Compare commits

...

8464 Commits

Author SHA1 Message Date
W. J. van der Laan
a0988140b7 Merge bitcoin/bitcoin#22921: Some small improvements to release notes
9f9ffe5bb0 Some small improvements to release notes (Pieter Wuille)

Pull request description:

  .

ACKs for top commit:
  laanwj:
    ACK 9f9ffe5bb0
  dongcarl:
    ACK 9f9ffe5bb0

Tree-SHA512: 63ff574eb1c5e7b5f2ac4fc99d3352d2de7fa05ccd0abbe58fcc0720b139ace05c855a6cf46ab7a5537b3f0792442d98dc155c3f3ee7bfd1127fe08f5fb01be5
2021-09-08 20:16:07 +02:00
Pieter Wuille
9f9ffe5bb0 Some small improvements to release notes 2021-09-08 13:49:11 -04:00
W. J. van der Laan
f75615ebdd doc: Manual pages update for 22.0 final
Tree-SHA512: 689698e7208782d9c7a6d926e9510b0e49223ee01f3fd878ae1b9b1d1f770e590982113e3c0f238046d65c5ddd7683ad357c7ebabc177bc8be9c7ec484c2944c
2021-09-08 17:21:09 +02:00
W. J. van der Laan
afbee409ba build: Bump version to 22.0 final
Tree-SHA512: c9280b1b19e6874ebdde89b6ab98933c56834cc0d74b98a420349cd93f96f526768e42b2e1e1d820c39f2f8b137bbeab72fc221d5ddb7f315c545675ee68e851
2021-09-08 17:20:58 +02:00
W. J. van der Laan
03f1422786 Merge bitcoin/bitcoin#22857: [22.x] Backports
303bc8a069 guix/prelude: Override VERSION with FORCE_VERSION (Carl Dong)
0640bf5c82 doc: mention bech32m/BIP350 in doc/descriptors.md (Pieter Wuille)

Pull request description:

  Backports:
  * #22847 - guix/prelude: Override VERSION with FORCE_VERSION
  * #22837 - doc: mention bech32m/BIP350 in doc/descriptors.md

  Theses are both minor enough that they would not require and rc4.

ACKs for top commit:
  laanwj:
    ACK 303bc8a069

Tree-SHA512: faac095f71abb537f1d2a338e4f79f8389be2362eec0841e3fb47aaee731ce242856db461f89351c2ca4e1129a3afdd49c3e918a7bf22af3e4d2f7deaff48ad8
2021-09-08 17:20:00 +02:00
W. J. van der Laan
fbf498d26b Merge bitcoin/bitcoin#22920: doc: Move 22.0 release notes from wiki
d447972417 doc: Move 22.0 release notes from wiki (W. J. van der Laan)

Pull request description:

  Reintroduce the release notes back into the branch for 22.0 final.

Top commit has no ACKs.

Tree-SHA512: ef2065a43db657a1351bcce18fe59a5eb5ee009da5739fdc077654696cd0e822546981df492924d61c74d2f2a276d7768c2cf0e57402befa8bd4d11c8e9b8fc5
2021-09-08 17:05:07 +02:00
W. J. van der Laan
d447972417 doc: Move 22.0 release notes from wiki
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-authored-by: Jon Atack <jon@atack.com>
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: Michael Folkson <michaelfolkson@gmail.com>
Co-authored-by: Pieter Wuille <github-sipa@wuille.net>
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
Co-authored-by: Vasil Dimov <vd@freebsd.org>
2021-09-08 16:49:54 +02:00
Carl Dong
303bc8a069 guix/prelude: Override VERSION with FORCE_VERSION
Previously, if the builder exported $VERSION in their environment (as
past Gitian-building docs told them to), but their HEAD does not
actually point to v$VERSION, their build outputs will differ from those
of other builders.

This is because the contrib/guix/guix-* scripts only ever act on the
current git worktree, and does not try to check out $VERSION if $VERSION
is set in the environment.

Setting $VERSION only makes the scripts pretend like the current
worktree is $VERSION.

This problem was seen in jonatack's attestation for all.SHA256SUMS,
where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
else's.

Here is my deduced sequence of events:

1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
   guix.sigs

2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
   changes from master in the same worktree where he guix-built 22.0rc3
   and ends up at 7be143a960

3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
   still on 7be143a960, he guix-codesigns. Normally, this would result
   in outputs going in guix-build-7be143a960e2, but he had
   VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
   like he was building 22.0rc3, and used 22.0rc3's guix-build directory
   to locate un-codesigned outputs and dump codesigned ones.

   However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
   (7be143a960), which made all timestamps in the resulting codesigned
   DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
   POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
   windows codesigning procedure does not consider SOURCE_DATE_EPOCH.

We resolve this by only allowing VERSION overrides via the FORCE_VERSION
environment variable.

Github-Pull: #22847
Rebased-From: 96cc6bb04f
2021-09-02 09:37:40 +08:00
Pieter Wuille
0640bf5c82 doc: mention bech32m/BIP350 in doc/descriptors.md
Github-Pull: #22837
Rebased-From: d2a09c8355
2021-09-02 09:37:01 +08:00
W. J. van der Laan
86de56776a doc: Manual pages update for rc3
Tree-SHA512: b8e2fa79fc6e46e74ebfe725fcdf12432167de9547138721e1d545426c52935e8f58f15fb4f7bfda9f11b02b17f77633238b5d0e883fd3362dc62b0ef4197a3c
2021-08-26 14:56:12 +02:00
W. J. van der Laan
c1c79f4c81 doc: Stop nixing - in manual pages
The version replacement here is not working anyway, not just that but it
is actively harmful as it removes all `-` from the text. So remove that
line. See discussion in #22681.

Tree-SHA512: 909c6225a50bbda1253bae65c09700a658bd36b5c91c91cea28d1df0aad5dc4adb095e0695259eca377536da34d17741ce453a5c5c50041d1163d94bbbe5355d
2021-08-26 14:51:31 +02:00
Jon Atack
f95b655ba9 Improve doc/i2p.md regarding I2P router options/versions
Github-Pull: bitcoin/bitcoin#22648
Rebased-From: b87a9c4d13

Tree-SHA512: 3567e6dbfb7c3f410ea84d3f005c86be0283f92e1be1bc2a7ce93240a0b8460b985b3cec5873a19eade52dc1d78ca383c99e023d0bb4247048ae7e43b47522a5
2021-08-26 14:43:41 +02:00
W. J. van der Laan
59d4afc272 build: Bump version to 22.0rc3
Tree-SHA512: 7e8c32d7b784329236090e09bb6c5f8ee7a64f46cfc136dae97d4838239292e94b28cfd429f56eaaddc1a23ebf33218b1b4396e126fe6cf60938f1909aaaf66e
2021-08-26 14:31:29 +02:00
W. J. van der Laan
99cd080db1 Merge bitcoin/bitcoin#22667: [22.x] qt: Pre-rc3 translations update
aa254a01c1 qt: Pre-rc3 translations update (Hennadii Stepanov)

Pull request description:

  A regularly updated PR with new translations fetched from Transifex.com.

ACKs for top commit:
  laanwj:
    Sanity-check (did not review any specific translations) ACK aa254a01c1

Tree-SHA512: bb380b1afb8af7895ac2de5ccdf489b9a73b6d47334d5eb5608370034795c064b5978106814ada96aaa5abd45a04901e6fd749bddc23149fb4a11dd4e6efe846
2021-08-26 14:29:13 +02:00
W. J. van der Laan
4a25e39624 Merge bitcoin/bitcoin#22629: [22.x] rc3 backports
32e1424f84 Fix build with Boost 1.77.0 (Rafael Sadowski)
cb34a0aafe qt: Handle new added plurals in bitcoin_en.ts (Hennadii Stepanov)
068985c02e doc: Mention the flat directory structure for uploads (Andrew Chow)
27d43e5bd4 guix: Don't include directory name in SHA256SUMS (Andrew Chow)
88fb7e37ad test: fix bug in 22686 (S3RK)
63fec7e295 clientversion: No suffix #if CLIENT_VERSION_IS_RELEASE (Carl Dong)
dfaffbeb63 test: Test for ApproximateBestSubset edge case with too little fees (Andrew Chow)
e86b023606 wallet: Assert that enough was selected to cover the fees (Andrew Chow)
ffc81e2048 wallet: Use GetSelectionAmount for target value calculations (Andrew Chow)
ce77b45a1f release: Release with separate SHA256SUMS and sig files (Carl Dong)
cb491bd5a7 guix-verify: Non-zero exit code when anything fails (Carl Dong)
6a611d2e3c gui: ensure external signer option remains disabled without signers (Andrew Chow)
e9b4487684 qt: Fix regression in "Encrypt Wallet" menu item (Hennadii Stepanov)
57fce067a3 consensus/params: simplify ValidDeployment check to avoid gcc warning (Anthony Towns)
e9d30fbb3a ci: Run fuzzer task for the master branch only (Hennadii Stepanov)

Pull request description:

  Backported:

  1) #22730
  1) https://github.com/bitcoin-core/gui/pull/393
  1) #22597
  1) https://github.com/bitcoin-core/gui/pull/396
  1) #22643
  1) #22642
  1) #22685
  1) #22686
  1) #22654
  1) #22742
  1) https://github.com/bitcoin-core/gui/pull/406
  1) #22713

ACKs for top commit:
  laanwj:
    Code list-of-backported-PRs review ACK 32e1424f84

Tree-SHA512: f5e2dd1be6cdcd39368eeb5d297b3ff4418d0bf2e70c90e59ab4ba1dbf16f773045d877b4997511de58c3aca75a978dcf043e338bad23951557e2a27ccc845f6
2021-08-26 14:26:01 +02:00
Rafael Sadowski
32e1424f84 Fix build with Boost 1.77.0
BOOST_FILESYSTEM_C_STR changed to accept the path as an argument

Github-Pull: bitcoin/bitcoin#22713
Rebased-From: acb7aad27e
2021-08-26 14:49:42 +03:00
Hennadii Stepanov
cb34a0aafe qt: Handle new added plurals in bitcoin_en.ts
This step was missed. See translation_process.md

Github-Pull: bitcoin-core/gui#406
Rebased-From: 2b3d8f3dde
2021-08-23 22:43:21 +03:00
Hennadii Stepanov
aa254a01c1 qt: Pre-rc3 translations update 2021-08-23 19:23:41 +03:00
Andrew Chow
068985c02e doc: Mention the flat directory structure for uploads
The uploaded binaries need to match the same flat directory structure of
the SHA256SUMS file in order for torrent downloaders to be able to
verify the download without moving files. Mention this in the release
process doc.

Github-Pull: bitcoin/bitcoin#22654
Rebased-From: 132cae44f2
2021-08-20 10:36:49 +03:00
Andrew Chow
27d43e5bd4 guix: Don't include directory name in SHA256SUMS
The SHA256SUMS file can be used in a sha256sum -c command to verify
downloaded binaries. However users are likely to download just a single
file and not place this file in the correct directory relative to the
SHA256SUMS file for the simple verification command to work. By not
including the directory name in the SHA256SUMS file, it will be easier
for users to verify downloaded binaries.

Co-authored-by: Carl Dong <contact@carldong.me>

Github-Pull: bitcoin/bitcoin#22654
Rebased-From: fb17c99e35
2021-08-20 10:36:48 +03:00
S3RK
88fb7e37ad test: fix bug in 22686
Github-Pull: bitcoin/bitcoin#22742
Rebased-From: 8dcbbbea64
2021-08-20 10:36:47 +03:00
Carl Dong
63fec7e295 clientversion: No suffix #if CLIENT_VERSION_IS_RELEASE
Previously, building from a release source tarball would result in a
version string like v22.0.0-<commithash>, but we expect just v22.0.0.
This commit solves this problem.

Also use PACKAGE_VERSION instead of reconstructing it.

Github-Pull: bitcoin/bitcoin#22685
Rebased-From: 5100deee58
2021-08-20 10:36:47 +03:00
Andrew Chow
dfaffbeb63 test: Test for ApproximateBestSubset edge case with too little fees
ApproximateBestSubset had an edge case (due to not using
GetSelectionAmount) where it was possible for it to return success but
fail to select enough to cover transaction fees. A test is added that
could trigger this failure prior to the fix being implemented.

Github-Pull: bitcoin/bitcoin#22686
Rebased-From: 92885c4f69
2021-08-20 10:36:46 +03:00
Andrew Chow
e86b023606 wallet: Assert that enough was selected to cover the fees
When the fee is not subtracted from the outputs, the amount that has
been reserved for the fee (change_and_fee - change_amount) must be
enough to cover the fee that is needed. It would be a bug to not do so,
so use an assert to make this obvious if such a situation were to occur.

Github-Pull: bitcoin/bitcoin#22686
Rebased-From: d9262324e8
2021-08-20 10:36:45 +03:00
Andrew Chow
ffc81e2048 wallet: Use GetSelectionAmount for target value calculations
For target value calculations, GetSelectionAmount should be used, not
m_effective_value or m_value.

Specifically, ApproximateBestSubset mistakenly uses m_value when
calculating whether the target value has been met. This has been changed
to use GetSelectionAmount.

Github-Pull: bitcoin/bitcoin#22686
Rebased-From: 2de222c401
2021-08-20 10:36:44 +03:00
Carl Dong
ce77b45a1f release: Release with separate SHA256SUMS and sig files
This allows us to remove the rfc4880 EOL hacks and release with a
SHA256SUMS.asc file that's a combination of all signer signatures.

Github-Pull: bitcoin/bitcoin#22642
Rebased-From: 90b3e482e9
2021-08-20 10:36:44 +03:00
Carl Dong
cb491bd5a7 guix-verify: Non-zero exit code when anything fails
Previously, if verification fails, the correct message will be printed,
but the exit code would still be 0.

Github-Pull: bitcoin/bitcoin#22643
Rebased-From: d451b60d22
2021-08-20 10:36:43 +03:00
Andrew Chow
6a611d2e3c gui: ensure external signer option remains disabled without signers
When no external signers are available, the option to enable external
signers should always be disabled. However the encrypt wallet checkbox
can erroneously re-enable the external signer checkbox. To avoid this,
CreateWalletDialog now stores whether signers were available during
setSigners so that future calls to external_signer_checkbox->setEnabled
can account for whether signers are available.

Github-Pull: bitcoin-core/gui#396
Rebased-From: a9b9ca82da
2021-08-20 10:36:42 +03:00
Hennadii Stepanov
e9b4487684 qt: Fix regression in "Encrypt Wallet" menu item
Adding a new item to the m_wallet_selector must follow the establishment
of signal-slot connections.

Github-Pull: bitcoin-core/gui#393
Rebased-From: d54d949598
2021-08-20 10:36:42 +03:00
Anthony Towns
57fce067a3 consensus/params: simplify ValidDeployment check to avoid gcc warning
Github-Pull: bitcoin/bitcoin#22597
Rebased-From: 059171009b
2021-08-20 10:36:41 +03:00
Hennadii Stepanov
e9d30fbb3a ci: Run fuzzer task for the master branch only
Github-Pull: bitcoin/bitcoin#22730
Rebased-From: 5a9e255e5a
2021-08-20 10:35:41 +03:00
W. J. van der Laan
d3bd5410f6 doc: Manual pages update for 22.0rc2
Tree-SHA512: 361c6efdd21396ebf4358195e91bde81e7f7414e7022efc4d4a10da7cf3df7ce8cdadd277237921fdde9bae4b811d69d95fec56c010b823324ca69b40b966d41
2021-08-11 13:43:13 +02:00
W. J. van der Laan
873fbc745d build: Bump RC to rc2
Tree-SHA512: 2e9db22da514b0ef4456f46524762420730cdd3c160a4012ca9d736d18d01abd2403b0ce800743b1f92212937436fe849b1c1b07bed0e3ad9f61165be55b7958
2021-08-02 15:41:13 +02:00
W. J. van der Laan
04222235a4 Merge bitcoin/bitcoin#22534: [22.x] rc2 backports
739d19053b doc: add info to i2p.md about IBD time and multiple networks (Jon Atack)
cc8838ce98 contrib, p2p: update I2P hardcoded seeds (Jon Atack)
cd57bb1a66 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information (Andrew Chow)
219900a123 guix: Remove extra \r from all.SHA256SUMS line ending (Andrew Chow)
38d18c01e2 guix, doc: Add a note that codesigners need to rebuild after tagging (Andrew Chow)
aa9b6aba03 guix: Allow changing the base manifest in guix-verify (Andrew Chow)
056e47d887 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS (Andrew Chow)
8f1e3b31b2 script, doc: guix touchups (jonatack)
3bbfc1b8e0 Updated Readme, Corrected the codesign typo (h)
34f9f88bc9 guix/build: Remove vestigial SKIPATTEST.TAG (Carl Dong)
9e52a30ebd guix/INSTALL: Misc fixups (Carl Dong)
45e0f3d608 guix: Silence getent(1) invocation (Carl Dong)

Pull request description:

  Currently backports #22511. We can collect up further backports and merge prior to rc2.

ACKs for top commit:
  laanwj:
    ACK 739d19053b

Tree-SHA512: 8fc795ee56b7757ff405636a2811bd606ea33ba1160f3f1ea42e0e1478ce8211bb60bf7b16a673b932db40a24b76d47c54e703bf2775d3b9385d9b080183b433
2021-08-02 15:38:51 +02:00
W. J. van der Laan
d7a3c7082d Merge bitcoin/bitcoin#22536: [22.x] qt: Pre-rc2 translations update
2d3fcf5760 qt: Pre-rc2 translations update (Hennadii Stepanov)

Pull request description:

  Since 6312b8370c translations are still updating everyday.

  I'm going to keep this PR updated until the moment just before tagging rc2, when it is expected to be merged.

  Hope this will make life of both translators and release maintainers a bit easier :)

Top commit has no ACKs.

Tree-SHA512: d2c44f3900ee313cfce24edaed30a08dbe340e5ecc627301bc3cc7162ddebe6c0403351ce0bf056395ab14e55908de1609f5e6e8183d3405bed86416888cb32d
2021-08-02 15:30:55 +02:00
Jon Atack
739d19053b doc: add info to i2p.md about IBD time and multiple networks
Github-Pull: #22589
Rebased-From: d2dffd5be4
2021-08-02 19:59:23 +08:00
Jon Atack
cc8838ce98 contrib, p2p: update I2P hardcoded seeds
Github-Pull: #22589
Rebased-From: 2962640c49
2021-08-02 19:59:07 +08:00
Hennadii Stepanov
2d3fcf5760 qt: Pre-rc2 translations update 2021-07-31 12:57:21 +03:00
Andrew Chow
cd57bb1a66 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information
If the user has set log.showSignature=true in their git config, then the
git log will always output GPG signature information. Since git log is
used to set EPOCH_SOURCE_DATE, this will mistakenly have GPG signature
information in it which causes issues for the build. To avoid this
issue, we override the config and force log.showSignature=false.

Github-Pull: #22531
Rebased-From: 9b313dfef1
2021-07-29 11:16:36 +08:00
Andrew Chow
219900a123 guix: Remove extra \r from all.SHA256SUMS line ending
guix-attest mistakenly added an extra \r to the line endings in
all.SHA256SUMS, causing guix-verify to erroneously fail.

Co-Authored-By: Carl Dong <contact@carldong.me>

Github-Pull: #22531
Rebased-From: 43225f0a2a
2021-07-29 11:16:05 +08:00
Andrew Chow
38d18c01e2 guix, doc: Add a note that codesigners need to rebuild after tagging
One of the issues observed during the 22.0rc1 release process was that a
codesigner's attestation mismatched non-codesigner attestations because
the guix-codesign step was performed prior to tagging the version in
bitcoin-detached-sigs.

Github-Pull: #22531
Rebased-From: d080c27066
2021-07-29 11:15:29 +08:00
Andrew Chow
aa9b6aba03 guix: Allow changing the base manifest in guix-verify
When verifying guix attestations, it is useful to set a particular
signer's manifest as the base to compare against.

Github-Pull: #22531
Rebased-From: 4a466388a0
2021-07-29 11:14:54 +08:00
Andrew Chow
056e47d887 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS
Github-Pull: #22531
Rebased-From: 33455c7696
2021-07-29 11:14:20 +08:00
jonatack
8f1e3b31b2 script, doc: guix touchups
Github-Pull: #22538
Rebased-From: 198ceb82f9
2021-07-29 11:05:48 +08:00
h
3bbfc1b8e0 Updated Readme, Corrected the codesign typo
Github-Pull: #22538
Rebased-From: d7b7f610a5
2021-07-29 11:04:59 +08:00
Carl Dong
34f9f88bc9 guix/build: Remove vestigial SKIPATTEST.TAG
Github-Pull: #22533
Rebased-From: 9f01feda0a
2021-07-23 14:09:16 +08:00
Carl Dong
9e52a30ebd guix/INSTALL: Misc fixups
Github-Pull: #22511
Rebased-From: a884a1edcd
2021-07-23 09:11:54 +08:00
Carl Dong
45e0f3d608 guix: Silence getent(1) invocation
Github-Pull: #22511
Rebased-From: 3c4d2c418e
2021-07-23 09:11:35 +08:00
W. J. van der Laan
6312b8370c qt: Pre-rc2 translations update
We forgot this for rc1. Thanks to Hebasto for fixing the import script.

Tree-SHA512: 127d0989dabf95867f4542e7f3134ef5d5045418b1411582772d60759e16a5090e30c83bffbbc44b2a496ba830a66bdb0d8ba0d2def43f3462a4f15edf64953a
2021-07-21 14:09:23 +02:00
W. J. van der Laan
f277b1782c build: Bump version to 22.0.0rc1
Tree-SHA512: 1bc98f4a6d15d8f810fa6d2ee26c495a8da08dff3efd3d2ec6bbc6bd8091751a21436efc39f04fb68c5279312644ff9a63bcbc3c4df02e0bf89cd1cd8473bac5
2021-07-20 15:25:24 +02:00
fanquake
42af9596ce Merge bitcoin/bitcoin#22499: Update assumed chain params
eeddd1c8fa Update assumed chain params (Sriram)

Pull request description:

  Update the relevant variables in `src/chainparams.cpp` for `mainnet`, `testnet`, and `signet` as given [here](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-branch-off).

  To review this PR, check out [this guide](https://github.com/fanquake/core-review/blob/master/update-assumevalid.md).

  Note: added a 10% overhead to the base value of `mainnet` in `m_assumed_blockchain_size`

ACKs for top commit:
  MarcoFalke:
    ACK eeddd1c8fa, checked against my node 🌮
  bfolkens:
    ACK eeddd1c - checked against `mainnet`
  achow101:
    Code Review ACK eeddd1c8fa
  0xB10C:
    ACK mainnet, testnet, and signet eeddd1c8fa
  jamesob:
    ACK eeddd1c8fa ([`jamesob/ackr/22499.1.sriramdvt.update_assumed_chain_par`](https://github.com/jamesob/bitcoin/tree/ackr/22499.1.sriramdvt.update_assumed_chain_par))
  darosior:
    ACK eeddd1c8fa mainnet and testnet

Tree-SHA512: 0ab19d2acc6a854c6aa38fba199d61c68cec40f005d1d54341ea32b59aae9b7d1aabfd21d7c0bc79f54be99d3e71d1d727196cab88f370259fd2c6e002d3e43c
2021-07-20 21:09:58 +08:00
MarcoFalke
539023ab41 Merge bitcoin/bitcoin#22492: wallet: Reorder locks in dumpwallet to avoid lock order assertion
9b85a5e2f7 tests: Test for dumpwallet lock order issue (Andrew Chow)
25d99e6511 Reorder dumpwallet so that cs_main functions go first (Andrew Chow)

Pull request description:

  When a wallet is loaded which has an unconfirmed transaction in the mempool, it will end up establishing the lock order of cs_wallet -> cs_main -> cs_KeyStore. If `dumpwallet` is used on this wallet, then a lock order of cs_wallet -> cs_KeyStore -> cs_main will be used, which causes a lock order assertion. This PR fixes this by reordering `dumpwallet` and `GetKeyBirthTimes` (only used by `dumpwallet`). Specifically, in both functions, the function calls which lock cs_main are done prior to locking cs_KeyStore. This avoids the lock order issue.

  Additionally, I have added a test case to `wallet_dump.py`. Of course testing this requires `--enable-debug`.

  Fixes #22489

ACKs for top commit:
  MarcoFalke:
    review ACK 9b85a5e2f7 🎰
  ryanofsky:
    Code review ACK 9b85a5e2f7. Nice to reduce lock scope, and good test!
  prayank23:
    tACK 9b85a5e2f7
  lsilva01:
    Tested ACK 9b85a5e2f7 under the same conditions reported in issue #22489 and the `dumpwallet` command completed successfully.

Tree-SHA512: d370a8f415ad64ee6a538ff419155837bcdbb167e3831b06572562289239028c6b46d80b23d227286afe875d9351f3377574ed831549ea426fb926af0e19c755
2021-07-20 15:04:07 +02:00
fanquake
8ed8164e6f Merge bitcoin/bitcoin#22261: [p2p/mempool] Two small fixes to node broadcast logic
5a77abd4e6 [style] Clean up BroadcastTransaction() (John Newbery)
7282d4c036 [test] Allow rebroadcast for same-txid-different-wtxid transactions (glozow)
cd48372b67 [mempool] Allow rebroadcast for same-txid-different-wtxid transactions (John Newbery)
847b6ed48d [test] Test transactions are not re-added to unbroadcast set (Duncan Dean)
2837a9f1ea [mempool] Only add a transaction to the unbroadcast set when it's added to the mempool (John Newbery)

Pull request description:

  1. Only add a transaction to the unbroadcast set when it's added to the mempool

      Currently, if BroadcastTransaction() is called to rebroadcast a
      transaction (e.g. by ResendWalletTransactions()), then we add the
      transaction to the unbroadcast set. That transaction has already been
      broadcast in the past, so peers are unlikely to request it again,
      meaning RemoveUnbroadcastTx() won't be called and it won't be removed
      from m_unbroadcast_txids.

      Net processing will therefore continue to attempt rebroadcast for the
      transaction every 10-15 minutes. This will most likely continue until
      the node connects to a new peer which hasn't yet seen the transaction
      (or perhaps indefinitely).

      Fix by only adding the transaction to the broadcast set when it's added to the mempool.

  2. Allow rebroadcast for same-txid-different-wtxid transactions

      There is some slightly unexpected behaviour when:

      - there is already transaction in the mempool (the "mempool tx")
      - BroadcastTransaction() is called for a transaction with the same txid
        as the mempool transaction but a different witness (the "new tx")

      Prior to this commit, if BroadcastTransaction() is called with
      relay=true, then it'll call RelayTransaction() using the txid/wtxid of
      the new tx, not the txid/wtxid of the mempool tx. For wtxid relay peers,
      in SendMessages(), the wtxid of the new tx will be taken from
      setInventoryTxToSend, but will then be filtered out from the vector of
      wtxids to announce, since m_mempool.info() won't find the transaction
      (the mempool contains the mempool tx, which has a different wtxid from
      the new tx).

      Fix this by calling RelayTransaction() with the wtxid of the mempool
      transaction in this case.

  The third commit is a comment/whitespace only change to tidy up the BroadcastTransaction() function.

ACKs for top commit:
  duncandean:
    reACK 5a77abd
  naumenkogs:
    ACK 5a77abd4e6
  theStack:
    re-ACK 5a77abd4e6
  lsilva01:
    re-ACK 5a77abd4e6

Tree-SHA512: d1a46d32a9f975220e5b432ff6633fac9be01ea41925b4958395b8d641680500dc44476b12d18852e5b674d2d87e4d0160b4483e45d3d149176bdff9f4dc8516
2021-07-20 20:57:58 +08:00
fanquake
e4487fd5bb Merge bitcoin/bitcoin#22096: p2p: AddrFetch - don't disconnect on self-announcements
5730a43703 test: Add functional test for AddrFetch connections (Martin Zumsande)
c34ad3309f net, rpc: Enable AddrFetch connections for functional testing (Martin Zumsande)
533500d907 p2p: Add timeout for AddrFetch peers (Martin Zumsande)
b6c5d1e450 p2p: AddrFetch - don't disconnect on self-announcements (Martin Zumsande)

Pull request description:

  AddrFetch connections (old name: oneshots) are intended to be short-lived connections on which we ask a peer for addresses via `getaddr` and disconnect after receiving them.

  This is done by disconnecting after receiving the first `addr`. However, it is no longer working as intended, because nowadays, the first `addr` a typical bitcoin core node sends is its self-announcement.
  So we'll disconnect before the peer gets a chance to answer our `getaddr`.

  I checked that this affects both `-seednode` peers specified manually, and DNS seeds when AddrFetch is used as a fallback if DNS doesn't work for us.

  The current behavior of getting peers via AddrFetch when starting with an empty addrman would be to connect to the peer, receive its self-announcement and add it to addrman, disconnect, reconnect to the same peer again as a full outbound (no other addresses in addrman) and then receive more `addr`. This is silly and not in line with AddrFetch peer being intended to be short-lived peers. 

  Fix this by only disconnecting after receiving an `addr` message of size > 1.

  [Edit] As per review discussion, this PR now also adds a timeout after which we disconnect if we haven't received any suitable `addr`, and a functional test.

ACKs for top commit:
  amitiuttarwar:
    reACK 5730a43703
  naumenkogs:
    ACK 5730a43703
  jnewbery:
    ACK 5730a43703

Tree-SHA512: 8a81234f37e827705138eb254223f7f3b3bf44a06cb02126fc7990b0d231b9bd8f07d38d185cc30d55bf35548a6fdc286b69602498d875b937e7c58332158bf9
2021-07-20 20:27:21 +08:00
fanquake
5c8820b399 Merge bitcoin/bitcoin#22436: build: use aarch64 Clang if cross-compiling for darwin on aarch64
54c7754f31 build: use aarch64 Clang if cross-compiling for darwin on aarch64 (fanquake)

Pull request description:

  If we're cross-compiling for darwin on aarch64 hardware, we need to
  use a Clang that will run on that hardware.

  Only tested in a Linux Docker container (aarch64-unknown-linux-gnu),
  running on an Apple M1 mac-mini (aarch64-apple-darwin20.5.0).

ACKs for top commit:
  hebasto:
    ACK 54c7754f31, I agree it can be merged (fix in #22448 is orthogonal to this one).

Tree-SHA512: 66c530097a5dc072a0a00dc22eb3d4a7d923dfa8ab8160f7c3e395cbe58da324f367548d673c0510606f5225d5d37bb5607a76b1703b8b03ac7d2cceeccbd542
2021-07-20 18:23:35 +08:00
fanquake
201c5e4aec Merge bitcoin/bitcoin#22465: guix: Pin kernel-header version, time-machine to upstream 1.3.0 commit
e6a94d4446 guix: Bump to version-1.3.0 from upstream (Carl Dong)
90fd13b954 guix: Pin kernel header version (Carl Dong)

Pull request description:

  ```
  - Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
  - Use 4.9 for all others (second-oldest LTS release, released in
    combination with glibc glibc 2.24 in Debian stretch)
  ```

  ```
  The chosen commit is the HEAD of Guix's version-1.3.0 branch as of July
  15th, 2021.

  Also fix visual indenting.
  ```

  -----

  This + the documentation PR should make our Guix system ready for release!

ACKs for top commit:
  MarcoFalke:
    review ACK e6a94d4446 to change to vanilla guix. Did not review the kernel change.
  laanwj:
    ACK e6a94d4446
  fanquake:
    ACK e6a94d4446

Tree-SHA512: a175e4ddb3ee786a39f5e800ce336932ad2f6797a3a28400a6f723875d0f19833fd36cedc41b3580e4604110517211bd9f557be36adf7265fd8e591c434ae032
2021-07-20 12:23:56 +08:00
fanquake
4fdd0ff9ee Merge bitcoin/bitcoin#22199: macdeploy: minor fixups and simplifications
0a5723beea macdeploy: cleanup .temp.dmg if present (fanquake)
ecffe8689d macdeploy: remove qt4 related code (fanquake)
639f064253 macdeploy: select the plugins we need, rather than excluding those we don't (fanquake)
3d26b6b9e9 macdeploy: fix framework printing when passing -verbose (fanquake)
dca6c90329 macdeploy: remove unused plistlib import (fanquake)

Pull request description:

  This includes [one followup](https://github.com/bitcoin/bitcoin/pull/20422#discussion_r534207899) and [one bug fix](3d26b6b9e9) from #20422, as well as some simplifications to the `macdeployqtplus` code.

ACKs for top commit:
  hebasto:
    ACK 0a5723beea, tested on macOS Big Sur 11.4 (20F71, x86_64) + Homebrew's Qt 5.15.2.

Tree-SHA512: cfad9505eacd32fe3a9d06eb13b2de0b6d2cad7b17778e90b503501cbf922e53d4e7f7f74952d1aed58410bdae9b0bb3248098583ef5b85689cb27d4dc06c029
2021-07-20 11:39:47 +08:00
fanquake
e7441a6a45 Merge bitcoin/bitcoin#21711: guix: Add full installation and usage documentation
fac4814106 doc/release-process: Add torrent creation details (Carl Dong)
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong)
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong)
318c60700b guix: Adapt release-process.md to new Guix process (Carl Dong)
fcab35b229 guix-attest: Produce and sign normalized documents (Carl Dong)
c2541fd0ca guix: Overhaul README (Carl Dong)
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys (Carl Dong)
fc4f8449f3 guix: Update various check_tools lists (Carl Dong)
263220a85c guix: Check for a sane services database (Carl Dong)

Pull request description:

  Based on: #21462

  Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.

  Current TODOs:
  - [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up
  - [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login
  - [x] Fanquake docker option: complete section
  - [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section
  - [x] Building from source option: Insert dependency tree diagram that I made
  - [x] Building from source option: redo sectioning, kind of a mess right now
  - [x] Optional setup: make clear which parts are only needed if building from source
  - [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
  - [x] Overall (after everything): Make the links work.

  Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.

ACKs for top commit:
  fanquake:
    ACK fac4814106 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.

Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a
2021-07-20 11:09:39 +08:00
fanquake
d542603c5a Merge bitcoin/bitcoin#22502: scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue"
facd56750c scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue" (MarcoFalke)

Pull request description:

  No longer needed, as it wouldn't help to debug this issue. See https://github.com/bitcoin/bitcoin/pull/22472#issuecomment-882692900

ACKs for top commit:
  fanquake:
    ACK facd56750c

Tree-SHA512: 13352b3529c43d6e65ab127134b32158d3072dc2fbbb326fea9adfeada5a8610d0477ea75748b8b68e7abb3b9869a989df3a3169e92bdd458053d64bae6ed379
2021-07-20 10:46:56 +08:00
fanquake
624a193330 Merge bitcoin/bitcoin#22497: scripted-diff: remove ResetI2PPorts() (revert e0a2b390c1)
d4b67c8ebc scripted-diff: remove ResetI2PPorts() (revert e0a2b390c1) (Vasil Dimov)

Pull request description:

  `CAddrMan::ResetI2PPorts()` was temporary. Remove it:
  * it has partially achieved its goal: probably ran on about half of the
    I2P nodes
  * it is hackish, deemed risky and two bugs where found in it:
    https://github.com/bitcoin/bitcoin/issues/22467
    https://github.com/bitcoin/bitcoin/issues/22470

  -BEGIN VERIFY SCRIPT-
  git show e0a2b390c1 |git apply -R
  -END VERIFY SCRIPT-

  Fixes https://github.com/bitcoin/bitcoin/issues/22467
  Fixes https://github.com/bitcoin/bitcoin/issues/22470

ACKs for top commit:
  laanwj:
    ACK d4b67c8ebc
  MarcoFalke:
    review ACK d4b67c8ebc 😲
  jonatack:
    ACK d4b67c8ebc per IRC discussions https://www.erisian.com.au/bitcoin-core-dev/log-2021-07-16.html#l-212 and https://www.erisian.com.au/bitcoin-core-dev/log-2021-07-19.html#l-210

Tree-SHA512: 60d8f0ea0f66a8fcedfcb9c8944a419b974b15509b54ddfeec58db49ae9418e6916df712bba3fbd6b29497d85f7951fb9aa2e48eb9c59f88d09435685bd00b4c
2021-07-20 09:02:34 +08:00
Carl Dong
fac4814106 doc/release-process: Add torrent creation details
Source: https://github.com/bitcoin/bitcoin/pull/21711#discussion_r668754244
2021-07-19 18:24:36 -04:00
Carl Dong
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir 2021-07-19 18:24:36 -04:00
Carl Dong
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting 2021-07-19 18:24:36 -04:00
Carl Dong
318c60700b guix: Adapt release-process.md to new Guix process
Also, clean up release-process.md
2021-07-19 18:24:36 -04:00
Carl Dong
fcab35b229 guix-attest: Produce and sign normalized documents
That way we can easily combine the document and detached signature to
produce cleartext signature files for upload during the release process.

See subsequent commits which modify doc/release-process.md for more
details.
2021-07-19 18:24:36 -04:00
Carl Dong
c2541fd0ca guix: Overhaul README
- Added detailed Guix bootstrap/installation instructions
2021-07-19 18:24:36 -04:00
MarcoFalke
facd56750c scripted-diff: Revert "fuzz: Add Temporary debug assert for oss-fuzz issue"
-BEGIN VERIFY SCRIPT-
git show faf1af58f8 | git apply --reverse
-END VERIFY SCRIPT-
2021-07-19 19:12:54 +02:00
Andrew Chow
9b85a5e2f7 tests: Test for dumpwallet lock order issue
Adds a test for the condition which can trigger a lock order assertion.
Specifically, there must be an unconfirmed transaction in the mempool
which belongs to the wallet being loaded. This will establish the order
of cs_wallet -> cs_main -> cs_KeyStore. Then dumpwallet is called on
that wallet. Previously, this would have used a lock order of cs_wallet
-> cs_KeyStore -> cs_main, but this should be fixed now. The test
ensures that.
2021-07-19 12:25:11 -04:00
Andrew Chow
25d99e6511 Reorder dumpwallet so that cs_main functions go first
DEBUG_LOCKORDER expects cs_wallet, cs_main, and cs_KeyStore to be
acquired in that order. However dumpwallet would take these in the order
cs_wallet, cs_KeyStore, cs_main. So when configured with
`--enable-debug`, it is possible to hit the lock order assertion when
using dumpwallet.

To fix this, cs_wallet and cs_KeyStore are no longer locked at the same
time. Instead cs_wallet will be locked first. Then the functions which
lock cs_main will be run. Lastly cs_KeyStore will be locked afterwards.
This avoids the lock order issue.

Furthermore, since GetKeyBirthTimes (only used by dumpwallet) also uses
a function that locks cs_main, and itself also locks cs_KeyStore, the
same reordering is done here.
2021-07-19 12:25:11 -04:00
Sriram
eeddd1c8fa Update assumed chain params
Note: 10% overhead to the base value of `mainnet` in `m_assumed_blockchain_size`
2021-07-19 19:34:30 +05:30
Vasil Dimov
d4b67c8ebc scripted-diff: remove ResetI2PPorts() (revert e0a2b390c1)
`CAddrMan::ResetI2PPorts()` was temporary. Remove it:
* it has partially achieved its goal: probably ran on about half of the
  I2P nodes
* it is hackish, deemed risky and two bugs where found in it
  https://github.com/bitcoin/bitcoin/issues/22467
  https://github.com/bitcoin/bitcoin/issues/22470

-BEGIN VERIFY SCRIPT-
git show e0a2b390c1 |git apply -R
-END VERIFY SCRIPT-

Fixes https://github.com/bitcoin/bitcoin/issues/22467
Fixes https://github.com/bitcoin/bitcoin/issues/22470
2021-07-19 14:33:21 +02:00
MarcoFalke
54e31742d2 Merge bitcoin/bitcoin#22455: addrman: detect on-disk corrupted nNew and nTried during unserialization
816f29eab2 addrman: detect on-disk corrupted nNew and nTried during unserialization (Vasil Dimov)

Pull request description:

  Negative `nNew` or `nTried` are not possible during normal operation.
  So, if we read such values during unserialize, report addrman
  corruption.

  Fixes https://github.com/bitcoin/bitcoin/issues/22450

ACKs for top commit:
  MarcoFalke:
    cr ACK 816f29eab2
  jonatack:
    ACK 816f29eab2
  lsilva01:
    Code Review ACK 816f29eab2.  This change provides a more accurate description of the error.

Tree-SHA512: 01bdd72d2d86a0ef770a319fee995fd1e147b24a8db84ddb8cd121688e7f94fed73fddc0084758e7183c4f8d08e971f0b1b224f5adb10928a5aa4dbbc8709d74
2021-07-19 14:25:53 +02:00
W. J. van der Laan
d3474b8df2 Merge bitcoin/bitcoin#22387: Rate limit the processing of rumoured addresses
a4bcd687c9 Improve tests using statistics (John Newbery)
f424d601e1 Add logging and addr rate limiting statistics (Pieter Wuille)
b4ece8a1cd Functional tests for addr rate limiting (Pieter Wuille)
5648138f59 Randomize the order of addr processing (Pieter Wuille)
0d64b8f709 Rate limit the processing of incoming addr messages (Pieter Wuille)

Pull request description:

  The rate at which IP addresses are rumoured (through ADDR and ADDRV2 messages) on the network seems to vary from 0 for some non-participating nodes, to 0.005-0.025 addr/s for recent Bitcoin Core nodes. However, the current codebase will happily accept and process an effectively unbounded rate from attackers. There are measures to limit the influence attackers can have on the addrman database (bucket restrictions based on source IPs), but still - there is no need to permit them to feed us addresses at a rate that's orders of magnitude larger than what is common on the network today, especially as it will cause us to spam our peers too.

  This PR implements a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) based rate limiter, allowing an average of 0.1 addr/s per connection, with bursts up to 1000 addresses at once. Whitelisted peers as well as responses to GETADDR requests are exempt from the limit. New connections start with 1 token, so as to not interfere with the common practice of peers' self-announcement.

ACKs for top commit:
  laanwj:
    ACK a4bcd687c9
  vasild:
    ACK a4bcd687c9
  jnewbery:
    ACK a4bcd687c9
  jonatack:
    ACK a4bcd687c9

Tree-SHA512: b757de76ad78a53035b622944c4213b29b3b55d3d98bf23585afa84bfba10808299d858649f92269a16abfa75eb4366ea047eae3216f7e2f6d3c455782a16bea
2021-07-19 12:42:07 +02:00
Samuel Dobson
e8f85e0e86 Merge bitcoin/bitcoin#22421: Make IsSegWitOutput return true for taproot outputs
8465978f23 Make IsSegWitOutput return true for taproot outputs (Pieter Wuille)

Pull request description:

  This fixes a bug: currently `utxoupdatepsbt` will not fill in UTXO data for PSBTs spending taproot outputs.

ACKs for top commit:
  achow101:
    Code Review ACK 8465978f23
  jonatack:
    ACK 8465978f23
  meshcollider:
    utACK 8465978f23

Tree-SHA512: 2f8f873450bef4b5a4ce5962a231297b386c6b1445e69ce5f36ab28eca7343be3a11bc09c38534b0f75e6f99ba15d78d3ba5d484f6c63e5a9775e1f3f55a74e0
2021-07-18 20:07:52 +12:00
MarcoFalke
0eea1dfe80 Merge bitcoin/bitcoin#22445: fuzz: Move implementations of non-template fuzz helpers from util.h to util.cpp
a2aca207b1 Move implementations of non-template fuzz helpers (Sriram)

Pull request description:

  There are 78 cpp files that include `util.h` (`grep -iIr "#include <test/fuzz/util.h>" src/test/fuzz | wc -l`). Modifying the implementation of a fuzz helper in `src/test/fuzz/util.h` will cause all fuzz tests to be recompiled. Keeping the declarations of these non-template fuzz helpers in `util.h` and moving their implementations to `util.cpp` will skip the redundant recompilation of all the fuzz tests, and builds these helpers only once in `util.cpp`.

  Functions moved from `util.h` to `util.cpp`:
  - `ConsumeTxMemPoolEntry`
  - `ContainsSpentInput`
  - `ConsumeNetAddr`
  - Methods of `FuzzedFileProvider::(open, read, write, seek, close)`

ACKs for top commit:
  MarcoFalke:
    review ACK a2aca207b1 🍂

Tree-SHA512: e7037ebb86d0fc56048e4f3d8733eefc21da11683b09d2b22926bda410719628d89c52ddd9b4c18aa243607a66fdb4d13a63e62ca010e66b3ec9174fd18107f0
2021-07-18 09:46:06 +02:00
Samuel Dobson
5341c3b1b3 Merge bitcoin/bitcoin#22461: wallet: Change ScriptPubKeyMan::Upgrade default to True
5012a7912e Test that descriptor wallet upgrade does nothing (Andrew Chow)
48bd7d3b77 Change ScriptPubKeyMan::Upgrade to default to return true (Andrew Chow)

Pull request description:

  When adding a new ScriptPubKeyMan, it's likely that there will be nothing for `Upgrade` to do. If it is called (via `upgradewallet`), then it should do nothing, successfully. This PR changes the default `ScriptPubKeyMan::Upgrade` function so that it returns a success instead of failure when doing nothing.

  Fixes #22460

ACKs for top commit:
  jonatack:
    ACK 5012a7912e
  meshcollider:
    utACK 5012a7912e

Tree-SHA512: 578c6521e997f7bb5cc44be2cfe9e0a760b6bd4aa301026a6b8b3282e8757473e4cb9f68b2e79dacdc2b42dddae718450072e0a38817df205dfea177a74d7f3d
2021-07-18 19:33:10 +12:00
fanquake
4371e635d6 Merge bitcoin/bitcoin#22410: Avoid GCC 7.1 ABI change warning in guix build
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build (Pieter Wuille)

Pull request description:

  The arm-linux-gnueabihf guix build output is littered with warnings like:

  ```
  /gnu/store/7a96hdqdb2qi8a39f09n84xjy2hr23rs-gcc-cross-arm-linux-gnueabihf-8.4.0/include/c++/bits/stl_vector.h:1085:4: note:
                   parameter passing for argument of type '__gnu_cxx::__normal_iterator<CRecipient*, std::vector<CRecipient> >' changed in GCC 7.1
  ```

  These are irrelevant for us. Disable them using `-Wno-psabi`.

ACKs for top commit:
  laanwj:
    ACK 1edddf5de4
  hebasto:
    ACK 1edddf5de4, after thorough reading related materials, I agree this change can be merged. As I mentioned above, I have been compiling my arm-32bit binaries with `-Wno-psabi` flag for two years, and no related flaws were observed.

Tree-SHA512: 485c7500547ac5da567ad23847341c18ff832607f5a1002676404cc647e437cf3445b6894ecff5b52929ca52bea946c06bd90eace1997c895e56204e787065e4
2021-07-18 14:33:22 +08:00
fanquake
4315dc02a1 Merge bitcoin/bitcoin#20641: depends: Use Qt top-level build facilities
1155978d8f build, qt: Do not install *.prl files (Hennadii Stepanov)
763793b60e build, qt: Fix wrong cross-compiling detection on macOS (Hennadii Stepanov)
30982721ab build, qt: Force bootstrap while building linguist tools (Hennadii Stepanov)
689320e307 build, qt: Drop translations.pro hack (Hennadii Stepanov)
6a1f98f253 build, qt: Drop lrelease dependency patch (Hennadii Stepanov)
39e561e087 build, qt: Add linguist_tools list (Hennadii Stepanov)
27d3def1c6 build: Use Qt top-level build facilities (Hennadii Stepanov)

Pull request description:

  This PR:
  - uses Qt top-level build facilities without the need to download all-in-one archive
  - is based on **BlockMechanic**'s [idea](https://github.com/bitcoin/bitcoin/pull/20600), and is an alternative to #20600
  - makes it easy to integrate [new modules](https://github.com/bitcoin/bitcoin/pull/16883) into static builds
  - has the minimal diff
  - makes the qt package build process streamlined by dropping some patches and hacks (an alternative to  #21420 and #20642)

  Fixes #18536 (a non-intrusive alternative to #21589 and #19785).

  Fixes #14648.

  Fixes #21588 (a non-intrusive alternative to #21591).

  Required for adding [Wayland support](https://github.com/bitcoin/bitcoin/issues/19950) on Linux.

  ---

  **Note for reviewers**: With 9046de8a4cbc3899fed9eae084115f423e7ac5bd from #21995 it is easy to verify that there are no changes in the resulted `qt` package archive on the per commit basis. For example, for `HOST=i686-pc-linux-gnu` no commit in this PR introduces any changes.

ACKs for top commit:
  fanquake:
    ACK 1155978d8f

Tree-SHA512: 667b06b72cb7ff26d68b9b88e8dddb51084783ca9e3d80b3392710794c1dc7fd77bbcc3ccf4ccece9919d33b9bf8fce13c5059502bd228021dc7c93fdb87ca7a
2021-07-18 14:05:39 +08:00
fanquake
b5889611c7 Merge bitcoin/bitcoin#22234: build: Mark print-% target as phony.
fb7be92b09 Mark print-% target as phony. (Dmitry Goncharov)

Pull request description:

  .PHONY does not take patterns (such as print-%) as prerequisites.
  Have print-% depend on force and mark force as phony.

  This change ensures print-% rule works even when there is a file that matches the target.

  ```
  $ # on master
  $ make print-host
  host=x86_64-pc-linux-gnu
  $ touch print-host
  $ make print-host
  make: 'print-host' is up to date.
  $
  $ git co mark_print_as_phony
  Switched to branch 'mark_print_as_phony'
  $ make print-host
  host=x86_64-pc-linux-gnu
  $ touch force
  $ make print-host
  host=x86_64-pc-linux-gnu
  ```

ACKs for top commit:
  hebasto:
    ACK fb7be92b09, tested on Linux Mint 20.2 (x86_64).

Tree-SHA512: b89ae66aa8c7aa6a7ab5f0956f9eb3b3ef9d56994b60dc2a97d498d4c1bba537845c190723e8a10310280b1b35df2cd935cc30aeb76735cac2dc621ad7823772
2021-07-18 13:41:24 +08:00
fanquake
6baabc4d1d Merge bitcoin/bitcoin#21430: build: Add -Werror=implicit-fallthrough compile flag
3c4c8e79ba build: Add -Werror=implicit-fallthrough compile flag (Hennadii Stepanov)
014110c47d Use C++17 [[fallthrough]] attribute, and drop -Wno-implicit-fallthrough (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 3c4c8e79ba - looks ok to me now. Checked that warnings occur in our code & leveldb by removing a `[[fallthrough]]` or `FALLTHROUGH_INTENDED`.
  jarolrod:
    ACK 3c4c8e79ba
  theStack:
    ACK 3c4c8e79ba

Tree-SHA512: 4dce91f0f26b8a3de09bd92bb3d7e1995e078e3a8b3ff861c4fbf6c0b32b2327d063633b07b89c4aa94a1141d7f78d46d9d43ab8df865273e342693ad30645b6
2021-07-18 11:06:10 +08:00
Andrew Chow
5012a7912e Test that descriptor wallet upgrade does nothing 2021-07-16 15:34:56 -04:00
MarcoFalke
f8b20fd35b Merge bitcoin/bitcoin#22464: bench: fix 32-bit narrowing warning in bench/peer_eviction.cpp
e49d50cf40 bench: fix 32-bit narrowing warning in bench/peer_eviction.cpp (Jon Atack)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/22459.

ACKs for top commit:
  hebasto:
    ACK e49d50cf40, tested on Debian 10.10 (i386):

Tree-SHA512: 8ca366fc296c633dbc8b8e0e7d80f4f6a64d02fb3da86d199881364f027d34b816a3c964b3fea2c1cc0b3ad51dd02d93c8bb14b5ebbd99fb4073cd1031766332
2021-07-16 08:04:43 +02:00
Carl Dong
e6a94d4446 guix: Bump to version-1.3.0 from upstream
The chosen commit is the HEAD of Guix's version-1.3.0 branch as of July
15th, 2021.

Also fix visual indenting.
2021-07-15 21:50:48 -04:00
Carl Dong
90fd13b954 guix: Pin kernel header version
- Use 4.19 for riscv64 (earliest LTS release w/ riscv64 support)
- Use 4.9 for all others (second-oldest LTS release, released in
  combination with glibc glibc 2.24 in Debian stretch)
2021-07-15 21:47:57 -04:00
Pieter Wuille
8465978f23 Make IsSegWitOutput return true for taproot outputs 2021-07-15 17:08:52 -07:00
John Newbery
a4bcd687c9 Improve tests using statistics 2021-07-15 16:31:47 -07:00
Jon Atack
e49d50cf40 bench: fix 32-bit narrowing warning in bench/peer_eviction.cpp 2021-07-15 23:05:10 +02:00
Pieter Wuille
f424d601e1 Add logging and addr rate limiting statistics
Includes logging improvements by Vasil Dimov and John Newbery.
2021-07-15 13:03:20 -07:00
Pieter Wuille
b4ece8a1cd Functional tests for addr rate limiting 2021-07-15 13:03:17 -07:00
Pieter Wuille
5648138f59 Randomize the order of addr processing 2021-07-15 12:59:23 -07:00
Pieter Wuille
0d64b8f709 Rate limit the processing of incoming addr messages
While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.

This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR and ADDRV2 messages.
Every connection gets an associated token bucket. Processing an
address in an ADDR or ADDRV2 message from non-whitelisted peers
consumes a token from the bucket. If the bucket is empty, the
address is ignored (it is not forwarded or processed). The token
counter increases at a rate of 0.1 tokens per second, and will
accrue up to a maximum of 1000 tokens (the maximum we accept in a
single ADDR or ADDRV2). When a GETADDR is sent to a peer, it
immediately gets 1000 additional tokens, as we actively desire many
addresses from such peers (this may temporarily cause the token
count to exceed 1000).

The rate limit of 0.1 addr/s was chosen based on observation of
honest nodes on the network. Activity in general from most nodes
is either 0, or up to a maximum around 0.025 addr/s for recent
Bitcoin Core nodes. A few (self-identified, through subver) crawler
nodes occasionally exceed 0.1 addr/s.
2021-07-15 12:52:38 -07:00
Andrew Chow
48bd7d3b77 Change ScriptPubKeyMan::Upgrade to default to return true
If a ScriptPubKeyMan does not implement Upgrade, then using upgraewallet
will fail unexpectedly. By changing the default to return true, then
this error can be avoided. This is still correct because a successful
upgrade can be that nothing happened.
2021-07-15 12:33:16 -04:00
W. J. van der Laan
a88fa1a555 Merge bitcoin/bitcoin#22211: net: relay I2P addresses even if not reachable (by us)
7593b06bd1 test: ensure I2P addresses are relayed (Vasil Dimov)
e7468139a1 test: make CAddress in functional tests comparable (Vasil Dimov)
33e211d2a4 test: implement ser/unser of I2P addresses in functional tests (Vasil Dimov)
86742811ce test: use NODE_* constants instead of magic numbers (Vasil Dimov)
ba45f02708 net: relay I2P addresses even if not reachable (by us) (Vasil Dimov)

Pull request description:

  Nodes that can reach the I2P network (have set `-i2psam=`) will relay
  I2P addresses even without this patch. However, nodes that can't reach
  the I2P network will not. This was done as a precaution in
  https://github.com/bitcoin/bitcoin/pull/20119 before anybody could
  connect to I2P because then, for sure, it would have been useless.

  Now, however, we have I2P support and a bunch of I2P nodes, so get all
  nodes on the network to relay I2P addresses to help with propagation,
  similarly to what we do with Tor addresses.

ACKs for top commit:
  jonatack:
    ACK 7593b06bd1
  naumenkogs:
    ACK 7593b06bd1.
  laanwj:
    Code review ACK 7593b06bd1
  kristapsk:
    ACK 7593b06bd1. Code looks correct, tested that functional test suite passes and also that `test/functional/p2p_addrv2_replay.py` fails if I undo changes in `IsRelayable()`.

Tree-SHA512: c9feec4a9546cc06bc2fec6d74f999a3c0abd3d15b7c421c21fcf2d610eb94611489e33d61bdcd5a4f42041a6d84aa892f7ae293b0d4f755309a8560b113b735
2021-07-15 16:53:34 +02:00
MarcoFalke
853ac47705 Merge bitcoin/bitcoin#22393: doc: added info to bitcoin.conf doc
fa84caebc7 doc: added info to bitcoin.conf doc (Brian Liotti)

Pull request description:

  Should probably be explicitly stated to not make modifications to the conf file while daemon is running. ref #11586

  For example, if rpc credentials are modified while bitcoind is running, `bitcoin-cli stop` is unable to stop bitcoind until the original credentials are restored in `bitcoin.conf`

ACKs for top commit:
  rajarshimaitra:
    ACK fa84caebc7
  Zero-1729:
    ACK fa84caebc7
  theStack:
    ACK fa84caebc7 🗄️

Tree-SHA512: f6ddffc25563c0b01e661b6abe43a7909938ad8eca38d6d0e2d4a2ce9fb850e51b54d950ef3118b74b6e340c64fe3f37205861720a2de2933db29782234869bb
2021-07-15 16:41:55 +02:00
W. J. van der Laan
d86e6625e8 Merge bitcoin/bitcoin#22369: doc: Add steps for Transifex to release process
a16378e501 doc: Remove unnecessary steps from translations update process (Wladimir J. van der Laan)
2584929823 doc: Add steps for transifex to release process (Wladimir J. van der Laan)

Pull request description:

  Document how to update settings on and for the transifex website before and after branch-off of a new release.

  (This is #21440, updated with the review feedback.)

ACKs for top commit:
  laanwj:
    ACK a16378e501
  hebasto:
    ACK a16378e501

Tree-SHA512: 9669cc3bc7ba056f913ee77c2ef9d9ee2313da947fc07f8cd955807c34c5d39e3af73587adfe734274ab2412a7dfb1e2dfe7ee4904ca28cfef9bf8061d26a573
2021-07-15 14:57:21 +02:00
W. J. van der Laan
21998bc028 Merge bitcoin/bitcoin#22284: p2p, refactor: performance improvements to ProtectEvictionCandidatesByRatio()
b1d905c225 p2p: earlier continuation when no remaining eviction candidates (Vasil Dimov)
c9e8d8f9b1 p2p: process more candidates per protection iteration (Jon Atack)
02e411ec45 p2p: iterate eviction protection only on networks having candidates (Jon Atack)
5adb064574 bench: add peer eviction protection benchmarks (Jon Atack)
566357f8f7 refactor: move GetRandomNodeEvictionCandidates() to test utilities (Jon Atack)

Pull request description:

  This follow-up to #21261 improves `ProtectEvictionCandidatesByRatio()` for better performance.

  Benchmarks are added; the performance improvement is between 2x and 5x for the benchmarked cases (CPU 2.50GHz, Turbo off, performance mode, Debian Clang 11 non-debug build).

  ```
  $ ./src/bench/bench_bitcoin -filter="EvictionProtection*.*"
  ```

  The refactored code is well-covered by existing unit tests and also a fuzzer.

  - `$ ./src/test/test_bitcoin -t net_peer_eviction_tests`
  - `$ FUZZ=node_eviction ./src/test/fuzz/fuzz ../qa-assets/fuzz_seed_corpus/node_eviction`

ACKs for top commit:
  klementtan:
    Tested and code review ACK b1d905c2.
  vasild:
    ACK b1d905c225
  jarolrod:
    ACK b1d905c225

Tree-SHA512: a3a6607b9ea2fec138da9780c03f63e177b6712091c5a3ddc3804b896a7585216446310280791f5e20cc023d02d2f03a4139237e12b5c1d7f2a1fa1011610e96
2021-07-15 14:49:45 +02:00
Vasil Dimov
816f29eab2 addrman: detect on-disk corrupted nNew and nTried during unserialization
Negative `nNew` or `nTried` are not possible during normal operation.
So, if we read such values during unserialize, report addrman
corruption.

Fixes https://github.com/bitcoin/bitcoin/issues/22450
2021-07-15 13:40:29 +02:00
MarcoFalke
c0224bc962 Merge bitcoin/bitcoin#22415: Make m_mempool optional in CChainState
ceb7b35a39 refactor: move UpdateTip into CChainState (James O'Beirne)
4abf0779d6 refactor: no mempool arg to GetCoinsCacheSizeState (James O'Beirne)
46e3efd1e4 refactor: move UpdateMempoolForReorg into CChainState (James O'Beirne)
617661703a validation: make CChainState::m_mempool optional (James O'Beirne)

Pull request description:

  Make `CChainState::m_mempool` optional by making it a pointer instead of a reference. This will allow a simplification to assumeutxo semantics (see https://github.com/bitcoin/bitcoin/pull/15606#pullrequestreview-692965905) and help facilitate the `-nomempool` option.

ACKs for top commit:
  jnewbery:
    ACK ceb7b35a39
  naumenkogs:
    ACK ceb7b35a39
  ryanofsky:
    Code review ACK ceb7b35a39 (just minor style and test tweaks since last review)
  lsilva01:
    Code review ACK and tested on Signet ACK ceb7b35a39
  MarcoFalke:
    review ACK ceb7b35a39 😌

Tree-SHA512: cc445ad33439d5918cacf80a6354eea8f3d33bb7719573ed5b970fad1a0dab410bcd70be44c862b8aba1b71263b82d79876688c553e339362653dfb3d8ec81e6
2021-07-15 13:40:03 +02:00
MarcoFalke
97153a7026 Merge bitcoin/bitcoin#22385: refactor: Use DeploymentEnabled to hide VB deployments
fa5658ed07 Use DeploymentEnabled to hide VB deployments (MarcoFalke)
fa11fecf0d doc: Move buried deployment doc to the enum that enumerates them (MarcoFalke)

Pull request description:

  Plus a doc commit.

ACKs for top commit:
  jnewbery:
    utACK fa5658ed07
  ajtowns:
    utACK fa5658ed07

Tree-SHA512: 2aeceee0674feb603d76656eff40695b7d7305de309f837bbb6a8c1dbb1d0b962b741f06ab7b9a8b1dbd1964c9c0c9aa5dc9588fd8e6d896e620b69e08eedbaa
2021-07-15 08:34:40 +02:00
fanquake
e2c4ac7cfb Merge bitcoin/bitcoin#22447: test: whitelist rpc_rawtransaction peers to speed up tests
a3d6ec5bb5 test: move rpc_rawtransaction tests to < 30s group (Jon Atack)
5a1ed96077 test: whitelist rpc_rawtransaction peers to speed up tests (Jon Atack)

Pull request description:

  Speed up the somewhat slow `rpc_rawtransaction.py` test by more than 3x (from 45-55 seconds to 15 seconds on a laptop running 2 x 2.5GHz).

ACKs for top commit:
  mjdietzx:
    ACK a3d6ec5bb5
  kristapsk:
    ACK a3d6ec5bb5
  theStack:
    ACK a3d6ec5bb5 🐎
  brunoerg:
    tACK a3d6ec5bb5

Tree-SHA512: f1d105594c9b5b257a7096b631a6fa5aeb50e330a351f75c2d6ffa7dd73abdb6e1f596a78c16d204a9bac3fe506e0519f9ad96bb8477ab6424c8e18125ccb659
2021-07-15 14:31:19 +08:00
fanquake
fdf9b3eba3 Merge bitcoin/bitcoin#22446: test: Fix wallet_listdescriptors.py if bdb is not compiled
0c845e3f89 test: Fix wallet_listdescriptors.py if bdb is not compiled (Hennadii Stepanov)

Pull request description:

  If build system is configured `--without-bdb`, the `wallet_listdescriptors.py` fails:
  ```
  $ test/functional/wallet_listdescriptors.py --descriptors
  2021-07-14T13:20:52.931000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_02p7o1c9
  2021-07-14T13:21:23.377000Z TestFramework (INFO): Test that the command is not available for legacy wallets.
  2021-07-14T13:21:23.381000Z TestFramework (ERROR): JSONRPC error
  Traceback (most recent call last):
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/test_framework.py", line 128, in main
      self.run_test()
    File "test/functional/wallet_listdescriptors.py", line 34, in run_test
      node.createwallet(wallet_name='w1', descriptors=False)
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/test_node.py", line 685, in createwallet
      return self.__getattr__('createwallet')(wallet_name, disable_private_keys, blank, passphrase, avoid_reuse, descriptors, load_on_startup, external_signer)
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/home/hebasto/GitHub/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: Compiled without bdb support (required for legacy wallets) (-4)
  2021-07-14T13:21:23.436000Z TestFramework (INFO): Stopping nodes
  2021-07-14T13:21:24.092000Z TestFramework (WARNING): Not cleaning up dir /tmp/bitcoin_func_test_02p7o1c9
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): Test failed. Test logging available at /tmp/bitcoin_func_test_02p7o1c9/test_framework.log
  2021-07-14T13:21:24.092000Z TestFramework (ERROR):
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): Hint: Call /home/hebasto/GitHub/bitcoin/test/functional/combine_logs.py '/tmp/bitcoin_func_test_02p7o1c9' to consolidate all logs
  2021-07-14T13:21:24.092000Z TestFramework (ERROR):
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
  2021-07-14T13:21:24.092000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
  2021-07-14T13:21:24.092000Z TestFramework (ERROR):
  ```

  This PR fixes this issue.

  Also see #20267.

ACKs for top commit:
  achow101:
    ACK 0c845e3f89

Tree-SHA512: d7116a9ae30c7b7e3f55f55d2eea66f9e293c38d6757ed66d0477e4256ff5fedca5ddedafa0ef0c09f4dc1f7f973163e5a46090da26b067fdddbd9ea2ee76633
2021-07-15 13:36:54 +08:00
Jon Atack
a3d6ec5bb5 test: move rpc_rawtransaction tests to < 30s group 2021-07-14 16:08:21 +02:00
Jon Atack
5a1ed96077 test: whitelist rpc_rawtransaction peers to speed up tests
in my testing from 45-55 seconds to 15.
2021-07-14 16:03:15 +02:00
Hennadii Stepanov
0c845e3f89 test: Fix wallet_listdescriptors.py if bdb is not compiled 2021-07-14 16:26:08 +03:00
Sriram
a2aca207b1 Move implementations of non-template fuzz helpers
Moved implementations of `ConsumeTxMemPoolEntry`, `ContainsSpentInput`, `ConsumeNetAddr`, and the methods(open, read, write, seek, close) of FuzzedFileProvider from test/fuzz/util.h to test/fuzz/util.cpp.
2021-07-14 18:45:53 +05:30
MarcoFalke
531c2b7c04 Merge bitcoin/bitcoin#20354: test: Add feature_taproot.py --previous_release
fa80e10d94 test: Add feature_taproot.py --previous_release (MarcoFalke)
85ccffa266 test: move releases download incantation to README (Sjors Provoost)
29d6b1da2a test: previous releases: add v0.20.1 (Sjors Provoost)

Pull request description:

  Disabling the new consensus code at runtime is fine, but potentially fragile and incomplete. Fix that by giving the option to run with a version that has been compiled without any taproot code.

ACKs for top commit:
  Sjors:
    tACK fa80e10
  NelsonGaldeman:
    tACK fa80e10d94

Tree-SHA512: 1a1feef823f08c05268759645a8974e1b2d39a024258f5e6acecbe25097aae3fa9302c27262978b40f1aa8e7b525b60c0047199010f2a5d6017dd6434b4066f0
2021-07-14 10:57:06 +02:00
James O'Beirne
ceb7b35a39 refactor: move UpdateTip into CChainState
Makes sense and saves on arguments.

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-13 11:16:37 -04:00
James O'Beirne
4abf0779d6 refactor: no mempool arg to GetCoinsCacheSizeState
Unnecessary argument since we can make use of this->m_mempool

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-13 11:16:30 -04:00
James O'Beirne
46e3efd1e4 refactor: move UpdateMempoolForReorg into CChainState
Allows fewer arguments and simplification of call sites.

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-13 11:12:16 -04:00
James O'Beirne
617661703a validation: make CChainState::m_mempool optional
Since we now have multiple chainstate objects, only one of them is active at any given
time. An active chainstate has a mempool, but there's no point to others having one.

This change will simplify proposed assumeutxo semantics. See the discussion here:
https://github.com/bitcoin/bitcoin/pull/15606#pullrequestreview-692965905

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2021-07-13 11:11:35 -04:00
W. J. van der Laan
d8f1e1327f Merge bitcoin/bitcoin#22112: Force port 0 in I2P
4101ec9d2e doc: mention that we enforce port=0 in I2P (Vasil Dimov)
e0a2b390c1 addrman: reset I2P ports to 0 when loading from disk (Vasil Dimov)
41cda9d075 test: ensure I2P ports are handled as expected (Vasil Dimov)
4f432bd738 net: do not connect to I2P hosts on port!=0 (Vasil Dimov)
1f096f091e net: distinguish default port per network (Vasil Dimov)
aeac3bce3e net: change I2P seeds' ports to 0 (Vasil Dimov)
38f900290c net: change assumed I2P port to 0 (Vasil Dimov)

Pull request description:

  _This is an alternative to https://github.com/bitcoin/bitcoin/pull/21514, inspired by https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-815049933. They are mutually exclusive. Just one of them should be merged._

  Change assumed ports for I2P to 0 (instead of the default 8333) as this is closer to what actually happens underneath with SAM 3.1 (https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-812632520, https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-816564719).

  Don't connect to I2P peers with advertised port != 0 (we don't specify a port to our SAM 3.1 proxy and it always connects to port = 0).

  Note, this change:
  * Keeps I2P addresses with port != 0 in addrman and relays them to others via P2P gossip. There may be non-bitcoin-core-22.0 peers using SAM 3.2 and for them such addresses may be useful.
  * Silently refuses to connect to I2P hosts with port != 0. This is ok for automatically chosen peers from addrman. Not so ok for peers provided via `-addnode` or `-connect` - a user who specifies `foo.b32.i2p:1234` (non zero port) may wonder why "nothing is happening".

  Fixes #21389

ACKs for top commit:
  laanwj:
    Code review ACK 4101ec9d2e
  jonatack:
    re-ACK 4101ec9d2e per `git range-diff efff9c3 0b0ee03 4101ec9`, built with DDEBUG_ADDRMAN, did fairly extensive testing on mainnet both with and without a peers.dat / -dnsseeds=0 to test boostrapping.

Tree-SHA512: 0e3c019e1dc05e54f559275859d3450e0c735596d179e30b66811aad9d5b5fabe3dcc44571e8f7b99f9fe16453eee393d6e153454dd873b9ff14907d4e6354fe
2021-07-13 14:52:41 +02:00
fanquake
54c7754f31 build: use aarch64 Clang if cross-compiling for darwin on aarch64
If we're cross-compiling for darwin on aarch64 hardware, we need need to
use a Clang that will run on that hardware.

Only tested in a Linux Docker container (aarch64-unknown-linux-gnu),
running on an Apple M1 mac-mini (aarch64-apple-darwin20.5.0).
2021-07-13 20:21:58 +08:00
MarcoFalke
7e1ba37b5d Merge bitcoin/bitcoin#22433: init: remove straggling boost thread_group related code
aa72ffb1c2 init: remove straggling boost thread_group code (fanquake)

Pull request description:

  `boost::thread_group` was removed in #21016.

ACKs for top commit:
  MarcoFalke:
    review ACK aa72ffb1c2

Tree-SHA512: c7ac3c2cde38fb752e0103d563b506732a403aad765a5db6be8d82399df3783044a77b071cc9c71aec3824397b04611894cf115576e63e8ee714eacf62729ab9
2021-07-12 19:08:25 +02:00
fanquake
aa72ffb1c2 init: remove straggling boost thread_group code
boost::thread_group usage was removed in #21016.
2021-07-12 21:46:59 +08:00
fanquake
839f5d06d6 Merge bitcoin/bitcoin#22432: doc: fix incorrect testmempoolaccept doc
9169be09f4 fix incorrect testmempoolaccept doc (glozow)

Pull request description:

  Sorry, I somehow missed this...

ACKs for top commit:
  jnewbery:
    Tested ACK 9169be09f4

Tree-SHA512: d44f81655669e338af298b7b5d616eb4ca15cbaac667c49251408cb92cee2fb9f440fcfbbac6a17744f24ceeafaf6cea6b9c49a37a464f7eaeeda6e655a56f7a
2021-07-12 20:31:29 +08:00
glozow
9169be09f4 fix incorrect testmempoolaccept doc 2021-07-12 10:57:52 +01:00
W. J. van der Laan
842e2a9c54 Merge bitcoin/bitcoin#20234: net: don't bind on 0.0.0.0 if binds are restricted to Tor
2feec3ce31 net: don't bind on 0.0.0.0 if binds are restricted to Tor (Vasil Dimov)

Pull request description:

  The semantic of `-bind` is to restrict the binding only to some address.
  If not specified, then the user does not care and we bind to `0.0.0.0`.
  If specified then we should honor the restriction and bind only to the
  specified address.

  Before this change, if no `-bind` is given then we would bind to
  `0.0.0.0:8333` and to `127.0.0.1:8334` (incoming Tor) which is ok -
  the user does not care to restrict the binding.

  However, if only `-bind=addr:port=onion` is given (without ordinary
  `-bind=`) then we would bind to `addr:port` _and_ to `0.0.0.0:8333` in
  addition.

  Change the above to not do the additional bind: if only
  `-bind=addr:port=onion` is given (without ordinary `-bind=`) then bind
  to `addr:port` (only) and consider incoming connections to that as Tor
  and do not advertise it. I.e. a Tor-only node.

ACKs for top commit:
  laanwj:
    Code review ACK 2feec3ce31
  jonatack:
    utACK 2feec3ce31 per `git diff a004833 2feec3c`
  hebasto:
    ACK 2feec3ce31, tested on Linux Mint 20.1 (x86_64):

Tree-SHA512: a04483af601706da928958b92dc560f9cfcc78ab0bb9d74414636eed1c6f29ed538ce1fb5a17d41ed82c9c9a45ca94899d0966e7ef93da809c9bcdcdb1d1f040
2021-07-12 10:08:22 +02:00
W. J. van der Laan
e0fe658b86 Merge bitcoin/bitcoin#22335: doc: recommend --disable-external-signer in OpenBSD build guide
e65d1d4986 doc: recommend `--disable-external-signer` in OpenBSD build guide (Sebastian Falbesoner)

Pull request description:

  Building the master branch with the default build settings (i.e. with external signer support enabled) leads to the following errors on my OpenBSD 6.9 machine:

  ```
  In file included from util/system.cpp:9:
  In file included from /usr/local/include/boost/process.hpp:25:
  In file included from /usr/local/include/boost/process/group.hpp:32:
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:38:17: error: no member named 'waitid' in the global namespace
          ret = ::waitid(P_PGID, p.grp, &status, WEXITED | WNOHANG);
                ~~^
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:38:24: error: use of undeclared identifier 'P_PGID'
          ret = ::waitid(P_PGID, p.grp, &status, WEXITED | WNOHANG);
                         ^
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:38:48: error: use of undeclared identifier 'WEXITED'
          ret = ::waitid(P_PGID, p.grp, &status, WEXITED | WNOHANG);
                                                 ^
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:144:17: error: no member named 'waitid' in the global namespace
          ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
                ~~^
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:144:24: error: use of undeclared identifier 'P_PGID'
          ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
                         ^
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:144:49: error: use of undeclared identifier 'WEXITED'
          ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
                                                  ^
  /usr/local/include/boost/process/detail/posix/wait_group.hpp:144:59: error: use of undeclared identifier 'WSTOPPED'
          ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
                                                            ^
  7 errors generated.
  ```

  This PR recommends passing `--disable-external-signer` in the OpenBSD build guide ([as suggested by laanwj](https://github.com/bitcoin/bitcoin/pull/22294#issuecomment-867452411)). The same commit also bumps the OpenBSD version mentioned in the header to 6.9 -- I recently used this document to setup a Bitcoin Core build on 6.9 and the description and all mentioned versions were still valid (before external signer support was enabled by default).

  Would be nice if another OpenBSD user could confirm the build error.

ACKs for top commit:
  laanwj:
    ACK e65d1d4986

Tree-SHA512: c3ae7eca29cf42b4b52024477e1c3fb7242bbf9d809bc95f8fa08b2f9bf4bcfd4f22457d58569a208ac1d8e5fe41b270addd13d85a5bba0521a0f9e325288448
2021-07-12 10:01:48 +02:00
Martin Zumsande
5730a43703 test: Add functional test for AddrFetch connections
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-07-12 02:16:54 +02:00
Martin Zumsande
c34ad3309f net, rpc: Enable AddrFetch connections for functional testing
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-07-12 02:16:45 +02:00
John Newbery
5a77abd4e6 [style] Clean up BroadcastTransaction() 2021-07-09 18:21:36 +01:00
glozow
7282d4c036 [test] Allow rebroadcast for same-txid-different-wtxid transactions
Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-09 18:21:34 +01:00
John Newbery
cd48372b67 [mempool] Allow rebroadcast for same-txid-different-wtxid transactions
This commit fixes some slightly unexpected behaviour when:

- there is already transaction in the mempool (the "mempool tx")
- BroadcastTransaction() is called for a transaction with the same txid
  as the mempool transaction but a different witness (the "new tx")

Prior to this commit, if BroadcastTransaction() is called with
relay=true, then it'll call RelayTransaction() using the txid/wtxid of
the new tx, not the txid/wtxid of the mempool tx. For wtxid relay peers,
in SendMessages(), the wtxid of the new tx will be taken from
setInventoryTxToSend, but will then be filtered out from the vector of
wtxids to announce, since m_mempool.info() won't find the transaction
(the mempool contains the mempool tx, which has a different wtxid from
the new tx).

Fix this by calling RelayTransaction() with the wtxid of the mempool
transaction in this case.
2021-07-09 17:24:08 +01:00
Duncan Dean
847b6ed48d [test] Test transactions are not re-added to unbroadcast set 2021-07-09 17:24:08 +01:00
John Newbery
2837a9f1ea [mempool] Only add a transaction to the unbroadcast set when it's added to the mempool
Currently, if BroadcastTransaction() is called to rebroadcast a
transaction (e.g. by ResendWalletTransactions()), then we add the
transaction to the unbroadcast set. That transaction has already been
broadcast in the past, so peers are unlikely to request it again,
meaning RemoveUnbroadcastTx() won't be called and it won't be removed
from m_unbroadcast_txids.

Net processing will therefore continue to attempt rebroadcast for the
transaction every 10-15 minutes. This will most likely continue until
the node connects to a new peer which hasn't yet seen the transaction
(or perhaps indefinitely).

Fix by only adding the transaction to the broadcast set when it's added
to the mempool.
2021-07-09 17:24:08 +01:00
W. J. van der Laan
8ab0c77299 Merge bitcoin/bitcoin#22253: validation: distinguish between same tx and same-nonwitness-data tx in mempool
b7a8cd9963 [test] submit same txid different wtxid as mempool tx (glozow)
fdb48163bf [validation] distinguish same txid different wtxid in mempool (glozow)

Pull request description:

  On master, if you submit a transaction with the same txid but different witness to the mempool, it thinks the transactions are the same. Users submitting through `BroadcastTransaction()` (i.e. `sendrawtransaction` or the wallet) don't get notified that there's a different transaction in the mempool, although it doesn't crash. Users submitting through `testmempoolaccept()` will get a "txn-already-in-mempool" error.

  This PR simply distinguishes between `txn-already-in-mempool` and `txn-same-nonwitness-data-in-mempool`, without handling them differently: `sendrawtransaction` still will not throw, but `testmempoolaccept` will give you a different error.

  I believe the intention of #19645 is to allow full swaps of transactions that have different witnesses but identical nonwitness data. Returning a different error message + adding a test was suggested: https://github.com/bitcoin/bitcoin/pull/19645#issuecomment-705109193 so this is that PR.

ACKs for top commit:
  naumenkogs:
    ACK b7a8cd9963
  jnewbery:
    Code review ACK b7a8cd9963
  theStack:
    Code-review ACK b7a8cd9963
  darosior:
    re-utACK b7a8cd9963

Tree-SHA512: 9c6591edaf8727ba5b4675977adb8cbdef7288584003b6cd659828032dc92d2ae915800a8ef8b6fdffe112c1b660df72297a3dcf2e2e3e1f959c6cb3678c63ee
2021-07-09 17:34:46 +02:00
Vasil Dimov
4101ec9d2e doc: mention that we enforce port=0 in I2P
Co-authored-by: Jon Atack <jon@atack.com>
2021-07-09 11:19:38 +02:00
Vasil Dimov
e0a2b390c1 addrman: reset I2P ports to 0 when loading from disk
This is a temporary change to convert I2P addresses that have propagated
with port 8333 to ones with port 0.

It would cause a problem some day if indeed some bitcoin software is
listening on port 8333 only and rejects connections to port 0 and we are
still using SAM 3.1 which only supports port 0. In this case we would
replace 8333 with 0 and try to connect to such nodes.

This commit should be included in 22.0 and be reverted before 23.0 is
released.
2021-07-09 11:19:38 +02:00
Vasil Dimov
41cda9d075 test: ensure I2P ports are handled as expected 2021-07-09 11:19:37 +02:00
Vasil Dimov
4f432bd738 net: do not connect to I2P hosts on port!=0
When connecting to an I2P host we don't specify destination port and it
is being forced to 0 by the SAM 3.1 proxy, so if we connect to the same
host on two different ports, that would be actually two connections to
the same service (listening on port 0).

Fixes https://github.com/bitcoin/bitcoin/issues/21389
2021-07-09 11:19:37 +02:00
Vasil Dimov
1f096f091e net: distinguish default port per network
Change `CChainParams::GetDefaultPort()` to return 0 if the network is
I2P.
2021-07-09 11:19:36 +02:00
Vasil Dimov
aeac3bce3e net: change I2P seeds' ports to 0 2021-07-09 11:19:35 +02:00
Vasil Dimov
38f900290c net: change assumed I2P port to 0
* When accepting an I2P connection, assume the peer has port 0 instead
  of the default 8333 (for mainnet). It is not being sent to us, so we
  must assume something.
* When deriving our own I2P listen CService use port 0 instead of the
  default 8333 (for mainnet). So that we later advertise it to peers
  with port 0.

In the I2P protocol SAM 3.1 and older (we use 3.1) ports are not used,
so they are irrelevant. However in SAM 3.2 and newer ports are used and
from the point of view of SAM 3.2, a peer using SAM 3.1 seems to have
specified port=0.
2021-07-09 11:19:35 +02:00
MarcoFalke
d1e4c56309 Merge bitcoin/bitcoin#22363: test: refactor: use script_util helpers for creating P2{PKH,SH,WPKH,WSH} scripts
905d672b74 test: use script_util helpers for creating P2W{PKH,SH} scripts (Sebastian Falbesoner)
285a65ccfd test: use script_util helpers for creating P2SH scripts (Sebastian Falbesoner)
b57b633b94 test: use script_util helpers for creating P2PKH scripts (Sebastian Falbesoner)
61b6a017a9 test: wallet util: fix multisig P2SH-P2WSH script creation (Sebastian Falbesoner)

Pull request description:

  PR #18788 (commit 08067aebfd) introduced functions to generate output scripts for various types. This PR replaces all manual CScript creations in the P2PKH, P2SH, P2WPKH, P2WSH formats with those helpers in order to increase readability and maintainability over the functional test codebase. The first commit fixes a bug in the wallet_util helper module w.r.t. to P2SH-P2WSH script creation (the result is not used in any test so far, hence it can still be seen as refactoring).

  The following table shows a summary of the output script patterns tackled in this PR:

  | Type | master branch  | PR branch |
  | ---------- | ------------- | ------------- |
  | P2PKH | `CScript([OP_DUP, OP_HASH160, hash160(key), OP_EQUALVERIFY, OP_CHECKSIG])` | `key_to_p2pkh_script(key)`  |
  |             | `CScript([OP_DUP, OP_HASH160, keyhash, OP_EQUALVERIFY, OP_CHECKSIG])` | `keyhash_to_p2pkh_script(keyhash)` |
  | P2SH  | `CScript([OP_HASH160, hash160(script), OP_EQUAL])`  | `script_to_p2sh_script(script)` |
  | P2WPKH | `CScript([OP_0, hash160(key)])` | `key_to_p2wpkh_script(key)` |
  | P2WSH | `CScript([OP_0, sha256(script)])` | `script_to_p2wsh_script(script)` |

  Note that the `key_to_...` helpers can't be used if an invalid key size (not 33 or 65 bytes) is passed, which is the case in some rare instances where the scripts still have to be created manually.

  Possible follow-up ideas:
  * further simplify by identifying P2SH-wrapped scripts and using `key_to_p2sh_p2wpkh_script()` and `script_to_p2sh_p2wsh_script()` helpers
  * introduce and use `key_to_p2pk_script()` helper for P2PK scripts

ACKs for top commit:
  rajarshimaitra:
    tACK 905d672b74
  LarryRuane:
    tACK 905d672b74
  0xB10C:
    ACK 905d672b74
  MarcoFalke:
    review ACK 905d672b74 🕹

Tree-SHA512: 7ccfe69699bc81168ac122b03536720013355c1b2fbb088355b616015318644c4d1cd27e20c4f56c89ad083ae609add4bc838cf6316794d0edb0ce9cf7fa0fd8
2021-07-09 11:17:58 +02:00
fanquake
efff9c3494 Merge bitcoin/bitcoin#22176: test: Correct outstanding -Werror=sign-compare errors
4e44f5bac4 test: Correct outstanding -Werror=sign-compare errors (Ben Woosley)

Pull request description:

  I'm unclear on why these aren't failing on CI, but they failed for me locally, e.g.:

  ```
  In file included from /usr/local/include/boost/test/test_tools.hpp:46:
  /usr/local/include/boost/test/tools/old/impl.hpp:107:17: error: comparison of integers of different signs: 'const unsigned int' and 'const int' [-Werror,-Wsign-compare]
      return left == right;
             ~~~~ ^  ~~~~~
  /usr/local/include/boost/test/tools/old/impl.hpp:130:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl<unsigned int, int>' requested here
          return equal_impl( left, right );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:145:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::call_impl<unsigned int, int>' requested here
          return call_impl( left, right, left_is_array() );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:92:50: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::operator()<unsigned int, int>' requested here
  BOOST_PP_REPEAT( BOOST_TEST_MAX_PREDICATE_ARITY, IMPL_FRWD, _ )
                                                   ^
  /usr/local/include/boost/preprocessor/repetition/repeat.hpp:30:26: note: expanded from macro 'BOOST_PP_REPEAT'
                           ^
  /usr/local/include/boost/preprocessor/cat.hpp:22:32: note: expanded from macro 'BOOST_PP_CAT'
                                 ^
  /usr/local/include/boost/preprocessor/cat.hpp:29:34: note: expanded from macro 'BOOST_PP_CAT_I'
                                   ^
  <scratch space>:153:1: note: expanded from here
  BOOST_PP_REPEAT_1
  ^
  test/streams_tests.cpp:122:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned int, int>' requested here
      BOOST_CHECK_EQUAL(varint, 54321);
      ^

  /usr/local/include/boost/test/tools/old/impl.hpp:107:17: error: comparison of integers of different signs: 'const unsigned long long' and 'const long' [-Werror,-Wsign-compare]
      return left == right;
             ~~~~ ^  ~~~~~
  /usr/local/include/boost/test/tools/old/impl.hpp:130:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl<unsigned long long, long>' requested here
          return equal_impl( left, right );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:145:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::call_impl<unsigned long long, long>' requested here
          return call_impl( left, right, left_is_array() );
                 ^
  /usr/local/include/boost/test/tools/old/impl.hpp:92:50: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::operator()<unsigned long long, long>' requested here
  BOOST_PP_REPEAT( BOOST_TEST_MAX_PREDICATE_ARITY, IMPL_FRWD, _ )
                                                   ^
  /usr/local/include/boost/preprocessor/repetition/repeat.hpp:30:26: note: expanded from macro 'BOOST_PP_REPEAT'
                           ^
  /usr/local/include/boost/preprocessor/cat.hpp:22:32: note: expanded from macro 'BOOST_PP_CAT'
                                 ^
  /usr/local/include/boost/preprocessor/cat.hpp:29:34: note: expanded from macro 'BOOST_PP_CAT_I'
                                   ^
  <scratch space>:161:1: note: expanded from here
  BOOST_PP_REPEAT_1
  ^
  test/serfloat_tests.cpp:41:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned long long, long>' requested here
      BOOST_CHECK_EQUAL(TestDouble(std::numeric_limits<double>::infinity()), 0x7ff0000000000000);
      ^

ACKs for top commit:
  theStack:
    ACK 4e44f5bac4

Tree-SHA512: 8d9e5245676c61207ceacdf78c78a78ccc9fd2a2551d4d8df023513795591334aa2f5e1f4a2a8ed2bfeb381f1e226b6ba84c07e0de29a1f3f00da71f3a257bc1
2021-07-09 10:22:43 +08:00
fanquake
34d1d6a112 Merge bitcoin/bitcoin#22381: guix: Test security-check sanity before performing them (with macOS)
5b4703c6a7 guix: Test security-check sanity before performing them (Carl Dong)
6cf3345297 scripts: adjust test-symbol-check for guix release environment (fanquake)
1946b5f77c scripts: more robustly test macOS symbol checks (fanquake)
a8127b34bc build: Use and test PE binutils with --reloc-section (Carl Dong)
678348db51 guix: Patch binutils to add security-related disable flags (Carl Dong)
9fdc8afe11 devtools: Improve *-check.py tool detection (Carl Dong)
bda62eab38 ci: skip running the Linux test-security-check target for now (fanquake)
d6ef3543ae lint: Run mypy with --show-error-codes (Carl Dong)

Pull request description:

  This is #20980 rebased (to include the Boost Process fix), and with an additional commit (892d6897f1e613084aa0517a660eab2412308e6e) to fix running the `test-security-check` target for the macOS build. It should pass inside Guix, as well as when cross-compiling on Ubuntu, or building natively on macOS.

  Note that the `test-security-check` may output some warnings (similar too):
  ```bash
  ld: warning: passed two min versions (10.14, 11.4) for platform macOS. Using 11.4.
  ld: warning: passed two min versions (10.14, 11.4) for platform macOS. Using 11.4.
  ld: warning: passed two min versions (10.14, 10.14) for platform macOS. Using 10.14.
  ```
  but those can be ignored, and come about due to us passing `-platform_version` when `-mmacosx-version-min` is already part of `CC`.

  Guix builds:
  ```bash
  71ed0c7a13a4726300779ffc87f7d271086a2744c36896fe6dc51fe3dc33df2e  guix-build-5b4703c6a70d/output/aarch64-linux-gnu/SHA256SUMS.part
  9273980a17052c8ec45b77579781c14ab5d189fa25aa29907d5115513dd302b1  guix-build-5b4703c6a70d/output/aarch64-linux-gnu/bitcoin-5b4703c6a70d-aarch64-linux-gnu-debug.tar.gz
  9c042179af43c8896eb95a34294df15d4910308dcdba40b2010cd36e192938b8  guix-build-5b4703c6a70d/output/aarch64-linux-gnu/bitcoin-5b4703c6a70d-aarch64-linux-gnu.tar.gz
  1ceddecac113f50a952ba6a201cdcdb722e3dc804e663f219bfac8268ce42bf0  guix-build-5b4703c6a70d/output/arm-linux-gnueabihf/SHA256SUMS.part
  759597c4e925e75db4a2381c06cda9b9f4e4674c23436148676b31c9be05c7aa  guix-build-5b4703c6a70d/output/arm-linux-gnueabihf/bitcoin-5b4703c6a70d-arm-linux-gnueabihf-debug.tar.gz
  34e3b6beabaf8c95d7c2ca0d2c3ac4411766694ef43e00bd9783badbbaf045a7  guix-build-5b4703c6a70d/output/arm-linux-gnueabihf/bitcoin-5b4703c6a70d-arm-linux-gnueabihf.tar.gz
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-5b4703c6a70d/output/dist-archive/SKIPATTEST.TAG
  3664f6ceee7898caa374281fd877a7597fe491fa2e9f0c174c28d889d60b559c  guix-build-5b4703c6a70d/output/dist-archive/bitcoin-5b4703c6a70d.tar.gz
  d6bc35ba0750c1440bb32831b8c12cddee62f6dce10fec2650897444c2bf4748  guix-build-5b4703c6a70d/output/powerpc64-linux-gnu/SHA256SUMS.part
  a836edf6474ba0c16c19bb217549bac7936c1b44306ed512df58f607ee5568f2  guix-build-5b4703c6a70d/output/powerpc64-linux-gnu/bitcoin-5b4703c6a70d-powerpc64-linux-gnu-debug.tar.gz
  7cc91c6805d5069ca3bd1771e77d95f83eb184b137198cbf84d1d11d0a5c5afe  guix-build-5b4703c6a70d/output/powerpc64-linux-gnu/bitcoin-5b4703c6a70d-powerpc64-linux-gnu.tar.gz
  93b4cb7b83c4975120ad5de5a92f050f5760a2a3f2c37c204c647f5a581c924a  guix-build-5b4703c6a70d/output/powerpc64le-linux-gnu/SHA256SUMS.part
  2266e2c5d0dafa28c6c057ccfc1c439baeab1d714d8c3f64a83015d2827116d2  guix-build-5b4703c6a70d/output/powerpc64le-linux-gnu/bitcoin-5b4703c6a70d-powerpc64le-linux-gnu-debug.tar.gz
  85f41f42c319b83d049d6fd2e2278c07b40a1e28a2eac596427822c0eef9dc3f  guix-build-5b4703c6a70d/output/powerpc64le-linux-gnu/bitcoin-5b4703c6a70d-powerpc64le-linux-gnu.tar.gz
  1499ca9119926083d8c3714ca10d8d4c8d864cbeee8848fd8445b7a1d081222d  guix-build-5b4703c6a70d/output/riscv64-linux-gnu/SHA256SUMS.part
  1995fc1a2e45c49d4b0718aff5dcdac931917e8ae9e762fd23f1126abcecc248  guix-build-5b4703c6a70d/output/riscv64-linux-gnu/bitcoin-5b4703c6a70d-riscv64-linux-gnu-debug.tar.gz
  266889eb58429a470f0fd7bb123f2ae09b0aef86c47b0390938b3634a8f748a9  guix-build-5b4703c6a70d/output/riscv64-linux-gnu/bitcoin-5b4703c6a70d-riscv64-linux-gnu.tar.gz
  cdc3a0dcf80b110443dac5ddf8bc951001a776a651c898c5ea49bb2d487bfe29  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/SHA256SUMS.part
  8538d1eab96c97866b24546c453d95822f24cf9c6638b42ba523eb7aa441cb26  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/bitcoin-5b4703c6a70d-osx-unsigned.dmg
  d1b73133f1da68586b07292a8425f7f851e93f599c016376f23728c041cf39cc  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/bitcoin-5b4703c6a70d-osx-unsigned.tar.gz
  5ad94c5f8a5f29405955ff3ab35d137de1acc04398d6c8298fb187b57a6e316a  guix-build-5b4703c6a70d/output/x86_64-apple-darwin18/bitcoin-5b4703c6a70d-osx64.tar.gz
  8c6d7b3f847faa7b4d16ceecf228f26f146ea982615c1d7a00c57f9230a0c484  guix-build-5b4703c6a70d/output/x86_64-linux-gnu/SHA256SUMS.part
  d0a8c99750319ad8046cfa132a54e5c13a08351f94439ae9af0f8e5486c2c2ea  guix-build-5b4703c6a70d/output/x86_64-linux-gnu/bitcoin-5b4703c6a70d-x86_64-linux-gnu-debug.tar.gz
  d816bb26dd4b0e309f2f576b1cccc6d78743fb2f357daad2da09bb1177330971  guix-build-5b4703c6a70d/output/x86_64-linux-gnu/bitcoin-5b4703c6a70d-x86_64-linux-gnu.tar.gz
  65caaa7f648c7eab1eb82c3331a2ca25b8cd4fe41439de55604501e02571de55  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/SHA256SUMS.part
  5bf6f7328cbceb0db22a2d7babb07b60cb6dcc19a6db84a1698589b7f5173a06  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win-unsigned.tar.gz
  7aabcb56115decef78d3797840b6e49dbc9b202d56f892490e92616fb06fec9e  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win64-debug.zip
  2f369694648ff9dc5ca1261a1e5874b1c7408ccf2802f9caef56c1334e8a5b7c  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win64-setup-unsigned.exe
  1c1f92513c4aad38419ff49a7b80bf10e6b1eca01ee8c5e3b2acd1768cf1e3d5  guix-build-5b4703c6a70d/output/x86_64-w64-mingw32/bitcoin-5b4703c6a70d-win64.zip
  ```

ACKs for top commit:
  hebasto:
    Approach ACK 5b4703c6a7.

Tree-SHA512: 2cd92a245ea64ef7176cf402a1fa5348a9421c30a4d30d01c950c48f6dcc15cf22ce69ffe1657be97e5fccc14bd933d64683c4439b695528ce3dc34d72dda927
2021-07-09 10:20:16 +08:00
MarcoFalke
df2b5dadfb Merge bitcoin/bitcoin#22408: test: add tests for bad-txns-prevout-null reject reason
1f449586a9 test: add `bad-txns-prevout-null` test to mempool_accept.py (Sebastian Falbesoner)
aa0a5bb70d test: add `bad-txns-prevout-null` test case to invalid_txs.py (Sebastian Falbesoner)

Pull request description:

  This simple PR adds missing tests for the reject reason `bad-txns-prevout-null`, which is thrown in the function `CheckTransaction()`: a62fc35a15/src/consensus/tx_check.cpp (L52-L54)

  Basically this condition is met for non-coinbase transactions (the code snippet above only hits if `!tx.IsCoinBase()`) with coinbase-like outpoints, i.e. hash=0, n=0xffffffff.

  Can be tested by running the functional tests `feature_block.py`, `p2p_invalid_tx.py` and `mempool_accept.py`. Not sure if the redundancy in the tests is desired (I guess it would make sense if the mempool acceptance test also makes use of the invalid_txs templates?).

ACKs for top commit:
  rajarshimaitra:
    tACK 1f449586a9
  brunoerg:
    tACK 1f449586a9
  kristapsk:
    ACK 1f449586a9, code looks correct and all tests pass.

Tree-SHA512: 2d4f940a6ac8e0d80d2670c9e1111cbf43ae6ac62809a2ccf17cffee9a41d387ea4d889ee300eb4a407c055b13bfa5d37102a32ed59964a9b6950bd907ba7204
2021-07-08 17:43:14 +02:00
Carl Dong
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys 2021-07-08 11:29:54 -04:00
Carl Dong
fc4f8449f3 guix: Update various check_tools lists 2021-07-08 11:29:26 -04:00
Carl Dong
263220a85c guix: Check for a sane services database
On bare systems, it is possible to be lacking a services database. Check
for basic entries before attempting a build.

See the error message in the diff for more context.
2021-07-08 11:28:07 -04:00
W. J. van der Laan
d968616422 Merge bitcoin/bitcoin#22179: Torv2 removal followups
00b875ba94 addrman: remove invalid addresses when unserializing (Vasil Dimov)
bdb62096f0 fuzz: reduce possible networks check (Vasil Dimov)
a164cd3ba6 net: simplify CNetAddr::IsRoutable() (Vasil Dimov)

Pull request description:

  * Simplify some code, now that we know `CNetAddr::IsRFC4193()` and `CNetAddr::IsTor()` cannot be `true` at the same time.
  * Drop Tor v2 addresses when loading addrman from `peers.dat` - they would have been loaded as dummy-all-zeros IPv6 addresses and linger in addrman, wasting space.

ACKs for top commit:
  sipa:
    ACK 00b875ba94. Reviewed the code, and tested with -DDEBUG_ADDRMAN (unit tests + mainnet run with peers.dat that contained v2 onions).
  laanwj:
    Code review and lightly tested ACK 00b875ba94
  jonatack:
    ACK 00b875ba94 reviewed, debug-built with -DEBUG_ADDRMAN rebased to current master, restarted node on mainnet/signet/testnet and verified that on each chain -addrinfo shows no change in address counts (as expected). Added some sanity check asserts, rebuilt/re-ran test. Checked that the new test fails on master with "test/addrman_tests.cpp(824): error: in "addrman_tests/remove_invalid": check addrman.size() == 2 has failed [4 != 2]"
  jarolrod:
    ACK 00b875ba94

Tree-SHA512: 6ed8e6745134b1b94fffaba28482de909ea39483b46b7f57bda61cdbae7a51251d15cb674de3631772fbeabe153d77a19269f96e62a89102a2d5c01e48f0ba06
2021-07-08 17:20:35 +02:00
Vasil Dimov
b1d905c225 p2p: earlier continuation when no remaining eviction candidates
in ProtectEvictionCandidatesByRatio().

With this change, `if (n.count == 0) continue;` will be true
if a network had candidates protected in the first iterations
and has no candidates remaining to be protected in later iterations.

Co-authored-by: Jon Atack <jon@atack.com>
2021-07-08 12:28:40 +02:00
Jon Atack
c9e8d8f9b1 p2p: process more candidates per protection iteration
for the usual case when some of the protected networks
don't have eviction candidates, to reduce the number
of iterations in ProtectEvictionCandidatesByRatio().

Picks up an idea in ef411cd2 that I had dropped.
2021-07-08 12:28:38 +02:00
Jon Atack
02e411ec45 p2p: iterate eviction protection only on networks having candidates
in ProtectEvictionCandidatesByRatio().

Thank you to Vasil Dimov, whose suggestions during a post-merge
discussion about PR 21261 reminded me that I had done this in
earlier versions of the PR, e.g. commits like ef411cd2.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-08 12:28:35 +02:00
Jon Atack
5adb064574 bench: add peer eviction protection benchmarks
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-08 12:28:23 +02:00
glozow
b7a8cd9963 [test] submit same txid different wtxid as mempool tx
Co-authored-by: Antoine Riard <ariard@student.42.fr>
Co-authored-by: Antoine Riard <antoine.riard@gmail.com>
2021-07-08 09:31:45 +01:00
glozow
fdb48163bf [validation] distinguish same txid different wtxid in mempool
Changes behavior.
2021-07-08 09:31:45 +01:00
Vasil Dimov
2feec3ce31 net: don't bind on 0.0.0.0 if binds are restricted to Tor
The semantic of `-bind` is to restrict the binding only to some address.
If not specified, then the user does not care and we bind to `0.0.0.0`.
If specified then we should honor the restriction and bind only to the
specified address.

Before this change, if no `-bind` is given then we would bind to
`0.0.0.0:8333` and to `127.0.0.1:8334` (incoming Tor) which is ok -
the user does not care to restrict the binding.

However, if only `-bind=addr:port=onion` is given (without ordinary
`-bind=`) then we would bind to `addr:port` _and_ to `0.0.0.0:8333` in
addition.

Change the above to not do the additional bind: if only
`-bind=addr:port=onion` is given (without ordinary `-bind=`) then bind
to `addr:port` (only) and consider incoming connections to that as Tor
and do not advertise it. I.e. a Tor-only node.
2021-07-07 15:46:38 +02:00
fanquake
4129134e84 Merge bitcoin/bitcoin#22402: doc: Install Rosetta on M1-macOS for qt in depends
cdb41d5573 doc: Install Rosetta on M1-macOS for qt in depends (Hennadii Stepanov)

Pull request description:

  On master (c609e10545) `make -C depends qt` on Apple Silicon based macOS 11.4 ends with an error:
  ```
  /bin/sh: /Users/hebasto/bitcoin/depends/work/build/aarch64-apple-darwin20.5.0/qt/5.12.11-6c4d47a8f8f/qtbase/bin/moc: Bad CPU type in executable
  ```

  Installing Rosetta 2 fixes it.

  Explanation. On Apple Silicon macOS the `qt` package in depends actually is cross compiled. All native tools (including `moc`) are x86_64 binaries, that require Rosetta 2 to run.

ACKs for top commit:
  promag:
    ACK cdb41d5573.
  fanquake:
    ACK cdb41d5573 - I have not tested after installing Rosetta 2, but I saw the same issue during my first cross-compile on an M1 box.
  Zero-1729:
    ACK cdb41d5573

Tree-SHA512: fb06a32d6fb40f405ce856b44f5d3af0c51089886f3be79e509e5c325614d7af58ce4480c064c17e0efb695a1f69f68d533c417f9631d46d8a630aba60ce4433
2021-07-07 20:59:35 +08:00
Carl Dong
5b4703c6a7 guix: Test security-check sanity before performing them 2021-07-07 20:29:05 +08:00
fanquake
6cf3345297 scripts: adjust test-symbol-check for guix release environment
Now that our release binaries are build in a glibc 2.24 and 2.27
environment, we can't use a symbol from glibc 2.28 to test our checks.
Replace renameat2() with nextup(), which was introduced in 2.24.

Note that this also means re-disabling the test for RISC-V, however
RISC-V is built in a glibc 2.27 environment, and our minimum required
glibc for that binary is 2.27.
2021-07-07 20:29:01 +08:00
fanquake
1946b5f77c scripts: more robustly test macOS symbol checks 2021-07-07 19:31:37 +08:00
Carl Dong
a8127b34bc build: Use and test PE binutils with --reloc-section
Also fix test-security-check.py to account for new PE PIE failure
indication.
2021-07-07 19:31:37 +08:00
Carl Dong
678348db51 guix: Patch binutils to add security-related disable flags
We use these flags in our test-security-check make target, but they are
only available because debian patches them in.

We can patch them in for our Guix builds so that we can check the sanity
of our security/symbol checking suite before running them.
2021-07-07 19:31:37 +08:00
Carl Dong
9fdc8afe11 devtools: Improve *-check.py tool detection
This is important to make sure that we're not testing tools different
from the one we're building with.

Introduce determine_wellknown_cmd, which encapsulates how we
should handle well-known tools specification (IFS splitting, env
override, etc.).
2021-07-07 19:31:37 +08:00
fanquake
bda62eab38 ci: skip running the Linux test-security-check target for now
The CI environment is a moving target, and these tests are somewhat
fragile, so for now, disable them.
2021-07-07 19:31:37 +08:00
Carl Dong
d6ef3543ae lint: Run mypy with --show-error-codes
When using mypy ignore directives, the error code needs to be specified.
Somehow mypy doesn't print it by default...
2021-07-07 19:30:47 +08:00
fanquake
137f3b9ba3 Merge bitcoin/bitcoin#22405: build: remove --enable-glibc-back-compat from Guix build
797b3ed909 script: remove gitian reference from symbol-check.py (fanquake)
15fc9a0299 guix: add additional documentation to patches (fanquake)
4516e5ec92 lint: exclude Guix patches from spell-checking (fanquake)
de6ca41a52 guix: no-longer pass --enable-glibc-back-compat to Guix (fanquake)
84dd81fb5b build: remove glibc backcompat requirement for Linux symbol checks (fanquake)

Pull request description:

  Now that our Guix toolchains are based on glibc 2.24 and 2.27 (RISCV), we don't need to use the `--enable-glibc-back-compat` option to produce binaries that don't use any symbols from glibc 2.17 and 2.27 or later.

  This also adds additional documentation to some Guix patches (pointed out in #22365) and removes Guix patches from the spelling linter, because that isn't our spelling.

  Symbol usage: https://gist.github.com/fanquake/d15604fc580718444c5aa4b3c3c75fdc.

  Guix Builds:
  ```bash
  bash-5.1# find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ed54e6a6cf4fab328557c0c72eb08c73f2a58c6c70959544cf4b1882e75ea69e  guix-build-797b3ed90900/output/aarch64-linux-gnu/SHA256SUMS.part
  83bd9dadc59f89f848d143fa4fc3964f16fe0b4bdf35e5093b577ff2c4bd1f43  guix-build-797b3ed90900/output/aarch64-linux-gnu/bitcoin-797b3ed90900-aarch64-linux-gnu-debug.tar.gz
  94cb8c35281f12dec6ea5b390b66cad5e27ac8c45a30c42c8d38c438695d54c0  guix-build-797b3ed90900/output/aarch64-linux-gnu/bitcoin-797b3ed90900-aarch64-linux-gnu.tar.gz
  7318b63d65c0aa52d2446de8e1f40658d2e47ab8fb0268820c3b7585d140fb23  guix-build-797b3ed90900/output/arm-linux-gnueabihf/SHA256SUMS.part
  95e1ffb372964b73f539653ca703b70cf0c018801a9c4c0ffc46a0b63539253c  guix-build-797b3ed90900/output/arm-linux-gnueabihf/bitcoin-797b3ed90900-arm-linux-gnueabihf-debug.tar.gz
  039d3842e6499626cf955ae0a7590dd6b3d0935cdc217c98aaf9d156b0ebd3b4  guix-build-797b3ed90900/output/arm-linux-gnueabihf/bitcoin-797b3ed90900-arm-linux-gnueabihf.tar.gz
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-797b3ed90900/output/dist-archive/SKIPATTEST.TAG
  2c4e7b6e7aff63ba811e5bf59362d16866c3a358f8844fba8739a61192870622  guix-build-797b3ed90900/output/dist-archive/bitcoin-797b3ed90900.tar.gz
  955029b949c368eabd517dd33040d2f01e2ac6a55e7b4f9107907a7c6e0c6060  guix-build-797b3ed90900/output/powerpc64-linux-gnu/SHA256SUMS.part
  fd6d6b137f8efedf58a879d11205b1d4649e1f97d7f91e193239ef206fcc285d  guix-build-797b3ed90900/output/powerpc64-linux-gnu/bitcoin-797b3ed90900-powerpc64-linux-gnu-debug.tar.gz
  51736ac8e77737999f1b5bd4c381b0016f19a8d5e40e786fe941ff04e84c11c9  guix-build-797b3ed90900/output/powerpc64-linux-gnu/bitcoin-797b3ed90900-powerpc64-linux-gnu.tar.gz
  8c244c16bfa46c1efdb120e1d91fdd14d3f14eefee8d7e1fbb0a9b4664a5c315  guix-build-797b3ed90900/output/powerpc64le-linux-gnu/SHA256SUMS.part
  704ee593251a1b1c65a5bebeef93b23f266af4e8cbf8ae556150c3b2e8f06a6c  guix-build-797b3ed90900/output/powerpc64le-linux-gnu/bitcoin-797b3ed90900-powerpc64le-linux-gnu-debug.tar.gz
  0ec06ae7d344de20d61e3965d8b383747ef20b0e9d93a3165733ea23bdf2ead8  guix-build-797b3ed90900/output/powerpc64le-linux-gnu/bitcoin-797b3ed90900-powerpc64le-linux-gnu.tar.gz
  2dd6c6ecc67b0ea40ca9c43f92efca81ccd054b8db8c197ad84ad9674d510a25  guix-build-797b3ed90900/output/riscv64-linux-gnu/SHA256SUMS.part
  5ebb27a855a677f7a188d83995be6b2a3ea8606be152abb7fc7832713fb0677a  guix-build-797b3ed90900/output/riscv64-linux-gnu/bitcoin-797b3ed90900-riscv64-linux-gnu-debug.tar.gz
  bdaf1783f5e1861597afa37c1880364e118d9a7a7af8017302d82202791019f6  guix-build-797b3ed90900/output/riscv64-linux-gnu/bitcoin-797b3ed90900-riscv64-linux-gnu.tar.gz
  726c9092b60ac2e7d7e14b2c24467fcf276a6f89170a871ddab9dce6ac230699  guix-build-797b3ed90900/output/x86_64-apple-darwin18/SHA256SUMS.part
  2af4d709b44952654f3c08c86593bf2ccc9a44ed422783a1b95b8a199a894db2  guix-build-797b3ed90900/output/x86_64-apple-darwin18/bitcoin-797b3ed90900-osx-unsigned.dmg
  fd49ba445aa6cf3d8c47019a05e9e5740cb0f53349344dd80671297127f49f1a  guix-build-797b3ed90900/output/x86_64-apple-darwin18/bitcoin-797b3ed90900-osx-unsigned.tar.gz
  3f51cbf8cf18420d4be70e656aa993675cf5e828a255c2030047ae2e059ed5b7  guix-build-797b3ed90900/output/x86_64-apple-darwin18/bitcoin-797b3ed90900-osx64.tar.gz
  afd1edee1447bb88d81e972abfae4c4e065b5b1827769f033cff9472084c7c1b  guix-build-797b3ed90900/output/x86_64-linux-gnu/SHA256SUMS.part
  ec468ef886d25e685f4f7a18b4f7d497dedf757495e0d5beb72c23cc32ab69b5  guix-build-797b3ed90900/output/x86_64-linux-gnu/bitcoin-797b3ed90900-x86_64-linux-gnu-debug.tar.gz
  1934d7294f0c9e083d38a3f68d4a61cd679defa79ce0a89f77386978692b9b18  guix-build-797b3ed90900/output/x86_64-linux-gnu/bitcoin-797b3ed90900-x86_64-linux-gnu.tar.gz
  94c11c328a628052eb6f50e9816aa768f87ea7acfbbbafdab60f6928da766811  guix-build-797b3ed90900/output/x86_64-w64-mingw32/SHA256SUMS.part
  fd371922ba93d81bd4a2b711d617af6756f9f0494db6d83aa0e5f491a24168ef  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win-unsigned.tar.gz
  4e4ad976bc029bbbf9596ad8493accaaba8b0d5c598dd342f8da330609bbdf21  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win64-debug.zip
  3a89a16b9101e9a17d98efb9234b5bdd264c0bba2c6326511017730e1a08311f  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win64-setup-unsigned.exe
  e285ab737e3c843fd3f1c26c2f053e421a3c39b33995747ce48281884d3f28d1  guix-build-797b3ed90900/output/x86_64-w64-mingw32/bitcoin-797b3ed90900-win64.zip
  ```

ACKs for top commit:
  sipa:
    utACK 797b3ed909
  hebasto:
    ACK 797b3ed909

Tree-SHA512: 3a569702d8832c155c5ce8d2f6d823f7f12603885576078bc5192bc9038a48261ecb541800f79d1e9bc86d71fa640265c5b8b89df9d8bb680b3bb05d9d78a666
2021-07-07 19:21:57 +08:00
MarcoFalke
914c0cad97 Merge bitcoin/bitcoin#22399: fuzz: Rework CTxDestination fuzzing
fabf17056c fuzz: Move CTxDestination fuzzing to script fuzz target (MarcoFalke)
fa42800a51 fuzz: Simplify CTxDestination fuzzing in the script target (MarcoFalke)
fab99865c0 fuzz: Improve ConsumeTxDestination (MarcoFalke)
fa40c0964b fuzz: Move ConsumeTxDestination to cpp file (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    cr ACK fabf17056c

Tree-SHA512: afd2cf384d04a810c0c462c6d80849bd0fefd017d7acac877f64f2bffae3fc8d687701bc479e67a727a05f43431a17cb4ccaf09c6b3c68106562c94b7ed19250
2021-07-07 12:03:49 +02:00
fanquake
692b78793b Merge bitcoin/bitcoin#22360: doc: Remove unused section from release process
fa051d3e5e doc: Remove unused section from release process (MarcoFalke)

Pull request description:

  The section does not apply to us, because it is being done by the maintainers of the `bitcoin.org` website, see https://github.com/bitcoin-dot-org/Bitcoin.org/issues/3641#issuecomment-819586576 . Documentation about the process is in their repo: https://github.com/bitcoin-dot-org/bitcoin.org/blob/master/docs/adding-events-release-notes-and-alerts.md#release-notes

ACKs for top commit:
  jarolrod:
    ACK fa051d3e5e
  Zero-1729:
    ACK fa051d3e5e
  fanquake:
    ACK fa051d3e5e

Tree-SHA512: eadc192958bb0d1009ad94c2ec79087b87514e3d19e2e6d759440392fa4abc432c512ba60692018e789c822a98a515a5f9a42966de4f8fe6b0a8b99470a76b5c
2021-07-07 14:45:41 +08:00
Pieter Wuille
1edddf5de4 Avoid GCC 7.1 ABI change warning in guix build 2021-07-06 10:41:43 -07:00
Jon Atack
566357f8f7 refactor: move GetRandomNodeEvictionCandidates() to test utilities
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-06 17:28:15 +02:00
Brian Liotti
fa84caebc7 doc: added info to bitcoin.conf doc 2021-07-06 09:32:37 -04:00
fanquake
797b3ed909 script: remove gitian reference from symbol-check.py 2021-07-06 20:50:47 +08:00
fanquake
15fc9a0299 guix: add additional documentation to patches 2021-07-06 20:50:47 +08:00
fanquake
4516e5ec92 lint: exclude Guix patches from spell-checking
Co-authored-by: Carl Dong <contact@carldong.me>
2021-07-06 20:50:46 +08:00
fanquake
de6ca41a52 guix: no-longer pass --enable-glibc-back-compat to Guix
Now that our Guix builds are performed on glibc 2.24 and 2.27 (RISCV),
we no-longer need to pass the --enable-glibc-back-compat option.

Replace it with --disable-threadlocal, to prevent the usage of symbols
from glibc 2.18.

None of the binaries produced required symbols later than 2.17, and 2.27
(RISCV).
2021-07-06 20:50:40 +08:00
fanquake
84dd81fb5b build: remove glibc backcompat requirement for Linux symbol checks 2021-07-06 15:21:17 +08:00
fanquake
088b348dbe Merge bitcoin/bitcoin#22406: build: remove --enable-determinism configure option
e46287853f build: remove --enable-determinism configure option (fanquake)

Pull request description:

  This was added by me a while back, with the intention of expanding what this did. That hasn't happened, and this hasn't gained much use. There's also been some discussion of some configure option fatigue, so just remove it for now. Note that `-Wl,--no-insert-timestamp` is also already used in the Guix build.

ACKs for top commit:
  MarcoFalke:
    review ACK e46287853f
  jarolrod:
    Code Review ACK e46287853f

Tree-SHA512: ac976f88203eca2a49e296a98693dbe53330e0cb0e273c5ff1fcded30daeb6070cc5beeae35cf9acfdc2279cd64c274d5aeb588aef077aa9bfde39bb23570491
2021-07-06 14:11:38 +08:00
Sebastian Falbesoner
1f449586a9 test: add bad-txns-prevout-null test to mempool_accept.py 2021-07-05 23:06:06 +02:00
Hennadii Stepanov
a62fc35a15 Merge bitcoin-core/gui#375: Emit dataChanged signal to dynamically re-sort Peers table
986bf78d7e qt: Emit dataChanged signal to dynamically re-sort Peers table (Hennadii Stepanov)

Pull request description:

  [By default](https://doc.qt.io/qt-5/qsortfilterproxymodel.html#details), the `PeerTableSortProxy`
  > dynamically re-sorts ... data whenever the original model changes.

  That is not the case on master (8cdf91735f) as in ecbd911538 (#164) no signals are emitted to notify about model changes.

  This PR uses a dedicated [`dataChanged`](https://doc.qt.io/qt-5/qabstractitemmodel.html#dataChanged) signal.

  Fixes #367.

  An alternative to #374.

ACKs for top commit:
  jarolrod:
    ACK 986bf78d7e

Tree-SHA512: dcb92c2f9a2c632880429e9528007db426d2ad938c64dfa1f1538c03e4b62620df52ad7daf33b582976c67b472ff76bc0dae707049f4bbbd4941232cee9ce3d4
2021-07-06 00:02:49 +03:00
Sebastian Falbesoner
aa0a5bb70d test: add bad-txns-prevout-null test case to invalid_txs.py
This reject reason is triggered for non-coinbase transactions with
a coinbase-like outpoint, i.e. hash=0, n=0xffffffff.

Note that the invalid tx templates are currently used in the
functional tests feature_block.py and p2p_invalid_tx.py.
2021-07-05 23:00:41 +02:00
Hennadii Stepanov
484d4ee226 Merge bitcoin-core/gui#365: Draw "eye" sign at the beginning of watch-only addresses
cd46c11577 qt: Draw "eye" sign at the beginning of watch-only addresses (Hennadii Stepanov)
9ea1da6fc9 qt: Do not extend recent transaction width to address/label string (Hennadii Stepanov)

Pull request description:

  This PR guaranties that the "eye" sign won't be hidden for very long addresses/labels.

  No longer need to extend `TransactionOverviewWidget` widget width to make "eye" signs shown:

  ![Screenshot from 2021-06-15 00-21-05](https://user-images.githubusercontent.com/32963518/121961807-9123b600-cd70-11eb-8cdd-8b2b0d1bf44f.png)

  Fixes https://github.com/bitcoin-core/gui/issues/373

ACKs for top commit:
  jarolrod:
    ACK cd46c11577

Tree-SHA512: 0602b5bb65d53c5b18e86260750006bba03adbae181917b5a2b7f89b17290bd1f57b4f80adaba32f42cc6fb468598a888b12c0b6b09005d2f2c07bd4d1ad334a
2021-07-05 23:48:21 +03:00
Sebastian Falbesoner
905d672b74 test: use script_util helpers for creating P2W{PKH,SH} scripts 2021-07-05 20:40:52 +02:00
Sebastian Falbesoner
285a65ccfd test: use script_util helpers for creating P2SH scripts 2021-07-05 20:40:47 +02:00
fanquake
e46287853f build: remove --enable-determinism configure option 2021-07-05 23:53:20 +08:00
fanquake
2711559845 Merge bitcoin/bitcoin#22365: guix: Avoid relying on newer symbols by rebasing our cross toolchains on older glibcs
647f7e5f1d guix: Also sort SHA256SUMS.part (Carl Dong)
dc4137a60c guix: Build depends/qt with our platform definition (Carl Dong)
16b0a936e1 guix: Rebase toolchain on glibc 2.24 (2.27 for riscv64) (Carl Dong)

Pull request description:

  After this PR, we'll have the following:
  - riscv64 -> build with a toolchain targeting glibc 2.27
  - everything else -> builds with a toolchain targeting glibc 2.24, but will not have symbols > 2.17 (checked by `symbol-check.py`)

ACKs for top commit:
  achow101:
    reACK 647f7e5f1d
  hebasto:
    ACK 647f7e5f1d
  MarcoFalke:
    review ACK 647f7e5f1d
  fanquake:
    ACK 647f7e5f1d - documentation can be fixed shortly.

Tree-SHA512: ddff57a5d7c053687b0a273720d4ad7d28c6fc8816226d4304869284d017af5e3630d4b57565d91e74f2e1b7583c9c83ee8b2e5e70e41d619ab618e602c97a94
2021-07-05 22:40:29 +08:00
Hennadii Stepanov
cdb41d5573 doc: Install Rosetta on M1-macOS for qt in depends 2021-07-05 13:42:14 +03:00
Hennadii Stepanov
3c4c8e79ba build: Add -Werror=implicit-fallthrough compile flag 2021-07-05 08:59:54 +03:00
Hennadii Stepanov
014110c47d Use C++17 [[fallthrough]] attribute, and drop -Wno-implicit-fallthrough 2021-07-05 08:59:38 +03:00
fanquake
c609e10545 Merge bitcoin/bitcoin#22292: bench, doc: benchmarking updates and fixups
d8513fe411 doc: update doc/benchmarking.md (Jon Atack)
84e2d5b781 bench: bench_bitcoin.cpp help fixups (Jon Atack)
10f4ce2078 bench: bench.h fixes and improvements (Jon Atack)

Pull request description:

  Fixups and updates I noticed while writing benchmarks for #22284.

ACKs for top commit:
  za-kk:
    ACK d8513fe411
  theStack:
    ACK d8513fe411 🚤

Tree-SHA512: d494956b5d6a3329e98e8b6f4405a10613b8fce51a04bbf4493d8b3497b8d5b177c1a9a3eeb828796eb4edb92b0ace769595151e223671c0dc8f09bcf631ebb5
2021-07-05 08:47:36 +08:00
MarcoFalke
fabf17056c fuzz: Move CTxDestination fuzzing to script fuzz target
No need to split it over several targets
2021-07-04 21:30:50 +02:00
MarcoFalke
fa42800a51 fuzz: Simplify CTxDestination fuzzing in the script target
The WitnessUnknown operators == and < are already called indirectly by
the corresponding CTxDestination operators.
2021-07-04 21:29:56 +02:00
MarcoFalke
fab99865c0 fuzz: Improve ConsumeTxDestination
* Assert when a type is missing
* Add missing WitnessV1Taproot
* Limit WitnessUnknown to version [2, 16], to avoid abiguity
* Limit WitnessUnknown to size [2, 40], to avoid invalid sizes
2021-07-04 21:28:35 +02:00
MarcoFalke
fa40c0964b fuzz: Move ConsumeTxDestination to cpp file
Moving the implementation out of the header will reduce compile time
2021-07-04 21:28:04 +02:00
Wladimir J. van der Laan
a16378e501 doc: Remove unnecessary steps from translations update process
These have been integrated into the script long ago.
2021-07-03 21:31:29 +02:00
Wladimir J. van der Laan
2584929823 doc: Add steps for transifex to release process
Document how to update settings on and for the transifex website before
and after branch-off of a new release.
2021-07-03 21:31:26 +02:00
Carl Dong
647f7e5f1d guix: Also sort SHA256SUMS.part 2021-07-02 14:10:33 -04:00
MarcoFalke
7a49fdc581 Merge bitcoin/bitcoin#22388: ci: use Ubuntu 20.04 as the default Docker container
7fc1e14ce6 ci: use Ubuntu 20.04 as the default Docker container (fanquake)

Pull request description:

  All but 2 of the Ubuntu CIs (native qt5 & nowallet) are already using 20.04 or 21.04.

ACKs for top commit:
  MarcoFalke:
    cr ACK 7fc1e14ce6

Tree-SHA512: f35d79a87af6c6955695b5e627884f94aed19bafaed4657d03ef4db66cf47cae5311464bb39961570140325652941283b9d88dff862776e8becfff9130162917
2021-07-02 07:29:12 +02:00
fanquake
7fc1e14ce6 ci: use Ubuntu 20.04 as the default Docker container
All but 2 of the Ubuntu CIs are already using 20.04 or 21.04.
2021-07-02 10:29:03 +08:00
Carl Dong
dc4137a60c guix: Build depends/qt with our platform definition
Our 'bitcoin-linux-g++' definition better integrates with our depends
system than the stock linux-g++-64 definition.

This fixes a bug whereby Guix builds on x86_64 for x86_64 did not
produce a QMinimalIntegrationPlugin and led to bitcoin-qt not being
built.
2021-07-01 16:20:28 -04:00
Carl Dong
16b0a936e1 guix: Rebase toolchain on glibc 2.24 (2.27 for riscv64)
Support for riscv64 in glibc landed in 2.27 so it's unavoidable that we
use 2.27.

Running a Bitcoin build with toolchains based on 2.24 for platforms
other than riscv64 seem to produce binaries which do not have 2.17
symbols. So use 2.24 since it's more recent and maintained by Debian
Stretch.
2021-07-01 16:17:03 -04:00
MarcoFalke
fa5658ed07 Use DeploymentEnabled to hide VB deployments
The helper was previously unused. This commit changes it to be more
meaningful and puts it to use.

See previous discussion at https://github.com/bitcoin/bitcoin/pull/19438/files#r650687320
2021-07-01 20:20:52 +02:00
MarcoFalke
fa11fecf0d doc: Move buried deployment doc to the enum that enumerates them
This is more visible than a comment hidden in an RPC helper function.
2021-07-01 20:04:07 +02:00
MarcoFalke
ddc6979b8b Merge bitcoin/bitcoin#19438: Introduce deploymentstatus
e48826ad87 tests: remove ComputeBlockVersion shortcut from versionbits tests (Anthony Towns)
c5f36725e8 [refactor] Move ComputeBlockVersion into VersionBitsCache (Anthony Towns)
4a69b4dbe0 [move-only] Move ComputeBlockVersion from validation to versionbits (Anthony Towns)
0cfd6c6a8f [refactor] versionbits: make VersionBitsCache a full class (Anthony Towns)
8ee3e0bed5 [refactor] rpc/blockchain.cpp: SoftForkPushBack (Anthony Towns)
92f48f360d deploymentinfo: Add DeploymentName() (Anthony Towns)
ea68b3a572 [move-only] Rename versionbitsinfo to deploymentinfo (Anthony Towns)
c64b2c6a0f scripted-diff: rename versionbitscache (Anthony Towns)
de55304f6e [refactor] Add versionbits deployments to deploymentstatus.h (Anthony Towns)
2b0d291da8 [refactor] Add deploymentstatus.h (Anthony Towns)
eccd736f3d versionbits: Use dedicated lock instead of cs_main (Anthony Towns)
36a4ba0aaa versionbits: correct doxygen comments (Anthony Towns)

Pull request description:

  Introduces helper functions to make it easy to bury future deployments, along the lines of the suggestion from [11398](https://github.com/bitcoin/bitcoin/pull/11398#issuecomment-335599326) "I would prefer it if a buried deployment wouldn't require all code paths that check the BIP9 status to require changing".

  This provides three functions: `DeploymentEnabled()` which tests if a deployment can ever be active, `DeploymentActiveAt()` which checks if a deployment should be enforced in the given block, and `DeploymentActiveAfter()` which checks if a deployment should be enforced in the block following the given block, and overloads all three to work both with buried deployments and versionbits deployments.

  This adds a dedicated lock for the versionbits cache, which is acquired internally by the versionbits functions, rather than relying on `cs_main`. It also moves moves versionbitscache into deploymentstatus to avoid a circular dependency with validation.

ACKs for top commit:
  jnewbery:
    ACK e48826ad87
  gruve-p:
    ACK e48826ad87
  MarcoFalke:
    re-ACK e48826ad87 🥈

Tree-SHA512: c846ba64436d36f8180046ad551d8b0d9e20509b9bc185aa2639055fc28803dd8ec2d6771ab337e80da0b40009ad959590d5772f84a0bf6199b65190d4155bed
2021-07-01 19:15:09 +02:00
MarcoFalke
a926d6dfd2 Merge bitcoin/bitcoin#22310: test: Add functional test for replacement relay fee check
c4ddee64c7 test: Add test for replacement relay fee check (Antoine Riard)

Pull request description:

  This PR adds rename the `reject_reason` of our implementation of BIP125 rule 4 and adds missing functional test coverage. Note, `insufficient fee` is already the `reject_reason` of few others `PreChecks` replacement checks and as such might be confusing.

  > The replacement transaction must also pay for its own bandwidth at or above the rate set by the node's minimum relay fee setting. For example, if the minimum relay fee is 1 satoshi/byte and the replacement transaction is 500 bytes total, then the replacement must pay a fee at least 500 satoshis higher than the sum of the originals.

  ```
          // Finally in addition to paying more fees than the conflicts the
          // new transaction must pay for its own bandwidth.
          CAmount nDeltaFees = nModifiedFees - nConflictingFees;
          if (nDeltaFees < ::incrementalRelayFee.GetFee(nSize))
          {
              return state.Invalid(TxValidationResult::TX_MEMPOOL_POLICY, "insufficient fee",
                      strprintf("rejecting replacement %s, not enough additional fees to relay; %s < %s",
                          hash.ToString(),
                          FormatMoney(nDeltaFees),
                          FormatMoney(::incrementalRelayFee.GetFee(nSize))));
          }
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK c4ddee64c7
  glozow:
    ACK c4ddee6, one small suggestion if you retouch.

Tree-SHA512: 7c5d1065db6e6fe57a9f083bf051a7a55eb9892de3a2888679d4a6853491608c93b6e35887ef383a9988d14713fa13a0b1d6134b7354af5fd54765f0d4e98568
2021-07-01 18:36:17 +02:00
Hennadii Stepanov
091d35c70e Merge bitcoin-core/gui#377: Translations update
c7f74f1a7f Translations update (Hennadii Stepanov)

Pull request description:

  Translation string freeze, see [Release schedule for 22.0](https://github.com/bitcoin/bitcoin/issues/20851).

ACKs for top commit:
  laanwj:
    ACK c7f74f1a7f, I get the same output

Tree-SHA512: 85c12a88290f46db0d6724ef51c2789bb1f7dfc242682b95420cb1310cb986e8d8a53e628fb7e184008ca23236e36bb5dc8ea65c4e41e01ca2c8f17863894125
2021-07-01 17:40:30 +03:00
Hennadii Stepanov
c7f74f1a7f Translations update
Translation string freeze, see Release schedule for 22.0.
2021-07-01 17:16:14 +03:00
fanquake
2749613020 Merge bitcoin/bitcoin#22348: build: Fix cross build for Windows with Boost Process
67669ab425 build: Fix Boost Process compatibility with mingw-w64 compiler (Hennadii Stepanov)

Pull request description:

  On master (9c3751a0c9) the cross build for Win64 is broken if configured with `--enable-external-signer`:
  ```
  ...
    CXX      crypto/libbitcoin_crypto_base_a-chacha_poly_aead.o
  In file included from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handles.hpp:11,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/used_handles.hpp:17,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/async_in.hpp:20,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/async.hpp:49,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process.hpp:23,
                   from util/system.cpp:9:
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp:208:51: error: expected ‘)’ before ‘*’ token
    208 | typedef ::boost::winapi::NTSTATUS_ (__kernel_entry *nt_system_query_information_p )(
        |                                    ~              ^~
        |                                                   )
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp:223:51: error: expected ‘)’ before ‘*’ token
    223 | typedef ::boost::winapi::NTSTATUS_ (__kernel_entry *nt_query_object_p )(
        |                                    ~              ^~
        |                                                   )
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp: In function ‘boost::winapi::NTSTATUS_ boost::process::detail::windows::workaround::nt_system_query_information(boost::process::detail::windows::workaround::SYSTEM_INFORMATION_CLASS_, void*, boost::winapi::ULONG_, boost::winapi::PULONG_)’:
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp:239:12: error: ‘nt_system_query_information_p’ does not name a type; did you mean ‘nt_system_query_information’?
    239 |     static nt_system_query_information_p f = reinterpret_cast<nt_system_query_information_p>(::boost::winapi::get_proc_address(h, "NtQuerySystemInformation"));
        |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        |            nt_system_query_information
  In file included from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handles.hpp:11,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/used_handles.hpp:17,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/async_in.hpp:20,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/async.hpp:49,
                   from /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process.hpp:23,
                   from util/system.cpp:9:
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp:241:14: error: ‘f’ was not declared in this scope
    241 |     return (*f)(SystemInformationClass, SystemInformation, SystemInformationLength, ReturnLength);
        |              ^
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp: In function ‘boost::winapi::BOOL_ boost::process::detail::windows::workaround::nt_query_object(boost::winapi::HANDLE_, boost::process::detail::windows::workaround::OBJECT_INFORMATION_CLASS_, void*, boost::winapi::ULONG_, boost::winapi::PULONG_)’:
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp:253:12: error: ‘nt_query_object_p’ does not name a type; did you mean ‘nt_query_object’?
    253 |     static nt_query_object_p f = reinterpret_cast<nt_query_object_p>(::boost::winapi::get_proc_address(h, "NtQueryObject"));
        |            ^~~~~~~~~~~~~~~~~
        |            nt_query_object
  /home/hebasto/GitHub/bitcoin/depends/x86_64-w64-mingw32/include/boost/process/detail/windows/handle_workaround.hpp:255:14: error: ‘f’ was not declared in this scope
    255 |     return (*f)(Handle, ObjectInformationClass, ObjectInformation, ObjectInformationLength, ReturnLength);
        |              ^
  make[2]: *** [Makefile:9906: util/libbitcoin_util_a-system.o] Error 1
  make[2]: *** Waiting for unfinished jobs....
    CXX      crypto/libbitcoin_crypto_base_a-chacha20.o
  make[2]: Leaving directory '/home/hebasto/GitHub/bitcoin/src'
  make[1]: *** [Makefile:16141: all-recursive] Error 1
  make[1]: Leaving directory '/home/hebasto/GitHub/bitcoin/src'
  make: *** [Makefile:820: all-recursive] Error 1
  ```

  The upstream bug: https://github.com/boostorg/process/issues/96
  Also see: https://stackoverflow.com/a/59338759

  https://github.com/bitcoin/bitcoin/pull/22348#issuecomment-871061160:
  > [This commit](7fc41b2815), containing the `__kernel_entry` [SAL annotations](https://docs.microsoft.com/en-us/cpp/code-quality/using-sal-annotations-to-reduce-c-cpp-code-defects?view=msvc-160) was included in Boost Process as part of the `1.71.0` release, which broke support for compiling with mingw-w64 because it doesn't define the `__kernel_entry` SAL annotation (but it does define some others, i.e see [`sal.h`](https://github.com/mirror/mingw-w64/blob/master/mingw-w64-headers/include/sal.h)).
  >
  > A [commit was made](d7a721ee0d) to remove the annotations, however, it hasn't made it into either of the two Boost releases that have happened since (1.75.0 & 1.76.0). Meaning that this is currently needed for all versions of Boost process from 1.71.0 onwards.

ACKs for top commit:
  fanquake:
    ACK 67669ab425 - thanks for updating this.

Tree-SHA512: 5931ca1fb77ce38c042cf5a7556add024ea2386c208bf26c792a8ca4a771d97fac9802c32fa8aa2e3de1ad35f3362d8c066f0a83ee675859d226c602fd0bcf93
2021-07-01 20:17:11 +08:00
fanquake
185acdb5e8 Merge bitcoin/bitcoin#22334: wallet: do not spam about non-existent spk managers
6084d2caed wallet: do not spam about non-existent spk managers (S3RK)

Pull request description:

  Avoid spam in logs during `loadwallet`, `listdescriptors` and probably other commands as well.

  **`loadwallet` Before:**
  ```
  2021-06-24T06:31:45Z init message: Loading wallet…
  2021-06-24T06:31:45Z [desc] Wallet File Version = 169900
  2021-06-24T06:31:45Z [desc] Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total. Unknown wallet records: 0
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Wallet completed loading in             197ms
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] setKeyPool.size() = 0
  2021-06-24T06:31:45Z [desc] mapWallet.size() = 0
  2021-06-24T06:31:45Z [desc] m_address_book.size() = 0
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  {
    "name": "desc",
    "warning": ""
  }
  ```

  **After:**
  ```
  2021-06-24T06:26:58Z init message: Loading wallet…
  2021-06-24T06:26:58Z [desc] Wallet File Version = 169900
  2021-06-24T06:26:58Z [desc] Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total. Unknown wallet records: 0
  2021-06-24T06:26:58Z [desc] Wallet completed loading in             158ms
  2021-06-24T06:26:58Z [desc] setKeyPool.size() = 0
  2021-06-24T06:26:58Z [desc] mapWallet.size() = 0
  2021-06-24T06:26:58Z [desc] m_address_book.size() = 0
  {
    "name": "desc",
    "warning": ""
  }
  ```

ACKs for top commit:
  achow101:
    ACK 6084d2caed

Tree-SHA512: c7d7345c3182a575db088fd731b7f6e428c42e4f3f2e10d5adb50bf74a2defe88768e65ebb91a08590be48cf766a5697e36fafa73f68ffe45e76a60600f072e2
2021-07-01 19:11:20 +08:00
fanquake
fa46e48982 Merge bitcoin/bitcoin#22379: wallet: erase spkmans rather than setting to nullptr
b945a31afa wallet: erase spkmans rather than setting to nullptr (Andrew Chow)

Pull request description:

  In many places in ScriptPubKeyMan managing code, we assume that the ScriptPubKeyMan being retrieved actually exists and is not a nullptr. Thus removing a ScriptPubKeyMan requires erasing the object from the map rather than setting it to a nullptr.

  This fixes a segmentation fault that can be reached with `test/functional/wallet_descriptors.py --descriptors`

ACKs for top commit:
  S3RK:
    ACK b945a31

Tree-SHA512: 344a4cf9b1c168428750c751dcd24c52032506f20c81977fe93c4b5307ea209de72bb62a9c5284820f225b03acdc9573fceb734833d29b82f49d5a799ddcaea7
2021-07-01 18:03:32 +08:00
Hennadii Stepanov
67669ab425 build: Fix Boost Process compatibility with mingw-w64 compiler
Boost 1.71 has a broken compatibility with mingw-w64 compiler due to the
added __kernel_entry SAL annotations.
2021-07-01 12:16:47 +03:00
MarcoFalke
3ef2d400fa Merge bitcoin/bitcoin#22376: ci: Do not clone bitcoin-core/qa-assets git repository if not necessary
30450a1bd5 Do not clone qa-assets git repository if not necessary (Kiminuo)

Pull request description:

  This PR attempts to remove an unnecessary step when CI runs.

  The main motivation for the change is that I locally use `MAKEJOBS="-j15" FILE_ENV="./ci/test/00_setup_env_android.sh" ./ci/test_run_all.sh` to find out if a patch of mine works or not. Cloning `bitcoin-core/qa-assets` is slow on my machine (which is by no means slow).

ACKs for top commit:
  MarcoFalke:
    cr ACK 30450a1bd5

Tree-SHA512: 5763b53da9554b06039c39f8fc729de1b106cce2a242de8f97528d001bfa01d4f48d2a128f458a3cdee3da36312354c6714839b947f313c089c2c5cb30233a39
2021-07-01 08:32:00 +02:00
Hennadii Stepanov
333ec8be82 Merge bitcoin-core/gui#330: Allow prompt icon to be colorized
2f23ad2c40 qt: allow prompt icon to be colorized (Jarol Rodriguez)

Pull request description:

  Opening the console on macOS, while in dark mode, the console prompt icon will not be colorized white like other icons. This applies the `platformStyle` to the icon so that It can be colorized white.

  While here, refactor the `promptIcon` widget from a `QPushButton` to `QLabel`; which is more appropriate, per [Qt Docs](https://doc.qt.io/qt-5/qlabel.html#details):
  > QLabel is used for displaying text or an image. No user interaction functionality is provided.

  | Master        | PR               |
  | ----------- | ----------- |
  | ![Screen Shot 2021-05-14 at 11 46 33 PM](https://user-images.githubusercontent.com/23396902/118347462-8f689780-b511-11eb-8335-329f7d2a9992.png) | ![Screen Shot 2021-05-14 at 11 45 41 PM](https://user-images.githubusercontent.com/23396902/118347463-92638800-b511-11eb-9044-073f51ef27ff.png) |

ACKs for top commit:
  hebasto:
    ACK 2f23ad2c40

Tree-SHA512: 21f8b1610e4820c9064bbd08608b5467e5b9499e2a3b149ff223e37b60e7d560497255c733eafa5434628a84b9f7b7c91d8b0f34b02be2f9ceb3ab21a4d555a8
2021-07-01 09:14:02 +03:00
Hennadii Stepanov
34f554dbd3 Merge bitcoin-core/gui#366: Dark Mode fixes/portability
9d5bf6bf01 GUI: Always call parent changeEvent handler (Luke Dashjr)
c901d4d8ce GUI: Enable palette change adaptation on all platforms (Luke Dashjr)

Pull request description:

  The changes to support macOS "Dark Mode" are valid for any platform, and should work so long as Qt implements the PaletteChange event. (Worst case, we're no worse off with trying.)

  Additionally, we shouldn't block the parent classes from implementing event handlers. Who knows what side effects that could have.

ACKs for top commit:
  hebasto:
    ACK 9d5bf6bf01, tested on Linux Mint 20.1 (Qt 5.12.8) with the [`qt5ct`](https://packages.ubuntu.com/focal/qt5ct) package installed.
  kristapsk:
    ACK 9d5bf6bf01. Tested on Gentoo Linux with Xfce4 and Qt 5.15.2, does not break anything on my computer.

Tree-SHA512: dce2fff0ff129eda208132390a37424ff9607539287dbdbfdfd659ed9c4ea0472541e987489a04fd935e391dc006a35bfc9cfa9bcff33602b7dbd29b81c51626
2021-07-01 09:02:00 +03:00
Andrew Chow
b945a31afa wallet: erase spkmans rather than setting to nullptr
In many places in ScriptPubKeyMan managing code, we assume that the
ScriptPubKeyMan being retrieved actually exists and is not a nullptr.
Thus removing a ScriptPubKeyMan requires erasing the object from the
map rather than setting it to a nullptr.
2021-07-01 01:22:38 -04:00
fanquake
5a95c5179c Merge bitcoin/bitcoin#20191: wallet, refactor: make DescriptorScriptPubKeyMan agnostic of internal flag
181181019c refactor: remove m_internal from DescriptorSPKman (S3RK)

Pull request description:

  Rationale: improve consistency between `CWallet` and `DescriptorScriptPubKeyMan`; simplify `ScriptPubKeyMan` interface.

  Descriptor in itself is neither internal or external. It's responsibility of a wallet to assign and manage descriptors for a specific purpose. Duplicating information about internalness of a descriptor could lead to inconsistencies and unexpected behaviour (for example misreporting keypool size).

ACKs for top commit:
  instagibbs:
    reACK 181181019c
  achow101:
    reACK 181181019c

Tree-SHA512: d5613b7f6795b290bfa0fd8cb0536de1714d0cf72cba402266bd06d550758ebad690b54fc0a336a1c7414b5814aa4a37c90a6ae89926474a97d30956d7e034ff
2021-07-01 10:16:33 +08:00
fanquake
045bb06ebd Merge bitcoin/bitcoin#19651: wallet: importdescriptors update existing
3efaf83c75 wallet: deactivate descriptor (S3RK)
6737d9655b test: wallet importdescriptors update existing (S3RK)
586f1d53d6 wallet: maintain SPK consistency on internal flag change (S3RK)
f1b7db1474 wallet: don't mute exceptions in importdescriptors (S3RK)
bf68ebc1cd wallet: allow to import same descriptor twice (S3RK)

Pull request description:

  Rationale: allow updating existing descriptors with `importdescriptors` command.

  Currently if you run same `importdescriptors` command twice with a descriptor containing private key you will get very confusing error — `Missing required fields`. What happens is that Wallet tries to write imported private key to the disk, but it exists already so we get `DB_KEYEXIST (-30995)` from BerkelyDB. Please note, that we set `DB_NOOVERWRITE` (I guess not to lose some keys accidentally). The exception is caught in `catch (...)` in rpcdump.cpp with a generic error.

  With this PR if a descriptor is already present than we will update its activeness, internalness, label, range and next_index.
  For the range only expansion is allowed (range start can only decrease, range end increase).

ACKs for top commit:
  achow101:
    re-ACK 3efaf83c75
  meshcollider:
    Code review ACK 3efaf83c75
  jonatack:
    Light ACK 3efaf83c75 per `git range-diff a000cb0 5d96704 3efaf83` and as a sanity check, re-debug-built on debian with gcc 10.2.1 and clang 11, ran wallet_importdescriptors.py

Tree-SHA512: 122c4b621d64ec8a3b625f3aed9f01a2b5cbaf2029ad0325b5ff38d67fff5cd35324335fabe2dd5169548b01b267c81be6ae0f5c834342f3d5f6eeed515c4843
2021-07-01 10:06:56 +08:00
Antoine Riard
c4ddee64c7 test: Add test for replacement relay fee check 2021-06-30 18:47:08 -04:00
Samuel Dobson
722776c0fd Merge bitcoin/bitcoin#21329: descriptor wallet: Cache last hardened xpub and use in normalized descriptors
e6cf0ed92d wallet, rpc: listdescriptors does not need unlocked (Andrew Chow)
3280704886 Pass in DescriptorCache to ToNormalizedString (Andrew Chow)
7a26ff10c2 Change DescriptorImpl::ToStringHelper to use an enum (Andrew Chow)
75530c93a8 Remove priv option for ToNormalizedString (Andrew Chow)
74fede3b8b wallet: Upgrade existing descriptor caches (Andrew Chow)
432ba9e543 wallet: Store last hardened xpub cache (Andrew Chow)
d87b544b83 descriptors: Cache last hardened xpub (Andrew Chow)
cacc391098 Move DescriptorCache writing to WalletBatch (Andrew Chow)
0b4c8ef75c Refactor Cache merging and writing (Andrew Chow)
976b53b085 Revert "Cache parent xpub inside of BIP32PubkeyProvider" (Andrew Chow)

Pull request description:

  Currently fetching a normalized descriptor requires the wallet to be unlocked as it needs the private keys to derive the last hardened xpub. This is not very user friendly as normalized descriptors shouldn't require and don't involve the private keys except for derivation. We solve this problem by caching the last hardened xpub (which has to be derived at some point when generating the address pool).

  However the last hardened xpub was not already being cached. We only cached the immediate parent xpub and derived child keys. For example, with a descriptor derivation path of `/84'/0'/0'/0/*`, the parent xpub that is cached is `m/84'/0'/0'/0`, and the child keys of `m/84'/0'/0'/0/i` (note that child keys would not be cached in this case). This parent xpub is not suitable for the normalized descriptor form as we want the key at `m/84'/0'/0'`. So this PR adds another field to `DescriptorCache` to cache the last hardened xpub so that we can use them for normalized descriptors.

  Since `DescriptorCache` is changing, existing descriptor wallets need to be upgraded to use this new cache. The upgrade will occur in the background either at loading time (if the wallet is not encrypted) or at unlocking time in the same manner that `UpgradeKeyMetadata` operates. It will use a new wallet flag `WALLET_FLAG_LAST_HARDENED_XPUB_CACHED` to indicate whether the descriptor wallet has the last hardened xpub cache.

  Lastly `listdescriptors` will not require the wallet to be locked and `getaddressinfo`'s `parent_desc` will always be output (assuming the upgrade has occurred).

ACKs for top commit:
  fjahr:
    tACK e6cf0ed92d
  S3RK:
    reACK e6cf0ed
  jonatack:
    Semi ACK e6cf0ed92d reviewed, debug-built and ran unit tests and some of the descriptor functional tests at each commit. I'm not very familiar with this code and it could be clearer to the uninitiated IMHO, so I'm not confident enough to give a full ACK. Various minor suggestions follow, most of them for readability, feel free to pick and choose.
  meshcollider:
    Code review + functional test run ACK e6cf0ed92d

Tree-SHA512: ac27aade8644525cd65bfcaf27ff32afb974085b1451faf4ff68c6671a690bd6a41d4f39a33cbf461ae0fbe85995c0a4c08dbd36171da1c1d2a1d00053ad298d
2021-07-01 09:58:40 +12:00
Jarol Rodriguez
2f23ad2c40 qt: allow prompt icon to be colorized 2021-06-30 13:27:31 -04:00
Kiminuo
30450a1bd5 Do not clone qa-assets git repository if not necessary 2021-06-30 11:59:49 +02:00
fanquake
3fc20abab0 Merge bitcoin/bitcoin#22353: ci: Upgrading pip version in macos environment
057750c09d ci: Upgrading pip version in macos environment (Tushar Singla)

Pull request description:

  During each CI run, in macos native environment, python packages lief and zmq are rebuilt everytime which wastes a lot of resources and time and fixes #22206. The latest version of pip directly fetches pre-built binaries. Through this commit pip version is upgraded in macos environment before installation of these packages.

ACKs for top commit:
  MarcoFalke:
    cr ACK 057750c09d

Tree-SHA512: e61d02e46c8fe6a89119014d025a26aba090f9507d725315680893290f5bbc20a375ef408c71fa8db2f485b44ec91cfa0c140198ca44a9d3e0a57055b6bb9582
2021-06-30 16:23:19 +08:00
MarcoFalke
e1a13f12e1 Merge bitcoin/bitcoin#18096: doc: IsFinalTx comment about nSequence & OP_CLTV
f9e37f33ce doc: IsFinalTx comment about nSequence & OP_CLTV (Yuval Kogman)

Pull request description:

  It's somewhat surprising that a transaction's `nLockTime` field is ignored
  when all `nSequence` fields are final, so this change aims to clarify this
  behavior and cross reference relevant details of `OP_CHECKLOCKTIMEVERIFY`.

ACKs for top commit:
  MarcoFalke:
    ACK f9e37f33ce

Tree-SHA512: 88460dacbe4b8115fb1948715f09b21d4f34ba1da9e88d52f0b774a969f845e9eddc5940e7fee66eacdd3062dc40d6d44c3f282b0e5144411fd47eb2320b44f5
2021-06-30 09:43:21 +02:00
S3RK
181181019c refactor: remove m_internal from DescriptorSPKman
Descriptor in itself is neither internal or external.
It's responsibility of a wallet to assign and manage descriptors
for a specific purpose. Duplicating such information could lead to
inconsistencies and unexpected behaviour.
2021-06-30 08:37:50 +02:00
fanquake
bfd910cae4 Merge bitcoin/bitcoin#18568: doc: Clarify developer notes about constant naming
05f9770c1f doc: Clarify developer notes about constant naming (Russell Yanofsky)

Pull request description:

  I'm pretty sure developer notes were intended to say constants should be upper case and variables should be lower case, but right now they are ambiguous about whether to write:

  ```c++
  extern const int SYMBOL;
  ```

  or:

  ```c++
  extern const int g_symbol;
  ```

  First convention above is better than the second convention because it tells you without having to look anything up that the value of `SYMBOL` won't change at runtime. Also I haven't seen other c++ projects using the second convention.

ACKs for top commit:
  MarcoFalke:
    cr ACK 05f9770c1f
  practicalswift:
    ACK 05f9770c1f
  jarolrod:
    ACK 05f9770c1f 🥃

Tree-SHA512: 766d0e25d9db818d45df4ad6386987014f2053584cbced4b755ceef8bda6b7e2cfeb34eb8516423bd03b140faaf577614d5e3be2799f7eed0eb439187ab85323
2021-06-30 09:23:15 +08:00
Anthony Towns
e48826ad87 tests: remove ComputeBlockVersion shortcut from versionbits tests 2021-06-30 08:19:12 +10:00
Anthony Towns
c5f36725e8 [refactor] Move ComputeBlockVersion into VersionBitsCache
This also changes ComputeBlockVersion to take the versionbits cache
mutex once, rather than once for each versionbits deployment.
2021-06-30 08:19:12 +10:00
Anthony Towns
4a69b4dbe0 [move-only] Move ComputeBlockVersion from validation to versionbits 2021-06-30 08:19:12 +10:00
Anthony Towns
0cfd6c6a8f [refactor] versionbits: make VersionBitsCache a full class
Moves the VersionBits* functions to be methods of the cache class,
and makes the cache and its lock private to the class.
2021-06-30 08:19:12 +10:00
Anthony Towns
8ee3e0bed5 [refactor] rpc/blockchain.cpp: SoftForkPushBack
Rename BIP9SoftForkPushBack and BuriedSoftForkPushBack to SoftForkPushBack
and have the compiler figure out which one to use based on the deployment
type. Avoids the need to update the file when burying a deployment.
2021-06-30 08:19:12 +10:00
Anthony Towns
92f48f360d deploymentinfo: Add DeploymentName() 2021-06-30 08:19:12 +10:00
Anthony Towns
ea68b3a572 [move-only] Rename versionbitsinfo to deploymentinfo 2021-06-30 08:19:12 +10:00
Anthony Towns
c64b2c6a0f scripted-diff: rename versionbitscache
-BEGIN VERIFY SCRIPT-
sed -i -e 's/versionbitscache/g_versionbitscache/g' $(git grep -l versionbitscache)
-END VERIFY SCRIPT-
2021-06-30 08:19:12 +10:00
Anthony Towns
de55304f6e [refactor] Add versionbits deployments to deploymentstatus.h
Adds support for versionbits deployments to DeploymentEnabled,
DeploymentActiveAfter and DeploymentActiveAt. Also moves versionbitscache
from validation to deploymentstatus.
2021-06-30 08:18:58 +10:00
Anthony Towns
2b0d291da8 [refactor] Add deploymentstatus.h
Provides DeploymentEnabled, DeploymentActiveAt, and DeploymentActiveAfter
helpers for checking the status of buried deployments. Can be overloaded
so the same syntax works for non-buried deployments, allowing future
soft forks to be changed from signalled to buried deployments without
having to touch the implementation code.

Replaces IsWitnessEnabled and IsScriptWitnessEnabled.
2021-06-29 17:11:12 +10:00
Anthony Towns
eccd736f3d versionbits: Use dedicated lock instead of cs_main 2021-06-29 17:11:12 +10:00
Anthony Towns
36a4ba0aaa versionbits: correct doxygen comments 2021-06-29 17:11:12 +10:00
MarcoFalke
a000cb013c Merge bitcoin/bitcoin#22361: refactor: Make httpserver work queue a unique_ptr
fa92e60f38 refactor: Make httpserver work queue a unique_ptr (MarcoFalke)

Pull request description:

  This simplifies the code a bit because `if (p) { delete p; p = nullptr; }` can be replaced by a call to the `reset()` member.

ACKs for top commit:
  promag:
    Core review ACK fa92e60f38.
  jonatack:
    ACK fa92e60f38 code review, debug build clean, ran test/functional/interface*.py tests locally as a sanity check
  hebasto:
    ACK fa92e60f38, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 6b122162317dd4ad6889341745c7ac1903a3ee510f6548f46dc356308442a6eff13eb8dc604c38ba18783e7a66d2b836d641a8594ff980a010c12c97f3856684
2021-06-29 08:24:26 +02:00
S3RK
6084d2caed wallet: do not spam about non-existent spk managers 2021-06-29 08:16:39 +02:00
fanquake
e0face9235 Merge bitcoin/bitcoin#22358: Remove unused wallet pointer from wallet signals
8888cf45f5 Remove unused wallet pointer from NotifyAddressBookChanged (MarcoFalke)
faf3640303 Remove unused wallet pointer from NotifyTransactionChanged signal (MarcoFalke)

Pull request description:

  The signals are members of the wallet, so passing the pointer would be redundant even if it was used.

  Also, fix `with` -> `without`, which was forgotten in commit ca4cf5cff6.

ACKs for top commit:
  jonatack:
    Code review ACK 8888cf45f5 also verified with/without lock cs_wallet status for each of the two functions and debian clang 11 debug build clean
  promag:
    Code review ACK 8888cf45f5.
  theStack:
    Code review ACK 8888cf45f5

Tree-SHA512: e3b80931ce9bcb05213619f5435ac7c21d3c7848643950a70db610902bd1803c92bb75e501d46b0e519bc576901f160e088e8882c4f1adce892a80df565f897b
2021-06-29 12:41:06 +08:00
fanquake
8071ec179d Merge bitcoin/bitcoin#21789: refactor: Remove ::Params() global from CChainState
fa0d9211ef refactor: Remove chainparams arg from CChainState member functions (MarcoFalke)
fa38947125 refactor: Remove ::Params() global from inside CChainState member functions (MarcoFalke)

Pull request description:

  The `::Params()` global is verbose and confusing. Also it makes tests a bit harder to write because they'd have to mock a global.

  Fix all issues by simply using a member variable that points to the right params.

  (Can be reviewed with `--word-diff-regex=.`)

ACKs for top commit:
  jnewbery:
    ACK fa0d9211ef
  kiminuo:
    utACK fa0d9211
  theStack:
    ACK fa0d9211ef 🍉

Tree-SHA512: 44676b19c9ed471ccb536331d3029bad192d7d50f394fd7b8527ec431452aeec8c4494164b9cf8e16e0123c4463b16be864366c6b599370032c17262625a0356
2021-06-29 11:22:57 +08:00
S3RK
3efaf83c75 wallet: deactivate descriptor 2021-06-28 21:44:50 +02:00
S3RK
6737d9655b test: wallet importdescriptors update existing 2021-06-28 21:44:50 +02:00
S3RK
586f1d53d6 wallet: maintain SPK consistency on internal flag change 2021-06-28 21:44:50 +02:00
S3RK
f1b7db1474 wallet: don't mute exceptions in importdescriptors 2021-06-28 21:37:44 +02:00
S3RK
bf68ebc1cd wallet: allow to import same descriptor twice 2021-06-28 21:37:37 +02:00
Sebastian Falbesoner
b57b633b94 test: use script_util helpers for creating P2PKH scripts 2021-06-28 20:14:07 +02:00
Sebastian Falbesoner
61b6a017a9 test: wallet util: fix multisig P2SH-P2WSH script creation 2021-06-28 20:14:01 +02:00
Hennadii Stepanov
986bf78d7e qt: Emit dataChanged signal to dynamically re-sort Peers table 2021-06-28 17:38:01 +03:00
MarcoFalke
3f56ef7bef Merge bitcoin/bitcoin#22146: Reject invalid coin height and output index when loading assumeutxo
fa9ebedec3 Reject invalid coin height and output index when loading assumeutxo (MarcoFalke)

Pull request description:

  It should be impossible to have a coin at a height higher than the height of the snapshot block, so reject those early to avoid integer wraparounds and hash collisions later on.

  Same for the outpoint index.

  Both issues were found by fuzzing:

  * The height issue by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34793
  * The outpoint issue by my fuzz server: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34793#c2

ACKs for top commit:
  practicalswift:
    cr ACK fa9ebedec3: patch looks correct
  jamesob:
    crACK fa9ebedec3
  theStack:
    Code review ACK fa9ebedec3
  benthecarman:
    crACK fa9ebedec3

Tree-SHA512: dae7caee4b3862b23ebdf2acb7edec4baf75b0dbf1409b370b1a73aa6b632b317ebfac596dcbaf4edfb1301b513f45465ea75328962460f35e2af0d7e547c9ac
2021-06-28 16:11:44 +02:00
MarcoFalke
fa92e60f38 refactor: Make httpserver work queue a unique_ptr 2021-06-28 11:47:05 +02:00
MarcoFalke
fa051d3e5e doc: Remove unused section from release process 2021-06-28 11:17:24 +02:00
Hennadii Stepanov
cd46c11577 qt: Draw "eye" sign at the beginning of watch-only addresses 2021-06-28 12:09:42 +03:00
Hennadii Stepanov
9ea1da6fc9 qt: Do not extend recent transaction width to address/label string 2021-06-28 12:05:15 +03:00
MarcoFalke
8888cf45f5 Remove unused wallet pointer from NotifyAddressBookChanged 2021-06-28 10:39:49 +02:00
MarcoFalke
faf3640303 Remove unused wallet pointer from NotifyTransactionChanged signal 2021-06-28 10:38:32 +02:00
Tushar Singla
057750c09d ci: Upgrading pip version in macos environment
During each CI run, for macos native environment, python packages lief
and zmq are rebuilt everytime which wastes a lot of resources and time.
The latest version of pip directly fetches pre-built binaries. Through
this commit pip version is upgraded in macos environment before
installation of these packages.
2021-06-28 11:57:19 +05:30
MarcoFalke
8cdf91735f Merge bitcoin/bitcoin#22349: doc: repository IRC updates
dd49f1b52e doc: doc/README.md IRC updates (Jon Atack)

Pull request description:

  Follow-up to https://github.com/bitcoin/bitcoin/pull/22109:

  - update remaining link and reference from Freenode to Libera
  - remove defunct #bitcoin-dev channel
  - other minor fixups

ACKs for top commit:
  fanquake:
    ACK dd49f1b52e

Tree-SHA512: de20a8f3889028def7139be1cd0fa15daa3351ca987a16090d0caa6b4cd397e370400e2b3033ec776e270aa3a1ab085ada20fbdcc063ac88e2ad3b8351a596e7
2021-06-28 08:00:47 +02:00
fanquake
ac238f76bf Merge bitcoin/bitcoin#22339: doc: Document minimum required libc++ version
18c2027c76 doc: Document minimum required libc++ version (Hennadii Stepanov)

Pull request description:

  The minimum required libc++ version to compile the current master is 7.0.

  See #22324.

ACKs for top commit:
  fanquake:
    ACK 18c2027c76

Tree-SHA512: a2811236a600caea5b9fd156993a0195fc3f1c231b48d0e369250ebb759f7153fd974390de388b91b367d9dc5e115922a2cab9d3b8ad872a2724c07b63b9fcf4
2021-06-28 11:06:02 +08:00
Hennadii Stepanov
18c2027c76 doc: Document minimum required libc++ version 2021-06-28 05:10:56 +03:00
Jon Atack
dd49f1b52e doc: doc/README.md IRC updates
- update remaining link and reference from freenode to libera
- remove no-longer-existing #bitcoin-dev IRC channel
- other minor fixups
2021-06-27 21:26:03 +02:00
MarcoFalke
9c3751a0c9 Merge bitcoin/bitcoin#22342: Avoid the use of P0083R3 std::set::merge
6cf4ea7187 Avoid the use of P0083R3 std::set::merge (Pieter Wuille)

Pull request description:

  This use was introduced in #21365, but as pointed out in #22339, this causes compatibility problems.

  Just avoid its use for now.

ACKs for top commit:
  jonatack:
    re-ACK 6cf4ea7187
  benthecarman:
    ACK 6cf4ea7187
  hebasto:
    ACK 6cf4ea7187, successfully compiled on the following systems:

Tree-SHA512: 2b3fdcadb7de98963ebb0b192bd956aa68526457fe5b374c74a69ea10d5b68902763148f11abbcc471010bcdc799e0804faef5f8e8ff8a509b3a053c0cb0ba39
2021-06-26 10:02:36 +02:00
Pieter Wuille
6cf4ea7187 Avoid the use of P0083R3 std::set::merge 2021-06-25 10:42:05 -07:00
MarcoFalke
3e306ee1d5 Merge bitcoin/bitcoin#19362: rpc/blockchain: Reset scantxoutset progress before inferring descriptors
8c4129b454 rpc: reset scantxoutset progress on finish (Pavol Rusnak)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/19361 by moving resetting the `g_scan_progress` variable **before** inferring the descriptors

ACKs for top commit:
  achow101:
    Code review ACK 8c4129b454

Tree-SHA512: 2b81e5c930b9c201a1b9cc742c37e0617582326acd9adbd6b14991d33c05d51b1d5ff6a26f3accab7d274dc8c6a48bc8cbccf0811b60c1f76dd805f9b31482c0
2021-06-25 11:21:33 +02:00
MarcoFalke
246daf1f53 Merge bitcoin/bitcoin#22322: fuzz: Check banman roundtrip
fa485d06ec fuzz: Check banman roundtrip (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    cr ACK fa485d06ec
  vasild:
    ACK fa485d06ec

Tree-SHA512: 84e297c0b90ef68d72afd2053bfda2888496c1b180233516a8caaf76d6c03403f1e4ed59f1eb32d799873fc34009634b4ce372244b9d546d04626af41ac4d1d7
2021-06-25 10:07:58 +02:00
Dmitry Goncharov
fb7be92b09 Mark print-% target as phony.
.PHONY does not take patterns (such as print-%) as prerequisites.
Have print-% depend on FORCE and mark FORCE as phony.

$ # on master
$ make print-host
host=x86_64-pc-linux-gnu
$ touch print-host
$ make print-host
make: 'print-host' is up to date.
$
$ git co mark_print_as_phony
Switched to branch 'mark_print_as_phony'
$ make print-host
host=x86_64-pc-linux-gnu
$ touch FORCE
$ make print-host
host=x86_64-pc-linux-gnu
2021-06-24 23:08:26 -04:00
Andrew Chow
e6cf0ed92d wallet, rpc: listdescriptors does not need unlocked
With the last hardened xpub cache, we don't neeed to have the wallet be
unlocked for listdescriptors.
2021-06-24 14:08:46 -04:00
Andrew Chow
3280704886 Pass in DescriptorCache to ToNormalizedString
Use the descriptor xpub cache in ToNormalizedString so that the wallet
does not need to be unlocked in order to get the normalized descriptor.
2021-06-24 14:08:45 -04:00
Andrew Chow
7a26ff10c2 Change DescriptorImpl::ToStringHelper to use an enum
Instead of having multiple, possibly conflicting, bools controlling the
flow of ToStringHelper, use an enum.
2021-06-24 14:08:30 -04:00
Andrew Chow
75530c93a8 Remove priv option for ToNormalizedString 2021-06-24 13:49:22 -04:00
Andrew Chow
74fede3b8b wallet: Upgrade existing descriptor caches
Add functions to upgrade existing descriptor caches to support the use
of last hardened xpub caching.
2021-06-24 13:49:22 -04:00
Andrew Chow
432ba9e543 wallet: Store last hardened xpub cache 2021-06-24 13:49:22 -04:00
Andrew Chow
d87b544b83 descriptors: Cache last hardened xpub
Cache the last hardenex xpub in the DescriptorCache
2021-06-24 13:49:22 -04:00
Andrew Chow
cacc391098 Move DescriptorCache writing to WalletBatch
Instead of adhoc writing of the items in DescriptorCache, move it all
into WalletBatch.
2021-06-24 13:49:22 -04:00
Andrew Chow
0b4c8ef75c Refactor Cache merging and writing
Instead of having a large blob of cache merging code in TopUp, refactor
this into DescriptorCache so that it can merge and provide a diff
(another DescriptorCache containing just the items that were added).
Then TopUp can just write everything that was in the diff.
2021-06-24 13:49:22 -04:00
Andrew Chow
976b53b085 Revert "Cache parent xpub inside of BIP32PubkeyProvider"
This reverts commit 09e25071f4.

The changes made in this commit have turned out to be unnecessary and
confusing, so it is being reverted.
2021-06-24 13:49:15 -04:00
MarcoFalke
b2f5c38333 Merge bitcoin/bitcoin#22327: cli: Avoid truncating -rpcwaittimeout
fa34cb8024 cli: Avoid truncating -rpcwaittimeout (MarcoFalke)

Pull request description:

  `seconds` is not enough precision to "exactly" store a timestamp n seconds into the future. Improve the precision by using `microseconds`. Fixes #22325

  Also, use chrono literals.

ACKs for top commit:
  jonatack:
    ACK fa34cb8024 review, debug-built, tested
  theStack:
    Tested ACK fa34cb8024

Tree-SHA512: 7158da8545f9998a82bcc8636e04564efdb1e1be43b4288298c151b4df29ad47a2760259eefadd4a01db92ea18a1e017f3febc1cd8c69a4b28c86180229d8c90
2021-06-24 16:01:21 +02:00
MarcoFalke
fa485d06ec fuzz: Check banman roundtrip 2021-06-24 15:57:34 +02:00
Sebastian Falbesoner
e65d1d4986 doc: recommend --disable-external-signer in OpenBSD build guide 2021-06-24 14:58:42 +02:00
W. J. van der Laan
0553d75268 Merge bitcoin/bitcoin#22154: Add OutputType::BECH32M and related wallet support for fetching bech32m addresses
754f134a50 wallet: Add error message to GetReservedDestination (Andrew Chow)
87a0e7a3b7 Disallow bech32m addresses for legacy wallet things (Andrew Chow)
6dbe4d1072 Use BECH32M for tr() desc, WitV1Taproot, and WitUnknown CTxDests (Andrew Chow)
699dfcd8ad Opportunistically use bech32m change addresses if available (Andrew Chow)
0262536c34 Add OutputType::BECH32M (Andrew Chow)
177c15d2f7 Limit LegacyScriptPubKeyMan address types (Andrew Chow)

Pull request description:

  Currently bech32m addresses are classfied as bech32. Because bech32m is incompatible with bech32, we need to define a new `OutputType` for it so that it can be handled correctly. This PR adds `OutputType::BECH32M`, updates all of the relevant `OutputType` classifications, and handle requests for bech32m addresses. There is now a `bech32m` address type string that can be used.

  * `tr()` descriptors now report their output type as `OutputType::BECH32M`. `WtinessV1Taproot` and `WitnessUnknown` are also classified as `OutputType::BECH32M`.
  * Bech32m addresses are completely disabled for legacy wallets. They cannot be imported (explicitly disallowed in `importaddress` and `importmulti`), will not be created when getting all destinations for a pubkey, and will not be added with `addmultisigaddress`. Additional protections have been added to `LegacyScriptPubKeyMan` to disallow attempting to retrieve bech32m addresses.
  * Since Taproot multisigs are not implemented yet, `createmultisig` will also disallow the bech32m address type.
  * As Taproot is not yet active, `DescriptorScriptPubKeyMan` cannot and will not create a `tr()` descriptor. Protections have been added to make sure this cannot occur.
  * The change address type detection algorithm has been updated to return `bech32m` when there is a segwit v1+ output script and the wallet has a bech32m `ScriptPubKeyMan`, falling back to bech32 if one is not available.

ACKs for top commit:
  laanwj:
    re-review ACK 754f134a50
  Sjors:
    re-utACK 754f134: only change is switching to `bech32m` in two `wallet_taproot.py` test cases.
  fjahr:
    re-ACK 754f134a50
  jonatack:
    ACK 754f134a50

Tree-SHA512: 6ea90867d3631d0d438e2b08ce6ed930f37d01323224661e8e38f183ea5ee2ab65b5891394a3612c7382a1aff907b457616c6725665a10c320174017b998ca9f
2021-06-24 14:20:28 +02:00
W. J. van der Laan
b7565c708d Merge bitcoin/bitcoin#22250: doc: add basic I2P documentation
78cdab1ad2 doc: sort entries in doc/README.md#Miscellaneous alphabetically (Vasil Dimov)
cad487bcfa doc: add basic I2P documentation (Vasil Dimov)

Pull request description:

  Add basic I2P documentation to help users to start using I2P and Bitcoin Core.

ACKs for top commit:
  jonatack:
    ACK 78cdab1ad2
  laanwj:
    ACK 78cdab1ad2

Tree-SHA512: 71cffc9419338a21cb9e811ee9463d6d09eef1a2a73190f766523bc1642a1f58984d5b2ab13da9b6560387ac9f431aace8bc7ae9b776585ecd3e3c1ff6af8e8d
2021-06-24 13:38:39 +02:00
MarcoFalke
e59ea87954 Merge bitcoin/bitcoin#22311: test: Add missing syncwithvalidationinterfacequeue in p2p_blockfilters
fadddd13ee test: Add missing syncwithvalidationinterfacequeue (MarcoFalke)
faa211fc6e test: Misc cleanup (MarcoFalke)
fa1668bf50 test: Run pep-8 (MarcoFalke)
facd97ae0f scripted-diff: Renames (MarcoFalke)

Pull request description:

  The index on the block filters is running in the background on the validation interface. To avoid intermittent test failures, it needs to be synced.

  Also other cleanups.

ACKs for top commit:
  lsilva01:
    Tested ACK fadddd13ee on Ubuntu 20.04

Tree-SHA512: d858405db426a2f9d5620059dd88bcead4e3fba3ccc6bd8023f624b768fbcfa2203246fb0b2db620490321730d990f0e78063b21a26988c969cb126d4bd697bd
2021-06-24 12:54:00 +02:00
MarcoFalke
d6a59166a1 Merge bitcoin/bitcoin#22257: test: refactor: various (de)serialization helpers cleanups/improvements
bdb8b9a347 test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative) (Sebastian Falbesoner)
1914054208 scripted-diff: test: rename `FromHex` to `from_hex` (Sebastian Falbesoner)
a79396fe5f test: remove `ToHex` helper, use .serialize().hex() instead (Sebastian Falbesoner)
2ce7b47958 test: introduce `tx_from_hex` helper for tx deserialization (Sebastian Falbesoner)

Pull request description:

  There are still many functional tests that perform conversions from a hex-string to a message object (deserialization) manually. This PR identifies all those instances and replaces them with a newly introduced helper `tx_from_hex`.

  Instances were found via
  * `git grep "deserialize.*BytesIO"`

  and some of them manually, when it were not one-liners.

  Further, the helper `ToHex` was removed and simply replaced by `.serialize().hex()`, since now both variants are in use (sometimes even within the same test) and using the helper doesn't really have an advantage in readability. (see discussion https://github.com/bitcoin/bitcoin/pull/22257#discussion_r652404782)

ACKs for top commit:
  MarcoFalke:
    review re-ACK bdb8b9a347 😁

Tree-SHA512: e25d7dc85918de1d6755a5cea65471b07a743204c20ad1c2f71ff07ef48cc1b9ad3fe5f515c1efaba2b2e3d89384e7980380c5d81895f9826e2046808cd3266e
2021-06-24 12:47:13 +02:00
fanquake
c31161f4f7 Merge bitcoin/bitcoin#22320: build: set minimum required Boost to 1.64.0
957f358427 build: remove check for Boost Process header (fanquake)
df2c933217 build: remove workaround for Boost and std::atomic (fanquake)
2bf211696b build: set minimum required Boost to 1.64.0 (fanquake)

Pull request description:

  Setting a newer minimum required Boost means we can remove the awkward header / compile check for Boost Process.

  If we don't do this, we should at-least make Boost Process being missing no longer a failure, otherwise anyone building using Boost < 1.64.0 would have to pass `--disable-external-signer` as well.

  The only system I can see that is affected here, (doesn't have new enough system packages) is Debian Oldstable. However, anyone compiling there, can use depends. They can also no-longer use the system GCC (6.0), and I'd assume would be using Clang 7, which would be the newest compiler available to them. It's extended, LTS support also end in 1 year from now, so anyone still using it should be considering upgrading.

  Debian Buster (Stable) has 1.67+, Ubuntu Bionic has 1.65+, any of the BSDs, recent Fedora, macOS etc all also have well and truly new enough Boost versions available.

  I think this is something we should just do for 22.0. If not, definitely for 23.0.

  Fixes #22319. Compiling Bitcoin Core should work, as `windows.h` will be included.
  Alternative to #22294.
  Would also close #22269.
  #19128 could be re-opened.

ACKs for top commit:
  laanwj:
    Tested ACK 957f358427 that this fixes #22269.
  MarcoFalke:
    review ACK 957f358427

Tree-SHA512: a8ffa7933dce8bf994892ef16664103d7b4e1008e52628e9becb918a7727232dfb51b23100a82dc2b60cd9af5877abc32dc2d3754a7e1b3ac5410a92fdf393f3
2021-06-24 18:35:25 +08:00
MarcoFalke
bfa885898a Merge bitcoin/bitcoin#22306: [test] Improvements to p2p_addr_relay.py
6168eb06b2 [test] Prevent intermittent issue (Amiti Uttarwar)
1d8193e2a2 [test] Remove GetAddrStore class (Amiti Uttarwar)
ef2f149bf2 [test] Update GetAddrStore callers to use AddrReceiver (Amiti Uttarwar)
e8c67ea19a [test] Add functionality to AddrReceiver (Amiti Uttarwar)
09dc073cff [test] Allow AddrReceiver to be used more generally (Amiti Uttarwar)

Pull request description:

  A test refactor broken out from #21528 & a fix to #22243.

  This PR:
  1. consolidates the two helper classes into one, with the intent of making the test logic more clear & usable as we add more subtests to the file

  2. hopefully fixes the test flakiness by bumping up the mocktime interval to ensure `m_next_addr_send` timer triggers

ACKs for top commit:
  mzumsande:
    Code-Review ACK 6168eb06b2
  lsilva01:
    Tested ACK 6168eb06b2 on Ubuntu 20.04
  brunoerg:
    tACK 6168eb06b2

Tree-SHA512: 248324f9d37e0e5ffe4acc437cd72ad9a2960abc868a97c6040a36e6ea8b59029127ac4f63fcf67d981a5bb4dbf2334bb2c23c541fae8e910d5523884bcedcba
2021-06-24 12:10:44 +02:00
Jon Atack
d8513fe411 doc: update doc/benchmarking.md 2021-06-24 11:15:29 +02:00
Jon Atack
84e2d5b781 bench: bench_bitcoin.cpp help fixups
- remove unneeded strprintf
- consistent punctuation (no EOL periods)
- sort helps by order they are printed (alphabetical order)
2021-06-24 11:13:12 +02:00
Jon Atack
10f4ce2078 bench: bench.h fixes and improvements 2021-06-24 11:13:10 +02:00
MarcoFalke
a196c89317 Merge bitcoin/bitcoin#22270: test: Add bitcoin-util tests (+refactors)
fa4017e7a0 refactor: Pass grind args vector as const reference (MarcoFalke)
fa08bc288f Remove gArgs from AppInitUtil (MarcoFalke)
fa751a47ff Remove unused OptionsCategory arg from AddCommand (MarcoFalke)
fa3c1eee7f Remove unused includes from bitcoin-util (MarcoFalke)
fa304929e2 test: Add bitcoin-util tests (MarcoFalke)
fa831e709a build_msvc: Add bitcoin-util.exe (MarcoFalke)

Pull request description:

  bitcoin-util has no tests

  See https://marcofalke.github.io/btc_cov/total.coverage/src/bitcoin-util.cpp.gcov.html (Coverage report showing 0%)

ACKs for top commit:
  klementtan:
    Code review and tested ACK fa4017e7a0
  theStack:
    Tested ACK fa4017e7a0
  jamesob:
    reACK fa4017e7a0 ([`jamesob/ackr/22270.1.MarcoFalke.test_add_bitcoin_util_te`](https://github.com/jamesob/bitcoin/tree/ackr/22270.1.MarcoFalke.test_add_bitcoin_util_te))

Tree-SHA512: 68e2791239bc48d28fbb6394155c39ea0357a96ec7e4896ca579feeef1a803657165a0ef9fa3cf6e2a381e5b0ca0dafa1b594158303a04997db784201d8dd66d
2021-06-24 09:42:07 +02:00
MarcoFalke
7317e14a44 Merge bitcoin/bitcoin#22263: refactor: wrap CCoinsViewCursor in unique_ptr
7ad414f4bf doc: add comment about CCoinsViewDBCursor constructor (James O'Beirne)
0f8a5a4dd5 move-only(ish): don't expose CCoinsViewDBCursor (James O'Beirne)
615c1adfb0 refactor: wrap CCoinsViewCursor in unique_ptr (James O'Beirne)

Pull request description:

  I tripped over this one for a few hours at the beginning of the week, so I've sort of got a personal vendetta against `CCoinsView::Cursor()` returning a raw pointer.

  Specifically in the case of CCoinsViewDB, if a raw cursor is allocated and not freed, a cryptic leveldb assertion failure occurs on CCoinsViewDB destruction (`Assertion 'dummy_versions_.next_ == &dummy_versions_' failed.`).

  This is a pretty simple change.

  Related to: https://github.com/bitcoin/bitcoin/issues/21766
  See also: https://github.com/google/leveldb/issues/142#issuecomment-414418135

ACKs for top commit:
  MarcoFalke:
    review ACK 7ad414f4bf 🔎
  jonatack:
    re-ACK 7ad414f4bf modulo suggestion
  ryanofsky:
    Code review ACK 7ad414f4bf. Two new commits look good and thanks for clarifying constructor comment

Tree-SHA512: 6471d03e2de674d84b1ea0d31e25f433d52aa1aa4996f7b4aab1bd02b6bc340b15e64cc8ea07bbefefa3b5da35384ca5400cc230434e787c30931b8574c672f9
2021-06-23 18:32:35 +02:00
MarcoFalke
c0e30933e0 Merge bitcoin/bitcoin#22323: doc: spelling update
184d4534f6 script, doc: spelling update (Jon Atack)

Pull request description:

  Clears out the report from `test/lint/lint-spelling.sh` and touches up the leftover nits in https://github.com/bitcoin/bitcoin/pull/22166#pullrequestreview-690454669. Happy to add any others people are aware of.

ACKs for top commit:
  MarcoFalke:
    cr ACK 184d4534f6
  Sjors:
    utACK 184d453

Tree-SHA512: 3b0ef6bd5ff227363b0bda79eeb66763151c74f607bc3a2a7bfe7823e3eef196587bccfe639e714e8e27b918ba57e8317eda06f225143c32c736685087dbcd24
2021-06-23 14:46:37 +02:00
MarcoFalke
fa34cb8024 cli: Avoid truncating -rpcwaittimeout 2021-06-23 14:40:55 +02:00
Jon Atack
184d4534f6 script, doc: spelling update 2021-06-23 13:33:18 +02:00
Samuel Dobson
567670bec5 Merge bitcoin/bitcoin#22166: Add support for inferring tr() descriptors
d637a9b397 Taproot descriptor inference (Pieter Wuille)
c7388e5ada Report address as solvable based on inferred descriptor (Pieter Wuille)
29e5dd1a5b consensus refactor: extract ComputeTapleafHash, ComputeTaprootMerkleRoot (Pieter Wuille)

Pull request description:

  Includes:
  * First commit from #21365, adding TaprootSpendData in SigningProvider
  * A refactor to expose ComputeTapleafHash and ComputeTaprootMerkleRoot from script/interpreter
  * A tiny change to make `getaddressinfo` report tr() descriptors as solvable (so that inferred descriptors are shown), despite not having signing code for them.
  * Logic to infer the script tree back from TaprootSpendData, and then use that to infer descriptors.

ACKs for top commit:
  achow101:
    re-ACK d637a9b397
  Sjors:
    re-utACK d637a9b
  meshcollider:
    Code review ACK d637a9b397

Tree-SHA512: 5ab9b95da662382d8549004be4a1297a577d7caca6b068f875c7c9343723931d03fa9cbf133de11f83b74e4851490ce820fb80413c77b9e8495a5f812e505d86
2021-06-23 22:42:30 +12:00
Vasil Dimov
78cdab1ad2 doc: sort entries in doc/README.md#Miscellaneous alphabetically 2021-06-23 12:21:39 +02:00
Vasil Dimov
cad487bcfa doc: add basic I2P documentation 2021-06-23 12:21:38 +02:00
MarcoFalke
d6e0d78c31 Merge bitcoin/bitcoin#20966: banman: save the banlist in a JSON format on disk
bb719a08db style: remove () from assert in rpc_setban.py (Vasil Dimov)
24b10ebda3 doc: fix grammar in doc/files.md (Vasil Dimov)
dd4e957dcd test: ensure banlist can be read from disk after restart (Vasil Dimov)
d197977ae2 banman: save the banlist in a JSON format on disk (Vasil Dimov)

Pull request description:

  Save the banlist in `banlist.json` instead of `banlist.dat`.

  This makes it possible to store Tor v3 entries in the banlist on disk
  (and any other addresses that cannot be serialized in addrv1 format).

  Only read `banlist.dat` if it exists and `banlist.json` does not exist (first start after an upgrade).

  Supersedes https://github.com/bitcoin/bitcoin/pull/20904
  Resolves https://github.com/bitcoin/bitcoin/issues/19748

ACKs for top commit:
  jonatack:
    Code review re-ACK bb719a08db per `git range-diff 6a67366 4b52c72 bb719a0`
  achow101:
    Code Review ACK bb719a08db

Tree-SHA512: fc135c3a1fe20bcf5d008ce6bea251b4135e56c78bf8f750b4bd8144c095b81ffe165133cdc7e4715875eec7e7c4e13ad9f5d2450b21102af063d7c8abf716b6
2021-06-23 10:01:56 +02:00
fanquake
957f358427 build: remove check for Boost Process header
Now that we require Boost 1.64.0+, Boost Process will be available.
2021-06-23 15:48:42 +08:00
fanquake
df2c933217 build: remove workaround for Boost and std::atomic 2021-06-23 15:48:42 +08:00
fanquake
2bf211696b build: set minimum required Boost to 1.64.0 2021-06-23 15:48:37 +08:00
MarcoFalke
03aa59a4e5 Merge bitcoin/bitcoin#22313: test: Add missing sync_all to feature_coinstatsindex
fafd9165e9 test: Add missing sync_all to feature_coinstatsindex (MarcoFalke)

Pull request description:

  Sync the blocks before invalidating them to ensure all nodes are on the right tip. Otherwise nodes[0] might stay on the "stale" block and the test fails (intermittently)

ACKs for top commit:
  jamesob:
    crACK fafd9165e9

Tree-SHA512: ca567b97b839b56c91d52831eaac18d8c843d376be90c9fd8b49d2eb4a46b801a1d2402996d5dfe2bef3e2c9bd75d19ed443e3f42cc4679c5f20043ba556efc8
2021-06-23 08:07:19 +02:00
Andrew Chow
754f134a50 wallet: Add error message to GetReservedDestination
Adds an error output parameter to all GetReservedDestination functions
so that callers can get the actual reason that a change address could
not be fetched. This more closely matches GetNewDestination. This allows
for more granular error messages, such as one that indicates that
bech32m addresses cannot be generated yet.
2021-06-22 21:57:04 -04:00
Andrew Chow
87a0e7a3b7 Disallow bech32m addresses for legacy wallet things
We don't want the legacy wallet to ever have bech32m addresses so don't
allow importing them. This includes addmultisigaddress as that is a
legacy wallet only RPC

Additionally, bech32m multisigs are not available yet, so disallow them
in createmultisig.
2021-06-22 21:57:04 -04:00
Andrew Chow
6dbe4d1072 Use BECH32M for tr() desc, WitV1Taproot, and WitUnknown CTxDests
The tr() descriptor, WitnessV1Taproot CTxDestination, and
WitnessUnknown CTxDestination are OutputType::BECH32M so they should
report as such.
2021-06-22 21:57:04 -04:00
Andrew Chow
699dfcd8ad Opportunistically use bech32m change addresses if available
If a transaction as a segwit output, use a bech32m change address if
they are available. If not, fallback to bech32. If bech32 change
addresses are unavailable, fallback to the default address type.
2021-06-22 21:53:11 -04:00
Andrew Chow
0262536c34 Add OutputType::BECH32M
Bech32m addresses need their own OutputType

We are not ready to create DescriptorScriptPubKeyMans which produce
bech32m addresses. So don't allow generating them.
2021-06-22 21:53:11 -04:00
Andrew Chow
177c15d2f7 Limit LegacyScriptPubKeyMan address types
Make sure that LegacyScriptPubKeyMan can only be used for legacy,
p2sh-segwit, and bech32 address types.
2021-06-22 21:53:11 -04:00
MarcoFalke
c93e123dc7 Merge bitcoin/bitcoin#22308: wallet: Add missing BlockUntilSyncedToCurrentChain
fa27baa9c8 Revert "test: Add temporary logging to debug #20975" (MarcoFalke)
fadb55085a wallet: Add missing BlockUntilSyncedToCurrentChain (MarcoFalke)

Pull request description:

  Fixes #20975

  Also replace the wallet pointer by a reference

ACKs for top commit:
  achow101:
    ACK fa27baa9c8

Tree-SHA512: 79047a30998104a12c2ff84a8e3cc5207151410bbe92b74cfedbe1c1aca3ffa5909391607fc597f3a3cf0725fa827528a4c57edaeacc8360536b1965e166be6a
2021-06-22 23:08:23 +02:00
Luke Dashjr
9d5bf6bf01 GUI: Always call parent changeEvent handler 2021-06-22 19:24:44 +00:00
Luke Dashjr
c901d4d8ce GUI: Enable palette change adaptation on all platforms 2021-06-22 19:24:42 +00:00
MarcoFalke
fafd9165e9 test: Add missing sync_all to feature_coinstatsindex 2021-06-22 20:10:33 +02:00
MarcoFalke
fadddd13ee test: Add missing syncwithvalidationinterfacequeue 2021-06-22 19:42:51 +02:00
MarcoFalke
faa211fc6e test: Misc cleanup
* Replace wait_until with assert_equal where possible
* Use send_and_ping helper where possible
2021-06-22 19:42:35 +02:00
MarcoFalke
fa1668bf50 test: Run pep-8
Can be reviewed with --word-diff-regex=.
2021-06-22 19:42:25 +02:00
MarcoFalke
facd97ae0f scripted-diff: Renames
-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s/\<$1\>/$2/g" test/functional/p2p_blockfilters.py ; }

 # Rename from "node" to "peer" to avoid confusion with self.nodes
 ren node0 peer_0
 ren node1 peer_1

 # Remove the confusing "C" prefix
 ren CFiltersClient FiltersClient

-END VERIFY SCRIPT-
2021-06-22 19:41:28 +02:00
MarcoFalke
327e2691f6 Merge bitcoin/bitcoin#22309: blockstorage: Add missing atomic include
fa2d21fec8 add missing atomic include (MarcoFalke)

Pull request description:

  `std::atomic` is used in the file, so to avoid compile issues, add the missing include.

ACKs for top commit:
  practicalswift:
    cr ACK fa2d21fec8
  jamesob:
    crACK fa2d21fec8
  hebasto:
    ACK fa2d21fec8, I have reviewed the code and it looks OK, I agree it can be merged.
  prayank23:
    crACK fa2d21fec8

Tree-SHA512: 307b15abd62006be4457b2437fb65de517c296bf0417e8acd181904eb6056dba5655dd5bc43b834bf68a087d06637f5e99ba5a6bc8be3e12388cea470dc155d0
2021-06-22 15:52:16 +02:00
MarcoFalke
fa2d21fec8 add missing atomic include 2021-06-22 11:06:30 +02:00
MarcoFalke
fa27baa9c8 Revert "test: Add temporary logging to debug #20975"
This reverts commit faa94961d6.
2021-06-22 10:04:06 +02:00
MarcoFalke
fadb55085a wallet: Add missing BlockUntilSyncedToCurrentChain 2021-06-22 10:01:50 +02:00
MarcoFalke
672870ab7b Merge bitcoin/bitcoin#22201: test: Fix TestShell to allow running in Jupyter Notebook
168b6c317c add dummy file param to fix jupyter (Josiah Baker)

Pull request description:

  this fixes argparse to use `parse_known_args`. previously, if an unknown argument was passed, argparse would fail with an `unrecognized arguments: %s` error.

  ## why
  the documentation mentions being able to run `TestShell` in a REPL interpreter or a jupyter notebook. when i tried to run inside a jupyter notebook, i got the following error:

  ![image](https://user-images.githubusercontent.com/7444140/121382910-57554880-c947-11eb-94f2-49da8679528c.png)

  this was due to the notebook passing the filename of the notebook as an argument. this is a known problem with notebooks and argparse, documented here: https://stackoverflow.com/questions/48796169/how-to-fix-ipykernel-launcher-py-error-unrecognized-arguments-in-jupyter

  ## testing
  to test, make sure you have jupyter notebooks installed. you can do this by running:
  ```
  pip install notebook
  ```
  or following instructions from [here](https://jupyterlab.readthedocs.io/en/stable/getting_started/installation.html).

  once installed, start a notebook (`jupyter notebook`), launch a python3 kernel and run the following snippet:

  ```python
  import sys

  # make sure this is the path for your system
  sys.path.insert(0, "/path/to/bitcoin/test/functional")
  from test_framework.test_shell import TestShell

  test = TestShell().setup(num_nodes=2, setup_clean_chain=True)
  ```

  you should see the following output, without errors:
  ![image](https://user-images.githubusercontent.com/7444140/121383301-a307f200-c947-11eb-83b6-6c50b2cada25.png)

  if you are unfamiliar with notebooks, here is a short guide on using them: https://jupyter.readthedocs.io/en/latest/running.html

ACKs for top commit:
  MarcoFalke:
    review ACK 168b6c317c
  jamesob:
    crACK 168b6c317c
  practicalswift:
    cr ACK 168b6c317c

Tree-SHA512: 4fee1563bf64a1cf9009934182412446cde03badf2f19553b78ad2cb3ceb0e5e085a5db41ed440473494ac047f04641311ecbba3948761c6553d0ca4b54937b4
2021-06-22 08:11:16 +02:00
Amiti Uttarwar
6168eb06b2 [test] Prevent intermittent issue
Since m_next_addr_send is on a Poisson distribution, increase the mocktime bump
to ensure we don't experience flakiness in the tests. Closes #22243.
2021-06-21 18:05:54 -07:00
Amiti Uttarwar
1d8193e2a2 [test] Remove GetAddrStore class 2021-06-21 18:05:48 -07:00
Amiti Uttarwar
ef2f149bf2 [test] Update GetAddrStore callers to use AddrReceiver 2021-06-21 18:05:43 -07:00
Amiti Uttarwar
e8c67ea19a [test] Add functionality to AddrReceiver
Add two simple helper functions to `AddrReceiver` to support callers currently
using `GetAddrStore` [used in next commit].
2021-06-21 18:05:27 -07:00
Amiti Uttarwar
09dc073cff [test] Allow AddrReceiver to be used more generally
The `on_addr` functionality of `AddrReceiver` tests logic specific to how the
addr messages are set up in the test bodies. To allow other callers to also use
`AddrReceiver`, only apply the assertion logic if the caller indicates
desirability by setting `test_addr_contents` to true when initializing the
class.
2021-06-21 10:25:06 -07:00
MarcoFalke
398dd67833 Merge bitcoin/bitcoin#22296: doc: Final merge of release notes snippets, mv to wiki
fa09fd1a09 doc: Final merge of release notes snippets (MarcoFalke)

Pull request description:

  None of the remaining pulls tagged for 22.0 have snippets, so merge them and move them to the wiki.

  Trivial to review with `--color-moved=dimmed-zebra`.

ACKs for top commit:
  laanwj:
    ACK fa09fd1a09
  jonatack:
    ACK fa09fd1a09

Tree-SHA512: d5d272a9fc2217a87988efa189000cae5330da22f0459dd5e4340a18aba3d67d8fe8661016cb777e2e15f5d137c1c2b3de7d576c942eaf4201dabdc4e9c783f4
2021-06-21 17:55:00 +02:00
Josiah Baker
168b6c317c add dummy file param to fix jupyter
testshell in jupyter was failing due to an extra arg.
this adds a dummy -f param, which allows TestShell to
be used in a command line or jupyter environment
2021-06-21 17:12:16 +02:00
MarcoFalke
fa09fd1a09 doc: Final merge of release notes snippets 2021-06-21 16:31:06 +02:00
MarcoFalke
74013641e0 Merge bitcoin/bitcoin#22089: test: MiniWallet: fix fee calculation for P2PK and check tx vsize
d6d2ab9845 test: MiniWallet: fix fee calculation for P2PK and check tx vsize (Sebastian Falbesoner)
ce024b1c0e test: MiniWallet: force P2PK signature to have fixed size (71 bytes) (Sebastian Falbesoner)

Pull request description:

  This PR is a follow-up to #21945. It aims to both fix the fee calculation for P2PK mode transactions and enable its vsize check. Currently, the latter assumes a fixed tx length, which is fine for anyone-can-spend txs but doesn't apply to P2PK output spends due to varying DER signature size; the vsize check is therefore disabled for P2PK mode on master branch.

  Creating one million DER signatures with MiniWallet shows the following distribution of sizes (smart people with better math skills probably could deduce the ratios without trying, but hey):

  | DER signature size [bytes]  | #occurences (ratio) |
  | ------------- | ------------- |
  | 71  | 498893 (49.89%) |
  | 70 | 497244 (49.72%) |
  | 69 | 3837 (0.38%) |
  | 68 | 22 (0.0022%) |

  Note that even smaller signatures are possible (for smaller R and S values with leading zero bytes), it's just that the probability decreases exponentially.     Instead of choosing a large vsize check range and hoping that smaller signatures are never created (potentially leading to flaky tests), the proposed solution is ~~to limit the signature size to the two most common sizes 71 and 70 (>99.6% probability) and then accordingly only check for two vsize values; the value to be used for fee calculation is a decimal right between the two possible sizes (167.5 vbytes) and for the vsize check it's rounded down/up integer values are used.~~ to simply grind the signature to a fixed size of 71 bytes (49.89% probability, i.e. on average each call to `sign_tx()`, on average two ECC signing operations are needed).

  ~~The idea of grinding signatures to a fixed size (similar to https://github.com/bitcoin/bitcoin/pull/13666 which grinds to low-R values) would be counter-productive, as the signature creation in the test suite is quite expensive and this would significantly slow down tests that calculate hundreds of signatures (like e.g. feature_csv_activation.py).~~

  For more about transaction sizes on different input/output types, see the following interesting article: https://medium.com/coinmonks/on-bitcoin-transaction-sizes-97e31bc9d816

ACKs for top commit:
  MarcoFalke:
    Concept ACK d6d2ab9845

Tree-SHA512: 011c70ee0e4adf9ba12902e4b6c411db9ae96bdd8bc810bf1d67713367998e28ea328394503371fc1f5087a819547ddaea56c073b28db893ae1c0031d7927f32
2021-06-21 16:11:13 +02:00
W. J. van der Laan
6556da77d7 Merge bitcoin/bitcoin#21056: rpc: Add a -rpcwaittimeout parameter to limit time spent waiting
b9e76f1bf0 rpc: Add test for -rpcwaittimeout (Christian Decker)
f76cb10d7d rpc: Prefix rpcwaittimeout error with details on its nature (Christian Decker)
c490e17ef6 doc: Add release notes for the `-rpcwaittimeout` cli parameter (Christian Decker)
a7fcc8eb59 rpc: Add a `-rpcwaittimeout` parameter to limit time spent waiting (Christian Decker)

Pull request description:

  Adds a new numeric `-rpcwaittimeout` that can be used to limit the
  time we spend waiting on the RPC server to appear. This is used by
  downstream projects to provide a bit of slack when `bitcoind`s RPC
  interface is not available right away.

  This makes the `-rpcwait` argument more useful, since we can now limit
  how long we'll ultimately wait, before potentially giving up and reporting
  an error to the caller. It was discussed in the context of the BTCPayServer
  wanting to have c-lightning wait for the RPC interface to become available
  but still have the option of giving up eventually ([4355]).

  I checked with laanwj whether this is already possible ([comment]), and
  whether this would be a welcome change. Initially I intended to repurpose
  the (optional) argument to `-rpcwait`, however I decided against it since it
  would potentially break existing configurations, using things like `rpcwait=1`,
  or `rpcwait=true` (the former would have an unintended short timeout, when
  old behavior was to wait indefinitely).

  ~Due to its simplicity I didn't implement a test for it yet, but if that's desired I
  can provide one.~ Test was added during reviews.

  [4355]: https://github.com/ElementsProject/lightning/issues/4355
  [comment]: https://github.com/ElementsProject/lightning/issues/4355#issuecomment-768288261

ACKs for top commit:
  laanwj:
    Code review ACK b9e76f1bf0
  promag:
    ACK b9e76f1bf0.

Tree-SHA512: 3cd6728038ec7ca7c35c2e7ccb213bfbe963f99a49bb48bbc1e511c4dd23d9957c04f9af1f8ec57120e47b26eaf580b46817b099d5fc5083c98da7aa92db8638
2021-06-21 15:54:56 +02:00
Vasil Dimov
bb719a08db style: remove () from assert in rpc_setban.py 2021-06-21 15:29:28 +02:00
Vasil Dimov
24b10ebda3 doc: fix grammar in doc/files.md 2021-06-21 15:27:57 +02:00
Vasil Dimov
dd4e957dcd test: ensure banlist can be read from disk after restart
With `banlist.dat` (being written in addrv1 format) if we would try to
write a Tor v3 subnet, it would serialize as a dummy-all-0s IPv6
address and subsequently, when deserialized will not result in the same
subnet.

This problem does not exist with `banlist.json` where the data is saved
in textual, human-readable form.
2021-06-21 15:27:03 +02:00
Vasil Dimov
d197977ae2 banman: save the banlist in a JSON format on disk
Save the banlist in `banlist.json` instead of `banlist.dat`.

This makes it possible to store Tor v3 entries in the banlist on disk
(and any other addresses that cannot be serialized in addrv1 format).

Only read `banlist.dat` if it exists and `banlist.json` does not
exist (first start after an upgrade).

Supersedes https://github.com/bitcoin/bitcoin/pull/20904
Resolves https://github.com/bitcoin/bitcoin/issues/19748
2021-06-21 14:39:44 +02:00
Sebastian Falbesoner
bdb8b9a347 test: doc: improve doc for from_hex helper (mention to_hex alternative) 2021-06-21 14:36:02 +02:00
Sebastian Falbesoner
1914054208 scripted-diff: test: rename FromHex to from_hex
-BEGIN VERIFY SCRIPT-
sed -i 's/\<FromHex\>/from_hex/g' $(git grep -l FromHex)
-END VERIFY SCRIPT-

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2021-06-21 14:33:59 +02:00
Sebastian Falbesoner
a79396fe5f test: remove ToHex helper, use .serialize().hex() instead 2021-06-21 14:30:03 +02:00
Sebastian Falbesoner
2ce7b47958 test: introduce tx_from_hex helper for tx deserialization
`FromHex` is mostly used for transactions, so we introduce a
shortcut `tx_from_hex` for `FromHex(CTransaction, hex_str)`.
2021-06-21 14:28:05 +02:00
W. J. van der Laan
6a67366fdc Merge bitcoin/bitcoin#19033: http: Release work queue after event base finish
4e353cb618 http: Release work queue after event base finish (João Barbosa)

Pull request description:

  This fixes a race between `http_request_cb` and `StopHTTPServer` where
  the work queue is used after release.

  Fixes #18856.

ACKs for top commit:
  fjahr:
    Code review ACK 4e353cb618
  achow101:
    ACK 4e353cb618
  LarryRuane:
    ACK 4e353cb618
  hebasto:
    ACK 4e353cb618, tested (rebased on top of master 9313c4e6aa) on Linux Mint 20.1 (x86_64) using MarcoFalke's [patch](https://github.com/bitcoin/bitcoin/pull/19033#issuecomment-640106647), including different `-rpcthreads`/`-rpcworkqueue` cases. The bug is fixed. The code is correct.

Tree-SHA512: 185d2a9744d0d5134d782bf321ac9958ba17b11a5b3d70b4897c8243e6b146dfd3f23c57aef8e10ae9484374120b64389c1949a9cf0a21dccc47ffc934c20930
2021-06-21 11:15:42 +02:00
W. J. van der Laan
f6a25bea82 Merge bitcoin/bitcoin#22147: p2p: Protect last outbound HB compact block peer
30aee2dfe6 tests: Add test for compact block HB selection (Pieter Wuille)
6efbcec4de Protect last outbound HB compact block peer (Suhas Daftuar)

Pull request description:

  If all our high-bandwidth compact block serving peers (BIP 152) stall block
  download, then we can be denied a block for (potentially) a long time. As
  inbound connections are much more likely to be adversarial than outbound
  connections, mitigate this risk by never removing our last outbound HB peer if
  it would be replaced by an inbound.

ACKs for top commit:
  achow101:
    ACK 30aee2dfe6
  ariard:
    Code ACK 30aee2dfe
  jonatack:
    ACK 30aee2dfe6

Tree-SHA512: 5c6c9326e3667b97e0864c371ae2174d2be9054dad479f4366127b9cd3ac60ffa01ec9707b16ef29cac122db6916cf56fd9985733390017134ace483278921d5
2021-06-21 08:18:55 +02:00
W. J. van der Laan
a305a687e7 Merge bitcoin/bitcoin#22244: devtools: Correctly extract symbol versions in symbol-check
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py (W. J. van der Laan)
a33381acf5 devtools: Add xkb version to symbol-check (W. J. van der Laan)
19e598bab0 devtools: Fix verneed section parsing in pixie (W. J. van der Laan)

Pull request description:

  I misunderstood the ELF specification for version symbols (verneed): The `vn_aux` pointer is relative to the main verneed record, not the start of the section.

  This caused many symbols to not be versioned properly in the return value of `elf.dyn_symbols`. This was discovered in #21454.

  Fix it by correcting the offset computation.

  - xkb versions symbols (using the prefix `V`), as this library is used by bitcoin-qt, add it to the valid versions in `symbol-check.py`

  This unfortunately brings to light some symbols that have been introduced since and weren't caught (from a gitian compile of master):

  ```
  bitcoin-cli: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-cli: failed IMPORTED_SYMBOLS
  bitcoind: symbol getrandom from unsupported version GLIBC_2.25
  bitcoind: symbol log from unsupported version GLIBC_2.29
  bitcoind: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoind: symbol pow from unsupported version GLIBC_2.29
  bitcoind: symbol exp from unsupported version GLIBC_2.29
  bitcoind: failed IMPORTED_SYMBOLS
  bitcoin-qt: symbol exp from unsupported version GLIBC_2.29
  bitcoin-qt: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol log from unsupported version GLIBC_2.29
  bitcoin-qt: symbol pow from unsupported version GLIBC_2.29
  bitcoin-qt: symbol statx from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getrandom from unsupported version GLIBC_2.25
  bitcoin-qt: symbol renameat2 from unsupported version GLIBC_2.28
  bitcoin-qt: symbol getentropy from unsupported version GLIBC_2.25
  bitcoin-qt: failed IMPORTED_SYMBOLS
  bitcoin-wallet: symbol exp from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol log from unsupported version GLIBC_2.29
  bitcoin-wallet: symbol fcntl64 from unsupported version GLIBC_2.28
  bitcoin-wallet: failed IMPORTED_SYMBOLS
  test_bitcoin: symbol getrandom from unsupported version GLIBC_2.25
  test_bitcoin: symbol log from unsupported version GLIBC_2.29
  test_bitcoin: symbol fcntl64 from unsupported version GLIBC_2.28
  test_bitcoin: symbol pow from unsupported version GLIBC_2.29
  test_bitcoin: symbol exp from unsupported version GLIBC_2.29
  test_bitcoin: failed IMPORTED_SYMBOLS
  ```

ACKs for top commit:
  hebasto:
    ACK e8cd3700ee

Tree-SHA512: 8c15e3478eb642f01a1ddaadef03f80583f088f9fa8e3bf171ce16b0ec05ffb4675ec147d7ffc6a4360637ed47fca517c6ca2bac7bb30d794c03783cfb964b79
2021-06-21 07:58:12 +02:00
MarcoFalke
965e937434 Merge bitcoin/bitcoin#22279: fuzz: add missing ECCVerifyHandle to base_encode_decode
906d791311 fuzz: add missing ECCVerifyHandle to base_encode_decode (Andrew Poelstra)

Pull request description:

  It is possible to trigger a fuzztest failure in the `base_encode_decode` by asking it to decode any PSBT that has HD keypaths in it. For example, this one

  ```
  cHNidP8BAFUCAAAAASeaIyOl37UfxF8iD6WLD8E+HjNCeSqF1+Ns1jM7XLw5AAAAAAD/////AaBa6gsAAAAAGXapFP/pwAYQl8w7Y28ssEYPpPxCfStFiKwAAAAAAAEBIJVe6gsAAAAAF6kUY0UgD2jRieGtwN8cTRbqjxTA2+uHIgIDsTQcy6doO2r08SOM1ul+cWfVafrEfx5I1HVBhENVvUZGMEMCIAQktY7/qqaU4VWepck7v9SokGQiQFXN8HC2dxRpRC0HAh9cjrD+plFtYLisszrWTt5g6Hhb+zqpS5m9+GFR25qaAQEEIgAgdx/RitRZZm3Unz1WTj28QvTIR3TjYK2haBao7UiNVoEBBUdSIQOxNBzLp2g7avTxI4zW6X5xZ9Vp+sR/HkjUdUGEQ1W9RiED3lXR4drIBeP4pYwfv5uUwC89uq/hJ/78pJlfJvggg71SriIGA7E0HMunaDtq9PEjjNbpfnFn1Wn6xH8eSNR1QYRDVb1GELSmumcAAACAAAAAgAQAAIAiBgPeVdHh2sgF4/iljB+/m5TALz26r+En/vykmV8m+CCDvRC0prpnAAAAgAAAAIAFAACAAAA=
  ```

  which I took straight from the PSBT test vectors. The reason is that in src/psbt.h we call `DeserializeHDKeypaths`, which in turn calls `CPubKey::IsFullyValid`, which in turn asserts that a secp context has been created.

  The error appears to be masked on many systems by the definition of `instance_of_eccryptoclosure` in src/script/bitcoinconsensus.cpp, which defines a static object which contains an `ECCVerifyHandle`. If you just comment out that line you can reliably trigger the fuzz test failure, e.g. by creating a file `crash` with the above PSBT, and runnnig

  ```
  ASAN_OPTIONS=symbolize=0:detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1 UBSAN_OPTIONS=suppressions=./test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1 FUZZ=base_encode_decode ./src/test/fuzz/fuzz -seed_inputs=crash
  ```

ACKs for top commit:
  practicalswift:
    cr ACK 906d791311

Tree-SHA512: b98b60573c21efe28503fe351883c6f0d9ac99d0dd6f100537b16ac53476617b8a3f899faf0c23d893d34a01b3bbe4a784499ec6f9c7000292e850bed449bd85
2021-06-19 09:40:10 +02:00
MarcoFalke
e172ea8804 Merge bitcoin/bitcoin#22210: test: Use MiniWallet in test_no_inherited_signaling RBF test
fa7d71f270 test: Run pep-8 on touched test (MarcoFalke)
fab7e99c2a test: Use MiniWallet in test_no_inherited_signaling RBF test (MarcoFalke)
fab871f649 test: Remove unused generate() from test (MarcoFalke)
faff3f35b7 test: Add txin.sequence option to MiniWallet (MarcoFalke)

Pull request description:

  This comes with nice benefits:
  * Less code and complexity
  * Test can be run without wallet compiled in

  Also add some additional checks for `getmempoolentry` (#22209) and other cleanups 🎨

ACKs for top commit:
  mjdietzx:
    Tested ACK fa7d71f270 thanks for the explanations, nicely done
  theStack:
    ACK fa7d71f270 🍷

Tree-SHA512: 0e9b8fe985779d8d7034d256deed627125bb374b6ae2972c461b3a220739a51061c6147ad69339bee16282f82716c7f3f8a7a89c693ceb1e47ea50709272332a
2021-06-19 08:47:52 +02:00
Andrew Poelstra
906d791311 fuzz: add missing ECCVerifyHandle to base_encode_decode 2021-06-18 23:13:07 +00:00
Pieter Wuille
d637a9b397 Taproot descriptor inference 2021-06-18 11:28:47 -07:00
Pieter Wuille
c7388e5ada Report address as solvable based on inferred descriptor 2021-06-18 11:23:17 -07:00
Pieter Wuille
29e5dd1a5b consensus refactor: extract ComputeTapleafHash, ComputeTaprootMerkleRoot 2021-06-18 11:23:17 -07:00
James O'Beirne
7ad414f4bf doc: add comment about CCoinsViewDBCursor constructor 2021-06-18 14:15:39 -04:00
James O'Beirne
0f8a5a4dd5 move-only(ish): don't expose CCoinsViewDBCursor
No need for this to be a part of the header anymore.

Includes a small reference type style change.
2021-06-18 14:14:15 -04:00
MarcoFalke
fa4017e7a0 refactor: Pass grind args vector as const reference 2021-06-18 20:10:07 +02:00
MarcoFalke
fa08bc288f Remove gArgs from AppInitUtil
Also fix incorrect {}
2021-06-18 20:09:41 +02:00
MarcoFalke
fa751a47ff Remove unused OptionsCategory arg from AddCommand 2021-06-18 20:09:23 +02:00
MarcoFalke
fa3c1eee7f Remove unused includes from bitcoin-util 2021-06-18 20:08:52 +02:00
MarcoFalke
fa304929e2 test: Add bitcoin-util tests 2021-06-18 20:08:45 +02:00
MarcoFalke
fa831e709a build_msvc: Add bitcoin-util.exe 2021-06-18 20:08:06 +02:00
W. J. van der Laan
0f47e01d7d Merge bitcoin/bitcoin#20923: signet miner followups
b3c712cb28 contrib/signet/miner: remove debug code (Anthony Towns)
297e35159f bitcoin-util: use AddCommand / GetCommand (Anthony Towns)
b6d493fd4d contrib/signet/README.md: Update miner description (Anthony Towns)
e66543827c contrib/signet/miner: Automatic timestamp for first block (Anthony Towns)
a383ce5b4a contrib/signet/miner: --grind-cmd is required for calibrate (Anthony Towns)
1a45cd2e51 contrib/signet: Fix typos (Anthony Towns)

Pull request description:

  Followups from #19937

ACKs for top commit:
  laanwj:
    Code review ACK b3c712cb28

Tree-SHA512: a1003f9ee3697438114b60872b50f4300c8b52f0d58551566eb61c421d787525807ae75be205dcab2c24358cd568f53260120880109a9d728773405ff987596f
2021-06-18 19:31:38 +02:00
MarcoFalke
da1e6d5911 Merge bitcoin/bitcoin#14604: tests: Add test and refactor feature_block.py
55311197c4 Added new test for future blocks reacceptance (sanket1729)
511a5af462 Fixed inconsistencies between code and comments (sanket1729)

Pull request description:

  This Commit does 3 things:
  1) Adds a test case for checking reacceptance a previously rejected block which
  was too far in the future.
  ~~2) clean up uses of rehash or calc_sha256 where it was not needed~~
  3) While constructing block 44, this commit makes the code consistent with the expected figure in
  the comment just above it by adding a transaction to the block.
  4) Fix comment describing `sign_tx()` function

ACKs for top commit:
  duncandean:
    reACK 5531119
  brunoerg:
    reACK 55311197c4

Tree-SHA512: d40c72fcdbb0b2a0715adc58441eeea08147ee2ec5e371a4ccc824ebfdc6450698bd40aaeecb7ea7bfdb3cd1b264dd821b890276fff8b8d89b7225cdd9d6b546
2021-06-18 18:08:49 +02:00
MarcoFalke
0844084c13 Merge bitcoin/bitcoin#22249: test: kill process group to avoid dangling processes when using --failfast
451b96f7d2 test: kill process group to avoid dangling processes (S3RK)

Pull request description:

  This is an alternative to #19281

  This PR fixes a problem when after test failure with `--failfast` option there could be dangling nodes. The nodes will continue to occupy rpc/p2p ports on the machine and will cause further test failures.

  If there are any dangling nodes left at the end of the test run we kill the whole process group.
  Pros: the operations is immediate and won't lead to CI timeout
  Cons: the test_runner process is also killed and exit code is 137

  Example output:
  ```
  ...
  Early exiting after test failure

  TEST                           | STATUS    | DURATION

  rpc_decodescript.py            | ✓ Passed  | 2 s
  rpc_deprecated.py              | ✓ Passed  | 2 s
  rpc_deriveaddresses.py         | ✓ Passed  | 2 s
  rpc_dumptxoutset.py            | ✖ Failed  | 2 s

  ALL                            | ✖ Failed  | 8 s (accumulated)
  Runtime: 4 s

  Killed: 9
  > echo $?
  137
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 451b96f7d2
  aitorjs:
    ACK 451b96f7d2. Manual testing with and without **--failfast**.

Tree-SHA512: 87e510a1411b9e7571e63cf7ffc8b9a8935daf9112ffc0f069d6c406ba87743ec439808181f7e13cb97bb200fad528589786c47f0b43cf3a2ef0d06a23cb86dd
2021-06-18 14:22:51 +02:00
W. J. van der Laan
e8cd3700ee devtools: Integrate ARCH_MIN_GLIBC_VER table into MAX_VERSIONS in symbol-check.py
The (ancient) versions specified here were deceptive. Entries older than
MAX_VERSIONS['GLIBC'], which is 2.17, are ignored here. So reorganize
the code to avoid confusion for other people reading this code.
2021-06-18 13:25:57 +02:00
fanquake
da69d9965a Merge bitcoin/bitcoin#21871: scripts: add checks for minimum required OS versions
aa80b5759d scripts: check macOS SDK version is set (fanquake)
c972345bac scripts: check minimum required Windows version is set (fanquake)
29615aef52 scripts: check minimum required macOS vesion is set (fanquake)
8732f7b6c9 scripts: LIEF 0.11.5 (fanquake)

Pull request description:

  macOS:
  We use a compile flag ([-mmacosx-version-min=10.14](https://github.com/bitcoin/bitcoin/blob/master/depends/hosts/darwin.mk#L96)) to set the minimum required version of macOS needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Clangs Darwin driver should infer the SDK version used during compilation, and forward that through to the linker. Add a check that this has been done, and the expected SDK version is set. Should help prevent issues like #21771 in future.

  Windows:
  We use linker flags ([-Wl,--major/minor-subsystem-version](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L683)) to set the minimum required version of Windows needed to run our binaries. This adds a sanity check that the version is being set as expected.

  Gitian builds:
  ```bash
  # macOS:
  8b6fcd61d75001c37b2af3fceb5ae09f5d2fe85e97d361f684214bd91c27954a  bitcoin-f015e1c2cac9-osx-unsigned.dmg
  3c1e412bc7f5a7a5d0f78e2cd84b7096831414e1304c1307211aa3e135d89bbf  bitcoin-f015e1c2cac9-osx-unsigned.tar.gz
  50b7b2804e8481f63c69c78e3e8a71c0d811bf2db8895dd6d3edae9c46a738ae  bitcoin-f015e1c2cac9-osx64.tar.gz
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  8a20f21b20673dfc8c23e22b20ae0839bcaf65bf0e02f62381cdf5e7922936f0  bitcoin-core-osx-22-res.yml

  # Windows:
  b01fcdc2a5673387050d6c6c4f96f1d350976a121155fde3f76c2af309111f9d  bitcoin-f015e1c2cac9-win-unsigned.tar.gz
  b95bdcbef638804030671d2332d58011f8c4ed4c1db87d6ffd211515c32c9d02  bitcoin-f015e1c2cac9-win64-debug.zip
  350bf180252d24a3d40f05e22398fec7bb00e06d812204eb5a421100a8e10638  bitcoin-f015e1c2cac9-win64-setup-unsigned.exe
  2730ddabe246d99913c9a779e97edcadb2d55309933d46f1dffd0d23ecf9aae5  bitcoin-f015e1c2cac9-win64.zip
  fe6b5c0a550096b76b6727efee30e85b60163a41c83f21868c849fdd9876b675  src/bitcoin-f015e1c2cac9.tar.gz
  aa60d7a753e8cb2d4323cfbbf4d964ad3645e74c918cccd66862888f8646d80f  bitcoin-core-win-22-res.yml
  ```

ACKs for top commit:
  hebasto:
    ACK aa80b5759d, tested by breaking tests:

Tree-SHA512: 10150219910e8131715fbfe20edaa15778387616ef3bfe1a5152c7acd3958fe8f88c74961c3d3641074eb72824680c22764bb1dc01a19e92e946c2d4962a8d2c
2021-06-18 15:21:47 +08:00
fanquake
ad0c8f356e Merge bitcoin/bitcoin#22238: build: improve detection of eBPF support
8f7704d032 build: improve detection of eBPF support (fanquake)

Pull request description:

  Just checking for the `sys/sdt.h` header isn't enough, as systems like macOS have the header, but it doesn't actually have the `DTRACE_PROBE*` probes, which leads to [compile failures](https://github.com/bitcoin/bitcoin/pull/22006#issuecomment-859559004). The contents of `sys/sdt.h` in the macOS SDK is:
  ```bash
  #ifndef _SYS_SDT_H
  #define _SYS_SDT_H

  /*
   * This is a wrapper header that wraps the mach visible sdt.h header so that
   * the header file ends up visible where software expects it to be.  We also
   * do the C/C++ symbol wrapping here, since Mach headers are technically C
   * interfaces.
   *
   * Note:  The process of adding USDT probes to code is slightly different
   * than documented in the "Solaris Dynamic Tracing Guide".
   * The DTRACE_PROBE*() macros are not supported on Mac OS X -- instead see
   * "BUILDING CODE CONTAINING USDT PROBES" in the dtrace(1) manpage
   *
   */
  #include <sys/cdefs.h>
  __BEGIN_DECLS
  #include <mach/sdt.h>
  __END_DECLS

  #endif  /* _SYS_SDT_H */
  ```

  The `BUILDING CODE CONTAINING USDT PROBES` section from the dtrace manpage is available [here](https://gist.github.com/fanquake/e56c9866d53b326646d04ab43a8df9e2), and outlines the more involved process of using USDT probes on macOS.

ACKs for top commit:
  jb55:
    utACK 8f7704d032
  practicalswift:
    cr ACK 8f7704d032
  hebasto:
    ACK 8f7704d032, tested on macOS Big Sur 11.4 (20F71) and on Linux Mint 20.1 (x86_64) with depends.

Tree-SHA512: 5f1351d0ac2e655fccb22a5454f415906404fdaa336fd89b54ef49ca50a442c44ab92d063cba3f161cb8ea0679c92ae3cd6cfbbcb19728cac21116247a017df5
2021-06-18 15:16:00 +08:00
Samuel Dobson
5c2e2afe99 Merge bitcoin/bitcoin#21365: Basic Taproot signing support for descriptor wallets
458a345b05 Add support for SIGHASH_DEFAULT in RPCs, and make it default (Pieter Wuille)
c0f0c8eccb tests: check spending of P2TR (Pieter Wuille)
a2380127e9 Basic Taproot signing logic in script/sign.cpp (Pieter Wuille)
49487bc3b6 Make GetInputUTXO safer: verify non-witness UTXO match (Pieter Wuille)
fd3f6890f3 Construct and use PrecomputedTransactionData in PSBT signing (Pieter Wuille)
5cb6502ac5 Construct and use PrecomputedTransactionData in SignTransaction (Pieter Wuille)
5d2e22437b Don't nuke witness data when signing fails (Pieter Wuille)
ce9353164b Permit full precomputation in PrecomputedTransactionData (Pieter Wuille)
e841fb503d Add precomputed txdata support to MutableTransactionSignatureCreator (Pieter Wuille)
a91d532338 Add CKey::SignSchnorr function for BIP 340/341 signing (Pieter Wuille)
e77a2839b5 Use HandleMissingData also in CheckSchnorrSignature (Pieter Wuille)
dbb0ce9fbf Add TaprootSpendData data structure, equivalent to script map for P2[W]SH (Pieter Wuille)

Pull request description:

  Builds on top of #22051, adding signing support after derivation support.

  Nothing is changed in descriptor features. Signing works for key path and script path spending, through the normal sending functions, and PSBT-based RPCs. However, PSBT usability is rather low as no extensions have been defined to convey Taproot-specific information, so all script information must be known to the signing wallet.

ACKs for top commit:
  achow101:
    re-ACK 458a345b05
  fjahr:
    Code review ACK 458a345b05
  Sjors:
    ACK 458a345b05

Tree-SHA512: 30ed212cf7754763a4a81624ebc084c51727b8322711ac0b390369213c1a891d367ed8b123882ac08c99595320c11ec57ee42304ff22a69afdc3d1a0d55cc711
2021-06-18 09:12:44 +12:00
Martin Zumsande
533500d907 p2p: Add timeout for AddrFetch peers
If AddrFetch peers don't send us addresses, disconnect them after
a while.
2021-06-17 22:00:07 +02:00
MarcoFalke
8cb43077b3 Merge bitcoin/bitcoin#22271: fuzz: Assert roundtrip equality for CPubKey
9550dffa0c fuzz: Assert roundtrip equality for `CPubKey` (Sebastian Falbesoner)

Pull request description:

  This PR is a (quite late) follow-up to #19237 (https://github.com/bitcoin/bitcoin/pull/19237#issuecomment-642203251). Looking at `CPubKey::Serialize` and `CPubKey::Unserialize` I can't think of a scenario where the roundtrip (serialization/deserialization) equality wouldn't hold.

ACKs for top commit:
  jamesob:
    crACK 9550dffa0c pending CI

Tree-SHA512: 640fb9e777d249769b22ee52c0b15a68ff0645b16c986e1c0bce9742155d14f1be601e591833e1dc8dcffebf271966c6b861b90888a44aae1feae2e0248e2c55
2021-06-17 21:40:51 +02:00
W. J. van der Laan
7b45c5e875 Merge bitcoin/bitcoin#20516: Well-defined CAddress disk serialization, and addrv2 anchors.dat
f8866e8c32 Add roundtrip fuzz tests for CAddress serialization (Pieter Wuille)
e2f0548b52 Use addrv2 serialization in anchors.dat (Pieter Wuille)
8cd8f37dfe Introduce well-defined CAddress disk serialization (Pieter Wuille)

Pull request description:

  Alternative to #20509.

  This makes the `CAddress` disk serialization format well defined, and uses it to enable addrv2 support in anchors.dat (in a way that's compatible with older software). The new format is:
  - The first 4 bytes store a format version number. Its low 19 bits are ignored (as those historically stored the `CLIENT_VERSION`), but its high 13 bits specify the actual serialization:
    - 0x00000000: LE64 encoding for `nServices`, V1 encoding for `CService` (like pre-BIP155 network serialization).
    - 0x20000000: CompactSize encoding for `nServices`, V2 encoding for `CService` (like BIP155 network serialization).
    - Any other value triggers an unsupported format error on deserialization, and can be used for future format changes.
  - The `ADDRV2_FORMAT` flag in the stream's version does not determine the actual serialization format; it only sets whether or not V2 encoding is permitted.

ACKs for top commit:
  achow101:
    ACK f8866e8c32
  laanwj:
    Code review ACK f8866e8c32
  vasild:
    ACK f8866e8c32
  jonatack:
    ACK f8866e8c32 tested rebased to master and built/run/restarted with DEBUG_ADDRMAN, peers.dat and anchors ser/deser seems fine
  hebasto:
    ACK f8866e8c32, tested on Linux Mint 20.1 (x86_64).

Tree-SHA512: 3898f8a8c51783a46dd0aae03fa10060521f5dd6e79315fe95ba807689e78f202388ffa28c40bf156c6f7b1fc2ce806b155dcbe56027df73d039a55331723796
2021-06-17 17:43:16 +02:00
Sebastian Falbesoner
9550dffa0c fuzz: Assert roundtrip equality for CPubKey 2021-06-17 17:03:03 +02:00
James O'Beirne
615c1adfb0 refactor: wrap CCoinsViewCursor in unique_ptr
Specifically with CCoinsViewDB, if a raw cursor is allocated and
not freed, a cryptic leveldb assertion failure occurs on
CCoinsViewDB destruction.

See: https://github.com/google/leveldb/issues/142#issuecomment-414418135
2021-06-17 09:47:08 -04:00
MarcoFalke
922abe8ca3 Merge bitcoin/bitcoin#22268: fuzz: Add temporary debug assert for oss-fuzz issue
faf1af58f8 fuzz: Add Temporary debug assert for oss-fuzz issue (MarcoFalke)

Pull request description:

  oss-fuzz is acting weird, so add an earlier assert to help troubleshooting

ACKs for top commit:
  practicalswift:
    cr ACK faf1af58f8

Tree-SHA512: 85830d7d47cf6b4edfe91a07bd5aa8f7110db0bade8df93868cf276ed04d5dd17e671f769e6a0fb5092012b86aa82bb411fb171411f15746981104ce634c88c1
2021-06-17 14:57:09 +02:00
MarcoFalke
6eafa81b32 Merge bitcoin/bitcoin#22267: fuzz: Speed up crypto fuzz target
fa483e9f68 fuzz: Speed up crypto fuzz target (MarcoFalke)

Pull request description:

  May fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34962

  Similar solution to https://github.com/bitcoin/bitcoin/pull/22005

ACKs for top commit:
  practicalswift:
    cr ACK fa483e9f68: patch looks correct and rationale makes sense

Tree-SHA512: 3788cf9f6ba0f7a0a217cd3a6a825839689425e99e4d6d657981d291a001b0da7c5abb50a68b4ee1c2a8300b87fb92e4e3ccc1171907792b40251e467c33bd53
2021-06-17 12:54:27 +02:00
MarcoFalke
faf1af58f8 fuzz: Add Temporary debug assert for oss-fuzz issue 2021-06-17 10:55:39 +02:00
MarcoFalke
fa483e9f68 fuzz: Speed up crypto fuzz target 2021-06-17 10:32:59 +02:00
MarcoFalke
dd24567a24 Merge bitcoin/bitcoin#22120: test: p2p_invalid_block: Check that a block rejected due to too-new tim…
754e802274 test: check rejected future block later accepted (Luke Dashjr)

Pull request description:

  (Luke) was unsure if the code sufficiently avoided caching a
  time-too-new rejection, so wrote this test to check it.  It looks like
  despite only exempting BLOCK_MUTATED, it is still okay because header
  failures never cache block invalidity.  This test will help ensure that
  if this ever changes, BLOCK_TIME_FUTURE gets excluded at the same time.

  This PR re-opens https://github.com/bitcoin/bitcoin/pull/17872 which went stale and addresses the nits raised by reviewers there.

ACKs for top commit:
  MarcoFalke:
    review ACK 754e802274

Tree-SHA512: a2bbc8fffb523cf2831e1ecb05f20868e30106a38cc2e369e4973fa549cca06675a668df16f76c49cc4ce3a22925404255e5c53c4232d63ba1b9fca878509aa0
2021-06-17 09:05:52 +02:00
fanquake
d50302625e Merge bitcoin/bitcoin#22182: guix: Overhaul how guix-{attest,verify} works and hierarchy
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected (Carl Dong)
4cc35daed5 Rewrite guix-{attest,verify} for new hier (Carl Dong)
28a9c9b839 Make SHA256SUMS fragment right after build (Carl Dong)

Pull request description:

  Based on:  #22075
  Code reviewers: I recommend reading the new `guix-{attest,verify}` files instead of trying to read the diff

  The following changes resolve many usability improvements which were pointed out to me:
  1. Some maintainers like to extract their "uncodesigned tarball" inside the `output/` directory, resulting in the older `guix-attest` mistakenly attesting to the extracted contents
  2. Maintainers whose GPG keys reside on an external smartcard often need to physically interact with the smartcard as a way to approve the signing operation, having one signature per platform means a lot of fidgeting
  3. Maintainers wishing to sign on a separate machine now has the option of transferring only a subtree of `output/`, namely `output/*/SHA256SUMS.part`, in order to perform a signature (you may need to specify an `$OUTDIR_BASE` env var)
  4. An `all.SHA256SUMS` file should be usable as the base `SHA256SUMS` in bitcoin core torrents and on the release server.

  For those who sign on an separate machine than the one you do builds on, the following steps will work:
  1. `env GUIX_SIGS_REPO=/home/achow101/guix.sigs SIGNER=achow101 NO_SIGN=1 ./contrib/guix/guix-attest`
  2. Copy `/home/achow101/guix.sigs/<tag>/achow101` (which does not yet have signatures) to signing machine
  3. Sign the `SHA256SUMS` files:
      ```bash
      for i in "<path-to-achow101>/*.SHA256SUMS"; do
          gpg --detach-sign --local-user "<your-key-here>" --armor --output "$i"{.asc,}
      done
      ```
  5. Upload `<path-to-achow101>` (now with signatures) to `guix.sigs`

  -----

  After this change, output directories will now include a `SHA256SUMS.part` fragment, created immediately after a successful build:
  ```
  output
  └── x86_64-w64-mingw32
      ├── bitcoin-4e069f7589da-win64-debug.zip
      ├── bitcoin-4e069f7589da-win64-setup-unsigned.exe
      ├── bitcoin-4e069f7589da-win64.zip
      ├── bitcoin-4e069f7589da-win-unsigned.tar.gz
      └── SHA256SUMS.part
  ```

  These `SHA256SUMS.part` fragments look something like:
  ```
  3ebd7262b1a0a5bb757fef1f70e7e14033c70f98c059bc4dbfee5d1992b25825  dist-archive/bitcoin-4e069f7589da.tar.gz
  def2e7d3de5ab3e3f955344e75151df4f33713f9101f5295bd13c9375bdf633b  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-debug.zip
  643049fe3ee4a4e83a1739607e67b11b7c9b1a66208a6f35a9ff634ba795500e  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64-setup-unsigned.exe
  a247a1ccec0ccc2e138c648284bd01f6a761f2d8d6d07d91b5b4a6670ec3f288  x86_64-w64-mingw32/bitcoin-4e069f7589da-win-unsigned.tar.gz
  fab76a836dcc592e39c04fd2396696633fb6eb56e39ecbf6c909bd173ed4280c  x86_64-w64-mingw32/bitcoin-4e069f7589da-win64.zip
  ```

  Meaning that they are valid `SHA256SUMS` files when `sha256sum --check`'d at the `guix-build-*/output` directory level

  When `guix-attest` is invoked, these `SHA256SUMS.part` files are combined and sorted (by `-k2`, `LC_ALL=C`) to create:

  1. `noncodesigned.SHA256SUMS` for a manifest of all non-codesigned outputs, and
  3. `all.SHA256SUMS` for a manifest of all outputs including non-codesigned outputs

  Then both files are signed, resulting in the following `guix.sigs` hierarchy:
  ```
  4e069f7589da/
  └── dongcarl
      ├── all.SHA256SUMS
      ├── all.SHA256SUMS.asc
      ├── noncodesigned.SHA256SUMS
      └── noncodesigned.SHA256SUMS.asc
  ```

ACKs for top commit:
  achow101:
    ACK e2c40a4ed5
  hebasto:
    ACK e2c40a4ed5, tested on Linux Mint 20.1 (x86_64) with and w/o `NO_SIGN=1`. Changes in `contrib/guix/libexec/codesign.sh` and `contrib/guix/guix-verify` are reviewed only.

Tree-SHA512: 618aacefb0eb6595735a9ab6a98ea6598fce65f9ccf33fa1e7ef93bf140c0f6cfc16e34870c6aa3e4777dd3f004b92a82a994141879870141742df948ec59c1f
2021-06-17 13:10:37 +08:00
fanquake
7c561bea52 Merge bitcoin/bitcoin#21935: Enable external signer support by default, reduce #ifdef
2f5bdcbc31 gui: misc external signer fixes and translation hints (Sjors Provoost)
d672404466 refactor: make ExternalSigner NetworkArg() and m_chain private (Sjors Provoost)
4455145e26 refactor: reduce #ifdef ENABLE_EXTERNAL_SIGNER usage (Sjors Provoost)
5be90c907e build: enable external signer by default (Sjors Provoost)
7d9453041b refactor: clean up external_signer.h includes (Sjors Provoost)
fc0eca31b3 fuzz: fix fuzz binary linking order (Sjors Provoost)

Pull request description:

  This follows the introduction of GUI support in https://github.com/bitcoin-core/gui/pull/4

  I don't think we should expect GUI users to self compile. This also enables external signer support by default for RPC users.

  In addition this PR reduces the number of `#ifdef ENABLE_EXTERNAL_SIGNER`, which also fixes #21919. When compiled with `--disable-external-signer` such wallets can't be created in RPC or GUI, but they can be loaded. Attempting any action that calls HWI will trigger an error.

  Side-note: this PR may or may not (currently) break CI for the GUI repository, as explained here: https://github.com/bitcoin-core/gui/pull/4#issuecomment-769859001

ACKs for top commit:
  achow101:
    ACK 2f5bdcbc31
  hebasto:
    re-ACK 2f5bdcbc31

Tree-SHA512: 1b71c5a8bea2be077ee9fa33a01130c957a0cf90951d4b7b04d3d0ef826bb77e474c3963abddfef2e2c1ea99d9c72cd2302d1eb9b5fcb7ba0bd2a625f006aa05
2021-06-17 12:47:37 +08:00
fanquake
65c4a36e57 Merge bitcoin/bitcoin#22258: build: Disable deprecated-copy warning only when external warnings are enabled
1111457d74 build: Disable deprecated-copy warning only when external warnings are enabled (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/18967

  Alternative to https://github.com/bitcoin/bitcoin/pull/22240

ACKs for top commit:
  fanquake:
    tACK 1111457d74

Tree-SHA512: 0fc826f26ebbeab662fa7eed2a5cc1630c6c4e612deb91734885fc8bae0352be657ec48ae94ff55a984ac36d27b95cea8d947cc5cf408231d56addecf79db83f
2021-06-17 11:44:18 +08:00
sanket1729
55311197c4 Added new test for future blocks reacceptance
Adds a test case for checking reacceptance a previously rejected block
that was too far in the future.
2021-06-16 16:43:20 -07:00
sanket1729
511a5af462 Fixed inconsistencies between code and comments
1) Makes the code for block 44 consistent with  the expected figure in
the comment above it by adding a transaction to the block
2) Fixed comment describing sign_tx() function
2021-06-16 16:43:20 -07:00
MarcoFalke
1111457d74 build: Disable deprecated-copy warning only when external warnings are enabled 2021-06-16 15:44:27 +02:00
Sjors Provoost
2f5bdcbc31 gui: misc external signer fixes and translation hints 2021-06-16 10:48:58 +02:00
Sjors Provoost
d672404466 refactor: make ExternalSigner NetworkArg() and m_chain private 2021-06-16 10:48:58 +02:00
Sjors Provoost
4455145e26 refactor: reduce #ifdef ENABLE_EXTERNAL_SIGNER usage
In particular this make the node interface independent on whether external signer support is compiled.
2021-06-16 10:48:58 +02:00
Sjors Provoost
5be90c907e build: enable external signer by default 2021-06-16 10:48:57 +02:00
Sjors Provoost
7d9453041b refactor: clean up external_signer.h includes
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-06-16 10:48:38 +02:00
Sjors Provoost
fc0eca31b3 fuzz: fix fuzz binary linking order
We encountered a linking error when attempting to include external_signer_scriptpubkeyman.cpp when configured with --disable-external-signer.

Everywhere else we have LIBBITCOIN_WALLET, it is always before LIBBITCOIN_COMMON. But if you go up to where FUZZ_SUITE_LD_COMMON is first set, you see that we will end up having LIBBITCOIN_COMMON set before LIBBITCOIN_WALLET which means that the linker will have problems linking things common things that the wallet uses. Because the order is correct for the other targets, we only see a linker error for test/fuzz/fuzz.

In this diff, LIBTEST_UTIL and LIBTEST_FUZZ are moved to the top because they include LIBBITCOIN_SERVER and LIBBITCOIN_COMMON. LIBBITCOIN_SERVER always needs to be the first item in the linker order since it has the most dependencies.

The makefiles for making the fuzz and test binaries should be revisited so that the linking order is made consistent with the rest of the code and to avoid other linker order issues that may crop up in the future.

Co-Authored-By: Andrew Chow <achow101-github@achow101.com>
2021-06-16 10:41:24 +02:00
fanquake
6bc1eca01b Merge bitcoin/bitcoin#22144: Randomize message processing peer order
79c02c88b3 Randomize message processing peer order (Pieter Wuille)

Pull request description:

  Right now, the message handling loop iterates the list of nodes always in the same order: the order they were connected in (see the `vNodes` vector). For some parts of the net processing logic, this order matters. Transaction requests are assigned explicitly to peers since #19988, but many other parts of processing work on a "first-served-by-loop-first" basis, such as block downloading. If peers can predict this ordering, it may be exploited to cause delays.

  As there isn't anything particularly optimal about the current ordering, just make it unpredictable by randomizing.

  Reported by Crypt-iQ.

ACKs for top commit:
  jnewbery:
    ACK 79c02c88b3
  Crypt-iQ:
    ACK 79c02c88b3
  sdaftuar:
    utACK 79c02c88b3
  achow101:
    Code Review ACK 79c02c88b3
  jamesob:
    crACK 79c02c88b3
  jonatack:
    ACK 79c02c88b3
  vasild:
    ACK 79c02c88b3
  theStack:
    ACK 79c02c88b3

Tree-SHA512: 9a87c4dcad47c2d61b76c4f37f59674876b78f33f45943089bf159902a23e12de7a5feae1a73b17cbc3f2e37c980ecf0f7fd86af9e6fa3a68099537a3c82c106
2021-06-16 11:27:16 +08:00
fanquake
8f7704d032 build: improve detection of eBPF support
Just checking for the `sys/sdt.h` header isn't enough, as systems like
macOS have the header, but it doesn't actually have the dtrace probes,
which leads to compile failures.
2021-06-16 10:16:03 +08:00
Hennadii Stepanov
1155978d8f build, qt: Do not install *.prl files 2021-06-16 02:23:29 +03:00
Luke Dashjr
754e802274 test: check rejected future block later accepted
(Luke) was unsure if the code sufficiently avoided caching a
time-too-new rejection, so wrote this test to check it.  It looks like
despite only exempting BLOCK_MUTATED, it is still okay because header
failures never cache block invalidity.  This test will help ensure that
if this ever changes, BLOCK_TIME_FUTURE gets excluded at the same time.

Co-authored-by: Will Clark <will8clark@gmail.com>
2021-06-15 21:35:29 +01:00
Hennadii Stepanov
763793b60e build, qt: Fix wrong cross-compiling detection on macOS 2021-06-15 22:36:43 +03:00
Hennadii Stepanov
30982721ab build, qt: Force bootstrap while building linguist tools
Qt lrelease tool depends on the xml module. This change guarantees that
it is always available after being bootstrapped.
2021-06-15 22:36:28 +03:00
Hennadii Stepanov
689320e307 build, qt: Drop translations.pro hack
It is no longer required after switching to Qt top-level build.
2021-06-15 21:32:07 +03:00
Hennadii Stepanov
6a1f98f253 build, qt: Drop lrelease dependency patch
It is no longer required after switching to Qt top-level build.
2021-06-15 21:31:09 +03:00
Hennadii Stepanov
39e561e087 build, qt: Add linguist_tools list 2021-06-15 21:23:16 +03:00
Hennadii Stepanov
27d3def1c6 build: Use Qt top-level build facilities 2021-06-15 21:23:16 +03:00
S3RK
451b96f7d2 test: kill process group to avoid dangling processes 2021-06-15 09:37:58 +02:00
fanquake
eb63b1db2c Merge bitcoin/bitcoin#22247: Switch Appveyor CI to VS2019 stable image
aab7fd0f8d Switch Appveyor CI to VS2019 stable image (Aaron Clauson)

Pull request description:

  The current appveyor config is using the VS2019 preview image so the latest prebuilt Qt5.12.11 binaries can be used, see #22224.

  Appveyor updated the Visual Studio 2019 image to msbuild v16.10.1 on 14th of June. This is the version used to build the latest Qt binaries and removes the need to use the Appveyor VS2019 preview image.

ACKs for top commit:
  MarcoFalke:
    review ACK aab7fd0f8d if green
  practicalswift:
    cr ACK aab7fd0f8d: patch looks correct
  hebasto:
    ACK aab7fd0f8d

Tree-SHA512: 42ea4e6e27a2099ddeed99a3352e8ff014df1e93fbcb1f0f2ebd6f22ec2fb71212275a2adb2a7858516203e566c6ba053367b4ac7e0b74457f7a35e941d6fdd4
2021-06-15 09:25:58 +08:00
Hennadii Stepanov
3f68f02db9 Merge bitcoin-core/gui#362: Add keyboard shortcuts to context menus
e4c916a0ea Bugfix: GUI: Use a different shortcut for "1 d&ay" banning, due to conflict with "&Disconnect" (Luke Dashjr)
94e7cdd7e0 GUI: Add keyboard shortcuts for other context menus (Luke Dashjr)
02b5263cd4 GUI: Restore keyboard shortcuts for context menu entries (Luke Dashjr)

Pull request description:

  Various keyboard shortcuts were lost in #263; this restores them, and also adds new ones for other context menus.

  Note that with a context menu open, simply the shortcut by itself (no Alt) is used.

ACKs for top commit:
  jarolrod:
    Code Review ACK e4c916a
  hebasto:
    ACK e4c916a0ea, tested on Linux Mint 20.1 (Qt 5.12.8).

Tree-SHA512: 949461acf7aac592bc48a1c5abad41b167365830e0cedb3aa11b6a87bd347e16126830ea87936f9c9efc4b7df5b09d3833fae784964d6d119ed45703cfba2ffd
2021-06-15 00:57:18 +03:00
Carl Dong
e2c40a4ed5 guix-attest: Error out if SHA256SUMS is unexpected 2021-06-14 17:05:26 -04:00
Aaron Clauson
aab7fd0f8d Switch Appveyor CI to VS2019 stable image
The current appveyor config is using the VS2019 preview image so the latest prebuilt Qt5.12.11 binaries can be used, see #22224.

Appveyor updated the Visual Studio 2019 image to msbuild v16.10.1 on 14th of June. This is the version used to build the latest Qt binaries and removes the need to use the Appveyor VS2019 preview image.
2021-06-14 20:35:00 +01:00
W. J. van der Laan
a33381acf5 devtools: Add xkb version to symbol-check
xkb versions symbols (using the prefix `V`), as this library is used by
bitcoin-qt, add it to the valid versions in `symbol-check.py`.
2021-06-14 20:32:09 +02:00
W. J. van der Laan
19e598bab0 devtools: Fix verneed section parsing in pixie
I misunderstood the ELF specification for version symbols (verneed):
The `vn_aux` pointer is relative to the main verneed record, not the
start of the section.

This caused many symbols to not be versioned properly in the return
value of `elf.dyn_symbols`. This was discovered in #21454.

Fix it by correcting the offset computation.
2021-06-14 20:31:58 +02:00
MarcoFalke
3a2c84a6b5 Merge bitcoin/bitcoin#19238: refactor: Make CAddrMan::cs non-recursive
ae98aec9c0 refactor: Make CAddrMan::cs non-recursive (Hennadii Stepanov)
f5d1c7fac7 Add AssertLockHeld to CAddrMan private functions (Hennadii Stepanov)
5ef1d0b698 Add thread safety annotations to CAddrMan public functions (Hennadii Stepanov)
b138973a8b refactor: Avoid recursive locking in CAddrMan::Clear (Hennadii Stepanov)
f79a664314 refactor: Apply consistent pattern for CAddrMan::Check usage (Hennadii Stepanov)
187b7d2bb3 refactor: Avoid recursive locking in CAddrMan::Check (Hennadii Stepanov)
f77d9c79aa refactor: Fix CAddrMan::Check style (Hennadii Stepanov)
06703973c7 Make CAddrMan::Check private (Hennadii Stepanov)
efc6fac951 refactor: Avoid recursive locking in CAddrMan::size (Hennadii Stepanov)
2da95545ea test: Drop excessive locking in CAddrManTest::SimConnFail (Hennadii Stepanov)

Pull request description:

  This PR replaces `RecursiveMutex CAddrMan::cs` with `Mutex CAddrMan::cs`.

  All of the related code branches are covered by appropriate lock assertions to insure that the mutex locking policy has not been changed by accident.

  Related to #19303.

  Based on #22025, and first three commits belong to it.

ACKs for top commit:
  vasild:
    ACK ae98aec9c0

Tree-SHA512: c3a2d3d955a75befd7e497a802b8c10730e393be9111ca263ad0464d32fae6c7edf9bd173ffb6bc9bb61c4b39073a74eba12979d47f26b0b7b4a861d100942df
2021-06-14 16:41:14 +02:00
Hennadii Stepanov
ae98aec9c0 refactor: Make CAddrMan::cs non-recursive 2021-06-14 17:28:38 +03:00
Hennadii Stepanov
f5d1c7fac7 Add AssertLockHeld to CAddrMan private functions 2021-06-14 17:28:38 +03:00
Hennadii Stepanov
5ef1d0b698 Add thread safety annotations to CAddrMan public functions 2021-06-14 17:28:38 +03:00
Hennadii Stepanov
b138973a8b refactor: Avoid recursive locking in CAddrMan::Clear
Co-authored-by: John Newbery <john@johnnewbery.com>
2021-06-14 17:28:37 +03:00
Hennadii Stepanov
f79a664314 refactor: Apply consistent pattern for CAddrMan::Check usage
Co-authored-by: John Newbery <john@johnnewbery.com>
2021-06-14 17:28:37 +03:00
Hennadii Stepanov
187b7d2bb3 refactor: Avoid recursive locking in CAddrMan::Check 2021-06-14 17:28:37 +03:00
Hennadii Stepanov
f77d9c79aa refactor: Fix CAddrMan::Check style
This change improves readability, and follows Developer Notes.
2021-06-14 17:28:36 +03:00
Hennadii Stepanov
06703973c7 Make CAddrMan::Check private
Change in the addrman.h header is move-only.
2021-06-14 17:28:30 +03:00
Hennadii Stepanov
efc6fac951 refactor: Avoid recursive locking in CAddrMan::size 2021-06-14 17:21:28 +03:00
Hennadii Stepanov
2da95545ea test: Drop excessive locking in CAddrManTest::SimConnFail
The unit test is single threaded, so there's no need to hold the mutex
between Good() and Attempt().

This change avoids recursive locking in the CAddrMan::Attempt function.

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-06-14 17:21:22 +03:00
W. J. van der Laan
5c4f0c4d46 Merge bitcoin/bitcoin#21261: p2p: update inbound eviction protection for multiple networks, add I2P peers
1b1088d52f test: add combined I2P/onion/localhost eviction protection tests (Jon Atack)
7c2284eda2 test: add tests for inbound eviction protection of I2P peers (Jon Atack)
ce02dd1ef1 p2p: extend inbound eviction protection by network to I2P peers (Jon Atack)
70bbc62711 test: add combined onion/localhost eviction protection coverage (Jon Atack)
045cb40192 p2p: remove unused m_is_onion member from NodeEvictionCandidate struct (Jon Atack)
310fab4928 p2p: remove unused CompareLocalHostTimeConnected() (Jon Atack)
9e889e8a5c p2p: remove unused CompareOnionTimeConnected() (Jon Atack)
787d46bb2a p2p: update ProtectEvictionCandidatesByRatio() doxygen docs (Jon Atack)
1e15acf478 p2p: make ProtectEvictionCandidatesByRatio() fully ratio-based (Jon Atack)
3f8105c4d2 test: remove combined onion/localhost eviction protection tests (Jon Atack)
38a81a8e20 p2p: add CompareNodeNetworkTime() comparator struct (Jon Atack)
4ee7aec47e p2p: add m_network to NodeEvictionCandidate struct (Jon Atack)
7321e6f2fe p2p, refactor: rename vEvictionCandidates to eviction_candidates (Jon Atack)
ec590f1d91 p2p, refactor: improve constness in ProtectEvictionCandidatesByRatio() (Jon Atack)
4a19f501ab test: add ALL_NETWORKS to test utilities (Jon Atack)
519e76bb64 test: speed up and simplify peer_eviction_test (Jon Atack)
1cde800523 p2p, refactor: rm redundant erase_size calculation in SelectNodeToEvict() (Jon Atack)

Pull request description:

  Continuing the work in #20197 and #20685, this pull updates and abstracts our inbound eviction protection to make it fully ratio-based and easily extensible to peers connected via high-latency privacy networks that we newly support, like I2P and perhaps others soon, as these peers are disadvantaged by the latency criteria of our eviction logic.

  It then adds eviction protection for peers connected over I2P.  As described in https://github.com/bitcoin/bitcoin/pull/20685#issuecomment-767486499, we've observed over the past few months that I2P peers have a min ping latency similar to or greater than that of onion peers.

  The algorithm is a basically a multi-pass knapsack:

  - Count the number of eviction candidates in each of the disadvantaged
    privacy networks.

  - Sort the networks from lower to higher candidate counts, so that
    a network with fewer candidates will have the first opportunity
    for any unused slots remaining from the previous iteration.  In
    the case of a tie in candidate counts, priority is given by array
    member order from first to last, guesstimated to favor more unusual
    networks.

  - Iterate through the networks in this order.  On each iteration,
    allocate each network an equal number of protected slots targeting
    a total number of candidates to protect, provided any slots remain
    in the knapsack.

  - Protect the candidates in that network having the longest uptime,
    if any in that network are present.

  - Continue iterating as long as we have non-allocated slots
    remaining and candidates available to protect.

  The goal of this logic is to favorise the diversity of our peer connections.

  The individual commit messages describe each change in more detail.

  Special thank you to Vasil Dimov for the excellent review feedback and the algorithm improvement that made this change much better than it would have been otherwise. Thanks also to Antoine Riard, whose review feedback nudged this change to protect disadvantaged networks having fewer, rather than more, eviction candidates.

ACKs for top commit:
  laanwj:
    Code review re-ACK 1b1088d52f
  vasild:
    ACK 1b1088d52f

Tree-SHA512: 722f790ff11f2969c79e45a5e0e938d94df78df8687e77002f32e3ef5c72a9ac10ebf8c7a9eb7f71882c97ab0e67b2778191effdb747d9ca54d7c23c2ed19a90
2021-06-14 15:04:32 +02:00
Jon Atack
1b1088d52f test: add combined I2P/onion/localhost eviction protection tests 2021-06-14 14:02:15 +02:00
Jon Atack
7c2284eda2 test: add tests for inbound eviction protection of I2P peers 2021-06-14 14:01:44 +02:00
Jon Atack
ce02dd1ef1 p2p: extend inbound eviction protection by network to I2P peers
This commit extends our inbound eviction protection to I2P peers to
favorise the diversity of peer connections, as peers connected
through the I2P network are otherwise disadvantaged by our eviction
criteria for their higher latency (higher min ping times) relative
to IPv4 and IPv6 peers, as well as relative to Tor onion peers.

The `networks` array is order-dependent in the case of a tie in
candidate counts between networks (earlier array members receive
priority in the case of a tie).

Therefore, we place I2P candidates before localhost and onion ones
in terms of opportunity to recover unused remaining protected slots
from the previous iteration, guesstimating that most nodes allowing
both onion and I2P inbounds will have more onion peers, followed by
localhost, then I2P, as I2P support is only being added in the
upcoming v22.0 release.
2021-06-14 14:01:35 +02:00
Jon Atack
70bbc62711 test: add combined onion/localhost eviction protection coverage 2021-06-14 14:00:12 +02:00
Jon Atack
045cb40192 p2p: remove unused m_is_onion member from NodeEvictionCandidate struct 2021-06-14 13:58:05 +02:00
Jon Atack
310fab4928 p2p: remove unused CompareLocalHostTimeConnected() 2021-06-14 13:58:03 +02:00
Jon Atack
9e889e8a5c p2p: remove unused CompareOnionTimeConnected() 2021-06-14 13:58:01 +02:00
Jon Atack
787d46bb2a p2p: update ProtectEvictionCandidatesByRatio() doxygen docs 2021-06-14 13:57:59 +02:00
Jon Atack
1e15acf478 p2p: make ProtectEvictionCandidatesByRatio() fully ratio-based
with a more abstract framework to allow easily extending inbound
eviction protection to peers connected through new higher-latency
networks that are disadvantaged by our inbound eviction criteria,
such as I2P and perhaps other BIP155 networks in the future like
CJDNS.  This is a change in behavior.

The algorithm is a basically a multi-pass knapsack:

- Count the number of eviction candidates in each of the disadvantaged
  privacy networks.

- Sort the networks from lower to higher candidate counts, so that
  a network with fewer candidates will have the first opportunity
  for any unused slots remaining from the previous iteration.  In
  the case of a tie in candidate counts, priority is given by array
  member order from first to last, guesstimated to favor more unusual
  networks.

- Iterate through the networks in this order.  On each iteration,
  allocate each network an equal number of protected slots targeting
  a total number of candidates to protect, provided any slots remain
  in the knapsack.

- Protect the candidates in that network having the longest uptime,
  if any in that network are present.

- Continue iterating as long as we have non-allocated slots
  remaining and candidates available to protect.

Localhost peers are treated as a network like Tor or I2P by aliasing
them to an unused Network enumerator: Network::NET_MAX.

The goal is to favorise diversity of our inbound connections.

Credit to Vasil Dimov for improving the algorithm from single-pass
to multi-pass to better allocate unused protection slots.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-06-14 13:57:49 +02:00
Luke Dashjr
e4c916a0ea Bugfix: GUI: Use a different shortcut for "1 d&ay" banning, due to conflict with "&Disconnect" 2021-06-14 07:08:04 +00:00
fanquake
de5512e28d Merge bitcoin/bitcoin#22230: build: Fix MSVC linker /SubSystem option for bitcoin-qt.exe
9edd713c18 build: Fix MSVC linker /SubSystem option for bitcoin-qt.exe (Hennadii Stepanov)

Pull request description:

  On master (6f3fbc062f), running `bitcoin-qt.exe`, which was built with MSVC, causes a terminal window open along with the GUI.

  This PR fixes such behavior. See Microsoft [docs](https://docs.microsoft.com/en-us/cpp/build/reference/subsystem-specify-subsystem?view=msvc-160).

  It is still possible to use the `-printtoconsole` option for debug builds.

ACKs for top commit:
  sipsorcery:
    tACK 9edd713c18.

Tree-SHA512: 02f2874b13e484f98344f6a7e3b01fa82a78a39865787c77bd674ead22a84a7f98a1849ccad26bd2b8c8603b3e29dcc1633b0ad731ce7d61be2d6b1f9584839c
2021-06-14 10:06:55 +08:00
fanquake
ad2ab88ccc Merge bitcoin/bitcoin#22224: Update msvc and appveyor builds to use Qt5.12.11 binaries
e25ea54dbf Update msvc and appveyor builds to use Qt5.12.11 binaries. (Aaron Clauson)

Pull request description:

  Synchronises the Qt version used in the msvc and Appveyor builds with #22054.

  I needed to use switch to the `Visual Studio 2019 Preview` Appveyor image because the compiler version on the non-preview image is too far behind and I had difficulty building a compatible Qt version for it. Once the main Appveyor `Visual Studio 2019` image reaches version `16.10.1` it can be used.

ACKs for top commit:
  hebasto:
    ACK e25ea54dbf

Tree-SHA512: c5e8dcafa342df7cd8ff7c349a8186bee4cdf7fd748c5d94039e30698775058bae8099dd75a50a5079f3cbb5251e695be187bae615159e3cd45054a972c4e6bd
2021-06-14 10:04:03 +08:00
Jon Atack
3f8105c4d2 test: remove combined onion/localhost eviction protection tests
as we are about the change the behavior sufficiently that when we
have multiple disadvantaged networks and a small number of peers
under test, the number of protected peers per network can be different.
2021-06-13 20:15:51 +02:00
Jon Atack
38a81a8e20 p2p: add CompareNodeNetworkTime() comparator struct
to compare and sort peer eviction candidates by the
passed-in is_local (localhost status) and network
arguments, and by longest uptime.
2021-06-13 20:15:49 +02:00
Jon Atack
4ee7aec47e p2p: add m_network to NodeEvictionCandidate struct 2021-06-13 20:15:47 +02:00
Jon Atack
7321e6f2fe p2p, refactor: rename vEvictionCandidates to eviction_candidates
in ProtectEvictionCandidatesByRatio()
per current style guide in doc/developer-notes.md
2021-06-13 20:15:45 +02:00
Jon Atack
ec590f1d91 p2p, refactor: improve constness in ProtectEvictionCandidatesByRatio() 2021-06-13 20:15:43 +02:00
Jon Atack
4a19f501ab test: add ALL_NETWORKS to test utilities 2021-06-13 20:15:41 +02:00
Jon Atack
519e76bb64 test: speed up and simplify peer_eviction_test
This speeds up the test significantly, which helps when
running it repeatedly.

Suggest reviewing the diff with:

colorMoved = dimmed-zebra
colorMovedWs = allow-indentation-change
2021-06-13 20:14:40 +02:00
MarcoFalke
d75a1df617 Merge bitcoin/bitcoin#21941: fuzz: Call const member functions in addrman fuzz test only once
faf7623106 fuzz: Call const member functions in addrman fuzz test only once (MarcoFalke)

Pull request description:

  Logically based on #21940

  Currently the fuzz test may spend a long time generating random numbers:

  ![Screenshot from 2021-05-13 12-14-09](https://user-images.githubusercontent.com/6399679/118112238-06ecd880-b3e5-11eb-8013-6e0c20e6159f.png)

  Fix that by calling const member functions only once.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34224

ACKs for top commit:
  practicalswift:
    cr ACK faf7623106: touches only `src/test/fuzz/addrman.cpp`

Tree-SHA512: 0fe9e0111eb1706fc39bd2f90d4b87a771882bada54c01e96d8e79c2afca2f1081139d5ab680285a81835cc5142e74ada422a181db34b01904975d1e167e64c2
2021-06-13 19:37:58 +02:00
MarcoFalke
964d91b193 Merge bitcoin/bitcoin#22130: test: refactor: dedup utility function chain_transaction()
01eedf3821 test: doc: improve doc for chain_transaction() helper (Sebastian Falbesoner)
6e63e366d6 test: refactor: dedup utility function chain_transaction() (Sebastian Falbesoner)

Pull request description:

  Both tests `mempool_packages.py` and `mempool_package_onemore.py` define a utility function `chain_transaction` with a similar implementation. This PR deduplicates it by moving it into the util package and keeping the more general properties:
  * pass a list of parent_txids/vouts instead of single values
  * always mark the BIP125-replaceable flag for txs, created via `createrawtransaction` (this is needed by the `mempool_package_onemore.py` test, but doesn't hurt the other one)

  This is a low-hanging fruit; as a potential follow-up one could probably also deduplicate the function `chain_transaction` in `rpc_packages.py`, which looks a bit different, as it also takes the parent locking script into account and doesn't send the tx.

ACKs for top commit:
  mjdietzx:
    reACK 01eedf3821
  klementtan:
    Code review ACK 01eedf3821
  MarcoFalke:
    review ACK 01eedf3821 🙅

Tree-SHA512: ac7105d02c23f53d76d4ec9dc8de1074dd8faefeecd44b107921b78665279498966152fed312ecbe252a1c34a9643d531166329a4fea0e773df3bb75d43092b0
2021-06-13 18:27:32 +02:00
MarcoFalke
faf7623106 fuzz: Call const member functions in addrman fuzz test only once 2021-06-13 13:52:21 +02:00
MarcoFalke
fa0d9211ef refactor: Remove chainparams arg from CChainState member functions
Passing this is confusing and redundant with the m_params member.
2021-06-13 09:43:54 +02:00
MarcoFalke
fa38947125 refactor: Remove ::Params() global from inside CChainState member functions
It is confusing and verbose to repeatedly access the global when a
member variable can simply refer to it.
2021-06-13 09:39:37 +02:00
MarcoFalke
9c1ec689f3 Merge bitcoin/bitcoin#22102: Remove Warning: from warning message printed for unknown new rules
6d7e46ce23 Remove `Warning:` (Prayank)

Pull request description:

  Reason: I noticed that `Warning` is printed 2 times in `-getinfo` while reviewing https://github.com/bitcoin/bitcoin/pull/21832#issuecomment-851004943

  Same string is used for GUI, log and stderr. If we need to add `Warning:` in GUI or other place we can always prepend to this string.

  CLI:

  ```
  Warnings: Unknown new rules activated (versionbit 28)

  ```

  GUI:

  ![image](https://user-images.githubusercontent.com/13405205/120110401-e36ab180-c18a-11eb-8031-4d52287dc263.png)

ACKs for top commit:
  MarcoFalke:
    review ACK 6d7e46ce23

Tree-SHA512: 25760cf6d850f6c2216d651fa46574d2d21a9d58f4577ecb58f9566ea8cd07bfcd6679bb8a1f53575e441b02d295306f492c0c99a48c909e60e5d977ec1861f6
2021-06-13 09:21:46 +02:00
Luke Dashjr
94e7cdd7e0 GUI: Add keyboard shortcuts for other context menus 2021-06-12 19:39:38 +00:00
Luke Dashjr
02b5263cd4 GUI: Restore keyboard shortcuts for context menu entries
This partially reverts f385ad7651.
2021-06-12 19:39:36 +00:00
Pieter Wuille
458a345b05 Add support for SIGHASH_DEFAULT in RPCs, and make it default
For non-Taproot signatures, this is interpreted as SIGHASH_ALL.
2021-06-12 12:38:17 -07:00
Pieter Wuille
c0f0c8eccb tests: check spending of P2TR 2021-06-12 12:38:15 -07:00
Pieter Wuille
a2380127e9 Basic Taproot signing logic in script/sign.cpp 2021-06-12 12:25:28 -07:00
Pieter Wuille
49487bc3b6 Make GetInputUTXO safer: verify non-witness UTXO match 2021-06-12 12:25:28 -07:00
Pieter Wuille
fd3f6890f3 Construct and use PrecomputedTransactionData in PSBT signing 2021-06-12 12:25:28 -07:00
Pieter Wuille
5cb6502ac5 Construct and use PrecomputedTransactionData in SignTransaction 2021-06-12 12:25:28 -07:00
Pieter Wuille
5d2e22437b Don't nuke witness data when signing fails 2021-06-12 12:25:28 -07:00
Pieter Wuille
ce9353164b Permit full precomputation in PrecomputedTransactionData
At verification time, the to be precomputed data can be inferred from
the transaction itself. For signing, the necessary witnesses don't
exist yet, so just permit precomputing everything in that case.
2021-06-12 12:25:28 -07:00
Pieter Wuille
e841fb503d Add precomputed txdata support to MutableTransactionSignatureCreator
This provides a means to pass in a PrecomputedTransactionData object to
the MutableTransactionSignatureCreator, allowing the prevout data to be
passed into the signature hashers. It is also more efficient.
2021-06-12 12:25:28 -07:00
Pieter Wuille
a91d532338 Add CKey::SignSchnorr function for BIP 340/341 signing 2021-06-12 12:25:28 -07:00
Pieter Wuille
e77a2839b5 Use HandleMissingData also in CheckSchnorrSignature 2021-06-12 12:25:28 -07:00
Pieter Wuille
dbb0ce9fbf Add TaprootSpendData data structure, equivalent to script map for P2[W]SH
This data structures stores all information necessary for spending a taproot
output (the internal key, the Merkle root, and the control blocks for every
script leaf).

It is added to signing providers, and populated by the tr() descriptor.
2021-06-12 12:25:28 -07:00
W. J. van der Laan
b0e5fbf6fa Merge bitcoin/bitcoin#22156: Allow tr() import only when Taproot is active
fbf485c9b2 Allow tr() import only when Taproot is active (Andrew Chow)

Pull request description:

  To avoid issues around fund loss, only allow descriptor wallets to import `tr()` descriptors after taproot has activated.

ACKs for top commit:
  sipa:
    utACK fbf485c9b2
  fjahr:
    Code review ACK fbf485c9b2
  laanwj:
    Code review ACK fbf485c9b2
  prayank23:
    utACK fbf485c9b2

Tree-SHA512: 83c43376515eea523dbc89bc5a0fde53e54aec492e49a40c2a33d80fc94aac459e232ae07b024b4bd75b58078c8d090bc7a2d69541c5d3d4834d2f4cfc9c8208
2021-06-12 17:22:41 +02:00
Hennadii Stepanov
9edd713c18 build: Fix MSVC linker /SubSystem option for bitcoin-qt.exe
It is still possible to use -printtoconsole option for debug builds.
2021-06-12 17:13:27 +03:00
Hennadii Stepanov
6f3fbc062f Merge bitcoin-core/gui#333: refactor: Signal-slot connections cleanup
f507681baa qt: Connect WalletView signal to BitcoinGUI slot directly (Hennadii Stepanov)
bd50ff9290 qt: Drop redundant OverviewPage::handleOutOfSyncWarningClicks slot (Hennadii Stepanov)
793f19599b qt: Drop redundant WalletView::requestedSyncWarningInfo slot (Hennadii Stepanov)

Pull request description:

  This PR:
  - removes slots whose only job is to emit a signal, since we can use the signal as a slot
  - connects the`WalletView::outOfSyncWarningClicked` signal to the `BitcoinGUI::showModalOverlay` slot directly, and removes intermediate `WalletFrame` slot and signal
  - split from #29

  This PR does not change behavior.

ACKs for top commit:
  Talkless:
    tACK f507681baa, tested on Debian Sid with Qt 5.15.2, no any behavioral changes noticed.
  promag:
    Code review ACK f507681baa.

Tree-SHA512: cd636a7e61881b2cbee84d5425d2107a8e39683b8eb32d79dc9ea942db55d5c1979be2f70da1660eaee5de622d10ed5a92f11fc2351de21b84324b10b23d0c96
2021-06-12 14:04:18 +03:00
W. J. van der Laan
4bff49bbb1 Merge bitcoin/bitcoin#22172: doc: update tor.md, release notes with removal of tor v2 support
2ad034a890 doc: update release notes with removal of tor v2 support (Jon Atack)
49938eee9c doc: update tor.md with removal of tor v2 support (Jon Atack)

Pull request description:

  Follow-up documentation to #22050 that removed support for Tor version 2 hidden services from Bitcoin Core.

ACKs for top commit:
  laanwj:
    ACK 2ad034a890

Tree-SHA512: 0f13f9d1db7e11f1e3d9967b6d17b8dc3144b3ab3a258c706464c5e6ac5cbcf2ce2db4ea54be9939f05a82ebd1e7f325f50b435f9822c08b4f21ed4ac58de0af
2021-06-12 12:23:17 +02:00
Jon Atack
2ad034a890 doc: update release notes with removal of tor v2 support 2021-06-12 12:09:25 +02:00
fanquake
96f828ba4d Merge bitcoin/bitcoin#22221: refactor: Pass block reference instead of pointer to PeerManagerImpl::BlockRequested
fa334b4054 refactor: Pass block reference instead of pointer to PeerManagerImpl::BlockRequested (MarcoFalke)

Pull request description:

  This allows to remove an assert and at the same time make it more obvious that the block is never nullptr.

  Also, add missing `{}` while touching the function.

ACKs for top commit:
  jnewbery:
    Code review ACK fa334b4054
  mjdietzx:
    crACK fa334b4054
  theStack:
    Code review ACK fa334b4054

Tree-SHA512: 9733d3e20e048fcb2ac7510eae3539ce8aaa7397bd944a265123f1ffd90e15637cdaad19dba16f76d83f3f0d1888f1b7014c191bb430e410a106c49ca61a725c
2021-06-12 15:54:38 +08:00
fanquake
1a369f006f Merge bitcoin/bitcoin#18722: addrman: improve performance by using more suitable containers
a92485b2c2 addrman: use unordered_map instead of map (Vasil Dimov)

Pull request description:

  `CAddrMan` uses `std::map` internally even though it does not require
  that the map's elements are sorted. `std::map`'s access time is
  `O(log(map size))`. `std::unordered_map` is more suitable as it has a
  `O(1)` access time.

  This patch lowers the execution times of `CAddrMan`'s methods as follows
  (as per `src/bench/addrman.cpp`):

  ```
  AddrMan::Add(): -3.5%
  AddrMan::GetAddr(): -76%
  AddrMan::Good(): -0.38%
  AddrMan::Select(): -45%
  ```

ACKs for top commit:
  jonatack:
    ACK a92485b2c2
  achow101:
    ACK a92485b2c2
  hebasto:
    re-ACK a92485b2c2, only suggested changes and rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/18722#pullrequestreview-666663681) review.

Tree-SHA512: d82959a00e6bd68a6c4c5a265dd08849e6602ac3231293b7a3a3b7bf82ab1d3ba77f8ca682919c15c5d601b13e468b8836fcf19595248116635f7a50d02ed603
2021-06-12 11:41:27 +08:00
fanquake
9795e8ec8c Merge bitcoin/bitcoin#22214: refactor: Rearrange fillPSBT arguments
f47e802839 Rearrange fillPSBT arguments (Russell Yanofsky)

Pull request description:

  Move fillPSBT inout argument before output-only arguments. This is a nice thing to do to keep the interface style [consistent](https://google.github.io/styleguide/cppguide.html#Inputs_and_Outputs). But motivation is to work around a current limitation of the libmultiprocess code generator (which figures out order of inout parameters by looking at input list, but more ideally would use the output list).

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). The commit was first part of larger PR #10102.

ACKs for top commit:
  achow101:
    ACK f47e802839
  theStack:
    Code-review ACK f47e802839

Tree-SHA512: 1787af3031ff7ed6b519f3b93054d8b257af96a3380a476a6dab0f759329039ecc5d624b785c5c2d14d594fc852dd81c626880c775c691ec9c79b7b3dbcfb257
2021-06-12 11:35:49 +08:00
fanquake
a55904a80c Merge bitcoin/bitcoin#21866: [Bundle 7/7] validation: Farewell, global Chainstate!
6f994882de validation: Farewell, global Chainstate! (Carl Dong)
972c5166ee qt/test: Reset chainman in ~ChainstateManager instead (Carl Dong)
6c3b5dc0c1 scripted-diff: tree-wide: Remove all review-only assertions (Carl Dong)
3e82abb8dd tree-wide: Remove stray review-only assertion (Carl Dong)
f323248aba qt/test: Use existing chainman in ::TestGUI (can be scripted-diff) (Carl Dong)
6c15de129c scripted-diff: wallet/test: Use existing chainman (Carl Dong)
ee0ab1e959 fuzz: Initialize a TestingSetup for test_one_input (Carl Dong)
0d61634c06 scripted-diff: test: Use existing chainman in unit tests (Carl Dong)
e197076219 test: Pass in CoinsTip to ValidateCheckInputsForAllFlags (Carl Dong)
4d99b61014 test/miner_tests: Pass in chain tip to CreateBlockIndex (Carl Dong)
f0dd5e6bb4 test/util: Use existing chainman in ::PrepareBlock (Carl Dong)
464c313e30 init: Use existing chainman (Carl Dong)

Pull request description:

  Based on:  #21767

  à la Mr. Sandman
  ```
  Mr. Chainman, bring me a tip (bung, bung, bung, bung)
  Make it the most work that I've ever seen (bung, bung, bung, bung)
  Rewind old tip till we're at the fork point (bung, bung, bung, bung)
  Then tell it that it's time to call Con-nectTip

  Chainman, I'm so alone (bung, bung, bung, bung)
  No local objects to call my own (bung, bung, bung, bung)
  Please make sure I have a ref
  Mr. Chainman, bring me a tip!
  ```

  This is the last bundle in the #20158 series. Thanks everyone for their diligent review.
  I would like to call attention to https://github.com/bitcoin/bitcoin/issues/21766, where a few leftover improvements were collated.

  - Remove globals:
    - `ChainstateManager g_chainman`
    - `CChainState& ChainstateActive()`
    - `CChain& ChainActive()`
  - Remove all review-only assertions.

ACKs for top commit:
  jamesob:
    reACK 6f994882de based on the contents of
  ariard:
    Code Review ACK 6f99488.
  jnewbery:
    utACK 6f994882de
  achow101:
    Code Review ACK 6f994882de
  ryanofsky:
    Code review ACK 6f994882de.

Tree-SHA512: 4052ea79360cf0efd81ad0ee3f982e1d93aab1837dcec75f875a56ceda085de078bb3099a2137935d7cc2222004ad88da94b605ef5efef35cb6bc733725debe6
2021-06-12 11:29:31 +08:00
Hennadii Stepanov
a8c8dbc98f Merge bitcoin-core/gui#361: Fix gui segfault caused by bitcoin/bitcoin#22216
d7f3b1af21 Fix gui segfault caused by bitcoin/bitcoin#22216 (Russell Yanofsky)

Pull request description:

  Reported by Hennadii Stepanov https://github.com/bitcoin/bitcoin/pull/22216#issuecomment-859790682

  Fixes bitcoin/bitcoin#22227

ACKs for top commit:
  hebasto:
    ACK d7f3b1af21, tested on Linux Mint 20.1 (Qt 5.12.8).
  jarolrod:
    ACK d7f3b1af21

Tree-SHA512: d672bfa9f1bcd500a879ec7ed27096086ae93b73ad5da8090f29cc5b6d985c46a76583cc384304d67210f87b6b839c2391f0fcc24fd3588c4a014e540283fdfe
2021-06-12 00:55:43 +03:00
Russell Yanofsky
d7f3b1af21 Fix gui segfault caused by bitcoin/bitcoin#22216
Reported by Hennadii Stepanov https://github.com/bitcoin/bitcoin/pull/22216#issuecomment-859790682

Fixes bitcoin/bitcoin#22227
2021-06-11 15:37:30 -04:00
Vasil Dimov
7593b06bd1 test: ensure I2P addresses are relayed
This test would fail if `CNetAddr::IsRelayable()` returns `false` for
I2P addresses, given that this test node does not have I2P connectivity.
2021-06-11 19:17:59 +02:00
Vasil Dimov
e7468139a1 test: make CAddress in functional tests comparable
This way we can compare CAddress objects using `==` or even
arrays of CAddress using `array1 == array2`.
2021-06-11 19:16:07 +02:00
Vasil Dimov
33e211d2a4 test: implement ser/unser of I2P addresses in functional tests 2021-06-11 19:16:06 +02:00
Aaron Clauson
e25ea54dbf Update msvc and appveyor builds to use Qt5.12.11 binaries. 2021-06-11 16:36:31 +01:00
Vasil Dimov
86742811ce test: use NODE_* constants instead of magic numbers
We just assigned `NODE_NETWORK | NODE_WITNESS` to `nServices` a few
lines above. Use that for verifying correctness instead of `9`.
2021-06-11 13:52:42 +02:00
Jon Atack
1cde800523 p2p, refactor: rm redundant erase_size calculation in SelectNodeToEvict()
as EraseLastKElements() called in the next line performs the same operation.
Thanks to Martin Zumsande (lightlike) for seeing this while reviewing.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2021-06-11 12:26:59 +02:00
MarcoFalke
fa334b4054 refactor: Pass block reference instead of pointer to PeerManagerImpl::BlockRequested 2021-06-11 09:56:16 +02:00
MarcoFalke
f66eceaecf Merge bitcoin/bitcoin#22216: refactor: Make SetupServerArgs callable without NodeContext
493fb47c57 Make SetupServerArgs callable without NodeContext (Russell Yanofsky)

Pull request description:

  `bitcoin-gui` code needs to call `SetupServerArgs` but will not have a `NodeContext` object if it is communicating with an external `bitcoin-node` process, so this just passes `ArgsManager` directly.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10). The commit was first part of larger PR #10102.

ACKs for top commit:
  MarcoFalke:
    review ACK 493fb47c57

Tree-SHA512: 94cda4350113237976e32f1935e3602d1e6ea90c29c4434db2094be70dddf4b63702c3094385258bdf1c3e5b52c7d23bbc1f0282bdd4965557eedd5aef9a0fd4
2021-06-11 09:18:34 +02:00
MarcoFalke
6985038046 Merge bitcoin/bitcoin#22118: test: check anchors.dat when node starts for the first time
ef99d03c2b test: check anchors.dat when node starts for the first time (bruno)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598406712, https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598406187, https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598405613.

ACKs for top commit:
  laanwj:
    Code review ACK ef99d03c2b

Tree-SHA512: 505f1f34fbc0c72a92968883be0f1c5f169a4ba3aa8a56e1ce8bc5e514f49e3a17ce51fd40be0073dc4bc06eaeda36dfe90ace843c181170e34643226afd78ef
2021-06-11 08:23:02 +02:00
fanquake
ce8276b531 Merge bitcoin/bitcoin#22190: Use latest signapple commit
683d197970 Use latest signapple commit (Andrew Chow)

Pull request description:

  Update gitian and guix to use the same latest signapple commit.

  Also changed guix to use the actual repo. The changes from the fork were incorporated upstream.

ACKs for top commit:
  fanquake:
    ACK 683d197970 - sanity checked that the updated package is built:

Tree-SHA512: a4981f8bbe33e6c5654632bc9b9f6f2f1e675741a19ac7296205e370f1e64a747101ecb632e0cc82a0134e4c2e9ce47b3f7b4d8c8f75f0f06dd069c078303759
2021-06-11 09:09:26 +08:00
fanquake
551933f9ec Merge bitcoin/bitcoin#22203: test: Use ConnmanTestMsg from test lib in denialofservice_tests
fa72fce7c9 test: Use ConnmanTestMsg from test lib in denialofservice_tests (MarcoFalke)

Pull request description:

  This allows to remove code.

  Also, required for https://github.com/bitcoin/bitcoin/pull/18470

ACKs for top commit:
  mjdietzx:
    crACK fa72fce7c9 👍👍
  fanquake:
    ACK fa72fce7c9

Tree-SHA512: 12aa68cde697c0f7c25d60bb0c02783e5462eb3ba39947b0d94a7798bc278e7d5f092f3ab2a3d0547947c3502cde7c4a599419055a57f78ef1f70f9f637e14c7
2021-06-11 09:07:44 +08:00
Andrew Chow
fbf485c9b2 Allow tr() import only when Taproot is active
To avoid issues around fund loss, only allow descriptor wallets
to import tr() descriptors after taproot has activated.
2021-06-10 15:45:47 -04:00
Carl Dong
6f994882de validation: Farewell, global Chainstate! 2021-06-10 15:05:25 -04:00
Carl Dong
972c5166ee qt/test: Reset chainman in ~ChainstateManager instead
There are some mutable, global state variables that are currently reset
by UnloadBlockIndex such as pindexBestHeader which should be cleaned up
whenever the ChainstateManager is unloaded/reset/destructed/etc.

Not cleaning them up leads to bugs like a use-after-free that happens
like so:

1. At the end of a test, ChainstateManager is destructed, which also
   destructs BlockManager, which calls BlockManager::Unload to free all
   CBlockIndexes in its BlockMap
2. Since pindexBestHeader is not cleaned up, it now points to an invalid
   location
3. Another test starts to init, and calls LoadGenesisBlock, which calls
   AddToBlockIndex, which compares the genesis block with an invalid
   location
4. Cute puppies perish by the hundreds

Previously, for normal codepaths (e.g. bitcoind), we relied on the fact
that our program will be unloaded by the operating system which
effectively resets these variables. The one exception is in QT tests,
where these variables had to be manually reset.

Since now ChainstateManager is no longer a global, we can just put this
logic in its destructor to make sure that callers are always correct.

Over time, we should probably move these mutable global state variables
into ChainstateManager or CChainState so it's easier to reason about
their lifecycles.
2021-06-10 15:05:25 -04:00
Carl Dong
6c3b5dc0c1 scripted-diff: tree-wide: Remove all review-only assertions
-BEGIN VERIFY SCRIPT-
find_regex='((assert|CHECK_NONFATAL)\(std::addressof|TODO: REVIEW-ONLY)' \
    && git grep -l -E "$find_regex" -- . \
        | xargs sed -i -E "/${find_regex}/d"
-END VERIFY SCRIPT-
2021-06-10 15:05:24 -04:00
Carl Dong
3e82abb8dd tree-wide: Remove stray review-only assertion
Unfortunately, these assertion don't fit the regex in the scripted-diff.
Therefore, we remove it manually.
2021-06-10 15:04:39 -04:00
Carl Dong
f323248aba qt/test: Use existing chainman in ::TestGUI (can be scripted-diff) 2021-06-10 15:04:39 -04:00
Carl Dong
6c15de129c scripted-diff: wallet/test: Use existing chainman
-BEGIN VERIFY SCRIPT-
git ls-files -- src/wallet/test \
    | xargs sed -i -E \
            -e 's@g_chainman\.m_blockman@m_node.chainman->m_blockman@g' \
            -e 's@([^:])(Chain(state|)Active)@\1::\2@g' \
            -e 's@::Chain(state|)Active\(\)@m_node.chainman->ActiveChain\1()@g'
-END VERIFY SCRIPT-
2021-06-10 15:04:39 -04:00
Carl Dong
ee0ab1e959 fuzz: Initialize a TestingSetup for test_one_input
For fuzz tests that need it.
2021-06-10 15:04:39 -04:00
Carl Dong
0d61634c06 scripted-diff: test: Use existing chainman in unit tests
-BEGIN VERIFY SCRIPT-
git ls-files -- src/test \
    | grep -v '^src/test/fuzz' \
    | xargs sed -i -E \
            -e 's@g_chainman\.m_blockman@m_node.chainman->m_blockman@g' \
            -e 's@([^:])(Chain(state|)Active)@\1::\2@g' \
            -e 's@::Chain(state|)Active\(\)@m_node.chainman->ActiveChain\1()@g'
-END VERIFY SCRIPT-
2021-06-10 15:04:39 -04:00
Carl Dong
e197076219 test: Pass in CoinsTip to ValidateCheckInputsForAllFlags 2021-06-10 15:04:39 -04:00
Carl Dong
4d99b61014 test/miner_tests: Pass in chain tip to CreateBlockIndex 2021-06-10 15:04:39 -04:00
Carl Dong
f0dd5e6bb4 test/util: Use existing chainman in ::PrepareBlock 2021-06-10 15:04:39 -04:00
Carl Dong
464c313e30 init: Use existing chainman 2021-06-10 15:04:39 -04:00
Russell Yanofsky
f47e802839 Rearrange fillPSBT arguments
Move fillPSBT input-output argument before output-only arguments. This is a
temporary workaround which can go away with improvements to libmultiprocess
code generator. Currently code generator figures out order of input-output
parameters by looking at input list, but it would make more sense for it to
take order from output list, so input-only parameters still have to be first
but there is more flexibility for the other parameters.
2021-06-10 10:58:45 -04:00
W. J. van der Laan
1704bbf226 Merge bitcoin/bitcoin#22141: net processing: Remove hash and fValidatedHeaders from QueuedBlock
2f4ad6b7ef scripted-diff: rename MarkBlockAs functions (John Newbery)
2c45f832e8 [net processing] Tidy up MarkBlockAsReceived() (John Newbery)
6299350733 [net processing] Add IsBlockRequested() function (John Newbery)
4e90d2dd0e [net processing] Remove QueuedBlock.hash (John Newbery)
156a19ee6a scripted-diff: rename nPeersWithValidatedDownloads (John Newbery)
b03de9c753 [net processing] Remove CNodeState.nBlocksInFlightValidHeaders (John Newbery)
b4e29f2436 [net processing] Remove QueuedBlock.fValidatedHeaders (John Newbery)
85e058b191 [net processing] Remove unnecessary hash arg from MarkBlockAsInFlight() (John Newbery)

Pull request description:

  The QueuedBlock struct contains a `fValidatedHeaders` field that indicates whether we have already validated a header for the requested block. Since headers-first syncing, we only request blocks where the header is already validated, so `fValidatedHeaders` is always true. Remove it and clean up the logic that uses that field.

  Likewise, QueuedBlock contains a `hash` field that is set to the block hash. Since headers-first syncing, we always have a CBlockIndex, which contains the block hash, so remove the redundant `hash` field.

  Tidy up the logic and rename functions to better indicate what they're doing.

ACKs for top commit:
  mjdietzx:
    crACK 2f4ad6b7ef
  sipa:
    utACK 2f4ad6b7ef
  MarcoFalke:
    review ACK 2f4ad6b7ef 📊

Tree-SHA512: 3d31d2bcb4d35d0fdb7c1da624c2878203218026445e8f76c4a2df68cc7183ce0e7d0c47c7c0a3242e55efaca7c9f5532b683cf6ec7c03d23fa83764fdb82fd2
2021-06-10 16:58:45 +02:00
Russell Yanofsky
493fb47c57 Make SetupServerArgs callable without NodeContext
bitcoin-gui code needs to call SetupServerArgs but will not have a
NodeContext object if it is communicating with an external bitcoin-node
process.
2021-06-10 09:58:45 -05:00
Pieter Wuille
30aee2dfe6 tests: Add test for compact block HB selection 2021-06-10 10:22:13 -04:00
Vasil Dimov
ba45f02708 net: relay I2P addresses even if not reachable (by us)
Nodes that can reach the I2P network (have set `-i2psam=`) will relay
I2P addresses even without this patch. However, nodes that can't reach
the I2P network will not. This was done as a precaution in
https://github.com/bitcoin/bitcoin/pull/20119 before anybody could
connect to I2P because then, for sure, it would have been useless.

Now, however, we have I2P support and a bunch of I2P nodes, so get all
nodes on the network to relay I2P addresses to help with propagation,
similarly to what we do with Tor addresses.
2021-06-10 14:16:41 +02:00
MarcoFalke
fa7d71f270 test: Run pep-8 on touched test
Can be reviewed with --ignore-all-space
2021-06-10 13:39:24 +02:00
MarcoFalke
fab7e99c2a test: Use MiniWallet in test_no_inherited_signaling RBF test 2021-06-10 13:38:56 +02:00
MarcoFalke
fab871f649 test: Remove unused generate() from test
This is already done by the test framework in setup_nodes()
2021-06-10 13:38:18 +02:00
MarcoFalke
faff3f35b7 test: Add txin.sequence option to MiniWallet 2021-06-10 13:38:10 +02:00
fanquake
ef8f2966ac Merge bitcoin/bitcoin#22084: package testmempoolaccept followups
ee862d6efb MOVEONLY: context-free package policies (glozow)
5cac95cd15 disallow_mempool_conflicts -> allow_bip125_replacement and check earlier (glozow)
e8ecc621be [refactor] comment/naming improvements (glozow)
7d91442461 [rpc] reserve space in txns (glozow)
6c5f19d9c4 [package] static_assert max package size >= max tx size (glozow)

Pull request description:

  various followups from #20833

ACKs for top commit:
  jnewbery:
    utACK ee862d6efb
  ariard:
    Code Review ACK ee862d6

Tree-SHA512: 96ecb41f7bbced84d4253070f5274b7267607bfe4033e2bb0d2f55ec778cc41e811130b6321131e0418b5835894e510a4be8a0f822bc9d68d9224418359ac837
2021-06-10 19:09:54 +08:00
MarcoFalke
e87fbee402 Merge bitcoin/bitcoin#22208: doc: Update REVIEWERS
3636d9be8f Update REVIEWERS: I've found that I keep track of PRs in need of review without the need for DrahtBot's automated notification :) (practicalswift)

Pull request description:

  Update `REVIEWERS`: I've found that I keep track of PRs in need of review without the need for DrahtBot's automated notification :)

ACKs for top commit:
  MarcoFalke:
    ACK 3636d9be8f

Tree-SHA512: 8602bb7ab10bd04ab8e65b63708a2bb33fecb1b7494ebc59030976fd3eb53ec2d57a3ed96fd9170e2de7771e630d9571e73f7d3b939452cf6bf9e75c3bfaa282
2021-06-10 11:25:33 +02:00
practicalswift
3636d9be8f Update REVIEWERS: I've found that I keep track of PRs in need of review without the need for DrahtBot's automated notification :) 2021-06-10 09:00:05 +00:00
fanquake
aa80b5759d scripts: check macOS SDK version is set
Clangs Darwin driver should infer the SDK version used during compilation, and
forward that through to the linker. Add a check that this has been done, and the
expected SDK version is set.

Should help prevent issues like #21771 in future.
2021-06-10 15:43:50 +08:00
MarcoFalke
356f421fb0 Merge bitcoin/bitcoin#22204: doc: remove obsolete okSafeMode RPC guideline from developer notes
6780a095d8 doc: remove obsolete `okSafeMode` RPC guideline from developer notes (Sebastian Falbesoner)

Pull request description:

  Since the flag has been removed from the RPC command table in commit ec6902d0ea (PR #11179), this guideline is not relevant anymore and can be removed.

ACKs for top commit:
  MarcoFalke:
    ACK 6780a095d8. Also, safe mode was removed completely in commit 2ae705d841

Tree-SHA512: 2a6b002ae302e979ce403171b79a892e32f5083792c3b0b8204edb5eb08c6b24ab77bbeeae0e3bb6d6564a6f1678cfce00eb7b5b82063b7741f89a96b0c0aef3
2021-06-10 07:38:20 +02:00
fanquake
c972345bac scripts: check minimum required Windows version is set
We use linker flags (-Wl,--major/minor-subsystem-version) to set the
minimum required version of Windows needed to run our binaries. This
adds a sanity check that the version is being set as expected.
2021-06-10 10:40:53 +08:00
fanquake
29615aef52 scripts: check minimum required macOS vesion is set
We use a compile flag (-mmacosx-version-min) to set the minimum required
version of macOS needed to run our binaries. This adds a sanity check
that the version is being set as expected.
2021-06-10 10:40:53 +08:00
fanquake
8732f7b6c9 scripts: LIEF 0.11.5 2021-06-10 10:40:52 +08:00
fanquake
92584d29ce Merge bitcoin/bitcoin#22186: build, qt: Fix compiling qt package in depends with GCC 11
d1d1cc9831 build, qt: Fix compiling qt package in depends with GCC 11 (Hennadii Stepanov)

Pull request description:

  The `qt` package in depends fails to compile with GCC 11 due to the missed `<limits>` headers.

  See: https://bugreports.qt.io/browse/QTBUG-90395

  Affected systems:
  - Ubuntu 21.04 + GCC 11.1.0
  - Fedora 34 + GCC 11.1.1

  Ubuntu 21.04 build log excerpt:
  ```
  Configuring qt...
  Creating qmake...
  gmake[1]: Entering directory '/home/hebasto/bitcoin/depends/work/build/x86_64-pc-linux-gnu/qt/5.12.11-1ff5c6c1f55/qtbase/qmake'
  In file included from /home/hebasto/bitcoin/depends/work/build/x86_64-pc-linux-gnu/qt/5.12.11-1ff5c6c1f55/qtbase/include/QtCore/qendian.h:1,
                   from /home/hebasto/bitcoin/depends/work/build/x86_64-pc-linux-gnu/qt/5.12.11-1ff5c6c1f55/qtbase/src/corelib/codecs/qutfcodec.cpp:43:
  /home/hebasto/bitcoin/depends/work/build/x86_64-pc-linux-gnu/qt/5.12.11-1ff5c6c1f55/qtbase/include/QtCore/../../src/corelib/global/qendian.h: In static member function ‘static constexpr QSpecialInteger<S> QSpecialInteger<S>::max()’:
  /home/hebasto/bitcoin/depends/work/build/x86_64-pc-linux-gnu/qt/5.12.11-1ff5c6c1f55/qtbase/include/QtCore/../../src/corelib/global/qendian.h:331:35: error: ‘numeric_limits’ is not a member of ‘std’
    331 |     { return QSpecialInteger(std::numeric_limits<T>::max()); }
        |                                   ^~~~~~~~~~~~~~
  /home/hebasto/bitcoin/depends/work/build/x86_64-pc-linux-gnu/qt/5.12.11-1ff5c6c1f55/qtbase/include/QtCore/../../src/corelib/global/qendian.h:331:54: error: ‘::max’ has not been declared; did you mean ‘std::max’?
    331 |     { return QSpecialInteger(std::numeric_limits<T>::max()); }
        |                                                      ^~~
        |                                                      std::max
  ```

ACKs for top commit:
  fanquake:
    ACK d1d1cc9831

Tree-SHA512: 2dd643efc0aefc492f9565c0900ba0f1657c016bc4a44792f85478b9fc6e0e4ecad78847114ef6ec702d2de4cdbc3f657e9b96634ea58f42b6cc98dfb5e09eab
2021-06-10 09:45:05 +08:00
Sebastian Falbesoner
6780a095d8 doc: remove obsolete okSafeMode RPC guideline from developer notes
Since the flag has been removed from the RPC command table in commit
ec6902d0ea (PR #11179), this guideline
is not relevant anymore and can be removed.
2021-06-10 02:02:29 +02:00
MarcoFalke
fa9ebedec3 Reject invalid coin height and output index when loading assumeutxo 2021-06-09 22:20:01 +02:00
MarcoFalke
46424e943c Merge bitcoin/bitcoin#22202: test: Add temporary coinstats suppressions
faca40ec68 test: Add temporary coinstats suppressions (MarcoFalke)

Pull request description:

  Needed for my fuzzer to continue to run

ACKs for top commit:
  practicalswift:
    cr ACK faca40ec68: suppression looks necessary (temporarily)

Tree-SHA512: 5bdff9a24a60546cfe31e775fa2aa5e238aefda2ed2604bef18c82b1b80c51ca3cbe058d6c7988fa75305258b70076036a3e430b9b7de13a111309fa7a66745b
2021-06-09 22:14:54 +02:00
MarcoFalke
fa72fce7c9 test: Use ConnmanTestMsg from test lib in denialofservice_tests 2021-06-09 21:00:48 +02:00
MarcoFalke
faca40ec68 test: Add temporary coinstats suppressions 2021-06-09 19:48:45 +02:00
MarcoFalke
ca424e242a Merge bitcoin/bitcoin#22200: zmq: use std::string in zmqError()
3f05a9e681 zmq: use msg: prefix over errno= in zmqError (fanquake)
9a7cb57bbc zmq: use std::string in zmqError() (fanquake)

Pull request description:

  This is two minor changes. The first is to change `zmqError` to take a `const std::string&` instead of a `const char*`. The second is to change the second portion of `zmqError` to print `msg: message` rather than `errno=message`, given that `zmq_strerror` returns a message. To me, this seems more readable / useful than output like: `Error: Unable to initialize context errno=No such file or directory`.

ACKs for top commit:
  practicalswift:
    cr ACK 3f05a9e681
  instagibbs:
    utACK 3f05a9e681
  theStack:
    Code-Review ACK 3f05a9e681

Tree-SHA512: 197cf381e8b3ced271d0e575e0c6d8e5e9ed93c4b284338b17873c5232eaabe64d6c4b66e1aeb5e76befc89e316abae2b28b7fd760f178481d7b9f4e3f85da67
2021-06-09 19:32:34 +02:00
Ben Woosley
4e44f5bac4 test: Correct outstanding -Werror=sign-compare errors
e.g.:
In file included from /usr/local/include/boost/test/test_tools.hpp:46:
/usr/local/include/boost/test/tools/old/impl.hpp:107:17: error: comparison of integers of different signs: 'const unsigned int' and 'const int' [-Werror,-Wsign-compare]
    return left == right;
           ~~~~ ^  ~~~~~
/usr/local/include/boost/test/tools/old/impl.hpp:130:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl<unsigned int, int>' requested here
        return equal_impl( left, right );
               ^
/usr/local/include/boost/test/tools/old/impl.hpp:145:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::call_impl<unsigned int, int>' requested here
        return call_impl( left, right, left_is_array() );
               ^
/usr/local/include/boost/test/tools/old/impl.hpp:92:50: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::operator()<unsigned int, int>' requested here
BOOST_PP_REPEAT( BOOST_TEST_MAX_PREDICATE_ARITY, IMPL_FRWD, _ )
                                                 ^
/usr/local/include/boost/preprocessor/repetition/repeat.hpp:30:26: note: expanded from macro 'BOOST_PP_REPEAT'
                         ^
/usr/local/include/boost/preprocessor/cat.hpp:22:32: note: expanded from macro 'BOOST_PP_CAT'
                               ^
/usr/local/include/boost/preprocessor/cat.hpp:29:34: note: expanded from macro 'BOOST_PP_CAT_I'
                                 ^
<scratch space>:153:1: note: expanded from here
BOOST_PP_REPEAT_1
^
test/streams_tests.cpp:122:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned int, int>' requested here
    BOOST_CHECK_EQUAL(varint, 54321);
    ^

/usr/local/include/boost/test/tools/old/impl.hpp:107:17: error: comparison of integers of different signs: 'const unsigned long long' and 'const long' [-Werror,-Wsign-compare]
    return left == right;
           ~~~~ ^  ~~~~~
/usr/local/include/boost/test/tools/old/impl.hpp:130:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl<unsigned long long, long>' requested here
        return equal_impl( left, right );
               ^
/usr/local/include/boost/test/tools/old/impl.hpp:145:16: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::call_impl<unsigned long long, long>' requested here
        return call_impl( left, right, left_is_array() );
               ^
/usr/local/include/boost/test/tools/old/impl.hpp:92:50: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::equal_impl_frwd::operator()<unsigned long long, long>' requested here
BOOST_PP_REPEAT( BOOST_TEST_MAX_PREDICATE_ARITY, IMPL_FRWD, _ )
                                                 ^
/usr/local/include/boost/preprocessor/repetition/repeat.hpp:30:26: note: expanded from macro 'BOOST_PP_REPEAT'
                         ^
/usr/local/include/boost/preprocessor/cat.hpp:22:32: note: expanded from macro 'BOOST_PP_CAT'
                               ^
/usr/local/include/boost/preprocessor/cat.hpp:29:34: note: expanded from macro 'BOOST_PP_CAT_I'
                                 ^
<scratch space>:161:1: note: expanded from here
BOOST_PP_REPEAT_1
^
test/serfloat_tests.cpp:41:5: note: in instantiation of function template specialization 'boost::test_tools::tt_detail::check_frwd<boost::test_tools::tt_detail::equal_impl_frwd, unsigned long long, long>' requested here
    BOOST_CHECK_EQUAL(TestDouble(std::numeric_limits<double>::infinity()), 0x7ff0000000000000);
    ^
2021-06-09 12:34:21 -04:00
Carl Dong
4cc35daed5 Rewrite guix-{attest,verify} for new hier 2021-06-09 11:06:00 -04:00
Carl Dong
28a9c9b839 Make SHA256SUMS fragment right after build 2021-06-09 11:06:00 -04:00
fanquake
3f05a9e681 zmq: use msg: prefix over errno= in zmqError
zmq_strerror() converts the passed errno into a description, meaning
currently you have output like: "errno=No such file or directory".

Using msg: would seem to make more sense here.
2021-06-09 19:25:13 +08:00
fanquake
9a7cb57bbc zmq: use std::string in zmqError() 2021-06-09 19:25:07 +08:00
Samuel Dobson
93e38d5c06 Merge bitcoin/bitcoin#22173: wallet: Do not load external signers wallets when unsupported
e60cd26ad4 Do not load external signers wallets when unsupported (Andrew Chow)

Pull request description:

  When external signer support is not compiled, do not load external signer wallets.

  Alternative to #22168.

ACKs for top commit:
  promag:
    Tested ACK e60cd26ad4.
  meshcollider:
    Code review ACK e60cd26ad4

Tree-SHA512: aed2d0038f448c2f89c6b48f412b106e63c9ed20e748e69aae21fb58c33fc7e4fa73375a52372c73788669eb2b968a8da6b022c65658fa4484f5bbcf205b1b15
2021-06-09 23:12:40 +12:00
Samuel Dobson
69577a27ab Merge bitcoin/bitcoin#21944: wallet: Fix issues when walletdir is root directory
d44a261acf Fix issues when `walletdir` is root directory (unknown)

Pull request description:

  + Remove one character less from wallet path

  + After testing lot of random strings with special chars in `wallet_name`, I found that the issue was not related to special characters in the name. Reviewing PR https://github.com/bitcoin/bitcoin/pull/21907 helped me resolve the issue.

  **Real issue**: If the path mentioned in `walletdir` is a root directory, first character of the wallet name or path is removed

  **Solution**: `if` statement to check `walletdir` is a root directory

  Fixes: https://github.com/bitcoin/bitcoin/issues/21510 https://github.com/bitcoin/bitcoin/issues/21501
  Related PR: https://github.com/bitcoin/bitcoin/pull/20080

  Consider the wallet directories `w1` and `w2` saved in `D:\`. Run `bitcoind.exe -walletdir=D:\`, Results for `bitcoin-cli.exe listwalletdir`:

  Before this PR:

  ```

  {
    "wallets": [
      {
        "name": "1"
      },
      {
        "name": "2"
      }
    ]
  }

  ```

  After this PR:
  ```
    "wallets": [
      {
        "name": "w1"
      },
      {
        "name": "w2"
      }
    ]
  }

  ```

ACKs for top commit:
  ryanofsky:
    Code review ACK d44a261acf
  meshcollider:
    utACK d44a261acf

Tree-SHA512: b09b00f727407e3771c8694861dae1bfd29d97a0d51ddcb5d9c0111dc618b3fff2f75829cbb4361c54457ee564e94fcefd9e2928262a1c918a2b6bbad724eb55
2021-06-09 22:53:36 +12:00
Samuel Dobson
58f8b156ed Merge bitcoin/bitcoin#22008: wallet: Cleanup and refactor CreateTransactionInternal
96c2c9520e scripted-diff: Rename SelectCoinsMinConf to AttemptSelection (Andrew Chow)
b583f73354 Move vin filling to before final fee setting (Andrew Chow)
d39cac0547 Set m_subtract_fee_outputs during recipients vector loop (Andrew Chow)
364e0698a5 Move variable initializations to where they are used (Andrew Chow)
32ab430651 Move recipients vector checks to beginning of CreateTransaction (Andrew Chow)
cd1d6d3324 Rename nSubtractFeeFromAmount in CreateTransaction (Andrew Chow)
dac21c793f Rename nValue and nValueToSelect (Andrew Chow)
d2aee3bbc7 Remove extraneous scope in CreateTransactionInternal (Andrew Chow)
b2995963b5 Move cs_wallet lock in CreateTransactionInternal to top of function (Andrew Chow)

Pull request description:

  #17331 did some refactors and cleanup of `CreateTransactionInternal` to make it easier to understand, however it is still a bit convoluted even though it doesn't have to be. This PR does additional cleanup and refactoring to `CreateTransactionInternal` so that it is easier to understand. Some unnecessary code was removed, some variables moved around to where they matter, and several indents removed.

ACKs for top commit:
  glozow:
    reACK 96c2c95
  ryanofsky:
    Code review ACK 96c2c9520e also acked previously (was reverted).
  meshcollider:
    re-utACK 96c2c9520e

Tree-SHA512: 3dba67ed436968a07bfd82d435d566ad74e116c6e50ac9baed7144a46ad5c0f630b1ba59d91e8e8972ac2af559d7c0576f0560f09684d2ab20fad6689902866f
2021-06-09 22:37:34 +12:00
fanquake
fe511454e1 Merge bitcoin/bitcoin#22174: build, qt: Fix libraries linking order for Linux hosts
a8bd5ea017 build, qt: Fix libraries linking order for Linux hosts (Hennadii Stepanov)

Pull request description:

  `-lxcb-shm` should follow `-lxcb` when linking libraries for Linux hosts.

  Fixes #22105.

ACKs for top commit:
  prayank23:
    ACK a8bd5ea017
  fanquake:
    ACK a8bd5ea017

Tree-SHA512: 6ea2a1ba5440dd01e64428e5c3aa3e7a6e1380b723071ad38a0516b23b56dc252cb2a6544e014b13a0b2f89b3fe45c04a82e498669b5cc74a5f9b70d9c270926
2021-06-09 15:35:43 +08:00
Samuel Dobson
68a89d7a46 Merge bitcoin-core/gui#4: UI external signer support (e.g. hardware wallet)
1c4b456e1a gui: send using external signer (Sjors Provoost)
24815c6309 gui: wallet creation detects external signer (Sjors Provoost)
3f845ea299 node: add externalSigners to interface (Sjors Provoost)
62ac119f91 gui: display address on external signer (Sjors Provoost)
450cb40a34 wallet: add displayAddress to interface (Sjors Provoost)
eef8d64529 gui: create wallet with external signer (Sjors Provoost)
6cdbc83e93 gui: add external signer path to options dialog (Sjors Provoost)

Pull request description:

  Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).

  This PR adds GUI support for external signers, based on the since merged bitcoin/bitcoin#16546 (RPC).

  The UX isn't amazing - especially the blocking calls - but it works.

  First we adds a GUI setting for the signer script (e.g. path to HWI):

  <img width="625" alt="Schermafbeelding 2019-08-05 om 19 32 59" src="https://user-images.githubusercontent.com/10217/62483415-e1ff1680-b7b7-11e9-97ca-8d2ce54ca1cb.png">

  Then we add an external signer checkbox to the wallet creation dialog:

  <img width="374" alt="Schermafbeelding 2019-11-07 om 19 17 23" src="https://user-images.githubusercontent.com/10217/68416387-b57ee000-0194-11ea-9730-127d60273008.png">

  It's checked by default if HWI detects a device. It also grabs the name. It then creates a fresh wallet and imports the keys.

  You can verify an address on the device (blocking...):
  <img width="673" alt="Schermafbeelding 2019-08-05 om 19 29 22" src="https://user-images.githubusercontent.com/10217/62483560-43bf8080-b7b8-11e9-9902-8a036116dc4b.png">

  Sending, including coin selection, Just Works(tm) as long the device is present.

  ~External signer support is enabled by default when the GUI is configured and Boost::Process is present.~

  External signer support remains disabled by default, see https://github.com/bitcoin/bitcoin/pull/21935.

ACKs for top commit:
  achow101:
    Code Review ACK 1c4b456e1a
  hebasto:
    ACK 1c4b456e1a, tested on Linux Mint 20.1 (Qt 5.12.8) with HWW `2.0.2-rc.1`.
  promag:
    Tested ACK 1c4b456e1a but rebased with e033ca1379, with HWI 2.0.2, with Nano S and Nano X.
  meshcollider:
    re-code-review ACK 1c4b456e1a

Tree-SHA512: 3503113c5c69d40adb6ce364d8e7cae23ce82d032a00474ba9aeb6202eb70f496ef4a6bf2e623e5171e524ad31ade7941a4e0e89539c64518aaec74f4562d86b
2021-06-09 18:59:59 +12:00
fanquake
0a5723beea macdeploy: cleanup .temp.dmg if present 2021-06-09 12:33:39 +08:00
fanquake
ecffe8689d macdeploy: remove qt4 related code 2021-06-09 12:33:38 +08:00
fanquake
639f064253 macdeploy: select the plugins we need, rather than excluding those we don't 2021-06-09 12:33:33 +08:00
fanquake
3d26b6b9e9 macdeploy: fix framework printing when passing -verbose 2021-06-09 12:16:47 +08:00
fanquake
dca6c90329 macdeploy: remove unused plistlib import
Unused since #20422, see: https://github.com/bitcoin/bitcoin/pull/20422#discussion_r534207899
2021-06-09 12:16:47 +08:00
fanquake
7cac26246a Merge bitcoin/bitcoin#22075: guix: Misc leftover usability improvements
108a6be92a guix: Check for disk space availability before building (Carl Dong)
d7dec89091 guix: Remove dest if OUTDIR mv fails (Carl Dong)

Pull request description:

  There seems to be some corner cases that can be hit when guix scripts unexpectedly fail in the middle of operation, see: https://gnusha.org/bitcoin-builds/2021-05-24.log

  - Perform an early disk space check for `guix-build`
  - Overwrite existing output directory after a successful build (the existing one might be malformed), and cleanup output directory if the `mv` somehow fails

ACKs for top commit:
  laanwj:
    Tested ACK 108a6be92a
  achow101:
    ACK 108a6be92a

Tree-SHA512: cf6438317da40bf55714cd2d8cce859b3d435cc66cabefe8d4a53552d7880966acfe84ffe8fadf1c80e368ae6b037992258a6d409df85ffc6ce8bf780e98e2e5
2021-06-09 09:09:38 +08:00
Andrew Chow
683d197970 Use latest signapple commit
Update gitian and guix to use the same latest signapple commit
2021-06-08 16:46:56 -04:00
Sebastian Falbesoner
d6d2ab9845 test: MiniWallet: fix fee calculation for P2PK and check tx vsize 2021-06-08 19:38:43 +02:00
Sebastian Falbesoner
ce024b1c0e test: MiniWallet: force P2PK signature to have fixed size (71 bytes)
In order to enable exact fee calculation for transactions that spend
P2PK outputs in the MiniWallet, we enforce the created signatures to
have a fixed length (>49.89% probability) by default. With that it is
easier to check the created transactions vsize and avoid flaky tests
that would appear whenever the signatures R- or S-values are smaller
(due to leading zero bytes).

Note that to get the total scriptSig size one has to add another
2 bytes, as there is also the OP_PUSHx instruction on the front and
the sighash type byte on the back, leading to a final scriptSig size
of 73 bytes.
2021-06-08 19:38:17 +02:00
MarcoFalke
82bc7faec8 Merge bitcoin/bitcoin#21946: Document and test lack of inherited signaling in RBF policy
2eb0eeda39 validation: document lack of inherited signaling in RBF policy (Antoine Riard)
906b6d9da6 test: Extend feature_rbf.py with no inherited signaling (Antoine Riard)

Pull request description:

  Contrary to BIP125 or other full-node implementation (e.g btcd), Bitcoin Core's mempool policy doesn't implement inherited signaling.

  This PR documents our mempool behavior on this and add a test demonstrating the case.

ACKs for top commit:
  jonatack:
    ACK 2eb0eeda39
  benthecarman:
    ACK 2eb0eeda39

Tree-SHA512: d41453d3b49bae3c1eb532a968f43bc047084913bd285929d4d9cba142777ff2be38163d912e28dfc635f4ecf446de68effad799c6e71be52f81e83410c712fb
2021-06-08 17:00:28 +02:00
MarcoFalke
45a8b01fad Merge bitcoin/bitcoin#22187: test: Add sync_blocks in wallet_orphanedreward.py
7a681d61b0 Add sync_blocks in wallet_orphanedreward.py. (Daniel Kraft)

Pull request description:

  Add an explicit `sync_blocks` call in `wallet_orphanedreward.py`, which was missing and could lead to intermittent failures of the test due to race conditions.

  This will presumably fix #22181.

ACKs for top commit:
  MarcoFalke:
    review ACK 7a681d61b0

Tree-SHA512: bb226c31bf3f2e7c52beb829d7b67496e5b38781245db5f9184e3f28c93ac3aa4d21fcf5bf3055e79d384cfd0ed916e79dccb3d77486e86fe1fedb5e35f894ad
2021-06-08 10:39:15 +02:00
Daniel Kraft
7a681d61b0 Add sync_blocks in wallet_orphanedreward.py.
Add an explicit sync_blocks call in wallet_orphanedreward.py, which was
missing and could lead to intermittent failures of the test due to
race conditions.

This will presumably fix https://github.com/bitcoin/bitcoin/issues/22181.
2021-06-08 09:52:10 +02:00
MarcoFalke
76d4018aa5 Merge bitcoin/bitcoin#22180: fuzz: Increase branch coverage of the float fuzz target
fa13f34bf3 fuzz: Increase branch coverage of the float fuzz target (MarcoFalke)
fad0c58c3e fuzz: Remove confusing return keyword from CallOneOf (MarcoFalke)

Pull request description:

  Currently the branch coverage for the float fuzz target is only 50% : https://marcofalke.github.io/btc_cov/fuzz.coverage/src/test/fuzz/float.cpp.gcov.html

  This is caused by the Fuzzed Data Provider only picking "nice" floats.

ACKs for top commit:
  practicalswift:
    cr ACK fa13f34bf3: patch looks correct

Tree-SHA512: 326822515e9a1c77647d41eab9a96185a3b320914d9264730fa72ffb76c2bf3dc5bf72cf6cd9beef14f4f032358d76a976860bf3e2418ae61943cf926c0ea086
2021-06-08 09:19:16 +02:00
Hennadii Stepanov
d1d1cc9831 build, qt: Fix compiling qt package in depends with GCC 11 2021-06-08 04:16:36 +03:00
Sebastian Falbesoner
01eedf3821 test: doc: improve doc for chain_transaction() helper
Change to docstring format and describe the functions
purpose, its parameters and return value in more detail.
2021-06-08 00:21:37 +02:00
Sebastian Falbesoner
6e63e366d6 test: refactor: dedup utility function chain_transaction() 2021-06-08 00:21:37 +02:00
Hennadii Stepanov
e638acf697 Merge bitcoin-core/gui#164: Handle peer addition/removal in a right way
ecbd911538 qt: Handle peer addition/removal in a right way (Hennadii Stepanov)
1b66f6e556 qt: Drop PeerTablePriv class (Hennadii Stepanov)
efb7e5aa96 qt, refactor: Use default arguments for overridden functions (Hennadii Stepanov)

Pull request description:

  This PR makes `PeerTableModel` handle a peer addition/removal in a right way. See:
  - https://doc.qt.io/qt-5/model-view-programming.html#inserting-and-removing-rows
  - https://doc.qt.io/qt-5/model-view-programming.html#resizable-models

  Fixes #160.

  Fixes #191.

ACKs for top commit:
  jarolrod:
    re-ACK ecbd911
  promag:
    reACK ecbd911538 just improvements to the comment since last review.

Tree-SHA512: 074935d67f78561724218e8b33822e2de16749f873c29054926b720ffcd642f08249a222b563983cf65a9b716290aa14e2372c47fc04e5f401f759db25ca710f
2021-06-07 22:13:05 +03:00
W. J. van der Laan
359f72105b Merge bitcoin/bitcoin#21573: Update libsecp256k1 subtree to latest master
5c7ee1b2da libsecp256k1 no longer has --with-bignum= configure option (Pieter Wuille)
bdca9bcb6c Squashed 'src/secp256k1/' changes from 3967d96bf1..efad3506a8 (Pieter Wuille)
cabb566123 Disable certain false positive warnings for libsecp256k1 msvc build (Pieter Wuille)

Pull request description:

  This updates our src/secp256k1 subtree to the latest upstream master. The changes include:

  * The introduction of safegcd-based modular inverses, reducing ECDSA signing time by 25%-30% and ECDSA verification time by 15%-17%.
    * [Original paper](https://gcd.cr.yp.to/papers.html) by Daniel J. Bernstein and Bo-Yin Yang
    * [Implementation](https://github.com/bitcoin-core/secp256k1/pull/767) by Peter Dettman; [final](https://github.com/bitcoin-core/secp256k1/pull/831) version
    * [Explanation](https://github.com/bitcoin-core/secp256k1/blob/master/doc/safegcd_implementation.md) of the algorithm using Python snippets
    * [Analysis](https://github.com/sipa/safegcd-bounds) of the maximum number of iterations the algorithm needs
    * [Formal proof in Coq](https://medium.com/blockstream/a-formal-proof-of-safegcd-bounds-695e1735a348) by Russell O'Connor, for a high-level equivalent algorithm
  * Removal of libgmp as an (optional) dependency (which wasn't used in the Bitcoin Core build)
  * CI changes (Travis -> Cirrus)
  * Build system improvements

ACKs for top commit:
  laanwj:
    Tested ACK 5c7ee1b2da

Tree-SHA512: ad8ac3746264d279556a4aa7efdde3733e114fdba8856dd53218588521f04d83950366f5c1ea8fd56329b4c7fe08eedf8e206f8f26dbe3f0f81852e138655431
2021-06-07 17:05:11 +02:00
Prayank
6d7e46ce23 Remove Warning:
+ Remove `Warning:` from warning message printed for unknown new rules
+ Change warning message in test

Author:    Prayank <prayank@tutanota.de>
2021-06-07 20:19:18 +05:30
Hennadii Stepanov
ecbd911538 qt: Handle peer addition/removal in a right way
This change fixes a bug when a multiple rows selection gets inconsistent
after a peer addition/removal.
2021-06-07 17:37:40 +03:00
W. J. van der Laan
3c393ef9e1 Merge bitcoin/bitcoin#22149: test: Add temporary logging to debug #20975
faa94961d6 test: Add temporary logging to debug #20975 (MarcoFalke)

Pull request description:

  to be reverted after a fix

ACKs for top commit:
  laanwj:
    Code review ACK faa94961d6

Tree-SHA512: 1f3103fcf4cad0af54e26c4d257bd824b128b5f2d2b81c302e861a829fd55d6a099fa476b79b30a71fe98975ae604b9e3ff31fd48a51d442389a9bd515e60ba0
2021-06-07 16:03:45 +02:00
Vasil Dimov
00b875ba94 addrman: remove invalid addresses when unserializing
The Tor v2 addresses, left over from when Tor v2 was supported will be
unserialized as a dummy, invalid `::` (all zeros) IPv6 address. Remove
them so that they do not take up space in addrman.
2021-06-07 14:42:11 +02:00
MarcoFalke
fa13f34bf3 fuzz: Increase branch coverage of the float fuzz target 2021-06-07 13:41:14 +02:00
MarcoFalke
fad0c58c3e fuzz: Remove confusing return keyword from CallOneOf
The return type is already enforced to be void by the
ternary operator:

./test/fuzz/util.h:47:25: error: right operand to ? is void, but left operand is of type *OTHER_TYPE*
    ((i++ == call_index ? callables() : void()), ...);
                        ^ ~~~~~~~~~~~   ~~~~~~
2021-06-07 13:41:13 +02:00
MarcoFalke
912cb59490 Merge bitcoin/bitcoin#21795: fuzz: Terminate immediately if a fuzzing harness tries to perform a DNS lookup (belt and suspenders)
3737d35fee fuzz: Terminate immediately if a fuzzing harness ever tries to perform a DNS lookup (belts and suspenders) (practicalswift)

Pull request description:

  Terminate immediately if a fuzzing harness tries to perform a DNS lookup (belt and suspenders).

  Obviously this _should_ never happen, but if it _does_ happen we want immediate termination instead of a DNS lookup :)

ACKs for top commit:
  MarcoFalke:
    review ACK 3737d35fee

Tree-SHA512: 51cd2d32def7f9f052e02f99c354656af1f807cc9fdf592ab765e620bfe660f1ed26e0484763f94aba650424b44959eafaf352bfd0f81aa273e350510e97356e
2021-06-07 09:03:44 +02:00
fanquake
791f985a60 Merge bitcoin/bitcoin#22137: util: Properly handle -noincludeconf on command line (take 2)
fa910b4765 util: Properly handle -noincludeconf on command line (MarcoFalke)

Pull request description:

  Before:

  ```
  $ ./src/qt/bitcoin-qt -noincludeconf
  (memory violation, can be observed with valgrind or similar)
  ```

  After:

  ```
  $ ./src/qt/bitcoin-qt -noincludeconf
  (passes startup)
  ```

  Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34884

ACKs for top commit:
  practicalswift:
    cr ACK fa910b4765: patch looks correct
  ryanofsky:
    Code review ACK fa910b4765. Nice cleanups!

Tree-SHA512: 5dfad82a78bca7a9a6bcc6aead2d7fbde166a09a5300a82f80dd1aee1de00e070bcb30b7472741a5396073b370898696e78c33038f94849219281d99358248ed
2021-06-07 13:20:57 +08:00
fanquake
260b1d74fe Merge bitcoin/bitcoin#22092: test: convert documentation into type annotations
68ace23fa3 test: convert docs into type annotations in test_framework/test_node.py (fanquake)
8bfcba36db test: convert docs into type annotations in test_framework/wallet.py (fanquake)
b043ca8e8b test: convert docs into type annotations in test_framework/util.py (fanquake)

Pull request description:

  Rather than having function types exist as documentation, make them type annotations, which enables more `mypy` checking.

ACKs for top commit:
  instagibbs:
    utACK 68ace23fa3

Tree-SHA512: b705f26b48baabde07b9b2c0a8d547b4dcca291b16eaf5ac70462bb3a1f9e9c2783d93a9d4290889d0cbb3f7db3671446754411a1f498b265d336f6ff33478df
2021-06-07 13:05:56 +08:00
fanquake
1cc123f405 Merge bitcoin/bitcoin#22169: p2p, rpc, fuzz: various tiny follow-ups
c274574458 p2p, rpc, fuzz: various tiny follow-ups (Jon Atack)

Pull request description:

  - p2p: pass `Span` by value per https://github.com/bitcoin/bitcoin/pull/22143#issuecomment-853953438 as a follow-up to 8be56f0f8e
  - rpc: remove duplicate `CAddress` constructor per https://github.com/bitcoin/bitcoin/pull/22043#discussion_r638535703
  - fuzz: rename 3 fuzz targets changed in eba9a94b9f back to their original names per https://github.com/bitcoin-core/qa-assets/pull/63#issuecomment-855281865

ACKs for top commit:
  MarcoFalke:
    cr ACK c274574458
  practicalswift:
    cr ACK c274574458: patch looks correct
  jarolrod:
    ACK  c274574458

Tree-SHA512: 3672b210d30b3a91f3a6455005e4d3cb1f89621820c417c645d24b06e53459440122a1f75758e0e04c3d04eff9d6f88ef62865216aa3e42301c6df783f7c0b4a
2021-06-07 13:03:53 +08:00
Hennadii Stepanov
a8bd5ea017 build, qt: Fix libraries linking order for Linux hosts
This change fixes configuring with Qt on Alpine Linux.
2021-06-06 23:25:07 +03:00
Andrew Chow
e60cd26ad4 Do not load external signers wallets when unsupported
When external signer support is not compiled, do not load external
signer wallets.
2021-06-06 13:52:27 -04:00
Jon Atack
49938eee9c doc: update tor.md with removal of tor v2 support 2021-06-06 19:05:01 +02:00
Jon Atack
c274574458 p2p, rpc, fuzz: various tiny follow-ups 2021-06-06 15:49:22 +02:00
Hennadii Stepanov
f507681baa qt: Connect WalletView signal to BitcoinGUI slot directly
This change removes redundant intermediate WalletFrame connections.

This commit does not change behavior.
2021-06-06 01:04:15 +03:00
Hennadii Stepanov
bd50ff9290 qt: Drop redundant OverviewPage::handleOutOfSyncWarningClicks slot
This change makes a connection directly to the signal that was emitted
in the removed slot.

This commit does not change behavior.
2021-06-06 01:04:15 +03:00
Hennadii Stepanov
793f19599b qt: Drop redundant WalletView::requestedSyncWarningInfo slot
This change makes a connection directly to the signal that was emitted
in the removed slot.

This commit does not change behavior.
2021-06-06 01:04:14 +03:00
Hennadii Stepanov
e033ca1379 Merge bitcoin-core/gui#29: refactor: Optimize signal-slot connections logic
62cb8d98d2 qt: Drop BitcoinGUI* WalletFrame data member (Hennadii Stepanov)
f73e5c972a qt: Move CreateWalletActivity connection from WalletFrame to BitcoinGUI (Hennadii Stepanov)
20e2e24e90 qt: Move WalletView connections from WalletFrame to BitcoinGUI (Hennadii Stepanov)

Pull request description:

  This PR:
  - implements an idea from https://github.com/bitcoin/bitcoin/pull/17937#issuecomment-575991765
  - simplifies `WalletFrame` class interface
  - as a side effect, removes `bitcoingui` -> `walletframe` -> `bitcoingui` circular dependency
  - is an alternative to https://github.com/bitcoin/bitcoin/pull/17500

ACKs for top commit:
  promag:
    Tested ACK 62cb8d98d2 on macos 11.2.3 with depends build.
  jarolrod:
    ACK 62cb8d98d2

Tree-SHA512: 633b526a8499ba9ab4b16928daf4de4f6d610284bb9fa51891cad35300a03bde740df3466a71b46e87a62121330fcc9e606eac7666ea5e45fa6d5785b60dcbbd
2021-06-06 01:02:54 +03:00
Hennadii Stepanov
21d87bbdfd Merge bitcoin-core/gui#256: Save/restore column sizes of the tables in the Peers tab
fb1b1e0f3e qt: Save/restore column sizes of the tables in the Peers tab (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  jonatack:
    ACK fb1b1e0f3e code review, debug-built and tested
  jarolrod:
    ACK fb1b1e0f3e

Tree-SHA512: f93495ecd13e4202aba61b407fffbeec855f5b0c1cc027197c78edddd7d11c87ebdb0fcb1daac242f0407323b31f4e7e0313bd76113a5241e4c868a8829af20a
2021-06-05 23:02:32 +03:00
Hennadii Stepanov
916f45eba5 Merge bitcoin-core/gui#329: Make console buttons look clickable
8b419b5163 qt: make console buttons look clickable (Jarol Rodriguez)

Pull request description:

  On master, for macOS, the console buttons' hitboxes are quite small. This makes clicking on the button with your mouse a little more tedious than it should be. The Issue is related to recent versions of Qt (>5.9.8) not playing so nice on macOS when there are "incorrect" `width` and `height` values set for a `QPushButton` (here is another example: https://github.com/bitcoin-core/gui/pull/319#pullrequestreview-652907740).

  This fixes this small hitbox issue by converting the buttons from `QPushButton` to `QToolButton`, which in turn makes the buttons look explicitly clickable. This approach was chosen as it helps us avoid having to play around with `width` and `height` values until we find values that play nice with macOS and look good on Linux & Windows. Also, `QToolButton` is an appropriate class for these buttons.

  Per [Qt Docs](https://doc.qt.io/qt-5/qtoolbutton.html#details):
  > A tool button is a special button that provides quick-access to specific commands or options. As opposed to a normal command button, a tool button usually doesn't show a text label, but shows an icon instead.

  Since we are changing the type of the buttons, we need to change the respective actions connection logic in `rpcconsole`. Instead of plugging in `QToolButton`, we abstract it to the base class: `QAbstractButton`.

  per [Qt Dev Notes](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Developer-Notes-for-Qt-Code#inherited-signals-and-slot)
  > Use base class functions as this makes the code more general, e.g., use QAbstractButton::clicked instead of QPushButton::clicked.

  While here, we also update the size of the icons to `22x22` to be consistent with other tool buttons.

  **macOS: Master vs PR:**

  | Master        | PR               |
  | ----------- | ----------- |
  | ![master-ss-macos](https://user-images.githubusercontent.com/23396902/118339460-e9079c80-b4e6-11eb-864b-d394aca5df61.png) | ![pr-ss-macos](https://user-images.githubusercontent.com/23396902/118339468-ec9b2380-b4e6-11eb-9a9e-30620216750e.png) |

  **Linux: Master vs PR:**

  | Master        | PR               |
  | ----------- | ----------- |
  | ![master-ss-linux](https://user-images.githubusercontent.com/23396902/118339520-13595a00-b4e7-11eb-86d0-96dd1264c198.png) | ![pr-ss-linux](https://user-images.githubusercontent.com/23396902/118339533-1c4a2b80-b4e7-11eb-8d7f-f733d999c8fd.png) |

ACKs for top commit:
  hebasto:
    ACK 8b419b5163, tested on Linux Mint 20.1 (Qt 5.12.8).
  promag:
    Tested ACK 8b419b5163 on macOS Big Sur M1, this drops only relevant usages to `flat` buttons.

Tree-SHA512: 3f3cdcbe83398136a1d1ee8fc2835be8681f2ed39e79db1e939cab6a00a779f528343d54992807a845cc84d9ef13591affb7a6dbca9e5753a2b8665b0af4d611
2021-06-05 21:51:51 +03:00
Hennadii Stepanov
38ab7d0765 Merge bitcoin-core/gui#325: Align numbers in the "Peer Id" column to the right
69b8b5d72c qt: Align numbers in the "Peer Id" column to the right (Hennadii Stepanov)

Pull request description:

  On master (6b49d88a5d):
  ![Screenshot from 2021-05-12 21-53-52](https://user-images.githubusercontent.com/32963518/118029170-9013f900-b36c-11eb-9e27-36140cbd618e.png)

  With this PR:
  ![Screenshot from 2021-05-12 21-48-19](https://user-images.githubusercontent.com/32963518/118028803-2a277180-b36c-11eb-83ae-77d3e8cdd343.png)

ACKs for top commit:
  jarolrod:
    ACK 69b8b5d72c
  jonatack:
    ACK 69b8b5d72c happy to re-ack with more separation or center-aligned
  kristapsk:
    ACK 69b8b5d72c

Tree-SHA512: 3279522745a0fe6db1ec9be723a2640a37983947ed1af4aad56f22ce4780cc943a25b2d774529a9eeb3b00309f0204cfbc3f38b7db2c52865d05bf6a7c87e747
2021-06-05 15:29:15 +03:00
Hennadii Stepanov
fb1b1e0f3e qt: Save/restore column sizes of the tables in the Peers tab 2021-06-05 13:36:53 +03:00
MarcoFalke
898dd9e262 Merge bitcoin/bitcoin#22150: test: Remove unused node from feature_nulldummy
fa2b6c62cd test: Remove unused node from feature_nulldummy (MarcoFalke)

Pull request description:

  This is confusing and might even slow down the test.

  This reverts a change that was added a year ago in d438d60 and then the need for it was removed by 95d5d5e six months ago.

Top commit has no ACKs.

Tree-SHA512: 9a86792e9a634cf7bbd4e7a21b1acdfc3baba1b1962fe2b9b73848436d10351d2326dca01313c097ba2342dde7207add73e731d053c0bfa888a5d8f2b233a7cf
2021-06-05 08:41:37 +02:00
MarcoFalke
8f5c9a7fd4 Merge bitcoin/bitcoin#22153: test: Fix p2p_leak.py intermittent failure
ca3a77068b test: Fix p2p_leak.py intermittent failure by lowering timeout (Martin Zumsande)

Pull request description:

  Fixes #22085

  Root cause: There was just 1 second between the wait (5 seconds) and the `-peertimeout=4`.
  Since `ShouldRunInactivityChecks` in `net.cpp` measures the timeout in seconds, its result can only change once per second, even though it is called more often.
  So in situations when the connection is established early in a given second like [here](https://bitcoinbuilds.org/index.php?ansilog=d7b3e075-683a-45cc-94d4-9645fc17e0b6.log#l3117) (2021-05-27T12:28:04.**001**913Z ), the 1 second leeway was not be sufficient, leading to the intermittent failures.

  Fix this by lowering the timeout by one second.

ACKs for top commit:
  MarcoFalke:
    re-ACK ca3a77068b

Tree-SHA512: e7e22356d276c65a5b4f0a1b7ee5a9ad07d27691220746c7d02af3fad22cab1d53fd0ef59a938167ec80e4571c96649132d6922ad10667fc91baa47892f27a3e
2021-06-05 08:38:22 +02:00
Martin Zumsande
ca3a77068b test: Fix p2p_leak.py intermittent failure by lowering timeout 2021-06-04 22:42:04 +02:00
Vasil Dimov
bdb62096f0 fuzz: reduce possible networks check
If an address classifies as `IsRFC4193()`, then it cannot be
`NET_ONION` (Tor v3), thus remove that condition from the assert.
2021-06-04 16:12:04 +02:00
Vasil Dimov
a164cd3ba6 net: simplify CNetAddr::IsRoutable()
Reduce the condition `IsRFC4193() && !IsTor()` to `IsRFC4193()`. We know
that if `IsRFC4193()` is `true` then, for sure, the address is not Tor,
so `!IsTor()` is also `true`.
2021-06-04 16:09:25 +02:00
MarcoFalke
fa2b6c62cd test: Remove unused node from feature_nulldummy 2021-06-04 15:18:44 +02:00
MarcoFalke
faa94961d6 test: Add temporary logging to debug #20975 2021-06-04 15:05:27 +02:00
fanquake
346e52afd6 Merge bitcoin/bitcoin#22121: doc: Various validation doc fixups
fa4245d884 doc: Various validation doc fixups (MarcoFalke)

Pull request description:

ACKs for top commit:
  michaelfolkson:
    Re-ACK fa4245d884
  jnewbery:
    ACK fa4245d884

Tree-SHA512: fa1086b09941247a4ffcbc1d7d27dc77a17a3ae093a5146dbb703db9ff4ba5d73ea77bd5b7747af79ea8a7dfe2c4c56a7e19ac5aac3417090e9ae127836022ae
2021-06-04 20:42:28 +08:00
MarcoFalke
3ac5209662 Merge bitcoin/bitcoin#18795: Test: wallet issue with orphaned rewards
e4356f6a6c Testcase for wallet issue with orphaned rewards. (Daniel Kraft)

Pull request description:

  This adds a new test case demonstrating the wallet issue when block rewards are orphaned (#14148).

ACKs for top commit:
  LarryRuane:
    ACK e4356f6a6c
  leonardojobim:
    reACK e4356f6a6c .

Tree-SHA512: e9a2310ee1b3d52cfa302f431ed3d272bbc1b9195439ff318d9eb1006c0b28968dbe840e1600b6ff185e5d7ea57e4dcc837cef16051b5537445e10bc363b8c22
2021-06-04 13:54:34 +02:00
MarcoFalke
fa910b4765 util: Properly handle -noincludeconf on command line
This bug was introduced in commit
fad0867d6a.

Unit test
Co-Authored-By: Russell Yanofsky <russ@yanofsky.org>
2021-06-04 11:08:00 +02:00
MarcoFalke
a748782a11 Merge bitcoin/bitcoin#15545: [doc] explain why CheckBlock() is called before AcceptBlock
3d552b0d78 [doc] explain why CheckBlock() is called before AcceptBlock() (Sjors Provoost)

Pull request description:

  Based on https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-February/016697.html and its PDF attachment.

ACKs for top commit:
  MarcoFalke:
    cr ACK 3d552b0d78

Tree-SHA512: d1ef39855317853e0e7e051ec6015054d0d227fcdf20281c2c1921056537f1f79044aa1bdd35f46475edd17596fbcae79aeb338c4865b1269a01b158f6cb2ac4
2021-06-04 09:50:54 +02:00
Suhas Daftuar
6efbcec4de Protect last outbound HB compact block peer
If all our high-bandwidth compact block serving peers (BIP 152) stall block
download, then we can be denied a block for (potentially) a long time. As
inbound connections are much more likely to be adversarial than outbound
connections, mitigate this risk by never removing our last outbound HB peer if
it would be replaced by an inbound.
2021-06-03 17:15:25 -04:00
W. J. van der Laan
c7dd9ff71b Merge bitcoin/bitcoin#22051: Basic Taproot derivation support for descriptors
2667366aaa tests: check derivation of P2TR (Pieter Wuille)
7cedafc541 Add tr() descriptor (derivation only, no signing) (Pieter Wuille)
90fcac365e Add TaprootBuilder class (Pieter Wuille)
5f6cc8daa8 Add XOnlyPubKey::CreateTapTweak (Pieter Wuille)
2fbfb1becb Make consensus checking of tweaks in pubkey.* Taproot-specific (Pieter Wuille)
a4bf84039c Separate WitnessV1Taproot variant in CTxDestination (Pieter Wuille)
41839bdb89 Avoid dependence on CTxDestination index order (Pieter Wuille)
31df02a070 Change Solver() output for WITNESS_V1_TAPROOT (Pieter Wuille)
4b1cc08f9f Make XOnlyPubKey act like byte container (Pieter Wuille)

Pull request description:

  This is a subset of #21365, to aide review.

  This adds support `tr(KEY)` or `tr(KEY,SCRIPT)` or `tr(KEY,{{S1,{{S2,S3},...}},...})` descriptors, describing Taproot outputs with specified internal key, and optionally any number of scripts, in nested groups of 2 inside `{`/`}` if there are more than one. While it permits importing `tr(KEY)`, anything beyond that is just laying foundations for more features later.

  Missing:
  * Signing support (see #21365)
  * Support for more interesting scripts inside the tree (only `pk(KEY)` is supported for now). In particular, a multisig policy based on the new `OP_CHECKSIGADD` opcode would be very useful.
  * Inferring `tr()` descriptors from outputs (given sufficient information).
  * `getaddressinfo` support.
  * MuSig support. Standardizing that is still an ongoing effort, and is generally kind of useless without corresponding PSBT support.
  * Convenient ways of constructing descriptors without spendable internal key (especially ones that arent't trivially recognizable as such).

ACKs for top commit:
  Sjors:
    utACK 2667366 (based on https://github.com/bitcoin/bitcoin/pull/21365#issuecomment-846945215 review, plus the new functional test)
  achow101:
    Code Review ACK 2667366aaa
  lsilva01:
    Tested ACK 2667366aaa
  meshcollider:
    utACK 2667366aaa

Tree-SHA512: 61046fef22c561228338cb178422f0b782ef6587ec8208d3ce2bd07afcff29a664b54b35c6b01226eb70b6540b43f6dd245043d09aa6cb6db1381b6042667e75
2021-06-03 21:58:41 +02:00
Sjors Provoost
3d552b0d78 [doc] explain why CheckBlock() is called before AcceptBlock()
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2021-06-03 19:09:28 +02:00
W. J. van der Laan
07ededa30c Merge bitcoin/bitcoin#22050: p2p: remove tor v2 support
5d82a57db4 contrib: remove torv2 seed nodes (Jon Atack)
5f7e086dac contrib: update generate-seeds.py to ignore torv2 addresses (Jon Atack)
8be56f0f8e p2p, refactor: extract OnionToString() from CNetAddr::ToStringIp() (Jon Atack)
5f9d3c09b4 p2p: remove torv2 from CNetAddr::ToStringIP() (Jon Atack)
3d39042144 p2p: remove torv2 in SetIP() and ADDR_TORV2_SIZE constant (Jon Atack)
cff5ec477a p2p: remove pre-addrv2 onions from SerializeV1Array() (Jon Atack)
4192a74413 p2p: ignore torv2-in-ipv6 addresses in SetLegacyIPv6() (Jon Atack)
1d631e956f p2p: remove BIP155Network::TORV2 from GetBIP155Network() (Jon Atack)
7d1769bc45 p2p: remove torv2 from SetNetFromBIP155Network() (Jon Atack)
eba9a94b9f fuzz: rename CNetAddr/CService deserialize targets (Jon Atack)
c56a1c9b18 p2p: drop onions from IsAddrV1Compatible(), no longer relay torv2 (Jon Atack)
f8e94002fc p2p: remove torv2/ADDR_TORV2_SIZE from SetTor() (Jon Atack)
0f1c58ae87 test: update feature_proxy to torv3 (Jon Atack)

Pull request description:

  ![image](https://user-images.githubusercontent.com/2415484/120018909-4d425a00-bfd7-11eb-83c9-95a3dac97926.jpeg)

  This patch removes support in Bitcoin Core for Tor v2 onions, which are already removed from the release of Tor 0.4.6.

  - no longer serialize/deserialize and relay Tor v2 addresses
  - ignore incoming Tor v2 addresses
  - remove Tor v2 addresses from the addrman and peers.dat on node launch
  - update generate-seeds.py to ignore Tor v2 addresses
  - remove Tor v2 hard-coded seeds

  Tested with tor-0.4.6.1-alpha (no v2 support) and 0.4.5.7 (v2 support). With the latest Tor (no v2 support), this removes all the warnings like those reported with current master in https://github.com/bitcoin/bitcoin/issues/21351

  ```
  <bitcoind debug log>
  Socks5() connect to […].onion:8333 failed: general failure

  <tor log>
  Invalid hostname [scrubbed]; rejecting
  ```

  and the addrman no longer has Tor v2 addresses on launching bitcoind.
  ```rake
  $ ./src/bitcoin-cli -addrinfo
  {
    "addresses_known": {
      "ipv4": 44483,
      "ipv6": 8467,
      "torv2": 0,
      "torv3": 2296,
      "i2p": 6,
      "total": 55252
    }
  }
  ```
  After recompiling back to current master and restarting with either of the two Tor versions (0.4.5.7 or 0.4.6.1), -addrinfo initially returns 0 Tor v2 addresses and then begins finding them again.

  Ran nodes on this patch over the past week on mainnet/testnet/signet/regtest after building with DEBUG_ADDRMAN.

  Verified that this patch bootstraps an onlynet=onion node from the Tor v3 hardcoded fixed seeds on mainnet and testnet and connects to blocks and v3 onion peers: `rm ~/.bitcoin/testnet3/peers.dat ; ./src/bitcoind -testnet -dnsseed=0 -onlynet=onion`

  ![Screenshot from 2021-05-28 00-26-17](https://user-images.githubusercontent.com/2415484/119905021-ea02ea00-bf3a-11eb-875f-27ef57640c49.png)

  Tested using `addnode`, `getaddednodeinfo`,`addpeeraddress`, `disconnectnode` and `-addrinfo` that a currently valid, connectable Tor v2 peer can no longer be added:

  ![Screenshot from 2021-05-30 11-32-05](https://user-images.githubusercontent.com/2415484/120099282-29435d80-c12a-11eb-81b6-5084244d7d2a.png)

  Thanks to Vasil Dimov, Carl Dong, and Wladimir J. van der Laan for their work on BIP155 and Tor v3 that got us here.

ACKs for top commit:
  laanwj:
    Code review ACK 5d82a57db4

Tree-SHA512: 590ff3d2f6ef682608596facb4b01f44fef69716d2ab3552ae1655aa225f4bf104f9ee08d6769abb9982a8031de93340df553279ce1f5023771f9f2b651178bb
2021-06-03 18:43:55 +02:00
John Newbery
2f4ad6b7ef scripted-diff: rename MarkBlockAs functions
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren  MarkBlockAsInFlight BlockRequested
ren  MarkBlockAsReceived RemoveBlockRequest
-END VERIFY SCRIPT-
2021-06-03 14:57:37 +01:00
John Newbery
2c45f832e8 [net processing] Tidy up MarkBlockAsReceived() 2021-06-03 14:57:37 +01:00
W. J. van der Laan
907d636e5e Merge bitcoin/bitcoin#21353: interfaces: Stop exposing wallet destdata to gui
f5ba424cd4 wallet: Add IsAddressUsed / SetAddressUsed methods (Russell Yanofsky)
62252c95e5 interfaces: Stop exposing wallet destdata to gui (Russell Yanofsky)
985430d9b2 test: Add gui test for wallet receive requests (Russell Yanofsky)

Pull request description:

  Stop giving GUI access to destdata rows in database. Replace with narrow API just for saving and reading receive request information.

  This simplifies code and should prevent the GUI from interfering with other destdata like address-used status. It also adds some more GUI test coverage.

  There are no changes in behavior.

ACKs for top commit:
  jarolrod:
    tACK f5ba424cd4
  laanwj:
    Code review ACK f5ba424cd4

Tree-SHA512: 5423df4786e537a59013cb5bfb9e1bc29a7ca4b8835360c00cc2165a59f925fdc355907a4ceb8bca0285bb4946ba235bffa7645537a951ad03fd3b4cee17b6b0
2021-06-03 15:57:30 +02:00
John Newbery
6299350733 [net processing] Add IsBlockRequested() function
MarkBlockAsReceived() should not be used for both removing the block
from mapBlocksInFlight and checking whether it was in the map.
2021-06-03 14:57:26 +01:00
fanquake
8837f1ebde Merge bitcoin/bitcoin#21654: build, qt: Make Qt rcc output always deterministic
a58868d201 build: Makes rcc output always deterministic (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2.

  The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732).

  This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the
  `QT_RCC_SOURCE_DATE_OVERRIDE` needless.

  ---

  Also it improves interaction with ccache:

  On master (f6c44e999b):
  ```
  $ make && make clean && ccache --zero-stats && make && ccache --show-stats
  ...
  cache directory                     /home/hebasto/.ccache
  primary config                      /home/hebasto/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Sun Apr 11 15:45:43 2021
  stats zeroed                        Sun Apr 11 15:45:05 2021
  cache hit (direct)                   638
  cache hit (preprocessed)               0
  cache miss                             1
  cache hit rate                     99.84 %
  called for link                       10
  cleanups performed                     0
  files in cache                     20023
  cache size                          13.2 GB
  max cache size                      15.0 GB
  ```

  The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`.

  With this PR:
  ```
  $ make && make clean && ccache --zero-stats && make && ccache --show-stats
  ...
  cache directory                     /home/hebasto/.ccache
  primary config                      /home/hebasto/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Sun Apr 11 15:28:46 2021
  stats zeroed                        Sun Apr 11 15:28:21 2021
  cache hit (direct)                   639
  cache hit (preprocessed)               0
  cache miss                             0
  cache hit rate                    100.00 %
  called for link                       10
  cleanups performed                     0
  files in cache                     20012
  cache size                          13.2 GB
  max cache size                      15.0 GB
  ```

ACKs for top commit:
  fanquake:
    ACK a58868d201

Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5
2021-06-03 21:25:30 +08:00
fanquake
fcfd37f3f9 Merge bitcoin/bitcoin#22123: depends: Fix qt.mk for mac arm64
3b36395b96 depends: Fix qt.mk for mac arm64 (João Barbosa)

Pull request description:

  With f16d4cd8c5 `depends/config.guess` gives `aarch64-apple-darwin20.3.0` where before would give `arm-apple-darwin20.3.0`. Fix `qt.mk` accordingly.

ACKs for top commit:
  hebasto:
    ACK 3b36395b96, I have reviewed the code and it looks OK, I agree it can be merged.
  fanquake:
    ACK 3b36395b96

Tree-SHA512: bd20402d0a6e9a5bb652198de189cf2b4f3f76fd03d0cba8c4d657c60b8a088cf3532efe6c1efbbedd94c00a155e6d180b77f1cd8bc24e0e35764839e8b77e30
2021-06-03 21:04:41 +08:00
Christian Decker
b9e76f1bf0 rpc: Add test for -rpcwaittimeout
Suggested-by: Jon Atack <@jonatack>
2021-06-03 14:49:15 +02:00
fanquake
68ace23fa3 test: convert docs into type annotations in test_framework/test_node.py 2021-06-03 20:45:03 +08:00
fanquake
8bfcba36db test: convert docs into type annotations in test_framework/wallet.py 2021-06-03 20:45:03 +08:00
fanquake
b043ca8e8b test: convert docs into type annotations in test_framework/util.py 2021-06-03 20:45:03 +08:00
fanquake
6fe012c6bd Merge bitcoin/bitcoin#22133: build, qt: Make QWindowsVistaStylePlugin available again (regression)
ab86ac7739 build, qt: Make QWindowsVistaStylePlugin available again (regression) (Hennadii Stepanov)

Pull request description:

  This PR is similar to 1be8e0f238, and, actually, it is a #21376 follow up.

  Required as in Qt 5.12.x style plugins are separated.

  Fixes #22132.
  Fixes https://github.com/bitcoin-core/gui/issues/303.

  **Note for reviewers**. Besides visual changes in the GUI, you could compare the first dozen of lines in the `debug.log` file.

  ---

  #### Guix build:
  ```
  $ HOSTS=x86_64-w64-mingw32 contrib/guix/guix-build
  $ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-ab86ac7739b2/output/dist-archive/SKIPATTEST.TAG
  d3f05888c95d6cf6f4a0bac952d4c0bf1e271f49cb7ab92b08080b753957e5e5  guix-build-ab86ac7739b2/output/dist-archive/bitcoin-ab86ac7739b2.tar.gz
  113bb86ffd2854e08d3c0c5cd3f9728f2e4fe6c7a1813bc1c4b73eb48ad6597f  guix-build-ab86ac7739b2/output/x86_64-w64-mingw32/bitcoin-ab86ac7739b2-win-unsigned.tar.gz
  3fdc99754d6f26d9655a308c8847076d8fd4db8d61f4f18d8ef545894a42c10d  guix-build-ab86ac7739b2/output/x86_64-w64-mingw32/bitcoin-ab86ac7739b2-win64-debug.zip
  932bb69e3eb1a617f9e337b83220c9d6a277bec421f4d78e32dffde5643eb00c  guix-build-ab86ac7739b2/output/x86_64-w64-mingw32/bitcoin-ab86ac7739b2-win64-setup-unsigned.exe
  852f4c03ad049fcd6bb2ce25b3bcf4e18b90d34f7232660024b341bd7c8dd710  guix-build-ab86ac7739b2/output/x86_64-w64-mingw32/bitcoin-ab86ac7739b2-win64.zip
  c6a33a193a6b128e665198dde94758acab3a5ab7fefe0c3c6617eb98ad01693d  guix-build-ab86ac7739b2/output/x86_64-w64-mingw32/inputs.SHA256SUMS
  ```

ACKs for top commit:
  laanwj:
    Concept and code review ACK ab86ac7739
  fanquake:
    ACK ab86ac7739

Tree-SHA512: 9f5498480379fad41de616da48331e123daf5b2294ca79c33ffea2b113b314634be9a2a70bf060cb5be1392a48e9d047891e1d22c129f408f1d76a0bc888441f
2021-06-03 20:40:13 +08:00
MarcoFalke
d331e262f5 Merge bitcoin/bitcoin#22138: script: fix spelling linter raising spuriously on "invokable"
8050eb43bf script: fix spelling linter raising spuriously on "invokable" (Jon Atack)

Pull request description:

  "invokable" is a valid word that means to be callable, but the linter is raising on it:
  ```
  $ test/lint/lint-spelling.sh
  contrib/guix/guix-attest:18: invokable ==> invocable
  contrib/guix/guix-clean:18: invokable ==> invocable
  contrib/guix/guix-verify:18: invokable ==> invocable
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK 8050eb43bf

Tree-SHA512: 43f8dc7b7adb00ae563ccfe04a64a7ceb50237f24ff87209062bf57b2564b4d38a409df80e0183aa4f40ab306b5e07d7a5fad1600d41705bd3c443ed66a6d1c1
2021-06-03 14:36:40 +02:00
Jon Atack
5d82a57db4 contrib: remove torv2 seed nodes 2021-06-03 14:04:06 +02:00
Jon Atack
5f7e086dac contrib: update generate-seeds.py to ignore torv2 addresses 2021-06-03 14:04:04 +02:00
John Newbery
4e90d2dd0e [net processing] Remove QueuedBlock.hash
It's redundant with CBlockIndex::GetBlockHash()
2021-06-03 12:57:43 +01:00
Jon Atack
8050eb43bf script: fix spelling linter raising spuriously on "invokable" 2021-06-03 13:55:45 +02:00
MarcoFalke
fa4245d884 doc: Various validation doc fixups
* Rename RewindBlockIndex -> NeedsRedownload (follow-up to commit
  d831e711ca)
* Fix typos
* Inline comments about faking chain data to avoid duplicating them
2021-06-03 13:53:31 +02:00
John Newbery
156a19ee6a scripted-diff: rename nPeersWithValidatedDownloads
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren nPeersWithValidatedDownloads  m_peers_downloading_from
-END VERIFY SCRIPT-
2021-06-03 12:49:27 +01:00
John Newbery
b03de9c753 [net processing] Remove CNodeState.nBlocksInFlightValidHeaders
nBlocksInFlightValidHeaders always has the same value as nBlocksInFlight, since we only download
blocks with valid headers.
2021-06-03 12:45:15 +01:00
John Newbery
b4e29f2436 [net processing] Remove QueuedBlock.fValidatedHeaders
Since headers-first syncing, we only ever request a block if we've already validated its headers.
Therefore QueuedBlock.fValidatedHeaders is always set to true. Remove it.
2021-06-03 12:39:56 +01:00
John Newbery
85e058b191 [net processing] Remove unnecessary hash arg from MarkBlockAsInFlight()
MarkBlockAsInFlight is always called with a non-null pindex. Just get the block hash
from that pindex inside the function.
2021-06-03 12:39:55 +01:00
Jon Atack
8be56f0f8e p2p, refactor: extract OnionToString() from CNetAddr::ToStringIp() 2021-06-03 13:36:50 +02:00
Jon Atack
5f9d3c09b4 p2p: remove torv2 from CNetAddr::ToStringIP() 2021-06-03 13:36:48 +02:00
Jon Atack
3d39042144 p2p: remove torv2 in SetIP() and ADDR_TORV2_SIZE constant 2021-06-03 13:36:46 +02:00
Jon Atack
cff5ec477a p2p: remove pre-addrv2 onions from SerializeV1Array() 2021-06-03 13:36:44 +02:00
Jon Atack
4192a74413 p2p: ignore torv2-in-ipv6 addresses in SetLegacyIPv6() 2021-06-03 13:36:42 +02:00
Jon Atack
1d631e956f p2p: remove BIP155Network::TORV2 from GetBIP155Network() 2021-06-03 13:36:40 +02:00
Christian Decker
f76cb10d7d rpc: Prefix rpcwaittimeout error with details on its nature
As proposed by @laanwj the error message is now prefixed with the
"timeout on transient error:" prefix, to explain why the error is
suddenly considered terminal.
2021-06-03 11:23:22 +02:00
Christian Decker
c490e17ef6 doc: Add release notes for the -rpcwaittimeout cli parameter 2021-06-03 11:23:22 +02:00
Christian Decker
a7fcc8eb59 rpc: Add a -rpcwaittimeout parameter to limit time spent waiting
Adds a new numeric `-rpcwaittimeout` that can be used to limit the
time we spend waiting on the RPC server to appear. This is used by
downstream projects to provide a bit of slack when `bitcoind`s RPC
interface is not available right away.
2021-06-03 11:23:22 +02:00
MarcoFalke
a9435e3445 Merge bitcoin/bitcoin#22065: Mark CheckTxInputs [[nodiscard]]. Avoid UUM in fuzzing harness coins_view.
37371268d1 Mark `CheckTxInputs` `[[nodiscard]]` (out-param `txfee` only set if call is successful). Avoid UUM in fuzzing harness `coins_view`. (practicalswift)

Pull request description:

  Mark `CheckTxInputs` `[[nodiscard]]` (out-param `txfee` only set if call is successful).

  Avoid use of uninitialised memory (UUM) in fuzzing harness `coins_view`.

ACKs for top commit:
  MarcoFalke:
    review ACK 37371268d1

Tree-SHA512: edada5b2e80ce9ad3bd57b4c445bedefffa0a2d1cc880957d6848e4b7d9fc1ce036cd17f8b18bc03a36fbf84fc29c166cd6ac3dfbfe03e69d6fdbda13697754d
2021-06-03 08:53:06 +02:00
MarcoFalke
0011167191 Merge bitcoin/bitcoin#22135: CRegTestParams: Use args instead of gArgs.
5f23531926 CRegTestParams: Use `args` instead of `gArgs`. (Kiminuo)

Pull request description:

  This PR is a very minor follow-up to #13311.

  I believe that `gArgs` was just overlooked at the modified line.

ACKs for top commit:
  MarcoFalke:
    cr ACK 5f23531926

Tree-SHA512: f4e4ed6b23fca60e88825b502f20a1341ee2e4429bc8a2a7e419057adb643abda11be2061fe7ee076931657736e629aff88fd2c33737c84c330dc9d64f368c30
2021-06-03 08:14:56 +02:00
MarcoFalke
fd7a770d32 Merge bitcoin/bitcoin#22122: ci: Bump macOS image to big-sur-xcode-12.5
faa8dfd6a1 ci: Bump macOS image to big-sur-xcode-12.5 (MarcoFalke)

Pull request description:

  Closes #22068

ACKs for top commit:
  hebasto:
    ACK faa8dfd6a1, I have reviewed the code and it looks OK, I agree it can be merged, and the Cirrus CI is green.

Tree-SHA512: e29f6290163f3727f3603a3d6b4cf47677f6b02fff370e8d9073962a42bd7ab1ae8d247306e4c41bcadf0a208784344a6229627fe1a883b1e5112df30ea88635
2021-06-03 07:46:12 +02:00
Hennadii Stepanov
ab86ac7739 build, qt: Make QWindowsVistaStylePlugin available again (regression)
In Qt 5.12.x style plugins are separated.

Co-authored-by: Jarol Rodriguez <jarolrod@tutanota.com>
2021-06-03 00:25:01 +03:00
Pieter Wuille
79c02c88b3 Randomize message processing peer order 2021-06-02 13:55:14 -07:00
Kiminuo
5f23531926 CRegTestParams: Use args instead of gArgs. 2021-06-02 22:36:48 +02:00
W. J. van der Laan
2aab8a6dd0 Merge bitcoin/bitcoin#21231: Add /opt/homebrew to path to look for boost libraries
9a0969585f build: Add /opt/homebrew to path to look for boost libraries (Fu Yong Quah)

Pull request description:

  Following the instruction in https://github.com/bitcoin/bitcoin/blob/master/doc/build-osx.md for building on the M1 Macs don't work out of the box, because homebrew now defaults to /opt/homebrew instead of /usr/local. This PR fixes that.

ACKs for top commit:
  jonasschnelli:
    utACK 9a0969585f
  promag:
    Tested ACK 9a0969585f.

Tree-SHA512: 472568b97fbd8623481fe6fd43b0509fa32fe7f1c1d8090321a6a6a5bdc7343d4ad4122c10dcc7c9c93068db8a3f009a73befaf1ba11e4af54a66afd2c2dbe14
2021-06-02 20:37:11 +02:00
glozow
ee862d6efb MOVEONLY: context-free package policies
Co-authored-by: ariard <antoine.riard@gmail.com>
2021-06-02 17:26:44 +01:00
W. J. van der Laan
1186910b6b Merge bitcoin/bitcoin#22088: doc: improve note on choosing posix mingw32
dafab2b1b3 doc: improve note on choosing posix mingw32 (Jarol Rodriguez)

Pull request description:

  The current [windows build doc](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md) can lead someone to believe that the step where you must choose the posix mingw32 g++ compiler option is only for `Ubuntu 18.04`. It is only until you (or just me) go through the cross-compile process and realize that it's not building because you didn't set this option on > `Ubuntu 18.04`. Then you come back and read the footnotes and see: `Starting from Ubuntu Xenial 16.04...`.

  This PR improves this portion of the doc by editing the text around this stage to state "this is now the next step". We could add a note saying `Ubuntu 18.04 and up`, but this is redundant as it's unlikely someone will be using < Ubuntu 18.04 since it's not officially supported by our build system. While here, some minor fixups and add some more guidance to completing this step.

  **Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md#building-for-64-bit-windows)
  **PR:** [render](dafab2b1b3/doc/build-windows.md (building-for-64-bit-windows))

ACKs for top commit:
  laanwj:
    ACK dafab2b1b3
  hebasto:
    ACK dafab2b1b3

Tree-SHA512: d8d50fa13c34be753700843183a1a4addad6447c813bfc039865ba11d5fa03c25206ed6f3782331df4c6a44448e3fec4138ae911b995b5cacc5b756bb5d74bb4
2021-06-02 17:01:02 +02:00
W. J. van der Laan
a7d17daa5c Merge bitcoin/bitcoin#22086: test: remove BasicTestingSetup from unit tests that don't need it
6c3fcd5591 test: remove BasicTestingSetup from util_threadnames unit tests (fanquake)
b53d3c1b1f test: remove BasicTestingSetup from uint256 unit tests (fanquake)
c0497a4928 test: remove BasicTestingSetup from torcontrol unit tests (fanquake)
ef8bb0473b test: remove BasicTestingSetup from sync unit tests (fanquake)
1aee83421f test: remove BasicTestingSetup from reverse_lock unit tests (fanquake)
57ba949ef5 test: remove BasicTestingSetup from policy_fee unit tests (fanquake)
3974c962b6 test: remove BasicTestingSetup from merkleblock tests (fanquake)
cd5bc4b470 test: remove BasicTestingSetup from hash unit tests (fanquake)
39cec22935 test: remove BasicTestingSetup from compilerbug unit tests (fanquake)
6d3b78c0e2 test: remove BasicTestingSetup from bswap unit tests (fanquake)
a13dc24831 test: remove BasicTestingSetup from bech32 unit tests (fanquake)
f4dcbe4498 test: remove BasicTestingSetup from base64 unit tests (fanquake)
fd144f6426 test: remove BasicTestingSetup from base32 unit tests (fanquake)
4c389ba04b test: remove BasicTestingSetup from arith_uint256 unit tests (fanquake)
05590651a0 test: remove BasicTestingSetup from amount unit tests (fanquake)
883a5c7d02 test: remove BasicTestingSetup from allocator unit tests (fanquake)

Pull request description:

  * Less setup/overhead for tests that don't need it. Some naive bench-marking would suggest that a full `test_bitcoin` run is a few % faster after this change.
  * Tests which don't need the BasicTestingSetup can't accidentally end up depending on it somehow.
  * Already the case in at least the scheduler and block_filter tests.

  This adds missing includes, but more significant is the removal of `setup_common.h` from tests where it isn't needed. This saves recompiling those tests when changes are made in the header.

ACKs for top commit:
  practicalswift:
    cr ACK 6c3fcd5591: patch looks correct
  laanwj:
    ACK 6c3fcd5591

Tree-SHA512: 69b891e2b4740402d62b86a4fc98c329a432d125971342a6f97334e166b3537ed3d4cdbb2531fa05c1feae32339c9fcb2dceda9afeeaed4edc70e8caa0962161
2021-06-02 16:53:05 +02:00
Daniel Kraft
e4356f6a6c Testcase for wallet issue with orphaned rewards.
This adds a new test case demonstrating the wallet issue when block
rewards are orphaned (https://github.com/bitcoin/bitcoin/issues/14148).
2021-06-02 13:53:50 +02:00
W. J. van der Laan
e3d644a127 Merge bitcoin-core/gui#351: Translations update
0680460041 qt: Translations update (Hennadii Stepanov)

Pull request description:

  This is a pre-translation-string-freeze update. See [Release schedule for 22.0](https://github.com/bitcoin/bitcoin/issues/20851).

ACKs for top commit:
  laanwj:
    ACK 0680460041

Tree-SHA512: 65f11cc20e82459ca3484d4f377ff38be5ba31ba906abcb58d3e5ea56ee0eefd5e74c1ef1ea387833812bb2b9a02995dbeed7b45707e1d6db3f49e3b8055af6f
2021-06-02 12:24:12 +02:00
João Barbosa
3b36395b96 depends: Fix qt.mk for mac arm64 2021-06-02 10:02:33 +01:00
glozow
5cac95cd15 disallow_mempool_conflicts -> allow_bip125_replacement and check earlier 2021-06-02 09:52:50 +01:00
glozow
e8ecc621be [refactor] comment/naming improvements 2021-06-02 09:40:40 +01:00
glozow
7d91442461 [rpc] reserve space in txns 2021-06-02 09:40:39 +01:00
glozow
6c5f19d9c4 [package] static_assert max package size >= max tx size 2021-06-02 09:40:39 +01:00
MarcoFalke
faa8dfd6a1 ci: Bump macOS image to big-sur-xcode-12.5
This also removes the "brew update" added in commit
b7381552cd.
2021-06-02 10:03:38 +02:00
MarcoFalke
2fccd9cf30 Merge bitcoin/bitcoin#22082: test: update nanobench from release 4.0.0 to 4.3.4
44d05d0a69 test: remove sanitizer suppression for nanobench (Martin Ankerl)
e3c866e3ca test: update nanobench from release 4.0.0 to 4.3.4 (Martin Ankerl)

Pull request description:

  This updates the third-party library nanobench with the latest release. It contains mostly minor bugfixes, a new pyperf output format, ability to suppress warnings with environment variable `NANOBENCH_SUPPRESS_WARNINGS`. Full changelog:

  v4.0.2
  * Changed `doNotOptimizeAway` to what google benchmark is doing. The old code did not work on some machines.
  * fix: display correct "total" value
  * minor Documentation updates

  v4.1.0
  * Updated link to new pyperf home
  * Adds ability to configure console output time unit
  *  Add support for environment variable `NANOBENCH_SUPPRESS_WARNINGS`
  * Nanobench is now usable with CMake's FetchContent (see documentation: https://nanobench.ankerl.com/tutorial.html#cmake-integration)

  v4.2.0
  * Ability to store and later compare results added, through `pyperf`.
  * See https://nanobench.ankerl.com/tutorial.html#pyperf-python-pyperf-module-output
  * Added lots of build targets to travis, similar to bitcoin's build.
  * Some minor API & documentation improvements

  v4.3.0
  * `ankerl::nanobench::Rng` can now return the state with `std::vector<uint64_t> Rng::state()`, and this can also be used to initialize the Rng.

  v4.3.1
  * Minor cmake improvements when integrationg as a third-party library: add alias `nanobench::nanobench`, default to C++17

  v4.3.2
  * Fixed a MSVC 2015 build problem
  * updates license to 2021.
  * build should now work with very old linux headers
  * Also disable UBSAN (bitcoin needed to add a suppression)

  v4.3.3
  * Do not use locale-dependent `std::to_string`

  v4.3.4
  * Add missing sanitizer suppression to `rotl`

ACKs for top commit:
  MarcoFalke:
    review ACK 44d05d0a69

Tree-SHA512: 3291c85057720cfc84a44bfaa305a7d0df4dc35779169d20de73d32e40d4cdbf3f005bf343f79710eca517441de2459e8118c195c5f5136f99d1f50ebd5dfd08
2021-06-02 09:20:16 +02:00
fanquake
a83bbf02c7 Merge bitcoin/bitcoin#22054: depends: Bump Qt version to 5.12.11
fa416cea84 depends: Bump Qt version to 5.12.11 (Hennadii Stepanov)

Pull request description:

  Qt 5.12.11:
  - [fixes](c5d904639d) macOS related [QTBUG-87014](https://bugreports.qt.io/browse/QTBUG-87014), and the `fix_bigsur_drawing.patch` (which is our workaround for QTBUG-87014) could be dropped
  - [upgrades](00326c9dc1) supported macOS SDK to 11.0, and removes related warnings
  - fixes tab widget rendering on macOS Big Sur ([here](4d6832d03f) and [here](28b015342d)):
    - master (5.12.10)
  ![DeepinScreenshot_select-area_20210525122725](https://user-images.githubusercontent.com/32963518/119474470-dd24a180-bd54-11eb-82e6-ca4d39b0b177.png)
  ![DeepinScreenshot_select-area_20210525123634](https://user-images.githubusercontent.com/32963518/119475548-ebbf8880-bd55-11eb-9c13-477016f8c23b.png)
    - this PR (5.12.11)
  ![Screenshot from 2021-05-25 12-28-02](https://user-images.githubusercontent.com/32963518/119474537-ed3c8100-bd54-11eb-8efe-7fe5ccae8a98.png)

  Closes https://github.com/bitcoin-core/gui/issues/136.

ACKs for top commit:
  goums:
    ACK fa416cea84
  fanquake:
    ACK fa416cea84 - merging this now to move it out of the way of other Qt related changes.

Tree-SHA512: 4e621b214f05d12a060757fadf2fa103c09b594d7b1c2ad9200f550e55c1e1fdf66f2f830550e724b9277448e5480be256cb0003f4597902944400e16c4a68b8
2021-06-02 12:12:33 +08:00
fanquake
0a3b8ea11a Merge bitcoin/bitcoin#22106: refactor: address ProcessNewBlock comments from #21713
e12f287498 net: cleanup newly added PeerManagerImpl::ProcessNewBlock (fanquake)
610151f5b0 validation: change ProcessNewBlock() to take a CBlock reference (fanquake)

Pull request description:

  Addresses some [post-merge comments](https://github.com/bitcoin/bitcoin/pull/21713#pullrequestreview-638777410) from #21713. Also makes `ChainstateManager::ProcessNewBlock` take a const reference argument, as it [was asked](https://github.com/bitcoin/bitcoin/pull/21713#discussion_r615229548) why it was not the case in that PR.

ACKs for top commit:
  jnewbery:
    Code review ACK e12f287498
  MarcoFalke:
    review ACK e12f287498 🚚

Tree-SHA512: 9c3e7353240c862d50bce2a0f58741c109dd628040b56ed46250103f8ebe9009238b131da710486791e28e3a83c985057b7be0a32aed1a929269b43097c7425b
2021-06-02 10:45:47 +08:00
unknown
d44a261acf Fix issues when walletdir is root directory
+ Remove one character less from wallet path if root directory
2021-06-01 20:34:20 +05:30
Hennadii Stepanov
0680460041 qt: Translations update 2021-06-01 17:09:20 +03:00
MarcoFalke
7e83e74e7f Merge bitcoin/bitcoin#21178: test: run mempool_reorg.py even with wallet disabled
a3f0cbf82d test: run mempool_reorg.py even with wallet disabled (Darius Parvin)

Pull request description:

  Run mempool_reorg.py test even when the wallet is disabled, as discussed in #20078.

  As part of this PR I created a new method in `MiniWallet`, `create_self_transfer`, to return a raw tx (without broadcasting it) and its associated utxo.

ACKs for top commit:
  MarcoFalke:
    cr ACK a3f0cbf82d

Tree-SHA512: 316a38faffadcb87499c1d6eca21e9696cef65362bbffcf621788a9b771bb1fa2971b1c7835cbd34b952d7612ad83afbca824cd8be39ecd6b994e8963027f991
2021-06-01 15:05:45 +02:00
MarcoFalke
0b9ed3f6e7 Merge bitcoin/bitcoin#22113: test: minor cleanups in feature_cltv.py
7e32fde912 test: feature_cltv.py: don't return tx copies in modification functions (Sebastian Falbesoner)
9ab2ce0a66 test: drop unused node parameters in feature_cltv.py (Sebastian Falbesoner)
0c2139a3f1 test: fix typo in feature_cltv.py (s/ctlv/cltv/) (Sebastian Falbesoner)

Pull request description:

  This tiny PR cleans up the test `feature_cltv.py` in the following ways:
  * fixes a typo (s/ctlv/cltv/); compared to CHECKLOCKTIMEVERIFY, CHECKTIMELOCKVERIFY probably also sounds good and you [even get some search results for it](https://www.google.com/search?q=%22CHECKTIMELOCKVERIFY%22), but it's still wrong ;)
  * drops the unused "node" parameters from the tx modification functions
  * don't return a copy from the tx modification functions; it's modified in-place, hence a copy is not needed and `cltv_validate(tx, ...)` looks more natural than `tx = cltv_validate(tx, ...)`

ACKs for top commit:
  MarcoFalke:
    review ACK 7e32fde912 📼

Tree-SHA512: d2e6230977442f6a511d0f7c99431a44ad3a423647f4f327ce2ce8efe78bf9616c0d2093f5e3c3550f690dcb3f625ddf53227505c01ced70227425f249c25364
2021-06-01 14:55:56 +02:00
MarcoFalke
f63fc53c2a Merge bitcoin/bitcoin#21767: [Bundle 6/n] Prune g_chainman usage in auxiliary modules
7a799c9c2b index: refactor-only: Reuse CChain ref (Carl Dong)
db33cde80f index: Add chainstate member to BaseIndex (Carl Dong)
f4a47a1feb bench: Use existing chainman in AssembleBlock (Carl Dong)
91226eb917 bench: Use existing NodeContext in DuplicateInputs (Carl Dong)
e6b4aa6eb5 miner: Pass in chainman to RegenerateCommitments (Carl Dong)
9ecade1425 rest: Add GetChainman function and use it (Carl Dong)
fc1c282845 rpc/blockchain: Use existing blockman in gettxoutsetinfo (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  The first 2 commits are fixups addressing review for the last bundle: #21391

  NEW note:
  1. I have opened #21766 which keeps track of potential improvements where the flaws already existed before the de-globalization work, please post on that issue about these improvements, thanks!

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

ACKs for top commit:
  jarolrod:
    ACK  7a799c9
  ariard:
    Code Review ACK 7a799c9
  fjahr:
    re-ACK 7a799c9c2b
  MarcoFalke:
    review ACK 7a799c9c2b 🌠
  ryanofsky:
    Code review ACK 7a799c9c2b. Basically no change since last review except fixed rebase conflicts and a new comment about REST Ensure()
  jamesob:
    conditional ACK 7a799c9c2b ([`jamesob/ackr/21767.1.dongcarl.bundle_6_n_prune_g_chai`](https://github.com/jamesob/bitcoin/tree/ackr/21767.1.dongcarl.bundle_6_n_prune_g_chai))

Tree-SHA512: 531c00ddcb318817457db2812d9a9d930bc664e58e6f7f1c746350732b031dd624270bfa6b9f49d8056aeb6321d973f0e38e4ff914acd6768edd8602c017d10e
2021-06-01 13:34:18 +02:00
Martin Ankerl
44d05d0a69 test: remove sanitizer suppression for nanobench
In #21738 an ASAN suppression for nanobench was added. This is not needed any more,as nanobench.h already includes the necessary suppressions for the relevant code.
2021-06-01 12:00:21 +02:00
Martin Ankerl
e3c866e3ca test: update nanobench from release 4.0.0 to 4.3.4
This updates the third-party library nanobench with the latest release. It contains mostly minor bugfixes, a new pyperf output format, ability to suppress warnings with environment variable `NANOBENCH_SUPPRESS_WARNINGS`. Full changelog:

v4.0.2
* Changed `doNotOptimizeAway` to what google benchmark is doing. The old code did not work on some machines.
* fix: display correct "total" value
* minor Documentation updates

v4.1.0
* Updated link to new pyperf home
* Adds ability to configure console output time unit
*  Add support for environment variable `NANOBENCH_SUPPRESS_WARNINGS`
* Nanobench is now usable with CMake's FetchContent (see documentation: https://nanobench.ankerl.com/tutorial.html#cmake-integration)

v4.2.0
* Ability to store and later compare results added, through `pyperf`.
* See https://nanobench.ankerl.com/tutorial.html#pyperf-python-pyperf-module-output
* Added lots of build targets to travis, similar to bitcoin's build.
* Some minor API & documentation improvements

v4.3.0
* `ankerl::nanobench::Rng` can now return the state with `std::vector<uint64_t> Rng::state()`, and this can also be used to initialize the Rng.

v4.3.1
* Minor cmake improvements when integrationg as a third-party library: add alias `nanobench::nanobench`, default to C++17

v4.3.2
* Fixed a MSVC 2015 build problem
* updates license to 2021.
* build should now work with very old linux headers
* Also disable UBSAN (bitcoin needed to add a suppression)

v4.3.3
* Do not use locale-dependent `std::to_string`

v4.3.4
* Add missing sanitizer suppression to `rotl`
2021-06-01 12:00:00 +02:00
MarcoFalke
c91589dc2d Merge bitcoin/bitcoin#22005: fuzz: Speed up banman fuzz target
fae0f836be fuzz: Speed up banman fuzz target (MarcoFalke)

Pull request description:

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34463

ACKs for top commit:
  practicalswift:
    cr ACK fae0f836be: patch looks correct and touches only `src/test/fuzz/banman.cpp`

Tree-SHA512: edbad168c607d09a5f4a29639f2d0b852605dd61403334356ad35a1eac667b6ce3922b1b316fdf37a991195fbc24e947df9e37359231663f8a364e5889e28417
2021-06-01 11:32:02 +02:00
MarcoFalke
5cf92c32d1 Merge bitcoin/bitcoin#21969: refactor: Switch serialize to uint8_t (Bundle 1/2)
ffff0d0442 refactor: Switch serialize to uint8_t (1/n) (MarcoFalke)

Pull request description:

  Replace `char` -> `uint8_t` in serialization where a sign doesn't make sense (char might be signed/unsigned).

ACKs for top commit:
  practicalswift:
    cr ACK ffff0d0442: patch looks correct and commit hash is ffffresh (was bbbbadass)
  kristapsk:
    ACK ffff0d0442

Tree-SHA512: cda682280c21d37cc3a6abd62569732079b31d18df3f157aa28bed80bd6f9f29a7db5c133b1f57b3a8f8d5ba181a76e473763c6e26a2df6d9244813f56f893ee
2021-06-01 09:01:29 +02:00
fanquake
62d9ff9e6e Merge bitcoin/bitcoin#22107: scripted-diff: rename GetSystemTimeInSeconds to GetTimeSeconds
feb72e5432 scripted-diff: rename GetSystemTimeInSeconds to GetTimeSeconds (fanquake)

Pull request description:

  This PR simply renames `GetSystemTimeInSeconds` to `GetTimeSeconds`, for uniformity amongst our time handling functions (`GetTimeMillis`, `GetTimeMicros`). I have a branch that does a chunk of `GetTime()` -> `GetSystemTimeInSeconds` (`GetTimeSeconds`) / `GetTime<T>` migration, so we can eventually remove the (2 year) deprecated `GetTime()`.

  However, splitting this off and doing the renaming first while the number of `GetSystemTimeInSeconds` instances is small seems worthwhile.

ACKs for top commit:
  practicalswift:
    cr ACK feb72e5432: patch looks correct
  promag:
    Code review ACK feb72e5432.

Tree-SHA512: e2ac30be9cbcd77b70c9f74bef820b558945d0fcc6f3dc59fde68a18d08a7d36f42088b804ffe7c03478c8db048615b4c4aa65a3d8d9f5d717d59b58c99f1c54
2021-06-01 11:07:39 +08:00
Hennadii Stepanov
684e687d42 Merge bitcoin-core/gui#123: rpc: Do not accept command while executing another one
38eb37c0bd qt, rpc: Do not accept command while executing another one (Hennadii Stepanov)
0c32b9c527 qt, rpc: Accept stop RPC even another command is executing (Hennadii Stepanov)
ccf790287c qt, rpc, refactor: Return early in RPCConsole::on_lineEdit_returnPressed (Hennadii Stepanov)
5b9c8c9cdd qt, rpc: Add "Executing…" message (Hennadii Stepanov)

Pull request description:

  On master (3f512f3d56) it is possible to enter another command while the current command is still being executed. That makes a mess in the output.

  With this PR:
  ![Screenshot from 2020-10-29 20-48-55](https://user-images.githubusercontent.com/32963518/97619690-329c0880-1a29-11eb-9f5b-6ae3c02c13b2.png)

  Some previous context: https://github.com/bitcoin-core/gui/pull/59#issuecomment-715275185

  ---

  It is still possible to enter and execute the `stop` command any time.

ACKs for top commit:
  jarolrod:
    ACK  38eb37c
  promag:
    Tested ACK 38eb37c0bd.

Tree-SHA512: 2b37a4b6838bf586b1b5c878192106721f713caeb6252514a6540356aab898986396e0777e73891d331b1be797a4926c20d3f9f38ba2c984ea90d55b0c34f664
2021-06-01 03:27:29 +03:00
Hennadii Stepanov
aedf71dcc5 Merge bitcoin-core/gui#331: Make RPC console welcome message translation-friendly
0f3d955a38 qt: Make RPC console welcome message translation-friendly (Hennadii Stepanov)

Pull request description:

  The best practice is do not split a translatable multi-line message into single lines. This helps translators to follow the context.

ACKs for top commit:
  jarolrod:
    re-ACK 0f3d955a38

Tree-SHA512: 30911ff3a972a7787804bb8b27d0b77bfff15939bb478c199261866bfb55d9acd12ab4d44b8b9fc1d4898222cabc4007cc897f9b65728924d121f31e914c44ac
2021-05-31 23:43:58 +03:00
Hennadii Stepanov
38eb37c0bd qt, rpc: Do not accept command while executing another one 2021-05-31 23:34:04 +03:00
Hennadii Stepanov
0c32b9c527 qt, rpc: Accept stop RPC even another command is executing
While here, clean up the command input by calling the trimmed function
on the input from the command prompt.
2021-05-31 23:34:04 +03:00
Hennadii Stepanov
ccf790287c qt, rpc, refactor: Return early in RPCConsole::on_lineEdit_returnPressed 2021-05-31 23:32:59 +03:00
Hennadii Stepanov
5b9c8c9cdd qt, rpc: Add "Executing…" message 2021-05-31 23:31:32 +03:00
MarcoFalke
933c6466c2 Merge bitcoin/bitcoin#22109: doc: Fix external links (IRC, ...)
9999e4c64b doc: Fix external links (IRC, ...) (MarcoFalke)

Pull request description:

  The freenode channel was deleted by their staff. Fix that by moving to libera chat.

  Also includes minor cosmetic fixups 🐷 💅

ACKs for top commit:
  hebasto:
    re-ACK 9999e4c64b, agree with the recent changes.
  kristapsk:
    re-ACK 9999e4c64b
  theStack:
    ACK 9999e4c64b
  jarolrod:
    ACK 9999e4c64b

Tree-SHA512: 67886783b1239153ac1631f6594fc2f873760da3c63aca400c0a84f852418d0e28de0400310f4d8b8fe213b7d4d243600607acc4a48c1662f9848f254c946cab
2021-05-31 21:12:46 +02:00
Sebastian Falbesoner
7e32fde912 test: feature_cltv.py: don't return tx copies in modification functions
The functions cltv_modify_tx(), cltv_invalidate() and cltv_validate()
all modify the passed transaction in-place, i.e. there is no need
to return a copy.
2021-05-31 19:47:30 +02:00
Sebastian Falbesoner
9ab2ce0a66 test: drop unused node parameters in feature_cltv.py
The following functions had a parameter "node" that is unsused
and hence can be removed:
    - cltv_modify_tx()
    - cltv_invalidate()
    - cltv_validate()
2021-05-31 19:34:00 +02:00
Sebastian Falbesoner
0c2139a3f1 test: fix typo in feature_cltv.py (s/ctlv/cltv/) 2021-05-31 19:18:04 +02:00
Hennadii Stepanov
11225905b7 Merge bitcoin-core/gui#309: Add access to the Peers tab from the network icon
d29ea72393 gui: Add access to the Peers tab from the network icon (Hennadii Stepanov)

Pull request description:

  This PR add a small context menu to the network activity icon that provides an access to the Peers tab:

  ![gui-network-icon](https://user-images.githubusercontent.com/32963518/116794314-d64b9b80-aad4-11eb-89ca-7f75c7442ba8.gif)

  Closes #93.

ACKs for top commit:
  Sjors:
    re-ACK d29ea72393
  kristapsk:
    re-ACK d29ea72393
  promag:
    Code review ACK d29ea72393.

Tree-SHA512: dd871415fe514a19c6a22100d58f31954d9e55b80585d5a3f26e17a8d51dadf912441786fc0d23beabd812f1b501658fec1dbe345cd41beae5832a8eda890f77
2021-05-31 19:37:44 +03:00
Darius Parvin
a3f0cbf82d test: run mempool_reorg.py even with wallet disabled
- run mempool_reorg.py even when the wallet is not compiled
- add `locktime` argument to `create_self_transfer` and `send_self_transfer`
- use more logs instead of comments
2021-05-31 09:28:10 -07:00
MarcoFalke
9999e4c64b doc: Fix external links (IRC, ...) 2021-05-31 17:27:57 +02:00
MarcoFalke
ffff0d0442 refactor: Switch serialize to uint8_t (1/n) 2021-05-31 14:56:17 +02:00
MarcoFalke
8462cd5601 Merge bitcoin/bitcoin#22080: doc: add maxuploadtarget to bitcoin.conf example
947f9734da doc: add maxuploadtarget to bitcoin.conf example (apitko)

Pull request description:

  picking up #21499, author has stated they [can't squash](https://github.com/bitcoin/bitcoin/pull/21499#issuecomment-849277632).

  This adds the maxuploadtarget option to the `bitcoin.conf` example file. This is useful for those looking to configure their bandwidth utilization.

  **Changes from Original PR:**
  - squash commits
  - fix typo in commit message + reword commit message to be more appropriate
  - Implement review suggestions ([1](https://github.com/bitcoin/bitcoin/pull/21499#discussion_r615409982), [2](https://github.com/bitcoin/bitcoin/pull/21499#discussion_r615410337), [3](https://github.com/bitcoin/bitcoin/pull/21499#pullrequestreview-659357756))
  - Fix spacing

ACKs for top commit:
  jonatack:
    ACK 947f9734da
  theStack:
    re-ACK 947f9734da

Tree-SHA512: 658ce6e1f204cd1c1c6f8977fd46ba1631f417d3f492e6d777b0ca49ffecc2ecfa4fbd86ca0e08c0dd5d6df4e32f0b56693e0a71bd7eafafd019e93bbcf9420f
2021-05-31 13:59:51 +02:00
MarcoFalke
8c6df2b4ca Merge bitcoin/bitcoin#22095: test: Additional BIP32 test vector for hardened derivation with leading zeros
91ef8344d4 Additional test vector for hardened derivation with leading zeros (Kristaps Kaupe)

Pull request description:

  See [Inconsistent BIP32 Derivations](https://blog.polychainlabs.com/bitcoin,/bip32,/bip39,/kdf/2021/05/17/inconsistent-bip32-derivations.html) and https://github.com/bitcoin/bips/pull/1030.

ACKs for top commit:
  practicalswift:
    cr ACK 91ef8344d4
  sipa:
    ACK 91ef8344d4. Verified that it matches the linked BIP32 update, and that it indeed tests derivation from a private key with leading 0 byte.

Tree-SHA512: 0a3ae7aed15e4e08e9bec5db8de53c6c03ed3b3632f390394eea422597755173cbd2228ff0cfa57f5aae3df9d4cdf03a8ef4725cc8bce86ab7d9c82ab9d479ad
2021-05-31 13:46:42 +02:00
MarcoFalke
c5ee0cc11a Merge bitcoin/bitcoin#21989: test: Use COINBASE_MATURITY in functional tests
bfa9309ad6 Use COINBASE_MATURITY constant in functional tests. (Kiminuo)
525448df9d Move COINBASE_MATURITY from `feature_nulldummy` test to `blocktools`. (Kiminuo)

Pull request description:

  `COINBASE_MATURITY` constant was added to `feature_nulldummy` test in #21373. This PR moves the constant to `blocktools.py` file and uses the constant in more tests as suggested [here](https://github.com/bitcoin/bitcoin/pull/21373#discussion_r605418462).

  Edit: Goal of this PR is to replace integer constants with `COINBASE_MATURITY` but not necessarily in *all* cases because that would mean to read and fully understand all tests. That's out of my time constraints. Any reports where `COINBASE_MATURITY` should be used are welcome though!

ACKs for top commit:
  theStack:
    ACK bfa9309ad6 🌇

Tree-SHA512: 01f04645f05a39028681f355cf3d42dd63ea3303f76d93c430e0fdce441934358a2d847a54e6068d61932f1b75e1d406f51859b057b3e4b569f7083915cb317f
2021-05-31 11:26:25 +02:00
fanquake
feb72e5432 scripted-diff: rename GetSystemTimeInSeconds to GetTimeSeconds
-BEGIN VERIFY SCRIPT-
sed -i -e 's/GetSystemTimeInSeconds/GetTimeSeconds/g' $(git grep -l GetSystemTimeInSeconds src)
-END VERIFY SCRIPT-
2021-05-31 15:11:18 +08:00
fanquake
e12f287498 net: cleanup newly added PeerManagerImpl::ProcessNewBlock
Addresses some post-merge comments.
2021-05-31 14:36:46 +08:00
fanquake
610151f5b0 validation: change ProcessNewBlock() to take a CBlock reference
Update ProcessNewBlock arguments to newer style.
2021-05-31 14:36:46 +08:00
Kiminuo
bfa9309ad6 Use COINBASE_MATURITY constant in functional tests. 2021-05-31 07:32:28 +02:00
MarcoFalke
d7a6bba949 Merge bitcoin/bitcoin#22103: test: Fix IPv6 check on BSD systems
2be3572506 test: Fix IPv6 check on BSD systems (nthumann)

Pull request description:

  I noticed that `test_ipv6_local()` always returns `False` on macOS or FreeBSD, even though IPv6 is working perfectly fine. This causes `test/functional/rpc_bind.py --ipv6` and `test/functional/feature_proxy.py` to skip their run.
  Apparently, there's a check if the port number is `0` (see [here](64881da478/sys/netinet6/udp6_usrreq.c (L248)) or [here](8f02f2a044/bsd/netinet6/udp6_usrreq.c (L282))), while Linux has no problem with this.
  This is fixed by specifying any other port number than `0`, e.g. `1`. Still, because of `SOCK_DGRAM`, no actual connection is made.

ACKs for top commit:
  fanquake:
    ACK 2be3572506 - nice improvement. I checked that with this change ipv6 related tests in `feature_proxy.py` are being run.
  theStack:
    ACK 2be3572506

Tree-SHA512: 8417c2d3cf71050529f3fa409a03872040fe5d249eae4172f276e62156e505a20d375b963712a186c9ad7967d8a497b5900d327c74a9693f68c33063871d4691
2021-05-31 07:24:54 +02:00
fanquake
619e930aa1 Merge bitcoin/bitcoin#22070: build: don't use cf-protection when targeting arm-apple-darwin
3caedb4c03 build: don't use cf-protection when targeting arm-apple-darwin (fanquake)

Pull request description:

  After two reports on IRC of issues building depends on an Apple M1 machine, this option (obviously) can't be used when targeting `arm-apple-darwin`.  For now, just use it for `x86_64-apple-darwin`.

  ```bash
  Apple clang version 12.0.5 (clang-1205.0.22.9)
  Target: x86_64-apple-darwin20.4.0

  error: option 'cf-protection=return' cannot be specified on this target
  error: option 'cf-protection=branch' cannot be specified on this target
  2 errors generated.
  ```

ACKs for top commit:
  promag:
    Tested ACK 3caedb4c03.

Tree-SHA512: 8763a5b94000016b0c2f0438e66002fdfcd2cbafd9d2d2acc1972f0c6f707e820186711dbd9d3f72673c179718da75588acb4732f8d84b85f0c1dfc862b6e944
2021-05-31 10:05:55 +08:00
nthumann
2be3572506 test: Fix IPv6 check on BSD systems 2021-05-30 23:47:50 +02:00
Andrew Chow
96c2c9520e scripted-diff: Rename SelectCoinsMinConf to AttemptSelection
SelectCoinsMinConf is a bit of a misnomer now since it really just does
all of the coin selection given some parameters. So rename this to
something less annoying to say and makes a bit more sense.

-BEGIN VERIFY SCRIPT-
sed -i 's/SelectCoinsMinConf/AttemptSelection/g' $(git grep -l SelectCoinsMinConf ./src)
-END VERIFY SCRIPT-
2021-05-30 14:10:10 -04:00
Andrew Chow
b583f73354 Move vin filling to before final fee setting
It's unnecessary to fill in the vin with dummy inputs, calculate the
fee, then fill in the vin with the actual inputs. Just fill the vin with
the actual inputs the first time.
2021-05-30 14:07:49 -04:00
Andrew Chow
d39cac0547 Set m_subtract_fee_outputs during recipients vector loop
Instead of setting this afterwards based on the results from the loop,
just do it inside of the loop itself.

Fixed some styling nearby
2021-05-30 14:06:40 -04:00
Andrew Chow
364e0698a5 Move variable initializations to where they are used
- txNew nLockTime setting to txNew init
- FeeCalc to the fee estimation fetching
- setCoins to prior to SelectCoins
- nBytes to CalculateMaximumSignedTxSize call
- tx_sizes to CalculateMaximumSignedTxSize call
- coin_selection_params.m_avoid_partial_spends to params init
2021-05-30 14:06:13 -04:00
Andrew Chow
32ab430651 Move recipients vector checks to beginning of CreateTransaction
Ensuring that the recipients vector is not empty and that the amounts
are non-negative can be done in CreateTransaction rather than
CreateTransactionInternal. Additionally, these checks should happen as
soon as possible, so they are done at the beginning of
CreateTransaction.
2021-05-30 14:05:42 -04:00
Andrew Chow
cd1d6d3324 Rename nSubtractFeeFromAmount in CreateTransaction
Renamed to outputs_to_subtract_fee_from for clarity.
2021-05-30 14:05:15 -04:00
Andrew Chow
dac21c793f Rename nValue and nValueToSelect
nValue is the sum of the intended recipient amounts, so name it that for
clarity.

nValueToSelect is the coin selection target value, so name it
selection_target for clarity.
2021-05-30 14:03:43 -04:00
Andrew Chow
d2aee3bbc7 Remove extraneous scope in CreateTransactionInternal
These brackets were restricting a scope for no apparent reason. Remove
them and dedent.
2021-05-30 14:01:29 -04:00
Andrew Chow
b2995963b5 Move cs_wallet lock in CreateTransactionInternal to top of function
It isn't necessary to not lock parts of this function. Just lock the
whole thing and get rid of an indent.
2021-05-30 13:59:32 -04:00
Samuel Dobson
55a156fca0 Merge bitcoin/bitcoin#21207: MOVEONLY: CWallet transaction code out of wallet.cpp/.h
c7bd5842e4 MOVEONLY: CWallet transaction code out of wallet.cpp/.h (Russell Yanofsky)

Pull request description:

  This commit just moves function without making any changes. It can be reviewed with `git log -p -n1 --color-moved=dimmed_zebra`

  Motivation for this change is to make `wallet.cpp/h` less monolithic and start to make wallet transaction state tracking comprehensible so bugs in https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking can be fixed safely without introducing new problems.

  This moves wallet classes and methods that deal with transactions out of `wallet.cpp/.h` into better organized files:

  - `transaction.cpp/.h` - CWalletTx and CMerkleTx class definitions
  - `receive.cpp/.h` - functions checking received transactions and computing balances
  - `spend.cpp/.h` - functions creating transactions and finding spendable coins

  After #20773, when loading is separated from syncing it will also be possible to move more `wallet.cpp/.h` functions to:

  - `sync.cpp/.h` - functions handling chain notifications and rescanning

  This commit arranges `receive.cpp` and `spend.cpp` functions in dependency order so it's possible to skim `receive.cpp` and get an idea of how computing balances works, and skim `spend.cpp` and get an idea of how transactions are created, without having to jump all over `wallet.cpp` where functions are not in order and there is a lot of unrelated code.

  Followup commit "refactor: Detach wallet transaction methods" in https://github.com/bitcoin/bitcoin/pull/21206 follows up this PR and tweaks function names and arguments to reflect new locations. The two commits are split into separate PRs because this commit is more work to maintain and less work to review, while the other commit is less work to maintain and more work to review, so hopefully this commit can be merged earlier.

ACKs for top commit:
  Sjors:
    re-utACK c7bd5842e4
  fjahr:
    utACK c7bd5842e4
  promag:
    Code review ACK c7bd5842e4, verified move only claim.
  meshcollider:
    Dimmed-zebra-check and functional test run ACK c7bd5842e4

Tree-SHA512: 4981de6911cb1196774db375494355cc9af59b52456129c002d264a77cd9ed6175f8ecbb6b2f492a59a4d5a0def21a39d96fa79c9f4d99be0992985f553be32f
2021-05-30 22:00:40 +12:00
Hennadii Stepanov
be1c512437 Merge bitcoin-core/gui#343: Improve the GUI responsiveness when progress dialogs are used
4935ac583b qt: Improve GUI responsiveness (Hennadii Stepanov)
75850106ae qt, macos: Fix GUIUtil::PolishProgressDialog bug (Hennadii Stepanov)

Pull request description:

  [`QProgressDialog`](https://doc.qt.io/qt-5/qprogressdialog.html) estimates the time the operation will take (based on time for steps), and only shows itself if that estimate is beyond [`minimumDuration`](https://doc.qt.io/qt-5/qprogressdialog.html#minimumDuration-prop).

  The default `minimumDuration` value is [4 seconds](https://doc.qt.io/qt-5/qprogressdialog.html#details), and it could make users think that the GUI is frozen.

  This PR sets `minimumDuration` to zero for all progress dialogs, that affects ones in the `WalletControllerActivity` class.

ACKs for top commit:
  ryanofsky:
    Code review ACK 4935ac583b. I'm not very familiar with this API but all the changes and explanations make sense and are very clear, and this seems like it should be an improvement.
  promag:
    Code review ACK 4935ac583b.
  jarolrod:
    ACK 4935ac583b

Tree-SHA512: 2ddd74e7fd87894d341d2439dbaa544d031a350f7f57d4c7e9fbba977dc24080fe60fd7a80a542b1647f1de9091d7fd04a36eab695088d4d75fb836548e99b5f
2021-05-29 17:15:21 +03:00
Hennadii Stepanov
62cb8d98d2 qt: Drop BitcoinGUI* WalletFrame data member
This changes removes bitcoingui->walletframe->bitcoingui circular
dependency.

This commit does not change behavior.
2021-05-29 15:49:30 +03:00
Hennadii Stepanov
f73e5c972a qt: Move CreateWalletActivity connection from WalletFrame to BitcoinGUI
This changes remove some pointers to the BitcoinGUI instance that is
required for the next commits.

This commit does not change behavior.
2021-05-29 15:49:30 +03:00
Hennadii Stepanov
20e2e24e90 qt: Move WalletView connections from WalletFrame to BitcoinGUI
This changes remove some pointers to the BitcoinGUI instance that is
required for the next commits.

This commit does not change behavior.
2021-05-29 15:49:29 +03:00
bruno
ef99d03c2b test: check anchors.dat when node starts for the first time 2021-05-28 16:50:07 -03:00
apitko
947f9734da doc: add maxuploadtarget to bitcoin.conf example
Introduce the maxuploadtarget option to the example bitcoin.conf file. This adds visibility for this option which is useful to those looking to configure bandwidth usage.
2021-05-28 12:53:17 -04:00
Martin Zumsande
b6c5d1e450 p2p: AddrFetch - don't disconnect on self-announcements
Disconnecting an AddrFetch peer only after receiving an addr
message of size >1 prevents dropping them before
they had a chance to answer the getaddr request.
2021-05-28 18:29:05 +02:00
Hennadii Stepanov
123b401e0a Merge bitcoin-core/gui#275: Support runtime appearance adjustment on macOS
c231254a65 qt: Make TransactionView aware of runtime palette change (Hennadii Stepanov)
2b622d4ace qt: Make CoinControlDialog aware of runtime palette change (Hennadii Stepanov)
97a6b5e06a qt: Make OverviewPage aware of runtime palette change (Hennadii Stepanov)
d05f1b278d qt: Make UnitDisplayStatusBarControl aware of runtime palette change (Hennadii Stepanov)
6b2ce65392 qt: Replace base class of ClickableLabel with ThemedLabel (Hennadii Stepanov)
ff530a2093 qt: Use GUIUtil::ThemedLabel class (Hennadii Stepanov)
d99ef327a8 qt: Add GUIUtil::ThemedLabel class (Hennadii Stepanov)
c054720e08 qt: Make SignVerifyMessageDialog aware of runtime palette change (Hennadii Stepanov)
0dcc3fac43 qt: Make SendCoinsEntry aware of runtime palette change (Hennadii Stepanov)
fa18d28e12 qt: Make RPCConsole aware of runtime palette change (Hennadii Stepanov)
f1083826e3 qt: Make BitcoinGUI aware of runtime palette change (Hennadii Stepanov)
ce17861dc4 qt: Make PlatformStyle aware of runtime palette change (Hennadii Stepanov)

Pull request description:

  On macOS switching appearance (Light -> Dark or Dark -> Light) when Bitcoin Core is running makes the GUI pretty unusable.
  This bug is especially important when a user chose the "Auto" mode to adjust appearance automatically.

  This PR fixes Bitcoin Core behavior.

  This is an alternative to #268.

ACKs for top commit:
  Sjors:
    tACK c231254a65 on macOS 11.4
  goums:
    ACK c231254a65
  promag:
    Tested ACK c231254a65 on macOS Big Sur arm64.
  jarolrod:
    tACK c231254a65

Tree-SHA512: 122dda3e4c9703f68cec60613c536ca59d04c93f2c03398559f2361b8d279ae534800e8e677d94a33e10e769d00be54295a704e98afa2e986a06146b9f164854
2021-05-28 18:07:44 +03:00
Vasil Dimov
a92485b2c2 addrman: use unordered_map instead of map
`CAddrMan` uses `std::map` internally even though it does not require
that the map's elements are sorted. `std::map`'s access time is
`O(log(map size))`. `std::unordered_map` is more suitable as it has a
`O(1)` access time.

This patch lowers the execution times of `CAddrMan`'s methods as follows
(as per `src/bench/addrman.cpp`):

```
AddrMan::Add(): -3.5%
AddrMan::GetAddr(): -76%
AddrMan::Good(): -0.38%
AddrMan::Select(): -45%
```
2021-05-28 16:40:15 +02:00
Kristaps Kaupe
91ef8344d4 Additional test vector for hardened derivation with leading zeros 2021-05-28 17:22:01 +03:00
fanquake
6c3fcd5591 test: remove BasicTestingSetup from util_threadnames unit tests 2021-05-28 19:14:09 +08:00
fanquake
b53d3c1b1f test: remove BasicTestingSetup from uint256 unit tests 2021-05-28 19:14:09 +08:00
fanquake
c0497a4928 test: remove BasicTestingSetup from torcontrol unit tests 2021-05-28 19:14:09 +08:00
fanquake
ef8bb0473b test: remove BasicTestingSetup from sync unit tests 2021-05-28 19:14:09 +08:00
fanquake
1aee83421f test: remove BasicTestingSetup from reverse_lock unit tests 2021-05-28 19:14:09 +08:00
fanquake
57ba949ef5 test: remove BasicTestingSetup from policy_fee unit tests 2021-05-28 19:14:09 +08:00
fanquake
3974c962b6 test: remove BasicTestingSetup from merkleblock tests 2021-05-28 19:14:08 +08:00
fanquake
cd5bc4b470 test: remove BasicTestingSetup from hash unit tests 2021-05-28 19:14:08 +08:00
fanquake
39cec22935 test: remove BasicTestingSetup from compilerbug unit tests 2021-05-28 19:14:08 +08:00
fanquake
6d3b78c0e2 test: remove BasicTestingSetup from bswap unit tests 2021-05-28 19:14:08 +08:00
fanquake
a13dc24831 test: remove BasicTestingSetup from bech32 unit tests 2021-05-28 19:14:08 +08:00
fanquake
f4dcbe4498 test: remove BasicTestingSetup from base64 unit tests 2021-05-28 19:14:08 +08:00
fanquake
fd144f6426 test: remove BasicTestingSetup from base32 unit tests 2021-05-28 19:14:07 +08:00
fanquake
4c389ba04b test: remove BasicTestingSetup from arith_uint256 unit tests 2021-05-28 19:14:07 +08:00
fanquake
05590651a0 test: remove BasicTestingSetup from amount unit tests 2021-05-28 19:14:07 +08:00
fanquake
883a5c7d02 test: remove BasicTestingSetup from allocator unit tests 2021-05-28 19:14:07 +08:00
fanquake
8115c2ad7d Merge bitcoin/bitcoin#22078: Add src/qt/android/.gitignore
7d07192dde Add src/qt/android/.gitignore (Hennadii Stepanov)

Pull request description:

  This PR makes `git` ignore files created by `make apk`.

ACKs for top commit:
  icota:
    ACK 7d07192dde

Tree-SHA512: 4be20bd84830217a10d8ea7634799e71ed50be73f4f60c91c56311a2c95b22ff1f28d3b7bc077f1417318bb75e446e3fc3bdbf9dbc037b4cbc8428f0875f2c77
2021-05-28 14:14:34 +08:00
fanquake
a6d984c08c Merge bitcoin/bitcoin#22071: depends: latest config.guess and config.sub
7e2a9890e5 depends: latest config.sub (2021-04-30) (fanquake)
f16d4cd8c5 depends: latest config.guess (2021-05-24) (fanquake)

Pull request description:

  This is split out of #21851. Updating these files should be mechanical, and shouldn't have to wait for that PR. Also, having support in depends for the new `arm-apple-darwin` target (added in [2593751ef276497e312d7c4ce7fd049614c7bf80](https://git.savannah.gnu.org/cgit/config.git/commit/?id=2593751ef276497e312d7c4ce7fd049614c7bf80)) is useful when debugging. i.e #22070.

  If you try and compile depends for a `arm-apple-darwin` target using master, on a x86_64 darwin machine, currently you'll get:
  ```bash
  gmake -C depends -j9 HOST=arm64-apple-darwin
  Invalid configuration `arm64-apple-darwin': machine `arm64-apple' not recognized
  shasum: hosts/.mk: No such file or directory
  <omitted>
  Makefile:111: hosts/.mk: No such file or directory
  gmake: *** No rule to make target 'hosts/.mk'.  Stop.
  ```

ACKs for top commit:
  laanwj:
    ACK 7e2a9890e5

Tree-SHA512: 8ed99b5d486c6cbca8929a752460338b6ee17f6bf93013c76589605678853c3a01ebd631b4d3f5d6aaeb6e5c21b7bbe39afc4454d3a697fafb27678f6d2c021e
2021-05-28 14:10:19 +08:00
Jon Atack
7d1769bc45 p2p: remove torv2 from SetNetFromBIP155Network() 2021-05-28 01:46:29 +02:00
Jon Atack
eba9a94b9f fuzz: rename CNetAddr/CService deserialize targets
as the changes that follow are incompatible with the inputs.
2021-05-28 01:46:18 +02:00
Jon Atack
c56a1c9b18 p2p: drop onions from IsAddrV1Compatible(), no longer relay torv2 2021-05-28 01:42:21 +02:00
Jarol Rodriguez
dafab2b1b3 doc: improve note on choosing posix mingw32 2021-05-27 19:00:34 -04:00
Jon Atack
f8e94002fc p2p: remove torv2/ADDR_TORV2_SIZE from SetTor() 2021-05-27 22:54:39 +02:00
W. J. van der Laan
7257e50dba Merge bitcoin/bitcoin#20833: rpc/validation: enable packages through testmempoolaccept
13650fe2e5 [policy] detect unsorted packages (glozow)
9ef643e21b [doc] add release note for package testmempoolaccept (glozow)
c4259f4b7e [test] functional test for packages in RPCs (glozow)
9ede34a6f2 [rpc] allow multiple txns in testmempoolaccept (glozow)
ae8e6df709 [policy] limit package sizes (glozow)
c9e1a26d1f [fuzz] add ProcessNewPackage call in tx_pool fuzzer (glozow)
363e3d916c [test] unit tests for ProcessNewPackage (glozow)
cd9a11ac96 [test] make submit optional in CreateValidMempoolTransaction (glozow)
2ef187941d [validation] package validation for test accepts (glozow)
578148ded6 [validation] explicit Success/Failure ctors for MempoolAcceptResult (glozow)
b88d77aec5 [policy] Define packages (glozow)
249f43f3cc [refactor] add option to disable RBF (glozow)
897e348f59 [coins/mempool] extend CCoinsViewMemPool to track temporary coins (glozow)
42cf8b25df [validation] make CheckSequenceLocks context-free (glozow)

Pull request description:

  This PR enables validation dry-runs of packages through the `testmempoolaccept` RPC. The expectation is that the results returned from `testmempoolaccept` are what you'd get from test-then-submitting each transaction individually, in that order (this means the package is expected to be sorted in topological order, for now at least). The validation is also atomic: in the case of failure, it immediately halts and may return "unfinished" `MempoolAcceptResult`s for transactions that weren't fully validated. The API for 1 transaction stays the same.

  **Motivation:**
  - This allows you to test validity for transaction chains (e.g. with multiple spending paths and where you don't want to broadcast yet); closes #18480.
  - It's also a first step towards package validation in a minimally invasive way.
  - The RPC commit happens to close #21074 by clarifying the "allowed" key.

  There are a few added restrictions on the packages, mostly to simplify the logic for areas that aren't critical to main package use cases:
  - No package can have conflicts, i.e. none of them can spend the same inputs, even if it would be a valid BIP125 replacement.
  - The package cannot conflict with the mempool, i.e. RBF is disabled.
  - The total count of the package cannot exceed 25 (the default descendant count limit), and total size cannot exceed 101KvB (the default descendant size limit).

  If you're looking for review comments and github isn't loading them, I have a gist compiling some topics of discussion [here](https://gist.github.com/glozow/c3acaf161c95bba491fce31585b2aaf7)

ACKs for top commit:
  laanwj:
    Code review re-ACK 13650fe2e5
  jnewbery:
    Code review ACK 13650fe2e5
  ariard:
    ACK 13650fe

Tree-SHA512: 8c5cbfa91a6c714e1c8710bb281d5ff1c5af36741872a7c5df6b24874d6272b4a09f816cb8a4c7de33ef8e1c2a2c252c0df5105b7802f70bc6ff821ed7cc1a2f
2021-05-27 22:40:24 +02:00
W. J. van der Laan
2e8f3928f1 Merge bitcoin/bitcoin#22017: Update Windows code signing certificate
167fb1fc72 Update Windows code signing certificate (Andrew Chow)

Pull request description:

  Updates the Windows code signing certificate to a new one issued by Digicert. This certificate has been issued to Bitcoin Core Code Signing LLC registered in Delaware, US. Note that this is different from the previous Bitcoin Core Code Signing Association registered in Zurich, Switzerland as it was unable to meet the validation requirements in time.

ACKs for top commit:
  Sjors:
    utACK 167fb1f
  laanwj:
    ACK 167fb1fc72

Tree-SHA512: 8d5308c710ef94330417955b9bc82c5894d282798cebece82b84b425e3354e566aa6a68693ec359391ea40ddd7e2032d35ce28d104683d75ec3010ddf00be209
2021-05-27 21:51:58 +02:00
Hennadii Stepanov
1b66f6e556 qt: Drop PeerTablePriv class
This commit does not change behavior.
2021-05-27 22:34:02 +03:00
Hennadii Stepanov
efb7e5aa96 qt, refactor: Use default arguments for overridden functions
See Qt docs for QAbstractTableModel and QAbstractItemModel classes.
2021-05-27 22:33:04 +03:00
Carl Dong
7a799c9c2b index: refactor-only: Reuse CChain ref 2021-05-27 13:50:11 -04:00
Carl Dong
db33cde80f index: Add chainstate member to BaseIndex 2021-05-27 13:50:11 -04:00
Carl Dong
f4a47a1feb bench: Use existing chainman in AssembleBlock 2021-05-27 13:50:11 -04:00
Carl Dong
91226eb917 bench: Use existing NodeContext in DuplicateInputs 2021-05-27 13:50:11 -04:00
Carl Dong
e6b4aa6eb5 miner: Pass in chainman to RegenerateCommitments
Pass in chainman instead of prev_block so that we can enforce the
block.hashPrevBlock refers to prev_block invariant in the function
itself.

We should probably rethink BlockAssembler's API and somehow include
commitment regeneration functionality in there. Something like a variant
of CreateNewBlock that takes in a std::vector<TxRef> and return a CBlock
instead of CBlockTemplate. That could avoid reaching for
LookupBlockIndex at all.
2021-05-27 13:50:11 -04:00
Carl Dong
9ecade1425 rest: Add GetChainman function and use it
This is not the cleanest change but:

1. It fixes the erroneous use of RPC's Ensure*() in rest.cpp, which
   cause crashes in REST contexts.

   RPC code wraps all calls in a try/except, REST code does not.
   Ensure*(), being part of RPC, expects that its throw's will get
   caught by a try/except. But if you use Ensure*() in REST code, since
   it doesn't have a try/except wrap, a crash will happen.

2. It is consistent with other functions like GetMemPool.

Someone can probably make this a bit prettier.
2021-05-27 13:49:09 -04:00
Jon Atack
0f1c58ae87 test: update feature_proxy to torv3 2021-05-27 19:47:52 +02:00
W. J. van der Laan
ea1e5c2c71 Merge bitcoin/bitcoin#22025: refactor: Group and re-order CAddrMan members by access type
8caf60dbbe move-only: Group and re-order CAddrMan members by access type (Hennadii Stepanov)
5cd7f8abe3 refactor: Do not expose CAddrMan members as protected without need (Hennadii Stepanov)

Pull request description:

  This PR is split from #19238 as all of its commits are trivial to review.
  The last commit is easy to review with `git diff --color-moved=dimmed-zebra`.

  Addressed the following comments from #19238:
  - 130b82ff35 (r550865131)
  > Can you consolidate all the private members and protected members to be next to each other? Multiple private and protected access specifiers make this harder to read than is necessary.

  - 130b82ff35 (r557271783)
  > Yeah, class declaration is easier to read if there is just one instance of `public:`, `protected:` and `private:` (in that order).

ACKs for top commit:
  jnewbery:
    ACK 8caf60dbbe
  laanwj:
    Code review ACK 8caf60dbbe
  jarolrod:
    ACK 8caf60dbbe
  vasild:
    ACK 8caf60dbbe

Tree-SHA512: e6127fc658da7876e36f22e2fae162dc19502ed7f8e931fdebc827dabd627e5346c6fbe6f6d0cd27fd3e5c96690ff35022ff6b48f2747b748ebd66a45c851c2b
2021-05-27 15:52:38 +02:00
W. J. van der Laan
e20745c1bd Merge bitcoin/bitcoin#22029: [fuzz] Improve transport deserialization fuzz test coverage
e337145577 [fuzz] Occasional valid magic bytes for transport serialization test (Dhruv Mehta)
35571d8d9e [fuzz] Occasional valid checksum for transport serialization fuzz test (Dhruv Mehta)
654472a461 [fuzz] Add serialization to deserialization test (Dhruv Mehta)

Pull request description:

  This PR has 3 commits that increase the fuzz test coverage:

  Before commit 1:
  ```
  #306853 REDUCE cov: 798 ft: 5820 corp: 150/375Kb lim: 68333 exec/s: 1382 rss: 461Mb L: 254/63171 MS: 1 EraseBytes-
  #1453105 REDUCE cov: 798 ft: 5820 corp: 150/369Kb lim: 79613 exec/s: 1467 rss: 461Mb L: 6027/60873 MS: 1 EraseBytes-
  ```

  After commit 1 (adds serialization to de-serialization test):
  ```
  #303389 NEW cov: 1202 ft: 8382 corp: 157/382Kb lim: 68189 exec/s: 1451 rss: 447Mb L: 1386/65459 MS: 1 CopyPart-
  #1428759 REDUCE cov: 1202 ft: 8512 corp: 169/389Kb lim: 78749 exec/s: 1528 rss: 463Mb L: 1627/60488 MS: 1 EraseBytes-
  ```

  After commit 2 (provides an occasional checksum assist to the fuzzer inputs):
  ```
  #304820 NEW cov: 1440 ft: 4452 corp: 92/12551b lim: 2237 exec/s: 3386 rss: 486Mb L: 47/1111 MS: 1 ChangeByte-
  #1416181 REDUCE cov: 1442 ft: 5681 corp: 125/59Kb lim: 4096 exec/s: 3522 rss: 535Mb L: 2164/4049 MS: 1 EraseBytes-
  ```

  After commit 3 (provides an occasional magic bytes assist to the fuzzer inputs):
  ```
  #302684 NEW cov: 1454 ft: 3936 corp: 84/7056b lim: 2424 exec/s: 4146 rss: 477Mb L: 65/1108 MS: 3 CopyPart-CrossOver-CMP- DE: "\x0e\x00\x00\x00"-
  #1383925 REDUCE cov: 1454 ft: 4828 corp: 102/14573b lim: 4096 exec/s: 3954 rss: 534Mb L: 116/4050 MS: 2 EraseBytes-ChangeByte-
  ```

  If reviewers only accept the first commit, the seeds are not invalidated and new seeds are at: https://github.com/bitcoin-core/qa-assets/pull/61. In this case, we can also revert the test name change.

  If reviewers accept all three commits, the existing seeds are invalidated.

ACKs for top commit:
  practicalswift:
    Tested ACK e337145577

Tree-SHA512: d37f06eea0249322b00a99c4827359eb53aeb711751e5571f4681eeca06dc257e0c4cd4887150fc37cc2f689e26986112d768066ad274361615ba9b6a522c61a
2021-05-27 15:02:57 +02:00
Sjors Provoost
1c4b456e1a gui: send using external signer 2021-05-27 14:37:59 +02:00
Hennadii Stepanov
7076bba841 Merge bitcoin-core/gui#346: English translations update
df4c81fda4 English translations update (Hennadii Stepanov)
bfb53ddda9 scripted-diff: Fix ellipsis after pr20773 (Hennadii Stepanov)

Pull request description:

  Update for Transifex.

  After changing translator comments in #332 this update will show if Transifex triggers strings to be re-translated.

ACKs for top commit:
  laanwj:
    ACK df4c81fda4
  jarolrod:
    ACK df4c81fda4

Tree-SHA512: 1e54812bc04db6ae39e0b4d735b220ed8730a9941b17a0a2d09e21bcdd08e829adba86c35cf43c9be5e492ccb13e53a90149fcd7d6c0f5fdd022b978a1ff785c
2021-05-27 15:31:42 +03:00
Sjors Provoost
24815c6309 gui: wallet creation detects external signer 2021-05-27 14:01:54 +02:00
Sjors Provoost
3f845ea299 node: add externalSigners to interface 2021-05-27 14:01:54 +02:00
Sjors Provoost
62ac119f91 gui: display address on external signer 2021-05-27 14:01:54 +02:00
Sjors Provoost
450cb40a34 wallet: add displayAddress to interface 2021-05-27 14:01:54 +02:00
Sjors Provoost
eef8d64529 gui: create wallet with external signer 2021-05-27 14:01:53 +02:00
Sjors Provoost
6cdbc83e93 gui: add external signer path to options dialog 2021-05-27 14:01:53 +02:00
Hennadii Stepanov
fa416cea84 depends: Bump Qt version to 5.12.11 2021-05-27 14:53:18 +03:00
W. J. van der Laan
22b845291c Merge bitcoin/bitcoin#22063: build: Use Qt archive of the same version as the compiled binaries
cb7eba2a57 build: Use Qt archive of the same version as the compiled binaries (Hennadii Stepanov)

Pull request description:

  This PR fixes broken Android APK build when the `depends/sources` directory contains Qt source archives of different versions (e.g., Qt version update [pull request](https://github.com/bitcoin/bitcoin/pull/22054) in CI with the cached `depends/sources` directory).

  This is an alternative to #22058.

ACKs for top commit:
  MarcoFalke:
    review ACK cb7eba2a57
  laanwj:
    Code review ACK cb7eba2a57

Tree-SHA512: cf63a9809fba5cb13719d7e7bb5afc718a2cff5233b0670d30d30a0018d91278fcfc2a1b9ae8b84e8e3a52c95157bc465603cc754bb8a9d1a3d62415f01ad70f
2021-05-27 13:48:56 +02:00
Hennadii Stepanov
df4c81fda4 English translations update 2021-05-27 13:47:26 +03:00
Hennadii Stepanov
bfb53ddda9 scripted-diff: Fix ellipsis after pr20773
-BEGIN VERIFY SCRIPT-
sed -i 's|_("Loading wallet...")|_("Loading wallet…")|' src/wallet/*.cpp
-END VERIFY SCRIPT-
2021-05-27 13:46:36 +03:00
Hennadii Stepanov
b789914f17 Merge bitcoin-core/gui#332: Replace disambiguation strings with translator comments
8b77133651 qt: Replace disambiguation strings with translator comments (Hennadii Stepanov)

Pull request description:

  Since https://github.com/bitcoin/bitcoin/pull/21694 is merged, translator comments is the right way to pass context to translators.

  This PR fixes changes were made:
  - in #220 before https://github.com/bitcoin/bitcoin/pull/21694
  - in https://github.com/bitcoin/bitcoin/pull/21694 on testing purpose
  - in #125

  Closes #288.

ACKs for top commit:
  jarolrod:
    ACK 8b77133651

Tree-SHA512: 466ade35f4969a41fbf3196780b1ae9fa810bab5d2f09077f8631604636cc63b24a901c719f6b5797366d2aa307993d0aa419ce35200c8d0a741a3d81cad3e6b
2021-05-27 13:19:39 +03:00
W. J. van der Laan
811aa24c71 Merge bitcoin/bitcoin#22060: contrib: add torv3 seed nodes for testnet, drop v2 ones
6fe0516858 contrib: add torv3 seed nodes for testnet, drop v2 ones (Jon Atack)

Pull request description:

  Replace the ancient (2015) Tor V2 hardcoded seeds with new Tor V3 ones. This needs to be done before 0.22 to make sure onion-only testnet nodes can still connect to the network. Continues #21560.

  Ways to test:
  - Re-generate ` src/chainparamsseeds.h` with `cd contrib/seeds && python3 generate-seeds.py . > ../../src/chainparamsseeds.h`, check if git tree stays the same.
  - Create a new testnet node with `bitcoind -testnet -onlynet=onion -proxy=127.0.0.1:9050` (or delete `~/.bitcoin/testnet3/peers.dat`), check if it is able to connect to the network and get blocks.
  - Check if the addresses are connectable for ex.:
  ```python3
  #!/usr/bin/env python3
  import subprocess
  with open('contrib/seeds/nodes_test.txt') as f:
      for line in (line for line in (line.rstrip().split('#', 1)[0] for line in f) if line):
          subprocess.call(["nc", "-v", "-x", "127.0.0.1:9050", "-z"] + line.split(':'))
  ```

  Thanks to jonatack for providing the list.

ACKs for top commit:
  jonatack:
    ACK 6fe0516858

Tree-SHA512: 61bfdb44dfab9d02b75e5cb06c089a3b1a1fe7134875e1d09166c4116e961d809aa25422fe03f068876e9423b571ecc4a0c7a7eeacba4aac3b2768717f3ee6d6
2021-05-27 11:25:29 +02:00
Hennadii Stepanov
5c041cb348 Merge bitcoin-core/gui#311: Peers Window rename 'Peer id' to 'Peer'
657b33ef2d qt: add translator comments for peers table columns (Jarol Rodriguez)
73a91c63ec gui: rename "Peer Id" to "Peer" in tab column and details area (Jon Atack)

Pull request description:

  Picking up https://github.com/bitcoin-core/gui/pull/290

  **Original PR Description:**
  - renames the peers tab column header from `Peer Id` to `Peer` to allow resizing the column more tightly (this will be particularly useful after #256) and does the same for the peer details area.

  While here, we also add Qt translator comments for the Peer Table columns.

  | Master        | PR               |
  | ----------- | ----------- |
  | ![Screen Shot 2021-05-03 at 1 23 05 AM](https://user-images.githubusercontent.com/23396902/116843818-20a14b00-abaf-11eb-913e-ddff11cda5cd.png) | ![Screen Shot 2021-05-05 at 4 08 45 AM](https://user-images.githubusercontent.com/23396902/117112825-a2cc7380-ad57-11eb-939b-1aceb4214ad1.png) |

ACKs for top commit:
  jonatack:
    utACK 657b33ef2d
  hebasto:
    re-ACK 657b33ef2d

Tree-SHA512: f50116f7ca8719cadf1f95f45e3594b3b92bde9c43eb954f3e963ed10629dd9406efdb5e96aa1f750a926e24a96321d824ed3780bd9cd748774e0b85fd0c9535
2021-05-27 00:16:18 +03:00
Hennadii Stepanov
7d07192dde Add src/qt/android/.gitignore
This change makes git ignore files created by `make apk`.
2021-05-26 23:29:19 +03:00
W. J. van der Laan
456c8d6cd8 Merge bitcoin-core/gui#313: qt: Optimize string concatenation by default
a02c970eb0 qt, refactor: Revert explicit including QStringBuilder (Hennadii Stepanov)
3fd3a0fc87 qt, build: Optimize string concatenation (Hennadii Stepanov)

Pull request description:

  From [Qt docs](https://doc.qt.io/qt-5/qstring.html#more-efficient-string-construction):
  > ... multiple uses of the \[`QString`\] '+' operator usually means multiple memory allocations. When concatenating n substrings, where n > 2, there can be as many as n - 1 calls to the memory allocator.

  With this PR
  > ... the '+' will automatically be performed as the `QStringBuilder` '%' everywhere.

  The change in the `src/Makefile.qt.include` file does not justify submitting this PR into the main repo, IMHO.

ACKs for top commit:
  laanwj:
    Code review ACK a02c970eb0
  Talkless:
    utACK a02c970eb0, built successfully on Debian Sid with Qt 5.15.2, but did not check if any displayed strings are "wrong" after refactoring.
  jarolrod:
    ACK a02c970eb0

Tree-SHA512: cbb476ee96f27c3bd6e125efab74d8bf24bbdb4c30576b3feea45e203405f3bf5b497dd7d3e11361fc825fcbf4b893b152921a9efdeaf73b42d1865d85f0ae84
2021-05-26 15:01:18 +02:00
Russell Yanofsky
c7bd5842e4 MOVEONLY: CWallet transaction code out of wallet.cpp/.h
This commit just moves functions without making any changes. It can be
reviewed with `git log -p -n1 --color-moved=dimmed_zebra`

Motivation for this change is to make wallet.cpp/h less monolithic and
start to make wallet transaction state tracking comprehensible so bugs
in
https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking
can be fixed safely without introducing new problems.

This commit moves wallet classes and methods that deal with transactions
out of wallet.cpp/.h into better organized files:

- transaction.cpp/.h - CWalletTx and CMerkleTx class definitions
- receive.cpp/.h - functions checking received transactions and computing balances
- spend.cpp/.h - functions creating transactions and finding spendable coins

After #20773, when loading is separated from syncing it will also be
possible to move more wallet.cpp/.h functions to:

- sync.cpp/.h - functions handling chain notifications and rescanning

This commit arranges receive.cpp and spend.cpp functions in dependency
order so it's possible to skim receive.cpp and get an idea of how
computing balances works, and skim spend.cpp and get an idea of how
transactions are created, without having to jump all over wallet.cpp
where functions are not in order and there is a lot of unrelated code.

Followup commit "refactor: Detach wallet transaction methods" in
https://github.com/bitcoin/bitcoin/pull/21206 follows up this PR and
tweaks function names and arguments to reflect new locations. The two
commits are split into separate PRs because this commit is more work to
maintain and less work to review, while the other commit is less work to
maintain and more work to review, so hopefully this commit can be merged
earlier.
2021-05-26 06:32:51 -05:00
fanquake
ecddd12482 Merge bitcoin/bitcoin#18418: wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100
e6fe1c37d0 rpc: Improve avoidpartialspends and avoid_reuse documentation (Fabian Jahr)
8f073076b1 wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100 (Fabian Jahr)

Pull request description:

  Follow-up to #17824.

  This increases OUTPUT_GROUP_MAX_ENTRIES to 100 which means that OutputGroups will now be up to 100 outputs large, up from previously 10. The main motivation for this change is that during the PR review club on #17824 [several participants signaled](https://bitcoincore.reviews/17824.html#l-339) that 100 might be a better value here.

  I think fees should be manageable for users but more importantly, users should know what they can expect when using the wallet with this configuration, so I also tried to clarify the documentation on `-avoidpartialspends` and `avoid_reuse` a bit. If there are other additional ways how or docs where users can be made aware of the potential consequences of using these parameters, please let me know. Another small upside is that [there seem to be a high number of batching transactions with 100 and 200 inputs](https://miro.medium.com/max/3628/1*sZ5eaBSbsJsHx-J9iztq2g.png)([source](https://medium.com/@hasufly/an-analysis-of-batching-in-bitcoin-9bdf81a394e0)) giving these transactions a bit of a larger anonymity set, although that is probably a very weak argument.

ACKs for top commit:
  jnewbery:
    ACK e6fe1c37d0
  Xekyo:
    retACK e6fe1c37d0
  rajarshimaitra:
    tACK `e6fe1c3`
  achow101:
    ACK e6fe1c37d0
  glozow:
    code review ACK e6fe1c37d0

Tree-SHA512: 79685c58bafa64ed8303b0ecd616fce50fc9a2b758aa79833e4ad9f15760e09ab60c007bc16ab4cbc4222e644cfd154f1fa494b0f3a5d86faede7af33a6f2826
2021-05-26 19:32:51 +08:00
Hennadii Stepanov
3ec033ed32 Merge bitcoin-core/gui#121: Early subscribe core signals in transaction table model
cafef080a2 qt: Refactor to remove unnecessary block in DispatchNotifications (João Barbosa)
57785fb7f6 qt: Early subscribe core signals in transaction table model (João Barbosa)
c6cbdf1a90 qt: Refactor ShowProgress to DispatchNotifications (João Barbosa)
3bccd50ad2 qt: Set flag after inital load on transaction table model (João Barbosa)

Pull request description:

  This fixes the case where transaction notifications arrive between `getWalletTxs` and `subscribeToCoreSignals`. Basically notifications are queued until `getWalletTxs` and wallet rescan complete.

  This is also a requirement to call `getWalletTxs` in a background thread.

  Motivated by https://github.com/bitcoin/bitcoin/issues/20241.

ACKs for top commit:
  jonatack:
    tACK cafef080a2
  ryanofsky:
    Code review ACK cafef080a2. Only change since last review is splitting commits and replacing m_progress with m_loading.
  meshcollider:
    Code review ACK cafef080a2

Tree-SHA512: 003caab2f2ae3522619711c8d02d521d2b8f7f280a467f6c3d08abf37ca81cc66b4b9fa10acfdf34e5fe250da7b696cfeec435f72b53c1ea97ccda96d8b4be33
2021-05-26 13:30:17 +03:00
W. J. van der Laan
707ba8692b Merge bitcoin/bitcoin#21966: Remove double serialization; use software encoder for fee estimation
66545da200 Remove support for double serialization (Pieter Wuille)
fff1cae43a Convert uses of double-serialization to {En,De}codeDouble (Pieter Wuille)
afd964d70b Convert existing float encoding tests (Pieter Wuille)
bda33f98e2 Add unit tests for serfloat module (Pieter Wuille)
2be4cd94f4 Add platform-independent float encoder/decoder (Pieter Wuille)
e40224d0c7 Remove unused float serialization (MarcoFalke)

Pull request description:

  Based on #21981.

  This adds a software-based platform-independent float/double encoder/decoder (platform independent in the sense that it only uses arithmetic and library calls, but never inspects the binary representation). This should strengthen our guarantee that encoded float/double values are portable across platforms. It then removes the functionality to serialize doubles from serialize.h, and replaces its only (non-test) use for fee estimation data serialization with the software encoder.

  At least on x86/ARM, the only difference should be how certain NaN values are encoded/decoded (but not *whether* they are NaN or not).

  It comes with tests that verify on is_iec559 platforms (which are the only ones we support, at least for now) that the serialized bytes exactly match the binary representation of floats in memory (for non-NaN).

ACKs for top commit:
  laanwj:
    Code review re-ACK 66545da200
  practicalswift:
    cr re-ACK 66545da200

Tree-SHA512: 62ad9adc26e28707b2eb12a919feefd4fd10cf9032652dbb1ca1cc97638ac21de89e240858e80d293d5112685c623e58affa3d316a9783ff0e6d291977a141f5
2021-05-26 10:16:41 +02:00
fanquake
7aa41fc581 Merge bitcoin/bitcoin#22042: Replace size/weight estimate tuple with struct for named fields
881a3e2e17 Replace size/weight estimate tuple with struct for named fields (Gregory Sanders)

Pull request description:

  For clarity of return values of size estimation functions.

ACKs for top commit:
  MarcoFalke:
    review ACK 881a3e2e17
  practicalswift:
    cr ACK 881a3e2e17

Tree-SHA512: 9aa97e3774fcaf1e5b905259418d827051325c2ad76d6c7388ad7c687c605c024a6c5d2f7e9565f4c180ad1a9f6b179557fcb8b23371ff1a4f3b81300ac1f3d1
2021-05-26 14:24:50 +08:00
MarcoFalke
48c86eca02 Merge bitcoin/bitcoin#22069: fuzz: don't try and use fopencookie() when building for Android
1be6267ce1 fuzz: don't try and use fopencookie when building for Android (fanquake)

Pull request description:

  When building for Android, `_GNU_SOURCE` will be defined:
  ```bash
  /home/ubuntu/android-sdk/ndk/22.1.7171670/toolchains/llvm/prebuilt/linux-x86_64/bin/aarch64-linux-android30-clang++ -dM -E -x c++ - < /dev/null
  #define _GNU_SOURCE 1
  #define _LP64 1
  #define __AARCH64EL__ 1
  #define __AARCH64_CMODEL_SMALL__ 1
  #define __ANDROID_API__ 30
  #define __ANDROID__ 1
  #define __ARM_64BIT_STATE 1
  .....
  ```
  but it doesn't have the [`fopencookie()` function](https://www.gnu.org/software/libc/manual/html_node/Streams-and-Cookies.html), or define the `cookie_io_functions_t` type, which results in compile failures:
  ```bash
  In file included from test/fuzz/addition_overflow.cpp:7:
  ./test/fuzz/util.h:388:15: error: unknown type name 'cookie_io_functions_t'
          const cookie_io_functions_t io_hooks = {
                ^
  15 warnings and 1 error generated.
  ```

  Just skip trying to use it if we are building for Android. Should fix #22062.

ACKs for top commit:
  practicalswift:
    cr ACK 1be6267ce1

Tree-SHA512: d62f63d0624af04b76c7e07b0332c71eca2bf9cd9e096a60aea9e212b7bbc1548e9fa9a76d065ec719bb345fe8726619c3bd2d0631f54d877c82972b7b289321
2021-05-26 07:44:34 +02:00
MarcoFalke
35b83e6e43 Merge bitcoin/bitcoin#22056: doc: describe in fuzzing.md how to reproduce a CI crash
d8f1ea7227 doc: describe in fuzzing.md how to reproduce a CI crash (Jon Atack)

Pull request description:

  Not sure if this is 100% accurate or missing any pertinent info, but I misremembered how to do this today and it seems like useful information to provide.

ACKs for top commit:
  practicalswift:
    ACK d8f1ea7227

Tree-SHA512: 1b74e4187e6ea13b04eb03b3c6e2615c4eb18cc38cce215ad1645f8b135c5c31a243748eb313ccec05f1f62187ba33d550119acf07088968d2d2c1c09bc4c653
2021-05-26 07:35:01 +02:00
fanquake
7e2a9890e5 depends: latest config.sub (2021-04-30) 2021-05-26 13:30:18 +08:00
fanquake
f16d4cd8c5 depends: latest config.guess (2021-05-24) 2021-05-26 13:28:47 +08:00
fanquake
3caedb4c03 build: don't use cf-protection when targeting arm-apple-darwin
After two reports on IRC of issues building depends on an Apple M1
machine, it turns out that this option can't be used when targeting
arm-apple-darwin. For now, just use it for x86_64-apple-darwin.

```bash
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.4.0

error: option 'cf-protection=return' cannot be specified on this target
error: option 'cf-protection=branch' cannot be specified on this target
2 errors generated.
```
2021-05-26 13:08:16 +08:00
fanquake
2e7c3461c0 Merge bitcoin/bitcoin#22061: ci: Bump multiprocess memory
fa0bfc5239 ci: Bump multiprocess memory (MarcoFalke)

Pull request description:

  Fixes #22059

ACKs for top commit:
  ryanofsky:
    Code review ACK fa0bfc5239. Thanks for the update, and interesting to know about #21869. It looks like relevant build https://cirrus-ci.com/task/4807455453478912 is succeeding too
  fanquake:
    ACK fa0bfc5239

Tree-SHA512: f6e49aadf33199ffa7960c8da0b81bdc5ffea61f373e1b0367d000cdbd214614374b9f1a8b3ce9b8270e6d13a24a2029ab07bddb48e44c86dcb687d645e5ef34
2021-05-26 11:26:26 +08:00
fanquake
1be6267ce1 fuzz: don't try and use fopencookie when building for Android
When building for Android, _GNU_SOURCE will be defined, but it doesn't
actually have the fopencookie() function, or define the
cookie_io_functions_t type.

For now just skip trying to use it if we are building for Android.

Should fix #22062.
2021-05-26 11:07:47 +08:00
Carl Dong
108a6be92a guix: Check for disk space availability before building 2021-05-25 22:37:50 -04:00
Jarol Rodriguez
657b33ef2d qt: add translator comments for peers table columns
Adds Qt Translator Comments to each Peers Table column to aid translators by providing context.
2021-05-25 21:09:39 -04:00
Gregory Sanders
881a3e2e17 Replace size/weight estimate tuple with struct for named fields 2021-05-26 07:33:09 +08:00
Hennadii Stepanov
3ad1b8899b Merge bitcoin-core/gui#297: Avoid unnecessary translations
19d51a2907 qt: Avoid unnecessary translations (Hennadii Stepanov)

Pull request description:

  Working on translation, I found these translations introduced in #79, that are unnecessary (assuming the universal nature of the "BTC" string).

ACKs for top commit:
  jarolrod:
    ACK 19d51a2907

Tree-SHA512: b45551a54a323c5ba3779f4c1d7c8e7ec4d19a2e95fe70153f48234393bf1449a08e6bd24519ec035ebd4a98080a56af45e7a21546b47152e493b8e1b8f4345e
2021-05-26 01:41:33 +03:00
practicalswift
37371268d1 Mark CheckTxInputs [[nodiscard]] (out-param txfee only set if call is successful). Avoid UUM in fuzzing harness coins_view. 2021-05-25 21:09:05 +00:00
Carl Dong
d7dec89091 guix: Remove dest if OUTDIR mv fails 2021-05-25 16:40:46 -04:00
Hennadii Stepanov
cb7eba2a57 build: Use Qt archive of the same version as the compiled binaries
This change fixes broken Android APK build when the `depends/sources`
directory contains Qt source archives of different versions (e.g., Qt
version update pull request in CI with the cached `depends/sources`
directory).
2021-05-25 23:06:39 +03:00
MarcoFalke
8600934018 Merge bitcoin/bitcoin#22057: test: use MiniWallet (P2PK mode) for feature_dersig.py
3e05a57297 test: use MiniWallet (P2PK mode) for feature_dersig.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (feature_dersig.py) to be run even with the Bitcoin Core wallet disabled. A valid DER-signature is created by using the recently introduced P2PK-Mode of the MiniWallet (#21945).

ACKs for top commit:
  MarcoFalke:
    cr ACK 3e05a57297

Tree-SHA512: 0fb8da8ed8b47f68bcb57301eb4f0171a6c9e44539b7554626969347e5d6f80b3b9085f2cc160cd038a990f0d81b8b614846260fbed43b5f950d77f1b7aa81cf
2021-05-25 17:32:00 +02:00
Dhruv Mehta
e337145577 [fuzz] Occasional valid magic bytes for transport serialization test
Before commit:
Unable to deserialize : 0%
Wrong message start   : ~45.62%
Header too large      : ~14.5%
Wrong checksum        : ~38.13%
Invalid message type  : ~1.78%

304820	NEW    cov: 1440 ft: 4452 corp: 92/12551b lim: 2237 exec/s: 3386 rss: 486Mb L: 47/1111 MS: 1 ChangeByte-
1416181	REDUCE cov: 1442 ft: 5681 corp: 125/59Kb lim: 4096 exec/s: 3522 rss: 535Mb L: 2164/4049 MS: 1 EraseBytes-

After commit:
Unable to deserialize : 0%
Wrong message start   : ~39.6%
Header too large      : ~30.85%
Wrong checksum        : ~25.54%
Invalid message type  : ~4.01%

302684	NEW    cov: 1454 ft: 3936 corp: 84/7056b lim: 2424 exec/s: 4146 rss: 477Mb L: 65/1108 MS: 3 CopyPart-CrossOver-CMP- DE: "\x0e\x00\x00\x00"-
1383925	REDUCE cov: 1454 ft: 4828 corp: 102/14573b lim: 4096 exec/s: 3954 rss: 534Mb L: 116/4050 MS: 2 EraseBytes-ChangeByte-
2021-05-25 08:20:43 -07:00
Dhruv Mehta
35571d8d9e [fuzz] Occasional valid checksum for transport serialization fuzz test
Before commit:
Unable to deserialize: 0%
Wrong message start  : ~1.27%
Header too large     : ~0.5%
Wrong checksum       : ~67.99%
Invalid message type : ~30.1%

303389	NEW    cov: 1202 ft: 8382 corp: 157/382Kb lim: 68189 exec/s: 1451 rss: 447Mb L: 1386/65459 MS: 1 CopyPart-
1428759	REDUCE cov: 1202 ft: 8512 corp: 169/389Kb lim: 78749 exec/s: 1528 rss: 463Mb L: 1627/60488 MS: 1 EraseBytes-

After commit(new seeds; old seeds invalidated):
Unable to deserialize: 0%
Wrong message start  : ~45.62%
Header too large     : ~14.5%
Wrong checksum       : ~38.13%
Invalid message type : ~1.78%

304820	NEW    cov: 1440 ft: 4452 corp: 92/12551b lim: 2237 exec/s: 3386 rss: 486Mb L: 47/1111 MS: 1 ChangeByte-
1416181	REDUCE cov: 1442 ft: 5681 corp: 125/59Kb lim: 4096 exec/s: 3522 rss: 535Mb L: 2164/4049 MS: 1 EraseBytes-
2021-05-25 08:09:14 -07:00
Dhruv Mehta
654472a461 [fuzz] Add serialization to deserialization test
Before commit:
306853	REDUCE cov: 798 ft: 5820 corp: 150/375Kb lim: 68333 exec/s: 1382 rss: 461Mb L: 254/63171 MS: 1 EraseBytes-
1453105	REDUCE cov: 798 ft: 5820 corp: 150/369Kb lim: 79613 exec/s: 1467 rss: 461Mb L: 6027/60873 MS: 1 EraseBytes-

After commit:
303389	NEW    cov: 1202 ft: 8382 corp: 157/382Kb lim: 68189 exec/s: 1451 rss: 447Mb L: 1386/65459 MS: 1 CopyPart-
1428759	REDUCE cov: 1202 ft: 8512 corp: 169/389Kb lim: 78749 exec/s: 1528 rss: 463Mb L: 1627/60488 MS: 1 EraseBytes-
2021-05-25 08:08:34 -07:00
MarcoFalke
fa0bfc5239 ci: Bump multiprocess memory 2021-05-25 16:57:29 +02:00
Jon Atack
6fe0516858 contrib: add torv3 seed nodes for testnet, drop v2 ones 2021-05-25 16:07:56 +02:00
Samuel Dobson
6b254814c0 Merge bitcoin/bitcoin#17331: Use effective values throughout coin selection
51a3ac242c Have OutputGroup determine the value to use (Andrew Chow)
6d6d278475 Change SelectCoins_test to actually test SelectCoins (Andrew Chow)
9d3bd74ab4 Remove CreateTransaction while loop and some related variables (Andrew Chow)
6f0d5189af Remove use_bnb and bnb_used (Andrew Chow)
de26eb0e1f Do both BnB and Knapsack coin selection in SelectCoinsMinConf (Andrew Chow)
01dc8ebda5 Have KnapsackSolver actually use effective values (Andrew Chow)
bf26e018de Roll static tx fees into nValueToSelect instead of having it be separate (Andrew Chow)
cc3f14b27c Move output reductions for fee to after coin selection (Andrew Chow)
d97d25d950 Make cost_of_change part of CoinSelectionParams (Andrew Chow)
af5867c896 Move some calculations to common code in SelectCoinsMinConf (Andrew Chow)
1bf4a62cb6 scripted-diff: rename some variables (Andrew Chow)

Pull request description:

  Changes `KnapsackSolver` to use effective values instead of just the nominal txout value. Since fees are taken into account during the selection itself, we finally get rid of the `CreateTransaction` loop as well as a few other things that only were only necessary because of that loop.

  This should not change coin selection behavior at all (except maybe remove weird edge cases that were caused by the loop). In order to keep behavior the same, `KnapsackSolver` will select outputs with a negative effective value (as it did before).

ACKs for top commit:
  ryanofsky:
    Code review ACK 51a3ac242c. Looks good to go!
  instagibbs:
    review ACK 51a3ac242c
  meshcollider:
    re-light-utACK 51a3ac242c

Tree-SHA512: 372c27e00edcd5dbf85177421ba88f20bfdaf1791b6e3dc022c44876ecc379403e2375ed69e71c512c49e6af87641001ff385c4b25ab93684b3a08a53bf3824e
2021-05-26 01:35:43 +12:00
Jon Atack
d8f1ea7227 doc: describe in fuzzing.md how to reproduce a CI crash
and add/improve a few headers
2021-05-25 15:18:50 +02:00
Sebastian Falbesoner
3e05a57297 test: use MiniWallet (P2PK mode) for feature_dersig.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-25 14:39:45 +02:00
fanquake
7041d256e3 Merge bitcoin/bitcoin#21788: build: Silence [-Wunused-command-line-argument] warnings
e9f948c727 build: Convert warnings into errors when testing for -fstack-clash-protection (Hennadii Stepanov)

Pull request description:

  Apple clang version 12.0.5 (clang-1205.0.22.9) that is a part of Xcode 12.5, and is based on LLVM clang 11.1.0, fires spammy warnings:

  ```
  clang: warning: argument unused during compilation: '-fstack-clash-protection' [-Wunused-command-line-argument]
  ```

  From the https://github.com/apple/llvm-project:
  ```
  $ git log --oneline | grep 'stack-clash-protection'
  00065d5cbd02 Revert "-fstack-clash-protection: Return an actual error when used on unsupported OS"
  4d59c8fdb955 -fstack-clash-protection: Return an actual error when used on unsupported OS
  df3bfaa39071 [Driver] Change -fnostack-clash-protection to  -fno-stack-clash-protection
  68e07da3e5d5 [clang][PowerPC] Enable -fstack-clash-protection option for ppc64
  515bfc66eace [SystemZ] Implement -fstack-clash-protection
  e67cbac81211 Support -fstack-clash-protection for x86
  454621160066 Revert "Support -fstack-clash-protection for x86"
  0fd51a4554f5 Support -fstack-clash-protection for x86
  658495e6ecd4 Revert "Support -fstack-clash-protection for x86"
  e229017732bc Support -fstack-clash-protection for x86
  b03c3d8c6209 Revert "Support -fstack-clash-protection for x86"
  4a1a0690ad68 Support -fstack-clash-protection for x86
  f6d98429fcdb Revert "Support -fstack-clash-protection for x86"
  39f50da2a357 Support -fstack-clash-protection for x86
  ```

  I suppose, that Apple clang-1205.0.22.9 ends with on of the "Revert..." commits.

  This PR prevents using of the `-fstack-clash-protection` flag if it causes warnings.

  ---

  System: macOS Big Sur 11.3 (20E232).

ACKs for top commit:
  jarolrod:
    re-ACK e9f948c727
  Sjors:
    tACK e9f948c727 on macOS 11.3.1

Tree-SHA512: 30186da67f9b0f34418014860c766c2e7f622405520f1cbbc1095d4aa4038b0a86014d76076f318a4b1b09170a96d8167c21d7f53a760e26017f486e1a7d39d4
2021-05-25 16:03:26 +08:00
MarcoFalke
40f7a2891f Merge bitcoin/bitcoin#22043: rpc, test: addpeeraddress test coverage, code simplify/constness
b36e0cd1b9 rpc: simplify addpeeraddress and improve code constness (Jon Atack)
6b1926cf1e test: addpeeraddress functional test coverage (Jon Atack)

Pull request description:

  - Add functional test coverage for rpc addpeeraddress
  - Simplify addpeeraddress and improve code constness

ACKs for top commit:
  klementtan:
    ACK [`b36e0cd`](b36e0cd1b9)
  MarcoFalke:
    review ACK b36e0cd1b9 💭

Tree-SHA512: 01773fb70f23db5abf46806bb27804e48feff27272b2e6582bd5b886e9715088eb2d84755106bce2ad6f88e21582f7f071a30a89d5b17286d899c3dd8553b4fc
2021-05-25 09:40:00 +02:00
MarcoFalke
db1aca01d5 Merge bitcoin/bitcoin#22021: rpc: bumpfee/psbtbumpfee fixes and updates
4f504f826b rpc: fix code comment for bumpfee/psbtbumpfee output (Jon Atack)
5cb7ac23fb rpc: fix docs for bumpfee psbt update (Jon Atack)

Pull request description:

  Follow-up to #21544 and #20891 for the `bumpfee_helper` used for RPCs bumpfee and psbtbumpfee:
  - "psbt" field is only returned in psbtbumpfee and not bumpfee
  - bumpfee raises if private keys are disabled, so the txid help "Only returned when wallet private keys are enabled." no longer makes sense; remove it
  - add missing space in RPC examples ("Bump the fee, get the new transaction'stxid")
  - update txid/psbt code comments

ACKs for top commit:
  klementtan:
    ACK [`4f504f8`](4f504f826b)

Tree-SHA512: 194faf8af52383eb8ac5cd22825265931bcde135dac79d8ecc4f84f698070da9b9373c00eef8623961881bb293157c7c9a0d71d1bcccf481ae3605a2d1444ed8
2021-05-25 09:00:27 +02:00
MarcoFalke
aeecb1c2eb Merge bitcoin/bitcoin#21992: p2p: Remove -feefilter option
a7a43e8fe8 Factor feefilter logic out (amadeuszpawlik)
c0385f10a1 Remove -feefilter option (amadeuszpawlik)

Pull request description:

  net: Remove -feefilter option, as it is debug only and isn't used in any tests. Checking this option for every peer on every iteration of the message handler is unnecessary, as described in #21545.
  refactor: Move feefilter logic out into a separate `MaybeSendFeefilter(...)` function to improve readability of the already long `SendMessages(...)`. fixes  #21545

  The configuration option `-feefilter` has been added in 9e072a6e66: _"Implement "feefilter" P2P message"_
  According to the [BIP133](https://github.com/bitcoin/bips/blob/master/bip-0133.mediawiki), turning the fee filter off was ment for:
  > [...] a node [...] using prioritisetransaction to accept transactions whose actual fee rates might fall below the node's mempool min fee [in order to] disable the fee filter to make sure it is exposed to all possible txid's

  `-feefilter` was subsequently set as debug only in #8150, with the motivation that the help message was too difficult to translate.

ACKs for top commit:
  jnewbery:
    Code review ACK a7a43e8fe8
  promag:
    Code review ACK a7a43e8fe8.
  MarcoFalke:
    review ACK a7a43e8fe8 🦁

Tree-SHA512: 8ef9a2f255597c0279d3047dcc968fd30fb7402e981b69206d08eed452c705ed568c24e646e98d06eac118eddd09205b584f45611d1c874abf38f48b08b67630
2021-05-25 08:42:30 +02:00
MarcoFalke
0a909073dc Merge bitcoin/bitcoin#22048: test: MiniWallet: introduce enum type for output mode
6cebac598e test: MiniWallet: introduce enum type for output mode (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR to #21945 which lifted the number of MiniWallet's tx output modes from 2 to 3 (by adding P2PK Support).
  Since the current way of specifying the mode on the ctor via two booleans is ugly and error-prone (see table in comment https://github.com/bitcoin/bitcoin/pull/21945#issuecomment-842526575), a new Enum type `MiniWalletMode` is introduced that can hold the following values:

  - ADDRESS_OP_TRUE
  - RAW_OP_TRUE
  - RAW_P2PK

  Also adds documentation that should guide the user on which mode is useful for what etc. with a summary table. (Can also be split up in a separate commit or shortened if that is desired, maybe it's considered to be too verbose).

ACKs for top commit:
  MarcoFalke:
    cr ACK 6cebac598e

Tree-SHA512: cbbc10806d9d9e62829548094415e9f1a281cd247b9a9fc7f7f33b923c723aa03e7bc3024623a77fb1f7da4d73455fa8244840f746980d32acdad97ee12100da
2021-05-25 07:26:31 +02:00
Pieter Wuille
f8866e8c32 Add roundtrip fuzz tests for CAddress serialization 2021-05-24 18:06:35 -07:00
Pieter Wuille
e2f0548b52 Use addrv2 serialization in anchors.dat 2021-05-24 18:06:35 -07:00
Pieter Wuille
8cd8f37dfe Introduce well-defined CAddress disk serialization
Before this commit, CAddress disk serialization was messy. It stored
CLIENT_VERSION in the first 4 bytes, optionally OR'ed with ADDRV2_FORMAT.
 - All bits except ADDRV2_FORMAT were ignored, making it hard to use for actual
   future format changes.
 - ADDRV2_FORMAT determines whether or not nServices is serialized in LE64
   format or in CompactSize format.
 - Whether or not the embedded CService is serialized in V1 or V2 format is
   determined by the stream's version having ADDRV2_FORMAT (as opposed to the
   nServices encoding, which is determined by the disk version).

To improve the situation, this commit introduces the following disk
serialization format, compatible with earlier versions, but better defined for
future changes:
 - The first 4 bytes store a format version number. Its low 19 bits are ignored
   (as it historically stored the CLIENT_VERSION), but its high 13 bits specify
   the serialization exactly:
   - 0x00000000: LE64 encoding for nServices, V1 encoding for CService
   - 0x20000000: CompactSize encoding for nServices, V2 encoding for CService
   - Any other value triggers an unsupported format error on deserialization,
     and can be used for future format changes.
 - The ADDRV2_FORMAT flag in the stream's version does not impact the actual
   serialization format; it only determines whether V2 encoding is permitted;
   whether it's actually enabled depends solely on the disk version number.

Operationally the changes to the deserializer are:
 - Failure when the stored format version number is unexpected.
 - The embedded CService's format is determined by the stored format version
   number rather than the stream's version number.

These do no introduce incompatibilities, as no code versions exist that write
any value other than 0 or 0x20000000 in the top 13 bits, and no code paths
where the stream's version differs from the stored version.
2021-05-24 18:06:31 -07:00
Pieter Wuille
66545da200 Remove support for double serialization 2021-05-24 16:15:05 -07:00
Pieter Wuille
fff1cae43a Convert uses of double-serialization to {En,De}codeDouble 2021-05-24 16:15:05 -07:00
Pieter Wuille
afd964d70b Convert existing float encoding tests 2021-05-24 16:15:02 -07:00
Pieter Wuille
bda33f98e2 Add unit tests for serfloat module 2021-05-24 16:04:44 -07:00
Pieter Wuille
2be4cd94f4 Add platform-independent float encoder/decoder 2021-05-24 16:04:44 -07:00
MarcoFalke
e40224d0c7 Remove unused float serialization 2021-05-24 16:04:44 -07:00
Pieter Wuille
2667366aaa tests: check derivation of P2TR 2021-05-24 14:59:34 -07:00
Pieter Wuille
7cedafc541 Add tr() descriptor (derivation only, no signing)
This adds a new descriptor with syntax e.g. tr(KEY,{S1,{{S2,S3},S4})
where KEY is a key expression for the internal key and S_i are
script expression for the leaves. They have to be organized in
nested {A,B} groups, with exactly two elements.

tr() only exists at the top level, and inside the script expressions
only pk() scripts are allowed for now.
2021-05-24 12:14:16 -07:00
Pieter Wuille
90fcac365e Add TaprootBuilder class
This class functions as a utility for building taproot outputs, from
internal key and script leaves.
2021-05-24 12:14:16 -07:00
Pieter Wuille
5f6cc8daa8 Add XOnlyPubKey::CreateTapTweak 2021-05-24 12:14:16 -07:00
Pieter Wuille
2fbfb1becb Make consensus checking of tweaks in pubkey.* Taproot-specific
That results in a much safer interface (making the tweak commit
to the key implicitly using a fixed tag means it can't be used for
unrelated tweaking).
2021-05-24 12:14:16 -07:00
Pieter Wuille
a4bf84039c Separate WitnessV1Taproot variant in CTxDestination 2021-05-24 12:14:16 -07:00
Pieter Wuille
41839bdb89 Avoid dependence on CTxDestination index order 2021-05-24 12:14:16 -07:00
Pieter Wuille
31df02a070 Change Solver() output for WITNESS_V1_TAPROOT
This is just a small simplification to prepare for the follow-up instruction
of a CTxDestination variant for taproot outputs.

In the old code, WITNESS_V1_TAPROOT and WITNESS_UNKNOWN both produced
{version, program} as Solver() output. Change this so that WITNESS_V1_TAPROOT
produces just {program}, like WITNESS_V0_* do.
2021-05-24 12:14:16 -07:00
Pieter Wuille
4b1cc08f9f Make XOnlyPubKey act like byte container 2021-05-24 12:14:16 -07:00
Hennadii Stepanov
8caf60dbbe move-only: Group and re-order CAddrMan members by access type
Easy to verify with `git diff --color-moved=dimmed-zebra`.
2021-05-24 19:26:49 +03:00
glozow
13650fe2e5 [policy] detect unsorted packages 2021-05-24 15:48:32 +01:00
glozow
9ef643e21b [doc] add release note for package testmempoolaccept 2021-05-24 15:48:32 +01:00
glozow
c4259f4b7e [test] functional test for packages in RPCs 2021-05-24 15:48:32 +01:00
glozow
9ede34a6f2 [rpc] allow multiple txns in testmempoolaccept
Only allow "packages" with no conflicts, sorted in order of dependency,
and no more than 25 for now.  Note that these groups of transactions
don't necessarily need to adhere to some strict definition of a package
or have any dependency relationships. Clients are free to pass in a
batch of 25 unrelated transactions if they want to.
2021-05-24 15:45:01 +01:00
Sebastian Falbesoner
6cebac598e test: MiniWallet: introduce enum type for output mode
For the MiniWallet constructor, the two boolean parameters
"raw_script" and "use_p2pk" are replaced by a single parameter of the
newly introduced type MiniWalletMode (derived by enum.Enum), which can
hold the following values:
	- ADDRESS_OP_TRUE
	- RAW_OP_TRUE
	- RAW_P2PK
2021-05-24 16:10:49 +02:00
glozow
ae8e6df709 [policy] limit package sizes
Maximum number of transactions allowed in a package is 25, equal to the
default mempool descendant limit: if a package has more transactions
than this, either it would fail default mempool descendant limit or the
transactions don't all have a dependency relationship (but then they
shouldn't be in a package together). Same rationale for 101KvB virtual
size package limit.

Note that these policies are only used in test accepts so far.
2021-05-24 14:42:10 +01:00
glozow
c9e1a26d1f [fuzz] add ProcessNewPackage call in tx_pool fuzzer 2021-05-24 14:42:10 +01:00
glozow
363e3d916c [test] unit tests for ProcessNewPackage
Key functionality = a transaction with UTXOs not present in UTXO set
or mempool can be fully validated instead of being considered an orphan.
2021-05-24 14:42:10 +01:00
glozow
cd9a11ac96 [test] make submit optional in CreateValidMempoolTransaction
This allows us to easily create transaction chains for package
validation. We don't test_accept if submit=false because we want to be
able to make transactions that wouldn't pass ATMP (i.e. a child
transaction in a package would fail due to missing inputs).
2021-05-24 14:42:10 +01:00
glozow
2ef187941d [validation] package validation for test accepts
Only allow test accepts for now. Use the CoinsViewTemporary to keep
track of coins created by each transaction so that subsequent
transactions can spend them. Uncache all coins since we only
ever do test accepts (Note this is different from ATMP which doesn't
uncache for valid test_accepts) to minimize impact on the coins cache.

Require that the input txns have no conflicts and be ordered
topologically. This commit isn't able to detect unsorted packages.
2021-05-24 14:42:10 +01:00
W. J. van der Laan
b295395664 Merge bitcoin/bitcoin#21239: guix: Add codesignature attachment support for osx+win
ee883201cf guix: repro: Sort find output in libtool for gcc-8 (Carl Dong)
ee0a67c32a codesigning: Use SHA256 as digest for osslsigncode (Windows) (Carl Dong)
38eb91eb06 guix: Add codesigning functionality (Carl Dong)
bac2690e6f guix: Package codesigning tools (Carl Dong)
0a2176d477 guix: Reindent existing manifest.scm (Carl Dong)
c090a3e923 Makefile.am: use APP_DIST_DIR instead of hard-coding dist (Carl Dong)

Pull request description:

  This is the last PR before we reach feature-parity with the Gitian process!

  Note: I tried using the `Makefile` inside the distsrc to make the dmg instead of manually listing out the commands, but `make` seems to want to re-make a lot of other files which broke the dmg.

  The workflow looks something like this:
  1. `env [ FOO=bar... ] ./contrib/guix/guix-build` (add additional env vars as necessary)
  2. Codesigners only:
      1.  Copy `guix-build-<short-id>/output/x86_64-apple-darwin18/bitcoin-<short-id>-osx-unsigned.tar.gz` and `guix-build-<short-id>/output/x86_64-w64-mingw32/bitcoin-<short-id>-win-unsigned.tar.gz` to signing computer
      2. Codesign with `./detached-sig-create.sh` inside the tarball
      3. Upload contents of `signature-{osx,win}.tar.gz` to https://github.com/bitcoin-core/bitcoin-detached-sigs (as a new tag)
  3. Checkout new tag for `bitcoin-core/bitcoin-detached-sigs` with the detached signatures
  4. `env [ FOO=bar... ] DETACHED_SIGS_REPO=<path/to/bitcoin-detached-sigs> ./contrib/guix/guix-codesign` (modify env vars as necessary)
  5. Make sure `guix.sigs` is cloned and updated
  6. `env GUIX_SIGS_REPO=<path/to/guix.sigs> SIGNER=0x96AB007F1A7ED999=dongcarl ./contrib/guix/guix-attest` (modify env vars as necessary)
  7. Commit your new signatures and SHA256SUMS in `guix.sigs`
  8. Optionally, after there are multiple signatures in `guix.sigs`: `env GUIX_SIGS_REPO=<path/to/guix.sigs> ./contrib/guix/guix-verify`

ACKs for top commit:
  laanwj:
    Tested ACK ee883201cf
  achow101:
    ACK ee883201cf

Tree-SHA512: e812a07a5f19f900600c70cb9c717769ef544a6c0c12760b5558b76b6b37df863257f3dbf38b0757e6e06e334470267e94c9f2bdbc27409d6837b1a0bfc6acbc
2021-05-24 15:33:16 +02:00
amadeuszpawlik
a7a43e8fe8 Factor feefilter logic out
Break SendMessages() function into smaller units to improve readability.
Adds lock assert, as `round()` isn't thread safe.
closes #21545
2021-05-24 14:59:39 +02:00
fanquake
2968417948 Merge bitcoin/bitcoin#22013: net: ignore block-relay-only peers when skipping DNS seed
fe3d17df04 net: ignore block-relay-only peers when skipping DNS seed (Anthony Towns)

Pull request description:

  Since #17428 bitcoind will attempt to reconnect to two block-relay-only anchors before doing any other outbound connections. When determining whether to use DNS seeds, it will currently see these two peers and decide "we're connected to the p2p network, so no need to lookup DNS" -- but block-relay-only peers don't do address relay, so if your address book is full of invalid addresses (apart from your anchors) this behaviour will prevent you from recovering from that situation.

  This patch changes it so that it only skips use of DNS seeds when there are two full-outbound peers, not just block-relay-only peers.

ACKs for top commit:
  Sjors:
    utACK fe3d17d
  amitiuttarwar:
    ACK fe3d17df04, this impacts the very common case where we stop/start a node, persisting anchors & have a non-empty addrman (although, to be clear, wouldn't be particularly problematic in the common cases where the addrman has valid addresses)
  mzumsande:
    ACK fe3d17df04
  jonatack:
    ACK fe3d17df04
  prayank23:
    tACK fe3d17df04

Tree-SHA512: 9814b0d84321d7f45b5013eb40c420a0dd93bf9430f5ef12dce50d1912a18d5de2070d890a8c6fe737a3329b31059b823bc660b432d5ba21f02881dc1d951e94
2021-05-24 20:42:08 +08:00
fanquake
d3fa42c795 Merge bitcoin/bitcoin#21186: net/net processing: Move addr data into net_processing
0829516d1f [refactor] Remove unused ForEachNodeThen() template (John Newbery)
09cc66c00e scripted-diff: rename address relay fields (John Newbery)
76568a3351 [net processing] Move addr relay data and logic into net processing (John Newbery)
caba7ae8a5 [net processing] Make RelayAddress() a member function of PeerManagerImpl (John Newbery)
86acc96469 [net processing] Take NodeId instead of CNode* as originator for RelayAddress() (John Newbery)

Pull request description:

  This continues the work of moving application layer data into net_processing, by moving all addr data into the new Peer object added in #19607.

  For motivation, see #19398.

ACKs for top commit:
  laanwj:
    Code review ACK 0829516d1f
  mzumsande:
    ACK 0829516d1f, reviewed the code and ran tests.
  sipa:
    utACK 0829516d1f
  hebasto:
    re-ACK 0829516d1f

Tree-SHA512: efe0410fac288637f203eb37d1999910791e345872d37e1bd5cde50e25bb3cb1c369ab86b3a166ffd5e06ee72e4508aa2c46d658be6a54e20b4f220d2f57d0a6
2021-05-24 20:28:31 +08:00
Jon Atack
b36e0cd1b9 rpc: simplify addpeeraddress and improve code constness 2021-05-24 14:10:23 +02:00
MarcoFalke
ce4a852475 Merge bitcoin/bitcoin#21848: refactor: Make CFeeRate constructor architecture-independent
fafd121026 refactor: Make CFeeRate constructor architecture-independent (MarcoFalke)

Pull request description:

  Currently the constructor is architecture dependent. This is confusing for several reasons:

  * It is impossible to create a transaction larger than the max value of `uint32_t`, so a 64-bit `size_t` is not needed
  * Policy (and consensus) code should be arch-independent
  * The current code will print spurious compile errors when compiled on 32-bit systems:

  ```
  policy/feerate.cpp:23:22: warning: result of comparison of constant 9223372036854775807 with expression of type 'size_t' (aka 'unsigned int') is always true [-Wtautological-constant-out-of-range-compare]
      assert(nBytes_ <= uint64_t(std::numeric_limits<int64_t>::max()));
  ```

  Fix all issues by making it arch-independent. Also, fix `{}` style according to dev notes.

ACKs for top commit:
  theStack:
    re-ACK fafd121026
  promag:
    Code review ACK fafd121026.

Tree-SHA512: e16f75bad9ee8088b87e873906d9b5633449417a6996a226a2f37d33a2b7d4f2fd91df68998a77e52163de20b40c57fadabe7fe3502e599cbb98494178591833
2021-05-24 11:14:23 +02:00
MarcoFalke
599000903e Merge bitcoin/bitcoin#21850: Remove GetDataDir(net_specific) function
aca0e5dcdb Remove `GetDataDir(bool fNetSpecific = true)` function (Kiminuo)
b3e67f20a0 scripted-diff: Replace `GetDataDir(true)` calls with `gArgs.GetDataDirNet()` calls (Kiminuo)
4c3a5dcbfc scripted-diff: Replace `GetDataDir()` calls with `gArgs.GetDataDirNet()` calls (Kiminuo)
13bd8bb053 Make `ArgsManager.GetDataDirPath` private and drop needless suffix (Kiminuo)
4d8189f620 scripted-diff: Change `ArgsManager.GetDataDirPath()` to `ArgsManager.GetDataDirBase()` in tests (Kiminuo)
0f53df47d5 Add `ArgsManager.GetDataDirBase()` and `ArgsManager.GetDataDirNet()` as an intended replacement for `ArgsManager.GetDataDirPath(net_identifier)` (Kiminuo)
716de29dd8 Make `m_cached_blocks_path` mutable. Make `ArgsManager::GetBlocksDirPath()` const. (Kiminuo)

Pull request description:

  This PR is a follow up PR to #21244. The PR attempts to move us an inch towards the [goal](https://github.com/bitcoin/bitcoin/pull/21244#discussion_r615307465) by removing `GetDataDir(net_specific)` and replacing it by `gArgs.GetDataDir(net_specific)` calls.

  The approach of this PR attempts to be similar to the one chosen in "De-globalize ChainstateManager" (#20158). The goal is to pass `ArgsManager` to functions (or ideally to have `ArgsManager` as a member of a class where needed; inspiration from here: #21789) instead of having it as a global variable (i.e. `gArgs`).

  **Notes:**
  * First commit makes `m_cached_blocks_path` `mutable` as was suggested [here](https://github.com/bitcoin/bitcoin/pull/21244#discussion_r615274095) but not fully applied in #21244. (`m_cached_datadir_path` and `m_cached_network_datadir_path` were marked as `mutable` in #21244) This commit can be in a separate PR too.
  * Other commits deal with removing of `GetDataDir(net_specific)` function.
      * This was originally part of #21244 but it was [left]((https://github.com/bitcoin/bitcoin/pull/21244#pullrequestreview-633779754)) for a follow up PR.
  * I think that the proposed changes show nicely where there is reliance on `gArgs` which is IMO a good thing.

  If you know about a better approach how to do this, please share it here.

ACKs for top commit:
  hebasto:
    ACK aca0e5dcdb
  MarcoFalke:
    re-ACK aca0e5dcdb 👃

Tree-SHA512: deec4d88edb32d7f4c818c3a74ffbb64709685819b88242dcf5dbaa1fb611f3ce2b29d2576ddb9e0dc5e75288e43538968224008c0a80e7149fc81c309f7c9da
2021-05-24 11:05:58 +02:00
Kiminuo
aca0e5dcdb Remove GetDataDir(bool fNetSpecific = true) function 2021-05-24 10:29:58 +02:00
Kiminuo
b3e67f20a0 scripted-diff: Replace GetDataDir(true) calls with gArgs.GetDataDirNet() calls
-BEGIN VERIFY SCRIPT-
git ls-files -- 'src' ':(exclude)src/util/system.h' ':(exclude)src/util/system.cpp' | xargs sed -i 's/GetDataDir(true)/gArgs.GetDataDirNet()/g';
-END VERIFY SCRIPT-
2021-05-24 10:29:58 +02:00
Kiminuo
4c3a5dcbfc scripted-diff: Replace GetDataDir() calls with gArgs.GetDataDirNet() calls
-BEGIN VERIFY SCRIPT-
git ls-files -- 'src' ':(exclude)src/util/system.h' ':(exclude)src/util/system.cpp' | xargs sed -i 's/GetDataDir()/gArgs.GetDataDirNet()/g';
-END VERIFY SCRIPT-
2021-05-24 10:29:58 +02:00
Kiminuo
13bd8bb053 Make ArgsManager.GetDataDirPath private and drop needless suffix 2021-05-24 10:29:58 +02:00
Kiminuo
4d8189f620 scripted-diff: Change ArgsManager.GetDataDirPath() to ArgsManager.GetDataDirBase() in tests
-BEGIN VERIFY SCRIPT-
git ls-files src/test/*_tests.cpp src/test/util/setup_common.cpp | xargs sed -i 's/.GetDataDirPath()/.GetDataDirBase()/g';
-END VERIFY SCRIPT-
2021-05-24 10:29:57 +02:00
Kiminuo
0f53df47d5 Add ArgsManager.GetDataDirBase() and ArgsManager.GetDataDirNet() as an intended replacement for ArgsManager.GetDataDirPath(net_identifier) 2021-05-24 10:29:55 +02:00
MarcoFalke
778b920179 Merge bitcoin/bitcoin#21945: test: add P2PK support to MiniWallet
4bea301692 test: use P2PK-MiniWallet for feature_csv_activation.py (Sebastian Falbesoner)
dc7eb64e83 test: MiniWallet: add P2PK support (Sebastian Falbesoner)

Pull request description:

  This PR adds support for creating and spending transactions with raw pubkey (P2PK) outputs to MiniWallet, [as suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/21900#discussion_r629524841). Using that  mode in the test `feature_csv_activation.py`, all txs submitted to the mempool follow the standard policy, i.e. `-acceptnonstdtxn=1` can be removed.

  Possible follow-ups:
  * Improve MiniWallet constructor Interface; an enum-like parameter instead of two booleans would probably be better
  * Look at other tests that could benefit from P2PK (e.g. feature_cltv.py?)
  * Check vsize also for P2PK txs (vsize varies due to signature, i.e. a range has to be asserted)

ACKs for top commit:
  laanwj:
    Code review ACK 4bea301692

Tree-SHA512: 9b428e6b7cfde59a8c7955d5096cea88af1384a5f49723f00052e9884d819d952d20a5ab39bb02f9d8b6073769c44462aa265d84a33e33da33c2d21670c488a6
2021-05-24 08:42:59 +02:00
Hennadii Stepanov
e9f948c727 build: Convert warnings into errors when testing for -fstack-clash-protection
When building with Clang, if `-fstack-clash-protection` is used with an
unsupported target, it may result in hundreds of
`-Wunused-command-line-argument` warnings at compile time. This is
currently the case when building for at least Darwin using Apple or LLVM
Clang.

Unsupported targets may also include *BSD, however that is changing; see
further discussion in https://reviews.llvm.org/D92245 and
https://reviews.freebsd.org/D27366. 

Note that this option is already skipped for Windows.
2021-05-24 08:57:22 +03:00
fanquake
3f3c4d2e2e Merge bitcoin/bitcoin#22002: Fix crash when parsing command line with -noincludeconf=0
fad0867d6a Cleanup -includeconf error message (MarcoFalke)
fa9f711c37 Fix crash when parsing command line with -noincludeconf=0 (MarcoFalke)

Pull request description:

  The error message has several issues:

  * It may crash instead of cleanly shutting down, when `-noincludeconf=0` is passed
  * It doesn't quote the value
  * It includes an erroneous trailing `\n`
  * It is redundantly mentioning `"-includeconf cannot be used from commandline;"` several times, when once should be more than sufficient

  Fix all issues by:
  * Replacing `get_str()` with `write()` to fix the crash and quoting issue
  * Remove the `\n` and only print the first value to fix the other issues

  Before:

  ```
  $ ./src/bitcoind -noincludeconf=0
  terminate called after throwing an instance of 'std::runtime_error'
    what():  JSON value is not a string as expected
  Aborted (core dumped)

  $ ./src/bitcoind -includeconf='a b' -includeconf=c
  Error: Error parsing command line arguments: -includeconf cannot be used from commandline; -includeconf=a b
  -includeconf cannot be used from commandline; -includeconf=c
  ```

  After:

  ```
  $ ./src/bitcoind -noincludeconf=0
  Error: Error parsing command line arguments: -includeconf cannot be used from commandline; -includeconf=true

  $ ./src/bitcoind -includeconf='a b' -includeconf=c
  Error: Error parsing command line arguments: -includeconf cannot be used from commandline; -includeconf="a b"
  ```

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34493

  Testcase: https://github.com/bitcoin/bitcoin/files/6515429/clusterfuzz-testcase-minimized-system-6328535926046720.log

  ```
  FUZZ=system ./src/test/fuzz/fuzz ./clusterfuzz-testcase-minimized-system-6328535926046720.log
  ```

  See https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md

ACKs for top commit:
  sipa:
    utACK fad0867d6a

Tree-SHA512: b44af93be6bf71b43669058c1449c4c6999f03b5b01b429851b149b12d77733408cb207e9a3edc6f0bffd6030c4c52165e8e23a1c2718ff5082a6ba254cc94a4
2021-05-24 11:02:00 +08:00
Hennadii Stepanov
5cd7f8abe3 refactor: Do not expose CAddrMan members as protected without need 2021-05-23 14:46:21 +03:00
Jon Atack
6b1926cf1e test: addpeeraddress functional test coverage 2021-05-23 10:39:08 +02:00
Hennadii Stepanov
4935ac583b qt: Improve GUI responsiveness
QProgressDialog estimates the time the operation will take (based on
time for steps), and only shows itself if that estimate is beyond
minimumDuration. The default minimumDuration value is 4 seconds, and it
could make users think that the GUI is frozen.
2021-05-22 22:04:27 +03:00
Hennadii Stepanov
75850106ae qt, macos: Fix GUIUtil::PolishProgressDialog bug
QProgressDialog shows itself if the estimated time an operation will
take is beyond the minimumDuration value.

Direct call show() breaks that behavior on macos.
2021-05-22 21:33:20 +03:00
Kiminuo
716de29dd8 Make m_cached_blocks_path mutable. Make ArgsManager::GetBlocksDirPath() const. 2021-05-22 15:43:42 +02:00
MarcoFalke
be4171679b Merge bitcoin/bitcoin#21953: fuzz: Add utxo_snapshot target
fa91994b1b fuzz: Add utxo_snapshot target (MarcoFalke)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK fa91994b1b

Tree-SHA512: a00f077102a4e4e321bd1464c3fa11e7a5b9e04324b9be87aa28cfdc77630db7fc772d3a3768dc6ec36bbdd2d67b7e0719f0cf3fd87b4a1087365b934e137b5c
2021-05-22 10:09:40 +02:00
Jon Atack
4f504f826b rpc: fix code comment for bumpfee/psbtbumpfee output 2021-05-22 09:30:26 +02:00
Jon Atack
5cb7ac23fb rpc: fix docs for bumpfee psbt update
- "psbt" field is only returned in psbtbumpfee and not bumpfee
- bumpfee raises if privkeys are disabled, so drop "Only returned when wallet private keys are enabled."
- add missing space in RPC example
2021-05-22 09:22:14 +02:00
practicalswift
3737d35fee fuzz: Terminate immediately if a fuzzing harness ever tries to perform a DNS lookup (belts and suspenders) 2021-05-21 19:41:43 +00:00
Andrew Chow
167fb1fc72 Update Windows code signing certificate 2021-05-21 11:49:01 -04:00
Kiminuo
525448df9d Move COINBASE_MATURITY from feature_nulldummy test to blocktools. 2021-05-21 14:33:40 +02:00
MarcoFalke
fad0867d6a Cleanup -includeconf error message
Remove the erroneous trailing newline '\n'. Also, print only the first
value to remove needless redundancy in the error message.
2021-05-21 10:54:12 +02:00
MarcoFalke
fa9f711c37 Fix crash when parsing command line with -noincludeconf=0 2021-05-21 10:53:09 +02:00
MarcoFalke
eb4df9a628 Merge bitcoin/bitcoin#22004: fuzz: Speed up transaction fuzz target
bbbb51877a fuzz: Speed up transaction fuzz target (MarcoFalke)

Pull request description:

  `hashBlock` and `include_addresses` are orthogonal, so no need to do an exhaustive "search".

  Might fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34491

ACKs for top commit:
  practicalswift:
    cr ACK bbbb51877a: patch looks correct, and `TxToUniv` surprisingly wide in the `transaction_fuzz_target` flame graph! Putting it on a diet makes sense.

Tree-SHA512: 1e7c30c7fecf96364a9a1597c0a22139389fdeb67db59f3c2c6fc088196e3332877b2865991a957980d542f99a2f48cc066dd7cc16c695a5113190fe06205089
2021-05-21 09:03:25 +02:00
MarcoFalke
ac5f7f47c1 Merge bitcoin/bitcoin#21936: fuzz: Terminate immediately if a fuzzing harness tries to create a TCP socket (belt and suspenders)
393992b049 fuzz: Terminate immediately if a fuzzing harness ever tries to create a TCP socket (belt and suspenders) (practicalswift)

Pull request description:

  Terminate immediately if a fuzzing harness ever to create a TCP socket (belt and suspenders).

  Obviously this _should_ never happen, but if it _does_ happen we want immediate termination instead of a TCP socket :)

ACKs for top commit:
  MarcoFalke:
    ACK 393992b049

Tree-SHA512: 5bbff1f7e9a58b3eae24f742b7daf3fc870424c985f29bed5931e47a708d9c0984bfd8762f43658cffa9c69d32f86d56deb48bc7e43821e3398052174b6a160e
2021-05-21 09:00:18 +02:00
MarcoFalke
1cc38d3e01 Merge bitcoin/bitcoin#22003: txmempool: add thread safety annotations
793b268284 txmempool: add thread safety annotations (Anthony Towns)

Pull request description:

  Add missing thread safety guards to CTxMempool members.

ACKs for top commit:
  MarcoFalke:
    cr ACK 793b268284
  hebasto:
    re-ACK 793b268284, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/22003#pullrequestreview-664529633) review.

Tree-SHA512: c5eb197c63375c80c325a276f322177e84e0181c94a124720b1a364e964ac223fc6fdfd89bd0e152b76959fb6b97bfbf82dd36ec105ed6e2dc045ede717df4ae
2021-05-21 08:27:59 +02:00
MarcoFalke
18cf1c5165 Merge bitcoin/bitcoin#21991: build: libevent 2.1.12-stable
f152c1ac66 build: libevent 2.1.12-stable (fanquake)

Pull request description:

  Start using [libevent 2.1.12-stable](https://github.com/libevent/libevent/releases/tag/release-2.1.12-stable) in depends. Changelog is available here: https://raw.githubusercontent.com/libevent/libevent/release-2.1.12-stable/ChangeLog. This release basically only contains bug fixes.

  Upgrading will also prevent some oss-fuzz reports which are coming from libevent code, such as https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34355 & https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34412.

  Guix builds:
  ```bash
  aa365feab851d2055c4c3cd787fc1db972a8f2efe1e7c75bc62b34594846a021  guix-build-f152c1ac66bf/output/aarch64-linux-gnu/bitcoin-f152c1ac66bf-aarch64-linux-gnu-debug.tar.gz
  80cf96c4618279f4420b3f15ce3d3331937506e7996d8c325ce741f4da431fad  guix-build-f152c1ac66bf/output/aarch64-linux-gnu/bitcoin-f152c1ac66bf-aarch64-linux-gnu.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/aarch64-linux-gnu/inputs.SHA256SUMS
  b6261b19ed8f55e655122d515a4f76a984e1f74f932630ce40ca34bb76fadfc6  guix-build-f152c1ac66bf/output/arm-linux-gnueabihf/bitcoin-f152c1ac66bf-arm-linux-gnueabihf-debug.tar.gz
  be1018237e5e739a742f777efdbdd28ff794cfc2faa39930d5da6dfcfa620da3  guix-build-f152c1ac66bf/output/arm-linux-gnueabihf/bitcoin-f152c1ac66bf-arm-linux-gnueabihf.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/arm-linux-gnueabihf/inputs.SHA256SUMS
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855  guix-build-f152c1ac66bf/output/dist-archive/SKIPATTEST.TAG
  52fa25961965b53e73319f5f8ba5a62f789b56ff3a07286ed5fe88771c2674bf  guix-build-f152c1ac66bf/output/dist-archive/bitcoin-f152c1ac66bf.tar.gz
  069dfc72b83527722552d12db86f3f7046be9a85d0ccc5168c71af0d38dbc9a9  guix-build-f152c1ac66bf/output/powerpc64-linux-gnu/bitcoin-f152c1ac66bf-powerpc64-linux-gnu-debug.tar.gz
  dba47d56dccdd4183762f888ab2e64bf8c074a6873dcf6dac663f88633932926  guix-build-f152c1ac66bf/output/powerpc64-linux-gnu/bitcoin-f152c1ac66bf-powerpc64-linux-gnu.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/powerpc64-linux-gnu/inputs.SHA256SUMS
  e43e2c631ed0dd587a52705b36e5e69e9e622d493eb6741d438976197a1416f2  guix-build-f152c1ac66bf/output/powerpc64le-linux-gnu/bitcoin-f152c1ac66bf-powerpc64le-linux-gnu-debug.tar.gz
  f47f006c68262e3cd30acc355478d4ea056bb1c055bc51c5adab56305c260725  guix-build-f152c1ac66bf/output/powerpc64le-linux-gnu/bitcoin-f152c1ac66bf-powerpc64le-linux-gnu.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/powerpc64le-linux-gnu/inputs.SHA256SUMS
  f62805fd8652e28d935da14b0328f62a583be24b5517a5c7f189970bdd3c6eb0  guix-build-f152c1ac66bf/output/riscv64-linux-gnu/bitcoin-f152c1ac66bf-riscv64-linux-gnu-debug.tar.gz
  db16cbddfc790798af8617213cc726002545f05cfb90ec00e462ba5d3aca10b1  guix-build-f152c1ac66bf/output/riscv64-linux-gnu/bitcoin-f152c1ac66bf-riscv64-linux-gnu.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/riscv64-linux-gnu/inputs.SHA256SUMS
  2c1afbc6ea44e5947ffcbdf031233f03f7a63b93ca08ddfec2c45c661c1666bc  guix-build-f152c1ac66bf/output/x86_64-apple-darwin18/bitcoin-f152c1ac66bf-osx-unsigned.dmg
  65cff5a1bc7eb411b1bb26eb407c2a944e5bf24b08cfcd7a9eeea47924106ac8  guix-build-f152c1ac66bf/output/x86_64-apple-darwin18/bitcoin-f152c1ac66bf-osx-unsigned.tar.gz
  98a5c002aed8a35912853cca5e5ffd7afc91eb839b81ca8bc73ddf010839e3ac  guix-build-f152c1ac66bf/output/x86_64-apple-darwin18/bitcoin-f152c1ac66bf-osx64.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/x86_64-apple-darwin18/inputs.SHA256SUMS
  84e3188d369aae01a459b477e1935a9466338bb9ea6d4beb93675275a5ac62fd  guix-build-f152c1ac66bf/output/x86_64-linux-gnu/bitcoin-f152c1ac66bf-x86_64-linux-gnu-debug.tar.gz
  ea3f4d928f82ea3d1079be3d4e020349e9d412c621149656ea409a61de906c4c  guix-build-f152c1ac66bf/output/x86_64-linux-gnu/bitcoin-f152c1ac66bf-x86_64-linux-gnu.tar.gz
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/x86_64-linux-gnu/inputs.SHA256SUMS
  6dc14c4e1d0b353b865a9f807dafe7ec56e829a8bc104a5ac6ef331bdc9bc726  guix-build-f152c1ac66bf/output/x86_64-w64-mingw32/bitcoin-f152c1ac66bf-win-unsigned.tar.gz
  bd382152587c2dc5816d615d71e36df80f54eac5b4913569148e44117b80517c  guix-build-f152c1ac66bf/output/x86_64-w64-mingw32/bitcoin-f152c1ac66bf-win64-debug.zip
  d2f38b52ea54e913ccadbda6c264a258ca9a55b40a8af96ab8123e8311170aef  guix-build-f152c1ac66bf/output/x86_64-w64-mingw32/bitcoin-f152c1ac66bf-win64-setup-unsigned.exe
  8846b6edcdb0b3c29bdbab08b3582ee13b4877724bf3811246360b14b73ae420  guix-build-f152c1ac66bf/output/x86_64-w64-mingw32/bitcoin-f152c1ac66bf-win64.zip
  53db9e6b78fa13e0753cab4a35da5e66156c54485020c6c49b1cd1ab406e978b  guix-build-f152c1ac66bf/output/x86_64-w64-mingw32/inputs.SHA256SUMS
  ```

  Gitian builds:
  ```bash
  # macOS:
  19401d11d0eda5cdcbab8c1353f7110963288e5223fd95d8cc3380107b66a7e4  bitcoin-f152c1ac66bf-osx-unsigned.dmg
  3eb2383c5af734a86eaf1895f261c1ba5fddc130d9682857eff08fa7d0599e93  bitcoin-f152c1ac66bf-osx-unsigned.tar.gz
  c2572b6f17613a380cf95417ed6a533d8f75fb1418428e234cb90cc0d3d5a655  bitcoin-f152c1ac66bf-osx64.tar.gz
  52fa25961965b53e73319f5f8ba5a62f789b56ff3a07286ed5fe88771c2674bf  src/bitcoin-f152c1ac66bf.tar.gz
  4bcf84076345c61736c63f33e6f9db57b626030c417c2a1b668752d5f87dab22  bitcoin-core-osx-22-res.yml

  # Windows:
  5857fda233fc9fdb04391bd1718522b774978d093912650d4dc6e49b70dde40b  bitcoin-f152c1ac66bf-win-unsigned.tar.gz
  beff745cde9f19d3d619c3c360a9eb37dbe94169cbc968ac7958e80521584787  bitcoin-f152c1ac66bf-win64-debug.zip
  e92010f76d8b286e291c4420349018029e72013ae8bf5b98eccfbf9f856a1490  bitcoin-f152c1ac66bf-win64-setup-unsigned.exe
  46c38498fb16e5353de24caffd11d00b23162ec92f126de57888b4f92f375d79  bitcoin-f152c1ac66bf-win64.zip
  52fa25961965b53e73319f5f8ba5a62f789b56ff3a07286ed5fe88771c2674bf  src/bitcoin-f152c1ac66bf.tar.gz
  851ebc1f3028ec421d733a15acf7514474a44837b75e2f94c1f80ab383b1a364  bitcoin-core-win-22-res.yml

  # Linux:
  4270014b016a309b6ccff39cf87b90f3b5a36a10eab5d2c41697b12b9eb5e63e  bitcoin-f152c1ac66bf-aarch64-linux-gnu-debug.tar.gz
  f226aaeb9cf7786fa5f4dc379611c8b1b3b580df503f79905158d0e2271c3862  bitcoin-f152c1ac66bf-aarch64-linux-gnu.tar.gz
  4cd6208dc30c15b71d80d9cd59d367aba676934609d59581c8f418f9e864994d  bitcoin-f152c1ac66bf-arm-linux-gnueabihf-debug.tar.gz
  21746b2bbf4fd6778ea0357271f569975029667dea2ad0000abb6751eeba8071  bitcoin-f152c1ac66bf-arm-linux-gnueabihf.tar.gz
  d73e6b2a779061037eaa3174615a7e5bb6082056ee4dc13de4026cc3ad21f3c5  bitcoin-f152c1ac66bf-powerpc64-linux-gnu-debug.tar.gz
  e6c5876dcdd30b8936b957afc8adb9341c64a1201ba8c4285f5f76bc37689c7c  bitcoin-f152c1ac66bf-powerpc64-linux-gnu.tar.gz
  fc0270f7c220519e91ecd27c37df7ec130e16db8508e51b52eb88eae1f0d05d2  bitcoin-f152c1ac66bf-powerpc64le-linux-gnu-debug.tar.gz
  3729a36717f704a6e30a0368a68d2356ec9af240b706a7a8c117f7ebcc5ebbca  bitcoin-f152c1ac66bf-powerpc64le-linux-gnu.tar.gz
  b661fc47dbf4a56cecfe02abfff78644524cd6d94aac51a38d83a4ba96a21b6e  bitcoin-f152c1ac66bf-riscv64-linux-gnu-debug.tar.gz
  8934ae4727e7908f626d2392877cad153696c6d8e832a9dbc0960e090f633ab4  bitcoin-f152c1ac66bf-riscv64-linux-gnu.tar.gz
  cc8511b71c89575f15cfbb360c18c0af298869fd4e8c4a27ef2c04b6bc0f0703  bitcoin-f152c1ac66bf-x86_64-linux-gnu-debug.tar.gz
  1cfb8cf750ca7af47cbab00d8eaee336fb11c22756a62b26b5c9a1263b5b0cd4  bitcoin-f152c1ac66bf-x86_64-linux-gnu.tar.gz
  52fa25961965b53e73319f5f8ba5a62f789b56ff3a07286ed5fe88771c2674bf  src/bitcoin-f152c1ac66bf.tar.gz
  3b5053c7392a1a6264f3d192136e50fc9db969834d9d3643d860a3f0e2f79535  bitcoin-core-linux-22-res.yml
  ```

ACKs for top commit:
  practicalswift:
    ~~cr ACK f152c1ac66: patch looks correct and this should resolve the OSS-Fuzz issues~~ ACK temporarily redacted while awaiting non-determinism fix
  MarcoFalke:
    Approach ACK f152c1ac66
  hebasto:
    ~ACK f152c1ac66bfe5c77d947d5c99ce5da3a88797fa~ [retracted due to non-reproducible builds](https://github.com/bitcoin/bitcoin/pull/21991#issuecomment-844005749)

Tree-SHA512: 43253131f6d7fadfcda541e00c13d5e2fd395555563bcc42218ba4657d8f94b09760cbad6b8f0a17e426eed817624b745464d2733f7d5ac00fbb92a6ffadab84
2021-05-21 08:12:35 +02:00
Anthony Towns
fe3d17df04 net: ignore block-relay-only peers when skipping DNS seed 2021-05-21 13:03:00 +10:00
Anthony Towns
793b268284 txmempool: add thread safety annotations 2021-05-21 12:14:01 +10:00
Hennadii Stepanov
e2b55cd201 Merge bitcoin-core/gui#335: test: Use QSignalSpy instead of QEventLoop
7eea659fc9 qt, test: use qsignalspy instead of qeventloop (Jarol Rodriguez)

Pull request description:

  This PR refactors our GUI `apptests` to use [QSignalSpy](https://doc.qt.io/qt-5/qsignalspy.html) instead of [QEventLoop](https://doc.qt.io/qt-5/qeventloop.html).

  `QSignalSpy` is more appropriate for our GUI test's as it is purpose-built for testing emission of signals and sets up its own `QEventLoop` when the `wait` function is called.

ACKs for top commit:
  hebasto:
    ACK 7eea659fc9, tested on Linux Mint 20.1 (Qt 5.12.8).
  promag:
    Code review ACK 7eea659fc9.

Tree-SHA512: 3adddbcc5efd726302b606980c9923025c44bb8ee16cb8a183e633e423179c0822db66de9ccba20dc5124fff34af4151a379c9cd18130625c60789ce809ee6fd
2021-05-21 00:02:06 +03:00
Hennadii Stepanov
0f3d955a38 qt: Make RPC console welcome message translation-friendly 2021-05-20 23:43:53 +03:00
glozow
578148ded6 [validation] explicit Success/Failure ctors for MempoolAcceptResult
Makes code more clear and prevents accidentally calling the wrong ctor.
2021-05-20 21:34:31 +01:00
glozow
b88d77aec5 [policy] Define packages
Define the Package type as an alias for a vector of transactions for now.
Add PackageValidationResult, similar to TxValidationResult and
BlockValidationResult for package-wide errors that cannot be reported
within a single transaction result, such as having too many
transactions in the package. We can update the concept of
what a package is and have different logic for packages vs lists of
transactions in the future, e.g. for package relay.
2021-05-20 21:34:31 +01:00
glozow
249f43f3cc [refactor] add option to disable RBF
This is a mere refactor for now. We will use this to disable RBFing in
package validation.
2021-05-20 21:34:31 +01:00
glozow
897e348f59 [coins/mempool] extend CCoinsViewMemPool to track temporary coins 2021-05-20 21:34:31 +01:00
glozow
42cf8b25df [validation] make CheckSequenceLocks context-free
Allow CheckSequenceLocks to use heights and coins from any CoinsView and
CBlockIndex provided. This means that CheckSequenceLocks() doesn't need
to hold the mempool lock or cs_main. The caller is responsible for
ensuring the CoinsView and CBlockIndex are consistent before passing
them in. The typical usage is still to create a CCoinsViewMemPool from
the mempool and grab the CBlockIndex from the chainstate tip.
2021-05-20 21:34:31 +01:00
W. J. van der Laan
710c8ba829 Merge bitcoin-core/gui#281: set shortcuts for console's resize buttons
2a45134b56 qt: Add shortcuts for console font resize buttons (Hennadii Stepanov)
a2e122f0fe qt: Add GUIUtil::AddButtonShortcut (Hennadii Stepanov)
4ee9ee7236 qt: Use native presentation of shortcut (Hennadii Stepanov)

Pull request description:

  On `master` the only way to resize the console font is to manually move your mouse and click the resize buttons. This PR introduces convenient keyboard shortcuts to resize the console font.

  The common resize shortcuts for applications are `Ctrl+=`/`Ctrl++` and `Ctrl+-`/`Ctrl+_`. This means that the resize QPushButtons need two shortcuts each, but you cannot assign multiple shortcuts to a QPushButton. See: https://doc.qt.io/qt-5/qabstractbutton.html#shortcut-prop

  To get around this, we introduce a new function in `guiutil`, which connects a supplied `QKeySequence` shortcut to a `QAbstractButton`. This function can be reused in other situations where more than one shortcut is needed for a button.

  | PR on macOS      | PR on Linux |
  | ---------------- | ------------ |
  |  ![mac-resize-shortcuts](https://user-images.githubusercontent.com/23396902/114750132-a2752580-9d21-11eb-9542-15716f2c257d.gif) | ![linux-resize-shortcuts](https://user-images.githubusercontent.com/23396902/114750165-aacd6080-9d21-11eb-8abc-5388690dcf0b.gif) |

ACKs for top commit:
  hebasto:
    re-ACK 2a45134b56
  Talkless:
    tACK 2a45134b56, tested on Debian Sid with Qt 5.15.2, shortcuts still work.

Tree-SHA512: e894ccb7e5c695ba83998c21a474d6c587c9c849f12ced665c5e0034feb6b143e41b32ba135cab6cfab22cbf153d5a52b1083b2a278e6dfca3f5ad14c0f6c573
2021-05-20 22:09:15 +02:00
practicalswift
393992b049 fuzz: Terminate immediately if a fuzzing harness ever tries to create a TCP socket (belt and suspenders) 2021-05-20 19:02:37 +00:00
W. J. van der Laan
37e9f07996 Merge bitcoin/bitcoin#21843: p2p, rpc: enable GetAddr, GetAddresses, and getnodeaddresses by network
ce6bca88e8 doc: release note for getnodeaddresses by network (Jon Atack)
3f89c0e990 test: improve getnodeaddresses coverage, test by network (Jon Atack)
6c98c09991 rpc: enable filtering getnodeaddresses by network (Jon Atack)
80ba294854 p2p: allow CConnman::GetAddresses() by network, add doxygen (Jon Atack)
a49f3ddbba p2p: allow CAddrMan::GetAddr() by network, add doxygen (Jon Atack)
c38981e748 p2p: pull time call out of loop in CAddrMan::GetAddr_() (João Barbosa)
d35ddca91e p2p: enable CAddrMan::GetAddr_() by network, add doxygen (Jon Atack)

Pull request description:

  This patch allows passing a network argument to CAddrMan::GetAddr(), CConnman::GetAddresses(), and rpc getnodeaddresses to return only addresses of that network.

  It also contains a performance optimisation by promag.

ACKs for top commit:
  laanwj:
    Code review and lightly tested ACK ce6bca88e8
  vasild:
    ACK ce6bca88e8

Tree-SHA512: 40e700d97091248429c73cbc0639a1f03ab7288e636a7b9026ad253e9708253c6b2ec98e7d9fb2d56136c0f762313dd648915ac98d723ee330d713813a43f99d
2021-05-20 20:53:05 +02:00
MarcoFalke
fae0f836be fuzz: Speed up banman fuzz target 2021-05-20 18:03:47 +02:00
MarcoFalke
bbbb51877a fuzz: Speed up transaction fuzz target 2021-05-20 17:26:24 +02:00
MarcoFalke
ea8b2e8e12 Merge bitcoin/bitcoin#21913: rpc: RPCHelpMan fixes
6e2eb0d63b rpc/wallet: use OMITTED_NAMED_ARG instead of Default(VNULL) (Karl-Johan Alm)
4983f4cba4 rpc/createwallet: omitted named arguments (Karl-Johan Alm)
dc4db23b30 rpc: address:amount dictionaries are OBJ_USER_KEYS (Karl-Johan Alm)
c8cf0a3d51 rpc/getpeerinfo: bytesrecv_per_msg is a dynamic dictionary (Karl-Johan Alm)
eb4fb7e507 rpc/gettxoutsetinfo: hash_or_height is a named argument (Karl-Johan Alm)

Pull request description:

  This is a follow-up to #21897, and I believe covers the remaining cases, at least that I could find.

  Edited to remove unrelated information about a side project.

ACKs for top commit:
  laanwj:
    Documentation diff ACK 6e2eb0d63b
  promag:
    Code review ACK 6e2eb0d63b.

Tree-SHA512: d26f6e074e13d64bbca2a114a0adc7f905d47d238c4e9bc49f70ca0b775afbebf9879fc3794ab29dc316a6dbd00ba8cbeb01197e236ee4ab2e9854db25f23f04
2021-05-20 07:43:55 +02:00
Carl Dong
fc1c282845 rpc/blockchain: Use existing blockman in gettxoutsetinfo
Was missed in last bundle
2021-05-19 16:34:32 -04:00
Andrew Chow
51a3ac242c Have OutputGroup determine the value to use
Instead of hijacking the effective_feerate to use the correct value
during coin selection, have OutputGroup be aware of whether we are
subtracting the fee from the outputs and provide the correct value to
use for selection.

To do this, OutputGroup now takes CoinSelectionParams and has a new
function GetSelectionAmount().
2021-05-19 15:35:11 -04:00
Carl Dong
ee883201cf guix: repro: Sort find output in libtool for gcc-8
Otherwise the resulting .a static libraries (e.g. libstdc++.a) will not
be reproducible and end up making the Bitcoin binaries non-reproducible
as well.

See: https://reproducible-builds.org/docs/archives/#gnu-libtool
2021-05-19 15:29:10 -04:00
MarcoFalke
7d19c85f4a Merge bitcoin/bitcoin#21970: fuzz: Add missing CheckTransaction before CheckTxInputs
fae4ee545a fuzz: Add missing CheckTransaction before CheckTxInputs (MarcoFalke)
faacb7eadb fuzz: Sanity check result of CheckTransaction (MarcoFalke)

Pull request description:

  This bug was introduced by myself in commit eeee8f5be1 (https://github.com/bitcoin/bitcoin/pull/21553)

  Reproducer: https://github.com/bitcoin/bitcoin/files/6492249/clusterfuzz-testcase-minimized-coins_view-6109460079706112.log

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34301

ACKs for top commit:
  practicalswift:
    cr ACK fae4ee545a: patch looks correct :)

Tree-SHA512: 9ece7a5c4bfa60f5e5ffeba3f0ee52a07944c9bd6102588dd7ff7405695e6b32449945b7c41bd25baf38814df5a2436521e655ceff87223ad03c69ed39053023
2021-05-19 21:24:32 +02:00
Andrew Chow
6d6d278475 Change SelectCoins_test to actually test SelectCoins
This was originally modified to use SelectCoinsMinConf in order to test
both BnB and Knapsack at the same time. But since SelectCoins does both
now, this is no longer necessary and we can revert back to actually
testing SelectCoins.
2021-05-19 15:03:49 -04:00
MarcoFalke
ff438a1966 Merge bitcoin/bitcoin#21996: p2p: pass strings to NetPermissions::TryParse functions by const ref
39393479c5 p2p: pass strings to NetPermissions::TryParse functions by const ref (Jon Atack)

Pull request description:

  instead of by value, as these are "in" params that are not cheap to copy.

  Reference: https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#f16-for-in-parameters-pass-cheaply-copied-types-by-value-and-others-by-reference-to-const

ACKs for top commit:
  MarcoFalke:
    cr ACK 39393479c5

Tree-SHA512: 294fe0f2d900293b4447d4e1f0ccc60c1ed27b3bdbd0f5d71d3dbf71de86879638b1b813fadfb44c58b4acff4e7d75b7ed6a4f9cc5fcf627108224e6a21b524c
2021-05-19 21:01:29 +02:00
Andrew Chow
9d3bd74ab4 Remove CreateTransaction while loop and some related variables
Remove the CreateTransaction while loop. Removes variables that were
only needed because of that loop. Also renames a few variables and
moves their declarations to where they are used.

Some subtractFeeFromOutputs handling is moved to after coin selection
in order to reduce their amounts once the fee is known.

If subtracting the fee reduces the change to dust, we will also now
remove the change output
2021-05-19 14:58:03 -04:00
Andrew Chow
6f0d5189af Remove use_bnb and bnb_used
These booleans are no longer needed
2021-05-19 14:37:17 -04:00
Andrew Chow
de26eb0e1f Do both BnB and Knapsack coin selection in SelectCoinsMinConf
Instead of switching which algorithm to use based on use_bnb, just run
both in SelectCoinsMinConf. If BnB fails, do Knapsack.
2021-05-19 14:31:28 -04:00
Andrew Chow
01dc8ebda5 Have KnapsackSolver actually use effective values
Although the CreateTransaction loop currently remains, it should be
largely unused. KnapsackSolver will now account for transaction fees
when doing its selection.

In the previous commit, SelectCoinsMinConf was refactored to have some
calculations become shared for KnapsackSolver and SelectCoinsBnB. In
this commit, KnapsackSolver will now use the not_input_fees and
effective_feerate so that it include the fee for non-input things
(excluding a change output) so that the algorithm will select enough to
cover those fees. This is necessary for selecting on effective values.

Additionally, the OutputGroups
created for KnapsackSolver will actually have their effective values
calculated and set, and KnapsackSolver will do its selection on those
effective values.

Lastly, SelectCoins is modified to use the same value for preselected
inputs for BnB and KnapsackSolver. While it will still use the real
value when subtracting the fee from outputs, this behavior will be
the same regardless of the algo used for selecting additional inputs.
2021-05-19 14:25:06 -04:00
Jon Atack
39393479c5 p2p: pass strings to NetPermissions::TryParse functions by const ref 2021-05-19 19:41:05 +02:00
Andrew Chow
bf26e018de Roll static tx fees into nValueToSelect instead of having it be separate
The fees for transaction overhead and recipient outputs are now included
in nTargetValue instead of being a separate parameter. For the coin
selection algorithms, it doesn't matter that these are separate as in
either case, the algorithm needs to select enough to cover these fees.

Note that setting nValueToSelect is changed as it now includes
not_input_fees. Without the change to how nValueToSelect is increased
for KnapsackSolver, this would result in overpaying fees. The change to
increase by the difference between nFeeRet and not_input_fees allows
this to have the same behavior as previously.

Additionally, because we assume that KnapsackSolver will always find a
solution that requires change (we assume that BnB always finds a
non-change solution), we also include the fee for the change output in
KnapsackSolver's target. As part of this, we also use the changeless
nFeeRet when iterating for KnapsackSolver. This is because we include
the change fee when doing KnapsackSolver, so nFeeRet on further
iterations won't include the change fee.
2021-05-19 13:33:31 -04:00
Andrew Chow
cc3f14b27c Move output reductions for fee to after coin selection
Simplifies CreateTransactionInternal without changing behavior. Removes
the pick_new_inputs variable by moving the subtract fee from amount
implementation to later in the loop to where it is possible to calculate
the fee for the transaction. This allows the fee to be subtracted from
the outputs within a single iteration, instead of calculating the fee in
the first iteration, and subtracting the fee in the second.

This also removes another scenario where a second iteration of the loop
finds a smaller input set (and thus smaller fees than the first
iteration) with no change and so a third iteration of the loop is done in order to make
a change output that contains the excess fees.

To handle these cases, we always create a change output which contains
the difference between selected input values and the recipient amounts.
Once the transaction fee is calculated, the change output is reduced (in
the normal case) or the recipient amounts are reduced (in the subtract
fee from amount case). All of this is done in a single iteration of the
loop.
2021-05-19 13:22:27 -04:00
amadeuszpawlik
c0385f10a1 Remove -feefilter option
Feefilter option is debug only and it isn't used in any tests, it's wasteful
to check this option for every peer on every iteration of the message handler
loop. refs #21545
2021-05-19 16:55:03 +02:00
W. J. van der Laan
d4c409cf09 Merge bitcoin/bitcoin#20773: refactor: split CWallet::Create
489ebb7b34 wallet: make chain optional for CWallet::Create (Ivan Metlushko)
d73ae93964 CWallet::Create move chain init message up into calling code (Ivan Metlushko)
44c430ffac refactor: Add CWallet:::AttachChain method (Russell Yanofsky)
e2a47ce085 refactor: move first run detection to client code (Ivan Metlushko)

Pull request description:

  This is a followup for https://github.com/bitcoin/bitcoin/pull/20365#discussion_r522265003
  First part of a refactoring with overall goal to simplify `CWallet` and de-duplicate code with `wallettool`

  **Rationale**: split `CWallet::Create` and create `CWallet::AttachChain`.

  `CWallet::AttachChain` takes chain as first parameter on purpose. In future I suggest we can remove `chain` from `CWallet` constructor.

  The second commit is based on be164f9cf89b123f03b926aa980996919924ee64 from #15719 (thanks ryanofsky)

  cc ryanofsky achow101

ACKs for top commit:
  ryanofsky:
    Code review ACK 489ebb7b34. Only changes since last review were adding a const variable declaration, and implementing suggestion not to move feerate option checks to AttachChain. Thanks for updates and fast responses!

Tree-SHA512: 00235abfe1b00874c56c449adcab8a36582424abb9ba27440bf750af8f3f217b68c11ca74eb30f78a2109ad1d9009315480effc78345e16a3074a1b5d8128721
2021-05-19 16:11:29 +02:00
W. J. van der Laan
39d597d362 Merge bitcoin/bitcoin#21659: net: flag relevant Sock methods with [[nodiscard]]
e286cd0d7b net: flag relevant Sock methods with [[nodiscard]] (Vasil Dimov)

Pull request description:

  Flag relevant Sock methods with `[[nodiscard]]` to avoid issues like the one fixed in https://github.com/bitcoin/bitcoin/pull/21631.

ACKs for top commit:
  practicalswift:
    cr ACK e286cd0d7b: the only changes made are additions of `[[nodiscard]]` and `(void)` where appropriate
  laanwj:
    Code review ACK e286cd0d7b

Tree-SHA512: addc361968d24912bb625b42f4db557791556bf0ffad818252a89a32d76ac22758ec70f8282dcfbfd77eebec20a8e6bb7557c8ed08d50a58de95378c34955973
2021-05-19 15:08:56 +02:00
W. J. van der Laan
087812864b Merge bitcoin/bitcoin#21962: wallet: refactor: dedup sqlite PRAGMA access
9938d610b0 wallet: refactor: dedup sqlite PRAGMA assignments (Sebastian Falbesoner)
dca8ef586c wallet: refactor: dedup sqlite PRAGMA integer reads (Sebastian Falbesoner)

Pull request description:

  This refactoring PR deduplicates repeated SQLite access to PRAGMA settings. Two functions `ReadPragmaInteger(...)` (reads a single integer value via statement `PRAGMA key`) and `SetPragma(...)` (sets a key to specified value via statement `PRAGMA key = value`) are introduced for this purpose.
  This should be more readable and less error-prone, e.g. in case other PRAGMA settings need to be read/set in the future or the error handling has to be adapted.

ACKs for top commit:
  achow101:
    Code Review ACK 9938d610b0
  laanwj:
    Looks good to me now, code review ACK 9938d610b0

Tree-SHA512: 5332788ead6d8d652e28cb0cef1bf0be2b22d6744f8d02dd9e04a4a68e32e14d4a21f94d9b940c37a0d815be3f0091d956c9f6e269b0a6819b62b40482d3bbd2
2021-05-19 14:05:33 +02:00
Jon Atack
ce6bca88e8 doc: release note for getnodeaddresses by network 2021-05-19 13:06:06 +02:00
Jon Atack
3f89c0e990 test: improve getnodeaddresses coverage, test by network 2021-05-19 13:06:04 +02:00
Jon Atack
6c98c09991 rpc: enable filtering getnodeaddresses by network 2021-05-19 13:06:02 +02:00
Jon Atack
80ba294854 p2p: allow CConnman::GetAddresses() by network, add doxygen 2021-05-19 13:05:54 +02:00
Jon Atack
a49f3ddbba p2p: allow CAddrMan::GetAddr() by network, add doxygen 2021-05-19 13:04:11 +02:00
João Barbosa
c38981e748 p2p: pull time call out of loop in CAddrMan::GetAddr_() 2021-05-19 13:04:09 +02:00
Jon Atack
d35ddca91e p2p: enable CAddrMan::GetAddr_() by network, add doxygen 2021-05-19 13:04:07 +02:00
W. J. van der Laan
4da26fb85d Merge bitcoin/bitcoin#21506: p2p, refactor: make NetPermissionFlags an enum class
7075f604e8 scripted-diff: update noban documentation in net_processing.cpp (Jon Atack)
a95540cf43 scripted-diff: rename NetPermissionFlags enumerators (Jon Atack)
810d0929c1 p2p, refactor: make NetPermissionFlags a uint32 enum class (Jon Atack)
7b55a94497 p2p: NetPermissions::HasFlag() pass flags param by value (Jon Atack)
91f6e6e6d1 scripted-diff: add NetPermissionFlags scopes where not already present (Jon Atack)

Pull request description:

  While reviewing #20196, I noticed the `NetPermissionFlags` enums are frequently called as if they were scoped, yet are still global. This patch upgrades `NetPermissionFlags` to a scoped class enum and updates the enumerator naming, similarly to #19771. See https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#enum-enumerations for more info.

  This change would eliminate the class of bugs like https://github.com/bitcoin/bitcoin/pull/20196#discussion_r610770148 and #21644, as only defined operations on the flags would compile.

ACKs for top commit:
  laanwj:
    Code review ACK 7075f604e8
  vasild:
    ACK 7075f604e8

Tree-SHA512: 7fcea66ee499f059efc78c934b5f729b3c8573fe304dee2c27c837c2f662b89324790568246d75b2a574cf9f059b42d3551d928996862f4358055eb43521e6f4
2021-05-19 11:57:24 +02:00
W. J. van der Laan
1ed859e90e Merge bitcoin/bitcoin#21173: util: faster HexStr => 13% faster blockToJSON
74bf850ac4 faster HexStr => 13% faster blockToJSON (Martin Ankerl)

Pull request description:

  `std::string`'s push_back is rather slow because it needs to check & update the string size. For
  `HexStr` the output string size is already easily know, so we can initially create the string with
  the correct size and then just assign the data.

  `HexStr` is heavily usd in `blockToJSON`, so this change is a noticeable benefit. Benchmark on an i7-8700 @3.2GHz:

  * 71,315,461.00 ns/op master
  * 62,842,490.00 ns/op this commit

  So this little change makes `blockToJSON` about ~13% faster.

ACKs for top commit:
  laanwj:
    Code review ACK 74bf850ac4
  theStack:
    re-ACK 74bf850ac4

Tree-SHA512: fc99105123edc11f4e40ed77aea80cf7f32e49c53369aa364b38395dcb48575e15040b0489ed30d0fe857c032a04e225c33e9d95cdfa109a3cb5a6ec9a972415
2021-05-19 10:07:53 +02:00
W. J. van der Laan
2fc111b6e3 Merge bitcoin/bitcoin#21985: net: Return IPv6 scope id in CNetAddr::ToStringIP()
6c280adcd8 net: Return IPv6 scope id in `CNetAddr::ToStringIP()` (W. J. van der Laan)

Pull request description:

  If a scope id is provided, return it back in the string representation. Also bring back the test (now in platform independent fashion). Closes #21982. Includes #21961 (apart from the MacOS remark).

ACKs for top commit:
  practicalswift:
    cr ACK 6c280adcd8

Tree-SHA512: 77792c35679b6c3545fd3a8d3d74c4f515ac2ee9f02d983251aeaaac715d55c122bbb0141abbeac272011f15520b439bd2db4ec8541a58df9b366921d212ca5f
2021-05-19 09:22:36 +02:00
Ivan Metlushko
489ebb7b34 wallet: make chain optional for CWallet::Create 2021-05-19 08:50:20 +02:00
Ivan Metlushko
d73ae93964 CWallet::Create move chain init message up into calling code 2021-05-19 08:50:20 +02:00
Russell Yanofsky
44c430ffac refactor: Add CWallet:::AttachChain method
This commit does not change behavior, it just moves code from
CWallet::CreateWalletFromFile to CWallet:::AttachChain so it can be updated in
the next commit.

This commit is most easily reviewed with
"git diff -w --color-moved=dimmed_zebra" or by diffing CWallet:::AttachChain
against the previous code with an external diff tool.
2021-05-19 08:50:20 +02:00
Ivan Metlushko
e2a47ce085 refactor: move first run detection to client code 2021-05-19 08:50:16 +02:00
W. J. van der Laan
2fa3f30050 Merge bitcoin/bitcoin#21988: doc: note that brew installed qt is not supported
33b0b26a03 doc: note that brew installed qt is not supported (Raul Siles)

Pull request description:

  picking up #21791, the author has stated they [cannot squash](https://github.com/bitcoin/bitcoin/pull/21791#issuecomment-828770283).

  This is a useful note to prevent any issues from being opened up about this. The reason that both cannot co-exist and build bitcoin is stated [here](https://github.com/bitcoin/bitcoin/pull/21791#issuecomment-837278123):
  > ... the reason is sharing /usr/local/include/ and /usr/local/lib/ directories by both qt5 and qt6 installations.

  Changes from original PR:
  - slightly move the note up in this section, this placement seems more appropriate to me
  - drop "Note:"

  [PR Render](33b0b26a03/doc/build-osx.md (qt))

ACKs for top commit:
  laanwj:
    LGTM ACK 33b0b26a03
  hebasto:
    ACK 33b0b26a03

Tree-SHA512: f9efac1921a7a33b5791a9f9f4bada4b5369d358fc42e9884c077bfb4dc3f273fdd4432ce012006a8009dfafb87e13bddd56c6336fe84b6133f4b22f849c289a
2021-05-18 22:03:34 +02:00
W. J. van der Laan
1da36b49ab Merge bitcoin/bitcoin#21983: build: Silence lupdate "unknown namespace/class" warnings
88bdc4d33e build: Silent lupdate "unknown namespace/class" warnings (Hennadii Stepanov)

Pull request description:

  This PR removes multiple _"Qualifying with unknown namespace/class"_ warnings in `make -C src translate` output.
  Also all of the `lupdate` options are moved before input files (as documented).

  The remaining warnings are fixed in Qt 5.12.2 (see [QTBUG-42736](https://bugreports.qt.io/browse/QTBUG-42736)).

ACKs for top commit:
  laanwj:
    Tested ACK 88bdc4d33e

Tree-SHA512: 9c12ff5425a84758aaf8073554891ffe7eb2d75650a12be14ece364b6b7a4461626654885e35c5543462f0941abb020cc36dc64e656c85c8d6d2da36efc7ea2c
2021-05-18 21:48:53 +02:00
W. J. van der Laan
6c280adcd8 net: Return IPv6 scope id in CNetAddr::ToStringIP()
If a scope id is provided, return it back in the string representation.
Also bring back the test. Closes #21982.

Co-authored-by: Jon Atack <jon@atack.com>
2021-05-18 21:01:32 +02:00
Hennadii Stepanov
88bdc4d33e build: Silent lupdate "unknown namespace/class" warnings
This change removes multiple "Qualifying with unknown namespace/class"
warnings.
Also all options are moved before input files (as documented).
2021-05-18 18:25:58 +03:00
fanquake
741749a615 Merge bitcoin/bitcoin#21920: build: improve macro for testing -latomic requirement
fa25ce45e9 build: improve macro for testing -latomic requirement (MarcoFalke)

Pull request description:

  This fixes the issue where `-latomic` is incorrectly omitted from the linker flags.

  Steps to reproduce on vanilla Ubuntu Focal:

  ```
  export DEBIAN_FRONTEND=noninteractive && apt update && apt install curl wget htop git vim ccache -y && git clone https://github.com/bitcoin/bitcoin.git && cd bitcoin && apt install build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq make automake cmake curl clang llvm g++-multilib libtool binutils-gold bsdmainutils pkg-config python3 patch bison -y  && ( cd depends && make DEBUG=1 HOST=i686-pc-linux-gnu NO_QT=1 NO_WALLET=1 NO_ZMQ=1 NO_UPNP=1 NO_NATPMP=1 -j $(nproc) ) && ./autogen.sh && CONFIG_SITE="$PWD/depends/i686-pc-linux-gnu/share/config.site" ./configure CC='clang -m32' CXX='clang++ -m32' --enable-fuzz --with-sanitizers=fuzzer && make  -j $(nproc)
  ```

  Before:
  ```
  /usr/bin/ld: libbitcoin_server.a(libbitcoin_server_a-net.o): in function `std::atomic<std::chrono::duration<long long, std::ratio<1ll, 1000000ll> > >::load(std::memory_order) const':
  net.cpp:(.text._ZNKSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1000000EEEEE4loadESt12memory_order[_ZNKSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1000000EEEEE4loadESt12memory_order]+0x51): undefined reference to `__atomic_load'
  /usr/bin/ld: libbitcoin_server.a(libbitcoin_server_a-net.o): in function `std::atomic<std::chrono::duration<long long, std::ratio<1ll, 1000000ll> > >::store(std::chrono::duration<long long, std::ratio<1ll, 1000000ll> >, std::memory_order)':
  net.cpp:(.text._ZNSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1000000EEEEE5storeES4_St12memory_order[_ZNSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1000000EEEEE5storeES4_St12memory_order]+0x5f): undefined reference to `__atomic_store'
  /usr/bin/ld: libbitcoin_server.a(libbitcoin_server_a-net.o): in function `std::atomic<ServiceFlags>::load(std::memory_order) const':
  net.cpp:(.text._ZNKSt6atomicI12ServiceFlagsE4loadESt12memory_order[_ZNKSt6atomicI12ServiceFlagsE4loadESt12memory_order]+0x51): undefined reference to `__atomic_load'
  /usr/bin/ld: libbitcoin_server.a(libbitcoin_server_a-net_processing.o): in function `std::atomic<ServiceFlags>::store(ServiceFlags, std::memory_order)':
  net_processing.cpp:(.text._ZNSt6atomicI12ServiceFlagsE5storeES0_St12memory_order[_ZNSt6atomicI12ServiceFlagsE5storeES0_St12memory_order]+0x6c): undefined reference to `__atomic_store'
  /usr/bin/ld: libbitcoin_server.a(libbitcoin_server_a-net_processing.o): in function `std::atomic<std::chrono::duration<long long, std::ratio<1ll, 1ll> > >::load(std::memory_order) const':
  net_processing.cpp:(.text._ZNKSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1EEEEE4loadESt12memory_order[_ZNKSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1EEEEE4loadESt12memory_order]+0x51): undefined reference to `__atomic_load'
  /usr/bin/ld: libbitcoin_server.a(libbitcoin_server_a-net_processing.o): in function `std::atomic<std::chrono::duration<long long, std::ratio<1ll, 1ll> > >::store(std::chrono::duration<long long, std::ratio<1ll, 1ll> >, std::memory_order)':
  net_processing.cpp:(.text._ZNSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1EEEEE5storeES4_St12memory_order[_ZNSt6atomicINSt6chrono8durationIxSt5ratioILx1ELx1EEEEE5storeES4_St12memory_order]+0x5f): undefined reference to `__atomic_store'
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  ```

  After:

  Clean

ACKs for top commit:
  jarolrod:
    ACK fa25ce45e9
  hebasto:
    ACK fa25ce45e9, tested on Ubuntu 20.04.2 LTS.
  fanquake:
    ACK fa25ce45e9

Tree-SHA512: 7b0e76876c28a4ef8d8cde1ea26c18aeec51da3d54e4da75829276b9b2a259f9d0feab215be0d39e651c6bd89071859bc29aa48131e627e745a828d9f8d4a98e
2021-05-18 20:33:10 +08:00
MarcoFalke
fafd121026 refactor: Make CFeeRate constructor architecture-independent 2021-05-18 07:13:25 +02:00
fanquake
f152c1ac66 build: libevent 2.1.12-stable 2021-05-18 10:19:10 +08:00
Raul Siles
33b0b26a03 doc: note that brew installed qt is not supported 2021-05-17 21:36:08 -04:00
Fabian Jahr
e6fe1c37d0 rpc: Improve avoidpartialspends and avoid_reuse documentation 2021-05-18 02:11:47 +02:00
Fabian Jahr
8f073076b1 wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100 2021-05-18 02:09:18 +02:00
Jarol Rodriguez
7eea659fc9 qt, test: use qsignalspy instead of qeventloop 2021-05-17 15:51:24 -04:00
Sebastian Falbesoner
4bea301692 test: use P2PK-MiniWallet for feature_csv_activation.py
Using the MiniWallet in P2PK mode, all transactions submitted to the
mempool are following the standard policy now, i.e. the node command
line parameter '-acceptnonstdtxn=1' is not needed anymore.
2021-05-17 20:12:34 +02:00
Hennadii Stepanov
2a45134b56 qt: Add shortcuts for console font resize buttons
Co-authored-by: Jarol Rodriguez <jarolrod@tutanota.com>
2021-05-17 14:01:38 -04:00
Hennadii Stepanov
a2e122f0fe qt: Add GUIUtil::AddButtonShortcut
Co-authored-by: Jarol Rodriguez <jarolrod@tutanota.com>
2021-05-17 14:01:00 -04:00
Sebastian Falbesoner
dc7eb64e83 test: MiniWallet: add P2PK support 2021-05-17 19:50:31 +02:00
Sebastian Falbesoner
9938d610b0 wallet: refactor: dedup sqlite PRAGMA assignments 2021-05-17 17:22:36 +02:00
Sebastian Falbesoner
dca8ef586c wallet: refactor: dedup sqlite PRAGMA integer reads 2021-05-17 17:22:31 +02:00
Hennadii Stepanov
6d1d33d334 Merge bitcoin-core/gui#323: English translations update
c34a49f77f qt: English translations update (W. J. van der Laan)
55c012d20c qt: Extract translations correctly from UTF-8 formatted source (Hennadii Stepanov)

Pull request description:

  Update for Transifex. Needed after bitcoin/bitcoin#21836.

ACKs for top commit:
  hebasto:
    ACK c34a49f77f, verified that `make -C src translate` returns zero updated strings (as all of them are updated in the top commit):

Tree-SHA512: 01170b4ef82dc01104e21ec9fe1115c3defb50ffd697bb59864ca4bb132765b6211d048b009d8d3dab4606e023ba33ba7c6c8c2dd086941046782b5009a83bb0
2021-05-17 16:18:10 +03:00
W. J. van der Laan
c34a49f77f qt: English translations update
Update for Transifex. Needed after bitcoin/bitcoin#21836.
2021-05-17 13:26:29 +02:00
Hennadii Stepanov
55c012d20c qt: Extract translations correctly from UTF-8 formatted source 2021-05-17 13:21:13 +02:00
W. J. van der Laan
7b87fca930 Merge bitcoin/bitcoin#21756: Avoid calling getnameinfo when formatting IPv6 addresses in CNetAddr::ToStringIP
54548bae80 net: Avoid calling getnameinfo when formatting IPv6 addresses in CNetAddr::ToStringIP (practicalswift)
c10f27fdb2 net: Make IPv6ToString do zero compression as described in RFC 5952 (practicalswift)

Pull request description:

  Avoid calling `getnameinfo` when formatting IPv6 addresses in `CNetAddr::ToStringIP`.

  Fixes #21466.
  Fixes #21967.

  The IPv4 case was fixed in #21564.

ACKs for top commit:
  laanwj:
    Code review ACK 54548bae80
  vasild:
    ACK 54548bae80

Tree-SHA512: 8404e458b29efdb7bf78b91adc075d05e0385969d1532cccaa2c7cb69cd77411c42d95fcefc4000137b9f2076fe395731c7d9844b7d42b58a6d3bec69eed6fce
2021-05-17 13:08:14 +02:00
MarcoFalke
fae4ee545a fuzz: Add missing CheckTransaction before CheckTxInputs 2021-05-17 10:04:57 +02:00
MarcoFalke
faacb7eadb fuzz: Sanity check result of CheckTransaction 2021-05-17 10:04:53 +02:00
fanquake
1ef34ee25e Merge bitcoin/bitcoin#21947: doc: Fix OSS-Fuzz links
fadd98d02f doc: Fix OSS-Fuzz links (MarcoFalke)

Pull request description:

  * Add missing link to the coverage report
  * Replace unqualified link with qualified one

ACKs for top commit:
  practicalswift:
    ACK fadd98d02f
  jarolrod:
    ACK fadd98d02f

Tree-SHA512: 5d5138288ae75d6a9ae536f307a63813ffe60c628a8c9a74b33f0d1be1a34fac2ff5eca9c9fe64be3139f2251736ba088c5f3731f0b3f50c70c5a5f1a89f0ea4
2021-05-16 19:59:27 +10:00
MarcoFalke
fa91994b1b fuzz: Add utxo_snapshot target 2021-05-16 11:34:27 +02:00
Hennadii Stepanov
8b77133651 qt: Replace disambiguation strings with translator comments
Translator comments is the right way to pass context to translators.
2021-05-16 01:06:22 +03:00
Hennadii Stepanov
a02c970eb0 qt, refactor: Revert explicit including QStringBuilder 2021-05-15 19:10:47 +03:00
Hennadii Stepanov
3fd3a0fc87 qt, build: Optimize string concatenation
The defined QT_USE_QSTRINGBUILDER macro means using the QStringBuilder
for efficient string concatenation in all Qt code by default.
2021-05-15 19:10:38 +03:00
Jarol Rodriguez
8b419b5163 qt: make console buttons look clickable
Change the type for the console's buttons to QToolButton which will make them look explicitly clickable, which in turn fixes the small hitbox issue for macOS.
With this change, we need to generalize the respective action connect logic from QPushButton to QAbstractButton.
While here, update width and height of icon for consistency with other tool buttons.
2021-05-15 10:01:02 -04:00
MarcoFalke
c857148636 Merge bitcoin/bitcoin#21948: test: Fix off-by-one in mockscheduler test RPC
fa2e614d16 test: Fix off-by-one in mockscheduler test RPC (MarcoFalke)

Pull request description:

  Fixes:

  ```
  fuzz: scheduler.cpp:83: void CScheduler::MockForward(std::chrono::seconds): Assertion `delta_seconds.count() > 0 && delta_seconds < std::chrono::hours{1}' failed.
  ==1059066== ERROR: libFuzzer: deadly signal
      #0 0x558f75449c10 in __sanitizer_print_stack_trace (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x5fec10)
      #1 0x558f753f32b8 in fuzzer::PrintStackTrace() fuzzer.o
      #2 0x558f753d68d3 in fuzzer::Fuzzer::CrashCallback() fuzzer.o
      #3 0x7f4a3cbbb3bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
      #4 0x7f4a3c7ff18a in raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
      #5 0x7f4a3c7de858 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
      #6 0x7f4a3c7de728  (/lib/x86_64-linux-gnu/libc.so.6+0x25728)
      #7 0x7f4a3c7eff35 in __assert_fail (/lib/x86_64-linux-gnu/libc.so.6+0x36f35)
      #8 0x558f7588a913 in CScheduler::MockForward(std::chrono::duration<long, std::ratio<1l, 1l> >) scheduler.cpp:83:5
      #9 0x558f75b0e5b1 in mockscheduler()::$_7::operator()(RPCHelpMan const&, JSONRPCRequest const&) const rpc/misc.cpp:435:30
      #10 0x558f75b0e5b1 in std::_Function_handler<UniValue (RPCHelpMan const&, JSONRPCRequest const&), mockscheduler()::$_7>::_M_invoke(std::_Any_data const&, RPCHelpMan const&, JSONRPCRequest const&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:285:9
      #11 0x558f7587a141 in std::function<UniValue (RPCHelpMan const&, JSONRPCRequest const&)>::operator()(RPCHelpMan const&, JSONRPCRequest const&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #12 0x558f7587a141 in RPCHelpMan::HandleRequest(JSONRPCRequest const&) const rpc/util.cpp:565:26
      #13 0x558f756c0086 in CRPCCommand::CRPCCommand(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, RPCHelpMan (*)())::'lambda'(JSONRPCRequest const&, UniValue&, bool)::operator()(JSONRPCRequest const&, UniValue&, bool) const ./rpc/server.h:110:91
      #14 0x558f756c0086 in std::_Function_handler<bool (JSONRPCRequest const&, UniValue&, bool), CRPCCommand::CRPCCommand(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, RPCHelpMan (*)())::'lambda'(JSONRPCRequest const&, UniValue&, bool)>::_M_invoke(std::_Any_data const&, JSONRPCRequest const&, UniValue&, bool&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:285:9
      #15 0x558f756b8592 in std::function<bool (JSONRPCRequest const&, UniValue&, bool)>::operator()(JSONRPCRequest const&, UniValue&, bool) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #16 0x558f756b8592 in ExecuteCommand(CRPCCommand const&, JSONRPCRequest const&, UniValue&, bool) rpc/server.cpp:480:20
      #17 0x558f756b8592 in ExecuteCommands(std::vector<CRPCCommand const*, std::allocator<CRPCCommand const*> > const&, JSONRPCRequest const&, UniValue&) rpc/server.cpp:444:13
      #18 0x558f756b8017 in CRPCTable::execute(JSONRPCRequest const&) const rpc/server.cpp:464:13
      #19 0x558f7552457a in (anonymous namespace)::RPCFuzzTestingSetup::CallRPC(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) test/fuzz/rpc.cpp:50:25
      #20 0x558f7552457a in rpc_fuzz_target(Span<unsigned char const>) test/fuzz/rpc.cpp:354:28
      #21 0x558f7544cf0f in std::_Function_handler<void (Span<unsigned char const>), void (*)(Span<unsigned char const>)>::_M_invoke(std::_Any_data const&, Span<unsigned char const>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:300:2
      #22 0x558f75c05197 in std::function<void (Span<unsigned char const>)>::operator()(Span<unsigned char const>) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #23 0x558f75c05197 in LLVMFuzzerTestOneInput test/fuzz/fuzz.cpp:74:5
      #24 0x558f753d8073 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) fuzzer.o
      #25 0x558f753c1f72 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) fuzzer.o
      #26 0x558f753c7d6a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) fuzzer.o
      #27 0x558f753f3a92 in main (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x5a8a92)
      #28 0x7f4a3c7e00b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
      #29 0x558f7539cc9d in _start (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x551c9d)

ACKs for top commit:
  practicalswift:
    cr ACK fa2e614d16

Tree-SHA512: cfa120265261f0ad019b46c426b915c1c007806b37aecb27016ce780a0ddea5e6fc9b09065fd40684b11183dcd3bf543558d7a655e604695021653540266baf7
2021-05-15 09:55:02 +02:00
Antoine Riard
2eb0eeda39 validation: document lack of inherited signaling in RBF policy 2021-05-14 14:27:30 -04:00
Antoine Riard
906b6d9da6 test: Extend feature_rbf.py with no inherited signaling 2021-05-14 14:12:29 -04:00
MarcoFalke
fa2e614d16 test: Fix off-by-one in mockscheduler test RPC 2021-05-14 12:49:16 +02:00
W. J. van der Laan
ecf5f2c1a0 Merge bitcoin/bitcoin#21889: macho: check for control flow instrumentation
42b589d18f scripts: test for MACHO control flow instrumentation (fanquake)
469a5bc4fa build: build Boost with -fcf-protection when targeting Darwin (fanquake)

Pull request description:

  Addresses the macOS portion of #21888.

  Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation.

  Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
  From the Intel control flow enforcement documentation:

  > The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.

  Follow up from #21135.

  Guix builds:
  ```bash
  663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
  45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
  0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
  85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001  guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz
  ```

  Gitian builds:
  ```bash
  # macOS:
  bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96  bitcoin-42b589d18fed-osx-unsigned.dmg
  f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9  bitcoin-42b589d18fed-osx-unsigned.tar.gz
  29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7  bitcoin-42b589d18fed-osx64.tar.gz
  663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9  src/bitcoin-42b589d18fed.tar.gz
  366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 42b589d18f

Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
2021-05-14 12:26:38 +02:00
W. J. van der Laan
b82c3a0075 Merge bitcoin/bitcoin#21929: fuzz: Remove incorrect float round-trip serialization test
fae814c9a6 fuzz: Remove incorrect float round-trip serialization test (MarcoFalke)

Pull request description:

  It tests the wrong way of the round-trip: `int -> float -> int`, but only `float -> int -> float` is allowed and used. See also `src/test/fuzz/float.cpp`.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34118

ACKs for top commit:
  laanwj:
    Anyhow, ACK fae814c9a6

Tree-SHA512: 8412a7985be2225109f382b7c7ea6d6fcfbea15711671fdf2f41dd1a9adbb3b4489592863751d78bedaff98e9b0b13571d9cae06ffd92db8fbf7ce0f47874a41
2021-05-14 12:14:30 +02:00
MarcoFalke
61fea52171 Merge bitcoin/bitcoin#21942: docs: improve make with parallel jobs description.
07bc22ef10 docs: improve make with parallel jobs description. (Klement Tan)

Pull request description:

  Changed `use -jX here for parallelism` to `use "-j N" for N parallel jobs`

  **Rationale**: In my opinion `use -jX here for parallelism` is quite ambiguous as it could be perceived as a single option without any argument. Ie running:
  ```sh
  make -jX
  ```

  Embarrassingly this caused me to be stuck for quite a long time until I opened the help menu for `make` but if I am the only one who faced this issue I would be happy to close this PR.

ACKs for top commit:
  jarolrod:
    ACK 07bc22ef10

Tree-SHA512: 2d119b6a461668906c63184b865d2cc9fb2f75abeba34e2e44bc1ef3bcb4adec4a49896ddaf3cc6a20c0095ad20d0de0908401b351eaca9443161d24d6b20d0b
2021-05-14 08:22:02 +02:00
MarcoFalke
fadd98d02f doc: Fix OSS-Fuzz links 2021-05-14 08:15:03 +02:00
Hennadii Stepanov
4ee9ee7236 qt: Use native presentation of shortcut 2021-05-14 02:01:23 -04:00
Klement Tan
07bc22ef10 docs: improve make with parallel jobs description. 2021-05-14 08:45:27 +08:00
Andrew Chow
d97d25d950 Make cost_of_change part of CoinSelectionParams 2021-05-13 16:40:56 -04:00
Andrew Chow
af5867c896 Move some calculations to common code in SelectCoinsMinConf
To prepare for KnapsackSolver to use effective values, these
calculations are moved out of the BnB if block to allow for them to be
shared with KnapsackSolver in the future.
2021-05-13 16:40:56 -04:00
Andrew Chow
1bf4a62cb6 scripted-diff: rename some variables
actual_target -> selection_target
nChange -> change_and_fee

-BEGIN VERIFY SCRIPT-
sed -i -e 's/actual_target/selection_target/g' src/wallet/coinselection.cpp
sed -i -e '2801,3691s/nChange /change_and_fee /g' src/wallet/wallet.cpp
sed -i -e '2801,3691s/nChange,/change_and_fee,/g' src/wallet/wallet.cpp
sed -i -e '2801,3691s/nChange;/change_and_fee;/g' src/wallet/wallet.cpp
-END VERIFY SCRIPT-
2021-05-13 16:40:56 -04:00
Carl Dong
ee0a67c32a codesigning: Use SHA256 as digest for osslsigncode (Windows) 2021-05-13 15:41:56 -04:00
Carl Dong
38eb91eb06 guix: Add codesigning functionality 2021-05-13 15:41:56 -04:00
Carl Dong
bac2690e6f guix: Package codesigning tools 2021-05-13 15:41:56 -04:00
Carl Dong
0a2176d477 guix: Reindent existing manifest.scm 2021-05-13 15:41:56 -04:00
Carl Dong
c090a3e923 Makefile.am: use APP_DIST_DIR instead of hard-coding dist 2021-05-13 15:41:56 -04:00
W. J. van der Laan
b34bf2b42c Merge bitcoin/bitcoin#21939: refactor: Replace memset calls with array initialization
1c9255c7dd refactor: Replace memset calls with array initialization (João Barbosa)

Pull request description:

  Follow up to https://github.com/bitcoin/bitcoin/pull/21905#pullrequestreview-657045699.

ACKs for top commit:
  laanwj:
    re-ACK 1c9255c7dd
  Crypt-iQ:
    Code review ACK 1c9255c7dd

Tree-SHA512: 4b61dec2094f4781ef1c0427ee3bda3cfea12111274eebc7bc40a84f261d9c1681dd0860c57200bea2456588e44e8e0aecd18545c25f1f1250dd331ab7d05f28
2021-05-13 19:25:21 +02:00
W. J. van der Laan
4741aec1dd Merge bitcoin/bitcoin#21914: net: use stronger AddLocal() for our I2P address
105941b726 net: use stronger AddLocal() for our I2P address (Vasil Dimov)

Pull request description:

  There are two issues:

  ### 1. Our I2P address not added to local addresses.

  * `externalip=` is used with an IPv4 address (this sets automatically `discover=0`)
  * No `discover=1` is used
  * `i2psam=` is used
  * No `externalip=` is used for our I2P address
  * `listenonion=1 torcontrol=` are used

  In this case `AddLocal(LOCAL_MANUAL)` [is used](94f83534e4/src/torcontrol.cpp (L354)) for our `.onion` address and `AddLocal(LOCAL_BIND)` [for our](94f83534e4/src/net.cpp (L2247)) `.b32.i2p` address, the latter being [ignored](94f83534e4/src/net.cpp (L232-L233)) due to `discover=0`.

  ### 2. Our I2P address removed from local addresses even if specified with `externalip=` on I2P proxy restart.

  * `externalip=` is used with our I2P address (this sets automatically `discover=0`)
  * No `discover=1` is used
  * `i2psam=` is used

  In this case, initially `externalip=` causes our I2P address to be [added](94f83534e4/src/init.cpp (L1266)) with `AddLocal(LOCAL_MANUAL)` which overrides `discover=0` and works as expected. However, if later the I2P proxy is shut down [we do](94f83534e4/src/net.cpp (L2234)) `RemoveLocal()` in order to stop advertising our I2P address (since we have lost I2P connectivity). When the I2P proxy is started and we reconnect to it, restoring the I2P connectivity, [we do](94f83534e4/src/net.cpp (L2247)) `AddLocal(LOCAL_BIND)` which does nothing due to `discover=0`.

  To resolve those two issues, use `AddLocal(LOCAL_MANUAL)` for I2P which is also what we do with Tor.

ACKs for top commit:
  laanwj:
    Code review ACK 105941b726

Tree-SHA512: 0c9daf6116b8d9c34ad7e6e9bbff6e8106e94e4394a815d7ae19287aea22a8c7c4e093c8dd8c58a4a1b1412b2575a9b42b8a93672c8d17f11c24508c534506c7
2021-05-13 15:36:44 +02:00
João Barbosa
1c9255c7dd refactor: Replace memset calls with array initialization 2021-05-13 12:42:21 +01:00
Samuel Dobson
a31a1ceec7 Merge bitcoin/bitcoin#21907: wallet: Do not iterate a directory if having an error while accessing it
29c9e2c2d2 wallet: Do not iterate a directory if having an error while accessing it (Hennadii Stepanov)

Pull request description:

  On Windows when `ListDatabases` tries to iterate any system folder, e.g., "System Volume Information", it falls into an infinite loop.

  This PR fixes this bug. Now the `debug.log` contains:
  ```
  2021-05-12T09:07:53Z ListDatabases: Access is denied D:/System Volume Information -- skipping.
  ```

  An easy way to reproduce the bug and test this PR is to pass the `-walletdir=D:\` command-line option, and run the `listwalletdir` RPC, or File -> Open Wallet in the GUI menu.

  Fixes #20081.
  Fixes #21136.
  Fixes #21904.

  Also https://bitcoin.stackexchange.com/questions/99243/listwalletdir-access-is-denied-d-system-volume-information

ACKs for top commit:
  prayank23:
    ACK 29c9e2c2d2
  promag:
    Code review ACK 29c9e2c2d2.
  meshcollider:
    Code review ACK 29c9e2c2d2

Tree-SHA512: b851c88e6d09626f4cb81acc2fa59a563b2aee64582963285715bf785c64b872e8bf738aa6b27bdbaf4c3e5c8565c2dc2c802135f9aa1f48b4b913435bc5d793
2021-05-13 21:09:32 +12:00
Samuel Dobson
386ba92e83 Merge bitcoin/bitcoin#21910: refactor: remove redundant fOnlySafe argument
c30dd02cd8 refactor: remove redundant fOnlySafe argument (t-bast)

Pull request description:

  The `fOnlySafe` argument to `AvailableCoins` is now redundant, since #21359 added a similar field inside the `CCoinControl` struct (see https://github.com/bitcoin/bitcoin/pull/21359#discussion_r591578684).

  Not all code paths create a `CCoinControl` instance, but when it's missing we can default to using only safe inputs which is backwards-compatible.

ACKs for top commit:
  instagibbs:
    utACK c30dd02cd8
  promag:
    Code review ACK c30dd02cd8.
  achow101:
    ACK c30dd02cd8
  meshcollider:
    Code review + test run ACK c30dd02cd8

Tree-SHA512: af3cb598d06f233fc48a7c9c45bb14da92b5cf4168b8dbd4f134dc3e0c2b615c6590238ddb1eaf380aea5bbdd3386d2ac8ecd7d22dfc93579adc39248542839b
2021-05-13 21:05:55 +12:00
MarcoFalke
db2990d01f Merge bitcoin/bitcoin#21925: doc: Update bips.md for 0.21.1
faf30f2ae0 doc: Update bips.md for 0.21.1 (MarcoFalke)

Pull request description:

ACKs for top commit:
  kristapsk:
    ACK faf30f2ae0
  jarolrod:
    ACK faf30f2ae0
  prayank23:
    ACK faf30f2ae0

Tree-SHA512: b09e8782306fe990d4eba871627cb7aa97a521a5aa08b7c7369fbfc24e49c3c9386cd06c590af076dff9b95ade207cc470911aa4b97cd5f366630974e71e1348
2021-05-13 09:15:15 +02:00
MarcoFalke
0ab6ff5e37 Merge bitcoin/bitcoin#21931: ci: Bump cirrus fuzz CPUs to avoid timeout
fa397a6a9c ci: Bump cirrus fuzz CPUs to avoid timeout (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fa397a6a9c, let's try it.

Tree-SHA512: 7e06dda66c71d76e5fd144f6b5bb10f0bcac72feb15bd0f400ef08ba4dcb92558319401ef5f9d3822376affceb2192df1903b3a79c0ab2d7283ca21454054dea
2021-05-12 21:25:34 +02:00
Hennadii Stepanov
69b8b5d72c qt: Align numbers in the "Peer Id" column to the right 2021-05-12 22:05:25 +03:00
W. J. van der Laan
ee9befe8b4 Merge bitcoin/bitcoin#21584: Fix assumeutxo crash due to invalid base_blockhash
fa340b8794 refactor: Avoid magic value of all-zeros in assumeutxo base_blockhash (MarcoFalke)
fae33f98e6 Fix assumeutxo crash due to invalid base_blockhash (MarcoFalke)
fa5668bfb3 refactor: Use type-safe assumeutxo hash (MarcoFalke)
0000007709 refactor: Remove unused code (MarcoFalke)
faa921f787 move-only: Add util/hash_type (MarcoFalke)

Pull request description:

  Starting with commit d6af06d68a, a block hash of all-zeros is invalid and will lead to a crash of the node. Can be tested by cherry-picking the test changes without the other changes.

  Stack trace (copied from https://github.com/bitcoin/bitcoin/pull/21584#discussion_r612673879):

  ```
  #0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
  #1  0x00007ffff583c8b1 in __GI_abort () at abort.c:79
  #2  0x00007ffff582c42a in __assert_fail_base (fmt=0x7ffff59b3a38 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
      assertion=assertion@entry=0x555556c8b450 "!hashBlock.IsNull()", file=file@entry=0x555556c8b464 "txdb.cpp", line=line@entry=89,
      function=function@entry=0x555556c8b46d "virtual bool CCoinsViewDB::BatchWrite(CCoinsMap &, const uint256 &)") at assert.c:92
  #3  0x00007ffff582c4a2 in __GI___assert_fail (assertion=0x555556c8b450 "!hashBlock.IsNull()", file=0x555556c8b464 "txdb.cpp", line=89,
      function=0x555556c8b46d "virtual bool CCoinsViewDB::BatchWrite(CCoinsMap &, const uint256 &)") at assert.c:101
  #4  0x000055555636738b in CCoinsViewDB::BatchWrite (this=0x5555577975c0, mapCoins=std::unordered_map with 110 elements = {...}, hashBlock=...) at txdb.cpp:89
  #5  0x00005555564a2e80 in CCoinsViewBacked::BatchWrite (this=0x5555577975f8, mapCoins=std::unordered_map with 110 elements = {...}, hashBlock=...) at coins.cpp:30
  #6  0x00005555564a43de in CCoinsViewCache::Flush (this=0x55555778eaf0) at coins.cpp:223
  #7  0x00005555563fc11d in ChainstateManager::PopulateAndValidateSnapshot (this=0x55555740b038 <g_chainman>, snapshot_chainstate=..., coins_file=..., metadata=...)
      at validation.cpp:5422
  #8  0x00005555563fab3d in ChainstateManager::ActivateSnapshot (this=0x55555740b038 <g_chainman>, coins_file=..., metadata=..., in_memory=true) at validation.cpp:5299
  #9  0x0000555555e8c893 in validation_chainstatemanager_tests::CreateAndActivateUTXOSnapshot<validation_chainstatemanager_tests::chainstatemanager_activate_snapshot::test_method()::$_12>(NodeContext&, boost::filesystem::path, validation_chainstatemanager_tests::chainstatemanager_activate_snapshot::test_method()::$_12) (node=...,
      root=..., malleation=...) at test/validation_chainstatemanager_tests.cpp:199
  #10 0x0000555555e8877a in validation_chainstatemanager_tests::chainstatemanager_activate_snapshot::test_method (this=0x7fffffffc8d0)
      at test/validation_chainstatemanager_tests.cpp:262

ACKs for top commit:
  laanwj:
    Code review re-ACK fa340b8794
  jamesob:
    ACK fa340b8794 ([`jamesob/ackr/21584.1.MarcoFalke.fix_assumeutxo_crash_due`](https://github.com/jamesob/bitcoin/tree/ackr/21584.1.MarcoFalke.fix_assumeutxo_crash_due))

Tree-SHA512: c2c4e66c1abfd400ef18a04f22fec1f302f1ff4d27a18050f492f688319deb4ccdd165ff792eee0a1f816e7b69fb64080662b79517ab669e3d26b9eb77802851
2021-05-12 21:00:12 +02:00
W. J. van der Laan
79da18a9eb Merge bitcoin/bitcoin#21911: build: add configure~ to .gitignore
bc4538806e build: add *~ to .gitignore (Sjors Provoost)

Pull request description:

  The file `configure~` recently started appearing for me on macOS (11.3.1) whenever configure is (re)run.

ACKs for top commit:
  hebasto:
    ACK bc4538806e, tested on Linux Mint 20.1 with different build scenarios including cross-compiling for Windows and macOS.

Tree-SHA512: 830c7baf392ff6d66250a79c6ed0a98dac3daaace54a6d2e7940b9a72e3bac79ab44bbecd7642c931fde8a446654e2260d6afdecc679a1743fae6ec5eeda79f1
2021-05-12 19:45:10 +02:00
MarcoFalke
fa397a6a9c ci: Bump cirrus fuzz CPUs to avoid timeout 2021-05-12 18:26:37 +02:00
Sjors Provoost
bc4538806e build: add *~ to .gitignore
Homebrew autoconf version 2.7.1 introduces configure~ as a build artifact.

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-05-12 18:10:47 +02:00
W. J. van der Laan
6b49d88a5d Merge bitcoin/bitcoin#21905: net: initialize nMessageSize to uint32_t max
9c891b64ff net: initialize nMessageSize to max uint32_t instead of -1 (eugene)

Pull request description:

  nMessageSize is uint32_t and is set to -1. This will warn with `-fsanitize=implicit-integer-sign-change` when V1TransportDeserializer calls into the ctor.  This pull initializes nMessageSize to `numeric_limits<uint32_t>::max()` instead and removes the ubsan suppression.

ACKs for top commit:
  laanwj:
    Code review ACK 9c891b64ff
  promag:
    Code review ACK 9c891b64ff.

Tree-SHA512: f05173d9553a01d207a5a7f8ff113d9e11354c50b494a67d44d3931c151581599a9da4e28f40edd113f4698ea9115e6092b2a5b7329c841426726772076c1493
2021-05-12 17:39:39 +02:00
Jon Atack
7075f604e8 scripted-diff: update noban documentation in net_processing.cpp
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src/net_processing.cpp | xargs sed -i "s/$1/$2/g"; }
s 'the noban permission'      'NetPermissionFlags::NoBan permission'
s 'the NOBAN permission flag' 'NetPermissionFlags::NoBan permission'
s 'noban permission'          'NetPermissionFlags::NoBan permission'
-END VERIFY SCRIPT-
2021-05-12 16:13:32 +02:00
Jon Atack
a95540cf43 scripted-diff: rename NetPermissionFlags enumerators
- drop redundant PF_ permission flags prefixes
- drop ALL_CAPS naming per https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Renum-caps
- rename IsImplicit to Implicit

-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; }

s 'PF_NONE'        'None'
s 'PF_BLOOMFILTER' 'BloomFilter'
s 'PF_RELAY'       'Relay'
s 'PF_FORCERELAY'  'ForceRelay'
s 'PF_DOWNLOAD'    'Download'
s 'PF_NOBAN'       'NoBan'
s 'PF_MEMPOOL'     'Mempool'
s 'PF_ADDR'        'Addr'
s 'PF_ISIMPLICIT'  'Implicit'
s 'PF_ALL'         'All'
-END VERIFY SCRIPT-
2021-05-12 16:13:30 +02:00
Jon Atack
810d0929c1 p2p, refactor: make NetPermissionFlags a uint32 enum class
and define/update operation methods to handle type conversions explicitly. See
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Renum-oper
for more info.
2021-05-12 16:13:23 +02:00
W. J. van der Laan
d2ec37221a Merge bitcoin/bitcoin#21796: index: Avoid async shutdown on init error
faad68fcd4 index: Avoid async shutdown on init error (MarcoFalke)

Pull request description:

  An async shutdown during init is confusing when a simple boolean return value can be used for a synchronous shutdown.

  This also changes the error message on stderr from:

  ```
  Error: A fatal internal error occurred, see debug.log for details
  Error: A fatal internal error occurred, see debug.log for details
  ```

  To:

  ```
  Error: basic block filter index best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)

ACKs for top commit:
  laanwj:
    Code review ACK faad68fcd4

Tree-SHA512: 92dd895266d6d15a6b1a5c081c9b83f83d5c82e9bfceb3ea0664f48540812239e274c829ff0271c4a0afb6d6a8f67d89c5af20d719982ad62999a41ca0623274
2021-05-12 14:58:47 +02:00
MarcoFalke
91db985bc9 Merge bitcoin/bitcoin#21927: fuzz: Run const CScript member functions only once
fa74bfc860 fuzz: Run const CScript member functions only once (MarcoFalke)

Pull request description:

  Those functions should be O(N) in the input size (or maybe worse, I didn't check), so if the fuzz input dictates to run them N times, the complexity is N^2.

  Fix this by calling them only once.

  Can be reviewed with: `--ignore-all-space  --word-diff-regex=.`

  Input: https://github.com/bitcoin/bitcoin/files/6464685/clusterfuzz-testcase-minimized-input.log

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34101

ACKs for top commit:
  practicalswift:
    cr ACK fa74bfc860: patch looks correct, rationale makes sense and patch touches only `src/test/fuzz/`

Tree-SHA512: d579f7a2103ec154bf482a872142e55a1d9e7673d33a22a4c4230186fdd1b6618846463f4e25941031cc8c4bd1ea8d06cb49ae1bb1ec4af115497f5e5de1e19c
2021-05-12 14:48:25 +02:00
MarcoFalke
fae814c9a6 fuzz: Remove incorrect float round-trip serialization test 2021-05-12 14:42:41 +02:00
W. J. van der Laan
176842daa8 Merge bitcoin/bitcoin#21912: doc: Remove mention of priority estimation
fa0ad7b9fe doc: Remove mention of priority estimation (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    Documentation review ACK fa0ad7b9fe

Tree-SHA512: 1be856efc0a25c6bec31e6e58879bbccce18f69cc4f180b285a24362b032f1abeaabc55f9bb064c4c30d3217c38b3f96f52bdf80e13c6069c86cdc4d21f57ef3
2021-05-12 14:15:40 +02:00
W. J. van der Laan
01624a7db3 Merge bitcoin/bitcoin#21462: guix: Add guix-{attest,verify} scripts
d420e5c1c0 guix-attest: Avoid incomplete sigdirs with ERR traps (Carl Dong)
feda2c8e31 guix: Skip attesting to dist-archive (Carl Dong)
d522d8006b guix: Attest to inputs in inputs.SHA256SUMS (Carl Dong)
f9e2960c01 guix: Construct $OUTDIR in ${DISTSRC}/output (Carl Dong)
022abc85fc guix: Minor quoting fix in libexec/build.sh (Carl Dong)
c83c4fa5b7 guix-attest: Allow skipping GPG signing with NO_SIGN (Carl Dong)
0e1c2e448c guix-attest: Use ascii-armor signatures (Carl Dong)
b5fd89c4c8 guix-attest: Only use cross-platform flags for find+xargs (Carl Dong)
5926432ba6 guix: Add guix-verify script (Carl Dong)
30daf76a97 guix: Add guix-attest script (Carl Dong)

Pull request description:

  Adds replacements for `gsign` and `gverify`.

  Personally I'm not a big fan of using the word "sign" as it's been used to refer to both codesigning and GPG signing.

ACKs for top commit:
  laanwj:
    Code review and tested ACK d420e5c1c0

Tree-SHA512: 93d82d201f4596eaea0e3825aa55b013dfb91790e6ccee79893833d37921513d7b4e735f0641103e1e2ea8308abe4cb6218b73160924708802f2e0e3f7f6caf1
2021-05-12 13:51:38 +02:00
Hennadii Stepanov
29c9e2c2d2 wallet: Do not iterate a directory if having an error while accessing it
This change prevents infinite looping for, for example, system folders
on Windows.
2021-05-12 12:11:47 +03:00
Jon Atack
7b55a94497 p2p: NetPermissions::HasFlag() pass flags param by value 2021-05-12 11:04:34 +02:00
Jon Atack
91f6e6e6d1 scripted-diff: add NetPermissionFlags scopes where not already present
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" -- 'src' ':!src/net_permissions.h' | xargs sed -i -E "s/([^:])$1/\1NetPermissionFlags::$1/"; }

s 'PF_NONE'
s 'PF_BLOOMFILTER'
s 'PF_RELAY'
s 'PF_FORCERELAY'
s 'PF_DOWNLOAD'
s 'PF_NOBAN'
s 'PF_MEMPOOL'
s 'PF_ADDR'
s 'PF_ISIMPLICIT'
s 'PF_ALL'
-END VERIFY SCRIPT-

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-05-12 10:50:58 +02:00
MarcoFalke
faad68fcd4 index: Avoid async shutdown on init error 2021-05-12 10:47:35 +02:00
MarcoFalke
fa74bfc860 fuzz: Run const CScript member functions only once 2021-05-12 10:20:59 +02:00
MarcoFalke
faf30f2ae0 doc: Update bips.md for 0.21.1 2021-05-12 10:06:37 +02:00
MarcoFalke
2e30e328a7 Merge bitcoin/bitcoin#19064: refactor: Cleanup thread ctor calls
792be53d3e refactor: Replace std::bind with lambdas (Hennadii Stepanov)
a508f718f3 refactor: Use appropriate thread constructor (Hennadii Stepanov)
30e4448215 refactor: Make TraceThread a non-template free function (Hennadii Stepanov)

Pull request description:

  This PR does not change behavior.
  Its goal is to improve readability and maintainability of the code.

ACKs for top commit:
  jnewbery:
    utACK 792be53d3e
  jonatack:
    tACK 792be53d3e
  MarcoFalke:
    cr ACK 792be53d3e

Tree-SHA512: a03142f04f370f6bc02bd3ddfa870819b51740fcd028772241d68c84087f95a2d78207cbd5edb3f7c636fcf2d76192d9c59873f8f0af451d3b05c0cf9cf234df
2021-05-12 08:51:32 +02:00
fanquake
03e16cb027 Merge bitcoin/bitcoin#21593: build, qt, refactor: Get rid of some sed command instances
b95f7f8ac0 build, qt, refactor: Drop sed commands for win32-g++/qmake.conf (Hennadii Stepanov)

Pull request description:

  Such possibility is [available](https://codereview.qt-project.org/c/qt/qtbase/+/165348) since Qt 5.8.0.

ACKs for top commit:
  fanquake:
    ACK b95f7f8ac0

Tree-SHA512: e56a3d208a6bd5d42c722f8b344010fe7d1b6f7a28486613dfcb03f0403a47cee8476e2366eeaac401a19836cd09f782e8741a1e781ab4d78f72c500a30e4929
2021-05-12 14:24:47 +10:00
fanquake
6754e1928b Merge bitcoin/bitcoin#21922: fuzz: Avoid timeout in EncodeBase58
faa0d94a7d fuzz: Avoid timeout in EncodeBase58 (MarcoFalke)

Pull request description:

  The complexity is O(N^2), so limit the size.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34126

  Oss-Fuzz testcase for `rpc` fuzzer: https://github.com/bitcoin/bitcoin/files/6461382/clusterfuzz-testcase-minimized-rpc-4831734974775296.log

ACKs for top commit:
  practicalswift:
    cr ACK faa0d94a7d: patch looks correct
  sipa:
    utACK faa0d94a7d

Tree-SHA512: 57ad9de8d811b828982d09a586782fc8a62fa3685590301d58120e2249caa30a9dccd3abe0b47e00ea8482de705fe0edbed298ab8761ea0d29496b50ed2db5d7
2021-05-12 11:03:04 +10:00
MarcoFalke
faa0d94a7d fuzz: Avoid timeout in EncodeBase58 2021-05-11 21:24:49 +02:00
MarcoFalke
f0a76b3dbc Merge bitcoin/bitcoin#21892: fuzz: Avoid excessively large min fee rate in tx_pool
99993f0664 fuzz: Avoid excessively large min fee rate in tx_pool (MarcoFalke)

Pull request description:

  Any fee rate above 1 BTC / kvB is clearly nonsense, so no need to fuzz this.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34078

ACKs for top commit:
  practicalswift:
    cr ACK 99993f0664: patch looks correct despite no `fa` prefix in commit hash

Tree-SHA512: bd3651d354b13d889ad1708d2b385ad0479de036de74a237346eefad5dbfb1df76ec02b55ec00487ec598657ef6102f992302b14c4e47f913a9962f81f4157e6
2021-05-11 20:35:20 +02:00
MarcoFalke
88dc09d759 Merge bitcoin/bitcoin#21909: fuzz: Limit max insertions in timedata fuzz test
fa95555a49 fuzz: Limit max insertions in timedata fuzz test (MarcoFalke)

Pull request description:

  It is debatable whether a size of the median filter other than `200` (the only size used in production) should be fuzzed. For now add a minimal patch to cap the max insertions. Otherwise the complexity is N^2 log(N), where N is the size of the fuzz input.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34167

ACKs for top commit:
  practicalswift:
    cr ACK fa95555a49: patch looks correct

Tree-SHA512: be7737e9f4c906053e355641de84dde31fed37ed6be4c5e92e602ca7675dffdaf06b7063b9235ef541b05d3d5fd689c99479317473bb15cb5271b8baabffd0f2
2021-05-11 20:32:20 +02:00
MarcoFalke
fa25ce45e9 build: improve macro for testing -latomic requirement 2021-05-11 20:07:20 +02:00
Karl-Johan Alm
6e2eb0d63b rpc/wallet: use OMITTED_NAMED_ARG instead of Default(VNULL) 2021-05-12 00:49:49 +09:00
eugene
9c891b64ff net: initialize nMessageSize to max uint32_t instead of -1
nMessageSize is uint32_t and is set to -1. This will warn with
-fsanitize=implicit-integer-sign-change.
2021-05-11 11:21:29 -04:00
Karl-Johan Alm
4983f4cba4 rpc/createwallet: omitted named arguments
The current output for passphrase results in a non-optional passphrase which is incorrect.
2021-05-11 21:04:24 +09:00
W. J. van der Laan
e175a20769 Merge bitcoin/bitcoin#21644: p2p, bugfix: use NetPermissions::HasFlag() in CConnman::Bind()
36fb036d25 p2p: allow NetPermissions::ClearFlag() only with PF_ISIMPLICIT (Jon Atack)
4e0d5788ba test: add net permissions noban/download unit test coverage (Jon Atack)
dde69f20a0 p2p, bugfix: use NetPermissions::HasFlag() in CConnman::Bind() (Jon Atack)

Pull request description:

  This is a bugfix follow-up to #16248 and #19191 that was noticed in #21506. Both v0.21 and master are affected.

  Since #19191, noban is a multi-flag that implies download, so the conditional in `CConnman::Bind()` using a bitwise AND on noban will return the same result for both the noban status and the download status. This means that download peers are incorrectly not being added to local addresses because they are mistakenly seen as noban peers.

  The second commit adds unit test coverage to illustrate and test the noban/download relationship and the `NetPermissions` operations involving them.

  The final commit adds documentation and disallows calling `NetPermissions::ClearFlag()` with any second param other than `NetPermissionFlags` "implicit" -- per current usage in the codebase -- because `ClearFlag()` should not be called with any second param that is a subflag of a multiflag, e.g. "relay" or "download," as that would leave the result in an invalid state corresponding to none of the existing NetPermissionFlags. Thanks to Vasil Dimov for noticing this.

ACKs for top commit:
  theStack:
    re-ACK 36fb036d25 
  vasild:
    ACK 36fb036d25
  hebasto:
    ACK 36fb036d25, I have reviewed the code and it looks OK, I agree it can be merged.
  kallewoof:
    Code review ACK 36fb036d25

Tree-SHA512: 5fbc7ddbf31d06b35bf238f4d77ef311e6b6ef2e1bb9893f32f889c1a0f65774a3710dcb21d94317fe6166df9334a9f2d42630809e7fe8cbd797dd6f6fc49491
2021-05-11 13:08:45 +02:00
Vasil Dimov
105941b726 net: use stronger AddLocal() for our I2P address
There are two issues:

1. Our I2P address not added to local addresses.

* `externalip=` is used with an IPv4 address (this sets automatically
  `discover=0`)
* No `discover=1` is used
* `i2psam=` is used
* No `externalip=` is used for our I2P address
* `listenonion=1 torcontrol=` are used

In this case `AddLocal(LOCAL_MANUAL)` is used for our `.onion` address
and `AddLocal(LOCAL_BIND)` for our `.b32.i2p` address, the latter being
ignored due to `discover=0`.

2. Our I2P address removed from local addresses even if specified
with `externalip=` on I2P proxy restart.

* `externalip=` is used with our I2P address (this sets automatically
  `discover=0`)
* No `discover=1` is used
* `i2psam=` is used

In this case, initially `externalip=` causes our I2P address to be added
with `AddLocal(LOCAL_MANUAL)` which overrides `discover=0` and works as
expected. However, if later the I2P proxy is shut down we do
`RemoveLocal()` in order to stop advertising our I2P address (since we
have lost I2P connectivity). When the I2P proxy is started and we
reconnect to it, restoring the I2P connectivity, we do
`AddLocal(LOCAL_BIND)` which does nothing due to `discover=0`.

To resolve those two issues, use `AddLocal(LOCAL_MANUAL)` for I2P which
is also what we do with Tor.
2021-05-11 12:46:45 +02:00
Karl-Johan Alm
dc4db23b30 rpc: address:amount dictionaries are OBJ_USER_KEYS 2021-05-11 19:07:08 +09:00
Karl-Johan Alm
c8cf0a3d51 rpc/getpeerinfo: bytesrecv_per_msg is a dynamic dictionary
It is not a dictionary with the single key 'msg'.
2021-05-11 19:00:52 +09:00
Karl-Johan Alm
eb4fb7e507 rpc/gettxoutsetinfo: hash_or_height is a named argument 2021-05-11 18:58:42 +09:00
MarcoFalke
94f83534e4 Merge bitcoin/bitcoin#21752: doc: Clarify that feerates are per virtual size
fae196147b doc: Clarify that feerates are per virtual size (MarcoFalke)
fa83e95ac6 scripted-diff: Clarify that feerates are per virtual size (MarcoFalke)

Pull request description:

  By implementing segwit, it is already clear that all feerates in Bitcoin Core are denoted in (amount/virtual size). Though, there is  inconsistency, as some places use kvB, some use kB. Thus, replace all with "kvB".

  See also commit 6da3afbaee, which did the replacement for wallet RPCs.

ACKs for top commit:
  ryanofsky:
    Code review ACK fae196147b. Checked instances where units were being added in the second commit and they all looked right.

Tree-SHA512: ab70d13cde7d55c1ac931bddc2b45aa218fc75ef46cb6ea9e5a30b1d4dbf27889c2b6357299a6c5427912443a46ec3592a4809dae335e03162bd2120a0f7f8ad
2021-05-11 11:55:20 +02:00
MarcoFalke
fa0ad7b9fe doc: Remove mention of priority estimation
Follow up to commit b2322e0fc6
2021-05-11 11:52:32 +02:00
MarcoFalke
fa340b8794 refactor: Avoid magic value of all-zeros in assumeutxo base_blockhash
Just use std::optional
2021-05-11 11:21:05 +02:00
W. J. van der Laan
39e306009b Merge bitcoin-core/gui#271: Don't clear console prompt when font resizing
7962e0dde8 qt: Do not clear console prompt when font resizing (Hennadii Stepanov)
d2cc339005 qt, refactor: Drop redundant history cleaning in RPC console (Hennadii Stepanov)
4f0ae472e2 qt: Untie irrelevant signal-slot parameters (Hennadii Stepanov)

Pull request description:

  On master, a console resize event will clear the prompt. To fix this, we store the content of the prompt and re-set it upon a resize. This preserves the prompt text throughout resizes. The text will still clear when you click the clear button, as it should.

  **Master**

  | Before Resize      | After Resize |
  | ----------------- | ------------ |
  |  ![master-beforeresize](https://user-images.githubusercontent.com/23396902/113553721-2a428d80-95c6-11eb-971b-bb77151bc6d5.png)  |  ![master-afterresize](https://user-images.githubusercontent.com/23396902/113553769-3d555d80-95c6-11eb-9cdb-9ad1fd7208a9.png) |

  **PR**
  | Before Resize      | After Resize |
  | ----------------- | ------------ |
  | ![pr-beforeresize](https://user-images.githubusercontent.com/23396902/113553885-6f66bf80-95c6-11eb-8317-0975f1ebd444.png) | ![pr-afterresize](https://user-images.githubusercontent.com/23396902/113553906-75f53700-95c6-11eb-9a32-b64d8aba98e5.png) |

  Closes #269

ACKs for top commit:
  laanwj:
    Code review ACK 7962e0dde8
  hebasto:
    ACK 7962e0dde8
  Talkless:
    tACK 7962e0dde8, tested on Debian Sid with Qt 5.15.2

Tree-SHA512: a6f19d3f80e2e47725cff5d6e15862b6cb793a65dfcaded15f23bba051088cd3317f068f93290c9b09d0a90f5fcac1c5a4610cc417cc5961ba6d005fe5049ab0
2021-05-11 11:06:47 +02:00
MarcoFalke
fae33f98e6 Fix assumeutxo crash due to invalid base_blockhash
Can be reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-05-11 10:41:03 +02:00
MarcoFalke
fa5668bfb3 refactor: Use type-safe assumeutxo hash
This avoids accidentally mixing it up with other hashes (like block
hashes).
2021-05-11 10:40:40 +02:00
MarcoFalke
0000007709 refactor: Remove unused code 2021-05-11 10:39:44 +02:00
MarcoFalke
faa921f787 move-only: Add util/hash_type
Can be reviewed with --color-moved=dimmed-zebra
2021-05-11 10:38:18 +02:00
t-bast
c30dd02cd8 refactor: remove redundant fOnlySafe argument
The fOnlySafe argument to AvailableCoins is now redundant, since #21359
added a similar field inside the CCoinControl struct.

Not all code paths set a CCoinControl instance, but when it's missing we
can default to using only safe inputs which is backwards-compatible.
2021-05-11 09:58:13 +02:00
MarcoFalke
fa95555a49 fuzz: Limit max insertions in timedata fuzz test 2021-05-11 08:54:24 +02:00
MarcoFalke
d2f6d2976f Merge bitcoin/bitcoin#21895: refactor: Add TSA annotations to the WorkQueue class members
34b04eec44 refactor: Add TSA annotations to the WorkQueue class members (Hennadii Stepanov)

Pull request description:

  Noted while reviewing #19033, and hoping this will not conflict with it :)

ACKs for top commit:
  promag:
    Code review ACK 34b04eec44.

Tree-SHA512: 4c15729acd95223263c19bc0dd64b9e7960872b48edee6eee97a5d0c2b99b8838185ac3a2ccd5bee992cb3a12498633427fe9919be5a12da9949fcf69a6275a0
2021-05-11 07:00:34 +02:00
fanquake
4a26705761 Merge bitcoin/bitcoin#21902: refactor: Remove useless extern keyword
fa4bbd306e refactor: Remove useless extern keyword (MarcoFalke)

Pull request description:

  It is redundant, confusing and useless.

  https://en.cppreference.com/w/cpp/language/storage_duration#external_linkage

ACKs for top commit:
  practicalswift:
    cr ACK fa4bbd306e: patch looks correct
  Talkless:
    utACK fa4bbd306e, built successfully on Debian Sid, looks OK.
  jonatack:
    Light code review ACK fa4bbd306e
  hebasto:
    ACK fa4bbd306e, I've verified that all of the remained `extern` keywords specify either (a) a variable with external linkage, or (b) a symbol with "C" language linkage.
  promag:
    Code review ACK fa4bbd306e.

Tree-SHA512: 1d77d661132defa52ccb2046f7a287deb3669b68835e40ab75a0d9d08fe6efeaf3bea7c0e76c754fd18bfe45972c253a39462014080d014cc5d810498784e3e4
2021-05-11 08:52:37 +10:00
Hennadii Stepanov
4bc3b16349 Merge bitcoin-core/gui#293: Enable wordWrap for Services
a0f7978674 qt: enable wordWrap for peers-tab detail services (randymcmillan)

Pull request description:

  Enable wordWrap for peers-tab detailView Services

ACKs for top commit:
  Talkless:
    tACK a0f7978674 on same environment as previously.
  hebasto:
    ACK a0f7978674, tested on Linux Mint 20.1 (Qt 5.12.8):
  kristapsk:
    re-ACK a0f7978674. Tested under Gentoo Linux with Xfce4 (Qt 5.15.2).

Tree-SHA512: 872e511d2ecfa72fea0fd3284a958b45ee8aee138469ce7f9cd853cd9098b9583917909934b0a5c96f9b81ea1567bcea6a037558829bb79f2a3f413a83df06e6
2021-05-11 00:45:15 +03:00
Hennadii Stepanov
b49fe0a75a Merge bitcoin-core/gui#280: Remove user input from URI error message
3bad0b3fad Remove user input from URI error message (unknown)

Pull request description:

  Removes the user input from error message to avoid it being used in attacks.

  Its not really a vulnerability in Bitcoin Core because involves social engineering, dependency on user environment etc. But this PR improves security and by avoiding abuse of URI error in future.

  Example of an attack:

  1. User opens a link in firefox:

  ```
  bitcoin:tb1qag2e6yhl52hr53vdxzaxvnjtueupvuftan4yfu%0A%0AWARNING%3A%20DO%20NOT%20CLOSE%20THIS%20WINDOW%20OR%20TURN%20OFF%20YOUR%20PC!%20IF%20YOU%20ABORT%20THIS%20PROCESS%2C%20YOU%20COULD%20DESTROY%20ALL%20OF%20YOU%20DATA!%20PLEASE%20ENSURE%20THAT%20YOUR%20POWER%20CABLE%20IS%20PLUGGED%20IN!%0A%0AYou%20became%20victim%20of%20the%20XYZ%20RANSOMWARE!%0A%0AThe%20hard%20disks%20of%20your%20computer%20have%20been%20encrypted%20with%20a%20military%20grade%20encryption%20algorithm.%20There%20is%20no%20way%20to%20restore%20your%20data%20without%20a%20special%20key.%20You%20can%20purchase%20this%20key%20on%20the%20darknet%20page%20shown%20in%20step%202.%0ATo%20purchase%20your%20key%20and%20restore%20your%20data%2C%20please%20follow%20these%20three%20easy%20steps%3A%0A%0A1.%20Download%20the%20Tor%20browser%20at%20%E2%80%9Chttps%3A%2F%2Fwww.torproject.org%2F%E2%80%9C.%0A2.%20Visit%20one%20of%20the%20following%20pages%20with%20the%20Tor%20Browser%3A%0Ahttp%3A%2F%2Frandomchars.onion%2Fabc123%0A3.%20Send%20BTC%20by%20following%20the%20instructions%20on%20the%20page
  ```

  2. User selects Bitcoin Core to open the link:

  ![image](https://user-images.githubusercontent.com/13405205/114619801-8ee9a080-9cc8-11eb-9fad-23a2b831e8df.png)

  3. User is asked to send BTC with some message convincing enough which can be different depending on the victim:

  ![image](https://user-images.githubusercontent.com/13405205/114620061-d3753c00-9cc8-11eb-8314-e3362ebb90ac.png)

  **After this PR** (_No user input mentioned in the error_):

  ![image](https://user-images.githubusercontent.com/13405205/114624342-2b627180-9cce-11eb-93a8-0b2438d71571.png)

ACKs for top commit:
  hebasto:
    ACK 3bad0b3fad, tested on Linux Mint 20.1 (Qt 5.12.8).
  jarolrod:
    tACK 3bad0b3fad

Tree-SHA512: aac2fdfcaa7a9cd6582750c1960682554795640f5aacb78bdae121724e1151da3cbb62b8f8b1e0bc37347afe78b3e9a446277cab8e009d2a1050c0e971f001b3
2021-05-11 00:27:51 +03:00
Hennadii Stepanov
a2bdbdb358 Merge bitcoin-core/gui#194: Save/restore RPCConsole geometry only for window
01d9586ae8 qt: Save/restore RPCConsole geometry only for window (Hennadii Stepanov)

Pull request description:

  After using the GUI with `-disablewallet` the "Node window" inherits the geometry of the main window, that could be unexpected for users.

  This PR provides independent geometry settings for `RPCConsole` in both modes:
  - window sizes and `QSplitter` sizes when `-disablewallet=0`
  - only `QSplitter` sizes when `-disablewallet=1`

ACKs for top commit:
  Talkless:
    tACK 01d9586ae8, tested on Debian Sid with Qt 5.15.2. I've managed to reproduce issue using https://github.com/bitcoin-core/gui/pull/194#issuecomment-782822663 instructions, and I see that this PR does detach main window and information window sizes. Built with `--enable-wallet` and `--disable-wallet`.
  jarolrod:
    ACK 01d9586ae8, tested on macOS 11.2 Qt 5.15.2
  promag:
    Code review ACK 01d9586ae8.

Tree-SHA512: 9934cf04d4d5070dfc4671ea950e225cda9988858227e5481dad1baafa14af477bdbf4f91307ca687fde0cad6e4e605a3a99377e70d67eb115a19955ce2516f5
2021-05-10 23:56:14 +03:00
Hennadii Stepanov
8d7125f80e Merge bitcoin-core/gui#257: refactor: Use template function qOverload in signal-slot connections
cdbc2bd1f1 qt: Use template function qOverload in signal-slot connections (Hennadii Stepanov)

Pull request description:

  A nice template function [`qOverload`](https://doc.qt.io/qt-5/qtglobal.html#qOverload) is available for us now (https://github.com/bitcoin/bitcoin/pull/20413, https://github.com/bitcoin/bitcoin/pull/21286).

  Its usage makes code much more readable.

  This PR does not change behavior.

ACKs for top commit:
  Talkless:
    utACK cdbc2bd1f1.
  promag:
    Code review ACK cdbc2bd1f1.

Tree-SHA512: 72002aa646b1a79bab62d498825b3f245dc7ebdc189280f8bd3b4076e1bb50be8802c02bc872ff6f70c1ea81faec66d3bec36471119dd98c9e70d87b990396ae
2021-05-10 23:42:05 +03:00
MarcoFalke
d8ae29ec8f Merge bitcoin/bitcoin#21900: test: use MiniWallet for feature_csv_activation.py
bd7f27d16d refactor: feature_csv_activation.py: move tx helper functions to methods (Sebastian Falbesoner)
2eca46b0aa test: use MiniWallet for feature_csv_activation.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (feature_csv_activation.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078.

  Short reviewers guideline:
  - Since we exclusively work with anyone-can-spend outputs here (raw scriptPubKey = OP_TRUE), signing is not needed anymore. The function `sign_transaction` and its calls are removed, after changing a tx (e.g. its scriptSig or nVersion) a simple `.rehash()` call is sufficient. Also, generating an address `self.nodeaddress` (and with that, passing it to the the various test tx creation/sending helper methods) is not needed anymore and removed.
  - The test repeatedly uses the same input for creating different txs (e.g. with different txversions 1 and 2). To let `MiniWallet` create a tx with a specific input, we have to call `.get_utxo()` before which also marks the UTXO as spent. The method is changed to also support keeping the UTXO in its internal list (`mark_as_spent=False`). With the behaviour on master, the second call to `.get_utxo()` with the same input would fail.
  - To keep the diff in the first commit short, the `miniwallet` is set as a global variable, to avoid passing it on every tx creation/spending helper. The global is eliminated in the second (refactoring) commit, where all the helpers are moved to the test class as methods. By that, we can use `self.nodes[0]` directly in the helpers and don't have to pass it again and again. I think there could still be a lot of improvements/refactoring done in the test, but that should hopefully serve as a good basis.

ACKs for top commit:
  laanwj:
    Code review ACK bd7f27d16d
  MarcoFalke:
    review ACK bd7f27d16d 🐕

Tree-SHA512: 24fb6a0f7702bae40d5271d197119827067d4b597e954d182e4c1aa5d0fa870368eb3ffed469b26713fa8ff8eb3ecc06abc80b2449cd68156d5559e7ae8a2b11
2021-05-10 17:50:21 +02:00
Hennadii Stepanov
a58868d201 build: Makes rcc output always deterministic
The Qt Resource Compiler (rcc) has a command-line option
`--format-version` which has the default value 2.

The only difference from `--format-version 1` is adding a last modified
timestamp to the output file. That, in turn, forces us to use
`QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds.

This change makes rcc output always deterministic by using
`--format-version 1` option that makes usage of the
`QT_RCC_SOURCE_DATE_OVERRIDE` needless. Also it improves interaction
with ccache.

Co-authored-by: fanquake <fanquake@gmail.com>
2021-05-10 18:31:49 +03:00
W. J. van der Laan
32692d2681 Merge bitcoin/bitcoin#21359: rpc: include_unsafe option for fundrawtransaction
11d6459b6e rpc: include_unsafe option for fundrawtransaction (t-bast)

Pull request description:

  Allow RPC users to opt-in to unsafe inputs when funding a raw transaction.

  Applications that need to manage a complex RBF flow (such as lightning nodes using anchor outputs) are very limited if they can only use safe inputs.

  I also added this option to `send` and `walletcreatefundedpsbt` who internally delegate to `fundrawtransaction`.

  Fixes #21299

ACKs for top commit:
  laanwj:
    Code review ACK 11d6459b6e

Tree-SHA512: 5e542a4febcfd6f41cf784678ff02ec9282eae2082c274983f72c5ea87b7ebbe1bd5fdc6a020d7a9d5996157754eb4966b8aeb6c1ceebf0b1519f735579b8bac
2021-05-10 16:05:23 +02:00
Hennadii Stepanov
8bed1706ea Merge bitcoin-core/gui#296: Do not use QObject::tr plural syntax for numbers with a unit symbol
3adde72bc9 qt: Do not use QObject::tr plural syntax for numbers with a unit symbol (Hennadii Stepanov)

Pull request description:

  Working on translation, I found this is useless and unnecessarily burdensome for translators. I guess, this statement is correct internationally wide :)

ACKs for top commit:
  jarolrod:
    ACK 3adde72bc9
  promag:
    Code review ACK 3adde72bc9. Agree with OP, looks reasonable to me.

Tree-SHA512: bde65c122ca0feb7771d932cce63fd1aef1e7a9dda0188d19c577d57b279172204ac1bfcb6106a78b2c4d55d628e6dc0967051e064ec40d3c5aeafd4a48f0589
2021-05-10 16:34:12 +03:00
W. J. van der Laan
c9b051b58f Merge bitcoin/bitcoin#21891: fuzz: Remove strprintf test cases that are known to fail
facfc0f65d fuzz: Remove strprintf test cases that are known to fail (MarcoFalke)

Pull request description:

  They are still waiting to be fixed (see https://github.com/c42f/tinyformat/issues/70 ), so no need for us to carry them around in our source code. They can be added back once upstream is fixed.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34082

ACKs for top commit:
  laanwj:
    Code review ACK facfc0f65d

Tree-SHA512: d9d3d35555b6d58740a041ae45797ca85149f60990e2ed632c5dadf363e1d2362d2447681d7ceaa1fbffcd6e7bc8da5bc15d3923b68829a86c25b364a599afc8
2021-05-10 15:23:01 +02:00
W. J. van der Laan
c49d246647 Merge bitcoin/bitcoin#21745: refactor: Add missing includes in pubkey.cpp/pubkey.h
71c824ed6c cleaned up and added missing "include" statements for pubkey.cpp and pubkey.h (William Bright)

Pull request description:

  #### Problem:
  Many symbols in the files were undefined and causing issues when I was working on building independent sections of the codebase. The hidden imports from the "secp256k1" library was a particular pain point.

  The other standard and missing includes are following best practices and will help with refactoring, build process and others.

  #### Changes:
  Clean up and declared imports/include for `pubkey.cpp` and `pubkey.h`

ACKs for top commit:
  jnewbery:
    utACK 71c824ed6c
  laanwj:
    Code review ACK 71c824ed6c

Tree-SHA512: bce605cfde24d8e3be82a596cabab7a8577fec0aef7c5e6f7a56603357046d8e8dea11ac8e3dbe79600550291be7784e35c7a55ebf40b46525b8949e4bedae96
2021-05-10 14:35:31 +02:00
W. J. van der Laan
f8176b768a Merge bitcoin/bitcoin#21836: scripted-diff: Replace three dots with ellipsis in the UI strings
d66f283ac0 scripted-diff: Replace three dots with ellipsis in the UI strings (Hennadii Stepanov)

Pull request description:

  This PR is split from #21463.
  The change was suggested on [Transifex.com](https://www.transifex.com/bitcoin/bitcoin/), and it does not touch `LogPrint` and `LogPrintf` calls.

  The only comment on #21463 [was](9030e4b5a6 (r597220100)):
  > Mind that these messages also end up in the log. In principle the log is already UTF-8 (as are all strings and text in bitcoind). But, just noting, that it might make browsing the log a less pleasant experience on systems with misconfigured locale like some BSDs by default.

ACKs for top commit:
  laanwj:
    ACK d66f283ac0

Tree-SHA512: 5ab1cb3160f3f996f1ad7d7486662da3eb7f06a857f4a1874963ce10caed5b86b0ad6151b1b9ebeb2b8aa5f0c85efad3b768ea9cafe5db86f78f88912b756d1e
2021-05-10 14:02:46 +02:00
W. J. van der Laan
1a60c547fd Merge bitcoin/bitcoin#21749: test: Bump shellcheck version
08f3dbb1b0 test: Bump shellcheck version (Hennadii Stepanov)

Pull request description:

  The changelog for v0.7.2 is available [here](https://github.com/koalaman/shellcheck/blob/v0.7.2/CHANGELOG.md).

  Only [SC2268](https://github.com/koalaman/shellcheck/wiki/SC2268) requires to update our code.

ACKs for top commit:
  jarolrod:
    ACK  08f3dbb1b0

Tree-SHA512: 4585cd1f4d9def2fbaafe5a2a57761288d432781eb8c6c6d37064727d7ca8fc3f35c552e6a2ffdf0820d753d4bde2c8e43e5f3f57d242f5f57591a9b1b03558d
2021-05-10 13:49:50 +02:00
W. J. van der Laan
1591e35049 Merge bitcoin/bitcoin#21708: build: Drop pointless sed commands
f52fafc935 build: Drop pointless sed commands (Hennadii Stepanov)

Pull request description:

  Since moving to Autotools build system (35b8af9226, #2943, 2013-09), tag strings created by Qt specialized compilers ([uic](https://doc.qt.io/qt-5/uic.html), [moc](https://doc.qt.io/qt-5/moc.html), [rcc](https://doc.qt.io/qt-5/rcc.html)) were being removed.

  A bit later (70c71c50ce, #4241, 2014-06) this rule was dropped for the uic, and since then all of the generated `ui_*.h` files contain the following string:
  ```
  ** Created by: Qt User Interface Compiler version 5.12.8
  ```

  Such strings do not contain any timestamps, and cannot cause any non-determinism. The removing of them seems pointless.

  Diffs for some files:
  ```diff
  --- master/intro.moc
  +++ pr/intro.moc
  @@ -1,6 +1,7 @@
   /****************************************************************************
   ** Meta object code from reading C++ file 'intro.cpp'
   **
  +** Created by: The Qt Meta Object Compiler version 67 (Qt 5.12.8)
   **
   ** WARNING! All changes made in this file will be lost!
   *****************************************************************************/
  ```
  ```diff
  --- master/moc_addressbookpage.cpp
  +++ pr/moc_addressbookpage.cpp
  @@ -1,6 +1,7 @@
   /****************************************************************************
   ** Meta object code from reading C++ file 'addressbookpage.h'
   **
  +** Created by: The Qt Meta Object Compiler version 67 (Qt 5.12.8)
   **
   ** WARNING! All changes made in this file will be lost!
   *****************************************************************************/
  ```
  ```diff
  --- master/qrc_bitcoin.cpp
  +++ pr/qrc_bitcoin.cpp
  @@ -1,6 +1,7 @@
   /****************************************************************************
   ** Resource object code
   **
  +** Created by: The Resource Compiler for Qt version 5.12.8
   **
   ** WARNING! All changes made in this file will be lost!
   *****************************************************************************/
  ```

ACKs for top commit:
  laanwj:
    ACK f52fafc935

Tree-SHA512: 31f5c19b37645b4914f17d8c234b7ae8781a0499c4b250ffef07d70b7552954fb682f58a75d76162f98ab5e1667288b3a041df2705573fb00523e87b9c1fd47f
2021-05-10 13:46:21 +02:00
MarcoFalke
fa4bbd306e refactor: Remove useless extern keyword 2021-05-10 09:09:24 +02:00
MarcoFalke
adf7843410 Merge bitcoin/bitcoin#21786: wallet: ensure sat/vB feerates are in range (mantissa of 3)
847288df07 test: fee rate values that cannot be represented as sat/vB (Jon Atack)
06a90fa038 rpc: for sat/vB fee rates, limit ParseFixedPoint decimals to 3 (Jon Atack)
0742c7840f rpc: enable passing decimals to AmountFromValue, add doxygen (Jon Atack)
8ce3ef57a3 test: ParseFixedPoint with 3 decimals for sat/vB fee rates (Jon Atack)
b503327597 test: type error and out of range fee rates where missing (Jon Atack)
c5fd4344f7 test: explicit fee rates with invalid amounts (Jon Atack)
ea6f76b66e test: improve zero-value explicit fee rate coverage (Jon Atack)

Pull request description:

  - Improve/close gaps in existing test coverage before making the change
  - Enable passing `decimals` to `ParseFixedPoint()` when calling `AmountFromValue()`
  - Limit explicit fee rates in sat/vB passed in by users to 3 decimals, and raise otherwise
  - Add regression test coverage

  Closes #20534.

ACKs for top commit:
  MarcoFalke:
    review ACK 847288df07 🔷

Tree-SHA512: c539d07ae9b21c0d6c8ea460beb9c8dad5559445518aace560abc3c05c588907bae189b6fd7602b3b397de4a42356136c3ec6f960d3dcf2d5d16377aef4ab5a2
2021-05-10 09:02:48 +02:00
MarcoFalke
2a22d903f3 Merge bitcoin/bitcoin#21897: rpc: adjust incorrect RPCHelpMan types
7031721f2c rpc/listaddressgroupings: redefine inner-most array as ARR_FIXED (Karl-Johan Alm)
8500f7bf54 rpc/createrawtransaction: redefine addresses as OBJ_USER_KEYS (Karl-Johan Alm)
d9e2183c50 rpc: include OBJ_USER_KEY in RPCArg constructor checks (Karl-Johan Alm)

Pull request description:

  This PR adjusts the two issues I encountered while developing a tool that converts RPCHelpMan objects into bindings for other language(s).

  The first is in createrawtransaction, where the address part, e.g. bc1qabc in

  > createrawtransaction '[]' '[{"bc1qabc": 1.0}]'

  is declared as a `Type::OBJ`, when in reality it should be a `Type::OBJ_USER_KEYS`, defined as such:

  5925f1e652/src/rpc/util.h (L126)

  (coincidentally, this is the first and only (afaict) usage of this `RPCArg::Type`).

  The second is in the `listaddressgroupings` RPC, which returns an array of arrays of arrays, where the innermost one is a tuple-thingie with an optional 3rd item; this is an `ARR_FIXED`, not an `ARR`.

ACKs for top commit:
  MarcoFalke:
    ACK 7031721f2c 🐀

Tree-SHA512: 769377416c6226d1738a956fb685498e009f9e7eb2d45bc679b81c5364b9520fdbcb49392c937ab45598aa0d33589e8e6a59ccc101cf8d8e7dfdafd58d4eefd0
2021-05-10 08:44:02 +02:00
MarcoFalke
3028a1e384 Merge bitcoin/bitcoin#21581: streams: Accept URef obj for VectorReader unserialize
fa2204f6ad streams: Accept URef obj for VectorReader unserialize (MarcoFalke)

Pull request description:

  Missed in commit 172f5fa738. An URef may collapse into an LRef or RRef depending on context. There is no reason to forbid RRef in `VectorReader::operator>>`, so add it for consistency.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa2204f6ad, just expanded test since last review

Tree-SHA512: 09ff4e8a918e15b08cebd8c125d37e78bfb3a635c38546fc8454a97a882b2c81c55ef552243617e78744799d31127e6fbf78c4e319c030480b370aab6f38b645
2021-05-10 08:22:18 +02:00
fanquake
86e04741dd Merge bitcoin/bitcoin#21890: fuzz: Limit ParseISO8601DateTime fuzzing to 32-bit
fa1aa6c571 fuzz: Limit ParseISO8601DateTime fuzzing to 32-bit (MarcoFalke)

Pull request description:

  2038 is more than 10 years in the future, so no need for us to waste time fuzzing a 3rd party lib that will be EOL by then.

  Hopefully fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=34092

ACKs for top commit:
  theStack:
    Concept and code review ACK fa1aa6c571

Tree-SHA512: fdd2fbc7b5c7ce33ad23b2e5431bb97eaf6ae8c2d2a55990a3ab73be79282c584b704dcd1471ba288de75283732970c70c9a03ddad059b97b66ba8b3de39effe
2021-05-10 14:20:01 +08:00
Sebastian Falbesoner
bd7f27d16d refactor: feature_csv_activation.py: move tx helper functions to methods
This allows to get rid of the global miniwallet variable and to specify
the used node self.nodes[0] at only one place, instead of passing it to
every tx creation/send method again and again.

Can be reviewed with --ignore-all-space --color-moved=dimmed-zebra
2021-05-10 01:31:33 +02:00
Sebastian Falbesoner
2eca46b0aa test: use MiniWallet for feature_csv_activation.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-10 01:31:27 +02:00
randymcmillan
a0f7978674 qt: enable wordWrap for peers-tab detail services 2021-05-09 18:00:38 -04:00
Karl-Johan Alm
7031721f2c rpc/listaddressgroupings: redefine inner-most array as ARR_FIXED
ARR_FIXED is for cases like this, where the elements are in an array for convenience, rather than due to being dynamically sized lists.
2021-05-09 22:22:26 +09:00
Karl-Johan Alm
8500f7bf54 rpc/createrawtransaction: redefine addresses as OBJ_USER_KEYS
The OBJ type is for actual objects with defined keys; OBJ_USER_KEYS is for objects with user-defined keys (such as the bitcoin address(es) in the createrawtransaction output object.
2021-05-09 22:22:25 +09:00
Karl-Johan Alm
d9e2183c50 rpc: include OBJ_USER_KEY in RPCArg constructor checks 2021-05-09 22:22:25 +09:00
Jon Atack
847288df07 test: fee rate values that cannot be represented as sat/vB 2021-05-09 12:50:04 +02:00
Jon Atack
06a90fa038 rpc: for sat/vB fee rates, limit ParseFixedPoint decimals to 3 2021-05-09 12:50:02 +02:00
Jon Atack
0742c7840f rpc: enable passing decimals to AmountFromValue, add doxygen 2021-05-09 12:50:00 +02:00
Jon Atack
8ce3ef57a3 test: ParseFixedPoint with 3 decimals for sat/vB fee rates 2021-05-09 12:49:58 +02:00
Jon Atack
b503327597 test: type error and out of range fee rates where missing 2021-05-09 12:49:56 +02:00
Jon Atack
c5fd4344f7 test: explicit fee rates with invalid amounts 2021-05-09 12:49:47 +02:00
Hennadii Stepanov
34b04eec44 refactor: Add TSA annotations to the WorkQueue class members 2021-05-09 13:48:07 +03:00
MarcoFalke
99993f0664 fuzz: Avoid excessively large min fee rate in tx_pool 2021-05-09 10:53:24 +02:00
MarcoFalke
facfc0f65d fuzz: Remove strprintf test cases that are known to fail 2021-05-09 10:25:21 +02:00
MarcoFalke
fa1aa6c571 fuzz: Limit ParseISO8601DateTime fuzzing to 32-bit 2021-05-09 10:04:01 +02:00
fanquake
42b589d18f scripts: test for MACHO control flow instrumentation 2021-05-09 14:26:09 +08:00
fanquake
469a5bc4fa build: build Boost with -fcf-protection when targeting Darwin
The LLVM Clang we use for cross-compilation supports this option, and it's expected
that any builders on macOS will also be using an Apple Clang that supports it.
2021-05-09 13:38:48 +08:00
MarcoFalke
5925f1e652 Merge bitcoin/bitcoin#21872: net: Sanitize message type for logging
09205b33aa net: Clarify message header validation errors (W. J. van der Laan)
955eee7680 net: Sanitize message type for logging (W. J. van der Laan)

Pull request description:

  - Use `SanitizeString` when logging message errors to make sure that the message type is sanitized. I have checked all logging in `net.cpp`.

  - For the `MESSAGESTART` error don't inspect and log header details at all: receiving invalid start bytes makes it likely that the packet isn't even formatted as valid P2P message. Logging the four unexpected start bytes (as hex) should be enough.

  - Update `p2p_invalid_messages.py` test to check this.

  - Improve error messages in a second commit.

  Issue reported by gmaxwell.

ACKs for top commit:
  MarcoFalke:
    re-ACK 09205b33aa only change is log message fixup 🔂
  practicalswift:
    re-ACK 09205b33aa

Tree-SHA512: 8fe5326af135cfcf39ea953d9074a8c966b9b85a810b06a2c45b8a745cf115de4f321e72fc769709d6bbecfc5953aab83176db6735b04c0bc6796f59272cadce
2021-05-09 07:01:26 +02:00
fanquake
8d5a0583c4 Merge bitcoin/bitcoin#21869: depends: Add missing -D_LIBCPP_DEBUG=1 to debug flags
fa9249aacc depends: Add missing -D_LIBCPP_DEBUG=1 to debug flags (MarcoFalke)

Pull request description:

  Commands that can be used for testing:

  ```
  $ cat 1.cpp
  #include <vector>

  int main() {
    std::vector<int> foo;
    foo.begin() + 7;
  }
  ```

  ```
  clang++ -stdlib=libc++ -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC -D_LIBCPP_DEBUG=1 -Wall 1.cpp -o exe && ./exe
  g++                    -D_GLIBCXX_DEBUG -D_GLIBCXX_DEBUG_PEDANTIC -D_LIBCPP_DEBUG=1 -Wall 1.cpp -o exe && ./exe

ACKs for top commit:
  practicalswift:
    cr ACK fa9249aacc: patch looks correct
  fanquake:
    ACK fa9249aacc - was going to suggest adding this to the macOS CPP flags as well, however it seems doing that is less straight forward. Could be looked at by someone in a followup.

Tree-SHA512: 2ffbaaf0ccb36bcc9fa1a15426566406c6115c8878ff211a4794d982c5d198672d444a20f6c7ae9f341193f6d8118c7cc50896daf98af9553834379e47ddb39e
2021-05-09 12:05:57 +08:00
fanquake
21d6fed815 Merge bitcoin/bitcoin#21884: fuzz: Remove unused --enable-danger-fuzz-link-all option
fa27d6d3ac fuzz: Remove unused --enable-danger-fuzz-link-all option (MarcoFalke)

Pull request description:

  Remove the unused build option, which was *dangerous* (as the name implies). Also remove the fuzzbuzz config, which was never used as part of this repo and seems redundant now that we integrate with oss-fuzz.

ACKs for top commit:
  practicalswift:
    cr ACK fa27d6d3ac: patch looks correct and rationale makes sense
  hebasto:
    ACK fa27d6d3ac, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 9bd65ed6a76d13d8d9c7a88aaae30f701215d5d0619693a3115d5ec350808aaf6a1aa4737466a5b96f3948513ec4d063808fe16219818366720e247880a15177
2021-05-09 11:26:22 +08:00
MarcoFalke
fa27d6d3ac fuzz: Remove unused --enable-danger-fuzz-link-all option 2021-05-08 09:32:45 +02:00
Hennadii Stepanov
d29ea72393 gui: Add access to the Peers tab from the network icon 2021-05-08 09:51:07 +03:00
MarcoFalke
9313c4e6aa Merge bitcoin/bitcoin#21874: fuzz: Add WRITE_ALL_FUZZ_TARGETS_AND_ABORT
fa5cb6b268 fuzz: Add WRITE_ALL_FUZZ_TARGETS_AND_ABORT (MarcoFalke)

Pull request description:

  This is needed when stdout is polluted by the fuzz engine. stderr can't be used instead because it is polluted by aborting the program.

Top commit has no ACKs.

Tree-SHA512: bf0a2a6bcd964ff1f0f3ef6e7e297b4c780430c4d6312332ed99ace0e1c58243c1483fd387e39405837d39b36072dfeb9ae03d2a7aa728ad6955159754fd5766
2021-05-07 15:44:19 +02:00
W. J. van der Laan
09205b33aa net: Clarify message header validation errors
Make the errors less shouty and more descriptive.
2021-05-07 11:28:21 +02:00
MarcoFalke
fa5cb6b268 fuzz: Add WRITE_ALL_FUZZ_TARGETS_AND_ABORT 2021-05-07 11:01:05 +02:00
MarcoFalke
eb9a1fe037 Merge bitcoin/bitcoin#21802: refactor: Avoid UB in util/asmap (advance a dereferenceable iterator outside its valid range)
fa09871320 refactor: Avoid sign-compare compiler warning in util/asmap (MarcoFalke)

Pull request description:

  Can be reproduced on current master with `D_GLIBCXX_DEBUG`:

  ```
  /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/debug/safe_iterator.h:883:
  In function:
      __gnu_debug::_Safe_iterator<type-parameter-0-0, type-parameter-0-1,
      std::random_access_iterator_tag>::_Self __gnu_debug::operator+(const
      __gnu_debug::_Safe_iterator<type-parameter-0-0, type-parameter-0-1,
      std::random_access_iterator_tag>::_Self &,
      __gnu_debug::_Safe_iterator<type-parameter-0-0, type-parameter-0-1,
      std::random_access_iterator_tag>::difference_type)

  Error: attempt to advance a dereferenceable iterator 369 steps, which falls
  outside its valid range.

  Objects involved in the operation:
      iterator @ 0x0x7ffd3d613138 {
        type = std::__cxx1998::_Bit_const_iterator (constant iterator);
        state = dereferenceable;
        references sequence with type 'std::__debug::vector<bool, std::allocator<bool> >' @ 0x0x7ffd3d663590
      }
  ==65050== ERROR: libFuzzer: deadly signal
      #0 0x559ab9787690 in __sanitizer_print_stack_trace (/bitcoin/src/test/fuzz/fuzz+0x5a1690)
      #1 0x559ab9733998 in fuzzer::PrintStackTrace() (/bitcoin/src/test/fuzz/fuzz+0x54d998)
      #2 0x559ab9718ae3 in fuzzer::Fuzzer::CrashCallback() (/bitcoin/src/test/fuzz/fuzz+0x532ae3)
      #3 0x7f70a0e723bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
      #4 0x7f70a0b3418a in raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
      #5 0x7f70a0b13858 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
      #6 0x7f70a0f21148  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xa1148)
      #7 0x559ab9f60a96 in __gnu_debug::operator+(__gnu_debug::_Safe_iterator<std::__cxx1998::_Bit_const_iterator, std::__debug::vector<bool, std::allocator<bool> >, std::random_access_iterator_tag> const&, long) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/debug/safe_iterator.h:881:2
      #8 0x559ab9f61062 in SanityCheckASMap(std::__debug::vector<bool, std::allocator<bool> > const&, int) util/asmap.cpp:159:21
      #9 0x559ab9e4fdfa in SanityCheckASMap(std::__debug::vector<bool, std::allocator<bool> > const&) netaddress.cpp:1242:12
      #10 0x559ab9793fcb in addrman_fuzz_target(Span<unsigned char const>) test/fuzz/addrman.cpp:43:14
      #11 0x559ab978a03c in std::_Function_handler<void (Span<unsigned char const>), void (*)(Span<unsigned char const>)>::_M_invoke(std::_Any_data const&, Span<unsigned char const>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:300:2
      #12 0x559aba2692c7 in std::function<void (Span<unsigned char const>)>::operator()(Span<unsigned char const>) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #13 0x559aba269132 in LLVMFuzzerTestOneInput test/fuzz/fuzz.cpp:63:5
      #14 0x559ab971a1a1 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/bitcoin/src/test/fuzz/fuzz+0x5341a1)
      #15 0x559ab97198e5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/bitcoin/src/test/fuzz/fuzz+0x5338e5)
      #16 0x559ab971bb87 in fuzzer::Fuzzer::MutateAndTestOne() (/bitcoin/src/test/fuzz/fuzz+0x535b87)
      #17 0x559ab971c885 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/bitcoin/src/test/fuzz/fuzz+0x536885)
      #18 0x559ab970b23e in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/bitcoin/src/test/fuzz/fuzz+0x52523e)
      #19 0x559ab9734082 in main (/bitcoin/src/test/fuzz/fuzz+0x54e082)
      #20 0x7f70a0b150b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
      #21 0x559ab96dffdd in _start (/bitcoin/src/test/fuzz/fuzz+0x4f9fdd)

ACKs for top commit:
  sipa:
    utACK fa09871320
  vasild:
    ACK fa09871320

Tree-SHA512: 802fda33bda40fe2521f1e3be075ceddc5fd9ba185bd494286e50019931dfd688da7a6513601138b1dc7bb8e80ae47c8572902406eb59f68990619ddb2656748
2021-05-07 10:27:49 +02:00
MarcoFalke
a33f360fcd Merge bitcoin/bitcoin#21873: test: minor fixes & improvements for files linter test
2227fc4e62 test: minor fixes & improvements for files linter test (windsok)

Pull request description:

  Couple of minor fixes & improvements for files linter test added in #21740

  - Use a context manager when opening files, so that files are closed are we are done with them

  - Use the `-z` flag when shelling out to `git ls-files` so that we can catch newlines and other weird control characters in filenames.

  From the `git ls-files` manpage:
  ```
  -z \0 line termination on output and do not quote filenames. See OUTPUT below for more information.

  Without the -z option, pathnames with "unusual" characters are quoted as explained for the configuration variable
  core.quotePath (see git-config(1)). Using -z the filename is output verbatim and the line is terminated by a NUL byte.
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK 2227fc4e62
  practicalswift:
    cr ACK 2227fc4e62: patch looks correct

Tree-SHA512: af059a805f4a7614162de85dea856052a45ab531895cb0431087e7fc9e037513fa7501bb5eb2fe43238adf5f09e77712ebdbb15b1486983359ad3661a3da0c60
2021-05-07 10:23:43 +02:00
fanquake
a0d1d487e9 Merge bitcoin/bitcoin#21845: net processing: Don't require locking cs_main before calling RelayTransactions()
39e19713cd [net processing] Add internal _RelayTransactions() (John Newbery)

Pull request description:

  As part of the general effort to reduce cs_main usage in net_processing, this removes the need to be holding `cs_main` when calling `RelayTransactions()` from outside net_processing. Internally, we lock `cs_main` and call an internal `_RelayTransactions()` function that _does_ require `cs_main`.

ACKs for top commit:
  MarcoFalke:
    re-unsigned-code-review ACK 39e19713cd
  promag:
    Code review ACK 39e19713cd, just included sync.h since last review.
  ajtowns:
    ACK 39e19713cd

Tree-SHA512: dc08441233adfb8eaac501cf497cb4bad029eb723bd3fa8a3d8b7e49cc984c98859b95780ad15f5701d62ac745a8223beb0df405e3d49d95a8c86c8be17c9543
2021-05-07 11:11:05 +08:00
windsok
2227fc4e62 test: minor fixes & improvements for files linter test
Updates the lint-files.py lint test:
* Use a context manager when opening files, so that files are closed.
* Use the -z flag when shelling out to git ls-files so that we can catch newlines
  and other weird control characters in filenames
2021-05-06 18:21:06 -07:00
W. J. van der Laan
955eee7680 net: Sanitize message type for logging
- Use `SanitizeString` when logging message errors to make sure that the
message type is sanitized.

- For the `MESSAGESTART` error don't inspect and log header details at
all: receiving invalid start bytes makes it likely that the packet isn't
even formatted as valid P2P message. Logging the four unexpected start
bytes should be enough.

- Update `p2p_invalid_messages.py` test to check this.

Issue reported by gmaxwell.
2021-05-06 17:30:52 +02:00
MarcoFalke
06d573f053 Merge bitcoin/bitcoin#21867: test: use MiniWallet for p2p_blocksonly.py
9f767e8438 test: use MiniWallet for p2p_blocksonly.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (p2p_blocksonly.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078.

  Note that MiniWallet creates segwit transactions by default, i.e. txid and wtxid are not identical and we have to return both from `check_p2p_tx_violation(...)`: wtxid is needed to match an expected `"received getdata for: wtx ..."` debug output, whereas the txid is needed to wait for a certain tx via `wait_for_tx(...)`.

ACKs for top commit:
  jonatack:
    ACK 9f767e8438 tested with `--disable-wallet`

Tree-SHA512: f08001f02c3c310ccdf713af0ba17304368a36414f412749908bbe8c03ad1e902190b8768b79f3b4909855762f285e7ab1b627cc4f45c90b42bb097a43cb4318
2021-05-06 16:52:55 +02:00
MarcoFalke
1fcf66af68 Merge bitcoin/bitcoin#21798: fuzz: Create a block template in tx_pool targets
fa03d0acd6 fuzz: Create a block template in tx_pool targets (MarcoFalke)
fa61ce5cf5 fuzz: Limit mocktime to MTP in tx_pool targets (MarcoFalke)
fab646b8ea fuzz: Use correct variant of ConsumeRandomLengthString instead of hardcoding a maximum size (MarcoFalke)
fae2c8bc54 fuzz: Allow to pass min/max to ConsumeTime (MarcoFalke)

Pull request description:

  Relatively simple check to ensure a block can always be created from the mempool

ACKs for top commit:
  practicalswift:
    Tested ACK fa03d0acd6

Tree-SHA512: e613376ccc88591cbe594db14ea21ebc9b2b191f6325b3aa4ee0cd379695352ad3b480e286134ef6ee30f043d486cf9792a1bc7e44445c41045ac8c3b931c7ff
2021-05-06 16:06:20 +02:00
Jon Atack
ea6f76b66e test: improve zero-value explicit fee rate coverage 2021-05-06 09:06:27 +02:00
fanquake
9c05da4a5c Merge bitcoin/bitcoin#21865: ci: Properly pass msan cflags
fa3bbcf81e ci: Properly pass msan cflags (MarcoFalke)

Pull request description:

  Uninstrumented libraries might cause false positives with msan. Fixes #21632

ACKs for top commit:
  practicalswift:
    cr ACK fa3bbcf81e: patch looks correct
  fanquake:
    ACK fa3bbcf81e

Tree-SHA512: 55af961f3d422074bed4eec15df6b8d03cf5d7687a2ca5ea04c2591c0292ad3749db821d6160ba0e4458f0dcf448f0b2f05cc496314839264ddc8b89cced9e14
2021-05-06 14:00:54 +08:00
MarcoFalke
fa9249aacc depends: Add missing -D_LIBCPP_DEBUG=1 to debug flags 2021-05-06 07:47:27 +02:00
Sebastian Falbesoner
9f767e8438 test: use MiniWallet for p2p_blocksonly.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-06 02:30:05 +02:00
MarcoFalke
fa3bbcf81e ci: Properly pass msan cflags 2021-05-05 21:47:12 +02:00
MarcoFalke
f8575bce31 Merge bitcoin/bitcoin#21864: fix permissions on 00_setup_env_native_fuzz_with_msan.sh
d48565d109 fix permissions on 00_setup_env_native_fuzz_with_msan (glozow)

Pull request description:

  I think there was a silent merge conflict between #21852 and #21740?

  I have a [linter failure](https://cirrus-ci.com/task/5436849834426368):

  ```
  File "ci/test/00_setup_env_native_fuzz_with_msan.sh" contains a shebang line, but has the file permission 644 instead of the expected executable permission 755. Do "chmod 755 ci/test/00_setup_env_native_fuzz_with_msan.sh" (or remove the shebang line).
  ERROR: There were 1 failed tests in the lint-files.py lint test. Please resolve the above errors.
  ^---- failure generated from test/lint/lint-files.sh
  ```

ACKs for top commit:
  MarcoFalke:
    ACK d48565d109

Tree-SHA512: 445bdd738faf007451f40bbcf360dd1fb4675e17a4c96546e6818c12e33dd336dadd95cf8d4b5f8df1d6ccfbc4bf5496864bb5528e416cea894857b6b732140c
2021-05-05 21:05:36 +02:00
glozow
d48565d109 fix permissions on 00_setup_env_native_fuzz_with_msan 2021-05-05 11:45:12 -07:00
MarcoFalke
fa2204f6ad streams: Accept URef obj for VectorReader unserialize 2021-05-05 20:19:03 +02:00
MarcoFalke
fa03d0acd6 fuzz: Create a block template in tx_pool targets 2021-05-05 20:12:00 +02:00
MarcoFalke
128b98fce3 Merge bitcoin/bitcoin#21681: validation: fix ActivateSnapshot to use hardcoded nChainTx
91d93aac4e validation: remove nchaintx from assumeutxo metadata (James O'Beirne)
931684b24a validation: fix ActivateSnapshot to use hardcoded nChainTx (James O'Beirne)

Pull request description:

  This fixes an oversight from the move of nChainTx from the user-supplied
  snapshot metadata into the hardcoded assumeutxo chainparams.

  Since the nChainTx is now unused in the metadata, it should be removed
  in a future commit.

  See: https://github.com/bitcoin/bitcoin/pull/19806#discussion_r612165410

ACKs for top commit:
  Sjors:
    utACK 91d93aac4e
  ryanofsky:
    Code review ACK 91d93aac4e. No change to previous commit, just new commit removing now unused utxo snapshot field and updating tests.

Tree-SHA512: 445bdd738faf007451f40bbcf360dd1fb4675e17a4c96546e6818c12e33dd336dadd95cf8d4b5f8df1d6ccfbc4bf5496864bb5528e416cea894857b6b732140c
2021-05-05 18:34:16 +02:00
MarcoFalke
32f1f021bf Merge bitcoin/bitcoin#21817: refactor: Replace &foo[0] with foo.data()
fac30eec42 refactor: Replace &foo[0] with foo.data() (MarcoFalke)
faece47c47 refactor: Avoid &foo[0] on C-Style arrays (MarcoFalke)
face961109 refactor: Use only one temporary buffer in CreateObfuscateKey (MarcoFalke)
fa05dddc42 refactor: Use CPubKey vector constructor where possible (MarcoFalke)
fabb6dfe6e script: Replace address-of idiom with vector data() method (Guido Vranken)

Pull request description:

  The main theme of this refactor is to replace `&foo[0]` with `foo.data()`.

  The first commit is taken from #21781 with the rationale:

  * In CSignatureCache::ComputeEntryECDSA, change the way a vector pointer is resolved to prevent invoking undefined behavior if the vector is empty.

  The other commits aim to remove all `&foo[0]`, where `foo` is any kind of byte representation. The rationale:

  * Sometimes alternative code without any raw data pointers is easier to read (refer to the respective commit message for details)
  * If the raw data pointer is needed, `foo.data()` should be preferred, as pointed out in the developer notes. This addresses the instances that have been missed in commit 592404f03f, and https://github.com/bitcoin/bitcoin/pull/9804

ACKs for top commit:
  laanwj:
    Code review ACK fac30eec42
  practicalswift:
    cr ACK fac30eec42: patch looks correct
  promag:
    Code review ACK fac30eec42.

Tree-SHA512: e7e73146edbc78911a8e8c728b0a1c6b0ed9a88a008e650aa5dbffe72425bd42c76df70199a9cf7e02637448d7593e0eac52fd0f91f59240283e1390ee21bfa5
2021-05-05 18:24:09 +02:00
W. J. van der Laan
1b9a5236e9 Merge bitcoin/bitcoin#21740: test: add new python linter to check file names and permissions
46b025e00d test: add new python linter to check file names and permissions (windsok)
6f6bb3ebc7 test: fix file permissions on various scripts (windsok)

Pull request description:

  Adds a new python linter test which tests for correct filenames and file permissions in the repository.

  Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050

  Summary of tests:
  * Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.

  * Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test)

  * Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.

  * Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test)

  * Checks every file that contains a shebang line to ensure it has an executable permission

  Additionally updates the permissions on various files to comply with the new tests.

  Fixes #21729

ACKs for top commit:
  practicalswift:
    cr re-ACK 46b025e00d: patch still looks correct
  kiminuo:
    code review ACK 46b025e00d if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK.
  laanwj:
    Code review ACK 46b025e00d

Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
2021-05-05 17:14:22 +02:00
W. J. van der Laan
b8b6801412 Merge bitcoin/bitcoin#21710: doc: update helps for addnode rpc and -addnode/-maxconnections config options
b4fcbcfb49 doc: update -maxconnections config option help (Jon Atack)
79685a8992 doc: update -addnode config option help (Jon Atack)
2896c6c4cc doc: update addnode rpc help (Jon Atack)

Pull request description:

  Since #9319 proposed by Gregory Maxwell and released in v0.14, peers manually added through the `-addnode` config option or using the `addnode` RPC have their own separate limit of 8 connections that does not compete with other inbound or outbound connection usage and is not subject to the limitation imposed by the `-maxconnections` option.

  This PR updates the `-addnode` and `-maxconnections` config options and the `addnode` RPC help docs with this information.

  `-addnode` config option help
  ```
  $ bitcoind -h | grep -A5 addnode=
    -addnode=<ip>
         Add a node to connect to and attempt to keep the connection open (see
         the addnode RPC help for more info). This option can be specified
         multiple times to add multiple nodes; connections are limited to
         8 at a time and are counted separately from the -maxconnections
         limit.

  $ bitcoind -h | grep -A3 maxconnections=
    -maxconnections=<n>
         Maintain at most <n> connections to peers (default: 125). This limit
         does not apply to connections manually added via -addnode or the
         addnode RPC, which have a separate limit of 8.
  ```

  `addnode` rpc help
  ```
  $ bitcoin-cli help addnode
  addnode "node" "command"

  Attempts to add or remove a node from the addnode list.
  Or try a connection to a node once.
  Nodes added using addnode (or -connect) are protected from DoS disconnection and are not required to be
  full nodes/support SegWit as other outbound peers are (though such peers will not be synced from).
  Addnode connections are limited to 8 at a time and are counted separately from the -maxconnections limit.
  ```

ACKs for top commit:
  prayank23:
    ACK b4fcbcfb49
  jarolrod:
    ACK b4fcbcfb49

Tree-SHA512: b6d69baa6cbf6d53f91bac5b39b549d49db6c95f92ea1bdd3588a6432794a25ac2c8b3c89e2c72bb9097e61f2717c8b5ecc404745d5992b88e523db03200898f
2021-05-05 16:58:22 +02:00
MarcoFalke
a0f46028fc Merge bitcoin/bitcoin#21814: test: Fix feature_config_args.py intermittent issue
fab1eb65b1 test: Fix feature_config_args.py intermittent issue (MarcoFalke)

Pull request description:

  Fix #21448

ACKs for top commit:
  laanwj:
    Code review ACK fab1eb65b1

Tree-SHA512: cad9f684f43aa801d0c1cb5f1684ffa624df1216be225cea46b1389ba2b67cbd6159ffb786fe144bf1ca865623dd9a10289d4293cfabb678bdb243d4ea00734d
2021-05-05 16:46:24 +02:00
MarcoFalke
576300afb0 Merge bitcoin/bitcoin#21856: doc: add OSS-Fuzz section to fuzzing.md doc
47c3ea021e doc: add OSS-Fuzz section to fuzzing.md doc (Adam Jonas)

Pull request description:

  This adds documentation about [Bitcoin Core's participation](https://github.com/google/oss-fuzz/pull/5699/files) in Google's OSS-Fuzz program and adds the caveat that the project may not disclose vulnerabilities within the 90-day window described in the [program's disclosure guidelines](https://google.github.io/oss-fuzz/getting-started/bug-disclosure-guidelines/).

ACKs for top commit:
  jonatack:
    ACK 47c3ea021e

Tree-SHA512: 87bf0146fb74d1e4b3b8839e6c8f3d53046008a6d5b926ffe5b95be3c396a5e47e47967533422f60b04c4446482f49d210ada410b742f69781a7afde623d704d
2021-05-05 16:34:00 +02:00
Adam Jonas
47c3ea021e doc: add OSS-Fuzz section to fuzzing.md doc
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2021-05-05 10:10:56 -04:00
W. J. van der Laan
54617fad99 Merge bitcoin/bitcoin#21709: doc: update reduce-memory.md and bitcoin.conf -maxconnections info
300234ab66 doc: update bitcoin.conf maxconnections info (Jon Atack)
926827065f doc: update reduce-memory.md peer connections info (Jon Atack)

Pull request description:

  This patch updates the documentation in `doc/reduce-memory.md` and `share/examples/bitcoin.conf` regarding the peer connections limits and `-maxconnections`

ACKs for top commit:
  jarolrod:
    re-ACK 300234ab66
  laanwj:
    ACK 300234ab66
  prayank23:
    ACK 300234ab66

Tree-SHA512: 90f53626124afb50706e6a3b644bc7bb800bb7cf41ae7062c20c17b3d9bdf4a8d73b4cf188faec9113d772596f7e4bc36a4a69481cacb92cc55d5956181d0c31
2021-05-05 16:10:55 +02:00
W. J. van der Laan
3275c6e578 Merge bitcoin/bitcoin#21727: refactor: Move more stuff to blockstorage
fa09a9eac8 style: Add { } to multi-line if (MarcoFalke)
fadafab833 move-only: Move functions to blockstorage (MarcoFalke)
fa7e64d586 move-only: Move constants to blockstorage (MarcoFalke)
fa247a327f refactor: Move block storage globals to blockstorage (MarcoFalke)
fa81c30c6f refactor: Move pruning/reindex/importing globals to blockstorage (MarcoFalke)

Pull request description:

  See #21575

ACKs for top commit:
  Sjors:
    ACK fa09a9eac8
  kiminuo:
    ACK fa09a9e
  laanwj:
    Code review ACK fa09a9eac8
  promag:
    Code review ACK fa09a9eac8. Since last review

Tree-SHA512: 2eb6962ff44da6b77f3058fc02ec66ab742e25ae8dcc8ec62b062896571910d43ca7c4bb16fb3ccb5e5245195b8dec6384b6c8d442fa97ca28d93bdff347d677
2021-05-05 16:03:03 +02:00
W. J. van der Laan
23109cc548 Merge bitcoin/bitcoin#21753: doc: add -addrinfo to tor docs
65f30e4c21 doc: add -addrinfo troubleshooting section to tor.md (Jon Atack)

Pull request description:

  Follow-up to #21595.

ACKs for top commit:
  jarolrod:
    ACK 65f30e4c21
  practicalswift:
    ACK 65f30e4c21
  0xB10C:
    ACK 65f30e4c21
  theStack:
    ACK 65f30e4c21

Tree-SHA512: d17fa007106b8f877d2632c99273c663a24f025febe52faec9b197c561df808fd6a92bb27992ccbf5c3cc0d82058a8c4b82a2f1b99325f0ddfdac5ef703ac7d7
2021-05-05 15:52:00 +02:00
W. J. van der Laan
75b3a32f03 Merge bitcoin/bitcoin#21821: test: Add missing test for empty P2WSH redeem
fa80a11c3b test: Add missing test for empty P2WSH redeem (MarcoFalke)

Pull request description:

ACKs for top commit:
  0xB10C:
    ACK fa80a11c3b

Tree-SHA512: 2d28fe2e80c0770d82e4164b235345a3a828e6b71445a82aa144b3efb3298e494a8bc456a329f175a3cb542fa1b669d869dc1fb1b711611feeb76ab312a661d5
2021-05-05 15:47:19 +02:00
W. J. van der Laan
fb43d7cb1e Merge bitcoin/bitcoin#21787: test: fix off-by-ones in rpc_fundrawtransaction assertions
f09e6b2585 test: fix off-by-ones in rpc_fundrawtransaction (Jon Atack)

Pull request description:

  The variables in these assertions should be the same within each line.

ACKs for top commit:
  laanwj:
    ACK f09e6b2585
  theStack:
    ACK f09e6b2585

Tree-SHA512: 7ac754eaadd8cb00a725afa55bccbb8de7547dedac9350d79a9a470918245617e075c56a91adc36fb653bbe8a0a325d59b00443155a7e1a81ebf22e4e4cf56d9
2021-05-05 15:32:28 +02:00
W. J. van der Laan
df21e500d6 Merge bitcoin/bitcoin#21427: depends: Fix id_string invocations
fa872c9af3 depends: Fix id_string invocations (Carl Dong)

Pull request description:

  Closes: #21242

  ```
  Reproduced from depends/Makefile comment:

  When invoking a shell, GNU Make special-cases exit code 127 (command not
  found) by not capturing the output but instead passing it through. This
  is not done for any other exit code.

  Therefore, we require a "|| true" to avoid this behaviour when in an
  environment where the build_* or host_* may not exist yet.
  ```

ACKs for top commit:
  laanwj:
    Concept and light code review ACK fa872c9af3

Tree-SHA512: 9ce88381aec579d956572cf70c4f69dc5a3873f0d2af14a71cf24814192a89452b8280258bed8cca804e4bd2644db056d213ab733df46a10560a47079524d8ac
2021-05-05 14:32:12 +02:00
W. J. van der Laan
cdcf82622d Merge bitcoin/bitcoin#21629: build: fix configuring when building depends with NO_BDB=1
a5491882a0 build: fix configuring when building depends with NO_BDB=1 (fanquake)

Pull request description:

  Currently, if you build depends using `NO_BDB=1` (only sqlite wallets), `./configure` will fail as it still tries to find bdb. i.e:
  ```bash
  make -C depends/ NO_QT=1 NO_BDB=1 NO_UPNP=1 NO_ZMQ=1 NO_NATPMP=1 -j8
  ...
  copying packages: native_b2 boost libevent sqlite

  ./autogen.sh
  ./configure --prefix=/home/ubuntu/bitcoin/depends/x86_64-pc-linux-gnu
  ...
  checking for Berkeley DB C++ headers... default
  configure: error: Found Berkeley DB other than 4.8, required for portable BDB wallets (--with-incompatible-bdb to ignore or --without-bdb to disable BDB wallet support)
  ```

  This PR fixes the build such that you can build depends, opting out of bdb, without opting out of wallets entirely, and still configure successfully. I think I've tested across most potential configurations. i.e:
  ```bash
   ./configure (bdb and sqlite on system)
  bdb & sqlite are both are available

  ./configure --without-bdb  (bdb and sqlite on system)
  only sqlite

  ./configure --without-sqlite  (bdb and sqlite on system)
  only bdb

  ./configure --disable-wallet  (bdb and sqlite on system)
  neither bdb or sqlite

  depends NO_WALLET=1
  ./configure --prefix=/bitcoin/depends/x86_64-apple-darwin19.6.0
  neither bdb or sqlite

  depends NO_BDB=1
  ./configure --prefix=/bitcoin/depends/x86_64-apple-darwin19.6.0
  only sqlite

  depends NO_SQLITE=1
  ./configure --prefix=/bitcoin/depends/x86_64-apple-darwin19.6.0
  only bdb

  depends
  ./configure --prefix=/bitcoin/depends/x86_64-apple-darwin19.6.0
  bdb and sqlite
  ```

ACKs for top commit:
  laanwj:
    Code review ACK a5491882a0
  jarolrod:
    ACK a5491882a0

Tree-SHA512: baf7d2543a401db0d846095415ff449c04ecfb4a74c734dc51e79453702f9051210daeef686970f11fcffd32cdfadbc58acd54f0706aceecfb3edb0ff17310d7
2021-05-05 13:29:05 +02:00
W. J. van der Laan
b7c2625703 Merge bitcoin/bitcoin#21664: contrib: use LIEF for macOS and Windows symbol & security checks
7fc5e865b9 test: install lief in CI (fanquake)
955140b326 contrib: consolidate PIE and NX security checks (fanquake)
2aa1631822 contrib: use LIEF in PE symbol checks (fanquake)
e93ac26b85 contrib: use LIEF in macOS symbol checks (fanquake)
a632cbcee5 contrib: use f strings in symbol-check.py (fanquake)
0f5d77c8e4 contrib: add PE PIE check to security checks (fanquake)
8e1f40dd9a contrib: use LIEF for PE security checks (fanquake)
a25b2e965c contrib: use LIEF for macOS security checks (fanquake)
7e7eae7aa8 contrib: use f strings in security-check.py (fanquake)
2e7a9f7ade guix: install LIEF in Guix container (fanquake)
465967b5ef gitian: install LIEF in gitian container (fanquake)

Pull request description:

  This PR is a proof of concept for using [LIEF](https://github.com/lief-project/LIEF) for the PE and MACHO symbol and security checks. It replaces our current approach of manually parsing the output of `objdump` & `otool`. If the consensus is that using LIEF is ok, then I also plan on replacing [pixie.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/pixie.py), and using LIEF for all checks. LIEF for Linux is also currently blocked (on the next release, unless we want to build master) on one change for RISC-V that I [sent upstream](https://github.com/lief-project/LIEF/pull/562).

  LIEF is seemingly well maintained, and is the basis for a number of other tools. It also has some very nice documentation; i.e the [Python API for ELF](https://lief.quarkslab.com/doc/latest/api/python/elf.html). It also has many builtins we can take advantage of. i.e [`is_pie`](https://lief.quarkslab.com/doc/latest/api/python/macho.html#lief.MachO.Binary.is_pie), [`has_nx`](https://lief.quarkslab.com/doc/latest/api/python/macho.html#lief.MachO.Binary.has_nx) etc. This means we can [consolidate some of our checks](9c5eeb5484). If/when end up using LIEF for lightning then we can consolidate further, and cleanup these scripts. i.e to not parse the binary inside the checks, but once at the start of the script.

  Guix builds:
  ```bash
  # find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  963a08638c46f9a3d75cd4b0c155d1ca091bbeba27167291adcd3dca03fd4c3d  guix-build-f51237d94d98/output/aarch64-linux-gnu/bitcoin-f51237d94d98-aarch64-linux-gnu-debug.tar.gz
  a3ce927c46b103789a010c41a6ebfafe4548d90ee7d88f2a735c9183b775da5c  guix-build-f51237d94d98/output/aarch64-linux-gnu/bitcoin-f51237d94d98-aarch64-linux-gnu.tar.gz
  2503ac8901068805d5e7251fd5cfeb7c1f8ba3528bdfcf3aa1e0c40bfd5c1cbc  guix-build-f51237d94d98/output/arm-linux-gnueabihf/bitcoin-f51237d94d98-arm-linux-gnueabihf-debug.tar.gz
  5798697e58e1788df85aa9e2e4d33fef0456169fcbd2521f13b3b5806ac0d84d  guix-build-f51237d94d98/output/arm-linux-gnueabihf/bitcoin-f51237d94d98-arm-linux-gnueabihf.tar.gz
  4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2  guix-build-f51237d94d98/output/dist-archive/bitcoin-f51237d94d98.tar.gz
  9b4b8756c5c84295eb6b61b6b32a07a8d07723fb38aaa8f519b6133935061bda  guix-build-f51237d94d98/output/powerpc64-linux-gnu/bitcoin-f51237d94d98-powerpc64-linux-gnu-debug.tar.gz
  cbd821aa464a9c16f7979dbec1a5e66939e777a567f55f7081499a8d528d42c5  guix-build-f51237d94d98/output/powerpc64-linux-gnu/bitcoin-f51237d94d98-powerpc64-linux-gnu.tar.gz
  abed530a82e97e3cf621c90a13c0881b0e39ccce2a6f42a3ff80de76e2abc5f7  guix-build-f51237d94d98/output/powerpc64le-linux-gnu/bitcoin-f51237d94d98-powerpc64le-linux-gnu-debug.tar.gz
  8b6d2bdd8b58ff1f6072bf8693abe3ce773ff3a7d8d2b7218207e69945b9d31b  guix-build-f51237d94d98/output/powerpc64le-linux-gnu/bitcoin-f51237d94d98-powerpc64le-linux-gnu.tar.gz
  d99cc705032d22ae819975992216899ed960ba25871a05c8789d00b80418511f  guix-build-f51237d94d98/output/riscv64-linux-gnu/bitcoin-f51237d94d98-riscv64-linux-gnu-debug.tar.gz
  5240ca4f4ef7c62088185224ac319ad9a4a9b40075df10af18d8a6355bca32fb  guix-build-f51237d94d98/output/riscv64-linux-gnu/bitcoin-f51237d94d98-riscv64-linux-gnu.tar.gz
  adc16eaee4b51e8615ce8b3be9f6c018698237df4ad6e0886cf0d4ab6bc9e5c4  guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx-unsigned.dmg
  b188af0572ee682d74cc82c7e6e464115205fc130a457cfe19d42ac9ddd267f8  guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx-unsigned.tar.gz
  e764062fde144e6fb5d6dd776c10fc2daa8d775831f7e43247d17a6c6e060c97  guix-build-f51237d94d98/output/x86_64-apple-darwin18/bitcoin-f51237d94d98-osx64.tar.gz
  dab3d26ac94c669140f7329d14e57ef02b0fe92b8a8f9d96c32a416adea0da0f  guix-build-f51237d94d98/output/x86_64-linux-gnu/bitcoin-f51237d94d98-x86_64-linux-gnu-debug.tar.gz
  ca59d4379fbe2b9a52deebeaf88508e0eda4215f28d319aff0781289dd159712  guix-build-f51237d94d98/output/x86_64-linux-gnu/bitcoin-f51237d94d98-x86_64-linux-gnu.tar.gz
  52b7c35321a85c4f6c95bf0e687574454b71ede9bec1c9cf17f37c578c888a94  guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win-unsigned.tar.gz
  a543895a00f8ffb3ba50ca68396d52ad5a18dd8efe38730e0049dd70d283a092  guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64-debug.zip
  aec050d03c65268a986148500f7341cceb8c5f85287e0e3cde8933ce4b4dee32  guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64-setup-unsigned.exe
  57ba33ed6ee8d3a885e342471359301473e83037d5442895beb686921a4c50e9  guix-build-f51237d94d98/output/x86_64-w64-mingw32/bitcoin-f51237d94d98-win64.zip
  ```

  Gitian builds:
  ```bash
  # macOS:
  2f066e852bdd30ac46e5ecdf7619d19d408035c318a3edf0f1893ec2e25efb69  bitcoin-41a1b3d1b130-osx-unsigned.dmg
  8cf8ac4d21740f490262453c330b5f4a5c5b8139dfc1b322efefce3f3b93d1b2  bitcoin-41a1b3d1b130-osx-unsigned.tar.gz
  cf1b84efdd9d2588a1ce9513580fb56b38bfafe60e18f8adbeedf03521c6c2b2  bitcoin-41a1b3d1b130-osx64.tar.gz
  14995244b0bb3e80e7b79975c9c70fdfb3ee3c04fda3efd5358ce1c4efa3a312  src/bitcoin-41a1b3d1b130.tar.gz
  93881069d5e1dc385c08895a7b035a94eb010325afc2776c99b6aafa21096eb8  bitcoin-core-osx-22-res.yml

  # Windows:
  4d56dd7713121684b7eaa448679c65df2fd0aa5319bf8d12fb6cfa9f0b005cf7  bitcoin-f51237d94d98-win-unsigned.tar.gz
  4558f4173152b084bcba25aa1a53c605208a70fe20392141b63cefb476528c85  bitcoin-f51237d94d98-win64-debug.zip
  b63feaca010e86d514cfe38d716e3c8a8b8058e4f969b868aaaeb8a8a3d3dc81  bitcoin-f51237d94d98-win64-setup-unsigned.exe
  de7d8586cc91ba391fe911853a99d9fd15fc6f9a60f9b91a0447940173aac67a  bitcoin-f51237d94d98-win64.zip
  4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2  src/bitcoin-f51237d94d98.tar.gz
  45efaca35b5fad0a04dfd06e44f7c00b990aa91c7bf2faea57e020d3491a6cf0  bitcoin-core-win-22-res.yml

  # Linux:
  055d646c5f8cf4708008374546176012ff758566a2645a3a01e1a33eab1002fe  bitcoin-f51237d94d98-aarch64-linux-gnu-debug.tar.gz
  bfc8b0efc36b0474c88546b12d2723c04b4dc629ae311082025c7e0b8f0d1aa9  bitcoin-f51237d94d98-aarch64-linux-gnu.tar.gz
  9dfaa5acfffadad8942b32996458013a155d12ed07be76601f232233627b5cb9  bitcoin-f51237d94d98-arm-linux-gnueabihf-debug.tar.gz
  54eb57905ff8513b9f628707b61aa4659c362fb2f6d17e0ee240b4da3674907d  bitcoin-f51237d94d98-arm-linux-gnueabihf.tar.gz
  ad98d876616eff578ad8cfd17dfbabe48ed14200823579687d66694bae3d2fe3  bitcoin-f51237d94d98-powerpc64-linux-gnu-debug.tar.gz
  fe1b421dd1cb6e04d5dc5d341459dc15fa6e15b80906e5d8e0405cf43495e0f7  bitcoin-f51237d94d98-powerpc64-linux-gnu.tar.gz
  9001d95cc7d2722d9d7dd83d9da8e5adf575fddf91b615b76b9bcfece30ecf6f  bitcoin-f51237d94d98-powerpc64le-linux-gnu-debug.tar.gz
  9e0650ad2aba70c0fd1608a077e95f335dc1bb4a79eab9b0b56ac87427a4fd4f  bitcoin-f51237d94d98-powerpc64le-linux-gnu.tar.gz
  fbfde0134944d3dbd32991455b0a8abdd334853ab8a4c1a1a4c060d9de071c50  bitcoin-f51237d94d98-riscv64-linux-gnu-debug.tar.gz
  2fa2cfddce98c44c65305326fc623a7f065129208337503d813a08d51580cb8a  bitcoin-f51237d94d98-riscv64-linux-gnu.tar.gz
  b2d6caeee0e3c350a43165c39876ebed8e588958007af0d06996e341c7060683  bitcoin-f51237d94d98-x86_64-linux-gnu-debug.tar.gz
  bfdb827e75d43d61462513c9a843620b93c9160d9d246cad13278baaa07f64ea  bitcoin-f51237d94d98-x86_64-linux-gnu.tar.gz
  4185adebc6a0abe7241a3cd409a6ab7be031c26f1c4245e30bb5f87eef0925d2  src/bitcoin-f51237d94d98.tar.gz
  34820a093916fa35b0fd98806a50092f46b20271af7422f43e2a4223ef6f9bb7  bitcoin-core-linux-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    re-ACK 7fc5e865b9

Tree-SHA512: 0c30838413448ecfcf55e6273f607fdb01cb1acafa1d2762afad59360fca7d8efa78ec55064f50cba56cb2c9e98741e13665cba8e9b4b8e5b62b8a53f9bf8990
2021-05-05 13:22:59 +02:00
W. J. van der Laan
00a9b0647e Merge bitcoin/bitcoin#21799: guix: Use gcc-8 across the board
c90f6e5109 guix: Consistently use gcc-8 for $HOST (Carl Dong)

Pull request description:

  Only non-base commit is the last commit: b5abb07d0d

  Right now, here's what we use in Gitian:
  - Linux: Focal's [`g++-8-<arch>-linux-gnu`](https://packages.ubuntu.com/focal/g++-8-aarch64-linux-gnu) (`8.4.0-3ubuntu1cross1`)
  - MinGW-w64: Focal's [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64) (`9.3.0-7ubuntu1+22~exp1ubuntu4`)

  In Guix right now we use `gcc-9` across the board.

  I think it makes more sense to use `gcc-8` across the board, as it doesn't suffer from the `memcmp` bug, and is what debian buster (stable) does, meaning it will be well tested ([`g++-mingw-w64`](https://packages.debian.org/buster/g++-mingw-w64), [`g++-aarch64-linux-gnu`](https://packages.debian.org/buster/g++-aarch64-linux-gnu)).

  We can accomplish this somewhat easily using Guix as we have tighter control over the toolchain (see: b5abb07d0d).

  Let me know your thoughts!

ACKs for top commit:
  MarcoFalke:
    Approach ACK c90f6e5109, haven't reviewed
  laanwj:
    Code review ACK c90f6e5109
  hebasto:
    ACK c90f6e5109, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 3e5b9297305232273323aa745ec417ed1be2418ead0e432db7742f5d5f45efe6e4a2ed44328731512cff4bfde80e5f2dc350a131b8b8fb9207a2ef66bce27ed2
2021-05-05 10:43:09 +02:00
MarcoFalke
779aaa7f03 Merge bitcoin/bitcoin#21754: test: Run feature_cltv with MiniWallet
fa066f1b66 test: Run feature_cltv with MiniWallet (MarcoFalke)
fa5591d38c test: Hide tx rehash in helper (MarcoFalke)
fa5f938cfe test: Remove new_tx reference (MarcoFalke)

Pull request description:

  Allows to run the test even with no wallet compiled in

ACKs for top commit:
  theStack:
    ACK fa066f1b66 💽

Tree-SHA512: 3f659a178ba3ee0baffd70fddf8b8a68e5551d85626c7f254b234d7f75e6a16430a32a7952037db358b579f045b4d296b46156f72e5d226f3e80334dc635ca10
2021-05-05 08:47:28 +02:00
Jon Atack
73a91c63ec gui: rename "Peer Id" to "Peer" in tab column and details area
to allow resizing the column more tightly
2021-05-05 00:32:12 -04:00
fanquake
dc8da2a685 Merge bitcoin/bitcoin#21824: refactor: [index] Replace deprecated char with uint8_t in serialization
fafb880e88 refactor: [index] Replace deprecated char with uint8_t in serialization (MarcoFalke)

Pull request description:

  All char representations are serialized in the same way, however the `char` one is deprecated according to d22e7ee933/src/serialize.h (L227) . Also, using `uint8_t` directly avoids casts.

ACKs for top commit:
  jonatack:
    Approach ACK fafb880e88
  laanwj:
    Code review ACK fafb880e88
  practicalswift:
    cr ACK fafb880e88: patch looks correct

Tree-SHA512: ed08fb1b18cb75a695e15924bcaa30ff8746bcd5f17cc83e79f94fe5ff8d9f2083435cb49b8245e3341ede2512140940d864299f4746bc40c8ed8bfdbdacac24
2021-05-05 09:59:46 +08:00
MarcoFalke
3f8f238deb Merge bitcoin/bitcoin#21849: fuzz: Limit toxic test globals to their respective scope
cf83b82cf0 fuzz: Limit toxic test globals to their respective scope (MarcoFalke)

Pull request description:

  Globals in one fuzz target are toxic to all other fuzz targets, because we link all fuzz targets into one binary. Any code called by constructing the global will affect all other targets. This leads to incorrect coverage stats, false-positive crashes, ...

ACKs for top commit:
  practicalswift:
    cr ACK cf83b82cf0: non-toxic is better than toxic!
  laanwj:
    Code review ACK cf83b82cf0

Tree-SHA512: 5b3a37bcb36fce4160c94f877b2c07704527e3e1842092375c793d2eca77b996ae62889326094020855666bb34fa019fcfe92e8ff8430ce0372227f03ab2b907
2021-05-04 17:21:43 +02:00
MarcoFalke
a1c6434e19 Merge bitcoin/bitcoin#21852: ci: Add msan fuzz config
fa0422c251 ci: Add msan fuzz config (MarcoFalke)
fa399a76c6 ci: Use clang-12 in msan task (MarcoFalke)
fab30174af ci: Set BASE_SCRATCH_DIR early, so that it can be used in test configs (MarcoFalke)

Pull request description:

  Similar to the valgrind config, this config is not run by any ci task in this repo, but it can be used by other repos or self-hosted infrastructure.

ACKs for top commit:
  practicalswift:
    cr ACK fa0422c251: patch looks correct

Tree-SHA512: 2122ac0948978a7b952efc80d4aa3674b27d48c6166e0ce917c61ac4ee6b68d701a83e5f71ee6868c208885ee45aae409ca022ebcb23ccbe37819a8c36e34872
2021-05-04 17:04:49 +02:00
MarcoFalke
fa0422c251 ci: Add msan fuzz config 2021-05-04 15:06:00 +02:00
MarcoFalke
fa399a76c6 ci: Use clang-12 in msan task 2021-05-04 15:04:11 +02:00
MarcoFalke
fab30174af ci: Set BASE_SCRATCH_DIR early, so that it can be used in test configs
Can be reviewed with --color-moved=dimmed-zebra
2021-05-04 15:03:14 +02:00
fanquake
7fc5e865b9 test: install lief in CI 2021-05-04 20:48:53 +08:00
fanquake
955140b326 contrib: consolidate PIE and NX security checks 2021-05-04 20:48:00 +08:00
fanquake
2aa1631822 contrib: use LIEF in PE symbol checks 2021-05-04 20:48:00 +08:00
fanquake
e93ac26b85 contrib: use LIEF in macOS symbol checks 2021-05-04 20:48:00 +08:00
fanquake
a632cbcee5 contrib: use f strings in symbol-check.py 2021-05-04 20:48:00 +08:00
fanquake
0f5d77c8e4 contrib: add PE PIE check to security checks 2021-05-04 20:48:00 +08:00
fanquake
8e1f40dd9a contrib: use LIEF for PE security checks 2021-05-04 20:47:59 +08:00
fanquake
a25b2e965c contrib: use LIEF for macOS security checks 2021-05-04 20:47:59 +08:00
fanquake
7e7eae7aa8 contrib: use f strings in security-check.py 2021-05-04 20:47:59 +08:00
fanquake
2e7a9f7ade guix: install LIEF in Guix container
Co-authored-by: Carl Dong <contact@carldong.me>
2021-05-04 20:47:59 +08:00
fanquake
465967b5ef gitian: install LIEF in gitian container 2021-05-04 20:47:58 +08:00
fanquake
0ca8b7e7ec Merge bitcoin/bitcoin#21812: ci: Enable D_GLIBCXX_DEBUG for multiprocess task
fa44f5119a ci: Clarify that previous_releases task is using DEBUG (MarcoFalke)
fad0f21c3c ci: Use clang in multiprocess task to avoid OOM (MarcoFalke)
faeabef4f3 ci: Enable D_GLIBCXX_DEBUG for multiprocess task (MarcoFalke)

Pull request description:

  Enable `-D_GLIBCXX_DEBUG` via the depends `DEBUG` flag. Also `--enable-debug` to get debug symbols in traces.

ACKs for top commit:
  hebasto:
    ACK fa44f5119a, I have reviewed the code and it looks OK, I agree it can be merged, and CI is green.

Tree-SHA512: ab2a216bb44ee462f9dd181ec9025962502bd4201a1118ff52b6a193398e7ea3ca465a45a5eb341e308758fc3ef34ea3521f8a1f85ed64478ef3c1f6c1b8b141
2021-05-04 19:18:35 +08:00
John Newbery
39e19713cd [net processing] Add internal _RelayTransactions()
Callers of the external RelayTransactions() no longer need to lock cs_main.
2021-05-04 09:31:03 +01:00
W. J. van der Laan
ab9a566ab3 Merge bitcoin/bitcoin#21825: net: add I2P hardcoded seeds
142e2da440 net: add I2P seeds to chainparamsseeds (Jon Atack)
e01f173fb9 contrib: add a few I2P seed nodes (Jon Atack)
ea269c7ef1 contrib: parse I2P addresses in generate-seeds.py (Jon Atack)

Pull request description:

  Follow-up to #21560 that updated the fixed seeds infra for BIP155 addresses and then added Tor v3 ones:

  - Update contrib/generate-seeds.py to parse I2P addresses

  - Add a few I2P nodes to contrib/seeds/nodes_main.txt

  - Run generate-seeds.py and add the I2P seeds to chainparamsseeds.h

  Reviewers, see contrib/seeds/README.md for more info and feel free to use the following CLI one-liner to check for and propose additional seeds for contrib/seeds/nodes_main.txt. You can also see how many I2P peers your node knows with cli -addrinfo.

  ```rake
  bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".b32.i2p"))) | .address' | sort
  ```

  I verified the I2P addresses are correctly BIP155-serialized/deserialized by building with all seeds removed from chainparamsseeds.h except those added here, restarting with `-datadir=newdir -dnsseed=0` and running rpc ` getnodeaddresses 0` that initially returns only the new I2P addresses.

ACKs for top commit:
  laanwj:
    ACK 142e2da440
  vasild:
    ACK 142e2da440

Tree-SHA512: 040576012d5f1f034e2bd566ad654a6fdfd8ff7f6b12fa40c9fda1e948ebf8417fcea64cfc14938a41439370aa4669bab3e97274f9d4f9a6906fa9520afa9cf8
2021-05-04 09:44:46 +02:00
MarcoFalke
cf83b82cf0 fuzz: Limit toxic test globals to their respective scope 2021-05-04 09:24:17 +02:00
MarcoFalke
fac30eec42 refactor: Replace &foo[0] with foo.data() 2021-05-04 06:55:31 +02:00
MarcoFalke
faece47c47 refactor: Avoid &foo[0] on C-Style arrays
This is confusing at best when parts of a class use the
redundant operators and other parts do not.
2021-05-04 06:55:30 +02:00
MarcoFalke
face961109 refactor: Use only one temporary buffer in CreateObfuscateKey 2021-05-04 06:53:37 +02:00
MarcoFalke
fa05dddc42 refactor: Use CPubKey vector constructor where possible 2021-05-04 06:53:32 +02:00
Guido Vranken
fabb6dfe6e script: Replace address-of idiom with vector data() method 2021-05-04 06:53:20 +02:00
MarcoFalke
e2d4e67a8f Merge bitcoin/bitcoin#21840: test: Misc refactor to get rid of &foo[0] raw pointers
fa8a88849c bench: Remove duplicate constants (MarcoFalke)
000098f964 test: Use throwing variant accessor (MarcoFalke)
fa2197c8b3 test: Use loop to register RPCs (MarcoFalke)

Pull request description:

  Simplify test code

ACKs for top commit:
  Empact:
    Code Review ACK fa8a88849c
  practicalswift:
    cr ACK fa8a88849c
  promag:
    Code review ACK fa8a88849c.

Tree-SHA512: 6a5bebaa9a3f43e9c332f4fbff606e9ece6dc8b95a769980082cc022f8e9bde6083c1e4a0145dcbf3741f514d6e97b4198f201a1bf1370ebf43bd3a5c0f85981
2021-05-04 06:49:22 +02:00
Carl Dong
d420e5c1c0 guix-attest: Avoid incomplete sigdirs with ERR traps
Sometimes GPG connects to the wrong agent... or you don't have your
smartcard handy...
2021-05-03 16:50:59 -04:00
MarcoFalke
bf5e6a7771 Merge bitcoin/bitcoin#21846: fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer (generateblock)
575792e6ff fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer (practicalswift)

Pull request description:

  Add `-fsanitize=integer` suppression needed for RPC fuzzer (`generateblock`).

  Context: https://github.com/bitcoin-core/qa-assets/pull/59/checks?check_run_id=2494624259

  ```
  miner.cpp:130:21: runtime error: implicit conversion from type 'int64_t' (aka 'long') of value 244763573890 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 4245405314 (32-bit, unsigned)
      #0 0x56143974eaf3 in BlockAssembler::CreateNewBlock(CScript const&) miner.cpp:130:21
      #1 0x56143993690d in generateblock()::$_4::operator()(RPCHelpMan const&, JSONRPCRequest const&) const rpc/mining.cpp:370:127
  ```

ACKs for top commit:
  practicalswift:
    > review ACK [575792e](575792e6ff), but this function shouldn't be called by the rpc fuzzer, at least not without sanitizing num_blocks
  MarcoFalke:
    review ACK 575792e6ff

Tree-SHA512: c2133d1064bf17df0e7749ef4a0f7664b5c8082040491a1035d39f0c6e5d96997b347ef2354411e285c7f1f973e34515f1c3c88eb3de60fab64ca4d2adf6dd74
2021-05-03 22:18:05 +02:00
practicalswift
575792e6ff fuzz: Add -fsanitize=integer suppression needed for RPC fuzzer 2021-05-03 19:33:46 +00:00
Carl Dong
c90f6e5109 guix: Consistently use gcc-8 for $HOST 2021-05-03 14:42:25 -04:00
MarcoFalke
8e1c06846e Merge bitcoin/bitcoin#21662: build: update Boost download URL
36c10b9f4b build,boost: update download url. (fdov)

Pull request description:

  - bintray is closing.
      - updated to jfrog.io.

  For reference:
  https://github.com/boostorg/boost/issues/502

ACKs for top commit:
  Sjors:
    ACK 36c10b9
  hebasto:
    ACK 36c10b9f4b

Tree-SHA512: 6746781296c3f395a2824750cf1a578ef9a2d6b8d5c4da196a8da5d084aea01f7f2583b41b2aade7bb94dd02be71efcfd6edf7491a1a25f2d36240b18322a68e
2021-05-03 19:57:43 +02:00
MarcoFalke
ea71726a54 Merge bitcoin/bitcoin#21810: fuzz: Various RPC fuzzer follow-ups
5252f86eb6 fuzz: Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of #ifdef forests (practicalswift)
54549dda31 fuzz: RPC fuzzer post-merge follow-ups. Remove unused includes. Update list of fuzzed RPC commands. (practicalswift)

Pull request description:

  Various RPC fuzzer follow-ups:
  * Remove unused includes.
  * Update list of fuzzed RPC commands.
  * Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of `#ifdef` forests.

  Context: https://github.com/bitcoin/bitcoin/pull/21169#pullrequestreview-646723483

ACKs for top commit:
  MarcoFalke:
    Concept ACK 5252f86eb6

Tree-SHA512: 286d70798131706ffb157758e1c73f7f00ed96ce120c7d9dc849e672b283f1362df47b206cfec9da44d5debb5869225e721761dcd5c38a7d5d1019dc6c912ab2
2021-05-03 19:47:18 +02:00
Carl Dong
feda2c8e31 guix: Skip attesting to dist-archive
We already attest to the relevant dist-archive in inputs.SHA256SUMS,
which is recorded at build-time.

We use a SKIPATTEST.TAG file to indicate output directories which do not
require attestation (much like the CACHEDIR.TAG specification).
Generally, it's better to have build scripts declare properties of
directories instead of introducing name-based special cases in attest
scripts since build scripts have a more detailed context of what is
going on.
2021-05-03 13:18:19 -04:00
Carl Dong
d522d8006b guix: Attest to inputs in inputs.SHA256SUMS
At build/codesigning-time, hash build inputs and output the digest to
${OUTDIR}/inputs.SHA256SUMS, which gets included in the final SHA256SUMS
constructed by guix-attest.

Example final SHA256SUMS:
ee832d2a35b7701bff581dea05a536118b118e3ad0a587a2855b6ee8cd6fba20  inputs/bitcoin-78199266af7b.tar.gz
ca765e70a0c12866dd63c0be228b675278a26329e5f8f5b5c52fd09200fedf21  bitcoin-78199266af7b-powerpc64le-linux-gnu-debug.tar.gz
dae95327d7f2c324e2728c4b73627be6cb2c0d2f2e5bea940d1d5e6463939327  bitcoin-78199266af7b-powerpc64le-linux-gnu.tar.gz
2021-05-03 13:18:19 -04:00
Carl Dong
f9e2960c01 guix: Construct $OUTDIR in ${DISTSRC}/output
While files are being output to $OUTDIR, it will be under
${DISTSRC}/output, and only when everything is done, will
${DISTSRC}/output be moved to the actual $OUTDIR.

This makes it so that a Ctrl-C in the middle of a build is less likely
to result in a partially-constructed $OUTDIR. In fact, if I understand
correctly, if $OUTDIR and $DISTSRC reside on the same filesystem, the
move (rename) is likely atomic.

Also, since the "working $OUTDIR" is under ${DISTSRC}/output, it will be
cleaned properly by the guix-clean script.
2021-05-03 13:18:19 -04:00
Carl Dong
022abc85fc guix: Minor quoting fix in libexec/build.sh 2021-05-03 13:18:19 -04:00
Carl Dong
c83c4fa5b7 guix-attest: Allow skipping GPG signing with NO_SIGN 2021-05-03 13:18:19 -04:00
Carl Dong
0e1c2e448c guix-attest: Use ascii-armor signatures 2021-05-03 13:18:19 -04:00
Carl Dong
b5fd89c4c8 guix-attest: Only use cross-platform flags for find+xargs 2021-05-03 13:18:19 -04:00
Carl Dong
5926432ba6 guix: Add guix-verify script 2021-05-03 13:18:19 -04:00
Carl Dong
30daf76a97 guix: Add guix-attest script 2021-05-03 13:18:19 -04:00
MarcoFalke
3692097d92 Merge bitcoin/bitcoin#21823: script: update REVIEWERS
48bd9ebc5d script: update REVIEWERS (Adam Jonas)

Pull request description:

  Meta: `git show a06eb03` indicates the commit was first made one year ago and the PR was merged in September 2020.

  It might be time for an update, if automated review requests via DrahtBot are operational.

  "Regular contributors are free to add their names to specific directories or files provided that they are willing to provide a review."

  Perhaps we can compile and squash suggested updates here, per that guideline.

ACKs for top commit:
  practicalswift:
    ACK 48bd9ebc5d: patch looks correct :)
  adamjonas:
    ACK 48bd9eb. I ran through the new files added with `*`s using `git ls-files`.
  hebasto:
    ACK 48bd9ebc5d, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 770fd60d78bd64100814c72211e4bd97718c87a55a4c91e136fababa0149d12cdc1b3e89462dee3a55e5f51d3263420cc6d856868469eb9a72ff8c930859c54f
2021-05-03 19:11:40 +02:00
Adam Jonas
48bd9ebc5d script: update REVIEWERS 2021-05-03 13:16:43 +02:00
MarcoFalke
fa8a88849c bench: Remove duplicate constants 2021-05-03 11:50:30 +02:00
MarcoFalke
000098f964 test: Use throwing variant accessor
It does not matter if the tests fail due to a BOOST_CHECK failure or
due to a thrown exception. Prefer the exception because it is less
code.

Example fail with the throwing accessor:

unknown location(0): fatal error: in "script_standard_tests/script_standard_ExtractDestinations": std::bad_variant_access: std::get: wrong index for variant
test/script_standard_tests.cpp(314): last checkpoint

*** 1 failure is detected in the test module "Bitcoin Core Test Suite"
2021-05-03 11:50:29 +02:00
MarcoFalke
fa2197c8b3 test: Use loop to register RPCs
The same loop is used by the server, so no need for
the tests to do this differently.
2021-05-03 11:50:05 +02:00
MarcoFalke
b8593616dc Merge bitcoin/bitcoin#21775: p2p: Limit m_block_inv_mutex
fac96d0265 p2p: Limit m_block_inv_mutex (MarcoFalke)

Pull request description:

  Keeping the lock longer than needed is confusing to reviewers and thread analysis. For example, keeping the lock while appending tx-invs, which requires the mempool lock, will tell thread analysis tools an incorrect lock order of `(1) m_block_inv_mutex, (2) pool.cs`.

ACKs for top commit:
  Crypt-iQ:
    crACK fac96d0265
  jnewbery:
    utACK fac96d0265
  theStack:
    Code-Review ACK fac96d0265

Tree-SHA512: fcfac0f1f8b16df7522513abf716b2eed3d2fc9153f231c8cb61f451e342f29c984a5c872deca6bab3e601e5d651874cc229146c9370e46811b4520747a21f2b
2021-05-03 11:13:41 +02:00
MarcoFalke
320e518b90 Merge bitcoin/bitcoin#21750: net: remove unnecessary check of CNode::cs_vSend
9096b13a47 net: remove unnecessary check of CNode::cs_vSend (Vasil Dimov)

Pull request description:

  It is not possible to have a node in `CConnman::vNodesDisconnected` and
  its reference count to be incremented - all `CNode::AddRef()` are done
  either before the node is added to `CConnman::vNodes` or while holding
  `CConnman::cs_vNodes` and the object being in `CConnman::vNodes`.

  So, the object being in `CConnman::vNodesDisconnected` and its reference
  count being zero means that it is not and will not start to be used by
  other threads.

  So, the lock of `CNode::cs_vSend` in `CConnman::DisconnectNodes()` will
  always succeed and is not necessary.

  Indeed all locks of `CNode::cs_vSend` are done either when the reference
  count is >0 or under the protection of `CConnman::cs_vNodes` and the
  node being in `CConnman::vNodes`.

ACKs for top commit:
  MarcoFalke:
    review ACK 9096b13a47 🏧
  jnewbery:
    utACK 9096b13a47

Tree-SHA512: 910899cdcdc8934642eb0c40fcece8c3b01b7e20a0b023966b9d6972db6a885cb3a9a04e9562bae14d5833967e45e2ecb3687b94d495060c3da4b1f2afb0ac8f
2021-05-03 08:13:53 +02:00
fanquake
77d569ccb5 Merge bitcoin/bitcoin#19817: build: macOS toolchain bump
a5550f877a build: use -stdlib++-isystem with Clang 10 (fanquake)
51d9d1607f guix: use Clang 10 for the macOS cross compile (fanquake)
b80a6af9e5 build: no longer patch threading out of ld64 (fanquake)
c29cba44b3 build: Xcode 12.1, macOS SDK 10.15.6 (fanquake)
9ed2f19d38 build: native cctools 973.0.1, ld64 609 (fanquake)
f48f187cce build: Clang 10.0.1 (Hennadii Stepanov)
9b193cd2a3 build: libtapi 1100.0.11 (fanquake)

Pull request description:

  Bumps our macOS toolchain to be using the following:
  * Clang 10.0.1 (gitian) & Clang 10.0.0 (Guix)
  * ld64 609
  * libtapi 1100.0.11
  * cctools  973.0.1
  * Xcode 12.1
  * macOS SDK 10.15.6

  which are currently the most recent releases available as open source. See upstream [`cctools`](https://github.com/tpoechtrager/cctools-port/commits/973.0.1-ld64-609) and [`libtapi`](https://github.com/tpoechtrager/apple-libtapi/tree/1100.0.11).

  This should improve the possibility of Apple ARM cross-compilation in depends.

  This also removes our [patching out of pthreads usage](https://github.com/bitcoin/bitcoin/blob/master/depends/patches/native_cctools/ld64_disable_threading.patch) in `ld64`. There have been multiple changes since `ld64 450.3`, which have likely fixed the non-determinism we were working around. i.e from [InputFiles.cpp](https://opensource.apple.com/source/ld64/ld64-609/src/ld/InputFiles.cpp.auto.html):
  ```cpp
  // <rdar://problem/15002251> make implicit dylib order be deterministic by sorting by install_name
  std::sort(implicitDylibs.begin(), implicitDylibs.end(), DylibByInstallNameSorter());
  ```

  ```cpp
  // <rdar://problem/42675402> ld64 output is not deterministic due to dylib processing order
  std::sort(unprocessedDylibs.begin(), unprocessedDylibs.end(), [](const ld::dylib::File* lhs, const ld::dylib::File* rhs) {
  return strcmp(lhs->path(), rhs->path()) < 0;
  });
  ```

  Guix Build:
  ```bash
  find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  f6c3817b8fe5f7370299d1ae2533e4a3acd313ba9f9aa8d423a8956117e52dd5  guix-build-a5550f877a2c/output/dist-archive/bitcoin-a5550f877a2c.tar.gz
  4954dcf563c2d496b8d9fecd48f8e3f7fba2f319ffa254a5bc8ee12cfee6acf0  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx-unsigned.dmg
  8f6095b445c7f1a8e6accd86bb7f0696d5849402084927d2b726b7d557831c3a  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx-unsigned.tar.gz
  cc40f25477b4defc1617ae694313d80f307ddf6742fe6cc85c6bc0e215ef8be0  guix-build-a5550f877a2c/output/x86_64-apple-darwin18/bitcoin-a5550f877a2c-osx64.tar.gz
  ```

  Gitian Build:
  ```bash
  Generating report
  506a8abdefe559999b43dd9f14905b9b2b5a3363b1cd013d45ae47acc1f7ef6c  bitcoin-a5550f877a2c-osx-unsigned.dmg
  f606997f74026dd12d110d683c6f116b40df324836904ef507dd7ac787e6ebe2  bitcoin-a5550f877a2c-osx-unsigned.tar.gz
  5b495ef15f2c3260c2950921b61326912a9bf533cccd51e13818809fd225489e  bitcoin-a5550f877a2c-osx64.tar.gz
  f6c3817b8fe5f7370299d1ae2533e4a3acd313ba9f9aa8d423a8956117e52dd5  src/bitcoin-a5550f877a2c.tar.gz
  9eb0221e962d2839770963bd03c6c9e98e8bf3078566bee2ae42f06233a710fa  bitcoin-core-osx-22-res.yml
  Done.
  ```

ACKs for top commit:
  hebasto:
    ACK a5550f877a

Tree-SHA512: 504c4b0f9cd3b939714a322298320c5bde07e9356a48a9a000060b36f8dce4d6134ed60c3a5188810476a28ec5b108733eabbc6fb8053231b9ea8a494cc91b12
2021-05-03 14:07:58 +08:00
fanquake
60132382a7 Merge bitcoin/bitcoin#20867: Support up to 20 keys for multisig under Segwit context
ebd4be43cc doc: add release notes for 20867 (Antoine Poinsot)
5aa50ab9cc rpc/util: multisig: only check redeemScript size is <= 520 for P2SH (Antoine Poinsot)
063df9e897 test/functional: standardness sanity checks for P2(W)SH multisig (Antoine Poinsot)
ae0429d3af script: allow up to 20 keys in wsh() descriptors (Antoine Poinsot)
9fc68faf35 script: match multisigs with up to MAX_PUBKEYS_PER_MULTISIG keys (Antoine Poinsot)

Pull request description:

  As described in https://github.com/bitcoin/bitcoin/issues/20620 multisigs are currently limited to 16 keys in descriptors and RPC helpers, even for P2WSH and P2SH-P2WSH.

  This adds support for multisig with up to 20 keys (which are already standard) for Segwit v0 context for descriptors (`wsh()`, `sh(wsh())`) and RPC helpers.

  Fixes https://github.com/bitcoin/bitcoin/issues/20620

ACKs for top commit:
  meshcollider:
    re-utACK ebd4be43cc
  instagibbs:
    re-ACK ebd4be43cc

Tree-SHA512: 36141f10a8288010d17d5c4fe8d24878bcd4533b88a8aba3a44fa8f74ceb3182d70fee01427e0ab7f53ce7fab46c88c1cd3ac3b18ab8a10bd4a6b8b74ed79e46
2021-05-03 12:44:23 +08:00
Hennadii Stepanov
d66f283ac0 scripted-diff: Replace three dots with ellipsis in the UI strings
-BEGIN VERIFY SCRIPT-
sed -i -E -e 's/\.\.\."\)(\.|,|\)| )/…"\)\1/' -- $(git ls-files -- 'src' ':(exclude)src/qt/bitcoinstrings.cpp')
sed -i -e 's/\.\.\.\\"/…\\"/' src/qt/sendcoinsdialog.cpp
sed -i -e 's|\.\.\.</string>|…</string>|' src/qt/forms/*.ui
sed -i -e 's|\.\.\.)</string>|…)</string>|' src/qt/forms/sendcoinsdialog.ui
-END VERIFY SCRIPT-
2021-05-02 22:17:16 +03:00
Jon Atack
142e2da440 net: add I2P seeds to chainparamsseeds 2021-05-02 13:37:50 +02:00
Jon Atack
e01f173fb9 contrib: add a few I2P seed nodes 2021-05-02 13:37:48 +02:00
Jon Atack
ea269c7ef1 contrib: parse I2P addresses in generate-seeds.py 2021-05-02 13:37:41 +02:00
W. J. van der Laan
2448457cca Merge #21830: doc: Add doc/release-notes/release-notes-0.21.1.md
fab53ea72f doc: Add doc/release-notes/release-notes-0.21.1.md (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK fab53ea72f

Tree-SHA512: f228c51c5fd4d18badc72ca185b91a05b248614a69646b234e03963626eccbafcf23c8057bbaa9ae3a3fc4e7d8f007fd69306fc87da229c0c44847de8be42aad
2021-05-02 10:36:46 +02:00
MarcoFalke
fab53ea72f doc: Add doc/release-notes/release-notes-0.21.1.md 2021-05-02 09:59:14 +02:00
fanquake
59869704c0 Merge bitcoin/bitcoin#21818: doc: fixup -coinstatsindex help, update bitcoin.conf and files.md
54133c59b8 doc: add indexes/coinstats/db/ to files.md (Jon Atack)
5d1050f516 doc: fix -coinstatsindex help, and test/rpc touchups (Jon Atack)
e041ee0a80 doc: add coinstatsindex to bitcoin.conf (Jon Atack)

Pull request description:

ACKs for top commit:
  Sjors:
    utACK 54133c59b8
  MarcoFalke:
    cr ACK 54133c59b8
  clarkmoody:
    utACK 54133c5

Tree-SHA512: 1a7f3e89873b7dc79ec71d5d39e9e3e4977ce43cc4bee208ad55291bef1bb319a9d1c34ed84a87d6a803db983bdfd0af4d9f396cec0bec86b1701ebbb6f34378
2021-05-02 13:50:38 +08:00
fanquake
f2865b7394 Merge bitcoin/bitcoin#21811: doc: Remove Visual Studio 2017 reference from readme
0a331456e4 Remove Visual Studio 2017 reference from readme (Aaron Clauson)

Pull request description:

  This PR was motivated by a comment in GUI PR (257) regarding a suggested improvement not being supported by VS2017.

  When checking whether master can still be built with the VS2017 toolset ABI issues were encountered. Most likely due to the pre-compiled Qt binaries that are used.

  It does not seem worth the effort to try and support VS2017, which would most likely require additional Qt binaries, or lengthy instructions on how to build static Qt binaries on Windows (which is very error prone and tedious).

ACKs for top commit:
  jarolrod:
    ACK 0a331456e4
  hebasto:
    ACK 0a331456e4, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 17e2326cd55a5cc3fc13622ba72bb44f9b9d55070cb77941df5fd943cd3f1baf02b9ac9504dfae5941c522748ad7b584c3c8da03fb323a87b3639eb926ce1699
2021-05-02 13:04:02 +08:00
unknown
3bad0b3fad Remove user input from URI error message
+ Detailed error messages for invalid address
+ Used `IsValidDestination` instead of `IsValidDestinationString`
+ Referred to https://github.com/bitcoin/bitcoin/pull/20832 for solution
2021-05-02 07:47:19 +05:30
MarcoFalke
fae196147b doc: Clarify that feerates are per virtual size 2021-05-01 18:21:05 +02:00
Aaron Clauson
0a331456e4 Remove Visual Studio 2017 reference from readme
This PR was motivated by a comment in GUI PR (257) regarding a suggested improvement not being supported by VS2017.

When checking whether master can still be built with the VS2017 toolset ABI issues were encountered. Most likely due to the pre-compiled Qt binaries that are used.

It does not seem worth the effort to try and support VS2017, which would most likely require additional Qt binaries, or lengthy instructions on how to build static Qt binaries on Windows (which is very error prone and tedious).

Added advisory note about build not working with earlier Visual Studio versions.

Fixed grammar.
2021-05-01 17:15:38 +01:00
Hennadii Stepanov
c231254a65 qt: Make TransactionView aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:12 +03:00
Hennadii Stepanov
2b622d4ace qt: Make CoinControlDialog aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:11 +03:00
Hennadii Stepanov
97a6b5e06a qt: Make OverviewPage aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:11 +03:00
Hennadii Stepanov
d05f1b278d qt: Make UnitDisplayStatusBarControl aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:11 +03:00
Hennadii Stepanov
6b2ce65392 qt: Replace base class of ClickableLabel with ThemedLabel
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:10 +03:00
Hennadii Stepanov
ff530a2093 qt: Use GUIUtil::ThemedLabel class
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:10 +03:00
Hennadii Stepanov
d99ef327a8 qt: Add GUIUtil::ThemedLabel class
The ThemedLabel class correctly handles appearance changes on macOS.
2021-05-01 15:07:10 +03:00
Hennadii Stepanov
c054720e08 qt: Make SignVerifyMessageDialog aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:09 +03:00
Hennadii Stepanov
0dcc3fac43 qt: Make SendCoinsEntry aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:09 +03:00
Hennadii Stepanov
fa18d28e12 qt: Make RPCConsole aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:09 +03:00
Hennadii Stepanov
f1083826e3 qt: Make BitcoinGUI aware of runtime palette change
This change fixes the GUI when changing appearance on macOS.
2021-05-01 15:07:08 +03:00
Hennadii Stepanov
ce17861dc4 qt: Make PlatformStyle aware of runtime palette change
This change is a prerequisite to support changeable appearance on macOS.
2021-05-01 15:07:02 +03:00
Jon Atack
54133c59b8 doc: add indexes/coinstats/db/ to files.md 2021-05-01 13:57:52 +02:00
Jon Atack
5d1050f516 doc: fix -coinstatsindex help, and test/rpc touchups 2021-05-01 13:57:26 +02:00
MarcoFalke
fafb880e88 refactor: [index] Replace deprecated char with uint8_t in serialization 2021-05-01 13:48:42 +02:00
MarcoFalke
d22e7ee933 Merge bitcoin/bitcoin#21822: test: resolve bug in interface_bitcoin_cli.py
c5bb142817 test: resolve bug in test/functional/interface_bitcoin_cli.py - Test -getinfo with -rpcwallet=unloaded wallet returns no balances (klementtan)

Pull request description:

  I think there is a bug in this test case where the new value of `cli_get_info` is not asserted.

ACKs for top commit:
  jonatack:
    ACK c5bb142817

Tree-SHA512: 50c0c2c8fe63c95f951dee892fbacedf92208f47efe5ed481fbb255f15137c799d9200fa3ff31a442df0691248d7ff04d899842722c3032cd7f35553622ba38c
2021-05-01 13:12:37 +02:00
klementtan
c5bb142817 test: resolve bug in test/functional/interface_bitcoin_cli.py - Test -getinfo with -rpcwallet=unloaded wallet returns no balances 2021-05-01 17:58:01 +08:00
MarcoFalke
fa44f5119a ci: Clarify that previous_releases task is using DEBUG 2021-05-01 11:17:59 +02:00
MarcoFalke
fad0f21c3c ci: Use clang in multiprocess task to avoid OOM 2021-05-01 11:17:35 +02:00
MarcoFalke
fa80a11c3b test: Add missing test for empty P2WSH redeem 2021-05-01 10:12:11 +02:00
MarcoFalke
fa83e95ac6 scripted-diff: Clarify that feerates are per virtual size
-BEGIN VERIFY SCRIPT-
sed -i 's|/kB|/kvB|g' $( git grep -l '/kB' ./src )
-END VERIFY SCRIPT-
2021-05-01 09:42:50 +02:00
fanquake
a5550f877a build: use -stdlib++-isystem with Clang 10 2021-05-01 13:40:50 +08:00
fanquake
51d9d1607f guix: use Clang 10 for the macOS cross compile 2021-05-01 13:39:45 +08:00
fanquake
b80a6af9e5 build: no longer patch threading out of ld64
Changes introduced in ld64-450.3 have likely removed the need for us to
patch out pthreads. See:
https://opensource.apple.com/source/ld64/ld64-450.3/src/ld/InputFiles.cpp.auto.html.
2021-05-01 13:39:45 +08:00
fanquake
c29cba44b3 build: Xcode 12.1, macOS SDK 10.15.6 2021-05-01 13:39:45 +08:00
fanquake
9ed2f19d38 build: native cctools 973.0.1, ld64 609 2021-05-01 13:39:44 +08:00
Hennadii Stepanov
f48f187cce build: Clang 10.0.1
LLVM 8 has inherent nondeterminism in the compiler, fixed in LLVM 9+.
2021-05-01 13:39:44 +08:00
fanquake
9b193cd2a3 build: libtapi 1100.0.11 2021-05-01 13:39:44 +08:00
fanquake
bb11a988fa Merge bitcoin/bitcoin#21793: build: use -isysroot over --sysroot on macOS
cf971c9ea0 build: use -isysroot over --sysroot on macOS (fanquake)

Pull request description:

  Not only does this seem to be the more correct behaviour when targeting Darwin, but if you use `-isysroot`, Clangs Darwin driver will [infer the deployment target](3e037f8f0e/clang/lib/Driver/ToolChains/Darwin.cpp (L1652)) from the SDK and use other SDK info when parsing arguments to the linker. In the case of [`-platform_version`](3e037f8f0e/clang/lib/Driver/ToolChains/Darwin.cpp (L2656)), which is added if the linker is [new enough](3e037f8f0e/clang/lib/Driver/ToolChains/Darwin.cpp (L342)), the version tuple is constructed from the SDKInfo, and SDKInfo, as far as I can tell, only exists when `-isysroot` has been passed, see [parseSDKSettings](3e037f8f0e/clang/lib/Driver/ToolChains/Darwin.cpp (L1785))

  As a result, the SDK version field in the `LC_BUILD_VERSION` command is filled out. i.e when building master:
  ```bash
        cmd LC_BUILD_VERSION
    cmdsize 32
   platform 1
      minos 10.14
        sdk n/a
     ntools 1
       tool 3
    version 650.9
  ```
  vs this PR:
  ```bash
        cmd LC_BUILD_VERSION
    cmdsize 32
   platform 1
      minos 10.14
        sdk 11.3
     ntools 1
       tool 3
    version 650.9
  ```

  This, from what I understand, will fix the issue we are having with Qt deciding wether or not to enable features like "Dark mode" on macOS, see #21771, however I have not tested that. Thus this is an alternative to #21782.

  Our usage of `--sysroot` was added in #17118.

  ```bash
  -isysroot <dir>         Set the system root directory (usually /)
  ```

ACKs for top commit:
  Sjors:
    tACK cf971c9ea0
  hebasto:
    re-ACK cf971c9ea0, only rebased and addressed comments since my [previous](https://github.com/bitcoin/bitcoin/pull/21793#pullrequestreview-647321518) review.

Tree-SHA512: f01138179fb85083b5505bbaa48810451098ffa4da5d3c9b673785448790aa76f2e64b2aab6e698f6ee378a21f70626445a3fabee7c61dbfc44e96f3e3964656
2021-05-01 13:38:43 +08:00
Hennadii Stepanov
13f24d135b Merge bitcoin-core/gui#298: Peertableview alternating row colors
e94920a0bb qt: peertableview alternating row colors (randymcmillan)

Pull request description:

  peers-tab: enable alternating row colors for peer table and banned table

ACKs for top commit:
  Bosch-0:
    tACK e94920a0bb on Windows 10 - works as intended. Before / after below:
  jarolrod:
    tACK e94920a0bb

Tree-SHA512: 05ba18e1db9700bbd68644fe02292409f4e5c52e301b1b2977c335d1ff16456a93fb0b15c8c8385d1b15f648141341990706d530f6b08ecb33098fa941b9af1f
2021-04-30 23:17:01 +03:00
Jon Atack
e041ee0a80 doc: add coinstatsindex to bitcoin.conf 2021-04-30 20:19:20 +02:00
t-bast
11d6459b6e rpc: include_unsafe option for fundrawtransaction
Allow RPC users to opt-in to unsafe inputs when funding a raw transaction.

Applications that need to manage a complex RBF flow (such as lightning
nodes using anchor outputs) are very limited if they can only use safe inputs.

Fixes #21299
2021-04-30 18:53:47 +02:00
W. J. van der Laan
2b45cf0bcd Merge bitcoin/bitcoin#19521: Coinstats Index
5f96d7d22d rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height (Fabian Jahr)
23fe50436b test: Add test for coinstatsindex behavior in reorgs (Fabian Jahr)
90c966b0f3 rpc: Allow gettxoutsetinfo and getblockstats for stale blocks (Fabian Jahr)
b9362392ae index, rpc: Add use_index option for gettxoutsetinfo (Fabian Jahr)
bb7788b121 test: Test coinstatsindex robustness across restarts (Fabian Jahr)
e0938c2909 test: Add tests for block_info in gettxoutsetinfo (Fabian Jahr)
2501576ecc rpc, index: Add verbose amounts tracking to Coinstats index (Fabian Jahr)
655d929836 test: add coinstatsindex getindexinfo coverage, improve current tests (Jon Atack)
ca01bb8d68 rpc: Add Coinstats index to getindexinfo (Fabian Jahr)
57a026c30f test: Add unit test for Coinstats index (Fabian Jahr)
6a4c0c09ab test: Add functional test for Coinstats index (Fabian Jahr)
3f166ecc12 rpc: gettxoutsetinfo can be requested for specific blockheights (Fabian Jahr)
3c914d58ff index: Coinstats index can be activated with command line flag (Fabian Jahr)
dd58a4de21 index: Add Coinstats index (Fabian Jahr)
a8a46c4b3c refactor: Simplify ApplyStats and ApplyHash (Fabian Jahr)
9c8a265fd2 refactor: Pass hash_type to CoinsStats in stats object (Fabian Jahr)
2e2648a902 crypto: Make MuHash Remove method efficient (Fabian Jahr)

Pull request description:

  This is part of the coinstats index project tracked in #18000

  While the review of the new UTXO set hash algorithm (MuHash) takes longer recently #19328 was merged which added the possibility to run `gettxoutsetinfo` with a specific hash type. As the first type it added `hash_type=none` which skips the hashing of the UTXO set altogether. This alone did not make `gettxoutsetinfo` much faster but it allows the use of an index for the remaining coin statistics even before a new hashing algorithm has been added. Credit to Sjors for the idea to take this intermediate step.

  Features summary:
  - Users can start their node with the option `-coinstatsindex` which syncs the index in the background
  - After the index is synced the user can  use `gettxoutsetinfo` with `hash_type=none` or `hash_type=muhash` and will get the response instantly out of the index
  - The user can specify a height or block hash when calling `gettxoutsetinfo` to see coin statistics at a specific block height

ACKs for top commit:
  Sjors:
    re-tACK 5f96d7d22d
  jonatack:
    Code review re-ACK 5f96d7d22d per `git range-diff 13d27b4 07201d3 5f96d7d`
  promag:
    Tested ACK 5f96d7d22d. Light code review ACK 5f96d7d22d.

Tree-SHA512: cbca78bee8e9605c19da4fbcd184625fb280200718396c694a56c7daab6f44ad23ca9fb5456d09f245d8b8d9659fdc2b3f3ce5e953c1c6cf4003dbc74c0463c2
2021-04-30 17:27:19 +02:00
John Newbery
0829516d1f [refactor] Remove unused ForEachNodeThen() template 2021-04-30 11:29:17 +01:00
John Newbery
09cc66c00e scripted-diff: rename address relay fields
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren fGetAddr     m_getaddr_sent
ren fSentAddr    m_getaddr_recvd
ren vAddrToSend  m_addrs_to_send
-END VERIFY SCRIPT-
2021-04-30 11:29:16 +01:00
John Newbery
76568a3351 [net processing] Move addr relay data and logic into net processing 2021-04-30 11:29:14 +01:00
MarcoFalke
fab1eb65b1 test: Fix feature_config_args.py intermittent issue 2021-04-30 10:49:49 +02:00
MarcoFalke
faeabef4f3 ci: Enable D_GLIBCXX_DEBUG for multiprocess task 2021-04-30 08:33:07 +02:00
MarcoFalke
480bf01c29 Merge bitcoin-core/gui#300: Remove progress bar on modal overlay
61fd8fe9a1 Remove progress bar on modal overlay (bruno)

Pull request description:

  This PR removes the progress bar (keeping only the percentage) on modal overlay

  resolves #279

  Before:
  ![1](https://user-images.githubusercontent.com/19480819/116625265-bde65000-a91f-11eb-93ee-72474fc8dd67.PNG)

  After:
  ![2](https://user-images.githubusercontent.com/19480819/116625272-c2126d80-a91f-11eb-80b7-839703f03f87.PNG)

ACKs for top commit:
  Bosch-0:
    tACK 61fd8fe9a1 on Windows 10. Unnecessary Progress bar no longer there :)
  jarolrod:
    tACK 61fd8fe9a1

Tree-SHA512: 96d72f168b26e950ce37e9f489bcbcc608473c44bce3be127ccd47d17b7642fa234d314596186ee16b430d943575c312d84133425507a17ae7ac58ecae986639
2021-04-30 07:51:53 +02:00
windsok
46b025e00d test: add new python linter to check file names and permissions
Replaces the existing tests in the test/lint/lint-filenames.sh and test/lint/lint-shebang.sh linter tests, as well as adding some new and increased testing.
Summary of tests:
- Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.
- Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing test/lint/lint-filenames.sh test)
- Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.
- Checks that for executable .py and .sh files, the shebang line used matches an allowable list of shebangs (This should replicate the existing test/lint/lint-shebang.sh test)
- Checks every file that contains a shebang line to ensure it has an executable permission

Fixes #21729
2021-04-29 17:00:58 -07:00
Hennadii Stepanov
cdbc2bd1f1 qt: Use template function qOverload in signal-slot connections
This commit does not change behavior.
2021-04-30 00:00:44 +03:00
Hennadii Stepanov
e1e1e708fa Merge bitcoin-core/gui#125: Enable changing the autoprune block space size in intro dialog
415fb2e1ab GUI/Intro: Move prune setting below explanation (Luke Dashjr)
2a84c6bcf6 GUI/Intro: Estimate max age of backups that can be restored with pruning (Luke Dashjr)
e2dcd957fa GUI/Intro: Rework UI flow to let the user set prune size in GBs (Luke Dashjr)
f2e5a6b54f GUI/Intro: Abstract GUI-to-option into Intro::getPrune (Luke Dashjr)
62932cc686 GUI/Intro: Return actual prune setting from showIfNeeded (Luke Dashjr)

Pull request description:

  ![Screenshot_20200911_095102](https://user-images.githubusercontent.com/1095675/92933661-0c4cea00-f436-11ea-9853-2456091ffab3.png)

  Moved from https://github.com/bitcoin/bitcoin/pull/18728

ACKs for top commit:
  ryanofsky:
    Code review ACK 415fb2e1ab. Changes since last review: mb/gib suffixes, constexpr QOverload expected_backup_days tweaks, new moveonly layout commit
  jarolrod:
    Tested ACK 415fb2e.
  Talkless:
    tACK 415fb2e1ab, tested on Debian Sid with Qt 5.15.2.
  hebasto:
    ACK 415fb2e1ab, my unresolved comments are not blockers, and they could be resolved in follow ups.

Tree-SHA512: bd4882a9c08e6a6eb14b7fb6366983db8581425b4949fea212785d34d8fad9e32fb81ca8c8cdbfb2c05ea394aaf5a746ba2cf16623795c7252c3bdb61d455f00
2021-04-29 23:47:11 +03:00
W. J. van der Laan
298fd02e4a Merge bitcoin/bitcoin#21045: build: adds switch to enable/disable randomized base address in MSVC builds
9bd3f35003 build: adds switch for disabling random base addresses in MSVC (Ethan Heilman)

Pull request description:

  In m4 builds we have the --disable-hardening switch that can be given in `./configure` to turn off randomized addresses. This PR provides a simple way of turning off randomized addresses in MSVC builds.

  This PR:
  * Adds this option the common-init project file so that it can be globally set across the project
  * Documents this switch in msvc build readme

  I have run the following test to verify this works

  I ran the msvc build with `<RandomizedBaseAddress>true</RandomizedBaseAddress>` then checked `bitcoind.exe` and `bitcoin-cli.exe` with `dumpbin.exe`:

  bitcoind
  ```
  > .\dumpbin.exe /headers src/bitcoind.exe
  Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
  ...
  OPTIONAL HEADER VALUES
               20B magic # (PE32+)
             14.28 linker version
            AE4600 size of code
            345C00 size of initialized data
                 0 size of uninitialized data
            6BED74 entry point (00000001406BED74) mainCRTStartup
              1000 base of code
         140000000 image base (0000000140000000 to 0000000140E2DFFF)
              1000 section alignment
               200 file alignment
              6.00 operating system version
              0.00 image version
              6.00 subsystem version
                 0 Win32 version
            E2E000 size of image
               400 size of headers
                 0 checksum
                 3 subsystem (Windows CUI)
              8160 DLL characteristics
                     High Entropy Virtual Addresses
                     Dynamic base
                     NX compatible
                     Terminal Server Aware
  ```

  bitcoin-cli
  ```
  > .\dumpbin.exe /headers src/bitcoin-cli.exe
  Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
  ...
  OPTIONAL HEADER VALUES
               20B magic # (PE32+)
             14.28 linker version
            1E3E00 size of code
             92C00 size of initialized data
                 0 size of uninitialized data
            104384 entry point (0000000140104384) mainCRTStartup
              1000 base of code
         140000000 image base (0000000140000000 to 0000000140279FFF)
              1000 section alignment
               200 file alignment
              6.00 operating system version
              0.00 image version
              6.00 subsystem version
                 0 Win32 version
            27A000 size of image
               400 size of headers
                 0 checksum
                 3 subsystem (Windows CUI)
              8160 DLL characteristics
                     High Entropy Virtual Addresses
                     Dynamic base
                     NX compatible
                     Terminal Server Aware
  ```

  Then I built with `<RandomizedBaseAddress>false</RandomizedBaseAddress>` then checked `bitcoind.exe` and `bitcoin-cli.exe` with `dumpbin.exe` and observed that `Dynamic base` was longer listed in `OPTIONAL HEADER VALUES`

  bitcoind
  ```
  PS C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.16.27023\bin\HostX64\x64> .\dumpbin.exe /headers C:\Users\e0\Documents\GitHub\bitcoin021noaslr/src/bitcoind.exe
  Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
  ...
  OPTIONAL HEADER VALUES
               20B magic # (PE32+)
             14.28 linker version
            AE4600 size of code
            33FE00 size of initialized data
                 0 size of uninitialized data
            6BED74 entry point (00000001406BED74) mainCRTStartup
              1000 base of code
         140000000 image base (0000000140000000 to 0000000140E27FFF)
              1000 section alignment
               200 file alignment
              6.00 operating system version
              0.00 image version
              6.00 subsystem version
                 0 Win32 version
            E28000 size of image
               400 size of headers
                 0 checksum
                 3 subsystem (Windows CUI)
              8120 DLL characteristics
                     High Entropy Virtual Addresses
                     NX compatible
                     Terminal Server Aware
  ```

  bitcoin-cli
  ```
  > .\dumpbin.exe /headers src/bitcoin-cli.exe
  Microsoft (R) COFF/PE Dumper Version 14.16.27045.0
  ...
  OPTIONAL HEADER VALUES
               20B magic # (PE32+)
             14.28 linker version
            1E3E00 size of code
             90C00 size of initialized data
                 0 size of uninitialized data
            104384 entry point (0000000140104384) mainCRTStartup
              1000 base of code
         140000000 image base (0000000140000000 to 0000000140277FFF)
              1000 section alignment
               200 file alignment
              6.00 operating system version
              0.00 image version
              6.00 subsystem version
                 0 Win32 version
            278000 size of image
               400 size of headers
                 0 checksum
                 3 subsystem (Windows CUI)
              8120 DLL characteristics
                     High Entropy Virtual Addresses
                     NX compatible
                     Terminal Server Aware
  ```

ACKs for top commit:
  sipsorcery:
    ACK 9bd3f35003.
  practicalswift:
    cr ACK 9bd3f35003: patch looks correct

Tree-SHA512: ddffdb4ff8a09c7cfef61c07a5db2a2828e9e3aa795ad8e5a1bf51ab489a68b40f87f6694518c5e0b8858c0fad4f93bb947b052e6b9d5e55eb38e764b746fc02
2021-04-29 22:14:34 +02:00
MarcoFalke
40310f5e8c Merge bitcoin/bitcoin#21792: test: Fix intermittent issue in p2p_segwit.py
fad6269916 test: Assert that exit code indicates failure (MarcoFalke)
faecb72c3c test: Fix intermittent issue in p2p_segwit.py (MarcoFalke)

Pull request description:

  Calling `start_node` might call `wait_for_rpc_connection`, which will fail.

  https://cirrus-ci.com/task/5669555591708672?logs=ci#L3504

  ```
    File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_segwit.py", line 1974, in test_upgrade_after_activation
      self.start_node(2, extra_args=["-reindex", f"-segwitheight={SEGWIT_HEIGHT}"])
    File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 508, in start_node
      node.wait_for_rpc_connection()
    File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 224, in wait_for_rpc_connection
      raise FailedToStartError(self._node_msg(
  test_framework.test_node.FailedToStartError: [node 2] bitcoind exited with status 1 during initialization

ACKs for top commit:
  jnewbery:
    ACK fad6269916
  dhruv:
    ACK fad6269

Tree-SHA512: 4c5e39ce25e135717ea433258518f93f09d1c528c4538a8627d3da13bc0c0ba4b45911703c26392ff0f5e0cb7831a6c7cc53e6e29102d3da9c8cfce7cef333cc
2021-04-29 20:44:24 +02:00
practicalswift
5252f86eb6 fuzz: Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of #ifdef forests 2021-04-29 18:40:25 +00:00
Hennadii Stepanov
792be53d3e refactor: Replace std::bind with lambdas
Lambdas are shorter and more readable.
Changes are limited to std::thread ctor calls only.
2021-04-29 18:39:31 +03:00
Hennadii Stepanov
a508f718f3 refactor: Use appropriate thread constructor 2021-04-29 18:39:01 +03:00
fanquake
d9ae6ec892 Merge bitcoin/bitcoin#21759: wallet: document coin selection code
6ba892126d refactor + document coin selection strategy (glozow)
58ea324fdd [docs] add doxygen comments to wallet code (glozow)
0c74716c50 [docs] format existing comments as doxygen (glozow)

Pull request description:

  I think it would help code review to have more documentation + doxygen comments

ACKs for top commit:
  Xekyo:
    ReACK 6ba892126d
  achow101:
    ACK 6ba892126d

Tree-SHA512: 74a78d9b0e0c1d5659bed566432a5b3511511d8b2432f440565f443da7b8257a1b90e70aa7505a7f8abf618748eeb43d166e84f278bdee3d34ce5d5c37dc573a
2021-04-29 22:18:28 +08:00
MarcoFalke
fad6269916 test: Assert that exit code indicates failure 2021-04-29 13:03:12 +02:00
MarcoFalke
faecb72c3c test: Fix intermittent issue in p2p_segwit.py 2021-04-29 13:02:33 +02:00
fanquake
16d0051c6a Merge bitcoin/bitcoin#21688: doc: note on SDK for macOS depends cross-compile
5e7d1997fc doc: note on SDK for macOS depends cross-compile (Jarol Rodriguez)

Pull request description:

  This PR adds a friendly note for those who are unfamiliar with our depends build system that an extracted SDK is needed before proceeding with a macOS cross-compile. Additionally, it refers a builder to look at [macdeploy](https://github.com/bitcoin/bitcoin/tree/master/contrib/macdeploy#sdk-extraction) for instructions on obtaining the SDK and provides context to where this SDK should reside.

  **Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md#for-macos-cross-compilation)
  **PR:** [render](241c4017ed/depends/README.md (for-macos-cross-compilation))

ACKs for top commit:
  jonatack:
    ACK  5e7d1997fc
  Sjors:
    ACK 5e7d199
  hebasto:
    re-ACK 5e7d1997fc

Tree-SHA512: bbdff4f6ea1dfae03bca7fd30dfba543c4d130960075195e42695875382f13b93ea02128f80686fa4d1b5c3bd3703ac600f8ba07356ea0caf0fec09b7dbd081e
2021-04-29 18:56:49 +08:00
MarcoFalke
a7a193789e Merge bitcoin/bitcoin#21804: [tests] Speed up p2p_segwit.py
a33bdb52d1 [tests] Speed up p2p_segwit.py (John Newbery)

Pull request description:

  Never sleep for more than 5 seconds when waiting for an
  inv-getdata exchange to time out.

  Shaves about 1 minute of the runtime of p2p_segwit.py.

ACKs for top commit:
  MarcoFalke:
    review ACK a33bdb52d1 🐳

Tree-SHA512: 7bd892ed0b1b817579f88910ba4714519bd0d871241e1b9a67968d297de1ed63d558115abad2aae4d105ff176c35a7079a3a789f3053442aed30d6e1aefb5c4a
2021-04-29 12:47:18 +02:00
fanquake
1daca9f9a9 Merge bitcoin/bitcoin#21738: test: Use clang-12 for ASAN, Add missing suppression
fa00bb2c5c test: Add missing shift-base:nanobench.h suppression (MarcoFalke)
00004565cc ci: Use clang-12 for asan task (MarcoFalke)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK fa00bb2c5c

Tree-SHA512: fe7cd1ad9f3e73c09f7f84dfb0f276d0cda603c4d591b9338a0914bf1276b0247fd2faee7052f5962c3ae3280e7fa8b72f5b773b84c2a8882a89ed1f8c08256c
2021-04-29 18:13:29 +08:00
fanquake
cf971c9ea0 build: use -isysroot over --sysroot on macOS 2021-04-29 17:40:45 +08:00
MarcoFalke
fa09871320 refactor: Avoid sign-compare compiler warning in util/asmap
This reverts commit eac6a3080d ("refactor:
Rework asmap Interpret to avoid ptrdiff_t"), because it is UB to form a
past-the-end iterator, even if it is never dereferenced.

Then fix the compiler warning in a different way:
Instead of comparing an uint32_t against a signed ptrdiff_t, just
promote both to a type that can represent both types.

Even though in this case the ptrdiff_t should never hold a negative
value, the overhead from promotion should be negligible.
2021-04-29 11:38:18 +02:00
John Newbery
a33bdb52d1 [tests] Speed up p2p_segwit.py
Never sleep for more than 5 seconds when waiting for an
inv-getdata exchange to time out.
2021-04-29 09:17:56 +01:00
MarcoFalke
fa066f1b66 test: Run feature_cltv with MiniWallet 2021-04-29 07:18:28 +02:00
MarcoFalke
fa5591d38c test: Hide tx rehash in helper
This avoids code-bloat and accidentally forgetting to call it
2021-04-29 07:17:36 +02:00
MarcoFalke
fa5f938cfe test: Remove new_tx reference
new_tx is a reference to tx. To avoid confusion and code-bloat, remove it
2021-04-29 07:17:32 +02:00
MarcoFalke
fb66dbe786 Merge bitcoin/bitcoin#21762: test: Speed up mempool_spend_coinbase.py
fa40eb5b6b test: Speed up mempool_spend_coinbase.py (MarcoFalke)
fa29382ab2 test: Fix test cache issue (MarcoFalke)
fa085b470a test: Create MiniWallet.create_self_transfer (MarcoFalke)
fa1bedb494 test: Add MiniWallet.sendrawtransaction (MarcoFalke)

Pull request description:

  Locally the test will run 4 seconds faster with `--valgrind` (18s vs 14s)

ACKs for top commit:
  mjdietzx:
    crACK fa40eb5b6b

Tree-SHA512: ecfb60dda5ca5d7e6367bb9c6210390d95ebf6396ce657728901d118b75bb90c98f9351df3b01004d00682234448d6c6a13338d12097f7dced2cf7f1bd84d924
2021-04-29 07:13:49 +02:00
fanquake
7f37a1d560 Merge bitcoin/bitcoin#21785: test: Fix intermittent issue in p2p_addr_relay.py
faa51ef4d3 test: Fix intermittent issue in p2p_addr_relay.py (MarcoFalke)
fa37116c82 test: Use self.send_addr_msg (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/pull/21707/files#r621048952

ACKs for top commit:
  mzumsande:
    ACK faa51ef4d3
  amitiuttarwar:
    ACK faa51ef4d3

Tree-SHA512: b1fbc1a4a13b020890b465083a82762981de17dd710388c651353aefc28dee23d39803c2a1dadbf88287e92598901afcb01b488115b0fecd746ab6d73775415b
2021-04-29 10:08:27 +08:00
fanquake
2cbb4ab7a9 Merge bitcoin/bitcoin#21658: build: fix make deploy for arm64-darwin
b353633bf4 build: mac_alias 2.2.0 (sgulls)

Pull request description:

  Fix make deploy for arm64-darwin

  Accidentally [closed](https://github.com/bitcoin/bitcoin/pull/21555) the PR

ACKs for top commit:
  promag:
    Tested ACK b353633bf4.

Tree-SHA512: 08043792d63894b6738ea93d076cecace1d8b30a623b944170a34492c3838269da87e09878164c760cf321663fb72641a7295070a847ad67d91fc9970ebe5c6a
2021-04-29 09:12:15 +08:00
João Barbosa
cafef080a2 qt: Refactor to remove unnecessary block in DispatchNotifications
Review with --ignore-all-space
2021-04-29 00:18:55 +01:00
João Barbosa
57785fb7f6 qt: Early subscribe core signals in transaction table model 2021-04-29 00:18:55 +01:00
João Barbosa
c6cbdf1a90 qt: Refactor ShowProgress to DispatchNotifications 2021-04-29 00:18:19 +01:00
João Barbosa
3bccd50ad2 qt: Set flag after inital load on transaction table model 2021-04-28 23:58:11 +01:00
MarcoFalke
4cfe6c37d9 Merge bitcoin/bitcoin#18847: compressor: use a prevector in CompressScript serialization [ZAP1]
83a425d25a compressor: use a prevector in compressed script serialization (William Casarin)

Pull request description:

  This function was doing millions of unnecessary heap allocations during IBD.

  I'm start to catalog unnecessary heap allocations as a pet project of mine: as-zero-as-possible-alloc IBD. This is one small step.

  before:
  ![May01-174536](https://user-images.githubusercontent.com/45598/80850964-9a38de80-8bd3-11ea-8eec-08cd38ee1fa1.png)

  after:
  ![May01-174610](https://user-images.githubusercontent.com/45598/80850974-a91f9100-8bd3-11ea-94a1-e2077391f6f4.png)

  ~should I type alias this?~ *I type aliased it*

  This is a part of the Zero Allocations Project #18849 (ZAP1). This code came up as a place where many allocations occur.

ACKs for top commit:
  Empact:
    ACK 83a425d25a
  elichai:
    tACK 83a425d25a
  sipa:
    utACK 83a425d25a

Tree-SHA512: f0ffa6ab0ea1632715b0b76362753f9f6935f05cdcc80d85566774401155a3c57ad45a687942a1806d3503858f0bb698da9243746c8e2edb8fdf13611235b0e0
2021-04-28 21:13:44 +02:00
MarcoFalke
fa61ce5cf5 fuzz: Limit mocktime to MTP in tx_pool targets
This is needed for the next commit to generate blocks.

Also, apply the same mocking strategies to both targets.
2021-04-28 20:54:07 +02:00
MarcoFalke
fab646b8ea fuzz: Use correct variant of ConsumeRandomLengthString instead of hardcoding a maximum size
This is technically a breaking change.

This allows the fuzz engine to pick the right size,
also larger sizes, if needed.
2021-04-28 20:53:50 +02:00
MarcoFalke
fae2c8bc54 fuzz: Allow to pass min/max to ConsumeTime 2021-04-28 20:52:29 +02:00
Hennadii Stepanov
328da33557 Merge bitcoin-core/gui#18: Add peertablesortproxy module
5a4a15d2b4 qt, refactor: Drop no longer used PeerTableModel::getRowByNodeId func (Hennadii Stepanov)
9a9f180df0 qt, refactor: Drop no longer used PeerTableModel::sort function (Hennadii Stepanov)
778a64af20 qt: Use PeerTableSortProxy for sorting peer table (Hennadii Stepanov)
df2d165ba9 qt: Add peertablesortproxy module (Hennadii Stepanov)

Pull request description:

  The "Peers" table in the "Node" window does not hold multiple selection after sorting.

  This PR introduces a `QSortFilterProxyModel` subclass, that is a standard Qt [practice](https://doc.qt.io/qt-5/model-view-programming.html#custom-sorting-models) for such cases.

  Now the sorting code is encapsulated into the dedicated Qt class, and we do not need to maintain it.

  Fixes #283 (additionally).

  ---

  On **master** (7ae86b3c68):
  - rows are sorted by "Ping", and a selection is made
  ![Screenshot from 2020-11-28 22-53-11](https://user-images.githubusercontent.com/32963518/100525900-96eaed00-31cc-11eb-86e7-72ede3b8b33c.png)

  - rows are sorted by "NodeId", and the previous selection is _lost_
  ![Screenshot from 2020-11-28 22-53-21](https://user-images.githubusercontent.com/32963518/100525904-9c483780-31cc-11eb-957c-06f53d7d31ab.png)

  With **this PR**:
  - rows are sorted by "Ping", and a selection is made
  ![Screenshot from 2020-11-28 22-39-41](https://user-images.githubusercontent.com/32963518/100525776-06aca800-31cc-11eb-8c4e-9c6566fe80fe.png)

  - rows are sorted by "NodeId", and the row are still selected
  ![Screenshot from 2020-11-28 22-39-53](https://user-images.githubusercontent.com/32963518/100525791-2348e000-31cc-11eb-8b78-716a5551d7ec.png)

ACKs for top commit:
  jarolrod:
    re-ACK 5a4a15d2b4, tested on macOS 11.2 Qt 5.15.2 after rebase
  promag:
    Tested ACK 5a4a15d2b4.

Tree-SHA512: f81c1385892fbf1a46ffb98b42094ca1cc97da52114bbbc94fedb553899b1f18c26a349e186bba6e27922a89426bd61e8bc88b1f7832512dbe211b5f834e076e
2021-04-28 20:57:52 +03:00
practicalswift
54549dda31 fuzz: RPC fuzzer post-merge follow-ups. Remove unused includes. Update list of fuzzed RPC commands. 2021-04-28 09:27:50 +00:00
MarcoFalke
549d20a31b Merge bitcoin/bitcoin#20772: fuzz: bolster ExtractDestination(s) checks
a29f522ba4 fuzz: bolster ExtractDestination(s) checks (Michael Dietz)

Pull request description:

ACKs for top commit:
  practicalswift:
    Tested ACK a29f522ba4

Tree-SHA512: 0fc194edb7b0fce77c7bb725fe65dec7976598edcd53882b5a0eb7cd83281a3ddcd2b3de00282468be659a7e5bc9991eb482816418f55b30e657cdc5a3bd7438
2021-04-28 11:02:25 +02:00
Antoine Poinsot
ebd4be43cc doc: add release notes for 20867
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:31 +02:00
Antoine Poinsot
5aa50ab9cc rpc/util: multisig: only check redeemScript size is <= 520 for P2SH
This increase the maximum number of pubkeys to 20 (valid in P2WSH and
P2SH-P2WSH) and only checks the redeemScript doesn't exceed
MAX_SCRIPT_ELEMENT_SIZE for P2SH, as this checked is removed under
Segwit context.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:30 +02:00
Antoine Poinsot
063df9e897 test/functional: standardness sanity checks for P2(W)SH multisig
Note that it also test for sortedmulti(), which the previous commit didn't.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:30 +02:00
Antoine Poinsot
ae0429d3af script: allow up to 20 keys in wsh() descriptors
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:29 +02:00
Antoine Poinsot
9fc68faf35 script: match multisigs with up to MAX_PUBKEYS_PER_MULTISIG keys
We were previously ruling out 17-20 pubkeys multisig, while they are
only invalid under P2SH context.
This makes multisigs with up to 20 keys be detected as valid by the
solver. This is however *not* a policy change as it would only apply
to bare multisigs, which are already limited to 3 pubkeys.

Note that this does not change the sigOpCount calculation (as it would
break consensus). Therefore 1-16 keys multisigs are counted as 1-16 sigops
and 17-20 keys multisigs are counted as 20 sigops.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:28 +02:00
MarcoFalke
e45863166f Merge bitcoin/bitcoin#21169: fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70%.
545404e7e1 fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70%. (practicalswift)

Pull request description:

  Add RPC interface fuzzing.

  This PR increases overall fuzzing line coverage from [~65%](https://marcofalke.github.io/btc_cov/fuzz.coverage/) to ~70% 🎉

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make -C src/ test/fuzz/fuzz
  $ FUZZ=rpc src/test/fuzz/fuzz
  ```

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for more information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    Concept ACK 545404e7e1

Tree-SHA512: 35fc1b508af42bf480ee3762326b09ff2eecdb7960a1917ad16345fadd5c0c21d666dafe736176e5a848ff6492483c782e4ea914cd9000faf50190df051950fd
2021-04-28 09:45:30 +02:00
practicalswift
545404e7e1 fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70%. 2021-04-28 06:34:20 +00:00
MarcoFalke
faa51ef4d3 test: Fix intermittent issue in p2p_addr_relay.py 2021-04-28 08:16:29 +02:00
MarcoFalke
fa37116c82 test: Use self.send_addr_msg 2021-04-28 08:13:05 +02:00
MarcoFalke
edf679503c Merge bitcoin/bitcoin#21783: refactor: Make MempoolAcceptResult members const
363df758a9 doc/style followups in MempoolAcceptResult (glozow)

Pull request description:

  Follow up to #21062. Was going to be a part of #20833 but I'm trying to break it down as much as possible.

  - Make members const (https://github.com/bitcoin/bitcoin/pull/21062#discussion_r573659273)
  - List fee units (https://github.com/bitcoin/bitcoin/pull/21062#discussion_r569329362)
  - Use default value for `TxValidationState` in the success case (https://github.com/bitcoin/bitcoin/pull/21062#discussion_r573659801).

ACKs for top commit:
  jnewbery:
    ACK 363df758a9
  practicalswift:
    cr ACK 363df758a9: patch looks correct and `const` is better than non-`const` (where possible :))
  ariard:
    Code Review ACK 363df75

Tree-SHA512: 0ff1a0279e08e03204e48d0f4c92428d7f39c32f52c1d20fe6a0283d605839898297344be82ca69640ba9f878ca4ebd5da2d717e26d719a183b211d709334082
2021-04-28 08:06:43 +02:00
MarcoFalke
bce09da122 Merge bitcoin/bitcoin#21773: fuzz: Ensure prevout is consensus-valid
fa1fdeb230 fuzz: Ensure prevout is consensus-valid (MarcoFalke)

Pull request description:

  It shouldn't be possible to create consensus-invalid prevouts, so there is no need to fuzz them.

  To reproduce:

  ```
  $ echo 'AAAAAAEAAAAAAQF0Rw0SGsrit4+YZSEfpcQT/o+bJbjgVjATUHqrCfRE+QsBAAAXFgAUlsvXHgGV
  ZxF3QXxitwe1tIOYdLj2NePHATl9CgAAAAAAGXapFOFHg1yqRFl7soeowwpIEOoe9G1NiKwCRzBE
  AiAx6F2Q008gvJnok6JiyOn7lPqCJJmDiI2omRNXT1Q7XAIgCQP6WJizAqhnvImpQqYMJkqePGvx
  Jy/pGRMy1iNL0ecDIQJr4tWomVTBfjpyMFMOD9aDAR5gkByOIYiaQOv8P/sRztP3pS8RDAAAEUUE
  NQBwYAAAAAC5F6kUTLIzj/lKP2Hmpwyzukns2eweRkOH' | base64 --decode  > /tmp/a

  $ FUZZ=script_flags ./src/test/fuzz/fuzz /tmp/a
  INFO: Running with entropic power schedule (0xFF, 100).
  INFO: Seed: 59714236
  INFO: Loaded 1 modules   (212532 inline 8-bit counters): 212532 [0x55987fb3f668, 0x55987fb7349c),
  INFO: Loaded 1 PC tables (212532 PCs): 212532 [0x55987fb734a0,0x55987feb17e0),
  ./src/test/fuzz/fuzz: Running 1 inputs 1 time(s) each.
  Running: /tmp/a
  fuzz: script/interpreter.cpp:1495: bool HandleMissingData(MissingDataBehavior): Assertion `!"Missing data"' failed.
  ==520092== ERROR: libFuzzer: deadly signal
      #0 0x55987f111180 in __sanitizer_print_stack_trace (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x5ea180)
      #1 0x55987f0ba828 in fuzzer::PrintStackTrace() fuzzer.o
      #2 0x55987f09de43 in fuzzer::Fuzzer::CrashCallback() fuzzer.o
      #3 0x7fd003d563bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
      #4 0x7fd00399a18a in raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
      #5 0x7fd003979858 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
      #6 0x7fd003979728  (/lib/x86_64-linux-gnu/libc.so.6+0x25728)
      #7 0x7fd00398af35 in __assert_fail (/lib/x86_64-linux-gnu/libc.so.6+0x36f35)
      #8 0x55987f8ce194 in HandleMissingData(MissingDataBehavior) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:1495:9
      #9 0x55987f8ce194 in GenericTransactionSignatureChecker<CTransaction>::CheckECDSASignature(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, CScript const&, SigVersion) const /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:1685:68
      #10 0x55987f8cbbc8 in EvalChecksigPreTapscript(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, prevector<28u, unsigned char, unsigned int, int>::const_iterator, prevector<28u, unsigned char, unsigned int, int>::const_iterator, unsigned int, BaseSignatureChecker const&, SigVersion, ScriptError_t*, bool&) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:363:24
      #11 0x55987f8cbbc8 in EvalChecksig(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, prevector<28u, unsigned char, unsigned int, int>::const_iterator, prevector<28u, unsigned char, unsigned int, int>::const_iterator, ScriptExecutionData&, unsigned int, BaseSignatureChecker const&, SigVersion, ScriptError_t*, bool&) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:421:16
      #12 0x55987f8c5a01 in EvalScript(std::vector<std::vector<unsigned char, std::allocator<unsigned char> >, std::allocator<std::vector<unsigned char, std::allocator<unsigned char> > > >&, CScript const&, unsigned int, BaseSignatureChecker const&, SigVersion, ScriptExecutionData&, ScriptError_t*) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:1094:26
      #13 0x55987f8d6d6e in ExecuteWitnessScript(Span<std::vector<unsigned char, std::allocator<unsigned char> > const> const&, CScript const&, unsigned int, SigVersion, BaseSignatureChecker const&, ScriptExecutionData&, ScriptError_t*) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:1843:10
      #14 0x55987f8d48fc in VerifyWitnessProgram(CScriptWitness const&, int, std::vector<unsigned char, std::allocator<unsigned char> > const&, unsigned int, BaseSignatureChecker const&, ScriptError_t*, bool) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:1904:20
      #15 0x55987f8d3d8b in VerifyScript(CScript const&, CScript const&, CScriptWitness const*, unsigned int, BaseSignatureChecker const&, ScriptError_t*) /root/fuzz_dir/scratch/fuzz_gen/code/src/script/interpreter.cpp:2045:22
      #16 0x55987f201d47 in script_flags_fuzz_target(Span<unsigned char const>) /root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/script_flags.cpp:54:30
      #17 0x55987f11447f in std::_Function_handler<void (Span<unsigned char const>), void (*)(Span<unsigned char const>)>::_M_invoke(std::_Any_data const&, Span<unsigned char const>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:300:2
      #18 0x55987f8aed17 in std::function<void (Span<unsigned char const>)>::operator()(Span<unsigned char const>) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #19 0x55987f8aed17 in LLVMFuzzerTestOneInput /root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz.cpp:63:5
      #20 0x55987f09f5e3 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) fuzzer.o
      #21 0x55987f0894e2 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) fuzzer.o
      #22 0x55987f08f2da in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) fuzzer.o
      #23 0x55987f0bb002 in main (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x594002)
      #24 0x7fd00397b0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
      #25 0x55987f06420d in _start (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x53d20d)

  NOTE: libFuzzer has rudimentary signal handlers.
        Combine libFuzzer with AddressSanitizer or similar for better crash reports.
  SUMMARY: libFuzzer: deadly signal

ACKs for top commit:
  practicalswift:
    Tested ACK fa1fdeb230

Tree-SHA512: 6130ed9ab6d8eeab901f64a1c069300e67d0b6009c42763262fe6edeab8192e088c1a3c1f61aee900b9ebbc48fbf6e837b41704bad592ec526398355766e208a
2021-04-28 07:13:07 +02:00
Jon Atack
f09e6b2585 test: fix off-by-ones in rpc_fundrawtransaction 2021-04-27 17:13:38 +02:00
Yuval Kogman
f9e37f33ce doc: IsFinalTx comment about nSequence & OP_CLTV
It's somewhat surprising that a transaction's nLockTime field is ignored
when all nSequence fields are final, so this change aims to clarify this
behavior and cross reference relevant details of OP_CHECKLOCKTIMEVERIFY.
2021-04-27 11:52:13 +00:00
MarcoFalke
c6d6bc8abb Merge bitcoin/bitcoin#21523: validation: run VerifyDB on all chainstates
844ad0ecca doc: IsSnapshotActive (James O'Beirne)
9b604c0207 validation: prepare VerifyDB for assumeutxo (James O'Beirne)
7901647d72 refactor: rename active_chainstate in VerifyDB (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: #15606)

  ---

  ~~Pretty cut and dry; parameterizes `CVerifyDB` methods so that we can run the verify procedure on multiple chainstates.~~

  Two minor tweaks to ensure that `VerifyDB` can be run on multiple chainstates and a corresponding rename.

ACKs for top commit:
  fjahr:
    Code review re-ACK 844ad0ecca
  MarcoFalke:
    review ACK 844ad0ecca 🐥

Tree-SHA512: 26a398cf4dabc1aa0850743921dba0452b4813848a3c777586dc981716737e98e17b8110254a5c41af95dd236e0c00dc8b4eee891d69bef825a5e1911fc499d0
2021-04-27 13:31:29 +02:00
W. J. van der Laan
ac219dcbcc Merge bitcoin/bitcoin#19160: multiprocess: Add basic spawn and IPC support
84934bf70e multiprocess: Add echoipc RPC method and test (Russell Yanofsky)
7d76cf667e multiprocess: Add comments and documentation (Russell Yanofsky)
ddf7ecc8df multiprocess: Add bitcoin-node process spawning support (Russell Yanofsky)
10afdf0280 multiprocess: Add Ipc interface implementation (Russell Yanofsky)
745c9cebd5 multiprocess: Add Ipc and Init interface definitions (Russell Yanofsky)
5d62d7f6cd Update libmultiprocess library (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR adds basic process spawning and IPC method call support to `bitcoin-node` executables built with `--enable-multiprocess`[*].

  These changes are used in https://github.com/bitcoin/bitcoin/pull/10102 to let node, gui, and wallet functionality run in different processes, and extended in https://github.com/bitcoin/bitcoin/pull/19460 and https://github.com/bitcoin/bitcoin/pull/19461 after that to allow gui and wallet processes to be started and stopped independently and connect to the node over a socket.

  These changes can also be used to implement new functionality outside the `bitcoin-node` process like external indexes or pluggable transports (https://github.com/bitcoin/bitcoin/pull/18988). The `Ipc::spawnProcess` and `Ipc::serveProcess` methods added here are entry points for spawning a child process and serving a parent process, and being able to make bidirectional, multithreaded method calls between the processes. A simple example of this is implemented in commit "Add echoipc RPC method and test."

  Changes in this PR aside from the echo test were originally part of #10102, but have been split and moved here for easier review, and so they can be used for other applications like external plugins.

  Additional notes about this PR can be found at https://bitcoincore.reviews/19160

  [*] Note: the `--enable-multiprocess` feature is still experimental, and not enabled by default, and not yet supported on windows. More information can be found in [doc/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/multiprocess.md)

ACKs for top commit:
  fjahr:
    re-ACK 84934bf70e
  ariard:
    ACK 84934bf. Changes since last ACK fixes the silent merge conflict about `EnsureAnyNodeContext()`. Rebuilt and checked again debug command `echoipc`.

Tree-SHA512: 52a948b5e18a26d7d7a09b83003eaae9b1ed2981978c36c959fe9a55abf70ae6a627c4ff913a3428be17400a3dace30c58b5057fa75c319662c3be98f19810c6
2021-04-27 11:59:26 +02:00
MarcoFalke
fa09a9eac8 style: Add { } to multi-line if
Can be reviewed with --word-diff-regex=. --ignore-all-space
2021-04-27 10:36:41 +02:00
MarcoFalke
fadafab833 move-only: Move functions to blockstorage 2021-04-27 10:36:23 +02:00
MarcoFalke
fa7e64d586 move-only: Move constants to blockstorage 2021-04-27 10:32:54 +02:00
MarcoFalke
fa247a327f refactor: Move block storage globals to blockstorage
However, keep a declaration in validation to make it possible to move
smaller chunks to blockstorage without breaking compilation.

Also, expose AbortNode in the header.

Can be reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-27 10:32:30 +02:00
MarcoFalke
fa81c30c6f refactor: Move pruning/reindex/importing globals to blockstorage
Can be reviewed with --color-moved=dimmed-zebra
2021-04-27 10:32:24 +02:00
W. J. van der Laan
19a56d1519 Merge bitcoin/bitcoin#21009: Remove RewindBlockIndex logic
d831e711ca [validation] RewindBlockIndex no longer needed (Dhruv Mehta)

Pull request description:

  Closes #17862

  Context from [original comment](https://github.com/bitcoin/bitcoin/issues/17862#issuecomment-744285188) (minor edits):

  `RewindBlockIndex()` is a mechanism to allow nodes to be upgraded after segwit activation, while still keeping their chainstate/datadir in a consistent state. It works as follows:

  - A pre-segwit (i.e. v0.13.0 or older) node is running.
  -  Segwit activates. The pre-segwit node remains sync'ed to the tip, but is not enforcing the new segwit rules.
  - The user upgrades the node to a segwit-aware version (v0.13.1 or newer).
  - On startup, in `AppInitMain()`, `RewindBlockIndex()` is called. This walks the chain backwards from the tip, disconnecting and erasing blocks that from after segwit activation that weren't validated with segwit rules.
  - those blocks are then redownloaded (with witness data) and validated with segwit rules.

  This logic probably isn't required any more since:

  - Segwit activated at height 481824, when the block chain was 130GB and the total number of txs was 250 million. Today, we're at height 667704, the blockchain is over 315GB and the total number of txs is over 600 million. Even if 20% of that added data is witness data (a high estimate), then around 150GB of transactions would need to be rewound to get back to segwit activation height. It'd probably be faster to simply validate from genesis, especially since we won't be validating any scripts before the assumevalid block. It's also unclear whether rewinding 150GB of transactions would even work. It's certainly never been tested.
  - Bitcoin Core v0.13 is hardly used any more. https://luke.dashjr.org/programs/bitcoin/files/charts/software.html shows less than 50 nodes running it. The software was EOL on Aug 1st 2018. It's very unlikely that anyone is running 0.13 and will want to upgrade to 0.22.

  This PR introduces `NeedsRedownload()` which merely checks for insufficiently validated segwit blocks and requests that the user restarts the node with `-reindex`. Reindexing the block files upon restart will make the node rebuild chain state and block index from the `blk*.dat` files on disk. The node won't be able to index the blocks with `BLOCK_OPT_WITNESS`, so they will be missing from the chain and be re-downloaded, with witness data.

  Removing this code allows the following (done in follow-up #21090):

  - removal of tests using `segwitheight=-1` in `p2p_segwit.py`.
  - in turn, that allows us to drop support for `-segwitheight=-1`, which is only supported for that test.
  - that allows us to always set `NODE_WITNESS` in our local services. The only reason we don't do that is to support `-segwitheight=-1`.
  - that in turn allows us to drop all of the `GetLocalServices() & NODE_WITNESS` checks inside `net_processing.cpp`, since our local services would always include `NODE_WITNESS`

ACKs for top commit:
  jnewbery:
    utACK d831e711ca
  jamesob:
    ACK d831e711ca
  laanwj:
    Cursory code review ACK d831e711ca. Agree with the direction of the change, thanks for simplifying the logic here.
  glozow:
    utACK d831e711ca

Tree-SHA512: 3eddf5121ccd081ad7f15a5c6478ef867083edc8ba0bf1ee759e87bc070ee3d2f0698a3feba8db8dc087987c8452887b6f72cff05b3e178f41cb10a515fb8053
2021-04-27 10:14:52 +02:00
MarcoFalke
f8f55522c4 Merge bitcoin/bitcoin#21746: refactor: init: mark fReset const
785f9cc46a refactor: init: mark fReset const (James O'Beirne)

Pull request description:

  Small thing, but hey - it doesn't change.

ACKs for top commit:
  theStack:
    Code-review ACK 785f9cc46a

Tree-SHA512: 3cb8d7037f517162f6315d561accc4932b0f1e340162c3283871433f2e355d57b3740c9d2e953ce33fbfa3b277c8437f91955fb70331b3fe9c8e6a8589dc2b49
2021-04-27 08:26:57 +02:00
randymcmillan
e94920a0bb qt: peertableview alternating row colors 2021-04-26 19:16:15 -04:00
glozow
6ba892126d refactor + document coin selection strategy
Co-authored-by: Xekyo <murch@murch.one>
2021-04-26 11:00:18 -07:00
glozow
58ea324fdd [docs] add doxygen comments to wallet code
Co-authored-by: Xekyo <murch@murch.one>
2021-04-26 11:00:18 -07:00
James O'Beirne
91d93aac4e validation: remove nchaintx from assumeutxo metadata
This value is no longer used and is instead specified statically
in chainparams. This change means that previously generated
snapshots will no longer be usable.
2021-04-26 13:22:37 -04:00
MarcoFalke
683dda2a70 Merge bitcoin/bitcoin#21777: test: Fix feature_notifications.py intermittent issue
fa4aec2b26 test: Fix feature_notifications.py intermittent issue (MarcoFalke)

Pull request description:

  Fixes #21683

Top commit has no ACKs.

Tree-SHA512: 256806d82877477f4b3d795658f61127c0de4eff07216f6071f40a8ec1f5d43f3c587f35dd436d480dc261ef6646ac5547db104d22f3fcfeeb61bbdbe04bcc31
2021-04-26 18:04:22 +02:00
glozow
363df758a9 doc/style followups in MempoolAcceptResult 2021-04-26 05:24:32 -07:00
MarcoFalke
74a960a220 Merge bitcoin/bitcoin#21714: refactor: Drop CCoinControl::SetNull
c5a470eee1 refactor: Drop CCoinControl::SetNull (João Barbosa)

Pull request description:

  The only external call to `SetNull` is changed as follow

  ```diff
  - m_coin_control->SetNull();
  + m_coin_control = std::make_unique<CCoinControl>();
  ```

ACKs for top commit:
  theStack:
    Code-Review ACK c5a470eee1
  MarcoFalke:
    review ACK c5a470eee1 🍤

Tree-SHA512: 2588828720cdcf405fc4fb06f2aa17ad4eef2a645e12d820311006127e732258dd084993f17f23742f8e7f782e18289a6199dcec3690efc9b92458f90b816a8f
2021-04-26 09:20:44 +02:00
MarcoFalke
0a0a95b9d6 Merge bitcoin/bitcoin#21707: test: Extend functional tests for addr relay
a732ee353c [test] Add tests for addr relay in -blocksonly mode (Amiti Uttarwar)
a6694eaed8 [test] Add address relay tests involving outbound peers (Martin Zumsande)
8188b77c17 [test] Add tests for getaddr behavior (Martin Zumsande)
d2dbfe6ff1 [test] Extract sending an addr message into a helper (Amiti Uttarwar)
c991943399 [test] Refactor the addr relay test to prepare for new tests (Amiti Uttarwar)

Pull request description:

  This extends the functional test `p2p_addr_relay.py`.
  It adds test coverage for address relay involving outbound peers, tests for both outgoing and incoming `GETADDR` requests and tests for `-blocksonly` mode.

  The initial refactors and some of the new tests were taken from Amiti Uttarwar's PR #21528 - they are general test improvements not directly tied to the change proposed there.

ACKs for top commit:
  amitiuttarwar:
    re-ACK a732ee353c, small diff based on code review
  MarcoFalke:
    Concept ACK a732ee353c 🌊

Tree-SHA512: e80d52683808ddd6b948a5134239f002f3fecf61b60e187877b07be6251721fde847104e495c75a1a5133a09c0b41a9255a0bec82932c0b304b516fa89bce33e
2021-04-26 09:14:34 +02:00
MarcoFalke
fa4aec2b26 test: Fix feature_notifications.py intermittent issue 2021-04-26 08:22:46 +02:00
bruno
61fd8fe9a1 Remove progress bar on modal overlay 2021-04-25 21:22:35 -03:00
Hennadii Stepanov
19d51a2907 qt: Avoid unnecessary translations 2021-04-25 21:58:15 +03:00
Hennadii Stepanov
3adde72bc9 qt: Do not use QObject::tr plural syntax for numbers with a unit symbol
This is useless and unnecessarily burdensome for translators.
2021-04-25 21:41:24 +03:00
MarcoFalke
fac96d0265 p2p: Limit m_block_inv_mutex 2021-04-25 20:39:59 +02:00
Hennadii Stepanov
c4571a0c39 Merge bitcoin-core/gui#284: refactor: Simplify SendCoinsDialog::updateCoinControlState
5f438d66c1 refactor, qt: Simplify SendCoinsDialog::updateCoinControlState (João Barbosa)

Pull request description:

  This PR doesn't change behaviour, removes the coin control argument from `updateCoinControlState` since it's a class member.

ACKs for top commit:
  hebasto:
    ACK 5f438d66c1, I have reviewed the code and it looks OK, I agree it can be merged.
  jonatack:
    Code review ACK 5f438d66c1
  kristapsk:
    utACK 5f438d66c1. Code looks correct.

Tree-SHA512: 14abaa3d561f8c8854fed989b6aca886dcca42135880bac76070043f61c0042ec8967f2b83e50bbbb82050ef0f074209e97fa300cb4dc51ee182316e0846506d
2021-04-25 17:06:56 +03:00
Hennadii Stepanov
30e4448215 refactor: Make TraceThread a non-template free function
Also it is moved into its own module.
2021-04-25 12:28:44 +03:00
MarcoFalke
fa1fdeb230 fuzz: Ensure prevout is consensus-valid 2021-04-25 10:36:00 +02:00
MarcoFalke
8f80092d78 Merge bitcoin/bitcoin#21563: net: Restrict period when cs_vNodes mutex is locked
8c8237a4a1 net, refactor: Fix style in CConnman::StopNodes (Hennadii Stepanov)
229ac1892d net: Combine two loops into one, and update comments (Hennadii Stepanov)
a3d090d110 net: Restrict period when cs_vNodes mutex is locked (Hennadii Stepanov)

Pull request description:

  This PR restricts the period when the `cs_vNodes` mutex is locked, prevents the only case when `cs_vNodes` could be locked before the `::cs_main`.

  This change makes the explicit locking of recursive mutexes in the explicit order redundant.

ACKs for top commit:
  jnewbery:
    utACK 8c8237a4a1
  vasild:
    ACK 8c8237a4a1
  ajtowns:
    utACK 8c8237a4a1 - logic seems sound
  MarcoFalke:
    review ACK 8c8237a4a1 👢

Tree-SHA512: a8277924339622b188b12d260a100adf5d82781634cf974320cf6007341f946a7ff40351137c2f5369aed0d318f38aac2d32965c9b619432440d722a4e78bb73
2021-04-25 10:08:57 +02:00
windsok
6f6bb3ebc7 test: fix file permissions on various scripts
Updates permissions on files to comply with the new test added in the following commit
2021-04-23 17:13:28 -07:00
James O'Beirne
844ad0ecca doc: IsSnapshotActive 2021-04-23 15:06:54 -04:00
James O'Beirne
9b604c0207 validation: prepare VerifyDB for assumeutxo
Removes assumptions of use only on the active chainstate.
2021-04-23 15:06:48 -04:00
James O'Beirne
7901647d72 refactor: rename active_chainstate in VerifyDB
To prepare VerifyDB semantics for multiple
chainstate use.
2021-04-23 15:02:35 -04:00
Pieter Wuille
5c7ee1b2da libsecp256k1 no longer has --with-bignum= configure option 2021-04-23 11:36:01 -07:00
Pieter Wuille
a5a447a352 Update libsecp256k1 subtree to latest upstream master 2021-04-23 11:35:15 -07:00
Pieter Wuille
bdca9bcb6c Squashed 'src/secp256k1/' changes from 3967d96bf1..efad3506a8
efad3506a8 Merge #906: Use modified divsteps with initial delta=1/2 for constant-time
cc2c09e3a7 Merge #918: Clean up configuration in gen_context
07067967ee add ECMULT_GEN_PREC_BITS to basic_config.h
a3aa2628c7 gen_context: Don't include basic-config.h
be0609fd54 Add unit tests for edge cases with delta=1/2 variant of divsteps
cd393ce228 Optimization: only do 59 hddivsteps per iteration instead of 62
277b224b6a Use modified divsteps with initial delta=1/2 for constant-time
376ca366db Fix typo in explanation
1e5d50fa93 Merge #889: fix uninitialized read in tests
c083cc6e52 Merge #903: Make argument of fe_normalizes_to_zero{_var} const
6e898534ff Merge #907: changed import to use brackets <> for openssl
4504472269 changed import to use brackets <> for openssl as they are not local to the project
26de4dfeb1 Merge #831: Safegcd inverses, drop Jacobi symbols, remove libgmp
23c3fb629b Make argument of fe_normalizes_to_zero{_var} const
24ad04fc06 Make scalar_inverse{,_var} benchmark scale with SECP256K1_BENCH_ITERS
ebc1af700f Optimization: track f,g limb count and pass to new variable-time update_fg_var
b306935ac1 Optimization: use formulas instead of lookup tables for cancelling g bits
9164a1b658 Optimization: special-case zero modulus limbs in modinv64
1f233b3fa0 Remove num/gmp support
20448b8d09 Remove unused Jacobi symbol support
5437e7bdfb Remove unused scalar_sqr
aa9cc52180 Improve field/scalar inverse tests
1e0e885c8a Make field/scalar code use the new modinv modules for inverses
436281afdc Move secp256k1_fe_inverse{_var} to per-impl files
aa404d53be Move secp256k1_scalar_{inverse{_var},is_even} to per-impl files
08d54964e5 Improve bounds checks in modinv modules
151aac00d3 Add tests for modinv modules
d8a92fcc4c Add extensive comments on the safegcd algorithm and implementation
8e415acba2 Add safegcd based modular inverse modules
de0a643c3d Add secp256k1_ctz{32,64}_var functions
4c3ba88c3a Merge #901: ci: Switch all Linux builds to Debian and more improvements
9361f360bb ci: Select number of parallel make jobs depending on CI environment
28eccdf806 ci: Split output of logs into multiple sections
c7f754fe4d ci: Run PRs on merge result instead of on the source branch
b994a8be3c ci: Print information about binaries using "file"
f24e122d13 ci: Switch all Linux builds to Debian
ebdba03cb5 Merge #891: build: Add workaround for automake 1.13 and older
3a8b47bc6d Merge #894: ctime_test: move context randomization test to the end
7d3497cdc4 ctime_test: move context randomization test to the end
99a1cfec17 print warnings for conditional-uninitialized
3d2cf6c5bd initialize variable in tests
f329bba244 build: Add workaround for automake 1.13 and older
24d1656c32 Merge #882: Use bit ops instead of int mult for constant-time logic in gej_add_ge
e491d06b98 Use bit ops instead of int mult for constant-time logic in gej_add_ge
f8c0b57e6b Merge #864: Add support for Cirrus CI
cc2a5451dc ci: Refactor Nix shell files
2480e55c8f ci: Remove support for Travis CI
2b359f1c1d ci: Enable simple cache for brewing valgrind on macOS
8c02e465c5 ci: Add support for Cirrus CI
659d0d4798 Merge #880: Add parens around ROUND_TO_ALIGN's parameter.
b6f649889a Add parens around ROUND_TO_ALIGN's parameter. This makes the macro robust against a hypothetical ROUND_TO_ALIGN(foo ? sizeA : size B) invocation.
a4abaab793 Merge #877: Add missing secp256k1_ge_set_gej_var decl.
5671e5f3fd Merge #874: Remove underscores from header defs.
db726782fa Merge #878: Remove unused secp256k1_fe_inv_all_var
b732701faa Merge #875: Avoid casting (void**) values.
75d2ae149e Remove unused secp256k1_fe_inv_all_var
482e4a9cfc Add missing secp256k1_ge_set_gej_var decl.
2730618604 Avoid casting (void**) values. Replaced with an expression that only casts (void*) values.
fb390c5299 Remove underscores from header defs. This makes them consistent with other files and avoids reserved identifiers.
f2d9aeae6d Merge #862: Autoconf improvements
328aaef22a Merge #845: Extract the secret key from a keypair
3c15130709 Improve CC_FOR_BUILD detection
47802a4762 Restructure and tidy configure.ac
252c19dfc6 Ask brew for valgrind include path
8c727b9087 Merge #860: fixed trivial typo
b7bc3a4aaa fixed typo
33cb3c2b1f Add secret key extraction from keypair to constant time tests
36d9dc1e8e Add seckey extraction from keypair to the extrakeys tests
fc96aa73f5 Add a function to extract the secretkey from a keypair
98dac87839 Merge #858: Fix insecure links
07aa4c70ff Fix insecure links
b61f9da54e Merge #857: docs: fix simple typo, dependecy -> dependency
18aadf9d28 docs: fix simple typo, dependecy -> dependency
2d9e7175c6 Merge #852: Add sage script for generating scalar_split_lambda constants
dc6e5c3a5c Merge #854: Rename msg32 to msghash32 in ecdsa_sign/verify and add explanation
6e85d675aa Rename tweak to tweak32 in public API
f587f04e35 Rename msg32 to msghash32 in ecdsa_sign/verify and add explanation
329a2e0a3f sage: Add script for generating scalar_split_lambda constants
8f0c6f1545 Merge #851: make test count iteration configurable by environment variable
f4fa8d226a forbid a test iteration of 0 or less
f554dfc708 sage: Reorganize files
3a106966aa Merge #849: Convert Sage code to Python 3 (as used by Sage >= 9)
13c88efed0 Convert Sage code to Python 3 (as used by Sage >= 9)
0ce4554881 make test count iteration configurable by environment variable
9e5939d284 Merge #835: Don't use reserved identifiers memczero and benchmark_verify_t
d0a83f7328 Merge #839: Prevent arithmetic on NULL pointer if the scratch space is too small
903b16aa6c Merge #840: Return NULL early in context_preallocated_create if flags invalid
1f4dd03838 Typedef (u)int128_t only when they're not provided by the compiler
ebfa2058e9 Return NULL early in context_preallocated_create if flags invalid
29a299e373 Run the undefined behaviour sanitizer on Travis
7506e064d7 Prevent arithmetic on NULL pointer if the scratch space is too small
e89278f211 Don't use reserved identifiers memczero and benchmark_verify_t

git-subtree-dir: src/secp256k1
git-subtree-split: efad3506a8937162e8010f5839fdf3771dfcf516
2021-04-23 11:35:15 -07:00
Russell Yanofsky
84934bf70e multiprocess: Add echoipc RPC method and test
Add simple interfaces::Echo IPC interface with one method that just takes and
returns a string, to test multiprocess framework and provide an example of how
it can be used to spawn and call between processes.
2021-04-23 03:02:50 -05:00
MarcoFalke
66fd3b28e8 Merge bitcoin/bitcoin#21732: MOVEONLY: Move common init code to init/common
615965cfd1 Move common package version code to init/common (Russell Yanofsky)
5bed2ab42c Move common logging start code to init/common (Russell Yanofsky)
1fb7fcfa52 Move common logging GetArgs code to init/common (Russell Yanofsky)
90469c1690 Move common logging AddArg code to init/common (Russell Yanofsky)
387c4cf588 Move common sanity check code to init/common (Russell Yanofsky)
a67b54855b Move common global init code to init/common (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This change is move-only and can be easily reviewed with `--color-moved=dimmed_zebra`. The moves are needed to avoid duplicating common init code between different binaries (`bitcoin-node`, `bitcoin-wallet`, etc) in #10102. In #10102, each binary has it's own init file (`src/init/bitcoin-node.cpp`, `src/init/bitcoin-wallet.cpp`) so this PR moves the common code to `src/init/common.cpp`.

ACKs for top commit:
  MarcoFalke:
    review ACK 615965cfd1 🖱
  practicalswift:
    cr ACK 615965cfd1: dimmed zebra looks correct

Tree-SHA512: 859e1d86aee17eb50a49d806cf62d30d12f6b15018e41c096da41d7e535a9d2d088481cb340fee59e6c68e512a74b61c7146f2683465f553dc4953bf32f2a7b4
2021-04-23 10:02:50 +02:00
Russell Yanofsky
7d76cf667e multiprocess: Add comments and documentation 2021-04-23 03:02:50 -05:00
Russell Yanofsky
ddf7ecc8df multiprocess: Add bitcoin-node process spawning support
Add bitcoin-node startup code to let it spawn and be spawned by other
processes
2021-04-23 03:02:50 -05:00
Russell Yanofsky
10afdf0280 multiprocess: Add Ipc interface implementation 2021-04-23 03:02:50 -05:00
Russell Yanofsky
745c9cebd5 multiprocess: Add Ipc and Init interface definitions 2021-04-23 03:02:50 -05:00
Russell Yanofsky
5d62d7f6cd Update libmultiprocess library
Fix "Disable GCC suggest-override warnings for proxy clients" https://github.com/chaincodelabs/libmultiprocess/pull/40 is needed to prevent cirrus GCC failure https://cirrus-ci.com/task/6000489311502336?command=ci#L4294

This also includes other recent changes

https://github.com/chaincodelabs/libmultiprocess/pull/35 Fix README.md markdown
https://github.com/chaincodelabs/libmultiprocess/pull/37 Add "make check" target to build and run tests
https://github.com/chaincodelabs/libmultiprocess/pull/38 Add "extends" inherited method support
https://github.com/chaincodelabs/libmultiprocess/pull/41 Avoid depending on argument default constructors
https://github.com/chaincodelabs/libmultiprocess/pull/42 Support attaching custom cleanup functions to proxy client and server classes
https://github.com/chaincodelabs/libmultiprocess/pull/43 Drop hardcoded #include lines in generated files
2021-04-23 03:02:50 -05:00
MarcoFalke
fa40eb5b6b test: Speed up mempool_spend_coinbase.py 2021-04-23 09:56:02 +02:00
MarcoFalke
fa29382ab2 test: Fix test cache issue
The documentation did not match the implementation, no coins
were mined to the OP_TRUE address.
2021-04-23 09:48:56 +02:00
MarcoFalke
fa085b470a test: Create MiniWallet.create_self_transfer 2021-04-23 09:39:21 +02:00
Hennadii Stepanov
7962e0dde8 qt: Do not clear console prompt when font resizing 2021-04-22 22:30:12 -04:00
Hennadii Stepanov
d2cc339005 qt, refactor: Drop redundant history cleaning in RPC console
The default clearHistory=true argument is passed in the RPCConsole ctor
only. This is needless, as the history and historyPtr members are
initialized properly.
2021-04-22 22:30:05 -04:00
Hennadii Stepanov
4f0ae472e2 qt: Untie irrelevant signal-slot parameters
QAbstractButton::clicked signal has the `checked` parameter that is
irrelevant to RPCConsole::clear slot parameter.
2021-04-22 22:29:57 -04:00
glozow
0c74716c50 [docs] format existing comments as doxygen
Co-authored-by: Xekyo <murch@murch.one>
2021-04-22 13:32:24 -07:00
Jarol Rodriguez
5e7d1997fc doc: note on SDK for macOS depends cross-compile 2021-04-22 16:14:06 -04:00
MarcoFalke
fa1bedb494 test: Add MiniWallet.sendrawtransaction
Can be reviewed with --ignore-all-space --color-moved=dimmed-zebra
2021-04-22 20:02:31 +02:00
practicalswift
54548bae80 net: Avoid calling getnameinfo when formatting IPv6 addresses in CNetAddr::ToStringIP 2021-04-22 15:52:14 +00:00
practicalswift
c10f27fdb2 net: Make IPv6ToString do zero compression as described in RFC 5952 2021-04-22 15:53:53 +00:00
William Bright
71c824ed6c cleaned up and added missing "include" statements for pubkey.cpp and pubkey.h
removed comments next to include statements.

removed comments in include statements.

changed assert.h to cassert based on pr comments
2021-04-22 11:02:14 -04:00
Hennadii Stepanov
8c8237a4a1 net, refactor: Fix style in CConnman::StopNodes 2021-04-22 17:31:06 +03:00
Hennadii Stepanov
229ac1892d net: Combine two loops into one, and update comments 2021-04-22 17:28:39 +03:00
Hennadii Stepanov
a3d090d110 net: Restrict period when cs_vNodes mutex is locked 2021-04-22 17:28:39 +03:00
Jon Atack
65f30e4c21 doc: add -addrinfo troubleshooting section to tor.md 2021-04-22 13:00:21 +02:00
MarcoFalke
4b5659c6b1 Merge bitcoin/bitcoin#19801: test: check for all possible OP_CLTV fail reasons in feature_cltv.py (BIP 65)
b01cd9471f test: check that _all_ invalid-CLTV txs are rejected after BIP65 activation (Sebastian Falbesoner)
dbc1981474 test: check that _all_ invalid-CLTV txs are allowed in a block pre-BIP65 (Sebastian Falbesoner)
8d0ce50c48 test: prepare cltv_invalidate to test all failure reasons in feature_cltv.py (Sebastian Falbesoner)
ce994e1202 test: add tx modfication helper function in feature_cltv.py (Sebastian Falbesoner)

Pull request description:

  The functional test for [BIP65](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki) / `OP_CHECKLOCKTIMEVERIFY` (`feature_cltv.py`) currently only tests one out of five conditions that lead to failure of the op-code -- by prepending the script `OP_1NEGATE OP_CHECKLOCKTIMEVERIFY OP_DROP` to a tx's first input's scriptSig, the case of "_the top item on the stack is less than 0_" is checked:

  f8462a6d27/test/functional/feature_cltv.py (L26-L35)

  This PR adds the other cases (5 in total) by taking an integer argument to the function `cltv_invalidate` that is called in a loop instead of only once per testing scenario. Here is the full list of failure conditions and how they are tested (note that the scriptSig should still be valid before activation of BIP65, when `OP_CLTV` is simply a no-op):
  * _the stack is empty_
  ➡️  prepending `OP_CHECKLOCKTIMEVERIFY` to scriptSig
  * _the top item on the stack is less than 0_
  ➡️  prepending `OP_1NEGATE OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  * _the lock-time type (height vs. timestamp) of the top stack item and the nLockTime field are not the same_
  ➡️  prepending `OPNum(1000) OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  ➡️ setting tx.vin[0].nSequence=0 and tx.nCheckTimeLock=1296688602 (genesis block timestamp)
  * _the top stack item is greater than the transaction's nLockTime field_
  ➡️  prepending `OPNum(1000) OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  ➡️ setting tx.vin[0].nSequence=0 and tx.nCheckTimeLock=500
  * _the nSequence field of the txin is 0xffffffff_
  ➡️  prepending `OPNum(500) OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  ➡️ setting tx.vin[0].nSequence=0xffffffff and tx.nCheckTimeLock=500

  The first commit creates a helper function for the tx modification and also includes some tidying up like turning single-line to multi-line Python imports where necessary and cleaning up some PEP8 warnings. The second commit prepares the invalidation function `cltv_invalidate` and the third and the fourth use it and check for the expected reject reason strings ("Operation not valid with the current stack size", "Negative locktime" and "Locktime requirement not satisfied").

ACKs for top commit:
  MarcoFalke:
    review ACK b01cd9471f 🐣

Tree-SHA512: dd82ae86e2bc4f3ab9bb1cfc9f04e4431b2b59c8aaf2a9f4b28654a1577e003fb43c500f99d76ff57e96262168e1cad7c1a0d71158e4b01063737e8f4be1e07d
2021-04-22 12:58:23 +02:00
MarcoFalke
aaf66413e1 Merge bitcoin/bitcoin#21564: net: Avoid calling getnameinfo when formatting IPv4 addresses in CNetAddr::ToStringIP
58580a827d net: Avoid calling getnameinfo when formatting IPv4 addresses in CNetAddr::ToStringIP (practicalswift)
5858057384 net: Add IPv4ToString (we already have IPv6ToString) (practicalswift)

Pull request description:

  Avoid calling `getnameinfo` when formatting IPv4 addresses in `CNetAddr::ToStringIP`.

ACKs for top commit:
  naumenkogs:
    ACK 58580a827d
  0xB10C:
    ACK 58580a827d
  vasild:
    ACK 58580a827d

Tree-SHA512: 25e3c416acb74908d001baf1cf64c04cbc0d94ce8e7ce5a601f1343062d5d748cb406a3404e6f2b6e7e979c6300b38439e1bfd70ea90ec8c0ec2d7568f09fbcd
2021-04-22 12:29:23 +02:00
Hennadii Stepanov
08f3dbb1b0 test: Bump shellcheck version 2021-04-22 12:14:05 +03:00
Vasil Dimov
9096b13a47 net: remove unnecessary check of CNode::cs_vSend
It is not possible to have a node in `CConnman::vNodesDisconnected` and
its reference count to be incremented - all `CNode::AddRef()` are done
either before the node is added to `CConnman::vNodes` or while holding
`CConnman::cs_vNodes` and the object being in `CConnman::vNodes`.

So, the object being in `CConnman::vNodesDisconnected` and its reference
count being zero means that it is not and will not start to be used by
other threads.

So, the lock of `CNode::cs_vSend` in `CConnman::DisconnectNodes()` will
always succeed and is not necessary.

Indeed all locks of `CNode::cs_vSend` are done either when the reference
count is >0 or under the protection of `CConnman::cs_vNodes` and the
node being in `CConnman::vNodes`.
2021-04-22 11:06:13 +02:00
James O'Beirne
785f9cc46a refactor: init: mark fReset const 2021-04-21 20:18:04 -04:00
Dhruv Mehta
d831e711ca [validation] RewindBlockIndex no longer needed
Instead of rewinding blocks, we request that the user restarts with
-reindex
2021-04-21 16:09:14 -07:00
Amiti Uttarwar
a732ee353c [test] Add tests for addr relay in -blocksonly mode
Co-Authored-By: Martin Zumsande <mzumsande@gmail.com>
2021-04-21 22:34:10 +02:00
Martin Zumsande
a6694eaed8 [test] Add address relay tests involving outbound peers
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-04-21 22:34:03 +02:00
Hennadii Stepanov
e7776e20ed Merge bitcoin-core/gui#286: qt: Bump transifex slug for 22.x
417305991a qt: Bump transifex slug for 22.x (W. J. van der Laan)

Pull request description:

  Opening the 22.x translations early because of experimentation with the new xliff translations format. So change the slug so that the `tx` tool will fetch the right translation.

  In this context, change `file_filter` to use `xlf` as well as the files pulled with `tx pull` are that format now. The setting only affects the naming not the format of the files.

ACKs for top commit:
  hebasto:
    ACK 417305991a, I have reviewed the code and it looks OK, I agree it can be merged. All changes are in accordance with the Transifex [docs](https://docs.transifex.com/client/client-configuration).

Tree-SHA512: e0c18aa5e6cbd4428d24324fee8e5761b70dae51d0236277577aded719798c6a32fc81c0598f280321f2816629e33a334f61f9e7f6180c4074abfda6550cefbe
2021-04-21 23:28:38 +03:00
Martin Zumsande
8188b77c17 [test] Add tests for getaddr behavior
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-04-21 22:16:39 +02:00
Amiti Uttarwar
d2dbfe6ff1 [test] Extract sending an addr message into a helper
Also reduces mocktime to prevent idle disconnects
Co-Authored-By: Martin Zumsande <mzumsande@gmail.com>
2021-04-21 22:16:39 +02:00
Amiti Uttarwar
c991943399 [test] Refactor the addr relay test to prepare for new tests
Moves setting up the addr message into a repeatable function, and breaks up the
existing tests into separate functions for legibility.
2021-04-21 22:14:16 +02:00
W. J. van der Laan
417305991a qt: Bump transifex slug for 22.x
Opening the 22.x translations early because of experimentation with the
new xliff translations format.

In this context, change file_filter to `xlf` as well as the files
pulled with `tx pull` are that format now (the setting only affects the naming
not the format of the files).

Tree-SHA512: e0c18aa5e6cbd4428d24324fee8e5761b70dae51d0236277577aded719798c6a32fc81c0598f280321f2816629e33a334f61f9e7f6180c4074abfda6550cefbe
2021-04-21 13:46:41 +02:00
fanquake
e16f8720dc Merge #21655: build, qt: No longer need to set QT_RCC_TEST=1 for determinism
c799a19b4b build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)

Pull request description:

  The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
  > This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.

  Since #3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.

  Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
  See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](5283a6c87b).

ACKs for top commit:
  fanquake:
    ACK c799a19b4b

Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
2021-04-21 15:16:47 +08:00
fanquake
e507acb118 Merge #20353: configure: Support -fdebug-prefix-map and -fmacro-prefix-map
7abac98d3e configure: Support -f{debug,macro}-prefix-map (Anthony Towns)

Pull request description:

  When bitcoin is checked out in two directories (eg via git worktree) object files between the two will differ due to the full path being included in the debug section. `-fdebug-prefix-map` is used to replace this with "." to avoid this unnecessary difference and allow ccache to share objects between worktrees (provided the source and compile options are the same).

  Also provide `-fmacro-prefix-map` if supported so that the working dir is not encoded in `__FILE__` macros.

ACKs for top commit:
  practicalswift:
    cr ACK 7abac98d3e: patch looks correct
  fanquake:
    ACK 7abac98d3e

Tree-SHA512: b6a37c1728ec3b2e552f244da0e66db113c1e7662c7ac502e12ff466f3dbfbfefae12695ca135137c50dbb1c4c5d84059116c0cd09b391a17466dc77b8726679
2021-04-21 13:26:49 +08:00
MarcoFalke
2bce9329e8 Merge bitcoin/bitcoin#21719: refactor: Add and use EnsureConnman in rpc code
fafb68add5 refactor: Add and use EnsureConnman in rpc code (MarcoFalke)
faabeb854a refactor: Mark member functions const (MarcoFalke)

Pull request description:

  This removes the 10 occurrences of `throw JSONRPCError(RPC_CLIENT_P2P_DISABLED, "Error: Peer-to-peer functionality missing or disabled");` and replaces them with `EnsureConnman`.

ACKs for top commit:
  jarolrod:
    re-ACK fafb68add5
  theStack:
    ACK fafb68add5
  ryanofsky:
    Code review ACK fafb68add5

Tree-SHA512: 84c63cfe31e548645d906f7191a3526c7bea99ed0d54c2a75c2041452a44fe149ede343d8e1943b0e7770816c828bb047dfec8bc541a1f2b89920a126ee54d68
2021-04-21 07:23:34 +02:00
Hennadii Stepanov
f385ad7651 Merge bitcoin-core/gui#263: Revamp context menus
16c157de3c qt, refactor: Use better QMenu::addAction overloaded function (Hennadii Stepanov)
79311750b5 qt: Do not assign Alt+<KEY> shortcuts to context menu actions (Hennadii Stepanov)
963e12058f qt: Drop menu separator that separates nothing (Hennadii Stepanov)
1398a6536c qt, refactor: Make AddressBookPage::deleteAction a local variable (Hennadii Stepanov)

Pull request description:

  This PR:
  1. removes useless `Alt` + `<KEY>` shortcuts from context menu items
  2. replaces 3 lines of code with the only call of [`QMenu::addAction`](https://doc.qt.io/qt-5/qmenu.html#addAction-5) for each context menu item (it became possible since https://github.com/bitcoin/bitcoin/pull/21286 was merged)
  3. makes other minor cleanups

  No behavior change.

ACKs for top commit:
  kristapsk:
    ACK 16c157de3c
  promag:
    Code review ACK 16c157de3c. Nice code cleanup that takes advantage of more recent Qt API.
  jarolrod:
    ACK 16c157de3c

Tree-SHA512: e5555fe957058cc67b351aaf9f09fe3635edb2d07a2223d3093913a25607ae538f0a2fde84c0b0cd43e7475b248949548eb4a5d4b21d8f7391fa2fa8541c04ff
2021-04-20 23:18:54 +03:00
MarcoFalke
bca00942ed Merge bitcoin/bitcoin#21731: Update msvc build to use Qt5.12.10 binaries.
f02ca7a354 Update msvc build to use Qt5.12.10 binaries. (Aaron Clauson)

Pull request description:

  ![bitcoin_qt5 12 10](https://user-images.githubusercontent.com/197660/115268334-12c0e400-a132-11eb-9f59-e2d1e5332842.png)

ACKs for top commit:
  hebasto:
    ACK f02ca7a354, I made a customized AppVeyor build with an artifact: https://ci.appveyor.com/project/hebasto/bitcoin/builds/38786868/artifacts

Tree-SHA512: 0f8998a5b72bca9a08fc5ec5c40b0b79e9247486f58f74824ebf045175d9e3ce7485c16d8de574b7316e79a8433af0646870abf4b0f3b47fc35a92f63a7b5dc9
2021-04-20 19:07:55 +02:00
W. J. van der Laan
90e0faaa44 Merge #21694: build: Use XLIFF file to provide more context to Transifex translators
35d52397e7 Add bitcoin_en.xlf intermediate translation file to the repo (Hennadii Stepanov)
99686b6519 qt [experimental]: Add a translation comment and a disambiguation string (Hennadii Stepanov)
f959b75e8c build: Add Qt lconvert tool to depends (Hennadii Stepanov)
2045e4cdd2 build: Use XLIFF file to provide more context to Transifex translators (Hennadii Stepanov)

Pull request description:

  Currently, only a class name is provided to the Transifex translators as a context. Neither `disambiguation` parameter of the `tr()` function nor [translator comments](https://doc.qt.io/qt-5/i18n-source-translation.html#translator-comments), being included as XML elements to `*.ts` translation files, are not parsed by the Transifex due to its [limited support](https://docs.transifex.com/formats/qt-ts) of such files.

  This PR makes possible to provide all of the context details via an intermediate [XLIFF](https://docs.transifex.com/formats/xliff) translation file.

  With this PR `make -C src translate` produces the `src/qt/locale/bitcoin_en.xlf` file which must be provided to the Transifex as a translation source instead of `src/qt/locale/bitcoin_en.ts`.

  Closes #21465.

  An example translatable string with additional `<context>` and `<note>` XML elements: 35d52397e7/src/qt/locale/bitcoin_en.xlf (L126-L132)

ACKs for top commit:
  laanwj:
    ACK 35d52397e7

Tree-SHA512: cc19e3c09501d240153550d75d7697b5f824cb553f4223beaff66be4d3e6f98d7b5bb14f2d1e1d5ad014eaaa498a7f672e7ff0054ced53ace8c1e6f7e49f6d8a
2021-04-20 16:10:24 +02:00
Hennadii Stepanov
35d52397e7 Add bitcoin_en.xlf intermediate translation file to the repo
As the bitcoin_en.xlf file is created by the `make -C src translate`
command, other translation-related files are also updated.
2021-04-20 15:55:55 +03:00
Hennadii Stepanov
99686b6519 qt [experimental]: Add a translation comment and a disambiguation string
The goal is to see the way the Transifex presents the added items to
translators using an intermediate XLIFF translation file.
2021-04-20 15:49:46 +03:00
Hennadii Stepanov
f959b75e8c build: Add Qt lconvert tool to depends 2021-04-20 15:49:45 +03:00
Hennadii Stepanov
2045e4cdd2 build: Use XLIFF file to provide more context to Transifex translators
Details: https://docs.transifex.com/formats/xliff
2021-04-20 15:48:48 +03:00
W. J. van der Laan
0180453471 Merge #21595: cli: create -addrinfo
06c43201a7 cli: use C++17 std::array class template argument deduction (CTAD) (Jon Atack)
edf3167151 addrinfo: raise helpfully on server error or incompatible server version (Jon Atack)
bb85cbc4f7 doc: add cli -addrinfo release note (Jon Atack)
5056a37624 cli: add -addrinfo command (Jon Atack)
db4d2c282a cli: create AddrinfoRequestHandler class (Jon Atack)

Pull request description:

  While looking at issue #21351, it turned out that the problem was a lack of tor v3 addresses known to the node. It became clear (e.g. https://github.com/bitcoin/bitcoin/issues/21351#issuecomment-811004779) that a CLI command returning the number of addresses the node knows per network (with a tor v2 / v3 breakdown) would be very helpful. This patch adds that.

  `-addrinfo` is useful to see if your node knows enough addresses in a network to use options like `-onlynet=<network>`, or to upgrade to the upcoming tor release that no longer supports tor v2, for which you'll need to be sure your node knows enough tor v3 peers.

  ```
  $ bitcoin-cli --help | grep -A1 addrinfo
    -addrinfo
         Get the number of addresses known to the node, per network and total.

  $ bitcoin-cli -addrinfo
  {
    "addresses_known": {
      "ipv4": 14406,
      "ipv6": 2511,
      "torv2": 5563,
      "torv3": 2842,
      "i2p": 8,
      "total": 25330
    }
  }

  $ bitcoin-cli -addrinfo 1
  error: -addrinfo takes no arguments
  ```

  This can be manually tested, for example, with commands like this:
  ```
  $ bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".onion")) | select(.address | length <= 22)) | .address' | wc -l
  5563
  $ bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".onion")) | select(.address | length > 22)) | .address' | wc -l
  2842
  $ bitcoin-cli getnodeaddresses 0 | jq '.[] | .address' | wc -l
  25330
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 06c43201a7

Tree-SHA512: b668b47718a4ce052aff218789f3da629bca730592c18fcce9a51034d95a0a65f8e6da33dd47443cdd8f60c056c02696db175b0fe09a688e4385a76c1d8b7aeb
2021-04-20 14:36:36 +02:00
MarcoFalke
30a86bb814 Merge bitcoin/bitcoin#21733: build: Re-add command to install vcpkg
de17d245b7 Re-add command to install vcpkg (dplusplus1024)

Pull request description:

  `vcpkg integrate install` must be executed so that msbuild will automatically install external dependencies.

  It was removed in 712f95d332

  It was originally added in 7644567758

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.

  GUI-related pull requests should be opened against
  https://github.com/bitcoin-core/gui
  first. See CONTRIBUTING.md
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  sipsorcery:
    ACK de17d245b7.
  hebasto:
    ACK de17d245b7, I use the same in #21551.

Tree-SHA512: b32428e450cf76297a0e78b5ffb0933ddcfcce010dbd6f5b87d434bf9c437258edb5b817161f6255d5d60523f79f8513a1fe3e92ba62e3fd3786ab042aaae849
2021-04-20 13:09:50 +02:00
MarcoFalke
fa00bb2c5c test: Add missing shift-base:nanobench.h suppression
Otherwise it is not possible to run bench_bitcoin with clang-12 + ASAN
compiled.

Output:

$ src/bench/bench_bitcoin
bench/nanobench.h:1107:15: runtime error: left shift of 4982565676696827473 by 27 places cannot be represented in type 'uint64_t' (aka 'unsigned long')
    #0 0x5623d6a13137 in ankerl::nanobench::Rng::rotl(unsigned long, unsigned int) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/./bench/nanobench.h:1107:15
    #1 0x5623d6a13137 in ankerl::nanobench::Rng::operator()() /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/./bench/nanobench.h:1075:10
    #2 0x5623d6a05c5b in ankerl::nanobench::Rng::Rng(unsigned long) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/./bench/nanobench.h:3135:9
    #3 0x5623d6a0ca51 in ankerl::nanobench::detail::IterationLogic::Impl::Impl(ankerl::nanobench::Bench const&) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/./bench/nanobench.h:2206:13
    #4 0x5623d69f8f73 in ankerl::nanobench::detail::IterationLogic::IterationLogic(ankerl::nanobench::Bench const&) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/./bench/nanobench.h:2215:18
    #5 0x5623d690f165 in ankerl::nanobench::Bench& ankerl::nanobench::Bench::run<AddrManAdd(ankerl::nanobench::Bench&)::$_0>(AddrManAdd(ankerl::nanobench::Bench&)::$_0&&) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/./bench/nanobench.h:1114:28
    #6 0x5623d690e26e in AddrManAdd(ankerl::nanobench::Bench&) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bench/addrman.cpp:76:11
    #7 0x5623d69279d6 in void std::__invoke_impl<void, void (*&)(ankerl::nanobench::Bench&), ankerl::nanobench::Bench&>(std::__invoke_other, void (*&)(ankerl::nanobench::Bench&), ankerl::nanobench::Bench&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:60:14
    #8 0x5623d6927921 in std::enable_if<is_invocable_r_v<void, void (*&)(ankerl::nanobench::Bench&), ankerl::nanobench::Bench&>, void>::type std::__invoke_r<void, void (*&)(ankerl::nanobench::Bench&), ankerl::nanobench::Bench&>(void (*&)(ankerl::nanobench::Bench&), ankerl::nanobench::Bench&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/invoke.h:110:2
    #9 0x5623d692775f in std::_Function_handler<void (ankerl::nanobench::Bench&), void (*)(ankerl::nanobench::Bench&)>::_M_invoke(std::_Any_data const&, ankerl::nanobench::Bench&) /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:291:9
    #10 0x5623d692dbd5 in std::function<void (ankerl::nanobench::Bench&)>::operator()(ankerl::nanobench::Bench&) const /usr/bin/../lib/gcc/x86_64-linux-gnu/10/../../../../include/c++/10/bits/std_function.h:622:14
    #11 0x5623d692cd44 in benchmark::BenchRunner::RunAll(benchmark::Args const&) /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bench/bench.cpp:65:13
    #12 0x5623d69282bf in main /bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bench/bench_bitcoin.cpp:63:5
    #13 0x7f6812010564 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x28564)
    #14 0x5623d685f4dd in _start (/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bench/bench_bitcoin+0x13754dd)

SUMMARY: UndefinedBehaviorSanitizer: invalid-shift-base bench/nanobench.h:1107:15 in

$ clang --version
Ubuntu clang version 12.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
2021-04-20 12:46:06 +02:00
MarcoFalke
00004565cc ci: Use clang-12 for asan task
Changes from Ubuntu Focal to Hirsute to get clang-12.
Generally, it can be expected that more recent compilers have
better support for sanitizers.
2021-04-20 12:46:03 +02:00
MarcoFalke
572b36d4ff Merge bitcoin/bitcoin#21736: doc: Fix doxygen comment silent merge conflict in descriptor.cpp
e5faec65bd doc: Fix doxygen comment silent merge conflict in descriptor.cpp (W. J. van der Laan)

Pull request description:

  It looks like #21238 introduced a silent merge conflict in the documentation, which fails with `-Wdocumentation` in the CI.

  (please merge only if CI passes)

ACKs for top commit:
  ajtowns:
    ACK e5faec65bd -- fixed it for me
  meshcollider:
    ACK e5faec65bd modulo CI

Tree-SHA512: b07d50fd12aa7c239a92aad8ef29f4e88583c3ce701ebedba7c426aac4981c79113adc4670b7d055ab9535a28bdc3f9a30e6ca1b1ed0d7b9a333a3d9c4b40d8a
2021-04-20 10:40:33 +02:00
Jon Atack
06c43201a7 cli: use C++17 std::array class template argument deduction (CTAD)
Credit to João Barbosa (promag) for the suggestions.

Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-04-20 10:35:53 +02:00
W. J. van der Laan
e5faec65bd doc: Fix doxygen comment silent merge conflict in descriptor.cpp
It looks like #21238 introduced a silent merge conflict in the
documentation, which fails with `-Wdocumentation` in the CI.
2021-04-20 09:29:43 +02:00
W. J. van der Laan
906ecb87c8 Merge #21238: A few descriptor improvements to prepare for Taproot support
0b188b751f Clean up context dependent checks in descriptor parsing (Pieter Wuille)
33275a9649 refactor: move uncompressed-permitted logic into ParsePubkey* (Pieter Wuille)
17e006ff8d refactor: split off subscript logic from ToStringHelper (Pieter Wuille)
6ba5dda0c9 Account for key cache indices in subexpressions (Pieter Wuille)
4441c6f3c0 Make DescriptorImpl support multiple subscripts (Pieter Wuille)
a917478db0 refactor: move population of out.scripts from ExpandHelper to MakeScripts (Pieter Wuille)
84f3939ece Remove support for subdescriptors expanding to multiple scripts (Pieter Wuille)

Pull request description:

  These are a few refactors and non-invasive improvements to the descriptors code to prepare for adding Taproot descriptors.

  None of the commits change behavior in any way, except the last one which improves error reporting a bit.

ACKs for top commit:
  S3RK:
    reACK 0b188b7
  Sjors:
    re-ACK 0b188b7
  achow101:
    re-ACK 0b188b751f

Tree-SHA512: cb4e999134aa2bace0e13d4883454c65bcf1369e1c8585d93cc6444ddc245f3def5a628d58af7dab577e9d5a4a75d3bb46f766421fcc8cc5c85c01a11f148b3f
2021-04-20 08:59:09 +02:00
MarcoFalke
de77cbc9d8 Merge bitcoin/bitcoin#21691: test: Check that no versionbits are re-used
fa8eaee6a8 test: Run versionbits_sanity for all chains (MarcoFalke)
faec1e9ee1 test: Address outstanding versionbits_test feedback (MarcoFalke)
fad4167871 test: Check that no versionbits are re-used (MarcoFalke)

Pull request description:

ACKs for top commit:
  jnewbery:
    Code review ACK fa8eaee6a8
  ajtowns:
    ACK fa8eaee6a8 code review only

Tree-SHA512: e99ffcca8970921fd07fa9e04cf1ea2515a317409865d34ddfd70be0f0b0616b29d1fad58262d96a3c3418c0cf7018a6a955802a178b8f78f6ecfaa30a37d91c
2021-04-20 07:15:12 +02:00
fanquake
a839303edc Merge #21244: Move GetDataDir to ArgsManager
bb8d1c6e02 Change ClearDataDirPathCache() to ArgsManager.ClearPathCache(). (Kiminuo)
b4190eff72 Change GetBlocksDir() to ArgsManager.GetBlocksDirPath(). (Kiminuo)
83292e2a70 scripted-diff: Modify unit tests to use the ArgsManager in the BasicTestingSetup class instead of implicitly relying on gArgs. (Kiminuo)
55c68e6f01 scripted-diff: Replace m_args with m_local_args in getarg_tests.cpp (Kiminuo)
511ce3a26b BasicTestingSetup: Add ArgsManager. (Kiminuo)
1cb52ba065 Modify "util_datadir" unit test to not use gArgs. (Kiminuo)
1add318704 Move GetDataDir(fNetSpecific) implementation to ArgsManager. (Kiminuo)
70cdf679f8 Move StripRedundantLastElementsOfPath before ArgsManager class. (Kiminuo)

Pull request description:

  This PR attempts to contribute to "Remove gArgs" (#21005).

  Main changes:

  * `GetDataDir()` function is moved to `ArgsManager.GetDataDirPath()`.
  * `GetBlocksDir()` function is moved to `ArgsManager.GetBlocksDirPath()`.

ACKs for top commit:
  ryanofsky:
    Code review ACK bb8d1c6e02. Just minor const/naming changes and splitting/scripting commits since last review
  MarcoFalke:
    review ACK bb8d1c6e02 📓
  hebasto:
    re-ACK bb8d1c6e02, addressed comments, and two commits made scripted-diffs since my [previous](https://github.com/bitcoin/bitcoin/pull/21244#pullrequestreview-638270583) review.

Tree-SHA512: ba9408c22129d6572beaa103dca0324131766f06d562bb7d6b9e214a0a4d40b0216ce861384562bde24b744003b3fbe6fac239061c8fd798abd3981ebc1b9019
2021-04-20 08:42:56 +08:00
fanquake
67a359313f Merge #21728: remove executable flag for src/net_processing.cpp
f2f2541ee7 remove executable flag for src/net_processing.cpp (Sebastian Falbesoner)

Pull request description:

  The file permissions for `src/net_processing.cpp` have been changed in #21713, as discovered by fanquake (https://github.com/bitcoin/bitcoin/pull/21713#issuecomment-822245960). This PR removes the executable flag again.

ACKs for top commit:
  kiminuo:
    ACK f2f2541ee7 :)
  jnewbery:
    ACK f2f2541ee7
  promag:
    ACK f2f2541ee7.

Tree-SHA512: 1d5a62afb1152029e69fccea2ae53dcb262a91724a5c03dfc4de8c409b280814d0c211c2f9a71f1a6e927f4ed571ba4ac311de9de8ebb797eaf1051674241bdb
2021-04-20 08:11:20 +08:00
dplusplus1024
de17d245b7 Re-add command to install vcpkg
`vcpkg integrate install` must be executed so that msbuild will automatically install external dependencies.

It was removed in 712f95d332

It was originally added in 7644567758
2021-04-19 16:41:34 -06:00
Fabian Jahr
5f96d7d22d rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height 2021-04-19 21:11:37 +02:00
Fabian Jahr
23fe50436b test: Add test for coinstatsindex behavior in reorgs
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
2021-04-19 21:11:37 +02:00
Fabian Jahr
90c966b0f3 rpc: Allow gettxoutsetinfo and getblockstats for stale blocks 2021-04-19 21:11:37 +02:00
Fabian Jahr
b9362392ae index, rpc: Add use_index option for gettxoutsetinfo 2021-04-19 21:11:33 +02:00
Fabian Jahr
bb7788b121 test: Test coinstatsindex robustness across restarts 2021-04-19 20:31:02 +02:00
Fabian Jahr
e0938c2909 test: Add tests for block_info in gettxoutsetinfo
This additional data will automatically be returned if the coinstats index is used.
2021-04-19 20:31:02 +02:00
Fabian Jahr
2501576ecc rpc, index: Add verbose amounts tracking to Coinstats index 2021-04-19 20:31:02 +02:00
Jon Atack
655d929836 test: add coinstatsindex getindexinfo coverage, improve current tests 2021-04-19 20:31:02 +02:00
Fabian Jahr
ca01bb8d68 rpc: Add Coinstats index to getindexinfo 2021-04-19 20:31:02 +02:00
Fabian Jahr
57a026c30f test: Add unit test for Coinstats index 2021-04-19 20:31:02 +02:00
Fabian Jahr
6a4c0c09ab test: Add functional test for Coinstats index 2021-04-19 20:31:01 +02:00
Fabian Jahr
3f166ecc12 rpc: gettxoutsetinfo can be requested for specific blockheights 2021-04-19 20:30:59 +02:00
Fabian Jahr
3c914d58ff index: Coinstats index can be activated with command line flag 2021-04-19 20:28:48 +02:00
Fabian Jahr
dd58a4de21 index: Add Coinstats index
The index holds the values previously calculated in coinstats.cpp
for each block, representing the state of the UTXO set at each
height.
2021-04-19 20:28:48 +02:00
Fabian Jahr
a8a46c4b3c refactor: Simplify ApplyStats and ApplyHash 2021-04-19 20:28:48 +02:00
Fabian Jahr
9c8a265fd2 refactor: Pass hash_type to CoinsStats in stats object 2021-04-19 20:28:48 +02:00
Fabian Jahr
2e2648a902 crypto: Make MuHash Remove method efficient
Division of MuHash objects are very expensive and multiplication relatively cheap. The whole idea of introducing and tracking numerator and denominators seperately as a representation of the internal state was so that divisions would be rare. So using divison in the Remove method did not make any sense and was just a silly mistake which is corrected here.
2021-04-19 20:28:46 +02:00
Aaron Clauson
f02ca7a354 Update msvc build to use Qt5.12.10 binaries. 2021-04-19 16:41:50 +01:00
MarcoFalke
fafb68add5 refactor: Add and use EnsureConnman in rpc code 2021-04-19 13:04:23 +02:00
Hennadii Stepanov
13d27b452d Merge bitcoin-core/gui#276: Elide long strings in their middle in the Peers tab
4e0613369f qt: Elide long strings in their middle in the Peers tab (Hennadii Stepanov)

Pull request description:

  The eliding of long addresses (Onion v3 etc) in the Peers tab in their middle was [discussed](https://github.com/bitcoin-core/gui/issues/262#issuecomment-810490396) in #262.

  On master (f0fa32450e):
  ![DeepinScreenshot_select-area_20210410141435](https://user-images.githubusercontent.com/32963518/114267903-24eea400-9a07-11eb-8c80-99f68d5cc522.png)

  With this PR:
  ![DeepinScreenshot_select-area_20210410140430](https://user-images.githubusercontent.com/32963518/114267796-83675280-9a06-11eb-921f-ca47c2075496.png)

  This PR suggests the minimal diff to achieve the goal. OTOH, this change in behavior is common for all columns in the Peers table, but it seems harmless.

ACKs for top commit:
  jarolrod:
    tACK 4e0613369f
  promag:
    Code review ACK 4e0613369f.

Tree-SHA512: 1d5a62afb1152029e69fccea2ae53dcb262a91724a5c03dfc4de8c409b280814d0c211c2f9a71f1a6e927f4ed571ba4ac311de9de8ebb797eaf1051674241bdb
2021-04-19 13:11:38 +03:00
Russell Yanofsky
615965cfd1 Move common package version code to init/common 2021-04-19 06:11:38 -04:00
Russell Yanofsky
5bed2ab42c Move common logging start code to init/common 2021-04-19 06:11:38 -04:00
Russell Yanofsky
1fb7fcfa52 Move common logging GetArgs code to init/common 2021-04-19 06:11:38 -04:00
Russell Yanofsky
90469c1690 Move common logging AddArg code to init/common 2021-04-19 06:11:38 -04:00
Russell Yanofsky
387c4cf588 Move common sanity check code to init/common 2021-04-19 06:11:38 -04:00
Russell Yanofsky
a67b54855b Move common global init code to init/common 2021-04-19 06:11:38 -04:00
MarcoFalke
cfec4a1dad Merge bitcoin/bitcoin#21280: test: bug fix in transaction_tests
b109bde46a [test] check that mapFlagNames is up to date (glozow)
5d3ced72f9 [test] remove unnecessary OP_1s from invalid tests (glozow)
5aee73d175 [test] minor improvements / followups (glozow)
8a365df558 [test] fix bug in ExcludeIndividualFlags (glozow)
8cac2923f5 [test] remove invalid test from tx_valid.json (glozow)

Pull request description:

  This is a followup to #19698.

  - There was a bug in the `ExcludeIndividualFlags` function which is fixed here.
  - Fixing this bug also showed that there is a test that's supposed to fail (already existing in tx_invalid.json) in tx_valid.json, so I removed it. Other than that, the tests should all pass.
  - Also implements a few suggestions I received offline: removing the `OP_1`s from the invalid tests (similar to 19db590d04), comments, and style.
  - A few other small fixes, like adding asserts, putting all the flags in `mapFlagNames`, better error messages

ACKs for top commit:
  jnewbery:
    utACK b109bde46a

Tree-SHA512: 7233a8c0f1ae1172fac8000ea6e05384ecf79074c39948d118464868505c7f02f17e96503c81bd05c07adb2087648a5d93d9899e16fdefa6b7efcb51319444a9
2021-04-19 11:26:42 +02:00
MarcoFalke
83c715415a Merge #20857: test: update documentation in feature_csv_activation.py
9053b88b1c update docstring in feature_csv_activation.py (Pierre K)

Pull request description:

  These changes in the test documentation reflect the changes introduced in #17921.

ACKs for top commit:
  MarcoFalke:
    review ACK  9053b88

Tree-SHA512: 17fb954baded8dab1c869dd48b76b516150bae616c792c573e4114d4adfdd40195745c56570aa3050cc0015ee496acd7ec178df8ba14831dd22f9722fda84da2
2021-04-19 09:47:57 +02:00
Sebastian Falbesoner
f2f2541ee7 remove executable flag for src/net_processing.cpp 2021-04-19 09:47:34 +02:00
MarcoFalke
d4300a10dd Merge #21679: rpc: Keep default argument value in correct type
bee56c78e9 rpc: Check default value type againts argument type (João Barbosa)
f81ef4303e rpc: Keep default argument value in correct type (João Barbosa)

Pull request description:

  Store default values of RPC arguments in the corresponding type instead of a string. The value is then serialized when the help output is needed. This change simplifies #20017.

  The following examples illustrates how to use the new `RPCArg::Default` and `RPCArg::DefaultHint`:

  ```diff
  - {"verbose", RPCArg::Type::BOOL, /* default */ "false", "True for a json object, false for array of transaction ids"}
  + {"verbose", RPCArg::Type::BOOL, RPCArg::Default(false), "True for a json object, false for array of transaction ids"}
  ```

  ```diff
  - {"nblocks", RPCArg::Type::NUM, /* default */ "one month", "Size of the window in number of blocks"}
  + {"nblocks", RPCArg::Type::NUM, RPCArg::DefaultHint("one month"), "Size of the window in number of blocks"}
  ```

  No behavior change is expected.

ACKs for top commit:
  LarryRuane:
    ACK bee56c78e9
  MarcoFalke:
    ACK bee56c78e9 🦅

Tree-SHA512: c47d78c918e996d36631d4ad3c933b270a34c5b446b8d736be94cf4a0a7b8c0e33d954149ec786cf9550639865b79deb6a130ad044de6030f95aac33f524293a
2021-04-19 09:04:31 +02:00
MarcoFalke
17b51cd5cb Merge #21713: Refactor ProcessNewBlock to reduce code duplication
9a0653553a Refactor ProcessNewBlock to reduce code duplication (R E Broadley)

Pull request description:

  There are probably a few issues with this code (maybe there's even a reason this code is duplicated as it currently is), so apologies in advance that I'm still a little (maybe very) bad with C++

ACKs for top commit:
  MarcoFalke:
    ACK 9a0653553a 💻
  promag:
    Code review ACK 9a0653553a.
  theStack:
    Code-review ACK 9a0653553a 🌴

Tree-SHA512: f8634ffad4b2370204d1a0945db4e27248b9e579d9912784da432b8ee3303cae424fa9f7500000dcfb31e6d29d04a8f7d322d17a6fe3d4adaddd10c539458a8c
2021-04-19 09:01:49 +02:00
W. J. van der Laan
ed133fe005 Merge #21615: script: Add trusted key for hebasto
55d85834cc script: Add trusted key for hebasto (Hennadii Stepanov)

Pull request description:

  It is assumed that my responsibility will be limited to the [GUI repo](https://github.com/bitcoin-core/gui).

ACKs for top commit:
  laanwj:
    ACK 55d85834cc
  MarcoFalke:
    matches the key I have locally ACK 55d85834cc 🍪
  jarolrod:
    ACK 55d85834cc 🥃

Tree-SHA512: 256d03e108c9a14e251340ac6e91234d076778cb6bd551439182176207051f4efc55d396754867e5a7191c8c698610f92016668e163037c67dde56f4136026b8
2021-04-19 08:53:51 +02:00
fanquake
a47ae618a0 Merge #21718: rpc: Improve error message for getblock invalid datatype.
a411494261 rpc: Improve getblock error message for invalid data type. (klementtan)

Pull request description:

  Improve error messages for getblock invalid datatype.

  fixes: #21717

ACKs for top commit:
  instagibbs:
    utACK a411494261
  theStack:
    ACK a411494261
  promag:
    Code review ACK a411494261.

Tree-SHA512: 6e7d8290681e8ab375629f81669d0f8e0c21f9eb7ed9e2455cd19ea013e69b2d95fa7a9ee795315b2d5c60c96035c6cefc3d6e1039a06fd88c1dc7fe275ee6a1
2021-04-19 14:16:19 +08:00
João Barbosa
5f438d66c1 refactor, qt: Simplify SendCoinsDialog::updateCoinControlState 2021-04-18 21:12:57 +01:00
João Barbosa
c5a470eee1 refactor: Drop CCoinControl::SetNull 2021-04-18 21:10:47 +01:00
Pierre K
9053b88b1c update docstring in feature_csv_activation.py
These changes in the test documentation reflect the changes introduced in #17921
2021-04-18 22:05:14 +02:00
Jon Atack
36fb036d25 p2p: allow NetPermissions::ClearFlag() only with PF_ISIMPLICIT
NetPermissions::ClearFlag() is currently only called in the codebase with
an `f` value of NetPermissionFlags::PF_ISIMPLICIT.

If that should change in the future, ClearFlag() should not be called
with `f` being a subflag of a multiflag, e.g. NetPermissionFlags::PF_RELAY
or NetPermissionFlags::PF_DOWNLOAD, as that would leave `flags` in an
invalid state corresponding to none of the existing NetPermissionFlags.

Therefore, allow only calling ClearFlag with the implicit flag for now.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-04-18 16:32:28 +02:00
Jon Atack
4e0d5788ba test: add net permissions noban/download unit test coverage
to clarify/test the relationship and NetPermissions operations
involving the NetPermissionFlags PF_NOBAN and PF_DOWNLOAD.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-04-18 16:32:16 +02:00
Kiminuo
bb8d1c6e02 Change ClearDataDirPathCache() to ArgsManager.ClearPathCache(). 2021-04-18 12:07:00 +02:00
Kiminuo
b4190eff72 Change GetBlocksDir() to ArgsManager.GetBlocksDirPath(). 2021-04-18 11:59:28 +02:00
Kiminuo
83292e2a70 scripted-diff: Modify unit tests to use the ArgsManager in the BasicTestingSetup class instead of implicitly relying on gArgs.
-BEGIN VERIFY SCRIPT-
git ls-files src/test/dbwrapper_tests.cpp src/test/denialofservice_tests.cpp src/test/flatfile_tests.cpp src/test/fs_tests.cpp src/test/settings_tests.cpp src/test/util_tests.cpp | xargs sed -i 's/GetDataDir()/m_args.GetDataDirPath()/g';
-END VERIFY SCRIPT-
2021-04-18 11:59:28 +02:00
Kiminuo
55c68e6f01 scripted-diff: Replace m_args with m_local_args in getarg_tests.cpp
-BEGIN VERIFY SCRIPT-
git ls-files src/test/getarg_tests.cpp | xargs sed -i "s/m_args/m_local_args/g";
-END VERIFY SCRIPT-
2021-04-18 11:59:28 +02:00
Kiminuo
511ce3a26b BasicTestingSetup: Add ArgsManager. 2021-04-18 11:59:28 +02:00
Kiminuo
1cb52ba065 Modify "util_datadir" unit test to not use gArgs. 2021-04-18 11:59:25 +02:00
klementtan
a411494261 rpc: Improve getblock error message for invalid data type. 2021-04-18 12:30:58 +08:00
Jon Atack
300234ab66 doc: update bitcoin.conf maxconnections info 2021-04-17 20:22:43 +02:00
Jon Atack
926827065f doc: update reduce-memory.md peer connections info 2021-04-17 20:17:59 +02:00
MarcoFalke
faabeb854a refactor: Mark member functions const 2021-04-17 20:13:34 +02:00
Kiminuo
1add318704 Move GetDataDir(fNetSpecific) implementation to ArgsManager. 2021-04-17 20:09:01 +02:00
Kiminuo
70cdf679f8 Move StripRedundantLastElementsOfPath before ArgsManager class. 2021-04-17 19:18:10 +02:00
MarcoFalke
0dd7b23489 Merge #21391: [Bundle 5/n] Prune g_chainman usage in RPC modules
586190f0b4 rpc/rest: Take and reuse local Chain/ChainState obj (Carl Dong)
bc3bd36902 rpc: style: Improve BuriedForkDescPushBack signature (Carl Dong)
f99913969f rpc: Remove unnecessary casting of block height (Carl Dong)
6a3d192020 rpc: Tidy up local references (see commit message) (Carl Dong)
038854f31e rest/rpc: Remove now-unused old Ensure functions (Carl Dong)
6fb65b49f4 scripted-diff: rest/rpc: Use renamed EnsureAny*() (Carl Dong)
1570c7ee98 rpc: Add renamed EnsureAny*() functions (Carl Dong)
306b1cd3ee rpc: Add alt Ensure* functions acepting NodeContext (Carl Dong)
d7824acdb9 rest: Use existing NodeContext (Carl Dong)
3f08934799 rest: Pass in NodeContext to rest_block (Carl Dong)
7be0671b95 rpc/rawtx: Use existing NodeContext (Carl Dong)
60dc05afc6 rpc/mining: Use existing NodeContext (Carl Dong)
d485e815e2 rpc/blockchain: Use existing NodeContext (Carl Dong)
d0abf0bf42 rpc/*,rest: Add review-only assertion to EnsureChainman (Carl Dong)
cced0f46c9 miner: Pass in previous CBlockIndex to RegenerateCommitments (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Based on:
  - [x] #21270 | [Bundle 4/n] Prune g_chainman usage in validation-adjacent modules
  - [x] #21525 | [Bundle 4.5/n] Followup fixups to bundle 4

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  	1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  	2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  	3. Remove `old_function`

ACKs for top commit:
  ryanofsky:
    Code review ACK 586190f0b4. Since last review, no changes to existing commits, just some simple new commits added: three new commits renaming std::any Ensure functions (scripted diff commit and manual pre/post commits), and one new commit factoring out a repeated `ActiveChain()` call made in a loop. Thanks for the updates!
  jnewbery:
    utACK 586190f0b4
  MarcoFalke:
    review ACK 586190f0b4 🍯

Tree-SHA512: 64b677fb50141805b55c3f1afe68fcd298f9a071a359bdcd63256d52e334f83e462f31fb3ebee9b630da8f1d912a03a128cfc38179e7aaec29a055744a98478c
2021-04-17 17:37:37 +02:00
MarcoFalke
4a1751a929 Merge bitcoin-core/gui#277: Do not use QClipboard::Selection on Windows and macOS.
7f3a5980c1 qt: Do not use QClipboard::Selection on Windows and macOS. (Hennadii Stepanov)

Pull request description:

  Windows and macOS do [not support](https://doc.qt.io/qt-5/qclipboard.html#notes-for-windows-and-macos-users) the global mouse selection.

  Fixes #258.

ACKs for top commit:
  promag:
    Code review ACK 7f3a5980c1.
  jarolrod:
    ACK 7f3a5980c1

Tree-SHA512: be2beeef7d25af6f4d4a4548325d8d29f08e4342f499666bc4a670ed468a63195d514077c2cd0dba197e12bd43316fd3e2813cdc0954364b6aa4ae6b90c118bf
2021-04-17 13:28:25 +02:00
Jon Atack
b4fcbcfb49 doc: update -maxconnections config option help 2021-04-17 12:33:06 +02:00
MarcoFalke
fa8eaee6a8 test: Run versionbits_sanity for all chains 2021-04-17 11:29:50 +02:00
MarcoFalke
f5e8bcf985 Merge #21689: test: Remove intermittently failing and not very meaningful BOOST_CHECK in cnetaddr_basic
63631beef6 test: Remove intermittently failing and not very meaningful `BOOST_CHECK` in `cnetaddr_basic` (practicalswift)

Pull request description:

  Remove intermittently failing and not very meaningful `BOOST_CHECK` in `cnetaddr_basic`.

  Fixes #21682.

  Rationale from https://github.com/bitcoin/bitcoin/issues/21682#issuecomment-819897122:

  > I've looked at that test before and I don't think that specific `BOOST_CHECK` makes much sense TBH :)
  >
  > 1.) I don't understand why we test if `ToString()` output includes `%zone_index`: it clearly doesn't on some platforms, so we cannot rely on it anyways. Then why test it?
  >
  > 2.) And perhaps more fundamentally: why would we even _want_ to have `%zone_index` in our textual `ToString()` output? I think the expectation is to get say `fe80::1ff:fe23:4567:890a` (without zone index) and not say `fe80::1ff:fe23:4567:890a%eth2 ` or `fe80::1ff:fe23:4567:890a%3 `when doing `ipv6_addr.ToString()` :)

ACKs for top commit:
  MarcoFalke:
    review ACK 63631beef6

Tree-SHA512: 06863d1edfb9ad1ca9bcae09cf3f0f47b58bb29d222b70799c3dc059b96452889026e4b99b132782846d9896e3e798d17c7f9406e0e6a0bec1bffc6edb54e9df
2021-04-17 11:18:51 +02:00
R E Broadley
9a0653553a Refactor ProcessNewBlock to reduce code duplication 2021-04-17 09:55:19 +01:00
MarcoFalke
faec1e9ee1 test: Address outstanding versionbits_test feedback
* https://github.com/bitcoin/bitcoin/pull/21377#discussion_r609585080
* https://github.com/bitcoin/bitcoin/pull/21377#discussion_r613702341
2021-04-17 10:40:47 +02:00
MarcoFalke
fad4167871 test: Check that no versionbits are re-used
This allows to remove check that windows for the same bit are disjoint

This addresses https://github.com/bitcoin/bitcoin/pull/21377#discussion_r611492633
2021-04-17 10:40:43 +02:00
MarcoFalke
8e69370b15 Merge #21712: qa: Test default include_mempool value of gettxout
44dab423eb qa: Test default include_mempool value of gettxout (João Barbosa)

Pull request description:

  With the following diff the functional test would pass. Fix by testing the default value.

  ```diff
  --- a/src/rpc/blockchain.cpp
  +++ b/src/rpc/blockchain.cpp
  @@ -1142,7 +1142,7 @@ static RPCHelpMan gettxout()
       uint256 hash(ParseHashV(request.params[0], "txid"));
       int n = request.params[1].get_int();
       COutPoint out(hash, n);
  -    bool fMempool = true;
  +    bool fMempool = false;
       if (!request.params[2].isNull())
           fMempool = request.params[2].get_bool();
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK 44dab423eb

Tree-SHA512: 14db21b29d6b2c01d1d1278e18a0cf35d6ae566e33e45515d1fe2983dda94ad1ff6065c217601d283f9515cae39b57e981b62ac71ec2002de5359bd8a9e3efa9
2021-04-17 07:50:51 +02:00
João Barbosa
bee56c78e9 rpc: Check default value type againts argument type 2021-04-17 00:22:53 +01:00
João Barbosa
f81ef4303e rpc: Keep default argument value in correct type 2021-04-17 00:22:53 +01:00
João Barbosa
44dab423eb qa: Test default include_mempool value of gettxout 2021-04-16 23:44:49 +01:00
Jon Atack
79685a8992 doc: update -addnode config option help 2021-04-16 20:39:40 +02:00
Jon Atack
2896c6c4cc doc: update addnode rpc help 2021-04-16 20:39:27 +02:00
Hennadii Stepanov
f52fafc935 build: Drop pointless sed commands
Strings that contain moc and rcc versions without timestamps cannot
cause any non-determinism.
2021-04-16 20:22:22 +03:00
fanquake
585cbe2257 Merge #21695: Remove no longer used contrib/bitcoin-qt.pro from the repo
5f2be6e71e Remove no longer used contrib/bitcoin-qt.pro from the repo (Hennadii Stepanov)

Pull request description:

  From [IRC](http://www.erisian.com.au/bitcoin-core-dev/log-2021-04-15.html#l-209):
  > \<hebasto> wumpus: I cannot see any way how the `contrib/bitcoin-qt.pro` is used in the  translation process, neither in the main repo nor in https://github.com/bitcoin-core/bitcoin-maintainer-tools. Besides it looks outdated and unmaintained. May I ask you to confirm/deny my assumption?
  > \<wumpus> hebasto: it is not used for anything, it exists to be able to edit the qt forms in qt designer nothing more
  > \<wumpus> i'm not sure if it is even *necessary* for that, but it is why it is there
  > \<hebasto> wumpus: thanks, qt designer does not need *.pro file at all
  > \<hebasto> maybe qt creator does
  > \<wumpus> feel free to create a PR to remove it, best way to find out if someone wants to keep it, you are right it hasn't been updated in a long time
  > \<hebasto> ok
  > \<wumpus> fwiw, the only question i get about it ever is why it exists
  > \<hebasto> it was in use with `qmake` years ago (what I found digging into the repo history)
  > \<wumpus> yes, that was the original reason, but when we switched to automake it was kept around for use w/ qt's GUI tools
  > \<hebasto> I've noticed it in https://github.com/bitcoin/bitcoin/blame/master/doc/translation_process.md#L25
  > \<wumpus> what it says there is definitely not true anymore

ACKs for top commit:
  laanwj:
    ACK 5f2be6e71e
  jarolrod:
    ACK 5f2be6e71e

Tree-SHA512: 7c105612f28185097fee9e4108b162b4c8b07cc527f4438bdf5bcab08c65421ea301de8584d58770cd113fa871f6781daa8145bd6463278523449e28bfc49d06
2021-04-16 11:08:32 +08:00
Hennadii Stepanov
5f2be6e71e Remove no longer used contrib/bitcoin-qt.pro from the repo 2021-04-15 15:39:42 +03:00
MarcoFalke
c6b30ccb2e Merge #21630: fuzz: split FuzzedSock interface and implementation
549c82ad3a fuzz: use ConsumeBool() instead of !ConsumeBool() (Vasil Dimov)
29ae1c13a5 fuzz: split FuzzedSock interface and implementation (Vasil Dimov)
9668e43d8e fuzz: make FuzzedSock::Wait() sometimes simulate an occurred event (Vasil Dimov)
0c90ff1429 fuzz: set errno from FuzzedSock::Wait() if it simulates a failure (Vasil Dimov)
5198a02de4 style: remove extra white space (Vasil Dimov)

Pull request description:

  * split FuzzedSock interface and implementation
  * make FuzzedSock::Wait() sometimes simulate an occurred event
  * set errno from FuzzedSock::Wait() if it simulates a failure

  (this is a followup from https://github.com/bitcoin/bitcoin/pull/21617)

ACKs for top commit:
  practicalswift:
    cr ACK 549c82ad3a: patch looks correct and touches only `src/test/fuzz/`
  MarcoFalke:
    re-ACK 549c82ad3a only change is rebase 🎬

Tree-SHA512: 8ba965a8319074ad2ef840219c35c77e37cc79f00fb3926f20ccbf5f58e9616f5a3ac96434ad33996b47d292fa760d5d00a529001ac0d1d254262e5df93f616f
2021-04-15 10:48:39 +02:00
MarcoFalke
7cb0bcb681 Merge #21686: Speedy trial activation parameters for Taproot
f979b3237f Add mainnet and testnet taproot activation params (Andrew Chow)

Pull request description:

  Adds the activation parameters for taproot as specified in https://github.com/bitcoin/bips/pull/1104

ACKs for top commit:
  gmaxwell:
    utACK f979b3237f
  ajtowns:
    ACK f979b3237f
  instagibbs:
    ACK f979b32
  clarkmoody:
    ACK f979b32
  Sjors:
    ACK f979b3237f
  jonatack:
    utACK f979b3237f verified with the BIP draft

Tree-SHA512: f95538bcec46c36f9532a99fcf697b143083c25b2427dd578b88514add0a807371530c18f0a8ed040dc885ad6eca8234235e1d762f6f837eafc5daed856a9dcf
2021-04-15 10:19:02 +02:00
MarcoFalke
a5e756b74e Merge #21676: test: Use mocktime to avoid intermittent failure in rpc_tests
fa40d6a1c4 test: Reset mocktime in the common setup (MarcoFalke)
fa78590a8f test: Use mocktime to avoid intermittent failure (MarcoFalke)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/21602#discussion_r611176103

ACKs for top commit:
  jonatack:
    Code review ACK fa40d6a1c4
  jarolrod:
    ACK fa40d6a1c4

Tree-SHA512: 4967e006f3d2c4eb92f03c9086a6abe3190ad54755d251c30d20422c574bb1a154c06f3d5bcb0d4deaa3c4abfd3864d743b71d84897edd358e829bb42233ad12
2021-04-15 10:05:21 +02:00
Vasil Dimov
549c82ad3a fuzz: use ConsumeBool() instead of !ConsumeBool()
The former is shorter and ends up with a "random" bool anyway.
2021-04-15 08:51:39 +02:00
Vasil Dimov
29ae1c13a5 fuzz: split FuzzedSock interface and implementation
Move the `FuzzedSock`'s implementation from `src/test/fuzz/util.h` to
`src/test/fuzz/util.cpp`.

A separate interface and implementation make the code more readable for
consumers who don't need to (better not) know the implementation
details.
2021-04-15 08:51:36 +02:00
Vasil Dimov
9668e43d8e fuzz: make FuzzedSock::Wait() sometimes simulate an occurred event 2021-04-15 08:19:49 +02:00
Vasil Dimov
0c90ff1429 fuzz: set errno from FuzzedSock::Wait() if it simulates a failure 2021-04-15 08:19:48 +02:00
Vasil Dimov
5198a02de4 style: remove extra white space 2021-04-15 08:19:44 +02:00
practicalswift
63631beef6 test: Remove intermittently failing and not very meaningful BOOST_CHECK in cnetaddr_basic 2021-04-15 06:19:30 +00:00
MarcoFalke
9712f75746 Merge #21677: fuzz: Avoid use of low file descriptor ids (which may be in use) in FuzzedSock
6262182b3f Avoid use of low file descriptor ids (which may be in use) in FuzzedSock and StaticContentsSock (practicalswift)

Pull request description:

  Avoid use of low file descriptor ids (which may be in use) in `FuzzedSock`.

  Context: https://github.com/bitcoin/bitcoin/pull/21630/files#r610694541

ACKs for top commit:
  vasild:
    ACK 6262182b3f

Tree-SHA512: e622acb4d01446c3db01adbbbb779038be7247e13f3f4e72c614bc2880c3efd710fd3b189f87abb00f236fa5ddf91f4c215f420ca4eb08a97aaba31593254c3d
2021-04-15 08:02:22 +02:00
Andrew Chow
f979b3237f Add mainnet and testnet taproot activation params 2021-04-14 22:53:54 -04:00
fanquake
2cd834e6c0 Merge #21377: Speedy trial support for versionbits
ffe33dfbd4 chainparams: drop versionbits threshold to 90% for mainnnet and signet (Anthony Towns)
f054f6bcd2 versionbits: simplify state transitions (Anthony Towns)
55ac5f568a versionbits: Add explicit NEVER_ACTIVE deployments (Anthony Towns)
dd07e6da48 fuzz: test versionbits delayed activation (Anthony Towns)
dd85d5411c tests: test versionbits delayed activation (Anthony Towns)
73d4a70639 versionbits: Add support for delayed activation (Anthony Towns)
9e6b65f6fa tests: clean up versionbits test (Anthony Towns)
5932744450 tests: test ComputeBlockVersion for all deployments (Anthony Towns)
63879f0a47 tests: pull ComputeBlockVersion test into its own function (Anthony Towns)

Pull request description:

  BIP9-based implementation of "speedy trial" activation specification, see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-March/018583.html

  Edge cases are tested by fuzzing added in #21380.

ACKs for top commit:
  instagibbs:
    tACK ffe33dfbd4
  jnewbery:
    utACK ffe33dfbd4
  MarcoFalke:
    review ACK ffe33dfbd4 💈
  achow101:
    re-ACK ffe33dfbd4
  gmaxwell:
    ACK ffe33dfbd4
  benthecarman:
    ACK ffe33dfbd4
  Sjors:
    ACK ffe33dfbd4
  jonatack:
    Initial approach ACK ffe33dfbd4 after a first pass of review, building and testing each commit, mostly looking at the changes and diffs. Will do a more high-level review iteration. A few minor comments follow to pick/choose/ignore.
  ariard:
    Code Review ACK ffe33df

Tree-SHA512: f79a7146b2450057ee92155cbbbcec12cd64334236d9239c6bd7d31b32eec145a9781c320f178da7b44ababdb8808b84d9d22a40e0851e229ba6d224e3be747c
2021-04-15 10:04:14 +08:00
fanquake
7fcf53f7b4 Merge #21672: doc: remove boostrap info from GUIX_COMMON_FLAGS doc
09b3e46848 doc: remove boostrap info from GUIX_COMMON_FLAGS doc (fanquake)

Pull request description:

  Passing `ADDITIONAL_GUIX_COMMON_FLAGS="--no-substitutes --bootstrap"` as suggested doesn't work:
  ```bash
        ...outputting in: '/bitcoin/guix-build-a1f0b8b62eb8/output/x86_64-linux-gnu'
            ...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
  guix time-machine: error: bootstrap: unrecognized option
  ```

  and I think bootstrapping is more than covered in the preceding "Choose your security model" section.

ACKs for top commit:
  dongcarl:
    ACK 09b3e46848
  jarolrod:
    ACK 09b3e46848
  laanwj:
    Documentation review ACK 09b3e46848

Tree-SHA512: e533a0b925a2ec091884ec04313f56376b4e85f615c8853dd51840181493d61bd01fee1c019e34880c32d7896e4871ea77e63398ba02b022f614c03bb16531aa
2021-04-15 08:51:14 +08:00
practicalswift
6262182b3f Avoid use of low file descriptor ids (which may be in use) in FuzzedSock and StaticContentsSock 2021-04-14 22:21:17 +00:00
James O'Beirne
931684b24a validation: fix ActivateSnapshot to use hardcoded nChainTx
This fixes an oversight from the move of nChainTx from the user-supplied
snapshot metadata into the hardcoded assumeutxo chainparams.

Since the nChainTx is now unused in the metadata, it should be removed
in a future commit.
2021-04-14 13:29:27 -04:00
Jon Atack
dde69f20a0 p2p, bugfix: use NetPermissions::HasFlag() in CConnman::Bind()
PF_NOBAN is a multi-flag that includes PF_DOWNLOAD, so the conditional
in CConnman::Bind() using a bitwise AND will return the same result
for both the "noban" status and the "download" status.

Example:

`PF_DOWNLOAD` is `0b1000000`
`PF_NOBAN`    is `0b1010000`

This makes a check like `flags & PF_NOBAN` return `true` even if `flags`
is equal to `PF_DOWNLOAD`.

If `-whitebind=download@1.1.1.1:8765` is specified, then `1.1.1.1:8765`
should be added to the list of local addresses. We only want to avoid
adding to local addresses (that are advertised) a whitebind that has a
`noban@` flag.

As a result of a mis-check in `CConnman::Bind()` we would not have added
`1.1.1.1:8765` to the local addresses in the example above.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-04-14 18:06:30 +02:00
Hennadii Stepanov
16c157de3c qt, refactor: Use better QMenu::addAction overloaded function
This overloaded function was introduced in Qt 5.6 and makes code more
concise.
2021-04-14 18:51:26 +03:00
Hennadii Stepanov
79311750b5 qt: Do not assign Alt+<KEY> shortcuts to context menu actions
Such shortcuts are useless as pressing the Alt key closes a context menu
widget immediately.
2021-04-14 18:47:40 +03:00
Hennadii Stepanov
963e12058f qt: Drop menu separator that separates nothing 2021-04-14 18:47:39 +03:00
Hennadii Stepanov
1398a6536c qt, refactor: Make AddressBookPage::deleteAction a local variable 2021-04-14 18:47:30 +03:00
MarcoFalke
fa40d6a1c4 test: Reset mocktime in the common setup
Doing it there will reduce code bloat and also ensure no test can "forget" to reset it
2021-04-14 17:38:07 +02:00
Carl Dong
586190f0b4 rpc/rest: Take and reuse local Chain/ChainState obj
In all rest/rpc-related modules, if there are multiple calls to
ActiveChain{,State}(), and the calls fall under the same ::cs_main lock,
we can simply take a local reference and use/reuse it instead of calling
ActiveChain{,State}() again and again.
2021-04-14 11:17:31 -04:00
Carl Dong
bc3bd36902 rpc: style: Improve BuriedForkDescPushBack signature 2021-04-14 11:13:09 -04:00
Carl Dong
f99913969f rpc: Remove unnecessary casting of block height 2021-04-14 11:13:09 -04:00
Carl Dong
6a3d192020 rpc: Tidy up local references (see commit message)
Organize local variables/references such that:

1. There is always a `ChainstateManager` reference before any `LOCK(cs_main)`.
2. NodeContext references are used with Ensure*() functions introduced in
   previous commit where appropriate to avoid duplicate assertions.
2021-04-14 11:13:09 -04:00
Carl Dong
038854f31e rest/rpc: Remove now-unused old Ensure functions
The scripted-diff in the previous commit should have removed all calls
to functions like: Ensure(?!Any)\(const std::any& (context|ctx)\), so we
can remove them now.
2021-04-14 11:10:08 -04:00
Carl Dong
6fb65b49f4 scripted-diff: rest/rpc: Use renamed EnsureAny*()
-BEGIN VERIFY SCRIPT-
sed -i -E 's@Ensure([^(]+)(\((request\.|)context\))@EnsureAny\1\2@g' \
    -- src/rest.cpp src/rpc/*.cpp
-END VERIFY SCRIPT-
2021-04-14 11:09:34 -04:00
Carl Dong
1570c7ee98 rpc: Add renamed EnsureAny*() functions
- The original Ensure*(const std::any& context) functions are kept and
  the parameter renamed to ctx so that the scripted-diff in the
  subsequent commit will work as expected

- The renaming avoids overloading mistakes arising out of the untyped
  std::any argument.
2021-04-14 10:54:27 -04:00
MarcoFalke
773f8c1a7d Merge #21678: test: Fix TestPotentialDeadLockDetected suppression
f2ef5a8afd test: Fix TSan suppression (Hennadii Stepanov)

Pull request description:

  This PR is a #21669 follow up, and fixes [locally running `make check`](https://github.com/bitcoin/bitcoin/pull/21669#issuecomment-819403540).

ACKs for top commit:
  MarcoFalke:
    cr ACK f2ef5a8afd

Tree-SHA512: bb0c4d1707c6194358d2e9abfed5aa8dd487e014199025fb89f6e5a66d774af041b46a03358a9a5412e1683675c05c42a3b719217d940412ee3fe1ed18a5274c
2021-04-14 16:51:30 +02:00
MarcoFalke
8c867ed4ec Merge #21675: ci: Only cache depends/sdk-sources for macos/apk task in cirrus
11115c8ee5 ci: Only cache depends/sdk-sources for macos/apk task in cirrus (MarcoFalke)

Pull request description:

  Only macos needs the sdk-sources, so move it there (and remove it from showing up in the other tasks)

ACKs for top commit:
  hebasto:
    ACK 11115c8ee5

Tree-SHA512: 9e80e74016052e7b00314e191acc84c5e400bbf2e0d93ee7b1494b6be99e671cc326ddaa1fbded82f0912a9e338ad1891019dce78199b08c5180af42cb59729a
2021-04-14 16:43:00 +02:00
MarcoFalke
a12962ca89 Merge #21585: Fix assumeutxo crash due to truncated file
fa73ce6e65 Fix assumeutxo crash due to truncated file (MarcoFalke)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK fa73ce6e65
  ryanofsky:
    Code review ACK fa73ce6e65. Easy fix. It seems like this could have been caught in review, though.

Tree-SHA512: 3a98687c386e3995114ddf0ad7194fadd9520989290681ef703b578e3ca21aee51eadfb83aa38a489bac13d12709ea137b9b184b08e5bfa2919cca177aab90be
2021-04-14 15:12:14 +02:00
W. J. van der Laan
03ecceedf6 Merge #260: Handle exceptions instead of crash
b8e5d0d3fe qt: Handle exceptions in SendCoinsDialog::sendButtonClicked slot (Hennadii Stepanov)
1ac2bc7ac0 qt: Handle exceptions in TransactionView::bumpFee slot (Hennadii Stepanov)
bc00e13bc8 qt: Handle exceptions in WalletModel::pollBalanceChanged slot (Hennadii Stepanov)
eb6156ba1b qt: Handle exceptions in BitcoinGUI::addWallet slot (Hennadii Stepanov)
f7e260a471 qt: Add GUIUtil::ExceptionSafeConnect function (Hennadii Stepanov)
64a8755af3 qt: Add BitcoinApplication::handleNonFatalException function (Hennadii Stepanov)
af7e365b15 qt: Make PACKAGE_BUGREPORT link clickable (Hennadii Stepanov)

Pull request description:

  This PR is an alternative to https://github.com/bitcoin/bitcoin/pull/18897, and is based on Russ' [idea](https://github.com/bitcoin/bitcoin/pull/18897#pullrequestreview-418703664):
  > IMO it would be nice to have a followup PR that eliminated the one-line forwarding methods ...

  Related issues
  - #91
  - https://github.com/bitcoin/bitcoin/issues/18643

  Qt docs: https://doc.qt.io/qt-5.12/exceptionsafety.html#exceptions-in-client-code

  With this PR the GUI handles the wallet-related exception, and:
  - display it to a user:

  ![Screenshot from 2021-04-01 02-55-59](https://user-images.githubusercontent.com/32963518/113226183-33ff8480-9298-11eb-8fe6-2168834ab09a.png)

  - prints a message to `stderr`:
  ```

  ************************
  EXCEPTION: 18NonFatalCheckError
  wallet/wallet.cpp:2677 (IsCurrentForAntiFeeSniping)
  Internal bug detected: '!chain.findBlock(block_hash, FoundBlock().time(block_time))'
  You may report this issue here: https://github.com/bitcoin/bitcoin/issues

  bitcoin in QPushButton->SendCoinsDialog

  ```

  - writes a message to the `debug.log`
  - and, if the exception is a non-fatal error, leaves the main window running.

ACKs for top commit:
  laanwj:
    Code review ACK b8e5d0d3fe
  ryanofsky:
    Code review ACK b8e5d0d3fe. This is great! I think more improvements are possible but implementation is very clean and I love how targeted each commit is. Changes since last review: adding more explanatory text, making links clickable, reorganizing.

Tree-SHA512: a9f2a2ee8e64b993b0dbc454edcbc39c68c8852abb5dc1feb58f601c0e0e8014dca81c72733aa3fb07b619c6f49b823ed20c7d79cc92088a3abe040ed2149727
2021-04-14 14:17:30 +02:00
Hennadii Stepanov
f2ef5a8afd test: Fix TSan suppression
This change fixes locally running tests.
2021-04-14 13:27:48 +03:00
MarcoFalke
fa78590a8f test: Use mocktime to avoid intermittent failure 2021-04-14 10:05:09 +02:00
MarcoFalke
b8e5bbdf93 Merge #21669: test: Remove spurious double lock tsan suppressions by bumping to clang-12
fadea0bf37 Revert "test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles" (MarcoFalke)
fadbd99885 test: Remove spurious double lock tsan suppressions by bumping to clang-12 (MarcoFalke)

Pull request description:

  The double lock warnings appeared in #19041, but they didn't make any sense. Also, our sync module would detect double locks, if there were any.

  Bumping to clang-12 allows us to remove the spurious suppressions needed to run the tests, so do that.

ACKs for top commit:
  practicalswift:
    cr ACK fadea0bf37 assuming CI passes and more specifically that newer Clang agrees that these TSan suppressions are no longer needed.

Tree-SHA512: c411221a4b74d0af6ca8d686639b4f40b41c15906ccbb6647e8d569d6ab088264faafe075e1ac9523d5c0024b85f15a597bb3eedc7f07d4f5816245f75cfc08b
2021-04-14 09:20:53 +02:00
MarcoFalke
11115c8ee5 ci: Only cache depends/sdk-sources for macos/apk task in cirrus 2021-04-14 09:15:42 +02:00
MarcoFalke
fadea0bf37 Revert "test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles"
I can no longer observe the need for this suppression.

This reverts commit fa1fc536bb.
2021-04-14 06:56:59 +02:00
fanquake
e7af2f35af Merge #21666: Miscellaneous external signer changes
c8f469c6d5 external_signer: remove ExternalSignerException (fanquake)
9e0b199b97 external_signer: use const where appropriate (fanquake)
aaa4e5a45b wallet: remove CWallet::GetExternalSigner() (fanquake)
06a0673351 external_signer: remove ignore_errors from Enumerate() (fanquake)
8fdbb899b8 refactor: unify external wallet runtime errors (fanquake)
f4652bf125 refactor: add missing includes to external signer code (fanquake)
54569cc6d6 refactor: move all signer code inside ENABLE_EXTERNAL_SIGNER #ifdefs (fanquake)

Pull request description:

  These are a few followups after #21467.

ACKs for top commit:
  Sjors:
    tACK c8f469c6d5
  instagibbs:
    utACK c8f469c6d5

Tree-SHA512: 3d5ac5df81680075e71e0e4a7595c520d746c3e37f016cf168c1e10da15541ebb1595aecaf2c08575636e9ff77d499644cae53180232b7049cfae0b923106e4e
2021-04-14 10:08:26 +08:00
fanquake
09b3e46848 doc: remove boostrap info from GUIX_COMMON_FLAGS doc
Passing ADDITIONAL_GUIX_COMMON_FLAGS="--no-substitutes --bootstrap" as
suggested doesn't work:
```bash
      ...outputting in: '/bitcoin/guix-build-a1f0b8b62eb8/output/x86_64-linux-gnu'
          ...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
guix time-machine: error: bootstrap: unrecognized option
```

and I think bootstrapping is more than covered in the preceding "Choose
your security model" section.
2021-04-14 09:08:38 +08:00
MarcoFalke
fadbd99885 test: Remove spurious double lock tsan suppressions by bumping to clang-12 2021-04-13 19:54:39 +02:00
Vasil Dimov
e286cd0d7b net: flag relevant Sock methods with [[nodiscard]] 2021-04-13 17:26:20 +02:00
MarcoFalke
a1f0b8b62e Merge #21634: tests: Skip SQLite fsyncs while testing
41f891da50 tests: Skip SQLite fsyncs while testing (Andrew Chow)

Pull request description:

  Since we want tests to run quickly, and since tests do a lot more db operations than expected we expect to see in actual usage, we disable sqlite's syncing behavior to make db operations run much faster. This syncing behavior is necessary for normal operation as it helps guarantee that data won't become lost or corrupted, but in tests, we don't care about that.

  Fixes #21628

ACKs for top commit:
  vasild:
    ACK 41f891da50

Tree-SHA512: f36f969a182c622691ae5113573a3250e8d367437e83a1a9d3d2b55dd3a9cdf3c6474169a7bd271007bb9ce47f585aa7a6aeae6eebbaeb02d79409b02f47fd8b
2021-04-13 16:31:12 +02:00
fanquake
1f14130cb0 Merge #21575: refactor: Create blockstorage module
fadcd3f78e doc: Remove irrelevant link to GitHub (MarcoFalke)
fa121b628d blockstorage: [refactor] Use chainman reference where possible (MarcoFalke)
fa0c7d9ad2 move-only: Move *Disk functions to blockstorage (MarcoFalke)
fa91b2b2b3 move-only: Move AbortNode to shutdown (MarcoFalke)
fa413f07a1 move-only: Move ThreadImport to blockstorage (MarcoFalke)
faf843c07f refactor: Move load block thread into ChainstateManager (MarcoFalke)

Pull request description:

  This picks up the closed pull request #21030 and is the first step toward fixing #21220.

  The basic idea is to move all disk access into a separate module with benefits:
  * Breaking down the massive files init.cpp and validation.cpp into logical units
  * Creating a standalone-module to reduce the mental complexity
  * Pave the way to fix validation related circular dependencies
  * Pave the way to mock disk access for testing, especially where it is performance critical (like fuzzing)

ACKs for top commit:
  promag:
    Code review ACK fadcd3f78e, checked (almost) moved only changes. This is a nice tidy up change and doesn't change behavior. Easily reviewed commit by commit.
  jamesob:
    ACK fadcd3f78e ([`jamesob/ackr/21575.1.MarcoFalke.refactor_create_blocksto`](https://github.com/jamesob/bitcoin/tree/ackr/21575.1.MarcoFalke.refactor_create_blocksto))
  ryanofsky:
    Code review ACK fadcd3f78e. New organization makes sense, moves extraneous things outside of validation.cpp. PR is also easy to review with helpfully split up moveonly commits.

Tree-SHA512: 917996592b6d8f9998289d8cb2b1b78b23d1fdb3b07216c9caec1380df33baa09dc2c1e706da669d440b497e79c9c62a01ca20dc202df5ad974a75f3ef7a143b
2021-04-13 22:00:28 +08:00
fanquake
88331aa8a7 Merge #21633: refactor: add [[noreturn]] attribute where applicable
003929c0d5 refactor: add [[noreturn]] attribute where applicable (fanquake)

Pull request description:

  Similar to #10843. We could build with `-Wmissing-noreturn`, however that would also mean modifying something like `--suppress-external-warnings` to suppress warnings for leveldb, which I don't think we want to do. In any case, the functions where this is applicable are only added/removed very rarely.

ACKs for top commit:
  vasild:
    ACK 003929c0d5

Tree-SHA512: 33dfa6547d6b84f38a941f24d4c2effe8fde7b93dbc0b27a9309716420e4a879fdbe689d789fa5439d65f5f78292f89fd9dc1b61c97acf69316dfed954086705
2021-04-13 21:17:20 +08:00
fanquake
3aa4935db7 Merge #17934: doc: Use CONFIG_SITE variable instead of --prefix option
223b1ba7d9 doc: Use CONFIG_SITE instead of --prefix (Hennadii Stepanov)

Pull request description:

  The current examples of `--prefix=...` option usage to point `configure` script to appropriate `depends` directory is not [standard](https://www.gnu.org/prep/standards/html_node/Directory-Variables.html). This causes some [confusion](https://github.com/bitcoin/bitcoin/pull/16691) and a bit of inconvenience.

  Consider a CentOS 7 32 bit system. Packages `libdb4-devel`, `libdb4-cxx-devel`, `miniupnpc-devel` and `zeromq-devel` are unavailable from repos. After recommended build with depends:
  ```
  cd depends
  make
  cd ..
  ./autogen.sh
  ./configure --prefix=$PWD/depends/i686-pc-linux-gnu
  make
  ```
  a user is unable to `make install` compiled binaries neither locally (to `~/.local`) nor system-wide (to `/usr/local`) as `--prefix` is set already.

  Meanwhile, the standard approach with using [`config.site`](https://www.gnu.org/software/automake/manual/html_node/config_002esite.html) files allows both possibilities:

  ```
  cd depends
  make
  cd ..
  ./autogen.sh
  CONFIG_SITE=$PWD/depends/i686-pc-linux-gnu/share/config.site ./configure --prefix ~/.local
  make
  make install
  ```

  or

  ```
  CONFIG_SITE=$PWD/depends/i686-pc-linux-gnu/share/config.site ./configure
  make
  sudo make install  # install to /usr/local
  ```

  Moreover, this approach is used in [Gitian descriptors](https://github.com/bitcoin/bitcoin/tree/master/contrib/gitian-descriptors) already.

ACKs for top commit:
  practicalswift:
    ACK 223b1ba7d9: patch looks correct
  fanquake:
    ACK 223b1ba7d9

Tree-SHA512: 46d97924f0fc7e95ee4566737cf7c2ae805ca500e5c49af9aa99ecc3acede4b00329bc727a110aa1b62618dfbf5d1ca2234e736f16fbdf96d6ece5f821712f54
2021-04-13 21:16:04 +08:00
fanquake
c8f469c6d5 external_signer: remove ExternalSignerException
It's not clear why this need it's own exception class, as opposed to just
throwing std::runtime_error().
2021-04-13 20:09:34 +08:00
fanquake
9e0b199b97 external_signer: use const where appropriate 2021-04-13 20:09:34 +08:00
fanquake
aaa4e5a45b wallet: remove CWallet::GetExternalSigner() 2021-04-13 20:09:33 +08:00
fanquake
06a0673351 external_signer: remove ignore_errors from Enumerate()
This is undocumented and unused.
2021-04-13 20:09:33 +08:00
fanquake
8fdbb899b8 refactor: unify external wallet runtime errors
Rather than 3 different messages that are confusing / leak
implementation details, use a single message, that is similar to other
wallet related messages. i.e:
"Compiled without sqlite support (required for descriptor wallets)".
2021-04-13 20:09:33 +08:00
fanquake
f4652bf125 refactor: add missing includes to external signer code 2021-04-13 20:09:33 +08:00
fanquake
54569cc6d6 refactor: move all signer code inside ENABLE_EXTERNAL_SIGNER #ifdefs 2021-04-13 20:09:33 +08:00
MarcoFalke
c1f480fb23 Merge #21653: ci: Fix previous releases cache order
fa4f0b301b ci: Fix previous releases cache order (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fa4f0b301b

Tree-SHA512: 997e46e5432abb1f24c0762dab6366e173a8afd13a02d655691dbe8d6f532f4c5748e0874a7d158d7e171b2991ed2ce9cfc1982a5d9cd30d1dbb30f43452025d
2021-04-13 13:18:48 +02:00
fanquake
f0b457212f Merge #21467: Move external signer out of wallet module
88d4d5ff2f rpc: add help for enumeratesigners and walletdisplayaddress (Sjors Provoost)
b0db187e5b ci: use --enable-external-signer instead of --with-boost-process (Sjors Provoost)
b54b2e7b1a Move external signer out of wallet module (Sjors Provoost)

Pull request description:

  In addition, this PR enables external signer testing on CI.

  This PR moves the ExternalSigner class and RPC methods out of the wallet module.

  The `enumeratesigners` RPC can be used without a wallet since #21417. With additional modifications external signers could be used without a wallet in general, e.g. via `signrawtransaction`.

  The `signerdisplayaddress` RPC is ranamed to `walletdisplayaddress` because it requires wallet context. A future `displayaddress` RPC call without wallet context could take a descriptor argument.

  This commit fixes a `rpc_help.py` failure when configured with `--disable-wallet`.

ACKs for top commit:
  ryanofsky:
    Code review ACK 88d4d5ff2f
  fanquake:
    ACK 88d4d5ff2f

Tree-SHA512: 3242a24e22313aed97eee32a520bfcb1c17495ba32a2b8e06a5e151e2611320e2da5ef35b572d84623af0a49a210d2f9377a2531250868d1a0ccf3e144352a97
2021-04-13 14:35:16 +08:00
MarcoFalke
1f50f0bb38 Merge #21631: i2p: always check the return value of Sock::Wait()
1c1467f51b i2p: cancel the Accept() method if waiting on the socket errors (Vasil Dimov)

Pull request description:

  If `Sock::Wait()` fails, then cancel the `Accept()` method.

  Not checking the return value may cause an uninitialized read a few lines below when we read the `occurred` variable.

  [Spotted](https://github.com/bitcoin/bitcoin/pull/21630#issuecomment-814765659) by MarcoFalke, thanks!

ACKs for top commit:
  laanwj:
    Code review ACK 1c1467f51b
  practicalswift:
    cr ACK 1c1467f51b: patch looks correct and agree with laanwj that `[[nodiscard]]` can be taken in a follow-up PR :)

Tree-SHA512: 57fa8a03a4e055999e23121cd9ed1566a585ece0cf68b74223d8c902804cb6890218c9356d60e0560ccacc6c8542a526356c226ebd48e7b299b4572be312d49b
2021-04-13 06:16:12 +02:00
fanquake
bd65a76b9d Merge #21330: Deal with missing data in signature hashes more consistently
725d7ae049 Use PrecomputedTransactionData in signet check (Pieter Wuille)
497718b467 Treat amount<0 also as missing data for P2WPKH/P2WSH (Pieter Wuille)
3820090bd6 Make all SignatureChecker explicit about missing data (Pieter Wuille)
b77b0cc507 Add MissingDataBehavior and make TransactionSignatureChecker handle it (Pieter Wuille)

Pull request description:

  Currently we have 2 levels of potentially-missing data in the transaction signature hashes:
  * P2WPKH/P2WSH hashes need the spent amount
  * P2TR hashes need all spent outputs (amount + scriptPubKey)

  Missing amounts are treated as -1 (thus leading to unexpected signature failures), while missing outputs in P2TR validation cause assertion failure. This is hard to extend for signing support, and also quite ugly in general.

  In this PR, an explicit configuration option to {Mutable,}TransactionSignatureChecker is added (MissingDataBehavior enum class) to either select ASSERT_FAIL or FAIL. Validation code passes ASSERT_FAIL (as at validation time all data should always be passed, and anything else is a serious bug in the code), while signing code uses FAIL.

  The existence of the ASSERT_FAIL option is really just an abundance of caution. Always using FAIL should be just fine, but if there were for some reason a code path in consensus code was introduced that misses certain data, I think we prefer as assertion failure over silently introducing a consensus change.

  Potentially useful follow-ups (not for this PR, in my preference):
  * Having an explicit script validation error code for missing data.
  * Having a MissingDataBehavior::SUCCEED option as well, for use in script/sign.cpp DataFromTransaction (if a signature is present in a witness, and we don't have enough data to fully validate it, we should probably treat it as valid and not touch it).

ACKs for top commit:
  sanket1729:
    reACK 725d7ae049
  Sjors:
    ACK 725d7ae049
  achow101:
    re-ACK 725d7ae049
  benthecarman:
    ACK 725d7ae049
  fjahr:
    Code review ACK 725d7ae049

Tree-SHA512: d67dc51bae9ca7ef6eb9acccefd682529f397830f77d74cd305500a081ef55aede0e9fa380648c3a8dd4857aa7eeb1ab54fe808979d79db0784ac94ceb31b657
2021-04-13 10:24:31 +08:00
fanquake
003929c0d5 refactor: add [[noreturn]] attribute where applicable 2021-04-13 08:59:21 +08:00
fanquake
89b72ce045 Merge #21661: doc: Fix name of script guix-build
08151e19d9 doc: Fix name of script guix-build (Stephan Oeste)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK 08151e19d9
  jarolrod:
    ACK 08151e19d9

Tree-SHA512: 50e52f91b489db6616b5c9a993474bc1b8c196c3cac4fd5ded7c8fece5a7d72f85d9f566ee6a3df56a132a22a91dd72801ce849ec5e430a7850ff05abcab6b37
2021-04-13 08:16:14 +08:00
Andrew Chow
41f891da50 tests: Skip SQLite fsyncs while testing
Since we want tests to run quickly, and since tests do a lot more db
operations than expected we expect to see in actual usage, we disable
sqlite's syncing behavior to make db operations run much faster. This
syncing behavior is necessary for normal operation as it helps guarantee
that data won't become lost or corrupted, but in tests, we don't care
about that.
2021-04-12 19:29:03 -04:00
Carl Dong
306b1cd3ee rpc: Add alt Ensure* functions acepting NodeContext 2021-04-12 18:25:13 -04:00
W. J. van der Laan
708517b2dd Merge #21663: ci: Fix macOS brew install command
b7381552cd ci: Fix macOS brew install command (Hennadii Stepanov)

Pull request description:

  A solution for https://bintray.com shutdown.

  Details: https://github.com/Homebrew/discussions/discussions/691.

ACKs for top commit:
  jamesob:
    ACK b7381552cd

Tree-SHA512: 4570c297421ae66043348acbe2f1e50b000e700406b38bc813c80714d4f109c6cadecff571ecbd8c2f0094ebdc959486fc55022ba6545bd592683a421e9c0655
2021-04-12 23:32:03 +02:00
Jon Atack
edf3167151 addrinfo: raise helpfully on server error or incompatible server version 2021-04-12 22:12:41 +02:00
Hennadii Stepanov
b7381552cd ci: Fix macOS brew install command
Details: https://github.com/Homebrew/discussions/discussions/691
2021-04-12 22:30:55 +03:00
fdov
36c10b9f4b build,boost: update download url.
- bintray is closing.
    - updated to jfrog.io.
2021-04-12 20:11:11 +02:00
Stephan Oeste
08151e19d9 doc: Fix name of script guix-build 2021-04-12 19:55:01 +02:00
Anthony Towns
ffe33dfbd4 chainparams: drop versionbits threshold to 90% for mainnnet and signet 2021-04-12 12:59:04 +10:00
Anthony Towns
f054f6bcd2 versionbits: simplify state transitions
This removes the DEFINED->FAILED transition and changes the
STARTED->FAILED transition to only occur if signalling didn't pass the
threshold. This ensures that it is always possible for activation to
occur, no matter what settings are chosen, or the speed at which blocks
are found.
2021-04-12 11:14:49 +10:00
Anthony Towns
55ac5f568a versionbits: Add explicit NEVER_ACTIVE deployments
Previously we used deployments that would timeout prior to Bitcoin's
invention, which allowed the deployment to still be activated in unit
tests. This switches those deployments to be truly never active.
2021-04-12 11:14:49 +10:00
Anthony Towns
dd07e6da48 fuzz: test versionbits delayed activation 2021-04-12 11:14:49 +10:00
Anthony Towns
dd85d5411c tests: test versionbits delayed activation 2021-04-12 11:14:49 +10:00
Anthony Towns
73d4a70639 versionbits: Add support for delayed activation 2021-04-12 11:14:49 +10:00
Anthony Towns
9e6b65f6fa tests: clean up versionbits test
Simplify the versionbits unit test slightly to make the next set of
changes a little easier to follow.
2021-04-12 10:47:42 +10:00
Anthony Towns
5932744450 tests: test ComputeBlockVersion for all deployments
This generalises the ComputeBlockVersion test so that it can apply to
any activation parameters we might set, and checks all the parameters
set for each deployment on each chain, to simultaneously ensure that the
deployments we have configured work sensibly, and that the test code
does not suffer bitrot in the event that all interesting deployments
are buried.
2021-04-12 10:47:42 +10:00
Anthony Towns
63879f0a47 tests: pull ComputeBlockVersion test into its own function
The intent here is to allow checking ComputeBlockVersion behaviour with
each deployment, rather than only testdummy on mainnet. This commit does
the trivial refactoring component of that change.
2021-04-12 10:44:04 +10:00
sgulls
b353633bf4 build: mac_alias 2.2.0
Fix make deploy for arm64-darwin
2021-04-11 16:38:33 -04:00
Hennadii Stepanov
c799a19b4b build, qt: No longer need to set QT_RCC_TEST=1 for determinism
Since Qt 5.3.1 hash seeding is disabled for rcc.
See commit 5283a6c87beac5a43f612786fefd6e43f2c70bf6.
2021-04-11 17:29:15 +03:00
MarcoFalke
fa4f0b301b ci: Fix previous releases cache order
The order was broken in commit ffff4e7373
2021-04-11 13:55:33 +02:00
MarcoFalke
f6c44e999b Merge #21602: rpc: add additional ban time fields to listbanned
d3b0b08b0f doc: release notes for new listbanned fields (Jarol Rodriguez)
60290d3f5e test: increase listbanned unit test coverage (Jon Atack)
3e978d1a5d rpc: add time_remaining field to listbanned (Jarol Rodriguez)
5456b34531 rpc: add ban_duration field to listbanned (Jarol Rodriguez)
c95c61657a doc: improve listbanned help (Jarol Rodriguez)
dd3c8eaa33 rpc: swap position of banned_until and ban_created fields (Jarol Rodriguez)

Pull request description:

  This PR adds a `ban_duration` and `time_remaining` field to the `listbanned` RPC command. Thanks to jonatack, this PR also expands the `listbanned` test coverage to include these new fields

  It's useful to keep track of `ban_duration` as this is another data point on which to sort banned peers. I found this helpful in adding additional context columns to the GUI `bantablemodel` as part of a follow-up PR. As [suggested](https://github.com/bitcoin/bitcoin/pull/21602#issuecomment-813486134) by jonatack, `time_remaining` is another useful user-centric data point.

  Since a ban always expires after its created, the `ban_created` field is now placed before the `banned_until` field. This new ordering is more logical.

  This PR also improves the `help listbanned` output by providing additional context to the descriptions of the `address`, `ban_created`, and `banned_until` fields.

  **Master: listbanned**
  ```
  [
    {
      "address": "1.2.3.4/32",
      "banned_until": 1617691101,
      "ban_created": 1617604701
    },
    {
      "address": "135.181.41.129/32",
      "banned_until": 1649140716,
      "ban_created": 1617604716
    }
  ]
  ```

  **PR: listbanned**
  ```
  [
    {
      "address": "1.2.3.4/32",
      "ban_created": 1617775773,
      "banned_until": 1617862173,
      "ban_duration": 86400,
      "time_remaining": 86392
    },
    {
      "address": "3.114.211.172/32",
      "ban_created": 1617753165,
      "banned_until": 1618357965,
      "ban_duration": 604800,
      "time_remaining": 582184
    }
  ]
  ```

ACKs for top commit:
  jonatack:
    re-ACK d3b0b08b0f
  hebasto:
    ACK d3b0b08b0f, tested on Linux Mint 20.1 (x86_64).
  MarcoFalke:
    review ACK d3b0b08b0f 🕙

Tree-SHA512: 5b83ed2483344e546d57e43adc8a1ed7a1fff292124b14c86ca3a1aa2aec8b0f7198212fabff2c5145e7f726ca04ae567fe667b141254c7519df290cf63774e5
2021-04-11 13:36:29 +02:00
MarcoFalke
1e3db6807d Merge #21619: ci: Run self-hosted ci
fa41a91735 ci: Run self-hosted ci (MarcoFalke)
fa52a40f0e ci: Make cirrus cache folders relative to cirrus base dir (MarcoFalke)
fa278412a0 ci: Restart docker before run (MarcoFalke)
fad4f48e07 ci: [refactor] Create setting for ephemeral config in .cirrus.yml (MarcoFalke)

Pull request description:

  Due to our heavy use of the Cirrus CI community cluster, some tasks may take a long time to get scheduled. While it is possible to use "Compute Credits" to get immediately scheduled on the cluster, I couldn't find a sponsor that'd be willing to cover the total cost, if all tasks were paid for with credits.

  However, it is also possible to bring our own runners to Cirrus CI.

  For testing purposes, a single task will be transformed to run on the DrahtBot infrastructure. If all goes well, the other tasks can be moved, too.

ACKs for top commit:
  hebasto:
    ACK fa41a91735, I have reviewed the code and it looks OK.

Tree-SHA512: 513738daec36da8cd48a8f11e687ff0b7dfaba1ae4ed2fa77e7b043f88fd52bf5c0dbad2768e13df88518323917f08348cb62be6376a423142921f8d2c59a938
2021-04-11 09:51:22 +02:00
fanquake
dd01dc6a1d Merge #21643: Ignore guix builds
8e84c1872c Ignore guix builds (Hennadii Stepanov)

Pull request description:

  This PR is a #21375 follow up.

ACKs for top commit:
  sipa:
    ACK 8e84c1872c

Tree-SHA512: 71d1c5acac3382f232074a025445d6d6660cb99e53233fade9ab29ad95b56da44e4e42e44411cfef175a0a8f3800633779ad1d24c2cfdcbc1a36239d38d5b8c3
2021-04-11 09:51:19 +08:00
Hennadii Stepanov
7f3a5980c1 qt: Do not use QClipboard::Selection on Windows and macOS.
Windows and macOS do not support the global mouse selection.
2021-04-10 21:34:38 +03:00
Hennadii Stepanov
4e0613369f qt: Elide long strings in their middle in the Peers tab 2021-04-10 14:07:13 +03:00
MarcoFalke
f0fa32450e Merge #21606: fuzz: Extend psbt fuzz target a bit
faaf3954e2 fuzz: Extend psbt fuzz target a bit (MarcoFalke)

Pull request description:

  Previously it only merged the psbt with itself, now it tries to merge another.

ACKs for top commit:
  practicalswift:
    Tested ACK faaf3954e2

Tree-SHA512: e1b1d31a47d35e1767285bc2fda176c79cb0550d6d383fe467104272e61e1c83f6cbc0c7d6bbc0c3027729eec13ae1f289f8950117ee91e0fb3703e66d5e6918
2021-04-09 18:54:17 +02:00
Hennadii Stepanov
8e84c1872c Ignore guix builds 2021-04-09 17:57:58 +03:00
MarcoFalke
fa41a91735 ci: Run self-hosted ci 2021-04-09 15:56:56 +02:00
MarcoFalke
fa52a40f0e ci: Make cirrus cache folders relative to cirrus base dir 2021-04-09 15:56:29 +02:00
MarcoFalke
fa278412a0 ci: Restart docker before run
Also, add setting for persistent worker in .cirrus.yml
2021-04-09 15:56:23 +02:00
MarcoFalke
fad4f48e07 ci: [refactor] Create setting for ephemeral config in .cirrus.yml
This allows easier switching between self-hosted runners and the
community cluster. Also, named variables can be documented better.
2021-04-09 15:56:13 +02:00
MarcoFalke
faaf3954e2 fuzz: Extend psbt fuzz target a bit 2021-04-09 13:17:37 +02:00
Hennadii Stepanov
223b1ba7d9 doc: Use CONFIG_SITE instead of --prefix 2021-04-09 12:25:18 +03:00
Jon Atack
bb85cbc4f7 doc: add cli -addrinfo release note 2021-04-09 09:02:09 +02:00
Jon Atack
5056a37624 cli: add -addrinfo command 2021-04-09 09:02:07 +02:00
Jon Atack
db4d2c282a cli: create AddrinfoRequestHandler class 2021-04-09 09:02:04 +02:00
MarcoFalke
4ad83a9597 Merge #21592: test: Remove option to make TestChain100Setup non-deterministic
fa6183d776 test: Remove option to make TestChain100Setup non-deterministic (MarcoFalke)
fa732bccb3 test: Use compressed keys in TestChain100Setup (MarcoFalke)

Pull request description:

  Seems odd to have an option for non-deterministic tests
  when the goal should be for all tests to be deterministic.

ACKs for top commit:
  jamesob:
    ACK fa6183d776
  practicalswift:
    cr ACK fa6183d776: patch looks deterministic!

Tree-SHA512: 6897a9f36e0dfb7d63b25dd6984414b3ee8a62458ad232cb21ed5077184fdb0bc626996e4ac84ef0bdd452b9f17c54aac75a71575b8e723b84cac07c9f9d5611
2021-04-09 07:43:10 +02:00
fanquake
265a3a774b Merge #21445: cirrus: Use SSD cluster for speedup
fa212391ce cirrus: Use SSD cluster for speedup (MarcoFalke)

Pull request description:

  Haven't tested, but this might be faster: https://twitter.com/fedor/status/1354505744293502980

ACKs for top commit:
  fanquake:
    ACK fa212391ce - going to merge this now given there is a speedup, and it's enough to fix the fuzz task the is continually timing out.

Tree-SHA512: b24161ba2ed16fcf23ac6098100b04f7f6eb8936bb312a35fcd8e632568b877bfc09a1e522836fe298a2cd87a53a91e3b501a7f2e1c81cc0edb57c59aecffb0d
2021-04-09 09:53:38 +08:00
Carl Dong
fa872c9af3 depends: Fix id_string invocations
We now use a script named gen_id to generate the base build_id/host_id.
This solves 2 problems:

1. GNU Make special-casing exit code 127 (command not found) meant that
   warnings about missing tools would propagate to the user's terminal
   and broke our opportunistic build_id construction.
2. This change ensures that we don't have arbitrary characters in our
   make variables that would be misinterpreted by Make.

See comments in depends/Makefile and depends/gen_id for more
information.
2021-04-08 20:17:20 -04:00
W. J. van der Laan
0c9597ce7d Merge #21304: guix: Add guix-clean script + establish gc-root for container profiles
867a5e172a guix: Register garbage collector root for containers (Carl Dong)
8f8b96fb54 guix: Update hint messages to mention guix-clean (Carl Dong)
44f6d4f56b guix: Record precious directories and add guix-clean (Carl Dong)
84912d4b24 build: Remove spaces from variable-printing rules (Carl Dong)

Pull request description:

  ```
  guix: Record precious directories and add guix-clean

  Many users have reported problems that stem from having an unclean
  working tree. To that end, I've written a guix-clean script which should
  help reset the working tree while respecting user-specified precious
  directories.

  Precious directories, such as:

  - SOURCES_PATH
  - BASE_CACHE
  - SDK_PATH
  - OUTDIR

  Should be preserved when cleaning the working tree, and are thus
  recorded in ./contrib/guix/var/precious_dirs.

  The ./contrib/guix/guix-clean script is able to parse that file and make
  sure to avoid them when cleaning out the working tree.
  ```

ACKs for top commit:
  laanwj:
    ACK 867a5e172a

Tree-SHA512: c498fad781ff5e6406639df2b91b687fc528273fdf266bcdba8f6eec3b3b37ecce544b6da0252f0b9c6717f9d88e844e4c7b72d1877bdbabfc6871ddd0172af5
2021-04-08 23:19:54 +02:00
Jarol Rodriguez
d3b0b08b0f doc: release notes for new listbanned fields 2021-04-08 13:21:30 -04:00
Sjors Provoost
88d4d5ff2f rpc: add help for enumeratesigners and walletdisplayaddress 2021-04-08 17:56:00 +02:00
Sjors Provoost
b0db187e5b ci: use --enable-external-signer instead of --with-boost-process
An earlier version of #16546 used both --with-boost-process and --enable-external-signer, which was simplified to only use the latter. However I forgot to update CI, so the external signer tests were not run.
2021-04-08 17:56:00 +02:00
Sjors Provoost
b54b2e7b1a Move external signer out of wallet module
This commit moves the ExternalSigner class and RPC methods out of the wallet module.

The enumeratesigners RPC can be used without a wallet since #21417.
With additional modifications external signers could be used without a wallet in general, e.g. via signrawtransaction.

The signerdisplayaddress RPC is ranamed to walletdisplayaddress because it requires wallet context.
A future displayaddress RPC call without wallet context could take a descriptor argument.

This commit fixes a rpc_help.py failure when configured with --disable-wallet.
2021-04-08 17:56:00 +02:00
Vasil Dimov
1c1467f51b i2p: cancel the Accept() method if waiting on the socket errors 2021-04-08 16:31:55 +02:00
MarcoFalke
fa212391ce cirrus: Use SSD cluster for speedup 2021-04-08 11:28:06 +02:00
MarcoFalke
6664211be2 Merge #21574: Drop JSONRPCRequest constructors after #21366
9044522ef7 Drop JSONRPCRequest constructors after #21366 (Russell Yanofsky)

Pull request description:

  This just makes an additional simplification after #21366 replaced
  util::Ref with std::any. It was originally suggested
  https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351 but
  delayed for a followup. It would have prevented usage bug
  https://github.com/bitcoin/bitcoin/pull/21572.

ACKs for top commit:
  promag:
    ACK 9044522ef7, fixed conflict in src/wallet/interfaces.cpp.

Tree-SHA512: e909411b8f75013620b94e1a609296befb832fdcb574cd2e6689bfe3c636b03cd4ac1ccb2b32b532daf0f2131bb043464024966310fffc7e3cad77713d4bd0ef
2021-04-08 09:08:03 +02:00
MarcoFalke
fa6183d776 test: Remove option to make TestChain100Setup non-deterministic
Seems odd to have an option for non-deterministic tests
when the goal should be for all tests to be deterministic.

Can be reviewed with `--ignore-all-space`.
2021-04-08 08:59:00 +02:00
MarcoFalke
fa732bccb3 test: Use compressed keys in TestChain100Setup
coinbaseKey.MakeNewKey(true); creates a compressed key and there is no reason
for the deterministic setup to use uncompressed ones.
2021-04-08 08:58:44 +02:00
glozow
b109bde46a [test] check that mapFlagNames is up to date
There is no way to iterate through all script verification flags, and
it's not guaranteed that every power of 2 is used. Just make sure that
all flags in STANDARD_SCRIPT_VERIFY_FLAGS are present in mapFlagNames;
this covers all consensus and policy flags. If mapFlagNames has more
flags than STANDARD_SCRIPT_VERIFY_FLAGS, that's okay. Nonexistent flags
will be caught by the compiler.
2021-04-07 19:00:23 -07:00
glozow
5d3ced72f9 [test] remove unnecessary OP_1s from invalid tests
Similar to 19db590d04, which removed these
for the valid tests. Not removing ones that cause a false/empty stack
error because these tests should fail due to being invalid with CSV/CLTV
2021-04-07 19:00:17 -07:00
glozow
5aee73d175 [test] minor improvements / followups
Add missing script verify flags to mapFlagNames.
iterate through mapFlagNames values instead of bits.

BOOST_CHECK_MESSAGE better reports which test failed exactly, whereas
BOOST_ERROR was just incrementing the error counter.
2021-04-07 19:00:17 -07:00
glozow
8a365df558 [test] fix bug in ExcludeIndividualFlags
PR #19168 introduced this function but it always returns an empty vector.
2021-04-07 19:00:17 -07:00
fanquake
2e9031f95d Merge #21626: doc: Fix typos from codespell
94c7dd9ac8 doc: Fix typos from codespell lint (Yerzhan Mazhkenov)

Pull request description:

  Typos from codespell linter: https://cirrus-ci.com/task/6677401661865984?logs=lint#L856
  - txrequest.cpp: `annoucements` ==> `announcements`
  - contrib/guix/README.md:298: `stil` ==> `still`
  - contrib/guix/guix-build:18: `invokable` ==> `invocable`
  - contrib/guix/libexec/prelude.bash:12: `invokable` ==> `invocable`
  - src/test/fuzz/tx_pool.cpp:37: `acess` ==> `access`
  - src/txorphanage.h:29: `orginating` ==> `originating`

ACKs for top commit:
  practicalswift:
    cr ACK 94c7dd9ac8: thnaks fro fiixng tpyos!
  jarolrod:
    ACK 94c7dd9ac8

Tree-SHA512: e0fac462a2f9e68b6a161c9f5d95b4d0648ce5c618fd7cd243d57db8f0256138b8823b166ea406b21e95586eae43047df1ef0df04616858082a39c1d1eb13a86
2021-04-08 08:16:04 +08:00
Carl Dong
867a5e172a guix: Register garbage collector root for containers
By registering the container profiles as garbage collector roots, it
will prevent `guix gc` from garbage collecting derivations which our
container needs and inconvieniencing the user with a rebuild.
2021-04-07 15:21:00 -04:00
Carl Dong
8f8b96fb54 guix: Update hint messages to mention guix-clean 2021-04-07 15:20:25 -04:00
Yerzhan Mazhkenov
94c7dd9ac8 doc: Fix typos from codespell lint 2021-04-07 19:26:25 +01:00
W. J. van der Laan
cb79cabdd9 Merge #21594: rpc: add network field to getnodeaddresses
5c446784b1 rpc: improve getnodeaddresses help (Jon Atack)
1b9189866a rpc: simplify/constify getnodeaddresses code (Jon Atack)
3bb6e7b655 rpc: add network field to rpc getnodeaddresses (Jon Atack)

Pull request description:

  This patch adds a network field to RPC `getnodeaddresses`, which is useful on its own, particularly with the addition of new networks like I2P and others in the future, and which I also found helpful for adding a new CLI command as a follow-up to this pull that calls `getnodeaddresses` and needs to know the network of each address.

  While here, also improve the `getnodeaddresses` code and help.

  ```
  $ bitcoin-cli -signet getnodeaddresses 3
  [
    {
      "time": 1611564659,
      "services": 1033,
      "address": "2600:1702:3c30:734f:8f2e:744b:2a51:dfa5",
      "port": 38333,
      "network": "ipv6"
    },
    {
      "time": 1617531931,
      "services": 1033,
      "address": "153.126.143.201",
      "port": 38333,
      "network": "ipv4"
    },
    {
      "time": 1617473058,
      "services": 1033,
      "address": "nsgyo7begau4yecc46ljfecaykyzszcseapxmtu6adrfagfrrzrlngyd.onion",
      "port": 38333,
      "network": "onion"
    }
  ]

  $ bitcoin-cli help getnodeaddresses
  getnodeaddresses ( count )

  Return known addresses, which can potentially be used to find new nodes in the network.

  Arguments:
  1. count    (numeric, optional, default=1) The maximum number of addresses to return. Specify 0 to return all known addresses.

  Result:
  [                         (json array)
    {                       (json object)
      "time" : xxx,         (numeric) The UNIX epoch time when the node was last seen
      "services" : n,       (numeric) The services offered by the node
      "address" : "str",    (string) The address of the node
      "port" : n,           (numeric) The port number of the node
      "network" : "str"     (string) The network (ipv4, ipv6, onion, i2p) the node connected through
    },
    ...
  ]
  ```
  Future idea: allow passing `getnodeaddresses` a network (or networks) as an argument to return only addresses in that network.

ACKs for top commit:
  laanwj:
    Tested ACK 5c446784b1
  jarolrod:
    re-ACK 5c446784b1
  promag:
    Code review ACK 5c446784b1.

Tree-SHA512: ab0101f50c76d98c3204133b9f2ab6b7b17193ada31455ef706ad11afbf48f472fa3deb33e96028682369b35710ccd07d81863d2fd55c1485f32432f2b75efa8
2021-04-07 18:56:01 +02:00
fanquake
a5491882a0 build: fix configuring when building depends with NO_BDB=1
Currently, if you build depends using `NO_BDB=1` (only sqlite wallets),
./configure will fail as it still tries to find bdb. i.e:
```bash
checking for Berkeley DB C++ headers... default
configure: error: Found Berkeley DB other than 4.8, required for portable BDB wallets (--with-incompatible-bdb to ignore or --without-bdb to disable BDB wallet support)
```

This PR fixes the build such that you can build depends, opting out of
bdb without opting out of wallets entirely, and still configure
successfully.
2021-04-07 20:50:10 +08:00
Jon Atack
5c446784b1 rpc: improve getnodeaddresses help 2021-04-07 12:57:11 +02:00
Jon Atack
1b9189866a rpc: simplify/constify getnodeaddresses code 2021-04-07 12:57:09 +02:00
Jon Atack
3bb6e7b655 rpc: add network field to rpc getnodeaddresses 2021-04-07 12:57:07 +02:00
MarcoFalke
aa69471ecd Merge #21572: Fix wrong wallet RPC context set after #21366
937fd4a66f Fix wrong wallet RPC context set after #21366 (Russell Yanofsky)

Pull request description:

  This bug doesn't have any effects currently because it only affects
  external signer RPCs which aren't currently using the wallet context,
  but it does cause an appveyor failure in a upcoming PR:

  https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/38512882

  This bug is subtle and could have been avoided if JSONRPCRequest didn't
  have constructors that were so loose with type checking.  Suggested
  change
  https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351
  eliminates these and would be a good followup for a future PR.

  This PR just implements the simplest possible fix.

ACKs for top commit:
  theStack:
    Code-review ACK 937fd4a66f
  meshcollider:
    Code review ACK 937fd4a66f

Tree-SHA512: 53e6265ed6c7abb47d2b3e77d1604edfeb993c3a2440f0c19679cfeb23516965e6707ff486196a0acfbeff21c79a9a08b5cd33bae9a232d33d0134bca1bd0ff3
2021-04-07 10:53:26 +02:00
Russell Yanofsky
9044522ef7 Drop JSONRPCRequest constructors after #21366
This just makes an additional simplification after #21366 replaced
util::Ref with std::any. It was originally suggested
https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351 but
delayed for a followup. It would have prevented usage bug
https://github.com/bitcoin/bitcoin/pull/21572.
2021-04-07 04:53:26 -04:00
fanquake
2b3e5bf4c0 Merge #21613: build: enable -Wdocumentation
a4e970adb6 build: enable -Wdocumentation if suppressing external warnings (fanquake)
3b0078f958 doc: fixup -Wdocumentation issues (fanquake)
c6edcf1c71 build: suppress libevent warnings if supressing external warnings (fanquake)

Pull request description:

  Enable `-Wdocumentation` by taking advantage of our `--enable-suppress-external-warnings` flag. Most of the CIs are using this flag now, so any regressions should be caught.

  This also required modifying libevents flags when suppressing warnings, as depending on the version being built against, that could generate a large number of warnings. i.e:
  ```bash
  In file included from httpserver.cpp:34:
  In file included from ./support/events.h:12:
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:464:11: warning: parameter 'req' not found in the function declaration [-Wdocumentation]
     @param req a request object
            ^~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:465:11: warning: parameter 'databuf' not found in the function declaration [-Wdocumentation]
     @param databuf the data chunk to send as part of the reply.
            ^~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:467:11: warning: parameter 'call' not found in the function declaration [-Wdocumentation]
     @param call back's argument.
            ^~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:939:4: warning: declaration is marked with '@deprecated' command but does not have a deprecation attribute [-Wdocumentation-deprecated-sync]
    @deprecated  This function is deprecated; you probably want to use
    ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:946:1: note: add a deprecation attribute to the declaration to silence this warning
  char *evhttp_decode_uri(const char *uri);
  ^
  __AVAILABILITY_INTERNAL_DEPRECATED
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:979:5: warning: declaration is marked with '@deprecated' command but does not have a deprecation attribute [-Wdocumentation-deprecated-sync]
     @deprecated This function is deprecated as of Libevent 2.0.9.  Use
     ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:987:1: note: add a deprecation attribute to the declaration to silence this warning
  int evhttp_parse_query(const char *uri, struct evkeyvalq *headers);
  ^
  __AVAILABILITY_INTERNAL_DEPRECATED
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:1002:11: warning: parameter 'query_parse' not found in the function declaration [-Wdocumentation]
     @param query_parse the query portion of the URI
            ^~~~~~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:1002:11: note: did you mean 'uri'?
     @param query_parse the query portion of the URI
            ^~~~~~~~~~~
            uri
  69 warnings generated.
  ```

  Note that a lot of these have already been fixed upstream.

ACKs for top commit:
  laanwj:
    Concept and code review ACK a4e970adb6
  practicalswift:
    cr ACK a4e970adb6: automatic compiler feedback comes sooner and is more reliable than manual reviewer feedback
  jonatack:
    Light ACK a4e970adb6 skimmed the changes, clang 11 build is clean with the change, verified -Wdocumentation build warnings with this change when a doc fix was reverted

Tree-SHA512: 57a1e30cffcc8bcceee72d85f58ebe29eae525861c70acb237541bd480c51ede89875c033042c0af376fdbb49fb7f588ef9282a47c6e78f9d4501c41f1b21eb6
2021-04-07 16:49:57 +08:00
MarcoFalke
6154291cf9 Merge #21617: fuzz: Fix uninitialized read in i2p test
33333755f2 fuzz: Fix uninitialized read in test (MarcoFalke)

Pull request description:

  Can be tested with:

  ```
  ./test/fuzz/test_runner.py -l DEBUG --valgrind ../btc_qa_assets/fuzz_seed_corpus/ i2p
  ```

  ```
  ==22582== Conditional jump or move depends on uninitialised value(s)
  ==22582==    at 0x6BB2D8: __sanitizer_cov_trace_const_cmp1 (in /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz)
  ==22582==    by 0xB305DB: ConnectSocketDirectly(CService const&, Sock const&, int, bool) (netbase.cpp:570)
  ==22582==    by 0x8AAA5D: i2p::sam::Session::Hello() const (i2p.cpp:284)
  ==22582==    by 0x8A6FA0: i2p::sam::Session::CreateIfNotCreatedAlready() (i2p.cpp:352)
  ==22582==    by 0x8A6742: i2p::sam::Session::Listen(i2p::Connection&) (i2p.cpp:134)
  ==22582==    by 0x7A6C42: i2p_fuzz_target(Span<unsigned char const>) (i2p.cpp:37)

ACKs for top commit:
  sipa:
    utACK 33333755f2
  vasild:
    ACK 33333755f2

Tree-SHA512: 36073582b26b541324b3e55f3fd4a44abf89cb3081f36d361525daf8c27602fbc25f736510ec30df7cb4ca0c4e395e8d8a60f531bf6af358b5a3e65dbabf72c0
2021-04-07 10:39:27 +02:00
fanquake
c0160ea52e Merge #21540: wallet: refactor: dedup sqlite statement preparations/deletions
ea19cc844e wallet: refactor: dedup sqlite statement deletions (Sebastian Falbesoner)
9a3670930e wallet: refactor: dedup sqlite statement preparations (Sebastian Falbesoner)

Pull request description:

  This refactoring PR deduplicates repeated SQLite statement preparation calls (`sqlite3_prepare_v2(...)`) / deletions (`sqlite3_finalize(...)`) and its surrounding logic by putting each prepared statement and its corresponding text representation into a ~std::map~ ~`std::array`~ `std::vector`. This should be more readable and less error-prone, e.g. in case an additional statement needs to be added in the  future or the error handling has to be adapted.

ACKs for top commit:
  achow101:
    ACK ea19cc844e
  meshcollider:
    utACK ea19cc844e

Tree-SHA512: ced89869b2147e088e7a4cda2acbbdd4a806f66dbc2d6999953d0d702c0655aa53c0eb699cc7e5e3732f2d24206d577a9d9e1b5de7f439100dead2696ade1092
2021-04-07 14:17:25 +08:00
Jon Atack
60290d3f5e test: increase listbanned unit test coverage
Add test coverage for the new ban_duration and time_remaining fields.
While here, some code improvements.
2021-04-07 01:57:26 -04:00
MarcoFalke
41a8d2b96f Merge #21582: Fix assumeutxo crash due to missing base_blockhash
fa9b74f5ea Fix assumeutxo crash due to missing base_blockhash (MarcoFalke)
fa8fffebe8 refactor: Prefer clean assert over UB in coinstats (MarcoFalke)

Pull request description:

  This fixes an UB (which results in a crash with sanitizers enabled). Can be reproduced by cherry-picking the test without the other code changes. The fix:

  * Adds an `Assert` to transform the UB into a clean crash, even when sanitizers are disabled
  * Adds an early-fail condition to avoid the crash

ACKs for top commit:
  jamesob:
    ACK fa9b74f5ea ([`jamesob/ackr/21582.1.MarcoFalke.fix_assumeutxo_crash_due`](https://github.com/jamesob/bitcoin/tree/ackr/21582.1.MarcoFalke.fix_assumeutxo_crash_due))
  ryanofsky:
    Code review ACK fa9b74f5ea with no code changes since last review, just splitting up combocommit a little.

Tree-SHA512: dd36808a09f49c647543a9eaa6fdb785b3f1109af48ba4cc983153b22a144da9ca61af22034dcfaa0e192a65b1ee7de744f187555079aff55bec0efa0ce87cd4
2021-04-07 07:33:27 +02:00
fanquake
245a5cd560 Merge #21166: Introduce DeferredSignatureChecker and have SignatureExtractorClass subclass it
a97a9298ce Test that signrawtx works when a signed CSV and CLTV inputs are present (Andrew Chow)
6965456c10 Introduce DeferringSignatureChecker and inherit with SignatureExtractor (Andrew Chow)

Pull request description:

  Previously SignatureExtractorChecker took a MutableTransactionSignatureChecker and passed through function calls to that. However not all functions were implemented so not everything passed through as it should have. To solve this, SignatureExctractorChecker now implements all of those functions via a new class - DeferredSignatureChecker. DeferredSignatureChecker is introduced to allow for future signature checkers which use another SignatureChecker but need to be able to do somethings outside of just the signature checking.

  Fixes #21151

ACKs for top commit:
  sipa:
    utACK a97a9298ce
  meshcollider:
    Code review ACK a97a9298ce
  instagibbs:
    utACK a97a9298ce

Tree-SHA512: bca784c75c2fc3fcb74e81f4e3ff516699e8debaa2db81e12843abdfe9cf265dac11db8619751cb9b3e9bbe779805d029fabe5f3cbca5e86bfd72de3664b0b94
2021-04-07 12:47:41 +08:00
Jarol Rodriguez
3e978d1a5d rpc: add time_remaining field to listbanned 2021-04-06 23:45:31 -04:00
Jarol Rodriguez
5456b34531 rpc: add ban_duration field to listbanned 2021-04-06 23:01:10 -04:00
Jarol Rodriguez
c95c61657a doc: improve listbanned help
Add descriptions for the address, ban_created, and banned_until fields.
2021-04-06 18:22:20 -04:00
Jarol Rodriguez
dd3c8eaa33 rpc: swap position of banned_until and ban_created fields
A ban expires after its creation. Therefore, for the listbanned RPC,
position banned_until after ban_created in help and output.
2021-04-06 18:22:20 -04:00
MarcoFalke
33333755f2 fuzz: Fix uninitialized read in test 2021-04-06 12:43:33 +02:00
Hennadii Stepanov
55d85834cc script: Add trusted key for hebasto 2021-04-06 12:27:32 +03:00
W. J. van der Laan
9be7fe4849 Merge #21560: net: Add Tor v3 hardcoded seeds
b2ee8b207d net: Deserialize hardcoded seeds from BIP155 blob (W. J. van der Laan)
9b29d5df7f contrib: Add explicit port numbers for testnet seeds (W. J. van der Laan)
2a257de113 contrib: Add a few TorV3 seed nodes (W. J. van der Laan)
06030f7a42 contrib: generate-seeds.py generates output in BIP155 format (W. J. van der Laan)

Pull request description:

  Closes #20239 and mitigates my node's problem in #21351.

  - Add a few hardcoded seeds for TorV3
    - As the [bitcoin-seeder](https://github.com/sipa/bitcoin-seeder) doesn't collect TorV3 addresses yet, I have extracted these from my own node using [a script](https://gist.github.com/laanwj/b3d7b01ef61ce07c2eff0a72a6b90183) and added them manually. This is intended to be a temporary stop gap until 22.0's seeds update.

  - Change hardcoded seeds to variable length BIP155 binary format.
    - It is stored as a single serialized blob in a byte array, instead of pseudo-IPv6 address slots. This is more flexible and, assuming most of the list is IPv4, more compact.
    - Only the (networkID, addr, port) subset (CService). Services and time are construed on the fly as before.

  - Change input format for `nodes_*.txt`.
    - Drop legacy `0xAABBCCDD` format for IPv4. It is never generated by `makeseeds.py`.
    - Stop interpreting lack of port as default port, interpret it as 'no port', to accomodate I2P and other port-less protocols (not handled in this PR). An explicit port is always generated by `makeseeds.py` so in practice this makes no difference right now.

  A follow-up to this PR could do the same for I2P.

ACKs for top commit:
  jonatack:
    ACK b2ee8b207d

Tree-SHA512: 11a6b54f9fb0192560f2bd7b218f798f86c1abe01d1bf37f734cb88b91848124beb2de801ca4e6f856e9946aea5dc3ee16b0dbb9863799e42eec1b239d40d59d
2021-04-06 10:47:51 +02:00
MarcoFalke
1a7dec77f6 Merge #21571: test: make sure non-IP peers get discouraged and disconnected (vasild)
81747b2171 test: make sure non-IP peers get discouraged and disconnected (Vasil Dimov)
637bb6da36 test: also check disconnect in denialofservice_tests/peer_discouragement (Vasil Dimov)
4d6e246fa4 test: use pointers in denialofservice_tests/peer_discouragement (Vasil Dimov)

Pull request description:

  Split up from #20966, so that it can be backported easier. Merging this ahead of #20966 will also reduce the number of conflicts for that pull.

ACKs for top commit:
  jonatack:
    ACK 81747b2171

Tree-SHA512: 8f0e30b95baba7f056920d7fc3b37bd49ee13e69392fe80e2d333c6bb09fd25f4603249301b8795cca26a2f2d15b9f8904798a55cd9c04fd28afb316e95c551c
2021-04-06 10:26:33 +02:00
MarcoFalke
fadcd3f78e doc: Remove irrelevant link to GitHub
The doc nicely explains why the directory exists and it is
irrelevant when it was introduced. Even if it was relevant,
it could be trivially found out via `git log ./src/node/ | tail`
without visiting GitHub
2021-04-06 09:34:21 +02:00
MarcoFalke
02fb5bdc21 Merge #21610: build: remove -Wdeprecated-register from NOWARN flags
1a011b3a82 build: remove -Wdeprecated-register from NOWARN flags (fanquake)

Pull request description:

  The `register` keyword was deprecated in C++11, and [removed in C++17](https://en.cppreference.com/w/cpp/keyword/register). Now that we require C++17, we shouldn't have to suppress warnings for a non-existent feature.

ACKs for top commit:
  sipa:
    utACK 1a011b3a82
  hebasto:
    ACK 1a011b3a82

Tree-SHA512: 7546b3870fe819507deea57d5c3179bc5debd4513df41b3f74d191995116b1507ff2a77cf2081ac32871194d4e5c2a2913c668c56244dff39853a5da18194f9f
2021-04-06 08:52:54 +02:00
fanquake
a4e970adb6 build: enable -Wdocumentation if suppressing external warnings
Co-authored-by: Ben Woosley <ben.woosley@gmail.com>
2021-04-06 14:50:26 +08:00
fanquake
3b0078f958 doc: fixup -Wdocumentation issues 2021-04-06 14:50:17 +08:00
MarcoFalke
7b4934e550 Merge #21557: test: small cleanup in RPCNestedTests tests
6526a1644c test: small cleanup in RPCNestedTests tests (fanquake)

Pull request description:

  Remove QtDir & QtGlobal (dea086f498)
  Add missing includes.
  Remove obsolete comment about Qt 5.3 (fd46c4c001)

Top commit has no ACKs.

Tree-SHA512: 097e603fc31a19be1817459ad4c5a9692708f8a39a0ae87e4a60eabc22bf4f6141b577ba68746044fd594f92e36848b7cd56d60dccd262f83f8ec7310ab7d1bc
2021-04-06 08:45:08 +02:00
fanquake
c6edcf1c71 build: suppress libevent warnings if supressing external warnings 2021-04-06 14:44:50 +08:00
MarcoFalke
b2ec7d398f Merge #21609: ci: increase CPU count of sanitizer job to increase memory limit
de3ae78eff ci: increase CPU count of sanitizer job to increase memory limit (fanquake)

Pull request description:

  According to the [docs](https://cirrus-ci.org/guide/linux/#linux-containers):
  > For each CPU you can't get more than 4G of memory.

  thus if we want this job to have 24GB of memory, we need to increase the CPU count to 6.

  It's currently [failing with](https://github.com/bitcoin/bitcoin/runs/2273962280):
  >  Requested memory is too high! You can request at most 4G per CPU

Top commit has no ACKs.

Tree-SHA512: 0a4da5649d061425190a373859274c78ca5587cd2d6e27905ec548f124ed114a0133215cb2eff22ffc182f50c3a53df58e7c9832b44db6e37d7ea59ec96a4775
2021-04-06 07:57:30 +02:00
MarcoFalke
9ac8f6d7dd Merge #21598: refactor: Remove negative lock annotations from globals
fa5eabe721 refactor: Remove negative lock annotations from globals (MarcoFalke)

Pull request description:

  They only make sense for mutexes that are private members. Until cs_main is a private member the negative annotations should be replaced by excluded annotations, which are optional.

ACKs for top commit:
  sipa:
    utACK fa5eabe721
  ajtowns:
    ACK fa5eabe721
  hebasto:
    ACK fa5eabe721
  vasild:
    ACK fa5eabe721

Tree-SHA512: 06f8a200304f81533010efcc42d9f59b8c4d0ae355920c0a28efb6fa161a3e3e68f2dfffb0c009afd9c2501e6a293c6e5a419a64d718f1f4e79668ab2ab1fcdc
2021-04-06 07:54:12 +02:00
fanquake
328aaac80d Merge #21597: test: Document race:validation_chainstatemanager_tests suppression
fab19871ba test: Document race:validation_chainstatemanager_tests suppression (MarcoFalke)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK fab19871ba
  practicalswift:
    ACK fab19871ba

Tree-SHA512: 3f1838b4cf11eba768ce06826cd4b57c9065669b61a5530af44216fc96535ebf37124b47a8de8f72aedf32345157a72d2208cd63214481a9cb56c063f05db5dd
2021-04-06 10:33:30 +08:00
fanquake
e2777b5328 Merge #21611: Fix a typo in guix-build output
5c09bcadc4 Fix a typo in guix-build output (Pieter Wuille)

Pull request description:

  This was overlooked in #21375.

ACKs for top commit:
  fanquake:
    ACK 5c09bcadc4

Tree-SHA512: 81d8ad4061abb17d5f16ae72ab0c88df76d5a2f100cb9f471ca700d2e87583103036367ea1958a1066f9613f63908d1b2b35734a7eb77ec9850d4c8b079732e1
2021-04-06 09:12:36 +08:00
Pieter Wuille
5c09bcadc4 Fix a typo in guix-build output 2021-04-05 17:58:49 -07:00
fanquake
de3ae78eff ci: increase CPU count of sanitizer job to increase memory limit
According to the docs,
https://cirrus-ci.org/guide/linux/#linux-containers, "For each CPU you
can't get more than 4G of memory.", thus if we want this job to have
24GB of memory, we need to increase the CPU count to 6.
2021-04-06 08:40:24 +08:00
fanquake
1a011b3a82 build: remove -Wdeprecated-register from NOWARN flags
The register keyword was deprecated in C++11, and removed in C++17. Now
that we require C++17, we shouldn't have to supress warnings for a
non-existant feature.
2021-04-06 08:12:22 +08:00
fanquake
511e58223b Merge #21565: build: make bitcoin_qt.m4 more generic
fecb3723b6 build: additional PKG_CHECK_MODULES calls in bitcoin_qt.m4 (fanquake)
a53eff3ce5 build: misc doc changes in bitcoin_qt.m4 (fanquake)
340fa6c0ff build: set QT_*_CFLAGS & QT_*_LIBS in PKG_CHECK_MODULES() calls (fanquake)
4b8ad97c5b build: use QT_*_LIBS rather than passing lib names (fanquake)
13313b2904 scripted-diff: replace Qt5 with ${qt_lib_prefix} in _BITCOIN_QT_CHECK_STATIC_LIBS (fanquake)
477df3623c build: reorder libs in _BITCOIN_QT_CHECK_STATIC_LIBS (move-only) (fanquake)

Pull request description:

  These are some small changes to simplify this code before we may end up supporting Qt6. Replaces usages of `Qt5` with `qt_lib_prefix`, minor docs cleanups and takes care of [a suggestion](https://github.com/bitcoin/bitcoin/pull/21376#discussion_r590693514) left over from #21376.

  Guix builds:
  ```bash
  8749dca4a86c39607ff86b0107b76daea60d415a15b5022824f4a4469c6edc37  output/bitcoin-fecb3723b63d-aarch64-linux-gnu-debug.tar.gz
  7b162ad6d953ceed9402f5fef03a8199a76c6dd300a71b03c1814c33ce0126f6  output/bitcoin-fecb3723b63d-aarch64-linux-gnu.tar.gz
  869c42616320fc5bb25e942d7e20c0a8a4b8510939f8f8f6021a784e4afda83e  output/bitcoin-fecb3723b63d-arm-linux-gnueabihf-debug.tar.gz
  278b1481045e93b3ac7c31ae14ea3b6a3030391c2abc90e04ab80072f4c33dd3  output/bitcoin-fecb3723b63d-arm-linux-gnueabihf.tar.gz
  e5501939838f930bf5d5a6da4f6d3665c008d736aa916ef0e7e0e06db91e1a36  output/bitcoin-fecb3723b63d-osx-unsigned.dmg
  834a94ea3fbc5e090618842b339b301bebd622d1f392918f0f500ec6637733ac  output/bitcoin-fecb3723b63d-osx-unsigned.tar.gz
  6cf70af01a48bea9270e6883ed15ce458c46c6840f826414df7a244fffa3338d  output/bitcoin-fecb3723b63d-osx64.tar.gz
  d80e9e2789fdee931f80c049c70912e9ff146d5fe2ae7bcd26244d7fe5dd4910  output/bitcoin-fecb3723b63d-powerpc64-linux-gnu-debug.tar.gz
  d924e161f1c32b3980c72b4949c99269e3d5fad8656749edae9fbbaafb1b7ff9  output/bitcoin-fecb3723b63d-powerpc64-linux-gnu.tar.gz
  d2f339fc78608ac6cdc505b21abc5f397e52283914113f45dc0adad0c023df54  output/bitcoin-fecb3723b63d-powerpc64le-linux-gnu-debug.tar.gz
  d537c014adbdd712b7b20f3171bc4342622ba56f4fa655bba85afbf35aee2840  output/bitcoin-fecb3723b63d-powerpc64le-linux-gnu.tar.gz
  0f42646fa5be154e8717df0e39ba843a61928225b00e0d48b1b2179966fea315  output/bitcoin-fecb3723b63d-riscv64-linux-gnu-debug.tar.gz
  f7b9cc85f74c006ac484e20915c84a12069447102d685124c5246b6eff8545c8  output/bitcoin-fecb3723b63d-riscv64-linux-gnu.tar.gz
  b60b51544a26dfa2e6a05a6daa99c63aeb62d7a1893191824867e1da9bbb73c4  output/bitcoin-fecb3723b63d-win-unsigned.tar.gz
  abc4cdb44146edf16fc47aa11d72648fa53c00d8f70937cab74cdaaaa9947c58  output/bitcoin-fecb3723b63d-win64-debug.zip
  3c6d437b09acc479060d922c64c1f6688df6385a6a3950d31c7a6e0977d757eb  output/bitcoin-fecb3723b63d-win64-setup-unsigned.exe
  d4f015fcfca15f23a797786fe67ca12ac73d3f594dd9e1b885e33d765f21beff  output/bitcoin-fecb3723b63d-win64.zip
  5fcb478027cb790ab7a3480fab91f764b10da4fd25d82b08f5ecfe42b4b7d316  output/bitcoin-fecb3723b63d-x86_64-linux-gnu-debug.tar.gz
  eef74c6b301bd799c6522b60c26d700a496ecf2bd357f09c83cbd88132086783  output/bitcoin-fecb3723b63d-x86_64-linux-gnu.tar.gz
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  output/src/bitcoin-fecb3723b63d.tar.gz
  ```

  Gitian builds:
  ```bash
  # macOS:
  b3926b7956b31b87ec96885369401141cc5a3f37e91a0ad9a03c6f7c1bd2bee7  bitcoin-fecb3723b63d-osx-unsigned.dmg
  61792260b3afd0b921d84ddb0bb3639c2b4e063c7b7fb715d515b3601c7ad38b  bitcoin-fecb3723b63d-osx-unsigned.tar.gz
  953081b66a1feababe041cae0b31664856260f7364f87c5c50f7f7c1fd6b710a  bitcoin-fecb3723b63d-osx64.tar.gz
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  src/bitcoin-fecb3723b63d.tar.gz
  aecaa381205e9ed6585cdffa22c7b46d3ece989c1774855d4a3b4cae5d75567a  bitcoin-core-osx-22-res.yml

  # Windows:
  21c23951c96bcbf79fa75a6b915c40c24445964cced6242f4e9181464f66ba40  bitcoin-fecb3723b63d-win-unsigned.tar.gz
  327cc290f5f709892e54cdc7bbf1745892eed497cd636c8f80d2310419e585c2  bitcoin-fecb3723b63d-win64-debug.zip
  e86a36c26897be064339cda866cb9563d08c867eceae59a677329390d729cbfe  bitcoin-fecb3723b63d-win64-setup-unsigned.exe
  844754a178b3b7ba62fc39bcc3f5e76b3903c9cd86adc26ec224ecf84d697239  bitcoin-fecb3723b63d-win64.zip
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  src/bitcoin-fecb3723b63d.tar.gz

  # Linux:
  b6224d86755e6c609fd1cfff79f6514bdd0cf1af8689eca692352c515e6cb7dd  bitcoin-fecb3723b63d-aarch64-linux-gnu-debug.tar.gz
  371a2533f3510c7a7e7bd2a9b504accffdcdd65bcc9d6c393570e0378f470f93  bitcoin-fecb3723b63d-aarch64-linux-gnu.tar.gz
  6c171cf1168f7bc6a1f78d5662c6cd1de7dd6157559fe4f7f95d34df7e398354  bitcoin-fecb3723b63d-arm-linux-gnueabihf-debug.tar.gz
  ac455dcb33c92191eb0124c78fa356cea78c88e3976bfe8b4b9f9052458b7b24  bitcoin-fecb3723b63d-arm-linux-gnueabihf.tar.gz
  166ec0874d09b1e3d1b682ed120873b2715dee5ed13dd721443ee29784074107  bitcoin-fecb3723b63d-powerpc64-linux-gnu-debug.tar.gz
  2b32d1a761625c28ed59fdbe8463dcb89b4f583a3119c57fd6815c9882241c78  bitcoin-fecb3723b63d-powerpc64-linux-gnu.tar.gz
  b9b37f0097f39fb16c2612166ffe21a09b436feea42394935831e2cb102c532a  bitcoin-fecb3723b63d-powerpc64le-linux-gnu-debug.tar.gz
  e98fb672a5205a8d18f6ca4a39dec0c26353031c455c4e1b3917b917ce687d8d  bitcoin-fecb3723b63d-powerpc64le-linux-gnu.tar.gz
  f92b07bd155fc469a18aa34571534f2c0bf12706f381ca973bb7a51692e25754  bitcoin-fecb3723b63d-riscv64-linux-gnu-debug.tar.gz
  94321b7db0446457c372bea24d71d84d0f0181f7607bf9f96a8b773e69ee1370  bitcoin-fecb3723b63d-riscv64-linux-gnu.tar.gz
  350bc9be945e3410bf73d9631fe4bb750f9a2ee3d538d2ce3efd42b15eba926a  bitcoin-fecb3723b63d-x86_64-linux-gnu-debug.tar.gz
  74aa3745942a9d76d3db23fa0251a27bbe1ea95683a901df37333caf9c6d13ab  bitcoin-fecb3723b63d-x86_64-linux-gnu.tar.gz
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  src/bitcoin-fecb3723b63d.tar.gz
  959c25666565646279659fccb93278875bb926b67faf685185f461bf55b4058e  bitcoin-core-linux-22-res.yml
  ```

ACKs for top commit:
  hebasto:
    ACK fecb3723b6

Tree-SHA512: 00edf084852bfab404fb32d0a726088a50283a58a240d4e03cbd746cbffbe085d309434b2c22a4ed18c28be4df6958eb51d92d4ccd7548dd0ee49f99d458ff03
2021-04-06 08:05:41 +08:00
Carl Dong
44f6d4f56b guix: Record precious directories and add guix-clean
Many users have reported problems that stem from having an unclean
working tree. To that end, I've written a guix-clean script which should
help reset the working tree while respecting user-specified precious
directories.

Precious directories, such as:

- SOURCES_PATH
- BASE_CACHE
- SDK_PATH
- OUTDIR

Should be preserved when cleaning the working tree, and are thus
recorded in ./contrib/guix/var/precious_dirs.

The ./contrib/guix/guix-clean script is able to parse that file and make
sure to avoid them when cleaning out the working tree.
2021-04-05 19:13:54 -04:00
Carl Dong
84912d4b24 build: Remove spaces from variable-printing rules
This simplifies parsing when using these rules from scripts.
2021-04-05 19:13:54 -04:00
W. J. van der Laan
0102f80b51 Merge #21375: guix: Misc feedback-based fixes + hier restructuring
7476b46f18 guix: Build dmg as a static binary (Carl Dong)
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb4 guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3 guix: Supply --link-profile (Carl Dong)
d55a1056ee guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b2 guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7b guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5 guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab38 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e9785 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f18
  laanwj:
    ACK 7476b46f18

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2021-04-06 01:00:27 +02:00
MarcoFalke
fa121b628d blockstorage: [refactor] Use chainman reference where possible
Also, add missing { } for style.

Can be reviewed with `--word-diff-regex=.`
2021-04-05 20:26:32 +02:00
MarcoFalke
fa0c7d9ad2 move-only: Move *Disk functions to blockstorage
Can be reviewed with the git options
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-05 20:26:14 +02:00
Carl Dong
d7824acdb9 rest: Use existing NodeContext 2021-04-05 11:14:33 -04:00
Carl Dong
3f08934799 rest: Pass in NodeContext to rest_block 2021-04-05 11:14:33 -04:00
Carl Dong
7be0671b95 rpc/rawtx: Use existing NodeContext
Also pass in appropriate object to:
- TxToJSON
2021-04-05 11:14:33 -04:00
Carl Dong
60dc05afc6 rpc/mining: Use existing NodeContext
Also pass in appropriate object to:
- GetNetworkHashPS
- [gG]enerateBlock{,s}

Also:
- Misc style/constness changes
2021-04-05 11:14:28 -04:00
Carl Dong
d485e815e2 rpc/blockchain: Use existing NodeContext
Also pass in appropriate object to:
- BIP9SoftForkDescPushBack
- BuriedForkDescPushBack
2021-04-05 11:13:54 -04:00
Carl Dong
d0abf0bf42 rpc/*,rest: Add review-only assertion to EnsureChainman 2021-04-05 11:13:54 -04:00
Carl Dong
cced0f46c9 miner: Pass in previous CBlockIndex to RegenerateCommitments 2021-04-05 11:13:51 -04:00
Carl Dong
7476b46f18 guix: Build dmg as a static binary
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.

This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
2021-04-05 11:00:39 -04:00
Carl Dong
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching
CMake's RPATH patching apparently causes non-reproducibility in the
executables which are produced, manifesting in a difference in padding
in the .dynstr section (we found this while investigating
non-reproducibility in the "dmg" tool). This RPATH patching can be
safely skipped for executables which don't depend on internal shared
libraries.

Documentation sources:
1. https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling
2. https://reproducible-builds.org/docs/deterministic-build-systems/#cmake-notes

Prior debugging art:
1. https://stackoverflow.com/questions/63438206/cmake-g-reproducible-build-issue-with-changing-build-path
2. https://github.com/NXPmicro/mfgtools/pull/229/files
2021-04-05 11:00:39 -04:00
Carl Dong
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball 2021-04-05 11:00:39 -04:00
Carl Dong
ca85679eb4 guix: Use clang-toolchain instead of clang 2021-04-05 11:00:39 -04:00
Carl Dong
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users 2021-04-05 11:00:39 -04:00
Carl Dong
1742f8e12d guix: Add early health check for guix-daemon 2021-04-05 11:00:39 -04:00
Carl Dong
c1ae726a13 guix: More thoroughly control native toolchain 2021-04-05 11:00:39 -04:00
Carl Dong
39741128d3 guix: Supply --link-profile 2021-04-05 11:00:39 -04:00
Carl Dong
d55a1056ee guix: Add troubleshooting documentation entries 2021-04-05 11:00:39 -04:00
Carl Dong
7f401c953f guix: Adapt guix-build to prelude, restructure hier 2021-04-05 11:00:39 -04:00
Carl Dong
4eccf063b2 guix: Remove guix-build.sh filename extension 2021-04-05 11:00:39 -04:00
Carl Dong
7753357a7b guix: Add source-able bash prelude and utils 2021-04-05 11:00:21 -04:00
MarcoFalke
1ea5c7ec78 Merge #21604: test: Document why no symbol names can be used for suppressions
fa04eb7d78 test: Document why no symbol names can be used for suppressions (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 02e5d9cb3ca299b5e7f1f0f780af514692147f5f0846e21e5375a99e90d3aed3a3489c674ac0eb90ae34d60f576410ba1d926d5ca70cdce4f9c7a111cd516adf
2021-04-05 16:25:58 +02:00
MarcoFalke
fa04eb7d78 test: Document why no symbol names can be used for suppressions 2021-04-05 16:23:13 +02:00
Hennadii Stepanov
b8e5d0d3fe qt: Handle exceptions in SendCoinsDialog::sendButtonClicked slot
Also, uic automatic connection replaced with an explicit one.
2021-04-05 16:47:31 +03:00
Hennadii Stepanov
1ac2bc7ac0 qt: Handle exceptions in TransactionView::bumpFee slot
Also the parameter list of the TransactionView::bumpFee slot is made
compatible with one of the QAction::triggered signal.
2021-04-05 16:47:31 +03:00
Hennadii Stepanov
bc00e13bc8 qt: Handle exceptions in WalletModel::pollBalanceChanged slot
Actually, the private QTimer::timeout signal has one
QTimer::QPrivateSignal parameter.
2021-04-05 16:47:08 +03:00
W. J. van der Laan
5c9b06db81 Merge #21302: wallet: createwallet examples for descriptor wallets
5039e0e55a test: HelpExampleCliNamed and HelpExampleRpcNamed (Ivan Metlushko)
591735ef0b rpc: Add HelpExampleCliNamed and use it for `createwallet` doc (Wladimir J. van der Laan)
5d5a90e819 rpc: Add HelpExampleRpcNamed (Ivan Metlushko)

Pull request description:

  Rationale: make descriptor wallets more visible and just a bit easier to setup

  `bitcoin-cli help createwallet`

  **Before**:
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

  **After**
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  > bitcoin-cli createwallet "descriptors" false false "" true true true
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["descriptors", false, false, "", true, true, true]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 5039e0e55a

Tree-SHA512: d37210e6ce639addee881377092d8f6fb2a537a60a259c561899e24cf68a0254d7ff45a213573c938f626677e46770cd21113aae5974f26c66b9a2e137699c14
2021-04-05 15:31:41 +02:00
W. J. van der Laan
b2ee8b207d net: Deserialize hardcoded seeds from BIP155 blob
Switch from IPv6 slot-based format to more compact and flexible BIP155
format.
2021-04-05 14:00:48 +02:00
W. J. van der Laan
9b29d5df7f contrib: Add explicit port numbers for testnet seeds
This is necessary now due to parsing change.
2021-04-05 14:00:48 +02:00
W. J. van der Laan
2a257de113 contrib: Add a few TorV3 seed nodes 2021-04-05 14:00:48 +02:00
W. J. van der Laan
06030f7a42 contrib: generate-seeds.py generates output in BIP155 format 2021-04-05 13:59:40 +02:00
MarcoFalke
fab19871ba test: Document race:validation_chainstatemanager_tests suppression 2021-04-05 13:31:04 +02:00
MarcoFalke
fa5eabe721 refactor: Remove negative lock annotations from globals 2021-04-05 08:42:15 +02:00
MarcoFalke
824eea5643 Merge #21599: test: Replace file level integer overflow suppression with function level suppression
585854ac66 test: Replace blanket UBSan signed integer overflow suppression for txmempool.cpp with specific suppression (practicalswift)

Pull request description:

  Replace file level (`txmempool.cpp`) signed integer overflow suppression with function level suppression (`CTxMemPool::PrioritiseTransaction`). The suppression was added yesterday in #21586.

  Rationale: To avoid risk hiding other signed integer overflows in `txmempool.cpp`.

  Obviously it would be better if this signed integer overflow fixed instead of suppressed - see details #20626. Any taker? :)

  To hit the issue via fuzzing:

  ```
  $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" FUZZ=validation_load_mempool src/test/fuzz/fuzz
  INFO: Seed: 1184244493
  INFO: Loaded 1 modules   (634418 inline 8-bit counters): 634418 [0x55a09fdfbf98, 0x55a09fe96dca),
  INFO: Loaded 1 PC tables (634418 PCs): 634418 [0x55a09fe96dd0,0x55a0a08450f0),
  INFO:     1264 files found in mempool/
  INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1040698 bytes
  INFO: seed corpus: files: 1264 min: 1b max: 1040698b total: 15997133b rss: 197Mb
  txmempool.cpp:847:15: runtime error: signed integer overflow: -7211388903327006720 + -7211353718954917888 cannot be represented in type 'long'
      #0 0x55a09c3ce2d8 in CTxMemPool::PrioritiseTransaction(uint256 const&, long const&) /home/thomas/bitcoin/src/txmempool.cpp:847:15
  ```

ACKs for top commit:
  JeremyRubin:
    utACK 585854a
  hebasto:
    ACK 585854ac66, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 5a343f028c1e1a1aba3b51a0eced605849184891ffafecb3cd2b424c6cfea01afd7c2672274936b0bac646075ec066408a570bf6b34bc9b87399a53ce20d8a23
2021-04-05 07:28:45 +02:00
practicalswift
585854ac66 test: Replace blanket UBSan signed integer overflow suppression for txmempool.cpp with specific suppression 2021-04-04 21:10:15 +00:00
MarcoFalke
fa91b2b2b3 move-only: Move AbortNode to shutdown
Can be reviewed with the git option
--color-moved=dimmed-zebra
2021-04-04 18:08:36 +02:00
MarcoFalke
fa413f07a1 move-only: Move ThreadImport to blockstorage
Can be reviewed with the git options
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-04 18:07:24 +02:00
Hennadii Stepanov
b95f7f8ac0 build, qt, refactor: Drop sed commands for win32-g++/qmake.conf 2021-04-04 14:40:06 +03:00
MarcoFalke
590e49ccf2 Merge #21586: test: Add missing suppression for signed-integer-overflow:txmempool.cpp
fad8a97b21 test: Add missing suppression for signed-integer-overflow:txmempool.cpp (MarcoFalke)

Pull request description:

  Otherwise the fuzzer will crash:

  ```
  txmempool.cpp:847:15: runtime error: signed integer overflow: 8138645194045128704 + 4611686018427387904 cannot be represented in type 'long'
      #0 0x558ff1838d4c in CTxMemPool::PrioritiseTransaction(uint256 const&, long const&) /root/fuzz_dir/scratch/fuzz_gen/code/src/txmempool.cpp:847:15
      #1 0x558ff196e723 in LoadMempool(CTxMemPool&, CChainState&, std::function<_IO_FILE* (boost::filesystem::path const&, char const*)>) /root/fuzz_dir/scratch/fuzz_gen/code/src/validation.cpp:5053:22
      #2 0x558ff13f37ab in validation_load_mempool_fuzz_target(Span<unsigned char const>) /root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/validation_load_mempool.cpp:32:11
      #3 0x558ff1083378 in std::_Function_handler<void (Span<unsigned char const>), void (*)(Span<unsigned char const>)>::_M_invoke(std::_Any_data const&, Span<unsigned char const>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:300:2
      #4 0x558ff22a749d in std::function<void (Span<unsigned char const>)>::operator()(Span<unsigned char const>) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #5 0x558ff22a70e8 in LLVMFuzzerTestOneInput /root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz.cpp:63:5
      #6 0x558ff0f83543 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) fuzzer.o
      #7 0x558ff0f6d442 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) fuzzer.o
      #8 0x558ff0f7323a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) fuzzer.o
      #9 0x558ff0f9ef82 in main (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x196df82)
      #10 0x7f1237f310b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
      #11 0x558ff0f4816d in _start (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x191716d)

Top commit has no ACKs.

Tree-SHA512: 94c13771054b4acfb83e3dcfa09beb3f9d0ca0e025d3993cdf2e46df6456f227565b31fd4377b8dd86c567aeee800f293ac57a470c6f5f81e9177d460e7bd705
2021-04-04 07:44:00 +02:00
MarcoFalke
fa9b74f5ea Fix assumeutxo crash due to missing base_blockhash 2021-04-04 07:38:02 +02:00
MarcoFalke
fa8fffebe8 refactor: Prefer clean assert over UB in coinstats 2021-04-04 07:37:43 +02:00
MarcoFalke
fa73ce6e65 Fix assumeutxo crash due to truncated file 2021-04-03 17:52:58 +02:00
MarcoFalke
fad8a97b21 test: Add missing suppression for signed-integer-overflow:txmempool.cpp 2021-04-03 17:39:41 +02:00
fanquake
fecb3723b6 build: additional PKG_CHECK_MODULES calls in bitcoin_qt.m4
Add checks for the edid, input and service support modules.
2021-04-03 15:27:04 +08:00
MarcoFalke
ad4bf8a945 Merge #20459: rpc: Fail to return undocumented return values
fa8192f42e rpc: Fail to return undocumented return values (MarcoFalke)

Pull request description:

  Currently a few return values are undocumented. This is causing confusion at the least. See for example #18476

  Fix this by treating it as an internal bug to return undocumented return values.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa8192f42e. Only changes: rebase, no const_cast suggestion, and tostring cleanups needed after suggestion

Tree-SHA512: c006905639bafe3045de152b00c34d9864731becb3c4f468bdd61a392f10d7e7cd89a54862c8daa8c11ac4eea0eb5f13b0f647d21e21a0a797b54191cff7238c
2021-04-03 09:26:31 +02:00
fanquake
a53eff3ce5 build: misc doc changes in bitcoin_qt.m4 2021-04-03 14:07:22 +08:00
fanquake
340fa6c0ff build: set QT_*_CFLAGS & QT_*_LIBS in PKG_CHECK_MODULES() calls 2021-04-03 14:04:58 +08:00
fanquake
4b8ad97c5b build: use QT_*_LIBS rather than passing lib names 2021-04-03 14:04:58 +08:00
fanquake
13313b2904 scripted-diff: replace Qt5 with ${qt_lib_prefix} in _BITCOIN_QT_CHECK_STATIC_LIBS
-BEGIN VERIFY SCRIPT-
sed -i -e "s/\[Qt5/\[\$\{qt_lib_prefix\}/g" build-aux/m4/bitcoin_qt.m4
sed -i -e "s/Qt5Core/\$\{qt_lib_prefix\}Core/g" build-aux/m4/bitcoin_qt.m4
-END VERIFY SCRIPT-
2021-04-03 14:04:58 +08:00
fanquake
477df3623c build: reorder libs in _BITCOIN_QT_CHECK_STATIC_LIBS (move-only) 2021-04-03 14:04:57 +08:00
fanquake
9565dafed5 Merge #21403: build: set --build when configuring packages in depends
735610940c build: set --build when configuring packages in depends (fanquake)

Pull request description:

  After reading [#Specifying-Target-Triplets](https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/autoconf.html#Specifying-Target-Triplets) in the autoconf manuel, my understanding is that this change should mostly be a no-op, as `--build` defaults to the output of [`config.guess`](https://github.com/bitcoin/bitcoin/blob/master/depends/config.guess), however, this may be slightly more correct

  >  For historical reasons, whenever you specify --host, be sure to specify --build too; this will be fixed in the future.

  and will quell some warnings in depends (#16354), i.e:
  > configure: WARNING: If you wanted to set the --build type, don't use --host.
      If a cross compiler is detected then cross compile mode will be used.

  If anything, this also explicitly enables cross-compilation mode when `--host` differs from `--build`. As for "fixed in the future", this is the case for Autoconf 2.70+.

  If we don't make this change, I think we should consider closing #16354. We've now got a good handle on our depends configure flags, and we can just leave the last few warnings as they are. It's also unlikely that we are going to add the `xorg-macros` package to depends.

  Guix builds at 735610940c:
  ```bash
  bash-5.1# find output -type f -name *$(git rev-parse --short HEAD)*.* -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  afabf99650d57abd3df2c7e6253a3281f9814ada77d01c28ffb57c740b14fd8e  output/bitcoin-735610940c0c-aarch64-linux-gnu-debug.tar.gz
  6e9dedced61489f7734c5387ee05823db68eb4b890fa05f11541c1b966576446  output/bitcoin-735610940c0c-aarch64-linux-gnu.tar.gz
  5333a3c5978a61966251e56fedab8e059d9d8a1a7231a0908095bb373204a81f  output/bitcoin-735610940c0c-arm-linux-gnueabihf-debug.tar.gz
  d28e1c166b96e58cf40b3f9801c3050c2785b25eeb261d44eeb31a32a75e134c  output/bitcoin-735610940c0c-arm-linux-gnueabihf.tar.gz
  e00b0149c52e9207be1cfb56ec3b715b55f41691401095f7a7038ac124f08d80  output/bitcoin-735610940c0c-osx-unsigned.dmg
  ae5052ad1a05916bc6656b034c169097471fd9bdba16b606e2bcb592c9395544  output/bitcoin-735610940c0c-osx-unsigned.tar.gz
  8586a9b8b82a8832397fcd0756aa9649adc3e61ae579fb370b99fe406298199f  output/bitcoin-735610940c0c-osx64.tar.gz
  d560a8781bae856b6593e9b731fa99de0a4ad7bd3e03084166dc98904ece9e7b  output/bitcoin-735610940c0c-powerpc64-linux-gnu-debug.tar.gz
  a24cf28b7efd5ebdc892e7e8809a05b3405bb4222f7d2eaa32de265214c78f28  output/bitcoin-735610940c0c-powerpc64-linux-gnu.tar.gz
  ad29421842dbbe96526eb597a7ff2970f0acae8c3e75dc6c2100716a2ba20606  output/bitcoin-735610940c0c-powerpc64le-linux-gnu-debug.tar.gz
  ee7a8ccfd68cc2297bb96eaaacf7b3eb939a5e39cecc3c710dbe828518f1bfbd  output/bitcoin-735610940c0c-powerpc64le-linux-gnu.tar.gz
  185bf4ea3b35072a8dd54aedbafc0892ec1e486c8ebd311f05f0fb47a19058cf  output/bitcoin-735610940c0c-riscv64-linux-gnu-debug.tar.gz
  803078b15a9bbaa567176827639895e762edbe80844b80400f8a9581e5311d32  output/bitcoin-735610940c0c-riscv64-linux-gnu.tar.gz
  d920327a32a42a5abadd247f0d3f3ee0ab0488a28892c11838d99bede6456723  output/bitcoin-735610940c0c-win-unsigned.tar.gz
  9f1741a37e294af153d57ad4a6dad3e5f1cdf75f1588c86cb467abf0c177ae27  output/bitcoin-735610940c0c-win64-debug.zip
  2d1a91552c0dd7bb400ce6c71d2e98de702743a97afe5f4b0945865f251f3aea  output/bitcoin-735610940c0c-win64-setup-unsigned.exe
  8a70020bb0cb68516acf4b2cc89832d72373f31611bd075826a30a817a071cd2  output/bitcoin-735610940c0c-win64.zip
  93b03837679474905538e96c44ba81ee5653795ad333dc00569b92c82dc6ea31  output/bitcoin-735610940c0c-x86_64-linux-gnu-debug.tar.gz
  6ccd046a83ef5089667ad426875319beb83e2b2a5fb4d4459d5c5ce8d75f1a31  output/bitcoin-735610940c0c-x86_64-linux-gnu.tar.gz
  12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc  output/src/bitcoin-735610940c0c.tar.gz
  ```

  Gitian builds:
  ```bash
  # macOS:
  7d087749f610e5e3eca91c730d41afd4d404808e276242d05cedb89403de247f  bitcoin-735610940c0c-osx-unsigned.dmg
  96d68b4f0042dc40f5e8e362fa84dcf8e122dcb566889feb48ad8a982847dd8b  bitcoin-735610940c0c-osx-unsigned.tar.gz
  2dea72cbe5c3e228babd88b516a6f43b0e040e9510ea95b62f07fc0815bf8d22  bitcoin-735610940c0c-osx64.tar.gz
  12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc  src/bitcoin-735610940c0c.tar.gz
  478dbda9d551a8bed985c61c625df3f543afab26989a52bc44d5a4745715af81  bitcoin-core-osx-22-res.yml

  # Windows:
  c31a5118dfa21259b9868e2c4858c635becb786c2ee8af92302da7811a1efe9b  bitcoin-735610940c0c-win-unsigned.tar.gz
  5c171ec3aee830abc8ae843d674d5fe7a57420e782ee765b6992914ae5b7671a  bitcoin-735610940c0c-win64-debug.zip
  ca010ced982f75d111e01f3759b961deb83e7bf511b1ee9c9041f4830570f2a0  bitcoin-735610940c0c-win64-setup-unsigned.exe
  62514b01281f81af6057fb12cc5e3813c1d8f6532d0d8ac8f6f2909a595aa1dd  bitcoin-735610940c0c-win64.zip
  12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc  src/bitcoin-735610940c0c.tar.gz
  e26b55d8302a419f594396db6ff441aaeb5e275e02fde7b1b4eb092270f902e7  bitcoin-core-win-22-res.yml

  # Linux:
  e5be4deb91cb372de484468c0575a04f907f982e35a9841659bb213540f770d0  bitcoin-735610940c0c-aarch64-linux-gnu-debug.tar.gz
  6169e822679ece2bb6af7027362a6f373eb939eef9d89edc58c21190b4083c7d  bitcoin-735610940c0c-aarch64-linux-gnu.tar.gz
  aee5308767c5c930853a80ab94325b2a4ab3dbe9a7a20c7a9d60d57af2b5383f  bitcoin-735610940c0c-arm-linux-gnueabihf-debug.tar.gz
  a9cf6e56d859fffd5a2b5f1e6360515bc6b06b048776b93cc08954859457a251  bitcoin-735610940c0c-arm-linux-gnueabihf.tar.gz
  7617d85c358e6614fd40026a8d346f9edba5b5139e3daade2ec2aec72aed3a1f  bitcoin-735610940c0c-powerpc64-linux-gnu-debug.tar.gz
  f76e46a8b130812541ebcaaba173b4cbe2ec1781e348579b44f96e9244cc2475  bitcoin-735610940c0c-powerpc64-linux-gnu.tar.gz
  19ada7abb801422bd7772d7ee559004cab11a46141a747aabd27085e8c3d200d  bitcoin-735610940c0c-powerpc64le-linux-gnu-debug.tar.gz
  f3a6bbefe9f47b3960e3b9eeb3f740ecddf8773ae76961ced6a3cdf82503e37f  bitcoin-735610940c0c-powerpc64le-linux-gnu.tar.gz
  42ca333e8f82b4ec0b2354efc7c4f1c27ed32d60c35b88e33d26833419f19e69  bitcoin-735610940c0c-riscv64-linux-gnu-debug.tar.gz
  dde930d9134fb5be1374014db3f6f503b614ae801a1d77f6f4b82a5f11eb2518  bitcoin-735610940c0c-riscv64-linux-gnu.tar.gz
  82e1b4cc8aac309a99458cc27fd0285215a8af53780094c4990303ce8948cf0c  bitcoin-735610940c0c-x86_64-linux-gnu-debug.tar.gz
  ee8caad2ccddd8eaf8aa2d19040d859130168183355b09a2a0f77a59a97fcaee  bitcoin-735610940c0c-x86_64-linux-gnu.tar.gz
  12771acb17de84cda8e81c59704c00b1d4725ff51fa84c9067dc61a8dc795bdc  src/bitcoin-735610940c0c.tar.gz
  d90545b13226a338c994d9e538f468a6fafd694273bf71e14614feeaf5727ad4  bitcoin-core-linux-22-res.yml
  ```

ACKs for top commit:
  dongcarl:
    Code Review ACK 735610940c
  hebasto:
    ACK 735610940c, tested on Linux Mint 20.1 (x86_64).

Tree-SHA512: 13ce05269309a12d03b81a539ba5585003785f7b1bb7894a9353385a6e9c709f920cc4f79322f28d7a256809754f201e70be6b917915c7b14e1461530075781d
2021-04-03 13:18:51 +08:00
Pieter Wuille
cabb566123 Disable certain false positive warnings for libsecp256k1 msvc build 2021-04-02 13:17:49 -07:00
MarcoFalke
faf843c07f refactor: Move load block thread into ChainstateManager 2021-04-02 20:39:14 +02:00
Russell Yanofsky
937fd4a66f Fix wrong wallet RPC context set after #21366
This bug doesn't have any effects currently because it only affects
external signer RPCs which aren't currently using the wallet context,
but it does cause an appveyor failure in a upcoming PR:

https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/38512882

This bug is subtle and could have been avoided if JSONRPCRequest didn't
have constructors that were so loose with type checking.  Suggested
change
https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351
eliminates these and would be a good followup for a future PR.
2021-04-02 12:48:20 -04:00
Vasil Dimov
81747b2171 test: make sure non-IP peers get discouraged and disconnected 2021-04-02 18:33:45 +02:00
Vasil Dimov
637bb6da36 test: also check disconnect in denialofservice_tests/peer_discouragement
Use `CConnmanTest` instead of `CConnman` and add the nodes to it
so that their `fDisconnect` flag is set during disconnection.
2021-04-02 18:32:51 +02:00
Vasil Dimov
4d6e246fa4 test: use pointers in denialofservice_tests/peer_discouragement
This is a non-functional change that replaces the `CNode` on-stack
variables with `CNode` pointers.

The reason for this is that it would allow us to add those `CNode`s
to `CConnman::vNodes[]` which in turn would allow us to check that they
are disconnected properly - a `CNode` object must be in
`CConnman::vNodes[]` in order for its `fDisconnect` flag to be set.

If we store pointers to the on-stack variables in `CConnman` then it
would crash at the end, trying to `delete` them.
2021-04-02 18:32:46 +02:00
Sebastian Falbesoner
ea19cc844e wallet: refactor: dedup sqlite statement deletions 2021-04-02 15:48:02 +02:00
Sebastian Falbesoner
9a3670930e wallet: refactor: dedup sqlite statement preparations 2021-04-02 15:47:11 +02:00
MarcoFalke
e08f3193b5 Merge bitcoin-core/gui#266: Doc: Copyright: Fix embedded font file location
e309646db6 Doc: Copyright: Fix embedded font file location (wodry)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK e309646db6, it was overlooked in 89e421918e.

Tree-SHA512: 016035b82b587fd195a53bcea938b2d1258b8e908ac545477e2002d713c063b3f18f01cd546aaac3ce0543aec029502fcde3b552ab3e837125a1c0f8a983996a
2021-04-02 09:35:03 +02:00
fanquake
7aa0d8adf8 Merge #21063: wallet, rpc: update listdescriptors response format
2e5f7def22 wallet, rpc: update listdescriptors response format (Ivan Metlushko)

Pull request description:

  Update `listdescriptors` response format according to [RPC interface guidelines](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#rpc-interface-guidelines).

  This is a follow up for #20226

  **Before:**
  ```
  Result:
  [                               (json array) Response is an array of descriptor objects
    {                             (json object)
      "desc" : "str",             (string) Descriptor string representation
      "timestamp" : n,            (numeric) The creation time of the descriptor
      "active" : true|false,      (boolean) Activeness flag
      "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
      "range" : [                 (json array, optional) Defined only for ranged descriptors
        n,                        (numeric) Range start inclusive
        n                         (numeric) Range end inclusive
      ],
      "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
    },
    ...
  ]
  ```

  **After:**
  ```
  Result:
  {                                 (json object)
    "wallet_name" : "str",          (string) Name of wallet this operation was performed on
    "descriptors" : [               (json array) Array of descriptor objects
      {                             (json object)
        "desc" : "str",             (string) Descriptor string representation
        "timestamp" : n,            (numeric) The creation time of the descriptor
        "active" : true|false,      (boolean) Activeness flag
        "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
        "range" : [                 (json array, optional) Defined only for ranged descriptors
          n,                        (numeric) Range start inclusive
          n                         (numeric) Range end inclusive
        ],
        "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
      },
      ...
    ]
  }
  ```

ACKs for top commit:
  achow101:
    re-ACK 2e5f7def22
  meshcollider:
    utACK 2e5f7def22
  jonatack:
    re-ACK 2e5f7def22

Tree-SHA512: 49bf73e46e2a61003ce594a4bfc506eb9592ccb799c2909c43a1a527490a4b4009f78dc09f3d47b4e945d3d7bb3cd2632cf48c5ace5feed5066158cc010dddc1
2021-04-02 13:18:35 +08:00
wodry
e309646db6 Doc: Copyright: Fix embedded font file location 2021-04-02 06:18:35 +02:00
Carl Dong
e5b49a01f5 guix: Create windeploy inside distsrc-*
./windeploy is a "working directory", and therefore belongs inside
distsrc-*. Many people have noticed their Guix builds failing after
hours simply because they did not remove windeploy (but did remove the
distsrc-* directories).
2021-04-01 16:53:17 -04:00
Carl Dong
3e9982ab38 contrib: Silence git-describe when looking for tag
Otherwise, it prints a rather disturbing message to stderr:

    fatal: no tag exactly matches '<hash>'
2021-04-01 16:53:17 -04:00
Carl Dong
d5a71e9785 guix: Use --cores instead of --max-jobs
In Guix, there are two flags for controlling parallelism:

Note: When I say "derivation," think "package"

--cores=n
  - controls the number of CPU cores to build each derivation. This is
    the value passed to `make`'s `--jobs=` flag.
  - defaults to 0: as many cores as is available

--max-jobs=n
  - controls how many derivations can be built in parallel
  - defaults to 1

Therefore, if set --max-jobs=$MAX_JOBS and don't set --cores, Guix could
theoretically spin up $MAX_JOBS * $(nproc) number of threads, and that's
no good.

So we could either default to --cores=1, --max-jobs=$MAX_JOBS

  - Pro: --cores=1 means that `make` will be invoked with `-j1`,
         avoiding problems with package whose build systems and test
         suites break when running multi-threaded.

  - Con: There will be times when only 1 or 2 derivations can be built
         at a time, because the rest of the dependency graph all depend
         on those 1 or 2 derivations. During these times, the machine
         will be severely under-utilized.

or --cores=$MAX_JOBS, --max-jobs=1

  - Pro: We don't encounter prolonged periods of
         severe under-utilization mentioned above.

  - Con: Many packages' build systems and test suites break when running
         multi-threaded.

or --cores=1, --max-jobs=1 and let the user override with
$ADDITIONAL_GUIX_COMMON_FLAGS
2021-04-01 16:53:17 -04:00
W. J. van der Laan
66daf4cb3b Merge #21567: docs: fix various misleading comments
4eca20d6f7 [doc] correct comment about ATMPW (glozow)
8fa74aeb5b [doc] correct comment in chainparams (glozow)
2f8272c2a4 [doc] GetBestBlock() doesn't do nothing (gzhao408)

Pull request description:

  Came across a few misleading comments, wanted to fix them

ACKs for top commit:
  jnewbery:
    ACK 4eca20d6f7
  MarcoFalke:
    ACK 4eca20d6f7
  laanwj:
    Code review ACK 4eca20d6f7

Tree-SHA512: 5bef1f1e7703f304128cf0eb8945e139e031580c99062bbbe15bf4db8443c2ba5a8c65844833132e6646c8980c678fc1d2ab0c63e17105585d583570ee350fd0
2021-04-01 19:12:10 +02:00
glozow
4eca20d6f7 [doc] correct comment about ATMPW
ATMPW stands for AcceptToMemoryPoolWorker, which was removed in #16400.
2021-04-01 08:35:34 -07:00
glozow
8fa74aeb5b [doc] correct comment in chainparams
There are more than 3 networks.
2021-04-01 08:35:34 -07:00
gzhao408
2f8272c2a4 [doc] GetBestBlock() doesn't do nothing
This has tripped people up multiple times because it looks like
GetBestBlock is a const function returning the value of hashBlock.
2021-04-01 08:33:11 -07:00
W. J. van der Laan
086226d98a Merge #21198: net: Address outstanding review comments from PR20721
5ed535a02f [net] Changes to RunInactivityChecks (John Newbery)

Pull request description:

  Updates the RunInactivityChecks() function:

  - rename to ShouldRunInactivityChecks (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r576394790)
  - take optional time now (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r575895661)
  - call from within InactivityChecks (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r575894665)
  - update comment (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r575894343)
  - change ordering of inequality (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r574925129)
  - ~make inline (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r574903578)~

ACKs for top commit:
  laanwj:
    Code review ACK 5ed535a02f

Tree-SHA512: e6ac8e8cce5cddc84a52a40c908634c25f58be74512d642840d7bd7fa65c3d90a0f46cc19e4865b3fae7c933138247f58356167a60a5c519305cfd6d05e51f51
2021-04-01 16:36:22 +02:00
MarcoFalke
6e22b522f9 Merge #21373: test: generate fewer blocks in feature_nulldummy to fix timeouts, speed up
ccd976dd3d test: use 327 fewer blocks in feature_nulldummy (Jon Atack)
68c280f197 test, refactor: abstract the feature_nulldummy blockheight values (Jon Atack)

Pull request description:

  The resolved timeout issue seen in the CI can be reproduced locally by running `test/functional/feature_nulldummy.py --valgrind --loglevel=debug`

  Speeds up the normal test runtime for me from 3.8 to 2.2 seconds (debug build). Thanks to Marco Falke for the approach suggestion.

ACKs for top commit:
  AnthonyRonning:
    reACK ccd976dd3d - ran a few times with the rest of the tests and still passing for me with just the fewer block change.
  MarcoFalke:
    review ACK ccd976dd3d 🏝

Tree-SHA512: 38339dca4276d1972e3a5a5ee436da64e9e58fd3b50b186e34b07ade9523ac4c03f6c3869c5f2a59c23b43c44f87e712f8297a01a8d83202049c081e3eeb4445
2021-04-01 13:17:12 +02:00
fanquake
ee35532409 Merge #21300: script: Add explanatory comment to tc.sh
3a0446fad4 script: Add explanatory comment to tc.sh (dscotese)

Pull request description:

  This is a replacement for #21289

  tc.sh is used to limit bandwidth. I ran it and it is limiting my bandwidth. When I ran it, I got one error. I have not found an explanation anywhere of what the error means, but my best guess is consistent with the result, so I propose the explanatory comment to save others time when they use it and also get the error.

ACKs for top commit:
  laanwj:
    that said, LGTM ACK 3a0446fad4

Tree-SHA512: 5403a2a0fec3724625c20402a96334c3c7a620324a930c5fd828017da8911d2867aecb7a2ad94a23d1f189009d3eb197a67eb59c8e4531fd215d9b1edb600440
2021-04-01 19:16:54 +08:00
John Newbery
5ed535a02f [net] Changes to RunInactivityChecks
- rename to ShouldRunInactivityChecks (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r576394790)
- take optional time now (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r575895661)
- call from within InactivityChecks (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r575894665)
- update comment (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r575894343)
- change ordering of inequality (https://github.com/bitcoin/bitcoin/pull/20721#discussion_r574925129)
2021-04-01 11:35:27 +01:00
MarcoFalke
fa8192f42e rpc: Fail to return undocumented return values 2021-04-01 12:16:28 +02:00
MarcoFalke
80a699fda9 Merge #21525: [Bundle 4.5/n] Followup fixups to bundle 4
693414d271 node/ifaces: ChainImpl: Use an accessor for ChainMan (Carl Dong)
98c4e252f0 node/ifaces: NodeImpl: Use an accessor for ChainMan (Carl Dong)
7e8b5ee814 validation: Make BlockManager::LookupBlockIndex const (Carl Dong)
88aead263c node: Avoid potential UB by asserting assumptions (Carl Dong)
1dd8ed7a84 net_processing: Move comments to declarations (Carl Dong)
07156eb387 node/coinstats: Replace #include with fwd-declaration (Carl Dong)
7b8e976cd5 miner: Add chainstate member to BlockAssembler (Carl Dong)
e62067e7bc Revert "miner: Pass in chainstate to BlockAssembler::CreateNewBlock" (Carl Dong)
eede0647b0 Revert "scripted-diff: Invoke CreateNewBlock with chainstate" (Carl Dong)
0c1b2bc549 Revert "miner: Remove old CreateNewBlock w/o chainstate param" (Carl Dong)

Pull request description:

  Chronological history of this changeset:
  1. Bundle 4 (#21270) got merged
  2. Posthumous reviews were posted
  3. These changes were prepended in bundle 5
  4. More reviews were added in bundle 5
  5. Someone suggested that we split the prepended changes up to another PR
  6. This is that PR

  In the future, I will just do posthumous review changes in another PR instead. I apologize for the confusion.

  Addresses posthumous reviews on bundle 4:
    - From jnewbery:
      - https://github.com/bitcoin/bitcoin/pull/21270#issuecomment-796738048
        - I didn't fix this one, but I added a `TODO` comment so that we don't lost track of it
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592291225
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592296942
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592299738
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592301704
    - From MarcoFalke:
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593096212
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593097032
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593097867
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593100570

  Addresses reviews on bundle 5:
  - Checking chainman existence before locking cs_main
    - MarcoFalke
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r596601776
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r596601876
  - Appropriate locking, usage of chainman, and control flow in `src/node/interfaces.cpp`
    - MarcoFalke
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r596601383
    - jnewbery
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597029360
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597029921
    - ryanofsky
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597163828
  - Style/comment formatting changes
    - jnewbery
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597026552
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597027186
  - Making LookupBlockIndex const
    - jnewbery
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597035062

ACKs for top commit:
  MarcoFalke:
    review ACK 693414d271 🛐
  ryanofsky:
    Code review ACK 693414d271. I reviewed this previously as part of #21391. I am a fan of the increasingly complicated bundle numbering, and kind of hope there in the next round there is some way we can get bundles 5.333333 and 5.666667!
  jamesob:
    ACK 693414d271 ([`jamesob/ackr/21525.1.dongcarl.bundle_4_5_n_followup_f`](https://github.com/jamesob/bitcoin/tree/ackr/21525.1.dongcarl.bundle_4_5_n_followup_f))

Tree-SHA512: 9bdc199f70400d01764e1bd03c25bdb6cff26dcef60e4ca3b649baf8d017a2dfc1f058099067962b4b6ccd32d078002b1389d733039f4c337558cb70324c0ee3
2021-04-01 10:58:53 +02:00
fanquake
c2caa0fc4d Merge #21311: rpc: document optional fields for getchaintxstats result
73e1f7d754 rpc: document optional fields for getchaintxstats result (Sebastian Falbesoner)

Pull request description:

  This mini-PR updates the result help of the `getchaintxstats` RPC by showing the following fields as "optional":
  - window_tx_count
  - window_interval
  - txrate

  Help output diff between master and PR branch:
  ```diff
  16,18c16,18
  <   "window_tx_count" : n,                (numeric) The number of transactions in the window. Only returned if "window_block_count" is > 0
  <   "window_interval" : n,                (numeric) The elapsed time in the window in seconds. Only returned if "window_block_count" is > 0
  <   "txrate" : n                          (numeric) The average rate of transactions per second in the window. Only returned if "window_interval" is > 0
  ---
  >   "window_tx_count" : n,                (numeric, optional) The number of transactions in the window. Only returned if "window_block_count" is > 0
  >   "window_interval" : n,                (numeric, optional) The elapsed time in the window in seconds. Only returned if "window_block_count" is > 0
  >   "txrate" : n                          (numeric, optional) The average rate of transactions per second in the window. Only returned if "window_interval" is > 0
  ```

ACKs for top commit:
  0xB10C:
    ACK 73e1f7d754

Tree-SHA512: 63c8db3e47a3c2d5564d53c564484b95b656e1e5deca1e9841bc90d122d3c81f02fd2b59313fd913ce81b16f7cc2969fe1dd9d6c3e23628b8ac057ea08f55daa
2021-04-01 16:26:22 +08:00
practicalswift
58580a827d net: Avoid calling getnameinfo when formatting IPv4 addresses in CNetAddr::ToStringIP 2021-04-01 08:06:01 +00:00
practicalswift
5858057384 net: Add IPv4ToString (we already have IPv6ToString) 2021-04-01 08:00:48 +00:00
fanquake
2b2ab9ab78 Merge #21544: rpc: Missing doc updates for bumpfee psbt update
1111896eb7 doc: Merge release notes (MarcoFalke)
faeba9819d rpc: Missing doc updates for bumpfee psbt update (MarcoFalke)

Pull request description:

  Stuff missed in #20891. Also merge release notes, so that it doesn't have to be done later.

ACKs for top commit:
  fanquake:
    ACK 1111896eb7

Tree-SHA512: c9be5a3c944e2981c83546c4761277f1ad5fb9ba97bec80d073db4229924cb48fd23cb5638217c844e05af51d80507718dd201099cbe50819986b3c47c5df7e5
2021-04-01 15:17:15 +08:00
John Newbery
caba7ae8a5 [net processing] Make RelayAddress() a member function of PeerManagerImpl 2021-04-01 08:08:11 +01:00
John Newbery
86acc96469 [net processing] Take NodeId instead of CNode* as originator for RelayAddress()
This makes the following commit easier.
2021-04-01 08:08:11 +01:00
MarcoFalke
539e4eec63 Merge #21236: net processing: Extract addr send functionality into MaybeSendAddr()
935d488922 [net processing] Refactor MaybeSendAddr() (John Newbery)
01a79ff924 [net processing] Fix overindentation in MaybeSendAddr() (John Newbery)
38c0be5da3 [net processing] Refactor MaybeSendAddr() - early exits (John Newbery)
c87423c58b [net processing] Change MaybeSendAddr() to take a reference (John Newbery)
ad719297f2 [net processing] Extract `addr` send functionality into MaybeSendAddr() (John Newbery)
4ad4abcf07 [net] Change addr send times fields to be guarded by new mutex (John Newbery)
c02fa47baa [net processing] Only call GetTime() once in SendMessages() (John Newbery)

Pull request description:

  This continues the work of moving application layer data into net_processing. It refactors `addr` send functionality into its own function `MaybeSendAddr()` and flattens/simplifies the code. Isolating and simplifying the addr handling code makes subsequent changes (which will move addr data and logic into net processing) easier to review.

  This is a pure refactor. There are no functional changes.

  For motivation of the project, see #19398.

ACKs for top commit:
  sipa:
    utACK 935d488922
  hebasto:
    ACK 935d488922, I have reviewed the code and it looks OK, I agree it can be merged.
  MarcoFalke:
    review ACK 935d488922 🐑

Tree-SHA512: 4e9dc84603147e74f479a211b42bcf315bdf5d14c21c08cf0b17d6c252775b90b012f0e0d834f1a607ed63c7ed5c63d5cf49b134344e7b64a1695bfcff111c92
2021-04-01 08:29:53 +02:00
Hennadii Stepanov
eb6156ba1b qt: Handle exceptions in BitcoinGUI::addWallet slot 2021-04-01 03:05:31 +03:00
Hennadii Stepanov
f7e260a471 qt: Add GUIUtil::ExceptionSafeConnect function
Throwing an exception from a slot invoked by Qt's signal-slot connection
mechanism is considered undefined behavior, unless it is handled within
the slot. The GUIUtil::ExceptionSafeConnect function should be used for
exception handling within slots.
2021-04-01 03:05:31 +03:00
Hennadii Stepanov
64a8755af3 qt: Add BitcoinApplication::handleNonFatalException function
This helper function will be used in the following commits.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2021-04-01 03:05:31 +03:00
Hennadii Stepanov
af7e365b15 qt: Make PACKAGE_BUGREPORT link clickable
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2021-04-01 03:05:00 +03:00
Sebastian Falbesoner
b01cd9471f test: check that _all_ invalid-CLTV txs are rejected after BIP65 activation 2021-03-31 21:30:25 +02:00
Sebastian Falbesoner
dbc1981474 test: check that _all_ invalid-CLTV txs are allowed in a block pre-BIP65 2021-03-31 21:09:59 +02:00
Sebastian Falbesoner
8d0ce50c48 test: prepare cltv_invalidate to test all failure reasons in feature_cltv.py
only the "top item on the stack is less than 0" is used in the test right now
2021-03-31 21:09:59 +02:00
Sebastian Falbesoner
ce994e1202 test: add tx modfication helper function in feature_cltv.py
+ reformat python imports
+ fix PEP8 warnings (all except E501 line too long)
2021-03-31 21:09:59 +02:00
W. J. van der Laan
602b038d43 Merge #21366: refactor: replace util::Ref with std::any (C++17)
916ab0195d remove unused class util::Ref and its unit test (Sebastian Falbesoner)
8dbb87a393 refactor: replace util::Ref by std::any (C++17) (Sebastian Falbesoner)
95cccf8a4b util: introduce helper AnyPtr to access std::any instances (Sebastian Falbesoner)

Pull request description:

  As described in `util/ref.h`: "_This implements a small subset of the functionality in C++17's std::any class, and **can be dropped when the project updates to C++17**_". For accessing the contained object of a `std::any` instance, a helper template function `AnyPtr` is introduced (thanks to ryanofsky).

ACKs for top commit:
  hebasto:
    re-ACK 916ab0195d, with command
  ryanofsky:
    Code review ACK 916ab0195d. Changes since last review: rebase and replacing types with `auto`. I might have used `const auto*` and `auto*` instead of plain `auto` because I think the qualifiers are useful, but this is all good.

Tree-SHA512: fe2c3e4f5726f8ad40c61128339bb24ad11d2c261f71f7b934b1efe3e3279df14046452b0d9b566917ef61d5c7e0fd96ccbf35ff810357e305710f5002c27d47
2021-03-31 20:17:39 +02:00
John Newbery
935d488922 [net processing] Refactor MaybeSendAddr()
Changes to make MaybeSendAddr simpler and easier to maintain/update:

- assert invariant that node.vAddrToSend.size() can never exceed
  MAX_ADDR_TO_SEND
- erase known addresses from vAddrToSend in one pass
- no check for (vAddr.size() >= MAX_ADDR_TO_SEND) during iteration,
  since vAddr can never exceed MAX_ADDR_TO_SEND.
2021-03-31 18:06:51 +01:00
glozow
8cac2923f5 [test] remove invalid test from tx_valid.json
This exact test is also already in tx_invalid.json#L29
It will also test with no-P2SH flags, so duplicating with
different flags is not necessary.
2021-03-31 06:14:26 -07:00
fanquake
735610940c build: set --build when configuring packages in depends
After reading
https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.70/autoconf.html#Specifying-Target-Triplets,
my understanding is that this change should mostly be a no-op, as
--build defaults to the output of config.guess, however, this may be
slightly more correct
>  For historical reasons, whenever you specify --host, be sure to
> specify --build too; this will be fixed in the future.

and will quell some warnings in depends (#16354). If anything, this
also explicitly enables cross-compilation mode when `--host` differs
from `--build`.

As for "fixed in the future", this is the case for Autoconf 2.70+.
2021-03-31 17:05:52 +08:00
fanquake
b14462083f Merge #21486: build: link against -lsocket if required for *ifaddrs
4783115fd4 net: add ifaddrs.h include (fanquake)
879215e665 build: check if -lsocket is required with *ifaddrs (fanquake)
87deac66aa rand: only try and use freeifaddrs if available (fanquake)

Pull request description:

  Fixes #21485 by linking against `-lsocket` when it's required for using `*ifaddrs` functions.

ACKs for top commit:
  laanwj:
    Code review ACK 4783115fd4
  hebasto:
    ACK 4783115fd4, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 4542e036e9b029de970eff8a9230fe45d9204bb22313d075f474295d49bdaf1f1cbb36c0c6e2fa8dbbcdba518d8d3a68a6116ce304b82414315f333baf9af0e4
2021-03-31 14:38:06 +08:00
fanquake
6526a1644c test: small cleanup in RPCNestedTests tests
Add missing includes.
Remove obsolete comment about Qt 5.3
(fd46c4c001)
2021-03-31 14:03:11 +08:00
fanquake
e4a2918238 Merge #21542: ci: Bump macOS VM image to the latest version
b8e76479ce ci: Bump macOS VM image to the latest version (Hennadii Stepanov)

Pull request description:

  On Cirrus CI bump macOS VM from Catalina to Big Sur.

ACKs for top commit:
  fanquake:
    ACK b8e76479ce - I'm always going to question bumping things for the sake of it/when there's no reasoning given. In this case, moving from building on macOS 10.x to 11.x shouldn't really lose us any coverage, and may turn up potential macOS quirks a bit earlier.

Tree-SHA512: ab2daee194683ab0553328020fd2fcb918160f466cd380c542e1a9b44f5bea3664fb40b032f1b611ee0107b0efbe278230e067316e2373c3cb0470b205dd2f9d
2021-03-31 13:10:20 +08:00
fanquake
1a0d1452a0 Merge #21457: build: split libtapi and clang out of native_cctools
765e0be534 build: split native_cctools (fanquake)

Pull request description:

  This splits our native cctools package into two additional packages: `native_clang` and `native_libtapi`. This is in an effort to not only make our mac toolchain more understandable, but also to reduce duplication, and as a nice side-effect, fix the issue mentioned [here](https://github.com/bitcoin/bitcoin/pull/19817#issuecomment-741763289).

  Everything about the current build process should remain the same. For gitian, that is:
  * Download LLVM Clang 8.0.0 binary.
  * Build libtapi using downloaded Clang.
  * Build cctools using downloaded Clang and libtapi.
  * Build the rest of depends..

  For Guix (`FORCE_USE_SYSTEM_CLANG=1`):
  * Build libtapi using using system Clang.
  * Build cctools using system Clang and libtapi.
  * Build the rest of depends..

  After this has been merged, my plan is to combine a modified version of #20454 and #21414 with #19817, and  from what I  understand that will be enough to support Apple Arm cross compilation.

  Builds at 477ed59f49f39ae4272219a1216ee19fb72fdde5:
  Guix:
  ```bash
  find output -type f -name *$(git rev-parse --short HEAD)*.* -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  7e57b9e5a2109d1a35f0091d86f975c1b1d73ac70ac2609cefbe1134efbf2c87  output/bitcoin-477ed59f49f3-osx-unsigned.dmg
  dd11e71c2634ac2fa883d1e45cbd6de194fad37624bb56b8b8a6213fd40d6050  output/bitcoin-477ed59f49f3-osx-unsigned.tar.gz
  64384eaa2fd9768992d86a06a1414c9e92e84ba21a875696483df2bb5828e2a2  output/bitcoin-477ed59f49f3-osx64.tar.gz
  8a889e88db952d2c82ef44713c04aba95b777441f578738ff6d8a0d251e51da3  output/src/bitcoin-477ed59f49f3.tar.gz
  ```

  Gitian:
  ```bash
  d0eee8542d5f3d662555ad7218d2dc9f3f862656e65bcb2f01f256bfa0deead2  bitcoin-477ed59f49f3-osx-unsigned.dmg
  ba7bc94897e42e7a037e352c4e4e1730f181c6d76b6d6a2785bbd7bf85614c83  bitcoin-477ed59f49f3-osx-unsigned.tar.gz
  c4426d1d310a2fbffcaf2b7df0da4ec97bd11aab07085006dae68777b03f6372  bitcoin-477ed59f49f3-osx64.tar.gz
  8a889e88db952d2c82ef44713c04aba95b777441f578738ff6d8a0d251e51da3  src/bitcoin-477ed59f49f3.tar.gz
  a746831467dc8ff17ec5df06fc9288a859c1961d8c0b632d97b42f080dbd825d  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  dongcarl:
    reACK 765e0be534
  hebasto:
    ACK 765e0be534, verified building of the `native_cctools` package step-by-step.

Tree-SHA512: 61cf2b092fb8b9724adda1084e0cac9db889cd5e391914b43592aebc470fae3c1cbabc8b59a0abce6e7bad8c646694fe927f26f4deb18b60d7fd92f374f62feb
2021-03-31 08:27:35 +08:00
MarcoFalke
267b60f800 Merge #21553: fuzz: Misc refactor
fa4926cca6 fuzz: [refactor] Use IsValidFlagCombination in signature_checker fuzz target (MarcoFalke)
eeee8f5be1 fuzz: Removed unused try-catch in coins_view (MarcoFalke)
fa98f3f66e fuzz: [refactor] Use ConsumeScript in signature_checker fuzz target (MarcoFalke)

Pull request description:

  Some small refactors to remove unused and redundant fuzz code

ACKs for top commit:
  practicalswift:
    cr re-ACK fa4926cca6

Tree-SHA512: eb07a2140caad7b31495b76385fc7634cf5b6daa4947f430ebb127eb1375583dc11e541a0a42d0e5d93d430480b8a815b93974450fd5ed897528a2d47c752f86
2021-03-30 20:07:53 +02:00
Carl Dong
693414d271 node/ifaces: ChainImpl: Use an accessor for ChainMan 2021-03-30 13:52:22 -04:00
Carl Dong
98c4e252f0 node/ifaces: NodeImpl: Use an accessor for ChainMan 2021-03-30 13:52:22 -04:00
Carl Dong
7e8b5ee814 validation: Make BlockManager::LookupBlockIndex const 2021-03-30 13:52:22 -04:00
Carl Dong
88aead263c node: Avoid potential UB by asserting assumptions 2021-03-30 13:52:22 -04:00
Carl Dong
1dd8ed7a84 net_processing: Move comments to declarations
Also:
- Remove extraneous blank line
2021-03-30 13:52:22 -04:00
Wladimir J. van der Laan
f9e86d8966 Merge #21387: p2p: Refactor sock to add I2P fuzz and unit tests
40316a37cb test: add I2P test for a runaway SAM proxy (Vasil Dimov)
2d8ac77970 fuzz: add tests for the I2P Session public interface (Vasil Dimov)
9947e44de0 i2p: use pointers to Sock to accommodate mocking (Vasil Dimov)
82d360b5a8 net: change ConnectSocketDirectly() to take a Sock argument (Vasil Dimov)
b5861100f8 net: add connect() and getsockopt() wrappers to Sock (Vasil Dimov)
5a887d49b2 fuzz: avoid FuzzedSock::Recv() repeated errors with EAGAIN (Vasil Dimov)
3088f83d01 fuzz: extend FuzzedSock::Recv() to support MSG_PEEK (Vasil Dimov)
9b05c49ade fuzz: implement unimplemented FuzzedSock methods (Vasil Dimov)

Pull request description:

  Change the networking code and the I2P code to be fully mockable and use `FuzzedSocket` to fuzz the I2P methods `Listen()`, `Accept()` and `Connect()`.

  Add a mocked `Sock` implementation that returns a predefined data on reads and use it for a regression unit test for the bug fixed in https://github.com/bitcoin/bitcoin/pull/21407.

ACKs for top commit:
  practicalswift:
    Tested ACK 40316a37cb
  MarcoFalke:
    Concept ACK 40316a37cb
  jonatack:
    re-ACK 40316a37cb reviewed `git range-diff 01bb3afb 23c861d 40316a3` and the new unit test commit, debug built, ran unit tests, ran bitcoind with an I2P service and network operation with seven I2P peers (2 in, 5 out) is looking nominal
  laanwj:
    Code review ACK 40316a37cb

Tree-SHA512: 7fc4f129849e16e0c7e16662d9f4d35dfcc369bb31450ee369a2b97bdca95285533bee7787983e881e5a3d248f912afb42b4a2299d5860ace7129b0b19623cc8
2021-03-30 17:41:13 +02:00
Wladimir J. van der Laan
dede9eb924 Merge #20197: p2p: protect onions in AttemptToEvictConnection(), add eviction protection test coverage
0cca08a8ee Add unit test coverage for our onion peer eviction protection (Jon Atack)
caa21f586f Protect onion+localhost peers in ProtectEvictionCandidatesByRatio() (Jon Atack)
8f1a53eb02 Use EraseLastKElements() throughout SelectNodeToEvict() (Jon Atack)
8b1e156143 Add m_inbound_onion to AttemptToEvictConnection() (Jon Atack)
72e30e8e03 Add unit tests for ProtectEvictionCandidatesByRatio() (Jon Atack)
ca63b53ecd Use std::unordered_set instead of std::vector in IsEvicted() (Jon Atack)
41f84d5ecc Move peer eviction tests to a separate test file (Jon Atack)
f126cbd6de Extract ProtectEvictionCandidatesByRatio from SelectNodeToEvict (Jon Atack)

Pull request description:

  Now that #19991 and #20210 have been merged, we can determine inbound onion peers using `CNode::m_inbound_onion` and add it to the localhost peers protection in `AttemptToEvictConnection`, which was added in #19670 to address issue #19500.

  Update 28 February 2021: I've updated this to follow gmaxwell's suggestion in https://github.com/bitcoin/bitcoin/pull/20197#issuecomment-713865992.

  This branch now protects up to 1/4 onion peers (connected via our tor control service), if any, sorted by longest uptime. If any (or all) onion slots remain after that operation, they are then allocated to protect localhost peers, or a minimum of 2 localhost peers in the case that no onion slots remain and 2 or more onion peers were protected, sorted as before by longest uptime.

  This patch also adds test coverage for the longest uptime, localhost, and onion peer eviction protection logic to build on the welcome initial unit testing of #20477.

  Suggest reviewing the commits that move code with `colorMoved = dimmed-zebra` and `colorMovedWs = allow-indentation-change`.

  Closes #11537.

ACKs for top commit:
  laanwj:
    Code review ACK 0cca08a8ee
  vasild:
    ACK 0cca08a8ee

Tree-SHA512: 2f5a63f942acaae7882920fc61f0185dcd51da85e5b736df9d1fc72343726dd17da740e02f30fa5dc5eb3b2d8345707aed96031bec143d48a2497a610aa19abd
2021-03-30 16:20:47 +02:00
MarcoFalke
1999baac30 Merge #20228: addrman: Make addrman a top-level component
3fc06d3d7b [net] remove fUpdateConnectionTime from FinalizeNode (John Newbery)
7c4cc67c0c [net] remove CConnman::AddNewAddresses (John Newbery)
bcd7f30b79 [net] remove CConnman::MarkAddressGood (John Newbery)
8073673dbc [net] remove CConnman::SetServices (John Newbery)
392a95d393 [net_processing] Keep addrman reference in PeerManager (John Newbery)
1c25adf6d2 [net] Construct addrman outside connman (John Newbery)

Pull request description:

  Addrman is currently a member variable of connman. Make it a top-level component with lifetime owned by node.context, and add a reference to addrman in peerman. This allows us to eliminate some functions in connman that are simply forwarding requests to addrman, and simplifies the connman-peerman interface.

  By constructing the addrman in init, we can also add parameters to the ctor, which allows us to test it better. See #20233, where we enable consistency checking for addrman in our functional tests.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3fc06d3d7b only change is squash 🏀
  vasild:
    ACK 3fc06d3d7b

Tree-SHA512: 17662c65cbedcd9bd1c194914bc4bb4216f4e3581a06222de78f026d6796f1da6fe3e0bf28c2d26a102a12ad4fbf13f815944a297f000e3acf46faea42855e07
2021-03-30 12:28:15 +02:00
MarcoFalke
fa4926cca6 fuzz: [refactor] Use IsValidFlagCombination in signature_checker fuzz target
Can be reviewed with --color-moved=dimmed-zebra
2021-03-30 10:42:45 +02:00
MarcoFalke
eeee8f5be1 fuzz: Removed unused try-catch in coins_view 2021-03-30 10:16:22 +02:00
MarcoFalke
fa98f3f66e fuzz: [refactor] Use ConsumeScript in signature_checker fuzz target 2021-03-30 09:30:43 +02:00
fanquake
765e0be534 build: split native_cctools 2021-03-30 14:54:08 +08:00
Pieter Wuille
0b188b751f Clean up context dependent checks in descriptor parsing
This changes all context dependent checks in the parser to be
disjunctions of equality checks, rather than also including inequalities.
This makes sure that adding a new context enum in the future won't change
semantics for existing checks.

The error messages are also made a bit more consistent.
2021-03-29 17:44:13 -07:00
Pieter Wuille
33275a9649 refactor: move uncompressed-permitted logic into ParsePubkey*
This is a preparation for parsing xonly pubkeys, which will complicate
this logic. It's cleaner to put the decision logic close to the public
key parsing itself.
2021-03-29 17:44:13 -07:00
Pieter Wuille
17e006ff8d refactor: split off subscript logic from ToStringHelper
This will allow subclasses to overwrite the serialization of subscript
arguments without needing to reimplement all the rest of the ToString
logic.
2021-03-29 17:44:07 -07:00
fanquake
3ececa76b7 Merge #21543: build: Enable safe warnings for msvc builds
7c543bc27b build: Enable safe warnings for msvc builds (Hennadii Stepanov)

Pull request description:

  Disabling of some specific warnings no longer needed.

ACKs for top commit:
  MarcoFalke:
    ACK 7c543bc27b if green
  practicalswift:
    ACK 7c543bc27b if green: compiler warnings are good, compiler safety warnings are great
  sipsorcery:
    tACK 7c543bc27b.

Tree-SHA512: a8253a5a0ccc22b8622ae66efae77a163c54bbd6dbd7e28b043ad4eb474b2c8e5c7b22d3870b15a2b9b91bade0a5ae5bfc5ee765d132dc32ddbbe690fb0146d9
2021-03-30 08:42:15 +08:00
Pieter Wuille
6ba5dda0c9 Account for key cache indices in subexpressions
This has no effect for now, as the only fragments with sub-script
expressions (sh, wsh) only allow one, and don't have key expressions
in them.

A future Taproot descriptor will however violate both, and we want
the keys in different sub-scripts to be assigned non-overlapping
cache indices.
2021-03-29 17:38:41 -07:00
Pieter Wuille
4441c6f3c0 Make DescriptorImpl support multiple subscripts
So far, no descriptor exists that supports more than one sub-script
descriptor. This will change with taproot, so prepare for this by
changing the m_subdescriptor_arg from a unique_ptr to a vector of
unique_ptr's.
2021-03-29 17:38:38 -07:00
Pieter Wuille
a917478db0 refactor: move population of out.scripts from ExpandHelper to MakeScripts
There are currently two DescriptorImpl subclasses that rely on the functionality
that ExpandHelper automatically adds subscripts to the output SigningProvider.

Taproot descriptors will have subscripts, but we don't want them in the
SigningProvider's bare script field. To avoid them ending up there, move this
functionality into the specific classes' MakeScripts implementation.
2021-03-29 16:40:22 -07:00
Pieter Wuille
84f3939ece Remove support for subdescriptors expanding to multiple scripts 2021-03-29 16:40:22 -07:00
Sebastian Falbesoner
916ab0195d remove unused class util::Ref and its unit test 2021-03-29 23:29:47 +02:00
Sebastian Falbesoner
8dbb87a393 refactor: replace util::Ref by std::any (C++17) 2021-03-29 23:29:42 +02:00
Sebastian Falbesoner
95cccf8a4b util: introduce helper AnyPtr to access std::any instances
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2021-03-29 22:37:59 +02:00
MarcoFalke
1111896eb7 doc: Merge release notes 2021-03-29 15:57:14 +02:00
MarcoFalke
faeba9819d rpc: Missing doc updates for bumpfee psbt update
Adds updates that have been missed in commit
ea0a7ec949:

* RPC help doc update
* Release notes update
* Remove "mutable" keyword from lambda
2021-03-29 15:56:51 +02:00
MarcoFalke
1c7be9ab90 Merge #20286: rpc: deprecate addresses and reqSigs from rpc outputs
90ae3d8ca6 doc: Add release notes for -deprecatedrpc=addresses and bitcoin-tx (Michael Dietz)
085b3a7299 rpc: deprecate `addresses` and `reqSigs` from rpc outputs (Michael Dietz)

Pull request description:

  Considering the limited applicability of `reqSigs` and the confusing output of `1` in all cases except bare multisig, the `addresses` and `reqSigs` outputs are removed for all rpc commands.

  1) add a new sane "address" field (for outputs that have an identifiable address, which doesn't include bare multisig)
  2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact (with all weird/wrong behavior they have now)
  3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely always.

  Note: Some light refactoring done to allow us to very easily delete a few chunks of code (marked with TODOs) when we remove this deprecated behavior.

  Using `IsDeprecatedRPCEnabled` in core_write.cpp caused some circular dependencies involving core_io

  Circular dependencies were caused by rpc/util unnecessarily importing node/coinstats and node/transaction. Really what rpc/util needs are some fundamental type/helper-function definitions. So this was cleaned up to make more sense.

  This fixes #20102.

ACKs for top commit:
  MarcoFalke:
    re-ACK 90ae3d8ca6 📢

Tree-SHA512: 8ffb617053b5f4a8b055da17c06711fd19632e0037d71c4c8135e50c8cd7a19163989484e4e0f17a6cc48bd597f04ecbfd609aef54b7d1d1e76a784214fcf72a
2021-03-29 15:14:31 +02:00
John Newbery
01a79ff924 [net processing] Fix overindentation in MaybeSendAddr()
Reviewer hint: review with `git diff --ignore-all-space`.
2021-03-29 12:15:23 +01:00
John Newbery
38c0be5da3 [net processing] Refactor MaybeSendAddr() - early exits
Add early exit guard clauses if node.RelayAddrsWithConn() is false or if
current_time < node.m_next_addr_send. Add comments.

This commit leaves some lines over-indented. Those will be fixed in a
subsequent whitespace-only commit.
2021-03-29 12:15:23 +01:00
John Newbery
c87423c58b [net processing] Change MaybeSendAddr() to take a reference
Change name of CNode parameter to node now that it's no longer a
pointer.
2021-03-29 12:15:23 +01:00
John Newbery
ad719297f2 [net processing] Extract addr send functionality into MaybeSendAddr()
Reviewer hint: review with

 `git diff --color-moved=dimmed-zebra --ignore-all-space`
2021-03-29 12:15:23 +01:00
John Newbery
4ad4abcf07 [net] Change addr send times fields to be guarded by new mutex 2021-03-29 12:15:23 +01:00
John Newbery
c02fa47baa [net processing] Only call GetTime() once in SendMessages()
We currently call GetTime() 4 times in SendMessages(). Consolidate this to
once GetTime() call.
2021-03-29 12:15:23 +01:00
MarcoFalke
4399dc8142 Merge #21509: p2p: Don't send FEEFILTER in blocksonly mode.
beead33a21 [test] no send feefilters when txrelay is turned off (glozow)
18a9b27dd6 p2p: Don't send FEEFILTER in blocksonly mode (Martin Zumsande)

Pull request description:

  The purpose of FEEFILTER messages (BIP 133) is to inform our peers that we do not want transactions below a specified fee rate.
  In blocksonly mode, we do not want our peer to send us any transactions at all (and will disconnect if a peer still sends a transaction INV or TX). 

  Therefore, I don't think that it makes sense to send FEEFILTER messages every 10 minutes on average in blocksonly mode - this PR disables it.

  Note that on block-relay-only connections, FEEFILTER is already disabled, just not in blocksonly mode.

ACKs for top commit:
  glozow:
    re ACK beead33a21 🙂 thanks for adding the test!
  amitiuttarwar:
    reACK beead33a21
  MarcoFalke:
    review ACK beead33a21
  jnewbery:
    reACK beead33a21

Tree-SHA512: e748cd52fe23d647fa49008b020389956ac508e16ce9fd108d8afb773bff95788298ae229162bd70215d7246fc25c796484966dc05890b0b4ef601f9cd35628b
2021-03-29 11:56:47 +02:00
MarcoFalke
cf11f9c22f Merge #21531: test: remove qt byteswap compattests
9ac86bcc0d test: remove qt byteswap compattests (fanquake)

Pull request description:

  These were added as part of #9366 when with fixing issues with Protobuf.

  Now that we no-longer use Protobuf, there's no reason to maintain a duplicate set of byteswap tests in the qt tests. Our other set of byteswap tests are here: https://github.com/bitcoin/bitcoin/blob/master/src/test/bswap_tests.cpp.

ACKs for top commit:
  laanwj:
    Code review ACK 9ac86bcc0d

Tree-SHA512: 72ba131a5f8fbd9fdbbc4e1f95baa794496c960b12e0271700c632c6511b7e1b331e8db07a201838b4d56b2aeeb43d4de4e10265ea07ab14241307fa14d3342e
2021-03-29 11:52:58 +02:00
MarcoFalke
3bcd278aa6 Merge bitcoin-core/gui#154: qt: Support macOS Dark mode
dc4551c22c remove incompatibility release note for darkmode on macos (Sylvain Goumy)
303cfc6227 allow darkmode on macos build (Sylvain Goumy)
78f75a2d60 Allow icon colorization on mac os to better support dark mode (Uplab)

Pull request description:

  Allow icons to be colorized on macOS to support native Dark mode color scheme.

  Rendering on macOS Big Sur before PR:
  ![macos-darkmode-before-pr](https://user-images.githubusercontent.com/5577626/102502739-43f3af80-407f-11eb-9263-5bbc27b371c2.png)

  Rendering on macOS Big Sur after PR:
  ![macos-darkmode-after-pr](https://user-images.githubusercontent.com/5577626/102502678-350cfd00-407f-11eb-8b98-e271f2688c36.png)

  Light mode stay visually unchanged.

  <del>Note, that this currently only affect the build from source, as the macos dmg includes an attributes to force light color scheme on macos windows (see https://github.com/bitcoin/bitcoin/pull/14593). </del>
  <del>But once all glitches are fixed, we will be able to remove this temporary fix. </del>
  Edit: this PR is know including the removal of `NSRequiresAquaSystemAppearance` on Info.plist file so that the color fix is apply to every build.

  Linked issues: #68 #136

ACKs for top commit:
  hebasto:
    re-ACK dc4551c22c
  jarolrod:
    ACK dc4551c22c

Tree-SHA512: 1c3a4dec796063e61fcaf80112afc2b15c8669a1cd30ebd537cea96647c20215f8f80289719f905820bb0c490c8c1f94bfae4bb32f9c6d1fdd4e8f199ebb559f
2021-03-29 11:17:19 +02:00
MarcoFalke
11840509fb Merge #21484: doc: Add release notes for #18335
51eef4a03e doc: Add release notes for #18335 (rpc work queue exceeded error) (Larry Ruane)

Pull request description:

  Follow-on to #18335

ACKs for top commit:
  darosior:
    ACK 51eef4a03e

Tree-SHA512: 863d92cb1c23493d9c8c42ed89b30ebd59092e44f159de4cefbabfe4101e7d7d40f24776ff3fcf39dedf90b45fc25845cf7a2177af38729ce2118d75c3cd779b
2021-03-29 11:09:08 +02:00
Hennadii Stepanov
7c543bc27b build: Enable safe warnings for msvc builds 2021-03-29 11:01:19 +03:00
MarcoFalke
97e6a7f98c Merge #21477: test: Add test for CNetAddr::ToString IPv6 address formatting (RFC 5952)
732c7bddeb tests: Add test for CNetAddr::ToString IPv6 address formatting (RFC 5952) (practicalswift)

Pull request description:

  Test that `CNetAddr::ToString` formats IPv6 addresses with zero compression and canonicalisation as described in [RFC 5952 ("A Recommendation for IPv6 Address Text Representation")](https://tools.ietf.org/html/rfc5952).

  Solving #21466 will hopefully be trivial with the ability to check zero compression correctness against these tests.

ACKs for top commit:
  vasild:
    ACK 732c7bddeb

Tree-SHA512: 31a1378aa435ba4171490a2e15d7280a175292270eb001b47d367e010c6ac9b83420b82bbeab22211f8f500c69e21878047c87adf216263b3420b6bb2a5d2bfb
2021-03-29 09:27:05 +02:00
MarcoFalke
ea3c9a92c6 Merge bitcoin-core/gui#251: Improve URI/file handling message
ef3e1d7272 qt: Improve URI/file handling message (Hennadii Stepanov)

Pull request description:

  This PR:
  - fixes missing spaces after full stops
  - makes the translation context much bigger

  The latter is the main motivation for this PR, as I became a translator 🐅

  Screenshots:
  - master (a9d1b40d53)

  ![DeepinScreenshot_select-area_20210317211750](https://user-images.githubusercontent.com/32963518/111527570-bd776880-8768-11eb-9035-96bb08067e74.png)

  - this PR:

  ![Screenshot from 2021-03-17 21-13-36](https://user-images.githubusercontent.com/32963518/111527727-e7308f80-8768-11eb-95c7-e8b802bfed5f.png)

ACKs for top commit:
  jarolrod:
    ACK ef3e1d7272

Tree-SHA512: 8fbd1e3731b75866356fae201b3129126001600ca0197e83c05825e8c5bbbcf0132d6a6b808d7a5cbfbdde75ed1865ecbb651c30017570abd7c5803eff2b9306
2021-03-29 07:39:42 +02:00
fanquake
9ac86bcc0d test: remove qt byteswap compattests
These were added as part of #9366 to fix issues with Protobuf.

Now that we no-longer use Protobuf, there's no reason to maintain a
duplicate set of byteswap tests for qt.
2021-03-29 11:12:26 +08:00
fanquake
4783115fd4 net: add ifaddrs.h include 2021-03-29 11:09:44 +08:00
fanquake
879215e665 build: check if -lsocket is required with *ifaddrs 2021-03-29 11:09:40 +08:00
fanquake
87deac66aa rand: only try and use freeifaddrs if available 2021-03-29 11:08:29 +08:00
fanquake
de4d3ba437 Merge #21541: ci: Build depends only once for Android build
fa52d7d3ad cirrus: Add missing depends_sources_cache to Android task (MarcoFalke)
ffff4e7373 cirrus: Only cache releases when needed (MarcoFalke)
fa97a17ac3 ci: Bump Android cross-build to Ubuntu Focal (MarcoFalke)
fac577d423 ci: Build depends only once for Android build (MarcoFalke)
fa908a41f3 ci: Set DEPENDS_DIR when setting BASE_ROOT_DIR (MarcoFalke)

Pull request description:

  Currently the Android task has several issues:

  * It is missing a cache instruction, thus failing the build on Cirrus CI
  * It is running the depends build twice

  Fix those issues

ACKs for top commit:
  ryanofsky:
    Code review ACK fa52d7d3ad. Only change since last review is adding descriptions and changing new RUN_UNIT_TESTS line from true to false. (I assume that change doesn't do anything because even though prior default was true, it's a cross compiled build and enabling unit tests would have no effect.)
  hebasto:
    ACK fa52d7d3ad, I have reviewed the code and it looks OK, I agree it can be merged after passing CI.

Tree-SHA512: ae7c68e357068176a0e58285b83521353321c65862cee4ff56f413c51b00398062f0ee6775bfbbf28fda420cf5a24000a2286fbcf6cc7f3729b7805bc8419726
2021-03-29 11:04:55 +08:00
MarcoFalke
fa52d7d3ad cirrus: Add missing depends_sources_cache to Android task
This cache entry is required for completeness. The file
src/Makefile.qt.include needs it in this line:
 QT_BASE_PATH = $(shell find ../depends/sources/ -maxdepth 1 -type f -regex ".*qtbase.*\.tar.xz")

This cache entry is tied to the depends_built_cache cache entry. Either
both are present and cached, or neither of them is. Otherwise, the build
will fail.
2021-03-28 20:06:39 +02:00
MarcoFalke
ffff4e7373 cirrus: Only cache releases when needed
This does not change behavior, but removes unneeded and empty cache
instructions for tasks that don't need them.
2021-03-28 20:06:36 +02:00
MarcoFalke
fa97a17ac3 ci: Bump Android cross-build to Ubuntu Focal
This does not change behavior, but bumping to Focal now means it doesn't
have to be done later when Bionic is no longer used and EOL.
2021-03-28 20:05:46 +02:00
MarcoFalke
fac577d423 ci: Build depends only once for Android build
Depends is currently built twice for the Android build. For example, the
same task building it twice:

* https://cirrus-ci.com/task/6673185279049728?logs=ci#L3418 (aarch64-linux-android)
* https://cirrus-ci.com/task/6673185279049728?logs=ci#L3422 (x86_64-pc-linux-gnu, 4 lines later)
2021-03-28 20:05:31 +02:00
MarcoFalke
fa908a41f3 ci: Set DEPENDS_DIR when setting BASE_ROOT_DIR
The depends dir can not be overwritten by a FILE_ENV file. Also, a FILE_ENV file
might depend on the DEPENDS_DIR value. Thus, set it before reading FILE_ENV.

This commit does not change behavior, but is required for later commits.

Can be reviewed with --color-moved=dimmed-zebra
2021-03-28 20:04:24 +02:00
Hennadii Stepanov
b8e76479ce ci: Bump macOS VM image to the latest version 2021-03-28 20:29:46 +03:00
MarcoFalke
c00852653f Merge bitcoin-core/gui#254: refactor: Drop redundant setEditTriggers(NoEditTriggers) calls
257f55c119 qt, refactor: Drop redundant setEditTriggers(NoEditTriggers) calls (Hennadii Stepanov)

Pull request description:

  The models of the both views have no `Qt::ItemIsEditable` flag:
  3c87dbe95c/src/qt/peertablemodel.cpp (L218-L224)
  3c87dbe95c/src/qt/bantablemodel.cpp (L148-L154)

ACKs for top commit:
  Talkless:
    utACK 257f55c119, seems reasonable.
  jarolrod:
    ACK 257f55c119, looks correct.

Tree-SHA512: 4356e4d785055935fba452488a5d97ed95995def97b26ab18af43a545835f9e9d4c347e4cad7952aa725179cf6e775a2208c48730feebf40e3b1a7ba5f402af0
2021-03-28 19:21:24 +02:00
fanquake
e658b0e49b Merge #21505: build: Remove unused header from the build system
0eabb2abed build: Remove unused header from the build system (Hennadii Stepanov)

Pull request description:

  The only `#include <miniupnpc/miniwget.h>` was removed in #16659.

ACKs for top commit:
  practicalswift:
    cr ACK 0eabb2abed
  fanquake:
    ACK 0eabb2abed

Tree-SHA512: 630da03875c851e80286561eae0f966c89624cbb17b90f70e2bec9a69146e79d088fc176e07a4906915770ac1cdb11341a7a431ea7cf6a59d2816e927486f335
2021-03-27 09:06:53 +08:00
MarcoFalke
19e3e65429 Merge bitcoin-core/gui#243: fix issue when disabling the auto-enabled blank wallet checkbox
915e34112b qt: fix issue when disabling the auto-enabled blank wallet checkbox (Jarol Rodriguez)

Pull request description:

  As detailed by #151, On `master` a user can create the confusing scenario where you have a disabled `Encrypt Wallet` checkbox and a selected `Disable Private Keys` checkbox after unselecting the auto-enabled `Blank Wallet` checkbox.

  This commit makes it so that when the `Blank Wallet` checkbox is auto-selected after the user selects `Disable Private keys`, unselecting it will also unselect the `Disable Private Keys` checkbox, which in turn re-enables the `Encrypt Wallet` checkbox.

  Below are screenshots comparing the behavior of selecting `Disable Private Keys` then unselecting the `Blank Wallet` between `master` and this `PR`:

  **Master:**
  | Select `Disable Private Keys` | Unselect `Blank Wallet` |
  | ----------------------------- | ------------------------ |
  | ![Screen Shot 2021-03-09 at 7 57 14 PM](https://user-images.githubusercontent.com/23396902/110560141-77405a80-8113-11eb-9285-5acba6241dcf.png) |   ![Screen Shot 2021-03-09 at 7 57 31 PM](https://user-images.githubusercontent.com/23396902/110560159-81faef80-8113-11eb-9b37-086aa39ecb9f.png)    |

  **PR:**
  | Select `Disable Private Keys` | Unselect `Blank Wallet` |
  | ----------------------------- | ------------------------ |
  | ![Screen Shot 2021-03-09 at 7 34 12 PM](https://user-images.githubusercontent.com/23396902/110560379-e3bb5980-8113-11eb-899a-3a4c6a1bc115.png) | ![Screen Shot 2021-03-09 at 7 34 20 PM](https://user-images.githubusercontent.com/23396902/110560412-f170df00-8113-11eb-8bd0-f7fe6fc0d739.png) |

ACKs for top commit:
  hebasto:
    ACK 915e34112b
  Talkless:
    ACK 915e34112b

Tree-SHA512: ce6ecbc35b94a08cabf0b8a24dbdfc874d82cc8918cc8623dce8172c7fc9c75d63a13b036bae5f7ab2c090f8d020574a542285d1651600813faf5d91e2506a8d
2021-03-26 17:30:21 +01:00
MarcoFalke
9b48b3ac42 Merge #21390: test: Test improvements for UTXO set hash tests
4f2653a890 test: Use deterministic chain in utxo set hash test (Fabian Jahr)
4973c5175c test: Remove wallet dependency of utxo set hash test (Fabian Jahr)
1a27af1d7b rpc: Improve gettxoutsetinfo help (Fabian Jahr)

Pull request description:

  Follow-ups to #19145:
  - Small improvement on the help text of RPC gettxoutsetinfo
  - Using deterministic blockchain in the test `functional/feature_utxo_set_hash.py`
  - Removing wallet dependency in the test `functional/feature_utxo_set_hash.py`

  Split out of #19521.

ACKs for top commit:
  MarcoFalke:
    review ACK 4f2653a890 👲

Tree-SHA512: 92927b3aa22b6324eb4fc9d346755313dec44d973aa69a0ebf80a8569b5f3a7cf3539721ebdba183737534b9e29b3e33f412515890f0d0b819878032a3bba8f9
2021-03-26 08:52:59 +01:00
glozow
beead33a21 [test] no send feefilters when txrelay is turned off 2021-03-25 21:41:56 +01:00
Sylvain Goumy
dc4551c22c remove incompatibility release note for darkmode on macos 2021-03-25 15:28:32 +01:00
MarcoFalke
9217f9fe73 Merge #21522: fuzz: [refactor] Use PickValue where possible
fa818ca202 fuzz: [refactor] Use PickValue where possible (MarcoFalke)

Pull request description:

  `PickValue` is a bit less typing, so I think it should be used where possible

ACKs for top commit:
  practicalswift:
    cr ACK fa818ca202: patch looks correct and `PickValue` is better :)

Tree-SHA512: 49ed030694e3b7676654f1615f033287d26e2f0bc29647e1db56e0d84e14d29080f3e1898f5df8d644d834b8ded3ce713d2425ea86a37c9279d01f86ad03c202
2021-03-25 08:02:11 +01:00
fanquake
8f94c70625 Merge #21495: build, qt: Fix static builds on macOS Big Sur
ec76bad655 build, qt: Fix static builds on macOS Big Sur (Hennadii Stepanov)

Pull request description:

  See details and the patch in https://bugreports.qt.io/browse/QTBUG-87014

  Fixes https://github.com/bitcoin-core/gui/issues/249

ACKs for top commit:
  fanquake:
    ACK ec76bad655

Tree-SHA512: f2fa4a6a1b7dd6d5adc6ef6f5169aedeb3bf45b2b087305e8ab78041755b9f04203d2b0550ee95656042d16775b06da0a17730915b7bff996dd14dd1b6d34ea7
2021-03-25 11:38:30 +08:00
Hennadii Stepanov
ec76bad655 build, qt: Fix static builds on macOS Big Sur
See details and the patch: https://bugreports.qt.io/browse/QTBUG-87014
2021-03-25 03:17:40 +02:00
Carl Dong
07156eb387 node/coinstats: Replace #include with fwd-declaration 2021-03-24 15:40:56 -04:00
Carl Dong
7b8e976cd5 miner: Add chainstate member to BlockAssembler 2021-03-24 15:40:56 -04:00
Carl Dong
e62067e7bc Revert "miner: Pass in chainstate to BlockAssembler::CreateNewBlock"
This reverts commit d0de61b764.
2021-03-24 15:40:56 -04:00
Carl Dong
eede0647b0 Revert "scripted-diff: Invoke CreateNewBlock with chainstate"
This reverts commit 46b7f29340.
2021-03-24 15:40:56 -04:00
Carl Dong
0c1b2bc549 Revert "miner: Remove old CreateNewBlock w/o chainstate param"
This reverts commit 2afcf24408.
2021-03-24 15:40:56 -04:00
MarcoFalke
ed49203daa Merge #21357: test: Unconditionally check for fRelay field in test framework
39a9ec579f Unconditionally check for fRelay field in test framework (Troy Giorshev)

Pull request description:

  picking up #20411 (rebased onto master)

  There is a discrepancy in the implementation of our p2p protocol between
  bitcoind and the testing framework.  The fRelay field is an optional
  field at the end of a version message as of protocol version 70001.
  However, when deserializing a message in bitcoind, we don't check the
  version to see if it should have an fRelay field or not.  Instead, we
  unconditionally attempt to deserialize into the field.

  This commit brings the testing framework in line with the implementation
  in core.

  This matters for a version message with the following fields:

  Version = 60000
  fRelay = 1

  Bitcoind would deserialize this into a version message with
  Version=60000 and fRelay=1, whereas (before this commit) our testing
  framework would deserialize this into a version message with
  Version=60000 and fRelay=0.

ACKs for top commit:
  jnewbery:
    utACK 39a9ec579f

Tree-SHA512: 13a23f1180b7121ba41cb85baa38094b41f4607a7c88b3384775177cb116e76faf5514760624f98a4e8a830767407c46753a7e0285158c33e0c6ce395de8f15c
2021-03-24 19:28:24 +01:00
Wladimir J. van der Laan
23b15601df Merge #17227: Qt: Add Android packaging support
246774e264 depends: fix Qt precompiled headers bug (Igor Cota)
8e7ad4146d depends: disable Qt Vulkan support on Android (Igor Cota)
ba46adaa1a CI: add Android APK build to cirrus (Igor Cota)
7563720e30 CI: add Android APK build script (Igor Cota)
ebfb10cb75 Qt: add Android packaging support (Igor Cota)

Pull request description:

  ![bitcoin-qt](https://user-images.githubusercontent.com/762502/67396157-62f3d000-f5a7-11e9-8a6f-9425823fcd6c.gif)
  This PR is the third and final piece of the basic Android support puzzle - it depends on https://github.com/bitcoin/bitcoin/pull/16110 and is related to https://github.com/bitcoin/bitcoin/pull/16883. It introduces an `android` directory under `qt` and a simple way to build an Android package of `bitcoin-qt`:

  1. Build depends for Android as described in the [README](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md)
  2. Configure with one of the resulting prefixes
  3. Run `make && make apk` in `src/qt`

  The resulting APK files will be in `android/build/outputs/apk`. You can install them manually or with [adb](https://developer.android.com/studio/command-line/adb). One can also open the `android` directory in Android Studio for that integrated development and debugging experience. `BitcoinQtActivity` is your starting point.

  Under the hood makefile `apk` target:

  1. Renames the `bitcoin-qt` binary to `libbitcoin-qt.so` and copies it over to a folder under `android/libs` depending on which prefix and corresponding [ABI](https://developer.android.com/ndk/guides/abis.html#sa) `bitcoin-qt` was built for
  2. Takes `libc++_shared.so` from the Android NDK and puts in the same place. It [must be included](https://developer.android.com/ndk/guides/cpp-support) in the APK
  3. Extracts Qt for Android Java support files from the `qtbase` archive in `depends/sources` to `android/src`

  There is also just a tiny bit of `ifdef`'d code to make the Qt Widgets menus usable. It's not pretty but it works and is a stepping stone towards https://github.com/bitcoin/bitcoin/pull/16883.

ACKs for top commit:
  MarcoFalke:
    cr ACK 246774e264
  laanwj:
    Code review ACK 246774e264

Tree-SHA512: ba30a746576a167545223c35a51ae60bb0838818779fc152c210f5af1413961b2a6ab6af520ff92cbc8dcd5dcb663e81ca960f021218430c1f76397ed4cead6c
2021-03-24 19:02:01 +01:00
MarcoFalke
b1281b5d8f Merge #21516: remove unnecessary newline from initWarning() argument
804ac10631 remove unnecessary newline from initWarning() argument (Larry Ruane)

Pull request description:

  Run: `src/bitcoind -wallet=nosuchfile`

  Without this patch, `debug.log` contains:
  ```
  2021-03-23T21:19:16Z init message: Verifying wallet(s)...
  2021-03-23T21:19:16Z Warning: Skipping -wallet path that doesn't exist. Failed to load database path '/home/larry/.bitcoin/wallets/nosuchfile'. Path does not exist.

  2021-03-23T21:19:16Z init message: Loading banlist...
  ```
  With this patch, the empty line isn't present. This PR fixes a similar problem with `src/bitcoind -conf=nosuchfile`

ACKs for top commit:
  practicalswift:
    cr ACK 804ac10631: patch looks correct!
  jarolrod:
    tACK 804ac10631, nice catch!
  theStack:
    Code-review ACK 804ac10631

Tree-SHA512: dfcbaaa72ca24ac40233ac56840cfba8827853711d3df6e229ce940686f2ebf8bf0560bafcaa73a4d82d179a5050af0d3cabdc47b3b1dfd6aaadf718a6635f11
2021-03-24 18:50:51 +01:00
Larry Ruane
804ac10631 remove unnecessary newline from initWarning() argument 2021-03-24 04:31:48 -06:00
fanquake
f67b5dca57 Merge #20734: build: Make platform-specific targets available for proper platform builds only
3d31abbaaa build: Make Windows-specific targets available for Windows builds only (Hennadii Stepanov)
92990e25b7 build: Make macOS-specific targets available for macOS builds only (Hennadii Stepanov)

Pull request description:

  On master (f1dbf92ff0) it is possible to point `make` to macOS and Windows specific targets even the build system is configured to build for Linux platforms:
  ```
  $ make Bitcoin-Core.dmg
  ...
  $ make bitcoin-21.99.0-win64-setup
  ...
  ```

  Such behavior makes no sense, and it is confused. Fixed in this PR.

ACKs for top commit:
  fanquake:
    ACK 3d31abbaaa - tested that nonsensical targets are no longer available. i.e

Tree-SHA512: bbd8450bf98fbccb0b828df2f753ed0dbbd203defa2f58ce21390ee2ea183c95d8ff585d62d52be870dbf0158e2bb0fbd47eda026b80174ee6fd617473f5ac03
2021-03-24 16:52:17 +08:00
MarcoFalke
7ce7ce5716 Merge #21338: test: add functional test for anchors.dat
581791c620 test: add functional test for anchors.dat (bruno)

Pull request description:

  This PR adds a functional test for anchors.dat.

  It creates a node and adds 2 outbound block-relay-only connections and 5 inbound connections.
  When the node is down, anchors.dat should contain the 2 addresses from the outbound block-relay-only connections.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 581791c620
  hebasto:
    ACK 581791c620

Tree-SHA512: 77038b09e36ee5ae473a26d6f566c0ed283af258c34df8486706a24f72b05abab621a293ac886d03849bc45bc28be7336137252225b25aff393baa6b5238688c
2021-03-24 08:27:03 +01:00
fanquake
f95071a3f5 Merge #21489: fuzz: cleanups for versionbits fuzzer
aa7f418fe3 fuzz: cleanups for versionbits fuzzer (Anthony Towns)

Pull request description:

  Followups for #21380, shouldn't change coverage. Marking as draft to avoid introducing conflicts for the speedy trial PRs.

ACKs for top commit:
  MarcoFalke:
    cr ACK aa7f418fe3
  practicalswift:
    Tested ACK aa7f418fe3

Tree-SHA512: 6792364e3bb036cc903b4a5f5805d00afceeae475ce84660da962d28335bd98e59d5f45e68718657d3aa526123e351edadda39e99e49f1c6cfab629e98df35ed
2021-03-24 14:15:35 +08:00
MarcoFalke
fa818ca202 fuzz: [refactor] Use PickValue where possible 2021-03-24 06:57:55 +01:00
Troy Giorshev
39a9ec579f Unconditionally check for fRelay field in test framework
There is a discrepancy in the implementation of our p2p protocol between
bitcoind and the testing framework.  The fRelay field is an optional
field at the end of a version message as of protocol version 70001.
However, when deserializing a message in bitcoind, we don't check the
version to see if it should have an fRelay field or not.  Instead we
unconditionally attempt to deserialize into the field.

This commit brings the testing framework in line with the implementation
in core.

This matters for a version message with the following fields:

Version = 60000
fRelay = 1

Bitcoind would deserialize this into a version message with
Version=60000 and fRelay=1, whereas (before this commit) our testing
framework would deserialize this into a version message with
Version=60000 and fRelay=0.
2021-03-23 19:57:17 -04:00
Wladimir J. van der Laan
fa2a5b8f3a Merge #20421: build: miniupnpc 2.2.2
180dc3c886 build: miniupnpc 2.2.2 (fanquake)

Pull request description:

  Creating the dll subdir is no-longer required.
  We can drop our wingen patch.

  One issue that came up in [#19867](https://github.com/bitcoin/bitcoin/pull/19867#discussion_r483516359):
  > unrelated to this change but: why are we inserting the architecture in here, seems like something not necessary to reveal

  > My assumption is that it was being inserted to make depends more deterministic. However I think we can improve this, as there's no reason to reveal more of the version information either. Could leave the version as is /2.0 and either drop the architecture, or insert something else?

  I've dropped our `sed` and added a patch that just removes the OS string and miniupnpc version from the User-Agent. i.e:
  ```bash
  # master
  strings depends/x86_64-apple-darwin19.6.0/lib/libminiupnpc.a | rg -i User-Agent
  User-Agent: x86_64-apple-darwin19.6.0, UPnP/1.1, MiniUPnPc/2.0.20180203
  User-Agent: x86_64-apple-darwin19.6.0, UPnP/1.1, MiniUPnPc/2.0.20180203

  # this PR
  strings depends/x86_64-apple-darwin19.6.0/lib/libminiupnpc.a | rg -i User-Agent
  User-Agent: UPnP/1.1
  User-Agent: UPnP/1.1
  ```

  Note that built unmodified (22c1386351), the User-Agent would be:
  ```bash
  strings libminiupnpc.dylib | rg User-Agent
  User-Agent: Darwin/19.6.0, UPnP/1.1, MiniUPnPc/2.2.0
  User-Agent: Darwin/19.6.0, UPnP/1.1, MiniUPnPc/2.2.0
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 180dc3c886
  hebasto:
    ACK 180dc3c886.

Tree-SHA512: b0b6e623dbc5499e28faedf992d84278d6a11887a45a3806957b9e08886c5e56044cdfa2e7d7ec81cb1dd55f89be99834367905315d6bc611ba530e91d889ad1
2021-03-23 21:06:41 +01:00
Fabian Jahr
4f2653a890 test: Use deterministic chain in utxo set hash test
Also melts the previously separated test cases into one.
2021-03-23 20:47:24 +01:00
Fabian Jahr
4973c5175c test: Remove wallet dependency of utxo set hash test 2021-03-23 20:32:50 +01:00
Fabian Jahr
1a27af1d7b rpc: Improve gettxoutsetinfo help 2021-03-23 20:32:47 +01:00
Martin Zumsande
18a9b27dd6 p2p: Don't send FEEFILTER in blocksonly mode
It is unnecessary to send FEEFILTER messages when we don't accept
transactions from our peers.
2021-03-23 18:57:59 +01:00
Jarol Rodriguez
915e34112b qt: fix issue when disabling the auto-enabled blank wallet checkbox
This commit makes it so that when the `Blank Wallet` checkbox is auto-selected after the user selects 'Disable Private' keys, unselecting it will also unselect the 'Disable Private Keys' checkbox, which in turn re-enables the 'Encrypt Wallet' checkbox.
2021-03-23 13:13:30 -04:00
Larry Ruane
51eef4a03e doc: Add release notes for #18335 (rpc work queue exceeded error) 2021-03-23 10:14:13 -06:00
MarcoFalke
681c21be9a Merge #21512: fuzz: Fix tx_pool target to properly fuzz immature outpoints
fac921f23f fuzz: Fix tx_pool target to properly fuzz immature outpoints (MarcoFalke)
fa2b95f861 fuzz: Style fixups (MarcoFalke)

Pull request description:

  Also includes a commit for minor style fixups

ACKs for top commit:
  glozow:
    utACK fac921f23f this fixes it 👍

Tree-SHA512: 1575ba115b2009b653921511c163bd846cd381d6fc92b04a899c0686d23a02bdcdd95c81776b515b80ae187bcec3ccaca3aa88fcecbec888f73ca2d875eef506
2021-03-23 16:04:31 +01:00
MarcoFalke
837e59eff6 Merge bitcoin-core/gui#248: Fix: For values of "Bytes transferred" and "Bytes/s" with 1000-based prefix names use 1000-based divisor instead of 1024-based
d09ebc4723  Fix wrong(1024) divisor for 1000-based prefixes (wodry)

Pull request description:

  v.0.21.0

  I saw in the GUI peer window in the "received" column `1007 KB`, and after increasing to >=1024 I guess, it switched to `1 MB`. I would have expected the display unit to change from KB to MB already at value >=1000.

  I looked into the code, and the values appear to be power-of-2 byte values, so the switching at >=1024 and not >=1000 seems correct.
  But the unit display is not precisely correct, binary prefixes should be used for power-of-2 byte values.

  To be correct, this PR changes ~~KB/MB/GB to KiB/MiB/GiB.~~ KB to kB and the divisor from 1024 to 1000.

ACKs for top commit:
  hebasto:
    ACK d09ebc4723, tested on Linux Mint 20.1 (Qt 5.12.8) the both "Network Traffic" and "Peers" tabs of the "Node Window".
  jarolrod:
    ACK d09ebc4723
  leonardojobim:
    Tested ACK d09ebc4723 on Ubuntu 20.04 Qt 5.12.8

Tree-SHA512: 8f830b08cc3fd36dc8a18f1192959fe55d1644938044bf31d770f7c3bf8475fba6da5019a2d2024d5b2c81a8dab112f360c555367814a14f4d05c89d130f25b0
2021-03-23 15:57:58 +01:00
Michael Dietz
90ae3d8ca6 doc: Add release notes for -deprecatedrpc=addresses and bitcoin-tx 2021-03-23 10:51:46 -04:00
Michael Dietz
085b3a7299 rpc: deprecate addresses and reqSigs from rpc outputs
1) add a new sane "address" field (for outputs that have an
   identifiable address, which doesn't include bare multisig)
2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact
   (with all weird/wrong behavior they have now)
3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely,
   always.
2021-03-23 10:51:43 -04:00
Hennadii Stepanov
257f55c119 qt, refactor: Drop redundant setEditTriggers(NoEditTriggers) calls
The models of the both views have no Qt::ItemIsEditable flag.
2021-03-23 16:04:42 +02:00
fanquake
3c87dbe95c Merge #21497: build: Do not opt-in unused CoreWLAN stuff in depends for macOS
810b1310d6 build: Do not build unused CoreWLAN stuff in depends for macOS (Hennadii Stepanov)

Pull request description:

  We [do not use](d2a78ee928/contrib/devtools/symbol-check.py (L96-L111)) any macOS CoreWLAN Framework stuff.

  Changes in Qt Configure summary with `HOST=x86_64-apple-darwin18`:
  ```diff
  --- wlan-master/summary2021-03-22 00:26:04.377387806 +0200
  +++ wlan-pr/summary2021-03-22 00:37:07.060997990 +0200
  @@ -49,7 +49,7 @@
       slog2 ................................ no
     Using system PCRE2 ..................... no
   Qt Network:
  -  CoreWLan ............................... yes
  +  CoreWLan ............................... no
     getifaddrs() ........................... yes
     IPv6 ifname ............................ yes
     libproxy ............................... no
  ```

ACKs for top commit:
  fanquake:
    ACK 810b1310d6 - Only obvious difference I could see in the build is skipping a configure check.

Tree-SHA512: 54e177c4ad528ef48cc80c3a39ab1b66267dd0ca4fe6cc4f70579c87b74051a04ebeeca1a26afee7fc29f750af456804578abbe7e8a9ad2717297291f206547e
2021-03-23 19:59:09 +08:00
MarcoFalke
55ceaeb8c4 Merge #18030: doc: Coin::IsSpent() can also mean never existed
1404c57403 [doc] Coin: explain that IsSpent() can also mean never existed (Sjors Provoost)

Pull request description:

  This can be especially confusing where `AccessCoin()` is used with logic like this:

  ```c++
      while (iter.n < MAX_OUTPUTS_PER_BLOCK) {
          const Coin& alternate = view.AccessCoin(iter);
          if (!alternate.IsSpent()) return alternate;
  ```

ACKs for top commit:
  practicalswift:
    ACK 1404c57403
  MarcoFalke:
    ACK 1404c57403
  jnewbery:
    utACK 1404c57403

Tree-SHA512: 418618dd7e08bd5cc8360e3501d0f57e34100e5101ad3b8e0a819923fa860f44c7f2fada0f8447a1af3c2601fd72bfe619b91ff2f26f7133ceaeb0c98b017b12
2021-03-23 11:11:02 +01:00
MarcoFalke
fac921f23f fuzz: Fix tx_pool target to properly fuzz immature outpoints 2021-03-23 10:58:36 +01:00
MarcoFalke
fa2b95f861 fuzz: Style fixups 2021-03-23 10:58:32 +01:00
MarcoFalke
fd2b22bf24 Merge #21142: fuzz: Add tx_pool fuzz target
faa9ef49d1 fuzz: Add tx_pool fuzz targets (MarcoFalke)

Pull request description:

ACKs for top commit:
  AnthonyRonning:
    reACK faa9ef49d1
  practicalswift:
    Tested ACK faa9ef49d1
  glozow:
    code review ACK faa9ef49d1, a bunch of comments but non blocking

Tree-SHA512: 8d404398faa46d8e7bf93060a2fe9afd5c0c2bd6e549ff6588d2f3dd1b912dff6c5416d5477c18edecc2e85b00db4fdf4790c3e6597a5149b0d40c9d5014d82f
2021-03-23 09:59:40 +01:00
MarcoFalke
d400e672a0 Merge #21487: fuzz: Use ConsumeWeakEnum in addrman for service flags
55554463c1 fuzz: Use ConsumeWeakEnum in addrman for service flags (MarcoFalke)

Pull request description:

  This has minimally better performance. Reported by me in https://github.com/bitcoin/bitcoin/pull/20228#discussion_r598081787

ACKs for top commit:
  practicalswift:
    Tested ACK 55554463c1
  vasild:
    ACK 55554463c1

Tree-SHA512: 4e5f51fe4f2bd7b2f37d0690e41203341ba45c0c9bc9247449cd26cfb5f77dc2ec61df3e4963276f68694e4b3ca3d0a76367a51c4d775501edeb3224d305a261
2021-03-23 09:43:15 +01:00
fanquake
1c3a8579c0 Merge #21421: build: don't try and use -fstack-clash-protection on Windows
7b3434f800 build: don't try and use -fstack-clash-protection on Windows (fanquake)

Pull request description:

  This has never worked with any of the mingw-w64 compilers we use, and the `-O0` is causing issues for builders applying spectre mitigations (see [IRC logs](http://www.erisian.com.au/bitcoin-core-dev/log-2021-03-12.html#l-15)).

  Recent discussion on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458 would also indicate that this should just not be used on Windows.

ACKs for top commit:
  laanwj:
    Concept and code review ACK (but untested) 7b3434f800
  hebasto:
    ACK 7b3434f800, I've verified that this change does not affect builds for `HOST=x86_64-w64-mingw32` by comparing sizes of the output `*.exe` files.

Tree-SHA512: 72b582321ddff8db3201460fa42a53304e70f141ae078d76a4d4eeb1ca27c8dd567ccb468cc8616179c8df784bd8ca038dcb9a277f9e29f9d98c3cc842916b18
2021-03-23 12:03:10 +08:00
fanquake
180dc3c886 build: miniupnpc 2.2.2
Creating the dll subdir is no-longer required.
We can also drop our wingen patch.
2021-03-23 08:39:16 +08:00
Hennadii Stepanov
0eabb2abed build: Remove unused header from the build system 2021-03-22 19:10:49 +02:00
Wladimir J. van der Laan
c46f1ce751 Merge #21418: contrib: Make systemd invoke dependencies only when ready
663f6cd9dd contrib: Use -daemonwait in systemd init script (Wladimir J. van der Laan)

Pull request description:

  Make systemd invoke dependencies only when ready by using `-daemonwait` in the service file instead of `-daemon`.

  Closes #21322 by making bitcoind conform to behavior specified for `type=forking`.

  This may need some tuning of timeouts.

ACKs for top commit:
  darosior:
    ACK 663f6cd
  hebasto:
    re-ACK 663f6cd9dd

Tree-SHA512: 890005852b632a202caa578e6c796ebdc9da0b2379a9157a4f56f7db9d193c0ffbb78d120bbf112ab2f273855f2a08c3da000b1f7a9fb5222a3b94dcdb16b878
2021-03-22 15:15:14 +01:00
Wladimir J. van der Laan
663f6cd9dd contrib: Use -daemonwait in systemd init script
Closes #21322 by making bitcoind conform to behavior specified for
`type=forking`.
2021-03-22 13:42:13 +01:00
John Newbery
3fc06d3d7b [net] remove fUpdateConnectionTime from FinalizeNode
PeerManager can just call directly into CAddrMan::Connected() now.
2021-03-22 10:25:39 +00:00
MarcoFalke
1e4a3c057a Merge #21317: util: Make Assume() usable as unary expression
fa4cebadcf util: Make Assume() usable as unary expression (MarcoFalke)

Pull request description:

  Assume shouldn't behave different at the call site depending on build flags. Currently compilation fails if it is used as expression. Fix that by using the lambda approach from `Assert()` without the `assert()`.

ACKs for top commit:
  jnewbery:
    ACK fa4cebadcf
  practicalswift:
    cr ACK fa4cebadcf: patch looks correct and commit hash starts with `fa`

Tree-SHA512: 9ec9ac8d410cdaf5e4e28df571a89e3d23d38e05a7027bb726cae3da6e9314734277e5a218e9e090cc17e10db763da71052c229ad642077ca5824ee42022f3ed
2021-03-22 08:35:21 +01:00
MarcoFalke
786654aa5e Merge #21498: refactor: return std::nullopt instead of {}
5294f0d5a9 refactor: return std::nullopt instead of {} (fanquake)

Pull request description:

  In #21415 [we decided](https://github.com/bitcoin/bitcoin/pull/21415#issuecomment-800236640) to return `std::optional` rather than `{}` for
  uninitialized values. This PR replaces the two remaining usages of `{}`
  with `std::nullopt`.

  As a side-effect, this also quells the spurious GCC 10.2.x warning that
  we've had reported quite a few times. i.e #21318, #21248, #20797.

  ```bash
  txmempool.cpp: In member function ‘CTxMemPool::setEntries CTxMemPool::GetIterSet(const std::set<uint256>&) const’:
  txmempool.cpp:898:13: warning: ‘<anonymous>’ may be used uninitialized in this function [-Wmaybe-uninitialized]
    898 |     return {};
        |             ^
  ```

ACKs for top commit:
  hebasto:
    ACK 5294f0d5a9, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 5b776be79ab26e5a3a5fc2b463b394ea5ce6797ed5558424873fa4ecee2898170eff76d6da9d69394d28f8f98974117fc63b922a3e19c52f5294c83073e79bb0
2021-03-22 06:42:04 +01:00
fanquake
80cb51cc6b Merge #21491: test: remove duplicate assertions in util_tests
7e3444805e test: remove duplicate assertions in util_tests (Jon Atack)

Pull request description:

  as noticed by Kiminuo in https://github.com/bitcoin/bitcoin/pull/21488#discussion_r598247676

ACKs for top commit:
  practicalswift:
    cr ACK 7e3444805e: patch looks correct
  vasild:
    ACK 7e3444805e

Tree-SHA512: ad3d5983ad3a665155d766843dfda7178ced47e82154838331e428ed0828a467c1cf4bf99270aaf191e94156d485fafd0a7d5bc68248c4c1304a00ca5a2a9d2e
2021-03-22 11:43:35 +08:00
fanquake
5294f0d5a9 refactor: return std::nullopt instead of {}
In #21415 we decided to return `std::optional` rather than `{}` for
uninitialized values. This PR repalces the two remaining usages of `{}`
with `std::nullopt`.

As a side-effect, this also quells the spurious GCC 10.2.x warning that
we've had reported quite a few times. i.e #21318, #21248, #20797.

```bash
txmempool.cpp: In member function ‘CTxMemPool::setEntries CTxMemPool::GetIterSet(const std::set<uint256>&) const’:
txmempool.cpp:898:13: warning: ‘<anonymous>’ may be used uninitialized in this function [-Wmaybe-uninitialized]
  898 |     return {};
      |             ^
```
2021-03-22 11:22:06 +08:00
Hennadii Stepanov
810b1310d6 build: Do not build unused CoreWLAN stuff in depends for macOS 2021-03-21 23:45:26 +02:00
Igor Cota
246774e264 depends: fix Qt precompiled headers bug
Fixed in 5.14, see QTBUG-85214
2021-03-21 22:33:35 +01:00
Igor Cota
8e7ad4146d depends: disable Qt Vulkan support on Android 2021-03-21 22:33:35 +01:00
Igor Cota
ba46adaa1a CI: add Android APK build to cirrus 2021-03-21 22:33:35 +01:00
Igor Cota
7563720e30 CI: add Android APK build script 2021-03-21 22:33:35 +01:00
Igor Cota
ebfb10cb75 Qt: add Android packaging support
Introduce an android directory under qt and allow one to package bitcoin-qt for Android by running make apk.
Add bitcoin-qt Android build instructions.
2021-03-21 22:33:27 +01:00
Jon Atack
7e3444805e test: remove duplicate assertions in util_tests
Co-authored-by: Kiminuo <kiminuo@protonmail.com>
2021-03-21 11:14:35 +01:00
MarcoFalke
d2a78ee928 Merge #21488: test: add ParseUInt16() unit test and fuzz coverage
3d086f42ab test: add ParseUInt16() test coverage (Jon Atack)

Pull request description:

  `ParseUInt16()` was just added in #21328.

ACKs for top commit:
  practicalswift:
    cr ACK 3d086f42ab: patch looks correct & more coverage is better than less coverage

Tree-SHA512: bf7f96deb7c1531419565907f0ea8a8e32b368d4b823a3e80928b2c118edbf643ea06e357b4b5504a89f855caeed289daa9f823c740231ed6ad1b8ed00285ce8
2021-03-21 08:12:56 +01:00
MarcoFalke
9dbec05600 Merge #21349: build: Fix fuzz-cuckoocache cross-compiling with DEBUG=1
52a43b0c7d build: Fix fuzz-cuckoocache cross-compiling for Windows with DEBUG=1 (Hennadii Stepanov)

Pull request description:

  Fix #21348.

ACKs for top commit:
  practicalswift:
    Tested ACK 52a43b0c7d

Tree-SHA512: 6592f829edfb740a1e9d0691acf04b2372e91b0a53ca395b08350cb0b80031d3b55fa7331bdaddf857d450eb30b605af9fe8fe02559cda19374a48f9634fae70
2021-03-21 08:05:00 +01:00
MarcoFalke
4132193617 Merge #21040: wallet: Fix already-loading message grammar
ae9d26a8f0 wallet: Fix already-loading error message grammar (Fotis Koutoupas)

Pull request description:

ACKs for top commit:
  practicalswift:
    cr ACK ae9d26a8f0
  prayank23:
    ACK ae9d26a8f0

Tree-SHA512: 2f58d309dd33954f47e3ed339887b11bfdad4519f89ed3dd9bf3fb0db246d78b89653d553d02350ec84ce68bbb904db9fac00a2aad8d37f48df694d6782f35df
2021-03-21 07:51:11 +01:00
Anthony Towns
aa7f418fe3 fuzz: cleanups for versionbits fuzzer 2021-03-21 11:21:41 +10:00
MarcoFalke
63952f73b3 Merge #20921: validation: don't try to invalidate genesis block in CChainState::InvalidateBlock
787df19b09 validation: don't try to invalidate genesis block (Sebastian Falbesoner)

Pull request description:

  In the block invalidation method (`CChainState::InvalidateBlock`), the code for creating the candidate block map assumes that the passed block's previous block (`pindex->pprev`) is available and otherwise segfaults due to null-pointer deference in `CBlockIndexWorkComparator()` (see analysis by practicalswift in #20914), i.e. it doesn't work with the genesis block. Rather than analyzing all possible code paths and implications for this corner case, simply fail early if the genesis block is passed.

  Fixes #20914.

ACKs for top commit:
  sipa:
    ACK 787df19b09. Tested invalidation of generic on regtest.
  practicalswift:
    Tested ACK 787df19b09

Tree-SHA512: 978be7cf2bd1c1faebfe945d191ac77dea72791bea826459abd308f77c74c5991efee495a38817c306e488ecd5208b5c888df7d9d044132dd9a06bbbdb256b6c
2021-03-20 12:46:11 +01:00
MarcoFalke
55554463c1 fuzz: Use ConsumeWeakEnum in addrman for service flags 2021-03-20 12:03:12 +01:00
John Newbery
7c4cc67c0c [net] remove CConnman::AddNewAddresses
It just forwards calls to CAddrMan::Add.
2021-03-20 10:24:40 +00:00
John Newbery
bcd7f30b79 [net] remove CConnman::MarkAddressGood
It just forwards calls to CAddrMan::Good.
2021-03-20 10:24:40 +00:00
John Newbery
8073673dbc [net] remove CConnman::SetServices
It just forwards calls to CAddrMan::SetServices.
2021-03-20 10:24:40 +00:00
John Newbery
392a95d393 [net_processing] Keep addrman reference in PeerManager 2021-03-20 10:24:40 +00:00
John Newbery
1c25adf6d2 [net] Construct addrman outside connman
node.context owns the CAddrMan. CConnman holds a reference to
the CAddrMan.
2021-03-20 10:24:36 +00:00
bruno
581791c620 test: add functional test for anchors.dat 2021-03-19 20:30:52 -03:00
Jon Atack
3d086f42ab test: add ParseUInt16() test coverage 2021-03-19 23:50:36 +01:00
practicalswift
732c7bddeb tests: Add test for CNetAddr::ToString IPv6 address formatting (RFC 5952) 2021-03-19 21:35:56 +00:00
MarcoFalke
3530d5d2d8 Merge #18335: bitcoin-cli: print useful error if bitcoind rpc work queue exceeded
8dd5946c0b add functional test (Larry Ruane)
b5a80fa7e4 util: Handle HTTP_SERVICE_UNAVAILABLE in bitcoin-cli (Hennadii Stepanov)

Pull request description:

  If `bitcoind` is processing 16 RPC requests, attempting to submit another request using `bitcoin-cli` produces this less-than-helpful error message: `error: couldn't parse reply from server`. This PR changes the error to: `error: server response: Work queue depth exceeded`.

ACKs for top commit:
  fjahr:
    tACK 8dd5946c0b
  luke-jr:
    utACK 8dd5946c0b (no changes since previous utACK)
  hebasto:
    re-ACK 8dd5946c0b, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/18335#pullrequestreview-460621350) review.
  darosior:
    ACK 8dd5946c0b

Tree-SHA512: 33e25f6ff05d9b56fae2bdb68b132557bb8e995f5438ac4fbbc53c304c5152a98aa43c43600c31d8a6a2830cbd48bf8ec7d89dce50190b29ec00a43830126913
2021-03-19 20:52:16 +01:00
MarcoFalke
18cd0888ef Merge #21328: net, refactor: pass uint16 CService::port as uint16
52dd40a9fe test: add missing netaddress include headers (Jon Atack)
6f09c0f6b5 util: add missing braces and apply clang format to SplitHostPort() (Jon Atack)
2875a764f7 util: add ParseUInt16(), use it in SplitHostPort() (Jon Atack)
6423c8175f p2p, refactor: pass and use uint16_t CService::port as uint16_t (Jon Atack)

Pull request description:

  As noticed during review today in https://github.com/bitcoin/bitcoin/pull/20685#discussion_r584873708 of the upcoming I2P network support, `CService::port` is `uint16_t` but is passed around the codebase and into the ctors as `int`, which causes uneeded conversions and casts. We can avoid these (including in the incoming I2P code without further changes to it) by using ports with the correct type. The remaining conversions are pushed out to the user input boundaries where they can be range-checked and raise with user feedback in the next patch.

ACKs for top commit:
  practicalswift:
    cr ACK 52dd40a9fe: patch looks correct
  MarcoFalke:
    cr ACK 52dd40a9fe
  vasild:
    ACK 52dd40a9fe

Tree-SHA512: 203c1cab3189a206c55ecada77b9548b810281cdc533252b8e3330ae0606b467731c75f730ce9deb07cbaab66facf97e1ffd2051084ff9077cba6750366b0432
2021-03-19 20:47:10 +01:00
Jon Atack
0cca08a8ee Add unit test coverage for our onion peer eviction protection 2021-03-19 20:13:11 +01:00
Jon Atack
caa21f586f Protect onion+localhost peers in ProtectEvictionCandidatesByRatio()
Now that we have a reliable way to detect inbound onion peers, this commit
updates our existing eviction protection of 1/4 localhost peers to instead
protect up to 1/4 onion peers (connected via our tor control service), sorted by
longest uptime. Any remaining slots of the 1/4 are then allocated to protect
localhost peers, or 2 localhost peers if no slots remain and 2 or more onion
peers are protected, sorted by longest uptime.

The goal is to avoid penalizing onion peers, due to their higher min ping times
relative to IPv4 and IPv6 peers, and improve our diversity of peer connections.

Thank you to Gregory Maxwell, Suhas Daftuar, Vasil Dimov and Pieter Wuille
for valuable review feedback that shaped the direction.
2021-03-19 20:13:04 +01:00
Jon Atack
8f1a53eb02 Use EraseLastKElements() throughout SelectNodeToEvict()
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-03-19 20:11:47 +01:00
Jon Atack
8b1e156143 Add m_inbound_onion to AttemptToEvictConnection()
and an `m_is_onion` struct member to NodeEvictionCandidate and tests.

We'll use these in the peer eviction logic to protect inbound onion peers
in addition to the existing protection of localhost peers.
2021-03-19 20:11:45 +01:00
Jon Atack
72e30e8e03 Add unit tests for ProtectEvictionCandidatesByRatio()
Thank you to Vasil Dimov (vasild) for the suggestion to use std::unordered_set
rather than std::vector for the IsProtected() peer id arguments.
2021-03-19 20:11:43 +01:00
Jon Atack
ca63b53ecd Use std::unordered_set instead of std::vector in IsEvicted()
An unordered set can tell if an element is present in ~O(1) time (constant on
average, worst case linear to the size of the container), which speeds up and
simplifies the lookup in IsEvicted().

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-03-19 20:11:41 +01:00
Jon Atack
41f84d5ecc Move peer eviction tests to a separate test file
out of net_tests, because the eviction tests:

- are a different domain of test coverage, with different dependencies

- run more slowly than the net tests

- will be growing in size, in this PR branch and in the future, as eviction
  test coverage is improved
2021-03-19 20:11:39 +01:00
Jon Atack
f126cbd6de Extract ProtectEvictionCandidatesByRatio from SelectNodeToEvict
to allow deterministic unit testing of the ratio-based peer eviction protection
logic, which protects peers having longer connection times and those connected
via higher-latency networks.

Add documentation.
2021-03-19 20:11:29 +01:00
MarcoFalke
7f3fd34117 Merge #21481: doc: Tell howto install clang-format on Debian/Ubuntu
ea76f4ac7d Doc: Tell howto install clang-format on Debian/Ubuntu (wodry)

Pull request description:

  Because only macOS wasy mentioned, I was unsure if this would be a macOS specific tool. I guess Linux is more used than Mac, so Linux guide should be there, too.

ACKs for top commit:
  hebasto:
    ACK ea76f4ac7d, every system upgrade via clean installation I do the same.

Tree-SHA512: 75c28540e8815cb41f4cf92784b6349978988b679e4deef9ae77ede951f93516ca13ec7b313ab72865b01273e115b49ed2b67cdcd68015af1b643a6186b190dd
2021-03-19 20:00:48 +01:00
wodry
d09ebc4723 Fix wrong(1024) divisor for 1000-based prefixes 2021-03-19 19:35:42 +01:00
wodry
ea76f4ac7d Doc: Tell howto install clang-format on Debian/Ubuntu
Because only macOS wasy mentioned, I was unsure if this would be a macOS specific tool. I guess Linux is more used than Mac, so Linux guide should be there, too.
2021-03-19 19:24:40 +01:00
MarcoFalke
3a12fdba51 Merge #21235: p2p: Clarify disconnect log message in ProcessGetBlockData, remove send bool
fa81773243 style-only: Remove whitespace (MarcoFalke)
fae77b9e6d net: Simplify ProcessGetBlockData execution by removing send flag. (Patrick Strateman)
fae7c0429f log: Clarify that block request below NODE_NETWORK_LIMITED_MIN_BLOCKS disconnects (MarcoFalke)

Pull request description:

  * Clarify that "ignoring" really means "disconnect" in the log
  * Revive a refactor I took from #13670

ACKs for top commit:
  jnewbery:
    utACK fa81773243
  sipa:
    utACK fa81773243

Tree-SHA512: 0a4fcb979cb82c4e26012881eeaf903c38dfbb85d461476c01e35294760744746a79c48ffad827fe31c1b830f40c6e4240529c71e375146e4d0313c3b7d784ca
2021-03-19 18:56:34 +01:00
Hennadii Stepanov
3d31abbaaa build: Make Windows-specific targets available for Windows builds only 2021-03-18 20:58:13 +02:00
Hennadii Stepanov
92990e25b7 build: Make macOS-specific targets available for macOS builds only 2021-03-18 20:58:04 +02:00
MarcoFalke
faa9ef49d1 fuzz: Add tx_pool fuzz targets 2021-03-18 18:43:52 +01:00
fanquake
a65e772fec Merge #21428: test: Cleanup in test-{security,symbol}-check.py
0fc0c00f7a test: Drop unused get_machine function (Hennadii Stepanov)
61a0f8f9cc test: Cleanup test files in test-{security,symbol}-check.py (Hennadii Stepanov)

Pull request description:

  1) Test source and executable files are neither ignored by `.gitignore` nor removed by `make clean` and `make distclean`.

  2) The `get_machine` function is no longer used since #21255.

ACKs for top commit:
  fanquake:
    ACK 0fc0c00f7a

Tree-SHA512: ef3fcf22d4a04b6e4f37f748bd4be57e09696d2a77982e26292843cb2a1297789c8325f5c4bdad37d8094fce7765c4cc9ab19809e07471487943361b2b1a252c
2021-03-18 17:45:18 +08:00
MarcoFalke
fa81773243 style-only: Remove whitespace
Can be reviewed with --ignore-all-space
2021-03-18 09:16:11 +01:00
Patrick Strateman
fae77b9e6d net: Simplify ProcessGetBlockData execution by removing send flag.
Setting the send flag to false can be replaced by simply returning.
2021-03-18 09:15:16 +01:00
MarcoFalke
fae7c0429f log: Clarify that block request below NODE_NETWORK_LIMITED_MIN_BLOCKS disconnects 2021-03-18 09:12:37 +01:00
MarcoFalke
6834e02c89 Merge #21425: refactor: Pass PeerManagerImpl members only once
fa2a80bf12 refactor: Pass PeerManagerImpl members only once (MarcoFalke)

Pull request description:

  Member variables are already passed to methods via `this`, so no need to pass them another time as function parameter.

ACKs for top commit:
  jnewbery:
    utACK fa2a80bf12
  amitiuttarwar:
    utACK fa2a80bf12

Tree-SHA512: 1743825c7560cc748235e3db03e4cea02ad1f670f1b898d7757da644f12693ba9bb2d3eb09b64b3d10dd2e68f52dea31e26d5e97bdc013759baa0515d3c7055c
2021-03-18 08:58:46 +01:00
fanquake
e057e01b7b Merge #21162: Net Processing: Move RelayTransaction() into PeerManager
680eb56d82 [net processing] Don't pass CConnman to RelayTransactions (John Newbery)
a38a4e8f03 [net processing] Move RelayTransaction into PeerManager (John Newbery)

Pull request description:

  This is the first part of #21160. It moves the RelayTransaction() function to be a member function of the PeerManager class. This is required in order to move the transaction inventory data into the Peer object, since Peer objects are only accessible from within PeerManager.

ACKs for top commit:
  ajtowns:
    ACK 680eb56d82

Tree-SHA512: 8c93491a4392b6369bb7f090de326a63cd62a088de59026e202f226f64ded50a0cf1a95ed703328860f02a9d2f64d3a87ca1bca9a6075b978bd111d384766235
2021-03-18 14:57:50 +08:00
fanquake
bf7c22f7ff Merge #21435: doc: Update dependencies.md
bb3f79fbdc doc: Update libnatpmp info in dependencies.md (Hennadii Stepanov)
1a01a5dc8d doc: Update zlib info in dependencies.md (Hennadii Stepanov)

Pull request description:

  Update docs according to the recent changes in the code:
  - #21209 (zlib)
  - #21376 (libnatpmp)

ACKs for top commit:
  fanquake:
    ACK bb3f79fbdc - thanks for keeping this updated.

Tree-SHA512: 48350ad07700aa071ad6c34e4c161aaadc050488fc068cf478e9781d632828187962a4384c1b67c2344145a2c00c3e16cddd09259130af8e9e86cd76cd32900d
2021-03-18 11:40:45 +08:00
fanquake
d6e3ac89d4 Merge #21343: doc: revamp macOS build doc
c180c911b8 doc: revamp macOS build doc (Jarol Rodriguez)

Pull request description:

  This PR makes the macOS build-docs more informative and adds in the following information:
  - Proper descriptions and delineation of required/optional dependencies
  - walk-through of optional dependencies
  - configuration walk-through
  - various other tidbits of information

  This is a part of the efforts done in https://github.com/bitcoin/bitcoin/pull/20601 and https://github.com/bitcoin/bitcoin/pull/20610 to update the docs and introduce some consistency between them.

  This update does not add instructions for arm-based M1 Macbooks as I do not have one to test with. It would be nice to have someone follow up with an update containing instructions for arm-based Macs.

  **Before/Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-osx.md)
  **After/PR:** [render](c180c911b8/doc/build-osx.md)

ACKs for top commit:
  fanquake:
    ACK c180c911b8 - I still think these are getting too verbose and we're duplicating information all over the place; dependencies, configure options, combinations of options etc. However if people are happy to maintain them, I guess it's fine for now, and this revamping has already happened for some of the other build READMEs.

Tree-SHA512: 1440046c723fe80d4158e4a429e3aa8bd93570acb84ad202d5d24c749ab9a89a3aca8b61b49e75e042a4bf4317acd632d3906e1b5808a9052e74209256528b45
2021-03-18 11:36:48 +08:00
Hennadii Stepanov
ef3e1d7272 qt: Improve URI/file handling message
This change:
- fixes missing spaces after full stops
- makes translation context bigger
2021-03-17 21:38:00 +02:00
Wladimir J. van der Laan
a9d1b40d53 Merge #21415: refactor: remove Optional & nullopt
ebc4ab721b refactor: post Optional<> removal cleanups (fanquake)
57e980d13c scripted-diff: remove Optional & nullopt (fanquake)

Pull request description:

  Same rationale & motivation as #21404, which turned out to be quite low in the number of potential conflicts. Lets see what the bot has to say here.

ACKs for top commit:
  practicalswift:
    cr ACK ebc4ab721b: patch looks correct
  jnewbery:
    utACK ebc4ab721b
  laanwj:
    Code review ACK ebc4ab721b

Tree-SHA512: 550fbeef09b9d35ddefaa805d1755c18c8fd499c4b0f77ebfece8c20296a7abd1cf6c699e2261f92fe3552deeb7555ec2a2287ffe3ab9e98bb9f8612a4d43be3
2021-03-17 12:17:33 +01:00
MarcoFalke
fa2a80bf12 refactor: Pass PeerManagerImpl members only once
Can be reviewed with

--word-diff-regex=. --ignore-all-space
2021-03-17 10:35:30 +01:00
fanquake
ebc4ab721b refactor: post Optional<> removal cleanups 2021-03-17 14:56:20 +08:00
fanquake
993ecafa5e Merge #21417: Misc external signer improvement and HWI 2 support
57ff5a42ab doc: specify minimum HWI version (Sjors Provoost)
03308b2bfa rpc: don't require wallet for enumeratesigners (Sjors Provoost)

Pull request description:

  HWI just released 2.0. See https://github.com/bitcoin-core/HWI/releases/tag/2.0.0

  As of #16546 we already rely on features that are in 2.0 and not in the previous 1.* releases:
  * `--chain` param

  This shouldn't be a problem, because HWI 2.0 has been released before we release v22.

  Misc improvements:
  * document that HWI 2.0 is required
  * drop wallet requirement for `enumeratesigners`

ACKs for top commit:
  achow101:
    Code Review ACK 57ff5a42ab

Tree-SHA512: 3fb6ba20894e52a116f89525a5f5a1f61d363ccd904e1cffd0e6d095640fc6d2edf0388cd6ae20f83bbc31e5f458255ec090b6e823798d426eba3e45b4336bf9
2021-03-17 09:54:27 +08:00
Samuel Dobson
d25e28c20b Merge #21083: wallet: Avoid requesting fee rates multiple times during coin selection
f9cd2bfbcc Rename CoinSelectionParams::effective_fee to m_effective_feerate (Andrew Chow)
bdd0c2934b wallet: Move discard feerate fetching to CreateTransaction (Andrew Chow)
448d04b931 wallet: Move long term feerate setting to CreateTransaction (Andrew Chow)
e2f429e6bb wallet: Replace nFeeRateNeeded with effective_fee (Andrew Chow)
1a6a0b0dfb wallet: Use existing feerate instead of getting a new one (Andrew Chow)

Pull request description:

  During coin selection, there are various places where we need to have a feerate. We need the feerate for the transaction itself, the discard fee rate, and long term feerate. Fetching these each time we need them can lead to a race condition where two feerates that should be the same are actually different. One particular instance where this can happen is during the loop in `CreateTransactionInternal`. After inputs are chosen, the expected transaction fee is calculated using a newly fetched feerate. If `pick_new_inputs == false`, the loop will go again with the assumption that the fee for the transaction remains the same. However because the feerate is fetched again, it is possible that it actually isn't and this causes coin selection to fail.

  Instead of fetching the feerate each time it is needed, we fetch them all at once at the top of `CreateTransactionInternal`, store them in `CoinSelectionParams`, and use them where needed.

  While some of these fee rates probably don't need this caching, I've done it for consistency and the guarantee that they remain the same.

  Fixes #19229

ACKs for top commit:
  glozow:
    reACK f9cd2bfbcc
  fjahr:
    Code review re-ACK f9cd2bfbcc
  Xekyo:
    tACK f9cd2bfbcc
  meshcollider:
    Code review + test run ACK f9cd2bfbcc

Tree-SHA512: be83ff64ba473c3cdd3469c812e214659b6e2a9584c22ed2b1595618fce0d4b35d0901e61068cd1069fc1a8fb911db01dd7312d05c3b8cbafbe2504ab7a3e863
2021-03-17 13:14:48 +13:00
Andrew Chow
f9cd2bfbcc Rename CoinSelectionParams::effective_fee to m_effective_feerate
It's a feerate, not a fee. Also follow the style guide for member names.
2021-03-16 17:16:57 -04:00
Andrew Chow
bdd0c2934b wallet: Move discard feerate fetching to CreateTransaction
Instead of fetching the discard feerate for each SelectCoinsMinConf
iteration, fetch and cache it once during CreateTransaction so that it
is shared for each SelectCoinsMinConf through
coin_selection_params.m_discard_feerate.

Does not change behavior.
2021-03-16 16:33:27 -04:00
Andrew Chow
448d04b931 wallet: Move long term feerate setting to CreateTransaction
Instead of setting the long term feerate for each SelectCoinsMinConf
iteration, set it once during CreateTransaction and let it be shared
with each SelectCoinsMinConf through
coin_selection_params.m_long_term_feerate.

Does not change behavior.
2021-03-16 16:32:38 -04:00
Jon Atack
52dd40a9fe test: add missing netaddress include headers 2021-03-16 19:52:37 +01:00
Jon Atack
6f09c0f6b5 util: add missing braces and apply clang format to SplitHostPort() 2021-03-16 19:52:35 +01:00
Jon Atack
2875a764f7 util: add ParseUInt16(), use it in SplitHostPort() 2021-03-16 19:52:33 +01:00
Jon Atack
6423c8175f p2p, refactor: pass and use uint16_t CService::port as uint16_t 2021-03-16 19:52:31 +01:00
MarcoFalke
5ef16038a1 Merge #21410: test: increase rpc_timeout for fundrawtx test_transaction_too_large
d09120b7d1 test: give fundraw more time for test_transaction_too_large (Jon Atack)

Pull request description:

  to hopefully fix timeouts from a new test added in 48a0319bab of #20536 merged March 8, 2021

  seen locally when running via the test runner

  ```
  File "/home/jon/projects/bitcoin/bitcoin/test/functional/rpc_fundrawtransaction.py", line 927, in test_transaction_too_large
  raise JSONRPCException({
      test_framework.authproxy.JSONRPCException: 'generatetoaddress' RPC took longer than 30.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```
  and in the CI like https://bitcoinbuilds.org/index.php?ansilog=28537952-2c92-46f2-9871-8918e5ba2738.log#l2398
  ```
  File "/home/ubuntu/src/test/functional/rpc_fundrawtransaction.py", line 927, in test_transaction_too_large
  test_framework.authproxy.JSONRPCException: 'generatetoaddress' RPC took longer than 240.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```

Top commit has no ACKs.

Tree-SHA512: f11c923439014fe12420f986c640fd301a26282eb41516957d73b9c751087cbae3d0e316f9ccb49bcb424f488540266f70d3f97948633e77c62bd7935df90452
2021-03-16 19:25:31 +01:00
Hennadii Stepanov
bb3f79fbdc doc: Update libnatpmp info in dependencies.md 2021-03-16 19:46:15 +02:00
Andrew Chow
e2f429e6bb wallet: Replace nFeeRateNeeded with effective_fee
Make sure that all fee calculations use the same feerate.
coin_selection_params.effective_fee is the variable we use for all fee
calculations, so get rid of remaining nFeeRateNeeded usages and just
directly set coin_selection_params.effective_fee.

Does not change behavior.
2021-03-16 12:32:56 -04:00
Andrew Chow
1a6a0b0dfb wallet: Use existing feerate instead of getting a new one
During each loop of CreateTransaction, instead of constantly getting a
new feerate, use the feerate that we have already fetched for all
fee calculations. Thix fixes a race condition where the feerate required
changes during each iteration of the loop.

This commit changes behavior as the "Fee estimation failed" error will
now take priority over "Signing transaction failed".
2021-03-16 12:30:01 -04:00
Jon Atack
d09120b7d1 test: give fundraw more time for test_transaction_too_large 2021-03-16 17:18:54 +01:00
Wladimir J. van der Laan
01bb3afb51 Merge #21447: Always add -daemonwait to known command line arguments
4d008f908e Always add -daemonwait to known command line arguments (Hennadii Stepanov)

Pull request description:

  This is a follow up of #21007.

  When `AC_CHECK_DECLS([fork])` fails:

  - on master (8e6532053f):
  ```
  $ src/bitcoind -daemonwait
  Error: Error parsing command line arguments: Invalid parameter -daemonwait

  ```

  - with this PR:
  ```
  $ src/bitcoind -daemonwait
  Error: -daemon is not supported on this operating system

  ```

ACKs for top commit:
  laanwj:
    Code review ACK 4d008f908e

Tree-SHA512: 7fcb5e9d76958adcf57e04fa74bd2a98d62459d81a3c57a97bd74c346cbf47c53e560a15455fb024e912c3b44e8487a83499e993b282871ba069953e665d88a9
2021-03-16 15:10:47 +01:00
Vasil Dimov
40316a37cb test: add I2P test for a runaway SAM proxy
Add a regression test for https://github.com/bitcoin/bitcoin/pull/21407.

The test creates a socket that, upon read, returns some data, but never
the expected terminator `\n`, injects that socket into the I2P code and
expects `i2p::sam::Session::Connect()` to fail, printing a specific
error message to the log.
2021-03-16 14:58:38 +01:00
Vasil Dimov
2d8ac77970 fuzz: add tests for the I2P Session public interface 2021-03-16 14:58:38 +01:00
Vasil Dimov
9947e44de0 i2p: use pointers to Sock to accommodate mocking
Change the types of `i2p::Connection::sock` and
`i2p::sam::Session::m_control_sock` from `Sock` to
`std::unique_ptr<Sock>`.

Using pointers would allow us to sneak `FuzzedSock` instead of `Sock`
and have the methods of the former called.

After this change a test only needs to replace `CreateSock()` with
a function that returns `FuzzedSock`.
2021-03-16 13:59:18 +01:00
Vasil Dimov
82d360b5a8 net: change ConnectSocketDirectly() to take a Sock argument
Change `ConnectSocketDirectly()` to take a `Sock` argument instead of a
bare `SOCKET`. With this, use the `Sock`'s (possibly mocked) methods
`Connect()`, `Wait()` and `GetSockOpt()` instead of calling the OS
functions directly.
2021-03-16 13:58:23 +01:00
Vasil Dimov
b5861100f8 net: add connect() and getsockopt() wrappers to Sock
Extend the `Sock` class with wrappers to `connect()` and `getsockopt()`.

This will make it possible to mock code which uses those.
2021-03-16 13:53:26 +01:00
Vasil Dimov
5a887d49b2 fuzz: avoid FuzzedSock::Recv() repeated errors with EAGAIN
If `recv(2)` returns an error (`-1`) and sets `errno` to a temporary
error like `EAGAIN` a proper application code is expected to retry the
operation.

If the fuzz data is exhausted, then `FuzzedSock::Recv()` will keep
returning `-1` and setting `errno` to the first element of
`recv_errnos[]` which happened to be `EAGAIN`. This may continue forever
or cause the fuzz test to run for a long time before some higher level
application "receive timeout" is triggered.

Thus, put `ECONNREFUSED` as first element of `recv_errnos[]`.
2021-03-16 13:53:26 +01:00
Vasil Dimov
3088f83d01 fuzz: extend FuzzedSock::Recv() to support MSG_PEEK
A conforming `recv(2)` call is supposed to return the same data on a
call following `recv(..., MSG_PEEK)`. Extend `FuzzedSock::Recv()` to do
that.

For simplicity we only return 1 byte when `MSG_PEEK` is used. If we
would return a buffer of N bytes, then we would have to keep track how
many of them were consumed on subsequent non-`MSG_PEEK` calls.
2021-03-16 13:53:25 +01:00
Vasil Dimov
9b05c49ade fuzz: implement unimplemented FuzzedSock methods
We want `Get()` to always return the same value, otherwise it will look
like the `FuzzedSock` implementation itself is broken. So assign
`m_socket` a random number in the `FuzzedSock` constructor.

There is nothing to fuzz about the `Get()` and `Release()` methods, so
use the ones from the base class `Sock`.

`Reset()` is just setting our socket to `INVALID_SOCKET`. We don't want
to use the base `Reset()` because it will close `m_socket` and given
that our `m_socket` is just a random number it may end up closing a real
opened file descriptor if it coincides with our random `m_socket`.
2021-03-16 13:53:25 +01:00
Wladimir J. van der Laan
1b6c463e03 Merge #21407: i2p: limit the size of incoming messages
7059e6d822 test: add a test to ensure RecvUntilTerminator() limit works (Vasil Dimov)
80a5a8ea2b i2p: limit the size of incoming messages (Vasil Dimov)

Pull request description:

  Put a limit on the amount of data `Sock::RecvUntilTerminator()` can read
  if no terminator is received.

  In the case of I2P this avoids a runaway (or malicious) I2P proxy
  sending us tons of data without a terminator before a timeout is
  triggered.

ACKs for top commit:
  laanwj:
    Re-ACK 7059e6d822

Tree-SHA512: 21f3f3468c765c726cdc877eae847cdb4dbe225d94c5bd1849bd752c9761fac881c554f16ea7a685ad40312159d554e126c481e21c5fd83a6d947060b920373d
2021-03-16 13:11:59 +01:00
Wladimir J. van der Laan
7723479300 Merge #220: Do not translate file extensions
88df300f20 qt: Do not translate file extensions (Hennadii Stepanov)

Pull request description:

  File extensions are untranslatable by their nature.

ACKs for top commit:
  laanwj:
    Concept and code review ACK 88df300f20
  Talkless:
    tACK 88df300f20, tested on Debian Sid with Qt 5.15.2. Tested all filters except for .psbt.
  jarolrod:
    re-ACK 88df300f20

Tree-SHA512: 104d383543edcee8fb825f98d3b6669a7aaae2c74b6602f9bc407bf1c88be121ec535f2f9be87afa6ca775dc79865165f620553f6f6ab1d31a3f9ea93f7f9593
2021-03-16 13:07:10 +01:00
Wladimir J. van der Laan
3b408d21e0 Merge #21438: test: add ParseUInt8() test coverage
76782e560b refactor: reuse test string with embedded null char in util_tests (Jon Atack)
24c6546946 test: add ParseUInt8() unit and fuzz test coverage (Jon Atack)

Pull request description:

  We have unit test and fuzzer coverage for
  - `ParseInt64()`
  - `ParseInt32()`
  - `ParseUInt64()`
  - `ParseUInt32()`

  but not `ParseUInt8()`, so this pull adds it.

  I was tempted to add a commit that applies clang formatting to the file, or one that updates the C-style casts to named casts, but resisted the temptation unless reviewers request it.

ACKs for top commit:
  laanwj:
    Code review ACK 76782e560b
  MarcoFalke:
    cr ACK 76782e560b

Tree-SHA512: 1d7948b3385632094a3b0f0e38f87dccddabf74002e68aa055a51408866b057828ffa15c4b22aa9adde458155fbb5e443b66a9dbf3d7713358fc98a14d64bdcf
2021-03-16 12:06:26 +01:00
Wladimir J. van der Laan
af6ee17545 Merge #21405: compat: remove memcpy -> memmove backwards compatibility alias
52f0be3a93 compat: remove memcpy -> memmove backwards compatibility alias (fanquake)

Pull request description:

  In glib 2.13 memcpy was changed such that the way it copied bytes was reversed.
  This caused all sorts of issues for existing software, which depended on the
  existing behavior (when they should have been using memmove). See:
  https://sourceware.org/bugzilla/show_bug.cgi?id=12518
  https://bugzilla.redhat.com/show_bug.cgi?id=638477

  Now that we require glibc 2.17+ (#17538), we should be well clear of having to
  maintain our memcpy ->  memmove aliasing, which was introduced in #4339.

  Gitian builds:
  ```bash
  # Linux:
  52dee59c8c7d5620ac9b140b79fcaf3d2f15a219293140190f9283ba871f5391  bitcoin-52f0be3a9332-aarch64-linux-gnu-debug.tar.gz
  8963473b8791c5c6033a992d7dd761832fe1fb5732be790a6e9f8c11d67ad8ae  bitcoin-52f0be3a9332-aarch64-linux-gnu.tar.gz
  1fb3365c1ef60ecd1eb2d18f671f8f1e8cde0585de7de74aa0c5121093100c26  bitcoin-52f0be3a9332-arm-linux-gnueabihf-debug.tar.gz
  305c5b032d51ba97459715211112204a09d119edd6ec2a12b796559ad3fde761  bitcoin-52f0be3a9332-arm-linux-gnueabihf.tar.gz
  1f950a3e3979a4e1a67696b3fddc3090a0489a43b49e2b58a348d4b02ada2aa8  bitcoin-52f0be3a9332-powerpc64-linux-gnu-debug.tar.gz
  0b9731dba768b30c91dadec4cd7a98c86e06fbf6354555f798b46b7c4fab7b5f  bitcoin-52f0be3a9332-powerpc64-linux-gnu.tar.gz
  c4a37aae56cc023964f8d9e82d1b66913079cab559cbfc1c9127969aa968a06f  bitcoin-52f0be3a9332-powerpc64le-linux-gnu-debug.tar.gz
  dfbaa4f3bf12988a0a7f82c4b10162e5e7a63382a7e29d0170bc32ce344c97c3  bitcoin-52f0be3a9332-powerpc64le-linux-gnu.tar.gz
  3a0280d2c06516e50b0841d6f42d9589355dc9a1f8bb9a0b123554cd91b08004  bitcoin-52f0be3a9332-riscv64-linux-gnu-debug.tar.gz
  cc199a0f254b2366e80a6a884120ec3ea442983990ba1a5eb993c36060686eba  bitcoin-52f0be3a9332-riscv64-linux-gnu.tar.gz
  eb8e7ca673cc06c167ab082fe457a41f73758ecd5b34941300e3cd378c29b197  bitcoin-52f0be3a9332-x86_64-linux-gnu-debug.tar.gz
  dad19226c0e4c54b78ca2fa85fc28c5bfd1e1178e3f765472bd2f895a1d57145  bitcoin-52f0be3a9332-x86_64-linux-gnu.tar.gz
  ef89be95b84bb7c6fef055634cd20caf2fa5b42441502918dbfbf758bb2daab6  src/bitcoin-52f0be3a9332.tar.gz
  dc61f5ca33330c1609bc56b23f39fef3c1ff5ec6a1799d5b7a18f3c3b3acc9f9  bitcoin-core-linux-22-res.yml
  ```

  Guix builds:
  ```bash
  b50d6399cb59e5e4a9247b12a3eda61de6e51bd87ef1f27b388b75b71dfccf92  output/bitcoin-52f0be3a9332-aarch64-linux-gnu-debug.tar.gz
  23d845dc13e60a581ebdfbaa6063f559a56cce06734e1b50790d2fc13e257793  output/bitcoin-52f0be3a9332-aarch64-linux-gnu.tar.gz
  79094406fe00939bbce17a6d65de5a2686625e871432350c69e674cc80b1491c  output/bitcoin-52f0be3a9332-arm-linux-gnueabihf-debug.tar.gz
  65a91913249a743015eceea5a56c497d606af17270cb7e8a3df10cf729b757ec  output/bitcoin-52f0be3a9332-arm-linux-gnueabihf.tar.gz
  5e75ca5e8cf6934ba5a5a1b4d26c1b361b118e10ef34b73845d038035ddb9b85  output/bitcoin-52f0be3a9332-osx-unsigned.dmg
  774b372696cde8ceab40f6909dadea3fc87b375b495fcfb4ee8a963afd7fbd3a  output/bitcoin-52f0be3a9332-osx-unsigned.tar.gz
  dc4bdfb7b32dcc0b6e876d6d7ab3cb8d1472f21f66546ab70515f96262292e21  output/bitcoin-52f0be3a9332-osx64.tar.gz
  ea178ff9e28439f80129445cf260215c74eea2e610f62ff045061f287675d3ff  output/bitcoin-52f0be3a9332-powerpc64-linux-gnu-debug.tar.gz
  0390687a7aaa3f0a8a78be2deab21116599e5b332f00a2d1fdce97a5bd30e3eb  output/bitcoin-52f0be3a9332-powerpc64-linux-gnu.tar.gz
  52c948719a27f252f5969558abc2718c1e365ea85496322cb4ec97eab8a234cc  output/bitcoin-52f0be3a9332-powerpc64le-linux-gnu-debug.tar.gz
  5a4a8748dffe7e6a5bd07f3f564b1f2052440c4199fe25aaa41675bfb69e61db  output/bitcoin-52f0be3a9332-powerpc64le-linux-gnu.tar.gz
  ba521bd2b4e73aea317821a9e08da9a326c0be3b38d923b35ba14bc68ee6c814  output/bitcoin-52f0be3a9332-riscv64-linux-gnu-debug.tar.gz
  783ea81ab2f6b642b13ebf7882aa822d12f95936574a8848a74b1b8978e6801d  output/bitcoin-52f0be3a9332-riscv64-linux-gnu.tar.gz
  376706fc12e58d7d559a87e1ce64be22eaac3fc32d95c60d603ad893d9128cc1  output/bitcoin-52f0be3a9332-win-unsigned.tar.gz
  7aa48242fb71e29b00992b2be8677f1ea49f2ca82c5355bf0c1d4c8d14635596  output/bitcoin-52f0be3a9332-win64-debug.zip
  41e6461ab573fa8f6ac0f198193e72a4a047bb7a4193f743b937e81739c929cc  output/bitcoin-52f0be3a9332-win64-setup-unsigned.exe
  e2c4ecb05f24577da12f722d848bf6ac89f3f549d6d2bfd30d65676099c0725b  output/bitcoin-52f0be3a9332-win64.zip
  60ed63b3b562fa2141f18f1556a03c2474b75797088cd68fdb3e7d057a6983a3  output/bitcoin-52f0be3a9332-x86_64-linux-gnu-debug.tar.gz
  adb0bb62dc8b99d025a863e921b8e670f4c8f4b5600cd6d79eb552ede10bc8b8  output/bitcoin-52f0be3a9332-x86_64-linux-gnu.tar.gz
  ef89be95b84bb7c6fef055634cd20caf2fa5b42441502918dbfbf758bb2daab6  output/src/bitcoin-52f0be3a9332.tar.gz
  ```

ACKs for top commit:
  laanwj:
    Concept and code review ACK 52f0be3a93

Tree-SHA512: 851634a633cc7d27b10f11436768f3695a7615d5850166c3718028c36d3a7dd56baa2dd1028f47802891703e9f5a1d382f559e388ecef2249e2004edc62d97bf
2021-03-16 11:52:54 +01:00
Wladimir J. van der Laan
cb0aafbd21 Merge #21444: net, doc: Doxygen updates and fixes in netbase.{h,cpp}
8348a3742b net: fix hSocket param in netbase.h::ConnectSocketDirectly() (Jon Atack)
e6bd74b2e5 net: move Doxygen docs from netbase.cpp to netbase.h (Jon Atack)
12cc5704db net: update incorrect Doxygen documentation in netbase.cpp (Jon Atack)

Pull request description:

  While doing #21328, I noticed docs that were out-of-date or in the wrong file.

  The second commit is essentially move-only and is best reviewed with `colorMoved = dimmed-zebra` and `colorMovedWs = allow-indentation-change`.

ACKs for top commit:
  laanwj:
    Code review ACK 8348a3742b

Tree-SHA512: 13dae4abd3009fc43dfffc98e0f7eebcd6ad02afdd6050a7685a2ad4e6aaad93480d93886a2d1bd2375c2439d426494e4a8bc0c60e0e3104bfaa1830831ca663
2021-03-16 11:47:36 +01:00
Vasil Dimov
7059e6d822 test: add a test to ensure RecvUntilTerminator() limit works 2021-03-16 11:00:57 +01:00
Vasil Dimov
80a5a8ea2b i2p: limit the size of incoming messages
Put a limit on the amount of data `Sock::RecvUntilTerminator()` can read
if no terminator is received.

In the case of I2P this avoids a runaway (or malicious) I2P proxy
sending us tons of data without a terminator before a timeout is
triggered.
2021-03-16 11:00:57 +01:00
Hennadii Stepanov
4d008f908e Always add -daemonwait to known command line arguments 2021-03-16 10:48:26 +02:00
MarcoFalke
8e6532053f Merge bitcoin-core/gui#246: Revert "qt: Use "fusion" style on macOS Big Sur with old Qt"
77833a364a Revert "qt: Use "fusion" style on macOS Big Sur with old Qt" (Hennadii Stepanov)

Pull request description:

  This PR reverts workaround introduced in #177.

  After bumping Qt version in depends to 5.12.10 in bitcoin/bitcoin#21376, there are no reasons to use the Fusion style on macOS.

ACKs for top commit:
  leonardojobim:
    tACK 77833a364a. Tested on macOS Big Sur v11.2.3
  jarolrod:
    ACK 77833a364a
  Talkless:
    utACK 77833a364a

Tree-SHA512: f704f2027dd380dfc604231e3606a036a8be891aeeddf643c474131014fa080e123b42836ac643a2064fe7a5a018fa8b9aa61a31f9da1d15880de6a36c4c0d54
2021-03-16 09:05:26 +01:00
Pieter Wuille
725d7ae049 Use PrecomputedTransactionData in signet check
This is out of an abundance of caution only, as signet currently doesn't
enable taproot validation flags. Still, it seems cleaner to make sure
that all non-test code that passes MissingDataBehavior::ASSERT_FAIL
also actually makes sure no data can be missing.
2021-03-15 17:29:39 -07:00
Pieter Wuille
497718b467 Treat amount<0 also as missing data for P2WPKH/P2WSH
Historically lack of amount data has been treated as amount==-1. Change
this and treat it as missing data, as introduced in the previous commits.

To be minimally invasive, do this at SignatureHash() call sites rather
than inside SignatureHash() (which currently has no means or returning
a failure code).
2021-03-15 17:29:39 -07:00
Pieter Wuille
3820090bd6 Make all SignatureChecker explicit about missing data
Remove the implicit MissingDataBehavior::ASSERT_FAIL in the
*TransationSignatureChecker constructors, and instead specify
it explicit in all call sites:
* Test code uses ASSERT_FAIL
* Validation uses ASSERT_FAIL (through CachingTransactionSignatureChecker)
  (including signet)
* libconsensus uses FAIL, matching the existing behavior of the
  non-amount API (and the extended required data for taproot validation
  is not available yet)
* Signing code uses FAIL
2021-03-15 17:29:39 -07:00
Pieter Wuille
b77b0cc507 Add MissingDataBehavior and make TransactionSignatureChecker handle it
This allows specifying how *TransactionSignatureChecker will behave when
presented with missing transaction data such as amounts spent, BIP341 data,
or spent outputs.

As all call sites still (implicitly) use MissingDataBehavior::ASSERT_FAIL,
this commit introduces no change in behavior.
2021-03-15 17:29:34 -07:00
Jon Atack
76782e560b refactor: reuse test string with embedded null char in util_tests 2021-03-15 23:13:52 +01:00
Jon Atack
24c6546946 test: add ParseUInt8() unit and fuzz test coverage 2021-03-15 20:50:20 +01:00
MarcoFalke
4ba1bab443 Merge #21446: Update vcpkg checkout commit
b9e3f35306 Update vcpkg checkout commit. (Aaron Clauson)

Pull request description:

  Previously vcpkg was relying on `https://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz` which is no longer available. The vcpkg source has been updated to use `http://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-2-any.pkg.tar.zst`.

  This PR updates the commit ID used to checkout vcpkg for the updated URL.

Top commit has no ACKs.

Tree-SHA512: 3e2f6d8cc14ef0ca93f75179da3e8303e19fb27d4a95e66e35376672744733fdbed9eb330638a48aa9962792674e584b702f4fbfc2c2564c5022bf8dbde6056c
2021-03-15 20:03:24 +01:00
Wladimir J. van der Laan
b650c9140e Merge #21141: wallet: Add new format string placeholders for walletnotify
06e1fb0b17 Add new format string placeholders for walletnotify to include relevant block information for transactions (Maayan Keshet)

Pull request description:

  This patch includes two new format placeholders for walletnotify:
  %b - the hash of the block containting the transaction (zeroed if a mempool transaction)
  %h - the height of the block containing the transaction (zero if a mempool transaction)

  I've included test suite changes to check and validate the above functional requirements as well as doc/help description changes.

  **Motivation**
  The walletnotify option is used to be notified of new transactions relevant to the wallet of the node.
  A common usage pattern is to perform afterwards additional RPC calls to determine:
  1. If this is a mempool transaction or not (i.e. are there any confirmations?)
  2. What block was it included in?
  3. Did this transaction was seen before and is now seen again because of a fork?

  All of these questions can be answered with the current features, but the resulting RPC calls may be expensive in a heavily used node. As this information is readily available when calling the walletnotify callback, it makes sense to save expensive round trips by optionally sending this information at that point in time. I can definitely say we would like to use it in Fireblocks, my employer.

  Please let me know of any questions and suggestions.

ACKs for top commit:
  laanwj:
    ACK 06e1fb0b17

Tree-SHA512: d2744e2a7a883f9c3a9fd32237110e048c4b6b11fea8221c33d10b74157f65bbc4351211f441e8c1a4af5d5d38e2ba6b1943a7673dc18860c0553d7b41e00775
2021-03-15 19:55:38 +01:00
MarcoFalke
67ec26cacf Merge #19259: fuzz: Add fuzzing harness for LoadMempool(...) and DumpMempool(...)
68afd3eeec tests: Add fuzzing harness for LoadMempool(...) and DumpMempool(...) (practicalswift)
91af6b97c9 validation: Make DumpMempool(...) and LoadMempool(...) easier to test/fuzz/mock (practicalswift)
af322c7494 tests: Set errno in FuzzedFileProvider. Implement seek(..., ..., SEEK_END). (practicalswift)

Pull request description:

  Add fuzzing harness for `LoadMempool(...)` and `DumpMempool(...)`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  jonatack:
    Tested re-ACK 68afd3eeec

Tree-SHA512: 4b5fcaa87e6eb478611d3b68eb6859645a5e121e7e3b056ad2815699dace0a6123706ff542def371b47f4ab3ce2b8a29782026d84fb505827121e9b4cc7dac31
2021-03-15 18:56:06 +01:00
MarcoFalke
3ba195aa48 Merge #21398: doc: Update fuzzing docs for afl-clang-lto
fab633d2db doc: Update fuzzing docs for afl-clang-lto (MarcoFalke)

Pull request description:

  Update the docs to default to `afl-clang-lto`. The afl-gcc (and other afl legacy fuzz engines) are still supported, though discouraged.

ACKs for top commit:
  fanquake:
    ACK fab633d2db - seems to work for me. Compiled and ran some fuzzers using Clang 11 on Bionic. Set `llvm-config` so that `clang-11` would be used over `clang` (10).
  jarolrod:
    ACK fab633d2db, tested on Ubuntu Focal

Tree-SHA512: 3d1969c167bea45a9d691f3b757f51213d550c9c1b895bed1fcf3c2f7345791787cfb13c376291b94eb3181caf4ae3126f4d01c7cebda7b2bb1c40a1294e9a68
2021-03-15 18:52:58 +01:00
Aaron Clauson
b9e3f35306 Update vcpkg checkout commit.
Previously vcpkg was relying on https://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-1-any.pkg.tar.xz which is no longer available. The vcpkg source has been updated to use http://repo.msys2.org/mingw/i686/mingw-w64-i686-pkg-config-0.29.2-2-any.pkg.tar.zst. This PR updates the commit ID used to checkout vcpkg for the updated URL.
2021-03-15 17:18:42 +00:00
Maayan Keshet
06e1fb0b17 Add new format string placeholders for walletnotify to include relevant block information for transactions 2021-03-15 18:45:36 +02:00
MarcoFalke
c771fc0dc1 Merge #21424: Net processing: Tidy up CNodeState ctor
6927933782 [net processing] Add ChainSyncTimeoutState default initializers (John Newbery)
55966e0cc0 [net processing] Remove CNodeState ctor body (John Newbery)

Pull request description:

  This addresses the two outstanding review comments from #21370.

ACKs for top commit:
  practicalswift:
    cr ACK 6927933782: patch looks correct
  hebasto:
    ACK 6927933782, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: b3ef5c8a096e447887df255406b3a760f01c73e2b942374595416b4b4031fc69b89cd93168c45040489d581f340b2a62d3fbabd207d4307f587c00a7a7daacd1
2021-03-15 17:36:16 +01:00
MarcoFalke
9a5e097435 Merge #21443: fuzz: Implement fuzzed_dns_lookup_function as a lambda
7c8c140ecc fuzz: Implement fuzzed_dns_lookup_function as lambda (practicalswift)

Pull request description:

  Implement `fuzzed_dns_lookup_function` as a lambda.

  As wisely suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/19415#discussion_r594244506. Thanks! :)

ACKs for top commit:
  MarcoFalke:
    cr ACK 7c8c140ecc
  vasild:
    ACK 7c8c140ecc

Tree-SHA512: b175f2ad42e9a2be1f769ac677b2872e73ae621731d27e9a24bedadc14d9a6682c7fd1946a0df436d37e7b0cc0d212c1eef69f0409fb975cf9c460cd45f6e4ac
2021-03-15 17:26:16 +01:00
Jon Atack
8348a3742b net: fix hSocket param in netbase.h::ConnectSocketDirectly() 2021-03-15 16:57:48 +01:00
Jon Atack
e6bd74b2e5 net: move Doxygen docs from netbase.cpp to netbase.h 2021-03-15 16:53:25 +01:00
practicalswift
68afd3eeec tests: Add fuzzing harness for LoadMempool(...) and DumpMempool(...) 2021-03-15 15:42:15 +00:00
Jon Atack
12cc5704db net: update incorrect Doxygen documentation in netbase.cpp 2021-03-15 16:41:09 +01:00
practicalswift
7c8c140ecc fuzz: Implement fuzzed_dns_lookup_function as lambda 2021-03-15 15:27:25 +00:00
MarcoFalke
eceb3f7707 Merge #19415: net: Make DNS lookup mockable, add fuzzing harness
e528075189 tests: Add fuzzing harness for Lookup(...)/LookupHost(...)/LookupNumeric(...)/LookupSubNet(...) (practicalswift)
c6b4bfb4b3 net: Make DNS lookup code testable (practicalswift)

Pull request description:

  Make DNS lookup mockable/testable/fuzzable.

  Add fuzzing harness for `Lookup(…)`/`LookupHost(…)`/`LookupNumeric(…)`/`LookupSubNet(…)`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  Crypt-iQ:
    cr ACK e528075189
  vasild:
    ACK e528075189

Tree-SHA512: 9984c2e2fedc3c1e1c3dbd701bb739ebd2f01766e6e83543dae5ae43eb8646c452bba0e101dd2f06079e5258bd5846c7d27a60ed5d77c1682b54c9544ffad443
2021-03-15 12:05:13 +01:00
MarcoFalke
c47979778d Merge #21426: rpc: remove scantxoutset EXPERIMENTAL warning
2f0b25a156 rpc: remove scantxoutset EXPERIMENTAL warning (Jon Atack)

Pull request description:

  Remove old warning per IRC wallet meeting discussion at http://www.erisian.com.au/bitcoin-core-dev/log-2021-03-12.html#l-467

  This RPC was merged 3 years ago in #12196.

ACKs for top commit:
  MarcoFalke:
    cr ACK 2f0b25a156

Tree-SHA512: 874ccd5bd19ecbbe91912171ac85af7a4658dc92f6db484ff3d03f07f1b9ba97e1c69d33a5c3ae5c5ec46cac3595a211f55fec0fbf81bac30d66a891c376ce26
2021-03-15 11:14:47 +01:00
Jon Atack
2f0b25a156 rpc: remove scantxoutset EXPERIMENTAL warning 2021-03-15 10:45:49 +01:00
MarcoFalke
1e57d14d96 Merge #21035: Remove pointer cast in CRPCTable::dumpArgMap
9048c58e10 Remove pointer cast in CRPCTable::dumpArgMap (Russell Yanofsky)
14f3d9b908 refactor: Add RPC server ExecuteCommands function (Russell Yanofsky)
6158a6d397 refactor: Replace JSONRPCRequest fHelp field with mode field (Russell Yanofsky)

Pull request description:

  This change is needed to fix the `rpc_help.py` test failing in #10102: https://cirrus-ci.com/task/5469433013469184?command=ci#L2275

  The [`CRPCTable::dumpArgMap`](16b784d953/src/rpc/server.cpp (L492)) method currently works by casting RPC `unique_id` integer field to a function pointer, and then calling it. The `unique_id` field wasn't supposed to be used this way (it's meant to be used to detect RPC aliases) and as a result, this code segfaults in the `rpc_help.py` test in multiprocess PR #10102 because wallet RPC functions aren't directly accessible from the node process.

  Fix this by adding a new `GET_ARGS` RPC request mode to retrieve argument information similar to the way the `GET_HELP` mode retrieves help information.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

ACKs for top commit:
  MarcoFalke:
    re-ACK 9048c58e10 👑

Tree-SHA512: cd1a01c1daa5bde2c2455b63548371ee4cf39688313969ad2016d9a0fd4344102e3fd43034058f253364518e9632d57cf21abffad0d6a2c0c94b7a6921cbe615
2021-03-15 10:13:58 +01:00
MarcoFalke
6bc51af5c2 Merge bitcoin-core/gui#250: scripted-diff: Drop redundant QString calls
def1e64bb4 scripted-diff: Drop redundant QString calls (Hennadii Stepanov)

Pull request description:

  The return type of `QObject::tr` function _is_ `QString` 🐅

ACKs for top commit:
  jarolrod:
    ACK def1e64bb4, tested on macOS 10.14.6 Qt 5.15.2

Tree-SHA512: ef405c87a30d6965f6887511d8666b6da57d258ca07833a3fa2dc9fd147d0539d33c57f7551ee13c1dd8024d6057139595c6ce5d088dd6efd7aa13db2a3eebdb
2021-03-15 09:05:18 +01:00
fanquake
16209b1b19 Merge #20556: rpc: Properly document return values (submitblock, gettxout, getblocktemplate, scantxoutset)
fa7ff0790e rpc: Properly document submitblock return value (MarcoFalke)
fae542c28b rpc: Properly document getblocktemplate return value (MarcoFalke)
fabaccf031 rpc: Properly document scantxoutset return value (MarcoFalke)
faa2059547 rpc: Properly document gettxout return value (MarcoFalke)

Pull request description:

  Currently a few return values are undocumented. This is causing confusion at the least. See for example #18476

ACKs for top commit:
  fjahr:
    utACK fa7ff0790e
  amitiuttarwar:
    tACK fa7ff0790e

Tree-SHA512: 933cb8f003163d93dbedb302d4c162514c2698ec6d58dbb9a053da8b8b9a4459b0701a3d9e830ecdabd7f278a46b7a07a3af49ec60703a80fcd75390877294ea
2021-03-15 15:32:15 +08:00
fanquake
d42078e93b Merge #21437: gitian, ci: Drop unneeded python3-dev package for macOS builds
e4c0cada79 ci, gitian: Drop unneeded python3-dev package for macOS builds (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK e4c0cada79 - gitian builds match and I checked that this doesn't end up installed as a side-effect of another package.

Tree-SHA512: 520a3909b106a0e005b195c5395691edf62b76ee2df43b6971b7aa193648d68e6dac69cb4f1dc474f594b015a2fc2074061865e571d89365174beb5c1780356f
2021-03-15 14:46:11 +08:00
fanquake
57e980d13c scripted-diff: remove Optional & nullopt
-BEGIN VERIFY SCRIPT-
git rm src/optional.h

sed -i -e 's/Optional</std::optional</g' $(git grep -l 'Optional<' src)

sed -i -e 's/{nullopt}/{std::nullopt}/g' $(git grep -l 'nullopt' src)
sed -i -e 's/ nullopt;/ std::nullopt;/g' $(git grep -l 'nullopt' src)
sed -i -e 's/ nullopt)/ std::nullopt)/g' $(git grep -l 'nullopt' src)
sed -i -e 's/(nullopt)/(std::nullopt)/g' $(git grep -l 'nullopt' src)
sed -i -e 's/ nullopt,/ std::nullopt,/g' $(git grep -l 'nullopt' src)
sed -i -e 's/? nullopt :/? std::nullopt :/g' $(git grep -l 'nullopt' src)
sed -i -e 's/: nullopt}/: std::nullopt}/g' $(git grep -l 'nullopt' src)

sed -i -e '/optional.h \\/d' src/Makefile.am

sed -i -e '/#include <optional.h>/d' src/test/fuzz/autofile.cpp src/test/fuzz/buffered_file.cpp src/test/fuzz/node_eviction.cpp

sed -i -e 's/#include <optional.h>/#include <optional>/g' $(git grep -l '#include <optional.h>' src)
-END VERIFY SCRIPT-
2021-03-15 10:41:30 +08:00
Hennadii Stepanov
def1e64bb4 scripted-diff: Drop redundant QString calls
-BEGIN VERIFY SCRIPT-
sed -i 's/QString(QObject::tr(\([^)]*\))/QObject::tr(\1/' src/qt/guiutil.cpp
-END VERIFY SCRIPT-
2021-03-15 01:23:21 +02:00
Hennadii Stepanov
e4c0cada79 ci, gitian: Drop unneeded python3-dev package for macOS builds 2021-03-14 14:55:12 +02:00
Hennadii Stepanov
1a01a5dc8d doc: Update zlib info in dependencies.md 2021-03-14 13:13:13 +02:00
MarcoFalke
3c631917f3 Merge #21111: Improve OpenRC initscript
95f97111dd contrib/init: (OpenRC) quote some unquoted variables. (parazyd)
737feadff7 contrib/init: (OpenRC) Do not fail if both rpcuser and rpcpassword are unset. (parazyd)

Pull request description:

  This pull request improves the available OpenRC initscripts in
  `contrib/init`.

  The first commit (737feadff7) reworks
  `checkconfig()` to not fail if **both** `rpcuser` and `rpcpassword`
  are unset, because this implies that bitcoind shall use the `.cookie`
  file for RPC authentication. Currently, the initscript does not allow
  starting bitcoind without a set `rpcuser` and `rpcpassword`.

  The second commit (95f97111dd) simply
  quotes some unquoted variables.

ACKs for top commit:
  kristapsk:
    ACK 95f97111dd

Tree-SHA512: 62bebcd07143c147e349c0cfc17b54ef21bd4684377b444f58c6bd1f509a4d3e1af58746fa7215f18e33021f691bbbc5e42f4df497458322b055e545b7f30d46
2021-03-13 18:52:35 +01:00
Hennadii Stepanov
0fc0c00f7a test: Drop unused get_machine function
The removed get_machine function is no longer used since #21255.
2021-03-13 18:29:08 +02:00
Hennadii Stepanov
61a0f8f9cc test: Cleanup test files in test-{security,symbol}-check.py 2021-03-13 18:27:11 +02:00
Hennadii Stepanov
77833a364a Revert "qt: Use "fusion" style on macOS Big Sur with old Qt"
This reverts commit 4e1154dfd1.

After bumping Qt version in depends to 5.12.10 in bitcoin/bitcoin#21376,
there are no reasons to use the Fusion style on macOS.
2021-03-12 18:55:44 +02:00
John Newbery
6927933782 [net processing] Add ChainSyncTimeoutState default initializers
Moves the default values closer to the member definitions.
2021-03-12 11:39:22 +00:00
John Newbery
55966e0cc0 [net processing] Remove CNodeState ctor body
It's a no-op. The ctor for RollingBloomFilter already calls reset().
2021-03-12 11:37:50 +00:00
MarcoFalke
e0bc27a14c Merge #21404: refactor: Remove MakeUnique<T>()
1a6323bdbe doc: update developer notes for removal of MakeUnique (fanquake)
3ba2840e7e scripted-diff: remove MakeUnique<T>() (fanquake)

Pull request description:

  Since requiring C++17, this is just pointless abstraction. I think we should just "tear the band-aid off" and remove it. Similar to the changes happening in #21366.

  Also, having a comment saying this is deprecated doesn't prevent it's usage in new code. i.e : https://github.com/bitcoin/bitcoin/pull/20946#discussion_r561949731.

  The repository is fairly quiet at the moment, so any potential complaints about having to rebase should be minimal. Might as well get this over and done with.

ACKs for top commit:
  jnewbery:
    utACK 1a6323bdbe
  practicalswift:
    cr ACK 1a6323bdbe: patch looks correct
  ajtowns:
    ACK 1a6323bdbe -- code review only
  glozow:
    ACK 1a6323bdbe looks correct

Tree-SHA512: 4a14b9611b60b9b3026b54d6f5a2dce4c5d9b63a7b93d7de1307512df736503ed84bac66e7b93372c76e3117f49bf9f29cd473d3a47cb41fb2775bc10234736f
2021-03-12 08:34:15 +01:00
fanquake
a13a8cd8e3 Merge #21394: [doc] Improve comment about protected peers
ebde946a52 [doc] Improve comment about protected peers (Amiti Uttarwar)

Pull request description:

  The comment currently suggests a long-standing node would infrequently protect peers under normal circumstances. Clarify that we also protect peers that are synced to the same work as our chain tip. [Relevant check here](ee0dc02c6f/src/net_processing.cpp (L1997)).

ACKs for top commit:
  Empact:
    ACK ebde946a52
  jnewbery:
    ACK ebde946a52

Tree-SHA512: 3692f4098e95f935d801e0ee6bbd3a7c9480e66ca070a7c68ba79c4fc2e62377f5d37080c7b6a7d15ab617aaf4d3df9b26abc4f1b090d572ba46fdd092a6a64a
2021-03-12 13:11:10 +08:00
fanquake
7b3434f800 build: don't try and use -fstack-clash-protection on Windows
This has never worked with any of the mingw-w64 compilers we use, and
the -O0 is causing issues for builders applying spectre mitigations.

Recent discussion on https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458
also indicates that this should just not be used on Windows.
2021-03-12 10:21:21 +08:00
practicalswift
91af6b97c9 validation: Make DumpMempool(...) and LoadMempool(...) easier to test/fuzz/mock 2021-03-11 22:34:39 +00:00
practicalswift
af322c7494 tests: Set errno in FuzzedFileProvider. Implement seek(..., ..., SEEK_END). 2021-03-11 22:34:39 +00:00
MarcoFalke
b972913c33 Merge bitcoin-core/gui#188: Write PSBTs to file with binary mode
cc3971c9ff GUI: Write PSBTs to file with binary mode (Andrew Chow)

Pull request description:

  As noted in https://github.com/bitcoin/bitcoin/issues/20959, PSBT files should be opened in binary mode as on windows, all newlines are turned into CRLF which produces invalid PSBTs.

  Fixes https://github.com/bitcoin/bitcoin/issues/20959

ACKs for top commit:
  Talkless:
    utACK cc3971c9ff.

Tree-SHA512: fee62b66da844017a44d7d6da6d2d2794b097a7dec33fb07711615df1e94dccc76f987ffcbb325ad1f8db2a2dd6eaf514b6cbd2453e7658b9f6c9fb5c4c41dab
2021-03-11 19:43:43 +01:00
MarcoFalke
e9b6400a1e Merge #21411: test: add logging, reduce blocks, move sync_all in wallet_ groups
c62f9bc0e9 test: use fewer blocks in wallet_groups and move sync call (Jon Atack)
3a16b5ef95 test: add missing logging to wallet_groups.py (Jon Atack)

Pull request description:

  - add logging (particularly useful as the tests are somewhat slow)
  - generate 101 blocks instead of 110
  - move `sync_all` call into the loop, so fewer blocks are synced on each call, to hopefully see fewer CI timeouts as in https://bitcoinbuilds.org/index.php?ansilog=88eee99e-1727-44ed-b778-3b9c75c33928.log

  ```
  L2742     File "/home/ubuntu/src/test/functional/wallet_groups.py", line 162, in run_test
  L2743       self.sync_all()
  test_framework.authproxy.JSONRPCException: 'syncwithvalidationinterfacequeue' RPC took longer than 960.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK c62f9bc0e9

Tree-SHA512: 711deafcd589cb8196cb207ff882e0f2ab6b70828a6abad91f83f535974cc430a56b9e8a960fd233d31d610932a0d48b49ee681aae564d145a3040288ecda8f8
2021-03-11 19:09:09 +01:00
MarcoFalke
c970c1477d Merge #21380: tests: Add fuzzing harness for versionbits
1639c3b76c tests: Add fuzzing harness for versionbits (Anthony Towns)

Pull request description:

  Adds a fuzzing harness for versionbits.

ACKs for top commit:
  sipa:
    utACK 1639c3b76c
  jnewbery:
    utACK 1639c3b76c
  MarcoFalke:
    cr ACK 1639c3b76c

Tree-SHA512: 6bcf4d302b2193b56c72eecdb79d156b90d05b02ce3a1ad8f4c8a0fcf5caab91e7c78fe61ea280a69cdadcb5006376d4ade877169cd56dc084c2e70359651f0b
2021-03-11 17:21:15 +01:00
fanquake
05757aa860 Merge #21423: build: Cleanups and follow ups after bumping Qt to 5.12.10
e67c0122a4 doc: Update dependencies.md with a new Qt version (Hennadii Stepanov)
cc25f892d2 build: Cleanup libxkbcommon_postprocess_cmds (Hennadii Stepanov)
72fc043954 build, qt: Drop redundant -lxcb-static flag (Hennadii Stepanov)
cba4a7e416 build, qt: Always test plugins/subdir before adding to search paths (Hennadii Stepanov)

Pull request description:

  1) Always test `plugins/subdir` before adding to search paths as the existence of each subdir is not guaranteed for all platforms:
    - https://github.com/bitcoin/bitcoin/pull/21376#discussion_r591613489

  2) Drop redundant `-lxcb-static` flag as it has been already linked with `Qt5XcbQpa`:
    - https://github.com/bitcoin/bitcoin/pull/21363#discussion_r588881613

  3) Cleanup `libxkbcommon_postprocess_cmds` as there is no `share/` directory in the staging one:
    - https://github.com/bitcoin/bitcoin/pull/21376#discussion_r588867355
    - https://github.com/bitcoin/bitcoin/pull/21376#issuecomment-794010534

  4) Update `dependencies.md`

ACKs for top commit:
  fanquake:
    ACK e67c0122a4

Tree-SHA512: 9113ee97d5e7424290778154d62a68af804ee82efedbbe9776a7f692104d65b07d151e9f7f1f98ec08d18f6d63efef3e44b207bee67ad913f5dbc4eddbb8ea41
2021-03-19 16:58:13 +08:00
Anthony Towns
1639c3b76c tests: Add fuzzing harness for versionbits 2021-03-19 15:05:21 +10:00
fanquake
47d79c941a Merge #21471: bugfix: fix bech32_encode calls in gen_key_io_test_vectors.py
5c0210e3e6 bugfix: fix bech32_encode calls in gen_key_io_test_vectors.py (Pieter Wuille)

Pull request description:

  This fixes the the calls to bech32_encode in the gen_key_io_test_vectors.py script.

  Bug introduced in #20861.

ACKs for top commit:
  fanquake:
    ACK 5c0210e3e6

Tree-SHA512: 8e8aee08741619c1700371ca1a8ca05ffdb2f48544d9fd3982f2665f6afb926b126478cf644f15a699f8c7e7da53c2777a56ce7989f05e4a3ef9fbe085f74d5a
2021-03-19 11:32:29 +08:00
Pieter Wuille
5c0210e3e6 bugfix: fix bech32_encode calls in gen_key_io_test_vectors.py 2021-03-18 14:28:46 -07:00
Wladimir J. van der Laan
8ec881d3b6 Merge #20861: BIP 350: Implement Bech32m and use it for v1+ segwit addresses
03346022d6 naming nits (Fabian Jahr)
2e7c80fb5b Add signet support to gen_key_io_test_vectors.py (Pieter Wuille)
fe5e495c31 Use Bech32m encoding for v1+ segwit addresses (Pieter Wuille)
25b1c6e13d Add Bech32m test vectors (Pieter Wuille)
da2bb6976d Implement Bech32m encoding/decoding (Pieter Wuille)

Pull request description:

  This implements [BIP 350](https://github.com/bitcoin/bips/blob/master/bip-0350.mediawiki):
  * For segwit v1+ addresses, a new checksum algorithm called Bech32m is used.
  * Segwit v0 address keep using Bech32 as specified in [BIP 173](https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki).

ACKs for top commit:
  Sjors:
    utACK 0334602
  jnewbery:
    utACK 03346022d6
  achow101:
    ACK 0334602
  fjahr:
    re-ACK 0334602
  benthecarman:
    ACK 03346022d6

Tree-SHA512: 4424cfd44869d813d6152fb3ed867b204036736bc2344a039b93700b6f36a43e9110478f138eb81c97c77ab27ecb776dada5ba632cb5a3a9d244924d2540a557
2021-03-18 20:37:21 +01:00
Fabian Jahr
03346022d6 naming nits 2021-03-17 17:59:22 -07:00
Pieter Wuille
2e7c80fb5b Add signet support to gen_key_io_test_vectors.py 2021-03-16 10:48:40 -07:00
Pieter Wuille
fe5e495c31 Use Bech32m encoding for v1+ segwit addresses
This also includes updates to the Python test framework implementation,
test vectors, and release notes.
2021-03-16 10:48:36 -07:00
Pieter Wuille
25b1c6e13d Add Bech32m test vectors 2021-03-15 17:26:39 -07:00
Pieter Wuille
da2bb6976d Implement Bech32m encoding/decoding 2021-03-15 17:26:35 -07:00
Hennadii Stepanov
e67c0122a4 doc: Update dependencies.md with a new Qt version
Qt version is bumped to 5.12.10 in #21376.
2021-03-12 14:51:11 +02:00
Hennadii Stepanov
cc25f892d2 build: Cleanup libxkbcommon_postprocess_cmds
There is no "share" directory in the staging one.
2021-03-12 14:51:10 +02:00
Hennadii Stepanov
72fc043954 build, qt: Drop redundant -lxcb-static flag
The removed flag has been already linked with Qt5XcbQpa.
2021-03-12 14:51:10 +02:00
Hennadii Stepanov
cba4a7e416 build, qt: Always test plugins/subdir before adding to search paths
The existence of each subdir is not guaranteed for all platforms.
2021-03-12 14:50:53 +02:00
Jon Atack
c62f9bc0e9 test: use fewer blocks in wallet_groups and move sync call
- generate 101 blocks instead of 110

- move a sync_all call into the loop so that fewer blocks are
  synced on each call to hopefully see fewer timeouts
2021-03-11 17:15:05 +01:00
Wladimir J. van der Laan
92cf3a22e3 Merge #21007: bitcoind: Add -daemonwait option to wait for initialization
e017a913d0 bitcoind: Add -daemonwait option to wait for initialization (Wladimir J. van der Laan)
c3e6fdee6d shutdown: Use RAII TokenPipe in shutdown (Wladimir J. van der Laan)
612f746a8f util: Add RAII TokenPipe (Wladimir J. van der Laan)

Pull request description:

  This adds a `-daemonwait` flag that does the same as `-daemon` except that it, from a user perspective, backgrounds the process only after initialization is complete. This is similar to the behaviour of some other software such as c-lightning.

  This can be useful when the process launching bitcoind wants to guarantee that either the RPC server is running, or that initialization failed, before continuing. The exit code indicates the initialization result.

  The use of the libc function `daemon()` is replaced by a custom implementation which is inspired by the [glibc implementation](https://github.com/lattera/glibc/blob/master/misc/daemon.c#L44), but which also creates a pipe from the child to the parent process for communication.

  An additional advantage of having our own `daemon()` implementation is that no MACOS-specific pragmas are needed anymore to silence a deprecation warning.

  TODO:

  - [x] Factor out `token_read` and `token_write` to an utility, and use  them in `shutdown.cpp` as well—this is exactly the same kind of communication mechanism.

      - [x] RAII-ify pipe endpoints.

  - [x] Improve granularity of the `configure.ac` checks. This currently  still checks for the function `daemon()` which makes no sense as  it's not used. It should check for individual functions such as
    `fork()` and `setsid()` etc—the former being required, the second optional.

  - [-] ~~Signal propagation during initialization: if say, pressing Ctrl-C during `-daemonwait` it would be good to pass this SIGINT on to the child process instead of detaching the parent process and letting the child run free.~~ This is not necessary, see https://github.com/bitcoin/bitcoin/pull/21007#issuecomment-769007341.

  Future:

  - Consider if it makes sense to use this in the RPC tests (there would be no more need for "is RPC ready" polling loops). I think this is out of scope for this PR.

ACKs for top commit:
  jonatack:
    Tested ACK e017a913d0 checked change since previous review is move-only

Tree-SHA512: 53369b8ca2247e4cf3af8cb2cfd5b3399e8e0e3296423d64be987004758162a7ddc1287b01a92d7692328edcb2da4cf05d279b1b4ef61a665b71440ab6a6dbe2
2021-03-11 15:27:47 +01:00
Sjors Provoost
57ff5a42ab doc: specify minimum HWI version 2021-03-11 15:26:50 +01:00
Sjors Provoost
03308b2bfa rpc: don't require wallet for enumeratesigners 2021-03-11 15:15:14 +01:00
Wladimir J. van der Laan
e828fc8f52 Merge #21376: depends: Qt 5.12.10
550ed1bed2 build: update qt qpaint non determinism patch for 5.12.10 (fanquake)
6093ae4d30 build: update qt no-xlib patch for 5.12.10 (fanquake)
84928c4e73 build: update qt android jni static patch for 5.12.10 (fanquake)
cc6f47d51a build: update qt lrelease patch for 5.12.10 (fanquake)
286d07ff17 build, qt: Fix lib paths in *.pc files (Hennadii Stepanov)
fa5e97e8c2 build: disable qt SDK version checking (fanquake)
1be8e0f238 build: Add QMacStyle support (Hennadii Stepanov)
e674e94302 build: revert to using Qts internal zlib (fanquake)
06cd0da21f build: qt 5.12.10 (fanquake)
3272e34f9c build: Add xkbcommon 0.8.4 (Hennadii Stepanov)
d769b3372d build: only pass -optimized-tools to qt in debug mode (fanquake)

Pull request description:

  Switch to Qt 5.12.10 in depends. Based on #21363. This is a much smaller changeset, and should be easier to review than #19716. Also postpones needing to bring a bunch of new libs into depends.

  Big thanks to Hebasto that has been helping with this.

ACKs for top commit:
  laanwj:
    Code review ACK 550ed1bed2
  hebasto:
    ACK 550ed1bed2
  jarolrod:
    ACK 550ed1bed2 , tested on macOS 11.2 built from depends

Tree-SHA512: cb6b70f5a5372ba0b64f7ddfa696eda0411922cd261c67bfa2d9332c685a7b358ab18e5cfaa677b414ae8ad78296bba6ed0eecd071fdacdf736a0d030f679fe5
2021-03-11 11:51:52 +01:00
Wladimir J. van der Laan
767bb7d5c5 Merge #21270: [Bundle 4/n] Prune g_chainman usage in validation-adjacent modules
a67983cd6d net_processing: Add review-only assertion to PeerManager (Carl Dong)
272d993e75 scripted-diff: net_processing: Use existing chainman (Carl Dong)
021a04a469 net_processing: Move some static functions to PeerManager (Carl Dong)
91c5b68acd node/ifaces: ChainImpl: Use existing NodeContext member (Carl Dong)
8a1d580b21 node/ifaces: NodeImpl: Use existing NodeContext member (Carl Dong)
4cde4a701b node: Use existing NodeContext (Carl Dong)
106bcd4f39 node/coinstats: Pass in BlockManager to GetUTXOStats (Carl Dong)
2c3ba00693 miner: Pass in blockman to ::RegenerateCommitments (Carl Dong)
2afcf24408 miner: Remove old CreateNewBlock w/o chainstate param (Carl Dong)
46b7f29340 scripted-diff: Invoke CreateNewBlock with chainstate (Carl Dong)
d0de61b764 miner: Pass in chainstate to BlockAssembler::CreateNewBlock (Carl Dong)
a04aac493f validation: Remove extraneous LoadGenesisBlock function prototype (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Based on:
  - [x] #21055 | [Bundle 3/n] Prune g_chainman usage in mempool-related validation functions

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

ACKs for top commit:
  laanwj:
    Code review ACK a67983cd6d
  ryanofsky:
    Code review ACK a67983cd6d. Only change since last review new first commit fixing header declaration, and rebase
  glozow:
    code review ACK a67983cd6d

Tree-SHA512: dce182a18b88be80cbf50978d4ba8fa6ab0f01e861d09bae0ae9364051bb78f9334859d164b185b07f1d70a583e739557fab6d820cac8c37b3855b85c2a6771b
2021-03-11 11:48:55 +01:00
fanquake
1a6323bdbe doc: update developer notes for removal of MakeUnique 2021-03-11 13:45:50 +08:00
fanquake
3ba2840e7e scripted-diff: remove MakeUnique<T>()
-BEGIN VERIFY SCRIPT-
git rm src/util/memory.h
sed -i -e 's/MakeUnique/std::make_unique/g' $(git grep -l MakeUnique src)
sed -i -e '/#include <util\/memory.h>/d' $(git grep -l '#include <util/memory.h>' src)
sed -i -e '/util\/memory.h \\/d' src/Makefile.am
-END VERIFY SCRIPT-
2021-03-11 13:45:14 +08:00
Jon Atack
ccd976dd3d test: use 327 fewer blocks in feature_nulldummy
by generating 100 blocks for coinbase maturity instead of 427.

This speeds up the test and should help avoid timeout errors.
2021-03-10 22:47:59 +01:00
Jon Atack
68c280f197 test, refactor: abstract the feature_nulldummy blockheight values
Refactoring only, no change in test behavior.
2021-03-10 22:38:21 +01:00
MarcoFalke
63314b8211 Merge bitcoin-core/gui#229: Fix regression with initial sorting after pr205
c524dc54bb qt: Fix regression with initial sorting after pr205 (Hennadii Stepanov)

Pull request description:

  Unfortunately, #205 introduced a regression. After opening the "Receive" or "Transaction" tab at first time despite of the "Date" header is marked as sorted, table rows are not sorted actually:

  ![Screenshot from 2021-02-27 17-49-54](https://user-images.githubusercontent.com/32963518/109392491-f7e9a480-7924-11eb-96cc-98b6f932e18e.png)

  It appears that sorting the table must be triggered _after_ the `QTableView::setModel` call.

  With this PR (and pre-#205):

  ![Screenshot from 2021-02-27 17-48-40](https://user-images.githubusercontent.com/32963518/109392505-08018400-7925-11eb-8107-8f8685744b83.png)

ACKs for top commit:
  Talkless:
    tACK c524dc54bb, tested on Debian Sid with Qt 5.15.2. I can confirm @leonardojobim observations.
  leonardojobim:
    Tested ACK c524dc54bb on Ubuntu 20.04.2 Qt 5.12.8
  jonatack:
    ACK c524dc54bb
  jarolrod:
    ACK c524dc54bb, tested on macOS 11.1 Qt 5.15.2

Tree-SHA512: e370229979a70d63a0b64dbc11c4eca338695a070881d4d8f015644617f180e6accc24d6bdf98a75e7c9ba9be2a0ace9a2b7eb9c783ebb2992c3b2c3b3deb408
2021-03-10 17:01:12 +01:00
MarcoFalke
7cdadf91d5 Merge #21395: Net processing: Remove unused CNode.address member
a6b0fe206f [net processing] Remove unused CNode.address member (John Newbery)

Pull request description:

ACKs for top commit:
  practicalswift:
    cr ACK a6b0fe206f: patch looks correct and unused code should be removed
  fanquake:
    ACK a6b0fe206f

Tree-SHA512: 6022674dabe79be580d8005ac9e308d444d35588f324a7bb9f1ab04e8ad8ac41355c58ddfb016b001fd80a1a01ebcbddb2919ae9d33faccec2044af88547a79f
2021-03-10 10:41:37 +01:00
MarcoFalke
eea6196c3d Merge #21331: rpc: replace wallet raw pointers with references (#18592 rebased)
7c90c67b7e rpc: refactor rpc wallet functions to take references instead of pointers (fanquake)
4866934008 rpc: remove calls to CWallet.get() (fanquake)

Pull request description:

  This is a rebased #18592.

  > This PR replaces raw pointers in `rpcwallet.cpp` and `rpcdump.cpp` with **shared_ptr**. The motivation for this PR is described here https://github.com/bitcoin/bitcoin/issues/18590

  > It seems that this PR is indirectly related to this issue: https://github.com/bitcoin/bitcoin/pull/13063#discussion_r186740049

  > Notice: I have deliberately **not** changed the class `WalletRescanReserver ` whose constructor expects a raw pointer, because it's external and affects other areas, which I didn't touch to avoid making this PR "viral".

  > Fixes https://github.com/bitcoin/bitcoin/issues/18590

ACKs for top commit:
  MarcoFalke:
    ACK 7c90c67b7e 🐧
  ryanofsky:
    Code review ACK 7c90c67b7e. Changes easy to review with `--word-diff-regex=. -U0`

Tree-SHA512: 32d69c813026b02260e8a89de9d6a5ab9e87826ba230687246583ac7a80c8c3fd00318da4658f1450e04c23d2c77ae765862de0d2a110b1312b3b69a1161e7ba
2021-03-10 08:24:53 +01:00
MarcoFalke
ceb6df391f Merge #21388: doc: Rename fuzz seed_dir to corpus_dir
fad0ae6bb8 doc: Rename fuzz seed_dir to corpus_dir (MarcoFalke)

Pull request description:

  The fuzz corpus directory might contain hand-crafted seeds, but generally it is a set of test inputs. See also https://github.com/google/fuzzing/blob/master/docs/glossary.md#corpus

ACKs for top commit:
  practicalswift:
    cr ACK fad0ae6bb8: patch looks correct and "why not?" :)
  fanquake:
    ACK fad0ae6bb8 - did not test

Tree-SHA512: 38c952feb07aeeeb038b3261a12c824fab9ce5153d75f0ecf6d3f43db4f50998eeb2b14b11b7155f529189c93783fa2c11c81059021a04398c43f3505b31a2d4
2021-03-10 07:32:15 +01:00
fanquake
52f0be3a93 compat: remove memcpy -> memmove backwards compatibility alias
In glib 2.13 memcpy was changed such that the way it copied bytes was reversed.
This caused all sorts of issues for existing software, which depended on the
existing behavior (when they should have been using memmove). See:
https://sourceware.org/bugzilla/show_bug.cgi?id=12518
https://bugzilla.redhat.com/show_bug.cgi?id=638477

Now that we require glibc 2.17+ (#17538), we should be well clear of having to
maintain our memcpy ->  memmove aliasing, which was introduced in #4339.
2021-03-10 14:31:10 +08:00
fanquake
550ed1bed2 build: update qt qpaint non determinism patch for 5.12.10 2021-03-10 12:57:27 +08:00
fanquake
6093ae4d30 build: update qt no-xlib patch for 5.12.10 2021-03-10 12:57:27 +08:00
fanquake
84928c4e73 build: update qt android jni static patch for 5.12.10 2021-03-10 12:57:27 +08:00
fanquake
cc6f47d51a build: update qt lrelease patch for 5.12.10 2021-03-10 12:57:27 +08:00
Hennadii Stepanov
286d07ff17 build, qt: Fix lib paths in *.pc files
See:
- QTBUG-72903, commit 9864d2c6f3b628ca9f07a56b197e77bd43931cca
- QTBUG-78873, commit e55a61a77f0c87c05661a0335dfdb12673c6a27f

Could be dropped for Qt 5.14+.
2021-03-10 12:57:27 +08:00
fanquake
fa5e97e8c2 build: disable qt SDK version checking
This tries to invoke xcrun, which is not available when cross-compiling.
Given we are in control of the SDK versions being used, removing this
check has minimal-no effect.
2021-03-10 12:57:26 +08:00
Hennadii Stepanov
1be8e0f238 build: Add QMacStyle support 2021-03-10 12:57:26 +08:00
fanquake
e674e94302 build: revert to using Qts internal zlib 2021-03-10 12:57:26 +08:00
fanquake
06cd0da21f build: qt 5.12.10
remove fix_configure_mac.patch
Fixed upstream: https://bugreports.qt.io/browse/QTBUG-67286

remove fix_riscv64_arch.patch
Was fixed upstream in 6a39e49a6cdeb28a04a3657bb6a22f848d5dfa9d

remove fix_rcc_determinism.patch
Fixed upstream in https://bugreports.qt.io/browse/QTBUG-62511

remove freetype_back_compat.patch
By the time we ship a release with Qt 5.12, we'll certainly no-longer be
supporting Ubuntu 14.04 and Ubuntu 16.04 ships with FreeType 2.6.1,
which is new enough that using the symbol is no-longer an issue.
The renaming of FT_Get_X11_Font_Format() happened in FreeType 2.6

remove xkb-default.patch
This was removed upstream in d5abf545971da717014d316127045fc19edbcd65

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-03-10 12:57:22 +08:00
Hennadii Stepanov
3272e34f9c build: Add xkbcommon 0.8.4
Co-authored-by: fanquake <fanquake@gmail.com>
2021-03-10 08:23:47 +08:00
fanquake
d769b3372d build: only pass -optimized-tools to qt in debug mode
Qt's configure tells us that "-optimized-tools is not useful in -release
mode.", so don't use it there.
2021-03-10 08:04:04 +08:00
fanquake
bca5ee6f38 Merge #21400: build: Fix regression introduced in #21363
ffdd7de690 build, qt: Fix regression introduced in #21363 (Hennadii Stepanov)

Pull request description:

  After #21363 the `_BITCOIN_QT_CHECK_STATIC_PLUGIN` never fails due to the ill-formed code.

  Sorry for breaking it.

ACKs for top commit:
  fanquake:
    ACK ffdd7de690

Tree-SHA512: 0af3334af5332f3b349dba219565ee336090541c2c4fae53d7dc4dd2aa05291b3c7c13bfc506219f642fb854f3c887f0f9621b42d0df68d5d794e0563b2cf5d4
2021-03-10 08:03:32 +08:00
Hennadii Stepanov
ffdd7de690 build, qt: Fix regression introduced in #21363
Due to the ill-formed code _BITCOIN_QT_CHECK_STATIC_PLUGIN never fails.
2021-03-09 23:31:47 +02:00
MarcoFalke
fab633d2db doc: Update fuzzing docs for afl-clang-lto 2021-03-09 19:00:10 +01:00
MarcoFalke
ee0dc02c6f Merge #21397: fuzz: Bump FuzzedDataProvider.h
fa7dc7ae95 fuzz: Bump FuzzedDataProvider.h (MarcoFalke)

Pull request description:

  Fixes #21309

ACKs for top commit:
  practicalswift:
    cr ACK fa7dc7ae95

Tree-SHA512: 7805a943ab173d8f3f1e7e55d76a1cc60f63abf9fbf4d537bfeeb0dcf84ecdfb0417d789bd3f3a0c1603fea38884abb643b4b26c27b262e617e6c9a82894f42e
2021-03-09 13:53:36 +01:00
MarcoFalke
fa7dc7ae95 fuzz: Bump FuzzedDataProvider.h
Latest version from 0cccccf0d2/compiler-rt/include/fuzzer/FuzzedDataProvider.h
2021-03-09 12:56:44 +01:00
John Newbery
a6b0fe206f [net processing] Remove unused CNode.address member 2021-03-09 09:44:20 +00:00
fanquake
e175ca9c65 Merge #21370: Use C++11 member initializer in CNodeState
fa476f188e Use C++11 member initializer in CNodeState (MarcoFalke)

Pull request description:

  This removes a bunch of boilerplate, makes the code easier to read. Also, C++11 member initialization avoids accidental uninitialized members.

ACKs for top commit:
  practicalswift:
    cr ACK fa476f188e
  hebasto:
    cr ACK fa476f188e
  jnewbery:
    utACK fa476f188e

Tree-SHA512: 5c876717d30ded975e29bfbc77804012179588a13f950f0b2ec93fa9dbd5cf6b52fe86414fd5d1cce021db2ec77e271d533b0f7a8d6eeaac0feb9e6dbaec9ff2
2021-03-09 17:42:53 +08:00
Ivan Metlushko
2e5f7def22 wallet, rpc: update listdescriptors response format 2021-03-09 09:04:35 +01:00
fanquake
461f0c781e Merge #21382: build: Clean remnants of QTBUG-34748 fix
173ef8980d build: Small libxcb.mk improvements (Hennadii Stepanov)
5129b36573 build: Clean remnants of QTBUG-34748 fix (Hennadii Stepanov)

Pull request description:

  Hope, this PR will make [transit](https://github.com/bitcoin/bitcoin/pull/21376) to Qt 5.12.10 neater.

  A fix for [QTBUG-34748](https://bugreports.qt.io/browse/QTBUG-34748) was introduced in #5915 (v0.11.0, Qt 5.2.1).

  [QTBUG-34748](https://bugreports.qt.io/browse/QTBUG-34748) was [fixed](b19b080894) in Qt 5.3.0.

  The separated [`fix-xcb-include-order.patch`](bb44d9e754/depends/patches/qt/fix-xcb-include-order.patch), provided by #5915, was dropped in #12971 while bumping Qt to 5.9.4 (5.9.6). But `libxcb.mk` remained unchanged.

  This PR reverts #5915 for `libxcb.mk` as well.

ACKs for top commit:
  practicalswift:
    cr ACK 173ef8980d: patch looks correct
  fanquake:
    ACK 173ef8980d

Tree-SHA512: 9815a7e532ff4aa08f9623ded8d5708eca1c9c73ac7a2684419a18c125da7627b44ac3191f2e7978946942c8d0580e73b1a93df624986fb2a13791a68ce1e025
2021-03-09 15:46:57 +08:00
fanquake
738fc4a595 Merge #21363: build, qt: Improve Qt static plugins/libs check code
32b8dc648f build, refactor: Fix indentation (Hennadii Stepanov)
57b65225dc build, qt: Make Qt static libs check regardless of plugindir (Hennadii Stepanov)
6203457915 build, refactor: Rename internal _BITCOIN_QT_FIND_STATIC_PLUGINS macro (Hennadii Stepanov)
bf2477402d build, qt: Refactor internal _BITCOIN_QT_CHECK_STATIC_PLUGINS macro (Hennadii Stepanov)

Pull request description:

  Qt version bump is around the corner (#21376).

  As a preparation, this PR makes Qt static plugins/libs check code more correct, and easier to reason about.

  `configure` script log (for `HOST=x86_64-pc-linux-gnu`):
  - master (b4d22654fe):
  ```
  checking for static Qt plugins: -lqminimal... yes
  checking for static Qt plugins: -lqxcb -lxcb-static... yes
  ```

  - this PR:
  ```
  checking for QMinimalIntegrationPlugin (-lqminimal)... yes
  checking for QXcbIntegrationPlugin (-lqxcb -lxcb-static)... yes
  ```

ACKs for top commit:
  fanquake:
    ACK 32b8dc648f

Tree-SHA512: a9ed8178cdbe3586898adc145b474355804ca00c5cee85dcdb8fb33a16575fc469298ef532c973d3aae53e1cb49569a9232595d609f232fae2bf05862fb34781
2021-03-09 15:34:19 +08:00
MarcoFalke
6c156e49cb Merge #18842: wallet: Mark replaced tx to not be in the mempool anymore
fa4e088cba wallet: Mark replaced tx to not be in the mempool anymore (MarcoFalke)

Pull request description:

  The wallet does not mark the replaced tx as out-of-mempool. This causes failures in user scripts, because later RPCs may depend on this state change from `bumpfee`.

  For example, the following might fail on current master:

  ```
  txid = sendtoaddress(...)
  bumpfee(txid)
  abandontransaction(txid)  # fails because txid is still marked as "in mempool"
  ```

  Fixes #18831

ACKs for top commit:
  meshcollider:
    utACK fa4e088cba
  ryanofsky:
    Code review ACK fa4e088cba, and previous ACK faeedff5c87091fd83d2fb2b29eb49c948363f29 is also still valid in case there's a preference for the original fix

Tree-SHA512: 9858f40f5fb5a43a7b584b5c4268b6befa82e6a84583be5206fe721bcb6c255e8d35479d347d0b9aed72703df49887c02b14ab680e8efdd28b90dd6b93d9439a
2021-03-09 07:54:18 +01:00
Amiti Uttarwar
ebde946a52 [doc] Improve comment about protected peers
The comment currently suggests a long-standing node would infrequently protect
peers under normal circumstances. Clarify that we also protect peers that are
synced to the same work as our chain tip.
2021-03-08 18:17:48 -08:00
practicalswift
e528075189 tests: Add fuzzing harness for Lookup(...)/LookupHost(...)/LookupNumeric(...)/LookupSubNet(...) 2021-03-08 23:17:56 +00:00
practicalswift
c6b4bfb4b3 net: Make DNS lookup code testable 2021-03-08 23:17:56 +00:00
Samuel Dobson
a8b0892b74 Merge #20536: wallet: Error with "Transaction too large" if the funded tx will end up being too large after signing
48a0319bab Add a test that selects too large if BnB is used (Andrew Chow)
3e69939b78 Fail if maximum weight is too large (Andrew Chow)
51e2cd322c Have CalculateMaximumSignedTxSize also compute tx weight (Andrew Chow)

Pull request description:

  Currently the `Transaction too large` is calculated on the transaction that is returned from `CreateTransaction`. This does not make sense for when `CreateTransaction` is being used for `fundrawtransaction` as no signing occurs so the final returned transaction is missing signatures. Thus users may successfully fund a transaction but fail to broadcast it after it has been fully signed.

  So instead we should figure out whether the transaction we are funding will be too large after it is signed. We can do this by having `CalculateMaximumSignedTxSize` also return the transaction weight and then comparing that weight against the maximum weight.

ACKs for top commit:
  instagibbs:
    ACK 48a0319bab
  meshcollider:
    utACK 48a0319bab
  Xekyo:
    utACK with nits 48a0319bab

Tree-SHA512: 1700c60b07f67e2d5c591c5ccd131ac9f1861fab3def961c3c9c4b3281ec1063fe8e4f0f7f1038cac72692340856406bcee8fb45c8104d2ad34357a0ec878ac7
2021-03-09 10:42:21 +13:00
Carl Dong
a67983cd6d net_processing: Add review-only assertion to PeerManager 2021-03-08 15:54:31 -05:00
Carl Dong
272d993e75 scripted-diff: net_processing: Use existing chainman
-BEGIN VERIFY SCRIPT-
sed -i -E \
    -e 's/g_chainman/m_chainman/g' \
    -e 's@([^:])(Chain(state|)Active)@\1::\2@g' \
    -e 's@::Chain(state|)Active\(\)@m_chainman.ActiveChain\1()@g' \
    -- src/net_processing.cpp
-END VERIFY SCRIPT-
2021-03-08 15:54:31 -05:00
Carl Dong
021a04a469 net_processing: Move some static functions to PeerManager
- BlockRequestAllowed
- AlreadyHaveBlock
- ProcessGetBlockData
- PrepareBlockFilterRequest
- ProcessGetCFilters
- ProcessGetCFHeaders
- ProcessGetCFCheckPt

Moved out of anonymous namespace:
- ProcessBlockAvailability
- UpdateBlockAvailability
- CanDirectFetch
2021-03-08 15:54:31 -05:00
Carl Dong
91c5b68acd node/ifaces: ChainImpl: Use existing NodeContext member 2021-03-08 15:54:31 -05:00
Carl Dong
8a1d580b21 node/ifaces: NodeImpl: Use existing NodeContext member 2021-03-08 15:54:31 -05:00
Carl Dong
4cde4a701b node: Use existing NodeContext 2021-03-08 15:54:31 -05:00
Carl Dong
106bcd4f39 node/coinstats: Pass in BlockManager to GetUTXOStats 2021-03-08 15:54:31 -05:00
Carl Dong
2c3ba00693 miner: Pass in blockman to ::RegenerateCommitments
REQUIRES ATTENTION
2021-03-08 15:54:31 -05:00
Carl Dong
2afcf24408 miner: Remove old CreateNewBlock w/o chainstate param 2021-03-08 15:54:31 -05:00
Carl Dong
46b7f29340 scripted-diff: Invoke CreateNewBlock with chainstate
-BEGIN VERIFY SCRIPT-
find_regex='(\.|->)CreateNewBlock\(' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/miner\.\(cpp\|h\)$' \
        | xargs sed -i -E 's@'"$find_regex"'@\0::ChainstateActive(), @g'
-END VERIFY SCRIPT-
2021-03-08 15:54:31 -05:00
Carl Dong
d0de61b764 miner: Pass in chainstate to BlockAssembler::CreateNewBlock 2021-03-08 15:54:31 -05:00
Carl Dong
a04aac493f validation: Remove extraneous LoadGenesisBlock function prototype
Leftover from last bundle.
2021-03-08 15:54:21 -05:00
MarcoFalke
2067f9e5e8 Merge bitcoin-core/gui#233: qt test: Don't bind to regtest port
e21276a82a qt test: Don't bind to regtest port (Andrew Chow)

Pull request description:

  The qt tests don't need to bind to the regtest port. By not binding, it will no longer conflict with existing regtest instances and the tests will run as normal.

  Fixes #10

ACKs for top commit:
  MarcoFalke:
    cr ACK e21276a82a
  jarolrod:
    re-ACK e21276a82a, tested on macOS 11.2

Tree-SHA512: 5a269ee043f9aff7900e092c166de71912a2bf86ebe2982b3fb0e26bdebfb91869ee5d0f62082fd608c1288bfb7981f6c8647e504b11176711d7fec993a09164
2021-03-08 19:52:25 +01:00
MarcoFalke
fad0ae6bb8 doc: Rename fuzz seed_dir to corpus_dir 2021-03-08 15:23:51 +01:00
MarcoFalke
a22653a636 Merge #21371: fuzz: fix gcc Woverloaded-virtual build warnings
36aa2955b8 fuzz: fix gcc Woverloaded-virtual build warnings (Jon Atack)

Pull request description:

  Possible fixup to gcc build warnings since merge of b22d4c1607. Closes #21369.

ACKs for top commit:
  practicalswift:
    cr ACK 36aa2955b8: patch looks correct
  achow101:
    ACK 36aa2955b8
  kristapsk:
    ACK 36aa2955b8, this fixes compiler warnings for me with GCC 9.3.0.

Tree-SHA512: b6c99690ff72b809ce8105696744546252691b618f54311a9d930d9975fc692071ef408450f618fbb4aa99ee5390028a6eabbc968e22b2e8d2bd56bbafef49f8
2021-03-08 08:23:10 +01:00
MarcoFalke
e797388f8b Merge #21384: doc: add signet to bitcoin.conf documentation
4a285107c1 doc: add signet to doc/bitcoin-conf.md (Jon Atack)
21b6a23373 doc: add signet to share/examples/bitcoin.conf (Jon Atack)

Pull request description:

ACKs for top commit:
  jarolrod:
    ACK 4a285107c1 🥃
  kristapsk:
    ACK 4a285107c1

Tree-SHA512: 2c2aa58ce5316cf986a1f3b7638b42d3939a482c38becb346d6aeab2887e57adf5c0e961de33f7721c89936881769e76b46504445428737ee810d21f4d9fb1a4
2021-03-08 08:15:11 +01:00
Jon Atack
4a285107c1 doc: add signet to doc/bitcoin-conf.md 2021-03-08 00:44:54 +01:00
Jon Atack
21b6a23373 doc: add signet to share/examples/bitcoin.conf 2021-03-08 00:22:57 +01:00
Hennadii Stepanov
173ef8980d build: Small libxcb.mk improvements 2021-03-07 23:17:28 +02:00
Hennadii Stepanov
5129b36573 build: Clean remnants of QTBUG-34748 fix
A fix for QTBUG-34748 was introduced in #5915 (v0.11.0, Qt 5.2.1).
QTBUG-34748 was fixed in version 5.3.0.
The separated patch file, provided by #5915, was dropped in #12971 while
bumping Qt to 5.9.4 (5.9.6). But libxcb.mk remained unchanged.

This change reverts #5915 for libxcb.mk.
2021-03-07 23:03:38 +02:00
Jon Atack
3a16b5ef95 test: add missing logging to wallet_groups.py 2021-03-07 20:25:11 +01:00
MarcoFalke
1a4a9305c2 Merge bitcoin-core/gui#221: qt, refactor: rpcconsole translatable string fixes and improvements
6242beeb06 Hoist repeated translated strings to RPCConsole struct members (Jon Atack)
0f035c12fb RPCConsole::updateDetailWidget: convert strings to translated strings (Jon Atack)

Pull request description:

  - fixups from #206 review feedback (thanks!), see commit message for details
  - hoists repeatedly used translatable strings to the `RPCConsole` class for reuse

ACKs for top commit:
  hebasto:
    re-ACK 6242beeb06
  Talkless:
    tACK 6242beeb06, tested on Debian Sid with Qt 5.15.2. I see "Ban for.." translated to my native language as before, "To/From/Yes/No" are not but that's expected, as `.ts` files are not updated.
  jarolrod:
    ACK 6242beeb06

Tree-SHA512: 20a296511c5ac03a816766237fa2731b0360dedebf1bea02711eb21d7e4eae2a63a051fe48f4726052edc3e6318952f01fef920cd4b22a8196c39c23d8e5cc3a
2021-03-07 18:56:23 +01:00
Hennadii Stepanov
5a4a15d2b4 qt, refactor: Drop no longer used PeerTableModel::getRowByNodeId func 2021-03-07 16:58:11 +02:00
Hennadii Stepanov
9a9f180df0 qt, refactor: Drop no longer used PeerTableModel::sort function 2021-03-07 16:58:11 +02:00
Hennadii Stepanov
778a64af20 qt: Use PeerTableSortProxy for sorting peer table 2021-03-07 16:57:17 +02:00
Hennadii Stepanov
df2d165ba9 qt: Add peertablesortproxy module 2021-03-07 16:57:11 +02:00
MarcoFalke
8c049fe9af Merge #19771: net: Replace enum CConnMan::NumConnections with enum class ConnectionDirection
c77de622dd net: Replace enum CConnMan::NumConnections with enum class ConnectionDirection (Luke Dashjr)

Pull request description:

  Refactor split out of #17167

ACKs for top commit:
  practicalswift:
    cr ACK c77de622dd: patch looks correct & `enum class` is strictly better

Tree-SHA512: 40a1bf69d8ab2651b04ba6adbab789369a5a1a29a64ba764c3e6aab575b7943ea8dfd6e35b0abf5bcffa10e7265f4b523a93aa899c0fd581a84fc51ae5377b90
2021-03-07 14:21:02 +01:00
MarcoFalke
8c21562180 Merge bitcoin-core/gui#166: refactor: Use enum type as switch argument in *TableModel
1d5d832d5c qt, refactor: Use enum type as switch argument in TransactionTableModel (Hennadii Stepanov)
52f122c11f qt, refactor: Use enum type as switch argument in PeerTableModel (Hennadii Stepanov)
a35223f1cd qt, refactor: Use enum type as switch argument in BanTableModel (Hennadii Stepanov)
ab8a747d1c qt, refactor: Use enum type as switch argument in AddressTableModel (Hennadii Stepanov)

Pull request description:

  This PR makes code more maintainable by leveraging `-Wswitch` compiler warnings.

  Only the `RecentRequestsTableModel` is not refactored, because its `enum ColumnIndex` contains additional `NUMBER_OF_COLUMNS` value.

  No behavior change.

ACKs for top commit:
  hebasto:
    Do you mind mentioning the _top_ pr commit with your ACK, i.e., 1d5d832d5c, not ab8a747d1ced9f20ca32f9898418be70670da71a?
  jarolrod:
    ACK 1d5d832d5c, tested on macOS 11.1 Qt 5.15.2
  leonardojobim:
    ACK 1d5d832d5c, tested on Ubuntu 20.04 Qt 5.12.8
  promag:
    Code review ACK 1d5d832d5c.

Tree-SHA512: 0d474d226a2fa0069495d1aa5ec13b2470708ec7b8a6ab35402236c7bf57cb9939577677a30dfa54f5e3bc0477c6cfffd20ed6f19e4eb394a938569cc9347851
2021-03-07 13:04:30 +01:00
MarcoFalke
1020b04c39 Merge #21334: test: Additional (refactored) BIP9 tests
0c471a5f30 tests: check never active versionbits (Anthony Towns)
3ba9283a47 tests: more helpful errors for failing versionbits tests (Anthony Towns)

Pull request description:

  Extracted from #19573 to make review easier. I also reviewed it myself.

  I added some comments to the test: bae9a45219 (r585486781)

  I also moved some `TestState` changes from the second to the first commit, to reduce the latter diff.

ACKs for top commit:
  ajtowns:
    ACK 0c471a5f30
  MarcoFalke:
    review ACK 0c471a5f30 🔓

Tree-SHA512: 61f8d1ecaf38a6cd13db1cf71c89b8c4d2f5852ef77c5e7ecb9bd78eb216545037411641bb101cf0740c5c47845ac327954ee25b676d63779c5f148719ac5caf
2021-03-07 12:47:50 +01:00
MarcoFalke
fa4e088cba wallet: Mark replaced tx to not be in the mempool anymore 2021-03-07 12:18:15 +01:00
Andrew Chow
e21276a82a qt test: Don't bind to regtest port
The qt tests don't need to bind to the regtest port. By not binding, it
will no longer conflict with existing regtest instances and the tests
will run as normal.
2021-03-06 15:19:08 -05:00
Hennadii Stepanov
88df300f20 qt: Do not translate file extensions 2021-03-06 20:49:46 +02:00
fanquake
48725e64fb Merge #21209: build: use newer source for libnatpmp
7af25024e9 build: compile libnatpmp with -DNATPMP_STATICLIB on Windows (fanquake)
ee35745754 build: use newer source for libnatpmp (fanquake)

Pull request description:

  The source we are currently using is from 2015. The upstream repo has
  received a small number of bug fixes and improvements since then.
  Including one that fixes an issue for Windows users:  https://github.com/miniupnp/libnatpmp/pull/13.

  The source we are currently using is the most recent "official" release,
  however I don't think it's worth waiting for a new one. The maintainer
  was prompted to do so in Oct 2020, then again in Jan of this year, and
  no release has eventuated. Given libnatpmp is a new inclusion into our
  repository, I think we should be using this newer source.

  This also cleans up a few warnings we currently see in Windows depends builds:
  ```bash
  Extracting libnatpmp...
  /home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK
  Preprocessing libnatpmp...
  Configuring libnatpmp...
  Building libnatpmp...
  make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
  x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o natpmp.o natpmp.c
  x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o getgateway.o getgateway.c
  natpmp.c:42: warning: "EWOULDBLOCK" redefined
     42 | #define EWOULDBLOCK WSAEWOULDBLOCK
        |
  In file included from natpmp.c:38:
  /usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition
    166 | #define EWOULDBLOCK 140
        |
  natpmp.c:43: warning: "ECONNREFUSED" redefined
     43 | #define ECONNREFUSED WSAECONNREFUSED
        |
  In file included from natpmp.c:38:
  /usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition
    110 | #define ECONNREFUSED 107
        |
  natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
    271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response)
        |     ^~~~~~~~~~~~~~~~~~~~~~~~~
  ar crs libnatpmp.a natpmp.o getgateway.o
  make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
  Staging libnatpmp...
  Postprocessing libnatpmp...
  Caching libnatpmp...
  ```

ACKs for top commit:
  hebasto:
    ACK 7af25024e9

Tree-SHA512: 6939014ea986149a5bfdd42b516d563a65ae643516e234579d3f28e7c2f877b0270cc4305ae7c7cb131d6d946a6e0aedc84b4cc880a412612a878a333398b9d7
2021-03-06 09:20:03 +08:00
Jon Atack
36aa2955b8 fuzz: fix gcc Woverloaded-virtual build warnings 2021-03-05 22:30:00 +01:00
MarcoFalke
fa476f188e Use C++11 member initializer in CNodeState 2021-03-05 11:46:13 +01:00
MarcoFalke
ed25cb58f6 Merge bitcoin-core/gui#217: qt: Make warning label look clickable
67c59ae479 qt: Make warning label look clickable (Jarol Rodriguez)

Pull request description:

  The warning icon on the overview page indicates that there is something important the user should know about, but a user may not be aware that they can click it because, on `master`, the warning label does not look clickable. As detailed in issue #23, the reason to make it look clickable is that it if they "had a more clickable-appearance (borders or beveled button edges) it could help users more quickly understand what they are being alerted to."

  This PR removes the `flat` property from both `QPushButton`'s to make them look like a button, and therefore clickable. Furthermore, it updates the `Maximum Width` to `45` to fix the small hit-box issue outlined in issue #215.

  Below are screenshots showing how the warning icon looks under `master` and this `PR`:

  **macOS 11.1: Qt 5.15**
  | Master        | PR               |
  | ----------- | ----------- |
  |  <img width="754" alt="Screen Shot 2021-02-22 at 5 00 40 PM" src="https://user-images.githubusercontent.com/23396902/108776135-f6d50380-752f-11eb-9f96-25163c6a2a02.png"> | <img width="754" alt="Screen Shot 2021-02-22 at 3 08 40 PM" src="https://user-images.githubusercontent.com/23396902/108776068-e0c74300-752f-11eb-9545-3580e2b8f187.png"> |

  **Ubuntu 20.04: Qt 5.12**

  | Master        | PR               |
  | ----------- | ----------- |
  | <img width="783" alt="Screen Shot 2021-02-22 at 4 57 32 PM" src="https://user-images.githubusercontent.com/23396902/108776249-284dcf00-7530-11eb-8325-7fe13a9243a7.png"> |   ![Screen Shot 2021-02-22 at 4 12 54 PM](https://user-images.githubusercontent.com/23396902/108776428-60eda880-7530-11eb-8999-59ddd70de85f.png) |

  Closes #23
  Closes #215

ACKs for top commit:
  Talkless:
    tACK 67c59ae479, tested on Debian Sid. Does look as expected.

Tree-SHA512: 2b7302fb990ea49e2f01df6f4a23e2bc3de0797da89deaeb299742e6b285a0c21ea80d8259dc0222640cccc2bccc4ea09df443b9a11bf8b88a828e5fb2aec12c
2021-03-05 08:20:44 +01:00
MarcoFalke
f7653eb5ae Merge #21345: test: bring p2p_leak.py up to date
a061a29970 test: bring p2p_leak.py up to date. (Martin Zumsande)

Pull request description:

  After the introduction of wtxidrelay and sendaddrv2 messages during version handshake, extend p2p_leak.py test to reflect this.
  Also, some minor fixes and doc improvements.

  I also added a test that peers not completing the version handshake will be disconnected for timeout, as suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/19723#issuecomment-699540294.

ACKs for top commit:
  brunoerg:
    Tested ACK a061a29970
  theStack:
    Tested ACK a061a29970

Tree-SHA512: 26c601491fa8710fc972d1b8f15da6b387a95b42bbfb629ec4c668769ad3824b6dd6a33d97363bca2171e403d8d1ce08abf3e5c9cab34f98d53e0109b1c7a0e5
2021-03-05 08:18:42 +01:00
Hennadii Stepanov
32b8dc648f build, refactor: Fix indentation 2021-03-05 09:11:13 +02:00
Hennadii Stepanov
57b65225dc build, qt: Make Qt static libs check regardless of plugindir
Qt static libs reside in libdir.
2021-03-05 09:11:12 +02:00
Hennadii Stepanov
6203457915 build, refactor: Rename internal _BITCOIN_QT_FIND_STATIC_PLUGINS macro
New _BITCOIN_QT_CHECK_STATIC_LIBS name describes the macro functionality
more precisely.
2021-03-05 09:11:12 +02:00
Hennadii Stepanov
bf2477402d build, qt: Refactor internal _BITCOIN_QT_CHECK_STATIC_PLUGINS macro
This change puts Q_IMPORT_PLUGIN(...) boilerplate into the macro, which
now accepts only one plugin to check, and it is renamed (plural ->
singular).
2021-03-05 09:10:56 +02:00
Andrew Chow
a97a9298ce Test that signrawtx works when a signed CSV and CLTV inputs are present 2021-03-04 21:54:34 -05:00
Andrew Chow
6965456c10 Introduce DeferringSignatureChecker and inherit with SignatureExtractor
Introduces a DeferringSignatureChecker which simply takes a
BaseSignatureChecker and passes through everything.
SignatureExtractorChecker now subclasses DeferringSignatureChecker. This
allows for all BaseSignatureChecker functions to be implemented for
SignatureExtractorChecker, while allowing for future signature checkers
which opreate similarly to SignatureExtractorChecker.
2021-03-04 21:54:32 -05:00
fanquake
da8c7edffe Merge #21364: fuzz: Avoid -Wreturn-type warnings
3f3646855c fuzz: Avoid -Wreturn-type warnings (practicalswift)

Pull request description:

  Avoid `-Wreturn-type` warnings.

  Closes #21355.

ACKs for top commit:
  MarcoFalke:
    cr ACK 3f3646855c
  fanquake:
    ACK 3f3646855c - thanks for cleaning this up.

Tree-SHA512: 6fa2640a26e64d2bea60e016ad14b5c434137fedc0b3bf2ac244f02f9b1cd303d1ebac4ac4e6791534560f8311c4cbe9395c2ce94d7ec022d3b192f1ea070809
2021-03-05 10:42:38 +08:00
fanquake
fbf5d16238 Merge #21246: doc: Correction for VerifyTaprootCommitment comments
6a0a6e7d05 Correction for VerifyTaprootCommitment comments (Russell O'Connor)

Pull request description:

  According to BIP-341, 'p' is called the taproot *internal* key, not inner key.

ACKs for top commit:
  sipa:
    ACK 6a0a6e7d05
  benthecarman:
    ACK 6a0a6e7d05
  theStack:
    ACK 6a0a6e7d05

Tree-SHA512: 94f553476a8404bff4b2d5724a1a54c5f530b987a616cd00a3800095f245c06e3c7a9066c729976f32069a56029406859a70ba523151d333dc1ed874f242bce8
2021-03-05 10:30:33 +08:00
fanquake
2620ac4ec3 Merge #21354: build, doc: Drop no longer required packages from macOS cross-compiling dependencies
f7f3829a68 build, doc: Drop libbz2-dev from macOS cross-compiling dependencies (Hennadii Stepanov)
d8239362e2 build, doc: Drop libcap-dev from macOS cross-compiling dependencies (Hennadii Stepanov)

Pull request description:

  The `libcap-dev` and `libbz2-dev` packages are no longer required when cross-compiling for macOS.

ACKs for top commit:
  fanquake:
    ACK f7f3829a68

Tree-SHA512: 820cdc2724f3346c0942d4d4115fc7206f7bf02889d9fa6cbdbd1d9e3afa03a067c1c3fa64dff596aefdc74898178b7c7d64027a6501486e3b606f4760de04ae
2021-03-05 09:41:33 +08:00
fanquake
7c90c67b7e rpc: refactor rpc wallet functions to take references instead of pointers
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-03-05 09:20:13 +08:00
fanquake
4866934008 rpc: remove calls to CWallet.get()
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-03-05 09:05:37 +08:00
practicalswift
3f3646855c fuzz: Avoid -Wreturn-type warnings 2021-03-04 20:29:28 +00:00
Luke Dashjr
c77de622dd net: Replace enum CConnMan::NumConnections with enum class ConnectionDirection 2021-03-04 19:54:17 +00:00
Wladimir J. van der Laan
e017a913d0 bitcoind: Add -daemonwait option to wait for initialization
This adds a `-daemonwait` flag that does the same as `-daemon` except
it, from a user perspective, backgrounds the process only after
initialization is complete.

This can be useful when the process launching bitcoind wants to
guarantee that either the RPC server is running, or that initialization
failed, before continuing. The exit code indicates the initialization
result.

This replaces the use of the libc function `daemon()` by a custom
implementation which is inspired by the glibc implementation, but also
creates a pipe from the child to the parent process for communication.

An additional advantage of having our own `daemon()` implementation is
that no MACOS-specific pragmas are needed anymore to silence a
deprecation warning.
2021-03-04 18:24:00 +01:00
Wladimir J. van der Laan
c3e6fdee6d shutdown: Use RAII TokenPipe in shutdown 2021-03-04 18:24:00 +01:00
Wladimir J. van der Laan
612f746a8f util: Add RAII TokenPipe 2021-03-04 18:24:00 +01:00
Wladimir J. van der Laan
b4d22654fe Merge #21337: guix: Update darwin native packages dependencies
c967fb7fb9 guix: Remove libcap from manifest (Hennadii Stepanov)
7bbb409314 guix: Update darwin native packages dependencies (Hennadii Stepanov)

Pull request description:

  It is a #20470 follow up.

ACKs for top commit:
  fanquake:
    ACK c967fb7fb9

Tree-SHA512: 66ce05770f578ba61a44c58747c5a2669f425a989ed987838058bd86e3b49e342ac5a4f8852fc49f2b3a86b58fb6a340fdf3e34c1fc19bdab910729febba4bc7
2021-03-04 16:01:25 +01:00
Wladimir J. van der Laan
702cfc8c53 Merge #21055: [Bundle 3/n] Prune remaining g_chainman usage in validation functions
e11b649650 validation: CVerifyDB::VerifyDB: Use locking annotation (Carl Dong)
03f75c42e1 validation: Use existing chain member in CChainState::LoadGenesisBlock (Carl Dong)
5e4af77380 validation: Use existing chain member in CChainState::AcceptBlock (Carl Dong)
fee73347c0 validation: Pass in chain to FindBlockPos+SaveBlockToDisk (Carl Dong)
a9d28bcd8d validation: Use *this in CChainState::ActivateBestChainStep (Carl Dong)
4744efc9ba validation: Pass in chainstate to CTxMemPool::check (Carl Dong)
1fb7b2c595 validation: Use *this in CChainState::InvalidateBlock (Carl Dong)
8cdb2f7e58 validation: Move LoadBlockIndexDB to CChainState (Carl Dong)
8b99efbcc0 validation: Move invalid block handling to CChainState (Carl Dong)
2bdf37fe18 validation: Pass in chainstate to CVerifyDB::VerifyDB (Carl Dong)
31eac50c72 validation: Remove global ::VersionBitsTip{State,SinceHeight,Statistics} (Carl Dong)
63e4c7316a validation: Pass in chainstate to ::PruneBlockFilesManual (Carl Dong)
4bada76237 validation: Pass in chainstate to UpdateTip (Carl Dong)
a3ba08ba7d validation: Remove global ::{{Precious,Invalidate}Block,ResetBlockFailureFlags} (Carl Dong)
4927c9e699 validation: Remove global ::LoadGenesisBlock (Carl Dong)
9da106be4d validation: Check chain tip is non-null in CheckFinalTx (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Based on:
  - [x] #20750 | [Bundle 2/n] Prune g_chainman usage in mempool-related validation functions

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

  Note to self:
  - [x] Address: https://github.com/bitcoin/bitcoin/pull/20750#discussion_r579400663

ACKs for top commit:
  laanwj:
    Code review ACK e11b649650

Tree-SHA512: 205a451a741e32f17d5966de289f2f5a3f0817738c0087b70ff4755ddd217b53d01050ed396669bda2b1d216a88d927b9778777f9ff95ab1fe20e59c5f341776
2021-03-04 14:55:47 +01:00
fanquake
83bdbbd300 Merge #21003: test: Move MakeNoLogFileContext to libtest_util, and use it in bench
fa576b4532 Move MakeNoLogFileContext to common libtest_util, and use it in bench (MarcoFalke)

Pull request description:

  To avoid verbose code duplication, which may lead to accidental mishaps https://github.com/bitcoin/bitcoin/pull/20998/files#r563624041.

  Also fix a nit I found in https://github.com/bitcoin/bitcoin/pull/20946#discussion_r561949731.

ACKs for top commit:
  dongcarl:
    Light Code-Review ACK fa576b4532
  fanquake:
    ACK fa576b4532

Tree-SHA512: d39ac9c0957813ebb20ed13bd25a8ef8469377ce2651245638bd761c796feac2a17e30dd16f1e5c8db57737fb918c81d56a3d784c33258275e426a1b11e69eb2
2021-03-04 20:29:09 +08:00
fanquake
7450a01691 Merge #21358: fuzz: Add missing include (test/util/setup_common.h)
fa59ad5130 fuzz: Add missing include (test/util/setup_common.h) (MarcoFalke)

Pull request description:

  `src/test/fuzz/socks5.cpp` is using the symbol `BasicTestingSetup`, which is defined in `src/test/util/setup_common.h`.

  Currently compilation happens to succeed because the needed dependency is indirectly included. Compilation will break as soon as the indirect dependency is broken. According to the dev notes, everything that is used must be included.

  Fix the issue by including the missing include.

ACKs for top commit:
  fanquake:
    ACK fa59ad5130

Tree-SHA512: 9359d5d288ebc5a53d753ebed1ee8d49ddcfe12aeb56054ea43654c0d915337bb0dce7c8a7178e94711ff8dacd1b3ea0a2871b21b1709cd9786efc0c1ef532b3
2021-03-04 20:20:43 +08:00
fanquake
33921379b6 Merge #21015: Make all of net_processing (and some of net) use std::chrono types
0eaea66e8b Make tx relay data structure use std::chrono types (Pieter Wuille)
55e82881a1 Make all Poisson delays use std::chrono types (Pieter Wuille)
c733ac4d8a Convert block/header sync timeouts to std::chrono types (Pieter Wuille)
4d98b401fb Change all ping times to std::chrono types (Pieter Wuille)

Pull request description:

  (Picking up #20044. Rebased against master.)

  This changes various uses of integers to represent timestamps and durations to `std::chrono` duration types with type-safe conversions, getting rid of various `.count()`, constructors, and conversion factors.

ACKs for top commit:
  jnewbery:
    utACK 0eaea66e8b
  vasild:
    ACK 0eaea66e8b
  MarcoFalke:
    re-ACK 0eaea66e8b, only changes: minor rename, using C++11 member initializer, using 2min chrono literal, rebase 🤚
  ajtowns:
    utACK 0eaea66e8b

Tree-SHA512: 2dbd8d53bf82e98f9b4611e61dc14c448e8957d1a02575b837fadfd59f80e98614d0ccf890fc351f960ade76a6fb8051b282e252e81675a8ee753dba8b1d7f57
2021-03-04 20:13:43 +08:00
MarcoFalke
fa4cebadcf util: Make Assume() usable as unary expression 2021-03-04 12:03:50 +01:00
John Newbery
680eb56d82 [net processing] Don't pass CConnman to RelayTransactions
Use the local m_connman instead
2021-03-04 10:22:57 +00:00
John Newbery
a38a4e8f03 [net processing] Move RelayTransaction into PeerManager
We don't mark RelayTransaction as const. Even though it doesn't mutate
PeerManagerImpl state, it _is_ mutating the internal state of a CNode
object, by updating setInventoryTxToSend. In a subsequent commit, that
field will be moved to the Peer object, which is owned by
PeerMangerImpl.

This requires PeerManagerImpl::ReattemptInitialBroadcast() to no longer
be const.
2021-03-04 10:22:42 +00:00
Wladimir J. van der Laan
92b7efcf54 Merge #21148: Split orphan handling from net_processing into txorphanage
5e50e2d1b9 txorphanage: comment improvements (Anthony Towns)
eeeafb324e net_processing: move AddToCompactExtraTransactions into PeerManagerImpl (Anthony Towns)
f8c0688b94 scripted-diff: Update txorphanage naming convention (Anthony Towns)
6bd4963c06 txorphanage: Move functions and data into class (Anthony Towns)
03257b832d txorphanage: Extract EraseOrphansForBlock (Anthony Towns)
3c4c3c2fdd net_processing: drop AddOrphanTx (Anthony Towns)
26d1a6ccd5 denialofservices_tests: check txorphanage's AddTx (Anthony Towns)
1041616d7e txorphanage: Extract OrphanageAddTx (Anthony Towns)
f294da7274 txorphanage: Extract GetOrphanTx (Anthony Towns)
83679ffc60 txorphanage: Extract HaveOrphanTx (Anthony Towns)
ee135c8d5b txorphanage: Extract AddChildrenToWorkSet (Anthony Towns)
38a11c355a txorphanage: Add lock annotations (Anthony Towns)
81dd57e5b1 txorphanage: Pass uint256 by reference instead of value (Anthony Towns)
9d5313df7e move-only: Add txorphanage module (Anthony Towns)

Pull request description:

  Splits orphan handling into its own module and reduces global usage.

ACKs for top commit:
  jnewbery:
    utACK 5e50e2d1b9
  amitiuttarwar:
    utACK 5e50e2d1b9
  glozow:
    re ACK 5e50e2d1b9, comment updates
  laanwj:
    Code review ACK 5e50e2d1b9

Tree-SHA512: 92a959bb5dd414c96f78cb8dcaa68adb85faf16b8b843a2cbe0bb2aa08df13ad6bd9424d29b98f57a82ec29c942fbdbea3011883d00bf0b0feb643e295174e46
2021-03-04 10:16:38 +01:00
MarcoFalke
fa59ad5130 fuzz: Add missing include (test/util/setup_common.h) 2021-03-04 08:39:27 +01:00
MarcoFalke
d099894ec1 Merge #20969: test: check that getblockfilter RPC fails without block filter index
233a886b42 test: check that getblockfilter RPC fails without block filter index (Sebastian Falbesoner)

Pull request description:

  If a node was started without compact block filter index (parameter `--blockfilterindex=0`), the `getblockfilter` RPC call should fail.

ACKs for top commit:
  MarcoFalke:
    review ACK 233a886b42

Tree-SHA512: c8824373fad7d1de2dcb43c1d9541d736b478235be243080d2b7479c2588eac0e5722337ec1307394b331e0002fbcabb368e4955c2dc98dd5fce76d8c089e8a1
2021-03-04 08:37:10 +01:00
fanquake
7af25024e9 build: compile libnatpmp with -DNATPMP_STATICLIB on Windows
This fixes linking issues and mirrors what we do with miniupnpc.
2021-03-04 12:34:46 +08:00
fanquake
ee35745754 build: use newer source for libnatpmp
The source we are currently using is from 2015. The upstream repo has
received a small number of bug fixes and improvements since then.
Including one that fixes an issue for Windows users:
https://github.com/miniupnp/libnatpmp/pull/13.

The source we are currently using is the most recent "official" release,
however I don't think it's worth waiting for a new one. The maintainer
was prompted to do so in Oct 2020, then again in Jan of this year, and
no release has eventuated. Given libnatpmp is a new inclusion into our
repository, I think we should be using this newer source.

This also cleans up a few warnings we currently see in depends builds:
```bash
Extracting libnatpmp...
/home/ubuntu/bitcoin/depends/sources/libnatpmp-20150609.tar.gz: OK
Preprocessing libnatpmp...
Configuring libnatpmp...
Building libnatpmp...
make[1]: Entering directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o natpmp.o natpmp.c
x86_64-w64-mingw32-gcc -Os -fPIC -Wall -DENABLE_STRNATPMPERR   -c -o getgateway.o getgateway.c
natpmp.c:42: warning: "EWOULDBLOCK" redefined
   42 | #define EWOULDBLOCK WSAEWOULDBLOCK
      |
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:166: note: this is the location of the previous definition
  166 | #define EWOULDBLOCK 140
      |
natpmp.c:43: warning: "ECONNREFUSED" redefined
   43 | #define ECONNREFUSED WSAECONNREFUSED
      |
In file included from natpmp.c:38:
/usr/share/mingw-w64/include/errno.h:110: note: this is the location of the previous definition
  110 | #define ECONNREFUSED 107
      |
natpmp.c:271:5: warning: ‘readnatpmpresponseorretry’ redeclared without dllimport attribute: previous dllimport ignored [-Wattributes]
  271 | int readnatpmpresponseorretry(natpmp_t * p, natpmpresp_t * response)
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~
ar crs libnatpmp.a natpmp.o getgateway.o
make[1]: Leaving directory '/home/ubuntu/bitcoin/depends/work/build/x86_64-w64-mingw32/libnatpmp/20150609-13efa1beb87'
Staging libnatpmp...
Postprocessing libnatpmp...
Caching libnatpmp...
```
2021-03-04 10:34:18 +08:00
Jarol Rodriguez
c180c911b8 doc: revamp macOS build doc
This pr makes the macOS build docs more informative and adds in the following information:
- Proper descriptions and delineation of required/optional dependencies
- walk-through of optional dependencies
- configuration walk-through
- various other tid-bits of information
2021-03-03 16:09:52 -05:00
Hennadii Stepanov
f7f3829a68 build, doc: Drop libbz2-dev from macOS cross-compiling dependencies 2021-03-03 22:43:08 +02:00
Hennadii Stepanov
d8239362e2 build, doc: Drop libcap-dev from macOS cross-compiling dependencies 2021-03-03 22:38:45 +02:00
Carl Dong
e11b649650 validation: CVerifyDB::VerifyDB: Use locking annotation
...instead of recursively locking unconditionally
2021-03-03 14:56:26 -05:00
Hennadii Stepanov
c967fb7fb9 guix: Remove libcap from manifest 2021-03-03 21:50:15 +02:00
Carl Dong
03f75c42e1 validation: Use existing chain member in CChainState::LoadGenesisBlock 2021-03-03 14:49:30 -05:00
Carl Dong
5e4af77380 validation: Use existing chain member in CChainState::AcceptBlock 2021-03-03 14:49:30 -05:00
Carl Dong
fee73347c0 validation: Pass in chain to FindBlockPos+SaveBlockToDisk 2021-03-03 14:49:30 -05:00
Carl Dong
a9d28bcd8d validation: Use *this in CChainState::ActivateBestChainStep 2021-03-03 14:49:30 -05:00
Carl Dong
4744efc9ba validation: Pass in chainstate to CTxMemPool::check
This is the only instance where validation reaches for something outside
of it.
2021-03-03 14:49:29 -05:00
Wladimir J. van der Laan
47b99ab1a9 Merge #20406: util: Avoid invalid integer negation in FormatMoney and ValueFromAmount
1f05dbd06d util: Avoid invalid integer negation in ValueFromAmount: make ValueFromAmount(const CAmount& n) well-defined also when n is std::numeric_limits<CAmount>::min() (practicalswift)
7cc75c9ba3 util: Avoid invalid integer negation in FormatMoney: make FormatMoney(const CAmount& n) well-defined also when n is std::numeric_limits<CAmount>::min() (practicalswift)

Pull request description:

  Avoid invalid integer negation in `FormatMoney` and `ValueFromAmount`.

  Fixes #20402.

  Before this patch:

  ```
  $ CC=clang CXX=clang++ ./configure --with-sanitizers=undefined
  $ make -C src/ test/test_bitcoin
  $ src/test/test_bitcoin -t rpc_tests/rpc_format_monetary_values -t util_tests/util_FormatMoney
  core_write.cpp:21:29: runtime error: negation of -9223372036854775808 cannot be represented in type 'CAmount'
    (aka 'long'); cast to an unsigned type to negate this value to itself
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior core_write.cpp:21:29 in
  test/rpc_tests.cpp(186): error: in "rpc_tests/rpc_format_monetary_values":
    check ValueFromAmount(std::numeric_limits<CAmount>::min()).write() == "-92233720368.54775808" has failed
    [--92233720368.-54775808 != -92233720368.54775808]
  util/moneystr.cpp:16:34: runtime error: negation of -9223372036854775808 cannot be represented in type 'CAmount'
    (aka 'long'); cast to an unsigned type to negate this value to itself
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior util/moneystr.cpp:16:34 in
  test/util_tests.cpp(1188): error: in "util_tests/util_FormatMoney":
    check FormatMoney(std::numeric_limits<CAmount>::min()) == "-92233720368.54775808" has failed
    [--92233720368.-54775808 != -92233720368.54775808]
  ```

  After this patch:

  ```
  $ CC=clang CXX=clang++ ./configure --with-sanitizers=undefined
  $ make -C src/ test/test_bitcoin
  $ src/test/test_bitcoin -t rpc_tests/rpc_format_monetary_values -t util_tests/util_FormatMoney
  ```

ACKs for top commit:
  laanwj:
    re-ACK 1f05dbd06d

Tree-SHA512: 5aaeb8e2178f1597921f53c12bdfc2f3d5993d10c41658dcd25943e54e8cc2116a411bc71d928f890b33bc0b3761a8ee4449b0532bce41125b6c60692808c8c3
2021-03-03 19:04:36 +01:00
Pieter Wuille
0eaea66e8b Make tx relay data structure use std::chrono types 2021-03-03 09:48:07 -08:00
Pieter Wuille
55e82881a1 Make all Poisson delays use std::chrono types 2021-03-03 09:48:07 -08:00
Pieter Wuille
c733ac4d8a Convert block/header sync timeouts to std::chrono types 2021-03-03 09:48:07 -08:00
Pieter Wuille
4d98b401fb Change all ping times to std::chrono types 2021-03-03 09:48:07 -08:00
Russell Yanofsky
f5ba424cd4 wallet: Add IsAddressUsed / SetAddressUsed methods
This simplifies code and adds a less cumbersome interface for accessing
address used information than CWallet AddDestData / EraseDestData /
GetDestData methods.

There is no change in behavior. Lower-level walletdb DestData methods
are also still available and not affected by this change. If there is
interest in consolidating destdata logic more and making it internal to
walletdb, #18608 could be considered as a followup.
2021-03-03 10:19:35 -04:00
Wladimir J. van der Laan
cabe63759c Merge #20877: netinfo: user help and argument parsing improvements
7d3343fb8e cli: update -netinfo help doc following the merge of 882ce251 (Jon Atack)
ef614bb408 cli: small -netinfo simplification and performance improvement (Jon Atack)
6b45ef3233 cli: improve -netinfo invalid argument error message (Jon Atack)
3732404afa cli: warn in help that -netinfo is not intended to be a stable API (Jon Atack)
7afdd72258 cli: enable -netinfo help to run without a remote server (Jon Atack)

Pull request description:

  A few updates, some per IRC discussion today at http://www.erisian.com.au/bitcoin-core-dev/log-2021-01-07.html#l-87 with respect to -netinfo:

  - enable `-netinfo help` to run without a remote server
  - warn in `-netinfo help` that -netinfo is not intended to be a stable API
  - improve the -netinfo invalid argument error message
  - make a performance improvement and simplification I noticed after the merge of #20764
  - update the -netinfo help doc following the merge of #21192
  -----

  How to test manually:  🔬 🧪  📈

  1. check out and build this branch locally; if you need help, don't hesitate to refer to https://jonatack.github.io/articles/how-to-review-pull-requests-in-bitcoin-core#pull-down-the-code-locally or https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests
  2. while it is compiling, look at the code changes
  3. stop signet (if it is running) with `./src/bitcoin-cli -signet stop`
  4. once the build is completed, run `./src/bitcoin-cli -signet -netinfo help`
  5. the help should be printed even though the signet server is not running
  6. near the top you should see the new warning, "This human-readable interface will change regularly and is not intended to be a stable API" as well as a bit more description about the integer argument values.
  7. start signet with `./src/bitcoind -signet`
  8. test the improved invalid argument error message if you run `./src/bitcoin-cli -signet -netinfo 256` or `./src/bitcoin-cli -signet -netinfo a` (valid values are from 0 to 255)
  9. leave review feedback or `ACK <commit hash>` -- done 🍻

ACKs for top commit:
  michaelfolkson:
    Re-ACK 7d3343fb8e
  pinheadmz:
    RE-ACK 7d3343fb8e

Tree-SHA512: 28c5e9f295ffccba5c2a70faac4987d45f35d4758cf8f10daa767e83212316c4cfc65930e4066f7ad627e9d15b92d43439d1ba9c2f755dfde61885c6a70aa155
2021-03-03 15:19:35 +01:00
Russell Yanofsky
62252c95e5 interfaces: Stop exposing wallet destdata to gui
Stop giving GUI access to destdata rows in database. Replace with narrow
API just for saving and reading receive request information.

This simplifies code and should prevent the GUI from interfering with
other destdata like address-used status.

Note: No user-visible behavior is changing in this commit. New
CWallet::SetAddressReceiveRequest() implementation avoids a bug in
CWallet::AddDestData() where a modification would leave the previous
value in memory while writing the new value to disk. But it doesn't
matter because the GUI doesn't currently expose the ability to modify
receive requests, only to add and erase them.
2021-03-03 10:19:35 -04:00
Russell Yanofsky
985430d9b2 test: Add gui test for wallet receive requests
Make sure wallet receive requests are saved and deleted correctly by GUI
code

Co-authored-by: Jarol Rodriguez <jarolrod@tutanota.com>
2021-03-03 09:19:35 -05:00
MarcoFalke
ebd8d66454 Merge #19203: net: Add regression fuzz harness for CVE-2017-18350. Add FuzzedSocket.
366e3e1f89 fuzz: Add FUZZED_SOCKET_FAKE_LATENCY mode to FuzzedSock to allow for fuzzing timeout logic (practicalswift)
b22d4c1607 fuzz: Add fuzzing harness for Socks5(...) (practicalswift)

Pull request description:

  Add [regression fuzz harness](https://twitter.com/kayseesee/status/1205287895923212289) for CVE-2017-18350. This fuzzing harness would have found CVE-2017-18350 within a minute of fuzzing :)

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  vasild:
    ACK 366e3e1f89

Tree-SHA512: 5d8e1863b635efd10ccb11678b71472ba1523c3ef16affa7f9cd638635c1a9c307e28f432d5b87eb0c9cd1c3c1aeafbb24fa7ae86fe4e5090fda2e20d542b6ca
2021-03-03 14:41:05 +01:00
Anthony Towns
0c471a5f30 tests: check never active versionbits 2021-03-03 14:14:40 +01:00
Anthony Towns
3ba9283a47 tests: more helpful errors for failing versionbits tests
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
2021-03-03 14:12:16 +01:00
fanquake
97a35f3ae5 Merge #21320: build: fix libnatpmp macos cross compile
bd49ac4168 build: fix libnatpmp macos cross compile (fanquake)

Pull request description:

  Currently, our cross-compile of libnatpmp for macOS doesn't work at all.
  The wrong archiver is used, which produces an archive the linker doesn't like.
  This becomes clear when configuring:
  ```bash
  configure:25722: checking for initnatpmp in -lnatpmp
  configure:25747: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/bitcoin/depends/x86_64-apple-darwin18/native/bin/clang++ --target=x86_64-apple-darwin18 <trim>  -Wl,-headerpad_max_install_names -Wl,-dead_strip -Wl,-dead_strip_dylibs conftest.cpp -lnatpmp   >&5
  ld: archive has no table of contents for architecture x86_64
  clang: error: linker command failed with exit code 1 (use -v to see invocation)
  ```

  Fix this by using the right `ar` (we do the same for upnp).

  While we're at it, fix the build so that we are using our c/ppflags.
  In practice this basically means building with `-O2` rather than `-Os`.

  Note that this fixes an issue that is also fixed by #21209. However, given there are reservations about updating to use a newer libnatpmp source, we should just fix this for now.

ACKs for top commit:
  hebasto:
    ACK bd49ac4168, tested:

Tree-SHA512: 2efc2c788ef3ebebfbf564ef07b6cf63a72d8a0bccc22b0ba36537216aa575436b7e87088477e85f6d9191ad34f0b13f1c22cf88c90e1cb81641bfee5dc3058a
2021-03-03 19:58:25 +08:00
Wladimir J. van der Laan
b1a457ab1d Merge #21324: doc: Update build instructions for Fedora
4899fa3abd doc: Update build instructions for Fedora (Hennadii Stepanov)

Pull request description:

  This PR updates build instructions for Fedora, as Fedora 33 has no `libdb4-devel` and `libdb4-cxx-devel` packages in its default repos.

ACKs for top commit:
  practicalswift:
    ACK 4899fa3abd

Tree-SHA512: 0c23e2ce0ea690ed5eeaa56514d4246e0057a77b7c71f28af4ee1e480521d465122f81cea37cc773ce2db4fc189d5ab3c8f8ffdd65f150cc006390aa1e2a4ac8
2021-03-03 12:49:59 +01:00
Wladimir J. van der Laan
dd8f474d52 Merge #19288: fuzz: Add fuzzing harness for TorController
10d4477dae tests: Add fuzzing harness for TorController (practicalswift)
64219c01dc torcontrol: Move TorControlReply, TorControlConnection and TorController to improve testability (practicalswift)

Pull request description:

  Add fuzzing harness for `TorController`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  laanwj:
    ACK 10d4477dae

Tree-SHA512: 2da4b1000afe0e65a234636b8fbf6a26fe9e257852bd837168ca73aa3575959e9aded19054620439e4ed0b2787c70cad4541a8c2d210f5238d7f5e9e0545b734
2021-03-03 11:50:00 +01:00
Hennadii Stepanov
52a43b0c7d build: Fix fuzz-cuckoocache cross-compiling for Windows with DEBUG=1 2021-03-03 12:23:41 +02:00
MarcoFalke
9212e67463 Merge #21346: doc: install qt5 when building on macOS
bec7f2caf7 doc: install qt5 when building on macOS (fanquake)

Pull request description:

  Brew has updated such that qt now refers to [Qt 6.0.1](https://github.com/Homebrew/homebrew-core/blob/master/Formula/qt.rb). If builders
  install this, configure will not pick up qt. For now, install
  [qt@5 (5.15.2)](https://github.com/Homebrew/homebrew-core/blob/master/Formula/qt@5.rb), until required build system and likely source changes
  are made.

ACKs for top commit:
  hebasto:
    ACK bec7f2caf7, tested on Tested on macOS Big Sur 11.2.2 (20D80).

Tree-SHA512: 86663cfbc68c8c6f5d608d60cd59b37d3faf1e7f33ae17ec2e1a7c076e835eb8200181a17575f121929ea6ecded74b1619096fe5a763106f56de0bdbea9ae4fa
2021-03-03 10:08:01 +01:00
Ivan Metlushko
5039e0e55a test: HelpExampleCliNamed and HelpExampleRpcNamed 2021-03-03 09:58:39 +01:00
Wladimir J. van der Laan
591735ef0b rpc: Add HelpExampleCliNamed and use it for createwallet doc 2021-03-03 09:40:06 +01:00
Ivan Metlushko
5d5a90e819 rpc: Add HelpExampleRpcNamed 2021-03-03 09:36:23 +01:00
MarcoFalke
fa576b4532 Move MakeNoLogFileContext to common libtest_util, and use it in bench
Can be reviewed with --color-moved=dimmed-zebra
2021-03-03 09:17:37 +01:00
fanquake
fca3e98f64 Merge #18298: build: Fix Qt processing of configure script for depends with DEBUG=1
76f52e3da3 build: Fix Qt processing of configure script for depends with DEBUG=1 (Hennadii Stepanov)

Pull request description:

  This PR:
  - makes the `configure` script correctly pickup Qt if depends is built with `DEBUG=1`:
      - for Windows -- fix #19266
      - for macOS -- fix #16391
  - is an alternative to #18117 (without downsides)

ACKs for top commit:
  fanquake:
    ACK 76f52e3da3. Tested native darwin, and darwin/win cross compile with `DEBUG=1`.

Tree-SHA512: 8fde99302b4b06faf109315bddba9e3063b156c50f8f9863c2bd51718538c719429a63fdced071730c18022f2e559d3b25c1dcec3efa81fe79f657253680956a
2021-03-03 15:48:11 +08:00
Hennadii Stepanov
7bbb409314 guix: Update darwin native packages dependencies 2021-03-03 08:44:49 +02:00
fanquake
bec7f2caf7 doc: install qt5 when building on macOS
Brew has updated such that qt now refers to Qt 6.0.1. If builders
install this, configure will not work pick up qt. For now, install
qt@5 (5.15.2), until required build system and likely source changes
are made.
2021-03-03 12:00:06 +08:00
fanquake
4f223e93e9 Merge #21339: build: Make AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER]) unconditional
a4128138b4 build: Make AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER]) unconditional (Hennadii Stepanov)
9fef209945 build, refactor: Fix indentation for if..then..fi (Hennadii Stepanov)

Pull request description:

  #16546 introduced a regression in the `configure`:

  ```
  $ ./autogen.sh
  $ ./configure --disable-wallet --without-utils --without-daemon --without-gui --disable-tests --disable-bench
  ...
  checking whether to build test_bitcoin... no
  checking whether to reduce exports... no
  checking that generated files are newer than configure... done
  configure: error: conditional "ENABLE_EXTERNAL_SIGNER" was never defined.
  Usually this means the macro was only invoked conditionally.
  ```

  This PR fixes this bug, and refactors indentation to make easier to spot similar bugs in the future.

ACKs for top commit:
  Sjors:
    utACK a4128138b4
  fanquake:
    ACK a4128138b4 - this fixes the bug described, and improves readability.

Tree-SHA512: 4469dcc006690f38f93c3cdf8d15b76f5fc8ea76e87a1b5db5ee891dc9851f6ec539f2a6fd02a361aa76baa4f4b2b9fe8289137f5d9734ee5984f265cb131ef5
2021-03-03 10:54:38 +08:00
fanquake
3a8fc51a56 Merge #21333: build: set Unicode true for NSIS installer
9086e0dd3c build: set Unicode true for NSIS installer (fanquake)

Pull request description:

  Now that we are using Focal for gitian builds, and have [NSIS 3.0+ available](https://packages.ubuntu.com/focal/nsis) (also in Guix), we can create installers that [support unicode](https://nsis.sourceforge.io/Docs/Chapter4.html#aunicodetarget).

  Unicode is only becoming the NSIS default [beginning with the 3.07 release](https://nsis.sourceforge.io/Docs/AppendixF.html#v3.07-cl), so we need to set this attribute to get support.

  Should close: #13817

  Gitian builds:
  ```bash
  b8553615b6b4be5e4459e03796e700b30b5d198a7f184f27be6983ff901b5592  bitcoin-9086e0dd3c92-win-unsigned.tar.gz
  a6b024a5a68e0196e8e118168c918285e820f2d0ffe9c38db680580459da8bf3  bitcoin-9086e0dd3c92-win64-debug.zip
  ff4003d4f61127c707e44b5235eaf924b30351f20cde27e775131982a1b4cf92  bitcoin-9086e0dd3c92-win64-setup-unsigned.exe
  1876bee55fa9ea99b91203975c13d0ad8a046b4b58068bde41c977fd1d12de13  bitcoin-9086e0dd3c92-win64.zip
  000f2778f8f166a89b4ab35f155156c1c34800be6e47d29b5308043c50128392  src/bitcoin-9086e0dd3c92.tar.gz
  d650a9b8f2dd1df777bf42439dfcbcf6bc358e30ec148b9992a18b39f76b1ecf  bitcoin-core-win-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    ACK 9086e0dd3c
  hebasto:
    ACK 9086e0dd3c, tested on Windows 10 Pro (20H2, build 19042.804):

Tree-SHA512: cc7b7ca05877571d0a29a7d36e40279f54d886d8ab27facfa722c2ee95a1fc06c2bad8ef1eb1980d283ae981659d737021a46c8f4618e24d510b5ab384990e09
2021-03-03 09:57:52 +08:00
fanquake
0459e7abba Merge #21336: Make .gitignore ignore src/test/fuzz/fuzz.exe
35da623574 Make .gitignore ignore src/test/fuzz/fuzz.exe (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    cr ACK 35da623574
  practicalswift:
    cr ACK 35da623574

Tree-SHA512: 7e88f4f3ad409b38269fcc7b81c075567aa8a5c2365ff8d3e5b148309e6110722eb84d5732a19f90b81e778bf134d93a886287da70dda708c15477f124e87b03
2021-03-03 08:07:54 +08:00
fanquake
0b294c0ba8 Merge #21342: doc: Remove outdated comment
f1f63ac3f8 doc: Remove outdated comment (Hennadii Stepanov)

Pull request description:

  The removed commit has been wrong [since v0.20.0](https://github.com/bitcoin/bitcoin/pull/18331).

ACKs for top commit:
  fanquake:
    ACK - f1f63ac3f8

Tree-SHA512: ef6191fef389fa0ee5e6cf224e4990a1804aeefd1c3e9d9a4870cf46e1833fbb8701c379b6ce4e13caa02ae2f4f86778fa2c1e994c89392c08fcf01701482d7a
2021-03-03 07:20:41 +08:00
Martin Zumsande
a061a29970 test: bring p2p_leak.py up to date.
After the introduction of wtxidrelay and sendaddrv2 messages during
version handshake, extend p2p_leak.py test to reflect this.
Also, some minor fixes and doc improvements.
2021-03-03 00:16:16 +01:00
practicalswift
366e3e1f89 fuzz: Add FUZZED_SOCKET_FAKE_LATENCY mode to FuzzedSock to allow for fuzzing timeout logic 2021-03-02 21:44:51 +00:00
practicalswift
b22d4c1607 fuzz: Add fuzzing harness for Socks5(...) 2021-03-02 21:43:42 +00:00
Hennadii Stepanov
f1f63ac3f8 doc: Remove outdated comment
The removed commit is wrong since v0.21.0.
2021-03-02 22:14:18 +02:00
Hennadii Stepanov
a4128138b4 build: Make AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER]) unconditional
This change fixes an error when the value of the "use_boost" variable is
equal to "no".
2021-03-02 19:07:31 +02:00
Hennadii Stepanov
9fef209945 build, refactor: Fix indentation for if..then..fi
This makes easier to spot conditional macros.
2021-03-02 18:55:07 +02:00
practicalswift
1f05dbd06d util: Avoid invalid integer negation in ValueFromAmount: make ValueFromAmount(const CAmount& n) well-defined also when n is std::numeric_limits<CAmount>::min() 2021-03-02 16:05:28 +00:00
practicalswift
7cc75c9ba3 util: Avoid invalid integer negation in FormatMoney: make FormatMoney(const CAmount& n) well-defined also when n is std::numeric_limits<CAmount>::min() 2021-03-02 16:05:28 +00:00
Hennadii Stepanov
35da623574 Make .gitignore ignore src/test/fuzz/fuzz.exe 2021-03-02 16:16:24 +02:00
practicalswift
10d4477dae tests: Add fuzzing harness for TorController 2021-03-02 12:21:32 +00:00
practicalswift
64219c01dc torcontrol: Move TorControlReply, TorControlConnection and TorController to improve testability 2021-03-02 12:21:32 +00:00
Wladimir J. van der Laan
b9f41df1ea Merge #20685: Add I2P support using I2P SAM
a701fcf01f net: Do not skip the I2P network from GetNetworkNames() (Vasil Dimov)
0181e24439 net: recognize I2P from ParseNetwork() so that -onlynet=i2p works (Vasil Dimov)
b905363fa8 net: accept incoming I2P connections from CConnman (Vasil Dimov)
0635233a1e net: make outgoing I2P connections from CConnman (Vasil Dimov)
9559bd1404 net: add I2P to the reachability map (Vasil Dimov)
76c35c60f3 init: introduce I2P connectivity options (Vasil Dimov)
c22daa2ecf net: implement the necessary parts of the I2P SAM protocol (Vasil Dimov)
5bac7e45e1 net: extend Sock with a method to check whether connected (Vasil Dimov)
42c779f503 net: extend Sock with methods for robust send & read until terminator (Vasil Dimov)
ea1845315a net: extend Sock::Wait() to report a timeout (Vasil Dimov)
78fdfbea66 net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions (Vasil Dimov)
34bcfab562 net: move the constant maxWait out of InterruptibleRecv() (Vasil Dimov)
cff65c4a27 net: extend CNetAddr::SetSpecial() to support I2P (Vasil Dimov)
f6c267db3b net: avoid unnecessary GetBindAddress() call (Vasil Dimov)
7c224fdac4 net: isolate the protocol-agnostic part of CConnman::AcceptConnection() (Vasil Dimov)
1f75a653dd net: get the bind address earlier in CConnman::AcceptConnection() (Vasil Dimov)
25605895af net: check for invalid socket earlier in CConnman::AcceptConnection() (Vasil Dimov)
545bc5f81d util: fix WriteBinaryFile() claiming success even if error occurred (Vasil Dimov)
8b6e4b3b23 util: fix ReadBinaryFile() returning partial contents (Vasil Dimov)
4cba2fdafa util: extract {Read,Write}BinaryFile() to its own files (Vasil Dimov)

Pull request description:

  Add I2P support by using the [I2P SAM](https://geti2p.net/en/docs/api/samv3) protocol. Unlike Tor, for incoming connections we get the I2P address of the peer (and they also receive ours when we are the connection initiator).

  Two new options are added:

  ```
    -i2psam=<ip:port>
         I2P SAM proxy to reach I2P peers and accept I2P connections (default:
         none)

    -i2pacceptincoming
         If set and -i2psam is also set then incoming I2P connections are
         accepted via the SAM proxy. If this is not set but -i2psam is set
         then only outgoing connections will be made to the I2P network.
         Ignored if -i2psam is not set. Notice that listening for incoming
         I2P connections is done through the SAM proxy, not by binding to
         a local address and port (default: true)
  ```

  # Overview of the changes

  ## Make `ReadBinary()` and `WriteBinary()` reusable

  We would need to dump the I2P private key to a file and read it back later. Move those two functions out of `torcontrol.cpp`.

  ```
  util: extract {Read,Write}BinaryFile() to its own files
  util: fix ReadBinaryFile() returning partial contents
  util: fix WriteBinaryFile() claiming success even if error occurred
  ```

  ## Split `CConnman::AcceptConnection()`

  Most of `CConnman::AcceptConnection()` is agnostic of how the socket was accepted. The other part of it deals with the details of the `accept(2)` system call. Split those so that the protocol-agnostic part can be reused if we accept a socket by other means.

  ```
  net: check for invalid socket earlier in CConnman::AcceptConnection()
  net: get the bind address earlier in CConnman::AcceptConnection()
  net: isolate the protocol-agnostic part of CConnman::AcceptConnection()
  net: avoid unnecessary GetBindAddress() call
  ```

  ## Implement the I2P [SAM](https://geti2p.net/en/docs/api/samv3) protocol (not all of it)

  Just the parts that would enable us to make outgoing and accept incoming I2P connections.

  ```
  net: extend CNetAddr::SetSpecial() to support I2P
  net: move the constant maxWait out of InterruptibleRecv()
  net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions
  net: extend Sock::Wait() to report a timeout
  net: extend Sock with methods for robust send & read until terminator
  net: extend Sock with a method to check whether connected
  net: implement the necessary parts of the I2P SAM protocol
  ```

  ## Use I2P SAM to connect to and accept connections from I2P peers

  Profit from all of the preceding commits.

  ```
  init: introduce I2P connectivity options
  net: add I2P to the reachability map
  net: make outgoing I2P connections from CConnman
  net: accept incoming I2P connections from CConnman
  net: recognize I2P from ParseNetwork() so that -onlynet=i2p works
  net: Do not skip the I2P network from GetNetworkNames()
  ```

ACKs for top commit:
  laanwj:
    re-ACK a701fcf01f
  jonatack:
    re-ACK a701fcf01f reviewed diff per `git range-diff ad89812 2a7bb34 a701fcf`, debug built and launched bitcoind with i2pd v2.35 running a dual I2P+Torv3 service with the I2P config settings listed below (did not test `onlynet=i2p`); operation appears nominal (same as it has been these past weeks), and tested the bitcoind help outputs grepping for `-i i2p` and the rpc getpeerinfo and getnetworkinfo helps

Tree-SHA512: de42090c9c0bf23b43b5839f5b4fc4b3a2657bde1e45c796b5f3c7bf83cb8ec6ca4278f8a89e45108ece92f9b573cafea3b42a06bc09076b40a196c909b6610e
2021-03-02 11:50:13 +01:00
MarcoFalke
cfce346508 Merge #21310: zmq test: fix sync-up by matching notification to generated block
8a8c6383f6 zmq test: fix sync-up by matching notification to generated block (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR for #21008, fixes #21216.

  In the course of investigating the problem with jnewbery (analyzing the Cirrus log https://cirrus-ci.com/task/4660108304056320), it turned out that the "sync up" procedure of repeatedly generating a block and waiting for a notification with timeout is too brittle in its current form, as the following scenario could happen:

  - generate block A
  - receive notification, timeout happens => repeat procedure
  - generate block B
  - node publishes block A notification
  - receive notification, we receive the one caused by block A (!!!) => sync-up procedure is completed
  - node publishes block B notification
  - the actual test starts
  - on the first notification reception, the one caused by block B is received, rather than the one actually caused by test code => assertion failure

  This change in the PR ensures that after each test block generation, we wait for the notification that is actually caused by that block and ignore others from possibly earlier blocks. The matching is kind of ugly, it assumes that one out of four components in the block is contained in the notification: the block hash, the tx id, the raw block data or the raw transaction data. (Unfortunately we have to support all publisher topics.)

  I'm aware that this is quite a lot of code now only for establishing a robust test setup. OTOH I wouldn't know of a better method right now, suggestions are very welcome.

  Note for potential reviewers: for both reproducing the issue on master branch and verifying on PR branch, one can simply generate two blocks in the sync-up procedure rather than one.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 8a8c6383f6

Tree-SHA512: a2eb78ba06dfd0fda7b1c111b6bbfb5dab4ab08500cc19c7ea02c3239495d5c74cc7d45250a8b3ecc78ca42d97ee6719bf73db8a137839e5e09a3cfcf08ed29e
2021-03-02 11:30:17 +01:00
Anthony Towns
5e50e2d1b9 txorphanage: comment improvements 2021-03-02 19:40:11 +10:00
fanquake
9086e0dd3c build: set Unicode true for NSIS installer
Now that we are using Focal for gitian builds, and have NSIS 3.0+
available (also in Guix), we can create installers that support unicode.

Unicode is only becoming the default beginning with the 3.07 release,
https://nsis.sourceforge.io/Docs/AppendixF.html#v3.07-cl,
so we need to set this attribute to get support.

Should close: #13817

Co-authored-by: Chun Kuan Lee <ken2812221@gmail.com>
2021-03-02 16:36:21 +08:00
Hennadii Stepanov
4899fa3abd doc: Update build instructions for Fedora
Fedora 33 requires adding the Cheese third-party repo to install
libdb4-devel and libdb4-cxx-devel packages.
2021-03-02 10:09:19 +02:00
fanquake
72e6979b31 Merge #21325: lint: Fix spelling errors in comments
fbbb2d4fc1 lint: Fix spelling errors in comments (fyquah)

Pull request description:

  Found some spelling errors while running spelling linter  https://github.com/bitcoin/bitcoin/pull/21245

  This PR fixes them.

ACKs for top commit:
  fanquake:
    ACK fbbb2d4fc1 - I thought we just fixed all of these.

Tree-SHA512: 95525040001f94e899b778c616cb66ebafb679dff88835b66fccf6349d8eb942d6b7374c536a44e393f13156bce9a32ed57e6a82bb02074d2b3cddb2696addb2
2021-03-02 09:43:52 +08:00
fanquake
a28c053c88 Merge #21298: guix: Bump time-machine, glibc, and linux-headers
c33b199456 guix: Bump glibc and linux-headers (Carl Dong)
65363a1bd8 guix: Rebase on 95aca2991b (1.2.0-12.dffc918) (Carl Dong)

Pull request description:

  On bumping the time-machine:

  ```
  A few changes which are useful for us:

  1. 'gnu: cross-gcc-arguments: Enable 128 bit long double for POWER9.' is
     now merged into master.
  2. gnutls is bumped to 3.6.15 and the temporal test failure in
     status-request-revoked is fixed. Note that this does not fix the case
     where one has installed Guix v1.2.0 and is running a substitute-less
     bootstrap build, since the `guix time-machine` command itself has a
     dependency on gnutls v3.6.12 (the one with the broken test) and will
     thus try to build it before attempting to jump forwards in time. This
     does however, mean that those who build a version of Guix that also
     contains this fix will not go backwards in time to build the broken
     gnutls v3.6.12.
  ```

  On bumping the rest:

  ```
  Bump glibc and linux-headers to match those of our Gitian counterparts.

  We also require a glibc >= 2.28 for the test-symbol-check scripts to
  work properly.

  The default BASE-GCC-FOR-LIBC also has to be bumped since glibc 2.31
  requires a gcc >= 6.2
  ```

  This is a prerequisite for #20980

ACKs for top commit:
  fanquake:
    ACK c33b199456 - I think going ahead with this now and to sycn back up to gitian is fine. It will also unblock #20980. Potential code signing related issues can be sorted out in #21239 and later PRs.

Tree-SHA512: 31f022aadb93ba44813b0da005b1f2e5d67d76e8cdcdb53368924d1ea6cb076a21218c26831a6b0dcdcfe33507f54934330489ba557371d740f5587b7d727b95
2021-03-02 09:37:27 +08:00
fanquake
d1abe54fce Merge #21323: guix, doc: Update default HOSTS value
a0a7a4337d guix, doc: Update default HOSTS value (Hennadii Stepanov)

Pull request description:

  This is a #21089 follow up.

ACKs for top commit:
  fanquake:
    ACK a0a7a4337d

Tree-SHA512: c1813cc2b9212a79fd34d4e25cd0816b58264e1890daf777cd59411bd20fcc9affe312871d06fab1308b8f55c1a78ac1101e631882c18360a4709ecef4529f05
2021-03-02 09:03:06 +08:00
Carl Dong
1fb7b2c595 validation: Use *this in CChainState::InvalidateBlock 2021-03-01 17:56:23 -05:00
Carl Dong
8cdb2f7e58 validation: Move LoadBlockIndexDB to CChainState
CChainState needed cuz setBlockIndexCandidates
2021-03-01 17:56:22 -05:00
Carl Dong
8b99efbcc0 validation: Move invalid block handling to CChainState
- InvalidChainFound
- CheckForkWarningConditions
2021-03-01 17:56:07 -05:00
Carl Dong
2bdf37fe18 validation: Pass in chainstate to CVerifyDB::VerifyDB 2021-03-01 17:56:07 -05:00
Carl Dong
31eac50c72 validation: Remove global ::VersionBitsTip{State,SinceHeight,Statistics}
Tip: versionbitscache is currently a global so we didn't need to pass it
     in to any of ::VersionBitsTip*'s callers
2021-03-01 17:56:07 -05:00
Carl Dong
63e4c7316a validation: Pass in chainstate to ::PruneBlockFilesManual 2021-03-01 17:56:07 -05:00
Carl Dong
4bada76237 validation: Pass in chainstate to UpdateTip 2021-03-01 17:56:07 -05:00
Wladimir J. van der Laan
05e821ee19 Merge #21170: bench: Add benchmark to write JSON into a string
e3e0a2432c Add benchmark to write JSON into a string (Martin Ankerl)

Pull request description:

  The benchmark `BlockToJsonVerbose` only tests generating (and destroying)
  the JSON data structure, but serializing into a string is also a
  performance critical aspect of the RPC calls.

  Extracts test setup into a `struct TestBlockAndIndex`, and uses it in
  both `BlockToJsonVerbose` and `BlockToJsonVerboseWrite`.

  Also, use `ankerl::nanobench::doNotOptimizeAway` to make sure the compiler
  can't optimize the result of the calls away.

  Here are benchmark results on my Intel i7-8700:

  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |       71,807,017.00 |               13.93 |    0.4% |  555,782,961.00 |  220,788,645.00 |  2.517 | 102,279,341.00 |    0.4% |      0.80 | `BlockToJsonVerbose`
  |       27,916,835.00 |               35.82 |    0.1% |  235,084,034.00 |   89,033,525.00 |  2.640 |  42,911,139.00 |    0.3% |      0.32 | `BlockToJsonVerboseWrite`

ACKs for top commit:
  laanwj:
    Code review ACK e3e0a2432c

Tree-SHA512: bc4d6d1588d47d4bd7af8e7908e44b8561bc391a2d73eccd7c0aa37fc40e8a9ce1fa1f3c29b416eef24a73c6bce3036839c0bbfe1b8dbd6d1bba3718b7ca5383
2021-03-01 19:12:09 +01:00
Vasil Dimov
a701fcf01f net: Do not skip the I2P network from GetNetworkNames()
So that help texts include "i2p" in:
* `./bitcoind -help` (in `-onlynet` description)
* `getpeerinfo` RPC
* `getnetworkinfo` RPC

Co-authored-by: Jon Atack <jon@atack.com>
2021-03-01 18:19:47 +01:00
Vasil Dimov
0181e24439 net: recognize I2P from ParseNetwork() so that -onlynet=i2p works 2021-03-01 18:19:47 +01:00
Vasil Dimov
b905363fa8 net: accept incoming I2P connections from CConnman 2021-03-01 18:19:47 +01:00
Vasil Dimov
0635233a1e net: make outgoing I2P connections from CConnman 2021-03-01 18:19:46 +01:00
Vasil Dimov
9559bd1404 net: add I2P to the reachability map
Update `CNetAddr::GetReachabilityFrom()` to recognize the I2P network so
that we would prefer to advertise our I2P address to I2P peers.
2021-03-01 18:19:46 +01:00
Vasil Dimov
76c35c60f3 init: introduce I2P connectivity options
Introduce two new options to reach the I2P network:

* `-i2psam=<ip:port>` point to the I2P SAM proxy. If this is set then
  the I2P network is considered reachable and we can make outgoing
  connections to I2P peers via that proxy. We listen for and accept
  incoming connections from I2P peers if the below is set in addition to
  `-i2psam=<ip:port>`

* `-i2pacceptincoming` if this is set together with `-i2psam=<ip:port>`
  then we accept incoming I2P connections via the I2P SAM proxy.
2021-03-01 18:19:46 +01:00
Vasil Dimov
c22daa2ecf net: implement the necessary parts of the I2P SAM protocol
Implement the following commands from the I2P SAM protocol:

* HELLO: needed for all of the remaining ones
* DEST GENERATE: to generate our private key and destination
* NAMING LOOKUP: to convert .i2p addresses to destinations
* SESSION CREATE: needed for STREAM CONNECT and STREAM ACCEPT
* STREAM CONNECT: to make outgoing connections
* STREAM ACCEPT: to accept incoming connections
2021-03-01 18:19:37 +01:00
Vasil Dimov
5bac7e45e1 net: extend Sock with a method to check whether connected
This will be convenient in the I2P SAM implementation.
2021-03-01 17:36:17 +01:00
Vasil Dimov
42c779f503 net: extend Sock with methods for robust send & read until terminator
Introduce two high level, convenience methods in the `Sock` class:

* `SendComplete()`: keep trying to send the specified data until either
  successfully sent all of it, timeout or interrupted.

* `RecvUntilTerminator()`: read until a terminator is encountered (never
  after it), timeout or interrupted.

These will be convenient in the I2P SAM implementation.

`SendComplete()` can also be used in the SOCKS5 implementation instead
of calling `send()` directly.
2021-03-01 17:36:16 +01:00
fyquah
fbbb2d4fc1 lint: Fix spelling errors in comments 2021-03-01 15:24:28 +00:00
Russell O'Connor
6a0a6e7d05 Correction for VerifyTaprootCommitment comments
According to BIP-341, 'p' is called the taproot *internal* key, not inner key.
2021-03-01 09:01:48 -05:00
Vasil Dimov
ea1845315a net: extend Sock::Wait() to report a timeout
Previously `Sock::Wait()` would not have signaled to the caller whether
a timeout or one of the requested events occurred since that was not
needed by any of the callers.

Such functionality will be needed in the I2P implementation, thus extend
the `Sock::Wait()` method.
2021-03-01 13:22:18 +01:00
Vasil Dimov
78fdfbea66 net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions
Deduplicate `MSG_NOSIGNAL` and `MSG_DONTWAIT` definitions from `net.cpp`
and `netbase.cpp` to `compat.h` where they can also be reused by other
code.
2021-03-01 13:22:17 +01:00
Vasil Dimov
34bcfab562 net: move the constant maxWait out of InterruptibleRecv()
Move `maxWait` out of `InterruptibleRecv()` and rename it to
`MAX_WAIT_FOR_IO` so that it can be reused by other code.
2021-03-01 13:22:17 +01:00
Vasil Dimov
cff65c4a27 net: extend CNetAddr::SetSpecial() to support I2P
Recognize also I2P addresses in the form `base32hashofpublickey.b32.i2p`
from `CNetAddr::SetSpecial()`.

This makes `Lookup()` support them, which in turn makes it possible to
manually connect to an I2P node by using
`-proxy=i2p_socks5_proxy:port -addnode=i2p_address.b32.i2p:port`

Co-authored-by: Lucas Ontivero <lucasontivero@gmail.com>
2021-03-01 13:22:11 +01:00
Vasil Dimov
f6c267db3b net: avoid unnecessary GetBindAddress() call
Our local (bind) address is already saved in `CNode::addrBind` and there
is no need to re-retrieve it again with `GetBindAddress()`.

Also, for I2P connections `CNode::addrBind` would contain our I2P
address, but `GetBindAddress()` would return something like
`127.0.0.1:RANDOM_PORT`.
2021-03-01 12:57:01 +01:00
Vasil Dimov
7c224fdac4 net: isolate the protocol-agnostic part of CConnman::AcceptConnection()
Isolate the second half of `CConnman::AcceptConnection()` into a new
separate method, which could be reused if we accept incoming connections
by other means than `accept()` (first half of
`CConnman::AcceptConnection()`).
2021-03-01 12:57:01 +01:00
Vasil Dimov
1f75a653dd net: get the bind address earlier in CConnman::AcceptConnection()
Call `GetBindAddress()` earlier in `CConnman::AcceptConnection()`. That
is specific to the TCP protocol and makes the code below it reusable for
other protocols, if the caller provides `addr_bind`, retrieved by other
means.
2021-03-01 12:57:01 +01:00
Vasil Dimov
25605895af net: check for invalid socket earlier in CConnman::AcceptConnection()
This check is related to an `accept()` failure. So do the check earlier,
closer to the `accept()` call.

This will allow to isolate the `accept()`-specific code at the beginning
of `CConnman::AcceptConnection()` and reuse the code that follows it.
2021-03-01 12:57:00 +01:00
Vasil Dimov
545bc5f81d util: fix WriteBinaryFile() claiming success even if error occurred
`fclose()` is flushing any buffered data to disk, so if it fails then
that could mean that the data was not completely written to disk.

Thus, check if `fclose()` succeeds and only then claim success from
`WriteBinaryFile()`.
2021-03-01 12:57:00 +01:00
Vasil Dimov
8b6e4b3b23 util: fix ReadBinaryFile() returning partial contents
If an error occurs and `fread()` returns `0` (nothing was read) then the
code before this patch would have returned "success" with a partially
read contents of the file.
2021-03-01 12:57:00 +01:00
Vasil Dimov
4cba2fdafa util: extract {Read,Write}BinaryFile() to its own files
Extract `ReadBinaryFile()` and `WriteBinaryFile()` from `torcontrol.cpp`
to its own `readwritefile.{h,cpp}` files, so that it can be reused from
other modules.
2021-03-01 12:56:56 +01:00
Wladimir J. van der Laan
ad89812c70 Merge #21321: guix: Add curl to required tool list
97f10621f4 guix: Add curl to required tool list (Hennadii Stepanov)

Pull request description:

  On Ubuntu Hirsute (minimum installation) with the system `guix` package:
  ```
  $ HOSTS=x86_64-linux-gnu ./contrib/guix/guix-build.sh
  make: Entering directory '/home/hebasto/bitcoin/depends'
  make[1]: Entering directory '/home/hebasto/bitcoin/depends'
  Checksum missing or mismatched for boost source. Forcing re-download.
  Fetching boost_1_71_0.tar.bz2 from https://dl.bintray.com/boostorg/release/1.71.0/source/
  /bin/sh: 1: curl: not found
  Fetching boost_1_71_0.tar.bz2 from https://bitcoincore.org/depends-sources
  /bin/sh: 1: curl: not found
  make[1]: *** [funcs.mk:276: /home/hebasto/bitcoin/depends/sources/download-stamps/.stamp_fetched-boost-boost_1_71_0.tar.bz2.hash] Error 127
  make[1]: Leaving directory '/home/hebasto/bitcoin/depends'
  make: *** [Makefile:281: download-linux] Error 2
  make: Leaving directory '/home/hebasto/bitcoin/depends'
  ```

  This PR fixes that issue.

ACKs for top commit:
  laanwj:
    ACK 97f10621f4
  jonasschnelli:
    utACK 97f10621f4

Tree-SHA512: b9e1ba966782b13e25c85d4a9cfd381bcda190244cb9821ae1a72fd51f82f23ed5e98d9a02384619e3299feb7a34e1da8b9190081a6005221f47f93ae18f3c0b
2021-03-01 12:08:04 +01:00
Wladimir J. van der Laan
362e901a17 Merge #18466: rpc: fix invalid parameter error codes for {sign,verify}message RPCs
a5cfb40e27 doc: release note for changed {sign,verify}message error codes (Sebastian Falbesoner)
9e399b9b2d test: check parameter validity in rpc_signmessage.py (Sebastian Falbesoner)
e62f0c71f1 rpc: fix {sign,message}verify RPC errors for invalid address/signature (Sebastian Falbesoner)

Pull request description:

  RPCs that accept address parameters usually return the intended error code `RPC_INVALID_ADDRESS_OR_KEY` (-5) if a passed address is invalid. The two exceptions to the rule are `signmessage` and `verifymessage`, which return `RPC_TYPE_ERROR` (-3) in this case instead. Oddly enough `verifymessage` returns `RPC_INVALID_ADDRESS_OR_KEY` when the _signature_ was malformed, where `RPC_TYPE_ERROR` would be more approriate.

  This PR fixes these inaccuracies and as well adds tests to `rpc_signmessage.py` that check the parameter validity and error codes for the related RPCs `signmessagewithprivkey`, `signmessage` and `verifymessage`.

  master branch:
  ```
  $ ./bitcoin-cli signmessage invalid_addr message
  error code: -3
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage invalid_addr dummy_sig message
  error code: -3
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX invalid_sig message
  error code: -5
  error message:
  Malformed base64 encoding
  ```
  PR branch:
  ```
  $ ./bitcoin-cli signmessage invalid_addr message
  error code: -5
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage invalid_addr dummy_sig message
  error code: -5
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX invalid_sig message
  error code: -3
  error message:
  Malformed base64 encoding
  ```

ACKs for top commit:
  laanwj:
    Code review ACK a5cfb40e27
  meshcollider:
    utACK a5cfb40e27

Tree-SHA512: bae0c4595a2603cea66090f6033785601837b45fd853052312b3a39d8520566c581994b68f693dd247c22586c638c3b7689c849085cce548cc36b9bf0e119d2d
2021-03-01 11:45:42 +01:00
Hennadii Stepanov
a0a7a4337d guix, doc: Update default HOSTS value 2021-03-01 10:34:30 +02:00
Hennadii Stepanov
97f10621f4 guix: Add curl to required tool list 2021-03-01 09:53:01 +02:00
fanquake
bd49ac4168 build: fix libnatpmp macos cross compile
Currently, our cross-compile of libnatpmp for macOS doesn't work at all.
The wrong archiver is used, which produces an archive the linker doesn't like.
This becomes clear when configuring:
```bash
configure:25722: checking for initnatpmp in -lnatpmp
configure:25747: env -u C_INCLUDE_PATH -u CPLUS_INCLUDE_PATH -u OBJC_INCLUDE_PATH -u OBJCPLUS_INCLUDE_PATH -u CPATH -u LIBRARY_PATH /home/ubuntu/bitcoin/depends/x86_64-apple-darwin18/native/bin/clang++ --target=x86_64-apple-darwin18 <trim>  -Wl,-headerpad_max_install_names -Wl,-dead_strip -Wl,-dead_strip_dylibs conftest.cpp -lnatpmp   >&5
ld: archive has no table of contents for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```

Fix this by using the right `ar` (we do the same for upnp).

While we're at it, we fixe the build so that we are using our c/ppflags.
This  means building with `-O2` rather than `-Os`.

Note that this fixes an issue that is also fixed by #21209.
However, given there are reservations about updating to use a newer libnatpmp source, we should just fix this for now.
2021-03-01 14:00:36 +08:00
Sebastian Falbesoner
8a8c6383f6 zmq test: fix sync-up by matching notification to generated block
It turned out that the "sync up" procedure of repeatedly generating a
block and waiting for a notification once with timeout is too naive in
its current form, as the following scenario could happen:
    - generate block A
    - receive notification, timeout happens -> repeat procedure
    - generate block B
    - node publishes block A notification
    - receive notification, we receive the one caused by block A
      -> sync-up procedure is completed
    - node publishes block B
    - the actual test starts
    - on the first notification reception, one caused by block B is received,
      rather than the one actually caused by test code, leading to failure

This change ensures that after each test block generation, we wait for
the notification that is actually caused by that block and ignore others
from possibly earlier blocks.

Co-authored-by: Jon Atack <jon@atack.com>
2021-03-01 01:31:41 +01:00
Ethan Heilman
9bd3f35003 build: adds switch for disabling random base addresses in MSVC 2021-02-28 14:49:09 -05:00
Jon Atack
6242beeb06 Hoist repeated translated strings to RPCConsole struct members
and add missing braces to the touched conditionals.

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-02-28 19:14:35 +01:00
Jon Atack
0f035c12fb RPCConsole::updateDetailWidget: convert strings to translated strings
and in the touched lines:

- replace 2 occurrences of `== ""` with `isEmpty()`

- replace an unneeded `+=` with `=`
2021-02-28 19:13:53 +01:00
fanquake
e52ce9f2b3 Merge #21286: build: Bump minimum Qt version to 5.9.5
faa06ecc9c build: Bump minimum Qt version to 5.9.5 (Hennadii Stepanov)

Pull request description:

  Close #20104.

ACKs for top commit:
  laanwj:
    Code review ACK faa06ecc9c
  jarolrod:
    ACK faa06ecc9c
  fanquake:
    ACK faa06ecc9c - this should be ok to do now.

Tree-SHA512: 7295472b5fd37ffb30f044e88c39d375a5a5187d3f2d44d4e73d0eb0c7fd923cf9949c2ddab6cddd8c5da7e375fff38112b6ea9779da4fecce6f024d05ba9c08
2021-02-28 13:14:04 +08:00
Sebastian Falbesoner
73e1f7d754 rpc: document optional fields for getchaintxstats result 2021-02-27 20:39:17 +01:00
Hennadii Stepanov
c524dc54bb qt: Fix regression with initial sorting after pr205
QTableView widget must be explicitly sorted after the setModel call.
2021-02-27 17:47:33 +02:00
MarcoFalke
fb67caebe2 Merge #21297: test: feature_blockfilterindex_prune.py improvements
88c4b9b761 test: remove unneeded node from feature_blockfilterindex_prune.py (Jon Atack)
ace3f4cbdf test: improve assertions in feature_blockfilterindex_prune.py (Jon Atack)

Pull request description:

  - improves the assertions
  - removes an unneeded node, reducing from two to one, and some unneeded `extra_arg` code

ACKs for top commit:
  MarcoFalke:
    ACK 88c4b9b761
  brunoerg:
    Tested ACK 88c4b9b761

Tree-SHA512: 295700da3a5f583ee02ae2d184db93cce0e13aba69115d5db07f83e96a66b4b850adaff2c3725b6585799565b9ee654b1fee8a6245eaba8c21e1cb5ce524eb2b
2021-02-27 13:11:58 +01:00
Wladimir J. van der Laan
5ba5becbb5 Merge #21272: guix: Passthrough SDK_PATH into container
13a9fd11a5 guix: Passthrough SDK_PATH into container (Carl Dong)

Pull request description:

  This is a usability improvement for Guix builders so that they don't have to extract the Xcode tarball into `depends/SDKs` every time.

  Inspiration: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-778639698

ACKs for top commit:
  laanwj:
    Tested ACK 13a9fd11a5

Tree-SHA512: 63392d537e48a0da9f0ee04a929613b139bef1ac5643187871c9ea5376afd2a3d95df0f5e0950ae0eccd2813b166667be98401e5a248ae9c187fe4e84e54d427
2021-02-26 20:26:21 +01:00
Anthony Towns
eeeafb324e net_processing: move AddToCompactExtraTransactions into PeerManagerImpl
Allows making vExtraTxnForCompact and vExtraTxnForCompactIt member vars
instead of globals.
2021-02-27 01:08:09 +10:00
Anthony Towns
f8c0688b94 scripted-diff: Update txorphanage naming convention
-BEGIN VERIFY SCRIPT-
sed -i 's/mapOrphanTransactionsByPrev/m_outpoint_to_orphan_it/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/mapOrphanTransactions/m_orphans/g' src/txorphanage.h src/txorphanage.cpp src/net_processing.cpp src/test/denialofservice_tests.cpp
sed -i 's/g_orphan_list/m_orphan_list/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/g_orphans_by_wtxid/m_wtxid_to_orphan_it/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/nMaxOrphans/max_orphans/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/COrphanTx/OrphanTx/g' src/txorphanage.h src/txorphanage.cpp src/test/denialofservice_tests.cpp
-END VERIFY SCRIPT-
2021-02-27 01:08:09 +10:00
Anthony Towns
6bd4963c06 txorphanage: Move functions and data into class
Collects all the orphan handling globals into a single member var in
net_processing, and ensures access is encapuslated into the interface
functions. Also adds doxygen comments for methods.
2021-02-27 01:07:55 +10:00
Anthony Towns
03257b832d txorphanage: Extract EraseOrphansForBlock
Extract code that erases orphans when a new block is found into
EraseOrphansForBlock.
2021-02-27 00:31:09 +10:00
Anthony Towns
3c4c3c2fdd net_processing: drop AddOrphanTx
All the interesting functionality of AddOrphanTx is already in other
functions, so call those functions directly in the one place that
AddOrphanTx was used.
2021-02-27 00:30:11 +10:00
Anthony Towns
26d1a6ccd5 denialofservices_tests: check txorphanage's AddTx
Rather than checking net_processing's internal implementation of
AddOrphanTx, test txorphanage's exported AddTx interface. Note that
this means AddToCompactExtraTransactions is no longer tested here.
2021-02-26 23:55:10 +10:00
Anthony Towns
1041616d7e txorphanage: Extract OrphanageAddTx
Extract code from AddOrphanTx into OrphanageAddTx.
2021-02-26 23:55:10 +10:00
Anthony Towns
f294da7274 txorphanage: Extract GetOrphanTx
Extract orphan lookup code into GetOrphanTx function.
2021-02-26 23:55:10 +10:00
Anthony Towns
83679ffc60 txorphanage: Extract HaveOrphanTx
Extract some common code into HaveOrphanTx function.
2021-02-26 23:55:10 +10:00
Anthony Towns
ee135c8d5b txorphanage: Extract AddChildrenToWorkSet
Extract some common code into AddChildrenToWorkSet function.

(It's a hard knock life)
2021-02-26 23:55:10 +10:00
Anthony Towns
38a11c355a txorphanage: Add lock annotations
EraseOrphansFor was called both with and without g_cs_orphans held,
correct that so that it's always called with it already held.

LimitOrphanTxSize was always called with g_cs_orphans held, so
add annotations and don't lock it a second time.
2021-02-26 23:55:10 +10:00
Anthony Towns
81dd57e5b1 txorphanage: Pass uint256 by reference instead of value 2021-02-26 23:55:07 +10:00
Anthony Towns
9d5313df7e move-only: Add txorphanage module
This module captures orphan tracking code for tx relay.

Can be reviewed with --color-moved=dimmed-zebra
2021-02-26 23:55:03 +10:00
Wladimir J. van der Laan
8d37841cdf Merge #21277: wallet: listdescriptors uses normalized descriptor form
a69c3b35f8 wallet: listdescriptors uses normalized descriptor form (Ivan Metlushko)

Pull request description:

  Rationale: show importable descriptors with `listdescriptors` RPC

  It uses #19136 to derive xpub at the last hardened step.

  **Before**:
  ```
  [
      {
        "desc": "wpkh(tpubD6NzVbkrYhZ4YUQRJL49TWw1VR5v3QKUNYaGGMUfJUm19x5ZqQ2hEiPiYbAQvD2nHoPGQGPg3snLPM8sjmYpvx7XQhkmyfk8xhsUwKbXzyh/84'/1'/0'/0/*)#p4cn3erf",
        "timestamp": 1613982591,
        "active": true,
        "internal": false,
        "range": [
          0,
          999
        ],
        "next": 0
      },
      ...
  ]
  ```

  **After**:
  ```
  [
    {
      "desc": "wpkh([d4ade89c/84'/1'/0']tpubDDUEYcVXy6Vh5meHvcXN3sAr4k3fWwLZGpAHbkAHL8EnkDxp4d99CjNhJHfM2fUJicANvAKnCZS6XaVAgwAeKYc1KesGCN5qbQ25qQHrRxM/0/*)#8wq8rcft",
      "timestamp": 1613982591,
      "active": true,
      "internal": false,
      "range": [
        0,
        999
      ],
      "next": 0
    },
    ...
  ]
  ```

ACKs for top commit:
  achow101:
    ACK a69c3b35f8

Tree-SHA512: 4f92e726cb8245aa0b520729cfd272945f0c66830f0555544e0618883aca516318543fa6ab1862058c64b4e4ed54ad1da953e032f4846eef7454122031c1b005
2021-02-26 10:08:02 +01:00
dscotese
3a0446fad4 script: Add explanatory comment to tc.sh 2021-02-25 19:09:46 -08:00
Sylvain Goumy
303cfc6227 allow darkmode on macos build 2021-02-25 22:21:22 +01:00
Uplab
78f75a2d60 Allow icon colorization on mac os to better support dark mode 2021-02-25 22:21:22 +01:00
Jon Atack
88c4b9b761 test: remove unneeded node from feature_blockfilterindex_prune.py 2021-02-25 18:05:59 +01:00
Jon Atack
ace3f4cbdf test: improve assertions in feature_blockfilterindex_prune.py 2021-02-25 15:10:19 +01:00
MarcoFalke
e49117470b Merge bitcoin-core/gui#226: Add "Last Block" and "Last Tx" rows to peer details area
70d3c5d0b9 gui: add "Last Block" (CNodeStats::nLastBlockTime) to peer details (Jon Atack)
a21be7c401 gui: add "Last Tx" (CNodeStats::nLastTXTime) to peer details (Jon Atack)
4dc2fd6c37 qt: add RPCConsole::TimeDurationField helper, call systime only once (Jon Atack)

Pull request description:

  - add `RPCConsole::TimeDurationField` helper to replace repeated code and call system time only once in `RPCConsole::updateDetailWidget`
  - add "Last Tx" (`CNodeStats::nLastTXTime`) field to peer details
  - add "Last Block" (`CNodeStats::nLastBlockTime`) field to peer details

ACKs for top commit:
  hebasto:
    re-ACK 70d3c5d0b9

Tree-SHA512: 2611b71fd358ba9ffb6a6206275c08ecb5e683b6f87d022faaaba9802a15030430113afdb434814a9ae2681d04429aa733164bc110b64337ceaae12a0420f4f1
2021-02-25 14:49:47 +01:00
MarcoFalke
c0e44ee8e4 Merge #21254: test: Avoid connecting to real network when running tests
fa730e9157 test: Avoid connecting to real network when running tests (MarcoFalke)
fa1b713941 test: Assume node is running in subtests (MarcoFalke)

Pull request description:

  Introduced in #19884

ACKs for top commit:
  Sjors:
    ACK fa730e9157

Tree-SHA512: fe132a9ffe2fae1ab16857a3dec9839526fdf74d27a1ae794fbffca8356f639c4b916dc888b260281e9cc793916706c18d1687ebb5a076d4e1c481d218d308d3
2021-02-25 14:44:22 +01:00
MarcoFalke
cac10e66d2 Merge #21264: fuzz: Two scripted diff renames
fae216a73d scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext (MarcoFalke)
fa4fbec03e scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTION (MarcoFalke)

Pull request description:

  Split out two renames from #21003:

  * `PROVIDE_FUZZ_MAIN_FUNCTION`. *Reason*: This in only used by fuzzing, so the name should indicate that.
  * `MakeNoLogFileContext`. *Reason*: Better reflects what the helper does. Also, prepares it to be used in non-fuzz tests in the future.

ACKs for top commit:
  practicalswift:
    cr ACK fae216a73d: scripted-diff looks correct

Tree-SHA512: e5d347746f5da72b0c86fd4f07ac2e4b3016e88e8c97a830c73bd79d0af6d0245fe7712487fc20344d6cc25958941716c1678124a123930407e3a437265b71df
2021-02-25 14:42:48 +01:00
MarcoFalke
1b1d8bde1c Merge #21252: test: Add missing wait for sync to feature_blockfilterindex_prune
fa560cc6c4 test: Intermittent issue in feature_blockfilterindex_prune (MarcoFalke)

Pull request description:

  https://cirrus-ci.com/task/4962244553342976?command=ci#L5131

  The index is built in a background thread, so we have to wait for it.

ACKs for top commit:
  jonatack:
    ACK fa560cc6c4

Tree-SHA512: e7a246fe43a28511581fe34b1f5a85303b1874b2535330afc0405269cce7306984ecc6af389791321e3aa4b224819e89d9b89dd5bc080d60baa20bd007412787
2021-02-25 14:32:46 +01:00
Jon Atack
70d3c5d0b9 gui: add "Last Block" (CNodeStats::nLastBlockTime) to peer details 2021-02-25 12:25:07 +01:00
Jon Atack
a21be7c401 gui: add "Last Tx" (CNodeStats::nLastTXTime) to peer details 2021-02-25 11:33:19 +01:00
Jon Atack
4dc2fd6c37 qt: add RPCConsole::TimeDurationField helper, call systime only once 2021-02-25 11:32:51 +01:00
Wladimir J. van der Laan
2059d32edb Merge #21200: test: Speed up rpc_blockchain.py by removing miniwallet.generate()
faa137eb9e test: Speed up rpc_blockchain.py by removing miniwallet.generate() (MarcoFalke)
fa1fe80c75 test: Change address type from P2PKH to P2WSH in rpc_blockchain (MarcoFalke)
fa4d8f3169 test: Cache 25 mature coins for ADDRESS_BCRT1_P2WSH_OP_TRUE (MarcoFalke)
fad25153f5 test: Remove unused bug workaround (MarcoFalke)
faabce7d07 test: Start only the number of nodes that are needed (MarcoFalke)

Pull request description:

  Speed up various tests:

  * Remove unused nodes, which only consume time on start/stop
  * Remove unused "bug workarounds"
  * Remove the need for `miniwallet.generate()` by adding `miniwallet.scan_blocks()`. (On my system, with valgrind, generating 105 blocks takes 3.31 seconds. Rescanning 5 blocks takes 0.11 seconds.)

ACKs for top commit:
  laanwj:
    Code review ACK faa137eb9e

Tree-SHA512: ead1988d5aaa748ef9f8520af1e0bf812cf1d72e281ad22fbd172b7306d850053040526f8adbcec0b9a971c697a0ee7ee8962684644d65b791663eedd505a025
2021-02-25 10:13:39 +01:00
MarcoFalke
8ca6bd0dac Merge bitcoin-core/gui#223: qt: Re-add and rename transaction "Edit Label" action
5440c07457 qt: Rename "Edit label" to "Edit address label" (Wladimir J. van der Laan)
22664d6287 Revert "qt: Remove Transactionview Edit Label Action" (Wladimir J. van der Laan)

Pull request description:

  This reverts PR #211.

  I disagree with this change, I use the functionality a lot, it was the primary way I used to organize and edit transactions labels and am sad to see this go.

  > you can edit a sending address in the send tab Address Book

  Using the address book should not be encouraged at all! A while ago it was even proposed to remove it. There's rarely need to scroll through all historical addresses used and unused. The transaction list does just fine for this.

  > While all other actions apply directly to the selected transaction, the Edit Label action applies to the selected transaction's address.

  **In practice** when bitcoin is used in the commonly advised way, generate a new address for each transaction, those are equivalent though.

  I doubt I (and **luke-jr**) will be the only users that will stumblle on this. Further discussion here: https://github.com/bitcoin-core/gui/pull/211#issuecomment-784755998

ACKs for top commit:
  hebasto:
    ACK 5440c07457, verified that 22664d6287 is a clean revert of 8f9644890a.

Tree-SHA512: 3a86a730279bc454d0bd25d874dbfb6b1c0492480e66c3164e7c60d8658d622d4522de11bf8564876dc3ee056b53db71ecbe8a37281bf25d41a27e6e0d72ad8f
2021-02-25 09:09:34 +01:00
MarcoFalke
09bc7bfed1 Merge bitcoin-core/gui#214: qt: Disable requests context menu actions when appropriate
bb3da8fe41 qt: Disable requests context menu actions when appropriate (Jarol Rodriguez)

Pull request description:

  The recent requests table will allow you to copy data points even if they do not exist. This PR implements checks to disable the `copy label`, `copy message`, and `copy amount` context menu actions if the respective fields are empty. This brings the recent requests table context menu behavior closer to the behavior seen in the transaction view.

  On a payment request entry which does not have a value for label, message, or amount:
  | Master        | PR               |
  | ----------- | ----------- |
  |<img width="169" alt="Screen Shot 2021-02-19 at 1 22 28 AM" src="https://user-images.githubusercontent.com/23396902/108466086-167adc00-7251-11eb-8bd6-13984042bdb3.png">| <img width="169" alt="Screen Shot 2021-02-19 at 1 21 49 AM" src="https://user-images.githubusercontent.com/23396902/108466185-3e6a3f80-7251-11eb-9dd8-492ed07fd638.png">|

  `copy URI` never needs to be disabled as an entry in the recent requests table must have a URI even if it doesn't have a label, message, or amount. #213 will add a `copy address` context menu action. This also does not need a check as an entry must be associated with an address.

  Below are some more examples of how this PR will behave:
  **Has Label, Message, and Amount**
  <img width="780" alt="Screen Shot 2021-02-19 at 12 05 38 AM" src="https://user-images.githubusercontent.com/23396902/108466507-c18b9580-7251-11eb-8875-f3aeb9c4c8e9.png">

  **Has Label and Amount, but no Message**
  <img width="780" alt="Screen Shot 2021-02-19 at 12 05 58 AM" src="https://user-images.githubusercontent.com/23396902/108466421-9b65f580-7251-11eb-97eb-a3bfaa21fa7d.png">

ACKs for top commit:
  hebasto:
    re-ACK bb3da8fe41

Tree-SHA512: d98f1a6e05ebf6d9d056bc5754aca78ca7ecda93f497dba88187b947ca4a261eb7dc5e8c872956315acaa0008cc39320fb5806e17117e0c873090ad917ca4a3d
2021-02-25 09:02:44 +01:00
MarcoFalke
434065a483 Merge bitcoin-core/gui#219: qt: Prevent the main window popup menu
ca5bd1c8e5 qt: Prevent the main window popup menu (Hennadii Stepanov)

Pull request description:

  https://github.com/bitcoin/bitcoin/issues/11168 is not fixed by https://github.com/bitcoin/bitcoin/pull/11169 completely, as users are allowed to right click on the menu bar:

  ![Screenshot from 2021-02-23 14-18-24](https://user-images.githubusercontent.com/32963518/108842753-699eb700-75e2-11eb-92ec-3aff9aa80bd4.png)

  This PR moves the context menu prohibition from `QToolBar` instance to its parent `BitcoinGUI` instance, which is derived from `QMainWindow`.

ACKs for top commit:
  jarolrod:
    ACK ca5bd1c8e5, tested on Ubuntu 20.04 Qt 5.12. Confirming that I can replicate the behavior described on `master` and this `PR` fixes it.
  leonardojobim:
    tACK ca5bd1c8e5

Tree-SHA512: a654ecf7ee35bb271df039be77077c1e1f9514e332587ba8622cea18da6a5b3ae8a7eb421e404ec5993c31a2f4d028e0e456fcc01facdbf61a2bc3b1e8423982
2021-02-25 08:59:59 +01:00
MarcoFalke
fa80e10d94 test: Add feature_taproot.py --previous_release 2021-02-25 08:51:11 +01:00
Sjors Provoost
85ccffa266 test: move releases download incantation to README 2021-02-25 08:50:34 +01:00
Sjors Provoost
29d6b1da2a test: previous releases: add v0.20.1
Can be reviewed with --ignore-all-space
2021-02-25 08:50:34 +01:00
MarcoFalke
fa7ff0790e rpc: Properly document submitblock return value
Can be reviewed with --ignore-all-space
2021-02-25 08:21:28 +01:00
MarcoFalke
fae542c28b rpc: Properly document getblocktemplate return value
Can be reviewed with --ignore-all-space
2021-02-25 08:21:26 +01:00
MarcoFalke
fabaccf031 rpc: Properly document scantxoutset return value
Can be reviewed with --ignore-all-space
2021-02-25 08:09:20 +01:00
MarcoFalke
faa2059547 rpc: Properly document gettxout return value
Can be reviewed with --ignore-all-space
2021-02-25 08:06:26 +01:00
Carl Dong
c33b199456 guix: Bump glibc and linux-headers
Bump glibc and linux-headers to match those of our Gitian counterparts.

We also require a glibc >= 2.28 for the test-symbol-check scripts to
work properly.

The default BASE-GCC-FOR-LIBC also has to be bumped since glibc 2.31
requires a gcc >= 6.2
2021-02-24 21:14:58 -05:00
Carl Dong
65363a1bd8 guix: Rebase on 95aca2991b (1.2.0-12.dffc918)
A few changes which are useful for us:

1. 'gnu: cross-gcc-arguments: Enable 128 bit long double for POWER9.' is
   now merged into master.
2. gnutls is bumped to 3.6.15 and the temporal test failure in
   status-request-revoked is fixed. Note that this does not fix the case
   where one has installed Guix v1.2.0 and is running a substitute-less
   bootstrap build, since the `guix time-machine` command itself has a
   dependency on gnutls v3.6.12 (the one with the broken test) and will
   thus try to build it before attempting to jump forwards in time. This
   does however, mean that those who build a version of Guix that also
   contains this fix will not go backwards in time to build the broken
   gnutls v3.6.12.
2021-02-24 21:14:58 -05:00
fanquake
56f06a9863 Merge #21271: guix: Explicitly set umask in build container
d98f4593cf guix: Explicitly set umask in build container (Carl Dong)

Pull request description:

  Opened as a separate PR to fix non-reproducibility found through testing here: https://github.com/bitcoin/bitcoin/pull/21089#issuecomment-783549633

  Many thanks to everyone who helped find this!

ACKs for top commit:
  laanwj:
    ACK d98f4593cf
  fanquake:
    ACK d98f4593cf - I'm seeing matching hashes.

Tree-SHA512: ea339c3902f2f4dea32e8ef5cc675a1df0679530881260ae999aaaf7339d5b12c46e01e58677cbb079f33e573ad105e2b443a835f3e944ef8e943a25f83027f1
2021-02-25 08:22:04 +08:00
MarcoFalke
b54a10e777 Merge #21292: build, doc: Fix configure script output indentation and typos
8b08d0f2a5 build, doc: Fix configure script output indentation and typos (Hennadii Stepanov)

Pull request description:

  This PR is follow up of #16546, that breaks the `configure` script output indentation for gui/qt/qr lines:
  ```
  Options used to compile and link:
    external signer = no
    multiprocess    = no
    with libs       = yes
    with wallet     = yes
      with sqlite   = yes
      with bdb      = yes
      with gui / qt = yes
    with qr         = yes
    with zmq        = yes
    with test       = yes
  ...
  ```

  With this PR:
  ```
  Options used to compile and link:
    external signer = no
    multiprocess    = no
    with libs       = yes
    with wallet     = yes
      with sqlite   = yes
      with bdb      = yes
    with gui / qt   = yes
      with qr       = yes
    with zmq        = yes
    with test       = yes
  ...
  ```

  Also typos are fixed.

ACKs for top commit:
  Sjors:
    utACK 8b08d0f2a5
  vasild:
    ACK 8b08d0f2a5

Tree-SHA512: 46dfcfb754192dbcc080348781327d1714e2f9a696f3ed9252746b426e3afc628d84adb197ba3b8080eacaa6053ccac07e670998930ae92cef8ed713dd457c10
2021-02-24 15:31:01 +01:00
Wladimir J. van der Laan
9307c588d0 Merge #21293: test: Replace accidentally placed bit-OR with logical-OR
df8f2a11dc test: Replace accidentally placed bit-OR with logical-OR (Hennadii Stepanov)

Pull request description:

  This PR is a follow up of #19698.

ACKs for top commit:
  glozow:
    utACK df8f2a11dc

Tree-SHA512: 36aba3e952850deafe78dd39775a568e89e867c8a352f511f152bce7062f614f5bb4f23266dbb33da5292c9ee6da5ccefce117e3168621c71d2140c8e7f58460
2021-02-24 13:19:56 +01:00
Hennadii Stepanov
df8f2a11dc test: Replace accidentally placed bit-OR with logical-OR 2021-02-24 12:57:03 +02:00
Hennadii Stepanov
8b08d0f2a5 build, doc: Fix configure script output indentation and typos 2021-02-24 12:28:35 +02:00
Wladimir J. van der Laan
b59f2787e5 Merge #18017: txmempool: split epoch logic into class
fd6580e405 [refactor] txmempool: split epoch logic into class (Anthony Towns)

Pull request description:

  Splits the epoch logic introduced in #17925 into a separate class.

  Uses clang's thread safety annotations and encapsulates the data more strongly to reduce chances of bugs from API misuse.

ACKs for top commit:
  jonatack:
    ACK fd6580e405 using clang thread safety annotations looks like a very good idea, and the encapsulation this change adds should improve robustness (and possible unit test-ability) of the code. Verified that changing some of the locking duly provoked build-time warnings with Clang 9 on Debian and that small changes in the new `Epoch` class were covered by failing functional test assertions in `mempool_updatefromblock.py`, `mempool_resurrect.py`, and `mempool_reorg.py`
  hebasto:
    re-ACK fd6580e405, since my [previous](https://github.com/bitcoin/bitcoin/pull/18017#pullrequestreview-569619362) review:

Tree-SHA512: 7004623faa02b56639aa05ab7a078320a6d8d54ec62d8022876221e33f350f47df51ddff056c0de5be798f8eb39b5c03c2d3f035698555d70abc218e950f2f8c
2021-02-24 09:57:21 +01:00
Wladimir J. van der Laan
5440c07457 qt: Rename "Edit label" to "Edit address label"
This makes it more specific what the action refers to.
(Suggested by Pieter Wuille)
2021-02-24 08:56:41 +01:00
Wladimir J. van der Laan
22664d6287 Revert "qt: Remove Transactionview Edit Label Action"
This reverts commit 8f9644890a.
2021-02-24 08:35:55 +01:00
fanquake
587c986ccf Merge #21285: wallet: fix doc typo in signer option
da30c1bb05 wallet: fix doc typo in signer option (William Casarin)

Pull request description:

ACKs for top commit:
  0xB10C:
    ACK da30c1bb05
  darosior:
    ACK da30c1bb05

Tree-SHA512: 1f4ba501121101bbe94a18a0276df6a3592123548110ed5d1e4e1937b75c9a832bc3f0d6fa26bab69b3304526ef0548b2540d446e05a6402793321a34f508b09
2021-02-24 06:49:00 +08:00
Hennadii Stepanov
faa06ecc9c build: Bump minimum Qt version to 5.9.5 2021-02-23 21:34:42 +02:00
William Casarin
da30c1bb05 wallet: fix doc typo in signer option
Signed-off-by: William Casarin <jb55@jb55.com>
2021-02-23 11:05:13 -08:00
Wladimir J. van der Laan
2e8116149c Merge #21250: build: make HAVE_O_CLOEXEC available outside LevelDB (bugfix)
9bac71350d build: make HAVE_O_CLOEXEC available outside LevelDB (bugfix) (Sebastian Falbesoner)
584fd91d2d init: only use pipe2 if availabile, check in configure (Sebastian Falbesoner)

Pull request description:

  The result of the O_CLOEXEC availability check is currently only set in the Makefile and passed to LevelDB (see `LEVELDB_CPPFLAGS_INT` in `src/Makefile.leveldb.include`), but not defined to be used in our codebase. This means that code within the preprocessor conditional `#if HAVE_O_CLOEXEC` was actually never compiled. On the master branch this is currently used for pipe creation in `src/shutdown.cpp`, PR #21007 moves this part to a new module (I found the issue while testing that PR).

  The fix is similar to the one in #19803, which solved the same problem for HAVE_FDATASYNC.

  In the course of working on the PR it turned out that pipe2 is not available an all platforms, hence a configure check and a corresponding define HAVE_PIPE2 is introduced and used.

  The PR can be tested by anyone with a system that has pipe2 and O_CLOEXEC available by putting gibberish into the HAVE_O_CLOEXEC block: on master, everything should compile fine, on PR, the compiler should abort with an error. At least that's my naive way of testing preprocessor logic, happy to hear more sophisticated ways :-)

ACKs for top commit:
  laanwj:
    Code review ACK 9bac71350d

Tree-SHA512: aec89faf6ba52b6f014c610ebef7b725d9e967207d58b42a4a71afc9f1268fcb673ecc85b33a2a3debba8105a304dd7edaba4208c5373fcef2ab83e48a170051
2021-02-23 18:56:44 +01:00
MarcoFalke
1b045b5eef Merge #21053: rpc, test: document {previous,next}blockhash as optional
ba7e17e073 rpc, test: document {previous,next}blockhash as optional (Sebastian Falbesoner)

Pull request description:

  This PR updates the result help of the following RPCs w.r.t. the `previousblockhash` and `nextblockhash` fields:
  - getblockheader
  - getblock

  Also adds trivial tests on genesis block (should not contain "previousblockhash") and best block (should not contain "nextblockhash").

Top commit has no ACKs.

Tree-SHA512: ef42c5c773fc436e1b4a67be14e2532e800e1e30e45e54a57431c6abb714d2c069c70d40ea4012d549293b823a1973b3f569484b3273679683b28ed40abf46bb
2021-02-23 18:28:23 +01:00
Carl Dong
d98f4593cf guix: Explicitly set umask in build container
A difference in system umask value can cause non-determinism when zip
archives are produced. Set it to a reasonable default.
2021-02-23 12:24:17 -05:00
Wladimir J. van der Laan
a9335e4f12 Merge #16546: External signer support - Wallet Box edition
f75e0c1edd doc: add external-signer.md (Sjors Provoost)
d4b0107d68 rpc: send: support external signer (Sjors Provoost)
245b4457cf rpc: signerdisplayaddress (Sjors Provoost)
7ebc7c0215 wallet: ExternalSigner: add GetDescriptors method (Sjors Provoost)
fc5da520f5 wallet: add GetExternalSigner() (Sjors Provoost)
259f52cc33 test: external_signer wallet flag is immutable (Sjors Provoost)
2655197e1c rpc: add external_signer option to createwallet (Sjors Provoost)
2700f09c41 rpc: signer: add enumeratesigners to list external signers (Sjors Provoost)
07b7c940a7 rpc: add external signer RPC files (Sjors Provoost)
8ce7767071 wallet: add ExternalSignerScriptPubKeyMan (Sjors Provoost)
157ea7c614 wallet: add external_signer flag (Sjors Provoost)
f3e6ce78fb test: add external signer test (Sjors Provoost)
8cf543f96d wallet: add -signer argument for external signer command (Sjors Provoost)
f7eb7ecc67 test: framework: add skip_if_no_external_signer (Sjors Provoost)
87a97941f6 configure: add --enable-external-signer (Sjors Provoost)

Pull request description:

  Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).

  This PR lets `bitcoind` call an arbitrary command `-signer=<cmd>`, e.g. a hardware wallet driver,  where it can fetch public keys, ask to display an address, and sign a transaction (using PSBT under the hood).

  It's design to work with https://github.com/bitcoin-core/HWI, which supports multiple hardware wallets. Any command with the same arguments and return values will work. It simplifies the manual procedure described [here](https://github.com/bitcoin-core/HWI/blob/master/docs/bitcoin-core-usage.md).

  Usage is documented in [doc/external-signer.md](
  https://github.com/Sjors/bitcoin/blob/2019/08/hww-box2/doc/external-signer.md), which also describes what protocol a different signer binary should conform to.

  Use `--enable-external-signer` to opt in, requires Boost::Process:

  ```
  Options used to compile and link:
    with wallet     = yes
    with gui / qt   = no
    external signer = yes
  ```

  It adds the following RPC methods:
  * `enumeratesigners`: asks <cmd> for a list of signers (e.g. devices) and their master key fingerprint
  * `signerdisplayaddress <address>`:  asks <cmd> to display an address

  It enhances the following RPC methods:
  * `createwallet`: takes an additional `external_signer` argument and fetches keys from device
  * `send`: automatically sends transaction to device and waits

  Usage TL&DR:
  * clone HWI repo somewhere and launch `bitcoind -signer=../HWI/hwi.py`
  * check if you can see your hardware device: `bitcoin-cli enumeratesigners`
  * create wallet and auto import keys `bitcoin-cli createwallet "hww" true true "" true true true`
  * display address on device: `bitcoin-cli signerdisplayaddress ...`
  * to spend, use `send` RPC and approve transaction on device

  Prerequisites:
  - [x] #21127 load wallet flags before everything else
  - [x] #21182 remove mostly pointless BOOST_PROCESS macro

  Potentially useful followups:
  - GUI support: bitcoin-core/gui#4
  - bumpfee support
  - (automatically) verify (a subset of) keys on the device after import, through message signing

ACKs for top commit:
  laanwj:
    re-ACK f75e0c1edd

Tree-SHA512: 7db8afd54762295c1424c3f01d8c587ec256a72f34bd5256e04b21832dabd5dc212be8ab975ae3b67de75259fd569a561491945750492f417111dc7b6641e77f
2021-02-23 17:56:43 +01:00
Jarol Rodriguez
bb3da8fe41 qt: Disable requests context menu actions when appropriate
The recent requests table will allow you to copy data points even if they do not exist.
This PR implements checks to disable the 'copy label', 'copy message', and 'copy amount' context menu action if the respective fields are empty.
2021-02-23 11:38:22 -05:00
MarcoFalke
78effb37f3 Merge #21222: log: Clarify log message when file does not exist
faf48f20f1 log: Clarify log message when file does not exist (MarcoFalke)

Pull request description:

  Shorter and broader alternative to #21181

  Rendered diff:

  ```diff
  @@ -1,4 +1,4 @@
  -Bitcoin Core version v21.99.0-db656db2ed5a (release build)
  +Bitcoin Core version v21.99.0-faf48f20f196 (release build)
   Qt 5.15.2 (dynamic), plugin=wayland (dynamic)
   No static plugins.
   Style: adwaita / Adwaita::Style
  @@ -24,8 +24,8 @@ scheduler thread start
   Using wallet directory /tmp/test_001/regtest/wallets
   init message: Verifying wallet(s)...
   init message: Loading banlist...
  -ERROR: DeserializeFileDB: Failed to open file /tmp/test_001/regtest/banlist.dat
  -Invalid or missing banlist.dat; recreating
  +Missing or invalid file /tmp/test_001/regtest/banlist.dat
  +Recreating banlist.dat
   SetNetworkActive: true
   Failed to read fee estimates from /tmp/test_001/regtest/fee_estimates.dat. Continue anyway.
   Using /16 prefix for IP bucketing
  @@ -63,9 +63,9 @@ Bound to [::]:18444
   Bound to 0.0.0.0:18444
   Bound to 127.0.0.1:18445
   init message: Loading P2P addresses...
  -ERROR: DeserializeFileDB: Failed to open file /tmp/test_001/regtest/peers.dat
  -Invalid or missing peers.dat; recreating
  -ERROR: DeserializeFileDB: Failed to open file /tmp/test_001/regtest/anchors.dat
  +Missing or invalid file /tmp/test_001/regtest/peers.dat
  +Recreating peers.dat
  +Missing or invalid file /tmp/test_001/regtest/anchors.dat
   0 block-relay-only anchors will be tried for connections.
   init message: Starting network threads...
   net thread start

ACKs for top commit:
  jnewbery:
    utACK faf48f20f1
  amitiuttarwar:
    utACK faf48f20f1, 👍 for consistency. also checked where we create / load other `.dat` files, looks good to me.
  practicalswift:
    cr ACK faf48f20f1

Tree-SHA512: 697a728ef2b9f203363ac00b03eaf23ddf80bee043ecd3719265a0d884e8cfe88cd39afe946c86ab849edd1c836f05ec51125f052bdc14fe184b84447567756f
2021-02-23 16:03:19 +01:00
Sjors Provoost
f75e0c1edd doc: add external-signer.md 2021-02-23 14:34:32 +01:00
Sjors Provoost
d4b0107d68 rpc: send: support external signer 2021-02-23 14:34:32 +01:00
Sjors Provoost
245b4457cf rpc: signerdisplayaddress 2021-02-23 14:34:31 +01:00
Sjors Provoost
7ebc7c0215 wallet: ExternalSigner: add GetDescriptors method 2021-02-23 14:34:31 +01:00
Sjors Provoost
fc5da520f5 wallet: add GetExternalSigner() 2021-02-23 14:34:31 +01:00
Sjors Provoost
259f52cc33 test: external_signer wallet flag is immutable 2021-02-23 14:34:31 +01:00
Sjors Provoost
2655197e1c rpc: add external_signer option to createwallet 2021-02-23 14:34:31 +01:00
Sjors Provoost
2700f09c41 rpc: signer: add enumeratesigners to list external signers 2021-02-23 14:34:31 +01:00
Sjors Provoost
07b7c940a7 rpc: add external signer RPC files 2021-02-23 14:34:30 +01:00
Sjors Provoost
8ce7767071 wallet: add ExternalSignerScriptPubKeyMan 2021-02-23 14:34:30 +01:00
Sjors Provoost
157ea7c614 wallet: add external_signer flag 2021-02-23 14:34:30 +01:00
Sjors Provoost
f3e6ce78fb test: add external signer test
Includes a mock to mimick the HWI interace.
2021-02-23 14:34:30 +01:00
Sjors Provoost
8cf543f96d wallet: add -signer argument for external signer command
Create basic ExternalSigner class with contructor. A Signer(<cmd>)
is added to CWallet on load if -signer=<cmd> is set.
2021-02-23 14:34:30 +01:00
MarcoFalke
84f6c695c6 Merge #21274: assumptions: Assume C++17
5e531e6beb assumptions: check C++17 assumption with MSVC (fanquake)
c7b46489f8 assumptions: assume a C++17 compiler (fanquake)

Pull request description:

  This has been the case since #20413.

  This should also enable the check for MSVC. From my reading of https://docs.microsoft.com/en-us/cpp/build/reference/zc-cplusplus?view=msvc-160 and https://devblogs.microsoft.com/cppblog/msvc-now-correctly-reports-__cplusplus/ if we set the `/Zc:__cplusplus` switch in additional options, MSVC will report the correct value for `__cplusplus`. However I have not tested this.

ACKs for top commit:
  laanwj:
    Code review ACK 5e531e6beb
  hebasto:
    ACK 5e531e6beb, checked the MS docs, and AppVeyor build is green.
  practicalswift:
    ACK 5e531e6beb

Tree-SHA512: a4fb525cf5c33abc944c614edb0313a39c8a39a1637a03c09342c15ba0925f4eb037062e65e51b42ade667506b7e554c7159acf86e6b8c35d0a87dd79a6f239b
2021-02-23 14:03:29 +01:00
Hennadii Stepanov
ca5bd1c8e5 qt: Prevent the main window popup menu
By default, a popup menu contains checkable entries for the toolbars
and dock widgets present in the main window. This allows users to
accidentally hide the toolbar.
2021-02-23 14:14:37 +02:00
Wladimir J. van der Laan
c263c3d7d2 Merge #19698: test: apply strict verification flags for transaction tests and assert backwards compatibility
5786a818e1 Verify that all validation flags are backward compatible (gzhao408)
b10ce9aa48 [test] check verification flags are minimal/maximal (gzhao408)
a260c22cad [test] Check for invalid flag combinations (gzhao408)
a7098a2a8d [refactor] use CheckTxScripts, TrimFlags, FillFlags (gzhao408)
7a77727b2f Apply minimal validation flags to tx_invalid tests (gzhao408)
9532591bed [test] add BADTX setting for invalid txns that fail CheckTransaction (gzhao408)
4c06ebf128 [test] fix two witness tests in invalid tests with empty vout (gzhao408)
158a0b268c Apply maximal validation flags to tx_valid tests (gzhao408)
0a76a39b63 [test] fix CSV test missing OP_ADD (gzhao408)
19db590d04 [test] remove unnecessary OP_1s from CSV and CLTV tests (gzhao408)

Pull request description:

  This uses the first 4 commits of #15045, rebased and added some comments. The diff is quite large already and I want to make it easy to review, so I'm splitting it into 2 PRs (transaction and script). Script one is WIP, I'll link it when I open it.

  Interpretation of scripts is dependent on the script verification flags passed in.
  In tests, we should always apply **maximal** verification flags when checking that a transaction is **valid**; any additional flags should invalidate the transaction. A transaction should not be valid because we forgot to include a flag, and we should apply all flags by default.
  We should apply **minimal** verification flags when asserting that a transaction is **invalid**; if verification flags are applied, removing any one of them should mean the transaction is valid.
  New verify flags must be backwards compatible; tests should check backwards compatibility and apply the new flags by default. All `tx_invalid` tests should continue to be invalid with the exact same verify flags. All `tx_valid` tests that don't pass with new flags should _explicitly_ indicate that the flags need to be excluded, and fail otherwise.

  1. Flip the meaning of `verifyFlags` in tx_valid.json to mean _excluded_ verification flags instead of included flags. Edit the test data accordingly.
  2. Trim unneeded flags from tx_invalid.json.
  3. Add check to verify that tx_valid tests have maximal flags and tx_invalid tests have minimal flags.
  4. Add checks to verify that flags are soft forks (#10699) i.e. adding any flag should only decrease the number of acceptable scripts. Test by adding/removing random flags.

ACKs for top commit:
  achow101:
    ACK 5786a818e1
  laanwj:
    ACK 5786a818e1

Tree-SHA512: 19195d8cf3299e62f47dd3443ae4a95430c5c9d497993a18ab80de9e24b1869787af972774993bf05717784879bc4592fdabaae0fddebd437963d8f3c96d9a73
2021-02-23 11:15:24 +01:00
MarcoFalke
d386b54239 Merge bitcoin-core/gui#213: qt: Add Copy Address Action to Payment Requests
e348d7ea2c qt: Add Copy Address Action to Payment Requests (Jarol Rodriguez)

Pull request description:

  Currently, the only way to copy the address of a payment request is to double-click on the payment request and then click on the copy address button. This PR adds a convenient context menu action to copy the address of a payment request.

  | Master        | PR                 |
  | ----------- | ------------ |
  |<img width="169" alt="Screen Shot 2021-02-18 at 8 33 08 PM" src="https://user-images.githubusercontent.com/23396902/108444489-b6703f80-7228-11eb-8684-945fbcd04772.png"> |<img width="169" alt="Screen Shot 2021-02-18 at 8 33 50 PM" src="https://user-images.githubusercontent.com/23396902/108444505-c12ad480-7228-11eb-9eee-473fee877ad7.png">|

ACKs for top commit:
  hebasto:
    re-ACK e348d7ea2c, only suggested changes since my [previous](https://github.com/bitcoin-core/gui/pull/213#pullrequestreview-595520204) review.

Tree-SHA512: 2b75930ca326ef1d695afc1c6f25853ef55d06d20b66c3c3c372188a6cdfa4686c07f9c56824b766e46b660c731f8a9c2e5b935aa26b316fd46f9e396b29b802
2021-02-23 11:09:16 +01:00
Ivan Metlushko
a69c3b35f8 wallet: listdescriptors uses normalized descriptor form 2021-02-23 08:51:01 +01:00
fanquake
5e531e6beb assumptions: check C++17 assumption with MSVC
From my reading of
https://docs.microsoft.com/en-us/cpp/build/reference/zc-cplusplus?view=msvc-160
and
https://devblogs.microsoft.com/cppblog/msvc-now-correctly-reports-__cplusplus/
if we set the `/Zc:__cplusplus` switch in additional options, MSVC will
report the correct value for `__cplusplus`.
2021-02-23 12:51:50 +08:00
fanquake
c7b46489f8 assumptions: assume a C++17 compiler
This has already been the case since #20413.
2021-02-23 12:48:15 +08:00
Jarol Rodriguez
e348d7ea2c qt: Add Copy Address Action to Payment Requests
Currently, the only way to copy the address of a payment request is to double-click on the payment request and then click on the copy address button. This commit adds a convenient context menu action to copy the address of a payment request.
2021-02-22 23:39:39 -05:00
Carl Dong
13a9fd11a5 guix: Passthrough SDK_PATH into container 2021-02-22 22:07:25 -05:00
fanquake
1e7dd584a5 Merge #21263: doc: Clarify that squashing should happen before review
fa1f3a26a7 doc: Clarify that squashing should happen before review (MarcoFalke)

Pull request description:

  Unlike other repos, in our repo code review happens before merge, ideally.

  Thus, rebases, solving merge conflicts and squashing should happen before review, which again happens before merge.

ACKs for top commit:
  theStack:
    ACK fa1f3a26a7
  fanquake:
    ACK fa1f3a26a7

Tree-SHA512: e9222191a6e9cf9867bd1f29982526dd7b746b70dd2cc94f485256ec98ff2d3941c9b40728935e151d13795239334e334b71ad41044909cb2849f57776811a94
2021-02-23 10:55:03 +08:00
fanquake
3371758af4 Merge #21078: guix: only download sources for hosts being built
a6a1b106dc guix: only download sources for hosts being built (fanquake)

Pull request description:

  For example, if a user is only interested in building for Linux, this saves downloading the macOS compiler and additional dependencies, which is meaningful on a slow/poor connection. This will result in a few additional `make` invocations, for the Linux hosts, however this is low overhead, and time-wise irrelevant in terms of the overall build.

ACKs for top commit:
  laanwj:
    Code review ACK a6a1b106dc

Tree-SHA512: 34c916ae6f69fed0d5845690b39111a8bee37208fd727176f375cf5eb4860f512abe12bde2680d697c859b4d50a3bc5688ddca7c2f28f9968fcf358753cf3f6d
2021-02-23 10:51:41 +08:00
fanquake
64343a6110 Merge #21089: guix: Add support for powerpc64{,le}
95990b9f32 guix: Update conservative space requirements (Carl Dong)
5e6df11326 guix: Add support for powerpc64{,le} (Carl Dong)

Pull request description:

  ```
  The new time-machine commit contains a few small changes that make the
  powerpc cross-toolchain work.
  ```

  See this compare to review my custom patches to Guix: 7d6bd44da5...6c9d16db96

ACKs for top commit:
  fanquake:
    ACK 95990b9f32

Tree-SHA512: 464b0fb93d65962d8c27499293edb618d13d18f40d44e3eed96935e86d430666dfb1c5b8a30f99ffdfd17b44514ad88e358977390b689a2e3831d521f6f7b86a
2021-02-23 10:41:35 +08:00
Jarol Rodriguez
67c59ae479 qt: Make warning label look clickable
The warning label shown on the overview page does not look clickable.
This PR makes the warning label look clickable by removing the 'flat' property.
Additionally, the Maximum Width is updated to fix the small hit-box issue.
2021-02-22 15:14:08 -05:00
Carl Dong
a3ba08ba7d validation: Remove global ::{{Precious,Invalidate}Block,ResetBlockFailureFlags} 2021-02-22 11:48:39 -05:00
Carl Dong
4927c9e699 validation: Remove global ::LoadGenesisBlock 2021-02-22 11:48:39 -05:00
Carl Dong
9da106be4d validation: Check chain tip is non-null in CheckFinalTx
...also update comments to remove mention of ::ChainActive()

From: https://github.com/bitcoin/bitcoin/pull/20750#discussion_r579400663

> Also, what about passing a const reference instead of a pointer? I
> know this is only theoretical, but previously if the tip was nullptr,
> then Height() evaluated to -1, now it evaluates to UB
2021-02-22 11:46:37 -05:00
Sebastian Falbesoner
9bac71350d build: make HAVE_O_CLOEXEC available outside LevelDB (bugfix) 2021-02-22 14:17:40 +01:00
Sebastian Falbesoner
584fd91d2d init: only use pipe2 if availabile, check in configure 2021-02-22 14:17:24 +01:00
Wladimir J. van der Laan
7fca189a2a Merge #79: Embed monospaced font
67f26319a0 gui: Add monospaced font settings (Hennadii Stepanov)
22e0114d05 qt: Choose monospaced font in C++ code rather in `*.ui` file (Hennadii Stepanov)
623de12d04 qt: Make GUIUtil::fixedPitchFont aware of embedded font (Hennadii Stepanov)
89e421918e gui: Add Roboto Mono font (Hennadii Stepanov)

Pull request description:

  Qt does not guarantee that the actual applied font matches to the requested one.
  It was noted (https://github.com/bitcoin/bitcoin/pull/16432#issuecomment-514486077):
  > the monospace font looks a bit weird no macOS

  ... because it is _not_ monospaced.
  Also some discrepancies I've noted on Windows while testing Qt 5.15 ([#19716](https://github.com/bitcoin/bitcoin/pull/19716)).

  Of course, we could check the actual font with `QFontInfo`, and try to choose another font.
  But this PR suggests to just embed a monospaced font, and get the GUI look (partially) independent from a platform.

  [Roboto Mono](https://fonts.google.com/specimen/Roboto+Mono) was chosen after discussion with Bitcoin Design community, and due to its [Apache License, Version 2.0](https://fonts.google.com/specimen/Roboto+Mono#license).

  Changes are scoped to the Overview page only.

  ---

  Screenshots on macOS 10.15.6 (images are simulated by code patching):

  - master (ca30d34cf9)
  ![Screenshot from 2020-09-03 14-10-03](https://user-images.githubusercontent.com/32963518/92107902-30357d80-edef-11ea-8a4f-b4c758eebf66.png)

  - this PR (3fdd5b6bd17a679d6e3876682266092159c52d59)
  ![Screenshot from 2020-09-03 15-41-36](https://user-images.githubusercontent.com/32963518/92116277-4b5aba00-edfc-11ea-8cb9-22fc44460bfb.png)

  ---

  More screenshots added after https://github.com/bitcoin-core/gui/pull/79#issuecomment-782909149:

  - Linux Mint 20.1 + Cinnamon DE

  ![DeepinScreenshot_select-area_20210221205410](https://user-images.githubusercontent.com/32963518/108635739-b327be80-7489-11eb-8851-ac89f61199ee.png)

  - Windows 10 (with depends)

  ![DeepinScreenshot_select-area_20210221205056](https://user-images.githubusercontent.com/32963518/108635741-b6bb4580-7489-11eb-8b6b-66be5551eb8c.png)

  - macOS Big Sur (with depends)

  ![DeepinScreenshot_select-area_20210221202917](https://user-images.githubusercontent.com/32963518/108635746-bd49bd00-7489-11eb-8cd2-cf4bb2273a6d.png)

ACKs for top commit:
  laanwj:
    Tested ACK 67f26319a0

Tree-SHA512: a59775570b8ce314669ede50a0b69f53e8a47a41e7eea428835013240f0ce9afcff6e4c258895455b56806417ed877e5b7a9522f1904e95a5f435db8ccf6078c
2021-02-22 13:01:06 +01:00
Wladimir J. van der Laan
84af29337c Merge #21255: contrib: run test-symbol-check for RISC-V
ccb7b6a976 contrib: run test-symbol check for RISCV (fanquake)

Pull request description:

  Now that we are using Focal for gitian building (glibc 2.31), we can use a newer introduced symbol, and include RISC-V in this test.

ACKs for top commit:
  laanwj:
    Code review re-ACK ccb7b6a976

Tree-SHA512: 4ddaafc0b6cced49a504847ed8da02b0a514eca69014fdc2ad01068c6f8ddf9ad992525f47b2a182c3b4a5845f8dacfe5a7e4acf2f303a5dab6a70daa3b89509
2021-02-22 12:31:09 +01:00
MarcoFalke
fae216a73d scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext
-BEGIN VERIFY SCRIPT-
 # Rename
 sed -i -e 's/MakeFuzzingContext/MakeNoLogFileContext/g' $(git grep -l MakeFuzzingContext)
 # Bump the copyright of touched files in this scripted diff to avoid touching them again later
 ./contrib/devtools/copyright_header.py update ./src/test/fuzz/
-END VERIFY SCRIPT-
2021-02-22 10:27:22 +01:00
MarcoFalke
fa4fbec03e scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTION
-BEGIN VERIFY SCRIPT-
sed -i -e 's/PROVIDE_MAIN_FUNCTION/PROVIDE_FUZZ_MAIN_FUNCTION/g' $(git grep -l PROVIDE_MAIN_FUNCTION)
-END VERIFY SCRIPT-
2021-02-22 10:27:17 +01:00
MarcoFalke
fa1f3a26a7 doc: Clarify that squashing should happen before review 2021-02-22 09:53:01 +01:00
MarcoFalke
34d7030063 Merge #21202: [validation] Two small clang lock annotation improvements
25c57d6409 [doc] Add a note about where lock annotations should go. (Amiti Uttarwar)
ad5f01b960 [validation] Move the lock annotation from function definition to declaration (Amiti Uttarwar)

Pull request description:

  Based on reviewing #21188

  the first commit switches the lock annotations on `CheckInputScripts` to be on the function declaration instead of on the function definition. this ensures that all call sites are checked, not just ones that come after the definition.

  the second commit adds a note to the developer-notes section to clarify where the annotations should be applied.

ACKs for top commit:
  MarcoFalke:
    ACK 25c57d6409 🥘
  promag:
    Code review ACK 25c57d6409.

Tree-SHA512: 61b6ef856bf6c6016d535fbdd19daf57b9e59fe54a1f30d47282a071b9b9d60b2466b044ee57929e0320cb1bdef52e7a1687cacaa27031bbc43d058ffffe22ba
2021-02-22 09:47:15 +01:00
MarcoFalke
4b6ca4a35a Merge #20845: net: Log to net debug in MaybeDiscourageAndDisconnect except for noban and manual peers
fa55159b9e net: Log to net debug in MaybeDiscourageAndDisconnect except for noban and manual peers (MarcoFalke)

Pull request description:

  The goal is to avoid local peers (e.g. untrusted peers on the local network or inbound onion peers via a local onion proxy) filling the debug log (and thus the disk).

ACKs for top commit:
  practicalswift:
    ACK fa55159b9e
  vasild:
    ACK fa55159b9e

Tree-SHA512: de233bf57334580f9b91f369fafd131d71c5ae25db25b09cc8fa8cbf34c0648f083c52260a6a912238751467e3c3c5f5d2309c145710753058d44a0003f88f4f
2021-02-22 09:43:57 +01:00
Hennadii Stepanov
1d5d832d5c qt, refactor: Use enum type as switch argument in TransactionTableModel 2021-02-22 09:36:48 +02:00
Hennadii Stepanov
52f122c11f qt, refactor: Use enum type as switch argument in PeerTableModel 2021-02-22 09:36:48 +02:00
Hennadii Stepanov
a35223f1cd qt, refactor: Use enum type as switch argument in BanTableModel 2021-02-22 09:34:27 +02:00
Hennadii Stepanov
ab8a747d1c qt, refactor: Use enum type as switch argument in AddressTableModel 2021-02-22 09:34:17 +02:00
MarcoFalke
5e4a3ca2f4 Merge bitcoin-core/gui#211: qt: Remove Transactionview Edit Label Action
8f9644890a qt: Remove Transactionview Edit Label Action (Jarol Rodriguez)

Pull request description:

  This PR removes the `Edit Label` action from the `transactionview` context menu. Since the `Edit Label` action will no longer be utilized in the `transactionview`, the `Edit Label` function logic is also removed.

  | Master        |        PR        |
  | ----------- | ----------- |
  |<img width="248" alt="Screen Shot 2021-02-17 at 8 34 34 PM" src="https://user-images.githubusercontent.com/23396902/108292189-9b86c800-7161-11eb-9e80-6238523bc27e.png">|<img width="248" alt="Screen Shot 2021-02-17 at 8 35 10 PM" src="https://user-images.githubusercontent.com/23396902/108292204-a17ca900-7161-11eb-8582-7f33d3e2ba8f.png">|

  Among the context menu actions for each transaction in the `transactionview` is the `Edit Label` action.
  While all other actions apply directly to the selected transaction, the `Edit Label` action applies to the selected transaction's address. As documented in issue #209 and [#1168](https://github.com/bitcoin/bitcoin/issues/1168) , this is an "unfortunate" placement for such an action. The current placement creates a confusing UX scenario where the outcome of the action is ambiguous.

  **Example of Ambiguous Behavior:**
  The context menu gives the wrong impression that the `Edit Label` action will edit a `Label` for the specific transaction that has been right-clicked on. This impression can be because all other actions in this menu will relate to the specific transaction and the misconception between `Comment` and `Label`.
  <img width="1062" alt="editlabel-start" src="https://user-images.githubusercontent.com/23396902/108296385-6da48200-7167-11eb-89f0-b21ccc58f6f4.png">

  Let's say I wanted to give the transaction selected in the screenshot above a comment of "2-17[17:43]". Given all the context clues, it will be reasonable to assume that the `Edit Label` function will give a label to this transaction. Instead, it edits the `Label` for the address behind this transaction. Thus, changing the `Label` for all transactions associated with this address.
  <img width="971" alt="editlabel-end" src="https://user-images.githubusercontent.com/23396902/108297179-e35d1d80-7168-11eb-86a9-0d2796c51829.png">

  **Maintaining `Edit Label` Functionality:**
  The action of Editing a Label should instead be reserved for the respective address tables of the `Send` and `Receive` tabs. As documented in this [comment](https://github.com/bitcoin-core/gui/issues/209#issuecomment-780922101), `Edit Label` is currently implemented in the `Send` tab and is missing in the `Receive` tab. A follow-up PR can add the `Edit Label` functionality to the `Receive` tab.

ACKs for top commit:
  MarcoFalke:
    review ACK 8f9644890a
  Talkless:
    tACK 8f9644890a, tested on Debian Sid.

Tree-SHA512: 70bbcc8be3364b0d4f476a9760aa14ad1ad1f53b0b130ce0ffe75190d76c386e6e26c530c0a55d1742402fe2b45c68a2af6dbfaf58ee9909ad93b06f0b6559d4
2021-02-22 08:33:51 +01:00
MarcoFalke
08eec6907a Merge bitcoin-core/gui#206: Display fRelayTxes and bip152_highbandwidth_{to, from} in peer details
142807af8b gui: display BIP152 high bandwidth relay in peer details (Jon Atack)
9476886353 gui: display fRelayTxes in peer details (Jon Atack)

Pull request description:

  This pull adds two fields to the peer details, "Wants Tx Relay" (fRelayTxes) and "High Bandwidth" (bip152_highbandwidth to/from). See the added tooltips for more info.

ACKs for top commit:
  MarcoFalke:
    review ACK 142807af8b
  jarolrod:
    ACK 142807af8b

Tree-SHA512: 956c7fa54c9c2ea76ee879d370711be0bed4af05484a17d35a1dd77713ed34ff441ed3957d0ef3a7ca7cf59a2f5d898be49b12af609a16b3e3cbfc4a1ba8f54e
2021-02-22 08:19:53 +01:00
MarcoFalke
0e9596c860 Merge bitcoin-core/gui#205: Save/restore TransactionView and recentRequestsView tables column sizes
964885d048 qt: Save/restore recentRequestsView table column sizes (Hennadii Stepanov)
f5c8093e77 qt: Move recentRequestsView properties settings to constructor (Hennadii Stepanov)
9c5f4f2169 qt: Save/restore TransactionView table column sizes (Hennadii Stepanov)
788205c3f7 qt: Move transactionView properties settings to constructor (Hennadii Stepanov)
ecdbaf71c0 qt, refactor: Drop intermediate assignment (Hennadii Stepanov)

Pull request description:

  Save/restore TransactionView and recentRequestsView tables column sizes.
  Sorting order is not saved/restored intentionally.

  Based on #204 (the first commit).

ACKs for top commit:
  jarolrod:
    ACK 964885d048, tested on macOS 11.1 Qt 5.15.2
  Talkless:
    tACK 964885d048, tested on Debian Sid, saving/restoring and resetting (with `-resetguisettings`) works as expected.

Tree-SHA512: c24e41bf4d95bb33dce16e9a0b952ffd0912e95f4d2a1bc5292fcf5a27100e70fea73433c4ff246d05b174fc23a7b6de1790a2e8b990a9089e4deca79a00dedc
2021-02-22 08:17:06 +01:00
MarcoFalke
fd725c2d79 Merge bitcoin-core/gui#204: Drop buggy TableViewLastColumnResizingFixer class
3913d1e8c1 qt: Drop buggy TableViewLastColumnResizingFixer class (Hennadii Stepanov)

Pull request description:

  In Qt 5 the last column resizing with dragging its left edge works out-of-the-box.

  The current `TableViewLastColumnResizingFixer` implementation could put the last column content out of the view port and confuse a user:
  ![Screenshot from 2021-01-31 18-04-32](https://user-images.githubusercontent.com/32963518/106390022-fd6bd180-63ee-11eb-9216-6e5117f8dc96.png)

  Historical context:
  - https://github.com/bitcoin/bitcoin/pull/2862
  - https://github.com/bitcoin/bitcoin/pull/3626
  - https://github.com/bitcoin/bitcoin/pull/3738
  - https://github.com/bitcoin/bitcoin/pull/3920

  #205 is a nice addition.

ACKs for top commit:
  jarolrod:
    ACK 3913d1e8c1, tested on macOS 11.1 Qt 5.15.2
  Talkless:
    tACK 3913d1e8c1, tested on Debian Sid. Can confirm that behavior in previous commit does not produce scroll bar, last column gets "hidden". This PR makes clear that there's more to see in the view.
  promag:
    Tested ACK 3913d1e8c1 on macos.

Tree-SHA512: 12582dfce54bb1db3d9934ae092e305d32e9760cc99b0265322e161fa7f54b7d6fb6cefedf700783f767d5c3a56a8545c8d2f5ade66596c4e67b8a5287063e8a
2021-02-22 08:13:51 +01:00
MarcoFalke
8d7d5f257b Merge bitcoin-core/gui#202: peers-tab: bug fix right panel toggle
8353e8cecc peers-tab: bug fix right panel toggle (randymcmillan)

Pull request description:

  Initial Presentation:

  ![Screen Shot 2021-01-28 at 8 36 15 PM](https://user-images.githubusercontent.com/152159/106220159-e2a81b80-61a8-11eb-84e9-f9b44375c9a1.png)

  When node row selected - panel is presented:

  ![Screen Shot 2021-01-28 at 8 36 22 PM](https://user-images.githubusercontent.com/152159/106220185-eb98ed00-61a8-11eb-9467-6a762941902d.png)

  When network disabled - right panel is hidden:

  ![Screen Shot 2021-01-28 at 8 36 32 PM](https://user-images.githubusercontent.com/152159/106220235-0a977f00-61a9-11eb-8a10-f31e4312ed31.png)

ACKs for top commit:
  jarolrod:
    ACK 8353e8cecc
  jonatack:
    ACK 8353e8cecc tested rebased on current master. Behavior is initially a bit surprising but this would allow more columns to be added to the peers tab window. Verified that selecting more than one peer, clicking on a column header, or running `disconnectnode "" <currently-selected-peer-id>` in the console (or on the CLI with the `-server` startup option) returns the window to its full size. If this is merged, it might be nice to have an obvious way to close the details area like a clickable "close this" icon in the upper left corner of the area.
  Talkless:
    tACK 8353e8cecc, tested on Debian Sid. Made `bitcoind` connect to `bitcoin-qt` with the PR changes, and after I quit the `bitcoind` instance, right panel do disappear, compared to the previous commit where it didn't.

Tree-SHA512: 8fc156f40bdd61e3ba8db333c729a2a07fd5f0fd1eed56f2fd2aa5ae5864756f8ab6fad74ae2fb0552ee7518b6d489f5800709e6c80c6f31f61fd8ce21cece5f
2021-02-22 08:11:35 +01:00
MarcoFalke
02fda8267a Merge bitcoin-core/gui#179: Add Type column to peers window, update peer details name/tooltip
be4cf4832f gui: update to "Direction/Type" peer details name/tooltip (Jon Atack)
151888383a gui: add "Type" column to Peers main window (Jon Atack)
6fc72bd6f0 gui: allow ConnectionTypeToQString to prepend direction optionally (Jon Atack)

Pull request description:

  This pull:

  - adds a sortable `Type` column to the GUI Peers tab window
  - updates the peer details row to `Direction/Type`, so the `Type` column without a direction makes sense (the tooltip is also updated)

  ![Screenshot from 2021-02-06 22-53-11](https://user-images.githubusercontent.com/2415484/107130646-973bee80-68c7-11eb-9025-b18394ac5c93.png)

ACKs for top commit:
  jarolrod:
    ACK be4cf4832f
  leonardojobim:
    Tested ACK be4cf4832f  on Ubuntu 20.04 on VMWare.

Tree-SHA512: 6c6d1dbe7d6bdb616acff0aaf8f4223546f1d2524566e9cd6e5b1b3bed2be1e9b20b1bc52ed3b627df53ba1f2fe0bc76f036cf16ad934d8a446b515d9bece3b1
2021-02-22 08:04:29 +01:00
fanquake
a6a1b106dc guix: only download sources for hosts being built
If a user is only interested in building for Linux, this saves
downloading the macOS compiler and additional dependencies.

This will result in a few additional `make` invocations, for the Linux
hosts, however this is quite low overhead.

Co-authored-by: Carl Dong <contact@carldong.me>
2021-02-22 10:49:05 +08:00
fanquake
ccb7b6a976 contrib: run test-symbol check for RISCV
Now that we are using Focal for Gitian building (glibc 2.31), we can
user a newer introduced  symbol, and include RISCV in this test.
2021-02-22 08:35:06 +08:00
Hennadii Stepanov
67f26319a0 gui: Add monospaced font settings 2021-02-21 21:01:42 +02:00
Hennadii Stepanov
22e0114d05 qt: Choose monospaced font in C++ code rather in *.ui file
Setting the "Monospace" font family in a `*.ui` file does not work on
macOS, at least on Big Sur with Qt 5.15 (neither via the "font" property
nor via the "styleSheet" property). Qt chooses the ".AppleSystemUIFont"
instead of ".AppleSystemUIFontMonospaced".

This change makes macOS choose the correct monospaced font.
2021-02-21 21:01:03 +02:00
Hennadii Stepanov
623de12d04 qt: Make GUIUtil::fixedPitchFont aware of embedded font 2021-02-21 21:01:03 +02:00
Hennadii Stepanov
89e421918e gui: Add Roboto Mono font 2021-02-21 21:01:02 +02:00
Sjors Provoost
f7eb7ecc67 test: framework: add skip_if_no_external_signer 2021-02-21 16:27:10 +01:00
Sjors Provoost
87a97941f6 configure: add --enable-external-signer
This option replaces --with-boost-process

This prepares external signer support to be disabled by default.
It adds a configure option to enable this feature and to check
if Boost::Process is present.

This also exposes ENABLE_EXTERNAL_SIGNER to the test suite via test/config.ini
2021-02-21 16:27:10 +01:00
MarcoFalke
fa730e9157 test: Avoid connecting to real network when running tests
Can be reviewed with --word-diff-regex=.
2021-02-21 11:02:58 +01:00
MarcoFalke
fa1b713941 test: Assume node is running in subtests
Every (sub)test in the framework assumes the node is running, except for
the (sub)tests in this file. Remove that confusion by stopping the node
at the start of every subtest, instead of at the end.
2021-02-21 11:01:53 +01:00
MarcoFalke
fa560cc6c4 test: Intermittent issue in feature_blockfilterindex_prune 2021-02-21 08:08:08 +01:00
fanquake
5bb64acd9d Merge #21243: ci: Avoid invoking curl on the host
fa330d8fed ci: Avoid invoking curl on the host (MarcoFalke)

Pull request description:

  The only requirement for the ci system are the programs `docker.io` and `bash`. However, the mac cross build invokes `curl` on the host. Fix that.

  Before:

  ```
  $ FILE_ENV="./ci/test/00_setup_env_mac.sh" ./ci/test_run_all.sh
  ...
  ./ci/test/05_before_script.sh: line 22: curl: command not found
  ```

  After:

  ```
  ... (command passes)

ACKs for top commit:
  promag:
    ACK fa330d8fed.

Tree-SHA512: 49120fd671a48a6599dd6c34f6d3502a6e9f84b4476061cab06f55cba374d8188f53b9b41363e90f5fafb0074767b581f30bd2545c0b6934580a7eccfa1ef5c4
2021-02-21 10:25:26 +08:00
MarcoFalke
fa330d8fed ci: Avoid invoking curl on the host 2021-02-20 09:52:55 +01:00
MarcoFalke
828bb776d2 Merge #20750: [Bundle 2/n] Prune g_chainman usage in mempool-related validation functions
e8ae1db864 style-only: Make AcceptToMemoryPool signature readable (Carl Dong)
8f5c100064 style-only: Make CheckSequenceLock signature readable (Carl Dong)
8c824819c8 validation: Use *this in CChainState::LoadMempool (Carl Dong)
0a9a24d8c7 validation: Pass in chainstate to UpdateMempoolForReorg (Carl Dong)
7142018812 validation: Pass in chainstate to CTxMemPool::removeForReorg (Carl Dong)
71734c65dc validation: Pass in chain to ::TestLockPointValidity (Carl Dong)
120aaba9ac tree-wide: Fix erroneous AcceptToMemoryPool replacements (Carl Dong)
417dafc1ee validation: Remove old AcceptToMemoryPool w/o chainstate param (Carl Dong)
3704433c4f scripted-diff: Invoke ::AcceptToMemoryPool with chainstate (Carl Dong)
229bc37b5f validation: Pass in chainstate to ::AcceptToMemoryPool (Carl Dong)
d0da7ea57a validation: Pass in chainstate to ::LoadMempool (Carl Dong)
3a205c43dc validation: Pass in chainstate to AcceptToMemoryPoolWithTime (Carl Dong)
d8a816329c validation: Add chainstate member to MemPoolAccept (Carl Dong)
4c15942b79 validation: Pass in chainstate to ::CheckSequenceLocks (Carl Dong)
577b774d0c validation: Remove old CheckFinalTx w/o chain tip param (Carl Dong)
7031cf89db scripted-diff: Invoke ::CheckFinalTx with chain tip (Carl Dong)
d015eaa550 validation: Pass in chain tip to ::CheckFinalTx (Carl Dong)
252b489c9f validation: Pass in coins tip to CheckInputsFromMempoolAndCache (Carl Dong)
73a6d2b7be validation: Pass in chainstate to IsCurrentForFeeEstimation (Carl Dong)
d1f932b0b0 validation: Pass in coins cache to ::LimitMempoolSize (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  	1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  	2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  	3. Remove `old_function`

ACKs for top commit:
  glozow:
    reACK e8ae1db864 via `git range-diff 15f0042...e8ae1db`, only change is fixing ATMP call from conflict
  MarcoFalke:
    ACK e8ae1db864 📣

Tree-SHA512: 6af50f04940a69c5c3d3796a24f32f963fa02503cdc1155cc11fff832a99172b407cd163a19793080a5af98580f051b48195b62ec4a797ba2763b4883174153d
2021-02-20 09:21:45 +01:00
Carl Dong
95990b9f32 guix: Update conservative space requirements 2021-02-19 11:29:56 -05:00
Carl Dong
5e6df11326 guix: Add support for powerpc64{,le}
The new time-machine commit contains a few small changes that make the
powerpc cross-toolchain work.
2021-02-19 11:29:56 -05:00
Wladimir J. van der Laan
c46fe4d107 Merge #21088: guix: Jump forwards in time-machine and adapt
d02076b885 guix: Jump forwards in time-machine and adapt (Carl Dong)
f8ca8c5c28 guix: Supply --keep-failed for debugging (Carl Dong)

Pull request description:

  ```
  The new time-machine commit is Guix v1.2.0 with a yet-unupstreamed patch
  for NSIS.

  A few important changes:

  1. Guix switched back from using CPATH to C{,PLUS}_INCLUDE_PATH as the
     way to indicate #include search paths.
  2. GCC's library is now split into a separate output, whereas before it
     was included in the default output. This means that our gcc toolchain
     packages need to propagate that output.
  3. A few package versions were bumped
  ```

  See this compare to review my custom patches to Guix: https://github.com/dongcarl/guix/compare/version-1.2.0...7d6bd44da57926e0d4af25eba723a61c82beef98

ACKs for top commit:
  laanwj:
    ACK d02076b885

Tree-SHA512: 896d5bf1b6e5fda2f0106013c568c119bbbb86cb31a8c0a22432bada9b7da51678b96374bf8fd7c15353698ba47ac9dd39874d40c39001281471db7c78bf1705
2021-02-19 15:24:19 +01:00
MarcoFalke
f1c339d452 Merge #21211: test: Move P2WSH_OP_TRUE to shared test library
22220ef6d5 test: Move P2WSH_OP_TRUE to shared test library (MarcoFalke)

Pull request description:

  Otherwise it can't be used in other tests (unit, fuzz, bench, ...)

ACKs for top commit:
  darosior:
    ACK 22220ef6d5

Tree-SHA512: 1b636e751281291f7c21ac51c3d014f6a565144c9482974391c516228e756442b077655eda970eb8bdb12974b97855a909b2b60d518026a8d5f41aa15ec7cbc8
2021-02-19 13:14:25 +01:00
MarcoFalke
09eb46c943 Merge #21187: Net processing: Only call PushAddress() from net_processing
3e68efa615 [net] Move checks from GetLocalAddrForPeer to caller (John Newbery)
d21d2b264c [net] Change AdvertiseLocal to GetLocalAddrForPeer (John Newbery)

Pull request description:

  This is the first part of #21186. It slightly disentangles addr handling in net/net_processing by making it explicit that net_processing is responsible for pushing addr records into `vAddrToSend`.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3e68efa615 🍅

Tree-SHA512: 9af50c41f5a977e2e277f24a589db38e2980b353401def5e74b108ac5f493d9b5d6b1b8bf15323a4d66321495f04bc271450fcef7aa7d1c095f051a4f8e9b15f
2021-02-19 12:58:41 +01:00
MarcoFalke
6a680a6236 Merge #21226: build: Fix fuzz binary compilation under windows
56ace907b9 Fix fuzz binary compilation under windows (Dan Benjamin)

Pull request description:

  Small change to allow the fuzz binary to compile under windows. Also removed --disable-fuzz-binary from the windows CI test. This fixes #21212.

ACKs for top commit:
  MarcoFalke:
    review ACK 56ace907b9 the best bugfixes are the ones removing code

Tree-SHA512: 6088fd955a5e511b5ca1b3eaa8469a889eb6d994c2827acac7695dac6e4e320a344b45f4015a2f279b16df0d4b23ec4df13304ae6315395ad2fe8c5b526cada4
2021-02-19 11:30:41 +01:00
MarcoFalke
f1c2f619a6 Merge #21221: [tools] Allow argument/parameter bin packing in clang-format
876ac3f6b6 [tools] Allow argument/parameter bin packing in clang-format (John Newbery)

Pull request description:

  clang-format documentation for BinPackArguments:

  If `false`, a function call’s arguments will either be all on the same line or will have one line each.

  ```
  true:
  void f() {
    f(aaaaaaaaaaaaaaaaaaaa, aaaaaaaaaaaaaaaaaaaa,
      aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa);
  }

  false:
  void f() {
    f(aaaaaaaaaaaaaaaaaaaa,
      aaaaaaaaaaaaaaaaaaaa,
      aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa);
  }
  ```

  https://clang.llvm.org/docs/ClangFormatStyleOptions.html#configurable-format-style-options

  There's no reason to forbid this format. Having multiple arguments or parameters per line can be just as readable as having one per line (and is certainly more readable than having extremely long lines).

ACKs for top commit:
  laanwj:
    ACK 876ac3f6b6
  MarcoFalke:
    review ACK 876ac3f6b6
  vasild:
    ACK 876ac3f6b6

Tree-SHA512: 7c401b4551b458c83dd70883860788b4a60e08a5399171fef27a2f5fdc6b933f6454fe0d396c32d826e3ab537791329da3275ae9b5e9ad36630a6dc2c167e88f
2021-02-19 11:28:39 +01:00
fanquake
f093310b2e Merge #21228: test: Avoid comparision of integers with different signs
bedb8d88bc Avoid comparision of integers with different signs (Jonas Schnelli)

Pull request description:

  Fixes an integer comparison of different signs (which errors out on `-Werror,-Wsign-compare`). Introduced in #21121.

  See https://bitcoinbuilds.org/index.php?ansilog=982c61cf-6969-4001-bebc-dc215e5d29a4.log

ACKs for top commit:
  MarcoFalke:
    review ACK bedb8d88bc
  amitiuttarwar:
    ACK bedb8d88bc
  vasild:
    ACK bedb8d88bc

Tree-SHA512: cb22a6239a1fc9d0be5573bf6ae4ec379eb7398c88edc8fa2ae4fd721f37f9ca3724896c1ac16de14a5286888a0b631813da32cb62d177ffbf9b2c31e716a7aa
2021-02-19 18:02:33 +08:00
Wladimir J. van der Laan
56fe4bb3c5 Merge #21210: doc: Rework internal and external links
77772a1b80 doc: Rework internal and external links (MarcoFalke)

Pull request description:

  Some minor changes:
  * Move Bitcoin Core download link to the very top. *Reason:* The download link has nothing to do with the section that explains Bitcoin. Also, anyone quickly looking for the download link will find it faster.
  * Add a new link to the doc folder. *Reason*: Apart from the documentation that is shipped with the binary software, the doc folder is the primary location for Bitcoin Core related documentation.
  * Remove dead link to pdf. *Reason*: The pdf can be trivially found by asking a search engine.
  * Remove reference to "build server". *Reason*: There is no "build server". The CI system is explained in the next sentence in detail.
  * Remove dead? link to translation mailing list. *Reason*: The translation process is explained in `doc/translation_process.md`, no need to explain it in detail in the main readme.

ACKs for top commit:
  laanwj:
    ACK 77772a1b80
  RiccardoMasutti:
    ACK 77772a1

Tree-SHA512: 365824c6da519892ff239842b0c9525f559513eb800afd77b14febdc9d1a79294d9448df6cd25b623f1734ff859d9e20951e78fb8091d19fdadde0dd24eeddd4
2021-02-19 10:48:37 +01:00
fanquake
04e01606e3 Merge #21205: build: actually fail when Boost is missing
c5da2749e2 build: actually stop configure if Boost isn't available (fanquake)
cad8b527ea build: explicitly install libboost-dev package (fanquake)

Pull request description:

  If Boost is not found via AX_BOOST_BASE, we don't actually stop
  configuring, only a warning is emitted:
  ```bash
  checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
  ```

  Instead we usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being
  removed, and in any case, it makes more sense to fail earlier if Boost is missing.

  If Boost is unavailable, the failure now looks like:
  ```bash
  checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
  configure: error: Boost is not available!
  ```

  Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than showing `MINIMUM_REQUIRED_BOOST`).

  This PR also has a commit that adds `libboost-dev` to our install instructions and CI. This package is currently installed as a side-effect of installing our other libboost-*-dev packages. However as those continue to disappear, it makes sense to install boost-dev explicitly.

ACKs for top commit:
  laanwj:
    Code review ACK c5da2749e2
  MarcoFalke:
    Concept ACK c5da2749e2

Tree-SHA512: f866062f9d7d3a2316b6c887f17c664b9cfff41fdc0cb99ca79d641240fb01a5ae0d34140e515bc465219e1b43d5ca84f7c55f48b9c5b45a80ff2795dafd072b
2021-02-19 17:45:19 +08:00
Jonas Schnelli
9f3ffa2938 Merge #21230: test: Fix NODE_NETWORK_LIMITED_MIN_BLOCKS disconnection
fa24247d0f test: Fix NODE_NETWORK_LIMITED_MIN_BLOCKS disconnection (MarcoFalke)
fab6995629 test: Make test actually test something (MarcoFalke)
fae8f35df8 test: pep8 touched test (MarcoFalke)

Pull request description:

  Fix several bugs. Also, fix #21227

ACKs for top commit:
  jonasschnelli:
    utACK fa24247d0f - thanks for fixing.
  ryanofsky:
    Code review ACK fa24247d0f with caveat above that I don't really understand the problem or fix. But the cleanups look good and the fix does seem perfectly safe. More description would be welcome!

Tree-SHA512: 67f6ec92f6493aa822ae3fa8a7426a5acdc684044b8bafc0c65b652f63ccce969d0a6f1d1f099d6a91d05f478724869345b70335f2cfcfd00df46aef05cc4f9e
2021-02-19 08:38:01 +01:00
Fu Yong Quah
9a0969585f build: Add /opt/homebrew to path to look for boost libraries 2021-02-19 01:37:31 +00:00
Samuel Dobson
3a2d5bfeb3 Merge #21201: rpc: Disallow sendtoaddress and sendmany when private keys disabled
6bfbc97d71 test: disallow sendtoaddress/sendmany when private keys disabled (Jon Atack)
0997019e76 Disallow sendtoaddress and sendmany when private keys disabled (Andrew Chow)

Pull request description:

  Since `sendtoaddress` and `sendmany` (which use the `SendMoney` function) create and commit a transaction, they should not do anything when the wallet does not have private keys. Otherwise a valid transaction cannot be made.

  Fixes #21104

ACKs for top commit:
  jonatack:
    ACK 6bfbc97d71
  meshcollider:
    utACK 6bfbc97d71
  kristapsk:
    ACK 6bfbc97d71. "Error: Private keys are disabled for this wallet" is definitely a better error message than "Insufficient funds" here. Hopefully change of error code from -6 to -4 doesn't break any software using Bitcoin JSON-RPC API.

Tree-SHA512: f277d6b5252e43942d568614032596f2c0827f00cd0cb71e44ffcb9822bfb15a71730a3e3688f31e59ba4eb7d275250c4e65ad4b6b3e96be6314c56a672432fb
2021-02-19 14:00:48 +13:00
Dan Benjamin
56ace907b9 Fix fuzz binary compilation under windows 2021-02-18 17:35:24 -05:00
Carl Dong
e8ae1db864 style-only: Make AcceptToMemoryPool signature readable 2021-02-18 14:49:10 -05:00
Carl Dong
8f5c100064 style-only: Make CheckSequenceLock signature readable 2021-02-18 14:49:10 -05:00
Carl Dong
8c824819c8 validation: Use *this in CChainState::LoadMempool 2021-02-18 14:49:10 -05:00
Carl Dong
0a9a24d8c7 validation: Pass in chainstate to UpdateMempoolForReorg 2021-02-18 14:49:10 -05:00
Carl Dong
7142018812 validation: Pass in chainstate to CTxMemPool::removeForReorg
Several other parameters are now redundant since they can be safely
obtained from the chainstate given that ::cs_main is locked. These are
now removed.
2021-02-18 14:49:10 -05:00
Carl Dong
71734c65dc validation: Pass in chain to ::TestLockPointValidity 2021-02-18 14:49:10 -05:00
Carl Dong
120aaba9ac tree-wide: Fix erroneous AcceptToMemoryPool replacements 2021-02-18 14:49:10 -05:00
Carl Dong
417dafc1ee validation: Remove old AcceptToMemoryPool w/o chainstate param 2021-02-18 14:49:10 -05:00
Carl Dong
3704433c4f scripted-diff: Invoke ::AcceptToMemoryPool with chainstate
-BEGIN VERIFY SCRIPT-
find_regex='\bAcceptToMemoryPool\(' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E 's@'"$find_regex"'@\0::ChainstateActive(), @g'
-END VERIFY SCRIPT-
2021-02-18 14:49:06 -05:00
MarcoFalke
fa24247d0f test: Fix NODE_NETWORK_LIMITED_MIN_BLOCKS disconnection 2021-02-18 20:45:05 +01:00
MarcoFalke
fab6995629 test: Make test actually test something
The context manager was not even created, so previously it did not check the debug log
2021-02-18 20:43:35 +01:00
MarcoFalke
fae8f35df8 test: pep8 touched test 2021-02-18 20:43:32 +01:00
Carl Dong
229bc37b5f validation: Pass in chainstate to ::AcceptToMemoryPool 2021-02-18 14:43:28 -05:00
Carl Dong
d0da7ea57a validation: Pass in chainstate to ::LoadMempool 2021-02-18 14:43:28 -05:00
Carl Dong
3a205c43dc validation: Pass in chainstate to AcceptToMemoryPoolWithTime 2021-02-18 14:43:28 -05:00
Carl Dong
d8a816329c validation: Add chainstate member to MemPoolAccept 2021-02-18 14:43:28 -05:00
Carl Dong
4c15942b79 validation: Pass in chainstate to ::CheckSequenceLocks 2021-02-18 14:43:28 -05:00
Carl Dong
577b774d0c validation: Remove old CheckFinalTx w/o chain tip param 2021-02-18 14:43:28 -05:00
Carl Dong
7031cf89db scripted-diff: Invoke ::CheckFinalTx with chain tip
-BEGIN VERIFY SCRIPT-
find_regex='\bCheckFinalTx\(' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E 's@'"$find_regex"'@\0::ChainActive().Tip(), @g'
-END VERIFY SCRIPT-
2021-02-18 14:43:28 -05:00
Carl Dong
d015eaa550 validation: Pass in chain tip to ::CheckFinalTx 2021-02-18 14:43:28 -05:00
Carl Dong
252b489c9f validation: Pass in coins tip to CheckInputsFromMempoolAndCache 2021-02-18 14:43:28 -05:00
Carl Dong
73a6d2b7be validation: Pass in chainstate to IsCurrentForFeeEstimation 2021-02-18 14:43:28 -05:00
Carl Dong
d1f932b0b0 validation: Pass in coins cache to ::LimitMempoolSize 2021-02-18 14:43:28 -05:00
Jonas Schnelli
bedb8d88bc Avoid comparision of integers with different signs 2021-02-18 20:34:10 +01:00
Carl Dong
d02076b885 guix: Jump forwards in time-machine and adapt
The new time-machine commit is Guix v1.2.0 with a yet-unupstreamed patch
for NSIS.

A few important changes:

1. Guix switched back from using CPATH to C{,PLUS}_INCLUDE_PATH as the
   way to indicate #include search paths.
2. GCC's library is now split into a separate output, whereas before it
   was included in the default output. This means that our gcc toolchain
   packages need to propagate that output.
3. A few package versions were bumped
2021-02-18 14:29:42 -05:00
Carl Dong
f8ca8c5c28 guix: Supply --keep-failed for debugging 2021-02-18 14:29:42 -05:00
MarcoFalke
faf48f20f1 log: Clarify log message when file does not exist
Also, run clang-format on the function
2021-02-18 15:08:35 +01:00
Wladimir J. van der Laan
b805dbb0b9 Merge #19809: log: Prefix log messages with function name and source code location if -logsourcelocations is set
b4511e2e2e log: Prefix log messages with function name if -logsourcelocations is set (practicalswift)

Pull request description:

  Prefix log messages with function name if `-logfunctionnames` is set.

  Yes, exactly like `-logthreadnames` but for function names instead of thread names :)

  This is a small developer ergonomics improvement: I've found this to be a cheap/simple way to correlate log output and originating function.

  For me it beats the ordinary cycle of 1.) try to figure out a regexp matching the static part of the dynamic log message, 2.) `git grep -E 'Using .* MiB out of .* requested for signature cache'`, 3.) `mcedit filename.cpp` (`openemacs filename.cpp` works too!) and 4.) search for log message and scroll up to find the function name :)

  Without any logging parameters:

  ```
  $ src/bitcoind -regtest
  2020-08-25T03:29:04Z Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z block tree size = 1
  2020-08-25T03:29:04Z nBestHeight = 0
  2020-08-25T03:29:04Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z 0 addresses found from DNS seeds
  ```

  With `-logthreadnames` and `-logfunctionnames`:

  ```
  $ src/bitcoind -regtest -logthreadnames -logfunctionnames
  2020-08-25T03:29:04Z [init] [ReportHardwareRand] Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z [init] [InitSignatureCache] Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [InitScriptExecutionCache] Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [LoadChainTip] Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z [init] [AppInitMain] block tree size = 1
  2020-08-25T03:29:04Z [init] [AppInitMain] nBestHeight = 0
  2020-08-25T03:29:04Z [loadblk] [LoadMempool] Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z [dnsseed] [ThreadDNSAddressSeed] 0 addresses found from DNS seeds
  ```

ACKs for top commit:
  laanwj:
    Code review ACK b4511e2e2e
  MarcoFalke:
    review ACK b4511e2e2e 🌃

Tree-SHA512: d100f5364630c323f31d275259864c597f7725e462d5f4bdedcc7033ea616d7fc0d16ef1b2af557e692f4deea73c6773ccfc681589e7bf6ba970b9ec169040c7
2021-02-18 14:37:51 +01:00
Wladimir J. van der Laan
860f916803 Merge #20524: test: Move MIN_VERSION_SUPPORTED to p2p.py
9f21ed4037 [test] Check user agent string from test framework connections (John Newbery)
9ce4c3c4c1 [test] Add P2P_SERVICES to p2p.py (John Newbery)
010542614d [test] Move MY_RELAY to p2p.py (John Newbery)
9b4054cb7a [test] Move MY_SUBVERSION to p2p.py (John Newbery)
7e158a6910 [test] Move MY_VERSION to p2p.py (John Newbery)
652311165c [test] Move MIN_VERSION_SUPPORTED to p2p.py (John Newbery)

Pull request description:

  The messages.py module should contain code and helpers for
  [de]serializing p2p messages. Specific usage of those messages should
  be in p2p.py. This PR moves test framework specific constants to p2p.py.

  It also changes the SUBVERSION constant to be a string instead of a bytes object. That means that it needs to be explicitly converted to a bytes object to serialize into a version message. Failing to do so would cause an easy-to-spot bug. This should avoid silent failures like the one solved in #20522.

ACKs for top commit:
  laanwj:
    Code review ACK 9f21ed4037

Tree-SHA512: 41d46575ac0ec36ad074d6c6a5b9cef50b05eeb8ddd8ed0a8f0d0c4617cc7b8baa6580af5b83a668230ce1ac27bf0e56914d0361a48b1b05fd75e2e60350eeaf
2021-02-18 14:01:57 +01:00
Jon Atack
142807af8b gui: display BIP152 high bandwidth relay in peer details 2021-02-18 11:17:07 +01:00
Jon Atack
9476886353 gui: display fRelayTxes in peer details 2021-02-18 11:11:40 +01:00
John Newbery
876ac3f6b6 [tools] Allow argument/parameter bin packing in clang-format 2021-02-18 10:07:37 +00:00
John Newbery
3e68efa615 [net] Move checks from GetLocalAddrForPeer to caller
GetLocalAddrForPeer() is only called in one place. The checks inside that
function make more sense to be carried out be the caller:

- fSuccessfullyConnected is already checked at the top of
  SendMessages(), so must be true when we call GetLocalAddrForPeer()
- fListen can go into the conditional before GetLocalAddrForPeer() is
  called.
2021-02-18 09:43:13 +00:00
John Newbery
d21d2b264c [net] Change AdvertiseLocal to GetLocalAddrForPeer
Gossiping addresses to peers is the responsibility of net processing.
Change AdvertiseLocal() in net to just return an (optional) address
for net processing to advertise. Update function name to reflect
new responsibility.
2021-02-18 09:28:06 +00:00
Samuel Dobson
db656db2ed Merge #19136: wallet: add parent_desc to getaddressinfo
de6b389d5d tests: Test getaddressinfo parent_desc (Andrew Chow)
e4ac869a0a rpc: Add parent descriptor to getaddressinfo output (Andrew Chow)
bbe4a36152 wallet: Add GetDescriptorString to DescriptorScriptPubKeyMan (Andrew Chow)
9be1437c49 descriptors: Add ToNormalizedString and tests (Andrew Chow)

Pull request description:

  Adds `parent_desc` field to the `getaddressinfo` RPC to export a public descriptor. Using the given address, `getaddressinfo` will look up which `DescriptorScriptPubKeyMan` can be used to produce that address. It will then return the descriptor for that `DescriptorScriptPubKeyMan` in the `parent_desc` field. The descriptor will be in a normalized form where the xpub at the last hardened step is derived so that the descriptor can be imported to other wallets. Tests are added to check that the correct descriptor is being returned for the wallet's addresses and that these descriptors can be imported and used in other wallets.

  As part of this PR, a `ToNormalizedString` function is added to the descriptor classes. This really only has an effect on `BIP32PubkeyProvider`s that have hardened derivation steps. Tests are added to check that normalized descriptors are returned.

ACKs for top commit:
  Sjors:
    utACK de6b389d5d
  S3RK:
    Tested ACK de6b389
  jonatack:
    Tested ACK de6b389d5d modulo a few minor comments
  fjahr:
    Code review ACK de6b389d5d
  meshcollider:
    Tested ACK de6b389d5d

Tree-SHA512: a633e4a39f2abbd95afd7488484cfa66fdd2651dac59fe59f2b80a0940a2a4a13acf889c534a6948903d701484a2ba1218e3081feafe0b9a720dccfa9e43ca2b
2021-02-18 21:51:16 +13:00
Jonas Schnelli
9017d55e7c Merge #15946: Allow maintaining the blockfilterindex when using prune
84716b134e Add "index/blockfilterindex -> validation -> index/blockfilterindex" to expected circular dependencies (Jonas Schnelli)
ab3a0a2fb9 Add functional test for blockfilterindex in prune-mode (Jonas Schnelli)
c286a22f7b Add debug startup parameter -fastprune for more effective pruning tests (Jonas Schnelli)
5e112269c3 Avoid pruning below the blockfilterindex sync height (Jonas Schnelli)
00d57ff768 Avoid accessing nullpointer in BaseIndex::GetSummary() (Jonas Schnelli)
6abe9f5b11 Allow blockfilter in conjunction with prune (Jonas Schnelli)

Pull request description:

  Maintaining the blockfilterindexes in prune mode is possible and may lead to efficient p2p based rescans of wallets (restore backups, import/sweep keys) beyond the prune height (rescans not part of that PR).

  This PR allows running the blockfilterindex(es) in conjunction with pruning.
  * Bitcoind/Qt will shutdown during startup when missing block data has been detected ([re]enable `-blockfilterindex` when we already have pruned)
  * manual block pruning is disabled during blockfilterindex sync
  * auto-pruning is delayed during blockfilterindex sync

  ToDos:
  * [x] Functional tests

ACKs for top commit:
  fjahr:
    Code review ACK 84716b1
  ryanofsky:
    Code review ACK 84716b134e. Only changes since last review were suggested new FindFilesToPrune argument and test.
  benthecarman:
    tACK 84716b134e

Tree-SHA512: 91d832c6c562c463f7ec7655c08956385413a99a896640b9737bda0183607fac530435d03d87c3c0e70c61ccdfe73fe8f3639bc7d26d33ca7e60925ebb97d77a
2021-02-18 09:40:42 +01:00
MarcoFalke
cd66d8b1d8 Merge #20429: refactor: replace (sizeof(a)/sizeof(a[0])) with C++17 std::size
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17) (Sebastian Falbesoner)
365539c846 refactor: init vectors via std::{begin,end} to avoid pointer arithmetic (Sebastian Falbesoner)
63d4ee1968 refactor: iterate arrays via C++11 range-based for loops if idx is not needed (Sebastian Falbesoner)

Pull request description:

  This refactoring PR picks up the idea of #19626 and replaces all occurences of `sizeof(x)/sizeof(x[0])` (or `sizeof(x)/sizeof(*x)`, respectively) with the now-available C++17 [`std::size`](https://en.cppreference.com/w/cpp/iterator/size)  (as [suggested by sipa](https://github.com/bitcoin/bitcoin/pull/19626#issuecomment-666487228)), making the macro `ARRAYLEN` obsolete.

  As preparation for this, two other changes are done to eliminate `sizeof(x)/sizeof(x[0])` usage:
  * all places where arrays are iterated via an index are changed to use C++11 range-based for loops If the index' only purpose is to access the array element (as [suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/19626#discussion_r463404541)).
  * `std::vector` initializations are done via `std::begin` and `std::end` rather than using pointer arithmetic to calculate the end (also [suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/20429#discussion_r567418821)).

ACKs for top commit:
  practicalswift:
    cr ACK e829c9afbf: patch looks correct
  fanquake:
    ACK e829c9afbf
  MarcoFalke:
    review ACK e829c9afbf 🌩

Tree-SHA512: b01d32c04b9e04d562b7717cae00a651ec9a718645047a90761be6959e0cc2adbd67494e058fe894641076711bb09c3b47a047d0275c736f0b2218e1ce0d193d
2021-02-18 07:53:37 +01:00
Jarol Rodriguez
8f9644890a qt: Remove Transactionview Edit Label Action
Among the context menu actions for each transaction in the transactionview is the 'Edit Label' action.
While all other actions apply directly to the selected transaction, the 'Edit Label' action applies to the address of the selected transaction. This creates a confusing UX scenario where the outcome of the action is ambiguous. The action of Editing a Label should instead be reserved for the Send and Receive tabs.

This PR removes the 'Edit Label' action from the transactionview context menu. Since the 'Edit Label' action will no longer be utilized in the transactionview, the 'Edit Label' function logic is also removed.
2021-02-17 19:57:49 -05:00
Amiti Uttarwar
25c57d6409 [doc] Add a note about where lock annotations should go. 2021-02-17 15:58:23 -08:00
Amiti Uttarwar
ad5f01b960 [validation] Move the lock annotation from function definition to declaration
When the annotation is on the definition, it does not check call sites between
the declaration and the definition.
2021-02-17 15:45:11 -08:00
Wladimir J. van der Laan
372dd8da24 Merge #21110: util: remove Boost posix_time usage from GetTime*
9266f7497f util: Use std::chrono for time getters (MarcoFalke)
3c2e16be22 time: add runtime sanity check (Cory Fields)

Pull request description:

  I have a followup that should remove the last of our `boost:posix_time` usage in `ParseISO8601DateTime`, but that will likely need more cross-platform testing/discussion, so have just split them up as this change is straight forward.

ACKs for top commit:
  practicalswift:
    Tested ACK 9266f7497f
  laanwj:
    Code review ACK 9266f7497f

Tree-SHA512: 5471a60e65e9fa8ef48320743ef637f1d162724e717e0f5509118e1e5732fc0844656a9c09d3d1300eb657dcc7a1e1e67305d8c9ef959c63be67393607dd4ceb
2021-02-17 20:38:08 +01:00
Wladimir J. van der Laan
04336086d3 Merge #21182: build: remove mostly pointless BOOST_PROCESS macro
7bf04e358a build: remove mostly pointless BOOST_PROCESS macro (fanquake)

Pull request description:

  Performing a series of link checks for a Boost component that is
  header-only doesn't make much sense, and currently means we just have
  another confusing Boost macro in our tree. I'm not sure why this was
  originally done this way; maybe Sjors or luke-jr can elaborate (#15382 (929cda5470))?

  The macro also has the side-effect of producing confusing error
  messages. i.e in #20744, the CI is currently failing with:
  ```bash
  checking for boostlib >= 1.58.0 (105800) lib path in "/tmp/cirrus-ci-build/depends/x86_64-pc-linux-gnu/lib"... yes
  checking for boostlib >= 1.58.0 (105800)... yes
  checking whether the Boost::Process library is available... yes
  configure: error: Could not find a version of the Boost::Process library!
  ```

  This isn't useful, given there is no such thing as a `Boost::Process` library.

  This PR just removes the macro entirely, but maintains a `--with-boost-process`
  (defaulting to off), flag to configure. Hopefully this will also be
  removed, in favour of `--enable/disable-external-signer` if/when #16546
  is merged.

ACKs for top commit:
  laanwj:
    ACK 7bf04e358a

Tree-SHA512: b270a0250f32df2078f986c165b8977967d8c06df80bf2773f3442f74b395a3bfa6544af1024d9b6524d90d47a0f6304194b3aced0e2ecb88e75916da945ccb6
2021-02-17 15:08:38 +01:00
Jon Atack
7d3343fb8e cli: update -netinfo help doc following the merge of 882ce251 2021-02-17 15:05:42 +01:00
Jon Atack
ef614bb408 cli: small -netinfo simplification and performance improvement
that removes code and particularly this code from the loop of all peers:

`m_is_i2p_on |= (network_id == NET_I2P);`
2021-02-17 15:05:03 +01:00
Jon Atack
6b45ef3233 cli: improve -netinfo invalid argument error message 2021-02-17 15:05:01 +01:00
Jon Atack
3732404afa cli: warn in help that -netinfo is not intended to be a stable API 2021-02-17 15:04:50 +01:00
Jon Atack
7afdd72258 cli: enable -netinfo help to run without a remote server 2021-02-17 14:55:19 +01:00
Wladimir J. van der Laan
4dc1ff809a Merge #21192: cli: Treat high detail levels as maximum in -netinfo
882ce25132 cli: Treat high detail levels as the maximum in -netinfo (Wladimir J. van der Laan)

Pull request description:

  I somehow often type `-netinfo 5` which gets treated as `-netinfo 0`, after this change it's `-netinfo 4` which seems more convenient behavior.

ACKs for top commit:
  jonatack:
    ACK 882ce25132
  theStack:
    Tested ACK 882ce25132

Tree-SHA512: 5ed13213d00940c81f70c5fe5092f5fcc78c0184e1cc83b6c58a7bf24cf0f634816ce28f468aac588a4d202a6a7c1b411c0690099f1a8bf1999e662de4afcccd
2021-02-17 12:49:18 +01:00
MarcoFalke
22220ef6d5 test: Move P2WSH_OP_TRUE to shared test library 2021-02-17 11:36:30 +01:00
MarcoFalke
569b5ba1dc Merge #21121: [test] Small unit test improvements, including helper to make mempool transaction
1363b6c27d [doc / util] Use comments to clarify time unit for int64_t type. (Amiti Uttarwar)
47a7a1687d [util] Introduce a SetMockTime that takes chrono time (Amiti Uttarwar)
df6a5fc1df [util] Change GetMockTime to return chrono type instead of int (Amiti Uttarwar)
a2d908e1da [test] Throw error instead of segfaulting in failure scenario (Amiti Uttarwar)
9a3bbe8fc5 [test] Introduce a unit test helper to create a valid mempool transaction. (Amiti Uttarwar)

Pull request description:

  Some miscellaneous improvements that came up when working on #21061
  - The first commit is a helper to make valid mempool transactions & submit via ATMP. Introducing in this PR, using in #21061.
  - The second commit is a small improvement in `miner_tests.cpp` that uses `BOOST_REQUIRE_EQUAL` to properly terminate the program instead of segfaulting in the failure scenario where the blocks do not include the expected number of transactions.
  - The third commit changes the function signature of `GetMockTime()` to return a chrono type.
  - The fourth & fifth commit overload `SetMockTime` to also accept chrono type, and adds documentation to indicate that the `int64_t` function signature is deprecated.

ACKs for top commit:
  vasild:
    ACK 1363b6c27d

Tree-SHA512: c72574d73668ea04ee4c33858f8de68b368780f445e05afb569aaf8564093f8112259b3afe93cf6dc2ee12a1ab5af1130ac73c16416132c1ba2851c054a67d78
2021-02-17 10:40:09 +01:00
John Newbery
9f21ed4037 [test] Check user agent string from test framework connections
Add a check that new connections from the test framework to the
node have the correct user agent string. This makes bugs easier
to detect if the user agent string ever changes.
2021-02-17 09:29:44 +00:00
John Newbery
9ce4c3c4c1 [test] Add P2P_SERVICES to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore specify the nServices value in the calling code,
not in the messages.py module.
2021-02-17 09:29:41 +00:00
John Newbery
010542614d [test] Move MY_RELAY to p2p.py
messages.py is for message and primitive data structures. Specifics
about the test framework's p2p implementation should be in p2p.py.

Also rename to P2P_VERSION_RELAY. Also rename msg_version.nRelay to
relay. In Bitcoin Core, this is referred to as fRelay, since it's a
bool, so this field has always been misnamed.
2021-02-17 09:23:32 +00:00
John Newbery
9b4054cb7a [test] Move MY_SUBVERSION to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore move MY_SUBVERSION to p2p.py.

Also rename to P2P_SUBVERSION.
2021-02-17 09:22:37 +00:00
John Newbery
7e158a6910 [test] Move MY_VERSION to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore move MY_VERSION to p2p.py.

Also rename to P2P_VERSION to distinguish it from
other versioning used in Bitcoin/Bitcoin Core.

Also always set the nVersion field in CBlockLocator to 0 and ignore the
field in deserialized messages. The field is not currently used for
anything in Bitcoin Core.
2021-02-17 09:00:53 +00:00
John Newbery
652311165c [test] Move MIN_VERSION_SUPPORTED to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore move MIN_VERSION_SUPPORTED to p2p.py.

Also rename to MIN_P2P_VERSION_SUPPORTED to distinguish it from
other versioning used in Bitcoin/Bitcoin Core.
2021-02-17 09:00:24 +00:00
MarcoFalke
8639c446d8 Merge #21188: scripted-diff: Remove redundant lock annotations in net processing
fafddfadda scripted-diff: Remove shadowing lock annotations (MarcoFalke)

Pull request description:

  Would be good to not redundantly copy the lock annotation from the class declaration to the member implementation. Otherwise it may not result in a compile failure if a new lock requirement is added to the member implementation, but not the class declaration.

ACKs for top commit:
  amitiuttarwar:
    ACK `fafddfadda`, confirmed that the annotations removed were all redundant. confirmed the claim of potential issue :)
  hebasto:
    ACK fafddfadda
  jonatack:
    Light utACK fafddfadda verified that the removed annotations in the definitions correspond to those in their respective declarations

Tree-SHA512: ea095c6d4e0bedd70d4e2d8a42b06cfd90c161ebfcaac13558c5dc065601a732e5f812f332104b7daa087aa57b8b0242b177799d22eef7628d77d4d87f443bf2
2021-02-17 09:53:29 +01:00
MarcoFalke
7f831346cb Merge #20380: doc: Add instructions on how to fuzz the P2P layer using Honggfuzz NetDriver
fd0be92cff doc: Add instructions on how to fuzz the P2P layer using Honggfuzz NetDriver (practicalswift)

Pull request description:

  Add instructions on how to fuzz the P2P layer using [Honggfuzz NetDriver](http://blog.swiecki.net/2018/01/fuzzing-tcp-servers.html).

  Honggfuzz NetDriver allows for very easy fuzzing of TCP servers such as Bitcoin Core without having to write any custom fuzzing harness. The `bitcoind` server process is largely fuzzed without modification.

  This makes the fuzzing highly realistic: a bug reachable by the fuzzer is likely also remotely triggerable by an untrusted peer.

Top commit has no ACKs.

Tree-SHA512: 9e98cb30f00664c00c8ff9fd224ff9822bff3fd849652172df48dbaeade1dd1a5fc67ae53203f1966a1d4210671b35656009a2d8b84affccf3ddf1fd86124f6e
2021-02-17 09:50:56 +01:00
MarcoFalke
69f7f50aa5 Merge #20993: test: store subversion (user agent) as string in msg_version
de85af5cce test: store subversion (user agent) as string in msg_version (Sebastian Falbesoner)

Pull request description:

  It seems more natural to treat the "subversion" field (=user agent string, see [BIP 14](https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki#Proposal)) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in `msg_version.strSubVer`: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore.

  Note that currently, in the master branch the `msg_version.strSubVer` is never read (only in `msg_version.__repr__`); However, one issue that is solved by this PR came up while testing #19509 (not merged yet): A decoding script for binary message capture files takes use of the functional test framework convert it into JSON format. Bytestrings will be convered to hexstrings, while pure strings will (surprise surprise) end up without modification in the file.

  So without this patch, we get:

  ```
  $ jq . out.json | grep -m5 strSubVer
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
        "strSubVer": "2f5361746f7368693a302e32302e312f"
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
        "strSubVer": "2f5361746f7368693a302e32302e312f"
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
  ```

  After this patch:

  ```
  $ jq . out2.json | grep -m5 strSubVer
        "strSubVer": "/Satoshi:21.99.0/"
        "strSubVer": "/Satoshi:0.20.1/"
        "strSubVer": "/Satoshi:21.99.0/"
        "strSubVer": "/Satoshi:0.20.1/"
        "strSubVer": "/Satoshi:21.99.0/"
  ```

ACKs for top commit:
  jnewbery:
    utACK de85af5cce

Tree-SHA512: ff23642705c858e8387a625537dfec82e6b8a15da6d99b8d12152560e52d243ba17431b602b26f60996d897e00e3f37dcf8dc8a303ffb1d544df29a5937080f9
2021-02-17 09:36:30 +01:00
MarcoFalke
77772a1b80 doc: Rework internal and external links 2021-02-17 09:18:46 +01:00
MarcoFalke
9266f7497f util: Use std::chrono for time getters 2021-02-17 12:26:39 +08:00
Cory Fields
3c2e16be22 time: add runtime sanity check
std::chrono::system_clock.time_since_epoch and time_t(0) are not guaranteed
to use the Unix epoch timestamp, but in practice they almost certainly will.
Any differing behavior will be assumed to be an error, unless certain
platforms prove to consistently deviate, at which point we'll cope with it
by adding offsets.

Do a quick runtime check to verify that
time_t(0) == std::chrono::system_clock's epoch time == unix epoch.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2021-02-17 12:26:04 +08:00
fanquake
36be9b821a Merge #21087: guix: Passthrough BASE_CACHE into container
901f54321b guix: Passthrough BASE_CACHE into container (Carl Dong)

Pull request description:

  This allows depends-built packages to be cached.

ACKs for top commit:
  MarcoFalke:
    Approach ACK 901f54321b
  fanquake:
    ACK 901f54321b

Tree-SHA512: 464815f41fc081d7956bec84380668834b6ee6751c7a3d56daad6e1fc91e582de4bbdd1a89f399b1136f2adc4d9941517cfe4db694f0ee5bf59bf2f44fc6fda0
2021-02-17 12:25:06 +08:00
fanquake
c5da2749e2 build: actually stop configure if Boost isn't available
If Boost is not found via AX_BOOST_BASE, we don't actually stop
configuring, only a warning is emitted:
```bash
checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
```

Instead we would usually fail when one of the other
AX_BOOST_* macros fails to find a library. These macros are slowly being
removed, and in any case, it makes more sense to fail earlier if Boost
is missing.

If Boost is unavailable, the failure now looks like:
```bash
checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option.  If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
configure: error: Boost is not available!
```

Note that we now just pass the version into AX_BOOST_BASE, which fixes
it's display in the output (rather than MINIMUM_REQUIRED_BOOST).
2021-02-17 09:17:37 +08:00
fanquake
cad8b527ea build: explicitly install libboost-dev package
This package is currently installed as a side-effect of installing our
other libboost-*-dev packages. However as those continue to dissapear,
it makes sense to install boost dev explicitly.
2021-02-17 09:04:20 +08:00
fanquake
7c8e605bf4 Merge #21159: test: fix sign comparison warning in socket tests
9cc8e30125 test: fix sign comparison warning in socket tests (fanquake)

Pull request description:

  This fixes:
  ```bash
  In file included from test/sock_tests.cpp:10:
  In file included from /usr/local/include/boost/test/unit_test.hpp:18:
  In file included from /usr/local/include/boost/test/test_tools.hpp:46:
  /usr/local/include/boost/test/tools/old/impl.hpp:107:17: warning: comparison of integers of different signs: 'const long' and 'const unsigned long' [-Wsign-compare]
      return left == right;
             ~~~~ ^  ~~~~~
  ```

  which was introduced in #20788.

ACKs for top commit:
  practicalswift:
    cr ACK 9cc8e30125
  vasild:
    ACK 9cc8e30125

Tree-SHA512: 7069a4fde5cec01be03f8477fe396e53658f170efbf1d9ef3339d553bb90a2be9f4acd6b348127b14cd2f91426e0cd1fc35d2d3c9f201cf748c0cf50f47e46a5
2021-02-17 08:26:42 +08:00
fanquake
7bf04e358a build: remove mostly pointless BOOST_PROCESS macro
Performing a series of link checks for a Boost component that is
header-only doesn't make much sense, and currently means we just have
another confusing Boost macro in our tree. I'm not sure why this was
originally done this way; maybe Sjors or luke-jr can elaborate
(#15382 (929cda5470))?

The macro also has the side-effect of producing confusing error
messages. i.e in #20744, the CI is currently failing with:
```bash
checking for boostlib >= 1.58.0 (105800) lib path in "/tmp/cirrus-ci-build/depends/x86_64-pc-linux-gnu/lib"... yes
checking for boostlib >= 1.58.0 (105800)... yes
checking whether the Boost::Process library is available... yes
configure: error: Could not find a version of the Boost::Process library!
```

This isn't useful, given there is no such thing as a `Boost::Process`
library.

This PR just removes the macro entirely, but maintains a `--with-boost-process`
(defaulting to off), flag to configure. Hopefully this will also be
removed, in favour of `--enable-disable-external-signer` if/when #16546
is merged.
2021-02-17 08:04:11 +08:00
Jon Atack
6bfbc97d71 test: disallow sendtoaddress/sendmany when private keys disabled 2021-02-16 15:49:28 -05:00
Andrew Chow
0997019e76 Disallow sendtoaddress and sendmany when private keys disabled 2021-02-16 15:49:28 -05:00
Amiti Uttarwar
1363b6c27d [doc / util] Use comments to clarify time unit for int64_t type. 2021-02-16 12:23:00 -08:00
Amiti Uttarwar
47a7a1687d [util] Introduce a SetMockTime that takes chrono time 2021-02-16 12:23:00 -08:00
Amiti Uttarwar
df6a5fc1df [util] Change GetMockTime to return chrono type instead of int 2021-02-16 12:23:00 -08:00
Amiti Uttarwar
a2d908e1da [test] Throw error instead of segfaulting in failure scenario
If the miner code is faulty and does not include any transactions in a block,
the code segfaults when it tries to access block transactions. Instead, add a
check that safely aborts the process.
2021-02-16 12:23:00 -08:00
Amiti Uttarwar
9a3bbe8fc5 [test] Introduce a unit test helper to create a valid mempool transaction. 2021-02-16 12:23:00 -08:00
Wladimir J. van der Laan
92fee79dab Merge #19806: validation: UTXO snapshot activation
1afc0e4aa1 doc: remove potentially confusing ChainstateManager comment (James O'Beirne)
769a1ef9fd test: Add tests with maleated snapshot data (Fabian Jahr)
4d8de04f32 tests: add snapshot activation test (James O'Beirne)
31d225274f tests: add deterministic chain generation unittest fixture (James O'Beirne)
6606a4f8c6 move-onlyish: break out CreateUTXOSnapshot from dumptxoutset (James O'Beirne)
ad949ba449 txdb: don't reset during in-memory cache resize (James O'Beirne)
f6e2da5fb7 simplify ChainstateManager::SnapshotBlockhash() return semantics (James O'Beirne)
7a6c46b37e chainparams: add allowed assumeutxo values (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This change proposes logic for activating UTXO snapshots, which is unused at the moment aside from an included unittest. There are a few moveonyish/refactoring commits to allow for halfway decent unittests.

  Basic structure is included for specifying and checking the assumeutxo hash values used to validate activated snapshots. Initially I had specified a few height/hash pairs for mainnet in this change, but because of the security-critical nature of those parameters, I figured it was better to leave their inclusion to a future PR that includes only that change - my intent being that reviewers will be more likely to verify those parameters firsthand in a dedicated PR.

  Aside from that and the snapshot activation logic, there are a few related changes:

  - ~~allow caching the `nChainTx` value in the CCoinsViewDB; this is set during snapshot activation. Because we don't necessarily have access to the full chain at the time of snapshot load, this value is communicated through the snapshot metadata and must be cached within the chainstate to survive restarts.~~
  - break out `CreateUTXOSnapshot()` from dumptxoutset. This is essentially a move-only change to allow the reuse of snapshot creation logic from within unittests.
  - ...and a few other misc. changes that are solely related to unittests.

  The move-onlyish commit is most easily reviewed with `--color-moved=zebra`.

ACKs for top commit:
  fjahr:
    Code review ACK 1afc0e4aa1
  laanwj:
    Code review ACK 1afc0e4aa1

Tree-SHA512: a4e4f0698f00a53ec298b5e8b7ef1c9fdf0185f95139d1b1f63cfdf6cbbd6d17b8c6e51bbf1de2e5f1a946bf49f8466232698ef55acce5a012c80b067da366ea
2021-02-16 19:23:06 +01:00
MarcoFalke
3c9d9d21e1 Merge #21008: test: fix zmq test flakiness, improve speed
ef21fb7313 zmq test: speedup test by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
5c6546362d zmq test: fix flakiness by using more robust sync method (Sebastian Falbesoner)
8666033630 zmq test: accept arbitrary sequence start number in ZMQSubscriber (Sebastian Falbesoner)
6014d6e1b5 zmq test: dedup message reception handling in ZMQSubscriber (Sebastian Falbesoner)

Pull request description:

  Fixes #20934 by using the "sync up" method described in https://github.com/bitcoin/bitcoin/issues/20538#issuecomment-738791868.

  After improving robustness with this approach (commits 1-3), it turned out that there were still some fails, but those were unrelated to zmq: Out of 500 runs, 3 times `sync_mempool()` or `sync_blocks()` timed out, which can happen because the trickle relay time has no upper bound -- hence in rare cases, it takes longer than 60s. This is fixed by enabling immediate tx relay on node1 (commit 4), which as a nice side-effect also gives us a rough 2x speedup for the test.

  For further details, also see the explanations in the commit messages.

  There is no guarantee that the test is still not flaky, but it would help if potential reviewers would run the following script locally and report how many runs failed (feel free to do less than 1000 runs, as this takes quite a long if ran with `--valgrind`):
  ```
  #!/bin/sh
  OUTPUT_FILE=./zmq_results
  echo ===== repeated zmq test ===== > $OUTPUT_FILE

  for i in `seq 1000`; do
      echo ------------------------
      echo ----- test run $i -----
      echo ------------------------
      echo --- $i --- >> $OUTPUT_FILE
      ./test/functional/interface_zmq.py --valgrind
      if [ $? -ne 0 ]; then
          echo "FAILED. /o\\" >> $OUTPUT_FILE
      else
          echo "PASSED. \\o/" >> $OUTPUT_FILE
      fi
  done

  echo Failed test runs:
  grep FAILED $OUTPUT_FILE | wc -l
  ```

ACKs for top commit:
  jonatack:
    Light ACK ef21fb7313 with the caveat that I was unable to make the test fail with valgrind both here and on master, so I can't vouch that it actually fixes the CI flakiness. The test does run ~2x faster with this.

Tree-SHA512: 7a1e7592fbbd98e69e1e1294486b91253e589c72b3c6bbb7f587028ec07cca59b7d984e4ebf256c4bc3e8a529ec77d31842f3dd874038aea0b684abfea50306a
2021-02-16 18:56:20 +01:00
Carl Dong
901f54321b guix: Passthrough BASE_CACHE into container
This allows depends-built packages to be cached.
2021-02-16 12:17:33 -05:00
MarcoFalke
faa137eb9e test: Speed up rpc_blockchain.py by removing miniwallet.generate() 2021-02-16 17:47:55 +01:00
MarcoFalke
fa1fe80c75 test: Change address type from P2PKH to P2WSH in rpc_blockchain
This change does not matter for the test, except that it increases
the bogosize due to the increase in the size of the scriptPubKey
2021-02-16 17:47:48 +01:00
MarcoFalke
fa4d8f3169 test: Cache 25 mature coins for ADDRESS_BCRT1_P2WSH_OP_TRUE 2021-02-16 16:49:08 +01:00
MarcoFalke
fad25153f5 test: Remove unused bug workaround 2021-02-16 16:28:40 +01:00
MarcoFalke
faabce7d07 test: Start only the number of nodes that are needed 2021-02-16 16:25:50 +01:00
MarcoFalke
fa55159b9e net: Log to net debug in MaybeDiscourageAndDisconnect except for noban and manual peers 2021-02-16 13:29:00 +01:00
fanquake
9bbf08bf98 Merge #20721: Net: Move ping data to net_processing
a5e15ae45c scripted-diff: rename ping members (John Newbery)
45dcf22661 [net processing] Move ping data fields to net processing (John Newbery)
dd2646d12c [net processing] Move ping timeout logic to net processing (John Newbery)
0b43b81f69 [net processing] Move send ping message logic into function (John Newbery)
1a07600b4b [net] Add RunInactivityChecks() (John Newbery)
f8b3058992 [net processing] Add Peer& arg to MaybeDiscourageAndDisconnect() (John Newbery)

Pull request description:

  This continues the work of moving application layer data into net_processing, by moving all ping data into the new Peer object added in #19607.

  For motivation, see #19398.

ACKs for top commit:
  glozow:
    reACK a5e15ae45c
  MarcoFalke:
    review ACK a5e15ae45c 🥉
  amitiuttarwar:
    ACK a5e15ae45c

Tree-SHA512: fb84241613d6a6e1f2832fa5378030b5877a02e8308188f57ab545a6eaf2ab731a93abb7dcd3a7f7285bb66700f938096378a8e90cd6a3e6f3309f81d85a344e
2021-02-16 18:48:30 +08:00
Jonas Schnelli
84716b134e Add "index/blockfilterindex -> validation -> index/blockfilterindex" to expected circular dependencies 2021-02-16 10:30:41 +01:00
Jonas Schnelli
ab3a0a2fb9 Add functional test for blockfilterindex in prune-mode 2021-02-16 10:30:37 +01:00
Jonas Schnelli
c286a22f7b Add debug startup parameter -fastprune for more effective pruning tests 2021-02-16 10:26:17 +01:00
Jonas Schnelli
5e112269c3 Avoid pruning below the blockfilterindex sync height 2021-02-16 10:26:15 +01:00
MarcoFalke
b55dc3ad84 Merge #21185: fuzz: Remove expensive and redundant muhash from crypto fuzz target
ffff84a9cb fuzz: Remove expensive and redundant muhash from crypto fuzz target (MarcoFalke)

Pull request description:

  Remove because it is redundant with `src/test/fuzz/muhash.cpp` and incredibly expensive

ACKs for top commit:
  practicalswift:
    Tested ACK ffff84a9cb

Tree-SHA512: c91ea2406db857127c789b9cdeb714a719d88b54132e9cef74fffd229532d874b6c043353793ec687504b5784afc74995f8982243d41f976b63d57454a5ed339
2021-02-16 07:54:28 +01:00
Martin Ankerl
74bf850ac4 faster HexStr => 13% faster blockToJSON
`std::string`'s push_back is rather slow because it needs to check & update the string size. For
`HexStr` the output string size is already easily know, so we can initially create the string with
the correct size and then just assign the data.

`HexStr` is heavily usd in `blockToJSON`, so this change is a noticeable benefit. Benchmark on an i7-8700 @3.2GHz:

* 71,315,461.00 ns/op master
* 62,842,490.00 ns/op this commit

So this little change makes `blockToJSON` about ~13% faster.
2021-02-16 07:33:55 +01:00
Wladimir J. van der Laan
882ce25132 cli: Treat high detail levels as the maximum in -netinfo
I somehow often type `-netinfo 5` which gets treated as `-netinfo 0`,
after this change it's `-netinfo 4` which seems more convenient behavior.
2021-02-15 20:01:52 +01:00
John Newbery
a5e15ae45c scripted-diff: rename ping members
-BEGIN VERIFY SCRIPT-
sed -i 's/fPingQueued/m_ping_queued/g' src/net_processing.cpp
sed -i 's/nMinPingUsecTime/m_min_ping_time/g' src/net.* src/net_processing.cpp src/test/net_tests.cpp
sed -i 's/nPingNonceSent/m_ping_nonce_sent/g' src/net_processing.cpp
sed -i 's/nPingUsecTime/m_last_ping_time/g' src/net.*
-END VERIFY SCRIPT-
2021-02-15 16:15:51 +00:00
John Newbery
45dcf22661 [net processing] Move ping data fields to net processing 2021-02-15 16:15:51 +00:00
John Newbery
dd2646d12c [net processing] Move ping timeout logic to net processing
Ping messages are an application-level mechanism. Move timeout
logic from net to net processing.
2021-02-15 16:02:43 +00:00
John Newbery
0b43b81f69 [net processing] Move send ping message logic into function 2021-02-15 16:02:43 +00:00
John Newbery
1a07600b4b [net] Add RunInactivityChecks()
Moves the logic to prevent running inactivity checks until
the peer has been connected for -peertimeout time into its
own function. This will be reused by net_processing later.
2021-02-15 16:02:43 +00:00
John Newbery
f8b3058992 [net processing] Add Peer& arg to MaybeDiscourageAndDisconnect()
Refactor only. No change in behaviour.
2021-02-15 16:02:43 +00:00
MarcoFalke
489030f2a8 Merge #20965: net, rpc: return NET_UNROUTABLE as not_publicly_routable, automate helps
96635e6177 init: use GetNetworkNames() in -onlynet help (Jon Atack)
0dbde700a6 rpc: use GetNetworkNames() in getnetworkinfo and getpeerinfo helps (Jon Atack)
1c3af37881 net: create GetNetworkNames() (Jon Atack)
b45eae4d53 net: update NET_UNROUTABLE to not_publicly_routable in GetNetworkName() (Jon Atack)

Pull request description:

  per the IRC discussion today at http://www.erisian.com.au/bitcoin-core-dev/log-2021-01-19.html#l-87

  - return a more helpful string name for `Network::NET_UNROUTABLE`: "not_publicly_routable" instead of "unroutable"
  - update the RPC getpeerinfo "network" help, and automate it and the getnetworkinfo "network#name" and the -onlynet help doc generation

ACKs for top commit:
  theStack:
    re-ACK 96635e6177 🌳
  MarcoFalke:
    review ACK 96635e6177 🐗

Tree-SHA512: 511a7d987126b48a7a090739aa7c4964b6186a3ff8f5f7eec9233dd816c6b7a6dc91b3ea6b824aa68f218a8a3ebdc6ffd214e9a88af38f2bf23f3257c4284c3a
2021-02-15 15:31:15 +01:00
MarcoFalke
cb073bed00 Merge #21167: net: make CNode::m_inbound_onion public, initialize explicitly
2ee4a7a9ec net: remove CNode::m_inbound_onion defaults for explicitness (Jon Atack)
24bda56c29 net: make CNode::m_inbound_onion public, drop getter, update tests (Jon Atack)

Pull request description:

  Refactoring only, no change in behavior. This is a quick follow-up to #20210 to address these review comments:

  - https://github.com/bitcoin/bitcoin/pull/20210#discussion_r528835313
  - https://github.com/bitcoin/bitcoin/pull/20210#discussion_r550860416
  - https://github.com/bitcoin/bitcoin/pull/20210#issuecomment-766093925

  Changes:
  - make the `CNode::m_inbound_onion class` member public, update the Doxygen comment, drop the getter, and update the tests
  - remove the `CNode::m_inbound_onion` default value initialization in the ctor declaration and the member initializer in favor of always passing it explicitly to the ctor where we initialize it dynamically, to both clarify the caller code and to allow the compiler to warn if it is uninitialized in the ctor or omitted in the caller

ACKs for top commit:
  MarcoFalke:
    review ACK 2ee4a7a9ec 🏀
  vasild:
    ACK 2ee4a7a9ec

Tree-SHA512: 72961c91168885a9d881756b10bad9d587f5ce297d5a6493c23e267b7178ff22b697bc6868e7761d6304e372d2781453d30011a020afd506b1e308b4ffa5feee
2021-02-15 15:22:21 +01:00
MarcoFalke
d19639d2b6 Merge #21096: Re-add dead code detection
3f8776a139 Re-add dead code detection (flack)

Pull request description:

  This re-adds unreachable code detection for Python based on `vulture`.

  Effectively, this reverts f4beb4996d. The difference to the previous version is that this runs with the `--min-confidence 100` setting. From https://pypi.org/project/vulture/:

  > Use `--min-confidence 100` to only report code that is guaranteed to be unused within the analyzed files.

  So this should avoid the previous issues where static analysis had wrong positives due to the dynamic nature of Python code by only reporting things that are unambiguous (such as code after a `return` statement). As such, there is not suppressions list.

  My motivation was mainly #21081 which would have been caught by this (as can be seen by the CI run failing). This is still marked as draft because #21081 is needed to get the linter to pass. Also, there is a second problem that this found (see https://github.com/bitcoin/bitcoin/pull/19509/files#r571454691). From what I can tell, this is a spurious type comment that could just be removed (or if that line has no side effects it could also be deleted altogether?). I could add a commit here to fix it, but I wanted to see if there is interest in having this linter again in the first place

ACKs for top commit:
  practicalswift:
    ACK 3f8776a139

Tree-SHA512: 52314ad4f627d969de1eb15375ca677ed86a2e816fe773756a1ce22421214ba407b5a09a4bf701a3aab1a10c7b336f548e4cef3327edf154acba55e987db21f6
2021-02-15 15:13:57 +01:00
MarcoFalke
ffff84a9cb fuzz: Remove expensive and redundant muhash from crypto fuzz target 2021-02-15 14:39:08 +01:00
MarcoFalke
fafddfadda scripted-diff: Remove shadowing lock annotations
Can be reviewed with --word-diff-regex=.

-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended 's/(PeerManagerImpl::.*\)).*LOCKS_.*\)/\1/g' ./src/net_processing.cpp
-END VERIFY SCRIPT-
2021-02-15 12:52:04 +01:00
MarcoFalke
8d6994f93d Merge #21100: test: remove unused function xor_bytes
f64adc1eed test: remove unused function xor_bytes (Sebastian Falbesoner)

Pull request description:

  The function `xor_bytes` was introduced in commit 3c226639eb (#19953, BIP340-342 validation), even [code-reviewed](https://github.com/bitcoin/bitcoin/pull/19953/files#r509383731), but actually never used. The [default signing algorithm in BIP340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki#Default_Signing) needs a xor operation, but this step is currently done by a single xor operation on large integer operands:

  ```
  t = (sec ^ int.from_bytes(TaggedHash("BIP0340/aux", aux), 'big')).to_bytes(32, 'big')
  ```

  Alternatively, we could keep the function and as well use it:
  ```diff
  --- a/test/functional/test_framework/key.py
  +++ b/test/functional/test_framework/key.py
  @@ -492,7 +492,7 @@ def sign_schnorr(key, msg, aux=None, flip_p=False, flip_r=False):
       P = SECP256K1.affine(SECP256K1.mul([(SECP256K1_G, sec)]))
       if SECP256K1.has_even_y(P) == flip_p:
           sec = SECP256K1_ORDER - sec
  -    t = (sec ^ int.from_bytes(TaggedHash("BIP0340/aux", aux), 'big')).to_bytes(32, 'big')
  +    t = xor_bytes(sec.to_bytes(32, 'big'), TaggedHash("BIP0340/aux", aux))
       kp = int.from_bytes(TaggedHash("BIP0340/nonce", t + P[0].to_bytes(32, 'big') + msg), 'big') % SECP256K1_ORDER
       assert kp != 0
       R = SECP256K1.affine(SECP256K1.mul([(SECP256K1_G, kp)]))
  ```

ACKs for top commit:
  practicalswift:
    cr ACK f64adc1eed: untested unused code should be removed

Tree-SHA512: e9afae303488f19c6f6f44874d3537ed1c8164a197490e2b4e34853db886b858825b719648fa1a30b95177dcee9cf241f94ee9b835f0a2cae07024ce38a8c090
2021-02-15 12:10:41 +01:00
MarcoFalke
45ea103f8f Merge #20942: [refactor] Move some net_processing globals into PeerManagerImpl
6452190841 net_processing: simplify MaybeSetPeerAsAnnouncingHeaderAndIDs args (Anthony Towns)
39c2a69bc2 net_processing: move MaybeSetPeerAsAnnouncingHeadersAndIDs into PeerManagerImpl (Anthony Towns)
7b7117efd0 net_processing: simplify ProcessGetData and FindTxForGetData args (Anthony Towns)
34207b9004 net_processing: move FindTxForGetData and ProcessGetData to PeerManagerImpl (Anthony Towns)
d44084883a net_processing: simplify PeerManageImpl method args (Anthony Towns)
a490f0a056 net_processing: move MarkBlockAs*, TipMayBeStale, FindNextBlocksToDL to PeerManagerImpl (Anthony Towns)
052d9bc7e5 net_processing: simplify AlreadyHaveTx args (Anthony Towns)
eeac506250 net_processing: move AlreadyHaveTx into PeerManageImpl (Anthony Towns)
9781c08a33 net_processing: move some globals into PeerManagerImpl (Anthony Towns)

Pull request description:

  Turns some globals into member variables, and simplifies the parameter list for some of net_processing's internal functions. Mostly just serves as a code cleanup at this point.

ACKs for top commit:
  jnewbery:
    Code review ACK 6452190841
  ariard:
    Code Review ACK 6452190, changes are pretty straightforward.
  MarcoFalke:
    Concept ACK 6452190841 I have not reviewed this, but I left a comment 🐡

Tree-SHA512: 381361f9dbfeb851a5522ead3165ce1447a0f212ddea4b483aa38975559ee5ed03a4ba69c24fd69f36847a1eddfef05785f5cbb2fcec5fe50f8b336e8047c3b1
2021-02-15 12:02:43 +01:00
Wladimir J. van der Laan
51397c0ff7 Merge #20629: depends: Improve id string robustness
5200929bfe depends: Include GUIX_ENVIRONMENT in id string (Carl Dong)
4c7d418588 depends: Improve id string robustness (Carl Dong)
b3bdff42b5 build: Proper quoting for var printing targets (Carl Dong)

Pull request description:

  ```
  Environment variables and search paths can drastically effect the
  operation of build tools.

  Include these in our id string to mitigate against false cache hits.
  ```

  Note to builders: This will invalidate all depends output caches in `BASE_CACHE`

ACKs for top commit:
  laanwj:
    re-ACK 5200929bfe

Tree-SHA512: e70c98da89cde90dc54bc3be89b925787cf94bbf246e27cc9345816b312073d78a02215448f731f21d8cf033c455234a2377ff1d66c00e1f3db69c9c9687d027
2021-02-15 11:43:00 +01:00
MarcoFalke
df8892dc9f Merge #20986: docs: update developer notes to discourage very long lines
aa929abf8d [docs] Update developer notes to discourage very long lines (John Newbery)

Pull request description:

  Mandatory rules on line lengths are bad - there will always be cases where a longer line is more readable than the alternative.

  However, very long lines for no good reason _do_ hurt readability. For example, this declaration in validation.h is 274 chars:

  ```c++
      bool ConnectTip(BlockValidationState& state, const CChainParams& chainparams, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool.cs);
  ```

  That won't fit on one line without wrapping on my 27" monitor with a comfortable font size. Much easier to read is something like:

  ```c++
      bool ConnectTip(BlockValidationState& state, const CChainParams& chainparams,
                      CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock,
                      ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool)
          EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool.cs);
  ```

  Therefore, _discourage_ (don't forbid) line lengths greater than 100 characters in our developer style guide.

  100 chars is somewhat arbitrary. The old standard was 80, but that seems very limiting with modern displays.

ACKs for top commit:
  fanquake:
    ACK aa929abf8d - this is basically just something to point too when a PR has unreasonably long lines for no particularly reason.
  practicalswift:
    ACK aa929abf8d
  amitiuttarwar:
    ACK aa929abf8d
  theStack:
    ACK aa929abf8d
  glozow:
    ACK aa929abf8d

Tree-SHA512: 17f1b11f811137497ede8851ede93fa612dc622922b5ad7ac8f065ea026d9a718db5b92325754b74d24012b4d45c4e2cd5cd439a6a8d34bbabf5da927d783970
2021-02-14 09:48:31 +01:00
Wladimir J. van der Laan
e4f0c4cd73 Merge #21163: doc: Guix is shipped in Debian and Ubuntu
fa051c2386 doc: Guix is shipped in Debian and Ubuntu (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa051c2386 🚀

Tree-SHA512: f72f546cfc20cf1cc0c26c2306ac06416ada87661596fe811b497cce646aa286dc4aee832145bf838b13fbd3c5f064519eb8c0b4525eb562f2f04f20e2876ffc
2021-02-13 23:39:20 +01:00
Wladimir J. van der Laan
43981ee2c8 Merge #21127: wallet: load flags before everything else
9305862f71 wallet: load flags before everything else (Sjors Provoost)

Pull request description:

  Load and set wallet flags before processing other records. That way we can take them into account while processing those other records.

  Suggested here:
  https://github.com/bitcoin/bitcoin/pull/16546#discussion_r572334983

ACKs for top commit:
  laanwj:
    Code review ACK 9305862f71
  gruve-p:
    ACK 9305862f71
  achow101:
    ACK 9305862f71

Tree-SHA512: 7104523e369ce3c670571fe5e8b52c67b9ca92b8e36a2da5eb6f9f8bf8ed0544897007257204b68f6f371d682b3ef0d0635d36e6e8416ac74af1999d9fbc869c
2021-02-13 23:30:50 +01:00
Wladimir J. van der Laan
b08cbd09b8 Merge #21028: doc/bips: Add BIPs 43, 44, 49, and 84
c943326d3c doc/bips: Add BIPs 43, 44, 49, and 84 (Luke Dashjr)

Pull request description:

  If you don't like what they say, please suggest alternatives ;)

ACKs for top commit:
  prusnak:
    ACK c943326

Tree-SHA512: 7db93f8491289657ec45df30e557eb8572b35201eb29aed1b11bf3949924fce56b4e2d71e1f0acf5d24a01278c0dec99790d632f04c15117010c4ac564368d6b
2021-02-13 18:36:37 +01:00
Martin Ankerl
e3e0a2432c Add benchmark to write JSON into a string
The benchmark BlockToJsonVerbose only tests generating (and destroying)
the JSON data structure, but serializing into a string is also a
performance critical aspect of the RPC calls.

Also, use ankerl::nanobench::doNotOptimizeAway to make sure the compiler
can't optimize the result of the calls away.
2021-02-13 13:14:16 +01:00
flack
3f8776a139 Re-add dead code detection 2021-02-13 09:57:50 +01:00
Jon Atack
2ee4a7a9ec net: remove CNode::m_inbound_onion defaults for explicitness
and to allow the compiler to warn if uninitialized in the ctor
or omitted in the caller.
2021-02-12 22:32:08 +01:00
Jon Atack
24bda56c29 net: make CNode::m_inbound_onion public, drop getter, update tests 2021-02-12 22:23:15 +01:00
Luke Dashjr
c943326d3c doc/bips: Add BIPs 43, 44, 49, and 84 2021-02-12 20:48:18 +00:00
MarcoFalke
bf3189eda6 Merge #21165: test: Use mocktime in test_seed_peers
d4187e4619 [test] Use mocktime in test_seed_peers() (Dhruv Mehta)
015637dd44 [refactor] Correct log message in net.cpp (Dhruv Mehta)

Pull request description:

  The test now takes less than 5 seconds instead of more than 2 minutes

  Further context: https://github.com/bitcoin/bitcoin/pull/19884/files#r575336503

  Before:
  ```
  2021-02-12T17:22:25.980000Z TestFramework (INFO): Test seed peers, this will take about 2 minutes
  2021-02-12T17:24:30.472000Z TestFramework (INFO): Test -networkactive option
  ```

  After:
  ```
  2021-02-12T17:33:39.224000Z TestFramework (INFO): Test seed peers
  2021-02-12T17:33:43.139000Z TestFramework (INFO): Test -networkactive option
  ```

Top commit has no ACKs.

Tree-SHA512: 6d8df7d4433c96268694577e4c10a346785e076d45fa220091875e55def200100e7b827fac2a1f7853a2c2c39e9661e06288dca8c645da9e13d4318a4ff2172e
2021-02-12 20:33:18 +01:00
Dhruv Mehta
d4187e4619 [test] Use mocktime in test_seed_peers()
Test case now takes < 5 seconds instead of > 2 minutes
2021-02-12 09:35:18 -08:00
Dhruv Mehta
015637dd44 [refactor] Correct log message in net.cpp 2021-02-12 09:23:03 -08:00
MarcoFalke
fa051c2386 doc: Guix is shipped in Debian and Ubuntu 2021-02-12 14:59:06 +01:00
James O'Beirne
1afc0e4aa1 doc: remove potentially confusing ChainstateManager comment 2021-02-12 07:53:41 -06:00
Fabian Jahr
769a1ef9fd test: Add tests with maleated snapshot data 2021-02-12 07:53:40 -06:00
James O'Beirne
4d8de04f32 tests: add snapshot activation test 2021-02-12 07:53:37 -06:00
James O'Beirne
31d225274f tests: add deterministic chain generation unittest fixture 2021-02-12 07:53:36 -06:00
James O'Beirne
6606a4f8c6 move-onlyish: break out CreateUTXOSnapshot from dumptxoutset
This move/refactor is needed to set up a decent unittest for UTXO snapshot activation.
2021-02-12 07:53:34 -06:00
James O'Beirne
ad949ba449 txdb: don't reset during in-memory cache resize
We can't support a reset of the dbwrapper object when in-memory configuration is used
because it results in the permanent loss of coins. This only affects unittest
configurations (since that's the only place we use in-memory CCoinsViewDB instances).
2021-02-12 07:53:32 -06:00
James O'Beirne
f6e2da5fb7 simplify ChainstateManager::SnapshotBlockhash() return semantics
Don't return null snapshotblockhash values to avoid caller complexity/confusion.
2021-02-12 07:53:29 -06:00
James O'Beirne
7a6c46b37e chainparams: add allowed assumeutxo values
Values for mainnet and testnet will be specified in a follow-up PR that can be
scrutinized accordingly. This structure is required for use in snapshot activation
logic.
2021-02-12 07:53:22 -06:00
Wladimir J. van der Laan
e9c037ba64 Merge #19884: p2p: No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty
fe3e993968 [p2p] No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty. Add -fixedseeds arg. (Dhruv Mehta)

Pull request description:

  Closes #19795

  Before PR: If `peers.dat` is empty and `-dnsseed=0`, bitcoind will fallback on to fixed seeds but only after a 60 seconds delay.
  After PR: There's no 60 second delay.

  To reproduce:
  `rm ~/.bitcoin/peers.dat && src/bitcoind -dnsseed=0` without and with patch code

  Other changes in the PR:
  - `-fixedseeds` command line argument added: `-dnsseed=0 -fixedseeds=0 -addnode=X` provides a trusted peer only setup. `-dnsseed=0 -fixedseeds=0` allows for a `addnode` RPC to add a trusted peer without falling back to hardcoded seeds.

ACKs for top commit:
  LarryRuane:
    re-ACK fe3e993968
  laanwj:
    re-ACK fe3e993968

Tree-SHA512: 79449bf4e83a315be6dbac9bdd226de89d2a3f7f76d9c5640a2cb3572866e6b0e8ed67e65674c9824054cf13119dc01c7e1a33848daac6b6c34dbc158b6dba8f
2021-02-12 11:49:34 +01:00
Wladimir J. van der Laan
9996b1806a Merge #21064: refactor: use std::shared_mutex & remove Boost Thread
060a2a64d4 ci: remove boost thread installation (fanquake)
06e1d7d81d build: don't build or use Boost Thread (fanquake)
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)

Pull request description:

  This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).

  Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.

  Two other points:

  [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
  > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.

  Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
  * The version of Boost.
  * The platform you're building for.
  * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
  * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.

  A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.

  With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.

  Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
  Also related to #21022 - pthread sanity checking.

ACKs for top commit:
  laanwj:
    Code review ACK 060a2a64d4
  vasild:
    ACK 060a2a64d4

Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-12 11:39:36 +01:00
Wladimir J. van der Laan
54b66a6e5f Merge #19522: build: fix building libconsensus with reduced exports for Darwin targets
de4238f92f build: consolidate reduced export checks (fanquake)
012bdec1b7 build: add building libconsensus to end-of-configure output (fanquake)
8f360e349e build: remove ax_gcc_func_attribute macro (fanquake)
f054a089ec build: remove AX_GCC_FUNC_ATTRIBUTE test for dllimport (fanquake)
7cd0a69664 build: test for __declspec(dllexport) in configure (fanquake)
1624e17b54 build: remove duplicate visibility attribute detection (fanquake)

Pull request description:

  Darwin targets do not have a `protected` visibility function attribute, see [LLVM explanation](8e9a505139/clang/lib/Basic/Targets/OSTargets.h (L131)). This means that the `AX_GCC_FUNC_ATTRIBUTE` check for `visibility` fails:
  ```bash
  configure:24513: checking for __attribute__((visibility))
  configure:24537: g++ -std=c++11 -o conftest -g -O2  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS -DMAC_OSX -DOBJC_OLD_DISPATCH_PROTOTYPES=0  -Wl,-headerpad_max_install_names conftest.cpp  >&5
  conftest.cpp:35:56: warning: target does not support 'protected' visibility; using 'default' [-Wunsupported-visibility]
                      int foo_pro( void ) __attribute__((visibility("protected")));
                                                         ^
  1 warning generated.
  configure:24537: $? = 0
  configure:24550: result: no
  ```

  This leads to `EXPORT_SYMBOL` being [defined to nothing](f4de89edfa/src/script/bitcoinconsensus.h (L29)), as `HAVE_FUNC_ATTRIBUTE_VISIBILITY` is not defined, and when building with reduced exports, you end up with a libbitcoinconsensus.dylib that doesn't export any  `_bitcoinconsensus_*` symbols.
  ```bash
  ➜  git:(master) nm -C src/.libs/libbitcoinconsensus.dylib | rg _bitcoinconsensus_
  ➜  git:(master)
  ```

  We do have a [second check](f4de89edfa/configure.ac (L882)) for the `visibility` attribute, which works for Darwin as it's only testing for default visibility, however the result of this check isn't used at all. It was added in #4725, along with the `--enable-reduce-exports` option, however when libbitcoinconsensus was added in #5235, it used the results of the added `AX_GCC_FUNC_ATTRIBUTE` calls.

  This PR removes our usage of the AX_GCC_FUNC_ATTRIBUTE macro entirely, in favour of our own checks in configure. This meant adding a check for `dllexport`, which I've tested as working with both [GCC](https://gcc.gnu.org/onlinedocs/gcc/Microsoft-Windows-Function-Attributes.html) and [Clang](https://releases.llvm.org/10.0.0/tools/clang/docs/AttributeReference.html#dllexport) when building for Windows. I haven't added an equivalent check for `dllimport`, as we weren't actually using the result of that check, we're just testing that `MSC_VER` was defined before using.

  With these changes building a libbitcoinconsensus with reduced exports, when targeting Darwin, works as expected:
  ```bash
  ./autogen.sh
  ./configure --disable-tests --disable-bench --with-utils=no --with-daemon=no --with-gui=no --disable-wallet --with-libs=yes --enable-reduce-exports
  make -j8
  ...
  nm -C src/.libs/libbitcoinconsensus.dylib | rg _bitcoinconsensus_
  000000000000a340 T _bitcoinconsensus_verify_script
  00000000000097e0 T _bitcoinconsensus_verify_script_with_amount
  000000000000a3c0 T _bitcoinconsensus_version
  ```

  ```python
  >>> import ctypes
  >>> consensus = ctypes.CDLL("src/.libs/libbitcoinconsensus.dylib")
  >>> print(consensus.bitcoinconsensus_version())
  1
  >>> exit()
  ```

  TODO: Modify a CI job to compile with --enable-reduce-exports and check for symbols in shared lib?

ACKs for top commit:
  laanwj:
    Code review ACK de4238f92f

Tree-SHA512: d148f3c55d14dac6e9e5b718cc65bb557bcf6f663218d24bc9044b86281bd5dd3d931ebea79c336a58e8ed50d683218c0a9e75494f2267b91097665043e252ae
2021-02-12 11:11:55 +01:00
Wladimir J. van der Laan
8d82eddee6 Merge #19145: Add hash_type MUHASH for gettxoutsetinfo
e987ae5a55 test: Add test for deterministic UTXO set hash results (Fabian Jahr)
6ccc8fc067 test: Add test for gettxoutsetinfo RPC with MuHash (Fabian Jahr)
0d3b2f643d rpc: Add hash_type MUHASH to gettxoutsetinfo (Fabian Jahr)
2474645f3b refactor: Separate hash and stats calculation in coinstats (Fabian Jahr)
a1fcceac69 refactor: Improve encapsulation between MuHash3072 and Num3072 (Fabian Jahr)

Pull request description:

  This is another Pr in the series PRs for Coinstatsindex (see overview in #18000). This PR adds the `hash_type` option `muhash` to `gettxoutsetinfo` through which the user can calculate the serialized muhash of the utxo set. This PR does not use the index yet.

ACKs for top commit:
  Sjors:
    tACK e987ae5
  achow101:
    ACK e987ae5a55
  jonatack:
    Tested re-ACK e987ae5a55 per `git diff 3506d90 e987ae5`, reviewed diff, debug built, ran gettxoutsetinfo -signet and help on this branch vs master, at height 23127 both returned `hash_serialized_2` of `2b72d65f3b6efb2311f58374ea2b939abf49684d44f4bafda45faa3b5452a454` and this branch returned `muhash` of `c9f1ff12d345ccf9939c6bbf087e6f7399b6115adee1569287e9c5c43dbb475c`
  ryanofsky:
    Code review ACK e987ae5a55. Looks very good. I left one suggestion to simplify code, but feel free to ignore it here and maybe consider it for later since PR has already had a lot of review.

Tree-SHA512: 9a739ce375e73749fa69a467262b60d3e5314ef384e2d7150b3bbc8e4125cd9fd1db95306623bb9a632fcbaf5d9d2bf2f5cc43bf717d4ff5e2c9c4b52dd9296c
2021-02-12 10:47:41 +01:00
fanquake
de4238f92f build: consolidate reduced export checks 2021-02-12 16:02:21 +08:00
fanquake
9cc8e30125 test: fix sign comparison warning in socket tests
This fixes:
```bash
In file included from test/sock_tests.cpp:10:
In file included from /usr/local/include/boost/test/unit_test.hpp:18:
In file included from /usr/local/include/boost/test/test_tools.hpp:46:
/usr/local/include/boost/test/tools/old/impl.hpp:107:17: warning: comparison of integers of different signs: 'const long' and 'const unsigned long' [-Wsign-compare]
    return left == right;
           ~~~~ ^  ~~~~~
```

which was introduced in #20788.
2021-02-12 10:12:00 +08:00
fanquake
012bdec1b7 build: add building libconsensus to end-of-configure output 2021-02-12 09:04:16 +08:00
fanquake
8f360e349e build: remove ax_gcc_func_attribute macro
This is no-longer used.
2021-02-12 09:04:16 +08:00
fanquake
f054a089ec build: remove AX_GCC_FUNC_ATTRIBUTE test for dllimport
The result of this test isn't currently used anywhere (we use dllimport based on
MSC_VER in libconsensus).
2021-02-12 09:04:16 +08:00
fanquake
7cd0a69664 build: test for __declspec(dllexport) in configure
This should work for GCC and Clang when building for Windows targets.
2021-02-12 09:04:16 +08:00
fanquake
1624e17b54 build: remove duplicate visibility attribute detection
We are already testing for this, and our test works correctly with a Darwin
target, where the macro does not. Darwin targets do not support "protected"
visibility.
2021-02-12 09:04:15 +08:00
Dhruv Mehta
fe3e993968 [p2p] No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty. Add -fixedseeds arg. 2021-02-11 16:10:40 -08:00
Wladimir J. van der Laan
937dfa8398 Merge #21041: log: Move "Pre-allocating up to position 0x[…] in […].dat" log message to debug category
25f899cc23 log: Move "Pre-allocating up to position 0x[...] in [...].dat" log message to debug category (practicalswift)
acd7980b37 log: Move "Leaving block file [...]: [...]" log message to debug category (practicalswift)

Pull request description:

  Move `Pre-allocating up to position 0x[…] in […].dat` log message to debug category.

  After the cleanup of `-debug=net` log messages PR (#20724) was merged recently the console log now has very high signal to noise ratio. That's great! :)

  This PR increases the signal to noise ratio slightly more by moving the most common remaining implementation detail log message (`Pre-allocating up to position 0x[…] in […].dat`) to the debug category where it belongs :)

  Expected standard output from `bitcoind` (when in steady state) before this patch:

  ```
  $ src/bitcoind
  …
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z Pre-allocating up to position 0x0000000 in blk00000.dat
  0000-00-00T00:00:00Z Pre-allocating up to position 0x000000 in rev00000.dat
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  ```

  Expected standard output from `bitcoind` (when in steady state) after this patch:

  ```
  $ src/bitcoind
  …
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  0000-00-00T00:00:00Z UpdateTip: new best=0000000000000000000000000000000000000000000000000000000000000000 height=000000 version=0x00000000 log0_work=00.000000 tx=000000000 date='0000-00-00T00:00:00Z' progress=0.000000 cache=000.0MiB(0000000txo)
  ```

  I find the latter alternative much easier to visually scan for anomalies (and more aesthetically pleasing TBH!).

  Non-GUI users deserve nice interfaces too :)

ACKs for top commit:
  laanwj:
    ACK 25f899cc23

Tree-SHA512: 5970798c41b041527ebdcbd843c5e136c257c28c3b21fc74102da8970406ca5c0c7e406305c5e6e67de5c1708dc1858af07a77a2e05f44159b7103423e8ab32f
2021-02-11 19:58:20 +01:00
MarcoFalke
0ad4656790 Merge #20370: fuzz: version handshake
fabce459bb fuzz: version handshake (MarcoFalke)

Pull request description:

  Not fuzzing the version handshake will limit fuzz coverage

ACKs for top commit:
  practicalswift:
    cr ACK fabce459bb: patch looks very much correct

Tree-SHA512: 4091d27d39edee781d033e471b352084bb54df250d0890e4821a325926a44dff9b26a2614d67dd0529f73bd366b075d7a0a1a570c2837de286a1b93a59a8fb91
2021-02-11 17:25:33 +01:00
MarcoFalke
8e1913ae02 Merge #21062: refactor: return MempoolAcceptResult from ATMP
53e716ea11 [refactor] improve style for touched code (gzhao408)
174cb5330a [refactor] const ATMPArgs and non-const Workspace (gzhao408)
f82baf0762 [refactor] return MempoolAcceptResult (gzhao408)
9db10a5506 [refactor] clean up logic in testmempoolaccept (gzhao408)

Pull request description:

  This is the first 4 commits of #20833, and does refactoring only. It should be relatively simple to review, and offers a few nice things:
  - It makes accessing values that don't make sense (e.g. fee) when the tx is invalid an error.
  - Returning `MempoolAcceptResult` from ATMP makes the interface cleaner. The caller can get a const instead of passing in a mutable "out" param.
  - We don't have to be iterating through a bunch of lists for package validation, we can just return a `std::vector<MempoolAcceptResult>`.
  - We don't have to refactor all ATMP call sites again if/when we want to return more stuff from it.

ACKs for top commit:
  MarcoFalke:
    ACK 53e716ea11 💿
  jnewbery:
    Code review ACK 53e716ea11
  ariard:
    Code Review ACK 53e716e, I did tweak a bit the touched paths to see if we had good test coverage. Didn't find holes.

Tree-SHA512: fa6ec324a08ad9e6e55948615cda324cba176255708bf0a0a0f37cedb7a75311aa334ac6f223be7d8df3c7379502b1081102b9589f9a9afa1713ad3d9ab3c24f
2021-02-11 14:45:41 +01:00
Wladimir J. van der Laan
a1be08405d Merge #20788: net: add RAII socket and use it instead of bare SOCKET
615ba0eb96 test: add Sock unit tests (Vasil Dimov)
7bd21ce1ef style: rename hSocket to sock (Vasil Dimov)
04ae846904 net: use Sock in InterruptibleRecv() and Socks5() (Vasil Dimov)
ba9d73268f net: add RAII socket and use it instead of bare SOCKET (Vasil Dimov)
dec9b5e850 net: move CloseSocket() from netbase to util/sock (Vasil Dimov)
aa17a44551 net: move MillisToTimeval() from netbase to util/time (Vasil Dimov)

Pull request description:

  Introduce a class to manage the lifetime of a socket - when the object
  that contains the socket goes out of scope, the underlying socket will
  be closed.

  In addition, the new `Sock` class has a `Send()`, `Recv()` and `Wait()`
  methods that can be overridden by unit tests to mock the socket
  operations.

  The `Wait()` method also hides the
  `#ifdef USE_POLL poll() #else select() #endif` technique from higher
  level code.

ACKs for top commit:
  laanwj:
    Re-ACK 615ba0eb96
  jonatack:
    re-ACK 615ba0eb96

Tree-SHA512: 3003e6bc0259295ca0265ccdeb1522ee25b4abe66d32e6ceaa51b55e0a999df7ddee765f86ce558a788c1953ee2009bfa149b09d494593f7d799c0d7d930bee8
2021-02-11 14:07:33 +01:00
MarcoFalke
fabce459bb fuzz: version handshake 2021-02-11 12:45:57 +01:00
MarcoFalke
685c16fcb2 Merge #21043: net: Avoid UBSan warning in ProcessMessage(...)
3ddbf22ed1 util: Disallow negative mocktime (MarcoFalke)
f5f2f97168 net: Avoid UBSan warning in ProcessMessage(...) (practicalswift)

Pull request description:

  Avoid UBSan warning in `ProcessMessage(...)`.

  Context: https://github.com/bitcoin/bitcoin/pull/20380#issuecomment-770427182 (thanks Crypt-iQ!)

ACKs for top commit:
  MarcoFalke:
    re-ACK 3ddbf22ed1 only change is adding patch written by me
  ajtowns:
    ACK 3ddbf22ed1 -- code review only

Tree-SHA512: e8d7af0457ca86872b75a4e406c0a93aafd841c2962e244e147e748cc7ca118c56be0fdafe53765f4b291410030b2c3cc8f76f733b37a955d34fc885ab6037b9
2021-02-11 12:40:12 +01:00
MarcoFalke
e498aeffbe Merge #20211: Use -Wswitch for TxoutType where possible
fa650ca7f1 Use -Wswitch for TxoutType where possible (MarcoFalke)
fa59e0b5bd test: Add missing script_standard_Solver_success cases (MarcoFalke)

Pull request description:

  This removes unused `default:` cases for all `switch` statements on `TxoutType` and adds the cases (`MULTISIG`, `NULL_DATA`, `NONSTANDARD`) to `ExtractDestination` for clarity.

  Also, the compiler is now able to use `-Wswitch`.

ACKs for top commit:
  practicalswift:
    cr ACK fa650ca7f1: patch looks correct and `assert(false);` is better than UB :)
  hebasto:
    ACK fa650ca7f1, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 282458b6523bd8923a0c0f5c423d1db2dce2a2d1b1d1dae455415c6fc995bb41ce82c1f9b0a1c0dcc6d874d171e04c30eca585f147582f52c7048c140358630a
2021-02-11 11:48:12 +01:00
Jonas Schnelli
00d57ff768 Avoid accessing nullpointer in BaseIndex::GetSummary() 2021-02-11 11:39:45 +01:00
Jonas Schnelli
6abe9f5b11 Allow blockfilter in conjunction with prune 2021-02-11 11:39:45 +01:00
MarcoFalke
a59e7ed0bc Merge #20915: fuzz: Fail if message type is not fuzzed
fa4bc897fc fuzz: Fail if message type is not fuzzed (MarcoFalke)
faefed8cd5 fuzz: Count message type fuzzers before main() (MarcoFalke)

Pull request description:

  `process_message_*` is a nice way to quickly fuzz a single message type. However, the offered message types are outdated and all BIPs implemented in the last years are missing.

  Fix that by adding them and failing when the number of message types don't add up.

ACKs for top commit:
  practicalswift:
    cr ACK fa4bc897fc: patch looks correct and touches only `src/test/fuzz/`

Tree-SHA512: 8c98374b50fb4ab2ff2550daeab4c6e9f486bfe847466d217d4bc97d119adc99a82b87b56f47535b1cf8f844232bc7fa1230712a9147cda514ae78851556f988
2021-02-11 10:50:15 +01:00
Vasil Dimov
615ba0eb96 test: add Sock unit tests 2021-02-11 10:44:53 +01:00
MarcoFalke
dd0521b640 Merge #21023: fuzz: Disable shuffle when merge=1
fabeb5b9c7 fuzz: Disable shuffle when merge=1 (MarcoFalke)

Pull request description:

  This should hopefully help make the deletion of fuzz inputs more deterministic.

  My tests (N=1) revealed that without this patch 7000 files differ (https://github.com/bitcoin-core/qa-assets/pull/44#issuecomment-768841467). With this patch, "only" 2000 files differ.

ACKs for top commit:
  practicalswift:
    cr ACK fabeb5b9c7: `-shuffle=0` and `-prefer_small=1` make sense

Tree-SHA512: 21a701f52450d402a91dd6e0b33d564c63a9c3b919738eb9a80c24d48fc5b964088e325470738f39af0d595612c844acc7bf0941590cc2dc8c6f6ee4cb69c861
2021-02-11 10:34:45 +01:00
MarcoFalke
fa4bc897fc fuzz: Fail if message type is not fuzzed 2021-02-11 09:40:41 +01:00
MarcoFalke
faefed8cd5 fuzz: Count message type fuzzers before main() 2021-02-11 09:39:32 +01:00
MarcoFalke
b69eab9025 Merge #20663: fuzz: Hide script_assets_test_minimizer
fac726b1b8 doc: Fixup docs in fuzz/script_assets_test_minimizer.cpp (MarcoFalke)
fafca47adc fuzz: Hide script_assets_test_minimizer (MarcoFalke)

Pull request description:

  This is not an actual fuzz target. It is a hack to exploit the built-in capability of fuzz engines to measure coverage.

ACKs for top commit:
  practicalswift:
    cr ACK fac726b1b8: patch looks correct and touches only `src/test/fuzz/`

Tree-SHA512: 0652dd8d9e95746b0906be4044467435d8204a34a30366ae9bdb75b9cb0788d429db7cedf2760fd543565d9d4f7ee206873ed10a29dd715a792a26337f65b53c
2021-02-11 07:54:22 +01:00
parazyd
95f97111dd contrib/init: (OpenRC) quote some unquoted variables. 2021-02-11 00:09:42 +01:00
parazyd
737feadff7 contrib/init: (OpenRC) Do not fail if both rpcuser and rpcpassword are unset.
This implies the .cookie file shall be used for RPC and having _both_ unset
in bitcoin.conf should allow bitcoind to start.
2021-02-11 00:09:42 +01:00
practicalswift
25f899cc23 log: Move "Pre-allocating up to position 0x[...] in [...].dat" log message to debug category 2021-02-10 20:46:25 +00:00
practicalswift
acd7980b37 log: Move "Leaving block file [...]: [...]" log message to debug category 2021-02-10 20:40:42 +00:00
Wladimir J. van der Laan
deb185d2fd Merge #21130: script: Make LXC container size suitable for gitian builds
166266a372 script: Make LXC container size suitable for gitian builds (Hennadii Stepanov)

Pull request description:

  This change prevents "No space left on device" error.

  See:
  - https://github.com/bitcoin/bitcoin/pull/21036#issuecomment-774771873
  - https://github.com/bitcoin/bitcoin/pull/21036#issuecomment-775031315

ACKs for top commit:
  jonasschnelli:
    ACK 166266a372 - I had to add this manually to my nighly build base image

Tree-SHA512: 47d84c3a65f0a17013b2cb970c34bfa4e600e83066be302ff10280aefefa0a7c6cb6c21a191b3e8e6fcd1c292d1c434cc4769e04626c4536050aced29b34d573
2021-02-10 19:37:06 +01:00
Wladimir J. van der Laan
3c514cfe62 Merge #21114: Deduplicate some block-to-JSON code
fa2c521115 Deduplicate some block-to-JSON code. (Daniel Kraft)

Pull request description:

  Some of the logic converting blocks and block headers to JSON for the blockchain RPC methods (`getblock`, `getlockheader`) was duplicated. Instead of that, the `blockToJSON` RPC method now calls `blockheaderToJSON` first, and then fills in the missing, block-specific bits explicitly.

ACKs for top commit:
  laanwj:
    Code review ACK fa2c521115

Tree-SHA512: 1b9b269e251d9c8c1056f253cfc2a795170d609f4c26ecc85b1ff9cdd567095a673dd89564e0d587b32dfc152ea01b2682169b018f2c9c3004c511a9998d772e
2021-02-10 14:49:44 +01:00
Wladimir J. van der Laan
b3dda0af0e Merge #21123: code style: Add EditorConfig file
7a135d57b2 Add EditorConfig file. (Kiminuo)

Pull request description:

  ### Motivation

  Developers are supposed to follow [Coding style](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#coding-style-general). However, [from time to time](https://github.com/bitcoin/bitcoin/pull/21075#discussion_r570125634) a PR is created and then its author is asked to change tabs to spaces, for example.

  Introducing an `.editorconfig` file can mitigate these formatting issues.

  ### User story

  A contributor wants to create a new PR. She clones Bitcoin Core repo, opens her editor, the editor loads `.editorconfig` rules and writes her patch with correct formatting rules. Less Coding Style issues is then discovered in the PR review process and thus less CI runs are needed.

  ### What is EditorConfig file?

  https://editorconfig.org provides very well and concise explanation:

  > What is EditorConfig?

  > EditorConfig helps maintain consistent coding styles for multiple developers working on the same project across various editors and IDEs. The EditorConfig project consists of a file format for defining coding styles and a collection of text editor plugins that enable editors to read the file format and adhere to defined styles. EditorConfig files are easily readable and they work nicely with version control systems.

  ### Support

  `.editorconfig` is supported by many IDEs and text editors. Sometimes, the support is out of the box and sometimes a plugin is needed. However, for example, VS Code detects `.editorconfig` presence and automatically offers you to install the missing plugin.

  See https://editorconfig.org/#pre-installed for details on support. To name a few:

  * Visual Studio (out of the box)
  * VS Code (plugin)
  * JetBrains IDEs (IntelliJ IDEA, PyCharm, etc.) (out of the box)
  * Sublime Text (plugin)
  * Emacs (plugin)
  * Vim (plugin)

  Not supported (AFAIK):

  * [mcedit](https://github.com/MidnightCommander/mc)

  ### My editor does not support `.editorconfig`

  Then nothing really changes for you.

  ### `.editorconfig` vs `.clang-format`

  As explained [here](https://devblogs.microsoft.com/cppblog/clangformat-support-in-visual-studio-2017-15-7-preview-1/):

  > Note that Visual Studio also supports EditorConfig, which works in a similar way. ClangFormat, however, has a [much larger variety of style options](https://clang.llvm.org/docs/ClangFormatStyleOptions.html) than EditorConfig, including some very C++ specific rules that can be set, and it is already used by C++ developers today.

  Having both `.editorconfig` and `.clang-format` in a project, may not always work correctly though, I think. As some editors may have a plugin for `.editorconfig` and a plugin for `clang-formatter` which may not work correctly in unison. In VS Code & Visual Studio EditorConfig [takes precedence over other settings](https://github.com/MicrosoftDocs/visualstudio-docs/blob/master/docs/ide/cpp-editorconfig-properties.md#c-editorconfig-formatting-conventions).

  ### Possible issues

  Your editor may change formatting for some 3rd party library if you edit the code. A solution for this would be to make EditorConfig rules more specific (include only certain paths). I'm not sure if it is an issue in practice.

  ### Testing

  Add some trailing whitespace to a Python file and save the file. You should see that the trailing whitespace is removed.

  ### Possible future work

  It would be great to define rules for Makefiles. This would be good start:

  ```
  # Makefiles
  [Makefile,*.am]
  indent_style = tab
  trim_trailing_whitespace = true
  ```

  I don't know makefiles in this project good enough to propose something reasonable. If this PR is well received, it would be great to add it in this PR.

  Also, there are actually many different file extensions and so the proposed `.editorconfig` file can be probably improved very much:

  ```powershell
  Get-ChildItem -Recurse | % {$_.Extension.ToLower()} | sort | unique
  ```

  <details><summary>Click to see the output</summary>

  ```
  .1
  .ac
  .adoc
  .am
  .bash-completion
  .bat
  .bmp
  .c
  .cc
  .cert
  .cfg
  .clang_complete
  .clang-format
  .cmake
  .cmd
  .cnf
  .com
  .conf
  .cpp
  .css
  .csv
  .doxyfile
  .dtd
  .empty
  .exe
  .exp
  .gci
  .gitattributes
  .github
  .gitignore
  .gitmodules
  .guess
  .h
  .hex
  .hpp
  .html
  .icns
  .ico
  .idb
  .ilk
  .in
  .include
  .ini
  .init
  .ipp
  .jam
  .js
  .json
  .lastbuildstate
  .lib
  .list
  .log
  .m
  .m4
  .md
  .mk
  .mm
  .moc
  .obj
  .openrc
  .openrcconf
  .patch
  .pc
  .pdb
  .pl
  .plist
  .png
  .po
  .pro
  .py
  .python-version
  .qbk
  .qm
  .qml
  .qrc
  .raw
  .rb
  .rc
  .recipe
  .res
  .s
  .sage
  .sass
  .scm
  .scss
  .service
  .sgml
  .sh
  .sln
  .spec
  .sub
  .supp
  .svg
  .targets
  .td
  .tlog
  .ts
  .tx
  .txt
  .ui
  .user
  .v2
  .vcxproj
  .verbatim
  .vscode
  .xml
  .xpm
  .xsl
  .y
  .yapf
  .yml
  .yy
  ```

  </details>

  Fixes #21092

ACKs for top commit:
  laanwj:
    Tested re-ACK 7a135d57b2
  MarcoFalke:
    Approach ACK 7a135d57b2

Tree-SHA512: c36a3424ecc751fbdd66101463b0c470f5c7adcdb4795b1cd267ff718eb345a04615fc1182338adf5b7db724469dca00c64815a9ef77064734a6536fba41a2ba
2021-02-10 14:30:58 +01:00
Vasil Dimov
7bd21ce1ef style: rename hSocket to sock
In the arguments of `InterruptibleRecv()`, `Socks5()` and
`ConnectThroughProxy()` the variable `hSocket` was previously of type
`SOCKET`, but has been changed to `Sock`. Thus rename it to `sock` to
imply its type, to distinguish from other `SOCKET` variables and to
abide to the coding style wrt variables' names.
2021-02-10 13:30:09 +01:00
Vasil Dimov
04ae846904 net: use Sock in InterruptibleRecv() and Socks5()
Use the `Sock` class instead of `SOCKET` for `InterruptibleRecv()` and
`Socks5()`.

This way the `Socks5()` function can be tested by giving it a mocked
instance of a socket.

Co-authored-by: practicalswift <practicalswift@users.noreply.github.com>
2021-02-10 13:30:08 +01:00
Vasil Dimov
ba9d73268f net: add RAII socket and use it instead of bare SOCKET
Introduce a class to manage the lifetime of a socket - when the object
that contains the socket goes out of scope, the underlying socket will
be closed.

In addition, the new `Sock` class has a `Send()`, `Recv()` and `Wait()`
methods that can be overridden by unit tests to mock the socket
operations.

The `Wait()` method also hides the
`#ifdef USE_POLL poll() #else select() #endif` technique from higher
level code.
2021-02-10 13:30:08 +01:00
Vasil Dimov
dec9b5e850 net: move CloseSocket() from netbase to util/sock
Move `CloseSocket()` (and `NetworkErrorString()` which it uses) from
`netbase.{h,cpp}` to newly added `src/util/sock.{h,cpp}`.

This is necessary in order to use `CloseSocket()` from a newly
introduced Sock class (which will live in `src/util/sock.{h,cpp}`).
`sock.{h,cpp}` cannot depend on netbase because netbase will depend
on it.
2021-02-10 13:30:08 +01:00
fanquake
cda914228a Merge #21138: ci: Re-run wine tests once if they fail
faa8afe70b ci: Re-run wine tests once if they fail (MarcoFalke)

Pull request description:

  Works around the intermittent wine issue: https://github.com/bitcoin/bitcoin/issues/21122#issuecomment-776517563

ACKs for top commit:
  fanquake:
    ACK faa8afe70b - thanks for following up with this.

Tree-SHA512: cb377a8e62c7fcf38fb5bdd6203af82c6c302c86f7fc219cb756eb7b6d3d4fe4ebf30a36ebba739199ff7814b26256a3701cf38dc5e681b18c2f249cdf6d852e
2021-02-10 19:51:01 +08:00
Vasil Dimov
aa17a44551 net: move MillisToTimeval() from netbase to util/time
Move `MillisToTimeval()` from `netbase.{h,cpp}` to
`src/util/system.{h,cpp}`.

This is necessary in order to use `MillisToTimeval()` from a newly
introduced `src/util/sock.{h,cpp}` which cannot depend on netbase
because netbase will depend on it.
2021-02-10 11:00:12 +01:00
MarcoFalke
f61c3a1090 Merge #21125: test: Change BOOST_CHECK to BOOST_CHECK_EQUAL for paths
059e8ccc1e Change BOOST_CHECK to BOOST_CHECK_EQUAL to see mismatched values when a check fails. (Kiminuo)

Pull request description:

  This is useful to see mismatched values when a check fails as specified in the [Boost documentation](https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_reference/testing_tool_ref/assertion_boost_level.html).

  This PR would make #20744 PR's diff smaller by a bit.

ACKs for top commit:
  MarcoFalke:
    review ACK 059e8ccc1e
  theStack:
    Code Review ACK 059e8ccc1e

Tree-SHA512: 82359ef38e0d1926f12a34aeff6fde6d1d307c703a080547749b908f873c2a2f894f6f094c33470b32987c229e3a1f17f7d1e877663c53293c023bde0e7272c1
2021-02-10 10:07:07 +01:00
Daniel Kraft
fa2c521115 Deduplicate some block-to-JSON code.
Some of the logic converting blocks and block headers to JSON for
the blockchain RPC methods (getblock, getlockheader) was duplicated.
Instead of that, the blockToJSON RPC method now calls blockheaderToJSON
first, and then fills in the missing, block-specific bits explicitly.
2021-02-10 09:16:58 +01:00
MarcoFalke
faa8afe70b ci: Re-run wine tests once if they fail 2021-02-10 08:59:35 +01:00
Kiminuo
7a135d57b2 Add EditorConfig file. 2021-02-10 08:00:06 +01:00
fanquake
09530b0255 Merge #21126: ci: Properly bump to focal for win cross build
fa272dfdff ci: Properly bump to focal for win cross build (MarcoFalke)

Pull request description:

  Fixes #21122

ACKs for top commit:
  fanquake:
    ACK fa272dfdff - Looks good to me. I can see the wine version output, `wine-5.0 (Ubuntu 5.0-3ubuntu1)`, in the [log](https://cirrus-ci.com/task/5743559502462976), and `make check` is running.

Tree-SHA512: cd37462afc5512e00cef5e9e7fd1bb5c43c600e833b30cdea2c1c443dc7b0e68f5f2cbaf9d7b655892059af0a226478211db1dc97c10fe4dcdfe666a784f3afc
2021-02-10 09:40:44 +08:00
Sebastian Falbesoner
787df19b09 validation: don't try to invalidate genesis block 2021-02-10 00:14:18 +01:00
Sebastian Falbesoner
ef21fb7313 zmq test: speedup test by whitelisting peers (immediate tx relay)
Speeds up the zmq test roughly by a factor of 2x (~20 sec. instead of
~40 sec.) and also avoids timeouts on the synchronization methods
(sync_mempool() / sync_blocks()) that happened with a slight chance.
This is due to the fact that there is no upper bound on the trickle
relay time, so even the default of 60s is sometimes too low. Fixed by
enabling immediate tx relay on node1.
2021-02-09 23:55:32 +01:00
Sebastian Falbesoner
5c6546362d zmq test: fix flakiness by using more robust sync method
After connecting the subscriber sockets to the node, there is no
guarantee that the node's zmq publisher interfaces are ready yet, which
means that potentially the first expected notification messages could
get lost and the test fails. Currently this is handled by just waiting
for a short period of time (200ms), which works most of the time but is
still problematic, as in some rare cases the setup time takes much
longer, even in the range of multiple seconds.

The solution in this commit approaches the problem by using a more
robust method of syncing up, originally proposed by instagibbs:
    1. Generate a block on the node
    2. Try to receive a notification on all subscribers
    3. If all subscribers get a message within the timeout (1 second),
       we are done, otherwise repeat starting from step 1
2021-02-09 23:55:23 +01:00
Sebastian Falbesoner
8666033630 zmq test: accept arbitrary sequence start number in ZMQSubscriber
The ZMQSubscriber reception methods currently assert that the first
received publisher message has a sequence number of zero. In order to
fix the current test flakiness via "syncing up" to nodes in the setup
phase, we have to cope with the situation that messages get lost and the
first actual received message has a sequence number larger than zero.
2021-02-09 22:54:01 +01:00
Sebastian Falbesoner
6014d6e1b5 zmq test: dedup message reception handling in ZMQSubscriber 2021-02-09 22:54:01 +01:00
Wladimir J. van der Laan
d202054675 Merge #21052: refactor: Replace fs::unique_path with GetUniquePath(path) calls
1bca2aa694 Introduce GetUniquePath(base) helper method to replace boost::filesystem::unique_path() which is not available in std::filesystem. (Kiminuo)

Pull request description:

  This PR makes it easier in #20744 to remove our dependency on the `boost::filesystem::unique_path()` function which does not have a direct equivalent in C++17.

  This PR attempts to re-implement `boost::filesystem::unique_path()` as `GetUniquePath(path)` but the implementations are not meant to be the same.

  Note:

  * Boost 1.75.0 implementation of `unique_path`: 9cab675b71/src/unique_path.cpp (L235)

  * In the previous implementation, I attempted to add:
      ```cpp
      fs::path GetUniquePath(const fs::path& base)
      {
          FastRandomContext rnd;
          fs::path tmpFile = base / HexStr(rnd.randbytes(8));
          return tmpFile;
      }
      ```

      to `fs.cpp` but this leads to a circular dependency: "fs -> random -> logging -> fs". That is why the modified implementation adds a new file.

ACKs for top commit:
  laanwj:
    Code review ACK 1bca2aa694
  ryanofsky:
    Code review ACK 1bca2aa694. It's a simple change and extra test coverage is nice

Tree-SHA512: f324bdf0e254160c616b5033c3ece33d87db23eb0135acee99346ade7b5cf0d30f3ceefe359a25a8e9b53ba8e4419f459c2bdd369e10fc0152ce95031d1f221c
2021-02-09 22:22:13 +01:00
MarcoFalke
fa272dfdff ci: Properly bump to focal for win cross build 2021-02-09 21:37:14 +01:00
MarcoFalke
d48f9e8ebb Merge #21124: test: remove unnecessary assignment in bdb
c9095b738f test: remove unnecessary assignment in bdb (Bruno Garcia)

Pull request description:

  This PR removes the unnecessary assignment to page_info['entries'] on line 54 since there is another assignment for it in line 59.

  I think a lint (#21096) would detect cases like this one.

ACKs for top commit:
  achow101:
    ACK c9095b738f
  theStack:
    Code Review ACK c9095b738f

Tree-SHA512: 23377077c015b04361fd416b41bf6806ad0bdd4d264be6760f0fd3bc88d694d2cd52cae250519925c5d3b3c70715772714c3863f8fa181a2eb4883204ccdbf9d
2021-02-09 19:21:34 +01:00
Sjors Provoost
9305862f71 wallet: load flags before everything else 2021-02-09 19:01:15 +01:00
Sebastian Falbesoner
f64adc1eed test: remove unused function xor_bytes 2021-02-09 17:58:21 +01:00
Hennadii Stepanov
166266a372 script: Make LXC container size suitable for gitian builds
This change prevents "No space left on device" error.
2021-02-09 18:56:45 +02:00
gzhao408
53e716ea11 [refactor] improve style for touched code 2021-02-09 07:01:52 -08:00
gzhao408
174cb5330a [refactor] const ATMPArgs and non-const Workspace
ATMPArgs should contain const arguments for validation.
The Workspace should contain state that may change
throughout validation.
2021-02-09 07:01:52 -08:00
gzhao408
f82baf0762 [refactor] return MempoolAcceptResult
This creates a cleaner interface with ATMP, allows us to make results const,
and makes accessing values that don't make sense (e.g. fee when tx is
invalid) an error.
2021-02-09 07:01:52 -08:00
Kiminuo
059e8ccc1e Change BOOST_CHECK to BOOST_CHECK_EQUAL to see mismatched values when a check fails.
See https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_reference/testing_tool_ref/assertion_boost_level_eq.html
2021-02-09 15:00:02 +01:00
Bruno Garcia
c9095b738f test: remove unnecessary assignment in bdb 2021-02-09 10:22:50 -03:00
Wladimir J. van der Laan
b847f49717 Merge #20557: addrman: Fix new table bucketing during unserialization
4676a4fb5b [addrman] Don't repeat "Bucketing method was updated" log multiple times (John Newbery)
436292367c [addrman] Improve serialization comments (John Newbery)
ac3547eddd [addrman] Improve variable naming/code style of touched code. (John Newbery)
a5c9b04959 [addrman] Don't rebucket new table entries unnecessarily (John Newbery)
8062d928ce [addrman] Rename asmap version to asmap checksum (John Newbery)
009b8e0fdf [addrman] Improve variable naming/code style of touched code. (John Newbery)
b4c5fda417 [addrman] Fix new table bucketing during unserialization (John Newbery)

Pull request description:

  This fixes three issues in addrman unserialization.

  1. An addrman entry can appear in up to 8 new table buckets. We store this entry->bucket indexing during shutdown so that on restart we can restore the entries to their correct buckets. Commit ec45646de9 broke the deserialization code so that each entry could only be put in up to one new bucket.

  2. Unserialization may result in an entry appearing in a 9th bucket. If the entry already appears in 8 buckets don't try to place it in another bucket.

  3. We unnecessarily rebucket when reading a peers.dat with file version 1. Don't do that.

ACKs for top commit:
  vasild:
    ACK 4676a4fb5b
  glozow:
    re-ACK 4676a4fb5b, changes were a rename, comments, and removing repeat-logging.
  naumenkogs:
    ACK 4676a4f
  laanwj:
    Code review ACK 4676a4fb5b
  dhruv:
    ACK 4676a4fb5b
  ryanofsky:
    Code review ACK 4676a4fb5b. I'm not previously familiar with this code but all the changes here do make sense and seem like improvements. Left some notes and comments, but they aren't important so feel to ignore.

Tree-SHA512: b228984f6dec5910be23c3740ae20258da33bcf66ceb7edb10e5a53163450f743bab349e47f09808b7e8d40f27143119ec3e0981d7e678aa494d8559a1c99c23
2021-02-09 12:36:32 +01:00
MarcoFalke
3641ec1aac Merge #21112: ci: use Focal for macOS cross builds
ac24af453d ci: use Ubuntu Focal for macOS cross build (fanquake)

Pull request description:

  I had assumed Cirrus was spinning up Docker containers to run the CI,
  however we are actaully running on the Cirrus machines themselves. See
  `DANGER_RUN_CI_ON_HOST` and in the logs:
  ```bash
  Running on host system without docker wrapper
  ```

  So with this change we will actually be using Focal for the macOS cross build.

  Follow up to #21036.

  This originally contained Windows changes, and an attempt to get Cirrus running without `DANGER_RUN_CI_ON_HOST`, however that seems non-trival, so Windows changes have been dropped from here for now.

ACKs for top commit:
  MarcoFalke:
    cr ACK ac24af453d

Tree-SHA512: 587ba5acf741bcefecf1bc262fa1177f565ebfa9de56125eca19ed3c7db7b9aabfb96866e9c140681b88cb7015a3ded2bc6b4b1b235543d6f6e9dfc6984d569f
2021-02-09 08:00:33 +01:00
MarcoFalke
19b1ceddc7 Merge #21075: doc: Fix markdown formatting
e1604b3d50 doc: Replace tabs for spaces (Gunar C. Gessner)
98db48d349 doc: Fix markdown formatting (Gunar Gessner)

Pull request description:

  Lines were being joined making it hard to read.

ACKs for top commit:
  RandyMcMillan:
    ACK e1604b3d50

Tree-SHA512: fd5a7c5e9a1cbbf0fbb13b5c30b87853c84751da7f0fad08151bda07f1933872ab51cad29a0c0a70ced48e60df6d83bff3f84c2f77d00d22723fae9a8c3534fc
2021-02-09 07:56:44 +01:00
MarcoFalke
c4214d0e0d Merge #21117: test: remove assert_blockchain_height
fa0a4d6c60 test: remove assert_blockchain_height (MarcoFalke)

Pull request description:

  This simplifies the code and solves intermittent timeouts caused by commit 0d39b5848a.

  E.g. https://cirrus-ci.com/task/5196092369272832?command=ci#L3126

  ```
   test  2021-02-08T12:27:56.275000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 127, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 180, in run_test
                                         self.assert_blockchain_height(self.nodes[0], 101)
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 92, in assert_blockchain_height
                                         assert False, "blockchain too short after timeout: %d" % current_height
                                     AssertionError: blockchain too short after timeout: 101

ACKs for top commit:
  glozow:
    ACK fa0a4d6c60

Tree-SHA512: 3859b0c1581c21f03c775f119204cc3a219e5d86346883634886f6da46feaf254b9c6c49c1ec4581ffe409cdf05f6e91ec38c3976c3daf4a9e67f96ddc1e0dce
2021-02-09 07:43:47 +01:00
fanquake
a0077a80b0 Merge #21116: build: Disable --disable-fuzz-binary for gitian/guix builds
cb151b797a build: Disable --disable-fuzz-binary for guix builds (Hennadii Stepanov)
fd7caae35f build: Disable --disable-fuzz-binary for gitian builds (Hennadii Stepanov)

Pull request description:

  Fuzz binary is not shipped to users.
  This PR saves hundreds MB of the disk space for containers.

ACKs for top commit:
  MarcoFalke:
    review ACK cb151b797a
  fanquake:
    ACK cb151b797a

Tree-SHA512: 858e3816576c307b47915bb05de79a28029beaef8835c01f1bd6a764a0cf7f7f63ef8c2dc2c5944cb36cc9f4788d9b0590b8a5dda96940167252ba371cdbd078
2021-02-09 14:22:11 +08:00
fanquake
ac24af453d ci: use Ubuntu Focal for macOS cross build 2021-02-09 13:58:59 +08:00
fanquake
d864696649 Merge #21115: test: Fix Windows cross build
723eb4326b test: Fix Windows cross build (Hennadii Stepanov)

Pull request description:

  On master (e51f6c4dee, after #20936 merge), Windows cross compiling fails:
  ```
  $ make > /dev/null
  In file included from ./policy/fees.h:12,
                   from policy/fees.cpp:6:
  policy/fees.cpp: In member function ‘unsigned int CBlockPolicyEstimator::HighestTargetTracked(FeeEstimateHorizon) const’:
  ./sync.h:232:104: warning: control reaches end of non-void function [-Wreturn-type]
    232 | #define LOCK(cs) DebugLock<decltype(cs)> PASTE2(criticalblock, __COUNTER__)(cs, #cs, __FILE__, __LINE__)
        |                                                                                                        ^
  policy/fees.cpp:680:5: note: in expansion of macro ‘LOCK’
    680 |     LOCK(m_cs_fee_estimator);
        |     ^~~~
  test/fuzz/netaddress.cpp:12:10: fatal error: netinet/in.h: No such file or directory
     12 | #include <netinet/in.h>
        |          ^~~~~~~~~~~~~~
  compilation terminated.
  make[2]: *** [Makefile:13039: test/fuzz/fuzz-netaddress.o] Error 1
  make[2]: *** Waiting for unfinished jobs....
  libtool: warning: undefined symbols not allowed in x86_64-w64-mingw32 shared libraries; building static only
  test/fuzz/string.cpp: In function ‘void string_fuzz_target(FuzzBufferType)’:
  test/fuzz/string.cpp:81:11: error: ‘ShellEscape’ was not declared in this scope
     81 |     (void)ShellEscape(random_string_1);
        |           ^~~~~~~~~~~
  make[2]: *** [Makefile:13543: test/fuzz/fuzz-string.o] Error 1
  make[1]: *** [Makefile:15078: all-recursive] Error 1
  make: *** [Makefile:812: all-recursive] Error 1
  ```

  This PR fixes both of errors.

ACKs for top commit:
  MarcoFalke:
    cr ACK 723eb4326b

Tree-SHA512: 5d2fba5ca806e64bf92011786d1f868c6624f786bfa753a10316feab7a802a28ec27a4bd25fc26dc289a399895a521c3878ffa1efeff0e540c7245cdb8e4942c
2021-02-09 13:48:26 +08:00
Anthony Towns
fd6580e405 [refactor] txmempool: split epoch logic into class 2021-02-09 15:10:46 +10:00
MarcoFalke
b09ad737ee Merge #20944: rpc: Return total fee in getmempoolinfo
fa362064e3 rpc: Return total fee in mempool (MarcoFalke)

Pull request description:

  This avoids having to loop over the whole mempool to query each entry's fee

ACKs for top commit:
  achow101:
    ACK fa362064e3
  glozow:
    ACK fa362064e3 🧸
  jnewbery:
    ACK fa362064e3

Tree-SHA512: e2fa1664df39c9e187f9229fc35764ccf436f6f75889c5a206d34fff473fc21efbf2bb143f4ca7895c27659218c22884d0ec4195e7a536a5a96973fc9dd82d08
2021-02-08 20:36:46 +01:00
MarcoFalke
fa0a4d6c60 test: remove assert_blockchain_height 2021-02-08 16:56:46 +01:00
Hennadii Stepanov
cb151b797a build: Disable --disable-fuzz-binary for guix builds 2021-02-08 16:41:05 +02:00
Hennadii Stepanov
fd7caae35f build: Disable --disable-fuzz-binary for gitian builds 2021-02-08 15:38:26 +02:00
Hennadii Stepanov
723eb4326b test: Fix Windows cross build 2021-02-08 15:17:48 +02:00
MarcoFalke
b401b09355 Merge #21107: test: remove type: comments in favour of actual annotations
9913419cc9 test: remove type: comments in favour of actual annotations (fanquake)

Pull request description:

  Now that we require Python 3.6+, we should be using variable type
  annotations directly rather than `# type:` comments.

  Also takes care of the discarded value issue in p2p_message_capture.py.
  See: https://github.com/bitcoin/bitcoin/pull/19509/files#r571674446.

ACKs for top commit:
  MarcoFalke:
    review ACK 9913419cc9
  jnewbery:
    Code review ACK 9913419cc9

Tree-SHA512: 63aba5eef6c1320578f66cf8a6d85ac9dbab9d30b0d21e6e966be8216e63606de12321320c2958c67933bf68d10f2e76e9c43928e5989614cea34dde4187aad8
2021-02-08 10:56:54 +01:00
MarcoFalke
ca85449f22 Merge #21036: gitian: Bump descriptors to Focal for 22.0
2ecaf21433 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b318 build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, #19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by #20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](16b784d953/contrib/devtools/test-security-check.py (L53)) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21433

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21433

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
2021-02-08 10:44:37 +01:00
MarcoFalke
c969ab43c3 Merge #21084: test: fix timeout decrease in feature_assumevalid
0d39b5848a test: fix timeout decrease in feature_assumevalid (Bruno Garcia)

Pull request description:

  This PR fixes the timeout decrease in assert_blockchain_height function.

ACKs for top commit:
  practicalswift:
    cr ACK 0d39b5848a: patch looks correct
  theStack:
    ACK 0d39b5848a ⏲️

Tree-SHA512: ae3c83420b4de4ad41f1b20b6e77c3a26a8c5ac4fb136b2645fde119545a413c61312f76a16473141774bc955d30ac4fc86e5ca6cf729f8978a17d0dab520feb
2021-02-08 10:38:52 +01:00
MarcoFalke
fac726b1b8 doc: Fixup docs in fuzz/script_assets_test_minimizer.cpp 2021-02-08 10:12:22 +01:00
MarcoFalke
fafca47adc fuzz: Hide script_assets_test_minimizer
Can be reviewed with --ignore-all-space
2021-02-08 10:11:59 +01:00
MarcoFalke
e51f6c4dee Merge #20936: build: build fuzz tests by default
32cbb06676 build: build fuzz tests by default. (Dan Benjamin)

Pull request description:

  This fixes issue #19388. The changes are as follows:
    - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing
    - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no
    - Add the following libraries to FUZZ_SUITE_LD_COMMON:
      - LIBBITCOIN_WALLET
      - SQLLITE_LIBS
      - BDB_LIBS
      - if necessary, some or all of:
        - NATPMP_LIBS
        - MINIUPNPC_LIBS
        - LIBBITCOIN_ZMQ / ZMQ_LIBS

  Fixes  #19388

ACKs for top commit:
  MarcoFalke:
    review ACK 32cbb06676 📭

Tree-SHA512: c91d713ffe54a3d055daaec02c4317d7e13eed6688821ddc10d894224950b18e276fbdd4acc758c7103b50f34a132b1882b68bc8b60409f97438e0759ced77e1
2021-02-08 10:07:15 +01:00
fanquake
8636288db1 Merge #20720: build: more robustly check for fcf-protection support
e9189a750b build: more robustly check for fcf-protection support (fanquake)

Pull request description:

  When using Clang 7, we may end up trying to use the flag when it won't
  work properly, which can lead to confusing errors. i.e:
  ```bash
  /usr/bin/ld: error: ... <corrupt x86 feature size: 0x8>
  ```

  Use `AX_CHECK_LINK_FLAG` & `--fatal-warnings` to ensure we wont use the flag in this case.

  We do this as even when the error is emitted, compilation succeeds, and the binaries produced will run. This means we can't just check if the compiler accepts the flag, or if compilation succeeds (without or without `-Werror`, and/or passing `-Wl,--fatal-warnings`, which may not be passed through to the linker).

  This was reported by someone configuring for fuzzing, on Debian 10, where Clang 7 is the default.

  See here for a minimal example of the problematic behaviour:
  https://gist.github.com/fanquake/9b33555fcfebef8eb8c0795a71732bc6

ACKs for top commit:
  pstratem:
    tested ACK e9189a750b
  MarcoFalke:
    not an ACK e9189a750b , I only tested configure on my system (gcc-10, clang-11):
  hebasto:
    ACK e9189a750b, tested with clang-7, clang-10 and gcc: the `-fcf-protection=full` is not applied for clang-7, but applied for others compilers.

Tree-SHA512: ec24b0cc5523b90139c96cbb33bb98d1e6a24d858c466aa7dfb3c474caf8c50aca53e570fdbc0ff88378406b0ac5d687542452637b1b5fa062e829291b886fc1
2021-02-08 13:45:23 +08:00
fanquake
9913419cc9 test: remove type: comments in favour of actual annotations
Now that we require Python 3.6+, we should be using variable type
annotations directly rather than # type: comments.

Also takes care of the discarded value issue in p2p_message_capture.py.
See: https://github.com/bitcoin/bitcoin/pull/19509/files#r571674446.
2021-02-08 13:24:44 +08:00
fanquake
cf26ca3911 Merge #21081: test: fix the unreachable code at feature_taproot
5e0cd25e29 fix the unreachable code at feature_taproot (Bruno Garcia)

Pull request description:

  This PR removes the unnecessary return statement at the beginning of the function that makes the rest of the function unreachable.

ACKs for top commit:
  practicalswift:
    cr ACK 5e0cd25e29: patch looks correct!
  sipa:
    ACK 5e0cd25e29.
  theStack:
    Tested ACK 5e0cd25e29 🏔️
  sanket1729:
    tACK 5e0cd25e29. I noted this a while ago while fixing feature_taproot.py for elements. Verified that the extreme ranges of CScriptNum are correct and the overflow case for `CHECKSIGADD` works as intended. Adding 1 to 2^31 - 1 results in an overflow, but the interpreter puts a `vch` of corresponding to 2^31 on stack. Even though it cannot be converted to CscriptNum(restricted to 4 bytes), it's result can still be compared by OP_EQUAL.

Tree-SHA512: fff9be3be94f4b3f3ccf24bf588d96e84d14806f82692dccd31631b0e5c79a7575a96c308cb5a4f610ab02e2f854b899f374437c33ecf6d52055d333f2de9b27
2021-02-08 10:24:37 +08:00
fanquake
1815847103 Merge #21105: docs: correctly identify script type
4ed064dbd9 docs: correctly identify script type (lisa neigut)

Pull request description:

  Fix a typo.

ACKs for top commit:
  sipa:
    ACK 4ed064dbd9
  darosior:
    ACK 4ed064dbd9
  theStack:
    ACK 4ed064dbd9

Tree-SHA512: 94572fde89865a085020767f9de58f41c6b1c8f714c0bc6c256a4fc419a2693ce8a33d953d4c75542495ae72882d10846354db751770e85d3d694d88e0378843
2021-02-08 08:54:05 +08:00
lisa neigut
4ed064dbd9 docs: correctly identify script type
fixes a typo
2021-02-07 12:20:01 -06:00
Jon Atack
be4cf4832f gui: update to "Direction/Type" peer details name/tooltip 2021-02-06 23:02:03 +01:00
Jon Atack
151888383a gui: add "Type" column to Peers main window 2021-02-06 22:59:54 +01:00
Jon Atack
6fc72bd6f0 gui: allow ConnectionTypeToQString to prepend direction optionally 2021-02-06 22:21:16 +01:00
Anthony Towns
b3c712cb28 contrib/signet/miner: remove debug code 2021-02-07 06:52:42 +10:00
Anthony Towns
297e35159f bitcoin-util: use AddCommand / GetCommand 2021-02-07 06:50:11 +10:00
Anthony Towns
b6d493fd4d contrib/signet/README.md: Update miner description 2021-02-07 06:50:11 +10:00
Anthony Towns
e66543827c contrib/signet/miner: Automatic timestamp for first block
When mining the first block of a new signet chain, pick a timestamp for
the first block so that after mining 100 blocks the timestamp will be
back to the current time -- this prevents an unnecessary delay before
any miner rewards have matured enough to be spent.  This takes into
account that the delta between blocks may be shorter than 10 minutes due
to attempting to increase the difficulty to match --nbits, but does not
take into account the time spent actually generating the 100 blocks.
2021-02-07 06:50:11 +10:00
Anthony Towns
a383ce5b4a contrib/signet/miner: --grind-cmd is required for calibrate
Thanks to muxator for spotting.
2021-02-07 06:50:11 +10:00
Anthony Towns
1a45cd2e51 contrib/signet: Fix typos
Thanks to muxator, JeremyRubin, and gruve-p for spotting.
2021-02-07 06:50:11 +10:00
Dan Benjamin
32cbb06676 build: build fuzz tests by default.
This fixes issue #19388. The changes are as follows:
  - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing
  - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no
  - Add the following libraries to FUZZ_SUITE_LD_COMMON:
    - LIBBITCOIN_WALLET
    - SQLLITE_LIBS
    - BDB_LIBS
    - if necessary, some or all of:
      - NATPMP_LIBS
      - MINIUPNPC_LIBS
      - LIBBITCOIN_ZMQ / ZMQ_LIBS
2021-02-05 19:52:45 -05:00
Jonas Schnelli
6c6140846f Merge bitcoin-core/gui#203: Display plain "Inbound" in peer details
506e6585a5 gui: display plain "Inbound" in peer details (Jon Atack)

Pull request description:

  Alternative version to #201.

ACKs for top commit:
  MarcoFalke:
    ACK 506e6585a5
  jonasschnelli:
    utACK 506e6585a5

Tree-SHA512: 88d141b14684c1dcdff47f7ba241e5a7c42c14da3d9aaa89f1649235a64fd26bc5a6055707dc07992cd9d8c05d143754f6dd51ccee69fd4309336dd07c52e61c
2021-02-05 18:48:39 +01:00
Wladimir J. van der Laan
173cf31299 Merge #20839: fuzz: Avoid extraneous copy of input data, using Span<>
faf7d7418c fuzz: Avoid extraneous copy of input data, using Span<> (MarcoFalke)

Pull request description:

  Seeing speedup here in the fuzz framework part (non-fuzz-target part). Speedup is only visible for input data larger than 100kB.

ACKs for top commit:
  practicalswift:
    cr ACK faf7d7418c: patch looks correct :)
  laanwj:
    Code review ACK faf7d7418c

Tree-SHA512: 41af7118846e0dfee237a6d5269a6c7cfbc775d7bd1cc2a85814cb60f6c2b37fe7fd35f1a788d4f08e6e0202c48b71054b67d2931160c445c79fc59e5347dadf
2021-02-05 14:57:08 +01:00
Wladimir J. van der Laan
b829894f84 Merge #20764: cli -netinfo peer connections dashboard updates 🎄
747cb5b994 netinfo: display only outbound block relay counts (Jon Atack)
76d198a5c1 netinfo: add i2p network (Jon Atack)
9d6aeca2c5 netinfo: add bip152 high-bandwidth to/from fields (Jon Atack)
5de7a6cf63 netinfo: display manual peers count (Jon Atack)
d3cca3be63 netinfo: update to use peer connection types (Jon Atack)
62bf5b7850 netinfo: add ConnectionTypeForNetinfo member helper function (Jon Atack)

Pull request description:

  Merry Bitcoin Christmas! Ho ho ho 🎄 

  This PR updates `-netinfo` to:
  - use the getpeerinfo `connection_type` field (and no longer use getpeerinfo `relaytxes` for block-relay detection)
  - display manual peers count, if any, in the outbound row
  - display the block relay counts in the outbound row only
  - display high-bandwidth BIP152 compact block relay peers (`hb` column, to `.` and from `*`)
  - add support for displaying I2P network peers, if any are present

  Testing and review welcome! How to test:

  - to run the full live dashboard (on Linux): `$ watch --interval 1 --no-title ./src/bitcoin-cli -netinfo 4`
  - to run the full dashboard: ``$ ./src/bitcoin-cli -netinfo 4``
  - to see the help: `$ ./src/bitcoin-cli -netinfo help`
  - to see the help summary: `$ ./src/bitcoin-cli -help | grep -A4 netinfo`

ACKs for top commit:
  laanwj:
    re-ACK 747cb5b994
  michaelfolkson:
    ACK 747cb5b994
  jonasschnelli:
    Tested ACK 747cb5b994 - works nicely. Great that this PR only changes bitcoin-cli.

Tree-SHA512: 48fe23dddf3005a039190fcbc84167cd25b0a63489617fe14ea5db9a641a829b46b6e8dc7924aab6577d82a13909d157e82f715bd2ed3a8a15071957c35c19f3
2021-02-05 14:43:10 +01:00
Wladimir J. van der Laan
a6b1bf6439 Merge #20267: Disable and fix tests for when BDB is not compiled
49797c3ccf tests: Disable bdb dump test when no bdb (Andrew Chow)
1194cf9269 Fix wallet_send.py wallet setup to work with descriptors (Andrew Chow)
fbaea7bfe4 Require legacy wallet for wallet_upgradewallet.py (Andrew Chow)
b1b679e0ab Explicitly mark legacy wallet tests as such (Andrew Chow)
09514e1bef Setup wallets for interface_zmq.py (Andrew Chow)
4d03ef9a73 Use MiniWallet in rpc_net.py (Andrew Chow)
4de23824b0 Setup wallets for interface_bitcoin_cli.py (Andrew Chow)
7c71c627d2 Setup wallets with descriptors for feature_notifications (Andrew Chow)
1f1bef8dba Have feature_filelock.py test both bdb and sqlite, depending on compiled (Andrew Chow)
c77975abc0 Disable upgrades tests that require BDB if BDB is not compiled (Andrew Chow)
1f20cac9d4 Disable wallet_descriptor.py bdb format check if BDB is not compiled (Andrew Chow)
3641597d7e tests: Don't make any wallets unless wallet is required (Andrew Chow)
b9b88f57a9 Skip legacy wallet reliant tests if BDB is not compiled (Andrew Chow)
6f36242389 tests: Set descriptors default based on compilation (Andrew Chow)

Pull request description:

  This PR fixes tests for when BDB is not compiled. Tests which rely on or test legacy wallet behavior are disabled and skipped when BDB is not compiled. For the components of some tests that are for legacy wallet things, those parts of the tests are skipped.

  For the majority of tests, changes are made so that they can be run with either legacy wallets or descriptor wallets without materially effecting the test. Most tests only need the wallet for balance and transactions, so the type of wallet is not an important part of those tests. Additionally, some tests are wallet agnostic and modified to instead use the test framework's MiniWallet.

ACKs for top commit:
  laanwj:
    ACK 49797c3ccf
  ryanofsky:
    Code review ACK 49797c3ccf. Only change since last review is dropping last commit. Previous review w/ suggestions for future followup is https://github.com/bitcoin/bitcoin/pull/20267#pullrequestreview-581508843

Tree-SHA512: 69659f8a81fb437ecbca962f4082c12835282dbf1fba7d9952f727a49e01981d749af9b09feda1c8ca737516c7d7a08ef17e782795df3fa69892d5021b41c1ed
2021-02-05 14:26:10 +01:00
Wladimir J. van der Laan
3931732191 Merge #20646: doc: refer to BIPs 339/155 in feature negotiation
e1e6714832 doc: refer to BIPs 339/155 in feature negotiation (Jon Atack)

Pull request description:

  of `wtxidrelay` and `addrv2`/`sendaddrv2`, and add `fSuccessfullyConnected` doxygen documentation to clarify that it is set to true on VERACK.

ACKs for top commit:
  laanwj:
    re-ACK e1e6714832

Tree-SHA512: 3e6af5b246e4ee1ec68ee34db525746717871bc986ad4840f5a8edce55740768389f6fd0ec69046eda2fb4c69939440a96571f79d36e6cbff4fd3b7f2ebc74c0
2021-02-05 11:15:37 +01:00
MarcoFalke
53730a78bc Merge #21077: doc: clarify -timeout and -peertimeout config options
eecb7ab105 [doc] clarify -peertimeout and -timeout descriptions (gzhao408)

Pull request description:

  The debug-only option `-peertimeout` is used to delay `InactivityCheck()`, whereas the `-timeout` option specifies socket timeouts (`nConnectTimeout`). The current descriptions are a bit misleading and hard to tell apart. I think it would save dev/review time to update them 🤷

ACKs for top commit:
  MarcoFalke:
    ACK eecb7ab105 nice doc fixup
  jnewbery:
    ACK eecb7ab105

Tree-SHA512: 71d2e6c31664b9f7f0b053ecf3be21c6c55472553fa7478d8526ba3be8d54979bceafca63d87b8b2488c11f409c332ac795da613ff8101546b18d9cd8bcceb50
2021-02-05 10:21:01 +01:00
MarcoFalke
01d2cf2674 Merge #21080: fuzz: Configure check for main function (take 2)
fac4be3048 fuzz: Configure check for main function (take 2) (MarcoFalke)

Pull request description:

  Actually fix https://github.com/google/honggfuzz/issues/336#issuecomment-702972138

  Follow-up to #20065

  Steps to test: `honggfuzz` section in doc/fuzzing.md

ACKs for top commit:
  practicalswift:
    cr ACK fac4be3048: patch looks correct!

Tree-SHA512: 893768c80439fe5d90b883ade89dc02f5bb80e27637916cf5575b6a9ed0b1c04942ff851342f5bbabb8666e6696715427feeb98f5301ad23c7b87b09e5872f97
2021-02-05 10:04:56 +01:00
MarcoFalke
4013e44c74 Merge #21082: refactor: Treat ArgsManager::Flags as uint32_t explicitly
faf3b4b533 refactor: Treat ArgsManager::Flags as uint32_t explicitly (MarcoFalke)

Pull request description:

  The underlying type might be implementation defined, which is probably why the sanitizer kills the fuzz tests.

  Fix that by pinning the underlying type.

  This refactor does not change behaviour and only affects the sanitizer in tests.

ACKs for top commit:
  practicalswift:
    cr ACK faf3b4b533

Tree-SHA512: d446824836e1037b4200ba3630c8628090678cfad45559866275d8e06349f7c8cdb7e816619f5afb35f9f65299cc00e046d2f81b73cd8eb843e2e15676b647d5
2021-02-05 09:54:43 +01:00
Bruno Garcia
0d39b5848a test: fix timeout decrease in feature_assumevalid 2021-02-04 21:43:43 -02:00
gzhao408
eecb7ab105 [doc] clarify -peertimeout and -timeout descriptions 2021-02-04 13:10:48 -08:00
MarcoFalke
faf3b4b533 refactor: Treat ArgsManager::Flags as uint32_t explicitly 2021-02-04 19:34:15 +01:00
Bruno Garcia
5e0cd25e29 fix the unreachable code at feature_taproot 2021-02-04 10:57:37 -02:00
MarcoFalke
fac4be3048 fuzz: Configure check for main function (take 2) 2021-02-04 13:11:57 +01:00
Gunar C. Gessner
e1604b3d50 doc: Replace tabs for spaces 2021-02-04 12:06:13 +00:00
Kiminuo
1bca2aa694 Introduce GetUniquePath(base) helper method to replace boost::filesystem::unique_path() which is not available in std::filesystem. 2021-02-04 11:38:09 +01:00
MarcoFalke
ea5a50f92a Merge #21042: doc, test: Improve setup_clean_chain documentation
590bda79e8 scripted-diff: Remove setup_clean_chain if default is not changed (Fabian Jahr)
98892f39e3 doc: Improve setup_clean_chain documentation (Fabian Jahr)

Pull request description:

  The first commit improves documentation on setup_clean_chain which is misunderstood quite frequently. Most importantly it fixes the TestShell docs which are simply incorrect.

  The second commit removes the instances of `setup_clean_clain` in functional tests where it is not changing the default.

  This used to be part of #19168 which also sought to rename`setup_clean_chain`.

ACKs for top commit:
  jonatack:
    ACK 590bda79e8

Tree-SHA512: a7881186e65d31160b8f84107fb185973b37c6e50f190a85c6e2906a13a7472bb4efa9440bd37fe0a9ac5cd2d1e8559870a7e4380632d9a249eca8980b945f3e
2021-02-04 10:30:06 +01:00
MarcoFalke
f1239b70d1 Merge #21025: validation: Guard chainman chainstates with cs_main
20677ffa22 validation: Guard all chainstates with cs_main (Carl Dong)

Pull request description:

  ```
  This avoids a potential race-condition where a thread is reading the
  ChainstateManager::m_active_chainstate pointer while another one is
  writing to it. There is no portable guarantee that reading/writing the
  pointer is thread-safe.

  This is also done in way that mimics ::ChainstateActive(), so the
  transition from that function to this method is easy.

  More discussion:
  1. https://github.com/bitcoin/bitcoin/pull/20749#discussion_r559544027
  2. https://github.com/bitcoin/bitcoin/pull/19806#discussion_r561023961
  3. https://github.com/bitcoin/bitcoin/pull/19806#issuecomment-768946522
  4. https://github.com/bitcoin/bitcoin/pull/19806#issuecomment-768955695
  ```

  Basically this PR removes the loaded-but-unfired footgun, which:
  - Is multiplied (but still unshot) in the chainman deglobalization PRs (#20158)
  - Is shot in the test framework in the au.activate PR (#19806)

ACKs for top commit:
  jnewbery:
    code review ACK 20677ffa22. I've verified by eye that neither of these members are accessed without cs_main.
  ryanofsky:
    Code review ACK 20677ffa22. It is safer to have these new `GUARDED_BY` annotations and locks than not to have them, but in the longer run I think every `LOCK(cs_main)` added here and added earlier in f92dc6557a from #20749 should be removed and replaced with `EXCLUSIVE_LOCKS_REQUIRED(cs_main)` on the accessor methods instead. `cs_main` is a high level lock that should be explicitly acquired at a high level to prevent the chain state from changing. It shouldn't be acquired recursively in low-level methods just to read pointer values atomically.

Tree-SHA512: 68a3a46d79a407b774eab77e1d682a97e95f1672db0a5fcb877572e188bec09f3a7b47c5d0cc1f2769ea276896dcbe97cb35c861acf7d8e3e513e955dc773f89
2021-02-04 10:22:44 +01:00
Gunar Gessner
98db48d349 doc: Fix markdown formatting
Lines were being joined making it hard to read.
2021-02-04 09:17:53 +00:00
MarcoFalke
4e946ebcf1 Merge #20715: util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet
fa61b9d1a6 util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet (MarcoFalke)
7777105a24 refactor: Move all command dependend checks to ExecuteWalletToolFunc (MarcoFalke)
fa06bce4ac test: Add tests (MarcoFalke)
fac05ccdad wallet: [refactor] Pass ArgsManager to WalletAppInit (MarcoFalke)

Pull request description:

  This not only moves the parsing responsibility out from the wallet tool, but it also makes it easier to implement bitcoin-util #19937

  Fixes: #20902

ACKs for top commit:
  ajtowns:
    ACK fa61b9d1a6
  fjahr:
    Code review ACK fa61b9d1a6

Tree-SHA512: 79622b806e8bf9dcd0dc24a8a6687345710df57720992e83a41cd8d6762a6dc112044ebc58fcf6e8fbf45de29a79b04873c5b8c2494a1eaaf902a2884703e47b
2021-02-04 09:12:05 +01:00
MarcoFalke
5a429d3d0f Merge #21049: Add release notes for listdescriptors RPC
51f3752fbe Add release notes for listdescriptors RPC (Ivan Metlushko)

Pull request description:

  Original PR is #20226

ACKs for top commit:
  jonatack:
    ACK 51f3752
  jonasschnelli:
    ACK 51f3752fbe

Tree-SHA512: e8091d01b99a3effcd6c1738e7363a44858ba9bcf6bd99bf60f2025a25db83fc8d61354ab2002365b56071b9f3693c7d534153a259b5ebc91cbcf8d13f6555f1
2021-02-04 08:51:37 +01:00
fanquake
5f18080c29 Merge #21065: build: make macOS HOST in download-osx generic
f22a3ec140 build: make macOS HOST in download-osx generic (fanquake)

Pull request description:

  This was missed in #20419, and the update before that, so just make this non-versioned so that we don't have to worry about it. This is fine, because it's just for downloading sources.

ACKs for top commit:
  RandyMcMillan:
    ACK f22a3ec140
  dongcarl:
    utACK f22a3ec140

Tree-SHA512: 3a6993a69594a793a5185e4ba48858443a1002a37b96ff881d39ca7719c79432b35d709bd9a9379f8046bdbeb716c5e1598f273a7e7e3f3bf528b6a807abe5ec
2021-02-04 10:11:06 +08:00
Fotis Koutoupas
ae9d26a8f0 wallet: Fix already-loading error message grammar 2021-02-04 00:19:11 +02:00
MarcoFalke
faf7d7418c fuzz: Avoid extraneous copy of input data, using Span<> 2021-02-03 19:30:14 +01:00
Carl Dong
5200929bfe depends: Include GUIX_ENVIRONMENT in id string 2021-02-03 12:10:10 -05:00
Carl Dong
4c7d418588 depends: Improve id string robustness
Environment variables and search paths can drastically effect the
operation of build tools.

Include these in our id string to mitigate against false cache hits.
2021-02-03 12:10:10 -05:00
Carl Dong
b3bdff42b5 build: Proper quoting for var printing targets
Previously, if the value contained syntax that was meaningful to make,
the printing would fail. Quoting properly avoids this.
2021-02-03 12:10:02 -05:00
fanquake
2ecaf21433 gitian: remove execstack workaround for ricv64 & powerpc64le
When building with g++-10 (or 8) on Focal, binaries are being produced
with noexecstack by default, so we can remove the workaround of
explicitly passing "-Wl,-z,noexecstack" for risvc46 and powerpc64le.

When building for powerpc64 this is still required.
2021-02-03 21:47:16 +08:00
fanquake
5baff2b318 build: use focal in gitian descriptors
Compilers used change as follows:
Linux native GCC 7.5 -> GCC 8.4
Linux cross GCC 8.4 -> GCC 8.4
Windows mingw-w64 7.3 -> mingw-w64 9.3
macOS Clang 8.0.0 -> Clang 8.0.0

The macOS and Win cross builds in the CI are updated to use Focal, and
per the op, running the security tests is disabled in the Windows
build.
2021-02-03 21:39:54 +08:00
Jon Atack
747cb5b994 netinfo: display only outbound block relay counts 2021-02-03 14:18:20 +01:00
Jon Atack
76d198a5c1 netinfo: add i2p network
the i2p peer counts column is displayed iff the node is connected
to at least one i2p peer, so this doesn't add clutter for users
who are not running an i2p service
2021-02-03 14:17:32 +01:00
Jon Atack
9d6aeca2c5 netinfo: add bip152 high-bandwidth to/from fields 2021-02-03 14:17:30 +01:00
Jon Atack
5de7a6cf63 netinfo: display manual peers count 2021-02-03 14:17:28 +01:00
Jon Atack
d3cca3be63 netinfo: update to use peer connection types 2021-02-03 14:17:18 +01:00
fanquake
ea96e17e1f Merge #21060: doc: More precise -debug and -debugexclude doc
572fd0f738 doc: More precise -debug and -debugexclude doc (wodry)

Pull request description:

  I wondered how one could enable debug logging with `-debug=<category>` for multiple categories. Found out solution is to specify that option multiple times for each wanted category.

  This PR documents this behavior and uses the same wording for the same behavior of `-debugexclude=<category>` to make that also clear and stringent.

ACKs for top commit:
  laanwj:
    ACK 572fd0f738
  MarcoFalke:
    ACK 572fd0f738
  theStack:
    ACK 572fd0f738

Tree-SHA512: 8d93db37602fd5ff4247e7c11478e55b99c0e3d47eaa2bb901937805d8f2a466b3a198b713b760981c5411576b74c52e2909c46c6d3f2e0e04215fd521b65cf7
2021-02-03 10:02:26 +08:00
gzhao408
5786a818e1 Verify that all validation flags are backward compatible
See #10699, i.e. adding a flag should always reduce the
number of acceptable scripts.

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 09:17:58 -08:00
gzhao408
b10ce9aa48 [test] check verification flags are minimal/maximal
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 09:17:58 -08:00
gzhao408
a260c22cad [test] Check for invalid flag combinations 2021-02-02 09:17:58 -08:00
gzhao408
a7098a2a8d [refactor] use CheckTxScripts, TrimFlags, FillFlags
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
7a77727b2f Apply minimal validation flags to tx_invalid tests
- Reduce the number of validation flags used, to a minimally required set to fail a test

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
9532591bed [test] add BADTX setting for invalid txns that fail CheckTransaction
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
4c06ebf128 [test] fix two witness tests in invalid tests with empty vout
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
158a0b268c Apply maximal validation flags to tx_valid tests
- Apply all validation flags by default
- Invert the meaning of verifyFlags as flags being excluded

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
0a76a39b63 [test] fix CSV test missing OP_ADD
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
19db590d04 [test] remove unnecessary OP_1s from CSV and CLTV tests
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
9db10a5506 [refactor] clean up logic in testmempoolaccept
Cleans up reundant code and reduces the diff of the next commit.
2021-02-02 06:56:16 -08:00
Jon Atack
62bf5b7850 netinfo: add ConnectionTypeForNetinfo member helper function 2021-02-02 15:22:22 +01:00
Jon Atack
e1e6714832 doc: refer to BIPs 339/155 in feature negotiation
and add fSuccessfullyConnected doxygen documentation
to clarify that it is set to true on VERACK
2021-02-02 14:49:06 +01:00
fanquake
f22a3ec140 build: make macOS HOST in download-osx generic
This was missed in #20419, and the update before that, so just
make this un-versioned so that we don't have to worry about it.
This is fine, because it's just for downloading sources.
2021-02-02 20:28:17 +08:00
MarcoFalke
384e090f93 Merge #19509: Per-Peer Message Capture
bff7c66e67 Add documentation to contrib folder (Troy Giorshev)
381f77be85 Add Message Capture Test (Troy Giorshev)
e4f378a505 Add capture parser (Troy Giorshev)
4d1a582549 Call CaptureMessage at appropriate locations (Troy Giorshev)
f2a77ff97b Add CaptureMessage (Troy Giorshev)
dbf779d5de Clean PushMessage and ProcessMessages (Troy Giorshev)

Pull request description:

  This PR introduces per-peer message capture into Bitcoin Core.  📓

  ## Purpose

  The purpose and scope of this feature is intentionally limited.  It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?".

  ## Functionality

  When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured.  The capture occurs in the MessageHandler thread.  When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue.  When sending, the message is captured just before the message is pushed onto the vSendMsg queue.

  The message capture is as minimal as possible to reduce the performance impact on the node.  Messages are captured to a new `message_capture` folder in the datadir.  Each node has their own subfolder named with their IP address and port.  Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so:

  ```
  message_capture/203.0.113.7:56072/msgs_recv.dat
  message_capture/203.0.113.7:56072/msgs_sent.dat
  ```

  Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON.  This script has been placed on its own and out of the way in the new `contrib/message-capture` folder.  Its usage is simple and easily discovered by the autogenerated `-h` option.

  ## Future Maintenance

  I sympathize greatly with anyone who says "the best code is no code".

  The future maintenance of this feature will be minimal.  The logic to deserialize the payload of the p2p messages exists in our testing framework.  As long as our testing framework works, so will this tool.

  Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small.

  ## FAQ

  "Why not just use Wireshark"

  Yes, Wireshark has the ability to filter and decode Bitcoin messages.  However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol.  This drives the design in a different direction than Wireshark, in two different ways.  First, this tool must be convenient and simple to use.  Using an external tool, like Wireshark, requires setup and interpretation of the results.  To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty.  This tool, on the other hand, "just works".  Turn on the command line flag, run your node, run the script, read the JSON.  Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible.  A lot can happen in the SocketHandler thread that would be missed by Wireshark.

  Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent.  As can be seen by the **many** previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done.

  Lastly, I truly believe that this tool will be used significantly more by being included in the codebase.  It's just that much more discoverable.

ACKs for top commit:
  MarcoFalke:
    re-ACK bff7c66e67 only some minor changes: 👚
  jnewbery:
    utACK bff7c66e67
  theStack:
    re-ACK bff7c66e67

Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47
2021-02-02 13:11:28 +01:00
Wladimir J. van der Laan
1e69800d5e Merge #21059: Drop boost/preprocessor dependencies
e99db77a6e Drop boost/preprocessor dependencies (Hennadii Stepanov)
12f5028d49 refactor: Move STRINGIZE macro to macros.h (Hennadii Stepanov)

Pull request description:

  Use own macros instead of boost's ones.

ACKs for top commit:
  laanwj:
    Code review ACK e99db77a6e
  practicalswift:
    cr ACK e99db77a6e

Tree-SHA512: 7ec15c2780a661e293c990f64c41b5b451d894cc191aa7872fbcaf96da91915a351209b1f1003ab12a7a16cb464e50ac58a028db02beeedfa5f6931752c2d9e2
2021-02-02 12:44:27 +01:00
MarcoFalke
3ddbf22ed1 util: Disallow negative mocktime
Signed-off-by: practicalswift <practicalswift@users.noreply.github.com>
2021-02-02 08:43:19 +00:00
practicalswift
f5f2f97168 net: Avoid UBSan warning in ProcessMessage(...) 2021-02-02 08:43:19 +00:00
Ivan Metlushko
51f3752fbe Add release notes for listdescriptors RPC
Original PR is #20226
2021-02-02 08:21:46 +01:00
fanquake
060a2a64d4 ci: remove boost thread installation
Adjust fuzzbuzz.yml to only install the Boost components we need.
2021-02-02 12:38:22 +08:00
fanquake
06e1d7d81d build: don't build or use Boost Thread 2021-02-02 12:38:22 +08:00
fanquake
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache
Co-authored-by: MarcoFalke falke.marco@gmail.com
Co-authored-by: sinetek pitwuu@gmail.com
2021-02-02 12:38:10 +08:00
fanquake
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests
Co-authored-by: MarcoFalke falke.marco@gmail.com
Co-authored-by: sinetek pitwuu@gmail.com
2021-02-02 12:35:40 +08:00
Carl Dong
20677ffa22 validation: Guard all chainstates with cs_main
Since these chainstates are:

1. Also vulnerable to the race condition described in the previous
   commit
2. Documented as having similar semantics as m_active_chainstate

we should also protect them with ::cs_main.
2021-02-01 22:09:03 -05:00
fanquake
f72d80b07a Merge #21051: Fix -Wmismatched-tags warnings
b6aadcd5b4 build: Add -Werror=mismatched-tags (Hennadii Stepanov)
1485124291 Fix -Wmismatched-tags warnings (Hennadii Stepanov)

Pull request description:

  Warnings were introduced in #20749:
  ```
  ./validation.h:43:1: warning: class 'CCheckpointData' was previously declared as a struct; this is valid, but may result in linker errors under the Microsoft C++ ABI [-Wmismatched-tags]
  class CCheckpointData;
  ^
  ./chainparams.h:24:8: note: previous use is here
  struct CCheckpointData {
         ^
  ./validation.h:43:1: note: did you mean struct here?
  class CCheckpointData;
  ^~~~~
  struct
  1 warning generated.
  ```

  This change fixes AppVeyor build: https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/37547435

ACKs for top commit:
  glozow:
    utACK b6aadcd5b4 🚗
  practicalswift:
    cr ACK b6aadcd5b4: patch looks correct

Tree-SHA512: 3ac887ebdbf9a1ae33c1fd5381b3b8d83388ad557ddeb55013acd42bb9752a5bd009e3a0eed52644a023a7a0dda1c159277981af82f58fb0abfe60b84e01bf29
2021-02-02 09:20:19 +08:00
Jon Atack
96635e6177 init: use GetNetworkNames() in -onlynet help 2021-02-02 00:16:42 +01:00
Jon Atack
0dbde700a6 rpc: use GetNetworkNames() in getnetworkinfo and getpeerinfo helps 2021-02-02 00:16:04 +01:00
Jon Atack
1c3af37881 net: create GetNetworkNames() 2021-02-02 00:01:36 +01:00
Jon Atack
b45eae4d53 net: update NET_UNROUTABLE to not_publicly_routable in GetNetworkName() 2021-02-02 00:00:39 +01:00
Fabian Jahr
590bda79e8 scripted-diff: Remove setup_clean_chain if default is not changed
-BEGIN VERIFY SCRIPT-
git grep -l "self.setup_clean_chain = False" test/functional/*.py | xargs sed -i "/self.setup_clean_chain = False/d";
-END VERIFY SCRIPT-
2021-02-01 23:13:38 +01:00
Fabian Jahr
98892f39e3 doc: Improve setup_clean_chain documentation 2021-02-01 23:13:35 +01:00
Hennadii Stepanov
b6aadcd5b4 build: Add -Werror=mismatched-tags 2021-02-01 23:03:10 +02:00
wodry
572fd0f738 doc: More precise -debug and -debugexclude doc
I wondered how one could enable debug logging with `-debug=<category>` for multiple categories. Found out solution is to specify that option multiple times for each wanted category.

This PR documents this behavior and uses the same wording for the same behavior of `-debugexclude=<category>` to make that also clear and stringent.
2021-02-01 21:33:31 +01:00
Hennadii Stepanov
e99db77a6e Drop boost/preprocessor dependencies 2021-02-01 22:30:06 +02:00
Hennadii Stepanov
12f5028d49 refactor: Move STRINGIZE macro to macros.h
This is a move-only change.
2021-02-01 22:30:05 +02:00
Wladimir J. van der Laan
2c0fc856a6 Merge #20464: refactor: Treat CDataStream bytes as uint8_t
fa29272459 Remove redundant MakeUCharSpan wrappers (MarcoFalke)
faf4aa2f47 Remove CDataStream::Init in favor of C++11 member initialization (MarcoFalke)
fada14b948 Treat CDataStream bytes as uint8_t (MarcoFalke)
fa8bdb048e refactor: Drop CDataStream constructors in favor of one taking a Span of bytes (MarcoFalke)
faa96f841f Remove unused CDataStream methods (MarcoFalke)

Pull request description:

  Using `uint8_t` for raw bytes has a style benefit:
  * The signedness is clear from reading the code, as it does not depend on the architecture

  Other clean-ups in this pull include:
  * Remove unused methods
  * Constructor is simplified with `Span`
  * Remove `Init()` member in favor of C++11 member initialization

ACKs for top commit:
  laanwj:
    code review ACK fa29272459
  theStack:
    ACK fa29272459 🍾

Tree-SHA512: 931ee28bd99843d7e894b48e90e1187ffb0278677c267044b3c0c255069d9bbd9298ab2e539b1002a30b543d240450eaec718ef4ee95a7fd4be0a295e926343f
2021-02-01 15:17:28 +01:00
Wladimir J. van der Laan
56fcf93349 Merge #21026: doc: Document use of make-tag script to make tags
cc30dfbd4b doc: Document use of make-tag script to make tags (Wladimir J. van der Laan)

Pull request description:

  To make release tags the `make-tag.py` script from the maintainer tools should be used. This ensures that all the various occurrences of the version in different files match the tagged version before proceeding. And move it into a separate section with the other per-release actions.

  Also replace other "ping wumpus" references.

ACKs for top commit:
  jonatack:
    ACK cc30dfbd4b

Tree-SHA512: c09748a0bea85573b3f04fdb86430a53b683ff4d956edc1f49d471e2e526715cbde7cf6ad83a43a1a3fca4ff3c5af011e3d1b8cb61f5d8e065cfa71ba0138c88
2021-02-01 14:27:09 +01:00
Hennadii Stepanov
1485124291 Fix -Wmismatched-tags warnings 2021-02-01 14:37:14 +02:00
Wladimir J. van der Laan
d0d256536c Merge #21016: refactor: remove boost::thread_group usage
dc8be12510 refactor: remove boost::thread_group usage (fanquake)

Pull request description:

  Post #18710, there isn't much left using `boost::thread_group`, so should just be able to replace it with the standard library. This also removes the last use of `boost::thread_interrupted`.

  After this change, last piece of Boost Thread we'd be using is `boost::shared_mutex`. See the commentary [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) as to why it may be non-trivial to swap that for `std::shared_mutex` in the near future.

  Closes #17307

ACKs for top commit:
  laanwj:
    Code review re-ACK dc8be12510
  MarcoFalke:
    review ACK dc8be12510 🔁
  jonatack:
    Non-expert code review ACK dc8be12510, also checked range-diff since last review and that local debug build is clean with gcc 10.2.1-6 on Debian

Tree-SHA512: 5510e2d760cce824234207dc86b1551ca8f21cbf3a2ce753c0254a0d03ffd83c94e449aec202fb7bd76e6fc64df783a6b70a736b0add9ece3734bb9c8ce8fc2f
2021-02-01 13:27:28 +01:00
Wladimir J. van der Laan
44f4bcd302 Merge #20749: [Bundle 1/n] Prune g_chainman usage related to ::LookupBlockIndex
67c9a83df1 style-only: Remove redundant sentence in ActivateBestChain comment (Carl Dong)
b8e95658d5 style-only: Make TestBlockValidity signature readable (Carl Dong)
0cdad75390 validation: Use accessible chainstate in ChainstateManager::ProcessNewBlock (Carl Dong)
ea4fed9021 validation: Use existing chainstate in ChainstateManager::ProcessNewBlockHeaders (Carl Dong)
e0dc305727 validation: Move LoadExternalBlockFile to CChainState (Carl Dong)
5f8cd7b3a5 validation: Remove global ::ActivateBestChain (Carl Dong)
2a696472a1 validation: Pass in chainstate to ::NotifyHeaderTip (Carl Dong)
9c300cc8b3 validation: Pass in chainstate to TestBlockValidity (Carl Dong)
0e17c833cd validation: Make CChainState.m_blockman public (Carl Dong)
d363d06bf7 validation: Pass in blockman to ContextualCheckBlockHeader (Carl Dong)
f11d11600d validation: Move GetLastCheckpoint to BlockManager (Carl Dong)
e4b95eefbc validation: Move GetSpendHeight to BlockManager (Carl Dong)
b026e318c3 validation: Move FindForkInGlobalIndex to BlockManager (Carl Dong)
3664a150ac validation: Remove global LookupBlockIndex (Carl Dong)
eae54e6e60 scripted-diff: Use BlockManager::LookupBlockIndex (Carl Dong)
15d20f40e1 validation: Move LookupBlockIndex to BlockManager (Carl Dong)
f92dc6557a validation: Guard the active_chainstate with cs_main (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

ACKs for top commit:
  jnewbery:
    utACK 67c9a83df1
  laanwj:
    re-ACK 67c9a83df1
  ryanofsky:
    Code review ACK 67c9a83df1. Changes since last review:

Tree-SHA512: 8744aba2dd57a40cd2fedca809b0fe24d771bc60da1bffde89601999384aa0df428057a86644a3f72fbeedbc8b04db6c4fd264ea0db2e73c279e5acc6d056cbf
2021-02-01 13:09:46 +01:00
MarcoFalke
636e754a81 Merge #20941: rpc: document RPC_TRANSACTION_ALREADY_IN_CHAIN exception
74d23bf7fb rpc: document RPC_TRANSACTION_ALREADY_IN_CHAIN exception (Jarol Rodriguez)

Pull request description:

  It is not documented in the `RPCHelpMan` of `sendrawtransaction` that if you attempt to send a transaction which already exists in a block, an `RPC_TRANSACTION_ALREADY_IN_CHAIN` exception will be raised. It is best to make developers aware of this so that it can be properly caught and avoid any headaches.

  Closes #5638

ACKs for top commit:
  jonatack:
    ACK 74d23bf7fb

Tree-SHA512: d1d5fc242574377c8a76b4ef7b12239996424d8bee186533b5a8fe337bbeb3186e51dbdd28c5eafb982601e44e17b68a7f52db5dd7bc647429f6f95e2de289f6
2021-02-01 10:59:50 +01:00
MarcoFalke
87394b6741 Merge #20868: validation: remove redundant check on pindex
c943282b5e validation: remove redundant check on pindex (jarolrod)

Pull request description:

  This removes a redundant check on `pindex` being a `nullptr`. By the time we get to this step `pindex` is always a `nullptr` as the branch where it has been set would have already returned.

  Closes #19223

ACKs for top commit:
  Zero-1729:
    re-ACK c943282
  ajtowns:
    ACK c943282b5e - code review only
  MarcoFalke:
    review ACK c943282b5e 📨
  theStack:
    re-ACK c943282b5e

Tree-SHA512: d2dc58206be61d2897b0703ee93af67abed492a80e84ea03dcfbf7116833173da3bdafa18ff80422d5296729d5254d57cc1db03fdaf817c8f57c62c3abef673c
2021-02-01 10:56:23 +01:00
Samuel Dobson
7dc4807691 Merge #20040: wallet: Refactor OutputGroups to handle fees and spending eligibility on grouping
5d4597666d Rewrite OutputGroups to be clearer and to use scriptPubKeys (Andrew Chow)
f6b3052739 Explicitly filter out partial groups when we don't want them (Andrew Chow)
416d74fb16 Move OutputGroup positive only filtering into Insert (Andrew Chow)
d895e98b59 Move EligibleForSpending into GroupOutputs (Andrew Chow)
99b399aba5 Move fee setting of OutputGroup to Insert (Andrew Chow)
6148a8acda Move GroupOutputs into SelectCoinsMinConf (Andrew Chow)
2acad03657 Remove OutputGroup non-default constructors (Andrew Chow)

Pull request description:

  Even after #17458, we still deal with setting fees of an `OutputGroup` and filtering the `OutputGroup` outside of the struct. We currently make all of the `OutputGroup`s in `SelectCoins` and then copy and modify them within each `SelectCoinsMinConf` scenario. This PR changes this to constructing the `OutputGroup`s within the `SelectCoinsMinConf` so that the scenario can be taken into account during the group construction. Furthermore, setting of fees and filtering for effective value is moved into `OutputGroup::Insert` itself so that we don't add undesirable outputs to an `OutputGroup` rather than deleting them afterwards.

  To facilitate fee calculation and effective value filtering during `OutputGroup::Insert`, `OutputGroup` now takes the feerates in its constructor and computes the fees and effective value for each output during `Insert`.

  While removing `OutputGroup`s in accordance with the `CoinEligibilityFilter` still requires creating the `OutputGroup`s first, we can do that within the function that makes them - `GroupOutput`s.

ACKs for top commit:
  Xekyo:
    Code review ACK: 5d4597666d
  fjahr:
    Code review ACK 5d4597666d
  meshcollider:
    Light utACK 5d4597666d

Tree-SHA512: 35965b6d49a87f4ebb366ec4f00aafaaf78e9282481ae2c9682b515a3a9f2cbcd3cd6e202fee29489d48fe7f3a7cede4270796f5e72bbaff76da647138fb3059
2021-02-01 22:43:17 +13:00
Sebastian Falbesoner
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17)
Removes the macro ARRAYLEN and also substitutes all other uses of the same
"sizeof(a)/sizeof(a[0])" pattern by std::size, available since C++17.
2021-01-31 17:35:16 +01:00
Sebastian Falbesoner
365539c846 refactor: init vectors via std::{begin,end} to avoid pointer arithmetic 2021-01-31 17:35:01 +01:00
Hennadii Stepanov
964885d048 qt: Save/restore recentRequestsView table column sizes
Sorting order is not saved/restored intentionally.
2021-01-31 18:09:30 +02:00
Hennadii Stepanov
3913d1e8c1 qt: Drop buggy TableViewLastColumnResizingFixer class
In Qt 5 the last column resizing with dragging its left edge works
out-of-the-box.
The current TableViewLastColumnResizingFixer implementation could put
the last column content out of the view port and confuse a user.
2021-01-31 18:09:30 +02:00
Hennadii Stepanov
f5c8093e77 qt: Move recentRequestsView properties settings to constructor
This is move-only change.
2021-01-31 18:09:30 +02:00
Hennadii Stepanov
9c5f4f2169 qt: Save/restore TransactionView table column sizes
Sorting order is not saved/restored intentionally.
2021-01-31 18:09:30 +02:00
Hennadii Stepanov
788205c3f7 qt: Move transactionView properties settings to constructor
This is move-only change.
2021-01-31 18:09:30 +02:00
Hennadii Stepanov
ecdbaf71c0 qt, refactor: Drop intermediate assignment
This change improves code readability.
2021-01-31 18:09:30 +02:00
Sebastian Falbesoner
63d4ee1968 refactor: iterate arrays via C++11 range-based for loops if idx is not needed 2021-01-31 17:07:46 +01:00
MarcoFalke
4c55f92c76 Merge #20954: test: Declare nodes type in test_framework.py.
5353b0c64d Change type definitions for "chain" and "setup_clean_chain" from type comments to Python 3.6+ types. Additionally, set type for "nodes". (Kiminuo)

Pull request description:

  ### Motivation

  When I wanted to understand better https://github.com/bitcoin/bitcoin/pull/19145/files#diff-4bebbd3b112dc222ea7e75ef051838ceffcee63b9e9234a98a4cc7251d34451b test, I noticed that navigation in PyCharm/VS Code did not work for `nodes` variable. I think this is frustrating, especially for newcomers.

  ### Summary

  * This PR modifies Python 3.5 [type comments](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#variables) to Python 3.6+ types and adds a proper type for `nodes` [instance attribute](https://mypy.readthedocs.io/en/stable/class_basics.html#instance-and-class-attributes).
  * This PR does not change behavior.
  * This PR is intentionally very small, if the concept is accepted, a follow-up PRs can be more ambitious.

  ### End result

  1. Open `test/functional/feature_abortnode.py`
  2. Move your caret to: `self.nodes[0].generate[caret here](3)`
  3. Use "Go to definition" [F12] should work now.

  I have tested this on PyCharm (Windows, Ubuntu) and VS Code (Windows, Ubuntu).

  Note: Some `TestNode` methods (e.g. `self.nodes[0].getblock(...)` ) use `__call__` mechanism and navigation does not work for them even with this PR.

ACKs for top commit:
  laanwj:
    ACK 5353b0c64d
  theStack:
    ACK 5353b0c64d

Tree-SHA512: 821773f052ab9b2889dc357d38c59407a4af09e3b86d7134fcca7d78e5edf3a5ede9bfb37595ea97caf9ebfcbda372bcf73763b7f89b0677670f21b3e396a12b
2021-01-31 09:28:21 +01:00
MarcoFalke
1f514332ed Merge #21037: fuzz: Avoid designated initialization (C++20) in fuzz tests
dee2d6fbf9 fuzz: Avoid designated initialization (C++20) in fuzz tests (practicalswift)

Pull request description:

  Avoid designated initialization (C++20) in fuzz tests.

  Context: https://github.com/bitcoin/bitcoin/pull/20197#discussion_r565270556, https://github.com/bitcoin/bitcoin/pull/20936#discussion_r566708730

ACKs for top commit:
  MarcoFalke:
    cr ACK dee2d6fbf9
  dhruv:
    code review ACK dee2d6fbf9
  ajtowns:
    utACK dee2d6fbf9

Tree-SHA512: 5940fab6e97a2b11dd3b1475d2cffa2840dc2e6ec34bd9f9df90f948709cab98fd1c513d5dd104816d33a525a6e9710b8715b02db941e35d84f92bc211f56d1d
2021-01-31 08:43:54 +01:00
Fabian Jahr
e987ae5a55 test: Add test for deterministic UTXO set hash results 2021-01-30 20:33:23 +01:00
Fabian Jahr
6ccc8fc067 test: Add test for gettxoutsetinfo RPC with MuHash 2021-01-30 20:33:20 +01:00
Jon Atack
506e6585a5 gui: display plain "Inbound" in peer details 2021-01-30 20:28:37 +01:00
Fabian Jahr
0d3b2f643d rpc: Add hash_type MUHASH to gettxoutsetinfo
Also small style fix in rpc/util.cpp
2021-01-30 17:38:21 +01:00
Fabian Jahr
2474645f3b refactor: Separate hash and stats calculation in coinstats 2021-01-30 16:51:02 +01:00
practicalswift
dee2d6fbf9 fuzz: Avoid designated initialization (C++20) in fuzz tests 2021-01-30 08:22:32 +00:00
Russell Yanofsky
9048c58e10 Remove pointer cast in CRPCTable::dumpArgMap
CRPCTable::dumpArgMap currently works by casting RPC command unique_id
integer field to a function pointer, and then calling the function. The
unique_id field wasn't supposed to be used this way (it's meant to be
used to detect RPC aliases), and this code segfaults in the rpc_help.py
test in multiprocess PR https://github.com/bitcoin/bitcoin/pull/10102
because wallet RPC functions aren't directly accessible from the node
process.

Fix this by adding a new GET_ARGS request mode to retrieve argument
information similar to the way the GET_HELP mode retrieves help
information.
2021-01-29 18:15:48 -05:00
Russell Yanofsky
14f3d9b908 refactor: Add RPC server ExecuteCommands function
No change in behavior. New function is split from CRPCTable::execute and
used in the next commit.
2021-01-29 18:12:19 -05:00
Russell Yanofsky
6158a6d397 refactor: Replace JSONRPCRequest fHelp field with mode field
No change in behavior
2021-01-29 18:09:46 -05:00
Wladimir J. van der Laan
16b784d953 Merge #20689: contrib: replace binary verification script verify.sh with python rewrite
c86b9a65eb contrib: remove verify.sh (Sebastian Falbesoner)
c84838e7af contrib: binary verification script verify.sh rewritten in python (Sebastian Falbesoner)

Pull request description:

  The rationale for the PR is the same as for #18132:
  > Most of our test scripts are written in python. We don't have enough reviewers for bash scripts and they tend to be clumsy anyway. Especially when it comes to argument parsing.

  Note that there are still a lot of things that could be improved in this replacement (e.g. using regexps for version string parsing, adding type annotations, dividing up into more functions, getting a pylint score closer to 10, etc.), but I found the original shell script quite hard to read, so it's possibly still a good first step for an improvement.
  ~Not sure though if it's worth the reviewers time, and if it's even continued to be used long-term (maybe there are plans to merge it with `get_previous_releases.py`, which partly does the same?), so chasing for Concept ACKs right now.~

ACKs for top commit:
  laanwj:
    Tested and code review ACK c86b9a65eb

Tree-SHA512: f7949eead4ef7e5913fe273923ae5c5299408db485146cf996cdf6f8ad8c0ee4f4b30bb6b08a5964000d97b2ae2e7a1bdc88d11c613c16d2d135d80b444e3b16
2021-01-29 20:08:22 +01:00
Anthony Towns
6452190841 net_processing: simplify MaybeSetPeerAsAnnouncingHeaderAndIDs args
No need to pass connman to PeerManagerImpl methods.
2021-01-30 02:27:18 +10:00
Anthony Towns
39c2a69bc2 net_processing: move MaybeSetPeerAsAnnouncingHeadersAndIDs into PeerManagerImpl
Allows making lNodesAnnouncingHeaderAndIDs and
nPeersWithValidatedDownloads member vars instead of globals.
2021-01-30 02:27:12 +10:00
Anthony Towns
7b7117efd0 net_processing: simplify ProcessGetData and FindTxForGetData args
No need to pass mempool or connman to PeerManagerImpl methods.
2021-01-30 02:18:49 +10:00
Anthony Towns
34207b9004 net_processing: move FindTxForGetData and ProcessGetData to PeerManagerImpl
Allows making mapRelay and vRelayExpiration members rather than globals.
2021-01-30 02:16:37 +10:00
Anthony Towns
d44084883a net_processing: simplify PeerManageImpl method args
No need to pass mempool to MarkBlockAsInFlight, or consensusParams to
TipMayBeStale or FindNextBlocksToDownload.
2021-01-30 02:11:01 +10:00
Anthony Towns
a490f0a056 net_processing: move MarkBlockAs*, TipMayBeStale, FindNextBlocksToDL to PeerManagerImpl
Allows converting mapBlocksInFlight and g_last_tip_update from globals
to member variables.
2021-01-30 02:06:58 +10:00
Anthony Towns
052d9bc7e5 net_processing: simplify AlreadyHaveTx args
No need to pass mempool to PeerManagerImpl methods.
2021-01-30 01:51:06 +10:00
Anthony Towns
eeac506250 net_processing: move AlreadyHaveTx into PeerManageImpl
Allows making recentRejects and g_recent_confirmed_transactions members
rather than globals.
2021-01-30 01:50:02 +10:00
Anthony Towns
9781c08a33 net_processing: move some globals into PeerManagerImpl
nSyncStarted, mapBlockSource, g_wtxid_relay_peers,
g_outbound_peers_with_protect_from_disconnect were all only used by
PeerManagerImpl methods already.
2021-01-30 01:26:09 +10:00
John Newbery
4676a4fb5b [addrman] Don't repeat "Bucketing method was updated" log multiple times
Thanks to Vasil Dimov <vd@FreeBSD.org> for these suggestions
2021-01-29 12:39:55 +00:00
John Newbery
436292367c [addrman] Improve serialization comments
Thanks to Vasil Dimov <vd@FreeBSD.org> for these suggestions
2021-01-29 12:39:55 +00:00
Wladimir J. van der Laan
cc30dfbd4b doc: Document use of make-tag script to make tags
To make release tags the `make-tag.py` script from the maintainer tools
should be used. This ensures that all the various occurences of the
version in different files match the tagged version before proceeding.

Also replace other "ping wumpus" references.
2021-01-29 08:46:11 +01:00
fanquake
dc8be12510 refactor: remove boost::thread_group usage 2021-01-29 15:39:44 +08:00
MarcoFalke
c8b83510f4 Merge #20724: Cleanup of -debug=net log messages
48c8a9b964 net_processing: log txrelay flag from version message (Anthony Towns)
98fab37ca0 net: use peer=N instead of from=N in debug log (Anthony Towns)
12302105bb net_processing: additional debug logging for ignored messages (Anthony Towns)
f7edea3b7c net: make debug logging conditional on -debug=net (Anthony Towns)
a410ae8cb0 net, net_processing: log disconnect reasons with -debug=net (Anthony Towns)

Pull request description:

  A few changes to -debug=net logging:

   * always log when disconnecting a peer
   * only log various connection errors when -debug=net is enabled, since errors from random untrusted peers is completely expected
   * log when ignoring a message due to violating protocol (primarily to make it easier to debug other implementations)
   * use "peer=123" rather than "from 123" to make grepping logs a bit easier
   * log the value of the bip-37 `fRelay` field in version messages both when sending and receiving a version message

ACKs for top commit:
  jnewbery:
    ACK 48c8a9b964
  MarcoFalke:
    re-ACK 48c8a9b964 only change is rebase 🚓
  practicalswift:
    re-ACK 48c8a9b964

Tree-SHA512: 6ac530d883dffc4fd7fe20b1dc5ebb5394374c9b499aa7a253eb4a3a660d8901edd72e5ad21ce4a2bf71df25e8f142087755f9756f3497f564ef453a7e9246c1
2021-01-29 07:44:22 +01:00
MarcoFalke
40dd757bf6 Merge #21012: ci: Fuzz with integer sanitizer
faff3991a9 ci: Fuzz with integer sanitizer (MarcoFalke)

Pull request description:

  Otherwise the suppressions file will go out of sync

ACKs for top commit:
  practicalswift:
    cr ACK faff3991a9: patch looks correct

Tree-SHA512: 349216d071a2c5ccf24565fe0c52d7a570ec148d515d085616a284f1ab9992ce10ff82eb17962dddbcda765bbd3a9b15e8b25f34bdbed99fc36922d4161d307c
2021-01-29 07:43:17 +01:00
MarcoFalke
bc5f26d4ee Merge bitcoin-core/gui#139: doc: Improve gui/src/qt README.md
5d1f260713 Improve gui/src/qt README.md (Jarol Rodriguez)

Pull request description:

  **Master/Before:** [Render of Master](https://github.com/bitcoin-core/gui/blob/master/src/qt/README.md)

  **PR/After:** [Render of PR](5d1f260713/src/qt/README.md)

  **Changes:**
  The README.md found in `gui/src/qt` seems to not have gotten any love in a while. This PR fixes some grammatical errors, makes it easier to follow, and modernizes the logic of using Qt Creator.
  1. Makes several sections more informative
  2. Directories under `Files and Directories` now end with a forward slash denoting that they are a directory
  3. Modernize the Qt Creator Logic for the current setup flow
  4. Add UNIX Qt Creator Setup Instructions (Ubuntu & Debian)

ACKs for top commit:
  RandyMcMillan:
    ACK 5d1f260 👍🏼
  jonatack:
    ACK 5d1f260713

Tree-SHA512: bd5cc24b95460f34a1efa489a6acc10d7632c84eabdcdc5729ef077d8303cf037ed664aae033af8883252433ea0999d8ec7d92e8b03d03a873d32b041a94f813
2021-01-29 07:40:44 +01:00
fanquake
e213382920 Merge #21029: bitcoin-cli: Correct docs (no "generatenewaddress" exists)
71430aec43 bitcoin-cli: Correct docs (no "generatenewaddress" exists) (Luke Dashjr)

Pull request description:

ACKs for top commit:
  jonatack:
    ACK 71430aec43

Tree-SHA512: 45ee7a51080598141fac4446563bdf99a59bfaa0ee00d9e092511087a968e8535d9208683ed589be1c975531be6d0319d33720f1f0dc1c3635c7d5ea6d726a41
2021-01-29 11:36:57 +08:00
randymcmillan
8353e8cecc peers-tab: bug fix right panel toggle 2021-01-28 20:37:20 -05:00
Luke Dashjr
71430aec43 bitcoin-cli: Correct docs (no "generatenewaddress" exists) 2021-01-29 00:38:57 +00:00
Sebastian Falbesoner
ba7e17e073 rpc, test: document {previous,next}blockhash as optional
Affects the following RPCs:
- getblockheader
- getblock

Also adds trivial tests on genesis block (should not contain
"previousblockhash") and best block (should not contain
"nextblockhash").
2021-01-29 01:07:18 +01:00
Wladimir J. van der Laan
6a726cb534 Merge #20963: gitian-linux: Build binaries for 64-bit POWER (continued)
543bf745d3 gitian-linux: Extend noexec-stack workaround to powerpc (Wladimir J. van der Laan)
00f67c8aa1 gitian-linux: Build binaries for 64-bit POWER (Luke Dashjr)
63fc2b1782 gitian: Properly quote arguments in wrappers (Luke Dashjr)
798bc0b29a Support glibc-back-compat on 64-bit POWER (Luke Dashjr)

Pull request description:

  Rebase of #14066 by luke-jr.

  Let's try to get PowerPC support in in the beginning of the 22.0 cycle so that it gets some testing, and is not a last-minute decision this time, like for last … 2 or 3 major versions.

  The symbol/security tooling-related changes have been dropped since they were part of #20434.

Top commit has no ACKs.

Tree-SHA512: df0f8cd320c90f359f8b512c5cb8b59bb277516b57a05482cc8923c656106513b7428e315aaa8ab53e0bd6f80556b07d3639c47f6d9913bcfbfe388b39ef47c4
2021-01-28 22:43:10 +01:00
Carl Dong
67c9a83df1 style-only: Remove redundant sentence in ActivateBestChain comment 2021-01-28 14:15:26 -05:00
Carl Dong
b8e95658d5 style-only: Make TestBlockValidity signature readable 2021-01-28 14:15:26 -05:00
Carl Dong
0cdad75390 validation: Use accessible chainstate in ChainstateManager::ProcessNewBlock
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.
2021-01-28 14:15:26 -05:00
Carl Dong
ea4fed9021 validation: Use existing chainstate in ChainstateManager::ProcessNewBlockHeaders
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.
2021-01-28 14:15:26 -05:00
Carl Dong
e0dc305727 validation: Move LoadExternalBlockFile to CChainState
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

LoadExternalBlockFile mainly acts on CChainState.
2021-01-28 14:15:26 -05:00
Carl Dong
5f8cd7b3a5 validation: Remove global ::ActivateBestChain
Instead use CChainState::ActivateBestChain, which is what the global one
calls anyway.
2021-01-28 14:15:26 -05:00
Carl Dong
2a696472a1 validation: Pass in chainstate to ::NotifyHeaderTip
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.
2021-01-28 14:15:26 -05:00
Carl Dong
9c300cc8b3 validation: Pass in chainstate to TestBlockValidity
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.
2021-01-28 14:15:26 -05:00
Carl Dong
0e17c833cd validation: Make CChainState.m_blockman public 2021-01-28 14:15:26 -05:00
Carl Dong
d363d06bf7 validation: Pass in blockman to ContextualCheckBlockHeader
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.
2021-01-28 14:15:26 -05:00
Carl Dong
f11d11600d validation: Move GetLastCheckpoint to BlockManager
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

GetLastCheckPoint mainly acts on BlockManager.
2021-01-28 14:15:26 -05:00
Carl Dong
e4b95eefbc validation: Move GetSpendHeight to BlockManager
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

GetSpendHeight only acts on BlockManager.
2021-01-28 14:15:26 -05:00
Carl Dong
b026e318c3 validation: Move FindForkInGlobalIndex to BlockManager
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

FindForkInGlobalIndex only acts on BlockManager.

Note to reviewers: Since FindForkInGlobalIndex is always called with
::ChainActive() as its first parameter, it is possible to move
FindForkInGlobalIndex to CChainState and remove this const CChain&
parameter to instead use m_chain. However, it seems like the original
intention was for FindForkInGlobalIndex to work with _any_ chain, not
just the current active chain. Let me know if this should be changed.
2021-01-28 14:15:26 -05:00
Carl Dong
3664a150ac validation: Remove global LookupBlockIndex 2021-01-28 14:15:26 -05:00
Carl Dong
eae54e6e60 scripted-diff: Use BlockManager::LookupBlockIndex
[META] In a previous commit, we moved ::LookupBlockIndex to become a
       member function of BlockManager. This commit is split out from
       that one since it can be expressed nicely as a scripted-diff.

-BEGIN VERIFY SCRIPT-
find_regex='LookupBlockIndex' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E "s@${find_regex}@g_chainman.m_blockman.LookupBlockIndex@g"
-END VERIFY SCRIPT-
2021-01-28 14:15:26 -05:00
Carl Dong
15d20f40e1 validation: Move LookupBlockIndex to BlockManager
[META] This commit should be followed up by a scripted-diff commit which
       fixes calls to LookupBlockIndex tree-wide.
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

LookupBlockIndex only acts on BlockManager.
2021-01-28 14:15:26 -05:00
Carl Dong
f92dc6557a validation: Guard the active_chainstate with cs_main
This avoids a potential race-condition where a thread is reading the
ChainstateManager::m_active_chainstate pointer while another one is
writing to it. There is no portable guarantee that reading/writing the
pointer is thread-safe.

This is also done in way that mimics ::ChainstateActive(), so the
transition from that function to this method is easy.

More discussion:
1. https://github.com/bitcoin/bitcoin/pull/20749#discussion_r559544027
2. https://github.com/bitcoin/bitcoin/pull/19806#discussion_r561023961
3. https://github.com/bitcoin/bitcoin/pull/19806#issuecomment-768946522
4. https://github.com/bitcoin/bitcoin/pull/19806#issuecomment-768955695
2021-01-28 14:14:22 -05:00
Jarol Rodriguez
5d1f260713 Improve gui/src/qt README.md
The current readme is a little bit outdated and contains some grammatical mistakes. This commit updates the doc so that:
  - It is easier to follow and is more informative
  - Fixes grammatical mistakes
  - Modernizes the Qt Creater setup instructions
  - Adds UNIX instructions for Qt Creator setup
2021-01-28 14:03:21 -05:00
Wladimir J. van der Laan
80e16cadd5 Merge #20012: rpc: Remove duplicate name and argNames from CRPCCommand
fa04f9b4dd rpc: Remove duplicate name and argNames from CRPCCommand (MarcoFalke)
fa92912b4b rpc: Use RPCHelpMan for check-rpc-mappings linter (MarcoFalke)
faf835680b rpc: [refactor] Use concise C++11 code in CRPCConvertTable constructor (MarcoFalke)

Pull request description:

  Currently, the RPC argument names are specified twice to simplify consistency linting. To avoid having to specify the argnames twice when adding new arguments, remove the linter and add an equivalent test based on RPCHelpMan.

ACKs for top commit:
  laanwj:
    ACK fa04f9b4dd

Tree-SHA512: 3f5f32f5a09b22d879f24aa67031639d2612cff481d6aebc6cfe6fd757cafb3e7bf72120b30466f59292a260747b71e57322c189d5478b668519b9f32fcde31a
2021-01-28 19:25:12 +01:00
MarcoFalke
6ba2ffd28b Merge #21004: contrib: Fix docker args conditional in gitian-build
624091b7b9 Fix docker args conditional (setpill)

Pull request description:

  The conditional that checks if docker needs to be installed has the side effect of triggering the default `lxc` branch in case docker comes preinstalled. This is clearly not intentional.

ACKs for top commit:
  laanwj:
    Code review ACK 624091b7b9
  theStack:
    Code review ACK 624091b7b9

Tree-SHA512: e37e2c35aaed813762223e5963e5416d5865b3fb53efb2aac86daaa03b95ccf07db9c3a779446029d055ab89491147c4d900117273e22caed201b21bdf287c58
2021-01-28 17:42:59 +01:00
MarcoFalke
4d5eaf7a90 Merge #20995: fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION
fad3d7625a fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION (MarcoFalke)
fa99e33aeb fuzz: move-only FillNode implementation to cpp file (MarcoFalke)

Pull request description:

  This fixes a fuzz bug introduced in #20881. Previously the nodes in the fuzz tests had their version initialized to a constant (`PROTOCOL_VERSION`). After #20881, the nodes have their version initialized to an arbitrary signed integer. This is problematic for several reasons:

  * Both `nVersion` and `m_greatest_common_version` may be initialized to `0`. If a `version` message is processed, this leads to a crash, because `m_greatest_common_version` must be `INIT_PROTO_VERSION` while the `version` message is processed. See #20138
  * The "valid" range for `nVersion` is `[MIN_PEER_PROTO_VERSION, std::numeric_limits<int32_t>::max()]` (see check in net_processing)
  * The "valid" range for `m_greatest_common_version` is `std::min(nVersion, PROTOCOL_VERSION)` (see net_processing)

  Fix all issues by initializing `nVersion` and `m_greatest_common_version` to their valid ranges.

  -----

  The crashers, if someone wants to try this at home:

  ```
  ( echo 'dmVyc2lvbgAWFhYWFhYWFhYWFhYWFhYWFhYWFhZp/29uAPX//xYWFhYWFhYWFhYWFhYWFhYWFhYW
  FhYWFhYWaW9uAOr1//8WFhYWFha0ZXJzaW9uAPX//wAAAAAAABAAAAAAAAAAAAC0ZXJzaW9uAPX/
  /wBPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08AAAAAABAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAB2ZXJzaW9uAACDJIO9vXYKAAAAAAAAAAAAAAAAAAAAAAB2ZfS1qmu1qhUVFWs=' | base64 --decode > /tmp/a ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/a
  ```
  ```
  ( echo 'dmVyc2lvbgD//wAhTmiqN///NDcAAACENDL/iv//8DYAAHL///////79/RtcAJqamhqa/QEAAAD/
  ///+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZyq8SaGVhZGVycwAAAAD/NDcAAACENDL/iv//
  8DYAAHL///////79/RtcAJqamhqa/QEAAAD////+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZ
  yq8SaGVhZGVycwAAAADPAQAAACAGIm5GERoLWS1wb3J061u/KMNPOkwFXqZ///b5IgIAAD+5ubkb
  XD5hZGRyAJqamhqasP0BAAAAAAAAAP0BAAAAIf39/R0dHQAAAAAAMgAA///7//+gXqZ///b5IgIA
  AD+5ubm5ubm5AAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAFgAAAAAAAAAAAAlBmv39/f1/f39B
  f39hZGRyAG5vAACaLgAdGzY2zwEAAAAgBiJuRhEaC1ktcG9ydOtbvyjDTzpMBV6mf//2+SICAAA/
  ubm5G1w+YWRkcgCampoamrD9AQAAAAAAAAD9AQAAACH9/f0dHR0AAAAAADIAAP//+///oF6mf//2
  +SICAAA/ubm5ubm5uQAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAABYAAAAAAAAAAAAJQZr9/f39
  f39/QX9/YWRkcgBubwAAmi4AHRs2NjY2NjY2NjYCAgI2NgIA/f39/f39Nv39/TUmABxc' | base64 --decode > /tmp/b ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/b
  ```

ACKs for top commit:
  practicalswift:
    cr ACK fad3d7625a

Tree-SHA512: ea64ee99b94d8e619e3949d2d21252c1236412c0e40f44f2b73595ca70cd2da0bdab005fb1a54f65fb291e7b07fdd33577ce4a3a078ca933246b511ebcb0e52a
2021-01-28 15:13:25 +01:00
MarcoFalke
fabeb5b9c7 fuzz: Disable shuffle when merge=1 2021-01-28 14:38:48 +01:00
MarcoFalke
fa362064e3 rpc: Return total fee in mempool
Also, add missing lock annotations
2021-01-28 10:43:22 +01:00
Jonas Schnelli
68692d33c6 Merge bitcoin-core/gui#85: Remove unused "What's This" button in dialogs on Windows OS
ac7ccd67d7 scripted-diff: Remove unused "What's This" button in dialogs on Windows (Hennadii Stepanov)
b6951483ec qt: Add flags to prevent a "What's This" button on Windows OS (Hennadii Stepanov)

Pull request description:

  Fix #74.

  From [Qt docs](https://doc.qt.io/qt-5/qdialog.html#QDialog):
  > The widget flags _f_ are passed on to the `QWidget` constructor. If, for example, you don't want a **What's This** button in the title bar of the dialog, pass `Qt::WindowTitleHint | Qt::WindowSystemMenuHint` in _f_.

  Screenshot on Windows 10 (2004):
  - master (3ba25e3bdd)
  ![Screenshot from 2020-09-07 16-55-42](https://user-images.githubusercontent.com/32963518/92402384-20dc6a00-f138-11ea-9dcb-3e0f6373ff22.png)

  - this PR (e322fe7e19ac504272d14b9b4f9b28b13df888ed)
  ![Screenshot from 2020-09-07 18-31-16](https://user-images.githubusercontent.com/32963518/92402509-5aad7080-f138-11ea-8b63-9bbbf8b9b9e1.png)

ACKs for top commit:
  Bosch-0:
    tACK ac7ccd67d7 Tested on Windows 10.0.18363 Build 18363.
  promag:
    Code review ACK ac7ccd67d7 but with some suggestions.
  jonasschnelli:
    utACK ac7ccd67d7

Tree-SHA512: f6750a17b7203106cb4db5870becba1cef6a505d4edcc710ba131338bd3aae051510627e62c9bcb8345a7f497c614709e11aeb8f6ae3ea85967bbce2a8c69e64
2021-01-28 10:16:55 +01:00
Jonas Schnelli
7595183543 Merge bitcoin-core/gui#72: util: Log static plugins meta data and used style
957895c715 util: Log static plugins meta data and style (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up of https://github.com/bitcoin/bitcoin/pull/17826, and adds additional info about the imported static plugins and the used style to the `debug.log` I found useful for testing (e.g., with `QT_QPA_PLATFORM`, `QT_QPA_PLATFORMTHEME`, `QT_STYLE_OVERRIDE` variables) and debugging issues (e.g., https://github.com/bitcoin/bitcoin/pull/19716#issuecomment-674052881).

  The excerpt from the log:
  ```
  2020-11-15T18:41:45Z [main] Bitcoin Core version v0.20.99.0-f0b933f78 (release build)
  2020-11-15T18:41:45Z [main] Qt 5.9.8 (static), plugin=xcb (static)
  2020-11-15T18:41:45Z [main] Static plugins:
  2020-11-15T18:41:45Z [main]  QXcbIntegrationPlugin, version 329992
  2020-11-15T18:41:45Z [main] Style: fusion / QFusionStyle
  ...
  ```

ACKs for top commit:
  jarolrod:
    ACK 957895c715, Tested on macOS 11.1
  jonasschnelli:
    utACK 957895c715

Tree-SHA512: 0e46db7560f380fbda8ce5e53faa5d419a456e90ca595ce46be8e3030c99d3a113586edad1988a97e9bf0279e944f975968ed1156817bc16723ed31c64850239
2021-01-28 10:14:32 +01:00
Jonas Schnelli
02b01651c5 Merge bitcoin-core/gui#177: Use "fusion" style on macOS Big Sur with old Qt
4e1154dfd1 qt: Use "fusion" style on macOS Big Sur with old Qt (Hennadii Stepanov)

Pull request description:

  The "macintosh" style is broken on macOS Big Sur:

  - https://github.com/bitcoin/bitcoin/issues/20555#issuecomment-756264648
  - #136

ACKs for top commit:
  MarcoFalke:
    review ACK 4e1154dfd1 can't test
  jarolrod:
    ACK 4e1154dfd1
  jonasschnelli:
    Tested ACK 4e1154dfd1

Tree-SHA512: c2e0f7be220c8b34b182c73e362f41d0e8c8c002e766fcb5491c62f3cfb9f70eabbd32b29baefa152135efc5f83b15534c1c2459e500a586b0f64c5aa8acf614
2021-01-28 10:07:33 +01:00
MarcoFalke
fa04f9b4dd rpc: Remove duplicate name and argNames from CRPCCommand 2021-01-28 08:19:52 +01:00
MarcoFalke
fa92912b4b rpc: Use RPCHelpMan for check-rpc-mappings linter 2021-01-28 08:16:34 +01:00
MarcoFalke
faf835680b rpc: [refactor] Use concise C++11 code in CRPCConvertTable constructor 2021-01-28 08:16:31 +01:00
Anthony Towns
48c8a9b964 net_processing: log txrelay flag from version message 2021-01-28 16:04:04 +10:00
Anthony Towns
98fab37ca0 net: use peer=N instead of from=N in debug log 2021-01-28 16:04:04 +10:00
Anthony Towns
12302105bb net_processing: additional debug logging for ignored messages 2021-01-28 16:04:04 +10:00
Anthony Towns
f7edea3b7c net: make debug logging conditional on -debug=net 2021-01-28 16:04:04 +10:00
Anthony Towns
a410ae8cb0 net, net_processing: log disconnect reasons with -debug=net 2021-01-28 16:04:04 +10:00
Samuel Dobson
9deba2de76 Merge #20226: wallet, rpc: add listdescriptors command
647b81b709 wallet, rpc: add listdescriptors command (Ivan Metlushko)

Pull request description:

  Looking for concept ACKs

  **Rationale**: allow users to inspect the contents of their newly created descriptor wallets.

  Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:
   * add an option to export xprv
   * with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes

  The output is compatible with `importdescriptors` command so it could be easily used for backup/recover purposes.

  **Output example:**
  ```json
  [
    {
      "desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
      "timestamp": 1296688602,
      "active": false,
      "range": [
        0,
        999
      ],
      "next": 0
    }
  ]
  ```

ACKs for top commit:
  jonatack:
    re-ACK 647b81b709 rebased to master, debug builds cleanly, reviewed diff since last review, tested with a descriptor wallet (and with a legacy wallet)
  achow101:
    re-ACK 647b81b

Tree-SHA512: 51a3620bb17c836c52cecb066d4fa9d5ff418af56809046eaee0528c4dc240a4e90fff5711ba96e399c6664e00b9ee8194e33852b1b9e75af18061296e19a8a7
2021-01-28 13:40:18 +13:00
Hennadii Stepanov
01d9586ae8 qt: Save/restore RPCConsole geometry only for window 2021-01-27 22:57:57 +02:00
Ivan Metlushko
647b81b709 wallet, rpc: add listdescriptors command 2021-01-27 21:22:13 +01:00
Wladimir J. van der Laan
d1ddead09a Merge #180: Peer details: connection type follow-ups
79a2576af1 doc: update ConnectionType Doxygen documentation (Jon Atack)
f3153dc08f gui: improve markup handling of connection type tooltip (Jon Atack)
4f09615733 gui: return inbound {full, block} relay type in peer details (Jon Atack)

Pull request description:

  Three follow-ups to #163:
  - return relay type for inbound peers
  - improve markup handling in the tooltip to facilitate translations
  - update ConnectionType doxygen documentation

  ![Screenshot from 2021-01-11 08-37-44](https://user-images.githubusercontent.com/2415484/104156081-50e69300-53e0-11eb-9b0f-880cb5626d68.png)

ACKs for top commit:
  hebasto:
    re-ACK 79a2576af1, only suggested changes since my [previous](https://github.com/bitcoin-core/gui/pull/180#pullrequestreview-564894781) review.
  jarolrod:
    ACK 79a2576af1, tested on macOS 11.1 with Qt 5.15.2
  laanwj:
    Code review ACK 79a2576af1

Tree-SHA512: 4a8d8f8bfbaefd68e8d1bf3b20d29e4a8e8cfe97b2f8d59d3a4c338a50b61de0a67d97bd8646c04bd5df5a9679c4954b9b46e7cba24bb89f4d0e44e94cf9d66c
2021-01-27 20:33:52 +01:00
Andrew Chow
49797c3ccf tests: Disable bdb dump test when no bdb 2021-01-27 12:52:46 -05:00
Andrew Chow
1194cf9269 Fix wallet_send.py wallet setup to work with descriptors
Fixes the wallet setup so this test works with descriptor wallets. Also
enabled explicit descriptor and legacy wallet testing in the test
runner.
2021-01-27 12:52:46 -05:00
Andrew Chow
fbaea7bfe4 Require legacy wallet for wallet_upgradewallet.py 2021-01-27 12:52:46 -05:00
Andrew Chow
b1b679e0ab Explicitly mark legacy wallet tests as such
Some tests are intended to test only legacy wallet behavior. With
automatic switching of wallet type, we need to make them explicit
2021-01-27 12:52:46 -05:00
Andrew Chow
09514e1bef Setup wallets for interface_zmq.py 2021-01-27 12:52:46 -05:00
Andrew Chow
4d03ef9a73 Use MiniWallet in rpc_net.py 2021-01-27 12:52:46 -05:00
Andrew Chow
4de23824b0 Setup wallets for interface_bitcoin_cli.py 2021-01-27 12:52:46 -05:00
Andrew Chow
7c71c627d2 Setup wallets with descriptors for feature_notifications 2021-01-27 12:52:46 -05:00
Andrew Chow
1f1bef8dba Have feature_filelock.py test both bdb and sqlite, depending on compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
c77975abc0 Disable upgrades tests that require BDB if BDB is not compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
1f20cac9d4 Disable wallet_descriptor.py bdb format check if BDB is not compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
3641597d7e tests: Don't make any wallets unless wallet is required 2021-01-27 12:52:46 -05:00
Andrew Chow
b9b88f57a9 Skip legacy wallet reliant tests if BDB is not compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
6f36242389 tests: Set descriptors default based on compilation
Determines whether descriptors should be used based on whether the
--descriptor or --legacy-wallet option is set,
and the compiled support. If no option is set and both BDB and SQLite
are available, it defaults to legacy.

This is used to switch descriptor agnostic tests between descriptors and
legacy wallet.
2021-01-27 12:52:46 -05:00
Wladimir J. van der Laan
15a9df0706 Merge #20964: rpc: Add specific error code for "wallet already loaded"
a6739cc868 rpc: Add specific error code for "wallet already loaded" (Wladimir J. van der Laan)

Pull request description:

  Add a separate RPC error code for "wallet already loaded" to avoid having to match on message to detect this.
  Requested by shesek for rust-bitcoinrpc.

  If concept ACKed needs:
  - [ ]  Release note
  - [x]  A functional test (updated the existing test to make it pass, I think this is enough)

ACKs for top commit:
  jonasschnelli:
    Code Review ACK a6739cc868
  promag:
    Code review ACK a6739cc868.

Tree-SHA512: 9091872e6ea148aec733705d6af330f72a02f23b936b892ac28f9023da7430af6332418048adbee6014305b812316391812039e9180f7f3362d11f206c13b7d0
2021-01-27 13:43:31 +01:00
MarcoFalke
11d3b58336 Merge #20757: doc: tor.md and -onlynet help updates
193f9a9c97 doc: update tor.md manual config, move after automatic config (Jon Atack)
9af99b6f39 doc: update/improve automatic tor section of tor.md (Jon Atack)
dfc4ce1273 doc: update -proxy, -onion and -onlynet info in tor.md (saibato)
784a278e87 doc: update -onlynet help in src/init.cpp (Jon Atack)

Pull request description:

  This continues the tor documentation and help improvements of #19961 and clarifies issues that contributors have been mentioning and noticing, like in https://github.com/bitcoin/bitcoin/issues/20555#issuecomment-750433818.

  More info:

  - https://github.com/bitcoin/bitcoin/pull/19961#discussion_r545051534
  - https://github.com/bitcoin/bitcoin/pull/19961#discussion_r544912964

ACKs for top commit:
  Rspigler:
    ACK 193f9a9c97
  prayank23:
    ACK 193f9a9c97 9af99b6f39 dfc4ce1273

Tree-SHA512: edb1b776c4624e1c2e30d829511c226a6492b719f5d1aaaeee1eaade47c108a99c09004d13a05f70b2d65f36db3db647902b5ea36807a87065f34acade33ccea
2021-01-27 13:21:04 +01:00
Jon Atack
193f9a9c97 doc: update tor.md manual config, move after automatic config 2021-01-26 15:13:29 +01:00
MarcoFalke
faff3991a9 ci: Fuzz with integer sanitizer 2021-01-26 12:50:43 +01:00
MarcoFalke
e130ff38c9 Merge bitcoin-core/gui#183: Add include for std::bind.
2a39ccf133 Add include for std::bind. (sinetek)

Pull request description:

  Hi, this patch adds in <functional> because the GUI code makes use of std::bind.
  That's all.

ACKs for top commit:
  jonasschnelli:
    utACK 2a39ccf133

Tree-SHA512: fb5ac07d9cd5d006182b52857b289a9926362a2f1bfa4f7f1c78a088670e2ccf39ca28214781df82e8de3909fa3e69685fe1124a7e3ead758575839f5f2277a9
2021-01-26 11:14:27 +01:00
Jonas Schnelli
5f65f13597 Merge bitcoin-core/gui#189: qt: drop workaround for QTBUG-42503 which was fixed in Qt 5.5.0
a2a3f4cd8d qt: drop workaround for QTBUG-42503 which was fixed in Qt 5.5.0 (Pavol Rusnak)

Pull request description:

  Fixes https://github.com/bitcoin-core/gui/issues/101

ACKs for top commit:
  jonasschnelli:
    Tested ACK a2a3f4cd8d -

Tree-SHA512: 9ccbc20ff7991ec70cac7d4e4413f9d80a1de453e217d03927a5f167e87eae7f369f0ad437c40c8107e5384c5c5c758659ed0db923837a38f8d705f01f9cb798
2021-01-26 10:41:23 +01:00
Jonas Schnelli
d38e2d9a19 Merge bitcoin-core/gui#186: Add information to "Confirm fee bump" window
232d1f92bb Add information to "Confirm fee bump" window (Prayank)

Pull request description:

  + Add information in bump fee confirmation box according to the documentation: https://bitcoincore.org/en/doc/0.20.0/rpc/wallet/bumpfee/

  + Workaround to fix issue: https://github.com/bitcoin/bitcoin/issues/20795 in which user isn't aware of new inputs, outputs added to replacement transaction before broadcasting. Initial transaction had used coin control features and custom change address. Until the issue is fixed by change in coin selection algorithm we can add this warning.

  + Waiting for comments from devs who are working on coin selection algorithm PRs or involved in related research. However got two comments from Luke Dashjr and Pieter Wuille:

  _luke-jr: Reducing the change output also could be a privacy problem, since it identifies which output was change._

  _sipa: Wallet doesn't know the original transaction was using coin control. So I think its expected that if you use automatic fee bumping, you'll get whatever the coin selection algorithm decides. As for why its not decreasing the change and instead adding another input, that may be a bug._  (IRC: #bitcoin-core-dev)

ACKs for top commit:
  jonasschnelli:
    Tested ACK - 232d1f92bb

Tree-SHA512: 2ff65db1ddb1d4a45f82670b6ca303a0bf48acf3d09defffc21f44ec81cb6182268959706f592f3442aae5db48f43b8ea86973d74ec2721be93d209ce0414953
2021-01-26 10:30:13 +01:00
MarcoFalke
32d44d2b1c Merge #21000: fuzz: Add UBSan suppressions needed for fuzz tests to not warn under -fsanitize=integer
f0f8b1a076 fuzz: Add UBSan suppressions needed for fuzz tests to not warn under -fsanitize=integer (practicalswift)
58232e3ffb fuzz: Avoid -fsanitize=integer warnings in fuzzing harnesses (practicalswift)

Pull request description:

  Add UBSan suppressions needed for fuzz tests to not warn under `-fsanitize=integer`.

  Avoid `-fsanitize=integer` warnings in fuzzing harnesses.

  Suppressed warnings (excluding warnings from `src/crypto/` and `src/test/`):

  ```
  addrman.cpp:306:24: runtime error: implicit conversion from type 'long' of value 5190149478 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 895182182 (32-bit, unsigned)
  addrman.h:446:43: runtime error: implicit conversion from type 'int' of value -22 (32-bit, signed) to type 'const uint8_t' (aka 'const unsigned char') changed the value to 234 (8-bit, unsigned)
  arith_uint256.cpp:32:35: runtime error: left shift of 1712128 by 24 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
  arith_uint256.cpp:47:39: runtime error: left shift of 4294966784 by 31 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
  chain.cpp:151:12: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'unsigned long' changed the value to 18446744073709551615 (64-bit, unsigned)
  coins.cpp:114:22: runtime error: unsigned integer overflow: 0 - 96 cannot be represented in type 'unsigned long'
  compressor.cpp:162:33: runtime error: unsigned integer overflow: 15617702637291228364 * 10 cannot be represented in type 'unsigned long'
  compressor.cpp:188:11: runtime error: unsigned integer overflow: 2265760372865400000 * 10 cannot be represented in type 'unsigned long'
  hash.cpp:13:15: runtime error: left shift of 1692305888 by 15 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
  pubkey.h:152:23: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'
  streams.h:570:31: runtime error: left shift of 350879 by 52 places cannot be represented in type 'uint64_t' (aka 'unsigned long')
  util/bip32.cpp:57:36: runtime error: left shift of 3241096244 by 1 places cannot be represented in type 'unsigned int'
  util/strencodings.cpp:562:38: runtime error: implicit conversion from type 'unsigned char' of value 255 (8-bit, unsigned) to type 'char' changed the value to -1 (8-bit, signed)
  util/strencodings.h:164:24: runtime error: implicit conversion from type 'int' of value -74 (32-bit, signed) to type 'unsigned long' changed the value to 18446744073709551542 (64-bit, unsigned)
  ```

  The warnings above happen here:

  32b191fb66/src/addrman.cpp (L306)
  32b191fb66/src/addrman.h (L446)
  32b191fb66/src/arith_uint256.cpp (L32)
  32b191fb66/src/arith_uint256.cpp (L47)
  32b191fb66/src/chain.cpp (L151)
  32b191fb66/src/coins.cpp (L114)
  32b191fb66/src/compressor.cpp (L162)
  32b191fb66/src/compressor.cpp (L188)
  32b191fb66/src/hash.cpp (L13)
  32b191fb66/src/pubkey.h (L152)
  32b191fb66/src/streams.h (L570)
  32b191fb66/src/util/bip32.cpp (L57)
  32b191fb66/src/util/strencodings.cpp (L562)
  32b191fb66/src/util/strencodings.h (L164)

ACKs for top commit:
  MarcoFalke:
    review ACK f0f8b1a076 🤚

Tree-SHA512: a8f04f7cc055d03653161de1d9d14d106a6280cea1e86a1243abcd57cf8e61dcf5f731d0ab0da5b390790e816022ff7a70759a641463bc7e3303076b8667009f
2021-01-26 10:00:43 +01:00
MarcoFalke
ce75fc36ed Merge #20971: test: Work around libFuzzer deadlock
fa39c8a3e8 test: Work around libFuzzer deadlock (MarcoFalke)

Pull request description:

  Only required part is `symbolize=0`, but the other changes shouldn't hurt

ACKs for top commit:
  practicalswift:
    cr ACK fa39c8a3e8: patch looks correct

Tree-SHA512: 9cddf1de46ad12aea9b8be2c1acb86ba0e07ffdb52f8155d943edf970955551c7cb049a3a6c027846b45dab0dc0966dec42999476ebde50aa761a08dbb751eae
2021-01-26 09:56:24 +01:00
MarcoFalke
cbe439629e Merge bitcoin-core/gui#167: raise helpMessageDialog
77114462f2 raise helpMessageDialog (randymcmillan)

Pull request description:

  the raise() method brings the helpMessageDialog to the top if it is obscured by another window.

ACKs for top commit:
  promag:
    Code review ACK 77114462f2.
  hebasto:
    ACK 77114462f2, tested on:

Tree-SHA512: 0d5b107aa9a5ce3891e88ef69f64461c8b23d17476b798691119e84bfc78e16b2491c798adb5d6cc347af3b7f18729593d7924090c336114a3cf34fbee344bfb
2021-01-26 09:47:52 +01:00
MarcoFalke
280d0bd0bd Merge #21010: refactor: remove straggling boost::mutex usage
f827e151a2 refactor: remove straggling boost::mutex usage (fanquake)

Pull request description:

  After the merge of #18710, the linter is warning:
  ```bash
  A new Boost dependency in the form of "boost/thread/mutex.hpp" appears to have been introduced:
  src/sync.cpp:#include <boost/thread/mutex.hpp>
  src/test/sync_tests.cpp:#include <boost/thread/mutex.hpp>

  ^---- failure generated from test/lint/lint-includes.sh
  ```

  #18710 removed `boost/thread/mutex.hpp` from lint-includes, however in the interim #19337 was merged, which introduced more `boost::mutex` usage.

  Given we no longer use `boost::mutex`, just remove the double lock test and remaining includes.

ACKs for top commit:
  laanwj:
    Code review ACK f827e151a2
  hebasto:
    ACK f827e151a2

Tree-SHA512: f738b12189fe5b39db3e8f8231e9002714413a962eaf98adc84a6614fa474df5616358cfb1c89b92a2b0564efa9b704a774c49d4a25dca18a0ccc3cd9eabfc0a
2021-01-26 09:30:02 +01:00
Prayank
232d1f92bb Add information to "Confirm fee bump" window
Check if "Coin Control features" are enabled to display warning before broadcasting replacement transaction
Workaround to fix issue: bitcoin/bitcoin#20795

Co-authored-by: Jon Atack <jon@atack.com>
2021-01-26 13:51:21 +05:30
fanquake
f827e151a2 refactor: remove straggling boost::mutex usage
After the merge of #18710, the linter is warning:
```bash
A new Boost dependency in the form of "boost/thread/mutex.hpp" appears to have been introduced:
src/sync.cpp:#include <boost/thread/mutex.hpp>
src/test/sync_tests.cpp:#include <boost/thread/mutex.hpp>

^---- failure generated from test/lint/lint-includes.sh
```

the interim #19337 was merged, which introduced more `boost::mutex` usage.

Given we no longer use `boost::mutex`, just remove the double lock test
and remaining includes.
2021-01-26 15:57:28 +08:00
Samuel Dobson
16ae3368f2 Merge #17350: doc: Add developer documentation to isminetype
40f05647ee doc: Add developer documentation to isminetype (HAOYUatHZ)

Pull request description:

  Closes: https://github.com/bitcoin/bitcoin/issues/17217

ACKs for top commit:
  meshcollider:
    utACK 40f05647ee

Tree-SHA512: 156ff3bc02613d65aed5fcf50250ec3f3365b6c83c810763673ecfdd081a1310e5235be05f0c782638f191be61ad0028511392c40e4106a56eb1c6a3a8ab73b9
2021-01-26 13:24:35 +13:00
Samuel Dobson
4b15ffe991 Merge #20832: rpc: Better error messages for invalid addresses
8f0b64fb51 Better error messages for invalid addresses (Bezdrighin)

Pull request description:

  This PR addresses #20809.

  We add more detailed error messages in case an invalid address is provided inside the 'validateaddress' and 'getaddressinfo' RPC calls. This also covers the case when a user provides an address from a wrong network.

  We also add a functional test to test the new error messages.

ACKs for top commit:
  kristapsk:
    ACK 8f0b64fb51
  meshcollider:
    Code review ACK 8f0b64fb51

Tree-SHA512: ca0f806ab573e96b79e98d9f8c810b81fa99c638d9b5e4d99dc18c8bd2568e6a802ec305fdfb2983574a97a19a46fd53b77645f8078fb77e9deb24ad2a22cf93
2021-01-26 13:15:13 +13:00
practicalswift
f0f8b1a076 fuzz: Add UBSan suppressions needed for fuzz tests to not warn under -fsanitize=integer 2021-01-25 20:55:36 +00:00
practicalswift
58232e3ffb fuzz: Avoid -fsanitize=integer warnings in fuzzing harnesses 2021-01-25 20:55:36 +00:00
Jon Atack
9af99b6f39 doc: update/improve automatic tor section of tor.md 2021-01-25 21:31:10 +01:00
saibato
dfc4ce1273 doc: update -proxy, -onion and -onlynet info in tor.md
Improve the description of what these options do with regards to
tor or network traffic.

Some of the wording is from a laanwj review in PR 19358.
2021-01-25 21:31:08 +01:00
Jon Atack
784a278e87 doc: update -onlynet help in src/init.cpp 2021-01-25 21:31:05 +01:00
Wladimir J. van der Laan
52d84a45e2 Merge #20987: doc: add instructions for generating RPC docs
e147af7624 doc: add instructions for generating RPC docs (benk10)

Pull request description:

  Added instructions on how to generate the up to date RPC docs for the bitcoincore.org website in the relevant release-process subsection.

ACKs for top commit:
  theStack:
    ACK e147af7624 ⚙️

Tree-SHA512: 1621d9915ac77f663aa7892237d3e5ff4d9566301c4c0fcb8615804b68d71f6c19bd6e6d84710cd2d2ae492e6b149cdc6edae6292bd20d88d7ca8cd530b4081a
2021-01-25 20:33:23 +01:00
Wladimir J. van der Laan
b386d37360 Merge #18710: Add local thread pool to CCheckQueue
bb6fcc75d1 refactor: Drop boost::thread stuff in CCheckQueue (Hennadii Stepanov)
6784ac471b bench: Use CCheckQueue local thread pool (Hennadii Stepanov)
dba30695fc test: Use CCheckQueue local thread pool (Hennadii Stepanov)
01511776ac Add local thread pool to CCheckQueue (Hennadii Stepanov)
0ef938685b refactor: Use member initializers in CCheckQueue (Hennadii Stepanov)

Pull request description:

  This PR:
  - gets rid of `boost::thread_group` in the `CCheckQueue` class
  - allows thread safety annotation usage in the `CCheckQueue` class
  - is alternative to #14464 (https://github.com/bitcoin/bitcoin/pull/18710#issuecomment-616618525, https://github.com/bitcoin/bitcoin/pull/18710#issuecomment-617291612)

  Also, with this PR (I hope) it could be easier to resurrect a bunch of brilliant ideas from #9938.

  Related: #17307

ACKs for top commit:
  laanwj:
    Code review ACK bb6fcc75d1
  LarryRuane:
    ACK bb6fcc75d1
  jonatack:
    Code review ACK bb6fcc75d1 and verified rebase to master builds cleanly with unit/functional tests green

Tree-SHA512: fddeb720d5a391b48bb4c6fa58ed34ccc3f57862fdb8e641745c021841c8340e35c5126338271446cbd98f40bd5484f27926aa6c3e76fa478ba1efafe72e73c1
2021-01-25 20:21:19 +01:00
randymcmillan
77114462f2 raise helpMessageDialog 2021-01-25 10:57:44 -05:00
setpill
624091b7b9 Fix docker args conditional 2021-01-25 15:33:07 +01:00
MarcoFalke
d0852f39a7 Merge #20998: test: Fix BlockToJsonVerbose benchmark
7487bc9900 Fix BlockToJsonVerbose benchmark (Martin Ankerl)

Pull request description:

  Currently it was not possible to run just the BlockToJsonVerbose benchmark because it did not set up everything it needed, running `bench_bitcoin -filter=BlockToJsonVerbose` caused this assert to fail:

  ```
  bench_bitcoin: chainparams.cpp:506: const CChainParams& Params(): Assertion `globalChainParams' failed.
  ```

  Initializing TestingSetup fixes this.

ACKs for top commit:
  theStack:
    Tested ACK 7487bc9900 🐎

Tree-SHA512: 27b9702cb4bacc0475710f7b31f41844e83b8a0787685380749505d179aba724728604d4e4e2e3b3cb38cde88ab12f170881b5d3eb615872ee84632e85312166
2021-01-25 08:42:53 +01:00
Wladimir J. van der Laan
a6739cc868 rpc: Add specific error code for "wallet already loaded" 2021-01-25 07:55:35 +01:00
Wladimir J. van der Laan
543bf745d3 gitian-linux: Extend noexec-stack workaround to powerpc 2021-01-24 23:33:16 +01:00
Luke Dashjr
00f67c8aa1 gitian-linux: Build binaries for 64-bit POWER 2021-01-24 23:27:39 +01:00
Wladimir J. van der Laan
f707fcde9a Merge #20958: gitian-keys: Add signer aliases, some historical keys
2b4cfb44bd gitian-keys: Add signer aliases, some historical keys (Wladimir J. van der Laan)

Pull request description:

  Update `keys.txt` for `gitian-verify.py`  (bitcoin-core/bitcoin-maintainer-tools#71)

  - Extend line format with signer aliases in parentheses
  - Add a few historical signers
  - Sort the file by name

  Co-authored-by: fanquake <fanquake@gmail.com>

Top commit has no ACKs.

Tree-SHA512: 0bd22161400358fb2507be3f49b0ad79f323fdc280ecb647e1728d11f9d8ddea4a71286e5808036399929ede79c541432191f93f314ccfd12e40aadad56b8faa
2021-01-24 21:48:01 +01:00
Wladimir J. van der Laan
2b4cfb44bd gitian-keys: Add signer aliases, some historical keys
Co-authored-by: fanquake <fanquake@gmail.com>
2021-01-24 19:01:26 +01:00
Wladimir J. van der Laan
e31af1065e Merge #20989: gitian-keys: add CoinForensics key
94271d2ce7 gitian-keys: add CoinForensics key (coinforensics)

Pull request description:

  in ref to:
  bitcoin-core/gitian.sigs#1427

  PGP Signing key: https://coinforensics.keybase.pub/pgp/signing_key.asc

  ```
  -----BEGIN PGP PUBLIC KEY BLOCK-----

  mQENBF/1zL0BCADD1H7gNIVYZ3ibv3HBl8ue5yQKc68DCZk9HAZy9OLRoFLnR0CF
  qwTT8WnjWjvlc/66Ql6kgNhZu3AlEQ4FuTZ9tbYwYeHvw/SBTZkFFlWOyrs5dNJ+
  P+pDHUQQLGyu7oBM/6EpsVVbcXTWXXv3fUI3t0zqhHccCONDi//zFC8vv4hNE3Ps
  mTqxDSU3H88Zkhcel0zgIQXebnAfoExoMcue+A0TZAvk5HC1VCbuacyC6yIfcnqb
  lgOylUc7EA9TNYw52U4hcACyusm+T8wkdjutOtO5j+bFIDKRMkez2MvZZdfr08B4
  FgfUwzlR5lnr3Ezf7tjx9KA4e1300eh3vidNABEBAAG0TENvaW5Gb3JlbnNpY3Mg
  KFNpZ25pbmdLZXkpIDw1OTU2NzI4NCtjb2luZm9yZW5zaWNzQHVzZXJzLm5vcmVw
  bHkuZ2l0aHViLmNvbT6JAVQEEwEIAD4WIQQQFZjcgjwbX5pmJKul4JB6A4DmwwUC
  X/XMvQIbAwUJCWYBgAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAKCRCl4JB6A4Dm
  w4LRCACpLnGtci9ADF6jzt82PcLUHDlL1c8HNs+KeLa3SQ+KF45gmfOPieRZlj5l
  1MNJ0F5Km5DdJA0u39mNINlfdIlPOkxqXnmRAKbVCXVSUF0b8cmIdwYAc70qRpV/
  y6zTgfC8Zs1974/NX95+1GSnw+97MEbR3ULW3mapQXht6gDVCDZXIWVm9WQHzsXQ
  MmAqoynPhKfGAPf75rdgRjpwUS/BUASP+jSAiG5cuEtFljh2sARz2/H3JAAkqHhQ
  6Sqy3rMvNdZ3JVlRI9JOLlVp81PlnQFt39UDV09wa88st/6lTT35xm3LfhcPqSzx
  6KA9zt87hNlRX9sXxVDTusYvwj5EuQENBF/1zL0BCACth5JAULLkySfkVAeGzpbF
  sRkQxS4GJa6EB9qE+nJOijcBVJGwJlyaKHYu1lOTQBiOAz1bJXYk1kiG+CupvzfW
  4gnZRBtF1G/irbEArg2uLvOukKVgusElxUG8iSE7u0B2ET+RPXRPABY75htLS6Id
  xvgZiVAt/w5hRt3jbdmwGjOxdorkdofODvMOKtdi0qqxi16GJSD5EqasBIYpKO8Y
  xKKFJNusHaEgUAXkL82UwcO6YyuSsCckRKJTukNisCza5u2gWNf0hQHbiouEbFz7
  /e7x4Iys9EbOS+N6PnfbyVokbWMN3rSeq56Lci2/DK1YC+PFaxU8zydg4rqscwab
  ABEBAAGJATwEGAEIACYWIQQQFZjcgjwbX5pmJKul4JB6A4DmwwUCX/XMvQIbDAUJ
  CWYBgAAKCRCl4JB6A4Dmw+DfCADBX9/1ABB6KIYOMfd9/OzUI03hXQruZMz3CT6H
  RVhpmjvZAc210kVGW8X7Y50op4NQbdgheQgCTft/iteLv7JIPo8UauBls0G90Kgm
  aCFa4z9y3cMF+tkylgubuPVaOVh+OCcF9ZBzPQeQViMN0u/K1whhs48paFfG6tT4
  g7gO11PnRnCwIFGbwULp+yWInG1F2Hosbt03dzHQdPRWMzvo4Cb+gzIYES4jb0uQ
  AganiOKOOpLUgQ/9Iw1hEvqqZ/cjFR+lCztLrXUyLDzGo636C0Q3LYLoMEudk3Cx
  9mf4vHJoGKqk4Op9ATBaN8ET/F99AcQJRz7U3wL/88WDSRND
  =6SYc
  -----END PGP PUBLIC KEY BLOCK-----
  ```

ACKs for top commit:
  laanwj:
    ACK 94271d2ce7

Tree-SHA512: 9e6e26b145d00b5835d1a52d169a13e0c2cbcaea64d3c39130e430ef446db3b0f4bec514b6b97e5aa78dd9898c14e55ed77d4bc95cd9aacf097ed61479d6ecab
2021-01-24 18:57:33 +01:00
Fabian Jahr
a1fcceac69 refactor: Improve encapsulation between MuHash3072 and Num3072
Also fixes a typo.
2021-01-24 16:28:27 +01:00
Martin Ankerl
7487bc9900 Fix BlockToJsonVerbose benchmark
Currently it was not possible to run just the BlockToJsonVerboes benchmarsk because it did not set up everything it needed, running `bench_bitcoin -filter=BlockToJsonVerbose` caused this assert to fail:

```
bench_bitcoin: chainparams.cpp:506: const CChainParams& Params(): Assertion `globalChainParams' failed.
```

Initializing TestingSetup fixes this.
2021-01-24 10:31:13 +01:00
Bezdrighin
8f0b64fb51 Better error messages for invalid addresses
This commit addresses #20809.

We add an additional 'error' property in the result of 'validateaddress' in case the address is not valid that gives a short description of why the address in invalid. We also change the error message returned by 'getaddressinfo' in case the address is invalid.
2021-01-24 02:44:53 +01:00
Troy Giorshev
bff7c66e67 Add documentation to contrib folder
This commit adds brief documentation for this feature.  Included in the
justification is the purpose of this feature as well as usage and
functionality tips.
2021-01-23 16:15:05 -05:00
Troy Giorshev
381f77be85 Add Message Capture Test
Add a functional test for CaptureMessage.  This connects and then
disconnects a peer so that the handshake can be used to check if capture
is being done correctly.

Included in a docstring in the test is the following:

From the data file we'll only check the structure.

We won't care about things like:
- Deserializing the payload of the message
    - This is managed by the deserialize methods in
      test_framework.messages
- The order of the messages
    - There's no reason why we can't, say, change the order of the
      messages in the handshake
- Message Type
    - We can add new message types

We're ignoring these because they're simply too brittle to test here.
2021-01-23 16:15:05 -05:00
Troy Giorshev
e4f378a505 Add capture parser
This commit adds contrib/message-capture/message-capture-parser.py, a python
script to be used alongside -capturemessages to parse the captured
messages.

It is complete with arguments and will parse any file given, sorting the
messages in the files when creating the output.  If an output file is
specified with -o or --output, it will dump the messages in json format
to that file, otherwise it will print to stdout.

The small change to the unused msg_generic is to bring it in line with
the other message classes, purely to avoid a bug in the future.
2021-01-23 16:01:39 -05:00
Troy Giorshev
4d1a582549 Call CaptureMessage at appropriate locations
These calls are toggled by a debug-only "capturemessages" flag.  Default
disabled.
2021-01-23 15:58:42 -05:00
Troy Giorshev
f2a77ff97b Add CaptureMessage
This commit adds the CaptureMessage function.  This will later be called
when any message is sent or received.  The capture directory is fixed,
in a new folder "message_capture" in the datadir.  Peers will then have
their own subfolders, named with their IP address and port, replacing
colons with underscores to keep compatibility with Windows.  Inside,
received and sent messages will be captured into two binary files,
msgs_recv.dat and msgs_sent.dat.

e.g.
message_capture/203.0.113.7_56072/msgs_recv.dat
message_capture/203.0.113.7_56072/msgs_sent.dat

The format has been designed as to result in a minimal performance
impact.  A parsing script is added in a later commit.
2021-01-23 15:58:42 -05:00
MarcoFalke
fad3d7625a fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION 2021-01-23 20:04:37 +01:00
MarcoFalke
fa99e33aeb fuzz: move-only FillNode implementation to cpp file
This allows to modify the implementation without having to recompile all
fuzz targets.

Can be reviewed with --color-moved=dimmed-zebra
2021-01-23 20:01:12 +01:00
Sebastian Falbesoner
de85af5cce test: store subversion (user agent) as string in msg_version 2021-01-23 15:04:35 +01:00
benk10
e147af7624 doc: add instructions for generating RPC docs
Added instructions on how to generate the up to date RPC docs for the bitcoincore.org website in the relevant release-process subsection.
2021-01-23 10:01:31 +02:00
coinforensics
94271d2ce7 gitian-keys: add CoinForensics key 2021-01-22 18:36:45 +01:00
MarcoFalke
32b191fb66 Merge #20927: [refactor] [net] Clean up InactivityCheck()
bf100f8170 [net] Cleanup InactivityChecks() and add commenting about time (John Newbery)
06fa85cd50 [net] InactivityCheck() takes a CNode reference (John Newbery)

Pull request description:

  This is a pure refactor and should not change any behavior. It clarifies and documents the InactivityCheck() function

  This makes #20721 easier to review. In particular, this function uses a mixture of (unmockable) system time and mockable time. It's important to understand where those are being used when reviewing #20721.

  #20721 doesn't require this change, so if others don't agree that it's useful and makes review easier, then I'm happy to close this and just do #20721 directly.

ACKs for top commit:
  fanquake:
    ACK bf100f8170
  MarcoFalke:
    review ACK bf100f8170 💫

Tree-SHA512: 7b001de2a5fbe8a6dc37baeae930db5775290afb2e8a6aecdf13161f1e5b06ef813bc6291d8ee5cefcf1e430c955ea702833a8db84192eebe6e6acf0b9304cb2
2021-01-22 13:13:00 +01:00
fanquake
b7e12b350d Merge #20985: doc: add xorriso to macOS depends packages
5b41d84b34 doc: add xorriso to macOS depends packages (fanquake)

Pull request description:

  This was missed in #20470.

ACKs for top commit:
  hebasto:
    ACK 5b41d84b34, tested on Linux Mint 20.1 (x86_64).

Tree-SHA512: bcfd8468a099c69175f8a9d295c1466764ab25d6a61121b28675a09c3e96f45b6309e1523d341f4cb21d0ddee4945f00ba060ba02da835f2f0db7e694fd6c44b
2021-01-22 18:25:20 +08:00
John Newbery
aa929abf8d [docs] Update developer notes to discourage very long lines 2021-01-22 09:55:13 +00:00
fanquake
5b41d84b34 doc: add xorriso to macOS depends packages
This was missed in #20470.
2021-01-22 16:22:11 +08:00
fanquake
4bd586607d Merge #20523: zmq: deduplicate 'sequence' publisher message creation/sending
962444295d zmq: deduplicate 'sequence' publisher message creation/sending (Sebastian Falbesoner)

Pull request description:

  This small PR deduplicates common low-level creation and sending code for the 'sequence' zmq publisher message (methods `NotifyBlock{Connect,Disconnect}()`, `NotifyTransaction{Acceptance,Removal}()` in the class `CZMQPublishSequenceNotifier`) by introducing a helper function.

ACKs for top commit:
  jonatack:
    Code review re-ACK 962444295d per `git diff f231ffd 9624442`
  instagibbs:
    utACK 962444295d

Tree-SHA512: de0750d923f36d1a5751331e88eec8a1605cb88c93318830913210485e2bff712310484f18a0fb626df6ef32ce0b0cf57f4421ce62656e05fce7011a0c3c2d0e
2021-01-22 12:58:47 +08:00
fanquake
019aa248d9 Merge #17920: guix: Build support for macOS
f1694757dd guix: Fix typo (Carl Dong)
771c4b98a8 guix: README: Add darwin HOSTS entry (Carl Dong)
8dbf18cb1d guix: Check for macOS SDK before building anything (Carl Dong)
34b23f597e guix: Set ZERO_AR_DATE for darwin build determinism (Carl Dong)
f3835dc6a3 build: Make xorrisofs reproducible with -volume_date (Carl Dong)
c9eb4cf3a0 guix: Add support for darwin builds (Carl Dong)
37fe73a092 build: Add var printing target to src/Makefile.am (Carl Dong)

Pull request description:

  This PR brings our Guix builds on par with Gitian in terms of supported architectures.

  Reviewers: if you run a build, please submit:

  ```
  find output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  ```

  So that we can compare hashes and ensure reproducibility!

ACKs for top commit:
  fanquake:
    ACK f1694757dd - I think we can make some small usability improvements, but this is ok to merge now.

Tree-SHA512: 4af2b71654a9736467dcc681d10601c6eee37800d7847011a50585455b67b55d61742ca5604585f310a2fd75335b674e5e27dfb5169cb2f26e112aa4c411d8be
2021-01-22 12:16:57 +08:00
fanquake
d6c3c43ef8 Merge #20983: Fix MSVC build after gui#176
c5354e4641 Fix MSVC build after gui#176 (Hennadii Stepanov)

Pull request description:

  Close #20982.

ACKs for top commit:
  fanquake:
    ACK c5354e4641

Tree-SHA512: 548ae0616fad0373950fa842d7009dbf285882e8317767c699903eeca90a90cefb7a03674ddeced9a4ac8c25f3c35dfed3a6d6da1aeb6609feb19a83b1b170ee
2021-01-22 11:26:56 +08:00
Hennadii Stepanov
c5354e4641 Fix MSVC build after gui#176 2021-01-21 23:09:17 +02:00
MarcoFalke
fa61b9d1a6 util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet
Co-Authored-by: Anthony Towns <aj@erisian.com.au>
2021-01-21 19:31:28 +01:00
MarcoFalke
7777105a24 refactor: Move all command dependend checks to ExecuteWalletToolFunc 2021-01-21 19:30:34 +01:00
MarcoFalke
fa06bce4ac test: Add tests 2021-01-21 19:29:58 +01:00
MarcoFalke
fac05ccdad wallet: [refactor] Pass ArgsManager to WalletAppInit 2021-01-21 19:29:29 +01:00
MarcoFalke
7f653c3b22 Merge bitcoin-core/gui#176: Fix TxViewDelegate layout
af58f5b12c qt: Stop the effect of hidden widgets on the size of QStackedWidget (Hennadii Stepanov)
f0d04795e2 qt: Fix TxViewDelegate layout (Hennadii Stepanov)
d439921406 qt: Add TransactionOverviewWidget class (Hennadii Stepanov)

Pull request description:

  This change:
  - prevents overlapping date and amount strings
  - guaranties that "eye" sign at the end of the watch-only address/label is always visible

  Fix https://github.com/bitcoin/bitcoin/issues/20826

  Here are some screenshots with this PR with the _minimum available width_ of the transaction list widget:

  ![Screenshot from 2021-01-03 20-23-56](https://user-images.githubusercontent.com/32963518/103486411-6408ca00-4e06-11eb-9c21-627a65e532c1.png)
  ![Screenshot from 2021-01-03 20-24-47](https://user-images.githubusercontent.com/32963518/103486413-6834e780-4e06-11eb-8221-478d98bbdf69.png)
  ![Screenshot from 2021-01-03 20-25-27](https://user-images.githubusercontent.com/32963518/103486418-6d923200-4e06-11eb-8625-a4ed3089b6ab.png)
  ![Screenshot from 2021-01-03 20-33-20](https://user-images.githubusercontent.com/32963518/103486420-708d2280-4e06-11eb-90c2-f2463fb3c4b3.png)

ACKs for top commit:
  dooglus:
    ACK af58f5b.
  jarolrod:
    re-ACK af58f5b12c

Tree-SHA512: 6dae682490ec50fa0335d220bc2d153fa3e6ed578f07c6353a3b180f8f6cf1c2f9e52ebd7b3076f51d7004d86bf5cca14e6b5db9cdf786e85a57a81eacbb4988
2021-01-21 18:54:19 +01:00
MarcoFalke
53bbbe5a20 Merge bitcoin-core/gui#171: Use layout manager for Create Wallet dialog
d4feb6812a qt: Use layout manager for Create Wallet dialog (Hennadii Stepanov)

Pull request description:

  On master (e75f91eae3) not using layout manager causes problems with resizing:

  ![Screenshot from 2021-01-01 13-03-13](https://user-images.githubusercontent.com/32963518/103437728-ce1d4580-4c33-11eb-8915-1e9482775653.png)
  ![Screenshot from 2021-01-01 13-03-26](https://user-images.githubusercontent.com/32963518/103437730-d6758080-4c33-11eb-9e0f-87d0dd487fcb.png)

  Also text labels are not resized properly on some window managers (https://github.com/bitcoin/bitcoin/issues/20777), or if their lengths are changed (after translation).

  This PR introduces a standard layout manager for the "Create Wallet" dialog that fixes all layout issues (actually, the `createwalletdialog.ui` has been re-written from scratch):

  ![Screenshot from 2021-01-01 13-10-03](https://user-images.githubusercontent.com/32963518/103437822-d0cc6a80-4c34-11eb-84fd-fcb10a16d9ef.png)
  ![Screenshot from 2021-01-06 23-50-36](https://user-images.githubusercontent.com/32963518/103823090-0b416780-507a-11eb-89dd-3f48a358e168.png)

  Additional visual changes:
  - advanced options are grouped in `QGroupBox` (https://github.com/bitcoin-core/gui/pull/96#issuecomment-726337165)
  - enabled the [size grip](https://doc.qt.io/qt-5/qsizegrip.html#details)

  Fix https://github.com/bitcoin/bitcoin/issues/20777

ACKs for top commit:
  jarolrod:
    ACK d4feb6812a
  Sjors:
    re-tACK d4feb6812a
  promag:
    Tested ACK d4feb6812a on macos.

Tree-SHA512: 4c055962e49f88624900b880b33a866976d224628784593428b712d2e94563d77ddefddea3397134d20e72f738a8cf9aa885c1272fd9ffc90213c104435fb9f4
2021-01-21 18:51:17 +01:00
MarcoFalke
45952dab9d Merge #20932: refactor: Replace fs::absolute calls with AbsPathJoin calls
da9caa1ced Replace fs::absolute calls with AbsPathJoin calls (Kiminuo)
66576c4fd5 test: Clear forced -walletdir setting after wallet init_tests (Kiminuo)

Pull request description:

  This adds better test coverage and will make it easier in #20744 to remove our dependency on the two-argument boost::filesystem::absolute() function which does not have a direct equivalent in C++17.

  This PR doesn't change behavior aside from adding an assert and fixing a test bug.

ACKs for top commit:
  jonatack:
    Code review ACK da9caa1ced only doxygen improvements since my last review per `git diff d867d7a da9caa1`
  MarcoFalke:
    review ACK da9caa1ced 📯
  ryanofsky:
    Code review ACK da9caa1ced. Just comment and test tweaks since previous review.

Tree-SHA512: c940ee60f3ba374d4927cf34cf12d27c4c735c94af591fbc0ca408c641b30f8f8fbcfe521d66bfbddf9877a1fc8cd99bd8a47ebcd2fa59789de6bd87a7b9cf4d
2021-01-21 18:48:03 +01:00
Carl Dong
f1694757dd guix: Fix typo 2021-01-21 10:58:12 -05:00
Carl Dong
771c4b98a8 guix: README: Add darwin HOSTS entry 2021-01-21 10:58:12 -05:00
Carl Dong
8dbf18cb1d guix: Check for macOS SDK before building anything 2021-01-21 10:58:12 -05:00
Carl Dong
34b23f597e guix: Set ZERO_AR_DATE for darwin build determinism
See comments inserted in this commit.
2021-01-21 10:58:12 -05:00
Carl Dong
f3835dc6a3 build: Make xorrisofs reproducible with -volume_date
We need this to be after a '--' as '-volume_date' is a xorriso
flag, not a xorrisofs flag. See the respective man pages.

For more details: https://issues.guix.info/issue/35283#2
2021-01-21 10:58:12 -05:00
Carl Dong
c9eb4cf3a0 guix: Add support for darwin builds 2021-01-21 10:58:08 -05:00
Carl Dong
37fe73a092 build: Add var printing target to src/Makefile.am
See 181989f6c9 for more info. I missed
this one last time.
2021-01-21 10:57:31 -05:00
MarcoFalke
11cbd4bb54 Merge #17556: test: Change feature_config_args.py not to rely on strange regtest=0 behavior
ff44cae279 test: Change feature_config_args.py not to rely on strange regtest=0 behavior (Russell Yanofsky)

Pull request description:

  Update test to simply generate a normal mainnet configuration file instead of using a crazy setup where a regtest=1 config file using an includeconf in the [regtest] section includes another config file that specifies regtest=0, retroactively switching the network to mainnet.

  This setup was fragile and only worked because the triggered InitError happened early enough that none of the ignored [regtest] options mattered (only affecting log output).

  This change was originally made as part of #17493

Top commit has no ACKs.

Tree-SHA512: 3f77305454f04438493dfc2abd78a00434b30869454d1c3f54587b9c1f63239c49c90fb3b4d3a777ad130f2184e0f2dac87cee4cd23c50f1b3496a375943da01
2021-01-21 16:51:19 +01:00
MarcoFalke
1f45e85509 Merge #20972: locks: Annotate CTxMemPool::check to require cs_main
b396467053 locks: Annotate CTxMemPool::check to require cs_main (Carl Dong)

Pull request description:

  ```
  Currently, CTxMemPool::check locks CTxMemPool's own cs member, then
  calls GetSpendHeight which locks cs_main. This can potentially cause an
  undesirable lock invesion since CTxMemPool's cs is supposed to be locked
  after cs_main.

  This does not cause us any problems right now because all callers of
  CTxMemPool already lock cs_main before calling CTxMemPool::check, which
  means that the LOCK(cs_main) in GetSpendHeight becomes benign.

  However, it is currently possible for new code to be added which calls
  CTxMemPool::check without locking cs_main (which would be dangerous).
  Therefore we should make it explicit that cs_main needs to be held
  before calling CTxMemPool::check.

  NOTE: After all review-only assertions are removed in "#20158 |
        tree-wide: De-globalize ChainstateManager", and assuming that we
        keep the changes in "validation: Pass in spendheight to
        CTxMemPool::check", we can re-evaluate to see if this annotation
        is still necessary.
  ```
  -----

  Previous discussions:
  1. https://github.com/bitcoin/bitcoin/pull/20158#discussion_r520639845
  2. https://github.com/bitcoin/bitcoin/pull/20158#pullrequestreview-557117202
  3. https://github.com/bitcoin/bitcoin/pull/20749#discussion_r559425521

ACKs for top commit:
  jnewbery:
    Code review ACK b396467053
  MarcoFalke:
    ACK b396467053
  jonatack:
    ACK b396467053 review and debug built, verified that `cs_main` is held by callers of `CTxMemPool::check()` in `PeerManagerImpl::ProcessOrphanTx()`, `PeerManagerImpl::ProcessMessage()`, and `CChainState::ActivateBestChainStep()`

Tree-SHA512: 4635cddb4aa1af9532bb657b2f9c4deec4568d16ba28c574eae91bb77368cd40e23c3c720a9de11cec78e7ad678a44a5e25af67f13214b86b56e777e0c35a026
2021-01-21 16:45:45 +01:00
MarcoFalke
85fee49c39 Merge #20946: fuzz: Consolidate fuzzing TestingSetup initialization
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate (Carl Dong)
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization (Carl Dong)

Pull request description:

  ```
  Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:

  1. Calling InitializeFuzzingContext, which implicitly constructs a static
     const BasicTestingSetup
  2. Directly constructing a static const BasicTestingSetup in the initialize_*
     function
  3. Directly constructing a static TestingSetup and reproducing the
     initialization arguments (I'm assuming because
     InitializeFuzzingContext only initializes a BasicTestingSetup)

  The new, relatively-simple MakeFuzzingContext function allows us to
  consolidate these methods of initialization by being flexible enough to
  be used in all situations. It:

  1. Is templated so that we can choose to initialize any of
     the *TestingSetup classes
  2. Has sane defaults which are often used in fuzzers but are also
     easily overridable
  3. Returns a unique_ptr, explicitly transferring ownership to the caller
     to deal with according to its situation
  ```

  ~~Question for fuzzing people: was it intentional that `src/test/fuzz/net.cpp` would directly instantiate the `BasicTestingSetup` and thus omit the `"-nodebuglogfile"` flag?~~ [Answered](https://github.com/bitcoin/bitcoin/pull/20946#issuecomment-761537108)

ACKs for top commit:
  MarcoFalke:
    ACK abb6fa7285

Tree-SHA512: 96a5ca6f4cd5ea0e9483b60165b31ae3e9003918c700a7f6ade48010f419f2a6312e10b816b3187f1d263798827571866e4c4ac0bbfb2e0c79dfad254cda68e7
2021-01-21 16:04:31 +01:00
Carl Dong
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate
A full TestingSetup is required for both coins_view and
load_external_block_file as they interact with the active chainstate.
2021-01-21 09:29:42 -05:00
Carl Dong
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization
Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:

1. Calling InitializeFuzzingContext, which implicitly constructs a static
   const BasicTestingSetup
2. Directly constructing a static const BasicTestingSetup in the initialize_*
   function
3. Directly constructing a static TestingSetup and reproducing the
   initialization arguments (I'm assuming because
   InitializeFuzzingContext only initializes a BasicTestingSetup)

The new, relatively-simple MakeFuzzingContext function allows us to
consolidate these methods of initialization by being flexible enough to
be used in all situations. It:

1. Is templated so that we can choose to initialize any of
   the *TestingSetup classes
2. Has sane defaults which are often used in fuzzers but are also
   easily overridable
3. Returns a unique_ptr, explicitly transferring ownership to the caller
   to deal with according to its situation
2021-01-21 09:29:42 -05:00
Sebastian Falbesoner
962444295d zmq: deduplicate 'sequence' publisher message creation/sending 2021-01-21 14:35:21 +01:00
fanquake
3734adba39 Merge #20953: test: dedup zmq test setup code (node restart, topics subscription)
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) (Sebastian Falbesoner)

Pull request description:

  This PR deduplicates common setup code for the ZMQ functional test. The following steps, previously duplicated in each sub-test, are put into a new method `setup_zmq_test(...)`:
  - create subscriber sockets (`zmq.SUB`) for each topic with the specified timeout (default 60s)
  - restart node0 with specified zmq notifications enabled (`-zmqpub...=tcp://127.0.0.1:...`...)
  - if desired, connect node0 with node1 (note done by default)
  - connect all susbcriber sockets to publisher (running on node0)
  - wait a bit (currently 200ms), to _"Relax so that the subscribers are ready before publishing zmq messages"_

  Note that the last point should be repaced by a more robust method, as this test is still flaky, see #20934 (also #20590 and #20538).

ACKs for top commit:
  instagibbs:
    ACK 4efb6c2d3b
  laanwj:
    Code review ACK 4efb6c2d3b

Tree-SHA512: d49626756a9c669f1133f1b73ce273994b58c760ce0d6a4bdaa384f043a74149dc2b9fa66fe990413d9105f9c3b6ea973e099669e8e02f2902a5b84fa995028c
2021-01-21 16:38:06 +08:00
MarcoFalke
fa39c8a3e8 test: Work around libFuzzer deadlock 2021-01-21 08:30:49 +01:00
Carl Dong
b396467053 locks: Annotate CTxMemPool::check to require cs_main
Currently, CTxMemPool::check locks CTxMemPool's own cs member, then
calls GetSpendHeight which locks cs_main. This can potentially cause an
undesirable lock invesion since CTxMemPool's cs is supposed to be locked
after cs_main.

This does not cause us any problems right now because all callers of
CTxMemPool already lock cs_main before calling CTxMemPool::check, which
means that the LOCK(cs_main) in GetSpendHeight becomes benign.

However, it is currently possible for new code to be added which calls
CTxMemPool::check without locking cs_main (which would be dangerous).
Therefore we should make it explicit that cs_main needs to be held
before calling CTxMemPool::check.

NOTE: After all review-only assertions are removed in "#20158 |
      tree-wide: De-globalize ChainstateManager", and assuming that we
      keep the changes in "validation: Pass in spendheight to
      CTxMemPool::check", we can re-evaluate to see if this annotation
      is still necessary.
2021-01-20 16:15:03 -05:00
Samuel Dobson
80486e7e2d Merge #20952: wallet: Add BerkeleyDB version sanity check at init time
ad57fb756b wallet: Add BerkeleyDB version sanity check at init time (Wladimir J. van der Laan)

Pull request description:

  Detect version conflicts between the run-time BerkeleyDB library and the one used during compilation.

  This is very unsafe (can result in anything from crashes to corruption) so shut down when one is detected.

ACKs for top commit:
  decryp2kanon:
    utACK ad57fb7
  achow101:
    ACK ad57fb756b
  theStack:
    utACK ad57fb756b
  meshcollider:
    Code review ACK ad57fb756b

Tree-SHA512: 99cd7d836bffbdeb3d4e14053f7139cc85a6d42e631a3f9a3058a848042446b364faee127500f5acb374616e6a61ab2bedebfac1ba9bc993b4d6227114c2a6c2
2021-01-20 16:51:42 +13:00
Sebastian Falbesoner
233a886b42 test: check that getblockfilter RPC fails without block filter index
If a node was started without compact block filter index, i.e. parameter
`--blockfilterindex=0`, the `getblockfilter` RPC call should fail.
2021-01-20 01:55:24 +01:00
fanquake
977bec1d93 Merge #20937: guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH
1fca9811e1 lint: Skip whitespace lint for guix patches (Carl Dong)
a91c46c57d guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH (Carl Dong)

Pull request description:

  ```
  When building nsis, if VERSION is not specified, it defaults to
  cvs_version which is non-deterministic as it includes the current date.

  This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
  nsis is reproducible.

  Upstream change: https://github.com/kichik/nsis/pull/13
  ```

  Sidenote: also a good demonstration of how Guix allows us to flexibly patch our tools!

  Note to reviewers: if you want to compare hashes, please build after Jan 16th 2021 without my substitute server enabled!

ACKs for top commit:
  fanquake:
    ACK 1fca9811e1

Tree-SHA512: b800e0ce5f73827ad353739effb9167ec3a6bdb362c725ae20dd3f025ce78660f85c70ce1d75cd0896facf1e8fe38a9e058459ed13dec71ab3a2fe41e20eaa5d
2021-01-20 07:43:44 +08:00
Wladimir J. van der Laan
bc51b99bd5 Merge #20891: rpc: Remove deprecated bumpfee behavior
ea0a7ec949 Remove deprecated bumpfee behavior (Andrew Chow)

Pull request description:

  Removes the deprecation message, behavior, and test.

  This was marked for removal in 22.0.

ACKs for top commit:
  promag:
    ACK ea0a7ec949, maybe add need release notes tag.

Tree-SHA512: d1626906849f6ee37213c32e5f8c1433ad8fb7beabcd88f5801b1964b322171a2341bdfbd9a3a5ab39b2fd9d9c6a05f73298583423a73cab1275653105c03e8e
2021-01-19 17:33:18 +01:00
Wladimir J. van der Laan
dd545c53a5 Merge #20906: contrib: embed C++11 patch in install_db4.sh
92370033a2 contrib: embed C++11 patch in install_db4.sh (jackielove4u)

Pull request description:

  This is a continuation of https://github.com/bitcoin/bitcoin/pull/20665.

  Closes #20722.

ACKs for top commit:
  laanwj:
    ACK 92370033a2
  fanquake:
    ACK 92370033a2.

Tree-SHA512: ebfd16f5301158de1acc1b8eeca43b3d94f0a6d438832133a30648e5e8a88268b4af983be0bb57f3018e3af8459f32f0de676c1b4e8942e199a4497c776631c5
2021-01-19 17:30:16 +01:00
Pavol Rusnak
a2a3f4cd8d qt: drop workaround for QTBUG-42503 which was fixed in Qt 5.5.0
Fixes https://github.com/bitcoin-core/gui/issues/101
2021-01-19 16:27:58 +01:00
Luke Dashjr
63fc2b1782 gitian: Properly quote arguments in wrappers 2021-01-19 14:41:22 +01:00
Luke Dashjr
798bc0b29a Support glibc-back-compat on 64-bit POWER 2021-01-19 14:41:22 +01:00
HAOYUatHZ
40f05647ee doc: Add developer documentation to isminetype 2021-01-19 19:04:45 +08:00
John Newbery
bf100f8170 [net] Cleanup InactivityChecks() and add commenting about time
Also clean up and better comment the function. InactivityChecks() uses a
mixture of (non-mockable) system time and mockable time. Make sure
that's well documented.

Despite being marked as const in CConnman before this commit, the
function did mutate the state of the passed in CNode, which is contained
in vNodes, which is a member of CConnman. To make the function truly
const in CConnman and all its data, instead make InactivityChecks() a
pure function, return whether the peer should be disconnected, and let
the calling function (SocketHandler()) update the CNode object. Also
make the CNode& argument const.
2021-01-19 10:50:36 +00:00
Andrew Chow
cc3971c9ff GUI: Write PSBTs to file with binary mode 2021-01-18 20:55:08 -05:00
Wladimir J. van der Laan
43f3ada27b Merge #19866: eBPF Linux tracepoints
22eb7930a6 tracing: add tracing framework (William Casarin)
933ab8a720 build: detect sys/sdt.h for eBPF tracing (William Casarin)

Pull request description:

  Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin 😅 )

  The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages:

  # Examples

  ## Network Messages

  ```
  #!/usr/bin/env bpftrace

  BEGIN
  {
    printf("bitcoin net msgs\n");
    @start = nsecs;
  }

  usdt:./src/bitcoind:net:push_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @outbound[$command]++;
  }

  usdt:./src/bitcoind:net:process_message
  {
    $ip = str(arg0);
    $peer_id = (int64)arg1;
    $command = str(arg2);
    $data_len = arg3;
    $data = buf(arg3,arg4);
    $t = (nsecs - @start) / 100000;

    printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);

    @inbound[$ip, $command]++;
  }

  ```

      $ sudo bpftrace netmsg.bt

  output: https://jb55.com/s/b11312484b601fb3.txt

  if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice!

  ## IBD Benchmarking

  ```
  #!/usr/bin/env bpftrace
  BEGIN
  {
    printf("IBD to 500,000 bench\n");
  }

  usdt:./src/bitcoind:CChainState:ConnectBlock
  {
    $height = (uint32)arg0;

    if ($height == 1) {
      printf("block 1 found, starting benchmark\n");
      @start = nsecs;
    }

    if ($height >= 500000) {
      @end = nsecs;
      @duration = @end - @start;
      exit();
    }
  }

  END {
    printf("duration %d ms\n", @duration / 1000000)
  }
  ```
  This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock

  Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging.

  Other ideas: tracepoints for lock contention, threads, what else?

  Let me know what ya'll think and if this is worth adding to bitcoin.

  ## TODO

  - [ ] docs?
  - [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header)
  - [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now

ACKs for top commit:
  laanwj:
    Tested ACK 22eb7930a6
  0xB10C:
    Tested ACK 22eb7930a6

Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
2021-01-18 22:09:05 +01:00
Wladimir J. van der Laan
f7fd76bcc0 Merge #20880: gitian: Use custom MacOS code signing tool
2c403279e2 gitian: Remove codesign_allocate and pagestuff from MacOS build (Andrew Chow)
f55eed2514 gitian: use signapple to create the MacOS code signature (Andrew Chow)
95b06d2185 gitian: use signapple to apply the MacOS code signature (Andrew Chow)
42bb1ea363 gitian: install signapple in gitian-osx-signer.yml (Andrew Chow)

Pull request description:

  The MacOS code signing issues that were encountered during the 0.21.0 release cycle have shown that it is necessary for us to use a code signing tool for which the source code is available and modifiable by us. Given that there appears to not be such a tool available, I have written such a tool, [signapple](https://github.com/achow101/signapple), that we can use. This tool is able to create a valid MacOS code signature, detach it in a way that we were doing previously, and attach it to the unsigned binary. This tool can also verify that the signature is correct.

  This PR implements the usage of that tool in the gitian build for the code signed MacOS binary. The code signer will use this tool to create the detached signature. Gitian builders will use this tool to apply the detached signature. The `gitian-osx-signer.yml` descriptor has been modified to install this tool so that the `detached-sig-apply.sh` script can use it. Additionally, the `codesign_allocate` and `pagestuff` tools are no longer necessary so they are no longer added to the tarball used in code signing. Lastly, both the `detached-sig-create.sh` and `detached-sig-apply.sh` scripts are made to be significantly less complex and to not do unexpected things such as unpacking an already unpacked tarball.

  The detached code signature that signapple creates is almost identical to that which we were previously creating. The only difference is that the cpu architecture name is included in the extension (e.g. we have `bitcoin-qt.x86_64sign` instead of `bitcoin-qt.sign`). This was done in order to support signing universal binaries which we may want to do in the future. However signapple can still apply existing code signatures as it will accept the `.sign` extension. If it is desired, it can be modified to produce signatures with just the `.sign` extension. However I do not think it is necessary to maintain compatibility with the old process.

ACKs for top commit:
  laanwj:
    Code review ACK 2c403279e2

Tree-SHA512: 2a0e01e9133f8859b9de26e7e8fe1d2610d2cbdee2845e6008b12c083c7e3622cbb2d9b83c50a269e2c3074ab95914a8225d3cd4108017f58b77a62bf10951e0
2021-01-18 22:04:27 +01:00
Wladimir J. van der Laan
ca8218301b Merge #20884: script: Improve robustness of bitcoind.service on startup
9d02654677 doc: Fix systemd spelling and link to doc/init.md (Hennadii Stepanov)
601778c310 script: Add Documentation key to bitcoind.service (Hennadii Stepanov)
d9392b724c script: Improve robustness of bitcoind.service on startup (Hennadii Stepanov)

Pull request description:

  If network interfaces are not properly up the following happens:
  ```
  ...
  2021-01-08T10:17:11Z scheduler thread start
  2021-01-08T10:17:11Z libevent: getaddrinfo: address family for nodename not supported
  2021-01-08T10:17:11Z Binding RPC on address 127.0.0.1 port 8332 failed.
  2021-01-08T10:17:11Z HTTP: creating work queue of depth 16
  2021-01-08T10:17:11Z Using random cookie authentication.
  2021-01-08T10:17:11Z Generated RPC authentication cookie /var/lib/bitcoind/.cookie
  2021-01-08T10:17:11Z HTTP: starting 2 worker threads
  2021-01-08T10:17:11Z init message: Loading banlist...
  2021-01-08T10:17:11Z SetNetworkActive: true
  2021-01-08T10:17:11Z Error: Cannot resolve -externalip address: <EDITED>
  2021-01-08T10:17:11Z Shutdown: In progress...
  2021-01-08T10:17:11Z scheduler thread exit
  2021-01-08T10:17:11Z Shutdown: done
  ```

  This PR improves robustness on startup in such cases in documented way:
  https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/

  Also minor doc improvements are added.

ACKs for top commit:
  Sjors:
    ACK 9d02654
  practicalswift:
    ACK 9d02654677: patch looks correct
  darosior:
    ACK 9d02654677 -- been using the first patch too

Tree-SHA512: 38294f5682c09e6ea9008de7d7459098c920cf1b98ad8ef8a5d2ca01f2f781c0fec5591dc40ef36eeb19d94991b0c7fb7cb38c4e716bc7219875c9bcd0a55e1b
2021-01-18 20:02:00 +01:00
Wladimir J. van der Laan
991e612c3b Merge #20955: test: Fix get_previous_releases.py for aarch64
fa1d5e5137 test: Fix get_previous_releases.py for aarch64 (MarcoFalke)

Pull request description:

  Otherwise it will fail with "Not sure which binary to download..."

ACKs for top commit:
  laanwj:
    Code review ACK fa1d5e5137

Tree-SHA512: 0db71e898a431665757ce835016a4e05c629a95abc4a2951eac9bd9b5876ec3dc3d6f156d58565e2bcdf918cde4f2649183d4a58038ac13c705a7e914c0094d1
2021-01-18 19:20:53 +01:00
Wladimir J. van der Laan
c763cacb88 Merge #20938: build: fix linking against -latomic when building for riscv
54ce4fac80 build: improve macro for testing -latomic requirement (fanquake)
2c010b9c56 add std::atomic include to bitcoin-util.cpp (fanquake)

Pull request description:

  Since the merge of #19937, riscv builds have been failing, due to a link issue with [`std::atomic_exchange`](https://en.cppreference.com/w/cpp/atomic/atomic_exchange) in `bitcoin-util`:
  ```bash
    CXXLD    bitcoin-util
  bitcoin_util-bitcoin-util.o: In function `grind_task':
  /home/ubuntu/build/bitcoin/distsrc-riscv64-linux-gnu/src/bitcoin-util.cpp:98: undefined reference to `__atomic_exchange_1'
  collect2: error: ld returned 1 exit status

  ```

  We have a [macro](https://github.com/bitcoin/bitcoin/blob/master/build-aux/m4/l_atomic.m4) that tries to determine when `-latomic` is required, however it doesn't quite work well enough, as it's currently determining it isn't needed:
  ```bash
  ./autogen.sh
  ./configure --prefix=/home/ubuntu/bitcoin/depends/riscv64-linux-gnu
  ...
  checking whether std::atomic can be used without link library... yes
  ```

  This PR adds a call to `std::atomic_exchange` to the macro, which will get us properly linked against `-latomic` on riscv:
  ```bash
  checking whether std::atomic can be used without link library... no
  checking whether std::atomic needs -latomic... yes
  ```

  Also adds an `<atomic>` include to `bitcoin-util.cpp`.

ACKs for top commit:
  laanwj:
    Tested ACK 54ce4fac80

Tree-SHA512: 963c875097ee96b131163ae8109bcf8fecf4451d20faa2f3d223f9938ea3d8d1ed5604e12ad82c2b4b1c605fd293a9b6b08fefc00dd3e68d09c49e95029c6f50
2021-01-18 18:33:24 +01:00
John Newbery
ac3547eddd [addrman] Improve variable naming/code style of touched code. 2021-01-18 13:23:16 +00:00
John Newbery
a5c9b04959 [addrman] Don't rebucket new table entries unnecessarily
Only rebucket if the asmap checksum has changed, not if the file format
has changed but no asmap is provided.

Also, don't try to add an entry to another bucket if it already appears
in ADDRMAN_NEW_BUCKETS_PER_ADDRESS buckets.
2021-01-18 13:23:16 +00:00
John Newbery
8062d928ce [addrman] Rename asmap version to asmap checksum
Version implies that higher numbers take precendence. This is really a
checksum, to check whether the provided asmap is the same as the one
used when the peers.dat file was serialized.

Also update the comments to explain where/why this is used.
2021-01-18 13:23:16 +00:00
John Newbery
009b8e0fdf [addrman] Improve variable naming/code style of touched code. 2021-01-18 13:23:16 +00:00
John Newbery
b4c5fda417 [addrman] Fix new table bucketing during unserialization
An addrman entry can appear in up to 8 new table buckets. We store this
entry->bucket indexing during shutdown so that on restart we can restore
the entries to their correct buckets.

Commit ec45646de9 broke the
deserialization code so that each entry could only be put in up to one
new bucket. Fix that.
2021-01-18 13:23:16 +00:00
MarcoFalke
fa1d5e5137 test: Fix get_previous_releases.py for aarch64 2021-01-18 09:06:24 +01:00
Kiminuo
5353b0c64d Change type definitions for "chain" and "setup_clean_chain" from type comments to Python 3.6+ types. Additionally, set type for "nodes". 2021-01-18 09:01:07 +01:00
Troy Giorshev
dbf779d5de Clean PushMessage and ProcessMessages
This brings PushMessage and ProcessMessages further in line with the
style guide by fixing their if statements.

LogMessage is later called, inside an if statement, inside both of these
methods.
2021-01-17 20:31:02 -05:00
Carl Dong
1fca9811e1 lint: Skip whitespace lint for guix patches 2021-01-17 18:43:58 -05:00
Carl Dong
a91c46c57d guix: Make nsis reproducible by respecting SOURCE-DATE-EPOCH
When building nsis, if VERSION is not specified, it defaults to
cvs_version which is non-deterministic as it includes the current date.

This patches nsis to default to SOURCE_DATE_EPOCH if it exists so that
nsis is reproducible.

Upstream change: https://github.com/kichik/nsis/pull/13
2021-01-17 18:43:58 -05:00
Wladimir J. van der Laan
7acda55c4f Merge #20939: build: fix RELOC_SECTION security check for bitcoin-util
c061800bb1 build: fix RELOC_SECTION security check for bitcoin-util (fanquake)

Pull request description:

  The binutils we use for gitian builds strips the reloc section from
  Windows binaries, which breaks ASLR. As a temporary workaround, export
  main(). This is the same workaround as #18702 (bitcoin-cli), and will
  fix the currently failing security check:
  ```bash
  + make -j1 -C src check-security
  make: Entering directory '/home/ubuntu/build/bitcoin/distsrc-x86_64-w64-mingw32/src'
  Checking binary security...
  bitcoin-util.exe: failed RELOC_SECTION
  make: *** [check-security] Error 1
  ```

  Relevant upstream issue:
  https://sourceware.org/bugzilla/show_bug.cgi?id=19011

ACKs for top commit:
  dongcarl:
    ACK c061800bb1
  laanwj:
    ACK c061800bb1

Tree-SHA512: a1a4da0b2cddfc377190b9044a04f42a859ca79f11ce2c2ab4c3d066a2786c34d5446d75f8eec634f308d2d3349ebbd7c9f76dcaebeeb28e471c829851592367
2021-01-17 18:12:08 +01:00
Wladimir J. van der Laan
ad57fb756b wallet: Add BerkeleyDB version sanity check at init time
Detect version conflicts between the run-time BerkeleyDB library and the one used during compilation.

This is very unsafe (can result in anything from crashes to corruption) so shut down when one is detected.
2021-01-17 18:10:20 +01:00
Sebastian Falbesoner
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) 2021-01-17 12:58:46 +01:00
Wladimir J. van der Laan
30e664dcce Merge #20945: Fix 0.21.0 release note to specify correct option BIP 157 support
9a42b5e655 Fix 0.21.0 release note to specify correct option BIP 157 support (benthecarman)

Pull request description:

  https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L452

ACKs for top commit:
  mjdietzx:
    ACK 9a42b5e655
  laanwj:
    ACK 9a42b5e655

Tree-SHA512: 901195434046fd8a18a2329104b69e0ecc1bdb2bba1fe0688fb4a7c5992e9bdd66b8e50f46873f4ddf5ffe3c9aed860ccb01584812b06d865bd06fcd2b14c42b
2021-01-16 11:17:23 +01:00
Jarol Rodriguez
74d23bf7fb rpc: document RPC_TRANSACTION_ALREADY_IN_CHAIN exception
It is not documented that if you attempt to send a transaction
which already exists in a block, an RPC_TRANSACTION_ALREADY_IN_CHAIN
exception will be raised. This should be documented so that developers
are aware that this exception is raised.
2021-01-15 16:49:42 -05:00
Kiminuo
da9caa1ced Replace fs::absolute calls with AbsPathJoin calls
This adds better test coverage and will make it easier in #20744 to remove our dependency on the two-argument boost::filesystem::absolute() function which does not have a direct equivalent in C++17.
2021-01-15 22:48:15 +01:00
benthecarman
9a42b5e655 Fix 0.21.0 release note to specify correct option BIP 157 support 2021-01-15 14:05:59 -06:00
Kiminuo
66576c4fd5 test: Clear forced -walletdir setting after wallet init_tests
Leaving this value set interfered with the CreateWallet test if it happened to execute later in the test ordering. Specifically it would cause CreateWallet test to write data to the current directory instead of temporary test directory.
2021-01-15 20:19:31 +01:00
MarcoFalke
32e59fc371 Merge #20916: rpc: Return wtxid from testmempoolaccept
fa0aa87071 rpc: Return wtxid from testmempoolaccept (MarcoFalke)

Pull request description:

  It would be nice if `testmempoolaccept` returned the unique wtxid directly to avoid a costly `decoderawtransaction` roundtrip

ACKs for top commit:
  mjdietzx:
    utACK fa0aa87071
  stackman27:
    utACK [`fa0aa87`](fa0aa87071)
  glozow:
    cr ACK fa0aa87071

Tree-SHA512: 05dbaf46d93e47e9eedb725c2f57175e6d4e1722da0531fe4f80e74fc2518911da87634f25f61fa2bc8d87a3017e82fd0684b09a0a9706d71deed970035c2e7a
2021-01-15 20:13:34 +01:00
MarcoFalke
0a1cf6c347 Merge #20908: fuzz: Use mocktime in process_message* fuzz targets
fa0a864b38 fuzz: Use mocktime in process_message* fuzz targets (MarcoFalke)

Pull request description:

  Use mocktime to allow time to advance deterministically during execution of a fuzz input. This also allows to drop the call to `JumpOutOfIbd`.

ACKs for top commit:
  practicalswift:
    cr ACK fa0a864b38

Tree-SHA512: e92fc70ec6bd49760173cb202549f364304e22b3f7127b9a4da8447cf9341008e477ad42c2599c2fde167bbcbc0e2d139709b4ef6371788bc2c1c3b7f589e11d
2021-01-15 19:56:18 +01:00
practicalswift
b4511e2e2e log: Prefix log messages with function name if -logsourcelocations is set 2021-01-15 09:57:32 +00:00
fanquake
c061800bb1 build: fix RELOC_SECTION security check for bitcoin-util
The binutils we use for gitian builds strips the reloc section from
Windows binaries, which breaks ASLR. As a temporary workaround, export
main(). This is the same workaround as #18702 (bitcoin-cli), and will
fix the currently failing security check:
```bash
+ make -j1 -C src check-security
make: Entering directory '/home/ubuntu/build/bitcoin/distsrc-x86_64-w64-mingw32/src'
Checking binary security...
bitcoin-util.exe: failed RELOC_SECTION
make: *** [check-security] Error 1
```

Relevant upstream issue:
https://sourceware.org/bugzilla/show_bug.cgi?id=19011
2021-01-15 11:53:14 +08:00
fanquake
54ce4fac80 build: improve macro for testing -latomic requirement
riscv builds are currently failing because -latomic isn't being linked
against, when it is needed:
```bash
/home/ubuntu/build/bitcoin/distsrc-riscv64-linux-gnu/src/bitcoin-util.cpp:98: undefined reference to `__atomic_exchange_1'
```

This exteneds our macro to ensure that -latomic is linked against when
required.
2021-01-15 10:54:07 +08:00
fanquake
2c010b9c56 add std::atomic include to bitcoin-util.cpp 2021-01-15 10:40:29 +08:00
fanquake
f91587f050 Merge #20834: locks and docs in ATMP and CheckInputsFromMempoolAndCache
2f463f57e3 [doc] for CheckInputsFromMempoolAndCache (gzhao408)
85cc6bed64 lock annotations for MemPoolAccept functions (gzhao408)

Pull request description:

  This is a very small PR that adds some lock annotations to clarify that, now, the `pool.cs` lock is held throughout tx validation for mempool.  The comments in `CheckInputsFromMempoolAndCache` were unclear/outdated so I updated those as well.

  ~This PR is a cleanup. It removes unnecessary code that doesn't do much.~

ACKs for top commit:
  sdaftuar:
    utACK 2f463f57e3
  jnewbery:
    utACK 2f463f57e3
  MarcoFalke:
    cr ACK 2f463f57e3
  fanquake:
    ACK 2f463f57e3

Tree-SHA512: 5863a546b00ef02eba8f208c2c04c32f64671f17c967a5e3c51332fc0f472e5e9addddae075d0b91c77caebe1be9331317646b0bec802e86d9550773fd9621a9
2021-01-14 23:35:41 +08:00
MarcoFalke
29d2aeb4a2 Merge #20828: fuzz: Introduce CallOneOf helper to replace switch-case
fa75d40ef8 fuzz: Introduce CallOneOf helper to replace switch-case (MarcoFalke)

Pull request description:

  The current `switch (fuzzed_data_provider.ConsumeIntegralInRange<int>(0, nn)) { case 0: ... case 1: ... case nn: ...` has several problems:

  * It makes it hard to review newly added targets, because it requires manual counting of cases
  * It makes it hard to update a target, because updating all case labels is trivial, but tedious to review and causes merge conflicts
  * ~~Updating the target raises the question whether the case labels should be preserved to not invalidate the existing fuzz inputs format. Fuzz input format might already change implicitly on every commit, so this isn't something worthwhile to pursue.~~ Edit: This pull doesn't fix this problem.

  Fix all issues by adding a new `CallOneOf` helper

ACKs for top commit:
  ajtowns:
    ACK fa75d40ef8 - code review only
  jnewbery:
    utACK fa75d40ef8

Tree-SHA512: 2daa602b240b86c8e85a024e008f03a57ba60349377eed771f4d21a97a9dba9b66e93fff16ff1992018d4330be7a1a276944c3dfdf698748ce135626c380e563
2021-01-14 11:07:22 +01:00
Wladimir J. van der Laan
ad571bd354 Merge #20931: doc: Add historic 0.21.0 release notes
faea902721 doc: Add historic 0.21.0 release notes (MarcoFalke)

Pull request description:

  Archive the notes

ACKs for top commit:
  laanwj:
    ACK faea902721

Tree-SHA512: a9b5f4ff9c8273b54c81cc9ece0e90dcb2383ee9971a552b9d5ae5ce77ae29b970a4a4240d22e8e07811cc7e64e78adab841592ab3cd5d729c80233e033baf51
2021-01-14 10:47:10 +01:00
MarcoFalke
faea902721 doc: Add historic 0.21.0 release notes 2021-01-14 10:40:43 +01:00
MarcoFalke
cb2c578451 Merge bitcoin-core/gui#148: Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes"
8775691383 Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes" (Luke Dashjr)

Pull request description:

  The SendConfirmationDialog is used for bumping the fee, where "Send" doesn't really make sense

  Originally https://github.com/bitcoin/bitcoin/pull/17463, but rewritten here much simpler based on other merged changes.

ACKs for top commit:
  hebasto:
    ACK 8775691383, tested on Linux Mint 20.1 (x86_64, Qt 5.12.8):

Tree-SHA512: 3953cc9c09613c9a629def8b4dc061b537f148ddcb378430645602e0be0f3a9f1cff083aa685b94b2e9372300d02ec97e0d9ea89db6e3c6feec86795090f0f77
2021-01-13 17:47:46 +01:00
MarcoFalke
22fa9673b0 Merge #20917: doc, rpc: add missing signet mentions in network name lists
fc726e0138 doc, rpc: add missing signet mentions in network name lists (Sebastian Falbesoner)

Pull request description:

  This small PR adds a few missing mentions of signet w.r.t. chain enumerations:

  - RPC `getblockchaininfo`: result description for `"chain"`
  - RPC `getmininginfo`: result description for `"chain"`
  - REST interface documentation:
      - default ports listing for each chain
      - `"chain"` description for `chaininfo` endpoint result

  The instances were identified via `git grep -i "main.*test.*reg"`.

ACKs for top commit:
  ajtowns:
    ACK fc726e0138 -- quick code review only
  benthecarman:
    ACK fc726e0138

Tree-SHA512: 62cdc6ef74fa10db75cc04b9eaf7367183f726b3fee3d21fdf741b3816669dd21508735e89da389ddac980f49773ab229263748d1399553375fefe4526361846
2021-01-13 17:31:47 +01:00
jarolrod
c943282b5e validation: remove redundant check on pindex
This removes a conditional that checks if pindex is equal to nullptr.
This check is redundant because the branch where pindex is set returns at an earlier time. Additionaly, The independence of the earlier and later pindex is made clearer.
2021-01-13 11:22:06 -05:00
John Newbery
06fa85cd50 [net] InactivityCheck() takes a CNode reference 2021-01-13 16:18:12 +00:00
Wladimir J. van der Laan
e7eb37128c Merge #20913: doc: Add manual page generation for bitcoin-util
bc99ae77e4 scripted-diff: Fix typo in stub manual pages (Wladimir J. van der Laan)
b5e93f873a doc: Add manual page generation for bitcoin-util (Wladimir J. van der Laan)

Pull request description:

  - Add `-version` option to `bitcoin-util`
  - Add `bitcoin-util` call to `gen-manpages.sh`
  - Add stub manual page `bitcoin-util.1`
  - Add install of `bitcoin-util.1` to build system

ACKs for top commit:
  fanquake:
    ACK bc99ae77e4

Tree-SHA512: 948df66c62bbca1cf6da26845dfa63f8f5d036a3d5744add468dd1ce7f442c123d7b0db7011c2e8e3ee6539fd391c7ee2c21b706ec81b21b02821c9501cd077d
2021-01-13 10:16:09 +01:00
Wladimir J. van der Laan
60427ee35f Merge #20811: refactor: move net_processing implementation details out of header
c97f70c861 net_processing: move Peer definition to .cpp (Anthony Towns)
e0f2e6d2df net_processing: move PeerManagerImpl into cpp file (Anthony Towns)
a568b82feb net_processing: split PeerManager into interface and implementation classes (Anthony Towns)
0df3d3fd6b net_processing: make more of PeerManager private (Anthony Towns)
0d246a59b6 net, net_processing: move NetEventsInterface method docs to net.h (Anthony Towns)

Pull request description:

  Moves the implementation details of `PeerManager` and all of `struct Peer` into net_processing.cpp.

ACKs for top commit:
  jnewbery:
    ACK c97f70c861
  Sjors:
    ACK c97f70c861
  laanwj:
    Code review ACK c97f70c861
  vasild:
    ACK c97f70c861

Tree-SHA512: 2e081e491d981c61bd78436a6a6c2eb41d3c2d86a1a8ef9d4d6f801b82cb64a8bf707a96db3429418d1704cffb60a657d1037a0336cbc8173fb79aef9eb72001
2021-01-13 09:48:06 +01:00
Wladimir J. van der Laan
8ffaf5c2f5 Merge #19935: Move SaltedHashers to separate file and add some new ones
281fd1a4a0 Replace KeyIDHasher with SaltedSipHasher (Andrew Chow)
210b693db6 Add generic SaltedSipHasher (Andrew Chow)
95e61c1cf2 Move Hashers to util/hasher.{cpp/h} (Andrew Chow)

Pull request description:

  There are existing `SaltedOutPointHasher` and `SaltedTxidHasher` classes used for `std::unordered_map` and `std::unordered_set` that could be useful in other places in the codebase. So we these to their own `saltedhash.{cpp/h}` file. An existing `KeyIDHasher` is moved there too. Additionally, `ScriptIDHasher`, `SaltedPubkeyHasher`, and `SaltedScriptHasher` are added so that they can be used in future work.

  `KeyIDHasher` and `ScriptIDHasher` are not salted so that equality comparisons of maps and sets keyed by `CKeyID` and `CScriptID` will actually work.

  Split from #19602 (and a few other PRs/branches I have).

ACKs for top commit:
  laanwj:
    Code review ACK 281fd1a4a0
  jonatack:
    ACK 281fd1a4a0, code review, debug build and ran bitcoind after rebasing to master @ dff0f6f753
  fjahr:
    utACK 281fd1a4a0

Tree-SHA512: bb03b231ccf3c9ecefc997b8da9c3770af4819f9be5b0a72997a103864e84046a2ac39b8eadf0dc9247bdccd53f86f433642e3a098882e6748341a9e7736271b
2021-01-13 08:49:17 +01:00
MarcoFalke
fa0a864b38 fuzz: Use mocktime in process_message* fuzz targets 2021-01-13 07:48:41 +01:00
sinetek
2a39ccf133 Add include for std::bind. 2021-01-13 02:05:00 +01:00
MarcoFalke
fa0aa87071 rpc: Return wtxid from testmempoolaccept 2021-01-12 18:43:43 +01:00
Sebastian Falbesoner
fc726e0138 doc, rpc: add missing signet mentions in network name lists 2021-01-12 18:43:31 +01:00
Wladimir J. van der Laan
bc99ae77e4 scripted-diff: Fix typo in stub manual pages
-BEGIN VERIFY SCRIPT-
sed -i 's/placefolder/placeholder/' $(git ls-files doc/man/\*.1)
-END VERIFY SCRIPT-
2021-01-12 16:46:55 +01:00
Wladimir J. van der Laan
b5e93f873a doc: Add manual page generation for bitcoin-util
- Add `-version` option to `bitcoin-util`
- Add `bitcoin-util` call to `gen-manpages.sh`
- Add stub manual page `bitcoin-util.1`
- Add install of `bitcoin-util.1` to build system
2021-01-12 14:09:21 +01:00
Wladimir J. van der Laan
7b975639ef Merge #19937: signet mining utility
595a34dbea contrib/signet: Document miner script in README.md (Anthony Towns)
ff7dbdc08a contrib/signet: Add script for generating a signet chain (Anthony Towns)
13762bcc96 Add bitcoin-util command line utility (Anthony Towns)
95d5d5e625 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns)
81c54dec20 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns)

Pull request description:

  Adds `contrib/signet/miner` for mining signet blocks.

  Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied.

  Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet).

ACKs for top commit:
  laanwj:
    code review ACK 595a34dbea

Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
2021-01-12 12:53:45 +01:00
fanquake
18017152c2 Merge #20619: guix: Quality of life improvements
570e43fe72 guix: Print build params inside/outside of container (Carl Dong)
2f9d1fdde6 guix: Move DISTSRC determination to guix-build.sh (Carl Dong)
0b7cd07bb5 guix: Move OUTDIR determination+creation to guix-build.sh (Carl Dong)
d27ff8b86a guix: Add more sanity checks to guix-build.sh (Carl Dong)
57f9533146 guix: Add section headings to guix-build.sh (Carl Dong)
38b7b2ed72 genbuild: Specify rev-parse length (Carl Dong)
036dc740da docs: Point to contrib/guix/README.md in doc/guix.md (Carl Dong)
34f0fda2d3 guix: Small updates to README wording (Carl Dong)
402e3a5b1e guix: Update HOSTS README entry for new architectures (Carl Dong)
cfa7ceb21b guix: Remove README development environment section (Carl Dong)
93b6a8544a guix: Add ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS options (Carl Dong)
0f31e24703 guix: Add SUBSTITUTE_URLS option (Carl Dong)
444fcfca90 guix: Make guix honor MAX_JOBS setting (Carl Dong)

Pull request description:

  After live-demo-ing a Guix build (which completed successfully!) on achow101's stream, I realized there were a few quality of life improvements which can be made to improve the user experience of our Guix build process. Here are a few of them.

  Notable changes:
  1. When `MAX_JOBS` is specified, both `guix time-machine` and `guix environment` will now build up to `MAX_JOBS` packages at a time when creating the build environment
  2. The instructions for using substitutes were incorrect, and has now been replaced with a `SUBSTITUTE_URLS` environment variable, which works well with shell's IFS splitting rules
  3. New `ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS` options, for more granular customization of the build process.
  4. README cleanup

ACKs for top commit:
  fanquake:
    ACK 570e43fe72 - lets move this forward.

Tree-SHA512: 4e8ab560522ade5efb5e8736aec0fb1a3f19ae9deb586c1ab87020816876f3f466a950b3f8c04d9fa1d072ae5ee780038c5c9063577049bdd9db17978e11c328
2021-01-12 18:53:35 +08:00
gzhao408
2f463f57e3 [doc] for CheckInputsFromMempoolAndCache 2021-01-12 02:27:09 -08:00
gzhao408
85cc6bed64 lock annotations for MemPoolAccept functions
We should already have the mempool lock when entering
CheckInputsFromMempoolAndCache
2021-01-12 02:27:09 -08:00
Anthony Towns
595a34dbea contrib/signet: Document miner script in README.md 2021-01-12 18:34:29 +10:00
Anthony Towns
ff7dbdc08a contrib/signet: Add script for generating a signet chain 2021-01-12 18:34:29 +10:00
Anthony Towns
13762bcc96 Add bitcoin-util command line utility 2021-01-12 18:34:25 +10:00
fanquake
7838db141b Merge #20495: sync: Use decltype(auto) return type for WITH_LOCK
3eb94ec81b sync: Use decltype(auto) return type for WITH_LOCK (Carl Dong)

Pull request description:

  > Now that we're using C++17, we can use the decltype(auto) return type
  > for functions and lambda expressions.
  >
  > As demonstrated in this commit, this can simplify cases where previously
  > the compiler failed to deduce the correct return type.
  >
  > Just for reference, for the "assign to ref" cases fixed here, there are
  > 3 possible solutions:
  >
  > - Return a pointer and immediately deref as used before this commit
  > - Make sure the function/lambda returns declspec(auto) as used after
  >   this commit
  > - Class& i = WITH_LOCK(..., return std::ref(...));
  >
  > -----
  >
  > References:
  > 1. https://en.cppreference.com/w/cpp/language/function#Return_type_deduction
  > 2. https://en.cppreference.com/w/cpp/language/template_argument_deduction#Other_contexts
  > 3. https://en.cppreference.com/w/cpp/language/auto
  > 4. https://en.cppreference.com/w/cpp/language/decltype
  >
  > Explanations:
  > 1. https://stackoverflow.com/a/21369192
  > 2. https://stackoverflow.com/a/21369170

  Thanks to sipa and ryanofsky for helping me understand this

ACKs for top commit:
  jnewbery:
    utACK 3eb94ec81b
  hebasto:
    ACK 3eb94ec81b, I have reviewed the code and it looks OK, I agree it can be merged. I have verified possible warnings:
  ryanofsky:
    Code review ACK 3eb94ec81b

Tree-SHA512: 5f55c7722aeca8ea70e5c1a8db93e93ba0e356e8967e7f607ada38003df4b153d73c29bd2cea8d7ec1344720d37d857ea7dbfd2a88da1d92e0e9cbb9abd287df
2021-01-12 15:56:19 +08:00
MarcoFalke
6af013792f Merge #19315: [tests] Allow outbound & block-relay-only connections in functional tests.
b4dd2ef800 [test] Test the add_outbound_p2p_connection functionality (Amiti Uttarwar)
602e69e427 [test] P2PBlocksOnly - Test block-relay-only connections. (Amiti Uttarwar)
8bb6beacb1 [test/refactor] P2PBlocksOnly - Extract transaction violation test into helper. (Amiti Uttarwar)
99791e7560 [test/refactor] P2PBlocksOnly - simplify transaction creation using blocktool helper. (Amiti Uttarwar)
3997ab9154 [test] Add test framework support to create outbound connections. (Amiti Uttarwar)
5bc04e8837 [rpc/net] Introduce addconnection to test outbounds & blockrelay (Amiti Uttarwar)

Pull request description:

  The existing functional test framework uses the `addnode` RPC to spin up manual connections between bitcoind nodes. This limits our ability to add integration tests for our networking code, which often executes different code paths for different connection types.

  **This PR enables creating `outbound` & `block-relay-only` P2P connections in the functional tests.** This allows us to increase our p2p test coverage, since we can now verify expectations around these connection types.

  This builds out the [prototype](https://github.com/bitcoin/bitcoin/issues/14210#issuecomment-527421978) proposed by ajtowns in #14210. 🙌🏽

  An overview of this branch:
  - introduces a new test-only RPC function `addconnection` which initiates opening an `outbound` or `block-relay-only` connection. (conceptually similar to `addnode` but for different connection types & restricted to regtest)
  - adds `test_framework` support so a mininode can open an `outbound`/`block-relay-only` connection to a `P2PInterface`/`P2PConnection`.
  - updates `p2p_blocksonly` tests to create a `block-relay-only` connection & verify expectations around transaction relay.
  - introduces `p2p_add_connections` test that checks the behaviors of the newly introduced `add_outbound_p2p_connection` test framework function.

  With these changes, there are many more behaviors that we can add integration tests for. The blocksonly updates is just one example.

  Huge props to ajtowns for conceiving the approach & providing me feedback as I've built out this branch. Also thank you to jnewbery for lots of thoughtful input along the way.

ACKs for top commit:
  troygiorshev:
    reACK b4dd2ef800
  jnewbery:
    utACK b4dd2ef800
  MarcoFalke:
    Approach ACK b4dd2ef800 🍢

Tree-SHA512: d1cba768c19c9c80e6a38b1c340cc86a90701b14772c4a0791c458f9097f6a4574b4a4acc7d13d6790c7b1f1f197e2c3d87996270f177402145f084ef8519a6b
2021-01-11 21:06:57 +01:00
fanquake
6d81d7aa87 Merge #20787: Use C++17 std::array deduction for OUTPUT_TYPES, ALL_FEE_ESTIMATE_HORIZONS
aaaa987840 refactor: Use C++17 std::array deduction for ALL_FEE_ESTIMATE_HORIZONS (MarcoFalke)
fa39cdd072 refactor: Use C++17 std::array deduction for OUTPUT_TYPES (MarcoFalke)

Pull request description:

  With the new C++17 array deduction rules, an array encompassing all values in an enum can be specified in the same header file that specifies the enum. This is useful to avoid having to repeatedly enumerate all enum values in the code. E.g. the RPC code, but also the fuzz code.

ACKs for top commit:
  theStack:
    cr ACK aaaa987840 ⚙️
  fanquake:
    ACK aaaa987840

Tree-SHA512: b71bd98f3ca07ddfec385735538ce89a4952e418b52dc990fb160187ccef1fc7ebc139d42988b6f7b48df24823af61f803b83d47fb7a3b82475f0c0b109bffb7
2021-01-11 21:46:09 +08:00
MarcoFalke
5cce607105 Merge #20373: refactor, net: Increase CNode data member encapsulation
3642b2ed34 refactor, net: Increase CNode data member encapsulation (Hennadii Stepanov)
acebb79d3f refactor, move-only: Relocate CNode private members (Hennadii Stepanov)

Pull request description:

  All protected `CNode` data members could be private.

ACKs for top commit:
  jnewbery:
    utACK 3642b2ed34
  MarcoFalke:
    review ACK 3642b2ed34 🏛

Tree-SHA512: 8435e3c43c3b7a3107d58cb809b8b5e1a1c0068677e249bdf0fc6ed24140ac4fc4efe2a280a1ee86df180d738c0c9e10772308690607954db6713000cf6e728d
2021-01-11 13:19:43 +01:00
Wladimir J. van der Laan
675af2a515 Merge #20852: net: allow CSubNet of non-IP networks
39b43298d9 test: add test for banning of non-IP addresses (Vasil Dimov)
94d335da7f net: allow CSubNet of non-IP networks (Vasil Dimov)

Pull request description:

  Allow creation of valid `CSubNet` objects of non-IP networks and only
  match the single address they were created from (like /32 for IPv4 or
  /128 for IPv6).

  This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
  and in `BanMan` which assume that creating a subnet from any address
  using the `CSubNet(CNetAddr)` constructor would later match that address
  only. Before this change a non-IP subnet would be invalid and would not
  match any address.

ACKs for top commit:
  jonatack:
    Code review re-ACK 39b43298d9 per `git diff 5e95ce6 39b4329`; only change since last review is improvements to the functional test; verified the test fails on master @ 616eace0 where expected (`assert(self.is_banned(node, tor_addr))` fails and unban unfails)
  laanwj:
    code review ACK 39b43298d9

Tree-SHA512: 3239b26d0f2fa2d1388b4fdbc1d05ce4ac1980be699c6ec46049409baefcb2006b1e72b889871e2210e897f6725c48e873f68457eea7e6e4958ab4f959d20297
2021-01-11 11:27:47 +01:00
MarcoFalke
fa75d40ef8 fuzz: Introduce CallOneOf helper to replace switch-case
Can be reviewed with --ignore-all-space
2021-01-11 10:37:16 +01:00
jackielove4u
92370033a2 contrib: embed C++11 patch in install_db4.sh 2021-01-11 10:34:27 +01:00
Jonas Schnelli
616eace02a Merge bitcoin-core/gui#161: Add PeerTableModel::StatsRole to prevent data layer violation
b3e9bcaac8 qt, refactor: Drop no longer used PeerTableModel::getNodeStats function (Hennadii Stepanov)
49c604077c qt: Use PeerTableModel::StatsRole (Hennadii Stepanov)
35007edf9c qt: Add PeerTableModel::StatsRole (Hennadii Stepanov)

Pull request description:

  This PR allows to access to the `CNodeCombinedStats` instance directly from any view object.

  The `PeerTableModel::getNodeStats` member function removed as a kind of layer violation.

  No behavior changes.

  Also other pulls (bugfixes) are based on this one: #18 and #164.

ACKs for top commit:
  jonatack:
    Tested re-ACK b3e9bcaac8 per `git range-diff ae8f797 4c05fe0 b3e9bca`
  promag:
    Code review ACK b3e9bcaac8.
  jonasschnelli:
    utACK b3e9bcaac8

Tree-SHA512: 6ba50d5dd2c0373655d491ce8b130c47d598da2db5ff4b00633f447404c7e70f8562ead53ddf166e851384d9632ff9146a770c99845c2cdd3ff7250677e4c130
2021-01-11 09:06:21 +01:00
MarcoFalke
9c0b76c709 Merge #20876: test: Replace getmempoolentry with testmempoolaccept in MiniWallet
faabc26a61 test: Replace getmempoolentry with testmempoolaccept in MiniWallet (MarcoFalke)

Pull request description:

  This is a refactor to not use the return value of `sendrawtransaction` and `getmempoolentry` with the goal that submitting the tx to the mempool will become optional.

ACKs for top commit:
  mjdietzx:
    ACK faabc26a61

Tree-SHA512: 4260a59e65fed1c807530dad23f1996ba6e881bcce91995f5b498a0be6001f67b3e0251507c8801750fe105410147c68bb2f393ebe678c6a3a4d045e5d72fc19
2021-01-11 08:57:28 +01:00
fanquake
bd6af53e1f Merge #20480: Replace boost::variant with std::variant
faa8f68943 Replace boost::variant with std::variant (MarcoFalke)

Pull request description:

  Now that we can use std::variant from the vanilla standard library, drop the third-party boost variant dependency

ACKs for top commit:
  fjahr:
    Code review ACK faa8f68943
  fanquake:
    ACK faa8f68943

Tree-SHA512: 6e3aecd33b00c2e31a763f999247944d5b2ce5e3018f1965c516c1000cd08ff6703a8d50fb0be64883153da2925ae72986b8a6b96586db74057bd05d6f4986e6
2021-01-11 12:05:46 +08:00
fanquake
c4458cc3a1 Merge #18819: net: Replace cs_feeFilter with simple std::atomic
fad1f0fd33 net: Remove unused cs_feeFilter (MarcoFalke)

Pull request description:

  A `RecursiveMutex` is overkill for setting or reading a plain integer. Even a `Mutex` is overkill, when a plain `std::atomic` can be used.

  This removes 11 lines of code. Also, it is cutting down on the number of locks put on the stack at the same time, which complicates review looking out for potential lock contention.

ACKs for top commit:
  jnewbery:
    utACK fad1f0fd33
  practicalswift:
    cr ACK fad1f0fd33: patch looks correct

Tree-SHA512: 647f9b954fbf52e138d3e710937eb9131b390fef0deae03fd6a162d5a18b9f194010800bbddc8f89208d91be2802dff11c3884d04b3dd233865abd12aa3cde06
2021-01-11 10:14:11 +08:00
Jon Atack
79a2576af1 doc: update ConnectionType Doxygen documentation 2021-01-10 21:34:17 +01:00
Jon Atack
f3153dc08f gui: improve markup handling of connection type tooltip
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-01-10 21:34:15 +01:00
Jon Atack
4f09615733 gui: return inbound {full, block} relay type in peer details 2021-01-10 21:33:10 +01:00
Vasil Dimov
39b43298d9 test: add test for banning of non-IP addresses
Co-authored-by: Jon Atack <jon@atack.com>
2021-01-10 15:51:25 +01:00
Vasil Dimov
94d335da7f net: allow CSubNet of non-IP networks
Allow creation of valid `CSubNet` objects of non-IP networks and only
match the single address they were created from (like /32 for IPv4 or
/128 for IPv6).

This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
and in `BanMan` which assume that creating a subnet from any address
using the `CSubNet(CNetAddr)` constructor would later match that address
only. Before this change a non-IP subnet would be invalid and would not
match any address.
2021-01-10 15:51:15 +01:00
Hennadii Stepanov
3642b2ed34 refactor, net: Increase CNode data member encapsulation
All protected CNode data members could be private.
2021-01-10 12:00:46 +02:00
Hennadii Stepanov
acebb79d3f refactor, move-only: Relocate CNode private members 2021-01-10 12:00:44 +02:00
MarcoFalke
0944024309 Merge bitcoin-core/gui#163: Peer details: replace Direction with Connection Type
06ba9b3008 rpc: move getpeerinfo connection_type help to correct place (Jon Atack)
c95fe6e38f gui: improve connection type tooltip (Hennadii Stepanov)
2c19ba2e1d gui: replace Direction with Connection Type in peer details (Jon Atack)
7e2beab2d2 gui: create GUIUtil::ConnectionTypeToQString utility function (Jon Atack)

Pull request description:

  ![Screenshot from 2021-01-09 11-23-17](https://user-images.githubusercontent.com/2415484/104089297-c5e18d80-5265-11eb-9251-49afcfdb562b.png)

  Closes #159.

ACKs for top commit:
  hebasto:
    re-ACK 06ba9b3008, the tooltip content is organized as unordered list.
  jarolrod:
    re-ACK 06ba9b3008

Tree-SHA512: 24f46494ceb42ed308e4a4f2a543dbc4f4e6409a6f738c145a9f16e175bf69d411cbc944a4fd969f1829d57644dfbc194182fa8d4e9e6bce82acbeca8c673748
2021-01-10 10:42:50 +01:00
MarcoFalke
555fc0789d Merge #20881: fuzz: net permission flags in net processing
fad327ca65 fuzz: net permission flags in net processing (MarcoFalke)

Pull request description:

  to increase coverage

ACKs for top commit:
  Crypt-iQ:
    cr ACK fad327c
  practicalswift:
    ACK fad327ca65

Tree-SHA512: f8643d1774ff13524ab97ab228ad070489e080435e5742af26e6e325fd002e4c1fd78b9887e11622e79d6fe0c4daaddce5e033e6cd4b32e50fd68b434aab7333
2021-01-10 10:33:57 +01:00
fanquake
708ef4424a Merge #20890: doc: Add explicit macdeployqtplus dependencies install step
3e61b8c800 doc: Add explicit macdeployqtplus dependencies install step (Hennadii Stepanov)

Pull request description:

  This PR adds to macOS docs an explicit step to install `macdeployqtplus` script dependencies that are not part of the [Python Standard Library](https://docs.python.org/3/library/index.html):
  - https://pypi.org/project/ds-store/
  - https://pypi.org/project/mac-alias/

  This change is required on macOS 11 Big Sur:

  -  #20371
  - #20878

  Close #20878.

ACKs for top commit:
  fanquake:
    ACK 3e61b8c800

Tree-SHA512: d177139ee142d47cb27ad878d721cafcd03403ef861965ff532d712da461416380ec5878f70accf223a552a1f1e65eedb1e0ad72cb7a96791f8a55536ce28645
2021-01-10 15:03:05 +08:00
Anthony Towns
c97f70c861 net_processing: move Peer definition to .cpp 2021-01-09 23:27:45 +10:00
Anthony Towns
e0f2e6d2df net_processing: move PeerManagerImpl into cpp file 2021-01-09 23:27:45 +10:00
Anthony Towns
a568b82feb net_processing: split PeerManager into interface and implementation classes 2021-01-09 23:27:45 +10:00
Anthony Towns
0df3d3fd6b net_processing: make more of PeerManager private 2021-01-09 23:27:45 +10:00
Anthony Towns
0d246a59b6 net, net_processing: move NetEventsInterface method docs to net.h 2021-01-09 23:27:45 +10:00
Jon Atack
06ba9b3008 rpc: move getpeerinfo connection_type help to correct place
per review feedback
2021-01-09 13:50:00 +01:00
Hennadii Stepanov
c95fe6e38f gui: improve connection type tooltip
- remove RPC and option names from the translatable string
- use non-breaking hyphens
2021-01-09 13:49:51 +01:00
Jon Atack
2c19ba2e1d gui: replace Direction with Connection Type in peer details 2021-01-09 13:49:20 +01:00
Wladimir J. van der Laan
5574e48963 Merge #20741: doc: Update 'Secure string handling'
7117d7503f Update 'Secure string handling' (Prayank)

Pull request description:

  - Add information about possible path traversal attack
  - [wallet_name](https://bitcoincore.org/en/doc/0.20.0/rpc/wallet/createwallet/) (string): _The name for the new wallet. If this is a 'path', the wallet will be created at the 'path' location._

  Fixes https://github.com/bitcoin/bitcoin/issues/20128 (Not really fixing it but workaround)

  This PR is an alternative to https://github.com/bitcoin/bitcoin/pull/20393

ACKs for top commit:
  michaelfolkson:
    ACK 7117d7503f
  RiccardoMasutti:
    ACK 7117d7503f
  benthecarman:
    ACK 7117d7503f

Tree-SHA512: 0d6c4f8db5feba848bbb583e87a99e6c4b655deaa2b566164e2632acc1aabf470d4626d2dc4b82c4997effc30d9b474d860d0e0d3e896648c5cc9bfdb623da6d
2021-01-09 09:00:23 +01:00
Hennadii Stepanov
3e61b8c800 doc: Add explicit macdeployqtplus dependencies install step
This change is required on macOS 11 Big Sur.
2021-01-09 09:37:12 +02:00
Andrew Chow
ea0a7ec949 Remove deprecated bumpfee behavior 2021-01-08 18:58:58 -05:00
Carl Dong
570e43fe72 guix: Print build params inside/outside of container 2021-01-08 11:40:02 -05:00
Carl Dong
2f9d1fdde6 guix: Move DISTSRC determination to guix-build.sh 2021-01-08 11:40:01 -05:00
Carl Dong
0b7cd07bb5 guix: Move OUTDIR determination+creation to guix-build.sh 2021-01-08 11:40:01 -05:00
Carl Dong
d27ff8b86a guix: Add more sanity checks to guix-build.sh 2021-01-08 11:40:01 -05:00
Carl Dong
57f9533146 guix: Add section headings to guix-build.sh 2021-01-08 11:40:01 -05:00
Carl Dong
38b7b2ed72 genbuild: Specify rev-parse length 2021-01-08 11:40:01 -05:00
Carl Dong
036dc740da docs: Point to contrib/guix/README.md in doc/guix.md 2021-01-08 11:40:01 -05:00
Carl Dong
34f0fda2d3 guix: Small updates to README wording 2021-01-08 11:40:01 -05:00
Carl Dong
402e3a5b1e guix: Update HOSTS README entry for new architectures 2021-01-08 11:40:01 -05:00
Carl Dong
cfa7ceb21b guix: Remove README development environment section 2021-01-08 11:40:01 -05:00
Carl Dong
93b6a8544a guix: Add ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS options 2021-01-08 11:40:01 -05:00
Carl Dong
0f31e24703 guix: Add SUBSTITUTE_URLS option 2021-01-08 11:40:01 -05:00
Carl Dong
444fcfca90 guix: Make guix honor MAX_JOBS setting 2021-01-08 11:40:01 -05:00
Hennadii Stepanov
9d02654677 doc: Fix systemd spelling and link to doc/init.md
See https://www.freedesktop.org/wiki/Software/systemd/
2021-01-08 17:55:34 +02:00
Hennadii Stepanov
601778c310 script: Add Documentation key to bitcoind.service 2021-01-08 17:55:34 +02:00
Hennadii Stepanov
d9392b724c script: Improve robustness of bitcoind.service on startup 2021-01-08 17:34:48 +02:00
Hennadii Stepanov
af58f5b12c qt: Stop the effect of hidden widgets on the size of QStackedWidget
Layouts of the hidden widgets, those are children of QStackedWidget,
could prevent to adjust the size of the parent widget in the
WalletFrame widget.
2021-01-08 16:53:20 +02:00
Jon Atack
7e2beab2d2 gui: create GUIUtil::ConnectionTypeToQString utility function 2021-01-08 15:24:31 +01:00
Hennadii Stepanov
4e1154dfd1 qt: Use "fusion" style on macOS Big Sur with old Qt
The "macintosh" style is broken on macOS Big Sur at least for Qt 5.9.8.
2021-01-08 16:20:49 +02:00
MarcoFalke
9158d6f341 Merge #20786: net: [refactor] Prefer integral types in CNodeStats
faecb74562 Expose integral m_conn_type in CNodeStats, remove m_conn_type_string (Jon Atack)

Pull request description:

  Currently, strings are stored for what are actually integral (strong) enum types. This is fine, because the strings are only used as-is for the debug log and RPC. However, it complicates using them in the GUI. User facing strings in the GUI should be translated and only string literals can be picked up for translation, not runtime `std::string`s.

  Fix that by removing the `std::string` members and replace them by strong enum integral types.

ACKs for top commit:
  jonatack:
    Code review ACK faecb74562
  theStack:
    Code review ACK faecb74562 🌲

Tree-SHA512: 24df2bd0645432060e393eb44b8abaf20fe296457d07a867b0e735c3e2e75af7b03fc6bfeca734ec33ab816a7c8e1f8591a5ec342f3afe3098a4e41f5c2cfebb
2021-01-08 15:14:53 +01:00
MarcoFalke
1e078f17b5 Merge #20882: fuzz: Add missing muhash registration
fa44417fcb fuzz: Add missing muhash registration (MarcoFalke)

Pull request description:

  otherwise it is not possible to run the target

ACKs for top commit:
  laanwj:
    Code review ACK fa44417fcb

Tree-SHA512: b6495b84890a778d3569deddfc5703d1bed95d7d5c1eb0766fd5bd9afe86a3a0ccf4726aa16cba4a5fd182d23bfb7e815f3af4eb635915397c90805585b5699e
2021-01-08 14:00:17 +01:00
Hennadii Stepanov
76f52e3da3 build: Fix Qt processing of configure script for depends with DEBUG=1
Now, if depends is built with DEBUG=1, the configure script correctly
finds Qt for macOS and Windows.
2021-01-08 10:48:22 +02:00
MarcoFalke
fa44417fcb fuzz: Add missing muhash registration 2021-01-08 09:40:50 +01:00
MarcoFalke
faabc26a61 test: Replace getmempoolentry with testmempoolaccept in MiniWallet 2021-01-08 09:07:33 +01:00
MarcoFalke
5082324225 Merge #20688: test: run mempool_compatibility.py even with wallet disabled
a7599c80eb test: run mempool_compatibility.py even with wallet disabled (Michael Dietz)

Pull request description:

  Another functional test rewritten as proposed in https://github.com/bitcoin/bitcoin/issues/20078

ACKs for top commit:
  MarcoFalke:
    review ACK a7599c80eb didn't test

Tree-SHA512: cc7a617e5489ed27bbdbdee85a82fa08525375061f7f4524577a6b8ecb340396adac88419b51f513be22ca53edd0a3bd5d572d9f43ffc2c18550b0ef9069d238
2021-01-08 08:35:01 +01:00
Wladimir J. van der Laan
76b45d5fd7 Merge #19683: depends: Pin clang search paths for darwin host
196b727649 depends: Add comment about cache invalidation (Carl Dong)
949c480e52 depends: Fully determine path for darwin cctools (Carl Dong)
880660acfa depends: Fully determine path for darwin_{CC,CXX} (Carl Dong)
8033110741 depends: Quote to prevent word splitting in config.site (Carl Dong)
77b1ef89a0 depends: Remove -fuse-ld line (Carl Dong)
3007339218 depends: Pin clang search paths for darwin host (Carl Dong)
107f33d434 depends: Delay expansion of per-package vars (Carl Dong)

Pull request description:

  > Hello clang/lib/frontend,
  > I search your headers once again.
  > Because it's time for some housekeeping,
  > Within the code I was tweaking,
  > And the targets I was making with my build,
  > Are unfulfilled,
  > It's just language compliance.
  >
  > In reference works I scroll alone
  > Pages cribbed from holy tomes
  > In the details of a template
  > My code's behaviour has now found its fate
  > When my hopes were dashed as a note left it as described:
  > As undefined
  > It's not in compliance
  >
  > And from the standard text I saw
  > Ten thousand errors, maybe more
  > Threading used without locking
  > Pointers referenced after freeing
  > Linters writing warnings that coders will never fix
  > But still they tick
  > The box that claims compliance
  >
  > "Fools," said I, "you do not know"
  > Errors, like a cancer, grow
  > Hear my words that I might reach you
  > Use -Wall and it might teach you
  > But my words and compiler errors fade.
  > Schedules forbade compliance.
  >
  > And the people bowed and prayed
  > With static checking torn and frayed
  > The markets flashed out their warning
  > In the words that they were forming
  > As recruiters said "The search for more profits leads to writing stuff in CSS,
  > And node.js.
  > Without a need for compliance"

  Many thanks to ajtowns for the above contribution!

  -----

  This PR is ready for review!

  When cross-compiling for macOS, the SDK gives us the entire context/sysroot on which we should base the build. This means that we can be extremely specific w/re our search path ordering in order to avoid build problems that arise out of a user's specific environment/system setup and improve the robustness of our macOS toolchain. This PR does 2 things to this end:

  1. Unset environment variables which are known to alter search paths.
  1. Makes us (in the case of macOS builds) explicitly specify the list of system include search paths and its ordering, rather than rely on `clang`'s unreliable autodetection routine. Here is the [rabbit-hole gist](https://gist.github.com/dongcarl/5cdc6990b7599e8a5bf6d2a9c70e82f9).

      See the added comments in `depends/hosts/darwin.mk` for more details:

      8b8296dc70/depends/hosts/darwin.mk (L37-L60)

  We can be this specific _only_ because macOS builds are neatly contained in an SDK, **and** we are cross-compiling. Native toolchains should rely on the environment/distro/user to know how best to build for the running system.

  Note: Although the `-u` flag of `env` is not a POSIX standard flag, it seems like it is useful enough to be implemented in [coreutils](https://www.gnu.org/software/coreutils/manual/html_node/env-invocation.html), [busybox](https://busybox.net/downloads/BusyBox.html#env), [FreeBSD](https://www.freebsd.org/cgi/man.cgi?env).

ACKs for top commit:
  laanwj:
    code review ACK 196b727649

Tree-SHA512: 406442df16d9aa0aef62f9fa94f72d7e48374301f3d826bf32f183e1610942aa44a4adfac7bead1f14aded0044fac400e1328fcd933b2337e55a024f034b5013
2021-01-08 06:10:24 +01:00
Wladimir J. van der Laan
86a8b35f32 Merge #14501: Fix possible data race when committing block files
ef712298c3 util: Check for file being NULL in DirectoryCommit (Luke Dashjr)
4574904038 Fix possible data race when committing block files (Evan Klitzke)
220bb16cbe util: Introduce DirectoryCommit commit function to sync a directory (Evan Klitzke)
ce5cbaea63 util.h: Document FileCommit function (Evan Klitzke)
844d650eea util: Prefer Mac-specific F_FULLSYNC over fdatasync in FileCommit (Evan Klitzke)
f6cec0bcaf util: Refactor FileCommit from an #if sequence nested in #else, to a sequence of #elif (Evan Klitzke)

Pull request description:

  Reviving #12696

ACKs for top commit:
  laanwj:
    Code review ACK ef712298c3

Tree-SHA512: 07d650990ef4c18d645dee3f9a199a940683ad17557d79d93979a76c4e710d8d70e6eae01d1a5991494a24a7654eb7db868be0c34a31e70b2509945d95bc9cce
2021-01-07 22:07:33 +01:00
Andrew Chow
2c403279e2 gitian: Remove codesign_allocate and pagestuff from MacOS build 2021-01-07 15:33:19 -05:00
Andrew Chow
f55eed2514 gitian: use signapple to create the MacOS code signature 2021-01-07 15:33:19 -05:00
Andrew Chow
95b06d2185 gitian: use signapple to apply the MacOS code signature 2021-01-07 15:33:19 -05:00
Andrew Chow
42bb1ea363 gitian: install signapple in gitian-osx-signer.yml 2021-01-07 15:33:19 -05:00
Carl Dong
196b727649 depends: Add comment about cache invalidation 2021-01-07 14:24:06 -05:00
Carl Dong
949c480e52 depends: Fully determine path for darwin cctools
See previous commit for description.
2021-01-07 14:24:06 -05:00
Carl Dong
880660acfa depends: Fully determine path for darwin_{CC,CXX}
Instead of doing the awkward /bin path prepending at config.site
creation time, set darwin_{CC,CXX} in a way that fully determines the
program's path (clang/clang++) similar to how AC_PATH_{TOOL,PROG} would
do.

Also see the added comment block in depends/Makefile for more context on
determining $PATH for our config.site.
2021-01-07 14:22:36 -05:00
Carl Dong
8033110741 depends: Quote to prevent word splitting in config.site
SC2086 is disabled in our linter script so this wasn't caught.
2021-01-07 14:04:33 -05:00
Carl Dong
77b1ef89a0 depends: Remove -fuse-ld line
clang warns when a command line option is unused, and some of our tests
use Werror, so unfortunately we cannot use this flag to pin our linker
for now. Leaving this commit in for future reference, as it would be
great if there's more granularity to Werror and we can be explicit about
what linker we want to use.
2021-01-07 14:02:39 -05:00
Carl Dong
3007339218 depends: Pin clang search paths for darwin host 2021-01-07 14:02:39 -05:00
Carl Dong
107f33d434 depends: Delay expansion of per-package vars
Prior to this commit, when int_vars was called for packages, it would
immediately expand the "single-dollar variables", which may be defined
in terms of variables which are not yet determined (e.g. variables
defined in package/*.mk, which are included after int_vars is called).

This is required for the next commit as after that commit, for darwin
cross-builds:

0. int_vars is defined in terms of $(1)_cc
1. $(1)_cc is defined in terms of darwin_CC
2. ... which is defined in terms of clang_resource_dir
3. ... which is defined in terms of native_cctools_clang_version
4. which is undetermined at the time when int_vars is being expanded and evaluated
2021-01-07 14:02:39 -05:00
Wladimir J. van der Laan
d7e2401c62 Merge #18077: net: Add NAT-PMP port forwarding support
a191e23b8e doc: Add release notes (Hennadii Stepanov)
ae749d12dd doc: Add libnatpmp stuff (Hennadii Stepanov)
e28f9be87a ci: Add libnatpmp-dev package to some builds (Hennadii Stepanov)
5a0185b6c9 gui: Add NAT-PMP network option (Hennadii Stepanov)
a39f7336a3 net: Add -natpmp command line option (Hennadii Stepanov)
28acffd9d5 net: Add NAT-PMP to port mapping loop (Hennadii Stepanov)
a8d9f275d0 net: Add libnatpmp support (Hennadii Stepanov)
58e8364dcd gui: Apply port mapping changes on dialog exit (Hennadii Stepanov)
cf151cc68c scripted-diff: Rename UPnP stuff (Hennadii Stepanov)
4e91b1e24d net: Add flags for port mapping protocols (Hennadii Stepanov)
8b50d1b5bb net: Keep trying to use UPnP when -upnp=1 (Hennadii Stepanov)
28e2961fd6 refactor: Replace magic number with named constant (Hennadii Stepanov)
02ccf69dd6 refactor: Move port mapping code to its own module (Hennadii Stepanov)

Pull request description:

  Close #11902
  This PR is an alternative to:
  - #12288
  - #15717

  To compile with NAT-PMP support on Ubuntu [`libnatpmp-dev`](https://packages.ubuntu.com/source/bionic/libnatpmp) should be available.

  Log excerpt:
  ```
  2020-02-05T20:12:28Z [mapport] NAT-PMP: public address = 95.164.65.194
  2020-02-05T20:12:28Z [mapport] AddLocal(95.164.65.194:18333,3)
  2020-02-05T20:12:28Z [mapport] NAT-PMP: port mapping successful.
  ```

  See: [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html)

  ---

  Some follow-ups are out of this PR's scope:
  - mention NAT-PMP library in the version message
  - ~integrate NAT-PMP into the GUI~ (already [added](https://github.com/bitcoin/bitcoin/pull/18077#issuecomment-589405068))

ACKs for top commit:
  laanwj:
    Tested and code review ACK a191e23b8e

Tree-SHA512: 10e19267c21bf30f20ff1abfc882d526049f0e790b95e12f109dc2bed7c0aef45de03eaf967f4e667e7509be04f1873a5c508087393d947205f3aab2ad6d7cf1
2021-01-07 19:41:55 +01:00
Michael Dietz
a7599c80eb test: run mempool_compatibility.py even with wallet disabled 2021-01-07 12:24:24 -06:00
Amiti Uttarwar
b4dd2ef800 [test] Test the add_outbound_p2p_connection functionality
Open max number of full-relay and block-relay-only connections from a
functional test with different sorts of behaviors to ensure it behaves as
expected.
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
602e69e427 [test] P2PBlocksOnly - Test block-relay-only connections.
Ensure we will disconnect if the peer sends us a transaction & we don't
announce transactions to the peer.
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
8bb6beacb1 [test/refactor] P2PBlocksOnly - Extract transaction violation test into helper.
This is in preparation for use in the next commit.
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
99791e7560 [test/refactor] P2PBlocksOnly - simplify transaction creation using blocktool helper. 2021-01-07 10:15:56 -08:00
Amiti Uttarwar
3997ab9154 [test] Add test framework support to create outbound connections.
In the interest of increasing our P2P test coverage, add support to create
full-relay or block-relay-only connections. To support this, a P2P connection
spins up a listening thread & uses a callback to trigger the node initiating
the connection.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
5bc04e8837 [rpc/net] Introduce addconnection to test outbounds & blockrelay
Add a new RPC endpoint to enable opening outbound connections from
the tests. The functional test framework currently uses the addnode RPC, which
has different behavior than general outbound peers. These changes enable
creating both full-relay and block-relay-only connections. The new RPC
endpoint calls through to a newly introduced AddConnection method on
CConnman that ensures we stay within the allocated max.
2021-01-07 10:15:56 -08:00
MarcoFalke
fad327ca65 fuzz: net permission flags in net processing 2021-01-07 19:07:02 +01:00
Wladimir J. van der Laan
b6a71b80d2 Merge #19055: Add MuHash3072 implementation
9815332d51 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr)
01297fb3ca fuzz: Add MuHash consistency fuzz test (Fabian Jahr)
b111410914 test: Add MuHash3072 fuzz test (Fabian Jahr)
c122527385 bench: Add Muhash benchmarks (Fabian Jahr)
7b1242229d test: Add MuHash3072 unit tests (Fabian Jahr)
adc708c98d crypto: Add MuHash3072 implementation (Fabian Jahr)
0b4d290bf5 crypto: Add Num3072 implementation (Fabian Jahr)
589f958662 build: Check for 128 bit integer support (Fabian Jahr)

Pull request description:

  This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`.

ACKs for top commit:
  laanwj:
    Code review ACK 9815332d51

Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
2021-01-07 17:57:17 +01:00
Hennadii Stepanov
a191e23b8e doc: Add release notes 2021-01-07 18:07:11 +02:00
Hennadii Stepanov
ae749d12dd doc: Add libnatpmp stuff 2021-01-07 18:07:10 +02:00
Hennadii Stepanov
e28f9be87a ci: Add libnatpmp-dev package to some builds 2021-01-07 18:07:10 +02:00
Hennadii Stepanov
5a0185b6c9 gui: Add NAT-PMP network option 2021-01-07 18:07:10 +02:00
Hennadii Stepanov
a39f7336a3 net: Add -natpmp command line option 2021-01-07 18:07:09 +02:00
Hennadii Stepanov
28acffd9d5 net: Add NAT-PMP to port mapping loop 2021-01-07 18:07:09 +02:00
Hennadii Stepanov
a8d9f275d0 net: Add libnatpmp support 2021-01-07 18:07:09 +02:00
Hennadii Stepanov
58e8364dcd gui: Apply port mapping changes on dialog exit
This commit does not change behavior. It is a prerequisite for NAT-PMP
support adding.
2021-01-07 18:07:08 +02:00
Hennadii Stepanov
cf151cc68c scripted-diff: Rename UPnP stuff
-BEGIN VERIFY SCRIPT-
sed -i 's/g_upnp_interrupt/g_mapport_interrupt/' src/mapport.cpp
sed -i 's/if(g_upnp_thread/if (g_mapport_thread/' src/mapport.cpp
sed -i 's/g_upnp_thread/g_mapport_thread/' src/mapport.cpp
sed -i 's/LOCAL_UPNP/LOCAL_MAPPED/' src/mapport.cpp
sed -i 's/\bupnp\b/mapport/' src/mapport.cpp
sed -i 's/LOCAL_UPNP,  /LOCAL_MAPPED,/' src/net.h
-END VERIFY SCRIPT-
2021-01-07 18:07:08 +02:00
Hennadii Stepanov
4e91b1e24d net: Add flags for port mapping protocols 2021-01-07 18:07:08 +02:00
Hennadii Stepanov
8b50d1b5bb net: Keep trying to use UPnP when -upnp=1 2021-01-07 18:07:07 +02:00
Hennadii Stepanov
28e2961fd6 refactor: Replace magic number with named constant 2021-01-07 18:07:07 +02:00
Hennadii Stepanov
02ccf69dd6 refactor: Move port mapping code to its own module
This commit does not change behavior.
2021-01-07 18:06:58 +02:00
MarcoFalke
3a6acd1772 Merge #20789: fuzz: Rework strong and weak net enum fuzzing
eeee43bc48 fuzz: Use ConsumeWeakEnum for ServiceFlags (MarcoFalke)
fa9949b914 fuzz: Add ConsumeWeakEnum helper, Extract ALL_NET_PERMISSION_FLAGS (MarcoFalke)
faaef9434c fuzz: [refactor] Extract ALL_CONNECTION_TYPES constant (MarcoFalke)
fa42da2d54 fuzz: Use ConsumeNode in process_message target (MarcoFalke)
fa121f058f fuzz: Use ConsumeNode in process_messages target (MarcoFalke)

Pull request description:

  The fuzz tests have several problems:
  * The array passed to the fuzz engine to pick `net_permission_flags` is outdated
  * The process_message* targets has the service flags as well as connection type hardcoded, limiting potential coverage
  * The service flags deserialization from the fuzz engine doesn't allow for easy "exact matches". The fuzz engine has to explore a 64-bit space to hit an "exact match" (only one bit set)

  Fix all issues in the commits in this pull

ACKs for top commit:
  mzumsande:
    ACK eeee43bc48 after rebase.

Tree-SHA512: 1ad9520c7e708b7f4994ae8f77886ffca33d7c542756e2a3e07dbbbe59e360f9fcaccf2e2fb57d9bc731d4aeb4938fb1c5c546e9d2744b007af5626f5cb377fe
2021-01-07 17:04:56 +01:00
MarcoFalke
fad1f0fd33 net: Remove unused cs_feeFilter 2021-01-07 15:25:47 +01:00
Jon Atack
faecb74562 Expose integral m_conn_type in CNodeStats, remove m_conn_type_string 2021-01-07 15:18:44 +01:00
MarcoFalke
4b8b71e630 Merge bitcoin-core/gui#173: Follow Qt docs when implementing rowCount and columnCount
195fcb53a0 qt: Follow Qt docs when implementing rowCount and columnCount (Hennadii Stepanov)

Pull request description:

  [`QAbstractItemModel::rowCount`](https://doc.qt.io/qt-5/qabstractitemmodel.html#rowCount):
  > **Note:** When implementing a table based model, `rowCount()` should return 0 when the parent is valid.

  [`QAbstractItemModel::columnCount`](https://doc.qt.io/qt-5/qabstractitemmodel.html#columnCount):
  > **Note:** When implementing a table based model, `columnCount()` should return 0 when the parent is valid.

ACKs for top commit:
  jarolrod:
    Tested ACK 195fcb53a0. Compiled and ran on macOS (Big Sur 11.1 and Catalina 10.15.7), Arch Linux, and FreeBSD. visually verified no weird effects with the `Address`, `Ban`, `Peer`, and `Transaction` tables. As already stated, the code change brings us inline with what the QT Docs recommend.

Tree-SHA512: 179a3430e68e77b22cdf642964cd96c023a2286ee256bbeb25b43df3d2eef6f59978c8d92173c6be5071d127fdcd6aa338142f6eaf003ff08e4abd65172d20ca
2021-01-07 14:59:15 +01:00
MarcoFalke
42675e7833 Merge #20864: net: Move SocketSendData lock annotation to header
fa210689e2 net: Move SocketSendData lock annotation to header (MarcoFalke)
fa0a71781a net: Move CConnman/NetEventsInterface after CNode in header file (MarcoFalke)

Pull request description:

  Lock annotations must be in the header, otherwise the will have limited or no effect

ACKs for top commit:
  jnewbery:
    utACK fa210689e2

Tree-SHA512: c25aac793662227f2bb163e2f98cd38e89b43a03bad925d192d6166dee76a456110eaf55be5e4b4f2ddcf20bde7e3fc82a7a4338670a1f2d027d01b4a8908303
2021-01-07 14:57:47 +01:00
Wladimir J. van der Laan
efe03ceb58 Merge #20859: gitian-keys: add miketwenty1 key
f2d93b229d gitian-keys: add miketwenty1 key (Michael Tidwell)

Pull request description:

  in ref to:
  bitcoin-core/gitian.sigs#1427

ACKs for top commit:
  laanwj:
    ACK f2d93b229d

Tree-SHA512: 895935981ada0c3a094adde556b86b73dc440b599f8d6e2a7dd1e82592351c820b797c64a3e0fa1e79ab4c2f992afb3363b4a12625a54bdfbdea75b5a19d7e95
2021-01-07 13:42:49 +01:00
MarcoFalke
fa210689e2 net: Move SocketSendData lock annotation to header
Also, add lock annotation to SendMessages

Can be reviewed with "--word-diff-regex=."
2021-01-07 09:41:34 +01:00
MarcoFalke
fa0a71781a net: Move CConnman/NetEventsInterface after CNode in header file
Can be reviewed with --color-moved=dimmed-zebra --patience
2021-01-07 09:40:49 +01:00
MarcoFalke
8a720ced5f Merge bitcoin-core/gui#165: Save QSplitter state in QSettings
90f9fc274b qt: Save QSplitter state in QSettings (Hennadii Stepanov)

Pull request description:

  This PR adds the ability to save the `QSplitter` widget state in `QSettings` during shutdown, and restore it on startup.

  A user no longer needs to adjust the splitter every time :)
  ![DeepinScreenshot_select-area_20201225211422](https://user-images.githubusercontent.com/32963518/103141024-046c3980-46f7-11eb-9a8c-83613527ffe1.png)

ACKs for top commit:
  jonasschnelli:
    utACK 90f9fc274b
  jonatack:
    ACK 90f9fc274b this sets the "PeersTabSplitterSizes" value in the RPCConsole dtor and restores it in the RPCConsole ctor; tested in Debian with various split settings, tab open/close sequences, and shutdown methods, and the Peers window split state was faithfully maintained.

Tree-SHA512: efbd6a4cee512982944955d36775e75a8a217b1dc49e62d42c6e402d2710dd44324b2c3c1edeb5fe38d9229e0e4a39734d1f4e63405ade8694762e1bbf72020b
2021-01-07 09:09:16 +01:00
MarcoFalke
f13e03cda2 Merge #20584: Declare de facto const reference variables/member functions as const
31b136e580 Don't declare de facto const reference variables as non-const (practicalswift)
1c65c075ee Don't declare de facto const member functions as non-const (practicalswift)

Pull request description:

  _Meta: This is the second and final part of the `const` refactoring series (part one: #20581). **I promise: no more refactoring PRs from me in a while! :)** I'll now go back to focusing on fuzzing/hardening!_

  Changes in this PR:
  * Don't declare de facto const member functions as non-const
  * Don't declare de facto const reference variables as non-const

  Awards for finding candidates for the above changes go to:
  * `clang-tidy`'s [`readability-make-member-function-const`](https://clang.llvm.org/extra/clang-tidy/checks/readability-make-member-function-const.html)  check ([list of `clang-tidy` checks](https://clang.llvm.org/extra/clang-tidy/checks/list.html))
  * `cppcheck`'s `constVariable` check ([list of `cppcheck` checks](https://sourceforge.net/p/cppcheck/wiki/ListOfChecks/))

  See #18920 for instructions on how to analyse Bitcoin Core using Clang Static Analysis, `clang-tidy` and `cppcheck`.

ACKs for top commit:
  ajtowns:
    ACK 31b136e580
  jonatack:
    ACK 31b136e580
  theStack:
    ACK 31b136e580 ❄️

Tree-SHA512: f58f8f00744219426874379e9f3e9331132b9b48e954d24f3a85cbb858fdcc98009ed42ef7e7b4619ae8af9fc240a6d8bfc1c438db2e97b0ecd722a80dcfeffe
2021-01-07 09:05:09 +01:00
Hennadii Stepanov
d4feb6812a qt: Use layout manager for Create Wallet dialog 2021-01-06 23:35:24 +02:00
Michael Tidwell
f2d93b229d gitian-keys: add miketwenty1 key 2021-01-06 10:12:22 -05:00
Wladimir J. van der Laan
3b6d1b61d3 Merge #20829: doc: add -netinfo help
6f2c4fd077 netinfo: add user help documentation (Jon Atack)

Pull request description:

  This is the help doc commit of #20764 without the rest of the PR or anything new since the 0.21.0 branch-off in order to target giving users a -netinfo help doc for 0.21.

  - to test the new help
  ```
  $ ./src/bitcoin-cli -netinfo help
  ```
  - to see the updated short help
  ```
  $ ./src/bitcoin-cli -help | grep -A4 netinfo
  ```

  <details><summary><code>-netinfo</code> help doc</summary><p>

  ```
  $ ./src/bitcoin-cli -netinfo help
  -netinfo level "help"

  Returns a network peer connections dashboard with information from the remote server.
  Under the hood, -netinfo fetches the data by calling getpeerinfo and getnetworkinfo.
  An optional integer argument from 0 to 4 can be passed for different peers listings.
  Pass "help" to see this detailed help documentation.
  If more than one argument is passed, only the first one is read and parsed.
  Suggestion: use with the Linux watch(1) command for a live dashboard; see example below.

  Arguments:
  1. level (integer 0-4, optional)  Specify the info level of the peers dashboard (default 0):
                                    0 - Connection counts and local addresses
                                    1 - Like 0 but with a peers listing (without address or version columns)
                                    2 - Like 1 but with an address column
                                    3 - Like 1 but with a version column
                                    4 - Like 1 but with both address and version columns
  2. help (string "help", optional) Print this help documentation instead of the dashboard.

  Result:

  * The peers listing in levels 1-4 displays all of the peers sorted by direction and minimum ping time:

    Column   Description
    ------   -----------
    <->      Direction
             "in"  - inbound connections are those initiated by the peer
             "out" - outbound connections are those initiated by us
    type     Type of peer connection
             "full"   - full relay, the default
             "block"  - block relay; like full relay but does not relay transactions or addresses
    net      Network the peer connected through ("ipv4", "ipv6", "onion", "i2p", or "cjdns")
    mping    Minimum observed ping time, in milliseconds (ms)
    ping     Last observed ping time, in milliseconds (ms)
    send     Time since last message sent to the peer, in seconds
    recv     Time since last message received from the peer, in seconds
    txn      Time since last novel transaction received from the peer and accepted into our mempool, in minutes
    blk      Time since last novel block passing initial validity checks received from the peer, in minutes
    age      Duration of connection to the peer, in minutes
    asmap    Mapped AS (Autonomous System) number in the BGP route to the peer, used for diversifying
             peer selection (only displayed if the -asmap config option is set)
    id       Peer index, in increasing order of peer connections since node startup
    address  IP address and port of the peer
    version  Peer version and subversion concatenated, e.g. "70016/Satoshi:21.0.0/"

  * The connection counts table displays the number of peers by direction, network, and the totals
    for each, as well as a column for block relay peers.

  * The local addresses table lists each local address broadcast by the node, the port, and the score.

  Examples:

  Connection counts and local addresses only
  > bitcoin-cli -netinfo

  Compact peers listing
  > bitcoin-cli -netinfo 1

  Full dashboard
  > bitcoin-cli -netinfo 4

  Full live dashboard, adjust --interval or --no-title as needed (Linux)
  > watch --interval 1 --no-title ./src/bitcoin-cli -netinfo 4

  See this help
  > bitcoin-cli -netinfo help

  ```
  </p></details>

ACKs for top commit:
  laanwj:
    ACK 6f2c4fd077

Tree-SHA512: dd49b1ce65546dacfb8ba9f9d57de0eae55560fd05533cf26c0b5d6ec65bf1de789c3287e90a0e2f47707532fab2fe62919a4192a7ffd58ac8eec18293e9aaeb
2021-01-06 16:11:09 +01:00
Wladimir J. van der Laan
68196a8910 Merge #20860: gitian-keys: add key for guggero
c3aa1a16b2 gitian-keys: add key for guggero (Oliver Gugger)

Pull request description:

  As asked in bitcoin-core/gitian.sigs#1427

  Full key(s) available at https://keybase.io/guggero/pgp_keys.asc

  ```
  -----BEGIN PGP SIGNED MESSAGE-----
  Hash: SHA512

  My key is F4FC70F07310028424EFC20A8E4256593F177720
  (Oliver Gugger <gugger@gmail.com>)
  -----BEGIN PGP SIGNATURE-----

  iQEzBAEBCgAdFiEEbgHuyWVpA7BUK48QA9tjIiZ8NzsFAl/0uXEACgkQA9tjIiZ8
  NzsMeQgAiuj0PqUQYRn3xyxdqAGpN/JdUgCC0HjolW44kDPvij1Tb1ZrvSQD1Amh
  H0XLca+eROGmdgSMNFIBcsh4/Ikh11hRVsbDtf2v2Q3xuM6abklzCFZto47QFgf6
  PZZIVUu9MespkOXOa8NK+nhsq241S6gp+dJkd39sW0q1+B7/7zVoV1v2tSCxkKeI
  x8TbHfH+BGhB0I7Kwanz/tz5rX5U7GdF8ajjKg4p/Z303UP6+j+j5eT4b8WJKVKN
  9N9mtx9h0sgo/i3Vk6KZ7Y74YYH3e3FL7ZVaoWxZgDPX79724F2l7K8n3q60F2X7
  jhB7uMTZ2trD7A2qtrpK7HnRVYlbVA==
  =AOM6
  -----END PGP SIGNATURE-----
  ```

ACKs for top commit:
  laanwj:
    ACK c3aa1a16b2

Tree-SHA512: 494b22bccb38b517fec642828d77e0ef50b07527d803be7374da1c54295e0e8958eee8a9ad0bbfbd47d4f553507011ac4417b02694a878ff4ce78639baa1801d
2021-01-06 08:49:18 +01:00
Wladimir J. van der Laan
e520e091db Merge #20844: test: Add sanitizer suppressions for AMD EPYC CPUs
fa6c114ae6 test: Add sanitizer suppressions for AMD EPYC CPUs (MarcoFalke)

Pull request description:

  Currently the ci system only runs on intel cpus (and some arm devices), but it won't run on CPUs `Using the 'shani(1way,2way)' SHA256 implementation` (excerpt from debug log).

  For reference, google cloud CPUs (which is what Cirrus CI uses) print `Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`

  The traceback I got:

  ```
  crypto/sha256_shani.cpp:87:18: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'size_t' (aka 'unsigned long')
      #0 0x55c0000e95ec in sha256_shani::Transform(unsigned int*, unsigned char const*, unsigned long) /root/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/crypto/sha256_shani.cpp:87:18
      #1 0x55bfffb926f8 in (anonymous namespace)::SelfTest() /root/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/crypto/sha256.cpp:517:9
      #2 0x55bfffb906ed in SHA256AutoDetect[abi:cxx11]() /root/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/crypto/sha256.cpp:626:5
      #3 0x55bfff87ab97 in BasicTestingSetup::BasicTestingSetup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&) /root/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/util/setup_common.cpp:104:5
      #4 0x55bffe885877 in main /root/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/test/test_main.cpp:52:27
      #5 0x7f20c3bf60b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
      #6 0x55bffe7a5f6d in _start (/root/bitcoin/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/qt/test/test_bitcoin-qt+0x1d00f6d)

  SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow crypto/sha256_shani.cpp:87:18 in

ACKs for top commit:
  laanwj:
    Anyhow ACK fa6c114ae6

Tree-SHA512: 968a1d28eedec58c337b1323862f583cb1bcd78c5f03396940b9ab53ded12f8c6652877909aba05ee5586532137418fd817ff979bd7bef6e07856094f9d7f9b1
2021-01-06 08:36:06 +01:00
MarcoFalke
4eada5d8b1 Merge #20816: net: Move RecordBytesSent() call out of cs_vSend lock
378aedc452 [net] Add cs_vSend lock annotations (John Newbery)
673254515a [net] Move RecordBytesSent() call out of cs_vSend lock (John Newbery)

Pull request description:

  RecordBytesSent() does not require cs_vSend to be locked, so reduce the scope of cs_vSend.

  Also correctly annotate the CNode data members that are guarded by cs_vSend.

  This is a simpler alternative to #19673.

ACKs for top commit:
  jnewbery:
    ok, reverting to commit 378aedc which has two ACKs already. Any style issues can be fixed up in future PRs.
  troygiorshev:
    ACK 378aedc452
  theStack:
    re-ACK 378aedc452
  MarcoFalke:
    review ACK 378aedc452 🔌

Tree-SHA512: e9cd6c472b7e1479120c1bf2d1c640cf6d18c7d589a5f9b7dfc4875e5790adaab403a7a1b945a47e79e7249a614b8583270e4549f89b22e8a9edb2e4818b0d07
2021-01-06 07:07:35 +01:00
Hennadii Stepanov
f0d04795e2 qt: Fix TxViewDelegate layout
This change (1) prevents overlapping date and amount strings,
and (2) guaranties that "eye" sign at the end of the watch-only
address/label is always visible.
2021-01-05 22:47:12 +02:00
Jon Atack
6f2c4fd077 netinfo: add user help documentation
and drop no longer needed sort description header to save screen space
2021-01-05 21:31:48 +01:00
Hennadii Stepanov
d439921406 qt: Add TransactionOverviewWidget class 2021-01-05 22:10:10 +02:00
Oliver Gugger
c3aa1a16b2 gitian-keys: add key for guggero
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

My key is F4FC70F07310028424EFC20A8E4256593F177720
(Oliver Gugger <gugger@gmail.com>)
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEbgHuyWVpA7BUK48QA9tjIiZ8NzsFAl/0uXEACgkQA9tjIiZ8
NzsMeQgAiuj0PqUQYRn3xyxdqAGpN/JdUgCC0HjolW44kDPvij1Tb1ZrvSQD1Amh
H0XLca+eROGmdgSMNFIBcsh4/Ikh11hRVsbDtf2v2Q3xuM6abklzCFZto47QFgf6
PZZIVUu9MespkOXOa8NK+nhsq241S6gp+dJkd39sW0q1+B7/7zVoV1v2tSCxkKeI
x8TbHfH+BGhB0I7Kwanz/tz5rX5U7GdF8ajjKg4p/Z303UP6+j+j5eT4b8WJKVKN
9N9mtx9h0sgo/i3Vk6KZ7Y74YYH3e3FL7ZVaoWxZgDPX79724F2l7K8n3q60F2X7
jhB7uMTZ2trD7A2qtrpK7HnRVYlbVA==
=AOM6
-----END PGP SIGNATURE-----
2021-01-05 20:06:33 +01:00
MarcoFalke
417f95fa45 Merge #19915: p2p, refactor: Use Mutex type for some mutexes in CNode class
0e51a35512 refactor: Use Mutex type for some mutexes in CNode class (Hennadii Stepanov)

Pull request description:

  No need the `RecursiveMutex` type for the `CNode::cs_vSend`, `CNode::cs_hSocket` and `CNode::cs_vRecv`.

  Related to #19303.

ACKs for top commit:
  jnewbery:
    utACK 0e51a35512
  MarcoFalke:
    review ACK 0e51a35512 🔊

Tree-SHA512: 678ee5e3c15ad21a41cb86ec7179741bd505a138638fdc07f41d6d677c38fbf2208219bfc0509e3675e721fc8d8816e858070db7b87c5d72ad93aae81f7e1636
2021-01-05 12:19:08 +01:00
MarcoFalke
fafd725a7c Merge #19846: build: enable unused member function diagnostic
819d03b932 refactor: took out unused member functions (Zero)
ed69213c2b build: enable unused member function diagnostic (Zero)

Pull request description:

  This PR enables the `-Wunused-member-function` compiler diagnostic, as discussed in #19702.

  > **Notice**: The `unused-member-function` diagnostic is only available on clang. Therefore, clang should be used to test this PR.

  - [x] Include the `-Wunused-member-function`diagnostic in `./configure.ac`. (ed69213c2b)
  - [x] Resolve the reported warnings. (819d03b932)

  Currently, enabling this flag no longer reports the following warnings:

  > **Note**: output from `make 2>&1 | grep "warning: unused member function" | sort | uniq -c`

  ```
  1 index/blockfilterindex.cpp:54:5: warning: unused member function 'DBHeightKey' [-Wunused-member-function]
  2 script/bitcoinconsensus.cpp:50:9: warning: unused member function 'GetType' [-Wunused-member-function]
  1 test/util_tests.cpp:1975:14: warning: unused member function 'operator=' [-Wunused-member-function]
  ```

  All tests have passed locally (from `make check` & `src/test/test_bitcoin`).

  This PR closes #19702.

ACKs for top commit:
  practicalswift:
    ACK 819d03b932 - patch still looks correct :)
  MarcoFalke:
    ACK 819d03b932
  pox:
    Tested ACK 819d03b932 with clang after `make clean`. No unused member function warnings.
  theStack:
    tested ACK 819d03b932

Tree-SHA512: 5fdfbbb02b3dc618a90a874a5caa5e01e596fc1d14a209e75a6981f01b253f9bca0cfac8fdd758dd7151986609fb76571c3745124a29cfd4f8cbb8d82a07272e
2021-01-05 12:06:40 +01:00
MarcoFalke
34322b7f5c Merge #20842: docs: consolidate typo & url fixing
1112035d32 doc: fix various typos (Ikko Ashimine)
e8640849c7 doc: Use https URLs where possible (Sawyer Billings)

Pull request description:

  Consolidates / fixes the changes from #20762, #20836, #20810. There is no output when  `test/lint/lint-all.sh` is run.

  Closes #20807.

ACKs for top commit:
  MarcoFalke:
    ACK 1112035d32

Tree-SHA512: 22ca824688758281a74e5ebc6a84a358142351434e34c88c6b36045d2d241ab95fd0958565fd2060f98317e62e683323b5320cc7ec13592bf340e6922294ed78
2021-01-05 11:53:20 +01:00
MarcoFalke
c37600777e Merge #20854: [mempool] Remove unnecessary try-block
7ff05358a9 [mempool] Remove error suppression on upgrade (Amiti Uttarwar)

Pull request description:

  In 0.21, we added unbroadcast txids to mempool.dat (#18038). When users upgraded from 0.21 to 0.22, this would throw a misleading "failed to deserialize mempool data" error even though everything actually loaded properly. So, commit 9c8a55d added a try-block to prevent throwing the error. After upgrading to 0.22, this exception handling is no longer useful, so now we can remove it.

ACKs for top commit:
  MarcoFalke:
    review ACK 7ff05358a9
  theStack:
    Code review ACK 7ff05358a9

Tree-SHA512: 0444eea2b1326904f9855fd0af6669a4990f0427cf7c9293252a5b7049cdcc785bdf9398fd08ed8dedacfdd78e75039ddf1087b3654c558ff52498df15f05daf
2021-01-05 11:48:03 +01:00
MarcoFalke
faa8f68943 Replace boost::variant with std::variant 2021-01-05 10:10:50 +01:00
MarcoFalke
b40254b232 Merge #20855: Revert "Add patch to make codesign_allocate compatible with Apple's"
a0eb4c551e Revert "Add patch to make codesign_allocate compatible with Apple's" (Pieter Wuille)

Pull request description:

  This reverts #20644.

  It appears that Apple has recently changed their `codesign_allocate` tool back to using 4k alignment on x86_64, at least in some cases, so this patch isn't causing our cctools-based version to be exactly compatible.

  Furthermore, if codesigning were to change to use https://github.com/achow101/signapple instead, there is no need anymore to try to mimick Apple.

ACKs for top commit:
  laanwj:
    ACK a0eb4c551e
  MarcoFalke:
    checked-clean-revert ACK a0eb4c551e
  jonasschnelli:
    ACK a0eb4c551e

Tree-SHA512: 529719a76811006122406689233d1e80995107fe1ac1fc862a4ac53ca21685748ed76cac7ca648dd70f0ea43dd8dcf2e29d559beeab10e1d30dc5542ac95fd97
2021-01-05 09:39:29 +01:00
fanquake
ff6adac5f4 Merge #20847: gitian-keys: add darosior's key
d825a3957e gitian-keys: add darosior's key (Antoine Poinsot)

Pull request description:

  As per https://github.com/bitcoin-core/gitian.sigs/issues/1427

ACKs for top commit:
  fanquake:
    ACK d825a3957e

Tree-SHA512: d5ad39e0484d3d244911fd152d75f82141ca8ac4a7f21955cd428eee7433869b76026105a83bf50319aca0518bc96d5a1ed087c263c31bdfbf89a583e94bc1f5
2021-01-05 10:18:24 +08:00
Antoine Poinsot
d825a3957e gitian-keys: add darosior's key
-----BEGIN PGP PUBLIC KEY BLOCK-----

mQGNBFvwGvYBDADiZ/8Vi95xgj1GX/1Bot+y+MM/OSoJaXXQRnBGUjO5WLGG79Wh
cdQR53JZ7+MN7kHk/R7ZNjtTJdQi3HYfeDUQdQpknuNJN3qLrLuZ93jIhagvKfmJ
MpXhxikz/D3O67kLmbh+qUEhyMB8SvyUxbAaA/5laLeAdxeFWuVvFmngpnRKIAoa
k2PgPEO36+ghAb72GjSHtSFFIK+zn/dimHpLIOjKvERLoWGUSaXo/M5Uy1r67gke
TA3MyMGe1TU2SyMZEZR6LLOtPs0kxf5eU4blEpWs26/UdIAfKfayTnkvqpBHI1TR
6wbH7HlQVB8I1QrDg51oFDZ6DazKyNgnIAlWiM10Q2PlUunnT7NFfPSgDkAw/tR/
xsAMFbPT5NAD4TYqnK3NBlSTVNTmvyyy7EwQv0b1kO2fM/IYZ8XsJtCVJvaNXSbW
VoueDX5ZQ5z3ja/Y/Y7SSUbdJcMiNYrQZ/GbvwiNvjkmBR9UCBi7fSeRMQgl6tIK
ctIwqb7a4UjO42cAEQEAAbQpQW50b2luZSBQb2luc290IDxkYXJvc2lvckBwcm90
b25tYWlsLmNvbT6JAdcEEwEIAEEFCQYonioFCwkIBwIGFQoJCAsCBBYCAwECHgEC
F4ACGQEWIQRZC3KSaVr/pbZyy7LhP8FFzT9DBAUCX8I0iwIbDwAKCRDhP8FFzT9D
BKOQDACedmkzuKME1kS3heZGx7fezOJF9vNUuqKiP8eSCauZrWZCe02biyB5MGF6
20nijrxIpsiOfcr2iJJGvk4FQxmuZUFwM+r3+lJZTmBiEMWk9U/G/FLHEbTgIMOT
ehSzQZmp4S98FcChvNdxdXBMZMf/xrDVVqBWk8oBTEkQipVuQp3Jx8phl2ICCkU5
ni9e5ZMVI3zkdSm2DmlhK08tW+5iMczeGt9n9LMr2w3AVRPVIfIhYTf4mwCVfoLF
ScpurBljyVNm1Rwjs63hdN01jx27pSqDIntGTS/R8hJZrxNEn1Tb5zXGHhtDDZ3i
CTb0ANw32bQoQEJOdCHjryWQnFUn81MP1uf5NoKwsWYqHU8JU5dt4egZslwGrg6w
32jGG3fonrcRMblrO3dxd2LGwFpim4cQ7vxwdqaotDs6Wo3NSGkjrDaN0p7Lt2VO
LkIMWm1IYyW9gGHYfaw7XYSA/Nj4SAqkQTX+yHBidl5rBSBNM2u6nGLOCSidWxu/
iDByidOJAjMEEAEKAB0WIQTcD5pmgKm5i5kyhYGqIUI/fgqb5gUCX7G19QAKCRCq
IUI/fgqb5s1KD/4itFySsRwQFkJkg/QTvA7oi4h/oyASvhXC1vECO5OHYaxt49ok
KdkZOJRblD3weyUbbQ54OxvPvIadgxzwWYm6qqsXmRCt7C4lY1xSgyk5ct8n2Ytb
DzzNBuYRhdu9Qti28SY6EHvvjOE1gQTx1B+bkv75ih3KZrlURvJ8Rgdbc11BJDk8
3Q6fP27peqC5WbQOufY7ncfI+Dw9ecgffOPFINKYDAg8A22bDci0LNQG5xOkp+pl
VGKTf3TJdQSUelF62gqrFrY1EcgR3wP8kh2J2S3hBKiSUzN5bX3db2r8y03MOmiD
BKIjoWqSBC1ZfRuqfqqAJgTPIp0NzSK3tOpQYPRX5PrrKW+uRmQQsqC2rrF1Wj9T
hQ0Yxn6gbpbhSP7fAAepmlE64IxeaIXRgcHQ2Jog+3TkSDXXwURstjvVzmOg0NeP
FCsSXXau9eIvvXoO1BYsbV1XN2Rvp8DxvsGiukqK7t6qOq5PZHJoBB0nzD9vmFyz
Vz6X9yeKbbx50ocWV4OC55pIZFz8sKzW0FyPn97QQftfVGAbn82bBjAEEAzRujMx
qqE7Y13r0OzVwOd4k1e5o2GbcHi6xJHHFOgcYDKm7ye5E8CDmwn4VHSh/YXSXQph
2fFCLOJxx3dAaJ+0xMZb8auHsjBRJpyKCWX8IptRoW4ZrI+KSYm4ncsGmYkB1wQT
AQgAQQIbAwUJBiieKgULCQgHAgYVCgkICwIEFgIDAQIeAQIXgBYhBFkLcpJpWv+l
tnLLsuE/wUXNP0MEBQJfscd7AhkBAAoJEOE/wUXNP0MEVOYMAIN8k5UOQz5/dFiC
iRLfuHVwzrKcPSeQnRSdNEgXoqOrKesvO0pWulSJwKHR2wD6jrYQKZNJH7p72pCy
dEXjyaa/yMY2Q5Nso9za5K3PyabyOHN+gRiRWaZ1xAGBfxQX+7m0PYaW9r8tHLv5
SBfjF4wpV7qHqVYqfBTO2S2NRAnRtZdv2bLNW+QuwTZbN6/jWQJ2wbevuS29aUpL
tzOXzMj/4gJ+oW4Pt4TEiu28H554z68jXjLzCh3SylpuSUp1YtvUSnbRR8KnGcpb
+BzSqhemgeWDprj18LRTgrXEkyFsyHbSlk2NxViPDHklHO1FbJL7azuuDCAwBeTS
3F6k3msSEWjdfdv55VorBBdnU+bKJMQsdUClnN/LB72qTRPnnbfqy9WIOndwOQaT
5CKLmLMPo6zSJ4J4p/uj2Dh9Xr+wD6PuXXF3qrh1x9HEYONZzIcaN95YtYcerBCp
xJaxSx94cmCzArIo/hVo5E4VATwOyUvIXcFBl6HJodLjvWnSQ4kB1AQTAQoAPhYh
BFkLcpJpWv+ltnLLsuE/wUXNP0MEBQJek0vNAhsDBQkGKJ4qBQsJCAcCBhUKCQgL
AgQWAgMBAh4BAheAAAoJEOE/wUXNP0MEMSgL/1opoJjNREeHa143GmwL/dRvjKtI
apiIpMX64OqZQKqatF0zhMTKmR+yqmmk1TBbIK2Zu/lUgEP/naidKpje7ZFgpI4t
C/Tdcb/em34aRCsAbZbLLGlNaqOZVOSAoQzKvzUx6oya3EdCJuZqNqQSvnNr+bSS
/HeKY9PhOLSLrNCBJl+9C58KwC6WpeZX576lKm5RexeF0JNWxFunE19e9Rs5QyZm
sBQbj9CsUcReI93cU4DrY9+T1sT7yXGtfsR0B8eFRREXF+I2hvJ6IJ4R3xUS7h11
Ifx9hTW+DkWhQgRbV2EeHpJExELsB1yC+luqNpWjo4Z8dT1ISAkOb3hRx4/L0jcv
lNqgZbp0FyvZnBttV6WwWNPTx+fAbVsoaM2DhTSl+EqOl6qPws/S66ETKp2OCGBt
WGYDnL9PJGdBKfCX9saH1yVsorqVRLxbhmd5tgwGRj4ZXfMz3UrSY/4ny0wU7dz+
JUtvcqdbGTiHUKCXwiCf8ybeTcPMmFccwIaiU7QhQW50b2luZSBQb2luc290IDxk
YXJvc2lvckBsbi5kZXY+iQG2BDABCAAgFiEEWQtykmla/6W2csuy4T/BRc0/QwQF
Al+81V8CHQAACgkQ4T/BRc0/QwQd7gwApliX71tXlxmKoEtplPsEZJAJmXLglwsv
JO0C5tLCUjD2ir8BGvfGehOeD9/ATOwHAhTAAs41nFEp8sw8F7eAFEhPLg0BLWZ5
eAgAofEO9qmOIvaZYivWEZx6vT84cc/yASThU9TeDoSoReczqMeATE53fIljh3Ce
ruBNN/fKjDXaDkpxZAcRzqfdj0Nu3vpA6a42VqaGS5rHPamgl+JriH966XvbCJxx
sK8U9FxsXYL5Yi2BzQRZK0Y6CVBRlG24T77lkw9PQ1SZz412X0SUFE8zYsYkJ6Cb
/ExpLY/3aiIoEKz/FLpEp8MeqB2W24vjnC2qqmNkMguIR7hUehHv0leZK4kc5WxM
l7/0NLxHYi43wfKv7FkMoEXDJlrefjJ46S3SFI3/UKnhVyXA3QZtCJHFYBbEL5Ho
zLo1HdURHPiVJ3yBDsHsCS11GlICaNyC72qpAJ+iotTOqO5dQfQyGummXmFHc/8/
8eyZVXGeXbTiyvO7gl8EhMn3bCPtWbmUiQHUBBMBCgA+FiEEWQtykmla/6W2csuy
4T/BRc0/QwQFAl6TTE0CGwMFCQYonioFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AA
CgkQ4T/BRc0/QwTJEwv/agA8Szf1JtOnPaVT3hkaeSSvRSBTcgx7RLH3QZwO8W+m
Yu402/6q5AT1+Ra9XYyk5S7QjcLwnnohpfCKRDpsIdOl5Td+S5xMGqacm85YyRl3
uAyuW+9eAEmOD14LzhK0mu4UZN8JseRg9l0uguX4k8gFJuQMhFmBA0aRcYBB/sQG
1uuyjoGGXDfPsxNTB0oFCd0cKXl9iU8WkWTQg0jjRbFJA301HCff7z5o6/W3t87p
db0KwCoa/8RPZ6+qiUZlx2K85A0dveU0K5JSs0hASLI7xhuOLMtxdzx74vkKwZis
YL2Nz5IKmrp2vO7pvYnC3YjycqJmiQSqHDyqo4j4QZm8/WxaDsSgHlvzHp/EjNHK
Bi35jfbctVjKPcRIGKfR4bcns8pFvBslQ6ar6CPtCKymkN+WuNRtZbc8HI4Yo2Xl
pXcneBsXDrVWfO8AzYEV8q0GynsXwEuTHfft0vyjV3laK2K4reMKjECh1BZ5FPUw
wrTnKsfmiddv4f7rDdTjiQIzBBABCgAdFiEE3A+aZoCpuYuZMoWBqiFCP34Km+YF
Al+xtdMACgkQqiFCP34Km+ZdvA//Wh0irz3TwANLR6tAb//CCXSTEAcKS+Dgtn92
Jd7+uKUure3PV4gVYkPK0v4fYb0cdfuGT97eXQOo2fcwbCA30Q04sN8X5grrmrkD
r4MdIsORquEnAmTqmN/mRUyoF/r1sb93LW5x/VNNlAa6gQJwEOdki6orP9xuRW9m
UKm/tNCSuK1LELR64mNfC2ETrZNCF2gxTFi4s2uleIQb5RqnAbg+HYV9dOuD64Ed
+FvNrlF/7uyDsI/8FawtyO61HEiE71maIwSboDxeaRwPGkF8OtobQg7EIJp2hG9N
cPV6+ENJCDlQwL4oTgpWFnls0E++OKW+NADyTECBYf1Pm5XZsMVV56RG1GJ5kmBJ
sPOvjvmtIqaEiEaE6Veor31uesbXT3peytWUWHldh9DCzCF9Ee6mbvTlJTywU3Mu
B4XI9jQYMbBdrPDv+dBUYgQxWJL9vkOQsYUFcrdgFhXehFiSQJ9VENcWr50zxLWN
SWn8cVnkdHHkZhLJzj2NtobNLnSjmhrsFtruNNd0ECMw6E612nw4M+W2CHurQ9VT
Nnuw+rTtH2DllckQD3rsnTM3Fc52V0qBfs8xJDjT7Z2Sg/mY7R1BuRGvy/8LYQCt
2XzwkEySNM95jnACoAD4qZPRn120lHq0vEb9MqD+HqbtgoZUjExQ98Br4C3fnO/m
4riELKu0JUFudG9pbmUgUG9pbnNvdCA8YW50b2luZUByZXZhdWx0LmRldj6JAdQE
EwEIAD4WIQRZC3KSaVr/pbZyy7LhP8FFzT9DBAUCX7HHsgIbAwUJBiieKgULCQgH
AgYVCgkICwIEFgIDAQIeAQIXgAAKCRDhP8FFzT9DBFqNC/45VPmwp4kvDxRjmI9T
X/OwLfXVJVehYFNAuKukeiw5Vnt2XC2PF1XM/qMSypr69ium/I83VeFdZbWR7nm1
MeDcPpmE7D8wX9zy53XbWElJD++72+q51RSdC5+g3HFcNQyD0rLI9lJ+MT42/Z56
+V4KnhcPrh6Jm+1JDCnBmJEW5vV4HInlapfwUEUgz0uKEigDfRQn7vRp2x1Hio8l
SBsLIp6J4pnDqSz0x5hroGFSW8DvyjBAvf+OrTLNS+DQf/p/x6JTRBPep/oIRWNL
XzN161jfBGoZtHSj+FGwed6CoTHpgr0uZtI/giYnnzPq/MWevHdo9Tc2uOGzasO6
JOBPE5p4JtSceExoCL8mNpUO5g8KfnhaRQNniyQ9LEt8n/zW5LFJ1GthKqwR++hr
Fnza3bZMMGknKP++ogrJqoPjWnVapiuZvGl+WGhfA/a+ybZ9qongEfu1DE3NnH2q
2vjYyMmFbKR+2HdISY17RuS84kOKlLHm+9+37Q1+eqOpjrG0GmRhcm9zaW9yIDxk
YXJvc2lvckBsbi5kZXY+iQHUBBMBCgA+FiEEWQtykmla/6W2csuy4T/BRc0/QwQF
Al6C+OkCGwMFCQYonioFCwkIBwIGFQoJCAsCBBYCAwECHgECF4AACgkQ4T/BRc0/
QwR2vAwA0TJI8+2fOFqrliAVEDMSXJGigPyZ7AHUljcpdVUBV7Pw4VcdwUnD759k
chsYyOBbSKk67emxQe4Sjgk9UzjpfKYaqe8jiW4X11/43qubtV/SRWRRluJ1+jq/
OBFYflpy+zQKTeymXBcw9y6/tUtGBBoo4o4LAvBo7oNyYNgBcYW/D2q/RO/GhDAu
pnEONJm6iy+r6Icdf33EC+JL2eJV7UHXrbHgcGIriPPyHYVbcv/Q5gECrkUVTpha
MYGmKCPy1z55zNh+j6b0WjanJo90I2rjJUJiujzyEc/pbIaZU8AlFaaayIIZg9z2
CXDOirzoRblINzadF6WgbZmXTjTM30Sp04znDjWGfRBQgjNCaAKO1ErE1EaytM1s
t20dE1qJMqJuL/ck5V8pHySvQVgEk1XbFsWrcLlwIlyJqo59oLL2I4PylHlIeVjS
0tFkyLgtfxK2jskfwYeQabmECLE5/iXpClmHzJplt9obgTI5v9XHptt6EtFAJYws
649S+XHOtCJkYXJvc2lvciA8ZGFyb3Npb3JAcHJvdG9ubWFpbC5jb20+iQHUBBMB
CgA+AhsDBQsJCAcCBhUKCQgLAgQWAgMBAh4BAheAFiEEWQtykmla/6W2csuy4T/B
Rc0/QwQFAl5WUiAFCQYonioACgkQ4T/BRc0/QwQ8zwv7BLPZ1feTaqgkPNcuN48q
8m0uxveoIU1aLlblAdpaCv1hz83Ui9kD8YpmtgbYHCX6dZbpclJcMD3OggnmD1PE
3c/g97iWopbUIDIDc2RZJN66CrO8v4cHBBAc6PE0Jz6ZaJ+r15jRsWvxO0ORK9wX
faq57D3DMZMt/Nh89fXXWTr4imHtcIZRUc2/eAYSJHOx89ZUTJLxuXuWSYuseh6g
jipArqvsih2Qjn62ZbCL7EAt0g/QIRU4eOZ71vB8W/t6qfDrAvPiEpa+vjm9vXf5
NpiVuKOhi02VVZYIW8udXbnuNDOE6gV/IVeAwW/1sNcjzqqaEFnnpHv8Gr8EhCLv
XiiWDzvi4zgdyprkz/lI41zQ6nz8ZGI+qwJn4qtcQKhB53Wm1Iootr5kHZbB+1T0
mAIb8uwEyeFLHUV2LKJqiNaE2K3nf3GaoxJ3g88P3YF8KmISSHgZvZ7leKz7Ox8S
1CLgqut1FmO9TyuvbxRn7fngp+B7x4H9MSjq4IKPq2DiiQHUBBMBCgA+FiEEWQty
kmla/6W2csuy4T/BRc0/QwQFAlvwGvYCGwMFCQPCZwAFCwkIBwIGFQoJCAsCBBYC
AwECHgECF4AACgkQ4T/BRc0/QwSJygv8CiL/P1lrc6Zx6pORDIGQshXvzFcAWQWW
27NRrHkBRl7YkiQTTVZx5CReH9cy86Y//LC88iPcJ2AjlSnozj6o2sW3moxRxGGO
rNBkwoglASesZmId69Yifu40Ogk35ctazOFsDYnzTsNFlcDrEMpK+OcbjchQByuo
iDnQ9Z0stQW79Qw0dPt+NtvDgOVzp4HKbW/F7ZK9i2Bir6eMbvNCOn7i4t9lQM9P
vP11y75/TFVM56HB9RijujP+WvpgchwCyo1nEGjPjnCLs+MVi2u+u5gr3Jjev2Mb
MTd4tI7O7dMhMb+JRjcSTXKDiaQAsaeAmjBC+MVU6KX7gRTdy4moocgrXtLcSvGB
tOQfuCyS6lx+6kjFhc1kZ0ZY9QCzqa+w6dkB52zOuu40GVgDpOhRAQi4rDVSZMAD
eRDq8Z9XIQV6J6fh/2U4WQOQa+DcSD9VGdn+adrPffsY1u9W42N7x5aK2gSmdCam
LHyCiae4UeMjzD/roccIHIHantrzXO00uQGNBFvwGvYBDADaPLBXQHawjIrhfmZ+
Fjjps8FtM1EaZDrpKNsNRA6if8ZbuJrHD1aR/sLnXQdi4OVHI3xqpqrpz/qXHLRC
hYxaLCcGECJIQLR8AMDlZbJjoKXRNnRHd742uN7TBCwzetyCjOjZ1Gp3Ij1u9x6K
lsiHEtBzq//Tz9SmIascrWPFGtoGruSj7maax7UL4y4bEv9Bh6Cg8yRwgMqrRinT
p0i0BI1VpNWBaar/p60slmqL+tWFkgS8dhlLoKlRj/c3ybhYF9o4duBPOV3t1xgt
Jp1xhW95ZaCYTKR/7LOq/eMkx5HZUi+5WtFuaaoQ2+JbO8q3831ObOZMMaJVR9JV
VkPs9JQrr4re50auVOkC05LjRXmLJ504A7/HYljf9A0MEcmJeZxN/Pb2S4+yBaAc
YgTAr/Aee2FODfa4yqyLZqqaQxq9p22ytqBcFZ655MCRt35oqioRFWYikIkCZ66g
ZCXKZmM1bx9HFU/qP6lBoX65xKEcuGZdYGqb4ydC9DPMrK0AEQEAAYkBvAQYAQoA
JhYhBFkLcpJpWv+ltnLLsuE/wUXNP0MEBQJb8Br2AhsMBQkDwmcAAAoJEOE/wUXN
P0MEtB8L/R6QJWfQyB6XWDS8rDiGRdgfXMaflj7xiYG2lpxwpbk8ZtvsVxXHRSoP
RShcNum0mg/y6WZSIUvmMfejYbP9YJYEfHVwzcWN7jXBnLMq2u6JvLvCnqeUBvbf
Jd83zbVNUQMpYQAMX4Twxolqr24qhR4Gc2wYYezFrylZ9NPoiIdGf3jpunG66khC
2nrAp/bKhEtwnsND3IaBYdulWjn8vkDZRyEjbOy9zGxaB3ZTtjfNj4JI+BZ7h/YB
Las6iSZViJX3L7GEysp9f7oZ5C60y1VFQo7TcEUEUSZ9eAKrOkEBwFrMlyeneeGo
BbHFllhBMdRPr92G06GgWEK1D5KnEPo85Q47POs0MU0XRThwd2wElz79J2xOf5Ad
1edOtzpHaQli6nqybfAh6ZOKz/tUOrFAtEw765bKTpUG1sHsvapqk8+HQoOPKbE8
4axUQwMSQcHqnooHkprzib2GC/+DGwGeg1HeN5g1H1/GBPicYP//Ez9w3QbSVQ+7
KG04r8hWbw==
=m4XT
-----END PGP PUBLIC KEY BLOCK-----

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-01-05 02:54:49 +01:00
fanquake
8d6715666d Merge #20846: Add benthecarman to keys.txt
50a6f8f821 Add benthecarman to keys.txt (benthecarman)

Pull request description:

  For https://github.com/bitcoin-core/gitian.sigs/issues/1427

ACKs for top commit:
  laanwj:
    ACK 50a6f8f821

Tree-SHA512: a667cd6bc3511feb6bfea37d9b7b9ec69dadb02f47ec6e71e478c4f2e213c5cc0b8c74beec9536d3b9aadfdc743d5fb0107bee1ce72057acf01117e0f46862c0
2021-01-05 09:46:32 +08:00
fanquake
8b6acaca2f Merge #20848: Add gitian PGP key for theStack
729e1d151c Add gitian PGP key for theStack (Sebastian Falbesoner)

Pull request description:

  As requested per https://github.com/bitcoin-core/gitian.sigs/issues/1427#issue-778159391 🔑
  (my key is published at https://keybase.io/thestack / https://gist.github.com/theStack/60ed7ccc7c0caffeef2d001b8b54148b)

ACKs for top commit:
  laanwj:
    ACK 729e1d151c

Tree-SHA512: 23eb4206040fc3933fab71d4a5905adcada7bb0953fafd8e9905ea02d375d5909aeaff5782710023e72e6dceaa403a3a9c0f8246a03ee60b5ab45332ed0e9b9a
2021-01-05 09:39:27 +08:00
Pieter Wuille
a0eb4c551e Revert "Add patch to make codesign_allocate compatible with Apple's"
This reverts commit a4118c6e20.
2021-01-04 16:35:18 -08:00
Amiti Uttarwar
7ff05358a9 [mempool] Remove error suppression on upgrade
In 0.21, we added unbroadcast txids to mempool.dat. Commit 9c8a55d
added a try-block to prevent throwing a "failed to deserialize mempool data"
error when a user upgrades from 0.21 to 0.22. This exception handling is no
longer useful, so now we can remove it.
2021-01-04 12:02:29 -08:00
Sebastian Falbesoner
729e1d151c Add gitian PGP key for theStack 2021-01-04 16:26:15 +01:00
benthecarman
50a6f8f821 Add benthecarman to keys.txt 2021-01-04 09:08:09 -06:00
MarcoFalke
fa6c114ae6 test: Add sanitizer suppressions for AMD EPYC CPUs 2021-01-04 12:29:41 +01:00
MarcoFalke
bc8ada1c15 Merge #20736: rpc: Replace boost::variant with std::variant for RPCArg.m_fallback
fa749fbea3 rpc: Replace boost::variant with std::variant for RPCArg.m_fallback (MarcoFalke)

Pull request description:

  Now that we can use std::variant from the vanilla standard library, drop the third-party boost variant dependency.

  Patch is split out from #20480. A step-by-step replacement is possible because we don't have our own `Variant` wrapper and the source code specifies `boost::variant` explicitly.
  I think a step-by-step replacement should be preferred, because it simplifies review.

ACKs for top commit:
  fjahr:
    re-ACK fa749fbea3
  Sjors:
    re-ACK fa749fbea3

Tree-SHA512: 5e3c12b7d535f73065b4afa8df0a488f78fb25d2234f5ecbf740e624db03a34c35fea100eb7d37e84741721310e6450b7fb4296a2207a7ed1fa24485b3650981
2021-01-04 09:07:38 +01:00
Ikko Ashimine
1112035d32 doc: fix various typos
Co-authored-by: Peter Yordanov <ppyordanov@yahoo.com>
2021-01-04 12:31:31 +08:00
Sawyer Billings
e8640849c7 doc: Use https URLs where possible 2021-01-04 12:23:16 +08:00
fanquake
f52f427b8e Merge #20817: lint: update list of spelling linter false positives, bump to codespell 2.0.0
f3ba916e8b lint: ignore gitian keys file for spelling linter (Sebastian Falbesoner)
da289a6c4a lint: update list of spelling linter false positives (Sebastian Falbesoner)
a0022f1cfb test: bump codespell linter version to 2.0.0 (Sebastian Falbesoner)

Pull request description:

  This small patch updates the ignore list for the spelling linter script (which uses `codespell`), both removing false-positives that are not relevant anymore and adding new ones. As [suggested by jonatack](https://github.com/bitcoin/bitcoin/pull/20762#issuecomment-750889701)~~, whose last name is now also part of the list :)~~. Also changed the linter script to not check the gitian keys file, as [suggested by hebasto](https://github.com/bitcoin/bitcoin/pull/20817#discussion_r550763409). The codespell version used is bumped to most recent version 2.0.0, which is more aware of some terms that were previously needed in the ignorelist for v1.17.1, see https://github.com/bitcoin/bitcoin/pull/20817#issuecomment-753428669.

  Running spelling linter on master branch (repeated findings in the same file are removed to keep the output short):
  ```
  $ ./test/lint/lint-spelling.sh
  contrib/gitian-keys/keys.txt:16: Atack ==> Attack
  doc/developer-notes.md:1284: inout ==> input, in out
  doc/psbt.md:122: Asend ==> Ascend, as end
  src/bench/verify_script.cpp:27: Keypair ==> Key pair
  src/blockencodings.h:30: Unser ==> Under, unset, unsure, user
  src/compressor.h:65: Unser ==> Under, unset, unsure, user
  src/core_read.cpp:131: presense ==> presence
  src/index/disktxpos.h:21: blockIn ==> blocking
  src/net_processing.h:67: anounce ==> announce
  src/netaddress.h:486: compatiblity ==> compatibility
  src/primitives/transaction.h:35: nIn ==> inn, min, bin, nine
  src/qt/bitcoinunits.cpp:101: nIn ==> inn, min, bin, nine
  src/rpc/blockchain.cpp:2150: nIn ==> inn, min, bin, nine
  src/rpc/misc.cpp:198: nIn ==> inn, min, bin, nine
  src/script/bitcoinconsensus.cpp:81: nIn ==> inn, min, bin, nine
  src/script/bitcoinconsensus.h:63: nIn ==> inn, min, bin, nine
  src/script/interpreter.cpp:1279: nIn ==> inn, min, bin, nine
  src/script/interpreter.h:222: nIn ==> inn, min, bin, nine
  src/script/sign.cpp:17: nIn ==> inn, min, bin, nine
  src/script/sign.h:39: nIn ==> inn, min, bin, nine
  src/serialize.h:181: Unser ==> Under, unset, unsure, user
  src/signet.cpp:142: nIn ==> inn, min, bin, nine
  src/test/base32_tests.cpp:17: fo ==> of, for
  src/test/base64_tests.cpp:17: fo ==> of, for
  src/test/script_tests.cpp:1509: nIn ==> inn, min, bin, nine
  src/test/sighash_tests.cpp:27: nIn ==> inn, min, bin, nine
  src/test/validation_tests.cpp:78: excercise ==> exercise
  src/undo.h:36: Unser ==> Under, unset, unsure, user
  src/validation.cpp:1403: nIn ==> inn, min, bin, nine
  src/validation.h:255: nIn ==> inn, min, bin, nine
  src/wallet/wallet.cpp:1532: nIn ==> inn, min, bin, nine
  src/wallet/walletdb.cpp:429: Crypted ==> Encrypted
  test/functional/feature_nulldummy.py:63: unnecssary ==> unnecessary
  test/functional/wallet_encryption.py:81: crypted ==> encrypted
  test/functional/wallet_upgradewallet.py:36: fpr ==> for, far, fps
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

  Running spelling linter on PR branch:
  ```
  $ ./test/lint/lint-spelling.sh
  src/core_read.cpp:131: presense ==> presence
  src/net_processing.h:67: anounce ==> announce
  src/netaddress.h:486: compatiblity ==> compatibility
  src/test/validation_tests.cpp:78: excercise ==> exercise
  src/wallet/walletdb.cpp:429: Crypted ==> Encrypted
  test/functional/feature_nulldummy.py:63: unnecssary ==> unnecessary
  test/functional/wallet_encryption.py:81: crypted ==> encrypted
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```
  This list of remaining findings doesn't contain false positives anymore -- the typos are fixed in PR https://github.com/bitcoin/bitcoin/pull/20762.
  Happy new year! 🍾

ACKs for top commit:
  hebasto:
    re-ACK f3ba916e8b, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/20817#pullrequestreview-560632881) review.
  jonatack:
    ACK f3ba916e8b I don't know if there are any particular issues with bumping codespell to v2.0.0, but locally running the spelling linter and the cirrus job at https://cirrus-ci.com/task/5004066998714368 both LGTM. Thanks for also verifying and removing the unused words from the ignore list.

Tree-SHA512: e92ae6f16c01d4ff3d54f8c3a0ee95e12741f7bfe031d307a785f5cfd8a80525b16b34275f413b914c4a318f5166f9887399c21f2dad9cc7e9be41647042ef37
2021-01-04 12:05:57 +08:00
MarcoFalke
9a2400b575 Merge #20760: test: Set correct nValue for multi-op-return policy check
fad140e311 test: Set correct nValue for multi-op-return policy check (MarcoFalke)

Pull request description:

  `CTxOut::nValue` is default-initialized to `-1`. The dust-threshold for `OP_RETURN` outputs is `0`. Thus, the policy failure would be `dust` instead of `multi-op-return`. The test only passes because the dust check is currently not run.

  Avoid that confusion by setting the value to `0`, to ensure the dust check passes.

ACKs for top commit:
  theStack:
    ACK fad140e311

Tree-SHA512: f0c7a68eb2c573d6595b2b129fa8fa2a34fa35c17691f448bf1c54ccf66059c37562e7480cde7b51c4de677038d7717873da4257147a5f60acc8bbcd25fb7e3f
2021-01-03 19:04:41 +01:00
MarcoFalke
aaaa987840 refactor: Use C++17 std::array deduction for ALL_FEE_ESTIMATE_HORIZONS 2021-01-03 18:38:31 +01:00
Michael Dietz
a29f522ba4 fuzz: bolster ExtractDestination(s) checks 2021-01-03 11:38:19 -06:00
MarcoFalke
fa39cdd072 refactor: Use C++17 std::array deduction for OUTPUT_TYPES 2021-01-03 18:37:59 +01:00
MarcoFalke
2dab2d239a Merge #20765: fuzz: check that certain script TxoutType are nonstandard
efaf80e9bb fuzz: check that certain script TxoutType are nonstandard (Michael Dietz)

Pull request description:

  - Every transaction of type NONSTANDARD must not be a standard script
  - The only know types of nonstandard scripts are NONSTANDARD and certain NULL_DATA and MULTISIG scripts

  When reviewing https://github.com/bitcoin/bitcoin/pull/20761 I figured this is very similar and might also be good to have

ACKs for top commit:
  MarcoFalke:
    ACK efaf80e9bb

Tree-SHA512: 6f563ee3104ea9d2633aad95f1d003474bea759d0f22636c37aa91b5536a6ff0800c42447285ca8ed12f1b3699bf781dae1e5e0a3362da578749cd3164a06ea4
2021-01-03 18:28:45 +01:00
MarcoFalke
6e70674cda Merge #20781: fuzz: remove no-longer-necessary packages from fuzzbuzz config
0dade9154d fuzz: remove no-longer-necessary packages from fuzzbuzz config (fanquake)

Pull request description:

  I take it this is actively being used, given [comments in #20560](https://github.com/bitcoin/bitcoin/pull/20560#issuecomment-743747317); so remove old dependencies from setup.

ACKs for top commit:
  practicalswift:
    ACK 0dade9154d

Tree-SHA512: 781466776575e6051d0dddf4101bd057e484648f63e8e967240fefbf4b5832cacda6f6543708a0c368214a1efe0d60d371da78d7a920646cb93f1a4752aaf639
2021-01-03 18:26:59 +01:00
Hennadii Stepanov
195fcb53a0 qt: Follow Qt docs when implementing rowCount and columnCount 2021-01-02 21:40:53 +02:00
MarcoFalke
9c3765ad7c Merge #20830: doc: update developer notes with signet
ee701a9204 doc: update developer notes for signet (Jon Atack)

Pull request description:

  Preview with working anchor links:

  https://github.com/jonatack/bitcoin/blob/add-signet-to-developer-notes/doc/developer-notes.md

ACKs for top commit:
  0xB10C:
    ACK ee701a9204
  michaelfolkson:
    ACK ee701a9204
  MarcoFalke:
    ACK ee701a9204

Tree-SHA512: 12f0a81a701d9fd48e1f314e7a66b865309fdcb8f139323410d35c0cfb093d23a2f899249d1b0a025078894cf4789ed7221ae42804a56ab704a8b9ad35077bac
2021-01-02 19:20:35 +01:00
Sebastian Falbesoner
f3ba916e8b lint: ignore gitian keys file for spelling linter
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-01-02 19:06:28 +01:00
Sebastian Falbesoner
da289a6c4a lint: update list of spelling linter false positives 2021-01-02 19:06:20 +01:00
John Newbery
378aedc452 [net] Add cs_vSend lock annotations 2021-01-02 16:51:44 +00:00
John Newbery
673254515a [net] Move RecordBytesSent() call out of cs_vSend lock 2021-01-02 16:51:42 +00:00
Jon Atack
ee701a9204 doc: update developer notes for signet 2021-01-02 17:31:30 +01:00
MarcoFalke
eeee43bc48 fuzz: Use ConsumeWeakEnum for ServiceFlags 2021-01-02 15:07:29 +01:00
MarcoFalke
fa9949b914 fuzz: Add ConsumeWeakEnum helper, Extract ALL_NET_PERMISSION_FLAGS 2021-01-02 15:07:07 +01:00
MarcoFalke
faaef9434c fuzz: [refactor] Extract ALL_CONNECTION_TYPES constant 2021-01-02 15:06:36 +01:00
MarcoFalke
fa42da2d54 fuzz: Use ConsumeNode in process_message target 2021-01-02 15:05:24 +01:00
MarcoFalke
fa121f058f fuzz: Use ConsumeNode in process_messages target 2021-01-02 15:05:21 +01:00
MarcoFalke
6f97172b42 Merge #20649: refactor: Remove nMyStartingHeight from CNode/Connman
faaa4f2b6a refactor: Remove nMyStartingHeight from CNode/Connman (MarcoFalke)

Pull request description:

  CNode and CConnman keep track of the active chain height when CNodes have been created, but apart from serializing the int once (when sending a version message), it is unused. So it can simply be removed in favor of a single int in PeerMan that can do the same.

ACKs for top commit:
  jnewbery:
    utACK faaa4f2b6a
  theStack:
    re-ACK faaa4f2b6a 

Tree-SHA512: 713919e49e0f799c4a5a2e5ab07d1ca1c92d4049829680c4f436f9b3b545c548ddbee78ff80611de50cc91a116a416bcc4602e80ee64953597dc04ca15f9806d
2021-01-02 13:05:39 +01:00
Sebastian Falbesoner
a0022f1cfb test: bump codespell linter version to 2.0.0 2021-01-02 12:21:03 +01:00
MarcoFalke
faaa4f2b6a refactor: Remove nMyStartingHeight from CNode/Connman 2021-01-02 10:24:45 +01:00
MarcoFalke
ae8f797135 Merge #20210: net: assert CNode::m_inbound_onion is inbound in ctor, add getter, unit tests
86c495223f net: add CNode::IsInboundOnion() public getter and unit tests (Jon Atack)
6609eb8cb5 net: assert CNode::m_inbound_onion is inbound in ctor (Jon Atack)
993d1ecd19 test, fuzz: fix constructing CNode with invalid inbound_onion (Jon Atack)

Pull request description:

  The goal of this PR is to be able to depend on `m_inbound_onion` in AttemptToEvictConnection in #20197:

  - asserts `CNode::m_inbound_onion` is inbound in the CNode ctor to have a validity check at the class boundary
  - fixes a unit test and a fuzz utility that were passing invalid inbound onion values to the CNode ctor
  - drops an unneeded check in `CNode::ConnectedThroughNetwork()` for its inbound status
  - adds a public getter `IsInboundOnion()` that also allows unit testing it
  - adds unit test coverage

ACKs for top commit:
  sipa:
    utACK 86c495223f
  LarryRuane:
    ACK 86c495223f
  vasild:
    ACK 86c495223f
  MarcoFalke:
    review ACK 86c495223f 🐍

Tree-SHA512: 21109105bc4e5e03076fadd489204be00eac710c9de0127708ca2d0a10a048ff81f640f589a7429967ac3eb51d35fe24bb2b12e53e7aa3efbc47aaff6396d204
2021-01-02 09:54:01 +01:00
MarcoFalke
fa749fbea3 rpc: Replace boost::variant with std::variant for RPCArg.m_fallback 2021-01-01 15:08:07 +01:00
MarcoFalke
e75f91eae3 Merge #20805: doc: Update license year range to 2021
ccc8d5513f doc: Update license year range to 2021 (Emil Engler)

Pull request description:

  See #17801, #15061
  The same procedure as every year. Happy new year to all of you :)

Top commit has no ACKs.

Tree-SHA512: 59ca9ba43d3722d5e2ada04fe6c361058a72c84b8437f6331f45e361d7ebf0f3586ccb02dc72af12b9d94f766897f0f041ce3a90efe95eeff22217f9f964ffc6
2020-12-31 18:53:15 +01:00
MarcoFalke
4a540683ec Merge #20813: scripted-diff: Bump copyright headers
fa0074e2d8 scripted-diff: Bump copyright headers (MarcoFalke)

Pull request description:

  Needs to be done because no one has removed the years yet

ACKs for top commit:
  practicalswift:
    ACK fa0074e2d8

Tree-SHA512: 210e92acd7d400b556cf8259c3ec9967797420cfd19f0c2a4fa54cb2b3d32ad9ae27e771269201e7d554c0f4cd73a8b1c1a42c9f65d8685ca4d52e5134b071a3
2020-12-31 18:50:19 +01:00
MarcoFalke
409afd7153 Merge #20812: fuzz: Bump FuzzedDataProvider.h
fa5b935840 fuzz: Remove duplicate ALL_OUTPUT_TYPE array (MarcoFalke)
fafce49336 fuzz: Bump FuzzedDataProvider.h (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa5b935840

Tree-SHA512: eacf97f26869fb226d7ac5ea1f8204b25ebb7551d084af0da02cc1a850a1fdf7d7a4919386b7f27ab9acb1873f63aafe9ad9f8ea2668aea366e11c7c0991a2f4
2020-12-31 14:00:03 +01:00
MarcoFalke
fa0074e2d8 scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-12-31 09:45:41 +01:00
MarcoFalke
fa29272459 Remove redundant MakeUCharSpan wrappers 2020-12-31 09:06:00 +01:00
MarcoFalke
faf4aa2f47 Remove CDataStream::Init in favor of C++11 member initialization 2020-12-31 09:05:51 +01:00
MarcoFalke
fa5b935840 fuzz: Remove duplicate ALL_OUTPUT_TYPE array 2020-12-31 08:51:42 +01:00
MarcoFalke
fafce49336 fuzz: Bump FuzzedDataProvider.h
Latest version from https://raw.githubusercontent.com/llvm/llvm-project/70de7e0d9a95b7fcd7c105b06bd90fdf4e01f563/compiler-rt/include/fuzzer/FuzzedDataProvider.h
2020-12-31 08:49:10 +01:00
Hennadii Stepanov
b3e9bcaac8 qt, refactor: Drop no longer used PeerTableModel::getNodeStats function 2020-12-30 21:01:20 +02:00
Hennadii Stepanov
49c604077c qt: Use PeerTableModel::StatsRole
This change prevents direct calls to the PeerTableModel object that is a
layer violation.
2020-12-30 20:59:56 +02:00
Hennadii Stepanov
35007edf9c qt: Add PeerTableModel::StatsRole
This change allows access to CNodeCombinedStats instance directly from
any view object.
2020-12-30 20:54:47 +02:00
Emil Engler
ccc8d5513f doc: Update license year range to 2021 2020-12-30 16:24:47 +01:00
MarcoFalke
f1f26b8d5b Merge #20377: fuzz: Fill various small fuzzing gaps
4ddbcd0d9a fuzz: Add coverage for CDataStream consumer (practicalswift)
546a0764f3 fuzz: Fill various small fuzzing gaps (practicalswift)

Pull request description:

  Fill various small fuzzing gaps.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    review ACK 4ddbcd0d9a

Tree-SHA512: d20f2cc0172f39948673846d088121782f39b4556df8b38fa14859cfa062c1519d18ee9601d4503ef1ba9613976cc5349c1fc0f0b9601a3d68127ffce1b1854e
2020-12-29 09:27:29 +01:00
MarcoFalke
069f37c0fd Merge #20791: p2p: remove unused legacyWhitelisted in AcceptConnection()
8f9ca31782 p2p: remove unused legacyWhitelisted variable (Jon Atack)

Pull request description:

  Noticed while compiling master:

  ```
  net.cpp: In member function ‘void CConnman::AcceptConnection(const CConnman::ListenSocket&)’:
  net.cpp:1041:10: warning: variable ‘legacyWhitelisted’ set but not used [-Wunused-but-set-variable]
   1041 |     bool legacyWhitelisted = false;
        |          ^~~~~~~~~~~~~~~~~
  ```

ACKs for top commit:
  glozow:
    utACK 8f9ca31782
  MarcoFalke:
    review ACK 8f9ca31782

Tree-SHA512: e3fb94d3d34b364b063a115ef9ed06cfd58729a790ba6ed27c7c32430fabf3ca0aa61bc6d33a1236bff802302779c8db28af351208f06c09ad12ce4873c244a6
2020-12-29 08:56:01 +01:00
Michael Dietz
efaf80e9bb fuzz: check that certain script TxoutType are nonstandard 2020-12-28 19:31:23 -06:00
Jon Atack
8f9ca31782 p2p: remove unused legacyWhitelisted variable 2020-12-29 00:53:57 +01:00
Wladimir J. van der Laan
d875bcc8f9 Merge #162: Add network to peers window and peer details
e262a19b0b gui: display network in peer details (Jon Atack)
9136953073 gui: rename peer tab column headers, initialize in .h (Hennadii Stepanov)
05c08c696a gui: add network column in peers tab/window (Jon Atack)
e0e55060bf gui: fix broken doxygen formatting in src/qt/guiutil.h (Jon Atack)
0d5613f9de gui: create GUIUtil::NetworkToQString() utility function (Jon Atack)
af9103cc79 net, rpc: change CNodeStats::m_network from string to Network (Jon Atack)

Pull request description:

  and rename peers window column headers from NodeId and Node/Service to Peer Id and Address.

  ![Screenshot from 2020-12-27 14-45-31](https://user-images.githubusercontent.com/2415484/103172228-efec8600-4849-11eb-8cee-04a3d2ab1273.png)

ACKs for top commit:
  laanwj:
    ACK e262a19b0b

Tree-SHA512: 709c2a805c109c2dd033aca7b6b6dc94ebe2ce7a0168c71249e1e661c9c57d1f1c781a5b9ccf3b776bedeb83ae2fb5c505637337c45b1eb9a418cb1693a89761
2020-12-28 23:56:23 +01:00
MarcoFalke
e3dd0a56cf Merge #20755: [rpc] Remove deprecated fields from getpeerinfo
454a4088a8 [doc] Add release notes for removed getpeerinfo fields. (Amiti Uttarwar)
b1a936d4ae [rpc] Remove deprecated "whitelisted" field from getpeerinfo (Amiti Uttarwar)
094c3beaa4 [rpc] Remove deprecated "banscore" field from getpeerinfo (Amiti Uttarwar)
537053336f [rpc] Remove deprecated "addnode" field from getpeerinfo (Amiti Uttarwar)

Pull request description:

  This PR removes support for 3 fields on the `getpeerinfo` RPC that were deprecated in v0.21- `addnode`, `banscore` & `whitelisted`.

ACKs for top commit:
  sipa:
    utACK 454a4088a8
  jnewbery:
    ACK 454a4088a8.

Tree-SHA512: ccc0e90c0763eeb8529cf0c46162dbaca3f7773981b3b52d9925166ea7421aed086795d56b320e16c9340f68862388785f52a9b78314865070917b33180d7cd6
2020-12-28 22:40:39 +01:00
Prayank
7117d7503f Update 'Secure string handling'
Add information about possible path traversal attack with example
2020-12-29 01:49:30 +05:30
Wladimir J. van der Laan
7f9ae87011 Merge #20451: lint: run mypy over contrib/devtools
1ef2138c0d lint: run mypy over contrib/devtools (fanquake)

Pull request description:

  wumpus mentioned on IRC that we don't currently run `mypy` over the `contrib/devtools` directory, and that it would likely be worthwhile given #20434. This just adds that dir to the linter, as well as some missing annotations to fix existing errors. Note that now we require Python 3.6 we can make use of variable annotations.

  master (patched to check contrib devtools):
  ```bash
  test/lint/lint-python.sh
  contrib/devtools/symbol-check.py:154: error: Incompatible types in assignment (expression has type "List[str]", variable has type "str")
  contrib/devtools/circular-dependencies.py:35: error: Need type annotation for 'deps' (hint: "deps: Dict[<type>, <type>] = ...")
  contrib/devtools/circular-dependencies.py:67: error: Need type annotation for 'closure' (hint: "closure: Dict[<type>, <type>] = ...")
  Found 4 errors in 3 files (checked 187 source files)
  ```

  I haven't quite gone as far as to add annotations like
  ```python
  CHECKS: Dict[str, List[Tuple[str, Callable[[Any], bool]]]] = {...
  ```
  to `symbol-check.py`.

ACKs for top commit:
  laanwj:
    ACK 1ef2138c0d

Tree-SHA512: a58c2ece588c640289dc1d35dad5b1b8732788272daa0965d6bf44ee8a7f7c8e8585f94d233ac41c84b9ffcfc97841a00fe2c9acba41f58fd164f01de4b6512b
2020-12-28 14:18:37 +01:00
fanquake
ddbf7a6f50 Merge #20684: build: Define .INTERMEDIATE target once only
5e0dedb111 build: Define .INTERMEDIATE target once only (Hennadii Stepanov)

Pull request description:

  A new warning was introduced in 22437fc72e (#20470):
  ```
  $ ./autogen.sh
  ...
  Makefile.am:335: warning: .INTERMEDIATE was already defined in condition !BUILD_DARWIN, which is included in condition TRUE ...
  Makefile.am:139: ... '.INTERMEDIATE' previously defined here
  ...
  ```

  Fixed in this PR.

ACKs for top commit:
  jonatack:
    Tested and very light review ACK 5e0dedb111

Tree-SHA512: ecf8de79ba394c36ee84e0b8d3ba78587e0f856259e9731e6bbb38d0baebfd083eb44d7ef6a386dd9e4508dd64fec1c2b9a007e175fbd4d986e845b1c300a649
2020-12-28 16:48:06 +08:00
fanquake
0dade9154d fuzz: remove no-longer-necessary packages from fuzzbuzz config 2020-12-28 14:37:48 +08:00
fanquake
1ef2138c0d lint: run mypy over contrib/devtools 2020-12-28 14:25:06 +08:00
fanquake
4a8f4ac4fc Merge #20771: refactor: Enable -Wswitch for FeeEstimateHorizon
faccf8b1e1 refactor: Enable -Wswitch for FeeEstimateHorizon (MarcoFalke)

Pull request description:

  This enables the `-Wswitch` compiler warning for `FeeEstimateHorizon` by removing the `default` case in `switch` statements.

ACKs for top commit:
  practicalswift:
    cr ACK faccf8b1e1
  jonatack:
    ACK faccf8b1e1
  hebasto:
    ACK faccf8b1e1, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 63a8dff6e8dead149ec2fa8319e7ff41022c9534d423d3086fd8f22be073dc4915f74c7fe9139ee681a8204730cf58c80ef40c93fb33032d586e68b4f78f557d
2020-12-28 14:14:20 +08:00
practicalswift
4ddbcd0d9a fuzz: Add coverage for CDataStream consumer 2020-12-27 19:27:41 +00:00
Jon Atack
e262a19b0b gui: display network in peer details 2020-12-27 14:38:16 +01:00
Hennadii Stepanov
9136953073 gui: rename peer tab column headers, initialize in .h 2020-12-27 14:30:33 +01:00
Jon Atack
05c08c696a gui: add network column in peers tab/window 2020-12-27 14:30:24 +01:00
Jon Atack
e0e55060bf gui: fix broken doxygen formatting in src/qt/guiutil.h 2020-12-27 14:29:26 +01:00
Jon Atack
0d5613f9de gui: create GUIUtil::NetworkToQString() utility function 2020-12-27 14:29:24 +01:00
Jon Atack
af9103cc79 net, rpc: change CNodeStats::m_network from string to Network 2020-12-27 14:28:31 +01:00
fanquake
0e1b57b4bb Merge #20763: test: Fix comment typo in BitcoinTestFramework
40fdb2a212 test: Fix Comment Typo in BitcoinTestFramework (Joel Klabo)

Pull request description:

  Missing "override" in comment describing use of set_test_params

ACKs for top commit:
  michaelfolkson:
    ACK 40fdb2a212

Tree-SHA512: bf893a0d5f8dc86a3ec2eaf48cd7c0f0f832f3b3d254b3d99953336db7e294571b1d2c8686030bf8a27cbe67b1a85a54e53ebefb2e57d6d8d6ac864a15dce4e7
2020-12-27 17:51:15 +08:00
fanquake
31e511658a Merge #20674: fuzz: Call SendMessages after ProcessMessage to increase coverage
fa09f97bea fuzz: Call SendMessages after ProcessMessage to increase coverage (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    Tested ACK fa09f97bea
  dhruv:
    tACK fa09f97
  Crypt-iQ:
    cr ACK fa09f97bea
  sipa:
    utACK fa09f97bea

Tree-SHA512: 87c52aa38f902c4f6c9c2380f486a3ab21edc0e21e48bb619cdb67cfd698154cc57b170eef31fc940c0bb2c878e155847de03fc6e4cd85bed25f10c4f80c747b
2020-12-27 17:37:32 +08:00
Amiti Uttarwar
454a4088a8 [doc] Add release notes for removed getpeerinfo fields. 2020-12-26 13:30:54 -08:00
Amiti Uttarwar
b1a936d4ae [rpc] Remove deprecated "whitelisted" field from getpeerinfo 2020-12-26 13:30:54 -08:00
Amiti Uttarwar
094c3beaa4 [rpc] Remove deprecated "banscore" field from getpeerinfo 2020-12-26 13:30:08 -08:00
Amiti Uttarwar
537053336f [rpc] Remove deprecated "addnode" field from getpeerinfo 2020-12-26 13:30:08 -08:00
MarcoFalke
02cf20b9f5 Merge #20756: [doc] Add missing field (permissions) to the getpeerinfo help
667d203687 [doc] Add permissions to the getpeerinfo help. (Amiti Uttarwar)

Pull request description:

  This field was previously being returned, but missing from the RPCHelpMan. This PR uses the existing `NET_PERMISSIONS_DOC` to inform RPC users about this field.

  ```
     "permissions" : [                 (json array) Any special permissions that have been granted to this peer
        "str",                          (string) bloomfilter (allow requesting BIP37 filtered blocks and transactions),
                                        noban (do not ban for misbehavior; implies download),
                                        forcerelay (relay transactions that are already in the mempool; implies relay),
                                        relay (relay even in -blocksonly mode, and unlimited transaction announcements),
                                        mempool (allow requesting BIP35 mempool contents),
                                        download (allow getheaders during IBD, no disconnect after maxuploadtarget limit),
                                        addr (responses to GETADDR avoid hitting the cache and contain random records with the most up-to-date info).

        ...
      ],

  ```

ACKs for top commit:
  Sjors:
    tACK 667d203687

Tree-SHA512: 973631b41d35d6333e3cb06b35277de869110f6ad6498c7e74f00c75202e8de1788a48755c21ac964903e5e6050a5e769a63866211aec9004cd665a727a54a3c
2020-12-26 21:26:12 +01:00
MarcoFalke
faccf8b1e1 refactor: Enable -Wswitch for FeeEstimateHorizon 2020-12-26 17:32:14 +01:00
Hennadii Stepanov
90f9fc274b qt: Save QSplitter state in QSettings 2020-12-25 21:12:18 +02:00
MarcoFalke
43fc7a569c Merge #19972: fuzz: Add fuzzing harness for node eviction logic
5a9ee0869b tests: Add fuzzing harness for node eviction logic (practicalswift)

Pull request description:

  Add fuzzing harness for node eviction logic.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    cr ACK 5a9ee0869b

Tree-SHA512: c2401d22134867e23dab1ba94ae7ef36fdf52aa0588fdc4705d9cb765ddf979fd775fdf153ce2359f1bc1787cf60bf0ebcd47c7aa29c672e6a253fa58cac292d
2020-12-25 13:32:31 +01:00
MarcoFalke
762cbd287f Merge #15451: [doc] clarify getdata limit after #14897
c119ba3c9b [doc] clarify getdata limit after #14897 (Michael Polzer)

Pull request description:

  GETDATA is limited to blocks and transactions now and can't be used for other non-block data

ACKs for top commit:
  laanwj:
    ACK c119ba3c9b
  theStack:
    ACK c119ba3c9b
  benthecarman:
    ACK c119ba3c9b

Tree-SHA512: d6e9c109bcce4ef004ec83a9ec591163279476524dec97ed5f5c34e322dca35af66a168f0878ff972bbcec79d81623903f3619fedf8f88cdced3f3f66a779173
2020-12-25 09:43:25 +01:00
MarcoFalke
22435f0cf6 Merge #20759: doc: [test] Remove outdated comment in fuzz runner
fa511042b0 doc: [test] Remove outdated comment in fuzz runner (MarcoFalke)

Pull request description:

  All folders are soft-created with `os.makedirs`

ACKs for top commit:
  RiccardoMasutti:
    ACK fa51104

Tree-SHA512: 4051688946a205a981bbb005300fe3263495ead26591042b38ae44f4297c7689a613b560052fb5405a62054734d2599cfb0554a37c7b7369fb3a3636743d04a8
2020-12-25 07:51:29 +01:00
MarcoFalke
443988d406 Merge #20761: fuzz: Check that NULL_DATA is unspendable
fa26303286 fuzz: Check that NULL_DATA is unspendable (MarcoFalke)

Pull request description:

  * Every script of type NULL_DATA must be unspendable
  * The only know types of unspendable scripts are NULL_DATA and certain NONSTANDARD scripts

ACKs for top commit:
  sipa:
    utACK fa26303286

Tree-SHA512: 8297fbacf32b4868b12accc1c052d352d02d96540a1fc883de9d04a3df8734116deecc33046495c9a3af6d79fec7f8d63afbfa5e401a2ca8d7c70f0f13735c0d
2020-12-25 07:41:43 +01:00
fanquake
1b31695c74 Merge #20673: depends: Use more legible qmake commands in qt package
87fe104537 depends: Use more legible qmake commands in qt package (Hennadii Stepanov)
bf35a8da6e depends: Do not set build_subdir for qt package (Hennadii Stepanov)

Pull request description:

  Rather than using `cd` to jump all over the place, perform all `(q)make` commands from the top level directory.

  Looking at bash like `cd ../../../..` gives me a headache.

  Credits to **fanquake**.

  This PR is an alternative to #20504 that works without any additional [non-trivial hack](https://github.com/bitcoin/bitcoin/pull/20504#issuecomment-734730336).

ACKs for top commit:
  promag:
    Tested ACK 87fe104537.
  fanquake:
    ACK 87fe104537

Tree-SHA512: 1d2a13b5358fc7406c5363ddd62fd363dbc0ec5ace68946e4d3e6e8620419afaa64ef2837488aaed226174e01e8897495085540f7126b80f8b2372d21b5b29f9
2020-12-25 10:56:08 +08:00
Joel Klabo
40fdb2a212 test: Fix Comment Typo in BitcoinTestFramework
Missing "override" in comment describing use of set_test_params
2020-12-24 10:54:56 -08:00
Andrew Chow
48a0319bab Add a test that selects too large if BnB is used
If BnB is used, the test will fail because the transaction is too large.
2020-12-24 12:39:54 -05:00
MarcoFalke
f656165e9c Merge #18772: rpc: calculate fees in getblock using BlockUndo data
66d012ad7f test: RPC: getblock fee calculations (Elliott Jin)
bf7d6e31b1 RPC: getblock: tx fee calculation for verbosity 2 via Undo data (Elliott Jin)

Pull request description:

  This change is progress towards #18771 .  It adapts the fee calculation part of #16083 and addresses some feedback.  The additional "verbosity level 3" features are planned for a future PR.

  **Original PR description:**

  > Using block undo data (like in #14802) we can now show fee information for each transaction in a block without the need for additional -txindex and/or a ton of costly lookups. For a start we'll add transaction fee information to getblock verbosity level 2. This comes at a negligible speed penalty (<1%).

ACKs for top commit:
  luke-jr:
    tACK 66d012ad7f
  fjahr:
    tACK 66d012ad7f
  MarcoFalke:
    review ACK 66d012ad7f 🗜

Tree-SHA512: be1fe4b866946a8dc36427f7dc72a20e10860e320a28fa49bc85bd2a93a0d699768179be29fa52e18b2ed8505d3ec272e586753ef2239b4230e0aefd233acaa2
2020-12-24 15:32:10 +01:00
MarcoFalke
fa26303286 fuzz: Check that NULL_DATA is unspendable 2020-12-24 14:16:59 +01:00
MarcoFalke
fad140e311 test: Set correct nValue for multi-op-return policy check 2020-12-24 13:43:25 +01:00
MarcoFalke
fa511042b0 doc: [test] Remove outdated comment in fuzz runner 2020-12-24 08:20:54 +01:00
MarcoFalke
cc592a85ea Merge #20189: test: Switch to BIP341's suggested scheme for outputs without script
812baaa1f8 Switch to BIP341's suggested scheme for outputs without script (Pieter Wuille)

Pull request description:

  BIP341 suggests using Hash<sub>TapTweak</sub>(pubkey) to derive the tweak in case of key-only outputs. The functional test framework currently uses Hash<sub>TapTweak</sub>(pubkey || 0x00...00) instead. Change this.

  There is no technical reason to prefer one over the other, but in case someone looks at it for inspiration, it's better to be consistent with the BIP.

ACKs for top commit:
  laanwj:
    ACK 812baaa1f8
  instagibbs:
    ACK 812baaa1f8

Tree-SHA512: 02576c38776ec786255f49d7edecdb1ed8a9dcf0f547d58c23099588b4c3296edf279b103a6eb80e0f07d3c5ee9743f67d152f5244fd63adc6613b004f6969ed
2020-12-24 07:56:33 +01:00
fanquake
1be6f2dba9 Merge #20747: net processing: Remove dropmessagestest
176325a5a4 [net processing] Remove dropmessagestest (John Newbery)

Pull request description:

  -dropmessagestest is a command line option that causes 1 in n received
  messages to be dropped. The Bitcoin P2P protocol is stateful and in
  general cannot handle messages being dropped. Dropped
  version/verack/ping/pong messages will cause the connection to time out
  and be torn down. Other dropped messages may also cause the peer to
  believe that the peer has stalled and tear down the connection.

  It seems difficult to uncover any actual issues with -dropmessagestest,
  and any coverage that could be generated would probably be easier to
  trigger with fuzz testing.

ACKs for top commit:
  MarcoFalke:
    cr ACK 176325a5a4
  practicalswift:
    cr ACK 176325a5a4
  dhruv:
    cr ACK 176325a
  amitiuttarwar:
    ACK 176325a5a4

Tree-SHA512: bd582e5e8c9eb272a5d8ec01ff07c36c0033fbb84c30d1c72c87a7a6c7290021dcaf7bf549179a8b95aeb4f7243158d5593bc7fcf1ec16213782e470fe36bb89
2020-12-24 09:55:27 +08:00
Hennadii Stepanov
957895c715 util: Log static plugins meta data and style 2020-12-23 22:30:32 +02:00
Amiti Uttarwar
667d203687 [doc] Add permissions to the getpeerinfo help.
This field was already being returned, but the RPCHelpMan did not indicate
this. So, this PR updates the help text to match.
2020-12-23 11:38:59 -08:00
MarcoFalke
e669c3156f Merge #20748: test: Add race:SendZmqMessage tsan suppression
fa957f8dc9 test: Add race:SendZmqMessage tsan suppression (MarcoFalke)

Pull request description:

  Add suppression for `race:SendZmqMessage`, which isn't covered by the existing `zmq::*` suppression

  Fixes #20618

ACKs for top commit:
  hebasto:
    re-ACK fa957f8dc9, as my previous comment is not directly related to this pull changes.

Tree-SHA512: 8642a8b79bbfa4bee89042b66e528f27fd78c5e84a33023df440662e9114e31445fd7b04940f44b11fa4ab7438d346385a21816289c818cce9958a9b16730452
2020-12-23 12:32:33 +01:00
MarcoFalke
98de9eb1aa Merge #19983: Drop some TSan suppressions
3e1571285f Update TSan suppressions (Hennadii Stepanov)

Pull request description:

  It seems possible now to drop some TSan suppressions.

Top commit has no ACKs.

Tree-SHA512: 94518fd2f3a7168b2989424de0696e42c8f509b833aafbc7e75f4c1180a0b8d9a47f43c50d06b03b26a924643afe86274b2062c9d456c17a68576d19566ed66f
2020-12-22 21:55:22 +01:00
MarcoFalke
fa957f8dc9 test: Add race:SendZmqMessage tsan suppression 2020-12-22 21:53:13 +01:00
Hennadii Stepanov
3e1571285f Update TSan suppressions 2020-12-22 20:24:23 +02:00
John Newbery
176325a5a4 [net processing] Remove dropmessagestest
-dropmessagestest is a command line option that causes 1 in n received
messages to be dropped. The Bitcoin P2P protocol is stateful and in
general cannot handle messages being dropped. Dropped
version/verack/ping/pong messages will cause the connection to time out
and be torn down. Other dropped messages may also cause the peer to
believe that the peer has stalled and tear down the connection.

It seems difficult to uncover any actual issues with -dropmessagestest,
and any coverage that could be generated would probably be easier to
trigger with fuzz testing.
2020-12-22 17:48:31 +00:00
MarcoFalke
9286b1cbce Merge #20745: qa: Correct epoll_ctl data race suppression
d71e29e3e8 qa: Correct epoll_ctl data race suppression (Hennadii Stepanov)

Pull request description:

  Fixup of #20218. Comments must start from the beginning of the line.

ACKs for top commit:
  MarcoFalke:
    review ACK d71e29e3e8

Tree-SHA512: 4d8663ab505c347bcb62c2f118656e3343d5179825be0d1b86761ffdfdae1e7462002bf226a54dfc94be5885ce7f2633abaf70421ea35bf06eddad8e99fb9683
2020-12-22 18:22:53 +01:00
Hennadii Stepanov
d71e29e3e8 qa: Correct epoll_ctl data race suppression
Fixup of #20218. Comments must start from the beginning of the line.
2020-12-22 17:34:29 +02:00
MarcoFalke
df127ecede Merge #19829: net processing: Move block inventory state to net_processing
3002b4af2b [net processing] Guard m_continuation_block with m_block_inv_mutex (John Newbery)
184557e8e0 [net processing] Move hashContinue to net processing (John Newbery)
c853ef002e scripted-diff: rename vBlockHashesToAnnounce and vInventoryBlockToSend (John Newbery)
53b7ac1b7d [net processing] Move block inventory data to Peer (John Newbery)
78040f9168 [net processing] Rename nStartingHeight to m_starting_height (John Newbery)
77a2c2f8f9 [net processing] Move nStartingHeight to Peer (John Newbery)
717a374e74 [net processing] Improve documentation for Peer destruction/locking (John Newbery)

Pull request description:

  This continues the work of moving application layer data into net_processing, by moving all block inventory state into the new Peer object added in #19607.

  For motivation, see #19398.

ACKs for top commit:
  jonatack:
    re-ACK 3002b4af2b per `git diff 9aad3e4 3002b4a`
  Sjors:
    Code review re-ACK 3002b4af2b
  MarcoFalke:
    re-ACK 3002b4af2b 🌓

Tree-SHA512: eb2b474b73b025791ee3e6e41809926b332b48468763219f31638ca390f427632f05902dfc6a2c6bdc1ce47b215782f67874ddbf05b97d77d5897b7e2abfe4d9
2020-12-22 12:44:19 +01:00
MarcoFalke
209974d379 Merge #20740: fuzz: Update FuzzedDataProvider.h from upstream (LLVM)
e3d2ba7c70 fuzz: Update FuzzedDataProvider.h from upstream (LLVM) (practicalswift)

Pull request description:

  Update `FuzzedDataProvider.h` from upstream (LLVM).

  Upstream revision: 6d0488f75b/compiler-rt/include/fuzzer/FuzzedDataProvider.h

  Changes since last update:
  * [[compiler-rt] FuzzedDataProvider: add ConsumeData and method.](20a604d3f5)
  * [[compiler-rt] Fix a typo in a comment in FuzzedDataProvider.h.](5517d3b80b)
  * [[compiler-rt] Add ConsumeRandomLengthString() version without arguments.](2136d17d8d)
  * [[compiler-rt] Refactor FuzzedDataProvider for better readability.](1262db1b6a)
  * [[compiler-rt] FuzzedDataProvider: make linter happy.](1e65209e04)
  * [[compiler-rt] Mark FDP non-template methods inline to avoid ODR violations.](6d0488f75b)

ACKs for top commit:
  MarcoFalke:
    ACK e3d2ba7c70 🌛

Tree-SHA512: 62cb27906f08fd07983f4a8fbbd381c12ed185617a58f1ebc8564c87e638086f952417f4f6481fbd91b9a313aff00e944215393734566c219c074512991f8057
2020-12-22 12:12:12 +01:00
Fabian Jahr
9815332d51 test: Change MuHash Python implementation to match cpp version again 2020-12-22 01:48:34 +01:00
Fabian Jahr
01297fb3ca fuzz: Add MuHash consistency fuzz test 2020-12-22 01:48:30 +01:00
practicalswift
e3d2ba7c70 fuzz: Update FuzzedDataProvider.h from upstream (LLVM)
Upstream revision: 6d0488f75b/compiler-rt/include/fuzzer/FuzzedDataProvider.h

Changes:
* [compiler-rt] FuzzedDataProvider: add ConsumeData and method.
* [compiler-rt] Fix a typo in a comment in FuzzedDataProvider.h.
* [compiler-rt] Add ConsumeRandomLengthString() version without arguments.
* [compiler-rt] Refactor FuzzedDataProvider for better readability.
* [compiler-rt] FuzzedDataProvider: make linter happy.
* [compiler-rt] Mark FDP non-template methods inline to avoid ODR violations.
2020-12-21 23:19:33 +00:00
Wladimir J. van der Laan
f4ac48d30a Merge #20735: script: Remove outdated extract-osx-sdk.sh
1fbb8b6fa3 script: Remove outdated extract-osx-sdk.sh (Hennadii Stepanov)

Pull request description:

  The removed script was used to prepare `MacOSX10.11.sdk` which we do not use since 0.20 (7e2104433c from #16392).

  See https://github.com/bitcoin-core/docs/pull/68.

ACKs for top commit:
  laanwj:
    ACK 1fbb8b6fa3

Tree-SHA512: ab5128ec2c505a43b1a12e06abf38d8fa8f62a03efe760de2c842550e82f68ca512dee1b6c220a4f8af2465a9c35a429ca91cb179cc4ce016fcd87c61fbc8b07
2020-12-21 20:52:24 +01:00
Wladimir J. van der Laan
cc2a5ef9b2 Merge #20683: test: Fix restart node race
fab46b34f4 test: Fix restart node race (MarcoFalke)

Pull request description:

  It is not allowed to start a node before it has been fully stopped. Otherwise it could lead to intermittent issues due to access issues (e.g. cookie file https://cirrus-ci.com/task/6409665024098304?command=ci#L4793)

  Fix that by waiting for the node to fully stop.

ACKs for top commit:
  laanwj:
    code review ACK fab46b34f4

Tree-SHA512: 7605cac0573a7b04f05ff110d0131e8940d87f7baf6d698505ed16b363d4d15b1e552c5ffd1a187c8fe5639f7e265c3122734c85283275746e46bd789614fd21
2020-12-21 20:20:30 +01:00
MarcoFalke
2556a973ed Merge #20731: rpc: Add missing description of vout in getrawtransaction help text
b23349b880 rpc: Add missing description of vout in getrawtransaction help text (Ben Carman)

Pull request description:

  In `getrawtransaction` the vout did not have a description. I gave it the same description as the one used in `decoderawtransaction`.

ACKs for top commit:
  MarcoFalke:
    ACK b23349b880 🏯

Tree-SHA512: 3833b97c82a46dfeb7ac825d4b2514b4b05ce54ac41f2144a8e2f2093b3411fe1d090c1e5b0c3d09200a2ea164c8d17ece12cdb43bbaeaeccc51a9da6dd7b7a3
2020-12-21 19:57:38 +01:00
Fabian Jahr
b111410914 test: Add MuHash3072 fuzz test 2020-12-21 19:57:33 +01:00
Fabian Jahr
c122527385 bench: Add Muhash benchmarks
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-21 19:57:33 +01:00
Fabian Jahr
7b1242229d test: Add MuHash3072 unit tests
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-12-21 19:57:28 +01:00
Fabian Jahr
adc708c98d crypto: Add MuHash3072 implementation
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-21 19:57:04 +01:00
MarcoFalke
1d7243d307 Merge #20737: test: Add missing assignment in mempool_resurrect.py
fada8b019a test: Add missing assignment in mempool_resurrect.py (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: f438d85cd14a91eabfc380d9ee120cc7a7f9103cf0cd1cf565f675f386f82d966901c0ad3f60b8c462642fbf0a3791dbbd774f9b07668d22b58eb575c8d702c1
2020-12-21 18:41:57 +01:00
Ben Carman
b23349b880 rpc: Add missing description of vout in getrawtransaction help text 2020-12-21 09:57:06 -06:00
MarcoFalke
fada8b019a test: Add missing assignment in mempool_resurrect.py 2020-12-21 16:02:29 +01:00
MarcoFalke
1077c93a34 Merge #20692: test: run mempool_resurrect.py even with wallet disabled
11a32722f0 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz)

Pull request description:

  Another functional test rewritten as proposed in #20078

  **Request for help:**
  `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected.

  However, `node.gettransaction(txid)` throws the error:
  ```sh
  Traceback (most recent call last):
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main
      self.run_test()
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test
      assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda>
      assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5)
  ```

  Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly?

ACKs for top commit:
  MarcoFalke:
    ACK 11a32722f0

Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
2020-12-21 15:23:57 +01:00
Hennadii Stepanov
1fbb8b6fa3 script: Remove outdated extract-osx-sdk.sh 2020-12-21 15:03:08 +02:00
Sebastian Falbesoner
c86b9a65eb contrib: remove verify.sh
This script has been replaced by verify.py.
2020-12-21 14:01:16 +01:00
Sebastian Falbesoner
c84838e7af contrib: binary verification script verify.sh rewritten in python 2020-12-21 14:00:17 +01:00
Wladimir J. van der Laan
68c7acf6bb Merge #20671: Replace boost::optional with std::optional
fa4435e22f Replace boost::optional with std::optional (MarcoFalke)
fa7e803f3e Remove unused MakeOptional (MarcoFalke)
fadd4029dc psbt: Assert that tx has a value in UpdatePSBTOutput (MarcoFalke)

Pull request description:

  Now that we can use std::optional from the vanilla standard library, drop the third-party boost dependency

ACKs for top commit:
  practicalswift:
    cr ACK fa4435e22f: patch looks correct!
  laanwj:
    code review ACK fa4435e22f
  hebasto:
    ACK fa4435e22f, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 50c5a1a130cac65e043e0177ba5b009fc2ba09343af4e23322ff2eb32184a55f8f2dea66e7a1b9d9acf56bc164eef4e47448750549a07f3b661199ac9bf9afef
2020-12-21 13:12:16 +01:00
MarcoFalke
f061da2887 Merge #20697: ci: Fix COMMIT_RANGE variable value for PRs
3c2478c385 ci: Print COMMIT_RANGE to the log as it was in Travis CI (Hennadii Stepanov)
c123892c2e ci: Drop Travis-specific workaround for shellcheck (Hennadii Stepanov)
10af252d97 ci: Drop Travis-specific way to set COMMIT_RANGE variable (Hennadii Stepanov)
93504da3a9 ci: Fix COMMIT_RANGE variable value for PRs (Hennadii Stepanov)

Pull request description:

  This PR:
  - is a #20658 and #20682  followup
  - set the `COMMIT_RANGE` variable correctly for PRs
  - cleans up Travis-specific code
  - prints COMMIT_RANGE value to the log for convenience as it was in Travis CI

ACKs for top commit:
  MarcoFalke:
    ACK 3c2478c385

Tree-SHA512: beb933352b10fd5eb3e66373ddb62439e4f3a03b50fb037ee89fa92c0706cec41d05f2d307f15bb18d1e634e6464f4e123b7e2f88703c8edfd145d8d6eff0b1a
2020-12-21 12:09:16 +01:00
Hennadii Stepanov
3c2478c385 ci: Print COMMIT_RANGE to the log as it was in Travis CI 2020-12-21 12:13:43 +02:00
Hennadii Stepanov
c123892c2e ci: Drop Travis-specific workaround for shellcheck 2020-12-21 12:11:00 +02:00
Hennadii Stepanov
10af252d97 ci: Drop Travis-specific way to set COMMIT_RANGE variable 2020-12-21 12:10:59 +02:00
Hennadii Stepanov
93504da3a9 ci: Fix COMMIT_RANGE variable value for PRs 2020-12-21 12:10:43 +02:00
MarcoFalke
e9efb64a07 Merge #20733: inline non-member functions with body in fuzzing headers
d8b9cec25b inline non-member functions with body in fuzzing headers (Patrick Strateman)

Pull request description:

  Resolves the issue noted [here](https://github.com/bitcoin/bitcoin/pull/20560#issuecomment-748339644)

ACKs for top commit:
  MarcoFalke:
    ACK d8b9cec25b

Tree-SHA512: fb34707e2d2c5b664d4160e0e4b56e3df9fb2c9045da6ddea7139e0b4982262c4e085812a8543a6221febc9cd0815423b8287fec66baae3236e5f3339cc9df8c
2020-12-21 08:13:08 +01:00
Patrick Strateman
d8b9cec25b inline non-member functions with body in fuzzing headers 2020-12-20 18:19:43 -05:00
Fabian Jahr
0b4d290bf5 crypto: Add Num3072 implementation
Num3072 is a specialized bignum implementation used in MuHash3072.

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-20 23:27:58 +01:00
Fabian Jahr
589f958662 build: Check for 128 bit integer support
Used in MuHash3072 implementation.

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-12-20 22:08:15 +01:00
fanquake
e9189a750b build: more robustly check for fcf-protection support
When using Clang 7, we may end up trying to use the flag when it won't
work properly, which can lead to confusing errors. i.e:
```bash
/usr/bin/ld: error: ... <corrupt x86 feature size: 0x8>
```

Use CHECK_LINK_FLAG & --fatal-warnings to ensure we wont use the flag in this case.
2020-12-20 21:43:58 +08:00
John Newbery
3002b4af2b [net processing] Guard m_continuation_block with m_block_inv_mutex 2020-12-20 10:06:14 +00:00
John Newbery
184557e8e0 [net processing] Move hashContinue to net processing
Also rename to m_continuation_block to better communicate meaning.
2020-12-20 10:03:38 +00:00
John Newbery
c853ef002e scripted-diff: rename vBlockHashesToAnnounce and vInventoryBlockToSend
-BEGIN VERIFY SCRIPT-
sed -i 's/vBlockHashesToAnnounce/m_blocks_for_headers_relay/g' src/net_processing.*
sed -i 's/vInventoryBlockToSend/m_blocks_for_inv_relay/g' src/net_processing.*
-END VERIFY SCRIPT-
2020-12-20 10:01:48 +00:00
John Newbery
53b7ac1b7d [net processing] Move block inventory data to Peer 2020-12-20 10:01:48 +00:00
John Newbery
78040f9168 [net processing] Rename nStartingHeight to m_starting_height
Not done as a scripted diff to avoid misnaming the local variable in
ProcessMessage().
2020-12-20 10:01:46 +00:00
John Newbery
77a2c2f8f9 [net processing] Move nStartingHeight to Peer 2020-12-20 10:01:26 +00:00
John Newbery
717a374e74 [net processing] Improve documentation for Peer destruction/locking
Suggested here:

- https://github.com/bitcoin/bitcoin/pull/19607#discussion_r467071878
- https://github.com/bitcoin/bitcoin/pull/19829#discussion_r546116786
2020-12-19 14:49:03 +00:00
MarcoFalke
fa4435e22f Replace boost::optional with std::optional 2020-12-19 09:46:55 +01:00
MarcoFalke
fa7e803f3e Remove unused MakeOptional
The only use was to work around a compiler warning in an ancient
compiler, which we no longer support.
2020-12-19 09:45:58 +01:00
MarcoFalke
fadd4029dc psbt: Assert that tx has a value in UpdatePSBTOutput
This was previously done implicitly in boost::optional by BOOST_ASSERT.
Also, it was checked at runtime by valgrind if for some reason the
assert was disabled.

std::optional dereference won't assert, so add the Assert here
explicitly.

The explicit Assert also helps to document the code better.
2020-12-19 09:45:39 +01:00
Wladimir J. van der Laan
f1dbf92ff0 Merge #20434: contrib: Parse ELF directly for symbol and security checks
a0a771843f contrib: Changes to checks for PowerPC64 (Luke Dashjr)
634f6ec4eb contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan)

Pull request description:

  Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly.

  Add a small dependency-less ELF parser specific to the checks.

  This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future.

  This removes the build-time dependency for `readelf`.

  It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE.

Top commit has no ACKs.

Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
2020-12-18 12:16:40 +01:00
MarcoFalke
83e4670fd7 Merge #20690: Clean up logging of outbound connection type
6d1e85f475 Clean up logging of outbound connection type (Suhas Daftuar)

Pull request description:

  We have a function that converts `ConnectionType` enums to strings, so use it.

  Suggested by ajtowns in https://github.com/bitcoin/bitcoin/pull/19858#discussion_r540791588

ACKs for top commit:
  amitiuttarwar:
    ACK 6d1e85f475
  naumenkogs:
    ACK 6d1e85f

Tree-SHA512: f5084d8b5257380696d9fde86a8873e190cd4553feb07fa49df39bbd9510bf5832d190a3bca1571c48370d16a17c7a34900857b21b27bec0777bfa710211d7bb
2020-12-18 10:02:55 +01:00
MarcoFalke
20f4a9421b Merge #20682: ci: Install missing lint packages
faeb40bee3 ci: Install missing lint packages (MarcoFalke)

Pull request description:

  The cirrus container is vanilla ubuntu, so we need to install the needed packages

ACKs for top commit:
  hebasto:
    ACK faeb40bee3, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: e56198108e26ea0ba2a344b1b74bc294652f34e9866cca053a25fb1b83bbd87ea40254c340e5e169fdfcbd4dcb39fdc2078b5157ca729a22a9a1792ec514a33e
2020-12-18 09:53:38 +01:00
MarcoFalke
3dd3f7c0af Merge bitcoin-core/gui#155: Fix checkbox layout in Create Wallet dialog
e71b656f31 qt: Align layout of checkboxes (Hennadii Stepanov)

Pull request description:

  From https://github.com/bitcoin/bitcoin/issues/20555#issuecomment-747742627:
  > 0.21.0rc3 compiled from source, Ubuntu 20.04 + i3: tested mainnet, testnet, and signet. created a descriptor wallet in qt, tested torv3 with addnode, and checked if anchors.dat is created on shutdown and removed on startup.
  >
  > I've noticed the "Advanced options" label being cut-off in the wallet creation dialog.

  This PR should fix this issue.

ACKs for top commit:
  MarcoFalke:
    review ACK e71b656f31

Tree-SHA512: dcc997a440a96fc936296323afaf9f4040ac06cf5ad32724a88c2e2e16c726c35b1b2bacbb3e081839d8a69b5a6086b2570d6b2e98976f29b5427db2d434eee9
2020-12-18 08:53:38 +01:00
MarcoFalke
b27104dc52 Merge #20687: wallet: Add missing check for -descriptors wallet tool option
fae32f295c wallet: Add missing check for -descriptors wallet tool option (MarcoFalke)
faf8f61368 test: Add missing check for is_sqlite_compiled (MarcoFalke)
fa7dde1c41 wallet: Pass ArgsManager into ExecuteWalletToolFunc instead of using global (MarcoFalke)

Pull request description:

  Also, fix a test failure when compiled without sqlite

ACKs for top commit:
  ryanofsky:
    Code review ACK fae32f295c. Thanks for implementing the -descriptors check and dealing with the test failure!
  jonatack:
    Code review utACK fae32f295c

Tree-SHA512: 3d7710694085822739a8316e4abc6db270799ca6ff6b0f9e5563ae240da65ae6a9cab7ba2647feae6ba540dac40b55b38ed41c8f6ed0bf02a3d1536284448927
2020-12-18 07:56:15 +01:00
MarcoFalke
faeb40bee3 ci: Install missing lint packages
Also merge script into ci/lint_run_all.sh because env vars weren't
exported properly
2020-12-18 07:40:57 +01:00
MarcoFalke
9b28bd73a3 Merge #20691: ci, doc: Travis CI features and mentions cleanup
95487b0553 doc: Drop mentions of Travis CI as it is no longer used (Hennadii Stepanov)
09d105ef0f ci: Drop travis_fold feature as Travis CI is no longer used (Hennadii Stepanov)

Pull request description:

  As Travis CI is no longer used, this PR:
  - drops `travis_fold` feature
  - drops mentions of Travis CI in docs

ACKs for top commit:
  MarcoFalke:
    ACK 95487b0553

Tree-SHA512: 2e259bb8b1e37bcefc1251737bb2716f06ddb57c490010b373825c4e70f42ca38efae69a2f63f21f577d7cee3725b94097bdddbd313f8ebf499281cf97c53cef
2020-12-18 07:32:28 +01:00
Michael Dietz
11a32722f0 test: run mempool_resurrect.py even with wallet disabled 2020-12-17 21:37:07 -06:00
Hennadii Stepanov
e71b656f31 qt: Align layout of checkboxes 2020-12-18 01:36:21 +02:00
Hennadii Stepanov
95487b0553 doc: Drop mentions of Travis CI as it is no longer used 2020-12-18 01:15:53 +02:00
Wladimir J. van der Laan
816314ef0f Merge #20644: Add patch to make codesign_allocate compatible with Apple's
a4118c6e20 Add patch to make codesign_allocate compatible with Apple's (Pieter Wuille)

Pull request description:

  This is an alternative to #20638.

  The problem is that Apple's codesign(_allocate) apparently rounds the "vmsize" attribute on the __LINKEDIT section to a multiple of 0x2000 on x86_64 rather than 0x1000 (as their published source code does). This divergence means that the binary signed by codesign is slightly different from the one recreated by our reattach-sig-to-gitian-output process, and the signature being invalid.

  This fixes it by patching our codesign_allocate source code to also use 0x2000. In tests, this appears to result in matching binaries.

ACKs for top commit:
  jonasschnelli:
    Tested ACK a4118c6e20 - removed the osx cache, built commit a4118c6e20 for osx in gitian (dependency where built, patch was applied), signed on my signing mac (detach-sig-create), ran gitian osx signer with the produces signature and the a4118c6e20 build (detach-sig-apply), signature then was successful verified on my Mac (codesign -v /Volumes/Bitcoin-Core/Bitcoin-Qt.app)
  MarcoFalke:
    Concept ACK a4118c6e20

Tree-SHA512: 07b8cdf8216249ddfe4bd38b39f2b48b2e190d4002b84d8981e62197bbbc9f25ac5c137bcc32057b23fbf38cbb2889ef95101ce008edfbf608cd170b88b3acbc
2020-12-17 21:34:35 +01:00
Hennadii Stepanov
09d105ef0f ci: Drop travis_fold feature as Travis CI is no longer used 2020-12-17 22:02:12 +02:00
MarcoFalke
fae32f295c wallet: Add missing check for -descriptors wallet tool option 2020-12-17 20:36:41 +01:00
MarcoFalke
faf8f61368 test: Add missing check for is_sqlite_compiled 2020-12-17 20:33:45 +01:00
Suhas Daftuar
6d1e85f475 Clean up logging of outbound connection type 2020-12-17 14:00:42 -05:00
Jon Atack
86c495223f net: add CNode::IsInboundOnion() public getter and unit tests 2020-12-17 19:56:12 +01:00
Jon Atack
6609eb8cb5 net: assert CNode::m_inbound_onion is inbound in ctor
and drop an unneeded check in CNode::ConnectedThroughNetwork()
2020-12-17 19:56:10 +01:00
Jon Atack
993d1ecd19 test, fuzz: fix constructing CNode with invalid inbound_onion
as CNode ctor should only be passed inbound_onion = true
when the connection is inbound
2020-12-17 19:56:00 +01:00
MarcoFalke
b7136c11ab Merge #20686: fuzz, refactor: replace CNode code with fuzz/util.h::ConsumeNode()
23d8f34689 fuzz: replace CNode code with fuzz/util.h::ConsumeNode() (Jon Atack)

Pull request description:

  Noticed this while updating the CNode fuzzing in #20210.

  - cc26fab48d created `test/fuzz/net.cpp` in May 2020

  - 79ef8324d4 created a CNode factory utility `test/fuzz/util.h::ConsumeNode()` in October 2020

  This PR updates `fuzz/net.cpp` from the first commit to use `ConsumeNode()` from the second commit.

ACKs for top commit:
  MarcoFalke:
    ACK 23d8f34689

Tree-SHA512: 26f7685395b3d48fcf40dde0d479d5c2fb4e953ec9371940b19eee16bb30aee4840b081e1a918b924a9704c1bef484302ea3e8fe63819a3bba73e7eb805164f1
2020-12-17 19:47:28 +01:00
MarcoFalke
fa7dde1c41 wallet: Pass ArgsManager into ExecuteWalletToolFunc instead of using global 2020-12-17 19:16:00 +01:00
Jon Atack
23d8f34689 fuzz: replace CNode code with fuzz/util.h::ConsumeNode() 2020-12-17 18:49:30 +01:00
Wladimir J. van der Laan
190d3d8a75 Merge #20681: doc: Convert depends options list from html to markdown
7b6887e75a doc: Convert depends options list from html to markdown (Wladimir J. van der Laan)

Pull request description:

  This makes it easier to read in `less`, which is important for install instructions.

  Rendered: [before](7ef6b1c51d/depends (dependency-options)) - [after](d97042406f/depends/README.md (dependency-options))

ACKs for top commit:
  jonatack:
    Code review re-ACK 7b6887e75a per `git diff d970424 7b6887e`
  hebasto:
    re-ACK 7b6887e75a

Tree-SHA512: 02970b2bb97d2e8fb2d66470f6d70662653fda176bf6f4861742823b361fdc7ab6a2b44143480ac1a525b8d7808b6a068e8b3677dbba16cd783b4cab90470af5
2020-12-17 18:49:26 +01:00
MarcoFalke
143bd108ed Merge #19137: wallettool: Add dump and createfromdump commands
23cac24dd3 tests: Test bitcoin-wallet dump and createfromdump (Andrew Chow)
a88c320041 wallettool: Add createfromdump command (Andrew Chow)
e1e7a90d5f wallettool: Add dump command (Andrew Chow)

Pull request description:

  Adds two commands to the `bitcoin-wallet` tool: `dump` and `createfromdump`. These commands will be useful for a wallet storage migration in the future. It is also generally useful to have a storage agnostic dump like this. These commands are similar to BDB's `db_dump` and `db_load` tools. This can also be useful for manual construction of a wallet file for tests.

  `dump` outputs every key-value pair from the wallet as comma separated hex. Each key-value pair is on its own line with the key and value in hex separated by a comma. This is output to the file specified by the new `-dumpfile` option.

  `createfromdump` takes a file produced by `dump` and creates a new wallet file with exactly the records specified in that file.

  A new option `-dumpfile` is added to the wallet tool. When used with `dump`, the records will be written to the specified file. When used with `createfromdump`, the file is read and the key-value pairs constructed from it. `createfromdump` requires `-dumpfile`.

  A simple round-trip test is added to the `tool_wallet.py`.

  This PR is based on #19334,

ACKs for top commit:
  Sjors:
    re-utACK 23cac24
  MarcoFalke:
    re review ACK 23cac24dd3 only change is rebase and removing useless shared_ptr wrapper 🎼
  ryanofsky:
    Code review ACK 23cac24dd3. Only changes since last review rebase and changing a pointer to a reference

Tree-SHA512: 2d63cf62baca3d16495aa698dc02f7d889c81b41015e9c92c23c275bb4a690fc176d351c3fd7f310bd6b17f5a936cc9be694cbecd702af741b96c0f530e72fa2
2020-12-17 15:18:37 +01:00
MarcoFalke
fab46b34f4 test: Fix restart node race 2020-12-17 15:06:31 +01:00
Wladimir J. van der Laan
af4ce674da Merge #20635: fix misleading comment about call to non-existing function
cc3044ccdb fix misleading comment about call to non-existing function (pox)

Pull request description:

  The comment seems to be describing the subsequent call to `SyncTransaction` but refers to it as `SyncNotifications`, which is not any function currently in the codebase.

  It's best to just remove the "what" aspect of the comment and focus on the "why", which also reduces the risk of similar documentation errors in the future, in case the function ever gets renamed, for example.

ACKs for top commit:
  laanwj:
    ACK cc3044ccdb

Tree-SHA512: 882ff17836ef585a603dc504f3dd21f56f682e49b28a0998f23fd16025826fbb083b7978db3ee70d0e0ff2c86fd6c3fd99a2361e5d45c765fdc5822c5f14c0a7
2020-12-17 15:06:01 +01:00
Hennadii Stepanov
5e0dedb111 build: Define .INTERMEDIATE target once only 2020-12-17 14:10:52 +02:00
Wladimir J. van der Laan
7b6887e75a doc: Convert depends options list from html to markdown
This makes it easier to read in `less`, which is important for install
instructions.
2020-12-17 13:00:36 +01:00
Wladimir J. van der Laan
cfbfd389f6 Merge #20668: doc: warn that incoming conns are unlikely when not using default ports
010eed3ce0 doc: warn that incoming conns are unlikely when not using default ports (Adam Jonas)

Pull request description:

  Closes #5150.

  This was mostly copied from #5285 by sulks, who has since quit GitHub.

  The issue has remained open for 6 years, but the extra explanation still seems useful.

ACKs for top commit:
  laanwj:
    re-ACK 010eed3ce0

Tree-SHA512: d240fb06bba41ad8898ced59356c10adefc09f3abb33e277f8e2c5980b40678f2d237f286b476451bb29d2b94032a7dee2ada3b2efe004ed1c2509e70b48e40f
2020-12-17 12:10:00 +01:00
Wladimir J. van der Laan
f0913f2f95 Merge #20677: doc: Remove shouty enums in net_processing comments
0c41c10830 doc: Remove shouty enums in net_processing comments (Suhas Daftuar)

Pull request description:

  This uses the `CNode::ConnectionTypeAsString()` strings in place of the all-caps enums in a couple of comments in `net_processing`, as suggested by ajtowns in https://github.com/bitcoin/bitcoin/pull/19858#discussion_r540821050.

ACKs for top commit:
  practicalswift:
    ACK 0c41c10830
  jnewbery:
    ACK 0c41c10830
  laanwj:
    ACK 0c41c10830

Tree-SHA512: c8ab905e151ebb144c3f878277dc59d77591e4b39632658407b69b80b80d65825d5a391b01e2aea6af2fdf174c143dfe7d2f3eba84a020a58d7926458fdcd0a5
2020-12-17 11:59:58 +01:00
Wladimir J. van der Laan
7ef6b1c51d Merge #19961: doc: tor.md updates
a34eceb4cc doc: update -externalip documentation in tor.md (Jon Atack)
dc8a591222 doc: add tor.md section on how to get tor info via bitcoind (Jon Atack)
e1765d8b04 doc: update tor.md address examples from onion v2 to v3 (Jon Atack)

Pull request description:

  It looks like `doc/tor.md` could use some updates and improvements, not only for Tor v3, but also for setting multiple addresses with `-externalip` (see the conversation from http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-16.html#l-39), how to see information about your Tor config via Bitcoin Core, and other improvements.

  Closes #19924.

ACKs for top commit:
  laanwj:
    ACK a34eceb4cc

Tree-SHA512: 3197cdca1188dbd645c8f9e6ed7c023da5ad9bcf246a6bcbfbe6078f40c01c563032b4906736cde253a2daf71aaed28a659121628891a5d0bf6e89f821a17a28
2020-12-17 11:40:00 +01:00
MarcoFalke
d0e76b5050 Merge bitcoin-core/gui#153: Define MAX_DIGITS_BTC for magic number in BitcoinUnits::format
198fff88f3 GUI: Define MAX_DIGITS_BTC for magic number in BitcoinUnits::format (Luke Dashjr)

Pull request description:

  A magic number snuck in with https://github.com/bitcoin/bitcoin/pull/16432

ACKs for top commit:
  hebasto:
    ACK 198fff88f3, I have reviewed the code and it looks OK, I agree it can be merged.
  kristapsk:
    utACK 198fff88f3

Tree-SHA512: 78dc23c2ae61bac41e5e34eebf57274599cb2ebb0b18d46e8a3228d42b256a1bc9bb17091c748f0f692ef1c4c241cfbd3e30a12bcd12222a234c1a9547ebe786
2020-12-17 10:53:34 +01:00
MarcoFalke
83abd6b126 Merge #20680: ci: Only use credits for pull requests to the main repo
facf5e37f6 ci: Only use credits for pull requests to the main repo (MarcoFalke)

Pull request description:

  No need to spend credits for the `master` branch, because the build shouldn't fail there anyway and it is not time-critical to get a fast feedback.

  Some other changes:

  * Disable `stateful` for faster scheduling
  * Reduce lint memory from 8G to 1G for faster scheduling

ACKs for top commit:
  hebasto:
    ACK facf5e37f6, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 4d052e68217086574b9ea3d603cde1b585833c289d47dfed5308ff001d02964dc75ec3b3ebf5b233ccd09c47ad4ff5ba0bef639bf6362d984e7c49fca8fec24b
2020-12-17 10:41:39 +01:00
MarcoFalke
8452f922d2 Merge #19050: doc: Add warning for rest interface limitation
5c3eaf9983 doc: Add warnings for http interfaces limitations (Fabian Jahr)

Pull request description:

  `libevent`, which is used for our rest interface, can use up all of the available file descriptors in a system if too many connections are opened at once. If a new block is connected at the same time and can not be written to disk because there are no file descriptors available, the node crashes. Based on my investigation so far the issue is best solved upstream which means we have to wait for the next release (2.2). In the meantime it would be good if we would warn users of this limitation.

  See #11368 for more background.

ACKs for top commit:
  MarcoFalke:
    ACK 5c3eaf9983

Tree-SHA512: 73914538588477ead19068f5832fdcc8e0eb736e51f73b3aca501c93165e5ad634c2511a3fcffff251adcd3efda23a742b48211ad9d3b2a29cdeac17201d06a1
2020-12-17 10:00:38 +01:00
MarcoFalke
facf5e37f6 ci: Only use credits for pull requests to the main repo 2020-12-17 09:30:37 +01:00
MarcoFalke
6f2ca726ce Merge #20658: ci: Move linter task to cirrus
4045a6722c ci: Use cpu=1 for linter (Dhruv Mehta)
739d39022d ci: Move linter task to cirrus (Dhruv Mehta)

Pull request description:

  Solves #20467: Move linter to Cirrus-CI as Travis-CI.org is shutting down

ACKs for top commit:
  MarcoFalke:
    ACK 4045a6722c

Tree-SHA512: 9aa7487ac86c91fc68bb584d29134e304dbd46702514a5d47d1ef0de6b877d96d42b7589870fc67ad9a31f5d3a789728446da4418688f336111a9ba0f8de5feb
2020-12-17 09:05:15 +01:00
Dhruv Mehta
4045a6722c ci: Use cpu=1 for linter 2020-12-16 17:34:25 -08:00
Wladimir J. van der Laan
1811e488d5 Merge #20575: Do not run functions with necessary side-effects in assert()
5021810650 Make CanFlushToDisk a const member function (practicalswift)
281cf99554 Do not run functions with necessary side-effects in assert() (practicalswift)

Pull request description:

  Do not run functions with necessary side-effects in `assert()`.

ACKs for top commit:
  laanwj:
    Code review ACK 5021810650
  sipa:
    utACK 5021810650
  theStack:
    Code Review ACK 5021810650 🟢

Tree-SHA512: 38b7faccc2f16a499f9b7b1b962b49eb58580b2a2bbf63ea49dcc418a5ecc8f21a0972fa953f66db9509c7239af67cfa2f9266423fd220963d091034d7332b96
2020-12-16 23:14:53 +01:00
Jon Atack
a34eceb4cc doc: update -externalip documentation in tor.md 2020-12-16 22:58:14 +01:00
Jon Atack
dc8a591222 doc: add tor.md section on how to get tor info via bitcoind 2020-12-16 22:58:12 +01:00
Jon Atack
e1765d8b04 doc: update tor.md address examples from onion v2 to v3 2020-12-16 22:58:10 +01:00
Wladimir J. van der Laan
4acbcfa97d Merge #20470: build: Replace genisoimage with xorriso
7587d11ec9 build: remove cdrkit package from depends (fanquake)
0df9819126 build: Replace genisoimage with xorriso (fanquake)
22437fc72e build: Run libdmg-hfsplus's DMG tool in make deploy (Carl Dong)

Pull request description:

  This is a redo of fanquake's https://github.com/bitcoin/bitcoin/pull/18151, which, aside from switching us from the deprecated `genisoimage` to the maintained `xorriso`, is also necessary for Guix to achieve determinism without using faketime.

  > xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
  > more maintained, and has the right toggles for us to achieve output
  > determinism without using blunt tools like faketime.
  >
  > In this commit, we use xorrisofs from the build environment rather than
  > building it ourselves using depends. This is not necessary and can be
  > changed in the future.
  >
  > From wiki.debian.org/genisoimage?action=recall&rev=11 :
  >
  > > The classical command line interface for production of ISO 9660
  > > filesystem images is the option set established by program mkisofs.
  > > For reasons of licensing and other problems with its author, Debian
  > > ships a fork of mkisofs, called genisoimage, which was split off in
  > > 2006 and then developed independently.
  > >
  > > Meanwhile, genisoimage gets no new features and not even bug fixes. It
  > > is first choice only if its options -udf or -hfs are needed.
  > >
  > > Replacement in most uses cases, especially for bootable ISO 9660
  > > filesystems, archiving, and backup, is xorrisofs which starts the -as
  > > mkisofs emulation mode of program xorriso.

ACKs for top commit:
  laanwj:
    ACK 7587d11ec9

Tree-SHA512: 62f3aad08fa8bf21192e951d7dd33b24975586d76834cfa3498f4b8cdb586cefec8cab2c073d1951a0884b5e182fd71ef2cf3accad98f84455016776ad3c5422
2020-12-16 22:12:38 +01:00
Suhas Daftuar
0c41c10830 doc: Remove shouty enums in net_processing comments 2020-12-16 14:36:09 -05:00
MarcoFalke
ae9ee5bdb1 Merge #20651: net: Make p2p recv buffer timeout 20 minutes for all peers
ea36a453e3 [net] Make p2p recv buffer timeout 20 minutes for all peers (John Newbery)

Pull request description:

  The timeout interval for the send and recv buffers was changed from 90
  minutes to 20 minutes in commit f1920e86 in 2013, except for peers that
  did not support the pong message (where the recv buffer timeout remained
  at 90 minutes). A few observations:

  - for peers that support BIP 31 (pong messages), this recv buffer
    timeout is almost redundant with the ping timeout. We send a ping
    message every two minutes, and set a timeout of twenty minutes to
    receive the pong response. If the recv buffer was really timing out,
    then the pong response would also time out.
  - BIP 31 is supported by all nodes of p2p version 60000 and higher, and
    has been in widespread use since 2013. I'd be very surprised if there
    are many nodes on the network that don't support pong messages.
  - The recv buffer timeout is not specified in any p2p BIP. We're free to
    set it at any value we want.
  - A peer that doesn't support BIP 31 and hasn't sent any message to us
    at all in 90 minutes is unlikely to be useful for us, and is more likely
    to be evicted AttemptToEvictConnection() since it'll have the worst
    possible ping time and isn't providing blocks/transactions.

  Therefore, we remove this check, and set the recv buffer timeout to 20
  minutes for all peers. This removes the final p2p version dependent
  logic from the net layer, so all p2p version data can move into the
  net_processing layer.

  Alternative approaches:

  - Set the recv buffer timeout to 90 minutes for all peers. This almost
    wouldn't be a behaviour change at all (pre-BIP 31 peers would still
    have the same recv buffer timeout, and we can't ever reach a recv buffer
    timeout higher than 21 minutes for post-BIP31 peers, because the pong
    timeout would be hit first).
  - Stop supporting peers that don't support BIP 31. BIP 31 has been in
    use since 2012, and implementing it is trivial.

ACKs for top commit:
  MarcoFalke:
    review ACK ea36a453e3
  promag:
    Code review ACK ea36a453e3.
  practicalswift:
    cr ACK ea36a453e3: patch looks correct
  ajtowns:
    ACK ea36a453e3
  sipa:
    utACK ea36a453e3
  jonatack:
    Code review ACK ea36a453e3

Tree-SHA512: df290bb32d2b5d9e59a0125bb215baa92787f9d01542a7437245f1c478c7f9b9831e5f170d3cd0db2811e1b11b857b3e8b2e03376476b8302148e480d81aab19
2020-12-16 20:29:31 +01:00
Wladimir J. van der Laan
427f8c2cff Merge #20248: test: fix length of R check in key_signature_tests
89895773b7 Fix length of R check in test/key_tests.cpp:key_signature_tests (Dmitry Petukhov)

Pull request description:

  The code before the fix only checked the length of R value of the last
  signature in the loop, and only for equality (but the length can be
  less than 32)

  The fixed code checks that length of the R value is less than or equal
  to 32 on each iteration of the loop

ACKs for top commit:
  laanwj:
    ACK 89895773b7

Tree-SHA512: 73eb2bb4a6c1c5fc11dd16851b28b43037ac06ef8cfc3b1f957429a1fca295c9422d67ec6c73c0e4bb4919f92e22dc5d733e84840b0d01ad1a529aa20d906ebf
2020-12-16 19:45:16 +01:00
Wladimir J. van der Laan
5b6f970e3f Merge #20171: Add functional test test_txid_inv_delay
bc4a230087 Remove redundant p2p lock tacking for tx download functional tests (Antoine Riard)
d3b5eac9a9 Add mutation for functional test test_preferred_inv (Antoine Riard)
06efb3163c Add functional test test_txid_inv_delay (Antoine Riard)
a07910abcd test: Makes wtxidrelay support a generic P2PInterface option (Antoine Riard)

Pull request description:

  This is a simple functional test to increase coverage of #19988, checking that txid announcements from txid-relay peers are delayed by TXID_RELAY_DELAY, assuming we have at least another wtxid-relay peer.

  You can verify new test with the following diff :

  ```
  diff --git a/src/net_processing.cpp b/src/net_processing.cpp
  index f14db379f..2a2805df5 100644
  --- a/src/net_processing.cpp
  +++ b/src/net_processing.cpp
  @@ -773,7 +773,7 @@ void PeerManager::AddTxAnnouncement(const CNode& node, const GenTxid& gtxid, std
       auto delay = std::chrono::microseconds{0};
       const bool preferred = state->fPreferredDownload;
       if (!preferred) delay += NONPREF_PEER_TX_DELAY;
  -    if (!gtxid.IsWtxid() && g_wtxid_relay_peers > 0) delay += TXID_RELAY_DELAY;
  +    //if (!gtxid.IsWtxid() && g_wtxid_relay_peers > 0) delay += TXID_RELAY_DELAY;
       const bool overloaded = !node.HasPermission(PF_RELAY) &&
           m_txrequest.CountInFlight(nodeid) >= MAX_PEER_TX_REQUEST_IN_FLIGHT;
       if (overloaded) delay += OVERLOADED_PEER_TX_DELAY;
  ```

ACKs for top commit:
  laanwj:
    ACK bc4a230087

Tree-SHA512: 150e806bc5289feda94738756ab375c7fdd23c80c12bd417d3112043e26a91a717dc325a01079ebd02a88b90975ead5bd397ec86eb745c7870ebec379a8aa711
2020-12-16 18:45:11 +01:00
Andrew Chow
23cac24dd3 tests: Test bitcoin-wallet dump and createfromdump 2020-12-16 12:33:09 -05:00
Andrew Chow
a88c320041 wallettool: Add createfromdump command
Creates a new wallet file using the dump file produced by the dump
command
2020-12-16 12:33:06 -05:00
Andrew Chow
e1e7a90d5f wallettool: Add dump command
Adds a new dump command to bitcoin-wallet which prints out all of the
wallet's records in hex.
2020-12-16 12:32:47 -05:00
fanquake
7587d11ec9 build: remove cdrkit package from depends 2020-12-16 12:29:14 -05:00
fanquake
0df9819126 build: Replace genisoimage with xorriso
xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
more maintained, and has the right toggles for us to achieve output
determinism without using blunt tools like faketime.

In this commit, we use xorrisofs from the build environment rather than
building it ourselves using depends. This is not necessary and can be
changed in the future.

From https://wiki.debian.org/genisoimage?action=recall&rev=11 :

> The classical command line interface for production of ISO 9660
> filesystem images is the option set established by program mkisofs.
> For reasons of licensing and other problems with its author, Debian
> ships a fork of mkisofs, called genisoimage, which was split off in
> 2006 and then developed independently.
>
> Meanwhile, genisoimage gets no new features and not even bug fixes. It
> is first choice only if its options -udf or -hfs are needed.
>
> Replacement in most uses cases, especially for bootable ISO 9660
> filesystems, archiving, and backup, is xorrisofs which starts the -as
> mkisofs emulation mode of program xorriso.
2020-12-16 12:29:14 -05:00
Wladimir J. van der Laan
ad3d4b3929 Merge #20661: Only select from addrv2-capable peers for torv3 address relay
37fe80e626 Only consider addrv2 peers for relay of non-addrv1 addresses (Pieter Wuille)
83f8821a6f refactor: add IsAddrCompatible() to CNode (Pieter Wuille)

Pull request description:

  When selecting peers to relay an address to, only pick addrv2-capable ones if the address cannot be represented in addr(v1).

  Without this I expect that propagation of torv3 addresses over the cleartext network will be very hard for a while.

ACKs for top commit:
  jonatack:
    ACK 37fe80e626
  vasild:
    ACK 37fe80e626

Tree-SHA512: 18a854ea43ad473cf89b9c5193b524109d7af75c26f7aa7e26cd72ad0db52f19c8001d566c607a7e6772bc314f770f09b6c3e07282d110c5daea193edc592cd2
2020-12-16 18:09:06 +01:00
MarcoFalke
69f1ee1922 Merge #20365: wallettool: add parameter to create descriptors wallet
173cc9b7be test: walettool create descriptors (Ivan Metlushko)
345e88eecf wallettool: add param to create descriptors wallet (Ivan Metlushko)
6d3af3ab62 wallettool: pass in DatabaseOptions into MakeWallet (Ivan Metlushko)

Pull request description:

  Rationale: expose and promote descriptor wallets in more places; make cli tool more consistent with `createwallet` rpc.

  Add `-descriptors` parameter which is off by default. When specified it will create a new descriptors wallet with sqlite backend, which is consistent with `createwallet` rpc.

  This PR is based on a suggestion from **ryanofsky** https://github.com/bitcoin/bitcoin/pull/19137#discussion_r516779603

  Example:
  ```
  $ ./src/bitcoin-wallet  -wallet=fewty -descriptors create
  Topping up keypool...
  Wallet info
  ===========
  Name: fewty
  Format: sqlite
  Descriptors: yes
  Encrypted: no
  HD (hd seed available): yes
  Keypool Size: 6000
  Transactions: 0
  Address Book: 0
  ```
  ```
  $ ./src/bitcoin-wallet  -wallet=fewty create
  Topping up keypool...
  Wallet info
  ===========
  Name: fewty
  Format: bdb
  Descriptors: no
  Encrypted: no
  HD (hd seed available): yes
  Keypool Size: 2000
  Transactions: 0
  Address Book: 0
  ```

ACKs for top commit:
  achow101:
    ACK 173cc9b7be
  ryanofsky:
    Code review ACK 173cc9b7be. This seems pretty nicely implemented now, with opportunities to clean up more and dedup later
  MarcoFalke:
    Concept ACK 173cc9b7be 🌠

Tree-SHA512: cc32ba336ff709de2707ee15f495b4617908e8700ede8401a58e894f44cda485c544d644023c9a6604d88a62db9d92152383ee2e8abf691688c25cf6e222c622
2020-12-16 17:43:20 +01:00
Wladimir J. van der Laan
3f205808a5 Merge #20605: init: Signal-safe instant shutdown
cd03513dc2 init: Signal-safe instant shutdown (Wladimir J. van der Laan)

Pull request description:

  Replace the 200ms polling loop with a faster and more efficient waiting operation. This should speed up short RPC tests.

  This change has been tried a few times before, but abandoned every time because solutions used a condition variable which is not safe for use in signals, as they need to be reentrant.

  On UNIX-ish OSes, use a safe way: a pipe. When shutdown is requested write a dummy byte to the pipe. Waiting for shutdown is a matter of a blocking read from the pipe.

  On Windows, there are no signals so using a condition variable is safe.

  This only affects bitcoind. The GUI is unaffected by this change, and keeps polling as before in `BitcoinGUI::detectShutdown()`. It might be possible to listen to a pipe there, too, but I'm not sure, and it's complicated by the GUI-node abstraction.

ACKs for top commit:
  jonatack:
    ACK cd03513dc2 tested on Debian 5.9.11-1 (2020-11-27) x86_64 GNU/Linux

Tree-SHA512: ed2f532f69fec4855c17bf7b8f3d0eb96e78ee2a3c13d374dd2c6add06e3ad6a190da8ed8f9d7a76532cf998222d67f57e35b206aec29675e96437448ae7e13c
2020-12-16 16:43:51 +01:00
MarcoFalke
9dbcd37105 Merge #20569: test: Fix intermittent wallet_multiwallet issue with got_loading_error
fab48da908 test: Fix intermittent wallet_multiwallet issue with got_loading_error (MarcoFalke)
fa8e15f7b7 test: pep8 wallet_multiwallet.py (MarcoFalke)

Pull request description:

  Failing the test after 10 iterations without a loading error is problematic because it may take 11 iterations to get a loading error.

  Fix that by running until a loading error occurs, which should happen in almost all runs within the first 10 iterations.

ACKs for top commit:
  ryanofsky:
    Code review ACK fab48da908. This seems like a good workaround. I think more ideally think load and unload RPCs would not have racy status reporting (suggested previously https://github.com/bitcoin/bitcoin/pull/19300#pullrequestreview-435362710 and

Tree-SHA512: 6b80b26d916276efe2a01af93bca7dbf71a3e67db9d3deb15175070719bf7d1325a1410d93e74c0316942e388faa2ba185dc9d3759c82d1c73c3c509b9997f05
2020-12-16 15:47:23 +01:00
Wladimir J. van der Laan
2d4eeaf9c5 Merge #20601: doc: Update for FreeBSD 12.2, add GUI Build Instructions
c175690561 doc: Update for FreeBSD 12.2, add GUI Build Instructions (Jarol Rodriguez)

Pull request description:

  The current FreeBSD Build documentation is a little outdated and underwhelming. This PR intends to keep the build-freebsd.md doc up to date. Here are the main improvements:
  - Introduce dependency information
  - New instructions for building the GUI
  - Instructions for supporting descriptor wallets
  - Various notes on the build and compile process

  **Before/Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-freebsd.md)

  **After/PR:** [render](2e8b9a5aac/doc/build-freebsd.md)

ACKs for top commit:
  laanwj:
    utACK c175690561

Tree-SHA512: 64aeb743c7f4ed167451454564b53d13c5c30d82bd2423799655c7b5b465a75733072fb0c574927c2588a46e89a68c6c57b008220acfd25336d445c9dc309f3b
2020-12-16 15:39:28 +01:00
practicalswift
546a0764f3 fuzz: Fill various small fuzzing gaps 2020-12-16 14:24:16 +00:00
Adam Jonas
010eed3ce0 doc: warn that incoming conns are unlikely when not using default ports 2020-12-16 09:24:03 -05:00
MarcoFalke
fa09f97bea fuzz: Call SendMessages after ProcessMessage to increase coverage 2020-12-16 15:08:27 +01:00
MarcoFalke
a023094fc4 Merge #20276: test: run mempool_expiry.py even with wallet disabled
3b064fcb9d test: run mempool_expiry.py even with wallet disabled (Michael Dietz)

Pull request description:

  Run the mempool expiry test even when the wallet was not compiled, as proposed in https://github.com/bitcoin/bitcoin/issues/20078.

ACKs for top commit:
  MarcoFalke:
    ACK 3b064fcb9d

Tree-SHA512: 5860dc021d02bc3752268ec1e859505bec87174953223b34b1af8a8e4ab66d645458fbf9571c0b816a9de891c3ff41314996e580869671fccd6972c093e78154
2020-12-16 14:52:34 +01:00
Hennadii Stepanov
87fe104537 depends: Use more legible qmake commands in qt package
This change gets rid of multiple `../` that makes reasoning about the
script and its maintaining much easier.

Co-authored-by: fanquake <fanquake@gmail.com>
2020-12-16 15:21:31 +02:00
Hennadii Stepanov
bf35a8da6e depends: Do not set build_subdir for qt package
This change makes the next commit possible without exporting the PATH
variable.
2020-12-16 15:21:24 +02:00
practicalswift
5a9ee0869b tests: Add fuzzing harness for node eviction logic 2020-12-16 13:00:47 +00:00
Jonas Schnelli
6af8a6232b Merge #20650: depends: Drop workaround for a fixed bug in Qt build system
267f259c0d depends: Drop workaround for a fixed bug in Qt build system (Hennadii Stepanov)

Pull request description:

  This PR drops workaround that was [introduced](1dec09b341) for Qt 5.2.1 for a bug in Qt build system that has been fixed in Qt 5.3.0.

  The bug reports:
   - https://bugreports.qt.io/browse/QTBUG-35444
   - https://bugreports.qt.io/browse/QTBUG-32519

  I've noted this change is a part of the #19716, but I think that a separate commit with the documented reason will benefit it.

ACKs for top commit:
  laanwj:
    Code review ACK 267f259c0d
  jonasschnelli:
    code Review ACK 267f259c0d
  practicalswift:
    cr ACK 267f259c0d: patch looks correct

Tree-SHA512: b994f94776b4f8bb2f996095c87c7fef55e74d1e64852a890d664275e3739ec890ee388b10baa15445dd24ec7b971ce57d396cb062dbed933c18b6b69525349f
2020-12-16 13:59:04 +01:00
MarcoFalke
b440c33179 Merge #20477: net: Add unit testing of node eviction logic
fee88237e0 Assert eviction at >= 29 candidates. Assert non-eviction at <= 20 candidates. (practicalswift)
685c428de0 test: Add unit testing of node eviction logic (practicalswift)
ed73f8cee0 net: Move eviction node selection logic to SelectNodeToEvict(...) (practicalswift)

Pull request description:

  Add unit testing of node eviction logic.

  Closes #19966.

ACKs for top commit:
  jonatack:
    ACK fee88237e0
  MarcoFalke:
    ACK fee88237e0 🐼

Tree-SHA512: 0827c35609122ca42bfabb17feaaee35c191ab4dc2e428282af425a6c176eaeaff2789b74a4f7eb4ca6b8cb10603068e90ca800e6ef3bc3904d50e76616f7a2b
2020-12-16 13:30:55 +01:00
practicalswift
fee88237e0 Assert eviction at >= 29 candidates. Assert non-eviction at <= 20 candidates. 2020-12-16 12:00:15 +00:00
practicalswift
685c428de0 test: Add unit testing of node eviction logic 2020-12-16 12:00:15 +00:00
practicalswift
ed73f8cee0 net: Move eviction node selection logic to SelectNodeToEvict(...) 2020-12-16 12:00:15 +00:00
Luke Dashjr
198fff88f3 GUI: Define MAX_DIGITS_BTC for magic number in BitcoinUnits::format 2020-12-15 22:21:26 +00:00
Wladimir J. van der Laan
dff0f6f753 Merge #20611: Move TX_MAX_STANDARD_VERSION to policy
fade6195b1 Move TX_MAX_STANDARD_VERSION to policy (MarcoFalke)

Pull request description:

  `primitives` should only be used for the raw datastructures (parsing and format). It is not the right place to document relay policy.

ACKs for top commit:
  laanwj:
    Code review ACK fade6195b1
  lontivero:
    Concept ACK fade6195b1

Tree-SHA512: f809c4aecd14d7e9feaa7b50b9c0697232991eef36190cd960bcfb0ad6e20c71a4f6aab48c7747cf8a681eb14feda60c55b09a37f128673d519567224f29cd97
2020-12-15 22:46:49 +01:00
Wladimir J. van der Laan
c434e2cca9 Merge #20615: cirrus: Schedule one task with paid credits for faster CI feedback
faf2c6e32e cirrus: Schedule one task with paid credits for faster CI feedback (MarcoFalke)

Pull request description:

  During times of high activity in the repo, the scheduling of Cirrus CI tasks might put them a few hours in the future. This is fine when all the tasks eventually pass. Though for failing tasks, a failure should ideally be shown to the author and reviewer as soon as possible.

  Compute credits can be used to schedule immediately: https://cirrus-ci.org/pricing/#compute-credits. Running all tasks with compute credits will probably be more expensive than our previous CI invoice. However, they are also more flexible.

  As a start we could enable only a single task and revisit/re-evaluate the next steps in a month.

ACKs for top commit:
  laanwj:
    Concept ACK faf2c6e32e
  fanquake:
    ACK faf2c6e32e
  practicalswift:
    cr ACK faf2c6e32e: patch looks correct

Tree-SHA512: df599e1c4cc0394f7f03413ad29954ddc87b163b02640d8bfc0497a5dc8d237b8c963c1dd9d01ac83c4a044f575300763097dac2ea6d1a4a163a1cece342b743
2020-12-15 21:49:46 +01:00
Pieter Wuille
37fe80e626 Only consider addrv2 peers for relay of non-addrv1 addresses 2020-12-15 12:45:41 -08:00
Pieter Wuille
83f8821a6f refactor: add IsAddrCompatible() to CNode 2020-12-15 12:45:32 -08:00
Wladimir J. van der Laan
d9a4738c9d Merge #20660: Move signet onion seed from v2 to v3
3e6657a14d Move signet onion seed from v2 to v3 (Sjors Provoost)

Pull request description:

  Since v0.21 hidden services use the longer v3 address format.

  It may make sense to backport this to the v0.21 branch, although onion nodes can always use the non-onion seeds.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 3e6657a14d
  laanwj:
    ACK 3e6657a14d
  Saibato:
    tACK 3e6657a14d  👍

Tree-SHA512: 9bb4d82345ab25d6ea971f8f106c30778ade2ba11a292e8d7449ea39581b91e51c4b34851b5c06f5dfbe07e7b04e5dc92c48c98c47a1c7ef3d5f350c3a3ad4f7
2020-12-15 21:38:33 +01:00
Dhruv Mehta
739d39022d ci: Move linter task to cirrus 2020-12-15 10:13:55 -08:00
MarcoFalke
8bb40d5f56 Merge #20560: fuzz: Link all targets once
fa13e1b0c5 build: Add option --enable-danger-fuzz-link-all (MarcoFalke)
44444ba759 fuzz: Link all targets once (MarcoFalke)

Pull request description:

  Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons:

  * It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times
  * It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files)
  * It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons
  * The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner
  * It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file
  * It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets

  Fixes #20088

ACKs for top commit:
  practicalswift:
    Tested ACK fa13e1b0c5
  sipa:
    ACK fa13e1b0c5. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all

Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
2020-12-15 19:00:36 +01:00
MarcoFalke
a35a3466ef Merge #20653: doc: Move addr relay comment in net to correct place
fa86217e97 doc: Move add relay comment in net to correct place (MarcoFalke)

Pull request description:

  The comment was previously attached to `m_addr_known`, but now it is attached to `id`, which is wrong.

  Fix that by moving the comment to `RelayAddrsWithConn`.

ACKs for top commit:
  practicalswift:
    cr ACK fa86217e97: patch looks correct
  jnewbery:
    ACK fa86217e97
  theStack:
    Code review ACK fa86217e97 🌳

Tree-SHA512: ec3d5f1996aded38947d2a5fd0bb63539e88f83964cd3254984002edfd51abb4dde813c7c81619a8a3a5c55b7e9ae83c8c5be8ad6c84b4593ed3bbf463fe8979
2020-12-15 17:56:11 +01:00
Wladimir J. van der Laan
ec6149c01e Merge #20616: Check CJDNS address is valid
f7264fff0a Check if Cjdns address is valid (Lucas Ontivero)

Pull request description:

  CJDNS addresses start with 0xFC and for that reason if a netaddr was unserialized with network type cjdns but its address prefix is not 0xFC then that netaddr should be considered invalid.

ACKs for top commit:
  jonatack:
    ACK f7264fff0a
  practicalswift:
    cr ACK f7264fff0a: patch looks correct
  theStack:
    ACK f7264fff0a ✔️

Tree-SHA512: 5300df2ffbbd69c40271b6d8df96cca98eb3e1ee76aba62c9c76025d083788ab1f1332775890c63b06e02ca593863a867cd53956bce5962383e8450487898669
2020-12-15 17:51:41 +01:00
Wladimir J. van der Laan
cd03513dc2 init: Signal-safe instant shutdown
Replace the 200ms polling loop with a faster and more efficient waiting
operation.

This was tried a few times before, but given up every time because
solutions use a condition variable which is not safe for use in signals
as they need to be reentrant.

On UNIX-ish OSes, use a safe way: a pipe. When shutdown is requested
write a dummy byte to the pipe. Waiting for shutdown is a matter of a
blocking read from the pipe.

On Windows, there are no signals so using a condition variable is safe.
2020-12-15 17:21:06 +01:00
MarcoFalke
70150824dc Merge #20437: fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime()
8c09c0c1d1 fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime() (practicalswift)

Pull request description:

  Avoid time-based "non-determinism" in fuzzing harnesses by using mocked `GetTime()`.

  Prior to this commit the fuzzing harnesses `banman`, `connman`, `net` and `rbf` had time-based "non-determinism". `addrman` is fixed in #20425. `process_message` and `process_messages` are left to fix: simply using mock time is not enough for them due to interaction with `IsInitialBlockDownload()`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    review ACK 8c09c0c1d1
  practicalswift:
    > review ACK [8c09c0c](8c09c0c1d1)

Tree-SHA512: 32dfbead3dfd18cf4ff56dc2ea341aa977441b4e19a54879cf54fa5820c7e2b14b92c7e238d32fd785654f3b28cc82826ae66c03e94c292633c63c41196ba9a8
2020-12-15 17:11:59 +01:00
Sjors Provoost
3e6657a14d Move signet onion seed from v2 to v3 2020-12-15 14:12:01 +01:00
fanquake
f1f2418433 Merge #20253: net: use std::chrono throughout maxOutbound logic
0475c8ba4d net: use std::chrono throughout maxOutbound logic (fanquake)
f805933e70 init: set nMaxOutboundLimit connection option directly (fanquake)
173d0d35f1 net: remove nMaxOutboundTimeframe from connection options (fanquake)
b117eb1486 net: remove SetMaxOutboundTimeframe (fanquake)
2f3f1aec1f net: remove SetMaxOutboundTarget (fanquake)

Pull request description:

  Switch to using `std::chrono` types for the max outbound related logic.
  Removes some unnecessary code from init.

ACKs for top commit:
  jnewbery:
    utACK 0475c8ba4d
  MarcoFalke:
    review ACK 0475c8ba4d 🎭

Tree-SHA512: 5a6d5b61e0d4c08a235cfc0257dae65d09a5df019d8d230b1a58a3e2483ddf4a31efdefc885c4a02e4715e4180b0ed92ebc0a1c08b2bf476a391945114593514
2020-12-15 20:48:21 +08:00
Jonas Schnelli
33d6337269 Merge bitcoin-core/gui#115: Replace "Hide tray icon" option with positive "Show tray icon" one
03edb52eee qt: Remove redundant BitcoinGUI::setTrayIconVisible (Hennadii Stepanov)
17174f8328 gui: Replace "Hide tray icon" option with positive "Show tray icon" one (Hennadii Stepanov)

Pull request description:

  This change makes easier both (1) using this option, and (2) reasoning about the code.

ACKs for top commit:
  jonasschnelli:
    utACK 03edb52eee

Tree-SHA512: 38e317492210d4fb13302dea383bd1f4f0ae1219d7ff2fdcb78607f15ac61a51969acaadb59b72c3f075b6356ef54368eb46fb49e6e1bd42db6d5804b97e232b
2020-12-15 09:28:50 +01:00
Wladimir J. van der Laan
94a9cd25fd Merge #20594: Fix getauxval calls in randomenv.cpp
836a3dc02c Avoid weak-linked getauxval support on non-linux platforms (like macOS) (Jonas Schnelli)
41a413b317 Define correct symbols for getauxval (Jonas Schnelli)

Pull request description:

  PR #20358 made use of the two preprocessor symbols `HAVE_STRONG_GETAUXVAL` as well as `HAVE_WEAK_GETAUXVAL`.

  These symbols have not been defined in configure.ac. They where only passed selective as CRC32 CPPFLAGS in https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include#L16.

  PR #20358 would have broken the macOS build since `getauxval` is not supported on macOS (but weak-linking does pass).

  This PR defines the two symbols correctly and reduces calls to `getauxval` to linux.

ACKs for top commit:
  laanwj:
    Code review ACK 836a3dc02c
  jonatack:
    utACK 836a3dc02c

Tree-SHA512: 6527f4a617b937f4c368a3cb1c162f1ac38a6f5e6341295554961eaf322906e9b27398a6f7b00819854ceebb5c828d3e6ce0a779edd769adc4053ce8beda3739
2020-12-14 21:03:40 +01:00
Lucas Ontivero
f7264fff0a Check if Cjdns address is valid 2020-12-14 14:48:52 -03:00
MarcoFalke
fa86217e97 doc: Move add relay comment in net to correct place
Can be reviewed with
--ignore-all-space --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2020-12-14 18:15:51 +01:00
MarcoFalke
fff7d056a9 Merge #20599: net processing: Tolerate sendheaders and sendcmpct messages before verack
b316dcb758 [net processing] Tolerate sendheaders and sendcmpct messages before verack (John Newbery)

Pull request description:

  BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
  which stage the `sendheaders` or `sendcmpct` messages should be sent.
  Therefore we should tolerate them being sent before the version-verack
  handshake is complete.

ACKs for top commit:
  MarcoFalke:
    review ACK b316dcb758 📒
  jonatack:
    Code review re-ACK b316dcb758 per `git range-diff b103fdcb 82d0a43 b316dcb`, rebase only
  luke-jr:
    utACK b316dcb758, only code movement from after verack check to before

Tree-SHA512: a8da3990a786a53c195a33cd278eb9d1b6b09e9a33717674d5bc6ef5629811189216f3eda1a3dd11f6972b9680c7b7a532bf7343b6e7440009dd831bef128e1d
2020-12-14 17:10:55 +01:00
MarcoFalke
fa13e1b0c5 build: Add option --enable-danger-fuzz-link-all 2020-12-14 16:55:56 +01:00
John Newbery
b316dcb758 [net processing] Tolerate sendheaders and sendcmpct messages before verack
BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
which stage the `sendheaders` or `sendcmpct` messages should be sent.
Therefore we should tolerate them being sent before the version-verack
handshake is complete.
2020-12-14 13:23:43 +00:00
John Newbery
ea36a453e3 [net] Make p2p recv buffer timeout 20 minutes for all peers
The timeout interval for the send and recv buffers was changed from 90
minutes to 20 minutes in commit f1920e86 in 2013, except for peers that
did not support the pong message (where the recv buffer timeout remained
at 90 minutes). A few observations:

- for peers that support BIP 31 (pong messages), this recv buffer
  timeout is almost redundant with the ping timeout. We send a ping
  message every two minutes, and set a timeout of twenty minutes to
  receive the pong response. If the recv buffer was really timing out,
  then the pong response would also time out.
- BIP 31 is supported by all nodes of p2p version 60000 and higher, and
  has been in widespread use since 2013. I'd be very surprised if there
  are many nodes on the network that don't support pong messages.
- The recv buffer timeout is not specified in any p2p BIP. We're free to
  set it at any value we want.
- A peer that doesn't support BIP 31 and hasn't sent any message to us
  at all in 90 minutes is unlikely to be useful for us, and is more likely
  to be evicted AttemptToEvictConnection() since it'll have the worst
  possible ping time and isn't providing blocks/transactions.

Therefore, we remove this check, and sent the recv buffer timeout to 20
minutes for all peers. This removes the final p2p version dependent
logic from the net layer, so all p2p version data can move into the
net_processing layer.

Alternative approaches:

- Set the recv buffer timeout to 90 minutes for all peers. This almost
  wouldn't be a behaviour change at all (pre-BIP 31 peers would still
  have the same recv buffer timeout, and we can't ever reach a recv buffer
  timeout higher than 21 minutes for post-BIP31 peers, because the pong
  timeout would be hit first).
- Stop supporting peers that don't support BIP 31. BIP 31 has been in
  use since 2012, and implementing it is trivial.
2020-12-14 12:59:30 +00:00
MarcoFalke
b103fdcb3b Merge #19763: net: don't try to relay to the address' originator
7fabe0f359 net: don't relay to the address' originator (Vasil Dimov)

Pull request description:

  For each address to be relayed we "randomly" pick 2 nodes to send the
  address to (in `RelayAddress()`). However we do not take into
  consideration that it does not make sense to relay the address back to
  its originator (`CNode::PushAddress()` will do nothing in that case).

  This means that if the originator is among the "randomly" picked nodes,
  then we will relay to one node less than intended.

  Fix this by skipping the originating node when choosing candidates to
  relay to.

ACKs for top commit:
  sdaftuar:
    ACK 7fabe0f359 (this time I looked at the test, and verified the test breaks in expected ways if I break the code).
  jnewbery:
    utACK 7fabe0f359 (only net_processing changes. I haven't reviewed the test changes)
  jonatack:
    re-ACK 7fabe0f359 per `git range-diff b76abae fd897f8 7fabe0f`, change since last review is rebase and more readable Doxygen documentation

Tree-SHA512: c6a9d11c7afc97ab4e8960513f6416648d4a8c0c64b713c145a7482a7b9e54946f81386a3351e3ec0011e5594ba5ccff4d10c6f656bb80680d9f0d0a63366165
2020-12-14 13:37:21 +01:00
Hennadii Stepanov
267f259c0d depends: Drop workaround for a fixed bug in Qt build system
The bug reports:
 - https://bugreports.qt.io/browse/QTBUG-35444
 - https://bugreports.qt.io/browse/QTBUG-32519
Fixed in Qt 5.3.0 (the workaround was introduced for Qt 5.2.1).
2020-12-14 14:07:49 +02:00
MarcoFalke
eec9366f7d Merge #20624: net processing: Remove nStartingHeight check from block relay
f6360088de [net processing] Clarify UpdatedBlockTip() (John Newbery)
94d2cc35be [net processing] Remove unnecesary nNewHeight variable in UpdatedBlockTip() (John Newbery)
8b57013473 [net processing] Remove nStartingHeight check from block relay (John Newbery)

Pull request description:

  nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
  0.2.9, P2P version 209) with the comment "better prevention of inventory
  relaying during initial download". At that time, there was no function
  to determine whether the node was still in Initial Block Download, so to
  prevent syncing nodes from relaying old blocks to their peers, a check
  was added to never relay a block to a peer where the height was lower
  than 2000 less than the peer's best block. That check was updated
  several times in later commits to ensure that we weren't relaying blocks
  before the latest checkpoint if the peer didn't provide a
  startingheight. The checkpoint comparison was changed to compare with an
  estimate of the highest block in commit eae82d8e.

  In commit 202e0194, all block relay was gated on being out of Initial
  Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
  removed since "we already checked IsIBD()".

  We can remove the check against nStartingHeight entirely. If the node is
  out of Initial Block Download, then its tip height must have been within
  24 hours of current time, so should not be more than ~144 blocks behind
  the most work tip.

  This simplifies moving block inventory state into the `Peer` object (#19829).

ACKs for top commit:
  Sjors:
    utACK f636008
  jonatack:
    ACK f6360088de
  MarcoFalke:
    ACK f6360088de 💽
  ariard:
    Code Review ACK f636008

Tree-SHA512: 4959cf35f1dcde46f34bffec1375729a157e1b2a1fd8a8ca33da9771c3c89a6c43e7050cdeeab8d90bb507b0795703db8c8bc304a1a5065ef00aae7a6992ca4f
2020-12-14 11:01:01 +01:00
MarcoFalke
096bd374da Merge #20592: doc: update wtxidrelay documentation per BIP339
4b7b58b3fe Update net_processing WTXID documentation per BIP339 (Jon Atack)

Pull request description:

  BIP339 currently states:

  *The wtxidrelay message MUST be sent in response to a version message from a peer whose protocol version is >= 70016 and prior to sending a verack. A wtxidrelay message received after a verack message MUST be ignored or treated as invalid.*

ACKs for top commit:
  MarcoFalke:
    ACK 4b7b58b3fe
  practicalswift:
    ACK 4b7b58b3fe
  RiccardoMasutti:
    ACK 4b7b58b

Tree-SHA512: 58ca6b197618cc73c70aa5de0a2d9d89a68b4cad9d5a708278ef17a9d6854d4362bcc384b6d29696642924977204a8fc120b31e91e2d97b6072b7b0d41c9f2dc
2020-12-14 10:52:28 +01:00
Anthony Towns
95d5d5e625 rpc: allow getblocktemplate for test chains when unconnected or in IBD 2020-12-14 18:32:57 +10:00
Anthony Towns
81c54dec20 rpc: update getblocktemplate with signet rule, include signet_challenge 2020-12-14 18:32:57 +10:00
fanquake
25bc840e03 Merge #20617: p2p: Remove m_is_manual_connection from CNodeState
a33442fdc7 Remove m_is_manual_connection from CNodeState (Antoine Riard)

Pull request description:

  Currently, this member is only used to exclude MANUAL peers from discouragement
  in MaybePunishNodeForBlock(). Manual connections are already protected in
  MaybeDiscourageAndDisconnect(), independently from their network
  processing behaviors.

ACKs for top commit:
  MarcoFalke:
    cr ACK a33442fdc7
  promag:
    Code review ACK a33442fdc7.
  jnewbery:
    utACK a33442fdc7
  amitiuttarwar:
    code review ACK a33442fdc7

Tree-SHA512: cfe3f3dfa131373e3299002d34ae9e22ca6e1a966831bab32fcf06ff1d08f06095b4ab020cc4d267f3ec05ae23fbdc22373382ab828b999c0db11b8c842a4f0c
2020-12-14 10:37:19 +08:00
Pieter Wuille
a4118c6e20 Add patch to make codesign_allocate compatible with Apple's 2020-12-13 12:20:39 -08:00
fanquake
0475c8ba4d net: use std::chrono throughout maxOutbound logic 2020-12-13 11:41:33 +08:00
fanquake
f805933e70 init: set nMaxOutboundLimit connection option directly
DEFAULT_MAX_UPLOAD_TARGET is a compile time constant.
2020-12-13 11:12:05 +08:00
fanquake
173d0d35f1 net: remove nMaxOutboundTimeframe from connection options
It's not actually possible to change this value, so remove the
indirection of it being a conn option.

DEFAULT_MAX_UPLOAD_TIMEFRAME is a compile time constant.
2020-12-13 11:10:40 +08:00
fanquake
b117eb1486 net: remove SetMaxOutboundTimeframe
This was introduced in 872fee3fcc and it's unclear
if it's ever been used.
2020-12-13 10:38:24 +08:00
fanquake
2f3f1aec1f net: remove SetMaxOutboundTarget
This has been unused since f3552da813.
2020-12-13 10:38:24 +08:00
fanquake
ade38b6ee8 Merge #20588: Remove unused and confusing CTransaction constructor
fac39c1983 wallet: document that tx in CreateTransaction is purely an out-param (MarcoFalke)
faac31521b Remove unused and confusing CTransaction constructor (MarcoFalke)

Pull request description:

  The constructor is confusing and dangerous (as explained in the TODO), fix that by removing it.

ACKs for top commit:
  laanwj:
    Code review ACK fac39c1983
  promag:
    Code review ACK fac39c1983.
  theStack:
    Code review ACK fac39c1983

Tree-SHA512: e0c8cffce8d8ee0166b8e1cbfe85ed0657611e26e2af0d69fde70eceaa5d75cbde3eb489af0428fe4fc431360b4c791fb1cc21b8dee7d4c7a4f17df00836229d
2020-12-13 10:36:22 +08:00
MarcoFalke
b18978066d Merge #20079: p2p: Treat handshake misbehavior like unknown message
faaad1bbac p2p: Ignore version msgs after initial version msg (MarcoFalke)
fad68afcff p2p: Ignore non-version msgs before version msg (MarcoFalke)

Pull request description:

  Handshake misbehaviour doesn't cost us more than any other unknown message, so it seems odd to treat it differently

ACKs for top commit:
  jnewbery:
    utACK faaad1bbac
  practicalswift:
    ACK faaad1bbac: patch looks correct

Tree-SHA512: 9f30c3b5c1f6604fd02cff878f10999956152419a3dd9825f8267cbdeff7d06787418b41c7fde8a00a5e557fe89204546e05d5689042dbf7b07fbb7eb95cddff
2020-12-12 12:32:46 +01:00
MarcoFalke
ffc4d04990 Merge #20275: wallet: List all wallets in non-SQLite and non-BDB builds
f3d870fc22 wallet: List all wallets in non-SQLite or non-BDB builds (Russell Yanofsky)
d70dc89e78 refactor: Consolidate redundant wallet database path and exists functions (Russell Yanofsky)
6a7a63644c refactor: Drop call to GetWalletEnv in wallet salvage code (Russell Yanofsky)
6ee9cbdd18 refactor: Replace ListWalletDir() function with ListDatabases() (Russell Yanofsky)
5aaeb6cf87 MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir (Russell Yanofsky)

Pull request description:

  This PR does not change behavior when bitcoin is built normally with both the SQLite and BDB libraries. It just makes non-SQLite and non-BDB builds more similar to the normal build. Specifically:

  - It makes wallet directory lists always include all wallets so wallets don't appear missing depending on the build.

  - It now triggers specific "Build does not support SQLite database format" and "Build does not support Berkeley DB database format" errors if a wallet can't be loaded instead of the more ambiguous and scary "Data is not in recognized format" error.

  Both changes are implemented in the last commit. The previous commits are just refactoring cleanups that make the last commit possible and consolidate and reduce code.

ACKs for top commit:
  achow101:
    ACK f3d870fc22
  promag:
    Tested ACK f3d870fc22. Tested a --without-sqlite build with sqlite wallets.

Tree-SHA512: 029ad21559dbc338b5f351d05113c51bc25bce830f4f4e18bcd82287bc528275347a60249da65b91d252632aeb70b25d057bd59c704bfcaafb9f790bc5b59762
2020-12-12 09:20:11 +01:00
pox
cc3044ccdb fix misleading comment about call to non-existing function 2020-12-12 07:01:38 +02:00
Carl Dong
22437fc72e build: Run libdmg-hfsplus's DMG tool in make deploy
Previously, the compression of the .iso file to a .dmg file was done
outside of `make deploy' in order to use the faketime-wrapped version of
libdmg-hfsplus's DMG tool.

Specifying the faketime-wrapped version of the DMG tool to ./configure
fixes this and simplifies build scripts.
2020-12-11 11:43:11 -05:00
John Newbery
f6360088de [net processing] Clarify UpdatedBlockTip() 2020-12-11 10:40:19 +00:00
John Newbery
94d2cc35be [net processing] Remove unnecesary nNewHeight variable in UpdatedBlockTip() 2020-12-11 10:35:32 +00:00
John Newbery
8b57013473 [net processing] Remove nStartingHeight check from block relay
nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
0.2.9, P2P version 209) with the comment "better prevention of inventory
relaying during initial download". At that time, there was no function
to determine whether the node was still in Initial Block Download, so to
prevent syncing nodes from relaying old blocks to their peers, a check
was added to never relay a block to a peer where the height was lower
than 2000 less than the peer's best block. That check was updated
several times in later commits to ensure that we weren't relaying blocks
before the latest checkpoint if the peer didn't provide a
startingheight. The checkpoint comparison was changed to compare with an
estimate of the highest block in commit eae82d8e.

In commit 202e0194, all block relay was gated on being out of Initial
Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
removed since "we already checked IsIBD()".

We can remove the check against nStartingHeight entirely. If the node is
out of Initial Block Download, then its tip height must have been within
24 hours of current time, so should not be more than ~144 blocks behind
the most work tip.
2020-12-11 10:08:37 +00:00
MarcoFalke
6a48063671 Merge #19858: Periodically make block-relay connections and sync headers
b3a515c0be Clarify comments around outbound peer eviction (Suhas Daftuar)
daffaf03fb Periodically make block-relay connections and sync headers (Suhas Daftuar)
3cc8a7a0f5 Use conn_type to identify block-relay peers, rather than m_tx_relay == nullptr (Suhas Daftuar)
91d61952a8 Simplify and clarify extra outbound peer counting (Suhas Daftuar)

Pull request description:

  To make eclipse attacks more difficult, regularly initiate outbound connections
  and stay connected long enough to sync headers and potentially learn of new
  blocks. If we learn a new block, rotate out an existing block-relay peer in
  favor of the new peer.

  This augments the existing outbound peer rotation that exists -- currently we
  make new full-relay connections when our tip is stale, which we disconnect
  after waiting a small time to see if we learn a new block.  As block-relay
  connections use minimal bandwidth, we can make these connections regularly and
  not just when our tip is stale.

  Like feeler connections, these connections are not aggressive; whenever our
  timer fires (once every 5 minutes on average), we'll try to initiate a new
  block-relay connection as described, but if we fail to connect we just wait for
  our timer to fire again before repeating with a new peer.

ACKs for top commit:
  ariard:
    Code Review ACK b3a515c, only change since last time is dropping a useless `cs_main` taking. I manually tested a previous version of the PR, and not substantial change has been introduced since then which would alter behavior IMO.
  jonatack:
    Tested ACK b3a515c0be over several weeks, though this change and behavior could benefit from test coverage and other follow-ups (refactoring, etc.) described in the review feedback. I did not verify the behavior of `m_start_extra_block_relay_peers` only being enabled after initial chain sync. Since my last review, one unneeded `cs_main` lock was removed.

Tree-SHA512: 75fc6f8e8003e88e93f86b845caf2d30b8b9c0dbb0a6b8aabe4e24ea4f6327351f736a068a3b2720a8a581b789942a3a47f921e2afdb47e88bc50d078aa37b6f
2020-12-11 10:20:01 +01:00
fanquake
736eb4d808 Merge #19982: test: Fix inconsistent lock order in wallet_tests/CreateWallet
e1e68b6305 test: Fix inconsistent lock order in wallet_tests/CreateWallet (Hennadii Stepanov)
cb23fe01c1 [skip ci] sync: Check precondition in LEAVE_CRITICAL_SECTION() macro (Hennadii Stepanov)
c5e3e74f70 sync: Improve CheckLastCritical() (Hennadii Stepanov)

Pull request description:

  This PR:
  - fixes #19049 that was caused by #16426
  - removes `wallet_tests::CreateWallet` suppression from the `test/sanitizer_suppressions/tsan`

  The example of the improved `CheckLastCritical()`/`LEAVE_CRITICAL_SECTION()` log (could be got when compiled without the last commit):
  ```
  2020-09-20T08:34:28.429485Z [test] INCONSISTENT LOCK ORDER DETECTED
  2020-09-20T08:34:28.429493Z [test] Current lock order (least recent first) is:
  2020-09-20T08:34:28.429501Z [test]  'walletInstance->cs_wallet' in wallet/wallet.cpp:4007 (in thread 'test')
  2020-09-20T08:34:28.429508Z [test]  'cs_wallets' in wallet/wallet.cpp:4089 (in thread 'test')
  ```

  Currently, there are other "naked" `LEAVE_CRITICAL_SECTION()` in the code base:
  b99a1633b2/src/rpc/mining.cpp (L698)

  b99a1633b2/src/checkqueue.h (L208)

ACKs for top commit:
  MarcoFalke:
    review ACK e1e68b6305 💂
  ryanofsky:
    Code review ACK e1e68b6305. Just trivial rebase and suggested switch to BOOST_CHECK_EXCEPTION since last review
  vasild:
    ACK e1e68b630

Tree-SHA512: a627680eac3af4b4c02772473d68322ce8d3811bf6b035d3485ccc97d35755bef933cffabd3f20b126f89e3301eccecec3f769df34415fb7c426c967b6ce36e6
2020-12-11 09:44:56 +08:00
Jarol Rodriguez
c175690561 doc: Update for FreeBSD 12.2, add GUI Build Instructions
The current FreeBSD Build documentation is a little outdated and underwhelming. This PR updates the doc to
be more informative. It also adds new instructions for building the GUI and adding support for  descriptor
wallets.

on vasild's recommendation: it is ok to point the user to download db5 which has an active port package
instead of building db4 from the provided script.

Co-authored-by: Vasil Dimov <vd@freebsd.org>
2020-12-10 15:38:20 -05:00
Hennadii Stepanov
e1e68b6305 test: Fix inconsistent lock order in wallet_tests/CreateWallet 2020-12-10 20:49:06 +02:00
Hennadii Stepanov
cb23fe01c1 [skip ci] sync: Check precondition in LEAVE_CRITICAL_SECTION() macro
This change reveals a bug in the wallet_tests/CreateWalletFromFile test,
that will be fixed in the following commit.
2020-12-10 20:46:39 +02:00
Hennadii Stepanov
c5e3e74f70 sync: Improve CheckLastCritical()
This commit adds actual lock stack logging if check fails.
2020-12-10 20:46:29 +02:00
Antoine Riard
a33442fdc7 Remove m_is_manual_connection from CNodeState
Currently, this member is only used to exclude MANUAL peers from discouragement
in MaybePunishNodeForBlock(). Manual connections are already protected in
MaybeDiscourageAndDisconnect(), independently from their network
processing behaviors.
2020-12-10 12:53:02 -05:00
MarcoFalke
da957cd62e Merge #20613: test: Use Popen.wait instead of RPC in assert_start_raises_init_error
fa918dd537 test: Use Popen.wait instead of RPC in assert_start_raises_init_error (MarcoFalke)

Pull request description:

  Using RPC (`wait_for_rpc_connection`) has several issue:

  * It polls in a loop, which might be slow
  * It tries to read the RPC cookie file, which might not be present, thus leading to intermittent issues

  Fix both by using `Popen.wait`

ACKs for top commit:
  laanwj:
    Code review ACK ~~faf7b05be9c86ee61c39e5314511fe2410128a6b~~ fa918dd537
  darosior:
    ACK fa918dd537

Tree-SHA512: 5368ad0d0ea2deb0af9582a42667c9290efe8f2705f37a236afc2c7908b04265ab342e2dd356a57156e99389f4a27ab6da9fa7bf9161fb7568240aa005e693b9
2020-12-10 18:38:18 +01:00
Michael Dietz
3b064fcb9d test: run mempool_expiry.py even with wallet disabled
Test coverage is also extended in this commit to:
Ensure that another transaction in the mempool is not evicted
when other transactions expire. Note: this other transaction does
not have any ancestors in the mempool that expired. Otherwise it
would be evicted from the mempool, and we already test this case.
2020-12-10 10:14:35 -06:00
MarcoFalke
faf2c6e32e cirrus: Schedule one task with paid credits for faster CI feedback 2020-12-10 15:04:54 +01:00
Suhas Daftuar
b3a515c0be Clarify comments around outbound peer eviction 2020-12-10 08:46:51 -05:00
Suhas Daftuar
daffaf03fb Periodically make block-relay connections and sync headers
To make eclipse attacks more difficult, regularly initiate outbound connections
and stay connected long enough to sync headers and potentially learn of new
blocks. If we learn a new block, rotate out an existing block-relay peer in
favor of the new peer.

This augments the existing outbound peer rotation that exists -- currently we
make new full-relay connections when our tip is stale, which we disconnect
after waiting a small time to see if we learn a new block.  As block-relay
connections use minimal bandwidth, we can make these connections regularly and
not just when our tip is stale.

Like feeler connections, these connections are not aggressive; whenever our
timer fires (once every 5 minutes on average), we'll try to initiate a new
block-relay connection as described, but if we fail to connect we just wait for
our timer to fire again before repeating with a new peer.
2020-12-10 08:46:39 -05:00
Wladimir J. van der Laan
b76abae387 Merge #20587: [doc] Tidy up Tor doc (more stringent)
32045bbfd5 [doc] Tidy up Tor doc (more stringent) (wodry)

Pull request description:

  This is a follow up to https://github.com/bitcoin/bitcoin/pull/19638 that left two deprecated "hidden service/server" naming occurences.

  It also shall make the chapter titles regarding creation of onion services stringent and easy to read and distinguish.

  It removes the one and only reference to the testnet (here the testnet onion service port), as it is not explained that it references to the testnet and I do not know why it is mentioned there. It is only confusing. Also, as said, the testnet is not referenced at any other place in this document.

ACKs for top commit:
  practicalswift:
    ACK 32045bbfd5
  laanwj:
    Review ACK 32045bbfd5
  RiccardoMasutti:
    ACK 32045bb

Tree-SHA512: c623387b76d68845c0fa47f67a6f8ef70c9c560e3f8f8754e45a4f51e43198c2092be789588acd4ada607f42fbe62d51a4b1888d81b225df19b6557a081835c0
2020-12-10 14:44:41 +01:00
Suhas Daftuar
3cc8a7a0f5 Use conn_type to identify block-relay peers, rather than m_tx_relay == nullptr 2020-12-10 08:41:57 -05:00
Suhas Daftuar
91d61952a8 Simplify and clarify extra outbound peer counting 2020-12-10 08:41:57 -05:00
Vasil Dimov
7fabe0f359 net: don't relay to the address' originator
For each address to be relayed we "randomly" pick 2 nodes to send the
address to (in `RelayAddress()`). However we do not take into
consideration that it does not make sense to relay the address back to
its originator (`CNode::PushAddress()` will do nothing in that case).

This means that if the originator is among the "randomly" picked nodes,
then we will relay to one node less than intended.

Fix this by skipping the originating node when choosing candidates to
relay to.
2020-12-10 14:41:41 +01:00
Jonas Schnelli
86f2007193 Merge #20527: build: Do not ignore Homebrew's SQLite on macOS
c932e0d67e doc: Update wallet database installation guide for macOS (Hennadii Stepanov)
ee7b84e63c build: Use Homebrew's sqlite package if it is available (Hennadii Stepanov)
c96d1f65a5 build, refactor: Check that Homebrew's qt5 package is actually installed (Hennadii Stepanov)

Pull request description:

  On master (7ae86b3c68) installed Homebrew `sqlite` package is ignored during build on macOS.

  This PR fixes this issue and update macOS build docs.

  Closes #20498.

ACKs for top commit:
  willcl-ark:
    > > That said, another tACK of [c932e0d](c932e0d67e)
  hebasto:
    > That said, another tACK of [c932e0d](c932e0d67e)
  laanwj:
    Code review ACK c932e0d67e
  jonasschnelli:
    code review re-ACK c932e0d67e

Tree-SHA512: 2563f25534d065556b17ee8c0fca957aea61b5ae288a2aa72743e77607843a45c39f209321e0f05b34283a74d2edcf961cf1dc54a35ed0cc21182304bb961505
2020-12-10 13:18:30 +01:00
Wladimir J. van der Laan
8e1f9d6273 Merge #20549: Support make src/bitcoin-node and src/bitcoin-gui
206f74e88c Support make src/bitcoin-node and src/bitcoin-gui (João Barbosa)

Pull request description:

  This change adds the following configure output variables
  ```
  dnl Multi Process
  BITCOIN_MP_NODE_NAME=bitcoin-node
  BITCOIN_MP_GUI_NAME=bitcoin-gui
  ```
  and adds support for
  ```sh
  make src/bitcoin-node src/bitcoin-gui
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 206f74e88c

Tree-SHA512: 4d1a694b9010ecc267ee955f4475127a58e6da72f30179ec740285ee6fe03cd91dcb6847317a47460dbd548edb88b7da6c7a98eac10f0dabe3ce4e83e0aa8093
2020-12-10 12:53:54 +01:00
MarcoFalke
fa918dd537 test: Use Popen.wait instead of RPC in assert_start_raises_init_error 2020-12-10 12:36:41 +01:00
Wladimir J. van der Laan
e20b488395 Merge #20567: test: Add option to git-subtree-check to do full check, add help
34c80d9eee test: Add option to git-subtree-check to do full check, add help (Wladimir J. van der Laan)

Pull request description:

  This adds a brief help text to `git-subtree-check.sh` and adds an option to do a full remote check instead of having two different code paths with a successful exit status. Also make it explicit that the CI is not doing this.

ACKs for top commit:
  fjahr:
    tested ACK 34c80d9eee

Tree-SHA512: 20f672fd3b3c1d633eccf9998fdd738194cdd7d10cc206691f2dcc28bbbf8187b8d06b87814f875a06145b179f5ca1f4f4f9922972be72759cf5ac6e0c11abd1
2020-12-10 12:04:28 +01:00
Wladimir J. van der Laan
dcff2ee1fb Merge #20589: log: Clarify that failure to read/write fee_estimates.dat is non-fatal
fa0d8359b3 log: Clarify that failure to read fee_estimates.dat is non-fatal (MarcoFalke)
faefa5db5f log: Clarify that failure to write fee_estimates.dat is non-fatal (MarcoFalke)

Pull request description:

  two minor logging fixups

ACKs for top commit:
  practicalswift:
    ACK fa0d8359b3: patch looks correct
  laanwj:
    Code review ACK fa0d8359b3

Tree-SHA512: d1e7e595d3b4a5e497ee7ab70f3be5783dafec2726ef8e012db836c15e8e622022859a4472d6b516fe19d327737b25fdfb509cd9aeb022ca847b13c54e55800a
2020-12-10 12:00:18 +01:00
Wladimir J. van der Laan
eb53c03b36 Merge #20595: Improve heuristic hex transaction decoding
0f949cde3d Add regression test for incorrect decoding (Pieter Wuille)
39c42c4420 Improve heuristic hex transaction decoding (Pieter Wuille)

Pull request description:

  The current hex tx decoding logic will refuse to decode valid extended-encoded transactions if the result fails the heuristic sanity check, even when the legacy-encoding fails. Fix this.

  Fixes #20579

ACKs for top commit:
  achow101:
    Code review ACK 0f949cde3d
  jonatack:
    Tested ACK 0f949cde3d
  laanwj:
    Code review ACK 0f949cde3d

Tree-SHA512: bd6dc80d824eb9a87026a623be910cac92173f8ce1c8b040c2246348c3cf0c6d64bcc40127b859e5e4da1efe88cf02a6945f7ebb91079799395145cb09d9c7a5
2020-12-10 11:22:10 +01:00
MarcoFalke
fade6195b1 Move TX_MAX_STANDARD_VERSION to policy
Also remove extraneous whitespace, should be reviewed with --ignore-all-space
2020-12-10 11:12:08 +01:00
MarcoFalke
c8fdbb46bd Merge #20609: configure: output notice that test binary is disabled by fuzzing
904d875cf5 configure: output notice that test binary is disabled by fuzzing (Andrew Poelstra)

Pull request description:

  I wasted a bit of time today running a stale `test_bitcoin` and not understanding why, until I remembered that I'd ./configured my working directory with --enable-fuzz.

Top commit has no ACKs.

Tree-SHA512: 6cbe30547332114ad3fe61c67e224f5a28aac4b1b58e0acecb29cb04f5a34f792c927797aa8000449aae076435bd45acf209b7323b0b48fa971705d6ed3e6529
2020-12-10 09:11:04 +01:00
MarcoFalke
38176dc665 Merge #20573: wallet, bugfix: allow send with string fee_rate amounts
6fa72ceb80 test: add coverage for passing fee rate as a string (Jon Atack)
ce207d6b93 wallet, bugfix: allow send to take string fee rate values (Jon Atack)

Pull request description:

  RPC send currently only accepts fee rates as numbers, which is a user-facing bug. It should accept fee rates as an amount, e.g. a string or a number, as documented in its help and like sendtoaddress, sendmany, fundrawtransaction, walletcreatefundedpsbt, and bumpfee. Provide a fix and regression test coverage.

ACKs for top commit:
  MarcoFalke:
    review ACK 6fa72ceb80
  achow101:
    Code review ACK 6fa72ceb80
  promag:
    Code review ACK 6fa72ceb80.

Tree-SHA512: 735f9269cb1b81953764b5283449c0b154bd62de034225be5bcedc515c84faf767fe8fe0741008679fe412922c847b00d116cb11aab775236b779c847ba87167
2020-12-10 08:46:01 +01:00
MarcoFalke
22f13c1e08 Merge #19776: net, rpc: expose high bandwidth mode state via getpeerinfo
343dc4760f test: add test for high-bandwidth mode states in getpeerinfo (Sebastian Falbesoner)
dab6583307 doc: release note for new getpeerinfo fields "bip152_hb_{from,to}" (Sebastian Falbesoner)
a7ed00f8bb rpc: expose high-bandwidth mode states via getpeerinfo (Sebastian Falbesoner)
30bc8fab68 net: save high-bandwidth mode states in CNodeStats (Sebastian Falbesoner)

Pull request description:

  Fixes #19676, "_For every peer expose through getpeerinfo RPC whether or not we selected them as HB peers, and whether or not they selected us as HB peers._" See [BIP152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki), in particular the [protocol flow diagram](https://github.com/bitcoin/bips/raw/master/bip-0152/protocol-flow.png).  The newly introduced states are changed on the following places in the code:
  * on reception of a `SENDCMPCT` message with valid version, the field `m_highbandwidth_from` is changed depending on the first integer parameter in the message (1=high bandwidth, 0=low bandwidth), i.e. it just mirrors the field `CNodeState.fPreferHeaderAndIDs`.
  * after adding a `SENDCMPCT` message to the send queue, the field `m_highbandwidth_to` is changed depending on how the first integer parameter is set (same as above)

  Note that after receiving `VERACK`, the node also sends `SENDCMPCT`, but that is only to announce the preferred version and never selects high-bandwidth mode, hence there is no need to change the state variables there, which are initialized to `false` anyways.

ACKs for top commit:
  naumenkogs:
    reACK 343dc4760f
  jonatack:
    re-ACK 343dc4760f per `git range-diff 7ea6499 4df1d12 343dc47`

Tree-SHA512: f4999e6a935266812c2259a9b5dc459710037d3c9e938006d282557cc225e56128f72965faffb207fc60c6531fab1206db976dd8729a69e8ca29d4835317b99f
2020-12-10 08:21:36 +01:00
MarcoFalke
f5b2ea3e59 Merge #20217: net: Remove g_relay_txes
34e33ab859 Remove g_relay_txes (John Newbery)
68334b3944 [net processing] Add m_ignores_incoming_txs to PeerManager and use internally (John Newbery)
4d510aa055 [init] Use MakeUnique<> to construct peerman (John Newbery)
f3f61d0eb9 [net processing] Add IgnoresIncomingTxs() function to PeerManager (John Newbery)
5805b8299f [net processing] Move PushNodeVersion into PeerManager (John Newbery)

Pull request description:

  `g_relay_txes` is only required inside net_processing and is set only once at startup. Instead of having a global, move it to be a const member of PeerManager.

  This requires moving `PushNodeVersion()` into `PeerManager`, which also allows us to remove the `connman` argument.

ACKs for top commit:
  narula:
    utACK 34e33ab859
  MarcoFalke:
    re-ACK 34e33ab85 💐

Tree-SHA512: 33f75b522e5f34b243731932eb96cd6c8ce9db69b5186395e3718858bc715cec1711a663c6afc5880462812cbc15040930e2dc648b2acad6bc6502ad1397c5e3
2020-12-10 07:29:12 +01:00
MarcoFalke
44444ba759 fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
fanquake
054710615c Merge #20608: contrib: add symbol check test for PE binaries
ae9b48995b contrib: add symbol check test for PE (fanquake)

Pull request description:

  Follow up to #20476. Adds a test for the PE symbol check. One failing case where we link against `-lpdh` and a pass case.

ACKs for top commit:
  laanwj:
    Code review ACK ae9b48995b
  dongcarl:
    Code Review ACK ae9b48995b

Tree-SHA512: 14109d2c7cb98fb445fe1a7f3078e1e88c49fd29583529c53c75bb625f3060d43df0c64542df72272cff81e1b073f74ce6e437ad0e6617ba2bcccacfd1dc8e53
2020-12-10 11:33:36 +08:00
fanquake
17918a987a Merge #20046: depends: Set CMAKE_INSTALL_RPATH for native packages
7d0271b5c3 depends: Set CMAKE_INSTALL_RPATH for native packages (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  After #19685 started setting `LDFLAGS`, the `INSTALL_RPATH_USE_LINK_PATH` cmake option used in the libmultiprocess build no longer works, so it is neccessary to set `CMAKE_INSTALL_RPATH` as a fallback.

  It's unclear currently whether the bad interaction between `INSTALL_RPATH_USE_LINK_PATH` and `LDFLAGS` is a bug, but the issue is reported:

  - https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877
  - https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892

  Fixes #19981

ACKs for top commit:
  fanquake:
    ACK 7d0271b5c3 - I haven't looked in depth, but I've re-read through #19981 and checked the failure by testing #19160 (with this reverted):
  dongcarl:
    ACK 7d0271b Looked into this a bit, it makes sense that for the things we build in depends, we want the library search to start in depends. It seems reasonable to expect this to happen automatically when `CMAKE_INSTALL_PREFIX` and `INSTALL_RPATH_USE_LINK_PATH` are set, but oh well...

Tree-SHA512: 97cc5801c3204c14cd33004423631456ca0701e2127ee5146810a76e2f4aac9de1f4b5437402a4329cda54e022dc99270fee7e38c2995765f36b3848215fa78e
2020-12-10 11:11:50 +08:00
Andrew Chow
5d4597666d Rewrite OutputGroups to be clearer and to use scriptPubKeys
Rewrite OutputGroups so that the logic is easier to follow and
understand.

There is a slight behavior change as OutputGroups will be grouped by
scriptPubKey rather than CTxDestination as before. This should have no
effect on users as all addresses are a CTxDestination. However by using
scriptPubKeys, we can correctly group outputs which fall into the
NoDestination case. But we also shouldn't have any NoDestination
outputs.
2020-12-09 20:18:05 -05:00
Andrew Poelstra
904d875cf5 configure: output notice that test binary is disabled by fuzzing 2020-12-09 23:46:49 +00:00
Luke Dashjr
8775691383 Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes"
The SendConfirmationDialog is used for bumping the fee, where "Send" doesn't really make sense
2020-12-09 22:53:03 +00:00
Wladimir J. van der Laan
0038ce9758 Merge #20603: build: Update crc32c subtree
90c0f267bd Squashed 'src/crc32c/' changes from 224988680f..b5ef9be675 (MarcoFalke)

Pull request description:

  Except for the ARM64 darwin fix this is just code-shuffling in files/functions we don't use

ACKs for top commit:
  jonasschnelli:
    Tested ACK fa7c8d136f - Tested this on an ARM Mac. Linking issue went away (successful depends compilation). Also tested that the ARM64 hardware acceleration code part was used.
  laanwj:
    Code review ACK fa7c8d136f

Tree-SHA512: 1fa156d72c75d22ead2677b165e566978331f795d52a637e478d83d1cf2adddd84eed259d617df6d11270af2e4e57ae6991aec3bc4c0bdf5dec959f44daa14eb
2020-12-09 21:37:23 +01:00
John Newbery
34e33ab859 Remove g_relay_txes
Also remove vestigial commend in init.cpp
2020-12-09 18:13:37 +00:00
John Newbery
68334b3944 [net processing] Add m_ignores_incoming_txs to PeerManager and use internally 2020-12-09 18:13:37 +00:00
John Newbery
4d510aa055 [init] Use MakeUnique<> to construct peerman 2020-12-09 18:10:38 +00:00
John Newbery
f3f61d0eb9 [net processing] Add IgnoresIncomingTxs() function to PeerManager 2020-12-09 18:10:38 +00:00
John Newbery
5805b8299f [net processing] Move PushNodeVersion into PeerManager 2020-12-09 18:09:42 +00:00
MarcoFalke
7212db4d2a Merge #20602: util: Allow use of C++14 chrono literals
fa11110bff util: Allow use of C++14 chrono literals (MarcoFalke)

Pull request description:

  I think we should allow the use of chrono literals for new code to make it less verbose. Obviously old code can stay as-is.

  This patch pulls in the needed namespace and replaces some lines for illustrative purposes.

ACKs for top commit:
  vasild:
    ACK fa11110bff
  jonatack:
    ACK fa11110bff

Tree-SHA512: ee2b72c8f28dee07b33b9a8ee8f7c87c0bc43b05c56a17b786cf9803ef204c7628e01b02de1af1a4eb01f5cdf6fc336f69c2833e17acd606ebda20ac6917e6bb
2020-12-09 17:15:11 +01:00
Wladimir J. van der Laan
42ed7f51fa Merge #20606: Remove unused bits from service flags enum
fa40168ab3 Remove unused bits from service flags enum (MarcoFalke)

Pull request description:

  Remove service bits that haven't been observed on the active network for years and won't ever be observed on the network with this meaning. Keeping this dead assignment in our source code forever doesn't add any value.

  I somehow forgot to do this in commit fa0d0ff6e1.

ACKs for top commit:
  laanwj:
    Code review ACK fa40168ab3
  practicalswift:
    cr ACK fa40168ab3
  fanquake:
    ACK fa40168ab3

Tree-SHA512: 376e5ac05940493cf2209fea60515c843e978c4b476f2524f6bf7a37a646d237c3ddcf6c0fa23641f9ba550f625609703d9b51b4be631a7f2a90e1092b557232
2020-12-09 16:37:56 +01:00
fanquake
ae9b48995b contrib: add symbol check test for PE 2020-12-09 23:13:24 +08:00
fanquake
795afe6e63 Merge #19910: net processing: Move peer_map to PeerManager
3025ca9e77 [net processing] Add RemovePeer() (John Newbery)
a20ab22786 [net processing] Make GetPeerRef const (John Newbery)
ed7e469cee [net_processing] Move peer_map to PeerManager (John Newbery)
a529fd3e3f [net processing] Move GetNodeStateStats into PeerManager (John Newbery)

Pull request description:

  This moves `g_peer_map` from a global in net_processing.cpp's unnamed namespace to being a member `m_peer_map` of `PeerManager`.

ACKs for top commit:
  theuni:
    Re-ACK 3025ca9e77.
  dongcarl:
    Re-ACK 3025ca9
  hebasto:
    re-ACK 3025ca9e77, since my [previous](https://github.com/bitcoin/bitcoin/pull/19910#pullrequestreview-545574237) review only reverted the change that introduced NRVO in `PeerManager::GetPeerRef`, and comments are fixed in the proper commits.

Tree-SHA512: 6369eb3c688ac5b84f89f7674115f78ff02edbed76063ac2ebb1759894c9e973883e10821a35dab92bd3d738280acc095bd5368f552a060b83cd309330387d47
2020-12-09 21:56:17 +08:00
MarcoFalke
a3586d5920 Merge #20323: tests: Create or use existing properly initialized NodeContexts
81137c60fe test: Add new ChainTestingSetup and use it (Carl Dong)
7e9e7fe567 qt/test: [FIX] Add forgotten Context setting in RPCNestedTests (Carl Dong)

Pull request description:

  This is part 1/n of the effort to [de-globalize `ChainstateManager`](https://github.com/bitcoin/bitcoin/pull/20158)

  Reviewers: Looking for tested/Code-Review/plain-ACKs

  ### Context

  In many of our tests, we manually instantiate `NodeContext`s or `ChainstateManager`s in the test code, which is error prone. Instead, we should create or use existing references because:
  1. Before we [de-globalize `ChainstateManager`](https://github.com/bitcoin/bitcoin/pull/20158), much of our code still acts on `g_chainman` (our global `ChainstateManager`), sometimes even when you're calling a method on a specific instance of `ChainstateManager`! This means that we may act on two instances of `ChainstateManager`, which is most likely not what we want.
  2. Using existing references (initialized by the `{Basic,}TestingSetup` constructors) means that you're acting on objects which are properly initialized, instead of "just initialized enough for this dang test to pass". Also, they're already there! It's free!
  3. By acting on the right object, we also allow the review-only assertions in future commits of [de-globalize `ChainstateManager`](https://github.com/bitcoin/bitcoin/pull/20158) to work and demonstrate correctness.

  Some more detailed debugging notes can be found in the first commit, reproduced below:
  ```
  Previously, the validation_chainstatemanager_tests test suite
  instantiated its own duplicate ChainstateManager on which tests were
  performed.

  This wasn't a problem for the specific actions performed in
  that suite. However, the existence of this duplicate ChainstateManager
  and the fact that many of our validation static functions reach for
  g_chainman, ::Chain(state|)Active means we may end up acting on two
  different CChainStates should we write more extensive tests in the
  future.

  This change adds a new ChainTestingSetup which performs all
  initialization previously done by TestingSetup except:

  1. Mempool sanity check frequency setting
  2. ChainState initialization
  3. Genesis Activation
  4. {Ban,Conn,Peer}Man initialization

  Means that we will no longer need to initialize a duplicate
  ChainstateManger in order to test the initialization codepaths of
  CChainState and ChainstateManager.

  Lastly, this change has the additional benefit of allowing for
  review-only assertions meant to show correctness to work in future work
  de-globalizing g_chainman.

  In the test chainstatemanager_rebalance_caches, an additional
  LoadGenesisBlock call is added as MaybeReblanaceCaches eventually calls
  FlushBlockFile, which tries to access vinfoBlockFile[nLastBlockFile],
  which is out of bounds when LoadGenesisBlock hasn't been called yet.

  -----

  Note for the future:

  In a previous version of this change, I put ChainTestingSetup between
  BasicTestingSetup and TestingSetup such that TestingSetup inherited from
  ChainTestingSetup.

  This was suboptimal, and showed how the class con/destructor inheritance
  structure we have for these TestingSetup classes is probably not the
  most suitable abstraction. In particular, for both TestingSetup and
  ChainTestingSetup, we need to stop the scheduler first before anything
  else. Otherwise classes depending on the scheduler may be referenced
  by the scheduler after said classes are freed. This means that there's
  no clear parallel between our teardown code and C++'s destructuring
  order for class hierarchies.

  Future work should strive to coalesce (as much as possible) test and
  non-test init codepaths and perhaps structure it in a more fail-proof
  way.
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 81137c60fe looking excellent now 🐩
  jnewbery:
    ACK 81137c60fe
  ryanofsky:
    Code review ACK 81137c60fe. This change is simpler after the rebase because wallet & bench commits are dropped.

Tree-SHA512: a8d84f08f2db6428b0b88449bdc814c9db35b7559156d536dfebd3225c2707dba65959e76d2152e3f8c96eacbf1e0b0000f745edf1e196deddb97ff1ef360953
2020-12-09 10:22:10 +01:00
MarcoFalke
90ef622ab5 Merge #20564: Don't send 'sendaddrv2' to pre-70016 software, and send before 'verack'
1583498fb6 Send and require SENDADDRV2 before VERACK (Pieter Wuille)
c5a8919660 Don't send 'sendaddrv2' to pre-70016 software (Pieter Wuille)

Pull request description:

  BIP155 defines addrv2 and sendaddrv2 for all protocol versions, but some implementations reject messages they don't know. As a courtesy, don't send it to nodes with a version before 70016, as no software is known to support BIP155 that doesn't announce at least that protocol version number.

  Also move the sending of sendaddrv2 earlier (before sending verack), as proposed in https://github.com/bitcoin/bips/pull/1043. This has the side effect that local address broadcast of torv3 will work (as it'll only trigger after we know whether or not the peer supports addrv2).

ACKs for top commit:
  MarcoFalke:
    ACK 1583498fb6
  jnewbery:
    ACK 1583498fb6
  jonatack:
    ACK 1583498fb6
  vasild:
    ACK 1583498

Tree-SHA512: 3bd5833fa8c8567b6dedd99e4a9b6bb71c127aa66d5284b217503c86d597dc59aa7382c41f3a4bf561bb658b89db81d1a7703a700eef4ffc17cb916660e23a82
2020-12-09 07:01:57 +01:00
Pieter Wuille
0f949cde3d Add regression test for incorrect decoding 2020-12-08 13:13:40 -08:00
Carl Dong
81137c60fe test: Add new ChainTestingSetup and use it
Previously, the validation_chainstatemanager_tests test suite
instantiated its own duplicate ChainstateManager on which tests were
performed.

This wasn't a problem for the specific actions performed in
that suite. However, the existence of this duplicate ChainstateManager
and the fact that many of our validation static functions reach for
g_chainman, ::Chain(state|)Active means we may end up acting on two
different CChainStates should we write more extensive tests in the
future.

This change adds a new ChainTestingSetup which performs all
initialization previously done by TestingSetup except:

1. RPC command registration
2. ChainState initialization
3. Genesis Activation
4. {Ban,Conn,Peer}Man initialization

Means that we will no longer need to initialize a duplicate
ChainstateManger in order to test the initialization codepaths of
CChainState and ChainstateManager.

Lastly, this change has the additional benefit of allowing for
review-only assertions meant to show correctness to work in future work
de-globalizing g_chainman.

In the test chainstatemanager_rebalance_caches, an additional
LoadGenesisBlock call is added as MaybeReblanaceCaches eventually calls
FlushBlockFile, which tries to access vinfoBlockFile[nLastBlockFile],
which is out of bounds when LoadGenesisBlock hasn't been called yet.

-----

Note for the future:

The class con/destructor inheritance structure we have for these
TestingSetup classes is probably not the most suitable abstraction. In
particular, for both TestingSetup and ChainTestingSetup, we need to stop
the scheduler first before anything else. Otherwise classes depending on
the scheduler may be referenced by the scheduler after said classes are
freed. This means that there's no clear parallel between our teardown
code and C++'s destructuring order for class hierarchies.

Future work should strive to coalesce (as much as possible) test and
non-test init codepaths and perhaps structure it in a more fail-proof
way.
2020-12-08 15:00:25 -05:00
Carl Dong
7e9e7fe567 qt/test: [FIX] Add forgotten Context setting in RPCNestedTests 2020-12-08 14:22:43 -05:00
MarcoFalke
fa7c8d136f build: Update crc32c subtree 2020-12-08 19:26:30 +01:00
Pieter Wuille
39c42c4420 Improve heuristic hex transaction decoding
Whenever both encodings are permitted, try both, and if only one succeeds,
return that one. Otherwise prefer the one for which the heuristic sanity
check passes. If that is the case for neither or for both, return the
extended-permitting deserialization.
2020-12-08 10:11:02 -08:00
MarcoFalke
e98d1d6740 Merge #19425: refactor: Get rid of more redundant chain methods
5baa88fd38 test: Remove no longer needed MakeChain calls (Russell Yanofsky)
6965f1352d refactor: Replace uses ChainActive() in interfaces/chain.cpp (Russell Yanofsky)
3fbbb9a640 refactor: Get rid of more redundant chain methods (Russell Yanofsky)

Pull request description:

  This just drops three interfaces::Chain methods replacing them with other calls.

  Motivation for removing these chain methods:

  - Need to get rid of findFirstBlockWithTimeAndHeight for #10102, which doesn't support overloaded methods
  - Followup from https://github.com/bitcoin/bitcoin/pull/16426#discussion_r412487403
  - phantomcircuit comments about findNextBlock test http://www.erisian.com.au/bitcoin-core-dev/log-2020-06-06.html#l-214

  Behavior is not changing in any way here. A TODO comment in ScanForWalletTransactions was removed, but just because it was invalid (see https://github.com/bitcoin/bitcoin/pull/19195#discussion_r448020762), not because it was implemented.

ACKs for top commit:
  MarcoFalke:
    re-ACK 5baa88fd38 🕶
  dongcarl:
    ACK 5baa88fd38

Tree-SHA512: d20a2a8cf8742e6100c6d3a4871ed67f184925712cf9e55d301abee60353bf3f74cd0d46a13be1715d1c2faef72102bb321654d08a128c2ef6880f72b72a6687
2020-12-08 18:49:02 +01:00
Pieter Wuille
1583498fb6 Send and require SENDADDRV2 before VERACK
See the corresponding BIP change: https://github.com/bitcoin/bips/pull/1043
2020-12-08 09:40:10 -08:00
MarcoFalke
fa40168ab3 Remove unused bits from service flags enum 2020-12-08 18:36:51 +01:00
MarcoFalke
90c0f267bd Squashed 'src/crc32c/' changes from 224988680f..b5ef9be675
b5ef9be675 Merge #1: Merge changes from upstream
9e7f512430 Merge remote-tracking branch 'origin/master' into bitcoin-fork
1f85030246 Add support for ARM64 darwin (#43)
3bb959c982 Remove unnecessary reinterpret_cast (#42)
2e97ab26b1 Fix (unused) ReadUint64LE for BE machines (#41)
47b40d2209 Bump dependencies. (#40)
ba74185625 Move CI to Visual Studio 2019.
efa301a7e5 Allow different C/C++ standards when this is used as a subproject.
cc6d71465e CMake: Use configure_package_config_file()

git-subtree-dir: src/crc32c
git-subtree-split: b5ef9be6755a2e61e2988bb238f13d1c0ee1fa0a
2020-12-08 17:08:07 +01:00
MarcoFalke
fa11110bff util: Allow use of C++14 chrono literals 2020-12-08 16:47:36 +01:00
fanquake
16b31cc4c5 Merge #20422: build: mac deployment unification
b685f60a08 build: mac_alias 2.1.1 (fanquake)
5d2cbdf772 macdeploy: use Python 3.6 (fanquake)
a42aa94c54 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run (fanquake)
adaa26202b macdeploy: remove existing Bitcoin-Core.dmg if present (fanquake)
ccb0325b1b macdeploy: move qt_conf to where it's used (fanquake)
6390a04862 macdeploy: consolidate .DS_Store generation (fanquake)
32347cd56a macdeploy: assume plistlib is available (fanquake)
0ab4018c12 macdeploy: have a single level of logging output (fanquake)
827d382aa7 macdeploy: remove add-resources argument (fanquake)
464b34d4c3 macdeploy: remove codesigning argument (fanquake)
4d70d3d7fe build: automatically determine macOS translations (fanquake)

Pull request description:

  This consolidates our macOS build code so that `.DS_Store` generation is the same when running `make deploy` for macOS when building on Linux and macOS, rather than maintaining two version of code that essentially do the same thing (just slightly differently).

  It also removes unused code and any AppleScript usage, automates finding translation files and generally simplifies `macdeployqtplus`. It also gets rid of the annoying "popping up" behaviour during DMG generation, names the created image `Bitcoin-Core.dmg` rather than `Bitcoin-Qt.dmg`.

ACKs for top commit:
  dergoegge:
    ACK b685f60a08 - Less and cleaner code looks good. I tested this with `make deploy` and everything still works + the popup during DMG generation is gone.

Tree-SHA512: dcd38344e2dfcfa7ffbccf6226a71425c4d16b421a4881d5ee37b8e7ef393b3e8077262444c39b11912269d8cf688aba897e6518cba8361eb24a03fdd03b8caf
2020-12-08 16:51:49 +08:00
Russell Yanofsky
5baa88fd38 test: Remove no longer needed MakeChain calls
These calls are no longer needed after edc316020e
from #19098 which started instantiating BasicTestingSetup.m_node.chain

Patch from MarcoFalke <falke.marco@gmail.com> in
https://github.com/bitcoin/bitcoin/pull/19425#discussion_r526701954

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-12-07 20:46:03 -05:00
Wladimir J. van der Laan
1a9fa4c5ba Merge #20561: p2p: periodically clear m_addr_known
65273fa0e7 Clear m_addr_known before our periodic self-advertisement (Suhas Daftuar)

Pull request description:

  We use a rolling bloom filter to track which addresses we've previously sent a peer, but after #7125 we no longer clear it every day before our own announcement.  This looks to me like an oversight which has the effect of reducing the frequency with which we actually self-announce our own address, so this reintroduces resetting that filter.

ACKs for top commit:
  naumenkogs:
    ACK 65273fa0e7
  laanwj:
    Code review ACK 65273fa0e7
  sipa:
    utACK 65273fa0e7

Tree-SHA512: 602c155fb6d2249b054fcb6f1c0dd17143605ceb87132286bbd90babf26d258ff6c41f9925482c17e2be41805d33f9b83926cb447f394969ffecd4bccfa0a64f
2020-12-07 23:57:55 +01:00
Jonas Schnelli
836a3dc02c Avoid weak-linked getauxval support on non-linux platforms (like macOS) 2020-12-07 22:16:39 +01:00
Jonas Schnelli
41a413b317 Define correct symbols for getauxval 2020-12-07 21:22:01 +01:00
Wladimir J. van der Laan
d38feb6134 Merge #20535: test: Fix intermittent feature_taproot issue
fa275e1539 test: Fix intermittent feature_taproot issue (MarcoFalke)

Pull request description:

  The nodes might disconnect (e.g. due to "Timeout downloading block" https://cirrus-ci.com/task/5313800947630080?command=ci#L1763) and the test fails to continue.

  Fix that by reconnecting the nodes.

ACKs for top commit:
  laanwj:
    code review ACK fa275e1539

Tree-SHA512: 2871183c8058d8292c9c4ef56ea3d19d5616ca712ebdaabb6609f8c9cd2e16c9ac2ce26aa1e94b346872b7b6fec56b59af151af83de3a5aa08bed01bfcc7187a
2020-12-07 20:37:18 +01:00
Jon Atack
4b7b58b3fe Update net_processing WTXID documentation per BIP339
as BIP339 currently states:

"The wtxidrelay message MUST be sent in response to a version
message from a peer whose protocol version is >= 70016 and
prior to sending a verack. A wtxidrelay message received after
a verack message MUST be ignored or treated as invalid."
2020-12-07 19:27:00 +01:00
Pieter Wuille
c5a8919660 Don't send 'sendaddrv2' to pre-70016 software 2020-12-07 09:13:57 -08:00
MarcoFalke
fac39c1983 wallet: document that tx in CreateTransaction is purely an out-param 2020-12-07 15:02:55 +01:00
MarcoFalke
faac31521b Remove unused and confusing CTransaction constructor 2020-12-07 14:59:33 +01:00
MarcoFalke
fa0d8359b3 log: Clarify that failure to read fee_estimates.dat is non-fatal
An uppercase "ERROR" in the log might indicate a fatal error. Though,
all read-failures for fee_estimates.dat are non-fatal, so avoid the
"ERROR".

Before:
ERROR: CBlockPolicyEstimator::Read(): up-version (149900) fee estimate file

After:
CBlockPolicyEstimator::Read(): unable to read policy estimator data (non-fatal): up-version (149900) fee estimate file
2020-12-07 14:13:20 +01:00
Russell Yanofsky
6965f1352d refactor: Replace uses ChainActive() in interfaces/chain.cpp
Suggested https://github.com/bitcoin/bitcoin/pull/19425#discussion_r456236407
2020-12-07 09:09:53 -04:00
Wladimir J. van der Laan
5c4911e7e7 Merge #20568: doc: Use FeeModes doc helper in estimatesmartfee
fa8abdc995 rpc: Use FeeModes doc helper in estimatesmartfee (MarcoFalke)

Pull request description:

  Not sure why this doesn't use the doc helper, probably an oversight?

ACKs for top commit:
  laanwj:
    Code review ACK fa8abdc995

Tree-SHA512: 1f2dc8356e3476ddcf9cafafa7f9865ad95bed1e3067c0edab8e3c483e374bdbdbecc066167554b4a1b479e28f6a52c4ae6a75a70c67ee4e1ff4f3ba36b04001
2020-12-07 14:09:53 +01:00
Russell Yanofsky
3fbbb9a640 refactor: Get rid of more redundant chain methods
This just drops three interfaces::Chain methods replacing them with other calls.

Motivation for removing these chain methods:

- Need to get rid of findFirstBlockWithTimeAndHeight for #10102, which doesn't
  support overloaded methods
- Followup from
  https://github.com/bitcoin/bitcoin/pull/16426#discussion_r412487403
- phantomcircuit comments about findNextBlock test
  http://www.erisian.com.au/bitcoin-core-dev/log-2020-06-06.html#l-214

Behavior is not changing in any way here. A TODO comment in
ScanForWalletTransactions was removed, but just because it was invalid (see
https://github.com/bitcoin/bitcoin/pull/19195#discussion_r448020762), not
because it was implemented.
2020-12-07 09:09:53 -04:00
MarcoFalke
faefa5db5f log: Clarify that failure to write fee_estimates.dat is non-fatal 2020-12-07 14:03:22 +01:00
MarcoFalke
03b1db6114 Merge #18766: Disable fee estimation in blocksonly mode (by removing the fee estimates global)
4e28753f60 feestimator: encapsulate estimation file logic (Antoine Poinsot)
e8ea6ad9c1 init: don't create a CBlockPolicyEstimator if we don't relay transactions (Antoine Poinsot)
86ff2cf202 Remove the remaining fee estimation globals (Antoine Poinsot)
03bfeee957 interface: remove unused estimateSmartFee method from node (Antoine Poinsot)

Pull request description:

  If the `blocksonly` mode is turned on after running with transaction
  relay enabled for a while, the fee estimation will serve outdated data
  to both the internal wallet and to external applications that might be
  feerate-sensitive and make use of `estimatesmartfee` (for example a
  Lightning Network node).

  This has already caused issues (for example https://github.com/bitcoin/bitcoin/issues/16840 (C-lightning), or https://github.com/lightningnetwork/lnd/issues/2562 (LND)) and it seems prudent to fail rather than to give inaccurate values.

  This fixes #16840, and closes #16890 which tried to fix the symptoms (RPC) but not the cause as mentioned by sdaftuar :
  > If this is a substantial problem, then I would think we should take action to protect our own wallet users as well (rather than hide the results of what our fee estimation would do!).

ACKs for top commit:
  MarcoFalke:
    re-ACK 4e28753f60 👋
  jnewbery:
    utACK 4e28753f60

Tree-SHA512: c869cf03b86d8194002970bbc84662dae76874967949b9be0d9a4511a1eabcb1627c38aca3154da9dcece1a4c49ec02bd4f9fcca2ec310986e07904559e63ba8
2020-12-07 12:59:48 +01:00
John Newbery
3025ca9e77 [net processing] Add RemovePeer()
This allows us to avoid repeated locking in FinalizeNode()
2020-12-07 11:59:24 +00:00
John Newbery
a20ab22786 [net processing] Make GetPeerRef const 2020-12-07 11:57:12 +00:00
John Newbery
ed7e469cee [net_processing] Move peer_map to PeerManager 2020-12-07 11:55:28 +00:00
MarcoFalke
00f4dcd552 Merge #20138: net: Assume that SetCommonVersion is called at most once per peer
fa0f415709 net: Assume that SetCommonVersion is called at most once per peer (MarcoFalke)

Pull request description:

  This restores the check removed in https://github.com/bitcoin/bitcoin/pull/17785#discussion_r503224381

  Instead of using `error`, which was used previously, it uses a newly introduced `Assume()`. `error` had several issues:
  * It logs unconditionally to the debug log
  * It doesn't abort the program when the error is hit in tests

ACKs for top commit:
  practicalswift:
    cr ACK fa0f415709: patch looks correct
  jnewbery:
    utACK fa0f415709

Tree-SHA512: cd7424a9485775e8c7093b725f8f52a90d47485185e79bac80f7810e450d0b3fda608d8805e9239094929f7bad2dca3fe772fb78ae606c2399d15405521e136b
2020-12-07 12:48:55 +01:00
Hennadii Stepanov
c932e0d67e doc: Update wallet database installation guide for macOS 2020-12-07 13:25:06 +02:00
Hennadii Stepanov
ee7b84e63c build: Use Homebrew's sqlite package if it is available 2020-12-07 13:24:59 +02:00
William Casarin
22eb7930a6 tracing: add tracing framework
Signed-off-by: William Casarin <jb55@jb55.com>
2020-12-07 11:55:51 +01:00
William Casarin
933ab8a720 build: detect sys/sdt.h for eBPF tracing 2020-12-07 11:55:51 +01:00
Wladimir J. van der Laan
f3e17686b3 Merge #20468: build: warn when generating man pages for binaries built from a dirty branch
6690adba08 Warn when binaries are built from a dirty branch. (Tyler Chambers)

Pull request description:

  - Adjusted `--version` flag behavior in bitcoind and bitcoin-wallet to have the same behavior.
  - Added `--version` flag to bitcoin-tx to match.
  - Added functionality in gen-manpages.sh to error when attempting to generate man pages for binaries built from a dirty branch.

  mitigates problem with  issue #20412

ACKs for top commit:
  laanwj:
    Tested ACK 6690adba08

Tree-SHA512: b5ca509f1a57f66808c2bebc4b710ca00c6fec7b5ebd7eef58018e28e716f5f2358e36551b8a4df571bf3204baed565a297aeefb93990e7a99add502b97ee1b8
2020-12-07 10:51:45 +01:00
Wladimir J. van der Laan
31438cc818 Merge #20482: Add depends qt fix for ARM macs
c23f6f84ef Add depends qt fix for ARM macs (Jonas Schnelli)

Pull request description:

  With this, depends builds fine on macOS 11 on an Apple Silicon Mac (ARM64).

ACKs for top commit:
  laanwj:
    Code review ACK c23f6f84ef

Tree-SHA512: a8354cec99969cff9e7dab150c335050ddb4b3c93a9f12a4db5e8046f02b11ce692ac17c2b96cbbe7f380c1aa110b15b8d6d48d51bc9c560282c702e99fd8a8d
2020-12-07 10:07:53 +01:00
MarcoFalke
fa8abdc995 rpc: Use FeeModes doc helper in estimatesmartfee
Can be reviewed with --ignore-all-space
2020-12-07 09:28:47 +01:00
Jonas Schnelli
eab63b971d Merge #19847: rpc, refactor: Avoid duplicate set lookup in gettxoutproof
52fc39917f rpc: Reject empty txids in gettxoutproof (João Barbosa)
73dc19a330 rpc, refactor: Avoid duplicate set lookup in gettxoutproof (João Barbosa)

Pull request description:

ACKs for top commit:
  jonasschnelli:
    code review ACK 52fc39917f

Tree-SHA512: 76b18e5235e8b2d394685515a4a60335666eeb0f6b31c1d397f7db2fbe681bc817b8cd3e8f6708b9dacd6113e4e1d94837072cae27834b8a1a22d2717db8191e
2020-12-07 09:17:08 +01:00
MarcoFalke
1a04f45fe9 Merge #19832: p2p: Put disconnecting logs into BCLog::NET category
1816327e53 p2p: Put disconnecting logs into BCLog::NET category (Hennadii Stepanov)

Pull request description:

  It's too noisy:
  ```
  $ cat debug.log | wc -l
  28529
  $ cat debug.log | grep "Disconnecting and discouraging peer" | wc -l
  10177
  ```

ACKs for top commit:
  MarcoFalke:
     noban, addnode and local peers are still unconditionally logged (as they should), but this one can go into a category, so cr-ACK 1816327e53
  practicalswift:
    ACK 1816327e53 for the reasons MarcoFalke gave above.
  ajtowns:
    ACK 1816327e53

Tree-SHA512: c312c1009090840659b2cb1364d8ad9b6ab8e742fc462aef169996d93c76c248507639a00257ed9d73a6916c01176b1793491b2305e92fdded5f9de0935b6ba6
2020-12-07 09:07:57 +01:00
fanquake
d0ca394596 Merge #20476: contrib: Add test for ELF symbol-check
ed1bbcefea contrib: add MACHO tests to symbol-check tests (fanquake)
5bab08df17 contrib: Add test for ELF symbol-check (Wladimir J. van der Laan)

Pull request description:

  Check both failure cases:
  - Use a glibc symbol from a version that is too new
  - Use a symbol from a library that is not in the allowlist

  And also check a conforming binary.

  Adding a similar check for Windows PE can be done in a separate PR.

ACKs for top commit:
  fanquake:
    ACK ed1bbcefea

Tree-SHA512: fd437612e003922465fe1396efa1fa3a64bd1c7b0a514d2a0a7a0caaaa9fb5cb43e0ed7caec15eb0a3508692c9eb3212d7ba3c7e8180b942dd3e50616ad6e557
2020-12-07 15:33:37 +08:00
wodry
32045bbfd5 [doc] Tidy up Tor doc (more stringent)
This is a follow up to https://github.com/bitcoin/bitcoin/pull/19638 that left some deprectaed "hidden service/server" naming occurences.

It also shall make the chapter titles regarding creation of onion services stringent and easy to read and distinguish.

It removes the one and only reference to the testnet (here the testnet onion service port), as it is not explained that it references to the testnet and I do not know why it is mentioned there. It is only confusing. Also, as said, the testnet is not referenced at any other place in this document.
2020-12-07 05:17:26 +01:00
fanquake
cef2efafce Merge #20577: doc: libconsensus: add missing error code description, fix NBitcoin link
cb0b7125c1 doc: libbitcoinconsensus: add missing error code description, fix NBitcoin link (Sebastian Falbesoner)

Pull request description:

  This PR improves the libbitcoinconsensus description in `shared-libraries.md` in two ways:
  * adds the missing error code description for `bitcoinconsensus_ERR_INVALID_FLAGS` (introduced by commit 5ca8ef299a, PR #8976)
  * updates and fixes the link to the NBitcoin implementation (introduced by commit 3361edd010, PR #6430)
      * the owner of the `NBitcoin` github repository changed from `NicolasDorier` to `MetacoSA` (redirection still worked though)
      * instead of dynamically referring to a file in master with a fixed line number (which is obviously always quickly outdated), use a permalink with a file numbers area

ACKs for top commit:
  MarcoFalke:
    cr ACK cb0b7125c1
  harding:
    Code (documentation) review ACK cb0b7125c1.  Text is clear and seems accurate, and the link checks out.

Tree-SHA512: 9840458db6fb40e71c9852104aefcec5abbaf5054c6123701181dd477cea8c81d3647f376b67692159adf577c9b6305b05b784728bf9f14a753fab5898075a4e
2020-12-07 10:17:23 +08:00
practicalswift
31b136e580 Don't declare de facto const reference variables as non-const 2020-12-06 18:44:31 +00:00
practicalswift
1c65c075ee Don't declare de facto const member functions as non-const 2020-12-06 18:44:25 +00:00
MarcoFalke
64156ad4d1 Merge #19893: test: Remove or explain syncwithvalidationinterfacequeue
fa6af31227 test: Document why syncwithvalidationinterfacequeue is needed in tests (MarcoFalke)
fa135a13b8 Revert "test: Add missing sync_all to wallet_balance test" (MarcoFalke)

Pull request description:

  syncwithvalidationinterfacequeue is a hidden test-only RPC, so it should not be used when it is not needed. Thus, either remove it or explain why it is needed.

ACKs for top commit:
  fjahr:
    Code review ACK fa6af31227

Tree-SHA512: de30db4ab521184091ee5beeab02989138cf7cf05088f766a2fb106151b239310b63d5380cb79e2a072f72c5ae9513aecae8eb9c1c7be713771585c3cb04d63a
2020-12-06 19:13:10 +01:00
MarcoFalke
9385549a31 Merge #20581: Don't make "in" parameters look like "out"/"in-out" parameters: pass by ref to const instead of ref to non-const
12dcdaaa54 Don't make "in" parameters look like "out"/"in-out" parameters: pass by ref to const instead of ref to non-const (practicalswift)

Pull request description:

  Don't make "in" parameters look like "out"/"in-out" parameters: pass by ref to const instead of ref to non-const.

ACKs for top commit:
  MarcoFalke:
    cr ACK 12dcdaaa54
  fjahr:
    Code review ACK 12dcdaaa54
  hebasto:
    ACK 12dcdaaa54, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: bf9e476dc27bfbd856b42cc4720fb2d4b74e64039776ce067691aaf536360b36902ef72988c7914505df5bf0ed2f1557e0eba52b87935d3a17db865ea17f6093
2020-12-06 19:10:19 +01:00
practicalswift
5021810650 Make CanFlushToDisk a const member function 2020-12-06 00:54:10 +00:00
practicalswift
281cf99554 Do not run functions with necessary side-effects in assert() 2020-12-06 00:48:09 +00:00
practicalswift
12dcdaaa54 Don't make "in" parameters look like "out"/"in-out" parameters: pass by ref to const instead of ref to non-const 2020-12-06 00:22:40 +00:00
Sebastian Falbesoner
cb0b7125c1 doc: libbitcoinconsensus: add missing error code description, fix NBitcoin link 2020-12-05 13:37:00 +01:00
MarcoFalke
f35e4d906f Merge #20572: ci: Adjust Cirrus CI task names (follow up)
667b6a29df ci: Adjust Cirrus CI task names (follow up) (Hennadii Stepanov)

Pull request description:

  "no depends" implies "only system libs".

  #20545 follow up .

ACKs for top commit:
  practicalswift:
    ACK 667b6a29df

Tree-SHA512: b3bf5dc931b7aeed30932e8d6c5f5b0040e4be5a5676ed23b41675ebad2f8a016b1abb27584b68970ee00bc5b2af7122ca9566f373db5f3a2e38fcf7eee6522a
2020-12-05 06:24:46 +01:00
Jon Atack
6fa72ceb80 test: add coverage for passing fee rate as a string 2020-12-04 22:35:31 +01:00
Hennadii Stepanov
667b6a29df ci: Adjust Cirrus CI task names (follow up)
"no depends" implies "only system libs".
2020-12-04 23:17:31 +02:00
Jon Atack
ce207d6b93 wallet, bugfix: allow send to take string fee rate values 2020-12-04 22:12:36 +01:00
João Barbosa
52fc39917f rpc: Reject empty txids in gettxoutproof 2020-12-04 18:38:23 +00:00
João Barbosa
73dc19a330 rpc, refactor: Avoid duplicate set lookup in gettxoutproof 2020-12-04 18:38:23 +00:00
Carl Dong
3eb94ec81b sync: Use decltype(auto) return type for WITH_LOCK
Now that we're using C++17, we can use the decltype(auto) return type
(available since C++14) for functions and lambda expressions.

As demonstrated in this commit, this can simplify cases where previously
the compiler failed to deduce the correct return type.

Just for reference, for the "assign to ref" cases fixed here, there are
3 possible solutions:

- Return a pointer and immediately deref as used before this commit
- Make sure the function/lambda returns declspec(auto) as used after
  this commit
- Class& i = WITH_LOCK(..., return std::ref(...));

-----

References:
1. https://en.cppreference.com/w/cpp/language/function#Return_type_deduction
2. https://en.cppreference.com/w/cpp/language/template_argument_deduction#Other_contexts
3. https://en.cppreference.com/w/cpp/language/auto
4. https://en.cppreference.com/w/cpp/language/decltype

Explanations:
1. https://stackoverflow.com/a/21369192
2. https://stackoverflow.com/a/21369170
3. Item 3 in Effective Modern C++ (Scott Meyers) via jnewbery
2020-12-04 12:23:05 -05:00
Russell Yanofsky
f3d870fc22 wallet: List all wallets in non-SQLite or non-BDB builds
This commit does not change behavior when bitcoin is built normally with both
the SQLite and BDB libraries. It just makes non-SQLite and non-BDB builds
more similar to the normal build. Specifically:

- It makes wallet directory lists always include all wallets so wallets don't
  appear missing depending on the build.

- It now triggers specific "Build does not support SQLite database format" and
  "Build does not support Berkeley DB database format" errors if a wallet can't
  be loaded instead of the more ambiguous and scary "Data is not in recognized
  format" error.
2020-12-04 11:03:28 -04:00
MarcoFalke
751ffaabad Merge #20562: tests: Test that a fully signed tx given to signrawtx is unchanged
773c42b265 tests: Test that a fully signed tx given to signrawtx is unchanged (Andrew Chow)

Pull request description:

  Tests that a fully signed transaction given to `signrawtransactionwithwallet` is both unchanged and marked as complete. This tests for a regression in 0.20 where the transaction would not be marked as complete.

ACKs for top commit:
  MarcoFalke:
    ACK 773c42b265 🕶

Tree-SHA512: b32afd5f2667c817ee59c37af78cb37352e7c22d0271833d446fd61f11b354974694128a3b408f214db459caef3fd67ecdbf1664280ac2de602215f05aa4a6de
2020-12-04 16:03:28 +01:00
Russell Yanofsky
d70dc89e78 refactor: Consolidate redundant wallet database path and exists functions
No change in behavior. Just remove a little bit of code, reduce macro usage,
remove duplicative functions, and make BDB and SQLite implementations more
consistent with each other.
2020-12-04 11:03:28 -04:00
Russell Yanofsky
6a7a63644c refactor: Drop call to GetWalletEnv in wallet salvage code
No observable change in behavior. This just avoids a redundant environment
lookup. Motivation is to be able to simplify the GetWalletEnv implementation in
an upcoming commit.
2020-12-04 11:03:28 -04:00
Russell Yanofsky
6ee9cbdd18 refactor: Replace ListWalletDir() function with ListDatabases()
No change to behavior. This is just cleanup after previous MOVEONLY commit to
make db.h list function fit conventions of surrounding functions.
2020-12-04 11:03:28 -04:00
Russell Yanofsky
5aaeb6cf87 MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir
This commit does not change to any code and behavior. It it is easily reviewed
with the --color-moved=dimmed_zebra git diff option.

Motivation for this change is to:

- Consolidate redundant functions
  IsBDBFile /ExistsBerkeleyDatabase / SplitWalletPath, and
  IsSQLiteFile / ExistsSQLiteDatabase in the next commits

- Detect SQLite wallets consistently regardless whether bitcoin is built with
  SQLite support in the next commits

- Avoid attempting to open SQLite databases with the BDB library when bitcoin
  is built without SQLite support in the next commits
2020-12-04 11:03:28 -04:00
MarcoFalke
c1604483d3 Merge #20566: refactor: Use C++17 std::array where possible
fac7ab1d5b refactor: Use C++17 std::array where possible (MarcoFalke)

Pull request description:

  Using the C++11 std::array with explicit template parameters is problematic because overshooting the size will fill the memory with default constructed types.

  For example,

  ```cpp
  #include <array>
  #include <iostream>

  int main()
  {
      std::array<int, 3> a{1, 2};
      for (const auto& i : a) {
          std::cout << i << std::endl;  // prints "1 2 0"
      }
  }
  ```

ACKs for top commit:
  jonasschnelli:
    Code Review ACK fac7ab1d5b
  practicalswift:
    cr ACK fac7ab1d5b
  vasild:
    ACK fac7ab1d
  promag:
    Code review ACK fac7ab1d5b.

Tree-SHA512: ef7e872340226e0d6160e6fd66c6ca78b2ef9c245fa0ab27fe4777aac9fba8d5aaa154da3d27b65dec39a6a63d07f1063c3a8ffb667a98ab137756a1a0af2656
2020-12-04 15:18:48 +01:00
fanquake
9601c0d7aa Merge #20565: Android : Ensure pic build for bdb
e373959d6f Android : Ensure pic build for bdb (Block Mechanic)

Pull request description:

  This pr ensures android builds for the BDB dependency have the pic flag enabled. Android builds were failing to link with reloc errors.

ACKs for top commit:
  laanwj:
    Code review ACK e373959d6f
  jonasschnelli:
    utACK e373959d6f

Tree-SHA512: 68319ed7cc0bd295eaa87dd53ba051daeb1456bc3ab9b48ca0c4b831a9c8da1073480478efde73689f0e403e37409a8459229264656f05ba5fef6c257a74f977
2020-12-04 22:13:22 +08:00
John Newbery
a529fd3e3f [net processing] Move GetNodeStateStats into PeerManager 2020-12-04 11:37:45 +00:00
Hennadii Stepanov
c96d1f65a5 build, refactor: Check that Homebrew's qt5 package is actually installed
This change unifies Homebrew packages workflow, and does not change
behavior.
2020-12-04 13:02:08 +02:00
MarcoFalke
fa0f415709 net: Assume that SetCommonVersion is called at most once per peer 2020-12-04 11:19:15 +01:00
MarcoFalke
dca80ffb45 Merge #20255: util: Add Assume() identity function
faa05854f8 util: Remove probably misleading TODO (MarcoFalke)
fac5efe730 util: Add Assume() identity function (MarcoFalke)
fa861569dc util: Allow Assert(...) to be used in all contexts (practicalswift)

Pull request description:

  This is needed for #20138. Please refer to the added documentation for motivation.

ACKs for top commit:
  practicalswift:
    cr ACK faa05854f8
  jnewbery:
    utACK faa05854f8
  hebasto:
    ACK faa05854f8, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 72165fbd898b92ab9a79b070993fa1faa86c2e3545b6645e72c652bda295d5107bc298d0482bf3aaf0926fc0c3e6418a445c0e073b08568c44231f547f76a688
2020-12-04 11:07:28 +01:00
Wladimir J. van der Laan
ca759bc743 Merge #20200: doc: rename CODEOWNERS to REVIEWERS
86e6add5ca doc: rename CODEOWNERS to REVIEWERS (Adam Jonas)

Pull request description:

  This PR renames the CODEOWNERS file to REVIEWERS and works with DrahtBot to leave a comment requesting a review.

  Testing of the functionality was done on https://github.com/adamjonas/bitcoin-codeowners-sandbox/pulls.

  ~EDIT: [after further testing of a fake organization](https://github.com/jonasorg/bitcoin-codeowners-sandbox-org/pulls), it appears that in order to be automatically tagged, the reviewer requires write level permissions. This is undocumented and will obviously close the circle of who can be tagged and so this PR reverts the addition of the file in #18949.~

  ~This removes a line not being parsed in the CODEOWNERS file introduced in #18949. While the pattern [checked out](https://github.com/bitcoin/bitcoin/pull/18949#issuecomment-648859076) using `git ls-files`, it is preventing the CODEOWNERS file from automatically tagging reviewers. For future modifications to this file, note that [any line failing to parse causes the entire file to stop identifying codeowners](http://www.benjaminoakes.com/git/2018/08/10/Testing-changes-to-GitHub-CODEOWNERS/#:~:text=warning).~

  ~I experimented in a [sandbox repo](https://github.com/adamjonas/bitcoin-codeowners-sandbox/pulls) to discover the offending line and this change appears to fix the problem.~

ACKs for top commit:
  laanwj:
    Doesn't look like github is going to fix this any time soon and besides REVIEWERS is a better name so ACK 86e6add5ca

Tree-SHA512: b3425eca4be62f829f0edcfa08232df310680835c2f60e9fa36956a6b59a2b0cd0ab580c572c87affccd843fbd849f4f99e4b3d9d7b6070ab117953f04e17f5a
2020-12-04 10:36:37 +01:00
MarcoFalke
fab48da908 test: Fix intermittent wallet_multiwallet issue with got_loading_error 2020-12-04 10:32:00 +01:00
MarcoFalke
fa8e15f7b7 test: pep8 wallet_multiwallet.py 2020-12-04 10:27:38 +01:00
Jonas Schnelli
257cf05f9b Merge #20563: build: Check that Homebrew's berkeley-db4 package is actually installed
d3ef947524 build: Check that Homebrew's berkeley-db4 package is actually installed (Hennadii Stepanov)

Pull request description:

  On master (a0489f3472) the `configure` script is not able to determine that Homebrew's `berkeley-db4` package is uninstalled. This causes a compile error on macOS.

  With this PR, and with the [uninstalled](https://stackoverflow.com/questions/20802320/detect-if-homebrew-package-is-installed) `berkeley-db4` package:
  ```
  % ./configure -q
  configure: error: libdb_cxx headers missing, Bitcoin Core requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)
  ```

  Related #20478.

ACKs for top commit:
  promag:
    Tested ACK d3ef947524.
  willcl-ark:
    tACK d3ef947524
  jonasschnelli:
    utACK d3ef947524

Tree-SHA512: 8dc532e08249ec63bd357594aa458d314b6e8537fc63f5b1d509c84d0d71d5b1f70172caa1a7efe2fc8af31c829e7982a0695cf3fbe5cbc477019550269915e1
2020-12-04 10:08:18 +01:00
Wladimir J. van der Laan
34c80d9eee test: Add option to git-subtree-check to do full check, add help
This adds a brief help text to `git-subtree-check.sh` and adds and an
option to do a full remote check instead of having two different code
paths with a successful exit status. Also make it explicit that the CI
is not doing this.
2020-12-04 09:36:17 +01:00
MarcoFalke
fac7ab1d5b refactor: Use C++17 std::array where possible 2020-12-04 08:22:45 +01:00
MarcoFalke
e2ae6a2bef Merge #20182: ci: Build with --enable-werror by default, and document exceptions
2f6fe4e4e9 ci: Build with --enable-werror by default, and document exceptions (Hennadii Stepanov)

Pull request description:

  This PR prevents introducing of new compiler warnings in the master branch, e.g., #19986, #20162.

ACKs for top commit:
  practicalswift:
    cr ACK 2f6fe4e4e9: patch looks correct
  MarcoFalke:
    re-ACK 2f6fe4e4e9 🏏
  vasild:
    ACK 2f6fe4e

Tree-SHA512: 23b5feb5bc472658c992d882ef61af23496f25adaa19f9c79bfaef5d2db273d44981aa93b1631a7d37cb58755283c1dacf3f2d68e501522d3fa8c965ab646d19
2020-12-04 07:23:58 +01:00
Block Mechanic
e373959d6f Android : Ensure pic build for bdb 2020-12-04 01:57:54 +02:00
Hennadii Stepanov
d3ef947524 build: Check that Homebrew's berkeley-db4 package is actually installed 2020-12-03 23:39:14 +02:00
Andrew Chow
773c42b265 tests: Test that a fully signed tx given to signrawtx is unchanged
Tests that a fully signed transaction given to
signrawtransactionwithwallet is both unchanged and marked as complete.
This tests for a regression in 0.20 where the transaction would not be
marked as complete.
2020-12-03 15:53:58 -05:00
Hennadii Stepanov
2f6fe4e4e9 ci: Build with --enable-werror by default, and document exceptions 2020-12-03 20:51:39 +02:00
Suhas Daftuar
65273fa0e7 Clear m_addr_known before our periodic self-advertisement
This behavior was apparently inadvertently broken in 5400ef6; without this
change our daily self-announcements frequently go unsent, because our
address is still in the peer's rolling bloom filter (for potentially many
days, depending on addr traffic).
2020-12-03 13:10:37 -05:00
Wladimir J. van der Laan
a0489f3472 Merge #20221: net: compat.h related cleanup
cadb77a6ab net: Add compat.h header for htonl function (Hennadii Stepanov)
f796f0057b net: Drop unneeded headers when compat.h included (Hennadii Stepanov)
467c346448 net: Drop unneeded Windows headers in compat.h (Hennadii Stepanov)

Pull request description:

  It is the `compat.h` header's job to provide platform-agnostic interfaces for internet operations.
  No need in `#include <arpa/inet.h>` scattered around.

ACKs for top commit:
  practicalswift:
    re-ACK cadb77a6ab: patch looks even better
  laanwj:
    Code review ACK cadb77a6ab

Tree-SHA512: 625ff90b2806310ab856a6ca1ddb6d9a85aa70f342b323e8525a711dd12219a1ecec8373ec1dca5a0653ffb11f9b421753887b25615d991ba3132c1cca6a3c6e
2020-12-03 14:54:07 +01:00
Wladimir J. van der Laan
3fa6a9fc8c Merge #20506: ci: AppVeyor fixes for Visual Studio 2019 16.8.1 image
2c69381f3d Removed redundant git pull from appveyor config. (Aaron Clauson)
8b99e609e7 Adjusted msvc compiler and linker settings to remove optimisations that are causing sporadic ABI issues on Visual Studio updates. (Aaron Clauson)

Pull request description:

  The motivation for this PR is twofold:

    1. Update the Qt binaries used by the appveyor CI job after a recent update to Visual Studio 2019 used in the Appveyor build image resulted in ABI incompatibilities,

    2. Remove optimisations and debug information from the Bitcoin Core `Release` msvc build to reduce the chance of future ABI incompatibility issues for future Visual Studio updates.

  The changes made in this PR are:

   - Changed appveyor config file hash to use a new version of Qt pre-compiled binaries built for Visual Studio 2019 v16.8.1.
   - Adjusted msvc compiler and linker settings to remove optimisations and debug information generation to help avoid future ABI issues on Visual Studio updates.
   - Tidied up debug and release configuration blocks in common project file to avoid duplication.
   - Updated appveyor config to use latest Visual Studio 2019 image.
   - Bumped vcpkg version to tag `2020.11-1` for binary caching feature*.

  See #20392 for related discussion.

  *Binary caching is a new [vcpkg feature](https://github.com/microsoft/vcpkg/blob/master/docs/specifications/binarycaching.md) that allows dependency caching. This PR is not using the feature but by updating the vcpkg version it means it can be optionally used by other contributors in their own Appveyor configs. By caching the vcpkg dependencies using this feature my build times are reduced by approx 10 minutes.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 2c69381f3d
  laanwj:
    Concept ACK 2c69381f3d

Tree-SHA512: 372f5b8b3b5f7e56b78045f9fc06a22fd9f5366d06e99e2f1eaad6d741680da74d0e6371e7bc580cb41f4d6696b2101b950167309d33e98242578b458ace813a
2020-12-03 14:36:51 +01:00
Wladimir J. van der Laan
a3186b6da6 Merge #20520: depends: Do not force Precompiled Headers (PCH) for building Qt on Linux
c82d15b6d1 depends: Do not force Precompiled Headers (PCH) for building Qt on Linux (Hennadii Stepanov)

Pull request description:

  On CentOS 8 (Cirrus CI job) the forced `-pch` option breaks Qt build.

  Removing `-pch` option does not affect build time for other systems:

  - master (e2ff5e7b35):
  ```
  $ time make -j 9 -C depends/ qt
  ...
  Caching qt...
  make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'

  real	4m22,359s
  user	18m3,719s
  sys     1m24,769s
  ```

  - this PR:
  ```
  $ time make -j 9 -C depends/ qt
  ...
  Caching qt...
  make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'

  real	4m14,862s
  user	18m3,355s
  sys 	1m24,506s
  ```

  Qt docs: https://doc.qt.io/qt-5/qmake-precompiledheaders.html

  Fixes #20423

ACKs for top commit:
  MarcoFalke:
    review ACK c82d15b6d1

Tree-SHA512: 0f2a3712e90de881d00f8e56c363edde33dd4f5c117df5744ab4e51d0a8146331de7236bc8329d68ddd91535cd853e68ee80ef4cceb6a909786abfd8881b01e8
2020-12-03 13:23:09 +01:00
Antoine Poinsot
4e28753f60 feestimator: encapsulate estimation file logic
This moves the fee_estimates file management to the CBlockPolicyEstimator
Flush() method.

Co-authored-by: John Newbery <john@johnnewbery.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-12-03 12:56:37 +01:00
Antoine Poinsot
e8ea6ad9c1 init: don't create a CBlockPolicyEstimator if we don't relay transactions
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-12-03 12:56:37 +01:00
Antoine Poinsot
86ff2cf202 Remove the remaining fee estimation globals
This moves the CBlockPolicyEstimator to the NodeContext, which get rids
of two globals and allows us to conditionally create the
CBlockPolicyEstimator (and to remove a circular dep).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-12-03 12:56:37 +01:00
Antoine Poinsot
03bfeee957 interface: remove unused estimateSmartFee method from node
Co-Authored-by: MarcoFalke <falke.marco@gmail.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-12-03 12:56:36 +01:00
fanquake
ed1bbcefea contrib: add MACHO tests to symbol-check tests 2020-12-03 12:16:19 +01:00
Wladimir J. van der Laan
5bab08df17 contrib: Add test for ELF symbol-check
Check both failure cases:
- Use a glibc symbol from a version that is too new
- Use a symbol from a library that is not in the allowlist

And also check a conforming binary.

Adding a similar check for Windows PE can be done in a separate PR.
2020-12-03 12:15:12 +01:00
Aaron Clauson
2c69381f3d Removed redundant git pull from appveyor config. 2020-12-03 09:23:22 +00:00
MarcoFalke
681ce59d0e Merge #20466: test: Fix intermittent p2p_fingerprint issue
fad7be584f test: Fix intermittent p2p_finerprint issue (MarcoFalke)

Pull request description:

  A single sync_with_ping can't be used to drop a block announcement, as the block might be sent *after* the ping has been responded to.

  Fix that by waiting for the block.

ACKs for top commit:
  theStack:
    ACK fad7be584f

Tree-SHA512: d43ba9d07273486858f65a26326cc6637ef743bf7b400e5048ba7eac266fb1893283e6503dd49f179caa1abab2977315fb70ba9fba34be9a817a74259d8e4034
2020-12-03 10:09:30 +01:00
João Barbosa
206f74e88c Support make src/bitcoin-node and src/bitcoin-gui 2020-12-02 23:05:35 +00:00
MarcoFalke
a35b948836 Merge #20545: ci: Adjust cirrus ci task names
fa5c4f12f5 ci: Adjust cirrus ci task names (MarcoFalke)

Pull request description:

  The task names are too long for GitHub to display them properly without truncation in the "checks-view". Fix that by using a new naming scheme:

  * Native builds don't mention "x86_64 Linux", as it is redundant, they do mention the OS (bionic or focal) in the name suffix
  * Cross builds mention the target in the prefix and the OS (always bionic) in the suffix
  * the macos native build simply says "macos native"

ACKs for top commit:
  practicalswift:
    ACK fa5c4f12f5: patch&nbsp;&nbsp;looks&nbsp;&nbsp;correct!
  hebasto:
    ACK fa5c4f12f5, I have reviewed the code and it looks OK, I agree it can be merged.
  RiccardoMasutti:
    ACK fa5c4f1

Tree-SHA512: 856deb0577c97c70069ef1d369991addc49522135c0ad9e382218fd79ba3d55a95d6c601288dcef0510764b92fbd30a9d7de32b08dc5be55482deab14049b892
2020-12-02 17:22:00 +01:00
fanquake
0a13d15c14 Merge #20530: lint, refactor: Update cppcheck linter to c++17 and improve explicit usage
1e62350ca2 refactor: Improve use of explicit keyword (Fabian Jahr)
c502a6dbfb lint: Use c++17 std in cppcheck linter (Fabian Jahr)

Pull request description:

  I found the `extended-lint-cppcheck` linter still uses `std=c++11` when reviewing #20471. The only difference in the output after this change is one line is missing:

  ```
  src/script/descriptor.cpp:159:5: warning: Struct 'PubkeyProvider' has a constructor with 1 argument that is not explicit. [noExplicitConstructor]
  ```

  After some digging, I am still not sure why this one is ignored with c++17 when 40 other`noExplicitConstructor` warnings were still appearing.

  In the second commit, I fix these warnings, adding `explicit` where appropriate and adding fixes to ignore otherwise.

ACKs for top commit:
  practicalswift:
    cr ACK 1e62350ca2: patch looks correct!
  MarcoFalke:
    review ACK 1e62350ca2

Tree-SHA512: dff7b324429a57160e217cf38d9ddbb6e70c6cb3d3e3e0bd4013d88e07afc2292c3df94d0acf7122e9d486322821682ecf15c8f2724a78667764c05d47f89a12
2020-12-02 20:52:19 +08:00
Hennadii Stepanov
c82d15b6d1 depends: Do not force Precompiled Headers (PCH) for building Qt on Linux
On CentOS 8 the forced '-pch' option breaks Qt build.
Removing '-pch' option does not affect build time for other Linux
systems.
2020-12-02 13:49:25 +02:00
Aaron Clauson
8b99e609e7 Adjusted msvc compiler and linker settings to remove optimisations that are causing sporadic ABI issues on Visual Studio updates.
Tidied up debug and release configuration blocks in common project file to avoid duplication.

Updated appveyor config to use latest Visual Studio 2019 image.

Changed appveyor config file hash to use a new version of Qt pre-compiled binaries built for Visual Studio 2019 v16.8.1.

Bumped vcpkg version to tag '2020.11-1' for binary caching feature.

See #20392 for related discussion.
2020-12-02 11:39:01 +00:00
MarcoFalke
fa5c4f12f5 ci: Adjust cirrus ci task names 2020-12-02 11:52:03 +01:00
MarcoFalke
3693fccc3a Merge #20543: ci: no-longer exclude feature_block in TSAN job
2b356117e9 ci: no-longer exclude feature_block in TSAN job (fanquake)

Pull request description:

  The TSAN job is now running on Cirrus.
  Increase the allocated memory to the [maximum allowed](https://cirrus-ci.org/guide/linux/#linux-containers).

ACKs for top commit:
  jonasschnelli:
    utACK 2b356117e9 - checked the CI run and confirmed that the feature_block runs: https://cirrus-ci.com/task/6008403543719936?command=ci#L3249
  MarcoFalke:
    review ACK 2b356117e9

Tree-SHA512: b774995600361c74bc3267b566e12add66a4604bdf88f6e3f69669edbb8d7aff6f20fdbf0ef98187be4730ce4e18b1939bbcecd993a5c5c1ff40b237c7921b71
2020-12-02 10:56:14 +01:00
Jonas Schnelli
c33662a0ea Merge #18948: qt: Call setParent() in the parent's context
8963b2c71f qt: Improve comments in WalletController::getOrCreateWallet() (Hennadii Stepanov)
5fcfee68af qt: Call setParent() in the parent's context (Hennadii Stepanov)
5659e73493 qt: Add ObjectInvoke template function (Hennadii Stepanov)

Pull request description:

  The `setParent(parent)` internally calls `QCoreApplication::sendEvent(parent, QChildEvent)` that implies running in the thread which created the parent object. That is not the case always, and an internal assertion fails in the debug mode.

  Steps to reproduce this issue on master (007e15dcd7) on Linux Mint 20 (x86_64):

  ```
  $ make -C depends DEBUG=1
  $ CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
  $ make
  $ QT_FATAL_WARNINGS=1 lldb src/qt/bitcoin-qt -- --regtest -debug=qt
  (lldb) target create "src/qt/bitcoin-qt"
  Current executable set to '/home/hebasto/GitHub/bitcoin/src/qt/bitcoin-qt' (x86_64).
  (lldb) settings set -- target.run-args  "--regtest" "-debug=qt"
  (lldb) run
  Process 431562 launched: '/home/hebasto/GitHub/bitcoin/src/qt/bitcoin-qt' (x86_64)
  # load wallet via GUI
  Process 431562 stopped
  * thread #24, name = 'QThread', stop reason = signal SIGABRT
      frame #0: 0x00007ffff794518b libc.so.6`__GI_raise(sig=2) at raise.c:51:1
  (lldb) bt
  * thread #24, name = 'QThread', stop reason = signal SIGABRT
    * frame #0: 0x00007ffff794518b libc.so.6`__GI_raise(sig=2) at raise.c:51:1
      frame #1: 0x00007ffff7924859 libc.so.6`__GI_abort at abort.c:79:7
      frame #2: 0x0000555556508ec4 bitcoin-qt`::qt_message_fatal((null)=<unavailable>, context=<unavailable>, message=<unavailable>) at qlogging.cpp:1690:15
      frame #3: 0x00005555565099cf bitcoin-qt`QMessageLogger::fatal(this=<unavailable>, msg=<unavailable>) const at qlogging.cpp:796:21
      frame #4: 0x000055555650479d bitcoin-qt`qt_assert_x(where=<unavailable>, what=<unavailable>, file=<unavailable>, line=<unavailable>) at qglobal.cpp:3088:46
      frame #5: 0x0000555556685733 bitcoin-qt`QCoreApplicationPrivate::checkReceiverThread(receiver=0x0000555557b27510) at qcoreapplication.cpp:557:5
      frame #6: 0x00005555567ced86 bitcoin-qt`QApplication::notify(this=0x00007fffffffd4a0, receiver=0x0000555557b27510, e=0x00007fff9a7f8ce0) at qapplication.cpp:2956:27
      frame #7: 0x0000555556685d31 bitcoin-qt`QCoreApplication::notifyInternal2(receiver=0x0000555557b27510, event=0x00007fff9a7f8ce0) at qcoreapplication.cpp:1024:24
      frame #8: 0x00005555566c9224 bitcoin-qt`QObjectPrivate::setParent_helper(QObject*) [inlined] QCoreApplication::sendEvent(event=<unavailable>, receiver=<unavailable>) at qcoreapplication.h:233:59
      frame #9: 0x00005555566c9210 bitcoin-qt`QObjectPrivate::setParent_helper(this=0x00007fff85855260, o=0x0000555557b27510) at qobject.cpp:2036
      frame #10: 0x00005555566c9b41 bitcoin-qt`QObject::setParent(this=<unavailable>, parent=<unavailable>) at qobject.cpp:1980:24
      frame #11: 0x0000555555710be8 bitcoin-qt`WalletController::getOrCreateWallet(std::unique_ptr<interfaces::Wallet, std::default_delete<interfaces::Wallet> >) + 2534

  ...
  ```

  Fixes #18835.

ACKs for top commit:
  ryanofsky:
    Code review ACK 8963b2c71f. No changes since last review, just rebase because of conflict on some adjacent lines
  jonasschnelli:
    utACK 8963b2c71f

Tree-SHA512: fef615904168717df3d8a0bd85eccc3eef990cc3e66c9fa280c8ef08ea009a7cb5a2a4f868ed0be3c0fe5bf683e8465850b5958deb896fdadd22d296186c9586
2020-12-02 10:52:09 +01:00
MarcoFalke
7f0f01f0ab Merge #20507: sync: print proper lock order location when double lock is detected
db058efeb0 sync: use HasReason() in double lock tests (Vasil Dimov)
a21dc469cc sync: const-qualify the argument of double_lock_detected() (Vasil Dimov)
6d3689fcf6 sync: print proper lock order location when double lock is detected (Vasil Dimov)

Pull request description:

  Before:
  ```
  Assertion failed: detected double lock at src/sync.cpp:153, details in debug log.
  ```

  After:
  ```
  Assertion failed: detected double lock for 'm' in src/test/sync_tests.cpp:40 (in thread ''), details in debug log.
  ```

ACKs for top commit:
  jonasschnelli:
    utACK db058efeb0
  ajtowns:
    ACK db058efeb0
  hebasto:
    ACK db058efeb0, tested on Linux Mint 20 (x86_64).

Tree-SHA512: 452ddb9a14e44bb174135b39f2219c76eadbb8a6c0e80d64a25f995780d6dbc7b570d9902616db94dbfabaee197b5828ba3475171a68240ac0958fb203a7acdb
2020-12-02 09:44:55 +01:00
fanquake
2b356117e9 ci: no-longer exclude feature_block in TSAN job
The TSAN job is now running on Cirrus.
Increase the allocated memory to the maximum allowed.
2020-12-02 16:42:07 +08:00
MarcoFalke
283f22cabb Merge #20461: rpc: Validate -rpcauth arguments
053b4fbad8 doc: Release note regarding -rpcauth validation (João Barbosa)
46001323b1 rpc: Validate -rpcauth arguments (João Barbosa)
d37c813a43 rpc: Refactor to process -rpcauth once (João Barbosa)

Pull request description:

  Invalid `-rpcauth` arguments are currently silently ignored. This make server initialization fail if any `-rpcauth` is invalid.

ACKs for top commit:
  MarcoFalke:
    review ACK 053b4fbad8
  jonatack:
    ACK 053b4fbad8
  ryanofsky:
    Code review ACK 053b4fbad8. Only changes since last review are moving a variable declaration and adding a comment, release notes, and a `const`.

Tree-SHA512: c99923d4a121f0c9f882b07f5402ea53e9b2d9455ad34468a094ffab1d64df26c82e1279734c0d42bc2e113eae7b581fbc3be52f3ed4a2d7450d11793afcf406
2020-12-02 09:37:37 +01:00
MarcoFalke
607c844f37 Merge #20540: test: Fix wallet_multiwallet issue on windows
fada2dfcac test: Fix wallet_multiwallet issue on windows (MarcoFalke)

Pull request description:

  The error message on windows:

  > 2020-11-30T18:10:47.536032Z ListWalletDir: Error scanning C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20201130_181042\wallet_multiwallet_0\node0\regtest\wallets\self_walletdat_symlink: boost::filesystem::status: The name of the file cannot be resolved by the system: "C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20201130_181042\wallet_multiwallet_0\node0\regtest\wallets\self_walletdat_symlink\wallet.dat"

ACKs for top commit:
  promag:
    Code review ACK fada2dfcac. Although it could ignore (don't log) directories that lead to no permission error.
  fanquake:
    ACK fada2dfcac

Tree-SHA512: b475162cc3cd1574209d916605b229a79c8089714295f5e16569b71f958f0007d54dc76833938492d931387784588b11b73e3ef00f963540af42c079417f8d72
2020-12-02 08:31:02 +01:00
fanquake
80d4231e16 Merge #19980: refactor: Some wallet cleanups
9b74461fa2 refactor: Assert before dereference in CWallet::GetDatabase (João Barbosa)
021feb3187 refactor: Drop redudant CWallet::GetDBHandle (João Barbosa)

Pull request description:

ACKs for top commit:
  achow101:
    Code Review ACK 9b74461fa2
  meshcollider:
    utACK 9b74461fa2
  ryanofsky:
    Code review ACK 9b74461fa2. Changes since last review: rebasing due to conflict, dropping wallet path commit c6a5cd7a64c78b162f545a3467d0fea7dcaadfcc as suggested in discussion, making GetDatabase() const in the earlier commit. Giving more descriptive title like

Tree-SHA512: 68cf3b5e9fe0acb3a5cd081086629989f213f1904cc344e5775767b56759a7d905b1e1c303afbe40f172ff81bf07f3719b59d8f6ec2de3fdd53cd0e2d220fb25
2020-12-02 08:23:00 +08:00
Fabian Jahr
1e62350ca2 refactor: Improve use of explicit keyword 2020-12-01 18:36:39 +01:00
Fabian Jahr
c502a6dbfb lint: Use c++17 std in cppcheck linter 2020-12-01 18:32:26 +01:00
MarcoFalke
f17e8ba3a1 Merge #20207: Follow-up extra comments on taproot code and tests
2d8099c713 Mention units of MAX_STANDARD_ policy constants (Pieter Wuille)
84e29c7c01 Mention in validation that IsWitnessStandard tests for P2TR (Pieter Wuille)
f867cbcc26 Clean up assets test minimizer LDFLAGS (Pieter Wuille)
ea0e78677b Document additional IsWitnessStandard behavior (Pieter Wuille)
6040de9a46 Add comments on CPubKey::IsValid (Pieter Wuille)
8dbb7de67c Add comments to VerifyTaprootCommitment (Pieter Wuille)
cdf900cbf2 Document need_vin_vout_mismatch argument to make_spender (Pieter Wuille)
18246ed5f0 Fix and improve taproot_construct comments (Pieter Wuille)

Pull request description:

  Addressing some review comments raised here: https://github.com/bitcoin/bitcoin/pull/19953#pullrequestreview-512238027 and https://github.com/bitcoin/bitcoin/pull/19953#pullrequestreview-513499921

ACKs for top commit:
  jonatack:
    ACK 2d8099c per `git range-diff 5009159 4f10965 2d8099c`
  ariard:
    ACK 2d8099c, only changes are comment light improvements on IsValid/IsWitnessStandard.

Tree-SHA512: c4881546c379ea8efc7ef99a43cbf3b9cd3f9dde5fd97a07ee66f2b593c78aef0bd8784853c5c9c737b66c269241a1048bbbdd6c964a3d872efd8ba0ec410b68
2020-12-01 15:11:51 +01:00
practicalswift
8c09c0c1d1 fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime() 2020-12-01 13:18:34 +00:00
MarcoFalke
dfd0b70088 Merge #20425: fuzz: Make CAddrMan fuzzing harness deterministic
17a5f172fa fuzz: Make addrman fuzzing harness deterministic (practicalswift)

Pull request description:

  Make `CAddrMan` fuzzing harness deterministic.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  Crypt-iQ:
    utACK 17a5f172fa

Tree-SHA512: 725f983745233e9b616782247fa18847e483c074ca4336a5beea8a9009128c3a74b4d50a12662d8ca2177c2e1fc5fc121834df6b459ac0af43c931d77ef7c4d8
2020-12-01 14:04:10 +01:00
MarcoFalke
fada2dfcac test: Fix wallet_multiwallet issue on windows 2020-12-01 13:38:23 +01:00
Jonas Schnelli
277c225b84 Merge #20478: Don't set BDB flags when configuring without
982e548a9a Don't set BDB flags when configuring without (Jonas Schnelli)

Pull request description:

  Configuring `--without-bdb` on MacOS leads to a compile error (when BerkeleyDB is not installed). `brew --prefix berkeley-db4` always reports the target directory (even if not installed).

  This PR prevents BDB_CFLAGS (et al) from being populated when configuring `--without-bdb`

  ```
  ld: warning: directory not found for option '-L/Users/user/Documents/homebrew/Cellar/berkeley-db@4/4.8.30/lib'
  ld: warning: directory not found for option '-L/Users/user/Documents/homebrew/Cellar/berkeley-db@4/4.8.30/lib'
  ld: library not found for -ldb_cxx-4.8
  ld: library not found for -ldb_cxx-4.8
  ```

ACKs for top commit:
  promag:
    Tested ACK 982e548a9a.
  hebasto:
    ACK 982e548a9a, tested on macOS 11 Big Sur.

Tree-SHA512: f8ca0adca0e18e2de4c0f99d5332cba70d957a9d31a357483b43dcf61c2ed4749d223eabadd45fdbf3ef0781c6b37217770e9aa935b5207eaf7f87c5bdfe9e95
2020-12-01 11:28:27 +01:00
Jonas Schnelli
dcb7518067 Merge #20496: build: Drop unneeded macOS framework dependencies
ec4a46dd9c build: Drop unneeded IOKit framework dependency (Hennadii Stepanov)
65afe4cb69 build: Drop unneeded ApplicationServices framework dependency (Hennadii Stepanov)

Pull request description:

  Bitcoin Core codebase does not contain direct dependencies on the `ApplicationServices` and `IOKit` frameworks.

ACKs for top commit:
  jonasschnelli:
    utACK ec4a46dd9c
  practicalswift:
    cr ACK ec4a46dd9c: patch looks correct!
  promag:
    Tested ACK ec4a46dd9c (not depends build).

Tree-SHA512: 47b5ad87d761992850133a921f07d485565c70ba2909a3289050f406e6dbd39ad49e1aeeb6cad79c6914385a72ddffd273dfadd69259a35545a13cd17d0e5043
2020-12-01 11:27:06 +01:00
Jonas Schnelli
335d27d081 Merge #20512: doc: Add bash as an OpenBSD dependency
1d578c078f doc: Add bash as an OpenBSD dependency (Emil Engler)

Pull request description:

  If we require Python for the test framework, we should also require
  bash. It is required for the linters and other scripts and does not
  comes in a default OpenBSD installation.

ACKs for top commit:
  practicalswift:
    ACK 1d578c078f
  RiccardoMasutti:
    ACK 1d578c0
  theStack:
    ACK 1d578c078f

Tree-SHA512: ef14e801983093a99d4c28d134adbc589ca06e5437bf1ff34f8e593968c59793e9d99e8a48f577f847acdfc5185e193276526894b4c632c59d66d87939977910
2020-12-01 11:23:23 +01:00
Jonas Schnelli
f2a673f15b Merge bitcoin-core/gui#137: refactor: Replace deprecated Qt::SystemLocale{Short,Long}Date
86b1ab64b1 refactor: Replace deprecated Qt::SystemLocale{Short,Long}Date (Hennadii Stepanov)

Pull request description:

  As all deprecated warning in Qt 5.15.0 were eliminated in #46, Qt 5.15.1 introduced another one that is fixed in this PR.

  Required for https://github.com/bitcoin/bitcoin/pull/20182.

  Details in Qt docs:
  - https://doc.qt.io/qt-5/qdatetime.html#toString-1
  - https://doc.qt.io/qt-5/qdate.html#toString-1

ACKs for top commit:
  jarolrod:
    Tested ACK 86b1ab6 on MacOS 10.15.7 and Arch Linux both with Qt 5.15.1
  jonasschnelli:
    Tested ACK 86b1ab64b1

Tree-SHA512: 1dbba8ee70c895bf58317172a9901cdbe5503b1d6258f51caaae88d88d332d9fbd4697c995192d31e3618ddfd532c5f5881289b3af1184422e5a9263a1224115
2020-12-01 10:59:09 +01:00
MarcoFalke
cd720337fe Merge #20222: refactor: CTxMempool constructor clean up
f15e780b9e refactor: Clean up CTxMemPool initializer list (Elle Mouton)
e3310692d0 refactor: Make CTxMemPool::m_check_ratio a const and a constructor argument (Elle Mouton)
9d4b4b2c2c refactor: Avoid double to int cast for nCheckFrequency (Elle Mouton)

Pull request description:

  This PR cleans up the CTxMemPool interface by including the ratio used to determine when a mempool sanity check should run in the constructor of CTxMempool instead of using nCheckFrequency which required a cast from a double to a uint32_t. Since nCheckFrequency (now called m_check_ratio) is set in the constructor and only every read from there after, it can be turned into a const and no longer needs to be guarded by the 'cs' lock.

  Since nCheckFrequency/m_check_ratio no longer needs to lock the 'cs' mutux, mutex lock line in the "CTxMempool::check" function can be moved below where the m_check_ratio variable is checked. Since the variable is 0 by default (meaning that "CTxMempool::check" will most likely not run its logic) this saves us from unnecessarily grabbing the lock.

ACKs for top commit:
  jnewbery:
    utACK f15e780b9e
  MarcoFalke:
    ACK f15e780b9e 👘
  glozow:
    utACK f15e780b9e
  theStack:
    Code Review ACK f15e780b9e

Tree-SHA512: d83f3b5311ca128847b621e5e999c7e1bf0f4e6261d4cc090fb13e229a0f7eecd66ad997f654f50a838baf708d1515740aa3bffc244909a001d01fd5ae398b68
2020-12-01 10:02:56 +01:00
MarcoFalke
24e4857b29 Merge #20494: refactor: Move node and wallet code out of src/interfaces
629a9299b2 Move WalletImpl from interfaces/wallet.cpp to wallet/interfaces.cpp (Russell Yanofsky)
2a26771d81 Move ChainImpl from interfaces/chain.cpp to node/interfaces.cpp (Russell Yanofsky)
12bd0fc9d7 Move NodeImpl from interfaces/node.cpp to node/interfaces.cpp (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  Move `NodeImpl` from `interfaces/node.cpp` to `node/interfaces.cpp`
  Move `ChainImpl` from `interfaces/chain.cpp` to `node/interfaces.cpp`
  Move `WalletImpl` from `interfaces/wallet.cpp` to `wallet/interfaces.cpp`

  No changes to any classes (can review with `git diff --color-moved=dimmed_zebra`)

  Motivation for this change is to move node and wallet code to respective directories where it might fit in better than `src/interfaces/`, but also to remove all unnecessary code from `src/interfaces/` to unblock #19160 review, which has been hung up partially because of code organization. Building on top of this PR, #19160 should now be able to organize interface implementations more understandably in `src/node/` `src/wallet/` `src/ipc/` and `src/init/` directories instead of having so much functionality all in `src/interfaces/`

ACKs for top commit:
  promag:
    Code review ACK 629a9299b2.
  MarcoFalke:
    review ACK 629a9299b2 🔺

Tree-SHA512: 87c2b8fd51519bbd4e5ad3539a79debcf88c3bf021eb28c63f3f555186538b62a0c4cc1a3f07cfb4ff13aea8b0b2fdde505d81f22a5e5fd12a6e375b55a92ab8
2020-12-01 09:39:56 +01:00
MarcoFalke
ffd5e7a856 Merge #20519: Handle rename failure in DumpMempool(...) by using the RenameOver(...) return value. Add [[nodiscard]] to RenameOver(...).
ce9dd45422 Add [[nodiscard]] to RenameOver(...) (practicalswift)
9429a398e2 Handle rename failure in DumpMempool(...) by using RenameOver(...) return value (practicalswift)

Pull request description:

  Handle rename failure in `DumpMempool(...)` by using the `RenameOver(...)` return value.

  Add `[[nodiscard]]` to `RenameOver(...)` to reduce the risk of similar rename issues in the future.

ACKs for top commit:
  vasild:
    ACK ce9dd454
  theStack:
    ACK ce9dd45422 🏷️

Tree-SHA512: 1e63d7f3061e1f6ea2df5750dbc1547a39bd50b6c529812a0c8a0c11d3100c241afdf14094e69b69a38bade7e54a12b2a42888545874398eaf5d02421b57e874
2020-12-01 08:28:34 +01:00
Andrew Chow
3e69939b78 Fail if maximum weight is too large
Our max weight check in CreateTransaction only worked if the transaction
was fully signed. However if we are funding a transaction, it is
possible that the tx weight will be too large for a standard tx. In that
case, we should also fail. So we use the tx weight returned by
CalculateMaximumSignedTxSize and check against the limit for those
transactions.
2020-11-30 16:39:20 -05:00
Andrew Chow
51e2cd322c Have CalculateMaximumSignedTxSize also compute tx weight 2020-11-30 16:38:07 -05:00
MarcoFalke
fa275e1539 test: Fix intermittent feature_taproot issue 2020-11-30 20:45:09 +01:00
Adam Jonas
86e6add5ca doc: rename CODEOWNERS to REVIEWERS
This reverts #18949 since CODEOWNERS require write permission.
Instead the REVIEWERS file uses DrahtBot to tag reviewers in PR
comments.
2020-11-30 13:53:50 -05:00
MarcoFalke
81d5af42f4 Merge #20499: Remove obsolete NODISCARD ifdef forest. Use [[nodiscard]] (C++17).
79bff8e48a Remove NODISCARD (practicalswift)
4848e71107 scripted-diff: Use [[nodiscard]] (C++17) instead of NODISCARD (practicalswift)

Pull request description:

  Remove obsolete `NODISCARD` `ifdef` forest. Use `[[nodiscard]]` (C++17).

ACKs for top commit:
  theStack:
    ACK 79bff8e48a
  fanquake:
    ACK 79bff8e48a

Tree-SHA512: 56dbb8e50ed97ecfbce28cdc688a01146108acae49a943e338a8f983f7168914710d36e38632f6a7c200ba6c6ac35b2519e97d6c985e8e7eb23223f13bf985d6
2020-11-30 15:42:36 +01:00
fanquake
817aeca57a Merge #20491: refactor: Drop noop gcc version checks
830ddf4139 Drop noop gcc version checks (Hennadii Stepanov)

Pull request description:

  Since #20413 the minimum required GCC version is 7.

ACKs for top commit:
  fanquake:
    ACK 830ddf4139

Tree-SHA512: 36264661d6ced1683a0c907efba7c700502acaf8e9fd50d9066bc9c7b877b25165b0684c2d7fe74bd58e500a77d7702bdbdd53691c274f29e4abccd241c10964
2020-11-30 16:10:41 +08:00
fanquake
b685f60a08 build: mac_alias 2.1.1 2020-11-30 15:39:09 +08:00
fanquake
5d2cbdf772 macdeploy: use Python 3.6 2020-11-30 14:54:20 +08:00
fanquake
a42aa94c54 macdeploy: remove runHDIUtil in favor of directly calling subprocess.run 2020-11-30 14:54:20 +08:00
fanquake
adaa26202b macdeploy: remove existing Bitcoin-Core.dmg if present 2020-11-30 14:54:20 +08:00
fanquake
ccb0325b1b macdeploy: move qt_conf to where it's used 2020-11-30 14:54:20 +08:00
fanquake
6390a04862 macdeploy: consolidate .DS_Store generation
Rather than two lots of logic doing roughly the same thing, dependent on if
you're compiling on Linux or macOS, combine the .DS store generation into
macdeployqtplus.

This also removes the -fancy and -volname options.
2020-11-30 14:54:19 +08:00
fanquake
32347cd56a macdeploy: assume plistlib is available
We already require Python 3.5 or later
2020-11-30 14:54:19 +08:00
fanquake
0ab4018c12 macdeploy: have a single level of logging output
4 different levels of verbosity is overkill for a fairly simple script, which
was always being run at 2 in any case.
2020-11-30 14:54:19 +08:00
fanquake
827d382aa7 macdeploy: remove add-resources argument 2020-11-30 14:54:19 +08:00
fanquake
464b34d4c3 macdeploy: remove codesigning argument 2020-11-30 14:54:19 +08:00
fanquake
4d70d3d7fe build: automatically determine macOS translations
Rather than using OSX_QT_TRANSLATIONS which must be manually updated,
and we forget to update anyway, i.e: #19059, automatically find and copy
available translations from the translations directory.
2020-11-30 14:54:18 +08:00
fanquake
2e1336dbfe Merge #20471: build: use C++17 in depends
2f5dfe4a7f depends: build qt in c++17 mode (fanquake)
104e859c97 builds: don't pass -silent to qt when building in debug mode (fanquake)
e2c500636c depends: build zeromq with -std=c++17 (fanquake)
2374f2fbef depends: build Boost with -std=c++17 (fanquake)
2dde55702d depends: build bdb with -std=c++17 (fanquake)

Pull request description:

  In packages where we are passing `-std=c++11` switch to `-std=c++17`, or, `-std=c++1z` in the case of Qt.

  This PR also contains a [commit](104e859c97) that improves debug output when building Qt for debugging (`DEBUG=1`).

  Now we'll get output like this:
  ```bash
  g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 -fno-exceptions <lots more> ../../corelib/kernel/qcoreapplication.cpp
  ```
  rather than just:
  ```bash
  compiling ../../corelib/kernel/qcoreapplication.cpp
  ```

  Note that when you look at the DEBUG output for these changes when building Qt, you'll see objects being compiled with a mix of C++11 and C++17. The breakdown is roughly:

  1. `qmake` built with `-std=c++11`:
  ```bash
  Creating qmake...
  make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/qmake'
  g++ -c -o project.o   -std=c++11 -ffunction-sections -O2 -g <trim> <trim>/qt/5.9.8-4110fa99945/qtbase/qmake/project.cpp

  # when qmake, Qt also builds some of it's corelib, such as corelib/global/qmalloc.cpp
  g++ -c -o qmalloc.o   -std=c++11 -ffunction-sections -O2 -g <trim> <trim>/qt/5.9.8-4110fa99945/qtbase/src/corelib/global/qmalloc.cpp
  ```

  2. `qmake` is run, and passed our build options, including `-c++std`:
  ```bash
  make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase'
  <trim>qt/5.9.8-4110fa99945/qtbase/bin/qmake -o Makefile qtbase.pro -- -bindir <trim>/native/bin -c++std c++1z -confirm-license <trim>
  ```

  3. After some cleaning and configuring, we actually start to build Qt, as well as it's tools and internal libs:
  ```bash
  Building qt...
  make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/src'

  # build libpng, zlib etc
  gcc -c -m64 -pipe -pipe -O1 <trim> -o .obj/png.o png.c

  # build libQt5Bootstrap, using C++11, which again compiles qmalloc.cpp
  make[2]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qtbase/src/tools/bootstrap'
  g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 <trim> -o .obj/qmalloc.o ../../corelib/global/qmalloc.cpp

  # build a bunch of tools like moc, rcc, uic, qfloat16-tables, qdbuscpp2xml, using C++11
  g++ -c -pipe -O2 -std=c++11 -fno-exceptions -Wall -W <trim> -o .obj/rcc.o rcc.cpp

  # from here, Qt is compiled with -std=c++1z, including qmalloc.cpp, for the third and final time:
  g++ -c -include .pch/Qt5Core <trim> -g -Og -fPIC -std=c++1z -fvisibility=hidden <trim> -o .obj/qmalloc.o global/qmalloc.cpp
  ```

  4.  Finally, build tools like `lrelease`, `lupdate`, etc, but back to using -std=c++11
  ```bash
  make[1]: Entering directory '<trim>/qt/5.9.8-4110fa99945/qttools/src/linguist/lrelease'
  g++ -c -pipe -O2 -std=c++11 -fno-exceptions -Wall -W <trim> -o .obj/translator.o ../shared/translator.cpp
  ```

  If you dump the debug info from the built Qt libs, they should also tell you that they were compiled with `C++17`:
  ```bash
  objdump -g bitcoin/depends/x86_64-pc-linux-gnu/lib/libQt5Core.a
  GNU C++17 9.3.0 -m64 -mtune=generic -march=x86-64 -g -O1 -Og -std=c++17 -fPIC -fvisibility=hidden -fvisibility-inlines-hidden -fasynchronous-unwind-tables -fstack-protector-strong -fstack-clash-protection -fcf-protection
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 2f5dfe4a7f
  practicalswift:
    cr ACK 2f5dfe4a7f: patch looks correct
  fjahr:
    Code review ACK 2f5dfe4a7f
  hebasto:
    ACK 2f5dfe4a7f, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: fc5e9d7c7518c68349c8228fb1aead829850373efc960c9b8c079096a83d1dad19c62a9730fce5802322bf07e320960fd47851420d429eda0a87c307f4e8b03a
2020-11-30 12:23:23 +08:00
Luke Dashjr
415fb2e1ab GUI/Intro: Move prune setting below explanation 2020-11-30 02:46:43 +00:00
Luke Dashjr
2a84c6bcf6 GUI/Intro: Estimate max age of backups that can be restored with pruning 2020-11-30 02:46:43 +00:00
Tyler Chambers
6690adba08 Warn when binaries are built from a dirty branch.
Adjusted version flag behavior in bitcoin-tx, bitcoin-wallet, and
bitcoind to match. Added functionality in gen-manpages.sh to warning when
attempting to generate man pages for binaries built from a dirty
branch.
2020-11-28 18:43:55 -05:00
MarcoFalke
7ae86b3c68 Merge #20522: [test] Fix sync issue in disconnect_p2ps
3ebde2143a [test] Fix wait condition in disconnect_p2ps (Amiti Uttarwar)

Pull request description:

  #19315 currently has a [test failure](https://cirrus-ci.com/task/4545582645641216) because of a race. `disconnect_p2ps` is intended to have a `wait_until` clause that prevents this race, but the conditional doesn't match since its comparing two different object types. `MY_SUBVERSION` is defined in messages.py as a byte string, but is compared to the value returned by the RPC. This PR simply converts types to ensure they match, which should prevent the race from occurring.

  HUGE PROPS TO jnewbery for discovering the issue 🔎

ACKs for top commit:
  jnewbery:
    ACK 3ebde2143a
  glozow:
    Code review ACK 3ebde2143a

Tree-SHA512: ca096b80a3e4d757a645f38846d6dc89d6a3d35c3435513a72d278e305faddd4aff9e75a767941b51b2abbf59c82679bac1e9a0140d6f285efe3053e51bcc2a8
2020-11-28 18:06:43 +01:00
MarcoFalke
854b36cfa2 Merge bitcoin-core/gui#138: unlock encrypted wallet "OK" button bugfix
8008ef770f qt: unlock wallet "OK" button bugfix (Michael Dietz)

Pull request description:

  When trying to send a transaction from an encrypted wallet, the ask
  passphrase dialog would not allow the user to click the "OK" button
  and proceed. Therefore it was impossible to send a transaction
  through the gui. It was not enabling the "OK" button after the
  passphrase was entered by the user, because it was using the same
  form validation logic as the "Change passphrase" flow.

  I reported this in a comment in https://github.com/bitcoin-core/gui/issues/136. But then I realized this seems to be a flat out bug.

ACKs for top commit:
  MarcoFalke:
    review ACK 8008ef770f
  hebasto:
    ACK 8008ef770f, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: cc09b34c7f3aea09729e1c7ccccff05dc11fec56fee2ad369f2d862979572b1edd8b7e738ffe6e91d35d071b819b0c3e0f5d48bf5e27427a80af4a28893f8aaf
2020-11-28 10:39:32 +01:00
MarcoFalke
1ae5758981 Merge #20448: RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint wallet
89bdad5b25 RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint (Luke Dashjr)

Pull request description:

  Allow specifying the `wallet_name` param to `unloadwallet` on RPC wallet endpoints, so long as it matches the endpoint wallet.

ACKs for top commit:
  jonatack:
    ACK 89bdad5b25
  MarcoFalke:
    review ACK 89bdad5b25

Tree-SHA512: efb399c33f7b5596870a26a8680f453ca47aa7a6db4e550f9435d13044f1c4bad0ae11e8f0205213409d08b75c4188c3be782e54aafab1f65b97eb8cf5c252a9
2020-11-28 10:14:45 +01:00
Amiti Uttarwar
3ebde2143a [test] Fix wait condition in disconnect_p2ps
MY_SUBVERSION is defined in messages.py as a byte string, but here we were
comparing this value to the value returned by the RPC. Convert to ensure the
types match.
2020-11-27 14:54:55 -08:00
Michael Dietz
8008ef770f qt: unlock wallet "OK" button bugfix
When trying to send a transaction from an encrypted wallet, the ask
passphrase dialog would not allow the user to click the "OK" button
and proceed. Therefore it was impossible to send a transaction
through the gui. It was not enabling the "OK" button after the
passphrase was entered by the user, because it was using the same
form validation logic as the "Change passphrase" flow.
2020-11-27 14:18:50 -06:00
practicalswift
ce9dd45422 Add [[nodiscard]] to RenameOver(...) 2020-11-27 12:41:26 +00:00
practicalswift
9429a398e2 Handle rename failure in DumpMempool(...) by using RenameOver(...) return value 2020-11-27 12:41:07 +00:00
MarcoFalke
e2ff5e7b35 Merge #20497: [Refactor] Add MAX_STANDARD_SCRIPTSIG_SIZE to policy
e416cfc92b Add MAX_STANDARD_SCRIPTSIG_SIZE to policy (sanket1729)

Pull request description:

  Bitcoin core has a standardness rule for max satisfaction script sig size.
  This PR adds to the policy header file so that it is documented along with
  along policy rules. The initial reasoning that 1650 is an implicit
  limit(would not reach assuming all other policy rules are being
  followed) is outdated.

  As we now know, bitcoin transactions can have spend conditions are more than
  just signatures and there may exist p2sh transactions involving 100 byte
  preimages that maybe non-standard because of this rule. Because this
  rule is no longer implicit, we should explicitly document it in policy
  header file

ACKs for top commit:
  sipa:
    utACK e416cfc92b
  practicalswift:
    cr ACK e416cfc92b
  theStack:
    Code Review ACK e416cfc92b

Tree-SHA512: 1a91ee23dfb6085807e04dd0687d7a443e0f3e0f52d0a995a6599dff28533b0b599afba2724735d93948a64a3e25d0bc016ce3e771c0bd453eef78b22dc2369d
2020-11-27 10:59:24 +01:00
Hennadii Stepanov
86b1ab64b1 refactor: Replace deprecated Qt::SystemLocale{Short,Long}Date 2020-11-27 10:04:13 +02:00
Pieter Wuille
2d8099c713 Mention units of MAX_STANDARD_ policy constants 2020-11-26 14:56:25 -08:00
Pieter Wuille
84e29c7c01 Mention in validation that IsWitnessStandard tests for P2TR 2020-11-26 14:56:25 -08:00
Pieter Wuille
f867cbcc26 Clean up assets test minimizer LDFLAGS 2020-11-26 14:56:25 -08:00
Pieter Wuille
ea0e78677b Document additional IsWitnessStandard behavior 2020-11-26 14:56:25 -08:00
Pieter Wuille
6040de9a46 Add comments on CPubKey::IsValid 2020-11-26 14:56:25 -08:00
Pieter Wuille
8dbb7de67c Add comments to VerifyTaprootCommitment 2020-11-26 14:56:25 -08:00
Pieter Wuille
cdf900cbf2 Document need_vin_vout_mismatch argument to make_spender 2020-11-26 14:56:25 -08:00
Pieter Wuille
18246ed5f0 Fix and improve taproot_construct comments 2020-11-26 14:56:25 -08:00
Emil Engler
1d578c078f doc: Add bash as an OpenBSD dependency
If we require Python for the test framework, we should also require
bash. It is required for the linters and other scripts and does not
comes in a default OpenBSD installation.
2020-11-26 21:34:25 +01:00
Vasil Dimov
db058efeb0 sync: use HasReason() in double lock tests
`HasReason()` is shorter than a lambda function.
2020-11-26 14:42:06 +01:00
Vasil Dimov
a21dc469cc sync: const-qualify the argument of double_lock_detected()
It is not modified in the function, so should be `const`.
2020-11-26 14:42:00 +01:00
Vasil Dimov
6d3689fcf6 sync: print proper lock order location when double lock is detected
Before:
```
Assertion failed: detected double lock at src/sync.cpp:153, details in debug log.
```

After:
```
Assertion failed: detected double lock for 'm' in src/test/sync_tests.cpp:40 (in thread ''), details in debug log.
```
2020-11-26 14:41:53 +01:00
practicalswift
79bff8e48a Remove NODISCARD 2020-11-26 09:07:33 +00:00
practicalswift
4848e71107 scripted-diff: Use [[nodiscard]] (C++17) instead of NODISCARD
-BEGIN VERIFY SCRIPT-
sed -i "s/NODISCARD/[[nodiscard]]/g" $(git grep -l "NODISCARD" ":(exclude)src/bench/nanobench.h" ":(exclude)src/attributes.h")
-END VERIFY SCRIPT-
2020-11-26 09:05:59 +00:00
sanket1729
e416cfc92b Add MAX_STANDARD_SCRIPTSIG_SIZE to policy
Bitcoin core has a standardness rule for max satisfaction script sig size.
This PR adds to the policy header file so that it is documented along with
along policy rules. The initial reasoning that 1650 is an implicit
limit(would not reached assuming all other policy rules are being
followed) is outdated.

As we now know, bitcoin transactions can have spend conditions are more than
just signatures and there may exist p2sh transactions involving 100 byte
preimages that maybe non-standard because of this rule. Because this
rule is no longer implicit, we should explicitly document it in policy
header file
2020-11-25 14:04:39 -06:00
Hennadii Stepanov
ec4a46dd9c build: Drop unneeded IOKit framework dependency 2020-11-25 18:25:52 +02:00
Hennadii Stepanov
65afe4cb69 build: Drop unneeded ApplicationServices framework dependency 2020-11-25 18:18:36 +02:00
Wladimir J. van der Laan
50091592dd Merge #19337: sync: detect double lock from the same thread
95975dd08d sync: detect double lock from the same thread (Vasil Dimov)
4df6567e4c sync: make EnterCritical() & push_lock() type safe (Vasil Dimov)

Pull request description:

  Double lock of the same (non-recursive) mutex from the same thread would produce an undefined behavior. Detect this from `DEBUG_LOCKORDER` and react similarly to the deadlock detection.

  This came up during discussion in another, related PR: https://github.com/bitcoin/bitcoin/pull/19238#discussion_r442394521.

ACKs for top commit:
  laanwj:
    code review ACK 95975dd08d
  hebasto:
    re-ACK 95975dd08d

Tree-SHA512: 375c62db7819e348bfaecc3bd82a7907fcd8f5af24f7d637ac82f3f16789da9fc127dbd0e37158a08e0dcbba01a55c6635caf1d8e9e827cf5a3747f7690a498e
2020-11-25 17:02:20 +01:00
MarcoFalke
19b8071eae Merge #20489: CI msvc: only build vcpkg dependencies for release (not debug) to reduce build times
fa18e7cbc5 This change to the appveyor CI config for msvc builds reverses a change introduced in #19960. It re-applies a setting to inform vcpkg to only build release vesions of the dependencies rather than the default of debug and release. (Aaron Clauson)

Pull request description:

  This change to the appveyor CI config for msvc builds reverses a change introduced in #19960. It re-applies a setting to inform vcpkg to only build release versions of the dependencies rather than the default of debug and release.

  It had been expected that the vcpkg manifest mechanism introduced in #19960 would do this automatically but it turns out not to be the case.

ACKs for top commit:
  MarcoFalke:
    ACK fa18e7cbc5 if green
  hebasto:
    ACK fa18e7cbc5, AppVeyor build takes 5 minutes less.

Tree-SHA512: 427e7e78190c20e0d85dad9b29beed2b6fa13f99c6bc72bcc1839dfb51237a7cc785ab707b4f851c527c1bb0d3e7ebad9e640969e19d29778584bbaeec75cecf
2020-11-25 16:31:43 +01:00
MarcoFalke
5c0aebfcd4 Merge #19387: span: update constructors to match c++20 draft spec and add lifetimebound attribute
e3e7446305 Add lifetimebound to attributes for general-purpose usage (Cory Fields)
1d58cc7cb0 span: add lifetimebound attribute (Cory Fields)
62733fee87 span: (almost) match std::span's constructor behavior (Cory Fields)

Pull request description:

  Replaces #19382 with a different approach. See [this comment](https://github.com/bitcoin/bitcoin/pull/19382#discussion_r446332852) for the reasoning behind the switch.

  --

  Description from #19382:

  See [here](http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0936r0.pdf) for more detail on lifetimebound.

  This is implemented using preprocesor macros rather than configure checks in order to keep span.h self-contained.

  The ```[[clang::lifetimebound]]``` syntax was chosen over ```__attribute__((lifetimebound))``` because the former is more flexible and works to guard ```this``` as well as function parameters, and also because at least for now, it's available only in clang.

  There are currently no violations in our codebase, but this can easily be tested by inserting one like this somewhere and compiling with a modern clang:
  ```c++
  Span<const int> bad(std::vector<int>{1,2,3});
  ```

  The result:
  > warning: temporary whose address is used as value of local variable 'bad' will be destroyed at the end of the full-expression [-Wdangling]
      Span<const int> bad(std::vector<int>{1,2,3});
  ```

ACKs for top commit:
  sipa:
    ACK e3e7446305
  ajtowns:
    ACK e3e7446305 (drive by; only a quick skim of code and some basic sanity checks)
  MarcoFalke:
    review ACK e3e7446305 🔗
  jonatack:
    ACK e3e7446 change since last review is adding `[[clang::lifetimebound]]` as `LIFETIMEBOUND` to src/attributes.h as suggested in https://github.com/bitcoin/bitcoin/pull/19387#issuecomment-650752959.

Tree-SHA512: 05a3440ee595ef0e8d693a2820b360707695c016a68e15df47c20cd8d053646cc6c8cca8addd7db40e72b3fce208879a41c8102ba7ae9223e4366e5de1175211
2020-11-25 15:18:33 +01:00
Hennadii Stepanov
8963b2c71f qt: Improve comments in WalletController::getOrCreateWallet() 2020-11-25 16:12:33 +02:00
Hennadii Stepanov
5fcfee68af qt: Call setParent() in the parent's context
setParent(parent) calls sendEvent(parent, QChildEvent) that implies
running in the thread which created the parent object.
2020-11-25 16:12:32 +02:00
Hennadii Stepanov
5659e73493 qt: Add ObjectInvoke template function
This is a replacement of the QMetaObject::invokeMethod functor overload
which is available in Qt 5.10+.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-11-25 16:12:23 +02:00
Dmitry Petukhov
89895773b7 Fix length of R check in test/key_tests.cpp:key_signature_tests
The code before the fix only checked the length of R value of the last
signature in the loop, and only for equality (but the length can be
less than 32)

The fixed code checks that length of the R value is less than or equal
to 32 on each iteration of the loop

The BOOST_CHECK(sig.size() <= 70) is merged with sig[3] <= 32 check,
and BOOST_CHECKs are moved outside the loop, for efficiency
2020-11-25 18:07:37 +05:00
Hennadii Stepanov
830ddf4139 Drop noop gcc version checks
Since #20413 the minimum required GCC version is 7.

Co-authored-by: practicalswift <practicalswift@users.noreply.github.com>
2020-11-25 14:38:33 +02:00
MarcoFalke
afdfd3c8c1 Merge #20403: wallet: upgradewallet fixes, improvements, test coverage
3eb6f8b2e6 wallet (not for backport): improve upgradewallet error messages (Jon Atack)
ca8cd893bb wallet: fix and improve upgradewallet error responses (Jon Atack)
99d56e3571 wallet: fix and improve upgradewallet result responses (Jon Atack)
2498b04ce8 Don't upgrade to HD split if it is already supported (Andrew Chow)
c46c18b788 wallet: refactor GetClosestWalletFeature() (Jon Atack)

Pull request description:

  This follows up on #18836 and #20282 to fix and improve the as-yet unreleased `upgradewallet` feature and also implement review follow-up in https://github.com/bitcoin/bitcoin/pull/18836#discussion_r519328607.

  This PR fixes 4 upgradewallet issues:

  - this bug: https://github.com/bitcoin/bitcoin/pull/20403#discussion_r526063920
  - it returns nothing in the absence of an RPC error, which isn't reassuring for users
  - it returns the same thing both in the case of a successful upgrade and when no upgrade took place
  - the error message object is currently dead code

  This PR fixes the above and provides:

  ...user feedback to not silently return without upgrading
  ```
  {
    "wallet_name": "disable private keys",
    "previous_version": 169900,
    "current_version": 169900,
    "result": "Already at latest version. Wallet version unchanged."
  }
  ```
  ...better feedback after successfully upgrading
  ```
  {
    "wallet_name": "watch-only",
    "previous_version": 159900,
    "current_version": 169900,
    "result": "Wallet upgraded successfully from version 159900 to version 169900."
  }
  ```
  ...helpful error responses
  ```
  {
    "wallet_name": "blank",
    "previous_version": 169900,
    "current_version": 169900,
    "error": "Cannot downgrade wallet from version 169900 to version 159900. Wallet version unchanged."
  }
  {
    "wallet_name": "blank",
    "previous_version": 130000,
    "current_version": 130000,
    "error": "Cannot upgrade a non HD split wallet from version 130000 to version 169899 without upgrading to support pre-split keypool. Please use version 169900 or no version specified."
  }
  ```
  updated help:
  ```
  upgradewallet ( version )

  Upgrade the wallet. Upgrades to the latest version if no version number is specified.
  New keys may be generated and a new wallet backup will need to be made.
  Arguments:
  1. version    (numeric, optional, default=169900) The version number to upgrade to. Default is the latest wallet version.

  Result:
  {                            (json object)
    "wallet_name" : "str",     (string) Name of wallet this operation was performed on
    "previous_version" : n,    (numeric) Version of wallet before this operation
    "current_version" : n,     (numeric) Version of wallet after this operation
    "result" : "str",          (string, optional) Description of result, if no error
    "error" : "str"            (string, optional) Error message (if there is one)
  }
  ```

ACKs for top commit:
  achow101:
    ACK  3eb6f8b
  MarcoFalke:
    review ACK 3eb6f8b2e6 🛡

Tree-SHA512: b767314069e26b5933b123acfea6aa40708507f504bdb22884da020a4ca1332af38a7072b061e36281533af9f4e236d94d3c129daf6fe5b55241127537038eed
2020-11-25 12:46:27 +01:00
Aaron Clauson
fa18e7cbc5 This change to the appveyor CI config for msvc builds reverses a change introduced in #19960. It re-applies a setting to inform vcpkg to only build release vesions of the dependencies rather than the default of debug and release.
It had been expected that the vcpkg manifest mechanism introduced in #19960 would do this automatically but it turns out not to be the case.
2020-11-25 11:07:10 +00:00
Wladimir J. van der Laan
1e9e4b68f3 Merge #20469: build: Avoid secp256k1.h include from system
e95aaefe25 build: Avoid secp256k1.h include from system (Niklas Gögge)

Pull request description:

  While building i ran into an error because i had a version of `secp256k1.h` under `/usr/local/include` that was incompatible with the secp256k1 code in the repository. This caused a problem because `$(BOOST_CPPFLAGS)` contained `-I/usr/local/include` and the include paths are searched by the compiler in order from left to right, so in the end `$(BITCOIN_INCLUDES)` contained `-I/usr/local/include` before `-I$(srcdir)/secp256k1/include` which caused the compiler to find  `secp256k1.h` under `/usr/local/include`.

  Looking at git blame i am wondering how this has not happened to anyone else in several years: cb89e18845/src/Makefile.am (L25)

  I am on macOS 10.15.

ACKs for top commit:
  laanwj:
    Code review ACK e95aaefe25
  hebasto:
    ACK e95aaefe25, tested on macOS 11 Big Sur by adding `#error` into `/usr/local/include/secp256k1.h`.

Tree-SHA512: 1f0b395725936c179ab60dee3582ec7b21e2f9c0f1895e160d84a487cf0db16d0c7aa47d05800e0aded31685b4362056cac9b9ecca1bb8c308a4c5a810e8dc1d
2020-11-25 09:48:47 +01:00
MarcoFalke
efd4cdb81f Merge #20456: test: Fix intermittent issue in mempool_compatibility
fa05d19bd6 test: Fix intermittent issue in mempool_compatibility (MarcoFalke)

Pull request description:

  Fixes https://cirrus-ci.com/task/5141306890518528?command=ci#L6076

  The version is too old to understand getmempoolinfo()[loaded], so it is never called when starting (See https://cirrus-ci.com/task/5141306890518528?command=ci#L5541)

ACKs for top commit:
  achow101:
    ACK fa05d19bd6

Tree-SHA512: e912d5dff6236d2d4ac608f9f3b3e255cc2b611f36d79bfe4e2a940709833a947c682d7703327887e1753eab30b95eb2615d7e2c21ce4bca4f089e717cbb51c4
2020-11-25 08:19:59 +01:00
MarcoFalke
ca4a784942 Merge #20410: wallet: Do not treat default constructed types as None-type
fa69c2c784 wallet: Do not treat default constructed types as None-type (MarcoFalke)
fac4e136fa refactor: Change pointer to reference because it can not be null (MarcoFalke)

Pull request description:

  Equating `0==None` and `""==None` is confusing, unneeded and undocumented

ACKs for top commit:
  jonatack:
    ACK fa69c2c784
  achow101:
    ACK fa69c2c784
  Sjors:
    tACK fa69c2c784 modulo `unset`

Tree-SHA512: c4c8d0ad80c6697621d356a9545caf28ca2facc82bb2fa8e70eceb52372d25f0685237c73688c4b01da0e75d213c77c0d45011a8bdfe81ea783d85f045786dac
2020-11-25 08:02:19 +01:00
Jonas Schnelli
c23f6f84ef Add depends qt fix for ARM macs 2020-11-24 21:00:34 +01:00
Russell Yanofsky
629a9299b2 Move WalletImpl from interfaces/wallet.cpp to wallet/interfaces.cpp 2020-11-24 10:20:16 -05:00
Russell Yanofsky
2a26771d81 Move ChainImpl from interfaces/chain.cpp to node/interfaces.cpp
No changes to ChainImpl or any related classes (review with `git diff --color-moved=dimmed_zebra`)
2020-11-24 10:13:23 -05:00
Russell Yanofsky
12bd0fc9d7 Move NodeImpl from interfaces/node.cpp to node/interfaces.cpp 2020-11-24 10:13:23 -05:00
Jonas Schnelli
982e548a9a Don't set BDB flags when configuring without 2020-11-24 15:08:28 +01:00
fanquake
31c9987976 Merge #20447: depends: Patch qt_intersect_spans to avoid non-deterministic behavior in LLVM 8
8f7d1b39ef Fix QPainter non-determinism on macOS (Andrew Chow)

Pull request description:

  Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans when compiling. The particular optimization that seems to be causing the problems is that a temp variable is being added for spans->y. For some reason, when it does this, it chooses different instructions to use when making that variable. We bypass this problem by patching qt_intersect_spans to always make and use this local variable.

  Potential alternative to #20436 and #20440

ACKs for top commit:
  hebasto:
    re-ACK 8f7d1b39ef ~for merging into the 0.21 branch, but [not into the master](https://github.com/bitcoin/bitcoin/pull/20454) branch.~
  fanquake:
    ACK 8f7d1b39ef

Tree-SHA512: b0d00a77643554021736524fb64611462ef2ec849a220543c12d99edb0f52f2e8128d2cc61fa82176b7e13b294574774a92d6b649badf8b7630c6d6a7e70ce10
2020-11-24 21:12:02 +08:00
MarcoFalke
9402159b9b Merge #20472: test: Add testing of ParseInt/ParseUInt edge cases with leading +/-/0:s
05c1095388 test: Add testing of ParseInt/ParseUInt edge cases with leading +/-/0:s (practicalswift)

Pull request description:

  Add testing of `ParseInt`/`ParseUInt` edge cases with leading `+`/`-`/`0`:s.

  Context: While working on #20457 and #20452 I noticed some edge cases which our unit tests are currently not covering.

ACKs for top commit:
  MarcoFalke:
    review ACK 05c1095388
  laanwj:
    Code review ACK 05c1095388
  jonatack:
    ACK 05c1095388
  promag:
    Code review ACK 05c1095388.

Tree-SHA512: bdfb94d8fa0293512dbba89907cb6dd0f8b1418d878267dd6d49c8c397a0e5b9714441345565d41a6a909a1cda052ef7cccece822f355ff604fcf85f2dc8136f
2020-11-24 12:10:23 +01:00
MarcoFalke
3a32b62fa7 Merge #20462: RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC request and wallet_name parameter specify a wallet
b1f59d55d9 RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC endpoint and wallet_name parameter specify a wallet (Luke Dashjr)

Pull request description:

  Just documentation clarifications from #20448

ACKs for top commit:
  MarcoFalke:
    review ACK b1f59d55d9
  jonatack:
    re-ACK b1f59d55d9  per `git diff e8303a0 b1f59d5`

Tree-SHA512: ac068b0aa7ceed49496367fdd9425b59dbba18b56e89b26afc22a6c8ece51f0b92a169cacd55740b1cadab2b32f4f8e8700e609066ab7e59d3b53c7891da585e
2020-11-24 12:07:09 +01:00
MarcoFalke
6f9dac5ede Merge #20473: doc: Document current boost dependency as 1.71.0
0918eb49d5 doc: Document current boost dependency as 1.71.0 (Wladimir J. van der Laan)

Pull request description:

  This was forgotten in #19764.

ACKs for top commit:
  practicalswift:
    ACK 0918eb49d5
  fanquake:
    ACK 0918eb49d5

Tree-SHA512: bd4a39b96b95adeb725767b283f4cf04d9f0d6ac352e7dc67f88cf575b00a24c6d3f4bf51fe362e0c89aeebb6c7e8e9add9f9f17e843121efd30f8edef6128bc
2020-11-24 12:04:22 +01:00
Wladimir J. van der Laan
0918eb49d5 doc: Document current boost dependency as 1.71.0
This was forgotten in #19764.
2020-11-24 10:36:07 +01:00
Wladimir J. van der Laan
7ebbee551d Merge #19764: depends: Split boost into build/host packages + bump + cleanup
f190343c96 depends: boost: Specify cflags+compileflags (Carl Dong)
b2328b7989 depends: boost: Remove unnecessary _archiver_ (Carl Dong)
ab9e047cc2 depends: boost: Cleanup toolset selection (Carl Dong)
86002e7e90 depends: boost: Cleanup architecture/address-model (Carl Dong)
d7048fa73f depends: boost: Disable all compression (Carl Dong)
9cf2ee54d3 depends: boost: Split into non-/native packages (Carl Dong)
a57b498560 depends: boost: Bump to 1.71.0 (Carl Dong)
800655ff31 depends: boost: Refer to version in URL (Carl Dong)

Pull request description:

  This PR improves the robustness of our boost package in depends, most notably:
  1. Bumps boost from `1.70.0` to `1.71.0`, because `1.71.0`:
  	1. Removes the need to patch out the unused variable.
  		f8462a6d27/depends/packages/boost.mk (L36)
  		Upstream boost patched it out in d20b64cf37, which was first included in the `1.71.0` release
  	2. Comes packaged with a version of `b2` which allows us to override its `CXX` and `CXXFLAGS`. Previously, choosing a toolset while building `b2` such as `clang` or `gcc` would force `b2`'s build system to invoke the compiler as a bare, hardcoded `clang` or `gcc`. However, our `depends` build system often want to customize this behaviour, adding extra flags or invoking the compiler by an alternate name. So this is useful.
  		1. Commit where `CXX` was introduced: 374f96516a
  		2. Commit where `CXXFLAGS` was introduced: 5d49abc1f2
  2. The boost package is now split into `native_b2` and `boost`, better representing what actually happens.
  	- In our `depends` build system, we have a distinction between `native` packages and non-`native` packages. The output of `native` packages are meant to be used on the machine that's performing the build, and the output of non-`native` packages are meant to be used on/for the machine that will ultimately be running bitcoin. Previously, `boost` existed in `depends` as a non-`native` package, but that's partly inaccurate because the `./bootstrap.sh` invocation in its `$(package)_config_cmds` stage actually produced a binary called `b2`, which is run on the machine that's performing the build. This means that `b2` is a `native` package which is being built in an environment set up for the non-`native` package `boost`. This reveals a hidden unintended behavior in our `depends` build system: for linux->darwin cross builds, we use `gcc` for `native` packages, and `clang` for non-`native` packages. But `b2` was actually being built using `clang`, since it was being built in an environment set up for non-`native` packages.

  theuni you might be interested in taking a look

ACKs for top commit:
  laanwj:
    Concept and code review ACK f190343c96

Tree-SHA512: f8b728a34da4f0a9a985a819a5762f2fc2689ea24c7eba1d24d26dfbd4c59f202227c699b0a4069dab10b6329cf9f4c6dd95082685776ee43dd5f7b659acdef1
2020-11-24 10:33:08 +01:00
MarcoFalke
faa05854f8 util: Remove probably misleading TODO
The TODO has been added by me, but I don't remember how to solve it. The
current code works fine, so just remove the TODO.
2020-11-24 10:09:48 +01:00
MarcoFalke
fac5efe730 util: Add Assume() identity function 2020-11-24 09:47:29 +01:00
practicalswift
fa861569dc util: Allow Assert(...) to be used in all contexts
Fixes the compile error when used inside operator[]:

./chain.h:404:23: error: C++11 only allows consecutive left square brackets when introducing an attribute
        return (*this)[Assert(pindex)->nHeight] == pindex;
                      ^
2020-11-24 09:46:49 +01:00
practicalswift
05c1095388 test: Add testing of ParseInt/ParseUInt edge cases with leading +/-/0:s 2020-11-24 08:36:48 +00:00
Luke Dashjr
89bdad5b25 RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint 2020-11-24 05:33:18 +00:00
Luke Dashjr
b1f59d55d9 RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC endpoint and wallet_name parameter specify a wallet 2020-11-24 05:31:58 +00:00
fanquake
2f5dfe4a7f depends: build qt in c++17 mode 2020-11-24 10:18:24 +08:00
fanquake
104e859c97 builds: don't pass -silent to qt when building in debug mode
This means we'll get build output like this when building with DEBUG=1:

g++ -c -pipe -ffunction-sections -O2 -fPIC -std=c++11 -fno-exceptions <lots more> ../../corelib/kernel/qcoreapplication.cpp

rather than just:

compiling ../../corelib/kernel/qcoreapplication.cpp
2020-11-24 10:18:21 +08:00
fanquake
e2c500636c depends: build zeromq with -std=c++17 2020-11-24 10:18:06 +08:00
fanquake
2374f2fbef depends: build Boost with -std=c++17 2020-11-24 10:17:35 +08:00
fanquake
2dde55702d depends: build bdb with -std=c++17 2020-11-24 07:29:28 +08:00
Niklas Gögge
e95aaefe25 build: Avoid secp256k1.h include from system 2020-11-23 23:33:53 +01:00
João Barbosa
053b4fbad8 doc: Release note regarding -rpcauth validation 2020-11-23 21:02:54 +00:00
João Barbosa
46001323b1 rpc: Validate -rpcauth arguments 2020-11-23 21:02:54 +00:00
João Barbosa
d37c813a43 rpc: Refactor to process -rpcauth once 2020-11-23 21:02:54 +00:00
Carl Dong
f190343c96 depends: boost: Specify cflags+compileflags 2020-11-23 15:50:59 -05:00
Carl Dong
b2328b7989 depends: boost: Remove unnecessary _archiver_
We already have $(package)_ar, so just use that instead
2020-11-23 15:50:58 -05:00
Carl Dong
ab9e047cc2 depends: boost: Cleanup toolset selection 2020-11-23 15:50:57 -05:00
Carl Dong
86002e7e90 depends: boost: Cleanup architecture/address-model 2020-11-23 15:50:56 -05:00
Carl Dong
d7048fa73f depends: boost: Disable all compression 2020-11-23 15:50:55 -05:00
Carl Dong
9cf2ee54d3 depends: boost: Split into non-/native packages 2020-11-23 15:50:54 -05:00
Carl Dong
a57b498560 depends: boost: Bump to 1.71.0 2020-11-23 15:50:53 -05:00
Carl Dong
800655ff31 depends: boost: Refer to version in URL 2020-11-23 15:50:52 -05:00
MarcoFalke
fada14b948 Treat CDataStream bytes as uint8_t
Also, rename CSerializeData to SerializeData
2020-11-23 21:19:50 +01:00
MarcoFalke
fa8bdb048e refactor: Drop CDataStream constructors in favor of one taking a Span of bytes 2020-11-23 21:19:22 +01:00
MarcoFalke
cb89e18845 Merge #19179: ci: Run ci configs on cirrus
fa7a4385d0 ci: Fix doc typos in .cirrus.yml (MarcoFalke)
fa73674738 ci: Run i686 centos ci config on cirrus (MarcoFalke)
fa1f949a4d ci: Run nowallet ci config on cirrus (MarcoFalke)

Pull request description:

  Travis CI Org is shutting down, so move the configs to cirrus ci

ACKs for top commit:
  practicalswift:
    ACK fa7a4385d0: patch looks correct

Tree-SHA512: 1b7125c7f0d2288931fb8c5e90345891d5f7c1f00c4af136afbeb36bd2836f72920a1877dacc7be787e2c8d84de2a733c1ca71931e5c101b222c1fea588619b3
2020-11-23 21:02:17 +01:00
MarcoFalke
fad7be584f test: Fix intermittent p2p_finerprint issue 2020-11-23 20:58:50 +01:00
MarcoFalke
faa96f841f Remove unused CDataStream methods 2020-11-23 19:24:54 +01:00
Andrew Chow
8f7d1b39ef Fix QPainter non-determinism on macOS
Aplies a patch to Qt that fixes the non-determinism by modifying Qt. The
source of the non-determinism is how LLVM 8 optimizes qt_intersect_spans
when compiling. The particular optimization that seems to be causing the
problems is that a temp variable is being added for spans->y. For some
reason, when it does this, it chooses different instructions to use when
making that variable. We bypass this problem by patching
qt_intersect_spans to always make and use this local variable.
2020-11-23 12:08:18 -05:00
MarcoFalke
fa7a4385d0 ci: Fix doc typos in .cirrus.yml 2020-11-23 17:11:05 +01:00
MarcoFalke
fa73674738 ci: Run i686 centos ci config on cirrus 2020-11-23 17:09:12 +01:00
MarcoFalke
fa1f949a4d ci: Run nowallet ci config on cirrus 2020-11-23 17:08:48 +01:00
MarcoFalke
2ee954daae Merge #20458: test: add is_bdb_compiled helper
b87caf10b5 test: add is_bdb_compiled helper (Sjors Provoost)

Pull request description:

  Followup for #20202, needed by #16546.

  Allow the functional test suite to skip tests that require BDB, as well as introduce specific logic to handle whether BDB support is enabled or not. It follows the same pattern as `skip_if_no_sqlite` and `is_sqlite_compiled`.

ACKs for top commit:
  laanwj:
    Code review ACK b87caf10b5

Tree-SHA512: e84fb22e017b28f0f75d17e5368fcba22a893484b31b12082cfe9354e6fbd566daf34b3b82f7deb7205b2061c9c61538e402df000e2f05428affae6dbea05c5e
2020-11-23 15:53:40 +01:00
Wladimir J. van der Laan
555b5d1bf9 Merge #20419: build: set minimum supported macOS to 10.14
a52ecc936a build: set minimum supported macOS to 10.14 (fanquake)

Pull request description:

  This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):

  > You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.

  > While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.

  > I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.

  We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).

ACKs for top commit:
  hebasto:
    ACK a52ecc936a, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
2020-11-23 14:24:58 +01:00
Wladimir J. van der Laan
b0c9024a7c Merge #20449: build: Fix Windows installer build
68c2ef13e9 Fix version string in Windows and Mac installers (Andrew Chow)

Pull request description:

  Apparently NSIS requires a 4 digit version number, and #20223 dropped the 4th digit. So this adds a 4th 0 digit so that building the Windows installer doesn't fail. Also fixes a typo in that version string that was also present in a plist file.

ACKs for top commit:
  fanquake:
    ACK 68c2ef13e9
  laanwj:
    Code review ACK 68c2ef13e9
  hebasto:
    Approach ACK 68c2ef13e9, tested on Linux Mint 20 (x86_64, nsis 3.05-2).

Tree-SHA512: 845560ff176eae8081096426790c928a773fa75d366f42a2a4631c1be2ae9234d7a5b72854ccfaa7fa1a32002b937ca393b12168ffacf9a5e3e311a76725483a
2020-11-23 13:36:41 +01:00
Sjors Provoost
b87caf10b5 test: add is_bdb_compiled helper 2020-11-23 11:30:55 +01:00
Wladimir J. van der Laan
86bf3ae3b5 Merge #20202: wallet: Make BDB support optional
d52f502b1e Fix mock SQLiteDatabases (Andrew Chow)
99309ab3e9 Allow disabling BDB in configure with --without-bdb (Andrew Chow)
ee47f11f73 GUI: Force descriptor wallets when BDB is not compiled (Andrew Chow)
71e40b33bd RPC: Require descriptors=True for createwallet when BDB is not compiled (Andrew Chow)
6ebc41bf9c Enforce salvage is only for BDB wallets (Andrew Chow)
a58b719cf7 Do not compile BDB things when USE_BDB is defined (Andrew Chow)
b33af48210 Include wallet/bdb.h where it is actually being used (Andrew Chow)

Pull request description:

  Adds a `--without-bdb` option to `configure` which disables the compilation of the BDB stuff. Legacy wallets will not be created when BDB is not compiled. A legacy-sqlite wallet can be loaded, but we will not create them.

  Based on #20156 to resolve the situation where both `--without-sqlite` and `--without-bdb` are provided. In that case, the wallet is disabled and `--disable-wallet` is effectively set.

ACKs for top commit:
  laanwj:
    Code review ACK d52f502b1e

Tree-SHA512: 5a92ba7a542acc2e27003e9d4e5940e0d02d5c1f110db06cdcab831372bfd83e8d89c269caff31dd5bff062c1cf5f04683becff12bd23a33be731676f346553d
2020-11-23 10:30:01 +01:00
Wladimir J. van der Laan
1b75f2542d Merge #20432: net: Treat raw message bytes as uint8_t
fabecce719 net: Treat raw message bytes as uint8_t (MarcoFalke)

Pull request description:

  Using `uint8_t` from the beginning when messages are `recv`ed has two style benefits:
  * The signedness is clear from reading the code, as it does not depend on the architecture
  * When passing the bytes on, the need for static signedness casts is dropped, making the code a bit less verbose and more coherent

ACKs for top commit:
  laanwj:
    Code review ACK fabecce719
  theStack:
    Code Review ACK fabecce719
  jonatack:
    Tested ACK fabecce719

Tree-SHA512: e6d9803c78633fde3304faf592afa961ff9462a7912d1da97a24720265274aa10ab4168d71b6ec2756b7448dd42585321afee0e5c889e705be778ce9a330d145
2020-11-23 10:26:25 +01:00
MarcoFalke
fa05d19bd6 test: Fix intermittent issue in mempool_compatibility 2020-11-23 09:27:36 +01:00
Luke Dashjr
a0a771843f contrib: Changes to checks for PowerPC64
Changes from #14066.
2020-11-22 11:11:32 +01:00
Wladimir J. van der Laan
634f6ec4eb contrib: Parse ELF directly for symbol and security checks
Instead of the ever-messier text parsing of the output of the readelf
tool (which is clearly meant for human consumption not to be machine
parseable), parse the ELF binaries directly.

Add a small dependency-less ELF parser specific to the checks.

This is slightly more secure, too, because it removes potential
ambiguity due to misparsing and changes in the output format of `elfread`. It
also allows for stricter and more specific ELF format checks in the future.

This removes the build-time dependency for `readelf`.

It passes the test-security-check for me locally, though I haven't
checked on all platforms.
2020-11-22 11:11:32 +01:00
fanquake
e9a1c9fbde Merge #19867: build: document and cleanup Qt hacks
e1f2553e11 build: remove global_init_link_order from mac qt qmake.conf (fanquake)
498fa16bea build: document preprocessing steps in qt package (fanquake)
bd5d9336d9 build: don't copy Info.plist.* into mkspec for macOS qt build (fanquake)
bfd7e33b4b build: remove plugin_no_soname from mac qt qmake.conf (fanquake)
fdde4c7ce6 build: pass XCODE_VERSION through to qt macOS cross compile conf (fanquake)
49473ef211 build: convert "echo" usage into a patch in qt package (fanquake)

Pull request description:

  Follow up on removing `sed` usage in #19761. Also nice to revisit & cleanup before 5.15.x.

ACKs for top commit:
  laanwj:
    Code review ACK e1f2553e11

Tree-SHA512: 4e6489d877aaa300f69e091d7117136da49611bd80afd45adfbd7ddeb5b3c9c76fb0f87a3249cbe63ba93129df56281fd4a9389daadc852211325c5ca9ac6567
2020-11-22 14:30:16 +08:00
Andrew Chow
68c2ef13e9 Fix version string in Windows and Mac installers
Fixes some typos that cause gitian to fail to make the installers.
2020-11-21 14:12:08 -05:00
MarcoFalke
816132e6eb Merge #20426: wallet: allow zero-fee fundrawtransaction/walletcreatefundedpsbt and other fixes
9f08780dd7 Use the correct incremental fee constant in bumpfee help (Jon Atack)
3f1e10b2b1 Update feeRate (BTC/kvB) to fee_rate (sat/vB) in wallet_bumpfee (Jon Atack)
1b3d700928 Allow zero-fee fundrawtxn and walletcreatefundedpsbt calls (Jon Atack)

Pull request description:

  - Fixes https://github.com/bitcoin/bitcoin/pull/20305/files#r525406176. A check to raise an error on zero-fee txns was mistakenly extended in a0d4957 from the bumpfee and send{toaddress, many} RPCs to also include fundrawtransaction and walletcreatefundedpsbt. This commit re-overrides zero fee rate checking for these two RPCs, not only for the feeRate (BTC/kvB) arg to return to previous behavior, but also for the new fee_rate (sat/vB) arg. Negative fee rates will still raise "amount out of range" by the MoneyRange check in src/bitcoin-tx.cpp::AmountFromValue.

  - Updates a wallet bumpfee test from feeRate (BTC/kvB) to fee_rate (sat/vB)

  - Fixes https://github.com/bitcoin/bitcoin/pull/20305/files#r525405363 to use the correct incremental fee rate constant in the bumpfee help  (thanks Marco Falke for the catch) and rectifies "1.000 sat/vB sat/vB" in the help to "1.000 sat/vB"

ACKs for top commit:
  MarcoFalke:
    review ACK 9f08780dd7  🐾
  promag:
    Code review ACK 9f08780dd7.
  Xekyo:
    Code review reACK 9f08780dd7.

Tree-SHA512: 413dfb4f23ebaf3d2ef210dd04610a843272e64eabba428699f5de4d646a86ac4911dab66b5e2f5ebea53b76e4be8347ef40824c1592c750d5eaa12579d3cdf6
2020-11-21 07:47:40 +01:00
MarcoFalke
d4159984c3 Merge #20223: build: Drop the leading 0 from the version number
8f7b930475 Drop the leading 0 from the version number (Andrew Chow)

Pull request description:

  Removes the leading 0 from the version number. The minor version, which we had been using as the major version, is now the major version. The revision, which we had been using as the minor version, is now the minor version. The revision number is dropped. The build number is promoted to being part of the version number. This also avoids issues where it was accidentally not included in the version number.

  The CLIENT_VERSION remains the same format as previous as previously, as the Major version was 0 so it never actually got included in it.

  The user agent string formatter is updated to follow this new versioning.

  ***

  Honestly I'm just tired of all of the people asking for "1.0" that maybe this'll shut them up. Skip the whole 1.0 thing and go straight to version 22.0!

  Also, this means that the terminology we commonly use lines up with how the variables are named. So major versions are actually bumping the major version number, etc.

ACKs for top commit:
  jnewbery:
    Code review ACK 8f7b930475
  MarcoFalke:
    review ACK 8f7b930475 🎻

Tree-SHA512: b5c3fae14d4c0a9c0ab3b1db7c949ecc0ac3537646306b13d98dd0efc17c489cdd16d43f0a24aaa28e9c4a92ea360500e05480a335b03f9fb308010cdd93a436
2020-11-20 15:42:07 +01:00
MarcoFalke
fabecce719 net: Treat raw message bytes as uint8_t 2020-11-20 15:11:21 +01:00
fanquake
ea460291f9 Merge #20430: sanitizers: Add suppression for unsigned-integer-overflow in libstdc++
0f020cdf0a sanitizers: Add suppression for unsigned-integer-overflow in libstdc++ basic_string.tcc (Jonas Schnelli)

Pull request description:

  Reported here: https://bitcoinbuilds.org/logs/e35cd579-0f0f-47e4-b49a-4ceba8ff9830.log
  Issue: https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/basic_string.tcc#L1271

ACKs for top commit:
  MarcoFalke:
    cr ACK 0f020cdf0a
  practicalswift:
    cr ACK 0f020cdf0a

Tree-SHA512: e304259a1eed878263bd715b4d16c57f8974264c23ccd6799f85e8141b2eb0b5c6468a6452ffbc7334f57c1957b6e43bb248760b3c0718d93f092d57764d0a8f
2020-11-20 17:56:03 +08:00
fanquake
4c2ee5c855 Merge #20424: build: Update univalue subtree
2a55a0ed30 Squashed 'src/univalue/' changes from 98261b1e7b..98fadc0909 (MarcoFalke)

Pull request description:

  Just a minor bugfix: Currently we don't push booleans into arrays, but if we did they'd be pushed as integers.

  Can be tested by reverting the diff in `include/` and observing a test failure.

ACKs for top commit:
  laanwj:
    ACK fa17eef627
  practicalswift:
    cr ACK fa17eef627

Tree-SHA512: d87ca5be6769b4cd0c9b9e319973bc0c4f2b7121779f9554e11f34a4edb0013997e875c7edb7bc6eb9309ff5c13379d22f436cd4fb9e6e68df6f0aee29fed914
2020-11-20 17:53:34 +08:00
Jon Atack
9f08780dd7 Use the correct incremental fee constant in bumpfee help
and remove redundant units ("Must be at least 1.000 sat/vB sat/vB" -> "1.00 sat vB")
2020-11-20 09:43:52 +01:00
Jon Atack
3f1e10b2b1 Update feeRate (BTC/kvB) to fee_rate (sat/vB) in wallet_bumpfee
as the feeRate argument should soon be deprecated.

Also loosen one test (and a similar one) that caused a one-off CI failure with:
expected message
'Insufficient total fee 0.00000141, must be at least 0.00001704 (oldFee 0.00000999 + incrementalFee 0.00000705)'
actual message
'Insufficient total fee 0.00000141, must be at least 0.00001712 (oldFee 0.00001007 + incrementalFee 0.00000705)'
2020-11-20 09:40:47 +01:00
Jon Atack
1b3d700928 Allow zero-fee fundrawtxn and walletcreatefundedpsbt calls
A check to raise an error on zero-fee txns was mistakenly extended in commit
a0d4957 from the bumpfee and send{toaddress, many} RPCs to also include
fundrawtransaction and walletcreatefundedpsbt.

This commit overrides zero fee rate checking for these two RPCs, not only for
the feeRate (BTC/kvB) arg to return to previous behavior, but also for the new
fee_rate (sat/vB) arg.
2020-11-20 09:40:44 +01:00
Jonas Schnelli
094663842d Merge bitcoin-core/gui#13: Hide peer detail view if multiple are selected
76277cc77d qt: Hide peer detail view if multiple are selected (João Barbosa)

Pull request description:

  Currently if multiple peers are selected the peer detail view shows the first new selected peer.

  With this PR the peer detail view is hidden when multiple peers are selected. It is also a slight refactor to simplify and remove duplicate code.

ACKs for top commit:
  jonasschnelli:
    Tested ACK 76277cc77d.
  hebasto:
    ACK 76277cc77d, tested on Linux Mint 20 (Qt 5.12.8).

Tree-SHA512: 16c9cfd6ccb7077a9f31917a6cb3532e32d17d21f735e43bf4720fb0c8bb1bd539d42569c105df4b551f5dccb4acaeedb6bb2362620a9cb9267a602d9d065b9f
2020-11-20 09:38:05 +01:00
Jonas Schnelli
c45e1d9aa7 Merge bitcoin-core/gui#21: Update pruning tooltip, original author BitcoinErrorLog
2fc5efc55c Update pruning tooltip, original author BitcoinErrorLog (Riccardo Spagni)

Pull request description:

  Squashed commits from BitcoinErrorLog at his request, per the original discussion on #15: this tooltip has been adjusted to be more user-friendly and reflect what the net effect of pruning is for the user.

ACKs for top commit:
  harding:
    Untested ACK 2fc5efc55c
  Sjors:
    utACK 2fc5efc55c and welcome to the dark side!
  jonasschnelli:
    ACK 2fc5efc55c

Tree-SHA512: 45d6a7efbf4d34d20b9de439c988a39c739591b854726b6682c4cffcb23dff7d9131afab572fa0c9a8bc033c46c3878efdfbf8a984aafde632e1dfc1caa1cbbb
2020-11-20 09:35:00 +01:00
MarcoFalke
3d632c3f49 Merge #20428: tests: shrink feature_taproot transfer of funds tx
7ffac12545 tests: shrink feature_taproot transfer of funds tx (Anthony Towns)

Pull request description:

  When moving funds from node 1 to node 0 for the pre-activation tests, there can be a large number of inputs, potentially resulting in a tx that is larger than standardness rules allow, or that takes a long time to sign. This just takes the top 500 outputs, which is plenty (~90% of the wallet balance).

ACKs for top commit:
  luke-jr:
    utACK 7ffac12545
  MarcoFalke:
    cr ACK 7ffac12545

Tree-SHA512: 68445b4827dddb9a8e8614d61a68f5bbd7152557bf940be0a75741cb49deeff1566198da1a777ac66cb3fed93e64a30bf895875d6cc6ae9e48275e3febb620a6
2020-11-20 09:30:12 +01:00
Jonas Schnelli
0f020cdf0a sanitizers: Add suppression for unsigned-integer-overflow in libstdc++ basic_string.tcc 2020-11-20 08:40:50 +01:00
Wladimir J. van der Laan
fdd068507d Merge #20056: net: Use Span in ReceiveMsgBytes
fa5ed3b4ca net: Use Span in ReceiveMsgBytes (MarcoFalke)

Pull request description:

  Pass a data pointer and a size as span in `ReceiveMsgBytes` to get the benefits of a span

ACKs for top commit:
  jonatack:
    ACK fa5ed3b4ca code review, rebased to current master 12a1c3ad1a, debug build, unit tests, ran bitcoind/-netinfo/getpeerinfo
  theStack:
    ACK fa5ed3b4ca

Tree-SHA512: 89bf111323148d6e6e50185ad20ab39f73ab3a58a27e46319e3a08bcf5dcf9d6aa84faff0fd6afb90cb892ac2f557a237c144560986063bc736a69ace353ab9d
2020-11-20 06:10:58 +01:00
Wladimir J. van der Laan
46f0b2f976 Merge #19851: refactor: Extract ParseOpCode from ParseScript
c92387232f refactor: Extract ParseOpCode from ParseScript (João Barbosa)

Pull request description:

  Seems more natural to have `mapOpNames` "hidden" in `ParseOpCode` than in `ParseScript`.

  A second lookup in `mapOpNames` is also removed.

ACKs for top commit:
  laanwj:
    ACK c92387232f
  theStack:
    re-ACK c92387232f

Tree-SHA512: d59d1964760622cf365479d44e3e676aa0bf46b60e77160140d967e012042df92121d3224c7551dc96eff5ff3294598cc6bade82adb3f60d28810e18e60e1257
2020-11-20 05:36:01 +01:00
Wladimir J. van der Laan
2878167c18 Merge #20000: test: fix creation of "std::string"s with \0s
ecc6cf1a3b test: fix creation of std::string objects with \0s (Vasil Dimov)

Pull request description:

  A string literal `"abc"` contains a terminating `\0`, so that is 4
  bytes. There is no need to write `"abc\0"` unless two terminating
  `\0`s are necessary.

  `std::string` objects do not internally contain a terminating `\0`, so
  `std::string("abc")` creates a string with size 3 and is the same as
  `std::string("abc", 3)`.

  In `"\01"` the `01` part is interpreted as one number (1) and that is
  the same as `"\1"` which is a string like `{1, 0}` whereas `"\0z"` is a
  string like `{0, 'z', 0}`. To create a string like `{0, '1', 0}` one
  must use `"\0" "1"`.

  Adjust the tests accordingly.

ACKs for top commit:
  laanwj:
    ACK ecc6cf1a3b
  practicalswift:
    ACK ecc6cf1a3b modulo happily green CI

Tree-SHA512: 5eb489e8533a4199a9324b92f7280041552379731ebf7dfee169f70d5458e20e29b36f8bfaee6f201f48ab2b9d1d0fc4bdf8d6e4c58d6102f399cfbea54a219e
2020-11-20 04:39:37 +01:00
Anthony Towns
7ffac12545 tests: shrink feature_taproot transfer of funds tx 2020-11-20 07:29:42 +10:00
Jon Atack
3eb6f8b2e6 wallet (not for backport): improve upgradewallet error messages 2020-11-19 20:00:56 +01:00
Jon Atack
ca8cd893bb wallet: fix and improve upgradewallet error responses 2020-11-19 20:00:53 +01:00
Jon Atack
99d56e3571 wallet: fix and improve upgradewallet result responses 2020-11-19 20:00:50 +01:00
MarcoFalke
0a267f4eb8 Merge bitcoin-core/gui#46: refactor: Fix deprecation warnings when building against Qt 5.15
705c1f0648 qt, refactor: Fix 'buttonClicked is deprecated' warnings (Hennadii Stepanov)
c2f4e5ea1d qt, refactor: Fix 'split is deprecated' warnings (Hennadii Stepanov)
8e12d69961 qt, refactor: Fix 'QFlags is deprecated' warnings (Hennadii Stepanov)
fa5749c805 qt, refactor: Fix 'pixmap is deprecated' warnings (Hennadii Stepanov)
b02264cb5d qt, refactor: Fix 'QDateTime is deprecated' warnings (Hennadii Stepanov)

Pull request description:

  [What's New in Qt 5.15](https://doc.qt.io/qt-5/whatsnew515.html#deprecated-modules):
  > To help preparing for the transition to Qt 6, numerous classes and member functions that will be removed from Qt 6.0 have been marked as deprecated in the Qt 5.15 release.

  Fixes #36

ACKs for top commit:
  jonasschnelli:
    utACK 705c1f0648
  promag:
    Tested ACK 705c1f0648 on macos with Apple clang version 11.0.3 (clang-1103.0.32.62) and brew qt 5.15.1.

Tree-SHA512: 29e00535b4583ceec0dfb29612e86ee29bdea13651b548c6d22167917a4a10464af49160a12b05151030699f690f437ebb9c4ae9f130f66a722415222165b44f
2020-11-19 19:04:22 +01:00
practicalswift
17a5f172fa fuzz: Make addrman fuzzing harness deterministic 2020-11-19 17:21:55 +00:00
MarcoFalke
db58b857f7 Merge #20329: docs/descriptors.md: Remove hardened marker in the path after xpub
dc80a7d0b0 docs/descriptors.md: Remove hardened marker in the path after xpub (Dmitry Petukhov)

Pull request description:

  As described in "Key origin identification" section, a descriptor
  that has hardened derivation after xpub does not let you compute scripts
  without access to the corresponding private keys. Such a descriptor is
  practically useless.

  The text after the descriptor said "with child key *1'/2* of the
  specified xpub", and clearly an xpub cannot have "child key" with
  hardened derivation. Therefore it makes sense to fix this inconsistency
  to not confuse the reader of the doc

Top commit has no ACKs.

Tree-SHA512: 9f35951d90868585303681c27ea2ef9d36d4036181e337cfbd48730de0c346320b08dd089350b116d4c8542f3b506868cf318796bb713e5f80600ccbd96f9970
2020-11-19 17:59:20 +01:00
Wladimir J. van der Laan
04670ef81e Merge #20385: test: run mempool_spend_coinbase.py even with wallet disabled
21f2433601 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz)

Pull request description:

  Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in #20078.

ACKs for top commit:
  laanwj:
    ACK 21f2433601

Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
2020-11-19 16:40:02 +01:00
Wladimir J. van der Laan
848d66519c Merge #20054: Remove confusing and useless "unexpected version" warning
0000a0c7e9 Remove confusing and almost useless "unexpected version" warning (MarcoFalke)

Pull request description:

  It is useless because it isn't displayed for most users:

  * It isn't displayed in normal operation (because the validation debug category is disabled by default)
  * It isn't displayed for users that sync up their nodes intermittently, e.g. once a day or once a week (because it is disabled for IBD)
  * It is only displayed in the debug log (as opposed to the versionbits warning, which is displayed more prominently)

  It is confusing because it doesn't have a use case:

  Despite the above, if a user *did* see the warning, it would most likely be a false positive (like it has been in the past). Even if it wasn't, there is nothing they can do about it. The only thing they could do is to check for updates and hope that a fixed version is available. But why would the user be so scrupulously precise in enabling the warning and reading the log, but then fail to regularly check update channels for updated software?

ACKs for top commit:
  practicalswift:
    ACK 0000a0c7e9
  decryp2kanon:
    ACK 0000a0c
  LarryRuane:
    ACK 0000a0c7e9

Tree-SHA512: 16e069c84be6ab6034baeefdc515d0e5cdf560b2005d2faec5f989d45494bd16cfcb4ffca6a17211d9556ae44f9737a60a476c08b5c2bb5e1bd29724ecd6d5c1
2020-11-19 16:39:31 +01:00
MarcoFalke
884bde510e Merge #20291: [net] Consolidate logic around calling CAddrMan::Connected()
0bfce9dc46 [addrman] Fix Connected() comment (John Newbery)
eefe194718 [net] Consolidate logic around calling CAddrMan::Connected() (John Newbery)

Pull request description:

  Currently, the logic around whether we called CAddrMan::Connected() for
  a peer is spread between verack processing (where we discard inbound
  peers) and FinalizeNode (where we discard misbehaving and
  block-relay-only peers). Consolidate that logic to a single place.

  Also remove the CNode.fCurrentlyConnected bool, which is now
  redundant. We can rely on CNode.fSuccessfullyConnected, since the two
  bools were only ever flipped to true in the same place.

ACKs for top commit:
  mzumsande:
    Code review ACK 0bfce9dc46
  amitiuttarwar:
    code review ACK 0bfce9dc46. nice tidy, and bonus that we get to remove an unnecessary call to `cs_main`

Tree-SHA512: 1ab74dae3bc12a6846da57c853033e546bb4f91caa39f4c50bf0cf7eca59cb917bdb2ef795da55363e7e9f70949cb28bb3be004cb3afa4389f970d2fe097d932
2020-11-19 16:30:54 +01:00
MarcoFalke
fa17eef627 Update univalue subtree 2020-11-19 15:48:24 +01:00
MarcoFalke
2a55a0ed30 Squashed 'src/univalue/' changes from 98261b1e7b..98fadc0909
98fadc0909 Merge #24: Push bool into array correctly
5f03f1f39a Push bool into array correctly

git-subtree-dir: src/univalue
git-subtree-split: 98fadc090984fa7e070b6c41ccb514f69a371c85
2020-11-19 15:47:07 +01:00
Wladimir J. van der Laan
c4d1e24f54 Merge #20047: test: use wait_for_{block,header} helpers in p2p_fingerprint.py
6b56c1f4d0 test: remove last_{block,header}_equals() in p2p_fingerprint.py (Sebastian Falbesoner)
136d96b71f test: use wait_for_{block,header} helpers in p2p_fingerprint.py (Sebastian Falbesoner)

Pull request description:

  This small PR takes use of the message receiving helper functions `wait_for_block()` and `wait_for_header()` (from module `test_framework.p2p`) in the test `p2p_fingerprint.py`.  It also simplifies the checks for very old stale blocks/headers requests by getting rid of the functions `last_block_equals()` and `last_header_equals()` and rather only testing that not any blocks/headers message is received at all. Unneeded sending of requests are also removed and calls to time.sleep(...) substituted by ping syncs.

ACKs for top commit:
  guggero:
    ACK 6b56c1f4

Tree-SHA512: 9114db70f3804adad4ab658236762d4fa73fef91158c5756dd1af2d24196ea740451b0028667e0c4047f1f89fe1355031921d3dfb973acc1370052a4bc12c2ab
2020-11-19 15:38:17 +01:00
Wladimir J. van der Laan
7aa94569ce Merge #20024: init: Fix incorrect warning "Reducing -maxconnections from N to N-1, because of system limitations"
ea93bbeb26 init: Fix incorrect warning "Reducing -maxconnections from N to N-1, because of system limitations" (practicalswift)

Pull request description:

  Fix incorrect warning `Reducing -maxconnections from N to N-1, because of system limitations`.

  Before this patch (only the first warning is correct):

  ```
  $ src/bitcoind -maxconnections=10000000 | grep Warning
  2020-09-26T01:23:45Z Warning: Reducing -maxconnections from 10000000 to 1048417, because of system limitations.

  $ src/bitcoind -maxconnections=1000000 | grep Warning
  2020-09-26T01:23:45Z Warning: Reducing -maxconnections from 1000000 to 999999, because of system limitations.

  $ src/bitcoind -maxconnections=100000 | grep Warning
  2020-09-26T01:23:45Z Warning: Reducing -maxconnections from 100000 to 99999, because of system limitations.

  $ src/bitcoind -maxconnections=10000 | grep Warning
  2020-09-26T01:23:45Z Warning: Reducing -maxconnections from 10000 to 9999, because of system limitations.

  $ src/bitcoind -maxconnections=1000 | grep Warning
  2020-09-26T01:23:45Z Warning: Reducing -maxconnections from 1000 to 999, because of system limitations.

  $ src/bitcoind -maxconnections=100 | grep Warning
  [no warning]
  ```

  After this patch (no incorrect warnings):

  ```
  $ src/bitcoind -maxconnections=10000000 | grep Warning
  2020-09-26T01:23:45Z Warning: Reducing -maxconnections from 10000000 to 1048417, because of system limitations.

  $ src/bitcoind -maxconnections=1000000 | grep Warning
  [no warning]

  $ src/bitcoind -maxconnections=100000 | grep Warning
  [no warning]

  $ src/bitcoind -maxconnections=10000 | grep Warning
  [no warning]

  $ src/bitcoind -maxconnections=1000 | grep Warning
  [no warning]

  $ src/bitcoind -maxconnections=100 | grep Warning
  [no warning]
  ```

ACKs for top commit:
  n-thumann:
    tACK ea93bbeb26, Ran on other systems running Debian 10.5 (4.19.0-8-amd64) and Debian bullseye/sid (5.3.0-1-amd64) and was able to reproduce the issue exactly as you described above on both of them. After applying your patch the issue is fixed ✌️
  laanwj:
    Code review ACK ea93bbeb26
  theStack:
    tACK ea93bbeb26

Tree-SHA512: 9b0939a1a51fdf991d11024a5d20b4f39cab1a80320b799a1d24d0250aa059666bcb1ae6dd79c941c2f2686f07f59fc0f6618b5746aa8ca6011fdd202828a930
2020-11-19 15:33:02 +01:00
MarcoFalke
71d068db40 Merge #18531: rpc: remove deprecated CRPCCommand constructor
faaf9c58e4 remove CRPCCommand constructor that takes rpcfn_type function pointer (MarcoFalke)
fa19bb2cd8 remove dead rpc code (MarcoFalke)

Pull request description:

  Remove the CRPCCommand arguments, now that they are asserted to be equal and thus redundant

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    tested ACK faaf9c58e4
  promag:
    Tested ACK faaf9c58e4.
  ryanofsky:
    Code review ACK faaf9c58e4. Two obviously good simplifications.

Tree-SHA512: 5de3b440f7b2ed2c3e86655d4f0e2e5df9c67e8ce3c7817d5ea5311d1a38690f2f3e28fab41aad6936be9fc884326d037e5f19e85d4d2fe281474dada13911ee
2020-11-19 14:19:05 +01:00
MarcoFalke
fa69c2c784 wallet: Do not treat default constructed types as None-type 2020-11-19 13:48:38 +01:00
Wladimir J. van der Laan
9ab9665c74 Merge #15710: wallet: Catch ios_base::failure specifically
7486e2771e Tests: Unit test related to WalletDB ReadKeyValue (Bushstar)
32def8d1c2 Catch ios_base::failure specifically (Peter Bushnell)

Pull request description:

  In https://github.com/bitcoin/bitcoin/pull/2950 a hash of the pubkey and private was added to speed up key import, this was made backwards compatible by reading the hash in a try block with an ellipses catch all in case the hash was not present.

  CDataStream::read() specifically throws std::ios_base::failure, backwards compatibility expects only that error to be thrown, if something else gets thrown we should not be catching it. The change in this commit is to catch that exception only. If any other exception is thrown other than std::ios_base::failure it will be caught by the wider try block and an error written to the log and/or console.

  CDataStream::read() throwing std::ios_base::failure.
  2c364fde42/src/streams.h (L191)

  Wider catch statements that pick up all others exceptions other than ios_base::failure.
  2c364fde42/src/wallet/walletdb.cpp (L425)

  2c364fde42/src/wallet/walletdb.cpp (L430)

ACKs for top commit:
  laanwj:
    Code review ACK 7486e2771e

Tree-SHA512: 5364bf935af8ec603bf5b8fef8c23b5cdaa4fe3506090cff988413221f2eaa99f7a91929afb42a35f8881ce2328744a0d32052da51ca0a5b2e65b6809e97f604
2020-11-19 12:32:48 +01:00
Wladimir J. van der Laan
d9180c50b6 Merge #20145: contrib: add getcoins.py script to get coins from (signet) faucet
e9c8e6eea2 doc: add contrib/signet readme (Karl-Johan Alm)
355d0c4f6b contrib: add getcoins.py script to get coins from (signet) faucet (Karl-Johan Alm)

Pull request description:

  This adds a small python script that can be used to fetch Signet coins from the default (or custom) faucet.

ACKs for top commit:
  laanwj:
    Code and documentation review ACK e9c8e6eea2

Tree-SHA512: 9aaeb96bf0c636a38e2dbe4cfc8b3ef907b1c05d0b782ee51223014952e07ce45a071c7e99aa9aa7700196a67f8a47d74d13e5e8d6890b9be503acd2bacd4d4f
2020-11-19 12:09:19 +01:00
Wladimir J. van der Laan
1cc5e693c1 Merge #20358: src/randomenv.cpp: fix build on uclibc
330cb33985 src/randomenv.cpp: fix build on uclibc (Fabrice Fontaine)

Pull request description:

  Check for HAVE_STRONG_GETAUXVAL or HAVE_WEAK_GETAUXVAL before using
  getauxval to avoid a build failure on uclibc

  Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>

ACKs for top commit:
  laanwj:
    Code review ACK 330cb33985

Tree-SHA512: 94fbbdb0e859f0220d64b2d04565f575b410327f080125fec7fb74205d0bea0e8133561c83a696033d6dc377871133871b72c1aad19aca61e972ce67e0fdf707
2020-11-19 12:00:10 +01:00
Wladimir J. van der Laan
fbb2bee82d Merge #20067: refactor: remove use of boost::algorithm::replace_first
6f4e393646 refactor: remove use of boost::algorithm::replace_first (Sebastian Falbesoner)

Pull request description:

  As discussed in #19851 (https://github.com/bitcoin/bitcoin/pull/19851#issuecomment-685424702), this trivial PR substitutes the (only) use of `boost::algorithm::replace_first` by a direct implementation.

ACKs for top commit:
  laanwj:
    Code review ACK 6f4e393646

Tree-SHA512: 2ef06498e19f864a4cbae10e8d1905e3440a2d1e8e5aae83de7597c23cdab92b4612d7fa1efbc49016e530debd127d1d50531c60ff159dbea0deaa8c836a2bfb
2020-11-19 11:50:53 +01:00
Wladimir J. van der Laan
e12ad7f383 Merge #19968: doc: clarify CRollingBloomFilter size estimate
d9141a0002 doc: clarify CRollingBloomFilter size estimate (Anthony Towns)

Pull request description:

  Based on #19130, this change improves the comment for `CRollingBloomFilter` in `bloom.h`:

  - Give examples to illustrate the heuristic "1.8 bytes per element per factor 0.1 of false positive rate"
  - Add some Python code which can be copy/pasted for convenient filter size calculation (in an interpreter)
  - Reconcile the newly added code with the existing approximation

ACKs for top commit:
  laanwj:
    ACK d9141a0002

Tree-SHA512: e7138b3c531883a750ead06368975c750863fde7ef6f2633b137eca011079226e9205316217322014399fba05a48f294c788dd700bb7d479c58fe1f23e40419f
2020-11-19 11:44:29 +01:00
Wladimir J. van der Laan
47b6ad837c Merge #20333: build: remove native_biplist dependency
7087440894 depends: native_ds_store 1.3.0 (fanquake)

Pull request description:

  `ds_store` [now takes advantage](36fb607940) of Pythons ability to decode binary [plists](https://docs.python.org/3/library/plistlib.html) (since 3.4), so we can drop its biplist dependency.

  The call to `biplist.Data()` in `custom_dsstore.py` doesn't seem to do anything, and from what I can tell can just be removed. i.e:
  ```diff
  diff --git a/contrib/macdeploy/custom_dsstore.py b/contrib/macdeploy/custom_dsstore.py
  index dc1c1882d..e475bc6c3 100755
  --- a/contrib/macdeploy/custom_dsstore.py
  +++ b/contrib/macdeploy/custom_dsstore.py
  @@ -47,6 +47,7 @@ alias.volume.disk_image_alias.target.filename = package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.carbon_path = 'Macintosh HD:Users:\x00bitcoinuser:\x00Documents:\x00bitcoin:\x00bitcoin:\x00' + package_name_ns + '.temp.dmg'
   alias.volume.disk_image_alias.target.posix_path = 'Users/bitcoinuser/Documents/bitcoin/bitcoin/' + package_name_ns + '.temp.dmg'
   alias.target.carbon_path = package_name_ns + ':.background:\x00background.tiff'
  +assert(biplist.Data(alias.to_bytes()) == alias.to_bytes())
   icvp['backgroundImageAlias'] = biplist.Data(alias.to_bytes())
   ds['.']['icvp'] = icvp
  ```

ACKs for top commit:
  laanwj:
    ACK 7087440894

Tree-SHA512: 8ba3cf561937efe4a3daae8b0cb4de3bf9e425b3a9244161b09d94ee2b1bd4c3e21315fa70e495b19a052aabdc1731b3b6f346b63272d72d2762ced83237d02f
2020-11-19 11:39:00 +01:00
Wladimir J. van der Laan
888c22e0dd Merge #20359: depends: Various config.site.in improvements and linting
46756a6987 depends: Fix PYTHONPATH setting in config.site.in (Carl Dong)
618cbd2c1a lint: Also lint files with shellcheck directive (Carl Dong)
6c7e8f067d depends: Allow relative CONFIG_SITE path env var (Carl Dong)

Pull request description:

  This changeset:

  1. Allows the `CONFIG_SITE` env var to be a relative path rather than requiring an absolute one
  2. Enables linting of the `config.site.in` file with `shellcheck` in our linting scripts
  3. Sets the `PYTHONPATH` var sensibly in `config.site.in`

  Please see commit messages for more details

ACKs for top commit:
  laanwj:
    ACK 46756a6987

Tree-SHA512: 744089b9f6e5604e56466d9a3e64563f9183a70f7e300ac9ae6248f0f17c0b53fe28a2c41d43c5ffe5da825f53c2ca21f21aacba0579442da3056fb0c4b81454
2020-11-19 11:32:16 +01:00
Wladimir J. van der Laan
cddcd22ab3 Merge #20288: script, doc: contrib/seeds updates
961f148cb1 doc: update contrib/seeds/README dnspython installation info (Jon Atack)
dd7b5f46d8 script: fix deprecation warning in makeseeds.py (Jon Atack)

Pull request description:

  Seen while reviewing #20237.

  1. Fix a deprecation warning in `contrib/seeds/makeseeds.py`
  ```
      makeseeds.py:139: DeprecationWarning: please use dns.resolver.resolve() instead
        asn = int([x.to_text() for x in dns.resolver.query('.'.join(
  ```
    - Per https://dnspython.readthedocs.io/en/latest/whatsnew.html, `dns.resolver.query()` was deprecated in `dnspython` version 2.0.0.

    - See https://dnspython.readthedocs.io/en/latest/resolver-class.html for more info on the resolver class.

  2. Update the `dnspython` dependency installation instructions in `contrib/seeds/README`

    - The markdown rendering can be seen here: https://github.com/jonatack/bitcoin/tree/contrib-seeds-fixups/contrib/seeds

ACKs for top commit:
  laanwj:
    code review ACK 961f148cb1

Tree-SHA512: f9c4f318a1a0d35b8de147d24b72c534a1f58eece31e7cfa00b4149a63b6a618d8ca0312f52fd8056f3c645cf2ee68574ca02319fddffdad919a70cd33395d33
2020-11-19 10:40:46 +01:00
MarcoFalke
faaad1bbac p2p: Ignore version msgs after initial version msg
Sending a version message after the intial version message is peer
misbehavior. Though, it seems arbitrary and confusing to disconnect only
after exactly 100 version messages. Duplicate version messages affect us
no different than any other unknown message. So remove the Misbehaving
and ignore the redundant msgs.
2020-11-19 08:07:16 +01:00
MarcoFalke
fad68afcff p2p: Ignore non-version msgs before version msg
Sending a non-version message before the initial version message is peer
misbehavior. Though, it seems arbitrary and confusing to disconnect only
after exactly 100 non-version messages. So remove the Misbehaving and
instead rely on the existing disconnect-due-to-handshake-timeout logic.
2020-11-19 08:06:30 +01:00
Andrew Chow
2498b04ce8 Don't upgrade to HD split if it is already supported
It is unnecessary to upgrade to FEATURE_HD_SPLIT if this feature is
already supported by the wallet. Because upgrading to FEATURE_HD_SPLIT
actually requires upgrading to FEATURE_PRE_SPLIT_KEYPOOL, users would
accidentally be upgraded to FEATURE_PRE_SPLIT_KEYPOOL instead of nothing
being done.

Fixes the issue described at
https://github.com/bitcoin/bitcoin/pull/20403#discussion_r526063920
2020-11-19 08:04:05 +01:00
Luke Dashjr
e2dcd957fa GUI/Intro: Rework UI flow to let the user set prune size in GBs 2020-11-19 03:31:12 +00:00
Luke Dashjr
f2e5a6b54f GUI/Intro: Abstract GUI-to-option into Intro::getPrune 2020-11-19 03:19:30 +00:00
Luke Dashjr
62932cc686 GUI/Intro: Return actual prune setting from showIfNeeded 2020-11-19 03:15:53 +00:00
fanquake
ea7926527c Merge #20413: build: Require C++17 compiler
fac7198728 Use std::make_unique (MarcoFalke)
faaee810e6 build: Require C++17 compiler (MarcoFalke)

Pull request description:

  Developers have been compiling with C++17 for a few months now (fuzz tests and the msvc build have it even enabled by default). According to #16684, the 22.0 release shall be compiled with C++17 enabled.

  This only sets the build flag, any other changes need more discussion and can be done later.

ACKs for top commit:
  elichai:
    utACK fac7198728
  hebasto:
    ACK fac7198728, I've locally compiled on ARM 32bit SBC without GUI.
  fanquake:
    ACK fac7198728

Tree-SHA512: 53eb40ba5d496376a2d2cf16e2000bef36616cc2a3696c3ec59a5366e41fa8b872817a7ca21751f030f9c1efb339dfa63cc655eaa5199b9a3d2e52c2de0ccb29
2020-11-19 10:29:05 +08:00
Andrew Chow
8f7b930475 Drop the leading 0 from the version number
Removes the leading 0 from the version number. The minor version, which
we had been using as the major version, is now the major version. The
revision, which we had been using as the minor version, is now the minor
version. The revision number is dropped. The build number is promoted to
being part of the version number. This also avoids issues where it was
accidentally not included in the version number.

The CLIENT_VERSION remains the same format as previous as previously,
the Major version was 0 so that was never a factor in CLIENT_VERSION.
2020-11-18 12:00:57 -05:00
Andrew Chow
d52f502b1e Fix mock SQLiteDatabases 2020-11-18 11:56:12 -05:00
Andrew Chow
99309ab3e9 Allow disabling BDB in configure with --without-bdb 2020-11-18 11:56:12 -05:00
Andrew Chow
ee47f11f73 GUI: Force descriptor wallets when BDB is not compiled 2020-11-18 11:56:12 -05:00
Andrew Chow
71e40b33bd RPC: Require descriptors=True for createwallet when BDB is not compiled 2020-11-18 11:56:12 -05:00
Andrew Chow
6ebc41bf9c Enforce salvage is only for BDB wallets 2020-11-18 11:56:12 -05:00
Andrew Chow
a58b719cf7 Do not compile BDB things when USE_BDB is defined 2020-11-18 11:56:08 -05:00
Andrew Chow
b33af48210 Include wallet/bdb.h where it is actually being used 2020-11-18 11:55:43 -05:00
Jon Atack
c46c18b788 wallet: refactor GetClosestWalletFeature() 2020-11-18 16:11:47 +01:00
MarcoFalke
fac7198728 Use std::make_unique 2020-11-18 15:15:37 +01:00
MarcoFalke
faaee810e6 build: Require C++17 compiler 2020-11-18 15:15:04 +01:00
fanquake
a52ecc936a build: set minimum supported macOS to 10.14 2020-11-18 21:46:09 +08:00
MarcoFalke
50e019a97a Merge #20414: doc: Remove generated manual pages from master branch
daf1ebf0b1 doc: Remove generated manual pages from master branch (Wladimir J. van der Laan)

Pull request description:

  Replace the generated manual pages on the master branch with placeholders with instructions. The master branch is too much in flux for anything generated from command output to be kept up to date, and having pages from versions ago looks silly.

  We can't remove them completely because `make dist` relies on the files being present.

  Resolves #20062.

ACKs for top commit:
  MarcoFalke:
    review ACK daf1ebf0b1
  RiccardoMasutti:
    ACK daf1ebf

Tree-SHA512: fdd0bb777ddf66c663e4f6b00d206b3f6283ae89028e00b38d16e68a224a781e9245a25edf6f3326f9252ae2ef7c2950c591667c56d4e9d10a16fc50a6c358f4
2020-11-18 14:42:37 +01:00
MarcoFalke
d020e8839f Merge #20418: doc: clean out release notes post branch-off
8a715a6b17 build: Bump gitian descriptors to 0.22 (fanquake)
dc5a35a507 doc: clean out release notes post branch-off (fanquake)

Pull request description:

  Usually done during a version bump. i.e: d84c9aa25d.

ACKs for top commit:
  MarcoFalke:
    review ACK 8a715a6b17 👜

Tree-SHA512: e185b4b51f52014777905c346185e1d6cfd147ce5431532eee4e188a70fc942da389d93cf61ccc9466126ec05a4d3896cab439029cb7ffc47b67e8af03b6b0a4
2020-11-18 14:02:25 +01:00
fanquake
8a715a6b17 build: Bump gitian descriptors to 0.22 2020-11-18 20:52:44 +08:00
fanquake
dc5a35a507 doc: clean out release notes post branch-off 2020-11-18 20:52:38 +08:00
fanquake
5bcae7967f Merge #20408: CConnman: move initialization to declaration
9d09132be4 CConnman: initialise at declaration rather than in Start() (Anthony Towns)

Pull request description:

  Ensure nMaxOutboundTotalBytesSentInCycle and nMaxOutboundCycleStartTime are initialized even if CConnman::Start() is not called. Prevents failures in test/fuzz/connman when run under valgrind.

ACKs for top commit:
  practicalswift:
    ACK 9d09132be4: patch looks correct!
  MarcoFalke:
    review ACK 9d09132be4 , checked that we call Start only once and in the same scope where connman is constructed (AppInitMain) 💸
  jnewbery:
    Code review ACK 9d09132be4

Tree-SHA512: 1c6c893e8c616a91947a8cc295b0ba508af3ecfcdcd94cdc5f95d808cc93c6d1a71fd24dcc194dc583854e9889fb522ca8523043367fb0263370fbcab08c6aaa
2020-11-18 20:45:46 +08:00
MarcoFalke
a64ff1c4d3 Merge #19905: Remove dead CheckForkWarningConditionsOnNewFork
fa7eed5be7 doc: Clarify that vpindexToConnect is in reverse order (MarcoFalke)
fa62304c97 Remove dead CheckForkWarningConditionsOnNewFork (MarcoFalke)

Pull request description:

  The function has several code and logic bugs, which prevent it from working at all:

  * `vpindexToConnect.back()` is passed to `CheckForkWarningConditionsOnNewFork`, which is the earliest connected block (least work block), *not* the new fork tip
  * `ActivateBestChainStep` will never try to connect a block that descends from an invalid block, so the invalid fork will only ever be of height 1, never hitting the 7 block minimum condition

  Instead of dragging the dead and wrong code around through every change in validation, remove it. In the future it could make sense to add a fork detection somewhere outside of the `ActivateBestChainStep` logic (maybe net_processing).

ACKs for top commit:
  jnewbery:
    utACK fa7eed5be7
  fjahr:
    Code review ACK fa7eed5be7
  glozow:
    utACK fa7eed5be7 I see that it's dead code

Tree-SHA512: 815bdbac7c1eb5b7594b0866a2dbd3c7619797afaadb03a5269fb96739ffb83b05b8e4f7c1e68d48d7886132dd0b12c14c3fb4ee0e72de1074726050ed203e1a
2020-11-18 11:23:56 +01:00
MarcoFalke
54532f46c4 Merge bitcoin-core/gui#109: wallet: Remove unused AskPassphraseDialog::Decrypt
4146a31ccb qt, wallet: Drop unused parameter in WalletModel::setWalletEncrypted (Hennadii Stepanov)
f886a20b02 qt, wallet: Drop unused parameter in Wallet{Frame|View}::encryptWallet (Hennadii Stepanov)
6e950118a3 qt, wallet: Remove unused AskPassphraseDialog::Decrypt (Hennadii Stepanov)

Pull request description:

  Grabbed from #42 with an additional commit.

  Fix #1.

ACKs for top commit:
  MarcoFalke:
    ACK 4146a31ccb
  promag:
    Code review ACK 4146a31ccb.

Tree-SHA512: 6070d8995525af826ad972cf1b8988ff98af0528eef285a07ec7ba0e2e92a7a6173a19dc371de94d4b437fa10f7921166e45a081de6ed2f4306e6502aafc94ee
2020-11-18 11:19:42 +01:00
MarcoFalke
4ae04090bb Merge bitcoin-core/gui#118: Remove BDB version from the Information tab
e4fc45a011 gui: Remove BDB version from the Information tab (Hennadii Stepanov)

Pull request description:

  Master (67d4643a1a):
  ![DeepinScreenshot_select-area_20201027161350](https://user-images.githubusercontent.com/32963518/97313812-b90cea80-186f-11eb-8fec-781b0724bd9c.png)

  This PR:
  ![DeepinScreenshot_select-area_20201027161449](https://user-images.githubusercontent.com/32963518/97313883-c924ca00-186f-11eb-8f82-ebb9f68414b1.png)

ACKs for top commit:
  achow101:
    ACK e4fc45a011
  jonasschnelli:
    utACK e4fc45a011
  Sjors:
    utACK e4fc45a
  promag:
    ACK e4fc45a011.
  jarolrod:
    Tested ACK e4fc45a

Tree-SHA512: 3660b78607a4dc364b7cb2bc74ccd8b57acd7a16084151e58945e1a92a9af97a37edb33e94a87067fabfe9ce1c45be11fab270bc146f742a2f1010530470c433
2020-11-18 11:18:02 +01:00
Wladimir J. van der Laan
daf1ebf0b1 doc: Remove generated manual pages from master branch
Replace the generated manual pages on the master branch with
placeholders with instructions. The master branch is too much in flux
for anything generated from command output to be kept up to date,
and having pages from versions ago looks silly.

We can't remove them completely because `make dist` relies
on the files being present.

Resolves #20031.
2020-11-18 11:12:46 +01:00
MarcoFalke
4b24c3962f Merge #19504: Bump minimum python version to 3.6
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them (Anthony Towns)
8ae9d314e9 Bump minimum python version to 3.6 (Anthony Towns)

Pull request description:

  Python 3.5 has reached [end-of-life](https://devguide.python.org/#status-of-python-branches) as of September 2020, and 3.6 has some moderately nice [features](https://docs.python.org/3/whatsnew/3.6.html):

  - `f'x = {x}'` as an alternative to `'x = {}'.format(x)` format strings (cf https://github.com/bitcoin/bitcoin/pull/13718#issuecomment-406591027)
  - underscore separators for large numbers, like `1_234_567`
  - improvements to async
  - improvements to typing module

  Note that 3.6 is not available in xenial (16.04), but is available in bionic (18.04), while focal (20.04) has 3.8. CentOS 7 and 8 have 3.6.8, Debian stable has 3.7.3, and [gentoo and arch already had 3.6 and 3.7 in 2018](https://github.com/bitcoin/bitcoin/pull/14954#issuecomment-447118707).

ACKs for top commit:
  MarcoFalke:
    re-ACK 97c738ff1b

Tree-SHA512: ec7fce68845edde4d61a42de12c065fd49e5217311a6fda1323206f091a0afd50f293645dffc27d420127e4e5deb864e953f1b67eff735a0dfbbedd7899a9d60
2020-11-18 10:24:22 +01:00
Wladimir J. van der Laan
132e1d897f build: Bump master version to 0.21.99
Tree-SHA512: 94c258b234b2412d92f312a1b38adf17249664a9e3e321de0ff683b59a48cee192cd42da5220df0726a782d98776610f4420534b3a1c51f4cf4a0180d5835622
2020-11-18 10:06:03 +01:00
MarcoFalke
fac4e136fa refactor: Change pointer to reference because it can not be null 2020-11-18 08:33:26 +01:00
Anthony Towns
9d09132be4 CConnman: initialise at declaration rather than in Start()
Ensure nMaxOutboundTotalBytesSentInCycle and nMaxOutboundCycleStartTime
are initialized even if CConnman::Start() is not called.
2020-11-18 02:29:33 +10:00
fanquake
831675c8dc Merge #20401: qt: Pre-splitoff translations update
bb6441b7a4 qt: Pre-splitoff translations update (Wladimir J. van der Laan)

Pull request description:

  0.21 split-off should be near now. Let's do one final translations update just before the split-off.
  (Hopefully it won't take too long, but might want to keep this open to be the last thing merged)

ACKs for top commit:
  hebasto:
    ACK bb6441b7a4
  MarcoFalke:
    ACK bb6441b7a4 (checked that only changes are translation changes in `src/qt`)

Tree-SHA512: 3273246923d3020e1f7ae46cbb59f1ed45a35acb5e1582b55486c5723f5aa1e5809fe2fd87b1ac34d308eef2902e621d0ace97181a044262b2c8f002bf50daac
2020-11-17 20:57:36 +08:00
MarcoFalke
80e32e120e Merge #20305: wallet: introduce fee_rate sat/vB param/option
05e82d86b0 wallet: override minfee checks (fOverrideFeeRate) for fee_rate (Jon Atack)
9a670b4f07 wallet: update sendtoaddress, send RPC examples with fee_rate (Jon Atack)
be481b72e2 wallet: use MIN_RELAY_TX_FEE in bumpfee help (Jon Atack)
449b730579 wallet: provide valid values if invalid estimate mode passed (Jon Atack)
6da3afbaee wallet: update remaining rpcwallet fee rate units to BTC/kvB (Jon Atack)
173b5b5fe0 wallet: update fee rate units, use sat/vB for fee_rate error messages (Jon Atack)
7f9835a05a wallet: remove fee rates from conf_target helps (Jon Atack)
b7994c01e9 wallet: add fee_rate unit warnings to bumpfee (Jon Atack)
410e471fa4 wallet: remove redundant bumpfee fee_rate checks (Jon Atack)
a0d4957473 wallet: introduce fee_rate (sat/vB) param/option (Jon Atack)
e21212f01b wallet: remove unneeded WALLET_BTC_KB_TO_SAT_B constant (Jon Atack)
6112cf20d4 wallet: add CFeeRate ctor doxygen documentation (Jon Atack)
3f72791613 wallet: fix bug in RPC send options (Jon Atack)

Pull request description:

  This PR builds on #11413 and #20220 to address #19543.

  - replace overloading the conf_target and estimate_mode params with `fee_rate` in sat/vB in the sendtoaddress, sendmany, send, fundrawtransaction, walletcreatefundedpsbt, and bumpfee RPCs

  - allow non-actionable conf_target value of `0` and estimate_mode value of `""` to be passed to use `fee_rate` as a positional argument, in addition to as a named argument

  - fix a bug in the experimental send RPC described in https://github.com/bitcoin/bitcoin/pull/20220#discussion_r513789526 where args were not being passed correctly into the options values

  - update the feerate error message units for these RPCs from BTC/kB to sat/vB

  - update the test coverage, help docs, doxygen docs, and some of the RPC examples

  - other changes to address the excellent review feedback

  See this wallet meeting log for more context: http://www.erisian.com.au/bitcoin-core-dev/log-2020-11-06.html#l-309

ACKs for top commit:
  achow101:
    re-ACK 05e82d8
  MarcoFalke:
    review ACK 05e82d86b0 did not test and found a few style nits, which can be fixed later 🍯
  Xekyo:
    tACK 05e82d86b0
  Sjors:
    utACK 05e82d86b0

Tree-SHA512: a4ee5f184ada53f1840b2923d25873bda88c5a2ae48e67eeea2417a0b35154798cfdb3c147b05dd56bd6608a784e1b91623bb985ee2ab9ef2baaec22206d0a9c
2020-11-17 13:49:12 +01:00
MarcoFalke
e7986c51bc Merge bitcoin-core/gui#96: Slight improve create wallet dialog
ac64cec4ce gui: create wallet: add advanced section (Sjors Provoost)
c99d6f644a gui: create wallet: name placeholder (Sjors Provoost)
5bff82540b [gui] create wallet: smarter checkbox toggling (Sjors Provoost)

Pull request description:

  Previously only users who needed a second wallet had to use to the create wallet dialog. With the merge of https://github.com/bitcoin/bitcoin/pull/15454 now all new users have to. I don't think it was user-friendly enough for that.

  <img width="403" alt="Schermafbeelding 2020-09-18 om 09 41 44" src="https://user-images.githubusercontent.com/10217/93574129-52ef9680-f998-11ea-9a6f-31144f66d3bf.png">

  This PR makes a few simple improvements so that new users don't have to think too much:

  <img width="369" alt="Schermafbeelding 2020-10-15 om 16 45 22" src="https://user-images.githubusercontent.com/10217/96145959-0c914700-0f06-11eb-9526-cf447d841d7a.png">

  It's lightly inspired by #77. It would be better if those changes made it into the upcoming release, but this PR is a good start imo.

  * wallet encryption is no longer checked by default, because such a change in the default needs a separate discussion (fwiw, I suspect it increases the number of users losing access to coins)
  * watch-only and descriptor wallet stuff is moved to advanced, so new users know they can safely ignore these check boxes
  * bonus: when you click on "disable private keys" it disables encrypt wallet and checks blank wallet
  * label changes: see screenshot
  * tooltip changes: see code diff

  Note that a blank wallet name isn't allowed in the dialog; I haven't addressed that.

  _Update 2020-10-30_, dropped the new strings for now:
  <img width="450" alt="Schermafbeelding 2020-10-30 om 11 26 55" src="https://user-images.githubusercontent.com/10217/97694591-1b99fc80-1aa3-11eb-8b85-e19f1ad5add4.png">

ACKs for top commit:
  fjahr:
    Tested ACK ac64cec4ce
  jonatack:
    re-ACK ac64cec4ce, per `git diff d393708 ac64cec` only change since my last review is improving the placeholder from "MyWallet" to "Wallet" and dropping the last commit. Tested creating a dozen wallets in signet with different combinations of options and then verifying/comparing their characteristics in the console with getwalletinfo. My remaining caveats are (1) the need for less user surprise by either (a) improving the user info or (b) with less auto-(un)selecting as mentioned in https://github.com/bitcoin-core/gui/pull/96#issuecomment-727017409 and (2) I prefer the "Encrypt private keys" and "Watch-only" wording and descriptions below over the current ones; hopefully these can be addressed in a follow-up.
  hebasto:
    re-ACK ac64cec4ce
  ryanofsky:
    Code review ACK ac64cec4ce. Only changes since last review are tweaking placeholder text and dropping "allow nameless" commit

Tree-SHA512: a25f84eb66ee4f99af441d73e33928df9d9cf592177398ef48f0037f5913699e47a162cf1301c83b34501546d43ff4ae12607fd078c5c03b92f573bf7604a9f2
2020-11-17 13:03:59 +01:00
MarcoFalke
c463f70fb0 Merge #20139: Wallet: do not return warnings from UpgradeWallet()
9636962889 [upgradewallet] removed unused warning param (Sishir Giri)

Pull request description:

  The `warning` variable was unused in `upgradewallet` so I removed it

ACKs for top commit:
  practicalswift:
    ACK 9636962889: diff looks correct
  MarcoFalke:
    review ACK 9636962889
  jonatack:
    ACK 9636962889

Tree-SHA512: 1d63186ce1e05e86a778340f2d7986c2cee1523de0a11cea39e8d148ac7ee26c49741dfa302b5c1cd1c8d74e67c1f9baee2763720c2d850b57da9a3fdce24565
2020-11-17 12:43:43 +01:00
fanquake
3457054c61 Merge #20346: script: modify security-check.py to use "==" instead of "is" for literal comparison
b6121edf70 swapped "is" for "==" in literal comparison (Tyler Chambers)

Pull request description:

  In Python 3.8+ literal comparisons using "is" instead of "==" produce a SyntaxWarning [source](https://docs.python.org/3.8/whatsnew/3.8.html#changes-in-python-behavior).

  I checked the entire devtools directory, this seems to be the only occurrence.

  This is a small fix, but removes the SyntaxWarning.
  Fixes: #20338

ACKs for top commit:
  hebasto:
    re-ACK b6121edf70, only squashed since my [previous](https://github.com/bitcoin/bitcoin/pull/20346#pullrequestreview-525934568) review.
  practicalswift:
    re-ACK b6121edf70: patch still looks correct
  theStack:
    utACK b6121edf70

Tree-SHA512: 82a43495d6552fbaa3b02b58f0930b049d27aa937fe44b47714e3c059f844cc494de20674557371cbccf24fb8873ecb7376fb965ae326847eed2b855ed2d59c6
2020-11-17 13:57:40 +08:00
fanquake
7c0d412a74 Merge #20405: p2p: avoid calculating onion address checksum when version is not 3
d355a302d9 Break circuit earlier (lontivero)

Pull request description:

  Currently when parsing an onion v3 address the pubic key checksum is calculated in order to compare it with the received address checksum. However this step is not necessary if the address version byte is not 3, in which case the method can return with false immediately.

ACKs for top commit:
  jonatack:
    ACK d355a302d9
  practicalswift:
    ACK d355a302d9 -- patch looks correct
  hebasto:
    ACK d355a302d9, I have reviewed the code and it looks OK, I agree it can be merged.
  sipa:
    utACK d355a302d9

Tree-SHA512: 9e4506793b7f4a62ce8edc41a260a8c125ae81ed2f90cd850eb2a9214d323c446edc7586c7b0590dcbf3aed5be534718b77bb19c45b48f8f52553d32a3663a65
2020-11-17 13:56:22 +08:00
Sishir Giri
9636962889 [upgradewallet] removed unused warning param 2020-11-16 13:22:42 -08:00
lontivero
d355a302d9 Break circuit earlier
There is no need to calculate the full checksum for an Tor v3 onion
address if the version byte is not the expected one.
2020-11-16 15:54:24 -03:00
Michael Dietz
21f2433601 test: run mempool_spend_coinbase.py even with wallet disabled 2020-11-16 09:05:34 -06:00
Wladimir J. van der Laan
bb6441b7a4 qt: Pre-splitoff translations update 2020-11-16 11:15:11 +01:00
Wladimir J. van der Laan
c48e788246 Merge #18836: wallet: upgradewallet fixes and additional tests
5f9c0b6360 wallet: Remove -upgradewallet from dummywallet (MarcoFalke)
a314271f08 test: Remove unused wallet.dat (MarcoFalke)
bf7635963c tests: Test specific upgradewallet scenarios and that upgrades work (Andrew Chow)
4b418a9dec test: Add test_framework/bdb.py module for inspecting bdb files (Andrew Chow)
092fc43485 tests: Add a sha256sum_file function to util (Andrew Chow)
0bd995aa19 wallet: upgrade the CHDChain version number when upgrading to split hd (Andrew Chow)
8e32e1c41c wallet: remove nWalletMaxVersion (Andrew Chow)
bd7398cc62 wallet: have ScriptPubKeyMan::Upgrade check against the new version (Andrew Chow)
5f720544f3 wallet: Add GetClosestWalletFeature function (Andrew Chow)
842ae3842d wallet: Add utility method for CanSupportFeature (Andrew Chow)

Pull request description:

  This PR cleans up the wallet upgrade mechanism a bit, fixes some probably bugs, and adds more test cases.

  The `nWalletMaxVersion` member variable has been removed as it made `CanSupportFeature` unintuitive and was causing a couple of bugs. The reason this was introduced originally was to allow a wallet upgrade to only occur when the new feature is first used. While this makes sense for the old `-upgradewallet` option, for an RPC, this does not quite make sense. It's more intuitive for an upgrade to occur if possible if the `upgradewallet` RPC is used as that's an explicit request to upgrade a particular wallet to a newer version. `nWalletMaxVersion` was only relevant for upgrades to `FEATURE_WALLETCRYPT` and `FEATURE_COMPRPUBKEY` both of which are incredibly old features. So for such wallets, the behavior of `upgradewallet` will be that the feature is enabled immediately without the wallet needing to be encrypted at that time (note that `FEATURE_WALLETCRYPT` indicates support for encryption, not that the wallet is encrypted) or for a new key to be generated.

  `CanSupportFeature` would previously indicate whether we could upgrade to `nWalletMaxVersion` not just whether the current wallet version supported a feature. While this property was being used to determine whether we should upgrade to HD and HD chain split, it was also causing a few bugs. Determining whether we should upgrade to HD or HD chain split is resolved by passing into `ScriptPubKeyMan::Upgrade` the version we are upgrading to and checking against that. By removing `nWalletMaxVersion` we also fix a bug where you could upgrade to HD chain split without the pre-split keypool.

  `nWalletMaxVersion` was also the version that was being reported by `getwalletinfo` which meant that the version reported was not always consistent across restarts as it depended on whether `upgradewallet` was used. Additionally to make the wallet versions consistent with actually supported versions, instead of just setting the wallet version to whatever is given to `upgradewallet`, we normalize the version number to the closest supported version number. For example, if given 150000, we would store and report 139900.

  Another bug where CHDChain was not being upgraded to the version supporting HD chain split is also fixed by this PR.

  Lastly several more tests have been added. Some refactoring to the test was made to make these tests easier. These tests check specific upgrading scenarios, such as from non-HD (version 60000) to HD to pre-split keypool. Although not specifically related to `upgradewallet`, `UpgradeKeyMetadata` is now being tested too.

  Part of the new tests is checking that the wallet files are identical before and after failed upgrades. To facilitate this, a utility function `sha256sum_file` has been added. Another part of the tests is to examine the wallet file itself to ensure that the records in the wallet.dat file have been correctly modified. So a new `bdb.py` module has been added to deserialize the BDB db of the wallet.dat file. This format isn't explicitly documented anywhere, but the code and comments in BDB's source code in file `dbinc/db_page.h` describe it. This module just dumps all of the fields into a dict.

ACKs for top commit:
  MarcoFalke:
    approach ACK 5f9c0b6360
  laanwj:
    Code review ACK 5f9c0b6360
  jonatack:
    ACK 5f9c0b6360, approach seems fine, code review, only skimmed the test changes but they look well done, rebased on current master, debug built and verified the `wallet_upgradewallet.py` test runs green both before and after running `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2`

Tree-SHA512: 7c4ebf420850d596a586cb6dd7f2ef39c6477847d12d105fcd362abb07f2a8aa4f7afc5bfd36cbc8b8c72fcdd1de8d2d3f16ad8e8ba736b6f4f31f133fe5feba
2020-11-16 11:03:25 +01:00
MarcoFalke
1e17114917 Merge #20238: doc: Missing comments for signet parameters
9c08f3332c doc: Missing comments for signet parameters (kanon)

Pull request description:

  We have such comment in chainparams.cpp. However in Signet the comments are missing.

  In example...

  - Mainnet
  d67883d01e/src/chainparams.cpp (L83-L84)

  - Testnet
  d67883d01e/src/chainparams.cpp (L196-L197)

  - Regtest
  d67883d01e/src/chainparams.cpp (L392-L393)

ACKs for top commit:
  theStack:
    ACK 9c08f3332c

Tree-SHA512: d4e488cf01e50d6320282b29d776c11e6b3d423f9268226749f738a57a51f456b6bd48334d2d5a43afa782df65ea15525a0af1688003c1be6ef915c05650e147
2020-11-16 10:40:56 +01:00
MarcoFalke
37a4634811 Merge #20390: CI/Cirrus: Skip merge_base step for non-PRs
20e491ddcb CI/Cirrus: Skip merge_base step for non-PRs (Luke Dashjr)

Pull request description:

  CIRRUS_BASE_BRANCH is a PR-specific variable and undocumented on non-PR builds.
  In practice (at the moment), it seems to be HEAD, which in private repositories can be pretty much anything, causing CI to fail if it can't be cleanly merged.

  By checking CIRRUS_PR first, we can reliably do CI builds of branches outside PRs.

ACKs for top commit:
  MarcoFalke:
    review ACK 20e491ddcb

Tree-SHA512: 9fd8db2e19a3145f7dccfca107631b20df8c94d385f624e2bcef2fa18e38bf3e23c6c68fc8241decedbf1413bf69ca572cff75e1ccf82c09ac50443001ec5ae5
2020-11-16 07:59:43 +01:00
MarcoFalke
2fa085a5d7 Merge #20033: refactor: minor whitespace fixups, s/const/constexpr/ and remove template (followup to #19845)
89836a82ee style: minor improvements as a followup to #19845 (Vasil Dimov)

Pull request description:

  Address suggestions:
  https://github.com/bitcoin/bitcoin/pull/19845#discussion_r495486760
  https://github.com/bitcoin/bitcoin/pull/19845#discussion_r495488051
  https://github.com/bitcoin/bitcoin/pull/19845#discussion_r495730125

ACKs for top commit:
  jonatack:
    re-ACK 89836a8 change since previous review is replacing std::runtime_error with std::exception, built/ran unit tests with gcc debian 10.2.0-15, then broke a few v3 net_tests involving `BOOST_CHECK_EXCEPTION`, rebuilt, ran `src/test/test_bitcoin -t net_tests -l all` and checked the error reporting.
  hebasto:
    re-ACK 89836a82ee
  theStack:
    ACK 89836a82ee

Tree-SHA512: 36477fdccabe5a8ad91fbabb4655cc363a3a7ca237a98ae6dd4a9fae4a4113762040f864d4ca13a47d081f7d16e5bd487edbfb61ab50a37e4a0424e9bec30b24
2020-11-16 07:57:34 +01:00
MarcoFalke
0ede354367 Merge #20386: Silence false positive GCC warning in wallet/rpcwallet.cpp
049feabf28 Add missing optional.h include (Kristaps Kaupe)
29c66ace5c Silence false positive GCC warning (Kristaps Kaupe)

Pull request description:

  Resolves #20381.

ACKs for top commit:
  jnewbery:
    utACK 049feabf28
  practicalswift:
    ACK 049feabf28: diagnostics signal to noise is increased by getting rid of false positives
  hebasto:
    ACK 049feabf28, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 05d84f51521c3b843ed6bf284a83a91db015ad0cd4fcf8b602275812575c1f6b4899286a89d360fbd3caef184abdfb9d834e119842d8740919892f05a0f9e1f8
2020-11-15 18:33:59 +01:00
Hennadii Stepanov
03edb52eee qt: Remove redundant BitcoinGUI::setTrayIconVisible
The removed BitcoinGUI::setTrayIconVisible just duplicates
QSystemTrayIcon::setVisible.
2020-11-15 19:12:19 +02:00
Hennadii Stepanov
17174f8328 gui: Replace "Hide tray icon" option with positive "Show tray icon" one
This change makes easier both (1) using this option, and (2) reasoning
about the code.
2020-11-15 19:07:06 +02:00
MarcoFalke
fb7726e56d Merge #20395: ci: Use the previous build worker image in AppVeyor
406097c810 ci: Use the previous build worker image in AppVeyor (Hennadii Stepanov)

Pull request description:

  This is a workaround as the [recent](https://www.appveyor.com/updates/2020/11/14/) Visual Studio 2019 image update breaks our builds.

  This PR is alternative to #20392 due to its build [failure](https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/36314660).

ACKs for top commit:
  MarcoFalke:
    review ACK 406097c810 also seems to pass

Tree-SHA512: f9b722d8e67dd7f0745de6da385064630adb27ecbc0a919be47f62217a2bb7a27a6fa00a7536a24bf17500a77160ca3b92b3c8619047171a6f5198b434015221
2020-11-15 09:44:05 +01:00
Hennadii Stepanov
406097c810 ci: Use the previous build worker image in AppVeyor
This is a workaround as the recent Visual Studio 2019 image update
breaks our builds.
2020-11-15 09:12:06 +02:00
Luke Dashjr
20e491ddcb CI/Cirrus: Skip merge_base step for non-PRs
CIRRUS_BASE_BRANCH is a PR-specific variable and undocumented on non-PR builds.
In practice (at the moment), it seems to be HEAD, which in private repositories can be pretty much anything, causing CI to fail if it can't be cleanly merged.

By checking CIRRUS_PR first, we can reliably do CI builds of branches outside PRs.
2020-11-14 17:51:59 +00:00
Sjors Provoost
ac64cec4ce gui: create wallet: add advanced section 2020-11-14 13:07:49 +01:00
Sjors Provoost
c99d6f644a gui: create wallet: name placeholder 2020-11-14 13:07:49 +01:00
Kristaps Kaupe
049feabf28 Add missing optional.h include 2020-11-14 10:56:38 +02:00
Kristaps Kaupe
29c66ace5c Silence false positive GCC warning 2020-11-14 03:20:07 +02:00
fanquake
e1f2553e11 build: remove global_init_link_order from mac qt qmake.conf
This has been around since the original import of Qt
(38be0d13830efd2d98281c645c3a60afe05ffece), however there
are now only two instatnces of it left in the qt codebase,
and from what I can gather, it's unused.
2020-11-14 08:54:24 +08:00
fanquake
498fa16bea build: document preprocessing steps in qt package 2020-11-14 08:54:24 +08:00
fanquake
bd5d9336d9 build: don't copy Info.plist.* into mkspec for macOS qt build
We generate our own Info.plist as part of make deploy, and as far as I
can tell, it doesn't seem to have an effect wether these are present
during qt's build.

I also can't find a single mention of the .app plist in the qt code,
whereas there are multiple instances of .lib.
2020-11-14 08:54:24 +08:00
fanquake
bfd7e33b4b build: remove plugin_no_soname from mac qt qmake.conf
plugin_no_soname was removed from Qt some time ago, see upstream commit
1d034244c261520d5e739534dc264c2500e02b5f. It was replaced with
plugin_with_soname, however that is currently only used (as of 5.15.x)
in the Android Clang mkspec.
2020-11-14 08:54:24 +08:00
fanquake
fdde4c7ce6 build: pass XCODE_VERSION through to qt macOS cross compile conf
This should mostly be a no-op, however it would seem to make more sense
that we pass through the XCODE_VERSION we now have in depends, rather
than leaving the version set to 4.3.
2020-11-14 08:54:24 +08:00
fanquake
49473ef211 build: convert "echo" usage into a patch in qt package 2020-11-14 08:54:17 +08:00
Ivan Metlushko
173cc9b7be test: walettool create descriptors 2020-11-13 17:52:28 +07:00
Jonas Schnelli
543693b92b Merge #20378: wallet: fix potential division by 0 in WalletLogPrintf
440f8d3abe fix potential devision by 0 (Jonas Schnelli)

Pull request description:

  #20344 removed the divide-by-zero sanitizer suppression in `wallet/wallet.cpp` but kept a potential devision by zero in `wallet.cpp`'s fee logging.

  Detected here https://bitcoinbuilds.org/index.php?job=ffb7d59f-379f-4f27-a273-a5595b8c5f07

ACKs for top commit:
  practicalswift:
    ACK 440f8d3abe
  laanwj:
    Code review ACK 440f8d3abe
  hebasto:
    re-ACK 440f8d3abe

Tree-SHA512: 9f7903d1e567497c5f972d39e9629c059151e705dbed0a6b88f7c6650c50ecf820f78e3e0f3e629c661d45a938c5d7659faae7c61e47ca8b3bdb029661bca55a
2020-11-13 08:26:57 +01:00
MarcoFalke
28cb61646b Merge #19065: tests: Add fuzzing harness for CAddrMan
d04a17a790 fuzz: Use ConsumeRandomLengthBitVector(...) in src/test/fuzz/connman and src/test/fuzz/net (practicalswift)
e6bb9fde85 tests: Add fuzzing harness for CAddrMan (practicalswift)

Pull request description:

  Add fuzzing harness for `CAddrMan`.

  ~~Fill some fuzzing coverage gaps for functions in `addrdb.h`, `merkleblock.h` and `outputtype.h`.~~

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    review ACK d04a17a790

Tree-SHA512: a6b627e3a0cb51e3a8cb02ad0f19088fc0e965ca34ab110b68d5822d0ea7f473207ae312b49fb217cb6cf2f9f211d00bb69c83bac9f50d79c9ed1e157e85775d
2020-11-13 07:40:35 +01:00
MarcoFalke
99fcc2b477 Merge #20379: tests: Remove no longer needed UBSan suppression (float divide-by-zero in validation.cpp)
0ccb3addf6 tests: Remove no longer needed UBSan suppression (float-divide-by-zero in validation.cpp) (practicalswift)

Pull request description:

  Remove no longer needed UBSan suppression.

  The float divide-by-zero in `validation.cpp` was fixed by instagibbs in ec30a79f1c (#15283).

ACKs for top commit:
  MarcoFalke:
    ACK 0ccb3addf6

Tree-SHA512: 89a4f4b7371fa5725d9f801cee7ebbd17523f66017c9acfa813657dcb8d837f42209eff44ce9e5d48296a630bab9599d75f10024a0c7da7defb228f4eae3392a
2020-11-13 07:19:23 +01:00
practicalswift
fd0be92cff doc: Add instructions on how to fuzz the P2P layer using Honggfuzz NetDriver 2020-11-12 20:20:29 +00:00
practicalswift
0ccb3addf6 tests: Remove no longer needed UBSan suppression (float-divide-by-zero in validation.cpp) 2020-11-12 18:58:59 +00:00
Jonas Schnelli
440f8d3abe fix potential devision by 0 2020-11-12 17:11:18 +01:00
Wladimir J. van der Laan
0bd4929cd0 Merge #20284: addrman: ensure old versions don't parse peers.dat
38ada892ed addrman: ensure old versions don't parse peers.dat (Vasil Dimov)

Pull request description:

  Even though the format of `peers.dat` was changed in a backwards
  incompatible way, it is not guaranteed that old versions will fail to
  parse it. There is a chance that old versions parse its contents as
  garbage and use it.

  Old versions expect the "key size" field to be 32 and fail the parsing
  if it is not. Thus, we put something other than 32 in it. This will make
  versions between 0.11.0 and 0.20.1 deterministically fail on the new
  format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
  will still parse it as garbage.

  Also, introduce a way to increment the `peers.dat` format in a way that
  does not necessary make older versions refuse to read it.

ACKs for top commit:
  jnewbery:
    ACK 38ada892ed
  laanwj:
    Code review ACK 38ada892ed
  MarcoFalke:
    re-ACK 38ada892ed 🥐

Tree-SHA512: 550bd660c5019dba0f9c334aca8a11c4a0463cfddf11efe7a4a5585ffb05549c82b95066fba5d073ae37893e0eccc158a7ffea9b33ea031d9be4a39e44f6face
2020-11-12 17:05:51 +01:00
practicalswift
d04a17a790 fuzz: Use ConsumeRandomLengthBitVector(...) in src/test/fuzz/connman and src/test/fuzz/net 2020-11-12 15:33:43 +00:00
practicalswift
e6bb9fde85 tests: Add fuzzing harness for CAddrMan 2020-11-12 14:23:17 +00:00
Jonas Schnelli
9bd1316697 Merge bitcoin-core/gui#120: Fix multiwallet transaction notifications
241434200e refactor: qt: Use vQueueNotifications.clear() (João Barbosa)
989e579d07 qt: Make transaction notification queue wallet specific (João Barbosa)
7b3b2303f4 move-only: Define TransactionNotification before  TransactionTablePriv (João Barbosa)

Pull request description:

  Currently `vQueueNotifications` holds transactions of any wallet, but the queue is dispatched on a given wallet and it assumes notifications are of that wallet.

  This means that some transactions can be missed if multiple wallets are loaded.

  Fix this by having a queue for each wallet.

ACKs for top commit:
  jonasschnelli:
    utACK 241434200e
  hebasto:
    ACK 241434200e, I have reviewed the code and it looks OK, I agree it can be merged.
  ryanofsky:
    Code review ACK 241434200e. Only change is dropping one commit

Tree-SHA512: 61beac5a16ed659e3a25ad145dbceafcef963aaf8f9838355298949ec2324e2bd760f59353cd251d30cf0334d8dc1642a1f3821d8a9eec092533b581f6ce86db
2020-11-12 12:23:55 +01:00
Jon Atack
05e82d86b0 wallet: override minfee checks (fOverrideFeeRate) for fee_rate
in RPCs fundrawtransaction and walletcreatefundedpsbt only.

This provides the existing feeRate (BTC/kvB) behavior in these two RPCs to the
new fee_rate (sat/vB) param also.

See these two GitHub review discussions for more info:
https://github.com/bitcoin/bitcoin/pull/10706/#discussion_r126560525
https://github.com/bitcoin/bitcoin/pull/20305#discussion_r520032533
2020-11-12 11:44:15 +01:00
Jon Atack
9a670b4f07 wallet: update sendtoaddress, send RPC examples with fee_rate 2020-11-12 11:43:22 +01:00
Jon Atack
be481b72e2 wallet: use MIN_RELAY_TX_FEE in bumpfee help
Co-authored-by: Murch <murch@murch.one>
2020-11-12 11:43:19 +01:00
Jon Atack
449b730579 wallet: provide valid values if invalid estimate mode passed
Co-authored-by: Murch <murch@murch.one>
2020-11-12 11:43:17 +01:00
Jon Atack
6da3afbaee wallet: update remaining rpcwallet fee rate units to BTC/kvB 2020-11-12 11:43:14 +01:00
Jon Atack
173b5b5fe0 wallet: update fee rate units, use sat/vB for fee_rate error messages
and BTC/kvB for feeRate error messages.
2020-11-12 11:43:03 +01:00
MarcoFalke
8a486158cb Merge #20188: tests: Add fuzzing harness for CConnman
79ef8324d4 tests: Add fuzzing harness for CConnman (practicalswift)

Pull request description:

  Add fuzzing harness for `CConnman`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    review ACK 79ef8324d4

Tree-SHA512: eb9ffae20e939b818f8b9def064544b9a8fcd127ca22d1a54af1afedf1d24143be42419f3a03d684be59a5ff07b29d8bfa34ef2aaf1d9f9f75c4c1aaa90a29a8
2020-11-12 10:06:42 +01:00
MarcoFalke
af8ec1d3e5 Merge #20375: fuzz: Improve coverage for CPartialMerkleTree fuzzing harness
3c77b8009d fuzz: Improve coverage for CPartialMerkleTree fuzzing harness (practicalswift)

Pull request description:

  Improve coverage for `CPartialMerkleTree` fuzzing harness.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    ACK 3c77b8009d

Tree-SHA512: a1fa0f7650a5ee5ff83f35e41b9faf6c34671fc304b9af00e5b83073f21d50bcbe91c2428fa64d05dc42a7c521bfd24031e307c7f4abf9ded469d69a55c5d64a
2020-11-12 09:57:35 +01:00
MarcoFalke
027e51f715 Merge #20372: Avoid signed integer overflow when loading a mempool.dat file with a malformed time field
ee11a412a5 Avoid signed integer overflow when loading a mempool.dat file with a malformed time field (practicalswift)

Pull request description:

  Avoid signed integer overflow when loading a `mempool.dat` file with a malformed time field.

  Avoid the following signed integer overflow:

  ```
  $ xxd -p -r > mempool.dat-crash-1 <<EOF
  0100000000000000000000000004000000000000000000000000ffffffff
  ffffff7f00000000000000000000000000
  EOF
  $ cp mempool.dat-crash-1 ~/.bitcoin/regtest/mempool.dat
  $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" src/bitcoind -regtest
  validation.cpp:5079:23: runtime error: signed integer overflow: 9223372036854775807 + 1209600 cannot be represented in type 'long'
      #0 0x5618d335197f in LoadMempool(CTxMemPool&) src/validation.cpp:5079:23
      #1 0x5618d3350df3 in CChainState::LoadMempool(ArgsManager const&) src/validation.cpp:4217:9
      #2 0x5618d2b9345f in ThreadImport(ChainstateManager&, std::vector<boost::filesystem::path, std::allocator<boost::filesystem::path> >, ArgsManager const&) src/init.cpp:762:33
      #3 0x5618d2b92162 in AppInitMain(util::Ref const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*)::$_14::operator()() const src/init.cpp:1881:9
  ```

  This PR was broken out from PR #20089. Hopefully this PR is trivial to review.

  Fixes a subset of #19278.

ACKs for top commit:
  MarcoFalke:
    review ACK ee11a412a5
  Crypt-iQ:
    crACK ee11a412a5

Tree-SHA512: 227ab95cd7d22f62f3191693b455eacfa8e36534961bee12c622fc9090957cfb29992eabafa74d806a336e03385aa8f98b7ce734f04b0b400e33aa187d353337
2020-11-12 09:34:48 +01:00
MarcoFalke
bcd142e479 Merge #20285: Remove references to CreateWalletFromFile
c82336c493 Remove references to CreateWalletFromFile (fanquake)

Pull request description:

  `CWallet::CreateWalletFromFile()` was removed in 8b5e7297c0 but these references remain.

ACKs for top commit:
  hebasto:
    ACK c82336c493

Tree-SHA512: 3dd50fe0cd5a60bbc96d265107d4739f3e08f943435f3772038963ac4be9e4a87a863412ac0d571226ea66d71550b17b52f01b9d46a6282d49feae1508fd682e
2020-11-12 09:32:56 +01:00
fanquake
c82336c493 Remove references to CreateWalletFromFile
CWallet::CreateWalletFromFile() was removed in
8b5e7297c0 but these references remain.
2020-11-12 13:12:29 +08:00
Samuel Dobson
c2d8ba6265 Merge #19502: Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks
24d2d3341d QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored (Luke Dashjr)
69f59af54d Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Luke Dashjr)

Pull request description:

  Previously, an exception would be thrown, which could kill the node in some circumstances.

  Includes test changes to cause failure.

  Review with `?w=1`

ACKs for top commit:
  hebasto:
    re-ACK 24d2d3341d, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/19502#pullrequestreview-520552944) review.
  promag:
    Tested ACK 24d2d3341d, test change fails on master.
  meshcollider:
    utACK 24d2d3341d

Tree-SHA512: f701f81b3aa3d3e15cee52ac9e7c31a73c0d8166e56bf077235294507cbcee099829fedc432a1c4b6d8780885f4e37897b44b980b08125771de3c849c000499e
2020-11-12 13:26:38 +13:00
practicalswift
3c77b8009d fuzz: Improve coverage for CPartialMerkleTree fuzzing harness 2020-11-11 22:31:40 +00:00
MarcoFalke
d9f5132736 Merge #20344: wallet: fix scanning progress calculation for single block range
5e146022da wallet: fix scanning progress calculation for single block range (Sebastian Falbesoner)

Pull request description:

  If the blockchain is rescanned for a single block (i.e. start and stop hashes are equal, and with that also the estimated start/stop verification progress values) the progress calculation could lead to a NaN value caused by a division by zero (0.0/0.0), resulting in an invalid JSON result for the `getwalletinfo` RPC.  This PR fixes this behaviour by setting the progress to zero in that special case. Fixes #20297.

  The behaviour can easily be reproduced by continuously running single block rescans in an endless loop, e.g. via
  ```bash
  #!/bin/bash
  while true
  do
      bitcoin-cli rescanblockchain $(bitcoin-cli getblockcount)
  done
  ```

  and at the same time perform some `getwalletinfo` RPCs.

  On the master branch, this leads to frequent invalid responses (tested on mainchain):
  ```
  $ bitcoin-cli getwalletinfo
  error: couldn't parse reply from server
  $ curl --user `cat ~/.bitcoin/.cookie` --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getwalletinfo", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  {"result":{"walletname":"","walletversion":169900,"format":"bdb","balance":0.00000000,"unconfirmed_balance":0.00000000,"immature_balance":0.00000000,"txcount":0,"keypoololdest":1603677276,"keypoolsize":1000,"hdseedid":"3196e33ecb47c7130e6ca60f2f895f9259860dca","keypoolsize_hd_internal":1000,"paytxfee":0.00000000,"private_keys_enabled":true,"avoid_reuse":false,"scanning":{"duration":0,"progress":},"descriptors":false},"error":null,"id":"curltest"}
  ```
  (note that missing value for "progress" in the JSON result).

  On the PR branch, the behaviour doesn't occur anymore.

ACKs for top commit:
  MarcoFalke:
    review ACK 5e146022da
  promag:
    Core review ACK 5e146022da.

Tree-SHA512: f0e6aad5a6cd08b36c5fe820fff0ef26663229b39169a4dbe757f3c795a41cf5c69c9dc90efe7515675ae1059307f8971123781a0514d10704123a6f28b125ab
2020-11-11 16:11:01 +01:00
Vasil Dimov
38ada892ed addrman: ensure old versions don't parse peers.dat
Even though the format of `peers.dat` was changed in an incompatible
way (old software versions <0.21 cannot understand the new file format),
it is not guaranteed that old versions will fail to parse it. There is a
chance that old versions parse its contents as garbage and use it.

Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
(<0.11.0) will still parse it as garbage.

Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
2020-11-11 16:05:15 +01:00
Jon Atack
7f9835a05a wallet: remove fee rates from conf_target helps 2020-11-11 15:56:05 +01:00
Jon Atack
b7994c01e9 wallet: add fee_rate unit warnings to bumpfee 2020-11-11 15:56:03 +01:00
Jon Atack
410e471fa4 wallet: remove redundant bumpfee fee_rate checks
SetFeeEstimateMode() handles these checks now and provides a more actionable
error message.
2020-11-11 15:56:01 +01:00
Jon Atack
a0d4957473 wallet: introduce fee_rate (sat/vB) param/option
Create a fee_rate (sat/vB) RPC param and replace overloading the conf_target and
estimate_mode params in the following 6 RPCs with it:

- sendtoaddress
- sendmany
- send
- fundrawtransaction
- walletcreatefundedpsbt
- bumpfee

In RPC bumpfee, the previously existing fee_rate remains but the unit is changed
from BTC/kvB to sat/vB. This is a breaking change, but it should not be an
overly risky one, as the units change by a factor of 1e5 and any fees specified
in BTC/kvB after this commit will either be too low and raise an error or be 1
sat/vB and can be RBFed.

Update the test coverage for each RPC.

Co-authored-by: Murch <murch@murch.one>
2020-11-11 15:55:59 +01:00
Jon Atack
e21212f01b wallet: remove unneeded WALLET_BTC_KB_TO_SAT_B constant 2020-11-11 15:55:56 +01:00
Jon Atack
6112cf20d4 wallet: add CFeeRate ctor doxygen documentation
as requested by reviewers
2020-11-11 15:55:53 +01:00
Jon Atack
3f72791613 wallet: fix bug in RPC send options
when empty, options were not being populated by arguments of the same name

found while adding test coverage in 603c0050
2020-11-11 15:55:46 +01:00
MarcoFalke
179ece4273 Merge #20218: test: Suppress epoll_ctl data race
fa949b3c13 test: Suppress epoll_ctl data race (MarcoFalke)

Pull request description:

  Happens intermittently: https://cirrus-ci.com/task/5462892373868544?command=ci#L5385

ACKs for top commit:
  hebasto:
    ACK fa949b3c13, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: d5aa559fc105053da594531722f2a03d898eadeb4413c3a728fc5116cc4d1a2c16c49649a24c75ea810e4ec6bb9728b0bcd2ea991886bb9d206170218eddf6d2
2020-11-11 15:45:19 +01:00
practicalswift
ee11a412a5 Avoid signed integer overflow when loading a mempool.dat file with a malformed time field 2020-11-11 14:45:16 +00:00
MarcoFalke
155bf91c3b Merge #20368: ci: Remove redundant valgrind fuzz task
fa92cf29d9 ci: Remove redundant valgrind fuzz task (MarcoFalke)

Pull request description:

  This task has several issues:

  * It slows down other tasks and times out: It needs a lot of resources (CPU, RAM, time), because it builds more than 100 fuzzers, clones a 2 GB repo with 100k seeds and pipes them all through valgrind
  * It doesn't add a lot of value: Except for one issue in the boost time library, it hasn't found any issues that the existing fuzz,asan,ubsan fuzzer has already found
  * It is redundant: It is already run in the bitcoin-core/qa-assets repo on every push of new seeds and once daily

  Fix all issues by removing it here.

Top commit has no ACKs.

Tree-SHA512: 76d16a3e5afc79ba9d89cfeb915d7e66fd1ad4e2035ae4ccd30a21cc060bfba3fb6a904346b7c41606e69d972e86bf660df962673028689227cc26072bba24bd
2020-11-11 14:21:37 +01:00
Sebastian Falbesoner
5e146022da wallet: fix scanning progress calculation for single block range
If the blockchain is rescanned for a single block (i.e. start and stop hashes
are equal, and with that also the estimated verification progress) the progress
calculation could lead to a NaN value caused by a division by zero, resulting in
an invalid JSON result for the getwalletinfo RPC.  Fixed by setting the progress
to zero in that special case.

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-11-11 13:15:00 +01:00
MarcoFalke
fa949b3c13 test: Suppress epoll_ctl data race 2020-11-11 10:10:31 +01:00
MarcoFalke
fa92cf29d9 ci: Remove redundant valgrind fuzz task 2020-11-11 09:03:11 +01:00
Ivan Metlushko
345e88eecf wallettool: add param to create descriptors wallet 2020-11-11 11:41:53 +07:00
Ivan Metlushko
6d3af3ab62 wallettool: pass in DatabaseOptions into MakeWallet 2020-11-11 11:40:02 +07:00
fanquake
7087440894 depends: native_ds_store 1.3.0
native_ds_store now takes advantage of Pythons ability to decode binary
plists (since 3.4), so we can drop its biplist dependency.

The call to biplist.Data() in custom_dsstore doesn't seem to do anything,
and from what I can tell can just be removed.
2020-11-11 08:01:02 +08:00
Andrew Chow
281fd1a4a0 Replace KeyIDHasher with SaltedSipHasher 2020-11-10 14:33:37 -05:00
Andrew Chow
210b693db6 Add generic SaltedSipHasher
SaltedSipHasher is a generic hasher that can be used with most things we
would hash in an unordered container.
2020-11-10 14:33:37 -05:00
Andrew Chow
95e61c1cf2 Move Hashers to util/hasher.{cpp/h}
Move the hashers that we use for hash tables to a common place.

Moved hashers:
- SaltedTxidHasher
- SaltedOutpointHasher
- FilterHeaderHasher
- SignatureCacheHasher
- BlockHasher
2020-11-10 14:33:37 -05:00
MarcoFalke
fa8dd34e91 Merge #20332: test: Mock IBD in net_processing fuzzers
fa4234d877 test: Mock IBD in net_processing fuzzers (MarcoFalke)

Pull request description:

  Without this the fuzzers fail to detect trivial crasher bugs, such as https://github.com/bitcoin/bitcoin/pull/20317#issuecomment-723047111

ACKs for top commit:
  practicalswift:
    Tested ACK fa4234d877

Tree-SHA512: ce5da5c0a604b7559805a98ffdde882b44ca4f91b003b493d6e1be230714ce4cccb11dbfc1fc175f9d8fc779551c0a4103ceb4b473552928207d7d78ae329e10
2020-11-10 19:51:11 +01:00
MarcoFalke
42f950cb27 Merge #20322: test: Fix intermittent issue in wallet_listsinceblock
444412821e test: Fix intermittent issue in wallet_listsinceblock (MarcoFalke)

Pull request description:

ACKs for top commit:
  Empact:
    Code Review ACK 444412821e

Tree-SHA512: 86d47b1e3c8681dd479654589c894016ac81a3c96a34c3b4a75278b2af85054ea8c6f768e518a5322a4928d82d5e99105bbce0f4fa6a7a18c40e3e0799f9ab54
2020-11-10 09:20:51 +01:00
MarcoFalke
0b69bb90ee Merge #20355: fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet
79b8f8d574 fuzz: Assert roundtrip equality for both addrv1 and addrv2 versions of CService (practicalswift)
0e3a78a8ab fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet (practicalswift)

Pull request description:

  Check for `addrv1` compatibility before using `addrv1` serializer/deserializer on `CSubNet`. As requested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/20289#issuecomment-724012969.

  Assert roundtrip equality for both `addrv1` and `addrv2` versions of `CService`.

ACKs for top commit:
  MarcoFalke:
    review ACK 79b8f8d574

Tree-SHA512: 3f758aa89ab0c253b593fbe8fe9adc5c6db9afec8856facfe635053a32b4feb438c951323ae0c9e27f1d7e89d12a9b62d81f094dc96159233c12f64d4b95c290
2020-11-10 07:57:02 +01:00
Carl Dong
46756a6987 depends: Fix PYTHONPATH setting in config.site.in
Previously, when running ./configure:

1. With CONFIG_SITE pointed to our depends config.site.in, and
2. PYTHONPATH was not set either in the environment or by the user

The configure would output something like:

PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages:'

When we really mean:

PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages'

...without the colon

This change makes sure that:

1. There's no trailing colon, and
2. We use the $PATH_SEPARATOR variable instead of a colon
2020-11-09 16:58:28 -05:00
Carl Dong
618cbd2c1a lint: Also lint files with shellcheck directive
Files like config.site.in are not referenced by any other script in our
tree, so we need to mark it manually with a "shellcheck shell="
directive and make sure that shellcheck is run on them.
2020-11-09 16:58:27 -05:00
Carl Dong
6c7e8f067d depends: Allow relative CONFIG_SITE path env var
Previously, if ./configure was invoked with:

```
$ env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
```

Where $CONFIG_SITE was a relative path, ./configure would fail with the
following misleading output:

```
checking for boostlib >= 1.58.0 (105800)... yes
checking whether the Boost::System library is available... yes
configure: error: Could not find a version of the Boost::System library!
```

Fully resolving depends_prefix in config.site.in fixes this. To make
sure that there are no other side effects I ran a diff on the
config.status generated by:

1. The scripts prior to this change with CONFIG_SITE set to a full path:
       env CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
2. The scripts after this change with CONFIG_SITE set to a relative path:
       env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure

And it looks good!

Diff: https://paste.sr.ht/~dongcarl/95b469fbc555c128046e85723d87a9082a754f6b
2020-11-09 16:58:26 -05:00
Fabrice Fontaine
330cb33985 src/randomenv.cpp: fix build on uclibc
Check for HAVE_STRONG_GETAUXVAL or HAVE_WEAK_GETAUXVAL before using
getauxval to avoid a build failure on uclibc

Signed-off-by: Fabrice Fontaine <fontaine.fabrice@gmail.com>
2020-11-09 21:21:30 +01:00
Wladimir J. van der Laan
1dfe19e284 Merge #20153: wallet: do not import a descriptor with hardened derivations into a watch-only wallet
538be4219a wallet: fix importdescriptor silent fail (Ivan Metlushko)

Pull request description:

  Currently `importdescriptor` command will successfully import a descriptor with hardened derivations into a watch-only wallet while silently failing to expand the descriptor to fill the cache. This leads to a broken wallet state and failure to load such wallet due to missing cache on subsequent restart.

ACKs for top commit:
  laanwj:
    Code review ACK 538be4219a
  achow101:
    ACK 538be4219a
  meshcollider:
    utACK 538be4219a

Tree-SHA512: 4bdd0ab4437d55b3f1a79c3a300a0b186089155c020fe220a73d0cce274de47d90371d88918d39fd795f9fccf8db328f1e322d29a6062f9ce94a1c254398f004
2020-11-09 20:19:00 +01:00
Wladimir J. van der Laan
79a3b59cc7 Merge #20120: net, rpc, test, bugfix: update GetNetworkName, GetNetworksInfo, regression tests
7b5bd3102e test: add getnetworkinfo network name regression tests (Jon Atack)
9a75e1e569 rpc: update GetNetworksInfo() to not return unsupported networks (Jon Atack)
ba8997fb2e net: update GetNetworkName() with all enum Network cases (Jon Atack)

Pull request description:

  Following up on the BIP155 addrv2 changes, and starting with 7be6ff6 in #19845, RPC getnetworkinfo began returning networks with empty names.

  <details><summary><code>getnetworkinfo</code> on current master</summary><p>

  ```
    "networks": [
      {
        "name": "ipv4",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "ipv6",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "onion",
        "limited": false,
        "reachable": true,
        "proxy": "127.0.0.1:9050",
        "proxy_randomize_credentials": true
      },
      {
        "name": "",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      }
    ],
  ```
  </p></details>

  <details><summary><code>getnetworkinfo</code> on this branch</summary><p>

  ```
    "networks": [
      {
        "name": "ipv4",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "ipv6",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "onion",
        "limited": false,
        "reachable": true,
        "proxy": "127.0.0.1:9050",
        "proxy_randomize_credentials": true
      }
    ],
  ```
  </p></details>

  This patch:
  - updates `GetNetworkName()` to the current Network enum
  - updates `getNetworksInfo()` to ignore as-yet unsupported networks
  - adds regression tests

ACKs for top commit:
  hebasto:
    re-ACK 7b5bd3102e
  vasild:
    ACK 7b5bd3102

Tree-SHA512: 8f12363eb430e6f45e59e3b1d69c2f2eb5ead254ce7a67547d116c3b70138d763157335a3c85b51f684a3b3b502c6aace4f6fa60ac3b988cf7a9475a7423c4d7
2020-11-09 17:07:23 +01:00
practicalswift
79b8f8d574 fuzz: Assert roundtrip equality for both addrv1 and addrv2 versions of CService 2020-11-09 15:29:15 +00:00
practicalswift
0e3a78a8ab fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CSubNet 2020-11-09 15:27:41 +00:00
Tyler Chambers
b6121edf70 swapped "is" for "==" in literal comparison
update lint-python.sh to include check F632
2020-11-09 10:21:51 -05:00
Wladimir J. van der Laan
4fd37d0a10 Merge #20292: test: Fix intermittent feature_taproot issue
fab900802d ci: Bump timeout factor (MarcoFalke)
50eb0c2512 Small improvements to the Taproot functional tests (Pieter Wuille)
fac865b72d test: Fix intermittent feature_taproot issue (MarcoFalke)
fa1dea19fc test: Fix deser issue in create_block (MarcoFalke)
fa762a3fd4 test: Remove unused unnamed parameter from block.serialize call (MarcoFalke)

Pull request description:

  This fixes three bugs. Also, fix some unrelated code style issues.

  Please refer to the commit messages for more information.

ACKs for top commit:
  laanwj:
    Code review ACK fab900802d

Tree-SHA512: 4e22c240cf345710f3b21fc63243126b90014b3656d0865ff87156e958dd1442e6572c6c0a5701dbbe503eee931a0ceb66eeeb3553137f3d1f5afd27a9f9cada
2020-11-09 15:47:04 +01:00
MarcoFalke
05aeeee34f Merge #20339: ci: Run more ci configs on cirrus
77777c8b5e ci: Run windows ci config on cirrus (MarcoFalke)
3333d6942e ci: Run macos ci config on cirrus (MarcoFalke)
fa8b1114e6 ci: Run arm ci config on cirrus (MarcoFalke)
fa0795f54d ci: Replace TRAVIS_OS_NAME with CI_OS_NAME (MarcoFalke)
fafce1a13a ci: Move documentation to correct config file (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK 77777c8b5e

Tree-SHA512: 581d3bdb2c3e3da20bd8492c6b23d90f3b4f7f9300ade384667e41de9fbbe42bbcef26dd99328839dc0b49be6e303c4c6d1b66ceb5d8009800c52c97685f9080
2020-11-09 15:31:20 +01:00
Wladimir J. van der Laan
663fd92b28 Merge #20266: wallet: fix change detection of imported internal descriptors
bd93fc9945 Fix change detection of imported internal descriptors (Andrew Chow)

Pull request description:

  Import internal descriptors were having address book entries added which meant they would be detected as non-change. Fix this and add a test for it.

ACKs for top commit:
  laanwj:
    Code review ACK bd93fc9945
  meshcollider:
    utACK bd93fc9945
  promag:
    Code review ACK bd93fc9945.

Tree-SHA512: 8fa9e364be317627ec171eedffdb505976c0e7f1e55bc7e8cfdffa3aeea5db24d231f55166602cd0e97a5ba621acc871de0a765c75d0c65678f83e93c3b657c5
2020-11-09 15:14:45 +01:00
Wladimir J. van der Laan
f70eb51b05 Merge #20318: build: Ensure source tarball has leading directory name
faa2f06f5e scripted-diff: [build] Ensure source tarball has leading directory name (MarcoFalke)

Pull request description:

  This has been fixed in 0.20, so it needs to be fixed on master as well to avoid a regression

  #18945

ACKs for top commit:
  laanwj:
    ACK faa2f06f5e
  hebasto:
    ACK faa2f06f5e, tested gitian builds only.
  promag:
    ACK faa2f06f5e.

Tree-SHA512: e3b025c29c45b025002abc35262bb5d771f6cbd807f1c256c477c243685e93cd43ad9f642b38e3cf218590912abe6ea0ddfec3bfbef36f99080aad74ed6cc0af
2020-11-09 15:06:20 +01:00
Anthony Towns
7abac98d3e configure: Support -f{debug,macro}-prefix-map
When bitcoin is checked out in two directories (eg via git worktree)
object files between the two will differ due to the full path being
included in the debug section. -fdebug-prefix-map is used to replace
this with "." to avoid this unnecessary difference and allow ccache to
share objects between worktrees (provided the source and compile options
are the same).

Also provide -fmacro-prefix-map if supported so that the working dir is
not encoded in __FILE__ macros.
2020-11-09 20:15:41 +10:00
MarcoFalke
77777c8b5e ci: Run windows ci config on cirrus 2020-11-09 10:16:45 +01:00
MarcoFalke
3333d6942e ci: Run macos ci config on cirrus 2020-11-09 10:15:59 +01:00
MarcoFalke
fa8b1114e6 ci: Run arm ci config on cirrus 2020-11-09 10:15:50 +01:00
MarcoFalke
fa0795f54d ci: Replace TRAVIS_OS_NAME with CI_OS_NAME
Also enable free disk space check unconditionally
2020-11-09 10:15:48 +01:00
MarcoFalke
fafce1a13a ci: Move documentation to correct config file 2020-11-09 10:15:13 +01:00
Anthony Towns
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them 2020-11-09 17:53:50 +10:00
Anthony Towns
8ae9d314e9 Bump minimum python version to 3.6 2020-11-09 17:53:47 +10:00
João Barbosa
9b74461fa2 refactor: Assert before dereference in CWallet::GetDatabase 2020-11-07 11:40:27 +00:00
João Barbosa
021feb3187 refactor: Drop redudant CWallet::GetDBHandle 2020-11-07 11:35:20 +00:00
MarcoFalke
fa4234d877 test: Mock IBD in net_processing fuzzers 2020-11-07 07:50:59 +01:00
MarcoFalke
7e373294a5 Merge #20315: travis: Remove s390x build
fa2c3c0d96 ci: Set LC_ALL=C to allow running the s390x tests in qemu (MarcoFalke)
fac0517836 travis: Remove s390x build (MarcoFalke)

Pull request description:

  This has been discussed in the last meeting.

  Refer to the commit body for more details.

Top commit has no ACKs.

Tree-SHA512: 8e0455286ce41c95ed2e5eb624ac534251bb4a321f13d26d14356497e0c39f841372e166373ffd4a0a9fa379636c2cfb535bd92534fff427cdcb827354e66b6c
2020-11-06 18:08:32 +01:00
MarcoFalke
4727c1ca24 Merge #20328: cirrus: Skip tasks on the gui repo main branch
66667acc53 cirrus: Skip tasks on the gui repo main branch (MarcoFalke)

Pull request description:

  No need to run every build twice, once in the main repo and then in the read-only gui mirror repo

ACKs for top commit:
  decryp2kanon:
    ACK 66667ac
  hebasto:
    ACK 66667acc53, though still preferring `only_if` as showing skipped tasks as successful ones seems a bit confused.

Tree-SHA512: 0d35bd115152e06ba4dc5f364130ba5496167d960c44eac2c76192ff9bf7c51f46ab72e2d054dcc6a91818a18dffbbc262f8a4c4483857158c0af4f55dfe9b28
2020-11-06 18:03:49 +01:00
Dmitry Petukhov
dc80a7d0b0 docs/descriptors.md: Remove hardened marker in the path after xpub
As described in "Key origin identification" section, a descriptor
that has hardened derivation after xpub does not let you compute scripts
without access to the corresponding private keys. Such a descriptor is
practically useless.

The text after the descriptor said "with child key *1'/2* of the
specified xpub", and clearly an xpub cannot have "child key" with
hardened derivation. Therefore it makes sense to fix this inconsistency
to not confuse the reader of the doc
2020-11-06 14:27:47 +05:00
MarcoFalke
66667acc53 cirrus: Skip tasks on the gui repo main branch 2020-11-06 10:05:30 +01:00
fanquake
a0c00ff7c0 Merge #20298: macOS deploy: use the new plistlib API
04a69c200e macOS deploy: use the new plistlib API (Jonas Schnelli)

Pull request description:

  See https://docs.python.org/3/library/plistlib.html.
  The old API was deprecated in 3.4 and removed in 3.9.

  ~~AFAIK the macdeployplus scripts is only used when calling `make deploy` locally (on macOS). The linux cross compile build (like gitian) are not affected by this PR.~~

ACKs for top commit:
  fanquake:
    ACK 04a69c200e - I checked that `make deploy` on macOS currently fails when building master and using Python 3.9. This PR fixes that, and it's fine to use (and backport) these changes as they only require Python 3.4. Related note: I think we could just about drop our native_biplist dependency entirely given some changes upstream.
  practicalswift:
    ACK 04a69c200e: patch looks correct

Tree-SHA512: c5bb60c5157b371d680c82e0978470a488f3edc58cd09e1be635fed59420f227dd113e901c28e15a463da6fe81dc64d08a701b1fdfeb4502f418785707dbebbc
2020-11-06 15:34:55 +08:00
MarcoFalke
c51c2753a4 Merge #20326: tests: Fix ecdsa_verify in test framework
568a1d7261 fix ecdsa verify in test framework (Stepan Snigirev)

Pull request description:

  This PR fixes a small bug in the test framework in `verify_ecdsa` function.
  `r` in ecdsa signature is modulo curve order, so if the point `R` calculated during verification has x-coordinate that is larger than the curve order, the verification will fail in the test framework but pass in libsecp256k1.

  Example (all in hex):
  public key: `0289d889551598a0263746c01e5882ccf9b7dc4ca5a37108482c9d80de40e0a8cf`
  der signature: `3006020104020104` (r = 4, s = 4)
  message: `3232323232323232323232323232323232323232323232323232323232323232`

  libsecp256k1 returns `true`, test framework returns `false`.

ACKs for top commit:
  sipa:
    utACK 568a1d7261

Tree-SHA512: 9e9c58498f10085d2ad85e95caff6c92793799d2a40696ef43febcd7d313c8c3d5ecec715ca903cbb8432a8a96bd0065d86d060966d4ee651c3871ce16c252bf
2020-11-06 08:08:27 +01:00
MarcoFalke
65460c207c Merge #20324: wallet: Set DatabaseStatus::SUCCESS in MakeSQLiteDatabase
faf5fa7413 wallet: Set DatabaseStatus::SUCCESS in MakeSQLiteDatabase (MarcoFalke)

Pull request description:

  This is a refactor to set the status to `SUCCESS` (like it is done in `MakeBerkeleyDatabase`, too). It also happens to fix a false positive valgrind warning (tested with bionic-gcc and focal-clang):

  ```
   node1 stderr ==28149== Conditional jump or move depends on uninitialised value(s)
  ==28149==    at 0x464471: LoadWallets(interfaces::Chain&) (load.cpp:105)
  ==28149==    by 0x44BFBA: interfaces::(anonymous namespace)::WalletClientImpl::load() (wallet.cpp:510)
  ==28149==    by 0x1640F9: AppInitMain(util::Ref const&, NodeContext&, interfaces::BlockAndHeaderTipInfo*) (init.cpp:1815)
  ==28149==    by 0x144F3F: AppInit (bitcoind.cpp:142)
  ==28149==    by 0x144F3F: main (bitcoind.cpp:172)
  ==28149==
  {
     <insert_a_suppression_name_here>
     Memcheck:Cond
     fun:_Z11LoadWalletsRN10interfaces5ChainE
     fun:_ZN10interfaces12_GLOBAL__N_116WalletClientImpl4loadEv
     fun:_Z11AppInitMainRKN4util3RefER11NodeContextPN10interfaces21BlockAndHeaderTipInfoE
     fun:AppInit
     fun:main
  }

  TEST                                             | STATUS    | DURATION

  wallet_hd.py --descriptors                       | ✖ Failed  | 69 s
  ```

ACKs for top commit:
  achow101:
    ACK faf5fa7413

Tree-SHA512: e8cbac195d05518467f89725d413bdf226d74671eba1c1eb80b3a61d65724af75a1fe93bcb5c608eaa0d54eddce992738bd923e7d83e493f54c3f4c67b66408c
2020-11-06 08:02:00 +01:00
Luke Dashjr
24d2d3341d QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored 2020-11-06 04:35:33 +00:00
Luke Dashjr
69f59af54d Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks 2020-11-06 04:17:54 +00:00
Stepan Snigirev
568a1d7261 fix ecdsa verify in test framework 2020-11-05 23:16:55 +01:00
MarcoFalke
faf5fa7413 wallet: Set DatabaseStatus::SUCCESS in MakeSQLiteDatabase 2020-11-05 20:48:41 +01:00
MarcoFalke
444412821e test: Fix intermittent issue in wallet_listsinceblock 2020-11-05 19:51:57 +01:00
MarcoFalke
f5cdc290d5 Merge #20316: test: Fix wallet_multiwallet test issue on Windows
fa00ff0399 test: Fix wallet_multiwallet test issue on Windows (MarcoFalke)

Pull request description:

  Fixes:

  ```
  Traceback (most recent call last):
    File "test\functional\test_framework\test_framework.py", line 126, in main
      self.run_test()
    File "test/functional/wallet_multiwallet.py", line 120, in run_test
      assert_equal(sorted(map(lambda w: w['name'], self.nodes[0].listwalletdir()['wallets'])), sorted(in_wallet_dir))
    File "test\functional\test_framework\util.py", line 49, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
  AssertionError: not(['', 'sub\\w5', 'w', 'w1', 'w2', 'w3', 'w7', 'w7_symlink', 'w8'] == ['', 'sub/w5', 'w', 'w1', 'w2', 'w3', 'w7', 'w7_symlink', 'w8'])

ACKs for top commit:
  promag:
    ACK fa00ff0399.

Tree-SHA512: 7a809a352677a216465cef59e866e4881272e302e897cebf7d9645bf87aebeaf54435bb0692bb5c1381c2dd680e8a34e640ea18ca6e2a4087e3233cd9c24ed04
2020-11-05 18:45:05 +01:00
MarcoFalke
faa2f06f5e scripted-diff: [build] Ensure source tarball has leading directory name
-BEGIN VERIFY SCRIPT-
sed -i 's|git archive --|git archive --prefix="${DISTNAME}/" --|g'                          $(git grep -l 'git archive' ./contrib)
sed -i 's|tar -xf "\?${\?GIT_ARCHIVE}\?"\?|tar --strip-components=1 -xf "${GIT_ARCHIVE}"|g' $(git grep -l 'tar -xf'     ./contrib)
-END VERIFY SCRIPT-
2020-11-05 17:31:24 +01:00
MarcoFalke
9bb078351b Merge #20308: wallet: Set bilingual error completely
090b8385af Set bilingual error completely (Hennadii Stepanov)

Pull request description:

  Fix https://github.com/bitcoin-core/gui/issues/128

ACKs for top commit:
  MarcoFalke:
    review ACK 090b8385af
  practicalswift:
    ACK 090b8385af: patch looks correct!

Tree-SHA512: ef400291a866c3116377a4439a23de89a1c5e3ef4597d682138f88d90612846aabb31228b98a8722e7f58b4b499a58adc732bc40ac28fae6d18fce1d4953c96a
2020-11-05 14:51:37 +01:00
MarcoFalke
fa00ff0399 test: Fix wallet_multiwallet test issue on Windows 2020-11-05 13:50:15 +01:00
MarcoFalke
d94777bd52 Merge #20302: net: Make it easier to reason about node eviction by removing unused NodeEvictionCandidate::addr (CAddress)
f1f433e8ca Make it easier to reason about node eviction by removing unused NodeEvictionCandidate::addr (CAddress) (practicalswift)

Pull request description:

  Make it easier to reason about node eviction by removing unused `NodeEvictionCandidate::addr` (`CAddress`).

ACKs for top commit:
  jnewbery:
    utACK f1f433e8ca

Tree-SHA512: fef91d7b412b8a4f172370cff6c37eb8c3db0ba618f5daf2dcc8737c8fcef7b9b820d7ee99cd0a9eae7dd653a096cf83d5113776b0d1d9a324147581674e9ede
2020-11-05 13:00:13 +01:00
MarcoFalke
fa2c3c0d96 ci: Set LC_ALL=C to allow running the s390x tests in qemu
Without the fix, the tests immediately abort:

terminate called after throwing an instance of 'std::runtime_error'
  what():  locale::facet::_S_create_c_locale name not valid
./qt/test/test_bitcoin-qt: line 2: 116150 Aborted
2020-11-05 12:54:51 +01:00
MarcoFalke
fac0517836 travis: Remove s390x build
The build didn't find too many issues, and they are generally only
endianness related, so easy to fix if they do make it into the main
developement branch.

With travis migrating its service, it fails too often intermittently. It
might be more appropriate to run it manually once before every release.
2020-11-05 12:00:18 +01:00
Hennadii Stepanov
090b8385af Set bilingual error completely 2020-11-05 11:28:37 +02:00
MarcoFalke
f33e332541 Merge #20303: fuzz: Assert expected DecodeHexTx behaviour when using legacy decoding
d7901ab8d2 fuzz: Assert expected DecodeHexTx behaviour when using legacy decoding (practicalswift)

Pull request description:

  Assert expected `DecodeHexTx` behaviour when using legacy decoding.

  As suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/20290#issuecomment-720989597.

ACKs for top commit:
  MarcoFalke:
    review ACK d7901ab8d2

Tree-SHA512: 3285680059e6fa73b0fb2c52b775f6319de1ac616f731206662b742764dc888cdfd1ac1f1fcfdfd5418d2006475a852d1c1a56a7035f772f0a6b2a84f5de93bc
2020-11-05 07:57:28 +01:00
fanquake
6954e4d16c Merge #20283: test: Only try witness deser when checking for witness deser failure
fae45c34d1 test: Only try witness deserialize when checking for witness deserialize failure (MarcoFalke)

Pull request description:

  Witness deserialize will fail always. (This is what the test is checking for)

  Consequently, non-witness deserialize is also tried, and it might succeed accidentally. Avoid that by not trying non-witness deserialize.

  Fixes #20249

ACKs for top commit:
  jnewbery:
    utACK fae45c34d1

Tree-SHA512: 45e65b31603e3ca839776a7ed30e363b32eba20dfb67b7b55bff06715876850d4f6ba22f8ea4911a62e1f8ffff395bf187b23c46ddc766516b97057df000deb3
2020-11-05 14:56:02 +08:00
MarcoFalke
83650e4df5 Merge #20199: wallet: ignore (but warn) on duplicate -wallet parameters
58cfbc38e0 Ignoring (but warn) on duplicate -wallet parameters (Jonas Schnelli)

Pull request description:

  I expect that there are many users with load on startup wallet definitions in `bitcoin.conf` or via startup CLI argument.
  With the new `settings.json` r/w configuration file, users unloading and loading a wallet through the GUI or via the RPC calls might end up with a duplicate `-wallet` entry (one that still remains in bitcoin.conf or CLI) plus the new duplication in `settings.json` due to the unload/load.

  Steps to reproduce
  * create wallet (if via RPC set `load_on_startup` or unloadwallet/loadwallet then set `load_on_startup`).
  * stop bitcoin
  * start bitcoind again with same `--wallet=mywallet`

  I guess it is acceptable to skip duplicates.

ACKs for top commit:
  achow101:
    Tested ACK 58cfbc38e0
  meshcollider:
    Code review ACK 58cfbc38e0
  ryanofsky:
    Code review ACK 58cfbc38e0. Changes since previous review: rebased, tweaked warning message, squashed/fixed test

Tree-SHA512: f94e5a999bdd7dc291f0bc142911b0a8033929350d6f6a35b58c4a06a3c8f83147be0f0c402d4e946dedbbcc85b7e023b672c731b6d7a8984d4780017c961cfb
2020-11-05 07:51:07 +01:00
practicalswift
d7901ab8d2 fuzz: Assert expected DecodeHexTx behaviour when using legacy decoding 2020-11-04 23:11:50 +00:00
MarcoFalke
6760088015 Merge #20300: fuzz: Add missing ECC_Start to descriptor_parse test
5cafe2b25c fuzz: Add missing ECC_Start to descriptor_parse test (Ivan Metlushko)

Pull request description:

  Fixes fuzzing harness.

  I also observed that the corpus for this test consists only of `xprv...` keys while we are using regtest parameters. So for proper fuzzing we need either A) to update the corpus and replace `xprv...` with `tprv...` B) switch to main net in the test

ACKs for top commit:
  MarcoFalke:
    review ACK 5cafe2b25c
  practicalswift:
    Tested ACK 5cafe2b25c

Tree-SHA512: 7415a98a445ce0f96219637d2362fecfc1191ad104f55d79ca92b0c92cde165e00646be5bf3fda956385e3cb22540eca457e575048493367cdf0e00a27d7cdb8
2020-11-04 20:38:18 +01:00
MarcoFalke
deb2b27c0d Merge #20294: ci: Run more ci configs on cirrus
fa8e494554 ci: Run ci configs on cirrus (MarcoFalke)

Pull request description:

  Now that cirrus ci runs more stable than travis ci, we can try to move more configs over there to see if any issues arise.

ACKs for top commit:
  practicalswift:
    ACK fa8e494554: patch looks correct
  decryp2kanon:
    reACK fa8e494554

Tree-SHA512: e2d1838050b6199d11fa06d1cc9d804883ec5df7d65386c950e8124c0067dc1aaa62ec84c9842c8263e2cf5b17fc819ce85689338113f8d69edb1954f06e76e2
2020-11-04 20:13:08 +01:00
MarcoFalke
5f9c0b6360 wallet: Remove -upgradewallet from dummywallet 2020-11-04 12:16:57 -05:00
MarcoFalke
a314271f08 test: Remove unused wallet.dat 2020-11-04 12:16:57 -05:00
Andrew Chow
bf7635963c tests: Test specific upgradewallet scenarios and that upgrades work 2020-11-04 12:16:54 -05:00
Andrew Chow
4b418a9dec test: Add test_framework/bdb.py module for inspecting bdb files
For upgrade tests and possibly other tests, it is useful to inspect the
bdb file for the wallet (i.e. the wallet.dat file).
test_framework/bdb.py is an implementation of bdb file deserialization
specific for Bitcoin Core's usage.
2020-11-04 12:15:29 -05:00
Andrew Chow
092fc43485 tests: Add a sha256sum_file function to util 2020-11-04 12:15:25 -05:00
Andrew Chow
0bd995aa19 wallet: upgrade the CHDChain version number when upgrading to split hd 2020-11-04 12:15:14 -05:00
Andrew Chow
8e32e1c41c wallet: remove nWalletMaxVersion
nWalletMaxVersion was used to allow an upgrade to a version only
when the new feature was used. This makes sense for the old
-upgradewallet startup option. But because upgradewallet is now a RPC,
putting off the version bump like this does not make sense. Instead,
immediately upgrading to the given version number makes sense.
2020-11-04 12:15:12 -05:00
Andrew Chow
bd7398cc62 wallet: have ScriptPubKeyMan::Upgrade check against the new version
Instead of using CanSupportFeature and relying on nWalletMaxVersion,
take the new version we are upgrading to and use IsSupportedFeature
with that and the previous wallet version.
2020-11-04 12:10:23 -05:00
MarcoFalke
ed9f547750 Merge #20299: test: Fix intermittent rpc_net issue
fa2ecadd0d test: Fix intermittent rpc_net issue (MarcoFalke)

Pull request description:

  The test fails because getpeerinfo and getnettotals are not synchronised, so a `wait_until` is needed for each RPC (separately).

  Fixes https://cirrus-ci.com/task/4663366629195776?command=ci#L5034

ACKs for top commit:
  jnewbery:
    utACK fa2ecadd0d

Tree-SHA512: 5ea7128801aab8dbe3d9e6737545ff4ee770e4a9c5a2096ba2339a688424f1879ccba6bf8bcb219983acf86eb28af06fc629586613e7fe28aeffadd2c98633e8
2020-11-04 17:16:41 +01:00
Ivan Metlushko
5cafe2b25c fuzz: Add missing ECC_Start to descriptor_parse test 2020-11-04 22:55:03 +07:00
MarcoFalke
1209b6c692 Merge #20212: net: fix output of peer address in version message
af3b0dfc54 net: fix output of peer address in version message (Vasil Dimov)

Pull request description:

  If `-logips -debug=net` is specified then we print the contents of the
  version message we send to the peer, including his address. Because the
  addresses in the version message use pre-BIP155 encoding they cannot
  represent a Tor v3 address and we would actually send 16 `0`s instead (a
  dummy IPv6 address). However we would print the full address in the log
  message. Before this fix:

  ```
  2020-10-21T12:24:17Z send version message: version 70016, blocks=653500, us=[::]:0, them=xwjtp3mj427zdp4tljiiivg2l5ijfvmt5lcsfaygtpp6cw254kykvpyd.onion:8333, peer=0
  ```

  This is confusing because we pretend to send one thing while we actually
  send another. Adjust the printout to reflect what we are sending. After
  this fix:

  ```
  2020-10-21T12:26:54Z send version message: version 70016, blocks=653500, us=[::]:0, them=[::]:0, peer=0
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK af3b0dfc54
  jnewbery:
    utACK af3b0dfc54

Tree-SHA512: f169d7b4f07c219e541f7c37ea23b82c77e50085fc72ec62f1dd46970389916e177268d07d45c7be94dd209d1903f8f23eaff62b7fa782f6057dd36bb96bba82
2020-11-04 13:41:52 +01:00
practicalswift
f1f433e8ca Make it easier to reason about node eviction by removing unused NodeEvictionCandidate::addr (CAddress) 2020-11-04 12:22:06 +00:00
MarcoFalke
fa2ecadd0d test: Fix intermittent rpc_net issue 2020-11-04 13:21:54 +01:00
John Newbery
0bfce9dc46 [addrman] Fix Connected() comment 2020-11-04 11:41:17 +00:00
Jonas Schnelli
04a69c200e macOS deploy: use the new plistlib API
See https://docs.python.org/3/library/plistlib.html.
The new API was added in 3.4 and old removed in 3.9.
2020-11-04 10:28:02 +01:00
MarcoFalke
88776c2926 Merge #20245: test: Run script_assets_test even if built --with-libs=no
fa3967efdb test: Replace ARRAYLEN with C++11 ranged for loop (MarcoFalke)
fafc529053 test: Run AssetTest even if built --with-libs=no (MarcoFalke)
faf58ab139 ci: Add --with-libs=no to one ci config (MarcoFalke)

Pull request description:

  `script_assets_test` doesn't call libbitcoinconsensus, so it seems confusing to require it

ACKs for top commit:
  fanquake:
    ACK fa3967efdb - looks ok to me.

Tree-SHA512: 8744fef64c5d7dc19a0ef4ae9b3df5b3f356253bf000f12723064794c5e50df071824d436059985f1112d94c1b530b65cbeb6b8435114a91b195480620eafc59
2020-11-04 08:51:14 +01:00
Samuel Dobson
5d32009f1a Merge #20220: wallet, rpc: explicit fee rate follow-ups/fixes for 0.21
0be29000c0 rpc: update conf_target helps for correctness/consistency (Jon Atack)
778b9be406 wallet, rpc: fix send subtract_fee_from_outputs help (Jon Atack)
603c005083 wallet: add rpc send explicit fee rate coverage (Jon Atack)
dd341e602d wallet: add sendtoaddress/sendmany explicit fee rate coverage (Jon Atack)
44e7bfa603 wallet: add walletcreatefundedpsbt explicit fee rate coverage (Jon Atack)
6e1ea4273e test: refactor for walletcreatefundedpsbt fee rate coverage (Jon Atack)
3ac7b0c6f1 wallet: fundrawtx fee rate coverage, fixup ParseConfirmTarget() (Jon Atack)
2d8eba8f84 wallet: combine redundant bumpfee invalid params and args tests (Jon Atack)
1697a40b6f wallet: improve bumpfee error/help, add explicit fee rate coverage (Jon Atack)
fc5721723d wallet: fix SetFeeEstimateMode() error message (Jon Atack)
052427eef1 wallet, bugfix: fix bumpfee with explicit fee rate modes (Jon Atack)

Pull request description:

  Follow-up to #11413 providing a base to build on for #19543:

  - bugfix for `bumpfee` raising a JSON error with explicit feerates, fixes issue #20219
  - adds explicit feerate test coverage for `bumpfee`, `fundrawtransaction`, `walletcreatefundedpsbt`, `send`, `sendtoaddress`, and `sendmany`
  - improves a few related RPC error messages and `ParseConfirmTarget()` / error message
  - fixes/improves the explicit fee rate information in the 6 RPC helps, of which 2 were also missing `conf_target` sat/B units

  This provides a spec and regression coverage for the potential next step of a universal `sat/vB` feerate argument (see #19543), as well as immediate coverage and minimum fixes for 0.21.

ACKs for top commit:
  kallewoof:
    Concept/Tested ACK 0be29000c0
  meshcollider:
    Code review + functional test run ACK 0be29000c0

Tree-SHA512: efd965003e991cba51d4504e2940f06ab3d742e34022e96a673606b44fad85596aa03a8c1809f06df7ebcf21a38e18a891e54392fe3d6fb4d120bbe4ea0cf5e0
2020-11-04 16:35:23 +13:00
Samuel Dobson
17c6fb176a Merge #20282: wallet: change upgradewallet return type to be an object
2ead31fb1b [wallet] Return object from upgradewallet RPC (Sishir Giri)

Pull request description:

  Change the return type of upgradewallet to be an object for future extensibility.

  Also return any error string returned from the `UpgradeWallet()` function.

ACKs for top commit:
  MarcoFalke:
    ACK 2ead31fb1b
  meshcollider:
    Tested ACK 2ead31fb1b

Tree-SHA512: bcc7432d2f35093ec2463ea19e894fa885b698c0e8d8e4bd2f979bd4d722cbfed53ec589d6280968917893c64649dc9e40800b8d854273b0f9a1380f51afbdb1
2020-11-04 14:51:42 +13:00
MarcoFalke
fa8e494554 ci: Run ci configs on cirrus 2020-11-03 20:26:58 +01:00
MarcoFalke
fab900802d ci: Bump timeout factor
This avoids timeouts when signing a large raw transaction

https://cirrus-ci.com/task/5009228131729408?command=ci#L4981

test_framework.authproxy.JSONRPCException: 'signrawtransactionwithwallet' RPC took longer than 120.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
2020-11-03 19:44:57 +01:00
Wladimir J. van der Laan
95bde34a71 Merge #20237: net: Hardcoded seeds update for 0.21
6866259fab net: Hardcoded seeds update for 0.21 (Wladimir J. van der Laan)
36e875b4c5 contrib: Add new versions to makeseeds.py and update gitignore (RandyMcMillan)

Pull request description:

  Stats:

  ```
    IPv4   IPv6  Onion Pass
  426728  59523   7900 Initial
  426728  59523   7900 Skip entries with invalid address
  426728  59523   7900 After removing duplicates
  426727  59523   7900 Skip entries from suspicious hosts
  123226  51785   7787 Enforce minimal number of blocks
  121710  51322   7586 Require service bit 1
    4706   1427   3749 Require minimum uptime
    4124   1098   3681 Require a known and recent user agent
    4033   1075   3681 Filter out hosts with multiple bitcoin ports
     512    140    512 Look up ASNs and limit results per ASN and per net
  ```
  I've credited RandyMcMillan for the first commit because of #20190.

  There are at least enough onions this time! Number of IPv6 nodes that pass all the requirements seems similar to last time in #18506.

  For the next major release we'll want TORv3 hardcoded peers as well. This makes no sense now as there are hardly any. But it'd make sense to think about how to collect them because they cannot come from the DNS seeds.

  ### Reviewing
  ```
  2020-10-28 12:04:45     jnewbery  wumpus: Do you have any suggestions for how to review #20237 ?
  2020-10-28 12:28:37     wumpus  jnewbery: previous PRs like it might be a guide there (#18506, #16999), e.g. people could try to repeat the last step in https://github.com/bitcoin/bitcoin/tree/master/contrib/seeds#seeds and see if it ends up with the same .h file, you could also repeat the entire process but as the list of peers from the seeder will be different every time that will give a (slightly, hopefully)
  2020-10-28 12:28:37     wumpus  different output
  2020-10-28 12:49:40     wumpus  testing what part of the peers are connectable is also useful
  2020-10-28 12:51:05     wumpus  or to go deeper, whether most part of the nodes are 'good nodes' and not say spy nodes, but i don't know what means of testing
  ```

ACKs for top commit:
  jonatack:
    ACK 6866259fab

Tree-SHA512: 6b913ec92932de03304301a0cbf7b4a912ed09d890b019deeb449b8fa787c4994222368c6bf08b3c6e2bfa474442612e1c9de9327ec46ba59c37a5f38af50c75
2020-11-03 15:04:07 +01:00
MarcoFalke
fa7eed5be7 doc: Clarify that vpindexToConnect is in reverse order
Also, style-fixups of touched code
2020-11-03 14:02:02 +01:00
MarcoFalke
fa62304c97 Remove dead CheckForkWarningConditionsOnNewFork 2020-11-03 14:01:40 +01:00
Pieter Wuille
50eb0c2512 Small improvements to the Taproot functional tests
The "whitelist" and "connect_nodes" is not needed in feature_taproot.py,
so remove it.

The changes to key.py are required when running the unit tests from the
test folder. Failure on current master:

[test]$ python -m unittest functional/test_framework/key.py
.E
======================================================================
ERROR: test_schnorr_testvectors (functional.test_framework.key.TestFrameworkKey)
Implement the BIP340 test vectors (read from bip340_test_vectors.csv).
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/functional/test_framework/key.py", line 526, in test_schnorr_testvectors
    with open(os.path.join(sys.path[0], 'test_framework', 'bip340_test_vectors.csv'), newline='', encoding='utf8') as csvfile:
FileNotFoundError: [Errno 2] No such file or directory: 'test/test_framework/bip340_test_vectors.csv'

----------------------------------------------------------------------
Ran 2 tests in 0.775s

FAILED (errors=1)
2020-11-03 12:26:17 +01:00
MarcoFalke
fac865b72d test: Fix intermittent feature_taproot issue
The transaction is too large to fit into the mempool, so put it into a
block.

https://travis-ci.org/github/bitcoin/bitcoin/jobs/740987240#L7217

test  2020-11-03T01:31:08.645000Z TestFramework (ERROR): JSONRPC error
        Traceback (most recent call last):
          File "./test/functional/test_framework/test_framework.py", line 126, in main
            self.run_test()
          File "./test/functional/feature_taproot.py", line 1448, in run_test
            self.nodes[1].sendtoaddress(address=addr, amount=int(self.nodes[1].getbalance() * 70000000) / 100000000)
          File "./test/functional/test_framework/coverage.py", line 47, in __call__
            return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
          File "./test/functional/test_framework/authproxy.py", line 146, in __call__
            raise JSONRPCException(response['error'], status)
        test_framework.authproxy.JSONRPCException: Transaction too large (-6)
2020-11-03 12:26:17 +01:00
MarcoFalke
fa1dea19fc test: Fix deser issue in create_block
Without the fix a hex-string can not be parsed:

File "./test/functional/test_framework/blocktools.py", line 82, in create_block
    txo.deserialize(io.BytesIO(tx))
TypeError: a bytes-like object is required, not 'str'

Also, remove io import and repace it with our FromHex() helper
2020-11-03 12:25:55 +01:00
MarcoFalke
fa762a3fd4 test: Remove unused unnamed parameter from block.serialize call
All blocks and transactions are serialized with witness, no need to set
this
2020-11-03 12:23:21 +01:00
Jonas Schnelli
58cfbc38e0 Ignoring (but warn) on duplicate -wallet parameters 2020-11-03 12:06:32 +01:00
John Newbery
eefe194718 [net] Consolidate logic around calling CAddrMan::Connected()
Currently, the logic around whether we called CAddrMan::Connected() for
a peer is spread between verack processing (where we discard inbound
peers) and FinalizeNode (where we discard misbehaving and
block-relay-only peers). Consolidate that logic to a single place.

Also remove the CNode.fCurrentlyConnected bool, which is now
redundant. We can rely on CNode.fSuccessfullyConnected, since the two
bools were only ever flipped to true in the same place.
2020-11-03 09:15:20 +00:00
MarcoFalke
218fe60d91 Merge #20290: fuzz: Fix DecodeHexTx fuzzing harness issue
28f8cb13d4 fuzz: Fix DecodeHexTx fuzzing harness issue (practicalswift)

Pull request description:

  Fix `DecodeHexTx` fuzzing harness issue.

  Before this patch:

  ```
  $ src/test/fuzz/decode_tx
  decode_tx: test/fuzz/decode_tx.cpp:29:
      void test_one_input(const std::vector<uint8_t> &):
      Assertion `result_try_witness_and_maybe_no_witness' failed.
  …
  ```

  After this patch:

  ```
  $ src/test/fuzz/decode_tx
  …
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 28f8cb13d4

Tree-SHA512: 2ed11b2f00a4c6fa3e8eea76a2a37d89a4b8d52815264676fe3de0a26ad7906cfafda9b843ceede2fd428815472e01fd1f87afb851282a8c7839bd4c87dc382b
2020-11-03 09:48:15 +01:00
MarcoFalke
5174b534da Merge #20289: fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CService
c2cf8a18c2 fuzz: Check for addrv1 compatibility before using addrv1 serializer on CService (practicalswift)

Pull request description:

  Check for addrv1 compatibility before using addrv1 serializer/deserializer on `CService`:

  Before this patch:

  ```
  $ src/test/fuzz/service_deserialize
  service_deserialize: test/fuzz/deserialize.cpp:85:
      void (anonymous namespace)::AssertEqualAfterSerializeDeserialize(const T &, const int) [T = CService]:
      Assertion `Deserialize<T>(Serialize(obj, version)) == obj' failed.
  ```

  After this patch:

  ```
  $ src/test/fuzz/service_deserialize
  …
  ```

  Related change: #20247

ACKs for top commit:
  MarcoFalke:
    review ACK c2cf8a18c2

Tree-SHA512: dba6ddc60e8ef621011d844281461f1741de08c4af1a2b7156c810af44306cef7ec582de5974752db02ca085cfd23da0296d70b694e59ee262589d829fa0626e
2020-11-03 09:14:22 +01:00
fanquake
8387f832d6 Merge #20187: Addrman: test-before-evict bugfix and improvements for block-relay-only peers
16d9bfc417 Avoid test-before-evict evictions of current peers (Suhas Daftuar)
e8b215a086 Refactor test for existing peer connection into own function (Suhas Daftuar)
4fe338ab3e Call CAddrMan::Good() on block-relay-only peer addresses (Suhas Daftuar)
daf5553126 Avoid calling CAddrMan::Connected() on block-relay-only peer addresses (Suhas Daftuar)

Pull request description:

  This PR does two things:

  * Block-relay-only interaction with addrman.
    * Calling `CAddrMan::Connected()` on an address that was a block-relay-only peer causes the time we report in `addr` messages containing that peer to be updated; particularly now that we use anchor connections with a our block-relay-only peers, this risks leaking information about those peers.  So, stop this.
    * Avoiding calling `CAddrMan::Good()` on block-relay-only peer addresses causes the addrman logic around maintaining the new and tried table to be less good, and in particular makes it so that block-relay-only peer addresses are more likely to be evicted from the addrman (for no good reason I can think of).  So, mark those addresses as good when we connect.

  * Fix test-before-evict bug. There's a bug where if we get a collision in the tried table with an existing address that is one of our current peers, and the connection is long-lived enough, then `SelectTriedCollisions()` might return that existing peer address to us as a test-before-evict connection candidate. However, our logic for new outbound connections would later prevent us from actually making a connection; the result would be that when we get a collision with a long-lived current peer, that peer's address is likely to get evicted from the tried table.  Fix this by checking to see if a test-before-evict candidate is a peer we're currently connected to, and if so, mark it as `Good()`.

ACKs for top commit:
  sipa:
    utACK 16d9bfc417
  amitiuttarwar:
    code review ACK 16d9bfc417
  mzumsande:
    Code-Review ACK 16d9bfc417.
  jnewbery:
    utACK 16d9bfc417
  ariard:
    Code Review ACK 16d9bfc.
  jonatack:
    Tested ACK 16d9bfc417

Tree-SHA512: 188ccb814e436937cbb91d29d73c316ce83f4b9c22f1cda56747f0949a093e10161ae724e87e4a2d85ac40f85f5f6b4e87e97d350a1ac44f80c57783f4423324
2020-11-03 09:38:35 +08:00
Antoine Riard
bc4a230087 Remove redundant p2p lock tacking for tx download functional tests
New functional test coverage of tx download was added by #19988,
but `with p2p_lock` is redundant for some tests with `wait_until`
test helper, already guaranteeing test lock tacking.
2020-11-02 18:29:49 -05:00
Antoine Riard
d3b5eac9a9 Add mutation for functional test test_preferred_inv
Add a booelan arg to test_preferred_inv to cover NONPREF_PEER_TX_DELAY
as applied as a TxRequestTracker parameter in AddTxAnnouncement.
2020-11-02 18:29:49 -05:00
Antoine Riard
06efb3163c Add functional test test_txid_inv_delay
Add a simple functional test to cover TXID_RELAY_DELAY as applied
as a TxRequestTracker parameter in AddTxAnnoucement.
2020-11-02 18:29:49 -05:00
Antoine Riard
a07910abcd test: Makes wtxidrelay support a generic P2PInterface option
Its usage is extended beyond p2p_segwit.py in next commit.
2020-11-02 18:29:48 -05:00
practicalswift
28f8cb13d4 fuzz: Fix DecodeHexTx fuzzing harness issue 2020-11-02 22:21:03 +00:00
practicalswift
c2cf8a18c2 fuzz: Check for addrv1 compatibility before using addrv1 serializer on CService 2020-11-02 21:45:40 +00:00
Jon Atack
961f148cb1 doc: update contrib/seeds/README dnspython installation info 2020-11-02 20:54:59 +01:00
Jon Atack
dd7b5f46d8 script: fix deprecation warning in makeseeds.py
makeseeds.py:139: DeprecationWarning: please use dns.resolver.resolve() instead
  asn = int([x.to_text() for x in dns.resolver.query('.'.join(

per https://dnspython.readthedocs.io/en/latest/whatsnew.html
dns.resolver.query() was deprecated in dnspython version 2.0.0
2020-11-02 20:54:57 +01:00
Wladimir J. van der Laan
ca18860563 Merge #20263: Update assumed chain params
fa90ba36d3 Update assumed chain params (MarcoFalke)

Pull request description:

  > Oh, by the way, the same procedure as last year, Miss Sophie?
  > Same procedure as every year, James.

  See https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-branch-off

ACKs for top commit:
  jonatack:
    ACK fa90ba36d3 per `git diff fa0c1b5 fa90ba3` and re-running getblockheader/getblockhash/getchaintxstats on the signet chain
  dergoegge:
    ACK fa90ba36d3 - mainnet and testnet data matches my node.
  theStack:
    re-ACK fa90ba36d3 ✔️
  darosior:
    re-ACK fa90ba36d3 for mainnet and testnet.

Tree-SHA512: 83044cc59d9fc873cb29f13008d00b54cb4bb4646c1ca53ab4f429e7333a32402becb888d3be117d390c1297c2f7d083f77ae12ac8633265edceb3cfefac087f
2020-11-02 18:53:23 +01:00
Wladimir J. van der Laan
ef4c7c4e0b Merge #18788: tests: Update more tests to work with descriptor wallets
c7b7e0a692 tests: Make only desc wallets for wallet_multwallet.py --descriptors (Andrew Chow)
d4b67ad214 Avoid creating legacy wallets in wallet_importdescriptors.py (Andrew Chow)
6c9c12bf87 Update feature_backwards_compatibility for descriptor wallets (Andrew Chow)
9a4c631e1c Update wallet_labels.py to not require descriptors=False (Andrew Chow)
242aed7cc1 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors (Andrew Chow)
388053e172 Disable some tests for tool_wallet when descriptors (Andrew Chow)
47d3243160 Make raw multisig tests legacy wallet only in rpc_rawtransaction.py (Andrew Chow)
59d3da5bce Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py (Andrew Chow)
25bc5dccbf Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py (Andrew Chow)
0bd1860300 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py (Andrew Chow)
08067aebfd Add script equivalent of functions in address.py (Andrew Chow)
86968882a8 Add descriptor wallet output to tool_wallet.py (Andrew Chow)
3457679870 Use separate watchonly wallet for multisig in feature_nulldummy.py (Andrew Chow)
a42652ec10 Move import and watchonly tests to be legacy wallet only in wallet_balance.py (Andrew Chow)
4b871909d6 Use importdescriptors for descriptor wallets in wallet_bumpfee.py (Andrew Chow)
c2711e4230 Avoid dumpprivkey in wallet_listsinceblock.py (Andrew Chow)
553dbf9af4 Make import tests in wallet_listtransactions.py legacy wallet only (Andrew Chow)
dc81418fd0 Use a separate watchonly wallet in rpc_fundrawtransaction.py (Andrew Chow)
a357111047 Update wallet_importprunedfunds to avoid dumpprivkey (Andrew Chow)

Pull request description:

  I went through all the tests and checked whether they passed with descriptor wallets. This partially informed some changes in #16528. Some tests needed changes to work with descriptor wallets. These were primarily due to import and watchonly behavior. There are some tests and test cases that only test legacy wallet behavior so those tests won't be run with descriptor wallets.

  This PR updates more tests to have to the `--descriptors` switch in `test_runner.py`. Additionally a mutually exclusive `--legacy-wallet` option has been added to force legacy wallets. This does nothing currently but will be useful in the future when descriptor wallets are the default. For the tests that rely on legacy wallet behavior, this option is being set so that we don't forget in the future. Those tests are `feature_segwit.py`, `wallet_watchonly.py`, `wallet_implicitsegwit.py`, `wallet_import_with_label.py`, and `wallet_import_with_label.py`.

  If you invert the `--descriptors`/`--legacy-wallet` default so that descriptor wallets are the default, all tests (besides the legacy wallet specific ones) will pass.

ACKs for top commit:
  MarcoFalke:
    review ACK c7b7e0a692 🎿
  laanwj:
    ACK c7b7e0a692

Tree-SHA512: 2f4e87815005d1d0a2543ea7947f7cd7593d8cf5312228ef85f8e096f19739b225769961943049cb44f6f07a35b8de988e2246ab9aca5bb5a0b2e62694d5637d
2020-11-02 18:50:37 +01:00
MarcoFalke
b6a00e76ab Merge #20279: doc: release process updates/fixups
e5f3e95a8e doc: fix getchaintxstats fields in release-process.md (Jon Atack)

Pull request description:

  ISTM the getchaintxstats fields should be `window_final_block_hash` rather than `window_last_block_hash`. While here, replace getblockchaininfo with getblockheader (and getblockhash) instead of getblockchaininfo for updating the nMinimumChainWork and defaultAssumeValid consensus params, update the example PR, and improve a link with a named anchor tag.

  Markdown rendering here: https://github.com/jonatack/bitcoin/blob/release-process-getchaintxstats-fix/doc/release-process.md

ACKs for top commit:
  theStack:
    re-ACK e5f3e95a8e

Tree-SHA512: 48c9c65f10d65e461da8d4935af56b6c67e6faca94e4593237f754d8c48f03bef2b9b4a71e5d1009b215a415ba7c4c4218aca6dce97238101ca1c81f5d098bdb
2020-11-02 15:58:25 +01:00
MarcoFalke
fae45c34d1 test: Only try witness deserialize when checking for witness deserialize failure 2020-11-02 14:19:44 +01:00
MarcoFalke
c5ec0367d7 Merge #20165: Only relay Taproot spends if next block has it active
3d0556d410 Increase feature_taproot inactive test coverage (Pieter Wuille)
525cbd425e Only relay Taproot spends if next block has it active (Pieter Wuille)

Pull request description:

  There should be no change to mempool transaction behavior for witness v1 transactions as long as no activation is defined. Until that point, we should treat the consensus rules as under debate, and for soft-fork safety, that means spends should be treated as non-standard.

  It's possible to go further: don't relay them unless the consensus rules are actually active for the next block. This extends non-relay to the period where a deployment is defined, started, locked in, or failed. I see no downsides to this, and the code change is very simple.

ACKs for top commit:
  Sjors:
    utACK 3d0556d410
  MarcoFalke:
    review ACK 3d0556d410 🏓
  jnewbery:
    utACK 3d0556d410

Tree-SHA512: ca625a2981716b4b44e8f3722718fd25fd04e25bf3ca1684924b8974fca49f7c1d438fdd9dcdfbc091a442002e20d441d42c41a0e2096e74a61068da6c60267a
2020-11-02 10:12:06 +01:00
Sishir Giri
2ead31fb1b [wallet] Return object from upgradewallet RPC 2020-11-02 08:38:38 +00:00
MarcoFalke
fa90ba36d3 Update assumed chain params 2020-11-02 09:22:08 +01:00
MarcoFalke
867dbeba5f Merge #20281: docs: Correct getblockstats documentation for (sw)total_weight
5d9917464a docs: Correct getblockstats documentation for (sw)total_weight (Nadav Ivgi)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    ACK 5d9917464a

Tree-SHA512: eb9e8f05c61d5363cf698f0b9b51ff42557e783be64c6a814cd8f4073499ff0cbfe00528b12cc0f5e4de245458a62cbd30eb817a7d1e020ee3cbfa83a95eb550
2020-11-02 08:44:59 +01:00
Samuel Dobson
26d7941224 Merge #20230: wallet: Fix bug when just created encrypted wallet cannot get address
bf6855a909 wallet: Fix bug when just created encrypted wallet cannot get address (Hennadii Stepanov)

Pull request description:

  Fix https://github.com/bitcoin-core/gui/issues/105

ACKs for top commit:
  achow101:
    Tested ACK bf6855a909
  kristapsk:
    ACK bf6855a909
  meshcollider:
    Tested ACK bf6855a909

Tree-SHA512: eca0ab306d7206f2e5db568e83217bd854caac104379f4d8fb261db832d4d6310cbb1eab44ce9b05a5ac2eb5879a623b729752a88810f8370c24518a8d81292d
2020-11-02 11:58:19 +13:00
Andrew Chow
c7b7e0a692 tests: Make only desc wallets for wallet_multwallet.py --descriptors 2020-11-01 17:54:19 -05:00
Andrew Chow
d4b67ad214 Avoid creating legacy wallets in wallet_importdescriptors.py 2020-11-01 17:54:19 -05:00
Andrew Chow
6c9c12bf87 Update feature_backwards_compatibility for descriptor wallets 2020-11-01 17:54:19 -05:00
Andrew Chow
9a4c631e1c Update wallet_labels.py to not require descriptors=False 2020-11-01 17:54:19 -05:00
Andrew Chow
242aed7cc1 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors
Although legacy wallet is still the default, for future use, add a
--legacy-wallet option to the test framework. Additional tests for
descriptor wallets have been enabled with the --descriptors option.
Tests that must be legacy wallet only are being started with
--legacy-wallet. Even though this option does not currently do anything,
this will be helpful in the future when descriptor wallets become the
default.
2020-11-01 17:54:19 -05:00
Andrew Chow
388053e172 Disable some tests for tool_wallet when descriptors
Some tests are legacy wallet only (and make legacy wallets) so they
shouldn't be run when doing descriptor wallet tests.
2020-11-01 17:54:19 -05:00
Andrew Chow
47d3243160 Make raw multisig tests legacy wallet only in rpc_rawtransaction.py
The traditional multisig workflow doesn't work with descriptor wallets
so make these tests legacy wallet only.
2020-11-01 17:54:19 -05:00
Andrew Chow
59d3da5bce Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py
addmultisigaddress is not available in descriptor wallets, so only run
these when testing legacy wallets
2020-11-01 17:54:19 -05:00
Andrew Chow
25bc5dccbf Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py
sethdseed and importmulti are not available for descriptor wallets, so
when doing descriptor wallet tests, use importdescriptors instead.

Also changes some output to match what descriptor wallets will return.
2020-11-01 17:54:19 -05:00
Andrew Chow
0bd1860300 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py
dumpprivkey and watchonly behavior don't work with descriptor wallets.

Test for multisigs is modified to not rely on watchonly behavior for
those multisigs. This has a side effect of removing listunspent, but
that's not the target of this test, so that's fine.
2020-11-01 17:54:19 -05:00
Andrew Chow
08067aebfd Add script equivalent of functions in address.py 2020-11-01 17:54:19 -05:00
Andrew Chow
86968882a8 Add descriptor wallet output to tool_wallet.py
Descriptor wallets output slightly different information in the wallet
tool, so check that output when in descriptor wallet mode.
2020-11-01 17:54:19 -05:00
Andrew Chow
3457679870 Use separate watchonly wallet for multisig in feature_nulldummy.py
Create and import the multisig into a separate watchonly wallet so that
feature_nulldummy.py works with descriptor wallets.

blocktools.create_raw_transaction is also updated to use multiple nodes
and wallets and to use PSBT so that this test passes.
2020-11-01 17:54:19 -05:00
Andrew Chow
a42652ec10 Move import and watchonly tests to be legacy wallet only in wallet_balance.py
Imports and watchonly behavior are legacy wallet only, so make them only
run when the test is in legacy wallet mode.
2020-11-01 17:54:19 -05:00
Andrew Chow
4b871909d6 Use importdescriptors for descriptor wallets in wallet_bumpfee.py
If using descriptor wallets, use importdescriptors instead of
importmulti.
2020-11-01 17:54:19 -05:00
Andrew Chow
c2711e4230 Avoid dumpprivkey in wallet_listsinceblock.py
Generate a privkey in the test framework instead of using dumpprivkey so
that descriptor wallets work in this test.
2020-11-01 17:54:19 -05:00
Andrew Chow
553dbf9af4 Make import tests in wallet_listtransactions.py legacy wallet only
Existing import* RPCs are disabled for descriptor wallets, so only do
these tests for legacy wallets.
2020-11-01 17:54:19 -05:00
Andrew Chow
dc81418fd0 Use a separate watchonly wallet in rpc_fundrawtransaction.py
Import things into a separate watchonly wallet to work with descriptor
wallets.
2020-11-01 17:54:19 -05:00
Andrew Chow
a357111047 Update wallet_importprunedfunds to avoid dumpprivkey
Removes the use of dumpprivkey so that descriptor wallets can pass on
this. Also does a few descriptor wallet specific changes due to
different IsMine semantics.
2020-11-01 17:54:19 -05:00
Samuel Dobson
f1fcbdea25 Merge #20271: doc: Document that wallet salvage is experimental
fab94534b6 doc: Document that wallet salvage is experimental (MarcoFalke)

Pull request description:

  See #20151

ACKs for top commit:
  practicalswift:
    ACK fab94534b6: user safety first
  hebasto:
    ACK fab94534b6, maybe capitalize into "WARNING"?
  meshcollider:
    Trivial ACK fab94534b6

Tree-SHA512: 94912c491facc485293e4333066057933d706d84c7172f615296e7ba998c583c8bd07e751e6f00cd6576e7791007ace321f959181f7bf6a4e15e10d7ec8a1b7e
2020-11-02 11:52:49 +13:00
Samuel Dobson
0c2eb7f8de Merge #20262: tests: Skip --descriptor tests if sqlite is not compiled
7411876c75 Ensure a legacy wallet for BDB format check (Andrew Chow)
586640381a Skip --descriptor tests if sqlite is not compiled (Andrew Chow)

Pull request description:

  #20156 allows sqlite to not be compiled by configuring `--without-sqlite`. However doing so and then running the test runner will result in all of the `--descriptor` tests to fail. We should be skipping those tests if sqlite was not compiled.

ACKs for top commit:
  practicalswift:
    ACK 7411876c75: patch looks correct
  Sjors:
    tACK 7411876c75
  ryanofsky:
    Code review ACK 7411876c75
  hebasto:
    ACK 7411876c75, tested on Linux Mint 20 (x86_64), tests pass for binaries compiled with:

Tree-SHA512: 1d635a66d2b7bb865300144dfefcfdaf86133aaaa020c8f440a471476ac1205d32f2df704906ce6c2ea48ddf791c3c95055f6291340b4f7b353c1b02cab5cabe
2020-11-02 11:48:22 +13:00
Samuel Dobson
5a6f3c5a01 Merge #20080: Strip any trailing / in -datadir and -blocksdir paths
ad5cef5dfd doc: Update data directory path comments (Hennadii Stepanov)
b19e88230f util: Add StripRedundantLastElementsOfPath function (Hennadii Stepanov)

Pull request description:

  Wallet names in `listwalletdir` RPC are correct now, even if the `-datadir` path has any number of trailing `/`.

  This PR is an alternative to #19933.

  Fixes #19928.

ACKs for top commit:
  MarcoFalke:
    review ACK ad5cef5dfd 🔙
  promag:
    Code review ACK ad5cef5dfd.
  meshcollider:
    Code review + test run ACK ad5cef5dfd

Tree-SHA512: bccabbd6c18243d48d15b2b27201cc0f5984623dcbc635c8740cf74523f359844c36eadd40391142874fcf452a43880bb6afbf89815ae736e499f9a98143a661
2020-11-02 11:41:38 +13:00
Jon Atack
e5f3e95a8e doc: fix getchaintxstats fields in release-process.md
also:

- use the getblockheader (and getblockhash) RPCs instead of getblockchaininfo
  for updating the nMinimumChainWork and defaultAssumeValid consensus params

- use "RPC" consistently

- update the example PR from 17002 to 20263

- improve a link with a named anchor tag
2020-11-01 22:19:24 +01:00
Nadav Ivgi
5d9917464a docs: Correct getblockstats documentation for (sw)total_weight 2020-11-01 19:23:05 +02:00
João Barbosa
241434200e refactor: qt: Use vQueueNotifications.clear() 2020-11-01 12:56:10 +00:00
Vasil Dimov
89836a82ee style: minor improvements as a followup to #19845
Address suggestions:
https://github.com/bitcoin/bitcoin/pull/19845#discussion_r495486760
https://github.com/bitcoin/bitcoin/pull/19845#discussion_r495488051
https://github.com/bitcoin/bitcoin/pull/19845#discussion_r495730125
2020-10-31 17:02:52 +01:00
Vasil Dimov
ecc6cf1a3b test: fix creation of std::string objects with \0s
A string literal `"abc"` contains a terminating `\0`, so that is 4
bytes. There is no need to write `"abc\0"` unless two terminating
`\0`s are necessary.

`std::string` objects do not internally contain a terminating `\0`, so
`std::string("abc")` creates a string with size 3 and is the same as
`std::string("abc", 3)`.

In `"\01"` the `01` part is interpreted as one number (1) and that is
the same as `"\1"` which is a string like `{1, 0}` whereas `"\0z"` is a
string like `{0, 'z', 0}`. To create a string like `{0, '1', 0}` one
must use `"\0" "1"`.

Adjust the tests accordingly.
2020-10-31 16:02:49 +01:00
Pieter Wuille
3d0556d410 Increase feature_taproot inactive test coverage 2020-10-30 15:52:38 -07:00
Pieter Wuille
525cbd425e Only relay Taproot spends if next block has it active 2020-10-30 15:52:19 -07:00
MarcoFalke
fab94534b6 doc: Document that wallet salvage is experimental 2020-10-30 13:53:28 +01:00
Sjors Provoost
5bff82540b [gui] create wallet: smarter checkbox toggling 2020-10-30 11:15:05 +01:00
Andrew Chow
bd93fc9945 Fix change detection of imported internal descriptors 2020-10-29 17:55:13 -04:00
Andrew Chow
7411876c75 Ensure a legacy wallet for BDB format check 2020-10-29 14:59:29 -04:00
Andrew Chow
586640381a Skip --descriptor tests if sqlite is not compiled 2020-10-29 12:34:16 -04:00
MarcoFalke
42b66a6b81 Merge #20186: wallet: Make -wallet setting not create wallets
01476a88a6 wallet: Make -wallet setting not create wallets (Russell Yanofsky)

Pull request description:

  This changes `-wallet` setting to only load existing wallets, not create new ones.

  - Fixes settings.json corner cases reported by sjors & promag: https://github.com/bitcoin-core/gui/issues/95, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

  - Prevents accidental creation of wallets reported most recently by jb55 http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

  - Simplifies behavior after #15454. #15454 took the big step of disabling creation of the default wallet. This PR extends it to avoid creating other wallets as well. With this change, new wallets just aren't created on startup, instead of sometimes being created, sometimes not. #15454 release notes are updated here and are simpler.

  This change should be targeted for 0.21.0. It's a bug fix and simplifies behavior of the #15937 / #19754 / #15454 features added in 0.21.0.

  ---

  This PR is implementing the simplest, most basic alternative listed in https://github.com/bitcoin-core/gui/issues/95#issuecomment-694236940. Other improvements mentioned there can build on top of this.

ACKs for top commit:
  achow101:
    ACK 01476a88a6
  hebasto:
    re-ACK 01476a88a6
  MarcoFalke:
    review ACK 01476a88a6 🏂

Tree-SHA512: 0d50f4e5dfbd04a2efd9fd66c02085a0ed705807bdec1cf5770d0ae8cb6af07080fb81306349937bf66acdb713d03fb35636f6442b650d0820e66cbae09c2f87
2020-10-29 15:01:39 +01:00
fanquake
8e9e190ea5 Merge #20257: Update secp256k1 subtree to latest master
6c0259fc2f Squashed 'src/secp256k1/' changes from c6b6b8f1bb..3967d96bf1 (Pieter Wuille)

Pull request description:

  Nothing important changed, but this silences this (erroneous) warning in certain GCC 9 versions:

  ```
  In file included from src/secp256k1.c:16:
  src/ecmult_impl.h: In function ‘secp256k1_ecmult’:
  src/ecmult_impl.h:496:48: warning: array subscript [1, 268435456] is outside array bounds of ‘struct secp256k1_strauss_point_state[1]’ [-Warray-bounds]
    496 |             secp256k1_gej tmp = a[state->ps[np].input_pos];
        |                                   ~~~~~~~~~~~~~^~~~~~~~~~
  src/ecmult_impl.h:565:42: note: while referencing ‘ps’
    565 |     struct secp256k1_strauss_point_state ps[1];
        |                                          ^~
  src/ecmult_impl.h:502:139: warning: array subscript [1, 268435456] is outside array bounds of ‘struct secp256k1_strauss_point_state[1]’ [-Warray-bounds]
    502 |             secp256k1_fe_mul(state->zr + np * ECMULT_TABLE_SIZE(WINDOW_A), state->zr + np * ECMULT_TABLE_SIZE(WINDOW_A), &(a[state->ps[np].input_pos].z));
        |                                                                                                                              ~~~~~~~~~~~~~^~~~~~~~~~
  src/ecmult_impl.h:565:42: note: while referencing ‘ps’
    565 |     struct secp256k1_strauss_point_state ps[1];
        |                                          ^~
  ```

  (see https://github.com/bitcoin-core/secp256k1/issues/834)

ACKs for top commit:
  fanquake:
    ACK 5803f5f5f6  - performed the update myself and got the same change: [check_20257_subtree](https://github.com/fanquake/bitcoin/tree/check_20257_subtree).
  hebasto:
    ACK 5803f5f5f6, tested on Linux Mint 20 (x86_64) with `gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0` -- no warnings are emitted.

Tree-SHA512: 386281d23aee93a3b1d1a09fec8319c3a477e46967430c935677eed54abddc62d5a7710f9eeab1ec476ace05adcb194b5b377712e44a6bb95a74ffa35faf77f3
2020-10-29 19:38:21 +08:00
Wladimir J. van der Laan
5b82f253b6 Merge #20195: build: fix mutex detection when building bdb on macOS
d0a829e963 build: fix mutex detection when building bdb on macOS (fanquake)

Pull request description:

  Starting with the Apple Clang shipped with Xcode 12, [Apple has enabled -Werror=implicit-function-declaration by default](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes):
  > Clang now reports an error when you use a function without an explicit declaration when building C or Objective-C code for macOS (-Werror=implicit-function-declaration flag is on). This additional error detection unifies Clang’s behavior for iOS/tvOS and macOS 64-bit targets for this diagnostic. (49917738)

  This causes bdbs mutex detection to fail when building on macOS (not cross-compiling):
  ```bash
  checking for mutexes... UNIX/fcntl
  configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
  configure: error: Unable to find a mutex implementation
  ```

  as previously emitted warnings are being turned into errors. i.e:
  ```bash
  configure:18704: checking for mutexes
  configure:18815: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mmacosx-version-min=10.12 --sysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o conftest -pipe -O2  -I/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/include -L/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/lib conftest.c  -lpthread >&5
  conftest.c:46:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
  main() {
  ^
  conftest.c:51:2: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
          exit (
          ^
  conftest.c:51:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
  1 warning and 1 error generated.
  ```

  Append `-Wno-error=implicit-function-declaration` to `cflags` so that `-Wimplicit-function-declaration` [returns to being a warning](https://clang.llvm.org/docs/UsersManual.html#cmdoption-wno-error), and the configure checks succeed.

  Fixes #19411.

ACKs for top commit:
  laanwj:
    Code review ACK d0a829e963

Tree-SHA512: 7813005b1fc0b370f843b6c0672acab32c999416e92c3f02b75d866e9c7aa41fe5822704fc74de6b65f0d7d94f2cdd05cc7c3ee83295ff1ecbc71d8492b9a2bf
2020-10-29 12:31:54 +01:00
Wladimir J. van der Laan
2e24197117 Merge #20115: cli: -netinfo quick updates/fixups for 0.21
398045ba8b cli -netinfo: print oversized/extreme ping times as "-" (Jon Atack)
773f4c99c0 cli -netinfo: handle longer tor v3 local addresses (Jon Atack)
33e987452f cli -netinfo: make age column variable-width (Jon Atack)
f8a1c4d946 cli -netinfo: various quick updates and fixes (Jon Atack)

Pull request description:

  Quick fixups and updates for v0.21.0:

  - [x] handle larger BIP155 `addrv2` addresses
  - [x] add Signet chain
  - [x] add an additional space between the `net` and `mping` columns; add missing `tinyformat` and `algorithm` headers
  - [x] s/uptime/age/ per 0xB10C suggestion, and make the column auto-adjusting variable width
  - [x] display `-` for oversized mping/ping times like `1.17348e+06`, as reported by practicalswift

  Edit: removed the release note commit, as this PR was not merged before the notes were moved to the wiki. It's here:
  ```
  - A new `bitcoin-cli -netinfo` command returns a network peer connections
    dashboard that displays data from the `getpeerinfo` and `getnetworkinfo` RPCs
    in a human-readable format. An optional integer argument from `0` to `4` may
    be passed to see various levels of detail. (#19643)
  ```

ACKs for top commit:
  michaelfolkson:
    ACK 398045ba8b
  Emzy:
    Tested ACK 398045ba8b

Tree-SHA512: 0625ee840141bafbfcaf8f1fce53f8f850ae91721b2bdad4279372da87c18a1fe3a214d90bfdbbabdf6da38d58290d7dd0f1109b4e2ca5d20cacf417d6ced0f9
2020-10-29 12:07:31 +01:00
Wladimir J. van der Laan
f3727fd735 Merge #20156: build: Make sqlite support optional (compile-time)
bbb42a6896 RPC: createwallet: Nicer error message if descriptor wallet requested and sqlite support not compiled in (Luke Dashjr)
6608fec332 GUI: Create Wallet: Nicely disable descriptor wallet checkbox if sqlite support not compiled in (Luke Dashjr)
7b54d768e1 Make sqlite support optional (compile-time) (Luke Dashjr)

Pull request description:

  As a new requirement, sqlite support should be optional. This PR aims to be only minimum/blocker changes for 0.21.

  Potential follow-up PRs after this:
  * Make BDB support optional
  * Nicer error messages when user tries to load an unsupported wallet
  * Don't compile descriptor wallet code if sqlite disabled

ACKs for top commit:
  jonasschnelli:
    Tested ACK bbb42a6896
  achow101:
    ACK bbb42a6896
  Sjors:
    re-utACK bbb42a6896
  hebasto:
    ACK bbb42a6896, tested on Linux Mint 20 (x86_64, Qt 5.12.8).

Tree-SHA512: 500209dd1971310fab8ae51543343ce0ba91f088ccccff6109b4cc27547cd5532289dca6cb7dac2a7d7c59cdf3c8f5aacc31e9b0f912e38cea52ec26b97100bd
2020-10-29 12:03:36 +01:00
Wladimir J. van der Laan
924a4ff7eb Merge #20242: fuzz: Properly initialize PrecomputedTransactionData
fa56d56d4e fuzz: Properly initialize PrecomputedTransactionData (MarcoFalke)

Pull request description:

  Fixes:

  ```
  script_flags: script/interpreter.cpp:1512: bool SignatureHashSchnorr(uint256 &, const ScriptExecutionData &, const T &, uint32_t, uint8_t, SigVersion, const PrecomputedTransactionData &) [T = CTransaction]: Assertion `cache.m_bip341_taproot_ready && cache.m_spent_outputs_ready' failed.
  ==34989== ERROR: libFuzzer: deadly signal
      #0 0x55e90077ff11 in __sanitizer_print_stack_trace (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x20cf11)
      #1 0x55e9006cb068 in fuzzer::PrintStackTrace() (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x158068)
      #2 0x55e9006b01b3 in fuzzer::Fuzzer::CrashCallback() (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x13d1b3)
      #3 0x7f6fb89383bf  (/lib/x86_64-linux-gnu/libpthread.so.0+0x153bf)
      #4 0x7f6fb855018a in raise (/lib/x86_64-linux-gnu/libc.so.6+0x4618a)
      #5 0x7f6fb852f858 in abort (/lib/x86_64-linux-gnu/libc.so.6+0x25858)
      #6 0x7f6fb852f728  (/lib/x86_64-linux-gnu/libc.so.6+0x25728)
      #7 0x7f6fb8540f35 in __assert_fail (/lib/x86_64-linux-gnu/libc.so.6+0x36f35)
      #8 0x55e9008275bd in bool SignatureHashSchnorr<CTransaction>(uint256&, ScriptExecutionData const&, CTransaction const&, unsigned int, unsigned char, SigVersion, PrecomputedTransactionData const&) /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/script/interpreter.cpp:1512:5
      #9 0x55e900825a3f in GenericTransactionSignatureChecker<CTransaction>::CheckSchnorrSignature(Span<unsigned char const>, Span<unsigned char const>, SigVersion, ScriptExecutionData const&, ScriptError_t*) const /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/script/interpreter.cpp:1699:10
      #10 0x55e900832503 in VerifyWitnessProgram(CScriptWitness const&, int, std::vector<unsigned char, std::allocator<unsigned char> > const&, unsigned int, BaseSignatureChecker const&, ScriptError_t*, bool) /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/script/interpreter.cpp:1900:26
      #11 0x55e90082ecb5 in VerifyScript(CScript const&, CScript const&, CScriptWitness const*, unsigned int, BaseSignatureChecker const&, ScriptError_t*) /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/script/interpreter.cpp:1977:18
      #12 0x55e9007a9b61 in test_one_input(std::vector<unsigned char, std::allocator<unsigned char> > const&) /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags.cpp:51:30
      #13 0x55e9007d0b49 in LLVMFuzzerTestOneInput /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz.cpp:36:5
      #14 0x55e9006b1871 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x13e871)
      #15 0x55e9006b0fb5 in fuzzer::Fuzzer::RunOne(unsigned char const*, unsigned long, bool, fuzzer::InputInfo*, bool*) (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x13dfb5)
      #16 0x55e9006b38d7 in fuzzer::Fuzzer::ReadAndExecuteSeedCorpora(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x1408d7)
      #17 0x55e9006b3c39 in fuzzer::Fuzzer::Loop(std::__Fuzzer::vector<fuzzer::SizedFile, fuzzer::fuzzer_allocator<fuzzer::SizedFile> >&) (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x140c39)
      #18 0x55e9006a290e in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x12f90e)
      #19 0x55e9006cb752 in main (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x158752)
      #20 0x7f6fb85310b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
      #21 0x55e9006776ad in _start (/tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/script_flags+0x1046ad)

ACKs for top commit:
  laanwj:
    Code review ACK fa56d56d4e

Tree-SHA512: 5637b0a0d982360df32d8cd39e913395967af02ec746508fc5f2fd649695c58bfaaf18ef76f4ca9da764d34fdd63dfe188317dd41b2ed57534bd4055a05ae870
2020-10-29 11:50:28 +01:00
Wladimir J. van der Laan
6196cf77e5 Merge #19753: p2p: don't add AlreadyHave transactions to recentRejects
d419fdedbe [net processing] Don't add AlreadyHave txs to recentRejects (Troy Giorshev)

Pull request description:

  If we already have a transaction, don't add it to recentRejects

  Now, we only add a transaction to our recentRejects filter if we didn't already have it, meaning that it is added at most once, as intended.

ACKs for top commit:
  jnewbery:
    Code review ACK d419fdedbe
  laanwj:
    Code review ACK d419fdedbe

Tree-SHA512: cff5c1ba36c4700e2d6ab3eec4a3e51e1bef28fb3cc1bc850c84e06d6e5a9f6c32825207c253cc9cdf596b2eaadb6b5be68b3f8ca752b4ef6c31cf85138e3c99
2020-10-29 11:40:15 +01:00
Jon Atack
0be29000c0 rpc: update conf_target helps for correctness/consistency
for sendtoaddress, sendmany, send, fundrawtransaction, walletcreatefundedpsbt, bumpfee
2020-10-29 00:22:12 +01:00
Jon Atack
778b9be406 wallet, rpc: fix send subtract_fee_from_outputs help 2020-10-29 00:22:10 +01:00
Jon Atack
603c005083 wallet: add rpc send explicit fee rate coverage 2020-10-29 00:22:08 +01:00
Jon Atack
dd341e602d wallet: add sendtoaddress/sendmany explicit fee rate coverage 2020-10-29 00:22:04 +01:00
Jon Atack
44e7bfa603 wallet: add walletcreatefundedpsbt explicit fee rate coverage 2020-10-29 00:22:02 +01:00
Jon Atack
6e1ea4273e test: refactor for walletcreatefundedpsbt fee rate coverage 2020-10-29 00:22:00 +01:00
Jon Atack
3ac7b0c6f1 wallet: fundrawtx fee rate coverage, fixup ParseConfirmTarget() 2020-10-29 00:21:57 +01:00
Hennadii Stepanov
4146a31ccb qt, wallet: Drop unused parameter in WalletModel::setWalletEncrypted 2020-10-28 18:44:55 +02:00
Hennadii Stepanov
e4fc45a011 gui: Remove BDB version from the Information tab 2020-10-28 18:26:50 +02:00
MarcoFalke
3f512f3d56 Merge #20258: tests: Remove getnettotals/getpeerinfo consistency test
778cd0d88d [tests] Remove getnettotals/getpeerinfo consistency test (John Newbery)

Pull request description:

  We make no guarantees about consistency between RPC calls.

  Alternative to 18784

ACKs for top commit:
  MarcoFalke:
    review ACK 778cd0d88d
  troygiorshev:
    ACK 778cd0d88d after reading discussion on 18784, code review, ran test

Tree-SHA512: 438333a111cc93a09680cec47f13fbe03557d4803e5d826aec6f72e5afea62a088622645f0756e8fd2c9182c2a69ccca867d4d6fed2250364bee2b6c834adb1a
2020-10-28 16:27:33 +01:00
John Newbery
778cd0d88d [tests] Remove getnettotals/getpeerinfo consistency test
We make no guarantees about consistency between RPC calls.
2020-10-28 10:18:09 +00:00
Wladimir J. van der Laan
db26eeba71 Merge #19877: [test] clarify rpc_net & p2p_disconnect_ban functional tests
47ff5098ad [test] Clarify setup of node topology. (Amiti Uttarwar)
0672522aed [move-only, test]: Match test order with run order (Amiti Uttarwar)

Pull request description:

  small improvements to clarify logic in the functional tests
  1. have test logic in `rpc_net.py` match run order of the test
  2. remove `connect_nodes` calls that are redundant with the automatic test setup executed by the test framework

  Noticed when I was trying to debug a test for #19725. Small changes but imo very helpful, because they initially confused me.

ACKs for top commit:
  laanwj:
    ACK 47ff5098ad

Tree-SHA512: 2843da2c0b4f06b2600b3adb97900a62be7bb2228770abd67d86f2a65c58079af22c7c20957474a98c17da85f40a958a6f05cb8198aa0c56a58adc1c31100492
2020-10-28 10:25:34 +01:00
MarcoFalke
488b77f72e Merge #20256: qt: Pre-splitoff translations update
a193f969ee qt: Pre-splitoff translations update (Wladimir J. van der Laan)

Pull request description:

  Do a pre-splitoff translations update. Pulls the translations from Transifex, and apparently, there are also some new English messages in the source code since the transtlations freeze, but not many.

ACKs for top commit:
  fanquake:
    ACK a193f969ee - I got the same changes (branch: [translation_check_0210](https://github.com/fanquake/bitcoin/tree/translation_check_0210)), except for a diff in `src/qt/locale/bitcoin_es_MX.ts`. Although from a look on Transifex it seems that transalation may have been updated in the last hour or so.

Tree-SHA512: 315c1f4327142caae01179a8398b0a8adb9108e1fc8122585274b7ed74ae878554399a06fd12a8103a4933ecf0ad6e211d6efc63edc6827c0168317f2b83528f
2020-10-28 08:50:12 +01:00
Pieter Wuille
6c0259fc2f Squashed 'src/secp256k1/' changes from c6b6b8f1bb..3967d96bf1
3967d96bf1 Merge #838: Make autotools check for all the used openssl functions
3734b68200 Configure echo if openssl tests are enabled
6f54e69f03 Merge #841: Avoids a potentially shortening size_t to int cast in strauss_wnaf_
8893f42438 Avoids a potentially shortening size_t to int cast in strauss_wnaf_
e6692778d3 Modify bitcoin_secp.m4's openssl check to call all the functions that we use in the tests/benchmarks. That way linking will fail if those symbols are missing
ac05f61fcf Merge #809: Stop treating ECDH as experimental
e6e3d5da2f travis: add schnorrsig to valgrind and big endian platform test
353dff156f Stop treating ECDH as experimental

git-subtree-dir: src/secp256k1
git-subtree-split: 3967d96bf184519eb98b766af665b4d4b072563e
2020-10-27 23:08:48 -07:00
Pieter Wuille
5803f5f5f6 Update secp256k1 subtree to latest master 2020-10-27 23:08:48 -07:00
Amiti Uttarwar
47ff5098ad [test] Clarify setup of node topology.
Since the test framework automatically sets up a connection between the nodes,
the second connect_nodes call was a no-op. Remove the redundant call & add
comments to explain the expected topology.
2020-10-27 19:23:02 -07:00
João Barbosa
989e579d07 qt: Make transaction notification queue wallet specific
Drop global vQueueNotifications and make one for each wallet.
2020-10-28 00:35:17 +00:00
João Barbosa
7b3b2303f4 move-only: Define TransactionNotification before TransactionTablePriv
This is needed because next commit moves vQueueNotifications to
TransactionTablePriv member.
2020-10-28 00:12:47 +00:00
Amiti Uttarwar
0672522aed [move-only, test]: Match test order with run order 2020-10-27 13:45:25 -07:00
Jon Atack
2d8eba8f84 wallet: combine redundant bumpfee invalid params and args tests
might be best reviewed with:

git show COMMIT_HASH -w --color-moved=dimmed-zebra
2020-10-27 21:33:43 +01:00
Jon Atack
1697a40b6f wallet: improve bumpfee error/help, add explicit fee rate coverage 2020-10-27 21:33:37 +01:00
Hennadii Stepanov
ad5cef5dfd doc: Update data directory path comments 2020-10-27 21:46:00 +02:00
Hennadii Stepanov
b19e88230f util: Add StripRedundantLastElementsOfPath function
Co-authored-by: saibato <saibato.naga@pm.me>
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-10-27 21:45:32 +02:00
Wladimir J. van der Laan
a193f969ee qt: Pre-splitoff translations update 2020-10-27 19:40:44 +01:00
Suhas Daftuar
16d9bfc417 Avoid test-before-evict evictions of current peers
Outbound peer logic prevents connecting to addresses that we're already
connected to, so prevent inadvertent eviction of current peers via
test-before-evict by checking this condition and marking current peer's
addresses as Good().

Co-authored-by: John Newbery <john@johnnewbery.com>
2020-10-27 11:15:21 -04:00
Suhas Daftuar
e8b215a086 Refactor test for existing peer connection into own function 2020-10-27 11:15:21 -04:00
Suhas Daftuar
4fe338ab3e Call CAddrMan::Good() on block-relay-only peer addresses
Being able to invoke Good() is important for address management (new vs tried
table, tried table eviction via test-before-evict). We mitigate potential
information leaks by not calling Connected() on these peer addresses.
2020-10-27 11:15:21 -04:00
Suhas Daftuar
daf5553126 Avoid calling CAddrMan::Connected() on block-relay-only peer addresses
Connected() updates the time we serve in addr messages, so avoid leaking
block-relay-only peer connections by avoiding these calls.
2020-10-27 11:14:58 -04:00
Jonas Schnelli
55b1ffcd25 Merge bitcoin-core/gui#116: Fix unreasonable default size of the main window without loaded wallets
7b2e42ecc4 qt: Add WalletFrame::sizeHint (Hennadii Stepanov)

Pull request description:

  This PR fixes a bug in master (d67883d01e) and in 0.20.1 that could be easily reproduced with
  ```
  $ src/qt/bitcoin-qt -regtest -resetguisettings -nowallet
  ```

  ![Screenshot from 2020-10-25 21-21-27](https://user-images.githubusercontent.com/32963518/97117179-b1800100-170a-11eb-87c9-3120d39b9455.png)
  ![Screenshot from 2020-10-25 21-23-32](https://user-images.githubusercontent.com/32963518/97117186-b644b500-170a-11eb-8b5d-234ff7205003.png)

  **With this PR:**

  ![Screenshot from 2020-10-25 21-20-35](https://user-images.githubusercontent.com/32963518/97117226-f441d900-170a-11eb-8d66-98b7718a2bb1.png)
  ![Screenshot from 2020-10-25 21-23-03](https://user-images.githubusercontent.com/32963518/97117232-f99f2380-170a-11eb-85ed-c7b5ece926b2.png)

  ---

  Fix #104
  Fix #113

  This PR is an alternative to #107 without [hard-coding a size in pixels](https://github.com/bitcoin-core/gui/pull/107#discussion_r511474021).

ACKs for top commit:
  jonasschnelli:
    Tested ACK 7b2e42ecc4 - I can confirm this fixes #104 (Ubuntu 20.04 - HiDPI 200%).

Tree-SHA512: eb0692dbeb3befdeecca0e41534c9783eab6637c14cc4f170ee42619235884f9354f8d22a10c20c08cc89dc5340a60b7dfa2523c12e64b3386b3fd2c6d5f934e
2020-10-27 14:51:30 +01:00
fanquake
d0a829e963 build: fix mutex detection when building bdb on macOS
Starting with the Clang shipped with Xcode 12, Apple has enabled
-Werror=implicit-function-declaration by default. This causes bdbs mutex
detection to fail when building on macOS (not cross-compiling):

checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation

as previously emitted warnings are being turned into errors. i.e:

error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]

Append -Wno-error=implicit-function-declaration to cflags so that
-Wimplicit-function-declaration returns to being a warning, and the
configure checks will succeed.

Fixes #19411.
2020-10-27 21:30:12 +08:00
Troy Giorshev
d419fdedbe [net processing] Don't add AlreadyHave txs to recentRejects
Now, we only add a transaction to our recentRejects filter if we didn't
already have it, meaning that it is added at most once, as intended.
2020-10-27 06:07:41 -04:00
Wladimir J. van der Laan
83363f7b62 Merge #20167: test: Add test for -blockversion
fa9b48549c test: Add test for -blockversion (MarcoFalke)
fa7fb0e442 test: Default blockversion to 4 in feature_block (MarcoFalke)
fa2b778d0c test: Remove unused -blockversion from tests (MarcoFalke)

Pull request description:

  `-blockversion` is currently untested, as in: The setting could be made a no-op without any tests failing. Fix that by adding an explicit test for it. Also, related minor cleanups.

ACKs for top commit:
  guggero:
    ACK fa9b48549c.

Tree-SHA512: 1b2e792f7ed0ec1db163476ee8a938f8f7cb3691f797c721bbe55fdeed92487c2ff83b55467440096917999406c86430cb3a615383cefb4f621828309ff6a1e7
2020-10-27 10:57:57 +01:00
Wladimir J. van der Laan
c1564baf3b Merge #19124: doc: Document ALLOW_HOST_PACKAGES dependency option
47e2a35fac doc: Document ALLOW_HOST_PACKAGES dependency option (skmcontrib)

Pull request description:

  Provided entry in depends, README.md to ensure that ALLOW_HOST_PACKAGES dependency option is documented, #19113

ACKs for top commit:
  hebasto:
    ACK 47e2a35fac.

Tree-SHA512: 10d9285885be25f092881e4886c6a804cd42b5224bdf1dfa8b8369463808ddaf533a8604f14f7fe45478434a22feae98053f4731b51d976c071d69882bdac72b
2020-10-27 10:17:14 +01:00
fanquake
67d4643a1a Merge #20152: doc: Update wallet files in files.md
defe48a51f doc: Update wallet files in files.md (Hennadii Stepanov)

Pull request description:

  This PR is a #19077 follow up, and it addresses the [comment](https://github.com/bitcoin/bitcoin/pull/19077#discussion_r504805234):

  > If need to update, there are two corrections that could be made:
  >
  >  * Line 69 "Wallets are Berkeley DB (BDB) databases" is no longer true
  >
  >  * Line 76 "Wallet lock file" should say "BDB wallet lock file"

ACKs for top commit:
  RiccardoMasutti:
    ACK defe48a
  meshcollider:
    ACK defe48a51f

Tree-SHA512: 39939f86a9c7842bf06913998305dcbd6209585f1da0fe9c274bac0572eb8464e59176884dd9e2b91312f34efad40cdeb4085ec72c2a2c1b33d16b6ab505140c
2020-10-27 15:43:53 +08:00
MarcoFalke
fa463f1163 Merge #20247: fuzz: Check for addrv1 compatibility before using addrv1 serializer. Fuzz addrv2 serialization.
903f3d0627 fuzz: Check for addrv1 compatibility before using addrv1 serializer (practicalswift)

Pull request description:

  Fuzz addrv2 address serialization.

  Check for addrv1 compatibility before using addrv1 serializer.

  Before this

  ```
  $ src/test/fuzz/netaddr_deserialize
  netaddr_deserialize: test/fuzz/deserialize.cpp:84: void
      (anonymous namespace)::AssertEqualAfterSerializeDeserialize(const T &) [T = CNetAddr]:
      Assertion `Deserialize<T>(Serialize(obj)) == obj' failed.
  ```

  After this patch:

  ```
  $ src/test/fuzz/netaddr_deserialize
  …
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 903f3d0627

Tree-SHA512: a9ddb71cc31c877fa3dd78dbc908d1e30b4790398fefe19e6541f1fca81e8560f7a11fa099ef3943b94401974c472e523484fdf66f1c23ff2e998558ba4b65de
2020-10-27 08:07:23 +01:00
practicalswift
903f3d0627 fuzz: Check for addrv1 compatibility before using addrv1 serializer 2020-10-26 16:37:35 +00:00
MarcoFalke
fa9b48549c test: Add test for -blockversion 2020-10-26 16:31:25 +01:00
MarcoFalke
fa3967efdb test: Replace ARRAYLEN with C++11 ranged for loop 2020-10-26 14:27:33 +01:00
MarcoFalke
fafc529053 test: Run AssetTest even if built --with-libs=no 2020-10-26 14:19:15 +01:00
MarcoFalke
faf58ab139 ci: Add --with-libs=no to one ci config 2020-10-26 14:18:50 +01:00
kanon
9c08f3332c doc: Missing comments for signet parameters 2020-10-26 04:38:23 -04:00
MarcoFalke
fa56d56d4e fuzz: Properly initialize PrecomputedTransactionData 2020-10-26 09:17:25 +01:00
Hennadii Stepanov
7b2e42ecc4 qt: Add WalletFrame::sizeHint
This change makes the layout of a WalletFrame widget correct even
without loaded wallets.
2020-10-25 21:17:18 +02:00
Jon Atack
398045ba8b cli -netinfo: print oversized/extreme ping times as "-" 2020-10-25 16:51:21 +01:00
Jon Atack
773f4c99c0 cli -netinfo: handle longer tor v3 local addresses 2020-10-25 16:51:18 +01:00
Jon Atack
33e987452f cli -netinfo: make age column variable-width
as it has a wide possible range and the new name ("age" instead of "uptime") is
much shorter.
2020-10-25 16:22:55 +01:00
Jon Atack
f8a1c4d946 cli -netinfo: various quick updates and fixes
- add new signet chain

- update change "uptime" column name to "age" per suggestion by 0xB10C (Timo)

- add an additional digit to mping field width

- change m_networks_size from size_t to uint8_t, as size_t was a holdover
  from m_networks_size being defined as size_t m_networks.size() in a draft

- order Peer struct members by decreasing memory size
2020-10-25 16:15:56 +01:00
Wladimir J. van der Laan
6866259fab net: Hardcoded seeds update for 0.21
Stats:

```
  IPv4   IPv6  Onion Pass
426728  59523   7900 Initial
426728  59523   7900 Skip entries with invalid address
426728  59523   7900 After removing duplicates
426727  59523   7900 Skip entries from suspicious hosts
123226  51785   7787 Enforce minimal number of blocks
121710  51322   7586 Require service bit 1
  4706   1427   3749 Require minimum uptime
  4124   1098   3681 Require a known and recent user agent
  4033   1075   3681 Filter out hosts with multiple bitcoin ports
   512    140    512 Look up ASNs and limit results per ASN and per net
```
2020-10-25 14:25:00 +01:00
RandyMcMillan
36e875b4c5 contrib: Add new versions to makeseeds.py and update gitignore 2020-10-25 14:08:33 +01:00
Sebastian Falbesoner
6f4e393646 refactor: remove use of boost::algorithm::replace_first 2020-10-25 12:29:54 +01:00
Jon Atack
fc5721723d wallet: fix SetFeeEstimateMode() error message
to clarify for the user the confusing error message that the missing fee rate
needs to be set in the conf_target param/option.
2020-10-25 00:35:38 +02:00
Jon Atack
052427eef1 wallet, bugfix: fix bumpfee with explicit fee rate modes 2020-10-24 22:03:11 +02:00
MarcoFalke
d67883d01e Merge bitcoin-core/gui#43: bugfix: Call setWalletActionsEnabled(true) only for the first wallet
20c9e03554 gui: Call setWalletActionsEnabled(true) only for the first wallet (Hennadii Stepanov)

Pull request description:

  On master (a78742830a) there is a bug:
  - open an encrypted wallet; please note that the "Encrypt Wallet..." menu item is disabled that is expected:
  ![Screenshot from 2020-08-03 12-38-37](https://user-images.githubusercontent.com/32963518/89169084-70060c80-d586-11ea-86b9-05ef38d08f41.png)
  - then open any other wallet; note that the "Encrypt Wallet..." menu item gets enabled that is wrong:
  ![Screenshot from 2020-08-03 12-42-36](https://user-images.githubusercontent.com/32963518/89169385-d68b2a80-d586-11ea-9813-a533a847e098.png)

  This PR fixes this bug.

ACKs for top commit:
  jonasschnelli:
    Tested ACK 20c9e03554 - I could reproduce the issue on master and have verify that this PR fixes it.
  achow101:
    ACK 20c9e03554

Tree-SHA512: 2c9ab94bde8c4f413b0a95c05bf3a1a29f5910e0f99d6639a11dd77758c78af25b060b3fecd78117066ef15b113feb79870bc1347cc04289da915c00623e5787
2020-10-24 09:01:37 +02:00
Hennadii Stepanov
f886a20b02 qt, wallet: Drop unused parameter in Wallet{Frame|View}::encryptWallet 2020-10-23 20:09:55 +03:00
Hennadii Stepanov
6e950118a3 qt, wallet: Remove unused AskPassphraseDialog::Decrypt
Co-authored-by: Sehyun Chung <sehyun@berkeley.edu>
2020-10-23 19:55:23 +03:00
Hennadii Stepanov
bf6855a909 wallet: Fix bug when just created encrypted wallet cannot get address 2020-10-23 19:24:24 +03:00
Elle Mouton
f15e780b9e refactor: Clean up CTxMemPool initializer list
Shorten the CTxMemPool initializer list using default initialization
for members that dont depend on the constuctor parameters.
2020-10-23 14:41:40 +02:00
Elle Mouton
e3310692d0 refactor: Make CTxMemPool::m_check_ratio a const and a constructor argument
Since m_check_ratio is only set once and since the CTxMemPool object is
no longer a global variable, m_check_ratio can be passed into the
constructor of CTxMemPool. Since it is only read from after
initialization, m_check_ratio can also be made a const and hence no
longer needs to be guarded by the cs mutex.
2020-10-23 14:41:30 +02:00
Elle Mouton
9d4b4b2c2c refactor: Avoid double to int cast for nCheckFrequency
Use a ratio instead of a frequency that requires a double to int cast
for determining how often a mempool sanity check should run.
2020-10-23 14:14:57 +02:00
Jonas Schnelli
49984b44cf Merge bitcoin-core/gui#71: Fix visual quality of text in QR image
6954156b40 qt: Fix visual quality of text in QR image (Hennadii Stepanov)
8071c75d45 qt, refactor: Limit scope of QPainter object (Hennadii Stepanov)

Pull request description:

  Master (197450f808):
  ![DeepinScreenshot_select-area_20200824001800](https://user-images.githubusercontent.com/32963518/90988962-96283680-e59f-11ea-8e20-42e9b23033f5.png)

  This PR (6954156b40):
  - macOS 10.15.6
  ![Screenshot from 2020-09-07 15-40-30](https://user-images.githubusercontent.com/32963518/92390251-2c716600-f123-11ea-96f0-0e9d35810c76.png)

  - Linux Mint 20
  ![Screenshot from 2020-09-07 15-48-13](https://user-images.githubusercontent.com/32963518/92390272-36936480-f123-11ea-8fee-4de23bb40ed9.png)

  Fix #54
  Fix https://github.com/bitcoin/bitcoin/issues/19103

  ---

  The first commit is easy to review with [`git diff --word-diff`](8071c75d45).

ACKs for top commit:
  jonasschnelli:
    Tested ACK 6954156b40 - tested on macOS 10.15. Fixes the problem.

Tree-SHA512: 6ecb3397d2a5094c5f00ee05fc09520751568404e000a8691b6de7e57f38c2d5da628694e5e45a2b4cc302a846bbc00014c40820233eb026d3ebd4f68c2c9913
2020-10-23 11:54:47 +02:00
fanquake
9af7c1993b Merge #20216: wallet: fix buffer over-read in SQLite file magic check
56a461f727 wallet: fix buffer over-read in SQLite file magic check (Sebastian Falbesoner)

Pull request description:

  Looking at our new SQLite database code, I noticed that there is a potential problem in the method `IsSQLiteFile()`:  If there is no terminating zero within the first 16 bytes of the file, the `magic` buffer would be over-read in the `std::string` constructor for `magic_str`. Fixed by using the "from buffer" variant of the string ctor (that also takes a size) rather than the "from c-string" variant (see http://www.cplusplus.com/reference/string/string/string/).

  The behaviour can be reproduced by the following steps:
  * Creating a file of at least 512 bytes in size (to pass the minimum size check) that doesn't contain zero bytes in the magic area, e.g. simply:
  `$ python3 -c "print('A'*512)" > /tmp/corrupt_wallet`
  * Showing content and size of the `magic_str` string in case the magic check fails
  * Create a simple unit test that simply calls `IsSQLiteFile` with the corrupt wallet file
  * Run the unit test and see the random gibberish output of `magic_str` after 16 `A`s :-)

  Or, TLDR variant, just get the branch https://github.com/theStack/bitcoin/tree/reproduce_sqlite_magic_overread, compile unit Tests and run the script `./reproduce_sqlite_magic_overread.sh`.

  Note that this is the minimal diff, probably it would be better to avoid `std::string` at all in this case and just use `memcmp`, strings that include null bytes are pretty confusing.

ACKs for top commit:
  promag:
    Code review ACK 56a461f727.
  practicalswift:
    ACK 56a461f727: patch looks correct
  achow101:
    ACK 56a461f727

Tree-SHA512: a7aadd4d38eb92337e6281df2980f4bde744dbb6cf112b9cd0f2cab8772730e302db9123a8fe7ca4e7e844c47e68957487adb2bed4518c40b4bed6a69d7922b4
2020-10-23 10:10:54 +08:00
Jonas Schnelli
9453fbf5a0 Merge #20: Wrap tooltips in the intro window
6ed4bcabc1 qt: Wrap tooltips in the intro window (Hennadii Stepanov)

Pull request description:

  On master (d6fe5b28df):

  ![DeepinScreenshot_select-area_20200702082514](https://user-images.githubusercontent.com/32963518/86319852-18e6e200-bc3e-11ea-902c-1b868540f4cf.png)

  With this PR:

  ![Screenshot from 2020-07-02 08-22-44](https://user-images.githubusercontent.com/32963518/86319881-28fec180-bc3e-11ea-9d76-522a5e71c32d.png)

  ---

  This is a move-only change.

  This was done while reviewing #15.

ACKs for top commit:
  jonasschnelli:
    utACK 6ed4bcabc1

Tree-SHA512: 67507c8c5628cd95fe8e377ce21e2c23d305faf1affa0b7579885553c20055ba07ed61e2aed3230d7cd22c3c90f76e0c58cb3238be9f153a8ca6e6d4e38753b0
2020-10-22 21:26:00 +02:00
Hennadii Stepanov
cadb77a6ab net: Add compat.h header for htonl function 2020-10-22 21:45:20 +03:00
Hennadii Stepanov
f796f0057b net: Drop unneeded headers when compat.h included 2020-10-22 21:45:20 +03:00
Hennadii Stepanov
467c346448 net: Drop unneeded Windows headers in compat.h
No interface from the mswsock.h header is used.
According to https://docs.microsoft.com/en-us/windows/win32/winsock/creating-a-basic-winsock-application
"The Winsock2.h header file internally includes core elements from
the Windows.h header file, so there is not usually an #include line
for the Windows.h header file in Winsock applications."
2020-10-22 21:45:10 +03:00
MarcoFalke
88271184e8 Merge #20112: test: Speed up wallet_resendwallettransactions with mockscheduler RPC
fa299ac273 test: Speed up wallet_resendwallettransactions test with mockscheduler RPC (MarcoFalke)

Pull request description:

  Also fixes #20143

ACKs for top commit:
  guggero:
    ACK fa299ac2

Tree-SHA512: 024ced4aa5f5c266e24fd0583d47b45b19c2a6ae25a06fabeacaa0ac996eec0c45f11cc34b2df17d01759b78ed31a991aa86978aafcc76cb0017382f601bf85a
2020-10-22 14:18:06 +02:00
MarcoFalke
7012db2a6b Merge #20176: test: Fix intermittent issue in p2p_feefilter
fa5a91a352 test: Fix typo (one tx is enough) in p2p_feefilter (MarcoFalke)
fa3af2c0d3 test: Fix intermittent issue in p2p_feefilter (MarcoFalke)

Pull request description:

  Fixes:

  ```
  Traceback (most recent call last):
    File "test/functional/test_framework/test_framework.py", line 126, in main
      self.run_test()
    File "test/functional/p2p_feefilter.py", line 63, in run_test
      self.test_feefilter()
    File "test/functional/p2p_feefilter.py", line 117, in test_feefilter
      txids = [miniwallet.send_self_transfer(fee_rate=Decimal('0.00020000'), from_node=node1)['wtxid'] for _ in range(3)]
    File "test/functional/p2p_feefilter.py", line 117, in <listcomp>
      txids = [miniwallet.send_self_transfer(fee_rate=Decimal('0.00020000'), from_node=node1)['wtxid'] for _ in range(3)]
    File "test/functional/test_framework/wallet.py", line 63, in send_self_transfer
      txid = from_node.sendrawtransaction(tx_hex)
    File "test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "test/functional/test_framework/authproxy.py", line 146, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: bad-txns-inputs-missingorspent (-25)

ACKs for top commit:
  guggero:
    ACK fa5a91a3

Tree-SHA512: 51d885753f72e1c91c4580709c15bdab60ff8c9d6f9bcb6db78a560e7e4dd7f76ce23add3303b374174afa3f11f74aa61db189a90c68d7f7655b15e64f51ed96
2020-10-22 11:02:33 +02:00
MarcoFalke
1cb4e339f9 Merge #20039: test: Convert amounts from float to decimal
5aadd4be18 Convert amounts from float to decimal (Prayank)

Pull request description:

  > decimal is preferred in accounting applications

  https://docs.python.org/3.8/library/decimal.html

  Decimal type saves an exact value so better than using float.

  ~~3 variables declared with type as 'Decimal' in [test/functional/mempool_accept.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/mempool_accept.py): fee, fee_expected, output_amount~~

  ~~Not required to convert to string anymore for using the above variables as decimal~~

  + fee, fee_expected, output_amount
  ~~+ 8 decimal places~~
  + Using value of coin['amount'] as decimal and removed 'int'
  + Removed unnecessary parentheses
  + Remove str() and use quotes

  Fixes https://github.com/bitcoin/bitcoin/issues/20011

ACKs for top commit:
  guggero:
    ACK 5aadd4be18

Tree-SHA512: 5877cf3837e5b65bec0fc8909de141a720bfa02a747513e21d20f3c41ec0cfecc524d2c347a96596b0a1a97900da2acf08b799f26b11d537e4dcddc6ce45f38e
2020-10-22 10:13:44 +02:00
Sebastian Falbesoner
56a461f727 wallet: fix buffer over-read in SQLite file magic check
If there is no terminating zero within the 16 magic bytes, the buffer would be
over-read in the std::string constructor. Fixed by using the "from buffer"
variant of the ctor (that also takes a size) rather than the "from c-string"
variant.
2020-10-22 03:39:55 +02:00
fanquake
dda18e7310 Merge #20214: test: Fix rpc_net intermittent issue
fa5f46600f test: Fix rpc_net intermittent issue (MarcoFalke)

Pull request description:

  Without the sync, the nodes might generate blocks at the same height and thus never be able to sync

ACKs for top commit:
  practicalswift:
    ACK fa5f46600f: patch looks correct

Tree-SHA512: 21255795c2121c71fc620beb766855e57c7af94a668331d1b625665e22eb4b485a2b5c3ad2bb9a7042744f3c3e49c71251bcec41ba25bca03fd54aae32968a3a
2020-10-22 08:40:38 +08:00
MarcoFalke
fa5f46600f test: Fix rpc_net intermittent issue 2020-10-21 21:07:28 +02:00
Prayank
5aadd4be18 Convert amounts from float to decimal
+ fee, fee_expected, output_amount
+ Using value of coin['amount'] as decimal and removed 'int'
+ Removed unnecessary parentheses
+ Remove str() and use quotes
2020-10-21 21:21:39 +05:30
Sebastian Falbesoner
6b56c1f4d0 test: remove last_{block,header}_equals() in p2p_fingerprint.py
Testing that requests to very old blocks / block headers fail can simply be
done by checking that the node doesn't respond with any "blocks" / "headers"
message at all.
Also removes unnecessary sending of block/header requests and replaces
time.sleep(3) with node0.sync_with_ping().
2020-10-21 15:31:24 +02:00
Russell Yanofsky
01476a88a6 wallet: Make -wallet setting not create wallets
This changes -wallet setting to only load existing wallets, not create new ones.

- Fixes settings.json corner cases reported by sjors & promag:
  https://github.com/bitcoin-core/gui/issues/95,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

- Prevents accidental creation of wallets reported most recently by jb55
  http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

- Simplifies behavior after #15454. #15454 took the big step of disabling
  creation of the default wallet. This PR extends it to avoid creating other
  wallets as well. With this change, new wallets just aren't created on
  startup, instead of sometimes being created, sometimes not. #15454 release
  notes are updated here and are simpler.

This change should be targeted for 0.21.0. It's a bug fix and simplifies
behavior of the #15937 / #19754 / #15454 features added in 0.21.0.
2020-10-21 08:48:43 -04:00
MarcoFalke
b46f37ba5e Merge #20198: Show name, format and if uses descriptors in bitcoin-wallet tool
fa4074b395 Show name, format and if uses descriptors in bitcoin-wallet tool (Jonas Schnelli)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    ACK fa4074b395
  jonatack:
    re-ACK fa4074b395

Tree-SHA512: cf6ee96ff21532fc4b0ba7a0fdfdc1fa485c9b1495447350fe65cd0bd919e0e0280613933265cdee069b8c29ccf015ac374535a70cac3d4fb89f4d08b3a03519
2020-10-21 14:48:43 +02:00
MarcoFalke
47fc883106 Merge #19967: test: Replace (dis)?connect_nodes globals with TestFramework methods
3c7d9ab8c8 test: Move (dis)?connect_nodes globals into TestFramework as helpers (Elliott Jin)
4b16c61461 scripted-diff: test: Replace uses of (dis)?connect_nodes global (Prayank)
be386840d4 test: Replace use of (dis)?connect_nodes globals (Elliott Jin)

Pull request description:

  `util.py` defines global helper functions `connect_nodes` and `disconnect_nodes`; however, these functions are confusing because they take a node and an index (instead of two indexes).

  The `TestFramework` object has enough context to convert from `i` to `self.nodes[i]`, so we can replace all instances of `connect_nodes(self.nodes[a], b)` with `self.connect_nodes(a, b)`. Similarly, we can replace instances of `disconnect_nodes`.

  The approach taken in this PR builds on #19945 but uses a scripted-diff for the majority of the changes.

  Fixes: #19821

ACKs for top commit:
  MarcoFalke:
    ACK 3c7d9ab8c8
  guggero:
    ACK 3c7d9ab8

Tree-SHA512: e027092748602904abcd986d7299624c8754c3236314b6d8e392e306741c212f266c2207e385adfb194f67ae6559a585ee7b15d639b1d65c4651dbf503e5931a
2020-10-21 14:43:12 +02:00
Vasil Dimov
af3b0dfc54 net: fix output of peer address in version message
If `-logips -debug=net` is specified then we print the contents of the
version message we send to the peer, including his address. Because the
addresses in the version message use pre-BIP155 encoding they cannot
represent a Tor v3 address and we would actually send 16 `0`s instead (a
dummy IPv6 address). However we would print the full address in the log
message. Before this fix:

```
2020-10-21T12:24:17Z send version message: version 70016, blocks=653500, us=[::]:0, them=xwjtp3mj427zdp4tljiiivg2l5ijfvmt5lcsfaygtpp6cw254kykvpyd.onion:8333, peer=0
```

This is confusing because we pretend to send one thing while we actually
send another. Adjust the printout to reflect what we are sending. After
this fix:

```
2020-10-21T12:26:54Z send version message: version 70016, blocks=653500, us=[::]:0, them=[::]:0, peer=0
```
2020-10-21 14:40:42 +02:00
MarcoFalke
0f86e7f128 Merge #20157: Bugfix: chainparams: Add missing (always enabled) Taproot deployment for Signet
2d5793c016 Bugfix: chainparams: Add missing (disabled) Taproot deployment for Signet (Luke Dashjr)

Pull request description:

  Is there a way we can trigger compiler warnings if a deployment is undefined?

ACKs for top commit:
  decryp2kanon:
    utACK 2d5793c016
  MarcoFalke:
    review ACK 2d5793c016

Tree-SHA512: 135cefae0f8dc552b0f682c2b87cabca7a4716290a36410a55968850e803a5049234e3cc597c8ef8d7917ae5d5ea3fb851e160df171b6793114c6bc01c5ea3e7
2020-10-21 14:19:33 +02:00
MarcoFalke
fa650ca7f1 Use -Wswitch for TxoutType where possible 2020-10-21 13:51:21 +02:00
MarcoFalke
fa59e0b5bd test: Add missing script_standard_Solver_success cases 2020-10-21 13:50:47 +02:00
Jonas Schnelli
fa4074b395 Show name, format and if uses descriptors in bitcoin-wallet tool 2020-10-21 13:28:15 +02:00
Luke Dashjr
bbb42a6896 RPC: createwallet: Nicer error message if descriptor wallet requested and sqlite support not compiled in 2020-10-20 13:44:43 +00:00
Luke Dashjr
6608fec332 GUI: Create Wallet: Nicely disable descriptor wallet checkbox if sqlite support not compiled in 2020-10-20 13:44:43 +00:00
Luke Dashjr
7b54d768e1 Make sqlite support optional (compile-time) 2020-10-20 13:44:43 +00:00
Luke Dashjr
2d5793c016 Bugfix: chainparams: Add missing (disabled) Taproot deployment for Signet 2020-10-20 13:32:23 +00:00
Elliott Jin
3c7d9ab8c8 test: Move (dis)?connect_nodes globals into TestFramework as helpers 2020-10-20 00:43:00 -07:00
Prayank
4b16c61461 scripted-diff: test: Replace uses of (dis)?connect_nodes global
-BEGIN VERIFY SCRIPT-

 # max-depth=0 excludes test/functional/test_framework/...
 FILES=$(git grep -l --max-depth 0 "connect_nodes" test/functional)

 # Replace (dis)?connect_nodes(self.nodes[a], b) with self.(dis)?connect_nodes(a, b)
 sed -i 's/\b\(dis\)\?connect_nodes(self\.nodes\[\(.*\)\]/self.\1connect_nodes(\2/g' $FILES

 # Remove imports in the middle of a line
 sed -i 's/\(dis\)\?connect_nodes, //g' $FILES
 sed -i 's/, \(dis\)\?connect_nodes//g' $FILES

 # Remove imports on a line by themselves
 sed -i '/^\s*\(dis\)\?connect_nodes,\?$/d' $FILES
 sed -i '/^from test_framework\.util import connect_nodes$/d' $FILES

-END VERIFY SCRIPT-

Co-authored-by: Elliott Jin <elliott.jin@gmail.com>
2020-10-20 00:42:00 -07:00
Elliott Jin
be386840d4 test: Replace use of (dis)?connect_nodes globals
A later scripted-diff commit replaces the majority of uses, which all
follow this pattern:

    (dis)?connect_nodes(self.nodes[a], b)

This commit replaces the few "special cases".
2020-10-20 00:27:45 -07:00
practicalswift
79ef8324d4 tests: Add fuzzing harness for CConnman 2020-10-20 06:18:54 +00:00
Samuel Dobson
f5bd46a4cc Merge #20125: rpc, wallet: Expose database format in getwalletinfo
624bab00dd test: add coverage for getwalletinfo format field (Jon Atack)
5e737a0092 rpc, wallet: Expose database format in getwalletinfo (João Barbosa)

Pull request description:

  Support for sqlite based wallets was added in #19077. This PR adds the `format` key in `getwalletinfo` response, that can be `bdb` or  `sqlite`.

ACKs for top commit:
  jonatack:
    Tested ACK 624bab00dd
  laanwj:
    Code review ACK 624bab00dd.
  MarcoFalke:
    doesn't hurt ACK 624bab00dd
  hebasto:
    ACK 624bab00dd, tested on Linux Mint 20 (x86_64).
  meshcollider:
    utACK 624bab00dd

Tree-SHA512: a81f8530f040f6381d33e073a65f281993eccfa717424ab6e651c1203cbaf27794dcb7175570459e7fdaa211565bc060d0a3ecbe70d2b6f9c49b8d5071e4441c
2020-10-20 12:35:33 +13:00
Pieter Wuille
812baaa1f8 Switch to BIP341's suggested scheme for outputs without script 2020-10-19 13:50:36 -07:00
MarcoFalke
45385018e1 Merge #20162: p2p: declare Announcement::m_state as uint8_t, add getter/setter
c8abbc9d1f p2p: declare Announcement::m_state as uint8_t, add getter/setter (Jon Atack)

Pull request description:

  Change `Announcement::m_state` in `tx_request.cpp` from type `State` to `uint8_t` and add a getter and setter for the conversion to/from `State`. This should silence these travis ci gcc compiler warnings:

  ```
  txrequest.cpp:73:21: warning: ‘{anonymous}::Announcement::m_state’ is
  too small to hold all values of ‘enum class {anonymous}::State’
       State m_state : 3;
                       ^
  ```

  The gcc warnings are based on the maximum value held by the underlying uint8_t enumerator type, even though the intention of the bitfield declaration is the maximum declared enumerator value. They have apparently been silenced in gcc 8.4+ and 9.3+ according to https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61414.

ACKs for top commit:
  sipa:
    utACK c8abbc9d1f
  ajtowns:
    ACK c8abbc9d1f -- quick code review
  hebasto:
    ACK c8abbc9d1f, tested on Bionic (x86_64, gcc 7.5.0):

Tree-SHA512: 026721dd7a78983a72da77638d3327d2b252bef804e489278a852f000046c028d6557bbd6c2b4cea391d4e01f9264a1be842d502047cb90b2997cc37bee59e61
2020-10-19 13:09:32 +02:00
fanquake
4f807348af Merge #20185: doc: Merge release notes
fa38093bee doc: Merge release notes (MarcoFalke)

Pull request description:

  Now that all features are merged, open release notes editing at the wiki

  https://github.com/bitcoin-core/bitcoin-devwiki/wiki/0.21.0-Release-Notes-Draft

ACKs for top commit:
  fanquake:
    ACK fa38093bee

Tree-SHA512: ced161a2fcb0366a77a05b020c8dd65a0cf0a0de17b8260bbca9b5833ed370f92b1b81116bfc59b83380bf28b55d8963c628cf13a0cad603e5c823341b446065
2020-10-19 18:47:55 +08:00
MarcoFalke
fa38093bee doc: Merge release notes 2020-10-19 12:00:16 +02:00
MarcoFalke
4769942d90 Merge #19624: Warn on unknown rw_settings
fa48405ef8 Warn on unknown rw_settings (MarcoFalke)

Pull request description:

  Log a warning to debug log if unknown settings are encountered. This should probably only ever happen when the software is upgraded.

  Something similar is already done for the command line and config file. See:

  * test: Add test for unknown args #16234 (commit fa7dd88b71)

ACKs for top commit:
  ryanofsky:
    Code review ACK fa48405ef8. Looks good and I could see this being helpful for debugging. Thanks for taking suggestions

Tree-SHA512: cec7d88adf84fa0a842f56b26245157736eb50df433db951e622ea07fd145b899822b24cdab1d8b36c066415ce4f0ef09b493fa8a8d691532822a59c573aafa7
2020-10-19 11:30:49 +02:00
MarcoFalke
fa5a91a352 test: Fix typo (one tx is enough) in p2p_feefilter 2020-10-19 09:42:40 +02:00
MarcoFalke
d9d9a29352 Merge #20179: test: Fix intermittent issue in wallet_import_rescan
faab86f6c8 test: Fix intermittent issue in wallet_send (MarcoFalke)
faca3734c0 test: Fix intermittent issue in wallet_import_rescan (MarcoFalke)

Pull request description:

  Fixes the issue

  ```
   node2 2020-10-16T14:26:28.699593Z (mocktime: 2020-10-16T16:26:46Z) [msghand] Timeout downloading block 1131f0318b913e078402524f1e63e53d52171844dd8246a03b970e540cb56924 from peer=0, disconnecting

ACKs for top commit:
  fjahr:
    utACK faab86f6c8

Tree-SHA512: 739e8ad488c50e6beae676fb6cb021033cca6192da4acb4512c182bd8843f2f42a4a07474118285cb1d91798904433e4abb09476cc9ce115aae87ce64db69eaf
2020-10-19 09:27:32 +02:00
MarcoFalke
152ddb3197 Merge #20180: test: Fix -Wunused-function warnings if configured --without-libs
76bbcc414f test: Fix -Wunused-function warning if configured --without-libs (Hennadii Stepanov)

Pull request description:

  On master (80c8a02f1b) compiling with gcc:
  ```
  $ ./configure --without-libs
  $ make clean && make
  ...
  test/script_tests.cpp:1369:23: warning: ‘CScriptWitness script_tests::ScriptWitnessFromJSON(const UniValue&)’ defined but not used [-Wunused-function]
   1369 | static CScriptWitness ScriptWitnessFromJSON(const UniValue& univalue)
        |                       ^~~~~~~~~~~~~~~~~~~~~
  test/script_tests.cpp:1357:28: warning: ‘std::vector<CTxOut> script_tests::TxOutsFromJSON(const UniValue&)’ defined but not used [-Wunused-function]
   1357 | static std::vector<CTxOut> TxOutsFromJSON(const UniValue& univalue)
        |                            ^~~~~~~~~~~~~~
  test/script_tests.cpp:1350:28: warning: ‘CMutableTransaction script_tests::TxFromHex(const string&)’ defined but not used [-Wunused-function]
   1350 | static CMutableTransaction TxFromHex(const std::string& str)
        |                            ^~~~~~~~~
  ...
  ```

  This change is move-only (nice to review with `git diff --color-moved`).

ACKs for top commit:
  practicalswift:
    ACK 76bbcc414f: diff looks correct
  fanquake:
    ACK 76bbcc414f - verified that this fixes the warnings. As mentioned can be reviewed with `git diff HEAD~ --color-moved=dimmed_zebra`.

Tree-SHA512: 7799ac190d1e3f15e38b36cfcd1f8d138be80cab6c6cfad8f7828e07deffc2037d52f1d967f7f233a3a8ed74eee184f5275076c2f364c3e363c77a1f40aa5030
2020-10-19 09:18:16 +02:00
fanquake
62af467ff0 Merge #20082: [bugfix] random: fixes read buffer to use min rather than max
bd5215103e random: fixes read buffer resizing in RandAddSeedPerfmon (Ethan Heilman)

Pull request description:

  As shown below when resizing the read buffer `vData` `std::max((vData.size() * 3) / 2, nMaxSize)` is used. This means that the buffer size immediately jumps to `nMaxSize`. I believe the intend of this code is to grow the buffer size through several steps rather than immediately resize it to the max size.

  ```cpp
      std::vector<unsigned char> vData(250000, 0);
      long ret = 0;
      unsigned long nSize = 0;
      const size_t nMaxSize = 10000000; // Bail out at more than 10MB of performance data
      while (true) {
          nSize = vData.size();
          ret = RegQueryValueExA(HKEY_PERFORMANCE_DATA, "Global", nullptr, nullptr, vData.data(), &nSize);
          if (ret != ERROR_MORE_DATA || vData.size() >= nMaxSize)
              break;
          vData.resize(std::max((vData.size() * 3) / 2, nMaxSize)); // Grow size of buffer exponentially
      }
  ```

  vData always starts at size 250,000 and nMaxSize is always 10,000,000 so the first time this line is reached:
  ```cpp
  vData.resize(std::max((vData.size() * 3) / 2, nMaxSize));
  ```
  the effect will always be to resize vData to nMaxSize. Then because the loop terminates when vData.size >= 10,000,000 only one resize operation will take place.

  To fix this issue we replace `std::min` with `std::max`

  This PR also adds a comment clarifying the behavior of this function the first time it is called.

ACKs for top commit:
  fanquake:
    ACK bd5215103e - thanks for taking a look at this Ethan. Swapping from `std::max` to `std::min` here certainly seems correct.

Tree-SHA512: 7c65f700e5bbe44bc2f1ffdcdc99ec19c542894c95b5ee9791facd09d02afae88d1f8f35af129719e4860db94bc790856e7adb1d218a395381e7c2913b95f1d0
2020-10-19 14:04:10 +08:00
Andrew Chow
5f720544f3 wallet: Add GetClosestWalletFeature function
Given a version number, get the closest supported WalletFeature
for a version number.
2020-10-19 00:14:38 -04:00
Andrew Chow
842ae3842d wallet: Add utility method for CanSupportFeature 2020-10-19 00:14:38 -04:00
fanquake
a1e0359618 Merge #19986: refactor: clean up -Wlogical-op warning in wallet/scriptpubkeyman.cp
95fedd33a2 refactor: Clean up -Wlogical-op warning (maskoficarus)

Pull request description:

  This is a quick patch that fixes #19912 . This change prevents a -Wlogical-op warning that occurs because we're treating a const int value as a boolean. There's no sense checking if a non-zero constant has a value, so I've removed the check.

  #18836 also addresses the same warning, but has a larger scope and will require more review. This pull request will act as a patch to prevent this compile warning until 18836 is merged.

ACKs for top commit:
  MarcoFalke:
    review ACK 95fedd33a2
  hebasto:
    ACK 95fedd33a2, tested on Linux Mint 20 (x86_64):

Tree-SHA512: 13a9d7f7cb472f4c22a01ca2f9771a75267ff769bdae9d0dc6b2c7f3b05369f6dfa859be2b172b39c15ede6c44cddf556380b3565e42850faa65ccd3fe6e175b
2020-10-19 11:07:11 +08:00
fanquake
c92aa8357c Merge #19911: net: guard vRecvGetData with cs_vRecv and orphan_work_set with g_cs_orphans
da0988daf1 scripted-diff: rename vRecvGetData (Neha Narula)
ba951812ec Guard vRecvGetData (now in net processing) with its own mutex (Neha Narula)
2d9f2fca43 Move vRecvGetData to net processing (Neha Narula)
673247b58c Lock before checking if orphan_work_set is empty; indicate it is guarded (Neha Narula)
8803aee668 Move m_orphan_work_set to net_processing (Neha Narula)
9c47cb29f9 [Rename only] Rename orphan_work_set to m_orphan_work_set. (Neha Narula)

Pull request description:

  Add annotations to guard `vRecvGetData` and `orphan_work_set` and fix up places where they were accessed without a lock. There is no current data race because they happen to be accessed by only one thread, but this might not always be the case.

  Original discussion: https://github.com/bitcoin/bitcoin/pull/18861#discussion_r451778445

ACKs for top commit:
  MarcoFalke:
    review ACK da0988daf1 🐬
  jnewbery:
    Code review ACK da0988daf1
  hebasto:
    ACK da0988daf1, I have reviewed the code and it looks correct, I agree it can be merged.

Tree-SHA512: 31cadd319ddc9273a87e77afc4db7339fd636e816b5e742eba5cb32927ac5cc07a672b2268d2d38a75a0f1b17d93836adab9acf7e52f26ea9a43f54efa57257e
2020-10-19 10:25:38 +08:00
Hennadii Stepanov
76bbcc414f test: Fix -Wunused-function warning if configured --without-libs
This is a move-only change.
2020-10-18 19:06:01 +03:00
MarcoFalke
faab86f6c8 test: Fix intermittent issue in wallet_send 2020-10-18 11:02:59 +02:00
MarcoFalke
faca3734c0 test: Fix intermittent issue in wallet_import_rescan 2020-10-18 10:10:20 +02:00
MarcoFalke
80c8a02f1b Merge #20159: test: mining_getblocktemplate_longpoll.py improvements (use MiniWallet, add logging)
b128b56672 test: add logging for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner)
8ee3536b2b test: remove unused helpers random_transaction(), make_change() and gather_inputs() (Sebastian Falbesoner)
fddce7e199 test: use MiniWallet for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (mining_getblocktemplate_longpoll.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. Also adds missing log messages for the subtests.

  This was the only functional test that used the `random_transaction` helper in `test_framework/util.py`, hence it is removed, together with other helpers (`make_change` and `gather_inputs`) that were again only used by `random_transaction`.

ACKs for top commit:
  MarcoFalke:
    ACK b128b56672

Tree-SHA512: 09a5fa7b0f5976a47040f7027236d7ec0426d5a4829a082221c4b5fae294470230e89ae3df0bca0eea26833162c03980517f5cc88761ad251c3df4c4a49bca46
2020-10-17 17:57:23 +02:00
MarcoFalke
fa3af2c0d3 test: Fix intermittent issue in p2p_feefilter 2020-10-17 17:48:03 +02:00
fanquake
5d644778da Merge #20169: Taproot follow-up: Make ComputeEntrySchnorr and ComputeEntryECDSA const to clarify contract
51365674e8 script: Make ComputeEntrySchnorr and ComputeEntryECDSA const to clarify contract (practicalswift)

Pull request description:

  Make `ComputeEntrySchnorr` and `ComputeEntryECDSA` `const` to clarify contract.

ACKs for top commit:
  benthecarman:
    ACK 51365674e8
  theStack:
    ACK 51365674e8 👌
  sipa:
    utACK 51365674e8

Tree-SHA512: 0f7a72bf6df7a97d21045ead9db398d2a9527c358aeeb894dec34a5386da4cc316e2f3326716e960ef8aa47bf73b99d1f92bb6d45dfa7871c84624bcad8a79f1
2020-10-17 22:46:56 +08:00
fanquake
b3527fd2e9 Merge #20168: contrib: Fix gen_key_io_test_vectors.py imports
fa68755364 contrib: Fix gen_key_io_test_vectors.py imports (MarcoFalke)

Pull request description:

  The script currently fails with

  ```
  Traceback (most recent call last):
    File "./gen_key_io_test_vectors.py", line 18, in <module>
      from segwit_addr import bech32_encode, decode, convertbits, CHARSET
  ImportError: cannot import name 'decode' from 'segwit_addr'
  ```

  Fix that.
  Also, unrelated cleanup to use the `bytearray.hex()` method instead of importing a library. https://docs.python.org/3.5/library/stdtypes.html#bytes.hex

ACKs for top commit:
  theStack:
    tested ACK fa68755364

Tree-SHA512: 45ff7d710de3d0ef5ac6d91543cff0edff6189d2cd00b0f8889f4361e66ef1825f12aea9e71d62038c14a7a531bfc95ffe9a1df83b85aa7f3dd666df07a6be81
2020-10-17 10:37:01 +08:00
Jon Atack
c8abbc9d1f p2p: declare Announcement::m_state as uint8_t, add getter/setter
to silence these Travis CI GCC compiler warnings:

txrequest.cpp:73:21: warning: ‘{anonymous}::Announcement::m_state’ is
too small to hold all values of ‘enum class {anonymous}::State’
     State m_state : 3;
                     ^
The warnings are based on the maximum value held by the underlying uint8_t
enumerator type, though the intention of the bitfield declaration is the
maximum declared enumerator value.

The warning been silenced in GCC 8.4+ and 9.3+ according to
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61414
2020-10-16 23:33:44 +02:00
Sebastian Falbesoner
b128b56672 test: add logging for mining_getblocktemplate_longpoll.py 2020-10-16 15:41:00 +02:00
Sebastian Falbesoner
8ee3536b2b test: remove unused helpers random_transaction(), make_change() and gather_inputs() 2020-10-16 15:40:54 +02:00
MarcoFalke
9e8d2bd076 Merge bitcoin-core/gui#97: Relax GUI freezes during IBD (when using wallets)
0d9d2a1f7c Only update the updateSmartFeeLabel once in sync (Jonas Schnelli)

Pull request description:

  Calling `updateSmartFeeLabel` and therefore `estimateSmartFee` is pointless during IBD.

  GUI freezes appear because `estimateSmartFee` competes with `processBlock` for the `m_cs_fee_estimator` lock leading to multiple seconds of blocking the GUI thread in `updateSmartFeeLabel`.

ACKs for top commit:
  ryanofsky:
    Code review ACK 0d9d2a1f7c. Clever fix. Didn't test but I remember I could reproduce the startup issue easily before by putting a sleep in estimateSmartFee.
  promag:
    Code review ACK 0d9d2a1f7c.
  hebasto:
    ACK 0d9d2a1f7c, tested on Linux Mint 20 (x86_64) with `QT_FATAL_WARNINGS=1` and `-debug=qt`.

Tree-SHA512: 85ec2266f06ddd7b523e24d2a462f10ed965d5b4d479005263056f81b7fe49996e1568dafb84658af406e9202ed3bfa846d59c10bb951e0f97cee230e30fafd5
2020-10-16 14:08:35 +02:00
MarcoFalke
fa68755364 contrib: Fix gen_key_io_test_vectors.py imports 2020-10-16 13:48:17 +02:00
MarcoFalke
fa7fb0e442 test: Default blockversion to 4 in feature_block
There is one tests that checks version=1 blocks are rejected. For all
other tests the version doesn't matter as long as it is large enough.
2020-10-16 13:28:17 +02:00
Sebastian Falbesoner
fddce7e199 test: use MiniWallet for mining_getblocktemplate_longpoll.py
This test can now be run even with the Bitcoin Core wallet disabled.
2020-10-16 13:05:31 +02:00
MarcoFalke
fa2b778d0c test: Remove unused -blockversion from tests 2020-10-16 13:01:07 +02:00
MarcoFalke
cb21d864c5 Merge #19401: QA: Use GBT to get block versions correct
d438d609cd QA: Use GBT to get block versions correct (Luke Dashjr)
1df2cd1c8f QA: blocktools: Accept block template to create_block (Luke Dashjr)

Pull request description:

  The goal here is to decouple unrelated tests from the details of block versions.

  Currently, these tests are forcing specific versions of blocks for no real reason.

ACKs for top commit:
  fjahr:
    re-ACK d438d609cd
  benthecarman:
    ACK d438d60

Tree-SHA512: 523b1cd4dac8d65c88432e126ce7f60df96ca4b94f7ecc8e83ba4ffbade23e2afe7055fdf586ce3c195a533f2004e63fff83add4267b39473a581c9f1c6d5340
2020-10-16 11:43:21 +02:00
Wladimir J. van der Laan
2947ae6f85 Merge #20035: signet: Fix uninitialized read in validation
fa723e3d43 Initialize default-initialized uint256 consensus params to zero explicitly (MarcoFalke)
fa729cdb2c doc: Move assumed-values doxygen comments to header (MarcoFalke)
fa64892b82 signet: Fix uninitialized read in validation (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    re-ACK fa723e3d43: patch still looks correct
  kallewoof:
    ReACK fa723e3d43
  theStack:
    re-ACK fa723e3d43 🍐

Tree-SHA512: db562bcc15af23bbcbf485f0bbf7564c64c144a4368230fd7682e8861d9500f6f5240351e31c560140df43b2e8456eafd9d27d1e8dd682b20afcc279a39dc329
2020-10-16 11:12:08 +02:00
fanquake
82d3596dfe Merge #20161: Minor Taproot follow-ups
1d22300b99 Address functional test nits (Pieter Wuille)
5669642a0b docs: mention BIPs 340-342 in doc/bips.md (Pieter Wuille)

Pull request description:

  This addresses some nits in the tests, and adds entries for BIP 340-342 to doc/bips.md.

ACKs for top commit:
  fanquake:
    ACK 1d22300b99
  benthecarman:
    ACK 1d22300b99

Tree-SHA512: ad8f937dc6a34db86c585f65beb80e7eceda1822d9a20c86346a319908870381062856d0b95b42049a2791317a038c77fbcbf896c9f4aaa7318e4864b7fcf7a4
2020-10-16 15:01:42 +08:00
practicalswift
51365674e8 script: Make ComputeEntrySchnorr and ComputeEntryECDSA const to clarify contract 2020-10-16 06:26:46 +00:00
MarcoFalke
fa723e3d43 Initialize default-initialized uint256 consensus params to zero explicitly 2020-10-16 06:29:23 +02:00
fanquake
cbb5f3a2d5 Merge #19836: rpc: Properly deserialize txs with witness before signing
3333077823 rpc: Adjust witness-tx deserialize error message (MarcoFalke)
cccc752569 rpc: Properly deserialize txs with witness before signing (MarcoFalke)

Pull request description:

  Signing a transaction can only happen when the transaction has inputs. A transaction with inputs can always be deserialized as witness-transaction. If `try_no_witness` decoding is attempted, this will lead to rare intermittent failures.

  Fixes #18803

ACKs for top commit:
  achow101:
    ACK 3333077823
  ajtowns:
    ACK 3333077823

Tree-SHA512: 73f8a5cdfe03fb0e68908d2fa09752c346406f455694a020ec0dd1267ef8f0a583b8e84063ea74aac127106dd193b72623ca6d81469a94b3f5b3c766ebf2c42b
2020-10-16 12:05:26 +08:00
Karl-Johan Alm
e9c8e6eea2 doc: add contrib/signet readme 2020-10-16 12:50:18 +09:00
Pieter Wuille
1d22300b99 Address functional test nits 2020-10-15 15:39:09 -07:00
Pieter Wuille
5669642a0b docs: mention BIPs 340-342 in doc/bips.md 2020-10-15 14:20:20 -07:00
Wladimir J. van der Laan
9ad7cd2887 Merge #20090: [doc] Tiny followups to new getpeerinfo connection type field
41dca087b7 [trivial] Extract connection type doc into file where it is used. (Amiti Uttarwar)
3069b56a45 [doc] Improve help for getpeerinfo connection_type field. (Amiti Uttarwar)

Pull request description:

  two commits addressing small followups from #19725

  * first commit adds a clarification in the release notes that this field shouldn't be expected to be stable (suggested by sdaftuar in https://github.com/bitcoin/bitcoin/pull/19725#issuecomment-697421878)

  * second commit moves the `CONNECTION_TYPE_DOC` object out of the header file to reduce the size of the binary (suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/19725#discussion_r495467895, he tested and found a decrease of 10kB)

ACKs for top commit:
  achow101:
    ACK 41dca087b7
  laanwj:
    Code review ACK 41dca087b7

Tree-SHA512: a555df978b4341fbe05deeb40a8a655f0d3c5c1c0adcc1737fd2cf61b204a5a24a301ca0c2b5a3616554d4abf8c57074d22dbda5a50d8450bc22c57679424985
2020-10-15 20:45:56 +02:00
Wladimir J. van der Laan
9855422e65 Merge #17428: p2p: Try to preserve outbound block-relay-only connections during restart
a490d074b3 doc: Add anchors.dat to files.md (Hennadii Stepanov)
0a85e5a7bc p2p: Try to connect to anchors once (Hennadii Stepanov)
5543c7ab28 p2p: Fix off-by-one error in fetching address loop (Hennadii Stepanov)
4170b46544 p2p: Integrate DumpAnchors() and ReadAnchors() into CConnman (Hennadii Stepanov)
bad16aff49 p2p: Add CConnman::GetCurrentBlockRelayOnlyConns() (Hennadii Stepanov)
c29272a157 p2p: Add ReadAnchors() (Hennadii Stepanov)
567008d2a0 p2p: Add DumpAnchors() (Hennadii Stepanov)

Pull request description:

  This is an implementation of #17326:
  - all (currently 2) outbound block-relay-only connections (#15759) are dumped to `anchors.dat` file
  - on restart a node tries to connect to the addresses from `anchors.dat`

  This PR prevents a type of eclipse attack when an attacker exploits a victim node restart to force it to connect to new, probably adversarial, peers.

ACKs for top commit:
  jnewbery:
    code review ACK a490d074b3
  laanwj:
    Code review ACK a490d074b3

Tree-SHA512: 0f5098a3882f2814be1aa21de308cd09e6654f4e7054b79f3cfeaf26bc02b814ca271497ed00018d199ee596a8cb9b126acee8b666a29e225b08eb2a49b02ddd
2020-10-15 20:19:55 +02:00
Jon Atack
7b5bd3102e test: add getnetworkinfo network name regression tests 2020-10-15 19:21:43 +02:00
Jon Atack
9a75e1e569 rpc: update GetNetworksInfo() to not return unsupported networks 2020-10-15 19:21:41 +02:00
Jon Atack
ba8997fb2e net: update GetNetworkName() with all enum Network cases 2020-10-15 19:21:38 +02:00
Wladimir J. van der Laan
0d22482353 Merge #20002: net, rpc, cli: expose peer network in getpeerinfo; simplify/improve -netinfo
6272604bef refactor: enable -netinfo to add future networks (i2p, cjdns) (Jon Atack)
82fd40216c refactor: promote some -netinfo localvars to class members (Jon Atack)
5133fab37e cli: simplify -netinfo using getpeerinfo network field (Jon Atack)
4938a109ad rpc, test: expose CNodeStats network in RPC getpeerinfo (Jon Atack)
6df7882029 net: add peer network to CNodeStats (Jon Atack)

Pull request description:

  This PR:

  - builds on #19991 and #19998
  - exposes peer networks via a new getpeerinfo `network` field ("ipv4", "ipv6", or "onion"), and adds functional tests
  - updates -netinfo to use getpeerinfo `network` rather than detecting the peer networks client-side
  - refactors -netinfo to easily add future networks

ACKs for top commit:
  laanwj:
    ACK 6272604bef

Tree-SHA512: 28883487585135ceaaf84ce09131f2336e3193407f2e3df0960e3f4ac340f500ab94ffecb9d06a4c49bc05e3cca4f914ea4379860bea0bd5df2f834f74616015
2020-10-15 17:44:38 +02:00
Jon Atack
624bab00dd test: add coverage for getwalletinfo format field 2020-10-15 16:46:34 +02:00
Ivan Metlushko
538be4219a wallet: fix importdescriptor silent fail 2020-10-15 18:02:58 +07:00
Wladimir J. van der Laan
711ddce943 Merge #20131: test: Remove unused nVersion=1 in p2p tests
faad92fe1c test: Remove unused nVersion=1 in p2p tests (MarcoFalke)

Pull request description:

  After commit ddefb5c0b7 nVersion is no
  longer used in p2p logic when sending messages. Only when receiving
  messages, but in this test no messages are received.

ACKs for top commit:
  laanwj:
    Code review ACK faad92fe1c
  fanquake:
    ACK faad92fe1c

Tree-SHA512: 9a7029187aaa5a7929a4a2199646131ff1ea72df6a855ce7022dd3bb2647dd525356dbc5e460c77007eebcdeab400a689db8cb77e8239af3b539c117a4e0d16e
2020-10-15 12:01:11 +02:00
MarcoFalke
560dea9b26 Merge #19770: RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions")
5b57dc5458 RPC: getpeerinfo: Wrap long help line for bytesrecv_per_msg (Luke Dashjr)
d681a28219 RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") (Luke Dashjr)

Pull request description:

  If we were going to continue support for "whitelisted", we should have probably made it true if any permission flag was set, rather than only if "default permissions" were used.

  This corrects the description, and deprecates it.

ACKs for top commit:
  laanwj:
    ACK 5b57dc5458

Tree-SHA512: a2e2137f8be8110357c1b2fef2c923fa8c7c4a49b0b2b3a2d78aedf12f8ed5cc7e140018a21b37e6ec7770ed4007542aeef7ad4558973901b107e8e0f81d6003
2020-10-15 11:49:34 +02:00
Wladimir J. van der Laan
e3b474c548 Merge #20140: Restore compatibility with old CSubNet serialization
886be97af5 Ignore incorrectly-serialized banlist.dat entries (Pieter Wuille)
883cea7dea Restore compatibility with old CSubNet serialization (Pieter Wuille)

Pull request description:

  #19628 changed CSubNet for IPv4 netmasks, using the first 4 bytes of `netmask` rather than the last 4 to store the actual mask. Unfortunately, CSubNet objects are serialized on disk in banlist.dat, breaking compatibility with existing banlists (and bringing them into an inconsistent state where entries reported in `listbanned` cannot be removed).

  Fix this by reverting to the old format (just for serialization). Also add a sanity check to the deserializer so that nonsensical banlist.dat entries are ignored (which would otherwise be possible if someone added IPv4 entries after #19628 but without this PR).

  Reported by Greg Maxwell.

ACKs for top commit:
  laanwj:
    Code review ACK 886be97af5
  vasild:
    ACK 886be97af

Tree-SHA512: d3fb91e8ecd933406e527187974f22770374ee2e12a233e7870363f52ecda471fb0b7bae72420e8ff6b6b1594e3037a5115984c023dbadf38f86aeaffcd681e7
2020-10-15 11:44:36 +02:00
MarcoFalke
fa729cdb2c doc: Move assumed-values doxygen comments to header 2020-10-15 11:28:13 +02:00
MarcoFalke
fa64892b82 signet: Fix uninitialized read in validation 2020-10-15 11:27:43 +02:00
Wladimir J. van der Laan
3956165903 Merge #17775: DecodeHexTx: Try case where txn has inputs first
27fc6a38f8 DecodeHexTx: Break out transaction decoding logic into own function (Gregory Sanders)
6020ce3c01 DecodeHexTx: Try case where txn has inputs first (Gregory Sanders)

Pull request description:

  Alternative/complementary to https://github.com/bitcoin/bitcoin/pull/17773 to avoid random `decoderawtransaction` failures. Most cases this is used now is on complete transactions, especially with the uptake of PSBT.

ACKs for top commit:
  ajtowns:
    ACK 27fc6a38f8
  achow101:
    ACK 27fc6a38f8

Tree-SHA512: 0a836d7c9951bf7d2764507788dbcc871d520f1ea9b77d6b22f051f4d6224ed779aba0e4f28c5c165040095ee0c70b67080c39164d82de61b19158f7ae6fddb2
2020-10-15 10:55:44 +02:00
Hennadii Stepanov
defe48a51f doc: Update wallet files in files.md 2020-10-15 11:32:26 +03:00
Wladimir J. van der Laan
3caee16946 Merge #19953: Implement BIP 340-342 validation (Schnorr/taproot/tapscript)
0e2a5e448f tests: dumping and minimizing of script assets data (Pieter Wuille)
4567ba034c tests: add generic qa-asset-based script verification unit test (Pieter Wuille)
f06e6d0345 tests: functional tests for Schnorr/Taproot/Tapscript (Pieter Wuille)
3c226639eb tests: add BIP340 Schnorr signature support to test framework (Pieter Wuille)
206fb180ec --- [TAPROOT] Tests --- (Pieter Wuille)
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342) (Pieter Wuille)
e9a021d7e6 Make Taproot spends standard + policy limits (Pieter Wuille)
865d2c37e2 --- [TAPROOT] Regtest activation and policy --- (Pieter Wuille)
72422ce396 Implement Tapscript script validation rules (BIP 342) (Johnson Lau)
330de894a9 Use ScriptExecutionData to pass through annex hash (Pieter Wuille)
8bbed4b7ac Implement Taproot validation (BIP 341) (Pieter Wuille)
0664f5fe1f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340) (Pieter Wuille)
5de246ca81 Implement Taproot signature hashing (BIP 341) (Johnson Lau)
9eb590894f Add TaggedHash function (BIP 340) (Pieter Wuille)
450d2b2371 --- [TAPROOT] BIP340/341/342 consensus rules --- (Pieter Wuille)
5d62e3a68b refactor: keep spent outputs in PrecomputedTransactionData (Pieter Wuille)
8bd2b4e784 refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script (Pieter Wuille)
107b57df9f scripted-diff: put ECDSA in name of signature functions (Pieter Wuille)
f8c099e220 --- [TAPROOT] Refactors --- (Pieter Wuille)

Pull request description:

  This is an implementation of the Schnorr/taproot consensus rules proposed by BIPs [340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), [341](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), and [342](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki).

  See the list of commits [below](https://github.com/bitcoin/bitcoin/pull/19953#issuecomment-691815830). No signing or wallet support of any kind is included, as testing is done entirely through the Python test framework.

  This is a successor to https://github.com/bitcoin/bitcoin/pull/17977 (see discussion following [this comment](https://github.com/bitcoin/bitcoin/pull/17977#issuecomment-682285983)), and will have further changes squashed/rebased. The history of this PR can be found in #19997.

ACKs for top commit:
  instagibbs:
    reACK 0e2a5e448f
  benthecarman:
    reACK 0e2a5e4
  kallewoof:
    reACK 0e2a5e448f
  jonasnick:
    ACK 0e2a5e448f almost only looked at bip340/libsecp related code
  jonatack:
    ACK 0e2a5e448f modulo the last four commits (tests) that I plan to finish reviewing tomorrow
  fjahr:
    reACK 0e2a5e448f
  achow101:
    ACK 0e2a5e448f

Tree-SHA512: 1b00314450a2938a22bccbb4e177230cf08bd365d72055f9d526891f334b364c997e260c10bc19ca78440b6767712c9feea7faad9a1045dd51a5b96f7ca8146e
2020-10-15 10:22:35 +02:00
Samuel Dobson
8ed37f6c84 Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04 Run dumpwallet for legacy wallets only in  wallet_backup.py (Andrew Chow)
6c6639ac9f Include sqlite3 in documentation (Andrew Chow)
f023b7cac0 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866 Set and check the sqlite user version (Andrew Chow)
9d3d2d263c Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8e walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225 Determine wallet file type based on file magic (Andrew Chow)
6045f77003 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4e Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7 Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2 Add SetupSQLStatements (Andrew Chow)
6636a2608a Implement SQLiteBatch::Close (Andrew Chow)
93825352a3 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372b Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe125 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df82580 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e Add libsqlite3 (Andrew Chow)

Pull request description:

  This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.

  For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

  We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.

  I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.

ACKs for top commit:
  Sjors:
    re-utACK c4a29d0a90
  promag:
    Tested ACK c4a29d0a90.
  fjahr:
    reACK c4a29d0a90
  S3RK:
    Re-review ACK c4a29d0a90
  meshcollider:
    re-utACK c4a29d0a90
  hebasto:
    re-ACK c4a29d0a90, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
  ryanofsky:
    Code review ACK c4a29d0a90. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
  jonatack:
    ACK c4a29d0a90, debug builds and test runs after rebase to latest master @ c2c4dbaebd, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.

Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
2020-10-15 20:12:29 +13:00
fanquake
f2e6d14430 Merge #20147: Update libsecp256k1 (endomorphism, test improvements)
52380bf304 Squashed 'src/secp256k1/' changes from 8ab24e8dad..c6b6b8f1bb (Pieter Wuille)

Pull request description:

  This updates the libsecp256k1 subtree to the latest master, which includes:

  * Enabling the GLV endomorphism optimization by default (and removing support for the non-GLV EC multiplication)
  * Added a proof for the correctness of the lambda split algorithm by roconnor-blockstream (other code was relying on the fact that it always outputs 128 bit results, which isn't at all obvious).
  * Improved exhaustive tests, in particular for the Schnorr signature module
  * Various other testing and CI improvements

ACKs for top commit:
  fanquake:
    ACK 9e5626d2a8 - performed a squash and checked that the changes were the same. The non-endomorphism code has now been ripped out.
  benthecarman:
    ACK 9e5626d

Tree-SHA512: 50fda5f3f934ee525f01cfc15e4f5efbc5261a97f2b77fe1b3453ee0edcf1281ad74ab4532a2fe1fe907652dd47023beff8cf3d73bf34f65ac914a694b9e7110
2020-10-15 11:27:47 +08:00
fanquake
661fe5d65c Merge #20146: net: Send post-verack handshake messages at most once
fa1f6f237d net: Send post-verack handshake messages at most once (MarcoFalke)

Pull request description:

  There is no need to send `SENDHEADERS` and `SENDCMPCT` messages as a reply to each `VERACK` that is received. For alive checks, a `PING`/`PONG` can be used.

ACKs for top commit:
  jonatack:
    Concept ACK fa1f6f237d this is the only code section that sets `fCurrentlyConnected` and `fSuccessfullyConnected` to true. Could add a test. I did not verify if this code is actually being called repeatedly post initial verack; was it?
  hebasto:
    ACK fa1f6f237d, I have reviewed the code and it looks OK, I agree it can be merged.
  naumenkogs:
    ACK fa1f6f237d
  laanwj:
    Code review ACK fa1f6f237d

Tree-SHA512: c841d5d3807254a49463bbcfac3b32881b34a9d3206899544c86322c20988e17ad2ae243cba227fd3825a914f0cb2584451edda2414aecee6d5e3f5a0636f08a
2020-10-15 07:59:19 +08:00
João Barbosa
5e737a0092 rpc, wallet: Expose database format in getwalletinfo 2020-10-14 21:47:42 +01:00
Pieter Wuille
52380bf304 Squashed 'src/secp256k1/' changes from 8ab24e8dad..c6b6b8f1bb
c6b6b8f1bb Merge #830: Rip out non-endomorphism code + dependencies
c582abade1 Consistency improvements to the comments
63c6b71616 Reorder comments/function around scalar_split_lambda
2edc514c90 WNAF of lambda_split output has max size 129
4232e5b7da Rip out non-endomorphism code
ebad8414b0 Check correctness of lambda split without -DVERIFY
fe7fc1fda8 Make lambda constant accessible
9d2f2b44d8 Add tests to exercise lambda split near bounds
9aca2f7f07 Add secp256k1_split_lambda_verify
acab934d24 Detailed comments for secp256k1_scalar_split_lambda
76ed922a5f Increase precision of g1 and g2
6173839c90 Switch to our own memcmp function
63150ab4da Merge #827: Rename testrand functions to have test in name
c5257aed0b Merge #821: travis: Explicitly set --with-valgrind
bb1f54280f Merge #818: Add static assertion that uint32_t is unsigned int or wider
a45c1fa63c Rename testrand functions to have test in name
5006895bd6 Merge #808: Exhaustive test improvements + exhaustive schnorrsig tests
4eecb4d6ef travis: VALGRIND->RUN_VALGRIND to avoid confusion with WITH_VALGRIND
66a765c775 travis: Explicitly set --with-valgrind
d7838ba6a6 Merge #813: Enable configuring Valgrind support
7ceb0b7611 Merge #819: Enable -Wundef warning
8b7dcdd955 Add exhaustive test for extrakeys and schnorrsig
08d7d89299 Make pubkey parsing test whether points are in the correct subgroup
87af00b511 Abstract out challenge computation in schnorrsig
63e1b2aa7d Disable output buffering in tests_exhaustive.c
39f67dd072 Support splitting exhaustive tests across cores
e99b26fcd5 Give exhaustive_tests count and seed cmdline inputs
49e6630bca refactor: move RNG seeding to testrand
b110c106fa Change exhaustive test groups so they have a point with X=1
cec7b18a34 Select exhaustive lambda in function of order
78f6cdfaae Make the curve B constant a secp256k1_fe
d7f39ae4b6 Delete gej_is_valid_var: unused outside tests
8bcd78cd79 Make secp256k1_scalar_b32 detect overflow in scalar_low
c498366e5b Move exhaustive tests for recovery to module
be31791543 Make group order purely compile-time in exhaustive tests
e73ff30922 Enable -Wundef warning
c0041b5cfc Add static assertion that uint32_t is unsigned int or wider
4ad408faf3 Merge #782: Check if variable=yes instead of if var is set in travis.sh
412bf874d0 configure: Allow specifying --with[out]-valgrind explicitly
34debf7a6d Modify .travis.yml to explictly pass no in env vars instead of setting to nothing
a0e99fc121 Merge #814: tests: Initialize random group elements fully
5738e8622d tests: Initialize random group elements fully
c9939ba55d Merge #812: travis: run bench_schnorrsig
a51f2af62b travis: run bench_schnorrsig
ef37761fee Change travis.sh to check if variables are equal to yes instead of not-empty. Before this, setting `VALGRIND=wat` was considered as true, and to make it evaluate as false you had to unset the variable `VALGRIND=` but not it checks if `VALGRIND=yes` and if it's not `yes` then it's evaluated to false

git-subtree-dir: src/secp256k1
git-subtree-split: c6b6b8f1bb044d7d1aa065ebb674adde98a36a8e
2020-10-14 11:41:15 -07:00
Pieter Wuille
9e5626d2a8 Update libsecp256k1 subtree to latest master 2020-10-14 11:41:15 -07:00
Wladimir J. van der Laan
c2c4dbaebd Merge #19988: Overhaul transaction request logic
fd9a0060f0 Report and verify expirations (Pieter Wuille)
86f50ed10f Delete limitedmap as it is unused now (Pieter Wuille)
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents (Pieter Wuille)
173a1d2d3f Expedite removal of tx requests that are no longer needed (Pieter Wuille)
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers (Pieter Wuille)
242d16477d Change transaction request logic to use txrequest (Pieter Wuille)
5b03121d60 Add txrequest fuzz tests (Pieter Wuille)
3c7fe0e5a0 Add txrequest unit tests (Pieter Wuille)
da3b8fde03 Add txrequest module (Pieter Wuille)

Pull request description:

  This replaces the transaction request logic with an encapsulated class that maintains all the state surrounding it. By keeping it stand alone, it can be easily tested (using included unit tests and fuzz tests).

  The major changes are:

  * Announcements from outbound (and whitelisted) peers are now always preferred over those from inbound peers. This used to be the case for the first request (by delaying the first request from inbound peers), and a bias afters. The 2s delay for requests from inbound peers still exists, but after that, if viable outbound peers remain for any given transaction, they will always be tried first.
  * No more hard cap of 100 in flight transactions per peer, as there is less need for it (memory usage is linear in the number of announcements, but independent from the number in flight, and CPU usage isn't affected by it). Furthermore, if only one peer announces a transaction, and it has over 100 in flight already, we still want to request it from them. The cap is replaced with a rule that announcements from such overloaded peers get an additional 2s delay (possibly combined with the existing 2s delays for inbound connections, and for txid peers when wtxid peers are available).
  * The limit of 100000 tracked announcements is reduced to 5000; this was excessive. This can be bypassed using the PF_RELAY permission (to accommodate locally dumping a batch of many transactions).

  This replaces #19184, rebased on #18044 and with many small changes.

ACKs for top commit:
  ariard:
    Code Review ACK fd9a006. I've reviewed the new TxRequestTracker, its integration in net_processing, unit/functional/fuzzing test coverage. I looked more for soundness of new specification rather than functional consistency with old transaction request logic.
  MarcoFalke:
    Approach ACK fd9a0060f0 🏹
  naumenkogs:
    Code Review ACK fd9a006. I've reviewed everything, mostly to see how this stuff works at the lower level (less documentation-wise, more implementation-wise), and to try breaking it with unexpected sequences of events.
  jnewbery:
    utACK fd9a0060f0
  jonatack:
    WIP light ACK fd9a0060f0 have read the code, verified that each commit is hygienic, e.g. debug build clean and tests green, and have been running a node on and off with this branch and grepping the net debug log. Am still unpacking the discussion hidden by GitHub by fetching it via the API and connecting the dots, storing notes and suggestions in a local branch; at this point none are blockers.
  ryanofsky:
    Light code review ACK fd9a0060f0, looking at txrequest implementation, unit test implementation, and net_processing integration, just trying to understand how it works and looking for anything potentially confusing in the implementation. Didn't look at functional tests or catch up on review discussion. Just a sanity check review focused on:

Tree-SHA512: ea7b52710371498b59d9c9cfb5230dd544fe9c6cb699e69178dea641646104f38a0b5ec7f5f0dbf1eb579b7ec25a31ea420593eff3b7556433daf92d4b0f0dd7
2020-10-14 18:36:59 +02:00
Russell Yanofsky
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets 2020-10-14 11:28:18 -04:00
Andrew Chow
310b0fde04 Run dumpwallet for legacy wallets only in wallet_backup.py
Descriptor wallets don't support dumpwallet, so make the tests that do
dumpwallet legacy wallet only.
2020-10-14 11:28:18 -04:00
Andrew Chow
6c6639ac9f Include sqlite3 in documentation 2020-10-14 11:28:18 -04:00
Andrew Chow
f023b7cac0 wallet: Enforce sqlite serialized threading mode 2020-10-14 11:28:18 -04:00
Andrew Chow
6173269866 Set and check the sqlite user version 2020-10-14 11:28:18 -04:00
Andrew Chow
9d3d2d263c Use network magic as sqlite wallet application ID 2020-10-14 11:28:18 -04:00
Andrew Chow
9af5de3798 Use SQLite for descriptor wallets
MakeWalletDatabase no longer has a default DatabaseFormat. Instead
callers, like CWallet::Create, need to specify the database type to
create if the file does not exist. If it exists and NONE is given, then
CreateWalletDatabase will try to autodetect the type.
2020-10-14 11:28:18 -04:00
Andrew Chow
9b78f3ce8e walletutil: Wallets can also be sqlite 2020-10-14 11:28:18 -04:00
Andrew Chow
ac38a87225 Determine wallet file type based on file magic 2020-10-14 11:28:18 -04:00
Andrew Chow
6045f77003 Implement SQLiteDatabase::MakeBatch 2020-10-14 11:28:18 -04:00
Andrew Chow
727e6b2a4e Implement SQLiteDatabase::Verify 2020-10-14 11:28:18 -04:00
Andrew Chow
b4df8fdb19 Implement SQLiteDatabase::Rewrite
Rewrite uses the VACUUM command which does exactly what we want. A
specific advertised use case is to compact a database and ensure that
any deleted data is actually deleted.
2020-10-14 11:28:18 -04:00
Andrew Chow
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort 2020-10-14 11:28:18 -04:00
Andrew Chow
ac5c1617e7 Implement SQLiteDatabase::Backup 2020-10-14 11:28:18 -04:00
Andrew Chow
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor 2020-10-14 11:28:18 -04:00
Andrew Chow
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey 2020-10-14 11:28:18 -04:00
Andrew Chow
7aa45620e2 Add SetupSQLStatements 2020-10-14 11:28:18 -04:00
Andrew Chow
6636a2608a Implement SQLiteBatch::Close 2020-10-14 11:28:18 -04:00
Andrew Chow
93825352a3 Implement SQLiteDatabase::Close 2020-10-14 11:28:18 -04:00
Andrew Chow
a0de83372b Implement SQLiteDatabase::Open 2020-10-14 11:28:18 -04:00
Andrew Chow
3bfa0fe125 Initialize and Shutdown sqlite3 globals
sqlite3 recommends that sqlite3_initialize be called when the
application starts, and sqlite3_shutdown when it stops. Since we don't
always use sqlite3, we initialize it when a SQLiteDatabse is constructed
(calling sqlite3_initialize after initialized is a no-op). We call
sqlite3_shutdown when we see that there are no databases opened. The
number of open databases is tracked by an atomic g_dbs_open.
2020-10-14 11:28:18 -04:00
Andrew Chow
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch 2020-10-14 11:28:17 -04:00
Andrew Chow
ca8b7e04ab Implement SQLiteDatabaseVersion 2020-10-14 11:27:40 -04:00
Andrew Chow
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes 2020-10-14 11:27:37 -04:00
Andrew Chow
e87df82580 Add sqlite to travis and depends 2020-10-14 11:18:13 -04:00
Andrew Chow
54729f3f4e Add libsqlite3 2020-10-14 11:18:12 -04:00
Jon Atack
6272604bef refactor: enable -netinfo to add future networks (i2p, cjdns)
After this commit, a new network may be added by changing 4 lines:
- increment the value of `m_networks_size`
- add the network name to `m_networks`
- add the network name to this line: `result += "        ipv4    ipv6   onion   total  block-relay\n";`
- add "counts.at(i).at(<m_networks pos>)" to this line: `result += strprintf("%-5s  %5i   %5i   %5i   %5i   %5i\n...`
2020-10-14 16:29:37 +02:00
Luke Dashjr
5b57dc5458 RPC: getpeerinfo: Wrap long help line for bytesrecv_per_msg 2020-10-14 14:16:43 +00:00
Luke Dashjr
d681a28219 RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") 2020-10-14 14:16:42 +00:00
Neha Narula
da0988daf1 scripted-diff: rename vRecvGetData
-BEGIN VERIFY SCRIPT-
sed -i 's/vRecvGetData/m_getdata_requests/g' src/net_processing.cpp
-END VERIFY SCRIPT-
2020-10-14 10:08:44 -04:00
Neha Narula
ba951812ec Guard vRecvGetData (now in net processing) with its own mutex
This requires slightly reorganizing the logic in GETBLOCKTXN to
maintain locking order.
2020-10-14 10:08:44 -04:00
Neha Narula
2d9f2fca43 Move vRecvGetData to net processing 2020-10-14 10:08:44 -04:00
Neha Narula
673247b58c Lock before checking if orphan_work_set is empty; indicate it is guarded 2020-10-14 10:08:44 -04:00
Neha Narula
8803aee668 Move m_orphan_work_set to net_processing 2020-10-14 10:08:37 -04:00
Wladimir J. van der Laan
99a1d572ea Merge #18750: build: optionally skip external warnings
ba8950ee01 build: optionally skip external warnings (Vasil Dimov)

Pull request description:

  Add an option to `./configure` to suppress compilation warnings from
  external headers. The option is off by default (no change in behavior,
  show warnings from external headers).

  This option is useful if e.g. Boost or Qt is installed outside of
  `/usr/include` (warnings from headers in `/usr/include` are already
  suppressed by default) and those warnings stand in the way of compiling
  Bitcoin Core with `-Werror[=...]` or they just clutter the build output
  too much and make our own warnings hard to spot.

  `-isystem /usr/include` bricks GCC's `#include_next`, so we use
  `-idirafter` instead. This way we don't have to treat `/usr/include`
  specially.

ACKs for top commit:
  practicalswift:
    ACK ba8950ee01: diff looks correct!
  hebasto:
    ACK ba8950ee01, tested on Linux Mint 20 (x86_64).
  luke-jr:
    utACK ba8950ee01

Tree-SHA512: 9b54fae8590be6c79f2688a5aca09e0a9067f481dabecdd49bb278c08a62ac2b0cc704c894fbd53240e77ac84da0c7a237845df0a696cfbdb0359e1c8e2e10c9
2020-10-14 14:57:15 +02:00
Jon Atack
82fd40216c refactor: promote some -netinfo localvars to class members 2020-10-14 14:56:07 +02:00
Jon Atack
5133fab37e cli: simplify -netinfo using getpeerinfo network field 2020-10-14 14:56:05 +02:00
Jon Atack
4938a109ad rpc, test: expose CNodeStats network in RPC getpeerinfo 2020-10-14 14:56:03 +02:00
Jon Atack
6df7882029 net: add peer network to CNodeStats 2020-10-14 14:55:55 +02:00
Wladimir J. van der Laan
9efa55c715 Merge #20130: Wallet: remove db mode string
135afa749c wallet: remove db mode string (Ivan Metlushko)

Pull request description:

  This is a [follow-up](https://github.com/bitcoin/bitcoin/pull/19077#discussion_r500261927) for #19077

  This PR simplifies DB interface by removing mode string from `WalletDatabase` and `WalletBatch`.

  The mode string was used to determine two flags for the instantiation of db connection:
  1) read-only flag. Never used on connection level. And on batch level Is only used within `BerkeleyDatabase::Rewrite` where it's replaced with bool flag.
  2) create flag. Is not required as we always check `require_existing` & `require_create` flags in `MakeDatabase()` before creating actual database instance. So we can safely default to always creating database if it doesn't exist yet.

ACKs for top commit:
  achow101:
    ACK 135afa749c
  laanwj:
    Code review ACK 135afa749c

Tree-SHA512: f49c07c7387c02e517a58199620a678a918f8dfc20d1347d29fd6adea0bc89698c26cb8eef42b0977961c11c207c4bbe109bc31059f47c126cc600b01fd987eb
2020-10-14 14:50:22 +02:00
MarcoFalke
3750f664b3 Merge #20129: tests: don't export in6addr_loopback
8e4d62280e tests: don't export in6addr_loopback (Vasil Dimov)

Pull request description:

  Don't export `in6addr_loopback` because that upsets
  `contrib/devtools/symbol-check.py`

  Fixes https://github.com/bitcoin/bitcoin/issues/20127

ACKs for top commit:
  sipa:
    utACK 8e4d62280e
  hebasto:
    ACK 8e4d62280e

Tree-SHA512: 216ffb53df55d2888317a81d18745308aaf93a3f3b45aa778166f7c91edb9741c28424d6333b35cefb5ece42b74e20ea21c761d93d8432798e7ec12097c2758f
2020-10-14 10:33:55 +02:00
MarcoFalke
fa1f6f237d net: Send post-verack handshake messages at most once 2020-10-14 10:09:50 +02:00
fanquake
e21b824386 Merge #20109: Release notes and followups from 19339
88197b0769 [doc] release notes for max fee checking (gzhao408)
c201d73df3 style and nits for fee-checking in BroadcastTransaction (gzhao408)

Pull request description:

  Pretty trivial... addresses some tiny comments from #19339. Also fixes a docs typo from #19940 and adds a release note about the error message change for testmempoolaccept.

ACKs for top commit:
  achow101:
    ACK 88197b0769
  MarcoFalke:
    cr re-ACK 88197b0769

Tree-SHA512: fff16d731426b9b4db5222df02633983402f4c7241551eec98bb1554145dbdc132f40ed8ca4abd5edcebe1f4d1e879fb6d11bd91730604f6552c10cdf65706a1
2020-10-14 12:19:42 +08:00
Karl-Johan Alm
355d0c4f6b contrib: add getcoins.py script to get coins from (signet) faucet 2020-10-14 08:27:32 +09:00
Neha Narula
9c47cb29f9 [Rename only] Rename orphan_work_set to m_orphan_work_set.
This helps distinguish the member from any local variables.
2020-10-13 17:38:38 -04:00
MarcoFalke
fa299ac273 test: Speed up wallet_resendwallettransactions test with mockscheduler RPC 2020-10-13 15:38:18 +02:00
MarcoFalke
3333077823 rpc: Adjust witness-tx deserialize error message 2020-10-13 14:57:52 +02:00
MarcoFalke
ec0453cd57 Merge bitcoin-core/gui#102: Fix SplashScreen crash when run with -disablewallet
c056064a4a gui: Fix SplashScreen crash when run with -disablewallet (Hennadii Stepanov)

Pull request description:

  This PR fixes the bug introduced in https://github.com/bitcoin/bitcoin/pull/19099:

  ```
  $ src/qt/bitcoin-qt -disablewallet
  bitcoin-qt: interfaces/node.cpp:236: auto interfaces::(anonymous namespace)::NodeImpl::walletClient()::(anonymous class)::operator()() const: Assertion `"m_context->wallet_client" && check' failed.
  Aborted (core dumped)
  ```

ACKs for top commit:
  Sjors:
    tACK c056064
  promag:
    ACK c056064a4a.

Tree-SHA512: 263d9efd5899cc6e447dfc5142bf911ca627149fac0a1c5e5b58dd196aa5e0d12fe13e3f750fb5f3c4338222f7959935d2f77391263f967dbca2e0e79a416a29
2020-10-13 14:42:12 +02:00
MarcoFalke
cd6e193d4c Merge #20126: test: p2p_leak_tx.py improvements (use MiniWallet, add p2p_lock acquires)
5b77f8098d test: add p2p_lock acquires in p2p_leak_tx.py (Sebastian Falbesoner)
cc8c6823b4 test: use MiniWallet for p2p_leak_tx.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (p2p_leak_tx.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. It also adds missing p2p_lock acquires that need to be held while modifying internal p2p Interface state (in this case the `last_message` dictionary) to avoid data races.

ACKs for top commit:
  laanwj:
    Code review ACK 5b77f8098d

Tree-SHA512: 6661bc6e3491a9af4bf040f379e5955c525136397e99d3eadde92e247580d0d87efff750e6d3b1f6d9a4e578144a433a982f574ef056b44dd6bca33873a1bae6
2020-10-13 14:27:30 +02:00
fanquake
ad90dd9f31 Merge #20141: Avoid the use of abs64 in timedata
d1292f25f2 Avoid the use of abs64 in timedata (Pieter Wuille)

Pull request description:

  Fixes #20135.

ACKs for top commit:
  kallewoof:
    ACK d1292f25f2
  jonatack:
    ACK d1292f25f2 code/logic review, verified there are no remaining callers of `abs64()`, verified no warnings in a debug build
  practicalswift:
    ACK d1292f25f2
  MarcoFalke:
    ACK d1292f25f2 🎹

Tree-SHA512: d17e95c668eb5e02ea546433b3d1b5a0ccbfb2c9cec62fa67dad1844d7e278a2576fbc0b75bddbf4db9af7331e978148c7bef7fce7e6a07e0eb917ef1392f302
2020-10-13 20:00:39 +08:00
Ivan Metlushko
135afa749c wallet: remove db mode string
We never need to open database in read-only mode as it's controlled
separately for every batch.

Also we can safely create database if it doesn't exist already
because require_existing option is verified in MakeDatabase
before creating a new WalletDatabase instance.
2020-10-13 18:42:59 +07:00
Pieter Wuille
d1292f25f2 Avoid the use of abs64 in timedata 2020-10-12 19:50:16 -07:00
Pieter Wuille
0e2a5e448f tests: dumping and minimizing of script assets data
This adds a --dumptests flag to the feature_taproot.py test, to dump all its
generated test cases to files, in a format compatible with the
script_assets_test unit test. A fuzzer for said format is added as well, whose
primary purpose is coverage-based minimization of those dumps.
2020-10-12 17:18:47 -07:00
Pieter Wuille
4567ba034c tests: add generic qa-asset-based script verification unit test
This adds a unit test that does generic script verification tests,
with positive/negative witnesses/scriptsigs, under various flags.
The test data is large (several MB) so it's stored in the qa-assets
repo.
2020-10-12 17:18:47 -07:00
Pieter Wuille
f06e6d0345 tests: functional tests for Schnorr/Taproot/Tapscript
A large functional test is added that automatically generates random transactions which
exercise various aspects of the new rules, and verifies they are accepted into the mempool
(when appropriate), and correctly accepted/rejected in (Python-constructed) blocks.

Includes sighashing code and many tests by Johnson Lau.
Includes a test by Matthew Zipkin.
Includes several tests and improvements by Greg Sanders.
2020-10-12 17:18:47 -07:00
Pieter Wuille
3c226639eb tests: add BIP340 Schnorr signature support to test framework
Add a pure Python implementation of BIP340 signing and verification, tested against
the BIP's test vectors.
2020-10-12 17:18:47 -07:00
Pieter Wuille
206fb180ec --- [TAPROOT] Tests --- 2020-10-12 17:18:47 -07:00
Pieter Wuille
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342)
Define a versionbits-based activation for the new consensus rules on regtest.
No activation or activation mechanism is defined for testnet or mainnet.
2020-10-12 17:18:47 -07:00
Pieter Wuille
e9a021d7e6 Make Taproot spends standard + policy limits
This adds a `TxoutType::WITNESS_V1_TAPROOT` for P2TR outputs, and permits spending
them in standardness rules. No corresponding `CTxDestination` is added for it,
as that isn't needed until we want wallet integration. The taproot validation flags
are also enabled for mempool transactions, and standardness rules are added
(stack item size limit, no annexes).
2020-10-12 17:18:47 -07:00
Pieter Wuille
865d2c37e2 --- [TAPROOT] Regtest activation and policy --- 2020-10-12 17:18:47 -07:00
Johnson Lau
72422ce396 Implement Tapscript script validation rules (BIP 342)
This adds a new `SigVersion::TAPSCRIPT`, makes the necessary interpreter
changes to make it implement BIP342, and uses them for leaf version 0xc0
in Taproot script path spends.
2020-10-12 17:18:24 -07:00
Pieter Wuille
330de894a9 Use ScriptExecutionData to pass through annex hash
Instead of recomputing the annex hash every time a signature is verified, compute it
once and cache it in a new ScriptExecutionData structure.
2020-10-12 17:18:15 -07:00
Pieter Wuille
8bbed4b7ac Implement Taproot validation (BIP 341)
This includes key path spending and script path spending, but not the
Tapscript execution implementation (leaf 0xc0 remains unemcumbered in
this commit).

Includes constants for various aspects of the consensus rules suggested
by Jeremy Rubin.
2020-10-12 17:17:56 -07:00
Pieter Wuille
0664f5fe1f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340)
This enables the schnorrsig module in libsecp256k1, adds the relevant types
and functions to src/pubkey, as well as in higher-level `SignatureChecker`
classes. The (verification side of the) BIP340 test vectors is also added.
2020-10-12 17:15:40 -07:00
Johnson Lau
5de246ca81 Implement Taproot signature hashing (BIP 341)
This implements the new sighashing scheme from BIP341, with all relevant
whole-transaction values precomputed once and cached.

Includes changes to PrecomputedTransactionData by Pieter Wuille.
2020-10-12 17:06:38 -07:00
Pieter Wuille
886be97af5 Ignore incorrectly-serialized banlist.dat entries 2020-10-12 15:35:08 -07:00
Pieter Wuille
883cea7dea Restore compatibility with old CSubNet serialization 2020-10-12 15:15:28 -07:00
Pieter Wuille
fd9a0060f0 Report and verify expirations 2020-10-12 12:14:53 -07:00
Pieter Wuille
86f50ed10f Delete limitedmap as it is unused now 2020-10-12 12:14:53 -07:00
Pieter Wuille
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents 2020-10-12 12:14:53 -07:00
Pieter Wuille
173a1d2d3f Expedite removal of tx requests that are no longer needed
Whenever a transaction is added to the mempool or orphan pool, both
its txid and wtxid are considered AlreadyHave, and thus will eventually
be removed from m_txrequest.

The same is true for hashes added to the reject filter, but note that sometimes
only the wtxid is added (in which case only the wtxid can be removed from
m_txrequest).
2020-10-12 12:14:53 -07:00
Pieter Wuille
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers
Maintaining up to 100000 INVs per peer is excessive, as that is far more
than fits in a typical mempool.

Also disable the "overload" penalty for PF_RELAY peers.
2020-10-12 12:14:53 -07:00
Pieter Wuille
242d16477d Change transaction request logic to use txrequest
This removes most transaction request logic from net_processing, and
replaces it with calls to a global TxRequestTracker object.

The major changes are:

* Announcements from outbound (and whitelisted) peers are now always
  preferred over those from inbound peers. This used to be the case for the
  first request (by delaying the first request from inbound peers), and
  a bias afters. The 2s delay for requests from inbound peers still exists,
  but after that, if viable outbound peers remain for any given transaction,
  they will always be tried first.

* No more hard cap of 100 in flight transactions per peer, as there is less
  need for it (memory usage is linear in the number of announcements, but
  independent from the number in flight, and CPU usage isn't affected by it).
  Furthermore, if only one peer announces a transaction, and it has over 100
  in flight and requestable already, we still want to request it from them.
  The cap is replaced with an additional 2s delay (possibly combined with the
  existing 2s delays for inbound connections, and for txid peers when wtxid
  peers are available).

Includes functional tests written by Marco Falke and Antoine Riard.
2020-10-12 12:14:11 -07:00
Pieter Wuille
5b03121d60 Add txrequest fuzz tests
This adds a fuzz test that reimplements a naive reimplementation of
TxRequestTracker (with up to 16 fixed peers and 16 fixed txhashes),
and compares the real implementation against it.
2020-10-12 12:08:47 -07:00
Pieter Wuille
3c7fe0e5a0 Add txrequest unit tests
Add unit tests for TxRequestTracker. Several scenarios are tested,
randomly interleaved with eachother.

Includes a test by Antoine Riard (ariard).
2020-10-12 12:08:43 -07:00
Pieter Wuille
da3b8fde03 Add txrequest module
This adds a new module (unused for now) which defines TxRequestTracker, a data
structure that maintains all information about transaction requests, and coordinates
requests.
2020-10-12 11:01:16 -07:00
Wladimir J. van der Laan
f79a4a8952 Merge #19998: net: Add CNode::ConnectedThroughNetwork member function
3984b78cd7 test: Add tests for CNode::ConnectedThroughNetwork (Hennadii Stepanov)
49fba9c1aa net: Add CNode::ConnectedThroughNetwork member function (Hennadii Stepanov)
d4dde24034 net: Add CNode::m_inbound_onion data member (Hennadii Stepanov)

Pull request description:

  This PR:
  - adds `CNode::ConnectedThroughNetwork` member function
  - is based on #19991, and only last two commits belong to it
  - is required for https://github.com/bitcoin-core/gui/pull/86 and #20002

ACKs for top commit:
  jonatack:
    re-ACK 3984b78cd7 per `git diff 3989fcf 3984b78c`
  laanwj:
    Code review ACK 3984b78cd7

Tree-SHA512: 23a9c8bca8dca75113b5505fe443b294f2d42d03c98c7e34919da12d8396beb8d0ada3a58ae16e3da04b7044395f72cf9c216625afc078256cd6c897ac42bf3d
2020-10-12 18:40:45 +02:00
Vasil Dimov
ba8950ee01 build: optionally skip external warnings
Add an option to `./configure` to suppress compilation warnings from
external headers. The option is off by default (no change in behavior,
show warnings from external headers).

This option is useful if e.g. Boost or Qt is installed outside of
`/usr/include` (warnings from headers in `/usr/include` are already
suppressed by default) and those warnings stand in the way of compiling
Bitcoin Core with `-Werror[=...]` or they just clutter the build output
too much and make our own warnings hard to spot.
2020-10-12 18:18:24 +02:00
MarcoFalke
faad92fe1c test: Remove unused nVersion=1 in p2p tests
After commit ddefb5c0b7 nVersion is no
longer used in p2p logic when sending messages. Only when receiving
messages, but in this test no messages are received.
2020-10-12 13:11:26 +02:00
Pieter Wuille
9eb590894f Add TaggedHash function (BIP 340)
This adds the TaggedHash function as defined by BIP340 to the hash module, which
is used in BIP340 and BIP341 to produce domain-separated hashes.
2020-10-12 02:06:32 -07:00
Pieter Wuille
450d2b2371 --- [TAPROOT] BIP340/341/342 consensus rules --- 2020-10-12 02:06:32 -07:00
Pieter Wuille
5d62e3a68b refactor: keep spent outputs in PrecomputedTransactionData
A BIP-341 signature message may commit to the scriptPubKeys and amounts
of all spent outputs (including other ones than the input being signed
for spends), so keep them available to signature hashing code.
2020-10-12 02:06:32 -07:00
Vasil Dimov
8e4d62280e tests: don't export in6addr_loopback
Don't export `in6addr_loopback` because that upsets
`contrib/devtools/symbol-check.py`

Fixes https://github.com/bitcoin/bitcoin/issues/20127
2020-10-12 10:25:08 +02:00
fanquake
af22322dab Merge #20119: BIP155 follow-ups
56f9dba015 Only relay IPv4, IPv6, Tor addresses (Pieter Wuille)
79f3d9b932 Mention BIP155 in doc/bips.md (Pieter Wuille)

Pull request description:

  This:
  * Documents BIP155 support in doc/bips.md
  * Restricts addrv2 relay to IPv4, IPv6, and Tor addresses. Relaying addresses in ranges that no network software has support for seems like a gratuitous spam vector.

ACKs for top commit:
  jonatack:
    ACK 56f9dba015
  naumenkogs:
    ACK 56f9dba
  hebasto:
    ACK 56f9dba015, verified both links.

Tree-SHA512: f0a2072b3d84a05cdbc7b961c18d7322a2e7260517f5306599ff52d8c728f9167de0a59a6d66cb95d84d69f3028680ce8bd05dab0db8c4f97938a287e5ce9631
2020-10-12 15:47:07 +08:00
Sebastian Falbesoner
5b77f8098d test: add p2p_lock acquires in p2p_leak_tx.py 2020-10-12 00:22:16 +02:00
Sebastian Falbesoner
cc8c6823b4 test: use MiniWallet for p2p_leak_tx.py
This test can now be run even with the Bitcoin Core wallet disabled.
2020-10-11 23:57:56 +02:00
Pieter Wuille
56f9dba015 Only relay IPv4, IPv6, Tor addresses 2020-10-11 11:29:11 -07:00
Pieter Wuille
79f3d9b932 Mention BIP155 in doc/bips.md 2020-10-11 11:29:07 -07:00
fanquake
0b2abaa666 Merge #19954: Complete the BIP155 implementation and upgrade to TORv3
dcf0cb4776 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d9 net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fd Support bypassing range check in ReadCompactSize (Pieter Wuille)

Pull request description:

  This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:

  `net: CAddress & CAddrMan: (un)serialize as ADDRv2`
  `net: advertise support for ADDRv2 via new message`

  plus one more commit:

  `tor: make a TORv3 hidden service instead of TORv2`

ACKs for top commit:
  jonatack:
    re-ACK dcf0cb4776 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
  sipa:
    ACK dcf0cb4776
  hebasto:
    re-ACK dcf0cb4776, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
  laanwj:
    Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb4776 merged on master (12a1c3ad1a).
  ariard:
    Code Review ACK dcf0cb4

Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
2020-10-11 08:51:57 +08:00
Amiti Uttarwar
41dca087b7 [trivial] Extract connection type doc into file where it is used.
This slightly reduces the size of the binary.
2020-10-09 16:09:51 -07:00
Amiti Uttarwar
3069b56a45 [doc] Improve help for getpeerinfo connection_type field. 2020-10-09 16:08:02 -07:00
Hennadii Stepanov
c056064a4a gui: Fix SplashScreen crash when run with -disablewallet 2020-10-09 21:14:47 +03:00
gzhao408
88197b0769 [doc] release notes for max fee checking 2020-10-09 08:58:47 -07:00
Elliott Jin
66d012ad7f test: RPC: getblock fee calculations 2020-10-09 08:50:49 -07:00
Elliott Jin
bf7d6e31b1 RPC: getblock: tx fee calculation for verbosity 2 via Undo data
Co-authored-by: Felix Weis <mail@felixweis.com>
2020-10-09 08:50:44 -07:00
Vasil Dimov
dcf0cb4776 tor: make a TORv3 hidden service instead of TORv2
TORv2 is deprecated [1], thus whenever we create the hidden service
ourselves create a TORv3 one instead.

[1] https://blog.torproject.org/v2-deprecation-timeline
2020-10-09 16:42:50 +02:00
Vasil Dimov
353a3fdaad net: advertise support for ADDRv2 via new message
Introduce a new message `sendaddrv2` to signal support for ADDRv2.
Send the new message immediately after sending the `VERACK` message.

Add support for receiving and parsing ADDRv2 messages.

Send ADDRv2 messages (instead of ADDR) to a peer if he has
advertised support for it.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:50 +02:00
Vasil Dimov
201a4596d9 net: CAddress & CAddrMan: (un)serialize as ADDRv2
Change the serialization of `CAddrMan` to serialize its addresses
in ADDRv2/BIP155 format by default. Introduce a new `CAddrMan` format
version (3).

Add support for ADDRv2 format in `CAddress` (un)serialization.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:49 +02:00
Andrew Chow
de6b389d5d tests: Test getaddressinfo parent_desc 2020-10-09 09:04:18 -04:00
Andrew Chow
e4ac869a0a rpc: Add parent descriptor to getaddressinfo output
Adds parent_desc field to getaddressinfo output which contains the
descriptor that was used to derive the address if it is available.
2020-10-09 09:04:18 -04:00
Andrew Chow
bbe4a36152 wallet: Add GetDescriptorString to DescriptorScriptPubKeyMan
GetDescriptorString returns a normalized descriptor for a
DescriptorScriptPubKeyMan.
2020-10-09 09:04:15 -04:00
Andrew Chow
9be1437c49 descriptors: Add ToNormalizedString and tests 2020-10-09 09:04:13 -04:00
Hennadii Stepanov
a490d074b3 doc: Add anchors.dat to files.md 2020-10-09 14:30:09 +03:00
Hennadii Stepanov
0a85e5a7bc p2p: Try to connect to anchors once 2020-10-09 14:29:05 +03:00
Hennadii Stepanov
5543c7ab28 p2p: Fix off-by-one error in fetching address loop
This is a move-only commit.
2020-10-09 14:29:05 +03:00
Hennadii Stepanov
4170b46544 p2p: Integrate DumpAnchors() and ReadAnchors() into CConnman 2020-10-09 14:29:05 +03:00
Hennadii Stepanov
bad16aff49 p2p: Add CConnman::GetCurrentBlockRelayOnlyConns() 2020-10-09 14:29:04 +03:00
Hennadii Stepanov
c29272a157 p2p: Add ReadAnchors() 2020-10-09 14:29:04 +03:00
Hennadii Stepanov
567008d2a0 p2p: Add DumpAnchors() 2020-10-09 14:28:58 +03:00
fanquake
12a1c3ad1a Merge #20107: doc: Collect release-notes snippets
faa0847dec doc: Add release notes for #20101 (MarcoFalke)
99992e7832 doc: Collect release-notes snippets (MarcoFalke)

Pull request description:

  Also add a note for #20101

ACKs for top commit:
  fanquake:
    ACK faa0847dec - no need to bike-shed here as these will all get massaged to death in the wiki anyways.

Tree-SHA512: 63d3597e2bbd422ec182e76112110477d22e3afedb479114ddec958405bcdd63492df9477267aac65605612af49c0aff6246b1bc3d41dd606d6d61c30117c109
2020-10-09 18:08:14 +08:00
fanquake
6854dbdc88 Merge #20076: doc: Update and improve files.md
2dc79c4264 doc: Update and improve files.md (Hennadii Stepanov)

Pull request description:

  This PR adds to the `files.md`:
  - the `signet` subdirectory
  - the `ip_asn.map` file
  - some small improvements

ACKs for top commit:
  practicalswift:
    ACK 2dc79c4264
  MarcoFalke:
    ACK 2dc79c4264

Tree-SHA512: f645486a26293e91eda826dee46e5798af9a81be410d48d07c2714f416da19b85e7e75b1a638b0e03a3e6dc486a8bb65c4be811eb2ff51b66f5817aecf89416d
2020-10-09 18:02:38 +08:00
Pieter Wuille
1d3ec2a1fd Support bypassing range check in ReadCompactSize
This is needed when we want to encode an arbitrary number as CompactSize
like node service flags, which is a bitmask and could be bigger than the
usual size of an object.
2020-10-09 10:32:19 +02:00
MarcoFalke
380705ef4f Merge #20106: cirrus: Use kvm to avoid spurious CI failures in the default virtualization cluster
faf2999e25 cirrus: Use kvm to avoid spurious CI failures in the default virtualization cluster (MarcoFalke)

Pull request description:

  Try to fix #20093

ACKs for top commit:
  practicalswift:
    ACK faf2999e25
  hebasto:
    ACK faf2999e25, the related doc:

Tree-SHA512: 156aa2ce5a5dde11570f7f90f9d51be540a5469f090033ab6f337b2c46347741469e109f9566f4b7c424339483d61192b91102021f5db38823ce4cf4428e5671
2020-10-09 08:44:35 +02:00
gzhao408
c201d73df3 style and nits for fee-checking in BroadcastTransaction 2020-10-08 14:11:16 -07:00
Wladimir J. van der Laan
d9de00b3e0 Merge #20105: [net] Remove CombinerAll
1afcd41a90 [net] Remove CombinerAll (John Newbery)

Pull request description:

  This was introduced in 9519a9a4 for use with boost signals. Boost signals
  have not been used in net since 8ad663c1, so this code is unused.

ACKs for top commit:
  MarcoFalke:
    review ACK 1afcd41a90
  laanwj:
    code review ACK 1afcd41a90

Tree-SHA512: a4313142afb88bf12f15abc4e717b3b0d0b40d2d5db2638494af3181e1cd680d7b036087050fc0e0dfe606228849a2e20ae85135908a9ebe8ff2130f163920e1
2020-10-08 20:42:19 +02:00
MarcoFalke
faa0847dec doc: Add release notes for #20101 2020-10-08 15:11:25 +02:00
MarcoFalke
392c6f4fb2 Merge #20101: rpc: change no wallet loaded message to be clearer
907f142fc7 rpc: change no wallet loaded message to be clearer (Andrew Chow)

Pull request description:

  Changes the no wallet is loaded rpc error message to be clearer that no wallet is loaded and how the user can load or create a wallet. Also changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as that makes more sense.

ACKs for top commit:
  MarcoFalke:
    review ACK 907f142fc7
  kristapsk:
    ACK 907f142fc7. In addition to standard tests, just in case tested that this doesn't break anything with JoinMarket.
  meshcollider:
    utACK 907f142fc7

Tree-SHA512: 4b413e6ab5430ec75a79de9db6583f2f3f38ccdf71aa373d8386a56e64f07f92200c8107c8c82c92c7c431d739615977c208b771a24c5960fa8676789b5497a2
2020-10-08 15:07:23 +02:00
MarcoFalke
faf2999e25 cirrus: Use kvm to avoid spurious CI failures in the default virtualization cluster 2020-10-08 14:47:21 +02:00
MarcoFalke
fa6af31227 test: Document why syncwithvalidationinterfacequeue is needed in tests 2020-10-08 14:23:51 +02:00
MarcoFalke
fa135a13b8 Revert "test: Add missing sync_all to wallet_balance test"
This reverts commit fa815255c7.

The underlying bug has been fixed in commit f77b1de16f.
2020-10-08 14:23:37 +02:00
John Newbery
1afcd41a90 [net] Remove CombinerAll
This was introduced in 9519a9a4 for use with boost signals. Boost signals
have not been used in net since 8ad663c1, so this code is unused
2020-10-08 12:28:38 +01:00
MarcoFalke
b337bd7bc0 Merge #20065: fuzz: Configure check for main function
fae7a1c188 fuzz: Configure check for main function (MarcoFalke)

Pull request description:

  Instead of the PP jungle, use a proper configure check

  Fixes https://github.com/google/honggfuzz/issues/336#issuecomment-702972138

ACKs for top commit:
  practicalswift:
    ACK fae7a1c188

Tree-SHA512: 2e55457d01f9ac598bb1e119d8b49dca55a28f88ec164cee6b5f071c29e9791f5a46cc8ee2b801b3a3faf906348da964ce32e7254da981c1104b9210a3508100
2020-10-08 13:17:45 +02:00
MarcoFalke
9dd4de2832 Merge #20027: Use mockable time everywhere in net_processing
b6834e312a Avoid 'timing mishap' warnings when mocking (Pieter Wuille)
ec3916f40a Use mockable time everywhere in net_processing (Pieter Wuille)

Pull request description:

  The fact that net_processing uses a mix of mockable tand non-mockable time functions made it hard to write functional tests for #19988.

  I'm opening this as a separate PR as I believe it's independently useful. In some ways this doesn't go quite as far as it could, as there are now several data structures that could be converted to `std::chrono` types as well now. I haven't done that here, but I'm happy to reconsider that.

ACKs for top commit:
  MarcoFalke:
    ACK b6834e312a 🌶
  jnewbery:
    utACK b6834e312a
  naumenkogs:
    utACK b6834e3

Tree-SHA512: 6528a167c57926ca12894e0c476826411baf5de2f7b01c2125b97e5f710e620f427bbb13f72bdfc3de59072e56a9c1447bce832f41c725e00e81fea019518f0e
2020-10-08 11:38:47 +02:00
Pieter Wuille
b6834e312a Avoid 'timing mishap' warnings when mocking 2020-10-08 01:04:29 -07:00
Pieter Wuille
ec3916f40a Use mockable time everywhere in net_processing 2020-10-08 01:04:29 -07:00
Andrew Chow
907f142fc7 rpc: change no wallet loaded message to be clearer
Changes the no wallet is loaded rpc error message to be clearer that no
wallet is loaded and how the user can load or create a wallet. Also
changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as
that makes more sense.
2020-10-07 21:36:44 -04:00
Ethan Heilman
bd5215103e random: fixes read buffer resizing in RandAddSeedPerfmon
+ Replaces std::max with std::min to resize buffer in RandAddSeedPerfmon
+ Documents behavior of RandAddSeedPerfmon
2020-10-07 20:46:49 -04:00
MarcoFalke
283a73d7ea Merge #20092: util: Do not use gArgs global in ArgsManager member functions
d103484fe8 util: Do not use gArgs global in ArgsManager member functions (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK d103484fe8: patch looks correct

Tree-SHA512: dda7a5062363170c6995f2fd8fda48c0a919e5ca67be9faa8f0fa66f9d3b535f134eb6f4860a0859bc5457c02230b34a8d1264045f22bed8d30668158ac2271f
2020-10-07 09:31:12 +02:00
fanquake
db88db4727 Merge #19339: validation: re-delegate absurd fee checking from mempool to clients
b048b275d9 [validation] Remove absurdfee from accepttomempool (John Newbery)
932564b9cf scripted-diff: update max-fee-exceeded error message to include RPC (gzhao408)
8f1290c601 [rpc/node] check for high fee before ATMP in clients (gzhao408)

Pull request description:

  Picked up from #15810. Add separate fee-checking logic for clients that need to enforce max fee rates, then remove the `absurdFee` logic from ATMP.

  ATMP's `nAbsurdFee` argument is used to enforce user-specific behavior (it is not policy since it isn't applied consistently: it is only ever used in RPC and wallet, and set to 0 everywhere else internally). It should be removed from `AcceptToMemoryPool` because (1) validation results/mempool behavior should not be user-specific and (2) enforcing a max fee rate should be the responsibility of the client instead of the mempool.

  Note: this PR does not intend to _remove_ protection from high fees, just re-delegate the responsibility to clients.

ACKs for top commit:
  jnewbery:
    utACK b048b275d9
  LarryRuane:
    re-ACK b048b275d9
  MarcoFalke:
    re-ACK b048b275d9 , only change is squashing one commit 🏦
  instagibbs:
    utACK b048b275d9

Tree-SHA512: 57c17ba16d230a4cae2896dd6a64c924f307757824e35784bf96da7b10aff2d8ea910710edf35e981035623a155f8766209a92a0fdb856549fde78bc3eaae4d2
2020-10-07 10:58:30 +08:00
Pieter Wuille
8bd2b4e784 refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script
The old name is confusing, as it doesn't store a scriptPubKey, but the
actually executed script.
2020-10-06 14:51:30 -07:00
Pieter Wuille
107b57df9f scripted-diff: put ECDSA in name of signature functions
In preparation for adding Schnorr versions of `CheckSig`, `VerifySignature`, and
`ComputeEntry`, give them an ECDSA specific name.

-BEGIN VERIFY SCRIPT-
sed -i 's/CheckSig(/CheckECDSASignature(/g' $(git grep -l CheckSig ./src)
sed -i 's/VerifySignature(/VerifyECDSASignature(/g' $(git grep -l VerifySignature ./src)
sed -i 's/ComputeEntry(/ComputeEntryECDSA(/g' $(git grep -l ComputeEntry ./src)
-END VERIFY SCRIPT-
2020-10-06 14:51:30 -07:00
Pieter Wuille
f8c099e220 --- [TAPROOT] Refactors --- 2020-10-06 14:51:30 -07:00
Pieter Wuille
d8cd7b137f Merge #20095: CI: Bump vcpkg commit ID to get new msys mirror list
c83155b406 Bump vcpkg commit ID to get new msys mirror list (Aaron Clauson)

Pull request description:

  This fixes the appveyor CI job, see #20066.

  Currently the job fails because some of the vcpkg dependencies need to install msys2 and the hardcoded mirror in the vcpkg config is down.

  Vcpkg commit 76a7e9248fb3c57350b559966dcaa2d52a5e4458 adds new mirrors to the hardcoded list.

ACKs for top commit:
  decryp2kanon:
    utACK c83155b406
  practicalswift:
    ACK c83155b406
  sipa:
    utACK c83155b406. Verified that 40230b8e3f6368dcb398d649331be878ca1e9007 is a recent commit in the https://github.com/microsoft/vcpkg repository's master branch.

Tree-SHA512: 0887454a18d341cca35448fc8003d82b522ca32b0e7e5929a3b79d72603b79769fe0af9cc6eec3565e905eb530ada7ca8e4b9f1c04881197a5ccf0f8c829274e
2020-10-06 12:00:24 -07:00
Aaron Clauson
c83155b406 Bump vcpkg commit ID to get new msys mirror list
This fixes the appveyor CI job, see #20066.

Currently the job fails because some of the vcpkg dependencies need to install msys2 and the hardcoded mirror in the vcpkg config is down.

Vcpkg commit 76a7e9248fb3c57350b559966dcaa2d52a5e4458 adds new mirrors to the hardcoded list.
2020-10-06 19:17:53 +01:00
Hennadii Stepanov
d103484fe8 util: Do not use gArgs global in ArgsManager member functions 2020-10-06 14:57:06 +03:00
João Barbosa
c92387232f refactor: Extract ParseOpCode from ParseScript
A second lookup in mapOpNames is also removed.
2020-10-06 12:34:05 +01:00
Hennadii Stepanov
2dc79c4264 doc: Update and improve files.md
Added the `signet` subdirectory and the `ip_asn.map` file.
2020-10-06 11:05:03 +03:00
John Newbery
b048b275d9 [validation] Remove absurdfee from accepttomempool
Mempool behavior should not be user-specific.
Checking that txfee is acceptable should be
the responsibility of the wallet or client, not
the mempool.
2020-10-05 04:55:01 -07:00
gzhao408
932564b9cf scripted-diff: update max-fee-exceeded error message to include RPC
-BEGIN VERIFY SCRIPT-
sed -i 's/Fee exceeds maximum configured by \-\maxtxfee/Fee exceeds maximum configured by user (e.g. -maxtxfee, maxfeerate)/g' src/util/error.cpp test/functional/rpc_fundrawtransaction.py test/functional/rpc_psbt.py test/functional/rpc_rawtransaction.py test/functional/wallet_bumpfee.py test/functional/wallet_create_tx.py
-END VERIFY SCRIPT-
2020-10-05 04:55:01 -07:00
gzhao408
8f1290c601 [rpc/node] check for high fee before ATMP in clients
Check absurd fee in BroadcastTransaction and RPC,
return TransactionError::MAX_FEE_EXCEEDED instead
of TxValidationResult::TX_NOT_STANDARD because this
is client preference, not a node-wide policy.
2020-10-05 04:54:05 -07:00
MarcoFalke
875e1ccc9f Merge #19947: Test: Cover "change_type" option of "walletcreatefundedpsbt" RPC
a56e9f5670 test: Assert exclusive PSBT funding options (Oliver Gugger)
64bc5efd39 test: Assert PSBT change type (Oliver Gugger)

Pull request description:

  Increases test coverage of the `walletcreatefundedpsbt` RPC.

  Tests the following combinations:
   - Make sure the global option `-changetype` is used as the default value for the `change_type` option if not specified.
   - Make sure the global option `-changetype` can be overwritten by explicitly setting the `change_type` option of the `walletcreatefundedpsbt` RPC call.
   - Make sure the options `change_type` and `changeAddress` are mutually exclusive.

ACKs for top commit:
  achow101:
    ACK a56e9f5670

Tree-SHA512: bf0fb20c890887b7228ad9277fdb32f367ba772eed6efbe2b4f471f808d4d435110256601e8ebd9bea57026d9f22f3cc3c26a009b017e3da6d8fc6896313def5
2020-10-05 09:45:11 +02:00
fanquake
dde104963b Merge #20064: RPC: remove duplicate line in getblock help
1885ad3546 RPC: remove duplicate line in getblock help (Fabian Jahr)

Pull request description:

  Line simply seems duplicated in error.

  Testing instructions:
  Run `src/bitcoin-cli help getblock` on master branch to reproduce. Then build this PR and compare its results.

ACKs for top commit:
  dhruv:
    tACK `1885ad3`
  kristapsk:
    ACK 1885ad3546
  Emzy:
    tACK 1885ad3546

Tree-SHA512: 870c035cb553b0e1d5ef72e64231ef277e0392efe94bc6ecf47129023bd94a6d5a276f46529807f68a1db55c7baa94d9119c7264d9947bc4e5dd9dcefd1b13e7
2020-10-05 09:40:17 +08:00
MarcoFalke
fae7a1c188 fuzz: Configure check for main function 2020-10-04 17:49:07 +02:00
MarcoFalke
cce1513179 Merge #19723: Ignore unknown messages before VERACK
675e55e013 Ignore unknown messages before VERACK (Suhas Daftuar)

Pull request description:

  This allows for feature negotiation to take place with messages between VERSION and VERACK in the future, without requiring additional software changes to specifically ignore messages for features that are unimplemented by our software.

ACKs for top commit:
  sipa:
    utACK 675e55e013
  practicalswift:
    ACK 675e55e013: patch looks correct
  MarcoFalke:
    ACK 675e55e013
  hebasto:
    ACK 675e55e013, the offender peer will be eventually disconnected due to the timeout.

Tree-SHA512: 8d2b1d8b9843f2ee26b2c30f7c5ff0bfcfbe3f46b32cd0369c48ece26624151091237e83ce3f18c6da004099026602cfab1642ac916db777f047d170b365c007
2020-10-04 15:39:26 +02:00
MarcoFalke
f0fd132226 Merge #20072: ci: Build Arm64 on Travis without functional tests
33df8d46bb ci: Build Arm64 on Travis without functional tests (Fabian Jahr)

Pull request description:

  The Travis Arm64 env is a pretty big PITA for quite a while now. It simply seems to time out due to a lack of resources as far as I can tell from my research into the matter.

  I have reported the issue in August and received no response: https://travis-ci.community/t/arm64-failing-without-message/9775. Others have complained about similar issues with Arm64 over the past year. The explanation for this is probably that this env is still considered to be in beta: https://docs.travis-ci.com/user/multi-cpu-architectures.

  Considering the high frequency of failures (I didn't run any numbers, just based on my personal observations) and the beta status I would suggest that we shorten the runtime by removing part of the test suite until the env is more stable. I would suggest removing functional tests since it should help for sure but maybe there are other ideas.

ACKs for top commit:
  MarcoFalke:
    ACK 33df8d46bb

Tree-SHA512: 55b0c658b526611d16a26e9508611ddeecbfbd842ec064ada61d188a8be2f473e080c2a35746a9aae8006b11b0c2ab9dd4639b0f3004ecf821506e26c7345972
2020-10-04 09:42:53 +02:00
MarcoFalke
2f7a53cc9d Merge #20069: test: Mention commit id in scripted diff error
3491bf358a test: Mention commit id in scripted diff error (Wladimir J. van der Laan)

Pull request description:

  Add commit id to make spotting the issue easier.

ACKs for top commit:
  robot-dreams:
    ACK 3491bf358a
  sipa:
    utACK 3491bf358a
  hebasto:
    ~ACK~ Concept ACK 3491bf358a, should help in situations like https://travis-ci.org/github/bitcoin/bitcoin/jobs/732481553

Tree-SHA512: 1ae66fa760f9e5d52e029bae71f6b5863f1efd7b95de3723ea09290944c9d7687f5ec6927aa115a3aebd6f2b993baa0c2433975c6ad5cd2858089013362eb599
2020-10-04 09:09:38 +02:00
MarcoFalke
06314fbb55 Merge #20071: ci, lint: Remove usage of TRAVIS_COMMIT_RANGE
a91ab86fae lint: Use TRAVIS_BRANCH in lint-git-commit-check.sh (Fabian Jahr)
c11dc995c9 lint: Don't use TRAVIS_COMMIT_RANGE in whitespace linter (Fabian Jahr)
1b41ce8f5f lint: Don't use TRAVIS_COMMIT_RANGE for commit-script-check (Fabian Jahr)

Pull request description:

  This is causing problems again, very similar to #19654.

  UPDATE: This now removes all remaining usages of TRAVIS_COMMIT_RANGE and instead uses TRAVIS_BRANCH for the range, including `lint-git-commit-check` where TRAVIS_COMMIT_RANGE had already been removed. For builds triggered by a pull request, TRAVIS_BRANCH is the name of the branch targeted by the pull request. In the linters there is still a fallback that assumes master as the target branch.

ACKs for top commit:
  sipa:
    ACK a91ab86fae. See test I tried in #20075.

Tree-SHA512: 1378bdebd5d8787a83fbda5d9999cc9447209423e7f0218fe5eb240e6a32dc1b51d1cd53b4f8cd1f71574d935ac5e22e203dfe09cce17e9976a48416038e1263
2020-10-04 09:05:59 +02:00
Fabian Jahr
a91ab86fae lint: Use TRAVIS_BRANCH in lint-git-commit-check.sh 2020-10-03 21:56:15 +02:00
Fabian Jahr
c11dc995c9 lint: Don't use TRAVIS_COMMIT_RANGE in whitespace linter 2020-10-03 21:56:13 +02:00
Fabian Jahr
1b41ce8f5f lint: Don't use TRAVIS_COMMIT_RANGE for commit-script-check 2020-10-03 21:55:52 +02:00
Fabian Jahr
33df8d46bb ci: Build Arm64 on Travis without functional tests 2020-10-03 19:20:04 +02:00
Hennadii Stepanov
3984b78cd7 test: Add tests for CNode::ConnectedThroughNetwork 2020-10-03 15:38:49 +03:00
Hennadii Stepanov
49fba9c1aa net: Add CNode::ConnectedThroughNetwork member function 2020-10-03 15:38:19 +03:00
Wladimir J. van der Laan
3491bf358a test: Mention commit id in scripted diff error 2020-10-03 13:50:52 +02:00
Hennadii Stepanov
d4dde24034 net: Add CNode::m_inbound_onion data member 2020-10-03 13:56:19 +03:00
fanquake
54fc96ffa7 Merge #19956: rpc: Improve invalid vout value rpc error message
f471a3be00 scripted diff: Improve invalid vout value rpc error message (Nima Yazdanmehr)

Pull request description:

  Since the `vout` value can start at `0`, the error message for *negative* values can be improved to something like: `vout cannot be negative`.

ACKs for top commit:
  fanquake:
    ACK f471a3be00
  promag:
    Code review ACK f471a3be00.

Tree-SHA512: fbdee3d0ddd5b58eb93934a1217b44e125a9ad39e672b1f35c7609c6c5fcf45ae1b731d3d6135b7225d98792dbfc34a50907b8c41274a5b029d7b5c59f886560
2020-10-03 11:13:21 +08:00
Fabian Jahr
1885ad3546 RPC: remove duplicate line in getblock help 2020-10-03 00:44:59 +02:00
Andrew Chow
f6b3052739 Explicitly filter out partial groups when we don't want them
Instead of hacking OutputGroup::m_ancestors to discourage the inclusion
of partial groups via the eligibility filter, add a parameter to the
eligibility filter that indicates whether we want to include the group.
Then for those partial groups, don't return them in GroupOutputs if we
indicate they aren't desired.
2020-10-02 12:35:22 -04:00
Andrew Chow
416d74fb16 Move OutputGroup positive only filtering into Insert 2020-10-02 12:35:04 -04:00
Oliver Gugger
a56e9f5670 test: Assert exclusive PSBT funding options
Make sure the options "change_type" and "changeAddress" of the
walletcreatefundedpsbt RPC cannot both be specified a the same time.
2020-10-02 17:15:52 +02:00
MarcoFalke
171cd05ae3 Merge #20034: test: Get rid of default wallet hacks
c1585bca8d test: Get rid of default wallet hacks (Russell Yanofsky)
ed3acda33b test, refactor: add default_wallet_name and wallet_data_filename variables (Russell Yanofsky)

Pull request description:

  Changes:

  - Get rid of setup_nodes (`-wallet`, `-nowallet`, `-disablewallet`) argument rewriting
  - Get rid of hardcoded wallet `""` names and `-wallet=""` args

  Motivation:

  - Simplify test framework behavior so it's easier to write new tests without having arguments mangled by the framework
  - Make tests more readable, replacing unexplained `""` string literals with `default_wallet_name` references
  - Make it trivial to update default wallet name and wallet data filename for sqlite #19077 testing
  - Stop relying on `-wallet` arguments to create wallets, so it is easy to change `-wallet` option in the future to only load existing wallets not create new ones (to avoid accidental wallet creation, and encourage use of wallet encryption and descriptor features)

ACKs for top commit:
  MarcoFalke:
    ACK c1585bca8d, only effective change is adding documentation 🎵

Tree-SHA512: f62dec7cbdacb5f330aa0e1eec89ab4d065540d91495bbedcb375eda1c080b45ce9edb310ce253c44c4839f1b4cc2c7df9816c58402d5d43f94a437e301ea8bc
2020-10-02 17:07:35 +02:00
Wladimir J. van der Laan
d9935222d5 Merge #19951: net, test: CNetAddr scoped ipv6 test coverage, rename scopeId to m_scope_id
f36887fa47 net: rename CNetAddr scopeId to m_scope_id, improve code doc (Jon Atack)
5cb5fd3005 test: add test coverage for CNetAddr ipv6 scoped addresses (Jon Atack)

Pull request description:

  Add some test coverage for the IPv6 scoped address feature in `netaddress`/`netbase` per https://tools.ietf.org/html/rfc4007, update the member name `scopeId` to `m_scope_id` per `doc/developer-notes.md`, and improve the code doc.

  ----

  Reviewers can manually verify the test with these steps:

  - [pull down the changes locally and check out this branch](https://jonatack.github.io/articles/how-to-review-pull-requests-in-bitcoin-core#pull-down-the-code-locally)

  - [build Bitcoin Core](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests)

  - run the test: `cd src ; test/test_bitcoin -t net_tests/cnetaddr_basic -l all`

  <details><summary><em>you should see passing test output like this</em></summary><p>

  ```
  Running 1 test case...
  Entering test module "Bitcoin Core Test Suite"
  test/net_tests.cpp(85): Entering test suite "net_tests"
  test/net_tests.cpp(204): Entering test case "cnetaddr_basic"
  test/net_tests.cpp(210): info: check LookupHost("0.0.0.0", addr, false) has passed
  test/net_tests.cpp(211): info: check !addr.IsValid() has passed
  test/net_tests.cpp(212): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(214): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(215): info: check addr.ToString() == "0.0.0.0" has passed
  test/net_tests.cpp(218): info: check LookupHost("255.255.255.255", addr, false) has passed
  test/net_tests.cpp(219): info: check !addr.IsValid() has passed
  test/net_tests.cpp(220): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(222): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(223): info: check addr.ToString() == "255.255.255.255" has passed
  test/net_tests.cpp(226): info: check LookupHost("12.34.56.78", addr, false) has passed
  test/net_tests.cpp(227): info: check addr.IsValid() has passed
  test/net_tests.cpp(228): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(230): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(231): info: check addr.ToString() == "12.34.56.78" has passed
  test/net_tests.cpp(234): info: check LookupHost("::", addr, false) has passed
  test/net_tests.cpp(235): info: check !addr.IsValid() has passed
  test/net_tests.cpp(236): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(238): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(239): info: check addr.ToString() == "::" has passed
  test/net_tests.cpp(242): info: check LookupHost(ipv6_addr, addr, false) has passed
  test/net_tests.cpp(243): info: check addr.IsValid() has passed
  test/net_tests.cpp(244): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(246): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(247): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(266): info: check LookupHost(ipv6_addr + "%0", addr, false) has passed
  test/net_tests.cpp(267): info: check addr.IsValid() has passed
  test/net_tests.cpp(268): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(269): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(270): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(274): info: check addr.IsValid() has passed
  test/net_tests.cpp(275): info: check addr.IsTor() has passed
  test/net_tests.cpp(277): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(278): info: check addr.ToString() == "6hzph5hv6337r6p2.onion" has passed
  test/net_tests.cpp(282): info: check !addr.IsValid() has passed
  test/net_tests.cpp(283): info: check addr.IsInternal() has passed
  test/net_tests.cpp(285): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(286): info: check addr.ToString() == "esffpvrt3wpeaygy.internal" has passed
  test/net_tests.cpp(204): Leaving test case "cnetaddr_basic"; testing time: 30933us
  test/net_tests.cpp(85): Leaving test suite "net_tests"; testing time: 30975us
  Leaving test module "Bitcoin Core Test Suite"; testing time: 31169us

  *** No errors detected
  ```
  </p>/</details>

  - change this line in the code to break the feature:
  ```diff
  --- a/src/netaddress.cpp
  +++ b/src/netaddress.cpp
  @@ -716,7 +716,7 @@ bool CService::GetSockAddr(struct sockaddr* paddr, socklen_t *addrlen) const
           memset(paddrin6, 0, *addrlen);
           if (!GetIn6Addr(&paddrin6->sin6_addr))
               return false;
  -        paddrin6->sin6_scope_id = m_scope_id;
  +        // paddrin6->sin6_scope_id = m_scope_id;
  ```

  - rebuild, e.g. `make`

  - run the test: `test/test_bitcoin -t net_tests/cnetaddr_basic -l all`, verify that the added tests break

  <details><summary><em>you should see test output with a few failed tests like this</em></summary><p>

  ```
  Running 1 test case...
  Entering test module "Bitcoin Core Test Suite"
  test/net_tests.cpp(85): Entering test suite "net_tests"
  test/net_tests.cpp(204): Entering test case "cnetaddr_basic"
  test/net_tests.cpp(210): info: check LookupHost("0.0.0.0", addr, false) has passed
  test/net_tests.cpp(211): info: check !addr.IsValid() has passed
  test/net_tests.cpp(212): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(214): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(215): info: check addr.ToString() == "0.0.0.0" has passed
  test/net_tests.cpp(218): info: check LookupHost("255.255.255.255", addr, false) has passed
  test/net_tests.cpp(219): info: check !addr.IsValid() has passed
  test/net_tests.cpp(220): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(222): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(223): info: check addr.ToString() == "255.255.255.255" has passed
  test/net_tests.cpp(226): info: check LookupHost("12.34.56.78", addr, false) has passed
  test/net_tests.cpp(227): info: check addr.IsValid() has passed
  test/net_tests.cpp(228): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(230): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(231): info: check addr.ToString() == "12.34.56.78" has passed
  test/net_tests.cpp(234): info: check LookupHost("::", addr, false) has passed
  test/net_tests.cpp(235): info: check !addr.IsValid() has passed
  test/net_tests.cpp(236): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(238): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(239): info: check addr.ToString() == "::" has passed
  test/net_tests.cpp(242): info: check LookupHost(ipv6_addr, addr, false) has passed
  test/net_tests.cpp(243): info: check addr.IsValid() has passed
  test/net_tests.cpp(244): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(246): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(247): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%1]
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%21]
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%19]
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%3]
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(266): info: check LookupHost(ipv6_addr + "%0", addr, false) has passed
  test/net_tests.cpp(267): info: check addr.IsValid() has passed
  test/net_tests.cpp(268): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(269): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(270): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(274): info: check addr.IsValid() has passed
  test/net_tests.cpp(275): info: check addr.IsTor() has passed
  test/net_tests.cpp(277): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(278): info: check addr.ToString() == "6hzph5hv6337r6p2.onion" has passed
  test/net_tests.cpp(282): info: check !addr.IsValid() has passed
  test/net_tests.cpp(283): info: check addr.IsInternal() has passed
  test/net_tests.cpp(285): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(286): info: check addr.ToString() == "esffpvrt3wpeaygy.internal" has passed
  test/net_tests.cpp(204): Leaving test case "cnetaddr_basic"; testing time: 32316us
  test/net_tests.cpp(85): Leaving test suite "net_tests"; testing time: 32354us
  Leaving test module "Bitcoin Core Test Suite"; testing time: 32522us

  *** 4 failures are detected in the test module "Bitcoin Core Test Suite"
  ```
  </p></details>

  - leave your review here

ACKs for top commit:
  laanwj:
    ACK f36887fa47

Tree-SHA512: 8e77e512db130642be7d3a910735ca803a2afdb5a704f713f163f8b5a80be3077a2be6f0a3ca43d299731dcd2545ac35571f8c74e5250a72a48233c26f9a3ab5
2020-10-02 17:06:59 +02:00
Wladimir J. van der Laan
597488d37c Merge #19871: doc: Clarify scope of eviction protection of outbound block-relay peers
d76925478e [doc] Clarify semantic of peer's m_protect w.r.t to outbound eviction logics (Antoine Riard)
ac71fe936d [doc] Clarify scope of eviction protection of outbound block-relay peers (Antoine Riard)

Pull request description:

  Block-relay-only peers were introduced by #15759. According to its
  author, it was intented to make them only immune to outbound peer
  rotation-based eviction and not from all eviction as modified comment
  leans to think of.

  Clearly indicate that outbound block-relay peers aren't protected
  from eviction by the bad/lagging chain logic.

  Fix #19863

ACKs for top commit:
  naumenkogs:
    ACK d76925478e
  jonatack:
    ACK d76925478e

Tree-SHA512: 597fbd62838a6e39276024165b11514cad20a2e9d33cf9202d261cbadcb62b2df427c858e0cb57e585840d4c1d4600104aa53916bb868541f2580e4eed9b4b52
2020-10-02 16:42:38 +02:00
Jon Atack
f36887fa47 net: rename CNetAddr scopeId to m_scope_id, improve code doc 2020-10-02 16:42:02 +02:00
Jon Atack
5cb5fd3005 test: add test coverage for CNetAddr ipv6 scoped addresses 2020-10-02 16:40:15 +02:00
MarcoFalke
fa5ed3b4ca net: Use Span in ReceiveMsgBytes 2020-10-02 16:26:33 +02:00
Oliver Gugger
64bc5efd39 test: Assert PSBT change type
Make sure the wallet's default change type is respected by default when
funding a PSBT but can be overwritten by the "change_type" option.
2020-10-02 15:48:45 +02:00
Wladimir J. van der Laan
df2129a234 Merge #19991: net: Use alternative port for incoming Tor connections
96571b3d4c doc: Update onion service target port numbers in tor.md (Hennadii Stepanov)
bb145c9050 net: Extend -bind config option with optional network type (Hennadii Stepanov)
92bd3c1da4 net, refactor: Move AddLocal call one level up (Hennadii Stepanov)
57f17e57c8 net: Pass onion service target to Tor controller (Hennadii Stepanov)
e3f07851f0 refactor: Rename TorController::target to m_tor_control_center (Hennadii Stepanov)
fdd3ae4d26 net, refactor: Refactor CBaseChainParams::RPCPort function (Hennadii Stepanov)
a5266d4546 net: Add alternative port for onion service (Hennadii Stepanov)
b3273cf403 net: Use network byte order for in_addr.s_addr (Hennadii Stepanov)

Pull request description:

  This PR adds ability to label incoming Tor connections as different from normal localhost connections.

  Closes #8973.
  Closes #16693.

  Default onion service target ports are:
  - 8334 on mainnnet
  - 18334 on testnet
  - 38334 on signet
  - 18445 on regtest

  To set the onion service target socket manually the extended `-bind` config option could be used:

  ```
  $ src/bitcoind -help | grep -A 6 -e '-bind'
    -bind=<addr>[:<port>][=onion]
         Bind to given address and always listen on it (default: 0.0.0.0). Use
         [host]:port notation for IPv6. Append =onion to tag any incoming
         connections to that address and port as incoming Tor connections
         (default: 127.0.0.1:8334=onion, testnet: 127.0.0.1:18334=onion,
         signet: 127.0.0.1:38334=onion, regtest: 127.0.0.1:18445=onion)

  ```

  Since [pr19991.02 update](https://github.com/bitcoin/bitcoin/pull/19991#issuecomment-698882284) this PR is an alternative to #19043.

ACKs for top commit:
  Sjors:
    re-utACK 96571b3d4c
  vasild:
    ACK 96571b3d4
  laanwj:
    Re-ACK 96571b3d4c

Tree-SHA512: cb0eade80f4b3395f405f775e1b89c086a1f09d5a4464df6cb4faf808d9c2245474e1720b2b538f203f6c1996507f69b09f5a6e35ea42633c10e22bd733d4438
2020-10-02 13:37:23 +02:00
MarcoFalke
99992e7832 doc: Collect release-notes snippets 2020-10-02 11:24:46 +02:00
fanquake
60ec57b3d7 Merge #20055: rpc: Set HTTP Content-Type in bitcoin-cli
7eab781a14 rpc: Set HTTP Content-Type in bitcoin-cli (Wladimir J. van der Laan)

Pull request description:

  We don't set any `Content-Type` in the client. It is more consistent with our other JSON-RPC use to set it to `application/json`.

  Note that our server doesn't enforce content types, so it doesn't make a difference in practice. But it is fairly strange HTTP behavior to not set it at all for a POST request.

  This came up in #18950.

ACKs for top commit:
  promag:
    ACK 7eab781a14.
  jonatack:
    Tested ACK 7eab781a14
  practicalswift:
    ACK 7eab781a14: patch looks correct
  fanquake:
    ACK 7eab781a14 - Looks fine to me.

Tree-SHA512: a9fa155324d0f7bff955585a336ead6bb60b721039f424521a435e4bb0fad3f4532e5cc7b7a9acc4e93585e8d3db3082c010138810f22c0e92b8f749b86ef653
2020-10-02 15:25:20 +08:00
MarcoFalke
a6a993a888 Merge #20060: refactor: fix -Wbraced-scalar-init warning in validation tests
82b70f15c7 refactor: fix -Wbraced-scalar-init warning in validation tests (fanquake)

Pull request description:

  Introduced in #20004 (fa29b5ae66).

  ```
  test/validation_tests.cpp:68:88: warning: braces around scalar initializer [-Wbraced-scalar-init]
      BOOST_CHECK(signet_params->GetConsensus().signet_challenge == std::vector<uint8_t>{{OP_TRUE}});
                                                                                         ^~~~~~~~~
  /usr/local/include/boost/test/tools/old/interface.hpp:83:6: note: expanded from macro 'BOOST_CHECK'
      (P), BOOST_TEST_STRINGIZE( P ), CHECK, CHECK_PRED, _ )
       ^
  /usr/local/include/boost/test/tools/old/interface.hpp:68:61: note: expanded from macro 'BOOST_TEST_TOOL_IMPL'
          BOOST_JOIN( BOOST_TEST_TOOL_PASS_PRED, frwd_type )( P, ARGS ),          \
                                                              ^
  /usr/local/include/boost/test/tools/old/interface.hpp:51:47: note: expanded from macro 'BOOST_TEST_TOOL_PASS_PRED2'
                                                ^
  1 warning generated.
  ```

ACKs for top commit:
  practicalswift:
    ACK 82b70f15c7
  MarcoFalke:
    ACK 82b70f15c7

Tree-SHA512: ae14acd52e2a0d370a6ee7321f257190c6a44094eb3fa5a6bd85b6bb2b4002e7784589cb34dcf78545238c33cbea38113061b2a46e092f1119731e70932fa469
2020-10-02 08:38:12 +02:00
Anthony Towns
d9141a0002 doc: clarify CRollingBloomFilter size estimate 2020-10-01 22:22:56 -07:00
fanquake
82b70f15c7 refactor: fix -Wbraced-scalar-init warning in validation tests
Introduced in #20004 (fa29b5ae66).

```bash
test/validation_tests.cpp:68:88: warning: braces around scalar initializer [-Wbraced-scalar-init]
    BOOST_CHECK(signet_params->GetConsensus().signet_challenge == std::vector<uint8_t>{{OP_TRUE}});
                                                                                       ^~~~~~~~~
/usr/local/include/boost/test/tools/old/interface.hpp:83:6: note: expanded from macro 'BOOST_CHECK'
    (P), BOOST_TEST_STRINGIZE( P ), CHECK, CHECK_PRED, _ )
     ^
/usr/local/include/boost/test/tools/old/interface.hpp:68:61: note: expanded from macro 'BOOST_TEST_TOOL_IMPL'
        BOOST_JOIN( BOOST_TEST_TOOL_PASS_PRED, frwd_type )( P, ARGS ),          \
                                                            ^
/usr/local/include/boost/test/tools/old/interface.hpp:51:47: note: expanded from macro 'BOOST_TEST_TOOL_PASS_PRED2'
                                              ^
1 warning generated.
```
2020-10-02 12:30:50 +08:00
fanquake
69a73803df Merge #20058: Update transifex slug for 0.21
7532c9a221 qt: Periodic translations update (Wladimir J. van der Laan)
598e3b1ed3 tx: Update transifex slug for 0.21 (Wladimir J. van der Laan)

Pull request description:

  Update transifex slug for 0.21 (as part of #18947), and do a periodic translations update.

ACKs for top commit:
  fanquake:
    ACK 7532c9a221

Tree-SHA512: 7e396c9a7bcf69088ec89b604332b44cfc2f1949665a7d464ad7cd1c6156cd32ca4a6f7e0b70e4782966f0904c82c4c9286b25ada98052adeaf97e1f408489f6
2020-10-02 12:16:26 +08:00
Wladimir J. van der Laan
7532c9a221 qt: Periodic translations update 2020-10-01 22:24:08 +02:00
Wladimir J. van der Laan
598e3b1ed3 tx: Update transifex slug for 0.21 2020-10-01 22:19:11 +02:00
Hennadii Stepanov
96571b3d4c doc: Update onion service target port numbers in tor.md 2020-10-01 19:19:43 +03:00
Hennadii Stepanov
bb145c9050 net: Extend -bind config option with optional network type 2020-10-01 19:19:20 +03:00
Hennadii Stepanov
92bd3c1da4 net, refactor: Move AddLocal call one level up
This change simplifies the following commit.
2020-10-01 19:00:10 +03:00
Hennadii Stepanov
57f17e57c8 net: Pass onion service target to Tor controller 2020-10-01 19:00:07 +03:00
Wladimir J. van der Laan
a0185d90a7 Merge #18309: zmq: Add support to listen on multiple interfaces
e66870c5a4 zmq: Append address to notify log output (nthumann)
241803da21 test: Add zmq test to support multiple interfaces (nthumann)
a0b2e5cb6a doc: Add release notes to support multiple interfaces (nthumann)
b1c3f180ec doc: Adjust ZMQ usage to support multiple interfaces (nthumann)
347c94f551 zmq: Add support to listen on multiple interfaces (Nicolas Thumann)

Pull request description:

  This PR adds support for ZeroMQ to listen on multiple interfaces, just like the RPC server.
  Currently, if you specify more than one e.g. `zmqpubhashblock` paramter, only the first one will be used. Therefore a user may be forced to listen on all interfaces (e.g. `zmqpubhashblock=0.0.0.0:28332`), which can result in an increased attack surface.
  With this PR a user can specify multiple interfaces to listen on, e.g.
  `-zmqpubhashblock=tcp://127.0.0.1:28332 -zmqpubhashblock=tcp://192.168.1.123:28332`.

ACKs for top commit:
  laanwj:
    Code review ACK e66870c5a4
  instagibbs:
    reACK e66870c5a4

Tree-SHA512: f38ab4a6ff00dc821e5f4842508cefadb701e70bb3893992c1b32049be20247c8aa9476a1f886050c5f17fe7f2ce99ee30193ce2c81a7482a5a51f8fc22300c7
2020-10-01 17:43:34 +02:00
Wladimir J. van der Laan
7eab781a14 rpc: Set HTTP Content-Type in bitcoin-cli
We don't set any `Content-Type` in the client. It is more
consistent with our other JSON-RPC use to set it to `application/json`.

Note that our server doesn't enforce content types, so it doesn't make a
difference in practice. But it is fairly strange HTTP behavior to not set it.

This came up in #18950.
2020-10-01 17:33:56 +02:00
MarcoFalke
0000a0c7e9 Remove confusing and almost useless "unexpected version" warning 2020-10-01 16:29:26 +02:00
MarcoFalke
9fc2f011ba Merge #20048: chainparams: do not log signet startup messages for other chains
6fccad7f71 signet: do not log signet startup messages for other chains (Jon Atack)

Pull request description:

  The following signet startup messages are printed to the debug log immediately on node startup for all chains. This behavior occurs on master as a side effect after the merge of #20014. This PR removes the first message and moves the signet derived magic logging to `init.cpp`.
  ```
  $ ./src/bitcoind
  2020-09-30T14:25:15Z Using default signet network
  2020-09-30T14:25:15Z Signet derived magic (message start): 0a03cf40
  2020-09-30T14:25:15Z Bitcoin Core version v0.20.99.0 ...
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 6fccad7f71
  kallewoof:
    utACK 6fccad7f71
  hebasto:
    ACK 6fccad7f71

Tree-SHA512: 33821dce89b24caf7b7c1ecb41e572ecfb26e6958a1316d359ff240e6ef97c4a1f2cf1b4b974596b252815f9df23960ce385c132ebdbc855bbe6123c3b0b003a
2020-10-01 13:29:05 +02:00
Jon Atack
6fccad7f71 signet: do not log signet startup messages for other chains
and move signet network magic logging from chainparams.cpp to init.cpp
2020-10-01 11:25:42 +02:00
MarcoFalke
40aab35e98 Merge #19253: Tests: tidy up address.py and segwit_addr.py
825fcae484 [tests] Replace bytes literals with hex literals (John Newbery)
64eca45100 [tests] Fix pep8 style violations in address.py (John Newbery)
b230f8b3f3 [tests] Correct docstring for address.py (John Newbery)
ea70e6a2ca [tests] Tidy up imports in address.py (John Newbery)
7f639df0b8 [tests] Remove unused optional verify_checksum parameter (John Newbery)
011e784f74 [tests] Rename segwit encode and decode functions (John Newbery)
e4557133f5 [tests] Move bech32 unit tests to test framework (John Newbery)

Pull request description:

  Lots of small fixes:

  - moving unit tests to test_framework implementation files
  - renaming functions to be clearer
  - removing multiple imports
  - removing unreadable byte literals from the code
  - fixing pep8 violations
  - correcting out-of-date docstring

ACKs for top commit:
  jonatack:
    re-ACK 825fcae484 per `git range-diff a0a422c 7edcdcd 825fcae` and verified `wallet_address_types.py` and `wallet_basic.py --descriptors` (the failure on one travis job) are green locally.
  MarcoFalke:
    ACK 825fcae484
  fanquake:
    ACK 825fcae484 - looks ok to me.

Tree-SHA512: aea509c27c1bcb94bef11205b6a79836c39c62249672815efc9822f411bc2e2336ceb3d72b3b861c3f4054a08e16edb28c6edd3aa5eff72eec1d60ea6ca82dc4
2020-10-01 09:43:11 +02:00
nthumann
e66870c5a4 zmq: Append address to notify log output 2020-10-01 00:33:38 +02:00
nthumann
241803da21 test: Add zmq test to support multiple interfaces 2020-10-01 00:33:38 +02:00
Nima Yazdanmehr
f471a3be00 scripted diff: Improve invalid vout value rpc error message
-BEGIN VERIFY SCRIPT-
r() { sed -i 's/vout must be positive/vout cannot be negative/g' $1 }
r $(git grep -l 'vout must be positive')
-END VERIFY SCRIPT-
2020-09-30 20:43:05 +03:30
Wladimir J. van der Laan
3487e421a7 Merge #20004: test: Add signet witness commitment section parse tests
fa29b5ae66 test: Add signet witness commitment section parse tests (MarcoFalke)
fa23308e9a Remove gArgs global from CreateChainParams to aid testing (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK fa29b5ae66

Tree-SHA512: f956407d690decbfb8178bcb8f101d107389fecc3aa7be515f7b0f5ceac26d798c165100f7ddf08cec569beabcc6514862dda23b667cc4fd0a784316784735c2
2020-09-30 17:00:19 +02:00
Sebastian Falbesoner
136d96b71f test: use wait_for_{block,header} helpers in p2p_fingerprint.py 2020-09-30 16:48:04 +02:00
Russell Yanofsky
7d0271b5c3 depends: Set CMAKE_INSTALL_RPATH for native packages
After #19685 started setting LDFLAGS, the INSTALL_RPATH_USE_LINK_PATH cmake
option used in the libmultiprocess build no longer works, so it is neccessary
to set CMAKE_INSTALL_RPATH as a fallback.

It's unclear currently whether the bad interaction between
INSTALL_RPATH_USE_LINK_PATH and LDFLAGS is a bug, but the issue is reported:

  https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877
  https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892

Commands useful for building / testing this change

  make -C depends MULTIPROCESS=1 print-libmultiprocess_cmake
  make -C depends MULTIPROCESS=1 print-native_libmultiprocess_cmake
  make -C depends MULTIPROCESS=1 HOST=x86_64-apple-darwin16 print-libmultiprocess_cmake

  rm -rvf depends/x86_64-pc-linux-gnu/native depends/work/staging depends/work/build
  make -C depends MULTIPROCESS=1 V=1 native_libmultiprocess_staged
  for f in `find -name mpgen`; do echo == $f ==; readelf -d $f | grep -i path; done

  make -C depends MULTIPROCESS=1 V=1 native_libmultiprocess_built
  find -name CMakeCache.txt

Fixes #19981
2020-09-30 10:03:02 -04:00
Wladimir J. van der Laan
72affcb16c Merge #19802: doc: elaborate on release notes wrt netmasks
8de51d1513 doc: elaborate on release notes wrt netmasks (Vasil Dimov)

Pull request description:

  doc: elaborate on release notes wrt netmasks

  A minor followup from https://github.com/bitcoin/bitcoin/pull/19628#issuecomment-679958713.

ACKs for top commit:
  laanwj:
    ACK 8de51d1513
  practicalswift:
    ACK 8de51d1513
  theStack:
    ACK 8de51d1513

Tree-SHA512: ccece7c3057e476d59e9996582e8594b3db9eaed397139217335a41307e542194c8d13ad72458eaa8580a5c469fef9cc9e3f11c1c609617757579530b465e5b0
2020-09-30 16:03:02 +02:00
MarcoFalke
7b7cb70f4c Merge #19498: Tidy up ProcessOrphanTx
001343f4bc ProcessOrphanTx: Move AddToCompactExtraTransactions call into ProcessOrphanTx (John Newbery)
4fce726bd1 ProcessOrphanTx: Remove aliases (John Newbery)
e07c5d9423 ProcessOrphanTx: Remove outdated commented (John Newbery)
4763b51bca ProcessOrphanTx: remove useless setMisbehaving set (John Newbery)
55c79a9cef ProcessOrphanTx: remove useless done variable (John Newbery)
6e8dd99ef1 [net processing] Add doxygen comments for orphan data and function (John Newbery)

Pull request description:

  Originally a follow-up to #19364, this simplifies the logic in ProcessOrphanTx() and removes unused variables.

ACKs for top commit:
  troygiorshev:
    ACK 001343f4bc
  sipa:
    utACK 001343f4bc
  MarcoFalke:
    ACK 001343f4bc 🌮

Tree-SHA512: be558457f2e08ebb6bddcd49bdd75bd410c3650da44a76c688fc9f07822f94d5a1af93fa1342678052b2c8163cdb9745c352c7884325ab0a41fa593c3eb89116
2020-09-30 15:53:25 +02:00
fanquake
301959fa74 Merge #20043: doc: Add 19501 release notes
fa710a6d67 doc: Add 19501 release notes (MarcoFalke)
faf60dee34 doc: Remove double-whitespace from help string, other whitespace fixups (MarcoFalke)

Pull request description:

  Adds release notes and fixes up some whitespace nits for the touched RPCs

ACKs for top commit:
  fanquake:
    ACK fa710a6d67
  laanwj:
    Code review ACK fa710a6d67

Tree-SHA512: b84a96386a9a8ed69f464c7dffdd600cf9a8b33a06120798b141b300991baed369ab91ae48df6446e89e1d62534ccd8ae721454e7a19b48900b317e9192afc47
2020-09-30 20:23:44 +08:00
Wladimir J. van der Laan
4f5ae52738 Merge #20006: Fix misleading error message: Clean stack rule
af57766182 Fix misleading error message: Clean stack rule (sanket1729)

Pull request description:

  Error messages in clean stack is misleading as it lets the user believe that there are extra
  elements on the stack which is incorrect if the stack is empty.

  Let me know if this requires additional test.

ACKs for top commit:
  instagibbs:
    re-ACK af57766182
  gzhao408:
    reACK af57766182
  theStack:
    re-ACK af57766182
  darosior:
    re ACK af57766182

Tree-SHA512: 88e77416e220b080246fec368f5552a891d102d072b7bee62ac560d5e31c4a8c2ee9cbe569740b253e9df177d21dc788d10d856b2a542ab47761bb81698e4082
2020-09-30 14:21:42 +02:00
Wladimir J. van der Laan
5c435c78e3 Merge #20036: signet: Add assumed values for default signet
fac966142e signet: Add assumed values for default signet (MarcoFalke)

Pull request description:

  Doesn't matter much right now, but when the default signet is bigger, this might come in handy

ACKs for top commit:
  jsarenik:
    Tested ACK fac966142e
  laanwj:
    Tested ACK fac966142e (did a new re-sync)
  kallewoof:
    utACK fac966142e

Tree-SHA512: ed2692f5896350f8dc81f9bc5d79fbf1a4544b8f724c5c667fcadec3a37e26e9833ac189a3067a0731fd7b17a0c94f6b44a641fffe448e42259f7b7b44910db1
2020-09-30 14:05:19 +02:00
MarcoFalke
fa710a6d67 doc: Add 19501 release notes 2020-09-30 09:28:50 +02:00
MarcoFalke
faf60dee34 doc: Remove double-whitespace from help string, other whitespace fixups 2020-09-30 09:28:33 +02:00
MarcoFalke
1769828684 Merge #19501: send* RPCs in the wallet returns the "fee reason"
69cf5d4eeb [test] Make sure send rpc returns fee reason (Sishir Giri)
d5863c0b3e [send] Make send RPCs return fee reason (Sishir Giri)

Pull request description:

  Whenever a wallet funds a transaction, the fee reason is reported to the user only if the verbose is set to true. I added an extra parameter to `CreateTransaction` function in wallet.cpp. Then I implemented the fee reason return logic in `SendMoney`  in rpcwallet.cpp, followed by verbose parameter in `sendtoaddress` and `sendmany` functions. I also added a fee reason test case in walletbasic.py.

  link to the issue: https://github.com/MarcoFalke/bitcoin-core/issues/22#issue-616251578

ACKs for top commit:
  instagibbs:
    ACK 69cf5d4eeb
  meshcollider:
    utACK 69cf5d4eeb

Tree-SHA512: 2e3af32dcfbd5511ba95f8bc8edca7acfe709a8430ff03e43172e5d0af3dfa4b2f57906978e7f272d878043b9ed8c6004674cf47d7496b005d5f612e9a58aa0e
2020-09-30 09:01:23 +02:00
MarcoFalke
36f5a58c02 Merge #19960: build: The vcpkg tool has introduced a proper way to use manifests
712f95d332 Update msvc build to use new vcpkg manifest (Aaron Clauson)

Pull request description:

  The vcpkg tool has introduced a proper way to use [manifests](https://devblogs.microsoft.com/cppblog/vcpkg-accelerate-your-team-development-environment-with-binary-caching-and-manifests/). This PR replaces the custom text file mechanism with the new manifest approach.

  It is planned that vckpg manifests will include the ability to version dependencies in the future. Dependency versions would solve a number of issues that currently require workarounds with the appveyor CI.

ACKs for top commit:
  fanquake:
    ACK 712f95d332 - This is a nice simplification. I tested this in a Windows VM; packages were downloaded and installed automatically as required:
  hebasto:
    Approach ACK 712f95d332, I've verified that changes comply MS docs:

Tree-SHA512: ff9b3d6ad3cacabcbec6566fd289b179af163dc0c4545f8ba666fc14ba07527557f72bc84ba8abfa3bdffb22e2b8ff0b180f41d909c6de76894ac50ddcf8646b
2020-09-30 08:51:38 +02:00
fanquake
c7ad94428a Merge #19958: doc: Better document features of feelers
2ea62cae48 Improve docs about feeler connections (Gleb Naumenko)

Pull request description:

  "feeler" and "test-before-evict" are two different strategies suggest in [Eclipse Attacks on Bitcoin’s Peer-to-Peer Network](https://www.usenix.org/system/files/conference/usenixsecurity15/sec15-paper-heilman.pdf). In our codebase, we use `ConnType::FEELER` to implement both.

  It is confusing, up to the point that our documentation was just incorrect.

  This PR:
  - ~clarifies this aspect by renaming "ConnType::FEELER" to "ConnType::PROBE", meaning that this connections only probes that the node is operational, and then disconnects.~
  - fixes the documentation

ACKs for top commit:
  amitiuttarwar:
    ACK 2ea62cae48. thank you!
  practicalswift:
    ACK 2ea62cae48

Tree-SHA512: c9c03c09eefeacec28ea199cc3f697b0a98723f2f849f7a8115edc43791f8165e296e0e25a82f0b5a4a781a7de38c8954b48bf74c714eba02cdc21f7460673e5
2020-09-30 12:41:22 +08:00
maskoficarus
95fedd33a2 refactor: Clean up -Wlogical-op warning
This commit fixes #19912 by removing a check that always returned true. That check was causing a -Wlogical-op warning because it treated a constant int as though it were a boolean.
2020-09-29 22:08:54 -05:00
Andrew Chow
d895e98b59 Move EligibleForSpending into GroupOutputs
Instead of filtering after the OutputGroups have been made, do it as
they are being made.
2020-09-29 14:26:03 -04:00
Andrew Chow
99b399aba5 Move fee setting of OutputGroup to Insert
OutputGroup will handle the fee and effective value computations
inside of Insert. It now needs to take the effective feerate and long
term feerates as arguments to its constructor.
2020-09-29 14:25:56 -04:00
Andrew Chow
6148a8acda Move GroupOutputs into SelectCoinsMinConf 2020-09-29 14:25:21 -04:00
Andrew Chow
2acad03657 Remove OutputGroup non-default constructors 2020-09-29 14:25:11 -04:00
Wladimir J. van der Laan
de4b7f25ac Merge #20015: doc: Added default signet config for linearize script
8a4dcda414 doc: Added default signet config for linearize script (gr0kchain)

Pull request description:

  Updated the example-linearize.cfg file to include support for the signet chain network config which is used by the ./linearize-data.py

  Problem:
  Without the signet magic, genesis hash and path config, the linearize-data.py script cannot generate a bootstrap.dat file.

  Example:

  ```
  ./linearize-data.py ./linearize.cfg
  Read 4776 hashes
  Genesis block not found in hashlist
  ```

  Solution:

  Added netmagic, genesis and input example parameters to file. Netmagic in terms of signet is derived from the signet-challenge and not static as with other networks. The provided signet magic is based on the default public signet.

  Resolution

  ```
  ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
  ./linearize-data.py ./linearize.cfg
  $ ./linearize-data.py ./linearize.cfg
  Read 4776 hashes
  Input file /Users/gr0kchain/.bitcoin/signet/blocks/blk00000.dat
  Output file /Users/gr0kchain/Downloads/bootstrap.dat
  1000 blocks scanned, 1000 blocks written (of 4776, 20.9% complete)
  2000 blocks scanned, 2000 blocks written (of 4776, 41.9% complete)
  3011 blocks scanned, 3000 blocks written (of 4776, 62.8% complete)
  4010 blocks scanned, 4000 blocks written (of 4776, 83.8% complete)
  Done (4776 blocks written)
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 8a4dcda414

Tree-SHA512: ad4d330358cf67e7424fb1d97ca828c28ca2758102e45747f4059c11a8acce801162da024a20cfb892f997fd4c3f4d5af988d1ca67c74efa78bf7d4ed27dd421
2020-09-29 16:38:22 +02:00
Wladimir J. van der Laan
9e6f56f6ea Merge #19984: log: Remove static log message "Initializing chainstate Chainstate [ibd] @ height -1 (null)"
f22d6a1142 log: Remove static log message "Initializing chainstate Chainstate [ibd] @ height -1 (null)" (practicalswift)

Pull request description:

  Remove static log message `Initializing chainstate Chainstate [ibd] @ height -1 (null)`.

  AFAICT `chainstate->ToString()` will always equal `"Chainstate [ibd] @ height -1 (null)"` here which makes the log message neither relevant nor interesting :)

ACKs for top commit:
  laanwj:
    ACK f22d6a1142
  promag:
    ACK f22d6a1142, just get rid of it.
  hebasto:
    ACK f22d6a1142, I agree that the removed log message in its current state is cryptic and useless.

Tree-SHA512: 1a65c0d14c9a433afcdaadef9bfcdd5d63276d5d2caee1bf3c48ac477e54fa28138f64020e6e26ca5e67872954a1e7d93fa24a12accc7c7211bc6e7a6039051d
2020-09-29 15:24:43 +02:00
Wladimir J. van der Laan
8aa6178961 Merge #20003: net: Exit with error message if -proxy is specified without arguments (instead of continuing without proxy server)
9b4fa0af40 net: Print error message if -proxy is specified without arguments (instead of continuing without proxy server) (practicalswift)

Pull request description:

  Exit with error message if `-proxy` is specified without arguments (instead of continuing without proxy server).

  Continuing without a proxy server when the end-user has specified `-proxy` may result in accidental loss of privacy. (The end-user might think he/she is using a proxy when he/she is not.)

  Before this patch:

  ```
  $ src/bitcoind -proxy
  …
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -listen=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -listen=0 -> setting -listenonion=0
  …
  2020-09-23T00:24:33Z init message: Starting network threads...
  ```

  `bitcoind` is now running *without* a proxy server (`GetProxy(…, …) == false`, `HaveNameProxy() == false`, etc.).

  Note that the "-proxy set" log messages above which the end-user might interpret as "good, my traffic is now routed via the proxy".

  After this patch:

  ```
  $ src/bitcoind -proxy
  Error: No proxy server specified. Use -proxy=<ip> or -proxy=<ip:port>.
  $ echo $?
  1
  ```

ACKs for top commit:
  laanwj:
    re-ACK 9b4fa0af40
  kristapsk:
    ACK 9b4fa0af40, I have tested the code.
  hebasto:
    re-ACK 9b4fa0af40

Tree-SHA512: 4ba7a011991699a54b5bb87ec68367c681231bf5dcd36f8c89ff9ddc2e8d29df453817b7e362597e652ad6b341a22b7274be0fd78d435e5f0fd8058e5221c4ce
2020-09-29 15:17:28 +02:00
MarcoFalke
fac966142e signet: Add assumed values for default signet 2020-09-29 15:01:18 +02:00
Aaron Clauson
712f95d332 Update msvc build to use new vcpkg manifest
The vcpkg tool has introduced a proper way to use manifests, https://devblogs.microsoft.com/cppblog/vcpkg-accelerate-your-team-development-environment-with-binary-caching-and-manifests/. This PR replaces the custom text file mechanism with the new manifest approach.

It is planned that vckpg manifests will include the ability to version dependencies in the future. Dependency versions would solve a number of issues that currently require workarounds with the appveyor CI.

Set vcpkg manifest version to 1 to avoid any perception it's related to any release or other version numbering.
2020-09-29 13:48:28 +01:00
fanquake
d82b2c6e65 Merge #19898: log: print unexpected version warning in validation log category
62dba9628d log: print unexpected version warning in validation log category (nthumann)

Pull request description:

  Fixes #19603: As suggested by practicalswift, instead of always printing `<n> of the last 100 blocks have unexpected version` as a warning appended to UpdateTip, it is now printed in the validation log category and therefore only visible with `-debug=validation` enabled.

  Before:
  `2020-09-06T15:56:00Z UpdateTip: new best=00000000000000000001b2872e107a98b57913120e5c6c87ce2715a34c40adf8 height=646969 version=0x20400000 log2_work=92.261571 tx=565651941 date='2020-09-06T10:35:36Z' progress=0.999888 cache=32.2MiB(237417txo) warning='72 of last 100 blocks have unexpected version'`
  After:
  `2020-09-06T16:31:26Z UpdateTip: new best=0000000000000000000b3bd786dc42745dd7be4a8c695500a04518cb9e2f4dc1 height=646971 version=0x20000000 log2_work=92.261607 tx=565655901 date='2020-09-06T10:57:19Z' progress=0.999883 cache=3.8MiB(27550txo)`
  `2020-09-06T16:31:26Z 71 of last 100 blocks have unexpected version`

  Ran unit & functional tests, confirmed that the warning is now only printed when validation category is enabled.

ACKs for top commit:
  theStack:
    ACK 62dba9628d
  MarcoFalke:
    re-ACK 62dba96
  practicalswift:
    ACK 62dba9628d -- only change since last ACK is `s/nUpgraded/num_unexpected_version/`
  hebasto:
    re-ACK 62dba9628d, https://github.com/bitcoin/bitcoin/pull/19898#pullrequestreview-483158708 is resolved now.

Tree-SHA512: 2100ca7d6d3fd67c92e81d75162d2506d6f1ecf1761d5180d76663fac06771b35e5c4235ebe1a00731b5f7db82db3cd19328627929c8f22912df592686ba51d3
2020-09-29 20:41:11 +08:00
fanquake
ec9b4492eb Merge #19630: Cleanup fee estimation code
a3abeec33a policy/fees: remove a floating-point division by zero (Antoine Poinsot)
c36869bbf6 policy/fees: unify some duplicated for loops (Antoine Poinsot)
569d92a4d2 policy/fees: small readability improvements (Antoine Poinsot)
5b8cb35621 policy/fee: remove requireGreater parameter in EstimateMedianVal() (Antoine Poinsot)
dba8196b44 policy/fees: correct decay explanation comments (Antoine Poinsot)

Pull request description:

  This (*does not* change behaviour and) cleans up a bit of unused code in `CBlockPolicyEstimator` and friends, and slightly improves readability of the rest (comment correction etc.). The last commit is a small reformatting one which I could not resist but am happy to remove at will.

ACKs for top commit:
  jnewbery:
    utACK a3abeec33a
  MarcoFalke:
    ACK a3abeec33a 💹
  ariard:
    Code Review ACK a3abeec.

Tree-SHA512: b7620bcd23a2ffa8f7ed859467868fc0f6488279e3ee634f6d408872cb866ad086a037e8ace76599a05b7e9c07768adf5016b0ae782d153196b9c030db4c34a5
2020-09-29 16:35:01 +08:00
Russell Yanofsky
c1585bca8d test: Get rid of default wallet hacks
- Get rid of hardcoded wallet "" names and -wallet="" args
- Get rid of setup_nodes (-wallet, -nowallet, -disablewallet) argument rewriting

Motivation:

- Simplify test framework behavior so it's easier to write new tests without
  having arguments mangled by the framework
- Make tests more readable, replacing unexplained "" string literals with
  default_wallet_name references
- Make it trivial to update default wallet name and wallet data filename for
  sqlite #19077 testing
- Stop relying on -wallet arguments to create wallets, so it is easy to change
  -wallet option in the future to only load existing wallets not create new
  ones (to avoid accidental wallet creation, and encourage use of wallet
  encryption and descriptor features)
2020-09-29 04:35:01 -04:00
Russell Yanofsky
ed3acda33b test, refactor: add default_wallet_name and wallet_data_filename variables
No changes in behavior
2020-09-29 04:35:01 -04:00
MarcoFalke
fa29b5ae66 test: Add signet witness commitment section parse tests 2020-09-29 10:23:10 +02:00
MarcoFalke
fa23308e9a Remove gArgs global from CreateChainParams to aid testing 2020-09-29 10:20:05 +02:00
fanquake
6af9b31bfc Merge #19107: p2p: Move all header verification into the network layer, extend logging
deb52711a1 Remove header checks out of net_processing (Troy Giorshev)
52d4ae46ab Give V1TransportDeserializer CChainParams& member (Troy Giorshev)
5bceef6b12 Change CMessageHeader Constructor (Troy Giorshev)
1ca20c1af8 Add doxygen comment for ReceiveMsgBytes (Troy Giorshev)
890b1d7c2b Move checksum check from net_processing to net (Troy Giorshev)
2716647ebf Give V1TransportDeserializer an m_node_id member (Troy Giorshev)

Pull request description:

  Inspired by #15206 and #15197, this PR moves all message header verification from the message processing layer and into the network/transport layer.

  In the previous PRs there is a change in behavior, where we would disconnect from peers upon a single failed checksum check.  In various discussions there was concern over whether this was the right choice, and some expressed a desire to see how this would look if it was made to be a pure refactor.

  For more context, see https://bitcoincore.reviews/15206.html#l-81.

  This PR improves the separation between the p2p layers, helping improvements like [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and #18989.

ACKs for top commit:
  ryanofsky:
    Code review ACK deb52711a1 just rebase due to conflict on adjacent line
  jnewbery:
    Code review ACK deb52711a1.

Tree-SHA512: 1a3b7ae883b020cfee1bef968813e04df651ffdad9dd961a826bd80654f2c98676ce7f4721038a1b78d8790e4cebe8060419e3d8affc97ce2b9b4e4b72e6fa9f
2020-09-29 16:14:40 +08:00
fanquake
e36aa351a3 Merge #19969: Send RPC bug fix and touch-ups
f7b331ea85 rpc: add brackets to ConstructTransaction (Sjors Provoost)
d813d26f06 [rpc] send: various touch-ups (Sjors Provoost)
0fc1c685e1 [rpc] send: fix parsing replaceable option (Sjors Provoost)
efc9b85e6f Mark send RPC experimental (Sjors Provoost)

Pull request description:

  Followup based on #16378 nits. It also fixes an argument parsing error (uncaught because the test wasn't sufficiently thorough).

  I marked the RPC as experimental so we can tweak it a bit over the next release cycle.

ACKs for top commit:
  meshcollider:
    utACK f7b331ea85
  fjahr:
    utACK f7b331ea85
  kallewoof:
    ACK f7b331ea85

Tree-SHA512: 82dd8ac76a6558872db3f5249d4d6440469400aaa339153bc627d1ee673a91ecfadecb486bc1939ba87ebbd80e26ff29698e93e358599f3d26fde0e526892afe
2020-09-29 15:14:08 +08:00
Hennadii Stepanov
e3f07851f0 refactor: Rename TorController::target to m_tor_control_center
`target` is a proper name for the onion service target address and port.
This change is required for the following commit.
2020-09-29 09:59:52 +03:00
Hennadii Stepanov
fdd3ae4d26 net, refactor: Refactor CBaseChainParams::RPCPort function 2020-09-29 09:59:52 +03:00
Hennadii Stepanov
a5266d4546 net: Add alternative port for onion service
This change allows to distinguish incoming Tor connections from others.
2020-09-29 09:59:51 +03:00
Hennadii Stepanov
b3273cf403 net: Use network byte order for in_addr.s_addr
It is documented in the ip(7) manual page.
2020-09-29 09:59:43 +03:00
fanquake
7ea649946d Merge #20028: test: Check that invalid peer traffic is accounted for
faa94cb167 test: Check that invalid peer traffic is accounted for (MarcoFalke)
fae243f0cb test: Remove confusing cast to same type (int to int) (MarcoFalke)

Pull request description:

  Couldn't find a test for this and it seems something we should test, so I wrote one.

ACKs for top commit:
  vasild:
    ACK faa94cb16
  practicalswift:
    ACK faa94cb167: patch looks correct

Tree-SHA512: efcdd35960045cdfbd14480e16e0d1d09e81eb01670ac541ac2b105e1a63818a157c95853270242234a224880873e79957832bf4231374d7fe81de30f35e3abf
2020-09-29 14:32:33 +08:00
fanquake
5db44c740e Merge #20014: doc: Mention signet in -help output
b3972bca9f doc: Mention signet in -help output (Hennadii Stepanov)

Pull request description:

  ```
  $ src/bitcoind -help | grep -A 4 -e '-chain=' | head -8
    -chain=<chain>
         Use the chain <chain> (default: main). Allowed values: main, test,
         signet, regtest

    -signet
         Use the signet chain. Equivalent to -chain=signet. Note that the network
         is defined by the -signetchallenge parameter

  ```

  ```
  $ src/bitcoind -help | grep -A 3 -e '-port='
    -port=<port>
         Listen for connections on <port> (default: 8333, testnet: 18333 signet:
         38333, regtest: 18444)

  ```

  ```
  $ src/bitcoind -help | grep -A 3 -e '-rpcport='
    -rpcport=<port>
         Listen for JSON-RPC connections on <port> (default: 8332, testnet:
         18332, signet: 38332, regtest: 18443)

  ```

ACKs for top commit:
  jonatack:
    ACK b3972bca9f
  kallewoof:
    ACK b3972bca9f
  ajtowns:
    ACK b3972bca9f - skimmed code only, looks fine to me

Tree-SHA512: 66c59cdc3c19e8f8a02d3f3f992ff1db06769df63244d4af62629e18aaf4a12b3b7e75e4a0b9f616033cdc4415da046053ba36fede8be145b2dc695b2aa69a02
2020-09-29 14:04:25 +08:00
Sebastian Falbesoner
343dc4760f test: add test for high-bandwidth mode states in getpeerinfo 2020-09-29 00:42:06 +02:00
Sebastian Falbesoner
dab6583307 doc: release note for new getpeerinfo fields "bip152_hb_{from,to}"
Co-authored-by: Jon Atack <jon@atack.com>
2020-09-29 00:42:06 +02:00
Sebastian Falbesoner
a7ed00f8bb rpc: expose high-bandwidth mode states via getpeerinfo 2020-09-29 00:42:06 +02:00
Sebastian Falbesoner
30bc8fab68 net: save high-bandwidth mode states in CNodeStats 2020-09-29 00:42:06 +02:00
Sishir Giri
69cf5d4eeb [test] Make sure send rpc returns fee reason 2020-09-28 15:04:56 -07:00
Pieter Wuille
655937ebcb Merge #19845: net: CNetAddr: add support to (un)serialize as ADDRv2
7be6ff6187 net: recognize TORv3/I2P/CJDNS networks (Vasil Dimov)
e0d73573a3 net: CNetAddr: add support to (un)serialize as ADDRv2 (Vasil Dimov)
fe42411b4b test: move HasReason so it can be reused (Vasil Dimov)
d2bb681f96 util: move HasPrefix() so it can be reused (Vasil Dimov)

Pull request description:

  (chopped off from #19031 to ease review)

  Add an optional support to serialize/unserialize `CNetAddr` in ADDRv2 format (BIP155). The new serialization is engaged by ORing a flag into the stream version.

  So far this is only used in tests to ensure the new code works as expected.

ACKs for top commit:
  Sjors:
    re-tACK 7be6ff6187
  sipa:
    re-utACK 7be6ff6187
  eriknylund:
    ACK 7be6ff6187 I built the PR on macOS Catalina 10.15.6, ran both tests and functional tests. I've reviewed the code and think the changes look good and according to BIP155. I verified that the added Base32 encoding test looks as proposed and working. I've run a node for a week only with Onion addresses `-onlynet=onion` without issues and I can connect to other peer reviewers running TorV3 on their nodes and I can connect both of my test nodes to each other.
  jonatack:
    re-ACK 7be6ff6187 per `git diff b9c46e0 7be6ff6`, debug build, ran/running bitcoind with this change and observed the log and `-netinfo` peer connections while connected as a tor v2 service to both tor v2 peers and also five tor v3 peers.
  hebasto:
    ACK 7be6ff6187, tested on Linux Mint 20 (x86_64): on top of this pull and #19031 I'm able to connect to onion v3 addresses, and jonatack is able to connect to my created onion v3 address.

Tree-SHA512: dc621411ac4393993aa3ccad10991717ec5f9f2643cae46a24a89802df0a33d6042994fc8ff2f0f397a3dbcd1c0e58fe4724305a2f9eb64d9342c3bdf784d9be
2020-09-28 12:27:08 -07:00
MarcoFalke
2552702000 Merge #15367: feature: Added ability for users to add a startup command
090530cc24 feature: Added ability for users to add a startup command (Ben Carman)

Pull request description:

  Thoughts for adding the feature is for users to be able to add things like electrum-personal-server or lnd to run whenever Bitcoin Core is running.  Open to feedback about the feature.

ACKs for top commit:
  MarcoFalke:
    re-ACK 090530cc24
  dongcarl:
    tACK 090530c

Tree-SHA512: ba514d2fc8b4fb12b781c1a9c89845a25fce0b80ba7c907761cde4abb81edd03fa643682edc895986dc20b273ac3b95769508806db7fbd99ec28623f85c41e67
2020-09-28 20:44:33 +02:00
Ben Carman
090530cc24 feature: Added ability for users to add a startup command 2020-09-28 10:38:36 -05:00
MarcoFalke
faa94cb167 test: Check that invalid peer traffic is accounted for 2020-09-28 10:14:00 +02:00
MarcoFalke
fae243f0cb test: Remove confusing cast to same type (int to int) 2020-09-28 10:13:51 +02:00
MarcoFalke
c95784e3d3 Merge #20016: uint256: 1 is a constant
4cc7171c98 wallet: no need for duplicate storage for ABANDON_HASH constant (Anthony Towns)
82cf4641f4 scripted-diff: Replace UINT256_ONE() with uint256::ONE (Anthony Towns)
183f308fff uint256: Update constructors to c++11, make ONE static (Anthony Towns)

Pull request description:

  `UINT256_ONE()` returns a reference to a global; mark it as const to be sure someone doesn't accidently modify it.

ACKs for top commit:
  promag:
    ACK 4cc7171c98
  MarcoFalke:
    re ACK 4cc7171c98, only change is some constexpr shuffling 🛁
  kallewoof:
    ACK 4cc7171c98

Tree-SHA512: 7f399658bfd9ffa4075bc2349049476d842b9579a67518fb7151f56eab36907ef24b1474ee1e89bdc69fe181abe7295dfe19e33b3623d43cec71fc00e356e347
2020-09-28 09:16:56 +02:00
MarcoFalke
e7e6f0bf18 Merge #20026: test: typo in a functional test doc
a5a6965157 [Trivial] python help message (kanon)

Pull request description:

  before:
  `  --segwit              Test behaviour with SegWit txn (which should fail`
  after:
  `  --segwit              Test behaviour with SegWit txn (which should fail)`

ACKs for top commit:
  kristapsk:
    ACK a5a6965157

Tree-SHA512: b1176bca8ec4d722fca71fe790ca21dcf05bf7b6562f56ef2758a3e6d7a0554c16a08efc5b154446f75b9a09bd0ff1dfcf22e71010c748caeb02b1f5c883f9ca
2020-09-28 08:38:20 +02:00
Anthony Towns
4cc7171c98 wallet: no need for duplicate storage for ABANDON_HASH constant 2020-09-28 12:14:19 +10:00
Anthony Towns
82cf4641f4 scripted-diff: Replace UINT256_ONE() with uint256::ONE
-BEGIN VERIFY SCRIPT-
sed -i '/inline.* UINT256_ONE() {/,+1d' src/uint256.h
sed -i 's/UINT256_ONE()/uint256::ONE/' $(git grep -l UINT256_ONE)
-END VERIFY SCRIPT-
2020-09-28 12:14:19 +10:00
Anthony Towns
183f308fff uint256: Update constructors to c++11, make ONE static
Replace the memset with C++11 value/aggregate initialisation of
the m_data array, which still ensures the unspecified values end
up as zero-initialised.

This then allows changing UINT256_ONE() from dynamically allocating an
object, to a simpler referencing a static allocation.
2020-09-28 12:14:19 +10:00
kanon
a5a6965157 [Trivial] python help message 2020-09-27 17:31:42 -04:00
Sishir Giri
d5863c0b3e [send] Make send RPCs return fee reason 2020-09-26 17:57:26 -07:00
MarcoFalke
055abfbc5a Merge #20023: test: remove unused constants in functional tests
92e28fa8b2 test: remove unused constants in functional tests (Sebastian Falbesoner)

Pull request description:

  This mini-PR gets rid of constants in functional tests that are not used anymore. Found by [vulture ](https://pypi.org/project/vulture/)via the following script that has been lying around here locally for quite some time (I think it was once proposed by practicalswift, but I don't remember the concrete topic/PR):
  ```
  #!/bin/sh
  for F in $(git ls-files -- "*.py"); do vulture "$F" | grep "unused variable"; done
  ```

ACKs for top commit:
  practicalswift:
    ACK 92e28fa8b2: patch looks correct.

Tree-SHA512: 16516abc8014207bcefdf0545dffaecff1fbba66f45b54c02371dcfd1f18194855c6b72598c11b5407009561eafe8048d47af3471f0efb1795d52477d5a0232e
2020-09-26 17:45:59 +02:00
MarcoFalke
4f45ea1f73 Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19 [doc] Release notes (Amiti Uttarwar)
50f94b34a3 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b50 [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca4 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19.
  sipa:
    utACK a512925e19
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19 🌇
  promag:
    Code review ACK a512925e19.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2020-09-26 17:24:54 +02:00
practicalswift
ea93bbeb26 init: Fix incorrect warning "Reducing -maxconnections from N to N-1, because of system limitations" 2020-09-26 14:56:39 +00:00
MarcoFalke
8aa3a4a498 Merge #20022: test: use explicit p2p objects where available
0fcaf73199 test: use explicit p2p objects where available (Oliver Gugger)

Pull request description:

  This is a follow-up patch to #19804 as suggested by MarcoFalke (https://github.com/bitcoin/bitcoin/pull/19804#discussion_r494950062).

  To make the intent of the tests easier to understand, we reference the
  p2p connection objects by their explicit names instead of the p2ps array.

ACKs for top commit:
  theStack:
    ACK 0fcaf73199

Tree-SHA512: 37db22185077beeadfa7245bb768b87d6b7a2cfb906c3c859ab92ec3d657122db7301777f0838e13dfc748f37303850144fb7553e6cb6c66903e304d6e10e659
2020-09-26 12:55:19 +02:00
Sebastian Falbesoner
92e28fa8b2 test: remove unused constants in functional tests 2020-09-26 12:50:10 +02:00
Oliver Gugger
0fcaf73199 test: use explicit p2p objects where available
To make the intent of the tests easier to understand, we reference the
p2p connection objects by their explicit names instead of the p2ps array.
2020-09-26 10:48:54 +02:00
gr0kchain
8a4dcda414 doc: Added default signet config for linearize script 2020-09-25 14:37:22 +02:00
Hennadii Stepanov
b3972bca9f doc: Mention signet in -help output 2020-09-25 15:23:01 +03:00
MarcoFalke
78f912c901 Merge #19804: test/refactor: reference p2p objects explicitly and remove confusing Test_Node.p2p property
10d61505fe [test] remove confusing p2p property (gzhao408)
549d30faf0 scripted-diff: replace p2p with p2ps[0] in p2p_invalid_tx (gzhao408)
7a0de46aea [doc] sample code for test framework p2p objects (gzhao408)
784f757994 [refactor] clarify tests by referencing p2p objects directly (gzhao408)

Pull request description:

  The `TestNode` has a `p2p` property which is an alias for `p2ps[0]`.

  I think this should be removed because it can be confusing and misleading (to both the test writer and reviewer), especially if a TestNode has multiple p2ps connected (which is the case for many tests).
  Another example is when a test has multiple subtests that connect 1 p2p and use the `p2p` property to reference it. If the subtests don't completely clean up after themselves, the subtests may affect one another.

  The best way to refer to a connected p2p is use the object returned by `add_p2p_connection` like this:
  ```py
  p2p_conn = node.add_p2p_connection(P2PInterface())
  ```
  A good example is [p2p_invalid_locator.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_invalid_locator.py), which cleans up after itself (waits in both `wait_for_disconnect` and in `disconnect_p2ps`) but wouldn't need so much complexity if it just referenced the connections directly.

  If there is only one connected, it's not really that tedious to just use `node.p2ps[0]` instead of `node.p2p` (and it can always be aliased inside the test itself).

ACKs for top commit:
  robot-dreams:
    utACK 10d61505fe
  jnewbery:
    utACK 10d61505fe
  guggero:
    Concept ACK 10d61505.

Tree-SHA512: 5965548929794ec660dae03467640cb2156d7d826cefd26d3a126472cbc2494b855c1d26bbb7b412281fbdc92b9798b9765a85c27bc1a97f7798f27f64db6f13
2020-09-25 14:18:31 +02:00
MarcoFalke
faaf9c58e4 remove CRPCCommand constructor that takes rpcfn_type function pointer 2020-09-24 19:53:52 +02:00
MarcoFalke
fa19bb2cd8 remove dead rpc code
Checking for fHelp, or the size of the args, is dead code because:
* fHelp is always false (src/qt/test/rpcnestedtests.cpp)
* It is already implicitly called by RPCHelpMan::Check
  (src/rpc/mining.cpp, src/rpc/misc.cpp, src/rpc/net.cpp)
2020-09-24 19:53:24 +02:00
sanket1729
af57766182 Fix misleading error message: Clean stack rule
Error messages in cleanstack is misleading as
it lets the user believe that there are extra
elements on stack which is incorrect if the
stack is empty.
2020-09-24 12:36:16 -05:00
Gleb Naumenko
2ea62cae48 Improve docs about feeler connections 2020-09-24 08:52:25 +03:00
Hennadii Stepanov
bb6fcc75d1 refactor: Drop boost::thread stuff in CCheckQueue 2020-09-24 06:55:34 +03:00
Hennadii Stepanov
6784ac471b bench: Use CCheckQueue local thread pool 2020-09-24 06:55:34 +03:00
Hennadii Stepanov
dba30695fc test: Use CCheckQueue local thread pool 2020-09-24 06:55:33 +03:00
Hennadii Stepanov
01511776ac Add local thread pool to CCheckQueue 2020-09-24 06:55:33 +03:00
Hennadii Stepanov
0ef938685b refactor: Use member initializers in CCheckQueue 2020-09-24 06:53:39 +03:00
nthumann
a0b2e5cb6a doc: Add release notes to support multiple interfaces 2020-09-23 23:14:29 +02:00
nthumann
b1c3f180ec doc: Adjust ZMQ usage to support multiple interfaces 2020-09-23 23:14:28 +02:00
Nicolas Thumann
347c94f551 zmq: Add support to listen on multiple interfaces 2020-09-23 23:14:28 +02:00
MarcoFalke
1b313cacc9 Merge #19927: validation: Reduce direct g_chainman usage
72a1d5c6f3 validation: Remove review-only comments + assertions (Carl Dong)
3756853b15 docs: Move FindFilesToPrune{,Manual} doxygen comment (Carl Dong)
485899a93c style: Make FindFilesToPrune{,Manual} match style guide (Carl Dong)
3f5b5f3f6d validation: Move FindFilesToPrune{,Manual} to BlockManager (Carl Dong)
f8d4975ab3 validation: Move PruneOneBlockFile to BlockManager (Carl Dong)
74f73c783d validation: Pass in chainman to UnloadBlockIndex (Carl Dong)
4668ded6d6 validation: Move ~CMainCleanup logic to ~BlockManager (Carl Dong)

Pull request description:

  This PR paves the way for de-globalizing `g_chainman` entirely by removing the usage of `g_chainman` in the following functions/methods:
  - `~CMainCleanup`
  - `CChainState::FlushStateToDisk`
  - `UnloadBlockIndex`

  The remaining direct uses of `g_chainman` are as follows:
  1. In initialization codepaths:
  	- `AppTests`
  	- `AppInitMain`
  	- `TestingSetup::TestingSetup`
  2. `::ChainstateActive`
  3. `LookupBlockIndex`
  	- Note: `LookupBlockIndex` is used extensively throughout the codebase and require a much larger set of changes, therefore I've left it out of this initial PR

ACKs for top commit:
  MarcoFalke:
    re-ACK 72a1d5c6f3 👚
  jnewbery:
    utACK 72a1d5c6f3

Tree-SHA512: 944a4fa8405eecf39706ff944375d6824373aaeea849d11473f08181eff26b12f70043a8348a5b08e6e9021b243b481842fbdfbc7c3140ca795fce3688b7f5c3
2020-09-23 20:35:54 +02:00
MarcoFalke
43305e9810 Merge #19868: build: Fix target name
7a89f2e6c5 build: Fix target name (Hennadii Stepanov)

Pull request description:

  It seems like a typo :)
  This PR:
  - fixes errors when building a package in depends for `HOST=x86_64-apple-darwin16` (fix #19799)
  - is a correct alternative to d25e0e308f from #19764

ACKs for top commit:
  icota:
    tACK 7a89f2e6c5
  dongcarl:
    Code Review ACK 7a89f2e6c5
  theuni:
    ACK 7a89f2e6c5.

Tree-SHA512: a0bcbc6805d3450e201476ef1e22e0eb53903db1586c5515314c19afd337bded887e56de0fbe62feaf359b2de15dbccd49a44f1a8b566b4c64f5ae3d94a2ab6d
2020-09-23 20:29:52 +02:00
MarcoFalke
5e14fafb31 Merge #19994: Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet)
fa14f57fbc Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet) (MarcoFalke)

Pull request description:

  This is the last part split out from #18531 to just touch some RPC methods. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    tACK fa14f57fbc
  ryanofsky:
    Code review ACK fa14f57fbc. Just straightforward replacements except code moved in `addnode`, and displatching updated in `bumpfee_helper`

Tree-SHA512: e07af150f1d95a88e558256ce197a6b7dc6cd722a6d6c13c75d944c49c2e2441f8b8237e9f94b03db69fa18f9bda627b0781d5e1da70bf5415e09b38728a8cb1
2020-09-23 20:11:08 +02:00
practicalswift
9b4fa0af40 net: Print error message if -proxy is specified without arguments (instead of continuing without proxy server) 2020-09-23 15:41:49 +00:00
MarcoFalke
8235dca621 Merge #19979: Replace LockAssertion with AssertLockHeld, remove LockAssertion
0bd1184adf Remove unused LockAssertion struct (Hennadii Stepanov)
ab2a44297f Replace LockAssertion with a proper thread safety annotations (Hennadii Stepanov)
73f71e1996 refactor: Use explicit function type instead of template (Hennadii Stepanov)

Pull request description:

  This PR replaces `LockAssertion` with `AssertLockHeld`, and removes `LockAssertion`.

  This PR is compared with alternatives in https://github.com/bitcoin-core/bitcoin-devwiki/wiki/AssertLockHeld-PRs

ACKs for top commit:
  MarcoFalke:
    ACK 0bd1184adf
  ajtowns:
    ACK 0bd1184adf
  vasild:
    ACK 0bd1184ad

Tree-SHA512: ef7780dd689faf0bb479fdb97c49bc652e2dd10c148234bb95502dfbb676442d8565ee37864d923ca21a25f9dc2a335bf46ee82c095e387b59a664ab05c0ae41
2020-09-23 16:37:07 +02:00
Wladimir J. van der Laan
9e217f5a6f Merge #19572: ZMQ: Create "sequence" notifier, enabling client-side mempool tracking
759d94e70f Update zmq notification documentation and sample consumer (Gregory Sanders)
68c3c7e1bd Add functional tests for zmq sequence topic and mempool sequence logic (Gregory Sanders)
e76fc2b84d Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas (Gregory Sanders)
1b615e61bf zmq test: Actually make reorg occur (Gregory Sanders)

Pull request description:

  This PR creates a new ZMQ notifier that gives a "total hash history" of block (dis)connection, mempool addition/substraction, all in one pipeline. It also exposes a "mempool sequence number" to both this notifier and `getrawmempool` results, which allows the consumer to use the results together without confusion about ordering of results and without excessive `getrawmempool` polling.

  See the functional test `interfaces_zmq.py::test_mempool_sync` which shows the proposed user flow for the client-side tracking of mempool contents and confirmations.

  Inspired by https://github.com/bitcoin/bitcoin/pull/19462#issuecomment-656140421
  Alternative to https://github.com/bitcoin/bitcoin/pull/19462 due to noted deficiencies in current zmq notification streams.

  Also fixes a legacy zmq test that didn't actually trigger a reorg because of identical blocks being generated on each side of the split(oops)

ACKs for top commit:
  laanwj:
    Code review ACK 759d94e70f

Tree-SHA512: 9daf0d7d996190f3a68ff40340a687519323d7a6c51dcb26be457fbc013217ea7b62fbd0700b74b654433d2e370704feb61e5584399290692464fcfcb72ce3b7
2020-09-23 13:55:24 +02:00
MarcoFalke
8219893825 Merge #19993: refactor: Signet fixups
facaf9e61f doc: Document signet BIP (MarcoFalke)
faf0a26711 doc: Update comments for new chain settings (-signet and -chain=signet) (MarcoFalke)
fae0548686 fuzz: Remove needless guard (MarcoFalke)
77771a03df refactor: Remove SignetTxs::m_valid and use optional instead (MarcoFalke)
fa2ad5dae1 test: Run signet test even when wallet was not compiled (MarcoFalke)

Pull request description:

  Some doc and test fixups for #18267

ACKs for top commit:
  ajtowns:
    ACK facaf9e61f -- code review only
  dr-orlovsky:
    Reviewed & ACK facaf9e61f
  kallewoof:
    Code review ACK facaf9e61f

Tree-SHA512: 8085027c488d84bb4bddccba78bd2d4c5af0d8e2644ee72265f1f30fa8c83f61a961d9da2c796f2940e69682291cbee7b1028b6a6ce123ad9134c0ebbf4723b0
2020-09-23 09:02:00 +02:00
Troy Giorshev
deb52711a1 Remove header checks out of net_processing
This moves header size and netmagic checking out of net_processing and
into net.  This check now runs in ReadHeader, so that net can exit early
out of receiving bytes from the peer.  IsValid is now slimmed down, so
it no longer needs a MessageStartChars& parameter.

Additionally this removes the rest of the m_valid_* members from
CNetMessage.
2020-09-22 22:05:18 -04:00
Troy Giorshev
52d4ae46ab Give V1TransportDeserializer CChainParams& member
This adds a CChainParams& member to V1TransportDeserializer member, and
use it in place of many Params() calls.  In addition to reducing the
number of calls to a global, this removes a parameter from GetMessage
(and will later allow us to remove one from CMessageHeader::IsValid())
2020-09-22 22:01:14 -04:00
Troy Giorshev
5bceef6b12 Change CMessageHeader Constructor
This commit removes the single-parameter contructor of CMessageHeader
and replaces it with a default constructor.

The single parameter contructor isn't used anywhere except for tests.
There is no reason to initialize a CMessageHeader with a particular
messagestart.  This messagestart should always be replaced when
deserializing an actual message header so that we can run checks on it.

The default constructor initializes it to zero, just like the command
and checksum.

This also removes a parameter of a V1TransportDeserializer constructor,
as it was only used for this purpose.
2020-09-22 22:01:14 -04:00
Troy Giorshev
1ca20c1af8 Add doxygen comment for ReceiveMsgBytes 2020-09-22 22:01:14 -04:00
Troy Giorshev
890b1d7c2b Move checksum check from net_processing to net
This removes the m_valid_checksum member from CNetMessage.  Instead,
GetMessage() returns an Optional.

Additionally, GetMessage() has been given an out parameter to be used to
hold error information.  For now it is specifically a uint32_t used to
hold the raw size of the corrupt message.

The checksum check is now done in GetMessage.
2020-09-22 22:01:14 -04:00
Troy Giorshev
2716647ebf Give V1TransportDeserializer an m_node_id member
This is intended to only be used for logging.

This will allow log messages in the following commits to keep recording
the peer's ID, even when logging is moved into V1TransportDeserializer.
2020-09-22 22:01:14 -04:00
MarcoFalke
b1291b2e8f Merge #19963: Clarify blocksonly whitelistforcerelay test
e15344889a Clarify blocksonly whitelistforcerelay test (t-bast)

Pull request description:

  As discussed in https://github.com/bitcoin/bitcoin/issues/19943, this test may be a bit misleading to newcomers.

  We underscore the fact that our peer needs to run a modified version of Bitcoin Core to actually relay transactions to a `blocksonly` node and benefit from the `whitelistforcerelay` parameter.

ACKs for top commit:
  naumenkogs:
    ACK e15344889a

Tree-SHA512: cc3526ac26c40a2d878b0ad863008663040683fd21092fcdc93866c2b0a79db8c2d29767d1f70bf56195092fca2aa2961cdbee52b5f0b1ae757cece9cd206301
2020-09-22 22:45:06 +02:00
MarcoFalke
c7eb85d005 Merge #19959: build: patch qt libpng to fix powerpc build
f07fb5a55e build: patch qt libpng to fix powerpc build (fanquake)

Pull request description:

  This is an alternative to #19751 that fixes the build without requiring
  splitting out libpng. This patch can be dropped once we are building qt
  5.12.0 or later.

  One of the [concerns posted in #19751](https://github.com/bitcoin/bitcoin/pull/19751#issuecomment-675455993) was:

  > the one bundled with Qt is at best a slower "bare minimum".

  However for our usage, I don't think the performance of libpng is a concern. If it is, I'd like to see some numbers/justification as to why we should be worried about it.

  This patch should be enough to unblock PowerPC binaries (combined with other changes) for 0.21.0, and for 0.22.0, when we switch to qt 5.15.x in depends, we should be able to drop it.

  Related upstream issue: https://bugreports.qt.io/browse/QTBUG-66388.

ACKs for top commit:
  laanwj:
    ACK f07fb5a55e
  theuni:
    ACK f07fb5a55e.
  hebasto:
    ACK f07fb5a55e, the patch is the same as https://codereview.qt-project.org/gitweb?p=qt/qtbase.git;a=blobdiff;f=src/3rdparty/libpng/libpng.pro;h=a2f56669b47e09629b7a88a0964091404d6a9b06;hp=577b61d833f7842f3f0d6b1c94a3b3920006695c;hb=dc2aead842f4cdf74f9259d3606c53c8bdae2c6b;hpb=ceeecbae510af6e2d1ebbf865761e4761d404033

Tree-SHA512: 865b843f5049eca80215774274fb7ae0dacccc3dd7f4a2eec93a73057115dcea85e715f919f96441424f9dd902dd97f0a238d96d4074babcee66b30577104009
2020-09-22 22:36:06 +02:00
MarcoFalke
facaf9e61f doc: Document signet BIP 2020-09-22 22:33:09 +02:00
MarcoFalke
faf0a26711 doc: Update comments for new chain settings (-signet and -chain=signet) 2020-09-22 22:32:25 +02:00
MarcoFalke
fae0548686 fuzz: Remove needless guard 2020-09-22 22:32:18 +02:00
MarcoFalke
77771a03df refactor: Remove SignetTxs::m_valid and use optional instead
m_valid implies the block solution has been checked, which is not the
case. It only means the txs could be parsed. C++17 comes with
std::optional, so just use that instead.
2020-09-22 22:31:31 +02:00
MarcoFalke
fa2ad5dae1 test: Run signet test even when wallet was not compiled 2020-09-22 22:28:36 +02:00
MarcoFalke
fa14f57fbc Assert that RPCArg names are equal to CRPCCommand ones (net, rpcwallet) 2020-09-22 20:49:30 +02:00
Gregory Sanders
759d94e70f Update zmq notification documentation and sample consumer 2020-09-22 11:34:30 -04:00
Gregory Sanders
68c3c7e1bd Add functional tests for zmq sequence topic and mempool sequence logic 2020-09-22 11:34:30 -04:00
Gregory Sanders
e76fc2b84d Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas
Using the zmq notifications to avoid excessive mempool polling can be difficult
given the current notifications available. It announces all transactions
being added to mempool or included in blocks, but announces no evictions
and gives no indication if the transaction is in the mempool or a block.

Block notifications for zmq are also substandard, in that it only announces
block tips, while all block transactions are still announced.

This commit adds a unified stream which can be used to closely track mempool:

1) getrawmempool to fill out mempool knowledge
2) if txhash is announced, add or remove from set
based on add/remove flag
3) if blockhash is announced, get block txn list,
remove from those transactions local view of mempool
4) if we drop a sequence number, go to (1)

The mempool sequence number starts at the value 1, and
increments each time a transaction enters the mempool,
or is evicted from the mempool for any reason, including
block inclusion. The mempool sequence number is published
via ZMQ for any transaction-related notification.

These features allow for ZMQ/RPC consumer to track mempool
state in a more exacting way, without unnecesarily polling
getrawmempool. See interface_zmq.py::test_mempool_sync for
example usage.
2020-09-22 11:34:30 -04:00
Gregory Sanders
1b615e61bf zmq test: Actually make reorg occur 2020-09-22 11:17:50 -04:00
MarcoFalke
d692d192cd Merge #19849: Assert that RPCArg names are equal to CRPCCommand ones (blockchain,rawtransaction)
fa6bb0ce5d Assert that RPCArg names are equal to CRPCCommand ones (rawtransaction) (MarcoFalke)
fa80c81487 Assert that RPCArg names are equal to CRPCCommand ones (blockchain) (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch some RPC methods. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    utACK fa6bb0ce5d
  tryphe:
    utACK fa6bb0ce5d. Reducing data duplication is nice. Code changes are minimal and concise.

Tree-SHA512: deb0edc3f999baf055526eaa199b98c500635e12502dece7aa3cad5319db330eb5ee7459a5c8f040a83671a7f20c560c19a2026fb76c8416f138aa332727cbce
2020-09-22 17:08:08 +02:00
Zero
819d03b932 refactor: took out unused member functions
Took out the following unused member functions:

- 'DBHeightKey()'
- 'GetType()'
- 'operator='
2020-09-22 14:41:15 +01:00
Amiti Uttarwar
a512925e19 [doc] Release notes 2020-09-21 19:03:40 -07:00
Amiti Uttarwar
50f94b34a3 [rpc] Deprecate getpeerinfo addnode field
This field is now redundant since the connection type field will indicate
MANUAL for addnode connections.
2020-09-21 19:01:29 -07:00
Amiti Uttarwar
df091b9b50 [refactor] Rename test file to allow any getpeerinfo deprecations.
Simple rename/restructure to allow for upcoming test additions.
2020-09-21 19:01:29 -07:00
Amiti Uttarwar
395acfa83a [rpc] Add connection type to getpeerinfo RPC, update tests 2020-09-21 19:01:29 -07:00
Amiti Uttarwar
49c10a9ca4 [log] Add connection type to log statement
In addition to adding more specificity to the log statement about the type of
connection, this change also consolidates two statements into one. Previously,
the second one should have never been hit, since block-relay connections would
match the "!IsInboundConn()" condition and return early.
2020-09-21 19:01:29 -07:00
Wladimir J. van der Laan
77376034d4 Merge #17785: p2p: Unify Send and Receive protocol versions
ddefb5c0b7 p2p: Use the greatest common version in peer logic (Hennadii Stepanov)
e084d45562 p2p: Remove SetCommonVersion() from VERACK handler (Hennadii Stepanov)
8d2026796a refactor: Rename local variable nSendVersion (Hennadii Stepanov)
e9a6d8b13b p2p: Unify Send and Receive protocol versions (Hennadii Stepanov)

Pull request description:

  On master (6fef85bfa3) `CNode` has two members to keep protocol version:
  - `nRecvVersion` for received messages
  - `nSendVersion` for messages to send

  After exchanging with `VERSION` and `VERACK` messages via protocol version `INIT_PROTO_VERSION`, both nodes set `nRecvVersion` _and_ `nSendVersion` to _the same_ value which is the greatest common protocol version.

  This PR:
  - replaces two `CNode` members, `nRecvVersion` `nSendVersion`, with `m_greatest_common_version`
  - removes duplicated getter and setter

  There is no change in behavior on the P2P network.

ACKs for top commit:
  jnewbery:
    ACK ddefb5c0b7
  naumenkogs:
    ACK ddefb5c0b7
  fjahr:
    Code review ACK ddefb5c0b7
  amitiuttarwar:
    code review but untested ACK ddefb5c0b7
  benthecarman:
    utACK `ddefb5c`

Tree-SHA512: 5305538dbaa5426b923b0afd20bdef4f248d310855d1d78427210c00716c67b7cb691515c421716b6157913e453076e293b10ff5fd2cd26a8e5375d42da7809d
2020-09-22 00:14:32 +02:00
Wladimir J. van der Laan
8c5f68118c Merge #18267: BIP-325: Signet [consensus]
8258c4c007 test: some sanity checks for consensus logic (Anthony Towns)
e47ad375bf test: basic signet tests (Karl-Johan Alm)
4c189abdc4 test: add small signet fuzzer (practicalswift)
ec9b25d046 test: signet network selection tests (Karl-Johan Alm)
3efe298dcc signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bca4e qt: update QT to support signet network (Karl-Johan Alm)
a8de47a1c9 consensus: add signet validation (Karl-Johan Alm)
e8990f1214 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b7cd add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7dad validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of #16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c007 per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c007
  MarcoFalke:
    Approach ACK 8258c4c007 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
2020-09-21 22:33:00 +02:00
Wladimir J. van der Laan
c0c409dcd3 Merge #19697: Improvements on ADDR caching
0d04784af1 Refactor the functional test (Gleb Naumenko)
83ad65f31b Address nits in ADDR caching (Gleb Naumenko)
81b00f8780 Add indexing ADDR cache by local socket addr (Gleb Naumenko)
42ec558542 Justify the choice of ADDR cache lifetime (Gleb Naumenko)

Pull request description:

  This is a follow-up on #18991 which does 3 things:
  - improves privacy of a node listening to multiple addresses via adding cache index by local socket addr (suggested [here](https://github.com/bitcoin/bitcoin/pull/18991#issuecomment-668219345))
  - documents on the choice of 24h cache lifetime
  - addresses nits from #18991

ACKs for top commit:
  jnewbery:
    utACK 0d04784af1
  vasild:
    ACK 0d04784
  jonatack:
    Code review ACK 0d04784

Tree-SHA512: bb65a34dd1ce2811186d3e4469bc33e8399cebaaa494ce13041c7cff23275870e4176a719f7a72f8d779c49f8b2344bf4fa1aeb3ea4e2626d5ae76514f00a750
2020-09-21 19:36:57 +02:00
Carl Dong
72a1d5c6f3 validation: Remove review-only comments + assertions
[META] This is a followup to "validation: Move FindFilesToPrune{,Manual}
       to BlockManager" removing comments and assertions meant only to
       show that the change is correct.
2020-09-21 13:30:27 -04:00
Carl Dong
3756853b15 docs: Move FindFilesToPrune{,Manual} doxygen comment
[META] This is a pure comment commit.

They belong in the member declarations in the header file.
2020-09-21 13:30:21 -04:00
Carl Dong
485899a93c style: Make FindFilesToPrune{,Manual} match style guide
[META] This is a pure style commit.
2020-09-21 13:28:08 -04:00
Carl Dong
3f5b5f3f6d validation: Move FindFilesToPrune{,Manual} to BlockManager
[META] No behaviour change is intended in this commit.
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

Also stop FindFilesToPrune{,Manual} from unnecessary reaching for
::ChainActive() by passing in the necessary information.
2020-09-21 13:27:44 -04:00
t-bast
e15344889a Clarify blocksonly whitelistforcerelay test
As discussed in https://github.com/bitcoin/bitcoin/issues/19943, this
test may be a bit misleading to newcomers.

We underscore the fact that our peer needs to run a modified version of
Bitcoin Core to actually relay transactions to a `blocksonly` node and
benefit from the `whitelistforcerelay` parameter.
2020-09-21 10:16:09 +02:00
Vasil Dimov
7be6ff6187 net: recognize TORv3/I2P/CJDNS networks
Recognizing addresses from those networks allows us to accept and gossip
them, even though we don't know how to connect to them (yet).

Co-authored-by: eriknylund <erik@daychanged.com>
2020-09-21 10:13:34 +02:00
practicalswift
f22d6a1142 log: Remove static log message "Initializing chainstate Chainstate [ibd] @ height -1 (null)" 2020-09-20 15:34:42 +00:00
MarcoFalke
b99a1633b2 Merge #19781: test: add parameterized constructor for msg_sendcmpct()
638441928a test: add parameterized constructor for msg_sendcmpct() (Sebastian Falbesoner)

Pull request description:

  While working on the test for #19776 I noticed that creating a `sendcmpct` message is quite cumbersome -- due to the lack of a parameterized constructor, one needs to create an empty (that is, initialized with default values) object and then set the two fields one by one. This PR replaces the default constructor with a parameterized constructor and uses it in the test `p2p_compactblocks.py`, reducing LOC. No need to pollute the namespace with temporary throw-away message objects anymore.

ACKs for top commit:
  guggero:
    Code review ACK 638441928a.
  epson121:
    Code review ACK 638441928a

Tree-SHA512: 3b58d276d714b73abc6cc98d1d52dec5f6026b33f03faaeb7dcbc5d83ac377555179f98b159b2b9ecc8957999c35a1dc082e3c69299c5fde4e35f1bd0587ce9d
2020-09-20 11:13:56 +02:00
MarcoFalke
38fd1bdcd4 Merge #18949: doc: Add CODEOWNERS file to automatically nominate PR reviewers
a06eb03ded doc: Add comments and additional reviewers to CODEOWNERS file (Adam Jonas)
e02da22906 doc: Add CODEOWNERS file (Wladimir J. van der Laan)

Pull request description:

  This PR brings back and builds on #17094. GitHub uses a CODEOWNERS magic file to automatically add tagged contributors to the "Reviewers" list for a PR.

  The goal of this is to make use of GitHub's suggested reviewers feature and not to confer ownership or give veto power to specific people. It would be better if this file could be named CODEREVIEWERS, but alas, that wouldn't work. The idea of a NAGFILE was proposed in [Bitcoin Core Dev meeting in 2018](https://diyhpl.us/wiki/transcripts/bitcoin-core-dev-tech/2018-03-07-priorities/#:~:text=NAGFILE). While this GitHub implementation has some complications, it's a step towards realizing the promise of automating "reviewing begging" and (hopefully) positively impacting the review process as a whole.

  Of secondary value, this file can serve as documentation for who the maintainers are and who it might be smart to check with for certain areas of code/features (i.e., fuzzing, PSBT, and Bech32) -- this is helpful information for new contributors.

  * The first commit is taken from #17094
  * The second commit adds comments and expands the list of reviewers based on the suggestions and comments from that PR
  * ~The third WIP commit~ This commit also uses the doc dir as an example of granular assignments based on lines of codes ~contributed~ written and/or general engagement with the project. (If interested, here is a report for [most lines of code per author for each file](https://gist.github.com/adamjonas/854a46a1918224927b186865baeac411)). The pro of this level of detail is that the best reviewer is more likely to be nominated. The con is that it will create churn as files are renamed, new files are added, or reviewers want to be added or removed.

  Some open questions:
  * How often should this file be changed?
  * What level of history does one need have on the project before being added to this file? When does it make sense to remove a reviewer?
  * These review notifications can [cause a lot of noise](https://github.community/t5/How-to-use-Git-and-GitHub/Team-based-notifications-or-rework-CODEOWNERS-notification/td-p/7811) and automatically subscribes the requested reviewer to the thread. A GitHub Team based approach would allow for adding or removing reviewers without modifying this file; however, this comes along with its [own set of problems](https://bionic.fullstory.com/taming-github-codeowners-with-bots/#problems-with-github-code-owners), including granting [write access](https://github.community/t5/How-to-use-Git-and-GitHub/CODEOWNERS-works-with-users-but-not-teams/td-p/4986#U4991). Other projects [have used bots](https://bionic.fullstory.com/taming-github-codeowners-with-bots/#using-a-github-bot) to sidestep this.

Top commit has no ACKs.

Tree-SHA512: aa674ac62478b8801f48750df869c802070dc83d0fa9ff93596e9d63406129d7fd3c0daeb35d7a1a259554d045c24746a6808878a7b9867c7ed66d251f0c918f
2020-09-20 08:25:48 +02:00
Hennadii Stepanov
0bd1184adf Remove unused LockAssertion struct 2020-09-19 18:02:42 +03:00
Hennadii Stepanov
ab2a44297f Replace LockAssertion with a proper thread safety annotations 2020-09-19 18:02:02 +03:00
Hennadii Stepanov
73f71e1996 refactor: Use explicit function type instead of template 2020-09-19 17:50:58 +03:00
fanquake
831b0ecea9 Merge #13686: ZMQ: Small cleanups in the ZMQ code
6fe2ef2acb scripted-diff: Rename SendMessage to SendZmqMessage. (Daniel Kraft)
a3ffb6ebeb Replace zmqconfig.h by a simple zmqutil. (Daniel Kraft)
7f2ad1b9ac Use std::unique_ptr for CZMQNotifierFactory. (Daniel Kraft)
b93b9d5456 Simplify and fix notifier removal on error. (Daniel Kraft)
e15b1cfc31 Various cleanups in zmqnotificationinterface. (Daniel Kraft)

Pull request description:

  This contains various small code cleanups that make the ZMQ code easier to read and maintain (at least in my opinion).  The only functional change is that a potential memory leak is fixed that would have occured when a notifier is removed from the `notifiers` list after its callback function returned `false` (which is likely not relevant in practice but still a bug).

ACKs for top commit:
  instagibbs:
    utACK 6fe2ef2acb
  hebasto:
    re-ACK 6fe2ef2acb, only the latest commit got a scripted-diff since my [previous](https://github.com/bitcoin/bitcoin/pull/13686#pullrequestreview-487649808) review.

Tree-SHA512: 8206f8713bf3698d7cd4cb235f6657dc1c4dd920f50a8c5f371a559dd17ce5ab6d94d6281165eef860a22fc844a6bb25489ada12c83ebc780efd7ccdc0860f70
2020-09-19 17:13:28 +08:00
fanquake
83b23848f7 Merge #18790: gui: Improve thread naming
ead771bf6f qt: Rename qt-init thread before logging start (Hennadii Stepanov)
ad5f614bf3 qt: Name ClientModel timer QThread (Hennadii Stepanov)
2c7f5d8c2e qt: Name WalletController worker QThread (Hennadii Stepanov)
27dcc37d42 qt: Name RPCConsole executor QThread (Hennadii Stepanov)

Pull request description:

  On **master** (eef90c14ed):
  - thread list from OS:
  ![Screenshot from 2020-04-28 00-25-41](https://user-images.githubusercontent.com/32963518/80425413-3de07100-88ec-11ea-8d7a-79bd9e152395.png)
  - log excerpt:
  ```
  2020-04-27T21:25:26Z [] GUI: initialize : Running initialization in thread
  ...
  2020-04-27T21:26:04Z [] Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
  2020-04-27T21:26:04Z [] Using wallet /home/hebasto/.bitcoin/testnet3/wallets/test2
  2020-04-27T21:26:04Z [] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:26:04Z [] init message: Loading wallet...
  2020-04-27T21:26:04Z [] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:26:04Z [] [test2] Wallet File Version = 169900
  2020-04-27T21:26:04Z [] [test2] Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total. Unknown wallet records: 0
  2020-04-27T21:26:04Z [] [test2] Wallet completed loading in              26ms
  2020-04-27T21:26:04Z [] GUI: TransactionTablePriv::refreshWallet
  2020-04-27T21:26:04Z [] [test2] setKeyPool.size() = 2000
  2020-04-27T21:26:04Z [] [test2] mapWallet.size() = 0
  2020-04-27T21:26:04Z [] [test2] m_address_book.size() = 0
  ```

  With **this PR**:
  - thread list from OS:
  ![Screenshot from 2020-04-28 00-21-40](https://user-images.githubusercontent.com/32963518/80425527-7a13d180-88ec-11ea-8a34-dfc774bb1c75.png)
  - log excerpt:
  ```
  2020-04-27T21:21:25Z [qt-init] GUI: initialize : Running initialization in thread
  ...
  2020-04-27T21:23:08Z [qt-walletctrl] Using BerkeleyDB version Berkeley DB 4.8.30: (April  9, 2010)
  2020-04-27T21:23:08Z [qt-walletctrl] Using wallet /home/hebasto/.bitcoin/testnet3/wallets/test2
  2020-04-27T21:23:08Z [qt-walletctrl] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:23:08Z [qt-walletctrl] init message: Loading wallet...
  2020-04-27T21:23:08Z [qt-walletctrl] BerkeleyEnvironment::Open: LogDir=/home/hebasto/.bitcoin/testnet3/wallets/test2/database ErrorFile=/home/hebasto/.bitcoin/testnet3/wallets/test2/db.log
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Wallet File Version = 169900
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Keys: 2001 plaintext, 0 encrypted, 2001 w/ metadata, 2001 total. Unknown wallet records: 0
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Wallet completed loading in              37ms
  2020-04-27T21:23:08Z [qt-walletctrl] init message: Rescanning...
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Rescanning last 112924 blocks (from block 1609206)...
  2020-04-27T21:23:08Z [qt-walletctrl] [test2] Rescan started from block 000000000000003761c81f7efbd8cebf217f39d353ec1ac59c624ac2dddfc2a8...
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] Rescan completed in           14157ms
  2020-04-27T21:23:22Z [qt-walletctrl] GUI: TransactionTablePriv::refreshWallet
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] setKeyPool.size() = 2000
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] mapWallet.size() = 0
  2020-04-27T21:23:22Z [qt-walletctrl] [test2] m_address_book.size() = 0
  ```

ACKs for top commit:
  Sjors:
    tACK ead771bf6f

Tree-SHA512: a3b2789990414ab23b69236ca36b656a3f026e11e88fb5940ef4fecfc2053df5ed886615afb37f98584f6e19b953209d3884baab057740b2e9eed68661880dd3
2020-09-19 16:28:04 +08:00
fanquake
c30f79d418 Merge #19940: rpc: Return fee and vsize from testmempoolaccept
23c35bf005 [test] add get_vsize util for more programmatic testing (gzhao408)
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept (codeShark149)

Pull request description:

  From #19093 and resolves #19057.

  Difference from #19093: return `vsize` and `fees` object (similar to `getmempoolentry`) when the test accept is successful. Updates release-notes.md.

ACKs for top commit:
  jnewbery:
    utACK 23c35bf005
  fjahr:
    utACK 23c35bf
  instagibbs:
    reACK 23c35bf005

Tree-SHA512: dcb81b7b817a4684e9076bc5d427a6f2d549d2edc66544e718260c4b5f8f1d5ae1d47b754175e9f0c8a3bd8371ce116c2dca0583588d513a7d733d5d614f2b04
2020-09-19 15:04:03 +08:00
fanquake
967be53aee Merge #19971: test: create default wallet in extended tests
a5f5374b43 test: create default wallet in extended tests (Sjors Provoost)

Pull request description:

  This was omitted from #15454

ACKs for top commit:
  ryanofsky:
    Code review ACK a5f5374b43. Just reverted a leftover diff since last review
  gzhao408:
    utACK a5f5374b43

Tree-SHA512: 573e215e3665cd23f58417a7ebf66a73420645450f8bc51a7bbb36dea6bfda838f6131bb4456aea35d9dac57b61741bba704a7df8ed11409c21fb8001ec55588
2020-09-19 14:31:43 +08:00
Sjors Provoost
a5f5374b43 test: create default wallet in extended tests
This was omitted from #15454
2020-09-18 17:54:42 +02:00
Jonas Schnelli
0d9d2a1f7c Only update the updateSmartFeeLabel once in sync 2020-09-18 11:07:53 +02:00
Anthony Towns
8258c4c007 test: some sanity checks for consensus logic 2020-09-18 10:19:43 +09:00
Karl-Johan Alm
e47ad375bf test: basic signet tests 2020-09-18 10:19:43 +09:00
practicalswift
4c189abdc4 test: add small signet fuzzer 2020-09-18 10:19:42 +09:00
Karl-Johan Alm
ec9b25d046 test: signet network selection tests 2020-09-18 10:19:42 +09:00
Karl-Johan Alm
3efe298dcc signet: hard-coded parameters for Signet Global Network VI (2020-09-07)
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-09-18 10:19:42 +09:00
Karl-Johan Alm
c7898bca4e qt: update QT to support signet network 2020-09-18 09:37:57 +09:00
Karl-Johan Alm
a8de47a1c9 consensus: add signet validation 2020-09-18 09:37:57 +09:00
Karl-Johan Alm
e8990f1214 add signet chain and accompanying parameters
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-09-18 09:37:57 +09:00
Samuel Dobson
652c45fdbb Merge #15454: Remove the automatic creation and loading of the default wallet
d26f0648f1 Tell users how to load or create a wallet when no wallet is loaded (Andrew Chow)
1bee1e6269 Do not create default wallet (Andrew Chow)

Pull request description:

  Instead of automatically creating and loading a default wallet, users should instead explicitly create their wallet or load it on start.

  Builds on #19754 which provides the `load_on_startup` behavior for the GUI.

ACKs for top commit:
  jnewbery:
    Manual test and very light code review ACK d26f0648f1
  ryanofsky:
    Code review ACK d26f0648f1. Just suggested changes to first commit (reusing MakeWalletDatabase and adding release notes), no changes to second commit
  jonatack:
    ACK d26f0648f1 light code review, debug build, ran tests, did manual testing with testnet, rebased on master, on linux debian.

Tree-SHA512: 091d785aef64736f7df661c576e815a87f3d029cfa32f3a75ba86fc25795f10b022ab3ae15c5b61a10b8cee16f5650f15cd79cbd6127e5e3ccbef631966d3c30
2020-09-18 12:03:55 +12:00
Vasil Dimov
e0d73573a3 net: CNetAddr: add support to (un)serialize as ADDRv2
Co-authored-by: Carl Dong <contact@carldong.me>
2020-09-17 22:17:17 +02:00
Sjors Provoost
f7b331ea85 rpc: add brackets to ConstructTransaction 2020-09-17 21:10:56 +02:00
Sjors Provoost
d813d26f06 [rpc] send: various touch-ups 2020-09-17 21:10:56 +02:00
Sjors Provoost
0fc1c685e1 [rpc] send: fix parsing replaceable option 2020-09-17 20:59:09 +02:00
Sjors Provoost
efc9b85e6f Mark send RPC experimental 2020-09-17 15:29:52 +02:00
Vasil Dimov
fe42411b4b test: move HasReason so it can be reused
Move the class `HasReason` from `miner_tests.cpp` to
`setup_common.h` so that it can be reused by other tests.
2020-09-17 14:45:17 +02:00
Hennadii Stepanov
7a89f2e6c5 build: Fix target name 2020-09-16 19:58:21 +03:00
Wladimir J. van der Laan
be3af4f310 Merge #19934: tests: Add fuzzing harness for Keccak and SHA3_256
fc7f84a9ca tests: Add fuzzing harness for Keccak and SHA3_256 (practicalswift)

Pull request description:

  Add fuzzing harness for Keccak and SHA3_256.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  laanwj:
    uACK fc7f84a9ca
  elichai:
    utACK :) fc7f84a9ca

Tree-SHA512: 01e1610e1c178d5f42578e2dd5644a4165596db34cf5037d574a5285e0ace4b06dc33ab81a308595246117537fe175294efd4bfc174ffc2e8eac98f0ec9dd3e9
2020-09-16 16:30:45 +02:00
Wladimir J. van der Laan
a518b1c26b Merge #19936: Test: batch rpc with params
e1fdd2963b Test batch rpc with params (Gregory Sanders)

Pull request description:

  Useful as an example and test case.

ACKs for top commit:
  laanwj:
    ACK e1fdd2963b
  theStack:
    ACK e1fdd2963b

Tree-SHA512: 2d2ba8960916342b264a14624857d6dd10005be12efafb3e970b82656f721c8f3700ebc9b8809de1b2f887d482b772043504aeaeebc7f2e1c8203f076a451526
2020-09-16 16:25:00 +02:00
gzhao408
23c35bf005 [test] add get_vsize util for more programmatic testing 2020-09-16 07:19:58 -07:00
codeShark149
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept
Return fee and vsize if tx would pass ATMP.
2020-09-15 18:01:32 -07:00
fanquake
1c4f59728c Merge #19879: [p2p] miscellaneous wtxid followups
a8a64acaf3 [BroadcastTransaction] Remove unsafe move operator (Amiti Uttarwar)
125c038126 [p2p] Remove dead code (Amiti Uttarwar)
fc66d0a65c [p2p] Check for nullptr before dereferencing pointer (Adam Jonas)
cb79b9dbf4 [mempool] Revert unbroadcast set to tracking just txid (Amiti Uttarwar)

Pull request description:

  Addresses some outstanding review comments from #18044

  - reverts unbroadcast txids to a set instead of a map (simpler, communicates intent better, takes less space, no efficiency advantages of map)
  - adds safety around two touchpoints (check for nullptr before dereferencing pointer, remove an inaccurate std::move operator)
  - removes some dead code

  Links to comments on wtxid PR: [1](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r460495254) [2](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r460496023) [3](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r463532611)

  thanks to jnewbery & adamjonas for flagging these ! !

ACKs for top commit:
  sdaftuar:
    utACK a8a64acaf3
  naumenkogs:
    utACK a8a64acaf3
  jnewbery:
    utACK a8a64acaf3

Tree-SHA512: 7be669cb30cc17fb9e06b50e636ef7887c6a27354697987e4e4d38dba4b8f50e175647587430cd9bc3295bec01ce8b1e6639a50a4249d8fff9b1ca1b9ead3277
2020-09-16 06:30:57 +08:00
Carl Dong
f8d4975ab3 validation: Move PruneOneBlockFile to BlockManager
[META] This is a pure refactor commit.

Move PruneBlockFile to BlockManager because:
1. PruneOneBlockFile only acts on BlockManager
2. Eliminates the need for callers (FindFilesToPrune{,Manual}) to have a
   reference to the larger ChainstateManager, just a reference to
   BlockManager is enough. See following commits.
2020-09-15 14:13:44 -04:00
Carl Dong
74f73c783d validation: Pass in chainman to UnloadBlockIndex 2020-09-15 14:11:34 -04:00
fanquake
f07fb5a55e build: patch qt libpng to fix powerpc build
This is an alternative to #19751 that fixes the build without requiring
splitting out libpng. This patch can be dropped once we are building qt
5.12.0 or later.
2020-09-15 21:48:59 +08:00
Wladimir J. van der Laan
62e3eb9888 Merge #19241: help: Generate checkpoint height from chainparams
916d3596c4 help: Generate checkpoint height from chainparams (Luke Dashjr)

Pull request description:

  Not sure if this is worth putting in Core, but might as well until checkpoints are removed entirely.

ACKs for top commit:
  laanwj:
    re-ACK 916d3596c4

Tree-SHA512: d8eb26b570ee730fdd75ca916507134db5f2f68987a911e33544b7f1c9ccfd1c76b9c9db63056971956b6daf16910f17ecfc197481c2f7b0773afdfbf7d381cf
2020-09-15 15:46:08 +02:00
Wladimir J. van der Laan
48a9968e50 Merge #19558: build: split pthread flags out of ldflags and dont use when building libconsensus
fc9278d162 build: AX_PTHREAD serial 27 (fanquake)
15c27c4441 build: split PTHREAD_* flags out of AM_LDFLAGS (fanquake)
68e3e22944 scripted-diff: add FUZZ_SUITE_LDFLAGS_COMMON (fanquake)
afecde8046 build: add PTHREAD_LIBS to LDFLAGS configure output (fanquake)

Pull request description:

  TLDR: Split pthread flags out of ldflags, and stop using them when building libconsensus.

  Building libconsensus on Linux using Clang currently warns. i.e:
  ```bash
  ./autogen.sh
  ./configure --disable-tests --disable-bench --with-utils=no --with-daemon=no --with-gui=no --disable-wallet --with-libs=yes CC=clang CXX=clang++
  make V=1 -j6
  ... -Wl,-z -Wl,relro -Wl,-z -Wl,now   -pthread -Wl,-soname -Wl,libbitcoinconsensus.so.0 -o .libs/libbitcoinconsensus.so.0.0.0
  clang: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
  clang: warning: argument unused during compilation: '-pthread' [-Wunused-command-line-argument]
  ```

  Besides wanting to quiet the warnings, after digging into this it seemed we could clean up how we are passing around the pthread flags. I also learnt a bit more about how libtools builds shared libraries, and that passing `-pthread` on the link line wouldn't be enough to link against pthreads anyways, due to libtools usage of -nostdlib (see [related discussion where we build DLLs](476436b2de/configure.ac (L603))).

  This can be demonstrated with a patch to libconsensus:
  ```patch
  diff --git a/src/script/bitcoinconsensus.cpp b/src/script/bitcoinconsensus.cpp
  index 15e204062..10bf3582f 100644
  --- a/src/script/bitcoinconsensus.cpp
  +++ b/src/script/bitcoinconsensus.cpp
  @@ -10,6 +10,8 @@
   #include <script/interpreter.h>
   #include <version.h>

  +#include <pthread.h>
  +
   namespace {

   /** A class that deserializes a single CTransaction one time. */
  @@ -127,3 +129,10 @@ unsigned int bitcoinconsensus_version()
       // Just use the API version for now
       return BITCOINCONSENSUS_API_VER;
   }
  +
  +void *func_pthread(void *x) { return x; }
  +
  +void f() {
  +	pthread_t t;
  +	pthread_create(&t,0,func_pthread,0);
  +}
  ```

  After building,  you'll find you have a `libbitcoinconsensus.so` using pthread symbols, but which isn't linked against libpthread:
  ```bash
  ldd -r src/.libs/libbitcoinconsensus.so
  	linux-vdso.so.1 (0x00007ffe49378000)
  	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f553cee7000)
  	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f553cda2000)
  	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f553cd88000)
  	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f553cbc5000)
  	/lib64/ld-linux-x86-64.so.2 (0x00007f553d15d000)
  undefined symbol: pthread_create	(src/.libs/libbitcoinconsensus.so)
  ```

  This libtool behaviour has been known about for some time, i.e this [thread from 2005](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460),  describes the same issue. The suggestion from libtool maintainers at the time is to add `-lpthread` to LDFLAGS.

  Also worth noting is that some of the users in those threads were also using the `AX_PTHREADS` macro, same as us, to determine how to compile with/link against pthreads. This macro has [recently been updated](https://git.savannah.gnu.org/gitweb/?p=autoconf-archive.git;a=commitdiff;h=2fb904589643eb6ca6122f834891b58d1d51b347), with reference to this issue. You can compare the output from the version we currently use, to the new version:
  ```bash
  # our ax_pthread macro:
    PTHREAD_CFLAGS = -pthread
    PTHREAD_LIBS  =
    PTHREAD_CC    = gcc / clang

  # the new ax_pthread macro
    PTHREAD_CFLAGS = -pthread
    PTHREAD_LIBS  = -lpthread
    PTHREAD_CC    = gcc / clang
  ```

  Note that as part of this PR I've also added `PTHREAD_LIBS` to the split out flags. Although we weren't using it anywhere previously (and wouldn't have seemed to matter for the most part, given it was likely empty for most builders), the macro assumes it's use. i.e:
  > NOTE: You are assumed to not only compile your program with these flags,
  > but also to link with them as well. For example, you might link with
  > $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS

ACKs for top commit:
  laanwj:
    Code review ACK fc9278d162
  hebasto:
    re-ACK fc9278d162, only rebased and renamed s/`AM_PTHREAD_FLAGS`/`PTHREAD_FLAGS`/ since my [previous](https://github.com/bitcoin/bitcoin/pull/19558#pullrequestreview-473487730) review..
  kallewoof:
    ACK fc9278d162

Tree-SHA512: 7c0a5b0f0de4f54b1d7dce0e69020b341c37a383bb7c715867cc96c648774a557b1ddb42eb1b676f7bb2b822b69795bec14dc6272362d80662a21f10cb80331c
2020-09-15 13:03:02 +02:00
Daniel Kraft
6fe2ef2acb scripted-diff: Rename SendMessage to SendZmqMessage.
Windows headers define SendMessage as a macro, which leads to problems
with the method name "SendMessage".  To circumvent this, we rename the
method to "SendZmqMessage".

-BEGIN VERIFY SCRIPT-
sed -i 's/SendMessage/SendZmqMessage/g' src/zmq/zmqpublishnotifier.*
-END VERIFY SCRIPT-
2020-09-15 09:38:56 +02:00
fanquake
a33651866c Merge #19643: Add -netinfo peer connections dashboard
bf1f913c44 cli -netinfo: display multiple levels of details (Jon Atack)
077b3ac928 cli: change -netinfo optional arg from bool to int (Jon Atack)
4e2f2ddd64 cli: add getpeerinfo last_{block,transaction} to -netinfo (Jon Atack)
644be659ab cli: add -netinfo server version check and error message (Jon Atack)
ce57bf6cc0 cli: create peer connections report sorted by dir, minping (Jon Atack)
f5edd66e5d cli: create vector of Peer structs for peers data (Jon Atack)
3a0ab93e1c cli: add NetType enum struct and NetTypeEnumToString() (Jon Atack)
c227100919 cli: create local addresses, ports, and scores report (Jon Atack)
d3f77b736e cli: create inbound/outbound peer connections report (Jon Atack)
19377b2fd2 cli: start dashboard report with chain and version header (Jon Atack)
a3653c159e cli: tally peer connections by type (Jon Atack)
54799b66b4 cli: add ipv6 and onion address type detection helpers (Jon Atack)
12242b17a5 cli: create initial -netinfo option, NetinfoRequestHandler class (Jon Atack)

Pull request description:

  This PR is inspired by laanwj's python script mentioned in #19405, which it turns out I ended up using every day and extending because I got hooked on using it to monitor Bitcoin peer connections.

  For the full experience, run `./src/bitcoin-cli -netinfo 4`

  On Linux, try it with watch `watch ./src/bitcoin-cli -netinfo 4`

  Help doc
  ```
  $ ./src/bitcoin-cli -help | grep -A3 netinfo
    -netinfo
         Get network peer connection information from the remote server. An
         optional integer argument from 0 to 4 can be passed for different
         peers listings (default: 0).
  ```

ACKs for top commit:
  vasild:
    ACK bf1f913
  0xB10C:
    ACK bf1f913c44
  practicalswift:
    ACK bf1f913c44 -- patch looks correct and is limited to `src/bitcoin-cli.cpp`

Tree-SHA512: b9d18e5cc2ffd2bb9f0295b5ac7609da8a9bbecaf823a26dfa706b5f07d5d1a8343081dad98b16aa9dc8efd8f41bc1a4acdc40259727de622dc7195ccf59c572
2020-09-15 15:01:50 +08:00
fanquake
5d5e335f96 Merge #19903: Update build-openbsd.md with GUI support
d11020019a Add OpenBSD instructions for building the Qt GUI (grubles)

Pull request description:

  Using OpenBSD as a desktop OS is prevalent enough IMO to warrant updating the documentation for building the GUI.

ACKs for top commit:
  fanquake:
    ACK d11020019a - looks fine. Have not tested.

Tree-SHA512: a8078334fdd35438bcf87c3f5eae851c2a1ce961eb48ae50770bf2c556489da86b6ee198fe9fb732dcaddb2e0f2f4f55a3126971aae8f7d4e2e320dbb024e204
2020-09-15 12:05:21 +08:00
Samuel Dobson
ffaac6e614 Merge #16378: The ultimate send RPC
92326d8976 [rpc] add send method (Sjors Provoost)
2c2a1445dc [rpc] add snake case aliases for transaction methods (Sjors Provoost)
1bc8d0fd59 [rpc] walletcreatefundedpsbt: allow inputs to be null (Sjors Provoost)

Pull request description:

  `walletcreatefundedpsbt` has some interesting features that `sendtoaddress` and `sendmany` don't have:
  * manual coin selection
  * outputting a PSBT (it was controversial to add this, see #18201)
  * create a transaction without adding to wallet (which leads to broadcasting, unless `-walletbroadcast=0`)

  At the same time `walletcreatefundedpsbt` can't broadcast a transaction, which is inconvenient for simple use cases.

  This PR introduces a new `send` RPC method which creates a PSBT, signs it if possible and adds it to the wallet by default. If it can't sign all inputs, it outputs a PSBT. If `add_to_wallet` is set to `false` it will return the transaction in both PSBT and hex format.

  Because it uses a PSBT internally, it will much easier to add hardware wallet support to this method (see #16546).

  For `bitcoin-cli` users, it tries to keep the simplest use case easy to use:

  ```sh
  bitcoin-cli -regtest send '{"ADDRESS": 0.1}' 1 sat/b
  ```

  This paves the way for deprecating `sendtoaddress` and `sendmany` though there's no rush. The only missing feature compared to these older methods is adding labels to a destination address.

  Depends on:
  - [x] #16377 (`[rpc] don't automatically append inputs in walletcreatefundedpsbt`)
  - [x] #11413 (`[wallet] [rpc] sendtoaddress/sendmany: Add explicit feerate option`)
  - [x] #18244 (`[rpc] have lockUnspents also lock manually selected coins`)

ACKs for top commit:
  meshcollider:
    Light re-utACK 92326d8976
  achow101:
    ACK 92326d8976 Reviewed code and test, ran tests.
  kallewoof:
    utACK 92326d8976

Tree-SHA512: 7552ef1b193d4c06e381c44932fdb0d54f64383e4c7d6b988f49d059c7d4bba45ce6aa7813e03df86360ad9dad6f3010eb76ee7da480551742d5fd98c2251c0f
2020-09-15 14:49:08 +12:00
Carl Dong
4668ded6d6 validation: Move ~CMainCleanup logic to ~BlockManager
~CMainCleanup:
1. Is vestigial
2. References the g_chainman global (we should minimize g_chainman refs)
3. Only acts on g_chainman.m_blockman
4. Does the same thing as BlockManager::Unload
2020-09-14 10:42:45 -04:00
Antoine Poinsot
a3abeec33a policy/fees: remove a floating-point division by zero
Reported-by: practicalswift <practicalswift@users.noreply.github.com>
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-14 16:23:23 +02:00
Antoine Poinsot
c36869bbf6 policy/fees: unify some duplicated for loops
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-14 15:28:27 +02:00
Antoine Poinsot
569d92a4d2 policy/fees: small readability improvements
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-14 15:26:44 +02:00
Antoine Poinsot
5b8cb35621 policy/fee: remove requireGreater parameter in EstimateMedianVal()
It was always passed as true, and complicates the (already complex)
logic of the function.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-14 15:11:30 +02:00
Antoine Poinsot
dba8196b44 policy/fees: correct decay explanation comments
This was confusing: which one is the good one ? After testing the value
is right but not the comment, so fix it.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-09-14 15:11:25 +02:00
fanquake
fc9278d162 build: AX_PTHREAD serial 27 2020-09-14 16:35:09 +08:00
fanquake
15c27c4441 build: split PTHREAD_* flags out of AM_LDFLAGS
Note that with this change we are no-longer including PTHREAD_* flags
when building libbitcoinconsensus.

Also note that we are including PTHREAD_LIBS in AM_PTHREAD_FLAGS
2020-09-14 16:35:09 +08:00
fanquake
68e3e22944 scripted-diff: add FUZZ_SUITE_LDFLAGS_COMMON
-BEGIN VERIFY SCRIPT-
sed -i -e 's/\$(RELDFLAGS) \$(AM_LDFLAGS) \$(LIBTOOL_APP_LDFLAGS)$/\$(FUZZ_SUITE_LDFLAGS_COMMON)/' src/Makefile.test.include
patch -p1 << "EOF"
--- a/src/Makefile.test.include
+++ b/src/Makefile.test.include
@@ -323,6 +323,8 @@ endif

 if ENABLE_FUZZ

+FUZZ_SUITE_LDFLAGS_COMMON = $(RELDFLAGS) $(AM_LDFLAGS) $(LIBTOOL_APP_LDFLAGS)
+
 test_fuzz_addition_overflow_CPPFLAGS = $(AM_CPPFLAGS) $(BITCOIN_INCLUDES)
 test_fuzz_addition_overflow_CXXFLAGS = $(AM_CXXFLAGS) $(PIE_FLAGS)
 test_fuzz_addition_overflow_LDADD = $(FUZZ_SUITE_LD_COMMON)
EOF
-END VERIFY SCRIPT-
2020-09-14 16:35:00 +08:00
fanquake
06dbbe76dd Merge #19931: Change CSipHasher's count variable to uint8_t
812037cb80 Change CSipHasher's count variable to uint8_t (Pieter Wuille)

Pull request description:

  SipHash technically supports arbitrarily long inputs (at least, I couldn't find a limit in the [paper](https://eprint.iacr.org/2012/351.pdf)), but only the low 8 bits of the length matter. Because of that we should use an unsigned type to track the length (as any signed type could overflow, which is UB). `uint8_t` is sufficient, however.

  Fixes #19930.

ACKs for top commit:
  laanwj:
    anyhow re-ACK 812037cb80
  elichai:
    utACK 812037cb80
  practicalswift:
    ACK 812037cb80
  theStack:
    ACK 812037cb80

Tree-SHA512: 5b1440c9e4591460da198991fb421ad47d2d96def2014e761726ce361aa9575752f2c4085656e7e9badee3660ff005cc76fbd1afe4848faefe4502f3412bd896
2020-09-14 16:30:17 +08:00
fanquake
afecde8046 build: add PTHREAD_LIBS to LDFLAGS configure output
Also moves $PTHREAD_CFLAGS to the CFLAGS.
2020-09-14 16:12:36 +08:00
fanquake
ba4b3fbcf2 Merge #19944: Update secp256k1 subtree (including BIP340 support)
b9c1a76481 Squashed 'src/secp256k1/' changes from 2ed54da18a..8ab24e8dad (Pieter Wuille)

Pull request description:

  This updates our src/secp256k1 subtree to the latest libsecp256k1 upstream version.

  As it adds BIP340 support (see https://github.com/bitcoin-core/secp256k1/pull/558), this is a prerequisite for #17977. In particular, it contains:
  * A few generic library improvements
  * Support for x-only public keys as used by BIP340.
  * Support for "key pair" objects, making signing more efficient by using a precomputed public key.
  * Signing support for BIP340 Schnorr (single-party) signatures.
  * Verification support for BIP340 Schnorr signatures.
  * Support for verifying tweaked x-only keys, as used by BIP341's Taproot construction.

  Things that are not included:
  * MuSig, nor any kind of multisignatures, threshold signatures, ... on top.
  * Batch verification.
  * Support for variable-length messages in BIP340 (which are still being discussed, but won't affect BIP341, or Bitcoin Core).
  * A few more generic improvements that are still in the pipeline, including faster modular inversions.

ACKs for top commit:
  instagibbs:
    ACK 894fb33f4c
  fanquake:
    ACK 894fb33f4c. Any Valgrind concerns will be addressed upstream, see discussion in https://github.com/bitcoin-core/secp256k1/pull/813, and if necessary, can be pulled into our tree prior to the 0.21.0 branch off. They are not a blocker for merging this PR in it's current state.
  benthecarman:
    ACK `894fb33`

Tree-SHA512: 6dc992f4477069b7fbd223316f1be955750923be1479c38adad2312649fdca1f316edb375c42ef9d97cea2407caaef49fb8c93abd6c037fe1a522910cbbc2479
2020-09-14 11:52:24 +08:00
Gregory Sanders
e1fdd2963b Test batch rpc with params 2020-09-13 13:43:03 -04:00
Samuel Dobson
be375b2206 Merge #19919: bugfix: make LoadWallet assigns status always
8b39a87558 bugfix: make LoadWallet assigns status always (Akio Nakamura)

Pull request description:

  In my enviroment, ```test/functional/wallet_multiwallet.py``` failed in line 237 for master( 147d50d63 ).
  It got an expected rpc-error-message, but error code was not (-4) but (-18).

  This is because that although loadwallet() in rpcwallet.cpp assumes LoadWallet() always assign some value to the 'status', but LoadWallet() does not do so in some situation.

  This PR intends to fix above and prevends loadwallet() returns ambiguous error code.

ACKs for top commit:
  hebasto:
    re-ACK 8b39a87558, that is the same as 1728059730abef04f3fa84de0b6e20044be7a9d6.
  ryanofsky:
    Code review ACK 8b39a87558 (same as previous)
  meshcollider:
    utACK 8b39a87558

Tree-SHA512: a75d8240f60325bfdb69a07d392269fec97de743f38fe108371eb63a0aba5d8ce3cc484ecc69e81febf8040f5ab64f3a9450b98f8e07a0c17803784bb6f342bf
2020-09-13 12:04:43 +12:00
Luke Dashjr
d438d609cd QA: Use GBT to get block versions correct 2020-09-12 18:24:26 +00:00
Luke Dashjr
1df2cd1c8f QA: blocktools: Accept block template to create_block 2020-09-12 18:24:26 +00:00
Pieter Wuille
b9c1a76481 Squashed 'src/secp256k1/' changes from 2ed54da18a..8ab24e8dad
8ab24e8dad Merge #558: Add schnorrsig module which implements BIP-340 compliant signatures
f3733c5433 Merge #797: Fix Jacobi benchmarks and other benchmark improvements
cb5524adc5 Add benchmark for secp256k1_ge_set_gej_var
5c6af60ec5 Make jacobi benchmarks vary inputs
d0fdd5f009 Randomize the Z coordinates in bench_internal
c7a3424c5f Rename bench_internal variables
875d68b95f Merge #699: Initialize field elements when resulting in infinity
54caf2e74f Merge #799: Add fallback LE/BE for architectures with known endianness + SHA256 selftest
f431b3f28a valgrind_ctime_test: Add schnorrsig_sign
16ffa9d97c schnorrsig: Add taproot test case
8dfd53ee3f schnorrsig: Add benchmark for sign and verify
4e43520026 schnorrsig: Add BIP-340 compatible signing and verification
7332d2db6b schnorrsig: Add BIP-340 nonce function
7a703fd97d schnorrsig: Init empty experimental module
eabd9bc46a Allow initializing tagged sha256
6fcb5b845d extrakeys: Add keypair_xonly_tweak_add
58254463f9 extrakeys: Add keypair struct with create, pub and pub_xonly
f0010349b8 Separate helper functions for pubkey_create and seckey_tweak_add
910d9c284c extrakeys: Add xonly_pubkey_tweak_add & xonly_pubkey_tweak_add_test
176bfb1110 Separate helper function for ec_pubkey_tweak_add
4cd2ee474d extrakeys: Add xonly_pubkey with serialize, parse and from_pubkey
f49c9896b0 Merge #806: Trivial: Add test logs to gitignore
aabf00c155 Merge #648: Prevent ints from wrapping around in scratch space functions
f5adab16a9 Merge #805: Remove the extremely outdated TODO file.
bceefd6547 Add test logs to gitignore
1c325199d5 Remove the extremely outdated TODO file.
47e6618e11 extrakeys: Init empty experimental module
3e08b02e2a Make the secp256k1_declassify argument constant
8bc6aeffa9 Add SHA256 selftest
670cdd3f8b Merge #798: Check assumptions on integer implementation at compile time
5e5fb28b4a Use additional system macros to figure out endianness
7c068998ba Compile-time check assumptions on integer types
02b6c87b52 Add support for (signed) __int128
979961c506 Merge #787: Use preprocessor macros instead of autoconf to detect endianness
887bd1f8b6 Merge #793: Make scalar/field choice depend on C-detected __int128 availability
0dccf98a21 Use preprocessor macros instead of autoconf to detect endianness
b2c8c42cf1 Merge #795: Avoid linking libcrypto in the valgrind ct test.
57d3a3c64c Avoid linking libcrypto in the valgrind ct test.
79f1f7a4f1 Autodetect __int128 availability on the C side
0d7727f95e Add SECP256K1_FE_STORAGE_CONST_GET to 5x52 field
805082de11 Merge #696: Run a Travis test on s390x (big endian)
39295362cf Test travis s390x (big endian)
6034a04fb1 Merge #778: secp256k1_gej_double_nonzero supports infinity
f60915906d Merge #779: travis: Fix argument quoting for ./configure
9e49a9b255 travis: Fix argument quoting for ./configure
18d36327fd secp256k1_gej_double_nonzero supports infinity
214cb3c321 Merge #772: Improve constant-timeness on PowerPC
40412b1930 Merge #774: tests: Abort if malloc() fails during context cloning tests
2e1b9e0458 tests: Abort if malloc() fails during context cloning tests
67a429f31f Suppress a harmless variable-time optimization by clang in _int_cmov
5b196338f0 Remove redundant "? 1 : 0" after comparisons in scalar code
3e5cfc5c73 Merge #741: Remove unnecessary sign variable from wnaf_const
66bb9320c0 Merge #773: Fix some compile problems on weird/old compilers.
1309c03c45 Fix some compile problems on weird/old compilers.
2309c7dd4a Merge #769: Undef HAVE___INT128 in basic-config.h to fix gen_context compilation
22e578bb11 Undef HAVE___INT128 in basic-config.h to fix gen_context compilation
3f4a5a10e4 Merge #765: remove dead store in ecdsa_signature_parse_der_lax
f00d6575ca remove dead store in ecdsa_signature_parse_der_lax
dbd41db16a Merge #759: Fix uninitialized variables in ecmult_multi test
2e7fc5b537 Fix uninitialized variables in ecmult_multi test
37dba329c6 Remove unnecessary sign variable from wnaf_const
6bb0b77e15 Fix test_constant_wnaf for -1 and add a test for it.
47a7b8382f Clear field elements when writing infinity
61d1ecb028 Added test with additions resulting in infinity
60f7f2de5d Don't assume that ALIGNMENT > 1 in tests
ada6361dec Use ROUND_TO_ALIGN in scratch_create
8ecc6ce50e Add check preventing rounding to alignment from wrapping around in scratch_alloc
4edaf06fb0 Add check preventing integer multiplication wrapping around in scratch_max_allocation

git-subtree-dir: src/secp256k1
git-subtree-split: 8ab24e8dad9d43fc6661842149899e3cc9213b24
2020-09-11 12:44:08 -07:00
Pieter Wuille
894fb33f4c Update src/secp256k1 subtree to upstream libsecp256k1 2020-09-11 12:44:08 -07:00
Wladimir J. van der Laan
f2d9934381 Merge #19922: test: Run rpc_txoutproof.py even with wallet disabled
faf251d854 test: gettxoutproof duplicate txid (João Barbosa)
faf5eb45c4 test: Test empty array in gettxoutproof (MarcoFalke)
fa56e866e8 test: Run rpc_txoutproof.py even with wallet disabled (MarcoFalke)
faba790bd4 test: MiniWallet: Default fee_rate in send_self_transfer, Pass in utxo_to_spend (MarcoFalke)
fa65a11d0c test: bugfix: Actually pick largest utxo (MarcoFalke)

Pull request description:

  Run the consensus test even when the wallet was not compiled. Also:

  * Minor bugfix in MiniWallet
  * Two new test cases (one cherry-picked from #19847)

ACKs for top commit:
  jnewbery:
    utACK faf251d854. Thanks Marco!
  kristapsk:
    ACK faf251d854

Tree-SHA512: a5ab33695c88cfb3c369021d4506069c08ce298e24e891db55159130693ed3817444c72f6aad3f472235aa4597b2c601010af714411c2ec8ad9c2d2e0b00ecbc
2020-09-11 16:16:48 +02:00
Vasil Dimov
d2bb681f96 util: move HasPrefix() so it can be reused
Move the function `HasPrefix()` from `netaddress.cpp` to `util/string.h`
so it can be reused by `CNetAddr` methods (and possibly others).
2020-09-11 13:35:39 +02:00
fanquake
9366a73d69 Merge #19916: build: allow user to specify DIR_FUZZ_SEED_CORPUS for cov_fuzz
fb3bacce69 .gitignore: ignore qa-assets/ folder (eugene)
a9f201439a build: use DIR_FUZZ_SEED_CORPUS if specified for cov_fuzz target (eugene)

Pull request description:

  This PR contains two commits:
  - The cov_fuzz target now uses `DIR_FUZZ_SEED_CORPUS` as the seed directory instead of the hard-coded `qa-assets/fuzz_seed_corpus`. Otherwise, running it requires me to copy the corpus to the bitcoin directory first. In case `DIR_FUZZ_SEED_CORPUS` is not specified, the original default is used.
  - add qa-assets folder to gitignore

  Example usage:
  `make cov_fuzz DIR_FUZZ_SEED_CORPUS=~/workspace/qa-assets/fuzz_seed_corpus`

  It can also just be an environment variable.

ACKs for top commit:
  fanquake:
    ACK fb3bacce69 - looks fine.
  practicalswift:
    ACK fb3bacce69 - patch looks correct

Tree-SHA512: 19ad7c6a2b0c088df14fb71a217d956e66a69eea78f016cd1e914d39c5d7cc196766a637e6c941c6706322663010e7162e85f57e888b8f3b05d0c37d44740847
2020-09-11 14:22:12 +08:00
fanquake
dffefda21d Merge #19870: doc: update PyZMQ install instructions, fix zmq_sub.py file permissions
062e6699c4 script: fix zmq_sub.py file permissions (Jon Atack)
36f8e0cce7 doc: update PyZMQ installation instructions, ZeroMQ link (Jon Atack)

Pull request description:

  Seen while reviewing #19572.

ACKs for top commit:
  theStack:
    ACK 062e6699c4 🧷
  fanquake:
    ACK 062e6699c4

Tree-SHA512: 2210d92385377d066984d0a83882c3ece9f0f41c901b7eb375af9cdb57296f50f227c68193ccf35926073c2b788d58976442791a9fce2fc0f76452804d5cee6a
2020-09-11 13:12:54 +08:00
Pieter Wuille
812037cb80 Change CSipHasher's count variable to uint8_t 2020-09-10 09:04:53 -07:00
practicalswift
fc7f84a9ca tests: Add fuzzing harness for Keccak and SHA3_256 2020-09-10 14:54:30 +00:00
gzhao408
10d61505fe [test] remove confusing p2p property 2020-09-10 07:39:14 -07:00
gzhao408
549d30faf0 scripted-diff: replace p2p with p2ps[0] in p2p_invalid_tx
-BEGIN VERIFY SCRIPT-
sed -i 's/\.p2p\./.p2ps[0]./g' test/functional/p2p_invalid_tx.py
-END VERIFY SCRIPT-
2020-09-10 07:39:01 -07:00
gzhao408
7a0de46aea [doc] sample code for test framework p2p objects 2020-09-10 07:38:28 -07:00
Wladimir J. van der Laan
a47e596486 Merge #19841: Implement Keccak and SHA3_256
ab654c7d58 Unroll Keccak-f implementation (Pieter Wuille)
3f01ddb01b Add SHA3 benchmark (Pieter Wuille)
2ac8bf9583 Implement keccak-f[1600] and SHA3-256 (Pieter Wuille)

Pull request description:

  Add a simple (and initially unoptimized) Keccak/SHA3 implementation based on https://github.com/mjosaarinen/tiny_sha3/blob/master/sha3.c, as one will be needed for TORv3 support (the conversion from BIP155 encoding to .onion notation uses a SHA3-based checksum). In follow-up commits, a benchmark is added, and the Keccakf function is unrolled for a (for me) 4.9x speedup.

  Test vectors are taken from https://csrc.nist.gov/projects/cryptographic-algorithm-validation-program/secure-hashing#sha3vsha3vss.

ACKs for top commit:
  practicalswift:
    ACK ab654c7d58 -- patch looks correct and no sanitizer complaints when doing some basic fuzz testing of the added code (remember: **don't trust: fuzz!**) :)
  laanwj:
    re-ACK ab654c7d58
  vasild:
    ACK ab654c7

Tree-SHA512: 8a91b18c46e8fb178b7ff82046cff626180362337e515b92fbbd771876e795da2ed4e3995eb4849773040287f6e687237f469a90474ac53f521fc12e0f5031d9
2020-09-10 16:37:21 +02:00
gzhao408
784f757994 [refactor] clarify tests by referencing p2p objects directly
Use object returned from add_p2p_connection to refer to
p2ps. Add a test class attribute if it needs to be used across
many methods. Don't use the p2p property.
2020-09-10 07:37:14 -07:00
grubles
d11020019a Add OpenBSD instructions for building the Qt GUI 2020-09-10 16:02:10 +02:00
Antoine Riard
d76925478e [doc] Clarify semantic of peer's m_protect w.r.t to outbound eviction logics
The field m_protect is used to protect from eviction both by bad/lagging
chain and extra outbound peers logics. Outbound block-relay peers are
always excluded from this protection.
2020-09-10 09:51:03 -04:00
Sjors Provoost
92326d8976 [rpc] add send method 2020-09-10 13:44:53 +02:00
Karl-Johan Alm
404682b7cd add signet basic support (signet.cpp)
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-09-10 10:47:40 +09:00
Karl-Johan Alm
a2147d7dad validation: move GetWitnessCommitmentIndex to consensus/validation 2020-09-10 10:47:40 +09:00
nthumann
62dba9628d log: print unexpected version warning in validation log category
Instead of printing "<n> of the last 100 blocks have unexpected version"
as a warning appended to UpdateTip, it is now printed in the validation
log category.
2020-09-09 20:57:06 +02:00
Akio Nakamura
8b39a87558 bugfix: make LoadWallet assigns status always
Although loadwallet() in rpcwallet.cpp assumes LoadWallet() always
assign some value to the 'status', but LoadWallet() does not do so
in some situation.

This fixes above and prevends loadwallet() returns ambiguous error code.
2020-09-10 00:47:31 +09:00
João Barbosa
faf251d854 test: gettxoutproof duplicate txid 2020-09-09 11:27:35 +02:00
MarcoFalke
faf5eb45c4 test: Test empty array in gettxoutproof 2020-09-09 11:27:33 +02:00
MarcoFalke
fa56e866e8 test: Run rpc_txoutproof.py even with wallet disabled 2020-09-09 11:27:28 +02:00
MarcoFalke
faba790bd4 test: MiniWallet: Default fee_rate in send_self_transfer, Pass in utxo_to_spend
Adds two new features to MiniWallet:

* The fee rate is irrelevant sometimes, so just set an arbitrary default
* The utxo to spend needs to be selected manually sometimes
2020-09-09 10:39:05 +02:00
MarcoFalke
fa65a11d0c test: bugfix: Actually pick largest utxo 2020-09-09 10:38:50 +02:00
MarcoFalke
564e1ab0f3 Merge #19800: test: Mockwallet
fa188c9c59 test: Use MiniWalet in p2p_feefilter (MarcoFalke)
fa39c62eb7 test: inline hashToHex (MarcoFalke)

Pull request description:

  This introduces a minimalistic test wallet, which can be used as a drop in replacement for the Bitcoin Core wallet to create dummy transactions with a given fee rate.

ACKs for top commit:
  jnewbery:
    utACK fa188c9c59

Tree-SHA512: 0aad9cb14eea4f0055bd6a47cc8c8f82a16941b152598c3bf1e083aae84cca4ffa23f0b854a362a68be1b917deba1b5ec7c0207b63b0805d747ba9a7d1d82efe
2020-09-09 09:06:22 +02:00
Andrew Chow
d26f0648f1 Tell users how to load or create a wallet when no wallet is loaded 2020-09-08 21:02:53 -04:00
Andrew Chow
1bee1e6269 Do not create default wallet
No longer create a default wallet. The default wallet will still be
loaded if it exists and not other wallets were specified (anywhere,
including settings.json, bitcoin.conf, and command line).

Tests are updated to be started with -wallet= if they need the default
wallet.

Added test to wallet_startup.py testing that no default wallet is
created and that it is loaded if it exists and no other wallets were
specified.
2020-09-08 21:02:53 -04:00
MarcoFalke
4f229d8904 Merge #19914: refactor: Do not pass chain params to CheckForStaleTipAndEvictPeers twice
fa7e407b50 Do not pass chain params to CheckForStaleTipAndEvictPeers twice (MarcoFalke)

Pull request description:

  `PeerManager` already keeps a reference to the chain params as a member variable. No need to pass it in once again as a function parameter.

ACKs for top commit:
  naumenkogs:
    utACK fa7e407b50
  jnewbery:
    code review ACK fa7e407b50
  epson121:
    Code review ACK fa7e407b50

Tree-SHA512: 640c2d8adf9f1d54d0bfbdf81989064be2f5ba4b534d07d42258b372dc130f7b9c3fd087c7d28f0439678d124127f5d6f82f3139b1766f59f5ed661e7ac2a923
2020-09-08 22:13:39 +02:00
Hennadii Stepanov
0e51a35512 refactor: Use Mutex type for some mutexes in CNode class 2020-09-08 11:28:03 +03:00
eugene
fb3bacce69 .gitignore: ignore qa-assets/ folder
This commit ignores the qa-assets/ folder in case a user is running
the cov_fuzz target without DIR_FUZZ_SEED_CORPUS set. In this case,
the qa-assets folder is assumed to live in the bitcoin directory and
should be ignored by git.
2020-09-08 02:47:53 -04:00
eugene
a9f201439a build: use DIR_FUZZ_SEED_CORPUS if specified for cov_fuzz target
This commit allows the user to specify the location of the
fuzz_seed_corpus directory on their machine when running the cov_fuzz
target. If DIR_FUZZ_SEED_CORPUS is specified, then this will be used.
Otherwise, qa-assets/fuzz_seed_corpus is assumed to be in the bitcoin
directory.
2020-09-08 02:45:42 -04:00
MarcoFalke
fa7e407b50 Do not pass chain params to CheckForStaleTipAndEvictPeers twice 2020-09-08 07:55:11 +02:00
John Newbery
001343f4bc ProcessOrphanTx: Move AddToCompactExtraTransactions call into ProcessOrphanTx 2020-09-07 20:12:02 +01:00
John Newbery
4fce726bd1 ProcessOrphanTx: Remove aliases 2020-09-07 20:10:17 +01:00
John Newbery
e07c5d9423 ProcessOrphanTx: Remove outdated commented
Also rename orphan_state to state. Both the comment and the variable
name are leftover from when this logic was part of ProcessMessage().
2020-09-07 20:08:43 +01:00
John Newbery
4763b51bca ProcessOrphanTx: remove useless setMisbehaving set
This starts empty, and is only added to if we're about to
exit the function (so we never read from it).
2020-09-07 20:07:43 +01:00
John Newbery
55c79a9cef ProcessOrphanTx: remove useless done variable
There is a keyword that allows us to break out of loops. Use it.

There's a small change in behaviour here: if we process multiple orphans
that are still orphans, then we'll only call mempool.check() once at the
end, instead of after processing each tx.
2020-09-07 19:57:32 +01:00
John Newbery
6e8dd99ef1 [net processing] Add doxygen comments for orphan data and function 2020-09-07 19:55:53 +01:00
Sjors Provoost
2c2a1445dc [rpc] add snake case aliases for transaction methods 2020-09-07 20:33:16 +02:00
Sjors Provoost
1bc8d0fd59 [rpc] walletcreatefundedpsbt: allow inputs to be null
This is of neglible use here, but it allows new RPC methods to take outputs as their first argument and make inputs optional.
2020-09-07 20:33:16 +02:00
Hennadii Stepanov
ddefb5c0b7 p2p: Use the greatest common version in peer logic 2020-09-07 21:03:55 +03:00
Hennadii Stepanov
e084d45562 p2p: Remove SetCommonVersion() from VERACK handler
SetCommonVersion() is already called from the VERSION message handler.
There is no change in behavior on the P2P network.
2020-09-07 21:03:54 +03:00
Hennadii Stepanov
8d2026796a refactor: Rename local variable nSendVersion 2020-09-07 21:03:54 +03:00
Hennadii Stepanov
e9a6d8b13b p2p: Unify Send and Receive protocol versions
There is no change in behavior on the P2P network.
2020-09-07 21:03:44 +03:00
Hennadii Stepanov
ac7ccd67d7 scripted-diff: Remove unused "What's This" button in dialogs on Windows
-BEGIN VERIFY SCRIPT-
git grep -l 'QDialog(parent)' -- src/qt | xargs sed -i -E 's/QDialog\(parent\)/QDialog\(parent, GUIUtil::dialog_flags\)/g'
-END VERIFY SCRIPT-
2020-09-07 19:09:33 +03:00
MarcoFalke
147d50d63e Merge #19791: [net processing] Move Misbehaving() to PeerManager
bb6a32ce99 [net processing] Move Misbehaving() to PeerManager (John Newbery)
aa114b1c9b [net_processing] Move SendBlockTransactions into PeerManager (John Newbery)
3115e00f75 [net processing] Move MaybePunishPeerForTx to PeerManager (John Newbery)
e662e2d42a [net processing] Move ProcessOrphanTx to PeerManager (John Newbery)
b70cd890e3 [net processing] Move MaybePunishNodeForBlock into PeerManager (John Newbery)
d7778351bf [net processing] Move ProcessHeadersMessage to PeerManager (John Newbery)
64f6162651 [whitespace] tidy up indentation after scripted diff (John Newbery)
58bd369b0d scripted-diff: [net processing] Rename PeerLogicValidation to PeerManager (John Newbery)
2297b26b3c [net_processing] Pass chainparams to PeerLogicValidation constructor (John Newbery)
824bbd1ffb [move only] Collect all private members of PeerLogicValidation together (John Newbery)

Pull request description:

  Continues the work of moving net_processing logic into PeerLogicValidation. See https://github.com/bitcoin/bitcoin/pull/19704 and https://github.com/bitcoin/bitcoin/pull/19607#discussion_r462032894 for motivation.

  This PR also renames `PeerLogicValidation` to `PeerManager` as suggested in https://github.com/bitcoin/bitcoin/pull/10756#pullrequestreview-53892618.

ACKs for top commit:
  MarcoFalke:
    re-ACK bb6a32ce99 only change is rebase due to conflict in struct NodeContext and variable rename 🤸
  hebasto:
    re-ACK bb6a32ce99, only rebased, and added renaming `s/peer_logic/peerman/` into scripted-diff since my [previous](https://github.com/bitcoin/bitcoin/pull/19791#pullrequestreview-483118079) review (verified with `git range-diff`).

Tree-SHA512: a2de4a521688fd25125b401e5575402c52b328a0fa27b3010567008d4f596b960aabbd02b2d81f42658f88f4365443fadb1008150a62fbcea123fb42d85a2c21
2020-09-07 18:09:15 +02:00
Hennadii Stepanov
b6951483ec qt: Add flags to prevent a "What's This" button on Windows OS 2020-09-07 19:08:15 +03:00
Antoine Riard
ac71fe936d [doc] Clarify scope of eviction protection of outbound block-relay peers
Block-relay-only peers were introduced by #15759. According to its
author, it was intented to make them only immune to outbound peer
rotation-based eviction and not from all eviction as modified comment
leans to think of.

Clearly indicate that outbound block-relay peers aren't protected
from eviction by the bad/lagging chain logic.
2020-09-07 10:48:21 -04:00
MarcoFalke
fa188c9c59 test: Use MiniWalet in p2p_feefilter 2020-09-07 15:06:24 +02:00
Hennadii Stepanov
6954156b40 qt: Fix visual quality of text in QR image 2020-09-07 15:49:37 +03:00
John Newbery
bb6a32ce99 [net processing] Move Misbehaving() to PeerManager 2020-09-07 11:16:12 +01:00
John Newbery
aa114b1c9b [net_processing] Move SendBlockTransactions into PeerManager 2020-09-07 11:16:12 +01:00
John Newbery
3115e00f75 [net processing] Move MaybePunishPeerForTx to PeerManager 2020-09-07 11:16:12 +01:00
John Newbery
e662e2d42a [net processing] Move ProcessOrphanTx to PeerManager 2020-09-07 11:16:12 +01:00
John Newbery
b70cd890e3 [net processing] Move MaybePunishNodeForBlock into PeerManager 2020-09-07 11:16:12 +01:00
John Newbery
d7778351bf [net processing] Move ProcessHeadersMessage to PeerManager 2020-09-07 11:16:12 +01:00
John Newbery
64f6162651 [whitespace] tidy up indentation after scripted diff 2020-09-07 11:16:12 +01:00
John Newbery
58bd369b0d scripted-diff: [net processing] Rename PeerLogicValidation to PeerManager
-BEGIN VERIFY SCRIPT-
sed -i 's/PeerLogicValidation/PeerManager/g' $(git grep -l PeerLogicValidation ./src ./test)
sed -i 's/peer_logic/peerman/g' $(git grep -l peer_logic ./src ./test)
-END VERIFY SCRIPT-

PeerLogicValidation was originally net_processing's implementation to
the validation interface. It has since grown to contain much of
net_processing's logic. Therefore rename it to reflect its
responsibilities.

Suggested in
https://github.com/bitcoin/bitcoin/pull/10756#pullrequestreview-53892618.
2020-09-07 11:15:48 +01:00
John Newbery
2297b26b3c [net_processing] Pass chainparams to PeerLogicValidation constructor
Keep a references to chainparams, rather than calling the global
Params() function every time it's needed. This is fine, since
globalChainParams does not get updated once it's been set, and it's
available at the point of constructing the PeerLogicValidation object.
2020-09-07 11:13:58 +01:00
John Newbery
824bbd1ffb [move only] Collect all private members of PeerLogicValidation together
We don't have a project style for ordering class members, but it always
makes sense to have no more than one of each public/protected/private
specifier.

Also move documentation for MaybeDiscourageAndDisconnect to the header.
2020-09-07 11:13:58 +01:00
MarcoFalke
2583966130 Merge #19478: Remove CTxMempool::mapLinks data structure member
296be8f58e Get rid of unused functions CTxMemPool::GetMemPoolChildren, CTxMemPool::GetMemPoolParents (Jeremy Rubin)
46d955d196 Remove mapLinks in favor of entry inlined structs with iterator type erasure (Jeremy Rubin)

Pull request description:

  Currently we have a peculiar data structure in the mempool called maplinks. Maplinks job is to track the in-pool children and parents of each transaction. This PR can be primarily understood and reviewed as a simple refactoring to remove this extra data structure, although it comes with a nice memory and performance improvement for free.

  Maplinks is particularly peculiar because removing it is not as simple as just moving it's inner structure to the owning CTxMempoolEntry. Because TxLinks (the class storing the setEntries for parents and children) store txiters to each entry in the mempool corresponding to the parent or child, it means that the TxLinks type is "aware" of the boost multiindex (mapTx) it's coming from, which is in turn, aware of the entry type stored in mapTx. Thus we used maplinks to store this entry associated data we in an entirely separate data structure just to avoid a circular type reference caused by storing a txiter inside a CTxMempoolEntry.

  It turns out, we can kill this circular reference by making use of iterator_to multiindex function and std::reference_wrapper. This allows us to get rid of the maplinks data structure and move the ownership of the parents/child sets to the entries themselves.

  The benefit of this good all around, for any of the reasons given below the change would be acceptable, and it doesn't make the code harder to reason about or worse in any respect (as far as I can tell, there's no tradeoff).

  ### Simpler ownership model
  No longer having to consistency check that mapLinks did have records for our CTxMempoolEntry, impossible to have a mapLinks entry outlive or incorrectly die before a CTxMempoolEntry.

  ### Memory Usage
  We get rid of a O(Transactions) sized map in the mempool, which is a long lived data structure.

  ### Performance
  If you have a CTxMemPoolEntry, you immediately know the address of it's children/parents, rather than having to do a O(log(Transactions)) lookup via maplinks (which we do very often). We do it in *so many* places that a true benchmark has to look at a full running node, but it is easy enough to show an improvement in this case.

  The ComplexMemPool shows a good coherence check that we see the expected result of it being 12.5% faster / 1.14x faster.
  ```
  Before:
  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 1.40462, 0.277222, 0.285339, 0.279793

  After:
  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 1.22586, 0.243831, 0.247076, 0.244596
  ```
  The ComplexMemPool benchmark only checks doing addUnchecked and TrimToSize for 800 transactions. While this bench does a good job of hammering the relevant types of function, it doesn't test everything.

  Subbing in 5000 transactions shows a that the advantage isn't completely wiped out by other asymptotic factors (this isn't the only bottleneck in growing the mempool), but it's only a bit proportionally slower (10.8%, 1.12x), which adds evidence that this will be a good change for performance minded users.

  ```
  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 59.1321, 11.5919, 12.235, 11.7068

  # Benchmark, evals, iterations, total, min, max, median
  ComplexMemPool, 5, 1, 52.1307, 10.2641, 10.5206, 10.4306
  ```
  I don't think it's possible to come up with an example of where a maplinks based design would have better performance, but it's something for reviewers to consider.

  # Discussion
  ## Why maplinks in the first place?

  I spoke with the author of mapLinks (sdaftuar) a while back, and my recollection from our conversation was that it was implemented because he did not know how to resolve the circular dependency at the time, and there was no other reason for making it a separate map.

  ## Is iterator_to weird?

  iterator_to is expressly for this purpose, see https://www.boost.org/doc/libs/1_51_0/libs/multi_index/doc/tutorial/indices.html#iterator_to

  >  iterator_to provides a way to retrieve an iterator to an element from a pointer to the element, thus making iterators and pointers interchangeable for the purposes of element pointing (not so for traversal) in many situations. This notwithstanding, it is not the aim of iterator_to to promote the usage of pointers as substitutes for real iterators: the latter are specifically designed for handling the elements of a container, and not only benefit from the iterator orientation of container interfaces, but are also capable of exposing many more programming bugs than raw pointers, both at compile and run time. iterator_to is thus meant to be used in scenarios where access via iterators is not suitable or desireable:
  >
  >     - Interoperability with preexisting APIs based on pointers or references.
  >     - Publication of pointer-based interfaces (for instance, when designing a C-compatible library).
  >     - The exposure of pointers in place of iterators can act as a type erasure barrier effectively decoupling the user of the code from the implementation detail of which particular container is being used. Similar techniques, like the famous Pimpl idiom, are used in large projects to reduce dependencies and build times.
  >     - Self-referencing contexts where an element acts upon its owner container and no iterator to itself is available.

  In other words, iterator_to is the perfect tool for the job by the last reason given. Under the hood it should just be a simple pointer cast and have no major runtime overhead (depending on if the function call is inlined).

  Edit by laanwj: removed at sign from the description

ACKs for top commit:
  jonatack:
    re-ACK 296be8f per `git range-diff ab338a19 3ba1665 296be8f`, sanity check gcc 10.2 debug build is clean.
  hebasto:
    re-ACK 296be8f58e, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19478#pullrequestreview-482400727) review (verified with `git range-diff`).

Tree-SHA512: f5c30a4936fcde6ae32a02823c303b3568a747c2681d11f87df88a149f984a6d3b4c81f391859afbeb68864ef7f6a3d8779f74a58e3de701b3d51f78e498682e
2020-09-07 12:06:55 +02:00
Daniel Kraft
a3ffb6ebeb Replace zmqconfig.h by a simple zmqutil.
zmqconfig.h is currently not really needed anywhere, except that
it declares zmqError (which is then defined in
zmqnotificationinterface.cpp).  Note in particular that there is
no need to conditionally include zmq.h only if ZMQ is enabled, because
the place in the core code where the ZMQ library itself is included
(init.cpp) is conditional already on that.

This commit removes zmqconfig.h and replaces it by a much simpler
zmqutil.h library for zmqError.  The definition of the function is
moved to the matching (newly created) zmqutil.cpp.
2020-09-07 10:56:22 +02:00
Daniel Kraft
7f2ad1b9ac Use std::unique_ptr for CZMQNotifierFactory.
Instead of returning a raw pointer from CZMQNotifierFactory and
implicitly requiring the caller to know that it has to take ownership,
return a std::unique_ptr to make this explicit.

This also changes the typedef for CZMQNotifierFactory to use the new
C++11 using syntax, which makes it (a little) less cryptic.
2020-09-07 10:55:55 +02:00
Daniel Kraft
b93b9d5456 Simplify and fix notifier removal on error.
This factors out the common logic to run over all ZMQ notifiers, call a
function on them, and remove them from the list if the function fails is
extracted to a helper method.

Note that this also fixes a potential memory leak:  When a notifier was
removed previously after its callback returned false, it would just be
removed from the list without destructing the object.  This is now done
correctly by std::unique_ptr behind the scenes.
2020-09-07 10:55:54 +02:00
Daniel Kraft
e15b1cfc31 Various cleanups in zmqnotificationinterface.
This is a pure refactoring of zmqnotificationinterface to make the
code easier to read and maintain.  It replaces explicit iterators
with C++11 for-each loops where appropriate and uses std::unique_ptr
to make memory ownership more explicit.
2020-09-07 10:55:06 +02:00
MarcoFalke
07087051af Merge #19556: Remove mempool global
fafb381af8 Remove mempool global (MarcoFalke)
fa0359c5b3 Remove mempool global from p2p (MarcoFalke)
eeee1104d7 Remove mempool global from init (MarcoFalke)

Pull request description:

  This refactor unlocks some nice potential features, such as, but not limited to:
  * Removing the fee estimates global (would avoid slightly fragile workarounds such as #18766)
  * Making the mempool optional for a "blocksonly" operation mode

  Even absent those features, the new code without the global should be easier to maintain, read and write tests for.

ACKs for top commit:
  jnewbery:
    utACK fafb381af8
  hebasto:
    ACK fafb381af8, I have reviewed the code and it looks OK, I agree it can be merged.
  darosior:
    ACK fafb381af8

Tree-SHA512: a2e696dc377e2e81eaf9c389e6d13dde4a48d81f3538df88f4da502d3012dd61078495140ab5a5854f360a06249fe0e1f6a094c4e006d8b5cc2552a946becf26
2020-09-07 09:47:28 +02:00
Samuel Dobson
78cb45d722 Merge #19738: wallet: Avoid multiple BerkeleyBatch in DelAddressBook
abac436760 wallet: Avoid multiple BerkeleyBatch in DelAddressBook (João Barbosa)

Pull request description:

ACKs for top commit:
  achow101:
    ACK abac436760
  jonatack:
    ACK abac436760
  meshcollider:
    re-utACK abac436760

Tree-SHA512: 92309fb74c48694160807326c0fe9793044a75cd77ed19400cceab54a7eefeb54ffc9334535e6021b3af7b9a364dbbeda3a9173540fff8144dfd437e96d76b5c
2020-09-07 15:56:31 +12:00
Pieter Wuille
ab654c7d58 Unroll Keccak-f implementation 2020-09-06 18:35:23 -07:00
Pieter Wuille
3f01ddb01b Add SHA3 benchmark 2020-09-06 18:35:23 -07:00
Pieter Wuille
2ac8bf9583 Implement keccak-f[1600] and SHA3-256 2020-09-06 18:35:18 -07:00
Samuel Dobson
56d47e19ed Merge #19619: Remove wallet.dat path handling from wallet.cpp, rpcwallet.cpp
7bf6dfbb48 wallet: Remove path checking code from bitcoin-wallet tool (Russell Yanofsky)
77d5bb72b8 wallet: Remove path checking code from createwallet RPC (Russell Yanofsky)
a987438e9d wallet: Remove path checking code from loadwallet RPC (Russell Yanofsky)
8b5e7297c0 refactor: Pass wallet database into CWallet::Create (Russell Yanofsky)
3c815cfe54 wallet: Remove Verify and IsLoaded methods (Russell Yanofsky)
0d94e60625 refactor: Use DatabaseStatus and DatabaseOptions types (Russell Yanofsky)
b5b414151a wallet: Add MakeDatabase function (Russell Yanofsky)
288b4ffb6b Remove WalletLocation class (Russell Yanofsky)

Pull request description:

  Get rid of file path handling in wallet application code and move it down to database layer.

  There is no change in behavior except for some changed error messages.

  Motivation for this change is to make code more understandable, but also to prepare for adding SQLite support in #19077 so SQLite implementation can be contained at the database layer and wallet loading code does not need to become more complicated.

ACKs for top commit:
  achow101:
    ACK 7bf6dfbb48
  meshcollider:
    Code re-review and functional test run ACK 7bf6dfbb48

Tree-SHA512: 23ad18324c9e8947f0cf88a3734c2e9fb25536b2cb4d552cf5d1a4ade320fbffb73bb2d1b3a99585c11630aa7092e0fcfc2dd4fe65b91e3a54161433a5cd13cb
2020-09-07 11:45:36 +12:00
Wladimir J. van der Laan
af8135e369 Merge #19897: Change FILE_CHAR_BLOCKLIST to FILE_CHARS_DISALLOWED
637d8bce74 Change FILE_CHAR_BLOCKLIST to FILE_CHARS_DISALLOWED (Benoit Verret)

Pull request description:

  Blocklist is ambiguous. It could mean a list of blocks.

  Example: "blocknotify" in the same file refers to Bitcoin blocks.

ACKs for top commit:
  MarcoFalke:
    ACK 637d8bce74
  laanwj:
    ACK 637d8bce74 — this is a clear variable name improvement
  theStack:
    ACK 637d8bce74
  jonatack:
    ACK 637d8bce74
  eriknylund:
    ACK 637d8bce74
  promag:
    ACK 637d8bce74.

Tree-SHA512: 028e7102eeaf61105736c55c119a7f5c05411f2b6715a7939c41cb9e8f13afb757bbb6e7a302b3aae21722e69dab91f6eff8099e5884d248299905b4c7687c02
2020-09-06 19:35:22 +02:00
Benoit Verret
637d8bce74 Change FILE_CHAR_BLOCKLIST to FILE_CHARS_DISALLOWED
Blocklist is ambiguous. It could mean a list of blocks.

Example: "blocknotify" in the same file refers to Bitcoin blocks.
2020-09-06 12:15:04 -04:00
Wladimir J. van der Laan
68d1f1698f Merge #19890: refactor: remove unused header <arpa/inet.h> in protocol.cpp
2f79e9d002 refactor: remove unused header <arpa/inet.h> in protocol.cpp (Sebastian Falbesoner)

Pull request description:

  There is no code using types or functions related to "internet operations" anymore in protocol.cpp (since #735, more than 8 years ago!), hence the header include can be removed.

ACKs for top commit:
  practicalswift:
    ACK 2f79e9d002 -- patch looks correct and CI is happy
  epson121:
    Code review ACK 2f79e9d002
  laanwj:
    ACK 2f79e9d002
  promag:
    Code review ACK 2f79e9d002.

Tree-SHA512: b3f75fa080125a34ce224f11eb13ec7b914cd9930e3bbed24f550031ce92a7e0830e8ff20159d737ffe487dfd28c24c273ad5e89c6932c8c6960d7fadb6c5e54
2020-09-06 13:17:51 +02:00
MarcoFalke
c91f955f44 Merge #19887: test: Fix flaky wallet_basic test
56b018ca7f test: Fix flaky wallet_basic test (Fabian Jahr)

Pull request description:

  Fixes #19853

  I investigated the issue in #19876 and I still intend to fix the underlying issue of a race when using wallet RPCs right after starting a node in that PR. However, since that is a bit more complicated than I initially thought it makes sense to merge the fix of the test so the intermittent test failures stop. This fix in the test is going to be needed, either way, #19876 will only provide an error where before it was reporting a false balance.

Top commit has no ACKs.

Tree-SHA512: 52bb2388a3e77aa20d26ab0fd45796bc1781483b1cffe49cbb44e2488a72e76998edfb1198495373f9c6fd2ec26064d4176bd1a64dd59806622d5e50a4f4e870
2020-09-06 13:03:43 +02:00
João Barbosa
abac436760 wallet: Avoid multiple BerkeleyBatch in DelAddressBook 2020-09-06 10:59:01 +01:00
MarcoFalke
0368931702 Merge #19881: ci: Double tsan CPU and Memory to avoid global timeout
fa8e148714 ci: Double tsan CPU and Memory to avoid global timeout (MarcoFalke)

Pull request description:

  Fix #19864

ACKs for top commit:
  practicalswift:
    ACK fa8e148714 -- patch looks correct
  hebasto:
    ACK fa8e148714, according to https://cirrus-ci.org/guide/linux/ the limits are:

Tree-SHA512: b6d522290bfe80ed7453387b811628bf42c7657aa6a84d2f5984c8bb16f9857a71eabc6b8a4d63b84227d59b41a8ed7dd85d86cae5628dc9cf6b85bd365248d7
2020-09-06 07:50:00 +02:00
Sebastian Falbesoner
2f79e9d002 refactor: remove unused header <arpa/inet.h> in protocol.cpp 2020-09-06 02:29:30 +02:00
Fabian Jahr
56b018ca7f test: Fix flaky wallet_basic test
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2020-09-05 19:55:59 +02:00
MarcoFalke
fafb381af8 Remove mempool global 2020-09-05 16:24:56 +02:00
MarcoFalke
fa0359c5b3 Remove mempool global from p2p 2020-09-05 16:24:52 +02:00
MarcoFalke
eeee1104d7 Remove mempool global from init
Can be reviewed with the git diff options

--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space --ignore-all-space
2020-09-05 16:24:08 +02:00
MarcoFalke
3ba25e3bdd Merge #19848: Remove mempool global from interfaces
fa9ee52556 doc: Add doxygen comment to IsRBFOptIn (MarcoFalke)
faef4fc9b4 Remove mempool global from interfaces (MarcoFalke)
fa831684e5 refactor: Add IsRBFOptInEmptyMempool (MarcoFalke)

Pull request description:

  The chain interface has an `m_node` member, which has a pointer to the mempool global. Use the pointer instead of the global to prepare the removal of the mempool global. See #19556

ACKs for top commit:
  jnewbery:
    utACK fa9ee52556
  darosior:
    ACK fa9ee52
  hebasto:
    re-ACK fa9ee52556, since my [previous](https://github.com/bitcoin/bitcoin/pull/19848#pullrequestreview-482403942) review:

Tree-SHA512: 11b4c1446f0860a743fdaa67f95c52bf0262d0a4f888be0eaf07ee497448965d32be414111bf016bd568f2989cde923430e3a3889e224057b73c499f06de7199
2020-09-05 15:33:14 +02:00
Wladimir J. van der Laan
416efcb7ab Merge #19728: Increase the ip address relay branching factor for unreachable networks
86d4cf42d9 Increase the ip address relay branching factor for unreachable networks (Pieter Wuille)

Pull request description:

  Onion addresses propagate very badly among the IPv4/IPv6 network, resulting
  in difficulty for those to find each other.

  The branching factor 1 is probably so low that propagations die out before
  they reach another onion peer. Increase it to 1.5 on average.

ACKs for top commit:
  practicalswift:
    ACK 86d4cf42d9 -- patch looks correct
  naumenkogs:
    ACK 86d4cf4
  jonatack:
    ACK 86d4cf42d9. Code review, built and running with some sanity check logging. `RelayAddress()` is called by `ProcessMessage() ADDR` msg handling, from within the loop while processing each new address to relay it to a limited number of other nodes. According to git blame, the line setting `nRelayNodes` hasn't been touched since 2016 in e736772c56 *Move network-msg-processing code out of main to its own file*, which moved the line but otherwise did not change it. Running a mixed clearnet/onion node with this patch and the logging below, I'm only seeing values of `fReachable 1, nRelayNodes 2`. IIUC, I need to use the settings in `init.cpp` that call `SetReachable(*, false)`. *Edit:* with `onlynet=onion` am now seeing entries of `fReachable 0` with `nRelayNodes` values of 1 and 2.
  vasild:
    ACK 86d4cf42d

Tree-SHA512: 22391e16d60bcfdec9a9336728da39d68a24a183b3d1b0e8fbc038d265ca6ddf71d16db018f3678745fd9f3e9281049e42197fa0a29124833c50a9170ed6f793
2020-09-05 14:08:16 +02:00
MarcoFalke
fa8e148714 ci: Double tsan CPU and Memory to avoid global timeout 2020-09-05 14:02:55 +02:00
MarcoFalke
81a19e7253 Merge #19852: refactor: Avoid duplicate map lookup in ScriptToAsmStr
ac2ff4fb1e refactor: Avoid duplicate map lookup in ScriptToAsmStr (João Barbosa)

Pull request description:

  Simple change that avoids a duplicate (unnecessary) `mapSigHashTypes` lookup.

ACKs for top commit:
  laanwj:
    re-ACK ac2ff4fb1e

Tree-SHA512: 7e7f5af51c1acd7a42af273e5ee5e2faddd250ba8b8f63ccb3172d95f153ae391b2816b79564b856571af52dc2a767b5736a5d10ffb5cd2c540cd9832bf86419
2020-09-05 13:43:36 +02:00
MarcoFalke
fa9ee52556 doc: Add doxygen comment to IsRBFOptIn 2020-09-05 11:45:16 +02:00
MarcoFalke
faef4fc9b4 Remove mempool global from interfaces 2020-09-05 11:44:43 +02:00
MarcoFalke
fa831684e5 refactor: Add IsRBFOptInEmptyMempool
Co-authored-by: John Newbery <jonnynewbs@gmail.com>
2020-09-05 11:44:25 +02:00
MarcoFalke
df75e9f3ee Merge #19878: rawtransaction: fix argument in combinerawtransaction help message
4294e70690 rawtransaction: fix argument in combinerawtransaction help message (Matthew Zipkin)

Pull request description:

  Minor correction in the help message provided for `rpc combinerawtransaction`. The input to the rpc is not an array of transaction hashes (txids) but an array of serialized transactions encoded in raw hex.

ACKs for top commit:
  achow101:
    ACK 4294e70690
  darosior:
    ACK 4294e70690

Tree-SHA512: 81fe7707632574030715a09e4fe1ad7c0e2630be7842f20c6656d908bbc9532fc14e71b6d36e3fc261a347a088491ef9f6f38d7c4173c4a0bbc746e1d625359d
2020-09-05 11:19:45 +02:00
Amiti Uttarwar
a8a64acaf3 [BroadcastTransaction] Remove unsafe move operator
Previously, `tx` was being read after having `std::move` called on it. The
std::move operator indicates to the compiler that this object may be "moved
from", so we shouldn't subsequently read from it. The current code is not
problematic since tx is passed in as a const ref. But this `std::move` is at
best misleading & at worst problematic, so remove it.
2020-09-04 14:42:59 -07:00
Amiti Uttarwar
125c038126 [p2p] Remove dead code
The else clause is dead code because the only way to not enter the if branch is
if TX_WITNESS_STRIPPED is true. In that case, it would not have a witness to
match the `tx.HasWitness()` else condition.

Co-authored-by: Adam Jonas <jonas@chaincode.com>
Co-authored-by: John Newbery <john@johnnewbery.com>
2020-09-04 14:42:30 -07:00
Adam Jonas
fc66d0a65c [p2p] Check for nullptr before dereferencing pointer 2020-09-04 14:37:44 -07:00
Amiti Uttarwar
cb79b9dbf4 [mempool] Revert unbroadcast set to tracking just txid
When I originally implemented the unbroadcast set in 18038, it just tracked
txids. After 18038 was merged, I offered a patch to 18044 to make the
unbroadcast changes compatible with wtxid relay. In this patch, I updated
`unbroadcast_txids` to a map of txid -> wtxid. Post merge review comments shed
light on the fact that this update was unnecessary, and distracting. So, this
commit updates the unbroadcast ids back to a set.
2020-09-04 14:29:29 -07:00
Matthew Zipkin
4294e70690 rawtransaction: fix argument in combinerawtransaction help message 2020-09-04 17:21:18 -04:00
Jeremy Rubin
296be8f58e Get rid of unused functions CTxMemPool::GetMemPoolChildren, CTxMemPool::GetMemPoolParents 2020-09-04 09:46:44 -07:00
Jeremy Rubin
46d955d196 Remove mapLinks in favor of entry inlined structs with iterator type erasure 2020-09-04 09:46:44 -07:00
Jon Atack
062e6699c4 script: fix zmq_sub.py file permissions 2020-09-04 17:48:32 +02:00
Jon Atack
36f8e0cce7 doc: update PyZMQ installation instructions, ZeroMQ link 2020-09-04 17:48:02 +02:00
Wladimir J. van der Laan
23d3ae7acc Merge #19405: rpc, cli: add network in/out connections to getnetworkinfo and -getinfo
581b343d5b Add in/out connections to cli -getinfo (Jon Atack)
d9cc13e88d UNIX_EPOCH_TIME fixup in rpc getnettotals (Jon Atack)
1ab49b81cf Add in/out connections to rpc getnetworkinfo (Jon Atack)

Pull request description:

  This is basic info that is present in the GUI that I've been wishing to have exposed via the RPC and CLI without needing a bash workaround or script. For human users it would also be useful to have it in `-getinfo`.

  `bitcoin-cli getnetworkinfo`
  ```
    "connections": 15,
    "connections_in": 6,
    "connections_out": 9,
  ```

  `bitcoin-cli -getinfo`
  ```
    "connections": {
      "in": 6,
      "out": 9,
      "total": 15
    },
  ```

  Update the tests, RPC help, and release notes for the changes. Also fixup the `getnettotals` timemillis help while touching `rpc/net.cpp`.

  -----

  Reviewers can manually test this PR by [building from source](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests), launching bitcoind, and then running `bitcoin-cli -getinfo`, `bitcoin-cli getnetworkinfo`, `bitcoin-cli help getnetworkinfo`, and `bitcoin-cli help getnettotals` (for the UNIX epoch time change).

ACKs for top commit:
  eriknylund:
    > tACK [581b343](581b343d5b) on master at [a0a422c](a0a422c34c), ran unit & functional tests and and confirmed changes on an existing datadir ✌️
  benthecarman:
    tACK `581b343`
  willcl-ark:
    tACK for 581b343d5b, this time rebased onto master at 862fde88be.
  shesek:
    tACK `581b343`. This provides what I needed, thanks!
  n-thumann:
    tACK 581b343 on master at a0a422c, ran unit & functional tests and and confirmed changes on an existing datadir ✌️

Tree-SHA512: 08dd3ac8fefae401bd8253ff3ac027603c528eeccba53cedcb127771316173a7052fce44af8fa33ac98ebc4cf2a2b11cdefd949995d55e9b9a5942b876d00dc5
2020-09-04 15:09:37 +02:00
Wladimir J. van der Laan
99a8eb6051 Merge #19854: Avoid locking CTxMemPool::cs recursively in simple cases
020f0519ec refactor: CTxMemPool::IsUnbroadcastTx() requires CTxMemPool::cs lock (Hennadii Stepanov)
7c4bd0387a refactor: CTxMemPool::GetTotalTxSize() requires CTxMemPool::cs lock (Hennadii Stepanov)
fa5fcb032b refactor: CTxMemPool::ClearPrioritisation() requires CTxMemPool::cs lock (Hennadii Stepanov)
7140b31b90 refactor: CTxMemPool::ApplyDelta() requires CTxMemPool::cs lock (Hennadii Stepanov)
66e47e5e50 refactor: CTxMemPool::UpdateChild() requires CTxMemPool::cs lock (Hennadii Stepanov)
939807768a refactor: CTxMemPool::UpdateParent() requires CTxMemPool::cs lock (Hennadii Stepanov)

Pull request description:

  This is another step to transit `CTxMemPool::cs` from `RecursiveMutex` to `Mutex`.

  Split out from #19306.
  Only trivial thread safety annotations and lock assertions added. No new locks. No behavior change.

  Refactoring `const uint256` to `const uint256&` was [requested](https://github.com/bitcoin/bitcoin/pull/19647#discussion_r468471022) by **promag**.

  Please note that now, since #19668 has been merged, it is safe to apply `AssertLockHeld()` macros as they do not swallow compile time Thread Safety Analysis warnings.

ACKs for top commit:
  promag:
    Core review ACK 020f0519ec.
  jnewbery:
    Code review ACK 020f0519ec
  vasild:
    ACK 020f0519e

Tree-SHA512: a31e389142d5a19b25fef0aaf1072a337278564528b5cc9209df88ae548a31440e1b8dd9bae0169fd7aa59ea06e22fe5e0413955386512b83ef1f3e7d941e890
2020-09-04 13:20:22 +02:00
João Barbosa
ac2ff4fb1e refactor: Avoid duplicate map lookup in ScriptToAsmStr 2020-09-04 10:25:44 +01:00
Jonas Schnelli
a0a422c34c Merge #19754: wallet, gui: Reload previously loaded wallets on startup
f1ee37319a wallet: Reload previously loaded wallets on GUI startup (Andrew Chow)

Pull request description:

  Enable the GUI to also use the load_on_startup feature. Wallets loaded in the GUI always have load_on_startup=true. When they are unloaded from the GUI, load_on_startup=false.

  To facilitate this change, UpdateWalletSetting is moved into the wallet module and called from within LoadWallet, RemoveWallet, and Createwallet. This change does not actually touch the GUI code but rather the wallet functions that are shared between the GUI and RPC.

ACKs for top commit:
  jonasschnelli:
    Tested ACK f1ee37319a - works as expected. Wallets loaded via bitcoin-cli (in `-server` mode) or through the RPC console won't be loaded on startup but wallets loaded via the GUI menu will.
  kristapsk:
    ACK f1ee37319a, I have tested the code.

Tree-SHA512: f5b44aa763cf761d919015c5fbc0600b72434aa71e3b57007fd7530a29c3da1a9a0c98c4f22cb6cdffba61150a31170056a7d4737625e7b76f6958f3d584da8c
2020-09-03 18:24:32 +02:00
Russell Yanofsky
7bf6dfbb48 wallet: Remove path checking code from bitcoin-wallet tool
This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
77d5bb72b8 wallet: Remove path checking code from createwallet RPC
This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
a987438e9d wallet: Remove path checking code from loadwallet RPC
This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
8b5e7297c0 refactor: Pass wallet database into CWallet::Create
No changes in behavior
2020-09-03 12:24:32 -04:00
Russell Yanofsky
3c815cfe54 wallet: Remove Verify and IsLoaded methods
Checks are now consolidated in MakeBerkeleyDatabase function instead of
happening in higher level code.

This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
0d94e60625 refactor: Use DatabaseStatus and DatabaseOptions types
No changes in behavior. Just replaces arguments and return types
2020-09-03 12:24:32 -04:00
Russell Yanofsky
b5b414151a wallet: Add MakeDatabase function
New function is not currently called but will be called in upcoming commits. It
moves database path checking, and existence checking, and already-loaded
checking, and verification into a single function so this logic does not need
to be repeated all over higher level wallet code, and so higher level code does
not need to change when SQLite support is added in
https://github.com/bitcoin/bitcoin/pull/19077. This also lets higher level
wallet code make fewer assumptions about the contents of wallet directories.

This commit just adds the new function and does not change behavior in any way.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
288b4ffb6b Remove WalletLocation class
This removes a source of complexity and indirection that makes it harder to
understand path checking code. Path checks will be simplified in upcoming
commits.

There is no change in behavior in this commit other than a slightly more
descriptive error message in `loadwallet` if the default "" wallet can't be
found. (The error message is improved more in upcoming commit "wallet: Remove
path checking code from loadwallet RPC".)
2020-09-03 12:24:32 -04:00
John Newbery
825fcae484 [tests] Replace bytes literals with hex literals
It's almost impossible to read bytes literals in code, so replace them
with the hex string literal and then convert them to a bytes object
using bytes.fromhex().
2020-09-03 16:47:49 +01:00
John Newbery
64eca45100 [tests] Fix pep8 style violations in address.py 2020-09-03 16:47:49 +01:00
John Newbery
b230f8b3f3 [tests] Correct docstring for address.py 2020-09-03 16:47:49 +01:00
John Newbery
ea70e6a2ca [tests] Tidy up imports in address.py
No need to import twice from util.py
2020-09-03 16:47:49 +01:00
John Newbery
7f639df0b8 [tests] Remove unused optional verify_checksum parameter
This optional parameter is never used, so remove it.
2020-09-03 16:47:49 +01:00
John Newbery
011e784f74 [tests] Rename segwit encode and decode functions
These functions can be exported to other modules,
so be explicit that they're encoding and decoding
segwit addresses
2020-09-03 16:47:49 +01:00
John Newbery
e4557133f5 [tests] Move bech32 unit tests to test framework 2020-09-03 16:47:46 +01:00
Wladimir J. van der Laan
bd60a9a8ed Merge #19818: p2p: change CInv::type from int to uint32_t, fix UBSan warning
7984c39be1 test framework: serialize/deserialize inv type as unsigned int (Jon Atack)
407175e0c2 p2p: change CInv::type from int to uint32_t (Jon Atack)

Pull request description:

  Fixes UBSan implicit-integer-sign-change issue per https://github.com/bitcoin/bitcoin/pull/19610#issuecomment-680686460.

  Credit to Crypt-iQ for finding and reporting the issue and to vasild for the original review suggestion in https://github.com/bitcoin/bitcoin/pull/19590#pullrequestreview-455788826.

  Closes #19678.

ACKs for top commit:
  laanwj:
    ACK 7984c39be1
  MarcoFalke:
    ACK 7984c39be1 🌻
  vasild:
    ACK 7984c39be

Tree-SHA512: 59f3a75f40ce066ca6f0bb1927197254238302b4073af1574bdbfe6ed580876437be804be4e47d51467d604f0d9e3a5875159f7f2edbb2351fdb2bb9465100b5
2020-09-03 17:23:52 +02:00
Wladimir J. van der Laan
69a13eb246 Merge #19670: Protect localhost and block-relay-only peers from eviction
752e6ad533 Protect localhost and block-relay-only peers from eviction (Suhas Daftuar)

Pull request description:

  Onion peers are disadvantaged under our eviction criteria, so prevent eventual
  eviction of them in the presence of contention for inbound slots by reserving
  some slots for localhost peers (sorted by longest uptime).

  Block-relay-only connections exist as a protection against eclipse attacks, by
  creating a path for block propagation that may be unknown to adversaries.
  Protect against inbound peer connection slot attacks from disconnecting such
  peers by attempting to protect up to 8 peers that are not relaying transactions
  but have provided us with blocks.

  Thanks to gmaxwell for suggesting these strategies.

ACKs for top commit:
  laanwj:
    Code review ACK 752e6ad533

Tree-SHA512: dbf089c77c1f747aa1dbbbc2e9c2799c628028b0918d0c336d8d0e5338acedd573b530eb3b689c7f603a17221e557268a9f5c3f585f204bfb12e5d2e76de39a3
2020-09-03 17:20:47 +02:00
Wladimir J. van der Laan
4053de04e2 Merge #19859: qa: Fixes failing functional test by changing version
6de9429087 qa: Changes v0.17.1 to v0.17.2 (nthumann)

Pull request description:

  As of 0374e821bd v0.17.2 is downloaded instead of v0.17.1 for functional testing. This causes `test/functional/feature_backwards_compatibility.py` to fail, because it [requires](https://github.com/bitcoin/bitcoin/blob/master/test/functional/feature_backwards_compatibility.py#L57) v0.17.1.

  Steps to reproduce:
  Run `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.1 v0.16.3 v0.15.2`. It cannot be downloaded at all because the sha256sum is missing [here](c1e0c2ad3b/test/get_previous_releases.py (L23)).
  Or adjust the command and run `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2`, then run `test/functional/test_runner.py feature_backwards_compatibility`. It´ll fail because the test is missing v0.17.1.

  This PR changes v0.17.1 to v0.17.2 in this test and in a few comments.

ACKs for top commit:
  laanwj:
    ACK 6de9429087
  fanquake:
    ACK 6de9429087 - looks correct. Surprised this wasn't caught/part of #19813. In future you could add any explanations & extra info as part of your commit message as well (even though PR descriptions are included as part of the merge).

Tree-SHA512: bbe50c4fd5c1aedd6dc1cdc3d93ef9005db1c67adca3f263b6b0d869c40b495a3221e706c9389fedea4748e31911dbd591062f60ce9836e58099fbdd9515b4d9
2020-09-03 13:38:21 +02:00
Wladimir J. van der Laan
620ac8c475 Merge #19724: [net] Cleanup connection types- followups
eb1c5d090f [doc] Follow developer notes, add comment about missing default. (Amiti Uttarwar)
d5a57cef62 [doc] Describe connection types in more depth. (Amiti Uttarwar)
4829b6fcc6 [refactor] Simplify connection type logic in ThreadOpenConnections (Amiti Uttarwar)
1e563aed78 [refactor] Simplify check for block-relay-only connection. (Amiti Uttarwar)
da3a0be61b [test] Add explicit tests that connection types get set correctly (Amiti Uttarwar)
1d74fc7df6 [trivial] Small style updates (Amiti Uttarwar)
ff6b9081ad [doc] Explain address handling logic in process messages (Amiti Uttarwar)
dff16b184b [refactor] Restructure logic to check for addr relay. (Amiti Uttarwar)
a6ab1e81f9 [net] Remove unnecessary default args on OpenNetworkConnection (Amiti Uttarwar)
8d6ff46f55 scripted-diff: Rename `OUTBOUND` ConnectionType to `OUTBOUND_FULL_RELAY` (Amiti Uttarwar)

Pull request description:

  This PR addresses outstanding review comments from #19316. It further simplifies `net.cpp` complexity and adds documentation about design goals about different connection types.

ACKs for top commit:
  naumenkogs:
    ACK eb1c5d090f
  laanwj:
    Code review ACK eb1c5d090f

Tree-SHA512: 2fe14e428404c95661e5518c8c90db07ab5b9ebb1bac921b3bdf82b181f444fae379f8fc0a2b619e6b4693f01c55bd246fbd8c8eedadd96849a30de3161afee5
2020-09-03 13:28:30 +02:00
fanquake
68f0ab26bc Merge #19805: wallet: Avoid deserializing unused records when salvaging
0bbe26a1af wallet: filter for keys only before record deser in salvage (Andrew Chow)
544e12a4e8 walletdb: Add KeyFilterFn to ReadKeyValue (Andrew Chow)

Pull request description:

  When salvaging a wallet, the only things that matter are the private keys. It is not necessary to attempt to deserialize any other records, especially if those records are corrupted too.

  This PR adds a `KeyFilterFn` function callback to `ReadKeyValue` that salvage uses to filter for only the records that it wants. Of course doing it this way also lets us do other filters in the future from other places should we so desire.

ACKs for top commit:
  ryanofsky:
    Code review ACK 0bbe26a1af. Looks great! This should make the recovery code more robust. Normally it'd be good to have a test case for the problem this fixes, but Marco already wrote one in #19078, so I think we're covered
  laanwj:
    Code review ACK 0bbe26a1af

Tree-SHA512: 8e3ee283a22a79273915711c4fb751f3c9b02ce94e6bf08dc468f1cfdf9fac35c693bbfd2435ce43c3a06c601b9b0a67e209621f6814bedfe3bc7a7ccc37bb01
2020-09-03 12:47:01 +08:00
fanquake
136fe4c5e9 Merge #19816: test: Rename wait until helper to wait_until_helper
fa1cd9e1dd test: Remove unused lock arg from BitcoinTestFramework.wait_until (MarcoFalke)
fad2794e93 test: Rename wait until helper to wait_until_helper (MarcoFalke)
facb41bf1d test: Remove unused p2p_lock in VersionBitsWarningTest (MarcoFalke)

Pull request description:

  This avoids confusion with the `wait_until` member functions, which should be preferred because they take the appropriate locks and scale the timeout appropriately on their own.

ACKs for top commit:
  laanwj:
    Code review ACK fa1cd9e1dd
  hebasto:
    ACK fa1cd9e1dd, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 319d400085606a4c738e314824037f72998e6657d8622b363726842aba968744f23c56d27275dfe506b8cbbb6e97fc39ca1d325db05d4d67df0e8b35f2244d5c
2020-09-03 12:07:53 +08:00
fanquake
9876ab8c74 Merge #19844: remove usage of boost::bind
e36f802fa4 lint: add C++ code linter (fanquake)
c4be50fea3 remove usage of boost::bind (fanquake)

Pull request description:

  `boost::bind` usage was removed in #13743. However a new usage snuck in as
  part of 2bc4c3eaf9 (#15225).

ACKs for top commit:
  hebasto:
    ACK e36f802fa4
  practicalswift:
    ACK e36f802fa4 -- patch looks correct

Tree-SHA512: 2b0387c5443c184bcbf7df4849db1ed1296ff82c7b4ff0aff18334a400e56a472a972d18234d3866531a088d7a8da64688e58dc9f15daaad4048697c759d55ce
2020-09-03 11:43:10 +08:00
fanquake
2d4574aad8 Merge #19861: build: add /usr/local/ to LCOV_FILTER_PATTERN for macOS builds
9bdde3c802 build: add /usr/local/ to LCOV_FILTER_PATTERN for macOS builds (eugene)

Pull request description:

  With this commit, the files in /usr/local/ will not be included in
  `make cov` or `make cov_fuzz` coverage reports. This behavior could
  be observed when generating the reports on macOS with brew-installed
  clang.

ACKs for top commit:
  fanquake:
    ACK 9bdde3c802

Tree-SHA512: 15cbe8d514651448f3d7b8b0a00939fd6bd6f15e6812f1959fcaaab7364ca2ef4ee34f2ff8950b7fdc8ae64d043dc5f7185c0601dd94780b41331337e5e84c45
2020-09-03 11:36:45 +08:00
Amiti Uttarwar
eb1c5d090f [doc] Follow developer notes, add comment about missing default. 2020-09-02 17:18:22 -07:00
Amiti Uttarwar
d5a57cef62 [doc] Describe connection types in more depth. 2020-09-02 17:18:22 -07:00
Amiti Uttarwar
4829b6fcc6 [refactor] Simplify connection type logic in ThreadOpenConnections
Consolidate the logic to determine connection type into one conditional to
clarify how they are chosen.
2020-09-02 17:18:22 -07:00
Amiti Uttarwar
1e563aed78 [refactor] Simplify check for block-relay-only connection.
Previously we deduced it was a block-relay-only based on presence of the
m_tx_relay structure. Now we have the ability to identify it directly via a
connection type accessor function.
2020-09-02 17:18:22 -07:00
Amiti Uttarwar
da3a0be61b [test] Add explicit tests that connection types get set correctly 2020-09-02 17:18:22 -07:00
Amiti Uttarwar
1d74fc7df6 [trivial] Small style updates 2020-09-02 17:18:21 -07:00
Amiti Uttarwar
ff6b9081ad [doc] Explain address handling logic in process messages
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2020-09-02 17:18:21 -07:00
Amiti Uttarwar
dff16b184b [refactor] Restructure logic to check for addr relay.
We previously identified if we relay addresses to the connection by checking
for the existence of the m_addr_known data structure. With this commit, we
answer this question based on the connection type.

IsAddrRelayPeer() checked for the existence of the m_addr_known
2020-09-02 17:18:21 -07:00
Amiti Uttarwar
a6ab1e81f9 [net] Remove unnecessary default args on OpenNetworkConnection 2020-09-02 13:36:29 -07:00
Amiti Uttarwar
8d6ff46f55 scripted-diff: Rename OUTBOUND ConnectionType to OUTBOUND_FULL_RELAY
-BEGIN VERIFY SCRIPT-
sed -i 's/OUTBOUND, /OUTBOUND_FULL_RELAY, /g' src/net.h
sed -i 's/ConnectionType::OUTBOUND/ConnectionType::OUTBOUND_FULL_RELAY/g' src/test/net_tests.cpp src/test/fuzz/process_message.cpp src/test/fuzz/process_messages.cpp src/net.cpp src/test/denialofservice_tests.cpp src/net.h src/test/fuzz/net.cpp
-END VERIFY SCRIPT-
2020-09-02 13:34:58 -07:00
eugene
9bdde3c802 build: add /usr/local/ to LCOV_FILTER_PATTERN for macOS builds
With this commit, the files in /usr/local/ will not be included in
`make cov` or `make cov_fuzz` coverage reports. This behavior could
be observed when generating the reports on macOS with brew-installed
clang.
2020-09-02 15:17:13 -04:00
Jon Atack
bf1f913c44 cli -netinfo: display multiple levels of details 2020-09-02 16:24:23 +02:00
Suhas Daftuar
752e6ad533 Protect localhost and block-relay-only peers from eviction
Onion peers are disadvantaged under our eviction criteria, so prevent eventual
eviction of them in the presence of contention for inbound slots by reserving
some slots for localhost peers (sorted by longest uptime).

Block-relay-only connections exist as a protection against eclipse attacks, by
creating a path for block propagation that may be unknown to adversaries.
Protect against inbound peer connection slot attacks from disconnecting such
peers by attempting to protect up to 8 peers that are not relaying transactions
but appear to be full-nodes, sorted by recency of last delivered block.

Thanks to gmaxwell for suggesting these strategies.
2020-09-02 09:21:33 -04:00
Russell Yanofsky
ff44cae279 test: Change feature_config_args.py not to rely on strange regtest=0 behavior
Update test to simply generate a normal mainnet configuration file instead of
using a crazy setup where a regtest=1 config file using an includeconf in the
[regtest] section includes another config file that specifies regtest=0,
retroactively switching the network to mainnet.

This setup was fragile and only worked because the triggered InitError happened
early enough that none of the ignored [regtest] options mattered (only
affecting log output).
2020-09-02 08:14:34 -05:00
Wladimir J. van der Laan
c157a50694 Merge #19840: Avoid callback when -blocknotify is empty
413e0d1d31 Avoid callback when -blocknotify is empty (João Barbosa)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    ACK 413e0d1d31
  practicalswift:
    ACK 413e0d1d31 -- patch looks correct
  laanwj:
    Code review ACK 413e0d1d31

Tree-SHA512: 915e796666b4e74dbb029ba5436e5573a4b881aad9e118f737bcff4024528b7ff3b00dd035138f63d30963cfd66195f6e53a2dbe429ee28cb6f0b9cc47218ecf
2020-09-02 15:14:34 +02:00
fanquake
8845b38b59 Merge #19685: depends: CMake invocation cleanup
b893688357 depends: Specify LDFLAGS to cmake as well (Carl Dong)
b3f541f618 depends: Prepend CPPFLAGS to C{,XX}FLAGS for CMake (Carl Dong)
8e121e5509 depends: Cleanup CMake invocation (Carl Dong)
8c7cd0c6d9 depends: More robust cmake invocation (Carl Dong)
3ecf0eca63 depends: Use $($(package)_cmake) instead of cmake (Carl Dong)

Pull request description:

  - Use `$($(package)_cmake)` instead of invoking `cmake` directly
  - Use well-known env vars instead of overriding CMake variables

ACKs for top commit:
  ryanofsky:
    Code review ACK b893688357. Only changes since last review are new commits adding whitespace, cppflags and ldflags to cmake invocation

Tree-SHA512: cfcd8cc9dcd0b336cf48b82fca9fe4bbc7930ed397cb7a68a07066680eb4c1906a6a9b5bd2589b4b4999e8f16232fa30ee9b376b60f4456d0fff931fbf9cc19a
2020-09-02 21:03:05 +08:00
fanquake
c17a003758 Merge #19857: net: improve nLastBlockTime and nLastTXTime documentation
d780293e1e net: improve nLastBlockTime and nLastTXTime documentation (Jon Atack)

Pull request description:

  Follow-up to #19731 to help alleviate confusion around `nLastBlockTime` and `nLastTXTime`, now also provided by the JSON-RPC API as `last_block` and `last_transaction` in `getpeerinfo` output.

  Thanks to John Newbery, credited in the commit, and to Dave Harding and Adam Jonas during discussions on how to best explain these in this week's Optech newsletter.

ACKs for top commit:
  practicalswift:
    ACK d780293e1e
  MarcoFalke:
    ACK d780293e1e
  harding:
    ACK d780293e1e .  The added documentation matches my reading of the code and answers a question I had after seeing #19731
  0xB10C:
    ACK d780293e1e

Tree-SHA512: 72d47cf50a099913c7e4753cb80e11785b26fb66fa3a8b6c382fde4ea725116f3d215f93d32a567246d269768e66159f8dcf017a1bbc6d5f2489a35f81c316fa
2020-09-02 20:35:25 +08:00
nthumann
6de9429087 qa: Changes v0.17.1 to v0.17.2 2020-09-02 13:51:36 +02:00
Wladimir J. van der Laan
505b39e72b Merge #19610: p2p: refactor AlreadyHave(), CInv::type, INV/TX processing
fb56d37612 p2p: ensure inv is GenMsgTx before ToGenTxid in inv processing (John Newbery)
aa3621385e test: use CInv::MSG_WITNESS_TX flag in p2p_segwit (Jon Atack)
24ee4f01ea p2p: make gtxid(.hash) and fAlreadyHave localvars const (Jon Atack)
b1c855453b p2p: use CInv block message helpers in net_processing.cpp (Jon Atack)
acd6642167 [net processing] Change AlreadyHaveTx() to take a GenTxid (John Newbery)
5fdfb80b86 [net processing] Change AlreadyHaveBlock() to take block_hash argument (John Newbery)
430e183b89 [net processing] Remove mempool argument from AlreadyHaveBlock() (John Newbery)
42ca5618ca [net processing] Split AlreadyHave() into separate block and tx functions (John Newbery)
39f1dc9445 p2p: remove nFetchFlags from NetMsgType TX and INV processing (Jon Atack)
471714e1f0 p2p: add CInv block message helper methods (Jon Atack)

Pull request description:

  Building on #19590 and the recent `wtxid` and `GenTxid` changes, this is a refactoring and cleanup PR to simplify and improve some of the net processing code.

  Some of the diffs are best reviewed with `-w` to ignore spacing.

  Co-authored by John Newbery.

ACKs for top commit:
  laanwj:
    Code review ACK fb56d37612
  jnewbery:
    utACK fb56d37612
  vasild:
    ACK fb56d3761

Tree-SHA512: ba39b58e6aaf850880a842fe5f6295e9f1870906ef690206acfc17140aae2ac854981e1066dbcd4238062478762fbd040ef772fdc2c50eea6869997c583e6a6d
2020-09-02 13:45:40 +02:00
Gleb Naumenko
0d04784af1 Refactor the functional test 2020-09-02 10:33:18 +03:00
Gleb Naumenko
83ad65f31b Address nits in ADDR caching 2020-09-02 10:33:17 +03:00
Jonas Schnelli
3a3e21dafb Merge #14687: zmq: enable tcp keepalive
c276df7759 zmq: enable tcp keepalive (mruddy)

Pull request description:

  This addresses https://github.com/bitcoin/bitcoin/issues/12754.

  These changes enable node operators to address the silent dropping (by network middle boxes) of long-lived low-activity ZMQ TCP connections via further operating system level TCP keepalive configuration. For example, ZMQ sockets that publish block hashes can be affected in this way due to the length of time it sometimes takes between finding blocks (e.g.- sometimes more than an hour).

  Prior to this patch, operating system level TCP keepalive configurations would not take effect since the SO_KEEPALIVE option was not enabled on the underlying socket.

  There are additional ZMQ socket options related to TCP keepalive that can be set. However, I decided not to implement those options in this changeset because doing so would require adding additional bitcoin node configuration options, and would not yield a better outcome. I preferred a small, easily reviewable patch that doesn't add a bunch of new config options, with the tradeoff that the fine tuning would have to be done via well-documented operating system specific configurations.

  I tested this patch by running a node with:
  `./src/qt/bitcoin-qt -regtest -txindex -datadir=/tmp/node -zmqpubhashblock=tcp://127.0.0.1:28332 &`
  and connecting to it with:
  `python3 ./contrib/zmq/zmq_sub.py`

  Without these changes, `ss -panto | grep 28332 | grep ESTAB | grep bitcoin` will report no keepalive timer information. With these changes, the output from the prior command will show keepalive timer information consistent with the configuration at the time of connection establishment, e.g.-: `timer:(keepalive,119min,0)`.

  I also tested with a non-TCP transport and did not witness any adverse effects:
  `./src/qt/bitcoin-qt -regtest -txindex -datadir=/tmp/node -zmqpubhashblock=ipc:///tmp/bitcoin.block &`

ACKs for top commit:
  adamjonas:
    Just to summarize for those looking to review - as of c276df7759 there are 3 tACKs (n-thumann, Haaroon, and dlogemann), 1 "looks good to me" (laanwj) with no NACKs or any show-stopping concerns raised.
  jonasschnelli:
    utACK c276df7759

Tree-SHA512: b884c2c9814e97e666546a7188c48f9de9541499a11a934bd48dd16169a900c900fa519feb3b1cb7e9915fc7539aac2829c7806b5937b4e1409b4805f3ef6cd1
2020-09-02 09:09:18 +02:00
Andrew Chow
f1ee37319a wallet: Reload previously loaded wallets on GUI startup
Enable the GUI to also use the load_on_startup feature.
Wallets loaded in the GUI always have load_on_startup=true.
When they are unloaded from the GUI, load_on_startup=false.

To facilitate this change, UpdateWalletSetting is moved into the wallet
module and called from within LoadWallet, RemoveWallet, and
Createwallet. This change does not actually touch the GUI code but
rather the wallet functions that are shared between the GUI and RPC.
2020-09-01 12:13:50 -04:00
Jon Atack
d780293e1e net: improve nLastBlockTime and nLastTXTime documentation
Co-authored-by: John Newbery <john@johnnewbery.com>
2020-09-01 17:46:28 +02:00
Adam Jonas
a06eb03ded doc: Add comments and additional reviewers to CODEOWNERS file 2020-09-01 11:23:58 -04:00
Wladimir J. van der Laan
48c1083632 Merge #19105: Add Muhash3072 implementation in Python
36ec9801a4 test: Add chacha20 test vectors in muhash (Fabian Jahr)
0e2b400fea test: Add basic Python/C++ Muhash implementation parity unit test (Fabian Jahr)
b85543cb73 test: Add Python MuHash3072 implementation to test framework (Pieter Wuille)
ab30cece0e test: Move modinv to util and add unit test (Fabian Jahr)

Pull request description:

  This is the second in a [series of pull requests](https://github.com/bitcoin/bitcoin/pull/18000) to implement an Index for UTXO set statistics.

  This pull request adds a Python implementation of Muhash3072, a homomorphic hashing algorithm to be used for hashing the UTXO set. The Python implementation can then be used to compare behavior with the C++ version.

ACKs for top commit:
  jnewbery:
    utACK 36ec9801a
  laanwj:
    Code review ACK 36ec9801a4

Tree-SHA512: a3519c6e11031174f1ae71ecd8bcc7f3be42d7fc9c84c77f2fbea7cfc5ad54fcbe10b55116ad8d9a52ac5d675640eefed3bf260c58a02f2bf3bc0d8ec208baa6
2020-09-01 17:12:20 +02:00
Vasil Dimov
8de51d1513 doc: elaborate on release notes wrt netmasks 2020-09-01 15:28:39 +02:00
Hennadii Stepanov
020f0519ec refactor: CTxMemPool::IsUnbroadcastTx() requires CTxMemPool::cs lock
No change in behavior, the lock is already held at call sites.
2020-09-01 12:36:27 +03:00
Hennadii Stepanov
7c4bd0387a refactor: CTxMemPool::GetTotalTxSize() requires CTxMemPool::cs lock
No change in behavior, the lock is already held at call sites.
2020-09-01 12:34:39 +03:00
Hennadii Stepanov
fa5fcb032b refactor: CTxMemPool::ClearPrioritisation() requires CTxMemPool::cs lock
No change in behavior, the lock is already held at call sites.
Also `const uint256` refactored to `const uint256&`.
2020-09-01 12:34:29 +03:00
Hennadii Stepanov
7140b31b90 refactor: CTxMemPool::ApplyDelta() requires CTxMemPool::cs lock
No change in behavior, the lock is already held at call sites.
Also `const uint256` refactored to `const uint256&`.
2020-09-01 12:34:29 +03:00
Hennadii Stepanov
66e47e5e50 refactor: CTxMemPool::UpdateChild() requires CTxMemPool::cs lock
No change in behavior, the lock is already held at call sites.
2020-09-01 12:34:19 +03:00
Hennadii Stepanov
939807768a refactor: CTxMemPool::UpdateParent() requires CTxMemPool::cs lock
No change in behavior, the lock is already held at call sites.
2020-09-01 12:34:11 +03:00
fanquake
e36f802fa4 lint: add C++ code linter
This currently only checks for boost::bind usage.

Co-authored-by: practicalswift <practicalswift@users.noreply.github.com>
2020-09-01 14:23:08 +08:00
MarcoFalke
bab4cce1b0 Merge #19668: Do not hide compile-time thread safety warnings
ea74e10acf doc: Add best practice for annotating/asserting locks (Hennadii Stepanov)
2ee7743fe7 sync.h: Make runtime lock checks require compile-time lock checks (Anthony Towns)
23d71d171e Do not hide compile-time thread safety warnings (Hennadii Stepanov)
3ddc150857 Add missed thread safety annotations (Hennadii Stepanov)
af9ea55a72 Use LockAssertion utility class instead of AssertLockHeld() (Hennadii Stepanov)

Pull request description:

  On the way of transit from `RecursiveMutex` to `Mutex` (see #19303) it is crucial to have run-time `AssertLockHeld()` assertion that does _not_ hide compile-time Clang Thread Safety Analysis warnings.

  On master (65e4ecabd5) using `AssertLockHeld()` could hide Clang Thread Safety Analysis warnings, e.g., with the following patch applied:
  ```diff
  --- a/src/txmempool.h
  +++ b/src/txmempool.h
  @@ -607,7 +607,7 @@ public:
       void addUnchecked(const CTxMemPoolEntry& entry, setEntries& setAncestors, bool validFeeEstimate = true) EXCLUSIVE_LOCKS_REQUIRED(cs, cs_main);

       void removeRecursive(const CTransaction& tx, MemPoolRemovalReason reason) EXCLUSIVE_LOCKS_REQUIRED(cs);
  -    void removeForReorg(const CCoinsViewCache* pcoins, unsigned int nMemPoolHeight, int flags) EXCLUSIVE_LOCKS_REQUIRED(cs, cs_main);
  +    void removeForReorg(const CCoinsViewCache* pcoins, unsigned int nMemPoolHeight, int flags) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
       void removeConflicts(const CTransaction& tx) EXCLUSIVE_LOCKS_REQUIRED(cs);
       void removeForBlock(const std::vector<CTransactionRef>& vtx, unsigned int nBlockHeight) EXCLUSIVE_LOCKS_REQUIRED(cs);

  ```
  Clang compiles the code without any thread safety warnings.

  See "Add missed thread safety annotations" commit for the actual thread safety warnings that are fixed in this PR.

ACKs for top commit:
  MarcoFalke:
    ACK ea74e10acf 🎙
  jnewbery:
    ACK ea74e10acf
  ajtowns:
    ACK ea74e10acf

Tree-SHA512: 8cba996e526751a1cb0e613c0cc1b10f027a3e9945fbfb4bd30f6355fd36b9f9c2e1e95ed3183fc254b42df7c30223278e18e5bdb5e1ef85db7fef067595d447
2020-09-01 08:18:26 +02:00
fanquake
a1d14f522c Merge #19671: wallet: Remove -zapwallettxes
3340dbadd3 Remove -zapwallettxes (Andrew Chow)

Pull request description:

  It's not clear what use there is to keeping `-zapwallettxes` given that it's intended usage has been superseded by `abandontransaction`. So this removes it outright.

  Alternative to #19700

ACKs for top commit:
  meshcollider:
    utACK 3340dbadd3
  fanquake:
    ACK 3340dbadd3 - remaining manpage references will get cleaned up pre-release.

Tree-SHA512: 3e58e1ef6f4f94894d012b93e88baba3fb9c2ad75b8349403f9ce95b80b50b0b4f443cb623cf76c355930db109f491b3442be3aa02972e841450ce52cf545fc8
2020-09-01 09:26:28 +08:00
Wladimir J. van der Laan
e796fdd4cb Merge #19507: Expand functional zmq transaction tests
7356292e1d Have zmq reorg test cover mempool txns (Gregory Sanders)
a0f4f9c983 Add zmq test for transaction pub during reorg (Gregory Sanders)
2399a0600c Add test case for mempool->block zmq notification (Gregory Sanders)
e70512a83c Make ordering of zmq consumption irrelevant to functional test (Gregory Sanders)

Pull request description:

  Tests written to better define what messages are sent when. Also did a bit of refactoring to make sure the exact notification channel ordering doesn't matter.

  Confusions below aside, I believe having these more descriptive tests helps describe what behavior we expect from ZMQ notificaitons.

  Remaining confusion:
  1) Notification patterns seem to vary wildly with the inclusion of mempool transactions being reorg'ed. See difference between "Add zmq test for transaction pub during reorg" and "Have zmq reorg test cover mempool txns" commits for specifics.
  2) Why does a reorg'ed transaction get announced 3 times? From what I understand it can get announced once for disconnected block, once for mempool entry. What's the third? It occurs a 4th time when included in a block(not added in test)

ACKs for top commit:
  laanwj:
    code review ACK 7356292e1d
  promag:
    Code review ACK 7356292e1d.

Tree-SHA512: 573662429523fd6a1af23dd907117320bc68cb51a93fba9483c9a2160bdce51fb590fcd97bcd2b2751d543d5c1148efa4e22e1c3901144f882b990ed2b450038
2020-08-31 20:46:27 +02:00
Andrew Chow
3340dbadd3 Remove -zapwallettxes
-zapwallettxes is made a hidden option to inform users that it is
removed and they should be using abandontransaction to do the stuck
transaction thing.
2020-08-31 12:39:19 -04:00
MarcoFalke
fa6bb0ce5d Assert that RPCArg names are equal to CRPCCommand ones (rawtransaction) 2020-08-31 18:29:57 +02:00
MarcoFalke
fa80c81487 Assert that RPCArg names are equal to CRPCCommand ones (blockchain) 2020-08-31 18:29:55 +02:00
MarcoFalke
89a8299a14 Merge #19717: rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining,zmq,rpcdump)
fa3d9ce325 rpc: Assert that RPCArg names are equal to CRPCCommand ones (rpcdump) (MarcoFalke)
fa32c1d5ec rpc: Assert that RPCArg names are equal to CRPCCommand ones (zmq) (MarcoFalke)
faaa46dc20 rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining) (MarcoFalke)
fa93bc14c7 rpc: Remove unused return type from appendCommand (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    tested ACK fa3d9ce325
  promag:
    Code review ACK fa3d9ce325.

Tree-SHA512: 068ade4b55cc195868d53b7f9a27151d45b440857bb069e261a49d102a49a38fdba5d68868516a1d66a54a73ba34681362f934ded7349e894042bde873b75719
2020-08-31 17:43:35 +02:00
MarcoFalke
068bc21188 Merge #19842: Update the vcpkg checkout commit ID in appveyor config
a104caeb40 Update the vcpkg checkout commit ID in appveyor config. (Aaron Clauson)

Pull request description:

  A recent appveyor vm update broke the build of the `berkeleydb` vcpkg dependency, see #19839. The temporary resolution was to switch back to the previous appveyor vm.

  This PR updates the pegged vcpkg commit ID to the most recent commit as of 31 Aug 2020. That commit ID has been tested against the latest appveyor vm and is able to build Bitcoin Core successfully.

  The vcpkg bump includes a [patch](https://github.com/microsoft/vcpkg/pull/12870) to the `berkeleydb` build config which allows it to be built on the latest appveyor vm.

ACKs for top commit:
  MarcoFalke:
    Concept ACK a104caeb40

Tree-SHA512: 6d363d1615c51bb3d4b324eb96d53950648fc97fc81ffaef91ee6e92f1336776d150d89f6e859f354ee75ce66afcef07aa19ed39b725dbb3f47ba67d26e111db
2020-08-31 17:02:20 +02:00
MarcoFalke
c1e0c2ad3b Merge #19813: util, ci: Hard code previous release tarball checksums
0374e821bd util: Hard code previous release tarball checksums (Hennadii Stepanov)
bd897ce79f scripted-diff: Move previous_release.py to test/get_previous_releases.py (Hennadii Stepanov)

Pull request description:

  #19205 introduced signature verifying for the downloaded `SHA256SUMS.asc`.
  This approach is brittle and does not work in CI environment for many reasons:
  - https://github.com/bitcoin/bitcoin/issues/19812#issuecomment-680760663
  - https://github.com/bitcoin/bitcoin/pull/19013#discussion_r459590779

  This PR:
  - implements **Sjors**' [idea](https://github.com/bitcoin/bitcoin/pull/19205#pullrequestreview-426080048):
  > Alternatively we might as well hard code the checksum for each `tar.gz` release in the source code, here.

  - is an alternative to 5a2c31e528e6bd60635096f233252f3c717f366d (#19013)

  - fixes #19812

  - updates v0.17.1 to v0.17.2

ACKs for top commit:
  MarcoFalke:
    cr ACK 0374e821bd
  Sjors:
    tACK 0374e821bd

Tree-SHA512: cacdcf9f5209eae7da357abb3445585ad2f980920fd5bf75527ce89974d3f531a4cf8b5b35edfc116b23bfdfb45c0437cb14cbc416d76ed2dc5b9e6d33cdad71
2020-08-31 16:18:29 +02:00
Jon Atack
077b3ac928 cli: change -netinfo optional arg from bool to int 2020-08-31 16:13:29 +02:00
Jon Atack
4e2f2ddd64 cli: add getpeerinfo last_{block,transaction} to -netinfo 2020-08-31 16:13:26 +02:00
Jon Atack
644be659ab cli: add -netinfo server version check and error message 2020-08-31 16:13:17 +02:00
Jon Atack
ce57bf6cc0 cli: create peer connections report sorted by dir, minping 2020-08-31 16:12:44 +02:00
Jon Atack
f5edd66e5d cli: create vector of Peer structs for peers data 2020-08-31 16:12:36 +02:00
Jon Atack
3a0ab93e1c cli: add NetType enum struct and NetTypeEnumToString() 2020-08-31 16:12:04 +02:00
Jon Atack
c227100919 cli: create local addresses, ports, and scores report 2020-08-31 16:12:01 +02:00
Jon Atack
d3f77b736e cli: create inbound/outbound peer connections report 2020-08-31 16:11:59 +02:00
Jon Atack
19377b2fd2 cli: start dashboard report with chain and version header 2020-08-31 16:11:51 +02:00
Jon Atack
a3653c159e cli: tally peer connections by type 2020-08-31 16:11:09 +02:00
fanquake
c4be50fea3 remove usage of boost::bind
boost::bind usage was removed in #13743. However a new usage snuck in as
part of 2bc4c3eaf9 (#15225).
2020-08-31 19:34:57 +08:00
Samuel Dobson
f98872f127 Merge #18244: rpc: fundrawtransaction and walletcreatefundedpsbt also lock manually selected coins
6d1f51343c [rpc] fundrawtransaction, walletcreatefundedpsbt lock manually selected coins (Sjors Provoost)

Pull request description:

  When using `fundrawtransaction` and `walletcreatefundedpsbt` with `lockUnspents`, it would only lock automatically selected coins, not manually selected coins. That doesn't make much sense to me if the goal is to prevent accidentally double-spending yourself before you broadcast a transaction.

  Note that when  creating a transaction, manually selected coins are automatic "unlocked" (or more accurately: the lock is ignored). Earlier versions of this PR introduced an error when a locked coin is manually selected, but this idea was abandoned after some discussion. An application that uses this RPC should either rely on automatic coin selection (with `lockUnspents`) or handle lock concurrency itself with manual coin selection. In particular it needs to make sure to avoid/pause calls with automatic coin selection between calling `lockunspent` and the subsequent spending RPC.

  See #7518 for historical background.

ACKs for top commit:
  meshcollider:
    Code review ACK 6d1f51343c
  fjahr:
    Code review ACK 6d1f51343c

Tree-SHA512: 8773c788d92f2656952e1beac147ba9956b8c5132d474e0880e4c89ff53642928b4cbfcd1cb3d17798b9284f02618a8830c93a9f7a4733e5bded96adff1d5d4d
2020-08-31 23:30:53 +12:00
Samuel Dobson
7721b31809 Merge #19773: wallet: Avoid recursive lock in IsTrusted
772ea4844c wallet: Avoid recursive lock in IsTrusted (João Barbosa)
819f10f671 wallet, refactor: Immutable CWalletTx::pwallet (João Barbosa)

Pull request description:

  This change moves `CWalletTx::IsTrusted` to `CWallet` in order to have TSAN. So now `CWallet::IsTrusted` requires `cs_wallet` and the recursive lock no longer happens.

  Motivated by https://github.com/bitcoin/bitcoin/pull/19289/files#r473308226.

ACKs for top commit:
  meshcollider:
    utACK 772ea4844c
  hebasto:
    ACK 772ea4844c, reviewed and tested on Linux Mint 20 (x86_64).

Tree-SHA512: 702ffd928b2f42a8b90de398790649a5fd04e1ac3877558da928e94cdeb19134883f06c3a73a6826c11c912facf199173375a70200737e164ccaea1bec515b2a
2020-08-31 22:45:27 +12:00
Zero
ed69213c2b build: enable unused member function diagnostic 2020-08-31 10:24:59 +01:00
MarcoFalke
61b8c04d78 Merge #19379: tests: Add fuzzing harness for SigHasLowR(...) and ecdsa_signature_parse_der_lax(...)
46fcac1e4b tests: Add fuzzing harness for ec_seckey_import_der(...) and ec_seckey_export_der(...) (practicalswift)
b667a90389 tests: Add fuzzing harness for SigHasLowR(...) and ecdsa_signature_parse_der_lax(...) (practicalswift)

Pull request description:

  Add fuzzing harness for `SigHasLowR(...)` and `ecdsa_signature_parse_der_lax(...)`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  Crypt-iQ:
    ACK 46fcac1e4b

Tree-SHA512: 11a4856a1efd9a04030a8c8aee2413fd5be1ea248147e649a48a55bacdf732bb48a19ee1ce2761d47d4dd61c9598aec53061b961b319ad824d539dda11a8ccf4
2020-08-31 10:56:34 +02:00
MarcoFalke
269a7ccb27 Merge #19099: refactor: Move wallet methods out of chain.h and node.h
24bf17602c gui refactor: Inline SplashScreen::ConnectWallet (Russell Yanofsky)
e4f4350471 refactor: Move wallet methods out of chain.h and node.h (Russell Yanofsky)
b266b3e0bf refactor: Create interfaces earlier during initialization (Russell Yanofsky)

Pull request description:

  Add WalletClient interface so node interface is cleaner and don't need wallet-specific methods.

  The new NodeContext::wallet_client pointer will also be needed to eliminate global wallet variables like ::vpwallets in #19101, because createWallet(), loadWallet(), getWallets(), etc methods called by the GUI need a way to get a reference to the list of open wallets if it is no longer a global variable.

ACKs for top commit:
  promag:
    Code review ACK 24bf17602c.
  MarcoFalke:
    ACK 24bf17602c 🐚

Tree-SHA512: a70d3776cd6723093db8912028c50075ec5fa0a48b961cb1a945f922658f5363754f8380dbb8378ed128c8c858913024f8264740905b8121a35c0d63bfaed7cf
2020-08-31 10:10:57 +02:00
Aaron Clauson
a104caeb40 Update the vcpkg checkout commit ID in appveyor config. 2020-08-31 08:10:02 +01:00
MarcoFalke
afffbb1bc6 Merge #19710: bench: Prevent thread oversubscription and decreases the variance of result values
3edc4e34fe bench: Prevent thread oversubscription (Hennadii Stepanov)
ce3e6a7cb2 bench: Allow skip benchmark (Hennadii Stepanov)

Pull request description:

  Split out from #18710.

  Some results (borrowed from #18710):
  ![89121718-a3329800-d4c1-11ea-8bd1-66da20619696](https://user-images.githubusercontent.com/32963518/90146614-ecb89800-dd89-11ea-80fe-bac0e46e735e.png)

ACKs for top commit:
  fjahr:
    Code review ACK 3edc4e34fe

Tree-SHA512: df7413ec9ea326564a8e8de54752c9d1444ff7de34edb03e1e0c2120fc333e4640767fdbe3e87eab6a7b389a4863c02e22ad2ae0dbf139fad6a9b85e00f563b4
2020-08-31 08:29:27 +02:00
MarcoFalke
5c910a6b7a Merge #19826: Pass mempool reference to chainstate constructor
fa0572d0f3 Pass mempool reference to chainstate constructor (MarcoFalke)

Pull request description:

  Next step toward #19556

  Instead of relying on the mempool global, each chainstate is given a reference to a mempool to keep up to date with the tip (block connections, disconnections, reorgs, ...)

ACKs for top commit:
  promag:
    Code review ACK fa0572d0f3.
  darosior:
    ACK fa0572d0f3
  hebasto:
    ACK fa0572d0f3, reviewed and tested on Linux Mint 20 (x86_64).

Tree-SHA512: 12184d33ae5797438d03efd012a07ba3e4ffa0d817c7a0877743f3d7a7656fe279280c751554fc035ccd0058166153b6c6c308a98b2d6b13998922617ad95c4c
2020-08-31 07:21:27 +02:00
fanquake
0adb80fe63 Merge #19803: Bugfix: Define and use HAVE_FDATASYNC correctly outside LevelDB
c4b85ba704 Bugfix: Define and use HAVE_FDATASYNC correctly outside LevelDB (Luke Dashjr)

Pull request description:

  Fixes a bug introduced in #19614

  The LevelDB-specific fdatasync check was only using `AC_SUBST`, which works for Makefiles, but doesn't define anything for C++. Furthermore, the #define is typically 0 or 1, never undefined.

  This fixes both issues by defining it and checking its value instead of whether it is merely defined.

  Pulled out of #14501 by fanquake's request

ACKs for top commit:
  fanquake:
    ACK c4b85ba704 - thanks for catching and fixing my mistake.
  laanwj:
     Code review ACK c4b85ba704

Tree-SHA512: 91d5d426ba000b4f3ee7e2315635e24bbb23ceff16269ddf4f65a63d25fc9e9cf94a3b236eed2f8031cc36ddcf78aeb5916efcb244f415943a8a12f907ede8f9
2020-08-31 13:07:24 +08:00
fanquake
21eda43cde Merge #19828: wallet, refactor: Remove duplicate map lookups in GetAddressBalances
b35e74ba37 wallet, refactor: Remove duplicate map lookups in GetAddressBalances (João Barbosa)

Pull request description:

  Now just one lookup in `balances` instead of three.

ACKs for top commit:
  achow101:
    ACK b35e74ba37
  theStack:
    ACK b35e74ba37
  practicalswift:
    ACK b35e74ba37

Tree-SHA512: a73c1b336406a569e3bb10290618c5950b944db58ed0b05ff202d097684bb3ba3a5942c8d30443960052aa16438c054e2d02977b67aa901cce665c4df0ee5602
2020-08-31 10:24:18 +08:00
fanquake
f89b4f895f Merge #19830: test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles
fa1fc536bb test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles (MarcoFalke)

Pull request description:

  Fixes #19712

ACKs for top commit:
  practicalswift:
    ACK fa1fc536bb -- patch looks correct
  hebasto:
    ACK fa1fc536bb

Tree-SHA512: 24d6a4e871fda11196a9f88e2ddbd1c1461d895c503a04b103791233e46638421836200eaaa7d70689564e51dee0d68d32b880dd90a5c259fb6a906f21d07853
2020-08-31 09:59:21 +08:00
Jon Atack
54799b66b4 cli: add ipv6 and onion address type detection helpers 2020-08-30 22:37:45 +02:00
Jon Atack
12242b17a5 cli: create initial -netinfo option, NetinfoRequestHandler class 2020-08-30 22:37:06 +02:00
MarcoFalke
6eb97fb823 Merge #19839: Set appveyor VM version to previous Visual Studio 2019 release
3ec8f4c080 Set appveyor vm version to previous Visual Studio 2019 release. (Aaron Clauson)

Pull request description:

  The Visual Studio 2019 Appveyor [update](https://www.appveyor.com/updates/2020/08/29/) includes a bump of `cmake` to `3.18.2`.

  The `berkeleydb` dependency from the pegged `vcpkg` commit now fails to build with the `cmake` update.

  Setting the Appveyor VM back to the previous version should fix the immediate issue while a solution is identified for updating the `berkeleydb` build configuration.

ACKs for top commit:
  hebasto:
    ACK 3ec8f4c080, AppVeyor build is ok now.

Tree-SHA512: 61e10d0260c1c51dc6c552c857b3013f55ee324e26b1229bf7ef6f14c3e296762148db66a994ecc33fdc43e81592a7f8442d17b407964edb774b482d84735757
2020-08-30 19:27:09 +02:00
João Barbosa
413e0d1d31 Avoid callback when -blocknotify is empty 2020-08-30 17:38:27 +01:00
Aaron Clauson
3ec8f4c080 Set appveyor vm version to previous Visual Studio 2019 release.
Latest vm version has updated cmake and the berkeleydb vcpkg now fails to build.
2020-08-30 12:34:43 +01:00
MarcoFalke
cccc752569 rpc: Properly deserialize txs with witness before signing 2020-08-30 10:34:59 +02:00
Hennadii Stepanov
ea74e10acf doc: Add best practice for annotating/asserting locks 2020-08-29 20:46:47 +03:00
Anthony Towns
2ee7743fe7 sync.h: Make runtime lock checks require compile-time lock checks 2020-08-29 20:46:47 +03:00
Hennadii Stepanov
23d71d171e Do not hide compile-time thread safety warnings 2020-08-29 20:46:23 +03:00
Hennadii Stepanov
3ddc150857 Add missed thread safety annotations
This is needed for upcoming commit "sync.h: Make runtime lock checks
require compile-time lock checks" to pass.
2020-08-29 20:46:23 +03:00
Hennadii Stepanov
af9ea55a72 Use LockAssertion utility class instead of AssertLockHeld()
This change prepares for upcoming commit "Do not hide compile-time
thread safety warnings" by replacing AssertLockHeld() with
LockAssertion() where needed.
2020-08-29 20:43:23 +03:00
Hennadii Stepanov
1816327e53 p2p: Put disconnecting logs into BCLog::NET category 2020-08-29 17:22:30 +03:00
Wladimir J. van der Laan
4631dc5c57 Merge #18921: build: add stack-clash and control-flow protection options to hardening flags
b536813cef build: add -fstack-clash-protection to hardening flags (fanquake)
076183b36b build: add -fcf-protection=full to hardening options (fanquake)

Pull request description:

  Beginning with Ubuntu `19.10`, it's packaged GCC now has some additional hardening options enabled by default (in addition to existing defaults like `-fstack-protector-strong` and reducing the minimum ssp buffer size). The new additions are`-fcf-protection=full` and `-fstack-clash-protection`.

  > -fcf-protection=[full|branch|return|none]
  > Enable code instrumentation of control-flow transfers to increase program security by checking that target addresses of control-flow transfer instructions (such as indirect function call, function return, indirect jump) are valid. This prevents diverting the flow of control to an unexpected target. This is intended to protect against such threats as Return-oriented Programming (ROP), and similarly call/jmp-oriented programming (COP/JOP).

  > -fstack-clash-protection
  > Generate code to prevent stack clash style attacks. When this option is enabled, the compiler will only allocate one page of stack space at a time and each page is accessed immediately after allocation. Thus, it prevents allocations from jumping over any stack guard page provided by the operating system.

  If your interested you can grab `gcc-9_9.3.0-10ubuntu2.debian.tar.xz` from https://packages.ubuntu.com/focal/g++-9. The relevant changes are part of the `gcc-distro-specs` patches, along with the relevant additions to the gcc manages:

  > NOTE: In Ubuntu 19.10 and later versions, -fcf-protection is enabled by default for C, C++, ObjC, ObjC++, if none of -fno-cf-protection nor -fcf-protection=* are found.

  > NOTE: In Ubuntu 19.10 and later versions, -fstack-clash-protection is enabled by default for C, C++, ObjC, ObjC++, unless -fno-stack-clash-protection is found.

  So, if you're C++ using GCC on Ubuntu 19.10 or later, these options will be active unless you explicitly opt out. This can be observed with a small test:

  ```c++
  int main() { return 0; }
  ```

  ```bash
  g++ --version
  g++ (Ubuntu 9.3.0-10ubuntu2) 9.3.0

  g++ test.cpp

  objdump -dC a.out
  ..
  0000000000001129 <main>:
      1129:	f3 0f 1e fa          	endbr64
      112d:	55                   	push   %rbp
      112e:	48 89 e5             	mov    %rsp,%rbp
      1131:	b8 00 00 00 00       	mov    $0x0,%eax
      1136:	5d                   	pop    %rbp
      1137:	c3                   	retq
      1138:	0f 1f 84 00 00 00 00 	nopl   0x0(%rax,%rax,1)
      113f:	00

  # recompile opting out of control flow protection
  g++ test.cpp -fcf-protection=none

  objdump -dC a.out
  ...
  0000000000001129 <main>:
      1129:	55                   	push   %rbp
      112a:	48 89 e5             	mov    %rsp,%rbp
      112d:	b8 00 00 00 00       	mov    $0x0,%eax
      1132:	5d                   	pop    %rbp
      1133:	c3                   	retq
      1134:	66 2e 0f 1f 84 00 00 	nopw   %cs:0x0(%rax,%rax,1)
      113b:	00 00 00
      113e:	66 90                	xchg   %ax,%ax
  ```

  Note the insertion of an `endbr64` instruction when compiling and _not_ opting out. This instruction is part of the Intel Control-flow Enforcement Technology [spec](https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf), which the GCC control flow implementation is based on.

  If we're still doing gitian builds for the `0.21.0` and `0.22.0` releases, we'd likely update the gitian image to Ubuntu Focal, which would mean that the GCC used for gitian builds would also be using these options by default. So we should decide whether we want to explicitly turn these options on as part of our hardening options (although not just for this reason), or, we should be opting-out.

  GCC has supported both options since 8.0.0. Clang has supported `-fcf-protection` from 7.0.0 and will support `-fstack-clash-protection` in it's upcoming [11.0.0 release](https://clang.llvm.org/docs/ReleaseNotes.html#id6).

ACKs for top commit:
  jamesob:
    ACK b536813cef ([`jamesob/ackr/18921.1.fanquake.build_add_stack_clash_an`](https://github.com/jamesob/bitcoin/tree/ackr/18921.1.fanquake.build_add_stack_clash_an))
  laanwj:
    Code review ACK b536813cef

Tree-SHA512: abc9adf23cdf1be384f5fb9aa5bfffdda86b9ecd671064298d4cda0440828b509f070f9b19c88c7ce50ead9ff32afff9f14c5e78d75f01241568fbfa077be0b7
2020-08-29 13:42:04 +02:00
Sebastian Falbesoner
a5cfb40e27 doc: release note for changed {sign,verify}message error codes 2020-08-29 10:42:43 +02:00
Sebastian Falbesoner
9e399b9b2d test: check parameter validity in rpc_signmessage.py
checks parameter validity and error codes for the related RPCs:
    - signmessagewithprivkey
    - signmessage
    - verifymessage
2020-08-29 10:42:43 +02:00
Sebastian Falbesoner
e62f0c71f1 rpc: fix {sign,message}verify RPC errors for invalid address/signature 2020-08-29 10:42:43 +02:00
Hennadii Stepanov
0374e821bd util: Hard code previous release tarball checksums 2020-08-29 11:28:53 +03:00
Hennadii Stepanov
bd897ce79f scripted-diff: Move previous_release.py to test/get_previous_releases.py
-BEGIN VERIFY SCRIPT-
OLD=contrib/devtools/previous_release.py
NEW=test/get_previous_releases.py
sed -i "s|$OLD|$NEW|g" $(git grep -l $OLD)
git mv $OLD $NEW
-END VERIFY SCRIPT-
2020-08-29 11:26:25 +03:00
MarcoFalke
baf9cedee8 Merge #18817: doc: Document differences in bitcoind and bitcoin-qt locale handling
ca185cf5a1 doc: Document differences in bitcoind and bitcoin-qt locale handling (practicalswift)

Pull request description:

  Document differences in `bitcoind` and `bitcoin-qt` locale handling.

  Since this seems to be the root cause to the locale dependency issues we've seen over the years I thought it was worth documenting :)

  Note that 1.) `QLocale` (used by Qt), 2.) C locale (used by locale-sensitive C standard library functions/POSIX functions and some parts of the C++ standard library such as `std::to_string`) and 3.) C++ locale (used by the C++ input/output library) are three separate things. This comment is about the perhaps surprising interference with the C locale (2) that takes place as part of the Qt initialization.

ACKs for top commit:
  hebasto:
    re-ACK ca185cf5a1

Tree-SHA512: e51c32f3072c506b0029a001d8b108125e1acb4f2b6a48a6be721ddadda9da0ae77a9b39ff33f9d9eebabe2244c1db09e8502e3e7012d7a5d40d98e96da0dc44
2020-08-29 10:03:45 +02:00
MarcoFalke
fa1fc536bb test: Add tsan supp for leveldb::DBImpl::DeleteObsoleteFiles 2020-08-29 09:55:44 +02:00
practicalswift
ca185cf5a1 doc: Document differences in bitcoind and bitcoin-qt locale handling 2020-08-29 01:55:27 +00:00
Suhas Daftuar
675e55e013 Ignore unknown messages before VERACK 2020-08-28 17:17:07 -04:00
Wladimir J. van der Laan
1cf73fb8eb Merge #19607: [p2p] Add Peer struct for per-peer data in net processing
8e35bf5906 scripted-diff: rename misbehavior members (John Newbery)
1f96d2e673 [net processing] Move misbehavior tracking state to Peer (John Newbery)
7cd4159ac8 [net processing] Add Peer (John Newbery)
aba03359a6 [net processing] Remove CNodeState.name (John Newbery)

Pull request description:

  We currently have two structures for per-peer data:

  - `CNode` in net, which should just contain connection layer data (eg socket, send/recv buffers, etc), but currently also contains some application layer data (eg tx/block inventory).
  - `CNodeState` in net processing, which contains p2p application layer data, but requires cs_main to be locked for access.

  This PR adds a third struct `Peer`, which is for p2p application layer data, and doesn't require cs_main. Eventually all application layer data from `CNode` should be moved to `Peer`, and any data that doesn't strictly require cs_main should be moved from `CNodeState` to `Peer` (probably all of `CNodeState` eventually).

  `Peer` objects are stored as shared pointers in a net processing global map `g_peer_map`, which is protected by `g_peer_mutex`. To use a `Peer` object, `g_peer_mutex` is locked, a copy of the shared pointer is taken, and the lock is released. Individual members of `Peer` are protected by different mutexes that guard related data. The lifetime of the `Peer` object is managed by the shared_ptr refcount.

  This PR adds the `Peer` object and moves the misbehaving data from `CNodeState` to `Peer`. This allows us to immediately remove 15 `LOCK(cs_main)` instances.

  For more motivation see #19398

ACKs for top commit:
  laanwj:
    Code review ACK 8e35bf5906
  troygiorshev:
    reACK 8e35bf5906 via `git range-diff master 9510938 8e35bf5`
  theuni:
    ACK 8e35bf5906.
  jonatack:
    ACK 8e35bf5906 keeping in mind Cory's comment (https://github.com/bitcoin/bitcoin/pull/19607#discussion_r470173964) for the follow-up

Tree-SHA512: ad84a92b78fb34c9f43813ca3dfbc7282c887d55300ea2ce0994d134da3e0c7dbc44d54380e00b13bb75a57c28857ac3236bea9135467075d78026767a19e4b1
2020-08-28 20:29:16 +02:00
Jon Atack
7984c39be1 test framework: serialize/deserialize inv type as unsigned int 2020-08-28 20:12:02 +02:00
João Barbosa
b35e74ba37 wallet, refactor: Remove duplicate map lookups in GetAddressBalances 2020-08-28 17:01:06 +01:00
MarcoFalke
ca30d34cf9 Merge bitcoin-core/gui#39: Add visual accenting for the 'Create new receiving address' button
4ec49f8d1e qt: Leverage the default "Create new receiving address" button (Hennadii Stepanov)
4227a8e1f3 qt: Make "Create new receiving address" default unconditionally (Hennadii Stepanov)

Pull request description:

  Fix #24

  The first commit:
  - visual improvement with no behavior change

  The second commit:
  - removes a bunch of LOCs
  - slightly change behavior and makes it standard

  With this PR:
  ![DeepinScreenshot_select-area_20200721213040](https://user-images.githubusercontent.com/32963518/88093294-7b2a6700-cb9a-11ea-89a2-a0e2678056a7.png)

ACKs for top commit:
  Saibato:
    Concept tACK  4227a8e1f3 4ec49f8d1e
  promag:
    Tested ACK 4ec49f8d1e on macos.

Tree-SHA512: 3403d5ee96ec139491c7e23b24a24d9239fe55c58d99cbd4cd13bc877f76f992ed011c09e2af35b2a63be1a2371b95f6ac719325396dcc8333cf3eb7fa2e3d2c
2020-08-28 17:54:05 +02:00
MarcoFalke
5edef20a65 Merge #19797: net: Remove old check for 3-byte shifted IP addresses from pre-0.2.9 nodes
7b6d0f10a7 Remove old check for 3-byte shifted IP addresses from pre-0.2.9 node messages (Raúl Martínez (RME))

Pull request description:

  The change removes an old check for IPv6 addresses in range ::ff:ff00:0:0:0/72 that were created due to a bug in size field of addr messages for 0.2.8 nodes and before.

  This check is no longer needed as they are no more pre 0.2.9 nodes on the network (as per bitnodes network snapshot).

  Credits for discovering this go to sipa in https://github.com/bitcoin/bitcoin/pull/19628#discussion_r475907453

  Thanks for the attention!

ACKs for top commit:
  sipa:
    utACK 7b6d0f10a7
  vasild:
    ACK 7b6d0f1

Tree-SHA512: c5fab59dda2acafe143f607a4c5b636a54ac76fba651cad1ad1b09c94e88ab39503a31c2244c8f2664da68456c2a870c601d8894139c55cde9ece8161913ed2e
2020-08-28 17:51:37 +02:00
Wladimir J. van der Laan
9632b7edc7 Merge #19739: refactor: remove c-string interfaces for DecodeBase58{Check}
d3e8adfada util: remove c-string interfaces for DecodeBase58{Check} (Sebastian Falbesoner)

Pull request description:

  This micro-PR gets rid of base58 function interfaces that are redundant in terms of c-string / std::string variants; the c-string interface for `DecodeBase58Check` is completely unused outside the base58 module, while the c-string interface for `DecodeBase58` is only used in unit tests, where an implicit conversion to std::string is not problematic.

ACKs for top commit:
  practicalswift:
    ACK d3e8adfada -- patch looks correct
  laanwj:
    Code review ACK d3e8adfada

Tree-SHA512: 006a4a1e23b11385f60820c188b8e6b1634a182ca36e29a6580f72150214c65a3fdb273ec439165f26ba88a42d2bf5bab1cf3666a9eaee222fb4e1c00aeba433
2020-08-28 16:50:57 +02:00
Wladimir J. van der Laan
22acd36d53 Merge #19646: doc: Updated outdated help command for getblocktemplate
c91b241b48 Updated outdated help command for getblocktemplate (fixes #19625) (Jake Leventhal)

Pull request description:

  **Summary of Changes**
  * Removed coinbasetxn from the help outputs
  * Added the missing name for transactions in the help outputs
  * Added help outputs for longpollid and default_witness_commitment
  * Added more clarity to capabilities, rules, and coinbaseaux

  **Rationale**
  The outputs from the help command for `getblocktemplate` are outdated and don't reflect the actual results from `getblocktemplate` (see #19625 for more details)

  Fixes #19625.

ACKs for top commit:
  laanwj:
    ACK c91b241b48
  fjahr:
    utACK c91b241b48

Tree-SHA512: ee443af4bc3b2838dfd92e2705f344256ee785ae720e505fffea9b0ec5b75930e3b1374bae59b36d5da57c85c9aefe4d62504b028b893d6f2914dccf1e34c658
2020-08-28 15:24:16 +02:00
fanquake
1dac4dcf08 Merge #19758: Drop deprecated and unused GUARDED_VAR and PT_GUARDED_VAR annotations
9034f6e30e Drop deprecated and unused GUARDED_VAR and PT_GUARDED_VAR annotations (Hennadii Stepanov)

Pull request description:

  https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#guarded-var-and-pt-guarded-var:
  > `GUARDED_VAR` and `PT_GUARDED_VAR`
  > Use of these attributes has been deprecated.

ACKs for top commit:
  MarcoFalke:
    ACK 9034f6e30e They seem to be deprecated for a long time already https://releases.llvm.org/4.0.0/tools/clang/docs/ThreadSafetyAnalysis.html#guarded-var-and-pt-guarded-var

Tree-SHA512: d86f55fe57c28d91eda4a0ad727e36a5b35ba4b50a557c59b83cf0c5291cc5ad37b6f4ba6daeba3c1aba143faadaea6bb21c723f4d221856d6e6c42d228e8aa2
2020-08-28 20:36:40 +08:00
Raúl Martínez (RME)
7b6d0f10a7 Remove old check for 3-byte shifted IP addresses from pre-0.2.9 node messages
The change removes an old check for IPv6 addresses in range ::ff:ff00:0:0:0/72 that were created due to a bug in size field of addr messages for 0.2.8 nodes and before.

This check is no longer needed as they are no more pre 0.2.9 nodes on the network (as per bitnodes network snapshot).

Credits for discovering this go to sipa.
2020-08-28 14:17:28 +02:00
fanquake
4326515f01 Merge #19822: chain: Fix CChain comparison UB by removing it (it was unused)
df536883d2 chain: Remove UB CChain comparison (Carl Dong)

Pull request description:

  Comparing two empty `CChain`s is currently undefined behaviour, and resulted in false assertion failures when comparing identical empty `CChain`s in local testing.

  Let's just remove this comparison operator since it doesn't seem to be used anywhere.

ACKs for top commit:
  practicalswift:
    ACK df536883d2 -- patch is guaranteed to be correct :)
  MarcoFalke:
    cr ACK df536883d2

Tree-SHA512: db10bac364fc965b56abf7a5bac48018786b14806ffe107e3e8eb24d5004a29331f3387dfe3409a3452a6750d3329e3f354265d787ebb3abfccabe77b28a54d5
2020-08-28 19:44:47 +08:00
João Barbosa
772ea4844c wallet: Avoid recursive lock in IsTrusted 2020-08-28 10:42:18 +01:00
João Barbosa
819f10f671 wallet, refactor: Immutable CWalletTx::pwallet 2020-08-28 10:42:18 +01:00
MarcoFalke
fa48405ef8 Warn on unknown rw_settings 2020-08-28 11:39:26 +02:00
MarcoFalke
fa0572d0f3 Pass mempool reference to chainstate constructor 2020-08-28 10:42:04 +02:00
MarcoFalke
862fde88be Merge #19067: tests: Add fuzzing harness for CNode
cc26fab48d tests: Add fuzzing harness for CNode (practicalswift)

Pull request description:

  Add fuzzing harness for `CNode`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: e6330e5de5b2eb44d3bd91a885e69ebb625bfd1cb2499338aeb3997ff0268848434e651126fe68a8cadd7235c391e61a40d6408ee26e457faf73572e0c375f6b
2020-08-28 08:30:57 +02:00
Carl Dong
df536883d2 chain: Remove UB CChain comparison
It was unused, and had UB
2020-08-27 20:07:27 -04:00
MarcoFalke
15886b08aa Merge bitcoin-core/gui#40: Clarify block height label
b6dcc6d741 gui: Clarify block height label (Hennadii Stepanov)

Pull request description:

  Prefer "block height" instead of "number of blocks".

  This was done while testing https://github.com/bitcoin/bitcoin/pull/16981.

ACKs for top commit:
  michaelfolkson:
    ACK b6dcc6d741. I don't think there are any other obvious examples in the GUI where "block height" should replace "number of blocks" except for translations.
  MarcoFalke:
    cr ACK b6dcc6d741

Tree-SHA512: ec3b48c1af5d613ed657ad51f2caddea774376736ecc02343d54518986e35ec37f1745b059814b5be92b5e5c2bb2970d17159b24c6e88b9316803d4de5327c31
2020-08-27 20:33:00 +02:00
Russell Yanofsky
24bf17602c gui refactor: Inline SplashScreen::ConnectWallet
Suggested https://github.com/bitcoin/bitcoin/pull/19099#discussion_r450522201
2020-08-27 14:33:00 -04:00
Russell Yanofsky
e4f4350471 refactor: Move wallet methods out of chain.h and node.h
Add WalletClient interface so node interface is cleaner and don't need
wallet-specific methods.

The new NodeContext::wallet_client pointer will also be needed to eliminate
global wallet variables like ::vpwallets, because createWallet(), loadWallet(),
getWallets(), etc methods called by the GUI need a way to get a reference to
the list of open wallets if it is no longer a global variable.

Also tweaks splash screen registration for load wallet events to be delayed
until after wallet client is created.
2020-08-27 14:33:00 -04:00
Russell Yanofsky
b266b3e0bf refactor: Create interfaces earlier during initialization
Add AppInitInterfaces function so wallet chain and chain client interfaces are
created earlier during initialization. This is needed in the next commit to
allow the gui splash screen to be able to register for wallet events through a
dedicated WalletClient interface instead managing wallets indirectly through
the Node interface. This only works if the wallet client interface is created
before the splash screen needs to use it.
2020-08-27 14:33:00 -04:00
MarcoFalke
b987e657cd Merge #19169: rpc: Validate provided keys for query_options parameter in listunspent
a99a3c0bd6 rpc: Validate provided keys for query_options parameter in listunspent (pasta)

Pull request description:

  At Dash, one of our developers was working with the `listunspent` RPC command, but instead of saying "minimumAmount" he said "minimmumAmount" as such the RPC wasn't working as expected.

  In https://github.com/dashpay/dash/pull/3507 we implemented a check so that `listunspent` returns an error if an unrecognized option is given. I figured I might as well adapt the code and throw up a PR here.

  Cheers!

ACKs for top commit:
  adaminsky:
    ACK `a99a3c0bd`
  meshcollider:
    Seems fine to me. utACK a99a3c0bd6

Tree-SHA512: 9fccf14979849879a51b352afa3e1932ce4a6cfc2ee97b8d405ec6e65673fe94e302795e3ec0b440e6d252f13acda620e1f6a0e86c3fa918883c3fb4600a372c
2020-08-27 20:17:25 +02:00
practicalswift
cc26fab48d tests: Add fuzzing harness for CNode 2020-08-27 17:50:39 +00:00
MarcoFalke
fa1cd9e1dd test: Remove unused lock arg from BitcoinTestFramework.wait_until 2020-08-27 18:50:08 +02:00
MarcoFalke
fad2794e93 test: Rename wait until helper to wait_until_helper 2020-08-27 18:50:05 +02:00
Wladimir J. van der Laan
91af7ef831 Merge #19289: wallet: GetWalletTx and IsMine require cs_wallet lock
b8405b833a wallet: IsChange requires cs_wallet lock (João Barbosa)
d8441f30ff wallet: IsMine overloads require cs_wallet lock (João Barbosa)
a13cafc6c6 wallet: GetWalletTx requires cs_wallet lock (João Barbosa)

Pull request description:

  This change removes some unlock/lock and lock/lock cases regarding `GetWalletTx` and `IsMine` overloads.

ACKs for top commit:
  laanwj:
    Code review ACK b8405b833a
  ryanofsky:
    Code review ACK b8405b833a. Just new commit since last review changing IsChange GetChange locks and adding annotations

Tree-SHA512: 40d37c4fe5d10a1407f57d899d5822bb285633d8dbfad8afcf15a9b41b428ed9971a9a7b1aae84318371155132df3002699a15dab56e004527d50c889829187d
2020-08-27 16:21:37 +02:00
MarcoFalke
facb41bf1d test: Remove unused p2p_lock in VersionBitsWarningTest 2020-08-27 11:41:40 +02:00
Jon Atack
407175e0c2 p2p: change CInv::type from int to uint32_t
fixes issue #19678 UBSan implicit-integer-sign-change

Credit to Eugene (Crypt-iQ) for finding and reporting the issue
and to Vasil Dimov (vasild) for the original suggestion
2020-08-27 10:57:20 +02:00
fanquake
2562d5d238 Merge #19761: build: improve sed robustness by not using sed
3de365e4f1 build: replace wingenminiupnpcstrings sed with a patch in miniupnpc package (fanquake)
bbc01a753d build: replace qtranslations lrelease sed with a patch in qt package (fanquake)
c723e4176e build: replace FreeType back-compat sed with a patch in qt package (fanquake)
3aaa39d436 build: replace pwd sed in qt package with a patch (fanquake)
9d440f4e11 build: remove no-longer needed qt workaround (fanquake)
bf85eace1a build: remove no-longer needed qt configure workaround (fanquake)
4af59a407a build: use patch rather than sed in zeromq package (fanquake)
cc107a3af1 build: use patch rather than sed in native_cctools package (fanquake)
865cb23a48 build: use patch rather than sed in fontconfig package (fanquake)
335bd7f8bc build: use patch rather than sed in Boost package (fanquake)
f36140d00c build: use patch rather than sed in bdb package (fanquake)

Pull request description:

  While using `sed` can be handy to use for a quick-fix, these instances accumulate, and can become unmaintainable. Not only that, but using sed isn't necessarily robust and it can fail silently. Most of our usage is also missing any documentation explaining why something is being done, when it should be updated/removed etc.

  Rather than relying on sed going forward, where possible, I've converted our sed usage into patches. These are easier to maintain, contain documentation, and should fail loudly when they don't apply.

  The remaining sed usage, (1 in miniupnpc, the rest in qt), are non-trivial to remove, as they are using build-time variables, or some input from the environment.

  This also steals 2 related commits out of #19716.

  Related to #16838.

ACKs for top commit:
  hebasto:
    re-ACK 3de365e4f1, only `drop_lrelease_dependency.patch` updated. Travis makes ARM build without errors now.
  theuni:
    ACK 3de365e4f1.

Tree-SHA512: b39afcb237e4421f9caabbd665af93fd2e749a1cdd42b1d5ee2261059120005c0e82994d315e679c317d23794eab5c7727f51cae403c94a9c4e4fd7eee9e7ee6
2020-08-27 16:03:14 +08:00
Gleb Naumenko
81b00f8780 Add indexing ADDR cache by local socket addr 2020-08-27 10:51:56 +03:00
Gleb Naumenko
42ec558542 Justify the choice of ADDR cache lifetime 2020-08-27 10:51:56 +03:00
MarcoFalke
28f4e53e16 Merge #19752: test: Update wait_until usage in tests not to use the one from utils
d841301010 test: Add docstring to wait_until() in util.py to warn about its usage (Seleme Topuz)
1343c86c7c test: Update wait_until usage in tests not to use the one from utils (Seleme Topuz)

Pull request description:

  Replace global (from [test_framework/util.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/test_framework/util.py#L228)) `wait_until()` usages with the ones provided by `BitcoinTestFramework` and `P2PInterface` classes.

  The motivation behind this change is that the `util.wait_until()` expects a timeout, timeout_factor and lock and it is not aware of the context of the test framework. `BitcoinTestFramework` offers a `wait_until()` which has an understandable amount of default `timeout` and a shared `timeout_factor`. Moreover, on top of these, `mininode.wait_until()` also has a shared lock.

  closes #19080

ACKs for top commit:
  MarcoFalke:
    ACK d841301010 🦆
  kallewoof:
    utACK d841301010

Tree-SHA512: 81604f4cfa87fed98071a80e4afe940b3897fe65cf680a69619a93e97d45f25b313c12227de7040e19517fa9c003291b232f1b40b2567aba0148f22c23c47a88
2020-08-27 08:21:59 +02:00
MarcoFalke
30568d3f1e Merge #19778: test: Fix intermittent issue in wallet_bumpfee
fafc9d5af4 test: Fix intermittent issue in wallet_bumpfee (MarcoFalke)
fa347b2f25 test: Select at least the fee in wallet_bumpfee to avoid negative amounts (MarcoFalke)

Pull request description:

  With a "dirty" mempool a transaction might fail to be accepted intermittently. For example,

  * https://travis-ci.org/github/bitcoin-core/gui/jobs/719916499#L6773 Fails acceptance
  * https://travis-ci.org/github/bitcoin-core/gui/jobs/719916499#L6954 Test fails

  Fix the issue by clearing the mempool between subtests

ACKs for top commit:
  promag:
    Code review ACK fafc9d5af4.

Tree-SHA512: 23fb6decb6343d19eafddcbdb7da0551f6be11325d1c97c30e563944000aeb02bcc4b24904d204b132c093dc1acf28445fa1fd08bfe8d8b52ddd1de51c33eeb6
2020-08-26 19:25:38 +02:00
Hennadii Stepanov
705c1f0648 qt, refactor: Fix 'buttonClicked is deprecated' warnings 2020-08-26 20:22:42 +03:00
Hennadii Stepanov
c2f4e5ea1d qt, refactor: Fix 'split is deprecated' warnings 2020-08-26 20:22:42 +03:00
Hennadii Stepanov
8e12d69961 qt, refactor: Fix 'QFlags is deprecated' warnings 2020-08-26 20:22:42 +03:00
Hennadii Stepanov
fa5749c805 qt, refactor: Fix 'pixmap is deprecated' warnings 2020-08-26 20:17:52 +03:00
Hennadii Stepanov
b02264cb5d qt, refactor: Fix 'QDateTime is deprecated' warnings 2020-08-26 20:17:45 +03:00
Seleme Topuz
d841301010 test: Add docstring to wait_until() in util.py to warn about its usage 2020-08-26 18:01:59 +02:00
Seleme Topuz
1343c86c7c test: Update wait_until usage in tests not to use the one from utils
Replace "wait_until()" usage from utils, with the ones from BitcoinTestFramework and P2PInterface.
closes #19080
2020-08-26 18:01:59 +02:00
MarcoFalke
93ab136a33 Merge bitcoin-core/gui#35: Parse params directly instead of through node (partial revert #10244)
519cae8fd6 gui: Delay interfaces::Node initialization (Russell Yanofsky)
102abff9eb gui: Replace interface::Node references with pointers (Russell Yanofsky)
91aced7c7e gui: Remove unused interfaces::Node references (Russell Yanofsky)
e133631625 gui: Partially revert #10244 gArgs and Params changes (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This is a partial revert of https://github.com/bitcoin/bitcoin/pull/10244. It changes gui code to go back to using gArgs and Params() functions directly instead of using interfaces::Node to handle arguments.

  These changes were originally pushed as part of https://github.com/bitcoin/bitcoin/pull/19461. Motivation is to support a new GUI process connecting to an already running node process. Details are explained in commit messages, but in addition to spawning a new bitcoin-node process, we want bitcoin-gui to connect to an existing bitcoin-node process. So for that reason it should be able to parse its own parameters, rather than rely on the node.

ACKs for top commit:
  MarcoFalke:
    re-ACK 519cae8fd6, only change is rebase and addressed nits of my previous review last week 🌄

Tree-SHA512: 9c339dd82ba78bcc7b887b84d872f35ccc7dfa3d271691e6eafe8a2048cbbe3bdde1e810ce33d0714d75d048c9de3470e9e9b6f8306a6047d1cb3548f6858dc8
2020-08-26 16:32:44 +02:00
Hennadii Stepanov
3edc4e34fe bench: Prevent thread oversubscription
This change decreases the variance of benchmark results.
2020-08-26 13:00:00 +03:00
John Newbery
fb56d37612 p2p: ensure inv is GenMsgTx before ToGenTxid in inv processing
and otherwise log that an unknown INV type was received.

In INV processing, when handling transaction type inv messages,
ToGenTxid() expects that we constructed the CInv ourselves or
that we verified that it is for a transaction type CInv.

Therefore, change this `else` branch into an `else if (inv.GenMsgTx())`
to make this safer and log any INVs that fall through.
2020-08-26 11:57:30 +02:00
Jon Atack
aa3621385e test: use CInv::MSG_WITNESS_TX flag in p2p_segwit 2020-08-26 11:57:27 +02:00
Jon Atack
24ee4f01ea p2p: make gtxid(.hash) and fAlreadyHave localvars const 2020-08-26 11:57:23 +02:00
Jon Atack
b1c855453b p2p: use CInv block message helpers in net_processing.cpp 2020-08-26 11:57:19 +02:00
John Newbery
acd6642167 [net processing] Change AlreadyHaveTx() to take a GenTxid 2020-08-26 11:57:15 +02:00
John Newbery
5fdfb80b86 [net processing] Change AlreadyHaveBlock() to take block_hash argument 2020-08-26 11:57:11 +02:00
John Newbery
430e183b89 [net processing] Remove mempool argument from AlreadyHaveBlock() 2020-08-26 11:57:07 +02:00
John Newbery
42ca5618ca [net processing] Split AlreadyHave() into separate block and tx functions 2020-08-26 11:57:03 +02:00
Jon Atack
39f1dc9445 p2p: remove nFetchFlags from NetMsgType TX and INV processing
The nFetchFlags code can be removed here because GetFetchFlags() can only add
the MSG_WITNESS_FLAG, which is added to the CInv::type field. That CInv is only
passed to AlreadyHave() or ToGenTxid(), and neither of those functions do
anything different depending on whether the CInv type is MSG_TX or
MSG_WITNESS_TX.

Co-authored by: John Newbery <john@johnnewbery.com>
2020-08-26 11:56:59 +02:00
Jon Atack
471714e1f0 p2p: add CInv block message helper methods 2020-08-26 11:56:55 +02:00
MarcoFalke
a12d9e5fd2 Merge #19687: refactor: make EncodeBase{32,64} consume Spans
e2aa1a585a util: make EncodeBase64 consume Spans (Sebastian Falbesoner)
2bc207190e util: make EncodeBase32 consume Spans (Sebastian Falbesoner)

Pull request description:

  To simplify the interface of the Base32/Base64 encoding functions for raw data, this PR changes them from taking two arguments (pointer and length) to just one Span. Most calls to `EncodeBase64` pass data from `CDataStream` instances, which unfortunately internally work with `char*` pointers rather than `unsigned char*`, but thanks to the recently introduced `MakeUCharSpan` helper, converting them is quite easy.

ACKs for top commit:
  MarcoFalke:
    ACK e2aa1a585a 🐮
  vasild:
    ACK e2aa1a585

Tree-SHA512: 43bd3bd2ee8e3be2474db0a81dae9d9e88fac2464b96d1b042147106ed7433799dcba3000c69990511ecfc697b0c7306ce85f2ecb2293e2e44fd356c9694b150
2020-08-26 11:52:31 +02:00
Russell Yanofsky
519cae8fd6 gui: Delay interfaces::Node initialization
This is needed to allow bitcoin-gui to connect to existing node process with
-ipcconnect instead of spawning a new process. It's possible to spawn a new
bitcoin-node process without knowing the current data dir or network, but
connecting to an existing bitcoin-node requires knowing the datadir and network
first.
2020-08-26 05:52:31 -04:00
Russell Yanofsky
102abff9eb gui: Replace interface::Node references with pointers
No change in behavior. Replacing references with pointers allows Node interface
creation to be delayed until later during gui startup next commit to support
implementing -ipcconnect option
2020-08-26 05:52:31 -04:00
Russell Yanofsky
91aced7c7e gui: Remove unused interfaces::Node references
Remove Node references no longer needed after previous commit
2020-08-26 05:52:31 -04:00
Russell Yanofsky
e133631625 gui: Partially revert #10244 gArgs and Params changes
Change gui code to use gArgs, Params() functions directly instead of going
through interfaces::Node.

Remotely accessing bitcoin-node ArgsManager from bitcoin-gui works fine in
https://github.com/bitcoin/bitcoin/pull/10102, when bitcoin-gui spawns a new
bitcoin-node process and controls its startup, but for bitcoin-gui to support
-ipcconnect option in https://github.com/bitcoin/bitcoin/pull/19461 and connect
to an existing bitcoin-node process, it needs ability to parse arguments itself
before connecting out.

This change also simplifies https://github.com/bitcoin/bitcoin/pull/10102 a
bit, by making the bitcoin-gui -> bitcoin-node startup sequence more similar to
the bitcoin-node -> bitcoin-wallet startup sequence where the parent process
parses arguments and passes them to the child process instead of the parent
process using the child process to parse arguments.
2020-08-26 05:52:31 -04:00
fanquake
e80e5b3e4f Merge #19760: test: Remove confusing mininode terminology
d5800da519 [test] Remove final references to mininode (John Newbery)
5e8df3312e test: resort imports (John Newbery)
85165d4332 scripted-diff: Rename mininode to p2p (John Newbery)
9e2897d020 scripted-diff: Rename mininode_lock to p2p_lock (John Newbery)

Pull request description:

  New contributors are often confused by the terminology in the test framework, and what the difference between a _node_ and a _peer_ is. To summarize:

  - a 'node' is a bitcoind instance. This is the thing whose behavior is being tested. Each bitcoind node is managed by a python `TestNode` object which is used to start/stop the node, manage the node's data directory, read state about the node (eg process status, log file), and interact with the node over different interfaces.
  - one of the interfaces that we can use to interact with the node is the p2p interface. Each connection to a node using this interface is managed by a python `P2PInterface` or derived object (which is owned by the `TestNode` object). We can open zero, one or many p2p connections to each bitcoind node. The node sees these connections as 'peers'.

  For historic reasons, the word 'mininode' has been used to refer to those p2p interface objects that we use to connect to the bitcoind node (the code was originally taken from the 'mini-node' branch of https://github.com/jgarzik/pynode/tree/mini-node). However that name has proved to be confusing for new contributors, so rename the remaining references.

ACKs for top commit:
  amitiuttarwar:
    ACK d5800da519
  MarcoFalke:
    ACK d5800da519 🚞

Tree-SHA512: 2c46c2ac3c4278b6e3c647cfd8108428a41e80788fc4f0e386e5b0c47675bc687d94779496c09a3e5ea1319617295be10c422adeeff2d2bd68378e00e0eeb5de
2020-08-26 15:43:17 +08:00
fanquake
6a2ba62685 Merge #19779: Remove gArgs global from init
fa9d5902f7 scripted-diff: gArgs -> args (MarcoFalke)
fa33bc2dab init: Capture copy of blocknotify setting for BlockNotifyCallback (MarcoFalke)
fa40017706 init: Pass reference to ArgsManager around instead of relying on global (MarcoFalke)

Pull request description:

  The gArgs global has several issues:

  * gArgs is used by each process (bitcoind, bitcoin-qt, bitcoin-wallet, bitcoin-cli, bitcoin-tx, ...), but it is hard to determine which arguments are actually used by each process. For example arguments that have never been registered, but are still used, will always return the fallback value.
  * Tests may run several sub-tests, which need different settings. So globals will have to be overwritten, but that is fragile on its own: e.g. https://github.com/bitcoin/bitcoin/pull/19704#issuecomment-678259092 or #19511

  The goal is to remove gArgs, but as a first step in that direction this pull will change gArgs in init to use a passed-in reference instead.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa9d5902f7. Looks good. Nice day to remove some globals, and add some lambdas 👍
  fanquake:
    ACK fa9d5902f7 - I'm not as familiar with the settings & argument handling code, but this make sense, and is a step in the right direction towards a reduction in the usage of globals. Not a huge fan of the clang-formatting in the scripted diff.
  jonasschnelli:
    Concept ACK fa9d5902f7

Tree-SHA512: ed00db5f826566c7e3b4d0b3d2ee0fc1a49a6e748e04e5c93bdd694ac7da5598749e73937047d5fce86150d764a067d2ca344ba4ae3eb2704cc5c4fa0d20940f
2020-08-26 15:18:38 +08:00
fanquake
92735e45ba Merge #19775: test: Activate segwit in TestChain100Setup
fad84b7e14 test: Activate segwit in TestChain100Setup (MarcoFalke)
fa11ff2980 test: Pass empty tx pool to block assembler (MarcoFalke)
fa96574b0d test: Move doxygen comment to header (MarcoFalke)

Pull request description:

  This fixes not only a TODO in the code, but also prevents a never ending source of uninitialized reads. E.g.

  * #18376
  * https://github.com/bitcoin/bitcoin/pull/19704#issuecomment-678259092
  * ...

ACKs for top commit:
  jnewbery:
    utACK fad84b7e14

Tree-SHA512: 64cf16a59656d49e022b603f3b06441ceae35a33a4253b4382bc8a89a56e08ad5412c8fa734d0fc7b58586f40ea6d57b348a3b4838bc6890a41ae2ec3902e378
2020-08-26 13:17:35 +08:00
fanquake
3de365e4f1 build: replace wingenminiupnpcstrings sed with a patch in miniupnpc package
We should be able to drop this once we are using 2.1 or later. See
upstream commit: 9663c55c61408fdcc39a82987d2243f816b22932.
2020-08-26 11:27:31 +08:00
fanquake
bbc01a753d build: replace qtranslations lrelease sed with a patch in qt package 2020-08-26 11:27:12 +08:00
Andrew Chow
0bbe26a1af wallet: filter for keys only before record deser in salvage
When salvaging a wallet, avoid deserializing any records that we don't
care about, i.e. filter for keys only before the deserialization.
2020-08-25 13:23:40 -04:00
Andrew Chow
544e12a4e8 walletdb: Add KeyFilterFn to ReadKeyValue
Add a KeyFilterFn callback to ReadKeyValue which allows the caller to
specify which types to actually deserialize. A KeyFilterFn takes the
type as the parameter and returns a bool indicating whether
deserialization should continue.
2020-08-25 13:23:40 -04:00
Sebastian Falbesoner
e2aa1a585a util: make EncodeBase64 consume Spans 2020-08-25 18:52:57 +02:00
Sebastian Falbesoner
2bc207190e util: make EncodeBase32 consume Spans 2020-08-25 18:52:51 +02:00
Luke Dashjr
ef712298c3 util: Check for file being NULL in DirectoryCommit 2020-08-25 16:46:46 +00:00
Evan Klitzke
4574904038 Fix possible data race when committing block files
It was recently pointed out to me that calling fsync() or fdatasync() on a new
file is not sufficient to ensure it's persisted to disk, a the existence of the
file itself is stored in the directory inode. This means that ensuring that a
new file is actually committed also requires an fsync() on the parent directory.

This change ensures that we call fsync() on the blocks directory after
committing new block files.
2020-08-25 16:46:46 +00:00
Evan Klitzke
220bb16cbe util: Introduce DirectoryCommit commit function to sync a directory 2020-08-25 16:46:46 +00:00
Evan Klitzke
ce5cbaea63 util.h: Document FileCommit function 2020-08-25 16:46:46 +00:00
Evan Klitzke
844d650eea util: Prefer Mac-specific F_FULLSYNC over fdatasync in FileCommit 2020-08-25 16:46:45 +00:00
Evan Klitzke
f6cec0bcaf util: Refactor FileCommit from an #if sequence nested in #else, to a sequence of #elif
This should not change the actual code generation.
2020-08-25 16:46:20 +00:00
Luke Dashjr
c4b85ba704 Bugfix: Define and use HAVE_FDATASYNC correctly outside LevelDB 2020-08-25 16:46:00 +00:00
MarcoFalke
8d6224fefe Merge #19628: net: change CNetAddr::ip to have flexible size
102867c587 net: change CNetAddr::ip to have flexible size (Vasil Dimov)
1ea57ad674 net: don't accept non-left-contiguous netmasks (Vasil Dimov)

Pull request description:

  (chopped off from #19031 to ease review)

  Before this change `CNetAddr::ip` was a fixed-size array of 16 bytes,
  not being able to store larger addresses (e.g. TORv3) and encoded
  smaller ones as 16-byte IPv6 addresses.

  Change its type to `prevector`, so that it can hold larger addresses and
  do not disguise non-IPv6 addresses as IPv6. So the IPv4 address
  `1.2.3.4` is now encoded as `01020304` instead of
  `00000000000000000000FFFF01020304`.

  Rename `CNetAddr::ip` to `CNetAddr::m_addr` because it is not an "IP" or
  "IP address" (TOR addresses are not IP addresses).

  In order to preserve backward compatibility with serialization (where
  e.g. `1.2.3.4` is serialized as `00000000000000000000FFFF01020304`)
  introduce `CNetAddr` dedicated legacy serialize/unserialize methods.

  Adjust `CSubNet` accordingly. Still use `CSubNet::netmask[]` of fixed 16
  bytes, but use the first 4 for IPv4 (not the last 4). Do not accept
  invalid netmasks that have 0-bits followed by 1-bits and only allow
  subnetting for IPv4 and IPv6.

  Co-authored-by: Carl Dong <contact@carldong.me>

ACKs for top commit:
  sipa:
    utACK 102867c587
  MarcoFalke:
    Concept ACK 102867c587
  ryanofsky:
    Code review ACK 102867c587. Just many suggested updates since last review. Thanks for following up on everything!
  jonatack:
    re-ACK 102867c587 diff review, code review, build/tests/running bitcoind with ipv4/ipv6/onion peers
  kallewoof:
    ACK 102867c587

Tree-SHA512: d60bf716cecf8d3e8146d2f90f897ebe956befb16f711a24cfe680024c5afc758fb9e4a0a22066b42f7630d52cf916318bedbcbc069ae07092d5250a11e8f762
2020-08-25 18:10:25 +02:00
MarcoFalke
fa39c62eb7 test: inline hashToHex 2020-08-25 14:27:25 +02:00
fanquake
f8462a6d27 Merge #19601: Refactoring CHashWriter & Get{Prevouts,Sequence,Outputs}Hash to SHA256 (Alternative to #18071)
9ab4cafabd Refactor Get{Prevout,Sequence,Outputs}Hash to Get{Prevouts,Sequences,Outputs}SHA256. (Jeremy Rubin)
6510d0ff41 Add SHA256Uint256 helper functions (Jeremy Rubin)
b475d7d0fa Add single sha256 call to CHashWriter (Jeremy Rubin)

Pull request description:

  Opened as an alternative to #18071 to be more similar to #17977.

  I'm fine with either, deferring to others.

  cc jnewbery Sjors

ACKs for top commit:
  jnewbery:
    Code review ACK 9ab4cafabd
  jonatack:
    Tested ACK 9ab4caf
  fjahr:
    tested ACK 9ab4cafabd
  instagibbs:
    reACK 9ab4cafabd

Tree-SHA512: 93a7a47697f1657f027b18407bdcce16963f6b23d12372e7ac8fd4ee96769b3e2639369f9956fee669cc881b6338641cddfeeef1516c7104cb50ef4b880bb0a7
2020-08-25 20:18:40 +08:00
John Newbery
d5800da519 [test] Remove final references to mininode 2020-08-25 10:04:25 +01:00
fanquake
c723e4176e build: replace FreeType back-compat sed with a patch in qt package 2020-08-25 14:37:45 +08:00
fanquake
3aaa39d436 build: replace pwd sed in qt package with a patch 2020-08-25 14:37:45 +08:00
fanquake
9d440f4e11 build: remove no-longer needed qt workaround 2020-08-25 14:37:45 +08:00
fanquake
bf85eace1a build: remove no-longer needed qt configure workaround
This was fixed upstream in c45595d64831990311f92fcebc4e34e2797f5352.
2020-08-25 14:37:44 +08:00
fanquake
4af59a407a build: use patch rather than sed in zeromq package 2020-08-25 14:37:44 +08:00
fanquake
cc107a3af1 build: use patch rather than sed in native_cctools package 2020-08-25 14:37:44 +08:00
fanquake
865cb23a48 build: use patch rather than sed in fontconfig package 2020-08-25 14:37:35 +08:00
fanquake
335bd7f8bc build: use patch rather than sed in Boost package
The depends comment is actually incorrect, and this can be dropped once
we move to 1.71.0 or later.
2020-08-25 13:19:56 +08:00
fanquake
f36140d00c build: use patch rather than sed in bdb package 2020-08-25 13:19:55 +08:00
fanquake
8e0f341779 Merge #15704: Move Win32 defines to configure.ac to ensure they are globally defined
1ccb9f30c0 Move Win32 defines to configure.ac to ensure they are globally defined (Luke Dashjr)

Pull request description:

  #9245 no longer needs this, since the main `_WIN32_WINNT` got bumped by something else.

  So rather than just lose it, might as well get it merged in independently.

  I'm not aware of any practical effects, but it seems safer to use the same API versions everywhere.

ACKs for top commit:
  fanquake:
    ACK 1ccb9f30c0 - checked that the binaries produced are the same.

Tree-SHA512: 273e9186579197be01b443b6968e26b9a8031d356fabc5b73aa967fcdb837df195b7ce0fc4e4529c85d9b86da6f2d7ff1bf56a3ff0cbbcd8cee8a9c2bf70a244
2020-08-25 11:52:52 +08:00
fanquake
c6b730dbfc Merge #18405: build: Drop all of the ZeroMQ patches
f642b49af7 build: Drop ZeroMQ patch for glibc < 2.12 (Hennadii Stepanov)
079df9609e build: Drop ZeroMQ patch for Mingw-w64 < 4.0 (Hennadii Stepanov)

Pull request description:

  This PR gets rid of the all patches for ZeroMQ:
  - the [Mingw-w64 5.0 (Ubuntu 18.04 bionic)](https://packages.ubuntu.com/bionic/mingw-w64) is used to build the [Windows](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md) binaries
  - it is safe to use `pthread_setname_np` since #17538 when the minimal `glibc` version is set to 2.17; see: https://github.com/bitcoin/bitcoin/pull/11986#issuecomment-366105050

ACKs for top commit:
  fanquake:
    ACK f642b49af7.

Tree-SHA512: a2c97cdb682cd7d96a666ad099f20725a32bf8fda0842fc5534ca08a1634c90ba80afde92f9054595ed2501fbc5c02abbe3da765934ecff12d836ff25e277fc5
2020-08-25 10:26:16 +08:00
Vasil Dimov
102867c587 net: change CNetAddr::ip to have flexible size
Before this change `CNetAddr::ip` was a fixed-size array of 16 bytes,
not being able to store larger addresses (e.g. TORv3) and encoded
smaller ones as 16-byte IPv6 addresses.

Change its type to `prevector`, so that it can hold larger addresses and
do not disguise non-IPv6 addresses as IPv6. So the IPv4 address
`1.2.3.4` is now encoded as `01020304` instead of
`00000000000000000000FFFF01020304`.

Rename `CNetAddr::ip` to `CNetAddr::m_addr` because it is not an "IP" or
"IP address" (TOR addresses are not IP addresses).

In order to preserve backward compatibility with serialization (where
e.g. `1.2.3.4` is serialized as `00000000000000000000FFFF01020304`)
introduce `CNetAddr` dedicated legacy serialize/unserialize methods.

Adjust `CSubNet` accordingly. Still use `CSubNet::netmask[]` of fixed 16
bytes, but use the first 4 for IPv4 (not the last 4). Only allow
subnetting for IPv4 and IPv6.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-08-24 21:50:59 +02:00
Vasil Dimov
1ea57ad674 net: don't accept non-left-contiguous netmasks
A netmask that contains 1-bits after 0-bits (the 1-bits are not
contiguous on the left side) is invalid [1] [2].

The code before this PR used to parse and accept such
non-left-contiguous netmasks. However, a coming change that will alter
`CNetAddr::ip` to have flexible size would make juggling with such
netmasks more difficult, thus drop support for those.

[1] https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#Subnet_masks
[2] https://tools.ietf.org/html/rfc4632#section-5.1
2020-08-24 21:50:59 +02:00
Jon Atack
581b343d5b Add in/out connections to cli -getinfo 2020-08-24 18:41:24 +02:00
Jon Atack
d9cc13e88d UNIX_EPOCH_TIME fixup in rpc getnettotals 2020-08-24 18:41:22 +02:00
Jon Atack
1ab49b81cf Add in/out connections to rpc getnetworkinfo 2020-08-24 18:41:14 +02:00
Wladimir J. van der Laan
7f609f68d8 Merge #19731: net, rpc: expose nLastBlockTime/nLastTXTime as last block/last_transaction in getpeerinfo
5da96210fc doc: release note for getpeerinfo last_block/last_transaction (Jon Atack)
cfef5a2c98 test: rpc_net.py logging and test naming improvements (Jon Atack)
21c57bacda test: getpeerinfo last_block and last_transaction tests (Jon Atack)
8a560a7d57 rpc: expose nLastBlockTime/TXTime as getpeerinfo last_block/transaction (Jon Atack)
02fbe3ae0b net: add nLastBlockTime/TXTime to CNodeStats, CNode::copyStats (Jon Atack)

Pull request description:

  This PR adds inbound peer eviction criteria `nLastBlockTime` and `nLastTXTime` to `CNodeStats` and `CNode::copyStats`, which then allows exposing them in the next commit as `last_transaction` and `last_block` Unix Epoch Time fields in RPC `getpeerinfo`.

  This may be useful for writing missing eviction tests. I'd also like to add `lasttx` and `lastblk` columns to the `-netinfo` dashboard as described in https://github.com/bitcoin/bitcoin/pull/19643#issuecomment-671093420.

  Relevant discussion at the p2p irc meeting http://www.erisian.com.au/bitcoin-core-dev/log-2020-08-11.html#l-549:
  ```text
  <jonatack> i was specifically trying to observe and figure out how to test https://github.com/bitcoin/bitcoin/issues/19500
  <jonatack> which made me realise that i didn't know what was going on with my peer conns in enough detail
  <jonatack> i'm running bitcoin locally with nLastBlockTime and nLastTXTime added to getpeerinfo for my peer connections dashboard
  <jonatack> sipa: is there a good reason why that (eviction criteria) data is not exposed through getpeerinfo currently?
  <sipa> jonatack: nope; i suspect just nobody ever added it
  <jonatack> sipa: thanks. will propose.
  ```

  The last commit is optional, but I think it would be good to have logging in `rpc_net.py`.

ACKs for top commit:
  jnewbery:
    Code review ACK 5da96210fc
  theStack:
    Code Review ACK 5da96210fc
  darosior:
    ACK 5da96210fc

Tree-SHA512: 2db164bc979c014837a676e890869a128beb7cf40114853239e7280f57e768bcb43bff6c1ea76a61556212135281863b5290b50ff9d24fce16c5b89b55d4cd70
2020-08-24 17:03:07 +02:00
fanquake
4fefd80f08 Merge #19704: Net processing: move ProcessMessage() to PeerLogicValidation
daed542a12 [net_processing] Move ProcessMessage to PeerLogicValidation (John Newbery)
c556770b5e [net_processing] Change PeerLogicValidation to hold a connman reference (John Newbery)

Pull request description:

  Rather than ProcessMessage() being a static function in net_processing.cpp, make it a private member function of PeerLogicValidation. This is the start of moving static functions and global variables into PeerLogicValidation to make it better encapsulated.

ACKs for top commit:
  jonatack:
    ACK daed542a12 code review and debug tested
  promag:
    Code review ACK daed542a12.
  MarcoFalke:
    re-ACK daed542a12, only change is removing second commit 🎴
  theStack:
    Code Review ACK daed542a12

Tree-SHA512: ddebf410d114d9ad5a9e536950018ff333a347c035d74fcc101fb4a3f20a281782c7eac2b7d1bd1c8f6bc7e59f5b5630fb52c2e1b4c32df454fa584673bd021e
2020-08-24 21:50:37 +08:00
fanquake
7ce7f2b251 Merge #17396: build: modest Android improvements
366913e307 build: AX_BOOST_THREAD serial 33 (Igor Cota)
cf0681133a build: disable D-Bus on Android by default (Igor Cota)

Pull request description:

  I've been trying to build for Android on different OSes/Gitian with varying success. Build system is quite the beast and sometimes it doesn't get it right. To make sure it does these three little tweaks make the Android build more robust:

  - disable D-Bus (Android doesn't support it and has its own way to trigger notifications)
  - don't flag `-lpthread` when linking Boost, [Bionic has built-in support](https://stackoverflow.com/questions/30801752/android-ndk-and-pthread)
  - ~~add `-static-libstdc++` to linker flags. This avoids having to bundle `libc++_shared` with CLI apps, still necessary with `bitcoin-qt` though (thanks Sjors)~~

  I think these are small and fairly straightforward so I put them all into this one PR.

ACKs for top commit:
  fanquake:
    ACK 366913e307

Tree-SHA512: 31465fd228a5877c20aa2a05f98242d4eeb328b9b35bd1a7a3dcfb1ef51379d84053a81ade5a65436ffc1bc8ccd21f11ed0539eb10e827d182c0c04394629af0
2020-08-24 21:27:29 +08:00
MarcoFalke
fafc9d5af4 test: Fix intermittent issue in wallet_bumpfee 2020-08-24 13:49:28 +02:00
MarcoFalke
fa347b2f25 test: Select at least the fee in wallet_bumpfee to avoid negative amounts 2020-08-24 13:44:07 +02:00
fanquake
8e94275f86 Merge #19786: doc: Remove label from good first issue template
fa30d5282c doc: Remove label from good first issue template (MarcoFalke)

Pull request description:

  Good first issues aren't that frequent that manually assigning the label is a problem, but this fixes the spam problem (e.g. https://twitter.com/GoodFirstIssues/status/1295455089491161088 )

ACKs for top commit:
  jnewbery:
    ACK fa30d5282c

Tree-SHA512: 59e7c707637cc328e2443c2b7e5d2c82ef151739ad5afb6cea1a60501318dc8c4c81c95591eed8172581ac99d43cf826dcdd547e096eff1038137853af67a975
2020-08-24 19:32:44 +08:00
MarcoFalke
fa30d5282c doc: Remove label from good first issue template 2020-08-24 09:31:24 +02:00
MarcoFalke
1d53d72948 Merge #19659: Add a seed corpus generation option to the fuzzing test_runner
15ae4a17c4 test/fuzz: add a seed corpus generation option to the test_runner (Antoine Poinsot)

Pull request description:

  This adds a startup option to test/fuzz/test_runner.py which allows to generate seed corpus to the passed `seed_dir` instead of using them.

ACKs for top commit:
  MarcoFalke:
    ACK 15ae4a17c4

Tree-SHA512: f80ad58e48cc45272eace33dbf377848f31cbd6a25433786d50e9700f70185dff6513f71d885d0727ed57a2aa49163bfbdbc51a8091e99b4b1bae71e1504e6a5
2020-08-24 08:02:27 +02:00
MarcoFalke
fa9d5902f7 scripted-diff: gArgs -> args
-BEGIN VERIFY SCRIPT-
 # Replace gArgs with args
 sed -i 's/\<gArgs\>/args/g' src/init.cpp src/bitcoind.cpp
 sed -i 's/&args;/\&gArgs;/g' src/init.cpp

 # Format changed lines
 git diff -U0 | clang-format-diff -p1 -i -v
-END VERIFY SCRIPT-
2020-08-24 07:52:17 +02:00
MarcoFalke
fa33bc2dab init: Capture copy of blocknotify setting for BlockNotifyCallback
Can be reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2020-08-24 07:51:48 +02:00
MarcoFalke
fa40017706 init: Pass reference to ArgsManager around instead of relying on global 2020-08-24 07:45:17 +02:00
fanquake
38c13a4a60 Merge #19689: build: Add Qt version checking
4af4672525 build, qt: Add Qt version checking (Hennadii Stepanov)
30e336f785 build: Drop unused bitcoin_cv_qt58 (Hennadii Stepanov)

Pull request description:

  Now `configure` script checks that Qt version is not less then minimum required (currently [5.5.1](https://github.com/bitcoin/bitcoin/pull/15393)).

  This PR is an alternative to #15706 (see https://github.com/bitcoin/bitcoin/pull/15706#issuecomment-629076962).

  Closes #15688.

  The first commit removes dead code (see https://github.com/bitcoin/bitcoin/pull/18297#issuecomment-603252662).

ACKs for top commit:
  fanquake:
    ACK 4af4672525 - this looks ok. I've tested this with Qt 5.15.0 and Qt 5.7.1 system libs, as well as 5.9.8 from depends.

Tree-SHA512: 8e3b82fa3a98926814923331038185633fabad962c271f31bd158e1ab293dcde52ab1dbf997745540a9ed27e16835cf5b5f3701d405876d877fa561eb03cc619
2020-08-24 12:14:18 +08:00
Antoine Poinsot
15ae4a17c4 test/fuzz: add a seed corpus generation option to the test_runner
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-23 23:09:26 +02:00
Hennadii Stepanov
8071c75d45 qt, refactor: Limit scope of QPainter object 2020-08-23 23:47:09 +03:00
Sebastian Falbesoner
638441928a test: add parameterized constructor for msg_sendcmpct() 2020-08-23 02:27:09 +02:00
MarcoFalke
197450f808 Merge #19777: docs: Correct description for getblockstats's txs field
4148f55dd0 docs: Correct description for getblockstats's txs field (Nadav Ivgi)

Pull request description:

  It does count the coinbase transaction.

  Refs #19766

ACKs for top commit:
  MarcoFalke:
    ACK 4148f55dd0
  theStack:
    ACK 4148f55dd0

Tree-SHA512: ccd420f19242efbbbecfe822c825363bc89e26618834de0d805f5cdb07461c8bdc6e077c61ea8cd0d40564a96c67d8a71c68175c8543bb849909d7ae375b2a92
2020-08-22 20:50:14 +02:00
MarcoFalke
fad84b7e14 test: Activate segwit in TestChain100Setup 2020-08-21 18:44:52 +02:00
MarcoFalke
fa11ff2980 test: Pass empty tx pool to block assembler 2020-08-21 18:44:50 +02:00
MarcoFalke
fa96574b0d test: Move doxygen comment to header
Also, unrelated formatting fixups.

Can be reviewed with --word-diff-regex=.
2020-08-21 18:44:27 +02:00
John Newbery
5e8df3312e test: resort imports 2020-08-21 15:53:59 +01:00
John Newbery
85165d4332 scripted-diff: Rename mininode to p2p
-BEGIN VERIFY SCRIPT-
sed -i 's/\.mininode/\.p2p/g' $(git grep -l "mininode")
git mv test/functional/test_framework/mininode.py test/functional/test_framework/p2p.py
-END VERIFY SCRIPT-
2020-08-21 15:52:20 +01:00
John Newbery
9e2897d020 scripted-diff: Rename mininode_lock to p2p_lock
-BEGIN VERIFY SCRIPT-
sed -i 's/mininode_lock/p2p_lock/g' $(git grep -l "mininode_lock")
-END VERIFY SCRIPT-
2020-08-21 15:52:13 +01:00
Nadav Ivgi
4148f55dd0 docs: Correct description for getblockstats's txs field
It does count the coinbase transaction.

Refs #19766
2020-08-21 17:41:02 +03:00
MarcoFalke
d254e6e795 Merge #19722: test: Add test for getblockheader verboseness
5067c5acc3 [test] Add test for getblockheader verboseness (Torhte Butler)

Pull request description:

  Improve test coverage by adding a test for getblockheader with verbose argument set to false.

ACKs for top commit:
  theStack:
    ACK 5067c5acc3

Tree-SHA512: e55593f1026a89dc7b796fa985b4cbcdb596e91d80d42dfb0660bda1692aaa35749ec29f9cd7032803f6225afb323f085df1ef6a9982de87be8e098f7253cdd5
2020-08-21 14:34:02 +02:00
John Newbery
daed542a12 [net_processing] Move ProcessMessage to PeerLogicValidation 2020-08-21 13:10:41 +01:00
fanquake
0d9e14a646 Merge #19733: Move comment about BaseIndex::DB from TxIndex::DB
8ed2f1ed78 Remove unused includes (Marcin Jachymiak)
cf095a53fc Move comment about BaseIndex::DB from TxIndex::DB (Marcin Jachymiak)

Pull request description:

  Moves a comment about the `BaseIndex::DB` from the `TxIndex::DB` into the correct place. Originally part of https://github.com/bitcoin/bitcoin/pull/14053.

ACKs for top commit:
  fanquake:
    ACK 8ed2f1ed78

Tree-SHA512: cb4e2b916c7ab996961cc2e1d910bc4b8a1700eb32b70fc1657ca720117a7a84f7337fe5e4fb30e047aa92c31eaa976eaaa5cb8f861877f2ff6f4a59bb94f4e9
2020-08-21 12:48:46 +08:00
João Barbosa
b8405b833a wallet: IsChange requires cs_wallet lock 2020-08-21 00:28:10 +01:00
Luke Dashjr
916d3596c4 help: Generate checkpoint height from chainparams 2020-08-20 18:20:27 +00:00
Luke Dashjr
1ccb9f30c0 Move Win32 defines to configure.ac to ensure they are globally defined
common.vcxproj used for MSVC builds
2020-08-20 17:55:06 +00:00
Wladimir J. van der Laan
27eeb0337b Merge #19550: rpc: Add getindexinfo RPC
124e1ee134 doc: Add release notes for getindexinfo RPC (Fabian Jahr)
c447b09458 test: Add tests for getindexinfo RPC (Fabian Jahr)
667bc7a7f7 rpc: Add getindexinfo RPC (Fabian Jahr)

Pull request description:

  As I was playing with indices a I was missing an RPC that gives information about the active indices in the node. I think this can be helpful for many users, especially since there are some new index candidates coming up (#14053, #18000) that can give a quick overview without the user having to parse the logs.

  Feature summary:
  - Adds new RPC `listindices` (placed in Util section)
  - That RPC only lists the actively running indices
  - For each index it gives the name, whether it is synced and up to which block height it is synced

ACKs for top commit:
  laanwj:
    Re-ACK 124e1ee134
  jonatack:
    Code review re-ACK 124e1ee per `git range-diff a57af89 47a5372 124e1ee` no change since my last re-ACK, rebase only

Tree-SHA512: 3b7174c87951e6457fef099f530337803906baf32fb64261410b8def2c0917853d6a1bf3059cd590b1cc1523608f8916dafb327a431d27ecbf8d7454406b5b35
2020-08-20 16:00:22 +02:00
fanquake
70d7ddbcbb Merge #19727: test: Remove unused classes from p2p_leak.py
ed5cd12869 test: Distinguish between nodes(bitcoind) and peers(mininodes) in p2p_leak.py (Dhruv Mehta)
f6f082b934 test: remove `CNodeNoVersionIdle` from p2p_leak.py (Dhruv Mehta)
45cf55ccac test: remove `CNodeNoVersionMisbehavior` from p2p_leak.py (Dhruv Mehta)

Pull request description:

  - Removes `CNodeNoVersionMisbehavior` per recommendation at https://github.com/bitcoin/bitcoin/pull/19657#issuecomment-669926458
  - Removes `CNodeNoVersionIdle` because it is similarly unnecessary
  - As someone new to the codebase, I found it easier to understand it if `no_version_disconnect_node` tries to overwhelm the peer with any message that is not version/verack.
  - Per recommendation at https://github.com/bitcoin/bitcoin/pull/19727#pullrequestreview-468093555, made a clear distinction between nodes(bitcoind) and peers(mininode interface implementations)

ACKs for top commit:
  jnewbery:
    tested ACK ed5cd12869
  amitiuttarwar:
    utACK ed5cd12869

Tree-SHA512: 310a24c91fd837e7f65177edb55fe6142fb3559fae7867c5cdd9c9a23b1a02202b935ca9a82633fa7649f3de2fa221f6da906a7b5e499fc20f7254085033757d
2020-08-20 10:17:14 +08:00
fanquake
44f66d2f10 Merge #19765: doc: Fix getmempoolancestors RPC result doc
333329dbda doc: Fix getmempoolancestor RPC result doc (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK 333329dbda

Tree-SHA512: 30a7568ec15d1af0c484b4d479e14ec3609a01b76f17f8285688b0c5e5b0480926bbf6f651da91193b66fd752e83e9707e4b08c52b69f8c670c430da84713359
2020-08-20 08:56:37 +08:00
Wladimir J. van der Laan
e9b3012654 Merge #19750: refactor: remove unused c-string variant of atoi64()
71e0f07e9c util: remove unused c-string variant of atoi64() (Sebastian Falbesoner)

Pull request description:

  This is another micro-PR "removing old cruft with potentially sharp edges" (quote by practicalswift, see #19739). Gets rid of the c-string variant of the function `atoi64()`, which is only used in fuzzers and on one place with `wallet/wallet.h` (where it is originally a `std::string` anyways and uses `.c_str()` -- this method call can simply be removed.)

ACKs for top commit:
  practicalswift:
    ACK 71e0f07e9c -- diff looks correct
  laanwj:
    ACK 71e0f07e9c

Tree-SHA512: 4d1d28e2f5274fdbe0652e7a0f83dd416f4d19c1e1a49979927960a3ad40b0990eeaa4374656bf2c6998a965a14d62c1bc78303b7d583d3307c17828030a8e3b
2020-08-19 15:04:34 +02:00
Wladimir J. van der Laan
44ddcd887d Merge #19706: refactor: make EncodeBase58{Check} consume Spans
356988e200 util: make EncodeBase58Check consume Spans (Sebastian Falbesoner)
f0fce0675d util: make EncodeBase58 consume Spans (Sebastian Falbesoner)

Pull request description:

  This PR improves the interfaces for the functions `EncodeBase58{Check}` by using Spans, in a similar fashion to e.g. PRs #19660, #19687. Note that on the master branch there are currently two versions of `EncodeBase58`: one that takes two pointers (marking begin and end) and another one that takes a `std::vector<unsigned char>` const-ref. The PR branch only leaves one generic Span-interface, both simplifying the interface and allowing more generic containers to be passed. The same is done for `EncodeBase58Check`, where only one interface existed but it's more generic now (e.g. a std::array can be directly passed, as done in the benchmarks).

ACKs for top commit:
  laanwj:
    Code review ACK 356988e200

Tree-SHA512: 47cfccdd7f3a2d4694bb8785e6e5fd756daee04ce1652ee59a7822e7e833b4a441ae9362b9bd67ea020d2b5b7d927629c9addb6abaa9881d8564fd3b1257f512
2020-08-19 14:20:15 +02:00
MarcoFalke
333329dbda doc: Fix getmempoolancestor RPC result doc 2020-08-19 10:41:27 +02:00
MarcoFalke
c6532fa6c1 Merge #19746: ci: Move valgrind fuzzer to cirrus
fa0538e94d ci: Set cirrus RAM to 8GB (MarcoFalke)
fa41810d0e ci: Run valgrind fuzzer on cirrus (MarcoFalke)

Pull request description:

  The first commit should fix the 50min timeout in forked repos. Similar to #19424. E.g. https://travis-ci.org/github/bitcoin-core/gui/builds/718322267

  The second commit should fix #19744

Top commit has no ACKs.

Tree-SHA512: c765098dfa913ca49b1d1eee99aaa83e4b9eb191b7ad5e652e3f04744fe8670dd3ef4215832b8e2b5bac0273d24f607fc275e72f566326108ba42ab57228ffd4
2020-08-19 10:12:49 +02:00
Marcin Jachymiak
8ed2f1ed78 Remove unused includes 2020-08-18 21:47:59 -04:00
Marcin Jachymiak
cf095a53fc Move comment about BaseIndex::DB from TxIndex::DB 2020-08-18 21:47:59 -04:00
Dhruv Mehta
ed5cd12869 test: Distinguish between nodes(bitcoind) and peers(mininodes) in p2p_leak.py
Also, remove "C" prefix from class names to match new style
2020-08-18 13:13:19 -07:00
Dhruv Mehta
f6f082b934 test: remove CNodeNoVersionIdle from p2p_leak.py 2020-08-18 13:06:15 -07:00
Dhruv Mehta
45cf55ccac test: remove CNodeNoVersionMisbehavior from p2p_leak.py
It's also clearer to have `no_version_disconnect_node` send a message
other than version or verack in order to reach the peer discouragement
threshold.
2020-08-18 13:06:09 -07:00
Carl Dong
b893688357 depends: Specify LDFLAGS to cmake as well 2020-08-18 15:29:19 -04:00
Carl Dong
b3f541f618 depends: Prepend CPPFLAGS to C{,XX}FLAGS for CMake
This is similar to how we do it for qt.mk.
2020-08-18 15:23:31 -04:00
Carl Dong
8e121e5509 depends: Cleanup CMake invocation 2020-08-18 15:21:12 -04:00
practicalswift
46fcac1e4b tests: Add fuzzing harness for ec_seckey_import_der(...) and ec_seckey_export_der(...) 2020-08-18 18:03:57 +00:00
practicalswift
b667a90389 tests: Add fuzzing harness for SigHasLowR(...) and ecdsa_signature_parse_der_lax(...) 2020-08-18 18:03:56 +00:00
Samuel Dobson
a2a250c7d0 Merge #19743: -maxapsfee follow-up
7e31ea9fa0 -maxapsfee: follow-up fixes (Karl-Johan Alm)
9f77b82176 doc: release notes for -maxapsfee (Karl-Johan Alm)

Pull request description:

  Addresses feedback from jonatack and meshcollider in #14582.

ACKs for top commit:
  jonatack:
    ACK 7e31ea9fa0
  meshcollider:
    re-ACK 7e31ea9fa0

Tree-SHA512: 5d159d78e917e41140e1394bef05e821430dbeac585e9bd708f897041dd7104c2a6b563bfab8b2c85e6d923441160a3880d7864d768aa8e0e66860e0a2c4f56b
2020-08-19 00:03:23 +12:00
Karl-Johan Alm
7e31ea9fa0 -maxapsfee: follow-up fixes
Co-authored-by: Jon Atack <jon@atack.com>
Co-authored-by: Samuel Dobson <dobsonsa68@gmail.com>
2020-08-18 19:24:39 +09:00
Hennadii Stepanov
9034f6e30e Drop deprecated and unused GUARDED_VAR and PT_GUARDED_VAR annotations 2020-08-18 10:46:53 +03:00
Karl-Johan Alm
9f77b82176 doc: release notes for -maxapsfee 2020-08-18 15:30:36 +09:00
MarcoFalke
e6e277f9ed Merge #19756: tests: add sync_all to fix race condition in wallet groups test
72ae20fc14 tests: add sync_all to fix race condition in wallet groups test (Karl-Johan Alm)

Pull request description:

  This most likely fixes #19749, the intermittent CI issues with wallet_groups.

  This fix is also included in #19743.

Top commit has no ACKs.

Tree-SHA512: dd6ef7f89829483e2278191c21fe0912b51fd2187c10a0fa158339c5ab9f22d93b733ae10f17ef25d8b64f44e596e66dba8d7db5c009343472f422ce4cd67d8f
2020-08-18 08:18:00 +02:00
fanquake
772cb03a28 Merge #19015: build: Enable some commonly enabled compiler diagnostics
2f8a4c9a06 build: Enable some commonly enabled compiler diagnostics (practicalswift)

Pull request description:

  Enable some commonly enabled compiler diagnostics as discussed in #17344.

  | Compiler diagnostic | no# of emitted unique GCC warnings in `master` | no# of emitted unique Clang warnings in `master` |
  | ------------- | ------------- | ------------- |
  | `-Wduplicated-branches`: Warn if `if`/`else` branches have duplicated code  | 0 | Not supported |
  | `-Wduplicated-cond`: Warn if `if`/`else` chain has duplicated conditions  | 0 | Not supported |
  | `-Wlogical-op`: Warn about logical operations being used where bitwise were probably wanted  | 0 | Not supported |
  | `-Woverloaded-virtual`: Warn if you overload (not `override`) a virtual function  | 0 | 0 |
  | ~~`-Wunused-member-function`: Warn on unused member function~~  | Not supported | 2 |
  | ~~`-Wunused-template`: Warn on unused template~~ | Not supported | 1 |

  There is a large overlap between this list and [Jason Turner's list of recommended compiler diagnostics in the Collaborative Collection of C++ Best Practices (`cppbestpractices`) project](https://github.com/lefticus/cppbestpractices/blob/master/02-Use_the_Tools_Available.md#gcc--clang). There is also an overlap with the recommendations given in the [C++ Core Guidelines](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines) (with editors Bjarne Stroustrup and Herb Sutter).

  Closes #17344.

ACKs for top commit:
  jonatack:
    ACK 2f8a4c9a06 no warnings for me with these locally on debian 5.7.10-1 (2020-07-26) x86_64 with gcc 10 and clang 12
  fanquake:
    ACK 2f8a4c9a06 - no-longer seeing any obvious issues with doing this.
  hebasto:
    ACK 2f8a4c9a06, no new warnings in Travis jobs.

Tree-SHA512: f669ea22b31263a555f999eff6a9d65750662e95831b188c3192a2cf0127fb7b5136deb762a6b0b7bbdfb0dc6a40caf48251a62b164fffb81dd562bdd15ec3c8
2020-08-18 13:00:13 +08:00
fanquake
f2d1b9881f Merge #19721: p2p: comment out unused MSG_FILTERED_WITNESS_BLOCK
4792cad88c doc: comment out and add annotation to unused MSG_FILTERED_WITNESS_BLOCK (Adam Jonas)

Pull request description:

  Commenting out and adding a note to unused `MSG_FILTERED_WITNESS_BLOCK` [defined in BIP144](https://github.com/bitcoin/bips/blob/master/bip-0144.mediawiki#relay).

  There was an attempt to make use of this in https://github.com/bitcoin/bitcoin/pull/10350, but it was closed due to lack of support. (h/t sdaftuar for pointing to the PR and jnewbery for the idea)

ACKs for top commit:
  jnewbery:
    Obvious ACK 4792cad88c
  theStack:
    ACK 4792cad88c 📜
  MarcoFalke:
    cr ACK 4792cad88c good to keep it around in a comment to avoid accidental future re-assignment
  practicalswift:
    ACK 4792cad88c

Tree-SHA512: 22327ddded643ae50fdb529e4529a9b464f74e90620d0d2079a11070eaa8afe8363f6e14cca52f3bec2c9f87ee13e318edc6c5193761c94b8ae77be353a8da1f
2020-08-18 12:15:48 +08:00
fanquake
53dac67a97 Merge #19719: build: Add Werror=range-loop-analysis
fa55c1d5fd build: Add Werror=range-loop-analysis (MarcoFalke)

Pull request description:

  The warning is implicitly enabled for Bitcoin Core. Also explicitly since commit d92204c900.

  To avoid "fix range loop" follow-up refactors, we have two options:

  * Disable the warning, so that issues never appear
  * Enable it as an error, so that issues are either caught locally or by ci

ACKs for top commit:
  fanquake:
    ACK fa55c1d5fd
  practicalswift:
    ACK fa55c1d5fd -- pre-review fix-up is better than post-review fix-up
  hebasto:
    re-ACK fa55c1d5fd

Tree-SHA512: 019aa133f254af8882c1d5d10c420d9882305db0fc2aa9dad7d285168e2556306c3eedcc03bd30e63f11eae4cc82b648d83fb6e9179d6a6364651fb602d70134
2020-08-18 11:33:34 +08:00
Karl-Johan Alm
72ae20fc14 tests: add sync_all to fix race condition in wallet groups test 2020-08-18 10:08:49 +09:00
Sebastian Falbesoner
71e0f07e9c util: remove unused c-string variant of atoi64() 2020-08-17 17:56:59 +02:00
Sebastian Falbesoner
d3e8adfada util: remove c-string interfaces for DecodeBase58{Check} 2020-08-17 13:16:09 +02:00
MarcoFalke
1bc8e8eae2 Merge #19734: Move CDiskTxPos to its own file
7668db3b08 Move only: Move CDiskTxPos to its own file (Marcin Jachymiak)

Pull request description:

  Moves `CDiskTxPos` it its own file so it can be used without the `txindex.h` include elsewhere. Originally part of #14053.

ACKs for top commit:
  jnewbery:
    utACK 7668db3b08
  promag:
    ACK 7668db3b08.

Tree-SHA512: b108e980ad04e43d1323410c3683a82bed70aee7795f5d8a2afbaf32a07ba598571f00b047bdde15048124b17178bcbd10654c48461beac988e9643cb2df664c
2020-08-17 12:47:11 +02:00
MarcoFalke
fa0538e94d ci: Set cirrus RAM to 8GB 2020-08-17 11:53:31 +02:00
MarcoFalke
fa41810d0e ci: Run valgrind fuzzer on cirrus 2020-08-17 11:52:02 +02:00
Torhte Butler
5067c5acc3 [test] Add test for getblockheader verboseness
Add test for getblockheader with verbose argument set to false.
2020-08-17 08:09:48 +00:00
Samuel Dobson
c831e105c5 Merge #14582: wallet: always do avoid partial spends if fees are within a specified range
7f13dfb587 test: test the implicit avoid partial spends functionality (Karl-Johan Alm)
b82067bf69 wallet: try -avoidpartialspends mode and use its result if fees are below threshold (Karl-Johan Alm)

Pull request description:

  The `-avoidpartialspends` feature is normally disabled, as it may affect the optimal fee for payments. This PR introduces a new parameter `-maxapsfee` (max avoid partial spends fee) which acts on the following values:
  * -1: disable partial spend avoidance completely (do not even try it)
  * 0: only do partial spend avoidance if fees are the same or better as the regular coin selection
  * 1..∞: use APS variant if the absolute fee difference is less than or equal to the max APS fee

  For values other than -1, the code will now try partial spend avoidance once, and if that gives a value within the accepted range, it will use that.

  Example: -maxapsfee=0.00001000 means the wallet will do regular coin select, APS coin select, and then pick AKS iff the absolute fee difference is <= 1000 satoshi.

  Edit: updated this to reflect the fact we are now using a max fee.

ACKs for top commit:
  fjahr:
    tested ACK 7f13dfb587
  achow101:
    ACK 7f13dfb587
  jonatack:
    ACK 7f13dfb58, code review, debug build, verified the test fails with `AssertionError: not(2 == 1)` for the number of vouts when `-maxapsfee=0.0001` is changed to 0, and verified the new logging with an added assertion.
  meshcollider:
    Code review ACK 7f13dfb587

Tree-SHA512: 475929df57f6191bb4e36bfbcad5a280a64bb0ecd8767b76cb2e44e2301235d0eb294a3f2fac5bbf15d35d7ecfba47acb2285feadb883c9ce31c08377e3afb3c
2020-08-17 16:18:28 +12:00
João Barbosa
d8441f30ff wallet: IsMine overloads require cs_wallet lock 2020-08-17 00:06:03 +01:00
João Barbosa
a13cafc6c6 wallet: GetWalletTx requires cs_wallet lock 2020-08-17 00:06:02 +01:00
Wladimir J. van der Laan
ffad348167 Merge #19705: Shrink CAddress from 48 to 40 bytes on x64
767073fb96 Shrink CAddress from 48 to 40 bytes on x64 (Vasil Dimov)

Pull request description:

  `CAddress` inherits `CService` which is 28 bytes (on 64 bit machines).
  `CAddress` then adds two member variables - one that requires 4 byte
  alignment (`nTime`) and one that requires 8 byte alignment
  (`nServices`).

  Declare the smaller one first so that it fits in bytes 29..32.

  On 32 bit machines this change has no effect and `CAddress` remains 40
  bytes.

ACKs for top commit:
  laanwj:
    ACK 767073fb96
  theStack:
    ACK 767073fb96

Tree-SHA512: 73d6a4fcfa2687b4076950801871252e369510ecf09f820576dbeca9ee3ee94d14672e7d5596cb45fedd9e4b973dd0716a2ea3f13fc3058b4b697d036a7c9db0
2020-08-16 18:56:25 +02:00
Igor Cota
366913e307 build: AX_BOOST_THREAD serial 33 2020-08-16 18:47:48 +02:00
Igor Cota
cf0681133a build: disable D-Bus on Android by default
Android uses a different notification system and doesn't support D-Bus
2020-08-16 15:16:36 +02:00
Fabian Jahr
124e1ee134 doc: Add release notes for getindexinfo RPC 2020-08-16 11:15:52 +02:00
Fabian Jahr
c447b09458 test: Add tests for getindexinfo RPC 2020-08-16 11:15:52 +02:00
Fabian Jahr
667bc7a7f7 rpc: Add getindexinfo RPC 2020-08-16 11:15:49 +02:00
MarcoFalke
a57af897ec Merge #19564: test: p2p_feefilter improvements (logging, refactoring, speedup)
9e7894357e test: speedup p2p_feefilter.py by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
fe3f0cc44e test: use wait_until for invs matching in p2p_feefilter.py (Sebastian Falbesoner)
6d941923c3 test: add logging for p2p_feefilter.py (Sebastian Falbesoner)

Pull request description:

  This PR gives some love to the functional test `p2p_feefilter.py` by introducing the following changes:
  * add missing log messages for the `test_feefilter` subtest (the main one)
  * deduplicate code by using the utility function `wait_until` (already using the [recently introduced version](12410b1feb)) instead of a manual condition checking loop with `time.sleep`
  * improve naming of the function `matchAllInvs` (more expressive name, snake_case) and moving it from global namespace to the connection class `FeefilterConn`
  * speeding up the test significantly by the good ol' method of activating immediate tx relay (as seen on e.g. https://github.com/bitcoin/bitcoin/pull/17121, https://github.com/bitcoin/bitcoin/pull/17124, https://github.com/bitcoin/bitcoin/pull/17340, https://github.com/bitcoin/bitcoin/pull/17362, ...):
  ```
  master branch:
  $ time ./p2p_feefilter.py
  ...
  real    0m39.367s
  user    0m1.227s
  sys 0m0.571s

  PR branch:
  $ time ./p2p_feefilter.py
  ...
  real    0m9.386s
  user    0m1.120s
  sys 0m0.577s
  ```

ACKs for top commit:
  instagibbs:
    code review ACK 9e7894357e
  jonatack:
    re-ACK 9e78943 per `git range-diff 3ab2582 ea74a3c 9e78943`

Tree-SHA512: fe21c1c5413df9165fea916b5d5f609d3ba33e7b5c3364b38eb824fcc55d9e6abddf27116cbc0b325913d451a73c44542040fb916aec9c46f805c6e12f6f10cf
2020-08-16 11:04:28 +02:00
Marcin Jachymiak
7668db3b08 Move only: Move CDiskTxPos to its own file 2020-08-15 18:23:55 -04:00
Jake Leventhal
c91b241b48 Updated outdated help command for getblocktemplate (fixes #19625)
* Removed coinbasetxn from the getblocktemplate help outputs
* Added the missing name for transactions in the help outputs
* Added getblocktemplate help outputs for longpollid and default_witness_commitment
* Added more clarity to capabilities, rules, and coinbaseaux for getblocktemplate help (credit to luke-jr)

Co-authored-by: Luke Dashjr <luke+github_public@dashjr.org>
2020-08-15 13:06:03 -07:00
MarcoFalke
3ab2582c7f Merge #19730: ci: Set increased --timeout-factor by default
fa330ec2fe test: Remove confusing and broken use of wait_until global (MarcoFalke)
fa6583c30b ci: Set increased --timeout-factor by default (MarcoFalke)

Pull request description:

  Assuming that tests don't have a logic error or race, setting a high timeout should not cause any issues. The tests will still pass just as fast in the fastest case, but it allows for some buffer in case of slow disks or otherwise starved ci machines.

  Fixes #19729

ACKs for top commit:
  hebasto:
    ACK fa330ec2fe, I have reviewed the code, and it looks OK, I agree it can be merged.

Tree-SHA512: 3da80ee008c7b08bab5fdaf7804d57c79d6fed49a7d37b9c54fc89756659fcb9981fd10afc4d07bd90d93c1699fd410a0110a3cd34d016873759d114ce3cd82d
2020-08-15 15:39:02 +02:00
Jon Atack
5da96210fc doc: release note for getpeerinfo last_block/last_transaction 2020-08-15 15:26:33 +02:00
Jon Atack
cfef5a2c98 test: rpc_net.py logging and test naming improvements 2020-08-15 15:13:40 +02:00
Jon Atack
21c57bacda test: getpeerinfo last_block and last_transaction tests 2020-08-15 15:13:13 +02:00
Jon Atack
8a560a7d57 rpc: expose nLastBlockTime/TXTime as getpeerinfo last_block/transaction 2020-08-15 14:34:37 +02:00
MarcoFalke
fa330ec2fe test: Remove confusing and broken use of wait_until global 2020-08-15 14:01:54 +02:00
Jon Atack
02fbe3ae0b net: add nLastBlockTime/TXTime to CNodeStats, CNode::copyStats 2020-08-15 12:03:16 +02:00
MarcoFalke
fa6583c30b ci: Set increased --timeout-factor by default 2020-08-15 09:24:46 +02:00
Pieter Wuille
86d4cf42d9 Increase the ip address relay branching factor for unreachable networks
Onion addresses propagate very badly among the IPv4/IPv6 network, resulting
in difficulty for those to find each other.

The branching factor 1 is probably so low that propagations die out before
they reach another onion peer. Increase it to 1.5 on average.
2020-08-14 23:02:42 -07:00
fanquake
4cde97374d Merge #19718: build: Add missed gcov files to 'make clean'
6cb8771173 build: Add missed gcov files to 'make clean' (Hennadii Stepanov)

Pull request description:

  On master (b4d0366b47):
  ```
  $ ./autogen.sh
  $ ./configure --enable-lcov
  $ make && make cov
  $ make clean
  $ find . -name '*.gcno'
  ./src/rpc/libbitcoin_server_a-blockchain.gcno
  ./src/rpc/libbitcoin_common_a-util.gcno
  ./src/rpc/libbitcoin_common_a-rawtransaction_util.gcno
  ./src/rpc/libbitcoin_server_a-rawtransaction.gcno
  ./src/rpc/libbitcoin_util_a-request.gcno
  ./src/rpc/libbitcoin_server_a-net.gcno
  ./src/rpc/libbitcoin_server_a-server.gcno
  ./src/rpc/libbitcoin_server_a-mining.gcno
  ./src/rpc/libbitcoin_server_a-misc.gcno
  ./src/rpc/libbitcoin_cli_a-client.gcno
  ./src/node/libbitcoin_server_a-coinstats.gcno
  ./src/node/libbitcoin_server_a-transaction.gcno
  ./src/node/libbitcoin_server_a-context.gcno
  ./src/node/libbitcoin_server_a-psbt.gcno
  ./src/node/libbitcoin_server_a-ui_interface.gcno
  ./src/node/libbitcoin_server_a-coin.gcno
  ./src/test/util/libtest_util_a-setup_common.gcno
  ./src/test/util/libtest_util_a-net.gcno
  ./src/test/util/libtest_util_a-blockfilter.gcno
  ./src/test/util/libtest_util_a-mining.gcno
  ./src/test/util/libtest_util_a-transaction_utils.gcno
  ./src/test/util/libtest_util_a-wallet.gcno
  ./src/test/util/libtest_util_a-str.gcno
  ./src/test/util/libtest_util_a-logging.gcno
  ./src/index/libbitcoin_server_a-txindex.gcno
  ./src/index/libbitcoin_server_a-base.gcno
  ./src/index/libbitcoin_server_a-blockfilterindex.gcno
  ./src/util/libbitcoin_util_a-error.gcno
  ./src/util/libbitcoin_util_a-rbf.gcno
  ./src/util/libbitcoin_util_a-message.gcno
  ./src/util/libbitcoin_util_a-time.gcno
  ./src/util/libbitcoin_util_a-moneystr.gcno
  ./src/util/libbitcoin_util_a-url.gcno
  ./src/util/libbitcoin_consensus_a-strencodings.gcno
  ./src/util/libbitcoin_util_a-settings.gcno
  ./src/util/libbitcoin_util_a-system.gcno
  ./src/util/libbitcoin_util_a-threadnames.gcno
  ./src/util/libbitcoin_util_a-fees.gcno
  ./src/util/libbitcoin_util_a-asmap.gcno
  ./src/util/libbitcoin_util_a-strencodings.gcno
  ./src/util/libbitcoin_util_a-string.gcno
  ./src/util/libbitcoin_util_a-bytevectorhash.gcno
  ./src/util/libbitcoin_util_a-bip32.gcno
  ./src/util/libbitcoin_util_a-spanparsing.gcno
  ./src/util/libbitcoinconsensus_la-strencodings.gcno
  ./src/interfaces/libbitcoin_wallet_a-wallet.gcno
  ./src/interfaces/libbitcoin_util_a-handler.gcno
  ./src/interfaces/libbitcoin_server_a-chain.gcno
  ./src/interfaces/libbitcoin_server_a-node.gcno
  ./src/crc32c/src/libcrc32c_a-crc32c_portable.gcno
  ./src/crc32c/src/libcrc32c_a-crc32c.gcno
  ./src/crc32c/src/libcrc32c_sse42_a-crc32c_sse42.gcno
  ```

  This PR fixes this issue.

ACKs for top commit:
  practicalswift:
    ACK 6cb8771173 -- patch looks correct

Tree-SHA512: d331b8fa18f2e0cb2c107de747a39a018f73bcc20b05ed403aa38cf316b5be30b3229e92fb6c85469747571d0048a34b2846432994d0911c8234d207d4e32f1a
2020-08-15 10:15:42 +08:00
fanquake
d052f5e6b7 Merge #16841: Replace GetScriptForWitness with GetScriptForDestination
7966aa424a Add variables for repeated scripts (MeshCollider)
fec8336ad9 Remove GetScriptForWitness function (MeshCollider)
b887060d06 Replace usage of GetScriptForWitness with GetScriptForDestination (MeshCollider)

Pull request description:

  As per this TODO in the code:

  > TODO: replace calls to GetScriptForWitness with GetScriptForDestination using the various witness-specific CTxDestination subtypes.

  The commit "Add additional check for P2SH before adding extra wrapper" also adds an additional check that the scriptPubKey is a P2SH before auto-wrapping the witness script. We shouldn't wrap the witness script if not. Note: #16251 is even better than this check, please review that.

ACKs for top commit:
  instagibbs:
    ACK 7966aa424a
  jonatack:
    Code review re-ACK 7966aa4 per `git range-diff b4d0366 ed266f7 7966aa4`
  achow101:
    re-ACK 7966aa424a only changes since last is rebase.

Tree-SHA512: 3449e0e83bd842acc7c94544a85367da97ac20d859eefc1a618caef0c98204398c266fe8fb9600b78326df5175402e1ae4a132eb766e2c4485e7cda6a2a95c43
2020-08-15 08:54:45 +08:00
Samuel Dobson
a0e75bd31d Merge #15937: Add loadwallet and createwallet load_on_startup options
642ad31b41 Add loadwallet and createwallet RPC load_on_startup options (Russell Yanofsky)

Pull request description:

  This maintains a persistent list of wallets stored in settings that will automatically be loaded on startup. Being able to load a wallet automatically on startup will be more useful in the GUI, but it's reasonable to expose this feature by RPC as well.

ACKs for top commit:
  achow101:
    re-ACK 642ad31b41 Only change is the test
  meshcollider:
    re-utACK 642ad31b41

Tree-SHA512: cca0b71bf1a83ad071830e6c459f1cd979b4add7144e899ec560da72b5910dd9bf9426e5c7d125ae96fad8990fbf81a76bc83c0459486c16086ada6cbde5eaa3
2020-08-15 12:19:48 +12:00
Samuel Dobson
f269165edc Merge #17458: Refactor OutputGroup effective value calculations and filtering to occur within the struct
9adc2f80fc Refactor OutputGroups to handle effective values, fees, and filtering (Andrew Chow)
7d07e864b8 Use real value when calculating OutputGroup value (Andrew Chow)

Pull request description:

  Currently, the effective values and filtering for positive effective values is done outside of the OutputGroup. We should instead have functions in Outputgroup to do this and call those for each OutputGroup. So this PR does that.

  This makes future changes for effective values in coin selection much easier.

ACKs for top commit:
  instagibbs:
    reACK 9adc2f80fc
  fjahr:
    re-ACK 9adc2f80fc
  meshcollider:
    Light code review ACK 9adc2f80fc

Tree-SHA512: 7445c94b7295b45bcd83a6f8a5c8f6961a89453fcc856335192d4b4a66aec7724513616b04e5111588ab208c89b311055399d6279cd9c4ce452aefb85f04b64a
2020-08-15 11:44:30 +12:00
Adam Jonas
4792cad88c doc: comment out and add annotation to unused MSG_FILTERED_WITNESS_BLOCK 2020-08-14 09:58:34 -04:00
MarcoFalke
fa55c1d5fd build: Add Werror=range-loop-analysis 2020-08-14 15:27:38 +02:00
MarcoFalke
30dd562fd2 Merge #19457: wallet: Cleanup wallettool salvage and walletdb extraneous declarations
0e279fe489 walletdb: Remove unused static functions from walletdb.h (Andrew Chow)
9f536d4fe9 wallettool: Have RecoverDatabaseFile return errors and warnings (Andrew Chow)
06e263a4e3 Call RecoverDatabaseFile directly from wallettool (Andrew Chow)

Pull request description:

  Followup to #19324 addressing some comments.

  Removes the `SalvageWallet` function in wallettool and instead directly calls `RecoverDatabaseFile` as suggested in https://github.com/bitcoin/bitcoin/pull/19324#discussion_r450379596

  Removes the `LogPrintf`s and `tfm::format`s in `RecoverDatabaseFile` as noted in https://github.com/bitcoin/bitcoin/pull/19324#discussion_r448027237

  Removes the declarations of `VerifyEnvironment` and `VerifyDatabaseFile` that were forgotten in `walletdb.h` as noted in https://github.com/bitcoin/bitcoin/pull/19324#issuecomment-654389079

ACKs for top commit:
  meshcollider:
    Code review ACK 0e279fe489
  ryanofsky:
    Code review ACK 0e279fe489, just dropped last commit

Tree-SHA512: ffd01f30536c2eab4bf40ba363c3ea916ecef3c8f0c5262040b40498776ffb00f95240204a40e38415d6931800851d0a3fa63ee91efc1d329b60ac317da0363d
2020-08-14 15:12:44 +02:00
MarcoFalke
1a43cd3f74 Merge #19709: test: Fix 'make cov' with clang
35cd2da623 test: Fix 'make cov' with clang (Hennadii Stepanov)

Pull request description:

  This is a follow up of #19688.

  With this PR it is possible to do the following:
  ```
  $ ./autogen.sh
  $ ./configure --enable-lcov CC=clang CXX=clang++
  $ make
  $ make cov
  ```

  Currently, on master (8a85377cd0), `make cov` fails to `Processing src/test/test_bitcoin-util_tests.gcda`.

ACKs for top commit:
  vasild:
    ACK 35cd2da
  Crypt-iQ:
    ACK 35cd2da

Tree-SHA512: aaf56118e2644064e9738a8279889c617db5805c5c804c904469b24c496bd609f9c5fc2aebcf1a422f8a5ed2eb38bd6e76b484680310b55c36d922b73a4c33cf
2020-08-14 14:57:49 +02:00
Hennadii Stepanov
ce3e6a7cb2 bench: Allow skip benchmark
Co-authored-by: Martin Ankerl <Martin.Ankerl@gmail.com>
2020-08-14 14:56:18 +03:00
Hennadii Stepanov
6cb8771173 build: Add missed gcov files to 'make clean' 2020-08-14 14:38:28 +03:00
MarcoFalke
fa3d9ce325 rpc: Assert that RPCArg names are equal to CRPCCommand ones (rpcdump) 2020-08-14 12:38:03 +02:00
MarcoFalke
fa32c1d5ec rpc: Assert that RPCArg names are equal to CRPCCommand ones (zmq) 2020-08-14 12:37:51 +02:00
MarcoFalke
faaa46dc20 rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining) 2020-08-14 12:37:19 +02:00
MarcoFalke
fa93bc14c7 rpc: Remove unused return type from appendCommand 2020-08-14 12:37:06 +02:00
Wladimir J. van der Laan
4d4bd5ed74 Merge #17204: wallet: Do not turn OP_1NEGATE in scriptSig into 0x0181 in signing code (sipa)
dca28634d7 test: ensure OP_1NEGATE satisfies BIP62 minimal push rule (Jon Atack)
e629d07199 Do not turn OP_1NEGATE in scriptSig into 0x0181 in signing code (Pieter Wuille)

Pull request description:

  A rebase of #13084 which additionally modifies the test code (unaddressed in the original, assuming sipa is too busy to deal with this at the moment).

  Relatively simple bugfix so it'd be good to have merged soon.

  Turning OP_1NEGATE into 0x0181 results in a larger-than-necessary data push instead of just actually using the OP_1NEGATE opcode (0x4f). This fails the minimal push rule of BIP 62 and makes the result non-standard.

ACKs for top commit:
  fjahr:
    Code review ACK dca28634d7
  luke-jr:
    ACK dca28634d7
  jonatack:
    ACK dca28634d7

Tree-SHA512: 706d9a2ef20c809dea923e477a873e2fd60db8d0ae64289e510b766a38005c1f31ab0b5883f16b9c7863ff0d3f705e8e413f6121320028ac196b79c3184a4113
2020-08-14 11:53:47 +02:00
Wladimir J. van der Laan
dabab06a1a Merge #19455: rpc generate: print useful help and error message
f0aa8aeea5 test: add rpc_generate functional test (Jon Atack)
92d94ffb8d rpc: print useful help and error message for generate (Jon Atack)
8d32d2011d test: consider generate covered in _get_uncovered_rpc_commands() (Jon Atack)

Pull request description:

  This was a requested follow-up to #19133 and #17700 to alleviate confusion and head-scratching by people following tutorials that use `generate`. See https://github.com/bitcoin/bitcoin/pull/19455#issuecomment-668172916 below, https://github.com/bitcoin/bitcoin/pull/19133#issuecomment-636860943 and https://github.com/bitcoin/bitcoin/pull/17700#issuecomment-566159096.

  before
  ```
  $ bitcoin-cli help generate
  help: unknown command: generate

  $ bitcoin-cli generate
  error code: -32601
  error message:
  Method not found
  ```

  after
  ```
  $ bitcoin-cli help generate
  generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.

  $ bitcoin-cli generate
  error code: -32601
  error message:
  generate ( nblocks maxtries ) has been replaced by the -generate cli option. Refer to -help for more information.
  ```

  In the general help it remains hidden, as requested by laanwj.
  ```
  $ bitcoin-cli help

  == Generating ==
  generateblock "output" ["rawtx/txid",...]
  generatetoaddress nblocks "address" ( maxtries )
  generatetodescriptor num_blocks "descriptor" ( maxtries )
  ```

ACKs for top commit:
  adamjonas:
    utACK f0aa8aeea5
  pinheadmz:
    ACK f0aa8aeea5

Tree-SHA512: d083652589ad3e8228c733455245001db22397559c3946e7e573cf9bd01c46e9e88b72d934728ec7f4361436ae4c74adb8f579670b09f479011924357e729af5
2020-08-14 11:29:05 +02:00
Hennadii Stepanov
35cd2da623 test: Fix 'make cov' with clang 2020-08-14 12:18:47 +03:00
fanquake
c0b1706964 Merge #19568: Wallet should not override signing errors
e7448d6680 wallet: Don't override signing errors (Fabian Jahr)

Pull request description:

  While reviewing #17204 I noticed that the errors in `input_errors` from `::SignTransaction` where being overridden by `CWallet::SignTransaction`. For example, a Script related error led to incomplete signature data which led to `CWallet::SignTransaction` reporting that keys were missing, which was a less precise error than the original one.

  Additionally, the error `"Input not found or already spent"` is [duplicated in `sign.cpp`](c7b4968552/src/script/sign.cpp (L481)), so the error here is redundant at the moment. So technically the whole error block could be removed, I think. However, this code is affected by the ongoing work on the wallet so there might be a reason why these errors are here. But even if there is a reason to keep them, I don't think existing, potentially more precise errors should be overridden here unless we want to hide them from the users. I am looking for feedback if this is a work in progress state where these errors could be more useful in the future or if they can be removed.

  On testing: even though [the errors in `CWallet` are covered](https://marcofalke.github.io/btc_cov/total.coverage/src/wallet/wallet.cpp.gcov.html), all tests still pass after removing them. I am not sure if there is a desire to cover these specific error messages, tests in `test/functional/rpc_signrawtransaction.py` seem to aim for a more generic approach.

ACKs for top commit:
  achow101:
    ACK e7448d6680
  meshcollider:
    Code review ACK e7448d6680

Tree-SHA512: 3e2bc11d05379d2aef87b093a383d1b044787efc70e35955b2f8ecd028b6acef02f386180566af6a1a63193635f5d685466e2f6141c96326c49ffc5c81ca3e23
2020-08-14 16:04:31 +08:00
MarcoFalke
31760bb7c9 Merge #19528: rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc)
fa77de2baa rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc) (MarcoFalke)
fa50bdc755 rpc: Limit echo to 10 args (MarcoFalke)
fa89ca9b5b refactor: Use C++11 range based for loops to simplify rpc code (MarcoFalke)
fa459bdc87 rpc: Treat all args after a hidden arg as hidden as well (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  laanwj:
    Code review ACK fa77de2baa
  fjahr:
    tested ACK fa77de2baa
  theStack:
    ACK fa77de2baa
  ryanofsky:
    Code review ACK fa77de2baa. Pretty straightfoward changes

Tree-SHA512: badae1606518c0b55ce2c0bb9025d14f05556532375eb20fd6f3bfadae1e5e6568860bff8599d037e655bf1d23f1f464ca17f4db10a6ab3d502b6e9e61c7b3d3
2020-08-14 09:26:37 +02:00
Samuel Dobson
609ce2d0da Merge #19644: rpc: document returned error fields as optional if applicable
f110b7c722 rpc: document returned error fields as optional if applicable (Sebastian Falbesoner)

Pull request description:

  The following RPCs return error fields (named `"error"` or `"errors"`) that are optional, but don't show up as optional in the help text yet:
  * `analyzepsbt`
  * `estimatesmartfee`
  * `signrawtransactionwithkey`
  * `signrawtransactionwithwallet`

  The following RPC has the errors field already marked as optional, but doesn't match the usual format in the description (like `"if there are any"` in parantheses):
  * `estimaterawfee`

  This PR adds the missing optional flags and adapts the description strings. Inspired by a recent PR #19634 by justinmoon.

  The instances were found via `git grep "RPCResult.*\"error"`. Note that there is one RPC so far where the return error is not optional (i.e. in case of no error, the field is included in the result, but is just empty), namely `bumpfee`.

ACKs for top commit:
  adaminsky:
    ACK `f110b7c`
  laanwj:
    ACK f110b7c722, new documentation looks consistent with actual behavior
  achow101:
    ACK f110b7c722
  meshcollider:
    utACK f110b7c722

Tree-SHA512: 30c00f78a575b60e32b4536496af986d53a25f33e6ebbf553adcdcf825ad21a44f90267f3d1ea53326dac83bcfa9983fdb3dad6d3126e20f97f3c08ce286e188
2020-08-14 09:57:58 +12:00
MeshCollider
7966aa424a Add variables for repeated scripts 2020-08-14 08:47:19 +12:00
MeshCollider
fec8336ad9 Remove GetScriptForWitness function 2020-08-14 08:47:19 +12:00
MeshCollider
b887060d06 Replace usage of GetScriptForWitness with GetScriptForDestination 2020-08-14 08:44:42 +12:00
Russell Yanofsky
642ad31b41 Add loadwallet and createwallet RPC load_on_startup options
This maintains a persistent list of wallets stored in settings that will
automatically be loaded on startup. Being able to load a wallet automatically
on startup will be more useful in the GUI when the option to create wallets is
added in #15006, but it's reasonable to expose this feature by RPC as well.
2020-08-13 09:44:48 -04:00
Wladimir J. van der Laan
b4d0366b47 Merge #19070: p2p: Signal support for compact block filters with NODE_COMPACT_FILTERS
f5c003d3ea [test] Add test for NODE_COMPACT_FILTER. (Jim Posen)
132b30d9c8 [net] Signal NODE_COMPACT_FILTERS if we're serving compact filters. (Jim Posen)
b3fbc94d4f Apply cfilters review fixups (John Newbery)

Pull request description:

  If -peerblockfilters is configured, signal the `NODE_COMPACT_FILTERS` service bit to indicate that we are able to serve compact block filters, headers and checkpoints.

ACKs for top commit:
  MarcoFalke:
    re-review and Concept ACK f5c003d3ea
  fjahr:
    Code review ACK f5c003d3ea
  clarkmoody:
    Concept ACK f5c003d3ea
  ariard:
    Concept and Code Review ACK f5c003d
  jonatack:
    ACK f5c003d3e

Tree-SHA512: 34d1c153530a0e55d09046fe548c9dc37344b5d6d50e00af1b4e1de1e7b49de770fca8471346a17c151de9fe164776296bb3dd5af331977f0c3ef1e6fc906f85
2020-08-13 15:44:48 +02:00
Hennadii Stepanov
ead771bf6f qt: Rename qt-init thread before logging start 2020-08-13 16:40:03 +03:00
Hennadii Stepanov
ad5f614bf3 qt: Name ClientModel timer QThread 2020-08-13 16:37:05 +03:00
Hennadii Stepanov
2c7f5d8c2e qt: Name WalletController worker QThread 2020-08-13 16:37:05 +03:00
Hennadii Stepanov
27dcc37d42 qt: Name RPCConsole executor QThread 2020-08-13 16:36:53 +03:00
Wladimir J. van der Laan
6757b3ac8f Merge #19655: rpc: Catch listsinceblock target_confirmations exceeding block count
c133cdcdc3 Cap listsinceblock target_confirmations param (Adam Stein)

Pull request description:

  This addresses an issue brought up in #19587.

  Currently, the `target_confirmations` parameter to `listsinceblock` is not checked for being too large. When `target_confirmations` is greater than one more than the current number of blocks, `listsinceblock` fails with error code -1. In comparison, when `target_confirmations` is less than 1,  a -8 "Invalid parameter" error code is thrown.

  This PR fixes the issue by returning a -8 "Invalid parameter" error if the `target_confirmations` value corresponds to a block with more confirmations than the genesis block. This happens if `target_confirmations` exceeds one more than the number of blocks.

ACKs for top commit:
  laanwj:
    Code review ACK c133cdcdc3
  ryanofsky:
    Code review ACK c133cdcdc3. Just suggested changes since last review. Thanks!

Tree-SHA512: 02680f4cb937d2c24d5019abd0ebfa188b8a50679a1e64e9c26bfe5c17eef6aea906832e6e2d492ba8a2ea160041bf185d66795ee691e340f6793db03c21b89a
2020-08-13 12:12:33 +02:00
Jonas Schnelli
1052b09031 Merge #19011: Reduce cs_main lock accumulation during GUI startup
386ec192a5 Reduce cs_main lock accumulation during GUI startup (Jonas Schnelli)
d42cb79068 Optionally populate BlockAndHeaderTipInfo during AppInitMain (Jonas Schnelli)
b354a1480a Add BlockAndHeaderTipInfo to the node interface/appInit (Jonas Schnelli)
25e1d0bf41 RPCConsole, take initial chaintip data as parameter (Jonas Schnelli)

Pull request description:

  During the GUI startup, there is currently an accumulation of cs_main locks due to setting initial chain state values at multiple locations (in the GUI main thread).

  This PR tries to cache the initial chain state (tip height, tip time, best header, etc.) short after loading the blockindex.

  The cached values are then used instead of fetching them again (and thus locking `cs_main`) during setting the client model.

  This should fix the initial GUI blocking often experienced during or short after the splashscreen.
  On mac, best tested together with #19007.

ACKs for top commit:
  promag:
    Code review ACK 386ec192a5.
  ryanofsky:
    Code review ACK 386ec192a5. Just rebased since last review due to conflicts

Tree-SHA512: caccca05360e6dc0c3aade5e7ed24be513607821a8bd6612d0337259304ab772799fb2d707a0d7c7e50fbff4bd394354643fd0aeaa3bb55960ccc28562f4763d
2020-08-13 10:37:46 +02:00
Samuel Dobson
8a85377cd0 Merge #18654: rpc: separate bumpfee's psbt creation function into psbtbumpfee
79d6332e9e moveonly: Fix indentation in bumpfee RPC (Andrew Chow)
431071c28a Hide bumpfee's psbt creation behavior behind -deprecatedrpc (Andrew Chow)
4638224f64 Add psbtbumpfee RPC (Andrew Chow)

Pull request description:

  Adds a new RPC `psbtbumpfee` which always creates a psbt. `bumpfee` will then only be able to create and broadcast fee bumping transactions instead of changing its behavior based on `IsWalletSet(WALLET_FLAG_DISABLE_PRIVATE_KEYS)`.

  Split from #18627

ACKs for top commit:
  Sjors:
    re-utACK 79d6332
  meshcollider:
    utACK 79d6332e9e
  fjahr:
    Code review ACK 79d6332e9e

Tree-SHA512: 1c92c4b4461bb30e78be3ee73165f624398ef33996ce36043b61a8931be667030d0fca12fd0b30097b78c56e4e9092c69582b237cbdac51d56f6be23d8c0f1bb
2020-08-13 12:21:11 +12:00
Adam Stein
c133cdcdc3 Cap listsinceblock target_confirmations param
Previously, listsinceblock would fail with error code -1 when the
target_confirmations exceeded the number of confirmations of the genesis
block. This commit allows target_confirmations to refer to a lastblock
hash with more confirmations than exist in the chain by setting the
lastblock hash to the genesis hash in this case. This allows for
`listsinceblock "" 6` to not fail if the block count is less than 5
which may happen on regtest.

Includes update to the functional test for listsinceblock to test for
this case.
2020-08-12 15:16:22 -07:00
Sebastian Falbesoner
9e7894357e test: speedup p2p_feefilter.py by whitelisting peers (immediate tx relay)
Most of the test time is spent in wait_for_invs() after sending to addresses,
i.e. the bottleneck is in relaying transactions. By whitelisting the peers via
-whitelist, the inventory is transmissioned immediately rather than on average
every 5 seconds, speeding up the test significantly:

before:
$ time ./p2p_feefilter.py
...
real    0m39.367s
user    0m1.227s
sys 0m0.571s

with this commit:
$ time ./p2p_feefilter.py
...
real    0m9.386s
user    0m1.120s
sys 0m0.577s
2020-08-12 22:38:09 +02:00
Sebastian Falbesoner
fe3f0cc44e test: use wait_until for invs matching in p2p_feefilter.py
additionally:
    -> rename function with snake_case (s/allInvsMatch/wait_for_invs_to_match)
    -> move it from global namespace to the class FeefilterConn
2020-08-12 22:30:38 +02:00
Sebastian Falbesoner
6d941923c3 test: add logging for p2p_feefilter.py 2020-08-12 22:21:38 +02:00
MarcoFalke
038a04eb80 Merge #19688: build: Add support for llvm-cov
75f9659d7a build: Add missed fuzz.coverage/ directory to .gitignore (Hennadii Stepanov)
8ebc0505e9 build: Add missed fuzz_filtered.info to COVERAGE_INFO (Hennadii Stepanov)
c71bdf93d7 build, test: Add support for llvm-cov (Hennadii Stepanov)

Pull request description:

  With this PR it is possible to use `lcov` with clang:
  ```
  $ ./autogen.sh
  $ ./configure --enable-lcov --enable-fuzz --with-sanitizers=fuzzer CC=clang CXX=clang++
  $ make
  $ make cov_fuzz
  ```

  ---

  NOTE: Unfortunately, on my system (`clang version 10.0.0-4ubuntu1`) due to unknown for me reasons `make cov` never finishes, trying to `Processing src/test/test_bitcoin-util_tests.gcda` forever (stopped waiting).

  Closes #12602

ACKs for top commit:
  Crypt-iQ:
    Tested ACK 75f9659d7a
  vasild:
    ACK 75f9659d7

Tree-SHA512: 4bc31b38fa62d70c21f890f17f0340e64d0509cea3c29ff6ac101e90ae65d2032640abf100a380c31557bea4c3f54301c2acc2b88a00cbc5261d54c01358ce4e
2020-08-12 22:21:04 +02:00
Wladimir J. van der Laan
13c4635a3e Merge #19696: rpc: Fix addnode remove command error
a51d0ad2de rpc: Improve addnode remove command error message (Fabian Jahr)

Pull request description:

  The `addnode` RPC with the `remove` command parameter is used to remove a node from the "added nodes". It did not have test coverage and in case of failure to remove the node it responded with the confusing message "Error: Node has not been added.".

  This PR adds test coverage and introduces a new error code as well as changes the error message to something that makes sense.

ACKs for top commit:
  laanwj:
    Code review ACK a51d0ad2de
  theStack:
    Tested ACK https://github.com/bitcoin/bitcoin/commit/a51d0ad2de

Tree-SHA512: 033ef5de0d4d49d58ef4df3759b838c9d19ee9dfb0aff9f814a3a63d124ca231a442c930efa7d343fe1f65727c4b59fc23dd5e26fe6ea69f9e84fda48b5c5cc2
2020-08-12 19:29:14 +02:00
Jonas Schnelli
386ec192a5 Reduce cs_main lock accumulation during GUI startup 2020-08-12 16:44:09 +02:00
Jonas Schnelli
d42cb79068 Optionally populate BlockAndHeaderTipInfo during AppInitMain 2020-08-12 16:44:09 +02:00
Jonas Schnelli
b354a1480a Add BlockAndHeaderTipInfo to the node interface/appInit 2020-08-12 16:44:06 +02:00
Jonas Schnelli
25e1d0bf41 RPCConsole, take initial chaintip data as parameter 2020-08-12 16:31:23 +02:00
Sebastian Falbesoner
356988e200 util: make EncodeBase58Check consume Spans 2020-08-12 16:25:50 +02:00
Sebastian Falbesoner
f0fce0675d util: make EncodeBase58 consume Spans 2020-08-12 16:25:50 +02:00
Vasil Dimov
767073fb96 Shrink CAddress from 48 to 40 bytes on x64
`CAddress` inherits `CService` which is 28 bytes (on 64 bit machines).
`CAddress` then adds two member variables - one that requires 4 byte
alignment (`nTime`) and one that requires 8 byte alignment
(`nServices`).

Declare the smaller one first so that it fits in bytes 29..32.

On 32 bit machines this change has no effect and `CAddress` remains 40
bytes.
2020-08-12 16:00:11 +02:00
John Newbery
c556770b5e [net_processing] Change PeerLogicValidation to hold a connman reference
Hold a reference to connman rather than a pointer because:

- PeerLogicValidation can't run without a connman
- The pointer never gets reseated

The alternative is to always assert that the pointer is non-null before
dereferencing.

Change the name from connman to m_connman at the same time to conform
with current style guidelines.
2020-08-12 14:25:28 +01:00
Wladimir J. van der Laan
bd00d3b1f2 Merge #19658: [rpc] Allow RPC to fetch all addrman records and add records to addrman
37a480e0cd [net] Add addpeeraddress RPC method (John Newbery)
ae8051bbd8 [test] Test that getnodeaddresses() can return all known addresses (John Newbery)
f26502e9fc [addrman] Specify max addresses and pct when calling GetAddresses() (John Newbery)

Pull request description:

  Currently addrman only allows a maximum of 1000 records or 23% of all records to be returned in a call to `GetAddr()`. Relax this limit and have the client specify the max records they want. For p2p, behaviour is unchanged (but the rate limiting is set inside net_processing, where it belongs). For RPC, `getnodeaddresses` can now return the complete addrman, which is helpful for testing and monitoring.

  Also add a test-only RPC `addpeeraddress`, which adds an IP address:port to addrman. This is helpful for testing (eg #18991).

ACKs for top commit:
  naumenkogs:
    utACK 37a480e0cd
  laanwj:
    Code review and lightly manually tested ACK 37a480e0cd

Tree-SHA512: f86dcd410aaebaf6e9ca18ce6f23556e5e4649c1325577213d873aa09967298e65ab2dc19a72670641ae92211a923afda1fe124a82e9d2c1cad73d478ef27fdc
2020-08-12 15:23:06 +02:00
John Newbery
8e35bf5906 scripted-diff: rename misbehavior members
-BEGIN VERIFY SCRIPT-
sed -i 's/nMisbehavior/m_misbehavior_score/g' src/net_processing.cpp src/net_processing.h src/rpc/net.cpp src/qt/rpcconsole.cpp
-END VERIFY SCRIPT-
2020-08-12 11:23:22 +01:00
John Newbery
1f96d2e673 [net processing] Move misbehavior tracking state to Peer
Misbehavior tracking state is now contained in Peer instead of
CNode. It is no longer guarded by cs_main, but instead by a
dedicated m_misbehavior_mutex lock.

This allows us to remove 14 cs_main locks from net_processing.
2020-08-12 11:23:21 +01:00
John Newbery
7cd4159ac8 [net processing] Add Peer
Peer is a struct for holding per-peer data. This structure is not
protected by cs_main since it does not contain validation-critical data.
2020-08-12 11:22:44 +01:00
John Newbery
aba03359a6 [net processing] Remove CNodeState.name
This has been unused since logging peer IPs was removed from
Misbehaving() in a8865f8b.
2020-08-12 10:10:22 +01:00
John Newbery
37a480e0cd [net] Add addpeeraddress RPC method
Allows addresses to be added to Address Manager for testing.
2020-08-12 09:22:10 +01:00
John Newbery
ae8051bbd8 [test] Test that getnodeaddresses() can return all known addresses 2020-08-12 09:22:10 +01:00
John Newbery
f26502e9fc [addrman] Specify max addresses and pct when calling GetAddresses()
CAddrMan.GetAddr() would previously limit the number and percentage of
addresses returned (to ADDRMAN_GETADDR_MAX (1000) and
ADDRMAN_GETADDR_MAX_PCT (23) respectively). Instead, make it the callers
responsibility to specify the maximum addresses and percentage they want
returned.

For net_processing, the maximums are MAX_ADDR_TO_SEND (1000) and
MAX_PCT_ADDR_TO_SEND (23). For rpc/net, the maximum is specified by the
client.
2020-08-12 09:22:07 +01:00
fanquake
ce3bdd0ed1 Merge #19316: [net] Cleanup logic around connection types
01e283068b [net] Remove unnecessary default args on CNode constructor (Amiti Uttarwar)
bc5d65b3ca [refactor] Remove IsOutboundDisconnectionCandidate (Amiti Uttarwar)
2f2e13b6c2 [net/refactor] Simplify multiple-connection checks (Amiti Uttarwar)
7f7b83deb2 [net/refactor] Rework ThreadOpenConnections logic (Amiti Uttarwar)
35839e963b [net] Fix bug where AddrFetch connections would be counted as outbound full relay (Amiti Uttarwar)
4972c21b67 [net/refactor] Clarify logic for selecting connections in ThreadOpenConnections (Amiti Uttarwar)
60156f5fc4 [net/refactor] Remove fInbound flag from CNode (Amiti Uttarwar)
7b322df629 [net/refactor] Remove m_addr_fetch member var from CNode (Amiti Uttarwar)
14923422b0 [net/refactor] Remove fFeeler flag from CNode (Amiti Uttarwar)
49efac5cae [net/refactor] Remove m_manual_connection flag from CNode (Amiti Uttarwar)
d3698b5ee3 [net/refactor] Add connection type as a member var to CNode (Amiti Uttarwar)
46578c03e9 [doc] Describe different connection types (Amiti Uttarwar)
442abae2ba [net/refactor] Add AddrFetch connections to ConnectionType enum (Amiti Uttarwar)
af59feb052 [net/refactor] Extract m_addr_known logic from initializer list (Amiti Uttarwar)
e1bc29812d [net/refactor] Add block relay only connections to ConnectionType enum (Amiti Uttarwar)
0e52a659a2 [net/refactor] Add feeler connections to ConnectionType enum (Amiti Uttarwar)
1521c47438 [net/refactor] Add manual connections to ConnectionType enum (Amiti Uttarwar)
26304b4100 [net/refactor] Introduce an enum to distinguish type of connection (Amiti Uttarwar)
3f1b7140e9 scripted-diff: Rename OneShot to AddrFetch (Amiti Uttarwar)

Pull request description:

  **This is part 1 of #19315, which enables the ability to test `outbound` and `block-relay-only` connections from the functional tests.** Please see that PR for more information of overall functionality.

  **This PR simplifies how we manage different connection types.** It introduces an enum with the various types of connections so we can explicitly define the connection type. The existing system relies on a series of independent flags, then has asserts scattered around to ensure that conflicting flags are not enabled at the same time. I find this approach to be both brittle and confusing. While making these changes, I found a small bug due to the silent assumptions.

  This PR also proposes a rename from `OneShot` to `AddrFetch`. I find the name `OneShot` to be very confusing, especially when we also have `onetry` manual connections. Everyone I've talked to offline has agreed that the name is confusing, so I propose a potential alternative. I think this is a good opportunity for a rename since I'm creating an enum to explicitly define the connection types.
  (some context for the unfamiliar: `oneshot` or `addrfetch` connections are short-lived connections created on startup. They connect to the seed peers, send a `getaddr` to solicit addresses, then close the connection.)

  Overview of this PR:
  * rename `oneshot` to `addrfetch`
  * introduce `ConnectionType` enum
  * one by one, add different connection types to the enum
  * expose the `conn_type` on CNode, and use this to reduce reliance on flags (& asserts)
  * fix the bug in counting different type of connections
  * some additional cleanup to simplify logic and make expectations explicit/inclusive rather than implicit/exclusive.

ACKs for top commit:
  jnewbery:
    utACK 01e283068b
  laanwj:
    Code review ACK 01e283068b, the commits are pretty straightforward to follow, and I think this is a move in the right direction overall
  vasild:
    ACK 01e283068
  sdaftuar:
    ACK 01e283068b.
  fanquake:
    ACK 01e283068b - I don't have as much experience with the networking code but these changes look fairly straight forward, the new code seems more robust/understandable and the additional documentation is great. I'm glad that a followup branch is already underway. There might be some more review comments here later today, so keep an eye on the discussion, however I'm going to merge this now.
  jb55:
    wow this code was messy before... ACK 01e283068b

Tree-SHA512: 7bb644a6ed5849913d777ebc2ff89133ca0fbef680355a9a344e07496a979e6f9ff21a958e8eea93dcd7d5c343682b0c7174b1a3de380a4247eaae73da436e15
2020-08-12 10:01:44 +08:00
Andrew Chow
9adc2f80fc Refactor OutputGroups to handle effective values, fees, and filtering
Instead of having callers set the fees, effective values, and filtering
of outputs, do these within OutputGroups themselves as member functions.

m_fee and m_long_term_fee is added to OutputGroup to track the fees of
the OutputGroup.
2020-08-11 14:25:02 -04:00
skmcontrib
47e2a35fac doc: Document ALLOW_HOST_PACKAGES dependency option 2020-08-11 10:26:45 -04:00
Fabian Jahr
a51d0ad2de rpc: Improve addnode remove command error message
This also adds test coverage for the remove command which was uncovered before.
2020-08-11 14:04:02 +02:00
practicalswift
2f8a4c9a06 build: Enable some commonly enabled compiler diagnostics 2020-08-11 12:03:28 +00:00
fanquake
cb1ee1551c Merge #19674: refactor: test: use throwaway _ variable for unused loop counters
dac7a111bd refactor: test: use _ variable for unused loop counters (Sebastian Falbesoner)

Pull request description:

  This tiny PR substitutes Python loops in the form of `for x in range(N): ...` by `for _ in range(N): ...` where applicable. The idea is indicating to the reader that a block (or statement, in list comprehensions) is just repeated N times, and that the loop counter is not used in the body, hence using the throwaway variable. This is already done quite often in the current tests (see e.g. `$ git grep "for _ in range("`). Another alternative would be using `itertools.repeat` (according to Python core developer Raymond Hettinger it's [even faster](https://twitter.com/raymondh/status/1144527183341375488)), but that doesn't seem to be widespread in use and I'm not sure about a readability increase.

  The only drawback I see is that whenever one wants to debug loop iterations, one would need to introduce a loop variable again. Reviewing this is basically a no-brainer, since tests would fail immediately if a a substitution has taken place on a loop where the variable is used.

  Instances to replace were found by `$ git grep "for.*in range("` and manually checked.

ACKs for top commit:
  darosior:
    ACK dac7a111bd
  instagibbs:
    manual inspection ACK dac7a111bd
  practicalswift:
    ACK dac7a111bd -- the updated code is easier to reason about since the throwaway nature of a variable is expressed explicitly (using the Pythonic `_` idiom) instead of implicitly. Explicit is better than implicit was we all know by now :)

Tree-SHA512: 5f43ded9ce14e5e00b3876ec445b90acda1842f813149ae7bafa93f3ac3d510bb778e2c701187fd2c73585e6b87797bb2d2987139bd1a9ba7d58775a59392406
2020-08-11 09:24:50 +08:00
Wladimir J. van der Laan
85fa648c85 Merge #19596: Deduplicate parent txid loop of requested transactions and missing parents of orphan transactions
4c0731f9c5 Deduplicate missing parents of orphan transactions (Suhas Daftuar)
8196176243 Rewrite parent txid loop of requested transactions (Suhas Daftuar)

Pull request description:

  I noticed a couple of places recently where we loop over all inputs of a transaction in order to do some processing on the txids we find in those inputs.  There may be thousands of inputs in a transaction, and the same txid may appear many times.  In a couple of places in particular, we loop over those txids and add them to a rolling bloom filter; doing that multiple times for the same txid wastes entries in that filter.

  This PR fixes that in two places relating to transaction relay: one on the server side, where we look for parent transactions of a tx that we are delivering to a peer to ensure that getdata requests for those parents will succeed; and the other on the client side, where when we process an orphan tx we want to loop over the parent txids and ensure that all are eventually requested from the peer who provided the orphan.

  This addresses a couple of [related](https://github.com/bitcoin/bitcoin/pull/19109#discussion_r455197217) [comments](https://github.com/bitcoin/bitcoin/pull/19109#discussion_r456820373) left in #19109.

ACKs for top commit:
  laanwj:
    Code review ACK 4c0731f9c5
  jonatack:
    ACK 4c0731f9c5
  ajtowns:
    ACK 4c0731f9c5

Tree-SHA512: 8af9df7f56c6e54b5915519d7d5465e081473ceb1bcc89bbebf83e78722cf51ff58145e588cf57126bce17071a8053273f4bcef0ad8166bec83ba14352e40f5d
2020-08-10 20:38:19 +02:00
Vasil Dimov
95975dd08d sync: detect double lock from the same thread
Double lock of the same (non-recursive) mutex from the same thread
is producing an undefined behavior. Detect this from DEBUG_LOCKORDER
and react similarly to the deadlock detection.
2020-08-10 18:43:08 +02:00
fanquake
6d5a9fec2d Merge #19605: doc: set CC_FOR_BUILD when building on OpenBSD
01cd24c226 doc: set CC_FOR_BUILD when building on OpenBSD (fanquake)

Pull request description:

  Closes: #19559

  While #19559 has been fixed upstream, it makes sense to not only
  recommend using `CC_FOR_BUILD`here  until the fix is pulled in as
  part of our next libsecp update, but after discussing with Cory,
  he suggested we should be setting this on OpenBSD (which still has
  the an ancient GCC) regardless.

ACKs for top commit:
  real-or-random:
    ACK 01cd24c226 I looked at the diff (but can't test the instructions on OpenBSD)
  laanwj:
    Code review ACK 01cd24c226

Tree-SHA512: 322802b9303771f1be2ad9628f268dfa71dc7ee77948fa2a34f21eceb19b2d8efdd8876c8f0778adbfcde48fa0f88cd4e698ae425428159abca38e8c7980da1d
2020-08-10 22:39:11 +08:00
fanquake
04a0676b3f Merge #17563: lib: fix a compiler warning: unused GetDevURandom()
ca2e474372 Fix a compiler warning: unused GetDevURandom() (Vasil Dimov)

Pull request description:

  ~~Only define GetDevURandom() if it is going to be used.~~

  Silence by planting a dummy reference to the `GetDevURandom` symbol
  in the places where we don't call the function.

ACKs for top commit:
  practicalswift:
    ACK ca2e474372 -- increased signal to noise in compiler diagnostics is good
  sipa:
    utACK ca2e474372
  hebasto:
    re-ACK ca2e474372, tested on macOS 10.15.6 + llvm clang 10.0.0

Tree-SHA512: 03c98f00dad5d9a3c5c9f68553d72ad5489ec02f18b9769108a22003ec7be7819a731b1eab6a9f64dafb5be0efddccf6980de7e3bb90cd20d4f4d72f74124675
2020-08-10 21:30:42 +08:00
fanquake
f306384f5a Merge #19622: build: Drop ancient hack in gitian-linux descriptor
33a84e8f40 build: Update and sort package list in gitian-linux.yml (Hennadii Stepanov)
95051682be build: Drop old hack which is unneeded now (Hennadii Stepanov)

Pull request description:

  The hack was aimed to fix an issue in Ubuntu Trusty 14.04 (see #8188).
  The current hack implementation was added in #8315.

  On master (8db23349fe) this hack is effectively noop, and it is no longer needed.

  I see this PR as a step to removing `libfaketime` from gitian builds.

ACKs for top commit:
  dongcarl:
    tACK 33a84e8f40
  laanwj:
    Code review ACK 33a84e8f40

Tree-SHA512: 90036c555a500649ccc3d108bf11f09a9cfd2c92c0b598f7e0c0df63a713ae7abaf78f350b68c025470619c967223f45f6a235ad37a6ce1d1a0341ed34963ba0
2020-08-10 20:15:09 +08:00
Hennadii Stepanov
4af4672525 build, qt: Add Qt version checking 2020-08-10 02:10:28 +03:00
Hennadii Stepanov
30e336f785 build: Drop unused bitcoin_cv_qt58 2020-08-10 02:10:12 +03:00
Wladimir J. van der Laan
be11f94e95 Merge #19631: test: Wait for 'cmpctblock' in p2p_compactblocks when it is expected
9e165d0de4 test: Wait for 'cmpctblock' in p2p_compactblocks when it is expected (Ben Woosley)

Pull request description:

  This is a more narrowly-construed wait which eliminates the possibility of the
  wait being triggered by other messages.

  Note `received_block_announcement` reflect three possible messages:
  edec7f7c25/test/functional/p2p_compactblocks.py (L34-L53)

  Prompted by looking into: #19449

ACKs for top commit:
  laanwj:
    Code review ACK 9e165d0de4
  theStack:
    ACK 9e165d0de4

Tree-SHA512: bc4a9c8bf031c8a7efb40d9625feaa3fd1f56f3b75da7034944af71ccea44328a6c708ab0c13fea85fb7cf4fd9043fe90eb94a25e95b2d42be44c2962b4904ce
2020-08-09 18:54:42 +02:00
Wladimir J. van der Laan
5e4fed9e58 Merge #19657: test: Wait until is_connected in add_p2p_connection
fa4dfd215f test: Wait until is_connected in add_p2p_connection (MarcoFalke)

Pull request description:

  Moving the wait_until from the individual test scripts to the test framework simplifies two tests

ACKs for top commit:
  jnewbery:
    Code review ACK fa4dfd215f
  theStack:
    ACK fa4dfd215f 

Tree-SHA512: 36eda7eb323614a4c4f9215f1d7b40b9f9c4036d1c08eb701ea705f3e2986fdabd2fc558965a6aadabeed861034aeaeef3c00f968ca17ed7a27e42e506cda87d
2020-08-09 18:35:31 +02:00
Wladimir J. van der Laan
214e6655c3 Merge #19649: Restore test case for p2p transaction blinding
566aada386 Test that wtxid relay peers add wtxid to reject filter (Gregory Sanders)
0fea6ede1b Restore test case for p2p transaction blinding (Gregory Sanders)

Pull request description:

  Introduced in ca10a03add then erroneously removed in 8d8099e97a. The restored line is how we are
  checking that the node will still re-request a specific txid given a witness-related failure.

ACKs for top commit:
  fjahr:
    tACK 566aada386

Tree-SHA512: be2b75b5eddb88019b79cc798f9922ca7347ccbb2210b8d4eae93fdde62e2cbb614b5247cb2fbd7ee3577dbe053875a9b62c5747aace8617f12790b8fccdeab4
2020-08-09 18:30:25 +02:00
Wladimir J. van der Laan
b75f2ad72d Merge #19660: refactor: Make HexStr take a span
0a8aa626dd refactor: Make HexStr take a span (Wladimir J. van der Laan)

Pull request description:

  Make `HexSt`r take a span of bytes, instead of an awkward pair of templated iterators. This simplifies most of the uses.

ACKs for top commit:
  elichai:
    Code review ACK 0a8aa626dd
  hebasto:
    re-ACK 0a8aa626dd
  jonatack:
    re-ACK 0a8aa626dd

Tree-SHA512: 6e178ece5cbac62119c857a10299b1e85422938084c3f03063e17119a5129e0c28016e05a6fabaa4c271a7e0a37c7cd89fa47c435ee19b38a5acfe80d00de992
2020-08-09 15:35:58 +02:00
Wladimir J. van der Laan
6ea7348122 Merge #19638: Replace hidden service with onion service
1e72b68ab3 Replace `hidden service` with `onion service` (Riccardo Masutti)

Pull request description:

  For a couple of years, Tor has made the term `hidden service` obsolete, in favor of `onion service`: [Tor Project | Onion Services](https://community.torproject.org/onion-services/)

  This PR updates all the references.

ACKs for top commit:
  laanwj:
    Code review ACK 1e72b68ab3
  hebasto:
    ACK 1e72b68ab3, tested on Linux Mint 20 (x86_64).

Tree-SHA512: 6a29e828e1c5e1ec934b5666f67326dbd84d77c8b2641f6740abac6d3d5923b7729763b9ff2230390b0bb23359a5f3731ccd9a30011ca69004f7c820aed17262
2020-08-09 15:08:20 +02:00
Hennadii Stepanov
75f9659d7a build: Add missed fuzz.coverage/ directory to .gitignore 2020-08-08 23:52:18 +03:00
Hennadii Stepanov
8ebc0505e9 build: Add missed fuzz_filtered.info to COVERAGE_INFO 2020-08-08 23:38:14 +03:00
Hennadii Stepanov
c71bdf93d7 build, test: Add support for llvm-cov 2020-08-08 22:53:15 +03:00
MarcoFalke
e349eeeb2c Merge #19672: build: make clean removes .gcda and .gcno files from fuzz directory
90bd476ea6 build: make clean removes .gcda and .gcno files from fuzz directory (eugene)

Pull request description:

  I believe these should also be deleted upon invoking `make clean`. It also garbles the coverage file if you try to fuzz the same harness again.

ACKs for top commit:
  practicalswift:
    ACK 90bd476ea6 -- patch looks correct
  hebasto:
    ACK 90bd476ea6, tested with hints from #12602 and #18107.
  darosior:
    ACK 90bd476ea6

Tree-SHA512: 4b2eb664f64d18bc0385c5a0040b0b9fa6fe470c941ae39c7cb4544c4283427a8d4985517475fe0295c3ab2794b9a2ad4f76b6a443c05d846c97c966add87ca9
2020-08-08 16:43:03 +02:00
Carl Dong
8c7cd0c6d9 depends: More robust cmake invocation
Specify well-known env vars instead of using a workaround to split up CC
and CXX.
2020-08-07 20:53:42 -04:00
Carl Dong
3ecf0eca63 depends: Use $($(package)_cmake) instead of cmake 2020-08-07 20:52:22 -04:00
Amiti Uttarwar
01e283068b [net] Remove unnecessary default args on CNode constructor 2020-08-07 17:18:17 -07:00
Amiti Uttarwar
bc5d65b3ca [refactor] Remove IsOutboundDisconnectionCandidate 2020-08-07 17:18:17 -07:00
Amiti Uttarwar
2f2e13b6c2 [net/refactor] Simplify multiple-connection checks
Extract logic that check multiple connection types into interface functions &
structure as switch statements. This makes it very clear what touch points are
for accessing `m_conn_type` & using the switch statements enables the compiler
to warn if a new connection type is introduced but not handled for these cases.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
7f7b83deb2 [net/refactor] Rework ThreadOpenConnections logic
Make the connection counts explicit and extract into interface functions around
m_conn_type. Using explicit counting and switch statements where possible
should help prevent counting bugs in the future.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
35839e963b [net] Fix bug where AddrFetch connections would be counted as outbound full relay
The desired logic is for us to only open feeler connections after we have hit
the max count for outbound full relay connections.  A short lived AddrFetch
connection (previously called oneshot) could cause ThreadOpenConnections to
miscount and mistakenly open a feeler instead of full relay.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
4972c21b67 [net/refactor] Clarify logic for selecting connections in ThreadOpenConnections 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
60156f5fc4 [net/refactor] Remove fInbound flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
7b322df629 [net/refactor] Remove m_addr_fetch member var from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
14923422b0 [net/refactor] Remove fFeeler flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
49efac5cae [net/refactor] Remove m_manual_connection flag from CNode 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
d3698b5ee3 [net/refactor] Add connection type as a member var to CNode
- Directly maintaining the connection type prevents having to deduce it from
  several flags.
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
46578c03e9 [doc] Describe different connection types 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
442abae2ba [net/refactor] Add AddrFetch connections to ConnectionType enum
- AddrFetch connections are short lived connections used to getaddr from a peer
- previously called "one shot" connections
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
af59feb052 [net/refactor] Extract m_addr_known logic from initializer list 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
e1bc29812d [net/refactor] Add block relay only connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
0e52a659a2 [net/refactor] Add feeler connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
1521c47438 [net/refactor] Add manual connections to ConnectionType enum 2020-08-07 17:18:16 -07:00
Amiti Uttarwar
26304b4100 [net/refactor] Introduce an enum to distinguish type of connection
- extract inbound & outbound types
2020-08-07 17:18:16 -07:00
Amiti Uttarwar
3f1b7140e9 scripted-diff: Rename OneShot to AddrFetch
-BEGIN VERIFY SCRIPT-
sed -i 's/a oneshot/an addrfetch/g' src/chainparams.cpp #comment
sed -i 's/oneshot/addrfetch/g' src/net.cpp #comment
sed -i 's/AddOneShot/AddAddrFetch/g' src/net.h src/net.cpp
sed -i 's/cs_vOneShots/m_addr_fetches_mutex/g' src/net.h src/net.cpp
sed -i 's/vOneShots/m_addr_fetches/g' src/net.h src/net.cpp
sed -i 's/fOneShot/m_addr_fetch/g' src/net.h src/net.cpp src/net_processing.cpp
sed -i 's/ProcessOneShot/ProcessAddrFetch/g' src/net.h src/net.cpp
-END VERIFY SCRIPT-
2020-08-07 17:18:12 -07:00
Jeremy Rubin
9ab4cafabd Refactor Get{Prevout,Sequence,Outputs}Hash to Get{Prevouts,Sequences,Outputs}SHA256.
Several proposals (Taproot, MuHash, CTV) require access to the single
hash.
2020-08-07 11:08:40 -07:00
Riccardo Masutti
1e72b68ab3 Replace hidden service with onion service
For a couple of years, Tor documentation has made
the term hidden service obsolete, in favor of onion
service.

This PR updates all the references in the code base.
2020-08-07 14:55:02 +02:00
Sjors Provoost
6d1f51343c [rpc] fundrawtransaction, walletcreatefundedpsbt lock manually selected coins
Previously only automatically selected coins were locked when lockUnspents is set.
It now also locks selected coins.
2020-08-07 14:13:15 +02:00
MarcoFalke
4b705b1c98 Merge #19098: test: Remove duplicate NodeContext hacks
edc316020e test: Remove duplicate NodeContext hacks (Russell Yanofsky)

Pull request description:

  Qt tests currently are currently using two NodeContext structs at the same time, one in interfaces::NodeImpl::m_context, and the other in BasicTestingSetup::m_node, and the tests have hacks transferring state between them.

  Fix this by getting rid of the NodeImpl::m_context struct and making it a pointer. This way a common BitcoinApplication object can be used for all qt tests, but they can still have their own testing setups.

  Non-test code is changing but non-test behavior is still the same as before.

  Motivation for this PR is to be able to remove the "std::move(test.m_node.connman)" and mempool hacks for swapping individual NodeContext members in Qt tests, because followup PR #19099 adds yet another member (wallet_client) that needs to be swapped. After this change, the whole NodeContext struct can be swapped instead of individual members, so the workarounds are less fragile and invasive.

ACKs for top commit:
  MarcoFalke:
    crACK edc316020e 🌮
  promag:
    ACK edc316020e.

Tree-SHA512: c1650e4127f43a4020304ca7c13b5d9122fb5723aacd8fa1cf855d03c6052fcfb7685810aa2a5ef708561015f0022fecaacbad479295104ca45d2c17579466a4
2020-08-07 08:07:37 +02:00
fanquake
6d8543504d Merge #19620: Add txids with non-standard inputs to reject filter
9f88ded82b test addition of unknown segwit spends to txid reject filter (Gregory Sanders)
7989901c7e Add txids with non-standard inputs to reject filter (Suhas Daftuar)

Pull request description:

  Our policy checks for non-standard inputs depend only on the non-witness
  portion of a transaction: we look up the scriptPubKey of the input being
  spent from our UTXO set (which is covered by the input txid), and the p2sh
  checks only rely on the scriptSig portion of the input.

  Consequently it's safe to add txids of transactions that fail these checks to
  the reject filter, as the witness is irrelevant to the failure. This is helpful
  for any situation where we might request the transaction again via txid (either
  from txid-relay peers, or if we might fetch the transaction via txid due to
  parent-fetching of orphans).

  Further, in preparation for future witness versions being deployed on the
  network, ensure that WITNESS_UNKNOWN transactions are rejected in
  AreInputsStandard(), so that transactions spending v1 (or greater) witness
  outputs will fall into this category of having their txid added to the reject
  filter.

ACKs for top commit:
  ajtowns:
    ACK 9f88ded82b - code review
  jnewbery:
    Code review ACK 9f88ded82b
  ariard:
    Code Review/Tested ACK 9f88ded
  naumenkogs:
    utACK 9f88ded82b
  jonatack:
    ACK 9f88ded82b

Tree-SHA512: 1e93c0a5b68cb432524780ffc0093db893911fdfed9e2ed17f888e59114cc75d2a07062aefad4e5ce2e87c9270886117a8abb3c78fb889c9b9f31967f1777148
2020-08-07 07:34:27 +08:00
Jeremy Rubin
6510d0ff41 Add SHA256Uint256 helper functions 2020-08-06 16:28:26 -07:00
Jeremy Rubin
b475d7d0fa Add single sha256 call to CHashWriter 2020-08-06 16:28:26 -07:00
Wladimir J. van der Laan
0a8aa626dd refactor: Make HexStr take a span
Make HexStr take a span of bytes, instead of an awkward pair of
templated iterators.
2020-08-06 19:41:43 +02:00
Sebastian Falbesoner
dac7a111bd refactor: test: use _ variable for unused loop counters
substitutes "for x in range(N):" by "for _ in range(N):"
indicates to the reader that a block is just repeated N times, and
that the loop counter is not used in the body
2020-08-06 18:39:33 +02:00
fanquake
82127d27c9 Merge #19667: build: set minimum required Boost to 1.58.0
70452a070b build: set minimum required Boost to 1.58 (fanquake)

Pull request description:

  Any systems which only have an older installable Boost can use depends.
  1.58.0 retains compatibility with the packages [installable on Ubuntu 16.04](https://packages.ubuntu.com/xenial/libboost-dev).

  The projects usage of Boost wont be going away any time soon, if ever (i.e #15382), and our usage of the test framework.

  Fixes: #19506

ACKs for top commit:
  practicalswift:
    ACK 70452a070b -- patch looks correct
  laanwj:
    ACK 70452a070b
  hebasto:
    ACK 70452a070b, tested on Linux Mint 20 (x86_64).

Tree-SHA512: d290415e3c70a394b3d7659c0480a35b4082bdce8d48b1c64a0025f7ad6e21567b4dc85813869513ad246d27f950706930410587c11c1aa3693ae6245084765c
2020-08-06 19:29:09 +08:00
eugene
90bd476ea6 build: make clean removes .gcda and .gcno files from fuzz directory
With this commit, make clean now removes coverage files from the
fuzzing directory. Without this, subsequent fuzzing runs would have
garbled coverage signals for files in the fuzz directory as
they were never deleted with make clean.
2020-08-06 00:44:03 -04:00
fanquake
19f812c57b Merge #19669: contrib: Fixup valgrind suppressions file
fa5288cf5c contrib: Fixup valgrind suppressions file (MarcoFalke)

Pull request description:

  I am observing this one on bionic with system boost::fs:

  ```
  {
     <insert_a_suppression_name_here>
     Memcheck:Cond
     fun:__wcsnlen_avx2
     fun:wcsnrtombs
     fun:_ZNKSt7codecvtIwc11__mbstate_tE6do_outERS0_PKwS4_RS4_PcS6_RS6_
     fun:_ZN5boost10filesystem11path_traits7convertEPKwS3_RNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEERKSt7codecvtIwc11__mbstate_tE
     fun:_ZN5boost10filesystem6detail11unique_pathERKNS0_4pathEPNS_6system10error_codeE
  ...

ACKs for top commit:
  practicalswift:
    ACK fa5288cf5c -- patch looks correct

Tree-SHA512: 067e10e932a7f5b13e516134e0cfd3030265c1b582cdfde1cea97042e31399aa40c4590710a39429854c68ad703a0ae9f0b06e9af4cdd81e3cacb042939a84b6
2020-08-06 10:14:44 +08:00
Karl-Johan Alm
7f13dfb587 test: test the implicit avoid partial spends functionality
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
2020-08-06 10:25:08 +09:00
Karl-Johan Alm
b82067bf69 wallet: try -avoidpartialspends mode and use its result if fees are below threshold
The threshold is defined by a new max avoid partial spends fee flag, which defaults to 0 (i.e. if fees are unchanged, use the grouped option).
2020-08-06 10:07:00 +09:00
Wladimir J. van der Laan
e3272ff290 Merge #19614: util: use HAVE_FDATASYNC to determine fdatasync() use
1d8338d6b7 util: use HAVE_FDATASYNC to determine fdatasync() use (fanquake)

Pull request description:

  Rather than just using on Linux and NetBSD, use `fdatasync()` based
  on whether it's available. i.e `fdatasync` is available in newer versions of FreeBSD.

  This also aligns more closely with what is being done in leveldb.

  Was pointed out by Luke in #19430.

ACKs for top commit:
  practicalswift:
    ACK 1d8338d6b7 -- patch looks correct
  laanwj:
    ACK 1d8338d6b7
  hebasto:
    ACK 1d8338d6b7

Tree-SHA512: 7dd6d87f5dc0c0ba21ae42f96b63fc12b34806cd536457fc4284f14bb8c235765344be228b000c6adf4cd1e8c4e6a03a18ca18ab22599c42cc3b706e0bcd1a17
2020-08-05 17:33:42 +02:00
Hennadii Stepanov
33a84e8f40 build: Update and sort package list in gitian-linux.yml
Virtual package 'binutils-gold' replaced with 'binutils'.
Explicitly added 'patch' package.
2020-08-05 18:08:47 +03:00
MarcoFalke
fa5288cf5c contrib: Fixup valgrind suppressions file 2020-08-05 16:43:30 +02:00
Wladimir J. van der Laan
4644b13b44 Merge #19632: test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli
82fc4017b7 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli (Ben Woosley)

Pull request description:

  `decimal.InvalidOperation` is a special case of a float parsing error, which
  presumably should be handled in the same way as a general parsing error,
  rather than blow up.

  Alternatives include: logging the error, or re-raising with more information.

  Example log output:
  ```
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
        self.sync_blocks(nodes)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
        best_hash = [x.getbestblockhash() for x in rpc_connections]
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
        return self.cli.send_cli(self.command, *args, **kwargs)
      File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
        return json.loads(cli_stdout, parse_float=decimal.Decimal)
      File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
        return cls(**kw).decode(s)
      File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
        obj, end = self.raw_decode(s, idx=_w(s, 0).end())
      File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
        obj, end = self.scan_once(s, idx)
    decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
  ```
  See: https://travis-ci.org/github/bitcoin/bitcoin/jobs/713502326

ACKs for top commit:
  laanwj:
    ACK 82fc4017b7

Tree-SHA512: 8c102b8bf831b05c5ca4b2e1feb5574dcbaed8cab0b2f22b013c5dfcb81788a38839a163dd1e2c6470ccbe5874214663b84485f45467738fd850ca38d539ae25
2020-08-05 16:27:54 +02:00
Samuel Dobson
e4df534c60 Merge #15382: util: add RunCommandParseJSON
31cf68a3ad [util] add RunCommandParseJSON (Sjors Provoost)
c17f54ee53 [ci] use boost::process (Sjors Provoost)
32128ba682 [doc] include Doxygen comments for HAVE_BOOST_PROCESS (Sjors Provoost)
3c84d85f7d [build] msvc: add boost::process (Sjors Provoost)
c47e4bbf0b [build] make boost-process opt-in (Sjors Provoost)
929cda5470 configure: add ax_boost_process (Sjors Provoost)
8314c23d7b [depends] boost: patch unused variable in boost_process (Sjors Provoost)

Pull request description:

  Prerequisite for external signer support in #16546. Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).

  This adds a new dependency [boost process](https://github.com/boostorg/process/tree/boost-1.64.0). This is part of Boost since 1.64 which is part of `depends`. Because the minimum Boost version is 1.47, this functionality is skipped for older versions of Boost.

  Use `./configure --with-boost-process` to opt in, which checks for the presence of Boost::Process.

  We add `UniValue runCommandParseJSON(const std::string& strCommand)` to `system.{h,cpp}` which calls an arbitrary command and processes the JSON returned by it. This is currently only called by the test suite.

  ~For testing purposes this adds a new regtest-only RPC method `runcommand`, as well as `test/mocks/command.py` used by functional tests.~ (this is no longer the case)

  TODO:
  - [ ] review boost process in #15440

ACKs for top commit:
  achow101:
    ACK 31cf68a3ad
  hebasto:
    re-ACK 31cf68a3ad, only rebased (verified with `git range-diff`) and removed an unintentional tab character since the [previous](https://github.com/bitcoin/bitcoin/pull/15382#pullrequestreview-458371035) review.
  meshcollider:
    Very light utACK 31cf68a3ad, although I am not very confident with build stuff.
  promag:
    Code review ACK 31cf68a3ad, don't mind the nit.
  ryanofsky:
    Code review ACK 31cf68a3ad. I left some comments below that could be ignored or followed up later. The current change is clean and comprehensive.

Tree-SHA512: c506e747014b263606e1f538ed4624a8ad7bcf4e025cb700c12cc5739964e254dc04a2bbb848996b170e2ccec3fbfa4fe9e2b3976b191222cfb82fc3e6ab182d
2020-08-05 23:43:43 +12:00
fanquake
70452a070b build: set minimum required Boost to 1.58
Any systems which only have an older install-able Boost can use depends.

Fixes: #19506
2020-08-05 17:13:45 +08:00
MarcoFalke
65e4ecabd5 Merge #19654: lint: Don't use TRAVIS_COMMIT_RANGE in commit message linter
72351784b3 lint: Remove travis env var from commit linter (Fabian Jahr)

Pull request description:

  #19439 was recently merged and seemed to work fine but I now noticed strange behavior when it was running in Travis, which I could not reproduce locally. It turns out `TRAVIS_COMMIT_RANGE` which is used in Travis to get the commits for the linter, uses all the commits that were in a push, which includes all rebase commits for example. This means that the linter can fail on a commit that the developer has never even seen before, which can be very confusing. See an example here which caused me to look into this: https://travis-ci.org/github/bitcoin/bitcoin/jobs/714296381 The commit that is reported as failing in my PR is not part of my PR.

  I think we rather want to use something like `git merge-base` to get the commit range by default and in Travis. I am leaving the env variable functionality in place with a different name but this is not a variable that can be expected to be present in the CI environments so the `merge-base` range should be used there by default.

ACKs for top commit:
  hebasto:
    ACK 72351784b3, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: afb27bb386855cb8d5cf84fd3a6c11ef1160b25af6175ed0aa146bf04b9a26eb77298df70df0a855f8c46f19f08b3f62c49872c12974fcfa5526a15ee05b3c10
2020-08-05 10:45:33 +02:00
Vasil Dimov
4df6567e4c sync: make EnterCritical() & push_lock() type safe
The functions `EnterCritical()` and `push_lock()` take a pointer to a
mutex, but that pointer used to be of type `void*` because we use a few
different types for mutexes. This `void*` argument was not type safe
because somebody could have send a pointer to anything that is not a
mutex. Furthermore it wouldn't allow to check whether the passed mutex
is recursive or not.

Thus, change the functions to templated ones so that we can implement
stricter checks for non-recursive mutexes. This also simplifies the
callers of `EnterCritical()`.
2020-08-05 09:42:42 +02:00
Suhas Daftuar
4c0731f9c5 Deduplicate missing parents of orphan transactions
In the logic for requesting missing parents of orphan transactions, parent
transactions with multiple outputs being spent by the given orphan were being
processed multiple times. Fix this by deduplicating the set of missing parent
txids first.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-08-04 13:59:16 -04:00
Suhas Daftuar
8196176243 Rewrite parent txid loop of requested transactions
Previously, we would potentially add the same txid many times to the rolling
bloom filter of recently announced transactions to a peer, if many outputs of
the same txid appeared as inputs in a transaction. Eliminate this problem and
avoid redundant lookups by asking the mempool for the unique parents of a
requested transaction.
2020-08-04 13:59:11 -04:00
Gregory Sanders
9f88ded82b test addition of unknown segwit spends to txid reject filter 2020-08-04 13:29:40 -04:00
Suhas Daftuar
7989901c7e Add txids with non-standard inputs to reject filter
Our policy checks for non-standard inputs depend only on the non-witness
portion of a transaction: we look up the scriptPubKey of the input being
spent from our UTXO set (which is covered by the input txid), and the p2sh
checks only rely on the scriptSig portion of the input.

Consequently it's safe to add txids of transactions that fail these checks to
the reject filter, as the witness is irrelevant to the failure. This is helpful
for any situation where we might request the transaction again via txid (either
from txid-relay peers, or if we might fetch the transaction via txid due to
parent-fetching of orphans).

Further, in preparation for future witness versions being deployed on the
network, ensure that WITNESS_UNKNOWN transactions are rejected in
AreInputsStandard(), so that transactions spending v1 (or greater) witness
outputs will fall into this category of having their txid added to the reject
filter.
2020-08-04 13:29:40 -04:00
Larry Ruane
8dd5946c0b add functional test 2020-08-04 11:25:55 -06:00
MarcoFalke
0f16212c59 Merge #19340: Preserve the LockData initial state if "potential deadlock detected" exception thrown
63e9e40b73 test: Add LockStackEmpty() (Hennadii Stepanov)
42b2a95373 test: Repeat deadlock tests (Hennadii Stepanov)
1f96be25b0 Preserve initial state if push_lock() throws exception (Hennadii Stepanov)

Pull request description:

  On master (e3fa3c7d67) if the `push_lock()` throws the "potential deadlock detected" exception (via the `potential_deadlock_detected()` call), the `LockData` instance internal state differs from one when the `push_lock()` was called. This non-well behaviour makes (at least) testing brittle.

  This PR preserves the `LockData` instance initial state if `push_lock()` throws an exception, and improves the `sync_tests` unit test.

ACKs for top commit:
  MarcoFalke:
    re-ACK 63e9e40b73
  vasild:
    ACK 63e9e40

Tree-SHA512: 7679182154ce5f079b44b790faf76eb5f553328dea70a326ff6b600db70e2f9ae015a33a104ca070cb660318280cb79b6b42e37ea5166f26f9e627ba721fcdec
2020-08-04 17:00:15 +02:00
MarcoFalke
fa4dfd215f test: Wait until is_connected in add_p2p_connection 2020-08-04 16:13:33 +02:00
MarcoFalke
3c93623be2 Merge #19489: test: Fail wait_until early if connection is lost
faa9a74c9e test: Fail wait_until early if connection is lost (MarcoFalke)

Pull request description:

  Calling `minonode.wait_until` needs a connection to make progress (e.g. waiting for an inv), unless the mininode waits for the initial connection or for a disconnection. So for test development and failure debugging, fail early in all `wait_until`, unless opted out.

ACKs for top commit:
  jnewbery:
    Code review ACK faa9a74c9e.

Tree-SHA512: 4be850b96e23b87bc2ff42c028a5045d6f5cdbc9482ce6a6ba01cc5eb26710dab9e2ed547c363aac4bd5825151ee9996fb797261420b631bceeddbfa698d1dec
2020-08-04 11:21:13 +02:00
Fabian Jahr
72351784b3 lint: Remove travis env var from commit linter 2020-08-04 10:48:56 +02:00
fanquake
bb2a9f9c8c Merge #19634: rpc: Document getwalletinfo's unlocked_until field as optional
f916847d2b rpc: Document getwalletinfo's unlocked_until field as optional (Justin Moon)

Pull request description:

  The `getwalletinfo` RPC command's `unlocked_until` field is [optional in the code](f916847d2b/src/wallet/rpcwallet.cpp (L2397)), but wasn't marked as optional in the docs.

ACKs for top commit:
  theStack:
    ACK f916847d2b
  achow101:
    ACK f916847d2b
  kristapsk:
    ACK f916847d2b

Tree-SHA512: 8d82f0992fdaf8160000acf4a6e7e7f9ff289a90a983be2e078cf754f4b03601637e5f405afa66bd55adef9b347fa5eac5cc1822033b2ac08c587609cf3dfe0f
2020-08-04 09:14:45 +08:00
Wladimir J. van der Laan
e02da22906 doc: Add CODEOWNERS file
Describes maintainers.

See https://help.github.com/en/articles/about-code-owners for syntax.

Ref #17094.
2020-08-03 16:43:08 -04:00
Wladimir J. van der Laan
34eb236258 Merge #19326: Simplify hash.h interface using Spans
77c507358b Make Hash[160] consume range-like objects (Pieter Wuille)
02c4cc5c5d Make CHash256/CHash160 output to Span (Pieter Wuille)
0ef97b1b10 Make MurmurHash3 consume Spans (Pieter Wuille)
e549bf8a9a Make CHash256 and CHash160 consume Spans (Pieter Wuille)
2a2182c387 Make script/standard's BaseHash Span-convertible (Pieter Wuille)
e63dcc3a67 Add MakeUCharSpan, to help constructing Span<[const] unsigned char> (Pieter Wuille)
567825049f Make uint256 Span-convertible by adding ::data() (Pieter Wuille)
131a2f0337 scripted-diff: rename base_blob::data to m_data (Pieter Wuille)

Pull request description:

  This makes use of the implicit constructions and conversions to Span introduced in #18468 to simplify the hash.h interface:

  * All functions that take a pointer and a length are changed to take a Span instead.
  * The Hash() and Hash160() functions are changed to take in "range" objects instead of begin/end iterators.

ACKs for top commit:
  laanwj:
    re-ACK 77c507358b
  jonatack:
    Code review re-ACK 77c5073 per `git range-diff 14ceddd 49fc016 77c5073`

Tree-SHA512: 9ec929891b1ddcf30eb14b946ee1bf142eca1442b9de0067ad6a3c181e0c7ea0c99c0e291e7f6e7a18bd7bdf78fe94ee3d5de66e167401674caf91e026269771
2020-08-03 17:27:49 +02:00
Wladimir J. van der Laan
14ceddd290 Merge #18991: Cache responses to GETADDR to prevent topology leaks
3bd67ba5a4 Test addr response caching (Gleb Naumenko)
cf1569e074 Add addr permission flag enabling non-cached addr sharing (Gleb Naumenko)
acd6135b43 Cache responses to addr requests (Gleb Naumenko)
7cc0e8101f Remove useless 2500 limit on AddrMan queries (Gleb Naumenko)
ded742bc5b Move filtering banned addrs inside GetAddresses() (Gleb Naumenko)

Pull request description:

  This is a very simple code change with a big p2p privacy benefit.

  It’s currently trivial to scrape any reachable node’s AddrMan (a database of all nodes known to them along with the timestamps).
  We do have a limit of one GETADDR per connection, but a spy can disconnect and reconnect even from the same IP, and send GETADDR again and again.

  Since we respond with 1,000 random records at most, depending on the AddrMan size it takes probably up to 100 requests for an spy to make sure they scraped (almost) everything.
  I even have a script for that. It is totally doable within couple minutes.

  Then, with some extra protocol knowledge a spy can infer the direct peers of the victim, and other topological stuff.

  I suggest to cache responses to GETADDR on a daily basis, so that an attacker gets at most 1,000 records per day, and can’t track the changes in real time. I will be following up with more improvements to addr relay privacy, but this one alone is a very effective. And simple!

  I doubt any of the real software does *reconnect to get new addrs from a given peer*, so we shouldn’t be cutting anyone.
  I also believe it doesn’t have any negative implications on the overall topology quality. And the records being “outdated” for at most a day doesn’t break any honest assumptions either.

ACKs for top commit:
  jnewbery:
    reACK 3bd67ba5a4
  promag:
    Code review ACK 3bd67ba5a4.
  ariard:
    Code Review ACK 3bd67ba

Tree-SHA512: dfa5d03205c2424e40a3f8a41af9306227e1ca18beead3b3dda44aa2a082175bb1c6d929dbc7ea8e48e01aed0d50f0d54491caa1147471a2b72a46c3ca06b66f
2020-08-03 14:48:52 +02:00
Hennadii Stepanov
20c9e03554 gui: Call setWalletActionsEnabled(true) only for the first wallet 2020-08-03 12:25:34 +03:00
Gregory Sanders
566aada386 Test that wtxid relay peers add wtxid to reject filter
h/t Anthony Towns
2020-08-02 23:34:11 -04:00
Gregory Sanders
0fea6ede1b Restore test case for p2p transaction blinding
Introduced in ca10a03add then erroneously removed in
8d8099e97a. The restored line is how we are
checking that the node will still re-request a specific txid given a witness-related failure.
2020-08-02 21:07:44 -04:00
MarcoFalke
fa77de2baa rpc: Assert that RPCArg names are equal to CRPCCommand ones (misc) 2020-08-02 21:33:27 +02:00
MarcoFalke
fa50bdc755 rpc: Limit echo to 10 args 2020-08-02 21:32:40 +02:00
MarcoFalke
fa89ca9b5b refactor: Use C++11 range based for loops to simplify rpc code 2020-08-02 21:32:36 +02:00
MarcoFalke
fa459bdc87 rpc: Treat all args after a hidden arg as hidden as well
This commit has no effect right now, but hardens the code for the future
2020-08-02 21:32:30 +02:00
Sebastian Falbesoner
f110b7c722 rpc: document returned error fields as optional if applicable
Affects the following RPCs:
- analyzepsbt
- estimatesmartfee
- signrawtransactionwithkey
- signrawtransactionwithwallet

For the RPC estimaterawfee, the description message was adapted
to match the other optional ones.
2020-08-02 18:44:41 +02:00
Hennadii Stepanov
63e9e40b73 test: Add LockStackEmpty() 2020-08-02 16:42:39 +03:00
Hennadii Stepanov
42b2a95373 test: Repeat deadlock tests 2020-08-02 16:42:39 +03:00
Hennadii Stepanov
1f96be25b0 Preserve initial state if push_lock() throws exception 2020-08-02 16:42:33 +03:00
MarcoFalke
a78742830a Merge #19639: doc: Remove Reference Links #19582
334de75885 scripted-diff: Remove Reference Links (Robert)

Pull request description:

  Removed all reference links.
  Found this issue from #19582.
  The decision to remove links instead of update them was made in #19584
  The author of that PR was slow to resolve his commit to use scripted diff so I made this PR instead.

ACKs for top commit:
  laanwj:
    ACK 334de75885
  MarcoFalke:
    ACK 334de75885

Tree-SHA512: a337116379912b27974867bd86ec7799a1d41d67b51771885467fbe1be003b415cb37ce8e521568bf3eae190ab2f6af0d6e29fd3ea25f2689b8fb31def8fec96
2020-08-02 12:42:36 +02:00
MarcoFalke
8443fb11ad Merge #19637: doc: Add historical release notes for 0.20.1
05e2740d92 doc: Add historical release notes for 0.20.1 (Wladimir J. van der Laan)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    ACK 05e2740d92

Tree-SHA512: 8f33403c51bcbaf97410642878bbc5ee4cbced2e079a61362a69c7eb9d682cf8d3d6ce16fe9aa99c2c89f21a57910032219312403b16d322d403b029f39c62ff
2020-08-02 09:50:22 +02:00
Robert
334de75885 scripted-diff: Remove Reference Links
-BEGIN VERIFY SCRIPT-
sed -i '/https:\/\/bitcoin.org\/en\/developer-reference/d' ./src/protocol.h
-END VERIFY SCRIPT-
2020-08-01 23:42:07 -04:00
Wladimir J. van der Laan
05e2740d92 doc: Add historical release notes for 0.20.1 2020-08-01 14:17:14 +02:00
Hennadii Stepanov
6ed4bcabc1 qt: Wrap tooltips in the intro window 2020-08-01 13:39:19 +03:00
Justin Moon
f916847d2b rpc: Document getwalletinfo's unlocked_until field as optional 2020-07-31 12:27:48 -05:00
Hennadii Stepanov
95051682be build: Drop old hack which is unneeded now
This hack is described in #8188. The current implementation was
introduced in #8315.
2020-07-31 17:10:40 +03:00
fanquake
f7c73b03d9 Merge #19569: Enable fetching of orphan parents from wtxid peers
10b7a6d532 refactor: make txmempool interface use GenTxid (Pieter Wuille)
5c124e1740 refactor: make FindTxForGetData use GenTxid (Pieter Wuille)
a2bfac8935 refactor: use GenTxid in tx request functions (Pieter Wuille)
e65d115b72 test: request parents of orphan from wtxid relay peer (Anthony Towns)
900d7f6c07 p2p: enable fetching of orphans from wtxid peers (Pieter Wuille)
9efd86a908 refactor: add GenTxid (=txid or wtxid) type and use it for tx request logic (Pieter Wuille)
d362f19355 doc: list support for BIP 339 in doc/bips.md (Pieter Wuille)

Pull request description:

  This is based on https://github.com/bitcoin/bitcoin/pull/18044#discussion_r450687076.

  A new type `GenTxid` is added to protocol.h, which represents a tagged txid-or-wtxid. The tx request logic is updated to use these instead of uint256s, permitting per-announcement distinguishing of txid/wtxid (instead of assuming that everything we want to request from a wtxid peer is wtx). Then the restriction of orphan-parent requesting to non-wtxid peers is lifted.

  Also document BIP339 in doc/bips.md.

ACKs for top commit:
  jnewbery:
    Code review ACK 10b7a6d532
  jonatack:
    ACK 10b7a6d532
  ajtowns:
    ACK 10b7a6d532 -- code review. Using gtxid to replace the is_txid_or_wtxid flag for the mempool functions is nice.
  naumenkogs:
    utACK 10b7a6d

Tree-SHA512: d518d13ffd71f8d2b3c175dc905362a7259689e6022a97a0b4f14f1f9fdd87475cf5af70cb12338d1e5d31b52c12e4faaea436114056a2ae9669cb506240758b
2020-07-31 19:52:18 +08:00
Sjors Provoost
31cf68a3ad [util] add RunCommandParseJSON 2020-07-31 13:38:10 +02:00
Sjors Provoost
c17f54ee53 [ci] use boost::process
Explictly opt-out on win64, in case the default changes.
2020-07-31 13:38:10 +02:00
Sjors Provoost
32128ba682 [doc] include Doxygen comments for HAVE_BOOST_PROCESS 2020-07-31 13:38:10 +02:00
Sjors Provoost
3c84d85f7d [build] msvc: add boost::process
* AppVeyor boost-process vcpkg package.
* Tell Boost linter to ignore it
* Add HAVE_BOOST_PROCESS for MSVC build (bitcoin_config.h)
2020-07-31 13:38:09 +02:00
Sjors Provoost
c47e4bbf0b [build] make boost-process opt-in 2020-07-31 13:38:09 +02:00
Sjors Provoost
929cda5470 configure: add ax_boost_process
Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
2020-07-31 13:38:09 +02:00
Sjors Provoost
8314c23d7b [depends] boost: patch unused variable in boost_process
Co-Authored-By: fanquake <fanquake@gmail.com>
2020-07-31 13:38:09 +02:00
MarcoFalke
a63a26f042 Merge #19585: rpc: RPCResult Type of MempoolEntryDescription should be OBJ.
ae4958be95 rpc: RPCResult Type of MempoolEntryDescription should be OBJ. If multiple entries are possible, wrapping Type should be OBJ_DYN. fixes #19579 (Chris L)

Pull request description:

  If multiple entries are possible, wrapping Type should be OBJ_DYN.

  fixes #19579

Top commit has no ACKs.

Tree-SHA512: 59cf9f6e9729a69a867e924d8306e0cd6b70a3d702fc5a4111345874bb1224ee51ac3f70cea61b25cfe6bde7f65cb02528d52acc20dda4eda692eddf34f217e8
2020-07-31 07:52:40 +02:00
Ben Woosley
82fc4017b7 test: Catch decimal.InvalidOperation from TestNodeCLI#send_cli
decimal.InvalidOperation is a special case of a float parsing error, which
presumably should be handled in the same way as a general parsing error,
rather than blow up.

Alternatives include: logging the error, or re-raising with more information.

Example log output:
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 603, in sync_all
      self.sync_blocks(nodes)
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in sync_blocks
      best_hash = [x.getbestblockhash() for x in rpc_connections]
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 568, in <listcomp>
      best_hash = [x.getbestblockhash() for x in rpc_connections]
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 571, in __call__
      return self.cli.send_cli(self.command, *args, **kwargs)
    File "/home/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_node.py", line 639, in send_cli
      return json.loads(cli_stdout, parse_float=decimal.Decimal)
    File "/usr/lib64/python3.6/json/__init__.py", line 367, in loads
      return cls(**kw).decode(s)
    File "/usr/lib64/python3.6/json/decoder.py", line 339, in decode
      obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/lib64/python3.6/json/decoder.py", line 355, in raw_decode
      obj, end = self.scan_once(s, idx)
  decimal.InvalidOperation: [<class 'decimal.InvalidOperation'>]
2020-07-30 18:45:53 -07:00
fanquake
a4a279b4f3 Merge #19617: doc: Clang 8 or later is required with FORCE_USE_SYSTEM_CLANG
e60ef21b8a doc: Clang 8 or later is required with FORCE_USE_SYSTEM_CLANG (fanquake)

Pull request description:

  The usage of pragmas within the macOS SDK requires LLVM Clang 8. This is
  the same version as our prebuilt Clang, however the minimum is worth noting
  here as they may diverge and/or expert users might expect they could use an
  earlier version.

  If you compile depends using `FORCE_USE_SYSTEM_CLANG=1` and Clang 7 you'll see output like:
  ```bash
  In file included from kernel/qcore_mac_objc.mm:44:
  In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSText.h:9:
  In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSView.h:19:
  In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSResponder.h:10:
  /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSEvent.h:19:1: error:
        expected 'push' or 'pop' after '#pragma clang attribute'
  /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/usr/include/os/availability.h:104:273: note: expanded from macro
        'API_UNAVAILABLE_BEGIN'
    ...__API_UNAVAILABLE_BEGIN5, __API_UNAVAILABLE_BEGIN4, __API_UNAVAILABLE_BEGIN3, __API_UNAVAILABLE_BEGIN2, __API_UNAVAILABLE_BEGIN1, 0)(__VA_A...
                                                                                                               ^
  fatal error: too many errors emitted, stopping now [-ferror-limit=]
  20 errors generated.
  ```

  I've got a [godbolt here](https://godbolt.org/z/j6r987) that contains a demo of the issue (based off the macOS SDK). It will compile with Clang 8 but not with Clang 7.

ACKs for top commit:
  laanwj:
    ACK e60ef21b8a

Tree-SHA512: affc082a4b28d9cd102a844e10f63f55f0f20a9f840e5d8037a41470ad1ea83edc3075ce262b1d618c59c9b75445e52ddd9ba9f84c5ac3da9ce20681473c47b1
2020-07-31 08:52:03 +08:00
Ben Woosley
9e165d0de4 test: Wait for 'cmpctblock' in p2p_compactblocks when it is expected
This is a more narrowly-construed wait which eliminates the possibility of the
wait being triggered by other messages.

Co-authored-by: Billy Garrison <billygarrison.btc@gmail.com>
2020-07-30 16:13:48 -07:00
Pieter Wuille
77c507358b Make Hash[160] consume range-like objects 2020-07-30 13:57:54 -07:00
Pieter Wuille
02c4cc5c5d Make CHash256/CHash160 output to Span 2020-07-30 13:57:54 -07:00
Pieter Wuille
0ef97b1b10 Make MurmurHash3 consume Spans 2020-07-30 13:57:54 -07:00
Pieter Wuille
e549bf8a9a Make CHash256 and CHash160 consume Spans 2020-07-30 13:57:53 -07:00
Pieter Wuille
2a2182c387 Make script/standard's BaseHash Span-convertible 2020-07-30 13:57:09 -07:00
Pieter Wuille
e63dcc3a67 Add MakeUCharSpan, to help constructing Span<[const] unsigned char>
Based on a suggestion by Russell Yanofsky.
2020-07-30 13:57:09 -07:00
Pieter Wuille
567825049f Make uint256 Span-convertible by adding ::data() 2020-07-30 13:57:09 -07:00
Pieter Wuille
131a2f0337 scripted-diff: rename base_blob::data to m_data
This is in preparation for exposing a ::data member function.

-BEGIN VERIFY SCRIPT-
sed -i "s/\([^.]\|other.\)data/\1m_data/g" src/uint256.h src/uint256.cpp
-END VERIFY SCRIPT-
2020-07-30 13:57:09 -07:00
Pieter Wuille
10b7a6d532 refactor: make txmempool interface use GenTxid 2020-07-30 13:45:03 -07:00
Pieter Wuille
5c124e1740 refactor: make FindTxForGetData use GenTxid 2020-07-30 13:45:02 -07:00
Pieter Wuille
a2bfac8935 refactor: use GenTxid in tx request functions 2020-07-30 13:45:02 -07:00
Anthony Towns
e65d115b72 test: request parents of orphan from wtxid relay peer 2020-07-30 13:45:02 -07:00
Pieter Wuille
900d7f6c07 p2p: enable fetching of orphans from wtxid peers
Based on a commit by Anthony Towns.
2020-07-30 13:45:02 -07:00
Pieter Wuille
9efd86a908 refactor: add GenTxid (=txid or wtxid) type and use it for tx request logic 2020-07-30 13:44:54 -07:00
Pieter Wuille
d362f19355 doc: list support for BIP 339 in doc/bips.md 2020-07-30 12:39:34 -07:00
Andrew Chow
7d07e864b8 Use real value when calculating OutputGroup value
OutputGroup::m_value is the true value, not the effective value,
so use the true values of the outputs, not their effective values.
2020-07-30 12:51:32 -04:00
MarcoFalke
edec7f7c25 Merge #19439: script: Linter to check commit message formatting
284a969cc0 Linter to check commit message formatting (Amir Ghorbanian)

Pull request description:

  Write linter to check that commit messages have a new line before the body or no body at all. fixes issue #19091.

ACKs for top commit:
  troygiorshev:
    ACK 284a969cc0 Reviewed, manually tested. Works great!
  fjahr:
    tested ACK 284a969cc0
  adamjonas:
    utACK 284a969cc0

Tree-SHA512: fa278f090780b54e4fa6e2967a62b4c1a4da55d112ec1ad6dd7e1181ac490c5c1af0165524b5781b463fdd6d0f79fd3d95b5160184e6eca432ccff1189f77390
2020-07-30 17:32:37 +02:00
MarcoFalke
ad2952d17a Merge #19604: Pass mempool pointer to UnloadBlockIndex/GetCoinsCacheSizeState
fae8c28dae Pass mempool pointer to GetCoinsCacheSizeState (MarcoFalke)
fac674db20 Pass mempool pointer to UnloadBlockIndex (MarcoFalke)
faec851b6e test: Simplify cs_main locks (MarcoFalke)

Pull request description:

  Split out from #19556

  Instead of relying on the implicit mempool global, pass a mempool pointer (which can be `0`). This helps with testing, code clarity and unlocks the features described in #19556.

ACKs for top commit:
  jnewbery:
    code review ACK fae8c28dae
  fjahr:
    Code review ACK fae8c28dae
  darosior:
    Tested ACK fae8c28dae
  jamesob:
    ACK fae8c28dae ([`jamesob/ackr/19604.1.MarcoFalke.pass_mempool_pointer_to`](https://github.com/jamesob/bitcoin/tree/ackr/19604.1.MarcoFalke.pass_mempool_pointer_to))

Tree-SHA512: fa687518c8cda4a095bdbdfe56e01fae2fb16c13d51efbb1312cd6dc007611fc47f53f475602e4a843e3973c9410e6af5a81d6847bd2399f8262ca7205975728
2020-07-30 17:30:52 +02:00
MarcoFalke
62d137ac3b Merge #19561: refactor: Pass ArgsManager into functions that register args
8ed9002cd1 refactor: use local argsmanager in CRegTestParams (Ivan Metlushko)
9b20f66828 scripted-diff: Replace gArgs with local argsman (Ivan Metlushko)
a316e9ce26 refactor: add unused ArgsManager to replace gArgs (Ivan Metlushko)

Pull request description:

  Rationale: reduce use of gArgs to decouple code and simplify future maintenance and easier unit testing.

  This PR is continuation of work started in  #18926 and #18662
  It covers only places that register args in ArgsManager with `AddArgs()` or `AddHiddenArgs()`.

  Closes #19511

ACKs for top commit:
  MarcoFalke:
    ACK 8ed9002cd1 👛

Tree-SHA512: 7e6ba8e8357a48833c71e9c3942a769acb3d93bdcc6748a8ef2b7c4461a2499419b60896abf1d8b6bf8e88ee2590284cdd5da64220243ac22375300bcb8fe3e8
2020-07-30 17:08:46 +02:00
MarcoFalke
37b765b962 Merge #19102: wallet: Introduce and use DummyDatabase instead of dummy BerkeleyDatabase
0fcff547d5 walletdb: Ensure that having no database handle is a failure (Andrew Chow)
da039d2a91 Remove BDB dummy databases (Andrew Chow)
0103d6434e Introduce DummyDatabase and use it in the tests (Andrew Chow)

Pull request description:

  In the unit tests, we use a dummy `WalletDatabase` which does nothing and always returns true. This is currently implemented by creating a `BerkeleyDatabase` in dummy mode. This PR instead adds a `DummyDatabase` class which does nothing and never fails for use in the tests. `CreateDummyWalletDatabase` is changed to return this `DummyDatabase` and `BerkeleyDatabase` is cleaned up to remove all of the checks for `IsDummy`.

  Based on `WalletDatabase` abstract class introduced in #19334

ACKs for top commit:
  instagibbs:
    utACK 0fcff547d5
  MarcoFalke:
    crACK 0fcff547d5 🚈

Tree-SHA512: 05fbf32e078753e9a55a05f4c080b6d365b909a2a3a8e571b7e64b59ebbe53da49394f70419cc793192ade79f312f5e0422ca7c261ba81bae5912671c5ff6402
2020-07-30 17:01:11 +02:00
Wladimir J. van der Laan
17de75b028 Merge #19590: p2p, refactor: add CInv transaction message helpers; use in net processing
c251d710a4 p2p, refactoring: use CInv helpers in net_processing.cpp (Jon Atack)
4254cd9f8f p2p: add CInv transaction message helper methods (Jon Atack)

Pull request description:

  Following the merge of wtxid relay in #18044, this is the first of three refactoring PRs (this one, #19610, and #19611) with no change in behavior, tightly scoped to ease review, to simplify the net processing code and improve encapsulation:

  - add `CInv` transaction message helper methods, defined in the class

  - use the new helpers in `net_processing.cpp` to simplify the code and improve encapsulation

  Test coverage is provided by the functional p2p tests, notably (from seeing which tests failed when breaking things to test coverage) `p2p_segwit`, `p2p_tx_download`, `p2p_feefilter`, and `p2p_permissions`.

ACKs for top commit:
  fjahr:
    Code review ACK c251d710a4
  laanwj:
    Code review ACK c251d710a4
  vasild:
    ACK c251d71
  theStack:
    Code-Review ACK c251d710a4
  hebasto:
    ACK c251d710a4, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: ead034b3c9e438909b4c5010c570d7930e69063c114290b051b7cebfd9bd5b19f573218bebe8a521256d32e830797f997adad3d85b4539c64ac5762b698e656d
2020-07-30 16:18:06 +02:00
Wladimir J. van der Laan
4ebe2f6e75 Merge #18011: Replace current benchmarking framework with nanobench
78c312c983 Replace current benchmarking framework with nanobench (Martin Ankerl)

Pull request description:

  Replace current benchmarking framework with nanobench

  This replaces the current benchmarking framework with nanobench [1], an
  MIT licensed single-header benchmarking library, of which I am the
  autor. This has in my opinion several advantages, especially on Linux:

  * fast: Running all benchmarks takes ~6 seconds instead of 4m13s on
    an Intel i7-8700 CPU @ 3.20GHz.

  * accurate: I ran e.g. the benchmark for SipHash_32b 10 times and
    calculate standard deviation / mean = coefficient of variation:

    * 0.57% CV for old benchmarking framework
    * 0.20% CV for nanobench

    So the benchmark results with nanobench seem to vary less than with
    the old framework.

  * It automatically determines runtime based on clock precision, no need
    to specify number of evaluations.

  * measure instructions, cycles, branches, instructions per cycle,
    branch misses (only Linux, when performance counters are available)

  * output in markdown table format.

  * Warn about unstable environment (frequency scaling, turbo, ...)

  * For better profiling, it is possible to set the environment variable
    NANOBENCH_ENDLESS to force endless running of a particular benchmark
    without the need to recompile. This makes it to e.g. run "perf top"
    and look at hotspots.

  Here is an example copy & pasted from the terminal output:

  |             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |                2.52 |      396,529,415.94 |    0.6% |           25.42 |            8.02 |  3.169 |           0.06 |    0.0% |      0.03 | `bench/crypto_hash.cpp RIPEMD160`
  |                1.87 |      535,161,444.83 |    0.3% |           21.36 |            5.95 |  3.589 |           0.06 |    0.0% |      0.02 | `bench/crypto_hash.cpp SHA1`
  |                3.22 |      310,344,174.79 |    1.1% |           36.80 |           10.22 |  3.601 |           0.09 |    0.0% |      0.04 | `bench/crypto_hash.cpp SHA256`
  |                2.01 |      496,375,796.23 |    0.0% |           18.72 |            6.43 |  2.911 |           0.01 |    1.0% |      0.00 | `bench/crypto_hash.cpp SHA256D64_1024`
  |                7.23 |      138,263,519.35 |    0.1% |           82.66 |           23.11 |  3.577 |           1.63 |    0.1% |      0.00 | `bench/crypto_hash.cpp SHA256_32b`
  |                3.04 |      328,780,166.40 |    0.3% |           35.82 |            9.69 |  3.696 |           0.03 |    0.0% |      0.03 | `bench/crypto_hash.cpp SHA512`

  [1] https://github.com/martinus/nanobench

ACKs for top commit:
  laanwj:
    ACK 78c312c983

Tree-SHA512: 9e18770b18b6f95a7d0105a4a5497d31cf4eb5efe6574f4482f6f1b4c88d7e0946b9a4a1e9e8e6ecbf41a3f2d7571240677dcb45af29a6f0584e89b25f32e49e
2020-07-30 15:34:17 +02:00
Gleb Naumenko
3bd67ba5a4 Test addr response caching 2020-07-30 14:38:50 +03:00
Gleb Naumenko
cf1569e074 Add addr permission flag enabling non-cached addr sharing 2020-07-30 14:38:50 +03:00
Gleb Naumenko
acd6135b43 Cache responses to addr requests
Prevents a spy from scraping victim's AddrMan by
reconnecting and re-requesting addrs.
2020-07-30 14:38:48 +03:00
MarcoFalke
2a784723f0 Merge #19597: test: test decodepsbt fee calculation (count input value only once per UTXO)
82dee87933 test: test decodepsbt fee calculation (count input value only once per UTXO) (Sebastian Falbesoner)

Pull request description:

  Fixes #19523, adding a simple test to `rpc_psbt.py` that checks that the decodepsbt fee matches the one given by the wallet (`walletcreatefundedpsbt`). This is in particular important for PSBTs with segwit inputs that have both a witness- and a non-witness-UTXO type set.

  Example test run after reverting commit 75122780e2 ("Increment input value sum only once per UTXO in decodepsbt"):

  ```
  $ test/functional/rpc_psbt.py
  2020-07-26T11:31:44.862000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test__sutcd4y
  20.00007580
  2020-07-26T11:31:47.073000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/honeybadger/buidl/bitcoin_thestack/test/functional/test_framework/test_framework.py", line 118, in main
      self.run_test()
    File "test/functional/rpc_psbt.py", line 166, in run_test
      assert_equal(decoded['fee'], created_psbt['fee'])
    File "/home/honeybadger/buidl/bitcoin_thestack/test/functional/test_framework/util.py", line 49, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
  AssertionError: not(20.00007580 == 0.00007580)
  2020-07-26T11:31:47.125000Z TestFramework (INFO): Stopping nodes
  ......
  ```

ACKs for top commit:
  achow101:
    ACK 82dee87933

Tree-SHA512: 296b8a701f851d482ef6200c6cbf0cf0257a79a828ac6dbc39b05d8c2d839c6fdb9d3f5a084015295cfa3eac7c11faa2f2d52e619c11627b04c75150eead8330
2020-07-30 09:46:16 +02:00
MarcoFalke
149eca433d Merge #19599: test: clean message_count and last_message
2c6a02e024 Clean message_count and last_message (Troy Giorshev)

Pull request description:

  From #19580

  This PR changes comments to clarify the intended usage of `message_count` and `last_message`.  Additionally it changes the only usage of `message_count` to use `last_message` instead, bringing the code into alignment with the intended usage.

  Note: Now `message_count` is completely unused.  However, it is ready to be used (i.e. the supporting code works) and likely will be used in some test in the future.

ACKs for top commit:
  jnewbery:
    utACK 2c6a02e024

Tree-SHA512: 07c7684c9586de4f845e10d7aac36c1aab9fb56b409949c1c70d5ca705bc3971ca7d5943245a0472def4efd7b4e1c5dad2f713db5ead8fca08404daf4891e98b
2020-07-30 09:15:49 +02:00
Andrew Chow
0fcff547d5 walletdb: Ensure that having no database handle is a failure
Previously having no database handle could still be considered a success
when BerkeleyDatabase and BerkeleyBatch were used for dummy database
things. With dedicated DummyDatabase and DummyBatch classes now, these
should fail.
2020-07-29 12:30:24 -04:00
Andrew Chow
da039d2a91 Remove BDB dummy databases 2020-07-29 12:30:23 -04:00
Andrew Chow
0103d6434e Introduce DummyDatabase and use it in the tests 2020-07-29 12:28:30 -04:00
Wladimir J. van der Laan
8db23349fe Merge #19335: wallet: Cleanup and separate BerkeleyDatabase and BerkeleyBatch
74507ce71e walletdb: Remove BerkeleyBatch friend class from BerkeleyDatabase (Andrew Chow)
00f0041351 No need to check for duplicate fileids in all dbenvs (Andrew Chow)
d86efab370 walletdb: Move Db->open to BerkeleyDatabase::Open (Andrew Chow)
4fe4b3bf1b walletdb: track database file use as m_refcount within BerkeleyDatabase (Andrew Chow)
65fb8807ac Combine BerkeleyEnvironment::Verify into BerkeleyDatabase::Verify (Andrew Chow)

Pull request description:

  `BerkeleyBatch` and `BerkeleyDatabase` are kind of messy. The goal of this is to clean up them up so that they are logically separated.

  `BerkeleyBatch` currently handles the creation of the `BerkeleyDatabase`'s `Db` handle. This is instead moved into `BerkeleyDatabase` and is called by `BerkeleyBatch`.

  Instead of having `BerkeleyEnvironment` track each database's usage, have `BerkeleyDatabase` track this usage itself with the `m_refcount` variable that is present in `WalletDatabase`.

  Lastly, instead of having each `BerkeleyEnvironment` store the fileids of the databases open in it, have a global `g_fileids` to track those fileids. We were already checking fileid uniqueness globally (by checking the fileids in every environment when opening a database) so it's cleaner to do this with a global variable.

  All of these changes allow us to make `BerkeleyBatch` and `BerkeleyDatabase` no longer be friend classes.

  The diff of this PR is currently the same as in ##18971

  Requires #19334

ACKs for top commit:
  laanwj:
    Code review ACK 74507ce71e
  ryanofsky:
    Code review ACK 74507ce71e. No changes since last review other than rebase

Tree-SHA512: 845d84ee1a470e2bf5d2e2e3d7738183d8ce43ddd06a0bbd57edecf5779b2f55d70728b1b57f5daab0f078650a8d60c3e19dc30b75b36e7aa952ce268399d5f6
2020-07-29 18:24:16 +02:00
Wladimir J. van der Laan
400f45ec9b Merge #19525: build: add -Wl,-z,separate-code to hardening flags
65d0f1a533 devtools: Add security check for separate_code (Wladimir J. van der Laan)
2e9e6377f1 build: add -Wl,-z,separate-code to hardening flags (fanquake)

Pull request description:

  TLDR: We are generally explicit about the hardening related flags we use,
  rather than letting the distro / toolchain decide via their defaults. This PR
  adds `-z,separate-code` which has been enabled by default for Linux targets
   since binutils 2.31. Ubuntu Bionic (currently used for gitian) ships with
  binutils 2.30, so this will enable the option for those builds.

  This flag was added to binutils/ld in the 2.30 release,
  see commit c11c786f0b45617bb8807ab6a57220d5ff50e414:

  > The new "-z separate-code" option will generate separate code LOAD
  segment which must be in wholly disjoint pages from any other data.

  It was made the default for Linux/x86 targets in the 2.31 release, see commit
  f6aec96dce1ddbd8961a3aa8a2925db2021719bb:

  > This patch adds --enable-separate-code to ld configure to turn on
  -z separate-code by default and enables it by default for Linux/x86.
  This avoids mixing code pages with data to improve cache performance
  as well as security.

  > To reduce x86-64 executable and shared object sizes, the maximum page
  size is reduced from 2MB to 4KB when -z separate-code is turned on by
  default.  Note: -z max-page-size= can be used to set the maximum page
  size.

  > We compared SPEC CPU 2017 performance before and after this change on
  Skylake server.  There are no any significant performance changes.
  Everything is mostly below +/-1%.

  Support was also added to LLVMs lld: https://reviews.llvm.org/D64903, however
  there it remains off by default.

  There were concerns about an increase in binary size, however in our case, the
  difference would seem negligible, given we are shipping a
  multi-megabyte binary, which then downloads 100's of GBs of data.

  Also note that most recent versions of distros are shipping a new enough version
  of binutils that this is available and/or already on by default (assuming the distro
  has not turned it off, I haven't checked everywhere):

  CentOS 8: 2.30
  Debian Buster 2.31.1
  Fedora 29: 2.31.1
  FreeBSD: 2.33
  GNU Guix: 2.33 / 2.34
  Ubuntu 18.04: 2.30

  Related threads / discussion:
  https://bugzilla.redhat.com/show_bug.cgi?id=1623218

  The ELF header when building on Debian Buster (where it's already enabled by default in binutils):
  ```bash
  Program Header:
      PHDR off    0x0000000000000040 vaddr 0x0000000000000040 paddr 0x0000000000000040 align 2**3
           filesz 0x00000000000002a0 memsz 0x00000000000002a0 flags r--
    INTERP off    0x00000000000002e0 vaddr 0x00000000000002e0 paddr 0x00000000000002e0 align 2**0
           filesz 0x000000000000001c memsz 0x000000000000001c flags r--
      LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**12
           filesz 0x0000000000038f10 memsz 0x0000000000038f10 flags r--
      LOAD off    0x0000000000039000 vaddr 0x0000000000039000 paddr 0x0000000000039000 align 2**12
           filesz 0x00000000006b9389 memsz 0x00000000006b9389 flags r-x
      LOAD off    0x00000000006f3000 vaddr 0x00000000006f3000 paddr 0x00000000006f3000 align 2**12
           filesz 0x0000000000204847 memsz 0x0000000000204847 flags r--
      LOAD off    0x00000000008f7920 vaddr 0x00000000008f8920 paddr 0x00000000008f8920 align 2**12
           filesz 0x00000000000183e0 memsz 0x0000000000022fd0 flags rw-
   DYNAMIC off    0x000000000090adb0 vaddr 0x000000000090bdb0 paddr 0x000000000090bdb0 align 2**3
           filesz 0x0000000000000240 memsz 0x0000000000000240 flags rw-
  ```
   vs when opting out using `-Wl,-z,noseparate-code`:
  ```bash
  Program Header:
      PHDR off    0x0000000000000040 vaddr 0x0000000000000040 paddr 0x0000000000000040 align 2**3
           filesz 0x0000000000000230 memsz 0x0000000000000230 flags r--
    INTERP off    0x0000000000000270 vaddr 0x0000000000000270 paddr 0x0000000000000270 align 2**0
           filesz 0x000000000000001c memsz 0x000000000000001c flags r--
      LOAD off    0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**12
           filesz 0x00000000008f6a87 memsz 0x00000000008f6a87 flags r-x
      LOAD off    0x00000000008f7920 vaddr 0x00000000008f8920 paddr 0x00000000008f8920 align 2**12
           filesz 0x00000000000183e0 memsz 0x0000000000022fd0 flags rw-
   DYNAMIC off    0x000000000090adb0 vaddr 0x000000000090bdb0 paddr 0x000000000090bdb0 align 2**3
           filesz 0x0000000000000240 memsz 0x0000000000000240 flags rw-
  ```

ACKs for top commit:
  laanwj:
    ACK 65d0f1a533

Tree-SHA512: 6e40e434efea8a8e39f6cb244dfd16aaa5a9db5a2ea762a05d1727357b20e33b7e47c1a652ee88490c9d7952a4caa2f992396fb30346239300d37ae123e36d49
2020-07-29 16:52:51 +02:00
Wladimir J. van der Laan
a76ccb01b9 Merge #19534: net: save the network type explicitly in CNetAddr
bcfebb6d55 net: save the network type explicitly in CNetAddr (Vasil Dimov)
100c64a95b net: document `enum Network` (Vasil Dimov)

Pull request description:

  (chopped off from https://github.com/bitcoin/bitcoin/pull/19031 to ease review)

  Before this change, we would analyze the contents of `CNetAddr::ip[16]`
  in order to tell which type is an address. Change this by introducing a
  new member `CNetAddr::m_net` that explicitly tells the type of the
  address.

  This is necessary because in BIP155 we will not be able to tell the
  address type by just looking at its raw representation (e.g. both TORv3
  and I2P are "seemingly random" 32 bytes).

  As a side effect of this change we no longer need to store IPv4
  addresses encoded as IPv6 addresses - we can store them in proper 4
  bytes (will be done in a separate commit). Also the code gets
  somewhat simplified - instead of
  `memcmp(ip, pchIPv4, sizeof(pchIPv4)) == 0` we can use
  `m_net == NET_IPV4`.

ACKs for top commit:
  troygiorshev:
    reACK bcfebb6d55 via `git range-diff master 64897c5 bcfebb6`
  jonatack:
    re-ACK bcfebb6 per `git diff 662bb25 bcfebb6`, code review, debug build/tests clean, ran bitcoind.
  laanwj:
    Code review ACK bcfebb6d55

Tree-SHA512: 9347e2a50feac617a994bfb46a8f77e31c236bde882e4fd4f03eea4766cd5110216f5f3d24dee91d25218bab7f8bb6e1d2d6212a44db9e34594299fd6ff7606b
2020-07-29 13:31:16 +02:00
MarcoFalke
fae8c28dae Pass mempool pointer to GetCoinsCacheSizeState 2020-07-29 12:30:11 +02:00
MarcoFalke
fac674db20 Pass mempool pointer to UnloadBlockIndex 2020-07-29 12:29:51 +02:00
Ivan Metlushko
8ed9002cd1 refactor: use local argsmanager in CRegTestParams 2020-07-29 16:39:03 +07:00
Ivan Metlushko
9b20f66828 scripted-diff: Replace gArgs with local argsman
-BEGIN VERIFY SCRIPT-
sed -i -e 's/gArgs.Add/argsman.Add/g' `git grep -l "gArgs.Add"`
-END VERIFY SCRIPT-
2020-07-29 16:39:00 +07:00
Ivan Metlushko
a316e9ce26 refactor: add unused ArgsManager to replace gArgs 2020-07-29 16:36:44 +07:00
MarcoFalke
faec851b6e test: Simplify cs_main locks 2020-07-29 08:00:54 +02:00
MarcoFalke
2f71a1ea35 Merge #18637: coins: allow cache resize after init
f19fdd47a6 test: add test for CChainState::ResizeCoinsCaches() (James O'Beirne)
8ac3ef4699 add ChainstateManager::MaybeRebalanceCaches() (James O'Beirne)
f36aaa6392 Add CChainState::ResizeCoinsCaches (James O'Beirne)
b223111da2 txdb: add CCoinsViewDB::ChangeCacheSize (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  In the assumeutxo implementation draft (#15056), once a UTXO snapshot is loaded, a new chainstate object is created after initialization. This means that we have to reclaim some of the cache that we've allocated to the original chainstate (per `dbcache=`) to repurpose for the snapshot chainstate.

  Furthermore, it makes sense to have different cache allocations depending on which chainstate is more active. While the snapshot chainstate is working to get to the network tip (and the background validation chainstate is idle), it makes sense that the snapshot chainstate should have the majority of cache allocation. And contrariwise once the snapshot has reached network tip, most of the cache should be given to the background validation chainstate.

  This set of changes (detailed in the commit messages) allows us to dynamically resize the various coins caches. None of the functionality introduced here is used at the moment, but will be in the next AU PR (which introduces `ActivateSnapshot`).

  `ChainstateManager::MaybeRebalanceCaches()` defines the (somewhat normative) cache allocations between the snapshot and background validation chainstates. I'd be interested in feedback if anyone has thoughts on the proportions I've set there.

ACKs for top commit:
  ajtowns:
    weak utACK f19fdd47a6 -- didn't find any major problems, but not super confident that I didn't miss anything
  fjahr:
    Code review ACK f19fdd4
  ryanofsky:
    Code review ACK f19fdd47a6. Only change since last review is constructor cleanup (no change in behavior). I think the suggestions here from ajtowns and others are good, but shouldn't delay merging the PR (and hold up assumeutxo)

Tree-SHA512: fffb7847fb6993dd4a1a41cf11179b211b0b20b7eb5f7cf6266442136bfe9d43b830bbefcafd475bfd4af273f5573500594aa41fff03e0ed5c2a1e8562ff9269
2020-07-29 07:53:19 +02:00
fanquake
e60ef21b8a doc: Clang 8 or later is required with FORCE_USE_SYSTEM_CLANG
The usage of pragmas within the macOS SDK requires LLVM Clang 8. This is
the version as our prebuilt Clang, however the minimum is worth noting here
as they may diverge and/or expert users might expect they could use an
earlier version.

If you compile using Clang 7 you'll see output like:
```bash
In file included from kernel/qcore_mac_objc.mm:44:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSText.h:9:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSView.h:19:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSResponder.h:10:
/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSEvent.h:19:1: error:
      expected 'push' or 'pop' after '#pragma clang attribute'
/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/usr/include/os/availability.h:104:273: note: expanded from macro
      'API_UNAVAILABLE_BEGIN'
  ...__API_UNAVAILABLE_BEGIN5, __API_UNAVAILABLE_BEGIN4, __API_UNAVAILABLE_BEGIN3, __API_UNAVAILABLE_BEGIN2, __API_UNAVAILABLE_BEGIN1, 0)(__VA_A...
                                                                                                             ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
```
2020-07-29 10:49:54 +08:00
MarcoFalke
a41ae68053 Merge #19613: travis: Re-enable s390x
fae656144e travis: Re-enable s390x (MarcoFalke)

Pull request description:

  According to travis, the issue has been solved. Quote

  > I would like to confirm that we have resolved this issue and most of our users are reported that this issue has been resolved on their end as well. Could you please re-check and see if that still exists for you?

ACKs for top commit:
  theStack:
    ACK fae656144e

Tree-SHA512: cf42f96d25474a9dcf0817a049e30e29714731d708f73c40a3042b0c70a71ff08f07dd96a89f0dcd5a50a63a355cf30b3511172a32b8af7d5a2e13ad222a4b49
2020-07-28 17:51:58 +02:00
fanquake
1d8338d6b7 util: use HAVE_FDATASYNC to determine fdatasync() use
Rather than just using it on Linux and NetBSD, use `fdatasync()` based
on whether it's available. i.e it is available in newer versions
of FreeBSD (11.1 and later).

This also aligns our code more closely with what is being done in leveldb.

Was pointed out by Luke in #19430.
2020-07-28 22:39:56 +08:00
MarcoFalke
fae656144e travis: Re-enable s390x 2020-07-28 16:01:53 +02:00
fanquake
01cd24c226 doc: set CC_FOR_BUILD when building on OpenBSD
Closes: #19559

While #19559 has been fixed upstream, it makes sense to not only
recommend using `CC_FOR_BUILD`here  until the fix is pulled in as
part of our next libsecp update, but after discussing with Cory,
he suggested we should be setting this on OpenBSD (which still has
the an ancient GCC) regardless.
2020-07-28 16:29:47 +08:00
MarcoFalke
24ead1a923 Merge #19519: ci: Increase CCACHE_SIZE in some builds on Travis
fdf697fe75 ci: Increase CCACHE_SIZE in some builds on Travis (Hennadii Stepanov)

Pull request description:

  This PR will decrease build time for the longest Travis builds.
  Up to ~20 min improvement with warmed up cache is expected.

Top commit has no ACKs.

Tree-SHA512: ee3a22162d03537be11e60b53de043247f6a65e35e630e0807b758a846b7e05ef2059d18846644aafa3cd5dc7d3e3f56a8ccb94cc71b95227debc9201d3142a2
2020-07-28 10:25:11 +02:00
fanquake
e0ed4cc50d Merge #19530: depends: build LTO support into Apple's ld64
5962522fbc depends: bump native_cctools for fixed lto with external clang (Cory Fields)
00d1ba7aaa depends: enable lto support for Apple's ld64 (Cory Fields)

Pull request description:

  This didn't work for a few reasons (various toolchain compatibility issues) the last time I tested it, but after the last round of bumps it works with no apparent issues.

  Note that this does not _enable_ LTO by default in any way, only hooks up the machinery for ```-flto``` to work correctly when specified.

  Lines were split for an easier rebase after #17919 is merged.

ACKs for top commit:
  fanquake:
    ACK 5962522fbc. The relevant option upstream is [here](https://github.com/tpoechtrager/cctools-port/blob/master/cctools/m4/llvm.m4#L4).

Tree-SHA512: df2775e74e7bc847e6cef94cb8457d503d6c9e2fdea861e51386fa6ed5a7ba688241db3685561ae1a32f66724c1b3801727252025f00c04b90a3bdc8a4f6f93b
2020-07-28 16:01:26 +08:00
fanquake
cea39fc268 Merge #19595: Fix .gitignore for src/test/fuzz directory
623f66554d Fix .gitignore for src/test/fuzz directory (Hennadii Stepanov)

Pull request description:

  On master (31d2b4098a):
  ```
  $ git ls-files --ignored --exclude-standard
  contrib/init/org.bitcoin.bitcoind.plist
  contrib/macdeploy/fancy.plist
  src/qt/Makefile
  src/qt/test/Makefile
  src/test/Makefile
  src/test/fuzz/FuzzedDataProvider.h
  src/test/fuzz/addition_overflow.cpp
  src/test/fuzz/addrdb.cpp
  src/test/fuzz/asmap.cpp
  src/test/fuzz/asmap_direct.cpp
  src/test/fuzz/autofile.cpp
  src/test/fuzz/banman.cpp
  src/test/fuzz/base_encode_decode.cpp
  src/test/fuzz/bech32.cpp
  src/test/fuzz/block.cpp
  src/test/fuzz/block_header.cpp
  src/test/fuzz/blockfilter.cpp
  src/test/fuzz/bloom_filter.cpp
  src/test/fuzz/buffered_file.cpp
  src/test/fuzz/chain.cpp
  src/test/fuzz/checkqueue.cpp
  src/test/fuzz/coins_view.cpp
  src/test/fuzz/crypto.cpp
  src/test/fuzz/crypto_aes256.cpp
  src/test/fuzz/crypto_aes256cbc.cpp
  src/test/fuzz/crypto_chacha20.cpp
  src/test/fuzz/crypto_chacha20_poly1305_aead.cpp
  src/test/fuzz/crypto_common.cpp
  src/test/fuzz/crypto_hkdf_hmac_sha256_l32.cpp
  src/test/fuzz/crypto_poly1305.cpp
  src/test/fuzz/cuckoocache.cpp
  src/test/fuzz/decode_tx.cpp
  src/test/fuzz/descriptor_parse.cpp
  src/test/fuzz/deserialize.cpp
  src/test/fuzz/eval_script.cpp
  src/test/fuzz/fee_rate.cpp
  src/test/fuzz/fees.cpp
  src/test/fuzz/flatfile.cpp
  src/test/fuzz/float.cpp
  src/test/fuzz/fuzz.cpp
  src/test/fuzz/fuzz.h
  src/test/fuzz/golomb_rice.cpp
  src/test/fuzz/hex.cpp
  src/test/fuzz/http_request.cpp
  src/test/fuzz/integer.cpp
  src/test/fuzz/key.cpp
  src/test/fuzz/key_io.cpp
  src/test/fuzz/kitchen_sink.cpp
  src/test/fuzz/load_external_block_file.cpp
  src/test/fuzz/locale.cpp
  src/test/fuzz/merkleblock.cpp
  src/test/fuzz/message.cpp
  src/test/fuzz/multiplication_overflow.cpp
  src/test/fuzz/net_permissions.cpp
  src/test/fuzz/netaddress.cpp
  src/test/fuzz/p2p_transport_deserializer.cpp
  src/test/fuzz/parse_hd_keypath.cpp
  src/test/fuzz/parse_iso8601.cpp
  src/test/fuzz/parse_numbers.cpp
  src/test/fuzz/parse_script.cpp
  src/test/fuzz/parse_univalue.cpp
  src/test/fuzz/policy_estimator.cpp
  src/test/fuzz/policy_estimator_io.cpp
  src/test/fuzz/pow.cpp
  src/test/fuzz/prevector.cpp
  src/test/fuzz/primitives_transaction.cpp
  src/test/fuzz/process_message.cpp
  src/test/fuzz/process_messages.cpp
  src/test/fuzz/protocol.cpp
  src/test/fuzz/psbt.cpp
  src/test/fuzz/random.cpp
  src/test/fuzz/rbf.cpp
  src/test/fuzz/rolling_bloom_filter.cpp
  src/test/fuzz/script.cpp
  src/test/fuzz/script_bitcoin_consensus.cpp
  src/test/fuzz/script_descriptor_cache.cpp
  src/test/fuzz/script_flags.cpp
  src/test/fuzz/script_interpreter.cpp
  src/test/fuzz/script_ops.cpp
  src/test/fuzz/script_sigcache.cpp
  src/test/fuzz/script_sign.cpp
  src/test/fuzz/scriptnum_ops.cpp
  src/test/fuzz/signature_checker.cpp
  src/test/fuzz/span.cpp
  src/test/fuzz/spanparsing.cpp
  src/test/fuzz/string.cpp
  src/test/fuzz/strprintf.cpp
  src/test/fuzz/system.cpp
  src/test/fuzz/timedata.cpp
  src/test/fuzz/transaction.cpp
  src/test/fuzz/tx_in.cpp
  src/test/fuzz/tx_out.cpp
  src/test/fuzz/util.h
  src/univalue/gen/gen.cpp
  test/functional/data/wallets/high_minversion/db.log
  test/functional/data/wallets/high_minversion/wallet.dat
  ```

  With this PR:
  ```
  $ git ls-files --ignored --exclude-standard
  contrib/init/org.bitcoin.bitcoind.plist
  contrib/macdeploy/fancy.plist
  src/qt/Makefile
  src/qt/test/Makefile
  src/test/Makefile
  src/univalue/gen/gen.cpp
  test/functional/data/wallets/high_minversion/db.log
  test/functional/data/wallets/high_minversion/wallet.dat
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 623f66554d seems like an improvement when writing new fuzz tests
  practicalswift:
    ACK 623f66554d -- thanks for fixing! ❤️
  theStack:
    tested ACK 623f66554d

Tree-SHA512: 16b3854bf4fd8c3096d915a4efc5cbc63d28b18854b051bafee374508dfbb5871ae7dc6f303dbf57469473082d2c3a7df0a8170da22d60d13878544679363b5c
2020-07-28 15:19:00 +08:00
fanquake
2979a7aff0 Merge #19583: p2p: clean up Misbehaving()
a8865f8b72 [net processing] Tidy up Misbehaving() (John Newbery)
d15b3afb4c [net processing] Always supply debug message to Misbehaving() (John Newbery)
634144a1c2 [net processing] Fixup MaybeDiscourageAndDisconnect() style (John Newbery)

Pull request description:

  This PR makes a few minor clean-ups to `Misbehaving()` in preparation to move it out of the cs_main lock.

  There are very minor logging changes but otherwise no functional changes.

ACKs for top commit:
  troygiorshev:
    tACK a8865f8b72
  jonatack:
    ACK a8865f8
  fjahr:
    Code review ACK a8865f8b72
  promag:
    Code review ACK a8865f8b72.

Tree-SHA512: 98fb4f5f76399715545a1ea19290dcebfc8cb4eff72a1d3555dd3de6e184040bb8668c9651dab21db0dfd8e674e53a5977105ef76547146c9f6fa6b4b9d2ba59
2020-07-28 15:15:35 +08:00
fanquake
a1da180b1b Merge #19589: rpc: Avoid useless mempool query in gettxoutproof
fa5979d12f rpc: Avoid useless mempool query in gettxoutproof (MarcoFalke)
fa1f7f28cb rpc: Style fixups in gettxoutproof (MarcoFalke)

Pull request description:

  `GetTransaction` implicitly and unconditionally asks the mempool global for a transaction. This is problematic for several reasons:

  * `gettxoutproof` is for on-chain txs only and asking the mempool for on-chain txs is confusing and minimally wasteful
  * Globals are confusing and make code harder to test with unit tests

  Fix both issues by passing in an optional mempool. This also helps with #19556

ACKs for top commit:
  hebasto:
    re-ACK fa5979d12f
  jnewbery:
    utACK fa5979d12f
  promag:
    Code review ACK fa5979d12f.

Tree-SHA512: 048361b82abfcc40481181bd44f70cfc9e97d5d6356549df34bbe30b9de7a0a72d2207a3ad0279b21f06293509b284d8967f58ca7e716263a22b20aa4e7f9c54
2020-07-28 14:54:00 +08:00
Wladimir J. van der Laan
65d0f1a533 devtools: Add security check for separate_code
Check that sections are appropriately separated in virtual memory,
based on their (expected) permissions. This checks for missing
-Wl,-z,separate-code and potentially other problems.

Co-authored-by: fanquake <fanquake@gmail.com>
2020-07-28 12:57:35 +08:00
fanquake
2e9e6377f1 build: add -Wl,-z,separate-code to hardening flags
This flag was added to binutils/ld in the 2.30 release, 
see commit c11c786f0b45617bb8807ab6a57220d5ff50e414:

> The new "-z separate-code" option will generate separate code LOAD
segment which must be in wholly disjoint pages from any other data.


It was made the default for Linux/x86 targets in the 2.31 release, see commit
f6aec96dce1ddbd8961a3aa8a2925db2021719bb:

> This patch adds --enable-separate-code to ld configure to turn on
-z separate-code by default and enables it by default for Linux/x86.
This avoids mixing code pages with data to improve cache performance
as well as security.

> To reduce x86-64 executable and shared object sizes, the maximum page
size is reduced from 2MB to 4KB when -z separate-code is turned on by
default.  Note: -z max-page-size= can be used to set the maximum page
size.

> We compared SPEC CPU 2017 performance before and after this change on
Skylake server.  There are no any significant performance changes.
Everything is mostly below +/-1%.

Support was also added to LLVMs lld: https://reviews.llvm.org/D64903, however
there is remains off by default.

There were concerns about an increase in binary size, however in our case, the
increase (1 page worth of bytes) would seem negligible, given we are shipping a
multi-megabyte binary, which then downloads 100's of GBs of data.

Also note that most recent versions of distros are shipping a new enough version
of binutils that this is available and/or on by default (assuming the distro has
not turned it off, I haven't checked everywhere):

CentOS 8: 2.30
Debian Buster 2.31.1
Fedora 29: 2.31.1
FreeBSD: 2.33
GNU Guix: 2.33 / 2.34
Ubuntu 18.04: 2.30

Related threads / discussion:
https://bugzilla.redhat.com/show_bug.cgi?id=1623218
2020-07-28 12:57:35 +08:00
Vasil Dimov
bcfebb6d55 net: save the network type explicitly in CNetAddr
Before this change, we would analyze the contents of `CNetAddr::ip[16]`
in order to tell which type is an address. Change this by introducing a
new member `CNetAddr::m_net` that explicitly tells the type of the
address.

This is necessary because in BIP155 we will not be able to tell the
address type by just looking at its raw representation (e.g. both TORv3
and I2P are "seemingly random" 32 bytes).

As a side effect of this change we no longer need to store IPv4
addresses encoded as IPv6 addresses - we can store them in proper 4
bytes (will be done in a separate commit). Also the code gets
somewhat simplified - instead of
`memcmp(ip, pchIPv4, sizeof(pchIPv4)) == 0` we can use
`m_net == NET_IPV4`.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-07-27 15:13:24 +02:00
Wladimir J. van der Laan
b62fbf9e1c Merge #19593: refactor: Drop unused CBufferedFile::Seek()
7b3851e947 refactor: Drop unused CBufferedFile::Seek() (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK 7b3851e947 -- deleted code is better than unused untested code:)
  MarcoFalke:
    ACK 7b3851e947, assuming that removing this should either be correct or result in a compile failure
  jonasschnelli:
    utACK 7b3851e947
  promag:
    Code review ACK 7b3851e947.

Tree-SHA512: 7bfd172aa4bbe349855c1303fd9cd58093d66833fefe46bd29081bfcca4ab434b84c6b84e76e94d06b8749a5abe1dc1e184f5189136cd1403d0e5bc25ad6d456
2020-07-27 13:57:34 +02:00
Troy Giorshev
2c6a02e024 Clean message_count and last_message
This commit clarifies the intended usage of message_count and
last_message.  Additionally it changes the only usage of message_count
to using last_message instead, bringing the code further along the
intended usage.
2020-07-27 07:55:49 -04:00
Jon Atack
c251d710a4 p2p, refactoring: use CInv helpers in net_processing.cpp
to simplify the code and reach less from it into the CInv class internals
2020-07-27 11:06:48 +02:00
Jon Atack
4254cd9f8f p2p: add CInv transaction message helper methods 2020-07-27 11:06:39 +02:00
Andrew Chow
0e279fe489 walletdb: Remove unused static functions from walletdb.h
VerifyEnvironment and VerifyDatabaseFile were removed, but their
declarations weren't. Remove those.
2020-07-26 20:22:49 -04:00
Andrew Chow
9f536d4fe9 wallettool: Have RecoverDatabaseFile return errors and warnings
Instead of logging or printing these errors and warnings, return them to
the caller.
2020-07-26 20:22:45 -04:00
Hennadii Stepanov
623f66554d Fix .gitignore for src/test/fuzz directory 2020-07-27 00:56:37 +03:00
Hennadii Stepanov
7b3851e947 refactor: Drop unused CBufferedFile::Seek() 2020-07-26 22:46:28 +03:00
MarcoFalke
31d2b4098a Merge #19508: Work around memory-aliasing in descriptor ParsePubkey
fa2ae0ac8d span: Add Span::empty() and use it in script/descriptor (MarcoFalke)
fa8a992589 Work around memory-aliasing in descriptor ParsePubkey (MarcoFalke)

Pull request description:

  While this is not undefined behaviour, the memory aliasing trick is confusing when reading the code. Having `a.size()==0` and then access `a[0]` works in this particular case, but should probably be avoided to harden the code for the future.

ACKs for top commit:
  theStack:
    re-ACK fa2ae0ac8d
  elichai:
    ACK fa2ae0ac8d
  jonatack:
    ACK fa2ae0ac8d

Tree-SHA512: 0ec7b09eef45504973a195923cdf1aa8522117c8e2f69b453e5ce9aa8a7e327c71138518022c32d05133dc99cb861101ed0f60fa891814ee3e9dab3a6fa61a84
2020-07-26 19:01:51 +02:00
MarcoFalke
fa5979d12f rpc: Avoid useless mempool query in gettxoutproof 2020-07-26 16:44:07 +02:00
MarcoFalke
fa1f7f28cb rpc: Style fixups in gettxoutproof 2020-07-26 15:43:35 +02:00
Sebastian Falbesoner
82dee87933 test: test decodepsbt fee calculation (count input value only once per UTXO)
Checks that the RPC decodepsbt calculates the fee correctly, in particular for
PSBTs with segwit inputs that have both a witness- and a non-witness-UTXO type
set. Before commit 75122780e2 ("Increment input
value sum only once per UTXO in decodepsbt") the values for those inputs were
double counted.
2020-07-26 13:25:16 +02:00
Hennadii Stepanov
b6dcc6d741 gui: Clarify block height label 2020-07-25 20:41:35 +03:00
John Newbery
a8865f8b72 [net processing] Tidy up Misbehaving()
- Make const things const.
- Replace conditional return with assert.
- Don't log the peer's IP address.
- Log the name Misbehaving directly instead of relying on __func__.
2020-07-25 15:52:23 +01:00
John Newbery
d15b3afb4c [net processing] Always supply debug message to Misbehaving()
Misbehaving() could optionally take a debug string for printing to the
log file. Make this mandatory and always provide the string.

A couple of additional minor changes:

- remove the unnecessary forward declaration of Misbehaving()
- don't include the nodeid or newline in the passed debug message.
Misbehaving() adds these itself.
2020-07-25 15:50:34 +01:00
John Newbery
634144a1c2 [net processing] Fixup MaybeDiscourageAndDisconnect() style
Based on review comments from Marco Falke and Jon Atack.
2020-07-25 15:49:24 +01:00
Fabian Jahr
e7448d6680 wallet: Don't override signing errors 2020-07-25 00:00:36 +02:00
Chris L
ae4958be95 rpc: RPCResult Type of MempoolEntryDescription should be OBJ. If multiple entries are possible, wrapping Type should be OBJ_DYN. fixes #19579 2020-07-24 18:15:51 +02:00
Wladimir J. van der Laan
40a04814d1 Merge #19472: [net processing] Reduce cs_main scope in MaybeDiscourageAndDisconnect()
655b195747 [net processing] Continue SendMessages processing if not disconnecting peer (John Newbery)
a49781e56d [net processing] Only call MaybeDiscourageAndDisconnect from SendMessages (John Newbery)
a1d5a428a2 [net processing] Fix bad indentation in SendMessages() (John Newbery)
1a1c23f8d4 [net processing] Change cs_main TRY_LOCK to LOCK in SendMessages() (John Newbery)

Pull request description:

  The motivation for this PR is to reduce the scope of cs_main locking in misbehavior logic. It is the first set of commits from a larger branch to move the misbehavior data out of CNodeState and into a new struct that doesn't take cs_main.

  There are some very minor behavior changes in this branch, such as:

  - Not checking for discouragement/disconnect in `ProcessMessages()` (and instead relying on the following check in `SendMessages()`)
  - Checking for discouragement/disconnect as the first action in `SendMessages()` (and not doing ping message sending first)
  - Continuing through `SendMessages()` if `MaybeDiscourageAndDisconnect()` doesn't disconnect the peer (rather than dropping out of `SendMessages()`

ACKs for top commit:
  jonatack:
    re-ACK 655b195 per `git range-diff 505b4ed f54af5e 655b195`, code/commit messages review, a bit of code history, and debug build.
  MarcoFalke:
    ACK 655b195747 only some style-nits 🚁
  promag:
    Code review ACK 655b195747.
  ariard:
    Code Review ACK 655b195

Tree-SHA512: fd6d7bc6bb789f5fb7771fb6a45f61a8faba32af93b766554f562144f9631d15c9cc849a383e71743ef73e610b4ee14853666f6fbf08a3ae35176d48c76c65d3
2020-07-24 17:20:58 +02:00
Gleb Naumenko
7cc0e8101f Remove useless 2500 limit on AddrMan queries 2020-07-24 18:02:20 +03:00
Gleb Naumenko
ded742bc5b Move filtering banned addrs inside GetAddresses() 2020-07-24 18:02:20 +03:00
Pavol Rusnak
8c4129b454 rpc: reset scantxoutset progress on finish 2020-07-24 11:04:48 +02:00
fanquake
007e15dcd7 Merge #19565: build: call AC_PATH_TOOL for dsymutil in macOS cross-compile
ef3d4ce4c3 build: call AC_PATH_TOOL for dsymutil in macOS cross-compile (fanquake)

Pull request description:

  While testing #19530 I noticed that we couldn't call [`dsymutil`](https://www.llvm.org/docs/CommandGuide/dsymutil.html) after LTO:
  ```bash
  ../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
  ```

  This updates configure to call `AC_PATH_TOOL` so that we end up with the
  full path to dsymutil, similar to `otool` and `install_name_tool`, ie:
  `/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-dsymutil`.

ACKs for top commit:
  laanwj:
    Code review ACK ef3d4ce4c3
  theuni:
    ACK ef3d4ce4c3.

Tree-SHA512: e4fa93e7f9f7945289143dfe2a6645ad8ee7f3bee0793412b3509901a30566d6f952e3b39e0e525a54f8dbd0c480f8da70fc6cb80b07800d11b0c6071fbb7466
2020-07-24 08:57:51 +08:00
MarcoFalke
f4cfa6d019 Merge #15935: Add <datadir>/settings.json persistent settings storage
9c69cfe4c5 Add <datadir>/settings.json persistent settings storage. (Russell Yanofsky)
eb682c5700 util: Add ReadSettings and WriteSettings functions (Russell Yanofsky)

Pull request description:

  Persistent settings are used in followup PRs #15936 to unify gui settings between bitcoin-qt and bitcoind, and #15937 to add a load_on_startup flag to the loadwallet RPC and maintain a dynamic list of wallets that should be loaded on startup that also can be shared between bitcoind and bitcoin-qt.

ACKs for top commit:
  MarcoFalke:
    Approach re-ACK 9c69cfe4c5 🌾
  jnewbery:
    utACK 9c69cfe4c5

Tree-SHA512: 39fcc6051717117c9141e934de1d0d3f739484be4685cdf97d54de967c8c816502b4fd0de12114433beaa5c5b7060c810fd8ae4e2b3ce7c371eb729ac01ba2e1
2020-07-23 18:39:42 +02:00
MarcoFalke
6ee36a263c Merge #19473: net: Add -networkactive option
2aac093a3d test: Add test coverage for -networkactive option (Hennadii Stepanov)
3c58129b12 net: Log network activity status change unconditionally (Hennadii Stepanov)
62fe6aa87e net: Add -networkactive option (Hennadii Stepanov)

Pull request description:

  Some Bitcoin Core activity is completely local (offline), e.g., reindexing.

  The `setnetworkactive` RPC command is already present. This PR adds the corresponding command-line argument / config option, and allows to start the client with disabled p2p network by providing `-networkactive=0` or `-nonetworkactive`.

  This was done while reviewing #16981.

ACKs for top commit:
  MarcoFalke:
    re-ACK 2aac093a3d 🏠
  LarryRuane:
    ACK 2aac093a3d

Tree-SHA512: 446d791b46d7b556d7694df7b1f88cd4fbc09301fe4eaf036b45cb8166ed806156353cc03788a07b633d5887d5eee30a7c02a2d4307141c8ccc75e0a88145636
2020-07-23 18:32:59 +02:00
Andrew Chow
74507ce71e walletdb: Remove BerkeleyBatch friend class from BerkeleyDatabase 2020-07-22 23:30:19 -04:00
Andrew Chow
00f0041351 No need to check for duplicate fileids in all dbenvs
Since we have .walletlock in each directory, we don't need the duplicate
fileid checks across all dbenvs as it shouldn't be possible anyways.
2020-07-22 23:30:19 -04:00
Andrew Chow
d86efab370 walletdb: Move Db->open to BerkeleyDatabase::Open
Instead of opening the Db handle in BerkeleyBatch, make BerkeleyDatabase
do that.
2020-07-22 23:30:19 -04:00
Andrew Chow
4fe4b3bf1b walletdb: track database file use as m_refcount within BerkeleyDatabase
Instead of having BerkeleyEnvironment track the file use count, make
BerkeleyDatabase do it itself.
2020-07-22 23:30:19 -04:00
Andrew Chow
65fb8807ac Combine BerkeleyEnvironment::Verify into BerkeleyDatabase::Verify 2020-07-22 23:30:19 -04:00
Samuel Dobson
9d4b3d86b6 Merge #19334: wallet: Introduce WalletDatabase abstract class
d416ae560e walletdb: Introduce WalletDatabase abstract class (Andrew Chow)
2179dbcbcd walletdb: Add BerkeleyDatabase::Open dummy function (Andrew Chow)
71d28e7cdc walletdb: Introduce AddRef and RemoveRef functions (Andrew Chow)
27b2766384 walletdb: Move BerkeleyDatabase::Flush(true) to Close() (Andrew Chow)

Pull request description:

  A `WalletDatabase` abstract class is created from `BerkeleyDatabase` and is implemented by `BerkeleyDatabase`. First, to get to the point that this is possible, 4 functions need to be added to `BerkeleyDatabase`: `AddRef`, `RemoveRef`, `Open`, and `Close`.

  First the increment and decrement of `mapFileUseCount` is refactored into separate functions `AddRef` and `RemoveRef`.

  `Open` is introduced as a dummy function. This will raise an exception so that it always fails.

  `Close` is refactored from `Flush`. The `shutdown` argument in `Flush` is removed and instead `Flush(true)` is now the `Close` function.

  Split from #18971

  Requires #19325

ACKs for top commit:
  ryanofsky:
    Code review ACK d416ae560e. Only changes since last review were rebasing after base PR #19334 merge, and adding cs_db lock in BerkeleyDatabase destructor, which should avoid races accessing env->m_databases and env->m_fileids
  fjahr:
    Code review ACK d416ae560e
  meshcollider:
    Code review & test run ACK d416ae560e

Tree-SHA512: 98d05ec093d7446c4488e2b0914584222a331e9a2f4d5be6af98e3f6d78fdd8e75526c12f91a8a52d4820c25bce02aa02aabe92d38bee7eb2fce07d0691b7b0d
2020-07-23 15:22:25 +12:00
Hennadii Stepanov
2aac093a3d test: Add test coverage for -networkactive option 2020-07-22 22:55:48 +03:00
Hennadii Stepanov
3c58129b12 net: Log network activity status change unconditionally 2020-07-22 22:55:47 +03:00
Hennadii Stepanov
62fe6aa87e net: Add -networkactive option
The `setnetworkactive' RPC command is already present.
This new option allows to start the client with disabled p2p network
activity for testing or reindexing.
2020-07-22 22:55:11 +03:00
Wladimir J. van der Laan
ccef10261e Merge #18044: Use wtxid for transaction relay
0a4f1422cd Further improve comments around recentRejects (Suhas Daftuar)
0e20cfedb7 Disconnect peers sending wtxidrelay message after VERACK (Suhas Daftuar)
cacd85209e test: Use wtxid relay generally in functional tests (Fabian Jahr)
8d8099e97a test: Add tests for wtxid tx relay in segwit test (Fabian Jahr)
9a5392fdf6 test: Update test framework p2p protocol version to 70016 (Fabian Jahr)
dd78d1d641 Rename AddInventoryKnown() to AddKnownTx() (Suhas Daftuar)
4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason (Suhas Daftuar)
97141ca442 Delay getdata requests from peers using txid-based relay (Suhas Daftuar)
46d78d47de Add p2p message "wtxidrelay" (Suhas Daftuar)
2d282e0cba ignore non-wtxidrelay compliant invs (Anthony Towns)
ac88e2eb61 Add support for tx-relay via wtxid (Suhas Daftuar)
8e68fc246d Add wtxids to recentRejects instead of txids (Suhas Daftuar)
144c385820 Add wtxids of confirmed transactions to bloom filter (Suhas Daftuar)
85c78d54af Add wtxid-index to orphan map (Suhas Daftuar)
08b39955ec Add a wtxid-index to mapRelay (Suhas Daftuar)
60f0acda71 Just pass a hash to AddInventoryKnown (Suhas Daftuar)
c7eb6b4f1f Add wtxid to mempool unbroadcast tracking (Amiti Uttarwar)
2b4b90aa8f Add a wtxid-index to the mempool (Suhas Daftuar)

Pull request description:

  Using txids (a transaction's hash, without witness) for transaction relay is problematic, post-segwit -- if a peer gives us a segwit transaction that fails policy checks, it could be because the txid associated with the transaction is definitely unacceptable to our node (regardless of the witness), or it could be that the transaction was malleated and with a different witness, the txid could be accepted to our mempool.

  We have a bloom filter of recently rejected transactions, whose purpose is to help us avoid redownloading and revalidating transactions that fail to be accepted, but because of this potential for witness malleability to interfere with relay of valid transactions, we do not use the filter for segwit transactions.  This issue is discussed at some length in #8279.  The effect of this is that whenever a segwit transaction that fails policy checks is relayed, a node would download that transaction from every peer announcing it, because it has no way presently to cache failure.  Historically this hasn't been a big problem, but if/when policy for accepting segwit transactions were to change (eg taproot, or any other change), we could expect older nodes talking to newer nodes to be wasting bandwidth because of this.

  As discussed in that issue, switching to wtxid-based relay solves this problem -- by using an identifier for a transaction that commits to all the data in our relay protocol, we can be certain if a transaction that a peer is announcing is one that we've already tried to process, or if it's something new.  This PR introduces support for wtxid-based relay with peers that support it (and remains backwards compatible with peers that use txids for relay, of course).

  Apart from code correctness, one issue to be aware of is that by downloading from old and new peers alike, we should expect there to be some bandwidth wasted, because sometimes we might download the same transaction via txid-relay as well as wtxid-relay.  The last commit in this PR implements a heuristic I want to analyze, which is to just delay relay from txid-relay peers by 2 seconds, if we have at least 1 wtxid-based peer.  I've just started running a couple nodes with this heuristic so I can measure how well it works, but I'm open to other ideas for minimizing that issue.  In the long run, I think this will be essentially a non-issue, so I don't think it's too big a concern, we just need to bite the bullet and deal with it during upgrade.

  Finally, this proposal would need a simple BIP describing the changes, which I haven't yet drafted.  However, review and testing of this code in the interim would be welcome.

  To do items:
  - [x] Write BIP explaining the spec here (1 new p2p message for negotiating wtxid-based relay, along with a new INV type)
  - [ ] Measure and evaluate a heuristic for minimizing how often a node downloads the same transaction twice, when connected to old and new nodes.

ACKs for top commit:
  naumenkogs:
    utACK 0a4f1422cd
  laanwj:
    utACK 0a4f1422cd

Tree-SHA512: d8eb8f0688cf0cbe9507bf738e143edab1f595551fdfeddc2b6734686ea26e7f156b6bfde38bad8bbbe8bec1857c7223e1687f8f018de7463dde8ecaa8f450df
2020-07-22 20:58:55 +02:00
Wladimir J. van der Laan
1397afc5ec Merge #19526: log: Avoid treating remote misbehvior as local system error
fa56eda58e log: Avoid treating remote misbehvior as local system error (MarcoFalke)
fa492895b5 refactor: Switch ValidationState mode to C++11 enum class (MarcoFalke)

Pull request description:

  When logging failures of `CheckBlockHeader` (high-hash), they are always logged as system error. This is problematic for several reasons:

  * Submitting a blockheader that fails `CheckBlockHeader` over RPC will result in a debug log line that starts with `ERROR`. Proper behaviour should be to log not anything and instead only return the failure reason to the RPC user. This pull does not fix this issue entirely, but is a good first step in the right direction.

  * A misbehaving peer that sends us an invalid block header that fails `CheckBlockHeader` will result in a debug log line that starts with `ERROR`. Proper behavior should be to log the remote peer misbehavior if logging for that category was enabled. This pull fixes this issue for `CheckBlockHeader` and other functions can be adjusted as well if needed in follow-ups. This should be a good first step in the right direction.

ACKs for top commit:
  practicalswift:
    re-ACK fa56eda58e

Tree-SHA512: 9793191f5cb57bdff7c93926e94877e8ca2ef89dcebcf9eb155899c733961839ec7c3f9b9f001dc082ada4234fe6e75f6df431301678d6822325840771166d77
2020-07-22 19:48:55 +02:00
Wladimir J. van der Laan
93decbc7a4 Merge #19370: Static asserts for consistency of fee defaults
1554b54d47 Static asserts for consistency of fee defaults. (Daniel Kraft)

Pull request description:

  This adds `static_assert`'s that ensure that the default values given for fee levels in the wallet (minimum fee and incremental feerate increase) are at least as high as the corresponding levels configured in the core node policy.  Since the core policy values are enforced by the network, it makes sense for the wallet to be conservative and above (or at least not below) this.

ACKs for top commit:
  laanwj:
    code review ACK 1554b54d47, these assumptions seem straightforward

Tree-SHA512: 50e5adf082f467062334377f82a3ee75bcfd436afc65bd0eb33c8d0549d6d90fd1f48c31f60cabe523eb59be9efa8ae0879e9e09cd51ca9c1bd466631ce03cf4
2020-07-22 19:25:07 +02:00
Andrew Chow
06e263a4e3 Call RecoverDatabaseFile directly from wallettool
When using the salvage command, call RecoverDatabaseFile directly
instead of SalvageWallet. Also removes SalvageWallet as it is no longer
needed.

SalvageWallet was doing an additional verify on the database which would
caause the salvage to sometimes fail. This is not needed.
2020-07-22 11:55:15 -04:00
Wladimir J. van der Laan
c7b4968552 Merge #19553: build: pass -fcommon when building genisoimage
2a701a1c42 build: pass -fcommon when building genisoimage (fanquake)

Pull request description:

  Starting with the 10.1 release, GCC [defaults to -fno-common](https://gcc.gnu.org/gcc-10/porting_to.html). This causes
  linking issues when building genisoimage:
  ```bash
  [ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
  [100%] Linking C executable genisoimage
  /usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  /usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
  ```

  Rather than patching genisoimage further, just pass -fcommon to preserve
  the legacy GCC behaviour.

  Noticed [while testing #19530](https://github.com/bitcoin/bitcoin/pull/19530#issuecomment-659802512).

ACKs for top commit:
  laanwj:
    ACK 2a701a1c42
  hebasto:
    ACK 2a701a1c42, tested on Fedora 32 (x86_64, GCC 10.1.1):

Tree-SHA512: 0e6dd1b4c7c3f5b6997616da27a7ea54f757c81677766ee6813c776356527787a32b959f834e699fee9798553b13bcb142469f442c5e7a04f810f2c06e3a9505
2020-07-22 13:57:22 +02:00
fanquake
ef3d4ce4c3 build: call AC_PATH_TOOL for dsymutil in macOS cross-compile
While testing #19530 I noticed that we couldn't call dsymutil after LTO:
```bash
../libtool: line 10643: x86_64-apple-darwin16-dsymutil: command not found
```

This updates configure to call `AC_PATH_TOOL` so that we end up with the
full path to dsymutil, similar to `otool` and `install_name_tool`, ie:
`/bitcoin/depends/x86_64-apple-darwin16/share/../native/bin/x86_64-apple-darwin16-otool`.
2020-07-22 18:22:56 +08:00
fanquake
2031aa92a3 Merge #19562: test: Fix fuzzer compilation on macOS
c8992e8959 test: Fix fuzzer compilation on macOS fixes #19557 (freenancial)

Pull request description:

  fixes #19557

  Before the fix:
  ```
  ➜  bitcoin git:(fix-fuzzer-macos) make
  Making all in src
    CXX      test/fuzz/addition_overflow-addition_overflow.o
  In file included from test/fuzz/addition_overflow.cpp:7:
  ./test/fuzz/util.h:335:13: error: no matching function for call to 'AdditionOverflow'
          if (AdditionOverflow((uint64_t)fuzzed_file->m_offset, random_bytes.size())) {
              ^~~~~~~~~~~~~~~~
  ./test/fuzz/util.h:201:16: note: candidate template ignored: deduced conflicting types for parameter 'T' ('unsigned long long' vs. 'unsigned long')
  NODISCARD bool AdditionOverflow(const T i, const T j) noexcept
                 ^
  ./test/fuzz/util.h:346:13: error: no matching function for call to 'AdditionOverflow'
          if (AdditionOverflow(fuzzed_file->m_offset, n)) {
              ^~~~~~~~~~~~~~~~
  ./test/fuzz/util.h:201:16: note: candidate template ignored: deduced conflicting types for parameter 'T' ('long long' vs. 'long')
  NODISCARD bool AdditionOverflow(const T i, const T j) noexcept
                 ^
  ```

  After the fix:
  ```
  ➜  bitcoin git:(fix-fuzzer-macos) ./configure --enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=/usr/local/opt/llvm/bin/clang CXX=/usr/local/opt/llvm/bin/clang++ --disable-asm && make clean && make -j5
  ...
  ...
    CXXLD    test/fuzz/uint256_deserialize
  Making all in doc/man
  make[1]: Nothing to be done for `all'.
  make[1]: Nothing to be done for `all-am'.
  ```

ACKs for top commit:
  fanquake:
    ACK c8992e8959 - tested that compiling works on macOS.
  MarcoFalke:
    review ACK c8992e8959

Tree-SHA512: 965cdc61b30db0e2209c91b29f0d42de927a9a5b85e1e70f22d1452e0955f876726c7a8c1d1a5f448f12bf24eec3000802071cd4ae28d8605343fd43d174ca84
2020-07-22 18:03:41 +08:00
fanquake
597d2f905e Merge #19548: fuzz: add missing overrides to signature_checker
c0f09c2c9d fuzz: add missing overrides to signature_checker (Jon Atack)

Pull request description:

  These functions in `fuzz/signature_checker.cpp` override virtual member functions and should be marked `override` instead of `virtual`, which is for introducing a new virtual function. The overridden virtual functions are in `script/interpreter.h:151/156/161`.

  Also, per MarcoFalke suggestion, add missing parentheses in `fuzz/scriptnum_ops.cpp` and remove useless `unsigned int >= 0` conditional in `fuzz/script.cpp`.

  These changes fix 5 compile warnings in gcc 10 and 3 in clang 11/12.

ACKs for top commit:
  vasild:
    ACK c0f09c2
  MarcoFalke:
    review ACK c0f09c2c9d

Tree-SHA512: 76ce73ec577c1f23cf8646c31d44dcd6c6303732c47187d041a8921d0d24a50163989a375352ebc221abf2ac337bc0902149be46b6f9eebc071d2f364c407f71
2020-07-22 17:32:36 +08:00
MarcoFalke
c7007babb7 Merge #18907: walletdb: Don't remove database transaction logs and instead error
d0ea9bab28 walletdb: Don't remove database transaction logs and instead error (Andrew Chow)

Pull request description:

  Instead of removing the database transaction logs and retrying the
  wallet loading, just return an error message to the user. Additionally,
  speciically for DB_RUNRECOVERY, notify the user that this could be due
  to different BDB versions.

  Kind of implements the suggestion from https://github.com/bitcoin/bitcoin/pull/18870#discussion_r421647964

ACKs for top commit:
  Sjors:
    re-utACK d0ea9bab28
  ryanofsky:
    Code review ACK d0ea9bab28. Only changes since last review are rebase and expanding error and commit messages.

Tree-SHA512: f6e67dc70f58188742a5c8af7cdc63a2b58779aa0d26ae7f1e75805a239f1a342433860e5a238d6577fae5ab04b9d15e7f11c55b867065dfd13781a6a62e4958
2020-07-22 08:58:55 +02:00
Jon Atack
c0f09c2c9d fuzz: add missing overrides to signature_checker
and also

- add missing parentheses in fuzz/scriptnum_ops.cpp

- remove useless unsigned int conditional in fuzz/script.cpp

These changes fix 5 compile warnings in gcc 10.
2020-07-22 05:27:13 +02:00
freenancial
c8992e8959 test: Fix fuzzer compilation on macOS
fixes #19557
2020-07-21 15:23:49 -07:00
Hennadii Stepanov
4ec49f8d1e qt: Leverage the default "Create new receiving address" button
When Enter or Return is pressed the default button will be always
clicked. All buttons can always be clicked from the keyboard by pressing
spacebar when the button has focus.
2020-07-21 21:29:09 +03:00
Hennadii Stepanov
4227a8e1f3 qt: Make "Create new receiving address" default unconditionally
This change makes this button visually accented, and gives to users a
hint about the primary action.
This commit does not change behavior.
2020-07-21 21:24:28 +03:00
MarcoFalke
2c0c3f8e8c Merge #19217: p2p: disambiguate block-relay-only variable names from blocksonly variables
ec4c6a17e8 scripted-diff: replace MAX_BLOCKS_ONLY_CONNECTIONS with MAX_BLOCK_RELAY_ONLY_CONNECTIONS (glowang)

Pull request description:

  We have two different concepts that have similar names: `-blocksonly` and `block-relay-only`, and the similarity of names could lead to confusion. `-blocksonly` disables all local receiving & relaying of transactions (with a few exceptions), while `block-relay-only`means that bitcoind will make 2 additional outbound connections that are only used for block relay.

  In net.h and init.cpp, `MAX_BLOCKS_ONLY_CONNECTIONS` is used to represent the maximum number of `block-relay-only` outbound peers, which is 2. But this name sounds ambiguous, and I proposed a better name,  `MAX_BLOCK_RELAY_ONLY_CONNECTION`.

ACKs for top commit:
  jnewbery:
    ACK ec4c6a17e8

Tree-SHA512: cfa592a7ff936f14d10cfc1e926a51b82bc0feaf104885a41ca8111b906cb3d1ec5536bab143a3cfca70aa49e9575c6995941eb6d3d7f4018d4535712342f155
2020-07-21 16:04:33 +02:00
MarcoFalke
edfeaf6836 Merge #19552: test: fix intermittent failure in p2p_ibd_txrelay
12410b1feb test: fix intermittent p2p_ibd_txrelay race, add test_framework.py#wait_until (Jon Atack)

Pull request description:

  To fix these intermittent failures in Travis CI.
  ```
  162/163 - p2p_ibd_txrelay.py failed, Duration: 2 s

  stdout:
  2020-07-19T05:44:17.213000Z TestFramework (INFO):
      Check that nodes set minfilter to MAX_MONEY while still in IBD
  2020-07-19T05:44:17.216000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/test_framework/test_framework.py", line 117, in main
      self.run_test()
    File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/p2p_ibd_txrelay.py", line 30, in run_test
      assert_equal(conn_info['minfeefilter'], MAX_FEE_FILTER)
    File "/Users/travis/build/bitcoin/bitcoin/ci/scratch/build/bitcoin-x86_64-apple-darwin16/test/functional/test_framework/util.py", line 49, in assert_equal
      raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))

  AssertionError: not(0E-8 == 0.09170997)
  2020-07-19T05:44:17.293000Z TestFramework (INFO): Stopping nodes
  ```

  At Marco's suggestion, cherry-picked part of #19134 to nicely simplify using `wait_until`.

ACKs for top commit:
  vasild:
    ACK 12410b1fe

Tree-SHA512: 615f509883682fd693e578b259cba35a9fa0bc519f1394e88c857e8b0650bfec5397bfa856cfa9e6d5ef81d0ee6ad02e4ad2b0eb0bd530b4c281cbe3e663790b
2020-07-21 16:01:59 +02:00
fanquake
910a8d9c6f Merge #19560: contrib: Clean up previous_releases.py
facdf530c7 contrib: Clean up previous_releases.py (MarcoFalke)

Pull request description:

ACKs for top commit:
  fjahr:
    tACK facdf530c7
  Sjors:
    tACK facdf53
  hebasto:
    ACK facdf530c7, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: c3543320572267035aa342dd170128bbdeb83ca4e2e36a8e46596dd76c8ff1b26ed6759a8073884228b133c45b06ec48889cd0ec83a13bef276b48073d8248e4
2020-07-21 20:32:20 +08:00
MarcoFalke
65a54d684f Merge #18984: Remove unnecessary input blockfile SetPos
5fa067a27d Remove unnecessary blockfile SetPos (Tom Harding)

Pull request description:

  Nothing could have changed the position since we retrieved it a few statements earlier. This dates from commit 16d5194165.

ACKs for top commit:
  LarryRuane:
    ACK 5fa067a27d

Tree-SHA512: 459cc7226e186c231ffb67f0613f550e8eb940f1b8933c3bc4a4e8dd519c8d5d45884e8cfd9347039dab90a093644bbbb31be063baed1c6fc7984b6cb4f17c9f
2020-07-21 11:28:35 +02:00
MarcoFalke
facdf530c7 contrib: Clean up previous_releases.py
* Replace curl single char options with their verbose counterpart
* Stricter check for tarballHash
2020-07-21 11:03:35 +02:00
MarcoFalke
ea595d39f7 Merge #19205: script: previous_release.sh rewritten in python
9c34aff393 Remove previous_release.sh (Brian Liotti)
e1e5960e10 script: Add previous_release.py (Brian Liotti)

Pull request description:

  Closes #18132

  Added functionality:
  1) checks file hash before untarring when using the binary download option

ACKs for top commit:
  fjahr:
    re-ACK 9c34aff393
  Sjors:
    tACK 9c34aff393

Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
2020-07-21 10:11:39 +02:00
MarcoFalke
b763ae02a6 Merge #16878: Fix non-deterministic coverage of test DoS_mapOrphans
4455949d6f Make test DoS_mapOrphans deterministic (David Reikher)

Pull request description:

  This pull request proposes a solution to make the test `DoS_mapOrphans` in denialofservice_tests.cpp have deterministic coverage.

  The `RandomOrphan` function in denialofservice_tests.cpp and the implicitly called function `ecdsa_signature_parse_der_lax` in pubkey.cpp were causing the non-deterministic test coverage.

  In the former, if a random orphan was selected the index of which is bigger than the max. orphan index in `mapOrphanTransactions`, the last orphan was returned from `RandomOrphan`. If the random number generated was never large enough, this condition would not be fulfilled and the corresponding branch wouldn't run. The proposed solution is to force one of the 50 dependant orphans to depend on the last orphan in `mapOrphanTransactions` using the newly introduced function `OrphanByIndex` (and passing it a large uint256), forcing this branch to run at least once.

  In the latter, if values for ECDSA `R` or `S` (or both) had no leading zeros, some code would not be executed. The solution was to find a constant signature that would be comprised of `R` and `S` values with leading zeros and calling `CPubKey::Verify` at the end of the test with this signature forcing this code to always run at least once at the end even if it hadn't throughout the test.

  To test that the coverage is (at least highly likely) deterministic, I ran

  `contrib/devtools/test_deterministic_coverage.sh denialofservice_tests/DoS_mapOrphans 1000`

  and the result was deterministic coverage across 1000 runs.

  Also - removed denialofservice_tests test entry from the list of non-deterministic tests in the coverage script.

ACKs for top commit:
  MarcoFalke:
    ACK 4455949d6f

Tree-SHA512: 987eb1f94b80d5bec4d4944e91ef43b9b8603055750362d4b4665b7f011be27045808aa9f4c6ccf8ae009b61405f9a1b8671d65a843c3328e5b8acce1f1c00a6
2020-07-21 09:38:39 +02:00
David Reikher
4455949d6f Make test DoS_mapOrphans deterministic
The RandomOrphan function and the function ecdsa_signature_parse_der_lax
in pubkey.cpp were causing non-deterministic test coverage.

Force seed in the beginning of the test to make it deterministic.
The seed is selected carefully so that all branches of the function
ecdsa_signature_parse_der_lax are executed. Prior to this fix, the test
was exhibiting non-deterministic coverage since none of the ECDSA
signatures that were generated during the test had leading zeroes in
either R, S, or both, resulting in some branches of said function not
being executed. The seed ensures that both conditions are hit.

Removed denialofservice_tests test entry from the list of non-deterministic
tests in the coverage script.
2020-07-21 09:18:57 +03:00
fanquake
caf1766daf Merge #19547: doc: Update macOS cross compilation dependencies for Focal
6be3eebd74 doc: Update macOS cross compilation dependencies for Focal (Hennadii Stepanov)

Pull request description:

  The [`libtinfo5`](https://packages.ubuntu.com/focal/libtinfo5) package is required on Ubuntu Focal for macOS cross compilation.

  Fixes #19546.

ACKs for top commit:
  fanquake:
    ACK 6be3eebd74 I have been [installing the libtinfo5 package](https://github.com/fanquake/core-review/blob/master/docker/debian9.dockerfile#L18) into my cross-compiling images, so mostly forgot about this.

Tree-SHA512: f40ab1efae549276e520f9cc3200438e70297c3081a3f9acc3075ec11d494fd6176ed3f1d4889b39eb2a5706eac835844a75b79335bb7f3c21c985e0779a7562
2020-07-21 14:02:42 +08:00
fanquake
f5000ae66c Merge #19555: rpc: deduplicate WriteHDKeypath() used in decodepsbt
55057ffc51 rpc: deduplicate WriteHDKeypath() used in decodepsbt (Sebastian Falbesoner)

Pull request description:

  The functionality is already provided in the BIP32 utility library `util/bip32.h` with the exact same name and function signature.

ACKs for top commit:
  achow101:
    ACK 55057ffc51
  instagibbs:
    utACK 55057ffc51
  jonatack:
    ACK 55057ffc51

Tree-SHA512: 074c1a71ffb32908926bf07f0c5428a46309f6e0d21e7c20b1008197c820b97776a441736d0b6fd8ab0c0852522a0b5a5ddb26a1e4a1100ca02aabc65a07a018
2020-07-21 09:11:48 +08:00
Vasil Dimov
100c64a95b net: document enum Network 2020-07-20 16:03:39 +02:00
MarcoFalke
fa2ae0ac8d span: Add Span::empty() and use it in script/descriptor 2020-07-19 20:51:06 +02:00
Sebastian Falbesoner
55057ffc51 rpc: deduplicate WriteHDKeypath() used in decodepsbt
The functionality is already provided in the BIP32 utility library util/bip32.h
with the exact same name and function signature.
2020-07-19 17:25:07 +02:00
Jon Atack
12410b1feb test: fix intermittent p2p_ibd_txrelay race, add test_framework.py#wait_until 2020-07-19 13:37:54 +02:00
fanquake
476436b2de Merge #19551: fuzz: fix unused variable compiler warning in addrdb
1cdc2a642b fuzz: fix unused variable addrdb compiler warning (Jon Atack)

Pull request description:

  Fixes #19549

ACKs for top commit:
  hebasto:
    ACK 1cdc2a642b, tested on Linux Mint 20 (x86_64, clang 10.0).
  MarcoFalke:
    review re-ACK 1cdc2a642b
  practicalswift:
    ACK 1cdc2a642b

Tree-SHA512: 5ce5ce06f0a3121911ae2ac4ed3953e5e56e6a7bd50a60ee6df84873eec75d2997488d2435bd55294e890bbf6fc368dd4302cf2a6ab6073ec74c7711eb0340e1
2020-07-19 19:25:44 +08:00
fanquake
2a701a1c42 build: pass -fcommon when building genisoimage
Starting with the 10.1 release, GCC defaults to -fno-common. This causes
linking issues when building genisoimage:
```bash
[ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
[100%] Linking C executable genisoimage
/usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
```

Rather than patching genisoimage further, pass -fcommon to preserve the
legacy GCC behaviour.
2020-07-19 14:41:57 +08:00
Jon Atack
1cdc2a642b fuzz: fix unused variable addrdb compiler warning 2020-07-19 08:31:34 +02:00
Suhas Daftuar
0a4f1422cd Further improve comments around recentRejects 2020-07-19 02:10:42 -04:00
Suhas Daftuar
0e20cfedb7 Disconnect peers sending wtxidrelay message after VERACK 2020-07-19 02:10:42 -04:00
Fabian Jahr
cacd85209e test: Use wtxid relay generally in functional tests 2020-07-19 02:10:42 -04:00
Fabian Jahr
8d8099e97a test: Add tests for wtxid tx relay in segwit test
Also cleans up some doublicate lines in the rest of the test.

co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-07-19 02:10:42 -04:00
Fabian Jahr
9a5392fdf6 test: Update test framework p2p protocol version to 70016
This new p2p protocol version allows to use WTXIDs for tx relay.
2020-07-19 02:10:42 -04:00
Suhas Daftuar
dd78d1d641 Rename AddInventoryKnown() to AddKnownTx() 2020-07-19 02:10:42 -04:00
Suhas Daftuar
4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason
Previously, TX_WITNESS_MUTATED could be returned during transaction validation
for either transactions that had a witness that was non-standard, or for
transactions that had no witness but were invalid due to segwit validation
rules.

However, for txid/wtxid-relay considerations, net_processing distinguishes the
witness stripped case separately, because it affects whether a wtxid should be
able to be added to the reject filter. It is safe to add the wtxid of a
witness-mutated transaction to the filter (as that wtxid shouldn't collide with
the txid, and hence it wouldn't interfere with transaction relay from
txid-relay peers), but it is not safe to add the wtxid (== txid) of a
witness-stripped transaction to the filter, because that would interfere with
relay of another transaction with the same txid (but different wtxid) when
relaying from txid-relay peers.

Also updates the comment explaining this logic, and explaining that we can get
rid of this complexity once there's a sufficient deployment of wtxid-relaying
peers on the network.
2020-07-19 02:10:42 -04:00
Suhas Daftuar
97141ca442 Delay getdata requests from peers using txid-based relay
Using both txid and wtxid-based relay with peers means that we could sometimes
download the same transaction twice, if announced via two different hashes from
different peers.

Use a heuristic of delaying txid-peer-getdata requests by 2 seconds, if we have
at least one wtxid-based peer.
2020-07-19 02:10:42 -04:00
Suhas Daftuar
46d78d47de Add p2p message "wtxidrelay"
When sent to and received from a given peer, enables using wtxid's for
announcing and fetching transactions with that peer.
2020-07-19 02:10:41 -04:00
Anthony Towns
2d282e0cba ignore non-wtxidrelay compliant invs 2020-07-19 02:05:42 -04:00
Suhas Daftuar
ac88e2eb61 Add support for tx-relay via wtxid
This adds a field to CNodeState that tracks whether to relay transactions with
that peer via wtxid, instead of txid. As of this commit the field will always
be false, but in a later commit we will add a way to negotiate turning this on
via p2p messages exchanged with the peer.
2020-07-19 02:05:29 -04:00
Suhas Daftuar
8e68fc246d Add wtxids to recentRejects instead of txids
Previously, we only added txids to recentRejects if we were sure that the
transaction couldn't have had the wrong witness (either because the witness was
malleated or stripped).

In preparation for wtxid-based relay, we can observe that txid == wtxid for
transactions that have no witness, and add the wtxid of rejected transactions,
provided the transaction wasn't a witness-stripped one. This means that we now
add more data to the filter (as prior to this commit, any transaction with a
witness that failed to be accepted was being skipped for inclusion in the
filter) but witness malleation should still not interfere with relay of a valid
segwit transaction, because the txid of a segwit transaction would not be added
to the filter after failing validation.

In the future, having wtxids in the recent rejects filter will allow us to
skip downloading the same wtxid multiple times, once our peers use wtxids for
transaction relay.
2020-07-18 19:00:02 -04:00
Suhas Daftuar
144c385820 Add wtxids of confirmed transactions to bloom filter
This is in preparation for wtxid-based invs (we need to be able to tell whether
we AlreadyHave() a transaction based on either txid or wtxid).

This also double the size of the bloom filter, which is overkill, but still
uses a manageable amount of memory.
2020-07-18 19:00:02 -04:00
Suhas Daftuar
85c78d54af Add wtxid-index to orphan map 2020-07-18 19:00:02 -04:00
Suhas Daftuar
08b39955ec Add a wtxid-index to mapRelay 2020-07-18 19:00:02 -04:00
Suhas Daftuar
60f0acda71 Just pass a hash to AddInventoryKnown
Since it's only used for transactions, there's no need to pass in an inv type.
2020-07-18 19:00:01 -04:00
Amiti Uttarwar
c7eb6b4f1f Add wtxid to mempool unbroadcast tracking 2020-07-18 19:00:01 -04:00
Suhas Daftuar
2b4b90aa8f Add a wtxid-index to the mempool 2020-07-18 19:00:01 -04:00
Hennadii Stepanov
6be3eebd74 doc: Update macOS cross compilation dependencies for Focal 2020-07-18 18:05:40 +03:00
MarcoFalke
090d877160 Merge #19143: tests: Add fuzzing harnesses for CAutoFile, CBufferedFile, LoadExternalBlockFile and other FILE* consumers
ad6c34881d tests: Add fuzzing harness for CBlockPolicyEstimator::{Read,Write} (policy/fees.h) (practicalswift)
614e0807a8 tests: Add fuzzing harness for CBufferedFile::{SetPos,GetPos,GetType,GetVersion} (stream.h) (practicalswift)
7bcc71e5f8 tests: Add fuzzing harness for LoadExternalBlockFile(...) (validation.h) (practicalswift)
9823376030 tests: Add fuzzing harness for CBufferedFile (streams.h) (practicalswift)
f3aa659be6 tests: Add fuzzing harness for CAutoFile (streams.h) (practicalswift)
e507c0799d tests: Add serialization/deserialization fuzzing helpers WriteToStream(…)/ReadFromStream(…) (practicalswift)
e48094a506 tests: Add FuzzedAutoFileProvider which provides a CAutoFile interface to FuzzedDataProvider (practicalswift)
9dbcd6854c tests: Add FuzzedFileProvider which provides a FILE* interface to FuzzedDataProvider using fopencookie (practicalswift)

Pull request description:

  Add fuzzing harnesses for `CAutoFile`, `CBufferedFile`, `LoadExternalBlockFile` and other `FILE*` consumers:
  * Add `FuzzedFileProvider` which provides a `FILE*` interface to `FuzzedDataProvider` using `fopencookie`
  * Add `FuzzedAutoFileProvider` which provides a `CAutoFile` interface to `FuzzedDataProvider`
  * Add serialization/deserialization fuzzing helpers `WriteToStream(…)`/`ReadFromStream(…)`
  * Add fuzzing harness for `CAutoFile` (`streams.h`)
  * Add fuzzing harness for `CBufferedFile` (`streams.h`)
  * Add fuzzing harness for `LoadExternalBlockFile(...)` (`validation.h`)
  * Add fuzzing harness for `CBlockPolicyEstimator::Read` and `CBlockPolicyEstimator::Write` (`policy/fees.h`)

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  Crypt-iQ:
    Tested ACK ad6c348

Tree-SHA512: a38e142608218496796a527d7e59b74e30279a2815450408b7c27a76ed600cebc6b88491e831665a0639671e2d212453fcdca558500bbadbeb32b267751f8f72
2020-07-18 09:24:58 +02:00
MarcoFalke
fd59670642 Merge #19538: ci: Add tsan suppression for race in DatabaseBatch
0cdf2a77dd ci: add tsan debug symbols option (Russell Yanofsky)
9a2f12680b ci: Add tsan suppression for race in DatabaseBatch (Hennadii Stepanov)

Pull request description:

  Since #19325 was merged, the corresponding change in TSan suppression file gets required.

  This PR is:
  - an analogous to #19226 and #19450, and
  - a temporary workaround for CI fail like https://cirrus-ci.com/task/5741795508224000?command=ci#L4993

ACKs for top commit:
  MarcoFalke:
    ACK 0cdf2a77dd

Tree-SHA512: 7832f143887c8a0df99dea03e00694621710378fbe923e3592185fcd3658546a590693b513abffc5ab96e9ef76c9c4bff3330eeee69a0c5dbe7574f34c417220
2020-07-17 17:02:41 +02:00
Russell Yanofsky
0cdf2a77dd ci: add tsan debug symbols option
https://github.com/bitcoin/bitcoin/issues/19049#issuecomment-639255156
2020-07-17 12:14:41 +03:00
fanquake
c04485850e Merge #19536: qt, build: Fix QFileDialog for static builds
6457361e90 qt: Fix QFileDialog for static builds (Hennadii Stepanov)

Pull request description:

  This change partially reverts 248e22bbc0 (#16386) and makes `QFileDialog`s work again for static builds.

  Fixes https://github.com/bitcoin-core/gui/issues/32.

ACKs for top commit:
  fanquake:
    ACK 6457361e90. Although it would be good to know exactly _why_ this fixes the issue. At this stage I also don't think this should be a blocker for 0.20.1.
  theuni:
    ACK 6457361e90

Tree-SHA512: 8ad27e0bcae6debd02f73b7c374743e37d4edd806922b103a2fe494cf2d9930fe9ef3107b5a6c61f3c466cf7462de2641171880398954e7f2c4f417f5bb820d7
2020-07-17 14:25:07 +08:00
MarcoFalke
19aaf7945e Merge #19423: test: add functional test for txrelay during and after IBD
cb31ee01b4 [test] feefilter during and after IBD (gzhao408)

Pull request description:

  This is a followup to #19204 which uses `minfeefilter=MAX_MONEY` to effectively shut off txrelay, thereby reducing inv traffic, when nodes are in IBD. It was [missing](https://github.com/bitcoin/bitcoin/pull/19204#issuecomment-644040070) a functional test.

ACKs for top commit:
  jnewbery:
    utACK cb31ee01b4

Tree-SHA512: a9effc8193fa95fb42a2f9c66b258cc7b0941fc04c1ce3a6092f4426c9bfc7e72f702aca559b3e30e90652497f411f22fae3cf5cdb6cfd6ef6d37fed712cda67
2020-07-17 07:51:24 +02:00
Cory Fields
5962522fbc depends: bump native_cctools for fixed lto with external clang
https://github.com/tpoechtrager/cctools-port/pull/85 was merged upstream, which
fixes lto detection for external clang with some Linux Distro's including
Ubuntu.
2020-07-16 20:04:07 +00:00
Hennadii Stepanov
9a2f12680b ci: Add tsan suppression for race in DatabaseBatch 2020-07-16 22:41:10 +03:00
Wladimir J. van der Laan
c57dc566b0 Merge #16525: Dump transaction version as an unsigned integer in RPC/TxToUniv
e80259f197 Additionally treat Tx.nVersion as unsigned in joinpsbts (Matt Corallo)
970de70bdd Dump transaction version as an unsigned integer in RPC/TxToUniv (Matt Corallo)

Pull request description:

  Consensus-wise we already treat it as an unsigned integer (the
  only rules around it are in CSV/locktime handling), but changing
  the underlying data type means touching consensus code for a
  simple cleanup change, which isn't really worth it.

  See-also, https://github.com/rust-bitcoin/rust-bitcoin/pull/299

ACKs for top commit:
  sipa:
    ACK e80259f197
  practicalswift:
    ACK e80259f197
  ajtowns:
    ACK e80259f197 code review -- checked all other uses of tx.nVersion treat it as unsigned (except for policy.cpp:IsStandard anyway), so looks good.
  naumenkogs:
    ACK e80259f

Tree-SHA512: 6760a2c77e24e9e1f79a336ca925f9bbca3a827ce02003c71d7f214b82ed3dea13fa7d9f87df9b9445cd58dff8b44a15571d821c876f22f8e5a372a014c9976b
2020-07-16 21:38:09 +02:00
Cory Fields
00d1ba7aaa depends: enable lto support for Apple's ld64
Note that this does not _enable_ lto by default in any way, only hooks up the
machinery for -flto to work correctly.

enable-lto-support is explicitly used for pinned-clang because we know it
works. It is neither enabled nor disabled in the external clang case so that
it can be auto-detected.
2020-07-16 19:04:57 +00:00
Hennadii Stepanov
6457361e90 qt: Fix QFileDialog for static builds
This change partially reverts 248e22bbc0.
2020-07-16 20:29:49 +03:00
Fabian Jahr
36ec9801a4 test: Add chacha20 test vectors in muhash 2020-07-16 18:10:48 +02:00
Fabian Jahr
0e2b400fea test: Add basic Python/C++ Muhash implementation parity unit test 2020-07-16 18:10:48 +02:00
Pieter Wuille
b85543cb73 test: Add Python MuHash3072 implementation to test framework 2020-07-16 18:10:48 +02:00
Fabian Jahr
ab30cece0e test: Move modinv to util and add unit test 2020-07-16 18:10:43 +02:00
fanquake
1148f3d630 Merge #17919: depends: Allow building with system clang
de4fedb6c3 depends: Add documentation for FORCE_USE_SYSTEM_CLANG make flag (Carl Dong)
fe98999dcf depends: Reformat make options as definition list (Carl Dong)
60c55b1b9b depends: Add justifications for macOS clang flags (Carl Dong)
6b8e497eea depends: specify libc++ header location for darwin (Cory Fields)
156b604203 depends: force a new host id string if FORCE_USE_SYSTEM_CLANG is in use (Cory Fields)
c9c572a367 depends: Allow building with system clang (Carl Dong)
e6e5c8d6ca depends: Decouple toolchain + binutils (Carl Dong)

Pull request description:

  This replaces: #17099

  -----

  This patchset allows us to force depends to use system clang.

  Previously, #17099 removes our dependency on a specific clang we download from llvm.org, but theuni pointed out that since OSX builds are only ever built with a version of clang that is chosen and "blessed" by Apple, it is more likely that the user will encounter problems if they use their system clang. This patchset forces the user to set `FORCE_USE_SYSTEM_CLANG=1` in order to use their system clang (when they know what they're doing)

ACKs for top commit:
  theuni:
    ACK de4fedb6c3.

Tree-SHA512: 8774121e035f90c27030bcce06e1b79f7729b5e17802c718e49652ab06e19780632db974df47423c1d1b04f1ab1b7a763554fb922fec05d1cd6445b26578be1d
2020-07-16 22:41:09 +08:00
MarcoFalke
0f6900e780 Merge #19533: [tests] Remove unnecessary cs_mains in denialofservice_tests
f58c4b538e [tests] Remove unnecessary cs_mains in denialofservice_tests (Matt Corallo)

Pull request description:

  9fdf05d70c resolved some lock
  inversion warnings in denialofservice_tests, but left in a number
  of cs_main locks that are unnecessary (introducing lock inversion
  warnings in future changes).

ACKs for top commit:
  promag:
    ACK f58c4b538e.
  jonatack:
    ACK f58c4b538e verified the test locks correspond to the locks in net/net_processing, and the debug build is clean/unit tests pass.

Tree-SHA512: de2d9b2a8f08081b2ce31e18585e4677b167a11752b797d790c281575d7dfef3587f8be4fc7f8f16771141b6ff0b0145c7488cf30e79256b0043947c67a6182c
2020-07-16 13:03:49 +02:00
MarcoFalke
6a53c3e390 Merge bitcoin-core/gui#14: scripted-diff: rename movie folder to animation
80968cff68 scripted-diff: rename movie folder to animation (Peter Bushnell)

Pull request description:

  Rename the movies directory and RES_MOVIES make variable to animation and RES_ANIMATION respectively. Movies is a bit of an unexpected term to be found.

ACKs for top commit:
  MarcoFalke:
    ACK 80968cff68
  hebasto:
    ACK 80968cff68, tested on Linux Mint 20 (Qt 5.12.8).

Tree-SHA512: 6bd31ce36e821f6a1bef8a7972086a2387d6258c48fc9df12d3ffdae07d0237036afbc2dec673384b78d9567b91d6e12eafa59fa2305aa79153dfd9b7c3a8655
2020-07-16 09:55:19 +02:00
MarcoFalke
9a714c51dc Merge bitcoin-core/gui#34: Show permissions instead of whitelisted
784ef8be41 gui: Show permissions instead of whitelisted (Wladimir J. van der Laan)

Pull request description:

  Show detailed permissions instead of legacy "whitelisted" flag in the peer list details.
  These are formatted with `&` in between just like services flags. It reuses the "N/A" translation message if there are no special permissions.
  This removes the one-but-last use of `legacyWhitelisted`.

Top commit has no ACKs.

Tree-SHA512: 11982da4b9d408c74bc56bb3c540c0eb22506be6353aa4d4d6c64461d140f0587be194e2daad1612fddaa2618025a856b33928ad89041558f418f721f6abd407
2020-07-16 08:11:36 +02:00
MarcoFalke
affed844ba Merge #19174: refactor: replace CConnman pointers by references in net_processing.cpp
0c8461a88e refactor: replace CConnman pointers by references in net_processing.cpp (Sebastian Falbesoner)

Pull request description:

  This is a follow-up to the recently merged PR https://github.com/bitcoin/bitcoin/pull/19053, replacing ~~two more types of~~ one more type of pointer (CConnman) by references to increase the code quality -- pointers should either check for `nullptr` or be replaced by references, and the latter strategy seems to be more reasonable.

  Again, to keep the review burden managable, the changes are kept simple,
  * only tackling `CConnman*` ~~and `BanMan*`~~ pointers
  * only within the net_processing module, i.e. no changes that would need adaption in other modules
  * keeping the names of the variables as they are

ACKs for top commit:
  jnewbery:
    utACK 0c8461a88e
  MarcoFalke:
    ACK 0c8461a88e 🕧

Tree-SHA512: 79dc05144bcfb5e0bbc62180285aadcc6199f044fa3016c0f54f7b7f45037415260970037bd63b18fafefb8aef448549dae14b780bafb540fa2373f493a17f71
2020-07-16 08:07:25 +02:00
gzhao408
cb31ee01b4 [test] feefilter during and after IBD
Co-authored-by: Jon Atack <jon@atack.com>
2020-07-15 16:42:06 -07:00
Wladimir J. van der Laan
784ef8be41 gui: Show permissions instead of whitelisted
Show detailed permissions instead of legacy "whitelisted" flag.
These are formatted with `&` in between just like services flags.
It reuses the "N/A" translation message if not.
This removes the one-but-last use of `legacyWhitelisted`.
2020-07-15 22:54:17 +02:00
Wladimir J. van der Laan
3864219d40 Merge #19360: net: improve encapsulation of CNetAddr
bc74a40a56 net: improve encapsulation of CNetAddr (Vasil Dimov)

Pull request description:

  Do not access `CNetAddr::ip` directly from `CService` methods.

  This improvement will help later when we change the type of
  `CNetAddr::ip` (in the BIP155 implementation).

  (chopped off from https://github.com/bitcoin/bitcoin/pull/19031 to ease review)

ACKs for top commit:
  dongcarl:
    ACK bc74a40a56
  naumenkogs:
    ACK bc74a40
  fjahr:
    Code review ACK bc74a40
  laanwj:
    code review ACK bc74a40a56
  jonatack:
    ACK bc74a40a56
  jnewbery:
    ACK bc74a40a5

Tree-SHA512: 29a203905538e8311e3249b78565abe69ce36dc4ec239bec85c726c30e1a7b55b0aaf5c6659b676935008e068cfa53d716f7a598469064108daf130f94329a5d
2020-07-15 21:17:18 +02:00
Wladimir J. van der Laan
31bdd86631 Merge #19353: Fix mistakenly swapped "previous" and "current" lock orders
0ecff9dd34 Improve "detected inconsistent lock order" error message (Hennadii Stepanov)
bbe9cf4fe4 test: Improve "potential deadlock detected" exception message (Hennadii Stepanov)
35599344c8 Fix mistakenly swapped "previous" and "current" lock orders (Hennadii Stepanov)

Pull request description:

  In master (8ef15e8a86) the "previous" and "current" lock orders are mistakenly swapped.

  This PR:
  - fixes printed lock orders
  - improves the `sync_tests` unit test
  - makes the "detected inconsistent lock order" error message pointing to the lock location rather `tfm::format()` location.

  Debugger output example with this PR (with modified code, of course):
  ```
  2020-06-22T15:46:56Z [msghand] POTENTIAL DEADLOCK DETECTED
  2020-06-22T15:46:56Z [msghand] Previous lock order was:
  2020-06-22T15:46:56Z [msghand]  (2) 'cs_main' in net_processing.cpp:2545 (in thread 'msghand')
  2020-06-22T15:46:56Z [msghand]  (1) 'g_cs_orphans' in net_processing.cpp:1400 (in thread 'msghand')
  2020-06-22T15:46:56Z [msghand] Current lock order is:
  2020-06-22T15:46:56Z [msghand]  (1) 'g_cs_orphans' in net_processing.cpp:2816 (in thread 'msghand')
  2020-06-22T15:46:56Z [msghand]  (2) 'cs_main' in net_processing.cpp:2816 (in thread 'msghand')
  Assertion failed: detected inconsistent lock order for 'cs_main' in net_processing.cpp:2816 (in thread 'msghand'), details in debug log.
  Process 131393 stopped
  * thread #15, name = 'b-msghand', stop reason = signal SIGABRT
      frame #0: 0x00007ffff775c18b libc.so.6`__GI_raise(sig=2) at raise.c:51:1
  (lldb) bt
  * thread #15, name = 'b-msghand', stop reason = signal SIGABRT
    * frame #0: 0x00007ffff775c18b libc.so.6`__GI_raise(sig=2) at raise.c:51:1
      frame #1: 0x00007ffff773b859 libc.so.6`__GI_abort at abort.c:79:7
      frame #2: 0x0000555555e5b196 bitcoind`(anonymous namespace)::potential_deadlock_detected(mismatch=0x00007fff99ff6f30, s1=size=2, s2=size=2, lock_location=0x00007fff99ff7010) at sync.cpp:134:9
      frame #3: 0x0000555555e5a1b1 bitcoind`(anonymous namespace)::push_lock(c=0x0000555556379220, locklocation=0x00007fff99ff7010) at sync.cpp:158:13
      frame #4: 0x0000555555e59e8a bitcoind`EnterCritical(pszName="cs_main", pszFile="net_processing.cpp", nLine=2816, cs=0x0000555556379220, fTry=false) at sync.cpp:177:5
      frame #5: 0x00005555555b0500 bitcoind`UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::Enter(this=0x00007fff99ff8c20, pszName="cs_main", pszFile="net_processing.cpp", nLine=2816) at sync.h:134:9
      frame #6: 0x00005555555b017f bitcoind`UniqueLock<AnnotatedMixin<std::recursive_mutex>, std::unique_lock<std::recursive_mutex> >::UniqueLock(this=0x00007fff99ff8c20, mutexIn=0x0000555556379220, pszName="cs_main", pszFile="net_processing.cpp", nLine=2816, fTry=false) at sync.h:160:13
      frame #7: 0x00005555556aa57e bitcoind`ProcessMessage(pfrom=0x00007fff90001180, msg_type=error: summary string parsing error, vRecv=0x00007fff9c005ac0, nTimeReceived=1592840815980751, chainparams=0x00005555564b7110, chainman=0x0000555556380880, mempool=0x0000555556380ae0, connman=0x000055555657aa20, banman=0x00005555565167b0, interruptMsgProc=0x00005555565cae90) at net_processing.cpp:2816:9
  ```

ACKs for top commit:
  laanwj:
    ACK 0ecff9dd34
  vasild:
    ACK 0ecff9dd

Tree-SHA512: ff285de8dd3198b5b33c4bfbdadf9b1448189c96143b9696bc4f41c07e784c00851ec169cf3ed45cc325f3617ba6783620803234f57fcce28bf6bc3d6a7234fb
2020-07-15 20:53:40 +02:00
MarcoFalke
804ca26629 Merge #19386: rpc: Assert that RPCArg names are equal to CRPCCommand ones (server)
fa7592bfa8 rpc: Update server to use new RPCHelpMan (MarcoFalke)
aaaaad5627 rpc: Add option to hide RPCArg (MarcoFalke)
fa9708f94c rpc: Assert that passed arg names are equal to hardcoded ones (MarcoFalke)
faaeb2b0b3 rpc: Add CRPCCommand constructor which takes RPCHelpMan (MarcoFalke)
fa8ec00061 rpc: Check that left section is not multiline (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in server. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  laanwj:
    ACK fa7592bfa8
  ryanofsky:
    Code review ACK fa7592bfa8. Looks great! Just some hidden arg and Check() and comment cleanups since last review

Tree-SHA512: e64b6a212f4a3aeedeee47557559bde104d5fd40cdc1746b27eb2f3d4c8885d5e6e4dd287595ea11cdbc6a939654fe103cae765fd505875444d851f0abb11308
2020-07-15 19:20:21 +02:00
Matt Corallo
f58c4b538e [tests] Remove unnecessary cs_mains in denialofservice_tests
9fdf05d70c resolved some lock
inversion warnings in denialofservice_tests, but left in a number
of cs_main locks that are unnecessary (introducing lock inversion
warnings in future changes).
2020-07-15 16:34:05 +01:00
Wladimir J. van der Laan
d626a3be31 Merge #19210: qt: Get rid of cursor in out-of-focus labels
bd315eb5e2 qt: Get rid of cursor in out-of-focus labels (Hennadii Stepanov)

Pull request description:

  After clicking on `QLabel` with selectable text the cursor remains forever:

  ![47532924-65e7b200-d8ba-11e8-9254-7bde658961cb](https://user-images.githubusercontent.com/32963518/84038485-ad945200-a9a8-11ea-89e3-c7c17d02a611.png)

  This PR fixes this visual bug.

  Earlier attempts to fix this issue:
  - #14577
  - #14810 (combined with other UX feature)

ACKs for top commit:
  promag:
    Code review ACK bd315eb5e2.
  laanwj:
    Tested ACK bd315eb5e2

Tree-SHA512: 6bf89362412e5ce9a4dec6944b62fe44fc31ca49cda7f6e2eb37e847fac9dccb68bca7ac6877b19e42add2333e40d0b4265757ead105ac0a5d28f8ab43b322c3
2020-07-15 16:48:03 +02:00
Wladimir J. van der Laan
21209c9cce Merge #19512: p2p: banscore updates to gui, tests, release notes
fa108d6a75 test: update tests for peer discouragement (Jon Atack)
1a9f462caa gui, doc: rm Ban Score in GUI Peers window/release notes updates (Jon Atack)

Pull request description:

  This is the third `-banscore` PR in the mini-series described in #19464. See that PR for the intention and reasoning.

  - no longer display "Ban Score" in the GUI peers window and add a release note, plus release note fixups per https://github.com/bitcoin/bitcoin/pull/19464#pullrequestreview-447452052
  - update tests (`src/test/denialofservice_tests.cpp` and `test/functional/p2p_leak.py`) from banning to discouragement and per https://github.com/bitcoin/bitcoin/pull/19464#issuecomment-658052518

ACKs for top commit:
  jnewbery:
    ACK fa108d6a75
  laanwj:
    ACK fa108d6a75

Tree-SHA512: 58a449b3f47b8cb5490b34e4442ee8675bfad1ce48af4e4fd5c67715b0c1a596fb8e731d42e576b4c3b64627f76e0a68cbb1da9ea9f588a5932fe119baf40d50
2020-07-15 16:32:27 +02:00
Amir Ghorbanian
284a969cc0 Linter to check commit message formatting
Write linter to check that commit messages have a new line before the body or no body at all.

reference: gist.github.com/agnivade/67b42d664ece2d4210c7

Fixes issue #19091.
2020-07-15 10:01:49 -04:00
Wladimir J. van der Laan
7ebc365047 Merge #19214: Auto-detect SHA256 implementation in benchmarks
addf18da95 Call SHA256AutoDetect in benchmark setup (Pieter Wuille)

Pull request description:

  It seems `SHA256AutoDetect()` was not being called in benchmarks, making the numbers only reflect the naive implementation. Fix this by calling it in bench_bitcoin's setup.

ACKs for top commit:
  fjahr:
    tested ACK addf18da95
  pstratem:
    ACK addf18da95
  laanwj:
    ACK addf18da95

Tree-SHA512: 3ba4b068145942df1429bf5913e3f685511e6ebeae2c1a3f9b8ac0144f6db1c7df456f88f480a2129f3e1602e3bf6a39530bb96e2c74c03ddb19324cec6799c7
2020-07-15 15:35:11 +02:00
MarcoFalke
fa56eda58e log: Avoid treating remote misbehvior as local system error 2020-07-15 14:53:58 +02:00
MarcoFalke
fa492895b5 refactor: Switch ValidationState mode to C++11 enum class 2020-07-15 14:52:16 +02:00
Wladimir J. van der Laan
43125596ce Merge #19296: tests: Add fuzzing harness for AES{CBC,}256{Encrypt,Decrypt}, poly1305_auth, CHKDF_HMAC_SHA256_L32, ChaCha20 and ChaCha20Poly1305AEAD
cca7c577d5 tests: Add fuzzing harness for ChaCha20Poly1305AEAD (practicalswift)
2fc4e5916c tests: Add fuzzing harness for ChaCha20 (practicalswift)
e9e8aac029 tests: Add fuzzing harness for CHKDF_HMAC_SHA256_L32 (practicalswift)
ec86ca1aaa tests: Add fuzzing harness for poly1305_auth(...) (practicalswift)
4cee53bba7 tests: Add fuzzing harness for AES256CBCEncrypt/AES256CBCDecrypt (practicalswift)
9352c32325 tests: Add fuzzing harness for AES256Encrypt/AES256Decrypt (practicalswift)

Pull request description:

  Add fuzzing harness for `AES{CBC,}256{Encrypt,Decrypt}`, `poly1305_auth`, `CHKDF_HMAC_SHA256_L32`, `ChaCha20` and `ChaCha20Poly1305AEAD`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  laanwj:
    ACK cca7c577d5

Tree-SHA512: cff9acefe370c12a3663aa55145371df835479c6ab8f6d81bbf84e0f81a9d6b0d94e45ec545f9dd5e1702744eaa7947a1f4ffed0171f446fc080369161afd740
2020-07-15 14:45:12 +02:00
Wladimir J. van der Laan
2c4093ebc3 Merge #18882: build: fix -Wformat-security check when compiling with GCC
6cef3652d1 build: fix -Wformat-security check when compiling with GCC (fanquake)

Pull request description:

  GCC expects `-Wformat` to be passed with [`-Wformat-security`](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html), which means
  when we test for it in configure it currently fails:

  ```bash
  checking whether C++ compiler accepts -Wformat-security... no
  ...
  configure:15907: checking whether C++ compiler accepts -Wformat-security
  configure:15926: g++ -std=c++11 -c -g -O2 -Werror -Wformat-security  conftest.cpp >&5
  cc1plus: error: '-Wformat-security' ignored without '-Wformat' [-Werror=format-security]
  cc1plus: all warnings being treated as errors
  ```

  and never gets added to our CXX flags. Note that Clang does not have this requirement and the check is working correctly there.

  The change in this PR is the simple fix, however we might want to consider using something like `-Wformat=2` in future, which in GCC is equivalent to `-Wformat -Wformat-nonliteral -Wformat-security -Wformat-y2k.` and similar [in Clang](https://clang.llvm.org/docs/DiagnosticsReference.html#wformat-2).

ACKs for top commit:
  practicalswift:
    ACK 6cef3652d1
  laanwj:
    ACK 6cef3652d1

Tree-SHA512: f9230d42af39f85ea9d2f55dbbebd2bae4740fe59b0da2e092af3ac9ef7e6799d3a4cf83eb64574c63982e5f6b14e226d44c84fa0335255d65c9947d86a1ea38
2020-07-15 13:57:41 +02:00
practicalswift
ad6c34881d tests: Add fuzzing harness for CBlockPolicyEstimator::{Read,Write} (policy/fees.h) 2020-07-15 11:41:21 +00:00
practicalswift
614e0807a8 tests: Add fuzzing harness for CBufferedFile::{SetPos,GetPos,GetType,GetVersion} (stream.h) 2020-07-15 11:41:21 +00:00
practicalswift
7bcc71e5f8 tests: Add fuzzing harness for LoadExternalBlockFile(...) (validation.h) 2020-07-15 11:41:21 +00:00
practicalswift
9823376030 tests: Add fuzzing harness for CBufferedFile (streams.h) 2020-07-15 11:41:21 +00:00
practicalswift
f3aa659be6 tests: Add fuzzing harness for CAutoFile (streams.h) 2020-07-15 11:41:21 +00:00
practicalswift
e507c0799d tests: Add serialization/deserialization fuzzing helpers WriteToStream(…)/ReadFromStream(…) 2020-07-15 11:41:21 +00:00
practicalswift
e48094a506 tests: Add FuzzedAutoFileProvider which provides a CAutoFile interface to FuzzedDataProvider 2020-07-15 11:41:21 +00:00
practicalswift
9dbcd6854c tests: Add FuzzedFileProvider which provides a FILE* interface to FuzzedDataProvider using fopencookie 2020-07-15 11:41:21 +00:00
fanquake
6cef3652d1 build: fix -Wformat-security check when compiling with GCC
Debian GCC ignores -Wformat-security, without -Wformat, which
means when we test for it, it currently fails:

```bash
checking whether C++ compiler accepts -Wformat-security... no
...
configure:15907: checking whether C++ compiler accepts -Wformat-security
configure:15926: g++ -std=c++11 -c -g -O2 -Werror -Wformat-security  conftest.cpp >&5
cc1plus: error: '-Wformat-security' ignored without '-Wformat' [-Werror=format-security]
cc1plus: all warnings being treated as errors
```

Fix this by just combining the -Wformat and -Wformat-security checks
together.
2020-07-15 17:50:01 +08:00
MarcoFalke
3d57015aa2 Merge #19491: util: Make Assert work with any value
fa53635381 util: Make Assert work with any value (MarcoFalke)

Pull request description:

  Goal is to avoid compile failures

ACKs for top commit:
  jonatack:
    ACK fa53635381
  ryanofsky:
    Code review ACK fa53635381. Looks like if argument is an lvalue this effectively does:

Tree-SHA512: a5cf47a8bb2fa1bd8b8895774f33de50ad803165d6f7b520351be1cfcd5612d5d97c51d118461331d30640186c470879e5ad19e3333e09e72685c5e4e4f23079
2020-07-15 09:03:41 +02:00
fanquake
032cfdc9a2 Merge #19517: psbt: Increment input value sum only once per UTXO in decodepsbt
75122780e2 Increment input value sum only once per UTXO in decodepsbt (Andrew Chow)

Pull request description:

  Refactors the UTXO processing of `decodepsbt` to extract the relevant `CTxOut` and handle the input amounts from that. This avoids double counting the input value.

  Fixes #19516

ACKs for top commit:
  sipa:
    utACK 75122780e2
  ryanofsky:
    Code review ACK 75122780e2

Tree-SHA512: 004ec1597790a88a98098f1a26534d10ab0130a438dec0913522a529a8d7f18ad679948617dbcad6e541fbab5bcb2682aeed386b67746807c03b64d76ce5441d
2020-07-15 12:01:06 +08:00
Jon Atack
dca28634d7 test: ensure OP_1NEGATE satisfies BIP62 minimal push rule
with a regression test for PR 13084 and PR 17204.
2020-07-15 15:29:46 +12:00
Pieter Wuille
e629d07199 Do not turn OP_1NEGATE in scriptSig into 0x0181 in signing code
Co-authored-by: Samuel Dobson <dobsonsa68@gmail.com>
2020-07-15 15:29:22 +12:00
Hennadii Stepanov
fdf697fe75 ci: Increase CCACHE_SIZE in some builds on Travis 2020-07-14 20:41:17 +03:00
Andrew Chow
75122780e2 Increment input value sum only once per UTXO in decodepsbt 2020-07-14 12:20:06 -04:00
MarcoFalke
f4de89edfa Merge #19429: test: Fix intermittent failure in wallet_encryption
fabd33b541 test: Fix intermittent failure in wallet_encryption (MarcoFalke)

Pull request description:

  Iterating all crypted keys might take time.

  E.g.

  ```
   node0 2020-07-01T14:41:19.227367Z [httpworker.0] ThreadRPCServer method=walletpassphrase user=__cookie__
   node0 2020-07-01T14:41:24.377142Z [httpworker.0] queue run of timer lockwallet() in 100000000 seconds (using HTTP)
  ...
   test  2020-07-01T14:41:24.379000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 117, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_encryption.py", line 88, in run_test
                                         assert_greater_than(expected_time + 5, actual_time) # 5 second buffer
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 54, in assert_greater_than
                                         raise AssertionError("%s <= %s" % (str(thing1), str(thing2)))
                                     AssertionError: 1693614483 <= 1693614484
  ```

  https://cirrus-ci.com/task/5322429885054976?command=ci#L4517

ACKs for top commit:
  achow101:
    ACK fabd33b541

Tree-SHA512: 7a3ccdfc0cdc05fef1f942d3167d100ed63422eb54c05405c884ed91162b7bdb5ce54cb5a981b99a6df2e4af1ea834ccd7d5156531c8c14ea13e735becd6b377
2020-07-14 17:08:19 +02:00
Andrew Chow
d416ae560e walletdb: Introduce WalletDatabase abstract class
Make WalletDatabase actually an abstract class and not just a typedef
for BerkeleyDatabase. Have BerkeleyDatabase inherit this class.
2020-07-14 11:07:16 -04:00
Andrew Chow
2179dbcbcd walletdb: Add BerkeleyDatabase::Open dummy function
Adds an Open function for the class abstraction that does nothing for
now.
2020-07-14 11:07:16 -04:00
Andrew Chow
71d28e7cdc walletdb: Introduce AddRef and RemoveRef functions
Refactor mapFileUseCount increment and decrement to separate functions
AddRef and RemoveRef
2020-07-14 11:07:16 -04:00
Andrew Chow
27b2766384 walletdb: Move BerkeleyDatabase::Flush(true) to Close()
Instead of having Flush optionally shutdown the database and
environment, add a Close() function that does that.
2020-07-14 11:07:16 -04:00
MarcoFalke
834ac4c0f5 Merge #19323: gui: Fix regression in *txoutset* in GUI console
314b49bd50 gui: Fix regression in GUI console (Hennadii Stepanov)

Pull request description:

  The regression was introduced in #19056: if the GUI is running without `-server=1`, the `*txoutset*` call in the console returns "Shutting down".

  Fix #19255.

ACKs for top commit:
  ryanofsky:
    Code review ACK 314b49bd50. Only change since last review is rebase

Tree-SHA512: 8ff85641a5c249858fecb1ab69c7a1b2850af651ff2a94aa41ce352b5b5bc95bc45c41e1767e871b51e647612d09e4d54ede3e20c313488afef5678826c51b62
2020-07-14 16:33:01 +02:00
MarcoFalke
a924f61cae Merge #19325: wallet: Refactor BerkeleyDatabase to introduce DatabaseBatch abstract class
b82f0ca4d5 walletdb: Add MakeBatch function to BerkeleyDatabase and use it (Andrew Chow)
eac9200814 walletdb: Refactor DatabaseBatch abstract class from BerkeleyBatch (Andrew Chow)

Pull request description:

  In order to support alternative database systems, we need to have a generic `Batch` class. This PR adds a `DatabaseBatch` abstract class which is implemented by `BerkeleyBatch`. `DatabaseBatch` is now the class that is used by `WalletBatch` to interact with the database. To be able to get the correct type of `DatabaseBatch`, `BerkeleyDatabase` now has a `MakeBatch` function which returns a newly constructed `std::unique_ptr<DatabaseBatch>`. For `BerkeleyDatabase`, that will be `std::unique_ptr<BerkeleyBatch>`.

  The `Read`, `Write`, `Erase`, and `Exists` template functions are moved from `BerkeleyBatch`.

  Part of #18971

  Requires #19308 and #19324

ACKs for top commit:
  Sjors:
    re-utACK b82f0ca4d5
  MarcoFalke:
    ACK b82f0ca4d5 🌘
  meshcollider:
    LGTM, utACK b82f0ca4d5

Tree-SHA512: 6d2d41631c0983391dbecd702e881c6775b155c90b275df97f7157e42608ed251744f9d7ce5173d02a6c5cc38d90b611880fac7fa635d3d8c4d590681f56ac6a
2020-07-14 16:21:01 +02:00
MarcoFalke
b26d62c49a Merge #18990: log: Properly log txs rejected from mempool
fa9f20b647 log: Properly log txs rejected from mempool (MarcoFalke)

Pull request description:

  Currently `CheckTxInputs` rejections from the mempool are the only rejections that log directly and unconditionally to debug.log instead of leaving it to the caller. This has multiple issues:

  * A rejected RPC transaction will log a redundant failure reason to debug log. All other failures are merely reported to the RPC user.
  * A rejected p2p transaction will log the failure twice. Once with the `MEMPOOLREJ` flag, and once unconditionally.
  * A rejected orphan transaction will log no failure.

  Fix all issues by simply returning the state to the caller, like it is done for all other rejections.

  The patch includes whitespace fixups to highlight relevant parts of the codebase and simplify review.

ACKs for top commit:
  naumenkogs:
    utACK fa9f20b647
  rajarshimaitra:
    Concept ACK. Compiled and ran tests. `fa9f20b`
  jnewbery:
    code review ACK fa9f20b647

Tree-SHA512: 86cc17b2a9239c01c4fc3f254ad48ee1d3883266966b9811030176338b9ac3deaea7ea5babfb8bbf739d7440154e30011fede8f9313175f199d4a062af6494f7
2020-07-14 16:15:07 +02:00
Gregory Sanders
7356292e1d Have zmq reorg test cover mempool txns 2020-07-14 10:14:11 -04:00
Sebastian Falbesoner
0c8461a88e refactor: replace CConnman pointers by references in net_processing.cpp 2020-07-14 16:00:24 +02:00
MarcoFalke
1a655e89cb Merge #19514: [net/net processing] check banman pointer before dereferencing
ca3585a483 [net/net processing] check banman pointer before dereferencing (John Newbery)

Pull request description:

  Although we currently don't do this, it should be possible to create a
  CConnman or PeerLogicValidation without a Banman instance. Therefore
  always check that banman exists before dereferencing the pointer.

  Also add comments to the m_banman members of CConnman and
  PeerLogicValidation to document that these may be nullptr.

ACKs for top commit:
  jonatack:
    ACK ca3585a
  theStack:
    ACK ca3585a483

Tree-SHA512: 726401c8921b9a502029ead34ae797473a1bc359d6e4e58dcbe3e25b70dde40bb100723be467fd3e2bf418892c493911998226de19c9d529d72034e3be26be48
2020-07-14 15:26:40 +02:00
Jon Atack
fa108d6a75 test: update tests for peer discouragement 2020-07-14 14:15:01 +02:00
MarcoFalke
faa9a74c9e test: Fail wait_until early if connection is lost 2020-07-14 12:29:47 +02:00
John Newbery
ca3585a483 [net/net processing] check banman pointer before dereferencing
Although we currently don't do this, it should be possible to create a
CConnman or PeerLogicValidation without a Banman instance. Therefore
always check that banman exists before dereferencing the pointer.

Also add comments to the m_banman members of CConnman and
PeerLogicValidation to document that these may be nullptr.
2020-07-14 10:24:43 +01:00
Jon Atack
1a9f462caa gui, doc: rm Ban Score in GUI Peers window/release notes updates 2020-07-14 10:18:17 +02:00
MarcoFalke
07c83ce039 Merge #19268: doc: Add non-thread-safe note to FeeFilterRounder::round()
d842e6ac96 doc: Add non-thread-safe note to FeeFilterRounder::round() (Hennadii Stepanov)

Pull request description:

  The `FastRandomContext` class is documented as not thread-safe.
  This PR adds a relevant note to the `FeeFilterRounder::round()` function declaration.

  Close #19254

ACKs for top commit:
  MarcoFalke:
    self ACK d842e6ac96
  practicalswift:
    ACK d842e6ac96: explicit is better than implicit
  naumenkogs:
    ACK d842e6a

Tree-SHA512: 538508f24b9cb29baece6a64108e2c5fc3960768c6475c4f2baf48a4a7bdb96dcef1a74d21a4822e1f8635e1375362986da4e3a20f5644129046a354c4b0a8a0
2020-07-14 09:30:02 +02:00
MarcoFalke
90b5fc9ce4 Merge #19495: ci: Disable macOS functional tests on forked repos to avoid timeouts
60824b3c3a ci: Fix configure options for macOS builds (Hennadii Stepanov)
687939e3d2 ci: Drop Homebrew caching while using Homebrew addon on Travis (Hennadii Stepanov)
557d3f1cc0 ci: Do not activate Travis ccache caching strategy (Hennadii Stepanov)
2d747428e2 ci: Disable functional tests on forked repos to avoid timeouts for macOS (Hennadii Stepanov)

Pull request description:

  See: https://github.com/bitcoin-core/gui/issues/5#issuecomment-656819184

  Additionally, this PR:
  - updates macOS image to the recent 10.15.5 version
  - drops Homebrew caching as the Travis Homebrew addon have been used since #18438

  My forked repo build: https://travis-ci.org/github/hebasto/bitcoin/jobs/707200431

Top commit has no ACKs.

Tree-SHA512: 398e935f965a04babeb10e7b26d2341562f21a1ef671c2e7cc97c9ec79d5c31643f81ca18561ab7714b5c52e19df2e4bffe4223eadbab984daa9418ffbf8c2a8
2020-07-14 08:40:32 +02:00
MarcoFalke
b93c4244b9 Merge #19464: net: remove -banscore configuration option
06059b0c2a net: rename DEFAULT_BANSCORE_THRESHOLD to DISCOURAGEMENT_THRESHOLD (Jon Atack)
1d4024bca8 net: remove -banscore configuration option (Jon Atack)

Pull request description:

  per https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652684340, https://github.com/bitcoin/bitcoin/pull/19219#discussion_r443074487 and https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652699592. Edit: now split into 3 straightforward PRs:
  - net: remove -banscore configuration option (this PR)
  - rpc: deprecate banscore field in getpeerinfo (#19469, *merged*)
  - gui: no longer display banscores (TBA in the gui repo)

ACKs for top commit:
  MarcoFalke:
    review ACK 06059b0c2a 📙
  vasild:
    ACK 06059b0c

Tree-SHA512: 03fad249986e0896697033fbb8ba2cbfaae7d7603b1fb2a38b3d41db697630d238623f4d732b9098c82af249ce5a1767dd432b7ca0fec10544e23d24fbd57c50
2020-07-14 08:13:25 +02:00
fanquake
5550fa5983 Merge #19109: Only allow getdata of recently announced invs
f32c408f3a Make sure unconfirmed parents are requestable (Pieter Wuille)
c4626bcd21 Drop setInventoryTxToSend based filtering (Pieter Wuille)
43f02ccbff Only respond to requests for recently announced transactions (Pieter Wuille)
b24a17f039 Introduce constant for mempool-based relay separate from mapRelay caching (Pieter Wuille)
a9bc563803 Swap relay pool and mempool lookup (Pieter Wuille)

Pull request description:

  This implements the follow-up suggested here: https://github.com/bitcoin/bitcoin/pull/18861#issuecomment-627630111 . Instead of checking `setInventoryTxToSend`, maintain an explicit bloom filter with the 3500 most recently announced invs, and permit fetching any of these as long as they're in the relay pool or the mempool. In addition, permit relay from the mempool after just 2 minutes instead of 15.

  This:

  * Fixes the brief opportunity an attacker has to request unannounced invs just after the connection is established (pointed out by naumenkogs, see https://github.com/bitcoin/bitcoin/pull/18861#issuecomment-627627010).
  * Guarantees that locally resubmitted invs after `filterInventoryKnown` rolls over can still be requested (pointed out by luke-jr, see https://github.com/bitcoin/bitcoin/pull/18861#discussion_r419695831).

  It adds 37 KiB of filter per peer.

  This is also a step towards dropping the relay pool entirely and always relaying from the mempool directly (see #17303), but that is still blocked by dealing properly with NOTFOUNDs (see #18238).

ACKs for top commit:
  jnewbery:
    reACK f32c408f3
  jonatack:
    re-ACK f32c408 per `git range-diff f7c19e8 2da7ee3 f32c408` and redid the following: code review, thought about motivation, DoS and privacy aspects, debug build to check for warnings after updating Clang from 6 to 11 since last review.
  ajtowns:
    re-ACK f32c408f3a

Tree-SHA512: aa05b9fd01bad59581c4ec91836a52d7415dc933fa49d4c4adced79aa25aaad51e11166357e8c8b29fbf6021a7401b98c21b850b5d8e8ad773fdb5d6608e1e85
2020-07-14 08:40:35 +08:00
Wladimir J. van der Laan
89fa736783 Merge #19494: doc: CONTRIBUTING.md improvements
b03697b68e doc: CONTRIBUTING.md improvements (Jon Atack)

Pull request description:

  The motivation here was to add a mention of hygienic commits following a discussion today, e.g. something along the lines of:

  *Make sure each individual commit is hygienic, building successfully on its own without warnings, errors, or regressions, and that all tests pass.*

  While here, made various fixups. They are optional and can be omitted.

ACKs for top commit:
  harding:
    ACK b03697b68e Locally reviewed the word diff.
  MarcoFalke:
    ACK b03697b68e 🚌
  practicalswift:
    ACK b03697b68e
  hebasto:
    ACK b03697b68e, I have reviewed the changes and they look OK, I agree they can be merged.

Tree-SHA512: 6fb56219c311d914ec18fcf5d50fdbe3a51e4743a8cace93e348cb4a10c83b6fce631518f1455a1804d1fc81558b235bef58a8be6ccb1a010f46aa4143b1ebf5
2020-07-14 02:11:09 +02:00
Hennadii Stepanov
60824b3c3a ci: Fix configure options for macOS builds 2020-07-14 02:37:15 +03:00
Hennadii Stepanov
687939e3d2 ci: Drop Homebrew caching while using Homebrew addon on Travis 2020-07-14 02:35:02 +03:00
Hennadii Stepanov
557d3f1cc0 ci: Do not activate Travis ccache caching strategy
It is sufficient to specify CCACHE_DIR to cache.
Also this change fixes ccache on native macOS build.
2020-07-14 02:35:02 +03:00
Hennadii Stepanov
2d747428e2 ci: Disable functional tests on forked repos to avoid timeouts for macOS 2020-07-14 02:34:37 +03:00
MarcoFalke
fa8a992589 Work around memory-aliasing in descriptor ParsePubkey 2020-07-13 21:58:10 +02:00
Gregory Sanders
a0f4f9c983 Add zmq test for transaction pub during reorg 2020-07-13 14:18:01 -04:00
Gregory Sanders
2399a0600c Add test case for mempool->block zmq notification 2020-07-13 14:18:01 -04:00
Gregory Sanders
e70512a83c Make ordering of zmq consumption irrelevant to functional test 2020-07-13 14:18:01 -04:00
Wladimir J. van der Laan
37485baa27 Merge #19483: doc: Merge release notes
fa740d4cea doc: move-only release notes (MarcoFalke)
faa3d2eee1 doc: Remove release notes of backports (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK fa740d4cea

Tree-SHA512: 86e7077f9050c65b0b33829892aafabbb783a603a1a77932893659bc1071fb2d2a0515662e7e9e6fdfffb59f86bfc41ad74ba6e86d618bf71a6616d9f7a18fa2
2020-07-13 18:33:00 +02:00
Andrew Chow
d0ea9bab28 walletdb: Don't remove database transaction logs and instead error
Instead of removing the database transaction logs and retrying the
wallet loading, just return an error message to the user. Additionally,
specifically for DB_RUNRECOVERY, notify the user that this could be due
to different BDB versions. This error is pretty much only caused by
compiling with a newer version of BDB and then trying to open the wallet
with a version compiled with an older version of BDB.
2020-07-13 11:00:54 -04:00
Russell Yanofsky
edc316020e test: Remove duplicate NodeContext hacks
Qt tests currently are currently using two NodeContext structs at the same
time, one in interfaces::NodeImpl::m_context, and the other in
BasicTestingSetup::m_node, and the tests have hacks transferring state between
them.

Fix this by getting rid of the NodeImpl::m_context struct and making it a
pointer. This way a common BitcoinApplication object can be used for all qt
tests, but they can still have their own testing setups.

Non-test code is changing but non-test behavior is still the same as before.

Motivation for this PR is to be able to remove the
"std::move(test.m_node.connman)" and mempool hacks for swapping individual
NodeContext members in Qt tests, because followup PR #19099 adds yet another
member (wallet_client) that needs to be swapped. After this change, the whole
NodeContext struct can be swapped instead of individual members, so the
workarounds are less fragile and invasive.
2020-07-13 04:34:27 -04:00
MarcoFalke
d52bfc4916 Merge bitcoin-core/gui#30: Disable the main window toolbar when the modal overlay is shown
d0cc1f6df7 qt: Disable toolbar when overlay is shown (Hennadii Stepanov)
e74cd2083d qt, refactor: Cleanup ModalOverlay slots (Hennadii Stepanov)

Pull request description:

  Keeping the main window toolbar activated while the modal overlay is shown could create the appearance of the non-responsive GUI.

  Fixes #22.

  ---

  On master (ca055885c6):

  ![Screenshot from 2020-07-11 13-07-00](https://user-images.githubusercontent.com/32963518/87221791-7504e100-c377-11ea-9689-ddd4b21b98f9.png)

  With this PR:

  ![Screenshot from 2020-07-11 13-07-39](https://user-images.githubusercontent.com/32963518/87221803-8817b100-c377-11ea-92c8-3602dc4d2451.png)

ACKs for top commit:
  harding:
    Tested ACK d0cc1f6df7.  Tested on Linux/X11 as much as I could given it's a pretty small change; seems like a nice improvement.  I'm not experienced in Qt, but I don't see anything obviously problematic about the code.
  jonatack:
    ACK d0cc1f6 tested on Debian 5.7.6-1 (2020-06-24) x86_64 GNU/Linux
  LarryRuane:
    ACK d0cc1f6df7 tested on Ubuntu 18.04.4 LTS

Tree-SHA512: e371b34231c01e77118deb100e0f280ba1cdef54e317f7f7d6ac322598bda811bd1bfe3035e90d87f8267f4f5d2095d34a8136911159db63694fd1b1b11335a1
2020-07-13 10:34:27 +02:00
MarcoFalke
631284f09a Merge #19486: Remove unused constants CADDR_TIME_VERSION and GETHEADERS_VERSION
7bb6f9bfdb [protocol] Remove unused GETHEADERS_VERSION (John Newbery)
37a934e6b3 [protocol] Remove unused CADDR_TIME_VERSION (John Newbery)

Pull request description:

  These constants are no longer required and can be removed.

  Additional code comments are added to explain CAddress serialization.

ACKs for top commit:
  MarcoFalke:
    ACK 7bb6f9bfdb already an improvement, but maybe getting rid of INIT_PROTO_VERSION here would be an even stronger improvement (can be done later)
  jonatack:
    ACK 7bb6f9bfdb
  vasild:
    ACK 7bb6f9bf

Tree-SHA512: 5382562c60fd677c86583754eca11aad3719064efe2e5ef4f307d693b583422ca8d385926c2582aaab899f502b151f2eb87a7ac23363b15f4fceaa06296f98e3
2020-07-13 10:32:19 +02:00
Hennadii Stepanov
b5a80fa7e4 util: Handle HTTP_SERVICE_UNAVAILABLE in bitcoin-cli 2020-07-12 13:31:16 -06:00
Jon Atack
b03697b68e doc: CONTRIBUTING.md improvements 2020-07-12 07:52:28 +02:00
MarcoFalke
fa740d4cea doc: move-only release notes
Review suggestion:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2020-07-12 07:15:50 +02:00
MarcoFalke
faa3d2eee1 doc: Remove release notes of backports 2020-07-12 07:15:42 +02:00
Samuel Dobson
4db44acf2d Merge #18202: refactor: consolidate sendmany and sendtoaddress code
08fc6f6cfc [rpc] refactor: consolidate sendmany and sendtoaddress code (Sjors Provoost)

Pull request description:

  I consolidated code between these two RPC calls, since `sendtoaddress` is essentially `sendmany` with 1 destination.

  Unless I overlooked something, the only behaviour change is that some `sendtoaddress` error codes changed from `-4` to `-6`. The release note mentions this.

  Salvaged from #18201.

ACKs for top commit:
  fjahr:
    Code review ACK 08fc6f6cfc
  jonatack:
    ACK 08fc6f6cfc
  meshcollider:
    Code review & functional test run ACK 08fc6f6cfc

Tree-SHA512: 7b66c52fa0444a4d02fc3f81d9c2a386794d447616026a30111eda35fb46510475eea6506a9ceda00bb4e0230ebb758da5d236b3ac05c954c044fa68a1e3e909
2020-07-12 14:42:35 +12:00
Samuel Dobson
32302e5c88 Merge #19490: wallet: Fix typo in comments; Simplify assert
facd7dd3d1 wallet: Fix typo in comments; Simplify assert (MarcoFalke)

Pull request description:

  Follow up to https://github.com/bitcoin/bitcoin/pull/19046#discussion_r443783101 and https://github.com/bitcoin/bitcoin/pull/19046#discussion_r443793690

ACKs for top commit:
  practicalswift:
    ACK facd7dd3d1
  jonatack:
    ACK facd7dd3d1
  hebasto:
    ACK facd7dd3d1, spelling verified with `test/lint/lint-spelling.sh`: all remaining warnings are false positive.

Tree-SHA512: 2b185d138058840db56726bb6bcc42e5288a954e2a410c49e04806a047fbbdaf0bb2decc70ecf7613c69caa766655705ca44151613e7ea5015b386d1e726d870
2020-07-12 13:34:37 +12:00
Jon Atack
06059b0c2a net: rename DEFAULT_BANSCORE_THRESHOLD to DISCOURAGEMENT_THRESHOLD
and move it from validation to net processing.
2020-07-11 19:41:24 +02:00
Jon Atack
1d4024bca8 net: remove -banscore configuration option 2020-07-11 19:41:21 +02:00
MarcoFalke
42fe6aad32 Merge #19493: wallet: Fix clang build in Mac
1e58bcc9af wallet: Fix clang build in Mac (Anthony Fieroni)

Pull request description:

  Signed-off-by: Anthony Fieroni <bvbfan@abv.bg>

Top commit has no ACKs.

Tree-SHA512: 19312929af14dab97c37cf4547fbd6589a6de960f1a499c2118bb684240639af4b127cf8dc4d201b41d253cfbb645614a0606d4ecce29f300b10c210d38a961b
2020-07-11 19:12:43 +02:00
Anthony Fieroni
1e58bcc9af wallet: Fix clang build in Mac
Signed-off-by: Anthony Fieroni <bvbfan@abv.bg>
2020-07-11 19:33:57 +03:00
MarcoFalke
fa53635381 util: Make Assert work with any value 2020-07-11 15:02:07 +02:00
MarcoFalke
facd7dd3d1 wallet: Fix typo in comments; Simplify assert 2020-07-11 14:24:36 +02:00
Samuel Dobson
160800ac10 Merge #19441: walletdb: don't reinitialize desc cache with multiple cache entries
a66a7a1a70 walletdb: don't reinitialize desc cache with multiple cache entries (Andrew Chow)

Pull request description:

  When loading descriptor caches, we would accidentally reinitialize the descriptor cache when seeing that one already exists. This should have only been initializing the cache when one does not exist. However this code itself is unnecessary as the act of looking up the cache to add to it will initialize it if it didn't already exist.

  This issue could be hit by trying to load a wallet that had imported a multisig descriptor. The wallet would fail to load.

  A test has been added to wallet_importdescriptors.py to catch this case. Another test case has also been added to check that loading a wallet with only single key descriptors works.

ACKs for top commit:
  hugohn:
    tACK [a66a7a1](a66a7a1a70)
  jonatack:
    ACK a66a7a1a70
  meshcollider:
    Code review ACK a66a7a1a70

Tree-SHA512: 3df746421a008708eaa3bbbdd12b9ddd3e2ec111d54625a212dca7414b971cc1f6e2b1757b3232c31a2f637d1b1ef43bf3ffa4ac4216646cf1e92db5f79954f1
2020-07-12 00:14:27 +12:00
Samuel Dobson
5f96bce9b7 Merge #18923: wallet: Never schedule MaybeCompactWalletDB when -flushwallet is off
fa73493930 refactor: Use C++11 range-based for loop (MarcoFalke)
fa7b164d62 wallet: Never schedule MaybeCompactWalletDB when -flushwallet is off (MarcoFalke)
faf8401c19 wallet: Pass unused args to StartWallets (MarcoFalke)
fa6c186436 gui tests: Limit life-time of dummy testing setup (MarcoFalke)
fa28a61897 test: Add smoke test to check that wallets are flushed by default (MarcoFalke)

Pull request description:

  User-facing, this is a refactor. Internally, the scheduler does not have to call a mostly empty function every half a second.

ACKs for top commit:
  jnewbery:
    utACK fa73493930
  meshcollider:
    utACK fa73493930
  ryanofsky:
    Code review ACK fa73493930. Just rebased since last review

Tree-SHA512: 99e1fe1b2c22a3f4b19de3e566241d38693f4fd8d5a68ba1838d86740aa6c08e3325c11a072e30fd262a8861af4278bed52eb9374c85179b8f536477f528247c
2020-07-11 23:23:28 +12:00
Samuel Dobson
89899a3448 Merge #19046: Replace CWallet::Set* functions that use memonly with Add/Load variants
3a9aba21a4 Split SetWalletFlags into Add/LoadWalletFlags (Andrew Chow)
d9cd095b59 Split SetActiveScriptPubKeyMan into Add/LoadActiveScriptPubKeyMan (Andrew Chow)
0122fbab4c Split SetHDChain into AddHDChain and LoadHDChain (Andrew Chow)

Pull request description:

  `SetHDChaiin`, `SetActiveScriptPubKeyMan`, and `SetWalletFlags` have a `memonly` argument which is kind of confusing, as noted in https://github.com/bitcoin/bitcoin/pull/17681#discussion_r427633081. This PR replaces those functions with `Add*` and `Load*` variants so that they follow the pattern used elsewhere in the wallet.

  `AddHDChain`, `AddActiveScriptPubKeyMan`, and `AddWalletFlags` both set their respective variables in `CWallet` and writes them to disk. These functions are used by the actions which modify the wallet such as `sethdseed`, `importdescriptors`, and creating a new wallet.

  `LoadHDChain`, `LoadActiveScriptPubKeyMan`, and `LoadWalletFlags` just set the `CWallet` variables. These functions are used by `LoadWallet` when loading the wallet from disk.

ACKs for top commit:
  jnewbery:
    Code review ACK 3a9aba21a4
  ryanofsky:
    Code review ACK 3a9aba21a4. Only changes since last review tweaks making m_wallet_flags updates more safe
  meshcollider:
    utACK 3a9aba21a4

Tree-SHA512: 365aeaafc5ba42879c0eb797ec3beb29ab70e27f917dc880763f743420b3be6ddf797240996beed8a9ad70fb212c2590253c6b44c9dc244529c3939d9538983f
2020-07-11 23:08:54 +12:00
Samuel Dobson
4fc9224ee7 Merge #18850: wallet: Fix ZapSelectTx to sync wallet spends
9c59f9c285 Fix ZapSelectTx to sync wallet spends (Anthony Fieroni)

Pull request description:

  Signed-off-by: Anthony Fieroni <bvbfan@abv.bg>

ACKs for top commit:
  achow101:
    ACK 9c59f9c285
  ryanofsky:
    Code review ACK 9c59f9c285. Only change since last review tweaking the for loop as suggested
  jonatack:
    ACK 9c59f9c285 tested rebased on current master b33136b6ba and the new unit test does indeed fail without the change.
  meshcollider:
    utACK 9c59f9c285

Tree-SHA512: 71672a5ab0c659550c3a40577614ea896412b79566b5672636ab18765e4c71b9d0a990d94dc6b6e623b03a05737022b04026b5699438809c7c54782d0fd0a5d2
2020-07-11 22:20:43 +12:00
Hennadii Stepanov
d0cc1f6df7 qt: Disable toolbar when overlay is shown 2020-07-11 12:59:58 +03:00
Russell Yanofsky
9c69cfe4c5 Add <datadir>/settings.json persistent settings storage.
Persistent settings are used in followup PRs #15936 to unify gui settings
between bitcoin-qt and bitcoind, and #15937 to add a load_on_startup flag to
the loadwallet RPC and maintain a dynamic list of wallets that should be loaded
on startup that also can be shared between bitcoind and bitcoin-qt.
2020-07-11 05:41:12 -04:00
MarcoFalke
a42631775a Merge #19222: tests: Add fuzzing harness for BanMan
97846d7f5b tests: Add fuzzing harness for BanMan (practicalswift)
deba199f1c tests: Add ConsumeSubNet(...). Move and increase coverage in ConsumeNetAddr(...). (practicalswift)

Pull request description:

  Add fuzzing harness for `BanMan`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: f4126c15bbb77638833367d73f58193c8f05d16bed0b1d6c33b39387d5b610ff34af78cd721adb51778062ce3ac5e79756d1c3895ef54c6c80c61dcf056e94ff
2020-07-11 11:41:12 +02:00
Russell Yanofsky
eb682c5700 util: Add ReadSettings and WriteSettings functions
Currently unused, but includes tests.
2020-07-11 05:41:12 -04:00
Hennadii Stepanov
e74cd2083d qt, refactor: Cleanup ModalOverlay slots 2020-07-11 12:16:29 +03:00
MarcoFalke
ca055885c6 Merge #19474: doc: Use precise permission flags where possible
fab5586122 doc: Use precise permission flags where possible (MarcoFalke)

Pull request description:

  Instead of mentioning the all-encompassing `-whitelist*` settings, change the docs to mention the exact permission flag that will influence the behaviour.

  This is needed because in the future, the too-broad `-whitelist*` settings (they either include *all* permission flags or apply to *all* peers) might be deprecated to require the permission flags to be enumerated.

  Alternatively, in the future there could be an RPC to set the net permission flags on an existing connection, in which case the `-whitelist*` terminology is of no help.

ACKs for top commit:
  jnewbery:
    reACK fab5586122
  fjahr:
    Code review ACK fab5586122
  jonatack:
    ACK fab5586122

Tree-SHA512: c7dea3e577d90103bb2b0ffab7b7c8640b388932a3a880f69e2b70747fc9213dc1f437085671fd54c902ec2a578458b8a2fae6dbe076642fb88efbf9fa9e679c
2020-07-11 10:23:09 +02:00
John Newbery
655b195747 [net processing] Continue SendMessages processing if not disconnecting peer
If we don't disconnect a peer in MaybeDiscourageAndDisconnect because it
has NOBAN permissions or it's a manual connection, continue SendMessages
processing rather than exiting early.

The previous behaviour was that we'd miss the SendMessages processing on
this iteration of the MessageHandler loop. That's not a problem since
SendMessages() would just be called again on the next iteration, but it
was slightly inefficient and confusing.
2020-07-11 07:13:05 +01:00
John Newbery
a49781e56d [net processing] Only call MaybeDiscourageAndDisconnect from SendMessages
`nMisbehavior` is a tally in `CNodeState` that can be incremented from
anywhere. That almost always happens inside a `ProcessMessages()` call
(because we increment the misbehavior score when receiving a bad
messages from a peer), but not always. See, for example, the call to
`MaybePunishNodeForBlock()` inside `BlockChecked()`, which is an
asynchronous callback from the validation interface, executed on the
scheduler thread.

As long as `MaybeDiscourageAndDisconnect()` is called regularly for the
node, then the misbehavior score exceeding the 100 threshold will
eventually result in the peer being punished. It doesn't really matter
where that `MaybeDiscourageAndDisconnect()` happens, but it makes most
sense in `SendMessages()` which is where we do general peer
housekeeping/maintenance.

Therefore, remove the `MaybeDiscourageAndDisconnect()` call in
`ProcessMessages()` and move the `MaybeDiscourageAndDisconnect()` call
in `SendMessages()` to the top of the function. This moves it out of the
cs_main lock scope, so take that lock directly inside
`MaybeDiscourageAndDisconnect()`.

Historic note: `MaybeDiscourageAndDisconnect()` was previously
`SendRejectsAndCheckIfBanned()`, and before that was just sending
rejects.  All of those things required cs_main, which is why
`MaybeDiscourageAndDisconnect()` was called after the ping logic.
2020-07-11 07:06:20 +01:00
Jon Atack
f0aa8aeea5 test: add rpc_generate functional test 2020-07-11 07:48:07 +02:00
Jon Atack
92d94ffb8d rpc: print useful help and error message for generate 2020-07-11 07:47:57 +02:00
Carl Dong
de4fedb6c3 depends: Add documentation for FORCE_USE_SYSTEM_CLANG make flag 2020-07-10 21:06:32 -04:00
Carl Dong
fe98999dcf depends: Reformat make options as definition list 2020-07-10 21:06:31 -04:00
Carl Dong
60c55b1b9b depends: Add justifications for macOS clang flags 2020-07-10 21:05:55 -04:00
Cory Fields
6b8e497eea depends: specify libc++ header location for darwin
For depends builds this was fixed by fbcfcf69, which deleted the conflicting
headers. When we no longer control the clang installation, we need to ensure
that the SDK's libc++ headers are used rather than the ones shipped with clang.
We can do that by turning off the default include path and hard-coding our own.
This hard-coded path is ok because we control (via SDK packaging) where these
headers end-up.

Side-note: Now that this path is hard-coded in depends, we can potentially
package the SDK differently, as the c++ folder can live wherever is most
convenient for us.
2020-07-10 21:05:54 -04:00
Cory Fields
156b604203 depends: force a new host id string if FORCE_USE_SYSTEM_CLANG is in use
This should be caught by the differing clang --version outputs, but because we
haven't yet extracted our pinned clang, the system one is actually used for the
version check. That's not a problem because bumping our pinned clang will
cause a rebuild of everything anyway.
2020-07-10 21:05:53 -04:00
John Newbery
7bb6f9bfdb [protocol] Remove unused GETHEADERS_VERSION
GETHEADERS_VERSION is unused. No need to keep it in the codebase
for just its historic comment.
2020-07-10 22:14:18 +01:00
John Newbery
37a934e6b3 [protocol] Remove unused CADDR_TIME_VERSION
Add comments to CAddress serialization code explaining why
it's no longer needed.
2020-07-10 22:14:18 +01:00
MarcoFalke
505b4eda55 Merge #19140: tests: Avoid fuzzer-specific nullptr dereference in libevent when handling PROXY requests
20d31bdd92 tests: Avoid fuzzer-specific nullptr dereference in libevent when handling PROXY requests (practicalswift)

Pull request description:

  Avoid constructing requests that will be interpreted by libevent as PROXY requests to avoid triggering a `nullptr` dereference. Split out from #19074 as suggested by MarcoFalke.

  The dereference (`req->evcon->http_server`) takes place in `evhttp_parse_request_line` and is a consequence of our hacky but necessary use of the internal function `evhttp_parse_firstline_` in the `http_request` fuzzing harness.

  The suggested workaround is not aesthetically pleasing, but it successfully avoids the troublesome code path.

  `" http:// HTTP/1.1\n"` was a crashing input prior to this workaround.

  Before this PR:

  ```
  $ echo " http:// HTTP/1.1" > input
  $ src/test/fuzz/http_request input
  src/test/fuzz/http_request: Running 1 inputs 1 time(s) each.
  Running: input
  AddressSanitizer:DEADLYSIGNAL
  =================================================================
  ==27905==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000108 (pc 0x55a169b7e053 bp 0x7ffd452f1160 sp 0x7ffd452f10e0 T0)
  ==27905==The signal is caused by a READ memory access.
  ==27905==Hint: address points to the zero page.
      #0 0x55a169b7e053 in evhttp_parse_request_line depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-36daee64dc1/http.c:1883:37
      #1 0x55a169b7d9ae in evhttp_parse_firstline_ depends/work/build/x86_64-pc-linux-gnu/libevent/2.1.11-stable-36daee64dc1/http.c:2041:7
      #2 0x55a1687f624e in test_one_input(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/test/fuzz/http_request.cpp:51:9
  …
  $ echo $?
  1
  ```

  After this PR:

  ```
  $ echo " http:// HTTP/1.1" > input
  $ src/test/fuzz/http_request input
  src/test/fuzz/http_request: Running 1 inputs 1 time(s) each.
  Running: input
  Executed input in 0 ms
  ***
  *** NOTE: fuzzing was not performed, you have only
  ***       executed the target code on a fixed set of inputs.
  ***
  $ echo $?
  0
  ```

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: 7a6b68e52cbcd6c117487e74e47760fe03566bec09b0bb606afb3b652edfd22186ab8244e8e27c38cef3fd0d4a6c237fe68b2fd22e0970c349e4ab370cf3e304
2020-07-10 22:51:56 +02:00
MarcoFalke
5802ea6bd3 Merge #19453: refactor: reduce DefaultRequestHandler memory allocations
f20b359bb9 cli: reduce DefaultRequestHandler memory allocations (Jon Atack)

Pull request description:

  per https://github.com/bitcoin/bitcoin/pull/16439#discussion_r443957125. Simpler code, fewer allocations. No change of behavior. The code has good test coverage in `interface_bitcoin_cli.py`.

ACKs for top commit:
  MarcoFalke:
    review ACK f20b359bb9
  fjahr:
    Code review ACK f20b359

Tree-SHA512: 745eab44dfdcc485ca2bbc1db8b8d364cbd3cf94982e46e033745ce05ab617c15320ee55da7fb930d365f4d26b172049d5f5efcf0b6d3af5b0a28185bdb93ea8
2020-07-10 22:41:31 +02:00
John Newbery
a1d5a428a2 [net processing] Fix bad indentation in SendMessages()
Hint for reviewers: review ignoring whitespace changes.
2020-07-10 18:20:07 +01:00
John Newbery
1a1c23f8d4 [net processing] Change cs_main TRY_LOCK to LOCK in SendMessages()
This was changed to TRY_LOCK in #1117 to fix a potential deadlock
between cs_main and cs_vSend. cs_vSend was split into cs_vSend and
cs_sendProcessing in #9535 (and cs_sendProcessing was changed from a
TRY_LOCK to a LOCK in the same PR).

Since cs_vSend can no longer be taken before cs_main, revert this to a
LOCK().

This commit leaves part of the code with bad indentation. That is fixed
by the next (whitespace change only) commit.
2020-07-10 18:20:07 +01:00
MarcoFalke
c0b0b0240f Merge #14033: p2p: Drop CADDR_TIME_VERSION checks now that MIN_PEER_PROTO_VERSION is greater
57b0c0a93a Drop CADDR_TIME_VERSION checks now that MIN_PEER_PROTO_VERSION is greater (Ben Woosley)

Pull request description:

  We do not connect to peers older than 31800

ACKs for top commit:
  sipa:
    Code reivew ACK 57b0c0a93a
  jnewbery:
    Code review ACK 57b0c0a93a
  vasild:
    ACK 57b0c0a9

Tree-SHA512: e1ca7c9203cbad83ab7c7a2312777ad07ed6a16119169b256648b8a8738c260a5168acdd4fb33f6e4b17f51ec7e033e110b76bde55b4e3b2d444dc02c01bc2b1
2020-07-10 19:16:48 +02:00
MarcoFalke
107b8559c5 Merge #18638: net: Use mockable time for ping/pong, add tests
fa3365430c net: Use mockable time for ping/pong, add tests (MarcoFalke)
faab4aaf2f util: Add count_microseconds helper (MarcoFalke)

Pull request description:

  Switch `CNode::m_ping_start` and `CNetMessage::m_time` to mockable time, so that tests can be added.

  Mockable time is also type-safe, since it uses `std::chrono`

ACKs for top commit:
  jonatack:
    Code review re-ACK fa33654 re-read code, verified rebase per `git range-diff 4b5c919 fa94d6f fa33654`, previous tested ACKs still valid
  troygiorshev:
    ACK fa3365430c

Tree-SHA512: 7d632bd6019ce7c882029e71b667a61517e783af82755a85dd979ef09380934e172dec8b8f91d57b200a30a6e096aeaf01f19fee7f3aed0e0e871c72eb44d70e
2020-07-10 16:06:28 +02:00
MarcoFalke
fab5586122 doc: Use precise permission flags where possible 2020-07-10 15:37:42 +02:00
Vasil Dimov
bc74a40a56 net: improve encapsulation of CNetAddr
Do not access `CNetAddr::ip` directly from `CService` methods.

This improvement will help later when we change the type of
`CNetAddr::ip` (in the BIP155 implementation).

Co-authored-by: Carl Dong <contact@carldong.me>
2020-07-10 14:58:42 +02:00
fanquake
a4eb6a51a7 Merge #19469: rpc: deprecate banscore field in getpeerinfo
41d55d3057 doc: getpeerinfo banscore deprecation release note (Jon Atack)
dd54e3796e test: getpeerinfo banscore deprecation test (Jon Atack)
8c7647b3fb rpc: deprecate banscore field in rpc getpeerinfo (Jon Atack)

Pull request description:

  Per https://github.com/bitcoin/bitcoin/pull/19219#discussion_r443074487 and https://github.com/bitcoin/bitcoin/pull/19219#issuecomment-652699592, this PR deprecates returning the `banscore` field in the `getpeerinfo` RPC, updates the help, adds a test, and updates the release notes. Related to #19464.

ACKs for top commit:
  fanquake:
    ACK 41d55d3057

Tree-SHA512: 8eca08332581e2fe191a2aafff6ba89ce39413f0491ed0de8b86577739f0ec430b1a8fbff2914b0f3138a229563dfcc1981c0cf5b7dd6061b5c48680a28423bc
2020-07-10 13:45:48 +08:00
João Barbosa
4e353cb618 http: Release work queue after event base finish
This fixes a race between http_request_cb and StopHTTPServer where
the work queue is used after release.
2020-07-09 20:12:53 +01:00
MarcoFalke
2aaff4813c Merge #19283: refactor: Remove unused BlockAssembler::pblock member var
fa6d5ab674 refactor: Remove unused BlockAssembler::pblock member var (MarcoFalke)

Pull request description:

  It seems odd to have a confusing and fragile "convenience pointer" member variable to be able to write `pblock->vtx` instead of `pblocktemplate->block.vtx` in a single place.

ACKs for top commit:
  promag:
    Code review ACK fa6d5ab674.

Tree-SHA512: e9f032b5ab702dbefffd370db3768ebfb95c13acc732972b695281ea34c91d70cd0a1700bc2c6f106dbc9de68e81bc6bb06c68c2afd53c17cba8ebee4f9931b9
2020-07-09 20:24:17 +02:00
Wladimir J. van der Laan
fd9db45c3e Merge #19258: doc: improve subtree check instructions
a4a3fc4cd2 doc: improve subtree check instructions (Sjors Provoost)

Pull request description:

  Running `git-subtree-check.sh` requires adding the subtree repository as a remote. I learned that several years ago and then forgot again.

  This PR also improves the error message if the subtree commit can't be found.

ACKs for top commit:
  laanwj:
    ACK a4a3fc4cd2
  fanquake:
    ACK a4a3fc4cd2 - this looks ok.

Tree-SHA512: 959bd923726c172d17f9f97f8a56988bf2df5a94d3131e5152a66150b941394cee9e82fdc6b86e09c0ba91d123a496599f07ca454212168d8d301738394c12c8
2020-07-09 17:52:54 +02:00
Andrew Chow
b82f0ca4d5 walletdb: Add MakeBatch function to BerkeleyDatabase and use it
Instead of having WalletBatch construct the BerkeleyBatch, have
BerkeleyDatabase do it and return a std::unique_ptr<BerkeleyBatch>
2020-07-09 11:43:54 -04:00
Andrew Chow
eac9200814 walletdb: Refactor DatabaseBatch abstract class from BerkeleyBatch 2020-07-09 11:43:52 -04:00
Jon Atack
8d32d2011d test: consider generate covered in _get_uncovered_rpc_commands() 2020-07-09 17:27:22 +02:00
MarcoFalke
cc9d09e73d Merge #19191: net: Extract download permission from noban
fa0540cd46 net: Extract download permission from noban (MarcoFalke)

Pull request description:

  It should be possible to grant nodes in a local network (e.g. home, university, enterprise, ...) permission to download blocks even after the maxuploadtarget is hit.

  Currently this is only possible by setting the `noban` permission, which has some adverse effects, especially if the peers can't be fully trusted.

  Fix this by extracting a `download` permission from `noban`.

ACKs for top commit:
  jonatack:
    ACK fa0540c
  Sjors:
    re-utACK fa0540cd46

Tree-SHA512: 255566baa43ae925d93f5d0a3aa66b475a556d1590f662a88278a4872f16a1a05739a6119ae48a293011868042e05cb264cffe5822a50fb80db7333bf44376d9
2020-07-09 17:03:27 +02:00
Wladimir J. van der Laan
0d69fdb9a0 Merge #19314: refactor: Use uint16_t instead of unsigned short
1cabbddbca refactor: Use uint16_t instead of unsigned short (Aaron Hook)

Pull request description:

  I wanted to see if the `up for grabs` label works and looked at PR #17822 originally opend by ahook I saw it had many acks for example by jonatack and practicalswift but needed rebasing.

  So I checked out the remote branch rebased it resolved three conflicts and continued the rebase.

  Hope everything is as expected (:

ACKs for top commit:
  sipsorcery:
    ACK 1cabbddbca.
  practicalswift:
    ACK 1cabbddbca -- patch looks correct :)
  laanwj:
    ACK 1cabbddbca
  hebasto:
    ACK 1cabbddbca, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 0e6bf64f274aae5dacb188358b4d5f65ccb207d4f70922f039bc4ed7934709418ddad19f8bfb7462517427837c3d2bb3f86ef284bb40e87119aad2a1e148d9d6
2020-07-09 16:56:05 +02:00
fanquake
22ccc27046 Merge #18993: qt: increase console command max length
fc6a637a01 qt: increase console command max length (10xcryptodev)

Pull request description:

  fix #17618
  Tested the examples https://github.com/bitcoin/bitcoin/issues/17618#issuecomment-559538070 and works

ACKs for top commit:
  MarcoFalke:
    Approach ACK fc6a637a01
  hebasto:
    ACK fc6a637a01, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 4975d7fa4c13a6b0f50f5754c3e04eb5a42b1411c385dc883d9948b6fc0dee38900ba2a418218a9a30ce39988a27d22f3ff3a02f0fa44f4136f01eef473efeca
2020-07-09 22:28:23 +08:00
Wladimir J. van der Laan
9a3c7afe29 Merge #19317: Add a left-justified width field to log2_work component for a uniform debug.log output
c858302280 Change format of log2_work for uniform output (zero-padded) (jmorgan)

Pull request description:

  Motivation:
  It's jarring to watch the output of `tail -f ~/btcdata/debug.log` scroll by and very frequently see columns not lining up correctly because `log2_work` somtimes has less precision than 8 digits.

  Current display:
  ```
  2020-06-18T02:54:42Z UpdateTip: new best=0000000000000000107f877e4920643f9fb06090fa7551cd1cdd83b857f520aa height=382038 version=0x00000003 log2_work=83.558653 tx=90953616 date='2015-11-04T17:11:44Z' progress=0.166675 cache=117.6MiB(966410txo)
  2020-06-18T02:54:51Z UpdateTip: new best=0000000000000000019a4de585d30d1a8cc13c7a1972d11b4945635c9556acb5 height=382039 version=0x00000003 log2_work=83.55868 tx=90955936 date='2015-11-04T17:19:39Z' progress=0.166679 cache=117.9MiB(968799txo)
  ```

  Display with this commit:
  ```
  2020-06-18T02:54:42Z UpdateTip: new best=0000000000000000107f877e4920643f9fb06090fa7551cd1cdd83b857f520aa height=382038 version=0x00000003 log2_work=83.558653 tx=90953616 date='2015-11-04T17:11:44Z' progress=0.166675 cache=117.6MiB(966410txo)
  2020-06-18T02:54:51Z UpdateTip: new best=0000000000000000019a4de585d30d1a8cc13c7a1972d11b4945635c9556acb5 height=382039 version=0x00000003 log2_work=83.55868  tx=90955936 date='2015-11-04T17:19:39Z' progress=0.166679 cache=117.9MiB(968799txo)
  ```

ACKs for top commit:
  practicalswift:
    ACK c858302280 -- patch looks great :)
  achow101:
    ACK c858302280
  laanwj:
    Tested ACK c858302280

Tree-SHA512: 16cbe419c4993ad51019c676e8ca409ef1025b803cc598437c780dd7ca003d7e4ad421f451e9a374e0070ee9b3ee601b7aba849e1f346798f9321d1bce5c4401
2020-07-09 16:03:27 +02:00
fanquake
45f58dbc9b Merge #19452: doc: afl fuzzing comment about afl-gcc and afl-g++
2b78a11b48 doc: afl fuzzing comment about afl-gcc and afl-g++ (nsa)

Pull request description:

  When trying to build the fuzz tests with `--enable-lcov` on a Ubuntu machine, noticed that the documentation was lacking with regards to the afl-gcc and afl-g++ options. `afl-clang-fast` and `afl-clang-fast++` in the examples just need to be replaced with `afl-gcc` and `afl-g++`. I also had to set the `-m` flag as well to get the fuzzers to run.

ACKs for top commit:
  practicalswift:
    ACK 2b78a11b48
  MarcoFalke:
    Concept ACK 2b78a11b48, haven't tested

Tree-SHA512: d8151afd79de949e8c6da49b69bbbf1470eb478c8ddcbc69b30e86bf9396c0f13835a655d4ae658f7dc4f36c35b02cd23b08358fb73a71e15bf14e76c1f365a4
2020-07-09 21:24:28 +08:00
Wladimir J. van der Laan
fc8da23fbe Merge #19445: build: Update msvc build to use ISO standard C++17
2894e94d17 Updates msvc build to use ISO standard C++17. (Aaron Clauson)

Pull request description:

  This PR adds a compiler option to the msvc build to specify ISO C++17 support as discussed in #16684.

  In order to allow Bitcoin Core to compile with the new option two pre-processor defines are also necessary to avoid an warning (treated as an error) for C++17 deprecated features:

   - _SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING
   - _SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING

  For anyone interested a sample compiler message for each of the warnings is shown below:

  ````
  c:\Dev\github\sipsorcery_bitcoin\src\support\allocators\zeroafterfree.h(21,5): error C4996: 'std::allocator<_Ty>::const_pointer': warning STL4010: Various members of std::allocator are deprecated in C++17. Use std::allocator_traits instead of accessing these members directly. You can define _SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING or _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS to acknowledge that you have received this warning. [c:\Dev\github\sipsorcery_bitcoin\build_msvc\libbitcoin_qt\libbitcoin_qt.vcxproj]
  ````

  ````
  c:\Dev\github\sipsorcery_bitcoin\src\fs.cpp(29,31): error C4996: 'std::codecvt_utf8_utf16<wchar_t,1114111,(std::codecvt_mode)0>': warning STL4017: std::wbuffer_convert, std::wstring_convert, and the <codecvt> header (containing std::codecvt_mode, std::codecvt_utf8, std::codecvt_utf16, and std::codecvt_utf8_utf16) are deprecated in C++17. (The std::codecvt class template is NOT deprecated.) The C++ Standard doesn't provide equivalent non-deprecated functionality; consider using MultiByteToWideChar() and WideCharToMultiByte() from <Windows.h> instead. You can define _SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING or _SILENCE_ALL_CXX17_DEPRECATION_WARNINGS to acknowledge that you have received this warning. [c:\Dev\github\sipsorcery_bitcoin\build_msvc\libbitcoin_util\libbitcoin_util.vcxproj]
  ````

ACKs for top commit:
  MarcoFalke:
    Approach ACK 2894e94d17
  laanwj:
    ACK 2894e94d17

Tree-SHA512: aff14726e05cb52f81dee32eafbd5b9ec829d3ed032ed4a03345458d6a22875a307ee8543952298a95d30d80720158586d33dbc8bf1970a99c403c3f1510d7fd
2020-07-09 15:22:00 +02:00
Wladimir J. van der Laan
21028ce9ff Merge #19468: refactor: Drop unused CDBWrapper methods
4b5ac25881 Drop unused CDBWrapper methods (Hennadii Stepanov)

Pull request description:

  `CDBWrapper::Flush()` and `CDBWrapper::Sync()` are not used in the code.

ACKs for top commit:
  promag:
    ACK 4b5ac25881.
  laanwj:
    ACK 4b5ac25881

Tree-SHA512: 06115c59e75995d496173a64ceea1b9bb1b4fe3eac8bf4f59df68b87b112b5b3e8065298dcd5c4c7408544f76ee62922325acc2208619d830fd5dbb420cdda5c
2020-07-09 15:11:04 +02:00
fanquake
64bf5d64df Merge #18896: qt: Reset toolbar after all wallets are closed
1e9bfd4926 qt: Reset toolbar after all wallets are closed (Hennadii Stepanov)

Pull request description:

  If the last open wallet is closed from the non-"Overview" tab, that tab remains active when a new wallet is opened:

  ![Screenshot from 2020-05-06 09-00-26](https://user-images.githubusercontent.com/32963518/81142394-46821880-8f78-11ea-84b5-1d02f2b7f3f1.png)

  This PR fixes this bug.

ACKs for top commit:
  promag:
    Code review ACK 1e9bfd4926.
  luke-jr:
    utACK 1e9bfd4926
  jonasschnelli:
    utACK 1e9bfd4926

Tree-SHA512: a8dfd7591267e9544ad40c87581d554e5cfaad4b2a5bbfdbaf2596dc6869d2ac6cf7877adfef3d528fc61b081d40c6e30d787bbd7280ef7946aa7f7d9bc8b18e
2020-07-09 20:51:10 +08:00
fanquake
e3b31255c5 Merge #19470: banlist: log post-swept banlist size at startup
0b8ba84659 banlist: log post-swept banlist size at startup (fanquake)

Pull request description:

  We are currently logging the size of the banlist from before `SweepBanned()` has been called, meaning the value may be incorrect.

  i.e banlist.dat had `1`ban. That ban is swept on startup. We log "loaded 1 banned node..". Actual banlist size is `0`.

ACKs for top commit:
  jonatack:
    Code review ACK 0b8ba84659 `m_banned` is set in SetBanned and is updated by SweepBanned before the logging.
  laanwj:
    Code review ACK 0b8ba84659
  jnewbery:
    Code review ACK 0b8ba84659

Tree-SHA512: 1d6e363d6c68d7cc214dd685df3d2d27572f6a58a4c0e43c03cfbb03bc01badb6a10ecae403d137094bb316d27f33feb6be15b4e23ef1e9496cd0b3c23c21698
2020-07-09 20:29:31 +08:00
fanquake
6b48c30541 Merge #19454: tools: .clang-format compat with clang versions < 9
b9253c7d20 tools: clang-format 6 compatibility (Jon Atack)

Pull request description:

  Our `.clang-format` settings inadvertently lost compatibility with Clang versions < 9 in #19095, including for Debian stable. This patch returns compatibility in the interim until the distros update. See discussion from https://github.com/bitcoin/bitcoin/pull/19095#issuecomment-651926138.

ACKs for top commit:
  MarcoFalke:
    Approach ACK b9253c7d20 , haven't tested

Tree-SHA512: 4af541a195f48d84ffb80e23aaefb624c66bc78f087c8d92b4af5a654420b69fedf25272c6e4fde2688ff88412d306b7a990ce1e15d8b24180374c625a253fb6
2020-07-09 19:59:15 +08:00
MarcoFalke
fa73493930 refactor: Use C++11 range-based for loop 2020-07-09 13:08:42 +02:00
MarcoFalke
fa7b164d62 wallet: Never schedule MaybeCompactWalletDB when -flushwallet is off 2020-07-09 13:07:41 +02:00
MarcoFalke
faf8401c19 wallet: Pass unused args to StartWallets
This refactor does not change behavior
2020-07-09 13:07:37 +02:00
MarcoFalke
fa6c186436 gui tests: Limit life-time of dummy testing setup
Its only purpose is to create a directory. So instead of keeping it
alive, use it only to get the path of the directory and then create it
explicitly.
2020-07-09 13:07:35 +02:00
MarcoFalke
fa28a61897 test: Add smoke test to check that wallets are flushed by default 2020-07-09 13:07:17 +02:00
MarcoFalke
c4a44186d8 Merge #19085: Refactor: clean up PeriodicFlush()
e846a2a1d9 refactor: clean up PeriodicFlush() (John Newbery)

Pull request description:

  `PeriodicFlush()` is much more convoluted than it needs to be: it has triple nesting, local variables counting refs and return values, and increments the `mapFileUseCount` iterator unnecessarily. Removing all of that makes the function much easier to understand.

ACKs for top commit:
  MarcoFalke:
    ACK e846a2a1d9 🎁
  jonatack:
    re-ACK e846a2a per `git range-diff f7c19e8 7c10020 e846a2a`
  promag:
    ACK e846a2a1d9.

Tree-SHA512: 22bc600a5268b139c0a2c16b5a9f14837b262670ec24aef00643fcedd1c3ebcbf46dea1633e76adc8acf78e8840b776e17127307c5ee95308caa94239dad5b88
2020-07-09 12:54:17 +02:00
MarcoFalke
fa0540cd46 net: Extract download permission from noban 2020-07-09 12:48:05 +02:00
John Newbery
e846a2a1d9 refactor: clean up PeriodicFlush() 2020-07-09 07:00:11 +01:00
Pieter Wuille
f32c408f3a Make sure unconfirmed parents are requestable 2020-07-08 18:33:51 -07:00
Pieter Wuille
c4626bcd21 Drop setInventoryTxToSend based filtering 2020-07-08 18:29:56 -07:00
Pieter Wuille
43f02ccbff Only respond to requests for recently announced transactions
... unless they're UNCONDITIONAL_RELAY_DELAY old, or there has been
a response to a MEMPOOL request in the mean time.

This is accomplished using a rolling Bloom filter for the last
3500 announced transactions. The probability of seeing more than 100
broadcast events (which can be up to 35 txids each) in 2 minutes for
an outbound peer (where the average frequency is one per minute), is
less than 1 in a million.
2020-07-08 18:29:56 -07:00
Pieter Wuille
b24a17f039 Introduce constant for mempool-based relay separate from mapRelay caching
This constant is set to 2 minutes, rather than 15. This is still many times
larger than the transaction broadcast interval (2s for outbound, 5s for
inbound), so it should be acceptable for peers to know what our contents of
the mempool was that long ago.
2020-07-08 18:29:51 -07:00
Pieter Wuille
a9bc563803 Swap relay pool and mempool lookup
This is in preparation to using the mempool entering time as part of
the decision for relay, but does not change behavior on itself.
2020-07-08 18:28:00 -07:00
MarcoFalke
f7c19e829e Merge #19320: wallet: Replace CDataStream& with CDataStream&& where appropriate
fa8a341b88 wallet: Replace CDataStream& with CDataStream&& where appropriate (MarcoFalke)
fa021e9a5b wallet: Remove confusing double return value ret+success (MarcoFalke)

Pull request description:

  The keys and values are only to be used once because their memory is set
  to zero. Make that explicit by moving the bytes into the lower level
  methods.

ACKs for top commit:
  sipa:
    utACK fa8a341b88
  ryanofsky:
    Code review ACK fa8a341b88. Nice changes.

Tree-SHA512: 5c0218bae0f3cd2a07346f1bbf4ad232e5dde7ef2f807d82cc6cfd208d11fe60c8b0f37e7986087b52fbfc79cdfd33c3c8a5822b3d4d9a44d1c6b09e354fc424
2020-07-09 01:01:21 +02:00
MarcoFalke
9f4c0a9694 Merge #19347: [net] Make cs_inventory nonrecursive
e8a2822119 [net] Don't try to take cs_inventory before deleting CNode (John Newbery)
3556227ddd [net] Make cs_inventory a non-recursive mutex (John Newbery)
344e831de5 [net processing] Remove PushBlockInventory and PushBlockHash (John Newbery)

Pull request description:

  - Remove PushBlockInventory() and PushBlockHash(). These are one-line functions that can easy be inlined into the calling code. Doing so also allows us to eliminate the one place that cs_inventory is recursively locked.
  - Make cs_inventory a nonrecursive mutex
  - Remove a redundant TRY_LOCK of cs_inventory when deleting CNode.

ACKs for top commit:
  sipa:
    utACK e8a2822119
  MarcoFalke:
    ACK e8a2822119 🍬
  hebasto:
    re-ACK e8a2822119

Tree-SHA512: dbc721d102cdef7b5827a8f2549daf8b54f543050266999a7ea56c9f36618565b71e31ce0beb1209ba2db43d15388be173355a03fb6db8ad24e2475b145050bd
2020-07-08 21:57:25 +02:00
Cory Fields
e3e7446305 Add lifetimebound to attributes for general-purpose usage
Co-authored-by: practicalswift <practicalswift@users.noreply.github.com>
2020-07-08 13:31:56 -04:00
Hennadii Stepanov
314b49bd50 gui: Fix regression in GUI console
This change prevents "Shutting down" message during "dumptxoutset",
"gettxoutsetinfo" and "scantxoutset" calls.
2020-07-08 19:16:33 +03:00
fanquake
0b8ba84659 banlist: log post-swept banlist size at startup
We are currently logging the size of the banlist before SweepBanned()
has been called, meaning the value may be incorrect.
2020-07-08 21:44:45 +08:00
Jon Atack
41d55d3057 doc: getpeerinfo banscore deprecation release note 2020-07-08 15:10:27 +02:00
Hennadii Stepanov
4b5ac25881 Drop unused CDBWrapper methods 2020-07-08 14:26:14 +03:00
Jon Atack
dd54e3796e test: getpeerinfo banscore deprecation test 2020-07-08 13:14:50 +02:00
Jon Atack
8c7647b3fb rpc: deprecate banscore field in rpc getpeerinfo 2020-07-08 13:14:48 +02:00
practicalswift
97846d7f5b tests: Add fuzzing harness for BanMan 2020-07-08 05:31:43 +00:00
practicalswift
deba199f1c tests: Add ConsumeSubNet(...). Move and increase coverage in ConsumeNetAddr(...). 2020-07-08 05:05:12 +00:00
Carl Dong
c9c572a367 depends: Allow building with system clang 2020-07-07 16:26:44 -04:00
Carl Dong
e6e5c8d6ca depends: Decouple toolchain + binutils
For now they remain the same, but in the next commit, we will assign
them differently according to wether or not we're using system clang.
2020-07-07 16:25:47 -04:00
Pieter Wuille
abdfd2d0e3 Merge #19219: Replace automatic bans with discouragement filter
2ad58381ff Clean up separated ban/discourage interface (Pieter Wuille)
b691f2df5f Replace automatic bans with discouragement filter (Pieter Wuille)

Pull request description:

  This patch improves performance and resource usage around IP addresses that are banned for misbehavior. They're already not actually banned since #14929, as connections from them are still allowed, but they are preferred for eviction if the inbound connection slots are full.

  Stop treating these like manually banned IP ranges, and instead just keep them in a rolling Bloom filter of misbehaving nodes, which isn't persisted to disk or exposed through the ban framework. The effect remains the same: preferred for eviction, avoided for outgoing connections, and not relayed to other peers.

  Also change the name of this mechanism to "discouraged" to better reflect reality.

ACKs for top commit:
  naumenkogs:
    utACK 2ad58381ff
  amitiuttarwar:
    code review ACK 2ad58381ff
  jonatack:
    ACK 2ad5838 per changes since last review `git range-diff 3276c14 1f7e0ca 2ad5838`
  jnewbery:
    Code review ACK 2ad58381ff

Tree-SHA512: 5dedef401d9cbfa026812651303e6286223563dbeed7a10766ed536ac9e3f29ed4bd0df29cc6deadceeb35cbe9f066346add14ef0833958ca9f93d123fe7aab5
2020-07-07 11:20:34 -07:00
MarcoFalke
b52e25cc1b Merge #19328: Add gettxoutsetinfo hash_type option
40506bf93f test: Test gettxouttsetinfo hash_type option (Fabian Jahr)
f17a4d1c4d rpc: Add hash_type NONE to gettxoutsetinfo (Fabian Jahr)
a712cf6f68 rpc: gettxoutsetinfo can specify hash_type (only legacy option for now) (Fabian Jahr)
605884ef21 refactor: Extract GetBogoSize function (Fabian Jahr)

Pull request description:

  This is another intermediate part of the Coinstats Index (tracked in #18000).

  Sjors suggested [here](https://github.com/bitcoin/bitcoin/pull/18000#issuecomment-641423019) that the part of the changes in #19145 that don't rely on the new `hash_type` muhash, i.e. that are for `hash_type=none`, could be merged separately from everything involving muhash. So these changes are extracted from #19145 here and can be merged without any other requirements.

  Building the index with no UTXO set hash is still valuable because `gettxoutsetinfo` can still be used to audit the `total_amount` for example. By itself this PR is not a huge improvement, `hash_type=none` is speeding up `gettxoutsetinfo` by about 10%, but it enables the implementation of an index on top of it in a follow-up and that means large parts of the index code of Coinstats Index can be merged while reviews for the hashing algorithm might take longer.

ACKs for top commit:
  MarcoFalke:
    ACK 40506bf93f 🖨
  Sjors:
    tACK 40506bf93f

Tree-SHA512: 3964c2b8eed427511b1aa9b2ef285dff27dc4d1537d72c3911e435b6e6b40912232da4acb3a09bd19a0372ddffa44103388d8a650169d95a4a727b970d210add
2020-07-06 08:06:40 -04:00
Jon Atack
b9253c7d20 tools: clang-format 6 compatibility 2020-07-06 03:52:37 +02:00
nsa
2b78a11b48 doc: afl fuzzing comment about afl-gcc and afl-g++
This commit includes a short comment in doc/fuzzing.md that gives
guidance on compiling Bitcoin Core with AFL instrumentation using
afl-gcc and afl-g++.
2020-07-05 20:55:11 -04:00
MarcoFalke
f4301e9107 Merge #19450: ci: Add tsan suppression for race in BerkeleyBatch
a76dafa51d ci: Add tsan suppression for race in BerkeleyBatch (Hennadii Stepanov)

Pull request description:

  A temporary workaround for #19448.

Top commit has no ACKs.

Tree-SHA512: 47b83ff373e710bc9ba8c3661f9850a14417436028c42eb7765d21337ef25faaac4cf8cf93be844ae592d40264934d7d2f6b7ba0ab6c7209fc0da8fc13067769
2020-07-05 19:06:03 -04:00
MarcoFalke
171f4a516b Merge #19324: wallet: Move BerkeleyBatch static functions to BerkeleyDatabase
d8e9ca66d1 walletdb: Move Rewrite into BerkeleyDatabase (Andrew Chow)
91d109156d walletdb: Move PeriodicFlush into WalletDatabase (Andrew Chow)
8f1bcf8b7b walletdb: Combine VerifyDatabaseFile and VerifyEnvironment (Andrew Chow)

Pull request description:

  The `BerkeleyBatch` class has 4 static functions that operate on `BerkeleyDatabase` or `BerkeleyEnvironment`. It doesn't make sense for these to be standalone nor for them to be static functions. So instead, move them from `BerkeleyBatch` into `BerkeleyDatabase` and make them member functions instead of static.

  `BerkeleyBatch::VerifyEnvironment` and `BerkeleyBatch::VerifyDatabaseFile` are combined into a single `BerkeleyDatabase::Verify` function that operates on that `BerkeleyDatabase` object.

  `BerkeleyBatch::Rewrite` and `BerkeleyBatch::PeriodicFlush` both took a `BerkeleyDatabase` as an argument and did stuff on it. So we just make it a member function so it doesn't need to take a database as an argument.

  Part of #18971

ACKs for top commit:
  MarcoFalke:
    re-ACK d8e9ca66d1 only change is test fixup 🤞
  promag:
    Code review ACK d8e9ca66d1, good stuff.

Tree-SHA512: 9847e55b13d98bf4e5636cc14bc3f5351d56737f7e320fafffaed128606240765599e5400382c5aecac06690f7e36265ca3e1031f3f6d8a9688f6d5cb1bacd2a
2020-07-05 18:06:00 -04:00
Hennadii Stepanov
a76dafa51d ci: Add tsan suppression for race in BerkeleyBatch 2020-07-05 18:23:26 +03:00
Jon Atack
f20b359bb9 cli: reduce DefaultRequestHandler memory allocations 2020-07-05 16:39:17 +02:00
Hennadii Stepanov
d842e6ac96 doc: Add non-thread-safe note to FeeFilterRounder::round()
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-07-05 13:46:38 +03:00
Brian Liotti
9c34aff393 Remove previous_release.sh 2020-07-05 04:05:02 -04:00
Brian Liotti
e1e5960e10 script: Add previous_release.py
closes #18132
added GPG verify for binaries

co-authored-by: bboot <bboot@cisco.com>
2020-07-05 04:04:52 -04:00
MarcoFalke
8783bcc099 Merge #19444: test: Remove cached directories and associated script blocks from appveyor config
961e667600 Remove cached directories and associated script blocks from appveyor CI configuration. (Aaron Clauson)

Pull request description:

  Appveyor CI jobs have been failing in the last 24 hours due to a seemingly corrupted cache, see #19440.

  It's possible that the appveyor cache issue is related to the[ recent update](https://www.appveyor.com/updates/2020/07/03/) of the Visual Studio 2019 image

  PR #19431 changes the "save cache or error" to false in an attempt to avoid a failing CI job from potentially corrupting the cache. In theory the only way a PR could affect the cache is if the `vcpkg` install list changed. That happens very rarely and did not happen in the last 24 hours and so was not the cause of the current cache problems.

  I have done some testing with appveyor build jobs on my own fork and found that installing the `vcpkg` dependencies from scratch and doing a full build can now be done in just under 60 minutes. This is the first time in over 5 months I have been able to build Bitcoin Core on appveyor. Either the new Visual Studio 2019 image has dramatically reduced the build time or appveyor images have had their CPU increased.

  This PR removes all use of dependency caching from the appveyor CI config. The trade-off is the 15 minutes saved on each build from having the dependencies cached versus the hours maintainers need to spend investigating when the CI jobs start failing.

ACKs for top commit:
  MarcoFalke:
    ACK 961e667600

Tree-SHA512: 788c7efbfe6e044739ec41b08df30e24e26bfe0f31d1f5695e7243222a2eb649a2b5fd0254a9238fd416661dc05f737b0545d39feea7aa0da2236fffd7683a1b
2020-07-04 14:13:48 -04:00
Aaron Clauson
2894e94d17 Updates msvc build to use ISO standard C++17. 2020-07-04 16:03:18 +01:00
MarcoFalke
5ec19df687 Merge #19277: util: Add Assert identity function
fab80fef61 refactor: Remove unused EnsureChainman (MarcoFalke)
fa34587f1c scripted-diff: Replace EnsureChainman with Assert in unit tests (MarcoFalke)
fa6ef701ad util: Add Assert identity function (MarcoFalke)
fa457fbd33 move-only: Move NDEBUG compile time check to util/check (MarcoFalke)

Pull request description:

  The utility function is primarily useful to dereference pointer types, which are known to be not null at that time.

  For example, the ArgsManager is known to exist when the wallets are started: https://github.com/bitcoin/bitcoin/pull/18923/files#diff-fdb2a1a1d8bc790fcddeb6cf5a42ac55R503 . Instead of silently relying on that assumption, `Assert` can be used to abort the program and avoid UB should the assumption ever be violated.

ACKs for top commit:
  promag:
    Tested ACK fab80fef61.
  ryanofsky:
    Code review ACK fab80fef61

Tree-SHA512: 830fba10152ba17d47c4dd42809c7e26f9fe6d38e17a2d5b3f054fd644a5c4c9841286ac421ec9bb28cea9f5faeb659740fcf00de6cc589d423fee7694c42d16
2020-07-04 08:44:45 -04:00
Aaron Clauson
961e667600 Remove cached directories and associated script blocks from appveyor CI configuration. 2020-07-04 13:43:18 +01:00
Pieter Wuille
2ad58381ff Clean up separated ban/discourage interface 2020-07-03 20:43:55 -07:00
Pieter Wuille
b691f2df5f Replace automatic bans with discouragement filter
This patch improves performance and resource usage around IP
addresses that are banned for misbehavior. They're already not
actually banned, as connections from them are still allowed,
but they are preferred for eviction if the inbound connection
slots are full.

Stop treating these like manually banned IP ranges, and instead
just keep them in a rolling Bloom filter of misbehaving nodes,
which isn't persisted to disk or exposed through the ban
framework. The effect remains the same: preferred for eviction,
avoided for outgoing connections, and not relayed to other peers.

Also change the name of this mechanism to better reflect reality;
they're not banned, just discouraged.

Contains release notes and several interface improvements by
John Newbery.
2020-07-03 20:43:55 -07:00
Andrew Chow
a66a7a1a70 walletdb: don't reinitialize desc cache with multiple cache entries
When loading descriptor caches, we would accidentally reinitialize the
descriptor cache when seeing that one already exists. This should have
only been initializing the cache when one does not exist. However this
code itself is unnecessary as the act of looking up the cache to add to
it will initialize it if it didn't already exist.

This issue could be hit by trying to load a wallet that had imported a
multisig descriptor. The wallet would fail to load.

A test has been added to wallet_importdescriptors.py to catch this case.
Another test case has also been added to check that loading a wallet
with only single key descriptors works.
2020-07-03 21:15:09 -04:00
Hennadii Stepanov
bd315eb5e2 qt: Get rid of cursor in out-of-focus labels
This change is a temporary fix of QTBUG-59514.
2020-07-03 18:29:24 +03:00
MarcoFalke
fa7592bfa8 rpc: Update server to use new RPCHelpMan
Also, move Check to inside HandleRequest
2020-07-03 11:09:05 -04:00
MarcoFalke
aaaaad5627 rpc: Add option to hide RPCArg
Also, update switch statments to our style guide
2020-07-03 11:08:36 -04:00
MarcoFalke
fa9708f94c rpc: Assert that passed arg names are equal to hardcoded ones 2020-07-03 10:32:36 -04:00
MarcoFalke
3276c148c4 Merge #19424: ci: Run tsan ci config on cirrus
fa8e6df282 ci: Run tsan ci config on cirrus (MarcoFalke)

Pull request description:

  Fixes bitcoin-core/gui#12

  Copied description from #19321:

  Currently it is not possible to use travis in forked repositories due to the 50 minute limit on builds. A fresh build (uncached) of the thread sanitizer config takes more than 50 minutes.

  One approach to fix this could be to throw away tests until the run time is less than 50 minutes. However, the risk of being blind of failures in the thrown away tests is not worth the gain. Also, to detect them, one has to run the tsan configuration nightly and failures could only be detected post-merge.

  Another approach would be to ask travis support to raise the limit for a forked repository. This is a tedious and manual one-by-one process, so I'd rather not.

  Finally, a different ci provider can be used, since the config files are designed to be platform-agnostic. This is what I picked.

  I kept all settings identical to the travis machine for now. Both providers run in the google cloud, so this should be a "move-only".

ACKs for top commit:
  fanquake:
    ACK fa8e6df282 - my understanding is that test coverage remains the same. Just swapping providers to work-around the Travis time-limit in other repos.

Tree-SHA512: 26fed248a4f743107160d3b9e5df57fa0be280fd065ae6fece83d254f59d58ccf3e11a245519d158da109c47b053f62ee8756215008541973c65dc28c4efb748
2020-07-03 07:40:43 -04:00
MarcoFalke
915ac8a861 Merge #19413: refactor: Remove confusing BlockIndex global
fa0dfdf447 refactor: Remove confusing BlockIndex global (MarcoFalke)

Pull request description:

  The global `::BlockIndex()` is problematic for several reasons:

  * It returns a mutable reference to the block tree, without the appropriate lock annotation (`m_block_index` is guarded by `cs_main`). The current code is fine, but in the future this might lead to accidental races and data corruption.
  * The rpc server shouldn't rely on node globals, but rather a context that is passed in to the RPC method.
  * Tests might want to spin up their own block tree, and thus should also not rely on a single global.

  Fix all issues by removing the global

ACKs for top commit:
  promag:
    Code review ACK fa0dfdf447.
  jonatack:
    re-ACK fa0dfdf

Tree-SHA512: 8f158fc5e1c67e73588a21c25677b3fa0fe442313b13ec24b87054806c59607d6ba0c062a865ce3e0ee568706bd0d1faa84febda21aff5bcd65dab172f74c52f
2020-07-03 07:38:16 -04:00
fanquake
f61019f5a2 Merge #19407: doc: explain why passing -mlinker-version is required when cross-compiling
a8d39b8840 doc: explain why passing -mlinker-version is required (fanquake)

Pull request description:

  I have been down a 🐇 hole. Closes #19359.

  When Clang is compiled, [a check is run](https://github.com/llvm/llvm-project/blob/release/8.x/clang/CMakeLists.txt#L353) to define `HOST_LINK_VERSION` as the output of `$CMAKE_LINKER -v`. Note the this is the version of the linker being used to compile Clang itself.. and this check is only run when compiling Clang for macOS.

  In the Clang driver, if `HOST_LINK_VERSION` has been defined, there is some additional runtime functionality. An `-mlinker-version` argument, with the value of `HOST_LINK_VERSION` [will be added to the linker arguments](89de0d8dfb/clang/lib/Driver/Driver.cpp (L382)), if `-mlinker-version` has not been passed in by the user.

  This is a bit weird, as by default, you are setting `-mlinker-version` to the version of the linker that was used to build the Clang binary, not the linker which will be used when compiling. The commit which introduced the functionality, 628fcf4e3b, described it as a "hack", that should be replaced. However, that was 10 years ago, and the behaviour is still here.

  In the Darwin driver, [a check is done](89de0d8dfb/clang/lib/Driver/ToolChains/Darwin.cpp (L208)) for the `-mlinker-version` argument. If there is no argument, the version will default to `0`. Given the above, this should never happen when using Clang for macOS. A series of comparisons are then performed, to check whether the linker version is modern enough to enable certain features, like [`-demangle`](89de0d8dfb/clang/lib/Driver/ToolChains/Darwin.cpp (L215)).

  ### What this means

  #### macOS

  A Clang compiled for macOS, i.e `clang+llvm-8.0.0-x86_64-apple-darwin`, will have `HOST_LINKER_VERSION` set to the version of the linker used to compile Clang itself.

  At runtime, `-mlinker-version=HOST_LINKER_VERSION` will be added to the linker args, if `-mlinker-version` wasn't passed in. In the Darwin driver, additional arguments, like `-demangle`, will be added to the linker arguments, because `HOST_LINKER_VERSION` was likely some very modern version of `lld` or `ld64`.

  #### Linux (cross compilation in depends)

  A Clang compiled for Linux, i.e `clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-14.04`, which we now use for macOS builds in depends, will behave differently. As it's built for Linux, `HOST_LINKER_VERSION` was not defined at compile time, and there will be no default behaviour of appending `-mlinker-version=HOST_LINKER_VERSION` to the linker args. Thus, unless you pass in `-mlinker-version` yourself, when the version checks are done in the Darwin driver, no modern linker features will be enabled, as the version will have defaulted to `0`.

  Therefore, it's important that we continue to pass `-mlinker-version="our LD64 version"` as part of our compilation flags, if we want to have "modern" linker features enabled for our macOS builds.

  #### Summary

  [Clang 8](https://releases.llvm.org/download.html#8.0.0). Building a macOS binary. Link line with path arguments trimmed.

  | | default behaviour | `-mlinker-version=100` (`-demangle threshold`) | `-mlinker-version=530` |
  | - | --------------- | --------------------- | ---------------------- |
  | macOS Clang | `-demangle -lto_library ../libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.15.0 -o a.out ../test-b8b9b3.o -lc++ -lSystem ../libclang_rt.osx.a` | `-demangle -dynamic -arch x86_64 -macosx_version_min 10.15.0 -o a.out ../test-a66966.o -lc++ -lSystem ../libclang_rt.osx.a` | same as default |
  | Linux Clang | `-dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-bfce57.o -lc++ -lSystem` | `-demangle -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-a846a3.o -lc++ -lSystem` | `-demangle -lto_library ../libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-de0280.o -lc++ -lSystem` |

  Note: Most links here are pointing to the 8.x branch of LLVM/Clang, as we are using that version in depends.

  Note: To add a little more confusion, you wont see `-mlinker-version X` in your compile flags, you'll see [`-target-linker-version X`](431daedee4/clang/lib/Driver/ToolChains/Clang.cpp (L4777)).

ACKs for top commit:
  laanwj:
    ACK a8d39b8840

Tree-SHA512: 92f93079a5e59a0d561e74336b5cb03e3bf5a34437f5850283b9128c7624494b8285ec16290b1fa8103fe87f8789a53ce44b17902b8c1db5fde24d74b76fb168
2020-07-03 17:38:27 +08:00
fanquake
7d9008f43e Merge #18307: build: Require pkg-config for all of the hosts
92bc268e4a build: Detect missed pkg-config early (Hennadii Stepanov)
1739eb23d8 build: Drop unused use_pkgconfig variable (Hennadii Stepanov)
a661449a2e build: Drop use_pkgconfig check for libmultiprocess check (Hennadii Stepanov)
90b95e7929 build: Drop dead non-pkg-config code for libevent check (Hennadii Stepanov)
44a14afbb8 build: Drop dead non-pkg-config code for qrencode check (Hennadii Stepanov)
10cbae0c39 build: Drop dead non-pkg-config code for ZMQ check (Hennadii Stepanov)
06cfc9cadf build: Fix indentation in UNIVALUE check (Hennadii Stepanov)
6fd2118e77 build: Drop dead non-pkg-config code for UNIVALUE check (Hennadii Stepanov)
e9edbe4dbd build: Always use pkg-config (Hennadii Stepanov)
9e2e753b06 build: Always define ZMQ_STATIC for MinGW (Hennadii Stepanov)

Pull request description:

  This PR:
  - is based on #18297 (already merged)
  - drops all of the non-pkg-config paths from the `configure` script

  Ref: #17768

ACKs for top commit:
  fanquake:
    ACK 92bc268e4a. I re-gitian-built. There are a couple follow-ups that I'll PR shortly. Thanks for addressing my feedback above. I took too long to get back to this.
  laanwj:
    ACK 92bc268e4a

Tree-SHA512: 83c2d9cf03518867a1ebf7e26a8fc5b6dd8962ef983fe0d84e0c7eb74717f4c36a834da02faf0e503ffd87167005351671cf040c0d4ddae57ee152a6ff84012b
2020-07-03 16:15:52 +08:00
fanquake
daae8b8a1b Merge #18649: tests: Add std::locale::global to list of locale dependent functions in lint-locale-dependence.sh
54b5eb2b14 tests: Add std::locale::global to list of locale dependent functions in lint-locale-dependence.sh (practicalswift)

Pull request description:

  Add `std::locale::global` to list of locale dependent functions in `lint-locale-dependence.sh`.

  We currently flag `setlocale(...)` as locale dependent, but prior to this commit we didn't flag
  `std::locale::global(...)` as such.

  In addition to setting the global C++ locale `std::locale::global(...)` also does the equivalent of `std::setlocale(LC_ALL, ...);`.

  Thus the functionality of `std::locale::global(...)` is a superset of `setlocale(...)` :)

ACKs for top commit:
  MarcoFalke:
    ACK 54b5eb2b14, fine with me

Tree-SHA512: bcf2f1c765add6ed09c3debca968b75eeea81602503f109c0f76ec98635911d453f4834a39e741703c3d470f123178e8952191a9b1a3429394b99c07765dcf1f
2020-07-03 10:42:18 +08:00
Samuel Dobson
a24806c25d Merge #19215: psbt: Include and allow both non_witness_utxo and witness_utxo for segwit inputs
84d295e513 tests: Check that segwit inputs in psbt have both UTXO types (Andrew Chow)
4600479058 psbt: always put a non_witness_utxo and don't remove it (Andrew Chow)
5279d8bc07 psbt: Allow both non_witness_utxo and witness_utxo (Andrew Chow)
72f6bec1da rpc: show both UTXOs in decodepsbt (Andrew Chow)

Pull request description:

  Due to recent changes to hardware wallets, the full previous transaction will need to be provided for segwit inputs. Since some software may be checking for the existence of a `witness_utxo` to determine whether to produce a segwit signature, we keep that field to ease the transition.

  Because all of the sanity checks implemented by the `IsSane` functions were related to having mixed segwit and non-segwit data in a PSBT, those functions are removed as those checks are no longer proper.

  Some tests are updated/removed to accommodate this and a simple test added to check that both UTXOs are being added to segwit inputs.

  As discussed in the wallet IRC meeting, our own signer will not require `non_witness_utxo` for segwit inputs.

ACKs for top commit:
  Sjors:
    utACK 84d295e513 (didn't retest compared to 836d6fc, but fortunately HWI's CI tracks our master branch, with a bunch of hardware wallet simulators)
  ryanofsky:
    Code review re-ACK 84d295e513. No changes since last review, but now I understand the context better. I think it would good to improve the comments as suggested https://github.com/bitcoin/bitcoin/pull/19215#discussion_r447889473 and maybe refer to
  meshcollider:
    utACK 84d295e513

Tree-SHA512: ccc1fd3c16ac3859f5aca4fa489bd40f68be0b81bbdc4dd51188bbf28827a8642dc8b605a37318e5f16cf40f1c4910052dace2f27eca21bb58435f02a443e940
2020-07-03 09:23:22 +12:00
MarcoFalke
fa8e6df282 ci: Run tsan ci config on cirrus 2020-07-02 12:22:39 -04:00
MarcoFalke
7027c67cac Merge #18288: build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory
870f0cd2a0 build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift)

Pull request description:

  Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory.

  First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :)

  Some historical context:
  * 2017: Continuous compilation with Clang Thread Safety analysis enabled (#10866, #10923)
  * 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (#12686)
  * 2018: Continuous testing of use of locale dependent functions (#13041)
  * 2018: Continuous testing of format strings (#13705)
  * 2018: Continuous compilation with MSVC `TreatWarningAsError` (#14151)
  * 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (#14252, #14673, #17006)
  * 2018: Continuous testing under AddressSanitizer – ASan (#14794, #17205, #17674)
  * 2018: Continuous testing under ThreadSanitizer – TSan (#14829)
  * 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (#15134)
  * 2019: Continuous compile-time testing of assumptions we're making (#15391)
  * 2019: Continuous testing of fuzz test cases under Valgrind (#17633, #18159, #18166)
  * 2020: Finally... MemorySanitizer – MSAN! :)

  What is the next step? What tools should we add to CI to keep bugs from entering `master`? :)

ACKs for top commit:
  MarcoFalke:
    ACK 870f0cd2a0

Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578
2020-07-02 10:26:50 -04:00
MarcoFalke
d77170d526 Merge #19422: ci: Add tsan suppression for race in wallet
fa12d8d3ed ci: Add tsan suppression for race in wallet (MarcoFalke)

Pull request description:

  Workaround to fix #19417 (Intermittent CI failure)

Top commit has no ACKs.

Tree-SHA512: 2d68783d6db1bf425ce830cb23eab2f7fa3b9ee18cfb08665e4187196af571547206646dc6dfac0b4444e3dc6c4c13ae45efb09607d2d50df20a3d0a4eec98bd
2020-07-02 10:21:43 -04:00
Wladimir J. van der Laan
7173a3c73b Merge #19396: refactor: Remove confusing OutputType::CHANGE_AUTO
fa927ff884 Enable Wswitch for OutputType (MarcoFalke)
faddad71f6 Remove confusing OutputType::CHANGE_AUTO (MarcoFalke)
fa2eb38352 interfaces: Remove unused getDefaultChangeType (MarcoFalke)

Pull request description:

  `OutputType::CHANGE_AUTO` is problematic for several reasons:

  * An output that is not change must never be described by `CHANGE_AUTO`. Simply allowing that option makes the code confusing and review harder than it needs to be.
  * To make review even harder, `CHANGE_AUTO` requires `-Wswitch` to be disabled for `OutputType`

  Fix both issues by removing `CHANGE_AUTO` and then enabling `-Wswitch` for `OutputType`

ACKs for top commit:
  promag:
    Code review ACK fa927ff884.
  laanwj:
    Code review ACK fa927ff884

Tree-SHA512: 24fd809757aa343866c94dafe9a7130b50cda4f77c97666d407f99b813f75b115a7d8e688a6bc2a737e87cba64ddd4e43f2b3c5538fd35fabb5845807bb39134
2020-07-02 16:10:49 +02:00
Riccardo Spagni
2fc5efc55c Update pruning tooltip, original author BitcoinErrorLog 2020-07-02 15:18:42 +02:00
MarcoFalke
d6fe5b28df Merge #19427: ci: Run all tests on native mac again
fa23fbb42f ci: Run all tests on native mac again (MarcoFalke)

Pull request description:

  They should pass again after f6072e601a

ACKs for top commit:
  practicalswift:
    ACK fa23fbb42f -- Travis is happy and so am I

Tree-SHA512: 49c16b6056d4e67d12a202744e1c56fee2788830213fe4a195955ad44c6b8ecce768a591463ffa0048821959a75b6fad4178629a8866c4a26799c4c8c13e933d
2020-07-01 20:35:10 -04:00
MarcoFalke
fabd33b541 test: Fix intermittent failure in wallet_encryption 2020-07-01 20:08:04 -04:00
MarcoFalke
501203aa91 Merge bitcoin-core/gui#17: doc: Remove outdated comment in TransactionTablePriv
faebb60b8d doc: Remove outdated comment in TransactionTablePriv (MarcoFalke)

Pull request description:

  Locks are no longer taken upfront, so remove the outdated comment

ACKs for top commit:
  hebasto:
    ACK faebb60b8d, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: cd6df24d49d17e58049ac9b261c5e07c8e85ed1aacb547b13c0e55139339d7fcc3b1f766ea2e27d758ea77deadc01f7e28781be1515323c82b9012cee8fd488b
2020-07-01 19:27:56 -04:00
MarcoFalke
01f857a1c9 Merge #19428: Fix boost/C++17 compile failures
fa575f3461 wallet: Replace boost::none with nullopt (MarcoFalke)
fac7bdb75e script: Fix boost/C++17 compile failure (MarcoFalke)

Pull request description:

  Compiling with C++17 enabled, but not the latest version of boost (e.g. 1.69) will result in a compile failure. I know that C++17 is not "officially" supported yet, especially not with all boost versions, since C++17 is meant to replace boost, but some of my systems can no longer compile Bitcoin Core and upstreaming the patches avoids others running into the same issue.

ACKs for top commit:
  sipa:
    utACK fa575f3461

Tree-SHA512: 028e0f0a96c68f6e3394263dd720f0288fff6584592fdf9a7d9551b8358ee64f64b7c5cb802cc866eaa435e0247b66a5a5e54bfdc61a7c9769f287cfd3509966
2020-07-01 19:25:07 -04:00
MarcoFalke
fa927ff884 Enable Wswitch for OutputType 2020-07-01 18:03:12 -04:00
MarcoFalke
faddad71f6 Remove confusing OutputType::CHANGE_AUTO 2020-07-01 18:02:38 -04:00
MarcoFalke
fa575f3461 wallet: Replace boost::none with nullopt 2020-07-01 17:24:49 -04:00
MarcoFalke
fac7bdb75e script: Fix boost/C++17 compile failure
script/standard.cpp:297:48: error: temporary of type 'boost::static_visitor<CScript>' has protected destructor
    return boost::apply_visitor(CScriptVisitor{}, dest);
                                               ^
/usr/include/boost/variant/static_visitor.hpp:53:5: note: declared protected here
    ~static_visitor() = default;
    ^
1 error generated.
2020-07-01 17:24:46 -04:00
MarcoFalke
fa23fbb42f ci: Run all tests on native mac again 2020-07-01 17:00:37 -04:00
James O'Beirne
f19fdd47a6 test: add test for CChainState::ResizeCoinsCaches() 2020-07-01 14:44:28 -04:00
James O'Beirne
8ac3ef4699 add ChainstateManager::MaybeRebalanceCaches()
Aside from in unittests, this method is unused at the moment. It will be used
in upcoming commits that enable utxo snapshot activation.
2020-07-01 14:44:28 -04:00
James O'Beirne
f36aaa6392 Add CChainState::ResizeCoinsCaches
Also adds CCoinsViewCache::ReallocateCache() to attempt to free
memory that the cacheCoins's allocator may be hanging onto when
downsizing the cache.

Adds `CChainState::m_coins{tip,db}_cache_size_bytes` data members
so that we can reference cache size on a per-chainstate basis for
flushing.
2020-07-01 14:44:28 -04:00
James O'Beirne
b223111da2 txdb: add CCoinsViewDB::ChangeCacheSize
We'll need this to dynamically update the cache size of the existing
CCoinsViewDB instance when we create a new one during snapshot activation.

This requires us to keep the CDBWrapper instance as a pointer instead of
a reference so that we're able to destruct it and create a new instance
when the cache size changes.

Also renames `db` to `m_db` since we're already modifying each usage.

Includes feedback from Russ Yanofsky.
2020-07-01 14:44:24 -04:00
Andrew Chow
d8e9ca66d1 walletdb: Move Rewrite into BerkeleyDatabase
Make Rewrite actually a member of BerkeleyDatabase instead of a static
function in BerkeleyBatch
2020-07-01 12:32:11 -04:00
Andrew Chow
91d109156d walletdb: Move PeriodicFlush into WalletDatabase
Make PeriodicFlush a non-static member of WalletDatabase instead of
WalletBatch.
2020-07-01 12:32:06 -04:00
Andrew Chow
8f1bcf8b7b walletdb: Combine VerifyDatabaseFile and VerifyEnvironment
Combine these two functions into a single Verify function that is a
member of WalletDatabase. Additionally, these are no longer static.
2020-07-01 12:32:03 -04:00
Wladimir J. van der Laan
85a68952f8 Merge #19408: doc: Windows WSL build recommendation to temporarily disable Win32 PE support
21a65756f5 Add Windows WSL build recommendation to temporarily disable Win32 PE support. (Aaron Clauson)

Pull request description:

  This is a solution for the issues described in #17277 and #18348

  When cross compiling Bitcoin Code for Windows the `Autoconf` configure scripts attempt to execute Win32 PE files. The configure scripts expect the attempt to fail, however, WSL supports forking the execution of Win32 PE files out to the underlying Windows OS. This can result in the executions failing for unanticipated reasons, which is the case in the two referenced issues.

  This PR adds an explanatory note and additional instructions to temporarily disable WLS's Win32 support.

ACKs for top commit:
  laanwj:
    ACK 21a65756f5

Tree-SHA512: afb014be5a63fa9a827aed30acb2faab15feed34ed89c788a7f6ae6ab1b2238f99e075e6e281d0cc581914db3a4ecc3d5a3d26442f11a520e4e457a40e75e533
2020-07-01 17:24:41 +02:00
Wladimir J. van der Laan
e1b20e2285 Merge #19028: test: Set -logthreadnames in unit tests
99993489da test: Set -logthreadnames in unit tests (MarcoFalke)
fa4ea997b4 init: Setup scheduler in tests and init in exactly the same way (MarcoFalke)

Pull request description:

  Generally the unit tests are single threaded, with the exception of the script check threads, the schedule, and optionally indexer threads.

  Like the functional tests, the thread name can serve additional debug information, so set `-logthreadnames` in unit tests.

  Can be tested with

  ```
  ./src/test/test_bitcoin -l test_suite -t validation_tests/test_combiner_all -- DEBUG_LOG_OUT

ACKs for top commit:
  laanwj:
    ACK 99993489da

Tree-SHA512: 3bdbfc211da146da64b50b0826246aff5c611a84b69ab896a55b3c9d1adc92c5975da36ab92aee577df82e229c4326b477f4105bfdd1a5df4c9a0b018cf61602
2020-07-01 16:54:54 +02:00
Wladimir J. van der Laan
9d92ee12fd Merge #19257: build: remove BIP70 configure option
c4ffcf07af build: remove BIP70 configure option (fanquake)

Pull request description:

  This was left in after #17165, so that anyone who had been compiling
  with (already disabled by default) BIP70 would realise that support
  had been completely removed in 0.20.0. However we should be able to
  remove it for 0.21.0.

ACKs for top commit:
  jnewbery:
    utACK c4ffcf07af
  MarcoFalke:
    ACK c4ffcf07af with or without the "catch-all reject"

Tree-SHA512: a5dd4231ed97c9dd1984fb90d69a8725df2fdda0b963269b0575601c74528e5d820a4a863c428f8ede86eaae2a1606671fe1fcebdeb96b1023f7a5f899270284
2020-07-01 16:31:04 +02:00
Wladimir J. van der Laan
ffa70801da Merge #19256: gui: change combiner for signals to optional_last_value
f1a0314c53 gui: change combiner for signals to optional_last_value (Cory Fields)

Pull request description:

  [`optional_last_value`](https://www.boost.org/doc/libs/1_73_0/doc/html/boost/signals2/optional_last_value.html), which does not throw, has replaced `last_value` as
  Boosts default combiner. Besides being better supported, it also doesn't
  trigger gcc's `-Wmaybe-unitialized` warning, presumably because exceptions no
  longer bubble-up out of signals:

  ```bash
  In file included from ui_interface.cpp:9:
  /bitcoin/depends/x86_64-pc-linux-gnu/share/../include/boost/signals2/last_value.hpp: In member function 'boost::signals2::detail::signal_impl<R(Args ...), Combiner, Group, GroupCompare, SlotFunction, ExtendedSlotFunction, Mutex>::result_type boost::signals2::detail::signal_impl<R(Args ...), Combiner, Group, GroupCompare, SlotFunction, ExtendedSlotFunction, Mutex>::operator()(Args ...) [with Combiner = boost::signals2::last_value<bool>; Group = int; GroupCompare = std::less<int>; SlotFunction = boost::function<bool(const bilingual_str&, const std::__cxx11::basic_string<char>&, unsigned int)>; ExtendedSlotFunction = boost::function<bool(const boost::signals2::connection&, const bilingual_str&, const std::__cxx11::basic_string<char>&, unsigned int)>; Mutex = boost::signals2::mutex; R = bool; Args = {const bilingual_str&, const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned int}]':
  /bitcoin/depends/x86_64-pc-linux-gnu/share/../include/boost/signals2/last_value.hpp:54:36: warning: '*((void*)& value +1)' may be used uninitialized in this function [-Wmaybe-uninitialized]
           if(value) return value.get();
                                      ^
  /bitcoin/depends/x86_64-pc-linux-gnu/share/../include/boost/signals2/last_value.hpp:43:21: note: '*((void*)& value +1)' was declared here
           optional<T> value;
                       ^~~~~
  /bitcoin/depends/x86_64-pc-linux-gnu/share/../include/boost/signals2/last_value.hpp: In member function 'boost::signals2::detail::signal_impl<R(Args ...), Combiner, Group, GroupCompare, SlotFunction, ExtendedSlotFunction, Mutex>::result_type boost::signals2::detail::signal_impl<R(Args ...), Combiner, Group, GroupCompare, SlotFunction, ExtendedSlotFunction, Mutex>::operator()(Args ...) [with Combiner = boost::signals2::last_value<bool>; Group = int; GroupCompare = std::less<int>; SlotFunction = boost::function<bool(const bilingual_str&, const std::__cxx11::basic_string<char>&, const std::__cxx11::basic_string<char>&, unsigned int)>; ExtendedSlotFunction = boost::function<bool(const boost::signals2::connection&, const bilingual_str&, const std::__cxx11::basic_string<char>&, const std::__cxx11::basic_string<char>&, unsigned int)>; Mutex = boost::signals2::mutex; R = bool; Args = {const bilingual_str&, const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, const std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, unsigned int}]':
  /bitcoin/depends/x86_64-pc-linux-gnu/share/../include/boost/signals2/last_value.hpp:54:36: warning: '*((void*)& value +1)' may be used uninitialized in this function [-Wmaybe-uninitialized]
           if(value) return value.get();
                                      ^
  /bitcoin/depends/x86_64-pc-linux-gnu/share/../include/boost/signals2/last_value.hpp:43:21: note: '*((void*)& value +1)' was declared here
           optional<T> value;
                       ^~~~~
  ```

  The change in default happened in [Boost 1.39.0](https://www.boost.org/users/history/version_1_39_0.html) (along with the introduction of the Signals2 library.

  More information is also available here https://www.boost.org/doc/libs/1_73_0/doc/html/signals2/rationale.html#id-1.3.36.9.4:
  > The default combiner for Boost.Signals2 has changed from the last_value combiner used by default in the original Boost.Signals library.
  > This is because last_value requires that at least 1 slot be connected to the signal when it is invoked (except for the last_value<void> specialization).
  >  In a multi-threaded environment where signal invocations and slot connections and disconnections may be happening concurrently, it is difficult to fulfill this requirement. When using optional_last_value, there is no requirement for slots to be connected when a signal is invoked, since in that case the combiner may simply return an empty boost::optional.

ACKs for top commit:
  laanwj:
    ACK f1a0314c53

Tree-SHA512: 3600f85019a3591b141dc9207f8a7e66d16d9996cf97fdf08f5133a212d55c591955ab835ffbdca20b5d62711578bc305d5525c75546fa957f180192e2a80c1e
2020-07-01 16:12:44 +02:00
Wladimir J. van der Laan
26291745ae Merge #19308: wallet: BerkeleyBatch Handle cursor internally
ca24edfbc1 walletdb: Handle cursor internally (Andrew Chow)

Pull request description:

  Instead of returning a Dbc (BDB cursor object) and having the caller deal with the cursor, make BerkeleyBatch handle the cursor internally.

  Split from #18971

ACKs for top commit:
  ryanofsky:
    Code review ACK ca24edfbc1. Changes since last review: StartCursor rename, moving CloseCursor calls near returns
  promag:
    Code review ACK ca24edfbc1.

Tree-SHA512: f029b498c7f275aedca53ce7ade7cb99c82975fd6cad17346a4990fb3bcc54e2a5309b32053bd13def9ee464d331b036ac79abb8fc4fa561170c6cfc85283447
2020-07-01 16:00:32 +02:00
Cory Fields
f1a0314c53 gui: change combiner for signals to optional_last_value
optional_last_value, which does not throw, has replaced optional_value as
boost's default combiner. Besides being better supported, it also doesn't
trigger gcc's -Wmaybe-unitialized warning, presumably because exceptions no
longer bubble-up out of signals:

```bash
boost/signals2/last_value.hpp:54:36: warning: '*((void*)& value +1)' may be used uninitialized in this function [-Wmaybe-uninitialized]
	if(value) return value.get();
```

The change in default happened in Boost 1.39.0 (along with the
introduction of the signals 2 library. More information is available here:

https://www.boost.org/doc/libs/1_73_0/doc/html/signals2/rationale.html#id-1.3.36.9.4

and here:

https://www.boost.org/doc/libs/1_73_0/doc/html/boost/signals2/optional_last_value.html

Co-authored-by: fanquake <fanquake@gmail.com>
2020-07-01 21:40:51 +08:00
Wladimir J. van der Laan
bb588669f9 Merge #19331: build: Do not include server symbols in wallet
faca73000f ci: Install fixed version of clang-format for linters (MarcoFalke)
fa4695da4c build: Sort Makefile.am after renaming file (MarcoFalke)
cccc2784a3 scripted-diff: Move ui_interface to the node lib (MarcoFalke)
fa72ca6a9d qt: Remove unused includes (MarcoFalke)
fac96e6450 wallet: Do not include server symbols (MarcoFalke)
fa0f6c58c1 Revert "Fix link error with --enable-debug" (MarcoFalke)

Pull request description:

  This reverts a hacky workaround from commit b83cc0f, which only happens to work due to compiler optimizations. Then, it actually fixes the linker error.

  The underlying problem is that the wallet includes symbols from the server (ui_interface), which usually results in linker failures. Though, in this specific case the linker failures have not been observed (unless `-O0`) because our compilers were smart enough to strip unused symbols.

  Fix the underlying problem by creating a new header-only with the needed symbol and move ui_interface to node to clarify that this is part of libbitcoin_server.

ACKs for top commit:
  Sjors:
    ACK faca730
  laanwj:
    ACK faca73000f
  hebasto:
    re-ACK faca73000f, since the [previous](https://github.com/bitcoin/bitcoin/pull/19331#pullrequestreview-434420539) review:

Tree-SHA512: e9731f249425aaea50b6db5fc7622e10078cf006721bb87989cac190a2ff224412f6f8a7dd83efd018835302337611f5839e29e15bef366047ed591cef58dfb4
2020-07-01 15:38:18 +02:00
Wladimir J. van der Laan
f6072e601a Merge #19368: test: improve functional tests compatibility with BSD/macOS
3a7e79478a test: retry when write to a socket fails on macOS (Ivan Metlushko)
8cf9d15b82 test: use pgrep for better compatibility (Ivan Metlushko)

Pull request description:

  Rationale: a few minor changes to make experience of running tests on macOS a bit better
  1.`pidof` is not available on BSD/macOS, while `pgrep` is present on BSD, Linux and macOS
  2. Add retry as a workaround for a weird behavior when writing to a socket (https://bugs.python.org/issue33450). Stacktrace attached

  Man pages:
  https://www.freebsd.org/cgi/man.cgi?query=pgrep&apropos=0&sektion=1&manpath=FreeBSD+6.0-RELEASE&arch=default&format=html
  https://man7.org/linux/man-pages/man1/pgrep.1.html

  Related to #19281

  Stacktrace example:
  ```
  ...
  33/161 - feature_abortnode.py failed, Duration: 63 s

  stdout:
  2020-06-11T10:46:43.947000Z TestFramework (INFO): Initializing test directory /var/folders/2q/d5w9zh614r7g5c8r74ln3g400000gq/T/test_runner_₿_🏃_20200611_174102/feature_abortnode_128
  2020-06-11T10:46:45.199000Z TestFramework (INFO): Waiting for crash
  2020-06-11T10:47:15.921000Z TestFramework (INFO): Node crashed - now verifying restart fails
  2020-06-11T10:47:47.068000Z TestFramework (INFO): Stopping nodes
  [node 1] Cleaning up leftover process

  stderr:
  Traceback (most recent call last):
    File "/Users/xxx/Projects/bitcoin/test/functional/feature_abortnode.py", line 50, in <module>
      AbortNodeTest().main()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 142, in main
      exit_code = self.shutdown()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 266, in shutdown
      self.stop_nodes()
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_framework.py", line 515, in stop_nodes
      node.stop_node(wait=wait)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/test_node.py", line 318, in stop_node
      self.stop(wait=wait)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/authproxy.py", line 142, in __call__
      response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File "/Users/xxx/Projects/bitcoin/test/functional/test_framework/authproxy.py", line 107, in _request
      self.__conn.request(method, path, postdata, headers)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1107, in request
      self._send_request(method, url, body, headers)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1152, in _send_request
      self.endheaders(body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 1103, in endheaders
      self._send_output(message_body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 936, in _send_output
      self.send(message_body)
    File "/Users/xxx/.pyenv/versions/3.5.6/lib/python3.5/http/client.py", line 908, in send
      self.sock.sendall(data)
  OSError: [Errno 41] Protocol wrong type for socket
  ```

ACKs for top commit:
  laanwj:
    ACK 3a7e79478a

Tree-SHA512: fefbe40ce94ab29f18bbbed2a434194b1384ffa5279b1d04db7a3708e3dd422bd9e450f1db3f95a1a851fac5a626ab533c6ebcfd7ede96f8ccae9e6f3e9fff92
2020-07-01 15:07:07 +02:00
Wladimir J. van der Laan
e491e55646 Merge #19375: build: target Windows 7 when building libevent and fix ipv6 usage
eb6b73540d build: pass _WIN32_WINNT=0x0601 when building libevent for Windows (fanquake)
03e056edcd depends: Patch libevent build to fix IPv6 -rpcbind on Windows (Luke Dashjr)

Pull request description:

  TLDR: This poaches a commit from #18287 and adds one more to adjust the Windows version targeted when building libevent. These changes combined should fully fix ipv6 usage with the RPC server on Windows.

  ---

  Binding the RPC server to a ipv6 address does not currently work on Windows.
  We currently try and bind to `127.0.0.1` and `::1` [by default](https://github.com/bitcoin/bitcoin/blob/master/src/httpserver.cpp#L304).

  On Windows you'll see lines like this in debug.log:
  ```bash
  2020-06-24T01:49:04Z libevent: getaddrinfo: nodename nor servname provided, or not known
  2020-06-24T01:49:04Z Binding RPC on address ::1 port 8332 failed
  ```

  This issue was bought up in, and supposedly fixed by #18287, however the two people that tested it, both said that it didn't fix the problem. I think I now understand why that change alone is incomplete.

  Our call into libevent starts with [evhttp_bind_socket_with_handle()](https://github.com/bitcoin/bitcoin/blob/master/src/httpserver.cpp#L325):
  ```bash
  evhttp_bind_socket_with_handle()
  	bind_socket()
  		make_addrinfo()
  			evutil_getaddrinfo()
  				if #USE_NATIVE_GETADDRINFO
  					#ifndef AI_ADDRCONFIG
  						evutil_adjust_hints_for_addrconfig_()
  							evutil_check_interfaces()
  								evutil_check_ifaddrs()
  									evutil_found_ifaddr()
  										// miss identifies ipv6 as ipv4?
  					#endif
  					evutil_getaddrinfo_common_()
  ```

  The problem is falling into ["#ifndef AI_ADDRCONFIG"](https://github.com/libevent/libevent/blob/master/evutil.c#L1580):

  ```cpp
  #ifndef AI_ADDRCONFIG
  	/* Not every system has AI_ADDRCONFIG, so fake it. */
  	if (hints.ai_family == PF_UNSPEC &&
  	    (hints.ai_flags & EVUTIL_AI_ADDRCONFIG)) {
  		evutil_adjust_hints_for_addrconfig_(&hints);
  	}
  #endif
  ```

  When this occurs, hints end up being adjusted, and it seems that ipv6 addresses end up being mis-identified as ipv4?

  However this shouldn't happen, as these `AI_` definitions are available on Windows.
  The issue is that in evutil.c, `_WIN32_WINNT` [is set to `0x501`](https://github.com/libevent/libevent/blob/master/evutil.c#L45) (XP).

  This obviously predates Vista (`0x0600`), which is when the `AI_ADDRCONFIG` definition (and others) became [available](https://docs.microsoft.com/en-us/windows/win32/api/ws2def/ns-ws2def-addrinfoa).

  The change here will override libevents internal D_WIN32_WINNT defines. This should be ok, because it's only making "more" of the Windows API available. It's also aligned with what we do in our own configure, we pass [`D_WIN32_WINNT=0x0601`](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L610). We also now use linker flags to restrict our binary from running on a Windows version [earlier than Windows 7](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L621).

  The combined fixes can be tested by running:

  `bitcoind -rpcbind=::1 rpcallowip='0.0.0.0/0' -debug=http`

  and then querying it using:

  `bitcoin-cli -rpcconnect=::1 getblockchaininfo`

  TODO:
  - [x] Open an issue upstream. https://github.com/libevent/libevent/issues/1041

ACKs for top commit:
  laanwj:
    ACK eb6b73540d

Tree-SHA512: e1e50f194911301981edaed0c216ed4efb9ebd4a1f9bc9b9f85bec7140b66c45c8666fd5db4aad359596559d4a08ab7c920e9d9736f3ecdbb841afc54e40586e
2020-07-01 14:56:43 +02:00
MarcoFalke
fa12d8d3ed ci: Add tsan suppression for race in wallet 2020-06-30 19:36:25 -04:00
MarcoFalke
faebb60b8d doc: Remove outdated comment in TransactionTablePriv 2020-06-30 19:29:23 -04:00
MarcoFalke
faaeb2b0b3 rpc: Add CRPCCommand constructor which takes RPCHelpMan
This allows the constructor to ask the rpc manager for the name of the
rpc method or the rpc argument names instead of having it manually
passed in.
2020-06-30 14:11:11 -04:00
fanquake
eb6b73540d build: pass _WIN32_WINNT=0x0601 when building libevent for Windows
This enables of the use of AI_* definitions in the Windows headers,
specifically AI_ADDRCONFIG, which fixes an issue with libevent and
ipv6 on Windows.

It also aligns with what we define in configure when building Core.
2020-06-30 13:25:09 +08:00
MarcoFalke
fa0dfdf447 refactor: Remove confusing BlockIndex global 2020-06-29 20:28:47 -04:00
Cory Fields
1d58cc7cb0 span: add lifetimebound attribute
See http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2018/p0936r0.pdf for
reference.

This helps to guard against dangling references caused by construction from
temporaries such as:

    Span<const int> sp(std::vector<int>{1,2,3});
2020-06-29 15:15:34 -04:00
João Barbosa
76277cc77d qt: Hide peer detail view if multiple are selected 2020-06-29 20:11:18 +01:00
Cory Fields
62733fee87 span: (almost) match std::span's constructor behavior
c++20's draft of std::span no longer includes move constructors.
2020-06-29 13:51:24 -04:00
Peter Bushnell
80968cff68 scripted-diff: rename movie folder to animation
-BEGIN VERIFY SCRIPT-
sed -i -e 's/movies/animation/g' `git grep -l "movies"`
sed -i -e 's/RES_MOVIES/RES_ANIMATION/g' `git grep -l "RES_MOVIES"`
git mv src/qt/res/movies/ src/qt/res/animation
-END VERIFY SCRIPT-
2020-06-29 18:49:13 +01:00
Wladimir J. van der Laan
2af56d6d5c Merge #19399: refactor: Replace RecursiveMutex with Mutex in rpc/server.cpp
6fdfeebcc7 refactor: Replace RecursiveMutex with Mutex in rpc/server.cpp (Hennadii Stepanov)

Pull request description:

  The functions that could lock this mutex, i.e., `SetRPCWarmupStatus()`, `SetRPCWarmupFinished()`, `RPCIsInWarmup()`, `CRPCTable::execute()`, do not call itself recursively, and do not call each other either directly or indirectly. Therefore, the `g_rpc_warmup_mutex` could be a non-recursive mutex.

  Related to #19303.

ACKs for top commit:
  laanwj:
    ACK 6fdfeebcc7
  MarcoFalke:
    ACK 6fdfeebcc7

Tree-SHA512: 05a8ac58c0cd6a3c9afad9e06ad78059642e3e97715e129f379c0bf6dccdb58e70d05d965f23e7432fd3f02d7f97967a778ffb8e424837891d9d785a9e98964c
2020-06-29 19:16:18 +02:00
MarcoFalke
8d3187fd0d Merge #19412: test: move TEST_RUNNER_EXTRA into native tsan setup
a92e48b02d test: move TEST_RUNNER_EXTRA into native tsan setup (fanquake)

Pull request description:

  `feature_block.py` is being run in the tsan job, i.e [here](https://travis-ci.org/github/bitcoin/bitcoin/jobs/703122309), even though it should be excluded. My hasty assumption is that this will fix it. In any case, all other instances of `TEST_RUNNER_EXTRA` seem to have moved out of `.travis.yml` and into the different CI configurations.

ACKs for top commit:
  MarcoFalke:
    ACK a92e48b02d
  practicalswift:
    ACK a92e48b02d -- patch looks correct
  hebasto:
    ACK a92e48b02d, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 86057bef2cc87c6acdbbf94f8cd7a5147510448c3e67aacde8daf247e3ccf649cfc5afbbd10693e084f426042d98150616c0e49bfa5f32b949dff9cebd2fd95d
2020-06-29 12:47:51 -04:00
MarcoFalke
5c3c7cc50c Merge #19300: wallet: Handle concurrent wallet loading
9b009fae6e qa: Test concurrent wallet loading (João Barbosa)
b9971ae585 wallet: Handle concurrent wallet loading (João Barbosa)

Pull request description:

  This PR handles concurrent wallet loading.

  This can be tested by running in parallel the following script a couple of times:
  ```sh
  for i in {1..10}
  do
    src/bitcoin-cli -regtest loadwallet foo
    src/bitcoin-cli -regtest unloadwallet foo
  done
  ```

  Eventually the error occurs:
  ```
  error code: -4
  error message:
  Wallet already being loading.
  ```

  For reference, loading and already loaded wallet gives:
  ```
  error code: -4
  error message:
  Wallet file verification failed. Error loading wallet w1. Duplicate -wallet filename specified.
  ```

  Fixes #19232.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 9b009fae6e I have not reviewed the code
  hebasto:
    ACK 9b009fae6e, tested on Linux Mint 20 (x86_64):
  ryanofsky:
    Code review good-but-not-ideal ACK 9b009fae6e

Tree-SHA512: 0ccd77b03c0926e4c4e51efb31e193b93cb4b9ffe8bac6bb018f7344c55dfd939b873b8cf5e657dca73e6202eb75aa672de2acb787cc133184b0b3b51e47b972
2020-06-29 11:14:26 -04:00
Wladimir J. van der Laan
dbadf746e2 Merge #19333: refactor: Fix clang compile failure
fa3b35a189 ci: Add test for clang-3.8 C++11 support (MarcoFalke)
faa7431fee refactor: Fix clang compile failure (MarcoFalke)

Pull request description:

  Fix

  ```
  script/standard.cpp:278:22: error: default initialization of an object of const type 'const (anonymous namespace)::CScriptVisitor' without a user-provided default constructor
  const CScriptVisitor g_script_visitor;
                       ^
                                       {}
  1 error generated.

ACKs for top commit:
  laanwj:
    ACK fa3b35a189

Tree-SHA512: b3251208945b44530224aadbc10fef1260b479c0b43a5e345501fbfd3579a9fe354b946090e023232852bbb99759da4429b58b137b7b286ddac6bd7960851f7f
2020-06-29 16:50:59 +02:00
fanquake
a92e48b02d test: move TEST_RUNNER_EXTRA into native tsan setup 2020-06-29 22:00:27 +08:00
Wladimir J. van der Laan
1269cab21a Merge #19403: build: improve __builtin_clz* detection
9952242c03 build: improve builtin_clz* detection (fanquake)

Pull request description:

  Fixes #19402.

  The way we currently test for `__builtin_clz*` support with `AC_CHECK_DECLS` does not work with Clang:
  ```bash
  configure:21492: clang++-10 -std=c++11 -c -g -O2  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS conftest.cpp >&5
  conftest.cpp:100:10: error: builtin functions must be directly called
    (void) __builtin_clz;
           ^
  1 error generated.
  ```

  This also removes the `__builtin_clz()` check, as we don't actually use it anywhere, and it's trvial to re-add detection if we do start using it at some point. If this is controversial then I'll add a test for it as well.

ACKs for top commit:
  sipa:
    ACK 9952242c03
  laanwj:
    ACK 9952242c03

Tree-SHA512: 695abb1a694a01a25aaa483b4fffa7d598842f2ba4fe8630fbed9ce5450b915c33bf34bb16ad16a16b702dd7c91ebf49fe509a2498b9e28254fe0ec5177bbac0
2020-06-29 15:49:08 +02:00
MarcoFalke
8edfc1715a Merge #19204: p2p: Reduce inv traffic during IBD
fa525e4d1c net: Avoid wasting inv traffic during IBD (MarcoFalke)
fa06d7e934 refactor: block import implies IsInitialBlockDownload (MarcoFalke)
faba65e696 Add ChainstateManager::ActiveChainstate (MarcoFalke)
fabf3d64ff test: Add FeeFilterRounder test (MarcoFalke)

Pull request description:

  Tx-inv messages are ignored during IBD, so it would be nice if we told peers to not send them in the first place. Do that by sending two `feefilter` messages: One when the connection is made (and the node is in IBD), and another one when the node leaves IBD.

ACKs for top commit:
  jamesob:
    ACK fa525e4d1c ([`jamesob/ackr/19204.1.MarcoFalke.p2p_reduce_inv_traffic_d`](https://github.com/jamesob/bitcoin/tree/ackr/19204.1.MarcoFalke.p2p_reduce_inv_traffic_d))
  naumenkogs:
    utACK fa525e4
  gzhao408:
    ACK fa525e4d1c
  jonatack:
    re-ACK fa525e4 checked diff `git range-diff 19612ca fa8a66c fa525e4`, re-reviewed, ran tests, ran a custom p2p IBD behavior test at 9321e0f223.
  hebasto:
    re-ACK fa525e4d1c, only rebased since the [previous](https://github.com/bitcoin/bitcoin/pull/19204#pullrequestreview-429519667) review (verified with `git range-diff`).

Tree-SHA512: 2c22a5def9822396fca45d808b165b636f1143c4bdb2eaa5c7e977f1f18e8b10c86d4c180da488def38416cf3076a26de15014dfd4d86b2a7e5af88c74afb8eb
2020-06-29 09:45:56 -04:00
MarcoFalke
748178f13e Merge #19394: build: Remove unused RES_IMAGES
53361ddc75 [build] Remove unused RES_IMAGES (Bushstar)

Pull request description:

  Remove RES_IMAGES. Seems to be unused since 2015 in the commit below.

  98c222b5aa (diff-9a4f3a253de77bf90b107bdf5283ebc3R317)

  The src/qt/res/images to which it was used with is no longer present either.

ACKs for top commit:
  hebasto:
    ACK 53361ddc75, tested on Linux Mint 20 (x86_64).

Tree-SHA512: d2f09ae225a4c6c171e1aae4c4a444064dc0502e96130e04ccb718f9fcf611a287c56630dec3e9a8937b5e29040d931a237da36180d2343c23cef30359e46323
2020-06-29 09:39:23 -04:00
Wladimir J. van der Laan
fb87f6d168 Merge #19367: doc: Span pitfalls
fab57e2b9b doc: Mention Span in developer-notes.md (Pieter Wuille)
3502a60418 doc: Document Span pitfalls (Pieter Wuille)

Pull request description:

  This is an attempt to document pitfalls with the use of `Span`, following up on comments like https://github.com/bitcoin/bitcoin/pull/18468#issuecomment-622846597 and https://github.com/bitcoin/bitcoin/pull/18468#discussion_r442998211

ACKs for top commit:
  laanwj:
    ACK fab57e2b9b

Tree-SHA512: 8f6f277d6d88921852334853c2b7ced97e83d3222ce40c9fe12dfef508945f26269b90ae091439ebffddf03f939797cb28126b2387f77959069ef8909c25ab53
2020-06-29 15:18:26 +02:00
Aaron Clauson
21a65756f5 Add Windows WSL build recommendation to temporarily disable Win32 PE support. 2020-06-29 13:25:59 +01:00
fanquake
a8d39b8840 doc: explain why passing -mlinker-version is required 2020-06-29 16:48:59 +08:00
fanquake
19612ca2eb Merge #19371: ci: Increase test timeout for sanitizer configs
fa74a54fad ci: Increase test timeout for sanitizer configs (MarcoFalke)

Pull request description:

  Hopefully fixes #19369

ACKs for top commit:
  practicalswift:
    ACK fa74a54fad -- patch looks correct!
  fanquake:
    ACK fa74a54fad - the test failure here is a different issue, and the problem referenced by this PR hasn't occurred, so I think this can be merged. It's also fixing the use of `--factor` which was replaced in #18986.

Tree-SHA512: bec44fff454f20b7c5f8a461560d2496765dea61186027cc0cdce5ac55be0488b6f7f172fec49b89fe59a75b455501e2b4ae91a98c4a17d5c1a722846d2b3b60
2020-06-29 14:35:20 +08:00
fanquake
b80ae6b4c0 Merge #19344: docs: update testgen usage example
e12e970df6 docs: match usage text to script and location (Peter Bushnell)

Pull request description:

  Update the usage text in the README to match the usage text in the Python script.

  02b26ba1c1/contrib/testgen/gen_key_io_test_vectors.py (L9)
  02b26ba1c1/contrib/testgen/gen_key_io_test_vectors.py (L10)

  Also to match the file names in the actual destination.

  02b26ba1c1/src/test/data/key_io_valid.json
  02b26ba1c1/src/test/data/key_io_invalid.json

  Following the README usage text generates new files when the user is likely to have wanted to update the existing files.

ACKs for top commit:
  fanquake:
    ACK e12e970df6 - this looks correct.

Tree-SHA512: b7ab61e19a54597a8fbd1844b9cfaef78879e53b882eefe4e0140fa115674df7f061e468835186963b89c963244a17d922f2ad0829b10f62b84f02019ee33edb
2020-06-29 14:21:51 +08:00
fanquake
2f54c76050 Merge #19390: doc/REST-interface: Remove stale info
fd9c213c6e doc/REST-interface: Remove stale info (Luke Dashjr)

Pull request description:

  Clean merge to 0.19+

ACKs for top commit:
  fanquake:
    ACK fd9c213c6e
  MarcoFalke:
    ACK fd9c213c6e

Tree-SHA512: ac3ffaa72226380ed8b8ab505518d0dc4350bfcc4625dfd27a2350fbb972a8d2bb4255307926eb331c49232023bcb283a659f0d87e4ecbf654982341242f7d36
2020-06-29 13:32:01 +08:00
fanquake
dd191b5832 Merge #19356: build: Fix search for brew-installed BDB 4 on OS X
8578c6fccd build: Fix search for brew-installed BDB 4 on OS X (Glenn Willen)

Pull request description:

  ~~NOTE: This PR contains one important fix that I need (to make Bitcoin Core build cleanly on my system without shenanigans), plus some related general cleanup that is not really necessary, and could be annoying. (I am prepared to defend my argument that BDB_CFLAGS is wrong here, and BDB_CPPFLAGS is right, but this could bite anybody who has gotten in the habit of -- or scripted -- setting the former.)~~

  Ok, I have been convinced that I was too clever with the refactor and I have removed it. Now it's just the tiny change to fix the build on my local machine.

  ---

  On OS X, when searching Homebrew keg-only packages for BDB 4.8, if we find it,
  use BDB_CPPFLAGS and BDB_LIBS instead of CFLAGS and LIBS for the result. This
  is (1) more correct, and (2) necessary in order to give this location
  priority over other directories in the include search path, which may include
  system include directories with other versions of BDB.

ACKs for top commit:
  theuni:
    ACK 8578c6fccd.

Tree-SHA512: a28f48fc81a25736f7e77c663f21cd9a6ae1cd115682031c5aa695c94cb5afa11920330a60cd6a54832822a2aec1eb23123ac2e2dcd4f0b3835aef9c9339ac97
2020-06-29 13:08:33 +08:00
Bushstar
53361ddc75 [build] Remove unused RES_IMAGES 2020-06-29 05:48:44 +01:00
fanquake
9952242c03 build: improve builtin_clz* detection
The way we currently test with AC_CHECK_DECLS do not work with Clang:
```bash
configure:21492: clang++-10 -std=c++11 -c -g -O2  -DHAVE_BUILD_INFO -D__STDC_FORMAT_MACROS conftest.cpp >&5
conftest.cpp:100:10: error: builtin functions must be directly called
  (void) __builtin_clz;
         ^
1 error generated.
```

This also removes the __builtin_clz() check, as we don't actually use
it anywhere, and it's trvial to re-add detection if we do start using
it at some point.
2020-06-29 11:31:17 +08:00
MarcoFalke
d3a5dbfd1f Merge #19114: scripted-diff: TxoutType C++11 scoped enum class
fa32adf9dc scripted-diff: TxoutType C++11 scoped enum class (MarcoFalke)
fa95a694c4 doc: Update outdated txnouttype documentation (MarcoFalke)
fa58469c77 rpc: Properly use underlying type in GetAllOutputTypes (MarcoFalke)
fa41c65702 rpc: Simplify GetAllOutputTypes with the Join helper (MarcoFalke)

Pull request description:

  Non-scoped enums can accidentally and silently decay into an integral type. Also, the symbol names of the keys are exported to the surrounding (usually global) namespace.

  Fix both issues by switching to an `enum class TxoutType` in a (mostly) scripted-diff.

ACKs for top commit:
  practicalswift:
    ACK fa32adf9dc -- patch looks correct
  hebasto:
    re-ACK fa32adf9dc, since fa5997bd6fc82e16b597ea96e3c5c665f1f174ab (https://github.com/bitcoin/bitcoin/pull/19114#pullrequestreview-421425198) rebased only (verified with `git range-diff`).

Tree-SHA512: f42a9db47f9be89fa4bdd8d2fb05a16726286d8b12e3d87327b67d723f91c7d5a57deb4b2ddae9e1d16fee7a5f8c00828b6dc8909c5db680fc5e0a3cf07cd465
2020-06-28 14:20:00 -04:00
Hennadii Stepanov
6fdfeebcc7 refactor: Replace RecursiveMutex with Mutex in rpc/server.cpp 2020-06-28 10:00:24 +03:00
MarcoFalke
d342a45ca7 Merge #19354: doc: add release note for -getinfo displaying multiwallet balances
6d35d0d18f doc: add release note for -getinfo displaying multiwallet balances (Jon Atack)

Pull request description:

  Release note for #18594. This is one of the commits from #19089, which had one concept ACK and approach ACK since late May. It seems better to submit the changes atomically.

Top commit has no ACKs.

Tree-SHA512: 38616d14b02c39f4ee4b93bf14f72043423cef177b595e85181bc9dc610fbe19d8271f2d2c9e5e17bb46423ffe27746e8e510b13a23ae6fd0e5bc4418a00dafa
2020-06-27 13:29:26 -04:00
MarcoFalke
faca73000f ci: Install fixed version of clang-format for linters 2020-06-27 11:49:44 -04:00
MarcoFalke
fa4695da4c build: Sort Makefile.am after renaming file 2020-06-27 11:49:35 -04:00
MarcoFalke
cccc2784a3 scripted-diff: Move ui_interface to the node lib
-BEGIN VERIFY SCRIPT-

 # Move files
 git mv src/ui_interface.h                                          src/node/ui_interface.h
 git mv src/ui_interface.cpp                                        src/node/ui_interface.cpp
 sed -i -e 's/BITCOIN_UI_INTERFACE_H/BITCOIN_NODE_UI_INTERFACE_H/g' src/node/ui_interface.h

 # Adjust includes and makefile
 sed -i -e 's|ui_interface|node/ui_interface|g' $(git grep -l ui_interface)

 # Sort includes
 git diff -U0 | clang-format-diff -p1 -i -v

-END VERIFY SCRIPT-
2020-06-27 11:49:28 -04:00
MarcoFalke
fa72ca6a9d qt: Remove unused includes 2020-06-27 11:39:23 -04:00
MarcoFalke
fac96e6450 wallet: Do not include server symbols
ui_interface is in libbitcoin_server and can not be included in the
wallet because the wallet does not link with server symbols.
2020-06-27 11:39:09 -04:00
MarcoFalke
fa0f6c58c1 Revert "Fix link error with --enable-debug"
This reverts commit b83cc0fc94.
2020-06-27 11:38:55 -04:00
MarcoFalke
fa2eb38352 interfaces: Remove unused getDefaultChangeType 2020-06-27 10:46:53 -04:00
MarcoFalke
d06cf34656 Merge bitcoin-core/gui#6: Do not truncate node flag strings in debugwindow peers details tab
0ac09c9793 qt: Do not truncate node flag strings in debugwindow.ui peers details tab. (saibato)

Pull request description:

  Fix: When fiddling around with new node flags other than the usual.

  I saw that not all possible node flag strings i.e. the UNKNOWN[..] where
  visible in peers details tab.
  Since v18.2 fixed size was set to 300 and sliding is thereby limited.

  A fix on my old linux cruft and small screen was to set minimumSize width to -1 or 0.
  Qt will then autosize the slider to the max string length.

  Thereby i had full display of all flags inclusive sliding without to fullscreen the window.

  Not sure if this is even an issue for those who can afford big screens or high res macs?
  Feedback welcome.

  BTW: nice side effect now again easy to scroll trough long version names of the node.
  can't wait to see strings like /Satoshi:0.23.99/NOX2NOX4NOX32  or what ever fits in the version string.

ACKs for top commit:
  hebasto:
    ACK 0ac09c9793, tested on Linux Mint 20 (x86_64, Qt 5.12.8).
  promag:
    Tested ACK 0ac09c9793 on macos.

Tree-SHA512: a1601b5e35f10b1fd9407b28142ca00c1b985a822be5d23be4d7d3376211450f06e17f962c44b8b40977f8f8bbbb701cac1c5abb4afb3618da76385dfac848a3
2020-06-27 08:28:22 -04:00
Luke Dashjr
fd9c213c6e doc/REST-interface: Remove stale info 2020-06-26 21:19:15 +00:00
Pieter Wuille
fab57e2b9b doc: Mention Span in developer-notes.md 2020-06-26 13:53:49 -07:00
Pieter Wuille
3502a60418 doc: Document Span pitfalls 2020-06-26 13:49:52 -07:00
MarcoFalke
4946400470 Merge bitcoin-core/gui#8: Fix regression in TransactionTableModel
d906aaa117 qt: Fix regression in TransactionTableModel (Hennadii Stepanov)

Pull request description:

  Since https://github.com/bitcoin/bitcoin/pull/17993 a crash is possible on exit.

  Steps to reproduce:
  - precondition: the old chain
  - start `bitcoin-qt`
  - wait until sync
  - on main window: Menu -> File -> Quit
  - crash

  This PR is based on ryanofsky's [suggestion](https://github.com/bitcoin-core/gui/issues/7#issuecomment-646639251).

  Fixes #7.

ACKs for top commit:
  promag:
    Code review ACK d906aaa117.
  ryanofsky:
    Code review ACK d906aaa117. Only changes are squashing, adding assert and adding const
  vasild:
    ACK d906aaa1

Tree-SHA512: 99a475fd90dff50407a58537fdc6099a2a074018e9078452bf86defc1a4b9e546aa94f916d242355900b21638c6cfef845598a5282661a9343556c4514eb155f
2020-06-26 14:45:28 -04:00
MarcoFalke
3bbd8225b9 Merge #19366: tests: Provide main(...) function in fuzzer. Allow building uninstrumented harnesses with --enable-fuzz.
1087807b2b tests: Provide main(...) function in fuzzer (practicalswift)

Pull request description:

  Provide `main(...)` function in fuzzer. Allow building uninstrumented harnesses with only `--enable-fuzz`.

  This PR restores the behaviour to how things worked prior to #18008. #18008 worked around an macOS specific issue but did it in a way which unnecessarily affected platforms not in need of the workaround :)

  Before this patch:

  ```
  # Build uninstrumented fuzzing harness (no libFuzzer/AFL/other-fuzzer-instrumentation)
  $ ./configure --enable-fuzz
  $ make
    CXXLD    test/fuzz/span
  /usr/lib/gcc/x86_64-linux-gnu/7/../../../x86_64-linux-gnu/Scrt1.o: In function `_start':
  (.text+0x20): undefined reference to `main'
  collect2: error: ld returned 1 exit status
  Makefile:7244: recipe for target 'test/fuzz/span' failed
  make[2]: *** [test/fuzz/span] Error 1
  make[2]: *** Waiting for unfinished jobs....
  $
  ```

  After this patch:

  ```
  # Build uninstrumented fuzzing harness (no libFuzzer/AFL/other-fuzzer-instrumentation)
  $ ./configure --enable-fuzz
  $ make
  $ echo foo | src/test/fuzz/span
  $
  ```

  The examples above show the change in non-macOS functionality. macOS functionality is unaffected by this patch.

ACKs for top commit:
  MarcoFalke:
    ACK 1087807b2b

Tree-SHA512: 9c16ea32ffd378057c4fae9d9124636d11e3769374d340f68a1b761b9e3e3b8a33579e60425293c96b8911405d8b96ac3ed378e669ea4c47836af06892aca73d
2020-06-26 14:38:38 -04:00
MarcoFalke
fa8ec00061 rpc: Check that left section is not multiline 2020-06-26 13:46:29 -04:00
Andrew Chow
79d6332e9e moveonly: Fix indentation in bumpfee RPC
Review this with -w to see that nothing actually changes.
2020-06-25 18:11:05 -04:00
practicalswift
1087807b2b tests: Provide main(...) function in fuzzer 2020-06-25 21:03:27 +00:00
Andrew Chow
431071c28a Hide bumpfee's psbt creation behavior behind -deprecatedrpc
With psbtbumpfee, we can deprecate bumpfee's psbt creation behavior.
So put that behind a -deprecatedrpc
2020-06-25 15:32:11 -04:00
Andrew Chow
4638224f64 Add psbtbumpfee RPC 2020-06-25 15:32:11 -04:00
Hennadii Stepanov
0ecff9dd34 Improve "detected inconsistent lock order" error message 2020-06-25 21:27:34 +03:00
Wladimir J. van der Laan
f32f7e907a Merge #11413: [wallet] [rpc] sendtoaddress/sendmany: Add explicit feerate option
25dac9fa65 doc: add release notes for explicit fee estimators and bumpfee change (Karl-Johan Alm)
05227a3554 tests for bumpfee / estimate_modes (Karl-Johan Alm)
3404c1b753 policy: optional FeeEstimateMode param to CFeeRate::ToString (Karl-Johan Alm)
6fcf448430 rpc/wallet: add two explicit modes to estimate_mode (Karl-Johan Alm)
b188d80c2d MOVEONLY: Make FeeEstimateMode available to CFeeRate (Karl-Johan Alm)
5d1a411eb1 fees: add FeeModes doc helper function (Karl-Johan Alm)
91f6d2bc8f rpc/wallet: add conf_target as alias to confTarget in bumpfee (Karl-Johan Alm)
69158b41fc added CURRENCY_ATOM to express minimum indivisible unit (Karl-Johan Alm)

Pull request description:

  This lets users pick their own fees when using `sendtoaddress`/`sendmany` if they prefer this over the estimators.

ACKs for top commit:
  Sjors:
    re-utACK 25dac9fa65: rebased, more fancy C++,
  jonatack:
    ACK 25dac9fa65 I think this should be merged after all this time, even though it looks to me like there are needed follow-ups, fixes and test coverage to be added (see further down), which I don't mind helping out with, if wanted.
  fjahr:
    Code review ACK 25dac9fa65

Tree-SHA512: f31177e6cabf3187a43cdfe93477144f8e8385c7344613743cbbd16e8490d53ff5144aec7b9de6c9a65eb855b55e0f99d7f164dee4b6bf3cfea4dce51cf11d33
2020-06-25 19:53:42 +02:00
MarcoFalke
910f0468a1 Merge #19348: test: Bump linter versions
39d526bde4 test: Bump linter versions (Duncan Dean)

Pull request description:

  As per #19346, `mypy==0.700` was incompatible with Python 3.8.

  I've bumped the versions of all the linters to their latest stable versions.

  Checked with both Python 3.7 and 3.8 and everything still seems to work fine.

ACKs for top commit:
  hebasto:
    ACK 39d526bde4, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f3ee7fda8095aa25aa68685e863076d52a6b82649770d24b0064d652763c0ceb8ebcbf9024fc74fca45c754e67b2a831dd070b3af23bc099140e6d27e89a5319
2020-06-25 11:08:52 -04:00
practicalswift
cca7c577d5 tests: Add fuzzing harness for ChaCha20Poly1305AEAD 2020-06-25 15:06:13 +00:00
practicalswift
2fc4e5916c tests: Add fuzzing harness for ChaCha20 2020-06-25 15:06:13 +00:00
practicalswift
e9e8aac029 tests: Add fuzzing harness for CHKDF_HMAC_SHA256_L32 2020-06-25 15:06:13 +00:00
practicalswift
ec86ca1aaa tests: Add fuzzing harness for poly1305_auth(...) 2020-06-25 15:06:13 +00:00
practicalswift
4cee53bba7 tests: Add fuzzing harness for AES256CBCEncrypt/AES256CBCDecrypt 2020-06-25 15:06:13 +00:00
practicalswift
9352c32325 tests: Add fuzzing harness for AES256Encrypt/AES256Decrypt 2020-06-25 15:06:13 +00:00
MarcoFalke
c8fa03d176 Merge #19378: refactor: Use Mutex type for g_cs_recent_confirmed_transactions
1307686798 refactor: Use Mutex type for g_cs_recent_confirmed_transactions (Hennadii Stepanov)

Pull request description:

  No need the `RecursiveMutex` type for the `g_cs_recent_confirmed_transactions`.

  Related to #19303.

ACKs for top commit:
  MarcoFalke:
    ACK 1307686798
  vasild:
    ACK 13076867

Tree-SHA512: 67f1be10c80ec18d0f80b9f5036e5a20986314da9b9364ef4e193ad1d9f3f4c8e4c2e16253ca79d649ff602d5b8c2aff58d7dd1085841afb760479a4875cffbe
2020-06-25 09:46:23 -04:00
MarcoFalke
90981b7d68 Merge #19286: tests: Add fuzzing harness for CHash{160,256}, C{HMAC_,}SHA{1,256,512}, CRIPEMD160, CSipHasher, etc.
67bb7be864 tests: Add fuzzing harness for CHash{160,256}, C{HMAC_,}SHA{1,256,512}, CRIPEMD160, CSipHasher, etc. (practicalswift)

Pull request description:

  Add fuzzing harness for `CHash{160,256}`, `C{HMAC_,}SHA{1,256,512}`, `CRIPEMD160`, `CSipHasher`, etc.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: 5377b361097211a7d0b90a26ed1c6dadb9ecce11349036d19f8c9ad2818cd98709bbcbf1c2361dd18eae122b8dbce1c71bb5aa2e85660677e235b8974ae33fcc
2020-06-25 09:35:52 -04:00
MarcoFalke
3a4a3729d9 Merge #19090: refactor: Misc scheduler cleanups
fa8337fcdb clang-format scheduler (MarcoFalke)
fa3d41b5ab doc: Switch scheduler to doxygen comments (MarcoFalke)
fac43f9889 scheduler: Replace stop(true) with StopWhenDrained() (MarcoFalke)
fa9cca0550 doc: Remove unused documentation about unimplemented features (MarcoFalke)
fab2950d70 doc: Switch boost::thread to std::thread in scheduler (MarcoFalke)
fa9819695a test: Remove unused scheduler.h include from the common setup (MarcoFalke)
fa609c4f76 scheduler: Remove unused REVERSE_LOCK (MarcoFalke)

Pull request description:

  This accumulates a bunch of cleanup that was long overdue, but I haven't yet gotten around to address. Specifically, but not limited to:

  * Remove unused code, documentation and includes
  * Upgrade to doxygen documentation

  Please refer to the individual commits for more details.

ACKs for top commit:
  jnewbery:
    utACK fa8337fcdb

Tree-SHA512: 0c825ad9767e2697a3ef1ec1be13fdc2b18eeb7493ad0be5b65cc9f209391e78b17ee66e35e094c5e171c12b0f1624f287a110f6bddaf3024b708877afa8552e
2020-06-25 09:24:58 -04:00
MarcoFalke
ead6d686eb Merge #18912: ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors
3f686d1a28 ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors (practicalswift)

Pull request description:

  Re-introduce the Travis valgrind fuzzing job which was removed by PR #18899. The removal seems to have been made by accident since the removed job does not appear to be the source of the problem the PR set out to fix.

  ---

  Run fuzz testing [test cases (bitcoin-core/qa-assets)](https://github.com/bitcoin-core/qa-assets) under `valgrind`.

  This would have caught `util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value` (#18162) and similar cases.

  This fuzzing job was introduced in #18166.

Top commit has no ACKs.

Tree-SHA512: 6e2681eb0ade6af465c5ea91ac163a337465d2130ec9880ba57a36d9af7c25682734586a32977dc25972d4f78483f339d680ea48c0ae13cf1dfa52b617aae401
2020-06-25 08:58:11 -04:00
MarcoFalke
c9d1040d25 Merge #19237: wallet: Check size after unserializing a pubkey
37ae687f95 Add tests for CPubKey serialization/unserialization (Elichai Turkel)
9b8907fade Check size after Unserializing CPubKey (Elichai Turkel)

Pull request description:

  Found by practicalswift, closes #19235
  Currently all the public API(except the pointer-like API) in CPubKey that sets/constructs a pubkey goes through `CPubKey::Set` which checks if that the length and size match and if not invalidates the key.

  This adds the same check to `CPubKey::Unserialize`, sadly I don't see an easy way to just push this to the existing checks in `CPubKey::Set` but it's only a simple condition.

  The problem with not invalidating is that if you write a pubkey like: `{0x02,0x00}` it will think the actual length is 33(because of `size()`) and will access uninitialized memory if you call any of the functions on CPubKey.

ACKs for top commit:
  practicalswift:
    re-ACK 37ae687f95
  jonatack:
    Code review re-ACK 37ae687 per `git diff eab8ee3 37ae687` only change since last review at eab8ee3 is passing the `pubkey` param by reference to const instead of by value in `src/test/key_tests.cpp::CmpSerializationPubkey`
  MarcoFalke:
    ACK 37ae687f95

Tree-SHA512: 30173755555dfc76d6263fb6a59f41be36049ffae7b4e1b92b922d668f5e5e2331f7374d5fa10d5d59fc53020d2966156905ffcfa8b8129c1f6d0ca062174ff1
2020-06-25 08:07:36 -04:00
Ivan Metlushko
3a7e79478a test: retry when write to a socket fails on macOS
If the socket is tearing down macOS will return EPROTOTYPE instead of EPIPE.
Because python doesn't handle this internally we have to do a workaround and retry the request.
See https://bugs.python.org/issue33450
2020-06-25 17:26:20 +07:00
Hennadii Stepanov
1307686798 refactor: Use Mutex type for g_cs_recent_confirmed_transactions 2020-06-25 10:25:24 +03:00
Luke Dashjr
03e056edcd depends: Patch libevent build to fix IPv6 -rpcbind on Windows
libevent uses getaddrinfo when available, and falls back to gethostbyname
Windows has both, but gethostbyname only supports IPv4
libevent fails to detect Windows's getaddrinfo due to not including the right headers
This patches libevent's configure script to check it correctly
2020-06-25 10:23:54 +08:00
Andrew Chow
84d295e513 tests: Check that segwit inputs in psbt have both UTXO types 2020-06-24 16:32:20 -04:00
Andrew Chow
4600479058 psbt: always put a non_witness_utxo and don't remove it
Offline signers will always need a non_witness_utxo so make sure it is
there.
2020-06-24 16:32:19 -04:00
Andrew Chow
5279d8bc07 psbt: Allow both non_witness_utxo and witness_utxo 2020-06-24 16:31:42 -04:00
Andrew Chow
72f6bec1da rpc: show both UTXOs in decodepsbt 2020-06-24 16:31:42 -04:00
MarcoFalke
67881de0e3 Merge #19272: net, test: invalid p2p messages and test framework improvements
56010f9256 test: hoist p2p values to test framework constants (Jon Atack)
75447f0893 test: improve msg sends and p2p disconnections in p2p_invalid_messages (Jon Atack)
57960192a5 test: refactor test_large_inv() into 3 tests with common method (Jon Atack)
e2b21d8a59 test: add p2p_invalid_messages logging (Jon Atack)
9fa494dc09 net: update misbehavior logging for oversized messages (Jon Atack)

Pull request description:

  ...seen while reviewing #19264, #19252, #19304 and #19107:

  in `net_processing.cpp`
  - make the debug logging for oversized message size misbehavior the same for `addr`, `getdata`, `headers` and `inv` messages

  in `p2p_invalid_messages`
  - add missing logging
  - improve assertions/message sends, move cleanup disconnections outside the assertion scopes
  - split a slowish 3-part test into 3 order-independent tests
  - add a few p2p constants to the test framework

ACKs for top commit:
  troygiorshev:
    reACK 56010f9256
  MarcoFalke:
    ACK 56010f9256 🎛

Tree-SHA512: db67b70278f8d4c318907e105af54b54eb3afd15500f9aa0c98034f6fd4bd1cf9ad1663037bd9b237ff4890f3059b37291a6498d8d6ae2cc38efb9f045f73310
2020-06-24 15:57:34 -04:00
MarcoFalke
532b134cb0 Merge #19373: refactor: Replace HexStr(o.begin(), o.end()) with HexStr(o)
bd93e32292 refactor: Replace HexStr(o.begin(), o.end()) with HexStr(o) (Wladimir J. van der Laan)

Pull request description:

  HexStr can be called with anything that bas `begin()` and `end()` functions, so clean up the redundant calls.

  (context: I tried to convert `HexStr` to use span, but this turns out to be somewhat more involved than I thought, because of the limitation to pre-c++17 Span lacking iterator-based constructor) . This commit is a first step which stands on its own though)

ACKs for top commit:
  jonatack:
    ACK bd93e32292
  troygiorshev:
    ACK bd93e32292
  MarcoFalke:
    review ACK bd93e32292 🔌

Tree-SHA512: 7e4c9d0259b8d23271d233095f1c51db1ee021e865361d74c05c10dd5129aa6d34a243323e2b4596d648e2d7b25c7ebdee37a3e4f99a27883cb4c3cd26432b08
2020-06-24 14:24:14 -04:00
Wladimir J. van der Laan
bd93e32292 refactor: Replace HexStr(o.begin(), o.end()) with HexStr(o)
HexStr can be called with anything that bas `begin()` and `end()` functions,
so clean up the redundant calls.
2020-06-24 18:41:45 +02:00
MarcoFalke
fa74a54fad ci: Increase test timeout for sanitizer configs 2020-06-24 09:17:37 -04:00
Wladimir J. van der Laan
205b87d2f6 Merge #19357: doc: add release note for bitcoin-cli -generate
9886c7d98d doc: add release note for bitcoin-cli -generate (Jon Atack)

Pull request description:

  Adds a release note for #19133.

ACKs for top commit:
  laanwj:
    ACK 9886c7d98d

Tree-SHA512: 1354e5db0098447788c9ded6f2cd868fd6ea4a1443c99bec8026881b962c92b7257bfea45757769071605faeb989e9db48eaa2fbe9d273513aa2905ee479a8ec
2020-06-24 14:21:34 +02:00
MarcoFalke
dae1bd61b2 Merge bitcoin-core/gui#11: Remove needless headers from qt/walletview.cpp
4f9d9efb4e qt: Remove needless headers (Hennadii Stepanov)

Pull request description:

  No symbols from the removed headers are used in the `qt/walletview.cpp`.

  This is a small followup of https://github.com/bitcoin/bitcoin/pull/18027.

Top commit has no ACKs.

Tree-SHA512: 986ed5c8f3bac4c0053736ce84d738f8593d3dbf713109af3cb9b7051cd838f23152a39bb3c1e9694a993c4e7accf14e94e5beff5e7881155638cd44fbf7f46f
2020-06-24 08:18:25 -04:00
Hennadii Stepanov
4f9d9efb4e qt: Remove needless headers 2020-06-24 14:10:01 +03:00
Daniel Kraft
1554b54d47 Static asserts for consistency of fee defaults.
This adds static asserts that ensure that the default values given for
fee levels in the wallet (minimum fee and incremental feerate increase)
are at least as high as the corresponding levels configured in the
core node policy.
2020-06-24 11:44:21 +02:00
Karl-Johan Alm
25dac9fa65 doc: add release notes for explicit fee estimators and bumpfee change 2020-06-24 16:01:38 +09:00
Karl-Johan Alm
05227a3554 tests for bumpfee / estimate_modes
* invalid parameter tests for bumpfee
* add tests for no conf_target explicit estimate_modes
2020-06-24 16:01:38 +09:00
Karl-Johan Alm
3404c1b753 policy: optional FeeEstimateMode param to CFeeRate::ToString 2020-06-24 16:01:38 +09:00
Karl-Johan Alm
6fcf448430 rpc/wallet: add two explicit modes to estimate_mode 2020-06-24 16:01:37 +09:00
Karl-Johan Alm
b188d80c2d MOVEONLY: Make FeeEstimateMode available to CFeeRate
Can verify move-only with:

    git log -p -n1 --color-moved

This commit is move-only and doesn't change code or affect behavior.
2020-06-24 15:52:06 +09:00
Karl-Johan Alm
5d1a411eb1 fees: add FeeModes doc helper function 2020-06-24 15:52:05 +09:00
Glenn Willen
8578c6fccd build: Fix search for brew-installed BDB 4 on OS X
On OS X, when searching Homebrew keg-only packages for BDB 4.8, if we find it,
use BDB_CPPFLAGS and BDB_LIBS instead of CFLAGS and LIBS for the result. This
is (1) more correct, and (2) necessary in order to give this location
priority over other directories in the include search path, which may include
system include directories with other versions of BDB.
2020-06-23 22:04:02 -07:00
Ivan Metlushko
8cf9d15b82 test: use pgrep for better compatibility
pidof is not available on BSD system, while pgrep is present on BSD, Linux and macOS
2020-06-24 09:29:50 +07:00
John Newbery
e8a2822119 [net] Don't try to take cs_inventory before deleting CNode
The TRY_LOCK(cs_inventory) in DisconnectNodes() is taken after the CNode
object has been removed from vNodes and when the CNode's nRefCount is
zero.

The only other places that cs_inventory can be taken are:

- In ProcessMessages() or SendMessages(), when the CNode's nRefCount
must be >0 (see ThreadMessageHandler(), where the refcount is
incremented before calling ProcessMessages() and SendMessages()).
- In a ForEachNode() lambda in PeerLogicValidation::UpdatedBlockTip().
ForEachNode() locks cs_vNodes and calls the function on the CNode
objects in vNodes.

Therefore, cs_inventory is never locked by another thread when the
TRY_LOCK(cs_inventory) is reached in DisconnectNodes(). Since the
only purpose of this TRY_LOCK is to ensure that the lock is not
taken by another thread, this always succeeds. Remove the check.
2020-06-23 08:46:05 -04:00
John Newbery
3556227ddd [net] Make cs_inventory a non-recursive mutex
cs_inventory is never taken recursively. Make it a non-recursive mutex.
2020-06-23 08:46:05 -04:00
John Newbery
344e831de5 [net processing] Remove PushBlockInventory and PushBlockHash
PushBlockInventory() and PushBlockHash() are functions that can
be replaced with single-line statements. This also eliminates
the single place that cs_inventory is taken recursively.
2020-06-23 08:46:05 -04:00
practicalswift
870f0cd2a0 build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory 2020-06-23 09:09:30 +00:00
fanquake
80fd474e40 Merge #19240: build: macOS toolchain simplification and bump
adf543d714 darwin: pass mlinker-version so that clang enables new features (Cory Fields)
2418f739f7 macos: Bump to xcode 11.3.1 and 10.15 SDK (Cory Fields)
5c2c835433 depends: bump MacOS toolchain (Cory Fields)
85b5e42088 contrib: macdeploy: Remove historical extraction notes (Carl Dong)
351beb5c9a contrib: macdeploy: Use apple-sdk-tools instead of xar+pbzx (Carl Dong)
fbcfcf6954 native_cctools: Don't use libc++ from pinned clang (Carl Dong)
3381e4a189 Adapt rest of tooling to new SDK naming scheme (Carl Dong)
b3394ab235 contrib: macdeploy: Correctly generate macOS SDK (Carl Dong)

Pull request description:

  This PR achieves 3 main things:
  1. It simplifies the macOS SDK generation by putting the logic inside a (semi-)portable python3 script `gen-sdk`
  2. It transitions us to using `libc++` headers extracted from the `Xcode.app`, which is more correct as those headers better match the `.tbd` library stubs we use from the `MacOSX.sdk` (located under the same `Xcode.app`). Previously, we used `libc++` headers copied from our downloaded, pinned clang (see `native_cctools.mk`).
  3. It bumps the macOS toolchain in a way that fulfills all of the following constraints:
      1. The new SDK should support compiling with C++17 (our current one doesn't)
      2. The new toolchain should not change our minimum supported macOS version (`-mmacosx-version-min`)
      3. The new toolchain should expect to use a version of `cctools` that is supported by https://github.com/tpoechtrager/cctools-port

  For the constraints in (3), you can reference [this chart](https://en.wikipedia.org/wiki/Xcode#Xcode_7.0_-_11.x_(since_Free_On-Device_Development)) to see that the newest toolchain we can use with our `cctools-port` is `11.3.1`, and the rest of the constraints were tested with local builds.

  #### But [the other Wikipedia chart](https://en.wikipedia.org/wiki/Xcode#Xcode_11.x_(since_SwiftUI_framework)) says that the "min macOS to run" for Xcode 11.3.1 is 10.14.4, doesn't that violate constraint (ii)?

  This confused me at first too, but the "min macOS to run" is for the Xcode.app App itself. The SDK still supports 10.12, as evident in a few plist files and as proven through local builds.

  #### Why bundle all of this together in a single PR?

  We need (1) and (2) together, because if we don't, manually adding the `libc++` headers and writing that out in a `README.md` is going to result in a lot of user error, so it's great to have these together to be more correct and also make it easier on the user at the same time.

  We need (3) together with everything else because bumping (or in the case of (1), renaming) the SDK requires some human coordination and may break some builds. And since it's not that complicated a change, it makes sense to do it together with the rest.

ACKs for top commit:
  theuni:
    ACK adf543d714.
  fanquake:
    ACK adf543d714 - I'll take a look at the linker issue.

Tree-SHA512: 3813b69ebfe9610bee14100f26296fb5438d9bf0dd184ea55e6c38f5ebd94f7c171d98b150fc9e52fde626533f347f7ec51a2b72b79859d946284f578c1084a3
2020-06-23 16:14:49 +08:00
Ben Woosley
57b0c0a93a Drop CADDR_TIME_VERSION checks now that MIN_PEER_PROTO_VERSION is greater 2020-06-23 00:49:50 -07:00
Hennadii Stepanov
92bc268e4a build: Detect missed pkg-config early 2020-06-23 09:02:11 +03:00
Hennadii Stepanov
1739eb23d8 build: Drop unused use_pkgconfig variable 2020-06-23 09:01:47 +03:00
Hennadii Stepanov
a661449a2e build: Drop use_pkgconfig check for libmultiprocess check 2020-06-23 09:00:54 +03:00
Jon Atack
9886c7d98d doc: add release note for bitcoin-cli -generate 2020-06-23 07:09:27 +02:00
Cory Fields
adf543d714 darwin: pass mlinker-version so that clang enables new features
Without this clang fails to add any newly-added linker features.

Removing this in ca5055a5aa was likely a
regression.

See https://github.com/bitcoin/bitcoin/pull/19240#issuecomment-647764049
for more discussion.
2020-06-22 17:00:45 -04:00
Hennadii Stepanov
d906aaa117 qt: Fix regression in TransactionTableModel
Since #17993 a crash is possible on exit.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-06-22 23:43:22 +03:00
Andrew Chow
ca24edfbc1 walletdb: Handle cursor internally
Instead of returning a Dbc (BDB cursor object) and having the caller
deal with the cursor, make BerkeleyBatch handle the cursor internally.

This prepares BerkeleyBatch to work with other database systems as Dbc
objects are BDB specific.
2020-06-22 15:36:23 -04:00
Andrew Chow
3a9aba21a4 Split SetWalletFlags into Add/LoadWalletFlags
Remove memonly bool and follow typical Add and Load pattern used
everywhere else.
2020-06-22 14:59:09 -04:00
Duncan Dean
39d526bde4 test: Bump linter versions
Updates Python linters, spellchecking, and ShellCheck versions. The PR links are updated for
the dependency versions in test/README.md. ShellCheck SC2230 removed to align with with new
behaviour in v0.7.1.

Fixes #19346.
2020-06-22 20:15:53 +02:00
Jon Atack
6d35d0d18f doc: add release note for -getinfo displaying multiwallet balances 2020-06-22 19:26:34 +02:00
Wladimir J. van der Laan
e3fa3c7d67 Merge #19305: doc: add C++17 release note for 0.21.0
f1d21ef1c3 doc: add C++17 release note for 0.21.0 (fanquake)

Pull request description:

  TLDR: Mention that the codebase is now compatible with C++17, and that the
  intention is to require C++17 starting with 0.22.0.

  Following some discussion with Cory/Carl, and in #16684, I think this is the next step in the C++17 migration.

  While #16684 mentions a gitian/Guix release with C++17, it's not yet clear how that would be done. Are we just going to pass `--enable-c++17` in gitian/Guix?. Are we changing our default in configure.ac?

  According to the [last comment](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643778757) in #16684, we wouldn't be changing anything in depends:
  > No, everything (including depends) will stay at C++11.

  However I don't think we want to be mixing C++11 built dependencies, with a C++17 built bitcoind, if there is any potential for compatibility issues.

  Instead, I'd suggest we build the 0.21.0 release as C++11, and do a complete switch to C++17 for 0.22.0. Also, if we actually wanted to use C++17 in depends for 0.21.0, we couldn't without breaking C++11 compat (Qt). See below.

  Here is a potential timeline/TODOs for the migration:

  Potential Timeline
  * 17 / 6 / 2020 - Today
  * Some time prior to split-off:
      * Confirm that compiling with C++17 works.
      * Confirm that C++11 compatibility has not been broken.
  * 1 / 11 / 2020
      * [0.21.0 split off happens](https://github.com/bitcoin/bitcoin/issues/18947).
  * 2 / 11 / 2020
      * Merge an "incompatible with C++11" change into master.
      * Switch configure to use C++17 mode by default.
      * Update minimum compiler requirements. At least:
          * Clang 5: https://clang.llvm.org/cxx_status.html#cxx17
          * GCC 7: https://gcc.gnu.org/projects/cxx-status.html#cxx17
              * While GCC has some support from 5, it seems a more complete support landed in GCC 7.
              * https://gcc.gnu.org/gcc-7/changes.html#cxx
      * Switch depends packages to use C++17 where applicable.
      * Bump Qt from 5.9.x (no c++17 mode) to, likely, 5.15.x (LTS).
      * Drop support for macOS < 10.14.x
          * The c++ dylib shipped with macOS [doesn't support c++17, prior to macOS 10.14](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538).
          * Building Qt 5.12 or 5.15 in C++17 mode will also require a minimum macOS deployment target of 10.14. https://codereview.qt-project.org/c/qt/qtbase/+/283832.
      * Begin merging PRs like #19183 and #19245.
          * I've left some comments in #19183 if the macOS runtime issue interests anyone.
  * 3 / 12 / 2020
      * 0.21.0 released.
      * Built as C++11.
      * Contains warning in release notes that compiling 0.22.0 will require C++17.
  * 3 / 6 / 2021
      * 0.22.0 released.
      * Full of C++17 code.

  One thing worth noting, is that we cannot bump our Qt to a newer LTS for 0.21.0, without breaking C++11 compatibility. Qt 5.12 is not compilable in C++11 mode, as the project has started using C++14 features throughout at least the macOS portions of it's codebase, and seemingly "forgotten" that the release is meant to be C++11 compatible.
  Upstream bug here: https://bugreports.qt.io/browse/QTBUG-77310.
  > Building Qt requires C+11, at a minimum, but in practice we use later features, usually under a feature define, or with a fallback of some kind. On platforms that support > C11, we've (apparently) not considered the fallback necessary, under the assumption C+14 is always available.

ACKs for top commit:
  MarcoFalke:
    ACK f1d21ef1c3 can't hurt to give an advance warning
  Sjors:
    ACK f1d21ef1c3
  laanwj:
    ACK f1d21ef1c3
  theStack:
    ACK f1d21ef1c3

Tree-SHA512: 706baceb07d9584783ba6e437cdf447531c20f586285b9797edc21f3adb1e9d386059d1c543c70eb298d0f8e555dafb6682a55d35c5836979fc12132e8ba06f5
2020-06-22 19:13:22 +02:00
Wladimir J. van der Laan
a6aac20019 Merge #19350: test: Refactor tests using restart_node
20b6e95944 test: refactor functional tests to use restart_node (Christopher Coverdale)

Pull request description:

  fixes #19345

  This PR replaces consecutive calls to `stop_node()` and `start_node()` with `restart_node()` where appropriate in the functional tests.

  The commit messages are repetitive but focused on each file changed with the intention of squashing if applicable.

ACKs for top commit:
  laanwj:
    ACK 20b6e95944

Tree-SHA512: 1cfa1fb8c5f01a7b00fe44e80dbef072147f21e3891098817acd4275b0c5d91dc1c787594209e117edd418f2fa3a7b2dfcbafdf87efc07f740040938d641f3a9
2020-06-22 18:27:28 +02:00
Wladimir J. van der Laan
f591a1a184 Merge #19351: test: add two edge case tests for CSubNet
ccef5d7bf0 test: add two edge case tests for CSubNet (Vasil Dimov)

Pull request description:

  This is chopped off from https://github.com/bitcoin/bitcoin/pull/19031. It is needed because later 19031 modifies the related code and the tests ensure that no surprising changes in behavior sneak in.

ACKs for top commit:
  practicalswift:
    ACK ccef5d7bf0 -- more test coverage is better than less test coverage :)
  laanwj:
    ACK ccef5d7bf0
  hebasto:
    ACK ccef5d7bf0, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 6d386672b6598aeddd33dabe3512e816cf548d5c1af56c4c9e6f897d513b62ba4659cde73405811a0df286ffee3a3f084ab7caf8e3a2086fa9ddecd1bdcb3c67
2020-06-22 18:01:02 +02:00
Hennadii Stepanov
bbe9cf4fe4 test: Improve "potential deadlock detected" exception message 2020-06-22 18:24:29 +03:00
Hennadii Stepanov
35599344c8 Fix mistakenly swapped "previous" and "current" lock orders 2020-06-22 18:12:26 +03:00
Cory Fields
2418f739f7 macos: Bump to xcode 11.3.1 and 10.15 SDK
This gets us a newer SDK with c++17 support and retains 10.12
back-compat.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-06-22 10:14:33 -04:00
Cory Fields
5c2c835433 depends: bump MacOS toolchain
clang   6.0.1  -> 8.0.0
cctools 921    -> 949.0.1
ld64    409.12 -> 530
2020-06-22 10:14:05 -04:00
Carl Dong
85b5e42088 contrib: macdeploy: Remove historical extraction notes 2020-06-22 10:14:03 -04:00
Carl Dong
351beb5c9a contrib: macdeploy: Use apple-sdk-tools instead of xar+pbzx 2020-06-22 10:14:02 -04:00
Carl Dong
fbcfcf6954 native_cctools: Don't use libc++ from pinned clang
Now that we include the macOS SDK libc++ headers in our macOS SDK
tarball, we no longer need this hack to use the libc++ from our pinned
clang.
2020-06-22 10:14:01 -04:00
Carl Dong
3381e4a189 Adapt rest of tooling to new SDK naming scheme 2020-06-22 10:13:55 -04:00
Vasil Dimov
ccef5d7bf0 test: add two edge case tests for CSubNet 2020-06-22 15:30:31 +02:00
Christopher Coverdale
20b6e95944 test: refactor functional tests to use restart_node 2020-06-22 12:58:14 +01:00
Aaron Hook
1cabbddbca refactor: Use uint16_t instead of unsigned short
removed trailing whitespace to make linter happy
2020-06-22 12:12:22 +02:00
saibato
0ac09c9793 qt: Do not truncate node flag strings in debugwindow.ui peers details tab.
Not all possible node flags are visible in details of peers tab since v18.2.
qt will now autoadapt the slider to the full string size.

Signed-off-by: saibato <saibato.naga@pm.me>
2020-06-22 09:44:28 +00:00
Hennadii Stepanov
90b95e7929 build: Drop dead non-pkg-config code for libevent check 2020-06-22 11:31:38 +03:00
Fabian Jahr
40506bf93f test: Test gettxouttsetinfo hash_type option 2020-06-22 01:55:41 +02:00
Fabian Jahr
f17a4d1c4d rpc: Add hash_type NONE to gettxoutsetinfo 2020-06-22 01:55:36 +02:00
Fabian Jahr
a712cf6f68 rpc: gettxoutsetinfo can specify hash_type (only legacy option for now) 2020-06-22 00:55:44 +02:00
jmorgan
c858302280 Change format of log2_work for uniform output (zero-padded) 2020-06-21 17:23:26 -04:00
practicalswift
3f686d1a28 ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors 2020-06-21 21:01:06 +00:00
MarcoFalke
8ef15e8a86 Merge #19198: test: Check that peers with forcerelay permission are not asked to feefilter
fac63eb5ea doc: Remove -whitelistforcerelay from comment (MarcoFalke)
faabd1514f test: Check that peers with forcerelay permission do not get a feefilter message (MarcoFalke)
fad676b8d2 test: Add connect_nodes method (MarcoFalke)
fac6ef4fb2 test: Add test for no net permission (MarcoFalke)
ffff3fe50a test: Replace self.nodes[0].p2p with conn (MarcoFalke)
faccdc8a31 test: remove redundant generate (MarcoFalke)
fab83b934a test: pep-8 p2p_feefilter.py (MarcoFalke)

Pull request description:

ACKs for top commit:
  jonatack:
    re-ACK fac63eb move-only change of two class member functions in test_framework.py and rebases since my review @ faccf0a per `git range-diff 4b5c919 faccf0a fac63eb`. Verified p2p_feefilter and p2p_permissions functional tests are running 🟢 locally.

Tree-SHA512: 30a1c83baee15a4236d127d199c4f264852045372918d5aa5c09ef3d48041762ce3920ff86ef2466d4b2c792ddf56943d12b16c6dce34c6c5aea2a4af2eb4d49
2020-06-21 13:21:00 -04:00
MarcoFalke
fac63eb5ea doc: Remove -whitelistforcerelay from comment
Instead, permission flags should be used. For example
-whitelist=forcerelay@127.0.0.1
2020-06-21 12:18:10 -04:00
MarcoFalke
faabd1514f test: Check that peers with forcerelay permission do not get a feefilter message 2020-06-21 12:17:55 -04:00
MarcoFalke
fad676b8d2 test: Add connect_nodes method 2020-06-21 11:36:07 -04:00
MarcoFalke
fac6ef4fb2 test: Add test for no net permission 2020-06-21 11:35:46 -04:00
MarcoFalke
ffff3fe50a test: Replace self.nodes[0].p2p with conn 2020-06-21 11:35:44 -04:00
MarcoFalke
faccdc8a31 test: remove redundant generate
setup_nodes takes care of getting out of ibd
2020-06-21 11:35:35 -04:00
MarcoFalke
fab83b934a test: pep-8 p2p_feefilter.py 2020-06-21 11:35:25 -04:00
MarcoFalke
4b5c9191e3 Merge #19208: test: move sync_blocks and sync_mempool functions to test_framework.py
cc84460c16 test: move sync_blocks and sync_mempool functions to test_framework.py (Roy Shao)

Pull request description:

  This PR moves `sync_blocks` and `sync_mempool` out from `test_framework/util.py` to `test_framework/test_framework.py` so they can take contextual information of test framework into account.

  * Change all reference callers to call functions from `test_framework.py`
  * Remove `**kwargs` which is not used
  * Take into account of `timeout_factor` when respecting timeout in function implementations.
  * Pass all tests by running `./test/functional/test_runner.py`

  fixes #18930

ACKs for top commit:
  MarcoFalke:
    ACK cc84460c16 , reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space 💫

Tree-SHA512: a79b2a3fa842fc26a7aacb834bb2aea88b3049916c0b754e60002a77ce94bb5954e0ea3b436bf268e9295efb62d721dfef263a09339a55c684ac3fda388c275e
2020-06-21 09:17:39 -04:00
Samuel Dobson
c27330897d Merge #18027: "PSBT Operations" dialog
931dd47608 Make lint-spelling.py happy (Glenn Willen)
11a0ffb29d [gui] Load PSBT from clipboard (Glenn Willen)
a6cb0b0c29 [gui] PSBT Operations Dialog (sign & broadcast) (Glenn Willen)
5dd0c03ffa FillPSBT: report number of inputs signed (or would sign) (Glenn Willen)
9e7b23b733 Improve TransactionErrorString messages. (Glenn Willen)

Pull request description:

  Add a "PSBT Operations" dialog, reached from the "Load PSBT..." menu item, giving options to sign or broadcast the loaded PSBT as appropriate, as well as copying the result to the clipboard or saving it to a file.

  This is based on Sjors' #17509, and depends on that PR going in first. (It effectively replaces the small "load PSBT" dialog from that PR with a more feature-rich one.)

  Some notes:
  * The way I display status information is maybe unusual (a status bar, rather than messageboxes.) I think it's helpful to have the information in it be persistent rather than transitory. But if people dislike it, I would probably move the "current state of the transaction" info to the top line of the main label, and the "what action just happened, and did it succeed" info into a messagebox.
  * I don't really know much about the translation/localization stuff. I put tr() in all the places it seemed like it ought to go. I did not attempt to translate the result of TransactionErrorString (which is shared by GUI and non-GUI code); I don't know if that's correct, but it matches the "error messages in logs should be googleable in English" heuristic. I don't know whether there are things I should be doing to reduce translator effort (like minimizing the total number of distinct message strings I use, or something.)
  * I don't really know how (if?) automated testing is applied to GUI code. I can make a list of PSBTs exercising all the codepaths for manual testing, if that's the right approach. Input appreciated.

ACKs for top commit:
  instagibbs:
    tested ACK 931dd47608
  Sjors:
    re-tACK 931dd47608
  jb55:
    ACK 931dd47608
  achow101:
    ACK 931dd47608

Tree-SHA512: ade52471a2242f839a8bd6a1fd231443cc4b43bb9c1de3fb5ace7c5eb59eca99b1f2e9f17dfdb4b08d84d91f5fd65677db1433dd03eef51c7774963ef4e2e74f
2020-06-21 22:57:33 +12:00
MarcoFalke
fa32adf9dc scripted-diff: TxoutType C++11 scoped enum class
-BEGIN VERIFY SCRIPT-
 # General rename helper: $1 -> $2
 rename_global() { sed -i "s/\<$1\>/$2/g" $(git grep -l "$1"); }

 # Helper to rename TxoutType $1
 rename_value() {
   sed -i "s/    TX_$1,/    $1,/g" src/script/standard.h;  # First strip the prefix in the definition (header)
   rename_global TX_$1 "TxoutType::$1";                    # Then replace globally
 }

 # Change the type globally to bring it in line with the style-guide
 # (clsses are UpperCamelCase)
 rename_global 'enum txnouttype' 'enum class TxoutType'
 rename_global      'txnouttype'            'TxoutType'

 # Now rename each enum value
 rename_value 'NONSTANDARD'
 rename_value 'PUBKEY'
 rename_value 'PUBKEYHASH'
 rename_value 'SCRIPTHASH'
 rename_value 'MULTISIG'
 rename_value 'NULL_DATA'
 rename_value 'WITNESS_V0_KEYHASH'
 rename_value 'WITNESS_V0_SCRIPTHASH'
 rename_value 'WITNESS_UNKNOWN'

-END VERIFY SCRIPT-
2020-06-21 06:41:55 -04:00
MarcoFalke
fa95a694c4 doc: Update outdated txnouttype documentation
Also, remove scope of txnouttype in fuzz tests temporarily. The next
commit will add scopes to all txnouttype.
2020-06-21 06:40:33 -04:00
MarcoFalke
fa58469c77 rpc: Properly use underlying type in GetAllOutputTypes
Don't blindly assume it is int.

In practice this is usually `unsigned` or `int`, so this commit should
not change behavior.
2020-06-21 06:40:26 -04:00
MarcoFalke
fa41c65702 rpc: Simplify GetAllOutputTypes with the Join helper
This commit does not change behavior
2020-06-21 06:40:18 -04:00
Samuel Dobson
47a30ef0c6 Merge #19133: rpc, cli, test: add bitcoin-cli -generate command
22cb303cf0 rpc: add missing space in JSON parsing error message, update test (Jon Atack)
bf53ebef06 test: add multiwallet tests for bitcoin-cli -generate (Jon Atack)
4b859cfff9 cli: add multiwallet capability to GetNewAddress and -generate (Jon Atack)
18f93545a1 test: add tests for bitcoin-cli -generate (Jon Atack)
4818124137 cli: create bitcoin-cli -generate command (Jon Atack)
ff41a36900 cli: extract ParseResult() and ParseError() (Jon Atack)
f4185b26d9 cli: create GenerateToAddressRequestHandler class (Harris)
f7c65a3350 cli: create GetNewAddress() (Jon Atack)
9be7fd35c5 rpc: make generatetoaddress locals const (Jon Atack)
cb00510dba rpc: create rpc/mining.h, hoist default max tries values to constant (Jon Atack)

Pull request description:

  This PR continues and completes the work begun in #17700 working on issue #16000 to create a client-side version of RPC `generate`.

  Basically, `bitcoin-cli -generate` wraps calling `generatenewaddress` followed by `generatetoaddress [nblocks] [maxtries]` and prints the following:

  ```
  $ bitcoin-cli -generate
  {
    "address": "bcrt1qn4aszr2y2xvpa70y675a76wsu70wlkwvdyyln6"
    "blocks": [
      "01d2ebcddf663da90b28da7f6805115e2ba7818f16fe747258836646a43a0bb5",
    ]
  }

  $ bitcoin-cli -rpcwallet=wallet-name -generate 3 100
  {
    "address": "bcrt1q4cunfw0gnsj7g7e6mk0v0uuvvau9mwr09dj45l",
    "blocks": [
      "7a6650ca5e0c614992ee64fb148a7e5e022af842e4b6003f81abd8baf1e75136",
      "01d2ebcddf663da90b28da7f6805115e2ba7818f16fe747258836646a43a0bb5",
      "3f8795ec40b1ad812b818c177680841be319a3f6753d4e32dc7dfb5bafe5d00e"
    ]
  }
  ```

  Help doc:
  ```
  $ bitcoin-cli -h | grep -A5 "\-generate"
    -generate
         Generate blocks immediately, equivalent to RPC generatenewaddress
         followed by RPC generatetoaddress. Optional positional arguments
         are number of blocks to generate (default: 1) and maximum
         iterations to try (default: 1000000), equivalent to RPC
         generatetoaddress nblocks and maxtries arguments. Example:
         bitcoin-cli -generate 4 1000
  ```

  Quite a bit of test coverage turned out to be needed to cover the change and the different cases (arguments, multiwallet mode) and error-handling.

  This PR also improves some things that working on these changes brought to light.

  Credit to Harris Brakmić for the initial work in #17700.

ACKs for top commit:
  adamjonas:
    utACK 22cb303cf0
  meshcollider:
    utACK 22cb303cf0

Tree-SHA512: 94f67f632fe093d076f614e0ecff09ce7342ac6e424579200d5211a6615260e438d857861767fb788950ec6da0b26ef56dc8268c430012a3b3d4822b24ca6fbf
2020-06-21 22:24:07 +12:00
MarcoFalke
fa8337fcdb clang-format scheduler
Can easily be reviewed with the git options
--ignore-all-space --word-diff-regex=. -U0
2020-06-21 06:02:59 -04:00
MarcoFalke
e6f807f87c Merge #19095: [tools] Update clang-format config for multi-line function declarations and calls
cc29d1e2c4 [tools] Update clang-format config (John Newbery)

Pull request description:

  In some cases, running clang-format has made code _less_ readable by joining declarations and calls for functions with many arguments into very long lines. For example:

  ```
  -    size_t getQueueInfo(std::chrono::system_clock::time_point &first,
  -                        std::chrono::system_clock::time_point &last) const;
  +    size_t getQueueInfo(std::chrono::system_clock::time_point& first, std::chrono::system_clock::time_point& last) const;
  ```

  (https://github.com/bitcoin/bitcoin/pull/19090#discussion_r431961148)

  This change to clang-format would allow arguments/parameters for func declarations/calls to be split over multiple lines, aligned with the opening parens. It does not force args/params to be on new lines (that setting is `BinPackParameters : true`).

ACKs for top commit:
  MarcoFalke:
    ACK cc29d1e2c4 fine with me
  practicalswift:
    ACK cc29d1e2c4

Tree-SHA512: a62474925e71aaff41bdce7960fd5ffd64317da810f694d8084080b054708cf71c2ab2ce3111db5a9260d1c1f9e02d59a2ecb5543b1b6172ce085cb42432160a
2020-06-21 05:58:08 -04:00
Peter Bushnell
e12e970df6 docs: match usage text to script and location
Update the usage text in the README to match the usage text in the Python script.

02b26ba1c1/contrib/testgen/gen_key_io_test_vectors.py (L9)

Also to match the file names in the actual destination.

02b26ba1c1/src/test/data/key_io_valid.json
02b26ba1c1/src/test/data/key_io_invalid.json

Following the README usage text generates new files when the user is likely to have wanted to update the existing files.
2020-06-21 10:31:18 +01:00
Samuel Dobson
02b26ba1c1 Merge #19200: rpc: remove deprecated getaddressinfo fields
bc01f7ae05 doc: release note for rpc getaddressinfo removals (Jon Atack)
90e989390e rpc: getaddressinfo RPCResult fixup (Jon Atack)
a8507c99da rpc: remove deprecated getaddressinfo `labels: purpose` (Jon Atack)
645a8653c8 rpc: remove deprecated getaddressinfo `label` field (Jon Atack)

Pull request description:

  These were deprecated in #17578 and #17585, with expected 0.21 removal notified in the 0.20 release notes.
  ```
  - The `getaddressinfo` RPC has had its `label` field deprecated
    (re-enable for this release using the configuration parameter
    `-deprecatedrpc=label`).  The `labels` field is altered from returning
    JSON objects to returning a JSON array of label names (re-enable
    previous behavior for this release using the configuration parameter
    `-deprecatedrpc=labelspurpose`).  Backwards compatibility using the
    deprecated configuration parameters is expected to be dropped in the
    0.21 release.  (#17585, #17578)
  ```

ACKs for top commit:
  Sjors:
    utACK bc01f7a
  adamjonas:
    utACK bc01f7a
  meshcollider:
    utACK bc01f7ae05

Tree-SHA512: ae1af381e32c4c3bde8b061a56382838513a9a82c88767843cdeae3a2ab8aa7d8c2e66e106d2b31ea07d74bb80c191a2f842c9aaecc7c5438ad9a9bc66d1b251
2020-06-21 21:07:00 +12:00
Samuel Dobson
6bb5f6d8e3 Merge #16377: [rpc] don't automatically append inputs in walletcreatefundedpsbt
e5327f947c [rpc] fundrawtransaction: add_inputs option to control automatic input adding (Sjors Provoost)
79804fe24b [rpc] walletcreatefundedpsbt: don't automatically append inputs (Sjors Provoost)

Pull request description:

  When the user doesn't specificy inputs, it makes sense to automatically select them. But when the user does specify inputs, `walletcreatefundedpsbt` now fails if the amount is insufficient, unless `addInputs` is set to `true`.

  Similarly for `fundrawtransaction` if the original transaction already specified inputs, we only add more if `addInputs` is set to `true`.

  This protects against fat finger mistakes in the amount or fee rate (see also #16257). The behavior is also more similar to GUI coin selection.

ACKs for top commit:
  achow101:
    ACK e5327f947c
  meshcollider:
    utACK e5327f947c

Tree-SHA512: d8653b820914396c7c25b0d0a2b7e92de214aa023bc1aa085feb37d3b20fab361ebea90416a7db989f19bdc37e26cf0adfbcb712c80985c87afa67a9bd44fecb
2020-06-21 20:52:34 +12:00
Samuel Dobson
bd331bd745 Merge #17938: Disallow automatic conversion between disparate hash types
4d7369125a Disallow automatic conversion between hash types (Ben Woosley)
fa9ef2cdbe Remove an apparently unnecessary conversion (Ben Woosley)
966a22d859 Explicitly support conversion between equivalent hash types (Ben Woosley)
f32c1e07fd Use explicit conversion from WitnessV0KeyHash -> CKeyID (Ben Woosley)
2c54217f91 Use explicit conversion from PKHash -> CKeyID (Ben Woosley)
a9e451f144 Convert CPubKey to WitnessV0KeyHash directly (Ben Woosley)
3fcc468123 Prefer explicit CScriptID construction (Ben Woosley)
0a5ea32ce6 Prefer explicit uint160 conversion (Ben Woosley)

Pull request description:

  This bases the script/standard hash types, TxDestination-related and CScriptID on a base template which does not silently convert the underlying `uintN` type.

  Inspired by and built on #17924. Commits are small and focused to ease review.

  Note some of these changes may be relative to existing bugs of the same sort as #17924. See particularly "Convert CPubKey to WitnessV0KeyHash directly" and "Remove an apparently unnecessary conversion".

ACKs for top commit:
  achow101:
    ACK 4d7369125a
  meshcollider:
    re-utACK 4d7369125a

Tree-SHA512: f1b3284ddc6fb6c6e726f2c22668b6d732d45eb5418262ed2b9c728f60be7be43dfb414b6ddd9915025c8dcd7f360dc3b46e997a945a2feb95b0e5c4f05d6b54
2020-06-21 20:26:59 +12:00
MarcoFalke
fa8a341b88 wallet: Replace CDataStream& with CDataStream&& where appropriate
The keys and values are only to be used once because their memory is set
to zero. Make that explicit by moving the bytes into the lower level
methods.
2020-06-20 08:42:35 -04:00
MarcoFalke
fa021e9a5b wallet: Remove confusing double return value ret+success
Also, remove redundant comments
2020-06-20 08:41:19 -04:00
MarcoFalke
879acc681a Merge #19018: docs: fixing description of the field sequence in walletcreatefundedpsbt RPC method
d0a3feea73 Change docs for walletcreatefundedpsbt RPC method (Ivan Vershigora)

Pull request description:

  `sequence` field in the list of inputs currently marked as "required". Actually it can be omitted and it's value depends on `locktime` and `options.replaceable` fields. Just the same as in `createpsbt` call.

ACKs for top commit:
  achow101:
    ACK d0a3feea73

Tree-SHA512: 3f429a2c2eea283a47fb5002a99f7e2a5ed6f67df9fd895c1ab938256c48a6497ed6ac2673d8fe8968dfb67b939f4a84570899d9faf52f3abd6ec90c0703d1bd
2020-06-20 07:30:24 -04:00
Karl-Johan Alm
91f6d2bc8f rpc/wallet: add conf_target as alias to confTarget in bumpfee 2020-06-20 15:35:21 +09:00
Karl-Johan Alm
69158b41fc added CURRENCY_ATOM to express minimum indivisible unit
also moved CURRENCY_* into feerate.h file to work around MSVC bug
2020-06-20 15:35:13 +09:00
MarcoFalke
fa3b35a189 ci: Add test for clang-3.8 C++11 support 2020-06-19 20:33:53 -04:00
MarcoFalke
faa7431fee refactor: Fix clang compile failure
script/standard.cpp:278:22: error: default initialization of an object of const type 'const (anonymous namespace)::CScriptVisitor' without a user-provided default constructor
const CScriptVisitor g_script_visitor;
                     ^
                                     {}
1 error generated.
2020-06-19 19:30:21 -04:00
MarcoFalke
d4f9ae0025 Merge #19054: wallet: Skip hdKeypath of 'm' when determining inactive hd seeds
951bca61d7 tests: feature_backwards_compatibility.py test 0.16 up/downgrade (Andrew Chow)
3a03a11e8c Skip hdKeypath of 'm' (Andrew Chow)

Pull request description:

  Previously the seed was stored with keypath 'm' so we need to skip this as well when determining inactive seeds.

  Fixes #19051

ACKs for top commit:
  Sjors:
    ACK 951bca61d7
  instagibbs:
    re-utACK 951bca61d7
  ryanofsky:
    Code review ACK 951bca61d7. No significant changes since last review, just updated comment and some test tweaks

Tree-SHA512: 930f77e7097c9cf4f1012e540bd2b1a72fd279262517f10c1531b2ad48c632ef95e0dd4edea81bcc3b3db306479d34e5e79e5d6c4ed31dfa4b77a4231436436e
2020-06-19 16:14:47 -04:00
Ben Woosley
4d7369125a Disallow automatic conversion between hash types
A templated BaseHash does not allow for automatic conversion, thus
conversions much be explicitly allowed / whitelisted, which will
reduce the risk of unintended conversions.
2020-06-19 12:14:08 -07:00
Ben Woosley
fa9ef2cdbe Remove an apparently unnecessary conversion
CScript -> CScriptID -> ScriptHash is unnecessary because
ScriptHash and CScriptID do the same thing.
2020-06-19 12:14:08 -07:00
Ben Woosley
966a22d859 Explicitly support conversion between equivalent hash types
ScriptHash <-> CScriptID
CKeyID -> PKHash
PKHash -> WitnessV0KeyHash
2020-06-19 12:14:08 -07:00
Ben Woosley
f32c1e07fd Use explicit conversion from WitnessV0KeyHash -> CKeyID
These types are equivalent, in data etc, so they need only their
data cast across.
2020-06-19 12:14:08 -07:00
Ben Woosley
2c54217f91 Use explicit conversion from PKHash -> CKeyID
These types are equivalent, in data etc, so they need only their
data cast across.

Note a function is used rather than a casting
operator as CKeyID is defined at a lower level than script/standard
2020-06-19 12:14:07 -07:00
Ben Woosley
a9e451f144 Convert CPubKey to WitnessV0KeyHash directly
The round-tripping through PKHash has no effect, and is
potentially misleading as such.
2020-06-19 12:14:07 -07:00
Ben Woosley
3fcc468123 Prefer explicit CScriptID construction 2020-06-19 12:14:07 -07:00
Ben Woosley
0a5ea32ce6 Prefer explicit uint160 conversion 2020-06-19 12:14:06 -07:00
MarcoFalke
f3d776b593 Merge #19309: refactor: Fix link error with --enable-debug
b83cc0fc94 Fix link error with --enable-debug (Hennadii Stepanov)

Pull request description:

  Fixes a link error on master (39bd9ddb87):
  ```
  $ ./configure --enable-debug
  $ make
  ...
  bitcoin_wallet-bitcoin-wallet.o:(.data.rel.ro+0x0): undefined reference to `InitError(bilingual_str const&)'
  libbitcoin_wallet_tool.a(libbitcoin_wallet_tool_a-wallettool.o):(.data.rel.ro+0x8): undefined reference to `InitError(bilingual_str const&)'
  libbitcoin_wallet.a(libbitcoin_wallet_a-salvage.o):(.data.rel.ro+0x8): undefined reference to `InitError(bilingual_str const&)'
  libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o):(.data.rel.ro+0x8): undefined reference to `InitError(bilingual_str const&)'
  libbitcoin_wallet.a(libbitcoin_wallet_a-walletdb.o):(.data.rel.ro+0x8): undefined reference to `InitError(bilingual_str const&)'
  libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o):(.data.rel.ro+0x8): more undefined references to `InitError(bilingual_str const&)' follow
  collect2: error: ld returned 1 exit status
  ```

  See:
  - https://github.com/bitcoin/bitcoin/pull/19295#issuecomment-645471771
  - https://github.com/bitcoin/bitcoin/pull/19295#issuecomment-645487182

ACKs for top commit:
  achow101:
    Re-ACK b83cc0fc94

Tree-SHA512: f563d978b6725284049449bb0b3a184d356f32e9b63bcadb0ba71352d3d521af3dbb4a7b4fc0a5a620ed99c357e59f62249c10d0defc0cbe7775f2c06791dabe
2020-06-19 14:38:13 -04:00
Carl Dong
b3394ab235 contrib: macdeploy: Correctly generate macOS SDK
Previously, we did not include the macOS SDK libc++ headers in our SDK
creation process and instead used whichever libc++ headers shipped with
the clang package we downloaded in depends.

This change adds a script (which works on both GNU/Linux and macOS) to
correctly generate the macOS SDK including the libc++ headers. This can
be thought of as a simplified rewrite of tpoechtrager's script:

d3392f4eae/tools/gen_sdk_package.sh

The location within the SDK where we place the libc++ headers is chosen
such that clang's search path detection logic for sysroots would pick up
the headers properly.

We also document this change.
2020-06-19 14:35:48 -04:00
MarcoFalke
6dc1b45087 Merge #19267: ci: Upgrade most ci configs to focal
fa05f44893 ci: Upgrade most ci configs to focal (MarcoFalke)
fad6720891 doc: move doc to ci readme (MarcoFalke)
fa880773b4 ci: Have one config run in xenial to test against python3.5 (MarcoFalke)
fa6ddb2fa1 travis: Always run multiprocess build (MarcoFalke)

Pull request description:

  Generally developers compile with recent compilers, so bumping the ci configs to a recent OS should be uncontroversial. Older OSes (especially with compiler sanitizers) need workarounds that can be dropped by running on a more recent OS.

  This pull changes the asan sanitizer and the experimental multiprocess build to use focal.
  Also, it runs the no_wallet config on xenial to test against python 3.5, according to `doc/dependencies.md`.

  Finally, all configs that mimic gitian (win and mac) will stay at bionic.

ACKs for top commit:
  Sjors:
    ACK fa05f44893, assuming Travis passes
  hebasto:
    ACK fa05f44893

Tree-SHA512: 55ec56c71ba2280d27c1a8856a1e6c310b1fbf469d5a8a1dde228063e3892e1dd1e51408ecff7a3d77ac2ae018daa9e9bbbb60598cdeaab8c32a146b11b3e7c4
2020-06-19 12:53:23 -04:00
MarcoFalke
fa05f44893 ci: Upgrade most ci configs to focal 2020-06-19 10:44:11 -04:00
MarcoFalke
fad6720891 doc: move doc to ci readme 2020-06-19 10:44:00 -04:00
MarcoFalke
fa880773b4 ci: Have one config run in xenial to test against python3.5
Also, bump the travis env to bionic. This shouldn't matter at all
because all ci configs run inside a docker, but it does seem to fix a
bug. See
https://github.com/bitcoin/bitcoin/pull/19267#issuecomment-643630309
2020-06-19 10:43:21 -04:00
MarcoFalke
fa6ddb2fa1 travis: Always run multiprocess build 2020-06-19 10:31:49 -04:00
MarcoFalke
febe5823b4 Merge #19321: ci: Run asan ci config on cirrus
fa2eb3d5d6 ci: Run asan ci config on cirrus (MarcoFalke)
fa93527738 cirrus: Clear dummy task (MarcoFalke)

Pull request description:

  Currently it is not possible to use travis in forked repositories due to the 50 minute limit on builds. A fresh build (uncached) of the address sanitizer config takes more than 50 minutes.

  One approach to fix this could be to throw away tests until the run time is less than 50 minutes. However, the risk of being blind of failures in the thrown away tests is not worth the gain. Also, to detect them, one has to run the asan configuration nightly and failures could only be detected post-merge.

  Another approach would be to ask travis support to raise the limit for a forked repository. This is a tedious and manual one-by-one process, so I'd rather not.

  Finally, a different ci provider can be used, since the config files are designed to be platform-agnostic. This is what I picked.

  I kept all settings identical to the travis machine for now. Both providers run in the google cloud, so this should be a "move-only".

ACKs for top commit:
  hebasto:
    ACK fa2eb3d5d6

Tree-SHA512: 159d7dc6f5b24583e941282cdd40465b15db787f0a658a3e81a7b1a22abdb4cb573709b9b5c4465523e0ba0060b17a68fbdbda7a9ecdeb649f31535d377bbe75
2020-06-19 10:26:37 -04:00
MarcoFalke
fa525e4d1c net: Avoid wasting inv traffic during IBD 2020-06-19 09:27:30 -04:00
MarcoFalke
fa06d7e934 refactor: block import implies IsInitialBlockDownload 2020-06-19 09:27:13 -04:00
MarcoFalke
faba65e696 Add ChainstateManager::ActiveChainstate 2020-06-19 09:27:00 -04:00
MarcoFalke
fabf3d64ff test: Add FeeFilterRounder test 2020-06-19 09:26:59 -04:00
Jon Atack
56010f9256 test: hoist p2p values to test framework constants 2020-06-19 14:14:35 +02:00
Jon Atack
75447f0893 test: improve msg sends and p2p disconnections in p2p_invalid_messages
- call disconnect_p2ps() outside of the assert_debug_log scopes
- send messages directly from the p2p conn rather than via nodes[0].p2p
- add an assertion
2020-06-19 14:14:32 +02:00
Jon Atack
57960192a5 test: refactor test_large_inv() into 3 tests with common method 2020-06-19 14:14:29 +02:00
Jon Atack
e2b21d8a59 test: add p2p_invalid_messages logging 2020-06-19 14:14:25 +02:00
Jon Atack
9fa494dc09 net: update misbehavior logging for oversized messages
so that oversized ADDR, GETDATA, HEADERS and INV messages print
the same consistent debug logs.
2020-06-19 14:14:16 +02:00
Fabian Jahr
605884ef21 refactor: Extract GetBogoSize function 2020-06-19 14:13:08 +02:00
MarcoFalke
5f72ddb7ee Merge #18863: refactor: Make CScriptVisitor stateless
3351c91ed4 refactor: Make CScriptVisitor stateless (João Barbosa)

Pull request description:

  `CScriptVisitor` was added in 1025440184 (#1357) and the visitor return type was never used. Now `CScriptVisitor` is stateless and `CScript` is the return type.

ACKs for top commit:
  MarcoFalke:
    ACK 3351c91ed4 🏤
  sipa:
    utACK 3351c91ed4

Tree-SHA512: d158ad2ebe8ea4dc8cc090b943dd66fa5421a84f9443e16ab2d661df38e1a85de16ff13cbaa56924489d8d43cba25fa3cd8b6904bbbcbf356b886ffe8ffba19a
2020-06-19 07:52:49 -04:00
MarcoFalke
fa3365430c net: Use mockable time for ping/pong, add tests 2020-06-19 07:25:36 -04:00
MarcoFalke
faab4aaf2f util: Add count_microseconds helper 2020-06-19 07:25:35 -04:00
MarcoFalke
62948caf44 Merge #18937: refactor: s/command/msg_type/ in CNetMsgMaker and CSerializedNetMsg
51e9393c1f refactor: s/command/msg_type/ in CNetMsgMaker and CSerializedNetMsg (Sebastian Falbesoner)

Pull request description:

  Follow-up PR for #18533 -- another small step towards getting rid of the confusing "command" terminology. Also see PR #18610 which tackled the functional tests.

ACKs for top commit:
  MarcoFalke:
    ACK 51e9393c1f

Tree-SHA512: bb6f05a7be6823d5c4eab1d05b31fee944e700946827ad9425d59a3957fd879776c88c606319cbe9832d9451b275baedf913b71429ea3e01e4e82bf2d419e819
2020-06-19 06:54:24 -04:00
fanquake
b536813cef build: add -fstack-clash-protection to hardening flags
This option causes the compiler to insert probes whenever stack space
is allocated statically or dynamically to reliably detect stack overflows
and thus mitigate the attack vector that relies on jumping over a stack
guard page as provided by the operating system.

This option is now enabled by default in Ubuntu GCC as of 19.10.

Available in GCC 8 and Clang 11.
2020-06-19 17:20:27 +08:00
fanquake
076183b36b build: add -fcf-protection=full to hardening options
Enables code instrumentation of control-flow transfers. Available in
GCC 8 and Clang 7.

This option is now on by default in Ubuntu GCC as of 19.10.
2020-06-19 17:20:27 +08:00
Glenn Willen
931dd47608 Make lint-spelling.py happy 2020-06-19 02:20:04 -07:00
Glenn Willen
11a0ffb29d [gui] Load PSBT from clipboard 2020-06-19 02:20:04 -07:00
Glenn Willen
a6cb0b0c29 [gui] PSBT Operations Dialog (sign & broadcast)
Add a "PSBT Operations" dialog, reached from the "Load PSBT..." menu
item, giving options to sign or broadcast the loaded PSBT as
appropriate, as well as copying the result to the clipboard or saving
it to a file.
2020-06-19 02:20:04 -07:00
fanquake
c940c1ad85 Merge #19293: net: Avoid redundant and confusing FAILED log
fa1904e5f0 net: Remove dead logging code (MarcoFalke)
fac12ebf4f net: Avoid redundant and confusing FAILED log (MarcoFalke)

Pull request description:

  Remove a redundant and confusing "FAILED" log message and gets rid of the unused return type in `ProcessMessage`

ACKs for top commit:
  jnewbery:
    utACK fa1904e5f0
  gzhao408:
    utACK fa1904e5f0
  troygiorshev:
    ACK fa1904e5f0
  naumenkogs:
    utACK fa1904e

Tree-SHA512: bfa553d5efa022727ed17877fb7c08c14849d804fe6d6a7ce172d513857beba35de41ea40b27ff1aedf68b81e2cda7b2a948ac985fcaaf1b6cfb96cce4837c90
2020-06-19 17:17:29 +08:00
Sjors Provoost
08fc6f6cfc [rpc] refactor: consolidate sendmany and sendtoaddress code
The only new behavior is some error codes are changed from -4 to -6.
2020-06-19 11:17:06 +02:00
fanquake
0101110f9b Merge #19322: [net] split PushInventory()
f52d403b81 [net] split PushInventory() (John Newbery)

Pull request description:

  PushInventory() is currently called with a CInv object, which can be a
  MSG_TX or MSG_BLOCK. PushInventory() only uses the type to determine
  whether to add the hash to setInventoryTxToSend or
  vInventoryBlockToSend.

  Since the caller always knows what type of inventory they're pushing,
  the CInv is wastefully constructed and thrown away, and tx/block relay
  is being split out, we split the function into PushTxInventory() and
  PushBlockInventory().

ACKs for top commit:
  amitiuttarwar:
    utACK f52d403b81. nice cleanup, this has bothered me :)
  naumenkogs:
    utACK f52d403
  sipa:
    utACK f52d403b81

Tree-SHA512: 331495199a3b1a2620e6a62beb336e494291b725d8fd64bb44726c02e80807f3974ff4f329bb0f059088e65cd7d41eff276c1065806d2dd6e72c5a9f368e82cd
2020-06-19 15:41:24 +08:00
Glenn Willen
5dd0c03ffa FillPSBT: report number of inputs signed (or would sign)
In FillPSBT, optionally report the number of inputs we successfully
signed, as an out parameter. If "sign" is false, instead report the
number of inputs for which GetSigningProvider does not return nullptr.
(This is a potentially overbroad estimate of inputs we could sign.)
2020-06-18 23:32:59 -07:00
Glenn Willen
9e7b23b733 Improve TransactionErrorString messages. 2020-06-18 23:32:59 -07:00
fanquake
057bd3189f Merge #19197: init: use std::thread for ThreadImport()
83fd3a6d73 init: use std::thread for ThreadImport() (fanquake)

Pull request description:

  [Mentioned](https://github.com/bitcoin/bitcoin/pull/19142#issuecomment-638090759) in #19142, which removed the `boost::interruption_point()`
  in `ThreadImport()`.

ACKs for top commit:
  hebasto:
    ACK 83fd3a6d73, I have reviewed the code and it looks OK, I agree it can be merged.
  donaloconnor:
    ACK 83fd3a6
  laanwj:
    Code review ACK 83fd3a6d73
  MarcoFalke:
    ACK 83fd3a6d73

Tree-SHA512: 0644947d669feb61eed3a944012dad1bd3dd75cf994aa2630013043c213a335b162b63e20aa37e0997740d8e3a3ec367b660b5196007a09e13f0ac455b36c821
2020-06-19 13:29:03 +08:00
João Barbosa
9b009fae6e qa: Test concurrent wallet loading 2020-06-19 02:09:26 +01:00
João Barbosa
b9971ae585 wallet: Handle concurrent wallet loading 2020-06-19 01:02:28 +01:00
MarcoFalke
dbd7a91fdf Merge #19310: wallet: BerkeleyDatabase make BerkeleyDatabase::Create, CreateMock, and CreateDummy non-static functions
da7a83c5ee Remove WalletDatabase::Create, CreateMock, and CreateDummy (Andrew Chow)
d6045d0ac6 scripted-diff: Replace WalletDatabase::Create* with CreateWalletDatabase (Andrew Chow)
45c08f8a7b Add Create*WalletDatabase functions (Andrew Chow)

Pull request description:

  Instead of having `Create`, `CreateMock`, and `CreateDummy` being static functions in `BerkeleyDatabase`, move these to standalone functions in `walletdb.cpp`. This prepares us for having different `WalletDatabase` classes.

  Part of #18971. This was originally one commit but has been split into 3 to make it (hopefully) easier to review.

ACKs for top commit:
  MarcoFalke:
    ACK da7a83c5ee 🎂
  ryanofsky:
    Code review ACK da7a83c5ee. Easy review, nice scripted-diff

Tree-SHA512: 1feb7cb3889168c555154bf3701a49095fd6b8cab911d44b7f7efbf6fcee2280ccb3d4afec8a83755b39a592ecd13b90a318faa655c321f87bdabdf1e2312327
2020-06-18 16:29:21 -04:00
John Newbery
f52d403b81 [net] split PushInventory()
PushInventory() is currently called with a CInv object, which can be a
MSG_TX or MSG_BLOCK. PushInventory() only uses the type to determine
whether to add the hash to setInventoryTxToSend or
vInventoryBlockToSend.

Since the caller always knows what type of inventory they're pushing,
the CInv is wastefully constructed and thrown away, and tx/block relay
is being split out, we split the function into PushTxInventory() and
PushBlockInventory().
2020-06-18 15:45:48 -04:00
MarcoFalke
fa2eb3d5d6 ci: Run asan ci config on cirrus 2020-06-18 14:33:54 -04:00
MarcoFalke
fa93527738 cirrus: Clear dummy task 2020-06-18 14:33:31 -04:00
MarcoFalke
c2bcb99c1d Merge #19071: doc: Separate repository for the gui
66666d55b1 doc: Mention repo split in the READMEs (MarcoFalke)
faceed753a doc: Add redirect for GUI issues and pull requests (MarcoFalke)

Pull request description:

  ## 🥅 Goals

  Splitting up the GUI (and splitting out modules in general) has been brought up often in recent years. Now that the GUI is primarily connected through (internal) interfaces with the node, it seems an appropriate time to revive this discussion.

  Before looking for solutions, we should define a set of goals that we want to achieve. I will start with some ideas to get started and I hope that others will chime in to share and prioritize their goals.

  ### Separate issue and patch management

  It is currently not possible to subscribe to only a subset of modules in Bitcoin Core, or exclude modules from issue and patch notifications. While it is possible to reactively mute conversations in the stream of all ongoing discussions, there is no way to proactively achieve this. Moreover, the list of open issues and pull request will always include GUI related ones by default. Only with [filters](https://github.com/bitcoin/bitcoin/pulls?q=is%3Aopen+is%3Apr+-label%3AGUI) it is possible to hide them.

  ### More focused review and interests

  Long term goals of the GUI are partially unclear #17395 . Bitcoin Core developers are generally fluent on the command line. Thus, they might not be interested or motivated to review improvements to the GUI, which might not affect their workflow on the command line at all.  Splitting up the GUI will hopefully attract similar minded people to a project whose primary goal is to build and improve the GUI.

  ### Maintain high quality assurance

  The quality of the GUI (and even more importantly Bitcoin Core in general) must not degrade. This means that code review itself is not negatively affected by splitting the GUI, but also the integration of the GUI into the rest of Bitcoin Core. One issue could arise when arbitrary version-combinations are allowed. We are struggling hard to test against all supported versions of Boost. Making the GUI version another dimension is going to make testing impossible.

  ### The GUI *is* Bitcoin Core

  When a user downloads Bitcoin Core from our website (or another package manager) they expect the GUI to be included. This should not change (at least not as a result of splitting up the GUI into another project).

  Similarly, when building Bitcoin Core, the gui should still be built when `--with-gui` is specified.

  ## 🌳 Proposed solution: Monotree

  TLDR. Everything stays the same, the development process for the GUI changes slightly.

  Long version:
  * An exact mirror of the master development branch is hosted at `bitcoin-core/gui`. The new repository is used to track gui-only issues and pull requests. Global changes that happen to touch gui code still go to the *main* repo.
  * All pull requests will be merged into `bitcoin/bitcoin`.
  * Decision making process and maintainers will be identical for both repos.

  ### Disadvantages

  * Review activity might decrease?

  * It doesn't go far enough. bitcoin/bitcoin#3440 is proposing a modularized Bitcoin Core. The GUI could be an "add-on", connected over RPC or capnproto (bitcoin/bitcoin#10102). Thus, the gui could even be hosted as a subtree or completely separate project.

  ### Advantages

  * Review activity might increase? It is impossible to predict the future, but for example the `libsecp256k1` subtree has a lot of domain specific experts, maintainers and reviewers. I think longer term it makes sense to at least try this route for the gui as well.

  * A smaller step is easier to undo when it turns out to come with any unforeseen downsides.

  * No substantial changes to the decision making progress.

  * Nothing changes in how developers set up their dev environment or how users build from the source. Also, the release binaries and process will stay exactly the same. No version drift. Finally, code sharing between the GUI and Bitcoin Core is not made any harder.

  * The organizational side. There are 72 open issues (~14%) and 61 open PRs (~16%) with the GUI label. If moved to its own repo, non-GUI developers wouldn't have to be distracted with GUI-only issues and PRs and GUI enhancements. GUI developers have their own repo to focus on GUI development exclusively.

  ### Implementation (outstanding TODOs)

  * Adjust maintainer merge script https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/57
  * Create bitcoin-core/gui repository (empty or with master branch only)
  * Assign all existing bitcoin core maintainers to the new repo
  * Celebrate? 🥳

  * Long-term: Think how long the grace period is for existing GUI related issues and pull requests. Issues can be transferred with a script after a grace period of some months?

ACKs for top commit:
  fjahr:
    ACK 66666d55b1
  Sjors:
    ACK 66666d55b1
  troygiorshev:
    re-ACK 66666d5
  practicalswift:
    re-ACK 66666d55b1
  hebasto:
    re-ACK 66666d55b1

Tree-SHA512: 2e1a8de945fa6995583059a2e322621763fccce74a869f9aa750f73546b26350487c4acc4222c03cb3ac1f88e80f0b9d9a3a80a200432fee0d785f52c5cb6174
2020-06-18 14:19:26 -04:00
Roy Shao
cc84460c16 test: move sync_blocks and sync_mempool functions to test_framework.py 2020-06-18 13:32:36 -04:00
MarcoFalke
0865a8881d Merge bitcoin-core/gui#3: scripted-diff: Make SeparatorStyle a scoped enum
25f3554351 scripted-diff: Make SeparatorStyle a scoped enum (Hennadii Stepanov)

Pull request description:

  This PR is [split](https://github.com/bitcoin/bitcoin/pull/17877#issuecomment-644751515) from https://github.com/bitcoin/bitcoin/pull/17877 and makes `BitcoinUnits::SeparatorStyle` a scoped enum.

ACKs for top commit:
  MarcoFalke:
    review ACK 25f3554351 🚐

Tree-SHA512: 578f1340a476cf79faa109a83815d3c75e26d9c18873e653d7624b52428ccb2677293116db0a60ae14c949d63b64988fc5a39c7184c2352b87b00e8ddaaaf474
2020-06-18 12:59:28 -04:00
MarcoFalke
c7ebab12f9 Merge #19292: wallet: Refactor BerkeleyBatch Read, Write, Erase, and Exists functions into non-template functions
a389ed52e8 walletdb: refactor Read, Write, Erase, and Exists into non-template func (Andrew Chow)

Pull request description:

  In order to override these later, the specific details of how the Read, Write, Erase, and Exists functions interact with the actual database file need to go into functions that are not templated.

  The functions `ReadKey`, `WriteKey`, `EraseKey`, and `HasKey` are introduced to handle the actual interaction with the database.

  This is mostly a moveonly.

  Based on #19290

ACKs for top commit:
  ryanofsky:
    Code review ACK a389ed52e8. No changes since last review, just non-conflicting rebase
  Sjors:
    utACK a389ed52e8
  MarcoFalke:
    ACK a389ed52e8 🔳

Tree-SHA512: 73bd2fe9ddc4a132d4db6b97e77f5d5f8aa68b8cb25192384f3bacd826365947763a9eee73672331d34578e3f5ade85ee6aa550ff4d89eb62e482250dd5973e4
2020-06-18 10:25:51 -04:00
Hennadii Stepanov
25f3554351 scripted-diff: Make SeparatorStyle a scoped enum
-BEGIN VERIFY SCRIPT-
# General rename helper: $1 -> $2
rename_global() { sed -i "s/\<$1\>/$2/g" $(git grep -l "$1"); }

# Helper to rename SeparatorStyle enumerators
rename_value() {
  sed -i "s/    $1/    $2/g" src/qt/bitcoinunits.h;
  rename_global $1 "SeparatorStyle::$2";
}

rename_global 'enum SeparatorStyle' 'enum class SeparatorStyle'
rename_value 'separatorNever' 'NEVER'
rename_value 'separatorStandard' 'STANDARD'
rename_value 'separatorAlways' 'ALWAYS'
-END VERIFY SCRIPT-
2020-06-18 15:30:15 +03:00
Wladimir J. van der Laan
b8740d6737 Merge #18468: Span improvements
26acc8dd9b Add sanity check asserts to span when -DDEBUG (Pieter Wuille)
2676aeadfa Simplify usage of Span in several places (Pieter Wuille)
ab303a16d1 Add Span constructors for arrays and vectors (Pieter Wuille)
bb3d38fc06 Make pointer-based Span construction safer (Pieter Wuille)
1f790a1147 Make Span size type unsigned (Pieter Wuille)

Pull request description:

  This improves our Span class by making it closer to the C++20 `std::span` one:
  * ~~Support conversion between compatible Spans (e.g. `Span<char>` to `Span<const char>`).~~ (done in #18591)
  * Make the size type `std::size_t` rather than `std::ptrdiff_t` (the C++20 one underwent the same change).
  * Support construction of Spans directly from arrays, `std::string`s, `std::array`s, `std::vector`s, `prevector`s, ... (for all but arrays, this only works for const containers to prevent surprises).

  And then make use of those improvements in various call sites.

  I realize the template magic used looks scary, but it's only needed to make overload resultion make the right choices. Note that the operations done on values are all extremely simple: no casts, explicit conversions, or warning-silencing constructions. That should hopefully make it simpler to review.

ACKs for top commit:
  laanwj:
    Code review ACK 26acc8dd9b
  promag:
    Code review ACK 26acc8dd9b.

Tree-SHA512: 5a5bd346a140edf782b5b3b3f04d9160c7b9e9def35159814a07780ab1dd352545b88d3cc491e0f80d161f829c49ebfb952fddc9180f1a56f1257aa51f38788a
2020-06-18 14:12:21 +02:00
MarcoFalke
343c0bfbf1 Merge #19304: test: Check that message sends successfully when header is split across two buffers
80d4423f99 Test buffered valid message (Troy Giorshev)

Pull request description:

  This PR is a tweak of #19302.  This sends a valid message.

  Additionally, this test includes logging in the same vein as #19272.

ACKs for top commit:
  MarcoFalke:
    tested ACK 80d4423f99 (added an assert(false) to observe deterministic coverage) 🌦
  gzhao408:
    ACK 80d4423f99 👊

Tree-SHA512: 3b1aa5ec480a1661917354788923d64595e2886448c9697ec0606a81293e8b4a4642b2b3cc9afb2206ce6f74e5c6d687308c5ad19cb73c5b354d3071ad8496f8
2020-06-18 07:39:37 -04:00
fanquake
931035b608 Merge #19301: build: don't warn when doxygen isn't found
fa84edb93c build: don't warn when doxygen isn't found (fanquake)

Pull request description:

  Doxygen isn't so important that we need to warn when it is missing. I'd
  assume it might even be missing more often than not for most builds.

ACKs for top commit:
  MarcoFalke:
    Fine with me ACK fa84edb93c
  hebasto:
    ACK fa84edb93c, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 793ebf01a8a5d48b78a70fdef0022633fca59b30074c960ebb21589e3bd98992b8304621a2d999195d12172ed30fe9eefeeb2a952d58853cf58e8d9902b0090c
2020-06-18 13:18:49 +08:00
Pieter Wuille
26acc8dd9b Add sanity check asserts to span when -DDEBUG 2020-06-17 15:10:50 -07:00
Hennadii Stepanov
b83cc0fc94 Fix link error with --enable-debug 2020-06-17 22:31:58 +03:00
Troy Giorshev
80d4423f99 Test buffered valid message
A message can be broken across two buffers, with the split inside its
header.  Usually this will occur when sending many messages, such that
the first buffer fills.

This test uses the RPC to verify that the message is actually being
received in two pieces.

There is a very rare chance of a race condition where the test framework
sends a message in between the two halves of the message under test.  In
this case the peer will almost certainly disconnect and the test will
fail.  An assert has been added to help debugging that rare case.
2020-06-17 15:23:06 -04:00
MarcoFalke
35ed88f187 Merge #19298: test: Add missing sync_blocks
fa195d4eba test: Add missing sync_blocks (MarcoFalke)

Pull request description:

  Bitcoin Core does not sort block and tx announcements for other peers, so generating 100 blocks and then sending out a transaction might reject it if it arrives too early. (non-final)

  Fix that by syncing the blocks first.

  Fix #19265
  Fix #19311

ACKs for top commit:
  Sjors:
    utACK fa195d4eba: sounds plausible

Tree-SHA512: fdc46aed59595e4189509e71bd4a3607a93893933cc01d806cec2ee7701d54d7422c5f22dd83b81ddb021f9113b3119a688fdd8cf8a6474fc12fea422aedd064
2020-06-17 14:45:20 -04:00
Andrew Chow
da7a83c5ee Remove WalletDatabase::Create, CreateMock, and CreateDummy
These are superseded by CreateWalletDatabase, CreateMockWalletDatabase,
and CreateDummyWalletDatabase
2020-06-17 14:13:17 -04:00
Andrew Chow
d6045d0ac6 scripted-diff: Replace WalletDatabase::Create* with CreateWalletDatabase
-BEGIN VERIFY SCRIPT-
sed -i -e 's/WalletDatabase::Create(/CreateWalletDatabase(/g' `git grep -l "WalletDatabase::Create("`
sed -i -e 's/WalletDatabase::CreateDummy(/CreateDummyWalletDatabase(/g' `git grep -l "WalletDatabase::CreateDummy("`
sed -i -e 's/WalletDatabase::CreateMock(/CreateMockWalletDatabase(/g' `git grep -l "WalletDatabase::CreateMock("`
-END VERIFY SCRIPT-
2020-06-17 14:12:41 -04:00
Andrew Chow
45c08f8a7b Add Create*WalletDatabase functions
These functions doing the same things as WalletDatabase::Create,
CreateMock, and CreateDummy
2020-06-17 12:31:29 -04:00
fanquake
f1d21ef1c3 doc: add C++17 release note for 0.21.0
Mention that the codebase is now compatible with C++17, and that the
intention is to require C++17 starting with 0.22.0.
2020-06-17 23:18:38 +08:00
Andrew Chow
a389ed52e8 walletdb: refactor Read, Write, Erase, and Exists into non-template func
In order to override these later, the specific details of how the Read,
Write, Erase, and Exists functions interact with the actual database
file need to go into functions that are not templated.
2020-06-17 10:29:27 -04:00
Elichai Turkel
37ae687f95 Add tests for CPubKey serialization/unserialization 2020-06-17 14:59:36 +03:00
MarcoFalke
39bd9ddb87 Merge #19295: refactor: Use AbortError in FatalError
fa02b47313 refactor: Use AbortError in FatalError (MarcoFalke)

Pull request description:

  `FatalError` has been copied from `AbortNode`, so the two should use the same style to avoid confusion.

  Follow-up to #18927

ACKs for top commit:
  hebasto:
    ACK fa02b47313, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 2cf6d18a6ffb5c2e5cf54f0a072a7cef6dc7e924152b2fee44e6ff2c6c53bad962afd364eda30d8a73883d656429ea68391090e6a27057e69eaefd7c4dad0a33
2020-06-17 06:36:34 -04:00
fanquake
fa84edb93c build: don't warn when doxygen isn't found
Doxygen isn't so important that we need to warn when it is missing. I'd
assume it might even be missing more often than not for most builds.
2020-06-17 18:27:00 +08:00
MarcoFalke
38389dd3a0 Merge #19252: test: wait for disconnect in disconnect_p2ps + bloomfilter test followups
9a40cfc558 [refactor] use waiting inside disconnect_p2ps (gzhao408)
aeb9fb414e [test] wait for disconnect_p2ps to be reflected in getpeerinfo (gzhao408)
e81942d2e1 [test] logging and style followups for bloomfilter tests (gzhao408)

Pull request description:

  Followup to #19083 which adds bloomfilter-related tests.

  1. Make test_node `disconnect_p2ps` wait until disconnection is complete to avoid race conditions (and not place the burden on tests) from MarcoFalke's [comment](https://github.com/bitcoin/bitcoin/pull/19083#discussion_r437383989). And clean up any redundant `wait_until`s in the functional tests.
  2. Clean up style + logging in p2p_filter.py and p2p_nobloomfilter_messages.py and jonatack's other [comments](https://github.com/bitcoin/bitcoin/pull/19083#pullrequestreview-428955784)

ACKs for top commit:
  jonatack:
    Code review ACK 9a40cfc from re-reviewing the diff and `git range-diff 5cafb46 8386ad5 9a40cfc`
  MarcoFalke:
    ACK 9a40cfc558 🐂

Tree-SHA512: 2e14b1c12fc08a355bd5ccad7a2a734a4ccda4bc7dc7bac171cb57359819fc1599d764290729af74832fac3e2be258c5d406c701e78ab6d7262835859b9a7d87
2020-06-17 06:20:10 -04:00
MarcoFalke
9a482d3604 Merge #19249: Add means to handle negative capabilities in the Clang Thread Safety annotations
f8213c05f0 Add means to handle negative capabilities in thread safety annotations (Hennadii Stepanov)

Pull request description:

  This commit is separated from #19238, and it adds support of [Negative Capabilities](https://clang.llvm.org/docs/ThreadSafetyAnalysis.html#negative) in the Clang Thread Safety Analysis attributes.

  > Negative requirements are an alternative `EXCLUDES` [`LOCKS_EXCLUDED`] that provide a stronger safety guarantee. A negative requirement uses the `REQUIRES` [`EXCLUSIVE_LOCKS_REQUIRED`] attribute, in conjunction with the ! operator, to indicate that a capability should not be held.

  Examples of usage:
  - #19238 (for a class)
  - https://github.com/hebasto/bitcoin/tree/200610-addrman-tsn (for the whole code base)

ACKs for top commit:
  MarcoFalke:
    Approach ACK f8213c05f0
  vasild:
    ACK f8213c05

Tree-SHA512: 86d992826b87579661bd228712ae5ee6acca6f70b885ef7e96458974eac184e4874a525c669607ba6b6c861aa4806409a8792d100e6914c858bcab43d31cfb1b
2020-06-17 06:07:37 -04:00
Samuel Dobson
62d863f915 Merge #19290: wallet: move BDB specific classes to bdb.{cpp/h}
61c16339da walletdb: Move BDB specific things into bdb.{cpp/h} (Andrew Chow)
8f033642a8 walletdb: moveonly: Move BerkeleyBatch Cursor and Txn funcs to cpp (Andrew Chow)
25a655794a walletdb: move IsWalletLoaded to walletdb.cpp (Andrew Chow)
f6fc5f3849 walletdb: Add IsBDBWalletLoaded to look for BDB wallets specifically (Andrew Chow)
c3538f435a walletdb: Make SpliWalletFilePath non-static (Andrew Chow)

Pull request description:

  Moves the BDB specific classes from db.{cpp/h} to bdb.{cpp/h}.

  To do this, `SplitWalletFilePath` is first made non-static. Then `IsWalletLoaded` functionality is moved to `IsBDBWalletLoaded` which is called by `IsWalletLoaded`. Then the bulk of db.{cpp/h} is moved to a new file bdb.{cpp/h}.

  While doing some moveonly stuff, an additional commit moves the `*Cursor` and `Txn*` implementations out of the header file and into the cpp file.

  Part of #18971

ACKs for top commit:
  laanwj:
    Code review ACK 61c16339da
  promag:
    Code review ACK 61c16339da.
  meshcollider:
    utACK 61c16339da

Tree-SHA512: cb676cd34c9cd3c838a4fef230d84711efe4cf0d2eefa64ebfd7f787ddc6f7379db0b29454874ddc46ca7ffee0f18f6f3fb96a85513cd10164048948fd03a80c
2020-06-17 21:49:42 +12:00
fanquake
09da0e46bf Merge #19287: contrib: Fix SyntaxWarning in Python base58 implementation
47b49a05ea contrib: Fix SyntaxWarning in Python base58 implementation (Alex Willmer)

Pull request description:

  In Python integers should be compared for equality (`i == j`), not identity (`i is j`). Recent versions of CPython 3.x emit a SyntaxWarning when they encounter this incorrect usage, e.g.

  ```
  $ python3 base58.py
  base58.py:110: SyntaxWarning: "is" with a literal. Did you mean "=="?
    assert get_bcaddress_version('15VjRaDX9zpbA8LVnbrCAFzrVzN7ixHNsC') is 0
  Tests passed
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 47b49a05ea

Tree-SHA512: 9f8962025dcdfa062c0515c68a1864f5bbeb86bd0510c0ec0e413a5edb6afbfd5f41b4c0255784e53db8eaf39c68b7cfa7cc8a33a2e5214aae463fda374f8719
2020-06-17 17:05:15 +08:00
fanquake
3faf3429e9 Merge #19299: refactor: Remove unused vars, Add missing includes
fa193c6b1b Add missing includes to fix compile errors (MarcoFalke)
fa09ec83f3 Remove unused variables (MarcoFalke)

Pull request description:

  This is required for #19183, but seems like good cleanup that can go in upfront.

ACKs for top commit:
  practicalswift:
    ACK fa193c6b1b -- patch looks correct
  hebasto:
    ACK fa193c6b1b, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 79b94e7f7ee3a1a8a8fb2ea1ecdf61f130f8b133a37865894da3dbbbf311979e7d1fc013b923fdd7dbf19a221e0232f664defbdb57aa44e0b8c45bfff3c71dcb
2020-06-17 16:51:06 +08:00
MarcoFalke
5cafb46fef Merge #19294: test: refactor: Inline adjust_bitcoin_conf_for_pre_17
fa41b0a6da pep-8 test/functional/test_framework/util.py (MarcoFalke)
faa841bc97 test: refactor: Inline adjust_bitcoin_conf_for_pre_17 (MarcoFalke)

Pull request description:

  This removes mental and code complexity as well as attack surface for bikeshedding

ACKs for top commit:
  Sjors:
    utACK fa41b0a6da

Tree-SHA512: 6e3c872e66d98ffaa7aecdfd64aa7dd8fbb51815a8fdaba170ce0772b4c3360084d0ebab4a5feac768ab5df50d04528d7daafc51ba07c15445c1ef94fa3efd34
2020-06-16 16:28:14 -04:00
MarcoFalke
fa193c6b1b Add missing includes to fix compile errors
fs.cpp:35:17: error: no member named 'strerror' in namespace 'std'
    return std::strerror(errno);
           ~~~~~^
fs.cpp:49:9: error: use of undeclared identifier 'close'
        close(fd);
        ^
2 errors generated.

./interfaces/chain.h:265:55: error: ‘std::function’ has not been declared
     virtual void rpcRunLater(const std::string& name, std::function<void()> fn, int64_t seconds) = 0;
                                                       ^~~
2020-06-16 15:15:46 -04:00
MarcoFalke
fa09ec83f3 Remove unused variables 2020-06-16 15:14:55 -04:00
MarcoFalke
fa195d4eba test: Add missing sync_blocks 2020-06-16 15:05:05 -04:00
MarcoFalke
23b2a68df5 Merge #18275: wallet: error if an explicit fee rate was given but the needed fee rate differed
44cc75f80e wallet: error if an explicit fee rate was given but the needed fee rate differed (Karl-Johan Alm)

Pull request description:

  This ensures that the code doesn't silently ignore too low fee reates. It will now trigger an error in the QT client, if the user provides a fee rate below the minimum, and becomes a necessary check for #11413.

ACKs for top commit:
  Sjors:
    utACK 44cc75f80e (rebased)
  fjahr:
    re-ACK 44cc75f80e

Tree-SHA512: cd5a60ee496e64f7ab37aaa53f7748a7393357b1629ccd9660839d366c6191b6413b871ce3aa7293fce1539336222c300ef6f86304f30a1ae8fe361b02310483
2020-06-16 13:46:10 -04:00
Wladimir J. van der Laan
86590c17fc Merge #19284: [net] Add seed.bitcoin.wiz.biz to DNS seeds
313a081b90 [net] Add seed.bitcoin.wiz.biz to DNS seeds (wiz)

Pull request description:

  I've created the `seed.bitcoin.wiz.biz` DNS seed for the benefit of the Bitcoin community, and will operate it in accordance with the [Bitcoin DNS seed operator policy](https://github.com/bitcoin/bitcoin/blob/master/doc/dnsseed-policy.md). Since this is my first PR to the Bitcoin Core project, I also ACK the [contributing guidelines](https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md).

  The data for this DNS seed is generated using redundant instances of TheBlueMatt's [dnsseed-rust implementation](https://github.com/TheBlueMatt/dnsseed-rust), which connects to all discoverable Bitcoin nodes to verify their capabilities and speed, and utilizes the full AS-MAP data from my network's BGP tables to select Bitcoin nodes which are fairly distributed across different networks.

  As for my qualifications, I currently operate Bitcoin nodes for the [mempool.space](https://mempool.space/) open-source block explorer project (mempool) and the [Bisq Network](https://bisq.network/) open-source P2P trading community (bisq-network). I have 20 years experience as a network engineer, and all of [my Bitcoin nodes](https://bitnodes.io/nodes/?q=AS54415) are hosted on [my own network](https://ipinfo.io/AS54415) across multiple datacenters. For personal references, the current Bitcoin DNS seed operators Emzy and TheBlueMatt can probably vouch for me.

  The DNS responses served from this instance are currently served with a TTL of 60 seconds, and the DNS resolvers do not log queries from users. Any inquiries related to the operation of this DNS seed can be sent to <noc@wiz.biz>.

  Here is a rough diagram of the `seed.bitcoin.wiz.biz` DNS seed architecture:

  ![seed bitcoin wiz biz](https://user-images.githubusercontent.com/232186/84641969-cb2c6300-af36-11ea-9e4c-392fe39f5f08.png)

ACKs for top commit:
  jonasschnelli:
    Tested ACK 313a081b90.
  laanwj:
    ACK 313a081b90

Tree-SHA512: 9e4ea7a929b7888eba748933c1581328aefcba4de503af96f99630d797d794859b22c99999c25c3fc90f6efaed2598f32784d3acea3e428d84bae3aa37f92a25
2020-06-16 19:33:26 +02:00
Wladimir J. van der Laan
f658c15bb9 Merge #18434: tests: add a test-security target and run it in CI
9fe71a57a6 test: use subprocess.run() in test-security-check.py (fanquake)
968aaae940 tests: run test-security-check.py in CI (fanquake)

Pull request description:

  [Wladimir asked](https://github.com/bitcoin/bitcoin/pull/18415#issuecomment-603843094) about running the `test-security-check.py` script in our CI. This PR adds a target for that: `make test-security` and adds it to a few CI jobs.

ACKs for top commit:
  laanwj:
    ACK 9fe71a57a6

Tree-SHA512: d00ebbefbd57ab22436f284837c320f73238ec9967495adc4f2f9a4d574b3b1595c19ce41d53ff4060d5cd7174dbc311235d5877c90e8af2f5587735e7236056
2020-06-16 18:22:30 +02:00
gzhao408
9a40cfc558 [refactor] use waiting inside disconnect_p2ps
-Use wait_for_disconnect instead of manual
wait after calling disconnect_p2ps.
2020-06-16 08:28:55 -07:00
gzhao408
aeb9fb414e [test] wait for disconnect_p2ps to be reflected in getpeerinfo
-Waiting is important to avoid race conditions,
especially if testing peer info through rpc later.
-Wait for mininodes to be disconnected only, even
though it's more complex, because we may still want
to be connected to test nodes.
2020-06-16 08:26:13 -07:00
gzhao408
e81942d2e1 [test] logging and style followups for bloomfilter tests
-Use peer to refer to mininodes instead of node
because they are not bitcoind nodes.
-Use log.debug for logs that give helpful but
not super necessary information.
-Adhere to style guidelines (newlines, capitalization).
2020-06-16 08:23:12 -07:00
MarcoFalke
fa02b47313 refactor: Use AbortError in FatalError
This is needed for consistency with AbortNode
2020-06-16 10:51:50 -04:00
MarcoFalke
4b30c41b4e Merge #18927: Pass bilingual_str argument to AbortNode()
5527be0627 refactor: Add AbortError alias (Hennadii Stepanov)
d924f2a596 Drop MSG_NOPREFIX flag (Hennadii Stepanov)
083daf7fba Pass bilingual_str argument to AbortNode() (Hennadii Stepanov)
d1cca129b4 refactor: Use bilingual_str::empty() (Hennadii Stepanov)

Pull request description:

  This PR is a [followup](https://github.com/bitcoin/bitcoin/issues/16218#issuecomment-625919724) of #16224, and it adds `bilingual_str` type argument support to the `AbortNode()` functions.

ACKs for top commit:
  MarcoFalke:
    ACK 5527be0627 👟

Tree-SHA512: bf8b15b14912b1f672e6e588fffa1e6eb6f00b4b23d15d0ced7f18fbdf76919244427feb7217007fe29617049308e13def893a03a87358db819cca9692f59905
2020-06-16 08:53:02 -04:00
MarcoFalke
fa41b0a6da pep-8 test/functional/test_framework/util.py
Can be reviewed with
--word-diff-regex=. -U0
2020-06-16 08:28:06 -04:00
MarcoFalke
faa841bc97 test: refactor: Inline adjust_bitcoin_conf_for_pre_17 2020-06-16 08:27:57 -04:00
fanquake
9fe71a57a6 test: use subprocess.run() in test-security-check.py 2020-06-16 19:52:30 +08:00
fanquake
968aaae940 tests: run test-security-check.py in CI 2020-06-16 19:52:30 +08:00
MarcoFalke
0afbeb73cc Merge #19153: test: mempool compatibility test
16d4b3fd6d test: mempool.dat compatibility between versions (Ivan Metlushko)

Pull request description:

  Rationale: Verify mempool.dat compatibility between versions

  The format of mempool.dat has been changed in #18038
  The tests verifies the fix made in #18807 and ensures that the file format is compatible between current version and v0.19.1
  The test verifies both backward and forward compatibility.

  This PR also adds a log when we fail to add a tx loaded from mempool.dat.
  It was useful when debugging this test and could be potentially useful to debug other scenarios as well.

  Closes #19037

ACKs for top commit:
  Sjors:
    tACK 16d4b3fd6d

Tree-SHA512: 00a38bf528c6478cb0da467af216488f83c1e3ca4d9166c109202ea8284023e99d87a3d6e252c4d88d08d9b5ed1a730b3e1970d6e5c0aef526fa7ced40de7490
2020-06-16 07:25:14 -04:00
MarcoFalke
fa1904e5f0 net: Remove dead logging code
fRet is never false, so the dead code can be removed and the return type
can be made void
2020-06-16 06:57:39 -04:00
MarcoFalke
fac12ebf4f net: Avoid redundant and confusing FAILED log
Every `return false` is preceeded by a detailed debug log message to
explain that a disconnect or misbehavior happened. Logging another
generic "FAILED" message seems redundant.

Also, the size of the message and the message type has already been
logged and is thus redundant as well.

Finally, claiming that message processing FAILED seems odd, because the
message was fully processed to the point where it was concluded that the
peer should be either disconnected or marked as misbehaving.
2020-06-16 06:57:30 -04:00
MarcoFalke
46bdd4b537 Merge #19178: Make mininode_lock non-reentrant
62068381a3 [tests] Make mininode_lock non-reentrant (John Newbery)
c67c1f2c03 [tests] Don't call super twice in P2PTxInvStore.on_inv() (John Newbery)
9d80762fa0 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() (John Newbery)
edae6075aa [tests] Only acquire lock once in p2p_compactblocks.py (John Newbery)

Pull request description:

  There's no need for mininode_lock to be reentrant.
  Use a simpler non-recursive lock.

ACKs for top commit:
  MarcoFalke:
    ACK 62068381a3 😃
  jonatack:
    ACK 62068381a3

Tree-SHA512: dcbc19e6c986970051705789be0ff7bec70c69cf76d5b468c2ba4cb732883ad512b1de5c3206c2eca41fa3f1c4806999df4cabbf67fc3c463bb817458e59a19c
2020-06-16 05:46:17 -04:00
fanquake
28ce05d06f Merge #19260: p2p: disconnect peers that send filterclear + update existing filter msg disconnect logic
3a10d935ac [p2p/refactor] move disconnect logic and remove misbehaving (gzhao408)
ff8c430c65 [test] test disconnect for filterclear (gzhao408)
1c6b787e03 [netprocessing] disconnect node that sends filterclear (gzhao408)

Pull request description:

  Nodes that don't have bloomfilters turned on (i.e. no `NODE_BLOOM` service) should disconnect peers that send them `filterclear` P2P messages.

  Non-bloomfilter nodes already disconnect peers for [`filteradd` and `filterload`](19e919217e/src/net_processing.cpp (L2218)), but #8709 removed `filterclear` so it could be used to reset tx relay. This isn't needed now because using `feefilter` message is much better for this purpose (See #19204).

  Also refactors existing disconnect logic for `filteradd` and `filterload` into respective message handlers and removes banning for them.

ACKs for top commit:
  jnewbery:
    Code review ACK 3a10d935ac
  naumenkogs:
    utACK 3a10d93
  gillichu:
    tested ACK: quick test_runner on macOS [`3a10d93`](3a10d935ac)
  MarcoFalke:
    re-ACK 3a10d935ac only change is replacing false with true 🚝

Tree-SHA512: 7aad8b3c0b0e776a47ad52544f0c1250feb242320f9a2962542f5905042f77e297a1486f8cdc3bf0fb93cd00c1ab66a67b2ec426eb6da3fe4cda56b5e623620f
2020-06-16 16:09:22 +08:00
Andrew Chow
61c16339da walletdb: Move BDB specific things into bdb.{cpp/h}
Leave wallet/db.{cpp/h} for generic WalletDatabase stuff. The BDB
specific stuff goes into bdb.{cpp/h}
2020-06-15 20:41:05 -04:00
Andrew Chow
8f033642a8 walletdb: moveonly: Move BerkeleyBatch Cursor and Txn funcs to cpp
Put the implementation in the cpp, not the h file.
2020-06-15 20:35:39 -04:00
Andrew Chow
25a655794a walletdb: move IsWalletLoaded to walletdb.cpp 2020-06-15 17:36:08 -04:00
Andrew Chow
f6fc5f3849 walletdb: Add IsBDBWalletLoaded to look for BDB wallets specifically 2020-06-15 17:36:06 -04:00
Alex Willmer
47b49a05ea contrib: Fix SyntaxWarning in Python base58 implementation
In Python integers should be compared for equality (`i == j`), not identity (`i is j`). Recent versions of CPython 3.x emit a SyntaxWarning when they encounter this incorrect usage, e.g.

```
$ python3 base58.py 
base58.py:110: SyntaxWarning: "is" with a literal. Did you mean "=="?
  assert get_bcaddress_version('15VjRaDX9zpbA8LVnbrCAFzrVzN7ixHNsC') is 0
Tests passed
```
2020-06-15 21:36:54 +01:00
Andrew Chow
c3538f435a walletdb: Make SpliWalletFilePath non-static 2020-06-15 14:14:51 -04:00
practicalswift
67bb7be864 tests: Add fuzzing harness for CHash{160,256}, C{HMAC_,}SHA{1,256,512}, CRIPEMD160, CSipHasher, etc. 2020-06-15 18:14:50 +00:00
Andrew Chow
951bca61d7 tests: feature_backwards_compatibility.py test 0.16 up/downgrade 2020-06-15 11:01:57 -04:00
Andrew Chow
3a03a11e8c Skip hdKeypath of 'm'
Previously the seed was stored with keypath 'm' so we need to skip this
as well when determining inactive seeds.
2020-06-15 10:58:31 -04:00
MarcoFalke
fa6d5ab674 refactor: Remove unused BlockAssembler::pblock member var 2020-06-15 08:08:19 -04:00
MarcoFalke
fab80fef61 refactor: Remove unused EnsureChainman 2020-06-15 07:39:35 -04:00
MarcoFalke
fa34587f1c scripted-diff: Replace EnsureChainman with Assert in unit tests
-BEGIN VERIFY SCRIPT-
sed --regexp-extended -i -e 's/EnsureChainman\((m?_?node)\)\./Assert(\1.chainman)->/g' $(git grep -l EnsureChainman)
-END VERIFY SCRIPT-
2020-06-15 07:39:26 -04:00
MarcoFalke
fa6ef701ad util: Add Assert identity function
The utility is primarily useful to dereference pointer types, which are
known to be not null at that time.

For example, the ArgsManager is known to exist when the wallets are
started. Instead of silently relying on that assumption, Assert can be
used to abort the program and avoid UB should the assumption ever be
violated.
2020-06-15 07:39:08 -04:00
MarcoFalke
1c86ed4148 Merge #19276: ci: Move travis workarounds to .travis.yml
fa71667597 ci: Move travis workarounds to .travis.yml (MarcoFalke)

Pull request description:

  It seems odd to have travis related workarounds in the general ci config files. Fix that oddity by moving the travis related workarounds to the travis yaml file.

  For unexplained reasons, this should also work around and thus close #19171

ACKs for top commit:
  hebasto:
    ACK fa71667597, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: b4419d38e2b41f6e4d6e6b7658f1d972c40c390a49fe78808f8640d28efd84cc6668ce292d45b7c539e65b9e2ecbad10e796cb8f9329a0f1e7d0132ce962d226
2020-06-15 06:25:06 -04:00
MarcoFalke
75edf2cf02 Merge #19282: RPC: Rephrase generatetoaddress help, and use PACKAGE_NAME
0f8f515445 RPC: Rephrase generatetoaddress help, and use PACKAGE_NAME (Luke Dashjr)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 357c6e0bd1b144213ca6cf0bfd649c7a482c2d6d5e98a254d20c8365d228dc71ae1b78aca4918fdbf065f8894ef82f8a475902d605204275bb99fe77d4b42fae
2020-06-15 06:14:37 -04:00
wiz
313a081b90 [net] Add seed.bitcoin.wiz.biz to DNS seeds 2020-06-15 18:22:52 +09:00
Luke Dashjr
0f8f515445 RPC: Rephrase generatetoaddress help, and use PACKAGE_NAME 2020-06-15 08:13:49 +00:00
gzhao408
3a10d935ac [p2p/refactor] move disconnect logic and remove misbehaving
-Increasing the banscore and/or banning is too harsh,
just disconnecting is enough.
-Return true from ProcessMessage because we already log
receipt of filterclear and disconnect.
2020-06-14 11:48:17 -07:00
gzhao408
ff8c430c65 [test] test disconnect for filterclear 2020-06-14 11:47:49 -07:00
gzhao408
1c6b787e03 [netprocessing] disconnect node that sends filterclear
-nodes not serving bloomfilters should disconnect peers
that send filterclear, just like filteradd and filterload
-nodes that want to enable/disable txrelay should use
feefilter
2020-06-14 11:47:12 -07:00
MarcoFalke
fa457fbd33 move-only: Move NDEBUG compile time check to util/check 2020-06-14 13:58:02 -04:00
MarcoFalke
fa71667597 ci: Move travis workarounds to .travis.yml 2020-06-14 11:33:25 -04:00
fanquake
195822f1e0 Merge #19264: [tests] Don't import asyncio to test magic bytes
49236be099 [tests] Don't import asyncio to test magic bytes (John Newbery)

Pull request description:

  Simplify the test for invalid start bytes. No need to import asyncio and the Network thread.

ACKs for top commit:
  MarcoFalke:
    review ACK 49236be099
  jonatack:
    ACK 49236be099
  troygiorshev:
    ACK 49236be.  +0.1 on the additional `cut_len` reformat.

Tree-SHA512: 75cb695603cdc1be7035d7b5117dbef2a1fdb29fd4414a73d75b53d563d6fa800c31bfa9475004622c8bdea4978b51b2055d6fa7be0fe47c7ae34ccc2b0e89a0
2020-06-14 13:31:17 +08:00
Hennadii Stepanov
44a14afbb8 build: Drop dead non-pkg-config code for qrencode check 2020-06-13 20:07:10 +03:00
Hennadii Stepanov
10cbae0c39 build: Drop dead non-pkg-config code for ZMQ check 2020-06-13 20:01:49 +03:00
Hennadii Stepanov
06cfc9cadf build: Fix indentation in UNIVALUE check 2020-06-13 20:01:36 +03:00
Hennadii Stepanov
6fd2118e77 build: Drop dead non-pkg-config code for UNIVALUE check 2020-06-13 20:01:27 +03:00
Hennadii Stepanov
e9edbe4dbd build: Always use pkg-config 2020-06-13 20:01:04 +03:00
Hennadii Stepanov
9e2e753b06 build: Always define ZMQ_STATIC for MinGW 2020-06-13 19:59:18 +03:00
John Newbery
49236be099 [tests] Don't import asyncio to test magic bytes 2020-06-13 10:49:31 -04:00
MarcoFalke
eac65d99dd Merge #19261: refactor: Drop ::HasWallets()
ccf1f6ea24 refactor: Drop ::HasWallets() (João Barbosa)

Pull request description:

  Minor follow-up of #19250. The global `HasWallets()` is used only once and at the call site there's already a way to know if any wallet is loaded.

ACKs for top commit:
  MarcoFalke:
    ACK ccf1f6ea24
  hebasto:
    ACK ccf1f6ea24, I have reviewed the changes and they look OK, I agree they can be merged.

Tree-SHA512: fb902c045cbd331eaf71716c04734520f2ce7f2b317db510c4ce140162bbc683327b5a40ac860f6cde5add37e069065274d39dfa147fac2091eedec505f2f7eb
2020-06-13 10:30:31 -04:00
Martin Ankerl
78c312c983 Replace current benchmarking framework with nanobench
This replaces the current benchmarking framework with nanobench [1], an
MIT licensed single-header benchmarking library, of which I am the
autor. This has in my opinion several advantages, especially on Linux:

* fast: Running all benchmarks takes ~6 seconds instead of 4m13s on
  an Intel i7-8700 CPU @ 3.20GHz.

* accurate: I ran e.g. the benchmark for SipHash_32b 10 times and
  calculate standard deviation / mean = coefficient of variation:

  * 0.57% CV for old benchmarking framework
  * 0.20% CV for nanobench

  So the benchmark results with nanobench seem to vary less than with
  the old framework.

* It automatically determines runtime based on clock precision, no need
  to specify number of evaluations.

* measure instructions, cycles, branches, instructions per cycle,
  branch misses (only Linux, when performance counters are available)

* output in markdown table format.

* Warn about unstable environment (frequency scaling, turbo, ...)

* For better profiling, it is possible to set the environment variable
  NANOBENCH_ENDLESS to force endless running of a particular benchmark
  without the need to recompile. This makes it to e.g. run "perf top"
  and look at hotspots.

Here is an example copy & pasted from the terminal output:

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                2.52 |      396,529,415.94 |    0.6% |           25.42 |            8.02 |  3.169 |           0.06 |    0.0% |      0.03 | `bench/crypto_hash.cpp RIPEMD160`
|                1.87 |      535,161,444.83 |    0.3% |           21.36 |            5.95 |  3.589 |           0.06 |    0.0% |      0.02 | `bench/crypto_hash.cpp SHA1`
|                3.22 |      310,344,174.79 |    1.1% |           36.80 |           10.22 |  3.601 |           0.09 |    0.0% |      0.04 | `bench/crypto_hash.cpp SHA256`
|                2.01 |      496,375,796.23 |    0.0% |           18.72 |            6.43 |  2.911 |           0.01 |    1.0% |      0.00 | `bench/crypto_hash.cpp SHA256D64_1024`
|                7.23 |      138,263,519.35 |    0.1% |           82.66 |           23.11 |  3.577 |           1.63 |    0.1% |      0.00 | `bench/crypto_hash.cpp SHA256_32b`
|                3.04 |      328,780,166.40 |    0.3% |           35.82 |            9.69 |  3.696 |           0.03 |    0.0% |      0.03 | `bench/crypto_hash.cpp SHA512`

[1] https://github.com/martinus/nanobench

* Adds support for asymptotes

  This adds support to calculate asymptotic complexity of a benchmark.
  This is similar to #17375, but currently only one asymptote is
  supported, and I have added support in the benchmark `ComplexMemPool`
  as an example.

  Usage is e.g. like this:

  ```
  ./bench_bitcoin -filter=ComplexMemPool -asymptote=25,50,100,200,400,600,800
  ```

  This runs the benchmark `ComplexMemPool` several times but with
  different complexityN settings. The benchmark can extract that number
  and use it accordingly. Here, it's used for `childTxs`. The output is
  this:

  | complexityN |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |     total | benchmark
  |------------:|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|----------:|:----------
  |          25 |        1,064,241.00 |              939.64 |    1.4% |    3,960,279.00 |    2,829,708.00 |  1.400 |      0.01 | `ComplexMemPool`
  |          50 |        1,579,530.00 |              633.10 |    1.0% |    6,231,810.00 |    4,412,674.00 |  1.412 |      0.02 | `ComplexMemPool`
  |         100 |        4,022,774.00 |              248.58 |    0.6% |   16,544,406.00 |   11,889,535.00 |  1.392 |      0.04 | `ComplexMemPool`
  |         200 |       15,390,986.00 |               64.97 |    0.2% |   63,904,254.00 |   47,731,705.00 |  1.339 |      0.17 | `ComplexMemPool`
  |         400 |       69,394,711.00 |               14.41 |    0.1% |  272,602,461.00 |  219,014,691.00 |  1.245 |      0.76 | `ComplexMemPool`
  |         600 |      168,977,165.00 |                5.92 |    0.1% |  639,108,082.00 |  535,316,887.00 |  1.194 |      1.86 | `ComplexMemPool`
  |         800 |      310,109,077.00 |                3.22 |    0.1% |1,149,134,246.00 |  984,620,812.00 |  1.167 |      3.41 | `ComplexMemPool`

  |   coefficient |   err% | complexity
  |--------------:|-------:|------------
  |   4.78486e-07 |   4.5% | O(n^2)
  |   6.38557e-10 |  21.7% | O(n^3)
  |   3.42338e-05 |  38.0% | O(n log n)
  |   0.000313914 |  46.9% | O(n)
  |     0.0129823 | 114.4% | O(log n)
  |     0.0815055 | 133.8% | O(1)

  The best fitting curve is O(n^2), so the algorithm seems to scale
  quadratic with `childTxs` in the range 25 to 800.
2020-06-13 12:24:18 +02:00
fanquake
265492723a Merge #18297: build: Use pkg-config in BITCOIN_QT_CONFIGURE for all hosts including Windows
8a26848c46 build: Fix m4 escaping (Hennadii Stepanov)
9123ec15db build: Remove extra tokens warning (Hennadii Stepanov)
fded4f48c3 build: Remove duplicated QT_STATICPLUGIN define (Hennadii Stepanov)
05a93d5d96 build: Fix indentation in bitcoin_qt.m4 (Hennadii Stepanov)
ddbb419310 build: Use pkg-config in BITCOIN_QT_CONFIGURE for all hosts (Hennadii Stepanov)
492971de35 build: Fix mingw pkgconfig file and dependency naming (Hennadii Stepanov)

Pull request description:

  This PR makes `bitcoin_qt.m4` to use `pkg-config` for all hosts and removes non-pkg-config paths from it. This is a step towards the idea which was clear [stated](https://github.com/bitcoin/bitcoin/pull/8314#issue-76644643) by Cory Fields:
  > I believe the consensus is to treat Windows like the others and require pkg-config across the board. We can drop all of the non-pkg-config paths, and simply AC_REQUIRE(PKG_PROG_PKG_CONFIG)

  There are two unsolved problems with this PR. If depends is built with `DEBUG=1` the `configure` script fails to pickup Qt:
  - for macOS host (similar to, but not the same as #16391)
  - for Windows host (regression)

  The fix is ~on its way~ submitted in #18298 (as a followup).

  Also this PR picks some small improvements from #17820.

ACKs for top commit:
  theuni:
    Code review ACK 8a26848c46
  dongcarl:
    Code Review ACK 8a26848c46
  laanwj:
    Code review ACK 8a26848c46

Tree-SHA512: 3b25990934b939121983df7707997b31d61063b1207d909f539d69494c7cb85212f353092956d09ecffebb9fef28b869914dd1216a596d102fcb9744bb5487f7
2020-06-13 15:41:39 +08:00
fanquake
8c97780db8 Merge #19228: Update libsecp256k1 subtree
e10439ce5a scripted-diff: rename privkey with seckey in secp256k1 interface (Pieter Wuille)
ca8bc42330 Drop --disable-jni from libsecp256k1 configure options (Pieter Wuille)
ddc2419c09 Update MSVC build config for libsecp256k1 (Pieter Wuille)
67f232b5d8 Squashed 'src/secp256k1/' changes from b19c000063..2ed54da18a (Pieter Wuille)

Pull request description:

  It's been abound a year since the subtree was updated.

  Here is a list of the included PRs:

  * bitcoin-core/secp256k1#755: Recovery signing: add to constant time test, and eliminate non ct operators
  * bitcoin-core/secp256k1#754: Fix uninit values passed into cmov
  * bitcoin-core/secp256k1#752: autoconf: Use ":" instead of "dnl" as a noop
  * bitcoin-core/secp256k1#750: Add macOS to the CI
  * bitcoin-core/secp256k1#701: Make ec_ arithmetic more consistent and add documentation
  * bitcoin-core/secp256k1#732: Retry if r is zero during signing
  * bitcoin-core/secp256k1#742: Fix typo in ecmult_const_impl.h
  * bitcoin-core/secp256k1#740: Make recovery/main_impl.h non-executable
  * bitcoin-core/secp256k1#735: build: fix OpenSSL EC detection on macOS
  * bitcoin-core/secp256k1#728: Suppress a harmless variable-time optimization by clang in memczero
  * bitcoin-core/secp256k1#722: Context isn't freed in the ECDH benchmark
  * bitcoin-core/secp256k1#700: Allow overriding default flags
  * bitcoin-core/secp256k1#708: Constant-time behaviour test using valgrind memtest.
  * bitcoin-core/secp256k1#710: Eliminate harmless non-constant time operations on secret data.
  * bitcoin-core/secp256k1#718: Clarify that a secp256k1_ecdh_hash_function must return 0 or 1
  * bitcoin-core/secp256k1#714: doc: document the length requirements of output parameter.
  * bitcoin-core/secp256k1#682: Remove Java Native Interface
  * bitcoin-core/secp256k1#713: Docstrings
  * bitcoin-core/secp256k1#704: README: add a section for test coverage
  * bitcoin-core/secp256k1#709: Remove secret-dependant non-constant time operation in ecmult_const.
  * bitcoin-core/secp256k1#703: Overhaul README.md
  * bitcoin-core/secp256k1#689: Remove "except in benchmarks" exception for fp math
  * bitcoin-core/secp256k1#679: Add SECURITY.md
  * bitcoin-core/secp256k1#685: Fix issue where travis does not show the ./tests seed…
  * bitcoin-core/secp256k1#690: Add valgrind check to travis
  * bitcoin-core/secp256k1#678: Preventing compiler optimizations in benchmarks without a memory fence
  * bitcoin-core/secp256k1#688: Fix ASM setting in travis
  * bitcoin-core/secp256k1#684: Make no-float policy explicit
  * bitcoin-core/secp256k1#677: Remove note about heap allocation in secp256k1_ecmult_odd_multiples_table_storage_var
  * bitcoin-core/secp256k1#647: Increase robustness against UB in secp256k1_scalar_cadd_bit
  * bitcoin-core/secp256k1#664: Remove mention of ec_privkey_export because it doesn't exist
  * bitcoin-core/secp256k1#337: variable sized precomputed table for signing
  * bitcoin-core/secp256k1#661: Make ./configure string consistent
  * bitcoin-core/secp256k1#657: Fix a nit in the recovery tests
  * bitcoin-core/secp256k1#650: secp256k1/src/tests.c:  Properly handle sscanf return value
  * bitcoin-core/secp256k1#654: Fix typo (∞)
  * bitcoin-core/secp256k1#583: JNI: fix use sig array
  * bitcoin-core/secp256k1#644: Avoid optimizing out a verify_check
  * bitcoin-core/secp256k1#652: README.md: update instruction to run tests
  * bitcoin-core/secp256k1#651: Fix typo in secp256k1_preallocated.h
  * bitcoin-core/secp256k1#640: scalar_impl.h: fix includes
  * bitcoin-core/secp256k1#655: jni: Use only Guava for hex encoding and decoding
  * bitcoin-core/secp256k1#634: Add a descriptive comment for secp256k1_ecmult_const.
  * bitcoin-core/secp256k1#631: typo in comment for secp256k1_ec_pubkey_tweak_mul ()
  * bitcoin-core/secp256k1#629: Avoid calling _is_zero when _set_b32 fails.
  * bitcoin-core/secp256k1#630: Note intention of timing sidechannel freeness.
  * bitcoin-core/secp256k1#628: Fix ability to compile tests without -DVERIFY.
  * bitcoin-core/secp256k1#627: Guard memcmp in tests against mixed size inputs.
  * bitcoin-core/secp256k1#578: Avoid implementation-defined and undefined behavior when dealing with sizes
  * bitcoin-core/secp256k1#595: Allow to use external default callbacks
  * bitcoin-core/secp256k1#600: scratch space: use single allocation
  * bitcoin-core/secp256k1#592: Use trivial algorithm in ecmult_multi if scratch space is small
  * bitcoin-core/secp256k1#566: Enable context creation in preallocated memory
  * bitcoin-core/secp256k1#596: Make WINDOW_G configurable
  * bitcoin-core/secp256k1#561: Respect LDFLAGS and #undef STATIC_PRECOMPUTATION if using basic config
  * bitcoin-core/secp256k1#533: Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...)
  * bitcoin-core/secp256k1#617: Pass scalar by reference in secp256k1_wnaf_const()
  * bitcoin-core/secp256k1#619: Clear a copied secret key after negation
  * bitcoin-core/secp256k1#612: Allow field_10x26_arm.s to compile for ARMv7 architecture

ACKs for top commit:
  real-or-random:
    ACK e10439ce5a I verified the diff (subtree matches my local tree, manual inspection of other commits) but I didn't tested the resulting code
  fanquake:
    ACK e10439ce5a
  Sjors:
    ACK e10439ce5a
  jonasnick:
    reACK e10439ce5a

Tree-SHA512: eb6284a485da78e9d2ed3f771df85560d47c770ebf480a0d4121ab356ad26be101a2b973efe412f26e6c142bc1dbd2efbb5cc08774233e41918c59fe3dff3387
2020-06-13 09:50:29 +08:00
João Barbosa
ccf1f6ea24 refactor: Drop ::HasWallets() 2020-06-13 01:09:15 +01:00
MarcoFalke
f154071ec8 Merge #19177: test: Fix and clean p2p_invalid_messages functional tests
af2a145e57 Refactor resource exhaustion test (Troy Giorshev)
5c4648d17b Fix "invalid message size" test (Troy Giorshev)
ff1e7b8844 Move size limits to module-global (Troy Giorshev)
57890abf2c Remove two unneeded tests (Troy Giorshev)

Pull request description:

  This PR touches only the p2p_invalid_messages.py functional test module.  There are two main goals accomplished here.  First, it fixes the "invalid message size" test, which previously made a message that was invalid for multiple reasons.  Second, it refactors the file into a single consistent style.  This file appears to have originally had two authors, with different styles and some test duplication.

  It should now be easier and quicker to understand this module, anticipating the upcoming [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and [AltNet](https://github.com/bitcoin/bitcoin/issues/18989) changes.

  This should probably go in ahead of #19107, but the two are not strictly related.

ACKs for top commit:
  jnewbery:
    ACK af2a145e57
  MarcoFalke:
    re-ACK af2a145e57 🍦

Tree-SHA512: 9b57561e142c5eaefac5665f7355c8651670400b4db1a89525d2dfdd20e872d6873c4f6175c4222b6f5a8e5210cf5d6a52da69b925b673a2e2ac30a15d670d1c
2020-06-12 15:29:53 -04:00
MarcoFalke
5af16a4db7 Merge #16756: test: Connection eviction logic tests
45eff751c6 Add functional test for P2P eviction logic of inbound peers (Martin Zumsande)

Pull request description:

  This adds a functional test for the eviction logic for inbound peers, which is triggered when the number of maximum connections is exceeded.

  The functional test covers eviction protection for peers that  have sent us blocks or txns recently, or that have faster pings. I couldn't find a way to test the logic of `CConnman::AttemptToEvictConnection` that is based on netgroup (see #14210 for related discussion)

  Fixes #16660 (at least partially).

  [Edit: Earlier, this PR also contained a unit test, which was removed after the discussion]

ACKs for top commit:
  jonatack:
    ACK 45eff751c6
  naumenkogs:
    Tested ACK 45eff75
  fjahr:
    re-ACK 45eff751c6
  andrewtoth:
    re-ACK 45eff751c6

Tree-SHA512: 177208ab6f30dc62da1cc5f51e654f7c9770d8c6b42aca6ae7ecb30e29d3096e04d75739578e7d149a0f29dd92652b4a707e93c0f1be8aa7ed315e6ec3ab07a4
2020-06-12 14:48:20 -04:00
MarcoFalke
19e919217e Merge #19250: wallet: Make RPC help compile-time static
fadf6bd04f refactor: Remove unused request.fHelp (MarcoFalke)
fad889cbf0 wallet: Make RPC help compile-time static (MarcoFalke)

Pull request description:

  Currently calling `help` on a wallet RPC method will either return `help: unknown command: getnewaddress` or the actual help. This runtime dependency of the help is a bug that complicates any tool that relies on documentation. Also, the code that enables the bug is overly complicated and confusing.

  The fix is split into two commits:

  * First, a commit that can be reviewed with the `--color-moved=dimmed-zebra` option and tested with the included test.
  * Second, a commit that removes the complicated and confusing code.

ACKs for top commit:
  achow101:
    re-ACK fadf6bd04f
  promag:
    Tested ACK fadf6bd04f.

Tree-SHA512: 65d4ff400467f57cb8415c30ce30f814dc76c5c157308b7a7409c59ac9db629e65dfba31cd9c389cfe60a008d3d87787ea0a0e0f2671fd65fd190543c915493d
2020-06-12 13:04:37 -04:00
Sjors Provoost
a4a3fc4cd2 doc: improve subtree check instructions 2020-06-12 13:31:42 +02:00
fanquake
83fd3a6d73 init: use std::thread for ThreadImport()
Mentioned in #19142, which removed the boost::interruption_point()
in ThreadImport().
2020-06-12 16:02:18 +08:00
fanquake
c4ffcf07af build: remove BIP70 configure option
This was left in after #17165, so that anyone who had been compiling
with (already disabled by default) BIP70 would realise that support
had been completely removed in 0.20.0. However we should be able to
remove it for 0.21.0.
2020-06-12 15:54:00 +08:00
MarcoFalke
b33136b6ba Merge #19083: test: msg_mempool, fRelay, and other bloomfilter tests
dca73941eb scripted-diff: rename node to peer for mininodes (gzhao408)
0474ea25af [test] fix race conditions and test in p2p_filter (gzhao408)
4ef80f0827 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect (gzhao408)
497a619386 [test] add BIP 37 test for node with fRelay=false (gzhao408)
e8acc60156 [test] add mempool msg test for node with bloomfilter enabled (gzhao408)

Pull request description:

  This PR adds a few tests that are bloomfilter-related, including behavior for when bloomfilters are turned _off_:
  1. Tests p2p message `msg_mempool`: a node that has `peerbloomfilters` enabled should send its mempool (disabled behavior already tested [here](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_mempool.py)).
  2. Tests that bloomfilter peers with [`fRelay=False`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#extensions-to-existing-messages) in the `version` message should not receive any invs until they set the filter. The rest is the same as what’s already tested in `p2p_filter.py`.
  3. Tests that peers get disconnected if they send `filterload` or `filteradd` p2p messages to a node with bloom filters disabled.
  4. Refactor: renames p2p_mempool.py to p2p_nobloomfilter_messages.py.
  5. Fixes race conditions in p2p_filter.py

ACKs for top commit:
  MarcoFalke:
    ACK dca73941eb only changes is restoring accidentally deleted test 🍮
  jonatack:
    ACK dca73941eb modulo a few nits if you retouch, happy to re-ACK if you take any of them but don't feel obliged to.

Tree-SHA512: 442aeab0755cb8b830251ea170d1d5e6da8ac9029b3276d407a20ee3d588cc61b77b8842368de18c244056316b8c63b911776d6e106bc7c023439ab915b27ad3
2020-06-11 14:35:08 -04:00
MarcoFalke
8682414d02 Merge #19247: tests: Add fuzzing harness for {Read,Write}{LE,BE}{16,32,64} (crypto/common.h)
cf5b8f64b3 tests: Add fuzzing harness for {Read,Write}{LE,BE}{16,32,64} (crypto/common.h) (practicalswift)
4a8181b303 tests: Add std::vector<uint8_t> ConsumeFixedLengthByteVector(FuzzedDataProvider& fuzzed_data_provider, const size_t length) (practicalswift)

Pull request description:

  Add fuzzing harness for `{Read,Write}{LE,BE}{16,32,64}` (`crypto/common.h`).

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    ACK cf5b8f64b3

Tree-SHA512: 26412daa6987add1c721ad0348a5a894d68a646e724f328f2db6d9c9358a533481d8888b89d4b0743e9d1c11aa4e0e5341eb4c0d05a4da77b15ab75489327749
2020-06-11 12:59:09 -04:00
MarcoFalke
85f7db2284 Merge #19239: tests: move generate_wif_key to wallet_util.py
3a83a01694 [tests] move generate_wif_key to wallet_util.py (John Newbery)
b216b0b71f [tests] sort imports in rpc_createmultisig.py (John Newbery)
e38081846d Revert "[TESTS] Move base58 to own module to break circular dependency" (John Newbery)

Pull request description:

  generate_wif_key is a wallet utility function. Move it from the EC key module to the wallet util module.

  This fixes the circular dependency issue in #17977

ACKs for top commit:
  MarcoFalke:
    ACK 3a83a01694 🍪

Tree-SHA512: 24985dffb75202721ccc0c6c5b52f1fa5d1ce7963bccde24389feb913cab4dad0c265274ca67892c46c8b64e6a065a0f23263a89be4fb9134dfefbdbe5c7238a
2020-06-11 12:49:38 -04:00
MarcoFalke
fadf6bd04f refactor: Remove unused request.fHelp 2020-06-11 12:39:22 -04:00
MarcoFalke
fad889cbf0 wallet: Make RPC help compile-time static 2020-06-11 12:38:36 -04:00
practicalswift
cf5b8f64b3 tests: Add fuzzing harness for {Read,Write}{LE,BE}{16,32,64} (crypto/common.h) 2020-06-11 14:05:54 +00:00
practicalswift
4a8181b303 tests: Add std::vector<uint8_t> ConsumeFixedLengthByteVector(FuzzedDataProvider& fuzzed_data_provider, const size_t length) 2020-06-11 14:05:54 +00:00
MarcoFalke
7a24cca829 Merge #19100: refactor: Combine GetWalletForJSONRPCRequest and EnsureWalletIsAvailable functions
f42f5e58f5 refactor: Combine GetWalletForJSONRPCRequest and EnsureWalletIsAvailable functions (Russell Yanofsky)

Pull request description:

  This simplifies control flow and also helps get rid of the ::vpwallets variable in #19101 since EnsureWalletIsAvailable doesn't have access to the request context.

ACKs for top commit:
  MarcoFalke:
    ACK f42f5e58f5 (reviewed code to check that this is a refactor) 💢
  promag:
    Tested ACK f42f5e58f5.

Tree-SHA512: eb10685de3db3c1d10c3a797d8da5c8c731e4a8c9024bbb7245929ba767a77a52783a739b8cb1fa7af6fcd233dcf9c8ebbe414eb8b902e2542601aac18625997
2020-06-11 09:52:16 -04:00
Hennadii Stepanov
f8213c05f0 Add means to handle negative capabilities in thread safety annotations 2020-06-11 15:49:39 +03:00
fanquake
45a6811d36 Merge #19206: test: Remove leftover comment in mining_basic
fa98e10d5e test: Remove leftover comment in mining_basic (MarcoFalke)
faedb50d89 test: pep-8 mining_basic (MarcoFalke)

Pull request description:

  Remove an accidental leftover comment from #19082, which no longer applies and thus might be confusing

ACKs for top commit:
  adamjonas:
    code review ACK fa98e10

Tree-SHA512: c7f7f8f579b3c6e92f45769be0a7af1a421438a3f5524db5278b2269511a9e0e08f44e3836afb26727644035897ee51ff8296d13ce23030549e7403f57b40e40
2020-06-11 16:38:11 +08:00
Pieter Wuille
e10439ce5a scripted-diff: rename privkey with seckey in secp256k1 interface
-BEGIN VERIFY SCRIPT-
sed -i 's/privkey/seckey/g' src/key.cpp
-END VERIFY SCRIPT-
2020-06-10 18:29:33 -07:00
Pieter Wuille
ca8bc42330 Drop --disable-jni from libsecp256k1 configure options 2020-06-10 18:15:38 -07:00
John Newbery
3a83a01694 [tests] move generate_wif_key to wallet_util.py
generate_wif_key is a wallet utility function. Move
it from the EC key module to the wallet util module.
2020-06-10 12:10:02 -04:00
John Newbery
b216b0b71f [tests] sort imports in rpc_createmultisig.py 2020-06-10 11:56:44 -04:00
John Newbery
e38081846d Revert "[TESTS] Move base58 to own module to break circular dependency"
This reverts commit c75de5da5f.
2020-06-10 11:54:25 -04:00
Sjors Provoost
1404c57403 [doc] Coin: explain that IsSpent() can also mean never existed 2020-06-10 17:46:54 +02:00
Elichai Turkel
9b8907fade Check size after Unserializing CPubKey 2020-06-10 17:35:43 +03:00
gzhao408
dca73941eb scripted-diff: rename node to peer for mininodes
-BEGIN VERIFY SCRIPT-
sed -i 's/FilterNode/P2PBloomFilter/g' test/functional/p2p_filter.py;
sed -i 's/filter_node/filter_peer/g' test/functional/p2p_filter.py;
-END VERIFY SCRIPT-
2020-06-10 07:28:45 -07:00
gzhao408
0474ea25af [test] fix race conditions and test in p2p_filter
-grab mininode_lock for every access to mininode attributes,
otherwise there are race conditions
2020-06-10 07:28:04 -07:00
gzhao408
4ef80f0827 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect
-A node with bloomfilters disabled should disconnect peers that send
msg_mempool, msg_filterload, or msg_filteradd.
-Renamed the test because it now has a wider scope and msg_mempool's
actual functionality makes more sense for p2p_filter.py.
2020-06-10 07:28:04 -07:00
gzhao408
497a619386 [test] add BIP 37 test for node with fRelay=false
A node with fRelay=False should not receive any invs until filter is set using
filterload; all other behavior should be identical.
2020-06-10 07:28:04 -07:00
gzhao408
e8acc60156 [test] add mempool msg test for node with bloomfilter enabled
-msg_mempool is currently only tested with bloomfilter disabled
(node is disconnected) in p2p_mempool.py
-msg_mempool should get mempool txns in response when bloomfilter
is enabled
-edit test that doesn't test msg_mempool as intended
2020-06-10 07:27:58 -07:00
MarcoFalke
6762a627ec Merge #19230: [TESTS] Move base58 to own module to break circular dependency
c75de5da5f [TESTS] Move base58 to own module to break circular dependency (Pieter Wuille)

Pull request description:

  I encountered difficulties with the test framework in #17977. This fixes them, and I think the change is generally useful.

ACKs for top commit:
  laanwj:
    Code review ACK c75de5da5f
  MarcoFalke:
    ACK c75de5da5f according to --color-moved=dimmed-zebra this is a move-only apart from the imports 👒

Tree-SHA512: 9e0493de3e279074f0c70e92c959b73ae30479ad6f2083a3c6bbf4b0191d65ef94854559a5b7c904f5dadc5e93129ed00f6dc0a8ccce6ba7921cd45f7119f74b
2020-06-10 10:23:58 -04:00
MarcoFalke
371a73e940 Merge #19233: Make SetMiscWarning() accept bilingual_str argument
d49612f98a Make SetMiscWarning() accept bilingual_str argument (Hennadii Stepanov)
d1ae7c0355 Make GetWarnings() return bilingual_str (Hennadii Stepanov)
38e33aa481 refactor: Make GetWarnings() bilingual_str aware internally (Hennadii Stepanov)

Pull request description:

  This is one more step for consistent usage of `bilingual_str`.

  No new translation messages are defined.

ACKs for top commit:
  laanwj:
    Code review ACK d49612f98a
  MarcoFalke:
    ACK d49612f98a 🌂

Tree-SHA512: 7413cb94a85291209c182845f6873350bb9e9ce940647d416c462a136603832fec8a63d792341bf634f07629767c78bc206d3a318cf10c7e87241c114c2496e9
2020-06-10 09:50:57 -04:00
Wladimir J. van der Laan
bc933aeaf0 Merge #19226: test: Add BerkeleyDatabase tsan suppression
fa7b46cc91 test: Add BerkeleyDatabase tsan suppression (MarcoFalke)

Pull request description:

  Suppresses/Fixes #19211 for now

ACKs for top commit:
  laanwj:
    ACK fa7b46cc91
  practicalswift:
    ACK fa7b46cc91 -- patch looks correct

Tree-SHA512: 749e606caf0f140c1a190e3273ff81d220daa3eb004ba2b2078e6b3c5b6ac196bd5fc91ef42841412cfd4fe1e2a8694fc2a28268fde8485db90076593fc51dc7
2020-06-10 15:25:13 +02:00
Hennadii Stepanov
d49612f98a Make SetMiscWarning() accept bilingual_str argument 2020-06-10 15:01:20 +03:00
Hennadii Stepanov
d1ae7c0355 Make GetWarnings() return bilingual_str 2020-06-10 15:01:20 +03:00
Hennadii Stepanov
38e33aa481 refactor: Make GetWarnings() bilingual_str aware internally 2020-06-10 15:01:09 +03:00
fanquake
20e9531379 Merge #19231: gui: add missing translation.h include to fix build
948f1134bc gui: add missing translation.h include to fix build (fanquake)

Pull request description:

  After #19176, building the gui on Bionic is failing with:

  ```bash
    CXX      qt/qt_libbitcoinqt_a-guiutil.o
  qt/bitcoin.cpp: In function 'int GuiMain(int, char**)':
  qt/bitcoin.cpp:460:35: error: 'Untranslated' was not declared in this scope
             node->initError(strprintf(Untranslated("Error parsing command line arguments: %s\n"), error));
  ```

  The merge commit also failed to compile with the same error:
  https://travis-ci.org/github/bitcoin/bitcoin/jobs/696627543

ACKs for top commit:
  hebasto:
    ACK 948f1134bc, tested on Linux Mint 19.3 (x86_64): it fixes compiling error with the `--disable-wallet` configure option.

Tree-SHA512: db0197b110b3a7d05af2ceb29fbe9eeb6521d28f53b6267aa6d07a975886adb5c6485af79506ab6c66ed101e32292feeaff3707cdbc11432e5b97400953d5631
2020-06-10 17:12:40 +08:00
Ivan Metlushko
16d4b3fd6d test: mempool.dat compatibility between versions 2020-06-10 11:06:03 +07:00
fanquake
948f1134bc gui: add missing translation.h include to fix build
After #19176, building the gui on Bionic is failing with:

```bash
  CXX      qt/qt_libbitcoinqt_a-guiutil.o
qt/bitcoin.cpp: In function 'int GuiMain(int, char**)':
qt/bitcoin.cpp:460:35: error: 'Untranslated' was not declared in this scope
node->initError(strprintf(Untranslated("Error parsing command line arguments: %s\n"), error));
```

The merge commit also failed to compile with the same error:
https://travis-ci.org/github/bitcoin/bitcoin/jobs/696627543
2020-06-10 10:08:16 +08:00
Pieter Wuille
c75de5da5f [TESTS] Move base58 to own module to break circular dependency
This breaks the script->key->address->script dependency cycle.
2020-06-09 17:50:50 -07:00
MarcoFalke
f8364df250 Merge #19176: refactor: Error message bilingual_str consistency
6fe989054f refactor: Change Node::initError to take bilingual_str (Wladimir J. van der Laan)
425e7cb8cf refactor: Put`TryParsePermissionFlags` in anonymous namespace (Wladimir J. van der Laan)
77b79fa6ef refactor: Error message bilingual_str consistency (Wladimir J. van der Laan)

Pull request description:

  A straightforward and hopefully uncontroversial refactor to improve consistency.

  - Move the decision whether to translate an individual error message to where it is defined. This simplifies call sites: no more `InitError(Untranslated(SomeFunction(...)))`.

  - Make all functions in `util/error.h` consistently return a   `bilingual_str`. We've decided to use this as error message type so let's roll with it.

  This has no functional changes: no messages are changed, no new translation messages are defined.

  Also make a function static that can be static.

ACKs for top commit:
  MarcoFalke:
    ACK 6fe989054f 🔣
  hebasto:
    ACK 6fe989054f, tested on Linux Mint 19.3 (x86_64).

Tree-SHA512: 1dd123ef285c4b50bbc429b2f11c9a63aaa669a84955a0a9b8134e9dc141bc38f863f798e8982ac68bbe83170e1067a87d1a87fe7f791928b7914e10bbc2ef8d
2020-06-09 17:18:04 -04:00
MarcoFalke
221873e158 Merge #19227: test: change blacklist to blocklist
6fc641644f change blacklist to blocklist (TrentZ)

Pull request description:

  Let's use a more appropriate and clear word and discard the usage of the blacklist. Blocklist is clear and shall make everyone happy.

ACKs for top commit:
  amitiuttarwar:
    ACK 6fc641644f
  jonatack:
    ACK 6fc641644f git grep shows these two lines to be the only uses of the word in the codebase other than for specifying colors for the GUI.
  sipsorcery:
    ACK 6fc641644f due to easy change.

Tree-SHA512: 12fd55ad5c79f1a227da90c7fa730972aae6b74ab1f9df79ec1e7d0eca05c383ef7d6ef5f353620a01da344db915005339b62ca0884179d0f47fbefb084c9efc
2020-06-09 17:15:52 -04:00
Pieter Wuille
ddc2419c09 Update MSVC build config for libsecp256k1 2020-06-09 13:41:38 -07:00
Pieter Wuille
67f232b5d8 Squashed 'src/secp256k1/' changes from b19c000063..2ed54da18a
2ed54da18a Merge #755: Recovery signing: add to constant time test, and eliminate non ct operators
28609507e7 Add tests for the cmov implementations
73596a85a2 Add ecdsa_sign_recoverable to the ctime tests
2876af4f8d Split ecdsa_sign logic into a new function and use it from ecdsa_sign and recovery
5e1c885efb Merge #754: Fix uninit values passed into cmov
f79a7adcf5 Add valgrind uninit check to cmovs output
05d315affe Merge #752: autoconf: Use ":" instead of "dnl" as a noop
a39c2b09de Fixed UB(arithmetics on uninit values) in cmovs
3a6fd7f636 Merge #750: Add macOS to the CI
5e8747ae2a autoconf: Use ":" instead of "dnl" as a noop
71757da5cc Explictly pass SECP256K1_BENCH_ITERS to the benchmarks in travis.sh
99bd661d71 Replace travis_wait with a loop printing "\a" to stdout every minute
bc818b160c Bump travis Ubuntu from xenial(16.04) to bionic(18.04)
0c5ff9066e Add macOS support to travis
b6807d91d8 Move travis script into a standalone sh file
f39f99be0e Merge #701: Make ec_ arithmetic more consistent and add documentation
39198a03ea Merge #732: Retry if r is zero during signing
59a8de8f64 Merge #742: Fix typo in ecmult_const_impl.h
4e284655d9 Fix typo in ecmult_const_impl.h
f862b4ca13 Merge #740: Make recovery/main_impl.h non-executable
ffef45c98a Make recovery/main_impl.h non-executable
2361b3719a Merge #735: build: fix OpenSSL EC detection on macOS
3b7d26b23c build: add SECP_TEST_INCLUDES to bench_verify CPPFLAGS
84b5fc5bc3 build: fix OpenSSL EC detection on macOS
37ed51a7ea Make ecdsa_sig_sign constant-time again after reverting 25e3cfb
93d343bfc5 Revert "ecdsa_impl: replace scalar if-checks with VERIFY_CHECKs in ecdsa_sig_sign"
7e3952ae82 Clarify documentation of tweak functions.
89853a0f2e Make tweak function documentation more consistent.
41fc785602 Make ec_privkey functions aliases for ec_seckey_negate, ec_seckey_tweak_add and ec_seckey_mul
22911ee6da Rename private key to secret key in public API (with the exception of function names)
5a73f14d6c Mention that value is unspecified for In/Out parameters if the function returns 0
f03df0e6d7 Define valid ECDSA keys in the documentation of seckey_verify
5894e1f1df Return 0 if the given seckey is invalid in privkey_negate, privkey_tweak_add and privkey_tweak_mul
8f814cddb9 Add test for boundary conditions of scalar_set_b32 with respect to overflows
3fec982608 Use scalar_set_b32_seckey in ecdsa_sign, pubkey_create and seckey_verify
9ab2cbe0eb Add scalar_set_b32_seckey which does the same as scalar_set_b32 and also returns whether it's a valid secret key
4f27e344c6 Merge #728: Suppress a harmless variable-time optimization by clang in memczero
01993878bb Add test for memczero()
52a03512c1 Suppress a harmless variable-time optimization by clang in memczero
8f78e208ad Merge #722: Context isn't freed in the ECDH benchmark
ed1b91171a Merge #700: Allow overriding default flags
85b35afa76 Add running benchmarks regularly and under valgrind in travis
ca4906b02e Pass num of iters to benchmarks as variable, and define envvar
02dd5f1bbb free the ctx at the end of bench_ecdh
e9fccd4de1 Merge #708: Constant-time behaviour test using valgrind memtest.
08fb6c4926 Run valgrind_ctime_test in travis
3d2302257f Constant-time behaviour test using valgrind memtest.
96d8ccbd16 Merge #710: Eliminate harmless non-constant time operations on secret data.
0585b8b2ee Merge #718: Clarify that a secp256k1_ecdh_hash_function must return 0 or 1
7b50483ad7 Adds a declassify operation to aid constant-time analysis.
34a67c773b Eliminate harmless non-constant time operations on secret data.
ca739cba23 Compile with optimization flag -O2 by default instead of -O3
eb45ef3384 Clarify that a secp256k1_ecdh_hash_function must return 0 or 1
856a01d6ad Merge #714: doc: document the length requirements of output parameter.
d72b9e2483 Merge #682: Remove Java Native Interface
4b48a43106 doc: document the length requirements of output parameter.
1b4d256e2e Merge #713: Docstrings
dabfea7e21 field: extend docstring of secp256k1_fe_normalize
dc7d8fd9e2 scalar: extend docstring of secp256k1_scalar_set_b32
074ab582dd Merge #704: README: add a section for test coverage
acb7f97eb8 README: add a section for test coverage
227a4f2d07 Merge #709: Remove secret-dependant non-constant time operation in ecmult_const.
d567b779fe Clarify comments about use of rzr on ge functions and abs function.
2241ae6d14 Remove secret-dependant non-constant time operation in ecmult_const.
642cd062bd Remove Java Native Interface
83fb1bcef4 Remove -O2 from default CFLAGS because this would override the -O3 flag (see AC_PROG_CC in the Autoconf manual)
ecba8138ec Append instead of Prepend user-CFLAGS to default CFLAGS allowing the user to override default variables
613c34cd86 Remove test in configure.ac because it doesn't have an effect
f45d897101 Merge #703: Overhaul README.md
2e759ec753 Overhaul README.md
d644dda5c9 Merge #689: Remove "except in benchmarks" exception for fp math
bde2a32286 Convert bench.h to fixed-point math
387d723c3f Merge #679: Add SECURITY.md
0db61d25c9 Merge #685: Fix issue where travis does not show the ./tests seed…
a0771d15e6 Explicitly disable buffering for stderr in tests
fb424fbba2 Make travis show the ./tests seed by removing stdout buffering and always cat tests.log after a travis run.
22a6031184 Merge #690: Add valgrind check to travis
544002c008 Merge #678: Preventing compiler optimizations in benchmarks without a memory fence
dd98cc988f travis: Added a valgrind test without endro and enabled recovery+ecdh
b4c1382a87 Add valgrind check to travis
0c774d89e6 Merge #688: Fix ASM setting in travis
5c5f71eea5 Fix ASM setting in travis
e2625f8a98 Merge #684: Make no-float policy explicit
bae1bea3c4 Make no-float policy explicit
78c3836341 Add SECURITY.md
362bb25608 Modified bench_scalar_split so it won't get optimized out
73a30c6b58 Added accumulators and checks on benchmarks so they won't get optimized out
770b3dcd6f Merge #677: Remove note about heap allocation in secp256k1_ecmult_odd_multiples_table_storage_var
b76142ff25 Remove note about heap allocation in secp256k1_ecmult_odd_multiples_table_storage_var which was removed in 47045270fa90f81205d989f7107769bce1e71c4d
137d304a6b Merge #647: Increase robustness against UB in secp256k1_scalar_cadd_bit
0d9540b13f Merge #664: Remove mention of ec_privkey_export because it doesn't exist
59782c68b4 Remove mention of ec_privkey_export because it doesn't exist
96cd94e385 Merge #337: variable sized precomputed table for signing
dcb2e3b3ff variable signing precompute table
b4bff99028 Merge #661: Make ./configure string consistent
a467047e11 Make ./configure string consistent
e729cc7f5a Merge #657: Fix a nit in the recovery tests
b64a2e2597 Fix a nit in the recovery tests
e028aa33d3 Merge #650: secp256k1/src/tests.c:  Properly handle sscanf return value
f1e11d363d Merge #654: Fix typo (∞)
ef83281c3a Merge pull request #656 from real-or-random/patch-1
556caad2ca Fix typo in docs for _context_set_illegal_callback
0d82732a9a Improve VERIFY_CHECK of overflow in secp256k1_scalar_cadd_bit. This added check ensures that any curve order overflow doesn't go undetected due a uint32_t overflow.
786dfb49f5 Merge #583: JNI: fix use sig array
e95f8ab098 Merge #644: Avoid optimizing out a verify_check
384f55606a Merge #652: README.md: update instruction to run tests
ee56accd47 Merge #651: Fix typo in secp256k1_preallocated.h
7b9b117230 Merge #640: scalar_impl.h: fix includes
d99bec2e21 Merge #655: jni: Use only Guava for hex encoding and decoding
2abcf951af jni: Use only Guava for hex encoding and decoding
271582b3b7 Fix typo
ce6d438266 README.md: update instruction to run tests
b1e68cb8e6 Fix typo in secp256k1_preallocated.h
a11c76c59a secp256k1/src/tests.c:  Properly handle sscanf return value
8fe63e5654 Increase robustness against UB. Thanks to elichai2 who noted that the literal '1' is a signed integer, and that shifting a signed 32-bit integer by 31 bits causes an overflow and yields undefined behaviour. While 'scalar_low_impl''s 'secp256k1_scalar_cadd_bit' is only used for testing purposes and currently the 'bit' parameter is only 0 or 1, it is better to avoid undefined behaviour in case the used domain of 'secp256k1_scalar_cadd_bit' expands.
94ae7cbf83 Moved a dereference so the null check will be before the dereferencing
2cb73b1064 scalar_impl.h: fix includes
fa33017135 Merge #634: Add a descriptive comment for secp256k1_ecmult_const.
ee9e68cd30 Add a descriptive comment for secp256k1_ecmult_const.
d0d738d32d Merge #631: typo in comment for secp256k1_ec_pubkey_tweak_mul ()
6914c25276 typo in comment for secp256k1_ec_pubkey_tweak_mul ()
e541a90ef6 Merge #629: Avoid calling _is_zero when _set_b32 fails.
f34b0c3f35 Merge #630: Note intention of timing sidechannel freeness.
8d1563b0ff Note intention of timing sidechannel freeness.
1669bb2865 Merge #628: Fix ability to compile tests without -DVERIFY.
ecc94abcc8 Merge #627: Guard memcmp in tests against mixed size inputs.
544435fc90 Merge #578: Avoid implementation-defined and undefined behavior when dealing with sizes
143dc6e9ee Merge #595: Allow to use external default callbacks
e49f7991c2 Add missing #(un)defines to base-config.h
77defd2c3b Add secp256k1_ prefix to default callback functions
908bdce64e Include stdio.h and stdlib.h explicitly in secp256k1.c
5db782e655 Allow usage of external default callbacks
6095a863fa Replace CHECKs for no_precomp ctx by ARG_CHECKs without a return
cd473e02c3 Avoid calling secp256k1_*_is_zero when secp256k1_*_set_b32 fails.
6c36de7a33 Merge #600: scratch space: use single allocation
98836b11f0 scratch: replace frames with "checkpoint" system
7623cf2b97 scratch: save a couple bytes of unnecessarily-allocated memory
a7a164f2c6 scratch: rename `max_size` to `size`, document that extra will actually be allocated
5a4bc0bb95 scratch: unify allocations
c2b028a281 scratch space: thread `error_callback` into all scratch space functions
0be1a4ae62 scratch: add magic bytes to beginning of structure
92a48a764d scratch space: use single allocation
40839e21b9 Merge #592: Use trivial algorithm in ecmult_multi if scratch space is small
dcf392027b Fix ability to compile tests without -DVERIFY.
a484e0008b Merge #566: Enable context creation in preallocated memory
0522caac8f Explain caller's obligations for preallocated memory
238305fdbb Move _preallocated functions to separate header
695feb6fbd Export _preallocated functions
814cc78d71 Add tests for contexts in preallocated memory
ba12dd08da Check arguments of _preallocated functions
5feadde462 Support cloning a context into preallocated memory
c4fd5dab45 Switch to a single malloc call
ef020de16f Add size constants for preallocated memory
1bf7c056ba Prepare for manual memory management in preallocated memory
248bffb052 Guard memcmp in tests against mixed size inputs.
36698dcfee Merge #596: Make WINDOW_G configurable
a61a93ff50 Clean up ./configure help strings
2842dc523e Make WINDOW_G configurable
1a02d6ce51 Merge #626: Revert "Merge #620: Install headers automatically"
662918cb29 Revert "Merge #620: Install headers automatically"
14c7dbd444 Simplify control flow in DER parsing
ec8f20babd Avoid out-of-bound pointers and integer overflows in size comparisons
01ee1b3b3c Parse DER-enconded length into a size_t instead of an int
912680ed86 Merge #561: Respect LDFLAGS and #undef STATIC_PRECOMPUTATION if using basic config
91fae3ace0 Merge #620: Install headers automatically
5df77a0eda Merge #533: Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...)
975e51e0d9 Merge #617: Pass scalar by reference in secp256k1_wnaf_const()
735fbde04e Merge #619: Clear a copied secret key after negation
16e86150d0 Install headers automatically
069870d92a Clear a copied secret key after negation
8979ec0d9a Pass scalar by reference in secp256k1_wnaf_const()
84a808598b Merge #612: Allow field_10x26_arm.s to compile for ARMv7 architecture
d4d270a59c Allow field_10x26_arm.s to compile for ARMv7 architecture
248f046611 Make sure we're not using an uninitialized variable in secp256k1_wnaf_const(...)
9ab96f7b12 Use trivial algorithm in ecmult_multi if scratch space is small
dbed75d969 Undefine `STATIC_PRECOMPUTATION` if using the basic config
310111e093 Keep LDFLAGS if `--coverage`
74e2dbd68e JNI: fix use sig array
3cb057f842 Fix possible integer overflow in DER parsing

git-subtree-dir: src/secp256k1
git-subtree-split: 2ed54da18add295668ec71c91534b640d2cc029b
2020-06-09 13:39:09 -07:00
Pieter Wuille
8903a1a0a7 Update src/secp256k1 subtree 2020-06-09 13:39:09 -07:00
TrentZ
6fc641644f change blacklist to blocklist
Let's use a more appropriate and clear word and discard the usage of the blacklist. Blocklist is clear. Happy for everyone.
2020-06-09 15:08:41 -05:00
MarcoFalke
fa7b46cc91 test: Add BerkeleyDatabase tsan suppression 2020-06-09 14:15:04 -04:00
glowang
ec4c6a17e8 scripted-diff: replace MAX_BLOCKS_ONLY_CONNECTIONS with MAX_BLOCK_RELAY_ONLY_CONNECTIONS
-BEGIN VERIFY SCRIPT-
sed -i 's/\<MAX_BLOCKS_ONLY_CONNECTIONS\>/MAX_BLOCK_RELAY_ONLY_CONNECTIONS/g' src/init.cpp
sed -i 's/\<MAX_BLOCKS_ONLY_CONNECTIONS\>/MAX_BLOCK_RELAY_ONLY_CONNECTIONS/g' src/net.h
-END VERIFY SCRIPT-
2020-06-09 08:05:05 -07:00
Wladimir J. van der Laan
6fe989054f refactor: Change Node::initError to take bilingual_str
Make it consistent with `Chain::initError`.
2020-06-09 15:40:02 +02:00
Wladimir J. van der Laan
425e7cb8cf refactor: PutTryParsePermissionFlags in anonymous namespace
It's only used inside `net_permissions.cpp`.
2020-06-09 15:39:44 +02:00
Wladimir J. van der Laan
77b79fa6ef refactor: Error message bilingual_str consistency
- Move the decision whether to translate an error message to where it is
  defined. This simplifies call sites: no more `InitError(Untranslated(...))`.

- Make all functions in `util/error.h` consistently return a
  `bilingual_str`. We've decided to use this as error message type so
  let's roll with it.

This has no functional changes: no messages are changed, no new
translation messages are defined.
2020-06-09 15:39:44 +02:00
MarcoFalke
9ad6f14175 Merge #19220: refactor: Replace RecursiveMutex with Mutex in warnings.cpp
bacbfb61ee refactor: Replace RecursiveMutex with Mutex in warnings.cpp (Hennadii Stepanov)

Pull request description:

  The functions that could lock this mutex, i.e., `SetMiscWarning()`, `{S,G}etfLargeWorkForkFound()`, `SetfLargeWorkInvalidChainFound()`, `GetWarnings()`, do not call itself recursively, and do not call each other either directly or indirectly. Therefore, the `g_warnings_mutex` could be a non-recursive mutex.

  Related to #19180.

ACKs for top commit:
  laanwj:
    Code review ACK bacbfb61ee
  MarcoFalke:
    ACK bacbfb61ee , reviewed with -W  --word-diff-regex=. 🎿

Tree-SHA512: cc06d3d30e4051115d176dcfbd496c8562a70087369bccde756c1de42d7dc3f415ef20d3d69ad2599c1d0cd4228d604d7564adc17beac7b6ff92b924b8c20d54
2020-06-09 08:54:57 -04:00
Hennadii Stepanov
bacbfb61ee refactor: Replace RecursiveMutex with Mutex in warnings.cpp 2020-06-09 09:35:10 +03:00
Pieter Wuille
addf18da95 Call SHA256AutoDetect in benchmark setup 2020-06-08 11:24:51 -07:00
Hennadii Stepanov
5527be0627 refactor: Add AbortError alias 2020-06-08 18:35:00 +03:00
MarcoFalke
a79bca2f1f Merge #19069: refactor: replace pointers by references within tx_verify.{h,cpp}
b00266fe0c refactor: replace pointers by references within tx_verify.{h,cpp} (Sebastian Falbesoner)

Pull request description:

  This PR gets rid of another unnecessary use of raw pointers, similar to PR #19053 (see also issue #19062 where useful commands for finding potential candidates are listed) but in the tx verification module.

  For the functions `CalculateSequenceLocks()` and `SequenceLocks()`, the  `prevHeights` vector parameter type is changed to be passed as a reference. Note that there were no checks for null pointers -- if one would pass `nullptr` to one of the functions, the following line would immediately lead to a crash:
  dcacea096e/src/consensus/tx_verify.cpp (L32)

ACKs for top commit:
  Empact:
    Code Review ACK b00266fe0c

Tree-SHA512: 0eb71591467905434082029128bdca4df94988c372af40dca325654f6c002c72a00c73776cb5e72d6de2b2f218649211a5dbf19300a2e01f1841d6034e0f01e0
2020-06-08 10:36:57 -04:00
MarcoFalke
297466b49f Merge #18826: Expose txinwitness for coinbase in JSON form from RPC
34645c4dd0 Test txinwitness is accessible on coinbase vin (Rod Vagg)
3e4421070a Expose txinwitness for coinbase in JSON form (Rod Vagg)

Pull request description:

  ## Rationale

  The CLI can provide you with everything about transactions and blocks that you need to reconstruct the block structure and raw block itself **except** for the witness commitment nonce which is stored in the `scriptWitness` of the coinbase and is not printed. You could manually parse the raw `"hex"` fields for transactions if you really wanted to, but this seems to defeat the point of having a JSONification of the raw block/transaction data.

  Without the nonce you can't:

  1. calculate and validate the witness commitment yourself, you can generate the witness tx merkle root but you don't have the nonce to combine it with
  2. reconstruct the raw block form because you don't have `scriptWitness` stack associated with the coinbase (although you know how big it will be and can guess the common case of `[0x000...000]`)

  I'm building some archiving tooling for block data and being able to do a validated two-way conversion is very helpful.

  ## What

  This PR simply makes the `txinwitness` field not dependent on whether we are working with the coinbase or not. So you get it for the coinbase as well as the rest.

  ## Examples

  Common case of a `[0x000...000]` nonce: 00000000000000000000140a7289f3aada855dfd23b0bb13bb5502b0ca60cdd7

  ```json
        "vin": [
          {
            "coinbase": "0368890904c1fe8d5e2f706f6f6c696e2e636f6d2ffabe6d6d5565843a681160cf7b08b1b74ac90a719e6d6ab28c16d336b924f0dc2fcabdc6010000000000000051bf2ad74af345dbe642154b2658931612a70d195e007add0100ffffffff",
            "txinwitness": [
              "0000000000000000000000000000000000000000000000000000000000000000"
            ],
            "sequence": 4294967295
          }
        ],
  ...
  ```

  Novel nonce value: 000000000000000000008c31945b2012258366cc600a3e9a3ee0598e8f797731

  ```json
        "vin": [
          {
            "coinbase": "031862082cfabe6d6d80c099b5e21f4c186d54eb292e17026932e52b1b807fa1380574c5adc1c843450200000000000000",
            "txinwitness": [
              "5b5032506f6f6c5d5b5032506f6f6c5d5b5032506f6f6c5d5b5032506f6f6c5d"
            ],
            "sequence": 4294967295
          }
        ],
  ...
  ```

  ## Alternatives

  This field could be renamed for the coinbase, `"witnessnonce"` perhaps. It could also be omitted when null/zero (`0x000...000`).

  ## Tests

  This didn't break any tests and I couldn't find an obvious way to include a test for this. If this is desired I'd apreicate some pointers.

ACKs for top commit:
  MarcoFalke:
    ACK 34645c4dd0

Tree-SHA512: b192facc1dfd210a5ec3f0d5d1ac6d0cae81eb35be15eaa71f60009a538dd6a79ab396f218434e7e998563f7f0df2c396cc925cb91619f6841c5a67806148c85
2020-06-08 10:18:42 -04:00
MarcoFalke
66666d55b1 doc: Mention repo split in the READMEs 2020-06-08 10:06:14 -04:00
MarcoFalke
faceed753a doc: Add redirect for GUI issues and pull requests 2020-06-08 10:06:02 -04:00
MarcoFalke
3e58734e55 Merge #18898: gui: Display warnings as rich text
a9d28afe23 qt: Display warnings as rich text (Hennadii Stepanov)

Pull request description:

  On master (6621be5351), warnings that contain `<hr />` HTML tag are not displayed correctly:

  ![Screenshot from 2020-05-06 11-30-10](https://user-images.githubusercontent.com/32963518/81177281-0e49fc80-8faf-11ea-8cac-8847aa517e86.png)

  Fixed:

  ![Screenshot from 2020-05-07 07-30-48](https://user-images.githubusercontent.com/32963518/81255618-ca9ad580-9036-11ea-90ad-7f4d89c1880d.png)

ACKs for top commit:
  jonasschnelli:
    utACK a9d28afe23
  promag:
    Code review ACK a9d28afe23.

Tree-SHA512: ba5b3837d5f6ea15c3255a3120c9753fc58ee67a370c388556214048ab993c45be720af7cb8d43bb0f12088956cb78abc77546ed1fc691082880438072fe774b
2020-06-08 09:54:38 -04:00
MarcoFalke
b3ec1fe811 Merge #18890: test: disconnect_nodes should warn if nodes were already disconnected
34e641a564 test: Remove unnecessary disconnect_nodes call in rpc_psbt.py (Danny Lee)
e6e7abd51a test: remove redundant two-way disconnect_nodes calls (Danny Lee)
a9bd1f9adf test: warn if nodes not connected before disconnect_nodes (Danny Lee)

Pull request description:

  There's no harm in calling `disconnect_nodes` for nodes that weren't connected (in this case it's a no-op).  However, detecting this case and logging a warning can help ensure that tests are behaving as expected.

  In addition, since `disconnect_nodes` works bidirectionally, I removed all instances of this pattern:

  ```
  disconnect_nodes(self.nodes[0], 1)
  disconnect_nodes(self.nodes[1], 0)
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 34e641a564 👔
  amitiuttarwar:
    ACK 34e641a564. Thanks for this test improvement!

Tree-SHA512: 344855ceb46c012d43c13d7c09f44d32dcb7645706d10ae1e4645d9edca54c6c6c13fee26b79480755cdfcdf39b4b5770b36bb03ce71ba002d5be8a27fe008af
2020-06-08 09:13:22 -04:00
Troy Giorshev
af2a145e57 Refactor resource exhaustion test
This is a simple refactor of the specified test.  It is now brought in
line with the rest of the tests in the module.  This should make things
easier to debug, as all of the tests are now grouped together at the
top.
2020-06-08 08:52:39 -04:00
MarcoFalke
fa98e10d5e test: Remove leftover comment in mining_basic 2020-06-08 08:10:37 -04:00
MarcoFalke
faedb50d89 test: pep-8 mining_basic
Can be reviewed with the git options
--word-diff-regex=. --ignore-all-space  -U0
2020-06-08 08:10:23 -04:00
fanquake
9573d2b55b Merge #19194: util: Don't reference errno when pthread fails.
cb38b069b0 util: Don't reference errno when pthread fails. (MIZUTA Takeshi)

Pull request description:

  Pthread library does not set errno.
  Pthread library's errno is returned by return value.

ACKs for top commit:
  practicalswift:
    ACK cb38b069b0 -- patch looks correct
  MarcoFalke:
    review ACK cb38b069b0
  hebasto:
    ACK cb38b069b0, only squashed commits since the [previous](https://github.com/bitcoin/bitcoin/pull/19194#pullrequestreview-425831739) review.

Tree-SHA512: e6c950e30726e5031db97a7b84c8a9215da5ad3e5d233bcc349f812ad15957ddfe378e26d18339b9e0a5dcac2f50b47a687b87a6a6beaf6139df84f31531321e
2020-06-08 19:36:28 +08:00
MarcoFalke
41fb69404c Merge #19192: doc: Extract net permissions doc
fa2c2b50d8 doc: Extract net permissions doc (MarcoFalke)

Pull request description:

  Moving the documentation of each flag form the already over-large init.cpp into the net permissions module should clean up the code a bit. Moreover, making the documentation available is also required for an (currently imaginary) `setnetpermissions` RPC.

ACKs for top commit:
  Sjors:
    re-utACK fa2c2b50d8

Tree-SHA512: c0a75facc9768913c28d2ffcdfaad8d60f7604d5584ee546adaf77d270563558d361aeaf354e49e349aca7e2e80814b27ffc24247e7b4f045c63cbdc079b449f
2020-06-08 07:27:58 -04:00
MarcoFalke
374fd6fc8b Merge #19189: refactor: Replace RecursiveMutex with Mutex in timedata.cpp
cc5c0d2299 refactor: Fix formatting of timedata.cpp (Hennadii Stepanov)
c2410ceb84 refactor: Replace RecursiveMutex with Mutex in timedata.cpp (Hennadii Stepanov)

Pull request description:

  Only `GetTimeOffset()` and `AddTimeData()` functions lock this mutex. They do not call itself recursively, and do not call each other either directly or indirectly. Therefore, the `g_timeoffset_mutex` could be a non-recursive mutex.

  Related to #19180.

ACKs for top commit:
  MarcoFalke:
    ACK cc5c0d2299 , checked the second commit with  --word-diff-regex=. --ignore-all-space -U0 🦉
  vasild:
    ACK cc5c0d22 verified that recursion is not happening

Tree-SHA512: 38f6df689374d4a1a0e9aedb3ed5e885d8285c4da6b75f9bc84ae036936a159ef8276462db33b4f4dd5c71c6312fa9b45380f7a5726959665bc71dc39031be88
2020-06-08 07:04:27 -04:00
MarcoFalke
8496dbeba6 Merge #19190: refactor: Replace RecursiveMutex with Mutex in netbase.cpp
78c8f4fe11 refactor: Replace RecursiveMutex with Mutex in netbase.cpp (Hennadii Stepanov)

Pull request description:

  The functions that could lock this mutex, i.e., `{S,G}etProxy()`, `{S,G}etNameProxy()`, `HaveNameProxy()`, `IsProxy()`, do not call itself recursively, and do not call each other either directly or indirectly. Therefore, the `g_proxyinfo_mutex` could be a non-recursive mutex.

  Related to #19180.

ACKs for top commit:
  MarcoFalke:
    ACK 78c8f4fe11 , reviewed with the -W git option 👮
  vasild:
    ACK 78c8f4fe verified that recursion does not happen

Tree-SHA512: fc077fb371f38af5d05f1383c6bebf9926167c257892936fefd2d4fe6f679ca40124d25099e09f645d8ec266df222f96c5d0f9fd39eddcad15cbde0b427bc205
2020-06-08 07:01:14 -04:00
MarcoFalke
fa2c2b50d8 doc: Extract net permissions doc 2020-06-08 06:59:03 -04:00
MarcoFalke
399a0d9dc7 Merge #19180: refactor: Replace RecursiveMutex with Mutex in Shutdown()
1a9ef1d398 refactor: Replace RecursiveMutex with Mutex in Shutdown() (Hennadii Stepanov)

Pull request description:

  Step by step, going to replace all of the `RecursiveMutex` instances with the `Mutex` ones throughout the code base :)

  Not sure if it is possible in all cases though...

  This one is a low-hanging fruit.

ACKs for top commit:
  MarcoFalke:
    ACK 1a9ef1d398 Shutdown is not recursive, so the same thread can never lock twice (UB)
  vasild:
    ACK 1a9ef1d3 verified manually that `Shutdown()` is not called from places that could be called from inside `Shutdown()`.

Tree-SHA512: 362a507b1a6f97dc351f708224aedbfe4bee03c4398f394d78ee31c24d76a7012ffff0e6766866cd5fd9a8e0d8840f05a2741111fe583aa20d45f0af3df0dcfa
2020-06-08 06:55:00 -04:00
Jon Atack
bc01f7ae05 doc: release note for rpc getaddressinfo removals 2020-06-08 10:38:36 +02:00
Jon Atack
90e989390e rpc: getaddressinfo RPCResult fixup 2020-06-08 10:38:34 +02:00
Jon Atack
a8507c99da rpc: remove deprecated getaddressinfo labels: purpose 2020-06-08 10:38:31 +02:00
Jon Atack
645a8653c8 rpc: remove deprecated getaddressinfo label field 2020-06-08 10:38:29 +02:00
MIZUTA Takeshi
cb38b069b0 util: Don't reference errno when pthread fails.
Pthread library does not set errno.
Pthread library's errno is returned by return value.

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2020-06-08 16:37:59 +09:00
fanquake
807b9f8114 Merge #19188: test: Avoid overwriting the NodeContext member of the testing setup [-Wshadow-field]
fac6b9b938 test: Avoid overwriting the NodeContext member of the testing setup (MarcoFalke)
fa16e7816b build: Add -Wshadow-field (MarcoFalke)

Pull request description:

  Adding this warning will eliminate unexpected test failures and hard to review code. Moreover, there shouldn't be a use case in Bitcoin Core that relies on fields to be shadowed.

ACKs for top commit:
  fanquake:
    ACK fac6b9b938 - Warnings compiling fa16e7816b are below. No warnings with fac6b9b938. The `-Wshadow-field` diagnostic has been available in Clang since 5.0.0. It's not available for GCC.
  practicalswift:
    ACK fac6b9b938 -- patch looks correct
  hebasto:
    ACK fac6b9b938, tested on Linux Mint 19.3 (x86_64):

Tree-SHA512: 824874ca10877efea7463cf934a2953147f3f99c486f04679426c14ff968975d8652cbba5729bfb7cb2c86c637ade5d1e5b873d611c06bad013a7cad8427e2bf
2020-06-08 13:47:58 +08:00
Troy Giorshev
5c4648d17b Fix "invalid message size" test
This test originally made a message with an invalid stated length, and
an invalid checksum.  This was because only the header was changed, but
the checksum stayed the same.  This was fine for now because we check
the header first to see if it has a valid stated size, and we disconnect
if it does not, so we never end up checking for the checksum.  If this
behavior was to change, this test would become a problem.  (Indeed I
discovered this when playing around with this behavior).  By instead
creating a message with an oversized payload from the start, we create a
message with an invalid stated length but a valid checksum, as intended.

Additionally, this takes advantage to the newly module-global
VALID_DATA_LIMIT as opposed to the magic 0x02000000.  Yes, 4MB < 32MiB,
but at the moment when receiving a message we check both, so this makes
the test tighter.
2020-06-08 00:55:34 -04:00
Troy Giorshev
ff1e7b8844 Move size limits to module-global
As well, this renames those variables to match PEP8 and this clears up
the comment relating to VALID_DATA_LIMIT.

Admittedly, this commit is mainly to make the following ones cleaner.
2020-06-08 00:54:52 -04:00
Troy Giorshev
57890abf2c Remove two unneeded tests
Test 1 is a duplicate of test_size() later in the file.  Inexplicably,
this test does not work on macOS, whereas test_size() does.

Test 2 is problematic for two reasons.  First, it always fails with an
invalid checksum, which is probably not what was intended.  Second, it's
not defined at this layer what the behavior should be.  Hypothetically,
if this test was fixed so that it gave messages with valid checksums,
then the message would pass successfully thought the network layer and
fail only in the processing layer.  A priori the network layer has no
idea what the size of a message "actually" is.

The "Why does behavior change at 78 bytes" is because of the following:

print(len(node.p2p.build_message(msg))) # 125
=> Payload size = 125 - 24 = 101
If we take 77 bytes, then there are 101 - 77 = 24 left
That's exactly the size of a header
So, bitcoind deserializes the header and rejects it for some other reason
(Almost always an invalid size (too large))
But, if we take 78 bytes, then there are 101 - 78 = 23 left
That's not enough to fill a header, so the socket stays open waiting for
more data.  That's why we sometimes have to push additional data in
order for the peer to disconnect.

Additionally, both of these tests use the "conn" variable.  For fun, go
look at where it's declared.  (Hint: test_large_inv().  Don't we all
love python's idea of scope?)
2020-06-08 00:54:43 -04:00
MarcoFalke
b3091b2be7 Merge #19202: log: remove deprecated db log category
c514a4f59a doc: release note for `db` log category removal (Jon Atack)
4c0c89307d log: remove deprecated `db` log category (Jon Atack)

Pull request description:

  The `db` log category was renamed to `walletdb` (like `coindb`) in #17410 and its upcoming removal announced in the 0.20 release notes.

  ```
  - The `-debug=db` logging category has been renamed to
    `-debug=walletdb` to distinguish it from `coindb`.  The `-debug=db`
    option has been deprecated and will be removed in the next major
    release.  (#17410)
  ```

  This PR removes the warning and reverts to the usual behavior for an unrecognised log category.
  ```
  $ bitcoin-cli logging '["db"]'
  error code: -8
  error message:
  unknown logging category db
  ```
  ```
  $ ./src/bitcoind -debug=db
  Warning: Unsupported logging category -debug=db.
  2020-06-07T15:30:45Z Bitcoin Core version v0.20.99.0-4c0c89307d (debug build)
  2020-06-07T15:30:45Z Warning: Unsupported logging category -debug=db.
  2020-06-07T15:30:45Z Assuming ancestors of block 0000000000000000000f2adce67e49b0b6bdeb9de8b7c3d7e93b21e7fc1e819d have valid signatures.
  2020-06-07T15:30:45Z Setting nMinimumChainWork=00000000000000000000000000000000000000000e1ab5ec9348e9f4b8eb8154
  2020-06-07T15:30:45Z Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
  2020-06-07T15:30:45Z Using RdSeed as additional entropy source
  ```

ACKs for top commit:
  MarcoFalke:
    ACK c514a4f59a 🔄

Tree-SHA512: fd62fd7ae0dc65446ba4401d75b4047e055396a33f7f1b176e79a7753250aec2a474ae604163d3f7e68710443c0ed2f45e44435d15f35612d794807e2142d5a3
2020-06-07 12:31:01 -04:00
Jon Atack
c514a4f59a doc: release note for db log category removal 2020-06-07 17:59:55 +02:00
Jon Atack
4c0c89307d log: remove deprecated db log category 2020-06-07 17:03:49 +02:00
MarcoFalke
43695b0cf8 Merge #19201: ci: Switch to bitcoincore.org download
fa4cd1fdae ci: Switch to bitcoincore.org download (MarcoFalke)

Pull request description:

  bitcoin.org is down and not in our control, so it seems odd to rely on it for our ci infrastructure

ACKs for top commit:
  troygiorshev:
    ACK fa4cd1f

Tree-SHA512: f9f0e9c69a52b8b1906ceae195e8bcc189799fb39be921b26e3a37d1f8f3999831f86c96c3546848c0d01429c36cfb2d7c5f314655ac5282d3e8e4cdd838960e
2020-06-07 10:41:24 -04:00
MarcoFalke
fa4cd1fdae ci: Switch to bitcoincore.org download 2020-06-07 10:30:38 -04:00
Hennadii Stepanov
cc5c0d2299 refactor: Fix formatting of timedata.cpp 2020-06-07 14:04:00 +03:00
MarcoFalke
1b90a7b61a Merge #19005: doc: Add documentation for 'checklevel' argument in 'verifychain' RPC…
501e6ab4e7 doc: Add documentation for 'checklevel' argument in 'verifychain' RPC call (Calvin Kim)

Pull request description:

  Rationale: When ```bitcoin-cli help verifychain``` is called, the user doesn't get any documentation about the ```checklevel``` argument, leading to issues like #18995.

  This PR addresses that issue and adds documentation for what each level does, and that each level includes the checks of the previous levels.

ACKs for top commit:
  jonatack:
    ACK 501e6ab4e7 `git diff 292ed3c 501e6ab` shows only change since last review is the verifychain RPCHelpMan edit; rebuild and retested manually anyway
  MarcoFalke:
    ACK 501e6ab4e7 🚝

Tree-SHA512: 09239f79c25b5c3022b8eb1f76198ba681305d7e8775038e46becffe5f6a14c572e0c5d06b0723fe9d4a015ec42c9f7ca7b80a2a93df0b1b66f5a84a80eeeeb1
2020-06-07 06:41:31 -04:00
Calvin Kim
501e6ab4e7 doc: Add documentation for 'checklevel' argument in 'verifychain' RPC call 2020-06-07 17:50:22 +09:00
Martin Zumsande
45eff751c6 Add functional test for P2P eviction logic of inbound peers 2020-06-06 22:30:29 +02:00
Hennadii Stepanov
78c8f4fe11 refactor: Replace RecursiveMutex with Mutex in netbase.cpp 2020-06-06 17:19:52 +03:00
MarcoFalke
b1b1739944 Merge #18968: doc: noban precludes maxuploadtarget disconnects
fa9604c46f doc: noban precludes maxuploadtarget disconnects (MarcoFalke)
fa3999fe35 net: Reformat excessively long if condition into multiple lines (MarcoFalke)

Pull request description:

  Whitelisting has been replaced by permission flags, so properly document this. See also #10131

ACKs for top commit:
  hebasto:
    ACK fa9604c46f, I have reviewed the code and it looks OK, I agree it can be merged.
  ariard:
    ACK fa9604c

Tree-SHA512: 5aee917ab9817719f01ec155487542118e17fa3d145ae7e4bc0e872b2cec39cde9e7fbdee2ae77e9a52700dd8bcc366de4224152e08e709d44d08e0d2f19c613
2020-06-06 09:51:21 -04:00
MarcoFalke
fac6b9b938 test: Avoid overwriting the NodeContext member of the testing setup 2020-06-06 09:50:32 -04:00
Hennadii Stepanov
c2410ceb84 refactor: Replace RecursiveMutex with Mutex in timedata.cpp 2020-06-06 16:41:02 +03:00
MarcoFalke
fa16e7816b build: Add -Wshadow-field 2020-06-06 08:12:37 -04:00
fanquake
17cfa52d38 Merge #19172: test: Do not swallow flake8 exit code
5d77549d8b doc: Add mypy to test dependencies (Hennadii Stepanov)
7dda912e1c test: Do not swallow flake8 exit code (Hennadii Stepanov)

Pull request description:

  After #18210 the `flake8` exit code in `test/lint/lint-python.sh` just not used that makes the linter broken.

  This PR:
  - combines exit codes of `flake8` and `mypy` into the  `test/lint/lint-python.sh` exit code
  - documents `mypy` as the test dependency

ACKs for top commit:
  MarcoFalke:
    Approach ACK 5d77549d8b, fine with me
  practicalswift:
    ACK 5d77549d8b

Tree-SHA512: e948ba04dc4d73393967ebf3c6a26c40d428d33766382a0310fc64746cb7972e027bd62e7ea76898b742a656cf7d0fcda2fdd61560a21bfd7be249cea27f3d41
2020-06-06 14:25:08 +08:00
João Barbosa
3351c91ed4 refactor: Make CScriptVisitor stateless 2020-06-06 00:41:09 +01:00
pasta
a99a3c0bd6 rpc: Validate provided keys for query_options parameter in listunspent
With this change listunspent will throw an error if there is a wrong key
in the query_option object.

Signed-off-by: pasta <pasta@dashboost.org>
2020-06-05 15:01:26 -05:00
Hennadii Stepanov
1a9ef1d398 refactor: Replace RecursiveMutex with Mutex in Shutdown() 2020-06-05 22:04:57 +03:00
John Newbery
62068381a3 [tests] Make mininode_lock non-reentrant
There's no need for mininode_lock to be reentrant.
Use a simpler non-recursive lock.
2020-06-05 11:01:54 -04:00
John Newbery
c67c1f2c03 [tests] Don't call super twice in P2PTxInvStore.on_inv() 2020-06-05 10:59:40 -04:00
John Newbery
9d80762fa0 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() 2020-06-05 10:58:25 -04:00
John Newbery
edae6075aa [tests] Only acquire lock once in p2p_compactblocks.py 2020-06-05 10:57:13 -04:00
Hennadii Stepanov
5d77549d8b doc: Add mypy to test dependencies 2020-06-05 16:16:16 +03:00
Hennadii Stepanov
7dda912e1c test: Do not swallow flake8 exit code 2020-06-05 16:15:31 +03:00
Russell Yanofsky
f42f5e58f5 refactor: Combine GetWalletForJSONRPCRequest and EnsureWalletIsAvailable functions
This simplifies control flow and also helps get rid of the ::vpwallets
variable, because EnsureWalletIsAvailable doesn't have access to the request
context.
2020-06-05 08:29:18 -04:00
MarcoFalke
0fc6ea216c Merge #19096: Remove g_rpc_chain global
4a7253ab6c Remove g_rpc_chain global (Russell Yanofsky)
e783197bf0 refactor: replace RegisterWalletRPCCommands with GetWalletRPCCommands (Russell Yanofsky)

Pull request description:

  Replace with RPC request reference to new WalletContext struct similar to the existing NodeContext struct and reference.

  This PR is a followup to #18740 removing the g_rpc_node global.

  Some later PRs will follow this up and move more wallet globals to the WalletContext struct.

ACKs for top commit:
  MarcoFalke:
    ACK 4a7253ab6c 🎋
  ariard:
    Code Review ACK 4a7253a, feel free to ignore comment it's super nit.

Tree-SHA512: 5bb5561c89f81811ca5232a58bf450e230d4218e62471c03227d142395fd36131672e99cb88329b33b9680a235db01e8b9d1c1e2a18288349e57205528deabab
2020-06-05 08:29:18 -04:00
MarcoFalke
aa35ea5502 Merge #19173: build: turn on --enable-c++17 by --enable-fuzz
0012471391 build: turn on --enable-c++17 by --enable-fuzz (Vasil Dimov)

Pull request description:

  Fuzzing code uses C++17 specific code (e.g. std::optional), so it is not
  possible to compile with --enable-fuzz and without --enable-c++17.

  Thus, turn on --enable-c++17 whenever --enable-fuzz is used.

ACKs for top commit:
  hebasto:
    ACK 0012471391, tested on Linux Mint 19.3 (x86_64); verified that it fails to compile with `--enable-fuzz` and without `--enable-c++17` on master.

Tree-SHA512: 290531ea8d79de3b9251ea4ad21e793478b18150cc0124eea1e50c3a4ed92bab89c3e70ed0aa526906f8723ea952cdba4268f1560ae4be9bd25b9e4f9b97436c
2020-06-05 07:15:44 -04:00
Vasil Dimov
0012471391 build: turn on --enable-c++17 by --enable-fuzz
Fuzzing code uses C++17 specific code (e.g. std::optional), so it is not
possible to compile with --enable-fuzz and without --enable-c++17.

Thus, turn on --enable-c++17 whenever --enable-fuzz is used.
2020-06-05 11:50:34 +02:00
fanquake
b55b5b6c3d Merge #19164: ci: tsan with wallet
fa7e002d52 ci: tsan with wallet (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa7e002d52 -- patch looks correct and Travis is happy
  hebasto:
    ACK fa7e002d52, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 1138459bbef72f402f32dae1e28d96f174901d4248d959b538b973747c8b06f221ecd81a386a1f915c0a2faaefb9fb8f11e3be39e6c5e2468bf9ae43d9f97757
2020-06-05 16:17:27 +08:00
Jonas Schnelli
f4f2220456 Merge #19132: qt: lock cs_main, m_cached_tip_mutex in that order
f46b678acf qt: lock cs_main, m_cached_tip_mutex in that order (Vasil Dimov)

Pull request description:

  Always lock the mutexes `cs_main` and `m_cached_tip_mutex` in
  the same order: `cs_main`, `m_cached_tip_mutex`. Otherwise we may end up
  in a deadlock.

  `ClientModel::m_cached_tip_blocks` is protected by
  `ClientModel::m_cached_tip_mutex`. There are two access paths that
  lock the two mutexes in opposite order:

  ```
  validation.cpp:2868 CChainState::ActivateBestChain(): lock cs_main
  validation.cpp:2916 CChainState::ActivateBestChain(): call uiInterface.NotifyBlockTip()
  ui_interface.cpp:52 CClientUIInterface::NotifyBlockTip(): go deep in boost
  ...
  qt/clientmodel.cpp:255 BlockTipChanged(): lock m_cached_tip_mutex
  ```

  and

  ```
  qt/clientmodel.cpp:119 ClientModel::getBestBlockHash(): lock m_cached_tip_mutex
  qt/clientmodel.cpp:121 ClientModel::getBestBlockHash(): call m_node.getBestBlockHash()
  interfaces/node.cpp:200 NodeImpl::getBestBlockHash(): lock cs_main
  ```

  From `debug.log`:

  ```
  POTENTIAL DEADLOCK DETECTED
  Previous lock order was:
   m_cs_chainstate validation.cpp:2851
   (1) cs_main validation.cpp:2868
   ::mempool.cs validation.cpp:2868
   (2) clientmodel->m_cached_tip_mutex qt/clientmodel.cpp:255
  Current lock order is:
   (2) m_cached_tip_mutex qt/clientmodel.cpp:119
   (1) ::cs_main interfaces/node.cpp:200
  ```

  The possible deadlock was introduced in #17993

ACKs for top commit:
  jonasschnelli:
    Tested ACK f46b678acf

Tree-SHA512: 904f24b39bdc97c4d0ecb897a6980d8d479814535eb167e23105238800ea2f1f85273e3370cf894db58bc597f94c4f2e81fb68d0ff3362d468c16af5ce8f5d78
2020-06-05 09:25:49 +02:00
Jonas Schnelli
7f9800caf9 Merge #15202: gui: Add Close All Wallets action
c4b574899a gui: Add Close All Wallets action (João Barbosa)
f30960adc0 gui: Add closeAllWallets to WalletController (João Barbosa)

Pull request description:

  This PR adds the action to close all wallets.

  <img width="405" alt="Screenshot 2020-06-01 at 01 06 12" src="https://user-images.githubusercontent.com/3534524/83365986-25a8b980-a3a4-11ea-9613-24dcd8eaa55c.png">

ACKs for top commit:
  jonasschnelli:
    Tested ACK c4b574899a

Tree-SHA512: 049ad77ac79949fb55f6bde47b583fbf946f4bfaf3d56d768e85f813d814cff0fe326b700f7b5e383cda4af7b5666e13043a6aaeee3798a69fc94385d88ce809
2020-06-05 09:18:46 +02:00
fanquake
4ede05d421 Merge #18758: Remove unused boost/thread
89f9fef1f7 refactor: Specify boost/thread/thread.hpp explicitly (Hennadii Stepanov)
fad8c890f5 txdb: Remove unused boost/thread (MarcoFalke)
faa958bc28 txindex: Remove unused boost/thread (MarcoFalke)

Pull request description:

  There are predefined interruption points for `boost::thread`: https://www.boost.org/doc/libs/1_71_0/doc/html/thread/thread_management.html#interruption_points

  However, non-boost threads such as `std::thread` or the `main()` thread can obviously not be interrupted. So remove all unused boost/thread from methods that are never executed in a `boost::thread`.

  Most of them were accompanied by a `ShutdownRequested` anyway. So even if the current thread was a `boost::thread`, the interruption point would be redundant. (We only interrupt threads during shutdown)

ACKs for top commit:
  fanquake:
    ACK 89f9fef1f7
  hebasto:
    ACK 89f9fef1f7, tested on Linux Mint 19.3 (x86_64), verified shutdown in different scenarios.

Tree-SHA512: 17221dadedf2d107e5bda9e4f371cc4f8ffce6ad27cae41aa2b8f1150d8f1adf23d396585ca4a2dd25b1dc6f0d5c81fecd950d8557966ccb45a6d4a85a331d90
2020-06-05 11:01:39 +08:00
MarcoFalke
fa7e002d52 ci: tsan with wallet 2020-06-04 18:26:01 -04:00
MarcoFalke
fa9604c46f doc: noban precludes maxuploadtarget disconnects 2020-06-04 16:39:23 -04:00
MarcoFalke
fa3999fe35 net: Reformat excessively long if condition into multiple lines
Can be reviewed with the git option
--word-diff-regex=.
2020-06-04 16:39:17 -04:00
MarcoFalke
01b45b2e01 Merge #19053: refactor: replace CNode pointers by references within net_processing.{h,cpp}
8b3136bd30 refactor: replace CNode pointers by references within net_processing.{h,cpp} (Sebastian Falbesoner)

Pull request description:

  This PR is inspired by a [recent code review comment](https://github.com/bitcoin/bitcoin/pull/19010#discussion_r426954791) on a PR that introduced new functions to the net_processing module. The point of the discussion was basically that whenever we pass something not by value (in the concrete example it was about `CNode*` and `CConnman*`) we should either use
  * a pointer (```CType*```) with null pointer check or
  * a reference (```CType&```)

  To keep things simple, this PR for a first approach
  * only tackles `CNode*` pointers
  * only within the net_processing module, i.e. no changes that would need adaption in other modules
  * keeps the names of the variables as they are

  I'm aware that PRs like this are kind of a PITA to review, but I think the code quality would increase if we get rid of pointers without nullptr check -- bloating up the code by adding all the missing checks would be the worse alternative, in my opinion.

  Possible follow-up PRs, in case this is received well:
  * replace CNode pointers by references for net module
  * replace CConnman pointers by references for net_processing module
  * ...

ACKs for top commit:
  MarcoFalke:
    ACK 8b3136bd30 🔻
  practicalswift:
    ACK 8b3136bd30

Tree-SHA512: 15b6a569ecdcb39341002b9f4e09b38ed4df077e3a3a50dfb1b72d98bdc9f9769c7c504f106456aa7748af8591af7bb836b72d46086df715ab116e4ac3224b3b
2020-06-04 14:45:32 -04:00
Hennadii Stepanov
d924f2a596 Drop MSG_NOPREFIX flag
Since bilingual_str type is fully supported, the MSG_NOPREFIX flag is no
longer needed.
2020-06-04 18:32:52 +03:00
Hennadii Stepanov
083daf7fba Pass bilingual_str argument to AbortNode() 2020-06-04 18:32:51 +03:00
Hennadii Stepanov
d1cca129b4 refactor: Use bilingual_str::empty() 2020-06-04 18:32:38 +03:00
Wladimir J. van der Laan
39afe5b1c6 Merge #19082: test: Moved the CScriptNum asserts into the unit test in script.py
7daffc6a90 [test] CScriptNum Decode Check as Unit Tests (Gillian Chu)

Pull request description:

  The CScriptNum test (#14816) is a roundtrip test of the test framework. Thus, it would be better suited as a unit test. This is now possible with the introduction of the unit test module for the functional tests. See #18576.

  This PR:
  1. Refactors the CScriptNum tests into 2 unit tests, one in script.py and one in blocktools.py.
  2. Extends the script.py CScriptNum test to trial larger numbers.

ACKs for top commit:
  laanwj:
    ACK 7daffc6a90

Tree-SHA512: 17a04a4bfff1b1817bfc167824c679455d9e06e6e0164c00a7e44f8aa5041c5f5080adcc1452fd80ba1a6d8409f976c982bc481d686c434edf97a5893a32a436
2020-06-04 17:28:55 +02:00
Wladimir J. van der Laan
365f1082e1 Merge #19112: rpc: Remove special case for unknown service flags
fa1433ac1b rpc: Remove special case for unknown service flags (MarcoFalke)

Pull request description:

  The special case to return a bit as an integer is clumsy and undocumented. Probably also irrelevant because there shouldn't currently be a non-misbehaving client that connects to Bitcoin Core and advertises an unknown service flag.

  Thus, simply remove the code.

ACKs for top commit:
  laanwj:
    ACK fa1433ac1b

Tree-SHA512: 942de6a577a9ee076ce12c92be121617640d53ee8c3424064c45a30a7ff789555d3722a4203670768faf81da2a40adfed3ec5cdeb5da06f04be81ddb53b9db7e
2020-06-04 17:16:49 +02:00
Wladimir J. van der Laan
011fe009f9 Merge #17994: validation: flush undo files after last block write
ac94141af0 validation: delay flushing undo files in syncing node case (Karl-Johan Alm)

Pull request description:

  Fixes #17890. Replaces #17892.

  Data files (`{blk|rev}<number>.dat`) pre-allocate space as they are written, and then trims down to the final size once they move on to the next sequence ("finalized flush"). The code currently assumes (incorrectly) that blk and rev files finish at the same time, but because blk files are written as blocks come in, and rev files are written in block height order, rev files end up being written to for awhile after moving on to the next block file, resulting in pre-allocation and waste of up to 1 MB of space per rev file.

  The exact point at which rev file writing finishes is the highest height block found inside the corresponding block file, which is already available in the CBlockFileInfo vector. This PR moves finalized flushing of undo files to to directly after the undo data for the previous block file has been written.

  There is a branch with annotation that demonstrates how this is handling flushing here: https://github.com/kallewoof/bitcoin/tree/200124-rev-files-annotated

ACKs for top commit:
  vasild:
    ACK ac94141af (no changes in the code since ed34e00da).
  fjahr:
    Code review re-ACK ac94141af0
  jonatack:
    Code review ACK ac94141af0

Tree-SHA512: 1d4e3b3d1d99bd7ebe7a2f632b1231146dd4f9f993c54db3a4090d9c086d95d2e4c327fd936066392b3afc6277b8f3a908d5c5993d4c8e49f72b92a417716dd2
2020-06-04 16:39:06 +02:00
Wladimir J. van der Laan
b46fb5cb10 Merge #19131: refactor: Fix unreachable code in init arg checks
eea8114657 build: Enable unreachable-code-loop-increment (Jonathan Schoeller)
d15db4b1fc refactor: Fix unreachable code in init arg checks (Jonathan Schoeller)

Pull request description:

  Closes: #19017

  In #19015 it's been suggested that we add some new compiler warnings to our build. Some of these, such as `-Wunreachable-code-loop-increment`, generate warnings. We'll likely want to fix these up if we're going to turn these warnings on.

  ```shell
  init.cpp:969:5: warning: loop will run at most once (loop increment never executed) [-Wunreachable-code-loop-increment]
       for (const auto& arg : gArgs.GetUnsuitableSectionOnlyArgs()) {
       ^~~
   1 warning generated.
   ```
   aa8d76806c/src/init.cpp (L968-L972)

  To fix this, collect all errors, and output them in a single error message after the loop completes. This resolves the unreachable code warning, and avoids popup hell that could result from outputting a seperate message for each error or warning one by one.

ACKs for top commit:
  laanwj:
    Code review ACK eea8114657
  hebasto:
    re-ACK eea8114657, only suggested changes applied since the [previous](https://github.com/bitcoin/bitcoin/pull/19131#pullrequestreview-421772387) review.

Tree-SHA512: 2aa3ceb7fab581b6ba2580900668388d8eba1c3001c8ff9c11c1f4a9a10fbc37f30e590249862676858446e3f4950140a252953ba1643ba3bfd772f8eae20583
2020-06-04 16:27:53 +02:00
Hennadii Stepanov
89f9fef1f7 refactor: Specify boost/thread/thread.hpp explicitly 2020-06-04 10:05:54 -04:00
MarcoFalke
fad8c890f5 txdb: Remove unused boost/thread 2020-06-04 10:05:36 -04:00
MarcoFalke
faa958bc28 txindex: Remove unused boost/thread 2020-06-04 10:05:22 -04:00
fanquake
584170a388 Merge #19142: validation: Make VerifyDB level 4 interruptible
fa3b4f9b8e validation: Make VerifyDB level 4 interruptible (MarcoFalke)
fa1d5800d9 validation: Remove unused boost interruption_point (MarcoFalke)

Pull request description:

  level 0,1,2, and 3 are already interruptible, so make level 4 also interruptible

ACKs for top commit:
  laanwj:
    Code review ACK fa3b4f9b8e
  fanquake:
    ACK fa3b4f9b8e

Tree-SHA512: d302c84a17add1b5993dd78339c88670d27eee45ce208c4d046ae188b50be9843ee5a9584739d5d25453b54ae08fd1cb6eeee8cb1307d84c05cde8a54a7c445b
2020-06-04 21:45:26 +08:00
fanquake
76e64525ff Merge #19159: test: Make valgrind.supp work on aarch64
fab7d954b2 test: Make valgrind.supp work on aarch64 (MarcoFalke)

Pull request description:

  Was easy to fix by simply removing a line

ACKs for top commit:
  practicalswift:
    ACK fab7d954b2 -- diff looks correct

Tree-SHA512: d2d7c6cac453a3177c20e256ec50a03066f8dbf5ae45299077ccf4a2b45a3a40252b1b5fcaf9224a59bb5c3df5bd90ac58af27eb0f47dc87c2640df5b2b460ca
2020-06-04 11:02:34 +08:00
fanquake
b42bc33d2d Merge #19152: build: improve build OS configure output
0fef60c63d build: improved output of configure for build OS (sachinkm77)

Pull request description:

  The purpose of this fix is to improve output of the configure script by providing the build OS. This is done by leveraging the build_os set by the script config.sub / config.guess. #18966

ACKs for top commit:
  fanquake:
    ACK 0fef60c63d - thanks for following up.

Tree-SHA512: b9f49df901a9d37eb16c67c063bb3611602a84391aa54d097a52b740f474c2785c24bf405522d15d724fde25070d354bf20b885add2ee4405a71cbe9ebab5ff3
2020-06-04 10:44:34 +08:00
MarcoFalke
a1c0e5fce1 Merge #19088: validation: use std::chrono throughout some validation functions
789e9dd3aa validation: use std::chrono in IsCurrentForFeeEstimation() (fanquake)
47be28c8bc validation: use std::chrono in CChainState::FlushStateToDisk() (fanquake)

Pull request description:

  Probably up for debate as to which type is used for the constants. Personally, swapping these to hours is more readable.

ACKs for top commit:
  MarcoFalke:
    ACK 789e9dd3aa
  jonatack:
    ACK 789e9dd3aa

Tree-SHA512: f4a25cbd00a49a54b7783a1f588be83706dd2a475cecb5c2e8b97b2d4b27c0955a7454d7486f2454e96351c44f233b300c4f4b9ca62fc7336277f10da34dd5c3
2020-06-03 13:13:54 -04:00
MarcoFalke
fab7d954b2 test: Make valgrind.supp work on aarch64
The equivalent suppression on aarch64 looks like:

{
   <insert_a_suppression_name_here>
   Memcheck:Param
   pwrite64(buf)
   fun:__libc_pwrite64
   fun:pwrite
   fun:__os_io
   obj:/usr/lib/aarch64-linux-gnu/libdb_cxx-5.3.so
   fun:__log_flush_int
   fun:__log_flush
   fun:__memp_sync_int
   fun:__db_sync
   fun:__db_refresh
   fun:__db_close
   fun:__fop_subdb_setup
   fun:__db_open
   fun:__db_open_pp
}
2020-06-03 10:58:27 -04:00
fanquake
234fabab90 Merge #18210: test: type hints in Python tests
bd7e530f01 This PR adds initial support for type hints checking in python scripts. (Kiminuo)

Pull request description:

  This PR adds initial support for type hints checking in python scripts.

  Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

  [Mypy](https://mypy.readthedocs.io/en/latest/index.html) is used in `lint-python.sh` to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

  **Notes:**

  * [--ignore-missing-imports](https://mypy.readthedocs.io/en/latest/command_line.html#cmdoption-mypy-ignore-missing-imports) switch is passed on to `mypy` checker for now. The effect of this is that one does not need `# type: ignore` for `import zmq`. More information about import processing can be found [here](https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports). This can be changed in a follow-up PR, if it is deemed useful.
  * We are stuck with Python 3.5 until 04/2021 (see https://packages.ubuntu.com/xenial/python3). When Python version is bumped to 3.6+, one can change:
      ```python
      _opcode_instances = []  # type: List[CScriptOp]
      ```
      to
      ```python
      _opcode_instances:List[CScriptOp] = []
      ```
      for type hints that are **not** function parameters and function return types.

  **Useful resources:**

  * https://docs.python.org/3.5/library/typing.html
  * https://www.python.org/dev/peps/pep-0484/

ACKs for top commit:
  fanquake:
    ACK bd7e530f01 - the type checking is not the most robust (there are things it fails to detect), but I think this is worth adopting (in a limited capacity while we maintain 3.5 compat).
  MarcoFalke:
    ACK bd7e530f01 fine with me

Tree-SHA512: 21ef213915fb1dec6012f59ef17484e6c9e0abf542a316b63d5f21a7778ad5ebabf8961ef5fc8e5414726c2ee9c6ae07c7353fb4dd337f8fcef5791199c8987a
2020-06-03 22:19:52 +08:00
Gillian Chu
7daffc6a90 [test] CScriptNum Decode Check as Unit Tests
Migrates the CScriptNum decode tests into a unit test, and moved some
changes made in #14816. Made possible by the integration of
test_framework unit testing in #18576. Further extends the original
test with larger ints, similar to the scriptnum_tests.cpp file. Adds
test to blocktools.py testing fn create_coinbase() with CScriptNum
decode.
2020-06-03 07:18:01 -07:00
fanquake
575589a62e Merge #19041: ci: tsan with -stdlib=libc++-10
faf62e6ed0 ci: Remove unused workaround (MarcoFalke)
fa7c850915 ci: Install llvm to get llvm symbolizer (MarcoFalke)
fa563cef61 test: Add more tsan suppressions (MarcoFalke)
fa0cc02c0a ci: Mute depends logs completely (MarcoFalke)
fa906bf298 test: Extend tsan suppressions for clang stdlib (MarcoFalke)
fa10d85079 ci: Use libc++ instead of libstdc++ for tsan (MarcoFalke)
fa0d5ee112 ci: Set halt_on_error=1 for tsan (MarcoFalke)
fa2ffe87f7 ci: Deduplicate DOCKER_EXEC (MarcoFalke)
fac2eeeb9d cirrus: Remove no longer needed install step (MarcoFalke)

Pull request description:

  According to the [ThreadSanitizer docs](https://clang.llvm.org/docs/ThreadSanitizer.html#current-status):
  >  C++11 threading is supported with **llvm libc++**.

  For example, the thread sanitizer build is currently not checking for double lock of mutexes.

  Fixes (partially) https://github.com/bitcoin/bitcoin/issues/19038#issuecomment-632138003

ACKs for top commit:
  practicalswift:
    ACK faf62e6ed0
  fanquake:
    ACK faf62e6ed0
  hebasto:
    ACK faf62e6ed0, maybe re-organize commits to modify suppressions in a single one?

Tree-SHA512: 98ce5154b4736dfb811ffdb6e6f63a7bc25fe50d3b73134404a8f3715ad53626c31f9c8132dbacf85de47b9409f1e17a4399e35f78b1da30b1577167ea2982ad
2020-06-03 21:55:32 +08:00
MarcoFalke
bdedfcf076 Merge #18974: test: Check that invalid witness destinations can not be imported
facede18a4 test: Check that invalid witness destinations can not be imported (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK facede18a4 -- thanks for adding!

Tree-SHA512: 87000606fac2e6f2780ca75cdeeb2dc1f0528d9b8f13e4156e8304ce7a6b1eb014781b6f0c59d11544bf360ba3dc5f99549470b0876132e189b9107f2c6bb38d
2020-06-03 07:31:06 -04:00
MarcoFalke
0f55294cc1 Merge #18875: fuzz: Stop nodes in process_message* fuzzers
fab860aed4 fuzz: Stop nodes in process_message* fuzzers (MarcoFalke)
6666c828e0 fuzz: Give CNode ownership to ConnmanTestMsg in process_message fuzz harness (MarcoFalke)

Pull request description:

  Background is that I saw an integer overflow in net_processing

  ```
  #30629113	REDUCE cov: 25793 ft: 142917 corp: 3421/2417Kb lim: 4096 exec/s: 89 rss: 614Mb L: 1719/4096 MS: 1 EraseBytes-
  net_processing.cpp:977:25: runtime error: signed integer overflow: 2147483624 + 100 cannot be represented in type 'int'
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior net_processing.cpp:977:25 in
  net_processing.cpp:985:9: runtime error: signed integer overflow: -2147483572 - 100 cannot be represented in type 'int'
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior net_processing.cpp:985:9 in
  ```

  Telling from the line numbers, it looks like `nMisbehavior` wrapped around.

  Fix that by calling `StopNodes` after each exec, which should clear the node state and thus `nMisbehavior`.

ACKs for top commit:
  practicalswift:
    ACK fab860aed4

Tree-SHA512: 891c081d5843565d891aec028b6c27ef3fa39bc40ae78238e81d8f784b4d4b49cb870998574725a5159dd03aeeb2e0b9bc3d3bb51d57d1231ef42e3394b2d639
2020-06-03 07:23:41 -04:00
Wladimir J. van der Laan
42065518d9 Merge #19146: doc: Remove release note fragments of 0.20.1 release, Add release-notes-0.20.0
fa13c180c7 doc: Move 0.21 fragments into the main release notes (MarcoFalke)
fa1a91657f doc: Add release-notes-0.20.0.md (MarcoFalke)
faabc6e445 doc: Remove release notes of 0.20.1 release (MarcoFalke)

Pull request description:

  Remove 0.20.1 fragments from master (which will become 0.21), also add the 0.20 release notes

ACKs for top commit:
  laanwj:
    ACK fa13c180c7
  fanquake:
    ACK fa13c180c7

Tree-SHA512: d5041f405cae255e6fde8d0346909a726fe41aa096478541e0ea3a45cadbb5c761b8da693aceb83f8e112575bc7e3cf896f2bf1f7b707240c6678cc48409ee2e
2020-06-03 13:05:06 +02:00
MarcoFalke
fa3b4f9b8e validation: Make VerifyDB level 4 interruptible 2020-06-03 06:06:58 -04:00
MarcoFalke
fa1d5800d9 validation: Remove unused boost interruption_point
ActivateBestChain (ABC) is only called in the "msghand" or one of the
RPC threads, neither of which is a boost::thread. However, ABC is also
called in ThreadImport (which currently happens to be a boost::thread).
In all cases, the interruption_point is redundant with the breakpoint in
ABC that triggers when ShutdownRequested()

VerifyDB is only called in the main thread ("init") or one of the RPC
threads, neither of which is a boost::thread.
2020-06-03 06:06:56 -04:00
sachinkm77
0fef60c63d build: improved output of configure for build OS 2020-06-03 04:06:36 -04:00
fanquake
657b82cef0 Merge #19084: net: improve code documentation for dns seed behaviour
5cb7ee67a5 net: improve code documentation for dns seed behaviour (Anthony Towns)

Pull request description:

  Some better internal documentation post #16939

ACKs for top commit:
  hebasto:
    ACK 5cb7ee67a5
  naumenkogs:
    ACK 5cb7ee6
  ariard:
    ACK 5cb7ee6
  fanquake:
    ACK 5cb7ee67a5 - thanks for following up.

Tree-SHA512: 5a12680651cd1e0436aed1cadcbf50047ffeabfdc9f7f2f81fa176c30b10673fc960154c73ec34ed08ace1a000a81cedd44d67587883152654dee46065991b45
2020-06-03 11:04:00 +08:00
MarcoFalke
fa13c180c7 doc: Move 0.21 fragments into the main release notes 2020-06-02 19:09:39 -04:00
MarcoFalke
fa1a91657f doc: Add release-notes-0.20.0.md 2020-06-02 18:16:51 -04:00
MarcoFalke
faabc6e445 doc: Remove release notes of 0.20.1 release 2020-06-02 18:16:07 -04:00
MarcoFalke
3657aee2d2 Merge #18982: wallet: Minimal fix to restore conflicted transaction notifications
7eaf86d3bf trivial: Suggested cleanups to surrounding code (Russell Yanofsky)
b604c5c8b5 wallet: Minimal fix to restore conflicted transaction notifications (Russell Yanofsky)

Pull request description:

  This fix is a based on the fix by Antoine Riard (ariard) in https://github.com/bitcoin/bitcoin/pull/18600.

  Unlike that PR, which implements some new behavior, this just restores previous wallet notification and status behavior for transactions removed from the mempool because they conflict with transactions in a block. The behavior was accidentally changed in two `CWallet::BlockConnected` updates: a31be09bfd and 7e89994133 from https://github.com/bitcoin/bitcoin/pull/16624, causing issue https://github.com/bitcoin/bitcoin/issues/18325.

  The change here could be improved and replaced with a more comprehensive cleanup, so it includes a detailed comment explaining future considerations.

  Fixes #18325

  Co-authored-by: Antoine Riard (ariard)

ACKs for top commit:
  jonatack:
    Re-ACK 7eaf86d3bf
  ariard:
    ACK 7eaf86d, reviewed, built and ran tests.
  MarcoFalke:
    ACK 7eaf86d3bf 🍡

Tree-SHA512: 9a1efe975969bb522a9dd73c41064a9348887cb67883cd92c6571fd2df4321b9f4568363891abdaae14a3b9b168ef8142e95c373fc04677e46289b251fb84689
2020-06-02 18:11:52 -04:00
fanquake
5879bfa9a5 Merge #18792: wallet: Remove boost from PeriodicFlush
fa1c74fd03 wallet: Remove unused boost::thread_interrupted (MarcoFalke)
fa7b885f51 walletdb: Remove unsed boost/thread (MarcoFalke)
5555d978b0 wallet: Make PeriodicFlush uninterruptible (MarcoFalke)

Pull request description:

  The `boost::this_thread::interruption_point()` in the code base currently block the replacement of `boost::thread` with `std::thread`. [1]

  Remove them from the wallet because they are either unused or useless.

  The feature to interrupt a periodic flush is useless because all wallets have just been flushed 9ccaee1d5e/src/init.cpp (L194) and another flush should be a noop. Also, they will be flushed again shortly after 9ccaee1d5e/src/init.cpp (L285), so even if repeated flushes weren't a noop, doing 3 instead of 2 shouldn't matter too much at this point. Also, the wallet is flushed every two seconds in the worst case, so if this is an expensive operation, that period should be readjusted. (Or bdb should be removed altogether #18916)

  [1] Replacement of `boost::thread` with `std::thread` should happen because:

  * The boost thread dependency is slow to compile
  * Boost thread is less maintained than the standard lib
  * Boost thread is mostly redundant to the standard lib
  * Global interruption points via exceptions are hard to keep track of during review and easy to get wrong during runtime (e.g. accidental `catch (...)`)

ACKs for top commit:
  fanquake:
    ACK fa1c74fd03

Tree-SHA512: b166619256de2ef4325480fa1367f68bc9371ad785ec503aed61eab41ba61f1a9807aab25451a24efda3db64855c9ba0025645b98bc58557bc3ec56c5b3297d0
2020-06-02 22:35:03 +08:00
practicalswift
20d31bdd92 tests: Avoid fuzzer-specific nullptr dereference in libevent when handling PROXY requests 2020-06-02 11:53:08 +00:00
MarcoFalke
9e8bd217cd Merge #13204: Faster sigcache nonce
152e8baf08 Use salted hasher instead of nonce in sigcache (Jeremy Rubin)
5495fa5850 Add Hash Padding Microbenchmarks (Jeremy Rubin)

Pull request description:

  This PR replaces nonces in two places with pre-salted hashers.

  The nonce is chosen to be 64 bytes long so that it forces the SHA256 hasher to process the chunk. This leaves the next 64 (or 56 depending if final chunk) open for data. In the case of the script execution cache, this does not make a big performance improvement because the nonce was already properly padded to fit into one buffer, but does make the code a little simpler. In the case of the sig cache, this should reduce the hashing overhead slightly because we are less likely to need an additional processing step.

  I haven't benchmarked this, but back of the envelope it should reduce the hashing by one buffer for all combinations except compressed public keys with compact signatures.

ACKs for top commit:
  ryanofsky:
    Code review ACK 152e8baf08. No code changes, just rebase since last review and expanded commit message

Tree-SHA512: b133e902fd595cfe3b54ad8814b823f4d132cb2c358c89158842ae27daee56ab5f70cde2585078deb46f77a6e7b35b4cc6bba47b65302b7befc2cff254bad93d
2020-06-02 07:32:15 -04:00
MarcoFalke
45a1489997 Merge #19122: test: Add missing sync_blocks to wallet_hd
fa7d3a8890 test: Add missing sync_blocks to wallet_hd (MarcoFalke)
eeeed51f58 test: pep-8 wallet_hd (MarcoFalke)

Pull request description:

  This fixes the `                                   test_framework.authproxy.JSONRPCException: non-final (-26)` error when node 1 is one block behind of node 0. (height 122 vs 123)

ACKs for top commit:
  promag:
    Code review ACK fa7d3a8890.

Tree-SHA512: b549dce2e08c58b949168ed2013bfa176802c963d0d7e890f643c8792da5dade14d91441dfa74372f1f1d34d696e8900a2b60b4861c0ba2dce99f2a633ab27ff
2020-06-02 07:13:51 -04:00
MarcoFalke
fa1c74fd03 wallet: Remove unused boost::thread_interrupted
FindWalletTx is only called by zapwallet, which is never called in a
boost::thread
2020-06-02 07:11:46 -04:00
MarcoFalke
1f7fe59460 Merge #19111: Limit scope of all global std::once_flag
fa9c675591 Limit scope of all global std::once_flag (MarcoFalke)

Pull request description:

  `once_flag` is  a helper (as the name might suggest) to execute a callable only once. Thus, the scope of the flag does never need to extend beyond where the callable is called. Typically this is function scope.

  Move all the flags to function scope to
  * simplify code review
  * avoid mistakes where similarly named flags are accidentally exchanged
  * avoid polluting the global scope

ACKs for top commit:
  hebasto:
    ACK fa9c675591, tested on Linux Mint 19.3 (x86_64).
  promag:
    Code review ACK fa9c675591.

Tree-SHA512: 095a0c11d93d0ddcb82b3c71676090ecc7e3de3d5e7a2a63ab2583093be279242acac43523bbae2060b4dcfa8f92b54256a0e91fbbae78fa92d2d49e9db62e57
2020-06-02 07:04:19 -04:00
Jon Atack
22cb303cf0 rpc: add missing space in JSON parsing error message, update test 2020-06-02 08:50:57 +02:00
Jon Atack
bf53ebef06 test: add multiwallet tests for bitcoin-cli -generate 2020-06-02 08:50:54 +02:00
Jon Atack
4b859cfff9 cli: add multiwallet capability to GetNewAddress and -generate 2020-06-02 08:50:51 +02:00
Jon Atack
18f93545a1 test: add tests for bitcoin-cli -generate 2020-06-02 08:50:48 +02:00
Jon Atack
4818124137 cli: create bitcoin-cli -generate command 2020-06-02 08:50:45 +02:00
Jon Atack
ff41a36900 cli: extract ParseResult() and ParseError()
and make callable higher up with (nRet == 0) check.
2020-06-02 08:50:41 +02:00
Harris
f4185b26d9 cli: create GenerateToAddressRequestHandler class 2020-06-02 08:50:36 +02:00
Jonas Schnelli
44307449f7 Merge #19104: gui, refactor: Register Qt meta types in application constructor
4f49d5222e gui, refactor: Register Qt meta types in application constructor (João Barbosa)

Pull request description:

  Removes a warning when running `QT_QPA_PLATFORM=cocoa src/qt/test/test_bitcoin-qt`.

ACKs for top commit:
  jonasschnelli:
    Re utACK 4f49d5222e
  hebasto:
    ACK 4f49d5222e, tested on macOS 10.15.5.

Tree-SHA512: e931a022ba83cb0ef04d82544ebd9b18242f8fc2b41443afce4d5c4222f222e8b3517bdb484a1a4f61377c5dceca067d8ccf250da3a727299448e54bec33ed6e
2020-06-02 08:48:04 +02:00
Jon Atack
f7c65a3350 cli: create GetNewAddress() 2020-06-02 08:24:29 +02:00
Jon Atack
9be7fd35c5 rpc: make generatetoaddress locals const
and named the same as in generatetodescriptor just above.
2020-06-02 08:23:34 +02:00
Kiminuo
bd7e530f01 This PR adds initial support for type hints checking in python scripts.
Support for type hints was introduced in Python 3.5. Type hints make it easier to read and review code in my opinion. Also an IDE may discover a potential bug sooner. Yet, as PEP 484 says: "It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention."

Mypy is used in lint-python.sh to do the type checking. The package is standard so there is little chance that it will be abandoned. Mypy checks that type hints in source code are correct when they are not, it fails with an error.

Useful resources:

* https://docs.python.org/3.5/library/typing.html
* https://www.python.org/dev/peps/pep-0484/
2020-06-02 08:03:02 +02:00
Sebastian Falbesoner
8b3136bd30 refactor: replace CNode pointers by references within net_processing.{h,cpp} 2020-06-02 01:42:55 +02:00
Jonathan Schoeller
eea8114657 build: Enable unreachable-code-loop-increment
Enable unreachable-code-loop-increment and treat as error.
refs: #19015
2020-06-02 06:24:10 +10:00
Jonathan Schoeller
d15db4b1fc refactor: Fix unreachable code in init arg checks
Building with -Wunreachable-code-loop-increment causes a warning
due to always returning on the first iteration of the loop that
outputs errors on invalid args.

Collect all errors, and output them in a single error message
after the loop completes, resolving the warning and avoiding
popup hell by outputting a seperate message for each error.
2020-06-02 06:20:04 +10:00
Vasil Dimov
f46b678acf qt: lock cs_main, m_cached_tip_mutex in that order
Always lock the mutexes `cs_main` and `m_cached_tip_mutex` in
the same order: `cs_main`, `m_cached_tip_mutex`. Otherwise we may end up
in a deadlock.

`ClientModel::m_cached_tip_blocks` is protected by
`ClientModel::m_cached_tip_mutex`. There are two access paths that
lock the two mutexes in opposite order:

```
validation.cpp:2868 CChainState::ActivateBestChain(): lock cs_main
validation.cpp:2916 CChainState::ActivateBestChain(): call uiInterface.NotifyBlockTip()
ui_interface.cpp:52 CClientUIInterface::NotifyBlockTip(): go deep in boost
...
qt/clientmodel.cpp:255 BlockTipChanged(): lock m_cached_tip_mutex
```

and

```
qt/clientmodel.cpp:119 ClientModel::getBestBlockHash(): lock m_cached_tip_mutex
qt/clientmodel.cpp:121 ClientModel::getBestBlockHash(): call m_node.getBestBlockHash()
interfaces/node.cpp:200 NodeImpl::getBestBlockHash(): lock cs_main
```

From `debug.log`:

```
POTENTIAL DEADLOCK DETECTED
Previous lock order was:
 m_cs_chainstate validation.cpp:2851
 (1) cs_main validation.cpp:2868
 ::mempool.cs validation.cpp:2868
 (2) clientmodel->m_cached_tip_mutex qt/clientmodel.cpp:255
Current lock order is:
 (2) m_cached_tip_mutex qt/clientmodel.cpp:119
 (1) ::cs_main interfaces/node.cpp:200
```

The possible deadlock was introduced in
https://github.com/bitcoin/bitcoin/pull/17993
2020-06-01 17:14:43 +02:00
Jon Atack
cb00510dba rpc: create rpc/mining.h, hoist default max tries values to constant 2020-06-01 15:08:36 +02:00
fanquake
9bc7751cad Merge #19115: doc: Add release notes for 17219
fac0ed16ec doc: Sync "how to upgrade" with 0.20.0 release notes (MarcoFalke)
fa861794d3 doc: Add release notes for 17219 (MarcoFalke)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK fac0ed16ec - seems to match the relevant changes in 79606b56b2, 36c1d981d4 and  4c17c85c6c

Tree-SHA512: f04c2b3a1cd094d7f50e3f8db06d726873d2412d651c94bf38d17fd5ee0c47c84480d6b20ec18641888ee88012e7c59e7e794467a0eed1caf44c0569feecc4ca
2020-06-01 16:07:05 +08:00
fanquake
a8327fd71f Merge #19072: doc: Expand section on Getting Started
facef3d413 doc: Explain that anyone can work on good first issues, move text to CONTRIBUTING.md (MarcoFalke)
fae2fb2a19 doc: Expand section on Getting Started (MarcoFalke)
100000d1b2 doc: Add headings to CONTRIBUTING.md (MarcoFalke)
fab893e0ca doc: Fix unrelated typos reported by codespell (MarcoFalke)

Pull request description:

  Some random doc changes:

  * Add sections to docs, so that they can be linked to
  * Explain that anyone (even maintainers) are allowed to work on good first issues
  * Expand section on Getting Started slightly

ACKs for top commit:
  hebasto:
    ACK facef3d413
  fanquake:
    ACK facef3d413

Tree-SHA512: 8998e273a76dbf4ca77e79374c14efe4dfcc5c6df6b7d801e1e1e436711dbe6f76b436f9cbc6cacb45a56827babdd6396f3bd376a9426ee7be3bb9b8a3b8e383
2020-06-01 15:38:57 +08:00
Jim Posen
f5c003d3ea [test] Add test for NODE_COMPACT_FILTER.
Test that a node configured to serve compact filters will signal
NODE_COMPACT_FILTER service bit.
2020-05-31 23:01:13 -04:00
Jim Posen
132b30d9c8 [net] Signal NODE_COMPACT_FILTERS if we're serving compact filters.
If -peerblockfilters is configured, signal the NODE_COMPACT_FILTERS service
bit to indicate that we are able to serve compact block filters, headers
and checkpoints.
2020-05-31 23:01:06 -04:00
John Newbery
b3fbc94d4f Apply cfilters review fixups 2020-05-31 22:58:42 -04:00
João Barbosa
c4b574899a gui: Add Close All Wallets action 2020-06-01 00:58:09 +01:00
João Barbosa
f30960adc0 gui: Add closeAllWallets to WalletController 2020-06-01 00:54:00 +01:00
MarcoFalke
a65b55fa45 Merge #18994: tests: Add fuzzing harnesses for functions in script/
f898ef65c9 tests: Add fuzzing harness for functions in script/sign.h (practicalswift)
c91d2f0615 tests: Add fuzzing harness for functions in script/sigcache.h (practicalswift)
d3d8adb79f tests: Add fuzzing harness for functions in script/interpreter.h (practicalswift)
fa80117cfd tests: Add fuzzing harness for functions in script/descriptor.h (practicalswift)
43fb8f0ca3 tests: Add fuzzing harness for functions in script/bitcoinconsensus.h (practicalswift)
8de72711c6 tests: Fill fuzzing coverage gaps for functions in script/script.h, script/script_error.h and script/standard.h (practicalswift)
c571ecb071 tests: Add fuzzing helper functions ConsumeDataStream, ConsumeTxDestination and ConsumeUInt160 (practicalswift)

Pull request description:

  Add fuzzing harnesses for functions in `script/`:
  * Add fuzzing helper functions `ConsumeDataStream` and `ConsumeUInt160`
  * Fill fuzzing coverage gaps for functions in `script/script.h`, `script/script_error.h` and `script/standard.h`
  * Add fuzzing harness for functions in `script/bitcoinconsensus.h`
  * Add fuzzing harness for functions in `script/descriptor.h`
  * Add fuzzing harness for functions in `script/interpreter.h`
  * Add fuzzing harness for functions in `script/sigcache.h`
  * Add fuzzing harness for functions in `script/sign.h`

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    ACK f898ef65c9 🔉

Tree-SHA512: f6e77b34dc79f23de5fa9e38ac06e6554b5b946ec3e9a67e2bd982e60aca37ce844f785457ef427a5e3b45e31c305456bca8587cc9f4a0b50b3852e39726eb04
2020-05-31 18:58:41 -04:00
Fabian Jahr
5c3eaf9983 doc: Add warnings for http interfaces limitations 2020-06-01 00:54:18 +02:00
MarcoFalke
faf62e6ed0 ci: Remove unused workaround 2020-05-31 18:37:56 -04:00
MarcoFalke
fa7c850915 ci: Install llvm to get llvm symbolizer 2020-05-31 18:37:43 -04:00
MarcoFalke
07d0e0d59f Merge #19044: net processing: Add support for getcfilters
9e36067d8c [test] Add test for cfilters. (Jim Posen)
11106a4722 [net processing] Message handling for getcfilters. (Jim Posen)
e535670726 [indexes] Fix default [de]serialization of BlockFilter. (Jim Posen)
bb911ae7f5 [refactor] Pass CNode and CConnman by reference (John Newbery)

Pull request description:

  Support `getcfilters` requests when `-peerblockfilters` is set.

  Does not advertise compact filter support in version messages.

ACKs for top commit:
  Empact:
    re-Code Review ACK 9e36067d8c
  MarcoFalke:
    re-ACK 9e36067d8c , only change is adding commit "[refactor] Pass CNode and CConnman by reference" 🥑
  jkczyz:
    ACK 9e36067d8c
  fjahr:
    Code review ACK 9e36067d8c

Tree-SHA512: b45b42a25905ef0bd9e195029185300c86856c87f78cbe17921f4a25e159ae0f6f003e61714fa43779017eb97cd89d3568419be88e47d19dc8095562939e7887
2020-05-31 18:20:17 -04:00
MarcoFalke
091cc4b94e Merge #16564: test: Always define the raii_event_tests test suite
9a19c9ada5 Always define the raii_event_tests test suite (Craig Andrews)

Pull request description:

  The test suite must always be defined (even when EVENT_SET_MEM_FUNCTIONS_IMPLEMENTED is not defined) so that the test harness doesn't fail due to not being able to find the raii_event_tests test.

  This improves upon 95f97f4 actually fixing https://github.com/bitcoin/bitcoin/issues/9493

ACKs for top commit:
  MarcoFalke:
    ACK 9a19c9ada5 🎹

Tree-SHA512: 3c42f17a9b5d56c8841f3aa9ac19da91c10aff210026266f31f7eb98a62528740d7c518c121452b68e8f801d6c80ecfb627d137ec6ed533289fa3beb08b4f176
2020-05-31 17:55:55 -04:00
MarcoFalke
fa563cef61 test: Add more tsan suppressions 2020-05-31 10:52:06 -04:00
MarcoFalke
fa7d3a8890 test: Add missing sync_blocks to wallet_hd 2020-05-31 07:51:44 -04:00
MarcoFalke
eeeed51f58 test: pep-8 wallet_hd 2020-05-31 07:36:22 -04:00
MarcoFalke
dec067f5a0 Merge #18965: tests: implement base58_decode
60ed33904c tests: implement base58_decode (10xcryptodev)

Pull request description:

  implements TODO: def base58_decode

ACKs for top commit:
  ryanofsky:
    Code review ACK 60ed33904c. Just suggested changes since last review. Thank you for taking suggestions!

Tree-SHA512: b3c06b4df041a6d88033cd077a093813a688e42d0b9aa777c715e5fd69cfba7b1bf984428bd98417d3c15232d3d48bc9c163317564f9e1d562db6611c21e2c10
2020-05-30 12:33:49 -04:00
MarcoFalke
fac0ed16ec doc: Sync "how to upgrade" with 0.20.0 release notes 2020-05-30 12:31:36 -04:00
MarcoFalke
fa861794d3 doc: Add release notes for 17219 2020-05-30 12:28:07 -04:00
MarcoFalke
826fe9c667 Merge #18807: [doc / test / mempool] unbroadcast follow-ups
9e1cb1adf1 [trivial/doc] Fix comment type (Amiti Uttarwar)
8f30260a67 [doc] Update unbroadcast description in RPC results (Amiti Uttarwar)
750456d6f2 [trivial] Remove misleading 'const' (Amiti Uttarwar)
fa32e676e5 [test] Manage node connections better in mempool persist test (Amiti Uttarwar)
1f94bb0c74 [doc] Provide rationale for randomization in scheduling. (Amiti Uttarwar)
9c8a55d9cb [mempool] Don't throw expected error message when upgrading (Amiti Uttarwar)
ba54983182 [test] Test that wallet transactions aren't rebroadcast before 12 hours (Amiti Uttarwar)
00d44a534b [test] P2P connection behavior should meet expectations (Amiti Uttarwar)
bd093ca15d [test] updates to unbroadcast test (Amiti Uttarwar)
dab298d9ab [docs] add release notes (Amiti Uttarwar)

Pull request description:

  This PR is a follow up to #18038 which introduced the idea of an unbroadcast set & focuses mostly on documentation updates and test fixes. One small functionality update to not throw an expected error in `LoadMempool` when you upgrade software versions.

  #18895 is another follow up to that addresses other functionality updates.

  Background context:
  The unbroadcast set is a mechanism for the mempool to track locally submitted transactions (via wallet or RPC). The node does a best-effort of delivering the transactions to the network via retries every 10-15 minutes until either a `GETDATA` is received or the transaction is removed from the mempool.

ACKs for top commit:
  MarcoFalke:
    ACK 9e1cb1adf1 👁
  gzhao408:
    ACK [`9e1cb1a`](9e1cb1adf1)

Tree-SHA512: 0cd51c4ca368b9dce92d50d73ec6e9df278a259e609eef2858f24cb8595ad07acc3db781d9eb0c351715f18fca5a2b4526838981fdb34a522427e9dc868bdaa6
2020-05-30 12:22:09 -04:00
MarcoFalke
fa0cc02c0a ci: Mute depends logs completely 2020-05-30 08:33:44 -04:00
MarcoFalke
fa906bf298 test: Extend tsan suppressions for clang stdlib 2020-05-30 08:33:06 -04:00
MarcoFalke
fa10d85079 ci: Use libc++ instead of libstdc++ for tsan 2020-05-30 08:33:02 -04:00
MarcoFalke
fa0d5ee112 ci: Set halt_on_error=1 for tsan 2020-05-30 08:32:58 -04:00
MarcoFalke
fa2ffe87f7 ci: Deduplicate DOCKER_EXEC 2020-05-30 08:32:33 -04:00
MarcoFalke
fac2eeeb9d cirrus: Remove no longer needed install step 2020-05-30 08:32:29 -04:00
MarcoFalke
e478b11db0 Merge #19110: test: Explain that a bug should be filed when the tests fail
fad21a1a7a test: Explain that a bug should be filed when the test fail (MarcoFalke)

Pull request description:

  Without a bug report it is harder to fix the issue

ACKs for top commit:
  hebasto:
    ACK fad21a1a7a, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged.
  fanquake:
    ACK fad21a1a7a

Tree-SHA512: db194e8f8c0f07b2f4c9ef27e456510959f89da69435cee71605d720e0ad06f18700973f5af25ea31a190b933eb35f2743f014878aa3f8293500e06b4907ebbd
2020-05-30 06:49:27 -04:00
practicalswift
f898ef65c9 tests: Add fuzzing harness for functions in script/sign.h 2020-05-30 10:37:01 +00:00
practicalswift
c91d2f0615 tests: Add fuzzing harness for functions in script/sigcache.h 2020-05-30 10:37:01 +00:00
practicalswift
d3d8adb79f tests: Add fuzzing harness for functions in script/interpreter.h 2020-05-30 10:37:01 +00:00
practicalswift
fa80117cfd tests: Add fuzzing harness for functions in script/descriptor.h 2020-05-30 10:37:01 +00:00
practicalswift
43fb8f0ca3 tests: Add fuzzing harness for functions in script/bitcoinconsensus.h 2020-05-30 10:37:01 +00:00
practicalswift
8de72711c6 tests: Fill fuzzing coverage gaps for functions in script/script.h, script/script_error.h and script/standard.h 2020-05-30 10:37:01 +00:00
fanquake
98ebe7896d Merge #18820: build: Propagate well-known vars into depends
f0d7ed10b4 depends: Propagate only specific CLI variables to sub-makes (Carl Dong)
0a33803f1c depends: boost: Use clang toolset if clang in CXX (Carl Dong)
1ce74bcde3 depends: boost: Split target-os from toolset (Carl Dong)
2d4e480813 depends: boost: Specify toolset to bootstrap.sh (Carl Dong)
3d6603e340 depends: Propagate well-known vars into depends (Carl Dong)

Pull request description:

  From: https://github.com/bitcoin/bitcoin/pull/18308#issuecomment-598301117

  The following monstrosity is quite useful when invoked inside `depends`, and reviewers can use it to compare the behaviour of this change against master.
  ```bash
  make print-{{,{host,{,{i686,x86_64,riscv64}_}linux}_}{CC,CXX},boost_{cc,cxx}}
  ```

  It would also be helpful to make sure that setting `HOST`, `CC`, and `CXX` does the right thing. The 3 hosts I found offered good coverage were: `{x86_64,i686,riscv64}-linux-gnu`. As we special-case the `x86_64` and `i686` hosts in `depends/hosts/linux.mk`, and `riscv64` is a sanity check for a non-special-cased host.

ACKs for top commit:
  hebasto:
    ACK f0d7ed10b4, tested on Linux Mint 19.3 (x86_64):
  practicalswift:
    ACK f0d7ed10b4 -- patch looks correct
  laanwj:
    Code review and concept ACK f0d7ed10b4
  ryanofsky:
    Code review ACK f0d7ed10b4. Changes since last review: adding comment explaining check for predefined make variables, dropping freetype commit, adding commit whitelisting overrides for recursive makes

Tree-SHA512: b6b8e76f713c26a0add6cd685824e2f5639109236ee9f89338f7c79cb1b1f2c3897bfb62b80b023d6d1943b5a6eb282a2f827f1f499c5e556eca015d6635fa65
2020-05-30 11:15:10 +08:00
MarcoFalke
fa1433ac1b rpc: Remove special case for unknown service flags 2020-05-29 18:17:18 -04:00
MarcoFalke
fa9c675591 Limit scope of all global std::once_flag 2020-05-29 17:22:07 -04:00
MarcoFalke
fad21a1a7a test: Explain that a bug should be filed when the test fail 2020-05-29 15:33:54 -04:00
MarcoFalke
cb88de3e3d Merge #18926: test: Pass ArgsManager into getarg_tests
f871f15c9d scripted-diff: replace gArgs with argsman (glowang)
357f02bf29 Create a local class inherited from BasicTestingSetup with a localized args manager and put it into the getarg_tests namespace (glowang)

Pull request description:

  Replaced the global argsManager gArgs with a locally defined one in getarg_tests. This is to avoid confusion in arg settings between the test's ArgsManager and the   #18804

ACKs for top commit:
  MarcoFalke:
    ACK f871f15c9d
  ryanofsky:
    Code review ACK f871f15c9d. Changes look good and thanks for updating. In future would recommend using clang-format-diff and following [coding style](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#coding-style-c) notes, because it's atypical to indent namespace content, or indent protected keywords or put spaces around ::. Also it's fragile to define test setup class in a namespace, but test setup methods outside of the namespace and inside the test fixture instead. Would be simpler to just define the testing setup completely before using it without a namespace like: 8ad5f1c376/src/test/rpc_tests.cpp (L23) and it would have been a slightly smaller change too.

Tree-SHA512: 016594639396d60667fadec8ea80ef7af634fbb2014c704f02406fe3251c5362757c21f1763d8bdb94ca4a3026ab9dc786a92a9a934efc8cd807655d9deee779
2020-05-29 14:23:43 -04:00
Jonas Schnelli
8ad5f1c376 Merge #19106: util: simplify the interface of serviceFlagToStr()
189ae0c38b util: dedup code in callers of serviceFlagToStr() (Vasil Dimov)
fbacad1880 util: simplify the interface of serviceFlagToStr() (Vasil Dimov)

Pull request description:

  Don't take two redundant arguments in `serviceFlagToStr()`.

  Introduce `serviceFlagsToStr()` which takes a mask (with more than one
  bit set) and returns a vector of strings.

  As a side effect this fixes an issue introduced in
  https://github.com/bitcoin/bitcoin/pull/18165 due to which the GUI could
  print something like `UNKNOWN[1033] & UNKNOWN[1033] & UNKNOWN[2^10]`
  instead of `NETWORK & WITNESS`.

ACKs for top commit:
  MarcoFalke:
    ACK 189ae0c38b
  jonasschnelli:
    Tested ACK 189ae0c38b

Tree-SHA512: 000c490f16ebbba04458c62ca4ce743abffd344d375d95f5bbd5008742012032787655db2874b168df0270743266261dccf1693761906567502dcbac902bda50
2020-05-29 19:43:08 +02:00
Vasil Dimov
189ae0c38b util: dedup code in callers of serviceFlagToStr()
Introduce `serviceFlagsToStr()` which hides the internals of the bitmask
and simplifies callers of `serviceFlagToStr()`.
2020-05-29 18:59:37 +02:00
Vasil Dimov
fbacad1880 util: simplify the interface of serviceFlagToStr()
Don't take two redundant arguments in `serviceFlagToStr()`.

As a side effect this fixes an issue introduced in
https://github.com/bitcoin/bitcoin/pull/18165 due to which the GUI could
print something like `UNKNOWN[1033] & UNKNOWN[1033] & UNKNOWN[2^10]`
instead of `NETWORK & WITNESS`.
2020-05-29 18:49:26 +02:00
practicalswift
c571ecb071 tests: Add fuzzing helper functions ConsumeDataStream, ConsumeTxDestination and ConsumeUInt160 2020-05-29 16:44:22 +00:00
Anthony Towns
5cb7ee67a5 net: improve code documentation for dns seed behaviour 2020-05-30 02:07:48 +10:00
MarcoFalke
c19fd96694 Merge #19022: test: Fix intermittent failure in feature_dbcrash
fa2ca0cbdd test: Fix intermittent failure in feature_dbcrash (MarcoFalke)

Pull request description:

  Example backtrace https://cirrus-ci.com/task/6005716207009792?command=functional_test#L817

ACKs for top commit:
  jnewbery:
    utACK fa2ca0cbdd

Tree-SHA512: 978b3ac222f4764c887719240cfd1b29f72cdd273a456345b631e622db0a38e345c25a70d0bae8d4063c1ff6c1af892a7ee37d0d66f47284c2524b663c3afe55
2020-05-29 11:28:00 -04:00
João Barbosa
4f49d5222e gui, refactor: Register Qt meta types in application constructor 2020-05-29 16:05:47 +01:00
Jonas Schnelli
e4bfd51acc Merge #18452: qt: Fix shutdown when waitfor* cmds are called from RPC console
da73f1513a qt: Fix shutdown when waitfor* cmds are called from RPC console (Hennadii Stepanov)

Pull request description:

  On master (7eed413e72), if the GUI has been started with`-server=1`, `bitcoin-qt` hangs on shutdown during calling any of the `waitfor*` commands in the GUI RPC console.

  This PR suggests minimal changes to fix this bug.

  Fix #17495

ACKs for top commit:
  jonasschnelli:
    utACK da73f1513a

Tree-SHA512: 469f5332945a5f2c57d19336cda5df79b123ccc494aea6d58a85eb1293be52708b2b9c5bb6bc2c402a90b7b4e9e8d7ab8fe84cf201cf7ce612c9290c57e43681
2020-05-29 15:49:46 +02:00
Jonas Schnelli
f2e2c5ebcc Merge #14988: wallet: Fix for confirmed column in csv export for payment to self transactions
9760293ce6 wallet: Fix for exported confirmation field in payment to self transactions (Ben Carman)

Pull request description:

  Closes #3455

ACKs for top commit:
  jonasschnelli:
    Tested ACK 9760293ce6

Tree-SHA512: 8207768771ad787f716b966c4aa7aeef2da8a602e32e3510e41c7b49ec5ec679a3835d248be5016d4b37764f9914846f7c41c11cf48cddb617cb7ef831318fd7
2020-05-29 14:42:48 +02:00
fanquake
73407ff65c Merge #16939: p2p: Delay querying DNS seeds
96954d1794 DNS seeds: don't query DNS while network is inactive (Anthony Towns)
fa5894f7f5 DNS seeds: wait for 5m instead of 11s if 1000+ peers are known (Anthony Towns)

Pull request description:

  Changes the logic for querying DNS seeds: after this PR, if there's less than 1000 entries in addrman, it will still usually query DNS seeds after 11s (unless the first few peers tried mostly succeed), but if there's more than 1000 entries it won't try DNS seeds until 5 minutes have passed without getting multiple outbound peers. (If there's 0 entries in addrman, it will still immediately query the DNS seeds). Additionally, delays querying DNS seeds while the p2p network is not active.

  Fixes #15434

ACKs for top commit:
  fanquake:
    ACK 96954d1794 - Ran some tests of different scenarios. More documentation is being added in #19084.
  ariard:
    Tested ACK 96954d1, on Debian 9.1. Both MANY_PEERS/FEW_PEERS cases work.
  Sjors:
    tACK 96954d1 (rebased on master) on macOS 10.15.4. It found it useful to run with `-debug=addrman` and change `DNSSEEDS_DELAY_MANY_PEERS` to something lower to test the behaviour, as well as renaming `peers.dat` to test the peer threshold.
  naumenkogs:
    utACK 96954d1794

Tree-SHA512: 73693db3da73bf8e76c3df9e9c82f0a7fb08049187356eac2575c4ffa455f76548dd1c86a11fc6beea8a3baf0ba020e047bebe927883c731383ec72442356005
2020-05-29 18:24:09 +08:00
Jonas Schnelli
d7f5e01e30 Merge #18424: qt: Use parent-child relation to manage lifetime of OptionsModel object
8e08d00598 qt: Use parent-child relation to manage lifetime of OptionsModel object (Hennadii Stepanov)

Pull request description:

  Both `BitcoinApplication` and `OptionsModel` classes are derived from the `QObject` class, therefore a parent-child relation could be established to manage the lifetime of an `OptionsModel` object:
  5236b2e267/src/qt/optionsmodel.cpp (L29-L30)

  This PR does not change behavior.

ACKs for top commit:
  jonasschnelli:
    utACK 8e08d00598
  promag:
    ACK 8e08d00598.

Tree-SHA512: 0223dddf5ba28b0bfaefeda1b03b4ff95bf7e7d0c1e7b32368171e561813e22129f2a664f09279fa3b4fa63259b7680d55aa3fe66db9c7ae0039b7f529777ec3
2020-05-29 11:09:09 +02:00
Jonas Schnelli
de369c7ea5 Merge #18165: Consolidate service flag bit-to-name conversion to a shared serviceFlagToStr function
c31bc5bcfd Consolidate service flag bit-to-name conversion to a shared serviceFlagToStr function (Luke Dashjr)
cea91a1e40 Bugfix: GUI: Use unsigned long long type to avoid implicit conversion of MSB check (Luke Dashjr)

Pull request description:

  Side effect: this results in the RPC showing unknown service bits as "UNKNOWN[n]" like the GUI.

  Note that there is no common mask-to-`vector<string>` function because both GUI and RPC would need to iterate through it to convert to their desired target formats.

ACKs for top commit:
  jonasschnelli:
    utACK ~~cea91a1e40e12029140ebfba969ce3ef2965029c~~ c31bc5bcfd

Tree-SHA512: 32c7ba8ac7ef2d4087f4f317447ae93a328ec9fb9ad81301df2fbaeeb21a3db7a503187a369552b05a9414251b7cf8e15bcde74c1ea2ef36591ea7ffb6721f60
2020-05-29 11:06:08 +02:00
Jonas Schnelli
f4b603cff6 Merge #17993: gui: Balance/TxStatus polling update based on last block hash.
a06e845e82 BlockTip struct created and connected to notifyHeaderTip and notifyBlockTip signals. (furszy)
2f867203b0 Added best block hash to the NotifyHeaderTip and NotifyBlockTip signals. (furszy)

Pull request description:

  Rationale:

  The height based polling in the GUI is an issue on chain reorgs. Any new tip signal with the same height as the one that it's cached in the model was not triggering the GUI update (interpreting it as the same same block when it could receive a different one). Ending up with bad information presented in the GUI.

  This PR essentially changes the last cached height to be a last cached block hash.

  ---

  Old historical information of this PR.

  As the tip height is cached and updated via signaling in clientModel, there is no need to continue locking `cs_main` on every balance poll (`m_node.getNumBlocks()` method call).

  Extra topic:

  Would suggest to change the `cachedNumBlocks` field inside `walletModel` to a more understandable name, maybe `nLastBalanceUpdateHeight`.

  And finally, this will have the equal height reorg issue mentioned [here](https://github.com/bitcoin/bitcoin/pull/17905#issuecomment-577324304), whatever is presented to fix it, this should use the same flow too.

  **[Edit - 24/01/2020]**

  Have added #[17905](https://github.com/bitcoin/bitcoin/pull/17905#issuecomment-577324304) comment fix here too.

ACKs for top commit:
  jonasschnelli:
    utACK a06e845e82 - it would be great to have QT unit tests  (in this case for a reorg) that either automatically inspect the window content based on accessibility and tests for expected values or at least allow for quick manual re-testing (screenshots, automatically create UI situations).
  hebasto:
    re-ACK a06e845e82, suggested style changes implemented since the [previous](https://github.com/bitcoin/bitcoin/pull/17993#pullrequestreview-417249705) review.
  ryanofsky:
    Code review ACK a06e845e82. A lot of changes since the last review: rebase after sync_state introduction #18152 and tryGetBalances revert #18587, reverting getLastBlockTime change, fixing spacing and initializations and renaming some variables

Tree-SHA512: 835e587a8296df9899cccd7b3e598a5970942b640e432e6a32de0b4eaea5b40f9271258f089ec033595311707b74a0f7187ecf8ed397c713e1153e2714072975
2020-05-29 11:01:09 +02:00
Jonas Schnelli
beb3844c45 Merge #17968: qt: Ensure that ModalOverlay is resized properly
4fc1df41d5 qt: Track QEvent::Resize during animation (Hennadii Stepanov)

Pull request description:

  In certain circumstances the `ModalOverlay` widget is not sized properly:
  - #17269
  - #17967
  - https://github.com/bitcoin/bitcoin/pull/17968#pullrequestreview-350753107

  On master (f018d0c9cd) this bug looks like this:
  ![DeepinScreenshot_bitcoin-qt_20200120193402](https://user-images.githubusercontent.com/32963518/72748165-298b2a80-3bbf-11ea-810d-2966f08e496a.png)

  With this PR the wallet frame looks ok:
  ![DeepinScreenshot_bitcoin-qt_20200120195241](https://user-images.githubusercontent.com/32963518/72748388-c64dc800-3bbf-11ea-8875-1ba1899b3513.png)

  Fix #17269
  Fix #17967

ACKs for top commit:
  promag:
    Code review ACK 4fc1df41d5.
  jonasschnelli:
    utACK 4fc1df41d5

Tree-SHA512: b5d303fbc139c9383cd22edecba05e51b0d6115631aeb7d4474e973e7250a84019c11c0e41b5200e4d9ab10e17908774b45234317535857dc5314c3c28614ad4
2020-05-29 10:30:36 +02:00
Jonas Schnelli
0d5c18204b Merge #17956: gui: Disable unavailable context menu items in transactions tab
2b18fd2242 Disable unavailable context menu items in transactions tab (Kristaps Kaupe)

Pull request description:

  Fixes #9192.

ACKs for top commit:
  jonatack:
    Re-ACK 2b18fd2242
  jonasschnelli:
    codereview utACK 2b18fd2242

Tree-SHA512: 4ea19c7b5976f1f0b1baecccb3077cf82f078af7257f92162686bcce2188efe49511a5f557853bc5fe0b10616708957d61c006944babbe60b8105e78751e865f
2020-05-29 10:29:14 +02:00
Jonas Schnelli
a9024a42fc Merge #17918: qt: Hide non PKHash-Addresses in signing address book
c4ea501e96 qt: Hide non PKHash-Addresses in signing address book (Emil Engler)

Pull request description:

  [Video Demo](https://www.youtube.com/watch?v=T-Rp2pFRmzY)

  This PR hides all non PKHash addresses in the signing GUI in the Address Book when it is opened through the signing dialog, as non PKHash addresses are useless there.

ACKs for top commit:
  jonasschnelli:
    Code Review ACK c4ea501e96

Tree-SHA512: e321d45e15534b2d68da5a1297b1c7551cdd784f03203f54c9385c2ce0bb2b7316c09f9e8c3eb41bfa1e7207ecc94c8ed08f012e2d6c117b803996ade26feb2f
2020-05-29 10:23:08 +02:00
Jonas Schnelli
c2c15ea54e Merge #17908: qt: Remove QFont warnings with QT_QPA_PLATFORM=minimal
1122817c19 qt: Remove QFont warnings with QPA=minimal (Hennadii Stepanov)

Pull request description:

  This PR removes massive warnings like:
  ```
  QWARN  : ... QFont::setPointSizeF: Point size <= 0 (...), must be greater than 0
  ```

  from `test_bitcoin-qt` output.

  On master (e258ce792a):
  ```
  $ ./src/qt/test/test_bitcoin-qt | grep QFont | wc -l
  ~BitcoinApplication : Stopping thread
  ~BitcoinApplication : Stopped thread
  57
  ```

  With this PR:
  ```
  $ ./src/qt/test/test_bitcoin-qt | grep QFont | wc -l
  ~BitcoinApplication : Stopping thread
  ~BitcoinApplication : Stopped thread
  0
  ```

ACKs for top commit:
  promag:
    Code review ACK 1122817c19.
  jonasschnelli:
    utACK 1122817c19

Tree-SHA512: 32fa72a5d3db1d4c73a2a324aa9cad807ee46f23fc5319f7d71202987dc73ea7c90082904489b323a432e1afaebd9976b7dd0374236a16153162aa75fe368723
2020-05-29 10:19:44 +02:00
Jonas Schnelli
3431699981 Merge #17597: qt: Fix height of QR-less ReceiveRequestDialog
73529f0859 qt: Rename slot to updateDisplayUnit() (Hennadii Stepanov)
68288ef0c1 qt: Overhaul ReceiveRequestDialog (Hennadii Stepanov)

Pull request description:

  If master (89a1f7a250) is compiled without QR support, the "Request payment to..." dialog looks ugly:

  ![Screenshot from 2019-11-25 19-58-59](https://user-images.githubusercontent.com/32963518/69566647-3d9c1c80-0fc0-11ea-8ff6-183cea9372c5.png)

  With this PR:

  ![Screenshot from 2019-12-26 00-42-46](https://user-images.githubusercontent.com/32963518/71451226-221c6100-277a-11ea-94ae-c19a5c6256f7.png)

  ![Screenshot from 2019-12-26 00-44-34](https://user-images.githubusercontent.com/32963518/71451228-29436f00-277a-11ea-8ac5-1bafe6d73b5c.png)

  ![Screenshot from 2019-12-26 00-48-51](https://user-images.githubusercontent.com/32963518/71451230-2e082300-277a-11ea-8c4c-726ca7b776e7.png)

  Other minor changes:
  - "URI" abbreviation is not translated now
  - wallet name is shown if available

ACKs for top commit:
  jonasschnelli:
    Tested ACK 73529f0859

Tree-SHA512: 45f9a41d3c72978d78eb2e8ca98e274b8be5abf5352fd3e1f4f49514ce744994545fb3012e80600ded936ae41c6be4d4825a0c5f7bcb3db671d69e0299f0f65b
2020-05-29 10:02:26 +02:00
Jonas Schnelli
5f75c5e7df Merge #16432: qt: Add privacy to the Overview page
8d75115844 qt: Add privacy feature to Overview page (Hennadii Stepanov)
73d8ef7274 qt: Add BitcoinUnits::formatWithPrivacy() function (Hennadii Stepanov)

Pull request description:

  This PR allows to hide/reveal values on the Overviewpage by checking/unchecking Menu->Settings-> Mask Values

  Closes #16407

  Privacy mode is OFF (the default behavior):
  ![Screenshot from 2020-01-02 15-08-28](https://user-images.githubusercontent.com/32963518/71669074-28ab6980-2d74-11ea-8e54-4973aa307192.png)

  Privacy mode is ON:
  ![Screenshot from 2020-01-02 15-10-23 cropped](https://user-images.githubusercontent.com/32963518/71669082-2d701d80-2d74-11ea-9df5-d4acc4982dbe.png)

ACKs for top commit:
  jonatack:
    Tested ACK 8d75115
  jonasschnelli:
    Tested ACK 8d75115844

Tree-SHA512: 42f396d5bf0d343b306fb7e925f86f66b3fc3a257af370a812f4be181b5269298f9b23bd8a3ce25ab61de92908c4018d8c2dc8591d11bc58d79c4eb7206fc6ec
2020-05-29 08:55:16 +02:00
fanquake
adc143fc2b Merge #19097: qt: Add missing QPainterPath include
79b0a69e09 Add missing QPainterPath include (Andrew Chow)

Pull request description:

  This is needed to compile with Qt 5.15.

ACKs for top commit:
  laanwj:
    Code review ACK 79b0a69e09
  MarcoFalke:
    Code review ACK 79b0a69
  promag:
    Code review ACK 79b0a69e09.

Tree-SHA512: 8dbc3fa4572ad9cacd72e9664926729947681b8ed4f4a0607e27e6389eb95c8b49e6883ae8dbdea7edbbfea267b4520c4844897a7b67f55f4b988b9feb689e60
2020-05-29 09:27:26 +08:00
fanquake
fc1fb4dd7c Merge #19094: build: Only allow ASCII identifiers
399d84da37 build: Only allow ASCII identifiers (Wladimir J. van der Laan)

Pull request description:

  While emoji and other symbols in C++ identifers (as accepted by newer compilers) are fun, they might create confusion during code review, for example because some symbols look very similar. Forbid such extended identifiers for now.

  This is done by providing `-fno-extended-identifiers`. Thanks to sipa for suggesting this compiler flag.

ACKs for top commit:
  practicalswift:
    ACK 399d84da37 -- patch looks correct
  promag:
    ACK 399d84da37.
  jonatack:
    ACK 399d84da37
  fanquake:
    ACK 399d84da37 - seems like a good sanity check to enable.

Tree-SHA512: 62bfbe8c7e0284ed505c2c8789c1ae74997202d90595f298c2ee1917e5d69fa9b7196a9404ba2cff61f3162b2bbb5616a1591bed3f0534c58617e22009291933
2020-05-29 08:42:57 +08:00
MarcoFalke
fa3d41b5ab doc: Switch scheduler to doxygen comments 2020-05-28 19:28:43 -04:00
MarcoFalke
fac43f9889 scheduler: Replace stop(true) with StopWhenDrained()
This helps understanding the code at the call site without having to
look up the name of the argument or the default value.
2020-05-28 19:28:21 -04:00
Andrew Chow
79b0a69e09 Add missing QPainterPath include
This is needed to compile with Qt 5.15.
2020-05-28 14:56:47 -04:00
Wladimir J. van der Laan
399d84da37 build: Only allow ASCII identifiers
While emoji and other symbols in C++ identifers (as accepted by newer
compilers) are fun, they might create confusion during code review, for
example because some symbols look very similar. Forbid such extended
identifiers for now.

This is done by providing `-fno-extended-identifiers`. Thanks to sipa
for suggesting this compiler flag.
2020-05-28 19:35:42 +02:00
John Newbery
cc29d1e2c4 [tools] Update clang-format config
Allow arguments and parameters to be split over multiple lines
if they don't fit on one line
2020-05-28 13:26:18 -04:00
MarcoFalke
facef3d413 doc: Explain that anyone can work on good first issues, move text to CONTRIBUTING.md 2020-05-28 11:59:20 -04:00
MarcoFalke
fae2fb2a19 doc: Expand section on Getting Started
Co-authored-by: Adam Jonas <jonas@chaincode.com>
Co-authored-by: Jon Atack <jon@atack.com>
2020-05-28 11:59:07 -04:00
MarcoFalke
100000d1b2 doc: Add headings to CONTRIBUTING.md
This makes it possible to link and refer to subsections

Also, minor clarifications
2020-05-28 11:58:54 -04:00
Wladimir J. van der Laan
ea3e9e0b84 Merge #18700: Fix locking on WSL using flock instead of fcntl
e8fa0a3d20 Fix WSL file locking by using flock instead of fcntl (Samuel Dobson)

Pull request description:

  Fixes #18622

  A bug in WSL means that fcntl does not exclusively lock files, allowing multiple instances of bitcoin to use the same datadir. If we instead use flock, it works correctly. Passes Travis, but testing on some OS variety would be sensible.

  From what I can tell, flock and fcntl don't work with each other on linux, so it would still be possible to run a node with this code change and a node before it with the same datadir (this isn't true for Mac/FreeBSD). flock also doesn't support NFS on MacOS and linux<2.6.12 while fcntl did. See here for example: https://gavv.github.io/articles/file-locks/

  If changing to flock for all systems is inadvisable, it would also be possible to just detect WSL and use flock when on that platform to avoid the bug.

ACKs for top commit:
  laanwj:
    Code review ACK e8fa0a3d20

Tree-SHA512: ca1009e171970101f1dc2332c5e998717aee00eebc80bb586b826927a74bd0d4c94712e46d1396821bc30533d76deac391b6e1c406c406865661f57fa062c702
2020-05-28 17:55:20 +02:00
MarcoFalke
082a417abc Merge #18635: Replace -Wthread-safety-analysis with broader -Wthread-safety
87766b355c build: Replace -Wthread-safety-analysis with broader -Wthread-safety (Hennadii Stepanov)
9cc6eb3c9e Get rid of -Wthread-safety-precise warnings (Hennadii Stepanov)
971a468ccf Use template function instead of void* parameter (Hennadii Stepanov)
dfb75ae49d refactor: Rename LockGuard to StdLockGuard for consistency with StdMutex (Hennadii Stepanov)
79be487420 Add thread safety annotated wrapper for std::mutex (Hennadii Stepanov)

Pull request description:

  This PR gets rid of `-Wthread-safety-attributes` and `-Wthread-safety-precise` warnings, and replaces `-Wthread-safety-analysis` compiler option with the broader `-Wthread-safety` one.

ACKs for top commit:
  practicalswift:
    ACK 87766b355c -- patch looks correct
  ajtowns:
    ACK 87766b355c
  MarcoFalke:
    ACK 87766b355c 👍
  vasild:
    ACK 87766b3

Tree-SHA512: b1fe29f2568c954c612f964f9022a1f02333fae4a62c8c16c45e83f5f50a27231fc60b6bd369ccd3bbdb42ef4a0f19defde350c31a62613082ffbc9d7e383a5f
2020-05-28 10:47:50 -04:00
MarcoFalke
fa9cca0550 doc: Remove unused documentation about unimplemented features 2020-05-28 10:22:57 -04:00
MarcoFalke
fab2950d70 doc: Switch boost::thread to std::thread in scheduler
After commit d0ebd93 the scheduler itself no longer cares if the
serviceQueue is run in a std::thread or boost::thread. Change the
documentation to std::thread because we switched to C++11.
2020-05-28 10:22:36 -04:00
fanquake
789e9dd3aa validation: use std::chrono in IsCurrentForFeeEstimation() 2020-05-28 21:43:36 +08:00
fanquake
47be28c8bc validation: use std::chrono in CChainState::FlushStateToDisk() 2020-05-28 21:39:41 +08:00
glowang
f871f15c9d scripted-diff: replace gArgs with argsman
-BEGIN VERIFY SCRIPT-
sed -i 's/\<gArgs\>/m_args/g' src/test/getarg_tests.cpp
-END VERIFY SCRIPT-
2020-05-28 06:22:19 -07:00
glowang
357f02bf29 Create a local class inherited from BasicTestingSetup with a localized args manager
and put it into the getarg_tests namespace
2020-05-28 06:21:43 -07:00
MarcoFalke
fa9819695a test: Remove unused scheduler.h include from the common setup
The common setup is included in virtually all tests, so it should be
as slim as possible.
2020-05-28 09:00:56 -04:00
MarcoFalke
fa609c4f76 scheduler: Remove unused REVERSE_LOCK
No longer needed after commit d61f2bb076
2020-05-28 09:00:20 -04:00
Hennadii Stepanov
87766b355c build: Replace -Wthread-safety-analysis with broader -Wthread-safety 2020-05-28 09:56:44 +03:00
Hennadii Stepanov
9cc6eb3c9e Get rid of -Wthread-safety-precise warnings 2020-05-28 09:55:39 +03:00
Hennadii Stepanov
971a468ccf Use template function instead of void* parameter
This change gets rid of -Wthread-safety-attributes warning spam.
2020-05-28 09:55:04 +03:00
Hennadii Stepanov
dfb75ae49d refactor: Rename LockGuard to StdLockGuard for consistency with StdMutex 2020-05-28 09:54:24 +03:00
Hennadii Stepanov
79be487420 Add thread safety annotated wrapper for std::mutex
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2020-05-28 09:54:09 +03:00
Russell Yanofsky
4a7253ab6c Remove g_rpc_chain global
Replace with RPC request reference to new WalletContext struct similar to the
existing NodeContext struct and reference.

This PR is a followup to 25ad2c623a
https://github.com/bitcoin/bitcoin/pull/18740 removing the g_rpc_node global.

Some later PRs will follow this up and move more wallet globals to the
WalletContext struct.

Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2020-05-28 02:13:19 -04:00
Russell Yanofsky
e783197bf0 refactor: replace RegisterWalletRPCCommands with GetWalletRPCCommands 2020-05-28 02:13:19 -04:00
MarcoFalke
55b4c65bd1 Merge #16127: More thread safety annotation coverage
5478d6c099 logging: thread safety annotations (Anthony Towns)
e685ca1992 util/system.cpp: add thread safety annotations for dir_locks (Anthony Towns)
a788789948 test/checkqueue_tests: thread safety annotations (Anthony Towns)
479c5846f7 rpc/blockchain.cpp: thread safety annotations for latestblock (Anthony Towns)
8b5af3d4c1 net: fMsgProcWake use LOCK instead of lock_guard (Anthony Towns)
de7c5f41ab wallet/wallet.h: Remove mutexScanning which was only protecting a single atomic bool (Anthony Towns)
c3cf2f5501 rpc/blockchain.cpp: Remove g_utxosetscan mutex that is only protecting a single atomic variable (Anthony Towns)

Pull request description:

  In a few cases we need to use `std::mutex` rather than the sync.h primitives. But `std::lock_guard<std::mutex>` doesn't include the clang thread safety annotations unless you also use clang's C library, which means you can't indicate when variables should be guarded by `std::mutex` mutexes.

  This adds an annotated version of `std::lock_guard<std::mutex>` to threadsafety.h to fix that, and modifies places where `std::mutex` is used to take advantage of the annotations.

  It's based on top of #16112, and turns the thread safety comments included there into annotations.

  It also changes the RAII classes in wallet/wallet.h and rpc/blockchain.cpp to just use the atomic<bool> flag for synchronisation rather than having a mutex that doesn't actually guard anything as well.

ACKs for top commit:
  MarcoFalke:
    ACK 5478d6c099 🗾
  hebasto:
    re-ACK 5478d6c099, only renamed s/`MutexGuard`/`LockGuard`/, and dropped the commit "test/util_threadnames_tests: add thread safety annotations" since the [previous](https://github.com/bitcoin/bitcoin/pull/16127#pullrequestreview-414184113) review.
  ryanofsky:
    Code review ACK 5478d6c099. Thanks for taking suggestions! Only changes since last review are dropping thread rename test commit d53072ec730d8eec5a5b72f7e65a54b141e62b19 and renaming mutex guard to lock guard

Tree-SHA512: 7b00d31f6f2b5a222ec69431eb810a74abf0542db3a65d1bbad54e354c40df2857ec89c00b4a5e466c81ba223267ca95f3f98d5fbc1a1d052a2c3a7d2209790a
2020-05-27 19:31:33 -04:00
Carl Dong
f0d7ed10b4 depends: Propagate only specific CLI variables to sub-makes
We want to supply well-known vars to ./configure scripts to do with as
they please. However, we do _not_ want to override these well-known vars
at make-time as certain build systems expect a self-mangled version of
these well-known vars.

For example, freetype and bdb will prepend `libtool --mode=compile' to
CC and CXX, which, if we override CC on the command line at make-time,
will break the build.
2020-05-27 17:39:52 -04:00
Carl Dong
0a33803f1c depends: boost: Use clang toolset if clang in CXX 2020-05-27 17:39:51 -04:00
Carl Dong
1ce74bcde3 depends: boost: Split target-os from toolset
Previously, we specified the target-os in the toolset (and sometimes
used the wrong command line flags), now we have a clear separation,
which is favored by ./bootstrap.sh and ./b2.

This means that all supported OSes will specify the correct target-os=
and toolset= on the command line.
2020-05-27 17:39:49 -04:00
Carl Dong
2d4e480813 depends: boost: Specify toolset to bootstrap.sh
b2 will pickup our user-config.jam just fine, however, bootstrap.sh has
its own toolset autodetect mechanism, which doesn't GAF about our
user-config.jam
2020-05-27 17:39:48 -04:00
Carl Dong
3d6603e340 depends: Propagate well-known vars into depends
For example, doing:

  make CC=clang CXX=clang++

Should now propagate these settings down to depends packages
2020-05-27 17:39:47 -04:00
MarcoFalke
fa7b885f51 walletdb: Remove unsed boost/thread 2020-05-27 13:41:56 -04:00
MarcoFalke
fab893e0ca doc: Fix unrelated typos reported by codespell 2020-05-27 12:37:08 -04:00
MarcoFalke
9ccaee1d5e Merge #19004: refactor: Replace const char* to std::string
c57f03ce17 refactor: Replace const char* to std::string (Calvin Kim)

Pull request description:

  Rationale: Addresses #19000
  Some functions should be returning std::string instead of const char*.
  This commit changes that.

  Main benefits/reasoning:

  1.  The functions never return nullptr, so returning a string makes code at call sites easier to review (reviewers don't have to read the source code to verify that a nullptr is never returned)
  2. All call sites convert to string anyway

ACKs for top commit:
  MarcoFalke:
    re-ACK c57f03ce17 (no changes since previous review) 🚃
  Empact:
    Fair enough, Code Review ACK c57f03ce17
  practicalswift:
    ACK c57f03ce17 -- patch looks correct
  hebasto:
    re-ACK c57f03ce17

Tree-SHA512: 9ce99bb38fe399b54844315048204cafce0f27fd8f24cae357fa7ac6f5d8094d57bbf5f5c1f5878a65f2d35e4a3f95d527eb17f49250b690c591c0df86ca84fd
2020-05-27 07:16:10 -04:00
fanquake
cffbf1eb9a Merge #19073: Remove outdated comment about DER encoding
4c825792dd Remove outdated comment about DER encoding (Elichai Turkel)

Pull request description:

  This comment got me confused about the status of BIP66 (Thanks jnewbery for explaining)
  The comment was added in: https://github.com/bitcoin/bitcoin/pull/3843
  But in https://github.com/bitcoin/bitcoin/pull/5713 strict DER encoding was enforced in consensus,
  and is now it's buried and enforced by the height of the block here: 4af01b37d4/src/validation.cpp (L1889)

  P.S. This is also quite confusing: 4af01b37d4/src/validation.cpp (L1560-L1563) But seems to be intentional: 4af01b37d4/src/validation.cpp (L1510-L1511)

ACKs for top commit:
  laanwj:
    ACK 4c825792dd

Tree-SHA512: 7afbbae84ed4ecfaa0a273ae024b14f2b7ffe65307f078086fe0b5b645c57722bc2952fb15d167d9e4fa5b052d1d0ac6e5e33f57e8fc881c0ea611d352bccc1e
2020-05-27 18:35:54 +08:00
fanquake
48e114e0a8 Merge #19061: doc: Add link to Visual Studio build readme
1c91ffefcf doc : add link to readme.md in the first section (pad)

Pull request description:

  I have searched how to do it in this doc for some time :-(
  I think it might help other newbies interested in building with visual studio.

ACKs for top commit:
  hebasto:
    ACK 1c91ffefcf, a new link works as expected :)

Tree-SHA512: 42ef3ba374bced9b4ab0010fe8c30de06f59ff8a84f8e02f8a91f33e7e403cf91d624fc7df3f45096df53171a90b9ff60277969cc30f1357d92094ad72ca9d53
2020-05-27 17:51:05 +08:00
Hennadii Stepanov
8d75115844 qt: Add privacy feature to Overview page 2020-05-27 11:55:44 +03:00
pad
1c91ffefcf doc : add link to readme.md in the first section 2020-05-27 08:02:00 +02:00
Samuel Dobson
520e435b5e Merge #18918: wallet: Move salvagewallet into wallettool
84ae0578b6 Add release notes about salvage changes (Andrew Chow)
ea337f2d03 Move RecoverKeysOnlyFilter into RecoverDataBaseFile (Andrew Chow)
9ea2d258b4 Move RecoverDatabaseFile and RecoverKeysOnlyFilter into salvage.{cpp/h} (Andrew Chow)
b426c7764d Make BerkeleyBatch::Recover and WalletBatch::RecoverKeysOnlyFilter standalone (Andrew Chow)
2741774214 Expose a version of ReadKeyValue and use it in RecoverKeysOnlyFilter (Andrew Chow)
ced95d0e43 Move BerkeleyEnvironment::Salvage into BerkeleyBatch::Recover (Andrew Chow)
07250b8dce walletdb: remove fAggressive from Salvage (Andrew Chow)
8ebcbc85c6 walletdb: don't automatically salvage when corruption is detected (Andrew Chow)
d321046f4b wallet: remove -salvagewallet (Andrew Chow)
cdd955e580 Add basic test for bitcoin-wallet salvage (Andrew Chow)
c87770915b wallettool: Add a salvage command (Andrew Chow)

Pull request description:

  Removes the `-salvagewallet` startup option and adds a `salvage` command to the `bitcoin-wallet` tool. As such, `-salvagewallet` is removed. Additionally, the automatic salvage that is done if the wallet file fails to load is removed.

  Lastly the salvage code entirely is moved out entirely into `bitcoin-wallet` from `walletdb.{cpp/h}` and `db.{cpp/h}`.

ACKs for top commit:
  jonatack:
    ACK 84ae0578b6 feedback taken, and compared to my previous review, the bitcoin-wallet salvage command now seems to run and it exits without raising. The new test passes at both 9454105 and 84ae057 so as a sanity check I'd agree there is room for improvement, if possible.
  MarcoFalke:
    re-ACK 84ae0578b6 🏉
  Empact:
    Code Review ACK 84ae0578b6
  ryanofsky:
    Code review ACK 84ae0578b6. Lot of small changes since previous review: added verify step before salvage, added basic test in new commit, removed unused scanstate variable and warnings parameter, tweaked various comments and strings, moved fsuccess variable declaration
  meshcollider:
    Concept / light code review ACK 84ae0578b6

Tree-SHA512: 05be116b56ecade1c58faca1728c8fe4b78f0a082dbc2544a3f7507dd155f1f4f39070bd1fe90053444384337bc48b97149df5c1010230d78f8ecc08e69d93af
2020-05-27 14:51:49 +12:00
MarcoFalke
5555d978b0 wallet: Make PeriodicFlush uninterruptible 2020-05-26 19:56:43 -04:00
10xcryptodev
fc6a637a01 qt: increase console command max length 2020-05-26 18:54:49 -03:00
Jim Posen
9e36067d8c [test] Add test for cfilters. 2020-05-26 17:38:26 -04:00
Jim Posen
11106a4722 [net processing] Message handling for getcfilters.
Handle getcfilters request if -peercfilter is configured.
2020-05-26 17:38:20 -04:00
Jim Posen
e535670726 [indexes] Fix default [de]serialization of BlockFilter.
This only changes network serialization. Disk serialization does not
include the filter_type and is defined in
ReadFilterFromDisk()/WriteFilterToDisk().
2020-05-26 17:27:15 -04:00
John Newbery
bb911ae7f5 [refactor] Pass CNode and CConnman by reference
Pass CNode and CConnman by reference instead of by pointer to
ProcessGetCFCheckPt() and ProcessGetCFHeaders().
2020-05-26 17:24:17 -04:00
Wladimir J. van der Laan
4af01b37d4 Merge #19060: test: Remove global wait_until from p2p_getdata
fa80b4788b test: Remove global wait_until from p2p_getdata (MarcoFalke)
999922baed test: Default mininode.wait_until timeout to 60s (MarcoFalke)
fab47375fe test: pep-8 p2p_getdata.py (MarcoFalke)

Pull request description:

  Using the global wait_until makes it impossible to adjust the timeout based on the hardware the test is running on.

  Fix that by using the mininode member function.

  So for example, `./test/functional/p2p_getdata.py  --timeout-factor=0.04` gives a timeout of 2.4 seconds.

ACKs for top commit:
  laanwj:
    ACK fa80b4788b

Tree-SHA512: ebb1b7860a64451de2b8ee9a0966faddb13b84af711f6744e8260d7c9bc0b382e8fb259897df5212190821e850ed30d4d5c2d7af45a97f207fd4511b06b6674a
2020-05-26 19:06:12 +02:00
Elichai Turkel
4c825792dd Remove outdated comment about DER encoding 2020-05-26 18:39:01 +03:00
Anthony Towns
5478d6c099 logging: thread safety annotations
Adds LockGuard helper in threadsafety.h to replace lock_guard<mutex>
when LOCK(Mutex) isn't available for use.
2020-05-27 01:31:51 +10:00
Sebastian Falbesoner
b00266fe0c refactor: replace pointers by references within tx_verify.{h,cpp}
affects "prevHeights" parameter of the functions
- CalculateSequenceLocks()
- SequenceLocks()
2020-05-26 16:05:51 +02:00
Wladimir J. van der Laan
dcacea096e Merge #19032: Serialization improvements: final step
71f016c6eb Remove old serialization primitives (Pieter Wuille)
92beff15d3 Convert LimitedString to formatter (Pieter Wuille)
ef17c03e07 Convert wallet to new serialization (Pieter Wuille)
65c589e45e Convert Qt to new serialization (Pieter Wuille)

Pull request description:

  This is the final step 🥳 of the serialization improvements extracted from #10785.

  It converts the LimitedString wrapper to a new-style formatter, and updates the wallet and Qt code to use the new serialization framework. Finally all remaining old primitives are removed.

ACKs for top commit:
  jonatack:
    ACK 71f016c6eb reviewed diff, builds/tests/re-fuzzed.
  laanwj:
    Code review ACK 71f016c6eb

Tree-SHA512: d952194bc73259f6510bd4ab1348a1febbbf9862af30f905991812fb0e1f23f15948cdb3fc662be54d648e8f6d95b11060055d2e7a8c2cb5bf008224870b1ea1
2020-05-26 15:45:50 +02:00
Anthony Towns
e685ca1992 util/system.cpp: add thread safety annotations for dir_locks 2020-05-26 23:23:15 +10:00
MarcoFalke
fe1357a03a Merge #18881: Prevent UB in DeleteLock() function
90eb027204 doc: Add and fix comments about never destroyed objects (Hennadii Stepanov)
26c093a995 Replace thread_local g_lockstack with a mutex-protected map (Hennadii Stepanov)
58e6881bc5 refactor: Refactor duplicated code into LockHeld() (Hennadii Stepanov)
f511f61dda refactor: Add LockPair type alias (Hennadii Stepanov)
8d8921abd3 refactor: Add LockStackItem type alias (Hennadii Stepanov)
458992b06d Prevent UB in DeleteLock() function (Hennadii Stepanov)

Pull request description:

  Tracking our instrumented mutexes (`Mutex` and `RecursiveMutex` types) requires that all involved objects should not be destroyed until after their last use. On master (ec79b5f86b) we have two problems related to the object destroying order:
  - the function-local `static` `lockdata` object that is destroyed at [program exit](https://en.cppreference.com/w/cpp/utility/program/exit)
  - the `thread_local` `g_lockstack` that is destroyed at [thread exit](https://en.cppreference.com/w/cpp/language/destructor)

  Both cases could cause UB at program exit in so far as mutexes are used in other static object destructors.

  Fix #18824

ACKs for top commit:
  MarcoFalke:
    re-ACK 90eb027204, only change is new doc commit 👠
  ryanofsky:
    Code review ACK 90eb027204 because all the changes look correct and safe. But I don't know the purpose of commit  26c093a995 "Replace thread_local g_lockstack with a mutex-protected map (5/6)." It seems like it could have a bad impact on debug performance, and the commit message and PR description don't give a reason for the change.

Tree-SHA512: 99f29157fd1278994e3f6eebccedfd9dae540450f5f8b980518345a89d56b635f943a85b20864cef087027fd0fcdb4880b659ef59bfe5626d110452ae22031c6
2020-05-26 08:14:57 -04:00
MarcoFalke
13397dc78f Merge #19056: rpc: Make gettxoutsetinfo/GetUTXOStats interruptible
fa756928c3 rpc: Make gettxoutsetinfo/GetUTXOStats interruptible (MarcoFalke)
fa7fc5a8e0 rpc: factor out RpcInterruptionPoint from dumptxoutset (MarcoFalke)

Pull request description:

  Make it interruptible, so that shutdown doesn't block for up to one hour.

  Fixes (partially) #13217

ACKs for top commit:
  Empact:
    Code Review ACK fa756928c3
  laanwj:
    Code review ACK fa756928c3

Tree-SHA512: 298261e0ff7d79fab542b8f6828cc0ac451cbafe396d5f0816c9d36437faba1330f5c4cb2a25c5540e202bfb9783da6ec858bd453056ce488d21e36335d3d42c
2020-05-26 07:33:43 -04:00
MarcoFalke
d3b0ef80f6 Merge #18867: tests: Add fuzzing harness for CCoinsViewCache
f9b22e3bdb tests: Add fuzzing harness for CCoinsViewCache (practicalswift)

Pull request description:

  Add fuzzing harness for `CCoinsViewCache`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    ACK f9b22e3bdb 📫

Tree-SHA512: 4fa79aab683875eef128b672cf199909c86e4d2ed7c406f006fa27a546dafc9cb0061c4de5e660e622458072f1dab69dbf6b6b03d5b863f81c5710bf4cee6c0c
2020-05-26 07:29:44 -04:00
MarcoFalke
7d32cce3e7 Merge #19010: net processing: Add support for getcfheaders
5308c97cca [test] Add test for cfheaders (Jim Posen)
f6b58c1506 [net processing] Message handling for getcfheaders. (Jim Posen)
3bdc7c2d39 [doc] Add comment for m_headers_cache (John Newbery)

Pull request description:

  Support `getcfheaders` requests when `-peerblockfilters` is set.

  Does not advertise compact filter support in version messages.

ACKs for top commit:
  jkczyz:
    ACK 5308c97cca
  MarcoFalke:
    re-ACK 5308c97cca , only change is doc related 🗂
  theStack:
    ACK 5308c97cca 🚀

Tree-SHA512: 240fc654f6f634c191d9f7628b6c4801f87ed514a1dd55c7de5d454d4012d1c09509a2d5a246bc7da445cd920252b4cd56a493c060cdb207b04af4ffe53b95f7
2020-05-26 07:27:00 -04:00
fanquake
14f8447777 Merge #19059: gui: update Qt base translations for macOS release
69bfcac27a gui: update Qt base translations for macOS release (fanquake)

Pull request description:

  These haven't been updated since their addition, so this updates the list that
  controls which qt base translations are bundled with the macOS binary, to all the
  languages that are available with qt 5.9.8.

  This could probably be improved in some way, however qt updates are infrequent,
  and I didn't want to spend any more time looking at this. Also given that no-one
  seems to have noticed and/or reported this it wouldn't seem high-priority.

  Could be backported to 0.20.1.

  Master:
  ![master](https://user-images.githubusercontent.com/863730/82729428-11bce200-9d2a-11ea-8569-ee65d46c7403.png)

  This PR:
  ![fixed](https://user-images.githubusercontent.com/863730/82729427-0f5a8800-9d2a-11ea-86dd-1e6a3e211efa.png)

ACKs for top commit:
  hebasto:
    ACK 69bfcac27a, tested on macOS 10.15.

Tree-SHA512: df142fb16097deb514e72e005b73aafc4eb4ff0c17e423ba5040a3ec6874020a733e1c5259a88923580e71ef73c16222aed28f482b8c270a544a85b745a7b327
2020-05-26 08:55:22 +08:00
Amiti Uttarwar
9e1cb1adf1 [trivial/doc] Fix comment type 2020-05-25 11:27:07 -07:00
Amiti Uttarwar
8f30260a67 [doc] Update unbroadcast description in RPC results 2020-05-25 11:27:07 -07:00
Amiti Uttarwar
750456d6f2 [trivial] Remove misleading 'const'
Returning by const value is only meaningful in a specific circumstance around
user defined types. In this case, the const is not enforcing any restrictions
on the call site, so is misleading.
2020-05-25 11:27:07 -07:00
Amiti Uttarwar
fa32e676e5 [test] Manage node connections better in mempool persist test
there were two calls to disconnect_nodes that were no-ops. fixed one & removed
the other & added assertions to confirm node has no connections when creating
the unbroadcast transaction.
2020-05-25 11:27:07 -07:00
Amiti Uttarwar
1f94bb0c74 [doc] Provide rationale for randomization in scheduling. 2020-05-25 11:27:07 -07:00
Amiti Uttarwar
9c8a55d9cb [mempool] Don't throw expected error message when upgrading 2020-05-25 11:27:07 -07:00
Amiti Uttarwar
ba54983182 [test] Test that wallet transactions aren't rebroadcast before 12 hours 2020-05-25 11:27:06 -07:00
Amiti Uttarwar
00d44a534b [test] P2P connection behavior should meet expectations
- P2PTxInvStore should supplement `on_inv` behavior of parent, not overwrite.
2020-05-25 11:27:06 -07:00
Amiti Uttarwar
bd093ca15d [test] updates to unbroadcast test
- add () to function to actually disconnect from p2pconn
- extract max interval into a constant
- disconnect at the end of a subtest rather than start of next
2020-05-25 11:27:06 -07:00
Amiti Uttarwar
dab298d9ab [docs] add release notes 2020-05-25 11:27:06 -07:00
Andrew Chow
84ae0578b6 Add release notes about salvage changes 2020-05-25 12:59:29 -04:00
Andrew Chow
ea337f2d03 Move RecoverKeysOnlyFilter into RecoverDataBaseFile 2020-05-25 12:59:29 -04:00
Andrew Chow
9ea2d258b4 Move RecoverDatabaseFile and RecoverKeysOnlyFilter into salvage.{cpp/h} 2020-05-25 12:59:29 -04:00
Andrew Chow
b426c7764d Make BerkeleyBatch::Recover and WalletBatch::RecoverKeysOnlyFilter standalone
Instead of having these be class static functions, just make them be
standalone. Also removes WalletBatch::Recover which just passed through
to BerkeleyBatch::Recover.
2020-05-25 12:59:29 -04:00
Andrew Chow
2741774214 Expose a version of ReadKeyValue and use it in RecoverKeysOnlyFilter
We need this exposed for BerkeleyBatch::Recover to be moved out.
2020-05-25 12:59:29 -04:00
Andrew Chow
ced95d0e43 Move BerkeleyEnvironment::Salvage into BerkeleyBatch::Recover 2020-05-25 12:59:29 -04:00
Andrew Chow
07250b8dce walletdb: remove fAggressive from Salvage
The only call to Salvage set fAggressive = true so remove that parameter
and always use DB_AGGRESSIVE
2020-05-25 12:59:29 -04:00
Andrew Chow
8ebcbc85c6 walletdb: don't automatically salvage when corruption is detected 2020-05-25 12:59:22 -04:00
Andrew Chow
d321046f4b wallet: remove -salvagewallet 2020-05-25 12:39:40 -04:00
Andrew Chow
cdd955e580 Add basic test for bitcoin-wallet salvage 2020-05-25 12:37:53 -04:00
Andrew Chow
c87770915b wallettool: Add a salvage command 2020-05-25 12:36:48 -04:00
practicalswift
f9b22e3bdb tests: Add fuzzing harness for CCoinsViewCache 2020-05-25 10:05:06 +00:00
Pieter Wuille
71f016c6eb Remove old serialization primitives 2020-05-24 10:35:00 -07:00
Pieter Wuille
92beff15d3 Convert LimitedString to formatter 2020-05-24 10:35:00 -07:00
Pieter Wuille
ef17c03e07 Convert wallet to new serialization 2020-05-24 10:34:52 -07:00
furszy
a06e845e82 BlockTip struct created and connected to notifyHeaderTip and notifyBlockTip signals. 2020-05-23 20:02:10 -03:00
furszy
2f867203b0 Added best block hash to the NotifyHeaderTip and NotifyBlockTip signals.
[ClientModel] best header/block hash cached.
2020-05-23 20:02:04 -03:00
MarcoFalke
fa80b4788b test: Remove global wait_until from p2p_getdata 2020-05-23 10:01:10 -04:00
MarcoFalke
999922baed test: Default mininode.wait_until timeout to 60s 2020-05-23 09:51:03 -04:00
Hennadii Stepanov
73d8ef7274 qt: Add BitcoinUnits::formatWithPrivacy() function 2020-05-23 16:47:10 +03:00
MarcoFalke
fab47375fe test: pep-8 p2p_getdata.py 2020-05-23 09:40:49 -04:00
Samuel Dobson
24f7029064 Merge #18594: cli: display multiwallet balances in -getinfo
5edad5ce5d test: add -getinfo multiwallet functional tests (Jon Atack)
903b6c117f rpc: drop unused JSONRPCProcessBatchReply size arg, refactor (Jon Atack)
afce85eb99 cli: use GetWalletBalances() functionality for -getinfo (Jon Atack)
9f01849a49 cli: create GetWalletBalances() to fetch multiwallet balances (Jon Atack)
743077544b cli: lift -rpcwallet logic up to CommandLineRPC() (Jon Atack)
29f2cbdeb7 cli: extract connection exception handler, -rpcwait logic (Jon Atack)

Pull request description:

  This PR is a client-side version of #18453, per review feedback there and [review club discussions](https://bitcoincore.reviews/18453#meeting-log). It updates `bitcoin-cli -getinfo` on the client side to display wallet name and balance for the loaded wallets when more than one is loaded (e.g. you are in "multiwallet mode") and `-rpcwallet=` is not passed; otherwise, behavior is unchanged.

  before
  ```json
  $ bitcoin-cli -getinfo -regtest
  {
    "version": 199900,
    "blocks": 15599,
    "headers": 15599,
    "verificationprogress": 1,
    "timeoffset": 0,
    "connections": 0,
    "proxy": "",
    "difficulty": 4.656542373906925e-10,
    "chain": "regtest",
    "balance": 0.00001000,
    "relayfee": 0.00001000
  }

  ```
  after
  ```json
  $ bitcoin-cli -getinfo -regtest
  {
    "version": 199900,
    "blocks": 15599,
    "headers": 15599,
    "verificationprogress": 1,
    "timeoffset": 0,
    "connections": 0,
    "proxy": "",
    "difficulty": 4.656542373906925e-10,
    "chain": "regtest",
    "balances": {
      "": 0.00001000,
      "Encrypted": 0.00003500,
      "day-to-day": 0.00000120,
      "side project": 0.00000094
    }
  }
  ```
  -----

  `Review club` discussion about this PR is here: https://bitcoincore.reviews/18453

  This PR can be manually tested by building, creating/loading/unloading several wallets with `bitcoin-cli createwallet/loadwallet/unloadwallet` and running `bitcoin-cli -getinfo` and `bitcoin-cli -rpcwallet=<wallet-name> -getinfo`.

  `wallet_multiwallet.py --usecli` provides regression test coverage on this change, along with `interface_bitcoin_cli.py` where this PR adds test coverage.

  Credit to Wladimir J. van der Laan for the idea in https://github.com/bitcoin/bitcoin/issues/17314 and https://github.com/bitcoin/bitcoin/pull/18453#issuecomment-605431806.

ACKs for top commit:
  promag:
    Tested ACK 5edad5ce5d.
  jnewbery:
    utACK 5edad5ce5d
  meshcollider:
    Code review ACK 5edad5ce5d

Tree-SHA512: 4ca36c5f6c49936b40afb605c44459c1d5b80b5bd84df634007ca276b3f6c102a0cb382f9d528370363ee32c94b0d7ffa15184578eaf8de74179e566c5c5cee5
2020-05-24 00:17:38 +12:00
MarcoFalke
793e0ff22c Merge #18698: Make g_chainman internal to validation
fab6b9d18f validation: Mark g_chainman DEPRECATED (MarcoFalke)
fa1d97b256 validation: Make ProcessNewBlock*() members of ChainstateManager (MarcoFalke)
fa24d49098 validation: Make PruneOneBlockFile() a member of ChainstateManager (MarcoFalke)
fa84b1cd84 validation: Make LoadBlockIndex() a member of ChainstateManager (MarcoFalke)
fa05fdf0f1 net: Pass chainman into PeerLogicValidation (MarcoFalke)
fa7b626d7a node: Add chainman alias for g_chainman (MarcoFalke)

Pull request description:

  The global `g_chainman` has recently been introduced in #17737. The chainstate manager is primarily needed for the assumeutxo feature, but it can also simplify testing in the future.

  The goal of this pull is to make the global chainstate manager internal to validation, so that all external code does not depend on globals and that unit or fuzz tests can pass in their (potentially mocked) chainstate manager.

  I suggest reviewing the pull request commit-by-commit. It should be relatively straightforward refactoring that does not change behavior at all.

ACKs for top commit:
  ryanofsky:
    Code review ACK fab6b9d18f. Had to be rebased but still looks good

Tree-SHA512: dcbf114aeef4f8320d466369769f22ce4dd8f46a846870354df176c3de9ff17c64630fbd777e7121d7470d7a8564ed8d37b77168746e8df7489c6877e55d7b4f
2020-05-23 07:58:13 -04:00
fanquake
492cdc56e0 Merge #19058: doc: Drop protobuf stuff
ea9fcfd130 doc: Drop protobuf stuff (Hennadii Stepanov)

Pull request description:

  This is a follow-up to #17165.

ACKs for top commit:
  fanquake:
    ACK ea9fcfd130 -  clicked the links and they seem to work.

Tree-SHA512: 0861bbac3a3ff781a413e15f5ed02c624bc15d572a001a53cd2fb9f7683456175f69e9d666b72f260abbb5114b67cefca9fada4d179c62384c90479534ae63d5
2020-05-23 19:03:40 +08:00
fanquake
69bfcac27a gui: update Qt base translations for macOS release
These haven't been updated since their addition, so this updates the list that
controls which qt base translations are bundled with the macOS binary, to all the
languages that are available with qt 5.9.8.

This could probably be improved in some way, however qt updates are infrequent,
and I didn't want to spend any more time looking at this. Also given that no-one
seems to have noticed and/or reported this it wouldn't seem high-priority.

Could be backported to 0.20.1.
2020-05-23 18:22:29 +08:00
Hennadii Stepanov
ea9fcfd130 doc: Drop protobuf stuff 2020-05-23 10:14:18 +03:00
MarcoFalke
fa3288cda1 Merge #19052: tests: Don't limit fuzzing inputs to 1 MB for afl-fuzz (now: ∞ ∀ fuzzers)
6a239e72eb tests: Don't limit fuzzing inputs to 1 MB for afl-fuzz (now: ∞ ∀ fuzzers) (practicalswift)

Pull request description:

  Don't limit fuzzing inputs to 1 MB for `afl-fuzz`.

  This change provides a level playing field for all fuzzers which allows for fair benchmarking using projects such as the excellent [FuzzBench](https://github.com/google/fuzzbench) project.

  Prior to this commit we limited `afl-fuzz` to ≤1 MB inputs but allowed unlimited length inputs for all other fuzzers.

ACKs for top commit:
  MarcoFalke:
    ACK 6a239e72eb The maximum data size should be a runtime option, not a compile time hardcoded value.

Tree-SHA512: dad176ae39aa09fe919e057008ab0670b9da72909bfeb8f0e8b9ae93b65514f2e25a1d51be89a32be9122fc412edf49234dfd9a44beb974b25fda387fd7bf174
2020-05-22 17:43:20 -04:00
Russell Yanofsky
7eaf86d3bf trivial: Suggested cleanups to surrounding code
https://github.com/bitcoin/bitcoin/pull/18982#pullrequestreview-416974841
2020-05-22 16:30:07 -04:00
MarcoFalke
fa756928c3 rpc: Make gettxoutsetinfo/GetUTXOStats interruptible
Also, add interruption points to scantxoutset
2020-05-22 15:53:50 -04:00
MarcoFalke
fa7fc5a8e0 rpc: factor out RpcInterruptionPoint from dumptxoutset 2020-05-22 15:52:37 -04:00
Jim Posen
5308c97cca [test] Add test for cfheaders 2020-05-22 11:59:58 -04:00
Jim Posen
f6b58c1506 [net processing] Message handling for getcfheaders.
if -peerblockfilters is configured, handle requests for cfheaders.
2020-05-22 11:59:58 -04:00
John Newbery
3bdc7c2d39 [doc] Add comment for m_headers_cache 2020-05-22 11:59:58 -04:00
practicalswift
6a239e72eb tests: Don't limit fuzzing inputs to 1 MB for afl-fuzz (now: ∞ ∀ fuzzers) 2020-05-22 15:15:46 +00:00
Hennadii Stepanov
90eb027204 doc: Add and fix comments about never destroyed objects 2020-05-22 15:45:21 +03:00
MarcoFalke
b5c423c48e Merge #19014: test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option
fad798be76 test: Default --previous-releases to false if dir is empty (MarcoFalke)
faf1c3cc58 test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option (MarcoFalke)

Pull request description:

  The "auto-detection" feature is kept in place, but making it an option allows to properly document it. For example, on my machine I get:

  ```
  $ ./test/functional/wallet_disable.py --help | grep previous-releases
    --previous-releases   Force test of previous releases (default: False)

ACKs for top commit:
  Sjors:
    re-tACK fad798b

Tree-SHA512: a7377d0d5378be0a50be278d76396cc403583617b5fc43467773eee706df698acf3f4e67651491183b9b43a8e1816b052e4c17b90272b7ec4b6ac134ad811400
2020-05-22 06:29:37 -04:00
Andrew Chow
d9cd095b59 Split SetActiveScriptPubKeyMan into Add/LoadActiveScriptPubKeyMan
Remove the memonly bool and follow the Add and Load pattern we use
everywhere else.
2020-05-21 23:01:24 -04:00
Andrew Chow
0122fbab4c Split SetHDChain into AddHDChain and LoadHDChain
Remove the memonly bool and follow our typical Add and Load pattern.
2020-05-21 22:43:58 -04:00
Samuel Dobson
df303ceb65 Merge #18787: wallet: descriptor wallet release notes and cleanups
ca2a09640f Change SetType to SetInternal and remove m_address_type (Andrew Chow)
89b1ce1140 Remove unimplemented SetCrypted from DescriptorScriptPubKeyMan (Andrew Chow)
b9073c8f13 rpc: createwallet warning that descriptor wallets are experimental (Andrew Chow)
610030d95c docs: Add release notes for descriptor wallets (Andrew Chow)

Pull request description:

  Some docs and cleanup following #16528.

  * Added release notes to explain a bit of motivation for descriptor wallets, what was changed, and how users will be effected by it. Also mentions the caveats regarding multsigs and watchonly that we have discussed on IRC.
  * Adds a warning to `createwallet` that descriptor wallets are experimental.
  * Removed unused `SetCrypted` as suggestioned: https://github.com/bitcoin/bitcoin/pull/16528#discussion_r415300916
  * Removed `m_address_type` as mentioned in https://github.com/bitcoin/bitcoin/pull/18782#issuecomment-620167077

ACKs for top commit:
  Sjors:
    tACK ca2a09640f
  instagibbs:
    utACK ca2a09640f
  meshcollider:
    utACK ca2a09640f

Tree-SHA512: 987188a912c191430e5d3f89bcef54ba6773692fc2d95b16a3ec11d9007ded210466ed980a3857e8b7196beef6422f07f9c85cc157f996c02d16f4dbde2e7b2a
2020-05-22 14:21:56 +12:00
Samuel Dobson
ccd85b57af Merge #17681: wallet: Keep inactive seeds after sethdseed and derive keys from them as needed
1ed52fbb4d Remove IBD check in sethdseed (Andrew Chow)
b1810a145a Test that keys from inactive seeds are generated (Andrew Chow)
c93082ece4 Generate new keys for inactive seeds after marking used (Andrew Chow)
45f2f6a0e8 Determine inactive HD seeds from key metadata and track them in LegacyScriptPubKeyMan (Andrew Chow)
b59b4504ab have GenerateNewKey and DeriveNewChildKey take a CHDChain as an argument (Andrew Chow)

Pull request description:

  Largely implements the suggestion from https://github.com/bitcoin/bitcoin/pull/17484#issuecomment-560845316.

  After `sethdseed` is called, the CHDChain for the old seed is kept in the wallet. It is kept on the file as a new `inactivehdseed` record and in memory in a map `m_inactive_hd_seeds`. In `LegacyScriptPubKeyMan::MarkUnusedAddresses` we check each used key's metadata for whether it was derived from an inactive seed. If it is, we then check to see how many keys after that key were derived from the inactive seed. If that number does not match the keypool parameter, we derive more keys from the inactive seed until it does match. This way we won't miss transactions belonging to keys outside of the range of the keypool initially.

  The indexes and internal-ness of a key is gotten by checking it's key origin data.

  Because of this change, we no longer need to wait for IBD to finish before `sethdseed` can work so that check is also removed.

  A test case for this is added as well which fails on master.

ACKs for top commit:
  ryanofsky:
    Code review ACK 1ed52fbb4d. Changes since last review: various commit message, code comment, log message, error checking improvements, and fix for topping up inactive seeds if wallet isn't reloaded after calling sethdseed and test for this
  ariard:
    Code Review ACK 1ed52fb
  jonatack:
    ACK 1ed52fbb4d thanks for addressing the previous review feedback; would be happy to see the new review questions answered and feedback addressed and re-ack.

Tree-SHA512: e658ae0e1dab94be55d2b62cdda506c94815e73a6881533fd30d41cc77477f82fee2095144957a3a1df0c129e256bdd7b7abe3737d515f393610446cae4edf1c
2020-05-22 13:48:26 +12:00
fanquake
ad3a61c5f5 Merge #18895: p2p: unbroadcast followups: rpcs, nLastResend, mempool sanity check
651f1d816f [test] wait for inital broadcast before comparing mempool entries (gzhao408)
9d3f7eb986 [mempool] sanity check that all unbroadcast txns are in mempool (gzhao408)
a7ebe48b94 [rpc] add unbroadcast info to mempool entries and getmempoolinfo (gzhao408)
d160069604 [wallet] remove nLastResend logic (gzhao408)

Pull request description:

  Followup to #18038 by amitiuttarwar which introduces the unbroadcast set: "a mechanism for the mempool to track locally submitted transactions" and decreases the frequency of rebroadcast from 10-15 minutes to 12-36 hours.

  This PR addresses some of the outstanding TODOs building on top of it:
  - remove `nLastResend` logic, which is used to ensure rebroadcast doesn't happen again if no new block has been mined (makes sense in 10-15 min period, but not necessary for 12-36 hour period). (#18038 [comment](https://github.com/bitcoin/bitcoin/pull/18038#discussion_r416826914))
  - expose unbroadcast info via RPCs, for more informative queries and testing (#18038 [comment](https://github.com/bitcoin/bitcoin/pull/18038#discussion_r416837980))
  - add sanity check to verify unbroadcast transactions still exist in mempool before rebroadcasting (#18038 [comment](https://github.com/bitcoin/bitcoin/pull/18038#discussion_r416861609))

ACKs for top commit:
  naumenkogs:
    Code review ACK 651f1d816f
  amitiuttarwar:
    ACK 651f1d816f 🎉
  MarcoFalke:
    Review ACK 651f1d816f

Tree-SHA512: d5327e95ef39d44152b48df5c610502ae11c168f43dbbfb2885340c93d1ba9426eb3a5794573f5fc843502109cb3ffb63efa3f2db4f8f112efcde8f76d9a8845
2020-05-22 07:51:51 +08:00
Wladimir J. van der Laan
9abed46871 Merge #16946: wallet: include a checksum of encrypted private keys
d67055e00d Upgrade or rewrite encrypted key checksums (Andrew Chow)
c9a9ddb414 Set fDecryptionThoroughlyChecked based on whether crypted key checksums are valid (Andrew Chow)
a8334f7ac3 Read and write a checksum for encrypted keys (Andrew Chow)

Pull request description:

  Adds a checksum to the encrypted key record in the wallet database so that encrypted keys can be checked for corruption on wallet loading, in the same way that unencrypted keys are. This allows for us to skip the full decryption of keys upon the first unlocking of the wallet in that session as any key corruption will have already been detected. The checksum is just the double SHA256 of the encrypted key and it is appended to the record after the encrypted key itself.

  This is backwards compatible as old wallets will be able to read the encrypted key and ignore that there is more data in the stream. Additionally, old wallets will be upgraded upon their first unlocking (so that key decryption is checked before we commit to a checksum of the encrypted key) and a wallet flag set indicating that. The presence of the wallet flag lets us skip the full decryption as if `fDecryptionThoroughlyChecked` were true.

  This does mean that the first time an old wallet is unlocked in a new version will take much longer, but subsequent unlocks will be instantaneous. Furthermore, corruption will be detected upon loading rather than on trying to send so wallet corruption will be detected sooner.

  Fixes #12423

ACKs for top commit:
  laanwj:
    code review ACK d67055e00d
  jonatack:
    Code review ACK d67055e00d
  meshcollider:
    Code review ACK d67055e00d

Tree-SHA512: d5c1c10cfcb5db9e10dcf2326423565a9f499290b81f3155ec72254ed5bd7491e2ff5c50e98590eb07842c20d7797b4efa1c3475bae64971d500aad3b4e711d4
2020-05-21 20:50:25 +02:00
Wladimir J. van der Laan
4479eb04d9 Merge #18960: indexes: Add compact block filter headers cache
0187d4c118 [indexes] Add compact block filter headers cache (John Newbery)

Pull request description:

  Cache block filter headers at heights of multiples of 1000 in memory.

  Block filter headers at height 1000x are checkpointed, and will be the most frequently requested. Cache them in memory to avoid costly disk reads.

ACKs for top commit:
  jkczyz:
    ACK 0187d4c118
  theStack:
    ACK 0187d4c118 🎉
  fjahr:
    re-utACK 0187d4c118
  laanwj:
    code review ACK 0187d4c118
  ariard:
    Code Review ACK 0187d4c.

Tree-SHA512: 2075ae36901ebcdc4a217eae5203ebc8582181a0831fb7a53a119f031c46bca960a610a38a3d0636a9a405f713efcf4200c85f10c8559fd80139036d89473c56
2020-05-21 19:34:29 +02:00
Calvin Kim
c57f03ce17 refactor: Replace const char* to std::string
Some functions should be returning std::string instead of const char*.
This commit changes that.
2020-05-22 01:40:31 +09:00
Wladimir J. van der Laan
fed1a9043f Merge #19020: net: Use C++11 member initialization in protocol
fa8bbb1368 net: Use C++11 member initialization in protocol (MarcoFalke)

Pull request description:

  This change removes `Init` from the constructors and instead uses C++11 member initialization. This removes a bunch of boilerplate, makes the code easier to read. Also, C++11 member initialization avoids accidental uninitialized members.

ACKs for top commit:
  laanwj:
    ACK fa8bbb1368

Tree-SHA512: f89f6c2fe1bbfccd92acd72c0129d43e464339ed17e95384a81ed33a1a4257dba7ecc1534c6fc8c4668f0d9ade7ba0807b57066c6c763c1b72f74fc51f40907a
2020-05-21 17:44:03 +02:00
Wladimir J. van der Laan
99e5e21b38 Merge #19023: test: Fix intermittent ETIMEDOUT on FreeBSD
fab908f18a test: Fix intermittent ETIMEDOUT on FreeBSD (MarcoFalke)

Pull request description:

  Example backtrace: https://cirrus-ci.com/task/5307019047469056?command=functional_test#L1059

ACKs for top commit:
  laanwj:
    ACK fab908f18a

Tree-SHA512: 06e383e9993e083d6da4c546dde8811ace02559ddbb1c24e307938307763b3abe630699054e7067cf4aea993c82865e259b77b4bee518666a03d26e0f81c0656
2020-05-21 17:27:27 +02:00
MarcoFalke
7418169364 Merge #18997: gui: Remove un-actionable TODO
4444dbf4d5 gui: Remove un-actionable TODO (MarcoFalke)

Pull request description:

  With encryption turned on by default for all wallets in consideration (#18889), I believe that wallet decryption will not be implemented ever or at least any time soon. So remove that TODO comment for now. If deemed important, a brainstorming issue can be opened instead.

  Also remove some TODOs in the RPC console, which I don't understand. Maybe the gui was meant to show the debug log interactively? In any case, if deemed important, this should be filed as a brainstorming feature request, so that trade-offs of different solutions can be discussed.

ACKs for top commit:
  laanwj:
    Thanks. ACK 4444dbf4d5
  achow101:
    ACK 4444dbf4d5

Tree-SHA512: f7ddb37a14178f575da5409ea1c34e34bde37d79b2b56eaaf606a069e2b91c9d7b734529f5c68664b2fa5aa831117c8d19cce823743671cd6c31b81d68b8c70c
2020-05-21 10:58:28 -04:00
MarcoFalke
fab6b9d18f validation: Mark g_chainman DEPRECATED 2020-05-21 09:56:25 -04:00
MarcoFalke
fa1d97b256 validation: Make ProcessNewBlock*() members of ChainstateManager 2020-05-21 09:56:16 -04:00
MarcoFalke
fa24d49098 validation: Make PruneOneBlockFile() a member of ChainstateManager 2020-05-21 09:56:16 -04:00
MarcoFalke
fa84b1cd84 validation: Make LoadBlockIndex() a member of ChainstateManager 2020-05-21 09:55:59 -04:00
MarcoFalke
fa05fdf0f1 net: Pass chainman into PeerLogicValidation 2020-05-21 09:55:58 -04:00
MarcoFalke
fa7b626d7a node: Add chainman alias for g_chainman 2020-05-21 09:55:51 -04:00
MarcoFalke
cfe22a5f9e Merge #18530: Add test for -blocksonly and -whitelistforcerelay param interaction
0ea5d70b47 Updated comment for the condition where a transaction relay is denied (glowang)
be01449cc8 Add test for param interaction b/w -blocksonly and -whitelistforcerelay (glowang)

Pull request description:

  Related to: #18428

  When -blocksonly is turned on, a node would still relay transactions from whitelisted peers. This funcitonality has not been tested.

ACKs for top commit:
  MarcoFalke:
    ACK 0ea5d70b47

Tree-SHA512: 4e99c88281cb518cc67f5f3be7171a7b413933047b5d24a04bb3ff2210a82e914d69079f64cd5bac9206ec435e21a622c8e69cedbc2ccb39d2328ac5c01668e5
2020-05-21 09:00:25 -04:00
MarcoFalke
99993489da test: Set -logthreadnames in unit tests 2020-05-21 08:13:23 -04:00
MarcoFalke
fa4ea997b4 init: Setup scheduler in tests and init in exactly the same way 2020-05-21 08:13:00 -04:00
MarcoFalke
fad798be76 test: Default --previous-releases to false if dir is empty 2020-05-21 07:48:06 -04:00
MarcoFalke
faf1c3cc58 test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option 2020-05-21 07:47:46 -04:00
MarcoFalke
25ad2c623a Merge #18740: Remove g_rpc_node global
b3f7f375ef refactor: Remove g_rpc_node global (Russell Yanofsky)
ccb5059ee8 scripted-diff: Remove g_rpc_node references (Russell Yanofsky)
6fca33b2ed refactor: Pass NodeContext to RPC and REST methods through util::Ref (Russell Yanofsky)
691c817b34 Add util::Ref class as temporary alternative for c++17 std::any (Russell Yanofsky)

Pull request description:

  This PR removes the `g_rpc_node` global, to get same benefits we see removing other globals and make RPC code more testable, modular, and reusable.

  This uses a hybrid of the approaches suggested in #17548. Instead of using `std::any`, which isn't available in c++11, or `void*`, which isn't type safe, it uses a small new `util::Ref` helper class, which acts like a simplified `std::any` that only holds references, not values.

  Motivation for writing this was to provide an simpler alternative to #18647 by Harris Brakmić (brakmic) which avoids some shortcomings of that PR (https://github.com/bitcoin/bitcoin/pull/18647#issuecomment-617878826)

ACKs for top commit:
  MarcoFalke:
    re-ACK b3f7f375ef, only change is adding back const and more tests 🚾
  ajtowns:
    ACK b3f7f375ef

Tree-SHA512: 56292268a001bdbe34d641db1180c215351503966ff451e55cc96c9137f1d262225d7d7733de9c9da7ce7d7a4b34213a98c2476266b58c89dbbb0f3cb5aa5d70
2020-05-21 06:53:39 -04:00
Jon Atack
5edad5ce5d test: add -getinfo multiwallet functional tests
and improve the existing -getinfo -rpcwallet tests.
2020-05-21 10:24:30 +02:00
Jon Atack
903b6c117f rpc: drop unused JSONRPCProcessBatchReply size arg, refactor 2020-05-21 10:24:26 +02:00
Jon Atack
afce85eb99 cli: use GetWalletBalances() functionality for -getinfo
and replace GetBoolArg with IsArgSet as we only want
to know if the arg is passed; we do not need the value.
2020-05-21 10:24:23 +02:00
Jon Atack
9f01849a49 cli: create GetWalletBalances() to fetch multiwallet balances 2020-05-21 10:24:20 +02:00
Jon Atack
743077544b cli: lift -rpcwallet logic up to CommandLineRPC()
to allow passing rpcwallet independently from the -rpcwallet user option, and to
move the logic to the top-level layer where most of the other option args are
handled.
2020-05-21 10:24:17 +02:00
Jon Atack
29f2cbdeb7 cli: extract connection exception handler, -rpcwait logic
to ConnectAndCallRPC() to be callable for individual connections.

This is needed for RPCs that need to be called and handled sequentially, rather
than alone or in a batch.

For example, when fetching the balances for each loaded wallet, -getinfo will
call RPC listwallets, and then, depending on the result, RPC getbalances.

It may be somewhat helpful to review this commit with `git show -w`.
2020-05-21 10:23:48 +02:00
fanquake
97b21b302a Merge #18677: Multiprocess build support
e2bab2aa16 multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e7 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52b build: multiprocess autotools changes (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.

  In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.

  The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649

ACKs for top commit:
  practicalswift:
    ACK e2bab2aa16
  Sjors:
    tACK e2bab2aa16 on macOS 10.15.4
  hebasto:
    ACK e2bab2aa16, tested on Linux Mint 19.3 (x86_64):

Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
2020-05-21 15:34:25 +08:00
fanquake
17df15efb5 Merge #18958: guix: Make V=1 more powerful for debugging
f852761aec guix: Add clarifying documentation for V env var (Carl Dong)
85f4a4b082 guix: Make V=1 more powerful for debugging (Carl Dong)

Pull request description:

  ```
  - Print commands in both unexpanded and expanded forms
  - Set VERBOSE=1 for CMake
  ```

  Ping MarcoFalke hopefully you use `V=1` already for the Guix builds on DrahtBot?

ACKs for top commit:
  fanquake:
    ACK f852761aec. Ran a Windows Guix build and compared the output from master and this PR when using `V=1`. i.e `HOSTS=x86_64-w64-mingw32 PATH="/root/.config/guix/current/bin${PATH:+:}$PATH" V=1 ./contrib/guix/guix-build.sh`.

Tree-SHA512: 8bc466fa7b869618bbd5a0a91c6b23d4785009289f8dfb93b0349317463a9ab9ece128c72436e02a0819722a63e703100aed15807867a716fda891292fcb9d9d
2020-05-21 15:09:48 +08:00
10xcryptodev
60ed33904c tests: implement base58_decode 2020-05-20 18:56:08 -03:00
MarcoFalke
3eda7ea9ba Merge #18764: refactor: test: replace inv type magic numbers by constants
4a614ff88a test: explicit imports from test_framework.messages in p2p_invalid_messages.py (Sebastian Falbesoner)
b35e1d2471 test: add inventory type constant MSG_CMPCT_BLOCK (Sebastian Falbesoner)
eeaaa58d2c test: replace inv type magic numbers by constants (Sebastian Falbesoner)

Pull request description:

  Many functional tests still use magic numbers for inventory types, either passed to the `CInv` constructor or for comparing the `type` member of `CInv`. This PR replaces all of those by constants in the module `test_framework.messages` that have been introduced in commit c32cf9f622: `MSG_TX` (1) or `MSG_BLOCK` (2).

  It also introduces a new constant `MSG_CMPCT_BLOCK` (naming as in `src/protocol.h`) and uses it to replace the remaining magic numbers.

  The occurences of the magic numbers were identified through `grep`ing for `CInv(` and `type ==`. The idea was first to create a scripted-diff, but since also adding missing `import`s is needed, this would be non-trivial. Besides, also some unneeded comments like `# 2 == "Block"` could be removed.

ACKs for top commit:
  gzhao408:
    ACK [`4a614ff`](4a614ff88a)

Tree-SHA512: 4ba4fdef9f3eef7fd5ac72cb03ca3524863d1ae292161c550424a4c1047283fa2d2e7e03017d1fbae3652b3cb14f08b8d4b368403f3f209993aef3f2e2b22784
2020-05-20 15:11:51 -04:00
Pieter Wuille
65c589e45e Convert Qt to new serialization 2020-05-20 10:16:41 -07:00
Carl Dong
f852761aec guix: Add clarifying documentation for V env var 2020-05-20 13:11:16 -04:00
MarcoFalke
fa8bbb1368 net: Use C++11 member initialization in protocol 2020-05-20 08:27:07 -04:00
MarcoFalke
448bdff263 Merge #18317: Serialization improvements step 6 (all except wallet/gui)
f9ee0f37c2 Add comments to CustomUintFormatter (Pieter Wuille)
4eb5643e35 Convert everything except wallet/qt to new serialization (Pieter Wuille)
2b1f85e8c5 Convert blockencodings_tests to new serialization (Pieter Wuille)
73747afbbe Convert merkleblock to new serialization (Pieter Wuille)
d06fedd1bc Add SER_READ and SER_WRITE for read/write-dependent statements (Russell Yanofsky)
6f9a1e5ad0 Extend CustomUintFormatter to support enums (Russell Yanofsky)
769ee5fa00 Merge BigEndian functionality into CustomUintFormatter (Pieter Wuille)

Pull request description:

  The next step of changes from #10785.

  This:
  * Adds support for enum serialization to `CustomUintFormatter`, used in `CAddress` for service flags.
  * Merges `BigEndian` into `CustomUintFormatter`, used in `CNetAddr` for port numbers.
  * Converts everything (except wallet and gui) to use the new serialization framework.

ACKs for top commit:
  MarcoFalke:
    re-ACK f9ee0f37c2, only change is new documentation commit for CustomUintFormatter 📂
  ryanofsky:
    Code review ACK f9ee0f37c2. Just new commit adding comment since last review
  jonatack:
    Code review re-ACK f9ee0f37c2 only change since last review is an additional commit adding Doxygen documentation for `CustomUintFormatter`.

Tree-SHA512: e7a0a36afae592d5a4ff8c81ae04d858ac409388e361f2bc197d9a78abca45134218497ab2dfd6d031e0cce0ca586cf857077b7c6ce17fccf67e2d367c1b6cd4
2020-05-20 07:30:29 -04:00
MarcoFalke
e20e964cb1 Merge #18996: net: Remove un-actionable TODO
fabea6d404 net: Run clang-format on protocol.h (MarcoFalke)
facdeea2b2 net: Remove un-actionable TODO (MarcoFalke)

Pull request description:

  The first commit removes a TODO that is infeasible to solve. Currently, most (de)serializable classes in Bitcoin Core have public members. For example `CMessageHeader`, `FlatFilePos`, `CBlock`, `CTransaction`, `CCoin`, ...

  So either this TODO comment should apply to all classes or to none. Fix that discrepancy by removing it from the source code for now. If deemed important, the TODO can be discussed in a brainstorming issue later.

  Also run clang format on the header file in a new commit. Happy to drop this commit if it is too controversial, but I think it is trivial to review and makes the workflow of developers using clang-format-diff easier.

ACKs for top commit:
  practicalswift:
    ACK fabea6d404
  naumenkogs:
    ACK fabea6d. Not sure why that TODO was there in the first place, but Marco's justification seems correct.
  hebasto:
    ACK fabea6d404, agree with both changes: removing TODO and applying the `clang-format-diff.py`.

Tree-SHA512: b79ae07be27e5a40fc9f411a5e9ae91aecb2fdedbcbf74699614a1004f4ef816bf396903ec6c06eb1395fd83a2047620c7583acbaadfb8c4e613319a63062c3c
2020-05-20 07:27:53 -04:00
MarcoFalke
bd5ec7c528 Merge #19006: rpc: Avoid crash when g_thread_http was never started
faf45d1f1f http: Avoid crash when g_thread_http was never started (MarcoFalke)
fa12a37b27 test: Replace inline-comments with logs, pep8 formatting (MarcoFalke)
fa83b39ff3 init: Remove confusing and redundant InitError (MarcoFalke)

Pull request description:

  Avoid a crash during shutdown when the init sequence failed for some reason

ACKs for top commit:
  promag:
    Tested ACK faf45d1f1f.
  ryanofsky:
    Code review ACK faf45d1f1f. Thanks for updates, this is much easier to parse for me now. Since previous reviews: split out and reverted some cleanups & replaced chmod with mkdir in test
  hebasto:
    ACK faf45d1f1f, tested on Linux Mint 19.3 with the following patch:

Tree-SHA512: 59632bf01c999e65c724e2728ac103250ccd8b0b16fac19d3a2a82639ab73e4f2efb86c78e63c588a5954625d8d0cf9545e2a7e070e6e15d2a54beeb50e00b61
2020-05-20 07:25:04 -04:00
Jonas Schnelli
a587f85853 Merge #18587: gui: Avoid wallet tryGetBalances calls in WalletModel::pollBalanceChanged
d3a56be77a Revert "gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged" (Russell Yanofsky)
bf0a510981 gui: Avoid wallet tryGetBalances calls before TransactionChanged or BlockTip notifications (Russell Yanofsky)
2bc9b92ed8 Cancel wallet balance timer when shutdown requested (Russell Yanofsky)
83f69fab3a Switch transaction table to use wallet height not node height (Russell Yanofsky)

Pull request description:

  Main commit `gui: Avoid wallet tryGetBalances calls` is one-line change to `WalletModel::pollBalanceChanged` that returns early if there hasn't been a new `TransactionChanged` or `BlockTip` notification since the previous poll call. This is the same behavior that was implemented in #18160, now implemented in a simpler way.

  The other commits are a straight revert of #18160, and two tweaks to avoid relying on `WalletModel::m_client_model` lifetime which were causing travis failures with earlier versions of this PR.

  Motivation for this change is to be able to revert #18160 and cut down on unnecessary cross-process calls that happen when #18160 is combined with #10102

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

ACKs for top commit:
  jonasschnelli:
    utACK d3a56be77a

Tree-SHA512: 3cd31ca515e77c3bd7160d3f1ea0dce5050d4038b2aa441b6f66b8599bd413d81ca5542a197806e773d6092dd1d26830932b1cecbc95298b1f1ab41099e2f12f
2020-05-20 11:09:29 +02:00
fanquake
0aa2ff0f66 Merge #18956: build: enforce minimum required Windows version (7)
e8a8cff07c build: enforce minimum required Windows version (7) (fanquake)

Pull request description:

  Instruct the linker to set the major & minor subsystem versions in the PE
  header to 6 & 1 (NT 6.1 which corresponds to Windows 7). Similar to
  the behaviour on macOS, the binary will now refuse to run on
  unsupported versions of Windows, which, for us, is XP & Vista.

  ![windows_no_run](https://user-images.githubusercontent.com/863730/81654555-38e0fd00-9468-11ea-9cc8-caf37dec5713.png)

ACKs for top commit:
  laanwj:
    ACK e8a8cff07c

Tree-SHA512: 2f7c6443b79b1c6b995e337452aa177e95b0a9c48e47bcf1893aad6fd598e45940ab8eaa5ee1c5d994a521239b4e1b55a55bb3e8ffe367e1349db2a46892a6d4
2020-05-20 10:15:32 +08:00
MarcoFalke
fab908f18a test: Fix intermittent ETIMEDOUT on FreeBSD 2020-05-19 19:12:41 -04:00
MarcoFalke
fa2ca0cbdd test: Fix intermittent failure in feature_dbcrash 2020-05-19 18:57:09 -04:00
Pieter Wuille
f9ee0f37c2 Add comments to CustomUintFormatter 2020-05-19 14:30:30 -07:00
gzhao408
651f1d816f [test] wait for inital broadcast before comparing mempool entries
- mempool entry 'unbroadcast' field changes when tx passes initial broadcast (receive getdata),
so anytime you compare mempool entries as a whole, you must wait for all broadcasts to complete
('unbroadcast' = False) otherwise the state may change in between calls
- update P2PTxInvStore to send msg_getdata for invs and add functionality to wait for a list
of txids to complete initial broadcast
- make mempool_packages.py wait because it compares entries using getrawmempool and
getmempoolentry
2020-05-19 14:24:27 -07:00
gzhao408
9d3f7eb986 [mempool] sanity check that all unbroadcast txns are in mempool
- before reattempting broadcast for unbroadcast txns, check they are in mempool and remove if not
- this protects from memory leaks and network spam just in case unbroadcast set (incorrectly) has extra txns
- check that tx is in mempool before adding to unbroadcast set to try to prevent this from happening
2020-05-19 14:23:19 -07:00
gzhao408
a7ebe48b94 [rpc] add unbroadcast info to mempool entries and getmempoolinfo
- expose info about number of txns in unbroadcast set and whether a mempool entry's tx has passed initial broadcast
- makes rpcs more informative and allows for more explicit testing, eg tracking if tx is in unbroadcast set
before and after originating node connects to peers (adds this in mempool_unbroadcast.py)
- adds mempool method IsUnbroadcastTx to query for tx inclusion in  mempool's unbroadcast set
2020-05-19 14:23:13 -07:00
Ivan Vershigora
d0a3feea73 Change docs for walletcreatefundedpsbt RPC method 2020-05-19 19:32:27 +03:00
MarcoFalke
faf45d1f1f http: Avoid crash when g_thread_http was never started
g_thread_http can not be joined when it is not joinable. Avoid crashing
the node by adding the required check and add a test.
2020-05-19 10:41:44 -04:00
MarcoFalke
fa12a37b27 test: Replace inline-comments with logs, pep8 formatting 2020-05-19 10:41:30 -04:00
MarcoFalke
fa83b39ff3 init: Remove confusing and redundant InitError
The "A fatal internal error occurred, see debug.log for details" is
redundant because init.cpp will already show an InitError with a better
error message as well as the hint to check the debug.log
2020-05-19 10:37:15 -04:00
MarcoFalke
aa8d76806c Merge #17946: Fix GBT: Restore "!segwit" and "csv" to "rules" key
412d5fe879 QA: feature_segwit: Check that template "rules" includes "!segwit" as appropriate (Luke Dashjr)
2abe8cc3b7 Bugfix: Include "csv","!segwit" in "rules" (Luke Dashjr)

Pull request description:

  #16060 removed CSV & segwit from versionbits, breaking the "rules" key returned by GBT.

  Without this, miners don't know they're mining segwit blocks, and should fall back to pre-segwit block creation.

ACKs for top commit:
  sipa:
    ACK 412d5fe879
  jnewbery:
    Tested ACK 412d5fe879.

Tree-SHA512: 825d72e257dc0dd4941f2fe498d8d4f4f2a21b9505cd21a8f9eb7fb5d6d7dd9219347928cf90bb57a777920ce24295859763e64fa8a22ebb58fc2380f80f5615
2020-05-19 08:54:23 -04:00
Jonas Schnelli
d44dd51322 Merge #18152: qt: Use SynchronizationState enum for signals to GUI
a0d0f1c6c3 refactor: Remove Node:: queries from GUI (Hennadii Stepanov)
06d519f0b4 qt: Add SynchronizationState enum to signal parameter (Hennadii Stepanov)
3c709aa69d refactor: Remove Node::getReindex() call from GUI (Hennadii Stepanov)
1dab574edf refactor: Pass SynchronizationState enum to GUI (Hennadii Stepanov)
2bec309ad6 refactor: Remove unused bool parameter in RPCNotifyBlockChange() (Hennadii Stepanov)
1df77014d8 refactor: Remove unused bool parameter in BlockNotifyGenesisWait() (Hennadii Stepanov)

Pull request description:

  This PR is a followup of #18121 and:
  - addresses confusion about GUI notification throttling conditions (**luke-jr**'s [comment](https://github.com/bitcoin/bitcoin/pull/18121#discussion_r378552386), **ryanofsky**'s [comment](https://github.com/bitcoin/bitcoin/pull/18121#discussion_r378975960))
  - removes `isInitialBlockDownload()` call from the GUI back to the node (on macOS). See:  **ryanofsky**'s [comment](https://github.com/bitcoin/bitcoin/pull/18121#pullrequestreview-357730284)

ACKs for top commit:
  jonasschnelli:
    Core Review ACK a0d0f1c6c3 (modulo [question](https://github.com/bitcoin/bitcoin/pull/18152#pullrequestreview-414140601)).
  ryanofsky:
    Code review ACK a0d0f1c6c3. Only changes since last review were rebase and tweaking SynchronizationState enum declaration as suggested (thanks!)

Tree-SHA512: b6a712a710666e763aeee0d5440de1391a4c6c8f7fa661888773e1ba59e9e0f83654ee384d4edc704031be7eb25616e5eca2a6e26058d3efb7f64c47f9ed7316
2020-05-19 14:35:02 +02:00
fanquake
e7736854ef Merge #19008: ci: tsan on clang-9
faf552117e ci: Set DEBIAN_FRONTEND=noninteractive (MarcoFalke)
fa006caa13 ci: tsan on clang-9 (MarcoFalke)

Pull request description:

  Bump the compiler runtime library that includes the sanitizers from clang-8 to clang-9 to get a more recent version. Also, bump the system packages from xenial to bionic to test packages closer to what is commonly used in production.

  The second commit is needed to install the `tzdata` package, which is missing on some operating systems. See https://travis-ci.org/github/MarcoFalke/bitcoin-core/jobs/688455828#L1727

ACKs for top commit:
  hebasto:
    ACK faf552117e
  practicalswift:
    ACK faf552117e -- patch looks correct and Travis is happy

Tree-SHA512: aa38fdae5f716966a83a21d5f7c121675cf7d663148ab3baa065142c8b3850bcd4bf88526d7da0fa51f5e08f2c317b537f950fcc9eb1e69fdacb0eac8863e1c6
2020-05-19 16:10:23 +08:00
fanquake
042ff52142 Merge #18999: log: Remove "No rpcpassword set" from logs
fa243be1dc log: Remove "No rpcpassword set" from logs (MarcoFalke)

Pull request description:

  rpcpassword is deprecated and not recommended anymore. So remove it from the logs, which indicate that an rpcpassword should be set and cause confusion. See #18998.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa243be1dc. New log message makes more sense
  elichai:
    Re Code Review ACK (Checked the diff) fa243be1dc

Tree-SHA512: de3e0800a204b15a59a59a7e6f345013ee9d38e8c5d0c9a94d6142780faa9cce672ed358c7571f53c1eb843bf5afb0b7bcbfd289d3b9e2e0bf8ff2fd361e98a9
2020-05-19 15:41:21 +08:00
fanquake
c73bd004ae Merge #18861: Do not answer GETDATA for to-be-announced tx
2896c412fa Do not answer GETDATA for to-be-announced tx (Pieter Wuille)
f2f32a3dee Push down use of cs_main into FindTxForGetData (Pieter Wuille)
c6131bf407 Abstract logic to determine whether to answer tx GETDATA (Pieter Wuille)

Pull request description:

  This PR intends to improve transaction-origin privacy.

  In general, we should try to not leak information about what transactions we have (recently) learned about before deciding to announce them to our peers. There is a controlled transaction dissemination process that reveals our transactions to peers that has various safeguards for privacy (it's rate-limited, delayed & batched, deterministically sorted, ...), and ideally there is no way to test which transactions we have before that controlled process reveals them. The handling of the `mempool` BIP35 message has protections in this regard as well, as it would be an obvious way to bypass these protections (handled asynchronously after a delay, also deterministically sorted).

  However, currently, if we receive a GETDATA for a transaction that we have not yet announced to the requester, we will still respond to it if it was announced to *some* other peer already (because it needs to be in `mapRelay`, which only happens on the first announcement). This is a slight privacy leak.

  Thankfully, this seems easy to solve: `setInventontoryTxToSend` keeps track of the txids we have yet to announce to a peer - which almost(*) exactly corresponds to the transactions we know of that we haven't revealed to that peer. By checking whether a txid is in that set before responding to a GETDATA, we can filter these out.

  (*) Locally resubmitted or rebroadcasted transactions may end up in setInventoryTxToSend while the peer already knows we have them, which could result in us incorrectly claiming we don't have such transactions if coincidentally requested right after we schedule reannouncing them, but before they're actually INVed. This is made even harder by the fact that filterInventoryKnown will generally keep known reannouncements out of setInventoryTxToSend unless it overflows (which needs 50000 INVs in either direction before it happens).

  The condition for responding now becomes:

  ```
    (not in setInventoryTxToSend) AND
    (
      (in relay map) OR
      (
        (in mempool) AND
        (old enough that it could have expired from relay map) AND
        (older than our last getmempool response)
      )
    )
  ```

ACKs for top commit:
  naumenkogs:
    utACK 2896c41
  ajtowns:
    ACK 2896c412fa
  amitiuttarwar:
    code review ACK 2896c412fa
  jonatack:
    ACK 2896c412fa per `git diff 2b3f101 2896c41` only change since previous review is moving the recency check up to be verified first in `FindTxForGetData`, as it was originally in 353a391 (good catch), before looking up the transaction in the relay pool.
  jnewbery:
    code review ACK 2896c412fa

Tree-SHA512: e7d5bc006e626f60a2c108a9334f3bbb67205ace04a7450a1e4d4db1d85922a7589e0524500b7b4953762cf70554c4a08eec62c7b38b486cbca3d86321600868
2020-05-19 15:18:06 +08:00
Anthony Towns
a788789948 test/checkqueue_tests: thread safety annotations 2020-05-19 16:33:10 +10:00
Anthony Towns
479c5846f7 rpc/blockchain.cpp: thread safety annotations for latestblock 2020-05-19 16:33:10 +10:00
Anthony Towns
8b5af3d4c1 net: fMsgProcWake use LOCK instead of lock_guard 2020-05-19 16:33:10 +10:00
Anthony Towns
de7c5f41ab wallet/wallet.h: Remove mutexScanning which was only protecting a single atomic bool 2020-05-19 16:33:02 +10:00
MarcoFalke
faf552117e ci: Set DEBIAN_FRONTEND=noninteractive
Also fix travis environment variables for this build

Previously the resulting env was:

FILE_ENV="./ci/test/00_setup_env_native_tsan.sh" TEST_RUNNER_EXTRA="--exclude feature_block" #= Not= enough= memory= on= travis= machines=
2020-05-18 20:38:56 -04:00
MarcoFalke
fa006caa13 ci: tsan on clang-9 2020-05-18 20:38:54 -04:00
Hennadii Stepanov
a0d0f1c6c3 refactor: Remove Node:: queries from GUI 2020-05-19 03:01:53 +03:00
Hennadii Stepanov
06d519f0b4 qt: Add SynchronizationState enum to signal parameter 2020-05-19 03:01:42 +03:00
MarcoFalke
362f9c60a5 Merge #18986: tests: Add capability to disable RPC timeout in functional tests
38c3dd9c70 docs: Add notes on how to diasble rpc timeout in functional tests while attatching gdb. (codeShark149)
784ae09625 test: Add capability to disable RPC timeout in functional tests. (codeShark149)

Pull request description:

  Many times, especially while debugging RPC callbacks to core using gdb, the test timeout kicks in before the response can get back. This can be annoying and requires restarting the functional test as well as gdb attachment.

  This PR adds a `--notimeout` flag into `test_framework` and sets the `rpc_timeout` accordingly if the flag is set.

  The same effect can be achieved with newly added `--factor` flag but keeping a separate flag that explicitly disables the timeout can be easier for new testers to find it out and separates its purpose from the `--factor` flag.

  Requesting review ryanofsky jnewbery as per the IRC discussion.

  Update: After initial round of review, the approach is modified to accommodate the functionality in already existing `--factor` flag. `--factor` is changed to `--timeout-factor` to express its intent better.

ACKs for top commit:
  MarcoFalke:
    ACK 38c3dd9c70 and thanks for fixing up all my typos 😅
  jnewbery:
    ACK 38c3dd9c70.

Tree-SHA512: 9458dd1010288c62f8bb83f7a4893284fbbf938882dd65fc9e08810a910db07ef676e3100266028e5d4c8ce407b2267b3860595015da070c84a9d4a9816797db
2020-05-18 20:01:13 -04:00
Hennadii Stepanov
3c709aa69d refactor: Remove Node::getReindex() call from GUI 2020-05-19 02:49:48 +03:00
Hennadii Stepanov
1dab574edf refactor: Pass SynchronizationState enum to GUI
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-05-19 02:49:32 +03:00
Hennadii Stepanov
2bec309ad6 refactor: Remove unused bool parameter in RPCNotifyBlockChange() 2020-05-19 02:39:45 +03:00
Hennadii Stepanov
1df77014d8 refactor: Remove unused bool parameter in BlockNotifyGenesisWait() 2020-05-19 02:39:33 +03:00
Hennadii Stepanov
26c093a995 Replace thread_local g_lockstack with a mutex-protected map
This change prevents UB in case of early g_lockstack destroying.

Co-authored-by: Wladimir J. van der Laan <laanwj@protonmail.com>
2020-05-19 01:14:08 +03:00
Hennadii Stepanov
58e6881bc5 refactor: Refactor duplicated code into LockHeld() 2020-05-19 01:14:07 +03:00
Hennadii Stepanov
f511f61dda refactor: Add LockPair type alias 2020-05-19 01:14:07 +03:00
Hennadii Stepanov
8d8921abd3 refactor: Add LockStackItem type alias 2020-05-19 01:14:07 +03:00
Hennadii Stepanov
458992b06d Prevent UB in DeleteLock() function 2020-05-19 01:13:49 +03:00
John Newbery
0187d4c118 [indexes] Add compact block filter headers cache
Cache block filter headers at heights of multiples of 1000 in memory.

Block filter headers at height 1000x are checkpointed, and will be the
most frequently requested. Cache them in memory to avoid costly disk
reads.
2020-05-18 12:54:07 -04:00
codeShark149
38c3dd9c70 docs: Add notes on how to diasble rpc timeout in functional tests while attatching gdb. 2020-05-18 21:18:40 +05:30
codeShark149
784ae09625 test: Add capability to disable RPC timeout in functional tests.
Modifies the existing --factor flag to --timeout-factor to better express intent.
Adds rules to disable timeout if --timeout-factor is set to 0.
Modfies --timeout-factor help doc to inform users about this feature.
2020-05-18 21:18:04 +05:30
gzhao408
d160069604 [wallet] remove nLastResend logic
remove nLastResend because it's unnecessary now that rebroadcasts always happen at least 12 hours later
2020-05-17 17:52:11 -07:00
MarcoFalke
fa243be1dc log: Remove "No rpcpassword set" from logs 2020-05-17 12:39:26 -04:00
glowang
0ea5d70b47 Updated comment for the condition where a transaction relay is denied 2020-05-17 08:33:09 -07:00
MarcoFalke
4444dbf4d5 gui: Remove un-actionable TODO 2020-05-17 10:52:31 -04:00
MarcoFalke
fabea6d404 net: Run clang-format on protocol.h
Can be reviewed with the git diff flags
-U0 --ignore-all-space --word-diff-regex=.
2020-05-17 10:26:19 -04:00
MarcoFalke
facdeea2b2 net: Remove un-actionable TODO 2020-05-17 10:24:16 -04:00
MarcoFalke
dc5333d31f Merge #18938: tests: Fill fuzzing coverage gaps for functions in consensus/validation.h, primitives/block.h and util/translation.h
cd34038cbd Switch from Optional<T> to std::optional<T> (C++17). Run clang-format. (practicalswift)
fb559c1170 tests: Fill fuzzing coverage gaps for functions in util/translation.h (practicalswift)
b74f3d6c45 tests: Fill fuzzing coverage gaps for functions in consensus/validation.h (practicalswift)
c0bbf8193d tests: Fill fuzzing coverage gaps for functions in primitives/block.h (practicalswift)

Pull request description:

  * Fill fuzzing coverage gaps for functions in `consensus/validation.h`
  * Fill fuzzing coverage gaps for functions in `primitives/block.h`
  * Fill fuzzing coverage gaps for functions in `util/translation.h`
  * Switch from `Optional<T>` to `std::optional<T>` (C++17). Run `clang-format`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: d6aa4634c3953ade173589a8239bd230eb317ef897835a8557acb73df01b25e5e17bf46f837838e59ec04c1f3d3b7d1309ba68c8a264d17b938215512c9e6085
2020-05-17 08:19:39 -04:00
MarcoFalke
fa9f20b647 log: Properly log txs rejected from mempool 2020-05-16 10:37:43 -04:00
MarcoFalke
f8123d483c Merge #18952: test: avoid os-dependant path
8a22fd0114 avoided os-dependant path (Ferdinando M. Ametrano)

Pull request description:

  The current code fails on windows because of the forward slashes; using os.path.join solves the problem and it is in general more robust

ACKs for top commit:
  MarcoFalke:
    ACK 8a22fd0114

Tree-SHA512: 813f27aea33f97c8afac52e716a55fc5d7fb69621023aba99d40df7e1d145e0ec8d1eee49ddd403b219bf0e0e168e0e987b05c78eaef611f744d99bf2fc8bc91
2020-05-16 06:16:47 -04:00
MarcoFalke
5a454d7882 Merge #18634: ci: Add fuzzbuzz integration configuration file
8d306862ef ci: Add fuzzbuzz integration (practicalswift)

Pull request description:

  Add fuzzbuzz integration.

  Just like #15338 enabled optional FreeBSD building via Cirrus CI (`.cirrus.yml`) this PR adds optional fuzzing via fuzzbuzz (`.fuzzbuzz.yml`).

  Having this merged makes is easier for people to fuzz Bitcoin Core (via their forked repos) using their fuzzbuzz account and then hopefully submit coverage increasing corpus additions upstreams to to https://github.com/bitcoin-core/qa-assets.

  Historically it has been mostly been me and MarcoFalke who submit test cases to `qa-assets`, but with this change hopefully more people will join the hunt for coverage increasing fuzzing inputs :)

Top commit has no ACKs.

Tree-SHA512: c7d8e354996c673da36cc9add260383c82a5325bfaa7ce6141ad6cd6b7d6adf3a6c900ea2db17fb70147b3625fa7f6a1ff8ba813aeaa299f316d8f6cabb3a65c
2020-05-16 06:14:45 -04:00
William Casarin
83a425d25a compressor: use a prevector in compressed script serialization
Use a prevector for stack allocation instead of heap allocation during
script compression and decompression. These functions were doing
millions of unnecessary heap allocations during IBD.

We introduce a CompressedScript type alias for this prevector. It is
size 33 as that is the maximum size of a compressed script.

Fix the DecompressScript header to match the variable name from
compressor.cpp

Signed-off-by: William Casarin <jb55@jb55.com>
2020-05-15 15:26:54 -07:00
Andrew Chow
1ed52fbb4d Remove IBD check in sethdseed
It is no longer necessary to wait for IBD to be complete before setting
a HD seed. This check was originally to ensure that restoring an old
seed on an out of sync node would scan the entire blockchain and thus
not miss transactions that involved keys that were not in the keypool.
This was necessary as once the seed was changed, no further keys would
be derived from the old seed(s).

As we are now topping up inactive seeds as we find those keys to be
used, this check is no longer necessary. During IBD, each time we
find a used key belonging to an inactive hd seed, we will still generate
more keys from that inactive seed.
2020-05-15 18:00:10 -04:00
Andrew Chow
b1810a145a Test that keys from inactive seeds are generated 2020-05-15 18:00:10 -04:00
Andrew Chow
c93082ece4 Generate new keys for inactive seeds after marking used
When a key from an inactive seed is used, generate replacements
to fill a keypool that would have been there.
2020-05-15 18:00:10 -04:00
Andrew Chow
45f2f6a0e8 Determine inactive HD seeds from key metadata and track them in LegacyScriptPubKeyMan 2020-05-15 18:00:04 -04:00
Tom Harding
5fa067a27d Remove unnecessary blockfile SetPos
Nothing could have changed the position since we retrieved it a few statements earlier. This dates from commit 16d5194165.
2020-05-15 12:19:10 -07:00
Russell Yanofsky
b604c5c8b5 wallet: Minimal fix to restore conflicted transaction notifications
This fix is a based on the fix by Antoine Riard <ariard@student.42.fr> in
https://github.com/bitcoin/bitcoin/pull/18600.

Unlike that PR, which implements some new behavior, this just restores previous
wallet notification and status behavior for transactions removed from the
mempool because they conflict with transactions in a block. The behavior was
accidentally changed in two `CWallet::BlockConnected` updates:
a31be09bfd and
7e89994133 from
https://github.com/bitcoin/bitcoin/pull/16624, causing issue
https://github.com/bitcoin/bitcoin/issues/18325.

The change here could be improved and replaced with a more comprehensive
cleanup, so it includes a detailed comment explaining future considerations.

Fixes #18325

Co-authored-by: Antoine Riard <ariard@student.42.fr>
2020-05-15 09:23:55 -04:00
MarcoFalke
951870807e Merge #18781: Add templated GetRandDuration<>
0000ea3265 test: Add test for GetRandMillis and GetRandMicros (MarcoFalke)
fa0e5b89cf Add templated GetRandomDuration<> (MarcoFalke)

Pull request description:

  A naive implementation of this template is dangerous, because the call site might accidentally omit the template parameter:

  ```cpp
  template <typename D>
  D GetRandDur(const D& duration_max)
  {
      return D{GetRand(duration_max.count())};
  }

  BOOST_AUTO_TEST_CASE(util_time_GetRandTime)
  {
      std::chrono::seconds rand_hour = GetRandDur(std::chrono::hours{1});
      // Want seconds to be in range [0..1hour), but always get zero :((((
      BOOST_CHECK_EQUAL(rand_hour.count(), 0);
  }
  ```

  Luckily `std::common_type` is already specialised in the standard lib for `std::chrono::duration` (https://en.cppreference.com/w/cpp/chrono/duration/common_type). And its effect seem to be that the call site must always specify the template argument explicitly.

  So instead of implementing the function for each duration type by hand, replace it with a templated version that is safe to use.

ACKs for top commit:
  laanwj:
    Code review ACK 0000ea3265
  promag:
    Code review ACK 0000ea3265.
  jonatack:
    ACK 0000ea3 thanks for the improved documentation. Code review, built, ran `src/test/test_bitcoin -t random_tests -l test_suite` for the new unit tests, `git diff fa05a4c 0000ea3` since previous review:
  hebasto:
    ACK 0000ea3265 with non-blocking [nit](https://github.com/bitcoin/bitcoin/pull/18781#discussion_r424924671).

Tree-SHA512: e89d46e31452be6ea14269ecbbb2cdd9ae83b4412cd14dff7d1084283092722a2f847cb501e8054394e4a3eff852f9c87f6d694fd008b3f7e8458cb5a3068af7
2020-05-15 08:58:49 -04:00
fanquake
e2f6866cca Merge #18975: test: Remove const to work around compiler error on xenial
050e2ee6f2 test: Remove const to work around compiler error on xenial (Wladimir J. van der Laan)

Pull request description:

  Fix the following error in travis:

      test/validationinterface_tests.cpp:26:36: error: default initialization of an object of const type 'const BlockValidationState' without a user-provided default constructor

      const BlockValidationState state_dummy;

ACKs for top commit:
  MarcoFalke:
    Tested ACK 050e2ee6f2 on xenial with clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
  fanquake:
    ACK 050e2ee6f2 - I see why we didn't hit this on master. We are installing the `clang-8` packages for the tsan job. However on the 0.20 branch we are still just installing `clang`, which is 3.8.

Tree-SHA512: 8a1d57289dbe9895ab79f81ca87b4fd723426b8d72f3a34bec9553226fba69f6dc19551c1f1d52db6c4b2652164a02ddc60f3187c3e2ad7bcacb0aaca7fa690a
2020-05-15 08:05:45 +08:00
practicalswift
cd34038cbd Switch from Optional<T> to std::optional<T> (C++17). Run clang-format. 2020-05-14 18:52:57 +00:00
practicalswift
fb559c1170 tests: Fill fuzzing coverage gaps for functions in util/translation.h 2020-05-14 18:52:57 +00:00
practicalswift
b74f3d6c45 tests: Fill fuzzing coverage gaps for functions in consensus/validation.h 2020-05-14 18:45:42 +00:00
practicalswift
c0bbf8193d tests: Fill fuzzing coverage gaps for functions in primitives/block.h 2020-05-14 18:45:42 +00:00
Wladimir J. van der Laan
553bb3fc3d Merge #18962: net processing: Only send a getheaders for one block in an INV
746736639e [net processing] Only send a getheaders for one block in an INV (John Newbery)

Pull request description:

  Headers-first is the primary method of announcement on the network. If a node fell back sending blocks by inv, it's probably for a re-org. The final block hash provided should be the highest, so send a getheaders and then fetch the blocks we need to catch up.

  Sending many GETHEADERS messages to the peer would cause them to send a large number of potentially large HEADERS messages with redundant data, which is a waste of bandwidth.

ACKs for top commit:
  sipa:
    utACK 746736639e
  mzumsande:
    utACK 746736639e as per ajtowns' reasoning.
  naumenkogs:
    utACK 7467366
  ajtowns:
    ACK 746736639e
  jonatack:
    ACK 746736639e

Tree-SHA512: 59e243b80d3f0873709dfacb2e4ffba34689aad7de31ec7f69a64e0e3a0756235a0150e4082ff5de823949ba4411ee1aed2344b4749b62e0eb1ea906e41f5ea9
2020-05-14 20:43:45 +02:00
Wladimir J. van der Laan
2d7489be8f Merge #18796: scripts: security-check.py refactors
eacedfb023 scripts: add additional type annotations to security-check.py (fanquake)
83d063e954 scripts: add run_command to security-check.py (fanquake)
13f606b4f9 scripts: remove NONFATAL from security-check.py (fanquake)
061acf62a1 scripts: no-longer check for 32 bit windows in security-check.py (fanquake)

Pull request description:

  * Remove 32-bit Windows checks.
  * Remove NONFATAL checking. Added in #8249, however unused since #13764.
  * Add `run_command` to de-duplicate all of the subprocess calls. Mentioned in #18713.
  * Add additional type annotations.
  * Print stderr when there is an issue running a command.

ACKs for top commit:
  laanwj:
    ACK eacedfb023

Tree-SHA512: 69a7ccfdf346ee202b3e8f940634c5daed1d2b5a5d15ac9800252866ba3284ec66e391a66a0b341f5a4e5e8482fe1b614d4671e8e766112ff059405081184a85
2020-05-14 20:00:06 +02:00
Wladimir J. van der Laan
4dd2e5255a Merge #18946: rpcwallet: Replace boost::optional<T>::emplace with simple assignment of T{}
fa1f840596 rpcwallet: Replace pwallet-> with wallet. (MarcoFalke)
fa182a8794 rpcwallet: Replace boost::optional<T>::emplace with simple assignment of T{} (MarcoFalke)

Pull request description:

  Closes #18943

ACKs for top commit:
  laanwj:
    ACK fa1f840596
  ryanofsky:
    Code review ACK fa1f840596 and thanks for using a standalone commit for the fix
  promag:
    Code review ACK fa1f840596.
  hebasto:
    ACK fa1f840596, tested on Linux Mint 19.3.

Tree-SHA512: 0838485d1f93f737ce5bf12740669dcafeebb78dbc3fa15dbcc511edce64bf024f60f0497a04149a1e799d893d57b0c9ffe442020c1b9cfc3c69db731f50e712
2020-05-14 19:26:17 +02:00
Wladimir J. van der Laan
050e2ee6f2 test: Remove const to work around compiler error on xenial
Fix the following error in travis:

    test/validationinterface_tests.cpp:26:36: error: default initialization of an object of const type 'const BlockValidationState' without a user-provided default constructor

    const BlockValidationState state_dummy;
2020-05-14 18:40:57 +02:00
MarcoFalke
facede18a4 test: Check that invalid witness destinations can not be imported 2020-05-14 09:36:51 -04:00
fanquake
b9c504cbc4 Merge #18742: miner: Avoid stack-use-after-return in validationinterface
7777f2a4bb miner: Avoid stack-use-after-return in validationinterface (MarcoFalke)
fa5ceb25fc test: Remove UninterruptibleSleep from test and replace it by SyncWithValidationInterfaceQueue (MarcoFalke)
fa770ce7fe validationinterface: Rework documentation, Rename pwalletIn to callbacks (MarcoFalke)
fab6d060ce test: Add unregister_validation_interface_race test (MarcoFalke)

Pull request description:

  When a validationinterface has itself unregistered in one thread, but is about to get executed in another thread [1], there is a race:

  * The validationinterface destructing itself
  * The validationinterface getting dereferenced for execution

  [1] 64139803f1/src/validationinterface.cpp (L82-L83)

  This happens in the miner. More generally it happens everywhere where at least one thread is generating notifications and another one is unregistering a validationinterface.

  This issue has been fixed in commit ab31b9d6fe, but the fix has not been applied to the miner.

  Example where this happened in practice: https://travis-ci.org/github/bitcoin/bitcoin/jobs/675322230#L4414

ACKs for top commit:
  promag:
    Code review ACK 7777f2a4bb.
  laanwj:
    Code review ACK 7777f2a4bb

Tree-SHA512: 8087119243c71ba18a823a63515f3730d127162625d8729024278b447af29e2ff206f4840ee3d90bf84f93a2c5ab73b76c7e7044c83aa93b5b51047a166ec3d3
2020-05-14 20:40:55 +08:00
fanquake
eacedfb023 scripts: add additional type annotations to security-check.py 2020-05-14 15:30:52 +08:00
fanquake
83d063e954 scripts: add run_command to security-check.py
Deduplicate all the subprocess code as mentioned in 18713.
2020-05-14 15:29:58 +08:00
fanquake
13f606b4f9 scripts: remove NONFATAL from security-check.py 2020-05-14 14:36:27 +08:00
fanquake
061acf62a1 scripts: no-longer check for 32 bit windows in security-check.py 2020-05-14 14:36:27 +08:00
fanquake
e8a8cff07c build: enforce minimum required Windows version (7)
Instruct the linker to set the major & minor subsystem versions in the PE
header to 6 & 1 (NT 6.1 which corresponds to Windows 7). Similar to
macOS, the binary will now refuse to run on unsupported versions of
Windows.
2020-05-14 09:46:18 +08:00
MarcoFalke
7777f2a4bb miner: Avoid stack-use-after-return in validationinterface
This is achieved by switching to a shared_ptr.

Also, switch the validationinterfaces in the tests to use shared_ptrs
for the same reason.
2020-05-13 19:58:20 -04:00
MarcoFalke
fa5ceb25fc test: Remove UninterruptibleSleep from test and replace it by SyncWithValidationInterfaceQueue
For the purpose of this test the two have the same outcome, but this one
is shorter and avoids a sleep for 0.1 seconds.
2020-05-13 19:58:11 -04:00
MarcoFalke
fa770ce7fe validationinterface: Rework documentation, Rename pwalletIn to callbacks 2020-05-13 19:57:55 -04:00
MarcoFalke
fab6d060ce test: Add unregister_validation_interface_race test
This commit is (intentionally) adding a broken test. The test is broken
because it registering a subscriber object that can go out of scope
while events are still being sent.

To run the broken test and reproduce the bug:
  - Remove comment /** and */
  - ./configure --with-sanitizers=address
  - export ASAN_OPTIONS=detect_leaks=0
  - make
  - while ./src/test/test_bitcoin -t validationinterface_tests/unregister_validation_interface_race --catch_system_errors=no  ; do true; done
2020-05-13 19:57:50 -04:00
Russell Yanofsky
b3f7f375ef refactor: Remove g_rpc_node global
This commit does not change behavior
2020-05-13 16:20:13 -04:00
Wladimir J. van der Laan
04c09553d8 Merge #18887: build: enable -Werror=gnu
a30b0a24e9 build: enable -Werror=gnu (Vasil Dimov)

Pull request description:

  Stop the build if a warning is emitted due to `-Wgnu` and
  `--enable-werror` has been used. As usual - this would help notice such
  a warning that is about to be introduced in new code.

  This is a followup to
  https://github.com/bitcoin/bitcoin/pull/18088 build: ensure we aren't using GNU extensions

ACKs for top commit:
  practicalswift:
    ACK a30b0a24e9
  Empact:
    ACK a30b0a24e9

Tree-SHA512: f81b71cf3ee4db88b6f664c571075e0d30800a604f067f44273f256695a1dea533779db2ac859dd0a4cd8b66289c3e45f4aff1cfadfa160a1c354237167b05e2
2020-05-13 22:20:13 +02:00
Russell Yanofsky
ccb5059ee8 scripted-diff: Remove g_rpc_node references
This commit does not change behavior

-BEGIN VERIFY SCRIPT-
git grep -l g_rpc_node | xargs sed -i 's/g_rpc_node->/node./g'
-END VERIFY SCRIPT-
2020-05-13 16:20:13 -04:00
Russell Yanofsky
6fca33b2ed refactor: Pass NodeContext to RPC and REST methods through util::Ref
This commit does not change behavior
2020-05-13 16:20:13 -04:00
Russell Yanofsky
691c817b34 Add util::Ref class as temporary alternative for c++17 std::any
This commit does not change behavior
2020-05-13 16:20:13 -04:00
MarcoFalke
a9a6d946e4 Merge #18888: test: Remove RPCOverloadWrapper boilerplate
faa26d3744 test: Remove RPCOverloadWrapper boilerplate (MarcoFalke)

Pull request description:

  There are too many wrappers in test_node already, so at least the code that implements the wrappers should be as minimal as possible.

ACKs for top commit:
  laanwj:
    code review ACK faa26d3744

Tree-SHA512: 94e593907de22187524e2445afb3101e40b3b599d4b4015aa8c6ca902d7586ff9daf520828759029d199a3af79e61b96b490a822a5a193ac7bf946beacb11a24
2020-05-13 15:36:10 -04:00
Jonas Schnelli
51825aea7f Merge #18922: gui: Do not translate InitWarning messages in debug.log
78be8d97d3 util: Drop OpOriginal() and OpTranslated() (Hennadii Stepanov)
da16f95c3f gui: Do not translate InitWarning messages in debug.log (Hennadii Stepanov)
4c9b9a4882 util: Enhance Join() (Hennadii Stepanov)
fe05dd0611 util: Enhance bilingual_str (Hennadii Stepanov)

Pull request description:

  This PR forces the `bitcoin-qt` to write `InitWarning()` messages to the `debug.log` file in untranslated form, i.e., in English.

  On master (376294cde6):
  ```
  $ ./src/qt/bitcoin-qt -lang=nl -debug=vladidation -printtoconsole | grep 'vladi'
  Warning: Niet-ondersteunde logcategorie -debug=vladidation.
  2020-05-09T12:39:59Z Warning: Niet-ondersteunde logcategorie -debug=vladidation.
  2020-05-09T12:40:02Z Command-line arg: debug="vladidation"
  ```

  With this PR:
  ```
  $ ./src/qt/bitcoin-qt -lang=nl -debug=vladidation -printtoconsole | grep 'vladi'
  Warning: Unsupported logging category -debug=vladidation.
  2020-05-09T12:42:04Z Warning: Unsupported logging category -debug=vladidation.
  2020-05-09T12:42:35Z Command-line arg: debug="vladidation"
  ```

  ![Screenshot from 2020-05-09 15-42-31](https://user-images.githubusercontent.com/32963518/81474073-c7a50e00-920b-11ea-8775-c41122dacafe.png)

  Related to #16218.

ACKs for top commit:
  laanwj:
    ACK 78be8d97d3
  jonasschnelli:
    utACK 78be8d97d3
  MarcoFalke:
    ACK 78be8d97d3 📢

Tree-SHA512: 48e9ecd23c4dd8ec262e3eb94f8e30944bcc9c6c163245fb837b2e0c484d4d0b4f47f7abc638c14edc27d635d340ba3ee4ba4506b062399e9cf59a1564c98755
2020-05-13 20:30:39 +02:00
Wladimir J. van der Laan
fc895d7700 Merge #18616: refactor: Cleanup clientversion.cpp
c269e618cf Drop unused GIT_COMMIT_DATE macro (Hennadii Stepanov)
8f9f4ba5e2 refactor: Remove duplicated code (Hennadii Stepanov)
35f1189ea7 build: Rename BUILD_* macros and the code self-descriptive (Hennadii Stepanov)
dc1fba9389 scripted-diff: Rename share/genbuild.sh macros to more meaningful ones (Hennadii Stepanov)
1e06bb68be Drop unused CLIENT_VERSION_SUFFIX macro (Hennadii Stepanov)

Pull request description:

  This PR:
  - removes unused macros and duplicated code
  - renames macros in a way, that makes the code self-descriptive.

ACKs for top commit:
  dongcarl:
    Yup! ACK c269e618cf

Tree-SHA512: c469f6269b578ccfae33d960e317eca8efaf27d49638f4c3830948c11b12ef728494d7e18c31e4a410945b7d83af5b246c7b83661b4eca17cf41ee4c4583649b
2020-05-13 20:14:51 +02:00
Wladimir J. van der Laan
5d18c0ae18 Merge #18862: Remove fdelt_chk back-compat code and sanity check
df6bde031b test: remove glibc fdelt sanity check (fanquake)
8bf1540cc2 build: remove fdelt_chk backwards compatibility code (fanquake)

Pull request description:

  ae30d40e50
  The return type of [`fdelt_chk`](https://sourceware.org/git/?p=glibc.git;a=blob;f=debug/fdelt_chk.c;h=f62ce7349707cb68f55831c1c591fd7387a90258;hb=HEAD) changed from `unsigned  long int` to `long int` in glibc 2.16. See [this commit](https://sourceware.org/git/?p=glibc.git;a=commit;h=ceb9e56b3d1f8c1922e0526c2e841373843460e2). Now that we require [glibc >=2.17](https://github.com/bitcoin/bitcoin/pull/17538) we can remove our back-compat code.

  ab7bce584a
  While looking at the above changes, I noticed that our glibc fdelt sanity check doesn't seem to be checking anything. `fdelt_warn()` also isn't something we'd want to actually "trigger" at runtime, as doing so would cause `bitcoind` to abort.

  The comments:
  > // trigger: Call FD_SET to trigger __fdelt_chk. FORTIFY_SOURCE must be defined
  > //   as >0 and optimizations must be set to at least -O2.

  suggest calling FD_SET to check the invocation of `fdelt_chk` (this is [aliased with fdelt_warn in glibc](https://sourceware.org/git/?p=glibc.git;a=blob;f=debug/fdelt_chk.c;h=f62ce7349707cb68f55831c1c591fd7387a90258;hb=HEAD)). However just calling `FD_SET()` will not necessarily cause the compiler to insert a call to `fd_warn()`.

  Whether or not GCC (recent Clang should work, but may use different heuristics) inserts a call to `fdelt_warn()` depends on if the compiler can determine if the value passed in is a compile time constant (using [`__builtin_constant_p`](https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html)) and whether the value is < 0 or >= `FD_SETSIZE`. The glibc implementation is [here](https://sourceware.org/git/?p=glibc.git;a=blob;f=misc/bits/select2.h;h=7e17430ed94dd1679af10afa3d74795f9c97c0e8;hb=HEAD). This means our check should never cause a call to be inserted.

  Compiling master without `--glibc-back-compat` (if you do pass `--glibc-back-compat` the outcome is still the same; however the abort will only happen with >=`FD_SETSIZE` as that is what our [fdelt_warn()](https://github.com/bitcoin/bitcoin/blob/master/src/compat/glibc_compat.cpp#L24) checks for), there are no calls to `fdelt_warn()` inserted by the compiler:
  ```bash
  objdump -dC bitcoind | grep sanity_fdelt
  ...
  0000000000399d20 <sanity_test_fdelt()>:
    399d20:       48 81 ec 98 00 00 00    sub    $0x98,%rsp
    399d27:       b9 10 00 00 00          mov    $0x10,%ecx
    399d2c:       64 48 8b 04 25 28 00    mov    %fs:0x28,%rax
    399d33:       00 00
    399d35:       48 89 84 24 88 00 00    mov    %rax,0x88(%rsp)
    399d3c:       00
    399d3d:       31 c0                   xor    %eax,%eax
    399d3f:       48 89 e7                mov    %rsp,%rdi
    399d42:       fc                      cld
    399d43:       f3 48 ab                rep stos %rax,%es:(%rdi)
    399d46:       48 8b 84 24 88 00 00    mov    0x88(%rsp),%rax
    399d4d:       00
    399d4e:       64 48 33 04 25 28 00    xor    %fs:0x28,%rax
    399d55:       00 00
    399d57:       75 0d                   jne    399d66 <sanity_test_fdelt()+0x46>
    399d59:       b8 01 00 00 00          mov    $0x1,%eax
    399d5e:       48 81 c4 98 00 00 00    add    $0x98,%rsp
    399d65:       c3                      retq
    399d66:       e8 85 df c8 ff          callq  27cf0 <__stack_chk_fail@plt>
    399d6b:       0f 1f 44 00 00          nopl   0x0(%rax,%rax,1)

  ```

  If you modify the sanity test to pass `-1` or `FD_SETSIZE` to `FD_SET`, you'll see calls to `fdelt_warn` inserted, and the runtime behaviour is an abort as expected.

  ```diff
  diff --git a/src/compat/glibc_sanity_fdelt.cpp b/src/compat/glibc_sanity_fdelt.cpp
  index 87140d0c7..16974bfa0 100644
  --- a/src/compat/glibc_sanity_fdelt.cpp
  +++ b/src/compat/glibc_sanity_fdelt.cpp
  @@ -20,7 +20,7 @@ bool sanity_test_fdelt()
   {
       fd_set fds;
       FD_ZERO(&fds);
  -    FD_SET(0, &fds);
  +    FD_SET(FD_SETSIZE, &fds);
       return FD_ISSET(0, &fds);
   }
   #endif
  ```

  ```bash
  0000000000399d20 <sanity_test_fdelt()>:
    399d20:	48 81 ec 98 00 00 00 	sub    $0x98,%rsp
    399d27:	b9 10 00 00 00       	mov    $0x10,%ecx
    399d2c:	64 48 8b 04 25 28 00 	mov    %fs:0x28,%rax
    399d33:	00 00
    399d35:	48 89 84 24 88 00 00 	mov    %rax,0x88(%rsp)
    399d3c:	00
    399d3d:	31 c0                	xor    %eax,%eax
    399d3f:	48 89 e7             	mov    %rsp,%rdi
    399d42:	fc                   	cld
    399d43:	f3 48 ab             	rep stos %rax,%es:(%rdi)
    399d46:	48 c7 c7 ff ff ff ff 	mov    $0xffffffffffffffff,%rdi
    399d4d:	e8 3e ff ff ff       	callq  399c90 <__fdelt_warn>
    399d52:	0f b6 04 24          	movzbl (%rsp),%eax
    399d56:	83 e0 01             	and    $0x1,%eax
    399d59:	48 8b 94 24 88 00 00 	mov    0x88(%rsp),%rdx
    399d60:	00
    399d61:	64 48 33 14 25 28 00 	xor    %fs:0x28,%rdx
    399d68:	00 00
    399d6a:	75 08                	jne    399d74 <sanity_test_fdelt()+0x54>
    399d6c:	48 81 c4 98 00 00 00 	add    $0x98,%rsp
    399d73:	c3                   	retq
    399d74:	e8 77 df c8 ff       	callq  27cf0 <__stack_chk_fail@plt>
    399d79:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)
   ```

   ```bash
   src/bitcoind
  *** buffer overflow detected ***: src/bitcoind terminated
  Aborted
   ```

  I think the test should should be removed and replaced (if possible) with additional checks in security-check.py. I was thinking about adding a version of [this script](https://github.com/fanquake/core-review/blob/master/fortify.py) as part of the output, but that needs more thought. I'll address this in a follow up.

ACKs for top commit:
  laanwj:
    ACK  df6bde031b

Tree-SHA512: d8b3af4f4eb2d6c767ca6e72ece51d0ab9042e1bbdfcbbdb7ad713414df21489ba3217662b531b8bfdac0265d2ce5431abfae6e861b6187d182ff26c6e59b32d
2020-05-13 19:35:25 +02:00
Wladimir J. van der Laan
99c03728c0 Merge #18878: test: Add test for conflicted wallet tx notifications
f963a68051 test: Add test for conflicted wallet tx notifications (Russell Yanofsky)

Pull request description:

  Add test coverage for conflicted wallet transaction notifications so we can improve current behavior and avoid future regressions

  https://github.com/bitcoin/bitcoin/pull/9240 - accidental break
  https://github.com/bitcoin/bitcoin/issues/9479 - bug report
  https://github.com/bitcoin/bitcoin/pull/9371 - fix
  https://github.com/bitcoin/bitcoin/pull/16624 - accidental break
  https://github.com/bitcoin/bitcoin/issues/18325 - bug report
  https://github.com/bitcoin/bitcoin/pull/18600 - potential fix

ACKs for top commit:
  laanwj:
    ACK f963a68051
  jonatack:
    re-ACK f963a68051
  MarcoFalke:
    ACK f963a68051

Tree-SHA512: d3a7952a2d3dc2ff0800ef857575ea4ef9759c0917d58a7fc91e2db0ca3cc3baf0dd0cf9af61683f691e5fefb11afe8120bb5810c7037ed9ecedee385dd4aa07
2020-05-13 15:52:58 +02:00
fanquake
6c647c89db Merge #18738: build: Suppress -Wdeprecated-copy warnings
0c63f80854 build: Suppress -Wdeprecated-copy warnings (Hennadii Stepanov)

Pull request description:

  Tomorrow, on Apr 23 the Ubuntu 20.04 release is expected. It packaged with Qt 5.12 LTS that has a nasty peculiarity to cause modern compilers, including Clang 10.0 and GCC 9.3, to emit spammy `-Wdeprecated-copy` warnings (#15822, #18419).

  This PR suppress such warnings _temporarily_, until the [upstream is fixed](https://codereview.qt-project.org/c/qt/qtbase/+/272258).

  Here are some affected systems (with system packages):
  - Ubuntu 20.04 LTS + Qt 5.12.8 LTS + { Clang 10.0 | GCC 9.3 }
  - Fedora 32 + Qt 5.13.2 + Clang 10.0

  Reference: [QTBUG-75210](https://bugreports.qt.io/browse/QTBUG-75210)

  Also see **fanquake**'s [comment](https://github.com/bitcoin/bitcoin/pull/18738#issuecomment-622956100).

ACKs for top commit:
  MarcoFalke:
    ACK 0c63f80854 seems fine to disable this warning for the 0.21.0 release temporarily and then enable it for 0.22.0, when boost is removed.
  fanquake:
    ACK 0c63f80854 - I think it's ok to suppress these for now, given that `-Wdeprecated-copy` is enabled (via `-Wextra`) in GCC 9 and Clang 10. The Qt output is pretty noisy, and there's a few warnings from Boost as well.

Tree-SHA512: 7064a3272bc9eae00b73a16c421ac58be148f374cbef87320e8f092f52761f6e98166eff60346b70867f8a69a9698a79455dc16b42d92f8fbe7c56519571ac08
2020-05-13 21:17:07 +08:00
Hennadii Stepanov
da73f1513a qt: Fix shutdown when waitfor* cmds are called from RPC console 2020-05-13 16:14:04 +03:00
Hennadii Stepanov
1122817c19 qt: Remove QFont warnings with QPA=minimal 2020-05-13 16:02:38 +03:00
fanquake
a33901cb6d Merge #18814: rpc: Relock wallet only if most recent callback
9f59dde974 rpc: Relock wallet only if most recent callback (João Barbosa)
a2e6db5c4f rpc: Add mutex to guard deadlineTimers (João Barbosa)

Pull request description:

  This PR fixes an early relocking race condition from #18811 where old relock callback runs after new wallet unlock code and nRelockTime update but before rpcRunLater call, causing early relock and incorrect nRelockTime time

  Issue introduced in #18487.
  Fixes #18811.

ACKs for top commit:
  MarcoFalke:
    ACK 9f59dde974
  ryanofsky:
    Code review ACK 9f59dde974. No changes since last review except squashing commits.
  jonatack:
    ACK 9f59dde974

Tree-SHA512: 2f7fc03e5ab6037337f2d82dfad432495cc337c77d07c968ee2355105db6292f24543c03456f5402e0e759577a4327758f9372f7ea29de6d56dc3695fda9b379
2020-05-13 17:36:06 +08:00
Jonas Schnelli
246e878e78 Merge #18894: gui: Fix manual coin control with multiple wallets loaded
a8b5f1b133 gui: Fix manual coin control with multiple wallets loaded (João Barbosa)

Pull request description:

  This PR ensures each loaded wallet has a dedicated coin control in the send view which is manipulated by the coin control dialog.

  This is an alternative to #17457. Two main differences are:
   - scope reduced - no unnecessary changes unrelated to the fix;
   - approach taken - coin control instance now belongs to the send view.

  All problems raised in #17457 reviews no longer apply due to the approach taken - https://github.com/bitcoin/bitcoin/pull/17457#pullrequestreview-319297589 and https://github.com/bitcoin/bitcoin/pull/17457#issuecomment-555920829)

  No change in behavior if only one wallet is loaded.

  Closes #15725.

ACKs for top commit:
  jonasschnelli:
    utACK a8b5f1b133
  ryanofsky:
    Code review ACK a8b5f1b133. Code changes are very straightforward, just replacing global CCoinControl object with SendCoinsDialog member. Not sure if this means coin control settings are reset between payments. It would be good to note in the PR description or release notes if single wallet behavior is affected
  hebasto:
    ACK a8b5f1b133
  Sjors:
    tACK a8b5f1b133

Tree-SHA512: 3ad9c51bab6f28ec0e90efbd6f43fa510c81dafb2eff0b8c3724efcee3e030054a10be013e27cefe35763374c5f6d7af8c02658736964f733d7e38b646b5df65
2020-05-13 10:15:32 +02:00
Jonas Schnelli
8d17f8dc17 Merge #18578: gui: Fix leak in CoinControlDialog::updateView
e8123eae40 gui: Fix itemWalletAddress leak when not tree mode (João Barbosa)

Pull request description:

  Taken from #17457, the first commit is a similar to 88a94f7bb8 but for test binary, and the second commit fixes a leak where `CCoinControlWidgetItem` are unnecessarily created and leaked.

ACKs for top commit:
  jonasschnelli:
    utACK e8123eae40
  hebasto:
    ACK e8123eae40, tested on Linux Mint 19.3.

Tree-SHA512: 8b43cb29de103842ce5f048de51222919540d3212d2873c16731145e856178644041924ad0e9a58c2ff08f209a9b4ac26dc9965289eb719da233c0984f93631e
2020-05-13 10:13:06 +02:00
fanquake
219c55da75 Merge #16710: build: Enable -Wsuggest-override if available
839add193b build: Enable -Wsuggest-override (Hennadii Stepanov)
de5e91c303 refactor: Add BerkeleyDatabaseVersion() function (Hennadii Stepanov)

Pull request description:

  From GCC [docs](https://gcc.gnu.org/onlinedocs/gcc-9.2.0/gcc/Warning-Options.html):
  > `-Wsuggest-override`
  > Warn about overriding virtual functions that are not marked with the override keyword.

  ~This PR is based on #16722 (the first commit).~ See: https://github.com/bitcoin/bitcoin/pull/16722#issuecomment-584111086

ACKs for top commit:
  fanquake:
    ACK 839add193b
  vasild:
    ACK 839add193
  practicalswift:
    ACK 839add193b assuming Travis is happy: patch looks correct

Tree-SHA512: 1e8cc085da30d41536deff9b181962c1882314ab252c2ad958294087ae1e5a0dfa4886bdbe36f21cf6ae71df776a8420f349f007d4b5b49fd79ba98ce308965a
2020-05-13 15:19:05 +08:00
Pieter Wuille
2896c412fa Do not answer GETDATA for to-be-announced tx 2020-05-12 15:33:18 -07:00
Pieter Wuille
2676aeadfa Simplify usage of Span in several places 2020-05-12 14:19:40 -07:00
Pieter Wuille
ab303a16d1 Add Span constructors for arrays and vectors 2020-05-12 14:12:46 -07:00
Pieter Wuille
bb3d38fc06 Make pointer-based Span construction safer
This prevents constructing a Span<A> given two pointers into an array
of B (where B is a subclass of A), at least without explicit cast to
pointers to A.
2020-05-12 14:12:46 -07:00
Pieter Wuille
1f790a1147 Make Span size type unsigned
This matches a change in the C++20 std::span proposal.
2020-05-12 14:12:46 -07:00
John Newbery
746736639e [net processing] Only send a getheaders for one block in an INV
Headers-first is the primary method of announcement on the network. If a
node fell back sending blocks by inv, it's probably for a re-org. The
final block hash provided should be the highest, so send a getheaders
and then fetch the blocks we need to catch up.
2020-05-12 16:29:49 -04:00
Pieter Wuille
f2f32a3dee Push down use of cs_main into FindTxForGetData 2020-05-12 13:17:42 -07:00
Pieter Wuille
c6131bf407 Abstract logic to determine whether to answer tx GETDATA 2020-05-12 13:16:55 -07:00
Hennadii Stepanov
839add193b build: Enable -Wsuggest-override 2020-05-12 18:03:39 +03:00
Russell Yanofsky
e2bab2aa16 multiprocess: add multiprocess travis configuration 2020-05-12 09:47:06 -04:00
fanquake
8da1e43b63 Merge #18910: p2p: add MAX_FEELER_CONNECTIONS constant
e3047edfb6 test: use p2p constants in denial of service tests (fanquake)
25d8264c95 p2p: add MAX_FEELER_CONNECTIONS constant (tryphe)

Pull request description:

  Extracted from #16003.

ACKs for top commit:
  naumenkogs:
    utACK e3047ed

Tree-SHA512: 14fc15292be4db2e825a0331dd189a48713464f622a91c589122c1a7135bcfd37a61e64af1e76d32880ded09c24efd54d3c823467d6c35367a380e0be33bd35f
2020-05-12 21:47:06 +08:00
Russell Yanofsky
603fd6a2e7 depends: add MULTIPROCESS depends option
Builds required packages and passes --enable-multiprocess option to bitcoin build
2020-05-12 09:47:06 -04:00
Russell Yanofsky
5d1377b52b build: multiprocess autotools changes
autoconf and automake changes to support multiprocess gui/node/wallet execution.

This adds a new --enable-multiprocess flag, and build configuration code to
detect libraries needed for multiprocess support. The --enable-multiprocess
flag builds new bitcoin-node and bitcoin-gui executables, which are updated in
https://github.com/bitcoin/bitcoin/pull/10102 to communicate across processes.
But for now they are functionally equivalent to existing bitcoind and
bitcoin-qt executables.
2020-05-12 09:47:06 -04:00
Carl Dong
85f4a4b082 guix: Make V=1 more powerful for debugging
- Print commands in both unexpanded and expanded forms
- Set VERBOSE=1 for CMake
2020-05-12 09:37:31 -04:00
MarcoFalke
e45fb7e0d2 Merge #18877: Serve cfcheckpt requests
23083856a5 [test] Add test for cfcheckpt (Jim Posen)
f9e00bb25a [net processing] Message handling for getcfcheckpt. (Jim Posen)
9ccaaba11e [init] Add -peerblockfilters option (Jim Posen)

Pull request description:

  Serve cfcheckpt messages if basic block filter index is enabled and `-peercfilters` is set.

  `NODE_COMPACT_FILTERS` is not signaled to peers, but functionality can be used for testing and serving pre-configured clients.

ACKs for top commit:
  jonatack:
    Code review re-ACK 23083856a5 the only change since my review @ 967e2b1 is an update required for #16224 that was merged yesterday.
  fjahr:
    re-ACK 23083856a5
  jkczyz:
    re-ACK 23083856a5
  ariard:
    re-Code Review ACK 2308385
  clarkmoody:
    Tested ACK 23083856a
  MarcoFalke:
    re-ACK 23083856a5 🌳
  theStack:
    ACK 23083856a5

Tree-SHA512: 8c751bbd7d1c31a413096462ae025c3d2f3163c7016cbec472a5f5ec267f8dd19a2dfc4d749876d7409c1db546e6fdd16461c6863effcfa0d3e993edcfa92a08
2020-05-12 09:03:07 -04:00
MarcoFalke
4fa3157014 Merge #18929: ci: Pass down LD_LIBRARY_PATH and MAKEJOBS to fuzz test_runner
cbd661122e Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
fa35c34df7 Remove unused ci configs that have been moved elsewhere (MarcoFalke)
3333cb9699 fuzz: Pass down MAKEJOBS to test_runner (MarcoFalke)

Pull request description:

  Just how `MAKEJOBS` is passed down to the functional test `test_runner`, do the same for the fuzz `test_runner`.

  Also includes a commit to remove unused config files, which have been moved elsewhere.

Top commit has no ACKs.

Tree-SHA512: 32557102c9e40599b432aeb004c8427e8fbb07cdf4048050cdc8241d1b029aaad306b1131007eeca8315a4f71c38a7efbb833310e056cd11b835676cd19b8902
2020-05-12 07:37:31 -04:00
MarcoFalke
fab860aed4 fuzz: Stop nodes in process_message* fuzzers 2020-05-12 07:28:12 -04:00
fanquake
e3047edfb6 test: use p2p constants in denial of service tests 2020-05-12 17:30:33 +08:00
tryphe
25d8264c95 p2p: add MAX_FEELER_CONNECTIONS constant 2020-05-12 17:30:33 +08:00
fanquake
0f2fa599ae Merge #18931: net: use CMessageHeader::HEADER_SIZE, add missing include
83da576f44 net: use CMessageHeader::HEADER_SIZE, add missing include (Jon Atack)

Pull request description:

  as suggested 16 months ago by Gleb Naumenko in https://github.com/bitcoin/bitcoin/pull/15197#issuecomment-456181865.

  `static constexpr CMessageHeader::HEADER_SIZE` is already used in this file, `src/net.cpp`, in 2 instances. This commit replaces the remaining 2 integer values in the file with it and adds the explicit include header.

  Co-authored by: Gleb Naumenko <naumenko.gs@gmail.com>

ACKs for top commit:
  naumenkogs:
    utACK 83da576
  practicalswift:
    ACK 83da576f44 -- patch looks correct
  theStack:
    ACK 83da576f44 -- verified that its just magic number elimination refactoring and additionally checked that all tests pass 👍

Tree-SHA512: 5b915483bca4ea162c259865a1b615d73b88a1b1db3f82db05f770d10b8a42494d948f5b21badbcce2d9efa5915b8cbb6af83073867c23d2f152c0d35ac37b96
2020-05-12 17:05:40 +08:00
fanquake
ad1b7b1817 Merge #18957: Add a link from ZMQ doc to ZMQ example in contrib/
d97fac422e Add a link from ZMQ doc to ZMQ example in contrib/ (Damian Mee)

Pull request description:

  No code changes :).  Only a small convenience improvement in zmq doc.

ACKs for top commit:
  fanquake:
    ACK d97fac422e

Tree-SHA512: f05a8a7a77c0a698637fd24ffc94d0d617743b434f46695a56576a53331ede254aeece416baf3f8275ae4dfad85ae6e14d1920aa32af53150847420a176d90fb
2020-05-12 16:42:12 +08:00
fanquake
49d237ce32 Merge #18928: build: don't pass -w when building for Windows
89fea68ffd build: don't pass -w when building for Windows (fanquake)

Pull request description:

  This has been around since the introduction of autotools. However at
  this point I'm not sure we'd ever want to suppress all warnings when
  performing a build, and given that CXX FLAGS will have been overriden
  when cross-compiling for Windows (using depends), this would rarely,
  if-ever be used anyways.

  From https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html:
  > -w
  >
  >     Inhibit all warning messages.

ACKs for top commit:
  hebasto:
    ACK 89fea68ffd

Tree-SHA512: 2b5bdef7fff5c87b28199f5822cab3cdf600c90c01a40db5cd85053eef5dcb5816e2e97ff61a30ff94b4f0c6cb7be22beaef34d82235bdf05ff9da865d40b381
2020-05-12 16:17:15 +08:00
Damian Mee
d97fac422e Add a link from ZMQ doc to ZMQ example in contrib/ 2020-05-12 16:06:28 +08:00
fanquake
7a5767423f Merge #18808: [net processing] Drop unknown types in getdata
9847e205bf [docs] Improve commenting in ProcessGetData() (John Newbery)
2f032556e0 [test] test that an invalid GETDATA doesn't prevent processing of future messages (Amiti Uttarwar)
e257cf71c8 [net processing] ignore unknown INV types in GETDATA messages (Amiti Uttarwar)
047ceac142 [net processing] ignore tx GETDATA from blocks-only peers (Amiti Uttarwar)

Pull request description:

  Currently we'll stall peers that send us an unknown INV type in a GETDATA message. Be a bit more friendly and just drop the invalid request.

  Ditto for blocks-relay-only peers that send us a GETDATA for a transaction.

  There's a test for the first part. The second is difficult to test in the functional test framework since we aren't able to make blocks-relay-only connections.

ACKs for top commit:
  sipa:
    utACK 9847e205bf
  brakmic:
    ACK 9847e205bf
  luke-jr:
    utACK 9847e205bf
  naumenkogs:
    utACK 9847e20
  ajtowns:
    utACK 9847e205bf

Tree-SHA512: 6007f2fd839ffe737727f6fb8e8f083b2d9e05a510748f1d40b8f9be8fdf7b5419a36d8f1039923eec1ba2983e8f6f0436ec5fc196d9f6dcb0657f2ff8ff8e4c
2020-05-12 09:13:48 +08:00
Ferdinando M. Ametrano
8a22fd0114 avoided os-dependant path
The current code fails on windows because of the forward slashes; using os.path.join solves the problem and it is in general more robust
2020-05-11 22:31:02 +02:00
MarcoFalke
6666c828e0 fuzz: Give CNode ownership to ConnmanTestMsg in process_message fuzz harness 2020-05-11 14:36:06 -04:00
Hennadii Stepanov
de5e91c303 refactor: Add BerkeleyDatabaseVersion() function 2020-05-11 20:42:55 +03:00
MarcoFalke
eb2ffbb7c1 Merge #18914: refactor: Apply override specifier consistently
d044e0ec7d refactor: Remove override for final overriders (Hennadii Stepanov)
1551cea2d5 refactor: Use override for non-final overriders (Hennadii Stepanov)

Pull request description:

  Two commits are split out from #16710 to make reviewing [easier](https://github.com/bitcoin/bitcoin/pull/16710#issuecomment-625760894).

  From [C++ FAQ](https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines.html#c128-virtual-functions-should-specify-exactly-one-of-virtual-override-or-final):
  > C.128: Virtual functions should specify exactly one of virtual, override, or final
  > **Reason** Readability. Detection of mistakes. Writing explicit `virtual`, `override`, or `final` is self-documenting and enables the compiler to catch mismatch of types and/or names between base and derived classes. However, writing more than one of these three is both redundant and a potential source of errors.

ACKs for top commit:
  practicalswift:
    ACK d044e0ec7d: consistent use of `override` prevents bugs + patch looks correct + Travis happy
  MarcoFalke:
    ACK d044e0ec7d, based on my understanding that adding `override` or `final` to a function must always be correct, unless it doesn't compile!?
  vasild:
    ACK d044e0ec7

Tree-SHA512: 245fd9b99b8b5cbf8694061f892cb3435f3378c97ebed9f9401ce86d21890211f2234bcc39c9f0f79a4d2806cb31bf8ce41a0f9c2acef4f3a2ac5beca6b077cf
2020-05-11 13:34:07 -04:00
MarcoFalke
f71a3a8829 Merge #18939: doc: add c++17-enable flag to fuzzing instructions
872aa25fa1 doc: add c++17-enable to fuzzing instructions (Martin Zumsande)

Pull request description:

  Update the fuzzing doc because after the merge of #18901, C++17 is required for compilation.

ACKs for top commit:
  practicalswift:
    ACK 872aa25fa1
  MarcoFalke:
    ACK 872aa25fa1

Tree-SHA512: 47e37c033690de1d1fa644bf0cebb256036b32a5784021cc0d3b32e6188822d7f517d4342990dc7ec98de6d650794aeb85483157e69e141d6bd011993e124575
2020-05-11 10:08:04 -04:00
MarcoFalke
fa1f840596 rpcwallet: Replace pwallet-> with wallet.
pwallet is never null everywhere where it is dereferenced, so simply
replace it with a reference, which can not be null by definition.
2020-05-11 09:59:00 -04:00
MarcoFalke
fa182a8794 rpcwallet: Replace boost::optional<T>::emplace with simple assignment of T{}
Optional::emplace() was only added in boost 1.56, see
2e583aaf30

To simply work around https://github.com/bitcoin/bitcoin/issues/18943,
replace it with assignment of T{}
2020-05-11 09:53:49 -04:00
fanquake
ec4d27fa8b Merge #18216: test, build: Enable -Werror=sign-compare
68537275bd build: Enable -Werror=sign-compare (Ben Woosley)
eac6a3080d refactor: Rework asmap Interpret to avoid ptrdiff_t (Ben Woosley)
df37377e30 test: Fix outstanding -Wsign-compare errors (Ben Woosley)

Pull request description:

  Disallowing sign-comparison mismatches can help to prevent the introduction of overflow and interpretation bugs.

  In this case, ~all~ most existing violations are in the tests, and most simply required annotating the literal as unsigned for comparison.

  This was previously prevented by violations in leveldb which were fixed upstream and merged in #17398. You can test that by building this branch against: 22d11187ee vs 75fb37ce68

ACKs for top commit:
  fjahr:
    re-ACK 68537275bd
  practicalswift:
    ACK 68537275bd

Tree-SHA512: 14b5daa38c496fb51548feb30fb4dd179e6f76a8d355f52bc8e2a18f2f9340f0bc98dcf36d8b3d6521045d013891c3103749a4eda88ceef00202a6a0cf93f73c
2020-05-11 12:20:25 +08:00
Martin Zumsande
872aa25fa1 doc: add c++17-enable to fuzzing instructions 2020-05-11 01:18:17 +02:00
Sebastian Falbesoner
51e9393c1f refactor: s/command/msg_type/ in CNetMsgMaker and CSerializedNetMsg 2020-05-11 00:20:57 +02:00
Hennadii Stepanov
78be8d97d3 util: Drop OpOriginal() and OpTranslated()
The current implementation of the Join() allows do not use OpOriginal()
and OpTranslated() unary operators at all.
2020-05-10 21:28:29 +03:00
Hennadii Stepanov
da16f95c3f gui: Do not translate InitWarning messages in debug.log 2020-05-10 18:01:28 +03:00
Hennadii Stepanov
4c9b9a4882 util: Enhance Join() 2020-05-10 18:00:19 +03:00
Russell Yanofsky
cbd661122e Set LD_LIBRARY_PATH consistently in travis tests
Remove inconsistency between functional and unit test environments and make it
possible to substitute bitcoin-qt and bitcoin-node in place of bitcoind in
python tests, or to link bitcoind against shared libraries.
2020-05-10 10:35:17 -04:00
Jon Atack
83da576f44 net: use CMessageHeader::HEADER_SIZE, add missing include
static constexpr CMessageHeader::HEADER_SIZE is already used in this file,
src/net.cpp, in 2 instances. This commit replaces the remaining 2 integer
values with it and adds the explicit include header.

Co-authored by: Gleb Naumenko <naumenko.gs@gmail.com>
2020-05-10 15:58:42 +02:00
MarcoFalke
fa35c34df7 Remove unused ci configs that have been moved elsewhere
They have been moved to https://github.com/MarcoFalke/btc_nightly
running on Cirrus CI https://cirrus-ci.com/build/6249975761862656
2020-05-10 07:51:31 -04:00
MarcoFalke
3333cb9699 fuzz: Pass down MAKEJOBS to test_runner 2020-05-10 07:49:09 -04:00
fanquake
89fea68ffd build: don't pass -w when building for Windows
This has been around since the introduction of autotools. However at
this point I'm not sure we'd every want to suppress all warnings when
performing a build, and given that CXX FLAGS will have been overriden
when cross-compiling for Windows (using depends), this would rarely,
if-ever be used anyways.

From https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html:
-w

    Inhibit all warning messages.
2020-05-10 19:27:15 +08:00
Hennadii Stepanov
fe05dd0611 util: Enhance bilingual_str 2020-05-09 16:43:51 +03:00
MarcoFalke
88d8b4e182 Merge #18901: fuzz: use std::optional for sep_pos_opt variable
420fa0770f fuzz: use std::optional for sep_pos variable (Harris)

Pull request description:

  This PR changes the original `size_t sep_pos` to `std::optional<size_t> sep_post_opt` to remove the warning when compiling fuzz tests.

  ```shell
  warning: variable 'sep_pos' may be uninitialized when used here [-Wconditional-uninitialized]
  ```

  Also, it adds `--enable-c++17` flag to CI fuzz scripts.

ACKs for top commit:
  practicalswift:
    ACK 420fa0770f
  MarcoFalke:
    ACK 420fa07

Tree-SHA512: e967d5d8ab8ee7394b243ff5b28bac72d30bd14774e4a206f8c87474fad22769da76e4ba4e03cbef83b8f60e5293e9d9293b613e2e2e59e187d4e59ae6b874ca
2020-05-09 07:30:37 -04:00
Harris
420fa0770f fuzz: use std::optional for sep_pos variable
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-05-09 11:09:52 +02:00
Ben Woosley
68537275bd build: Enable -Werror=sign-compare
Explicitly add -Wsign-compare as well - not required for all compilers, as GCC activates it
under -Wall, but may impact clang, etc.
2020-05-09 00:20:09 -07:00
Ben Woosley
eac6a3080d refactor: Rework asmap Interpret to avoid ptrdiff_t 2020-05-09 00:20:00 -07:00
MarcoFalke
376294cde6 Merge #18866: test: Fix verack race to avoid intermittent test failures
fae153b409 test: Fix verack race to avoid intermittent test failures (MarcoFalke)

Pull request description:

  Fixes #18832

ACKs for top commit:
  laanwj:
    ACK fae153b409

Tree-SHA512: 071de8c8e2b2787c9433c7460e18b9a54beaf471a52ce848c5ac7263fc2a40f5b976d4f558ecc494fd0fa07284b7c98d29267cade58f80ab74fe9a7d18d94298
2020-05-08 19:31:17 -04:00
MarcoFalke
b55866969e Merge #18917: fuzz: fix vector size problem in system fuzzer
095bc9a106 fuzz: fix vector size problem in system fuzzer (Harris)

Pull request description:

  This PR fixes a problem with vector resizing in system fuzzer (*case 7* there). Originally, this problem was discussed in PR https://github.com/bitcoin/bitcoin/pull/18908

ACKs for top commit:
  MarcoFalke:
    ACK 095bc9a106
  practicalswift:
    ACK 095bc9a106
  brakmic:
    > ACK [095bc9a](095bc9a106)

Tree-SHA512: 73e6004ee51d68a34b49c79d1329a8c4865c21da888801c0fcc7f1bcacb510bf371bb61675eda83e53d08e0f24712e671369719523b0ced0eb2a22607bfa1d3d
2020-05-08 18:11:43 -04:00
Jim Posen
23083856a5 [test] Add test for cfcheckpt 2020-05-08 16:36:19 -04:00
Jim Posen
f9e00bb25a [net processing] Message handling for getcfcheckpt.
If -peerblockfilters is configured, handle requests for cfcheckpt.
2020-05-08 16:36:19 -04:00
Jim Posen
9ccaaba11e [init] Add -peerblockfilters option
When a node is configured with --blockfilterindex=basic and
-peerblockfilters it can serve compact block filters to its peers.

This commit adds the configuration option handling. Future commits
add compact block serving and service bits signaling.
2020-05-08 16:36:18 -04:00
Harris
095bc9a106 fuzz: fix vector size problem in system fuzzer
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2020-05-08 20:21:48 +02:00
Ben Woosley
df37377e30 test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
MarcoFalke
5b24f6084e Merge #16224: gui: Bilingual GUI error messages
18bd83b1fe util: Cleanup translation.h (Hennadii Stepanov)
e95e658b8e doc: Do not translate technical or extremely rare errors (Hennadii Stepanov)
7e923d47ba Make InitError bilingual (Hennadii Stepanov)
917ca93553 Make ThreadSafe{MessageBox|Question} bilingual (Hennadii Stepanov)
23b9fa2e5e gui: Add detailed text to BitcoinGUI::message (Hennadii Stepanov)

Pull request description:

  This is an alternative to #15340 (it works with the `Chain` interface; see: https://github.com/bitcoin/bitcoin/pull/15340#issuecomment-502674004).
  Refs:
  - #16218 (partial fix)
  - https://github.com/bitcoin/bitcoin/pull/15894#issuecomment-487947077

  This PR:
  - makes GUI error messages bilingual: user's native language + untranslated (i.e. English)
  - insures that only untranslated messages are written to the debug log file and to `stderr` (that is not the case on master).

  If a translated string is unavailable only an English string appears to a user.

  Here are some **examples** (updated):

  ![Screenshot from 2020-04-24 17-08-37](https://user-images.githubusercontent.com/32963518/80222043-e2458780-864e-11ea-83fc-197b7121dba5.png)

  ![Screenshot from 2020-04-24 17-12-17](https://user-images.githubusercontent.com/32963518/80222051-e5407800-864e-11ea-92f7-dfef1144becd.png)

  * `qt5ct: using qt5ct plugin` message is my local environment specific; please ignore it.

  ---

  Note for reviewers: `InitWarning()` is out of this PR scope.

ACKs for top commit:
  Sjors:
    re-tACK 18bd83b1fe
  MarcoFalke:
    ACK 18bd83b1fe 🐦

Tree-SHA512: 3cc8ec44f84403e54b57d11714c86b0855ed90eb794b5472e432005073354b9e3f7b4e8e7bf347a4c21be47299dbc7170f2d0c4b80e308205ff09596e55a4f96
2020-05-08 12:17:55 -04:00
MarcoFalke
3930014abc Merge #18864: Add v0.16.3 backwards compatibility test, bump v0.19.0.1 to v0.19.1
d135c29476 [ci] make list of previous releases to download a setting (Sjors Provoost)
9c246b873c [test] backwards compatibility: bump v0.19.0.1 to v0.19.1 (Sjors Provoost)
89a28e02fa [test] add v0.16.3 backwards compatibility test (Sjors Provoost)

Pull request description:

  Thanks to #18774's `adjust_bitcoin_conf_for_pre_17` we can now test backwards compatibility for v0.16.3, both for sync and loading a recent wallet.

  This PR bumps v0.19.0.1 to v0.19.1.

  I also made the version list consistent for the `contrib/devtools/previous_release.sh` instruction, between both tests.

ACKs for top commit:
  MarcoFalke:
    ACK d135c29476

Tree-SHA512: 5ff137a7a934237fa220f1c2807ce9abeeb75929266558bf3e4045bec7dfcd0a8747fa74d700065c568330b18badf58c60c308eb13d1eed444d4bbfe6decc48b
2020-05-08 11:23:40 -04:00
Sjors Provoost
d135c29476 [ci] make list of previous releases to download a setting
Co-Authored-By: MarcoFalke <falke.marco@gmail.com>
2020-05-08 16:13:29 +02:00
Hennadii Stepanov
d044e0ec7d refactor: Remove override for final overriders 2020-05-08 10:37:05 +03:00
Hennadii Stepanov
1551cea2d5 refactor: Use override for non-final overriders 2020-05-08 10:36:58 +03:00
Rod Vagg
34645c4dd0 Test txinwitness is accessible on coinbase vin 2020-05-08 12:19:34 +10:00
Rod Vagg
3e4421070a Expose txinwitness for coinbase in JSON form
txinwitness is used as the witness commitment nonce so is necessary if
reconstructing block data from RPC data.
2020-05-08 12:19:26 +10:00
Danny Lee
34e641a564 test: Remove unnecessary disconnect_nodes call in rpc_psbt.py 2020-05-07 10:32:01 -07:00
Danny Lee
e6e7abd51a test: remove redundant two-way disconnect_nodes calls 2020-05-07 10:32:01 -07:00
Danny Lee
a9bd1f9adf test: warn if nodes not connected before disconnect_nodes 2020-05-07 10:32:01 -07:00
MarcoFalke
f54753293f Merge #18905: travis: Remove s390x
8c705ff129 travis: Remove s390x (MarcoFalke)

Pull request description:

  Fixes #18868
  Fixes #18016

Top commit has no ACKs.

Tree-SHA512: 1007b761c7e01dd2b75aa34e92e01a92a84a100c0a3a53863c755d93b2438a74601e3f2083919b8a9cfc92fb104d0c8415fad3f6c9504c76b02ad2e9712660c0
2020-05-07 10:26:59 -04:00
MarcoFalke
8c705ff129 travis: Remove s390x 2020-05-07 09:58:57 -04:00
fanquake
56611b0e24 Merge #18743: depends: Add --sysroot option to mac os native compile flags
1e94a2bcbc depends: Add --sysroot option to mac os native compile flags (Russell Yanofsky)

Pull request description:

  Catalina SDK clang stopped automatically searching the SDK include paths when invoked without `--sysroot`:

  - https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-594600985
  - https://github.com/Homebrew/homebrew-core/issues/45061

  This hasn't been a problem for current native depends packages because are passing their own `--sysroot` values, and hasn't been a problem for current host packages because they use `darwin_` commands instead of `build_darwin_` commands.  But the current `build_darwin_CC` and `build_darwin_CXX` commands are still unnecessarily fragile, and incompatible with new native depends packages added in https://github.com/bitcoin/bitcoin/pull/18677.

  Cory Fields (theuni) suggested in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-595393546 switching compiler from SDK clang to native clang (from $PATH) to avoid this problem. This is easy and makes a certain amount of sense for building native packages, as opposed to host packages. But Michael (fanquake) pointed out in https://github.com/bitcoin/bitcoin/pull/18677#discussion_r409934309 that it would be inconsistent to switch to non-SDK compilers while still using other SDK tools like `ranlib` and `install_name_tool`. So simplest, minimal fix seems to be just adding the missing `--sysroot` option.

ACKs for top commit:
  ryanofsky:
    > ACK [1e94a2b](1e94a2bcbc) - I think this change is ok, and I prefer it to the previous patch.
  fanquake:
    ACK 1e94a2bcbc - I think this change is ok, and I prefer it to the previous patch. Thanks for the summary in the PR description. I played around with Xcode and the CLT; I think previously I didn't fully grok the slight differences between the two.

Tree-SHA512: 4d4bbb7f49acb76d934a872a15b4e14f36290b508cb9e728815f959767ec174bcfb6d2ca7dcd995cc550d86980d64d4247ea5ecfca2301f0953006e50744fdb4
2020-05-07 20:59:57 +08:00
Sebastian Falbesoner
4a614ff88a test: explicit imports from test_framework.messages in p2p_invalid_messages.py 2020-05-07 14:34:55 +02:00
Sebastian Falbesoner
b35e1d2471 test: add inventory type constant MSG_CMPCT_BLOCK 2020-05-07 14:34:55 +02:00
Sebastian Falbesoner
eeaaa58d2c test: replace inv type magic numbers by constants 2020-05-07 14:34:55 +02:00
MarcoFalke
c1cd2b5a97 Merge #18899: travis: Remove valgrind
fa082d0a57 travis: Remove valgrind (MarcoFalke)

Pull request description:

  When the valgrind run was added, it took 2 hours. Travis kindly raised the timeout limit to the maximum possible of 3 hours.

  Today, a full build of Bitcoin Core with all tests takes more than three hours. Thus, it is impossible to run all tests on travis.

  Moreover, the feedback loop for developers that create a pull request takes at least 2 hours, but in some cases (when the travis queue is full) until the next day. This is unacceptable.

  Fix both issues by removing the build from travis.

  Please note that the `ci/test/` configurations are *not* removed. They will stay in the repo and can be executed anywhere (just not on travis).

ACKs for top commit:
  jamesob:
    ACK fa082d0a57
  jnewbery:
    utACK fa082d0a57

Tree-SHA512: 9acaa0e2d3926014fadb7dd2e86c4e01df382e9399f6ae99f989fa609da66a77bdd1b75d6ff42d2686f38f730b8564e6dc722aa597a473290c9d30c2abe7ef0f
2020-05-07 07:56:33 -04:00
fanquake
df6bde031b test: remove glibc fdelt sanity check
As is, this sanity check doesn't seem to be testing fdelt_chk, because
passing a value of "0" to FD_SET wont cause the compiler to insert any
calls to fdelt_chk().

The documentation is a little misleading. If we actually triggered fdelt_chk
at runtime, bitcoind would abort. I think this check would be better replaced
(if possible) by additional checks in security-check.py.

The compiler may insert a call to fdelt_warn() (aliased with fdelt_chk
in glibc) at compile time if it can determine that an invalid value is
being passed to FD_SET.

These checks are essentially; value < 0 or value >= FD_SETSIZE along
with a check for wether the value is a compile time constant.

If the compiler can determine an invalid value is being passed, a call
to fdelt_warn will be inserted. Passing 0 should never cause a call to
be inserted.

You can check this after compiling:
```bash
objdump -dC bitcoind | grep sanity_fdelt
...
0000000000399d20 <sanity_test_fdelt()>:
  399d20:	48 81 ec 98 00 00 00 	sub    $0x98,%rsp
  399d27:	b9 10 00 00 00       	mov    $0x10,%ecx
  399d2c:	64 48 8b 04 25 28 00 	mov    %fs:0x28,%rax
  399d33:	00 00
  399d35:	48 89 84 24 88 00 00 	mov    %rax,0x88(%rsp)
  399d3c:	00
  399d3d:	31 c0                	xor    %eax,%eax
  399d3f:	48 89 e7             	mov    %rsp,%rdi
  399d42:	fc                   	cld
  399d43:	f3 48 ab             	rep stos %rax,%es:(%rdi)
  399d46:	48 8b 84 24 88 00 00 	mov    0x88(%rsp),%rax
  399d4d:	00
  399d4e:	64 48 33 04 25 28 00 	xor    %fs:0x28,%rax
  399d55:	00 00
  399d57:	75 0d                	jne    399d66 <sanity_test_fdelt()+0x46>
  399d59:	b8 01 00 00 00       	mov    $0x1,%eax
  399d5e:	48 81 c4 98 00 00 00 	add    $0x98,%rsp
  399d65:	c3                   	retq
  399d66:	e8 85 df c8 ff       	callq  27cf0 <__stack_chk_fail@plt>
  399d6b:	0f 1f 44 00 00       	nopl   0x0(%rax,%rax,1)

```

To test, you could modify this test to pass -1 to FD_SET, and check
that a call to fdelt_warn() is inserted, and that running bitcoind
fails. i.e:

```bash
0000000000399d20 <sanity_test_fdelt()>:
  399d20:	48 81 ec 98 00 00 00 	sub    $0x98,%rsp
  399d27:	b9 10 00 00 00       	mov    $0x10,%ecx
  399d2c:	64 48 8b 04 25 28 00 	mov    %fs:0x28,%rax
  399d33:	00 00
  399d35:	48 89 84 24 88 00 00 	mov    %rax,0x88(%rsp)
  399d3c:	00
  399d3d:	31 c0                	xor    %eax,%eax
  399d3f:	48 89 e7             	mov    %rsp,%rdi
  399d42:	fc                   	cld
  399d43:	f3 48 ab             	rep stos %rax,%es:(%rdi)
  399d46:	48 c7 c7 ff ff ff ff 	mov    $0xffffffffffffffff,%rdi
  399d4d:	e8 3e ff ff ff       	callq  399c90 <__fdelt_warn>
  399d52:	0f b6 04 24          	movzbl (%rsp),%eax
  399d56:	83 e0 01             	and    $0x1,%eax
  399d59:	48 8b 94 24 88 00 00 	mov    0x88(%rsp),%rdx
  399d60:	00
  399d61:	64 48 33 14 25 28 00 	xor    %fs:0x28,%rdx
  399d68:	00 00
  399d6a:	75 08                	jne    399d74 <sanity_test_fdelt()+0x54>
  399d6c:	48 81 c4 98 00 00 00 	add    $0x98,%rsp
  399d73:	c3                   	retq
  399d74:	e8 77 df c8 ff       	callq  27cf0 <__stack_chk_fail@plt>
  399d79:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)

```

```bash
./src/bitcoind
*** buffer overflow detected ***: src/bitcoind terminated
Aborted
```
2020-05-07 15:45:09 +08:00
fanquake
8bf1540cc2 build: remove fdelt_chk backwards compatibility code
Now that we require glibc 2.17 or later, we no longer need to check for
different return types in fdelt_chk. It was changed from unsigned long
int to long int in glibc 2.16 . See this commit:
https://sourceware.org/git/?p=glibc.git;a=commit;h=ceb9e56b3d1f8c1922e0526c2e841373843460e2
and related issue:
https://sourceware.org/bugzilla/show_bug.cgi?id=14210.
2020-05-07 15:44:56 +08:00
fanquake
3b1e289248 Merge #18535: build: remove -Qunused-arguments workaround for clang + ccache
a029805f57 build: remove -Qunused-arguments workaround for clang + ccache (fanquake)

Pull request description:

  This was added in 386efb7695 to address spammy Clang warnings when building with ccache.

  The issue was addressed in [ccache 3.2](https://bugzilla.samba.org/show_bug.cgi?id=8118), and from a look at most major distros, it's only Debian Jessie that has a version of ccache older than that ([3.1](https://packages.debian.org/jessie/ccache)).

  Therefore I think it's acceptable to drop this workaround, and re-enable warnings for unused driver arguments (when compiling using Clang and ccache).

ACKs for top commit:
  hebasto:
    ACK a029805f57.
  vasild:
    utACK a029805f57

Tree-SHA512: f887b9bd12f9c1c8d209943b86e8dafe33cfd1572912f2cafabe08ffe403973e48f0f7289280a8c6db9263c57aad43fbd4bb72f42db762eb090f3b1ef0538f43
2020-05-07 15:41:59 +08:00
Anthony Fieroni
9c59f9c285 Fix ZapSelectTx to sync wallet spends
Signed-off-by: Anthony Fieroni <bvbfan@abv.bg>
2020-05-07 08:40:10 +03:00
Hennadii Stepanov
1e9bfd4926 qt: Reset toolbar after all wallets are closed 2020-05-07 08:15:46 +03:00
Hennadii Stepanov
a9d28afe23 qt: Display warnings as rich text 2020-05-07 07:41:01 +03:00
João Barbosa
9f59dde974 rpc: Relock wallet only if most recent callback 2020-05-07 01:42:07 +01:00
MarcoFalke
fa082d0a57 travis: Remove valgrind 2020-05-06 16:10:46 -04:00
Wladimir J. van der Laan
7bcc42b403 Merge #18873: test: Fix intermittent sync_blocks failures
fa3f9a0566 test: Fix intermittent sync_blocks failures (MarcoFalke)

Pull request description:

  Fixes #18872
  Fixes #18737
  Fixes #18801

  See docstring for motivation and description

ACKs for top commit:
  laanwj:
    Code review ACK fa3f9a0566

Tree-SHA512: acd52d386a6849f7ff1cb1a51a439dc3c76e0e3a4dd8d22030df0ebb09b44497c61c56331ff65724b695d82d86b0ebb24608f9e637008e5dacb7676b0c448889
2020-05-06 15:30:31 +02:00
Wladimir J. van der Laan
c6b15ec0ee Merge #17874: build: make linker checks more robust
03da4c7781 build: make linker checks more robust (Cory Fields)

Pull request description:

  Check for a flag to turn linker warnings into errors. When flags are passed to
  linkers via the compiler driver using a -Wl,-foo flag, linker warnings may be
  swallowed rather than bubbling up.

  This is one of [Corys commits](b9acd3d33e) that I've modified to also add `-Wl,-fatal_warnings`
  for darwin.

ACKs for top commit:
  vasild:
    re-ACK 03da4c778

Tree-SHA512: 212031d619ed88e52aaae30cf3b711681d72c4d670884406403605d1d86c784c84cb07e2e0d6c30926e659db8f14f8dabd5af3de5291637f8080d6dfee358248
2020-05-06 15:09:55 +02:00
Wladimir J. van der Laan
f763283b65 Merge #18512: Improve asmap checks and add sanity check
748977690e Add asmap_direct fuzzer that tests Interpreter directly (Pieter Wuille)
7cf97fda15 Make asmap Interpreter errors fatal and fuzz test it (Pieter Wuille)
c81aefc537 Add additional effiency checks to sanity checker (Pieter Wuille)
fffd8dca2d Add asmap sanity checker (Pieter Wuille)
5feefbe6e7 Improve asmap Interpret checks and document failures (Pieter Wuille)
2b3dbfa5a6 Deal with decoding failures explicitly in asmap Interpret (Pieter Wuille)
1479007a33 Introduce Instruction enum in asmap (Pieter Wuille)

Pull request description:

  This improves/documents the failure cases inside the asmap interpreter. None of the changes are bug fixes (they only change behavior for corrupted asmap files), but they may make things easier to follow.

  In a second step, a sanity checker is added that effectively executes every potential code path through the asmap file, checking the same failure cases as the interpreter, and more. It takes around 30 ms to run for me for a 1.2 MB asmap file.

  I've verified that this accepts asmap files constructed by https://github.com/sipa/asmap/blob/master/buildmap.py with a large dataset, and no longer accepts it with 1 bit changed in it.

ACKs for top commit:
  practicalswift:
    ACK 748977690e modulo feedback below.
  jonatack:
    ACK 748977690e code review, regular build/tests/ran bitcoin with -asmap, fuzz build/ran both fuzzers overnight.
  fjahr:
    ACK 748977690e

Tree-SHA512: d876df3859735795c857c83e7155ba6851ce839bdfa10c18ce2698022cc493ce024b5578c1828e2a94bcdf2552c2f46c392a251ed086691b41959e62a6970821
2020-05-06 14:59:28 +02:00
Wladimir J. van der Laan
88b2652fad Merge #18853: wallet: Fix typo in assert that is compile-time true
fa47cf9d95 wallet: Fix typo in assert that is compile-time true (MarcoFalke)

Pull request description:

  Commit 92bcd70808 presumably added a check that a `dest` of type `CNoDestination` implies an empty `scriptChange`.

  However, it accidentally checked for `boost::variant::empty`, which always returns false: https://www.boost.org/doc/libs/1_72_0/doc/html/boost/variant.html#id-1_3_46_5_4_1_1_16_2-bb

ACKs for top commit:
  Sjors:
    utACK fa47cf9d95

Tree-SHA512: 9626b1e2947039853703932a362c2ee204e002d3344856eb93eef0e0f833401336f2dfa80fd43b83c8ec6eac624e6302aee771fb67aec436ba6483be02b8d615
2020-05-06 14:19:41 +02:00
Wladimir J. van der Laan
6621be5351 Merge #18843: build: warn on potentially uninitialized reads
71f183a49b build: warn on potentially uninitialized reads (Vasil Dimov)

Pull request description:

  * Enable `conditional-uninitialized` warning class to show potentially uninitialized
  reads.

  * Fix the sole such warning in Bitcoin Core in `GetRdRand()`: `r1` would be
  set to `0` on `rdrand` failure, so initializing it to `0` is a non-functional
  change.

ACKs for top commit:
  practicalswift:
    ACK 71f183a49b
  laanwj:
    ACK 71f183a49b

Tree-SHA512: 2c1d8caacd86424b16a9d92e5df19e0bedb51ae111eecad7e3bfa46447bc88e5fff1f32dacf6c4a28257ebb3d87e79f80f074ce2c523ce08b1a0c0a67ab44204
2020-05-06 13:49:49 +02:00
Wladimir J. van der Laan
dd3310bbb8 Merge #18854: doc: Fix typo in Coin doxygen comment
fa09110ebb doc: Fix typo in Coin doxygen comment (MarcoFalke)

Pull request description:

  `CTxOutCompressor` has been renamed in commit 4de934b9b5, so rename it in the docs as well.

ACKs for top commit:
  laanwj:
    ACK fa09110ebb
  hebasto:
    ACK fa09110ebb

Tree-SHA512: e16a21ac3112a67ee7d5ffabb3f47103aed8f91fdebf1bf96311cd0b7bdb9b7323ed826bfa95517386d4128ff0ae2c7c13bad047a7c5a0cc2458be7a43119157
2020-05-06 13:15:28 +02:00
Cory Fields
03da4c7781 build: make linker checks more robust
Check for a flag to turn linker warnings into errors. When flags are passed to
linkers via the compiler driver using a -Wl,-foo flag, linker warnings may be
swallowed rather than bubbling up.

Co-authored-by: fanquake <fanquake@gmail.com>
2020-05-06 17:39:07 +08:00
fanquake
551dc7f664 Merge #18806: net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix
1ad8ea2b73 net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix (Sebastian Falbesoner)

Pull request description:

  The BIP37 bloom filter class `CBloomFilter` contains two flags `isEmpty`/`isFull` together with an update method with the purpose to, according to the comments, "avoid wasting cpu", i.e. the mechanism should serve as an optimization for the trivial cases of empty (all bits zero) or full (all bits one) filters.
  However, the real reason of adding those flags (introduced with commit 37c6389c5a by gmaxwell) was a _covert fix_ of [CVE-2013-5700](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-5700), a vulnerability that allowed a divide-by-zero remote node crash.
  According to gmaxwell himself (https://github.com/bitcoin/bitcoin/pull/9060#issuecomment-257749165):
  > the IsEmpty/IsFull optimizations were largely a pretextual optimization intended to make unexploitable a remote crash vulnerability (integer division by zero) that existed in the original bloom filtering code without disclosing it. I'm doubtful that they are all that useful. :)

  For more information on how to trigger this crash, see PR https://github.com/bitcoin/bitcoin/pull/18515 which contains a detailled description and a regression test. It has also been discussed on a [recent PR club meeting on fuzzing](https://bitcoincore.reviews/18521.html).

  The covert fix code already led to issues and PR based on the wrong assumption that the flags are there for optimization reasons (see #16886 and #16922). This PR gets rid of the flags and the update method and just focuses on the CVE fix itself, i.e. it can be seen as a revert of the covert fix commit modulo the actual fix.

ACKs for top commit:
  meshcollider:
    utACK 1ad8ea2b73
  laanwj:
    Concept and code review ACK 1ad8ea2b73
  jkczyz:
    ACK 1ad8ea2b73
  MarcoFalke:
    ACK 1ad8ea2b73
  fjahr:
    Code review ACK 1ad8ea2b73

Tree-SHA512: 29f7ff9faece0285e11e16c024851f5bcb772dec64118ccc3f9067ec256267ec8e1b1e3105c7de2a72fd122c3b085e8fc840ab8f4e49813f1cc7a444df1867f7
2020-05-06 15:40:06 +08:00
Anthony Towns
c3cf2f5501 rpc/blockchain.cpp: Remove g_utxosetscan mutex that is only protecting a single atomic variable 2020-05-06 15:28:49 +10:00
fanquake
d96fdc2a39 Merge #18741: guix: Make source tarball using git-archive
bfe1ba2f5b rel-builds: Specify core.abbrev for git-rev-parse (Carl Dong)
27e63e01cc build: Accomodate makensis v2.x (Carl Dong)
1f2c39a30e guix: Remove logical cores requirement (Carl Dong)
a4f6ffa71e lint: Also enable source statements for non-gitian (Carl Dong)
d256f91cb1 rel-builds: Directly deploy win installer to OUTDIR (Carl Dong)
fa791da02f nsis: Specify OutFile path only once (Carl Dong)
14701604d0 guix: Expose GIT_COMMON_DIR in container as readonly (Carl Dong)
f5a6ac4f48 guix: Make source tarball using git-archive (Carl Dong)
395c1137f6 gitian: Limit sourced script to just assignments (Carl Dong)

Pull request description:

  Based on: #18556
  Related: https://github.com/bitcoin/bitcoin/pull/17595#discussion_r399728721

ACKs for top commit:
  fanquake:
    ACK bfe1ba2f5b - I agree with Carl, and am going to merge this. I'd like for Linux Guix builds to be working again, and we can rebase #18818.

Tree-SHA512: c87ada7e3de17ca0b692a91029b86573442ded5780fc081c214773f6b374a0cdbeaf6f6898c36669c2e247ee32aa7f82defb1180f8decac52c65f0c140f18674
2020-05-06 13:13:36 +08:00
Samuel Dobson
60091d20f9 Merge #9381: Remove CWalletTx merging logic from AddToWallet
28b112e9bd Get rid of BindWallet (Russell Yanofsky)
d002f9d15d Disable CWalletTx copy constructor (Russell Yanofsky)
65b9d8f8dd Avoid copying CWalletTx in LoadToWallet (Russell Yanofsky)
bd2fbc7cdb Get rid of unneeded CWalletTx::Init parameter (Russell Yanofsky)
2b9cba2065 Remove CWalletTx merging logic from AddToWallet (Russell Yanofsky)

Pull request description:

  This is a pure refactoring, no behavior is changing.

  Instead of AddToWallet taking a temporary CWalletTx object and then potentially merging it with a pre-existing CWalletTx, have it take a callback so callers can update the pre-existing CWalletTx directly.

  This makes AddToWallet simpler because now it is only has to be concerned with saving CWalletTx objects and not merging them.

  This makes AddToWallet calls clearer because they can now make direct updates to CWalletTx entries without having to make temporary objects and then worry about how they will be merged.

  Motivation for this change came from the bumpfee PR #8456 where we wanted to be able to call AddToWallet to make a simple update to an existing transaction, but were reluctant to, because the existing CWalletTx merging logic did not apply and seemed dangerous try to update as part of that PR. After this refactoring, the bumpfee PR could call AddToWallet safely instead of implementing a duplicate AddToWallet function.

  This also allows getting rid of the CWalletTx copy constructor to prevent unintentional copying.

ACKs for top commit:
  MarcoFalke:
    Anyway, re-ACK 28b112e9bd

Tree-SHA512: 528dd088714472a237500b200f4433db850bdb7fc29c5e5d81cae48072061dfb967f7c37edd90b33f24901239f9be982988547c1f8c80abc25fb243fbf7330ef
2020-05-06 11:36:32 +12:00
João Barbosa
a8b5f1b133 gui: Fix manual coin control with multiple wallets loaded 2020-05-05 23:56:21 +01:00
MarcoFalke
fa3f9a0566 test: Fix intermittent sync_blocks failures 2020-05-05 14:37:46 -04:00
MarcoFalke
faa26d3744 test: Remove RPCOverloadWrapper boilerplate 2020-05-05 11:30:36 -04:00
MarcoFalke
fbd522721c Merge #18885: contrib: Move optimize-pngs.py script to the maintainer repo
fa13090d20 contrib: Remove optimize-pngs.py script, which lives in the maintainer repo (MarcoFalke)

Pull request description:

  Moved to https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py

  Bitcoin Core should focus on the full node implementation, not on scripts to compress png images.

  This script is only used when new PNG files are added to the repo. This happens about once every two years. So fetching the script from the other repo should not be a burden, but removing it from this repo is going to cut down on the meta files we need to maintain in the main repo.

ACKs for top commit:
  practicalswift:
    ACK fa13090d20 -- `+0 lines, -82 lines` :)
  promag:
    ACK fa13090d20.
  hebasto:
    ACK fa13090d20, verified that script is already [moved](https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/56).

Tree-SHA512: 37d111adae769bcddc6ae88041032d5a2b8b228fec67f555c8333c38de3992f5138b30bea868d7d6d6b7f966a47133e5853134373b149ab23cba3b8b560ecb31
2020-05-05 10:40:39 -04:00
MarcoFalke
52ce396b2a Merge #18879: valgrind: remove outdated suppressions
d7120f7f78 valgrind : remove duplicate BCLog::Logger suppression (fanquake)
708e3c7e85 valgrind: remove rest_blockhash_by_height suppression (fanquake)

Pull request description:

  708e3c7e85: `Suppress rest_blockhash_by_height` should no longer be needed after #18785.

  d7120f7f78 : Removes a duplicate `Suppress BCLog::Logger::StartLogging()` suppression that was added in #17770.

ACKs for top commit:
  MarcoFalke:
    ACK d7120f7f78
  practicalswift:
    ACK d7120f7f78 -- patch looks correct and valgrind Travis job is happy

Tree-SHA512: 45f5b9fa64bf83cada3cd9ad33c245f660376d5b29f51a2531d83133940090df945f5ef26c5847d6ec024ffab9528d55573c5cf9ca5e73795f9abfc971b3d29b
2020-05-05 10:06:35 -04:00
Vasil Dimov
a30b0a24e9 build: enable -Werror=gnu
Stop the build if a warning is emitted due to `-Wgnu` and
`--enable-werror` has been used. As usual - this would help notice such
a warning that is about to be introduced in new code.

This is a followup to
https://github.com/bitcoin/bitcoin/pull/18088
build: ensure we aren't using GNU extensions
2020-05-05 14:47:59 +02:00
MarcoFalke
fa13090d20 contrib: Remove optimize-pngs.py script, which lives in the maintainer repo
https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/master/optimize-pngs.py
2020-05-05 07:25:58 -04:00
Karl-Johan Alm
44cc75f80e wallet: error if an explicit fee rate was given but the needed fee rate differed
This avoids cases where a user requests a fee rate below the minimum and is silently overruled by the wallet.
2020-05-05 14:09:20 +09:00
glowang
be01449cc8 Add test for param interaction b/w -blocksonly and -whitelistforcerelay 2020-05-04 21:57:47 -07:00
Andrew Chow
ca2a09640f Change SetType to SetInternal and remove m_address_type
m_address_type was used for two things:
1. Determine the type of descriptor to generate during
   SetupDescriptorGeneration
2. Sanity check during GetNewDestination.

There is no need to have this variable to accomplish those things.
1. Add a argument to SetupDescriptorGeneration indicating the address
   type to use
2. Use Descriptor::GetOutputType for the sanity check.
2020-05-05 00:24:46 -04:00
Andrew Chow
89b1ce1140 Remove unimplemented SetCrypted from DescriptorScriptPubKeyMan 2020-05-05 00:24:06 -04:00
Andrew Chow
b9073c8f13 rpc: createwallet warning that descriptor wallets are experimental 2020-05-05 00:24:06 -04:00
Andrew Chow
610030d95c docs: Add release notes for descriptor wallets 2020-05-05 00:24:00 -04:00
Samuel Dobson
ec79b5f86b Merge #18782: wallet: Make sure no DescriptorScriptPubKeyMan or WalletDescriptor members are left uninitialized after construction
2a78098098 wallet: Make sure no WalletDescriptor members are uninitialized after construction (practicalswift)
ff046aeeba wallet: Make sure no DescriptorScriptPubKeyMan members are uninitialized after construction (practicalswift)

Pull request description:

  This is a small folllow-up to #16528 ("Native Descriptor Wallets using DescriptorScriptPubKeyMan") which was merged in to `master` a couple of hours ago.

  Make sure no `DescriptorScriptPubKeyMan` or `WalletDescriptor` members are left uninitialized after construction.

  Before this change `bool m_internal` was left uninitialized when using the `DescriptorScriptPubKeyMan(WalletStorage&, WalletDescriptor&)` ctor.

  The same goes for the now initialized integers which were left uninitialized when using the `WalletDescriptor()` ctor.

ACKs for top commit:
  instagibbs:
    utACK  2a78098098
  fjahr:
    Code review ACK 2a78098098
  Sjors:
    utACK 2a78098
  achow101:
    ACK 2a78098098
  brakmic:
    Code review ACK 2a78098098
  meshcollider:
    utACK 2a78098098

Tree-SHA512: c98e035268fdc7f65a423b73ac0cf010b0ef7c5e679b3cf170c1813efac8ab5c657dcbaf43c746770bea59e4772bfefe4caa834f1175260c39c7f35d92946ba5
2020-05-05 15:56:04 +12:00
Hennadii Stepanov
0c63f80854 build: Suppress -Wdeprecated-copy warnings 2020-05-05 06:21:52 +03:00
fanquake
d7120f7f78 valgrind : remove duplicate BCLog::Logger suppression
This was added in #17770, but is identical to the supression above.
2020-05-05 11:10:36 +08:00
fanquake
708e3c7e85 valgrind: remove rest_blockhash_by_height suppression
This should no-longer be necessary after #18785.
2020-05-05 11:10:35 +08:00
Hennadii Stepanov
73529f0859 qt: Rename slot to updateDisplayUnit()
This commit does not change behavior.
2020-05-05 05:57:08 +03:00
Hennadii Stepanov
68288ef0c1 qt: Overhaul ReceiveRequestDialog 2020-05-05 05:56:50 +03:00
Russell Yanofsky
f963a68051 test: Add test for conflicted wallet tx notifications
Add test coverage for conflicted wallet transaction notifications so we improve
current behavior and avoid future regressions

https://github.com/bitcoin/bitcoin/pull/9240 - accidental break
https://github.com/bitcoin/bitcoin/issues/9479 - bug report
https://github.com/bitcoin/bitcoin/pull/9371 - fix
https://github.com/bitcoin/bitcoin/pull/16624 - accidental break
https://github.com/bitcoin/bitcoin/issues/18325 - bug report
https://github.com/bitcoin/bitcoin/pull/18600 - potential fix
2020-05-04 22:27:19 -04:00
Hennadii Stepanov
18bd83b1fe util: Cleanup translation.h 2020-05-05 04:51:29 +03:00
Hennadii Stepanov
e95e658b8e doc: Do not translate technical or extremely rare errors 2020-05-05 04:46:08 +03:00
Hennadii Stepanov
7e923d47ba Make InitError bilingual 2020-05-05 04:46:04 +03:00
Hennadii Stepanov
917ca93553 Make ThreadSafe{MessageBox|Question} bilingual 2020-05-05 04:45:59 +03:00
Hennadii Stepanov
23b9fa2e5e gui: Add detailed text to BitcoinGUI::message 2020-05-05 04:40:56 +03:00
fanquake
e727c2bdca Merge #18088: build: ensure we aren't using GNU extensions
0ae8f18dfe build: add -Wgnu to compile flags (fanquake)
3a0fd7726b Remove use of non-standard zero variadic macros (Ben Woosley)
49f6178c3e Drop unused LOG_TIME_MICROS helper (Ben Woosley)
5d4999951e prevector: Avoid unnamed struct, which is a GNU extension (DesWurstes)

Pull request description:

  Since we [started using](https://github.com/bitcoin/bitcoin/pull/7165) the `ax_cxx_compile_stdcxx.m4` macro we've been passing `[noext]` to indicate that we don't want to use an extended mode, i.e GNU extensions. Speaking to Cory he clarified that the intention was to "require only vanilla c++11 and turn _off_ extension support so they would fail to compile".

  However in the codebase we are currently making use of some GNU extensions. We should either remove there usage, or at least amend our CXX compiler checks. I'd prefer the former.

  #### anonymous structs
  ```bash
  ./prevector.h:153:9: warning: anonymous structs are a GNU extension [-Wgnu-anonymous-struct]
          struct {
  ```

  This is fixed in b849212c1e.

  #### variadic macros

  ```bash
  ./undo.h:57:50: warning: must specify at least one argument for '...' parameter of variadic macro [-Wgnu-zero-variadic-macro-arguments]
              ::Unserialize(s, VARINT(nVersionDummy));
  ```

  This is taken care of in #18087.

  The `LOG_TIME_*` macros introduced in #16805 make use of a [GNU extension](https://gcc.gnu.org/onlinedocs/cpp/Variadic-Macros.html).

  ```bash
  In file included from validation.cpp:22:
  ./logging/timer.h:99:99: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
      BCLog::Timer<std::chrono::milliseconds> PASTE2(logging_timer, __COUNTER__)(__func__, end_msg, ## __VA_ARGS__)
                                                                                                    ^
  ./logging/timer.h:99:99: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
  ./logging/timer.h:99:99: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
  ./logging/timer.h:99:99: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
  ./logging/timer.h:99:99: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
  ./logging/timer.h:101:92: warning: token pasting of ',' and __VA_ARGS__ is a GNU extension [-Wgnu-zero-variadic-macro-arguments]
      BCLog::Timer<std::chrono::seconds> PASTE2(logging_timer, __COUNTER__)(__func__, end_msg, ## __VA_ARGS__)
                                                                                             ^
  6 warnings generated.
  ```

  This is fixed in 081a0ab64eb442bc85c4d4a4d3bc2c8e97ac2a6d and 612e8e138b97fc5ad2f38847300132a8fc423c3f.

  #### prevention
  To ensure that usage doesn't creep back in we can add [`-Wgnu`](https://clang.llvm.org/docs/DiagnosticsReference.html#wgnu) to our compile time flags, which will make Clang warn whenever it encounters GNU extensions.

  This would close #14130.
  Also related to #17230, where it's suggested we use a GNU extension, the `gnu::pure` attribute.

ACKs for top commit:
  practicalswift:
    ACK 0ae8f18dfe -- diff looks correct
  MarcoFalke:
    ACK 0ae8f18dfe
  vasild:
    utACK 0ae8f18df
  dongcarl:
    ACK 0ae8f18dfe

Tree-SHA512: c517404681ef8edf04c785731d26105bac9f3c9c958605aa24cbe399c649e7c5ee0c4aa8e714fd2b2d335e2fbea4d571e09b0dec36678ef871f0a6683ba6bb7f
2020-05-05 07:44:23 +08:00
Carl Dong
bfe1ba2f5b rel-builds: Specify core.abbrev for git-rev-parse
Chose 12 because the kernel uses it:

  https://public-inbox.org/git/CA+55aFy0_pwtFOYS1Tmnxipw9ZkRNCQHmoYyegO00pjMiZQfbg@mail.gmail.com/raw

And also because it's a nice number.
2020-05-04 13:00:08 -04:00
Hennadii Stepanov
c269e618cf Drop unused GIT_COMMIT_DATE macro 2020-05-04 19:53:58 +03:00
Hennadii Stepanov
8f9f4ba5e2 refactor: Remove duplicated code 2020-05-04 19:53:21 +03:00
MarcoFalke
fa47cf9d95 wallet: Fix typo in assert that is compile-time true 2020-05-04 10:40:48 -04:00
Wladimir J. van der Laan
b549cb1bd2 Merge #18443: lockedpool: avoid sensitive data in core files (FreeBSD)
f85203097f lockedpool: avoid sensitive data in core files (FreeBSD) (Vasil Dimov)

Pull request description:

  This is a followup to
  23991ee53 / https://github.com/bitcoin/bitcoin/pull/15600
  to also use madvise(2) on FreeBSD to avoid sensitive data allocated
  with secure_allocator ending up in core files in addition to preventing
  it from going to the swap.

ACKs for top commit:
  sipa:
    ACK f85203097f if someone verifies this works as intended on *BSD.
  laanwj:
    ACK f85203097f
  practicalswift:
    Code-review ACK f85203097f assuming a reviewer with FreeBSD access verifies that the PR goal is achieved :)

Tree-SHA512: 2e6d4ab6a9fbe18732c8ba530eacc17f58128c97140758b80c905b5b838922a2bcaa5f9abc45ab69d5a1a2baa0cba322f006048b60a877228e089c7e64dadd2a
2020-05-04 16:31:07 +02:00
Wladimir J. van der Laan
23c926d859 Merge #18699: wallet: Avoid translating RPC errors
fa2cce4391 wallet: Remove trailing whitespace from potential translation strings (MarcoFalke)
fa59cc1c97 wallet: Report full error message in wallettool (MarcoFalke)
fae7776690 wallet: Avoid translating RPC errors when creating txs (MarcoFalke)
fae51a5c6f wallet: Avoid translating RPC errors when loading wallets (MarcoFalke)

Pull request description:

  Common errors and warnings should be translated when displayed in the
  GUI, but not translated when displayed elsewhere. The wallet method
  `CreateWalletFromFile` does not know its caller, so this commit changes it
  to return a `bilingual_str` to the caller.

  Fixes #17072

ACKs for top commit:
  laanwj:
    ACK fa2cce4391, checked that no new translation messages are added compared to master.
  hebasto:
    ACK fa2cce4391

Tree-SHA512: c6a943ae9c3689ea3c48c20d26de6e4970de0257a1f1eec57a2bded67a4af9dcc5c45b2d64659d6fb4c4bc4d8103e28483ea3d14bb850df8db0ff9e8e5c77ee2
2020-05-04 16:29:22 +02:00
Wladimir J. van der Laan
42fd503819 Merge #18786: init: Remove boost from ThreadImport
faec3dc2ad init: Remove boost from ThreadImport (MarcoFalke)

Pull request description:

  Can be tested by calling `-reindex` or `-loadblock` and then pressing `CTRL`+`C`.

  Should print something like:

  ```
  ...
  2020-04-27T19:34:31Z [loadblk] Reindexing block file blk00005.dat...
  ^C2020-04-27T19:34:32Z [loadblk] Shutdown requested. Exit ThreadImport
  2020-04-27T19:34:32Z [qt-init] Interrupting HTTP server
  ...
  ```

ACKs for top commit:
  laanwj:
    Code review ACK faec3dc2ad
  hebasto:
    ACK faec3dc2ad, tested on Linux Mint 19.3 (x86_64) both `bitcoind` and `bitcoin-qt` binaries.

Tree-SHA512: e105af18d98296d82ec99f48e478cf44577e3c32f7e4b47617a7bc7cbf71d6becb92722f229a1be38d58ad29712704509ad9740d8ab8cd3104cf90057664b437
2020-05-04 16:06:42 +02:00
MarcoFalke
0a729b0e42 Merge #18783: tests: Add fuzzing harness for MessageSign, MessageVerify and other functions in util/message.h
38e49ded8b tests: Add fuzzing harness for MessageSign, MessageVerify and other functions in util/message.h (practicalswift)

Pull request description:

  Add fuzzing harness for `MessageSign`, `MessageVerify` and other functions in `util/message.h`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  vasild:
    utACK 38e49ded8b

Tree-SHA512: 4f83718365d9c7e772a4ccecb31817bf17117efae2bfaf6e9618ff17908def0c8b97b5fa2504d51ab38b2e6f82c046178dd751495cc37ab4779c0b1ac1a4d211
2020-05-04 09:02:21 -04:00
MarcoFalke
fae153b409 test: Fix verack race to avoid intermittent test failures 2020-05-04 08:56:30 -04:00
MarcoFalke
74a1152f25 Merge #18859: Remove CCoinsViewCache::GetValueIn(...)
b56607a89b Remove CCoinsViewCache::GetValueIn(...) (practicalswift)

Pull request description:

  Remove `CCoinsViewCache::GetValueIn(...)`.

  Fixes #18858.

  It seems like `GetValueIn` was added in #748 ("Pay-to-script-hash (OP_EVAL replacement)", merged in 2012) and the last use in validation code was removed in #8498 ("Near-Bugfix: Optimization: Minimize the number of times it is checked that no money...", merged in 2017).

  `CCoinsViewCache::GetValueIn(…)` performs money summation like this:

  ```c++
  CAmount CCoinsViewCache::GetValueIn(const CTransaction& tx) const
  {
      if (tx.IsCoinBase())
          return 0;

      CAmount nResult = 0;
      for (unsigned int i = 0; i < tx.vin.size(); i++)
          nResult += AccessCoin(tx.vin[i].prevout).out.nValue;

      return nResult;
  }
  ```

  Note that no check is done to make sure that the resulting `nResult` is such that it stays within the money bounds (`MoneyRange(nResult)`), or that the summation does not trigger a signed integer overflow.

  Proof of concept output:

  ```
  coins.cpp:243:17: runtime error: signed integer overflow: 9223200000000000000 + 2100000000000000 cannot be represented in type 'long'
  GetValueIn = -9221444073709551616
  ```

  Proof of concept code:

  ```c++
  CMutableTransaction mutable_transaction;
  mutable_transaction.vin.resize(4393);

  Coin coin;
  coin.out.nValue = MAX_MONEY;
  assert(MoneyRange(coin.out.nValue));

  CCoinsCacheEntry coins_cache_entry;
  coins_cache_entry.coin = coin;
  coins_cache_entry.flags = CCoinsCacheEntry::DIRTY;

  CCoinsView backend_coins_view;
  CCoinsViewCache coins_view_cache{&backend_coins_view};
  CCoinsMap coins_map;
  coins_map.emplace(COutPoint{}, std::move(coins_cache_entry));
  coins_view_cache.BatchWrite(coins_map, {});

  const CAmount total_value_in = coins_view_cache.GetValueIn(CTransaction{mutable_transaction});
  std::cout << "GetValueIn = " << total_value_in << std::endl;
  ```

ACKs for top commit:
  MarcoFalke:
    ACK b56607a89b
  promag:
    Code review ACK b56607a89b.
  jb55:
    ACK b56607a89b
  hebasto:
    ACK b56607a89b, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged.

Tree-SHA512: 2c8402b5753ec96703d12c57c3eda8eccf999ed3519134a87faaf0838cfe44b94ef384296af2a524c06c8756c0245418d181af9083548e360905fac9d79215e6
2020-05-04 07:48:23 -04:00
Sjors Provoost
9c246b873c [test] backwards compatibility: bump v0.19.0.1 to v0.19.1 2020-05-04 13:36:40 +02:00
Sjors Provoost
89a28e02fa [test] add v0.16.3 backwards compatibility test 2020-05-04 13:36:40 +02:00
João Barbosa
e8123eae40 gui: Fix itemWalletAddress leak when not tree mode 2020-05-04 12:05:42 +01:00
Jonas Schnelli
afa577c323 Merge #15768: gui: Add close window shortcut
f5a3a5b9ab gui: Add close window shortcut (Miguel Herranz)

Pull request description:

  CMD+W is the standard shortcut in macOS to close a window without
  exiting the program.

  This adds support to use the shortcut in both main and debug windows.

ACKs for top commit:
  jonasschnelli:
    Tested ACK f5a3a5b9ab
  hebasto:
    ACK f5a3a5b9ab, tested on Linux Mint 19.3 by manually opening available dialogs and sub-windows, and applying the `Ctrl+W` shortcut. Also tested with "Minimize on close" option enabled / disabled.

Tree-SHA512: 39851f6680cf97c334d5759c6f8597cb45685359417493ff8b0566672edbd32303fa15ac4260ec8ab5ea1458a600a329153014f25609e1db9cf399aa851ae2f9
2020-05-04 11:53:34 +02:00
practicalswift
b56607a89b Remove CCoinsViewCache::GetValueIn(...) 2020-05-03 18:42:14 +00:00
Vasil Dimov
71f183a49b build: warn on potentially uninitialized reads
Enable -Wconditional-uninitialized to warn on potentially uninitialized
reads.

Fix the sole such warning in Bitcoin Core in GetRdRand(): r1 would be
set to 0 on rdrand failure, so initializing it to 0 is a non-functional
change.

From "Intel 64 and IA-32 ArchitecturesSoftware Developer's Manual" [1],
page 1711: "CF=1 indicates that the data in the destination is valid.
Otherwise CF=0 and the data in the destination operand will be returned
as zeros for the specified width."

[1] https://software.intel.com/sites/default/files/managed/39/c5/325462-sdm-vol-1-2abcd-3abcd.pdf
2020-05-03 17:21:45 +02:00
MarcoFalke
2e6a16b968 Merge #18855: tests: feature_backwards_compatibility.py test downgrade after upgrade
489ebfd7a1 tests: feature_backwards_compatibility.py test downgrade after upgrade (Andrew Chow)

Pull request description:

  After upgrading the node, try to go back to the original version to make sure that using a newer node version does not prevent the wallet file from being downgraded again.

ACKs for top commit:
  MarcoFalke:
    ACK 489ebfd7a1

Tree-SHA512: 86231de6514b3657912fd9d6621212166fd2b29b591fc97120092c548babcf1d6f50b5bd103b28cecde395a26809134f01c1a198725596c3626420de3fd1f017
2020-05-03 10:35:21 -04:00
MarcoFalke
cce034b028 Merge #18839: test: Fix intermittent issues
fab7ee3990 test: Fix p2p_leak intermittent issue (MarcoFalke)
fa8614aea9 test: Fix intermittent p2p_segwit issue (MarcoFalke)

Pull request description:

  Fixes #18801
  Fixes #18802

ACKs for top commit:
  practicalswift:
    ACK fab7ee3990 -- diff looks correct

Tree-SHA512: b5d0473ec1133aded127bef4c0c90f6b55906327cb49cbbd0776c4971cd9d5c9375fb70cc3dc8dd1f4bd8b1245c4414e803dc0d29ed4553b00eb131d27c9c128
2020-05-03 10:30:47 -04:00
MarcoFalke
ddc0a600b3 Merge #18617: test: add factor option to adjust test timeouts
2742c34286 test: add factor option to adjust test timeouts (Harris)

Pull request description:

  This PR adds a new option **factor** that can be used to adjust timeouts in various functional tests.
  Several timeouts and functions from `authproxy`, `mininode`, `test_node` and `util` have been adapted to use this option. The factor-option definition is located in `test_framework.py`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18266
  Also Fixes https://github.com/bitcoin/bitcoin/issues/18834

ACKs for top commit:
  MarcoFalke:
    Thanks! ACK 2742c34286

Tree-SHA512: 6d8421933ba2ac1b7db00b70cf2bc242d9842c48121c11aadc30b0985c4a174c86a127d6402d0cd73b993559d60d4f747872d21f9510cf4806e008349780d3ef
2020-05-03 08:58:56 -04:00
Andrew Chow
489ebfd7a1 tests: feature_backwards_compatibility.py test downgrade after upgrade
After upgrading the node, try to go back to the original version to make
sure that using a newver node version does not prevent the wallet file
from being downgraded again.
2020-05-02 23:45:17 -04:00
Harris
2742c34286 test: add factor option to adjust test timeouts
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-05-03 01:42:40 +02:00
MarcoFalke
fa09110ebb doc: Fix typo in Coin doxygen comment 2020-05-02 19:30:58 -04:00
fanquake
68ef9523d1 Merge #18413: script: prevent UB when computing abs value for num opcode serialize
2748e87932 script: prevent UB when computing abs value for num opcode serialize (pierrenn)

Pull request description:

  This was reported by practicalswift here #18046

  It seems that the original author of the line used a reference to glibc `abs`: https://github.com/lattera/glibc/blob/master/stdlib/abs.c

  However depending on some implementation details this can be undefined behavior for unusual values.

  A detailed explanation of the UB is provided here : https://stackoverflow.com/questions/17313579/is-there-a-safe-way-to-get-the-unsigned-absolute-value-of-a-signed-integer-with (by [Billy O'Neal](https://twitter.com/malwareminigun))

  Simple relevant godbolt example :  https://godbolt.org/z/yRwtCG

  Thanks!

ACKs for top commit:
  sipa:
    ACK 2748e87932
  MarcoFalke:
    ACK 2748e87932, only checked that the bitcoind binary does not change with clang -O2 🎓
  practicalswift:
    ACK 2748e87932

Tree-SHA512: 539a34c636c2674c66cb6e707d9d0dfdce63f59b5525610ed88da10c9a8d59d81466b111ad63b850660cef3750d732fc7755530c81a2d61f396be0707cd86dec
2020-05-02 21:24:05 +08:00
Andrew Chow
b59b4504ab have GenerateNewKey and DeriveNewChildKey take a CHDChain as an argument 2020-05-01 18:46:00 -04:00
Hennadii Stepanov
35f1189ea7 build: Rename BUILD_* macros and the code self-descriptive 2020-05-02 01:00:07 +03:00
Hennadii Stepanov
dc1fba9389 scripted-diff: Rename share/genbuild.sh macros to more meaningful ones
-BEGIN VERIFY SCRIPT-
sed -i 's/\bDESC\b/GIT_TAG/' share/genbuild.sh
sed -i 's/\bSUFFIX\b/GIT_COMMIT/g' share/genbuild.sh
-END VERIFY SCRIPT-
2020-05-02 00:59:20 +03:00
MarcoFalke
844d2070a2 Merge #18828: test: Strip down previous releases boilerplate
fa359d14c0 test: Strip down previous releases boilerplate (MarcoFalke)

Pull request description:

  Reduces code bloat and mental load to write compatibility tests

ACKs for top commit:
  Sjors:
    tACK fa359d14c0 on macOS

Tree-SHA512: dc66286b24b2f137e5bca99412850ec7eee8cc61cf9cdc7ab532d529220808189baea8d1b077f8b7f40d3e8881d981e1ffc5a877adb394816f1225b1186253e4
2020-05-01 14:57:59 -04:00
Carl Dong
27e63e01cc build: Accomodate makensis v2.x
Apparently the -X flag doesn't work as expected in makensis v2.x

For example:

  makensis -V2 share/setup.nsi -X'OutFile "test.exe"'

Will output:

  OutFile expects 1 parameters, got 0.
  Usage: OutFile install_output.exe

So let's instead construct the file using POSIX-compliant commands and
shell constructs
2020-05-01 14:27:57 -04:00
Carl Dong
1f2c39a30e guix: Remove logical cores requirement
Thanks MarcoFalke for pushing this to its limits and testing :-)
2020-05-01 12:33:47 -04:00
Carl Dong
a4f6ffa71e lint: Also enable source statements for non-gitian 2020-05-01 12:31:07 -04:00
Carl Dong
d256f91cb1 rel-builds: Directly deploy win installer to OUTDIR 2020-05-01 12:30:20 -04:00
MarcoFalke
fab7ee3990 test: Fix p2p_leak intermittent issue 2020-05-01 08:45:26 -04:00
MarcoFalke
fa8614aea9 test: Fix intermittent p2p_segwit issue 2020-05-01 08:36:35 -04:00
MarcoFalke
fa2cce4391 wallet: Remove trailing whitespace from potential translation strings
If the potential translation strings are translated in the future,
trailing whitespace is going to make translation effort harder.
2020-05-01 07:41:32 -04:00
MarcoFalke
fa59cc1c97 wallet: Report full error message in wallettool 2020-05-01 07:39:35 -04:00
MarcoFalke
fae7776690 wallet: Avoid translating RPC errors when creating txs
Also, mark feebumper bilingual_str as Untranslated

They are technical and have previously not been translated either.
It is questionable whether they can even appear in the GUI.
2020-05-01 07:39:06 -04:00
MarcoFalke
fae51a5c6f wallet: Avoid translating RPC errors when loading wallets
Common errors and warnings should be translated when displayed in the
GUI, but not translated when displayed elsewhere. The wallet method
CreateWalletFromFile does not know its caller, so this commit changes it
to return a bilingual_str to the caller.
2020-05-01 07:39:00 -04:00
Russell Yanofsky
d3a56be77a Revert "gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged"
This reverts commit 0933a37078 from
https://github.com/bitcoin/bitcoin/pull/18160 which no longer an optimization
since commit "gui: Avoid wallet tryGetBalances calls before TransactionChanged
or BlockTip notifications".
2020-05-01 06:59:09 -04:00
Russell Yanofsky
28b112e9bd Get rid of BindWallet
CWalletTx initialization has been fixed so it's no longer necessary to change
which wallet a transaction is bound to.
2020-05-01 05:59:09 -05:00
MarcoFalke
608359b071 Merge #16426: Reverse cs_main, cs_wallet lock order and reduce cs_main locking
6a72f26968 [wallet] Remove locked_chain from CWallet, its RPCs and tests (Antoine Riard)
841178820d [wallet] Move methods from Chain::Lock interface to simple Chain (Antoine Riard)
0a76287387 [wallet] Move getBlockHash from Chain::Lock interface to simple Chain (Antoine Riard)
de13363a47 [wallet] Move getBlockHeight from Chain::Lock interface to simple Chain (Antoine Riard)
b855592d83 [wallet] Move getHeight from Chain::Lock interface to simple Chain (Antoine Riard)

Pull request description:

  This change is intended to make the bitcoin node and its rpc, network and gui interfaces more responsive while the wallet is in use. Currently, because the node's `cs_main` mutex is always locked before the wallet's `cs_wallet` mutex (to prevent deadlocks), `cs_main` currently stays locked while the wallet does relatively slow things like creating and listing transactions.

  Switching the lock order so `cs_main` is acquired after `cs_wallet` allows `cs_main` to be only locked intermittently while the wallet is doing slow operations, so the node is not blocked waiting for the wallet.

  To review the present PR, most of getting right the move is ensuring any `LockAssertion` in `Chain::Lock` method is amended as a `LOCK(cs_main)`. And in final commit, check that any wallet code which was previously locking the chain is now calling a  method, enforcing the lock taking job. So far the only exception I found is `handleNotifications`, which should be corrected.

ACKs for top commit:
  MarcoFalke:
    re-ACK 6a72f26968 🔏
  fjahr:
    re-ACK 6a72f26968
  ryanofsky:
    Code review ACK 6a72f26968. Only difference compared to the rebase I posted is reverting unneeded SetLastBlockProcessed change in wallet_disableprivkeys test

Tree-SHA512: 9168b3bf3432d4f8bc4d9fa9246ac057050848e673efc264c8f44345f243ba9697b05c22c809a79d1b51bf0de1c4ed317960e496480f8d71e584468d4dd1b0ad
2020-05-01 06:59:09 -04:00
Russell Yanofsky
bf0a510981 gui: Avoid wallet tryGetBalances calls before TransactionChanged or BlockTip notifications
interfaces::Wallet::tryGetBalances was recently updated in
https://github.com/bitcoin/bitcoin/pull/18160 to avoid computing balances
internally, but this not efficient as it could be with #10102 because
tryGetBalances is an interprocess call.

Implementing the TransactionChanged / BlockTip check outside of tryGetBalances
also allows tryGetBalances to be simplified in next commit 'Revert "gui: Avoid
Wallet::GetBalance in WalletModel::pollBalanceChanged"'.
2020-05-01 06:59:09 -04:00
Russell Yanofsky
d002f9d15d Disable CWalletTx copy constructor
Disable copying of CWalletTx objects to prevent bugs where instances get copied
in and out of the mapWallet map and fields are updated in the wrong copy.
2020-05-01 05:59:09 -05:00
Russell Yanofsky
2bc9b92ed8 Cancel wallet balance timer when shutdown requested
This doesn't fix any current problem, but it makes balance checking code less
fragile, and prevents use-after free travis error in next commit:
https://travis-ci.org/github/bitcoin/bitcoin/jobs/675367629#L4240
2020-05-01 06:59:09 -04:00
Russell Yanofsky
65b9d8f8dd Avoid copying CWalletTx in LoadToWallet
The change in walletdb.cpp is easier to review ignoring whitespace.

This change is need to get rid of CWalletTx copy constructor.
2020-05-01 05:59:09 -05:00
Russell Yanofsky
83f69fab3a Switch transaction table to use wallet height not node height
Tweak of #17905 to make gui display of transactions and balances more
consistent. This change shouldn't cause visible effects in normal cases, just
make GUI wallet code more internally correct and consistent.
2020-05-01 06:59:09 -04:00
Russell Yanofsky
bd2fbc7cdb Get rid of unneeded CWalletTx::Init parameter 2020-05-01 05:59:09 -05:00
Russell Yanofsky
2b9cba2065 Remove CWalletTx merging logic from AddToWallet
Instead of AddToWallet taking a temporary CWalletTx object and then potentially
merging it with a pre-existing CWalletTx, have it take a callback so callers
can update the pre-existing CWalletTx directly.

This makes AddToWallet simpler because now it is only has to be concerned with
saving CWalletTx objects and not merging them.

This makes AddToWallet calls clearer because they can now make direct updates to
CWalletTx entries without having to make temporary objects and then worry about
how they will be merged.

This is a pure refactoring, no behavior is changing.
2020-05-01 05:59:09 -05:00
Antoine Riard
6a72f26968 [wallet] Remove locked_chain from CWallet, its RPCs and tests
This change is intended to make the bitcoin node and its rpc, network
and gui interfaces more responsive while the wallet is in use. Currently
because the node's cs_main mutex is always locked before the wallet's
cs_wallet mutex (to prevent deadlocks), cs_main currently stays locked
while the wallet does relatively slow things like creating and listing
transactions.

This commit only remmove chain lock tacking in wallet code, and invert
lock order from cs_main, cs_wallet to cs_wallet, cs_main.
must happen at once to avoid any deadlock. Previous commit were only
removing Chain::Lock methods to Chain interface and enforcing they
take cs_main.

Remove LockChain method from CWallet and Chain::Lock interface.
2020-04-30 14:41:24 -04:00
Antoine Riard
841178820d [wallet] Move methods from Chain::Lock interface to simple Chain
Remove findPruned and findFork, no more used after 17954.
2020-04-30 14:37:21 -04:00
Antoine Riard
0a76287387 [wallet] Move getBlockHash from Chain::Lock interface to simple Chain 2020-04-30 14:37:21 -04:00
Antoine Riard
de13363a47 [wallet] Move getBlockHeight from Chain::Lock interface to simple Chain
Add HaveChain to assert chain access for wallet-tool in LoadToWallet.
2020-04-30 14:37:21 -04:00
Antoine Riard
b855592d83 [wallet] Move getHeight from Chain::Lock interface to simple Chain
Instead of calling getHeight, we rely on CWallet::m_last_block
processed_height where it's possible.
2020-04-30 14:31:19 -04:00
MarcoFalke
e5b9308920 Merge #18829: ci: Merge C++17 build with one of the existing ones
aaaacff107 ci: Merge C++17 build with one of the existing ones (MarcoFalke)

Pull request description:

  No need to spin up an extra vm for each pull request for a simple sanity check that any of the other already running machines can test.

Top commit has no ACKs.

Tree-SHA512: ce1609aa4cbc057fc9e85e61f300eac2317cc206647a20f6b001180ffed6623e2243a408476546e16baacb91cb2dd9a2e7b6e2402c8fd829a72860b2c3eb7be6
2020-04-30 14:04:24 -04:00
MarcoFalke
ef8ef31fd2 Merge #18775: tests: Add fuzzing harnesses for various classes/functions in policy/ (CBlockPolicyEstimator, IsRBFOptIn(…), etc.)
2bcc2bd742 tests: Clarify how we avoid hitting the signed integer overflow in CFeeRate::GetFeePerK() when fuzzing (practicalswift)
13c1f6b24f tests: Add fuzzing harness for IsRBFOptIn(...) (practicalswift)
3439c88a5d tests: Add fuzzing harness for CBlockPolicyEstimator (practicalswift)

Pull request description:

  Add fuzzing harnesses for various classes/functions in `policy/` (`CBlockPolicyEstimator`, `IsRBFOptIn(…)`, etc.).

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: a756687216802a3b260def5706798a1eb673b3447408561728af77a1d61c8bfb3a7d9b874e16bf619565e9d093f9b595e07d6e6fa430ac08dfeed4f45b01fbc3
2020-04-30 12:53:42 -04:00
MarcoFalke
aaaacff107 ci: Merge C++17 build with one of the existing ones 2020-04-30 12:50:36 -04:00
MarcoFalke
fa359d14c0 test: Strip down previous releases boilerplate 2020-04-30 11:45:16 -04:00
practicalswift
2bcc2bd742 tests: Clarify how we avoid hitting the signed integer overflow in CFeeRate::GetFeePerK() when fuzzing 2020-04-30 14:19:49 +00:00
MarcoFalke
a66ba6d029 Merge #18576: test: use unittest for test_framework unit testing
de8905adf2 test: use unittest and test_runner for test framework unit testing (Gloria Zhao)

Pull request description:

  Proposal for unit testing on test_framework functions:
  1. Use the python `unittest` library. Don't use test_framework to test itself.
  2. Put the tests inside the same file as the functions they are testing.
  3. Call the tests from `test_runner.py`. To include more Test Framework tests, add the filename to the list `TEST_FRAMEWORK_MODULES`. Don't add new files or change the list of accepted script prefixes.

  Makes these changes for `bn2vch` (followup to [this comment](https://github.com/bitcoin/bitcoin/pull/18378#pullrequestreview-377271264)).

ACKs for top commit:
  jnewbery:
    Tested ACK de8905adf2. Great stuff gzhao408 . Thanks for this!

Tree-SHA512: 91572d43e203a1864765b93a9472667994115ec38b271f2b2f9fcd0f0112b393fc24ba7d2371d5a34b0a6a4522f6b934fc5164363819aa7ed8d6c6c9a60cc101
2020-04-30 09:30:13 -04:00
practicalswift
13c1f6b24f tests: Add fuzzing harness for IsRBFOptIn(...) 2020-04-30 13:19:24 +00:00
practicalswift
3439c88a5d tests: Add fuzzing harness for CBlockPolicyEstimator 2020-04-30 13:19:24 +00:00
MarcoFalke
0000ea3265 test: Add test for GetRandMillis and GetRandMicros 2020-04-30 09:19:16 -04:00
MarcoFalke
fa0e5b89cf Add templated GetRandomDuration<> 2020-04-30 09:19:14 -04:00
MarcoFalke
00c1a4d9a9 Merge #18809: rpc: Do not advertise dumptxoutset as a way to flush the chainstate
fac0cf6e55 rpc: Do not advertise dumptxoutset as a way to flush the chainstate (MarcoFalke)

Pull request description:

  The help message leaks several implementation details: leveldb and flush.

  Neither of them are relevant to the end user and I don't see why we should make them part of the API contract.

ACKs for top commit:
  laanwj:
    ACK fac0cf6e55

Tree-SHA512: 273fb85dc5be6cdccf17c43f183fa83c57d0a1cbb30555838f32c074218b713a753930009f6c98c85659421f2285f09c0a713b22f7e34d446e56737ac03870f7
2020-04-30 07:20:14 -04:00
MarcoFalke
cf5e3be5ea Merge #18825: test: fix message for ECC_InitSanityCheck test
06e434d7d9 test: fix message for ECC_InitSanityCheck test (fanquake)

Pull request description:

  OpenSSL is long gone.

ACKs for top commit:
  laanwj:
    Good catch. ACK 06e434d7d9

Tree-SHA512: 1a920fd6493e0374ca00633407e0130f987b136bc68d2062402747bda16a1e588a12bd8b0b8cdef828c9911f210386cfbdb25d478cb9b684d52769d197032064
2020-04-30 07:09:05 -04:00
fanquake
64673b1037 Merge #18780: validation: add const for minimum witness commitment size
692f8307fc test: add test for witness commitment index (fanquake)
06442549f8 validation: Add minimum witness commitment size constant (fanquake)

Pull request description:

  16101de5f3: Per [BIP 141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Commitment_structure), the witness commitment structure is at least 38 bytes,
  OP_RETURN (0x6a) + 36 (0x24) + 4 byte header (0xaa21a9ed) + 32 byte
  SHA256 hash. It can be longer, however any additional data has no
  consensus meaning.

  54f8c48d6a: As per BIP 141, if there is more than 1 pubkey that matches the witness
  commitment structure, the one with the highest output index should be
  chosen. This adds a sanity check that we are doing that, which will fail
  if anyone tries to "optimize" GetWitnessCommitmentIndex() by returning
  early.

ACKs for top commit:
  MarcoFalke:
    ACK 692f8307fc 🌵
  jonatack:
    Code review ACK 692f830
  ajtowns:
    ACK 692f8307fc
  jnewbery:
    utACK 692f8307fc
  laanwj:
    ACK 692f8307fc

Tree-SHA512: 7af3fe4b8a52fea2cdd0aec95f7bb935351a77b73d934bc88d6625a3503311b2a062cba5190b2228f97caa76840db3889032d910fc8e318ca8e7810a8afbafa0
2020-04-30 18:50:26 +08:00
fanquake
0ae8f18dfe build: add -Wgnu to compile flags
When compiling with Clang, this will warn when GNU extensions are
used.

Info: https://clang.llvm.org/docs/DiagnosticsReference.html#wgnu
2020-04-30 18:02:04 +08:00
Ben Woosley
3a0fd7726b Remove use of non-standard zero variadic macros
These are a gnu extension warned against by: gnu-zero-variadic-macro-arguments
2020-04-30 18:02:04 +08:00
Ben Woosley
49f6178c3e Drop unused LOG_TIME_MICROS helper 2020-04-30 18:02:04 +08:00
DesWurstes
5d4999951e prevector: Avoid unnamed struct, which is a GNU extension 2020-04-30 18:02:03 +08:00
Wladimir J. van der Laan
afed2e98b0 Merge #18810: doc: update rest info on block size and json
ff6549c3c8 fix: update rest info on block size and json (Chris Abrams)

Pull request description:

  Addressing the ambiguous block size text in rest docs: https://github.com/bitcoin/bitcoin/issues/18703

  Also makes sure to let developers know there is `.json` option for the rest output format.

ACKs for top commit:
  MarcoFalke:
    ACK ff6549c3c8
  promag:
    ACK ff6549c3c8.

Tree-SHA512: 9ef93c1432d650b1f9599778ba092c1ca5b084a537af257078e1c713c76c5d3a4cc4b1ede8a2489964be8ed0303ad8bea58c1cb4759bbb9b24dbdebfec8001d3
2020-04-30 11:45:06 +02:00
Wladimir J. van der Laan
35ef3c15ef Merge #18591: Add C++17 build to Travis
c31cbe7cfe Add C++17 test to Travis (Pieter Wuille)
7829685e27 Add configure option for c++17 (Pieter Wuille)
0fbde488b2 Support conversion between Spans of compatible types (Pieter Wuille)
7cbfebbf3d Update ax_cxx_compile_stdcxx.m4 (Pieter Wuille)

Pull request description:

  This adds a `--enable-c++17` option to the configure script, fixes the only C++17 incompatibility (with a commit taken from #18468), and adds a Travis test for it.

  This is all off by default, and release builds remain C++11.

  It implements the first step of the plan in https://github.com/bitcoin/bitcoin/issues/16684.

ACKs for top commit:
  elichai:
    tACK c31cbe7cfe
  practicalswift:
    Tested ACK c31cbe7cfe
  hebasto:
    ACK c31cbe7cfe, tested on Linux Mint 19.3 both C++11 and C++17 modes. Compiled and passed tests locally.

Tree-SHA512: a4b00776dbceef9c12abbb404c6bcd48f7916ce24c8c7a14116355f64e817578b7fcddbedd5ce435322319d1e4de43429b68553f4d96d970c308fe3e3e59b9d1
2020-04-30 11:16:56 +02:00
fanquake
06e434d7d9 test: fix message for ECC_InitSanityCheck test
OpenSSL is long gone.
2020-04-30 16:57:46 +08:00
Wladimir J. van der Laan
63d5ed2fc4 Merge #18437: util: Detect posix_fallocate() instead of assuming
182dbdf0f4 util: Detect posix_fallocate() instead of assuming (Vasil Dimov)

Pull request description:

  Don't assume that `posix_fallocate()` is available on Linux and not
  available on other operating systems. At least FreeBSD has it and we
  are not using it.

  Properly check whether `posix_fallocate()` is present and use it if it
  is.

ACKs for top commit:
  laanwj:
    ACK 182dbdf0f4

Tree-SHA512: f9ed4bd661f33ff6b2b1150591e860b3c1f44e12b87c35e870d06a7013c4e841ed2bf17b41ad6b18fe471b0b23a4b5e42cf1400637180888e0bc56c254fe0766
2020-04-30 10:45:17 +02:00
John Newbery
9847e205bf [docs] Improve commenting in ProcessGetData() 2020-04-29 19:34:01 -04:00
Amiti Uttarwar
2f032556e0 [test] test that an invalid GETDATA doesn't prevent processing of future messages
Co-Authored-By: John Newbery <john@johnnewbery.com>
2020-04-29 19:34:01 -04:00
MarcoFalke
36c0abd8f6 Merge #18726: test: check misbehavior more independently in p2p_filter.py
cd543d9193 test: check misbehavior more independently in p2p_filter.py (Danny Lee)

Pull request description:

  This expands on #18672 in two ways:

  - Check positive cases (`filterload` accepted, `filteradd` accepted) in addition to the negative cases added in #18672
  - Address MarcoFalke 's [suggestion](https://github.com/bitcoin/bitcoin/pull/18672#discussion_r412101752) to successfully load a filter before testing `filteradd`

ACKs for top commit:
  theStack:
    re-ACK cd543d9193

Tree-SHA512: f82402f6287ccddf08b38b6432d5e2b2b2ef528802a981d04c24bac459022f732d9090d4849d72d3d1eb2c757161dcb18c4c036b6e11dc80114e9cd49f21c3bd
2020-04-29 18:56:25 -04:00
MarcoFalke
95a9165016 Merge #18736: test: Add fuzzing harnesses for various classes/functions in util/
32b6b386a5 tests: Sort fuzzing harnesses (practicalswift)
e1e181fad1 tests: Add fuzzing coverage for JSONRPCTransactionError(...) and RPCErrorFromTransactionError(...) (practicalswift)
103b6ecce0 tests: Add fuzzing coverage for TransactionErrorString(...) (practicalswift)
dde508b8b0 tests: Add fuzzing coverage for ParseFixedPoint(...) (practicalswift)
1532259fca tests: Add fuzzing coverage for FormatHDKeypath(...) and WriteHDKeypath(...) (practicalswift)
90b635e84e tests: Add fuzzing coverage for CHECK_NONFATAL(...) (practicalswift)
a4e3d13df6 tests: Add fuzzing coverage for StringForFeeReason(...) (practicalswift)
a19598cf98 tests: Add fuzzing harness for functions in system.h (ArgsManager) (practicalswift)

Pull request description:

  Add fuzzing harnesses for various classes/functions in `util/`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: d27947220850c2a202c7740f44140c17545f45522596912452ccab0c2f5379abeb07cc769982c7855cb465059425206371a2b75ee1c285b03984161c9619d0b0
2020-04-29 18:54:34 -04:00
MarcoFalke
0f204dd3f2 Merge #18727: test: Add CreateWalletFromFile test
7918c1b019 test: Add CreateWalletFromFile test (Russell Yanofsky)

Pull request description:

  Add unit test calling CreateWalletFromFile, which isn't currently called from other unit tests, with some basic checks to make sure it rescans and registers for notifications correctly.

  Motivation for this change was to try to write a test that would fail without the early `handleNotifications` call in ef8c6ca60767cac589d98ca57ee33179608ccda8 from https://github.com/bitcoin/bitcoin/pull/16426, but succeed with it:

  ef8c6ca607/src/wallet/wallet.cpp (L3978-L3986)

  However, writing a full test for the race condition that call prevents isn't possible without the locking changes from #16426. So this PR just adds as much test coverage as is possible now.

  This new test is also useful for https://github.com/bitcoin/bitcoin/pull/15719, since it detects the stale notifications.transactionAddedToMempool notifications that PR eliminates.

ACKs for top commit:
  MarcoFalke:
    ACK 7918c1b019
  jonatack:
    ACK 7918c1b019

Tree-SHA512: 44035aee698ecb722c6039d061d8fac2011e9da0b314e4aff19be1d610b53cacff99016b34d6b84669bb3b61041b2318d9d8e3363658f087802ae4aa36ca17b8
2020-04-29 15:23:39 -04:00
Hennadii Stepanov
4fc1df41d5 qt: Track QEvent::Resize during animation 2020-04-29 21:45:17 +03:00
Jeremy Rubin
152e8baf08 Use salted hasher instead of nonce in sigcache
Use salted hasher instead of nonce in Script Execution Cache

Don't read more than 32 bytes from GetRand

Apply g_* naming convention to scriptExecutionCache in validation.cpp

Fully apply g_* naming convention to scriptCacheHasher

Write same uint256 nonce twice for cache hash rather than calling getrand twice

Use salted hasher instead of nonce in sigcache

Use salted hasher instead of nonce in Script Execution Cache

Don't read more than 32 bytes from GetRand

Apply g_* naming convention to scriptExecutionCache in validation.cpp

Fully apply g_* naming convention to scriptCacheHasher

Write same uint256 nonce twice for cache hash rather than calling getrand twice
2020-04-29 10:50:19 -07:00
MarcoFalke
978c5a2122 Merge #18485: test: Add mempool_updatefromblock.py
8098dea069 test: Add mempool_updatefromblock.py (Hennadii Stepanov)

Pull request description:

  This PR adds a new test for mempool update of transaction descendants/ancestors information (count, size) when transactions have been re-added from a disconnected block to the mempool.

  It could be helpful for working on PRs like #17925, #18191.

ACKs for top commit:
  ariard:
    ACK 8098dea

Tree-SHA512: 7e808fa8df8d7d7a7dbdc3f79361049b49c7bce9b58fd5539b28c9636bedac747695537e500d7ed68dc8bdb80167ad3f1c01086f7551691405d2ba2e38ef1d06
2020-04-29 12:07:13 -04:00
MarcoFalke
ab91a0e0fc Merge #18798: ci: Fix default retry script usage
45615de26c ci: Fix default retry script usage (Hennadii Stepanov)

Pull request description:

  On master (5352d14b37) `CI_RETRY_EXE=${CI_RETRY_EXE:retry}` works as a [Substring Expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html), and that is wrong.
  If `CI_RETRY_EXE` variable was unset initially, its new value becomes an empty string, but not "retry" as one could expect. Consequently, the `${CI_RETRY_EXE} ...` command does _not_ use `ci/retry/retry` script.

  This PR makes for `CI_RETRY_EXE` variable a usual parameter expansion, i.e., `${parameter:-word}`.

  Reference: https://github.com/bitcoin/bitcoin/pull/18735#issuecomment-620095489

Top commit has no ACKs.

Tree-SHA512: 108173f6b2677979b9ddf2f9b9df4a6c56f5efa81c36543a1816bb3b984e42984bf3c83fe413ea3a5ca1e2317c4efb02fea7180a6b44863af7cfe6202e9cf94d
2020-04-29 11:14:14 -04:00
MarcoFalke
e302830fae Merge #18774: test: added test for upgradewallet RPC
66fe7b1a98 test: added test for upgradewallet RPC (Harris)

Pull request description:

  This PR adds tests for the newly merged *upgradewallet* RPC.

  Additionally, it expands `test_framework/util.py` by adding the function `adjust_bitcoin_conf_for_pre_17` to support nodes that don't parse configuration sections.

  This test uses two older node versions, v0.15.2 and v0.16.3, to create older wallet versions to be used by `upgradewallet`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18767

Top commit has no ACKs.

Tree-SHA512: bb72ff1e829e2c3954386cc308842820ef0828a4fbb754202b225a8748f92d4dcc5ec77fb146bfd5484a5c2f29ce95adf9f3fb4483437088ff3ea4a8d2c442c1
2020-04-29 11:09:05 -04:00
Amiti Uttarwar
e257cf71c8 [net processing] ignore unknown INV types in GETDATA messages
Co-Authored-By: John Newbery <john@johnnewbery.com>
2020-04-29 10:54:55 -04:00
Amiti Uttarwar
047ceac142 [net processing] ignore tx GETDATA from blocks-only peers
Co-Authored-By: John Newbery <john@johnnewbery.com>
2020-04-29 10:54:48 -04:00
Harris
66fe7b1a98 test: added test for upgradewallet RPC 2020-04-29 16:11:49 +02:00
MarcoFalke
af2ec6b037 Merge #18759: bench: Start nodes with -nodebuglogfile
fabe44e815 bench: Start nodes with -nodebuglogfile (MarcoFalke)

Pull request description:

  For benchmarking we don't want to depend on the speed of the disk or the amount of debug logging

ACKs for top commit:
  fanquake:
    ACK fabe44e815 - This makes some of these benchmarks significantly faster to run. MempoolEviction total runtime is down from ~46s to 11s on my machine:

Tree-SHA512: d99700901650325896b9115d20b84a27042152f46266f595bf7ea1414528c0b346f4e707a12ee8b8ba99c35cf155e645e67971c1b2a679c4e609c400ff8b08ae
2020-04-29 08:30:15 -04:00
MarcoFalke
ecca2ea1d5 Merge #18785: Prevent valgrind false positive in rest_blockhash_by_height
fcb7261625 Prevent valgrind false positive in rest_blockhash_by_height (Russell Yanofsky)

Pull request description:

  A bad interaction between valgrind and clang 6.0.0-1ubuntu2 with -O2 optimizations makes valgrind misleadingly imply C++ code is reading an uninitialized blockheight value in `rest_blockhash_by_height` just because that's what clang optimized code is doing. The C++ code looks like:

  ```c++
  int32_t blockheight;
  if (!ParseInt32(height_str, &blockheight) || blockheight < 0) {
  ```

  while the optimized code looks like:

  ```
  0x00000000000f97ab <+123>:   callq  0x4f8860 <ParseInt32(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int*)>
  0x00000000000f97b0 <+128>:   mov    0xc(%rsp),%ebx
  0x00000000000f97b4 <+132>:   test   %ebx,%ebx
  0x00000000000f97b6 <+134>:   js     0xf98aa <rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+378>
  0x00000000000f97bc <+140>:   xor    $0x1,%al
  0x00000000000f97be <+142>:   jne    0xf98aa <rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+378>
  ```

  During the rest_interface.py test:

  eef90c14ed/test/functional/interface_rest.py (L266)

  when `height_str` is empty, `ParseInt32` returns false and `blockheight` value is never assigned. The optimized code reads the uninitialized `blockheight` value in `0xc(%rsp)` before the checking the `ParseInt32` return value in `%al`, which is harmless, but triggers the following error from valgrind:

  ```
  ==30660== Thread 13 b-httpworker.2:
  ==30660== Conditional jump or move depends on uninitialised value(s)
  ==30660==    at 0x2017B6: rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (rest.cpp:614)
  ==30660==    by 0x2041B9: operator() (rest.cpp:670)
  ==30660==    by 0x2041B9: std::_Function_handler<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), StartREST(util::Ref const&)::$_1>::_M_invoke(std::_Any_data const&, HTTPRequest*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (std_function.h:301)
  ==30660==    by 0x3EC994: operator() (std_function.h:706)
  ==30660==    by 0x3EC994: HTTPWorkItem::operator()() (httpserver.cpp:55)
  ==30660==    by 0x3ED16D: WorkQueue<HTTPClosure>::Run() (httpserver.cpp:114)
  ==30660==    by 0x3E9168: HTTPWorkQueueRun(WorkQueue<HTTPClosure>*, int) (httpserver.cpp:342)
  ==30660==    by 0x3EDAAA: __invoke_impl<void, void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int> (invoke.h:60)
  ==30660==    by 0x3EDAAA: __invoke<void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int> (invoke.h:95)
  ==30660==    by 0x3EDAAA: _M_invoke<0, 1, 2> (thread:234)
  ==30660==    by 0x3EDAAA: operator() (thread:243)
  ==30660==    by 0x3EDAAA: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(WorkQueue<HTTPClosure>*, int), WorkQueue<HTTPClosure>*, int> > >::_M_run() (thread:186)
  ==30660==    by 0x64256DE: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
  ==30660==    by 0x54876DA: start_thread (pthread_create.c:463)
  ==30660==    by 0x6DC888E: clone (clone.S:95)
  ==30660==  Uninitialised value was created by a stack allocation
  ==30660==    at 0x20173A: rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (rest.cpp:608)
  ==30660==
  {
     <insert_a_suppression_name_here>
     Memcheck:Cond
     fun:_ZL24rest_blockhash_by_heightRKN4util3RefEP11HTTPRequestRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
     fun:operator()
     fun:_ZNSt17_Function_handlerIFbP11HTTPRequestRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEZ9StartRESTRKN4util3RefEE3$_1E9_M_invokeERKSt9_Any_dataOS1_S9_
     fun:operator()
     fun:_ZN12HTTPWorkItemclEv
     fun:_ZN9WorkQueueI11HTTPClosureE3RunEv
     fun:_ZL16HTTPWorkQueueRunP9WorkQueueI11HTTPClosureEi
     fun:__invoke_impl<void, void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int>
     fun:__invoke<void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int>
     fun:_M_invoke<0, 1, 2>
     fun:operator()
     fun:_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJPFvP9WorkQueueI11HTTPClosureEiES6_iEEEEE6_M_runEv
     obj:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
     fun:start_thread
     fun:clone
  }
  ```

  This is a known bad interaction between clang and valgrind. The clang optimized code is correct but valgrind has no way of knowing that accessing the uninitialized value isn't a problem. Issue has been reported previously:

  - https://bugs.llvm.org/show_bug.cgi?id=32604#c4
  - https://github.com/Z3Prover/z3/issues/972

  This commit just sets blockheight to -1 as a workaround.

  This change was originally made in 41d5d651594c6c939add7a58b7e30c97dccdf24a from #18740 to fix the travis error there (https://travis-ci.org/github/bitcoin/bitcoin/jobs/678453061#L7157) but MarcoFalke suggested https://github.com/bitcoin/bitcoin/pull/18740#discussion_r414772851 moving to a new PR, since apparently the error's been seen on travis previously

ACKs for top commit:
  MarcoFalke:
    ACK fcb7261625
  practicalswift:
    ACK fcb7261625

Tree-SHA512: ec8abf45bd3d6c6e0e7e404d0b2a749efd43910619b84b0b5fe7dab22881598d1011a0f3ff2e146bf46320b63eb152bf63c62c06f1ab84c35dd640abc468f18f
2020-04-29 08:23:06 -04:00
João Barbosa
a2e6db5c4f rpc: Add mutex to guard deadlineTimers 2020-04-29 11:47:57 +01:00
fanquake
0ef0d33f75 Merge #18038: P2P: Mempool tracks locally submitted transactions to improve wallet privacy
50fc4df6c4 [mempool] Persist unbroadcast set to mempool.dat (Amiti Uttarwar)
297a178536 [test] Integration tests for unbroadcast functionality (Amiti Uttarwar)
6851502472 [refactor/test] Extract P2PTxInvStore into test framework (Amiti Uttarwar)
dc1da48dc5 [wallet] Update the rebroadcast frequency to be ~1/day. (Amiti Uttarwar)
e25e42f20a [p2p] Reattempt initial send of unbroadcast transactions (Amiti Uttarwar)
7e93eecce3 [util] Add method that returns random time in milliseconds (Amiti Uttarwar)
89eeb4a333 [mempool] Track "unbroadcast" transactions (Amiti Uttarwar)

Pull request description:

  This PR introduces mempool tracking of unbroadcast transactions and periodic reattempts at initial broadcast. This is a part of the rebroadcast project, and a standalone privacy win.

  The current rebroadcast logic is terrible for privacy because 1. only the source wallet rebroadcasts transactions and 2. it does so quite frequently. In the current system, if a user submits a transaction that does not immediately get broadcast to the network (eg. they are offline), this "rebroadcast" behavior is the safety net that can actually serve as the initial broadcast. So, keeping the attempts frequent is important for initial delivery within a reasonable timespan.

  This PR aims to improve # 2 by reducing the wallet rebroadcast frequency to ~1/day from ~1/15 min. It achieves this by separating the notion of initial broadcast from rebroadcasts. With these changes, the mempool tracks locally submitted transactions & periodically reattempts initial broadcast. Transactions submitted via the wallet or RPC are added to an "unbroadcast" set & are removed when a peer sends a `getdata` request, or the transaction is removed from the mempool. Every 10-15 minutes, the node reattempts an initial broadcast. This enables reducing the wallet rebroadcast frequency while ensuring the transactions will be propagated to the network.

  For privacy improvements around # 1, please see #16698.
  Thank you to gmaxwell for the idea of how to break out this subset of functionality (https://github.com/bitcoin/bitcoin/pull/16698#issuecomment-571399346)

ACKs for top commit:
  fjahr:
    Code review ACK 50fc4df6c4
  MarcoFalke:
    ACK 50fc4df6c4, I think this is ready for merge now 👻
  amitiuttarwar:
    The current tip `50fc4df` currently has 6 ACKs on it, so I've opened #18807 to address the last bits.
  jnewbery:
    utACK 50fc4df6c4.
  ariard:
    Code Review ACK 50fc4df (minor points no need to invalid other ACKs)
  robot-visions:
    ACK 50fc4df6c4
  sipa:
    utACK 50fc4df6c4
  naumenkogs:
    utACK 50fc4df

Tree-SHA512: 2dd935d645d5e209f8abf87bfaa3ef0e4492705ce7e89ea64279cb27ffd37f4727fa94ad62d41be331177332f8edbebf3c7f4972f8cda10dd951b80a28ab3c0f
2020-04-29 16:32:37 +08:00
Jeremy Rubin
5495fa5850 Add Hash Padding Microbenchmarks 2020-04-29 00:31:41 -07:00
fanquake
692f8307fc test: add test for witness commitment index
As per BIP 141, if there is more than 1 pubkey that matches the witness
commitment structure, the one with the highest output index should be
chosen. This adds a sanity check that we are doing that, which will fail
if anyone trys to "optimise" GetWitnessCommitmentIndex() be returning
early.
2020-04-29 11:20:31 +08:00
fanquake
06442549f8 validation: Add minimum witness commitment size constant
Per BIP 141, the witness commitment structure is atleast 38 bytes,
OP_RETURN (0x6a) + 36 (0x24) + 4 byte header (0xaa21a9ed) + 32 byte
SHA256 hash. It can be longer, however any additional data has no
consensus meaning.
2020-04-29 11:20:22 +08:00
Chris Abrams
ff6549c3c8 fix: update rest info on block size and json 2020-04-28 20:17:03 -05:00
MarcoFalke
fac0cf6e55 rpc: Do not advertise dumptxoutset as a way to flush the chainstate 2020-04-28 20:40:47 -04:00
Hennadii Stepanov
1e06bb68be Drop unused CLIENT_VERSION_SUFFIX macro 2020-04-28 23:10:58 +03:00
Sebastian Falbesoner
1ad8ea2b73 net: remove is{Empty,Full} flags from CBloomFilter, clarify CVE fix 2020-04-28 19:27:22 +02:00
MarcoFalke
ba348dbc51 Merge #18805: tests: Add missing sync_all to wallet_importdescriptors.py
7b2b06dfe3 tests: Add missing sync_all to wallet_importdescriptors.py (Andrew Chow)

Pull request description:

  node1 will sometimes do sendtoaddress before it has received a funding transaction which will cause the test to fail. sync_all to ensure it gets the transaction first.

  Fixes #18800

ACKs for top commit:
  instagibbs:
    ACK 7b2b06d The wallet endpoint right after is indeed node 1.

Tree-SHA512: b610a771d062b5f955cd70b34337577a1ab8dacbf4be20aa74e1e8234495b0be9faff138eb1713f29decb5574446e0583e221bc2c9a6eea13611b422ea3a296a
2020-04-28 13:14:26 -04:00
MarcoFalke
6a60bfc76c Merge #18765: test: Fix wallet_bumpfee intermittent error
fa301fec96 test: Fix wallet_bumpfee intermittent error (MarcoFalke)

Pull request description:

  Remove incorrect and undocumented `connect_nodes(self.nodes[0], 1)`.

  Issue is that transactions are re-relayed (going full circle) between the two nodes, that have two connections between each other.

  https://travis-ci.org/github/bitcoin/bitcoin/jobs/679201559#L6992

  Also fix some pep8 while touching the file

  This bug has been introduced by accident in c1dde3a949

ACKs for top commit:
  achow101:
    ACK fa301fec96

Tree-SHA512: a6565ca30dbe44b02e3f58f159d2515c2ea4a74030118fafc1a3391ce980a4b6d4505dcf51315fda24843f72550a7dea7407b877b3b796883dd73d3b6f009e6f
2020-04-28 13:11:57 -04:00
Andrew Chow
7b2b06dfe3 tests: Add missing sync_all to wallet_importdescriptors.py
node1 will sometimes do sendtoaddress before it has received a funding
transaction which will cause the test to fail. sync_all to ensure it
gets the transaction first.
2020-04-28 12:08:33 -04:00
MarcoFalke
fa301fec96 test: Fix wallet_bumpfee intermittent error 2020-04-28 11:53:00 -04:00
MarcoFalke
a483ffd689 Merge #18735: ci: Add and document BASE_BUILD_DIR
fae49f6e42 ci: Add and document BASE_BUILD_DIR (MarcoFalke)

Pull request description:

  Also fixes #18768

ACKs for top commit:
  hebasto:
    re-ACK fae49f6e42, which is essentially the same as the previously [reviewed changes](https://github.com/bitcoin/bitcoin/pull/18735#pullrequestreview-400581536).

Tree-SHA512: 216565a05ccd513dd9f114b2333d3c283fd71914d32f9b05f145cb7c70633b083ff8ef60798d6f22f4be6a4d652b03806551fd74b5b596c92968501a4d9726d2
2020-04-28 11:50:21 -04:00
Carl Dong
fa791da02f nsis: Specify OutFile path only once
Previously, we would specify the makensis output file path twice:

  1. At the top of Makefile.am as BITCOIN_WIN_INSTALLER, and
  2. In share/setup.nsi.ini

This commit uses the -X flag of makensis to eliminate the need for the
second instance mentioned above, referring makensis directly to the
value of BITCOIN_WIN_INSTALLER
2020-04-28 10:36:38 -04:00
Carl Dong
14701604d0 guix: Expose GIT_COMMON_DIR in container as readonly
When using worktrees or submodules, you'll see a `.git' plain text file
at the root of your working tree instead of the usual `.git' directory.

This plain text file will point to the real GIT_DIR, under the
GIT_COMMON_DIR. From experimentation, the full GIT_COMMON_DIR is
required to exist for operations such as git-archive(1), so we expose it
as readonly inside the container.
2020-04-28 10:36:37 -04:00
Carl Dong
f5a6ac4f48 guix: Make source tarball using git-archive 2020-04-28 10:36:36 -04:00
Carl Dong
395c1137f6 gitian: Limit sourced script to just assignments
Previously, the sourced script would create the source tarball. Now, it
only assigns variables and the source-ing script has more flexibility in
determining what to do with these variables.

See later commit showing how this flexibility is useful in our Guix
builds.
2020-04-28 10:36:35 -04:00
MarcoFalke
6e3fc7492a Merge #18663: doc: mention build docs in README.md
bda62e87e6 Adding build instructions to Bitcoin Core, fixes #18658 (Saahil Shangle)

Pull request description:

  Making the instructions for building Bitcoin Core more clear in the main `README.md` will reduce confusion between the `build_msvc` and `doc` folders.

ACKs for top commit:
  laanwj:
    ACK bda62e87e6

Tree-SHA512: ee4c394661eba48d4229e3d1e9ddb67ccb79589429bfa9986cb0242cd615d1f3cc5332063562c1e89c0cdd9ae2e609f61e8bfb209926d8363d35d3da6d94ae9c
2020-04-28 08:51:57 -04:00
MarcoFalke
ad04130742 Merge #18799: ci: Document why tests can not be run on mac
fa72a75102 ci: Document why tests can not be run on mac (MarcoFalke)

Pull request description:

  Fixes #18794

Top commit has no ACKs.

Tree-SHA512: 297652eda412aa8cf7255e20a6f294d22773dad8637a3d7b5204f3b638e911ce5b2e40e85f81395a34c1b5a5b497665944c2d6ea17c70c30c0c9e0ab553f956e
2020-04-28 08:25:51 -04:00
MarcoFalke
fa72a75102 ci: Document why tests can not be run on mac 2020-04-28 07:10:19 -04:00
fanquake
9fb95ae8e3 Merge #18779: doc: Better explain GNU ld's dislike of ld64's options
cd24f37ea9 doc: Better explain GNU ld's dislike of ld64's options (fanquake)

Pull request description:

  There's also now more than a single option being special cased for
  darwin. If we didn't special case these options they would still end
  up on the link line and the binaries produced would just segfault.

  I'm going to plug #17874 here as well, because adding
  `-fatal-warnings` to our `AX_CHECK_LINK_FLAG` calls would
  mostly prevent this sort of option mangling from happening.

  An example of the warning behaviour:
  ```bash
  echo "int main() {}" | g++ -x c++ -std=c++11 -Wl,-dead_strip -
  /usr/bin/ld: warning: cannot find entry symbol ad_strip; defaulting to 0000000000001040

  nm -C a.out
  0000000000001000 t _init
  0000000000001040 T _start
                   U ad_strip
  ```

ACKs for top commit:
  dongcarl:
    ACK cd24f37ea9

Tree-SHA512: 8c5ff11b647e7d44dbb3f509a07caf8606a6b481c114403f0de72b3ad65395dbe9a3436e731ae1b46a823431ed23c3c6aacab8942d78629d59cd8c258c5dbf02
2020-04-28 17:34:22 +08:00
fanquake
b9ba76f1c9 Merge #18769: qt: remove bug fix for Qt < 5.5
e3ec4924a7 qt: remove todo bug fix for old versions of Qt (10xcryptodev)

Pull request description:

  Remove the code used to fix a Qt bug in versions before Qt 5.5.0 as described in this link https://bugreports.qt.io/browse/QTBUG-43473

  Now the minimum requirement is Qt 5.5.1 as described in https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md

  This code is not necessary anymore

ACKs for top commit:
  hebasto:
    re-ACK e3ec4924a7, since the [previous review](https://github.com/bitcoin/bitcoin/pull/18769#pullrequestreview-400517155) only the prefix of the commit message has been changed.

Tree-SHA512: 57802974fccae863dde0e186847db09832b2433b11e8410a0137b27f1ae8a95bdcd9206a5ea0d79f7a2b56adc6b4bac8bb0c4db583158db36a349a6b28b81aac
2020-04-28 17:15:41 +08:00
fanquake
65fb3dfc8d Merge #18556: build: Drop make dist in gitian builds
2aa48edec0 refactor: Drop unused ${WRAP_DIR}/${HOST} directory (Hennadii Stepanov)
1362be0447 build: Drop make dist in gitian builds (Hennadii Stepanov)

Pull request description:

  After the merge of #18331, the packaged source tarball is created by `git archive`, but the binaries are built from another one which is made by `make dist`.

  With this PR the only source tarball, created by `git archive`, is used both for binaries building and for packaging to users.

  Close #16588.
  Close #18547.

  As a good side-effect, #18349 becomes redundant.

  **Change in behavior**

  The following variables 1b151e3ffc/configure.ac (L2-L6)

  are no longer used for naming of directories and tarballs.

  Instead of them the gitian descriptors use a git tag (if available) or a commit hash.

  ---

  Also a small refactor commit picked from #18404.

ACKs for top commit:
  dongcarl:
    ACK 2aa48edec0
  MarcoFalke:
    ACK 2aa48edec0
  fanquake:
    ACK 2aa48edec0 - I've had a quick look over this, and don't want to block merging if this actually gets as closer to finally having this all sorted out. Obviously we've still got #18741, and after speaking to Carl this morning, there will likely be even more changes after that (not Guix specific).

Tree-SHA512: d3b16f87e48d1790a3264940c28acd5d881bfd10f3ce94fb0c8a6af76d8039289d01e0cd4972adac49ae24362857251f6c1e5e09e3e9fbf636c10708b4015a7c
2020-04-28 16:44:17 +08:00
Hennadii Stepanov
45615de26c ci: Fix default retry script usage 2020-04-28 10:52:31 +03:00
fanquake
ac21090f20 Merge #18629: scripts: add PE .reloc section check to security-check.py
3e38023af7 scripts: add PE .reloc section check to security-check.py (fanquake)

Pull request description:

  The `ld` in binutils has historically had a few issues with PE binaries, there's a good summary in this [thread](https://sourceware.org/bugzilla/show_bug.cgi?id=19011).

  One issue in particular was `ld` stripping the `.reloc` section out of PE binaries, even though it's required for functioning ASLR. This was [reported by a Tor developer in 2014](https://sourceware.org/bugzilla/show_bug.cgi?id=17321) and they have been patching their [own binutils](https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/binutils) ever since. However their patch only made it into binutils at the [start of this year](https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=dc9bd8c92af67947db44b3cb428c050259b15cd0). It adds an `--enable-reloc-section` flag, which is turned on by default if you are using `--dynamic-base`. In the mean time this issue has also been worked around by other projects, such as FFmpeg, see [this commit](91b668acd6).

  I have checked our recent supported Windows release binaries, and they do contain a `.reloc` section. From what I understand, we are using all the right compile/linker flags, including `-pie` & `-fPIE`, and have never run into the crashing/entrypoint issues that other projects might have seen.

  One other thing worth noting here, it how Debian/Ubuntu patch the binutils that they distribute, because that's what we end up using in our gitian builds.

  In the binutils-mingw-w64 in Bionic (18.04), which we currently use in gitian, PE hardening options/security flags are enabled by default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8ubuntu1/changelog) and the [relevant commit](452b3013b8).

  However in Focal (20.04), this has now been reversed. PE hardening options are no-longer the default. See the [changelog](https://changelogs.ubuntu.com/changelogs/pool/universe/b/binutils-mingw-w64/binutils-mingw-w64_8.8/changelog) and [relevant commit](7bd8b2fbc2), which cites same .reloc issue mentioned here.

  Given that we explicitly specify/opt-in to everything that we want to use, the defaults aren't necessarily an issue for us. However I think it highlights the importance of continuing to be explicit about what we want, and not falling-back or relying on upstream.

  This was also prompted by the possibility of us doing link time garbage collection, see #18579 & #18605. It seemed some sanity checks would be worthwhile in-case the linker goes haywire while garbage collecting.

  I think Guix is going to bring great benefits when dealing with these kinds of issues. Carl you might have something to say in that regard.

ACKs for top commit:
  dongcarl:
    ACK 3e38023af7

Tree-SHA512: af14d63bdb334bde548dd7de3e0946556b7e2598d817b56eb4e75b3f56c705c26aa85dd9783134c4b6a7aeb7cb4de567eed996e94d533d31511f57ed332287da
2020-04-28 13:32:45 +08:00
MarcoFalke
5352d14b37 Merge #18778: ci: Run functional tests on mac again
fac24dea00 ci: Run functional tests on mac again (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fac24dea00, verified travis build log.

Tree-SHA512: 406282a7ac03e5c193830b727366c7b1350639f1850aff951bf7ddd4b0c3e3ffb396b950ccb3a64ddc59500fa2739766f3c34806b4d144bc4535bb2bd765b959
2020-04-27 20:36:39 -04:00
MarcoFalke
8bdb2134fc Merge #18777: wallet: Recommend absolute path for dumpwallet
fa501700e9 wallet: Recommned absolute path for dumpwallet (MarcoFalke)

Pull request description:

  Avoids misunderstandings such as #9564

ACKs for top commit:
  kristapsk:
    utACK fa501700e9

Tree-SHA512: f675ef607992857ffeb556a2945b5436a70b39c5d83f05a8be15a6fccc84cbe9d03e52f8239e28d159e41ed7c6f119b7a38e8ab327029f04609f63c559c12c49
2020-04-27 18:02:52 -04:00
MarcoFalke
faec3dc2ad init: Remove boost from ThreadImport 2020-04-27 15:35:26 -04:00
practicalswift
38e49ded8b tests: Add fuzzing harness for MessageSign, MessageVerify and other functions in util/message.h 2020-04-27 17:06:59 +00:00
MarcoFalke
fae49f6e42 ci: Add and document BASE_BUILD_DIR
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2020-04-27 12:27:34 -04:00
practicalswift
2a78098098 wallet: Make sure no WalletDescriptor members are uninitialized after construction 2020-04-27 14:20:26 +00:00
practicalswift
ff046aeeba wallet: Make sure no DescriptorScriptPubKeyMan members are uninitialized after construction 2020-04-27 14:20:00 +00:00
fanquake
cd24f37ea9 doc: Better explain GNU ld's dislike of ld64's options
There's also now more than a single option being special cased for
darwin.
2020-04-27 11:08:51 +08:00
MarcoFalke
fac24dea00 ci: Run functional tests on mac again 2020-04-26 21:03:16 -04:00
Samuel Dobson
eef90c14ed Merge #16528: Native Descriptor Wallets using DescriptorScriptPubKeyMan
223588b1bb Add a --descriptors option to various tests (Andrew Chow)
869f7ab30a tests: Add RPCOverloadWrapper which overloads some disabled RPCs (Andrew Chow)
cf06062859 Correctly check for default wallet (Andrew Chow)
886e0d75f5 Implement CWallet::IsSpentKey for non-LegacySPKMans (Andrew Chow)
3c19fdd2a2 Return error when no ScriptPubKeyMan is available for specified type (Andrew Chow)
388ba94231 Change wallet_encryption.py to use signmessage instead of dumpprivkey (Andrew Chow)
1346e14831 Functional tests for descriptor wallets (Andrew Chow)
f193ea889d add importdescriptors RPC and tests for native descriptor wallets (Hugo Nguyen)
ce24a94494 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly (Andrew Chow)
1cb42b22b1 Generate new descriptors when encrypting (Andrew Chow)
82ae02b165 Be able to create new wallets with DescriptorScriptPubKeyMans as backing (Andrew Chow)
b713baa75a Implement GetMetadata in DescriptorScriptPubKeyMan (Andrew Chow)
8b9603bd0b Change GetMetadata to use unique_ptr<CKeyMetadata> (Andrew Chow)
72a9540df9 Implement FillPSBT in DescriptorScriptPubKeyMan (Andrew Chow)
84b4978c02 Implement SignMessage for descriptor wallets (Andrew Chow)
bde7c9fa38 Implement SignTransaction in DescriptorScriptPubKeyMan (Andrew Chow)
d50c8ddd41 Implement GetSolvingProvider for DescriptorScriptPubKeyMan (Andrew Chow)
f1ca5feb4a Implement GetKeypoolOldestTime and only display it if greater than 0 (Andrew Chow)
586b57a9a6 Implement ReturnDestination in DescriptorScriptPubKeyMan (Andrew Chow)
f866957979 Implement GetReservedDestination in DescriptorScriptPubKeyMan (Andrew Chow)
a775f7c7fd Implement Unlock and Encrypt in DescriptorScriptPubKeyMan (Andrew Chow)
bfdd073486 Implement GetNewDestination for DescriptorScriptPubKeyMan (Andrew Chow)
58c7651821 Implement TopUp in DescriptorScriptPubKeyMan (Andrew Chow)
e014886a34 Implement SetupGeneration for DescriptorScriptPubKeyMan (Andrew Chow)
46dfb99768 Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file (Andrew Chow)
4cb9b69be0 Implement several simple functions in DescriptorScriptPubKeyMan (Andrew Chow)
d1ec3e4f19 Add IsSingleType to Descriptors (Andrew Chow)
953feb3d27 Implement loading of keys for DescriptorScriptPubKeyMan (Andrew Chow)
2363e9fcaa Load the descriptor cache from the wallet file (Andrew Chow)
46c46aebb7 Implement GetID for DescriptorScriptPubKeyMan (Andrew Chow)
ec2f9e1178 Implement IsHDEnabled in DescriptorScriptPubKeyMan (Andrew Chow)
741122d4c1 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan (Andrew Chow)
2db7ca765c Implement IsMine for DescriptorScriptPubKeyMan (Andrew Chow)
db7177af8c Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet (Andrew Chow)
78f8a92910 Implement SetType in DescriptorScriptPubKeyMan (Andrew Chow)
834de0300c Store WalletDescriptor in DescriptorScriptPubKeyMan (Andrew Chow)
d8132669e1 Add a lock cs_desc_man for DescriptorScriptPubKeyMan (Andrew Chow)
3194a7f88a Introduce WalletDescriptor class (Andrew Chow)
6b13cd3fa8 Create LegacyScriptPubKeyMan when not a descriptor wallet (Andrew Chow)
aeac157c9d Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet (Andrew Chow)
96accc73f0 Add WALLET_FLAG_DESCRIPTORS (Andrew Chow)
6b8119af53 Introduce DescriptorScriptPubKeyMan as a dummy class (Andrew Chow)
06620302c7 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it (Andrew Chow)

Pull request description:

  Introducing the wallet of the glorious future (again): native descriptor wallets. With native descriptor wallets, addresses are generated from descriptors. Instead of generating keys and deriving addresses from keys, addresses come from the scriptPubKeys produced by a descriptor. Native descriptor wallets will be optional for now and can only be created by using `createwallet`.

  Descriptor wallets will store descriptors, master keys from the descriptor, and descriptor cache entries. Keys are derived from descriptors on the fly. In order to allow choosing different address types, 6 descriptors are needed for normal use. There is a pair of primary and change descriptors for each of the 3 address types. With the default keypool size of 1000, each descriptor has 1000 scriptPubKeys and descriptor cache entries pregenerated. This has a side effect of making wallets large since 6000 pubkeys are written to the wallet by default, instead of the current 2000. scriptPubKeys are kept only in memory and are generated every time a descriptor is loaded. By default, we use the standard BIP 44, 49, 84 derivation paths with an external and internal derivation chain for each.

  Descriptors can also be imported with a new `importdescriptors` RPC.

  Native descriptor wallets use the `ScriptPubKeyMan` interface introduced in #16341 to add a `DescriptorScriptPubKeyMan`. This defines a different IsMine which uses the simpler model of "does this scriptPubKey exist in this wallet". Furthermore, `DescriptorScriptPubKeyMan` does not have watchonly, so with native descriptor wallets, it is not possible to have a wallet with both watchonly and non-watchonly things. Rather a wallet with `disable_private_keys` needs to be used for watchonly things.

  A `--descriptor` option was added to some tests (`wallet_basic.py`, `wallet_encryption.py`, `wallet_keypool.py`, `wallet_keypool_topup.py`, and `wallet_labels.py`) to allow for these tests to use descriptor wallets. Additionally, several RPCs are disabled for descriptor wallets (`importprivkey`, `importpubkey`, `importaddress`, `importmulti`, `addmultisigaddress`, `dumpprivkey`, `dumpwallet`, `importwallet`, and `sethdseed`).

ACKs for top commit:
  Sjors:
    utACK 223588b1bb (rebased, nits addressed)
  jonatack:
    Code review re-ACK 223588b1bb.
  fjahr:
    re-ACK 223588b1bb
  instagibbs:
    light re-ACK 223588b
  meshcollider:
    Code review ACK 223588b1bb

Tree-SHA512: 59bc52aeddbb769ed5f420d5d240d8137847ac821b588eb616b34461253510c1717d6a70bab8765631738747336ae06f45ba39603ccd17f483843e5ed9a90986
2020-04-27 12:23:05 +12:00
Russell Yanofsky
7918c1b019 test: Add CreateWalletFromFile test
Add unit test calling CreateWalletFromFile, which isn't currently called from
other unit tests, with some basic checks to make sure it rescans and registers
for notifications correctly.

Motivation for this change was to try to write a test that would fail without
the early `handleNotifications` call in ef8c6ca60767cac589d98ca57ee33179608ccda8
from https://github.com/bitcoin/bitcoin/pull/16426, but succeed with it:

ef8c6ca607/src/wallet/wallet.cpp (L3978-L3986)

However, writing a full test for the race condition that call prevents isn't
possible without the locking changes from #16426. So this PR just adds as much
test coverage as is possible now.

This new test is also useful for https://github.com/bitcoin/bitcoin/pull/15719,
since it detects the stale notifications.transactionAddedToMempool notifications
that PR eliminates.
2020-04-26 20:23:05 -04:00
Russell Yanofsky
fcb7261625 Prevent valgrind false positive in rest_blockhash_by_height
A bad interaction between valgrind and clang 6.0.0-1ubuntu2 with -O2
optimizations makes valgrind misleadingly imply C++ code is reading an
uninitialized blockheight value in rest_blockhash_by_height just because that's
what clang optimized code is doing. The C++ code looks like:

    int32_t blockheight;
    if (!ParseInt32(height_str, &blockheight) || blockheight < 0) {

while the optimized code looks like:

    0x00000000000f97ab <+123>:   callq  0x4f8860 <ParseInt32(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, int*)>
    0x00000000000f97b0 <+128>:   mov    0xc(%rsp),%ebx
    0x00000000000f97b4 <+132>:   test   %ebx,%ebx
    0x00000000000f97b6 <+134>:   js     0xf98aa <rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+378>
    0x00000000000f97bc <+140>:   xor    $0x1,%al
    0x00000000000f97be <+142>:   jne    0xf98aa <rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+378>

During the rest_interface.py test:

   self.test_rest_request("/blockhashbyheight/", ret_type=RetType.OBJ, status=400)

when height_str is empty, ParseInt32 returns false and blockheight value is
never assigned. The optimized code reads the uninitialized blockheight value
in 0xc(%rsp) before the checking the ParseInt32 return value in %al, which is
harmless, but triggers the following error from valgrind:

==30660== Thread 13 b-httpworker.2:
==30660== Conditional jump or move depends on uninitialised value(s)
==30660==    at 0x2017B6: rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (rest.cpp:614)
==30660==    by 0x2041B9: operator() (rest.cpp:670)
==30660==    by 0x2041B9: std::_Function_handler<bool (HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), StartREST(util::Ref const&)::$_1>::_M_invoke(std::_Any_data const&, HTTPRequest*&&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (std_function.h:301)
==30660==    by 0x3EC994: operator() (std_function.h:706)
==30660==    by 0x3EC994: HTTPWorkItem::operator()() (httpserver.cpp:55)
==30660==    by 0x3ED16D: WorkQueue<HTTPClosure>::Run() (httpserver.cpp:114)
==30660==    by 0x3E9168: HTTPWorkQueueRun(WorkQueue<HTTPClosure>*, int) (httpserver.cpp:342)
==30660==    by 0x3EDAAA: __invoke_impl<void, void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int> (invoke.h:60)
==30660==    by 0x3EDAAA: __invoke<void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int> (invoke.h:95)
==30660==    by 0x3EDAAA: _M_invoke<0, 1, 2> (thread:234)
==30660==    by 0x3EDAAA: operator() (thread:243)
==30660==    by 0x3EDAAA: std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(WorkQueue<HTTPClosure>*, int), WorkQueue<HTTPClosure>*, int> > >::_M_run() (thread:186)
==30660==    by 0x64256DE: ??? (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
==30660==    by 0x54876DA: start_thread (pthread_create.c:463)
==30660==    by 0x6DC888E: clone (clone.S:95)
==30660==  Uninitialised value was created by a stack allocation
==30660==    at 0x20173A: rest_blockhash_by_height(util::Ref const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (rest.cpp:608)
==30660==
{
   <insert_a_suppression_name_here>
   Memcheck:Cond
   fun:_ZL24rest_blockhash_by_heightRKN4util3RefEP11HTTPRequestRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
   fun:operator()
   fun:_ZNSt17_Function_handlerIFbP11HTTPRequestRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEZ9StartRESTRKN4util3RefEE3$_1E9_M_invokeERKSt9_Any_dataOS1_S9_
   fun:operator()
   fun:_ZN12HTTPWorkItemclEv
   fun:_ZN9WorkQueueI11HTTPClosureE3RunEv
   fun:_ZL16HTTPWorkQueueRunP9WorkQueueI11HTTPClosureEi
   fun:__invoke_impl<void, void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int>
   fun:__invoke<void (*)(WorkQueue<HTTPClosure> *, int), WorkQueue<HTTPClosure> *, int>
   fun:_M_invoke<0, 1, 2>
   fun:operator()
   fun:_ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJPFvP9WorkQueueI11HTTPClosureEiES6_iEEEEE6_M_runEv
   obj:/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25
   fun:start_thread
   fun:clone
}

This is a known bad interaction between clang and valgrind. The clang optimized
code is correct but valgrind has no way of knowing that accessing the
uninitialized value isn't a problem. Issue has been reported previously:

    https://bugs.llvm.org/show_bug.cgi?id=32604#c4
    https://github.com/Z3Prover/z3/issues/972

This commit just sets blockheight to 0 as a workaround.
2020-04-26 20:23:05 -04:00
MarcoFalke
fa501700e9 wallet: Recommned absolute path for dumpwallet 2020-04-26 20:22:42 -04:00
MarcoFalke
ae32e5ce3d Merge #18669: log: Use Join() helper when listing log categories
faec063887 log: Use Join() helper when listing log categories (MarcoFalke)

Pull request description:

  This removes the global `ListLogCategories` and replaces it with a one-line member function `LogCategoriesString`, which just calls `Join`.

  Should be a straightforward refactor to get rid of a few LOC.

ACKs for top commit:
  laanwj:
    ACK faec063887
  promag:
    ACK faec063887, I also think it's fine as it is (re https://github.com/bitcoin/bitcoin/pull/18669#discussion_r412944724).

Tree-SHA512: 2f51f9ce1246eda5630015f3a869e36953c7eb34f311baad576b92d7829e4e88051c6189436271cd0a13732a49698506345b446b98fd28e58edfb5b62169f1c9
2020-04-26 19:57:41 -04:00
Gloria Zhao
de8905adf2 test: use unittest and test_runner for test framework unit testing
Test the test_framework, but don't use test_framework objects or functions to test itself

Use python unittest library and put test_framework's unit tests inside their respective files
Add the filename to TEST_FRAMEWORK_MODULES in test_runner
Aggregate all test_framework tests into one TestSuite to run before the functional tests in test_runner
Delete framework_test_script, move test_bn2vch to script.py and add to TEST_FRAMEWORK_MODULES in test_runner
2020-04-26 13:31:39 -07:00
practicalswift
32b6b386a5 tests: Sort fuzzing harnesses 2020-04-26 20:25:40 +00:00
practicalswift
e1e181fad1 tests: Add fuzzing coverage for JSONRPCTransactionError(...) and RPCErrorFromTransactionError(...) 2020-04-26 20:23:56 +00:00
practicalswift
103b6ecce0 tests: Add fuzzing coverage for TransactionErrorString(...) 2020-04-26 20:23:56 +00:00
practicalswift
dde508b8b0 tests: Add fuzzing coverage for ParseFixedPoint(...) 2020-04-26 20:23:56 +00:00
practicalswift
1532259fca tests: Add fuzzing coverage for FormatHDKeypath(...) and WriteHDKeypath(...) 2020-04-26 20:23:56 +00:00
practicalswift
90b635e84e tests: Add fuzzing coverage for CHECK_NONFATAL(...) 2020-04-26 20:23:56 +00:00
practicalswift
a4e3d13df6 tests: Add fuzzing coverage for StringForFeeReason(...) 2020-04-26 20:23:56 +00:00
practicalswift
a19598cf98 tests: Add fuzzing harness for functions in system.h (ArgsManager) 2020-04-26 20:23:56 +00:00
10xcryptodev
e3ec4924a7 qt: remove todo bug fix for old versions of Qt 2020-04-26 13:57:46 -03:00
MarcoFalke
9ddfce6712 Merge #18753: test: Fix intermittent failure in wallet_importmulti
fa8b9b5d1f test: Fix intermittent failure in wallet_importmulti (MarcoFalke)

Pull request description:

  The wallet is async, so after generating a block, we must call `syncwithvalidationinterfacequeue`. Otherwise the timestamp will be of the previous block.

  https://travis-ci.org/github/bitcoin/bitcoin/jobs/677685073#L2648

ACKs for top commit:
  promag:
    ACK fa8b9b5d1f.

Tree-SHA512: c21f9912aabbe22019d4ac9d0da06d6e46ef7f2a84d2781110e04c9836eb0ecf90a22cf2bae7f608be611670d17b20600135d1c5e5404aa1e762839816285fb4
2020-04-26 10:47:37 -04:00
MarcoFalke
3cf464de2e Merge #18770: test: Remove raw-tx byte juggling in mempool_reorg
fa489011d9 test: Remove raw-tx byte juggling in mempool_reorg (MarcoFalke)

Pull request description:

ACKs for top commit:
  robot-visions:
    ACK fa489011d9

Tree-SHA512: 2ae537a5b34e48d2954ba02d9e050ae1a99043080266d5decca3f983b13bc3ddcbee1f3f3c7457e16ee76ea17b9ce08cac2e3205cf46b5e3c3e4ca5f758f00de
2020-04-26 07:41:01 -04:00
Samuel Dobson
e8fa0a3d20 Fix WSL file locking by using flock instead of fcntl
Co-authored-by: sipa <pieter@wuille.net>
2020-04-26 12:16:22 +12:00
MarcoFalke
fa489011d9 test: Remove raw-tx byte juggling in mempool_reorg 2020-04-25 19:36:02 -04:00
Danny Lee
cd543d9193 test: check misbehavior more independently in p2p_filter.py 2020-04-25 09:28:22 -07:00
MarcoFalke
65276c7737 Merge #18744: test: Add fuzzing harnesses for various classes/functions in primitives/
fd8e99da57 tests: Add fuzzing harness for functions in primitives/transaction.h (practicalswift)
d5a31b7cb4 tests: Add fuzzing harness for functions in primitives/block.h (practicalswift)

Pull request description:

  Add fuzzing harnesses for various classes/functions in `primitives/`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: ed54bd5b37ff5e40cfa8d3cd8c65d91a2f64fca87b6a5c3b8ddd6becd876ed172735fb53da4d00a86f318fb94517afd179e07cb28a43edf301ffe4dad703cca4
2020-04-25 09:50:12 -04:00
MarcoFalke
3e7c118d65 Merge #18688: fuzz: Run in parallel
fa66280396 fuzz: Run in parallel (MarcoFalke)

Pull request description:

  Can be reviewed with `--ignore-all-space`

ACKs for top commit:
  practicalswift:
    ACK fa66280396 -- patch looks correct and the live demo in https://github.com/bitcoin-core/qa-assets/pull/11#issuecomment-615439226 convinced me: this is good!
  fanquake:
    ACK fa66280396 - this seems sane, and clearly provides some speedup. I'd post benchmarks but can't seem to get through a full run of `./test/fuzz/test_runner.py` without hitting at least a few crashes; see #18763, 18762.

Tree-SHA512: d3b545ca90c75bed27f08fe712399d0ed1ac36b13fb289c83e5606eee8dec4c19f5f5cf91758f0a6b1606d8d6b455fbe46df2588faffe7462b185bd34dc2baaf
2020-04-25 09:28:33 -04:00
MarcoFalke
bab81f7e4d Merge #18739: doc: Document how to fuzz Bitcoin Core using Honggfuzz
bb1ec36fb1 doc: Document how to fuzz Bitcoin Core using honggfuzz (practicalswift)

Pull request description:

  Document how to fuzz Bitcoin Core using Honggfuzz.

ACKs for top commit:
  fanquake:
    ACK bb1ec36fb1 - did a couple quick runs on a severely under powered VM.

Tree-SHA512: 117944c52763a5672f988c62fecb01b85f19f3827fad5582a51464aefdaac4d9a9cd81e2118199f6ea1bb3ab0893c8459ca3d1df7f67bfcf215d5e305225f210
2020-04-25 09:26:52 -04:00
MarcoFalke
fd48e7ad99 Merge #18585: test: use zero-argument super() shortcut (Python 3.0+)
0956e46bff test: use zero-argument super() shortcut (Python 3.0+) (Sebastian Falbesoner)

Pull request description:

  This mini-PR replaces all calls to `super(...)` with arguments with the zero-argument shortcut `super()` where applicable. See [PEP 3135](https://www.python.org/dev/peps/pep-3135/#specification):

  > The new syntax:
  >
  >     super()
  >
  > is equivalent to:
  >
  >     super(__class__, <firstarg>)
  >
  > where __class__ is the class that the method was defined in, and <firstarg> is
  > the first parameter of the method (normally self for instance methods, and cls
  > for class methods).

ACKs for top commit:
  fanquake:
    ACK 0956e46bff

Tree-SHA512: 4ac66fe7ab2be2e8a514e5fcfc41dbb298f21b23ebb7b7b0310d704b0b3cef8adf287a8d80346d1ea9418998c597b4f0ff1f66148d0d806bb43db6607e0fe1cf
2020-04-25 09:25:50 -04:00
MarcoFalke
6f51f6f357 Merge #18754: bench: add CAddrMan benchmarks
a9b957740e bench: add CAddrMan benchmarks (Vasil Dimov)

Pull request description:

  The added benchmarks exercise the public methods Add(), GetAddr(),
  Select() and Good().

ACKs for top commit:
  naumenkogs:
    utACK a9b9577
  MarcoFalke:
    ACK a9b957740e

Tree-SHA512: af54b2fbd97db34faf4cc6c9bacb20d2c97d0aaddb9cf91b220bc2e09227b55345402ed17e34450745493e3a2b286c176c031cdeb477415570a757cee16b06a8
2020-04-25 08:38:39 -04:00
MarcoFalke
9fac600aba Merge #17383: Refactor: Move consts to their correct translation units
e9ea95a30d [net processing] Move all const declarations to top of net_processing.cpp (John Newbery)
507b36dd1b [validation] Move all const declarations to top of validation.h (John Newbery)
0109622b08 [validation] Move validation-only consts to validation.cpp (John Newbery)
b8580cacc7 [net processing] Move net processing consts to net_processing.cpp (John Newbery)

Pull request description:

  Following the main.cpp split, there are still some constants in the wrong places, eg net_processing constants in validation.h. Move them all to their rightful homes. At the same time, make them constexpr.

  Also move all const declarations to the top of their files, and ensure that they all have doxygen comments.

ACKs for top commit:
  practicalswift:
    ACK e9ea95a30d -- patch looks correct
  MarcoFalke:
    ACK e9ea95a30d 🚉

Tree-SHA512: 44d81da73c7be01e1d36b939789d793f297d3b94f84ea4e7ac853c621cc7054b5a05c7c9e7b83db506db44c16f344541be8f240d955694211e53a84c32b0d2c5
2020-04-25 08:36:39 -04:00
Sebastian Falbesoner
0956e46bff test: use zero-argument super() shortcut (Python 3.0+)
as defined in PEP 3135:

"The new syntax:

    super()

is equivalent to:

    super(__class__, <firstarg>)

where __class__ is the class that the method was defined in, and <firstarg> is
the first parameter of the method (normally self for instance methods, and cls
for class methods)."
2020-04-25 13:49:08 +02:00
MarcoFalke
5f19155e5b Merge #18724: test: add coverage for -rpcwallet cli option
2495110012 test: add coverage for -rpcwallet cli option (Jon Atack)

Pull request description:

  The bitcoin-cli `-rpcwallet=` option is an essential RPC/CLI option when more than one wallet is loaded (see `bitcoin-cli -help | grep -A5 rpcwallet` or `src/bitcoin-cli.cpp::L61`) and it currently has no test coverage.

  It is not only used by users, but also by the test framework and ~10 test files via `get_wallet_rpc()`.

  This PR adds coverage, while simultaneously improving the `-getinfo` coverage when multiple wallets are loaded. This is similar to the test coverage that would be added in #18594.

ACKs for top commit:
  robot-visions:
    ACK 2495110012

Tree-SHA512: caaa8b99fb8fa481ab2c6b2a287ed29720bb4553c3f66657462c44fa2990acaaf36cabeaaf81408678e5fdce4e105d729dd94b5ed8588dd1a6f2cb03fc25acf3
2020-04-24 18:41:22 -04:00
MarcoFalke
fabe44e815 bench: Start nodes with -nodebuglogfile 2020-04-24 16:46:54 -04:00
MarcoFalke
a215c61333 Merge #18756: refactor: test: use wait_for_getdata() in p2p_compactblocks.py
c4027e7350 refactor: test: use wait_for_getdata() in p2p_compactblocks.py (Sebastian Falbesoner)

Pull request description:

  The method `wait_for_getdata()` was recently changed to be more precise by waiting for a specified list of hashes, instead of only matching _any_ `getdata` message (see Issue #18614 and PR  #18690). This PR replaces the remaining occurences of manual inspection of `last_messages` with this call.

ACKs for top commit:
  robot-visions:
    ACK c4027e7350

Tree-SHA512: e10b346742f235b6ee2ef1f32f7fd74406c1a277389f020fb9913a93e94cc9530e1e9414872b83c9d2ae652ebce2b09b2c8c8372260c1afb4e0e54fbf7a935b0
2020-04-24 16:37:17 -04:00
MarcoFalke
29637a5c56 Merge #18745: test: Check submitblock return values
fa262712ca test: Check submitblock return values (MarcoFalke)

Pull request description:

  Add `assert_equal` in some tests to check the `submitblock` return value

ACKs for top commit:
  robot-visions:
    ACK fa262712ca

Tree-SHA512: 25d9effe82a4f6852184b9ac848f96336cc2cafb0bb07edb2792f00cd363f0759575bc9c164dd62f64425d3754028b4acd0675600c07d51277aa80bf66c6f960
2020-04-24 16:34:43 -04:00
Jon Atack
2495110012 test: add coverage for -rpcwallet cli option
and add -getinfo coverage with multiple wallets loaded.
2020-04-24 21:51:00 +02:00
MarcoFalke
8c0f86f284 Merge #18757: test: Remove enumeration of expected deserialization exceptions in ProcessMessage(...) fuzzer
fdceb63283 fuzz: Remove enumeration of expected deserialization exceptions in ProcessMessage(...) fuzzer (practicalswift)

Pull request description:

  Remove enumeration of expected deserialization exceptions in `ProcessMessage(...)` fuzzer.

  Closes #18749.

Top commit has no ACKs.

Tree-SHA512: fe0411dd1e574fe635019d9e3329202798295c8be1b0c5cb9c092ecc27ab7d4d2883104d7bd781ff5d422d13480d858fc8a7f5578d09268d142ae966afb79002
2020-04-24 14:38:23 -04:00
MarcoFalke
7812889849 Merge #18157: doc: fixing init.md documentation to not require rpcpassword
a2a03c3ca9 fixing documentation to not require rpcpassword (“jkcd”)

Pull request description:

  Configuration section in [doc/init.md](https://github.com/bitcoin/bitcoin/blob/master/doc/init.md) says user must set rpcpassword in order to run bitcoind. Since [71cbea](71cbeaad9a) fixed the code to use a cookie for authentication, it is not mandatory to set rpcpassword in the configuration.

  Fixes #16346

ACKs for top commit:
  hebasto:
    ACK a2a03c3ca9, modulo nit

Tree-SHA512: a62816fef78bed32200bb278cfc7aacf6ea154a60fdf5181927e48b806a1bd694bdf3ccec8362f5e58aad694d636c63f540323d54d85b61deaa417b95b8b56eb
2020-04-24 14:22:14 -04:00
Hennadii Stepanov
8098dea069 test: Add mempool_updatefromblock.py 2020-04-24 18:18:34 +03:00
practicalswift
fdceb63283 fuzz: Remove enumeration of expected deserialization exceptions in ProcessMessage(...) fuzzer 2020-04-24 14:53:59 +00:00
MarcoFalke
d1aa0ae1ad Merge #18712: test: display command line options passed to send_cli() in debug log
8f5dc8800a test: display command line options passed to send_cli() in debug log (Jon Atack)

Pull request description:

  as per https://github.com/bitcoin/bitcoin/pull/18691#discussion_r411382589, and revert two cli calls changed in #18691 from rpc commands back to command line options (these were the only occurrences).

ACKs for top commit:
  MarcoFalke:
    ACK 8f5dc8800a

Tree-SHA512: fcb3eca00aa4099066028c90d5e50a02e074366e09a17f5f5b937d9f7562dd054ff65681aa0ad4c94f6de1e98b1e2b9ac4cd084ddc297010253989a80483b1b9
2020-04-24 08:41:15 -04:00
practicalswift
fd8e99da57 tests: Add fuzzing harness for functions in primitives/transaction.h 2020-04-24 12:16:03 +00:00
MarcoFalke
85bae24d06 Merge #18752: test: Fix intermittent error in mempool_reorg
fae98668d1 test: Fix intermittent error in mempool_reorg (MarcoFalke)

Pull request description:

  Example: https://travis-ci.org/github/bitcoin/bitcoin/jobs/677689899#L4717

  Also speed up tx relay and fix two pep8 errors while touching the file anyway.

ACKs for top commit:
  vasild:
    utACK fae9866

Tree-SHA512: 23a7894e71ad0e1a59c74c73643708fca21b505fa4e980038d554294063fd63c396669eefb233ffdffb0083968e51b702c643cb449df8f656dd8345a20f33907
2020-04-24 07:57:48 -04:00
Sebastian Falbesoner
c4027e7350 refactor: test: use wait_for_getdata() in p2p_compactblocks.py 2020-04-24 13:55:20 +02:00
fanquake
d8ca51db5d Merge #18589: Fix naming of macOS SDK and clarify version
eb37275a6f Fix naming of macOS SDK and clarify version (Andrew Chow)

Pull request description:

  Fixes the `MacOSX10.14.sdk.tar.gz` creation command to have `MacOSX.sdk` be correctly named as `MacOSX10.14.sdk` and for the resulting file to be placed in the current directory. Gitian requires that `tar.gz` contains a folder named `MacOSX10.14.sdk` and the command did not do this originally. Having the file be placed in the current directory is a convenience so builders don't have to go find it.

  Also clarifies which version of Xcode to download and where it can be downloaded.

ACKs for top commit:
  fanquake:
    ACK eb37275a6f - tested the macOS and Linux SDK extraction. Also noticed something seemingly broken with Apple `tar`, but will open an issue to follow up.
  Sjors:
    ACK eb37275 for the macOS instruction

Tree-SHA512: d691e14711cf195999291dd6fb7ffe552c86f8b30d2b1a77e88b4db6050dd817ba128b047cf36d29b0bb0d4183e709b7c03aa27f31b64e562ea8cd948434ca55
2020-04-24 17:22:54 +08:00
MarcoFalke
fa8b9b5d1f test: Fix intermittent failure in wallet_importmulti 2020-04-23 19:01:26 -04:00
MarcoFalke
fae98668d1 test: Fix intermittent error in mempool_reorg 2020-04-23 18:43:44 -04:00
Amiti Uttarwar
50fc4df6c4 [mempool] Persist unbroadcast set to mempool.dat
Ensure that the unbroadcast set will still be meaningful if the node is
restarted.
2020-04-23 14:42:25 -07:00
Amiti Uttarwar
297a178536 [test] Integration tests for unbroadcast functionality
Check that...
- mempool tracks & reattempts delivery of a transaction where a GETDATA hasn't
  been requested by a peer yet.
- transaction delivery is not attempted again after GETDATA is received.
- transaction is removed from the unbroadcast set when its removed from the
  mempool.
2020-04-23 14:42:25 -07:00
Amiti Uttarwar
6851502472 [refactor/test] Extract P2PTxInvStore into test framework 2020-04-23 14:42:25 -07:00
Amiti Uttarwar
dc1da48dc5 [wallet] Update the rebroadcast frequency to be ~1/day.
Since the mempool unbroadcast mechanism handles the reattempts for initial
broadcast, the wallet rebroadcast attempts can be much less frequent
(previously ~1/30 min)
2020-04-23 14:42:25 -07:00
Amiti Uttarwar
e25e42f20a [p2p] Reattempt initial send of unbroadcast transactions
Every 10-15 minutes, the scheduler kicks off a job that queues unbroadcast
transactions onto each node.
2020-04-23 14:42:25 -07:00
Amiti Uttarwar
7e93eecce3 [util] Add method that returns random time in milliseconds 2020-04-23 14:42:25 -07:00
Amiti Uttarwar
89eeb4a333 [mempool] Track "unbroadcast" transactions
- Mempool tracks locally submitted transactions (wallet or rpc)
- Transactions are removed from set when the node receives a GETDATA request
  from a peer, or if the transaction is removed from the mempool.
2020-04-23 14:42:25 -07:00
Andrew Chow
223588b1bb Add a --descriptors option to various tests
Adds a --descriptors option globally to the test framework. This will
make the test create and use descriptor wallets. However some tests may
not work with this.

Some tests are modified to work with --descriptors and run with that
option in test_runer:
* wallet_basic.py
* wallet_encryption.py
* wallet_keypool.py
* wallet_keypool_topup.py
* wallet_labels.py
* wallet_avoidreuse.py
2020-04-23 13:59:48 -04:00
Andrew Chow
869f7ab30a tests: Add RPCOverloadWrapper which overloads some disabled RPCs
RPCOverloadWrapper overloads some deprecated or disabled RPCs with
an implementation using other RPCs to avoid having a ton of code churn
around replacing those RPCs.
2020-04-23 13:59:48 -04:00
Andrew Chow
cf06062859 Correctly check for default wallet 2020-04-23 13:59:48 -04:00
Andrew Chow
886e0d75f5 Implement CWallet::IsSpentKey for non-LegacySPKMans 2020-04-23 13:59:48 -04:00
Andrew Chow
3c19fdd2a2 Return error when no ScriptPubKeyMan is available for specified type
When a CWallet doesn't have a ScriptPubKeyMan for the requested type
in GetNewDestination, give a meaningful error. Also handle this in
Qt which did not do anything with errors.
2020-04-23 13:59:48 -04:00
Andrew Chow
388ba94231 Change wallet_encryption.py to use signmessage instead of dumpprivkey 2020-04-23 13:59:48 -04:00
Andrew Chow
1346e14831 Functional tests for descriptor wallets 2020-04-23 13:59:48 -04:00
Hugo Nguyen
f193ea889d add importdescriptors RPC and tests for native descriptor wallets
Co-authored-by: Andrew Chow <achow101-github@achow101.com>
2020-04-23 13:59:48 -04:00
Andrew Chow
ce24a94494 Add IsLegacy to CWallet so that the GUI knows whether to show watchonly 2020-04-23 13:59:48 -04:00
Andrew Chow
1cb42b22b1 Generate new descriptors when encrypting 2020-04-23 13:59:48 -04:00
Andrew Chow
82ae02b165 Be able to create new wallets with DescriptorScriptPubKeyMans as backing 2020-04-23 13:59:48 -04:00
Andrew Chow
b713baa75a Implement GetMetadata in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
8b9603bd0b Change GetMetadata to use unique_ptr<CKeyMetadata> 2020-04-23 13:59:48 -04:00
Andrew Chow
72a9540df9 Implement FillPSBT in DescriptorScriptPubKeyMan
FillPSBT will add our own scripts to the PSBT if those inputs are ours.
If an input also lists pubkeys that we happen to know the private keys
for, we will sign those inputs too.
2020-04-23 13:59:48 -04:00
Andrew Chow
84b4978c02 Implement SignMessage for descriptor wallets 2020-04-23 13:59:48 -04:00
Andrew Chow
bde7c9fa38 Implement SignTransaction in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
d50c8ddd41 Implement GetSolvingProvider for DescriptorScriptPubKeyMan
Internally, a GetSigningProvider function is introduced which allows for
some private keys to be optionally included. This can be called with a
script as the argument (i.e. a scriptPubKey from our wallet when we are
signing) or with a pubkey. In order to know what index to expand the
private keys for that pubkey, we need to also cache all of the pubkeys
involved when we expand the descriptor. So SetCache and TopUp are
updated to do this too.
2020-04-23 13:59:48 -04:00
Andrew Chow
f1ca5feb4a Implement GetKeypoolOldestTime and only display it if greater than 0 2020-04-23 13:59:48 -04:00
Andrew Chow
586b57a9a6 Implement ReturnDestination in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
f866957979 Implement GetReservedDestination in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
a775f7c7fd Implement Unlock and Encrypt in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
bfdd073486 Implement GetNewDestination for DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
58c7651821 Implement TopUp in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
e014886a34 Implement SetupGeneration for DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
46dfb99768 Implement writing descriptorkeys, descriptorckeys, and descriptors to wallet file 2020-04-23 13:59:48 -04:00
Andrew Chow
4cb9b69be0 Implement several simple functions in DescriptorScriptPubKeyMan
Implements a bunch of one liners: UpgradeKeyMetadata, IsFirstRun, HavePrivateKeys,
KeypoolCountExternalKeys, GetKeypoolSize, GetTimeFirstKey, CanGetAddresses,
RewriteDB
2020-04-23 13:59:48 -04:00
Andrew Chow
d1ec3e4f19 Add IsSingleType to Descriptors
IsSingleType will return whether the descriptor will give one or multiple scriptPubKeys
2020-04-23 13:59:48 -04:00
Andrew Chow
953feb3d27 Implement loading of keys for DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
2363e9fcaa Load the descriptor cache from the wallet file 2020-04-23 13:59:48 -04:00
Andrew Chow
46c46aebb7 Implement GetID for DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
ec2f9e1178 Implement IsHDEnabled in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
741122d4c1 Implement MarkUnusedAddresses in DescriptorScriptPubKeyMan 2020-04-23 13:59:48 -04:00
Andrew Chow
2db7ca765c Implement IsMine for DescriptorScriptPubKeyMan
Adds a set of scriptPubKeys that DescriptorScriptPubKeyMan tracks.
If the given script is in that set, it is considered ISMINE_SPENDABLE
2020-04-23 13:59:48 -04:00
Andrew Chow
db7177af8c Add LoadDescriptorScriptPubKeyMan and SetActiveScriptPubKeyMan to CWallet 2020-04-23 13:59:42 -04:00
Andrew Chow
78f8a92910 Implement SetType in DescriptorScriptPubKeyMan 2020-04-23 13:25:50 -04:00
Andrew Chow
834de0300c Store WalletDescriptor in DescriptorScriptPubKeyMan 2020-04-23 13:25:50 -04:00
Andrew Chow
d8132669e1 Add a lock cs_desc_man for DescriptorScriptPubKeyMan 2020-04-23 13:25:50 -04:00
Andrew Chow
3194a7f88a Introduce WalletDescriptor class
WalletDescriptor is a Descriptor with other wallet metadata
2020-04-23 13:25:50 -04:00
Andrew Chow
6b13cd3fa8 Create LegacyScriptPubKeyMan when not a descriptor wallet 2020-04-23 13:25:50 -04:00
Andrew Chow
aeac157c9d Return nullptr from GetLegacyScriptPubKeyMan if descriptor wallet 2020-04-23 13:25:50 -04:00
Andrew Chow
96accc73f0 Add WALLET_FLAG_DESCRIPTORS 2020-04-23 13:25:50 -04:00
Andrew Chow
6b8119af53 Introduce DescriptorScriptPubKeyMan as a dummy class 2020-04-23 13:25:50 -04:00
Andrew Chow
06620302c7 Introduce SetType function to tell ScriptPubKeyMans the type and internal-ness of it 2020-04-23 13:25:50 -04:00
Andrew Chow
eb37275a6f Fix naming of macOS SDK and clarify version 2020-04-23 13:22:16 -04:00
John Newbery
e9ea95a30d [net processing] Move all const declarations to top of net_processing.cpp 2020-04-23 12:54:06 -04:00
John Newbery
507b36dd1b [validation] Move all const declarations to top of validation.h 2020-04-23 12:54:06 -04:00
John Newbery
0109622b08 [validation] Move validation-only consts to validation.cpp 2020-04-23 12:54:06 -04:00
John Newbery
b8580cacc7 [net processing] Move net processing consts to net_processing.cpp 2020-04-23 12:54:03 -04:00
MarcoFalke
64139803f1 Merge #18690: test: Check object hashes in wait_for_getdata
9f5608c289 test: check for matching object hashes in wait_for_getdata (Danny Lee)

Pull request description:

  Previously, `wait_for_getdata` only looked for the presence of a recent `"getdata"` message.  Additionally checking the object hashes inside the message should make tests involving `wait_for_getdata` more robust.

  `p2p_sendheaders.py` already overrides `wait_for_getdata` do this check; we can use the same approach consistently across all tests that call `wait_for_getdata`.

  This PR is progress towards #18614 , but closing that issue would also involve some additional changes to `wait_for_getheaders`.

ACKs for top commit:
  theStack:
    ACK 9f5608c289 🍻

Tree-SHA512: 8e7f95881c19631db014d4bb2399fea0d14686a32542f6ca3b60809744b0d684eac4e4c107c87143991f3cd0c2d4ab09d0c17486239768a9b40bee25f2e4d54a
2020-04-23 06:37:53 -04:00
Samuel Dobson
e890c15e2c Merge #18671: wallet: Add BlockUntilSyncedToCurrentChain to dumpwallet
fa60afc4fb wallet: Add BlockUntilSyncedToCurrentChain to dumpwallet (MarcoFalke)

Pull request description:

  dumpwallet includes the block hash in the output, so this method depends on the chainstate. According to the developer notes e84a5f0004/doc/developer-notes.md (L1095) it must include a `BlockUntilSyncedToCurrentChain`.

  This is a minor fix and does not need backport, I think.

  It fixes test failures such as https://travis-ci.org/github/bitcoin/bitcoin/jobs/675487097#L2657 , which can only happen in master because the test was not backported.

ACKs for top commit:
  promag:
    Code review ACK fa60afc4fb.
  ryanofsky:
    Code review ACK fa60afc4fb
  meshcollider:
    utACK fa60afc4fb

Tree-SHA512: 8df70b06b226b2cdf880dec9264adb72d66fd81b09b404fd1665a79e5f5236d26122eebf15df00fe71ee292b5c91b2dc23a0a42b2aa50a8d690604b23832723f
2020-04-23 14:12:35 +12:00
Samuel Dobson
4f802e59a0 Merge #17509: gui: save and load PSBT
764bfe4cba [psbt] add file size limit (Sjors Provoost)
1cd8dc2556 [gui] load PSBT (Sjors Provoost)
f6895301f7 [gui] save PSBT to file (Sjors Provoost)
1d05a9d80b Move DEFAULT_MAX_RAW_TX_FEE_RATE to node/transaction.h (Sjors Provoost)
86e22d23bb [util] GetFileSize (Sjors Provoost)
6ab3aad9a5 [gui] send dialog: split on_sendButton_clicked (Sjors Provoost)

Pull request description:

  This adds:
  * a dialog after Create Unsigned, which lets you save a PSBT file in binary format, e.g. to an SD card
  * a "Load PSBT" menu entry lets you pick a PSBT file. We broadcast the transaction if complete

  ## Save flow
  <img width="482" alt="Schermafbeelding 2020-01-04 om 20 39 34" src="https://user-images.githubusercontent.com/10217/71765684-ba60d580-2f32-11ea-8dea-0c4398eb6e15.png">

  <img width="287" alt="Schermafbeelding 2020-01-04 om 20 40 35" src="https://user-images.githubusercontent.com/10217/71765677-a0bf8e00-2f32-11ea-8172-12dfd34a89f3.png">

  <img width="594" alt="Schermafbeelding 2020-01-04 om 20 41 12" src="https://user-images.githubusercontent.com/10217/71765681-aa48f600-2f32-11ea-8e2c-c4f6bf9f5309.png">

  <img width="632" alt="Schermafbeelding 2020-01-04 om 20 41 28" src="https://user-images.githubusercontent.com/10217/71765691-d19fc300-2f32-11ea-97ff-70f5dd59987a.png">

  By default the file name contains the destination address(es) and amount(s).

  We only use the binary format for files, in order to avoid compatibility hell. If we do want to add base64 file format support, we should use a different extension for that (`.psbt64`?).

  ## Load flow

  Select a file:
  <img width="649" alt="Schermafbeelding 2020-01-04 om 21 08 57" src="https://user-images.githubusercontent.com/10217/71766089-2ba28780-2f37-11ea-875d-074794b5707d.png">

  Offer to send if complete:

  <img width="308" alt="Schermafbeelding 2020-01-04 om 21 09 06" src="https://user-images.githubusercontent.com/10217/71766088-2a715a80-2f37-11ea-807d-394c8b840c59.png">

  Tell user if signatures are missing, offer to copy to clipboard:
  <img width="308" alt="Schermafbeelding 2020-01-04 om 21 15 57" src="https://user-images.githubusercontent.com/10217/71766115-702e2300-2f37-11ea-9f62-a6ede499c0fa.png">

  Incomplete for another reason:

  <img width="309" alt="Schermafbeelding 2020-01-04 om 21 07 51" src="https://user-images.githubusercontent.com/10217/71766090-2c3b1e00-2f37-11ea-8a22-6188377b67a1.png">

ACKs for top commit:
  instagibbs:
    re-ACK  764bfe4cba
  achow101:
    ACK 764bfe4cba
  jb55:
    Tested ACK 764bfe4cba
  jonatack:
    ACK 764bfe4c
  promag:
    Code review ACK 764bfe4cba.

Tree-SHA512: d284ed6895f3a271fb8ff879aac388ad217ddc13f72074725608e1c3d6d90650f6dc9e9e254479544dd71fc111516b02c8ff92158153208dc40fb2726b37d063
2020-04-23 13:16:23 +12:00
fanquake
3e38023af7 scripts: add PE .reloc section check to security-check.py 2020-04-23 08:40:24 +08:00
MarcoFalke
fa262712ca test: Check submitblock return values 2020-04-22 16:56:56 -04:00
practicalswift
d5a31b7cb4 tests: Add fuzzing harness for functions in primitives/block.h 2020-04-22 19:51:42 +00:00
MarcoFalke
47b94a337e Merge #18732: test: Remove unused, undocumented and misleading CScript.__add__
faff9e4bb4 test: Remove unused, undocumented and misleading CScript.__add__ (MarcoFalke)

Pull request description:

  See the corresponding pull #18612

ACKs for top commit:
  laanwj:
    ACK faff9e4bb4 provided it passes Travis

Tree-SHA512: 5d9c4d5b6453c70b24a6960d3b42834e9b31f6dbb99ac47a6abfd85f2739d5372563e7188c22aceabeee1c37eb218bf580848356f4a77268d65f178a9419b269
2020-04-22 15:49:22 -04:00
MarcoFalke
fb89af26d8 Merge #18733: doc: Add wallet release notes for 0.21.0
facaefadd3 doc: Add wallet release notes for 0.21.0 (MarcoFalke)
faa4243c11 Add release notes skeleton, so that notes can be filled easier (MarcoFalke)

Pull request description:

ACKs for top commit:
  fjahr:
    ACK facaefadd3
  achow101:
    ACK facaefadd3

Tree-SHA512: a17ab86e422ca3d3e53deffa7fecf09cdd9b816588deeded3a15e80a1c268ff1e8b56a0e052a417f1a091872099cd3d2b89993d4773a86516b0bdef880a949a0
2020-04-22 15:45:56 -04:00
Danny Lee
9f5608c289 test: check for matching object hashes in wait_for_getdata 2020-04-22 10:46:08 -07:00
practicalswift
bb1ec36fb1 doc: Document how to fuzz Bitcoin Core using honggfuzz 2020-04-22 15:22:36 +00:00
MarcoFalke
a7a6f1ff41 Merge #18575: bench: Remove requirement that all benches use same testing setup
fa1fdb02fc bench: Replace ::mempool globabl with test_setup.mempool (MarcoFalke)
fab1170964 bench: Remove requirement that all benches use RegTestingSetup (MarcoFalke)

Pull request description:

  The benches have always set up one global testing setup. This makes it hard to pick no testing setup at all or one with different params.

  Fix this by removing any global state setup from the main `bench.cpp` and leave the setup to each individual bench.

  One reason to have one global testing setup is to set the datadir location to a tempdir to avoid reading or writing in the default datadir location. But #13687 should prevent this already.

Top commit has no ACKs.

Tree-SHA512: 7c98aea7725a20f4b9225221f4279b9e9f7257ed5c14712ad01ea80d87c3b0fed760b40f413892498bbb354a917ee02d4c575cbe8423a403b86755e8ee11f33b
2020-04-22 10:52:40 -04:00
MarcoFalke
facaefadd3 doc: Add wallet release notes for 0.21.0 2020-04-22 09:22:49 -04:00
Russell Yanofsky
1e94a2bcbc depends: Add --sysroot option to mac os native compile flags
Catalina SDK clang stopped automatically searching the SDK include paths when
invoked without --sysroot:

https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-594600985
https://github.com/Homebrew/homebrew-core/issues/45061

This hasn't been a problem for current native depends packages because are
passing their own --sysroot values, and hasn't been a problem for current host
packages because they use `darwin_` commands instead of `build_darwin_`
commands.  But the current `build_darwin_CC` and `build_darwin_CXX` commands
are still unnecessarily fragile, and incompatible with new native depends
packages added in https://github.com/bitcoin/bitcoin/pull/18677.

Cory Fields <cory-nospam-@coryfields.com> suggested in
https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-595393546 switching
compiler from SDK clang to native clang (from $PATH) to avoid this problem.
This is easy and makes a certain amount of sense for building native packages,
as opposed to host packages. But fanquake <fanquake@gmail.com> pointed out in
https://github.com/bitcoin/bitcoin/pull/18677#discussion_r409934309 that it
would be inconsistent use switch to non-SDK compilers while still using other
SDK tools like ranlib and install_name_tool. So simplest, minimal fix seems to
be just adding the missing --sysroot option.
2020-04-22 08:18:11 -05:00
Wladimir J. van der Laan
5dcb061589 Merge #18702: build: fix ASLR for bitcoin-cli on Windows
315a4d36f7 build: fix ASLR for bitcoin-cli on Windows (fanquake)

Pull request description:

  ASLR is not currently working for the `bitcoin-cli.exe` binary. This is
  due to it not having a .reloc section, which is stripped by default by
  the mingw-w64 ld we use for gitian builds. A good summary of issues with
  ld and mingw-w64 is available in this thread:
  https://sourceware.org/bugzilla/show_bug.cgi?id=19011.

  All other Windows binaries that we distribute (bitcoind, bitcoin-qt,
  bitcoin-wallet, bitcoin-tx and test_bitcoin) do not suffer this issue,
  and currently having working ASLR. This is due to them exporting
  (inadvertent or not) libsecp256k1 symbols, and, as a result, the .reloc
  section is not stripped by ld.

  This change is a temporary workaround, also the same one described here:
  https://www.kb.cert.org/vuls/id/307144/, that causes main() to be
  exported. Exporting a symbol will mean that the .reloc section is not
  stripped, and ASLR will function correctly.

  Ultimately, this will be fixed by using a newer version of binutils (that has this [change](https://sourceware.org/git/gitweb.cgi?p=binutils-gdb.git;a=commit;h=dc9bd8c92af67947db44b3cb428c050259b15cd0)). Whether that's through bumping our gitian distro, or Guix.

  Related to #18629, which has a bunch of additional information in the PR description. If you would like to verify whether or not ASLR is indeed working, with or without this change. One easy way to check is using a tool like [VMMap](https://docs.microsoft.com/en-us/sysinternals/downloads/vmmap).

  Here are the memory mappings for the 0.20.0rc1 `bitcoind.exe` and `bitcoin-cli.exe` binaries. You'll notice that over machine restarts, even though the image is marked `(ASLR)` (which I assume may be due to the header bit being set), no ASLR is actually occuring for `bitcoin-cli.exe`:

  #### bitcoind.exe

  ![bitcoind-1](https://user-images.githubusercontent.com/863730/79678203-74065c80-822b-11ea-90bc-9c883d0aeefa.png)

  ![bitcoind-2](https://user-images.githubusercontent.com/863730/79678204-7668b680-822b-11ea-9263-3e7ba22f904c.png)

  ![bitcoind-3](https://user-images.githubusercontent.com/863730/79678206-7963a700-822b-11ea-972f-af31a514b9b4.png)

  #### bitcoin-cli.exe

  ![bitcoin-cli-1](https://user-images.githubusercontent.com/863730/79678208-7ec0f180-822b-11ea-8480-a4b5d1762945.png)

  ![bitcoin-cli-2](https://user-images.githubusercontent.com/863730/79678213-81bbe200-822b-11ea-964d-994f58ff12b0.png)

  ![bitcoin-cli-3](https://user-images.githubusercontent.com/863730/79678215-84b6d280-822b-11ea-9cd6-fee2e239c003.png)

ACKs for top commit:
  dongcarl:
    ACK 315a4d36f7
  laanwj:
    ACK 315a4d36f7

Tree-SHA512: 95f4dc15420ed9bcdeacb763e11c3c7e563eec594a172746fa0346c13f97db3a8769357dffc89fea1e57ae67133f337b1013a73b584662f5b6c4d251ca20a2b1
2020-04-22 15:18:11 +02:00
MarcoFalke
faa4243c11 Add release notes skeleton, so that notes can be filled easier 2020-04-22 09:16:05 -04:00
Wladimir J. van der Laan
ce4e1f0282 Merge #18553: Avoid non-trivial global constants in SHA-NI code
8508473094 Avoid non-trivial global constants in SHA-NI code (Pieter Wuille)

Pull request description:

  This is a potential solution for #18456.

  It seems that the compiler cannot turn `_mm_set_epi64x(<constant>,<constnant>)` into a constant itself, and thus emits a global initializer for the `MASK`, `INIT0`, and `INIT1` global constants in the sha-ni SHA256 implementation.

  Change this by turning them into dumb byte arrays, loading them into an SSE variable whenever needed.

  Tested on a SHA-NI capable machine. I do not observe any obvious performance impact (but this is hard to measure, it's already very fast...).

ACKs for top commit:
  laanwj:
    Code review ACK 8508473094
  elichai:
    ACK 8508473094

Tree-SHA512: 07049cf1a33624c22df2be48b814d5636c037b368861eb13ee073bdce2b7c902a56e96518218961f55a2a1631a40825ded6dbbc28d7fe0e7fec267d704e39112
2020-04-22 15:09:19 +02:00
MarcoFalke
faff9e4bb4 test: Remove unused, undocumented and misleading CScript.__add__ 2020-04-22 09:00:56 -04:00
Wladimir J. van der Laan
9e8e813df5 Merge #18410: Docs: Improve commenting for coins.cpp|h
21fa0a44ab [docs] use consistent naming for possible_overwrite (John Newbery)
2685c214cc [tests] small whitespace fixup (John Newbery)
e9936966c0 scripted-diff: Rename PRUNED to SPENT in coins tests (John Newbery)
c205979031 [docs] Improve commenting in coins.cpp|h (John Newbery)

Pull request description:

  - Add full commenting for spentness / DIRTYness / FRESHness and which combinations are valid
  - Remove the 'pruned' terminology, which doesn't make sense since per-txout chainstate db was merged (#10195).
  - Rename `potential_overwrite` to `possible_overwrite` to standardize terminology (there were previously examples of both, which made searching the codebase difficult).
  - Make other minor improvements to the comments

ACKs for top commit:
  jonatack:
    Re-ACK 21fa0a4 per `git diff 98bee55 21fa0a4` the only change since my previous review is the following code commenting diff in `src/coins.cpp::L177-179`;  rebuilt/ran unit tests anyway as a sanity check on the unit test changes.

Tree-SHA512: 391e01588ef5edb417250080cec17361f982c4454bc5f8c6d78bbd528c68a2bb94373297760691295c24660ce1022ad3ef7599762f736c8eed772ce096d38c3d
2020-04-22 14:23:56 +02:00
Wladimir J. van der Laan
acb4fa0741 Merge #18665: Do not expose and consider -logthreadnames when it does not work
b91e4ae0d8 Do not expose and consider -logthreadnames when it does not work (Hennadii Stepanov)

Pull request description:

  There are conditions when the `HAVE_THREAD_LOCAL` macro is undefined what causes the `-logthreadnames` option does not work -- instead of thread names empty strings `[]` only are printed in the `debug.log` file.

  This PR does not exposes the `-logthreadnames` option in such cases.

  Refs:
  - #16059
  - #18652

ACKs for top commit:
  MarcoFalke:
    ACK b91e4ae0d8, looked at the diff, didn't test

Tree-SHA512: 3bd58e5ea603c69686589ddc94d6fa441cab4f712004378f2f1661e12638804ca03cfb6426e6393e55b6a095b325f3161d3c5371af05d7fc79d6d328227bf40c
2020-04-22 14:18:06 +02:00
Wladimir J. van der Laan
19032c750c Merge #18612: script: Remove undocumented and unused operator+
ccccd51908 script: Remove undocumented and unused operator+ (MarcoFalke)

Pull request description:

  This operator has no documented use case and is also unused outside of test code. The test code and all other (imaginary) code that might use this operator is written more clear and concise by the existing CScript push operators for opcodes and data.

  Removing the operator is also going to protect against accidentally reintroducing bugs like this 6ff5f718b6 (diff-8458adcedc17d046942185cb709ff5c3L1135) (last time it was used).

ACKs for top commit:
  laanwj:
    ACK ccccd51908

Tree-SHA512: 43898ac77e4d9643d9f8ac6f8f65497a4f0bbb1fb5dcaecc839c3719aa36181ba77befb213e59a9f33a20a29e0173a0e9c4763b1930940b32c3d1598b3e39af9
2020-04-22 14:17:01 +02:00
fanquake
c90a9e6fff Merge #18713: scripts: Add MACHO stack canary check to security-check.py
8334ee31f8 scripts: add MACHO LAZY_BINDINGS test to test-security-check.py (fanquake)
7b99c7454c scripts: add MACHO Canary check to security-check.py (fanquake)

Pull request description:

  7b99c7454c uses `otool -Iv` to check for `___stack_chk_fail` in the macOS binaries. Similar to the [ELF check](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/security-check.py#L105). Note that looking for a triple underscore prefixed function (as opposed to two for ELF) is correct for the macOS binaries. i.e:
  ```bash
  otool -Iv bitcoind | grep chk
  0x00000001006715b8   509 ___memcpy_chk
  0x00000001006715be   510 ___snprintf_chk
  0x00000001006715c4   511 ___sprintf_chk
  0x00000001006715ca   512 ___stack_chk_fail
  0x00000001006715d6   517 ___vsnprintf_chk
  0x0000000100787898   513 ___stack_chk_guard
  ```

  8334ee31f8 is a follow up to #18295 and adds test cases to `test-security-check.py` that for some reason I didn't add at the time. I'll sort out #18434 so that we can run these tests in the CI.

ACKs for top commit:
  practicalswift:
    ACK 8334ee31f8: Mitigations are important. Important things are worth asserting :)
  jonasschnelli:
    utACK 8334ee31f8.

Tree-SHA512: 1aa5ded34bbd187eddb112b27278deb328bfc21ac82316b20fab6ad894f223b239a76b53dab0ac1770d194c1760fcc40d4da91ec09959ba4fc8eadedb173936a
2020-04-22 16:40:32 +08:00
fanquake
7d1a3bda21 Merge #18709: doc: note why we can't use thread_local with glibc back compat
b155fcda51 doc: fix typo in configure.ac (fanquake)
20a30922fb doc: note why we can't use thread_local with glibc back compat (fanquake)

Pull request description:

  Given that we went through a [gitian build](https://github.com/bitcoin/bitcoin/pull/18681) to remember why this is the case, we might as well make a note of it in configure.ac.

  [From #18681](https://github.com/bitcoin/bitcoin/pull/18681#issuecomment-615526634):

  Looking at the Linux build log, this has failed with:
  ```bash
  Checking glibc back compat...
  bitcoind: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoind: failed IMPORTED_SYMBOLS
  bitcoin-cli: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoin-cli: failed IMPORTED_SYMBOLS
  bitcoin-tx: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoin-tx: failed IMPORTED_SYMBOLS
  bitcoin-wallet: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bitcoin-wallet: failed IMPORTED_SYMBOLS
  test/test_bitcoin: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  test/test_bitcoin: failed IMPORTED_SYMBOLS
  bench/bench_bitcoin: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  bench/bench_bitcoin: failed IMPORTED_SYMBOLS
  qt/bitcoin-qt: symbol __cxa_thread_atexit_impl from unsupported version GLIBC_2.18
  ```

  `__cxa_thread_atexit_impl` is used for [thread_local variable destruction](https://sourceware.org/glibc/wiki/Destructor%20support%20for%20thread_local%20variables):

  > To implement this support, glibc defines __cxa_thread_atexit_impl exclusively for use by libstdc++ (which has the __cxa_thread_atexit to wrap around it), that registers destructors for thread_local variables in a list. Upon thread or process exit, the destructors are called in reverse order in which they were added.

  As suggested, this only became available in glibc 2.18. From the [2.18 release notes](https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html):

  > * Add support for calling C++11 thread_local object destructors on thread
    and program exit.  This needs compiler support for offloading C++11
    destructor calls to glibc.

ACKs for top commit:
  hebasto:
    ACK b155fcda51

Tree-SHA512: 5b9567e4a70598a4b0b91956f44ae0d93091db17c84cbf9817dac6cfa992c97d3438a8b1bb66644c74891f2149e44984daed445d22de93ca8858c5b0eabefb40
2020-04-22 14:46:19 +08:00
John Newbery
21fa0a44ab [docs] use consistent naming for possible_overwrite
And other general comment improvements for adding coins.
2020-04-21 14:19:15 -04:00
John Newbery
2685c214cc [tests] small whitespace fixup
Required after scripted-diff in previous commit.
2020-04-21 14:19:15 -04:00
John Newbery
e9936966c0 scripted-diff: Rename PRUNED to SPENT in coins tests
-BEGIN VERIFY SCRIPT-
sed -i -e 's/PRUNED,/SPENT ,/g' ./src/test/coins_tests.cpp
sed -i -e 's/PRUNED/SPENT/g' ./src/test/coins_tests.cpp
-END VERIFY SCRIPT-
2020-04-21 14:19:15 -04:00
John Newbery
c205979031 [docs] Improve commenting in coins.cpp|h
Remove references to 'pruned' coins, which don't exist since the move
to per-txout coins db.
2020-04-21 14:18:03 -04:00
MarcoFalke
b6a5dc90bf Merge #18384: [test] more specific feature_segwit test error messages and fixing incorrect comments
3c21db7b78 [test] add 8 error messages to feature_segwit and change version to type (Gloria Zhao)

Pull request description:

  Followup to [this](https://github.com/bitcoin/bitcoin/pull/15169/files#r303673472) comment on functional test feature_segwit.py verifying that unsigned witness transactions are invalid.

  (1) Changes 8 error messages from "mandatory-script-verify-flag" to "non-mandatory-script-verify-flag" and with more specific error messages.
  (2) Edits comments that incorrectly describe the test, namely that the `v` variable corresponds to using P2WSH versus P2WPKH, not witness versions.

ACKs for top commit:
  MarcoFalke:
    ACK 3c21db7b78 🍾

Tree-SHA512: 3734ea3762667636c4fb20f5285634ab94d6b3527b7390fcc5e41b4582829dfe0099beabeaed42098613d168ede3385a6ffcd73989d1fa9dbd18004f5e9cf083
2020-04-21 13:42:21 -04:00
Gloria Zhao
3c21db7b78 [test] add 8 error messages to feature_segwit and change version to type
P2WPKH witness program without signature -> throws "hash mismatch" error
P2WSH witness program without signature -> throws "empty witness" error
same errors for P2SH_P2WPKH and P2SH_P2WSH respectively when passed redeemScript but no signature
P2SH_P2WPKH and P2SH_P2WSH with no signature fail with "Operation not valid with current stack size" when not signed due to missing input
change VER to TYPE and constants WIT_V0 to P2WPKH=0 and WIT_V1 to P2WSH=1
2020-04-21 08:17:42 -07:00
MarcoFalke
9ea4d8326a Merge #18704: test: Increase debugging to hunt down mempool_reorg intermittent failure
fac2fc4dd8 test: Increase debugging to hunt down mempool_reorg intermittent failure (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 4094b44afaa623e58b69f8d0332e60f0150b9ae2fd8bb265210d85546d887672ab8a3435cd9b086be14f69ab5b17e0f9fae06bd8aec1e7947ca766dd72b577c4
2020-04-21 10:58:10 -04:00
Jon Atack
8f5dc8800a test: display command line options passed to send_cli() in debug log
and fixup two cli calls from rpc commands to command line options.

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-04-21 15:38:26 +02:00
Vasil Dimov
a9b957740e bench: add CAddrMan benchmarks
The added benchmarks exercise the public methods Add(), GetAddr(),
Select() and Good().
2020-04-21 15:06:59 +02:00
MarcoFalke
977328338e Merge #18721: test: Fix linter issue
60cdcf30a4 test: Fix linter issue (Hennadii Stepanov)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 4fa0103526fed4b1399d3a6b83cea1e509cefff36d7e16ed1499d22afdc49e72053ac3d0634d858838ec6c296af7134131ea4d509c46de99da557567bc75d711
2020-04-21 08:05:42 -04:00
Hennadii Stepanov
60cdcf30a4 test: Fix linter issue 2020-04-21 14:58:51 +03:00
MarcoFalke
4ad6144ed0 Merge #18672: test: add further BIP37 size limit checks to p2p_filter.py
c743718558 test: add further BIP37 size limit checks to p2p_filter.py (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR to #18628. In addition to the hash-functions limit test introduced with commit fa4c29bc1d, it adds checks for the following size limits as defined in [BIP37](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki):

  ad message type `filterload`:
  > The filter itself is simply a bit field of arbitrary byte-aligned size. The maximum size is **36,000 bytes**.

  ad message type `filteradd`:
  > The data field must be smaller than or equal to **520 bytes** in size (the maximum size of any potentially matched object).

  Also introduces new constants for the limits (or reuses the max script size constant in case for the `filteradd` limit).

  Also fixes #18711 by changing the misbehaviour check on "filteradd without filterset" (introduced with #18544) below to also use the more commonly used `assert_debug_log` method.

ACKs for top commit:
  MarcoFalke:
    ACK c743718558
  robot-visions:
    ACK c743718558
  jonasschnelli:
    utACK c743718558. Seems to fix it: https://bitcoinbuilds.org/index.php?build=2524

Tree-SHA512: a03e7639263eb36a381922afb4e1d0ed2ae286f2ad2e7bbd922509a043ddf6cfd08747e01d54d29bfb8f54b66908f653974b9c347e4ca4f43332b586778893be
2020-04-21 07:26:18 -04:00
fanquake
8334ee31f8 scripts: add MACHO LAZY_BINDINGS test to test-security-check.py
I didn't add the relevant test in #18295.
2020-04-21 11:32:06 +08:00
fanquake
7b99c7454c scripts: add MACHO Canary check to security-check.py 2020-04-21 11:32:01 +08:00
fanquake
b155fcda51 doc: fix typo in configure.ac
s/liner/linker/
2020-04-21 10:05:43 +08:00
MarcoFalke
c4c3f110eb Merge #18190: tests: Add fuzzing harness for Golomb-Rice coding (GolombRiceEncode/GolombRiceDecode)
69749fbe6a tests: Add fuzzing harness for Golomb-Rice coding (GolombRiceEncode/GolombRiceDecode) (practicalswift)

Pull request description:

  Add fuzzing harness for Golomb-Rice coding (`GolombRiceEncode`/`GolombRiceDecode`).

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/golomb_rice
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 1b26512301b8c22ab3b804d9b9e4baf933f26f8c05e462d583863badcec7e694548a34849a0d7c4ff7d58b19f6338b51819976ecf642bc4659b04ef71182d748
2020-04-20 15:32:41 -04:00
Wladimir J. van der Laan
56d2ff8a8f Merge #18676: build: Check libevent minimum version in configure script
b68e717967 build: Set libevent minimum version to 2.0.21 (Hennadii Stepanov)

Pull request description:

  The non-`pkg-config` path is ignored as there is a hope to get rid of all of them in #18307.

  As xenial has [libevent 2.0.21](https://packages.ubuntu.com/xenial-updates/libevent-2.0-5) only, the default bionic Docker image is used in the _"[no depends, only system libs, sanitizers: thread (TSan), no wallet]"_ CI test.

ACKs for top commit:
  theStack:
    utACK b68e717967
  laanwj:
    ACK b68e717967

Tree-SHA512: 9825c42aeb166165e99fe5eaf74dbb47c2b51aecdbe53c5ae949fe126e1b8e8b6fe8d228fdde4e8daa4243e5907954202f42eb23c71629e4b2b92a7d4eb892e4
2020-04-20 18:43:07 +02:00
Sebastian Falbesoner
c743718558 test: add further BIP37 size limit checks to p2p_filter.py
also unified method of detecting misbehaviour
(using assert_debug_log instead of checking peer's banscore)
2020-04-20 18:22:21 +02:00
practicalswift
69749fbe6a tests: Add fuzzing harness for Golomb-Rice coding (GolombRiceEncode/GolombRiceDecode) 2020-04-20 14:57:48 +00:00
MarcoFalke
fc00e651e4 Merge #18705: ci: Remove xenial tsan workaround
faebcd4e8d ci: Remove xenial tsan workaround (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK faebcd4e8d

Tree-SHA512: 5d7e15be211e526948f863f573dbb5a97005262241ba4a07858346cba3a17cb24b1473df347224e0e4f2b22201750e27fba80ffe0d1dddf85f4e8f9341a8c129
2020-04-20 10:16:12 -04:00
MarcoFalke
3be119c0f6 Merge #17579: [refactor] Merge getreceivedby tally into GetReceived function
a1d5b12ec0 Merge getreceivedby tally into GetReceived function (Andrew Toth)

Pull request description:

  This PR merges the tally code of `getreceivedbyaddress` and `getreceivedbylabel` into a single function `GetReceived`. This reduces repeated code and makes it similar to `listreceivedbyaddress` and `listreceivedbylabel`, which use the function `ListReceived`. It will also make the change in #14707 simpler and easier to review.

ACKs for top commit:
  theStack:
    re-ACK a1d5b12ec0
  meshcollider:
    utACK a1d5b12ec0

Tree-SHA512: 43d9cd92f7c2c6a8b9c7509aa85a9b9233a6cfec1c43a9062e3bdfb83515413d1feafa8938c828351278ba22bd31c47e62ab5341e4bddc2493103b094d73b047
2020-04-20 10:05:32 -04:00
Hennadii Stepanov
b91e4ae0d8 Do not expose and consider -logthreadnames when it does not work 2020-04-20 14:17:49 +03:00
MarcoFalke
5e5dd9918e Merge #17831: rpc: doc: Fix and extend getblockstats examples
709998467e rpc: doc: Fix and extend getblockstats examples (Adam Soltys)

Pull request description:

  This pull fixes the example curl command for `getblockstats` which doesn't work as is because it's missing a comma between the params and has single quotes around the second parameter.

  It also adds an additional example of getting block stats by hash by using a known workaround (#15412) to get bitcoin-cli to treat the hash parameter as JSON instead of a string since there is ongoing deliberation about how or whether to fix the root issue (#15448).

ACKs for top commit:
  theStack:
    ACK 709998467e

Tree-SHA512: 84a5b7f449f06fff785bc0afbc1a7dfd55454bc76c52a8945e91556f87f3edfdc5a1780faab8fcfd6c415b734295b7c67d2e04ba7b6cfa91a77758af5dda53ae
2020-04-20 07:15:45 -04:00
MarcoFalke
da4cbb7927 Merge #18544: net: limit BIP37 filter lifespan (active between 'filterload'..'filterclear')
a9ecbdfcaa test: add more inactive filter tests to p2p_filter.py (Sebastian Falbesoner)
5eae034996 net: limit BIP37 filter lifespan (active between 'filterload' and 'filterclear') (Sebastian Falbesoner)

Pull request description:

  This PR fixes https://github.com/bitcoin/bitcoin/issues/18483. On the master branch, there is currently _always_ a BIP37 filter set for every peer: if not a specific filter is set through a `filterload` message, a default match-everything filter is instanciated and pointed to via the `CBloomFilter` default constructor; that happens both initially, when the containing structure `TxRelay` is constructed:

  c0b389b335/src/net.h (L812)

  and after a loaded filter is removed again through a `filterclear` message:

  c0b389b335/src/net_processing.cpp (L3201)

  The behaviour was introduced by commit 37c6389c5a (an intentional covert fix for [CVE-2013-5700](https://github.com/bitcoin/bitcoin/pull/18515), according to gmaxwell).

  This default match-everything filter leads to some unintended side-effects:
  1. `getdata` request for filtered blocks (i.e. type `MSG_FILTERED_BLOCK`) are always responded to with `merkleblock`s, even if no filter was set by the peer, see issue #18483 (strictly speaking, this is a violation of BIP37) c0b389b335/src/net_processing.cpp (L1504-L1507)
  2. if a peer sends a `filteradd` message without having loaded a filter via `filterload` before, the intended increasing of the banscore never happens (triggered if `bad` is set to true, a few lines below) c0b389b335/src/net_processing.cpp (L3182-L3186)

  This PR basically activates the `else`-branch code paths for all checks of `pfilter` again (on the master branch, they are dead code) by limiting the pointer's lifespan: instead of always having a filter set, the `pfilter` is only pointing to a `CBloomFilter`-instance after receiving a `filterload` message and the instance is destroyed again (and the pointer nullified) after receiving a `filterclear` message.

  Here is a before/after comparison in behaviour:
  | code part / scenario                          |    master branch                   |   PR branch                                          |
  | --------------------------------------------- | ---------------------------------- | ---------------------------------------------------- |
  | `getdata` processing for `MSG_FILTERED_BLOCK` | always responds with `merkleblock` | only responds if filter was set via `filterload`     |
  | `filteradd` processing, no filter was loaded  | nothing                            | peer's banscore increases by 100 (i.e. disconnect)   |

  On the other code parts where `pfilter` is checked there is no change in the logic behaviour (except that `CBloomFilter::IsRelevantAndUpdate()` is unnecessarily called and immediately returned in the master branch).
  Note that the default constructor of `CBloomFilter` is only used for deserializing the received `filterload` message and nowhere else. The PR also contains a functional test checking that sending `getdata` for filtered blocks is ignored by the node if no bloom filter is set.

ACKs for top commit:
  MarcoFalke:
    re-ACK a9ecbdfcaa, only change is in test code 🕙

Tree-SHA512: 1a656a6d74ccaf628e7fdca063ba63fbab2089e0b6d0a11be9bbd387c2ee6d3230706ff8ffc1a55711481df3d4547137dd7c9d9184d89eaa43ade4927792d0b6
2020-04-20 06:59:53 -04:00
MarcoFalke
dc5da7fe7b Merge #18691: test: add wait_for_cookie_credentials() to framework for rpcwait tests
92fe537cf7 test: fix intermittent race condition in interface_bitcoin_cli.py (Jon Atack)
c648e636b2 test: add wait_for_cookie_credentials() to test framework (Jon Atack)

Pull request description:

  This PR adds a `wait_for_cookie_credentials()` method to the test framework and calls it before the `-rpcwait` tests, to avoid an intermittent race condition on the CI run with Valgrind where the cookie file isn't written yet when the CLI call with `-rpcwait` arrives to `get_auth_cookie()`.

  To reproduce/test, build with
  ```diff
  diff --git a/src/httprpc.cpp b/src/httprpc.cpp
  index 60c4d06f12..3dd06c4758 100644
  --- a/src/httprpc.cpp
  +++ b/src/httprpc.cpp
  @@ -291,6 +291,7 @@ static bool InitRPCAuthentication()
   bool StartHTTPRPC()
   {
       LogPrint(BCLog::RPC, "Starting HTTP RPC server\n");
  +    UninterruptibleSleep(std::chrono::seconds{11});
       if (!InitRPCAuthentication())
  ```
  then run the test normally and with valgrind
  ```
  test/functional/interface_bitcoin_cli.py -l debug
  valgrind test/functional/interface_bitcoin_cli.py -l debug
  ```
  Thanks to Marco Falke for all the help. Closes #18684.

Top commit has no ACKs.

Tree-SHA512: 1b76635b5b1d6b05138affef7ab788aa3bc3fc75b0c69ba778ecdf81063cfe02a8dd7667cfd63a6c6e19b2dac47d7a8b755e334d8af5c0ab9d4026808ee96c83
2020-04-20 06:13:59 -04:00
fanquake
20a30922fb doc: note why we can't use thread_local with glibc back compat 2020-04-20 16:58:54 +08:00
Jon Atack
92fe537cf7 test: fix intermittent race condition in interface_bitcoin_cli.py
by calling wait_for_cookie_credentials() to ensure the cookie file is written
and auth credentials available for testing the CLI -rpcwait option before the
RPC connection is up.
2020-04-19 18:12:05 +02:00
Jon Atack
c648e636b2 test: add wait_for_cookie_credentials() to test framework
to be able to ensure the cookie file is written and auth credentials available
when testing CLI/RPC commands before the RPC connection is up.
2020-04-19 18:12:03 +02:00
MarcoFalke
f8102d9088 Merge #18601: wallet: Refactor WalletRescanReserver to use wallet reference
fc289b7898 wallet: Refactor WalletRescanReserver to use wallet reference (João Barbosa)

Pull request description:

  Simple refactor to `WalletRescanReserver` to use wallet reference instead of pointer.

  Complements #18259.

ACKs for top commit:
  MarcoFalke:
    ACK fc289b7898

Tree-SHA512: b03e33f2d9df2870436aa3284137fd022dd89ea96a1b170fa27f8685ad4f986e6c4ba5975a84966c30d18430a4014d7d8740a1dff2f985c9ef8226ed18e69db9
2020-04-19 10:32:43 -04:00
MarcoFalke
d2882a012b Merge #18610: scripted-diff: test: replace command with msgtype (naming)
9df32e820d scripted-diff: test: replace command with msgtype (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR to https://github.com/bitcoin/bitcoin/pull/18533, which changed the naming of `strCommand` to `msg_type` in the network processing code. The same approach is done here for the function test framework, to get rid of the wrong "command" terminology for network mesage types. (Commands are usually used in the CLI or RPC context, so using the same name in the network message context would only be confusing.)

  The commit was created through the following steps:
  1. search for all occurences of the string "command" within the folder `test/functional`
  ```git grep -i command test/functional > command_finds```
  2. manually sort out all false-positives, i.e. occurences of "command" which describe commands in the correct sense (mostly CLI or RPC related, also some with Socks5)
  3. put the remaining occurences into a scripted-diff (a quite simple one, actually) that renames "command" to "msgtype" in the concerned files.

  The name `msgtype` was intentionally chosen without the underscore `_` as classes beginning with `msg_` define concrete types of messages.

ACKs for top commit:
  MarcoFalke:
    ACK 9df32e820d . Makes sense that tests use the same naming as Bitcoin Core. See `NetMsgType` here: https://doxygen.bitcoincore.org/namespace_net_msg_type.html

Tree-SHA512: cd0ee08a382910b7f10ce583acdaf4f8a39f9ba4a22434a914415727eedd98bac538de9bf6633574d5eb86f62558bc8dcb638a3289d99b04f8481f34e7a9a0c7
2020-04-19 09:18:21 -04:00
João Barbosa
fc289b7898 wallet: Refactor WalletRescanReserver to use wallet reference 2020-04-19 14:04:37 +01:00
MarcoFalke
faebcd4e8d ci: Remove xenial tsan workaround 2020-04-19 08:52:49 -04:00
MarcoFalke
fac2fc4dd8 test: Increase debugging to hunt down mempool_reorg intermittent failure 2020-04-19 08:40:20 -04:00
MarcoFalke
b470c75847 Merge #15761: Replace -upgradewallet startup option with upgradewallet RPC
0d32d66148 Remove -upgradewallet startup option (Andrew Chow)
92263cce5b Add upgradewallet RPC (Andrew Chow)
1e48796c99 Make UpgradeWallet a member function of CWallet (Andrew Chow)
c988f27937 Have UpgradeWallet take the version to upgrade to and an error message out parameter (Andrew Chow)
1833237123 Only run UpgradeWallet if the wallet needs to be upgraded (Andrew Chow)
9c16b1735f Move wallet upgrading to its own function (Andrew Chow)

Pull request description:

  `-upgradewallet` is largely incompatible with many recent wallet features and versions. For example, it was disabled if multiple wallets were used and would not work with encrypted wallets that were being upgraded to HD.

  This PR does away with the old method of upgrading upon startup and instead allows users to upgrade their wallets via an `upgradewallet` RPC. This does largely the same thing as the old `-upgradewallet` option but because the wallet is loaded, it can be unlocked to upgrade to HD. Furthermore it is compatible with multiwallet as it works on the individual wallet that is specified by the RPC.

ACKs for top commit:
  meshcollider:
    Code review ACK 0d32d66148
  darosior:
    ACK 0d32d66148
  MarcoFalke:
    ACK 0d32d66148 🚵

Tree-SHA512: b425bf6f5d605e26506889d63c780895482f07cbc086193218e031e8504d3072d41e90d65cd41bcc98ee4c1eb048954bc5d4ac85435f7394892373aac89a3b0a
2020-04-19 07:06:42 -04:00
MarcoFalke
a998c5185b Merge #18675: tests: Don't initialize PrecomputedTransactionData in txvalidationcache tests
3718ae2ef8 [tests] Don't initialize PrecomputedTransactionData in txvalidationcache tests (John Newbery)

Pull request description:

  PrecomputedTransactionData is initialized inside CheckInputScripts(). No need to pre-initialize it before calling into CheckInputScripts().

  Normally, I wouldn't bother, but we're making changes to `PrecomputedTransactionData` in #17977 which would break these tests without removing these constructions. Might as well get these changes out of the way here.

ACKs for top commit:
  robot-visions:
    ACK 3718ae2ef8
  sipa:
    utACK 3718ae2ef8

Tree-SHA512: bc9c095035a7072a2a91941df38cdbb969e817264efbaa6dcb88cc3ab132d9264aa0751fa588d1a5e45f37b4d2bb1903cda078765f0bbcc87d9cc47cbec5356a
2020-04-19 06:18:21 -04:00
MarcoFalke
b690b24eb2 Merge #18633: test: Properly raise FailedToStartError when rpc shutdown before warmup finished (take 2)
fa03713e13 test: Properly raise FailedToStartError when rpc shutdown before warmup finished (take 2) (MarcoFalke)

Pull request description:

  actually (?) fix #18561

  See most recent traceback https://travis-ci.org/github/bitcoin/bitcoin/jobs/674668692#L7062

  I believe the reason the error is still there is that ConnectionResetError is derived from OSError:

  ConnectionResetError(ConnectionError(OSError))

  And IOError is an alias for OSError since python 3.3, see https://docs.python.org/3/library/exceptions.html#IOError

  So fix that by renaming IOError to the alias OSError and move the less specific catch clause down a few lines.

ACKs for top commit:
  jonatack:
    ACK fa03713e13

Tree-SHA512: 6e5b214ed9101bf8ebe7472dcc1f9e9d128e2575c93ec00c8d0774ae1a9b52a8c2a653a45a0eab8d881570b08dd5ffeddf5aca88a10438c366e1f633253cb0b5
2020-04-19 06:10:06 -04:00
fanquake
d65631171c Merge #18695: test: Replace boost::mutex with std::mutex
27abd1a4f4 test: Replace boost::mutex with std::mutex (Hennadii Stepanov)

Pull request description:

  This PR replaces `boost::mutex` with `std::mutex` in the `scheduler_tests` test suite.

ACKs for top commit:
  theStack:
    ACK 27abd1a4f4
  sipa:
    utACK 27abd1a4f4

Tree-SHA512: 062eed360a68910fb71552fd892bfd097442718a237446cfb8350bfd5d807da7251ead2b9755e1d7022598774ed23fa5432a589ac6f8cadddab404b439883466
2020-04-19 11:53:25 +08:00
fanquake
315a4d36f7 build: fix ASLR for bitcoin-cli on Windows
ASLR is not currently working for the bitcoin-cli.exe binary. This is
due to it not having a .reloc section, which is stripped by default by
the mingw-w64 ld we use for gitian builds. A good summary of issues with
ld and mingw-w64 is available in this thread:
https://sourceware.org/bugzilla/show_bug.cgi?id=19011.

All other Windows binaries that we distribute (bitcoind, bitcoin-qt,
bitcoin-wallet, bitcoin-tx and test_bitcoin) do not suffer this issue,
and currently having working ASLR. This is due to them exporting
(inadvertent or not) libsecp256k1 symbols, and, as a result, the .reloc
section is not stripped by ld.

This change is a temporary workaround, also the same one described here:
https://www.kb.cert.org/vuls/id/307144/, that causes main() to be
exported. Exporting a symbol will mean that the .reloc section is not
stripped, and ASLR will function correctly.
2020-04-19 10:05:29 +08:00
MarcoFalke
fa03713e13 test: Properly raise FailedToStartError when rpc shutdown before warmup finished (take 2) 2020-04-18 14:58:58 -04:00
MarcoFalke
6ae99aab5d Merge #18692: test: Bump timeout in wallet_import_rescan
fabfcad876 test: Bump timeout in wallet_import_rescan (MarcoFalke)

Pull request description:

  Avoid timeouts when starting the node, also make error message more verbose

ACKs for top commit:
  practicalswift:
    ACK fabfcad876 -- patch looks correct

Tree-SHA512: 8fd60a05380349f521d0e814d2f268702dfbe57c7567a4f6e94435498dfdd32909179d75fded44757ecb1a93a4045842bc6d00bfd6cd18ba751513461359c7b0
2020-04-18 12:57:16 -04:00
Samuel Dobson
bbb1ba1814 Merge #17219: wallet: allow transaction without change if keypool is empty
92bcd70808 [wallet] allow transaction without change if keypool is empty (Sjors Provoost)
709f8685ac [wallet] CreateTransaction: simplify change address check (Sjors Provoost)
5efc25f963 [wallet] translate "Keypool ran out" message (Sjors Provoost)

Pull request description:

  Extracted from #16944

  First this PR simplifies the check when generating a change address, by dropping `CanGetAddresses` and just letting `reservedest.GetReservedDestination` do this check.

  Second, when the keypool is empty, instead of immediately giving up, we create a dummy change address and pass that to coin selection. If we didn't need the change address (e.g. when spending the entire balance), then it's all good. If we did need a change address, we throw the original error.

ACKs for top commit:
  fjahr:
    Code review ACK 92bcd70808
  jonasschnelli:
    utACK 92bcd70808
  achow101:
    ACK 92bcd70808
  meshcollider:
    Code review ACK 92bcd70808

Tree-SHA512: 07b8c8251f57061c58a85ebf0359be63583c23bac7a2c4cefdc14820c0cdebcc90a2bb218e5ede0db11d1e204cda149e056dfd18614642070b3d56efe2735006
2020-04-18 22:00:26 +12:00
Adam Soltys
709998467e rpc: doc: Fix and extend getblockstats examples
This fixes the example curl command for `getblockstats` which is missing
a comma between the params and has single quotes around the second
parameter.

Besides fixing the existing example, this commit adds an additional
example of getting block stats by hash by using a known workaround with
bitcoin-cli to get it to treat the hash parameter as a JSON string by
wrapping it in both single and double quotes.

Co-Authored-By: Andrew Toth <andrewstoth@gmail.com>
Co-Authored-By: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2020-04-17 20:40:08 -07:00
Hennadii Stepanov
27abd1a4f4 test: Replace boost::mutex with std::mutex 2020-04-18 01:51:05 +03:00
MarcoFalke
895c71e535 Merge #18682: fuzz: http_request workaround for libevent < 2.1.1
6f8b498d18 fuzz: http_request workaround for libevent < 2.1.1 (Sebastian Falbesoner)

Pull request description:

  The fuzz test `http_request` calls the following two internal libevent functions:
  * `evhttp_parse_firstline_`
  * `evhttp_parse_headers_`

  Before libevent 2.1.1 however, internal functions names didn't end with an underscore (see libevent commit 8ac3c4c25b and [Changelog for 2.1.1.-alpha](https://github.com/libevent/libevent/blob/master/ChangeLog#L1830) when the change was first mentioned) hence the build fails with a linking error.
  This PR adds a preprocessor workaround to the test that checks for the libevent version (via ~`_EVENT_NUMERIC_VERSION`~ `LIBEVENT_VERSION_NUMBER`) and creates wrapper functions mapping to naming scheme without underscore in case the version is older than 2.1.1.

  Tested with Ubuntu Xenial 16.04.6 LTS and clang-8.

ACKs for top commit:
  hebasto:
    ACK 6f8b498d18, tested on xenial:

Tree-SHA512: 3b9e0147b8aea22e417d418e3b6d4905f5be131c2b0ae4b0f8b9411c5606d2e22f1b23e1ecc6980ecab907c61404de09e588aae1ac43cf70cf9e8d006bbdee73
2020-04-17 17:17:11 -04:00
MarcoFalke
fabfcad876 test: Bump timeout in wallet_import_rescan 2020-04-17 17:09:27 -04:00
MarcoFalke
fa66280396 fuzz: Run in parallel 2020-04-17 15:43:23 -04:00
MarcoFalke
c54295c1a2 Merge #18641: test: Create cached blocks not in the future
fa32097541 test: Create cached blocks not in the future (MarcoFalke)

Pull request description:

  This avoids test failures when tests assume blocks are not from the future, like in wallet_dump: https://cirrus-ci.com/task/6607130193035264?command=ci#L3306

ACKs for top commit:
  jonatack:
    ACK fa32097541

Tree-SHA512: 60b6882e0e1df8c5d67f034533407a45d3685983891b67ff4631072bfd0a93a325c7ca18758d7a2df252e4fcdb7c87321cb1e84458b22782e57e719eec634c22
2020-04-17 14:04:14 -04:00
MarcoFalke
d7bdba460c Merge #18683: ci: Disable valgrind functionl tests on forked repos to avoid timeouts
faceeae49a ci: Disable valgrind functionl tests on forked repos to avoid timeouts (MarcoFalke)

Pull request description:

  Allows people to fork our repo and run the tests again

  Also print more cache stats

ACKs for top commit:
  hebasto:
    ACK faceeae49a, tested on my own repo: https://travis-ci.org/github/hebasto/bitcoin/jobs/676257500

Tree-SHA512: 50e44edf94fcb997438eeaf7308b2b58a0854141ecb1fbb0ba7bf5ed662f19b60899f966f579cca90ad5e789234d0e90122d8c2c854da70339058adc3a475fa6
2020-04-17 14:00:04 -04:00
Sebastian Falbesoner
6f8b498d18 fuzz: http_request workaround for libevent < 2.1.1
Before libevent 2.1.1, internal functions names didn't end with an underscore.
2020-04-17 19:00:19 +02:00
MarcoFalke
244daa4821 Merge #18607: rpc: Fix named arguments in documentation
fa168d7542 rpc: Document all aliases for first arg of listtransactions (MarcoFalke)
fa5b1f067f rpc: Document all aliases for second arg of getblock (MarcoFalke)
fa86a4bbfc rpc: Rename first arg of generateblock RPC to "output" (MarcoFalke)

Pull request description:

  This fixes a bug found with #18531:

  * Currently the named argument for `generateblock` is documented as `address/descriptor`, but the server only accepts a named argument of `address`. Fix it by changing the name to `output` for both the documentation and the server code. Also, add tests to prove the server understands the new name `output`.

  * Unrelated to that, there have been a bunch of aliases in the server code that are not present in the source code of the documentation. Fix that by adding the alias to the source code of the documentation. Only the first alias is displayed in the rendered documentation. Also, add tests to prove the server actually understands all aliases.

ACKs for top commit:
  pierreN:
    Tested ACK fa168d7 tests, help messages

Tree-SHA512: 05e15628e3a667b296f3783d20f764b450b959451b5360c7eaf5993156582d47a0f5882330ca2493b851eb46324d504953b90c875bc88a15c9e8c89eb3ef8d92
2020-04-17 12:16:42 -04:00
MarcoFalke
fa1fdb02fc bench: Replace ::mempool globabl with test_setup.mempool
This is a refactor, since they are aliases for each other
2020-04-17 10:20:54 -04:00
MarcoFalke
fab1170964 bench: Remove requirement that all benches use RegTestingSetup 2020-04-17 10:19:32 -04:00
MarcoFalke
54f812d9d2 Merge #18673: scripted-diff: Sort test includes
fa4632c417 test: Move boost/stdlib includes last (MarcoFalke)
fa488f131f scripted-diff: Bump copyright headers (MarcoFalke)
fac5c37300 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c417
  jonatack:
    ACK fa4632c417, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2020-04-17 10:12:13 -04:00
MarcoFalke
ecc2e4e363 Merge #18664: fuzz: fix unused variable compiler warning
eab7367e25 fuzz: fix unused variable compiler warning (Jon Atack)

Pull request description:

  Fixes the compiler warning while hopefully not invalidating the existing seeds. Added an explanatory comment.
  ```
  test/fuzz/locale.cpp:59:19: warning: unused variable 'random_int32' [-Wunused-variable]
      const int32_t random_int32 = fuzzed_data_provider.ConsumeIntegral<int32_t>();
  ```

ACKs for top commit:
  practicalswift:
    ACK eab7367e25

Tree-SHA512: 4c90784518027cd3f85acd18030201efe4018f9da46365fef934e9a53a0b923031fec4c884a2da2f14232b6060aeb9016ac09950a18e31395de048548ecbc836
2020-04-17 09:09:59 -04:00
Hennadii Stepanov
2aa48edec0 refactor: Drop unused ${WRAP_DIR}/${HOST} directory
This commit removes the directory that is no longer used since #16667.
2020-04-17 16:09:11 +03:00
Hennadii Stepanov
1362be0447 build: Drop make dist in gitian builds 2020-04-17 16:09:04 +03:00
MarcoFalke
faceeae49a ci: Disable valgrind functionl tests on forked repos to avoid timeouts 2020-04-17 09:00:43 -04:00
MarcoFalke
c2e53ff064 Merge #18467: rpc: Improve documentation and return value of settxfee
38677274f9 rpc: settxfee respects -maxtxfee wallet setting (Fabian Jahr)
bda84a08a0 rpc: Add documentation for deactivating settxfee (Fabian Jahr)

Pull request description:

  ~~Closes 18315~~

  `settxfee` can be deactivated by passing 0 as the fee. That does not seem to be documented for the user so this PR adds it in the description. The return value of a simple boolean seems also too simplified given the multiple dimensions that this deactivation feature enables. I.e. it does not seem intuitive if the returned boolean shows that the call succeeded or if means that `settxfee` is active. My suggested solution is to change the return value to a JSON object that included the "active" state and the currently set fee rate.

  Examples:
  ```
  $ src/bitcoin-cli settxfee 0.0000000
  {
    "active": false,
    "fee_rate": "0.00000000 BTC/kB"
  }
  $ src/bitcoin-cli settxfee 0.0001
  {
    "active": true,
    "fee_rate": "0.00010000 BTC/kB"
  }
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 38677274f9, seems useful to error out early instead of later #16257 🕍
  jonatack:
    ACK 38677274f9
  meshcollider:
    LGTM, utACK 38677274f9

Tree-SHA512: 642813b5cf6612abb4b6cb63728081a6bd1659d809e0149c8f56060b6da7253fee989b3b202854f3051df3773c966799af30b612648c466b099f00590f356548
2020-04-17 07:55:55 -04:00
Jon Atack
eab7367e25 fuzz: fix unused variable compiler warning 2020-04-17 13:45:43 +02:00
Samuel Dobson
c189bfd260 Merge #17824: wallet: Prefer full destination groups in coin selection
a2324e4d3f test: Improve naming and logging of avoid_reuse tests (Fabian Jahr)
1abbdac677 wallet: Prefer full destination groups in coin selection (Fabian Jahr)

Pull request description:

  Fixes #17603 (together with #17843)

  In the case of destination groups of >10 outputs existing in a wallet with `avoid_reuse` enabled, the grouping algorithm is adding left-over outputs as an "incomplete" group to the list of groups even when a full group has already been added. This leads to the strange behavior that if there are >10 outputs for a destination the transaction spending from that will effectively use `len(outputs) % 10` as inputs for that transaction.

  From the original PR and the code comment I understand the correct behavior should be the usage of 10 outputs. I opted for minimal changes in the current code although there maybe optimizations possible for cases with >20 outputs on a destination this sounds like too much of an edge case right now.

ACKs for top commit:
  jonatack:
    Re-ACK a2324e4
  achow101:
    ACK a2324e4d3f
  kallewoof:
    ACK a2324e4d3f
  meshcollider:
    Tested ACK a2324e4d3f (verified the new test fails on master without this change)

Tree-SHA512: 4743779c5d469fcd16df5baf166024b1d3c8eaca151df1e8281b71df62b29541cf7bfee3f8ab48d83e3b34c9256e53fd38a7b146a54c79f9caa44cce3636971a
2020-04-17 23:05:48 +12:00
Hennadii Stepanov
b68e717967 build: Set libevent minimum version to 2.0.21 2020-04-17 13:53:34 +03:00
MarcoFalke
4a71c46905 Merge #18670: refactor: Remove unused methods CBloomFilter::reset()/clear()
69ffddc83e refactor: Remove unused methods CBloomFilter::reset()/clear() (Sebastian Falbesoner)

Pull request description:

  The method `CBloomFilter::reset()` was introduced by commit d2d7ee0e86 in 2015, but was never ever used, as far as I could find. As discovered by MarcoFalke, the method `clear()` is also unused outside of unit tests and is hence also removed.

ACKs for top commit:
  MarcoFalke:
    re-ACK 69ffddc83e
  jonatack:
    ACK 69ffddc83e, code review, compiled a fuzz build and started the bloom_filter fuzz test as a sanity check.
  promag:
    ACK 69ffddc83e.

Tree-SHA512: 6c53678545ad8e2fa1ffc0a8838e450462f26748a60632f738dc020f0eb494ae2c32841e6256e266ed9140177257a78b707123421942f3819a14ffcb9a99322f
2020-04-17 06:49:00 -04:00
MarcoFalke
fa4632c417 test: Move boost/stdlib includes last 2020-04-17 06:36:04 -04:00
Samuel Dobson
0856c15706 Merge #18262: bnb: exit selection when best_waste is 0
9b5950db86 bnb: exit selection when best_waste is 0 (Andrew Chow)

Pull request description:

  If we find a solution which has no waste, just use that. This solution
  is what we would consider to be optimal, and other solutions we find
  would have to also have 0 waste, so they are equivalent to the first
  one with 0 waste. Thus we can optimize by just choosing the first one
  with 0 waste.

  Closes #18257

ACKs for top commit:
  instagibbs:
    utACK 9b5950db86
  meshcollider:
    utACK 9b5950db86

Tree-SHA512: 59565ff4a3d8281e7bc0ce87065a34c8d8bf8a95f628ba96b4fe89f1274979165aea6312e5f1f21b418c8c484aafc5166d22d9eff9d127a8192498625d58c557
2020-04-17 22:32:13 +12:00
Saahil Shangle
bda62e87e6 Adding build instructions to Bitcoin Core, fixes #18658 2020-04-16 21:01:00 -07:00
MarcoFalke
fa60afc4fb wallet: Add BlockUntilSyncedToCurrentChain to dumpwallet 2020-04-16 19:15:32 -04:00
Sebastian Falbesoner
69ffddc83e refactor: Remove unused methods CBloomFilter::reset()/clear()
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-04-17 01:09:39 +02:00
MarcoFalke
969ee85494 Merge #18662: test: Replace gArgs with local argsman in bench
faf989f936 util: Document why ArgsManager (con/de)structor is not inline (MarcoFalke)
fae00a77e2 bench: Remove unused argsman.ClearArgs (MarcoFalke)
fa46aebeb1 scripted-diff: Replace gArgs with local argsman in bench (MarcoFalke)
fa2bc4141d tools: Add unused argsman to bench_bitcoin (MarcoFalke)

Pull request description:

  All utilities use the same gArgs global that the node uses. This is generally fine and does not lead to issues unless a bench test is going to spin up a NodeContext via the TestingSetup. In that case the two uses of gArgs conflict and currently it needs to be cleared:

  544709763e/src/bench/bench_bitcoin.cpp (L76)

  One solution would be to do nothing, because the current code works with that workaround. Another solution would be to not use the same global in all binaries.

ACKs for top commit:
  promag:
    ACK faf989f936.
  ryanofsky:
    Code review ACK faf989f936. Just new commit added restoring forward declaration

Tree-SHA512: 8ee4b28eee294d41c002f801fa844b0c23c919a3061f5109638701db0947b3b0ea28caa7311ae5f126fc660648bbaa0890853e6b06bdc5868692f52ba8c05f66
2020-04-16 16:37:27 -04:00
Wladimir J. van der Laan
8f2497941e Merge #18598: gitian: Add missing automake package to gitian-win-signer.yml
e44aeefaae gitian: Add missing automake package to gitian-win-signer.yml (Andrew Chow)

Pull request description:

  automake is needed to build osslsigncode otherwise autogen.sh fails with the docker virtualization method.

ACKs for top commit:
  hebasto:
    ACK e44aeefaae, for `osslsigncode-1.7.1` we did not run `autogen.sh` in the past.
  fanquake:
    ACK e44aeefaae
  jonatack:
    ACK e44aeef

Tree-SHA512: a0e615c1b099ee1c469ce41f886f2ece6746234a5a800743a4e8be671e4114fd30e1c35bc0ddcb75778409564129d0fde7ac4e3d70b0f7691f97f729f34c8e0c
2020-04-16 22:27:46 +02:00
MarcoFalke
447f8676b2 Merge #18667: ci: Limit cache size regardless of NO_DEPENDS
0c6318788b ci: Limit cache size regardless of NO_DEPENDS (Hennadii Stepanov)

Pull request description:

  Close #18666.

ACKs for top commit:
  MarcoFalke:
    ACK 0c6318788b . Depends has ccache disabled anyway and is cached regardless of whether ccache is there or not, see #17248

Tree-SHA512: b1bf98be0f844b4704abd177841b014f3900be8160496f0d12596310db607b4f544547e8c3cbfcf17c086a78afd251653363f3dd467b769ac0062bc19adc8144
2020-04-16 16:12:30 -04:00
John Newbery
3718ae2ef8 [tests] Don't initialize PrecomputedTransactionData in txvalidationcache tests
PrecomputedTransactionData is initialized inside CheckInputScripts(). No need
to pre-initialize it before calling into CheckInputScripts().
2020-04-16 15:20:57 -04:00
MarcoFalke
fa488f131f scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-04-16 13:33:09 -04:00
MarcoFalke
fac5c37300 scripted-diff: Sort test includes
-BEGIN VERIFY SCRIPT-
 # Mark all lines with #includes
 sed -i --regexp-extended -e 's/(#include <.*>)/\1 /g' $(git grep -l '#include' ./src/bench/ ./src/test ./src/wallet/test/)
 # Sort all marked lines
 git diff -U0 | ./contrib/devtools/clang-format-diff.py -p1 -i -v
-END VERIFY SCRIPT-
2020-04-16 13:32:36 -04:00
MarcoFalke
d8dfcea5d9 Merge #17669: tests: have coins simulation test also use CCoinsViewDB
bee88b8c58 tests: have coins simulation test also use CCoinsViewDB (James O'Beirne)

Pull request description:

  Before this change, the coins simulation test uses a base view of type
  CCoinsViewTest, which has no relevance outside of the unittest suite. Might as
  well reuse this testcase with a more realistic configuration that has
  CCoinsViewDB (i.e. in-memory leveldb) at the bottom of the view structure.

  This adds explicit use of CCoinsViewDB in the unittest suite.

  #### Before change
  ```
  ./src/test/test_bitcoin --run_test=coins_tests --catch_system_errors=no  21.99s user 0.04s system 99% cpu 22.057 total
  ```

  #### After change
  ```
  ./src/test/test_bitcoin --run_test=coins_tests --catch_system_errors=no  78.80s user 0.04s system 100% cpu 1:18.82 total
  ```

ACKs for top commit:
  ryanofsky:
    Code review ACK bee88b8c58

Tree-SHA512: 75296b2bcbae2f46e780489aafb032592544a15c384d569d016005692fe79fe60d7f05857cf25cc7b0f9ab1c53b47886a6c71cca074a03fb9afec30e1f376858
2020-04-16 12:56:36 -04:00
MarcoFalke
faf989f936 util: Document why ArgsManager (con/de)structor is not inline 2020-04-16 12:26:09 -04:00
Kristaps Kaupe
2b18fd2242 Disable unavailable context menu items in transactions tab 2020-04-16 19:06:58 +03:00
MarcoFalke
f4c0ad4aef Merge #18660: test: Verify findCommonAncestor always initializes outputs
9986608ba9 test: Verify findCommonAncestor always initializes outputs (Russell Yanofsky)

Pull request description:

  Also add code comment to clarify surprising code noted by practicalswift
  https://github.com/bitcoin/bitcoin/pull/18657#issuecomment-614278450

ACKs for top commit:
  MarcoFalke:
    ACK 9986608ba9
  jonatack:
    ACK 9986608ba9 modulo @practicalswift's https://github.com/bitcoin/bitcoin/pull/18660#issuecomment-614487724

Tree-SHA512: d79c910291d68b770ef4b09564d274c0e19a6acf43ef1a6691dc889e3944ab3462b86056eeb794fd0c6f2464cfad6cc00711a833f84b32079c69ef9b3c8da24c
2020-04-16 11:44:03 -04:00
MarcoFalke
faec063887 log: Use Join() helper when listing log categories 2020-04-16 11:08:46 -04:00
Hennadii Stepanov
0c6318788b ci: Limit cache size regardless of NO_DEPENDS 2020-04-16 17:56:21 +03:00
MarcoFalke
661e8df1b6 Merge #18653: test: add coverage for bitcoin-cli -rpcwait
c28c7b882b test: add -getinfo "unlocked_until" and "headers" coverage (Jon Atack)
bb13f46fb1 test: verify cli.getwalletinfo in wallet section (Jon Atack)
becc8b9747 test: verify bitcoin-cli -version with node stopped (Jon Atack)
727b67e7b8 test: add coverage for bitcoin-cli -rpcwait (Jon Atack)

Pull request description:

  and other coverage improvements in `interface-bitcoin_cli.py`

ACKs for top commit:
  kristapsk:
    ACK c28c7b882b

Tree-SHA512: c3ac73b673872cba05496e3f9debc2c4dfa4291c6426bf0be9bb0242035613b25c0aecebb39abb14a1d87b13e1d28a5f34b21d4b77d93de56c679a8be92bbe82
2020-04-16 10:11:12 -04:00
MarcoFalke
79b0459648 Merge #18650: qt: Make bitcoin.ico non-executable
a95af77eb2 qt: Make bitcoin.ico non-executable (practicalswift)

Pull request description:

  Make `bitcoin.ico` non-executable.

  No need to execute icons and having +x bits laying around breaks `find … -executable` :)

  Before this patch:

  ```sh
  $ find $(git ls-files -- ":(exclude)*.sh" ":(exclude)*.py") -executable
  ci/retry/retry
  contrib/macdeploy/macdeployqtplus
  depends/config.guess
  depends/config.sub
  src/qt/res/icons/bitcoin.ico
  src/secp256k1/src/modules/recovery/main_impl.h
  ```

  After this patch:

  ```sh
  $ find $(git ls-files -- ":(exclude)*.sh" ":(exclude)*.py") -executable
  ci/retry/retry
  contrib/macdeploy/macdeployqtplus
  depends/config.guess
  depends/config.sub
  src/secp256k1/src/modules/recovery/main_impl.h
  ```

  FWIW:

  ```
  $ file $(find $(git ls-files -- ":(exclude)*.sh" ":(exclude)*.py") -executable)
  ci/retry/retry:                                 Bourne-Again shell script, UTF-8 Unicode text executable
  contrib/macdeploy/macdeployqtplus:              Python script, ASCII text executable
  depends/config.guess:                           POSIX shell script, ASCII text executable
  depends/config.sub:                             POSIX shell script, ASCII text executable
  src/qt/res/icons/bitcoin.ico:                   MS Windows icon resource - 10 icons, 48x48, 16 colors, 4 bits/pixel, 32x32, 16 colors, 4 bits/pixel
  src/secp256k1/src/modules/recovery/main_impl.h: C source, ASCII text
  ```

ACKs for top commit:
  MarcoFalke:
    ACK a95af77eb2 gitian build finished, so it doesn't look like the icon used in Windows resource files needs to be executable. Though, I didn't read the documentation.
  jonatack:
    ACK a95af77eb2

Tree-SHA512: ecf9154077824ae4c274b4341e985797f3648c0cb0c31cb25ce382163b923a3acbc7048683720be4ae3663501801129cd0f48c441a36f049cc304ebe9f30994e
2020-04-16 09:59:23 -04:00
MarcoFalke
e16718a8b3 Merge #18401: Refactor: Initialize PrecomputedTransactionData in CheckInputScripts
f63dec189c [REFACTOR] Initialize PrecomputedTransactionData in CheckInputScripts (Pieter Wuille)

Pull request description:

  This is a single commit taken from the Schnorr/Taproot PR #17977.

  Add a default constructor to `PrecomputedTransactionData`, which doesn't initialize the struct's members. Instead they're initialized inside the `CheckInputScripts()` function. This allows a later commit to add the spent UTXOs to that structure. The spent UTXOs are required for the schnorr signature hash, since it commits to the scriptPubKeys. See https://github.com/bitcoin/bips/blob/master/bip-0341.mediawiki#common-signature-message for details.

  By itself, this isn't really an improvement to the code, but I think it makes sense to separate out the refactor/moveonly commits from PR #17977 so that PR is only the logical changes needed for Schnorr/Taproot.

ACKs for top commit:
  jonatack:
    Re-ACK f63dec1  `git diff 851908d f63dec1` shows no change since last ACK.
  sipa:
    utACK f63dec189c
  theStack:
    re-ACK f63dec189c
  fjahr:
    Re-ACK f63dec189c
  ariard:
    Code Review ACK f63dec1

Tree-SHA512: ecf9154077824ae4c274b4341e985797f3648c0cb0c31cb25ce382163b923a3acbc7048683720be4ae3663501801129cd0f48c441a36f049cc304ebe9f30994e
2020-04-16 08:51:54 -04:00
MarcoFalke
fa168d7542 rpc: Document all aliases for first arg of listtransactions 2020-04-16 08:45:33 -04:00
MarcoFalke
fa5b1f067f rpc: Document all aliases for second arg of getblock 2020-04-16 08:45:05 -04:00
MarcoFalke
fae00a77e2 bench: Remove unused argsman.ClearArgs 2020-04-15 20:33:36 -04:00
MarcoFalke
fa46aebeb1 scripted-diff: Replace gArgs with local argsman in bench
-BEGIN VERIFY SCRIPT-
sed -i -e 's/gArgs/argsman/g' src/bench/bench_bitcoin.cpp
-END VERIFY SCRIPT-
2020-04-15 20:32:30 -04:00
MarcoFalke
fa2bc4141d tools: Add unused argsman to bench_bitcoin 2020-04-15 20:22:18 -04:00
MarcoFalke
544709763e Merge #18571: fuzz: Disable debug log file
fa69f88486 fuzz: Disable debug log file (MarcoFalke)
fa0cbd48c4 test: Add optional extra_args to testing setup (MarcoFalke)
fad4fa7e2f node: Add args alias for gArgs global (MarcoFalke)

Pull request description:

  There are several issues with writing to a debug log file when fuzzing:

  * Disk access is slow, but fuzzing should be fast (Note: I could not verify this claim with data)
  * Disks have a limited size and will eventually run out of space, but fuzzing should run continuous

  Fix both issues by disabling the debug log file for fuzz tests

ACKs for top commit:
  practicalswift:
    ACK fa69f88486 -- patch looks correct

Tree-SHA512: f61beb6c94a9ab664deb191685fcad601e228b77bb1c43db6ec40616ae393c9dd35c51474f1b0759ac0bc29b5ca8456a329906a3695bd0f18fa4372210c8b54a
2020-04-15 17:41:56 -04:00
Russell Yanofsky
9986608ba9 test: Verify findCommonAncestor always initializes outputs
Also add code comment to clarify surprising code noted by practicalswift
https://github.com/bitcoin/bitcoin/pull/18657#issuecomment-614278450
2020-04-15 17:07:44 -04:00
MarcoFalke
e84a5f0004 Merge #18645: [doc] Update thread information in developer docs
808ef36b89 [doc] Update thread information in developer docs (John Newbery)

Pull request description:

  - DumpAddresses thread was removed in #5964
  - Shutdown thread was removed in #5679
  - Add new threads (scheduler, RPC worker, indexer, tor control)
  - Small changes to documentation of other threads

ACKs for top commit:
  MarcoFalke:
    ACK 808ef36b89
  hebasto:
    ACK 808ef36b89.

Tree-SHA512: 85b6ace7bcc4dee030c63461bef1ded1a9581d4fa249c59f6fcd5d33d89c4357a6b8b35888ce0960f276d397b5e38a21e6c5d4b7b79544827a28c950e097b36d
2020-04-15 16:30:11 -04:00
MarcoFalke
6db8ef2431 Merge #18640: appveyor: Remove clcache
fac0c8db9f appveyor: Remove clcache (MarcoFalke)

Pull request description:

  The build time without cache seems to be 47 minutes and with cache 46 minutes. Maybe we can save more time by not installing clcache.

Top commit has no ACKs.

Tree-SHA512: ce503641a465f5e49deb7aa6b566aaab5e567148c001704451891c49ba7ade3219ca788bc0d8e55565febb9aadd579e26894f6c03b99755eb8b5d1498acf4dc9
2020-04-15 16:19:52 -04:00
MarcoFalke
fa69f88486 fuzz: Disable debug log file 2020-04-15 15:13:11 -04:00
MarcoFalke
fa0cbd48c4 test: Add optional extra_args to testing setup 2020-04-15 15:13:04 -04:00
MarcoFalke
fad4fa7e2f node: Add args alias for gArgs global 2020-04-15 15:05:18 -04:00
MarcoFalke
4bd6bc5cb4 Merge #18615: test: Avoid accessing free'd memory in validation_chainstatemanager_tests
fa176e253f test: Avoid accessing free'd memory in validation_chainstatemanager_tests (MarcoFalke)

Pull request description:

ACKs for top commit:
  ryanofsky:
    Code review ACK fa176e253f, though if you have to update this again, would suggest separating txindex test cleanup and the chainstatemanager test fix in separate commits, or identifying which part of the change is the bugfix fix in the commit description. Also to clean up the txindex test it might make sense to call SyncWithValidationInterfaceQueue in the test destructor to prevent nondeterminism in other tests

Tree-SHA512: 34c5dca283a7c205cd42b6aa59f00a71fd1bd980bc3d6640a18b280be11470bfabb2fd8c93fadde6fb8e084bcf96c80ec3aa72bbccccfde8a8260d173eaad08f
2020-04-15 14:38:02 -04:00
John Newbery
808ef36b89 [doc] Update thread information in developer docs 2020-04-15 12:02:39 -04:00
MarcoFalke
fa176e253f test: Avoid accessing free'd memory in validation_chainstatemanager_tests 2020-04-15 11:46:24 -04:00
Jon Atack
c28c7b882b test: add -getinfo "unlocked_until" and "headers" coverage 2020-04-15 16:40:24 +02:00
Jon Atack
bb13f46fb1 test: verify cli.getwalletinfo in wallet section 2020-04-15 16:40:22 +02:00
Jon Atack
becc8b9747 test: verify bitcoin-cli -version with node stopped
in interface_bitcoin_cli.py and improve/harmonize the test logging.
2020-04-15 16:40:15 +02:00
Jon Atack
727b67e7b8 test: add coverage for bitcoin-cli -rpcwait
in interface_bitcoin_cli.py
2020-04-15 16:39:42 +02:00
MarcoFalke
ccccd51908 script: Remove undocumented and unused operator+ 2020-04-15 10:01:55 -04:00
Sebastian Falbesoner
9df32e820d scripted-diff: test: replace command with msgtype
This is the functional test framework pendant for
7777e3624f, which renamed "strCommand" with
"msg_type" in the network processing code.

-BEGIN VERIFY SCRIPT-
 # Rename in test framework
 sed -i 's/command/msgtype/g' ./test/functional/test_framework/messages.py ./test/functional/test_framework/mininode.py
 # Rename in individual tests
 sed -i 's/command/msgtype/g' ./test/functional/p2p_invalid_messages.py ./test/functional/p2p_leak.py
-END VERIFY SCRIPT-
2020-04-15 15:41:49 +02:00
MarcoFalke
20c0e2e0f0 Merge #18628: test: Add various low-level p2p tests
fa4c29bc1d test: Add various low-level p2p tests (MarcoFalke)

Pull request description:

ACKs for top commit:
  jonatack:
    ACK fa4c29bc1d

Tree-SHA512: 842821b97359d4747c763398f7013415858c18a300cd882887bc812d039b5cbb67b9aa6f68434575dbc3c52f7eb8c43d1b293a59555a7242c0ca615cf44dc0aa
2020-04-15 08:26:34 -04:00
MarcoFalke
18f11fb24b Merge #18646: gui: use PACKAGE_NAME in exception message
1b04302e43 gui: use PACKAGE_NAME in exception message (fanquake)

Pull request description:

  Should address [this comment](https://github.com/bitcoin/bitcoin/issues/18643#issuecomment-613761345).

  ![qt_exception](https://user-images.githubusercontent.com/863730/79310875-348bf780-7f2f-11ea-9970-ba1ac28c7849.png)

  Can be triggered with a diff like:
  ```diff
  diff --git a/src/qt/modaloverlay.cpp b/src/qt/modaloverlay.cpp
  index 6243a71c7..e5d499012 100644
  --- a/src/qt/modaloverlay.cpp
  +++ b/src/qt/modaloverlay.cpp
  @@ -6,6 +6,7 @@
   #include <qt/forms/ui_modaloverlay.h>

   #include <qt/guiutil.h>
  +#include <util/check.h>

   #include <chainparams.h>

  @@ -179,6 +180,7 @@ void ModalOverlay::showHide(bool hide, bool userRequested)

   void ModalOverlay::closeClicked()
   {
  +    CHECK_NONFATAL(false);
       showHide(true);
       userClosed = true;
   }
  ```
  and clicking close in the modal.
  ```bash
  EXCEPTION: 18NonFatalCheckError
  qt/modaloverlay.cpp:183 (closeClicked)
  Internal bug detected: 'false'
  You may report this issue here: https://github.com/bitcoin/bitcoin/issues
  ```

ACKs for top commit:
  jonasschnelli:
    Thanks. utACK 1b04302e43
  emilengler:
    utACK 1b04302
  hebasto:
    ACK 1b04302e43.

Tree-SHA512: 20118269882fbbb5602dafe8101c6ca6c92d7b97efc1ef280a68a83c6a58340018d05f837ecabefee618dc11a62e82828d5ecde4350a71a4c2fd24257b90d45c
2020-04-15 07:30:39 -04:00
practicalswift
a95af77eb2 qt: Make bitcoin.ico non-executable 2020-04-15 10:59:10 +00:00
practicalswift
54b5eb2b14 tests: Add std::locale::global to list of locale dependent functions in lint-locale-dependence.sh
We currently flag `setlocale(...)` as locale dependent, but prior to this commit we didn't flag
`std::locale::global(...)` as such.

In addition to setting the global C++ locale `std::locale::global(...)` also does the equivalent
of `std::setlocale(LC_ALL, ...);`.

Thus the functionality of `std::locale::global(...)` is a superset of `setlocale(...)` :)
2020-04-15 10:05:03 +00:00
fanquake
e831f18b1e Merge #18619: gitian: add jonatack gpg key fingerprint
905e2e85ba gitian: add jonatack gpg key fingerprint (Jon Atack)

Pull request description:

  per request https://github.com/bitcoin-core/gitian.sigs/pull/1221#issuecomment-612778063

ACKs for top commit:
  laanwj:
    ACK 905e2e85ba
  fanquake:
    ACK 905e2e85ba

Tree-SHA512: bddd734f13c53859280db2fa94b47cbb2a8b3f17ed6a6fdda2bf04f7e201e310ae24930e8f4be2f8b65a949659a9d3369704ed70031da9653a66a513fe597b67
2020-04-15 15:38:02 +08:00
fanquake
1b04302e43 gui: use PACKAGE_NAME in exception message 2020-04-15 15:35:47 +08:00
fanquake
5447d57bff Merge #18624: Added my fingerprint Stephan Oeste (Emzy)
c47adf8df4 Added my fingerprint Stephan Oeste (Emzy) (Stephan Oeste)

Pull request description:

  By request from laanwj added my PGP fingerprint.
  See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442

ACKs for top commit:
  Sjors:
    ACK c47adf8. Fingerprint matches Twitter profile: https://twitter.com/emzy (haven't verified it in any other way)
  fanquake:
    ACK c47adf8df4

Tree-SHA512: 3e39ae88f507a12f11fb2d5c779eba79ee2daeddecd0dc3f1fddfa29ce963d0e9af3fa5a10357157812597c10205a6beae31cc70af9471a782da23d8753b7cbd
2020-04-15 15:31:40 +08:00
fanquake
ae486b263c Merge #17929: build: add linker optimisation flags to gitian & guix (Linux)
f2b5b0a3b4 build: add linker optimization flags to guix (fanquake)
b8b050a8d6 build: add linker optimization flags to gitian descriptors (fanquake)

Pull request description:

  This PR adds `-Wl,O2` to our gitian and guix LDFLAGS. This makes the linker perform certain optimisations (and is different from LTO).

  Any -O argument will enable optimizations in GNU ld. We can use -O2 here, as this matches our compile flags. Note that this would also enable additional optimizations if using the lld or gold linkers, when compared to -O0.

  A nice writeup + diagrams of some of these optimizations is  available here: http://lwn.net/Articles/192624/.

  #### master
  ```bash
  # bitcoind
  Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
   Length  Number     % of total  Coverage
        0  1          ( 33.3%)       0.0%
        1  0          (  0.0%)       0.0%
        2  1          ( 33.3%)      40.0%
        3  1          ( 33.3%)     100.0%
  ```
  ```bash
  # bitcoin-qt
  Histogram for `.gnu.hash' bucket list length (total of 3 buckets)
   Length  Number     % of total  Coverage
        0  0          (  0.0%)       0.0%
        1  1          ( 33.3%)      10.0%
        2  0          (  0.0%)      10.0%
        3  0          (  0.0%)      10.0%
        4  1          ( 33.3%)      50.0%
        5  1          ( 33.3%)     100.0%
  ```

  #### this PR:
  ```bash
  # bitcoind
  Histogram for `.gnu.hash' bucket list length (total of 8 buckets)
   Length  Number     % of total  Coverage
        0  3          ( 37.5%)       0.0%
        1  5          ( 62.5%)     100.0%
  ```
  ```bash
  # bitcoin-qt
  Histogram for `.gnu.hash' bucket list length (total of 19 buckets)
   Length  Number     % of total  Coverage
        0  9          ( 47.4%)       0.0%
        1  10         ( 52.6%)     100.0%
  ```

  #### GNU ld -O

  > If level is a numeric values greater than zero ld optimizes the output. This might take significantly longer and therefore probably should only be enabled for the final binary. At the moment this option only affects ELF shared library generation. Future releases of the linker may make more use of this option. Also currently there is no difference in the linker’s behaviour for different non-zero values of this option. Again this may change with future releases.

  #### lld -O

  > Optimize output file size

ACKs for top commit:
  dongcarl:
    ACK f2b5b0a3b4
  laanwj:
    ACK f2b5b0a3b4

Tree-SHA512: e53f3a4338317dbec65d3a93b57b5a6204aabdf9ac82d99447847a3c8627facc53c58c2cf947376f13edd979fc8129a80f18d9ebeccd191a576c83f1dad5c513
2020-04-15 15:17:04 +08:00
fanquake
903be99ee6 Merge #18621: script: Disallow silent bool -> CScript conversion
88884ee8d8 script: Disallow silent bool -> CScript conversion (MarcoFalke)

Pull request description:

  Makes nonsensical stuff like `ScriptToAsmStr(false,false);` a compile failure

ACKs for top commit:
  practicalswift:
    ACK 88884ee8d8
  laanwj:
    ACK 88884ee8d8
  promag:
    ACK 88884ee8d8.
  instagibbs:
    utACK 88884ee8d8
  jb55:
    ACK 88884ee8d8
  ryanofsky:
    Code review ACK 88884ee8d8

Tree-SHA512: 419d79c03b44a979c061b0540662928251ad68d53e65996bf370bb55ed1526ac7a22710cb7536c9954db5fec07bc312884bf8828f97a4ba180a5b07969a17f54
2020-04-15 14:56:40 +08:00
fanquake
4d793bcfe8 Merge #18632: doc: Fix macos comments in release-notes
fa6cb00ce5 doc: Fix macos comments in release-notes (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fa6cb00ce5.
  fanquake:
    ACK fa6cb00ce5 - this has also been done in the release notes in the wiki.

Tree-SHA512: ce325a9a32e28923ad34bdc7c1c69e9d14636a18d05604158e99f80683686bd2ba5550e84f85479336658b84c40dbe2940840e47d69c10ae3d41180dad49c8f3
2020-04-15 09:22:36 +08:00
MarcoFalke
fa32097541 test: Create cached blocks not in the future 2020-04-14 21:04:36 -04:00
MarcoFalke
fac0c8db9f appveyor: Remove clcache 2020-04-14 20:18:38 -04:00
MarcoFalke
75fcfdaf8b Merge #18631: appveyor: Disable functional tests for now
fa50d11dd1 appveyor: Disable functional tests for now (MarcoFalke)

Pull request description:

ACKs for top commit:
  jonasschnelli:
    ACK fa50d11dd1

Tree-SHA512: 86ea9bc0ff47dc309120748627cd83b19aa61fb1db3cca5e04a9e0d56f34c248cd2bec20b037c663ab0f21f7ee6cd0ff8266a36fe79c773de95e5032eb48b118
2020-04-14 17:35:39 -04:00
practicalswift
8d306862ef ci: Add fuzzbuzz integration 2020-04-14 16:38:26 +00:00
Sebastian Falbesoner
a9ecbdfcaa test: add more inactive filter tests to p2p_filter.py
check the following expected behaviors if no filter is set:
-> filtered block requests are ignored by the node
-> sending a 'filteradd' message is treated as misbehavior
   (i.e. the peer's banscore increases by 100)

also fixes a bug in the on_inv() callback method, which
directly modified the type from BLOCK to FILTERED_BLOCK
in the received 'inv' message rather than just for the reply

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-04-14 16:40:37 +02:00
MarcoFalke
fa6cb00ce5 doc: Fix macos comments in release-notes 2020-04-14 09:59:58 -04:00
Fabian Jahr
38677274f9 rpc: settxfee respects -maxtxfee wallet setting 2020-04-14 15:52:42 +02:00
MarcoFalke
fa50d11dd1 appveyor: Disable functional tests for now
Also add a draft for a Windows build on cirrus. The draft can be
finished later.
2020-04-14 09:15:18 -04:00
Fabian Jahr
a2324e4d3f test: Improve naming and logging of avoid_reuse tests 2020-04-14 15:03:14 +02:00
Fabian Jahr
1abbdac677 wallet: Prefer full destination groups in coin selection
When a wallet uses avoid_reuse and has a large number of outputs in
a single destination, it groups these outputs in OutputGroups that
are no larger than OUTPUT_GROUP_MAX_ENTRIES. The goal is to spend
as many outputs as possible from the destination while not breaking
consensus due to a huge number of inputs and also not surprise the
use with high fees. If there are n outputs in a destination and
n > OUTPUT_GROUP_MAX_ENTRIES then this results in one or many groups
of size OUTPUT_GROUP_MAX_ENTRIES and possibly one group of size
< OUTPUT_GROUP_MAX_ENTRIES.

Prior to this commit the coin selection in the case where
n > OUTPUT_GROUP_MAX_ENTRIES was skewed towards the one group of
size < OUTPUT_GROUP_MAX_ENTRIES if it exists and the amount to be
spent by the transaction is smaller than the aggregate of those
of the group size < OUTPUT_GROUP_MAX_ENTRIES. The reason is that
the coin selection decides between the different groups based on
fees and mostly the smaller group will cause smaller fees.

The behavior that users of the avoid_reuse flag seek is that the
full groups of size OUTPUT_GROUP_MAX_ENTRIES get used first. This
commit implements this by pretending that the small group has
a large number of ancestors (one smallet than the maximum allowed
for this wallet). This dumps the small group to the bottom of the
list of priorities in the coin selection algorithm.
2020-04-14 15:02:06 +02:00
MarcoFalke
4702cadca9 Merge #17954: wallet: Remove calls to Chain::Lock methods
48973402d8 wallet: Avoid use of Chain::Lock in CWallet::GetKeyBirthTimes (Russell Yanofsky)
e958ff9ab5 wallet: Avoid use of Chain::Lock in CWallet::CreateTransaction (Russell Yanofsky)
c0d07dc4cb wallet: Avoid use of Chain::Lock in CWallet::ScanForWalletTransactions (Russell Yanofsky)
1be8ff280c wallet: Avoid use of Chain::Lock in rescanblockchain (Russell Yanofsky)
3cb85ac594 wallet refactor: Avoid use of Chain::Lock in CWallet::RescanFromTime (Russell Yanofsky)
f7ba881bc6 wallet: Avoid use of Chain::Lock in listsinceblock (Russell Yanofsky)
bc96a9bfc6 wallet: Avoid use of Chain::Lock in importmulti (Russell Yanofsky)
25a9fcf9e5 wallet: Avoid use of Chain::Lock in importwallet and dumpwallet (Russell Yanofsky)
c1694ce6bb wallet: Avoid use of Chain::Lock in importprunedfunds (Russell Yanofsky)
ade5f87971 wallet refactor: Avoid use of Chain::Lock in qt wallettests (Russell Yanofsky)
f6da44ccce wallet: Avoid use of Chain::Lock in tryGetTxStatus and tryGetBalances (Russell Yanofsky)
bf30cd4922 refactor: Add interfaces::FoundBlock class to selectively return block data (Russell Yanofsky)

Pull request description:

  This is a set of changes updating wallet code to make fewer calls to `Chain::Lock` methods, so the `Chain::Lock` class will be easier to remove in #16426 with fewer code changes and small changes to behavior.

ACKs for top commit:
  MarcoFalke:
    re-ACK 48973402d8, only change is fixing bug 📀
  fjahr:
    re-ACK 48973402d8, reviewed rebase and changes since last review, built and ran tests locally
  ariard:
    Coce Review ACK 4897340, only changes are one suggested by last review on more accurate variable naming, human-readable output, args comments in `findCommonAncestor`

Tree-SHA512: cfd2f559f976b6faaa032794c40c9659191d5597b013abcb6c7968d36b2abb2b14d4e596f8ed8b9a077e96522365261299a241a939b3111eaf729ba0c3ef519b
2020-04-14 07:18:12 -04:00
Vasil Dimov
182dbdf0f4 util: Detect posix_fallocate() instead of assuming
Don't assume that `posix_fallocate()` is available on Linux and not
available on other operating systems. At least FreeBSD has it and we
are not using it.

Properly check whether `posix_fallocate()` is present and use it if it
is.
2020-04-14 09:49:45 +02:00
MarcoFalke
fa4c29bc1d test: Add various low-level p2p tests 2020-04-13 21:38:29 -04:00
MarcoFalke
6110ae8326 Merge #18451: test: shift coverage from getunconfirmedbalance to getbalances
0306d78cb4 Use getbalances in wallet_address_types tests (Jon Atack)
7eacdc5167 Shift coverage from getunconfirmedbalance to getbalances in wallet_abandonconflict tests (Jon Atack)
3e6f7377f6 Improve getbalances coverage in wallet_balance tests (Jon Atack)

Pull request description:

  <strike>This PR updates several tests and then removes the `getunconfirmedbalance` RPC which was deprecated in facfb4111d a year ago.

  Next steps: remove the deprecated `getwalletinfo` fields and the `getbalance` RPC in follow-ups, if there seems to be consensus on those removals.</strike>

  Update:

  `getunconfirmedbalance` RPC was deprecated in facfb4111d a year ago, but following the review comments below, this PR now only updates the test coverage to use `getbalances` while still leaving basic coverage for `getunconfirmedbalance` in wallet_balance.py.

  That said, I've seen 3 regular contributors confused in the past 10 days by "DEPRECATED" warnings in the code that are not following the deprecation policy in [JSON-RPC-interface.md#versioning](https://github.com/bitcoin/bitcoin/blob/master/doc/JSON-RPC-interface.md#versioning).

  ISTM these warnings should either be removed, or the calls deprecated (`-deprecatedrpc`), or the policy updated to describe these warnings as a pre-deprecation practice.

ACKs for top commit:
  jnewbery:
    utACK 0306d78cb

Tree-SHA512: 692e43e9bed5afa97d905740666e365f0b64e559e1c75a6a398236d9e943894e3477947fc11324f420a6feaffa0c0c1532aa983c50090ca39d06551399e6ddd1
2020-04-13 17:57:18 -04:00
Andrew Chow
0d32d66148 Remove -upgradewallet startup option 2020-04-13 13:28:04 -04:00
Andrew Chow
92263cce5b Add upgradewallet RPC 2020-04-13 13:28:01 -04:00
Andrew Chow
1e48796c99 Make UpgradeWallet a member function of CWallet 2020-04-13 13:21:41 -04:00
Andrew Chow
c988f27937 Have UpgradeWallet take the version to upgrade to and an error message out parameter 2020-04-13 13:21:18 -04:00
Andrew Chow
1833237123 Only run UpgradeWallet if the wallet needs to be upgraded 2020-04-13 13:21:01 -04:00
Andrew Chow
9c16b1735f Move wallet upgrading to its own function 2020-04-13 13:20:39 -04:00
MarcoFalke
6ef45bc968 Merge #18581: ci: Print ccache stats, add pip cache, and cleanups
73f8303629 ci: Add pip cache (Hennadii Stepanov)
6d480667b2 ci: Print ccache statistics summary (Hennadii Stepanov)
a6eea715ff ci: Drop unused WINEDEBUG variable (Hennadii Stepanov)

Pull request description:

  The Travis [pip cache](https://docs.travis-ci.com/user/caching/#pip-cache) is free and saves a dozen of seconds :)

  Here are some excerpts from the Travis logs with `ccache` statistics (I found useful):

  2) [Arm64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507749)
  ```
  ccache version 3.6
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Fri Apr 10 18:21:06 2020
  stats zeroed                        Fri Apr 10 18:03:04 2020
  cache hit (direct)                   156
  cache hit (preprocessed)             198
  cache miss                           143
  cache hit rate                     71.23 %
  called for link                        8
  cleanups performed                     9
  files in cache                      1255
  cache size                          80.6 MB
  max cache size                     100.0 MB
  ```
  3) [s390x](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507750)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:15:35 2020
  cache hit (direct)                   115
  cache hit (preprocessed)             163
  cache miss                            94
  cache hit rate                     74.73 %
  called for link                        6
  cleanups performed                     0
  files in cache                       804
  cache size                         304.8 MB
  max cache size                       5.0 GB
  ```
  4) [Win64](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507751)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:38:07 2020
  cache hit (direct)                   147
  cache hit (preprocessed)             199
  cache miss                           138
  cache hit rate                     71.49 %
  called for link                        7
  cleanups performed                     7
  files in cache                      1242
  cache size                          87.0 MB
  max cache size                     100.0 MB
  ```
  5) [CentOS 7](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507752)
  ```
  ccache version 3.7.7
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Fri Apr 10 17:45:59 2020
  stats zeroed                        Fri Apr 10 17:34:27 2020
  cache hit (direct)                   152
  cache hit (preprocessed)             200
  cache miss                           144
  cache hit rate                     70.97 %
  called for link                        8
  cleanups performed                     4
  files in cache                      1352
  cache size                          86.1 MB
  max cache size                     100.0 MB
  ```
  6) [bionic](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507753)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:21:10 2020
  cache hit (direct)                   136
  cache hit (preprocessed)              35
  cache miss                           330
  cache hit rate                     34.13 %
  called for link                        8
  cleanups performed                    18
  files in cache                      1302
  cache size                          83.8 MB
  max cache size                     100.0 MB
  ```
  7) [xenial](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507754)
  ```
  ccache version 3.2.4
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  cache hit (direct)                   161
  cache hit (preprocessed)             174
  cache miss                            95
  called for link                        7
  files in cache                      3411
  cache size                           1.1 GB
  max cache size                       5.0 GB
  ```
  10) [focal](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507757)
  ```
  ccache version 3.7.7
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats updated                       Fri Apr 10 17:35:57 2020
  stats zeroed                        Fri Apr 10 17:21:32 2020
  cache hit (direct)                    78
  cache hit (preprocessed)             125
  cache miss                           107
  cache hit rate                     65.48 %
  called for link                      120
  cleanups performed                     0
  files in cache                      6218
  cache size                           1.8 GB
  max cache size                       5.0 GB
  ```
  14) [macOS 10.12](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507761)
  ```
  ccache version 3.4.1
  cache directory                     /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /home/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /etc/ccache.conf
  stats zero time                     Fri Apr 10 17:47:19 2020
  cache hit (direct)                    28
  cache hit (preprocessed)               1
  cache miss                           469
  cache hit rate                      5.82 %
  called for link                        8
  cleanups performed                    17
  files in cache                      1946
  cache size                          82.6 MB
  max cache size                     100.0 MB
  ```
  15) [macOS 10.14 native](https://travis-ci.org/github/bitcoin/bitcoin/jobs/673507762)
  ```
  ccache version 3.7.8
  cache directory                     /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache
  primary config                      /Users/travis/build/bitcoin/bitcoin/ci/scratch/.ccache/ccache.conf
  secondary config      (readonly)    /usr/local/Cellar/ccache/3.7.8/etc/ccache.conf
  stats updated                       Fri Apr 10 17:38:44 2020
  stats zeroed                        Fri Apr 10 17:22:36 2020
  cache hit (direct)                   213
  cache hit (preprocessed)             293
  cache miss                           144
  cache hit rate                     77.85 %
  called for link                       11
  called for preprocessing              56
  compile failed                        30
  preprocessor error                    56
  bad compiler arguments                14
  autoconf compile/link                 77
  no input file                         66
  cleanups performed                     0
  files in cache                      1564
  cache size                         284.6 MB
  max cache size                       5.0 GB
  ```

Top commit has no ACKs.

Tree-SHA512: f9f2c6d72680bafc7a982efc04dc37ab9bba77ffd2a4cbbc56bb0e86c5c3501d57aa2f0ebeac0113e9d8daac1ee4eac1eddf74c1172c267b37ee2d9324366c8a
2020-04-13 11:45:11 -04:00
MarcoFalke
130b64415f Merge #18493: rpc: Remove deprecated "size" from mempool txs
0753efd9dc rpc: Remove deprecated "size" from mempool txs (Vasil Dimov)

Pull request description:

  Remove the "size" property of a mempool transaction from RPC replies.

  Deprecated in e16b6a718 in 0.19, about 1 year ago.

ACKs for top commit:
  kristapsk:
    ACK 0753efd9dc

Tree-SHA512: 392ced6764dd6a1d47c6d1dc9de78990cf3384910d801253f8f620bd1751b2676a6b52bee8a30835d28e845d84bfb37431c1d2370f48c9d4dc8e6a48a5ae8b9e
2020-04-13 11:39:44 -04:00
MarcoFalke
e28e5353c4 Merge #18545: test: refactor rpc_signrawtransaction and add logging
9cdddae3b4 test: add rpc_signrawtransaction logging (Jon Atack)
4d6cde38ce test: refactor rpc_signrawtransaction witness script tests (Jon Atack)

Pull request description:

  As a follow-up to #18484, the new tests are good but bury the one non-duplicate line in each test that sets the witness script, and there is no logging in the testfile. This PR makes it easy to see what is unique to each of the new tests and adds logging.

ACKs for top commit:
  theStack:
    ACK 9cdddae3b4 🥚 🐰

Tree-SHA512: 7b1ca303326658afb90b7635abc9fe8bb65f0be004124d4dcf38702bb6f38bc06ce33c0642be4ad5d511453d003cdefeea691e66e3b963a4feb66f6237a3c241
2020-04-13 11:19:56 -04:00
Stephan Oeste
c47adf8df4 Added my fingerprint Stephan Oeste (Emzy)
By request from added my PGP fingerprint.
See: https://github.com/bitcoin-core/gitian.sigs/pull/1220#issuecomment-612778442
2020-04-13 16:39:42 +02:00
MarcoFalke
d9fd7b5a67 Merge #18596: test: Try once more when RPC connection fails on Windows
fab9899204 test: Try once more when RPC connection fails on Windows (MarcoFalke)
faa655731e test: Document why connection is re-constructed on windows (MarcoFalke)
fa9f4f663c test: Remove python 3.4 workaround (MarcoFalke)
fae760f2b2 cirrus: Bump freebsd to 12.1 (MarcoFalke)

Pull request description:

  Fixes: #18548

ACKs for top commit:
  hebasto:
    ACK fab9899204, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: c4e9ed8d995b63a820ca66984f152ac216c83ba1f318b61b15c6d375c0e936c08f6bc3d38c255dddf3ee8952f848c7ababf684854e07a7c1b1d8504e6b7208ba
2020-04-13 09:35:48 -04:00
MarcoFalke
88884ee8d8 script: Disallow silent bool -> CScript conversion 2020-04-13 08:56:35 -04:00
Jon Atack
905e2e85ba gitian: add jonatack gpg key fingerprint 2020-04-13 14:23:06 +02:00
MarcoFalke
d5783985eb Merge #18502: doc: Update docs for getbalance (default minconf should be 0)
c0af173da2 doc: default minconf for getbalance should be 0 (U-Zyn Chua)

Pull request description:

  - Default `minconf` for `getbalance` is `0` but example in doc was showing as `1`.
  - `at least 6 blocks confirmed` now updated to be `at least 6 confirmations` to be more consistent with the terminology used elsewhere in the codebase and documentations.

ACKs for top commit:
  theStack:
    re-ACK c0af173da2

Tree-SHA512: 8f67af78a222a4bd2957658b37fae2224783274f355af84f39a5ce0da90b21f03dc798a6408d44a724c353ff5ed7dfec943fb28726ec423028b64fc579f937ad
2020-04-13 07:27:45 -04:00
MarcoFalke
4d26312c17 Merge #18597: test: Extend wallet_dump test to cover comments
555567ace9 test: Extend wallet_dump test to cover comments (MarcoFalke)

Pull request description:

ACKs for top commit:
  ryanofsky:
    Code review ACK 555567ace9. Nice new checks in this test. I confirmed this catches the missing FormatISO8601DateTime call you discovered in https://github.com/bitcoin/bitcoin/pull/17954#discussion_r406891999

Tree-SHA512: 71aa23dd039f3bcdee642b01151edd1a0d44f48cedd070f5858148c8cb8abd6f5edfd212daeba38e35c843da5ea6c799e5a952105fdecedac355a5a843c05a84
2020-04-13 06:28:35 -04:00
U-Zyn Chua
c0af173da2 doc: default minconf for getbalance should be 0 2020-04-13 16:22:30 +08:00
MarcoFalke
c49971f3c9 Merge #18584: test: Check that the version message does not leak the local address
fa404f1e47 test: Check that the version message does not leak the local address of the node (MarcoFalke)

Pull request description:

  Add test for #8740

ACKs for top commit:
  theStack:
    ACK fa404f1e47

Tree-SHA512: 4d1c10d1c02fba4b51bd8b9eb3a0d9a682b6aac8c3f6924e295fdca3faefa5ecc3eaa87d347cfec5d2b2bc49963c10fe0a37c463f36088ed0304a2e3716b963b
2020-04-12 19:49:34 -04:00
MarcoFalke
a2b282c9d0 Merge #18609: test: Remove REJECT message code
b1b0cfecb6 test: Remove REJECT message code (Hennadii Stepanov)

Pull request description:

  We no longer use REJECT p2p message:
  - #15437
  - #17004

ACKs for top commit:
  theStack:
    ACK b1b0cfecb6 (nice dead code find)

Tree-SHA512: 0a662b282e921c3991aeb15f54d077837f1ef20bc2e3b0b35117bb97a21d1bd1c3e21458e5c18ba0ca02030d559e3e8e74dbd3d3e2b46dbe7bede550948c3b55
2020-04-12 19:46:25 -04:00
MarcoFalke
59d6249290 Merge #18593: test: complete impl. of msg_merkleblock and wait_for_merkleblock
854382885f refactor: test: improve wait_for{header,merkleblock} interface (Sebastian Falbesoner)
1356a45ef0 test: complete impl. of msg_merkleblock and wait_for_merkleblock (Sebastian Falbesoner)

Pull request description:

  Implements the missing initialization/serialization methods for `msg_merkleblock`, based on the already present class `CMerkleBlock`. Also changes the method `wait_for_merkleblock()` to be more precise by waiting for a merkleblock with a specified blockhash instead of an arbitrary one.

  In the BIP37 test `p2p_filter.py`, this new method is used to make the test of receiving merkleblock and tx if a filter is set to be more precise, by checking if they also arrive in the right order.

  In the course of this PR, also the interface for the methods `wait_for_merkleblock()` and `wait_for_header()` are improved to take a hex string instead of an integer, which is more typesafe and less of a burden to the caller.

ACKs for top commit:
  MarcoFalke:
    ACK 854382885f

Tree-SHA512: adaf0ac728ef0b9929cb417a7a7b4c1346c400b2d365bf6914515c67b6cfe8f4a7ecc62fb514afdce9792f0bed833416f6bca6b9620f3d5dcdc66e4d5b0b7ea3
2020-04-12 19:38:18 -04:00
MarcoFalke
fa86a4bbfc rpc: Rename first arg of generateblock RPC to "output" 2020-04-12 17:04:03 -04:00
Hennadii Stepanov
b1b0cfecb6 test: Remove REJECT message code 2020-04-12 21:56:36 +03:00
MarcoFalke
76143bf714 Merge #18495: rpc: Remove deprecated migration code
2599d13c94 rpc: Remove deprecated migration code (Vasil Dimov)

Pull request description:

  Don't accept a second argument to `sendrawtransaction` and
  `testmempoolaccept` of type `bool`. Actually even the code before this
  change would not accept `bool`, but it would print a long explanatory
  message when rejecting it: "Second argument must be numeric (maxfeerate)
  and no longer supports a boolean. To allow a transaction with high fees,
  set maxfeerate to 0."

  This was scheduled for removal in 6c0a6f73e.

ACKs for top commit:
  MarcoFalke:
    ACK 2599d13c94 📅

Tree-SHA512: e2c74c0bde88e20149d0deab0845851bb3979143530a6bae4f46769d61b607ad2e2347f8969093c2461a80c47661732dc0b3def140f8ce84081719adda3b3811
2020-04-12 09:13:10 -04:00
MarcoFalke
fab9899204 test: Try once more when RPC connection fails on Windows 2020-04-12 09:04:15 -04:00
MarcoFalke
faa655731e test: Document why connection is re-constructed on windows 2020-04-12 08:50:41 -04:00
MarcoFalke
fa9f4f663c test: Remove python 3.4 workaround 2020-04-12 08:50:39 -04:00
MarcoFalke
fae760f2b2 cirrus: Bump freebsd to 12.1 2020-04-12 08:50:05 -04:00
fanquake
f2b5b0a3b4 build: add linker optimization flags to guix
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
2020-04-12 18:38:00 +08:00
fanquake
b8b050a8d6 build: add linker optimization flags to gitian descriptors
Any -O argument will enable optimizations in GNU ld. We can use -O2
here, as this matches our compile flags. Note that this would also
enable additional optimizations if using the lld or gold linkers,
when compared to -O0.
2020-04-12 18:36:56 +08:00
fanquake
ed3b8eada8 Merge #17595: guix: Enable building for x86_64-w64-mingw32 target
a35e323589 guix: Appease travis. (Carl Dong)
0b66d22da5 guix: Use gcc-9 for mingw-w64 instead of 8 (Carl Dong)
ba0b99bdd6 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends (Carl Dong)
93439a71ed guix: Bump to upstream commit with mingw-w64 changes (Carl Dong)
35a96792dd guix: Check mingw symbols, improve SSP fix docs (Carl Dong)
449d8fe25b guix: Expand on INT trap message (Carl Dong)
3f1f03c67a guix: Spelling fixes (Carl Dong)
ff821dd2a1 guix: Reinstate make-ssp-fixed-gcc (Carl Dong)
360a9e0ad5 guix: Bump time-machine for mingw-w64 patches (Carl Dong)
93e41b7e3b guix: Use gcc-8 for mingw-w64 instead of 7 (Carl Dong)
ef4f7e4c45 guix: Set the well-known timezone env var (Carl Dong)
acf4b3b3b5 guix: Make x86_64-w64-mingw32 builds reproducible (Carl Dong)
c4cce00eac guix: Remove dead links from README. (Carl Dong)
df953a4c9a guix: Appease shellcheck. (Carl Dong)
91897c95e1 guix: Improve guix-build.sh documentation (Carl Dong)
570d769c6c guix: Build support for Windows (Carl Dong)

Pull request description:

  ~~Based on: https://github.com/bitcoin/bitcoin/pull/16519~~
  Based on: #17933 (Time Machines are... shall we say... superior 😁)

  This PR allows us to perform Guix builds for the `x86_64-w64-mingw32` target. We do this _without_ splitting up the build script like we do in Gitian by using this newfangled alien technology called `case` statements. (This is WIP and might be changed to `if` statements soon)

ACKs for top commit:
  fanquake:
    ACK a35e323589 2/3

Tree-SHA512: c471951c23eb2cda919a71285d8b8f2580cb20f09d5db17b53e13dbd8813e01b3e7a83ea848e4913fd0f2bc12c6c133c5f76b54e65c0d89fed4dfd2e0be19875
2020-04-12 18:34:18 +08:00
Jon Atack
9cdddae3b4 test: add rpc_signrawtransaction logging 2020-04-12 12:12:03 +02:00
Jon Atack
4d6cde38ce test: refactor rpc_signrawtransaction witness script tests
to see what is distinct in each test.
2020-04-12 12:12:01 +02:00
Pieter Wuille
f63dec189c [REFACTOR] Initialize PrecomputedTransactionData in CheckInputScripts
Add a default constructor to `PrecomputedTransactionData`, which doesn't
initialize the struct's members. Instead they're initialized inside the
`CheckInputScripts()` function. This allows a later commit to add the
spent UTXOs to that structure.
2020-04-11 21:32:45 -04:00
Pieter Wuille
c31cbe7cfe Add C++17 test to Travis 2020-04-11 11:30:27 -07:00
Andrew Chow
e44aeefaae gitian: Add missing automake package to gitian-win-signer.yml
automake is needed to build osslsigncode otherwise autogen.sh fails.
2020-04-11 14:15:05 -04:00
MarcoFalke
555567ace9 test: Extend wallet_dump test to cover comments 2020-04-11 12:57:44 -04:00
Sebastian Falbesoner
854382885f refactor: test: improve wait_for{header,merkleblock} interface
The interfaces for the methods wait_for_header() and wait_for_merkleblock() are
changed to take a hex string instead of an integer, improving type safety and
removing the burden from the caller to always do the transformation via
`int(...)`. As suggested by MarcoFalke in
https://github.com/bitcoin/bitcoin/pull/18593#discussion_r407062253
2020-04-11 18:40:28 +02:00
Sebastian Falbesoner
1356a45ef0 test: complete impl. of msg_merkleblock and wait_for_merkleblock
Implements the missing initialization/serialization methods for
msg_merkleblock, based on the already present class CMerkleBlock.
Also changes the method wait_for_merkleblock() to be more precise by waiting
for a merkleblock with a specified blockhash instead of an arbitrary one.

In the BIP37 test p2p_filter.py, this new method is used to make the test of
receiving merkleblock and tx if a filter is set to be more precise, by checking
if they also arrive in the right order.
2020-04-11 18:40:16 +02:00
MarcoFalke
a5623ba89f Merge #18588: Revert "Merge #16367: Multiprocess build support"
f29bd546ec Revert "Merge #16367: Multiprocess build support" (MarcoFalke)

Pull request description:

  Reverting the changes temporarily is going to help with the following:

  * Discussion about the next steps for the multiprocess concept and the experimental libmultiprocess library without having code already commited in the master branch, potentially influencing the discussion
  * Allowing for more conceptual as well as code review ACKs to accumulate, since the pull only had one ACK (two if I count mine, which didn't make it to GitHub)

  Can be reviewed with `git diff HEAD HEAD~2 | wc` or `git diff 1b307613604883daea4913a65da30ae073c9dc4d~ | wc` (should be all zeros)

  Context here: https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-612260496

ACKs for top commit:
  ryanofsky:
    Code review ACK f29bd546ec. Confirmed revert with
  fanquake:
    ACK f29bd546ec

Tree-SHA512: 3ce06c30de23c81c2d69cfb3ada20b3458c48efda1a5ba96aee678e946c499f701bc83e9eae91580f0156c0f30a90e5d015ef8b1806ad611d433c482fa55723e
2020-04-11 10:05:35 -04:00
Pieter Wuille
7829685e27 Add configure option for c++17 2020-04-11 02:15:25 -07:00
Pieter Wuille
0fbde488b2 Support conversion between Spans of compatible types 2020-04-11 02:15:25 -07:00
Pieter Wuille
7cbfebbf3d Update ax_cxx_compile_stdcxx.m4 2020-04-11 02:15:20 -07:00
MarcoFalke
f29bd546ec Revert "Merge #16367: Multiprocess build support"
This reverts the changes made in merge commit
1b30761360:

This reverts commit b919efadff.
This reverts commit d54f64c6c7.
This reverts commit 787f40668d.
This reverts commit d630646662.
This reverts commit e6e44eedd5.
2020-04-10 19:38:21 -04:00
MarcoFalke
3eb8b1c392 Merge #17905: gui: Avoid redundant tx status updates
96cb597325 gui: Avoid redundant tx status updates (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  In `TransactionTablePriv::index`, avoid calling `interfaces::Wallet::tryGetTxStatus` if the status is up to date as of the most recent `NotifyBlockTip` notification.  Store height from the most recent notification in a new `ClientModel::cachedNumBlocks` variable in order to check this.

  This avoids floods of IPC traffic from `tryGetTxStatus` with #10102 when there are a lot of transactions. It might also make the GUI a little more efficient even when there is no IPC.

ACKs for top commit:
  promag:
    Code review ACK 96cb597325.
  hebasto:
    ACK 96cb597325

Tree-SHA512: fce597bf52a813ad4923110d0a39229ea09e1631e0d580ea18cffb09e58cdbb4b111a40a9a9270ff16d8163cd47b0bd9f1fe7e3a6c7ebb19198f049f8dd1aa46
2020-04-10 16:34:30 -04:00
Wladimir J. van der Laan
75917591c8 Merge #18586: build: Bump gitian descriptors to 0.21
dabe2bb11a build: Bump gitian descriptors to 0.21 (Wladimir J. van der Laan)

Pull request description:

  Per the release process.

ACKs for top commit:
  MarcoFalke:
    ACK dabe2bb11a, same as last bump: fa1ad8f06e

Tree-SHA512: 6548aa6f6adb31ac98e65290060c5a8a84e2c44e6634ddcc0299cea8f32be6966c7dba91696e1e7b759d251ccba5ac421c949ed0f62b7b6d07b7c2b3ef51248b
2020-04-10 21:43:44 +02:00
Vasil Dimov
2599d13c94 rpc: Remove deprecated migration code
Don't accept a second argument to `sendrawtransaction` and
`testmempoolaccept` of type `bool`. Actually even the code before this
change would not accept `bool`, but it would print a long explanatory
message when rejecting it: "Second argument must be numeric (maxfeerate)
and no longer supports a boolean. To allow a transaction with high fees,
set maxfeerate to 0."

This was scheduled for removal in 6c0a6f73e.
2020-04-10 21:27:01 +02:00
Wladimir J. van der Laan
dabe2bb11a build: Bump gitian descriptors to 0.21
Per the release process.
2020-04-10 19:46:39 +02:00
MarcoFalke
51e2ce45d6 Merge #17693: rpc: Add generateblock to mine a custom set of transactions
7524b6479c Add tests for generateblock (Andrew Toth)
dcc8332543 Add generateblock rpc (Andrew Toth)

Pull request description:

  The existing block generation rpcs for regtest, `generatetoaddress` and `generatetodescriptor`, mine everything in the mempool up to the block weight limit. This makes it difficult to test a system for several scenarios where a different set of transactions are mined. For example:
  - Testing the common scenario where a transaction is replaced in the mempool but the replaced transaction is mined instead.
  - Testing for a double-spent transaction where a transaction that conflicts with the mempool is mined.
  - Testing for non-standard transactions that are mined.
  - Testing the scenario where several blocks are mined without a specific transaction in the mempool being included in a block.

  This PR introduces a new rpc, `generateblock`, that takes an array of raw transactions and txids and mines only those and the coinbase. Any txids must be in the mempool, but the raw txs can be anything conforming to consensus rules. The coinbase can be specified as either an address or descriptor.

  This reopens #17653 since it was closed by mistake.

  Thanks to instagibbs for code suggestions that I used here.

ACKs for top commit:
  MarcoFalke:
    re-ACK 7524b6479c 📁

Tree-SHA512: 857106007465b5b9b8a84b6d07c17cbf8378a33a72d32ff79abea1d5ab4babb4d53a11ddbb14595aa1fac9dfa1391e3a11403d742f69951beea2f683e8a01cd4
2020-04-10 13:12:30 -04:00
MarcoFalke
10358a381a Merge #17737: Add ChainstateManager, remove BlockManager global
c9017ce3bc protect g_chainman with cs_main (James O'Beirne)
2b081c4568 test: add basic tests for ChainstateManager (James O'Beirne)
4ae29f5f0c use ChainstateManager to initialize chainstate (James O'Beirne)
5b690f0aae refactor: move RewindBlockIndex to CChainState (James O'Beirne)
89cdf4d569 validation: introduce unused ChainstateManager (James O'Beirne)
8e2ecfe249 validation: add CChainState.m_from_snapshot_blockhash (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This changeset introduces `ChainstateManager`, which is responsible for creating and managing access to multiple chainstates. Until we allow chainstate creation from UTXO snapshots (next assumeutxo PR?) it's basically unnecessary, but it is a prerequisite for background IBD support.

  Changes are also made to the initialization process to make use of `g_chainman` and thus clear the way for multiple chainstates being loaded on startup.

  One immediate benefit of this change is that we no longer have the `g_blockman` global, but instead have the ChainstateManager inject a reference of its shared BlockManager into any chainstate it creates.

  Another immediate benefit is that uses of `ChainActive()` and `ChainstateActive()` are now covered by lock annotations. Because use of `g_chainman` is annotated to require cs_main, these two functions subsequently follow.

  Because of whitespace changes, this diff looks bigger than it is. E.g., 4813167d98 is most easily reviewed with
  ```sh
  git show --color-moved=dimmed_zebra -w 4813167d98
  ```

ACKs for top commit:
  MarcoFalke:
    re-ACK c9017ce3bc 📙
  fjahr:
    Code Review Re-ACK c9017ce3bc
  ariard:
    Code Review ACK c9017ce
  ryanofsky:
    Code review ACK c9017ce3bc. No changes since last review other than a straight rebase

Tree-SHA512: 3f250d0dc95d4bfd70852ef1e39e081a4a9b71a4453f276e6d474c2ae06ad6ae6a32b4173084fe499e1e9af72dd9007f4a8a375c63ce9ac472ffeaada41ab508
2020-04-10 13:02:01 -04:00
MarcoFalke
a9213bbe75 Merge #18422: [consensus] MOVEONLY: Move single-sig checking EvalScript code to EvalChecksig
14e8cf974a [consensus] MOVEONLY: Move single-sig checking EvalScript code to EvalChecksig (Pieter Wuille)

Pull request description:

  This is another small refactor pulled out of the Schnorr/Taproot PR #17977.

  This is in preparation for adding different signature verification rules,
  specifically tapscript (BIP 342), which interprets opcode 0xac and 0xad
  as Schnorr signature verifications.

ACKs for top commit:
  sipa:
    ACK 14e8cf974a, verified move-only.
  MarcoFalke:
    ACK 14e8cf974a, reviewed with "git show 14e8cf974a --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space -W" 👆
  fjahr:
    Code-review ACK 14e8cf974a, verified that it's move-only.
  instagibbs:
    code review ACK 14e8cf974a, verified move-only
  theStack:
    Code-Review ACK 14e8cf974a
  jonatack:
    ACK 14e8cf974a

Tree-SHA512: af2efce9ae39d5ec01db5b9ef0ff383fe252ef5f33b3483927308ae17d91a619266cb45951f32ea1ce54807a4c0f052bcdefb47e244465d3a726393221c227b1
2020-04-10 12:59:29 -04:00
MarcoFalke
99d6a5be8b Merge #17999: refactor: Add ChainClient setMockTime, getWallets methods
3ce16ad2f9 refactor: Use psbt forward declaration (Russell Yanofsky)
1dde238f2c Add ChainClient setMockTime, getWallets methods (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  These changes are needed to set mock times, and get wallet interface pointers correctly when
  wallet code is running in a different process from node code in #10102

ACKs for top commit:
  MarcoFalke:
    re-ACK 3ce16ad2f9 🔙
  promag:
    Code review ACK 3ce16ad2f9.

Tree-SHA512: 6c093bfcd68adf5858a1aade4361cdb7fb015496673504ac7a93d0bd2595215047184551d6fd526baa27782331cd2819ce45c4cf923b205ce93ac29e485b5dd8
2020-04-10 12:57:35 -04:00
Hennadii Stepanov
73f8303629 ci: Add pip cache 2020-04-10 19:56:53 +03:00
MarcoFalke
1b30761360 Merge #16367: Multiprocess build support
b919efadff depends: Use default macos clang compiler (Russell Yanofsky)
d54f64c6c7 Add multiprocess travis configuration (Russell Yanofsky)
787f40668d Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
d630646662 libmultiprocess depends build (Russell Yanofsky)
e6e44eedd5 Multiprocess build changes (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  This splits autotools, depends build, and travis changes out of #10102, so code changes and build system changes can be reviewed separately.

ACKs for top commit:
  hebasto:
    re-ACK b919efadff, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-605514556) review.

Tree-SHA512: ebc5e403cc99a0d9629ed7fe1595e01d57e6d1255cbf03968a3196ff6f528f734c78060fdc065724ee1f923bcc5aa2b29470fcb36a7f15957eb57c76d58178a4
2020-04-10 12:55:12 -04:00
Hennadii Stepanov
6d480667b2 ci: Print ccache statistics summary 2020-04-10 19:54:32 +03:00
MarcoFalke
4eb1eeb02c Merge #18504: build: Drop bitcoin-tx and bitcoin-wallet dependencies on libevent
01a3392b1b Drop bitcoin-wallet dependency on libevent (Russell Yanofsky)
0660119ac3 Drop unintended bitcoin-tx dependency on libevent (Russell Yanofsky)

Pull request description:

  This fixes compile errors trying to build bitcoin-tx and bitcoin-wallet without libevent, which were reported by Luke Dashjr in https://github.com/bitcoin/bitcoin/issues/18465

  The fix avoiding `bitcoin-tx` dependency on libevent just adds a conditional build rule. This is implemented in the first commit (more details in commit description).

  The fix avoiding `bitcoin-wallet` dependency on libevent requires minor code changes, because `bitcoin-wallet` (unlike `bitcoin-tx`) links against code that calls `urlDecode` / `evhttp_uridecode`. This fix is implemented in the second commit (again details in the commit description).

ACKs for top commit:
  jonasschnelli:
    utACK 01a3392b1b.

Tree-SHA512: d2245e912ab494cccceeb427a1eca8e55b01a0006ff93eebcfb5461ae7cecd1083ac2de443d9db036b18bdc6f0fb615546caaa20c585046f66d234937f74870a
2020-04-10 12:52:37 -04:00
Hennadii Stepanov
a6eea715ff ci: Drop unused WINEDEBUG variable 2020-04-10 19:51:21 +03:00
MarcoFalke
6ab96ec546 Merge #18574: cli: call getbalances.ismine.trusted instead of getwalletinfo.balance
5df0877f91 test: update and harden interface_bitcoin_cli tests (Jon Atack)
75019774c9 cli -getinfo: use getbalances instead of deprecated getwalletinfo balance (Jon Atack)

Pull request description:

  Extracted from #18453 to preserve that PR as a discussion on multiwallet RPC/CLI.

  This PR updates `bitcoin-cli -getinfo` to fetch the wallet balance from `getbalances` in order to no longer depend on `getwalletinfo.balance` which was deprecated a year ago in facfb41.

  I found this when removing the getwalletinfo() `balance`, `unconfirmed_balance`, and `immature_balance` fields to see what broke from depending on them.

  I didn't see any perceivable change in `-getinfo` run time from the change.

  Test coverage for this change is provided by `test/functional/interface_bitcoin_cli.py`, which the second commit updates to (a) no longer depend on getwalletinfo.balances and (b) test the -getinfo blockcount and balance fields against non-default, non-zero values.

ACKs for top commit:
  robot-visions:
    ACK 5df0877
  MarcoFalke:
    ACK 5df0877
  vasild:
    re-ACK 5df0877f9
  promag:
    Code review ACK 5df0877f91.
  theStack:
    ACK 5df0877f91

Tree-SHA512: 0dd8c62f915b1c0112e42b132dcf74a141bdd1f51e7c17d4a698b374ec296f4f9836f7058dbe237cf24f9bfb32ea5000e14f7089e2e86472d9c6a175be26e910
2020-04-10 12:50:51 -04:00
Wladimir J. van der Laan
d84c9aa25d build: Bump version to 0.20.99
Now that 0.20 branch has been split off, master is 0.20.99 (pre-0.21).

Also clean out release notes.

Tree-SHA512: bba6133ae9708f75206c8934901b9f9909a233330f4dfefb3c24175bf8e11631cdc89a5d24a22421a73083f7eb743e977db8020b87dfbd3c1e6043929a19a285
2020-04-10 18:01:01 +02:00
MarcoFalke
a840dab2a5 Merge #18541: rpc: Make verifychain default values static, not depend on global args
fad691cafe rpc: Make verifychain default values static, not depend on global args (MarcoFalke)

Pull request description:

  This fixes several issues:

  * The documentation is not compile-time static and depends on run-time arguments, making it impossible to host it on a static resource like a website or pdf. See also a similar change in the wallet rpc code: #18499
  * The same call (relying on default values) will run different code on different machines, depending on the command line args that were used to start the server. This might lead to hard-to-debug-remote issues.

  This is a small behaviour change, and I will add release notes.

ACKs for top commit:
  theStack:
    ACK fad691cafe
  promag:
    Code review ACK fad691cafe.

Tree-SHA512: 1c7a253ff0ec13a973b10d3777b71c70954ded5805b65a3ab06317327014de4cd0601d71d30c6ce89a581722c150cb5567acc1bd3e0c789cb51bab6ef0dcfc4a
2020-04-10 11:21:57 -04:00
MarcoFalke
3347ca4881 Merge #18526: Remove PID file at the very end
7fcdec0f32 Remove PID file at the very end (Hennadii Stepanov)

Pull request description:

  While reproducing the bug from #18517, I've noticed that the `bitcoind.pid` file has already been removed when the `bitcoind` hangs.

  This PR makes `Shutdown()` keep the `bitcoind.pid` file available until the end.

ACKs for top commit:
  MarcoFalke:
    ACK 7fcdec0f32
  emilengler:
    utACK 7fcdec0f32
  promag:
    Code review ACK 7fcdec0f32.
  theStack:
    Code review ACK 7fcdec0f32

Tree-SHA512: 9732ef34e137dbee70a06d922b316b8ea7b9a1c959cf8861b6940cd789336dc19ee468a4c3a28d95d1458076a48270c676b0ff27fec30cf57eced6ddab0a2a9b
2020-04-10 11:06:27 -04:00
MarcoFalke
fa404f1e47 test: Check that the version message does not leak the local address of the node 2020-04-10 10:48:56 -04:00
MarcoFalke
29893ec875 Merge #18454: net: Make addr relay mockable, add test
fa1da3d4bf test: Add basic addr relay test (MarcoFalke)
fa1793c1c4 net: Pass connman const when relaying address (MarcoFalke)
fa47a0b003 net: Make addr relay mockable (MarcoFalke)

Pull request description:

  As usual:

  * Switch to std::chrono time to be type-safe and mockable
  * Add basic test that relies on mocktime to add code coverage

ACKs for top commit:
  naumenkogs:
    utACK  fa1da3d
  promag:
    ACK fa1da3d4bf (fabe56e44b6f683e24e37246a7a8851190947cb3 before https://github.com/bitcoin/bitcoin/pull/18454#issuecomment-607866453), fa5bf23d527a450e72c2bf13d013e5393b664ca3 was dropped since last review.

Tree-SHA512: 0552bf8fcbe375baa3cab62acd8c23b2994efa47daff818ad1116d0ffaa0b9e520dc1bca2bbc68369b25584e85e54861fe6fd0968de4f503b95439c099df9bd7
2020-04-10 10:12:46 -04:00
fanquake
2b9a4a1332 Merge #18577: doc: Correct scripted-diff example link
478c11dde3 Correct scripted-diff example link (Yahia Chiheb)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 478c11dde3

Tree-SHA512: 3bc741a79db9bd7abb17ef11f697b768565ec01303a5823ee6a7d8dfa6e888a99a15e9eda69f97a912abc3fd56a54f698f9a580596511bc9bcf62a6870b273f6
2020-04-10 08:21:50 +08:00
fanquake
d486991aa5 Merge #18295: scripts: add MACHO lazy bindings check to security-check.py
5ca90f8b59 scripts: add MACHO lazy bindings check to security-check.py (fanquake)

Pull request description:

  This is a slightly belated follow up to #17686 and some discussion with Cory. It's not entirely clear if we should make this change due to the way the macOS dynamic loader appears to work. However I'm opening this for some discussion. Also related to #17768.

  #### Issue:
  [`LD64`](https://opensource.apple.com/source/ld64/) doesn't set the [MH_BINDATLOAD](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) bit in the header of MACHO executables, when building with `-bind_at_load`. This is in contradiction to the [documentation](https://opensource.apple.com/source/ld64/ld64-450.3/doc/man/man1/ld.1.auto.html):
  ```bash
  -bind_at_load
       Sets a bit in the mach header of the resulting binary which tells dyld to
       bind all symbols when the binary is loaded, rather than lazily.
  ```

  The [`ld` in Apples cctools](https://opensource.apple.com/source/cctools/cctools-927.0.2/ld/layout.c.auto.html) does set the bit, however the [cctools-port](https://github.com/tpoechtrager/cctools-port/) that we use for release builds, bundles `LD64`.

  However; even if the linker hasn't set that bit, the dynamic loader ([`dyld`](https://opensource.apple.com/source/dyld/)) doesn't seem to ever check for it, and from what I understand, it looks at a different part of the header when determining whether to lazily load symbols.

  Note that our release binaries are currently working as expected, and no lazy loading occurs.

  #### Example:

  Using a small program, we can observe the behaviour of the dynamic loader.

  Conducted using:
  ```bash
  clang++ --version
  Apple clang version 11.0.0 (clang-1100.0.33.17)
  Target: x86_64-apple-darwin18.7.0

  ld -v
  @(#)PROGRAM:ld  PROJECT:ld64-530
  BUILD 18:57:17 Dec 13 2019
  LTO support using: LLVM version 11.0.0, (clang-1100.0.33.17) (static support for 23, runtime is 23)
  TAPI support using: Apple TAPI version 11.0.0 (tapi-1100.0.11)
  ```

  ```cpp
  #include <iostream>
  int main() {
  	std::cout << "Hello World!\n";
  	return 0;
  }
  ```

  Compile and check the MACHO header:
  ```bash
  clang++ test.cpp -o test
  otool -vh test
  ...
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE

  # Run and dump dynamic loader bindings:
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=no_bind.txt ./test
  Hello World!
  ```

  Recompile with `-bind_at_load`. Note still no `BINDATLOAD` flag:
  ```bash
  clang++ test.cpp -o test -Wl,-bind_at_load
  otool -vh test
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    16       1424   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ...
  DYLD_PRINT_BINDINGS=1 DYLD_PRINT_TO_FILE=bind.txt ./test
  Hello World!
  ```

  If we diff the outputs, you can see that `dyld` doesn't perform any lazy bindings when the binary is compiled with `-bind_at_load`, even if the `BINDATLOAD` flag is not set:
  ```diff
  @@ -1,11 +1,27 @@
  +dyld: bind: test:0x103EDF030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x103EDF030 = 0x7FFF70C9FA58
  +dyld: bind: test:0x103EDF038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x103EDF038 = 0x7FFF70CA12C2
  +dyld: bind: test:0x103EDF068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x103EDF068 = 0x7FFF70CA12B6
  +dyld: bind: test:0x103EDF070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x103EDF070 = 0x7FFF70CA1528
  +dyld: bind: test:0x103EDF080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x103EDF080 = 0x7FFF70C9FAE6
  <trim>
  -dyld: lazy bind: test:0x10D4AC0C8 = libsystem_platform.dylib:_strlen, *0x10D4AC0C8 = 0x7FFF73C5C6E0
  -dyld: lazy bind: test:0x10D4AC068 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryC1ERS3_, *0x10D4AC068 = 0x7FFF70CA12B6
  -dyld: lazy bind: test:0x10D4AC038 = libc++.1.dylib:__ZNKSt3__18ios_base6getlocEv, *0x10D4AC038 = 0x7FFF70CA12C2
  -dyld: lazy bind: test:0x10D4AC030 = libc++.1.dylib:__ZNKSt3__16locale9use_facetERNS0_2idE, *0x10D4AC030 = 0x7FFF70C9FA58
  -dyld: lazy bind: test:0x10D4AC080 = libc++.1.dylib:__ZNSt3__16localeD1Ev, *0x10D4AC080 = 0x7FFF70C9FAE6
  -dyld: lazy bind: test:0x10D4AC070 = libc++.1.dylib:__ZNSt3__113basic_ostreamIcNS_11char_traitsIcEEE6sentryD1Ev, *0x10D4AC070 = 0x7FFF70CA1528
  ```

  Note: `dyld` also has a `DYLD_BIND_AT_LAUNCH=1` environment variable, that when set, will force any lazy bindings to be non-lazy:
  ```bash
  dyld: forced lazy bind: test:0x10BEC8068 = libc++.1.dylib:__ZNSt3__113basic_ostream
  ```

  #### Thoughts:
  After looking at the dyld source, I can't find any checks for `MH_BINDATLOAD`. You can see the flags it does check for, such as MH_PIE or MH_BIND_TO_WEAK [here](https://opensource.apple.com/source/dyld/dyld-732.8/src/ImageLoaderMachO.cpp.auto.html).

  It seems that the lazy binding of any symbols depends on whether or not [lazy_bind_size](https://opensource.apple.com/source/xnu/xnu-6153.11.26/EXTERNAL_HEADERS/mach-o/loader.h.auto.html) from the `LC_DYLD_INFO_ONLY` load command is > 0. Which was mentioned in [#17686](https://github.com/bitcoin/bitcoin/pull/17686#issue-350216254).

  #### Changes:
  This PR is one of [Corys commits](7b6ba26178), that I've rebased and modified to make build. I've also included an addition to the `security-check.py` script to check for the flag.

  However, given the above, I'm not entirely sure this patch is the correct approach. If the linker no-longer inserts it, and the dynamic loader doesn't look for it, there might be little benefit to setting it. Or, maybe this is an oversight from Apple and needs some upstream discussion. Looking for some thoughts / Concept ACK/NACK.

  One alternate approach we could take is to drop the patch and modify security-check.py to look for `lazy_bind_size` == 0 in the `LC_DYLD_INFO_ONLY` load command, using `otool -l`.

ACKs for top commit:
  theuni:
    ACK 5ca90f8b59

Tree-SHA512: 444022ea9d19ed74dd06dc2ab3857a9c23fbc2f6475364e8552d761b712d684b3a7114d144f20de42328d1a99403b48667ba96885121392affb2e05b834b6e1c
2020-04-10 07:42:20 +08:00
MarcoFalke
405713d00f Merge #18529: Add fuzzer version of randomized prevector test
b1d24d1d03 Reorder the test instructions by number (Pieter Wuille)
c2ccadc26a Merge and generalize case 3 and case 6 (Pieter Wuille)
402ad5aaca Only run sanity check once at the end (Pieter Wuille)
eda8309bfc Assert immediately rather than caching failure (Pieter Wuille)
55608455cb Make a fuzzer-based copy of the prevector randomized test (Pieter Wuille)

Pull request description:

  The current prevector test effectively randomly generates a number of operations to perform on a prevector and a normal vector, and checks consistency between the two.

  By converting this into a fuzzer the operations can be targetted rather than random.

ACKs for top commit:
  MarcoFalke:
    ACK b1d24d1d03 🍬

Tree-SHA512: 2b5c62abcd5fee94f42db03400531484d98c59e7f4308e0e683c61aabcd9ce42f85c5d058d2d5e7f8221124f71d2112b6a5f3c80e5d0fdae265a70647747e92f
2020-04-09 15:00:57 -04:00
Yahia Chiheb
478c11dde3 Correct scripted-diff example link 2020-04-09 19:41:49 +01:00
Jon Atack
5df0877f91 test: update and harden interface_bitcoin_cli tests
- no longer depend on the deprecated balance field of getwalletinfo

- test blockcount and balance against non-default, non-zero expected values
2020-04-09 17:34:07 +02:00
MarcoFalke
081dcbde66 Merge #18561: test: Properly raise FailedToStartError when rpc shutdown before warmup finished
faede1b293 test: Properly raise FailedToStartError when rpc shutdown before warmup finished (MarcoFalke)

Pull request description:

  Should fix issues such as https://travis-ci.org/github/bitcoin/bitcoin/jobs/671910152#L7034

Top commit has no ACKs.

Tree-SHA512: ac659f29c5ec91985c916b734e24911cbf4e2c5c4b1f1891a7e6c2d2511ec285167550fb03848eee4a7a3cbc9f8cdb0c766f4e881d9e44368c7415d007006368
2020-04-09 22:21:17 +08:00
MarcoFalke
1ae366ecb0 Merge #18569: ci: Add big endian native s390x build
6136a96cdf ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST (Hennadii Stepanov)
97ba77aa8e ci: Add native s390x (Hennadii Stepanov)

Pull request description:

  Unlike the Docker wrapped solution (#17591) this PR suggests running on host system directly.

  This approach makes builds quick and stable (see: #18106).

  The excerpt from the Travis log:
  ```
  ...
  Running on host system without docker wrapper
  ...
  Byte Order: Big Endian
  ...
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 6136a96cdf

Tree-SHA512: 1b591de13e38d10a35217e1de11cbd648a359d18d16eed166fac18ea5788b58cc9fc6d407086ed342b99e57e479efd951a0ea693710177e500eb116316b9a788
2020-04-09 22:19:22 +08:00
Jon Atack
75019774c9 cli -getinfo: use getbalances instead of deprecated getwalletinfo balance 2020-04-09 15:42:43 +02:00
MarcoFalke
87374d80a7 Merge #18566: test: Set -use_value_profile=1 when merging fuzz inputs
fa5e973b44 test: Set -use_value_profile=1 when merging fuzz inputs (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa5e973b44

Tree-SHA512: 4fdf92c137a5f771240ceb35c5a0c0033b1c334455cff83af333eedffe6a636e88c358ec7b7b9141b230b50f03de7c894fae406494a85bd7f6a9147a6d57dd7f
2020-04-09 20:20:59 +08:00
MarcoFalke
dcef5ad6ec Merge #18562: ci: Run unit tests sequential once
fa7af33b4c ci: Run unit tests sequential once (MarcoFalke)
fa68a3e764 appveyor: Enable minimal unit test logging to aid debugging (MarcoFalke)

Pull request description:

  Fixes #16976

Top commit has no ACKs.

Tree-SHA512: 1f1ee8776a67afa8c1c5a16ef170c9975b6a486c087c8eba12e97d23382befd1c2801622ec70ca8e4cd1fbedce1dec46be67677ceaf07f35f1d3f3bead0200f0
2020-04-09 20:07:09 +08:00
Sebastian Falbesoner
5eae034996 net: limit BIP37 filter lifespan (active between 'filterload' and 'filterclear')
Previously, a default match-everything bloom filter was set for every peer,
i.e. even before receiving a 'filterload' message and after receiving a
'filterclear' message code branches checking for the existence of the filter
by testing the pointer "pfilter" were _always_ executed.
2020-04-09 11:26:24 +02:00
Hennadii Stepanov
6136a96cdf ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST 2020-04-09 02:59:37 +03:00
Hennadii Stepanov
97ba77aa8e ci: Add native s390x 2020-04-09 02:52:52 +03:00
pierrenn
2748e87932 script: prevent UB when computing abs value for num opcode serialize 2020-04-09 08:32:00 +09:00
Pieter Wuille
748977690e Add asmap_direct fuzzer that tests Interpreter directly 2020-04-08 16:26:06 -07:00
Pieter Wuille
7cf97fda15 Make asmap Interpreter errors fatal and fuzz test it 2020-04-08 16:26:06 -07:00
Pieter Wuille
c81aefc537 Add additional effiency checks to sanity checker 2020-04-08 16:26:06 -07:00
Pieter Wuille
fffd8dca2d Add asmap sanity checker 2020-04-08 16:26:06 -07:00
Pieter Wuille
5feefbe6e7 Improve asmap Interpret checks and document failures 2020-04-08 16:26:06 -07:00
Pieter Wuille
2b3dbfa5a6 Deal with decoding failures explicitly in asmap Interpret 2020-04-08 16:26:06 -07:00
Pieter Wuille
1479007a33 Introduce Instruction enum in asmap 2020-04-08 16:26:06 -07:00
Russell Yanofsky
05f9770c1f doc: Clarify developer notes about constant naming
I'm pretty sure developer notes were intended to say constants should be upper
case and variables should be lower case, but right now they are ambiguous about
whether to write:

```c++
// foo.h
extern const int SYMBOL;

// foo.cpp
const int SYMBOL = 1;
```

or:

```c++
// foo.h
extern const int g_symbol;

// foo.cpp
const int g_symbol = 1;
```

First convention above is better than the second convention because it tells
you without having to look anything up that the value of `SYMBOL` will never
change at runtime. Also I've never seen any c++ project anywhere using the
second convention
2020-04-08 15:08:35 -04:00
MarcoFalke
661bd5dea3 Merge #18363: tests: Add fuzzing harness for HTTPRequest, libevent's evhttp and related functions
cdfb8e7afa tests: Add fuzzing harness for HTTPRequest, libevent's evhttp and related functions (practicalswift)

Pull request description:

  Add fuzzing harness for `HTTPRequest`, `libevent`'s `evhttp` and related functions.

ACKs for top commit:
  laanwj:
    ACK cdfb8e7afa

Tree-SHA512: da481afed5eb3232d3f3d0583094e56050e6234223dfcb356d8567fe0616336eb1b78c5e6821325fc9767e385e5dfaf3c96f0d35ffdb67f18d74f9a9a9464e24
2020-04-09 02:45:37 +08:00
MarcoFalke
fa5e973b44 test: Set -use_value_profile=1 when merging fuzz inputs 2020-04-09 00:51:51 +08:00
MarcoFalke
3410fe6887 Merge #18521: fuzz: Add process_messages harness
fa6a008434 fuzz: Add process_messages harness (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    Tested ACK fa6a008434

Tree-SHA512: 2d8788308c7f45c97ca003378f58a9d51f51265958557a65e5e505b1666b4cb928f0d010622870175090a0ad25e2d10b41f26f4eef14b6ff334a024baa250f8c
2020-04-09 00:14:31 +08:00
MarcoFalke
4c59236376 Merge #18533: scripted-diff: Replace strCommand with msg_type
7777e3624f scripted-diff: Replace strCommand with msg_type (MarcoFalke)

Pull request description:

  Receiving a message is not a command, but simply a message of some type

ACKs for top commit:
  promag:
    ACK 7777e3624f.
  naumenkogs:
    ACK 7777e36
  practicalswift:
    ACK 7777e3624f -- I've always thought the `strCommand` name is confusing :)
  theStack:
    ACK 7777e36

Tree-SHA512: 662bac579064c621191916274314b85111cfb4df488f00893ceb16def1c47af4b2a0f34cd7349722099b5a9d23160edb8eb999841f1d64af3e0da02e4870b4bf
2020-04-09 00:12:39 +08:00
MarcoFalke
bfef72d0fb Merge #18565: tests: Add fuzzing harnesses for classes/functions in policy/fees.h, checkqueue.h and cuckoocache.h. Add fuzzing coverage.
283bd72156 tests: Add coverage of {,Incremental}DynamicUsage(const std::set<X, Y>& s) to existing fuzzer (practicalswift)
bf76000493 tests: Add fuzzing harness for classes/functions in cuckoocache.h (practicalswift)
57890b2555 tests: Add fuzzing harness for classes/functions in checkqueue.h (practicalswift)
2df5701e90 tests: Add coverage of GetVirtualTransactionSize(...) to existing fuzzer (practicalswift)
7b9a2dc864 tests: Add fuzzing harness for AdditionOverflow(...) (practicalswift)
44fb2a596b tests: Add fuzzing harness for FeeFilterRounder (practicalswift)

Pull request description:

  Includes:

  ```
  tests: Add fuzzing harness for FeeFilterRounder
  tests: Add fuzzing harness for classes/functions in checkqueue.h
  tests: Add fuzzing harness for classes/functions in cuckoocache.h
  tests: Add coverage of {,Incremental}DynamicUsage(const std::set<X, Y>& s) to existing fuzzer
  tests: Add coverage of GetVirtualTransactionSize(...) to existing fuzzer
  tests: Add fuzzing harness for AdditionOverflow(...)
  ```

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core.

ACKs for top commit:
  MarcoFalke:
    ACK 283bd72156

Tree-SHA512: 2361edfb5c47741b22d9fb996836c5250c5a26bc5e956039ea6a0c55ba2d36c78f241d66f85bc02f5b85b9b83d5fde56a5c4702b9d1b7ac4a9a3ae391ca79eaa
2020-04-08 23:34:19 +08:00
MarcoFalke
2392566284 Merge #18563: test: Fix unregister_all_during_call cleanup
13d2a33537 Fix unregister_all_during_call cleanup (Russell Yanofsky)

Pull request description:

  Use `TestingSetup` fixture to fix `unregister_all_during_call` test not calling `UnregisterBackgroundSignalScheduler`, which could trigger an assert in `RegisterBackgroundSignalScheduler` when called in later tests

  Failure reported by fanquake https://github.com/bitcoin/bitcoin/pull/18551#issuecomment-610974251

ACKs for top commit:
  MarcoFalke:
    ACK 13d2a33537 if appveyor unit tests pass

Tree-SHA512: d2ec8ff14c54d97903af50031abfac1f38ec1c3aabc90371cfd5b79481fa69d3d77f339bfdf7d2178fd85e83402f72eda7cf4d339e5bbfa7e6e1a68836643b93
2020-04-08 23:24:54 +08:00
practicalswift
283bd72156 tests: Add coverage of {,Incremental}DynamicUsage(const std::set<X, Y>& s) to existing fuzzer 2020-04-08 14:45:27 +00:00
practicalswift
bf76000493 tests: Add fuzzing harness for classes/functions in cuckoocache.h 2020-04-08 14:45:27 +00:00
practicalswift
57890b2555 tests: Add fuzzing harness for classes/functions in checkqueue.h 2020-04-08 14:45:27 +00:00
practicalswift
2df5701e90 tests: Add coverage of GetVirtualTransactionSize(...) to existing fuzzer 2020-04-08 14:45:27 +00:00
practicalswift
7b9a2dc864 tests: Add fuzzing harness for AdditionOverflow(...) 2020-04-08 14:45:27 +00:00
practicalswift
44fb2a596b tests: Add fuzzing harness for FeeFilterRounder 2020-04-08 14:45:27 +00:00
Wladimir J. van der Laan
b3c3d9a518 Merge #18558: build: Fix boost detection for arch armv7l
da0842dcd4 build: Update ax_boost_mase.m4 to the latest serial (Hennadii Stepanov)

Pull request description:

  Picked from the upstream 90814f1895

  Fix #17010.

  This PR is [alternative](https://github.com/bitcoin/bitcoin/issues/17010#issuecomment-610651736) to #18501.

ACKs for top commit:
  laanwj:
    ACK da0842dcd4

Tree-SHA512: 5e43e12c524e4ea6b967c9be02c81a75948eac6cf55b819e3339222a2e3414731581d40af3524ad865abae7c5247c190448ebf2aa5e0d9a338edb501cc23ba38
2020-04-08 16:28:50 +02:00
MarcoFalke
fa7af33b4c ci: Run unit tests sequential once 2020-04-08 22:16:55 +08:00
MarcoFalke
fa68a3e764 appveyor: Enable minimal unit test logging to aid debugging 2020-04-08 22:14:44 +08:00
Russell Yanofsky
13d2a33537 Fix unregister_all_during_call cleanup
Use TestingSetup fixture to fix unregister_all_during_call test not calling
UnregisterBackgroundSignalScheduler, which could trigger an assert in
RegisterBackgroundSignalScheduler when called in later tests

Failure reported by fanquake <fanquake@gmail.com>
https://github.com/bitcoin/bitcoin/pull/18551#issuecomment-610974251
2020-04-08 10:11:46 -04:00
MarcoFalke
faede1b293 test: Properly raise FailedToStartError when rpc shutdown before warmup finished 2020-04-08 21:44:39 +08:00
Hennadii Stepanov
da0842dcd4 build: Update ax_boost_mase.m4 to the latest serial
This change fixes boost detection for arch armv7l.
2020-04-08 16:14:31 +03:00
MarcoFalke
1f70185a80 Merge #18551: Do not clear validationinterface entries being executed
2276339a17 Add test for UnregisterAllValidationInterfaces bug (Russell Yanofsky)
3c61abbbc8 Do not clear validationinterface entries being executed (Pieter Wuille)

Pull request description:

  The previous code for MainSignalsInstance::Clear would decrement the reference
  count of every interface, including ones that were already Unregister()ed but
  still being executed.

  This fixes the issue pointed out here: https://github.com/bitcoin/bitcoin/pull/18524/files#r404395685 . It's not currently observable.

ACKs for top commit:
  jonasschnelli:
    utACK 2276339a17 - reviewed code and test (thanks @ryanofsky for adding the test).
  MarcoFalke:
    ACK 2276339a17 🎎
  ryanofsky:
    Code review ACK 2276339a17. No change to bugfix, just rebased and new test commit added since last review

Tree-SHA512: c1d68e7c681a45c6cadc84e407c2266bcb4b12d34264e1232a61c4eadb74b551231c5a3b1d041de39f507aef4dfa7d4589b8bfe1833f069c739c6270d2a05dbe
2020-04-08 20:43:37 +08:00
Carl Dong
a35e323589 guix: Appease travis. 2020-04-07 19:27:31 -04:00
Carl Dong
0b66d22da5 guix: Use gcc-9 for mingw-w64 instead of 8
The libtool unsorted 'find' determinism issue seemed to have been solved
in gcc-9's git: d41cd173e23ebea7c758644d6ad6e0fde1c2e3a6 or SVN: r262451

Furthermore, it seems that Ubuntu Focal 20.04 LTS is going to ship with
gcc 9 and mingw-w64 7, which will match what we have now.

-----

A note on this:

Careful observers will see that previously I stated that all released
versions of gcc were bootstrapped with a libtool 2.2.7a, meaning that
they all had the unsorted 'find' determinism issue first resolved in
libtool 2.2.7b.

However, I was mistaken, gcc's ltmain.sh CLAIMS it was generated by
libtool 2.2.7a, but it was in fact edited manually. It seems that gcc
maintains their own versions of ltmain.sh and libtool.m4, and only
sometimes backports patches from upstream.

Quite confusing.
2020-04-07 19:01:26 -04:00
Carl Dong
ba0b99bdd6 guix: Don't set MINGW_HAS_SECURE_API CFLAG in depends
This is no longer needed after 3bef7c22 in the mingw-w64 git repository,
which is first included in mingw-w64 v7.0.0.

As of the previous bump to our Guix time machine, we now use mingw-w64
v7.0.0.
2020-04-07 19:00:50 -04:00
Carl Dong
93439a71ed guix: Bump to upstream commit with mingw-w64 changes
Most of the mingw-w64 toolchain changes have now been upstreamed, we can
point to a commit that exists upstream.

NOTE: I'm not changing the URL yet until we see that Guix upstream will
accept all my patches for macOS.

-----

The Guix tree that's referred to by this commit contains the following
changes relevant to our mingw-w64 build:

b066c25026

  Adds a PACKAGES-WITH-*PATCHES procedure which we can use in the future
  to apply patches to packages if those patches are not considered
  appropriate to upstream Guix

4719b71572

  Adds mingw-w64 (the libc itself) reproducibility patches, taken from
  debian.

79825bee07 + 401d28e433 + c1c50cb5b0

  Add mingw-w64 specific binutils patches, taken from debian.
  Specifically, the "Make DLL import libraries reproducible" patch made
  libbitcoinconsensus.dll.a build reproducibly. The followup commits
  were hotfixes for my mistakes.

0f864175dc

  Bumps mingw-w64 to v7.0.0. This is the first release that enables
  secure APIs by default (which we need), and gains _FORTIFY_SOURCE
  support. This will also be what Ubuntu Focal 20.04 LTS releases with.

cdf00cf75d

  Bumps NSIS to v3.05. This is the first release that includes a fix for
  a reproducibility bug found by some of the electrum developers. See
  details here: https://sourceforge.net/p/nsis/bugs/1230/
2020-04-07 19:00:49 -04:00
Pieter Wuille
8508473094 Avoid non-trivial global constants in SHA-NI code 2020-04-07 13:57:34 -07:00
Russell Yanofsky
2276339a17 Add test for UnregisterAllValidationInterfaces bug
Bug in MainSignalsInstance::Clear could cause validation interface callbacks to
be deleted during execution if UnregisterAllValidationInterfaces was called
more than once.

Bug was introduced in https://github.com/bitcoin/bitcoin/pull/18524 and is
fixed by https://github.com/bitcoin/bitcoin/pull/18551
2020-04-07 12:54:41 -07:00
Pieter Wuille
3c61abbbc8 Do not clear validationinterface entries being executed
The previous code for MainSignalsInstance::Clear would decrement the reference
count of every interface, including ones that were already Unregister()ed but
still being executed.
2020-04-07 12:53:40 -07:00
MarcoFalke
1b151e3ffc Merge #18532: rpc: Avoid initialization-order-fiasco on static CRPCCommand tables
fa1a92224d rpc: Avoid initialization-order-fiasco on static CRPCCommand tables (MarcoFalke)

Pull request description:

  Currently the fiasco is only theoretical because all content of the table are compile-time constants. However, the fiasco materializes should they ever become run-time constants (e.g. #18531).

ACKs for top commit:
  promag:
    ACK fa1a92224d.
  practicalswift:
    ACK fa1a92224d -- fiasco bad :)

Tree-SHA512: cccadb0ad56194599b74f04264d74c34fa865958580a850efc6474bbdc56f30cadce6b2e9a6ad5472ff46c3f4c793366acd8090fad409a45b25d961f2d89da19
2020-04-07 23:46:17 +08:00
MarcoFalke
63dad67348 Merge #18546: Bugfix: Wallet: Safely deal with change in the address book [part 2]
7a2ecf16df Wallet: Change IsMine check in CWallet::DelAddressBook from assert to failure (Luke Dashjr)
2952c46b92 Wallet: Replace CAddressBookData.name with GetLabel() method (Luke Dashjr)
d7092c392e QA: Test that change doesn't turn into non-change when spent in an avoid-reuse wallet (Luke Dashjr)

Pull request description:

  Follow-up to #18192, not strictly necessary for 0.20

ACKs for top commit:
  MarcoFalke:
    re-ACK 7a2ecf16df, only change is adding an assert_equal in the test 🔰
  jnewbery:
    utACK 7a2ecf16df

Tree-SHA512: e0933ee40f705b751697dc27249e1868ed4874254b174ebdd0a7150125d8c818402e66df2371718c7eeb90e67ee2317215fb260aa9b9d7b9b45ee436de2988ff
2020-04-07 18:47:38 +08:00
MarcoFalke
d12568e2f8 Merge #18549: qt: Fix Window -> Minimize menu item
56fe839e4e qt: Fix Window -> Minimize menu item (Hennadii Stepanov)

Pull request description:

  Now Window -> Minimize menu item is broken on Linux.

  Steps to reproduce:

  1. start `bitcoin-qt`
  2. activate Window -> Minimize menu item with a keyboard (not by a shortcut) or a mouse

  **Expected behavior**

  The main window gets minimized.

  **Actual behavior**

  The main window still unchanged. Even worse: the menu widget becomes a separate window:

  ![Screenshot from 2020-04-07 00-32-02](https://user-images.githubusercontent.com/32963518/78608129-ffb1dd80-7868-11ea-8e73-62ecc140ac1f.png)

  This PR does not touch the macOS specific code as `qApp->focusWindow()` seems work on macOS flawlessly.

ACKs for top commit:
  promag:
    Tested ACK 56fe839e4e on bionic with qt 5.9.5.

Tree-SHA512: 3582e44ba181d859f5994b9cddc6ce1b60aa1db520a31dd3a0684336c79d558d7410ce7a1ab5b0860c6431b54d8acc3aa16e399717b4c70839861e6b6c4290c0
2020-04-07 18:43:07 +08:00
Pieter Wuille
b1d24d1d03 Reorder the test instructions by number 2020-04-06 14:51:38 -07:00
Pieter Wuille
c2ccadc26a Merge and generalize case 3 and case 6 2020-04-06 14:39:42 -07:00
Pieter Wuille
402ad5aaca Only run sanity check once at the end 2020-04-06 14:39:42 -07:00
Pieter Wuille
eda8309bfc Assert immediately rather than caching failure 2020-04-06 14:39:38 -07:00
Pieter Wuille
55608455cb Make a fuzzer-based copy of the prevector randomized test 2020-04-06 14:25:25 -07:00
Hennadii Stepanov
56fe839e4e qt: Fix Window -> Minimize menu item 2020-04-07 00:19:19 +03:00
Luke Dashjr
7a2ecf16df Wallet: Change IsMine check in CWallet::DelAddressBook from assert to failure 2020-04-06 20:52:04 +00:00
Luke Dashjr
2952c46b92 Wallet: Replace CAddressBookData.name with GetLabel() method 2020-04-06 20:52:04 +00:00
Luke Dashjr
d7092c392e QA: Test that change doesn't turn into non-change when spent in an avoid-reuse wallet 2020-04-06 20:52:04 +00:00
MarcoFalke
abc145c9a4 Merge #18543: test: Use one node to avoid a race due to missing sync in rpc_signrawtransaction
fa2251df5e test: Use one node to avoid a race due to missing sync in rpc_signrawtransaction (MarcoFalke)

Pull request description:

  Node 0 creates a transaction in a block, and node 1 sends a spending transaction without properly syncing the utxo set.

  Fixes intermittent test failure in rpc_signrawtransaction

  ```
  test  2020-04-01T00:14:03.400000Z TestFramework (ERROR): JSONRPC error
                                     Traceback (most recent call last):
                                       File "C:\projects\bitcoin\test\functional\test_framework\test_framework.py", line 112, in main
                                         self.run_test()
                                       File "C:\projects\bitcoin/test/functional/rpc_signrawtransaction.py", line 213, in run_test
                                         self.witness_script_test()
                                       File "C:\projects\bitcoin/test/functional/rpc_signrawtransaction.py", line 208, in witness_script_test
                                         self.nodes[1].sendrawtransaction(spending_tx_signed['hex'])
                                       File "C:\projects\bitcoin\test\functional\test_framework\coverage.py", line 47, in __call__
                                         return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                                       File "C:\projects\bitcoin\test\functional\test_framework\authproxy.py", line 141, in __call__
                                         raise JSONRPCException(response['error'], status)
                                     test_framework.authproxy.JSONRPCException: bad-txns-inputs-missingorspent (-25)
  ```

  Full log: https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/31864368

ACKs for top commit:
  achow101:
    ACK fa2251df5e

Tree-SHA512: 9450d216d9989d6c44028ae4b9818790cfb00796e0de22331422f775f74d697bb14ebae0e88dca20c6b641363780da384fe94c708e20fce9cfde929fb343b12f
2020-04-07 04:21:47 +08:00
MarcoFalke
c5966a87d1 Merge #18192: Bugfix: Wallet: Safely deal with change in the address book
b5795a7886 Wallet: Add warning comments and assert to CWallet::DelAddressBook (Luke Dashjr)
6d2905f57a Wallet: Avoid unnecessary/redundant m_address_book lookups (Luke Dashjr)
c751d886f4 Wallet: Avoid treating change-in-the-addressbook as non-change everywhere (Luke Dashjr)
8e64b8c84b Wallet: New FindAddressBookEntry method to filter out change entries (and skip ->second everywhere) (Luke Dashjr)
65b6bdc2b1 Wallet: Add CAddressBookData::IsChange which returns true iff label has never been set (Luke Dashjr)
144b2f85da Wallet: Require usage of new CAddressBookData::setLabel to change label (Luke Dashjr)
b86cd155f6 scripted-diff: Wallet: Rename mapAddressBook to m_address_book (Luke Dashjr)

Pull request description:

  In many places, our code assumes that presence in the address book indicates a non-change key, and absence of an entry in mapAddressBook indicates change.

  This no longer holds true after #13756 (first released in 0.19) since it added a "used" DestData populated even for change addresses. Only avoid-reuse wallets should be affected by this issue.

  Thankfully, populating DestData does not write a label to the database, so we can retroactively fix this (so long as the user didn't see the change address and manually assign it a real label).

  Fixing it is accomplished by:

  * Adding a new bool to CAddressBookData to track if the label has ever been assigned, either by loading one from the database, or by assigning one at runtime.
  * `CAddressBookData::IsChange` and `CWallet::FindAddressBookEntry` are new methods to assist in excluding change from code that doesn't expect to see them.
  * For safety in merging, `CAddressBookData::name` has been made read-only (the actual data is stored in `m_label`, a new private member, and can be changed only with `setLabel` which updates the `m_change` flag), and `mapAddressBook` has been renamed to `m_address_book` (to force old code to be rebased to compile).

  A final commit also does some minor optimisation, avoiding redundant lookups in `m_address_book` when we already have a pointer to the `CAddressBookData`.

ACKs for top commit:
  ryanofsky:
    Code review ACK b5795a7886. Pretty clever and nicely implemented fix!
  jonatack:
    ACK b5795a7886 nice improvements -- code review, built/ran tests rebased on current master ff53433fe4 and tested manually with rpc/cli
  jnewbery:
    Good fix. utACK b5795a788.

Tree-SHA512: 40525185a0bcc1723f602243c269499ec86ecb298fecb5ef24d626bbdd5e3efece86cdb1084ad7eebf7eeaf251db4a6e056bcd25bc8457b417fcbb53d032ebf0
2020-04-07 03:51:18 +08:00
Wladimir J. van der Laan
c31bcaf203 Merge #18458: net: Add missing cs_vNodes lock
fa369651c5 net: Add missing cs_vNodes lock (MarcoFalke)

Pull request description:

  Fixes #18457

ACKs for top commit:
  promag:
    Code review ACK fa369651c5.
  laanwj:
    ACK fa369651c5

Tree-SHA512: 60d7000f2f3d480bb0953ce27a0020763e7102da16a0006b619e0a236cfc33cbd4f83d870e9f0546639711cd877c1f9808d419184bbc153bb328885417e0066c
2020-04-06 21:06:09 +02:00
Wladimir J. van der Laan
75021e80ee Merge #18487: rpc: Fix rpcRunLater race in walletpassphrase
7b8e15728d rpc: Fix rpcRunLater race in walletpassphrase (João Barbosa)

Pull request description:

  Release locks before calling `rpcRunLater`.

  Quick explanation: `rpcRunLater` leads to `event_free` which calls `event_del` which can wait for the event callback to finish if it's already running and that callback will try to lock wallet mutex - which is already locked in http thread.

  Fixes #14995 , fixes #18482. Best reviewed with whitespace changes hidden.

ACKs for top commit:
  MarcoFalke:
    ACK 7b8e15728d, only tested that this avoids the node freezing. Did not look at how libevent works or how the deadlock happens or if this breaks other stuff. 📞
  ryanofsky:
    Code review ACK 7b8e15728d. Just updated comment since last review

Tree-SHA512: 17874a2fa7b0e164fb0d7ee4cb7d59650275b8c03476fb291d60af8b758495457660d3912623fb26259fefe84aeba21c0a9e0c6467982ba511f19344ed5413ab
2020-04-06 20:29:35 +02:00
MarcoFalke
54d5ba3d9c Merge #18540: test: wallet_bumpfee assertion fixup
b224b4e7bd test: wallet_bumpfee assertion fixup (Jon Atack)

Pull request description:

  Follow-up to #18516 to fix up an assertion as per suggested change in https://github.com/bitcoin/bitcoin/pull/18516#discussion_r404191587.

ACKs for top commit:
  jnewbery:
    ACK b224b4e7bd

Tree-SHA512: 4973bba73a67c1ffaf460921b3d454e9d66a40a67f73b7df742e24a0e389adba3946a3958a729391ee6bfa4ef844be759ebf71d14d788434c248e48a2bbe5bde
2020-04-07 02:21:53 +08:00
MarcoFalke
fa2251df5e test: Use one node to avoid a race due to missing sync in rpc_signrawtransaction 2020-04-07 02:19:08 +08:00
MarcoFalke
c0b389b335 Merge #18484: rpc: Correctly compute redeemScript from witnessScript for signrawtransaction
cd3b1569d9 Correctly compute redeemScript from witnessScript for signrawtransaction (Andrew Chow)

Pull request description:

  `ParsePrevouts` uses `GetScriptForWitness` on the given witnessScript to find the corresponding redeemScript. This is incorrect when the witnessScript is either a P2PK or P2PKH script as it returns the corresponding P2WPK script instead of turning the witnessScript into a P2WSH script. Instead this should make the script a `WitnessV0ScriptHash` destination and get the script for that.

  Test cases are also added. These will fail on master with a `redeemScript does not correspond to witnessScript`

  Reported on [Bitcointalk](https://bitcointalk.org/index.php?topic=5236818.0)

ACKs for top commit:
  MarcoFalke:
    weak ACK cd3b1569d9, only checked that the test fails without the code change 🚰
  instagibbs:
    utACK cd3b1569d9

Tree-SHA512: afac671dbb52ce88bfb4a9ca3dd6065427ad52c9778d0549ad40e9286778f308adad24fb3b3c3089545d7f88c57c53d41224fd7a4bb207550eff2fe06600118f
2020-04-07 00:59:48 +08:00
MarcoFalke
fad691cafe rpc: Make verifychain default values static, not depend on global args 2020-04-07 00:53:49 +08:00
Jon Atack
b224b4e7bd test: wallet_bumpfee assertion fixup 2020-04-06 18:37:50 +02:00
MarcoFalke
425a7f900e Merge #18516: test: relax bumpfee dust_to_fee txsize an extra vbyte
25e03ba1ff test: relax bumpfee dust_to_fee txsize an extra vbyte (Jon Atack)

Pull request description:

  Hopefully closes #18511 by allowing the transaction size to be 140-141 vbytes rather than strictly 141, and bumps with a slightly larger fee to ensure dust in the 140 vbyte case.

ACKs for top commit:
  jnewbery:
    utACK 25e03ba1ff

Tree-SHA512: 76a04e1ce090e48befe048ed6d412222d7f8bc951ff822850833061a0606b1bebc5289f7249737d3fb9aa26eb857f99543981037cea6babe3e578e2cfe8afcdb
2020-04-06 23:38:09 +08:00
Jon Atack
25e03ba1ff test: relax bumpfee dust_to_fee txsize an extra vbyte
and add explanatory documentation for the reasoning.
2020-04-06 17:07:33 +02:00
Wladimir J. van der Laan
fdeb445a34 Merge #18524: refactor: drop boost::signals2 in validationinterface
d6815a2313 refactor: drop boost::signals2 in validationinterface (Russell Yanofsky)

Pull request description:

  Stop using boost::signals2 internally in validationinterface. Replace with std::list and Add/Remove/Clear/Iterate helper functions.

  Motivation for change is to reduce dependencies and avoid issues happening with boost versions before 1.59: https://github.com/bitcoin/bitcoin/issues/18517, https://github.com/bitcoin/bitcoin/pull/18471

ACKs for top commit:
  MarcoFalke:
    ACK d6815a2313
  laanwj:
    ACK d6815a2313
  hebasto:
    re-ACK d6815a2313
  promag:
    ACK d6815a2313.

Tree-SHA512: 4fc0f14a8446e8616cc142af6c3d36815f3254525d30348ba8e4d4bc74c249a5a8c9bc119bdd1be7ebd7abe0b784bc0c5551a3e156a766890cb2fdd891a95919
2020-04-06 16:46:07 +02:00
Hennadii Stepanov
7fcdec0f32 Remove PID file at the very end 2020-04-06 17:12:32 +03:00
practicalswift
cdfb8e7afa tests: Add fuzzing harness for HTTPRequest, libevent's evhttp and related functions 2020-04-06 13:58:51 +00:00
MarcoFalke
299544f9c5 Merge #18534: test: skip backwards compat tests if not compiled with wallet
c0c43ae147 test: skip backwards compat tests if not compiled with wallet (fanquake)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: d9975a1490e69134408b6b724cea26a6c1397d43f59850283b9e338ae38e00fefbcd868fb141e0a4bb55f02076690a99331f29cfa2d0fa66c165032b24a94081
2020-04-06 21:27:56 +08:00
Wladimir J. van der Laan
adac12ae73 Merge #18506: net: Hardcoded seeds update for 0.20
0eeb0468e7 net: Hardcoded seeds update for 0.20 (Wladimir J. van der Laan)

Pull request description:

  Update hardcoded seeds from http://bitcoin.sipa.be/seeds.txt.gz,
  according to release process.

  Output from makeseeds.py:
  ```
    IPv4   IPv6  Onion Pass
  1364173 244127   2454 Initial
  1364173 244127   2454 Skip entries with invalid address
  1129552 213117   2345 After removing duplicates
  1129548 213117   2345 Skip entries from suspicious hosts
  338216 191944   2249 Enforce minimal number of blocks
  336851 188993   2189 Require service bit 1
    6998   1520    150 Require minimum uptime
    5682   1290     89 Require a known and recent user agent
    5622   1279     89 Filter out hosts with multiple bitcoin ports
     512    146     89 Look up ASNs and limit results per ASN and per net
  ```

Top commit has no ACKs.

Tree-SHA512: ce1c2cda18dd5bd22586a5283a0877f3bd890437cc29dc1d85452ba4a4d28032f591c8b37f3329e8e649556cf83750b6949a068fad76d1773853d93014609da0
2020-04-06 13:38:32 +02:00
fanquake
a029805f57 build: remove -Qunused-arguments workaround for clang + ccache
This was added in 386efb7695 to
address spammy Clang warnings when building with ccache.

The issue was addressed in ccache 3.2
(https://bugzilla.samba.org/show_bug.cgi?id=8118, Nov 2014),
and from a look at all major distros, it's only Debian Jessie
that has a version of ccache older than that (3.1).

Therefore I think it's acceptable to drop this workaround, and
re-enable warnings for unused driver arguments (when compiling
using Clang).
2020-04-06 14:08:56 +08:00
Russell Yanofsky
b919efadff depends: Use default macos clang compiler
Suggested by Cory Fields <cory-nospam-@coryfields.com>
https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-595393546
as alternate workaround for problem described
https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-594600985
2020-04-05 20:48:21 -05:00
fanquake
516ebe8a62 Merge #18514: test: remove rapidcheck integration and tests
9e071b0089 test: remove rapidcheck integration and tests (fanquake)

Pull request description:

  Whilst the property tests are interesting, ultimately [rapidcheck](https://github.com/emil-e/rapidcheck) integration in this repository has not gained much traction. We have a limited number of tests, and they are rarely (if ever) run. Have discussed this with Chris Stewart.

ACKs for top commit:
  practicalswift:
    ACK 9e071b0089

Tree-SHA512: d0c12af3163382eee8413da420c63e39265a7b700709a05d518445832d45e049aed9508e32524db5228fe3ac114609a00b7bb890be047c07032e44a5ef4611e9
2020-04-06 09:48:21 +08:00
Russell Yanofsky
d54f64c6c7 Add multiprocess travis configuration 2020-04-05 21:48:21 -04:00
Russell Yanofsky
787f40668d Set LD_LIBRARY_PATH consistently in travis tests
Remove inconsistency between functional and unit test environments and make it
possible to substitute bitcoin-qt and bitcoin-node in place of bitcoind in
python tests, or to link bitcoind against shared libraries.
2020-04-05 21:48:21 -04:00
Russell Yanofsky
d630646662 libmultiprocess depends build 2020-04-05 21:48:21 -04:00
Russell Yanofsky
e6e44eedd5 Multiprocess build changes
autotools and automake changes to support multiprocess execution.

This adds a new --enable-multiprocess flag, and build configuration code to
detect libraries needed for multiprocess support. The --enable-multiprocess
flag builds new bitcoin-node and bitcoin-gui executables, which are updated in
https://github.com/bitcoin/bitcoin/pull/10102 to communicate across processes.
But for now they are functionally equivalent to existing bitcoind and
bitcoin-qt executables.
2020-04-05 21:48:21 -04:00
fanquake
c0c43ae147 test: skip backwards compat tests if not compiled with wallet 2020-04-06 09:03:17 +08:00
MarcoFalke
7777e3624f scripted-diff: Replace strCommand with msg_type
-BEGIN VERIFY SCRIPT-
sed -i 's/\<strCommand\>/msg_type/g' ./src/net_processing.cpp ./src/test/fuzz/process_message.cpp
-END VERIFY SCRIPT-
2020-04-06 08:00:34 +08:00
Jon Atack
0306d78cb4 Use getbalances in wallet_address_types tests 2020-04-05 19:40:28 +02:00
Jon Atack
7eacdc5167 Shift coverage from getunconfirmedbalance to getbalances in wallet_abandonconflict tests 2020-04-05 19:40:25 +02:00
Jon Atack
3e6f7377f6 Improve getbalances coverage in wallet_balance tests
and shift some getunconfirmedbalance calls to getbalances, as the former is
deprecated, while leaving essential coverage for it in test_balances().
2020-04-05 19:39:38 +02:00
MarcoFalke
fa1a92224d rpc: Avoid initialization-order-fiasco on static CRPCCommand tables 2020-04-06 00:20:00 +08:00
MarcoFalke
cf21293ef7 Merge #18515: test: add BIP37 remote crash bug [CVE-2013-5700] test to p2p_filter.py
0ed2d8e07d test: add BIP37 remote crash bug [CVE-2013-5700] test to p2p_filter.py (Sebastian Falbesoner)

Pull request description:

  Integrates the missing message type `filteradd` to the test framework and checks that the BIP37 implementation is not vulnerable to the "remote crash bug" [CVE-2013-5700](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-5700) anymore. Prior to v.0.8.4, it was possible to trigger a division-by-zero error on the following line in the function `CBloomFilter::Hash()`:
  f0d6487e29/src/bloom.cpp (L45)
  By setting a zero-length filter via `filterload`, `vData.size()` is 0, so the modulo operation above, called on any .insert() or .contains() operation then crashed the node. The test uses the approach of just sending an arbitrary `filteradd` message after, which calls `CBloomFilter::insert()` (and in turn `CBloomFilter::Hash()`) on the node. The vulnerability was fixed by commit 37c6389c5a (an intentional covert fix, [according to gmaxwell](https://github.com/bitcoin/bitcoin/issues/18483#issuecomment-608224095)), which introduced flags `isEmpty`/`isFull` that wouldn't call the `Hash()` member function if `isFull` is true (set to true by default constructor).

  To validate that the test fails if the implementation is vulnerable, one can simply set the flags to false in the member function `UpdateEmptyFull()` (that is called after a filter received via `filterload` is constructed), which activates the vulnerable code path calling `Hash` in any case on adding or testing for data in the filter:
  ```diff
  diff --git a/src/bloom.cpp b/src/bloom.cpp
  index bd6069b..ef294a3 100644
  --- a/src/bloom.cpp
  +++ b/src/bloom.cpp
  @@ -199,8 +199,8 @@ void CBloomFilter::UpdateEmptyFull()
           full &= vData[i] == 0xff;
           empty &= vData[i] == 0;
       }
  -    isFull = full;
  -    isEmpty = empty;
  +    isFull = false;
  +    isEmpty = false;
   }
  ```
  Resulting in:
  ```
  $ ./p2p_filter.py
  [...]
  2020-04-03T14:38:59.593000Z TestFramework (INFO): Check that division-by-zero remote crash bug [CVE-2013-5700] is fixed
  2020-04-03T14:38:59.695000Z TestFramework (ERROR): Assertion failed
  [...]
  [... some exceptions following ...]
  ```

ACKs for top commit:
  naumenkogs:
    utACK 0ed2d8e07d

Tree-SHA512: 02d0253d13eab70c4bd007b0750c56a5a92d05d419d53033523eeb3ed80318bc95196ab90f7745ea3ac9ebae7caee3adbf2a055a40a4124e0915226e49018fe8
2020-04-05 21:18:11 +08:00
MarcoFalke
fa6a008434 fuzz: Add process_messages harness 2020-04-05 10:46:24 +08:00
MarcoFalke
96a30b98c9 Merge #18528: build: create test_fuzz library from src/test/fuzz/fuzz.cpp
691e2a7af7 build: create test_fuzz library from src/test/fuzz/fuzz.cpp (Harris)

Pull request description:

  This PR creates a static library **libtest_fuzz.a** to speed up the compilation of fuzz tests. It is functionally similar to https://github.com/bitcoin/bitcoin/pull/17542

  Fixes https://github.com/bitcoin/bitcoin/issues/18527

ACKs for top commit:
  MarcoFalke:
    ACK 691e2a7af7 🦁

Tree-SHA512: 39d7d2731ca4370db518dbb969eb17ddbf9c030c3fe0dec0d04ff6578f24a128563fe5aced78300c92ce296623a7079fea5aea70619819a20c56fb34191f00ef
2020-04-05 10:33:56 +08:00
Harris
691e2a7af7 build: create test_fuzz library from src/test/fuzz/fuzz.cpp 2020-04-05 01:01:13 +02:00
MarcoFalke
4830077494 Merge #18510: fuzz: Add CScriptNum::getint coverage
faa64af960 fuzz: Add CScriptNum::getint coverage (MarcoFalke)

Pull request description:

  Add coverage for

  * https://marcofalke.github.io/btc_cov/fuzz.coverage/src/script/script.h.gcov.html#311
  * https://marcofalke.github.io/btc_cov/fuzz.coverage/src/script/script.h.gcov.html#511

ACKs for top commit:
  practicalswift:
    ACK faa64af960 -- more fuzzing coverage is better than less fuzzing coverage :)

Tree-SHA512: 1a66a2edc3740e8c286049f6c27458c59c45b01052e51684eec0e1be63ffcee94b4ba3d41d88ad715ceb3e4754fd997cf03899085982454905e86d0553d58199
2020-04-05 04:53:19 +08:00
MarcoFalke
94fd4a56ed Merge #18496: test: remove redundant sync_with_ping after add_p2p_connection
4670006762 test: remove redundant sync_with_ping after add_p2p_connection (Jon Atack)

Pull request description:

  Now that #18247 is merged, these calls are redundant.

ACKs for top commit:
  vasild:
    utACK 4670006

Tree-SHA512: bdbfe8bcf9dbdde0a8115e3a62bfe359910798d7a3010d920ffca07049cb5f97bf8fb9b6f70079b0607105192b61a6d665774e59a2b678597b47ad6237595ad5
2020-04-05 04:51:44 +08:00
MarcoFalke
e16da90d95 Merge #18518: fuzz: Extend descriptor fuzz test
fa0189955a fuzz: Extend descriptor fuzz test (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa0189955a

Tree-SHA512: 6d6a6417f06d90732bbf055ff54102530d6956f3082f1ff65598f790d588170768aee98e4835996876d28bca2a9c62f22fe122c3fc7eafd4b7660696f72f9835
2020-04-05 04:49:55 +08:00
MarcoFalke
16b6d3422b Merge #18519: fuzz: Extend script fuzz test
fa86edf66d fuzz: Extend script fuzz test (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa86edf66d

Tree-SHA512: 611adee9e673183e67f9711e49289fa59e410bb3ac1bb3fcbb7f1ed331bf0d288c7065e256a82eb41a30a4afe53544c836463cf58865d6e40b18795c8716e57c
2020-04-05 04:48:47 +08:00
MarcoFalke
490ae0e87b Merge #18520: test: remaining replacements of (send_message+sync_with_ping) with send_and_ping
3dc8c012f0 test: remaining replacements of (send_message+sync_with_ping) with send_and_ping (Sebastian Falbesoner)

Pull request description:

  This is a tiny follow-up PR to #18494, substituting the remaining occurences of `send_message(...)`/`sync_with_ping(...)` pairs with `send_and_ping(...)`, as suggested in the comment https://github.com/bitcoin/bitcoin/pull/18494#pullrequestreview-386418913. Thanks to jonatack and [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/18494#issuecomment-608496342) for giving me the hint to do this follow-up.

ACKs for top commit:
  practicalswift:
    ACK 3dc8c012f0

Tree-SHA512: 44d64332933c23a7f59c0415e008ce1b2b2e07177f81cb9473b7c71558188f1c698e8973de5cc940280e4697f9553af852d9a42841304f82469673d1c8162852
2020-04-05 04:47:21 +08:00
MarcoFalke
a3b61cf641 Merge #18509: fuzz: Avoid running over all inputs after merging them
facc332dc5 fuzz: Avoid running over all inputs after merging them (MarcoFalke)

Pull request description:

  This cuts the time it takes to merge inputs by half

ACKs for top commit:
  practicalswift:
    ACK facc332dc5

Tree-SHA512: bb22992c463dd985d3b1e9b8908c591d0c8e620c38eba0a932d880f87133bfe4ca2036b166c4f79b92ddf7940f56c044e9cb8cc50309c74204df122b369c167d
2020-04-05 04:46:12 +08:00
MarcoFalke
4839560ee1 Merge #18407: tests: Add proof-of-work fuzzing harness
acf269e146 tests: Add proof-of-work fuzzing harness (practicalswift)

Pull request description:

  Add proof-of-work fuzzing harness.

Top commit has no ACKs.

Tree-SHA512: dcdfa211cf1ec3018b61f378bb0f95793bbbe5d00e2f4d17f9db2c7263fe8ce919760c56cae7122c62c82e05c90e7056eb1778871674bdb3c42869e5fe4c2b60
2020-04-05 04:41:07 +08:00
practicalswift
acf269e146 tests: Add proof-of-work fuzzing harness 2020-04-04 17:23:50 +00:00
Russell Yanofsky
d6815a2313 refactor: drop boost::signals2 in validationinterface
Stop using boost::signals2 internally in validationinterface. Replace with
std::list and Add/Remove/Clear/Iterate helper functions.

Motivation for change is to reduce dependencies and avoid issues happening with
boost versions before 1.59:

https://github.com/bitcoin/bitcoin/issues/18517
https://github.com/bitcoin/bitcoin/pull/18471
2020-04-04 11:44:39 -04:00
fanquake
5ca90f8b59 scripts: add MACHO lazy bindings check to security-check.py 2020-04-04 09:54:25 +08:00
Sebastian Falbesoner
3dc8c012f0 test: remaining replacements of (send_message+sync_with_ping) with send_and_ping 2020-04-03 22:11:08 +02:00
MarcoFalke
c8971547d9 Merge #18499: rpc: Make rpc documentation not depend on call-time rpc args
fab32557f2 rpc: Make rpc documentation not depend on rpc args (MarcoFalke)

Pull request description:

  This is required to host the documentation on a static resource (like a website or pdf)

ACKs for top commit:
  emilengler:
    utACK fab32557f2
  promag:
    ACK fab32557f2.

Tree-SHA512: 3ca2691c7fbd5f17c75df2887753da152f66521dcb7dee4c29af6339fdea011cecdd51f825b96bde9c6aaf82f4d915cbd5aacb52e4eae3898d9dbc216f627171
2020-04-04 03:15:00 +08:00
MarcoFalke
e35e118656 Merge #18508: RPC: Fix more formatting nits
f32ab443a9 Bugfix: RPC: JSON null is not "None" (Luke Dashjr)
26dcf39581 Bugfix: RPC: Don't use a continuation elipsis after an elision elipsis (Luke Dashjr)
eca65caadc Bugfix: RPC: Add missing commas and correct indentation of explicit ELISION (Luke Dashjr)

Pull request description:

  1. listsinceblock had a double ellipsis (elision + continuation); this looks ugly, just one is needed.
  2. Elision ellipsis wasn't getting a comma, so was truncated to `".."` by comma-removal code.
  3. Elision ellipsis was indented incorrectly (as if it was a subitem).
  4. Similarly, type "none" would get truncated to `"Non"`, when it should really be `"null"` anyway.

ACKs for top commit:
  MarcoFalke:
    ACK f32ab443a9 🐰

Tree-SHA512: 34e1c72673790ed11cdee838d64ea5e0ac498de19258df99d54b5322e003060123c65ad27ac2fd4729a1dfe52066a0629602a132b1ef85d4154affd99a065a3f
2020-04-04 03:08:05 +08:00
MarcoFalke
fa86edf66d fuzz: Extend script fuzz test 2020-04-04 01:32:17 +08:00
MarcoFalke
fa0189955a fuzz: Extend descriptor fuzz test 2020-04-04 01:16:19 +08:00
fanquake
9e071b0089 test: remove rapidcheck integration and tests 2020-04-03 22:47:59 +08:00
Wladimir J. van der Laan
0eeb0468e7 net: Hardcoded seeds update for 0.20
Update hardcoded seeds from seeds_emzy.txt seeds_lukejr.txt
seeds_sipa.txt seeds_sjors.txt, according to release process.

Output from makeseeds.py:
```
  IPv4   IPv6  Onion Pass
1364173 244127   2454 Initial
1364173 244127   2454 Skip entries with invalid address
1129552 213117   2345 After removing duplicates
1129548 213117   2345 Skip entries from suspicious hosts
338216 191944   2249 Enforce minimal number of blocks
336851 188993   2189 Require service bit 1
  6998   1520    150 Require minimum uptime
  5682   1290     89 Require a known and recent user agent
  5622   1279     89 Filter out hosts with multiple bitcoin ports
   512    146     89 Look up ASNs and limit results per ASN and per net
```
2020-04-03 16:29:26 +02:00
Sebastian Falbesoner
0ed2d8e07d test: add BIP37 remote crash bug [CVE-2013-5700] test to p2p_filter.py 2020-04-03 16:00:12 +02:00
fanquake
f0d6487e29 Merge #18513: doc: fix git add argument
4928a995e9 [doc] fix git add argument (Michael Polzer)

Pull request description:

  [`A`](https://git-scm.com/docs/git-add#Documentation/git-add.txt--A) is the correct flag.

ACKs for top commit:
  fanquake:
    ACK 4928a995e9 - checked that [`A`](https://git-scm.com/docs/git-add#Documentation/git-add.txt--A) and not `a` is the correct flag.

Tree-SHA512: 7e656ca9688b04ad2ef577aa1847799a34a377f5e6dfe4fd052a95d3dd98798dc10957e7f54164900ac1271f05e788ec4861026f53b910e369b0845532387cf4
2020-04-03 19:37:11 +08:00
fanquake
08c4994969 Merge #18503: init: Replace URL_WEBSITE with PACKAGE_URL
fad2f68353 init: Replace URL_WEBSITE with PACKAGE_URL (MarcoFalke)

Pull request description:

  This is needed for rebranding efforts such as #18489

ACKs for top commit:
  hebasto:
    ACK fad2f68353, tested on Linux Mint 19.3:
  fanquake:
    ACK fad2f68353 - clicked a link.

Tree-SHA512: c26e18cd328d3dd3fd7e25413e1bab780026687a148f126b8673e5f6cc13249f6c16689e45eba9da1545915c6001f96cd33f4e656c08cda3eae1c3fd88da23ea
2020-04-03 19:11:07 +08:00
Michael Polzer
4928a995e9 [doc] fix git add argument
error: unknown switch `a'
usage: git add [<options>] [--] <pathspec>...

    -n, --dry-run         dry run
    -v, --verbose         be verbose

    -i, --interactive     interactive picking
    -p, --patch           select hunks interactively
    -e, --edit            edit current diff and apply
    -f, --force           allow adding otherwise ignored files
    -u, --update          update tracked files
    --renormalize         renormalize EOL of tracked files (implies -u)
    -N, --intent-to-add   record only the fact that the path will be added later
    -A, --all             add changes from all tracked and untracked files
    --ignore-removal      ignore paths removed in the working tree (same as --no-all)
    --refresh             don't add, only refresh the index
    --ignore-errors       just skip files which cannot be added because of errors
    --ignore-missing      check if - even missing - files are ignored in dry run
    --chmod (+|-)x        override the executable bit of the listed files
2020-04-03 12:52:36 +02:00
fanquake
be60e37e40 Merge #18505: doc: Update webchat URLs in README.md
7b2975ae8f doc: Update webchat URLs in README.md (Suriyaa Sundararuban)

Pull request description:

  #### What happend?
  Web links in `doc/README.md` redirected from `http://webchat.freenode.net?channels=bitcoin` to `https://webchat.freenode.net/#bitcoin`.

  #### What did I changed?
  * Remove URL redirection. (Update all webchat links.)
  * Use HTTPS protocol instead of HTTP.

ACKs for top commit:
  fanquake:
    ACK 7b2975ae8f

Tree-SHA512: e55970ad368ce6dbb79295d12629d0f16318e7c43d8d194876e16e81c6e325bb136c9e8b361d61c5a04e2f3624350f81c99131b1e85d9ee5410ac22f524a4ef2
2020-04-03 18:24:38 +08:00
fanquake
d478595fad Merge #18382: doc: note the costs of fetching all pull requests
d695eb4c21 doc: note the costs of fetching all pull requests (Vasil Dimov)

Pull request description:

  Also mention that it is possible to fetch just one pull request.

ACKs for top commit:
  MarcoFalke:
    ACK d695eb4c21
  fanquake:
    ACK d695eb4c21

Tree-SHA512: afe080fd018b2e773fb974956937e819085831bf0c1c5623f7f12c728639906b80666b785234058ee39fd98115a53a2fad431c54ee0840667e60bb317e4a828d
2020-04-03 18:17:09 +08:00
fanquake
6ec42df32b Merge #18426: scripts: previous_release: improve behaviour on failed download
332f373a9d [scripts] previous_release: improve failed download error message (Sebastian Falbesoner)

Pull request description:

  Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output:
  ```
  $ contrib/devtools/previous_release.sh -r -b v0.9.5
  [...]
  gzip: stdin: not in gzip format
  tar: Child returned status 1
  tar: Error is not recoverable: exiting now
  ```
  This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f

  On the PR branch, the output on failed download looks now the following:
  ```
  $ contrib/devtools/previous_release.sh -r -b v0.9.5
  [...]
  curl: (22) The requested URL returned error: 404 Not Found
  Download failed.
  ```

ACKs for top commit:
  fanquake:
    ACK 332f373a9d

Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
2020-04-03 18:10:28 +08:00
MarcoFalke
faa64af960 fuzz: Add CScriptNum::getint coverage 2020-04-03 09:02:34 +08:00
MarcoFalke
facc332dc5 fuzz: Avoid running over all inputs after merging them 2020-04-03 08:30:40 +08:00
MarcoFalke
dce6f3b29b Merge #18383: refactor: Check for overflow when calculating sum of tx outputs
f65c9ad40f Check for overflow when calculating sum of outputs (Elichai Turkel)

Pull request description:

  This was reported by practicalswift here #18046
  The exact order of the if, is important, we first do `!MoneyRange(tx_out.nValue)` to make sure the amount is non-negative. and then `std::numeric_limits<CAmount>::max() - tx_out.nValue < nValueOut` checks that the addition cannot overflow (if we won't check that the amount is positive this check can also overflow! (by doing something like `max - -max`))
  and only then we make sure that the some is also valid `!MoneyRange(nValueOut + tx_out.nValue)`
  if any of these conditions fail we throw.

  the overflowing logic:
  ```
  a + b > max // we want to fail if a+b is more than the maximum -> will overflow
  b > max - a
  max - a < b
  ```

  Closes: #18046

ACKs for top commit:
  MarcoFalke:
    ACK f65c9ad40f, checked that clang with O2 produces identical binaries 💕
  practicalswift:
    ACK f65c9ad40f
  instagibbs:
    utACK f65c9ad40f
  vasild:
    ACK f65c9ad40f modulo `s/assert.h/cassert/`

Tree-SHA512: 512d6cf4762f24c41cf9a38da486b17b19c634fa3f4efbdebfe6608779e96fc3014d5d2d29adb8001e113152c0217bbd5b3900ac4edc7b8abe77f82f36209e33
2020-04-03 06:50:29 +08:00
Carl Dong
35a96792dd guix: Check mingw symbols, improve SSP fix docs 2020-04-02 17:20:05 -04:00
Carl Dong
449d8fe25b guix: Expand on INT trap message 2020-04-02 17:20:04 -04:00
Carl Dong
3f1f03c67a guix: Spelling fixes 2020-04-02 17:20:03 -04:00
Carl Dong
ff821dd2a1 guix: Reinstate make-ssp-fixed-gcc
Unfortunately, gcc is still not smart enough to detect whether or not
mingw-w64 provides ssp, so let's put it back just for mingw-w64.
2020-04-02 17:20:02 -04:00
Carl Dong
360a9e0ad5 guix: Bump time-machine for mingw-w64 patches
This bump will includes a couple of commits which improve the
reproducibility of the mingw-w64 toolchain. Most of which came from
debian. They will be upstreamed as upstream Guix release timeline
allows.
2020-04-02 17:20:01 -04:00
Carl Dong
93e41b7e3b guix: Use gcc-8 for mingw-w64 instead of 7
We're using mingw-w64 6.0.0, which is paired with gcc-8 in most distros.
2020-04-02 17:20:00 -04:00
Carl Dong
ef4f7e4c45 guix: Set the well-known timezone env var 2020-04-02 17:19:59 -04:00
Carl Dong
acf4b3b3b5 guix: Make x86_64-w64-mingw32 builds reproducible
- Add "--no-insert-timestamp" LDFLAG for x86_64-w64-mingw32 builds

"The option --no-insert-timestamp can be used to insert a zero value for
the timestamp, this ensuring that binaries produced from identical
sources will compare identically." - ld(1)

- Set "SetDateSave off" in NSIS script

From https://nsis.sourceforge.io/Docs/Chapter4.html#flags

"This command sets the file date/time saving flag which is used by the
File command to determine whether or not to save the last write date and
time of the file, so that it can be restored on installation. Valid
flags are 'on' and 'off'. 'on' is the default."

- Add commented out NSIS options for reproducibility debugging in NSIS
  script

- Make ZIPs deterministic by reseting file modification times to
  SOURCE_DATE_EPOCH using touch(1) (Reference:
  https://reproducible-builds.org/docs/archives/)
2020-04-02 17:19:57 -04:00
Carl Dong
c4cce00eac guix: Remove dead links from README. 2020-04-02 17:19:56 -04:00
Carl Dong
df953a4c9a guix: Appease shellcheck. 2020-04-02 17:19:55 -04:00
Carl Dong
91897c95e1 guix: Improve guix-build.sh documentation 2020-04-02 17:19:54 -04:00
Carl Dong
570d769c6c guix: Build support for Windows 2020-04-02 17:19:53 -04:00
MarcoFalke
0d71395848 Merge #18464: doc: block-relay-only vs blocksonly
fa6e01f2a1 doc: block-relay-only is not blocksonly (MarcoFalke)

Pull request description:

  Those are different concepts, see https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes/release-notes-0.19.0.1.md#p2p-changes for the block-relay-only nodes.

ACKs for top commit:
  jonatack:
    ACK fa6e01f
  hebasto:
    ACK fa6e01f2a1

Tree-SHA512: 6de2c81201b62ed59e504a3a6f164068600182e1bbf63eda7f9db3160507bdba091c13882ee0e75e713f0832bfaf5973a86eba3b94588d5b72196f05ae0a9c9a
2020-04-03 05:06:23 +08:00
MarcoFalke
ff53433fe4 Merge #18494: test: replace (send_message + sync_with_ping) with send_and_ping
6112a20982 test: replace (send_message + sync_with_ping) with send_and_ping (Jon Atack)

Pull request description:

  This is a follow-up to faf1d04731 yesterday.

ACKs for top commit:
  vasild:
    utACK 6112a20
  MarcoFalke:
    ACK 6112a20982 🎞

Tree-SHA512: 749644ac9a1ef0e1aa6c3ac5e899eb3fa7fb9c0909352f922a80412df2bc0e539692a7757af550eff4d4914cbe57b0c75ce3948f569acc7a52852e91a55ad457
2020-04-03 02:12:46 +08:00
Luke Dashjr
b5795a7886 Wallet: Add warning comments and assert to CWallet::DelAddressBook 2020-04-02 16:37:42 +00:00
Luke Dashjr
6d2905f57a Wallet: Avoid unnecessary/redundant m_address_book lookups 2020-04-02 16:37:41 +00:00
João Barbosa
7b8e15728d rpc: Fix rpcRunLater race in walletpassphrase 2020-04-02 17:25:27 +01:00
Luke Dashjr
c751d886f4 Wallet: Avoid treating change-in-the-addressbook as non-change everywhere 2020-04-02 16:25:17 +00:00
Luke Dashjr
8e64b8c84b Wallet: New FindAddressBookEntry method to filter out change entries (and skip ->second everywhere) 2020-04-02 16:02:56 +00:00
Luke Dashjr
65b6bdc2b1 Wallet: Add CAddressBookData::IsChange which returns true iff label has never been set 2020-04-02 16:02:07 +00:00
Luke Dashjr
144b2f85da Wallet: Require usage of new CAddressBookData::setLabel to change label 2020-04-02 16:01:36 +00:00
Luke Dashjr
b86cd155f6 scripted-diff: Wallet: Rename mapAddressBook to m_address_book
Previous versions assumed absence of an entry in mapAddressBook indicated change.
This no longer holds true (due to bugs) and will shortly be made intentional.
Renaming the field helps ensure that old code using mapAddressBook directly gets checked for necessary rebasing.

-BEGIN VERIFY SCRIPT-
sed -i -e 's/mapAddressBook/m_address_book/g' $(git grep -l 'mapAddressBook' ./src)
-END VERIFY SCRIPT-
2020-04-02 16:00:28 +00:00
MarcoFalke
fa1da3d4bf test: Add basic addr relay test 2020-04-02 23:29:32 +08:00
Luke Dashjr
f32ab443a9 Bugfix: RPC: JSON null is not "None" 2020-04-02 15:28:05 +00:00
Luke Dashjr
26dcf39581 Bugfix: RPC: Don't use a continuation elipsis after an elision elipsis 2020-04-02 15:24:01 +00:00
Luke Dashjr
eca65caadc Bugfix: RPC: Add missing commas and correct indentation of explicit ELISION 2020-04-02 15:24:00 +00:00
MarcoFalke
fa1793c1c4 net: Pass connman const when relaying address 2020-04-02 21:56:20 +08:00
Wladimir J. van der Laan
b83565625e Merge #18500: chainparams: Bump assumed valid hash
222253e37d chainparams: Bump assumed valid hash (MarcoFalke)

Pull request description:

  See https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-branch-off

ACKs for top commit:
  fanquake:
    ACK 222253e37d - matches output on my node. Also have 3.8GB for the chainstate dir. If anyone wants to know what commands they should run / what to check; I have some docs on [assumevalid updates](https://github.com/fanquake/core-review/blob/master/update-assumevalid.md).
  elichai:
    ACK 222253e37d
  theStack:
    ACK 222253e37d
  instagibbs:
    ACK 222253e37d all mainnet params, did not check testnet

Tree-SHA512: 856e75f0c6663ab478768d80618f11ac109ef30058661f58ed7f2a3a615dd5fb8f8d648c996eff77949036c1633119085b25cd4c25250a0850c943c02cddf3d7
2020-04-02 15:43:49 +02:00
Suriyaa Sundararuban
7b2975ae8f doc: Update webchat URLs in README.md 2020-04-02 13:09:55 +00:00
Wladimir J. van der Laan
6bdd515ccf Merge #16923: wallet: Handle duplicate fileid exception
9eefc6e92f gui: Delete progress dialog instead of hidding it (João Barbosa)
ee9e88ba27 wallet: Handle duplicate fileid exception (João Barbosa)

Pull request description:

  Handle the duplicate fileid exception thrown at `CheckUniqueFileid` in tow cases:
   - when duplicate wallets are set on the command line - catch in `LoadWallets`;
   - when a duplicate wallet is loaded dynamically - catch in `LoadWallet`.

  Fixes #16776.

ACKs for top commit:
  jonatack:
    Re-ACK 9eefc6e92f no change since last review 68e0ff0e1f530c942721aab49cf67ffc07104628
  hebasto:
    re-ACK 9eefc6e92f

Tree-SHA512: 46e3c1cd6708b54e2d1c4973a74c8d5428822e04cecbc147cf200eb034efa385e867bd749c7c639020e83c9813fae8fed64a851bdd99abf60c33b07e0363f5d5
2020-04-02 15:07:55 +02:00
MarcoFalke
fad2f68353 init: Replace URL_WEBSITE with PACKAGE_URL 2020-04-02 20:52:47 +08:00
Russell Yanofsky
01a3392b1b Drop bitcoin-wallet dependency on libevent
Don't require urlDecode function in wallet code since urlDecode implementation
currently uses libevent. Just call urlDecode indirectly though URL_DECODE
function pointer constant if available.

In bitcoind and bitcoin-qt, URL_DECODE is implemented and used to interpret RPC
wallet requests. In bitcoin-wallet, URL_DECODE is null to avoid depending on
libevent.
2020-04-02 08:35:10 -04:00
Russell Yanofsky
0660119ac3 Drop unintended bitcoin-tx dependency on libevent
Don't include util/url.cpp to libbitcoin_util.a when libevent isn't available.
This fixes a compile error trying to build bitcoin-tx without libevent reported
by Luke Dashjr in https://github.com/bitcoin/bitcoin/issues/18465

Fixes #18465
2020-04-02 08:28:11 -04:00
Wladimir J. van der Laan
5c1ba3a10a Merge #18358: util: fix compilation with mingw-w64 7.0.0
a46484c8b3 build: Detect gmtime_* definitions via configure (Ben Woosley)

Pull request description:

  Something has changed in the mingw-w64 headers such that we
  no-longer compile when using 7.0.0.

  ```bash
  util/time.cpp: In function 'std::__cxx11::string FormatISO8601DateTime(int64_t)':
  util/time.cpp:84:9: error: 'gmtime_r' was not declared in this scope
       if (gmtime_r(&time_val, &ts) == nullptr) {
           ^~~~~~~~
  util/time.cpp: In function 'std::__cxx11::string FormatISO8601Date(int64_t)':
  util/time.cpp:97:9: error: 'gmtime_r' was not declared in this scope
       if (gmtime_r(&time_val, &ts) == nullptr) {
  ```

  Looking at time.h, it seems that `gmtime_r()` is only available when
  `_POSIX_C_SOURCE` is defined. This must have been the case for 6.0.0
  (which we compile fine using), but no-longer seems to be for 7.0.0?

  I've checked that adding `-D_POSIX_C_SOURCE=200112L` to our compile
  flags does fix the issue above.

  However, an alternative solution seems to be to just use `gmtime_s()`
  instead, when compiling with `mingw-w64`, as `gmtime_r()` [just wraps
  `gmtime_s()` anyways](7c03b11bf1/mingw-w64-headers/crt/time.h (L284)).

  I've tested this change crosss-compiling on Debian Bullseye ([mingw-w64 7.0.0](https://packages.debian.org/source/bullseye/mingw-w64))
  and Buster ([mingw-w64 6.0.0](https://packages.debian.org/source/buster/mingw-w64)).

ACKs for top commit:
  laanwj:
    ACK a46484c8b3

Tree-SHA512: 7cf1a81060b9625d64de40b77341d74704cc8ae1358d25d7e2909685dc83a7a9762260d72e47806e9f0a5cbabf88d0239ec9e0fd0ebd3731b1d206b075f43a63
2020-04-02 12:47:13 +02:00
Ben Woosley
a46484c8b3 build: Detect gmtime_* definitions via configure
This improves the portability of the codebase and fixes compilation
with mingw-w64 7.0+.

Co-authored-by: fanquake <fanquake@gmail.com>
2020-04-02 12:31:54 +08:00
MarcoFalke
222253e37d chainparams: Bump assumed valid hash 2020-04-02 05:34:47 +08:00
MarcoFalke
fab32557f2 rpc: Make rpc documentation not depend on rpc args 2020-04-02 00:25:13 +08:00
Jon Atack
6112a20982 test: replace (send_message + sync_with_ping) with send_and_ping 2020-04-01 17:30:02 +02:00
Jon Atack
4670006762 test: remove redundant sync_with_ping after add_p2p_connection 2020-04-01 16:18:59 +02:00
Vasil Dimov
0753efd9dc rpc: Remove deprecated "size" from mempool txs
Remove the "size" property of a mempool transaction from RPC replies.

Deprecated in e16b6a718 in 0.19, about 1 year ago.
2020-04-01 14:19:34 +02:00
Wladimir J. van der Laan
41fa2926d8 Merge #18492: qt: Translations update pre-branch
740614e3c3 qt: Translations update pre-branch (Wladimir J. van der Laan)

Pull request description:

  Update translations from transifex before the 0.20 branch-off. As we don't have translations for the master branch (only for release versions), this is the last opportunity (for this release) to get up to date translations on master.

ACKs for top commit:
  elichai:
    utACK 740614e3c3
  fanquake:
    ACK 740614e3c3

Tree-SHA512: 108b82bac967009cab2257d77f0fbc38d18497ffdfe34b06aba35a1894b9b7ebff8ba4671219f4e20ebc42f984ed9ef65c5d41d21ac08dadfef798474f0b2b3b
2020-04-01 14:11:23 +02:00
Wladimir J. van der Laan
740614e3c3 qt: Translations update pre-branch 2020-04-01 12:49:15 +02:00
fanquake
b97e3a7349 Merge #18477: ci: Use Focal for fuzzers
faa9491870 ci: Use Focal for fuzzers (MarcoFalke)

Pull request description:

  This gives us access to clang-10, as well as a newer version of valgrind

ACKs for top commit:
  fanquake:
    ACK faa9491870 - [Clang 10](https://packages.ubuntu.com/focal/clang) and [valgrind 3.15](https://packages.ubuntu.com/focal/valgrind).
  practicalswift:
    ACK faa9491870 -- diff looks correct & contemporary clang is better than vintage clang

Tree-SHA512: 0e67232673434c0309db79c1054e3e981115083585945967e346f4d58792635832100f89911428aab928155e44e5f401207a023681ae008fdb5280cf02c4d427
2020-04-01 15:00:22 +08:00
MarcoFalke
67a72466e1 Merge #18247: test: Wait for both veracks in add_p2p_connection
faf1d04731 test: Remove redundant sync_with_ping after add_p2p_connection (MarcoFalke)
fa90647045 test: Wait for both veracks in add_p2p_connection (MarcoFalke)

Pull request description:

  This fixes the race in p2p_blocksonly

  E.g. https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/657038844#L4500

  ```
   ...
   test  2020-03-01T20:58:28.825000Z TestFramework.mininode (DEBUG): Closed connection to: 127.0.0.1:11828
   node0 2020-03-01T20:58:28.825642Z [net] disconnecting peer=0
   node0 2020-03-01T20:58:28.825826Z [net] Cleared nodestate for peer=0
   node0 2020-03-01T20:58:28.875835Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.876067Z [httpworker.0] ThreadRPCServer method=getmempoolinfo user=__cookie__
   test  2020-03-01T20:58:28.877000Z TestFramework.mininode (DEBUG): Connecting to Bitcoin Node: 127.0.0.1:11828
   test  2020-03-01T20:58:28.878000Z TestFramework.mininode (DEBUG): Connected & Listening: 127.0.0.1:11828
   test  2020-03-01T20:58:28.878000Z TestFramework.mininode (DEBUG): Send message to 127.0.0.1:11828: msg_version(nVersion=70014 nServices=9 nTime=Sun Mar  1 20:58:28 2020 addrTo=CAddress(nServices=1 ip=127.0.0.1 port=11828) addrFrom=CAddress(nServices=1 ip=0.0.0.0 port=0) nNonce=0x164D5DEB952A4A0B strSubVer=b'/python-mininode-tester:0.0.3/' nStartingHeight=-1 nRelay=1)
   node0 2020-03-01T20:58:28.883808Z [net] Added connection peer=1
   node0 2020-03-01T20:58:28.883950Z [net] connection from 127.0.0.1:33798 accepted
   node0 2020-03-01T20:58:28.884300Z [msghand] received: version (116 bytes) peer=1
   node0 2020-03-01T20:58:28.884483Z [msghand] sending version (114 bytes) peer=1
   node0 2020-03-01T20:58:28.884700Z [msghand] send version message: version 70015, blocks=200, us=[::]:0, peer=1
   node0 2020-03-01T20:58:28.884765Z [msghand] sending verack (0 bytes) peer=1
   test  2020-03-01T20:58:28.885000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_version(nVersion=70015 nServices=1033 nTime=Sun Mar  1 20:58:28 2020 addrTo=CAddress(nServices=0 ip=0.0.0.0 port=0) addrFrom=CAddress(nServices=1033 ip=0.0.0.0 port=0) nNonce=0x4A0F2F4C549B3399 strSubVer=b'/Satoshi:0.19.99(testnode0)/' nStartingHeight=200 nRelay=0)
   test  2020-03-01T20:58:28.885000Z TestFramework.mininode (DEBUG): Send message to 127.0.0.1:11828: msg_verack()
   test  2020-03-01T20:58:28.885000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_verack()
   node0 2020-03-01T20:58:28.885004Z [msghand] receive version message: /python-mininode-tester:0.0.3/: version 70014, blocks=-1, us=127.0.0.1:11828, peer=1
   test  2020-03-01T20:58:28.886000Z TestFramework (INFO): Check that txs from rpc are not rejected and relayed to other peers
   node0 2020-03-01T20:58:28.886556Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.886783Z [httpworker.1] ThreadRPCServer method=getpeerinfo user=__cookie__
   node0 2020-03-01T20:58:28.889032Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.889294Z [httpworker.2] ThreadRPCServer method=testmempoolaccept user=__cookie__
   node0 2020-03-01T20:58:28.891655Z [http] Received a POST request for / from 127.0.0.1:53448
   node0 2020-03-01T20:58:28.891963Z [httpworker.3] ThreadRPCServer method=sendrawtransaction user=__cookie__
   node0 2020-03-01T20:58:28.893115Z [httpworker.3] Enqueuing TransactionAddedToMempool: txid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f wtxid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f
   node0 2020-03-01T20:58:28.893443Z [scheduler] TransactionAddedToMempool: txid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f wtxid=af34fc5ff9ea8babbd4083fbb79ffd2ad5aff1d6def803c07ca5aeed880bd60f
   node0 2020-03-01T20:58:28.894814Z [msghand] received: verack (0 bytes) peer=1
   node0 2020-03-01T20:58:28.894937Z [msghand] sending sendheaders (0 bytes) peer=1
   node0 2020-03-01T20:58:28.895087Z [msghand] sending sendcmpct (9 bytes) peer=1
   node0 2020-03-01T20:58:28.895235Z [msghand] sending sendcmpct (9 bytes) peer=1
   node0 2020-03-01T20:58:28.895430Z [msghand] sending ping (8 bytes) peer=1
   node0 2020-03-01T20:58:28.895896Z [msghand] initial getheaders (199) to peer=1 (startheight:-1)
   test  2020-03-01T20:58:28.896000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_sendheaders()
   node0 2020-03-01T20:58:28.896016Z [msghand] sending getheaders (645 bytes) peer=1
   node0 2020-03-01T20:58:28.896607Z [msghand] sending feefilter (8 bytes) peer=1
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_sendcmpct(announce=False, version=2)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_sendcmpct(announce=False, version=1)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_ping(nonce=f735096062d217b5)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Send message to 127.0.0.1:11828: msg_pong(nonce=f735096062d217b5)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_getheaders(locator=CBlockLocator(nVersion=70014 vHave=[48924041037103782797700918670732352379567180837453042168545380831411841797392, 28010422273815860773972769588722664110955084223364219183119416607410792753789, 5954376895683677137597080246740451260829355661937599865380797589540815086241, 14500403275336359851183244421245184901482464358719551678581030092830439955257, 17853919108052771837249729512111680264864054213441538187113939176285784834878, 28843166929059356839755035875664073555480989477... (msg truncated)
   test  2020-03-01T20:58:28.897000Z TestFramework.mininode (DEBUG): Received message from 127.0.0.1:11828: msg_feefilter(feerate=000003e8)
   node0 2020-03-01T20:58:28.898144Z [msghand] received: pong (8 bytes) peer=1
   node0 2020-03-01T20:59:28.338539Z [scheduler] Feeding 13446 bytes of dynamic environment data into RNG
   test  2020-03-01T20:59:28.908000Z TestFramework.utils (ERROR): wait_until() failed. Predicate: ''''
                                             def test_function():
                                                 assert self.is_connected
                                                 if not self.last_message.get('tx'):
                                                     return False
                                                 return self.last_message['tx'].tx.rehash() == txid
                                     '''
   test  2020-03-01T20:59:28.908000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 112, in main
                                         self.run_test()
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_blocksonly.py", line 57, in run_test
                                         self.nodes[0].p2p.wait_for_tx(txid)
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/mininode.py", line 369, in wait_for_tx
                                         wait_until(test_function, timeout=timeout, lock=mininode_lock)
                                       File "/home/travis/build/MarcoFalke/bitcoin-core/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/util.py", line 234, in wait_until
                                         raise AssertionError("Predicate {} not true after {} seconds".format(predicate_source, timeout))
                                     AssertionError: Predicate ''''
                                             def test_function():
                                                 assert self.is_connected
                                                 if not self.last_message.get('tx'):
                                                     return False
                                                 return self.last_message['tx'].tx.rehash() == txid
                                     ''' not true after 60 seconds

ACKs for top commit:
  jonatack:
    ACK faf1d04

Tree-SHA512: 3b1a38a5c87d11c610eee0988f0c4af9bfcd978df9ac718ef611f663df2fd4a0eb04e077df5e940d15971bb2f22328fb6021cacccb6902f1e527f288ad2c4a2c
2020-04-01 05:08:41 +02:00
MarcoFalke
eb773f1063 Merge #18486: doc: Explain new test logging
7777703958 doc: Explain new test logging (MarcoFalke)

Pull request description:

  Explain logging added in #18472 and #16975

ACKs for top commit:
  jonatack:
    ACK 7777703

Tree-SHA512: 3a0aa7bab32a6753d8894d29cf82604b044b23e512102dd275b717eefda3c2212dbf43ea7e9155267350dd9f3bc5badba2eb660152db3efeab30a04f52126c95
2020-04-01 05:06:55 +02:00
Andrew Chow
cd3b1569d9 Correctly compute redeemScript from witnessScript for signrawtransaction
ParsePrevouts uses GetScriptForWitness on the given witnessScript
to find the corresponding redeemScript. This is incorrect when the
witnessScript is either a P2PK or P2PKH script as it returns the
corresponding P2WPK script instead of turning the witnessScript
into a P2WSH script. Instead this should make the script a
WitnessV0ScriptHash destination and get the script for that.

Test cases are also added.
2020-03-31 18:41:52 -04:00
MarcoFalke
7777703958 doc: Explain new test logging 2020-03-31 17:11:47 -04:00
MarcoFalke
faf1d04731 test: Remove redundant sync_with_ping after add_p2p_connection
Also replace the two-line (send_message + sync_with_ping) with the one-line send_and_ping
2020-03-31 17:03:24 -04:00
MarcoFalke
fa90647045 test: Wait for both veracks in add_p2p_connection 2020-03-31 17:03:03 -04:00
MarcoFalke
d2db25233c Merge #18474: test: check that peer is connected when calling sync_*
fac3716b09 test: check that peer is connected when calling sync_* (MarcoFalke)

Pull request description:

  Without a connection there is no way to sync, so we can fail early and don't have to wait for the timeout

ACKs for top commit:
  jonatack:
    ACK fac3716b09

Tree-SHA512: 12f771473c23e152dae4bfb201fadb2c3530cf439de64fea07d048734614543080a5d05c9c36e6e398c6a69c8279f609d34706599571814172a11bcfbea4a3b9
2020-03-31 16:35:03 -04:00
MarcoFalke
72ef366564 Merge #18472: test: Remove unsafe BOOST_TEST_MESSAGE
fa3cc0bfc4 test: Remove unsafe BOOST_TEST_MESSAGE (MarcoFalke)

Pull request description:

  Fixes #17987

  Can be tested with

  ```
  ./src/test/test_bitcoin -l test_suite  -- DEBUG_LOG_OUT

ACKs for top commit:
  fjahr:
    tested ACK fa3cc0bfc4
  mzumsande:
    Tested ACK fa3cc0bfc4

Tree-SHA512: f63b110d77882cd7c0d7574ff6c9c948db8febb3400ecdac45164746b587b0fa223463041801271b3959267ddc1d9a4a67ba76939e242e7dd2f92a2834a400a0
2020-03-31 15:39:15 -04:00
Fabian Jahr
bda84a08a0 rpc: Add documentation for deactivating settxfee 2020-03-31 17:07:33 +02:00
Russell Yanofsky
48973402d8 wallet: Avoid use of Chain::Lock in CWallet::GetKeyBirthTimes
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip, where it will treat the last block processed as the
current tip.
2020-03-31 08:36:02 -05:00
MarcoFalke
d52ba21dff Merge #18481: test: add BIP37 'filterclear' test to p2p_filter.py
0055922958 test: add BIP37 'filterclear' test to p2p_filter.py (Sebastian Falbesoner)

Pull request description:

  Integrates the message type `filterclear` to the test framework and adds a simple test to `p2p_filter.py`, checking that arbitrary txs get relayed again after deleting the filter.

ACKs for top commit:
  naumenkogs:
    utACK 0055922958

Tree-SHA512: fe64e99a526865770707d8077b9968d3923f248045ec7fa56cd380dba85ac77a71a473d244ef3aede2fc0d287b8d7c6bc0156b6033b0c949c2058cc08e255697
2020-03-31 09:36:02 -04:00
Russell Yanofsky
e958ff9ab5 wallet: Avoid use of Chain::Lock in CWallet::CreateTransaction
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip, where it may set a different lock time.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
c0d07dc4cb wallet: Avoid use of Chain::Lock in CWallet::ScanForWalletTransactions
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change affects behavior in a few small ways.

- If there's no max_height specified, percentage progress is measured ending at
  wallet last processed block instead of node tip

- More consistent error reporting: Early check to see if start_block is on the
  active chain is removed, so start_block is always read and the triggers an
  error if it's unavailable
2020-03-31 08:36:02 -05:00
Russell Yanofsky
1be8ff280c wallet: Avoid use of Chain::Lock in rescanblockchain
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip. The rescanblockchain error height error checking
will just be stricter in this case and only accept values up to the last
processed height
2020-03-31 08:36:02 -05:00
Russell Yanofsky
3cb85ac594 wallet refactor: Avoid use of Chain::Lock in CWallet::RescanFromTime
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change has no effect on behavior.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
f7ba881bc6 wallet: Avoid use of Chain::Lock in listsinceblock
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip. Previously listsinceblock might not have returned
all transactions up to the claimed "lastblock" value in this case, resulting in
race conditions and potentially missing transactions in cases where
listsinceblock was called in a loop like
https://github.com/bitcoin/bitcoin/issues/14338#issuecomment-426706574
2020-03-31 08:36:02 -05:00
Russell Yanofsky
bc96a9bfc6 wallet: Avoid use of Chain::Lock in importmulti
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip, in which case it may use a more accurate rescan
time.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
25a9fcf9e5 wallet: Avoid use of Chain::Lock in importwallet and dumpwallet
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip, in which case it will use more accurate backup and
rescan timestamps.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
c1694ce6bb wallet: Avoid use of Chain::Lock in importprunedfunds
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change only affects behavior in the case where wallet last block processed
falls behind the chain tip, in which case the "Block not found in chain" error
will be stricter and not allow importing data from a blocks between the wallet
last processed tip and the current node tip.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
ade5f87971 wallet refactor: Avoid use of Chain::Lock in qt wallettests
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

This change doesn't affect behavior.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
f6da44ccce wallet: Avoid use of Chain::Lock in tryGetTxStatus and tryGetBalances
This is a step toward removing the Chain::Lock class and reducing cs_main
locking.

It also helps ensure the GUI display stays up to date in the case where the
node chain height runs ahead of wallet last block processed height.
2020-03-31 08:36:02 -05:00
Russell Yanofsky
bf30cd4922 refactor: Add interfaces::FoundBlock class to selectively return block data
FoundBlock class allows interfaces::Chain::findBlock to return more block
information without having lots of optional output parameters. FoundBlock class
is also used by other chain methods in upcoming commits.

There is mostly no change in behavior. Only exception is
CWallet::RescanFromTime now throwing NonFatalCheckError instead of
std::logic_error.
2020-03-31 08:36:02 -05:00
João Barbosa
9eefc6e92f gui: Delete progress dialog instead of hidding it 2020-03-31 14:23:43 +01:00
João Barbosa
ee9e88ba27 wallet: Handle duplicate fileid exception 2020-03-31 14:23:43 +01:00
Wladimir J. van der Laan
9a2b5f22c1 Merge #18338: Fix wallet unload race condition
41b0baf43c gui: Handle WalletModel::unload asynchronous (João Barbosa)
ab31b9d6fe Fix wallet unload race condition (Russell Yanofsky)

Pull request description:

  This PR consists in two fixes. The first fixes a concurrency issues with `boost::signals2`. The second fixes a wallet model destruction while it's being used.

  From boost signal documentation at https://www.boost.org/doc/libs/1_72_0/doc/html/signals2/thread-safety.html:

  > When a signal is invoked by calling signal::operator(), the invocation first acquires a lock on the signal's mutex. Then it obtains a handle to the signal's slot list and combiner. Next it releases the signal's mutex, before invoking the combiner to iterate through the slot list.

  This means that `UnregisterValidationInterface` doesn't prevent more calls to that interface. The fix consists in capturing the `shared_ptr<CValidationInterface>` in each internal slot.

  The GUI bug is fixed by using a `Qt::QueuedConnection` in the `WalletModel::unload` connection.

ACKs for top commit:
  ryanofsky:
    Code review ACK 41b0baf43c. Only change is moving assert as suggested
  hebasto:
    ACK 41b0baf43c, tested on Linux Mint 19.3.

Tree-SHA512: 4f712d8de65bc1214411831250de5dc0a9fd505fb84da5baf9f2cc4d551bc3abffc061616f00afe43dba7525af2cd96c9b54aeead9383145e3b8801f25d85f50
2020-03-31 15:07:06 +02:00
Vasil Dimov
d695eb4c21 doc: note the costs of fetching all pull requests
Also, update the example to skip downloading the merge commits when
downloading all PRs.
2020-03-31 14:42:09 +02:00
Wladimir J. van der Laan
f2880e21ef Merge #18160: gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged
0933a37078 gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged (João Barbosa)

Pull request description:

  Each 250ms the slot `WalletModel::pollBalanceChanged` is called which, at worst case, calls `Wallet::GetBalance`. This is a waste of resources since most of the time there aren't new transactions or new blocks. Fix this by early checking if cache is dirty or not.

  The actual balance computation can still hang the GUI thread but that is tracked in #16874 and should be fixed with a solution similar to #17135.

ACKs for top commit:
  hebasto:
    ACK 0933a37078, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged.
  jonasschnelli:
    utACK 0933a37078
  instagibbs:
    ACK 0933a37078
  ryanofsky:
    Code review ACK 0933a37078, but I would prefer (not strongly) for #17905 to be merged first. This PR can be simpler if it is based on #17905, so tryGetBalances can just be left alone instead of changing into to a more complicated tryGetBalancesIfNeeded function, and then getting changed back later when we want to optimize it out.
  jonatack:
    ACK 0933a37078 based primarily on code review, despite a lot of manual testing with a large 177MB wallet.

Tree-SHA512: 18db35bf33a7577666658c8cb0b57308c8474baa5ea95bf1468cd8531a69857d8915584f6ac505874717aa6aabeb1b506ac77630f8acdb6651afab89275e38a1
2020-03-31 14:23:30 +02:00
Sebastian Falbesoner
0055922958 test: add BIP37 'filterclear' test to p2p_filter.py 2020-03-31 11:14:48 +02:00
Pieter Wuille
4eb5643e35 Convert everything except wallet/qt to new serialization 2020-03-30 16:10:30 -07:00
Pieter Wuille
2b1f85e8c5 Convert blockencodings_tests to new serialization 2020-03-30 16:10:30 -07:00
Pieter Wuille
73747afbbe Convert merkleblock to new serialization 2020-03-30 16:09:51 -07:00
MarcoFalke
965c0c37d5 Merge #18420: test: listsinceblock block height checks
83e1d92413 test: listsinceblock block height checks (Jon Atack)

Pull request description:

  This is the second commit of #17535.

  This PR extends a listsinceblock test to check the new transaction 'blockheight' field recently added in #17437. It also cleans up code in the test function without changing or removing existing checks.

ACKs for top commit:
  fjahr:
    tested ACK 83e1d92413
  ryanofsky:
    Code review ACK 83e1d92413. Nice test improvements!

Tree-SHA512: 92874b49a3bc0236500495f32dfcf683e1971ca3d4c51702c69ed4ce7dfce21273754f02f93d1243d73793701d9fdf49e14b149477cd249cbbd9e4e8d5bd49f8
2020-03-30 19:06:43 -04:00
MarcoFalke
faa9491870 ci: Use Focal for fuzzers 2020-03-30 18:57:59 -04:00
Russell Yanofsky
d06fedd1bc Add SER_READ and SER_WRITE for read/write-dependent statements
Extracted and extended by Pieter Wuille from a comment by Russ
Yanofsky (see
https://github.com/bitcoin/bitcoin/pull/18317#discussion_r398625457).
2020-03-30 15:36:16 -07:00
Russell Yanofsky
6f9a1e5ad0 Extend CustomUintFormatter to support enums
Extracted by Pieter Wuille from a comment by Russ Yanofsky, see
https://github.com/bitcoin/bitcoin/pull/18317#discussion_r398821936.
2020-03-30 15:36:16 -07:00
Pieter Wuille
769ee5fa00 Merge BigEndian functionality into CustomUintFormatter 2020-03-30 15:36:14 -07:00
MarcoFalke
fac3716b09 test: check that peer is connected when calling sync_* 2020-03-30 15:51:05 -04:00
MarcoFalke
7e1fc03b18 Merge #18334: test: Add basic test for BIP 37
fa15699969 test: Add basic test for BIP 37 (MarcoFalke)

Pull request description:

  This does not add full coverage, but should be a good start and can be extended in the future. Currently, none of the BIP 37 p2p code has test coverage.

ACKs for top commit:
  practicalswift:
    Code review ACK fa15699969 -- more testing coverage is better than less testing coverage

Tree-SHA512: d52e8be79240dffb769105c087ae0ae9305d599282546e4ca7379c4c7add2dbcd668265b46670aa07c357638044cf0f61a6fab7dba8971dd0f80c8f99768686e
2020-03-30 15:28:02 -04:00
MarcoFalke
fa3cc0bfc4 test: Remove unsafe BOOST_TEST_MESSAGE 2020-03-30 15:18:42 -04:00
fanquake
6a11d9e330 Merge #18433: serialization: prevent int overflow for big Coin::nHeight
e980214bc4 serialization: prevent int overflow for big Coin::nHeight (pierrenn)

Pull request description:

  This is an attempt to fix fuzzer issues 1,2,8 reported by practicalswift here : https://github.com/bitcoin/bitcoin/issues/18046

  The fuzzer harness doesn't prevent deserialization of unrealistic high values for `Coin::nHeight`.  In the [provided examples](https://github.com/bitcoin/bitcoin/issues/18046), we have :
  - `blockundo_deserialize` : the varint `0x8DD88DD700` is deserialized as `3944983552` in `Coin::nHeight` (`TxInOutFormatter::Unser`)
  - `coins_deserialize` : the varint `0x8DD5D5EC40` is deserialized as `3939874496` similarly
  - `txundo_deserialize`: the varint `0x8DCD828F01` is deserialized as `3921725441` in `Coin::nHeight` (`Coin::Unserialize`)

  Since `Coin::nHeight` is 31 bit long, multiplying a large value by 2 triggers the fuzzer.

  AFAIK those values are unrealistic (~70k years for the smallest..). I've looked a bit a reducing the range of values the fuzzer can deserialize, but this seems to be too much code change for not much.

  Hence this PR chooses to static cast `nHeight` when re-serializing; it seems to be the less intrusive/safest way to prevent the fuzzer output.

  Another more "upstream" approach would be to limit `Coin::nHeight` values to something more realistic, e.g. `0xFFFFFFF` (~5k years) :

  de3a30bab2/src/undo.h (L39) and de3a30bab2/src/coins.h (L71)

  Thanks !

  NB: i was also not sure about the component/area to prefix the PR/commit with.. ?

ACKs for top commit:
  practicalswift:
    ACK e980214bc4 -- patch looks correct
  promag:
    ACK e980214bc4.
  sipa:
    utACK e980214bc4
  MarcoFalke:
    re-ACK e980214bc4 🎑
  ryanofsky:
    Code review ACK e980214bc4. Just removed ternary ? 1 : 0 and replaced / 2 with >> 1 since last review

Tree-SHA512: 905fc9e5e52a6857abee4a1c863751767835965804bb8c39474f27a120f65399ff4ba7a49ef1da0ba565379f8c12095bd384b6c492cf06776f01b2db68d522b8
2020-03-30 22:16:12 +08:00
MarcoFalke
fa6e01f2a1 doc: block-relay-only is not blocksonly
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2020-03-30 09:09:12 -04:00
MarcoFalke
fa369651c5 net: Add missing cs_vNodes lock 2020-03-29 11:45:46 -04:00
MarcoFalke
5f9cd62f33 Merge #18455: tests: Add fuzzing harness for functions/classes in flatfile.h, merkleblock.h, random.h, serialize.h and span.h
11a520f679 tests: Add fuzzing harness for functions/classes in random.h (practicalswift)
64d277bbbc tests: Add fuzzing harness for LimitedString (serialize.h) (practicalswift)
f205cf7fef tests: Add fuzzing harness for functions/classes in span.h (practicalswift)
9718f38f54 tests: Add fuzzing harness for functions/classes in merkleblock.h (practicalswift)
a16ea051f9 tests: Add fuzzing harness for functions/classes in flatfile.h (practicalswift)

Pull request description:

  * Add fuzzing harness for functions/classes in `flatfile.h`
  * Add fuzzing harness for functions/classes in `merkleblock.h`
  * Add fuzzing harness for functions/classes in `span.h`
  * Add fuzzing harness for `LimitedString` (`serialize.h`)
  * Add fuzzing harness for functions/classes in `random.h`

Top commit has no ACKs.

Tree-SHA512: 6f7e0f946f1062d51216990cde9672b4e896335152548ace3d8711e4969c3e3c8566d01d915b72adcda5c1caa9c2e34da6b7473b55a229f5b77239d3b0ba4b67
2020-03-29 10:32:05 -04:00
practicalswift
11a520f679 tests: Add fuzzing harness for functions/classes in random.h 2020-03-29 13:17:04 +00:00
practicalswift
64d277bbbc tests: Add fuzzing harness for LimitedString (serialize.h) 2020-03-29 13:17:04 +00:00
practicalswift
f205cf7fef tests: Add fuzzing harness for functions/classes in span.h 2020-03-29 13:17:04 +00:00
practicalswift
9718f38f54 tests: Add fuzzing harness for functions/classes in merkleblock.h 2020-03-29 13:17:04 +00:00
practicalswift
a16ea051f9 tests: Add fuzzing harness for functions/classes in flatfile.h 2020-03-29 13:17:04 +00:00
MarcoFalke
6cfb3dbbdb Merge #18391: doc: Update init and reduce-traffic docs for -blocksonly
621e86ee8d Update -blocksonly documentation (glowang)

Pull request description:

  When -blocksonly is set to 1, it interacts with the -walletbroadcast
  parameter and sets it to 0.

  This behavior is not captured by the current documentation, which
  claims that -blocksonly does not impact any wallet transactions at
  all.

  Fixes #17294

ACKs for top commit:
  MarcoFalke:
    ACK 621e86ee8d

Tree-SHA512: f47bfb40a196c23e62505e1d4f79094011ac7c21fc9b920fad60cdadb5c4f48e993be1f015e26e568ce329967c24848fd7b665a6cffd3881f4cfcd2fd0081ed8
2020-03-29 08:15:55 -04:00
glowang
621e86ee8d Update -blocksonly documentation
When -blocksonly is set to 1, it interacts with the -walletbroadcast
parameter and sets it to 0 if it has not been set already.This behavior
is not captured by the current documentation, which claims that -blocksonly
does not impact any wallet transactions.

Update the max number of outgoing peers from 8 to 10, due to the
addition of two -blocksonly peers.
2020-03-29 05:12:30 -07:00
MarcoFalke
27a82d347e Merge #18459: rpc: remove unused getbalances() code
6e0d82c55b rpc: remove unused getbalances() code (Jon Atack)

Pull request description:

  This line from 999931cf8f appears to be extraneous and replaced 2 lines after by `UniValue balances{UniValue::VOBJ};`.

ACKs for top commit:
  Empact:
    ACK 6e0d82c55b
  hebasto:
    ACK 6e0d82c55b, the `obj` local variable is not used until the end of the scope.

Tree-SHA512: a220ca9cda091e78144d9b7fbe4bf90e8338d6e8c8dc7bea27a8e62f3a8ac1d983ad12a48a0a3366b2d8b9586878dfc69c1ec34bf846b34c91e42cda48a59850
2020-03-28 17:09:52 -04:00
Jon Atack
6e0d82c55b rpc: remove unused getbalances() code 2020-03-28 20:25:53 +01:00
MarcoFalke
6b4f182806 Merge #18444: RPC: Remove final comma for last entry of fixed-size arrays/objects in RPCResult
c34164896c Bugfix: RPC: Remove final comma for last entry of fixed-size Arrays and Objects in RPCResult (Luke Dashjr)

Pull request description:

  JSON doesn't allow a trailing comma in arrays

Top commit has no ACKs.

Tree-SHA512: 761502a05f447afc09c120f13bf23abd2aee83a7f5e5dadaf54c7e1c0c1280d83ee041ca6ca45998fb561e41b32d01067ec52a187c3bcc9d53303ea813bc212c
2020-03-28 15:13:13 -04:00
Wladimir J. van der Laan
1668c80bdc Merge #18449: util: Remove unused itostr
faaf1cb5b9 util: Replace i64tostr with ToString (MarcoFalke)
fac96fff62 util: Remove unused itostr (MarcoFalke)

Pull request description:

  Currently unused, but if someone really needed to use a helper with this functionality in the future, they could use `ToString`.

ACKs for top commit:
  laanwj:
    ACK faaf1cb5b9
  promag:
    Code review ACK faaf1cb5b9.

Tree-SHA512: 42180c03f51d677f7b69da23c7868bdd88944335fad0752fcc307f2c3e3c69f1cc1b316ac0875bcefb9a69c5d55200d7cf66843ea4c0f0f26baf7a054b96c1bb
2020-03-28 19:26:45 +01:00
Luke Dashjr
c34164896c Bugfix: RPC: Remove final comma for last entry of fixed-size Arrays and Objects in RPCResult
JSON doesn't allow a trailing comma in Arrays/Objects
2020-03-28 17:32:28 +00:00
Wladimir J. van der Laan
1277ca401a Merge #18415: scripts: add MACHO tests to test-security-check.py
7142d50ac3 scripts: rename test_64bit_PE to test_PE (fanquake)
edaca2dd12 scripts: add MACHO NX check to security-check.py (fanquake)
1a4e9f32ef scripts: add MACHO tests to test-security-check.py (fanquake)

Pull request description:

  Adds tests for the MACHO checks in security-check.py:
  ac579ada7e/contrib/devtools/security-check.py (L212-L214)

  I'm planning on following up with more checks in security-check.py, and corresponding tests in test-security-check.py.

  Note that you'll probably have to be on macOS to run them. You can run just this suite with `python3 test-security-check.py TestSecurityChecks.test_MACHO`.

ACKs for top commit:
  laanwj:
    ACK 7142d50ac3

Tree-SHA512: ace3ca9f6df5d4fedd5988938fb7dc7563ec7dc587aa275f780b5f51e9b8d7d6f7768e0a1e05ce438510a07b8640aba92c76847b30c2990f46c66b78a0acf960
2020-03-28 11:49:04 +01:00
pierrenn
e980214bc4 serialization: prevent int overflow for big Coin::nHeight 2020-03-28 08:38:07 +09:00
MarcoFalke
fa47a0b003 net: Make addr relay mockable 2020-03-27 17:58:51 -04:00
Wladimir J. van der Laan
f2c416bcf5 Merge #16995: Fix gcc 9 warnings
ff9c671b11 refactor: Work around GCC 9 `-Wredundant-move` warning (Russell Yanofsky)
b837b334db net: Fail instead of truncate command name in CMessageHeader (Wladimir J. van der Laan)

Pull request description:

  Fixes all 3 from #16992 (see commits)

  - net: Fail instead of truncate command name in CMessageHeader
  - refactor: Use std::move workaround for unique_ptr upcast only when necessary

ACKs for top commit:
  practicalswift:
    ACK ff9c671b11 -- patch looks correct
  sipa:
    utACK ff9c671b11
  ryanofsky:
    Code review ACK ff9c671b11. Looks good and seems to pass travis, modulo a timeout on one build
  hebasto:
    ACK ff9c671b11, tested on Fedora 31:

Tree-SHA512: 52d8c13aaf0d56f9bc546a98d7f853eae21f7e325b202fdeb2286b19a9a0ee308634c644b039f60ad8043421e382381cbf1bce58d9f807547f928621c7d245d0
2020-03-27 20:04:46 +01:00
Andrew Toth
a1d5b12ec0 Merge getreceivedby tally into GetReceived function 2020-03-27 14:26:00 -04:00
Andrew Toth
7524b6479c Add tests for generateblock 2020-03-27 12:09:31 -04:00
João Barbosa
41b0baf43c gui: Handle WalletModel::unload asynchronous
This change prevents deleting a WalletModel instance while it's
being used.
2020-03-27 15:17:35 +00:00
Russell Yanofsky
ab31b9d6fe Fix wallet unload race condition
Currently it's possible for ReleaseWallet to delete the CWallet pointer while
it is processing BlockConnected, etc chain notifications.

To fix this, unregister from notifications earlier in UnloadWallet instead of
ReleaseWallet, and use a new RegisterSharedValidationInterface function to
prevent the CValidationInterface shared_ptr from being deleted until the last
notification is actually finished.
2020-03-27 15:17:35 +00:00
Andrew Toth
dcc8332543 Add generateblock rpc 2020-03-27 11:00:10 -04:00
MarcoFalke
faaf1cb5b9 util: Replace i64tostr with ToString 2020-03-27 10:14:08 -04:00
Sjors Provoost
764bfe4cba [psbt] add file size limit 2020-03-27 14:59:19 +01:00
Sjors Provoost
1cd8dc2556 [gui] load PSBT
co-authored-by: Glenn Willen <gwillen@nerdnet.org>
2020-03-27 14:59:19 +01:00
Sjors Provoost
f6895301f7 [gui] save PSBT to file
co-authored-by: Glenn Willen <gwillen@nerdnet.org>
2020-03-27 14:59:19 +01:00
Sjors Provoost
1d05a9d80b Move DEFAULT_MAX_RAW_TX_FEE_RATE to node/transaction.h
So it can be used in the GUI.
2020-03-27 14:58:39 +01:00
MarcoFalke
210b533a11 Merge #18447: test: Add coverage for script parse error in ParseScript
dcda81c471 test: add coverage for script parse error in ParseScript (pierrenn)

Pull request description:

  Follow up on this suggestion :  https://github.com/bitcoin/bitcoin/pull/18416#issuecomment-603966799

  This adds a test case to raise the `script parse error` in `ParseScript`.

ACKs for top commit:
  instagibbs:
    utACK dcda81c471

Tree-SHA512: ae0ef2c00f34cee818c83582f190d5f4043159e922862f2b442b7b895b8ff3ca421533699247c12c367be77813b5205830a771cd47a18e8932807ccace2d6a1c
2020-03-27 09:58:12 -04:00
Wladimir J. van der Laan
bdc2644b72 Merge #18107: build: Add cov_fuzz target
faf7d4fa86 build: Add cov_fuzz target (MarcoFalke)
fac71e364e build: link fuzz/test_runner.py for out-of-tree builds (MarcoFalke)
faf2c5aca0 build: Remove unused USE_COVERAGE (MarcoFalke)

Pull request description:

  Only libFuzzer is supported right now, so clang is required. Thus, this needs a workaround such as https://github.com/bitcoin/bitcoin/issues/12602#issuecomment-562788247

  Can be tested with:

  ```
  mkdir build && cd build
  ../configure --enable-fuzz --with-sanitizers=fuzzer --enable-lcov --enable-lcov-branch-coverage CC=clang CXX=clang++
  make $MAKEJOBS
  make cov_fuzz

ACKs for top commit:
  practicalswift:
    ACK faf7d4fa86

Tree-SHA512: 6828f8f81d95f6781713d0b09d7eba2ffdb50217e09ca839db61791a4ed70024859c7a0cb01d9eede79166d574dd57ece01f9d9fe2610d4a72a4ca4a4ce0b838
2020-03-27 14:29:48 +01:00
MarcoFalke
fac96fff62 util: Remove unused itostr 2020-03-27 08:59:06 -04:00
fanquake
7eed413e72 Merge #18398: rpc: fix broken RPCExamples for waitforblock(height)
ef35604c9c rpc: fix broken RPCExamples for waitforblock(height) (Sebastian Falbesoner)

Pull request description:

  This PR fixes several broken RPCExamples from the "blockchain" category:
  - `HelpExampleCli` for `waitforblock` (disturbing comma between arguments)
  - `HelpExampleCli` for `waitforblockheight` (disturbing comma between arguments)
  - `HelpExampleRpc` for `waitforblockheight` (disturbing quotation marks around integer argument)

  Note that the CLI example for `waitforblockheight` would also work with the first argument in quotation marks (in contrast to the RPC example), but I removed them as well as they are not needed.

  Outputs for the non-working examples in the master branch:
  ```
  $ ./bitcoin-cli waitforblock "0000000000079f8ef3d2c688c244eb7a4570b24c9ed7b4a8c619eb02596f8862", 1000
  error code: -8
  error message:
  blockhash must be of length 64 (not 65, for '0000000000079f8ef3d2c688c244eb7a4570b24c9ed7b4a8c619eb02596f8862,')
  ```

  ```
  $ ./bitcoin-cli waitforblockheight "100", 1000
  error: Error parsing JSON:100,
  ```

  ```
  $ curl --user __cookie__ --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "waitforblockheight", "params": ["100", 1000]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  Enter host password for user '__cookie__':
  {"result":null,"error":{"code":-1,"message":"JSON value is not an integer as expected"},"id":"curltest"}
  ```

  Outputs for the fixed examples in the PR branch:
  ```
  $ ./bitcoin-cli waitforblock "0000000000079f8ef3d2c688c244eb7a4570b24c9ed7b4a8c619eb02596f8862" 1000
  {
    "hash": "0000000000000000000910ae4d56120e0ddd55c0552e80ed12dba147abc68080",
    "height": 622416
  }
  ```

  ```
  $ ./bitcoin-cli waitforblockheight 100 1000
  {
    "hash": "0000000000000000000910ae4d56120e0ddd55c0552e80ed12dba147abc68080",
    "height": 622416
  }
  ```

  ```
  $ curl --user __cookie__ --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "waitforblockheight", "params": [100, 1000]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  Enter host password for user '__cookie__':
  {"result":{"hash":"0000000000000000000910ae4d56120e0ddd55c0552e80ed12dba147abc68080","height":622416},"error":null,"id":"curltest"}
  ```

ACKs for top commit:
  fanquake:
    ACK ef35604c9c

Tree-SHA512: b98c6681d1aa24b3ee3ef4ef450cb630082a9f8695af18f3b6d418e5b0b1e472b787ccf6397cd719b4d5fe0082ea5f1d0ca553c1cc56066ee2d288be34c601e3
2020-03-27 15:24:11 +08:00
Wladimir J. van der Laan
b53af72b82 Merge #18416: util: Limit decimal range of numbers ParseScript accepts
9ab14e4d21 Limit decimal range of numbers ParseScript accepts (pierrenn)

Pull request description:

  Following up on this suggestion : https://github.com/bitcoin/bitcoin/pull/18413#issuecomment-602966490, prevent the output of `atoi64` in the `core_read.cpp:ParseScript` helper to send to `CScriptNum::serialize` values wider than 32-bit.

  Since the `ParseScript` helper is only used by the tool defined in `bitcoin-tx.cpp`, this only prevents users to provide too much unrealistic values.

ACKs for top commit:
  laanwj:
    ACK 9ab14e4d21

Tree-SHA512: ee228269d19d04e8fee0aa7c0ae2bb0a2b437b8e574356e8d9b2279318242057d51fcf39a842aa3afe27408d0f2d5276df245d07a3f4828644a366f80587b666
2020-03-27 08:02:51 +01:00
pierrenn
9ab14e4d21 Limit decimal range of numbers ParseScript accepts 2020-03-27 15:51:05 +09:00
fanquake
54646167db Merge #18388: Make VerifyWitnessProgram use a Span stack
2b0fcff7f2 Make VerifyWitnessProgram use a Span stack (Pieter Wuille)

Pull request description:

  Here is a follow-up to #18002, again with the goal of simplifying (potential) BIP341 code.

  Instead of passing a begin and end iterator of the initial stack to `ExecuteWitnessScript`, they are turned into a `Span<const valtype>`, representing a span of `valtype`s in memory. This allows `VerifyWitnessProgram` to operate on that span directly, instead of juggling iterators around (which would be exacerbated by #17977 if trying to avoid copying the stack).

ACKs for top commit:
  ajtowns:
    ACK 2b0fcff7f2
  elichai:
    ReACK on the diff 2b0fcff7f2
  instagibbs:
    re-ACK 2b0fcff7f2
  theStack:
    re-ACK 2b0fcff7f2
  Empact:
    ACK 2b0fcff7f2
  jnewbery:
    utACK 2b0fcff7f2

Tree-SHA512: 38eb4ce17f1947674c1c274caa40feb6ea8266bd96134d9cf1bc41e6fbf1114d4dde6c7a9e26e1ca8f3d0155429ef0911cc8ec0c1037d8fe7d6ec7f9e7184e93
2020-03-27 14:49:50 +08:00
pierrenn
dcda81c471 test: add coverage for script parse error in ParseScript 2020-03-27 12:10:31 +09:00
MarcoFalke
e3154aacf4 Merge #18445: tests: Add fuzzing harnesses for functions/classes in chain.h and protocol.h
7834c3b9ec tests: Add fuzzing harness for functions/classes in chain.h (practicalswift)
d7930c4326 tests: Add fuzzing harness for functions/classes in protocol.h (practicalswift)

Pull request description:

  Add fuzzing harnesses for functions/classes in `chain.h` and `protocol.h`.

Top commit has no ACKs.

Tree-SHA512: ac2d66bc678ebba0ffbbc42e77806eaf3bb07413ff19219c7a83b171ccd4601e0aa8546ee7ffe8018ca4de12d080f79f693d184cc337c234cde641803279f00c
2020-03-26 20:37:48 -04:00
MarcoFalke
0dc6218c79 Merge #18270: util: Fail to parse whitespace-only strings in ParseMoney(...) (instead of parsing as zero)
100213c5c2 util: Fail to parse space-only strings in ParseMoney(...) (instead of parsing as zero) (practicalswift)

Pull request description:

  Fail to parse whitespace-only strings in `ParseMoney(...)` (instead of parsing as `0`).

  This is a follow-up to #18225 ("util: Fail to parse empty string in `ParseMoney`") which made `ParseMoney("")` fail instead of parsing as `0`.

  Context: https://github.com/bitcoin/bitcoin/pull/18225#issuecomment-592994765

  Current non-test call sites:

  ```
  $ git grep ParseMoney ":(exclude)src/test/"
  src/bitcoin-tx.cpp:    if (!ParseMoney(strValue, value))
  src/init.cpp:        if (!ParseMoney(gArgs.GetArg("-incrementalrelayfee", ""), n))
  src/init.cpp:        if (!ParseMoney(gArgs.GetArg("-minrelaytxfee", ""), n)) {
  src/init.cpp:        if (!ParseMoney(gArgs.GetArg("-blockmintxfee", ""), n))
  src/init.cpp:        if (!ParseMoney(gArgs.GetArg("-dustrelayfee", ""), n))
  src/miner.cpp:    if (gArgs.IsArgSet("-blockmintxfee") && ParseMoney(gArgs.GetArg("-blockmintxfee", ""), n)) {
  src/util/moneystr.cpp:bool ParseMoney(const std::string& str, CAmount& nRet)
  src/util/moneystr.h:NODISCARD bool ParseMoney(const std::string& str, CAmount& nRet);
  src/wallet/wallet.cpp:        if (!ParseMoney(gArgs.GetArg("-mintxfee", ""), n) || 0 == n) {
  src/wallet/wallet.cpp:        if (!ParseMoney(gArgs.GetArg("-fallbackfee", ""), nFeePerK)) {
  src/wallet/wallet.cpp:        if (!ParseMoney(gArgs.GetArg("-discardfee", ""), nFeePerK)) {
  src/wallet/wallet.cpp:        if (!ParseMoney(gArgs.GetArg("-paytxfee", ""), nFeePerK)) {
  src/wallet/wallet.cpp:        if (!ParseMoney(gArgs.GetArg("-maxtxfee", ""), nMaxFee)) {
  ```

ACKs for top commit:
  Empact:
    ACK 100213c5c2
  sipa:
    ACK 100213c5c2
  theStack:
    ACK 100213c5c2

Tree-SHA512: cadfb1ac8276cf54736c3444705f2650e7a08023673aedc729fabe751ae80f6c490fc0945ee38dbfd02c95e4d9853d1e4c84f5d3c310f44eaf3585afec8a4c22
2020-03-26 20:25:55 -04:00
practicalswift
7834c3b9ec tests: Add fuzzing harness for functions/classes in chain.h 2020-03-26 21:21:34 +00:00
practicalswift
d7930c4326 tests: Add fuzzing harness for functions/classes in protocol.h 2020-03-26 21:21:34 +00:00
Vasil Dimov
f85203097f lockedpool: avoid sensitive data in core files (FreeBSD)
This is a followup to
23991ee53 / https://github.com/bitcoin/bitcoin/pull/15600
to also use madvise(2) on FreeBSD to avoid sensitive data allocated
with secure_allocator ending up in core files in addition to preventing
it from going to the swap.
2020-03-26 20:43:17 +01:00
MarcoFalke
7f9dedb22d Merge #18441: ci: Remove misplaced comments from folded block scalar
e41e46cee0 ci: Remove misplaced comments from folded block scalar (Hennadii Stepanov)

Pull request description:

  On master (f3a91ab0ed) the [build log for ARM](https://travis-ci.org/github/bitcoin/bitcoin/jobs/667247758) contains:
  ```
  $ export FILE_ENV="./ci/test/00_setup_env_arm.sh"
  $ export QEMU_USER_CMD=""
  $ export Can=
  $ export run=
  $ export the=
  $ export tests=
  $ export natively=
  $ export without=
  $ export qemu=

  Setting up build cache
  ...
  ```
  and
  ![Screenshot from 2020-03-26 18-58-50](https://user-images.githubusercontent.com/32963518/77674223-e28d2d00-6f93-11ea-82a1-7f805ccb678d.png)

  With this PR:
  ```
  $ export FILE_ENV="./ci/test/00_setup_env_arm.sh"
  $ export QEMU_USER_CMD=""

  Setting up build cache
  ...
  ```

  and
  ![Screenshot from 2020-03-26 19-01-56](https://user-images.githubusercontent.com/32963518/77674456-4d3e6880-6f94-11ea-9c83-546c81f1cdf7.png)

  ---

  Also Travis [build config validation](https://docs.travis-ci.com/user/build-config-validation) added:
  ![Screenshot from 2020-03-26 19-04-19](https://user-images.githubusercontent.com/32963518/77674686-9ee6f300-6f94-11ea-900f-fdef8a673113.png)

Top commit has no ACKs.

Tree-SHA512: a3ea05d543a4d46c65fffd7c67df9bbd4ca8d426fa215c5ce26653de34145edd419a54884da0416608a2c90bf5b421cab3ec32290c8916f10399354b44ddeb99
2020-03-26 14:04:41 -04:00
Hennadii Stepanov
e41e46cee0 ci: Remove misplaced comments from folded block scalar
Also Travis build config validation added.
2020-03-26 19:55:29 +02:00
Wladimir J. van der Laan
694f4cbd78 Merge #18312: wallet: remove deprecated fee bumping by totalFee
c3857c5fcb wallet: remove CreateTotalBumpTransaction() (Jon Atack)
4a0b27bb01 wallet: remove totalfee from createBumpTransaction() (Jon Atack)
e347cfa9a7 rpc: remove deprecated totalFee arg from RPC bumpfee (Jon Atack)
bd05f96d79 test: delete wallet_bumpfee_totalfee_deprecation.py (Jon Atack)
a6d1ab8caa test: update bumpfee testing from totalFee to fee_rate (Jon Atack)

Pull request description:

  Since 0.19, fee-bumping using `totalFee` was deprecated in #15996 and replaced by `fee_rate` in #16727. This changeset removes it.

ACKs for top commit:
  laanwj:
    ACK c3857c5fcb

Tree-SHA512: c1bb15d664baf4d2dea06981f36384af02057d125c51fcbc8640b9d5563532187c7b84aa952f7b575255a88ce383ed4d7495bec920a47b05b6fc0d432dce1f00
2020-03-26 18:34:49 +01:00
Jon Atack
c3857c5fcb wallet: remove CreateTotalBumpTransaction() 2020-03-26 17:54:23 +01:00
Jon Atack
4a0b27bb01 wallet: remove totalfee from createBumpTransaction() 2020-03-26 17:54:21 +01:00
Jon Atack
e347cfa9a7 rpc: remove deprecated totalFee arg from RPC bumpfee 2020-03-26 17:54:18 +01:00
MarcoFalke
94d3063c93 Merge #18412: script: fix SCRIPT_ERR_SIG_PUSHONLY error string
41ff4992e5 script: fix SCRIPT_ERR_SIG_PUSHONLY error string (Sebastian Falbesoner)

Pull request description:

  Fixes #18411, changing the error message from `"Only non-push operators allowed in signatures"` to `"Only push operators allowed in signatures"`.

ACKs for top commit:
  laanwj:
    ACK 41ff4992e5

Tree-SHA512: 3b75d83e2198d638d599ef6a4a8da986f0158600fe3f89f55b3759554588157acf2b0cba3f6a907164617264e7aee727d6d460b510c8b37ca7728aa79e11ad80
2020-03-26 12:34:27 -04:00
Wladimir J. van der Laan
23991ee53a Merge #15600: lockedpool: When possible, use madvise to avoid including sensitive information in core dumps
d831831822 lockedpool: When possible, use madvise to avoid including sensitive information in core dumps (Luke Dashjr)

Pull request description:

  If we're mlocking something, it's because it's sensitive information. Therefore, don't include it in core dump files, ~~and unmap it from forked processes~~.

  The return value is not checked because the madvise calls might fail on older kernels as a rule (unsure).

ACKs for top commit:
  practicalswift:
    Code review ACK d831831822 -- patch looks correct
  laanwj:
    ACK d831831822
  jonatack:
    ACK d831831822
  vasild:
    ACK d831831822

Tree-SHA512: 9a6c1fef126a4bbee0698bfed5a01233460fbcc86380d984e80dfbdfbed3744fef74527a8e3439ea226167992cff9d3ffa8f2d4dbd5ae96ebe0c12f3eee0eb9e
2020-03-26 16:56:03 +01:00
MarcoFalke
f3a91ab0ed Merge #18438: ci: Use Homebrew addon on native macOS
25c8b73656 ci: Use Homebrew addon on native macOS (Hennadii Stepanov)
596c627a1e ci: Fix brew in Travis (Hennadii Stepanov)

Pull request description:

  Recently almost every macOS image update on Travis breaks our builds:
  - #17848
  - #18436

  This PR:
  - fixes the error caused by the recent [update](https://changelog.travis-ci.com/xcode-11-3-1-xcode-11-2-1-xcode-11-1-and-xcode11-images-updated-142286) from 10.14.4 (18E226) to 10.14.6 (18G3020) on March 25
  - leverages [Homebrew addon](https://config.travis-ci.com/ref/job/addons/homebrew) to install packages

  Homebrew is not told to install `automake` and `pkg-config` packages, as the [docs](https://docs.travis-ci.com/user/reference/osx/#compilers-and-build-toolchain) states that they are pre-installed:
  >    - automake 1.16.1
  >    - pkg-config 0.29.2

Top commit has no ACKs.

Tree-SHA512: 1a70c06468fbe162503081b03dcf54614d67abf8ff0ce07d118b5bb50bbb92c182a76f769bea586c691aa82b9281a29cdef88091acc16895817a2e7cddafec6e
2020-03-26 09:10:31 -04:00
Hennadii Stepanov
25c8b73656 ci: Use Homebrew addon on native macOS
Also the macOS image has been updated.
2020-03-26 14:57:14 +02:00
Hennadii Stepanov
596c627a1e ci: Fix brew in Travis 2020-03-26 14:56:43 +02:00
fanquake
e99ca783cd Merge #18429: build: remove double LIBBITCOIN_SERVER from bench-Makefile
1f97b69ba2 build: remove double LIBBITCOIN_SERVER from bench-Makefile (Harris)

Pull request description:

  This PR removes the redundant **LIBBITCOIN_SERVER** linking from bench's Makefile.
  This PR is similar to https://github.com/bitcoin/bitcoin/pull/17910

  Originally, this PR was part of https://github.com/bitcoin/bitcoin/pull/18377, which later got replaced by a better one https://github.com/bitcoin/bitcoin/pull/18397 written by **hebasto**.

ACKs for top commit:
  Empact:
    Code Review ACK 1f97b69ba2
  theStack:
    ACK 1f97b69ba2
  hebasto:
    ACK 1f97b69ba2

Tree-SHA512: e43035262361d4458a7dcfc920445540f19301387814cde1be0539c936fc20da0dcbe49e5ea25385e6d36d9639515b7a4171228223da568d93427e9c32810945
2020-03-26 17:20:28 +08:00
fanquake
d8ce27ff9f Merge #18397: build: Fix libevent linking for bench_bitcoin binary
cd04286825 build: Fix typo in EVENT_CFLAGS variable (Hennadii Stepanov)
f709ad0c90 build: Fix libevent linking for bench_bitcoin binary (Hennadii Stepanov)

Pull request description:

  This change fixes `libevent` linking error for the `bench_bitcoin` binary.

  This PR is an alternative to #18377.
  Fix #18373.

  Also fixed a typo in `EVENT_CFLAGS` variable noted by **brakmic**.

ACKs for top commit:
  fanquake:
    ACK cd04286825

Tree-SHA512: a62f7457e86b11d3a55d603ea5d83f3a413792e2f28a0c72300e54d12591bd6f0acc1d76a4bd4b591e0223bc6d530e7a4b9a8b939fe2fdbf2dddfda5b1b537be
2020-03-26 16:22:24 +08:00
fanquake
7142d50ac3 scripts: rename test_64bit_PE to test_PE 2020-03-26 11:39:34 +08:00
fanquake
edaca2dd12 scripts: add MACHO NX check to security-check.py 2020-03-26 11:39:34 +08:00
fanquake
1a4e9f32ef scripts: add MACHO tests to test-security-check.py 2020-03-26 11:39:34 +08:00
Wladimir J. van der Laan
2e97d80017 Merge #18134: Replace std::to_string with locale-independent alternative
d056df033a Replace std::to_string with locale-independent alternative (Ben Woosley)

Pull request description:

  Addresses #17866 following practicalswift's suggestion:
  https://github.com/bitcoin/bitcoin/issues/17866#issuecomment-584287299

  ~Used ::ToString to avoid aliasing issues. Left uses in QT and test.~

ACKs for top commit:
  practicalswift:
    ACK d056df033a
  laanwj:
    ACK d056df033a

Tree-SHA512: 9e6966a9cdd14f4a1a40d9f0fa7c402aed22b2f1ad8681708e22b050d51a91c5d62220a9ec4c425be2d57acf5c964fca87a5e981b5cbff048bc3b6720dae92b7
2020-03-25 20:11:47 +01:00
MarcoFalke
7c94225026 Merge #18430: ci: Only clone bitcoin-core/qa-assets when fuzzing
fae1e99289 ci: Only clone bitcoin-core/qa-assets when fuzzing (MarcoFalke)

Pull request description:

  Currently the only content of that repo are some seeds, so we can speed up some ci builds

ACKs for top commit:
  laanwj:
    ACK fae1e99289 (provided this passes travis)

Tree-SHA512: ed813738e7f24bb56a2f12aa3b398e414eb4f0ba98379836a33ff3e5602cbf42a28e89aad10e346468191ecddc03e60d5b236097112e27c07cb1c2293533ea58
2020-03-25 13:36:50 -04:00
Wladimir J. van der Laan
60a39a96fc Merge #18425: releases: Update with new Windows code signing certificate
3e0df92bf2 Update with new Windows code signing certificate (Andrew Chow)

Pull request description:

  The current Windows code signing certificate is about expire (on March 26th 2020). As I have volunteered to take over the Windows code signing duties, I've purchased a new Windows code signing certificate with the same CA and under the same organization (Bitcoin Core Code Signing Association).

  A signature by the old certificate over the new certificate has been provided to me. This signature can be verified using
  ```
  openssl cms -verify -inform pem -purpose any -content path/to/new/win-codesign.cert -CAfile path/to/old/win-codesign.cert -certfile path/to/old/win-codesign.cert
  ```
  The verification should succeed and the new certificate will be printed out. This can be compared to the contents of `win-codesign.cert`.

  ```
  -----BEGIN PKCS7-----
  MIIC3AYJKoZIhvcNAQcCoIICzTCCAskCAQExDzANBglghkgBZQMEAgEFADALBgkq
  hkiG9w0BBwExggKkMIICoAIBATCBkTB8MQswCQYDVQQGEwJHQjEbMBkGA1UECBMS
  R3JlYXRlciBNYW5jaGVzdGVyMRAwDgYDVQQHEwdTYWxmb3JkMRgwFgYDVQQKEw9T
  ZWN0aWdvIExpbWl0ZWQxJDAiBgNVBAMTG1NlY3RpZ28gUlNBIENvZGUgU2lnbmlu
  ZyBDQQIRALWcUnSOxv9FQW3xdaMDO6swDQYJYIZIAWUDBAIBBQCggeQwGAYJKoZI
  hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMjAwMzI0MjA0ODM3
  WjAvBgkqhkiG9w0BCQQxIgQgtLkmnuSQyczDlJSnJeqbi61p3iJ/rpFABrY8JWBO
  o74weQYJKoZIhvcNAQkPMWwwajALBglghkgBZQMEASowCwYJYIZIAWUDBAEWMAsG
  CWCGSAFlAwQBAjAKBggqhkiG9w0DBzAOBggqhkiG9w0DAgICAIAwDQYIKoZIhvcN
  AwICAUAwBwYFKw4DAgcwDQYIKoZIhvcNAwICASgwDQYJKoZIhvcNAQEBBQAEggEA
  XaCl3Q8HwI9VpLCb9OY9eQh0QOPyl1KWEc3TP3UvwZwR4/gXkfPOKKf19UnS8eRB
  48SgUKRMYWoDYfSVUJRMda9BLkbJbQlHG3LFXhSY2alajpPXEHcMto/XPhVAmqzL
  w6aSNY0Gaorow696JHpetpKqAAlL1r2GjeaPYi2aZyIAifuhay/qwA+ig0SqzGOw
  UdgFZWMyS5yanq8/WlLCCql6kKOzT4tEqUaleD7R1q8BTcG2+fmhWR8WwJLpIV6y
  7GAqt0Cocu8sYpTNBNk8iKHxzZ2hMZKJpH9lHZuiJ/9vSercrvDy2R4/MG+KnBWb
  OyiFAt2mC51+63RhLOMJfg==
  -----END PKCS7-----
  ```

ACKs for top commit:
  laanwj:
    ACK 3e0df92bf2
  theuni:
    ACK 3e0df92bf2.

Tree-SHA512: 4210f4db1e805ab11231fbae49ea197257c6f7e44f1f6219685b63831704984d824ac2f9e0a3b1bd2655953af72636a474f077cb859fb35852551f5a9f8fbde3
2020-03-25 17:04:53 +01:00
MarcoFalke
fae1e99289 ci: Only clone bitcoin-core/qa-assets when fuzzing 2020-03-25 11:10:18 -04:00
Wladimir J. van der Laan
3e50fdbe4e Merge #18395: scripts: add PE dylib checking to symbol-check.py
1a0993ae35 scripts: add PE dylib checking to symbol-check.py (fanquake)

Pull request description:

  Uses `objdump -x` and looks for `DLL Name:` lines. i.e:
  ```bash
  objdump -x src/qt/bitcoin-qt.exe | grep "DLL Name:"
  	DLL Name: ADVAPI32.dll
  	DLL Name: dwmapi.dll
  	DLL Name: GDI32.dll
  	DLL Name: IMM32.dll
  	DLL Name: IPHLPAPI.DLL
  	DLL Name: KERNEL32.dll
  	DLL Name: msvcrt.dll
  	DLL Name: ole32.dll
  	DLL Name: OLEAUT32.dll
  	DLL Name: SHELL32.dll
  	DLL Name: SHLWAPI.dll
  	DLL Name: USER32.dll
  	DLL Name: UxTheme.dll
  	DLL Name: VERSION.dll
  	DLL Name: WINMM.dll
  	DLL Name: WS2_32.dll
  ```

ACKs for top commit:
  dongcarl:
    Concept ACK 1a0993ae35
  hebasto:
    ACK 1a0993ae35, tested on Linux Mint 19.3:

Tree-SHA512: 0099a50e2c616d5239a15cafa9a7c483e9c40244af41549e4738be0f5360f27a2afb956eb50b47cf446b242f4cfc6dc9d111306a056fb83789eefbd71eddabd2
2020-03-25 15:35:36 +01:00
Wladimir J. van der Laan
244e88e6b5 Merge #18402: gui: display mapped AS in peers info window
76db4b260e gui: avoid QT Designer/Form Editor re-formatting (Jon Atack)
aae26053f9 gui: display Mapped AS in peers info window (Jon Atack)

Pull request description:

  Continuing the asmap integration of #16702 which added `mapped_as` to the rpc getpeerinfo output, this adds the mapped AS to the Peers detail window in the GUI wallet.

  `$ src/qt/bitcoin-qt -asmap=<path-to-asmap-file>` (asmap on)

  ![Screenshot from 2020-03-22 12-29-56](https://user-images.githubusercontent.com/2415484/77248754-c0ae4600-6c33-11ea-9d27-a06560c180c0.jpg)

  `$ src/qt/bitcoin-qt` (asmap off)

  ![Screenshot from 2020-03-22 12-32-46](https://user-images.githubusercontent.com/2415484/77248749-bdb35580-6c33-11ea-925c-6e19ecc083ab.jpg)

  Added a tooltip and a couple of minor fixups.

ACKs for top commit:
  laanwj:
    ACK 76db4b260e

Tree-SHA512: 5f44c05c247bfabc9c161884d3af47c50a571cd02777b320ce389e61efa47706adbf0ea5e6644ae40423cb579d8bd0bb3c84fc6b618293a7add8e4327f07f63f
2020-03-25 15:11:10 +01:00
Wladimir J. van der Laan
baa72cd9a2 Merge #18331: build: Use git archive as source tarball
e4d366788b build: Drop needless EXTRA_DIST content (Hennadii Stepanov)
6c4da59f5b build: Drop SOURCEDIST reordering (Hennadii Stepanov)
5e6b8b3912 build: Use git archive as source tarball (Hennadii Stepanov)

Pull request description:

  This PR:
  - is an alternative to #17104
  - closes #16734
  - closes #6753

  The idea is clear described by some developers:
  - [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540691850):
  > This whole concept of explicitly listing each and every file manually (or with a fragile wildcard) is an obvious sisyphean task. I'd say all we need to do is run git archive and be done with it forever, see #16734, #6753, #11530 ...

  - [laanwj](https://github.com/bitcoin/bitcoin/pull/17097#issuecomment-540706025):
  > I agree, I've never been a fan of it. I don't think we have any files in the git repository we don't want to ship in the source tarball.

  ---

  The suggested changes have a downside which is pointed by [**luke-jr**](https://github.com/bitcoin/bitcoin/pull/17104#issuecomment-540828045):
  > ... but the distfile needs to include autogen-generated files.

  This means that a user is not able to run `./configure && make` right away. One must run `./autogen.sh` at first.

  Here are opinions about mandatory use of `./autogen.sh`:
  - [ryanofsky](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-534139356):
  > It's probably ok to require autogen. I think historically configure scripts were supposed to work on obscure unix systems that would just have a generic shell + make tool + c compiler, and not necessarily need gnu packages like m4 which are needed for autogen.

  - [laanwj](https://github.com/bitcoin/bitcoin/issues/16734#issuecomment-540729483):
  > I also think it's fine to require autogen. What is one dependency more, if you're building from source.

  ---

  ~Also this PR provides Windows users with ZIP archives of the sources. Additionally the commit ID is stored in these ZIP files as a file comment:~

  ---

  Note for reviewers: please verify is `git archive` output deterministic?

ACKs for top commit:
  MarcoFalke:
    re-ACK e4d366788b, only change is adding two dots in a the path 🛳
  laanwj:
    ACK e4d366788b

Tree-SHA512: d1153d3ca4a580696019b92be3555ab004d197d9a2146aacff9d3150eb7093b7d40eebd6eea12d861d93ff62d62b68706e04e64dbe5ea796ff6757486e462193
2020-03-25 14:48:44 +01:00
Harris
1f97b69ba2 build: remove double LIBBITCOIN_SERVER from bench-Makefile 2020-03-25 13:33:44 +01:00
MarcoFalke
3f5107d008 Merge #18423: tests: Add fuzzing harness for classes/functions in blockfilter.h. Add integer {de,}serialization fuzzing.
102f3267e9 tests: Add fuzzing harness for classes/functions in blockfilter.h (practicalswift)
87d24e67bb tests: Add integer serialization/deserialization fuzzing harness (practicalswift)

Pull request description:

  Add fuzzing harness for classes/functions in `blockfilter.h`.

  Add integer serialization/deserialization fuzzing harness.

Top commit has no ACKs.

Tree-SHA512: 729e6bc1adf5873a64ca334a0ddc279c6cddf208923ca37cec712e9c73d0216a641045e10084925b055230f9d31fbd85ba61e59e4da3f865a544c5f8afc05e05
2020-03-25 07:35:03 -04:00
MarcoFalke
fa15699969 test: Add basic test for BIP 37 2020-03-25 06:45:43 -04:00
Sebastian Falbesoner
332f373a9d [scripts] previous_release: improve failed download error message
before:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
------------------------------------------------------------

now:
------------------------------------------------------------
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
------------------------------------------------------------
2020-03-25 09:06:48 +01:00
fanquake
5b4a9f4bdf Merge #18342: doc: Add fuzzing quickstart guides for libFuzzer and afl-fuzz
33dd764984 doc: Add fuzzing quickstart guides for libFuzzer and afl-fuzz. Simplify instructions. (practicalswift)

Pull request description:

  Add fuzzing quickstart guide to make it trivial to start fuzzing Bitcoin Core.

  Fuzzing is fun and having more people contributing coverage-increasing inputs to https://github.com/bitcoin-core/qa-assets would be awesome :)

ACKs for top commit:
  MarcoFalke:
    ACK 33dd764984
  fanquake:
    ACK 33dd764984 - ran through the quick start and process message instructions. macOS users might see issues with afl-fuzz.

Tree-SHA512: f3ca972ce6ed0df8bb8177bdbb1e16d8a235941ffe4fa7b95ce9520b6454694ee26d2c545eac0b8b81856a77e26befda0922a9121a445dd936a0e9f9dd034160
2020-03-25 15:40:54 +08:00
Pieter Wuille
14e8cf974a [consensus] MOVEONLY: Move single-sig checking EvalScript code to EvalChecksig
This is in preparation for adding different signature verification rules,
specifically tapscript (BIP 342), which interprets opcode 0xac and 0xad
as Schnorr signature verifications.
2020-03-24 21:48:27 -04:00
Hennadii Stepanov
8e08d00598 qt: Use parent-child relation to manage lifetime of OptionsModel object
Both BitcoinApplication and OptionsModel classes are derived from the
QObject class, therefore a parent-child relation could be established to
manage the lifetime of an OptionsModel object.
This commit does not change behavior.
2020-03-24 22:53:07 +02:00
Sjors Provoost
86e22d23bb [util] GetFileSize 2020-03-24 20:29:49 +01:00
Sjors Provoost
6ab3aad9a5 [gui] send dialog: split on_sendButton_clicked
This commit does not change behavior.
2020-03-24 20:29:49 +01:00
practicalswift
102f3267e9 tests: Add fuzzing harness for classes/functions in blockfilter.h 2020-03-24 17:01:54 +00:00
practicalswift
87d24e67bb tests: Add integer serialization/deserialization fuzzing harness 2020-03-24 16:48:28 +00:00
Andrew Chow
3e0df92bf2 Update with new Windows code signing certificate 2020-03-24 12:22:46 -04:00
Jon Atack
83e1d92413 test: listsinceblock block height checks 2020-03-24 16:44:26 +01:00
MarcoFalke
5236b2e267 Merge #18417: tests: Add fuzzing harnesses for functions in addrdb.h, net_permissions.h and timedata.h
4308aa67e3 tests: Add fuzzing harness for functions in net_permissions.h (practicalswift)
43ff0d91f8 tests: Add fuzzing harness for functions in timedata.h (practicalswift)
a8695db785 tests: Add fuzzing harness for functions in addrdb.h (practicalswift)

Pull request description:

  Add fuzzing harnesses for functions in `addrdb.h`, `net_permissions.h` and `timedata.h`.

Top commit has no ACKs.

Tree-SHA512: ea41431e7f1944ecd0c102e6ea04e70d6763dc9b6e3a0949a4f7299897a92fa3e8e7139f9f65b9508ce8d45613ea24ec0fd6d4a8be3cfd7c23136512b17770eb
2020-03-24 11:35:32 -04:00
MarcoFalke
98fbb2a184 Merge #17720: test: add unit test for non-standard "scriptsig-not-pushonly" txs
5aab011805 test: add unit test for non-standard "scriptsig-not-pushonly" txs (Sebastian Falbesoner)

Pull request description:

  Approaches another missing unit test of issue #17394: Checks that the function `IsStandardTx()` returns rejection reason "scriptsig-not-pushonly" if any one of the input's scriptSig consists of any other ops than just PUSHs.

ACKs for top commit:
  MarcoFalke:
    ACK 5aab011805 🍟
  practicalswift:
    ACK 5aab011805 -- patch looks correct

Tree-SHA512: fbe25bcf57e5f0c8d2397eb67e61fe8d9145ba83032789adb2b67d6fcbcd87e6427e9d965e8cd7bbaaea482e39ec2f110f71ef2de079c7d1fba2712848caa9ba
2020-03-24 11:12:33 -04:00
practicalswift
4308aa67e3 tests: Add fuzzing harness for functions in net_permissions.h 2020-03-24 14:39:23 +00:00
practicalswift
43ff0d91f8 tests: Add fuzzing harness for functions in timedata.h 2020-03-24 14:39:23 +00:00
practicalswift
a8695db785 tests: Add fuzzing harness for functions in addrdb.h 2020-03-24 14:39:23 +00:00
Jon Atack
76db4b260e gui: avoid QT Designer/Form Editor re-formatting
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2020-03-24 14:04:16 +01:00
Jon Atack
aae26053f9 gui: display Mapped AS in peers info window 2020-03-24 14:04:14 +01:00
Karl-Johan Alm
ac94141af0 validation: delay flushing undo files in syncing node case
Data files are pre-allocated, and upon flush/finalization, they are trimmed down to their resulting size.
Block (blk) files are written to disk as blocks come in, which is often out of order, whereas undo (rev) files are written sequentially, as each block is added to the top of the chain.
When a block file hits the size limit, the system flushes and trims the file down to its final size, and moves on to the next block file.

Case 1: blocks are added to the chain as they come in (synced up node case) -- in this case, we will flush and finalize the undo file together with the block file.

Case 2: blocks are added to the chain after they have been downloaded (syncing node case) -- in this case, we postpone finalizing the undo file until we know the undo data for the last block in the file has been written to disk.
2020-03-24 11:57:19 +09:00
Pieter Wuille
2b0fcff7f2 Make VerifyWitnessProgram use a Span stack
This allows for very cheap transformations on the range of elements that
are to be passed to ExecuteWitnessScript.
2020-03-23 14:44:58 -07:00
MarcoFalke
ac579ada7e Merge #18278: interfaces: Describe and follow some code conventions
3dc27a1524 doc: Add internal interface conventions to developer notes (Russell Yanofsky)
1dca9dc4c7 refactor: Change createWallet, fillPSBT argument order (Russell Yanofsky)
96dfe5ced6 refactor: Change Chain::broadcastTransaction param order (Russell Yanofsky)
6ceb21909c refactor: Rename Chain::Notifications methods to be consistent with other interfaces methods (Russell Yanofsky)
1c2ab1a6d2 refactor: Rename Node::disconnect methods (Russell Yanofsky)
77e4b06572 refactor: Get rid of Wallet::IsWalletFlagSet method (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  This PR doesn't change behavior at all, it just cleans up code in [`src/interfaces`](https://github.com/bitcoin/bitcoin/tree/master/src/interfaces) to simplify #10102, and [documents](https://github.com/ryanofsky/bitcoin/blob/pr/ipc-conv/doc/developer-notes.md#internal-interface-guidelines) coding conventions there better

ACKs for top commit:
  hebasto:
    re-ACK 3dc27a1524, the only change since the [previous](https://github.com/bitcoin/bitcoin/pull/18278#pullrequestreview-372582146) review is rebasing.
  MarcoFalke:
    ACK 3dc27a1524 🕍

Tree-SHA512: 62e6a0f2488e3924e559d2074ed460b92e7a0a5d98eab492221cb20d59d04bbe32aef2a8aeba5e4ea9168cfa91acd5bc973dce6677be0180bd7a919354df53ed
2020-03-23 16:37:50 -04:00
Sebastian Falbesoner
41ff4992e5 script: fix SCRIPT_ERR_SIG_PUSHONLY error string 2020-03-23 17:33:48 +01:00
Elichai Turkel
f65c9ad40f Check for overflow when calculating sum of outputs 2020-03-23 16:36:24 +02:00
MarcoFalke
97b0687501 Merge #18393: tests: Don't assume presence of __builtin_mul_overflow(…) in MultiplicationOverflow(…) fuzzing harness
7c1ac70c01 tests: Don't assume presence of __builtin_mul_overflow in MultiplicationOverflow(...) fuzzing harness (practicalswift)

Pull request description:

  Don't assume presence of `__builtin_mul_overflow(…)` in `MultiplicationOverflow(…)` fuzzing harness.

  Fixes #18389.

ACKs for top commit:
  naumenkogs:
    ACK 7c1ac70

Tree-SHA512: b6f1040a088088ff7e4f5c038f0f710ca2b515387bac3cd249afe97613641f7f3754f61d73d7233f23b8296115fab5bbf656168624a2cb74909577440a49a359
2020-03-23 07:39:47 -04:00
MarcoFalke
d478a737db Merge #18392: ci: Run feature_block in valgrind
fa92af5af3 ci: Run feature_block and feature_abortnode in valgrind (MarcoFalke)
fa01febeaf test: Remove ci timeout restriction in test_runner (MarcoFalke)

Pull request description:

  Also revert commit 0a4912e46a, because some tests take too long for this to be useful anymore.

Top commit has no ACKs.

Tree-SHA512: 363f14766e1f4a5860ab668a516b41acebc6fbdf11d8defb3a95a772dbf82304ca1f5f14b1dbad97f2029503e03d92e8c69df0466a8872409c20665838f617ed
2020-03-23 07:38:07 -04:00
fanquake
57a8fa9c3b Merge #18406: test: add executable flag for rpc_estimatefee.py
f0dfac7da3 test: add executable flag for rpc_estimatefee.py (Sebastian Falbesoner)

Pull request description:

  Again a functional test without executable flag set sneaked in (see e.g. https://github.com/bitcoin/bitcoin/pull/17806 and https://github.com/bitcoin/bitcoin/pull/16742 for previous similar PRs, setting the filemode from 644 to 755). Maybe a linter like suggested in https://github.com/bitcoin/bitcoin/pull/17830 would be worth considering to avoid future (trivial) PRs like this?

ACKs for top commit:
  promag:
    ACK f0dfac7da3.
  kristapsk:
    ACK f0dfac7da3

Tree-SHA512: b37c11bdef439aa9d5736c9e0e0bbcc19aff876744f0c4e099ca5c67c9ff1293f1f9140f0d167ea13fee5396ae017aa4a0f1bae4f7aec8fa80b46beb421561c1
2020-03-23 10:46:07 +08:00
Sebastian Falbesoner
f0dfac7da3 test: add executable flag for rpc_estimatefee.py 2020-03-22 23:20:35 +01:00
Hennadii Stepanov
f642b49af7 build: Drop ZeroMQ patch for glibc < 2.12
It is safe to use pthread_setname_np since #17538 when the minimal glibc
version is set to 2.17.
2020-03-22 21:27:56 +02:00
Hennadii Stepanov
079df9609e build: Drop ZeroMQ patch for Mingw-w64 < 4.0
The Mingw-w64 5.0 (Ubuntu 18.04 Bionic) is used to build the Windows
binaries now.
2020-03-22 21:26:39 +02:00
MarcoFalke
fa92af5af3 ci: Run feature_block and feature_abortnode in valgrind 2020-03-22 10:12:40 -04:00
practicalswift
7c1ac70c01 tests: Don't assume presence of __builtin_mul_overflow in MultiplicationOverflow(...) fuzzing harness 2020-03-22 13:29:00 +00:00
Jon Atack
bd05f96d79 test: delete wallet_bumpfee_totalfee_deprecation.py 2020-03-22 11:02:57 +01:00
Jon Atack
a6d1ab8caa test: update bumpfee testing from totalFee to fee_rate 2020-03-22 11:02:54 +01:00
fanquake
1a0993ae35 scripts: add PE dylib checking to symbol-check.py 2020-03-22 10:47:38 +08:00
“jkcd”
a2a03c3ca9 fixing documentation to not require rpcpassword 2020-03-21 19:51:31 +05:30
Sebastian Falbesoner
ef35604c9c rpc: fix broken RPCExamples for waitforblock(height)
Fixes the following RPCExamples:
-> ExampleCli waitforblock (removed comma between arguments)
-> ExampleCli waitforblockheight (removed comma between arguments)
-> ExampleRpc waitforblockheight (removed quotation marks around integer argument)
2020-03-21 14:56:33 +01:00
MarcoFalke
5504703a9f Merge #18396: rpc: add missing HelpExampleRpc for getblockfilter
d18bf0c0b0 rpc: add missing HelpExampleRpc for getblockfilter (Sebastian Falbesoner)

Pull request description:

  From all RPCs in the "blockchain" category, `getblockfilter` is the only one where there is only a CLI example present but not a curl RPC example (all other RPCs in this category have either both or none). This PR adds the missing `HelpExampleRpc` string.

ACKs for top commit:
  emilengler:
    utACK d18bf0c

Tree-SHA512: b37c11bdef439aa9d5736c9e0e0bbcc19aff876744f0c4e099ca5c67c9ff1293f1f9140f0d167ea13fee5396ae017aa4a0f1bae4f7aec8fa80b46beb421561c1
2020-03-21 09:17:38 -04:00
Hennadii Stepanov
cd04286825 build: Fix typo in EVENT_CFLAGS variable
Co-authored-by: Harris <brakmic@gmail.com>
2020-03-21 14:23:30 +02:00
Hennadii Stepanov
f709ad0c90 build: Fix libevent linking for bench_bitcoin binary 2020-03-21 14:18:07 +02:00
Sebastian Falbesoner
d18bf0c0b0 rpc: add missing HelpExampleRpc for getblockfilter 2020-03-21 10:03:40 +01:00
MarcoFalke
fa01febeaf test: Remove ci timeout restriction in test_runner 2020-03-20 20:58:13 -04:00
Vasil Dimov
ca2e474372 Fix a compiler warning: unused GetDevURandom()
```
random.cpp:255:13: error: unused function 'GetDevURandom' [-Werror,-Wunused-function]

```

Clang 9.0.0, FreeBSD 12.1

Silence by planting a dummy reference to the `GetDevURandom` symbol
in the places where we don't call the function.
2020-03-20 20:49:10 +01:00
MarcoFalke
5bf45fe2a9 Merge #18386: doc: Temporary note that release notes should be edited in wiki
b492684063 doc: Temporary note that release notes should be edited in wiki (Wladimir J. van der Laan)

Pull request description:

  Replace release notes with temporary note that 0.20.0 release notes should be edited in wiki.

ACKs for top commit:
  MarcoFalke:
    ACK b492684063

Tree-SHA512: 7a8835f7807e3cd6e4fea2969cf4dfa21d2aab2be7bfc1a6403926ea60e1193573e967d7ff512a640395e06de4877fec7f7a5c48619856f69fd5f894d27f1875
2020-03-19 16:26:04 -04:00
Russell Yanofsky
3dc27a1524 doc: Add internal interface conventions to developer notes 2020-03-19 15:26:04 -05:00
Russell Yanofsky
1dca9dc4c7 refactor: Change createWallet, fillPSBT argument order
Move output arguments after input arguments for consistency with other methods,
and to work more easily with IPC framework in #10102
2020-03-19 15:26:04 -05:00
Russell Yanofsky
96dfe5ced6 refactor: Change Chain::broadcastTransaction param order
Make output argument last argument so it works more easily with IPC framework
in #10102, and for consistency with other methods
2020-03-19 15:26:04 -05:00
Russell Yanofsky
6ceb21909c refactor: Rename Chain::Notifications methods to be consistent with other interfaces methods
This also simplifies #10102 removing overrides needed to deal with inconsistent
case convention
2020-03-19 15:26:04 -05:00
Russell Yanofsky
1c2ab1a6d2 refactor: Rename Node::disconnect methods
Avoid overloading method name to work more easily with IPC framework
2020-03-19 15:26:04 -05:00
Russell Yanofsky
77e4b06572 refactor: Get rid of Wallet::IsWalletFlagSet method
Replace by privateKeysDisabled method to avoid need for GUI to reference
internal wallet flags.

Also remove adjacent WalletModel canGetAddresses wrapper that serves no purpose
and make Wallet::canGetAddresses non-const so it can be implemented by IPC
classes in #10102.
2020-03-19 15:26:04 -05:00
Wladimir J. van der Laan
b492684063 doc: Temporary note that release notes should be edited in wiki 2020-03-19 20:57:11 +01:00
Wladimir J. van der Laan
312d27b11c Merge #17477: Remove the mempool's NotifyEntryAdded and NotifyEntryRemoved signals
e57980b473 [mempool] Remove NotifyEntryAdded and NotifyEntryRemoved callbacks (John Newbery)
2dd561f361 [validation] Remove pool member from ConnectTrace (John Newbery)
969b65f3f5 [validation] Remove NotifyEntryRemoved callback from ConnectTrace (John Newbery)
5613f9842b [validation] Remove conflictedTxs from PerBlockConnectTrace (John Newbery)
cdb893443c [validation interface] Remove vtxConflicted from BlockConnected (John Newbery)
1168394d75 [wallet] Notify conflicted transactions in TransactionRemovedFromMempool (John Newbery)

Pull request description:

  These boost signals were added in #9371, before we had a `TransactionRemovedFromMempool` method in the validation interface. The `NotifyEntryAdded` callback was used by validation to build a vector of conflicted transactions when connecting a block, which the wallet was notified of in the `BlockConnected` CValidationInterface callback.

  Now that we have a `TransactionRemovedFromMempool` callback, we can fire that signal directly from the mempool for conflicted transactions.

  Note that #9371 was implemented to ensure `-walletnotify` events were fired for these conflicted transaction. We inadvertently stopped sending these notifications in #16624 (Sep 2019 commit 7e89994). We should probably fix that, but in a different PR.

ACKs for top commit:
  jonatack:
    Re-ACK e57980b
  ryanofsky:
    Code review ACK e57980b473, no code changes since previous review, but helpful new code comments have been added and the PR description is now more clear about where the old code came from

Tree-SHA512: 3bdbaf1ef2731e788462d4756e69c42a1efdcf168691ce1bbfdaa4b7b55ac3c5b1fd4ab7b90bcdec653703600501b4224d252cfc086aef28f9ce0da3b0563a69
2020-03-19 17:26:51 +01:00
Wladimir J. van der Laan
527c3989e7 Merge #18340: doc: mention MAKE=gmake workaround when building on a BSD
84a46a9b93 doc: mention MAKE=gmake workaround when building on a BSD (emu)

Pull request description:

  Fixes: #14404.
  Replaces: #18129.

ACKs for top commit:
  vasild:
    ACK 84a46a9b93
  laanwj:
    ACK 84a46a9b93

Tree-SHA512: 7a28c17c5d8a5d98aaedfb849d10a3a809f0d6d4b8f03add2cd6927e9d9689613b8b5c53e62d8e0fce8f4732efcee9ed3a83b0ed325b38934ceff6057a6db163
2020-03-19 15:50:31 +01:00
Wladimir J. van der Laan
3d3d834324 Merge #18359: build: fix sysctl() detection on macOS
e90e3e684f build: fix sysctl() detection on macOS (fanquake)

Pull request description:

  [`sysctl()` on *BSD](https://www.unix.com/man-page/FreeBSD/3/sysctl/) takes a "const int *name", whereas [`sysctl()` on macOS](https://developer.apple.com/library/archive/documentation/System/Conceptual/ManPages_iPhoneOS/man3/sysctl.3.html)
  it takes an "int *name". So our configure check and `sysctl()` detection on
  macOS currently fails:

  ```bash
  /usr/include/sys/sysctl.h:759:9: note: candidate function not viable:
  	no known conversion from 'const int [2]' to 'int *' for 1st argument
  int     sysctl(int *, u_int, void *, size_t *, void *, size_t);
  ```

  The simplest change seems to be to change the param to a "int *name", which
  will work during configure on macOS and *BSD systems.

  For consistency I've changed both calls, but note that macOS doesn't
  have `KERN_ARND`, so that check will always fail regardless. We can revert/add
  documentation if preferred.

ACKs for top commit:
  laanwj:
    Re-ACK e90e3e684f

Tree-SHA512: 29e9348136fc72882f63079bf10d2490e845d7656aae2c003e282bea49dd2778204a7776a67086bd88c2852af9a07dd04ba358eede7e37029e1c10f73c85d6a5
2020-03-19 15:23:51 +01:00
Wladimir J. van der Laan
67de1ee8bc Merge #18378: Bugfix & simplify bn2vch using int.to_bytes
a733ad514a Add bn2vch test to functional tests (Pieter Wuille)
a3ad6459b7 Simplify bn2vch using int.to_bytes (Pieter Wuille)

Pull request description:

  Alternative to #18374, fixing the incorrect padding added sometimes in `bn2vch`.

  Since we're using Python 3.2+, a much simpler implementation of `bn2vch` is possible using `int.to_bytes`.

  This also adds a "functional" test for bn2vch, in a new "framework_test_script.py", where the "framework_test_" prefix is intended for tests of the framework itself.

ACKs for top commit:
  laanwj:
    nice, ACK a733ad514a
  jnewbery:
    Tested ACK a733ad514a.

Tree-SHA512: aeacc4e7fd84279023d38e8b4a5175fb16d7b3a7f93c61b9dcb59cd9927547732983c76f28564b62e37088399fc0121b38a514d73b0ea38b3983836539e9ca90
2020-03-19 09:58:11 +01:00
fanquake
e90e3e684f build: fix sysctl() detection on macOS
sysctl() on *BSD takes a "const int *name", whereas sysctl() on macOS
it takes an "int *name". So our configure check and sysctl() detection on
macOS currently fails:

```bash
/usr/include/sys/sysctl.h:759:9: note: candidate function not viable:
	no known conversion from 'const int [2]' to 'int *' for 1st argument
int     sysctl(int *, u_int, void *, size_t *, void *, size_t);
```

This change removes the name argument from the sysctl() detection check,
meaning we will detect correctly on macOS and *BSD.

For consistency we also switch to using the more generic, non-const
version of the name parameter in the rest of our usage.
2020-03-19 11:36:38 +08:00
practicalswift
33dd764984 doc: Add fuzzing quickstart guides for libFuzzer and afl-fuzz. Simplify instructions. 2020-03-18 22:32:01 +00:00
Pieter Wuille
a733ad514a Add bn2vch test to functional tests 2020-03-18 13:54:01 -07:00
Pieter Wuille
a3ad6459b7 Simplify bn2vch using int.to_bytes 2020-03-18 13:54:01 -07:00
MarcoFalke
a421e0a22f Merge #18376: net: fix use-after-free in tests
7d8e1dec3b net: fix use-after-free in tests (Vasil Dimov)

Pull request description:

  In PeerLogicValidation::PeerLogicValidation() we would schedule a lambda
  function to execute later, capturing the local variable
  `consensusParams` by reference.

  Presumably this was considered safe because `consensusParams` is a
  reference itself to a global variable which is not supposed to change,
  but it can in tests.

  Fixes https://github.com/bitcoin/bitcoin/issues/18372

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  sipa:
    ACK 7d8e1dec3b
  practicalswift:
    ACK 7d8e1dec3b
  MarcoFalke:
    ACK 7d8e1dec3b

Tree-SHA512: fe0f6e5fac1976d38dfb249517eef142dcb8837e178d7d199e5e854e3ab428822c6da9d96fe312293d39b6c6cac0c97896f3b5760013db200cccd729ae1b0710
2020-03-18 16:40:47 -04:00
MarcoFalke
5c9d408b2d Merge #18300: fuzz: Add option to merge input dir to test runner
fa3fa27c45 fuzz: Remove option --export_coverage from test_runner (MarcoFalke)
aaaa055ff7 fuzz: Add option to merge input dir to test runner (MarcoFalke)
fa4fa88d76 doc: Remove --disable-ccache from docs (MarcoFalke)

Pull request description:

  This is mainly useful for myself to merge pull requests like https://github.com/bitcoin-core/qa-assets/pull/4

  I thought it wouldn't hurt to share the code.

  Also remove the `--disable-ccache` from the docs to speed up builds when developing fuzzers.

Top commit has no ACKs.

Tree-SHA512: 818d85a90db86a7f4e8b001cc88342e5b28b02029d2bd4174440b28a8c4cc29b5406bd6348f72ddf909bb3d0f9bf7b1011976f6480e4418c8b7da5ecccae93e8
2020-03-18 15:51:44 -04:00
MarcoFalke
e83a1de4c0 Merge #18155: tests: Add harness which fuzzes EvalScript and VerifyScript using a fuzzed signature checker
5e47b19e50 tests: Add harness which fuzzes EvalScript and VerifyScript using a fuzzed signature checker (practicalswift)

Pull request description:

  Add harness which fuzzes `EvalScript` and `VerifyScript` using a fuzzed signature checker.

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/signature_checker
  …
  ```

  Closes #17986.

Top commit has no ACKs.

Tree-SHA512: a9988f8fa7919fe470756ca3e4e75764a589f590769aab452c8f4c254cf41667793e52131d470a12629ec3681fa7fc20091f371b8f3e3eec105674c2769e7d7e
2020-03-18 15:48:27 -04:00
MarcoFalke
dbb067da38 Merge #18379: doc: Comment fix merkle.cpp
5b59a19731 Update merkle.cpp (4d55397500)

Pull request description:

  Change comment from `The reason is that if the number of hashes in the list at a given time
  is odd`, to ` The reason is that if the number of hashes in the list at a given level
         is odd` (to be a bit more precise: replacing `time` with `level`)

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  MarcoFalke:
    ACK 5b59a19731
  instagibbs:
    ACK 5b59a19731

Tree-SHA512: 30d29b9855b30de8b54033ca4884cfb5bf8ab9e52cf61da237abba0e15ebff947c65f8ba82175694bc60ee0d54f940a098cadcb0404d3c3bcf577006ab0561a5
2020-03-18 14:02:22 -04:00
4d55397500
5b59a19731 Update merkle.cpp
Change comment from `The reason is that if the number of hashes in the list at a given time
is odd`, to ` The reason is that if the number of hashes in the list at a given level
       is odd` (to be a bit more precise)
2020-03-18 10:34:53 -07:00
MarcoFalke
3a8d25064e Merge #18371: test: use fs namespace in dbwrapper unicodepath test
6afaf2f680 test: use fs namespace in dbwrapper unicodepath test (fanquake)

Pull request description:

  Use our `fs` namespace rather than `boost::filesystem`. Test was added in #17641.

ACKs for top commit:
  sipsorcery:
    ACK 6afaf2f680.

Tree-SHA512: 5ee024a6d90183b6c344f6a94cfbcacb006973f1f6d98cc421c1c6ef08c09b590d31c78b70b86d855e825241ffea25989cfc40d1bdd53e38a75cda0718ac4489
2020-03-18 09:42:25 -04:00
Vasil Dimov
7d8e1dec3b net: fix use-after-free in tests
In PeerLogicValidation::PeerLogicValidation() we would schedule a lambda
function to execute later, capturing the local variable
`consensusParams` by reference.

Presumably this was considered safe because `consensusParams` is a
reference itself to a global variable which is not supposed to change,
but it can in tests.

Fixes https://github.com/bitcoin/bitcoin/issues/18372
2020-03-18 13:03:26 +01:00
fanquake
6afaf2f680 test: use fs namespace in dbwrapper unicodepath test 2020-03-18 11:10:20 +08:00
emu
84a46a9b93 doc: mention MAKE=gmake workaround when building on a BSD
Fixes: #14404
2020-03-18 07:55:07 +08:00
MarcoFalke
ce87d5613a Merge #18289: refactor: Make scheduler methods type safe
fa36f3a295 refactor: move DUMP_BANS_INTERVAL to banman.h (MarcoFalke)
fadafb83cf scheduler: Make schedule* methods type safe (MarcoFalke)
fa70ccc6c4 scheduler: Use C++11 member initialization, add shutdown assert (MarcoFalke)

Pull request description:

  Main benefit is that stuff like `15 * 60 * 1000` is replaced by `minutes{15}`

ACKs for top commit:
  vasild:
    ACK fa36f3a (code review, not tested)
  ajtowns:
    ACK fa36f3a295
  jonatack:
    ACK fa36f3a

Tree-SHA512: f35f1a1d643dfa676bd47474659f6492ed05cca04cdb556064b126f654a6a44a4b93fcaddcdcd41faf81b8f11439c11e5c7ab88685ba2eef12f7188843d17ad8
2020-03-17 16:34:53 -04:00
practicalswift
5e47b19e50 tests: Add harness which fuzzes EvalScript and VerifyScript using a fuzzed signature checker 2020-03-17 19:10:59 +00:00
MarcoFalke
39497d1f32 Merge #15283: log: Fix UB with bench on genesis block
ec30a79f1c Fix UB with bench on genesis block (Gregory Sanders)

Pull request description:

  During the loading of the genesis block, the bench print lines in ConnectTip divide by zero due to early return in ConnectBlock.

ACKs for top commit:
  practicalswift:
    ACK ec30a79f1c
  sipa:
    utACK ec30a79f1c
  promag:
    ACK ec30a79, `nBlocksTotal` is only used in logging.

Tree-SHA512: b3bdbb58d10d002a2293d7f99196b227ed9f4ca8c6cd08981e95cc964be47efed98b91fad276ee6da5cf7e6684610998ace7ce9bace172dd6c51c386d985b83c
2020-03-17 14:27:42 -04:00
James O'Beirne
c9017ce3bc protect g_chainman with cs_main
I'd previously attempted to create a specialized lock for ChainstateManager,
but it turns out that because that lock would be required for functions like
ChainActive() and ChainstateActive(), it created irreconcilable lock inversions
since those functions are used so broadly throughout the codebase.

Instead, I'm just using cs_main to protect the contents of g_chainman.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-03-17 14:07:58 -04:00
James O'Beirne
2b081c4568 test: add basic tests for ChainstateManager
Feedback incorporated from Russell Yanofsky.

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-03-17 14:03:40 -04:00
James O'Beirne
4ae29f5f0c use ChainstateManager to initialize chainstate
This allows us to easily initialize multiple chainstates on startup in future
commits. It retires the g_chainstate global in lieu of g_chainman.
2020-03-17 14:03:40 -04:00
James O'Beirne
5b690f0aae refactor: move RewindBlockIndex to CChainState
This is in preparation for multiple chainstate initialization in init.
2020-03-17 14:03:40 -04:00
MarcoFalke
ad04f0d8a5 Merge #17319: Tests: remove bignum module
3ed772d221 [tests] remove bignum.py (John Newbery)
f950ec2520 [tests] remove bn2bin() (John Newbery)
3b9b38579c [tests] remove bn_bytes() function (John Newbery)
a760aa14a9 [tests] remove mpi2vch() function (John Newbery)
9a60bef50d [tests] don't encode the integer size in bignum (John Newbery)
1dc68aee66 [tests] add function comments to bignum (John Newbery)
f31fc0e92e [tests] fix flake8 warnings in script.py and bignum.py (John Newbery)

Pull request description:

  Only one function is imported in script.py. Just move that function to script.py and remove the bignum.py module.

  Remove unused functionality and fix some flake8 warnings along the way.

Top commit has no ACKs.

Tree-SHA512: 015f543ab545b5d5451896e2751d9c19334d9155b03faacd2023781e89833a2440f7f28741e9a8ac49badd9cdc012cbb6e038cdcdebeefaf9cb9d461c0689157
2020-03-17 13:38:00 -04:00
James O'Beirne
89cdf4d569 validation: introduce unused ChainstateManager
ChainstateManager is responsible for creating and managing multiple
chainstates, and will provide a high-level interface for accessing the
appropriate chainstate based upon a certain use.

Incorporates feedback from Marco Falke. Additional documentation written
by Russ Yanofsky.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-03-17 13:27:39 -04:00
James O'Beirne
8e2ecfe249 validation: add CChainState.m_from_snapshot_blockhash
This parameter is unused, but in future commits will allow ChainstateManager to
differentiate between chainstates created from a UTXO snapshot from those that
weren't.
2020-03-17 13:27:39 -04:00
MarcoFalke
d2d0a04a66 Merge #18353: tests: Add fuzzing harnesses for classes CBlockHeader, CFeeRate and various functions
44abf417eb tests: Add fuzzing harness for various functions taking std::string as input (practicalswift)
d69145acb7 tests: Add fuzzing harness for MultiplicationOverflow(...) (practicalswift)
7726f3bc46 tests: Add fuzzing harness for CFeeRate (practicalswift)
0579a27630 tests: Add fuzzing harness for CBlockHeader (practicalswift)
cb4eec13c0 tests: Add fuzzing harness for count_seconds(...) (practicalswift)

Pull request description:

  Add fuzzing harnesses for classes `CBlockHeader`, `CFeeRate` and various functions.

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/block_header
  ^c (ctrl-c)
  $ src/test/fuzz/fee_rate
  ^c (ctrl-c)
  $ src/test/fuzz/integer
  ^c (ctrl-c)
  $ src/test/fuzz/multiplication_overflow
  ^c (ctrl-c)
  $ src/test/fuzz/string
  ^c (ctrl-c)
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 44abf417eb 🏉

Tree-SHA512: 2b382a7bc8efdcc6dd8b79f1637f194ecdca3e522c6618ae6c4b0bf6f86d2e79b1bb1c7160522083600616d1ed509b2f577f3a512ea3a7825a0a3794578d9d90
2020-03-17 13:07:42 -04:00
Wladimir J. van der Laan
1d64dfe4fa Merge #18364: random: remove getentropy() fallback for macOS < 10.12
f9f210d8de doc: fix GetTimeMicros() comment in random.cpp (fanquake)
a889711562 rand: remove getentropy() fallback for macOS < 10.12 (fanquake)

Pull request description:

  We [no longer support macOS < 10.12](https://github.com/bitcoin/bitcoin/pull/17550) (our binaries will not run), so remove the fallback for when `getentropy()` wasn't available. From the manpage:

  ```bash
  HISTORY
       The getentropy() function appeared in OSX 10.12
  ```

  Note that compiling on macOS you'll see a new unused function warning:
  ```bash
  random.cpp:256:13: warning: unused function 'GetDevURandom' [-Wunused-function]
  static void GetDevURandom(unsigned char *ent32)
              ^
  1 warning generated.
  ```

  This will likely be addressed as part of #17563.

ACKs for top commit:
  vasild:
    ACK f9f210d8 (code review, not tested)
  elichai:
    utACK f9f210d8de
  practicalswift:
    ACK f9f210d8de -- patch looks correct
  laanwj:
    ACK f9f210d8de
  hebasto:
    ACK f9f210d8de, tested on macOS 10.13.6: compiled

Tree-SHA512: 6bd2a721f23605a8bca0b7b51f42d628ebf92a18e74eb43194331ba745ee449223aff84119892781c40b188c70b75417447f4e390e3d9ac549292de2b1e8b308
2020-03-17 17:03:20 +01:00
MarcoFalke
8fc5f2bdf0 Merge #18366: tests: simplify next_block() function in feature_block
612a931d1a tests: simplify next_block() function in feature_block (John Newbery)

Pull request description:

  The solve parameter is unnecessary. Remove it and add comments.

ACKs for top commit:
  MarcoFalke:
    ACK 612a931d1a
  TheQuantumPhysicist:
    ACK 612a931

Looks good. Thanks for improving it 😄
  practicalswift:
    ACK 612a931d1a -- simpler is better and patch looks correct :)

Tree-SHA512: 25b4879842ea37a3f598be886f02ce4c2fb0b5a618d02b266dbd380f5cbfdd71a8bd35ddd9d6f2cf83920e37c02caf9955a841a02b17ba75ac63f88d32f8b60b
2020-03-17 10:23:58 -04:00
John Newbery
612a931d1a tests: simplify next_block() function in feature_block
The solve parameter is unnecessary. Remove it and add
comments.
2020-03-17 09:45:00 -04:00
fanquake
f9f210d8de doc: fix GetTimeMicros() comment in random.cpp 2020-03-17 13:09:52 +08:00
fanquake
a889711562 rand: remove getentropy() fallback for macOS < 10.12 2020-03-17 13:00:31 +08:00
Anthony Towns
96954d1794 DNS seeds: don't query DNS while network is inactive 2020-03-17 12:10:47 +10:00
Anthony Towns
fa5894f7f5 DNS seeds: wait for 5m instead of 11s if 1000+ peers are known
If 1000 potential peers are known, wait for 5m before querying DNS seeds
for more peers, since eventually the addresses we already know should
get us connected. Also check every 11s whether we've got enough active
outbounds that DNS seeds aren't worth querying, and exit the dnsseed
thread early if so.
2020-03-17 11:33:07 +10:00
MarcoFalke
7060d2d97a Merge #18350: test: Fix mining to an invalid target + ensure that a new block has the correct hash internally
7a6627ae87 Fix mining to an invalid target + ensure that a new block has the correct hash internally in Python tests (Samer Afach)

Pull request description:

  Test with block 47 in the `feature_block.py` creates a block with a hash higher than the target, which is supposed to fail. Now two issues exist there, and both have low probability of showing up:

  1. The creation is done with `while (hash < target)`, which is wrong, because hash = target is a valid mined value based on the code in the function `CheckProofOfWork()` that validates the mining target:
  ```
      if (UintToArith256(hash) > bnTarget)
          return false;
  ```
  2. As we know the hash stored in CBlock class in Python is stateful, unlike how it's in C++, where calling `CBlock::GetHash()` will actively calculate the hash and not cache it anywhere. With this, blocks that come out of the method `next_block` can have incorrect hash value when `solve=False`. This is because the `next_block` is mostly used with `solve=True`, and solving does call the function `rehash()` which calculates the hash of the block, but with `solve=False`, nothing calls that method. And since the work to be done in regtests is very low, the probably of this problem showing up is very low, but it practically happens (well, with much higher probability compared to issue No. 1 above).

  This PR fixes both these issues.

Top commit has no ACKs.

Tree-SHA512: f3b54d18f5073d6f1c26eab89bfec78620dda4ac1e4dde4f1d69543f1b85a7989d64c907e091db63f3f062408f5ed1e111018b842819ba1a5f8348c7b01ade96
2020-03-16 16:31:42 -04:00
Wladimir J. van der Laan
25424cf57e Merge #18346: rpc: Document an RPCResult for all calls; Enforce at compile time
fac52253f8 rpc: Document an RPCResult for all calls; Enforce at compile time (MarcoFalke)
fadd99f610 rpc: Add missing newline in RPCResult description (MarcoFalke)

Pull request description:

  This documents the RPC Result (type and description, if applicable) everywhere it was missing. The patch can be reviewed with the `git diff` option `-W`/`--function-context`.

  Also, code won't compile without having an RPCResult documented.

ACKs for top commit:
  laanwj:
    Lightly tested ACK fac52253f8
  promag:
    Tested ACK fac52253f8, built and verified listunspent help output.

Tree-SHA512: af2c1af1432beb944993776026c320814bfaecaf202f47359f5758849096ca7051ec6560395a2cc6678dcc111e7c9cf4917d0f0b221bdcf3ed1642e14d0e5b3c
2020-03-16 18:04:08 +01:00
MarcoFalke
8662387309 Merge #17997: refactor: Remove mempool global from net
fa7fea3654 refactor: Remove mempool global from net (MarcoFalke)

Pull request description:

  To increase modularisation and simplify testing, remove the mempool global from net in favour of a mempool member.

  This is done in the same way it was done for the connection manager global.

ACKs for top commit:
  jnewbery:
    code review ACK fa7fea3654

Tree-SHA512: 0e3e1eefa8d6e46367bc6991d5f36c636b15ae4a3bda99b6fe6715db3240771c3d87943c6eb257d69f31929fa2f1d0973e14fc9d1353a27551dbe746eae36857
2020-03-16 11:05:45 -04:00
Wladimir J. van der Laan
a64e5bed6b Merge #18344: doc: Fix nit in getblockchaininfo
fb15bfd99e Fix nit in getblockchaininfo (Steven Roose)

Pull request description:

  Noticed that the statistics are not always shown.

ACKs for top commit:
  laanwj:
    ACK fb15bfd99e
  promag:
    ACK fb15bfd99e.

Tree-SHA512: bccbfdff03107d14967f6530eec0bcada7ba8eb16c61b829119533a73f2ead742a0da6a473b7962b15e25cd685c8f155506ab16d4a95b20352d3fd1b4b0164a3
2020-03-16 13:19:35 +01:00
Wladimir J. van der Laan
9ed429c50a Merge #18341: doc: Replace remaining literal BTC with CURRENCY_UNIT.
7df0cf719f Replace remaining literals BTC with CURRENCY_UNIT (Daniel Kraft)

Pull request description:

  This replaces one remaining instance of the literal `"BTC"` string with the `CURRENCY_UNIT` constant, as is done in most of the codebase already.

  After this change, no instance of literal `"BTC"` remains anywhere in the RPC help texts.

ACKs for top commit:
  MarcoFalke:
    ACK 7df0cf719f
  laanwj:
    ACK 7df0cf719f

Tree-SHA512: 7f7d52b366e084c93a7d6a3c45b1bbfc4f4f50bca6956594077e6d46295977c8cc18499232878869869c73a8ab9a1c41245029ae7425a87cec2ccb0cb52eea13
2020-03-16 13:14:10 +01:00
fanquake
69f4f2c433 Merge #18357: Fix missing header in sync.h
686c5456f2 Fix missing header in sync.h (João Barbosa)

Pull request description:

  `std::string` is referenced in `sync.h` but the relevant header is not explicitly included as required by current guideline. Furthermore on osx 10.14.6 with clang-900.0.31 the following error occurs:
  ```
  In file included from threadinterrupt.cpp:6:
  In file included from ./threadinterrupt.h:8:
  ./sync.h:206:21: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
          std::string lockname;
  ```

ACKs for top commit:
  practicalswift:
    ACK 686c5456f2
  laanwj:
    ACK 686c5456f2

Tree-SHA512: 7c1acdfa5b0dd148d1114e14c9450d5907006e63e1a04e82ed8a1e29757925476e6f8ef6024b0c6d1bb596623115209ad580d5035be1e4785337bd01b738c9f2
2020-03-16 20:04:54 +08:00
Wladimir J. van der Laan
d31f92586a Merge #18360: Bump transifex slug and update English translations for 0.20
2a1b85f3c5 tx: Bump transifex slug to 020x (Wladimir J. van der Laan)
82dd8860bb qt: Periodical translations update (Wladimir J. van der Laan)

Pull request description:

  Need to merge this so that translations for 0.20 can start.

Top commit has no ACKs.

Tree-SHA512: c78307eea3130b9bbc301f23790340e4c86a3f3877ab308bf67d9e1c9e977f7dda4b2cd83b96d9398b01dce616a6c962b98115123628bba35170eb0f301c4dd8
2020-03-16 11:29:26 +01:00
Wladimir J. van der Laan
2a1b85f3c5 tx: Bump transifex slug to 020x 2020-03-16 10:52:55 +01:00
Wladimir J. van der Laan
82dd8860bb qt: Periodical translations update 2020-03-16 10:51:11 +01:00
Hennadii Stepanov
8a26848c46 build: Fix m4 escaping 2020-03-16 11:09:23 +02:00
Hennadii Stepanov
9123ec15db build: Remove extra tokens warning 2020-03-16 11:09:22 +02:00
Hennadii Stepanov
fded4f48c3 build: Remove duplicated QT_STATICPLUGIN define
QT_STATICPLUGIN is defined in BITCOIN_QT_CONFIGURE macro.
2020-03-16 11:09:22 +02:00
Hennadii Stepanov
05a93d5d96 build: Fix indentation in bitcoin_qt.m4 2020-03-16 11:09:22 +02:00
Hennadii Stepanov
ddbb419310 build: Use pkg-config in BITCOIN_QT_CONFIGURE for all hosts
This change adds to the BITCOIN_QT_CONFIGURE script ability to use
pkg-config for MinGW. All of the non-pkg-config paths are removed as
needless.
If depends is built with DEBUG=1 the configure script fails to pickup
Qt:
- for macOS host (similar, but not the same as issue 16391)
- for Windows host (regression)
2020-03-16 11:08:07 +02:00
João Barbosa
686c5456f2 Fix missing header in sync.h 2020-03-15 22:43:39 +00:00
Hennadii Stepanov
e4d366788b build: Drop needless EXTRA_DIST content
Some EXTRA_DIST content is needless since a git archive is used as the
source tarball.
2020-03-15 18:34:27 +02:00
practicalswift
44abf417eb tests: Add fuzzing harness for various functions taking std::string as input 2020-03-15 16:25:29 +00:00
practicalswift
d69145acb7 tests: Add fuzzing harness for MultiplicationOverflow(...) 2020-03-15 16:25:29 +00:00
practicalswift
7726f3bc46 tests: Add fuzzing harness for CFeeRate 2020-03-15 16:25:29 +00:00
practicalswift
0579a27630 tests: Add fuzzing harness for CBlockHeader 2020-03-15 15:19:50 +00:00
practicalswift
cb4eec13c0 tests: Add fuzzing harness for count_seconds(...) 2020-03-15 15:19:50 +00:00
MarcoFalke
d402c1e4d3 Merge #18345: test: Bump timeouts to avoid valgrind failures
fa19295250 test: Bump timeouts to avoid valgrind failures (MarcoFalke)

Pull request description:

  This should fix ci timeout issues such as:

  * https://travis-ci.org/github/bitcoin/bitcoin/jobs/661946972#L3109
  * https://travis-ci.org/github/bitcoin/bitcoin/jobs/662521570#L4922

ACKs for top commit:
  practicalswift:
    ACK fa19295250

Tree-SHA512: 150f804957575033a83fd47c68fd6adff2c11b110ec5803bd77f121256349805b595c39a3a5047738ec538d082ee38cebbcb6792c787ef22f56b8dd0d9618c1f
2020-03-15 10:14:40 -04:00
MarcoFalke
fa19295250 test: Bump timeouts to avoid valgrind failures 2020-03-14 22:15:52 -04:00
Wladimir J. van der Laan
67dfd18f44 Merge #16902: O(1) OP_IF/NOTIF/ELSE/ENDIF script implementation
e6e622e5a0 Implement O(1) OP_IF/NOTIF/ELSE/ENDIF logic (Pieter Wuille)
d0e8f4d5d8 [refactor] interpreter: define interface for vfExec (Anthony Towns)
89fb241c54 Benchmark script verification with 100 nested IFs (Pieter Wuille)

Pull request description:

  While investigating what mechanisms are possible to maximize the per-opcode verification cost of scripts, I noticed that the logic for determining whether a particular opcode is to be executed is O(n) in the nesting depth. This issue was also pointed out by Sergio Demian Lerner in https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/, and this PR implements a variant of the O(1) algorithm suggested there.

  This is not a problem currently, because even with a nesting depth of 100 (the maximum possible right now due to the 201 ops limit), the slowdown caused by this on my machine is around 70 ns per opcode (or 0.25 s per block) at worst, far lower than what is possible with other opcodes.

  This PR mostly serves as a proof of concept that it's possible to avoid it, which may be relevant in discussions around increasing the opcode limits in future script versions. Without it, the execution time of scripts can grow quadratically with the nesting depth, which very quickly becomes unreasonable.

  This improves upon #14245 by completely removing the `vfExec` vector.

ACKs for top commit:
  jnewbery:
    Code review ACK e6e622e5a0
  MarcoFalke:
    ACK e6e622e5a0 🐴
  fjahr:
    ACK e6e622e5a0
  ajtowns:
    ACK e6e622e5a0
  laanwj:
    concept and code review ACK e6e622e5a0
  jonatack:
    ACK e6e622e5a0 code review, build, benches, fuzzing

Tree-SHA512: 1dcfac3411ff04773de461959298a177f951cb5f706caa2734073bcec62224d7cd103767cfeef85cd129813e70c14c74fa8f1e38e4da70ec38a0f615aab1f7f7
2020-03-14 21:03:45 +01:00
Ben Woosley
d056df033a Replace std::to_string with locale-independent alternative 2020-03-14 12:23:01 -07:00
Samer Afach
7a6627ae87 Fix mining to an invalid target + ensure that a new block has the
correct hash internally in Python tests
2020-03-14 14:44:56 +01:00
Daniel Kraft
7df0cf719f Replace remaining literals BTC with CURRENCY_UNIT
This replaces one remaining instance of the literal "BTC" string with
the CURRENCY_UNIT constant, as is done in most of the codebase already.

The other remaining instance (which is just part of a log message and thus
not really user-visible) is just removed.

After this change, no instance of literal "BTC" remains anywhere in the
non-Qt and non-test codebase.
2020-03-14 09:24:21 +01:00
Wladimir J. van der Laan
7f8176a1eb Merge #18204: descriptors: improve descriptor cache and cache xpubs
09e25071f4 Cache parent xpub inside of BIP32PubkeyProvider (Andrew Chow)
deb791c7ba Only cache xpubs that have a hardened last step (Andrew Chow)
f76733eda5 Cache the immediate derivation parent xpub (Andrew Chow)
58f54b686f Add DescriptorCache* read_cache and DescriptorCache* write_cache to Expand and GetPubKey (Andrew Chow)
66c2cadc91 Rename BIP32PubkeyProvider.m_extkey to m_root_extkey (Andrew Chow)
df55d44d0d Track the index of the key expression in PubkeyProvider (Andrew Chow)
474ea3b927 Introduce DescriptorCache struct which caches xpubs (Andrew Chow)

Pull request description:

  Improves the descriptor cache by changing it from a `std::vector<unsigned char>` to a newly introduced `DescriptorCache` class. Instead of serializing pubkeys and whatever else we would want to cache in a way that may not be backwards compatible, we instead create a `DescriptorCache` object and populate it. This object contains only an xpub cache. Since the only `PubkeyProvider` that used the cache is the `BIP32PubkeyProvider` we just have it store the xpubs instead of the pubkeys. This allows us to have both the parent xpub and the child xpubs in the same container. The map is keyed by `KeyOriginInfo`.

  Sine we are caching `CExtPubKey`s in `DescriptorCache`, `BIP32PubKeyProviders` can use the cached parent xpubs to derive the children if unhardened derivation is used in the last step. This also means that we can still derive the keys for a `BIP32PubkeyProvider` that has hardened derivation steps. When combined with descriptor wallets, this should allow us to be able to import a descriptor with an `xprv` and hardened steps and still be able to derive from it. In that sense, this is an alternative to #18163

  To test that this works, the tests have been updated to do an additional `Expand` at the `i + 1` position. This expansion is not cached. We then do an `ExpandFromCache` at `i + 1` and use the cache that was produced by the expansion at `i`. This way, we won't have the child xpubs for `i + 1` but we will have the parent xpubs. So this checks whether the parent xpubs are being stored and can be used to derive the child keys. Descriptors that have a hardened last step are skipped for this part of the test because that will always require private keys.

ACKs for top commit:
  instagibbs:
    code review re-re-ACK 09e25071f4
  Sjors:
    re-ACK 09e25071f4

Tree-SHA512: 95c8d0092274cdf115ce39f6d49dec767679abf3758d5b9e418afc308deca9dc6f67167980195bcc036cd9c09890bbbb39ec1dacffbfacdc03efd72a7e23b276
2020-03-13 22:45:09 +01:00
Wladimir J. van der Laan
e5cb0dffd5 Merge #18002: Abstract out script execution out of VerifyWitnessProgram()
c8e24ddce3 [REFACTOR] Abstract out script execution out of VerifyWitnessProgram() (Pieter Wuille)

Pull request description:

  This is a refactoring cherry-picked out of #17977. As it touches consensus code, I don't think this would ordinarily meet the bar for review cost vs benefit. However, it simplifies the changes for Taproot significantly, and if it's going to be necessitated by inclusion of that code, I may as well give it some additional attention by PRing it independently.

ACKs for top commit:
  fjahr:
    Re-ACK c8e24ddce3
  theStack:
    re-ACK c8e24ddce3
  Empact:
    Code Review Re-ACK c8e24ddce3
  ajtowns:
    ACK c8e24ddce3
  jnewbery:
    ACK c8e24ddce3
  jonatack:
    ACK c8e24dd

Tree-SHA512: 96c2aa5d2f9c7c802bcc008f5cde55b1dfedfaf42e34101331e6c0d594acdf6437661102dc939718f0877c20451336855dfbaa8aa8f57d9e722a7fa7329e3a46
2020-03-13 22:30:21 +01:00
MarcoFalke
fac52253f8 rpc: Document an RPCResult for all calls; Enforce at compile time 2020-03-13 15:36:15 -04:00
MarcoFalke
fadd99f610 rpc: Add missing newline in RPCResult description 2020-03-13 15:35:41 -04:00
Steven Roose
fb15bfd99e Fix nit in getblockchaininfo 2020-03-13 14:18:24 +00:00
fanquake
3f9e6a3c1c Merge #18316: util: HelpExampleRpc formatting
a33cffbeab util: HelpExampleRpc formatting fixup (Jon Atack)

Pull request description:

  Minor visual fixup of the HelpExampleRpc template; conforms to the JSON-RPC spec as per https://www.jsonrpc.org/specification#examples. (I'm... somewhat embarassed to open such a minor change, but this is what is shown in all the CLI/RPC help docs.)

ACKs for top commit:
  laanwj:
    ACK a33cffbeab

Tree-SHA512: 8f1dee080c224742fff60a33fec6f5fb1d59c9fa51f3f2a67bf2e1837dbfa25f12a69e34518936588940013b0e61f55378b4f1a571c47c3cb081ca5b245e1091
2020-03-13 10:23:19 +08:00
practicalswift
100213c5c2 util: Fail to parse space-only strings in ParseMoney(...) (instead of parsing as zero) 2020-03-12 14:15:10 +00:00
MarcoFalke
fa7fea3654 refactor: Remove mempool global from net
This refactor does two things:
* Pass mempool in to PeerLogicValidation
* Pass m_mempool around where needed
2020-03-12 09:23:56 -04:00
MarcoFalke
e2d36639ca Merge #18228: test: Add missing syncwithvalidationinterfacequeue
faf6f156ff test: Add missing syncwithvalidationinterfacequeue (MarcoFalke)

Pull request description:

  The wallet rebroadcast functionality learns about new blocks via the validation interface queue. To avoid test failures such as https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/31119387#L466 , we can sync with the queue before advancing the test.

ACKs for top commit:
  jonatack:
    ACK faf6f156 this makes sense; the fix was previously added to mempool_persist.py and wallet_zapwallettxes.py in #12217 and to wallet_balance.py in #16302. It is also used in src/test/validation_block_tests.cpp (processnewblock_signals_ordering) and src/bench/wallet_balance.cpp.

Tree-SHA512: d72fd4b597b669d8111007902b523e946712913cd6eea6f9a695b0f04ecbe2321d05019873af999a95b9e0aa0f5c140a17109b37503723e40c9eab24ec358eb7
2020-03-12 09:20:50 -04:00
MarcoFalke
2737197ff3 Merge #18213: test: Fix race in p2p_segwit
fa2cf85e6f test: Fix race in p2p_segwit (MarcoFalke)

Pull request description:

  Fixes #11696

Top commit has no ACKs.

Tree-SHA512: 09de07ea26236547586f5c373a0df2b68d84af5cfa8f40bd2ca9f951fc083c5f4b8a472a60668d99118bbd9f3942ec3d6a34f05944d47345acca41c95475bb27
2020-03-12 09:15:43 -04:00
Sjors Provoost
e5327f947c [rpc] fundrawtransaction: add_inputs option to control automatic input adding 2020-03-12 13:07:17 +01:00
Sjors Provoost
79804fe24b [rpc] walletcreatefundedpsbt: don't automatically append inputs
When the user doesn't specificy inputs, it makes sense to automatically select them. But when the user does specify inputs, we now fail if the amount is insufficient, unless addInputs is set to true.
2020-03-12 13:07:17 +01:00
Hennadii Stepanov
6c4da59f5b build: Drop SOURCEDIST reordering
Making SOURCEDIST deterministic is needless since a git archive is used
as the source tarball.
2020-03-12 11:43:16 +02:00
fanquake
4c42a13205 Merge #18320: guix: Remove now-unnecessary gcc make flag
0ae42a16c7 guix: Remove now-unnecessary gcc make flag (Carl Dong)

Pull request description:

  ```
  Previously, Guix would produce a gcc which did not know to use the SSP
  function from glibc, and required a gcc make flag for it to do so, in my
  attempt to fix it upstream I realized that this is no longer the case.

  This can be verified by performing a Guix build and doing

    readelf -s ... | grep __stack_chk

  to check that symbols are coming from glibc, and doing

    readelf -d ... | grep NEEDED | grep ssp

  to see that libssp.so is not being depended on
  ```

ACKs for top commit:
  fanquake:
    ACK 0ae42a16c7 - ran a Guix build (hashes below) and checked all the linux binaries:

Tree-SHA512: 701b91e7c323b12a29af9539cb2656d10ce0a93af573a02e57f0b7fea05a6e1819798536eadb24d0a17e7f35b503f5e863fee5e7409db1b8a3973c4375e49d4e
2020-03-12 17:36:37 +08:00
Hennadii Stepanov
5e6b8b3912 build: Use git archive as source tarball 2020-03-12 11:34:43 +02:00
John Newbery
e57980b473 [mempool] Remove NotifyEntryAdded and NotifyEntryRemoved callbacks
NotifyEntryAdded never had any subscribers so can be removed.

Since ConnectTrace no longer subscribes to NotifyEntryRemoved, there are
now no subscribers.

The CValidationInterface TransactionAddedToMempool and
TransactionRemovedFromMempool methods can now provide this
functionality. There's no need for a special notifications framework for
the mempool.
2020-03-11 18:38:33 -04:00
John Newbery
2dd561f361 [validation] Remove pool member from ConnectTrace
It's no longer used for anything.
2020-03-11 18:38:33 -04:00
John Newbery
969b65f3f5 [validation] Remove NotifyEntryRemoved callback from ConnectTrace
ConnectTrace used to subscribe to the mempool's NotifyEntryRemoved
callback to be notified of transactions removed for conflict. Since
PerBlockConnectTrace no longer tracks conflicted transactions,
ConnectTrace no longer requires these notifications.
2020-03-11 18:38:33 -04:00
John Newbery
5613f9842b [validation] Remove conflictedTxs from PerBlockConnectTrace
Since we don't add a vtxConflicted vector to BlockConnected the
conflictedTxs member of PerBlockConnectTrace is no longer used.
2020-03-11 18:38:33 -04:00
John Newbery
cdb893443c [validation interface] Remove vtxConflicted from BlockConnected
The wallet now uses TransactionRemovedFromMempool to be notified about
conflicted wallet, and no other clients use vtxConflicted.
2020-03-11 18:38:33 -04:00
John Newbery
1168394d75 [wallet] Notify conflicted transactions in TransactionRemovedFromMempool
The only CValidationInterface client that cares about transactions that
are removed from the mempool because of CONFLICT is the wallet.

Start using the TransactionRemovedFromMempool method to notify about
conflicted transactions instead of using the vtxConflicted vector in
BlockConnected.
2020-03-11 18:38:27 -04:00
MarcoFalke
9cc7eba1b5 Merge #18318: test: Bump rpc timeout in feature_assumevalid to avoid valgrind timeouts
fa9b3040e7 test: Bump rpc timeout in feature_assumevalid to avoid valgrind timeouts (MarcoFalke)
fa72d270ad test: Bump walletpassphrase timeouts in wallet_createwallet to avoid valgrind timeouts (MarcoFalke)

Pull request description:

  Fixes:

  * https://travis-ci.org/github/bitcoin/bitcoin/jobs/661135188#L3137
  * https://travis-ci.org/github/bitcoin/bitcoin/jobs/661066901#L3137
  * https://travis-ci.org/github/bitcoin/bitcoin/jobs/661121674#L3828

ACKs for top commit:
  practicalswift:
    ACK fa9b3040e7

Tree-SHA512: 9c9f844da28c08d335145cd28da84bfd6dd81285eee7410fcf8e4b3707ff6f9bd8f4c60afaa0389dbebe4b1f3a4ad209d58d0d5b8739799cc25acd920ffb2404
2020-03-11 16:17:15 -04:00
Carl Dong
0ae42a16c7 guix: Remove now-unnecessary gcc make flag
Previously, Guix would produce a gcc which did not know to use the SSP
function from glibc, and required a gcc make flag for it to do so, in my
attempt to fix it upstream I realized that this is no longer the case.

This can be verified by performing a Guix build and doing

  readelf -s ... | grep __stack_chk

to check that symbols are coming from glibc, and doing

  readelf -d ... | grep NEEDED | grep ssp

to see that libssp.so is not being depended on
2020-03-11 16:16:31 -04:00
MarcoFalke
5c73645ac7 Merge #18319: fuzz: Add missing ECC_Start to key_io test
bbbbb53dd1 fuzz: Add missing ECC_Start to key_io test (MarcoFalke)

Pull request description:

  Fixes

  ```
  $ ./src/test/fuzz/key_io ../btc_qa_assets/fuzz_seed_corpus/key_io
  INFO: Seed: 2023332714
  INFO: Loaded 1 modules   (470791 inline 8-bit counters): 470791 [0x55d4f15d46e0, 0x55d4f16475e7),
  INFO: Loaded 1 PC tables (470791 PCs): 470791 [0x55d4f16475e8,0x55d4f1d76658),
  INFO:      203 files found in ../btc_qa_assets/fuzz_seed_corpus/key_io
  INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 4096 bytes
  INFO: seed corpus: files: 203 min: 1b max: 465b total: 16482b rss: 99Mb
  key.cpp:154:39: runtime error: null pointer passed as argument 1, which is declared to never be null
  secp256k1/include/secp256k1.h:521:3: note: nonnull attribute specified here
  SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior key.cpp:154:39 in

ACKs for top commit:
  practicalswift:
    ACK bbbbb53dd1

Tree-SHA512: a0deaf62489a8a6be610d9b23bbcc457f9e55494e9421b93b0e361f14dfa033f5c4dfabd3c760e01ae7735d910930dfd7e0981a33717d9c685bb311592457308
2020-03-11 15:57:14 -04:00
MarcoFalke
309b0c4c19 Merge #13693: [test] Add coverage to estimaterawfee and estimatesmartfee
111880aaf7 [test] Add coverage to estimaterawfee and estimatesmartfee (Ben Woosley)

Pull request description:

  This adds light functional coverage to estimaterawfee - a subset of
  the testing applied to estimatesmartfee, and argument validation
  testing to both estimaterawfee and estimatesmartfee.

  One valid estimatesmartfee signature test is commented out because it
  fails currently.

  Extracted from #12940

Top commit has no ACKs.

Tree-SHA512: 361a883457b28b2dc75081666e49d6dc6b5d76eed40d858abe2dd4f35ece152cf1f99c94480a91f42a896aa2a73cf55f57921316fe66970b2d7ba691a3b17e2d
2020-03-11 15:55:28 -04:00
MarcoFalke
bbbbb53dd1 fuzz: Add missing ECC_Start to key_io test 2020-03-11 15:16:54 -04:00
MarcoFalke
fa9b3040e7 test: Bump rpc timeout in feature_assumevalid to avoid valgrind timeouts 2020-03-11 13:37:17 -04:00
MarcoFalke
58c72880ff Merge #18268: rpc: Remove redundant types from descriptions
8a2a652e6f Remove redundant type information from rpc docs (David O'Callaghan)

Pull request description:

  Simple edit of the RPC calls to remove redundant text ("A json object/array ...") from the beginning of help.

  Fixes: #18258

Top commit has no ACKs.

Tree-SHA512: cbbf760e0b7b4eda61c40b420ed77f5d878318e37b0eb13e63567212240b2c4ecc15d84030e98075e21c9ae9016539adfd201e5661ea824166a76d335180c32f
2020-03-11 13:28:57 -04:00
MarcoFalke
fa72d270ad test: Bump walletpassphrase timeouts in wallet_createwallet to avoid valgrind timeouts 2020-03-11 13:23:59 -04:00
MarcoFalke
249114b1a6 Merge #18314: tests: Add deserialization fuzzing of SnapshotMetadata (utxo_snapshot). Increase fuzzing coverage.
08eab0f599 tests: Add fuzzing of CSubNet, CNetAddr and CService related functions (practicalswift)
7a861a62c1 tests: Fuzz HasAllDesirableServiceFlags(...) and MayHaveUsefulAddressDB(...) (practicalswift)
47a263108b tests: Fuzz DecodeBase64PSBT(...) (practicalswift)
d3d4892ef4 tests: Simplify code by removing unwarranted use of unique_ptr:s (practicalswift)
e57e67057a tests: Fuzz DecodeHexBlk(...) (practicalswift)
117a706fab tests: Fuzz RecursiveDynamicUsage(const std::shared_ptr<X>& p) (practicalswift)
81b58a3161 tests: Fuzz operator!= of CService (practicalswift)
c2c58f6f59 tests: Increase fuzzing coverage of DecompressScript(...) (practicalswift)
9f8d74a8c7 tests: Fuzz currently uncovered code path in TxToUniv(...) (practicalswift)
46ef4cfe5f tests: Re-arrange test cases in parse_univalue to increase coverage (practicalswift)
516cc6fc78 tests: Remove unit test from fuzzing harness (practicalswift)
7b169cae20 tests: Add deserialization fuzzing of SnapshotMetadata (utxo_snapshot), uint160 and uint256 (practicalswift)

Pull request description:

  Add deserialization fuzzing of `SnapshotMetadata` (`utxo_snapshot`).

  Increase fuzzing coverage.

ACKs for top commit:
  MarcoFalke:
    ACK 08eab0f599 🗾

Tree-SHA512: 5dca2316d64b9eb1da9bbbb3831de285b1524cbe815e3dba0f9c4eac7f39b403eb26ee0bdd3d9409a1838e7226d783946ec0d251e514a99f68267a95ac56d416
2020-03-11 13:02:43 -04:00
MarcoFalke
0eebe45cf7 Merge #18208: rpc: Change RPCExamples to bech32
3e32499909 Change example addresses to bech32 (Yusuf Sahin HAMZA)

Pull request description:

  This is a follow-up PR to #18197 that fixes RPCExamples.

  Fixes #18185.

ACKs for top commit:
  MarcoFalke:
    ACK 3e32499909
  jonatack:
    ACK 3e32499

Tree-SHA512: c7a6410ef8b6e169016c2c5eac3e6b9501caabd0e8a0871ec31e56bfc44589f056d3f5cb55b5a13bba36f6c15136c2352f883e30e4dcc0997ffd36b27f9173b9
2020-03-11 12:42:47 -04:00
Wladimir J. van der Laan
24a727e23e Merge #18255: test: Add bad-txns-*-toolarge test cases to invalid_txs
faae5a9a35 test: Add bad-txns-*-toolarge test cases to invalid_txs (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK faae5a9a35

Tree-SHA512: 93962de02104de220cc76f3759e7276423668bbd7f2b5c32e256ece2daf55501d72804bb9eb009a5d7b3a6631c88859cf6cc3e51da19dddf73b4e7df6e8c4ce4
2020-03-11 16:45:47 +01:00
Jon Atack
a33cffbeab util: HelpExampleRpc formatting fixup
to conform to json-rpc spec as per
https://www.jsonrpc.org/specification#examples
2020-03-11 16:32:15 +01:00
Wladimir J. van der Laan
3b21a78634 Merge #18290: build: Set minimum Automake version to 1.13
ddc7e42d60 build: Set minimum Automake version to 1.13 (Hennadii Stepanov)

Pull request description:

  This PR suggests to set the required minimum Automake version to `1.13` explicitly for the following reasons:
  - it guarantees that [CVE-2012-3386](https://lists.gnu.org/archive/html/automake/2012-07/msg00023.html) has been fixed
  - `AC_CONFIG_MACRO_DIR` macro support, which we already use; from the [release notes](https://lists.gnu.org/archive/html/automake/2012-12/msg00038.html):
  > Improvements to aclocal and related rebuilds rules:
  >    - Autoconf-provided macros AC_CONFIG_MACRO_DIR and AC_CONFIG_MACRO_DIRS are now traced by aclocal, and can be used to declare the local m4 include directories.  Formerly, one had to specify it with an explicit '-I' option to the 'aclocal' invocation.
  - `AM_SILENT_RULES` macro support (since version `1.11`)

  Automake `1.13` requires Autoconf `2.65` or greater. We already have `2.69` since #17769.

  ---

  For reference, Automake `1.13` was released in [December of 2012](https://lists.gnu.org/archive/html/automake/2012-12/msg00038.html).

  CentOS 7 uses Automake [`1.13.4`](https://centos.pkgs.org/7/centos-x86_64/automake-1.13.4-3.el7.noarch.rpm.html)

  See the Automake docs for more info:
  - [`AM_INIT_AUTOMAKE`](https://www.gnu.org/software/automake/manual/automake.html#Public-Macros)
  - [List of Automake options](https://www.gnu.org/software/automake/manual/automake.html#List-of-Automake-options)

ACKs for top commit:
  laanwj:
    so also ACK ddc7e42d60
  fanquake:
    ACK ddc7e42d60 - I think adding a minimum required version here is fine. I'd be surprised if someone who is currently building Bitcoin Core was unable to after this change.

Tree-SHA512: a1f97864bc3a513450c03d041498f28e823e6f8cd9710d81df081435d72bd4b6cd2f3deb997dbf902f950215a859e48a2ee7ca1f8ebf4271778dd951ab78abf4
2020-03-11 16:31:38 +01:00
Wladimir J. van der Laan
d20d5dc824 Merge #18285: test: Check that wait_until returns if time point is in the past
fab7d14ea5 test: Check that wait_until returns if time point is in the past (MarcoFalke)

Pull request description:

  Add an explicit regression test for the condvar bug (#18227), so that this doesn't happen again

ACKs for top commit:
  laanwj:
    ACK fab7d14ea5

Tree-SHA512: 6ec0d0b3945cae87a001e367af34cca1953a8082b4a0d9f8a20d30acd1f36363e98035d4eb173ff786cf6692d352d41f960633415c46394af042eb44e3b5ad71
2020-03-11 16:11:56 +01:00
Wladimir J. van der Laan
414f6c496d Merge #18283: doc: Explain rebase policy in CONTRIBUTING.md
fa1244783c doc: Explain rebase/squash policy in CONTRIBUTING.md (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    re-ACK fa1244783c, typos fixed.
  laanwj:
    ACK fa1244783c

Tree-SHA512: b8f790a8ffa4f59f26b64befbea8acd92f3c548e577c35750a60dde94b4340258a540ad0db60a6de650db0156cbff9e1330c589c300db5253531fe6a16cdbcc9
2020-03-11 16:01:25 +01:00
MarcoFalke
2926cbcebb Merge #18219: doc: Add warning against wallet.dat re-use
c1e0742308 doc: Warn about wallet.dat re-use and backups (Albert)

Pull request description:

  Following discussion in #18205, this PR adds a warning against re-use of the same wallet file on two different nodes, as that can cause problems due to race conditions between nodes (eg: both nodes using the same addresses at the same time for different things because they are not aware of the other node).

  I've also included the rationale behind the warning but I've kept it short to make it clearer to users, not sure if I should have written a longer explanation instead.

  Also, while this PR may help some users avoid problems, the changes are largely inconsequential, so feel free to close it if it's not worth the effort.

  On an unrelated note, I've also set up [this site](https://corollari.github.io/bitcoin-core-docs/), which periodically pulls bitcoin core and turns its docs into a webpage. Browsing the docs can also be done locally or on github, so this doesn't add much value, but I personally find that more comfortable and it makes them more searchable.

Top commit has no ACKs.

Tree-SHA512: 5ce06026176917304932714470be8c3410d35698f925875b0955ecd3b1756ef52793feb469dd4bdac4921f1a24daf59001e9911f1f096f559fb28c250baae378
2020-03-11 10:28:15 -04:00
MarcoFalke
377eed564e Merge #18170: doc: Minor grammatical changes and flow improvements
7ba962276e doc: Minor grammatical changes and flow improvements (Travin Keith)

Pull request description:

  **Grammar:**

  Line 49: There shouldn't be a period at the end of a phrase.

  Lines 56, 57, 116, 137, and 177: Adding necessary commas

  Lines 103 and 136: Run-on sentence issues fixed.

  Line 176: Fixed punctuation and added necessary conjunction

  Line 178: Singular noun when it should be plural

  **Flow:**

  Line 49: Adding "for" makes it more natural.

  Line 54: Though it's not grammatically incorrect to end a sentence with a preposition in an informal document such as this, the word "followed" is much easier to understand anyway, especially for those who don't have English as their native language.

Top commit has no ACKs.

Tree-SHA512: 17fdb5fc6146e025f160e860dbcbdbfa07b7608b8cb611c3b9d4ed91c426100ef772915251bc1f6bacb3a62df57b72c2003fb72cb2c8542454638545985313da
2020-03-11 10:25:47 -04:00
MarcoFalke
90b3e59caf Merge #17833: doc: Added running functional tests in valgrind
155a11f897 doc: Added running functional tests in valgrind (Elichai Turkel)

Pull request description:

  Technically the notes only show an "example" of how to run valgrind with the suppression file,
  but now that https://github.com/bitcoin/bitcoin/pull/17633 is merged then maybe this can encourage more people to run also the functional tests in valgrind

Top commit has no ACKs.

Tree-SHA512: b8417249b720d0ed5e10b732648f2e07e8889bfc7aa7e94192d1c049b4b7837971678d30c535f273c227848f1290cf11e14369fd6c1924b734f2e47e2af41401
2020-03-11 10:21:05 -04:00
MarcoFalke
b813fbe804 Merge #17159: doc: Add a note about backporting
2a6bce482c doc: Add a note about backporting (Carnhof Daki)

Pull request description:

  See laanwj's comment in #17158
  https://github.com/bitcoin/bitcoin/pull/17158#issuecomment-542627090

Top commit has no ACKs.

Tree-SHA512: ac5248a796050ce1a5bd0718955f941f6a3c025e192599948f12566eb55296079404b999676b9a2c8fe10616fc8334698dfa415af0fb4db6c98038d52218af1f
2020-03-11 10:17:43 -04:00
MarcoFalke
9c507a77f7 Merge #18310: doc: asmap release note
ac73afbcbf doc: asmap release note (Jon Atack)

Pull request description:

  Unsure how much detail to provide. Feedback welcome.

ACKs for top commit:
  laanwj:
    ACK ac73afbcbf
  naumenkogs:
    ACK ac73afb

Tree-SHA512: 6fd26f83084db913f2683ced799a4d50a26cd81cdf282bd0ae5364d2ab48b7508b358fd88e0aca5b31bbabb8a0015ca4aca49d690c136776d33b83ca95603393
2020-03-11 10:05:33 -04:00
practicalswift
08eab0f599 tests: Add fuzzing of CSubNet, CNetAddr and CService related functions 2020-03-11 12:51:26 +00:00
practicalswift
7a861a62c1 tests: Fuzz HasAllDesirableServiceFlags(...) and MayHaveUsefulAddressDB(...) 2020-03-11 12:51:26 +00:00
practicalswift
47a263108b tests: Fuzz DecodeBase64PSBT(...) 2020-03-11 12:51:26 +00:00
practicalswift
d3d4892ef4 tests: Simplify code by removing unwarranted use of unique_ptr:s 2020-03-11 12:51:26 +00:00
practicalswift
e57e67057a tests: Fuzz DecodeHexBlk(...) 2020-03-11 12:51:26 +00:00
practicalswift
117a706fab tests: Fuzz RecursiveDynamicUsage(const std::shared_ptr<X>& p) 2020-03-11 12:51:26 +00:00
practicalswift
81b58a3161 tests: Fuzz operator!= of CService 2020-03-11 12:51:26 +00:00
practicalswift
c2c58f6f59 tests: Increase fuzzing coverage of DecompressScript(...) 2020-03-11 12:51:26 +00:00
MarcoFalke
f1064c1b0d Merge #17989: tests: Add fuzzing harness for ProcessMessage(...). Enables high-level fuzzing of the P2P layer.
9220a0fdd0 tests: Add one specialized ProcessMessage(...) fuzzing binary per message type for optimal results when using coverage-guided fuzzing (practicalswift)
fd1dae10b4 tests: Add fuzzing harness for ProcessMessage(...) (practicalswift)

Pull request description:

  Add fuzzing harness for `ProcessMessage(...)`. Enables high-level fuzzing of the P2P layer.

  All code paths reachable from this fuzzer can be assumed to be reachable for an untrusted peer.

  Seeded from thin air (an empty corpus) this fuzzer reaches roughly 20 000 lines of code.

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/process_message
  …
  ```

  Worth noting about this fuzzing harness:
  * To achieve a reasonable number of executions per seconds the state of the fuzzer is unfortunately not entirely reset between `test_one_input` calls. The set-up (`FuzzingSetup` ctor) and tear-down (`~FuzzingSetup`) work is simply too costly to be run on every iteration. There is a trade-off to handle here between a.) achieving high executions/second and b.) giving the fuzzer a totally blank slate for each call. Please let me know if you have any suggestion on how to improve this situation while maintaining >1000 executions/second.
  * To achieve optimal results when using coverage-guided fuzzing I've chosen to create one specialised fuzzing binary per message type (`process_message_addr`, `process_message_block`, `process_message_blocktxn `, etc.) and one general fuzzing binary (`process_message`) which handles all messages types. The latter general fuzzer can be seeded with inputs generated by the former specialised fuzzers.

  Happy fuzzing friends!

ACKs for top commit:
  MarcoFalke:
    ACK 9220a0fdd0 🏊

Tree-SHA512: c314ef12b0db17b53cbf3abfb9ecc10ce420fb45b17c1db0b34cabe7c30e453947b3ae462020b0c9f30e2c67a7ef1df68826238687dc2479cd816f0addb530e5
2020-03-11 08:51:24 -04:00
MarcoFalke
16dfaed1b7 Merge #18304: ci: Enable all functional tests in valgrind
4444edc2e6 ci: Enable all functional tests in valgrind (MarcoFalke)

Pull request description:

  The travis timeout for our repo has been bumped to 2h, so we can run all tests in valgrind now

ACKs for top commit:
  practicalswift:
    ACK 4444edc2e6 -- regarding the three disabled cases (`feature_abortnode`, `feature_block` and `rpc_bind`): not a big deal since MSan will take care of those once #18288 is merged. More is more :)

Tree-SHA512: ea2f798112911b6d1f3d88cfcdf0a7cdb698687248343703d6fe55da144542c961c15d888bffb41672c10aa76765615cb7c7ff93d468bfad3c51f962f24e7abb
2020-03-11 08:39:09 -04:00
practicalswift
9f8d74a8c7 tests: Fuzz currently uncovered code path in TxToUniv(...) 2020-03-11 12:38:10 +00:00
practicalswift
46ef4cfe5f tests: Re-arrange test cases in parse_univalue to increase coverage 2020-03-11 12:37:59 +00:00
MarcoFalke
fa1244783c doc: Explain rebase/squash policy in CONTRIBUTING.md 2020-03-11 07:51:23 -04:00
practicalswift
516cc6fc78 tests: Remove unit test from fuzzing harness 2020-03-11 11:42:27 +00:00
practicalswift
7b169cae20 tests: Add deserialization fuzzing of SnapshotMetadata (utxo_snapshot), uint160 and uint256 2020-03-11 11:42:27 +00:00
practicalswift
9220a0fdd0 tests: Add one specialized ProcessMessage(...) fuzzing binary per message type for optimal results when using coverage-guided fuzzing 2020-03-11 06:57:55 +00:00
practicalswift
fd1dae10b4 tests: Add fuzzing harness for ProcessMessage(...) 2020-03-11 06:57:55 +00:00
MarcoFalke
4444edc2e6 ci: Enable all functional tests in valgrind 2020-03-10 18:38:42 -04:00
MarcoFalke
b5c7665e30 Merge #18311: Bumpfee test fix
f1b4503114 bumpfee test: exit loop at proper time with new fee value being compared (Gregory Sanders)
2e4edc68f9 Add some test logging to wallet_bumpfee.py (Gregory Sanders)

Pull request description:

  In the loop we accidentally used `origfee` which is not the value to check, and also allowed the loop to exit too early since the new fee must be strictly greater than `0.0005`.

  Also converted/added a bunch of logging from comments.

  Resolves https://github.com/bitcoin/bitcoin/issues/17716

ACKs for top commit:
  MarcoFalke:
    ACK f1b4503114 🏈

Tree-SHA512: eb73297fc82b09b9ec08d85ba3f0bec662119d0ff63ccf5d978a7bad6a674b5915f5ed021ec42f72a732c9ee7af43212d1de87361f50a970df7755caec96f6d8
2020-03-10 14:15:39 -04:00
Gregory Sanders
f1b4503114 bumpfee test: exit loop at proper time with new fee value being compared 2020-03-10 13:56:52 -04:00
Gregory Sanders
2e4edc68f9 Add some test logging to wallet_bumpfee.py 2020-03-10 13:56:52 -04:00
MarcoFalke
cf4cb28efc Merge #18305: test: Explain why test logging should be used
ffff9dcdc3 test: Explain why test logging should be used (MarcoFalke)

Pull request description:

  Background is that some tests don't have any `self.log` call at all. Thus there are no "anchor points" and those tests are hard to debug because the logs can't easily be parsed by a human.

ACKs for top commit:
  jonatack:
    ACK ffff9dcdc3
  instagibbs:
    ACK ffff9dcdc3
  fanquake:
    re-ACK ffff9dcdc3

Tree-SHA512: 08d962e85c4892c2a0c58feb5dc697c680a9d68e41a79417da6fcd415e0c5c735c4533a985cf225bb89deb5ca717d9bedf990657958079185804caa512b10f5a
2020-03-10 12:45:38 -04:00
Jon Atack
ac73afbcbf doc: asmap release note 2020-03-10 17:35:01 +01:00
MarcoFalke
fa3fa27c45 fuzz: Remove option --export_coverage from test_runner
The coverage statistics are not stable across clang versions
2020-03-10 11:15:11 -04:00
MarcoFalke
fa36f3a295 refactor: move DUMP_BANS_INTERVAL to banman.h 2020-03-10 09:52:53 -04:00
MarcoFalke
fadafb83cf scheduler: Make schedule* methods type safe 2020-03-10 09:47:32 -04:00
MarcoFalke
ffff9dcdc3 test: Explain why test logging should be used 2020-03-10 08:24:49 -04:00
fanquake
5d92ac26ed Merge #18264: build: Remove Boost Chrono
ad345909b2 doc: remove Boost Chrono from install docs (fanquake)
e21fa542b1 test: remove Boost Chrono installation from CI (fanquake)
bd37f2bc26 build: remove Boost Chrono detection from build system (fanquake)
1d0a87e712 build: remove chrono package from depends Boost (fanquake)

Pull request description:

  We no longer use Boost Chrono.

ACKs for top commit:
  practicalswift:
    ACK ad345909b2
  MarcoFalke:
    ACK ad345909b2
  kallewoof:
    ACK ad345909b2

Tree-SHA512: d987ab5461c76c982318c65ec8d625094356716b79fd3a462beea75f07db0f82608ace13ec4c4b0233f352508715a4505ac2b4ed1c1e9d9d78f0da936b80f0f3
2020-03-10 20:24:10 +08:00
MarcoFalke
1a97c9a483 Merge #15443: qa: Add getdescriptorinfo functional test
cbf2d75d8f qa: Add getdescriptorinfo functional test (João Barbosa)

Pull request description:

  The `getdescriptorinfo` RPC was added in #15368, this PR adds some tests.

Top commit has no ACKs.

Tree-SHA512: 5bf3fb5842b975089821c7ac52202ecb23df255f655862646eb532e38e335ff963f8973bcf5b8bba386183281dc9bfe7279ba1cf25fd518c9a45fb45a9243e4d
2020-03-10 08:17:23 -04:00
João Barbosa
cbf2d75d8f qa: Add getdescriptorinfo functional test 2020-03-10 08:14:04 +00:00
Bushstar
7486e2771e Tests: Unit test related to WalletDB ReadKeyValue 2020-03-10 07:42:31 +00:00
Peter Bushnell
32def8d1c2 Catch ios_base::failure specifically 2020-03-10 06:05:16 +00:00
MarcoFalke
6fb4bbfb0e Merge #18176: tests: Add fuzzing harness for CScript and CScriptNum operations
e37f53648e Make lifetime correctness easier to see (avoid reference lifetime extension) (practicalswift)
e7ddbd9893 tests: Add fuzzing harness for CScriptNum operations (practicalswift)
65a52a0024 tests: Add fuzzing harness for CScript operations (practicalswift)
eb7c50ca1f tests: Add common Consume* fuzzing functions (practicalswift)

Pull request description:

  Add fuzzing harness for `CScript` and `CScriptNum` operations.

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/script_ops
  …
  $ src/test/fuzz/scriptnum_ops
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK e37f53648e 🦂

Tree-SHA512: 5165d918ffe3f1e3e85ab0e61d8b05934f682d324cf63ce188da5890899df2b5727aba9ed10c0437260ecff8055250e60c79d81d764bc740a7652d543a7c5fa3
2020-03-09 23:01:31 -04:00
practicalswift
e37f53648e Make lifetime correctness easier to see (avoid reference lifetime extension) 2020-03-09 20:39:48 +00:00
practicalswift
e7ddbd9893 tests: Add fuzzing harness for CScriptNum operations 2020-03-09 20:39:48 +00:00
MarcoFalke
ccb2c9e789 Merge #18306: test: add logging to wallet_listsinceblock.py
d484279a46 test: add logging to wallet_listsinceblock.py (Jon Atack)

Pull request description:

  This is the first commit from #17535.

Top commit has no ACKs.

Tree-SHA512: bb4f527a41bca3ffbf69e910311ce7f85dcc7a2be41350b3c653a27f4044f392b7e528f330e9691f497212469f6b16ce263230bb7a919548dd4e3e21cc72142f
2020-03-09 16:16:21 -04:00
Jon Atack
d484279a46 test: add logging to wallet_listsinceblock.py 2020-03-09 21:12:03 +01:00
Samuel Dobson
dcf2ccbfde Merge #18115: wallet: Pass in transactions and messages for signing instead of exporting the private keys
d2774c09cf Clear any input_errors for an input after it is signed (Andrew Chow)
dc174881ad Replace GetSigningProvider with GetSolvingProvider (Andrew Chow)
6a9c429084 Move direct calls to MessageSign into new SignMessage functions in CWallet and ScriptPubKeyMan (Andrew Chow)
82a30fade7 Move key and script filling and signing from CWallet::FillPSBT to ScriptPubKeyMan::FillPSBT (Andrew Chow)
3d70dd99f9 Move FillPSBT to be a member of CWallet (Andrew Chow)
a4af324d15 Use CWallet::SignTransaction in CreateTransaction and signrawtransactionwithwallet (Andrew Chow)
f37de92744 Implement CWallet::SignTransaction using ScriptPubKeyMan::SignTransaction (Andrew Chow)
d999dd588c Add SignTransaction function to ScriptPubKeyMan and LegacyScriptPubKeyMan (Andrew Chow)
2c52b59d0a Refactor rawtransaction's SignTransaction into generic SignTransaction function (Andrew Chow)

Pull request description:

  Following #17261, the way to sign transactions, PSBTs, and messages was to use `GetSigningProvider()` and get a `SigningProvider` containing the private keys. However this may not be feasible for future `ScriptPubKeyMan`s, such as for hardware wallets. Instead of exporting a `SigningProvider` containing private keys, we need to pass these things into the `ScriptPubKeyMan` (via `CWallet`) so that they can do whatever is needed internally to sign them. This is largely a refactor as the logic of processing transactions, PSBTs, and messages for is moved into `LegacyScriptPubKeyMan` and `CWallet` instead of being handled by the caller (e.g. `signrawtransaction`).

  To help with this, I've refactored the 3(!) implementations of a `SignTransaction()` function into one generic one. This function will be called by `signrawtransactionwithkey` and `LegacyScriptPubKeyMan::SignTransaction()`. `CWallet::CreateTransaction()` is changed to call `CWallet::SignTransaction()` which in turn, calls `LegacyScriptPubKeyMan::SignTransaction()`. Other `ScriptPubKeyMan`s may implement `SignTransaction()` differently.

  `FillPSBT()` is moved to be a member function of `CWallet` and the `psbtwallet.cpp/h` files removed. It is further split so that `CWallet` handles filling the UTXOs while the `ScriptPubKeyMan` handles adding keys, derivation paths, scripts, and signatures. In the end `LegacyScriptPubKeyMan::FillPSBT` still calls `SignPSBTInput`, but the `SigningProvider` is internal to `LegacyScriptPubKeyMan`. Other `ScriptPubKeyMan`s may do something different.

  A new `SignMessage()` function is added to both `CWallet` and `ScriptPubKeyMan`. Instead of having the caller (i.e. `signmessage` or the sign message dialog) get the private key, hash the message, and sign, `ScriptPubKeyMan` will now handle that (`CWallet` passes through to the `ScriptPubKeyMan`s as it does for many functions). This signing code is thus consolidated into `LegacyScriptPubKeyMan::SignMessage()`, though other `ScriptPubKeyMan`s may implement it differently. Additionally, a `SigningError` enum is introduced for the different errors that we expect to see from `SignMessage()`.

  Lastly, `GetSigningProvider()` is renamed to `GetPublicSigningProvider()`. It will now only provide pubkeys, key origins, and scripts. `LegacySigningProvider` has it's `GetKey` and `HaveKey` functions changed to only return false. Future implementations should return `HidingSigningProvider`s where private keys are hidden.

  Other things like `dumpprivkey` and `dumpwallet` are not changed because they directly need and access the `LegacyScriptPubKeyMan` so are not relevant to future changes.

ACKs for top commit:
  instagibbs:
    reACK d2774c09cf
  Sjors:
    re-utACK d2774c09cf
  meshcollider:
    re-utACK d2774c09cf

Tree-SHA512: 89c83e7e7e9315e283fae145a2264648a9d7f7ace8f3281cb3f44f0b013c988d67ba4fa9726e50c643c0ed921bdd269adaec984840d11acf4a681f3e8a582cc1
2020-03-10 09:02:12 +13:00
Wladimir J. van der Laan
ffdde475b0 Merge #18303: doc: Add historical release notes for 0.19.1
46a1ee7732 doc: Add historical release notes for 0.19.1 (Wladimir J. van der Laan)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 9ff578fbaec5694d63256d65cbe7e9d0ba99ac50803c8192f9b5e90a614dc89778bdb17b03f5f7725397f820e085e7d704deb1d6a58b18cd4e4fc91ea413b6ae
2020-03-09 20:54:19 +01:00
practicalswift
65a52a0024 tests: Add fuzzing harness for CScript operations 2020-03-09 19:24:50 +00:00
practicalswift
eb7c50ca1f tests: Add common Consume* fuzzing functions 2020-03-09 19:24:50 +00:00
MarcoFalke
5518eeec27 Merge #18047: tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h)
6590395f60 tests: Remove FUZZERS_MISSING_CORPORA (practicalswift)
815c7a6793 tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) (practicalswift)

Pull request description:

  Add basic fuzzing harness for `CNetAddr`/`CService`/`CSubNet` related functions (`netaddress.h`).

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/netaddress
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 69dc0e391d56d5e9cdb818ac0ac4b69445d0195f714442a06cf662998e38b6e0bbaa635dce78df37ba797feed633e94abba4764b946c1716d392756e7809112d
2020-03-09 13:53:46 -04:00
Albert
c1e0742308 doc: Warn about wallet.dat re-use and backups
See discussion in #18205 and https://github.com/bitcoin/bitcoin/pull/18219#issuecomment-594871230.
2020-03-10 01:51:03 +08:00
Wladimir J. van der Laan
46a1ee7732 doc: Add historical release notes for 0.19.1 2020-03-09 18:41:39 +01:00
practicalswift
6590395f60 tests: Remove FUZZERS_MISSING_CORPORA 2020-03-09 17:20:52 +00:00
MarcoFalke
be00ef2aca Merge #18292: fuzz: Add assert(script == decompressed_script)
fab0e5ba7f fuzz: Add assert(script == decompressed_script) (MarcoFalke)

Pull request description:

  Presumably an oversight in https://github.com/bitcoin/bitcoin/pull/17926#discussion_r389262151

ACKs for top commit:
  practicalswift:
    Tested ACK fab0e5ba7f

Tree-SHA512: 6dcec06169df497a540fd6ebbcd89f5db22257241b2bbe756de868742f9bc324b80d38dbababfa07e5f3a830aaae9fc6d168dcc2ca5d75da437bdf4dc4e0f370
2020-03-09 11:19:29 -04:00
practicalswift
815c7a6793 tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) 2020-03-09 15:16:36 +00:00
Andrew Chow
d2774c09cf Clear any input_errors for an input after it is signed
Make sure that there are no errors set for an input after it is signed.
This is useful for when there are multiple ScriptPubKeyMans. Some may
fail to sign, but one may be able to sign, and after it does, we don't
want there to be any more errors there.
2020-03-09 11:16:20 -04:00
Andrew Chow
dc174881ad Replace GetSigningProvider with GetSolvingProvider
Not all ScriptPubKeyMans will be able to provide private keys,
but pubkeys and scripts should be. So only provide public-only
SigningProviders, i.e. ones that can help with Solving.
2020-03-09 11:16:20 -04:00
Andrew Chow
6a9c429084 Move direct calls to MessageSign into new SignMessage functions in CWallet and ScriptPubKeyMan
Instead of getting a SigningProvider and then going to MessageSign,
have ScriptPubKeyMan handle the message signing internally.
2020-03-09 11:16:20 -04:00
Andrew Chow
82a30fade7 Move key and script filling and signing from CWallet::FillPSBT to ScriptPubKeyMan::FillPSBT
Instead of fetching a SigningProvider from ScriptPubKeyMan in order
to fill and sign the keys and scripts for a PSBT, just pass that
PSBT to a new FillPSBT function that does all that for us.
2020-03-09 11:16:20 -04:00
Andrew Chow
3d70dd99f9 Move FillPSBT to be a member of CWallet 2020-03-09 11:16:17 -04:00
MarcoFalke
aaaa055ff7 fuzz: Add option to merge input dir to test runner 2020-03-09 11:13:57 -04:00
MarcoFalke
fa4fa88d76 doc: Remove --disable-ccache from docs 2020-03-09 11:13:48 -04:00
MarcoFalke
31b2ce70fb Merge #18299: tests: Update FUZZERS_MISSING_CORPORA to enable regression fuzzing for all harnesses in master
bf06641819 tests: Reset FUZZERS_MISSING_CORPORA to enable regression fuzzing for more harnesses (practicalswift)

Pull request description:

  Update `FUZZERS_MISSING_CORPORA` to enable regression fuzzing for all harnesses in `master`.

  Note that this PR is expected to fail in Travis until https://github.com/bitcoin-core/qa-assets/pull/4 is merged :)

Top commit has no ACKs.

Tree-SHA512: 6b4d23726ca99c7ab9454087f0354e35d307e7e16b90741cdf7838265a07a631cf54d9533b4f1bf9a0fd900c5ed909756a35cab738284bf39b09ab1c2ed54368
2020-03-09 11:05:52 -04:00
practicalswift
bf06641819 tests: Reset FUZZERS_MISSING_CORPORA to enable regression fuzzing for more harnesses 2020-03-09 14:30:18 +00:00
fanquake
6ddf435493 Merge #18274: rpc/wallet: initialize nFeeRequired to avoid using garbage value on failure
a652ba6293 rpc/wallet: initialize nFeeRequired to avoid using garbage value on failure (Karl-Johan Alm)

Pull request description:

  Initialize the `nFeeRequired` variable to avoid using an uninitialized value for errors happening before it is set to 0.

  Note: this originally fixed `nFeeRet` in `wallet.cpp`.

ACKs for top commit:
  promag:
    ACK a652ba6293.
  Sjors:
    utACK a652ba6293
  practicalswift:
    ACK a652ba6293 -- patch looks correct
  meshcollider:
    utACK a652ba6293

Tree-SHA512: 0d12f1ffd0851ed5ce6d109d2c87f55e8b1d57da297e684feeabb57229200c4078f029c55ca5aa5712bd18e26dda3ce538443dfe68a7a6d504428068f81fded0
2020-03-09 19:17:51 +08:00
Karl-Johan Alm
a652ba6293 rpc/wallet: initialize nFeeRequired to avoid using garbage value on failure 2020-03-09 10:20:41 +09:00
Hennadii Stepanov
492971de35 build: Fix mingw pkgconfig file and dependency naming
This change adds the correct suffix to debug mode .pc filenames for
MinGW and also to the Qt libraries listed in the `Requires` field.
The filename adjustment fixes the accidental overwriting of release
mode .pc files with the debug mode variant which required the wrong
variant of the libraries when `debug_and_release` is active.
Note that macOS also supports the `debug_and_release' configuration
but may use the regular library names together with DYLD_IMAGE_SUFFIX.
Creation of *_debug.pc files is turned off as they're identical to their
non-debug counterparts.
More info:
- QTBUG-4155
- Qt commit a0d8fb4ac3cb7bafdb39f340055eacee4f957513
2020-03-09 02:02:31 +02:00
Andrew Chow
a4af324d15 Use CWallet::SignTransaction in CreateTransaction and signrawtransactionwithwallet
Instead of duplicating signing code, just use the function we already
have.
2020-03-08 12:27:05 -04:00
Andrew Chow
f37de92744 Implement CWallet::SignTransaction using ScriptPubKeyMan::SignTransaction 2020-03-08 12:26:32 -04:00
Andrew Chow
d999dd588c Add SignTransaction function to ScriptPubKeyMan and LegacyScriptPubKeyMan 2020-03-08 12:26:32 -04:00
Andrew Chow
2c52b59d0a Refactor rawtransaction's SignTransaction into generic SignTransaction function 2020-03-08 12:26:27 -04:00
MarcoFalke
fab0e5ba7f fuzz: Add assert(script == decompressed_script) 2020-03-07 16:55:34 -05:00
MarcoFalke
5e12a61044 Merge #17926: tests: Add key_io fuzzing harness. Fuzz additional functions in existing fuzzing harnesses.
52fed696d2 tests: Fuzz additional functions in the script fuzzing harness (practicalswift)
5fc10f3cb5 tests: Fuzz additional functions in the transaction fuzzing harness (practicalswift)
1d324ce922 tests: Fuzz additional functions in the integer fuzzing harness (practicalswift)
4fe4de6364 tests: Fuzz additional functions in the hex fuzzing harness (practicalswift)
c7ea12d098 tests: Add key_io fuzzing harness (practicalswift)

Pull request description:

  Add `key_io` fuzzing harness.

  Fuzz additional functions in the `hex` fuzzing harness.

  Fuzz additional functions in the `integer` fuzzing harness.

  Fuzz additional functions in the `script` fuzzing harness.

  Fuzz additional functions in the `transaction` fuzzing harness.

  **How to test this PR**

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/key_io
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 52fed696d2 🛫

Tree-SHA512: a57ab66c18d260c2e39d987cab9fa576f7a5520dc1ea7fd607d64d8e005e16558312ddb4c9f4d4f3147dc6194d8ae0b0fb86ed5e58ba6aef5383ea726463df97
2020-03-07 11:19:52 -05:00
Hennadii Stepanov
ddc7e42d60 build: Set minimum Automake version to 1.13 2020-03-07 17:38:06 +02:00
Andrew Chow
09e25071f4 Cache parent xpub inside of BIP32PubkeyProvider
Optimize Expand by having BIP32PubkeyProvider also cache the parent
(or only) xpub within itself. Since Expand does not provide a read
cache, it is useful to internally cache this xpub to avoid re-deriving
the same xpub.
2020-03-07 10:13:47 -05:00
Andrew Chow
deb791c7ba Only cache xpubs that have a hardened last step
Also adds tests for this:
For ranged descriptors with unhardened derivation, we expect to
find parent keys in the cache but no child keys.

For descriptors containing an xpub but do not have unhardened derivation
(i.e. hardened derivation or single xpub with or without derivation),
we expect to find all of the keys in the cache, and the same
number of keys in the cache as in the SigningProvider.

For everything else (no xpub), nothing should be cached at all.
2020-03-07 10:13:47 -05:00
Andrew Chow
f76733eda5 Cache the immediate derivation parent xpub
If unhardened derivation is used, cache the immediate derivation
parent xpub and use it for unhardened derivation
2020-03-07 10:13:47 -05:00
Andrew Chow
58f54b686f Add DescriptorCache* read_cache and DescriptorCache* write_cache to Expand and GetPubKey
Have Expand, ExpandFromCache, and ExpandHelper take additional DescriptorCache
parameters. These are then passed into PubkeyProvider::GetPubKey which
also takes them as arguments.

Reading and writing to the cache is pushed down into GetPubKey. The old cache where
pubkeys are serialized to a vector is completely removed and instead xpubs are being
cached in DescriptorCache.
2020-03-07 10:13:47 -05:00
Andrew Chow
66c2cadc91 Rename BIP32PubkeyProvider.m_extkey to m_root_extkey
Renaming clarifies that m_extkey is actually the root
extkey that keys are derived from.
2020-03-07 10:13:47 -05:00
Andrew Chow
df55d44d0d Track the index of the key expression in PubkeyProvider 2020-03-07 10:13:47 -05:00
Andrew Chow
474ea3b927 Introduce DescriptorCache struct which caches xpubs 2020-03-07 10:13:43 -05:00
practicalswift
52fed696d2 tests: Fuzz additional functions in the script fuzzing harness 2020-03-07 14:35:49 +00:00
practicalswift
5fc10f3cb5 tests: Fuzz additional functions in the transaction fuzzing harness 2020-03-07 14:35:49 +00:00
MarcoFalke
fa70ccc6c4 scheduler: Use C++11 member initialization, add shutdown assert
"Initializing the members in the declaration makes it easy to spot
uninitialized ones".
https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#c-data-structures
2020-03-07 08:58:38 -05:00
practicalswift
1d324ce922 tests: Fuzz additional functions in the integer fuzzing harness 2020-03-07 13:40:19 +00:00
practicalswift
4fe4de6364 tests: Fuzz additional functions in the hex fuzzing harness 2020-03-07 13:39:25 +00:00
practicalswift
c7ea12d098 tests: Add key_io fuzzing harness 2020-03-07 13:39:25 +00:00
fanquake
ad345909b2 doc: remove Boost Chrono from install docs 2020-03-07 08:34:31 +08:00
fanquake
e21fa542b1 test: remove Boost Chrono installation from CI 2020-03-07 08:34:31 +08:00
fanquake
bd37f2bc26 build: remove Boost Chrono detection from build system 2020-03-07 08:34:31 +08:00
fanquake
1d0a87e712 build: remove chrono package from depends Boost 2020-03-07 08:34:30 +08:00
fanquake
3d28c886f0 Merge #18286: build: Add locale fuzzer to FUZZERS_MISSING_CORPORA (and unbreak Travis! :))
0d0bc3b5c1 build: Add locale fuzzer to FUZZERS_MISSING_CORPORA (practicalswift)

Pull request description:

  Add `locale` fuzzer to `FUZZERS_MISSING_CORPORA`.

  This is a follow-up to #18126 which broke Travis. Sorry about that :)

ACKs for top commit:
  fanquake:
    ACK 0d0bc3b5c1

Tree-SHA512: c0968dc798839f87c891d1dfccf5541883ac56b51a29f52244e78c221c9c087d2dea0a959612d907d53b29fca1f486b340227b17653227ecbf6ca5ab0e85b0d3
2020-03-07 08:04:53 +08:00
practicalswift
0d0bc3b5c1 build: Add locale fuzzer to FUZZERS_MISSING_CORPORA 2020-03-06 23:29:23 +00:00
fanquake
4d80274b99 Merge #18241: wallet/refactor: refer to CWallet immutably when possible
79facb11e9 wallet: use constant CWallets in rpcwallet.cpp (Karl-Johan Alm)
d9b0ebc1da wallet: make ReserveDestination pwallet ivar const (Karl-Johan Alm)
57c569e4d9 wallet: make BackupWallet() const (Karl-Johan Alm)
df3a818d2a wallet: make getters const (Karl-Johan Alm)
227b9dd2d6 wallet/spkm: make GetOldestKeyPoolTime() const (Karl-Johan Alm)
22d329ad0e wallet: use constant CWallets in rpcdump.cpp (Karl-Johan Alm)
7b3587b29d wallet/db: make IsDummy() const (Karl-Johan Alm)
d366795d18 wallet/db: make Backup() const (Karl-Johan Alm)
8cd0b86340 wallet: make CanGetAddresses() const (Karl-Johan Alm)
037fa770eb wallet: make KeypoolCountExternalKeys() const (Karl-Johan Alm)
ddc93557ad wallet: make CanGenerateKeys() const (Karl-Johan Alm)
dc2d0650fd make BlockUntilSyncedToCurrentChain() const (Karl-Johan Alm)

Pull request description:

  A lot of places refer to `CWallet*`'s as `CWallet * const`, which translates to *"an immutable pointer to a mutable `CWallet` instance"*; this is

  1. often not what the author meant, especially as a lot of these places do not at all modify the wallet object, and
  2. confusing, as it tends to suggest that this is a proper way to refer to a constant `CWallet` instance.

  This PR changes references to wallets to `const CWallet* const` whenever immutability is expected. This should result in no behavioral changes at all, and improved compile-time error checking.

  Note from irc:

  > &lt;sipa&gt; sounds good to me; this is the sort of change that as long as it compiles, the behavior shouldn't change
  > &lt;sipa&gt; though in general it may lead to introducing automatic copying of objects sometimes (e.g. trying to std::move a const object will work, but generally result in a copy rather than an efficient move)
  > &lt;sipa&gt; CWallet objects aren't copied or moved though

ACKs for top commit:
  laanwj:
    ACK 79facb11e9
  Empact:
    ACK 79facb11e9
  promag:
    ACK 79facb11e9.
  fjahr:
    ACK 79facb11e9

Tree-SHA512: 80a80c1a52f0f788d0ccb268b53bc0f46c796643a3c5a22b55bbbde4ffa6c7e347784e5e53b1e488a3b4e14399e31d5be9417ad5b6319c74a462609e9b1a98e8
2020-03-07 07:24:54 +08:00
MarcoFalke
fab7d14ea5 test: Check that wait_until returns if time point is in the past 2020-03-06 16:08:12 -05:00
Wladimir J. van der Laan
3516a31eaa Merge #18234: refactor: Replace boost::mutex,condition_var,chrono with std equivalents in scheduler
70a6b529f3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR (Anthony Towns)
294937b39d scheduler_tests: re-enable mockforward test (Anthony Towns)
cea19f6859 Drop unused reverselock.h (Anthony Towns)
d0ebd93270 scheduler: switch from boost to std (Anthony Towns)
b9c4260127 sync.h: add REVERSE_LOCK (Anthony Towns)
306f71b4eb scheduler: don't rely on boost interrupt on shutdown (Anthony Towns)

Pull request description:

  Replacing boost functionality with C++11 stuff.

  Motivated by #18227, but should stand alone. Changing from `boost::condition_var` to `std::condition_var` means `threadGroup.interrupt_all` isn't enough to interrupt `serviceQueue` anymore, so that means calling `stop()` before `join_all()` is needed. And the existing reverselock.h code doesn't work with sync.h's DebugLock code (because the reversed lock won't be removed from `g_lockstack` which then leads to incorrect potential deadlock warnings), so I've replaced that with a dedicated class and macro that's aware of our debug lock behaviour.

  Fixes #16027, Fixes #14200, Fixes #18227

ACKs for top commit:
  laanwj:
    ACK 70a6b529f3

Tree-SHA512: d1da13adeabcf9186d114e2dad9a4fdbe2e440f7afbccde0c13dfbaf464efcd850b69d3371c5bf8b179d7ceb9d81f4af3cc22960b90834e41eaaf6d52ef7d331
2020-03-06 20:51:56 +01:00
MarcoFalke
45cdcd47d9 Merge #18126: tests: Add fuzzing harness testing the locale independence of the strencodings.h functions
259e290db8 tests: Add fuzzing harness for locale independence testing (practicalswift)

Pull request description:

  Context: [C and C++ locale assumptions in bitcoind and bitcoin-qt](https://github.com/bitcoin/bitcoin/pull/18124)

  Add fuzzing harness for locale independence testing of functions in `strencodings.h` and `tinyformat.h`.

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/locale
  …
  ```

  The tested functions (`ParseInt32(…)`, `ParseInt64(…)`, `atoi(const std::string&)`, `atoi64(const std::string& str)`, `i64tostr(const char*)`, `itostr(…)`, `strprintf(…)`) all call locale dependent functions (such as `strtol(…)`, `strtoll(…)`, `atoi(const char*)`, etc.) but are assumed to do so in a way that the tested functions return same results regardless of the chosen C locale (`setlocale`).

  This fuzzer aims to test that those assumptions hold up also in practice now and over time.

Top commit has no ACKs.

Tree-SHA512: d108d2f85aa6f482839dafbc7579465ffd4bacf7bc52835ad0fbaa1c71aed9b3870c83447b3d453a03b9ce307e76a3cfdd350a0c77024ab094c93c7d62c8a527
2020-03-06 14:31:11 -05:00
Anthony Towns
70a6b529f3 lint-cppcheck: Remove -DHAVE_WORKING_BOOST_SLEEP_FOR 2020-03-07 04:20:41 +10:00
practicalswift
259e290db8 tests: Add fuzzing harness for locale independence testing 2020-03-06 13:29:21 +00:00
Anthony Towns
294937b39d scheduler_tests: re-enable mockforward test 2020-03-06 23:14:10 +10:00
Anthony Towns
cea19f6859 Drop unused reverselock.h 2020-03-06 23:14:10 +10:00
Anthony Towns
d0ebd93270 scheduler: switch from boost to std
Changes from boost::chrono to std::chrono, boost::condition_var to
std::condition_var, boost::mutex to sync.h Mutex, and reverselock.h to
sync.h REVERSE_LOCK. Also adds threadsafety annotations to CScheduler
members.
2020-03-06 23:14:08 +10:00
Anthony Towns
b9c4260127 sync.h: add REVERSE_LOCK 2020-03-06 23:13:31 +10:00
Anthony Towns
306f71b4eb scheduler: don't rely on boost interrupt on shutdown
Calling interrupt_all() will immediately stop the scheduler, so it's
safe to invoke stop() beforehand, and this removes the reliance on boost
to interrupt serviceQueue().
2020-03-06 23:13:31 +10:00
fanquake
97aadf98d0 Merge #16117: util: Replace boost sleep with std sleep
fae86c38bc util: Remove unused MilliSleep (MarcoFalke)
fa9af06d91 scripted-diff: Replace MilliSleep with UninterruptibleSleep (MarcoFalke)
fa4620be78 util: Add UnintrruptibleSleep (MarcoFalke)

Pull request description:

  We don't use the interruptible feature of boost's sleep anywhere, so replace it with the sleep in `std::thread`

ACKs for top commit:
  ajtowns:
    ACK fae86c38bc quick code review
  practicalswift:
    ACK fae86c38bc -- patch looks correct
  sipa:
    Concept and code review ACK fae86c38bc
  fanquake:
    ACK fae86c38bc - note that an instance of `DHAVE_WORKING_BOOST_SLEEP_FOR` was missed in the [linter](https://github.com/bitcoin/bitcoin/blob/master/test/lint/extended-lint-cppcheck.sh#L69), but that can be cleaned up later.

Tree-SHA512: 7c0f8eb197664b9f7d9fe6c472c77d384f11c797c913afc31de4b532e3b4fd9ea6dd174f92062ff9d1ec39b25e0900ca7c597435add87f0f2477d9557204848c
2020-03-06 15:41:00 +08:00
MarcoFalke
3f826598a4 Merge #17972: tests: Add fuzzing harness for CKey and key related functions
f4691b6c21 tests: Add fuzzing harness for CKey related functions (practicalswift)

Pull request description:

  Add fuzzing harness for `CKey` and key related functions.

  **How to test this PR**

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/key
  …
  #4096   pulse  cov: 5736 ft: 6960 corp: 27/833b lim: 67 exec/s: 2048 rss: 122Mb
  #8192   pulse  cov: 5736 ft: 6960 corp: 27/833b lim: 103 exec/s: 2048 rss: 143Mb
  #13067  NEW    cov: 5736 ft: 6965 corp: 28/865b lim: 154 exec/s: 2177 rss: 166Mb L: 32/32 MS: 1 ChangeBit-
  #16384  pulse  cov: 5736 ft: 6965 corp: 28/865b lim: 182 exec/s: 2048 rss: 181Mb
  #32768  pulse  cov: 5736 ft: 6965 corp: 28/865b lim: 347 exec/s: 2184 rss: 258Mb
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 5b17ffb70c31966d3eac06d2258c127ae671d28d6cdf4e6ac20b45cd59ad32f80952c9c749930b97d317c72d5f840a3b75d466fd28fb6c351424a72c3e41bcbc
2020-03-05 16:43:16 -05:00
practicalswift
f4691b6c21 tests: Add fuzzing harness for CKey related functions 2020-03-05 21:11:10 +00:00
MarcoFalke
a2b5aae9f3 Merge #17996: tests: Add fuzzing harness for serialization/deserialization of floating-points and integrals
9ff41f6419 tests: Add float to FUZZERS_MISSING_CORPORA (temporarily) (practicalswift)
8f6fb0a85a tests: Add serialization/deserialization fuzzing for integral types (practicalswift)
3c82b92d2e tests: Add fuzzing harness for functions taking floating-point types as input (practicalswift)
c2bd588860 Add missing includes (practicalswift)

Pull request description:

  Add simple fuzzing harness for functions with floating-point parameters (such as `ser_double_to_uint64(double)`, etc.).

  Add serialization/deserialization fuzzing for integral types.

  Add missing includes.

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/float
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 9b5a0c4838ad18d715c7398e557d2a6d0fcc03aa842f76d7a8ed716170a28f17f249eaede4256998aa3417afe2935e0ffdfaa883727d71ae2d2d18a41ced24b5
2020-03-05 15:41:30 -05:00
practicalswift
9ff41f6419 tests: Add float to FUZZERS_MISSING_CORPORA (temporarily) 2020-03-05 20:35:26 +00:00
practicalswift
8f6fb0a85a tests: Add serialization/deserialization fuzzing for integral types 2020-03-05 20:35:26 +00:00
MarcoFalke
d7134b306a Merge #17917: tests: Add amount compression/decompression fuzzing to existing fuzzing harness
7e9c7113af compressor: Make the domain of CompressAmount(...) explicit (practicalswift)
4a7fd7a712 tests: Add amount compression/decompression fuzzing to existing fuzzing harness: test compression round-trip (practicalswift)

Pull request description:

  Small fuzzing improvement:

  Add amount compression/decompression fuzzing to existing fuzzing harness: test compression round-trip (`DecompressAmount(CompressAmount(…))`).

  Make the domain of `CompressAmount(…)` explicit.

  Amount compression primer:

  ```
      Compact serialization for amounts

      Special serializer/deserializer for amount values. It is optimized for
      values which have few non-zero digits in decimal representation. Most
      amounts currently in the txout set take only 1 or 2 bytes to
      represent.
  ```

  **How to test this PR**

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/integer
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 0f7c05b97012ccd5cd05a96c209e6b4d7d2fa73138bac9615cf531baa3f614f9003e29a198015bcc083af9f5bdc752bb52615b82c5df3c519b1a064bd4fc6664
2020-03-05 15:25:36 -05:00
practicalswift
7e9c7113af compressor: Make the domain of CompressAmount(...) explicit 2020-03-05 20:22:47 +00:00
MarcoFalke
891464950b Merge #18109: tests: Avoid hitting some known minor tinyformat issues when fuzzing strprintf(...)
470e2ac602 tests: Avoid hitting some known minor tinyformat issues when fuzzing strprintf(...) (practicalswift)

Pull request description:

  Avoid hitting some known minor tinyformat issues when fuzzing `strprintf(...)`. These can be removed when the issues have been resolved upstreams :)

  Note to reviewers: The `%c` and `%*` issues are also present for `%<some junk>c` and `%<some junk>*`. That is why simply matching on `"%c"` or `"%*"` is not enough. Note that the intentionally trivial skipping logic overshoots somewhat (`c[…]%` is filtered in addition to `%[…]c`).

Top commit has no ACKs.

Tree-SHA512: 2b002981e8b3f2ee021c3013f1260654ac7e158699313849c9e9660462bb8cd521544935799bb8daa74925959dc04d63440e647495e0b008cfe1b8a8b2202d40
2020-03-05 15:08:31 -05:00
Wladimir J. van der Laan
727857d12d Merge #18112: Serialization improvements step 5 (blockencodings)
353f376277 Convert blockencodings.h to new serialization framework (Pieter Wuille)
e574fff53e Add CustomUintFormatter (Pieter Wuille)
10633398f2 Add DifferenceFormatter (Russell Yanofsky)
56dd9f04c7 Make VectorFormatter support stateful formatters (Russell Yanofsky)
3ca574cef0 Convert CCompactSize to proper formatter (Pieter Wuille)

Pull request description:

  This is probably the most involved change in the sequence of changes extracted from #10785.

  In order to implement the differential encoding of BIP152, this change changes `VectorFormatter` to permit a stateful sub-formatter, which is then used by `DifferenceFormatter`. A `CustomUintFormatter` is added as well to do the 48-bit serialization of short ids.

ACKs for top commit:
  laanwj:
    ACK 353f376277, nice change
  ryanofsky:
    Code review ACK 353f376277. Only changes since last review are suggested assert change and MASK->MAX rename

Tree-SHA512: 976618991a8be62ba0738725b7cfa166a56cde998ebf1031ba6f28557032f1577b666ac7ae25cd498c0e1e740108c3c56a342620b724df41d6cc9d8bdafac037
2020-03-05 19:56:26 +01:00
MarcoFalke
aaf09469fb Merge #18260: refactor: Fix implicit value conversion in formatPingTime
1891245e73 refactor: Cast ping values to double before output (Ben Woosley)
7a810b1d7a refactor: Convert ping wait time from double to int64_t (Ben Woosley)
e6fc63ec7e refactor: Convert min ping time from double to int64_t (Ben Woosley)
b054c46977 refactor: Convert ping time from double to int64_t (Ben Woosley)

Pull request description:

  Alternative to #18252, see motivation there.

  This changes `CNodeStats` to handle ping timestamps as their original incoming usec `int64_t` values until the time they need to be displayed.

ACKs for top commit:
  vasild:
    ACK 1891245
  practicalswift:
    ACK 1891245e73 -- patch looks correct
  promag:
    ACK 1891245e73, added cast to double and also braces.

Tree-SHA512: 7cfcba941d9751b522b8c512c25da493338b444637bd0bb711b152d7d86b431ca0968956be3c844ee9dbfea25edab44a0de2afa44f2c9c0bf5b8df53eba66272
2020-03-05 13:50:10 -05:00
MarcoFalke
96488e6784 Merge #18263: rpc: change setmocktime check to use IsMockableChain
2455aa5d7f [rpc] changed MineBlocksOnDemand to IsMockableChain (Gloria Zhao)

Pull request description:

  Change: Update the if statement in `setmocktime` to use `IsMockableChain` chainparams function (aka `m_is_mockable_chain`) instead of `MineBlocksOnDemand`

  Rationale: It's a more appropriate check for whether or not chain is in RegTest, as [discussed](https://github.com/bitcoin/bitcoin/pull/18037#discussion_r376509388) in #18037

ACKs for top commit:
  MarcoFalke:
    ACK 2455aa5d7f 🙇
  jonatack:
    ACK 2455aa5d7f

Tree-SHA512: 1d8c8b7ff0b3c1bcbf5755194969b6664fe05a35003375ad08d18e34bcefd2df4f64d0e60078a10bbef3c8f469a9b9d07db467089b55c14cf532304bc965bffc
2020-03-05 13:40:06 -05:00
Gloria Zhao
2455aa5d7f [rpc] changed MineBlocksOnDemand to IsMockableChain 2020-03-05 10:09:28 -08:00
David O'Callaghan
8a2a652e6f Remove redundant type information from rpc docs
Co-Authored-By: MarcoFalke <falke.marco@gmail.com>
2020-03-05 15:35:47 +00:00
Wladimir J. van der Laan
a2a77ba34f Merge #18056: ci: Check for submodules
2a95c7c956 ci: Check for submodules (Emil Engler)

Pull request description:

  See #18019.
  The current solution looks like this (I also tested with multiple submodules):
  ```
  These submodules were found, delete them:
   355a5a310019659d9bf6818d2fd66fbb214dfed7 curl (curl-7_68_0-108-g355a5a310)
  ```
  The submodule example command was `git submodule add https://github.com/curl/curl.git curl`

ACKs for top commit:
  laanwj:
    ACK 2a95c7c956

Tree-SHA512: 64bf388123f0a88d12e3e41ff29bc190339377a0615c35dc3f2700bb7773470a8fa426e0ff57188a60ed88bded39f75082ff0b73118651ff403b163422395005
2020-03-05 16:33:54 +01:00
fanquake
d0601e67f1 Merge #17812: config, net, test: asmap feature refinements and functional tests
1ba3e1cc21 init: move asmap code earlier in init process (Jon Atack)
5ba829e12e rpc: fix getpeerinfo RPCResult `mapped_as` type (Jon Atack)
c90b9a2399 net: extract conditional to bool CNetAddr::IsHeNet (Jon Atack)
819fb5549b logging: asmap logging and #include fixups (Jon Atack)
dcaf543ba0 test: add functional test for an empty, unparsable asmap (Jon Atack)
b8d0412b21 config: separate the asmap finding and parsing checks (Jon Atack)
81c38a2497 config: enable passing -asmap an absolute file path (Jon Atack)
fbe9b024f0 config: use default value in -asmap config (Jon Atack)
08b992675c test: add feature_asmap functional tests (Jon Atack)

Pull request description:

  This PR builds on PR #16702 to add functional tests / sanity checks and user-facing refinements for passing `-asmap` to configure ASN-based IP bucketing in addrman. As per our review discussion in that PR, the idea here is to handle aspects like functional tests and config arg handling that can help the PR be merged while enabling the author to focus on the bucketing itself.

  - [x] add feature functional tests to verify node behaviour and debug log output when launching

    - `bitcoind` with no `-asmap` arg

    - `bitcoind -asmap=RELATIVE_FILENAME` to the unit test data skeleton asmap

    - `bitcoind -asmap` with no filename specified using the default asmap file

    - `bitcoind -asmap` with no filename specified and a missing default asmap file

  - [x] add the ability to pass absolute path filenames to the `-asmap` config arg in addition to datadir-relative path filenames as per https://github.com/bitcoin/bitcoin/pull/16702#discussion_r361300447, and add test coverage

  - [x] separate the asmap file finding and parsing checks, which allows adding tests for the case of a found but unparseable or empty asmap

  - [x] add test for an empty asmap

  - [x] various asmap fixups

  - [x] move the asmap init code earlier in the init process to provide immediate feedback when passing an  `-asmap` config arg. This speeds up the `feature_asmap` functional test from 60 to 5 seconds! Credit to Wladimir J. van der Laan for the suggestion.

ACKs for top commit:
  practicalswift:
    ACK 1ba3e1cc21 -- diff looks correct
  fanquake:
    ACK 1ba3e1cc21

Tree-SHA512: e9094460a597ac5597449acfe631c87b71d3ede6a12c7ae61b26d1161b3eefed8e7e25c4fb0505864cebd89300b7c4cf9378060aa9155441029315df15fa3283
2020-03-05 20:13:33 +08:00
Ben Woosley
1891245e73 refactor: Cast ping values to double before output
Note the divisor is a floating point literal so presumably
also floating point.
2020-03-05 04:44:50 -05:00
fanquake
cbc32d67dc Merge #18249: test: Bump timeouts to accomodate really slow disks
fa6df0de53 test: Bump timeouts to accomodate really slow disks (MarcoFalke)

Pull request description:

  Needed these patches locally for some arm machines with slow storage

ACKs for top commit:
  practicalswift:
    ACK fa6df0de53
  fanquake:
    ACK fa6df0de53

Tree-SHA512: 22f2f6f7ed05f26013431126bb179b029dbc931f02d0e58f8970c6d477f43e3106d76c9732942034cb2cfcb827191e338a082f953ccb69531a19ee6dab9a7e1a
2020-03-05 15:06:52 +08:00
fanquake
2492dc0068 Merge #18259: doc: Merge release notes for 0.20.0 release
fa8b6020ec doc: Merge release notes for 0.20.0 release (MarcoFalke)

Pull request description:

  mostly move-only. Can be reviewed with the `--color-moved=dimmed-zebra` option.

ACKs for top commit:
  laanwj:
    ACK fa8b6020ec
  fanquake:
    ACK fa8b6020ec - any changes are basically headers and newlines.

Tree-SHA512: 7273c7625d60c3b28bafc4371e17545bd9fcaa672fde8492a0b4ab88081d616dd41c77389d18a9a3b39b595c5409a354bf511745ddcb9834c9a25d91b94edb28
2020-03-05 10:25:39 +08:00
Andrew Chow
9b5950db86 bnb: exit selection when best_waste is 0
If we find a solution which has no waste, just use that. This solution
is what we would consider to be optimal, and other solutions we find
would have to also have 0 waste, so they are equivalent to the first
one with 0 waste. Thus we can optimize by just choosing the first one
with 0 waste.
2020-03-04 16:35:15 -05:00
Luke Dashjr
d831831822 lockedpool: When possible, use madvise to avoid including sensitive information in core dumps 2020-03-04 19:52:16 +00:00
Ben Woosley
7a810b1d7a refactor: Convert ping wait time from double to int64_t 2020-03-04 13:45:29 -05:00
Ben Woosley
e6fc63ec7e refactor: Convert min ping time from double to int64_t 2020-03-04 13:44:57 -05:00
Ben Woosley
b054c46977 refactor: Convert ping time from double to int64_t 2020-03-04 13:44:25 -05:00
MarcoFalke
fa8b6020ec doc: Merge release notes for 0.20.0 release 2020-03-04 10:06:23 -05:00
Jon Atack
1ba3e1cc21 init: move asmap code earlier in init process
and update feature_asmap.py and test_runner.py

This commit moves the asmap init.cpp code from the end of "Step 12: start node"
to "Step 6: network initialization" to provide feedback on passing an -asmap
config arg much more quickly. This change speeds up the feature_asmap.py
functional test file from 60 to 5 seconds by accelerating the 2 tests that use
`assert_start_raises_init_error`.

Credit to Wladimir J. van der Laan for the suggestion.
2020-03-04 14:54:30 +01:00
Jon Atack
5ba829e12e rpc: fix getpeerinfo RPCResult mapped_as type
and mention it is only available if the asmap config flag is set.
2020-03-04 14:54:27 +01:00
Jon Atack
c90b9a2399 net: extract conditional to bool CNetAddr::IsHeNet
and remove redundant public declaration
2020-03-04 14:31:31 +01:00
Jon Atack
819fb5549b logging: asmap logging and #include fixups
- move asmap #includes to sorted positions in addrman and init (move-only)

- remove redundant quotes in asmap InitError, update test

- remove full stops from asmap logging to be consistent with debug logging,
  update tests
2020-03-04 14:24:19 +01:00
Jon Atack
dcaf543ba0 test: add functional test for an empty, unparsable asmap
This is now testable after separating the asmap finding and parsing checks in
the previous commit.
2020-03-04 14:24:17 +01:00
Jon Atack
b8d0412b21 config: separate the asmap finding and parsing checks
and update the tests.
2020-03-04 14:24:15 +01:00
Jon Atack
81c38a2497 config: enable passing -asmap an absolute file path
- allow passing an absolute file path to the -asmap config arg

- update the -asmap config help

- add a functional test in feature_asmap.py
2020-03-04 14:24:13 +01:00
Jon Atack
fbe9b024f0 config: use default value in -asmap config
and move to sorted position
2020-03-04 14:24:11 +01:00
Jon Atack
08b992675c test: add feature_asmap functional tests
to verify node behaviour and debug log when launching bitcoind in these cases:

1. `bitcoind` with no -asmap arg, using /16 prefix for IP bucketing

2. `bitcoind -asmap=<relative path>`, using the unit test skeleton asmap

3. `bitcoind -asmap/-asmap=` with no file specified, using the default asmap

4. `bitcoind -asmap` with no file specified, and a missing default asmap file

The tests are order-independent. The slowest test (missing default asmap file)
is placed last.
2020-03-04 14:24:08 +01:00
MarcoFalke
a71c34742c Merge #17809: rpc: Auto-format RPCResult
fa6b061fc1 rpc: Auto-format RPCResult (MarcoFalke)
fa7d0503d3 rpc: Move OuterType enum to header (MarcoFalke)

Pull request description:

  This enforces most syntax rules of the RPCResult at compile time (or some at run time during unit and functional tests)

  Apart from normalizing the syntax, by separating stylistic formatting from the structure, we could in theory directly generate the html for e.g. https://bitcoincore.org/en/doc/0.19.0/rpc/wallet/importmulti/

ACKs for top commit:
  Sjors:
    Indeed, re-ACK fa6b061fc1
  ajtowns:
    ACK fa6b061fc1 -- skimmed code changes and differences to rpc help output

Tree-SHA512: 5b510b3aa0b7c7b9189a48c77593159409069f939145b9a00c5478e894cf65f994d44d633eb7bb7dbea40ee820645a2930976c24772379d96929002b120efa28
2020-03-04 08:16:57 -05:00
fanquake
088604221b Merge #18253: doc: Correct spelling errors in comments
9b0e16226e doc: Correct spelling errors in comments (Ben Woosley)

Pull request description:

  And ci script output.

  Identified via test/lint/lint-spelling

  Before:
  ```
  $ test/lint/lint-spelling.sh
  ci/test/05_before_script.sh:29: explicitely  ==> explicitly
  src/compressor.h:43: Ser  ==> Set
  src/compressor.h:78: Ser  ==> Set
  src/logging/timer.h:88: outputing  ==> outputting
  src/node/psbt.cpp:87: minumum  ==> minimum
  src/qt/coincontroldialog.cpp:372: UnSelect  ==> deselect
  src/qt/coincontroldialog.cpp:443: unselect  ==> deselect
  src/qt/coincontroldialog.cpp:448: UnSelect  ==> deselect
  src/qt/coincontroldialog.cpp:699: UnSelect  ==> deselect
  src/serialize.h:211: Ser  ==> Set
  src/serialize.h:213: Ser  ==> Set
  src/serialize.h:228: Ser  ==> Set
  src/serialize.h:246: Ser  ==> Set
  src/serialize.h:484: Ser  ==> Set
  src/serialize.h:490: Ser  ==> Set
  src/serialize.h:510: Ser  ==> Set
  src/serialize.h:622: Ser  ==> Set
  src/serialize.h:740: Ser  ==> Set
  src/test/base32_tests.cpp:14: fo  ==> of, for
  src/test/base64_tests.cpp:14: fo  ==> of, for
  src/txmempool.h:756: incomaptible  ==> incompatible
  src/undo.h:26: Ser  ==> Set
  src/wallet/coincontrol.h:74: UnSelect  ==> deselect
  test/functional/feature_backwards_compatibility.py:116: Abondon  ==> Abandon
  test/functional/rpc_getaddressinfo_label_deprecation.py:7: superceded  ==> superseded
  test/lint/lint-shell.sh:44: desriptor  ==> descriptor
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

  After:
  ```
  $ test/lint/lint-spelling.sh
  src/test/base32_tests.cpp:14: fo  ==> of, for
  src/test/base64_tests.cpp:14: fo  ==> of, for
  test/functional/rpc_getaddressinfo_label_deprecation.py:7: superceded  ==> superseded
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
  ```

ACKs for top commit:
  practicalswift:
    ACK 9b0e16226e
  MarcoFalke:
    ACK 9b0e16226e

Tree-SHA512: 9ce203700b11596e4b920b3c5b04f59bc7784fe5b495868d43423608180a9a553ec7efcc5ad70384f3ce462b036c2a682260efebce493c5e6a3d48716b268179
2020-03-04 12:22:17 +08:00
MarcoFalke
faae5a9a35 test: Add bad-txns-*-toolarge test cases to invalid_txs 2020-03-03 12:42:52 -05:00
Ben Woosley
9b0e16226e doc: Correct spelling errors in comments
And ci script output.

Identified via test/lint/lint-spelling
2020-03-02 23:07:21 -08:00
MarcoFalke
fa6df0de53 test: Bump timeouts to accomodate really slow disks 2020-03-02 16:25:05 -05:00
Wladimir J. van der Laan
ac5c5d0162 Merge #18168: httpserver: use own HTTP status codes
aff2748f8a httpserver: use own HTTP status codes (Filip Gospodinov)

Pull request description:

  Before, macros defined in `<event2/http.h>` have been used for some HTTP status codes. `<event2/http.h>` is included implicitly and the usage of its status code macros is inconsistent with the majority HTTP response implementations in this file.

  Now, the `HTTPStatusCode` enum from `<rpc/protocol.h>` is consistently used for all HTTP response implementations.

ACKs for top commit:
  practicalswift:
    ACK aff2748f8a -- patch looks correct
  laanwj:
    ACK aff2748f8a

Tree-SHA512: 6a7043488b88dcd584215d16b5f16f7bd668fe5553d31298d1beef134d2b0648aef81533014e34d1cd600baa36ee4e853f195352f4a00c866bd5ab1ff688bd50
2020-03-02 17:10:55 +01:00
Samuel Dobson
1f886243e4 Merge #18224: Make AnalyzePSBT next role calculation simple, correct
1ef28b4f7c Make AnalyzePSBT next role calculation simple, correct (Gregory Sanders)

Pull request description:

  Sniped test and alternative to https://github.com/bitcoin/bitcoin/pull/18220

  Sjors documenting the issue:
  ```
  A PSBT signed by ColdCard was analyzed as follows (see #17509 (comment))

  {
    "inputs": [
      {
        "has_utxo": true,
        "is_final": false,
        "next": "finalizer"
      }
    ],
    "estimated_vsize": 141,
    "estimated_feerate": 1e-05,
    "fee": 1.41e-06,
    "next": "signer"
  }
  I changed AnalyzePSBT so that it returns "next": "finalizer" instead.
  ```

  It makes it much clearer that the role has been decided before hitting the `calc_fee` block, and groups all state-deciding in one spot instead of 2.

  Note that this assumes that PSBT roles are a complete ordering, which for now and in the future seems to be a correct assumption.

ACKs for top commit:
  Sjors:
    ACK 1ef28b4f7c, much nicer. Don't forget to document the bug fix.
  achow101:
    ACK 1ef28b4f7c
  Empact:
    ACK 1ef28b4f7c

Tree-SHA512: 22ba4234985c6f9c1445b14565c71268cfaa121c4ef000ee3d5117212b09442dee8d46d9701bceddaf355263fe25dfe40def2ef614d4f2fe66c9ce876cb49934
2020-03-02 22:47:59 +13:00
Karl-Johan Alm
79facb11e9 wallet: use constant CWallets in rpcwallet.cpp
* GetAvoidReuseFlag: simply gets the flag, without modifying the wallet
* ListReceived: helper function to produce lists
* ListTransactions: produces a list of transactions, without modifications; two cases of map [] -> .at() for verified-existing keys
* DescribeWalletAddress: generates a description of a given wallet address without changing the wallet
* The following functions produce a list without making any modifications to the wallet:
  * listaddressgroupings
  * listreceivedbyaddress
  * listreceivedbylabel
  * listtransactions
  * listsinceblock
  * listlockunspent
  * listunspent
  * listlabels
  * getreceivedbyaddress
  * getreceivedbylabel
  * getaddressesbylabel
* signmessage: uses the wallet to procure a private key for signing, but does no modifications
* getbalance, getunconfirmedbalance: calculates the wallet balance, without any modifications
* gettransaction: procures transaction without any modifications
* backupwallet: makes a backup of the wallet to disk, without changing said wallet
* getwalletinfo: produces info about wallet without any modifications
* signrawtransactionwithwallet: modifies incoming transaction on the fly by signing with private key procured from within wallet; no modifications to wallet
* getaddressinfo: gets information about the given address, with no modifications done to the wallet; one case of [] -> .at() and one ::iterator -> ::const_iterator
* walletprocesspsbt: processes the given PSBT on the fly, without modifying the wallet
2020-03-02 17:27:36 +09:00
Karl-Johan Alm
d9b0ebc1da wallet: make ReserveDestination pwallet ivar const 2020-03-02 17:27:35 +09:00
Karl-Johan Alm
57c569e4d9 wallet: make BackupWallet() const 2020-03-02 17:27:35 +09:00
Karl-Johan Alm
df3a818d2a wallet: make getters const 2020-03-02 17:27:35 +09:00
Karl-Johan Alm
227b9dd2d6 wallet/spkm: make GetOldestKeyPoolTime() const
The method checks the oldest key time for key pools and returns the oldest. It does no modifications.
2020-03-02 17:26:31 +09:00
Karl-Johan Alm
22d329ad0e wallet: use constant CWallets in rpcdump.cpp
* GetWalletAddressesForKey is, as the name implies, immutable; the one change besides the parameter constness is a [] -> .at() change, to a verified-existing key.
* dumpprivkey and dumpwallet are both similarly immutable, for obvious reasons.
2020-03-02 17:26:30 +09:00
Karl-Johan Alm
7b3587b29d wallet/db: make IsDummy() const
This method does a simple check and no modifications.
2020-03-02 17:26:30 +09:00
Karl-Johan Alm
d366795d18 wallet/db: make Backup() const
This method is the to-disk equivalent of serialize methods which are also const.
2020-03-02 17:26:30 +09:00
Karl-Johan Alm
8cd0b86340 wallet: make CanGetAddresses() const
CWallet::CanGetAddresses() is used to check whether the wallet has available or is able to produce keys for addresses. It uses the ScriptPubKeyMan::CanGetAddresses(), which in turn uses the const KeypoolCountExternalKeys() method, all which do counting and no modifications.
2020-03-02 17:26:30 +09:00
Karl-Johan Alm
037fa770eb wallet: make KeypoolCountExternalKeys() const
This method returns the sum of the key pool sizes. It does no modification.
2020-03-02 17:26:30 +09:00
Karl-Johan Alm
ddc93557ad wallet: make CanGenerateKeys() const
This method simply checks if HD is or can be enabled and does not require mutability.
2020-03-02 17:26:30 +09:00
Karl-Johan Alm
dc2d0650fd make BlockUntilSyncedToCurrentChain() const
The method checks the chain tip for the best block, and calls SyncWithValidationInterfaceQueue() (a standalone function) if necessary.
2020-03-02 17:26:30 +09:00
MarcoFalke
54a7ef612a Merge #17399: validation: Templatize ValidationState instead of subclassing
10efc0487c Templatize ValidationState instead of subclassing (Jeffrey Czyz)
10e85d4adc Remove ValidationState's constructor (Jeffrey Czyz)
0aed17ef28 Refactor FormatStateMessage into ValidationState (Jeffrey Czyz)

Pull request description:

  This removes boilerplate code in the subclasses which otherwise only
  differ by the result type.

  The subclassing was introduced in a27a295.

ACKs for top commit:
  MarcoFalke:
    ACK 10efc0487c 🐱
  ajtowns:
    ACK 10efc0487c -- looks good to me
  jonatack:
    ACK 10efc048 code review, build/tests green, nice cleanup

Tree-SHA512: 765dd52dde7d49b9a5c6d99d97c96f4492673e2aed0b0604faa88db0308fa4500a26bf755cca0b896be283874096c215932e1110a2d01dc012cd36a5fce58a42
2020-03-01 15:34:05 -05:00
Yusuf Sahin HAMZA
3e32499909 Change example addresses to bech32 2020-03-01 18:13:35 +03:00
fanquake
715dbbe9e8 Merge #18229: random: drop unused MACH time headers
d36146009f Drop unused mach time headers (Ben Woosley)

Pull request description:

  Now that we're no longer special-casing clock usage for MacOS (see #17800), we're
  not referencing anything defined in these headers.

  Incidentally, this removes our last reference to the `__MACH__` system def. 🎉

ACKs for top commit:
  jonasschnelli:
    utACK d36146009f
  fanquake:
    ACK d36146009f - thanks.

Tree-SHA512: 246045b0683a705ad034416e8ace2024e652026a6c0517b6797320e52fc18a6e111ec2e405ca40653bd1d6421bb7755232e8fec22651fff8e448eb7d5646a954
2020-02-29 18:08:11 +08:00
fanquake
9027960932 Merge #18225: util: Fail to parse empty string in ParseMoney
8888461f68 util: Fail to parse empty string in ParseMoney (MarcoFalke)
fab30b61eb util: Remove unused ParseMoney that takes a c_str (MarcoFalke)

Pull request description:

  Supplying a fee rate or an amount on the command line as an empty string, which currently parses as `0` seems fragile and confusing. See for example the confusion in #18214.

  Fixes #18214

ACKs for top commit:
  Empact:
    Code Review ACK 8888461f68
  achow101:
    ACK 8888461f68
  instagibbs:
    utACK 8888461f68

Tree-SHA512: ac2d6b7fa89fe5809c34d5f49831042032591c34fb3c76908d72fed51e8bced41bf2b41dc1b3be34ee691a40463355649857a7a8f378709d38ae89503feb11c2
2020-02-29 09:44:48 +08:00
Ben Woosley
d36146009f Drop unused mach time headers
Now that we're no longer special-casing clock usage for MacOS, we're
not referencing anything defined in these headers.
2020-02-28 14:56:49 -08:00
MarcoFalke
eca4d8ef6a Merge #16562: Refactor message transport packaging
16d6113f4f Refactor message transport packaging (Jonas Schnelli)

Pull request description:

  This PR factors out transport packaging logic from `CConnman::PushMessage()`.
  It's similar to #16202 (where we refactor deserialization).

  This allows implementing a new message transport protocol like BIP324.

ACKs for top commit:
  dongcarl:
    ACK 16d6113f4f FWIW
  ariard:
    Code review ACK 16d6113
  elichai:
    semiACK 16d6113f4f ran functional+unit tests.
  MarcoFalke:
    ACK 16d6113f4f 🙎

Tree-SHA512: 8c2f8ab9f52e9b94327973ae15019a08109d5d9f9247492703a842827c5b5d634fc0411759e0bb316d824c586614b0220c2006410851933613bc143e58f7e6c1
2020-02-28 17:01:58 -05:00
Wladimir J. van der Laan
1a51cd1ac5 Merge #17800: random: don't special case clock usage on macOS
dc9305b616 random: don't special case clock usage on macOS (fanquake)

Pull request description:

  `clock_gettime()`, `CLOCK_MONOTONIC` and `CLOCK_REALTIME` are all available for use on
  macOS (now that we require macOS >=10.12 and build against 10.14). Use them rather than the [deprecated](https://developer.apple.com/library/archive/documentation/Darwin/Conceptual/KernelProgramming/Mach/Mach.html) `mach_timespec_t` time API.

  I mentioned the possibility for this change [in #17270](https://github.com/bitcoin/bitcoin/pull/17270#discussion_r346090606).

  [master](1dbf3350c6):
  ```bash
  2019-12-23T20:49:43Z Feeding 216 bytes of dynamic environment data into RNG
  2019-12-23T20:50:43Z Feeding 216 bytes of dynamic environment data into RNG
  ```

  This PR:
  ```bash
  2019-12-23T20:32:41Z Feeding 232 bytes of dynamic environment data into RNG
  2019-12-23T20:33:42Z Feeding 232 bytes of dynamic environment data into RNG
  ```

  ~~Depends on #16392.~~ Merged.

ACKs for top commit:
  laanwj:
    ACK dc9305b616

Tree-SHA512: 18c2f336ea628f9cf7339b817381d230a18893fd9c0351bf99a39ca6f45c5b0a20af9d599d48d6c09515627d5edafa91337c17f9f790264251d2cdcb3763bbd5
2020-02-28 22:51:54 +01:00
MarcoFalke
faf6f156ff test: Add missing syncwithvalidationinterfacequeue 2020-02-29 04:04:27 +07:00
MarcoFalke
7a266a679d Merge #18173: refactor: test/bench: deduplicate SetupDummyInputs()
7bf4ce4f64 refactor: test/bench: dedup SetupDummyInputs() (Sebastian Falbesoner)

Pull request description:

  The only difference between `SetupDummyInputs()` in `test/transaction_tests.cpp` and the one in `bench/ccoins_caching.cpp` was the nValue amounts of the outputs, so we allow to pass those in an extra (fixed-size) array parameter.

ACKs for top commit:
  MarcoFalke:
    re-ACK 7bf4ce4f64, only change is schuffling includes 🚶
  Empact:
    ACK 7bf4ce4f64

Tree-SHA512: e13643b2470f6b6ab429da0c0a8eebd4cb41e2ff2e421ef36f85fa4847bf4ea8aab88d59a01e94cac4c4eb85edb561463f02215b174c50b573ac6bbcc2bf98a3
2020-02-29 03:23:04 +07:00
Sebastian Falbesoner
5aab011805 test: add unit test for non-standard "scriptsig-not-pushonly" txs
The function IsStandardTx() returns rejection reason "scriptsig-not-pushonly"
if the transaction has at least one input for which the scriptSig consists of
any other ops than just PUSHs.
2020-02-28 21:20:31 +01:00
Sebastian Falbesoner
7bf4ce4f64 refactor: test/bench: dedup SetupDummyInputs()
The only difference between SetupDummyInputs() in test/transaction_tests.cpp
and the one in bench/ccoins_caching.cpp was the nValue amounts of the outputs,
so we allow to pass those in an extra (fixed-size) array parameter.
2020-02-28 21:09:03 +01:00
MarcoFalke
9aa8145bc0 Merge #17959: test: check specific reject reasons in feature_csv_activation.py
54be4e71d8 test: check specific reject reasons in feature_csv_activation.py (Sebastian Falbesoner)

Pull request description:

  This is kind of a prequel to #17921: increases the general quality of the functional test `feature_csv_activation.py` by checking for the specific reject reasons whenever the sending of a block fails. To get the reason, we have to limit the script threads to 1 via the parameter `-par=1`, like it is also done in `feature_cltv.py`:
  a654626f07/test/functional/feature_cltv.py (L57-L61)

  The commit also fixes a bug that was uncovered with this checks: for the BIP112 version 1 tx tests, txs from `bip112txs_vary_OP_CSV_v1` have been add twice to the list `failed_txs`:
  a654626f07/test/functional/feature_csv_activation.py (L396-L397)

  leading also to a block rejection as expected but for the wrong reason. It seems one of those two tx lists was meant to be `bip112txs_vary_OP_CSV_v1` (without the `_9`) and it was a typo.

ACKs for top commit:
  MarcoFalke:
    ACK 54be4e71d8 📶

Tree-SHA512: 9aac11aee3f53f1ae95ddb346a2f268872038f4d118c8dcf81b8201dee869774c9f3c3f1c326e370b8fd4eaf8e0673371689a96d9b1cb91be4286c88824725c3
2020-02-29 03:07:40 +07:00
Sebastian Falbesoner
54be4e71d8 test: check specific reject reasons in feature_csv_activation.py
this also fixes a bug that was uncovered with this checks:
for the BIP112 version 1 tx tests, certain txs (bip112txs_vary_OP_CSV_v1) have
been sent twice due to a typo, leading also to a failure as expected but for the
wrong reason
2020-02-28 20:42:23 +01:00
Wladimir J. van der Laan
5ad80bec3f Merge #18135: build: add --enable-determinism configure flag
3d9b41ecc0 build: add --enable-determinism configure flag (fanquake)

Pull request description:

  This adds a `--enable-determinsm` configure flag, which if used, will enable additional compile / link time flags to make subsequent builds of bitcoind deterministic.

  The first flag enabled is `--no-insert-timestamp`. This prevents the linker from embedding timestamps, and makes consecutive builds of `bitcoind.exe` deterministic. This will likely also be used for [Guix Windows builds](https://github.com/bitcoin/bitcoin/pull/17595#issuecomment-582696467).

  [`--no-insert-timestamp`](https://manpages.debian.org/buster/binutils-mingw-w64-x86-64/x86_64-w64-mingw32-ld.bfd.1.en.html):
  > The option --no-insert-timestamp can be used to insert a zero value for the timestamp, this ensuring that binaries produced from identical sources will compare identically.

  Diff of consecutive builds of [master](2bdc476d4d):
  ```diff
  --- bitcoind.exe.1
  +++ bitcoind.exe.2
  @@ -2,20 +2,20 @@
   00000060: 7420 6265 2072 756e 2069 6e20 444f 5320  t be run in DOS
   00000070: 6d6f 6465 2e0d 0d0a 2400 0000 0000 0000  mode....$.......
  -00000080: 5045 0000 6486 1400 57e8 445e 00da 6900  PE..d...W.D^..i.
  +00000080: 5045 0000 6486 1400 e3e9 445e 00da 6900  PE..d.....D^..i.
   00000090: e015 0100 f000 2600 0b02 021f 00de 4900  ......&.......I.
   000000a0: 00b0 5b00 008a 0000 e014 0000 0010 0000  ..[.............
   000000b0: 0000 4000 0000 0000 0010 0000 0002 0000  ..@.............
   000000c0: 0400 0000 0000 0000 0500 0200 0000 0000  ................
  -000000d0: 00f0 6a00 0006 0000 bd31 af00 0300 6001  ..j......1....`.
  +000000d0: 00f0 6a00 0006 0000 d434 af00 0300 6001  ..j......4....`.
   000000e0: 0000 2000 0000 0000 0010 0000 0000 0000  .. .............
  @@ -373594,15 +373594,15 @@
   005b35f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
  -005b3600: 0000 0000 57e8 445e 0000 0000 7ce1 5b00  ....W.D^....|.[.
  +005b3600: 0000 0000 e2e9 445e 0000 0000 7ce1 5b00  ......D^....|.[.
   005b3610: 0100 0000 2200 0000 2200 0000 28e0 5b00  ...."..."...(.[.
  ```

ACKs for top commit:
  practicalswift:
    ACK 3d9b41ecc0 -- patch looks correct
  laanwj:
    ACK 3d9b41ecc0

Tree-SHA512: 1ff9dab7fa818b1fc6b0eb3b7a1e0468aac9e2578f4451aa300a648f883fa83f83722067f1adf27ebad54157790857d1501d573adbd7ebdf6962858cc669960d
2020-02-28 19:08:38 +01:00
MarcoFalke
73cfa070e5 Merge #18195: test: Add cost_of_change parameter assertions to bnb_search_test
c72a11a1a0 test: Add cost_of_change parameter assertions to bnb_search_test (Yancy Ribbens)

Pull request description:

  If the `cost_of_change` variable is removed from the method body `SelectCoinsBnB`, there are currently no failing unit tests.  This PR adds assertions about the behavior of the `cost_of_change`:  If the cost of creating a change output is greater than what's leftover, then consume the output and create no change, otherwise, don't consume the output (no match found).

ACKs for top commit:
  achow101:
    ACK c72a11a1a0

Tree-SHA512: 613aa411df5e2911446e0e8bf3309336faaadf2d3c56e7d125b76454e7c6f9e4f5e8f0910dc6222282628e38cd8a4a7c56bb3d36b564a17f396b9b503ecc64c8
2020-02-29 00:39:52 +07:00
MarcoFalke
8888461f68 util: Fail to parse empty string in ParseMoney 2020-02-29 00:25:58 +07:00
MarcoFalke
fab30b61eb util: Remove unused ParseMoney that takes a c_str 2020-02-29 00:25:37 +07:00
Gregory Sanders
1ef28b4f7c Make AnalyzePSBT next role calculation simple, correct 2020-02-28 11:31:35 -05:00
MarcoFalke
e5753fa4e8 Merge #17921: test: test OP_CSV empty stack fail in feature_csv_activation.py
5ffaf883b9 test: eliminiated magic numbers in feature_csv_activation.py (Sebastian Falbesoner)
09f706ab8e test: check for OP_CSV empty stack fail reject reason in feature_csv_activation.py (Sebastian Falbesoner)
cbd345a75c test: test OP_CSV empty stack fail in feature_csv_activation.py (Sebastian Falbesoner)

Pull request description:

  Adds an empty stack failure check for OP_CSV (BIP112) to the functional test `feature_csv_activation.py` by prepending a valid scriptSig with `OP_CHECKSEQUENCEVERIFY`.
  If BIP112 is inactive, the operator just behaves as a NOP (for both tx versions 1 and 2) and the transaction remains valid -- if it is active, the tx is invalid due to an empty stack (for both tx versions 1 and 2, as well).

Top commit has no ACKs.

Tree-SHA512: 81102aaead5be11e02b894867fa9a9cc17358ec0eb2f21ce2d3db845b87691d305e6ed7c525f9c7e5bcb3c5c609eb28deca0fbaa3d5e9ff928cecd3b91ff129a
2020-02-28 22:49:29 +07:00
Sebastian Falbesoner
5ffaf883b9 test: eliminiated magic numbers in feature_csv_activation.py 2020-02-28 09:04:59 +01:00
Sebastian Falbesoner
09f706ab8e test: check for OP_CSV empty stack fail reject reason in feature_csv_activation.py 2020-02-28 09:04:59 +01:00
Sebastian Falbesoner
cbd345a75c test: test OP_CSV empty stack fail in feature_csv_activation.py
With BIP112 activated, the operation OP_CHECKSEQUENCEVERIFY (former OP_NOP3)
leads to script interpreter termination with an error if one of the following
conditions is true:
    -> stack is empty
    -> top item on stack is negative (< 0)
    -> top item on stack has disable flag unset and at least one of
       four other conditions is true (contains the core CSV logic)

This commits adds the missing empty stack failure test to the functional test
by prepending a valid scriptSig with just OP_CHECKSEQUENCEVERIFY. If BIP112 is
inactive, the operator just behaves as a NOP (for both tx versions 1 and 2) and
the transaction remains valid -- if it is active, the tx is invalid due to an
empty stack (for both tx versions 1 and 2, as well).
2020-02-28 09:04:18 +01:00
fanquake
eae48ec84c Merge #18209: test: Reduce unneeded whitelist permissions in tests
fa45d60646 test: Reduce unneeded whitelist permissions in tests (MarcoFalke)

Pull request description:

  It makes the tests confusing and fragile when overwriting default command line values that are not needed to be overwritten.

ACKs for top commit:
  fanquake:
    ACK fa45d60646
  laanwj:
    ACK fa45d60646

Tree-SHA512: 8ae5ad8c6be156b1a983adccbca8d868ef841e00605ea88e24227f1b7493987c50b3e62e68dd7dc785ad73d6e14279eb13d7a151cb0a976426fe2fd63ce5cbcd
2020-02-28 12:32:31 +08:00
Jeffrey Czyz
10efc0487c Templatize ValidationState instead of subclassing
This removes boilerplate code in the subclasses which otherwise only
differ by the result type.
2020-02-27 17:59:21 -08:00
Jeffrey Czyz
10e85d4adc Remove ValidationState's constructor 2020-02-27 17:59:21 -08:00
Jeffrey Czyz
0aed17ef28 Refactor FormatStateMessage into ValidationState 2020-02-27 17:59:07 -08:00
fanquake
fe63d79eab Merge #18212: doc: add missing step in win deployment instructions
7644567758 Add missing step in win deployment instructions (Dan Gershony)

Pull request description:

  As explained in #17864 there is a missing step that was required to finish the compilation for Bitcoin Core on Windows.

ACKs for top commit:
  sipsorcery:
    ACK 7644567758.

Tree-SHA512: 0d9ed248f511ea4f440d6c2f3e1235abbb3f9c0c576ca715df3cda91682d668991001197930e687ee48709eedbcf148d8ac9236464e9ce1d2ed15d8b3b4b252d
2020-02-28 08:37:03 +08:00
MarcoFalke
fa2cf85e6f test: Fix race in p2p_segwit
Co-Authored-By: Suhas Daftuar <sdaftuar@gmail.com>
2020-02-28 03:26:36 +07:00
MarcoFalke
1615043935 Merge #17461: test: check custom descendant limit in mempool_packages.py
b902bd66b0 test: check custom descendant limit in mempool_packages.py (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR to #17435, testing the custom descendant limit, passed by the argument `-limitdescendantcount`. ~~It was more tricky than expected, mainly because we don't know for sure at which point node1 has got all the transactions broadcasted from node0 (for the ancestor test this wasn't a problem since the txs were immediately available through `invalidateblock`) -- a simple `sync_mempools()` doesn't work here since the mempool contents are not equal due to different ancestor/descendant limits. Hence I came up with a "hacky manual sync":~~
  1. ~~wait until the mempool has the _expected_ tx count (see conditions below)~~
  2. ~~after that, wait some time and get sure that the mempool contents haven't changed in-between~~

  ~~Like for~~ Similar to the ancestor test, we overall check for ~~three~~ four conditions:
  - the # of txs in the node1 mempool is equal to the descendant limit (plus 1 for the parent tx, plus the # txs from the previous ancestor test which are still in) ~~(done by the hacky sync above)~~
  - all txs in node1 mempool are a subset of txs in node0 mempool
  - part of the constructed descendant-chain (the first ones up to the limit) are contained in node1 mempool
  - the remaining part of the constructed descendant-chain (all after the first ones up to the limit) is *not* contained in node1 mempool

ACKs for top commit:
  JeremyRubin:
    Excellent. utACK b902bd6

Tree-SHA512: 7de96dd248f16ab740e178ac5b64b57ead18cdcf74adfe989709d215e4a67b6b6d20de22c48e885d5f2edc55caaddd44a4261e996c5c87687ceb6a47f1d1fdaf
2020-02-28 03:02:24 +07:00
MarcoFalke
324a6dfeaf Merge #17771: tests: Add fuzzing harness for V1TransportDeserializer (P2P transport)
2f63ffd15c tests: Add fuzzing harness for V1TransportDeserializer (P2P transport) (practicalswift)

Pull request description:

  Add fuzzing harness for `V1TransportDeserializer` (P2P transport).

  **Testing this PR**

  Run:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/p2p_transport_deserializer
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 2f63ffd15c

Tree-SHA512: 8507d4a0414d16f1b8cc9649e3e638f74071dddc990d7e5d7e6faf77697f50bdaf133e49e2371edd29068a069a074469ef53148c6bfc9950510460b81d87646a
2020-02-28 02:35:14 +07:00
Dan Gershony
7644567758 Add missing step in win deployment instructions
As explained in #17864  there is a missing step that was required to finish the compilation for bitcoin core on windows
2020-02-27 15:11:30 +01:00
fanquake
3d9b41ecc0 build: add --enable-determinism configure flag
If used, this will enable additional compile / link time flags
that will make subsequent builds of bitcoind determinisitic.
2020-02-27 12:12:18 +08:00
fanquake
4502ed7cd1 Merge #18211: test: Disable mockforward scheduler unit test for now
fab2527515 test: Disable mockforward scheduler unit test for now (MarcoFalke)

Pull request description:

  This should be a workaround to fix #18174 in the short run and buy us more time to investigate the issue while ci runs are green again 🙏

ACKs for top commit:
  fanquake:
    ACK fab2527515 - be good to get Travis back.
  laanwj:
    ACK fab2527515

Tree-SHA512: 027e86b3dfec203a464e5bf528e9933c208c36633c2d4bfcdbc10da1799637a5d6ea0a63af33a4174fb1ad7115df631a4cb838f56e31f4cbd15498e1e9fdf9cc
2020-02-27 09:42:13 +08:00
Sebastian Falbesoner
b902bd66b0 test: check custom descendant limit in mempool_packages.py
To test the custom descendant limit on node1 (passed by the argument
-limitdescendantcount), we check for four conditions:
    -> the # of txs in the node1 mempool is equal to the limit
       (plus 1 for the parent tx, plus the # txs from the previous ancestor
        test which are still in)
    -> all txs in node1 mempool are a subset of txs in node0 mempool
    -> part of the constructed descendant-chain (the first ones up to the
       limit) are contained in node1 mempool
    -> the remaining part of the constructed descendant-chain (all after the
       first ones up to the limit) is *not* contained in node1 mempool
2020-02-27 00:24:33 +01:00
Wladimir J. van der Laan
e5d47ed8fd Merge #18167: Fix a violation of C++ standard rules where unions are used for type-punning
0653939ac1 Add static_asserts to ser_X_to_Y() methods (Samer Afach)
be94096dfb Fix a violation of C++ standard rules that unions cannot be switched. (Samer Afach)

Pull request description:

  Type punning in C++ is not like C. As per the C++ standard, one cannot use unions to convert the bit type. A discussion about this can be found [here](https://stackoverflow.com/questions/25664848/unions-and-type-punning). In C++, a union is supposed to only hold one type at a time. It's intended to be used only as `std::variant`. Switching types is undefined behavior.

  In fact, C++20 has a special casting function, called [`bit_cast`](https://en.cppreference.com/w/cpp/numeric/bit_cast) that solved this problem.

  Why has it been working so far? Because some compilers tolerate using unions and switching types, like gcc. More information [here](https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html#Type-punning).

  One important thing to mention is that performance is generally not affected by that memcpy. Compilers are smart enough to convert that to a memory cast when possible. But we have to do it the right way, otherwise, it's jut undefined behavior that depends on the compiler.

ACKs for top commit:
  practicalswift:
    ACK 0653939ac1
  elichai:
    ACK 0653939ac1
  laanwj:
    Code review ACK 0653939ac1
  kristapsk:
    ACK 0653939ac1

Tree-SHA512: f6e89de39fc964750429139bab6b5a1346f7060334b7afa020e315bdad8f8c195bce2b8a9e343f06e7fff175e2dfb1cdabfcb6fe405bea0febe4962f0cc62557
2020-02-26 19:00:24 +01:00
Wladimir J. van der Laan
89a97a71f2 Merge #17985: net: Remove forcerelay of rejected txs
facb71576c net: Remove forcerelay of rejected txs (MarcoFalke)

Pull request description:

  This removes the code that supposedly handled the forced relay of txs from a permissioned peer that were rejected from our mempool. The removal should be fine, because it is dead code for the following reasons:

  * While `RelayTransaction` enqueues the inv for all peers, the inv is never processed because it can not be found in the mempool. See 4a07233076/src/net_processing.cpp (L3862-L3866)

  * Even if the peers we intended to send the inv to can somehow reply with a getdata to the never-received inv, they won't receive the tx as a reply because it was never added to the "relay memory" (`mapRelay`)

  The dead code is (obviously) untested: https://marcofalke.github.io/btc_cov/total.coverage/src/net_processing.cpp.gcov.html#2574

  This feature was (intentionally or accidentally) removed in 4d8993b346, which was released in Bitcoin Core 0.13.0. So all currently supported versions of Bitcoin Core ship without this feature. I am not aware of any complaints about this feature or actual documented use-cases. So instead of reviving an unneeded feature, just remove the dead code.

ACKs for top commit:
  hebasto:
    ACK facb71576c, locally running the unit and functional tests.

Tree-SHA512: bfceae6f2983c1510fa0649a9a63c343cbbc1c4ab3a3698039cccf454c81e58c8f5114b147ed42a1bc867da74c43a5b53764ab14f942e191b6f59079044108b5
2020-02-26 18:46:05 +01:00
MarcoFalke
fab2527515 test: Disable mockforward scheduler unit test for now 2020-02-27 00:19:16 +07:00
Pieter Wuille
353f376277 Convert blockencodings.h to new serialization framework 2020-02-25 14:10:44 -08:00
Pieter Wuille
e574fff53e Add CustomUintFormatter 2020-02-25 14:10:38 -08:00
MarcoFalke
c3b4715923 Merge #18206: tests: Add fuzzing harness for bloom filter classes (CBloomFilter + CRollingBloomFilter)
eabbbe409f tests: Add fuzzing harness for rolling bloom filter class CRollingBloomFilter (practicalswift)
2a6a6ea0f5 tests: Add fuzzing harness for bloom filter class CBloomFilter (practicalswift)

Pull request description:

  Add fuzzing harness for bloom filter classes (`CBloomFilter` + `CRollingBloomFilter`).

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/bloom_filter
  …
  $ src/test/fuzz/rolling_bloom_filter
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK eabbbe409f 🤞

Tree-SHA512: 765d30bc52e3eb04dbd4d2b8f517387aa61312416e8fea3767250ef5c074e08641699019ee4600d42303de32f98379c20bfc0c0e60cb5154d0338088c1d29cb6
2020-02-26 02:37:43 +07:00
practicalswift
eabbbe409f tests: Add fuzzing harness for rolling bloom filter class CRollingBloomFilter 2020-02-25 17:04:03 +00:00
practicalswift
2a6a6ea0f5 tests: Add fuzzing harness for bloom filter class CBloomFilter 2020-02-25 17:04:03 +00:00
MarcoFalke
fa45d60646 test: Reduce unneeded whitelist permissions in tests 2020-02-26 00:00:29 +07:00
MarcoFalke
fa6b061fc1 rpc: Auto-format RPCResult 2020-02-25 22:35:58 +07:00
MarcoFalke
fa7d0503d3 rpc: Move OuterType enum to header
This is needed so that it can be used by RPCResult

Also,
* rename NAMED_ARG to NONE for generalization.
* change RPCArg constructors to initialize the members by moving values
2020-02-25 22:33:01 +07:00
Russell Yanofsky
3ce16ad2f9 refactor: Use psbt forward declaration
Also clean up forward other forward declarations in interfaces/wallet.h with !sort

Original motivation for this change was to fix a circular dependencies lint
error: "interfaces/chain.h -> interfaces/wallet.h -> psbt -> node/transaction
-> node/context -> interfaces/chain.h" from an earlier commit in this PR adding
a "interfaces/chain.h -> interfaces/wallet.h" include. Now, the wallet include
is no longer added, but it is still good to clean up the psbt include for
efficiency, and to sort the forward declarations.
2020-02-25 05:50:39 -05:00
Samuel Dobson
31c0006a6c Merge #17264: rpc: set default bip32derivs to true for psbt methods
5bad7921d0 [test] PSBT RPC: check that bip32_derivs are present by default (Sjors Provoost)
29a21c9061 [rpc] set default bip32derivs to true for psbt methods (Sjors Provoost)

Pull request description:

  In https://github.com/bitcoin/bitcoin/pull/13557#pullrequestreview-135905054 I recommended not including bip32 deriviation by default in PSBTs:

  > _Bit of a privacy issue_: let's say person A and B are about to spend from a multisig address, sending everything to person A. Person A gives their address to person B, their wallet wallet creates a PSBT, but doesn't sign it. Wallet A then calls `walletprocesspsbt` which signs it and _spontaneously adds the master_fingerprint and bip32 path_. Same issue with `walletcreatefundedpsbt`.
  >
  > Adding `bip32_derivs` should probably be opt-in.

  In practice I find this default quite annoying because I forget it and end up with a confused hardware wallet.

  More importantly, in the multisig example I provided, it's actually essential for the other side to know the derivation details (in addition to an xpub). This allows them to check that change is going to an address you can still co-sign for (because the spending policy is unchanged except for an index).

ACKs for top commit:
  instagibbs:
    utACK 5bad7921d0
  jonatack:
    ACK 5bad7921d0 code review, built, ran tests, inspected/messed around with/pprinted values from the new tests. Thanks for adding the tests.
  meshcollider:
    utACK 5bad7921d0

Tree-SHA512: 22ad71dda96856060a96758c4ae7aafa22d5e9efba30e0c8287c711e7579849bd72593cbc0f41a2e9e8821315d78bda04e848dbb006283b841b2795e2faebcfd
2020-02-25 23:50:39 +13:00
Russell Yanofsky
1dde238f2c Add ChainClient setMockTime, getWallets methods
Needed to set mock times, and get wallet interface pointers correctly when
wallet code is running in a different process from node code.
2020-02-25 05:50:39 -05:00
Samuel Dobson
03f98b15ad Merge #17577: refactor: deduplicate the message sign/verify code
e193a84fb2 Refactor message hashing into a utility function (Jeffrey Czyz)
f8f0d9893d Deduplicate the message signing code (Vasil Dimov)
2ce3447eb1 Deduplicate the message verifying code (Vasil Dimov)

Pull request description:

  The message signing and verifying logic was replicated in a few places
  in the code. Consolidate in a newly introduced `MessageSign()` and
  `MessageVerify()` and add unit tests for them.

ACKs for top commit:
  Sjors:
    re-ACK e193a84fb2
  achow101:
    ACK e193a84fb2
  instagibbs:
    utACK e193a84fb2
  meshcollider:
    utACK e193a84fb2

Tree-SHA512: b0e02a7d4623a98c8f8c77627af1725e6df07700de4630c2f75da6beacdf55414c38ba147bc6d2a757491ab07c827dddf93e8632fe600478760e255714ddab88
2020-02-25 23:29:54 +13:00
fanquake
a674e89d27 Merge #18162: util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value
12a2f37718 util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t nTime) by checking gmtime_s/gmtime_r return value (practicalswift)

Pull request description:

  Avoid potential uninitialized read in `FormatISO8601DateTime(int64_t)` by checking `gmtime_s`/`gmtime_r` return value.

  Before this patch `FormatISO8601DateTime(67768036191676800)` resulted in:

  ```
  ==5930== Conditional jump or move depends on uninitialised value(s)
  ==5930==    at 0x4F44C0A: std::ostreambuf_iterator<char, std::char_traits<char> > std::num_put<char, std::ostreambuf_iterator<char, std::char_traits<char> > >::_M_insert_int<long>(std::ostreambuf_iterator<char, std::char_traits<char> >, std::ios_base&, char, long) const (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
  ==5930==    by 0x4F511A4: std::ostream& std::ostream::_M_insert<long>(long) (in /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.25)
  ==5930==    by 0x4037C3: void tinyformat::formatValue<int>(std::ostream&, char const*, char const*, int, int const&) (tinyformat.h:358)
  ==5930==    by 0x403725: void tinyformat::detail::FormatArg::formatImpl<int>(std::ostream&, char const*, char const*, int, void const*) (tinyformat.h:543)
  ==5930==    by 0x402E02: tinyformat::detail::FormatArg::format(std::ostream&, char const*, char const*, int) const (tinyformat.h:528)
  ==5930==    by 0x401B16: tinyformat::detail::formatImpl(std::ostream&, char const*, tinyformat::detail::FormatArg const*, int) (tinyformat.h:907)
  ==5930==    by 0x4017AE: tinyformat::vformat(std::ostream&, char const*, tinyformat::FormatList const&) (tinyformat.h:1054)
  ==5930==    by 0x401765: void tinyformat::format<int, int, int, int, int, int>(std::ostream&, char const*, int const&, int const&, int const&, int const&, int const&, int const&) (tinyformat.h:1064)
  ==5930==    by 0x401656: std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<int, int, int, int, int, int>(char const*, int const&, int const&, int const&, int const&, int const&, int const&) (tinyformat.h:1073)
  ==5930==    by 0x4014CC: FormatISO8601DateTime[abi:cxx11](long) (…)
  ```

  The same goes for other very large positive and negative arguments.

  Fix by simply checking the `gmtime_s`/`gmtime_r` return value :)

ACKs for top commit:
  MarcoFalke:
    ACK 12a2f37718
  theStack:
    re-ACK 12a2f37718
  elichai:
    re ACK 12a2f37718

Tree-SHA512: 066142670d9bf0944d41fa3f3c702b1a460b5471b93e76a619b1e818ff9bb9c09fe14c4c37e9536a04c99533f7f21d1b08ac141e1b829ff87ee54c80d0e61d48
2020-02-25 10:06:38 +08:00
MarcoFalke
225aa5d6d5 Merge #18193: scripted-diff: Wallet: Rename incorrectly named *UsedDestination
bca8665d08 scripted-diff: Wallet: Rename incorrectly named *UsedDestination (Luke Dashjr)

Pull request description:

  These functions are used to mark/check if a key of our own has been used to spend (and only for avoid-reuse wallets), which has nothing to do with the destination/address itself.
  Give them more accurate names to avoid confusion.

  -BEGIN VERIFY SCRIPT-
  sed -i -e 's/UsedDestination/SpentKey/g' $(git grep -l 'UsedDestination' ./src)
  -END VERIFY SCRIPT-

ACKs for top commit:
  practicalswift:
    ACK bca8665d08 -- patch looks correct and rationale makes sense
  instagibbs:
    ACK bca8665d08, much more meaningful name, thanks
  kallewoof:
    ACK bca8665d08

Tree-SHA512: ff13d9061ffa748e92eb41ba962c3ec262a43e4b6abd62408b38c6f650395d6ae5851554257d1900fb02767a88d08380d592a27210192ee9abb72d0945976686
2020-02-24 23:01:24 +07:00
Yancy Ribbens
c72a11a1a0 test: Add cost_of_change parameter assertions to bnb_search_test 2020-02-22 11:11:09 -06:00
MarcoFalke
ab9de43588 Merge #18181: test: Remove incorrect assumptions in validation_flush_tests
faca8eff39 test: Remove incorrect assumptions in validation_flush_tests (MarcoFalke)
fa31eebfe9 test: Tabs to spaces in all tests (MarcoFalke)

Pull request description:

  The tests assume standard library internals that may not hold on all supported archs or when the code is instrumented for sanitizer or debug use cases

  Fixes #18111

ACKs for top commit:
  jamesob:
    ACK faca8eff39 pending passing tests
  fjahr:
    ACK faca8eff39

Tree-SHA512: 60a5ae824bdffb0762f82f67957b31b185385900be5e676fcb12c23d53f5eea734601680c2e3f0bdb8052ce90e7ca1911b1342affb67e43d91a506b111406f41
2020-02-22 22:18:46 +07:00
MarcoFalke
36e507227e Merge #18183: test: Set catch_system_errors=no on boost unit tests
fac52dafa0 test: Set catch_system_errors=no on boost unit tests (MarcoFalke)

Pull request description:

  Closes #16700

  Can be tested by adding an `assert(0)` and then running either `make check` or `./src/test/test_bitcoin -t bla_tests --catch_system_errors=no/yes`

ACKs for top commit:
  practicalswift:
    ACK fac52dafa0
  Empact:
    Tested ACK fac52dafa0

Tree-SHA512: ec00636951b2c1137aaf43610739d78d16f823f7da76a726d47f93b8b089766fb66b21504b3c5413bcf8b6b5c3db0ad74027d677db24a44487d6d79a6bdee2e0
2020-02-21 15:00:22 -08:00
Luke Dashjr
bca8665d08 scripted-diff: Wallet: Rename incorrectly named *UsedDestination
These functions are used to mark/check if a key of our own has been used to spend (and only for avoid-reuse wallets), which has nothing to do with the destination/address itself.
Give them more accurate names to avoid confusion.

-BEGIN VERIFY SCRIPT-
sed -i -e 's/UsedDestination/SpentKey/g' $(git grep -l 'UsedDestination' ./src)
-END VERIFY SCRIPT-
2020-02-21 21:16:40 +00:00
Luke Dashjr
c31bc5bcfd Consolidate service flag bit-to-name conversion to a shared serviceFlagToStr function
Side effect: this results in the RPC showing unknown service bits as "UNKNOWN[n]" like the GUI.

Note that there is no common mask-to-vector<string> function because both GUI and RPC would need to iterate through it to convert to their desired target formats.
2020-02-21 19:10:04 +00:00
Luke Dashjr
cea91a1e40 Bugfix: GUI: Use unsigned long long type to avoid implicit conversion of MSB check 2020-02-21 19:10:04 +00:00
Samuel Dobson
9dd7bd47be Merge #18034: Get the OutputType for a descriptor
7e80f646b2 Get the OutputType for a descriptor (Andrew Chow)

Pull request description:

  Adds a `GetOutputType()` method to get the OutputType of a descriptor. Some descriptors don't have a determinate OutputType, so we actually use an `Optional<OutputType>`. For descriptors with indeterminate OutputType, we return `nullopt`.

  `addr()` and `raw()` use OutputTypes as determined by the CTxDestination they have. For simplicity, `ScriptHash` destinations are `LEGACY` even though they could be `P2SH_SEGWIT`.
  `combo()`, `pk()`, and `multi()` are `nullopt` as they either don't have an OutputType or they have multiple. `DescriptorImpl` defaults to `nullopt`.
  `pkh()` is `LEGACY` as expected
  `wpkh()` and `wsh()` are `BECH32` as expected.
  `sh()` checks whether the sub-descriptor is `BECH32`. If so, it is `P2SH_SEGWIT`. Otherwise it is `LEGACY`.

  The descriptor tests are updated to check the OutputType too.

ACKs for top commit:
  fjahr:
    ACK 7e80f646b2
  meshcollider:
    utACK 7e80f646b2
  instagibbs:
    cursory ACK 7e80f646b2
  Sjors:
    Code review ACK 7e80f646b2
  jonatack:
    ACK 7e80f64 code review/build/tests

Tree-SHA512: c5a813447b62e982435e1c948066f8d6c148c9ebffb0a5eb5a9028b173b01d5ead2f076a5ca3f7f37698538baa346f82a977ee48f583d89cb4e5ebd9111b2341
2020-02-22 08:02:52 +13:00
MarcoFalke
e9fc8f6e7f Merge #18172: test: Transaction expiry from mempool
d6d2602a32 add: test that transactions expire from mempool (0xb10c)

Pull request description:

  This adds the functional test `mempool_expiry.py` covering mempool transaction expiry. Both the default `DEFAULT_MEMPOOL_EXPIRY` of 336 hours (two weeks, set in #9312) and the user definable mempool expiry via the `-mempoolexpiry=<n>` command line option are tested. The test checks that descendants of expired transactions are removed as well.

  *Notes for reviewers*
  - `LimitMempoolSize()` (which is the only caller of `CTxMemPool::Expire()`) is only called when a transaction is added to the mempool. In order to test expiry of a transaction-that-should-expire, the mocktime is set and a random transaction is broadcast to trigger `LimitMempoolSize()`. The transaction-that-should-expire is then checked for expiry. LMK if there is another way, but I don't think there is.

ACKs for top commit:
  MarcoFalke:
    ACK d6d2602a32
  theStack:
    ACK d6d2602a32
  promag:
    Code review ACK d6d2602a32.

Tree-SHA512: eb68cd9e2d870872b8e8e1522fed8954fb99cc9e4edda4b28bb2a4e41cddbc53fe6f7d9c090f1e0e98ab49beb24bf37ff3787a9e9801a95e8ae9ca9eb34fe6f0
2020-02-21 10:23:20 -08:00
MarcoFalke
fae86c38bc util: Remove unused MilliSleep 2020-02-21 10:06:27 -08:00
MarcoFalke
fa9af06d91 scripted-diff: Replace MilliSleep with UninterruptibleSleep
This is safe because MilliSleep is never executed in a boost::thread,
the only type of thread that is interruptible.

* The RPC server uses std::thread
* The wallet is either executed in an RPC thread or the main thread
* bitcoin-cli, benchmarks and tests are only one thread (the main thread)

-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/MilliSleep\((\S+)\);/UninterruptibleSleep(std::chrono::milliseconds{\1});/g' $(git grep -l MilliSleep)
-END VERIFY SCRIPT-
2020-02-21 10:06:21 -08:00
MarcoFalke
fa4620be78 util: Add UnintrruptibleSleep 2020-02-21 09:49:37 -08:00
fanquake
eb3c6b0912 Merge #18070: doc: add note about brew doctor
63ce882760 doc: link to homebrew's troubleshooting page (Gastón I. Silva)

Pull request description:

  A trivial documentation update.

  When I was following the build steps for mac, I had some errors installing the dependencies. After searching on the Internet, and correcting the errors, I found that `brew doctor` had all the answers I needed. Could have skipped the Internet searches all together.

ACKs for top commit:
  fanquake:
    ACK 63ce882760 - a link to the troubleshooting page seems fine. I wouldn't really want our README to have anything more specific than that.

Tree-SHA512: 12c96cd9c9bd39ada21f3f27cbec3ed4bef4b8e74dec7872c892fc6a92a70418a5cc0882ff449883e91d96c01e1ca7104b076590917f397334c82931ec7fda1c
2020-02-20 20:56:41 +08:00
fanquake
56fc2dfcc3 Merge #18122: rpc: update validateaddress RPCExamples to bech32
7f1475c711 rpc: update validateaddress RPCExamples to bech32 (Sebastian Falbesoner)

Pull request description:

  Another small step to get rid of legacy addresses in the RPC help texts and by that encourage the use of bech32 addresses by default. The (invalid) address is the same as in the `getaddressinfo` RPC (see 2ee0cb3330, kudos to jonatack!), I don't think it adds any value to have a different example address per RPC.

ACKs for top commit:
  fanquake:
    ACK 7f1475c711
  MarcoFalke:
    ACK 7f1475c711

Tree-SHA512: 2350f61fa942a9053f9f5c860ea446965dc7209c71c81bdb98a859d03ca23b225ad72c9c506e4a55c8d8988823d9cfbe808c1a452a1eeadb70ab186b146dd4ca
2020-02-20 20:28:46 +08:00
MarcoFalke
fac52dafa0 test: Set catch_system_errors=no on boost unit tests 2020-02-19 16:14:50 -08:00
practicalswift
12a2f37718 util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t nTime) by checking gmtime_s/gmtime_r return value 2020-02-19 22:41:06 +00:00
MarcoFalke
faca8eff39 test: Remove incorrect assumptions in validation_flush_tests 2020-02-19 11:52:25 -08:00
MarcoFalke
fa31eebfe9 test: Tabs to spaces in all tests
Spaces are used in all of the source code except in these two instances
2020-02-19 11:51:40 -08:00
Samer Afach
0653939ac1 Add static_asserts to ser_X_to_Y() methods 2020-02-19 18:44:46 +01:00
MarcoFalke
eddcbfb109 Merge #18166: ci: Run fuzz testing test cases (bitcoin-core/qa-assets) under valgrind to catch memory errors
f2472f6460 tests: Improve test runner output in case of target errors (practicalswift)
733bbec34f tests: Add --exclude integer,parse_iso8601 (temporarily) to make Travis pass until uninitialized read issue in FormatISO8601DateTime is fixed (practicalswift)
5ea81449f3 tests: Add support for excluding fuzz targets using -x/--exclude (practicalswift)
555236f769 tests: Remove -detect_leaks=0 from test/fuzz/test_runner.py - no longer needed (practicalswift)
a3b539a924 ci: Run fuzz testing test cases under valgrind (practicalswift)

Pull request description:

  Run fuzz testing [test cases (bitcoin-core/qa-assets)](https://github.com/bitcoin-core/qa-assets) under `valgrind`.

  This would have caught `util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t) by checking gmtime_s/gmtime_r return value` (#18162) and similar cases.

ACKs for top commit:
  MarcoFalke:
    ACK f2472f6460 👼

Tree-SHA512: bb0879d40167cf6906bc0ed31bed39db83c39c7beb46026f7b0ee53f28ff0526ad6fabc3f4cb3f5f18d3b8cafdcbf5f30105b35919f4e83697c71e838ed71493
2020-02-19 08:20:38 -08:00
practicalswift
f2472f6460 tests: Improve test runner output in case of target errors 2020-02-19 14:27:19 +00:00
practicalswift
733bbec34f tests: Add --exclude integer,parse_iso8601 (temporarily) to make Travis pass until uninitialized read issue in FormatISO8601DateTime is fixed 2020-02-19 14:11:54 +00:00
practicalswift
5ea81449f3 tests: Add support for excluding fuzz targets using -x/--exclude 2020-02-19 14:10:22 +00:00
practicalswift
555236f769 tests: Remove -detect_leaks=0 from test/fuzz/test_runner.py - no longer needed 2020-02-19 13:36:03 +00:00
0xb10c
d6d2602a32 add: test that transactions expire from mempool
This tests that a mempool transaction expires after a given timeout
and its children are removed as well.

Both the default expiry timeout defied by DEFAULT_MEMPOOL_EXPIRY and
a user definable expiry timeout via the -mempoolexpiry=<n> command
line argument (<n> is the timeout in hours) are tested.
2020-02-19 10:03:48 +01:00
Samuel Dobson
68e841e0af Merge #18067: wallet: Improve LegacyScriptPubKeyMan::CanProvide script recognition
a304a3632f Revert "Store p2sh scripts in AddAndGetDestinationForScript" (Russell Yanofsky)
eb7d8a5b07 [test] check for addmultisigaddress regression (Sjors Provoost)
005f8a92cc wallet: Improve LegacyScriptPubKeyMan::CanProvide script recognition (Russell Yanofsky)

Pull request description:

  Make `LegacyScriptPubKeyMan::CanProvide` method able to recognize p2sh scripts when the redeem script is present in the `mapScripts` map without the p2sh script also having to be added to the `mapScripts` map. This restores behavior prior to #17261, which I think broke backwards compatibility with old wallet files by no longer treating addresses created by `addmultisigaddress` calls before #17261 as solvable.

  The reason why tests didn't fail with the CanProvide implementation in #17261 is because of a workaround added in 4a7e43e846 "Store p2sh scripts in AddAndGetDestinationForScript", which masked the problem for new `addmultisigaddress` RPC calls without fixing it for multisig addresses already created in old wallet files.

  This change adds a lot of comments and allows reverting commit 4a7e43e846 "Store p2sh scripts in AddAndGetDestinationForScript", so the `AddAndGetDestinationForScript()` function, `CanProvide()` method, and `mapScripts` map should all be more comprehensible

ACKs for top commit:
  Sjors:
    re-ACK a304a3632f (rebase, slight text changes and my test)
  achow101:
    re-ACK a304a3632f
  meshcollider:
    utACK a304a3632f

Tree-SHA512: 03b625220c49684c376a8062d7646aeba0e5bfe043f977dc7dc357a6754627d594e070e4d458d12d2291888405d94c1dbe08c7787c318374cedd5755e724fb6e
2020-02-19 14:28:41 +13:00
Filip Gospodinov
aff2748f8a httpserver: use own HTTP status codes
Before, macros defined in `<event2/http.h>` have been used
for some HTTP status codes.
`<event2/http.h>` is included implicitly and the usage
of its status code macros is inconsistent with the majority
HTTP response implementations in this file.

Now, the `HTTPStatusCode` enum from `<rpc/protocol.h>` is
consistently used for all HTTP response implementations.
2020-02-18 08:29:35 +01:00
practicalswift
a3b539a924 ci: Run fuzz testing test cases under valgrind 2020-02-18 06:56:26 +00:00
Travin Keith
7ba962276e doc: Minor grammatical changes and flow improvements
Grammar:

Line 49: There shouldn't be a period at the end of a phrase.

Lines 56, 57, 116, 137, and 177: Adding necessary commas

Lines 103 and 136: Run-on sentence issues fixed. 

Line 176: Fixed punctuation and added necessary conjunction

Line 178: Singular noun when it should be plural

Flow:

Line 49: Adding "for" makes it more natural. 

Line 54: Though it's not grammatically incorrect to end a sentence with a preposition in an informal document such as this, the word "followed" is much easier to understand anyway, especially for those who don't have English as their native language.
2020-02-18 04:32:35 +01:00
MarcoFalke
36f42e1bf4 Merge #18037: Util: Allow scheduler to be mocked
8bca30ea17 [rpc] expose ability to mock scheduler via the rpc (Amiti Uttarwar)
7c8b6e5b52 [lib] add scheduler to node context (Amiti Uttarwar)
930d837542 [test] add chainparams property to indicate chain allows time mocking (Amiti Uttarwar)
1cd43e83c6 [test] unit test for new MockForward scheduler method (Amiti Uttarwar)
a6f63598ad [util] allow scheduler to be mocked (Amiti Uttarwar)

Pull request description:

  This PR is to support functional tests by allowing the scheduler to be mocked via the RPC.

  It adds a `MockForward` method to the scheduler class that iterates through the task queue and reschedules them to be `delta_seconds` sooner.

  This is currently used to support functional testing of the "unbroadcast" set tracking in #18038. If this patch is accepted, it would also be useful to simplify the code in #16698.

ACKs for top commit:
  MarcoFalke:
    ACK 8bca30ea17, only change is some style fixups 🕓

Tree-SHA512: 2a97fe8ade2b7fd1fb5cdfa1dcafb3227a377d7a847e3845a228bc119eb77824b4aefa43d922a06d583939b22725e223f308cf092961048079d36f6b1d9a639b
2020-02-17 17:01:50 -08:00
Amiti Uttarwar
8bca30ea17 [rpc] expose ability to mock scheduler via the rpc 2020-02-17 14:49:34 -08:00
Amiti Uttarwar
7c8b6e5b52 [lib] add scheduler to node context
- also update test setup & access point in denial of service test
2020-02-17 14:49:34 -08:00
Samer Afach
be94096dfb Fix a violation of C++ standard rules that unions cannot be switched. 2020-02-17 20:53:50 +01:00
Wladimir J. van der Laan
179504ccb6 Merge #17948: build: pass -fno-ident in Windows gitian descriptor
530d02addb build: pass -fno-ident in Windows gitian descriptor (fanquake)

Pull request description:

  `-fno-ident` prevents compilers from emitting compiler name and version number information that can needlessly bloat binaries.

  For example, in the `v0.19.0.1` Windows release binaries, there are > 1000 GCC compiler version strings embedded:
  ```bash
  # GCC: (GNU) 7.3-posix 20180312... & GCC: (GNU) 6.3.0 20170415.......
  strings bitcoind.exe | rg GCC | wc -l
      1021
  ```

  They end up collected in the end of the`.rdata` section, and cannot be removed by `strip`. i.e:

  ```bash
  objdump --section=.rdata --full-contents bitcoind.exe
  ...
   cfcc00 00000000 00000000 00000000 00000000  ................
   cfcc10 00000000 00000000 00000000 00000000  ................
   cfcc20 4743433a 2028474e 55292036 2e332e30  GCC: (GNU) 6.3.0
   cfcc30 20323031 37303431 35000000 00000000   20170415.......
   cfcc40 4743433a 2028474e 55292037 2e332d70  GCC: (GNU) 7.3-p
   cfcc50 6f736978 20323031 38303331 32000000  osix 20180312...
   cfcc60 4743433a 2028474e 55292037 2e332d70  GCC: (GNU) 7.3-p
   cfcc70 6f736978 20323031 38303331 32000000  osix 20180312...
  ```

  The flag is available for [Clang](https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-qn) and [GCC](https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-ident).

  Relevant code in [GCC](https://github.com/gcc-mirror/gcc/blob/master/gcc/toplev.c#L565-L578):
  ```c
    /* Attach a special .ident directive to the end of the file to identify
       the version of GCC which compiled this code.  The format of the .ident
       string is patterned after the ones produced by native SVR4 compilers.  */
    if (!flag_no_ident)
      {
        const char *pkg_version = "(GNU) ";
        char *ident_str;

        if (strcmp ("(GCC) ", pkgversion_string))
  	pkg_version = pkgversion_string;

        ident_str = ACONCAT (("GCC: ", pkg_version, version_string, NULL));
        targetm.asm_out.output_ident (ident_str);
      }
  ```

ACKs for top commit:
  practicalswift:
    ACK 530d02addb
  laanwj:
    ACK 530d02addb

Tree-SHA512: b3b28f43ec483dee28d1df8548fe72425bf00e750701825c256395f6aa7b23256eb27609b51779b86aed108b6eaa3912181a9d8282e23eebf9cee7784f9fabe0
2020-02-17 12:05:44 +01:00
Wladimir J. van der Laan
051439813e Merge #13339: wallet: Replace %w by wallet name in -walletnotify script
4e9efac678 test: Check wallet name in -walletnotify script (João Barbosa)
9a5b5ee81f wallet: Replace %w by wallet name in -walletnotify script (João Barbosa)

Pull request description:

  Fixes #13237.

ACKs for top commit:
  laanwj:
    ACK 4e9efac678

Tree-SHA512: 189dd1c785485f2e974d7c12531851b2a977778b3b954aa95efd527322ba3345924cfd587fb9c90b0fa979202af0ab2d90e53d125fe266a36c94f757e4176203
2020-02-17 11:59:23 +01:00
MarcoFalke
263f53e2d0 Merge #18098: scripted-diff: Add missing spaces in RPCResult, Normalize type names
fad027fb0c scripted-diff: Add missing spaces in RPCResult, Fix type names (MarcoFalke)

Pull request description:

  This makes the rendered diff smaller when the RPCResult is machine generated later on (Previous attempts: #14601 and #14459)

ACKs for top commit:
  Sjors:
    ACK fad027fb0c

Tree-SHA512: 48afd571b1cd349ca0b29bb444c1c7cda657e07dd96c610d479f931ccd938186aec98e533d0552b5b10afc9a3d7b911359260a49448e8e1106e3647b2c71f3ba
2020-02-16 17:26:21 -08:00
MarcoFalke
94c0287aec Merge #18159: tests: Add --valgrind option to test/fuzz/test_runner.py for running fuzzing test cases under valgrind
1b068c50dd tests: Add --valgrind option to test/fuzz/test_runner.py for running fuzzing test cases under valgrind (practicalswift)

Pull request description:

  Add `--valgrind` option to `test/fuzz/test_runner.py` for running fuzzing test cases under `valgrind`.

  Test this PR using:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=fuzzer
  $ make
  $ git clone https://github.com/bitcoin-core/qa-assets
  $ test/fuzz/test_runner.py --valgrind -l DEBUG qa-assets/fuzz_seed_corpus/
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 1b068c50dd 🌒

Tree-SHA512: e6eb99af1bceaa6f36f49092a05de415848099ccc1497cc098a62e925954c978cb37a46410b44ed5eef2c6464ca4ecb06397b75b5d35701f5a8525436e47b9fd
2020-02-16 09:47:54 -08:00
MarcoFalke
33861a8367 Merge #18145: build: add Wreturn-type to Werror flags, check on more Travis machines
c98c26ee99 ci: use --enable-werror on more hosts (Sjors Provoost)
6ba617dbe2 build: add Wreturn-type to Werror flags (Sjors Provoost)

Pull request description:

  I overlooked a missing `return false` in https://github.com/bitcoin/bitcoin/pull/17577#discussion_r379137776 and the warning only showed up on one Travis machine (`warning: control reaches end of non-void function [-Wreturn-type]`).

  This PR promotes `Wreturn-type` to an error when configured with `--enable-werror`. I also added `--enable-werror` to the Travis machine that happened to catch this particular instance.

ACKs for top commit:
  vasild:
    ACK c98c26e.
  practicalswift:
    ACK c98c26ee99

Tree-SHA512: 64e86c67fef2c5048aab201a8400b7e4a6f27b93d626159ba0b2807b5f119d2b0a83e3372db88f692cb4b0d059722d6a642d130c74a4f991a27f3a6b21780b5f
2020-02-16 09:22:06 -08:00
João Barbosa
0933a37078 gui: Avoid Wallet::GetBalance in WalletModel::pollBalanceChanged 2020-02-16 11:38:49 +00:00
practicalswift
1b068c50dd tests: Add --valgrind option to test/fuzz/test_runner.py for running fuzzing test cases under valgrind 2020-02-16 11:11:54 +00:00
Russell Yanofsky
10633398f2 Add DifferenceFormatter 2020-02-15 19:49:24 -08:00
Russell Yanofsky
56dd9f04c7 Make VectorFormatter support stateful formatters 2020-02-15 19:49:24 -08:00
MarcoFalke
2a2631fb0d Merge #18108: Fix .gitignore policy in build_msvc directory
dcb7af053d Ignore only auto-generated .vcxproj files (Hennadii Stepanov)
79c811ca2b Specify ignored bitcoin-qt file precisely (Hennadii Stepanov)

Pull request description:

  This PR makes `git` to ignore only auto-generated `.vcxproj` files, i.e.:
  4c2578706c/build_msvc/msvc-autogen.py (L14-L25)

  Fix #17287

ACKs for top commit:
  sipsorcery:
    ACK dcb7af053d.

Tree-SHA512: 5acfc3cb50e1239655bc9715711c8a0489093ed7cc287756614740f67908f11a55976cec75b29e263d4d3579b6b8af99bb3823515d43adf204cd5935fb2e1a3f
2020-02-15 10:40:44 -08:00
Pieter Wuille
3ca574cef0 Convert CCompactSize to proper formatter 2020-02-14 19:22:39 -08:00
Jeffrey Czyz
e193a84fb2 Refactor message hashing into a utility function
And add unit test for it.

The purpose of using a preamble or "magic" text as part of signing and
verifying a message was not given when the code was repeated in a few
locations. Make a test showing how it is used to prevent inadvertently
signing a transaction.
2020-02-14 10:45:41 +01:00
Vasil Dimov
f8f0d9893d Deduplicate the message signing code
The logic of signing a message was duplicated in 3 places:

src/qt/signverifymessagedialog.cpp
  SignVerifyMessageDialog::on_signMessageButton_SM_clicked()

src/rpc/misc.cpp
  signmessagewithprivkey()

src/wallet/rpcwallet.cpp
  signmessage()

Move the logic into

src/util/message.cpp
  MessageSign()

and call it from all the 3 places.
2020-02-14 10:45:40 +01:00
Vasil Dimov
2ce3447eb1 Deduplicate the message verifying code
The logic of verifying a message was duplicated in 2 places:

src/qt/signverifymessagedialog.cpp
  SignVerifyMessageDialog::on_verifyMessageButton_VM_clicked()

src/rpc/misc.cpp
  verifymessage()

with the only difference being the result handling. Move the logic into
a dedicated

src/util/message.cpp
  MessageVerify()

which returns a set of result codes, call it from the 2 places and just
handle the results differently in the callers.
2020-02-14 10:45:40 +01:00
Sjors Provoost
c98c26ee99 ci: use --enable-werror on more hosts 2020-02-14 10:40:23 +01:00
Sjors Provoost
6ba617dbe2 build: add Wreturn-type to Werror flags
This is supported by GCC and Clang.
https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
https://clang.llvm.org/docs/DiagnosticsReference.html#wreturn-type
2020-02-14 10:40:23 +01:00
Wladimir J. van der Laan
470664f2b7 Merge #17746: refactor: rpc: Remove vector copy from listtransactions
25bc17fceb refactor: rpc: Remove vector copy from listtransactions (João Barbosa)

Pull request description:

  Current approach
   - copy accumulated `ret` vector to `arrTmp`
   - drop unnecessary elements from `arrTmp`
   - reverse `arrTmp`
   - clear `ret`
   - copy `arrTmp` to the `ret`

  New approach
   - create a vector from the accumulated `ret` with just the necessary elements already reversed
   - copy it to the result

  This PR doesn't change behavior.

ACKs for top commit:
  ryanofsky:
    Code review ACK 25bc17fceb. Just comment and commit message tweaks since last review

Tree-SHA512: 87906561e3accdbdb0f4a8194cbcd76ea53ae53d0ce135b90bc54a5f77e300b14ef08505e7daf1fe52426f135442a743da5a027416a769bd454922357cebe7c0
2020-02-13 18:50:02 +01:00
Amiti Uttarwar
930d837542 [test] add chainparams property to indicate chain allows time mocking 2020-02-13 08:59:51 -08:00
Amiti Uttarwar
1cd43e83c6 [test] unit test for new MockForward scheduler method 2020-02-13 08:59:51 -08:00
Amiti Uttarwar
a6f63598ad [util] allow scheduler to be mocked
Add MockForward method to the scheduler that mimics going into the future by rescheduling all items on the taskQueue to be sooner.
2020-02-13 08:59:51 -08:00
João Barbosa
25bc17fceb refactor: rpc: Remove vector copy from listtransactions
No change in behavior.
2020-02-13 15:43:35 +00:00
Sebastian Falbesoner
7f1475c711 rpc: update validateaddress RPCExamples to bech32
also contains the following changes:
- rpc: factor out example bech32 address for RPCExamples
- doc: update developer notes wrt RPCExamples addresses
 (mention the EXAMPLE_ADDRESS constant as an example for an invalid bech32
  address suitable for RPCExamples help documentation)
2020-02-13 12:57:37 +01:00
fanquake
530d02addb build: pass -fno-ident in Windows gitian descriptor
This prevents compilers from emitting compiler name and
version number info that can needlessly bloat binaries.

Accepted by Clang and GCC. See:

https://clang.llvm.org/docs/ClangCommandLineReference.html#cmdoption-clang-qn

https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fno-ident
2020-02-13 18:20:43 +08:00
Jonas Schnelli
0c20809da8 Merge #18121: gui: Throttle GUI update pace when -reindex
c9fe61291e gui: Throttle GUI update pace when -reindex (Hennadii Stepanov)

Pull request description:

  This is grabbed from #17565.

  All **laanwj**'s and **ryanofsky**'s suggestions are implemented.

  With this PR,  the GUI does not freeze when a user runs:
  ```
  $ ./src/qt/bitcoin-qt -reindex
  ```

ACKs for top commit:
  jonasschnelli:
    utACK c9fe61291e

Tree-SHA512: c7be316cb73d3d286bdf8429a960f71777d13a73d059869a64e23ad276499252b561a3a5b9613c4c1ad58cc0de26283c1ec72be745c401f604eaa05f70bf7d64
2020-02-13 08:48:07 +01:00
Jonas Schnelli
b6a16fa44e Merge #18123: gui: Fix race in WalletModel::pollBalanceChanged
bf36a3ccc2 gui: Fix race in WalletModel::pollBalanceChanged (Russell Yanofsky)

Pull request description:

  Poll function was wrongly setting cached height to the current chain height instead of the chain height at the time of polling.

  This bug could cause balances to appear out of date, and was first introduced a0704a8996 (diff-2e3836af182cfb375329c3463ffd91f8L117). Before that commit, there wasn't a problem because cs_main was held during the poll update.

  Currently, the problem should be rare. But if 8937d99ce81a27ae5e1012a28323c0e26d89c50b from #17954 were merged, the problem would get worse, because the wrong cachedNumBlocks value would be set if the wallet was polled in the interval between a block being connected and it processing the BlockConnected notification.

  MarcoFalke also points out that a0704a8996 could lead to GUI hangs as well, because previously the pollBalanceChanged method, which runs on the GUI thread, would only make a nonblocking TRY_LOCK(cs_main) call, but after could make blocking LOCK(cs_main) calls, potentially locking up the GUI.

  Thanks to John Newbery for finding this bug this while reviewing https://github.com/bitcoin/bitcoin/pull/17954.

ACKs for top commit:
  Empact:
    utACK bf36a3ccc2
  jonasschnelli:
    utACK bf36a3c

Tree-SHA512: 1f4f229fa70a6d1fcf7be3806dca3252e86bc1755168fb421258389eb95aae67f863cb1216e6dc086b596c33560d1136215a4c87b5ff890abc8baaa3333b47f4
2020-02-13 08:44:36 +01:00
Hennadii Stepanov
c9fe61291e gui: Throttle GUI update pace when -reindex
Co-authored-by: Barry Deeney <mxaddict@codedmaster.com>
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-02-12 23:51:41 +02:00
Pieter Wuille
c8e24ddce3 [REFACTOR] Abstract out script execution out of VerifyWitnessProgram()
This removes the unclear reliance on "falling through" to get to the
script execution part.

Also fix some code style issues.
2020-02-12 11:20:38 -08:00
Russell Yanofsky
a304a3632f Revert "Store p2sh scripts in AddAndGetDestinationForScript"
This reverts commit 4a7e43e846.
2020-02-12 11:48:30 -05:00
Wladimir J. van der Laan
2bdc476d4d Merge #17708: prevector: avoid misaligned member accesses
5f26855f10 test: Remove ubsan alignment suppressions (Wladimir J. van der Laan)
9d933ef919 prevector: avoid misaligned member accesses (Anthony Towns)

Pull request description:

  Ensure prevector data is appropriately aligned. Earlier discussion in #17530.

  **Edit laanwj**: In contrast to #17530, it does this without increase in size of any of the coin cache data structures (x86_64, clang)

  | Struct        | (size,align) before           | (size,align) after  |
  | ------------- | ------------- | ------- |
  | Coin | 48, 8        |     48, 8   |
  | CCoinsCacheEntry | 56, 8    |   56, 8  |
  | CScript | 32, 1       |      32, 8  |

ACKs for top commit:
  laanwj:
    ACK 5f26855f10
  practicalswift:
    ACK 5f26855f10
  jonatack:
    ACK 5f26855f10

Tree-SHA512: 98d112d6856f683d5b212410b73f3071d2994f1efb046a2418a35890aa1cf1aa7c96a960fc2e963fa15241e861093c1ea41951cf5b4b5431f88345eb1dd0a98a
2020-02-12 17:48:30 +01:00
Sjors Provoost
eb7d8a5b07 [test] check for addmultisigaddress regression 2020-02-12 17:48:30 +01:00
Russell Yanofsky
005f8a92cc wallet: Improve LegacyScriptPubKeyMan::CanProvide script recognition
Make LegacyScriptPubKeyMan::CanProvide method able to recognize p2sh scripts
when the redeem script is present in the mapScripts map without the p2sh script
also having to be added to the mapScripts map. This restores behavior prior to
https://github.com/bitcoin/bitcoin/pull/17261, which I think broke backwards
compatibility with old wallet files by no longer treating addresses created by
`addmultisigaddress` calls before #17261 as solvable.

The reason why tests didn't fail with the CanProvide implementation in #17261
is because of a workaround added in 4a7e43e846
"Store p2sh scripts in AddAndGetDestinationForScript", which masked the problem
for new `addmultisigaddress` RPC calls without fixing it for multisig addresses
already created in old wallet files.

This change adds a lot of comments and allows reverting commit
4a7e43e846 "Store p2sh scripts in
AddAndGetDestinationForScript", so the AddAndGetDestinationForScript() function,
CanProvide() method, and mapScripts map should all be more comprehensible
2020-02-12 11:48:30 -05:00
Wladimir J. van der Laan
d4fc9aeb8b Merge #18125: doc: remove PPA note from release-process.md
53234fd0c7 doc: remove PPA note from release-process.md (fanquake)

Pull request description:

  The PPA is [no longer maintained](https://launchpad.net/~bitcoin/+archive/ubuntu/bitcoin) (in favour of the [snap](https://github.com/bitcoin-core/packaging/tree/master/snap)), so no need to bug the bluematt.

  Also fixes a link to the macdeploy instructions.

ACKs for top commit:
  laanwj:
    LGTM ACK 53234fd0c7
  practicalswift:
    ACK 53234fd0c7

Tree-SHA512: 11b32f9b8256f3250139a74522f35496af717cc611ff6de92bca13e0300c049630a61a2ce21976907f4bf8d2cabc54647e862d7ebffc07f2ef6b7d3ba24b3f32
2020-02-12 16:45:17 +01:00
practicalswift
470e2ac602 tests: Avoid hitting some known minor tinyformat issues when fuzzing strprintf(...) 2020-02-12 14:27:19 +00:00
MarcoFalke
caa2f3af29 Merge #12134: Build previous releases and run functional tests
c456145b2c [test] add 0.19 backwards compatibility tests (Sjors Provoost)
b769cd142d [test] add v0.17.1 wallet upgrade test (Sjors Provoost)
9d9390dab7 [tests] add wallet backwards compatility tests (Sjors Provoost)
c7ca630896 [scripts] support release candidates of earlier releases (Sjors Provoost)
8b1460dbd1 [tests] check v0.17.1 and v0.18.1 backwards compatibility (Sjors Provoost)
ae379cf7d1 [scripts] build earlier releases (Sjors Provoost)

Pull request description:

  This PR adds binaries for 0.17, 0.18 and 0.19 to Travis and runs a basic block propagation test.

  Includes test for upgrading v0.17.1 wallets and opening master wallets with older versions.

  Usage:

  ```sh
  contrib/devtools/previous_release.sh -f -b v0.19.0.1 v0.18.1 v0.17.1
  test/functional/backwards_compatibility.py
  ```

  Travis caches these earlier releases, so it should be able to run these tests with little performance impact.

  Additional scenarios where it might be useful to run tests against earlier releases:

  * creating a wallet with #11403's segwit implementation, copying it to an older node and making sure the user didn't lose any funds (although this PR doesn't support `v0.15.1`)
  * future consensus changes
  * P2P changes (e.g. to make sure we don't accidentally ban old nodes)

ACKs for top commit:
  MarcoFalke:
    ACK c456145b2c 🔨

Tree-SHA512: 360bd870603f95b14dc0cd629532cc147344f632b808617c18e1b585dfb1f082b401e5d493a48196b719e0aeaee533ae0a773dfc9f217f704aae898576c19232
2020-02-12 06:20:12 -08:00
fanquake
53234fd0c7 doc: remove PPA note from release-process.md
Also fixes a link to the macdeploy instructions.
2020-02-12 09:42:28 +08:00
fanquake
73a396e028 Merge #18004: build: don't embed a build-id when building libdmg-hfsplus
cb9e88e73a build: don't embed a build-id when building libdmg-hfsplus (fanquake)

Pull request description:

  There was a [reproducibility issue (IRC logs)](http://www.erisian.com.au/bitcoin-core-dev/log-2020-01-25.html) with the osx `0.19.1rc1` gitian builds. The `build-id` embedded into the `dmg` tool was mismatching. It's possible that differing versions of binutils/ld were the cause.

  While it was resolved after rebuilding the base gitian image, whether an upstream package issue or fluke, we can remove the possibility of it happening in future by just not embedding a build-id into the `dmg` tool at all. Can close if it's not deemed worth it.

  You can test this change using the following:

  ```bash
  # build libdmg
  make native_libdmg-hfsplus_built -C depends/ HOST=x86_64-apple-darwin16 -j6 V=1

  # master
  readelf --string-dump .note.gnu.build-id /bitcoin/depends/work/build/x86_64-apple-darwin16/native_libdmg-hfsplus/7ac55ec64c96f7800d9818ce64c79670e7f02b67-3830944ef98/build/dmg/dmg

  String dump of section '.note.gnu.build-id':
    [     c]  GNU
    [    11]  CjRa?]?^V8?v?;%n??

  # this pr
  readelf --string-dump  .note.gnu.build-id /bitcoin/depends/work/build/x86_64-apple-darwin16/native_libdmg-hfsplus/7ac55ec64c96f7800d9818ce64c79670e7f02b67-a72f53ab110/build/dmg/dmg
  readelf: Warning: Section '.note.gnu.build-id' was not dumped because it does not exist!
  ```

ACKs for top commit:
  laanwj:
    tested ACK cb9e88e73a

Tree-SHA512: 191eed32ed0a04a908f9c1b22188180b2db2f35bae0281940f0f9da2450c5c6807cd6ff5bbcce7c933a9133387b127c3478f7a39a2918c14f17be19fd9ea19b4
2020-02-12 08:50:07 +08:00
Russell Yanofsky
bf36a3ccc2 gui: Fix race in WalletModel::pollBalanceChanged
Poll function was wrongly setting cached height to the current chain height
instead of the chain height at the time of polling.

This bug could cause balances to appear out of date, and was first introduced
a0704a8996 (r378452145)
Before that commit, there wasn't a problem because cs_main was held during the
poll update.

Currently, the problem should be rare. But if
8937d99ce81a27ae5e1012a28323c0e26d89c50b from #17954 were merged, the problem
would get worse, because the wrong cachedNumBlocks value would be set if the
wallet was polled in the interval between a block being connected and it
processing the BlockConnected notification.

MarcoFalke <falke.marco@gmail.com> also points out that a0704a8996 could lead
to GUI hangs as well, because previously the pollBalanceChanged method, which
runs on the GUI thread, would only make a nonblocking TRY_LOCK(cs_main) call,
but after could make blocking LOCK(cs_main) calls, potentially locking up the
GUI.

Thanks to John Newbery <john@johnnewbery.com> for finding this bug this while
reviewing https://github.com/bitcoin/bitcoin/pull/17954.
2020-02-11 16:53:53 -05:00
Sjors Provoost
c456145b2c [test] add 0.19 backwards compatibility tests 2020-02-11 21:46:23 +01:00
Sjors Provoost
b769cd142d [test] add v0.17.1 wallet upgrade test 2020-02-11 21:46:23 +01:00
Sjors Provoost
9d9390dab7 [tests] add wallet backwards compatility tests 2020-02-11 21:46:23 +01:00
Sjors Provoost
c7ca630896 [scripts] support release candidates of earlier releases 2020-02-11 21:46:22 +01:00
Sjors Provoost
8b1460dbd1 [tests] check v0.17.1 and v0.18.1 backwards compatibility 2020-02-11 21:46:21 +01:00
Sjors Provoost
ae379cf7d1 [scripts] build earlier releases 2020-02-11 19:37:37 +01:00
Andrew Chow
7e80f646b2 Get the OutputType for a descriptor 2020-02-11 13:23:51 -05:00
MarcoFalke
facb71576c net: Remove forcerelay of rejected txs 2020-02-11 07:44:12 -08:00
fanquake
98264e2ccb Merge #18104: build: Skip i686 build by default in guix and gitian
fae9084ac5 build: Skip i686 build by default in guix and gitian (MarcoFalke)
fa55a2554c depends: Remove reference to win32 (MarcoFalke)

Pull request description:

  Closes #17504

  Now that we no longer provide downloads for i686 on our website (https://bitcoincore.org/en/download/), there is no need to build them by default.

  i686 can still be built in depends (tested by ci/travis) and in guix/gitian by setting the appropriate `HOSTS`.

ACKs for top commit:
  practicalswift:
    ACK fae9084ac5 -- patch looks correct
  dongcarl:
    ACK fae9084ac5 patch looks correct
  laanwj:
    Code review ACK fae9084ac5
  hebasto:
    ACK fae9084ac5, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: b000c19a2cd2a596a52028fa298c4022c24cfdfc1bdb3795a90916d0a00a32e4dd22278db93790b6a11724e08ea8451f4f05c77bc40d1664518e11a8c82d6e29
2020-02-11 16:32:17 +08:00
fanquake
35b7a8e539 Merge #18087: Get rid of VARINT default argument
0e0fa27acb Get rid of VARINT default argument (Pieter Wuille)

Pull request description:

  This removes the need for the non-strandard use of variadic macros.

ACKs for top commit:
  ryanofsky:
    Code review ACK 0e0fa27acb. Only change since last review reverting outdated documentation change from earlier version of pr
  jonatack:
    ACK 0e0fa27 code review, built/ran tests/bitcoind
  practicalswift:
    ACK 0e0fa27acb -- diff looks correct
  MarcoFalke:
    ACK 0e0fa27acb 📯

Tree-SHA512: 6e335e4b586d62112b7260a12481cd949d1b3bbdb83edf8db690348f0a01852e68504336ff3e072e5131a7c8cb404ef11a2f786f842b8d08bbf6ea0e688777b1
2020-02-11 15:26:31 +08:00
Pieter Wuille
0e0fa27acb Get rid of VARINT default argument
This removes the need for the GNU C++ extension of variadic macros.
2020-02-10 12:00:10 -08:00
Wladimir J. van der Laan
646f0ada02 Merge #18051: build: Fix behavior when ALLOW_HOST_PACKAGES unset
0e519fe284 build: Fix behavior when ALLOW_HOST_PACKAGES unset (Hennadii Stepanov)

Pull request description:

  On master (f05c1ac444) during building with depends host packages are always considered by `pkg-config` regardless of `ALLOW_HOST_PACKAGES` environment variable. This causes issues like #18042.

  This is an alternative to #18042 and #18045.

  On master:
  ```
  $ make HOST=x86_64-apple-darwin16 -C depends
  $ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin16/share/config.site ./configure
  ...
  checking for QT_DBUS... yes
  ...
  checking whether to build GUI with support for D-Bus... yes
  ...
  ```

  ---

  With this PR:
  1) `ALLOW_HOST_PACKAGES` unset
  ```
  $ make HOST=x86_64-apple-darwin16 -C depends
  $ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin16/share/config.site ./configure
  ...
  checking for QT_DBUS... no
  ...
  checking whether to build GUI with support for D-Bus... no
  ...
  ```
  2) `ALLOW_HOST_PACKAGES=1`
  ```
  $ make HOST=x86_64-apple-darwin16 ALLOW_HOST_PACKAGES=1 -C depends
  $ CONFIG_SITE=$PWD/depends/x86_64-apple-darwin16/share/config.site ./configure
  ...
  checking for QT_DBUS... yes
  ...
  checking whether to build GUI with support for D-Bus... yes
  ...
  ```

ACKs for top commit:
  jonasschnelli:
    Tested ACK 0e519fe284 - After this PR (and after a `make clean` & `make HOST=x86_64-apple-darwin16` in depends), the macOS depends build on a Debian machine with qt as host package worked (it fails on master due to `ENABLE_DBUS=1`).

Tree-SHA512: d11e1c2bc8ce8a07f3f9b465b01c9b2c814afe75b085a8b88aab74fd3a922aa98180c447457dfc4174515513181c5f4edc521978a1d3d0a112106c98b5c73c0e
2020-02-10 18:11:49 +01:00
Wladimir J. van der Laan
ceb3d45f7d Merge #17947: test: add unit test for non-standard txs with too large tx size
4537ba5f21 test: add unit test for non-standard txs with too large tx size (Sebastian Falbesoner)

Pull request description:

  Approaches another missing unit test of issue #17394: Checks that the function `IsStandardTx()` returns rejection reason `"tx-size"` if the transaction weight is larger than `MAX_STANDARD_TX_WEIGHT` (=400000 vbytes).

ACKs for top commit:
  Empact:
    Code Review ACK 4537ba5f21
  instagibbs:
    ACK 4537ba5f21

Tree-SHA512: ab32e3e47e0b337253aef3da9b7c97d01f4130d00d5860588dfed02114eec3ba49473acc6419448affd63e883fd827bf308716965606eaddee242c4c5a4eb799
2020-02-10 17:59:50 +01:00
Hennadii Stepanov
dcb7af053d Ignore only auto-generated .vcxproj files 2020-02-10 18:47:26 +02:00
Hennadii Stepanov
79c811ca2b Specify ignored bitcoin-qt file precisely
This commit prevents ignoring the build_msvc/bitcoin-qt directory.
2020-02-10 18:42:13 +02:00
Emil Engler
2a95c7c956 ci: Check for submodules 2020-02-10 16:48:15 +01:00
Wladimir J. van der Laan
4c2578706c Merge #18021: Serialization improvements step 4 (undo.h)
3c94b0039d Convert undo.h to new serialization framework (Pieter Wuille)
3cd8ab9d11 Make std::vector and prevector reuse the VectorFormatter logic (Pieter Wuille)
abf8624356 Add custom vector-element formatter (Pieter Wuille)
37d800bea0 Add a constant for the maximum vector allocation (5 Mbyte) (Pieter Wuille)

Pull request description:

  The next step of changes from #10785.

  This one adds:
  * A meta-formatter for vectors, which serializes the vector elements using another formatter
  * Switch the undo.h code to the new framework, using the above (where undo entries are serialized as a vector, each of which uses a modified serializer for the UTXOs).

ACKs for top commit:
  laanwj:
    code review ACK 3c94b0039d
  jonatack:
    Qualified ACK 3c94b0039d
  ryanofsky:
    Code review ACK 3c94b0039d. Changes since last review: renaming formatter classes, adding suggested static_assert, and removing temporary in VectorFormatter

Tree-SHA512: 44eebf51a303f6adbbc1ca2b9d043e8ae7fd37e06778e026590892f8d09f8253067862a68ba8ca5d733fd2f8e7c84edd255370f5a4b6560259427a65f94632df
2020-02-10 16:10:34 +01:00
Wladimir J. van der Laan
b063cb690f Merge #18081: test: set a name for CI Docker containers
9e111db088 test: set a name for CI Docker containers (fanquake)

Pull request description:

  Addresses one part of #16664, by making it easier to identify CI containers that are running locally. By default Docker will generate random names, like `peaceful_rubin`, with this change, we explicitly set names for all containers.

ACKs for top commit:
  MarcoFalke:
    ACK 9e111db088

Tree-SHA512: 0a29ada0d8cf6b0e9ae7a35f4f6df7a3dcc448523ceaed01371124360d6e3d1bf351172104a5fb629488eeaa57994ba04134dcb83c261eb1dfd2f0d73edf5f60
2020-02-10 14:52:51 +01:00
MarcoFalke
faf7d4fa86 build: Add cov_fuzz target 2020-02-10 05:41:34 -08:00
MarcoFalke
fac71e364e build: link fuzz/test_runner.py for out-of-tree builds 2020-02-10 05:34:45 -08:00
MarcoFalke
faf2c5aca0 build: Remove unused USE_COVERAGE 2020-02-10 05:34:11 -08:00
MarcoFalke
ab7915f804 Merge #18106: test: Disable s390 build on travis
b0a01299ed test: Disable s390 build on travis (Wladimir J. van der Laan)

Pull request description:

  Travis is consistently failing on s390 due to out of disk space issues,
  which causes false positives. Disable the s390 build for now until
  this is fixed some other way.

ACKs for top commit:
  MarcoFalke:
    ACK b0a01299ed 🍠
  jonasschnelli:
    ACK b0a01299ed

Tree-SHA512: ae8a78fa03bc620d802e4992d46929a0a4b801b50a6066d8f093f93fc5ebbc010d221c04787cbfc75436a2c40f9292257bf85fbb251593e21afff11756b6d8d4
2020-02-10 04:49:40 -08:00
fanquake
9e111db088 test: set a name for CI Docker containers 2020-02-10 20:03:52 +08:00
fanquake
657c5e5f1c Merge #18099: Update univalue subtree
97aa5740c0 Squashed 'src/univalue/' changes from 5a58a46671..98261b1e7b (MarcoFalke)

Pull request description:

  Closes #17742

ACKs for top commit:
  fanquake:
    ACK fad9ea8fdb

Tree-SHA512: 6316cb0e974ee6575e2a98930203dc7d155b346d2d2fe5a322e3d8b77a87d378d31fde16ea2f90ff93736429ddb89799a26945de13ce4a20132550bbcec0a48e
2020-02-10 19:57:35 +08:00
Wladimir J. van der Laan
b0a01299ed test: Disable s390 build on travis
Travis is consistently failing on s390 due to out of disk space issues,
which causes false positives. Disable the s390 build for now until
this is fixed some other way.
2020-02-10 12:44:35 +01:00
Wladimir J. van der Laan
9e77726fb7 Merge #18101: qt: Fix deprecated QCharRef usage
ac57859e53 qt: Fix deprecated QCharRef usage (Hennadii Stepanov)

Pull request description:

  From Qt docs:
  - [`QKeyEvent::text()`](https://doc.qt.io/qt-5/qkeyevent.html#text):
  > Return values when modifier keys such as Shift, Control, Alt, and Meta are pressed differ among platforms and could return an empty string.

  - [`QString::operator[]()`](https://doc.qt.io/qt-5/qstring.html#operator-5b-5d):

  > **Note:** Before Qt 5.14 it was possible to use this operator to access a character at an out-of-bounds position in the string, and then assign to such a position, causing the string to be automatically resized. Furthermore, assigning a value to the returned `QCharRef` would cause a detach of the string, even if the string has been copied in the meanwhile (and the `QCharRef` kept alive while the copy was taken). These behaviors are deprecated, and will be changed in a future version of Qt.

  Since Qt 5.14 this causes a `QCharRef` warning if any modifier key is pressed while the splashscreen is still displayed.

  Fix #18080.

  Note: Ctrl+Q will also close the spashscreen now.

ACKs for top commit:
  jonasschnelli:
    utACK ac57859e53

Tree-SHA512: a7e5559410bd05c406007ab0243f458b82d434b0543276ed331254c8d7a6b1aaa54d0b406f799b830859294975004380160f8af04ba403d3bf185d51e6784f54
2020-02-10 12:30:28 +01:00
Wladimir J. van der Laan
407d7c831a Merge #18091: Qt: pass clientmodel changes from walletframe to walletviews
2af3e16ca9 Qt: pass clientmodel changes from walletframe to walletviews (Jonas Schnelli)

Pull request description:

  Fixes #18090

  We currently don't pass `clientmodel` changes from the `walletframe` to the `walletviews` leading to possible invalid access during shutdown because all walletviews miss the nullifying of the clientmodel.

  TODO: needs investigation if this is should be backported.

ACKs for top commit:
  laanwj:
    Good catch, code review ACK 2af3e16ca9

Tree-SHA512: f8c0a114f01deac07fb311112d144f3bfc1c1882dd19e8742b372dd597d7a5d59cd0af99fc50494de2334cad98d6701675317474e40fe8820d04c058aeca1b75
2020-02-10 12:24:15 +01:00
Wladimir J. van der Laan
0193fd766b Merge #18082: logging: enable thread_local usage on macOS
d76894987d logging: enable thread_local usage on macOS (fanquake)

Pull request description:

  Now that we're building against a newer SDK (`10.14`), we should be able to enable `thread_local` usage on macOS. Have tested building and running locally, as well as cross-compiling and running the binaries on a macOS 10.14 system.

  #### master 8a56f79d49
  ```bash
  src/bitcoind -logthreadnames=1
  2020-02-06T04:38:33Z [] Bitcoin Core version v0.19.99.0-8a56f79d4 (release build)
  2020-02-06T04:38:33Z [] Assuming ancestors of block 00000000000000000005f8920febd3925f8272a6a71237563d78c2edfdd09ddf have valid signatures.
  2020-02-06T04:38:33Z [] Setting nMinimumChainWork=000000000000000000000000000000000000000008ea3cf107ae0dec57f03fe8
  2020-02-06T04:38:33Z [] Using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation
  2020-02-06T04:38:33Z [] Using RdSeed as additional entropy source
  ```

  #### this PR d76894987d
  ```bash
  checking for thread_local support... yes
  ...
  src/bitcoind -logthreadnames=1
  2020-02-06T04:17:49Z [net] net thread start
  2020-02-06T04:17:49Z [opencon] opencon thread start
  2020-02-06T04:17:49Z [dnsseed] dnsseed thread start
  2020-02-06T04:17:49Z [init] init message: Done loading
  2020-02-06T04:17:49Z [msghand] msghand thread start
  2020-02-06T04:17:49Z [addcon] addcon thread start
  ...
  2020-02-06T04:17:54Z [init] tor: Thread interrupt
  2020-02-06T04:17:54Z [init] Shutdown: In progress...
  ```

  From the [Xcode 8 release notes](https://developer.apple.com/library/archive/releasenotes/DeveloperTools/RN-Xcode/Chapters/Introduction.html#//apple_ref/doc/uid/TP40001051-CH1-SW78)
  > C++ now supports the thread_local keyword, which declares thread-local storage (TLS) and supports C++ classes with non-trivial constructors and destructors. (9001553)

ACKs for top commit:
  jonasschnelli:
    Tested ACK d76894987d
  nijynot:
    ACK d768949
  hebasto:
    ACK d76894987d

Tree-SHA512: 48f3e4104b80bd7b6aedcef10bb1957b073530130f33af7c5cb59e876ac3f5480e53d7af1c0b226d809fe9eef1add3d6c3fb6de4af174966202c6030060ea823
2020-02-10 12:22:32 +01:00
Wladimir J. van der Laan
22d11187ee Merge #17398: build: Update leveldb to 1.22+
677fb8e923 test: Add ubsan surpression for crc32c (Wladimir J. van der Laan)
8e68bb1dde build: Disable msvc warning 4722 for leveldb build (Aaron Clauson)
be23949765 build: MSVC changes for leveldb update (Aaron Clauson)
9ebdf04757 build: CRC32C build system integration (Wladimir J. van der Laan)
402252a808 build: Add LCOV exception for crc32c (Wladimir J. van der Laan)
3a037d0067 test: Add crc32c exception to various linters and generation scripts (Wladimir J. van der Laan)
84ff1b2076 test: Add crc32c to subtree check linter (Wladimir J. van der Laan)
7cf13a5134 doc: Add crc32c subtree to developer notes (Wladimir J. van der Laan)
24d02a9ac0 build: Update build system for new leveldb (Wladimir J. van der Laan)
2e1819311a Squashed 'src/crc32c/' content from commit 224988680f7673cd7c769963d4035cb315aa3388 (Wladimir J. van der Laan)
66480821b3 Squashed 'src/leveldb/' changes from f545dfabff4c2e9836efed094dba99a34fbc6b88..f8ae182c1e5176d12e816fb2217ae33a5472fdd7 (Wladimir J. van der Laan)

Pull request description:

  This updates leveldb to currently newest upstream commit 0c40829872:

  - CRC32C hardware acceleration is now an external library [crc32c](https://github.com/google/crc32c). This adds acceleration on ARM, and should be faster on x86 because of using prefetch. It also makes it easy to support similar instruction sets on other platforms in the future.
  - Thread handling uses C++11, instead of platform specific code.
  - Native windows environment was added. No need to maintain our own hacky one, anymore.
  - Upstream now builds using CMake. This doesn't mean we need to use that (phew), but internal configuration changed to a a series of checks, instead of OS profiles. This means the blanket error "Cannot build leveldb for $host. Please file a bug report' is removed.

  All changes: a53934a3ae...0c40829872

  Pretty much all our changes have been subsumed by upstream, so we figured it was cleaner to start over with a new branch from upstream with the still-relevant patches applied: https://github.com/bitcoin-core/leveldb/tree/bitcoin-fork-new

  There's quite some testing to be done (see below). See https://github.com/bitcoin-core/leveldb/issues/25 and https://github.com/bitcoin-core/leveldb/pull/26 for more history and context.

  TODO:
  - [x] Subtree `crc32c`
  - [x] Make linters happy about crc32 subtree
  - [x] Integrate `crc32c` library into build system
  - [x] MSVC build system

ACKs for top commit:
  sipa:
    ACK 677fb8e923

Tree-SHA512: 37ee92a750e053e924bc4626b12bb3fd81faa9f8c5ebaa343931fee810c45ba05aa6051fdea82535fa351bf2be7297801b98af9469865fc5ead771650a5d6240
2020-02-10 11:36:09 +01:00
MarcoFalke
fae9084ac5 build: Skip i686 build by default in guix and gitian 2020-02-09 13:22:53 -08:00
MarcoFalke
fa55a2554c depends: Remove reference to win32
win32 is no longer mentioned in doc/build-windows.md
2020-02-09 13:22:29 -08:00
Hennadii Stepanov
ac57859e53 qt: Fix deprecated QCharRef usage 2020-02-09 18:59:14 +02:00
MarcoFalke
fad9ea8fdb Update univalue subtree 2020-02-09 07:44:29 -08:00
MarcoFalke
97aa5740c0 Squashed 'src/univalue/' changes from 5a58a46671..98261b1e7b
98261b1e7b Merge #22: Clamp JSON object depth to PHP limit
54c4015415 Clamp JSON object depth to PHP limit

git-subtree-dir: src/univalue
git-subtree-split: 98261b1e7be4ce9820e25c8ce37d40cdef19ab20
2020-02-09 07:43:12 -08:00
MarcoFalke
fad027fb0c scripted-diff: Add missing spaces in RPCResult, Fix type names
This makes the rendered diff smaller when the RPCResult is machine
generated later on

-BEGIN VERIFY SCRIPT-
 # Add space after dictionary key and before colon
 sed -i --regexp-extended -e 's/(^ +" +\\"[a-zA-Z_]+\\"): ?/\1 : /g' $(git grep -l '\\":')
 # Rename (array) to (json array)
 sed -i -e 's/ (array) / (json array) /g' $(git grep -l '(array)' ./src)
 # Rename (object) to (json object)
 sed -i -e 's/ (object) / (json object) /g' $(git grep -l '(object)' ./src)
 # Rename (bool) to (boolean)
 sed -i -e 's/ (bool) / (boolean) /g' $(git grep -l '(bool)' ./src)
 # Rename (int) to (numeric)
 sed -i -e 's/  (int) /  (numeric) /g' $(git grep -l '(int)' ./src)
-END VERIFY SCRIPT-
2020-02-09 05:12:43 -08:00
MarcoFalke
75fb37ce68 Merge #18032: rpc: Output a descriptor in createmultisig and addmultisigaddress
19a354b11f Output a descriptor in createmultisig and addmultisigaddress (Andrew Chow)

Pull request description:

  Give a descriptor from `createmultisig` and `addmultisigaddress`.

  Extracted from #16528 with `addmultisgaddress` and tests added.

ACKs for top commit:
  Sjors:
    tACK 19a354b11f
  MarcoFalke:
    ACK 19a354b11f
  promag:
    Code review ACK 19a354b11f.
  meshcollider:
    utACK 19a354b11f

Tree-SHA512: e813125fbbc358ea8d45b1748de16a29a94efd83175b748fb8fa3b0bfc8e783ed36b6c554d84f5d4ead1ba252a83a3e937b6c3f75da7b8d3b4e55f94d6013771
2020-02-09 04:55:45 -08:00
Jonas Schnelli
2af3e16ca9 Qt: pass clientmodel changes from walletframe to walletviews 2020-02-07 18:40:38 +01:00
Pieter Wuille
3c94b0039d Convert undo.h to new serialization framework 2020-02-06 19:08:11 -08:00
Pieter Wuille
3cd8ab9d11 Make std::vector and prevector reuse the VectorFormatter logic 2020-02-06 19:06:09 -08:00
Pieter Wuille
abf8624356 Add custom vector-element formatter
This allows a very compact notation for serialization of vectors whose
elements are not serialized using their default encoding.
2020-02-06 18:58:44 -08:00
Gastón I. Silva
63ce882760 doc: link to homebrew's troubleshooting page 2020-02-06 09:05:55 -08:00
Russell Yanofsky
ff9c671b11 refactor: Work around GCC 9 -Wredundant-move warning
Use std::move workaround for unique_ptr, for when the C++ compiler lacks
a fix for this issue:
http://www.open-std.org/jtc1/sc22/wg21/docs/cwg_defects.html#1579
Do this in a way that avoids a GCC 9 `-Wredundant-move` warning.
2020-02-06 13:43:15 +01:00
Wladimir J. van der Laan
b837b334db net: Fail instead of truncate command name in CMessageHeader
Replace the memset/strncpy dance in `CMessageHeader::CMessageHeader`
with explicit code that copies then name and asserts the length.

This removes a warning in g++ 9.1.1 and IMO makes the code more readable
by not relying on strncpy padding and silent truncation behavior.
2020-02-06 13:24:40 +01:00
fanquake
23fab1a3df Merge #18062: gui: Fix unintialized WalletView::progressDialog
acf8abc7f3 gui: Fix unintialized WalletView::progressDialog (João Barbosa)

Pull request description:

  #17911 shows that it's possible to read the unintialized `progressDialog` in f32564f0a7/src/qt/walletview.cpp (L296-L297).

  And the debugger shows
  ```
  (gdb) bt
  #0  0x0000555556687c60 in QProgressDialog::wasCanceled() const ()
  #1  0x000055555572989f in WalletView::showProgress (this=0x5555577d7a70,
      title=..., nProgress=1) at qt/walletview.cpp:322
  ```

  Closes #17911.

ACKs for top commit:
  hebasto:
    ACK acf8abc7f3, I have reviewed the code and it looks OK, I agree it can be merged.
  elichai:
    utACK acf8abc7f3
  kristapsk:
    ACK acf8abc7f3
  MarcoFalke:
    ACK acf8abc7f3

Tree-SHA512: f5e6d873192d08d1a572e66e17c2e06d1ce27d01aa196b2a7ed591008641295bb02cda8ac90919ff2d2fc778316c2e143f8d36599e0d377779758853dfaf0a31
2020-02-06 14:43:38 +08:00
fanquake
4d211c8da1 Merge #18003: build: remove --large-address-aware linker flag
acd644b83d build: remove --large-address-aware linker flag (fanquake)

Pull request description:

  This flag was used when building 32-bit Windows executables, which we no-longer
  do, and is not accepted by the linker for any of the hosts we currently build
  for. i.e:

  ```bash
  checking whether the linker accepts -Wl,--large-address-aware... no
  ```

  --large-address-aware
      If given, the appropriate bit in the "Characteristics" field of the COFF
      header is set to indicate that this executable supports virtual addresses
      greater than 2 gigabytes. This should be used in conjunction with the /3GB
      or /USERVA=value megabytes switch in the "[operating systems]" section of
      the BOOT .INI. Otherwise, this bit has no effect. [This option is specific
      to PE targeted ports of the linker]

  You can check that the appropriate bit in the COFF header of our 64-bit
  Windows binaries is still be set using dumpbin. i.e:

  ```powershell
  dumpbin /headers .\bitcoind.exe

  FILE HEADER VALUES
  <snip>
  26 characteristics
       Executable
       Line numbers stripped
       Application can handle large (>2GB) addresses
  ```

ACKs for top commit:
  laanwj:
    ACK acd644b83d

Tree-SHA512: 9711e07bc08e843fcefd0517091a59cb7670dd107d03623a146d03fe73054d0e64f78489490b37f4708eab2c4800037f923b9ec92e7f53c3df9a590242f52b55
2020-02-06 14:29:59 +08:00
fanquake
d76894987d logging: enable thread_local usage on macOS 2020-02-06 12:21:00 +08:00
fanquake
cb9e88e73a build: don't embed a build-id when building libdmg-hfsplus 2020-02-06 12:02:09 +08:00
fanquake
dc9305b616 random: don't special case clock usage on macOS
clock_gettime(), CLOCK_MONOTONIC and CLOCK_REALTIME are all available for use on
macOS (now that we require macOS >=10.12). Use them rather than the deprecated
mach_timespec_t time API.

master:
2019-12-23T20:49:43Z Feeding 216 bytes of dynamic environment data into RNG
2019-12-23T20:50:43Z Feeding 216 bytes of dynamic environment data into RNG

this commit:
2019-12-23T20:32:41Z Feeding 232 bytes of dynamic environment data into RNG
2019-12-23T20:33:42Z Feeding 232 bytes of dynamic environment data into RNG
2020-02-06 08:24:13 +08:00
Wladimir J. van der Laan
8a56f79d49 Merge #17482: util: Disallow network-qualified command line options
900d8f6f70 util: Disallow network-qualified command line options (Russell Yanofsky)

Pull request description:

  Previously these were allowed but ignored.

  This change implements one of the settings simplifications listed in #17508. Change includes release notes.

ACKs for top commit:
  laanwj:
    ACK 900d8f6f70

Tree-SHA512: ab020a16a86c1e8ec709fbf798d533879d32c565eceeb7eb785c33042c49c6b4d1108c5453d8166e4a2abffc2c8802fbb6d3b895e0ddeefa8f274fd647e3c8ad
2020-02-05 16:23:53 +01:00
Wladimir J. van der Laan
712b7d9b47 Merge #17804: doc: Misc RPC help fixes
fa5c6622c8 doc: Use proper RPC help syntax in importmulti (MarcoFalke)
fab63111be doc: Remove duplicate "comment" from listsinceblock RPC help (MarcoFalke)
fa04cd6cfc doc: Properly document proxy_randomize_credentials as bool in getnetworkinfo (MarcoFalke)
fa9dec7c39 doc: Fix syntax error (trailing square bracket) in finalizepsbt (MarcoFalke)
faff5a60ed doc: Fix syntax error (trailing square bracket) in walletprocesspsbt (MarcoFalke)
fa0545901d doc: Add missing "optional" to "long" estimaterawfee RPC help (MarcoFalke)

Pull request description:

  This fixes documentation of the following RPCs:

  * estimaterawfee (hidden)
  * https://bitcoincore.org/en/doc/0.19.0/rpc/wallet/walletprocesspsbt/
  * https://bitcoincore.org/en/doc/0.19.0/rpc/rawtransactions/finalizepsbt/
  * https://bitcoincore.org/en/doc/0.19.0/rpc/network/getnetworkinfo/
  * https://bitcoincore.org/en/doc/0.19.0/rpc/wallet/listsinceblock/
  * https://bitcoincore.org/en/doc/0.19.0/rpc/wallet/importmulti/

  <!-- Also, it comes with a scripted diff to normalize whitespace and type names. (Previous attempts: #14601 and #14459)

ACKs for top commit:
  laanwj:
    ACK fa5c6622c8

Tree-SHA512: 5a10956e12f8ce23e93a2ce8bafd6cae759d8a21658f79397e3bfce3e4aabd9658bdbd40acde49323dca958a9befee7166654994208c182dd60f483109621e17
2020-02-05 14:54:42 +01:00
Wladimir J. van der Laan
b30a1f3e39 Merge #18052: Remove false positive GCC warning
e9434ee03e Remove false positive GCC warning (Hennadii Stepanov)

Pull request description:

  On master (f05c1ac444) GCC compiler fires a false positive `-Wmaybe-uninitialized`:

  ```
  wallet/wallet.cpp: In static member function ‘static std::shared_ptr<CWallet> CWallet::CreateWalletFromFile(interfaces::Chain&, const WalletLocation&, std::__cxx11::string&, std::vector<std::__cxx11::basic_string<char> >&, uint64_t)’:
  wallet/wallet.cpp:3913:27: warning: ‘*((void*)& time_first_key +8)’ may be used uninitialized in this function [-Wmaybe-uninitialized]
           Optional<int64_t> time_first_key;
                             ^~~~~~~~~~~~~~
  ```

  The same as #15292.

  This PR leverages a workaround and removes the warning.

ACKs for top commit:
  laanwj:
    ACK e9434ee03e, removes the warning for me (gcc 7.4.0)
  kristapsk:
    ACK e9434ee03e

Tree-SHA512: 8820a8ba6a75aa6b1ac675a38c883a77f12968b010533b6383180aa66e7e0d570bf6300744903ead91cf9084e5345144959cd6b0cea1b763190b8dd49bacce75
2020-02-05 14:43:28 +01:00
Wladimir J. van der Laan
c8ce2632eb Merge #16392: build: macOS toolchain update
7e2104433c build: use macOS 10.14 SDK (fanquake)
ca5055a5aa depends: native_cctools 921, ld64 409.12, libtapi 1000.10.8 (fanquake)
1de8c067c7 depends: clang 6.0.1 (fanquake)

Pull request description:

  TLDR: This updates our macOS toolchain to use a newer version of Clang, cctools (including new [dependency on libtapi](https://github.com/tpoechtrager/cctools-port/tree/master#dependencies)), LD64 and the macOS SDK.

  I've been testing depends builds (`HOST=x86_64-apple-darwin16`) inside a Debian Buster [Docker container](https://github.com/fanquake/core-review/blob/master/docker/debian.dockerfile), and running the resultant `bitcoind` and `bitcoin-qt` binaries on a macOS `10.14.4` system. The `.dmg` generated by a `make deploy` also mounts correctly on the same macOS system.

  #### Clang
  Upgraded from `3.7.1` to [`6.0.1`](https://releases.llvm.org/6.0.1/docs/ReleaseNotes.html)

  #### cctools
  * cctools `877.8` -> [`921`](https://opensource.apple.com/tarballs/cctools/)
  * LD64 `253.9` -> [`409.12`](https://opensource.apple.com/source/ld64/)
  * TAPI [`1000.10.8`](https://opensource.apple.com/tarballs/tapi/)

  See [tpoechtrager/cctools-port](https://github.com/tpoechtrager/cctools-port/) and [tpoechtrager/apple-libtapi](https://github.com/tpoechtrager/apple-libtapi/).

  #### macOS SDK
  Upgraded from building against the macOS `10.11` SDK to the macOS `10.14` SDK.

  #### TODO
  - [x] Make the `10.14` SDK available to Travis.

  Fixes: #16052
  Closes: #14797

ACKs for top commit:
  Sjors:
    re-ACK 7e2104433c (rebased from 248526e)
  dongcarl:
    ACK 7e21044

Tree-SHA512: fd36a33dbfb98c144240f8c69b77343e3f5bc18d8cf7d40fff61f51ad48925ec1872e6daba34c4045b18b4c2c84c22c744ebf4cba11061a0305eed13975ceefe
2020-02-05 14:27:32 +01:00
Wladimir J. van der Laan
01668839de Merge #17660: build: remove deprecated key from macOS Info.plist
c0bc453135 build: remove deprecated key from macOS Info.plist (fanquake)

Pull request description:

  Note that the current release binaries show correct version numbers everywhere in the GUI and macOS info dialogs (except for when you "space" click the app, shown in screenshots), and we haven't reintroduced the issue that #14701 fixed. This is just swapping a deprecated field for a newer one, as well as using the entire version string in two fields that we hadn't been previously.

  Follows up discussion in #14701.

  0.19.0.1
  ![0 19 0 1](https://user-images.githubusercontent.com/863730/70089170-a0576e80-15e5-11ea-975c-a6902a1ed95a.png)

  This PR.
  ![master](https://user-images.githubusercontent.com/863730/70089178-a3525f00-15e5-11ea-9d63-7db67de014a5.png)

ACKs for top commit:
  laanwj:
    ACK c0bc453135

Tree-SHA512: 6191056d0cb6072b8a2170c8441ebfe500cf00cd41014bf5ee68fbf60b5bb5642e2fad9541f1c5abfaafdae6db3102c3add6169cefce3fc4a63d8b913ea35865
2020-02-05 14:26:00 +01:00
Wladimir J. van der Laan
adea5e1b54 Merge #18023: Fix some asmap issues
c86bc14408 Make asmap Interpret tolerant of malicious map data (Pieter Wuille)
38c2395d7a Use ASNs for mapped IPv4 addresses correctly (Pieter Wuille)
6f8c937312 Mark asmap const in statistics code (Pieter Wuille)
d58bcdc4b5 Avoid asmap copies in initialization (Pieter Wuille)

Pull request description:

  Here are a few things to improve in the asmap implementation. The first two commits are just code improvements. The last one is a bugfix (the exsting code wouldn't correctly apply ASN lookups to mapped/embedded IPv4 addresses).

ACKs for top commit:
  practicalswift:
    ACK c86bc14408 -- patch looks correct
  naumenkogs:
    utACK c86bc14
  laanwj:
    ACK c86bc14408
  jonatack:
    ACK c86bc14408 code looks correct, built/ran tests, bitcoind with -asmap pointed to asmap/demo.map

Tree-SHA512: 1036f43152754d621bfbecfd3b7c7276e4670598fcaed42a3d275e51fa2cf3653e2c9e9cfa714f6c7719362541510e92171e076ac4169b55a0cc8908b2d514c0
2020-02-05 13:59:42 +01:00
MarcoFalke
bd5c4c6971 Merge #18069: test: replace 'regtest' leftovers by self.chain
eca56f8929 test: replace 'regtest' leftovers by self.chain (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR to #16681 (fixes #18068), replacing all remaining hardcoded `"regtest"` strings in functional tests by `self.chain`.

Top commit has no ACKs.

Tree-SHA512: 96524649b33164938e5a95215991103ed7855ebab55ef788d4816b3fa5cbc03d8f3b0d39f2247a87522f289fd7f4daf25e059900b8462b5127eb154bbee89054
2020-02-05 10:41:40 +00:00
Wladimir J. van der Laan
554d89fb29 Merge #18029: tests: Add fuzzing harness for AS-mapping (asmap)
4d2aceaad8 tests: Add fuzzer asmap to FUZZERS_MISSING_CORPORA (temporarily) (practicalswift)
8d07706985 tests: Add fuzzing harness for AS-mapping (asmap) (practicalswift)

Pull request description:

  Add fuzzing harness for AS-mapping (`asmap`).

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/asmap
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 4d2aceaad8
  jonatack:
    ACK 4d2aceaad8

Tree-SHA512: bc4c63b48cd98c0cec9d10ecb43775b1bf1215241ff821fc7a866c7e2738605641fb88d044eabf2f48a8c16f2ced9ffce5165c9e6a83c73ece004350da7153e7
2020-02-05 11:40:22 +01:00
Miguel Herranz
f5a3a5b9ab gui: Add close window shortcut
CMD+W/CTRL+W is the standard shortcut to close a window without
exiting the program.
2020-02-05 07:13:01 +01:00
Sebastian Falbesoner
eca56f8929 test: replace 'regtest' leftovers by self.chain
Commit 1abcecc40c replaced 'regtest' by self.chain
'regtest' "in almost all current tests", this commit takes care of the remaining
ones.
2020-02-05 04:28:40 +01:00
fanquake
a064e005fa Merge #18059: build: add missing attributes to Win installer
6c22315223 build: add additional attributes to Win installer (fanquake)

Pull request description:

  Fixes: #17170.

ACKs for top commit:
  hebasto:
    ACK 6c22315223, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: d2ff2006b8df6a34b3a16270d3eb895b03cf6b3ca69404bc39adeb7d5e3b896ddab6ba831566dc966d8bdfba3f57ddf325762cddf3ad76d1427971d1bcc68255
2020-02-05 11:16:06 +08:00
fanquake
8625446b4d Merge #17336: scripts: search for first block file for linearize-data with some block files pruned
317fb96de9 Add search for first blk file with pruned node (Rjected)

Pull request description:

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->
  When bitcoind is running in pruned mode, producing a hashlist with `./linearize-hashes.py linearize.cfg > hashlist.txt` and then executing `linearize-data.py linearize.cfg` will produce:
  ```
  Read 313001 hashes
  Input file /home/dan/.bitcoin/blocks/blk00000.dat
  Premature end of block data
  ```
  This happens because `linearize-data` starts by attempting to process `blk00000.dat` regardless of whether or not `blk00000.dat` actually exists - this may not be the case if working with a pruned node.
  This PR adds a function which finds the first block file that does exist, and calls that function when the `BlockDataCopier` is initialized.

  This is a refactor of #16431.

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  darosior:
    ACK 317fb96de9
  laanwj:
    Code review ACK 317fb96de9
  theStack:
    Code review ACK 317fb96de9

Tree-SHA512: fc8014282df6cfe7b267e64db8ce7d82b86b758c302fbfea4a3c39b62d93512f5c2e31a0de4e9c5ec18fc0268c917f011257d37b45afaef6033eec90e4aa585f
2020-02-05 08:45:06 +08:00
MarcoFalke
f32564f0a7 Merge #16681: Tests: Use self.chain instead of 'regtest' in all current tests
1abcecc40c Tests: Use self.chain instead of 'regtest' in almost all current tests (Jorge Timón)

Pull request description:

  Simply avoiding the hardcoded string in more places for consistency.
  It can also allow for more easily reusing tests for other chains other than regtest.

  Separated from #8994 .
  Continues #16509 .

  It is still not complete (ie to be complete, we need the -chain parameter in #16680 and make whether acceptnonstdtxs is allowed for that chain or not customizable for regtest [or for custom chains like in #8994 ] ). But while being incomplete like #16509 , it's quite simple to review and another step forward IMO.

ACKs for top commit:
  Sjors:
    re-ACK 1abcecc. I think it's an improvement even if incomplete and if some PR's might accidentally bring "regtest" back. Subsequent improvements hopefully don't have to touch 16 files.
  elichai:
    Code review ACK 1abcecc40c
  ryanofsky:
    Code review ACK 1abcecc40c.
  ryanofsky:
    Code review ACK 1abcecc40c

Tree-SHA512: 5620de6dab235ca8bd8670d6366c7b9f04f0e3ca9c5e7f87765b38e16ed80c17d7d1630c0d5fd7c5526f070830d94dc74cc2096d8ede87dc7180ed20569509ee
2020-02-04 20:55:26 +00:00
Sjors Provoost
92bcd70808 [wallet] allow transaction without change if keypool is empty 2020-02-04 11:20:25 +01:00
Sjors Provoost
709f8685ac [wallet] CreateTransaction: simplify change address check 2020-02-04 11:20:25 +01:00
Sjors Provoost
5efc25f963 [wallet] translate "Keypool ran out" message 2020-02-04 11:18:02 +01:00
fanquake
02fafdd12c Merge #18060: gui: Drop PeerTableModel dependency to ClientModel
ff59bcd321 gui: Drop PeerTableModel dependency to ClientModel (João Barbosa)

Pull request description:

  Class `PeerTableModel` doesn't actually depend on `ClientModel`.

ACKs for top commit:
  Empact:
    Code Review ACK ff59bcd321
  hebasto:
    ACK ff59bcd321, tested on Linux Mint 19.3. No changes in behavior are observed.

Tree-SHA512: 29fa3c316c05b8f7b9340e5859bbb8c3a0b826aa7c865c892cfa13b5ad30f822fcaae4e01555f7860cd1727f20b7ef555a808235522a04a6eebaaa7b605f8595
2020-02-04 14:57:53 +08:00
fanquake
6c22315223 build: add additional attributes to Win installer 2020-02-04 10:46:47 +08:00
João Barbosa
acf8abc7f3 gui: Fix unintialized WalletView::progressDialog 2020-02-03 20:05:04 +00:00
João Barbosa
ff59bcd321 gui: Drop PeerTableModel dependency to ClientModel 2020-02-03 14:48:40 +00:00
fanquake
7e2104433c build: use macOS 10.14 SDK
Co-Authored-By: Carl Dong <accounts@carldong.me>
2020-02-03 19:49:46 +08:00
fanquake
ca5055a5aa depends: native_cctools 921, ld64 409.12, libtapi 1000.10.8
This also removes the obsolete mlinker-version option

Co-Authored-By: Cory Fields <cory-nospam-@coryfields.com>
2020-02-03 19:49:46 +08:00
fanquake
1de8c067c7 depends: clang 6.0.1
This also removes some now-unnecessary cctools hacks.

Co-Authored-By: Cory Fields <cory-nospam-@coryfields.com>
2020-02-03 19:49:46 +08:00
Wladimir J. van der Laan
651e343888 Merge #16974: Walk pindexBestHeader back to ChainActive().Tip() if it is invalid
0a50019fde Walk pindexBestHeader back to ChainActive().Tip() if it is invalid (Matt Corallo)

Pull request description:

  Instead of keeping pindexBestHeader set to the best header we've
  ever seen, reset it back to our validated tip if we find an ancestor
  of it turns out to be invalid. While the name is now a bit confusing,
  this matches much better with how it is used in practice, see below.
  Further, this opens up more use-cases for it in the future, namely
  aggressively searching for new peers in case we have discovered
  (possibly via some covert channel) headers which we do not know to be
  invalid, but which we cannot find block data for.

  Places pindexBestHeader is used:

   * Various GUI displays of the best header and getblockchaininfo["headers"],
     I don't think changing this is bad, and if anything this is less confusing
     in the presence of an invalid block.
   * IsCurrentForFeeEstimation(): If anything I think ensuring pindexBestHeader
     isn't some crazy invalid chain is better than the alternative, even in the
     case where you are rejecting the current chain due to hardware error (since
     hopefully in that case you won't get any new blocks anyway).
   * ConnectBlock assumevalid checks: We use pindexBestHeader to check that the
     block we're connecting leads to something with nMinimumChainWork (preventing
     a user-set assumevalid from having bogus work) and that the block we're
     connecting leads to pindexBestHeader (I'm not too worried about this one -
     it's nice to "disable" assumevalid if we have a long invalid headers chain,
     but I don't see it as a critical protection).
   * BlockRequestAllowed() uses pindexBestHeader as its target to ensure the
     requested block is within a month of the "current chain". I don't think this
     is a meaningful difference, if we're rejecting the current tip we're
     trivially fingerprintable anyway, and if the chain really does have a bunch
     of invalid crap near the tip, using the best not-invalid header is likely a
     better criteria.
   * ProcessGetBlockData uses pindexBestHeader as the "current chain" definition
     of whether a block request is "historical" for the purpose of bandwidth
     limiting. Similarly, I don't see why this is a meaningful change.
   * We use pindexBestHeader for requesting missing headers on receipt of a
     headers/compact block message or block inv as well as for initial getheaders.
     I think this is definitely wrong, using the best not-invalid header for such
     requests is much better.
   * We use pindexBestHeader to define the "current chain" for deciding when
     we're close to done with initial headers sync. I don't think this is a
     meaningful change.
   * We use pindexBestHeader to decide if initial headers sync has timed out. If
     we're rejecting the chain due to hardware error this may result in
     additional cases where we ban a peer, but this is already true, so I think
     its fine.

ACKs for top commit:
  fjahr:
    ACK 0a50019fde
  kallewoof:
    ACK 0a50019fde
  ariard:
    utACK 0a50019

Tree-SHA512: 2ecfa973a9878a00313ae7ede94a9bd7710e0caf55b544b10bbc46dc463a0478cbaf477e6cdd072356d5a0c5fb3848e9339284af785a2995c20bae8bd23f23e5
2020-02-03 12:28:38 +01:00
Wladimir J. van der Laan
b2df21b32c Merge #17925: Improve UpdateTransactionsFromBlock with Epochs
bd5a026928 Make UpdateTransactionsFromBlock use Epochs (Jeremy Rubin)
2ccb7cca4a Add Epoch Guards to CTXMemPoolEntry and CTxMemPool (Jeremy Rubin)

Pull request description:

  UpdateTransactionsFromBlock is called during a re-org. When a re-org occurs, all of the transactions in the mempool may be descendants from a transaction which is in the pre-reorg block. This can cause us to propagate updates, worst case, to every transaction in the mempool.

  Because we construct a `setEntries setChildren`, which is backed by a `std::set`, it is possible that this algorithm is `O(N log N)`.

  By using an Epoch visitor pattern, we can limit this to `O(N)` worst case behavior.

  Epochs are also less resource intensive than almost any set option (e.g., hash set) because they are allocation free.

  This PR is related to https://github.com/bitcoin/bitcoin/pull/17268, it is a small subset of the changes which have been refactored slightly to ease review. If this PR gets review & merge, I will follow up with more PRs (similar to #17268) to improve the mempool

ACKs for top commit:
  sdaftuar:
    ACK bd5a026928
  adamjonas:
    Just to summarize for those looking to review - as of bd5a026 there are 3 ACKs (@sdaftuar, @ariard, and @hebasto) and one "looks good" from @ajtowns with no NACKs or any show-stopping concerns raised.
  ajtowns:
    ACK bd5a026928 (code review)
  ariard:
    Code review ACK bd5a026
  hebasto:
    ACK bd5a026928, modulo some nits and a typo.

Tree-SHA512: f0d2291085019ffb4e1119edeb9f4a89c1a572d1cb5b4bdf5743dd0152e721e1935f5155dcae84e1e5bda5ffdf6224c488c1e200bd33bedca9f5ca22d5f5139f
2020-02-03 11:54:34 +01:00
fanquake
365c83e6a8 Merge #18054: net: reference instead of copy in BlockConnected range loop
9a299a59cc net: reference instead of copy in BlockConnected range loop (Jon Atack)

Pull request description:

  Reference elements in range for loop instead of copying them and
  fix Clang `-Wrange-loop-analysis` warning introduced in a029e18

  ```
  net_processing.cpp:1185:25: warning: loop variable 'ptx' of
  type 'const std::shared_ptr<const CTransaction>' creates a copy from
  type 'const std::shared_ptr<const CTransaction>' [-Wrange-loop-analysis]
          for (const auto ptx : pblock->vtx) {
                          ^
  net_processing.cpp:1185:14: note: use reference type
  'const std::shared_ptr<const CTransaction> &' to prevent copying
          for (const auto ptx : pblock->vtx) {
               ^~~~~~~~~~~~~~~~
  1 warning generated.
  ```

ACKs for top commit:
  Empact:
    ACK 9a299a59cc
  MarcoFalke:
    ACK 9a299a59cc
  promag:
    ACK 9a299a59cc.
  elichai:
    ACK 9a299a59cc
  emilengler:
    ACK 9a299a5.

Tree-SHA512: 9284d1b00684877505454a05071212758c8cea083534e2eec09bfc8a9c3059eea811d2008f6a5a678539444f0d5b3134db1bd23da6514b3d3a1440634c8b53be
2020-02-03 12:02:23 +08:00
Jon Atack
9a299a59cc net: reference instead of copy in BlockConnected range loop
to fix -Wrange-loop-analysis warning introduced in a029e18
2020-02-02 10:27:47 +01:00
Samuel Dobson
6d0e532ae0 Merge #17585: rpc: deprecate getaddressinfo label
d3bc184081 doc: update release notes with getaddressinfo label deprecation (Jon Atack)
72af93f364 test: getaddressinfo label deprecation test (Jon Atack)
d48875fa20 rpc: deprecate getaddressinfo label field (Jon Atack)
dc0cabeda4 test: remove getaddressinfo label tests (Jon Atack)
c7654af6f8 doc: address pr17578 review feedback (Jon Atack)

Pull request description:

  This PR builds on #17578 (now merged) and deprecates the rpc getaddressinfo `label` field. The deprecated behavior can be re-enabled by starting bitcoind with `-deprecatedrpc=label`.

  See http://www.erisian.com.au/bitcoin-core-dev/log-2019-11-22.html#l-622 and https://github.com/bitcoin/bitcoin/pull/17283#issuecomment-554458001 for more context.

  Reviewers: This PR may be tested manually by building, then running bitcoind with and without the `-deprecatedrpc=label` flag while verifying the rpc getaddressinfo output and help text.

  Next step: add support for multiple labels.

ACKs for top commit:
  jnewbery:
    ACK d3bc184081
  laanwj:
    ACK d3bc184081
  meshcollider:
    utACK d3bc184081

Tree-SHA512: f954402884ec54977def332c8160fd892f289b0d2aee1e91fed9ac3220f7e5b1f7fc6421b84cc7a5c824a0582eca4e6fc194e4e33ddd378c733c8941ac45f56d
2020-02-02 21:35:46 +13:00
Hennadii Stepanov
e9434ee03e Remove false positive GCC warning 2020-02-01 23:07:19 +02:00
Hennadii Stepanov
0e519fe284 build: Fix behavior when ALLOW_HOST_PACKAGES unset 2020-02-01 13:32:32 +02:00
Jonas Schnelli
f05c1ac444 Merge #17937: gui: Remove WalletView and BitcoinGUI circular dependency
cb8a86d9f9 gui: Remove WalletView and BitcoinGUI circular dependency (João Barbosa)
ac3d10777d gui: Add transactionClicked and coinsSent signals to WalletView (João Barbosa)

Pull request description:

  Essentially moves the code in `WalletView::setBitcoinGUI` to the only caller. Two new signals are added beforehand in the first commit so that the connections in `WalletFrame` are all from the wallet view.

ACKs for top commit:
  hebasto:
    ACK cb8a86d9f9, tested on Linux Mint 19.3.
  jonasschnelli:
    utACK cb8a86d9f9

Tree-SHA512: 250316cd3689e51c8cded9ccd75963c836dcafa6db25d684f2aa691dea9738895f9140793e0f925784909e39f8257f7e1c7d611e8bd6d6634e1a50333f4ddb1e
2020-02-01 10:11:24 +01:00
Jonas Schnelli
cadb9d3342 Merge #18036: gui: Break trivial circular dependencies
3aee10b80b gui: Drop ShutdownWindow dependency to BitcoinGUI (João Barbosa)
61eb058cc1 gui: Drop BanTableModel dependency to ClientModel (João Barbosa)

Pull request description:

  `ShutdownWindow::showShutdownWindow` just needs a widget to center the shutdown window and to borrow its title.

ACKs for top commit:
  hebasto:
    ACK 3aee10b80b, since previous review only suggested change `QWidget` --> `QMainWindow`
  jonasschnelli:
    utACK 3aee10b80b

Tree-SHA512: e15cb6ee274730bd071d3d97b540c5059e5c655248d69a37c3fd00f2aacc6cfcb36b9a65755718027e15482ec8e5e85534c1dc13d0ddb4e0680df03fbf6571f2
2020-02-01 10:07:42 +01:00
Pieter Wuille
c86bc14408 Make asmap Interpret tolerant of malicious map data 2020-01-31 14:51:37 -08:00
Pieter Wuille
38c2395d7a Use ASNs for mapped IPv4 addresses correctly 2020-01-31 14:51:37 -08:00
Pieter Wuille
6f8c937312 Mark asmap const in statistics code 2020-01-31 14:51:37 -08:00
Pieter Wuille
d58bcdc4b5 Avoid asmap copies in initialization 2020-01-31 14:51:37 -08:00
João Barbosa
4e9efac678 test: Check wallet name in -walletnotify script 2020-01-31 17:43:26 -05:00
Jonas Schnelli
d104aa0ace Merge #17951: Use rolling bloom filter of recent block txs for AlreadyHave() check
a029e18c2b Use rolling bloom filter of recent block tx's for AlreadyHave() check (Suhas Daftuar)

Pull request description:

  In order to determine whether to download or process a relayed transaction, we first try to check whether we already have the transaction -- either in the mempool, in our filter of recently rejected transactions, in our orphan pool, or already confirmed in a block.

  Prior to this commit, the heuristic for checking whether a transaction was confirmed in a block is based on whether there's a coin cache entry corresponding to the 0- or 1-index vout of the tx. While that is a quick check, it is very imprecise (eg if those outputs were already spent in another block, we wouldn't detect that the transaction has already been confirmed) -- we can do better by just keeping a rolling bloom filter of the transactions in recent blocks, which will better capture the case of a transaction which has been confirmed and then fully spent.

  This should reduce the bandwidth that we waste by requesting transactions which will not be accepted to the mempool.

  To avoid relay problems for transactions which have been included in a recent block but then reorged out of the chain, we clear the bloom filter whenever a block is disconnected.

ACKs for top commit:
  MarcoFalke:
    re-ACK a029e18c2b only stylistic and comment fixups 🍴
  sipa:
    utACK a029e18c2b
  jonatack:
    Code review ACK a029e18c2b also built/ran tests and am running bitcoind with mempool debug logging and custom logging. Looked a bit into CRollingBloomFilter and also the mempool median time past checks mentioned above; I don't have a deep understanding of those areas yet but the concept here and changes LGTM. Tests and other optimisations could be added as a follow-up. In favor of seeing this move forward if no major immediate concerns.

Tree-SHA512: 784c9a35bcd3af5db469063ac7d26b4bac430e451e5637a34d8a538c3ffd1433abdd3f06e5584e7a84bfa9e791449e61819397b5a6c7890fa59d78ec3ba507b2
2020-01-31 14:42:59 +01:00
practicalswift
3c82b92d2e tests: Add fuzzing harness for functions taking floating-point types as input 2020-01-31 12:36:13 +00:00
practicalswift
c2bd588860 Add missing includes 2020-01-31 12:36:13 +00:00
João Barbosa
3aee10b80b gui: Drop ShutdownWindow dependency to BitcoinGUI 2020-01-31 11:49:51 +00:00
Sjors Provoost
5bad7921d0 [test] PSBT RPC: check that bip32_derivs are present by default 2020-01-31 10:38:04 +01:00
João Barbosa
61eb058cc1 gui: Drop BanTableModel dependency to ClientModel 2020-01-31 08:18:27 +00:00
Andrew Chow
19a354b11f Output a descriptor in createmultisig and addmultisigaddress 2020-01-30 23:55:36 -05:00
fanquake
3e1bf71064 Merge #18025: doc: Add missing supported rpcs to doc/descriptors.md
c7ec9a1888 Add missing supported rpcs to doc/descriptors.md (Andrew Toth)

Pull request description:

  Improve descriptor docs by adding missing rpcs.

ACKs for top commit:
  fanquake:
    ACK c7ec9a1888 - I think this has been bikeshed enough.
  jonatack:
    ACK c7ec9a1888

Tree-SHA512: 783219928ed7edc904b507bb30e2eefd8ca9f11225e1460fedecd755f9511055adcc52cc49f66ba840e121883e40753061db76a243ee6e0091daf1fc396ae59a
2020-01-31 08:43:58 +08:00
fanquake
d176aeafde Merge #18031: Remove GitHub Actions CI workflow.
085423b978 Remove GitHub Actions CI workflow. (Aaron Clauson)

Pull request description:

  While the GitHub Action CI workflow has permissions to make commits it's not suitable.

  As per #17803.

ACKs for top commit:
  fanquake:
    ACK 085423b978

Tree-SHA512: 62c19dda6164a563dc87e394314afc57ef2494b7e4af148b83220f1b2b0bef9ff309b5cde5d9bed3633429b77041871993ac5c73b0cbe7f720d6a4e60e3e816a
2020-01-31 08:21:26 +08:00
Andrew Toth
c7ec9a1888 Add missing supported rpcs to doc/descriptors.md 2020-01-30 18:34:25 -05:00
MarcoFalke
1d1f8bbf57 Merge #16115: On bitcoind startup, write config args to debug.log
b951b0973c on startup, write config options to debug.log (Larry Ruane)

Pull request description:

  When a developer is examining `debug.log` after something goes wrong, it's often useful to know the exact options the failing instance of `bitcoind` was started with. Sometimes the `debug.log` file is all that's available for the analysis. This PR logs the `bitcoin.conf` entries and command-line arguments to `debug.log` on startup.

ACKs for top commit:
  MarcoFalke:
    ACK b951b0973c 🐪
  jonatack:
    ACK b951b0973c reviewed diff, re-code review, built, ran tests, launched bitcoind and reviewed debug log output, verified value of  `str` debug log in the added unit test.

Tree-SHA512: bbca4fb3d49f99261758302bde0b8b67300ccc72e7380b01f1f66a146ae8a008a045df0ca5ca9664caff034d0ee38ea7ef38a50f38374525608c07ba52790358
2020-01-31 11:10:56 +13:00
Andrew Chow
d67055e00d Upgrade or rewrite encrypted key checksums
If fDecryptionThoroughlyChecked is false, after a key has been checked,
write (or rewrite) its checksum. This serves to upgrade wallets
and correct those which have the checksum corrupted but not the key.
2020-01-30 14:44:22 -05:00
Andrew Chow
c9a9ddb414 Set fDecryptionThoroughlyChecked based on whether crypted key checksums are valid
Change fDecryptionThoroughlyChecked to default to true so that it can
latch to false when an invalid checksum is seen. Checksums may be invalid
if the wallet does not have checksums or if the wallet became corrupted.

It is safe to default fDecryptionThoroughlyChecked to true because any
existing wallet without a checksum will set it to false. Any new or
blank wallet where encrypted keys are added will then set this to true
when the first encrypted key is generated by virtue of CheckDecryptionKey
doing that during the initial Unlock prior to keys being added.
2020-01-30 14:44:22 -05:00
Andrew Chow
a8334f7ac3 Read and write a checksum for encrypted keys 2020-01-30 14:42:50 -05:00
Aaron Clauson
085423b978 Remove GitHub Actions CI workflow. 2020-01-30 18:45:28 +00:00
practicalswift
4d2aceaad8 tests: Add fuzzer asmap to FUZZERS_MISSING_CORPORA (temporarily) 2020-01-30 16:06:02 +00:00
practicalswift
8d07706985 tests: Add fuzzing harness for AS-mapping (asmap) 2020-01-30 16:04:38 +00:00
Wladimir J. van der Laan
3b69310beb Merge #17984: test: Add p2p test for forcerelay permission
aaaae4d0eb test: Add p2p test for forcerelay permission (MarcoFalke)
fa6b57bcaa test: Fix whitespace in p2p_permissions.py (MarcoFalke)
faf40810d7 test: Make msg_tx a witness tx (MarcoFalke)

Pull request description:

  The commit `test: Make msg_tx a witness tx` is needed so that the python mininode does not strip the witness from transactions before sending them over p2p. The commit should also be done to keep symmetry with msg_block. See:

  *  tests: Make msg_block a witness block #15982

ACKs for top commit:
  laanwj:
    ACK aaaae4d0eb

Tree-SHA512: b4b546c88f7f0576cb512f0872bc6bef9d4df65783803f226986e56175937f418aa1ed906417ac909f27f1fd521d64629621fda83250fa925c46ef9513db0e4c
2020-01-30 15:41:59 +01:00
MarcoFalke
7fcaa8291c Merge #18009: tests: Add fuzzing harness for strprintf(…)
cc668d06fb tests: Add fuzzing harness for strprintf(...) (practicalswift)
ccc3c76e2b tests: Add fuzzer strprintf to FUZZERS_MISSING_CORPORA (temporarily) (practicalswift)
6ef04912af tests: Update FuzzedDataProvider.h from upstream (LLVM) (practicalswift)

Pull request description:

  Add fuzzing harness for `strprintf(…)`.

  Update `FuzzedDataProvider.h`.

  Avoid hitting some issues in tinyformat (reported upstreams in https://github.com/c42f/tinyformat/issues/70).

  ---

  Found issues in tinyformat:

  **Issue 1.** The following causes a signed integer overflow followed by an allocation of 9 GB of RAM (or an OOM in memory constrained environments):

  ```
  strprintf("%.777777700000000$", 1.0);
  ```

  **Issue 2.** The following causes a stack overflow:

  ```
  strprintf("%987654321000000:", 1);
  ```

  **Issue 3.** The following causes a stack overflow:

  ```
  strprintf("%1$*1$*", -11111111);
  ```

  **Issue 4.** The following causes a `NULL` pointer dereference:

  ```
  strprintf("%.1s", (char *)nullptr);
  ```

  **Issue 5.** The following causes a float cast overflow:

  ```
  strprintf("%c", -1000.0);
  ```

  **Issue 6.** The following causes a float cast overflow followed by an invalid integer negation:

  ```
  strprintf("%*", std::numeric_limits<double>::lowest());
  ```

Top commit has no ACKs.

Tree-SHA512: 9b765559281470f4983eb5aeca94bab1b15ec9837c0ee01a20f4348e9335e4ee4e4fecbd7a1a5a8ac96aabe0f9eeb597b8fc9a2c8faf1bab386e8225d5cdbc18
2020-01-31 02:56:49 +13:00
MarcoFalke
0130abbdb7 Merge #18018: tests: reset fIsBareMultisigStd after bare-multisig tests
1b96a3cd1e tests: reset fIsBareMultisigStd after bare-multisig tests (fanquake)

Pull request description:

  Fixes: #18015

  The bug this fixes is two-part.

  1. The `fIsBareMultisigStd` global is being reused by other tests,
  such as [script_p2sh_tests(set)](https://github.com/bitcoin/bitcoin/blob/master/src/test/script_p2sh_tests.cpp#L150), after being set to false.

  2. The order our tests run in doesn't always? seem to be random,
  which meant that the `script_p2sh` tests would only fail if they
  were run in an order where the `transaction_tests` ran first,
  mutating the `fIsBareMultisigStd` global.

  This doesn't seem to happen when running make check, but if you
  run `src/test/test_bitcoin and pass --random=99999`, the failure
  in `script_p2sh` will occur (on most, but maybe not all systems):

  ```bash
  src/test/test_bitcoin --random=99999
  Running 389 test cases...
  test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[1].IsStandard
  test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[2].IsStandard
  test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[3].IsStandard

  *** 3 failures are detected in the test module "Bitcoin Core Test Suite"
  ```

  The new test for bare multisig was introduced in #17502.

ACKs for top commit:
  Empact:
    Code Review ACK 1b96a3cd1e
  theStack:
    ACK https://github.com/bitcoin/bitcoin/pull/18018/commits/1b96a3cd1ebe725896f59614903184289fe62c

Tree-SHA512: fd7578f9f3faa44d236cd007fc25e31f061acabdb8458559fde0e67d11ab5cafed15305993270c9943a50326574bc5f5301b09494a5b0d2de69e64978093ed45
2020-01-31 02:50:47 +13:00
João Barbosa
cb8a86d9f9 gui: Remove WalletView and BitcoinGUI circular dependency 2020-01-30 11:38:06 +00:00
João Barbosa
ac3d10777d gui: Add transactionClicked and coinsSent signals to WalletView 2020-01-30 11:31:07 +00:00
fanquake
44c2400bcc Merge #18026: psbt_wallet_tests: use unique_ptr for GetSigningProvider
1115ba693b psbt_wallet_tests: use unique_ptr for GetSigningProvider (Anthony Towns)

Pull request description:

  #17261 changed GetSigningProvider to return a unique_ptr, but #17156 made psbt_wallet_tests use it as well, and wasn't correspondingly updated.

ACKs for top commit:
  fanquake:
    ACK 1115ba693b
  meshcollider:
    Thanks! utACK 1115ba693b

Tree-SHA512: f0191c9b00780e6d1445fa4ec531456758b468b5bca8660474d22b1edb5f48a636a940656c9bdbe466b8bffad7af1e57e0756239906e901d60c69c3124d3bff4
2020-01-30 15:23:53 +08:00
Anthony Towns
1115ba693b psbt_wallet_tests: use unique_ptr for GetSigningProvider 2020-01-30 16:09:13 +10:00
Pieter Wuille
37d800bea0 Add a constant for the maximum vector allocation (5 Mbyte) 2020-01-29 20:56:05 -08:00
Samuel Dobson
2d6e76af24 Merge #17261: Make ScriptPubKeyMan an actual interface and the wallet to have multiple
3f373659d7 Refactor: Replace SigningProvider pointers with unique_ptrs (Andrew Chow)
3afe53c403 Cleanup: Drop unused GUI learnRelatedScripts method (Andrew Chow)
e2f02aa59e Refactor: Copy CWallet signals and print function to LegacyScriptPubKeyMan (Andrew Chow)
c729afd0a3 Box the wallet: Add multiple keyman maps and loops (Andrew Chow)
4977c30d59 refactor: define a UINT256_ONE global constant (Andrew Chow)
415afcccd3 HD Split: Avoid redundant upgrades (Andrew Chow)
01b4511206 Make UpgradeKeyMetadata work only on LegacyScriptPubKeyMan (Andrew Chow)
4a7e43e846 Store p2sh scripts in AddAndGetDestinationForScript (Andrew Chow)
501acb5538 Always try to sign for all pubkeys in multisig (Andrew Chow)
81610eddbc List output types in an array in order to be iterated over (Andrew Chow)
eb81fc3ee5 Refactor: Allow LegacyScriptPubKeyMan to be null (Andrew Chow)
fadc08ad94 Locking: Lock cs_KeyStore instead of cs_wallet in legacy keyman (Andrew Chow)
f5be479694 wallet: Improve CWallet:MarkDestinationsDirty (João Barbosa)

Pull request description:

  Continuation of wallet boxes project.

  Actually makes ScriptPubKeyMan an interface which LegacyScriptPubkeyMan. Moves around functions and things from CWallet into LegacyScriptPubKeyMan so that they are actually separate things without circular dependencies.

  ***

  Introducing the `ScriptPubKeyMan` (short for ScriptPubKeyManager) for managing scriptPubKeys and their associated scripts and keys. This functionality is moved over from `CWallet`. Instead, `CWallet` will have a pointer to a `ScriptPubKeyMan` for every possible address type, internal and external. It will fetch the correct `ScriptPubKeyMan` as necessary. When fetching new addresses, it chooses the `ScriptPubKeyMan` based on address type and whether it is change. For signing, it takes the script and asks each `ScriptPubKeyMan` for whether that `ScriptPubKeyMan` considers that script `IsMine`, whether it has that script, or whether it is able to produce a signature for it. If so, the `ScriptPubKeyMan` will provide a `SigningProvider` to the caller which will use that in order to sign.

  There is currently one `ScriptPubKeyMan` - the `LegacyScriptPubKeyMan`. Each `CWallet` will have only one `LegacyScriptPubKeyMan` with the pointers for all of the address types and change pointing to this `LegacyScriptPubKeyMan`. It is created when the wallet is loaded and all keys and metadata are loaded into it instead of `CWallet`. The `LegacyScriptPubKeyMan` is primarily made up of all of the key and script management that used to be in `CWallet`. For convenience, `CWallet` has a `GetLegacyScriptPubKeyMan` which will return the `LegacyScriptPubKeyMan` or a `nullptr` if it does not have one (not yet implemented, but callers will check for the `nullptr`). For purposes of signing, `LegacyScriptPubKeyMan`'s `GetSigningProvider` will return itself rather than a separate `SigningProvider`. This will be different for future `ScriptPubKeyMan`s.

  The `LegacyScriptPubKeyMan` will also handle the importing and exporting of keys and scripts instead of `CWallet`. As such, a number of RPCs have been limited to work only if a `LegacyScriptPubKeyMan` can be retrieved from the wallet. These RPCs are `sethdseed`, `addmultisigaddress`, `importaddress`, `importprivkey`, `importpubkey`, `importmulti`, `dumpprivkey`, and `dumpwallet`. Other RPCs which relied on the wallet for scripts and keys have been modified in order to take the `SigningProvider` retrieved from the `ScriptPubKeyMan` for a given script.

  Overall, these changes should not effect how everything actually works and the user should experience no difference between having this change and not having it. As such, no functional tests were changed, and the only unit tests changed were those that were directly accessing `CWallet` functions that have been removed.

  This PR is the last step in the [Wallet Structure Changes](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Class-Structure-Changes).

ACKs for top commit:
  instagibbs:
    re-utACK 3f373659d7
  Sjors:
    re-utACK 3f373659d7 (it still compiles on macOS after https://github.com/bitcoin/bitcoin/pull/17261#discussion_r370377070)
  meshcollider:
    Tested re-ACK 3f373659d7

Tree-SHA512: f8e2b8d9efa750b617691e8702d217ec4c33569ec2554a060141d9eb9b9a3a5323e4216938e2485c44625d7a6e0925d40dea1362b3af9857cf08860c2f344716
2020-01-30 17:21:21 +13:00
fanquake
1b96a3cd1e tests: reset fIsBareMultisigStd after bare-multisig tests
The bug this fixes is two-part.

1.The fIsBareMultisigStd global is being reused by other tests,
i.e script_p2sh_tests(set), after being set to false.

2. The order our tests run in doesn't always? seem to be random,
which meant that the script_p2sh tests would only fail if they
were run in an order where transaction_tests ran first, mutating
the fIsBareMultisigStd global.

This doesn't seem to happen when running make check, but if you
run src/test/test_bitcoin and pass --random=99999, the failure
in script_p2sh:

test/script_p2sh_tests.cpp:200: error: in "script_p2sh_tests/set": txTo[1].IsStandard

will occur (on most systems).

The new test was introduced in 1bb5d517aa.
2020-01-30 08:41:24 +08:00
MarcoFalke
638239de75 Merge #18022: test: Fix appveyor test_bitcoin build of *.raw
fa1a46e7f4 build: Fix appveyor test_bitcoin build of *.raw (MarcoFalke)

Pull request description:

  Fixes #18020

Top commit has no ACKs.

Tree-SHA512: c0b3ca4f95b46543bb3bc6d254300c832a69feca79f5de4e13cafd4c962ae53903069ec7a8c9573761eefa5cec617992b70750b067ee42231dc74170ba6c3b10
2020-01-29 18:12:29 -05:00
Larry Ruane
b951b0973c on startup, write config options to debug.log 2020-01-29 15:44:00 -07:00
MarcoFalke
fa1a46e7f4 build: Fix appveyor test_bitcoin build of *.raw 2020-01-29 16:57:41 -05:00
Samuel Dobson
aabec94541 Merge #17719: Document better -keypool as a look-ahead safety mechanism
f41d589669 Document better -keypool as a look-ahead safety mechanism (Antoine Riard)

Pull request description:

  If after a backup, an address is issued beyond the initial
  keypool range and none of the addresses in this range
  is seen onchain, if a wallet is restored from backup, even in
  case of rescan, funds may be loss due to the look-ahead
  buffer not being incremented and so restored wallet not detecting
  onchain out-of-range address as derived from its seed.

  This scenario is theoretically unavoidable due to the requirement
  of the keypool to have a max size. However, given the default
  keypool size, this is unlikely. Document better keypool size
  implications to avoid user setting a too low value.

  While reviewing #17681, it took me a while to figure out the safety implications of keypool, I find it would be better to document this a bit farther to avoid users shooting themselves in the foot. For further context & discussion, see https://github.com/bitcoin/bitcoin/pull/17681#issuecomment-563613452

ACKs for top commit:
  ryanofsky:
    Code review ACK f41d589669. Just "Warning:" prefix added since the last review
  jonatack:
    ACK f41d589669 code review and build/test. The added `Warning:` since last review is a good addition.

Tree-SHA512: d3d0ee88fcdfc5c8841a2bd4bada0e4eeb412a0dce5054e5fb023643c2fa57206a0f3efb06890c245528dc4431413ed2fd5645b9319d26245d044c490b7f0db0
2020-01-30 09:10:15 +13:00
Suhas Daftuar
a029e18c2b Use rolling bloom filter of recent block tx's for AlreadyHave() check
In order to determine whether to download or process a relayed transaction, we
try to determine if we already have the transaction, either in the mempool, in
our recently rejected filter, in our orphan pool, or already confirmed in the
chain itself.

Prior to this commit, the heuristic for checking the chain is based on whether
there's an output corresponding to the 0- or 1-index vout in our coin cache.
While that is a quick check, it is very imprecise (say if those outputs were
already spent in a block) -- we can do better by just keeping a rolling bloom
filter of the transactions in recent blocks, which will capture the case of a
transaction which has been confirmed and then fully spent already.

To avoid relay problems for transactions which have been included in a recent
block but then reorged out of the chain, we clear the bloom filter whenever a
block is disconnected.
2020-01-29 09:37:21 -05:00
Wladimir J. van der Laan
c1607b5df4 Merge #17957: Serialization improvements step 3 (compression.h)
4de934b9b5 Convert compression.h to new serialization framework (Pieter Wuille)
ca34c5cba5 Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters (Pieter Wuille)

Pull request description:

  This is the next piece of the puzzle from #10785. It includes:
  * The `FORMATTER_METHODS` macro, similar to `SERIALIZE_METHODS`, for defining a formatter with a unified serialization/deserialization implementation.
  * Updating `compression.h` to consist of 3 formatters, rather than old-style wrappers (`ScriptCompression`, `AmountCompression`, `TxOutCompression`).

ACKs for top commit:
  laanwj:
    code review ACK 4de934b9b5
  ryanofsky:
    Code review ACK 4de934b9b5. Only change since last review is removing REF usages

Tree-SHA512: d52ca21eb1ce87d9bc3c90d00c905bd4fada522759aaa144c02a58b4d738d5e8647c0558b8ce393c707f6e3c4d20bf93781a2dcc1e1dcbd276d9b5ffd0e02cd6
2020-01-29 15:10:59 +01:00
MarcoFalke
3b5b276734 Merge #17942: doc: Improve fuzzing docs for macOS users
b6c3e84e87 doc: Improve fuzzing docs for macOS users (Fabian Jahr)

Pull request description:

  Adds several helpful hints for macOS users trying to get fuzzers to run locally using AFL or libFuzzer. These are partly based on this comment https://github.com/bitcoin/bitcoin/issues/17657#issuecomment-562869600 and discussions in the review club for #17860. See: https://bitcoincore.reviews/17860.html

  Based on the doc in the current state I could not compile fuzzers for AFL or libFuzzer. Using these hints, I can
  - compile and run fuzzers with AFL
  - compile but **not** run fuzzers with libFuzzer

  Fuzzers compiled with libFuzzers may be running but don't produce any output. Looking for others to test this to see if it is an issue with my local system. Especially interesting if you have been running libFuzzer fuzzers successfully on macOS before.

  Edit: Closes #17914

ACKs for top commit:
  MarcoFalke:
    ACK b6c3e84e87
  Sjors:
    ACK b6c3e84
  fanquake:
    ACK b6c3e84e87 - I think this has been nitpicked enough, and importantly the commands look better now.

Tree-SHA512: fdbacbcf10e9353a4ac3d22edf88663e33185ad2f244b986ff74c513de05f9fa62c4d8b17985d2f9288834c124b352cf52280627b5ff095735b411b12482e2ec
2020-01-29 08:45:09 -05:00
Wladimir J. van der Laan
01fc5891fb Merge #16702: p2p: supplying and using asmap to improve IP bucketing in addrman
3c1bc40205 Add extra logging of asmap use and bucketing (Gleb Naumenko)
e4658aa8ea Return mapped AS in RPC call getpeerinfo (Gleb Naumenko)
ec45646de9 Integrate ASN bucketing in Addrman and add tests (Gleb Naumenko)
8feb4e4b66  Add asmap utility which queries a mapping (Gleb Naumenko)

Pull request description:

  This PR attempts to solve the problem explained in #16599.
  A particular attack which encouraged us to work on this issue is explained here  [[Erebus Attack against Bitcoin Peer-to-Peer Network](https://erebus-attack.comp.nus.edu.sg/)] (by @muoitranduc)

  Instead of relying on /16 prefix to diversify the connections every node creates, we would instead rely on the (ip -> ASN) mapping, if this mapping is provided.

  A .map file can be created by every user independently based on a router dump, or provided along with the Bitcoin release. Currently we use the python scripts written by @sipa to create a .map file, which is no larger than 2MB (awesome!).

  Here I suggest adding a field to peers.dat which would represent a hash of asmap file used while serializing addrman (or 0 for /16 prefix legacy approach).
  In this case, every time the file is updated (or grouping method changed), all buckets will be re-computed.
  I believe that alternative selective re-bucketing for only updated ranges would require substantial changes.

  TODO:
  - ~~more unit tests~~
  - ~~find a way to test the code without including >1 MB mapping file in the repo.~~
  - find a way to check that mapping file is not corrupted (checksum?)
  - comments and separate tests for asmap.cpp
  - make python code for .map generation public
  - figure out asmap distribution (?)

  ~Interesting corner case: I’m using std::hash to compute a fingerprint of asmap, and std::hash returns size_t. I guess  if a user updates the OS to 64-bit, then the hash of asap will change? Does it even matter?~

ACKs for top commit:
  laanwj:
    re-ACK 3c1bc40205
  jamesob:
    ACK 3c1bc40205 ([`jamesob/ackr/16702.3.naumenkogs.p2p_supplying_and_using`](https://github.com/jamesob/bitcoin/tree/ackr/16702.3.naumenkogs.p2p_supplying_and_using))
  jonatack:
    ACK 3c1bc40205

Tree-SHA512: e2dc6171188d5cdc2ab2c022fa49ed73a14a0acb8ae4c5ffa970172a0365942a249ad3d57e5fb134bc156a3492662c983f74bd21e78d316629dcadf71576800c
2020-01-29 13:55:43 +01:00
fanquake
c434282d2c Merge #18008: test: only declare a main() when fuzzing with AFL
b35567fe0b test: only declare a main() when fuzzing with AFL (fanquake)

Pull request description:

  This fixes fuzzing using [libFuzzer](https://llvm.org/docs/LibFuzzer.html) on macOS, which caused a few issues during the recent review club. macOS users could only fuzz using afl, or inside a VM.

  It seems that the `__attribute__((weak))` marking is not quite enough to properly mark `main()` as weak on macOS. See Apples docs on [Frameworks and Weak Linking](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Concepts/WeakLinking.html#//apple_ref/doc/uid/20002378-107262-CJBJAEID).

  Have tested fuzzing using libFuzzer and AFL with this patch.

ACKs for top commit:
  MarcoFalke:
    ACK b35567fe0b
  fjahr:
    ACK b35567f

Tree-SHA512: b881fdd98c7e1587fcf44debd31f5e7a52df938059ab91c41d0785077b3329b793e051a2bf2eee64488b9f6029d9288c911052ec23ab3ab8c0561a2be1682dae
2020-01-29 20:48:30 +08:00
fanquake
e061b8dc8f Merge #17971: refactor: Remove redundant conditional
e80317be5f refactor: Remove redundant conditional (Bushstar)

Pull request description:

  Conditional check against fMaster is now redundant as it is already checked as true. This originally made sense as the outer conditional was:

  f9cae832e6/src/checkqueue.h (L86)

  Removal of fQuit happened in the commit below.

  30ded3e3d8 (diff-88316c9aa9514c038c9304297e672da5)

ACKs for top commit:
  theStack:
    ACK e80317be5f
  hebasto:
    ACK e80317be5f, I have reviewed the code, and it looks OK, I agree it can be merged.
  promag:
    ACK e80317be5f.
  emilengler:
    re-ACK e80317be5f
  practicalswift:
    ACK e80317be5f
  Empact:
    ACK e80317be5f

Tree-SHA512: 136ea1d02e3d65100a8758730617ccede7864e08e8404e42e65d45d4bf95a3bfea2ab9895c6e8833abd654557d3efbba02b25297a2a5eefc36a11e97bbe9134f
2020-01-29 20:16:00 +08:00
fanquake
1326092e6c Merge #17156: psbt: check that various indexes and amounts are within bounds
deaa6dd144 psbt: check output index is within bounds before accessing (Andrew Chow)
f1ef7f0aa4 Don't calculate tx fees for PSBTs with invalid money values (Andrew Chow)

Pull request description:

  Fixes #17149

  Two classes of issues were found by the psbt fuzzer: values out of range and causing overflows, and prevout indexes being out of range. This PR fixes both.

  When accessing a specific output using the index given in the tx, check that it is actually a possible output before trying to access the output.

  When summing and checking amounts for `decodepsbt` and `analyzepsbt`, make sure that the values are actually valid money values.. Otherwise, stop summing and don't show the fee. For `analyzepsbt`, return that the next role is the Creator since the Creator needs to remake the transaction to be valid.

ACKs for top commit:
  practicalswift:
    ACK deaa6dd144 -- only change since last ACK was the addition of tests
  gwillen:
    tested ACK deaa6dd, would also like to see this merged!

Tree-SHA512: 06c36720bbb5a7ab1c29f7d15878bf9f0d3e5760c06bff479d412e1bf07bb3e0e9ab6cca820a4bfedaab71bfd7af813807e87cbcdf0af25cc3f66a53a06dbcfd
2020-01-29 19:39:50 +08:00
fanquake
b35567fe0b test: only declare a main() when fuzzing with AFL
libFuzzer will provide a main(). This also fixes a weak linking
issue when fuzzing with libFuzzer on macOS.
2020-01-29 08:18:22 +08:00
Fabian Jahr
b6c3e84e87 doc: Improve fuzzing docs for macOS users 2020-01-29 00:46:21 +01:00
MarcoFalke
fe48ac8580 Merge #18013: bench: Fix benchmarks filters
0dae5a5c34 Fix benchmarks filters (Elichai Turkel)

Pull request description:

  The bug was introduced in https://github.com/bitcoin/bitcoin/pull/17781
  before this fix `./src/bench/bench_bitcoin -filter=*` will fail with:

  ```
  # Benchmark, evals, iterations, total, min, max, median
  bench_bitcoin: bench/bench.cpp:119: static void benchmark::BenchRunner::RunAll(benchmark::Printer&, uint64_t, double, const string&, bool): Assertion `g_testing_setup == nullptr' failed.
  Aborted (core dumped)
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 0dae5a5c34

Tree-SHA512: 43de4c7f4a5f29593972cf3bc822429466d0609c159c95d37c9e5370be392ace698b218a65542c7d53bfa52db7377ebdab808501ae109c2249f7f956bd318312
2020-01-28 14:02:13 -05:00
Wladimir J. van der Laan
677fb8e923 test: Add ubsan surpression for crc32c 2020-01-28 17:01:48 +01:00
Aaron Clauson
8e68bb1dde build: Disable msvc warning 4722 for leveldb build
This prevents AppVeyor from failing on a warning in leveldb's new
Windows environment.
2020-01-28 17:01:48 +01:00
Aaron Clauson
be23949765 build: MSVC changes for leveldb update 2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
9ebdf04757 build: CRC32C build system integration 2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
402252a808 build: Add LCOV exception for crc32c 2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
3a037d0067 test: Add crc32c exception to various linters and generation scripts 2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
84ff1b2076 test: Add crc32c to subtree check linter 2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
7cf13a5134 doc: Add crc32c subtree to developer notes 2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
24d02a9ac0 build: Update build system for new leveldb
Upstream leveldb switched build systems, which means we need to define
a few different values.
2020-01-28 17:01:48 +01:00
Wladimir J. van der Laan
3acaa13b15 Import crc32c using subtree merge as as 'src/crc32c' 2020-01-28 17:00:01 +01:00
Wladimir J. van der Laan
2e1819311a Squashed 'src/crc32c/' content from commit 224988680f7673cd7c769963d4035cb315aa3388
git-subtree-dir: src/crc32c
git-subtree-split: 224988680f7673cd7c769963d4035cb315aa3388
2020-01-28 16:59:58 +01:00
Wladimir J. van der Laan
20a6babfa9 Update to leveldb upstream using subtree merge 2020-01-28 16:59:07 +01:00
Wladimir J. van der Laan
66480821b3 Squashed 'src/leveldb/' changes from f545dfabff4c2e9836efed094dba99a34fbc6b88..f8ae182c1e5176d12e816fb2217ae33a5472fdd7
f8ae182c1e5176d12e816fb2217ae33a5472fdd7 Adds unicode support to Windows environment.
92ae82c78f225de84040c51e07fd0b4a61caed99 Increase maximum read-only mmap()s used from 1000 to 4096 on 64-bit systems
d42e63d49d9df05b12cd00af4ffc5f2b3edf7e21 Do not crash if filesystem can't fsync
bf2c2090b7ee12c5d85b85f08649b6e685f8715f Add filename to corruption errors
0c40829872a9f00f38e11dc370ff8adb3e19f25b Remove redundant PROJECT_SOURCE_DIR usage from CMake config.
5abdf4c019e51fce59d34c21b13bf4e0a948828a Fix installed target definition.
cf4d9ab23de7ec36b8e00536b7450f02c639cd87 Test CMake installation on Travis.
95d0ba1cb046bfd76619b8b80e14ee1b2897d219 Renamed local variable in DBImpl::Write.
657ba514298a726c7533f3106d3778062b59d75f Added return in Version::Get::State::Match to quiet warning.
370d532a00581ca79c87af7d7811e56de0ca52a8 Using CMake's check_cxx_compiler_flag to check support for -Wthread-safety.
45ee61579c1eb3accd6c88c922ec468dd61beea8 Update Travis CI configuration.
60db170a43a373d734c5b9f19693d36c75251c39 Fix tsan problem in env_test.
21304d41f77990b8edabbdab33b222bd5ceb5f18 Merge pull request #698 from neal-zhu:master
5e921896eedf87b0fb06bc8a1fd0991b9ac64131 drop fileds in State that are duplicates of fileds in Saver and fix typo
53e280b56866ac4c90a9f5fcfe02ebdfd4a19832 Simplify unlocking in DeleteObsoleteFiles.
046216a7ca6fb17a40cf8aa5598d90c825212a3d Add "leveldb" subdirectory to public include paths.
9ee91ac747ddf26f484d54f9aa474ccc4a2e0359 Ending sentences with periods in README.md.
e0d5f83a4f80060fe5b5d80025f0ad049bca430e Align EnvPosix and EnvWindows.
69061b464ab1da287da9b7ffec1ed911b754403b Disable exceptions and RTTI in CMake configuration.
107a75b62c19cce901ce10619b63c4b7acc9a0be cache Saver in State object
76ca1162768e5c89f1a49946a1f286c702ae27ae fix bug(uninitialized options pointer in State)
f668239bb262609146496b854e1ec3cea9cd1a83 remove TODO in Version::ForEachOverlapping
177cd08629883c409f7a01f90f7084bc5518f1ef format
8fa7a937ee8f38d8869357b0f27f120c5c58f4c9 fix bug
6a90bb91ee72642241fdbeefa673f88370c7b245 use ForEachOverlapping to impl Get
4cb80b7ddce6ff6089b15d8cfebf746fc1572477 Merge pull request #386 from ivanabc:master
72a38ff7f206b3924ac009a12a1838d6a0bdab03 Replace "> >" with ">>"
863f185970eff21e826e5fe1164a6215a515c23b unsigned char -> uint8_t
a3b71c1ff65e30ced00e85ebbca9ae5786af6626 Use GCC 9 on Travis CI
ae49533210e96bdee9c9479a7fa547f375a39c8b Add explicit typecasts to avoid compiler warning.
63d5315e1c224e52da8ec68d118c5b73ba2a63fc Merge branch 'master' into master
c00e177f3613068eda4bff4abfbd3bd4165a86e8 Guard DBImpl::versions_ by mutex_.
1d0b101165ddd34f26cc5c62b76f2a2e0d622483 Converted two for-loops to while-loops.
28e6d238be73e743c963fc0a26395b783a7565e2 Switch to using C++ 11 override specifier.
85cd40d108d8f8d91f58fd263c0f8428d11c34d5 Added unit test for InternalKey::DecodeFrom with empty string.
1aae5c9f29ea43ceca745efae012c4aa731e9374 Merge pull request #411 from proller:assert1
b7b86baec9ce47569affc5db54a20a6cc520e0f0 Using std::ostringstream in key DebugString.
3e6c000e18519cb22e0a44d0dea45b34daee4ee1 Merge pull request #457 from jellor:patch-2
1d94fe2f4d1dfdf1a6312bf4b36efcbe0c1bf576 Merge branch 'master' into patch-2
27dc99fb2642cadc87c9aaec82c54a2c725ee0d6 Fix EnvPosix tests on Travis CI.
9521545b062841409cf66eff0655feff09d9fd82 Formatting changes for prior O_CLOEXEC fix.
900f7d37eb3224059dd37afc6614d3158ddaeb8d Merge pull request #624 from adam-azarchs:master
a7528a5d2bd29126b60a277b528ed606b67c1771 Clean up util/coding.{h,cc}.
142035edd4b1ab431c0ecbd547d4a77f1eca0667 Initialize Stats::start_ before first use in Stats::Start().
e22b1cec6e1e0e2dec4c93b658acbfc56fb692c0 Merge pull request #365 from allangj:c-strict-prototypes
cd1ec032cd276409ba403cab4d0b2548dd26b890 Add argument definition for void c functions.
4bd052d7e8b0469b2b87664388e2a99cb212ecdb Consolidate benchmark code to benchmarks/.
506b1722ef1a58d87325575d9bbcd3c8869381c7 Convert missed virtual -> override in db_test.cc.
24424a1ef2c284f4ec30544a3458023362cbeacd Style cleanup.
9a56c49ed415df1b72ba1c84c8e7ed00de497f68 Merge pull request #679 from smartxworks:optimize-readseq
abf441b657c7e75091e2bd59449df6849358b812 Merge pull request #278 from wankai:master
78b39d68c15ba020c0d60a3906fb66dbf1697595 Bump the version number from 1.21 to 1.22.
9bd23c767601a2420478eec158927882b879bada Correct class/structure declaration order.
c784d63b931d07895833fb80185b10d44ad63cce Moved port/README to port/README.md.
297e66afc1dda3f3d7a7cc2022030164c302cb7a Format all files IAW the Google C++ Style Guide.
3724030179716fd8d95cf79339884c49afade8f9 Update Travis CI configuration.
d3d1c8a0f40a7eaa12a5bb702fa01786b7c3a646 don't check current key in DBIter::Next()
3dc9202f78a3eb30ee8c0267e4e4be2e3f986e45 [leveldb] Specifically export the WriteBatch::Handler inner class for Windows link
2ccb45c33aecd8b15000c0c622f45eb119b6b478 Check for possibly invalid offset in test.
7b1174519044339f07a023dc445b0d36425bd6db Changed Windows specific highlighting from bash to cmd.
2f008ac19ec783e4d0ba2161320241c99e9897e1 Initialize class members to default values in constructors.
ffabb1ae86cc4eb4516a7c0824c878c3b2d19e5d Merge pull request #665 from cheng-chang:coding
7da571cf2b954a107fa060698bfbfbba8e8318f8 Merge pull request #669 from pavel-pimenov:fix-readme-windows-mkdir
df4a323aafbf65996fec23de8b2dbb9d7e27ae11 Merge pull request #472 from zhoudayang:patch-1
5a2a472741f36ecf5b994439da5a64c6ab90c47f Fixed missing std namespaces and make_unique.
08e771901f454ac32643bd8e8cb2bcfa08026c0c Simplify issue320_test.
65e86f75ea30e44bc65327f92a16328684269acb Fix formatting of recent snapshot compaction fix.
7711e76766231bf93e0487c4530b2655e8c4c0b1 Merge pull request #339 from richcole-at-amazon:master
71ed7c401ec1b1e38d6f7cb9eb2fcff93c24d1f1 Fixed typo in comment in version_set.h.
09fa8868dbe0cb2701f0560c59ebb63cc17f1271 Align version/soversion CMake setup closer with other repositories.
20fb601aa9f68ff0aa147df22524b7d01758552b Fix snapshot compaction bug
37300aa54b8256dd2edfd504942eb2bd20823647 Restore soname versioning with CMake build
952be04df6edb936b8f7d0f652861100a7f61e97 Fix mkdir (windows)
56178ddaf4d3ba6c8d1cfb218610b1be3f5aa710 Update the version to 1.21 in preparation for a new release.
35619d248d909b197f68226c7d0a9ff947b82e8a Project import generated by Copybara.
416344de2fdffb3f17c565b984885d0122bfa1e9 leveldb: Register in copybara whitelist.
da94ac67e91679842a56a876f0b19b429d72de25 leveldb: Minor cleanup in ports.
bd24b963060861518c6648925f9708178562c992 leveldb: Silence unused argument warnings in MSVC.
6188a54ce95b47cc6bd398d7f2eb45d061857e45 leveldb: Add tests for empty keys and values.
cf1b5f473259e46c667f3fb5a28bcd884ee3a102 Remove unnecessary bit operation.
7035af5fc36657447054617759854a726d31dbe0 Two small fixes for the Windows implementation (#661)
6571279d6de21fe33caa31b2ea4170d34b15b10e fix a typo in the comment of skiplist_test.cc (#664)
15e227896621d01ebad4c5d4b3cc82a7a9b5b30b Use override consistently in leveldb::test::ErrorEnv.
ea49b27d062c4bc998616cef7944f7f9088a327d Switch corruption_test to use InMemEnv.
ce399ac28af7023b1aff0ede4986cb6d89b3c0b5 Always copy bytes to scratch buffer when reading w/MemEnv.
201f77d137f30ea46e789a2ad60e9119b6f990fc Inline defaults in options.
9ce30510d482f5b2fa2965201453f0fc914f700c Deleted dangling reference to deleted atomic_pointer.h.
7d8e41e49b8fddda66a2c5f0a6a47f1a916e8d26 leveldb: Replace AtomicPointer with std::atomic.
dd906262fd364c08a652dfa914f9995f6b7608a9 Make InMemoryEnv more consistent with filesystem based Env's.
cf1d1ab255de2a741695aec53d83e4f808f9e819 leveldb: Remove unused file port/win/stdint.h.
a20508dc6a18a34e05a6fc476a8d587fa9bb6608 Fix typo (#565)
04470825ac96cab0d9d16e4ed410349d082fbf82 Add AppVeyor (Windows CI) badge to README.
ed76289b259d42d0a57c147e791e2c235ed28805 Align windows_logger with posix_logger.
808e59ec6a160244960cda64b393968ffbdae72c Improve CI configuration.
c69d33b0ec3dad2a8063ad66da9d51a1d6309f4e Added native support for Windows.
75fceae7003e217e16b04433831da7528ae56881 Add O_CLOEXEC to open calls.
fe4494804f5e3a2e25485d32aeb0eb7d2f25732e leveldb: Make WriteBatch::ApproximateSize() const.
296de8d5b8e4e57bd1e46c981114dfbe58a8c4fa leveldb: Fix PosixWritableFile::Sync() on Apple systems.
b70493ca8586285b49e9888e2b528f71806bdc6e Fix fdatasync() feature detection in opensource build.
af7abf06ea061222c2c34d98e1995c5a901f374f Add back space to POSIX Logger.
58d70545af9ec7f30821f973b604f8e2a2f9ebdb Update Travis CI configuration.
1cb384088184be9840bd59b4040503a9fa9aee66 Clean up env_posix.cc.
a7dc502e9f11c2e5c911ba45b999676c43eaa51f Rework once initialization in env_posix.cc.
c43565dd398b2233db8eb49ba05234d62fb42e03 C++11 cleanup for util/mutexlock.h.
0145a94ab6bec48e596df499e8f6103e138a74ab Update .gitignore.
73d5834eceee8efa9a8ccfec77dc096a9e8ba18a Rework threading in env_posix.cc.
05709fb43eea34936c9f535edcb74d5e91a0b495 Remove InitOnce from the port API.
bb88f25115d20a6d73dfb6b16cc298db2f66948b Clean up PosixWritableFile in env_posix.cc.
7b945f200339aa47c24788d3ee9910c09c513843 Clean up posix_logger.h.
89af27bde59fbbb3025653812b45fec10a655cb7 Remove ssize_t from code that is not POSIX-specific.
03064cbbb2c00c3e6e41a78e8111d14a020f7d6f Simplify Limiter in env_posix.cc.
9b44da73d9b1d839c437e3fdaaa14ea08260dce4 Clarify comments for leveldb::Env file reading methods.
0ef2310f67f0c0b4ba3e6ad86d8138440af30d67 Remove GCC on OSX from the Travis CI matrix.
16a2b8bb3af5b1f54676256e55a5d3f0ec02da42 Expose WriteBatch::Append in the C API.
f7b0e1d901da26ac5ce6ad7f0a9806ce1440197e Expose WriteBatch::Append().
6caf73ad9dae0ee91873bcb39554537b85163770 Clean up Iterator.
6a6bdafcf10f5d4bef1ca52697c38d10c28b1a8b Corrected typo in docs: "cache" to "block_cache".
18683981505dc374ce29211c80a9552f8f2f4571 Clean up SnapshotImpl.
e7840de9f3db1a5eddedfecbbbc1ff72a4c2631a Fix documentation for log file growth.
bc23e00f955eadb9e26f8ce07c1c664e7b985ff0 Update default log file size in doc.
4de9594f6fbfd69043239a5705b5f32065f02d34 Add move constructor to Status.
d177a0263cce4344d05188521ad53459c369b940 Replace port_posix with port_stdcxx.
14cce848e7b8a040a8f457d5a796722a55e19597 Fix sign mismatch warnings in GCC.
8046a51b21114d3575421bfc78b1d98b1678720a Add forgotten <limits> header to util/logging.cc.
a0008deb679480fd30e845d7e52421af72160c2c Reimplement ConsumeDecimalNumber.
1f7dd5d5f6822f2b0b9f9e4c7d87d4535c122c0e Add tests for ConsumeDecimalNumber.
1cc8b10b8232e174d5bd1313959825727e03faa7 Document the building process.
09217fd0677a4fd9713c7a4d774c494a7d3c1f15 Replace NULL with nullptr in C++ files.
6a3b915166fce75aaf9ac209114a3ad9caa34171 Remove PLATFORM_IS_LITTLE_ENDIAN from port/posix.h.
260655b4c294991fe03bf6ab8b6d722ccfc41d32 Define LEVELDB_HAS_PORT_CONFIG_H for old compilers.
6fa45666703add49f77652b2eadd874d49aedaf6 Rename CMake project / targets from Leveldb to leveldb.
0db30413a4cfa8c980e675ba5cb96717d688af92 leveldb: Add more thread safety annotations.
04f39105c5a418905da8b7657ca244d672c99d3b Take <atomic> for granted in port/atomic_pointer.h.
74f032ff6f2465160366d865b1bb89a45dc2046b leveldb: Require C++11.
8e75db8623703cdc25ec3cd06f82129296672489 Remove build configuration for make.
df9a841a4fc9a04c7713542d75f50e749fb64b7b Add export.h to CMakeLists.txt
50fbc87e8c62a816d6afd4740e0652a13ac6dc3e Replace SIZE_MAX with std::numeric_limits.
739c25100e46576cdcdfff2d6f43f9f7008103c7 Add CMake build support.
0fa5a4f7b1ad9dc16b705bcad1f3ca913f187325 Extend thread safety annotations.
8143c12f3fc483b1ba61cdce11f9c1faf6d01bea Fix includes in util/testharness.h.
aece2068d7375f987685b8b145288c5557f9ce50 Remove extern from function declarations.
ddab751002588fe58955357d68d12b062e038d0d Add tests for {Old}InfoLogFileName().
7fd7c0072159abbca2660d91fc0667d5c17c4d16 Remove unused function ExtractValueType.
594cc987af2e0af6417c4ac2b947ee8cdad59e5e Bypass OSMemoryBarrier() warning on Mac.
49f35d3fc940a1e2d599d6ee3306eeb31a205e4b leveldb: Update Travis CI configuration for open source build.
623d014a54f8cf9b74ad6aaba9181ca1e65c43a1 Expose Env::GetTempDirectory() for use in C test.
8c8024ea33d8efc8c415597fb7fa1745002961d6 Switch HAVE_ library detection macros to 0/1.
41172a24016bc29fc795ed504737392587f54e3d Enable thread safety annotations in open source version.
47cb9e2a211e1d7157078ba7bab536beb29e56dc Add leveldb_options_set_max_file_size to the C API.
b5d4a22e64c7a6615b412f464026c808b58b1d34 Fixed style guide link in CONTRIBUTING.md
3da4d8b9899257386aeb5ffa345a6477c62ff7bf Deleted unused assignments in Reader.
0509414f858ae7c7225e29f3659a709afb324355 leveldb::DestroyDB will now delete empty directories.
23162ca1c6d891a9c5fe0e0fab1193cd54ed1b4f Fix typo (forgotten reference operator) in test.
5c39524f3639e6bf6ab49215152d24273e662986 Replace SSE-optimized CRC32C in POSIX port with external library.
ca216e493f32278f50a823811ab95f64cf0f839b leveldb: Rename SNAPPY to HAVE_SNAPPY.
25767d066ca995c055f04b78a31a6e518087e667 leveldb: Remove *_unlocked feature detection from POSIX port.
4a7e7f50dcf661cfffe71737650b0fb18e195d18 Add LEVELDB_EXPORT macro to export public symbols.
542590d2a8eee3838f40b01405baa6d2f6f8c700 leveldb: Include <algorithm> in util/env_test.cc.
8ae7998aabae4f208d77afcb930dafabade1b28d Fix FD leak in POSIX Env.
d9a9e02edf2b8187aa481416b36c49710026ab37 leveldb: Add tests for CL 170769101.
4447f9caced2bd09585c90f1b203c3aa8f4bbc40 Remove handling for unused LRUHandle representation special case.
2372ac574fdeb1235e70cdd86a2681d1ce05cf65 Fix file writing bug in CL 170738066.
1c75e88055e06da2939f9f4bd294625b76792815 Fix use of uninitialized value in LRUHandle.
7e12c00ecf1bb725e212618e7026e4d34d6cd3bb Fix issue 474: a race between the f*_unlocked() STDIO calls in env_posix.cc and concurrent application calls to fflush(NULL).
bcd9a8ea4a8aad23a3e101a23c61615bab2a093f Use portable CRC32C from google/crc32c.
ea0a7586b8615fd39c6b8f5a8a21a1f242129c2f Remove confusing and unnecessary if.
141e7671359d5e6c65ff70460774b53b94371df1 Simplify Table::Open() flow and remove a delete call.
09a3c8e7417547829b94bcdaa62cdf9e896f29a9 Switched variable type from int to uint64_t in ConsumeDecimalNumber.
2964b803b857932ff7499d7bebb61dc5514dab7c leveldb: Fix alignment code in SSE4.2-optimized CRC32C.
02f43c0fcde39823830493503e8a3f72fed43d24 Remove dead code.
0b402e96a76b19cd98e82402de636449a2613228 Use __APPLE__ instead of OS_MACOS. The former is compiler-provided.
8415f00eeedd96934d3578572d3802900e61a556 leveldb: Report missing CURRENT manifest file as database corruption.
69e2bd224b7f11e021527cb95bab18f1ee6e1b3b LevelDB: Add WriteBatch::ApproximateSize().
471f0b84ec3420c7565511eb6e2fee8e0a0550e8 fix comment
5b817400a0a5afe3badbb8859706a571882ababc fix comment
7d060117fa0d5cab7cb15b0cf127533bea9ffbc7 broken db: fix assertion in leveldb::InternalKey::Encode, mark base as corrupt
2883fcd849ca7b479d8a2f4fc929f0b6c7b9e372 set const property
e5f0a51fa44115fb083c1e71d5ddcd07a7aba719 reduce lock's range in DeleteObsoleteFiles
dd598676cd655dc2a2aaef47715ce18175d4a550 block_builder header file dependency fixed
REVERT: f545dfabff4c2e9836efed094dba99a34fbc6b88 Merge #18: Use utf-8 to decode filename
REVERT: f8e797a058b7a3993314e985dfdff8124214ba99 Use utf-8 to decode filename
REVERT: 2fc114812a04e6b88852fa37eedc556a464241f7 Merge #14: Fixes to allow building with msvc.
REVERT: 524b7e36a8e3bce6fcbcd1b5df09024283f325ba Merge #19: Increase maximum read-only mmap()s used from 1000 to 4096 on 64-bit systems
REVERT: 4874cb8d3e1dc7b9026b9faf51b9282c91f8ef40 Increase maximum number of read-only mmap()s used from 1000 to 4096 on 64 bit systems.
REVERT: 64052c76c567cff3dad32d1db0ef969d97b5882f Merge #15: Add filename to corruption errors
REVERT: 135ed0fb4e5d6440b174c4b80c147e915dd58969 Add filename to corruption errors
REVERT: d6eab93138884ee6c466fad5dadf2a1bfeb7cffd Fixes to allow building with msvc.
REVERT: c521b3ac654cfbe009c575eacf7e5a6e189bb5bb Merge #11: fixup define checks. Cleans up some oopses from #5.
REVERT: 8b1cd3753b184341e837b30383832645135d3d73 fixup define checks. Cleans up some oopses from #5.
REVERT: 6b1508d6d58caabf76cec2688b3428c9070b7bc9 Merge #6: Fixes typo
REVERT: fceb805426c66c8b79e2d75b83b4a35c57ad3a6e Merge #10: Clean up compile-time warnings (gcc 7.1)
REVERT: 0ec2a343f3be66ef6e25f9b9badc0256ac0911b7 Clean up compile-time warnings (gcc 7.1)
REVERT: d4c268a3571a66b3712ad24dfaf4b9f9671bcdf2 Merge #5: Move helper functions out of sse4.2 object
REVERT: 8d4eb0847041a26377dc99b1c4fb5c22d4841d5e Add HasAcceleratedCRC32C to port_win.h
REVERT: 77cfbfd250a690978a3b81d364054039467ed549 crc32: move helper functions out of port_posix_sse.cc
REVERT: 4c1e9e01688864a32217e541102fa8d2df9a3d59 silence compiler warnings about uninitialized variables
REVERT: 4953164851d1bc2fc653f60a98df5aa5c1dfcebd Merge #2: Prefer std::atomic over MemoryBarrier
REVERT: 2953978ef8cd8f0babcac86a52f5c688a5ad8fa8 Fixes typo
REVERT: f134284a1ce6e8e3ccc375a0a44300d9a87c51ab Merge #1: Merge upstream LevelDB 1.20
REVERT: 196962ff01c39b4705d8117df5c3f8c205349950 Add AcceleratedCRC32C to port_win.h
REVERT: ba8a445fdaa7cf3cb888a151e055330483b946f6 Prefer std::atomic over MemoryBarrier
REVERT: 1bdf1c34c5d903e466673a15103124568d995db4 Merge upstream LevelDB v1.20
REVERT: d31721eb0a115ac55506bb6735034bf915adc914 Merge #17: Fixed file sharing errors
REVERT: fecd449021504dc647c1a1226d72ab0d5efb84ad Fixed file sharing error in Win32Env::GetFileSize(), Win32SequentialFile::_Init(), Win32RandomAccessFile::_Init() Fixed error checking in Win32SequentialFile::_Init()
REVERT: 5b7510f1b79d9af1c5fe272a4587517a2579d3b7 Merge #14: Merge upstream LevelDB 1.19
REVERT: 0d969fd5708c9fd559d63be28664e1e840beb8ca Merge #16: [LevelDB] Do no crash if filesystem can't fsync
REVERT: c8c029b5b5793d3c9afef34afa53d10a910adf4e [LevelDB] Do no crash if filesystem can't fsync
REVERT: a31c8aa408d5594830f7cb20ead1ef1dff51b79e Add NewAppendableFile for win32 environment
REVERT: d40bc3fa5aaa5438d4d8f55ee83e6b3cd161ce02 Merge #13: Typo
REVERT: ebbd772d33d8596e5765a4d1251308d732d61355 Typo
REVERT: 1913d718ef8b07288229a75553862fcb343bf3ab Merge upstream LevelDB 1.19
REVERT: 20ca81f08fb7fa108923a091668e447dcf5c6b9d Merge pull request #9
REVERT: 7aa105e1a34e6e52b1e0de16d9d659a2af26fa0a leveldb: Win32WritableFile without memory mapping
REVERT: 7d41e6f89ff04ce9e6a742932924796f69c6e23d Merge upstream LevelDB 1.18
REVERT: 42dcc7edfc98c50038e4604fa630c626db17bf42 Merge upstream LevelDB 1.17.
REVERT: e991315d7fe4ca84a98902578106cbffa3dcccfd Merge upstream LevelDB 1.15.
REVERT: 02ac9f170b1c47e2c613cd47b8d7da45743af575 Merge upstream LevelDB 1.14.
REVERT: 936b4613ea4551992e6096b1e05eeefc09a20e3b Merge upstream LevelDB 1.13.
REVERT: be1b0ff1fcd6ad820a7fd111ac671fb51cc68001 On Mac OS X fsync does not guarantee write to disk. Use fcntl F_FULLFSYNC instead.
REVERT: a02ddf9b14d145e88185ee209ab8b01d8826663a Added GNU/kFreeBSD kernel name (TARGET_OS)
REVERT: 848746862caf337254a8a3e3a6bd3fa355db4fc8 CondVar::SignalAll was broken, leading to deadlocks on Windows builds. http://code.google.com/p/leveldb/issues/detail?id=149
REVERT: f6d84d1baf74a15ee8a0f73a81c647058bf816e9 Allow files to be opened for reading multiple times
REVERT: cb8e3f7adfaa48e09fb7a467086d69e4b6f948bd Checking whether closing succeeds
REVERT: d5317e8eda06d8dbbf04f08866c92323ccdbb43f Print actual Win32 error that occurred on file creation failure.
REVERT: 907f3084998fa4ce96b7abc6d9b12c7aa7b81c8c Port leveldb to MinGW32
REVERT: 9def2bfbf18dfbc0c3c95e90c91f043a6de3c1cb Mingw support for Windows LevelDB port
REVERT: 0a7b0748c71e64fd920eed94c26d69bc9ae77870 Pre-Vista leveldb::port::InitOnce implementation
REVERT: 31a2b09985842c833fbbd81e17f207c377217754 Native Windows LevelDB port
REVERT: 058a0357cd9650b214a199f81669a07d3eb4a298 Remove Snappy support

git-subtree-dir: src/leveldb
git-subtree-split: f8ae182c1e5176d12e816fb2217ae33a5472fdd7
2020-01-28 16:59:07 +01:00
Elichai Turkel
0dae5a5c34 Fix benchmarks filters 2020-01-28 17:10:15 +02:00
fanquake
2755b2b109 Merge #18010: test: rename test suite name "tx_validationcache_tests" to match filename
b3c4d9bac6 test: rename test suite name "tx_validationcache_tests" to match filename (Sebastian Falbesoner)

Pull request description:

  Quoting `src/test/README.md`, '`Adding test cases`':

  >     "The file naming convention is `<source_filename>_tests.cpp`
  >      and such files should wrap their tests in a test suite
  >      called `<source_filename>_tests`."

  Currently the unit test source file `txvalidationcache_tests.cpp` contains a unit test suite with the name `tx_validationcache_tests`, which is fixed by this PR. The following shell script shows that this is the only mismatch and for all other unit test source files the test suite names are correct:

  ```
   #!/bin/bash
   shopt -s globstar
   for test_full_filename in **/*_tests.cpp; do
       test_name_file=`basename $test_full_filename .cpp`
       test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
       if [ $test_name_file != $test_name_suite ]; then
           echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
       fi
   done
  ```

ACKs for top commit:
  practicalswift:
    ACK b3c4d9bac6 -- expected naming is better than unexpected naming :)
  kristapsk:
    ACK b3c4d9bac6

Tree-SHA512: 29d409b1eb22057ee2cc407508e2580d2bc03f412401df11b8ecf77be5ada6bda8f7d2cb5338c5e079490fa12242c1fd6230a09e47252c1b0d9fe535a828ca4c
2020-01-28 18:25:54 +08:00
fanquake
3774281327 Merge #17933: guix: Pin Guix using guix time-machine
88c83636d5 guix: Update documentation for time-machine (Carl Dong)
e6050884fd guix: Pin Guix using `guix time-machine` (Carl Dong)

Pull request description:

  An alternative to #16519, pinning our version of Guix and eliminating a `guix pull` and changing the default Guix profile of builders.

  I think this method might be superior, as it:
  - Eliminates the possibility of future changes to the `guix environment` command line interface breaking our builds
  - Eliminates the need to set up a separate channel repo

  It is a more general pinning solution than #16519.

  -----

  The reason why I didn't originally propose this is because `guix time-machine` is a recent addition to Guix, only available since `f675f8dec73d02e319e607559ed2316c299ae8c7`

ACKs for top commit:
  fanquake:
    ACK 88c83636d5

Tree-SHA512: 85e03b0987ffa86da73e02801e1cd8b7622698d70c4ba4e60561611be1e9717d661c2811a59b3e137b1b8eef2d0ba37c313867d035ebc89c3bd06a23a078064a
2020-01-28 17:14:50 +08:00
Sebastian Falbesoner
b3c4d9bac6 test: rename test suite name "tx_validationcache_tests" to match filename
Quoting src/test/README.md, 'Adding test cases':
    "The file naming convention is `<source_filename>_tests.cpp`
     and such files should wrap their tests in a test suite
     called `<source_filename>_tests`."

Currently the unit test source file txvalidationcache_tests.cpp contains a unit
test suite with the name tx_validationcache_tests, which is fixed by this commit.
The following shell script shows that this is the only mismatch and for all other
unit test source files the test suite names are correct:

 #!/bin/bash
 shopt -s globstar
 for test_full_filename in **/*_tests.cpp; do
     test_name_file=`basename $test_full_filename .cpp`
     test_name_suite=`sed -n "s/^.*TEST_SUITE(\(.*_tests\).*$/\1/p" $test_full_filename`
     if [ $test_name_file != $test_name_suite ]; then
         echo "TestFilename: $test_name_file != TestSuitname: $test_name_suite"
     fi
 done
2020-01-27 22:44:02 +01:00
Carl Dong
88c83636d5 guix: Update documentation for time-machine
Wait a minute, doc. Are you telling me you built a time machine... Out
of a functional package manager?
2020-01-27 16:41:20 -05:00
Carl Dong
e6050884fd guix: Pin Guix using guix time-machine 2020-01-27 16:38:07 -05:00
practicalswift
cc668d06fb tests: Add fuzzing harness for strprintf(...) 2020-01-27 21:31:42 +00:00
practicalswift
ccc3c76e2b tests: Add fuzzer strprintf to FUZZERS_MISSING_CORPORA (temporarily) 2020-01-27 21:31:42 +00:00
practicalswift
6ef04912af tests: Update FuzzedDataProvider.h from upstream (LLVM)
Upstream revision: a44ef027eb/compiler-rt/include/fuzzer/FuzzedDataProvider.h
2020-01-27 21:31:42 +00:00
Jonas Schnelli
b89f2d0599 Merge #17453: gui: Fix intro dialog labels when the prune button is toggled
4f7127d1e3 gui: Make Intro consistent with prune checkbox (Hennadii Stepanov)
4824a7d36c gui: Add Intro::UpdateFreeSpaceLabel() (Hennadii Stepanov)
daa3f3fa90 refactor: Add Intro::UpdatePruneLabels() (Hennadii Stepanov)
e4caa82a03 refactor: Replace static variable with data member (Hennadii Stepanov)
2bede28cd9 util: Add PruneGBtoMiB() function (Hennadii Stepanov)
e35e4b2ba0 util: Add PruneMiBtoGB() function (Hennadii Stepanov)

Pull request description:

  On master (a6f6333ba2) and on 0.19.0.1 the intro dialog with prune enabled (checkbox "Discard blocks..." is checked) provides a user with wrong info about the required disk space:

  ![DeepinScreenshot_bitcoin-qt_20191208112228](https://user-images.githubusercontent.com/32963518/70387510-8daab400-19ae-11ea-9338-29add9c31118.png)

  Also the paragraph "If you have chosen to limit..." is missed.

  ---

  With this PR when prune checkbox is toggled, the related text labels and the amount of required space shown are updated (previously they were only updated when the data directory was updated):
  ![Screenshot from 2019-12-08 11-34-53](https://user-images.githubusercontent.com/32963518/70387542-eed28780-19ae-11ea-9565-49d8a64b2f33.png)

  ---

  This PR is an alternative to #17035.

  **ryanofsky**'s [suggestion](https://github.com/bitcoin/bitcoin/pull/17035#discussion_r337594268) also has been implemented.

ACKs for top commit:
  emilengler:
    ACK 4f7127d1e3
  Sjors:
    tACK 4f7127d1e3
  ryanofsky:
    Code review ACK 4f7127d1e3. It seems like there are a few visible changes here:
  jonasschnelli:
    utACK 4f7127d1e3

Tree-SHA512: fa0bbdcfafde97d7906cda066cbd4608b936a71cae1b4cda3ee3aa2eed3a9795f279f14c6b1b4997278e094db891c7d3bb695368ba0882347aa42165a86e5172
2020-01-27 18:15:45 +01:00
Jonas Schnelli
ef8e2cee9f Merge #18007: Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open
4c524f0aad Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open (Luke Dashjr)

Pull request description:

  To reproduce bug, open 2 wallets, and close 1. You end up left without the HD/encrypt icons, despite having a wallet open still.

  This works because the icons are re-shown after we remove the current wallet (if there's another wallet still open).

ACKs for top commit:
  promag:
    Tested ACK 4c524f0aad.
  jonasschnelli:
    utACK 4c524f0aad
  hebasto:
    ACK 4c524f0aad, tested on Linux Mint 19.3.

Tree-SHA512: 4ef1bd4a0ae2f20ace9d02bc5d778640c11e46a86f30b762f8502e577f85114f0644d51a70cfbc4c23b51869c3caf20e94548aa64f51fdb85aea5f194a23fca6
2020-01-27 15:21:57 +01:00
Jonas Schnelli
3253b5dcf4 Merge #17096: gui: rename debug window
44f15cfdcf gui: renamed 'debug window' to 'node window' (Zero)

Pull request description:

  **Edit**: I have now limited the change in this PR to only renaming the window title from `Debug Window` to `Node Window`. Check [this comment](https://github.com/bitcoin/bitcoin/pull/17096#issuecomment-542837511) for more details.

  This PR is in response to #17082, which aims to rename the `Debug window` title to a more user friendly term; `Node window`.

  Closes #17082

ACKs for top commit:
  hebasto:
    ACK 44f15cfdcf, tested on Linux Mint 19.3:
  theStack:
    ACK 44f15cfdcf, tested on Linux (Lubuntu 16.04):

Tree-SHA512: 9fc73f2e67badb38525c550ce4c313288858b3fde30ef17fee85230be5bf31cf94408c699265b5e1256dfed60f8d04f48927d9b2831ba9f25498b98e6fa7180f
2020-01-27 11:31:09 +01:00
Jonas Schnelli
36ded99717 Merge #17998: gui: Shortcut to close ModalOverlay
1a638e1105 gui: Shortcut to close ModalOverlay (Emil Engler)

Pull request description:

  This adds the shortcut `Esc` to hide the ModalOverlay.
  The motivation is that it is annoying to always move the cursor to "Hide" when quickly testing something in the GUI with an outdated chain.

ACKs for top commit:
  kristapsk:
    ACK 1a638e1105. Agree with @promag, Esc feels more natural than Enter here.
  jonasschnelli:
    ACK 1a638e1105

Tree-SHA512: ea764349ec145ce9a34cbc66c3ac0eace9233a3fb3e9c22694a77882478afa22d4e686ce2c1d7b3938f6769f96ba995577b0216ba9d98954dcf3e55d2187f2e0
2020-01-27 11:14:13 +01:00
Luke Dashjr
4c524f0aad Bugfix: GUI: Hide the HD/encrypt icons earlier so they get re-shown if another wallet is open 2020-01-26 22:55:10 +00:00
fanquake
acd644b83d build: remove --large-address-aware linker flag
This flag was used when building 32-bit Windows executables, which we no-longer 
do, and is not accepted by the linker for any of the hosts we currently build 
for. i.e:

```bash
checking whether the linker accepts -Wl,--large-address-aware... no
```

--large-address-aware
    If given, the appropriate bit in the "Characteristics" field of the COFF
    header is set to indicate that this executable supports virtual addresses
    greater than 2 gigabytes. This should be used in conjunction with the /3GB
    or /USERVA=value megabytes switch in the "[operating systems]" section of
    the BOOT .INI. Otherwise, this bit has no effect. [This option is specific
    to PE targeted ports of the linker]

You can check that the appropriate bit in the COFF header of our current
Windows binaries is still be set using dumpbin. i.e:

```powershell
dumpbin /headers .\bitcoind.exe 

FILE HEADER VALUES
<snip>
26 characteristics
     Executable
     Line numbers stripped
     Application can handle large (>2GB) addresses
```
2020-01-26 10:43:10 +08:00
MarcoFalke
b4f832e24e Merge #18001: Updated appveyor job to checkout a specific vcpkg commit ID
2bcc70531a Updated appveyor job to checkout a specific vcpkg commit ID. (Aaron Clauson)

Pull request description:

  This PR sets the vcpkg packages in stone by checking out a specific commit ID whenever they need to be reinstalled. The commit ID was chosen as the most recent commit at the time of this PR.

  As per discussion on #17995 (and prior PR's/issues).

ACKs for top commit:
  fanquake:
    ACK 2bcc70531a - thanks for trying/suggesting all the different approaches, however this looks like the way to go. Should prevent `vcpkg` packages from changing out from under us.

Tree-SHA512: ced9c9c6df2287214a966d10110edda43a06380bae270a0d9ba1fd3dde48c27c109881423978e173b3e02512f6380600096b30510e90d37a6967fd8bf9186cb6
2020-01-25 20:42:30 -05:00
Aaron Clauson
2bcc70531a Updated appveyor job to checkout a specific vcpkg commit ID. 2020-01-25 19:07:03 +00:00
MarcoFalke
c26b05c2b7 Merge #17770: test: bump test timeouts so that functional tests run in valgrind
2d23082cbe bump test timeouts so that functional tests run in valgrind (Micky Yun Chan)

Pull request description:

  ci/tests: Bump timeouts so all functional tests run on travis in valgrind #17763

Top commit has no ACKs.

Tree-SHA512: 5a8c6e2ea02b715facfcb58c761577be15ae58c45a61654beb98c2c2653361196c2eec521bcae4a9a1bab8e409d6807de771ef4c46d3d05996ae47a22d499d54
2020-01-25 11:20:47 -05:00
Micky Yun Chan
2d23082cbe bump test timeouts so that functional tests run in valgrind 2020-01-25 15:51:35 +08:00
Emil Engler
1a638e1105 gui: Shortcut to close ModalOverlay 2020-01-24 20:27:49 +01:00
fanquake
28fbe68fdc Merge #17740: build: remove configure checks for win libraries we don't link against
2525c096b0 build: remove configure checks for win libraries we don't link against (fanquake)

Pull request description:

  While cross compiling, `HOST=x86_64-w64-mingw32`, none of these libs actually seem to be passed to the linker. i.e tailing a build with `make -j5 V=1 | rg -i 'mingwthrd|winspool|rpcrt4|crypt32'`.

  I'm not 100% sure about `crypt32`, even though the majority of our Windows cryptography usage, i.e [`CryptAcquireContextW`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontextw) or [`CryptGenRandom`](https://docs.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom) is provided by `advapi32`.

  Note that `rpcrt4` and `mingwthrd` are already missing from the MSVC build, so we can sync the remainder once it's clear what's actually needed. Hopefully sipsorcery can add some MSVC insight.

ACKs for top commit:
  practicalswift:
    ACK 2525c096b0 -- diff looks correct
  sipsorcery:
    ACK 2525c096b0.

Tree-SHA512: c756618f85ce2ab1e14e5514dbdc490d94c1c6dfd7a3e3d3b16344ae302fb789585dd10b5c2d784f961f3115bec1d914615051b3184bea00dfbcc3c23884ab4a
2020-01-24 15:35:57 +08:00
Andrew Chow
3f373659d7 Refactor: Replace SigningProvider pointers with unique_ptrs
Needed for future ScriptPubKeyMans which may need to create
SigningProviders dynamically and thus a normal pointer is not enough

This commit does not change behavior.
2020-01-23 16:35:08 -05:00
Andrew Chow
3afe53c403 Cleanup: Drop unused GUI learnRelatedScripts method
This commit does not change behavior.
2020-01-23 16:35:08 -05:00
Andrew Chow
e2f02aa59e Refactor: Copy CWallet signals and print function to LegacyScriptPubKeyMan
This commit does not change behavior.
2020-01-23 16:35:08 -05:00
Andrew Chow
c729afd0a3 Box the wallet: Add multiple keyman maps and loops
Add wallet logic for dealing with multiple ScriptPubKeyMan instances. This
doesn't change current behavior because there is still only a single
LegacyScriptPubKeyMan. But in the future the new logic will be used to support
descriptor wallets.
2020-01-23 16:35:08 -05:00
Andrew Chow
4977c30d59 refactor: define a UINT256_ONE global constant
Instead of having a uint256 representations of one scattered throughout
where it is used, define it globally in uint256.h
2020-01-23 16:35:08 -05:00
Andrew Chow
415afcccd3 HD Split: Avoid redundant upgrades
This avoids repeaded upgrades when support for more multiple keyman references
is added in the next commit:
https://github.com/bitcoin/bitcoin/pull/16341#discussion_r322370108
2020-01-23 16:35:08 -05:00
Andrew Chow
01b4511206 Make UpgradeKeyMetadata work only on LegacyScriptPubKeyMan 2020-01-23 16:35:08 -05:00
Andrew Chow
4a7e43e846 Store p2sh scripts in AddAndGetDestinationForScript 2020-01-23 16:35:08 -05:00
Andrew Chow
501acb5538 Always try to sign for all pubkeys in multisig 2020-01-23 16:35:08 -05:00
Andrew Chow
81610eddbc List output types in an array in order to be iterated over 2020-01-23 16:35:06 -05:00
Andrew Chow
eb81fc3ee5 Refactor: Allow LegacyScriptPubKeyMan to be null
In CWallet::LoadWallet, use this to detect and empty wallet with no keys

This commit does not change behavior.
2020-01-23 16:34:28 -05:00
Andrew Chow
fadc08ad94 Locking: Lock cs_KeyStore instead of cs_wallet in legacy keyman
This commit only affects locking behavior and doesn't have other changes.
2020-01-23 16:34:28 -05:00
João Barbosa
f5be479694 wallet: Improve CWallet:MarkDestinationsDirty 2020-01-23 16:34:28 -05:00
Gleb Naumenko
3c1bc40205 Add extra logging of asmap use and bucketing 2020-01-23 14:23:06 -05:00
Gleb Naumenko
e4658aa8ea Return mapped AS in RPC call getpeerinfo
If ASN bucketing is used, return a corresponding AS
used in bucketing for a given peer.
2020-01-23 14:22:56 -05:00
Wladimir J. van der Laan
fe3b58b959 Merge #17992: doc: Correct a small typo in help / manpage
2fa8dc5454 src/init: correct a typo (darosior)

Pull request description:

  Just a little typo I noticed while reading the manpage.

ACKs for top commit:
  emilengler:
    ACK 2fa8dc5
  practicalswift:
    ACK 2fa8dc5454 -- also small defects should be fixed
  laanwj:
    ACK 2fa8dc5454
  kristapsk:
    utACK 2fa8dc5454

Tree-SHA512: 3f25793313348f3584cd56894cc54a88bc3883c19e8e690b9624e1ab66eb2497ce270668684422c01ada2ec72f8349ec19106590739293e54fb458dbe9c561f3
2020-01-23 16:52:53 +01:00
MarcoFalke
fa5c6622c8 doc: Use proper RPC help syntax in importmulti 2020-01-23 10:23:30 -05:00
MarcoFalke
fab63111be doc: Remove duplicate "comment" from listsinceblock RPC help
Also, properly document all (json object) and (json array)
2020-01-23 10:21:00 -05:00
MarcoFalke
fa04cd6cfc doc: Properly document proxy_randomize_credentials as bool in getnetworkinfo
Also, fix indentation of some fields
2020-01-23 10:20:46 -05:00
MarcoFalke
fa9dec7c39 doc: Fix syntax error (trailing square bracket) in finalizepsbt 2020-01-23 10:20:26 -05:00
MarcoFalke
faff5a60ed doc: Fix syntax error (trailing square bracket) in walletprocesspsbt 2020-01-23 10:19:51 -05:00
MarcoFalke
fa0545901d doc: Add missing "optional" to "long" estimaterawfee RPC help
Also, use multi-line for arrays and objects
2020-01-23 10:19:47 -05:00
darosior
2fa8dc5454 src/init: correct a typo 2020-01-23 15:45:21 +01:00
MarcoFalke
aaaae4d0eb test: Add p2p test for forcerelay permission 2020-01-23 09:01:37 -05:00
MarcoFalke
fa6b57bcaa test: Fix whitespace in p2p_permissions.py 2020-01-23 09:00:20 -05:00
MarcoFalke
faf40810d7 test: Make msg_tx a witness tx 2020-01-23 08:59:51 -05:00
fanquake
2525c096b0 build: remove configure checks for win libraries we don't link against
While cross compiling, HOST=x86_64-w64-mingw32, none of these
libs actually seem to be passed to the linker.
2020-01-23 07:59:54 +08:00
Wladimir J. van der Laan
7a311fa54a Merge #17738: build: remove linking librt for backwards compatibility
f7453dcc03 build: remove linking librt for backwards compatibility (fanquake)

Pull request description:

  Now that we require glibc 2.17+, see #17538, we can remove linking librt
  for backwards compatibility purposes. The `clock_*` functions from librt
  were merged into glibc as part of the [2.17 release](https://sourceware.org/ml/libc-announce/2012/msg00001.html):

  * The `clock_*` suite of functions (declared in <time.h>) is now available
    directly in the main C library.  Previously it was necessary to link with
    -lrt to use these functions.  This change has the effect that a
    single-threaded program that uses a function such as `clock_gettime' (and
    is not linked with -lrt) will no longer implicitly load the pthreads
    library at runtime and so will not suffer the overheads associated with
    multi-thread support in other code such as the C++ runtime library.

  Note that `librt` is already unused by the RISC-V and AARCH64 binaries as their librts don't export any `clock_*` functions. As an example, you can find a diff of the arm32 vs arm64 librt symbols [here](https://gist.github.com/fanquake/b08cb1f0d14df3133395d7796ebf030c).

  Below is the library usage for the `v0.19.0.1` release (can delete these tables pre-merge).

  #### RISC-V
  ```bash
  riscv/bin/bitcoin-cli: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1']
  riscv/bin/bitcoin-qt: ['libpthread.so.0', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1', 'libatomic.so.1']
  riscv/bin/bitcoin-wallet: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1', 'libatomic.so.1']
  riscv/bin/bitcoind: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1', 'libatomic.so.1']
  riscv/bin/bitcoin-tx: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1']
  riscv/bin/test_bitcoin: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1', 'libatomic.so.1']
  ```

  #### AARCH64
  ```bash
  aarch64/bin/bitcoin-cli: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  aarch64/bin/bitcoin-qt: ['libpthread.so.0', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  aarch64/bin/bitcoin-wallet: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  aarch64/bin/bitcoind: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  aarch64/bin/bitcoin-tx: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  aarch64/bin/test_bitcoin: ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  ```

  #### ARM LINUX GNUEABIHF
  ```bash
  arm32/bin/bitcoin-cli: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  arm32/bin/bitcoin-qt: ['libpthread.so.0', 'librt.so.1', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  arm32/bin/bitcoin-wallet: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  arm32/bin/bitcoind: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  arm32/bin/bitcoin-tx: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  arm32/bin/test_bitcoin: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  ```

  #### LINUX X86_64
  ```bash
  x86_64/bin/bitcoin-cli: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  x86_64/bin/bitcoin-qt: ['libpthread.so.0', 'librt.so.1', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  x86_64/bin/bitcoin-wallet: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  x86_64/bin/bitcoind: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  x86_64/bin/bitcoin-tx: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  x86_64/bin/test_bitcoin: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  ```

  #### LINUX i686
  ```bash
  i686/bin/bitcoin-cli: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  i686/bin/bitcoin-qt: ['libpthread.so.0', 'librt.so.1', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  i686/bin/bitcoin-wallet: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  i686/bin/bitcoind: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  i686/bin/bitcoin-tx: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  i686/bin/test_bitcoin: ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  ```

ACKs for top commit:
  laanwj:
    ACK f7453dcc03

Tree-SHA512: b418260edcda88583abfa386a592ebfb977d111e8e2ba887a30bf830b0b10dba429b9cfd615fad453ff0bb824225914ccb91433064b158ae1fbb9d20fc0b9937
2020-01-22 22:24:51 +01:00
Wladimir J. van der Laan
4a07233076 Merge #17767: ci: Fix qemu issues
fa569e1a9c ci: Set LC_ALL=C when running in qemu-s390x (MarcoFalke)
fa3d77623e ci: Use debian to avoid apt install 404 errors (MarcoFalke)
fabb946090 ci: Install needed gcc and qemu-user iff cross-compiling (MarcoFalke)
faba4672b6 ci: Fix QEMU_USER_CMD parse issues (MarcoFalke)
fa5d709fb2 ci: Move wrap-qemu into separate script (MarcoFalke)

Pull request description:

  Now that the ci system no longer propagates files from the docker container back to the host, the wrap-qemu script is broken.

  To fix it, every statement in the script needs to be executed in the docker (with `DOCKER_EXEC`). Instead of juggling with triple escape sequences like `\\\"`, just move the script to a separate file and call it with `DOCKER_EXEC`.

  Also, fix a bunch of other bugs that prevent running the ci system in qemu

  See the `ci/README.md` on how to test. TLDR: Can be tested with (replace `arm` with `s390x` to run the s390x build):

  ```
  FILE_ENV="./ci/test/00_setup_env_arm.sh" MAKEJOBS="-j9" ./ci/test_run_all.sh
  ```

ACKs for top commit:
  laanwj:
    Code review ACK fa569e1a9c

Tree-SHA512: 84ebc44a4f0261ee6c29605a6896a1833ff6c81d729e6d08dd111941f570ce73221422bd3303e1108a266ec5eab2148bd5ee1cf6bc01477d8cc9a6c5bf2b34c2
2020-01-22 21:09:13 +01:00
Jonas Schnelli
16d6113f4f Refactor message transport packaging 2020-01-22 14:55:24 -05:00
Wladimir J. van der Laan
a3d198c93c Merge #17863: scripts: Add MACHO dylib checks to symbol-check.py
c491368d8c scripts: add MACHO dylib checking to symbol-check.py (fanquake)
76bf97213f scripts: fix check-symbols & check-security argument passing (fanquake)

Pull request description:

  Based on #17857.

  This adds dynamic library checks for MACHO executables to symbol-check.py. The script has been modified to function more like `security-check.py`. The error output is now also slightly different. i.e:
  ```bash
  # Linux x86
  bitcoin-cli: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
  bitcoin-cli: export of symbol vtable for std::basic_ios<char, std::char_traits<char> > not allowed
  bitcoin-cli: NEEDED library libstdc++.so.6 is not allowed
  bitcoin-cli: failed IMPORTED_SYMBOLS EXPORTED_SYMBOLS LIBRARY_DEPENDENCIES

  # RISCV (skips exported symbols checks)
  bitcoin-tx: symbol operator new[](unsigned long) from unsupported version GLIBCXX_3.4
  bitcoin-tx: NEEDED library libstdc++.so.6 is not allowed
  bitcoin-tx: failed IMPORTED_SYMBOLS LIBRARY_DEPENDENCIES

  # macOS
  Checking macOS dynamic libraries...
  libboost_filesystem.dylib is not in ALLOWED_LIBRARIES!
  bitcoind: failed DYNAMIC_LIBRARIES
  ```

  Compared to `v0.19.0.1` the macOS allowed dylibs has been slimmed down somewhat:
  ```diff
   src/qt/bitcoin-qt:
   /usr/lib/libSystem.B.dylib
  -/System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
   /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
   /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
   /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
   /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
   /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
   /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/Security.framework/Versions/A/Security
  -/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
   /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
  -/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
  -/System/Library/Frameworks/AGL.framework/Versions/A/AGL
   /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
   /usr/lib/libc++.1.dylib
  -/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
   /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
   /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
   /usr/lib/libobjc.A.dylib
  ```

ACKs for top commit:
  laanwj:
    ACK c491368d8c

Tree-SHA512: f8624e4964e80b3e0d34e8d3cc33f3107938f3ef7a01c07828f09b902b5ea31a53c50f9be03576e1896ed832cf2c399e03a7943a4f537a1e1c705f3804aed979
2020-01-22 20:33:44 +01:00
Wladimir J. van der Laan
1ae46dce60 Merge #17754: net: Don't allow resolving of std::string with embedded NUL characters. Add tests.
7a046cdc14 tests: Avoid using C-style NUL-terminated strings as arguments (practicalswift)
fefb9165f2 tests: Add tests to make sure lookup methods fail on std::string parameters with embedded NUL characters (practicalswift)
9574de86ad net: Avoid using C-style NUL-terminated strings as arguments in the netbase interface (practicalswift)

Pull request description:

  Don't allow resolving of `std::string`:s with embedded `NUL` characters.

  Avoid using C-style `NUL`-terminated strings as arguments in the `netbase` interface

  Add tests.

  The only place in where C-style `NUL`-terminated strings are actually needed is here:

  ```diff
  +    if (!ValidAsCString(name)) {
  +        return false;
  +    }
  ...
  -    int nErr = getaddrinfo(pszName, nullptr, &aiHint, &aiRes);
  +    int nErr = getaddrinfo(name.c_str(), nullptr, &aiHint, &aiRes);
       if (nErr)
           return false;
  ```

  Interface changes:

  ```diff
  -bool LookupHost(const char *pszName, std::vector<CNetAddr>& vIP, unsigned int nMaxSolutions, bool fAllowLookup);
  +bool LookupHost(const std::string& name, std::vector<CNetAddr>& vIP, unsigned int nMaxSolutions, bool fAllowLookup);

  -bool LookupHost(const char *pszName, CNetAddr& addr, bool fAllowLookup);
  +bool LookupHost(const std::string& name, CNetAddr& addr, bool fAllowLookup);

  -bool Lookup(const char *pszName, CService& addr, int portDefault, bool fAllowLookup);
  +bool Lookup(const std::string& name, CService& addr, int portDefault, bool fAllowLookup);

  -bool Lookup(const char *pszName, std::vector<CService>& vAddr, int portDefault, bool fAllowLookup, unsigned int nMaxSolutions);
  +bool Lookup(const std::string& name, std::vector<CService>& vAddr, int portDefault, bool fAllowLookup, unsigned int nMaxSolutions);

  -bool LookupSubNet(const char *pszName, CSubNet& subnet);
  +bool LookupSubNet(const std::string& strSubnet, CSubNet& subnet);

  -CService LookupNumeric(const char *pszName, int portDefault = 0);
  +CService LookupNumeric(const std::string& name, int portDefault = 0);

  -bool ConnectThroughProxy(const proxyType &proxy, const std::string& strDest, int port, const SOCKET& hSocketRet, int nTimeout, bool *outProxyConnectionFailed);
  +bool ConnectThroughProxy(const proxyType &proxy, const std::string& strDest, int port, const SOCKET& hSocketRet, int nTimeout, bool& outProxyConnectionFailed);
  ```

  It should be noted that the `ConnectThroughProxy` change (from `bool *outProxyConnectionFailed` to `bool& outProxyConnectionFailed`) has nothing to do with `NUL` handling but I thought it was worth doing when touching this file :)

ACKs for top commit:
  EthanHeilman:
    ACK 7a046cdc14
  laanwj:
    ACK 7a046cdc14

Tree-SHA512: 66556e290db996917b54091acd591df221f72230f6b9f6b167b9195ee870ebef6e26f4cda2f6f54d00e1c362e1743bf56785d0de7cae854e6bf7d26f6caccaba
2020-01-22 20:20:45 +01:00
Wladimir J. van der Laan
04f78b818f Merge #17887: bug-fix macos: give free bytes to F_PREALLOCATE
75163f4729 bug-fix macos: give free bytes to F_PREALLOCATE (Karl-Johan Alm)

Pull request description:

  The macos manpage for `fcntl` (for `F_PEOFPOSMODE`) states:

  > Allocate from the physical end of file.  In this case, fst_length indicates the number of newly allocated bytes desired.

  This would result in the rev files being essentially pre-allocating 2x their necessary size (this is the case for block files as well, but these are flushed down to their right sizes every time) as they would pre-allocate `pos + length` **free** bytes, rather than allocating `length` bytes after `pos`, as expected.

  Fixes #17827.

ACKs for top commit:
  eriknylund:
    ACK 75163f4729 built locally. All tests passing. Manual test as per my previous comment above on an older commit, using an APFS unencrypted disk image with 3 GB.
  laanwj:
    code review ACK 75163f4729

Tree-SHA512: 105c8d56c20acad8febdf0583f1e5721b63376ace325a7a62c2e4b15a442c7131404ed604c32c0cda716791d7ca5aa9f5b6a774ff86e39838bc7e87ca3c42760
2020-01-22 16:48:31 +01:00
Wladimir J. van der Laan
0038e536de Merge #17965: qt: Revert changes of pr17943
70e4706093 Revert "refactor: Remove never used default parameter" (Hennadii Stepanov)
219417b388 Revert "refactor: Simplify connection syntax" (Hennadii Stepanov)

Pull request description:

  The code, the `bool* ret = nullptr` parameter in the `BitcoinGUI::message()` slot, removed in #17943 is not dead actually. It is used in `ThreadSafeMessageBox()` function:
  a654626f07/src/qt/bitcoingui.cpp (L1363-L1368)

  Now in master (a654626f07):
  ```
  $ ./src/qt/bitcoin-qt -prune=-1
  Error: Prune cannot be configured with a negative value.
  bitcoin-qt: qt/bitcoingui.cpp:1369: bool ThreadSafeMessageBox(BitcoinGUI*, const string&, const string&, unsigned int): Assertion `invoked' failed.
  Aborted (core dumped)
  ```

  This PR reverts all commits of #17943

  Additional notes: the bug was missed due to dynamic function call `QMetaObject::invokeMethod()` which cannot be checked at compile time. See #16348 for more discussion.

  Sorry for introducing a bug.

ACKs for top commit:
  Sjors:
    ACK 70e4706093
  laanwj:
    ACK 70e4706093

Tree-SHA512: b968a026eaa4f5f39fd36ddc715d8e233f3c6420e6580f11d4ca422a5ff5d1d9d3df9ac11b353c3d4f434d67d6a69e37d2e26b8248d72bedd14ecba0a545a327
2020-01-22 16:12:17 +01:00
MarcoFalke
0a8b68cdf7 Merge #17483: build: Set gitian arch back to amd64
fae75306ba scripted-diff: Set gitian arch back to amd64 (MarcoFalke)

Pull request description:

  This was required to allow gitian builds on non-amd64 architecture, however, it seems to break the current builds (with lxc), see https://github.com/bitcoin/bitcoin/pull/17409#issuecomment-554099626

  Also, the gititan builds wouldn't be deterministic across arches anyway, see #17468

  So instead of wasting more time on this, revert the change and hope that guix allows to compile on non-amd64 architectures.

Top commit has no ACKs.

Tree-SHA512: 801e9a30ae1b0882ef45d5eb3a3cf80f3ace3b99db046069dbd95b6162119e977e3cf3134287d1ac5d09483906206acc71e1ac34d6b74dbc533d46aaf73f5cc2
2020-01-22 09:50:26 -05:00
practicalswift
2f63ffd15c tests: Add fuzzing harness for V1TransportDeserializer (P2P transport) 2020-01-22 13:08:34 +00:00
practicalswift
4a7fd7a712 tests: Add amount compression/decompression fuzzing to existing fuzzing harness: test compression round-trip 2020-01-22 13:06:52 +00:00
fanquake
a51aa2880d Merge #17897: init: Stop indexes on shutdown after ChainStateFlushed callback.
9dd58ca611 init: Stop indexes on shutdown after ChainStateFlushed callback. (Jim Posen)

Pull request description:

  Replaces https://github.com/bitcoin/bitcoin/pull/17852.

  Currently, the latest index state may not be committed to disk on shutdown. The state is committed on `ChainStateFlushed` callbacks and the current init order unregisters the indexes as validation interfaces before the final `ChainStateFlushed` callback is called on them.

  Issue identified by paulyc.

  For review: an alternative or supplemental solution would be to call `Commit` at the end of `BaseIndex::Stop`. I don't see any harm in doing so and it makes the less prone to user error. However, the destructor would have to be modified to not call `Stop` because `Commit` calls a virtual method, so I figured it wasn't worth it. But I'm curious how others feel.

ACKs for top commit:
  fjahr:
    tested ACK 9dd58ca611
  paulyc:
    > Code review ACK [9dd58ca](9dd58ca611), but failed to test because I can't reproduce the original problem.
  kallewoof:
    Tested ACK 9dd58ca611
  promag:
    Code review ACK 9dd58ca611, but failed to test because I can't reproduce the original problem.

Tree-SHA512: 2918380b699833cb7eab07456d1667dbf8ebbe2d2b5988300a3cf5b6a6cfc818b6d9086e1936ffe7881f67e409306c4b91d61a08a169cfd0a301383479d4f3cb
2020-01-22 17:42:27 +08:00
fanquake
e45463a06a Merge #17980: test: add missing #include to fix compiler errors
a5a2654bbc test: add missing #include to fix compiler errors (Karl-Johan Alm)

Pull request description:

  I believe this fixes AppVeyor errors in master. Will close if that is not the case.

  Closes #17976

ACKs for top commit:
  fanquake:
    ACK a5a2654bbc - glad the fix turned out to be this simple.

Tree-SHA512: 8fed8c2050d0f435e7ed6db1c2927d5daccc3540c6cf9e57e644d0931a740359550a5270201c893f40200960101f11cd039d807d4ed0190f1e0c674f86fd7290
2020-01-22 17:25:15 +08:00
Karl-Johan Alm
a5a2654bbc test: add missing #include to fix compiler errors 2020-01-22 17:13:53 +09:00
Pieter Wuille
4de934b9b5 Convert compression.h to new serialization framework 2020-01-21 20:29:11 -08:00
fanquake
742f84d0de Merge #17492: QT: bump fee returns PSBT on clipboard for watchonly-only wallets
3c30d7118a QT: Change bumpFee asserts to simple error message (Gregory Sanders)
e3b19d8696 QT: bump fee returns PSBT on clipboard for watchonly-only wallets (Gregory Sanders)

Pull request description:

  Very small set of changes to support PSBT-based fee bumping on watchonly wallets in QT.

  quasi-companion to https://github.com/bitcoin/bitcoin/pull/16373

ACKs for top commit:
  gwillen:
    code review ACK 3c30d71
  promag:
    Code review ACK 3c30d7118a.
  Sjors:
    utACK 3c30d71
  achow101:
    ACK 3c30d7118a

Tree-SHA512: 7a706141e46d7fd0ad513a08a96c16f2e7e531427a6776b689362f82e32cbd9d4b7eeb98f6936aa3f9347d23ccc94128516fcffa695efacd9cac43606ea916e2
2020-01-22 08:15:20 +08:00
Bushstar
e80317be5f refactor: Remove redundant conditional 2020-01-21 14:07:42 +00:00
Wladimir J. van der Laan
631df3ee87 Merge #17916: windows: Enable heap terminate-on-corruption
3d5d7aad26 windows: remove call to SetProcessDEPPolicy (fanquake)
f2645c2601 windows: Enable heap terminate-on-corruption (fanquake)

Pull request description:

  This PR is currently two separate changes:

  #### Enable heap terminate-on-corruption
  This is default behavior from Windows 8 onwards, however we still support Windows 7, so it should make sense to explicitly enable this. This is also done by projects like tor, chromium etc.

  > Enables the terminate-on-corruption feature. If the heap manager detects an error in any heap used by the process, it calls the Windows Error Reporting service and terminates the process.
  After a process enables this feature, it cannot be disabled.

  More info [here](https://docs.microsoft.com/en-us/windows/win32/api/heapapi/nf-heapapi-heapsetinformation).

  #### Remove call to SetProcessDEPPolicy()

  DEP is always enabled on 64-bit Windows processes, and `SetProcessDEPPolicy()` only works when called from a 32-bit process. I've tested that our current usage always fails ([as expected](https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setprocessdeppolicy#remarks)) with [ERROR_NOT_SUPPORTED](16151c441e/mingw-w64-headers/include/error.h (L42)).

  Please don't add a "Needs gitian build" tag here yet.

ACKs for top commit:
  sipsorcery:
    ACK 3d5d7aad26.
  laanwj:
    ACK 3d5d7aad26

Tree-SHA512: 0948bcf165685b6b573f2cd950680c34356b856690de655ced2b93d497e02e7b22aa195c99f6ce33202f182622c67302ff31c98ab51b7d050574af3debdee5ce
2020-01-20 20:55:13 +01:00
Wladimir J. van der Laan
daae6403d8 Merge #17777: tests: Add fuzzing harness for DecodeHexTx(…)
3f95fb085e build: Sort fuzzing harnesses to avoid future merge conflicts (practicalswift)
bcad0144ef tests: Add fuzzing harness for DecodeHexTx(...) (practicalswift)

Pull request description:

  Add fuzzing harness for `DecodeHexTx(…)`.

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/decode_tx
  …
  ```

ACKs for top commit:
  jonatack:
    ACK 3f95fb0

Tree-SHA512: 0f476d0cc26f1e03812664373118754042074bdab6c1e3a57c721f863feb82ca2986cceeaceb03192d893b9aa1d4ad8a5fb4c74824b9547fd8567805931a9ebd
2020-01-20 20:38:57 +01:00
Wladimir J. van der Laan
5d2ff75e20 Merge #17945: doc: Fix doxygen errors
297e098557 Fix doxygen errors (Ben Woosley)

Pull request description:

  These are all the remaining errors identified via -Werror=documentation, e.g.:
  ```
    ./rpc/rawtransaction_util.h:31:13: error: parameter 'prevTxs' not found in the function declaration [-Werror,-Wdocumentation]
      * @param  prevTxs       Array of previous txns outputs that tx depends on but may not yet be in the block chain
                ^~~~~~~
    ./rpc/rawtransaction_util.h:31:13: note: did you mean 'prevTxsUnival'?
      * @param  prevTxs       Array of previous txns outputs that tx depends on but may not yet be in the block chain
                ^~~~~~~
                prevTxsUnival

    netbase.cpp:766:11: error: parameter 'outProxyConnectionFailed[out]' not found in the function declaration [-Werror,-Wdocumentation]
     * @param outProxyConnectionFailed[out] Whether or not the connection to the
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    netbase.cpp:766:11: note: did you mean 'outProxyConnectionFailed'?
     * @param outProxyConnectionFailed[out] Whether or not the connection to the
              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
              outProxyConnectionFailed
  ```

  You can use this to run with `-Wdocumentation` yourself: #14920

ACKs for top commit:
  laanwj:
    ACK 297e098557

Tree-SHA512: a232d893b170873d923e77fa56c56a6567e7fd120b5af1f52cfeeae1093eec55621604cc80a523678f6fedc8bbb31228c4aa8dc2a630ce9ffc91525988522af7
2020-01-20 20:35:17 +01:00
Wladimir J. van der Laan
7e841f3f9b Merge #17823: scripts: Read suspicious hosts from a file instead of hardcoding
e1c582cbaa contrib: makeseeds: Read suspicious hosts from a file instead of hardcoding (Sanjay K)

Pull request description:

  referring to: https://github.com/bitcoin/bitcoin/issues/17020
  good first issue: reading SUSPICIOUS_HOSTS from a file.
  I haven't changed the base hosts that were included in the original source, just made it readable from a file.

ACKs for top commit:
  practicalswift:
    ACK e1c582cbaa -- diff looks correct

Tree-SHA512: 18684abc1c02cf52d63f6f6ecd98df01a9574a7c470524c37e152296504e2e3ffbabd6f3208214b62031512aeb809a6d37446af82c9f480ff14ce4c42c98e7c2
2020-01-20 20:24:38 +01:00
Hennadii Stepanov
70e4706093 Revert "refactor: Remove never used default parameter"
This reverts commit 7d0a8f4f53.
2020-01-19 09:38:51 +02:00
Hennadii Stepanov
219417b388 Revert "refactor: Simplify connection syntax"
This reverts commit 1a53b0da60.
2020-01-19 09:37:34 +02:00
Pieter Wuille
ca34c5cba5 Add FORMATTER_METHODS, similar to SERIALIZE_METHODS, but for formatters 2020-01-18 07:43:22 -08:00
Emil Engler
c4ea501e96 qt: Hide non PKHash-Addresses in signing address book 2020-01-18 09:00:40 +01:00
fanquake
a654626f07 Merge #17896: Serialization improvements (step 2)
9b66083788 Convert chain to new serialization (Pieter Wuille)
2f1b2f4ed0 Convert VARINT to the formatter/Using approach (Pieter Wuille)
ca62563df3 Add a generic approach for (de)serialization of objects using code in other classes (Pieter Wuille)

Pull request description:

  This is a second carve-out from #10785.

  This introduces a const-correct generic approach for serializing objects using custom serializers (defined separately from the object being serialized), then converts VARINT to use that approach, and then converts chain.h to the new framework (including the new const-correct VARINT macro).

ACKs for top commit:
  jamesob:
    ACK 9b66083788 ([`jamesob/ackr/17896.1.sipa.serialization_improvemen`](https://github.com/jamesob/bitcoin/tree/ackr/17896.1.sipa.serialization_improvemen))
  ryanofsky:
    Code review ACK 9b66083788. Only change since last review is suggested lvalue reference tweak

Tree-SHA512: 2da4af1754699cb223d6beae44c587555e39ef6951448488a04783c92e2dfd4a305934f71cc3a75d06faf6d722723d8cdbd5ccb12039783f8d62039b83987bb8
2020-01-18 08:59:00 +08:00
fanquake
0deba68064 Merge #17943: qt, refactor: Remove never used default parameter
1a53b0da60 refactor: Simplify connection syntax (Hennadii Stepanov)
7d0a8f4f53 refactor: Remove never used default parameter (Hennadii Stepanov)

Pull request description:

  In `BitcoinGUI::message()` slot the `bool* ret = nullptr` parameter is never used.

  This PR removes it and simplifies connections syntax by replacing lambdas with the `&BitcoinGUI::message` slot.

ACKs for top commit:
  promag:
    Code review ACK 1a53b0da60.
  Sjors:
    Tested ACK 1a53b0da60
  Empact:
    Code review ACK 1a53b0da60

Tree-SHA512: e287c3218d31a387338d50da3de79c27e8691829449c3a75a2f75bb1c680bd81eb9de43e4dd3646560a422d4a45c84debfce9783c4376b50aa5cde491f300688
2020-01-17 20:33:15 +08:00
fanquake
c20fbb7be8 Merge #17939: gui: Remove warning "unused variable 'wallet_model'"
c279a81e9c gui: Remove warning "unused variable 'wallet_model'" (João Barbosa)

Pull request description:

  This was part of the abandoned #15150.

ACKs for top commit:
  theStack:
    utACK c279a81e9c
  fanquake:
    ACK c279a81e9c - tested wallet loading/unloading in the qt rpc console.

Tree-SHA512: 8fbd55c7e213599c7be843b52e960a16cf965b3e01489f426ac3ed9d579d78bb4b2ac230bcccd8abe0397a8b1166ee10e0d685738441a77a5dcb5135c15790fa
2020-01-17 16:10:05 +08:00
fanquake
2ddf041a8d Merge #17928: depends: Consistent use of package variable
22c5a986e9 depends: Consistent use of package variable (Peter Bushnell)

Pull request description:

  All other mk files use the package variable consistently except for the two instances here, which have always been here, since depends was introduced in 0.10.

ACKs for top commit:
  fanquake:
    ACK 22c5a986e9 - tested a `make boost -C depends/ -j8`.

Tree-SHA512: 41766a328603db2ebb1f23ea0c5b2936de043587dd86396eaba73524d2f5bdeff25447040e33d61de2ef612a920281cd81c6fac097913270287f344beb839c5d
2020-01-17 15:02:27 +08:00
Luke Dashjr
412d5fe879 QA: feature_segwit: Check that template "rules" includes "!segwit" as appropriate 2020-01-17 05:28:48 +00:00
Luke Dashjr
2abe8cc3b7 Bugfix: Include "csv","!segwit" in "rules"
They have been missing since buried deployments were merged
2020-01-17 05:28:48 +00:00
Ben Woosley
297e098557 Fix doxygen errors
Identified via -Wdocumentation, e.g.:

  ./rpc/rawtransaction_util.h:31:13: error: parameter 'prevTxs' not found in the function declaration [-Werror,-Wdocumentation]
    * @param  prevTxs       Array of previous txns outputs that tx depends on but may not yet be in the block chain
              ^~~~~~~
  ./rpc/rawtransaction_util.h:31:13: note: did you mean 'prevTxsUnival'?
    * @param  prevTxs       Array of previous txns outputs that tx depends on but may not yet be in the block chain
              ^~~~~~~
              prevTxsUnival

  netbase.cpp:766:11: error: parameter 'outProxyConnectionFailed[out]' not found in the function declaration [-Werror,-Wdocumentation]
   * @param outProxyConnectionFailed[out] Whether or not the connection to the
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  netbase.cpp:766:11: note: did you mean 'outProxyConnectionFailed'?
   * @param outProxyConnectionFailed[out] Whether or not the connection to the
            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
            outProxyConnectionFailed
2020-01-16 18:25:11 -08:00
Samuel Dobson
7fb94c0ed4 Merge #17889: wallet: Improve CWallet:MarkDestinationsDirty
2b1641492f wallet: Improve CWallet:MarkDestinationsDirty (João Barbosa)

Pull request description:

  Improve `CWallet:MarkDestinationsDirty` by skipping transactions that already have the cache invalidated. Skipping a transaction avoids at worst case extracting all output destinations.

ACKs for top commit:
  meshcollider:
    re-utACK 2b1641492f

Tree-SHA512: 479dc2dde4b653b856e3d6a0c59a34fe33e963eb131a2d88552a8b30471b8725a087888fe5d7db6e4ee19b74072fe64441497f033be7d1931637f756e0d8fef5
2020-01-17 14:15:33 +13:00
João Barbosa
2b1641492f wallet: Improve CWallet:MarkDestinationsDirty 2020-01-17 01:12:17 +00:00
MarcoFalke
95ca6aeec7 Merge #17691: doc: Add missed copyright headers
fac86ac7b3 scripted-diff: Add missed copyright headers (Hennadii Stepanov)
6fde9d5e47 script: Update EXLUDE list in copyright_header.py (Hennadii Stepanov)
1998152f15 script: Add empty line after C++ copyright (Hennadii Stepanov)
071f2fc204 script: Add ability to insert copyright to *.sh (Hennadii Stepanov)

Pull request description:

  This PR improves `contrib/devtools/copyright_header.py` script and adds copyright headers to the files in `src` and `test` directories with two exceptions:
  - [`src/reverse_iterator.h`](https://github.com/bitcoin/bitcoin/blob/master/src/reverse_iterator.h) (added to exceptions)
  - [`src/test/fuzz/FuzzedDataProvider.h`](https://github.com/bitcoin/bitcoin/blob/master/src/test/fuzz/FuzzedDataProvider.h) (added to exceptions)

  On master 5622d8f315:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
    25 with zero copyrights
  ```

  With this PR:
  ```
  $ ./contrib/devtools/copyright_header.py report . | grep zero
     2 with zero copyrights
  ```

  ~I am uncertain about our copyright policy with `build_msvc` and `contrib` directories content, so they are out of scope of this PR.~

ACKs for top commit:
  MarcoFalke:
    ACK fac86ac7b3

Tree-SHA512: d7832c4a7a1a3b7806119775b40ec35d7982f49ff0e6199b8cee4c0e0a36e68d51728b6ee9924b1c161df4bc6105bd93391b79d42914357fa522f499cb113fa8
2020-01-16 15:58:35 -05:00
MarcoFalke
2aaeca50b2 Merge #17900: ci: Combine 32-bit build with CentOS 7 build
ef63f5fc11 ci: Combine 32-bit build with CentOS 7 build (Sebastian Falbesoner)

Pull request description:

  Combines the CentOS build with the 32-bit (i686) build to avoid Travis bottlenecks, as suggested in #17757 by MarcoFalke. This keeps most of the properties of the 32-bit build (dash as config shell, building QT5 GUI) and just builds it with depends inside the CentOS docker container.

  Making the depends in `05_before_script.sh` with unset config shell (`CONFIG_SHELL=`)

  6196e93001/ci/test/05_before_script.sh (L28)

  caused problems for building the library libevent (resulting in a Makefile with no shell set (`SHELL=`)), that's why I set it explicitely to `/bin/bash` if we have a CentOS Docker container.

  A Travis output of this 32-bit CentOS build can be seen here: https://travis-ci.org/theStack/bitcoin/jobs/634472394 (has been restarted once due to too long build time and appearance of the `CACHE_ERR_MSG`).

  For anyone wanting to verify the outputs, I found these instructions useful to reproduce a Travis build locally: https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine (steps 1-3). In this case it's a bit tricky since you run Docker inside Docker -- within the Travis Docker container, the CentOS Docker container is created. To make this possible, the Docker socket has to be exposed to the Travis container via bind-mounting (`docker run -v /var/run/docker.sock:/var/run/docker.sock ...`), as suggested in https://stackoverflow.com/a/33003273.

Top commit has no ACKs.

Tree-SHA512: af508241cec3a10a66c37673d56691717b78375340e910fcdd3fb3870741eba623a436e1e85b26b54f013375611896f5411c5a7fec2437d367d27172230129fe
2020-01-16 15:45:24 -05:00
MarcoFalke
ec9b964cc9 Merge #17541: test: add functional test for non-standard bare multisig txs
1be0b1fb2a test: add functional test for non-standard bare multisig txs (Sebastian Falbesoner)

Pull request description:

  Approaches another missing functional test of issue #17394 (counterpart to unit test in PR #17502): A transaction is rejected by the mempool with reason `"bare-multisig"` if any of the outputs' scriptPubKey has bare multisig format (`M <PubKey1> <PubKey2> ... <PubKeyN> N OP_CHECKSIG`) and bitcoind is started with the argument `-permitbaremultisig=0`.

ACKs for top commit:
  instagibbs:
    utACK 1be0b1fb2a
  kristapsk:
    ACK 1be0b1fb2a

Tree-SHA512: 2cade68c4454029b62278b38d0f137c2605a0e4450c435cdda2833667234edd4406f017ed12fa8df9730618654acbaeb68b16dcabb9f5aa84bad9f1c76c6d476
2020-01-16 15:41:34 -05:00
MarcoFalke
218274de7d Merge #17819: doc: developer notes guideline on RPCExamples addresses
42ec499489 doc: developer notes guideline on RPCExamples addresses (Jon Atack)

Pull request description:

  to make explicit the use of invalid addresses for user safety and to encourage
  the use of bech32 addresses by default. See https://github.com/bitcoin/bitcoin/pull/17578#discussion_r361752570 and https://github.com/bitcoin/bitcoin/pull/17578#discussion_r362564492.

  Fix a typo to appease the linter.

ACKs for top commit:
  promag:
    ACK 42ec499489, no strong opinion as whether this belongs to developer notes or not but why not.
  fjahr:
    ACK 42ec499
  michaelfolkson:
    ACK 42ec499489

Tree-SHA512: 64f90e227d256aa194c4fd48435440bdc233a51213dd4a6ac5b05d04263f729c6b4bb5f3afd3b87719b20cb1b159d5a9673d58a11b72823a4a6a16e8a26ae10e
2020-01-16 15:23:40 -05:00
Hennadii Stepanov
1a53b0da60 refactor: Simplify connection syntax 2020-01-16 20:44:21 +02:00
Hennadii Stepanov
7d0a8f4f53 refactor: Remove never used default parameter 2020-01-16 20:42:57 +02:00
Wladimir J. van der Laan
f018d0c9cd Merge #17924: Bug: IsUsedDestination shouldn't use key id as script id for ScriptHash
6dd59d2e49 Don't allow implementers to think ScriptHash(Witness*()) results in nesting computation (Gregory Sanders)
4b8f1e989f IsUsedDestination shouldn't use key id as script id for ScriptHash (Gregory Sanders)

Pull request description:

  Regression introduced in https://github.com/bitcoin/bitcoin/pull/17621 which causes p2sh-segwit addresses to be erroneously missed.

  Tests are only failing in 0.19 branch, likely because that release still uses p2sh-segwit addresses rather than bech32 by default.

  I'll devise a test case to catch this going forward.

ACKs for top commit:
  achow101:
    ACK 6dd59d2e49
  MarcoFalke:
    ACK 6dd59d2
  meshcollider:
    Code review ACK 6dd59d2e49

Tree-SHA512: b3e0f320c97b8c1f814cc386840240cbde2761fee9711617b713d3f75a4a5dce2dff2df573d80873df42a1f4b74e816ab8552a573fa1d62c344997fbb6af9950
2020-01-16 19:23:33 +01:00
Sebastian Falbesoner
4537ba5f21 test: add unit test for non-standard txs with too large tx size
The function IsStandardTx() returns rejection reason "tx-size" if the
transaction weight is larger than MAX_STANDARD_TX_WEIGHT (=400000 vbytes).
2020-01-16 15:10:28 +01:00
fanquake
a9c789ed96 Merge #17935: gui: hide HD & encryption icons when no wallet loaded
486f51099f gui: hide HD & encryption icons when no wallet loaded (Harris)

Pull request description:

  This PR takes care of removing (hiding) the HD wallet and encryption icons when no wallet is loaded.

  Fixes #17927

ACKs for top commit:
  Sjors:
    ACK 486f51099f
  theStack:
    ACK 486f51099f
  fanquake:
    ACK 486f51099f - tested that this fixes #17927. Thanks for following up so quick.
  emilengler:
    ACK 486f510

Tree-SHA512: 6e3e5305a9eefe1692614097c05393aa0dffd561c89cefb40d501e70a8102eafcadfbc1c86a35c0b256b0f94f41598545d7a043954d6b9669c169d31d95aaf24
2020-01-16 21:56:04 +08:00
João Barbosa
c279a81e9c gui: Remove warning "unused variable 'wallet_model'" 2020-01-16 14:27:43 +01:00
Peter Bushnell
22c5a986e9 depends: Consistent use of package variable
All other mk files use the package variable consistently except for the two instances here, which have always been here, since depends was introduced in 0.10.
2020-01-16 13:23:02 +00:00
fanquake
4e8b564df0 Merge #17931: test: Fix p2p_invalid_messages failing in Python 3.8 because of warning
f117fb00da Replace coroutine with async def in p2p_invalid_messages.py (Elichai Turkel)

Pull request description:

  In Python 3.8 `p2p_invalid_messages.py` fails because of the following warning python produce:
  ```
  2020-01-15T13:02:14.486000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_3xq0f6uh
  ./test/functional/p2p_invalid_messages.py:154: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
    asyncio.run_coroutine_threadsafe(asyncio.coroutine(swap_magic_bytes)(), NetworkThread.network_event_loop).result()
  2020-01-15T13:02:15.306000Z TestFramework (INFO): Sending a bunch of large, junk messages to test memory exhaustion. May take a bit...
  2020-01-15T13:02:17.971000Z TestFramework (INFO): Waiting for node to drop junk messages.
  2020-01-15T13:02:18.042000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:12826 due to [Errno 104] Connection reset by peer
  2020-01-15T13:02:18.141000Z TestFramework (INFO): Sending a message with incorrect size of 2
  2020-01-15T13:02:18.293000Z TestFramework (INFO): Sending a message with incorrect size of 77
  2020-01-15T13:02:18.344000Z TestFramework.mininode (WARNING): Connection lost to 127.0.0.1:12826 due to [Errno 104] Connection reset by peer
  2020-01-15T13:02:18.445000Z TestFramework (INFO): Sending a message with incorrect size of 78
  2020-01-15T13:02:18.597000Z TestFramework (INFO): Sending a message with incorrect size of 79
  2020-01-15T13:02:18.902000Z TestFramework (INFO): Stopping nodes
  2020-01-15T13:02:19.154000Z TestFramework (INFO): Cleaning up /tmp/bitcoin_func_test_3xq0f6uh on exit
  2020-01-15T13:02:19.154000Z TestFramework (INFO): Tests successful
  ```

  so as it says I replaced the co-routine with `async def` which IIUC is supported since Python 3.5, so this makes the test pass both on 3.5+ and on 3.8
  https://docs.python.org/3.5/library/asyncio-task.html ("The async def type of coroutine was added in Python 3.5, and is recommended if there is no need to support older Python versions")

ACKs for top commit:
  laanwj:
    ACK f117fb00da if it passes travis
  fanquake:
    ACK f117fb00da - observed the failure (it's the only test that fails) with Python 3.8.1, tested the fix with 3.5.6 and 3.8.1. This is our only usage of `asyncio.coroutine`.

Tree-SHA512: c21d50b23ef4d8a777fd1d9dfe433c85b0b5fff35afbd338817021ffcd42caea64b4c70e46cb3a8a543a1bf2aaa9a6b4f075f6493ab64192bc12bf8bafc54a87
2020-01-16 17:57:29 +08:00
Harris
486f51099f gui: hide HD & encryption icons when no wallet loaded 2020-01-15 23:13:16 +01:00
Gregory Sanders
3c30d7118a QT: Change bumpFee asserts to simple error message 2020-01-15 10:51:01 -05:00
Elichai Turkel
f117fb00da Replace coroutine with async def in p2p_invalid_messages.py 2020-01-15 14:16:27 +02:00
João Barbosa
9a5b5ee81f wallet: Replace %w by wallet name in -walletnotify script
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-01-15 11:47:26 +00:00
Samuel Dobson
ac61ec9da6 Merge #17843: wallet: Reset reused transactions cache
6fc554f591 wallet: Reset reused transactions cache (Fabian Jahr)

Pull request description:

  Fixes #17603 (together with #17824)

  `getbalances` is using the cache within `GetAvailableCredit` under certain conditions [here](35fff5be60/src/wallet/wallet.cpp (L1826)). For a wallet with `avoid_reuse` activated this can lead to inconsistent reporting of `used` transactions/balances between `getbalances` and `listunspent` as pointed out in #17603. When an address is reused before the first transaction is spending from this address, the cache is not updated even after the transaction is sent. This means the remaining outputs at the reused address are not showing up as `used` in `getbalances`.

  With this change, any newly incoming transaction belonging to the wallet marks all the other outputs at the same address as dirty.

ACKs for top commit:
  kallewoof:
    Code review re-ACK 6fc554f591
  promag:
    ACK 6fc554f591.
  achow101:
    Re-ACK 6fc554f591
  meshcollider:
    Code review ACK 6fc554f591

Tree-SHA512: c4cad2c752176d16d77b4a4202291d20baddf9f27250896a40274d74a6945e0f6b34be04c2f9b1b2e756d3ac669b794969df8f82a98e0b16f10e92f276649ea2
2020-01-15 22:11:33 +13:00
fanquake
002f9e9b40 Merge #17923: refactor: Use PACKAGE_NAME in GUI modal overlay and bitcoin-wallet
5855cc564f bitcoin-wallet: Use PACKAGE_NAME in usage help (Luke Dashjr)
7f5db163a4 GUI: Use PACKAGE_NAME in modal overlay (Luke Dashjr)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK 5855cc564f, checked with
  fanquake:
    ACK 5855cc564f - checked `bitcoin-wallet` and a `--disable-wallet` `bitcoin-qt`.

Tree-SHA512: 3526eb122bfdbc63349d12251f17ffa20c7f3754af4ac9c554e6d36bb14b351f31c413c30401bb3d6e0e6200b72614dfc8475489b1f742b0423bd83fba758b94
2020-01-15 14:19:42 +08:00
fanquake
af05bd9e1e Merge #17891: scripted-diff: Replace CCriticalSection with RecursiveMutex
e09c701e01 scripted-diff: Bump copyright of files changed in 2020 (MarcoFalke)
6cbe620964 scripted-diff: Replace CCriticalSection with RecursiveMutex (MarcoFalke)

Pull request description:

  `RecursiveMutex` better clarifies that the mutex is recursive, see also the standard library naming: https://en.cppreference.com/w/cpp/thread/recursive_mutex

  For that reason, and to avoid different people asking me the same question repeatedly (e.g. https://github.com/bitcoin/bitcoin/pull/15932#pullrequestreview-339175124 ), remove the outdated alias `CCriticalSection` with a scripted-diff

ACKs for top commit:
  Empact:
    ACK e09c701e01 diff and scripts look correct
  promag:
    ACK e09c701e01
  practicalswift:
    ACK e09c701e01 -- scripted diff looks correct

Tree-SHA512: 4bd7b5de1befdcf91dc8f43c127a1fee49679e06895a43216f160344a395c8e426dc68d529fbd2d5e1c215625a5a392dc415b1bce4127316aae7ecf98030c855
2020-01-15 13:46:21 +08:00
Jeremy Rubin
bd5a026928 Make UpdateTransactionsFromBlock use Epochs 2020-01-14 19:30:13 -08:00
Jeremy Rubin
2ccb7cca4a Add Epoch Guards to CTXMemPoolEntry and CTxMemPool 2020-01-14 19:30:13 -08:00
Karl-Johan Alm
75163f4729 bug-fix macos: give free bytes to F_PREALLOCATE
The macos manpage for fcntl (for F_PEOFPOSMODE) states:

> Allocate from the physical end of file.  In this case, fst_length indicates the number of newly allocated bytes desired.
2020-01-15 08:56:39 +09:00
Gregory Sanders
6dd59d2e49 Don't allow implementers to think ScriptHash(Witness*()) results in nesting computation 2020-01-14 17:10:53 -05:00
MarcoFalke
e09c701e01 scripted-diff: Bump copyright of files changed in 2020
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-01-15 02:18:00 +07:00
MarcoFalke
6cbe620964 scripted-diff: Replace CCriticalSection with RecursiveMutex
-BEGIN VERIFY SCRIPT-
 # Delete outdated alias for RecursiveMutex
 sed -i -e '/CCriticalSection/d'                 ./src/sync.h
 # Replace use of outdated alias with RecursiveMutex
 sed -i -e 's/CCriticalSection/RecursiveMutex/g' $(git grep -l CCriticalSection)
-END VERIFY SCRIPT-
2020-01-15 01:43:46 +07:00
Gregory Sanders
4b8f1e989f IsUsedDestination shouldn't use key id as script id for ScriptHash 2020-01-14 13:23:24 -05:00
Luke Dashjr
5855cc564f bitcoin-wallet: Use PACKAGE_NAME in usage help 2020-01-14 18:19:00 +00:00
Luke Dashjr
7f5db163a4 GUI: Use PACKAGE_NAME in modal overlay 2020-01-14 18:14:10 +00:00
Hennadii Stepanov
4f7127d1e3 gui: Make Intro consistent with prune checkbox
When prune checkbox is toggled, the related text labels and the amount
of required space shown are updated (previously they were only updated
when the data directory was updated).
2020-01-14 19:14:14 +02:00
Hennadii Stepanov
4824a7d36c gui: Add Intro::UpdateFreeSpaceLabel()
If a new custom datadir has enough free space, the prune checkbox gets
unchecked, unless -prune=NNN command-line option is provided.
2020-01-14 18:53:08 +02:00
Hennadii Stepanov
daa3f3fa90 refactor: Add Intro::UpdatePruneLabels()
This is a move-only commit and it does not change behavior.
2020-01-14 18:53:02 +02:00
Hennadii Stepanov
e4caa82a03 refactor: Replace static variable with data member 2020-01-14 18:47:50 +02:00
fanquake
ceb789cf3a Merge #17873: doc: Add to Doxygen documentation guidelines
c902c4c0c6 doc: Add to Doxygen documentation guidelines (Jon Layton)

Pull request description:

  Completes the up-for-grabs PR #16948.

  Changes can be tested here: [doc/developer-notes.md](https://github.com/jonatack/bitcoin/blob/doxygen-developer-notes-improvements/doc/developer-notes.md)

  Co-authored-by: Jon Layton <me@jonl.io>

ACKs for top commit:
  fanquake:
    ACK c902c4c0c6 - quick read, checked the new links work.
  laanwj:
    ACK c902c4c0c6

Tree-SHA512: 3b4cebba23061ad5243b2288c2006bf8527e74c689223825f96a44014875d15b2ab6ff54b8aa342ca657a14cf6ce3ab7d6e25bea5befd91162bc2645a74ddb7e
2020-01-14 10:02:26 +08:00
fanquake
a4a93a0bad Merge #17906: gui: Set CConnman byte counters earlier to avoid uninitialized reads
8313fa8e81 gui: Set CConnman byte counters earlier to avoid uninitialized reads (Russell Yanofsky)

Pull request description:

  Initialize CConnman byte counters during construction, so GetTotalBytesRecv() and GetTotalBytesSent() methods don't return garbage before Start() is called.

  Change shouldn't have any effect outside of the GUI. It just fixes a race condition during a qt test that was observed on travis: https://travis-ci.org/bitcoin/bitcoin/jobs/634989685

ACKs for top commit:
  MarcoFalke:
    ACK 8313fa8e81
  promag:
    ACK 8313fa8e81.

Tree-SHA512: 97c246da4e28e6e0b48f685b840f96746ad75c4b157a692201c6c4702db328a88ead8507d8e1b4e608aa1882513174ec60cf3977c31b7a9d76678cc9f49b45f8
2020-01-14 08:53:34 +08:00
Pieter Wuille
9b66083788 Convert chain to new serialization 2020-01-13 08:24:44 -08:00
Pieter Wuille
2f1b2f4ed0 Convert VARINT to the formatter/Using approach 2020-01-13 08:24:44 -08:00
Pieter Wuille
ca62563df3 Add a generic approach for (de)serialization of objects using code in other classes
This adds the (internal) Wrapper class, and the Using function that uses it. Given
a class F that implements Ser(stream, const object&) and Unser(stream, object&)
functions, this permits writing e.g. READWRITE(Using<F>(object)).
2020-01-13 08:24:44 -08:00
Fabian Jahr
6fc554f591 wallet: Reset reused transactions cache
If a destination is reused we mark the cache of the other transactions going to that destination dirty so they are not accidentally reported as trusted when the cache is hit.
2020-01-13 13:40:06 +01:00
Wladimir J. van der Laan
2ed74a43a0 Merge #16945: refactor: introduce CChainState::GetCoinsCacheSizeState
02b9511d6b tests: add tests for GetCoinsCacheSizeState (James O'Beirne)
b17e91d842 refactoring: introduce CChainState::GetCoinsCacheSizeState (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This pulls out the routine for detection of how full the coins cache is from
  FlushStateToDisk. We use this logic independently when deciding when to flush
  the coins cache during UTXO snapshot activation ([see here](231fb5f17e (diff-24efdb00bfbe56b140fb006b562cc70bR5275))).

ACKs for top commit:
  ariard:
    Code review ACK 02b9511.
  ryanofsky:
    Code review ACK 02b9511d6b. Just rebase, new COIN_SIZE comment, and new test message since last review

Tree-SHA512: 8bdd78bf68a4a5d33a776e73fcc2857f050d6d102caa4997ed19ca25468c1358e6e728199d61b423033c02e6bc8f00a1d9da52cf17a2d37d70860fca9237ea7c
2020-01-13 12:42:38 +01:00
Wladimir J. van der Laan
f2f9fdf579 Merge #17910: build: remove double LIBBITCOIN_SERVER linking
831e1220bc build: remove double LIBBITCOIN_SERVER linking (fanquake)

Pull request description:

  Seems that this is no longer required. Have tested building on macOS and Debian.

ACKs for top commit:
  promag:
    ACK 831e1220bc.
  practicalswift:
    ACK 831e1220bc
  laanwj:
    ACK 831e1220bc

Tree-SHA512: d226d9fa0292189fae7e2af14781a511c3633f1352324f19ae642e941d06c34e2abf8b1df97d2330d76dba6024a93d8d341e02cc4882d7066f97e82585631fe1
2020-01-13 11:47:50 +01:00
fanquake
3d5d7aad26 windows: remove call to SetProcessDEPPolicy
SetProcessDEPPolicy() is only supported by 32-bit Windows processes. On 64-bit
Windows it fails with ERROR_NOT_SUPPORTED (DEP is always enabled on 64-bit).

More info:
https://docs.microsoft.com/en-us/windows/win32/api/winbase/nf-winbase-setprocessdeppolicy
2020-01-13 16:40:50 +08:00
fanquake
f2645c2601 windows: Enable heap terminate-on-corruption
This is default behavior from Windows 8 onwards, however we still support
Windows 7, so it should make sense to explicitly enable this.

More info:
https://docs.microsoft.com/en-us/windows/win32/api/heapapi/nf-heapapi-heapsetinformation
2020-01-13 16:36:16 +08:00
fanquake
831e1220bc build: remove double LIBBITCOIN_SERVER linking 2020-01-12 08:49:40 +08:00
Jon Atack
d3bc184081 doc: update release notes with getaddressinfo label deprecation 2020-01-11 13:35:10 +01:00
fanquake
e258ce792a Merge #17907: doc: Fix improper Doxygen inline comments
498cdbb426 Fix improper Doxygen inline comments (Ben Woosley)

Pull request description:

  The proper syntax is `//!<`
  http://www.doxygen.nl/manual/docblocks.html#memberdoc

  Identified via `-Wdocumentation`:

  ```
  In file included from ./util/system.h:26:
  ./util/settings.h:74:41: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
      const SettingsValue* begin() const; //<! Pointer to first non-negated value.
                                          ^~~~
                                          ///<
  ./util/settings.h:75:41: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
      const SettingsValue* end() const;   //<! Pointer to end of values.
                                          ^~~~
                                          ///<
  ./util/settings.h:76:41: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
      bool empty() const;                 //<! True if there are any non-negated values.
                                          ^~~~
                                          ///<
  ./util/settings.h:77:41: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
      bool last_negated() const;          //<! True if the last value is negated.
                                          ^~~~
                                          ///<
  ./util/settings.h:78:41: error: not a Doxygen trailing comment [-Werror,-Wdocumentation]
      size_t negated() const;             //<! Number of negated values.
                                          ^~~~
                                          ///<
  ```

ACKs for top commit:
  fanquake:
    ACK 498cdbb426

Tree-SHA512: 2851fc1cbbcf700d198d82ce4923b2ef4a700f8ce19dff431ecf24f4e6fecda9fed1b4b4d148f3c1adfb6b0c6bff5d5315ee01bbcd855eb3d83e1a69b0c98893
2020-01-11 12:20:15 +08:00
Ben Woosley
498cdbb426 Fix improper Doxygen inline comments
The proper syntax is "//!<"
http://www.doxygen.nl/manual/docblocks.html#memberdoc
2020-01-10 16:32:40 -08:00
fanquake
f8c69677a1 Merge #17893: qa: Fix double-negative arg test
8b2f471a1b qa: Fix double-negative arg test (Hennadii Stepanov)

Pull request description:

  Commit 67518f7cc61bf59ddfa0fd7c8dbbdec3653b9556 tests do not catch that a pointer is returned instead of a value.

  This PR makes test to not accept trailing characters after 0.

  From [IRC](http://www.erisian.com.au/bitcoin-core-dev/log-2020-01-07.html#l-358):
  >  \<hebasto\> ryanofsky: hmm, why test/functional/feature_config_args.py passed on 67518f7cc61bf59ddfa0fd7c8dbbdec3653b9556 ?
  >  \<hebasto\> I see now: test is broken.
  >  \<ryanofsky\> test should be unaffected by that change, do you see a break somewhere?
  >  \<hebasto\> yes: "-connect=0x7fff50369968" != "-connect=0"
  > ...
  >  \<ryanofsky\> Oh I see how that would happen, it should not be a problem in the current PR.
  >  \<hebasto\> going to submit a pr to fix test
  >  \<ryanofsky\> in the commit you mentioned, value is a pointer to a string, and it was printing the pointer address instead of the string on: LogPrintf("Warning: parsed potentially confusing double-negative -%s=%s\n", key, value);
  >  \<hebasto\> correct
  >  \<ryanofsky\> oh I see, test could be fixed to more robust and not accept trailing characters after 0

ACKs for top commit:
  ryanofsky:
    Code review ACK 8b2f471a1b. I don't know how you found this but it's a nice catch! This change should make the test more reliable.

Tree-SHA512: 454b3d4415771d353a2da766f6ae6e0bfae7bdf485aaa7bfdd323595282356eeaf3f40e556b39f753bc35f578cbe9684368887eef2d63c5d7f0d7d9fa971697a
2020-01-11 07:53:33 +08:00
fanquake
62b189b91d Merge #17899: msvc: Ignore msvc linker warning and update to msvc build instructions.
0874a109da Ignore msvc linker warning and update to msvc build instructions. (Aaron Clauson)

Pull request description:

  - Update Visual Studio instructions.
  - Remove x64 platform conditional from bitcoin-qt project configuration.
  - Set use native environment toolset to fix linker warning.
  - Ignore linker warning about precompiled type information missing for test_bitcoin_qt.

ACKs for top commit:
  fanquake:
    ACK 0874a109da - tested building `bitcoind` and `bitcoin-qt`. Didn't open anything in Visual Studio.

Tree-SHA512: 83a4e4dfb8a52b024feadbf06bb1bf87993b6ebcb2a1b7dc3e2385815400f0beffc43591408b4abc8b6ffa406ce066c0af5028e7f53c707dca88ea5bba18346c
2020-01-11 07:35:00 +08:00
Russell Yanofsky
8313fa8e81 gui: Set CConnman byte counters earlier to avoid uninitialized reads
Initialize CConnman byte counters during construction, so GetTotalBytesRecv()
and GetTotalBytesSent() methods don't return garbage before Start() is called.

Change shouldn't have any effect outside of the GUI. It just fixes a race
condition during a qt test that was observed on travis:
https://travis-ci.org/bitcoin/bitcoin/jobs/634989685
2020-01-10 14:55:10 -05:00
Jim Posen
9dd58ca611 init: Stop indexes on shutdown after ChainStateFlushed callback.
Currently, the latest index state may not be committed to disk on shutdown.
2020-01-09 19:46:57 -05:00
Russell Yanofsky
96cb597325 gui: Avoid redundant tx status updates
In TransactionTablePriv::index, avoid calling
interfaces::Wallet::tryGetTxStatus if the status is up to date as of the most
recent NotifyBlockTip notification.  Store height from the most recent
notification in a new ClientModel::cachedNumBlocks variable in order to check
this.

This avoids floods of IPC traffic from tryGetTxStatus with #10102 when there
are a lot of transactions. It might also make the GUI a little more efficient
even when there is no IPC.
2020-01-09 16:05:35 -04:00
Wladimir J. van der Laan
e7f8450357 Merge #16688: log: Add validation interface logging
f9abf4ab6d Add logging for CValidationInterface events (Jeffrey Czyz)
6edebacb21 Refactor FormatStateMessage for clarity (Jeffrey Czyz)
72f3227c83 Format CValidationState properly in all cases (Jeffrey Czyz)
428ac70095 Add VALIDATION to BCLog::LogFlags (Jeffrey Czyz)

Pull request description:

  Add logging of `CValidationInterface` callbacks using a new `VALIDATIONINTERFACE` log flag (see #12994). A separate flag is desirable as the logging can be noisy and thus may need to be disabled without affecting other logging.

  This could help debug issues where there may be race conditions at play, such as #12978.

ACKs for top commit:
  jnewbery:
    ACK f9abf4ab6d
  hebasto:
    ACK f9abf4ab6d
  ariard:
    ACK f9abf4a, only changes since 0cadb12 are replacing log indication `VALIDATIONINTERFACE` by `VALIDATION` and avoiding a forward declaration with a new include
  ryanofsky:
    Code review ACK f9abf4ab6d. Just suggested changes since last review (thanks!)

Tree-SHA512: 3e0f6e2c8951cf46fbad3ff440971d95d526df2a52a2e4d6452a82785c63d53accfdabae66b0b30e2fe0b00737f8d5cb717edbad1460b63acb11a72c8f5d4236
2020-01-09 21:05:35 +01:00
Jon Atack
72af93f364 test: getaddressinfo label deprecation test 2020-01-09 18:08:24 +01:00
Jon Atack
d48875fa20 rpc: deprecate getaddressinfo label field 2020-01-09 18:08:18 +01:00
Jon Atack
dc0cabeda4 test: remove getaddressinfo label tests 2020-01-09 17:30:58 +01:00
Jon Atack
c7654af6f8 doc: address pr17578 review feedback
- https://github.com/bitcoin/bitcoin/pull/17578#discussion_r363975411
- https://github.com/bitcoin/bitcoin/pull/17578#discussion_r363969721
- https://github.com/bitcoin/bitcoin/pull/17578#discussion_r362703553
2020-01-09 17:29:49 +01:00
Aaron Clauson
0874a109da Ignore msvc linker warning and update to msvc build instructions.
- Update Visual Studio instructions.
- Remove x64 platform conditional from bitcoin-qt project configuration.
- Set use native environment toolset to fix linker warning.
- Ignore linker warning about precompiled type information missing for test_bitcoin_qt.
2020-01-09 09:19:36 +00:00
Sebastian Falbesoner
ef63f5fc11 ci: Combine 32-bit build with CentOS 7 build 2020-01-08 23:36:46 +01:00
Hennadii Stepanov
2bede28cd9 util: Add PruneGBtoMiB() function
This commit does not change behavior.
2020-01-08 17:21:40 +02:00
Hennadii Stepanov
e35e4b2ba0 util: Add PruneMiBtoGB() function
Now the text of prune QCheckBox shows the space in GB instead of
thousands MiB, which is consistent with other parts of the GUI.
2020-01-08 17:21:34 +02:00
Wladimir J. van der Laan
6196e93001 Merge #16963: wallet: Fix unique_ptr usage in boost::signals2
6d6a7a8403 gui: Fix duplicate wallet showing up (João Barbosa)
81ea66c30e Drop signal CClientUIInterface::LoadWallet (Russell Yanofsky)

Pull request description:

  This PR includes 2 fixes:
   - prevent GUI LoadWallet handlers from crashing on startup when multiple handlers are attached, because the first handler takes ownership of the wallet unique pointer. Now every handler will receive its own unique pointer;

   - prevent showing a wallet twice in the GUI on startup due to a race with `loadwallet`.

  Fixes #16937

ACKs for top commit:
  fjahr:
    code review ACK 6d6a7a8403
  ryanofsky:
    Code review ACK 6d6a7a8403. No changes since last ACK other than rebase due to #17070
  kallewoof:
    Code review ACK 6d6a7a8403

Tree-SHA512: 7f0658c9011f81dfa176a094c2263448ee1d14fda7dc94e8b55ee9c8b81538bd2d1e4bf8a8dbfcd029ebfc9feb6d3cda9dee3f911122df0a4b1e0ca75f653ba4
2020-01-08 15:58:33 +01:00
Wladimir J. van der Laan
295211e668 Merge #17445: zmq: Fix due to invalid argument and multiple notifiers
3e730bf90a zmq: Fix due to invalid argument and multiple notifiers (João Barbosa)

Pull request description:

  ZMQ initialization is interrupted if any notifier fails, and in that case all notifiers are destroyed. The notifier shutdown assumes that the initialization had occurred. This is not valid when there are multiple notifiers and any except the last fails to initialize.

  Can be tested by running test/functional/interface_zmq.py from this branch with bitcoind from master.

  Closes #17185.

ACKs for top commit:
  laanwj:
    Code review ACK 3e730bf90a, thanks for adding a test

Tree-SHA512: 5da710e97dcbaa94896d019e75162d470f6d381ee07c60e5b3e9db93d11e8f7ca9bf2c509efa4486199e88c96c3e720cc96b4e35b62725d4c7db8e8e9bf6e09d
2020-01-08 15:20:34 +01:00
fanquake
7f3675b3ce Merge #17696: qt: Force set nPruneSize in QSettings after the intro dialog
af112ab628 qt: Rename SetPrune() to InitializePruneSetting() (Hennadii Stepanov)
b0bfbe5028 refactor: Drop `bool force' parameter (Hennadii Stepanov)
68c9bbe9bc qt: Force set nPruneSize in QSettings after intro (Hennadii Stepanov)
a82bd8fa57 util: Replace magics with DEFAULT_PRUNE_TARGET_GB (Hennadii Stepanov)

Pull request description:

  On master (5622d8f315), having `QSettings` set already
  ```
  $ grep nPruneSize ~/.config/Bitcoin/Bitcoin-Qt-testnet.conf
  nPruneSize=6
  ```

  enabling prune option in the intro dialog

  ```
  $ ./src/qt/bitcoin-qt -choosedatadir -testnet
  ```
  ![DeepinScreenshot_select-area_20191208120425](https://user-images.githubusercontent.com/32963518/70388183-eed68580-19b6-11ea-9aa1-f9ad9aaa68a6.png)

  has no effect:
  ```
  $ grep Prune ~/.bitcoin/testnet3/debug.log
  2019-12-08T10:04:41Z Prune configured to target 5722 MiB on disk for block and undo files.
  ```

  ---

  With this PR:
  ```
  $ grep Prune ~/.bitcoin/testnet3/debug.log
  2019-12-08T10:20:35Z Prune configured to target 1907 MiB on disk for block and undo files.
  ```

  This PR has been split of #17453 (the first two commits) as it fixes an orthogonal bug.

  Refs:
  - https://github.com/bitcoin/bitcoin/pull/17453#discussion_r345424240
  - https://github.com/bitcoin/bitcoin/pull/17453#discussion_r350960201

ACKs for top commit:
  Sjors:
    Code review re-ACK af112ab628
  ryanofsky:
    Code review ACK af112ab628. Just suggested changes since last review (thanks!)
  promag:
    Tested ACK af112ab628. Latest suggestions and changes look good to me.

Tree-SHA512: 8ddad34b30dcc2cdcad6678ba8a0b36fa176e4e3465862ef6eee9be0f98d8146705138c9c7995dd8c0990af41078ca743fef1a90ed9240081f052f32ddec72b9
2020-01-08 22:03:49 +08:00
Wladimir J. van der Laan
40a495a38a Merge #16975: test: Show debug log on unit test failure
fa37e0a68b test: Show debug log on unit test failure (MarcoFalke)

Pull request description:

  Often, it is hard to debug unit test failures without the debug log. Especially when the failure happens remotely (e.g. on a ci system).

  Fix that by printing the log on failure.

ACKs for top commit:
  jamesob:
    ACK fa37e0a68b ([`jamesob/ackr/16975.1.MarcoFalke.test_show_debug_log_on_u`](https://github.com/jamesob/bitcoin/tree/ackr/16975.1.MarcoFalke.test_show_debug_log_on_u))

Tree-SHA512: 2ca4150c4ae3d4ad47e03b5e5e70da2baffec928ddef1fdf53a3ebc061f14aee249205387cb1b12ef6d4eb55711ef0080c0b41d9d18000b5da124ca80299793b
2020-01-08 14:57:32 +01:00
Wladimir J. van der Laan
b065df803c Merge #17880: build: add -Wdate-time to Werror flags
b0a254019c build: add Wdate-time to Werror flags (fanquake)

Pull request description:

  `-Wdate-time`
  Warn when macros __TIME__, __DATE__ or __TIMESTAMP__ are encountered as
  they might prevent bit-wise-identical reproducible compilations.

  This is supported by [GCC](https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html) and [Clang](https://clang.llvm.org/docs/DiagnosticsReference.html#wdate-time).

  Example output:
  ```bash
    CXX      bitcoind-bitcoind.o
  bitcoind.cpp:48:20: warning: expansion of date or time macro is not reproducible [-Wdate-time]
      printf("%s\n", __TIMESTAMP__);
                     ^
  bitcoind.cpp:49:20: warning: expansion of date or time macro is not reproducible [-Wdate-time]
      printf("%s\n", __TIME__);
                     ^
  bitcoind.cpp:50:20: warning: expansion of date or time macro is not reproducible [-Wdate-time]
      printf("%s\n", __DATE__);
                     ^
  3 warnings generated.
  ```

ACKs for top commit:
  practicalswift:
    ACK b0a254019c -- diff looks correct and guarding against potential non-reproducibility is good :)
  promag:
    Tested ACK b0a254019c on macos with clang. Already had `--enable-werror`, added a wild `printf("%s\n", __TIMESTAMP__)` and got the following error:
  laanwj:
    ACK b0a254019c
  hebasto:
    ACK b0a254019c

Tree-SHA512: b3a0b426e06dcd0c0baa94118c31158760b9690a8d0a15b5a2d544cb0879522e02817e134ef7346c707de09719818fc7e4bad1b3ad6b2dfe5e3c4169cdf5cb0d
2020-01-08 14:40:38 +01:00
Wladimir J. van der Laan
07efb3fe2b Merge #17826: qt: Log Qt related info
a004673c54 qt: Add LogQtInfo() function (Hennadii Stepanov)

Pull request description:

  This PR adds some info to `debug.log` I found useful for testing (e.g., on Wayland) and debugging issues like #17153:

  ```
  $ ./src/qt/bitcoin-qt -printtoconsole | head -n 6
  2020-01-04T14:57:40Z [main] Bitcoin Core version v0.19.99.0-0df287f4e (release build)
  2020-01-04T14:57:40Z [main] InitParameterInteraction: parameter interaction: -externalip set -> setting -discover=0
  2020-01-04T14:57:40Z [main] Qt 5.9.5 (dynamic), plugin=xcb (dynamic)
  2020-01-04T14:57:40Z [main] System: Linux Mint 19.3, x86_64-little_endian-lp64
  2020-01-04T14:57:40Z [main] Screen: HDMI-1 1600x1200, pixel ratio=1.0
  2020-01-04T14:57:40Z [main] Assuming ancestors of block 00000000000000b7ab6ce61eb6d571003fbe5fe892da4c9b740c49a07542462d have valid signatures.
  ```

ACKs for top commit:
  laanwj:
    ACK a004673c54

Tree-SHA512: 496bcfd4870a2730eab92b96b3e74989a7904b21369c372b6d4368f4ca2c141e2fdc1348a1fdd18cb68bb144dcea01d3023bb782f9d030e330c187f6a5a1a082
2020-01-08 14:22:45 +01:00
Wladimir J. van der Laan
6dd982aa9e Merge #17886: Restore English translation option
63bf06afc3 Restore English translation option (Andrew Chow)

Pull request description:

  It was [reported on Bitcointalk](https://bitcointalk.org/index.php?topic=5204167.msg53540137#msg53540137) that the normal English language option was lost in 0.19. This PR restores it. For some reason it was removed during the last periodic translation update.

ACKs for top commit:
  laanwj:
    ACK 63bf06afc3

Tree-SHA512: 94c7c7407f69e8df91fbbd8f8c5e3e8e031d308b72d775a00bcee564f2762a92f65c140029ce805faccdb767a25c0e222a396708c6ce29a5882bab939a45b772
2020-01-08 14:04:30 +01:00
practicalswift
7a046cdc14 tests: Avoid using C-style NUL-terminated strings as arguments 2020-01-08 12:35:59 +00:00
practicalswift
fefb9165f2 tests: Add tests to make sure lookup methods fail on std::string parameters with embedded NUL characters 2020-01-08 12:35:59 +00:00
practicalswift
9574de86ad net: Avoid using C-style NUL-terminated strings as arguments in the netbase interface 2020-01-08 12:35:59 +00:00
Hennadii Stepanov
8b2f471a1b qa: Fix double-negative arg test 2020-01-08 00:53:40 +02:00
fanquake
fcef6dbc15 Merge #17452: test: update fuzz directory in .gitignore
77ef48d532 gitignore: ignore fuzz binaries, remove test_bitcoin_fuzzy (Jon Atack)

Pull request description:

  The fuzzing gitignores haven't been updated since a4153e2 in 2016 that added an initial simple fuzzing framework.

  This commit:
  - removes `src/test/test_bitcoin_fuzzy` which is no longer used in favor of `src/test/fuzz`
  - ignores the `src/test/fuzz` directory, then un-ignores files in it with an extension, to de-clutter the git status from all the generated binary files.

  Co-authored-by: Karl-Johan Alm <karljohan-alm@garage.co.jp>

ACKs for top commit:
  practicalswift:
    ACK 77ef48d532
  MarcoFalke:
    ACK 77ef48d532

Tree-SHA512: 1fef4fbe88ded1ecf039205ace0da2abbaabcaae6ac6674eb29f84ae2f2fc560c1341e75e664cc2e243aca5767253756dba73b90ef32c7dd07c7c638abe6daf0
2020-01-08 06:47:18 +08:00
Samuel Dobson
cab3859a35 Merge #17677: Activate watchonly wallet behavior for LegacySPKM only
e1e1442f3e Activate no-privkey -> ISMINE_WATCH_ONLY behavior for LegacySPKM only (Gregory Sanders)

Pull request description:

  Slight cleanup following https://github.com/bitcoin/bitcoin/pull/16944

  This should allow future scriptpubkeymans to transparently work, since the current plan is to have ismine always be spendable.

ACKs for top commit:
  achow101:
    ACK e1e1442f3e
  Sjors:
    Code review ACK e1e1442f3e
  meshcollider:
    Code review ACK e1e1442f3e

Tree-SHA512: c0a86587d33b8b1646494a5cb0bf8681ee4a88e6913918157746943a0996b501903e0e6ee954cf04154c1e0faee0cbb375c74ca789f46ba9244eb5296632b042
2020-01-08 11:30:10 +13:00
Samuel Dobson
7ea3b85ecf Merge #17578: rpc: simplify getaddressinfo labels, deprecate previous behavior
8925df86c4 doc: update release notes (Jon Atack)
8bb405bbad test: getaddressinfo labels purpose deprecation test (Jon Atack)
60aba1f2f1 rpc: simplify getaddressinfo labels, deprecate previous behavior (Jon Atack)
7851f14ccf rpc: incorporate review feedback from PR 17283 (Jon Atack)

Pull request description:

  This PR builds on #17283 (now merged) and is followed by #17585.

  It modifies the value returned by rpc getaddressinfo `labels` to an array of label name strings and deprecates the previous behavior of returning an array of JSON hash structures containing label `name` and address `purpose` key/value pairs.

  before
  ```
    "labels": [
      {
        "name": "DOUBLE SPEND",
        "purpose": "receive"
      }
  ```
  after
  ```
    "labels": [
      "DOUBLE SPEND"
    ]
  ```

  The deprecated behavior can be re-enabled by starting bitcoind with `-deprecatedrpc=labelspurpose`.

  For context, see:
  - https://github.com/bitcoin/bitcoin/pull/17283#issuecomment-554458001
  - http://www.erisian.com.au/bitcoin-core-dev/log-2019-12-13.html#l-425 (lines 425-427)
  - http://www.erisian.com.au/bitcoin-core-dev/log-2019-11-22.html#l-622

  Reviewers: This PR may be tested manually by building, then running bitcoind with and without the `-deprecatedrpc=labelspurpose` flag while verifying the rpc getaddressinfo help text and `labels` output.

  Next steps: deprecate the rpc getaddressinfo `label` field (EDIT: done in #17585) and add support for multiple labels per address. This PR will unblock those.

ACKs for top commit:
  jnewbery:
    reACK 8925df8
  promag:
    Code review ACK 8925df86c4.
  meshcollider:
    Code review ACK 8925df86c4

Tree-SHA512: c2b717209996da32b6484de7bb8800e7048410f9ce6afdb3e02a6866bd4a8f2c730f905fca27b10b877b91cf407f546e69e8c4feb9cd934325a6c71c166bd438
2020-01-08 11:25:14 +13:00
Hennadii Stepanov
af112ab628 qt: Rename SetPrune() to InitializePruneSetting()
This function now resets the prune size and is only called after the
intro sequence.
2020-01-08 00:16:37 +02:00
Hennadii Stepanov
b0bfbe5028 refactor: Drop `bool force' parameter 2020-01-08 00:14:56 +02:00
Hennadii Stepanov
68c9bbe9bc qt: Force set nPruneSize in QSettings after intro
If QSettings is set already, it is required to force set nPruneSize
after the intro dialog.
2020-01-08 00:12:32 +02:00
Gregory Sanders
e3b19d8696 QT: bump fee returns PSBT on clipboard for watchonly-only wallets 2020-01-07 16:48:37 -05:00
Samuel Dobson
45f151913e Merge #16373: bumpfee: Return PSBT when wallet has privkeys disabled
091a876664 Test watchonly wallet bumpfee with PSBT return (Gregory Sanders)
e9b4f9419c bumpfee: Return PSBT when wallet has privkeys disabled (Gregory Sanders)
75a5e478b6 Change bumpfee to use watch-only funds for legacy watchonly wallets (Gregory Sanders)

Pull request description:

  The main use-case here is for using with watch-only wallets with PSBT-signing cold wallets of all kinds.

ACKs for top commit:
  achow101:
    ACK 091a876664
  Sjors:
    Tested ACK 091a876664
  meshcollider:
    utACK 091a876664

Tree-SHA512: f7cf663e1af0b029e5c99eac88c5fdc3bc9e9a3841da8a608e8a9957e9bcf6a78864b8c2706fcaf78a480ffe11badd80c4fad29f97c0bb929e0470fafda5c22e
2020-01-08 10:41:19 +13:00
Samuel Dobson
bcb4cdcca3 Merge #17621: IsUsedDestination should count any known single-key address
09502452bb IsUsedDestination should count any known single-key address (Gregory Sanders)

Pull request description:

  This plugs the privacy leak detailed at https://github.com/bitcoin/bitcoin/issues/17605, at least for the single-key case.

ACKs for top commit:
  meshcollider:
    Code Review ACK 09502452bb

Tree-SHA512: e1d68281675f05072b3087171cba1df9416a69c9ccf70c72e8555e55eadda2d0fd339e5a894e3a3438ff94b9e3827fb19b8b701faade70c08756b19ff157ee0c
2020-01-08 10:31:51 +13:00
Andrew Chow
63bf06afc3 Restore English translation option 2020-01-06 22:58:47 -05:00
Sanjay K
e1c582cbaa contrib: makeseeds: Read suspicious hosts from a file instead of hardcoding 2020-01-06 19:48:58 -05:00
Zero
44f15cfdcf gui: renamed 'debug window' to 'node window'
- Renamed 'Debug window' to a more user friendly term - 'Node window' - in the window title and tray menu.

fixes #17082
2020-01-06 20:29:45 +01:00
Andrew Chow
deaa6dd144 psbt: check output index is within bounds before accessing 2020-01-06 12:57:21 -05:00
Carnhof Daki
2a6bce482c doc: Add a note about backporting
See laanwj's comment in #17158
https://github.com/bitcoin/bitcoin/pull/17158#issuecomment-542627090

Co-Authored-By: Wladimir J. van der Laan <laanwj@protonmail.com>
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-Authored-By: João Barbosa <joao.paulo.barbosa@gmail.com>
Co-Authored-By: Michael <fanquake@gmail.com>
Co-Authored-By: Luke Dashjr <luke-jr+git@utopios.org>
2020-01-06 13:57:13 +01:00
fanquake
e8e79958a7 Merge #17857: scripts: fix symbol-check & security-check argument passing
71af793512 scripts: fix check-symbols & check-security argument passing (fanquake)

Pull request description:

  The first argument in `bin_PROGRAMS` (`bitcoind`) was being silently consumed and never passed into the [`security-check.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/security-check.py) or [`symbol-check.py`](https://github.com/bitcoin/bitcoin/blob/master/contrib/devtools/symbol-check.py) scripts.

  This seems to have been the case since the scripts were added to the makefile in f3d3eaf78e.

  Example of the behavior:

  ```python
  # touch a, touch b, touch c
  # python3 args.py < a b c

  import sys
  if __name__ == '__main__':
      print(sys.argv)
      # ['args.py', 'b', 'c']

      # if you add some lines to "a",
      # you'll see them here..
      for line in sys.stdin:
          print(line)
  ```

ACKs for top commit:
  laanwj:
    ACK 71af793512

Tree-SHA512: 9d0b975a11f66fd87a76654d210808000a629c9cce4c760f71e8a2bcb4e99b9109419f2306db67cf9b12c28e40b96ae722b7c9b4569b2b8bacd469fb99db30c3
2020-01-06 15:46:26 +08:00
fanquake
b0a254019c build: add Wdate-time to Werror flags
-Wdate-time

    Warn when macros __TIME__, __DATE__ or __TIMESTAMP__ are encountered as 
    they might prevent bit-wise-identical reproducible compilations.

This is supported by GCC and Clang.
https://gcc.gnu.org/onlinedocs/gcc/Warning-Options.html
https://clang.llvm.org/docs/DiagnosticsReference.html#wdate-time
2020-01-06 09:02:01 +08:00
Hennadii Stepanov
a004673c54 qt: Add LogQtInfo() function 2020-01-05 23:40:01 +02:00
Jon Layton
c902c4c0c6 doc: Add to Doxygen documentation guidelines
and update the table of contents.

Co-authored-by: Jon Layton <me@jonl.io>
2020-01-05 08:05:35 +01:00
fanquake
b949ac9697 Merge #17393: doc: Added regtest config for linearize script
582e66b6e7 doc: Added regtest config for linearize script (Gr0kchain)

Pull request description:

  Updated the example-linearize.cfg file to include support for the regtest chain network config which is used by the ./linearize-data.py

  Problem:
  Without the regtest magic, genesis hash and path config, the `linearize-data.py` script cannot generate a bootstrap.dat file.

  Example:

  ./linearize-data.py ./linearize.cfg
  Read 102 hashes
  Genesis block not found in hashlist

  Solution:

  Added netmagic, genesis and input example parameters to file.

  Resolution

  1. Starting bitcoind in regtest mode
  2. bitcoin-cli generatetoaddress 101 $(bitcoin-cli getnewaddress)
  3. ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
  4. ./linearize-data.py ./linearize.cfg

  ```
  $ ./linearize-data.py ./linearize.cfg
  Read 102 hashes
  Input file /Users/gr0kchain/.bitcoin/regtest/blocks/blk00000.dat
  Output file /Users/gr0kchain/Downloads/bootstrap.dat
  Done (102 blocks written)
  ```

ACKs for top commit:
  fanquake:
    ACK 582e66b6e7

Tree-SHA512: 699e92e740e68e2e5190ba37538efbbe3e4d4e725ebd6af704a0cf5517683b691754f7ea097bf840845d2b53b793c63258d406e9bd37922db810cf58bed053c3
2020-01-05 11:26:26 +08:00
fanquake
da279fe0ee Merge #17869: refactor: Remove unused defines in qt/bitcoinunits.h
7754d7ab1e refactor: Remove unused defines in bitcoinunits.h (Hennadii Stepanov)

Pull request description:

  In `bitcoinunits.h` some `#define`s introduced in #4167 are unused now.

ACKs for top commit:
  emilengler:
    ACK 7754d7ab1e
  fanquake:
    ACK 7754d7ab1e
  promag:
    ACK 7754d7ab1e.

Tree-SHA512: 688836a434d87530f99c309d8af60f63cdfdcfe583c9297636fbbed0f16a3dc0920d4249457303c00a83dc82d28edd8a99aab0b191c7ffbbd38c5d9fc8ee0df1
2020-01-05 11:10:29 +08:00
Hennadii Stepanov
fac86ac7b3 scripted-diff: Add missed copyright headers
-BEGIN VERIFY SCRIPT-
s() { contrib/devtools/copyright_header.py insert "$1"; }

s build_msvc/bitcoin_config.h
s build_msvc/msvc-autogen.py
s build_msvc/testconsensus/testconsensus.cpp
s contrib/devtools/circular-dependencies.py
s contrib/devtools/gen-manpages.sh
s contrib/filter-lcov.py
s contrib/gitian-build.py
s contrib/install_db4.sh
s src/crypto/sha256_avx2.cpp
s src/crypto/sha256_sse41.cpp
s src/fs.cpp
s src/qt/test/addressbooktests.cpp
s src/qt/test/addressbooktests.h
s src/qt/test/util.cpp
s src/qt/test/util.h
s src/qt/test/wallettests.cpp
s src/qt/test/wallettests.h
s src/test/blockchain_tests.cpp
s test/functional/combine_logs.py
s test/lint/lint-locale-dependence.sh
sed -i '1G' test/lint/lint-shebang.sh
s test/lint/lint-shebang.sh
-END VERIFY SCRIPT-
2020-01-04 20:18:28 +02:00
Wladimir J. van der Laan
593f5e239f Merge #17850: Serialization improvements (minimal initial commits)
9250a087d2 Convert addrdb/addrman to new serialization (Pieter Wuille)
ca33451535 Introduce new serialization macros without casts (Pieter Wuille)

Pull request description:

  This is a minimal subset of #10785 that still does *something*.

  It adds a new saner serialization macro, which can be used in parallel with the old one. Then the addrdb code is converted to use this new macro.

  I'll add follow-up PRs that add more functionality + converting of other modules as things get merged.

ACKs for top commit:
  jamesob:
    ACK 9250a087d2 ([`jamesob/ackr/17850.1.sipa.serialization_improvemen`](https://github.com/jamesob/bitcoin/tree/ackr/17850.1.sipa.serialization_improvemen))
  kallewoof:
    ACK 9250a087d2
  laanwj:
    code review ACK 9250a087d2

Tree-SHA512: d4f58c7f85d8ada7543ee43159be57d320746abe003af11395508d280d339fac7faa198e707d1a689fb0a775fc36b3945178c3ae1c0cf9ffe685773c6ddc10c1
2020-01-04 13:28:51 +01:00
Hennadii Stepanov
7754d7ab1e refactor: Remove unused defines in bitcoinunits.h 2020-01-04 13:16:38 +02:00
fanquake
392edbd49a Merge #17817: build: Add default configure cache file to .gitignore
0661a3c4a6 build: Add default configure cache to .gitignore (Hennadii Stepanov)

Pull request description:

  Ref: [Autoconf - 7.4.2 Cache Files](https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf-2.69/autoconf.html#Cache-Files)

ACKs for top commit:
  fanquake:
    ACK 0661a3c4a6 - sure; going to merge this. However lets not start adding every file that might occur from using any autoconf option to our .gitignore..

Tree-SHA512: 8be8fdd7fda35ae190c1613e5b3ac4860d6f9ec08f06b66b1278be26e11a1616ec781e0b88d0761690c99600b4de2306c01dd9798f9143531ddacb373e3fc677
2020-01-04 14:49:14 +08:00
fanquake
9a4cc52d73 Merge #17688: doc: Add "ci" prefix to CONTRIBUTING.md
19267cbc82 doc: Add ci prefix to CONTRIBUTING.md (Hennadii Stepanov)

Pull request description:

  It seems our maintainers like `ci` prefix for commits and PRs:
  ```
  git log | grep 'ci:'
  ```

  and
  ![Screenshot from 2019-12-07 11-49-51](https://user-images.githubusercontent.com/32963518/70372457-ec592a80-18e7-11ea-9320-73412a1ccd25.png)

  So let's document it.

ACKs for top commit:
  fanquake:
    ACK 19267cbc82 - this looks ok

Tree-SHA512: ef467513f9562f51d0471c5cc900336caab4e9494299bcd2c9cc9e1b296536a86467807d71b8d7100a5c78715174cf58b6ecfe6c3bd958060c15eba4fba7067f
2020-01-04 14:28:24 +08:00
fanquake
c491368d8c scripts: add MACHO dylib checking to symbol-check.py 2020-01-04 11:24:42 +08:00
fanquake
76bf97213f scripts: fix check-symbols & check-security argument passing
The first argument in bin_PROGRAMS (bitcoind) was being silently
dropped and never passed into the check-security.py or check-symbols.py scripts.

This has been the case since the scripts were added to the makefile in
f3d3eaf78e.

Example of the behavior:

```python
# touch a, touch b, touch c
# python3 args.py < a b c

import sys
if __name__ == '__main__':
    print(sys.argv)
    # ['args.py', 'b', 'c']

    # if you add some lines to "a",
    # you'll see them here..
    for line in sys.stdin:
        print(line)
```
2020-01-04 08:56:37 +08:00
Gregory Sanders
09502452bb IsUsedDestination should count any known single-key address 2020-01-03 17:20:46 -05:00
MarcoFalke
816464198c Merge #17851: tests: Add std::to_string to list of locale dependent functions
1f0adb3dac tests: Add std::to_string to list of locale dependent functions (practicalswift)

Pull request description:

  Add `std::to_string` to list of locale dependent functions:

  > `std::to_string` relies on the current locale for formatting purposes […]

  Context https://github.com/bitcoin/bitcoin/pull/17808#issuecomment-570329665

ACKs for top commit:
  hebasto:
    ACK 1f0adb3dac, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 4cd6f567f5931dd166cdb9b065a939fb0bc02c93de18a9501655d98caf18b7c4d81f1881ea900dcdf2ec103d3ab1bdc9c68d3257b76dd2468a59e74d278b0d8d
2020-01-03 15:43:17 -05:00
Jeffrey Czyz
f9abf4ab6d Add logging for CValidationInterface events
This could help debug issues where there may be race conditions at play,
such as #12978.

Fixes #12994.
2020-01-03 11:49:21 -08:00
Jeffrey Czyz
6edebacb21 Refactor FormatStateMessage for clarity
All cases of CValidationState were condensed into one strprintf call.
This is no longer suitable as more cases are added (e.g., IsValid).
2020-01-03 11:38:44 -08:00
Jeffrey Czyz
72f3227c83 Format CValidationState properly in all cases
FormatStateMessage does not properly handle the case where
CValidationState::IsValid() returns true. Use "Valid" for the state in
this case.
2020-01-03 11:37:26 -08:00
Jeffrey Czyz
428ac70095 Add VALIDATION to BCLog::LogFlags
This flag is for logging from within CValidationInterface (see #12994).
A separate flag is desirable as the logging can be noisy and thus may
need to be disabled without affecting other logging.
2020-01-03 10:59:06 -08:00
Jon Atack
8925df86c4 doc: update release notes
Update the release notes regarding the change in rpc getaddressinfo `labels`.
2020-01-03 19:51:11 +01:00
Jon Atack
8bb405bbad test: getaddressinfo labels purpose deprecation test 2020-01-03 19:51:08 +01:00
Jon Atack
60aba1f2f1 rpc: simplify getaddressinfo labels, deprecate previous behavior
- change the value returned in the RPC getaddressinfo `labels` field to an array
  of label name strings

- deprecate the previous behavior of returning a JSON hash structure containing
  label `name` and address `purpose` key/value pairs

- update the relevant tests
2020-01-03 19:46:20 +01:00
Hennadii Stepanov
6fde9d5e47 script: Update EXLUDE list in copyright_header.py 2020-01-03 17:48:37 +02:00
Hennadii Stepanov
1998152f15 script: Add empty line after C++ copyright 2020-01-03 17:48:37 +02:00
Hennadii Stepanov
071f2fc204 script: Add ability to insert copyright to *.sh 2020-01-03 17:47:54 +02:00
Jon Atack
77ef48d532 gitignore: ignore fuzz binaries, remove test_bitcoin_fuzzy
The fuzzing gitignores haven't been updated since a4153e2
in 2016 that added an initial simple fuzzing framework.

This commit:

- removes `src/test/test_bitcoin_fuzzy` which is no longer
  used in favor of `src/test/fuzz`

- ignores the src/test/fuzz directory, then un-ignores any
  files in it with an extension

Co-authored-by: Karl-Johan Alm <karljohan-alm@garage.co.jp>
2020-01-03 14:22:36 +01:00
MarcoFalke
fa569e1a9c ci: Set LC_ALL=C when running in qemu-s390x
This avoids std::runtime_error in the unit tests of the type
"locale::facet::_S_create_c_locale name not valid"
2020-01-03 07:07:11 -05:00
MarcoFalke
fa3d77623e ci: Use debian to avoid apt install 404 errors
The default ubuntu mirror does not have s390x or armhf packages:

Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Ign:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:6 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [19.2 kB]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [6781 B]
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Get:9 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [761 kB]
Ign:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Get:12 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Get:17 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [795 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Ign:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Ign:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Err:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
  404  Not Found [IP: 91.189.88.174 80]
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Ign:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1057 kB]
Ign:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [10.5 kB]
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1322 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [32.7 kB]
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Ign:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
Get:30 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4244 B]
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Get:32 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [2496 B]
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Ign:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Ign:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Ign:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Err:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
  404  Not Found [IP: 91.189.88.149 80]
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Ign:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Ign:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Err:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
  404  Not Found [IP: 91.189.88.149 80]
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Err:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
  404  Not Found [IP: 91.189.88.149 80]
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Fetched 17.4 MB in 2s (7076 kB/s)
Reading package lists...
E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/multiverse/binary-armhf/Packages  404  Not Found [IP: 91.189.88.174 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/main/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/restricted/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/main/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
2020-01-03 07:06:12 -05:00
MarcoFalke
fabb946090 ci: Install needed gcc and qemu-user iff cross-compiling 2020-01-03 07:05:39 -05:00
MarcoFalke
faba4672b6 ci: Fix QEMU_USER_CMD parse issues
* Fix bash syntax by adding missing -

* Disable QEMU_USER_CMD fallback when it is set

  Apparently bash can't distinguish an unset variable from a variable
  that is set to the empty string

* Export the environment variable to the docker env, otherwise it
  couldn't be used there
2020-01-03 07:04:45 -05:00
MarcoFalke
fa5d709fb2 ci: Move wrap-qemu into separate script 2020-01-03 07:03:46 -05:00
fanquake
71af793512 scripts: fix check-symbols & check-security argument passing
The first argument in bin_PROGRAMS (bitcoind) was being silently
dropped and never passed into the check-security.py or check-symbols.py scripts.

This has been the case since the scripts were added to the makefile in
f3d3eaf78e.

Example of the behavior:

```python
# touch a, touch b, touch c
# python3 args.py < a b c

import sys
if __name__ == '__main__':
    print(sys.argv)
    # ['args.py', 'b', 'c']

    # if you add some lines to "a",
    # you'll see them here..
    for line in sys.stdin:
        print(line)
```
2020-01-03 19:24:58 +08:00
MarcoFalke
fa37e0a68b test: Show debug log on unit test failure 2020-01-02 18:00:05 -05:00
MarcoFalke
17e14ac92f Merge #17781: rpc: Remove mempool global from miner
faa92a2297 rpc: Remove mempool global from miner (MarcoFalke)
6666ef13f1 test: Properly document blockinfo size in miner_tests (MarcoFalke)

Pull request description:

  The miner needs read-only access to the mempool. Instead of using the mutable global `::mempool`, keep a immutable reference to a mempool that is passed to the miner. Apart from the obvious benefits of removing a global and making things immutable, this might also simplify testing with multiple mempools.

ACKs for top commit:
  promag:
    ACK faa92a2297.
  fjahr:
    ACK faa92a2297
  jnewbery:
    Code review ACK faa92a2297

Tree-SHA512: c44027b5d2217a724791166f3f3112c45110ac1dbb37bdae27148a0657e0d1a1d043b0d24e49fd45465ec014224d1b7eb15c92a33069ad883fa8ffeadc24735b
2020-01-02 17:50:56 -05:00
practicalswift
1f0adb3dac tests: Add std::to_string to list of locale dependent functions 2020-01-02 20:58:36 +00:00
Wladimir J. van der Laan
190a4051fd Merge #17762: net: Log to net category for exceptions in ProcessMessages
4bdd68f301 Add missing typeinfo includes (Wladimir J. van der Laan)
4d88c3dcb6 net: Log to net category for exceptions in ProcessMessages (Wladimir J. van der Laan)

Pull request description:

  Remove the forest of special exceptions based on string matching, and simply log a short message to the NET logging category when an exception happens during packet processing. It is not good to panick end users with verbose errors (let alone writing to stderr) when any peer can generate them.

ACKs for top commit:
  MarcoFalke:
    re-ACK 4bdd68f301 (only change is adding includes) 🕕
  promag:
    ACK 4bdd68f301, could squash.

Tree-SHA512: a005591a3202b005c75e01dfa54249db3992e2f9eefa8b3d9d435acf66130417716ed926ce4e045179cf43788f1abc7362d999750681a9c80b318373d611c366
2020-01-02 21:39:00 +01:00
Wladimir J. van der Laan
4bdd68f301 Add missing typeinfo includes
The use of `typeid()` for logging exception types requires this include
according to https://en.cppreference.com/w/cpp/language/typeid.
2020-01-02 20:54:15 +01:00
Pieter Wuille
9250a087d2 Convert addrdb/addrman to new serialization 2020-01-02 11:05:36 -08:00
Pieter Wuille
ca33451535 Introduce new serialization macros without casts
This new approach uses a static method which takes the object as
a argument. This has the advantage that its constness can be a
template parameter, allowing a single implementation that sees the
object as const for serialization and non-const for deserialization,
without casts.

More boilerplate is included in the new macro as well.
2020-01-02 11:05:27 -08:00
MarcoFalke
52900a764c Merge #17849: ci: Fix brew python link
87744b16b0 ci: Fix brew python link (Hennadii Stepanov)

Pull request description:

  During the native macOS build on Travis brew-version python update from 3.7.5 to 3.7.6_1 causes link failure:

  ```
  ==> Upgrading python3
  ==> Downloading https://homebrew.bintray.com/bottles/python-3.7.6_1.mojave.bottl
  ==> Downloading from https://akamai.bintray.com/64/643d627c2b4fc03a3286c397d2992
  ######################################################################## 100.0%
  ==> Pouring python-3.7.6_1.mojave.bottle.tar.gz
  Error: The `brew link` step did not complete successfully
  The formula built, but is not symlinked into /usr/local
  ```

  Close #17848

Top commit has no ACKs.

Tree-SHA512: 09164805c557e3bd21df2d0765a1c6815e786040e9ec0e81a916b2df6c4f03974cf92c31eca999b997f8c4ed0998bdd6e35c3de7ccbaaed3bf131521ecc637dc
2020-01-02 13:51:33 -05:00
MarcoFalke
3f8dbcd655 Merge #16658: validation: Rename CheckInputs to CheckInputScripts
3bd8db80d8 [validation] fix comments in CheckInputScripts() (John Newbery)
6f6465cefc scripted-diff: [validation] Rename CheckInputs to CheckInputScripts (John Newbery)

Pull request description:

  CheckInputs() used to check no double spends, scripts & sigs and amounts. Since
  832e074, the double spend and amount checks
  have been moved to CheckTxInputs(), and CheckInputs() now just validates
  input scripts. Rename the function to CheckInputScripts().

  Also fix incorrect comments.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3bd8db80d8, did the rebase myself, checked the scripted diff 👡
  promag:
    ACK 3bd8db80d8 :trollface:

Tree-SHA512: 7b3f8597d210492798fb784ee8ea47ea6377519111190161c7cc34a967509013f4337304f52e9bedc97b7710de7b0ff8880e08cd7f867754567f82e7b02c794c
2020-01-02 11:09:00 -05:00
Hennadii Stepanov
87744b16b0 ci: Fix brew python link 2020-01-02 18:08:10 +02:00
Wladimir J. van der Laan
0655c7a94c Merge #17787: scripts: add MACHO PIE check to security-check.py
7c9e821c4e scripts: add MACHO NOUNDEFS check to security-check.py (fanquake)
4ca92dc6d3 scripts: add MACHO PIE check to security-check.py (fanquake)

Pull request description:

  This uses `otool -vh` to print the mach header and look for the `PIE` flag:
  ```bash
  otool -vh src/bitcoind
  Mach header
        magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
  MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    24       2544   NOUNDEFS DYLDLINK TWOLEVEL WEAK_DEFINES BINDS_TO_WEAK PIE
  ```

  From [`mach-o/loader.h`](https://opensource.apple.com/source/cctools/cctools-927.0.2/include/mach-o/loader.h.auto.html):
  ```c
  #define	MH_PIE 0x200000			/* When this bit is set, the OS will
  					   load the main executable at a
  					   random address.  Only used in
  					   MH_EXECUTE filetypes. */
  ```

ACKs for top commit:
  laanwj:
    code review ACK 7c9e821c4e

Tree-SHA512: 5ba2f60440d0e31c70371a355c91ca4f723d80f7287d04e2098bf5b11892cc74216ff8f1454603c4db9675d4f7983614843b992b8dcfca0309aadf2aa7ab2e4b
2020-01-02 12:48:49 +01:00
fanquake
7c9e821c4e scripts: add MACHO NOUNDEFS check to security-check.py 2020-01-02 14:42:23 +08:00
fanquake
4ca92dc6d3 scripts: add MACHO PIE check to security-check.py 2020-01-02 14:42:21 +08:00
fanquake
f7453dcc03 build: remove linking librt for backwards compatibility
Now that we require glibc 2.17+, #17538, we can remove linking in librt
for backwards compatibility purposes. The clock_* functions from librt
were merged into glibc as part of the 2.17 release.

* The `clock_*' suite of functions (declared in <time.h>) is now available
  directly in the main C library.  Previously it was necessary to link with
  -lrt to use these functions.  This change has the effect that a
  single-threaded program that uses a function such as `clock_gettime' (and
  is not linked with -lrt) will no longer implicitly load the pthreads
  library at runtime and so will not suffer the overheads associated with
  multi-thread support in other code such as the C++ runtime library.

https://sourceware.org/ml/libc-announce/2012/msg00001.html

Note that librt is already not linked by the RISC-V and AARCH64 binaries.
2019-12-31 21:58:00 +03:00
fanquake
c0bc453135 build: remove deprecated key from macOS Info.plist
Note that the current release binaries don't show incorrect version
numbers anywhere in the GUI, and we haven't reintroduced the issue that
#14701 fixed. This is just swapping a deprecated field for a newer
one and removing the additional version number.
2019-12-31 21:39:05 +03:00
fanquake
35fff5be60 Merge #17825: doc: Update dependencies.md
df97e59207 doc: Update dependencies.md (Hennadii Stepanov)

Pull request description:

  Updated info about:
  - Qt-bundled FreeType library on Android
  - Qt-bundled HarfBuzz-NG library
  - Qt dependency version (5.9.7 -> 5.9.8, #16413)

ACKs for top commit:
  fanquake:
    ACK df97e59207
  promag:
    ACK df97e59207.
  elichai:
    ~ACK df97e592078a33f6bf535979e05967c563de317b~

Tree-SHA512: 8e36520026e9e59376a1128576fbddf8313f9f7ddc7155ca462a4350ed0354d423a56127d2000b80159c677b74259eaf9348519d79a215f4fb46931bea3c2c3a
2019-12-31 21:32:26 +03:00
fanquake
99813a9745 Merge #17829: scripted-diff: Bump copyright of files changed in 2019
aaaaad6ac9 scripted-diff: Bump copyright of files changed in 2019 (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK aaaaad6ac9
  promag:
    ACK aaaaad6ac9 🎉
  fanquake:
    ACK aaaaad6ac9 - going to merge this now because the year is over and conflicts are minimal.

Tree-SHA512: 58cb1f53bc4c1395b2766f36fabc7e2332e213780a802762fff0afd59468dad0c3265f553714d761c7a2c44ff90f7dc250f04458f4b2eb8eef8b94f8c9891321
2019-12-31 20:52:26 +03:00
Gregory Sanders
27fc6a38f8 DecodeHexTx: Break out transaction decoding logic into own function 2019-12-30 13:24:23 -05:00
MarcoFalke
8830cb58ab Merge #17741: build: Included test_bitcoin-qt in msvc build
592af5ad3a Moved the include of the system projects to before the build depends on task. Otherwise it doesn't get run. (Aaron Clauson)
6e2215187e Included test_bitcoin-qt in msvc build. (Aaron Clauson)

Pull request description:

  This PR includes the `test_bitcoin-qt` project in the msvc build. The project is already in the repo but is not part of the solution and therefore does not get built.

  The test executable output from this project does not pass successfully on Windows (it may never have). This PR only builds the project and does not add a step to execute the tests.

  MarcoFalke mentioned the fact that it's missing in #17571.

Top commit has no ACKs.

Tree-SHA512: ebb8fc7710022b995e842708bbf83b9c0084dac5e1a22fa3f53d9ea472faabe96d75ad53150cc915ded5f782515a896f2b60840390bde38c1d0f80d0fd56a4ea
2019-12-31 01:09:49 +13:00
Elichai Turkel
155a11f897 doc: Added running functional tests in valgrind 2019-12-30 12:38:08 +02:00
MarcoFalke
aaaaad6ac9 scripted-diff: Bump copyright of files changed in 2019
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2019-12-30 10:42:20 +13:00
Hennadii Stepanov
df97e59207 doc: Update dependencies.md 2019-12-29 13:07:50 +02:00
Jon Atack
42ec499489 doc: developer notes guideline on RPCExamples addresses
to make explicit the use of invalid addresses for user safety and to encourage
the use of bech32 addresses by default.

Fix a typo to appease the linter.
2019-12-28 21:45:36 +01:00
MarcoFalke
b931f61b9a Merge #17801: doc: Update license year range to 2020
8dc9aa90c3 doc: Update license year range to 2020 (Emil Engler)

Pull request description:

  See #15061
  The same procedure as every year. Happy new year to all of you :)

Top commit has no ACKs.

Tree-SHA512: f2d924a739f6becc050a22cd0e37d97653ac1ef78ec645c98b5647ae2e65b9668851e24090d3ab3585503235113e1c48ea20580c35538afac5043026589bf830
2019-12-28 20:53:33 +01:00
Jon Atack
7851f14ccf rpc: incorporate review feedback from PR 17283
- (reverted after follow-on review by maintainers: provide a valid address in getaddressinfo RPCExample)
- remove unneeded code comments
2019-12-28 19:33:05 +01:00
Hennadii Stepanov
0661a3c4a6 build: Add default configure cache to .gitignore 2019-12-28 20:25:17 +02:00
MarcoFalke
6cb80a068d Merge #17806: test: Change filemode of rpc_whitelist.py
90df92206c test: Change filemode of rpc_whitelist.py (Emil Engler)

Pull request description:

  All python tests have the file mode `755`.
  Probably due to a mistake `rpc_whitelist.py` is the only test with the permission `644`.
  This PR makes it coherent with the other tests and updates it to `755` as well.

ACKs for top commit:
  practicalswift:
    ACK 90df92206c -- all tests should be executable

Tree-SHA512: b9e69cb5184a3bbee4c7b14ac35985145a9fd3403d0e449d79f15c18e9660cafec495d639f5f730e0c69dde5f4a3d7590b4e42d385e794cd02add1f4e3b785e7
2019-12-27 22:52:50 +01:00
Emil Engler
90df92206c test: Change filemode of rpc_whitelist.py 2019-12-27 21:19:44 +01:00
Emil Engler
8dc9aa90c3 doc: Update license year range to 2020 2019-12-26 23:11:21 +01:00
Gleb Naumenko
ec45646de9 Integrate ASN bucketing in Addrman and add tests
Instead of using /16 netgroups to bucket nodes in Addrman for connection
diversification, ASN, which better represents an actor in terms
of network-layer infrastructure, is used.
For testing, asmap.raw is used. It represents a minimal
asmap needed for testing purposes.
2019-12-25 08:59:08 -05:00
fanquake
1dbf3350c6 Merge #17793: ci: Update GitHub Actions CI vcpkg cache on MSBuild update
0b5a366bd7 ci: Update vcpkg cache on MSBuild update (Hennadii Stepanov)
b6fa752bc7 ci: Update Qt binaries for GitHub Actions (Hennadii Stepanov)

Pull request description:

  On master (0cda557340) Visual Studio update in GitHub Actions CI virtual environment could break a build as the `vcpkg` cache is not updated accordingly (see #17788).

  This PR:
  - force vcpkg cache update on MSBuild update
  - is an alternative to #17789
  - fixes #17788

ACKs for top commit:
  fanquake:
    ACK 0b5a366bd7

Tree-SHA512: b9e69cb5184a3bbee4c7b14ac35985145a9fd3403d0e449d79f15c18e9660cafec495d639f5f730e0c69dde5f4a3d7590b4e42d385e794cd02add1f4e3b785e7
2019-12-23 18:38:00 -05:00
Hennadii Stepanov
0b5a366bd7 ci: Update vcpkg cache on MSBuild update 2019-12-23 20:05:18 +02:00
Hennadii Stepanov
b6fa752bc7 ci: Update Qt binaries for GitHub Actions
On 2019-12-09 Visual Studio has been upgraded to 16.4.0 in Windows
Server 2019 (windows-latest) virtual environment.
2019-12-23 20:04:24 +02:00
MarcoFalke
faa92a2297 rpc: Remove mempool global from miner 2019-12-23 06:12:10 +07:00
MarcoFalke
0cda557340 Merge #17751: doc: use recommended shebang approach in documentation code block
6094222de7 use preferred shebang approach for documentation (hackerrdave)

Pull request description:

  Documentation update to use recommended shebang approach mentioned in the [developer notes](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#shebang)

ACKs for top commit:
  hebasto:
    ACK 6094222de7, I have reviewed the code, and it looks OK, I agree it can be merged.

Tree-SHA512: fc58632f0a6fa82c7abdddfac4897f082110d647426d2b468cba6fabf6b34a015fcad47e5b26be98e629b8b0417b8781e8d89da67189e20da228b97b17f1a532
2019-12-21 02:13:21 +07:00
MarcoFalke
6666ef13f1 test: Properly document blockinfo size in miner_tests
This fixes a typo in the test documentation
2019-12-20 22:24:17 +07:00
Russell Yanofsky
900d8f6f70 util: Disallow network-qualified command line options
Previously these were allowed but ignored.
2019-12-19 16:27:15 -05:00
practicalswift
3f95fb085e build: Sort fuzzing harnesses to avoid future merge conflicts 2019-12-19 20:20:05 +00:00
practicalswift
bcad0144ef tests: Add fuzzing harness for DecodeHexTx(...) 2019-12-19 20:20:05 +00:00
MarcoFalke
6677be64f6 Merge #17473: refactor: Settings code cleanups
e9fd366044 refactor: Remove null setting check in GetSetting() (Russell Yanofsky)
cba2710220 scripted-diff: Remove unused ArgsManager type flags in tests (Russell Yanofsky)
425bb30725 refactor: Add util_CheckValue test (Russell Yanofsky)
0fa54358b0 refactor: Add ArgsManager::GetSettingsList method (Russell Yanofsky)
3e185522ac refactor: Get rid of ArgsManagerHelper class (Russell Yanofsky)
dc0f148074 refactor: Replace FlagsOfKnownArg with GetArgFlags (Russell Yanofsky)
57e8b7a727 refactor: Clean up includeconf comments (Russell Yanofsky)
3f7dc9b808 refactor: Clean up long lines in settings code (Russell Yanofsky)

Pull request description:

  This PR doesn't change behavior. It just implements some suggestions from #15934 and #16545 and few other small cleanups.

ACKs for top commit:
  jnewbery:
    Code review ACK e9fd366044
  MarcoFalke:
    ACK e9fd366044 🚟

Tree-SHA512: 6e100d92c72f72bc39567187ab97a3547b3c06e5fcf1a1b74023358b8bca552124ca6a53c0ab53179b7f1329c03d9a73faaef6d73d2cd1a2321568a0286525e2
2019-12-20 03:05:28 +07:00
MarcoFalke
3e94938072 Merge #17772: doc: Mention PR Club in CONTRIBUTING.md
34d826ea5f doc: Mention PR Club in CONTRIBUTING.md (Emil Engler)

Pull request description:

  As we have a ["Review Club"](https://github.com/bitcoin/bitcoin/labels/Review%20club) label it would be worth mentioning it in the CONTRIBUTING.md file for beginners.

ACKs for top commit:
  practicalswift:
    ACK 34d826ea5f -- The Bitcoin Core PR Review Club is a great effort to introduce new contributors to the project in a positive and friendly way! We need that kind of newcomer friendly on-ramps to attract and educate the next generation of contributors! Kudos to @ jnewbery for helping secure the project also in the super long-term :)

Tree-SHA512: 9c6cbe9a82e1f13db10ad19c50a55566dbe3f5ccaf5b91f75c5a743f7f4f690515274d64c132460791bd4d8e76255b5cbfb2877908459734b52e3fbdba71fbb3
2019-12-19 23:51:47 +07:00
Gregory Sanders
6020ce3c01 DecodeHexTx: Try case where txn has inputs first 2019-12-19 11:44:41 -05:00
fanquake
7f9fbb89e8 Merge #17769: build: set AC_PREREQ to 2.69
4f4ae6f97e build: set AC_PREREQ to 2.69 (fanquake)

Pull request description:

  We use build macros such as `AX_CHECK_LINK_FLAG`, that require >=2.64, so our configure should also require Autoconf >= 2.64. The build would already blow up if 2.64 wasn't available. i.e:
  ```bash
  configure.ac:320: error: Autoconf version 2.64 or higher is required
  build-aux/m4/ax_check_link_flag.m4:74: AX_CHECK_LINK_FLAG is expanded from...
  ```
  For reference, Autoconf 2.69 was released in [April of 2012](https://lists.gnu.org/archive/html/autoconf/2012-04/msg00041.html).

  See the [Autoconf Versioning docs](https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Versioning.html) for more info on `AC_PREREQ`.

ACKs for top commit:
  hebasto:
    re-ACK 4f4ae6f97e, Autoconf 2.69 seems wide available.
  laanwj:
    ACK 4f4ae6f97e

Tree-SHA512: b77de9164ae6667513d40edaf9e16c6e7734c100643297b2dbb2ff54072774fdeab7b3b15d52979b99e204c1c4dcca4725ff155d7f6fdab7a867629130e10185
2019-12-19 11:10:28 -05:00
fanquake
4f4ae6f97e build: set AC_PREREQ to 2.69
We use build macros such as AX_CHECK_LINK_FLAG, that require >=2.64, so
our configure should also require Autoconf >= 2.64. The build would
already blow up if 2.64 wasn't available. i.e:

configure.ac:320: error: Autoconf version 2.64 or higher is required
build-aux/m4/ax_check_link_flag.m4:74: AX_CHECK_LINK_FLAG is expanded from...

For reference, Autoconf 2.69 was released in April of 2012.
https://lists.gnu.org/archive/html/autoconf/2012-04/msg00041.html

See https://www.gnu.org/software/autoconf/manual/autoconf-2.69/html_node/Versioning.html
for more info on AC_PREREQ.
2019-12-18 16:36:17 -05:00
Gleb Naumenko
8feb4e4b66 Add asmap utility which queries a mapping
The scripts for creating a compact IP->ASN mapping are here:
https://github.com/sipa/asmap

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2019-12-18 14:27:03 -05:00
Antoine Riard
f41d589669 Document better -keypool as a look-ahead safety mechanism
If after a backup, an address is issued beyond the initial
keypool range and none of the addresses in this range
is seen onchain, if a wallet is restored from backup, even in
case of rescan, funds may be loss due to the look-ahead
buffer not being incremented and so restored wallet not detecting
onchain out-of-range address as derived from its seed.

This scenario is theoretically unavoidable due to the requirement
of the keypool to have a max size. However, given the default
keypool size, this is unlikely. Document better keypool size
implications to avoid user setting a too low value.
2019-12-18 13:31:32 -05:00
Emil Engler
34d826ea5f doc: Mention PR Club in CONTRIBUTING.md 2019-12-18 17:52:46 +01:00
Gregory Sanders
091a876664 Test watchonly wallet bumpfee with PSBT return
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
2019-12-18 09:03:36 -05:00
Gregory Sanders
e9b4f9419c bumpfee: Return PSBT when wallet has privkeys disabled 2019-12-18 09:03:36 -05:00
Gregory Sanders
75a5e478b6 Change bumpfee to use watch-only funds for legacy watchonly wallets 2019-12-18 09:03:36 -05:00
fanquake
6fef85bfa3 Merge #17764: doc: Add formatting to the good first issue template
faede70882 doc: Add formatting to the good first issue template (MarcoFalke)

Pull request description:

  Add minor formatting to the good first issue template so that it is easier to see with one glance what the required skills are.

  Preview is here: https://github.com/MarcoFalke/bitcoin-core/issues/new/choose

ACKs for top commit:
  fanquake:
    ACK faede70882

Tree-SHA512: 0b0fcd051166981455061442e69f42c9fa726eaa228856e57434e012f7224781f4f3f12c31ce0a7a322df9999e79a8fbe63bf800b7933bc52c7cdaed90f37598
2019-12-17 17:02:01 -05:00
fanquake
47f45b6776 Merge #17686: build: add -bind_at_load to macOS hardened LDFLAGS
c78b123982 build: add -bind_at_load to hardened LDFLAGS (fanquake)

Pull request description:

  This performs the same function as `-Wl,-z,now`, except for ld on macOS.

  You can check the binaries using `otool -l`, and looking for the `LC_DYLD_INFO_ONLY` section; `lazy_bind_off` and `lazy_bind_size` should both be 0.

  This seems to be the case with our current release binaries. However we can make the check, and applying the flag explicit in configure.

  man ld:
  ```bash
  -bind_at_load
  Sets a bit in the mach header of the resulting binary which tells dyld
  to bind all symbols when the binary is loaded, rather than lazily.
  ```
  TODO:
  - [ ] Follow up with `MH_BINDATLOAD` flag.

ACKs for top commit:
  theuni:
    ACK c78b123982.

Tree-SHA512: 12259558b84f7e3d75d6fcde63b517685e42b18fcf8e8cfcf347483c5ba089d3b4b6d330e7b7f61f83a328fe4d141b771e8e52ddee9cac6da87dfc073ab1183d
2019-12-17 16:32:18 -05:00
hackerrdave
6094222de7 use preferred shebang approach for documentation 2019-12-17 12:18:13 -05:00
fanquake
ab4e6ad762 Merge #17756: build: remove WINDOWS_BITS from build system
abc147de95 build: remove WINDOWS_BITS from build system (fanquake)

Pull request description:

  We no longer build/ship 32 bit windows executables.

ACKs for top commit:
  laanwj:
    LGTM ACK abc147de95

Tree-SHA512: 7101393cddb7e578740e4c79532dac981eb963630ce63c28dfebf0f5ecde266c1836ac0efd1fd82e6010a6151755ad2cc2b09bc2f67edd7c0c77060ac046a9cd
2019-12-17 12:16:19 -05:00
fanquake
e6acd9f72c Merge #17537: wallet: Cleanup and move opportunistic and superfluous TopUp()s
6e77a7b65c keypool: Add comment about TopUp and when to use it (Andrew Chow)
ea50e34b28 keypool: Move opportunistic TopUps from LegacyScriptPubKeyMan to CWallet and ReserveDestination (Andrew Chow)
bb2c8ce23c keypool: Remove  superfluous topup from CWallet::GetNewChangeDestination (Andrew Chow)

Pull request description:

  * The `TopUp()` in `CWallet::GetNewChangeDestination` is unnecessary as currently m_spk_man calls TopUp further down the call stack inside LegacyScriptPubKeyMan::ReserveKeyFromKeyPool (called by LegacyScriptPubKeyMan::GetReservedDestination). This also lets us prepare for future changes with multiple ScriptPubKeyMans in the wallet.
  * An opportunistic `TopUp()` is moved from `LegacyScriptPubKeyMan::GetNewDestination` to `CWallet::GetNewDestination`.
  * Another opportunistic `TopUp()` is moved from `LegacyScriptPubKeyMan::ReserveKeyFromKeyPool`

  Moving opportunistic TopUps ensures that ScriptPubKeyMans will always be topped up before requesting Destinations from them as we cannot  always rely on future ScriptPubKeyMan implementaions topping up internally.

  See also: https://github.com/bitcoin/bitcoin/pull/17373#discussion_r348598174

ACKs for top commit:
  instagibbs:
    utACK 6e77a7b65c only change is slight elaboration on comment
  ryanofsky:
    Code review ACK 6e77a7b65c. Only the comment changed since my previous review.

Tree-SHA512: bdfc8d303842c3fb7c3d40af7abfa6d9dac4ef71a24922bb92229674ee89bfe3113ebb46d3903ac48ef99f0a7d6eaac33282495844f2b31f91b8df55084c421f
2019-12-17 12:01:18 -05:00
MarcoFalke
faede70882 doc: Add formatting to the good first issue template 2019-12-17 11:58:32 -05:00
MarcoFalke
890eac8f82 Merge #17725: ci: Add valgrind run
facb416ad5 ci: Add valgrind run (MarcoFalke)

Pull request description:

  Fixes #17460

ACKs for top commit:
  practicalswift:
    ACK facb416ad5

Tree-SHA512: 55396e548a76f976d7b7170b68bc5f93cfd44656162267172f66db7eb549699a2a22d3b1bb0d5f180fe0697931939e652c8cdb86b435e81e7ce572485798009d
2019-12-17 11:46:22 -05:00
MarcoFalke
a54c16cdb6 Merge #17661: ci: use depends for s390x
e190000869 ci-s390x: Add qemu and depends support in the ci script (Elichai Turkel)

Pull request description:

  Related: #17599

  This adds qemu support just like we have in arm and compile the depends.

  other than that I also fixed some missing includes to make the depends compile.

ACKs for top commit:
  MarcoFalke:
    ACK e190000 (first commit only, didn't look at second commit)

Tree-SHA512: 2b8a39772b86408569f52cdc33832dbce7e5e9cdd710524295f3d259628cdfc017e740f6f94941307d7f8e413236814a95ba851153c617eb5fb75b4bd9a7e52f
2019-12-17 10:46:38 -05:00
Wladimir J. van der Laan
7df6a7ea98 Merge #17758: Fix CNetAddr::IsRFC2544 comment + tests
529d332fbf test: add IsRFC2544 tests (Mark Tyneway)
419ef3b7cc CNetAddr: fix IsRFC2544 comment (Mark Tyneway)

Pull request description:

  The comment describing the functionality of `CNetAddr::IsRFC2544` is incorrect.

  46d6930f8c/src/netaddress.h (L57)

  It should actually read `198.18.0.0/15` based on [RFC 3330](https://tools.ietf.org/html/rfc3330):

  ```
     198.18.0.0/15 - This block has been allocated for use in benchmark
     tests of network interconnect devices.  Its use is documented in
     [RFC2544].
  ```

  See [RFC 2544](https://tools.ietf.org/html/rfc2544) here.

  See the implementation here:

  47d981e827/src/netaddress.cpp (L142-L145)

  This PR also adds tests for the minimum and maximum values that are valid RFC 2544 addresses.

ACKs for top commit:
  practicalswift:
    ACK 529d332fbf
  laanwj:
    ACK 529d332fbf
  promag:
    ACK 529d332fbf, nit could squash.
  jonatack:
    ACK 529d332fbf

Tree-SHA512: 954a9582856d77564e0ea5fd2e3d287d0cfc4ecfe0588115692d01005e8ca7ad8ab20ff390ded867dc91af2bfb758d4e73a336e6c0b7798846c30a6d69b8ae3d
2019-12-17 16:19:29 +01:00
Wladimir J. van der Laan
4d88c3dcb6 net: Log to net category for exceptions in ProcessMessages
Remove the forest of special exceptions, and simply log a short
message to the NET logging category when an exception happens during
packet processing. It is not good to panick end users with errors
that any peer can generate (let alone writing to stderr).
2019-12-17 15:56:49 +01:00
Mark Tyneway
529d332fbf test: add IsRFC2544 tests 2019-12-16 19:43:15 -08:00
Mark Tyneway
419ef3b7cc CNetAddr: fix IsRFC2544 comment 2019-12-16 19:42:18 -08:00
MarcoFalke
03dfa36641 Merge #17229: tests: Add fuzzing harnesses for various Base{32,58,64} and hex related functions
c18405732e tests: Add fuzzing harness for various hex related functions (practicalswift)
526dd78bed tests: Add fuzzing harness for various Base{32,58,64} related functions (practicalswift)
32e27129ff util: Move TrimString(...). Introduce default pattern (trims whitespace). Add NODISCARD. (practicalswift)
22d9bae36f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)

Pull request description:

  Add fuzzing harnesses for various Base{32,58,64} and hex related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/base_encode_decode
  …
  $ src/test/fuzz/hex
  …
  ```

ACKs for top commit:
  MarcoFalke:
    ACK c18405732e 🔁

Tree-SHA512: 4fcbe4f641fc553e43fd5c3c40a6beec0d2ce90c5ffc718213b37fc18aba4c055e51e26f93d01ea1248fd89473d07c9dce77db7f014b47d3abd045f61b5f1905
2019-12-16 19:10:58 -05:00
practicalswift
c18405732e tests: Add fuzzing harness for various hex related functions 2019-12-16 22:50:49 +00:00
practicalswift
526dd78bed tests: Add fuzzing harness for various Base{32,58,64} related functions 2019-12-16 22:50:49 +00:00
practicalswift
32e27129ff util: Move TrimString(...). Introduce default pattern (trims whitespace). Add NODISCARD. 2019-12-16 22:50:49 +00:00
practicalswift
22d9bae36f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-16 22:50:49 +00:00
MarcoFalke
94c6f2bba4 Merge #17593: test: move more utility functions into test utility library
78e283e656 [test] move wallet helper functions into test library (Martin Zumsande)
f613e5dfda [test] move mining helper functions into test library (Martin Zumsande)
2cb4e8bdc7 [test] move string helper functions into test library (Martin Zumsande)

Pull request description:

  This disbands `test/util.h` and `test/util.cpp` and moves the content into the test utility library recently created in #17542, so that all test utility functions are in one place.

  The content of the original files are split into three modules:
  1) string helper functions go to `test/util/str`
  2) mining helper functions go to the newly created `test/util/mining`
  3) wallet helper functions go to the newly created `test/util/wallet`

ACKs for top commit:
  MarcoFalke:
    ACK 78e283e656 🔧

Tree-SHA512: f182a61e86e76c32bcb84e37f44904d3a4a9c5a321f7a8efdda5368a6623cb8b5a5384ec4f96e67f0357b0c22099f6e3ecd0ac4cb467e3fa3f3128f8d36edfb8
2019-12-16 16:08:56 -05:00
MarcoFalke
f9fd3a27fd Merge #17750: util: change GetWarnings parameter to bool
7aab8d1024 [style] Code style fixups in GetWarnings() (John Newbery)
492c6dc1e7 util: change GetWarnings parameter to bool (John Newbery)
869b6314fd [qt] remove unused parameter from getWarnings() (John Newbery)

Pull request description:

  `GetWarnings()` changes the format of the output warning string based on a passed-in string argument that can be set to "gui" or "statusbar".

  Change the argument to a bool:

  - there are only two types of behaviour, so a bool is a more natural argument type
  - changing the name to `verbose` does not set any expectations for the how the calling code will use the returned string (currently, `statusbar` is used for RPC warnings, not a status bar)
  - removes some error-handling code for when the passed-in string is not one of the two strings expected.

ACKs for top commit:
  laanwj:
    code review ACK 7aab8d1024
  practicalswift:
    ACK 7aab8d1024 -- diff looks correct :)
  MarcoFalke:
    ACK 7aab8d1024 otherwise.
  promag:
    Code review ACK 7aab8d1024.

Tree-SHA512: 75882c6e3e44aa9586411b803149b36ba487f4eb9cac3f5c8f07cd9f586870bba4488a51e674cf8147f05718534f482836e6a4e3f66e0d4ef6821900c7dfd04e
2019-12-16 16:07:20 -05:00
MarcoFalke
48d64d73c0 Merge #17564: rpc: Use mempool from node context instead of global
fa8e650b52 rest: Use mempool from node context instead of global (MarcoFalke)
fa660d65d7 node: Use mempool from node context instead of global (MarcoFalke)
facbaf092f rpc: Use mempool from node context instead of global (MarcoFalke)

Pull request description:

  Currently they are identical, but in the future we might want to turn
  the mempool into a unique_ptr. Replacing the global with the mempool
  pointer from the node context simplifies this step.

ACKs for top commit:
  jnewbery:
    Code review ACK fa8e650b5
  ryanofsky:
    Code review ACK fa8e650b52, Only the discussed REST server changes since the last review.

Tree-SHA512: 0836f3f39cf90306455962918446e5f8612e88c32072b92afc30929aea1f17430bbda0e2b3668d36c9d6b97d63a93cf4903185194571108642b7bf5a39b89125
2019-12-16 16:05:06 -05:00
fanquake
abc147de95 build: remove WINDOWS_BITS from build system
We no longer build/ship 32 bit windows executables.
2019-12-16 13:12:29 -05:00
fanquake
988eaf2fcb Merge #17752: doc: fix directory path for secp256k1 subtree in developer-notes
a5089f62bd fix directory path for secp256k1 subtree in developer-notes (hackerrdave)

Pull request description:

  Documentation update to fix the directory path of the `secp256k1` subtree in the developer notes

ACKs for top commit:
  laanwj:
    ACK a5089f62bd

Tree-SHA512: d0986721d7091af26edaee769db78c9aabac25bbaddb2a1bfa96c7208187226e280e9c38897b5227ee6c9e40d5a1af86bb7c58e72c6a30a94a478c4bf54c086e
2019-12-16 11:00:03 -05:00
MarcoFalke
fa8e650b52 rest: Use mempool from node context instead of global 2019-12-16 10:45:22 -05:00
MarcoFalke
806a2c602c Merge #17071: tests: Add fuzzing harness for CheckBlock(...) and other CBlock related functions
893aa207e8 tests: Add fuzzing harness for CheckBlock(...) and other CBlock related functions (practicalswift)
ec8dcb0199 tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)

Pull request description:

  Add fuzzing harness for `CheckBlock(...)` and other `CBlock` related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/block
  …
  # And to to quickly verify that the relevant code regions are triggered, that the
  # fuzzing throughput seems reasonable, etc.
  $ contrib/devtools/test_fuzzing_harnesses.sh '^block$'
  ```

  `test_fuzzing_harnesses.sh` can be found in PR #17000.

Top commit has no ACKs.

Tree-SHA512: 275abd46d8ac970b28d8176f59124988b1e07c070173e001acd55995b830333417f301c309199fc589da08a6ac4c03aa74650d5e1638f6e3023dfbd3c9f6921d
2019-12-16 10:23:22 -05:00
MarcoFalke
6b51cce65a Merge #17753: util: Don't allow Base32/64-decoding or ParseMoney(…) on strings with embedded NUL characters. Add tests.
137c80d579 tests: Add tests for decoding/parsing of base32, base64 and money strings containing NUL characters (practicalswift)
a6fc26da55 util: Don't allow DecodeBase32(...) of strings with embedded NUL characters (practicalswift)
93cc18b0f6 util: Don't allow DecodeBase64(...) of strings with embedded NUL characters (practicalswift)
ccc53e43c5 util: Don't allow ParseMoney(...) of strings with embedded NUL characters (practicalswift)

Pull request description:

  Don't allow Base32/64-decoding or `ParseMoney(…)` on strings with embedded `NUL` characters. Add tests.

  Added tests before:

  ```
  $ src/test/test_bitcoin
  Running 385 test cases...
  test/base32_tests.cpp(31): error: in "base32_tests/base32_testvectors":
      check failure == true has failed [false != true]
  test/base64_tests.cpp(31): error: in "base64_tests/base64_testvectors":
      check failure == true has failed [false != true]
  test/util_tests.cpp(1074): error: in "util_tests/util_ParseMoney":
      check !ParseMoney(std::string("\0-1", 3), ret) has failed
  test/util_tests.cpp(1076): error: in "util_tests/util_ParseMoney":
      check !ParseMoney(std::string("1\0", 2), ret) has failed

  *** 4 failures are detected in the test module "Bitcoin Core Test Suite"
  ```

  Added tests after:

  ```
  $ src/test/test_bitcoin
  Running 385 test cases...

  *** No errors detected
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 137c80d579

Tree-SHA512: 9486a0d32b4cf686bf5a47a0778338ac571fa39c66ad6d6d6cede58ec798e87bb50a2f9b7fd79ecd1fef1ba284e4073c1b430110967073ff87bdbbde7cada447
2019-12-16 10:17:17 -05:00
Wladimir J. van der Laan
5e4912f990 Merge #17730: depends: remove Qt networking features
244501fc85 depends: disable unused qt networking features (fanquake)
29d56c62b7 depends: -optimized-qmake is now -optimized-tools (fanquake)
ccdda96804 depends: skip building qt proxies (fanquake)

Pull request description:

  Somewhat of a followup to removing BIP70 support in #17165. This removes networking features from our Qt build. This also removes the need to link against the `CFNetwork` and `SystemConfiguration` libraries on macOS.

  ```diff
  src/qt/bitcoin-qt:
   /usr/lib/libSystem.B.dylib
   /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
   /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
   /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
   /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
   /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
   /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
   /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
   /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
   /usr/lib/libc++.1.dylib
  -/System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
   /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
   /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
   /usr/lib/libobjc.A.dylib
  ```

  > Introduced the -optimized-tools option; supersedes -optimized-qmake.

  `optimized-qmake` became `optimized-tools` in Qt 5.6.0. While the former still works, we can use the newer flag.

  A diff of the removed symbols is available [here](https://gist.github.com/fanquake/9c8d5961c91f90a2966191367adfb391).

  We still need to actually build the network module, because we are using `QLocalServer` & `QLocalSocket` in the payment server.

ACKs for top commit:
  Sjors:
    Code review ACK 244501fc85: just a rebase (_updated since I accidentally repeated the previous hash_)
  practicalswift:
    ACK 244501fc85 -- diff looks correct
  promag:
    Code review ACK 244501fc85.

Tree-SHA512: 79734e3c96c40e7e484c86ac4cd4f738c05fcebe4771aeac443883f618a6c766e667909d5f8f14f9bd82f43206387c952458c5fa765cd0830f8beda6e6ac80ae
2019-12-16 13:28:57 +01:00
Wladimir J. van der Laan
c3628e4483 Merge #17743: doc: Add release note for RPC Whitelist
7965e0b41a doc: Add release note for RPC Whitelist (Emil Engler)

Pull request description:

  A release note for #12763

ACKs for top commit:
  laanwj:
    ACK 7965e0b41a

Tree-SHA512: 4ac3e62029a403e64e4cd3183433dc7aa071d42688b689d7cffb8f08dc4b26d2a586d32fa791d2b5679d6b95cd6e34c56e40a5592b9af446ad9429307f7267fe
2019-12-16 11:02:09 +01:00
practicalswift
137c80d579 tests: Add tests for decoding/parsing of base32, base64 and money strings containing NUL characters 2019-12-16 09:23:19 +00:00
practicalswift
a6fc26da55 util: Don't allow DecodeBase32(...) of strings with embedded NUL characters 2019-12-16 09:04:36 +00:00
practicalswift
93cc18b0f6 util: Don't allow DecodeBase64(...) of strings with embedded NUL characters 2019-12-16 09:04:36 +00:00
practicalswift
ccc53e43c5 util: Don't allow ParseMoney(...) of strings with embedded NUL characters 2019-12-16 09:04:35 +00:00
hackerrdave
a5089f62bd fix directory path for secp256k1 subtree in developer-notes 2019-12-15 19:58:51 -05:00
practicalswift
893aa207e8 tests: Add fuzzing harness for CheckBlock(...) and other CBlock related functions 2019-12-15 21:38:34 +00:00
practicalswift
ec8dcb0199 tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-15 21:27:38 +00:00
John Newbery
7aab8d1024 [style] Code style fixups in GetWarnings() 2019-12-15 17:05:52 -03:00
Emil Engler
7965e0b41a doc: Add release note for RPC Whitelist 2019-12-15 20:49:49 +01:00
Hennadii Stepanov
19267cbc82 doc: Add ci prefix to CONTRIBUTING.md 2019-12-15 21:28:56 +02:00
John Newbery
492c6dc1e7 util: change GetWarnings parameter to bool
GetWarnings() changes the format of the output warning string based on a
passed-in string argument that can be set to "gui" or "statusbar".
Change the argument to a bool:

- there are only two types of behaviour, so a bool is a more natural
argument type
- changing the name to 'verbose' does not set any expectations for the
how the calling code will use the returned string (currently,
'statusbar' is used for RPC warnings, not a status bar)
- removes some error-handling code for when the passed-in string is not
one of the two strings expected.
2019-12-15 13:24:48 -03:00
John Newbery
869b6314fd [qt] remove unused parameter from getWarnings() 2019-12-15 13:23:30 -03:00
Wladimir J. van der Laan
a595011f5a Merge #17728: rpc: require second argument only for scantxoutset start action
7d263571be rpc: require second argument only for scantxoutset start action (Andrew Chow)

Pull request description:

  It was reported on [IRC](http://www.erisian.com.au/bitcoin-core-dev/log-2019-12-11.html#l-377) that `scantxoutset`'s API was broken in 0.19.0:

  ```
  <belcher> i think scantxoutset may have been broken in bitcoin core 0.19 ? regardless of what parameters i run it with (e.g. "scantxoutset abort", "scantxoutset status") it just returns the help doc, according to the release notes the only change was https://github.com/bitcoin/bitcoin/pull/16285/files but i dont see anything that wouldve broken it, it works fine in 0.18
  <belcher> im on regtest, in case its important
  <harding> I can confirm `scantxoutset abort` returns the help doc on latest master.  Waiting for 0.18.1 to start now to attempt to reproduce there.
  <harding> It looks like it's expecting a second parameter (even though that doesn't make sense with "abort").
  <jonatack> Same for me as well
  <harding> Can also confirm that `scantxoutset abort` returns the expected result on 0.18.1.
  ```

  As noted in the conversation, previously, the second argument of `scanobjects` is only required for the `start` action. `Stop` and `abort` actions did not and could work without them.

  It appears that this was broken by #16240 which enforced the size of the arguments to match the listed required arguments.

  To fix this issue, this PR makes the `scanobjects` argument an optional argument. Then only in the `start` action do we check whether the `scanobjects` argument is there and throw an informative error about that. Also a test is added for this case.

ACKs for top commit:
  laanwj:
    ACK 7d263571be
  promag:
    ACK 7d263571be.

Tree-SHA512: 828bdfe47f4fffa5d00a2cf88db6cea4a2714d9c49276841ca5cbdd1603b87bb6862147b86edcf36d7b40314ddb80b1a07fd399faf288572c55cc788c5cf9526
2019-12-15 13:07:07 +01:00
Elichai Turkel
e190000869 ci-s390x: Add qemu and depends support in the ci script 2019-12-14 18:27:13 +02:00
Wladimir J. van der Laan
ddecb671f0 Merge #17654: Unbreak build with Boost 1.72.0
a64e97dd47 wallet: unbreak with boost 1.72 (Jan Beich)

Pull request description:

  Regressed by https://github.com/boostorg/filesystem/commit/9a14c37d6f95. See [error log](http://package22.nyi.freebsd.org/data/113amd64-default-PR241449/2019-11-27_11h48m22s/logs/bitcoin-0.19.0.1.log).
  35eda631ed/src/fs.h (L14)

ACKs for top commit:
  MarcoFalke:
    ACK a64e97dd47

Tree-SHA512: 0aad2b8ec211bb81021a2f8cd2059364f949be716ebaf154dd97d5c2f7119f42553892e90e6c375018ff2155b996690c7520374762259778de88014cb531ad3b
2019-12-13 15:39:32 +01:00
fanquake
c78b123982 build: add -bind_at_load to hardened LDFLAGS
This performs the same function as -Wl,-z,now, except for ld on macOS.

You can check the binaries using otool -l, looking for the
LC_DYLD_INFO_ONLY section. lazy_bind_off and lazy_bind_size should both
be 0.

man ld:

-bind_at_load
Sets a bit in the mach header of the resulting binary which tells dyld
to bind all symbols when the binary is loaded, rather than lazily.
2019-12-13 09:33:20 -05:00
Aaron Clauson
592af5ad3a Moved the include of the system projects to before the build depends on task. Otherwise it doesn't get run. 2019-12-13 14:01:52 +00:00
fanquake
244501fc85 depends: disable unused qt networking features 2019-12-13 08:30:26 -05:00
fanquake
29d56c62b7 depends: -optimized-qmake is now -optimized-tools 2019-12-13 08:30:26 -05:00
fanquake
ccdda96804 depends: skip building qt proxies 2019-12-13 08:30:26 -05:00
Aaron Clauson
6e2215187e Included test_bitcoin-qt in msvc build. 2019-12-13 12:07:32 +00:00
Wladimir J. van der Laan
988fe5b1ad Merge #12763: Add RPC Whitelist Feature from #12248
2081442c42 test: Add test for rpc_whitelist (Emil Engler)
7414d3820c Add RPC Whitelist Feature from #12248 (Jeremy Rubin)

Pull request description:

  Summary
  ====

  This patch adds the RPC whitelisting feature requested in #12248. RPC Whitelists help enforce application policies for services being built on top of Bitcoin Core (e.g., your Lightning Node maybe shouldn't be adding new peers). The aim of this PR is not to make it advisable to connect your Bitcoin node to arbitrary services, but to reduce risk and prevent unintended access.

  Using RPC Whitelists
  ====
  The way it works is you specify (in your bitcoin.conf) configurations such as

  ```
  rpcauth=user1:4cc74397d6e9972e5ee7671fd241$11849357f26a5be7809c68a032bc2b16ab5dcf6348ef3ed1cf30dae47b8bcc71
  rpcauth=user2:181b4a25317bff60f3749adee7d6bca0$d9c331474f1322975fa170a2ffbcb176ba11644211746b27c1d317f265dd4ada
  rpcauth=user3:a6c8a511b53b1edcf69c36984985e$13cfba0e626db19061c9d61fa58e712d0319c11db97ad845fa84517f454f6675
  rpcwhitelist=user1:getnetworkinfo
  rpcwhitelist=user2:getnetworkinfo,getwalletinfo, getbestblockhash
  rpcwhitelistdefault=0
  ```

  Now user1 can only call getnetworkinfo, user2 can only call getnetworkinfo or getwalletinfo, while user3 can still call all RPCs.

  If any rpcwhitelist is set, act as if all users are subject to whitelists unless rpcwhitelistdefault is set to 0. If rpcwhitelistdefault is set to 1 and no rpcwhitelist is set, act as if all users are subject to whitelists.

  Review Request
  =====
  In addition to normal review, would love specific review from someone working on LN (e.g., @ roasbeef) and someone working on an infrastructure team at an exchange (e.g., @ jimpo) to check that this works well with their system.

  Notes
  =====

  The rpc list is spelling sensitive -- whitespace is stripped though. Spelling errors fail towards the RPC call being blocked, which is safer.

  It was unclear to me if HTTPReq_JSONRPC is the best function to patch this functionality into, or if it would be better to place it in exec or somewhere else.

  It was also unclear to me if it would be preferred to cache the whitelists on startup or parse them on every RPC as is done with multiUserAuthorized. I opted for the cached approach as I thought it was a bit cleaner.

  Future Work
  =====

  In a future PR, I would like to add an inheritance scheme. This seemed more controversial so I didn't want to include that here. Inheritance semantics are tricky, but it would also make these whitelists easier to read.

  It also might be good to add a `getrpcwhitelist` command to facilitate permission discovery.

  Tests
  =====
  Thanks to @ emilengler for adding tests for this feature. The tests cover all cases except for where `rpcwhitelistdefault=1` is used, given difficulties around testing with the current test framework.

ACKs for top commit:
  laanwj:
    ACK 2081442c42

Tree-SHA512: 0dc1ac6a6f2f4b0be9c9054d495dd17752fe7b3589aeab2c6ac4e1f91cf4e7e355deedcb5d76d707cbb5a949c2f989c871b74d6bf129351f429569a701adbcbf
2019-12-13 11:27:36 +01:00
Wladimir J. van der Laan
995b6c83e1 Merge #17721: util: Don't allow Base58 decoding of non-Base58 strings. Add Base58 tests.
d945c6f5e6 util: Don't allow base58-decoding of std::string:s containing non-base58 characters (practicalswift)
ff7a999226 tests: Add tests for base58-decoding of std::string:s containing non-base58 characters (practicalswift)

Pull request description:

  Don't allow Base58 decoding of non-Base58 strings. Add Base58 tests.

  Fixes #17718.

  Added tests before the Base58 decoding patch:

  ```
  $ make check
  …
  test/base58_tests.cpp(62): error: in "base58_tests/base58_DecodeBase58":
      check !DecodeBase58(std::string("\0invalid", 8), result) has failed
  test/base58_tests.cpp(67): error: in "base58_tests/base58_DecodeBase58":
      check !DecodeBase58(std::string("good\0bad0IOl", 12), result) has failed
  test/base58_tests.cpp(76): error: in "base58_tests/base58_DecodeBase58":
      check !DecodeBase58Check(std::string("3vQB7B6MrGQZaxCuFg4oh\00IOl", 26), result) has failed
  *** 3 failures are detected in the test module "Bitcoin Core Test Suite"
  …
  $ echo $?
  1
  ```

  Added tests before the Base58 decoding patch:

  ```
  $ make check
  …
  OK
  …
  $ echo $?
  0
  ```

ACKs for top commit:
  MarcoFalke:
    ACK d945c6f5e6 🚓
  laanwj:
    ACK d945c6f5e6

Tree-SHA512: 78fee3a18718c9cfbf2e4b26daaf8f24b4deca00475b7b254fec7f8be740f8898c696d9cd0eaa7c50bca55909b9dff3b516b6fe4db92dc132dcc0a1c5e3d61af
2019-12-13 11:15:28 +01:00
Wladimir J. van der Laan
d4b335c60a Merge #17617: doc: unify unix epoch time descriptions
d94d34f05f doc: update developer notes wrt unix epoch time (Jon Atack)
e2f32cb5c5 qa: unify unix epoch time descriptions (Jon Atack)

Pull request description:

  Closes #17613.

  Updated call sites: mocktime, getblockheader, getblock, pruneblockchain,
  getchaintxstats, getblocktemplate, setmocktime, getpeerinfo, setban,
  getnodeaddresses, getrawtransaction, importmulti, listtransactions,
  listsinceblock, gettransaction, getwalletinfo, getaddressinfo

  Commands for testing manually:
  ```
  bitcoind -help-debug | grep -A1 mocktime
  bitcoin-cli help getblockheader
  bitcoin-cli help getblock
  bitcoin-cli help pruneblockchain
  bitcoin-cli help getchaintxstats
  bitcoin-cli help getblocktemplate
  bitcoin-cli help setmocktime
  bitcoin-cli help getpeerinfo
  bitcoin-cli help setban
  bitcoin-cli help getnodeaddresses
  bitcoin-cli help getrawtransaction
  bitcoin-cli help importmulti
  bitcoin-cli help listtransactions
  bitcoin-cli help listsinceblock
  bitcoin-cli help gettransaction
  bitcoin-cli help getwalletinfo
  bitcoin-cli help getaddressinfo
  ```

ACKs for top commit:
  laanwj:
    re-ACK d94d34f05f

Tree-SHA512: 060713ea4e20ab72c580f06c5c7e3ef344ad9c2c9cb034987d980a54e3ed2ac0268eb3929806daa5caa7797c45f5305254fd499767db7f22862212cf77acf236
2019-12-13 10:53:47 +01:00
Jon Atack
d94d34f05f doc: update developer notes wrt unix epoch time 2019-12-13 02:05:05 +01:00
Jon Atack
e2f32cb5c5 qa: unify unix epoch time descriptions
to "UNIX epoch time".

Call sites updated:
```
mocktime
getblockheader
getblock
pruneblockchain
getchaintxstats
getblocktemplate
setmocktime
getpeerinfo
setban
getnodeaddresses
getrawtransaction
importmulti
listtransactions
listsinceblock
gettransaction
getwalletinfo
getaddressinfo
```
2019-12-13 02:02:29 +01:00
MarcoFalke
c5e318aea6 Merge #17736: Update msvc build for Visual Studio 2019 v16.4
75d9317bc1 Update msvc build for Visual Studio 2019 v16.4 (Aaron Clauson)

Pull request description:

  msvc warning C4834 for the Bitcoin Core build was introduced by Visual Studio 16.4.0. This PR adds an ignore rule for the warning (it's related to the nodiscard attribute and is not considered relevant).

  An additional side effect of the msvc compiler update is the prebuilt Qt5.9.8 libraries cannot be linked due to being built with an earlier version of the compiler. To fix this a new Qt5.9.8 version has been compiled and the appveyor job updated to use them.

  The GitHub Actions job needs to continue to use the original Qt5.9.8 libraries until the latest GitHub Windows image also updates to >= Visual Studio 2019 v16.4.

Top commit has no ACKs.

Tree-SHA512: c28d64d78a968eb0bd614932b2d42d762d68853120c345970072b473e2c43fb34e99865062ae1517b10e76f269de6b8f4eed119cf05d59aa883a3553d6a76812
2019-12-12 15:55:33 -05:00
Emil Engler
2081442c42 test: Add test for rpc_whitelist 2019-12-12 11:52:26 -08:00
Aaron Clauson
75d9317bc1 Update msvc build for Visual Studio 2019 v16.4
msvc warning C4834 for the Bitcoin Core build was introduced by Visual Studio 16.4.0. This PR adds an ignore rule for the warning (it's related to the nodiscard attribute and is not considered relevant).
An additional side effect of the msvc compiler update is the prebuilt Qt5.9.8 libraries cannot be linked due to being built with an earlier version of the compiler. To fix this a new Qt5.9.8 version has been compiled and the appveyor job updated to use them. The GitHub Actions job needs to continue to use the original Qt5.9.8 libraries until the latest GitHub Windows image also updates to >= Visual Studio 2019 v16.4.
2019-12-12 18:51:30 +00:00
James O'Beirne
02b9511d6b tests: add tests for GetCoinsCacheSizeState 2019-12-12 11:55:27 -05:00
James O'Beirne
b17e91d842 refactoring: introduce CChainState::GetCoinsCacheSizeState
This separates out some logic for detecting how full the coins cache is from
FlushStateToDisk. We'll want to reuse this logic when deciding when to flush
the coins cache during UTXO snapshot activation.
2019-12-12 11:20:17 -05:00
MarcoFalke
54e11a39e1 Merge #17735: ci: fix typo
5096baf26b build: fix typo (Harris)

Pull request description:

  This PR fixes a typo in .github/workflows/ci.yml.

  test_bticoin => test_bitcoin.

ACKs for top commit:
  practicalswift:
    ACK 5096baf26b

Tree-SHA512: 478fb906adad493ae75872157d269e5060002878784968cfa44b23973b6fccb30cd643728d081a9ed20cff652a8784a33bc281ca5804935ed3c918200190cc9e
2019-12-12 10:54:53 -05:00
Harris
5096baf26b build: fix typo 2019-12-12 16:11:05 +01:00
fanquake
cf2f439876 Merge #17687: cli: fix Fatal LevelDB error when specifying -blockfilterindex=basic twice
034561f9cd cli: fix Fatal LevelDB error when specifying -blockfilterindex=basic twice (Harris)

Pull request description:

  This PR fixes #17679 by replacing BlockFilterType-vector with a set of the same type to make sure that only unique filter types get inserted.

ACKs for top commit:
  MarcoFalke:
    ACK 034561f9cd 📖
  laanwj:
    ACK 034561f9cd
  fanquake:
    ACK 034561f9cd - Tested with `src/bitcoind --blockfilterindex=basic --blockfilterindex=basic`

Tree-SHA512: 64ccec4d23528abfbb564f2b41fb846137875260ce06ea461da12175819985964a1a7442788d5ff7282b5de0c5fd46524d9a793788ee3b876626cbdf05b28c16
2019-12-12 08:04:31 -05:00
fanquake
8a01450b64 Merge #17598: doc: Update release process with latest changes
fab2f351f2 doc: Update release process with latest changes (MarcoFalke)

Pull request description:

  Mainly adding the reminder to bump the flatpak

ACKs for top commit:
  laanwj:
    ACK fab2f351f2
  fanquake:
    ACK fab2f351f2

Tree-SHA512: fe279a6cdee881e8dd608cb7d09d992c4b668b01b9d0d2dbfaf92f12f3032b8fcb2c256b20fcee861397451add1338f162b6e5fa7b3c21e76c247cc419315284
2019-12-12 07:06:21 -05:00
fanquake
75a2a4f357 Merge #17726: ci: Use python 3.7 on Windows Github Actions
fabd5b444e ci: Use python 3.7 on Windows Github Actions (MarcoFalke)

Pull request description:

  This mirrors the appveyor config 7da9e3a817/.appveyor.yml (L10) and is needed for PEP 540

ACKs for top commit:
  sipsorcery:
    tACK fabd5b444e.
  laanwj:
    ACK fabd5b444e

Tree-SHA512: 2d0118bf4eb5ec510d1ad6e287d35bf28cc800101fa18704c119c7bc84f545aaa236ffe45dc425559e6bd896610302a133b2c50ccdcd3ced6e4d6f8302de7cdb
2019-12-12 07:01:51 -05:00
Wladimir J. van der Laan
0192bd0652 Merge #17369: Refactor: Move encryption code between KeyMan and Wallet
7cecf10ac3 Replace LegacyScriptPubKeyMan::IsCrypted with LegacyScriptPubKeyMan::HasEncryptionKeys (Andrew Chow)
bf6417142f Remove SetCrypted() and fUseCrypto; Change IsCrypted()'s implementation (Andrew Chow)
77a777118e Rename EncryptKeys to Encrypt and pass in the encrypted batch to use (Andrew Chow)
35f962fcf0 Clear mapKeys before encrypting (Andrew Chow)
14b5efd66f Move fDecryptionThoroughlyChecked from CWallet to LegacyScriptPubKeyMan (Andrew Chow)
97c0374a46 Move Unlock implementation to LegacyScriptPubKeyMan (Andrew Chow)
e576b135d6 Replace LegacyScriptPubKeyMan::vMasterKey with GetDecryptionKey() (Andrew Chow)
fd9d6eebc1 Add GetEncryptionKey() and HasEncryptionKeys() to WalletStorage (Andrew Chow)

Pull request description:

  Let wallet class handle locked/unlocked status and master key, and let keyman
  handle encrypting its data and determining whether there is encrypted data.

  There should be no change in behavior, but state is tracked differently. The
  fUseCrypto atomic bool is eliminated and replaced with equivalent
  HasEncryptionKeys checks.

  Split from #17261

ACKs for top commit:
  laanwj:
    ACK 7cecf10ac3

Tree-SHA512: 95a997c366ca539abba0c0a7a0015f39d27b55220683d8d86344ff2d926db4724da67700d2c8ec2d82ed75d07404318c6cb81544af8aadeefab312167257e673
2019-12-12 12:17:10 +01:00
practicalswift
d945c6f5e6 util: Don't allow base58-decoding of std::string:s containing non-base58 characters 2019-12-12 11:01:56 +00:00
practicalswift
ff7a999226 tests: Add tests for base58-decoding of std::string:s containing non-base58 characters 2019-12-12 11:01:56 +00:00
Wladimir J. van der Laan
3914e877c4 Merge #17511: Add bounds checks before base58 decoding
5909bcd3bf Add bounds checks in key_io before DecodeBase58Check (Pieter Wuille)
2bcf1fc444 Pass a maximum output length to DecodeBase58 and DecodeBase58Check (Pieter Wuille)

Pull request description:

  Fixes #17501.

ACKs for top commit:
  laanwj:
    code review ACK 5909bcd3bf
  practicalswift:
    ACK 5909bcd3bf -- code looks correct

Tree-SHA512: 4807f4a9508dee9c0f1ad63f56f70f4ec4e6b7e35eb91322a525e3da3828521a41de9b8338a6bf67250803660b480d95fd02ce6b2fe79c4c88bc19b54f9d8889
2019-12-12 10:56:31 +01:00
fanquake
3f1966ead6 Merge #17705: test: re-enable CLI test support by using EncodeDecimal in json.dumps()
b6f9e3576a test: re-enable CLI test support by using EncodeDecimal in json.dumps() (fanquake)

Pull request description:

  As mentioned in https://github.com/bitcoin/bitcoin/pull/17675#issuecomment-563188648.

ACKs for top commit:
  practicalswift:
    ACK b6f9e3576a assuming Travis is happy too -- diff looks correct :)
  MarcoFalke:
    > ACK b6f9e35 assuming Travis is happy too -- diff looks correct :)

Tree-SHA512: 79fa535cc1756c8ee610a3d6a316a1c4f036797d6990a5620e44985393a2e52f78450f8e0021d0a148c08705fd1ba765508464a365f9030ae0d2cacbd7a93e19
2019-12-11 20:33:28 -05:00
Andrew Chow
7d263571be rpc: require second argument only for scantxoutset start action
The second argument of scanobjects is only required for the start action.
Stop and abort actions do not need this.
2019-12-11 17:19:33 -05:00
MarcoFalke
5948398b18 Merge #17474: Bugfix: GUI: Recognise NETWORK_LIMITED in formatServicesStr
4341bffb6e GUI: Refactor formatServicesStr to warn when a ServicesFlag is missing (Luke Dashjr)
df77de8c21 Bugfix: GUI: Recognise NETWORK_LIMITED in formatServicesStr (Luke Dashjr)

Pull request description:

  Currently, only the bottom 8 service bits are shown in the GUI peer details view.

  `NODE_NETWORK_LIMITED` is the 11th bit (2^10).

  The first commit expands the range to cover the full 64 bits, and properly label `"NETWORK_LIMITED"`.
  The second commit refactors the code so that any future omitted service bits will trigger a compile warning.

ACKs for top commit:
  jonasschnelli:
    utACK 4341bffb6e
  jonasschnelli:
    Tested ACK 4341bffb6e
  hebasto:
    Concept ACK 4341bffb6e

Tree-SHA512: 8338737d03fbcd92024159aabd7e632d46e13c72436d935b504d2bf7ee92b7d124e89a5917bf64d51c87f12a64de703270c2d7b4c6711fa8ed08ea7887d817c7
2019-12-11 17:00:27 -05:00
James O'Beirne
bee88b8c58 tests: have coins simulation test also use CCoinsViewDB
Before this change, the coins simulation test uses a base view of type
CCoinsViewTest, which has no relevance outside of the unittest suite. Might as
well reuse this testcase with a more realistic configuration that has
CCoinsViewDB at the bottom of the view structure.
2019-12-11 15:44:21 -05:00
Jeremy Rubin
7414d3820c Add RPC Whitelist Feature from #12248 2019-12-11 12:33:54 -08:00
MarcoFalke
fabd5b444e ci: Use python 3.7 on Windows Github Actions 2019-12-11 15:30:23 -05:00
Jan Beich
a64e97dd47 wallet: unbreak with boost 1.72
wallet/walletutil.cpp:77:23: error: no member named 'level' in 'boost::filesystem::recursive_directory_iterator'
        } else if (it.level() == 0 && it->symlink_status().type() == fs::regular_file && IsBerkeleyBtree(it...
                   ~~ ^
2019-12-11 18:51:16 +00:00
MarcoFalke
7da9e3a817 Merge #17050: tests: Add fuzzing harnesses for functions parsing scripts, numbers, JSON and HD keypaths (bip32)
a1308b7e12 tests: Add fuzzing harnesses for various JSON/univalue parsing functions (practicalswift)
e3d2bcf5cf tests: Add fuzzing harnesses for various number parsing functions (practicalswift)
fb8c12093a tests: Add ParseScript(...) (core_io) fuzzing harness (practicalswift)
074cb6451b tests: Add ParseHDKeypath(...) (bip32) fuzzing harness (practicalswift)
0dc5907d0f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)

Pull request description:

  Add fuzzing harnesses for `DecodeRawPSBT(...)`, `ParseHDKeypath(...)`, `ParseScript(...)`, various number parsing functions and various JSON/univalue parsing functions.

  **Testing this PR**
  As usual the best way to test proposed fuzzing harnesses is to use `test_fuzzing_harnesses.sh` (#17000) to quickly verify that the relevant code regions are triggered, that the fuzzing throughput seems reasonable, etc.

  `test_fuzzing_harnesses.sh 'psbt|hd_keypath|numbers|parse_script|univalue' 10` runs all fuzzers matching the regexp and gives them ten seconds of runtime each.

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ contrib/devtools/test_fuzzing_harnesses.sh 'psbt|hd_keypath|numbers|parse_script|univalue' 10
  Testing fuzzer parse_hd_keypath during 10 second(s)
  A subset of reached functions:
          NEW_FUNC[0/2]: 0x55bc23a76940 in ParsePrechecks(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:267
          NEW_FUNC[1/2]: 0x55bc23a77300 in ParseUInt32(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int*) src/util/strencodings.cpp:309
  stat::number_of_executed_units: 34237
  stat::average_exec_per_sec:     3112
  stat::new_units_added:          113
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              282
  Number of unique code paths taken during fuzzing round: 30

  Testing fuzzer parse_numbers during 10 second(s)
  A subset of reached functions:
  stat::number_of_executed_units: 31309
  stat::average_exec_per_sec:     2846
  stat::new_units_added:          688
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              234
  Number of unique code paths taken during fuzzing round: 149

  Testing fuzzer parse_script during 10 second(s)
  A subset of reached functions:
          NEW_FUNC[1/11]: 0x5636ff61ba00 in IsDigit(char) src/./util/strencodings.h:70
          NEW_FUNC[0/14]: 0x5636fe6c6280 in CScript::operator<<(opcodetype) src/./script/script.h:448
          NEW_FUNC[1/14]: 0x5636fe6e0290 in prevector<28u, unsigned char, unsigned int, int>::insert(prevector<28u, unsigned char, unsigned int, int>::iterator, unsigned char const&) src/./prevector.h:342
          NEW_FUNC[2/14]: 0x5636fe6e1040 in prevector<28u, unsigned char, unsigned int, int>::size() const src/./prevector.h:277
          NEW_FUNC[3/14]: 0x5636fe6e1250 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295
          NEW_FUNC[4/14]: 0x5636fe6e1cb0 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) src/./prevector.h:196
          NEW_FUNC[0/10]: 0x5636fe6c5650 in CScript::operator<<(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/./script/script.h:462
          NEW_FUNC[2/10]: 0x5636fe6e0a20 in void prevector<28u, unsigned char, unsigned int, int>::insert<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > > >(prevector<28u, unsigned char, unsigned int, int>::iterator, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<[32/1902]
  char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >) src/./prevector.h:368
          NEW_FUNC[5/10]: 0x5636fe6e2350 in void prevector<28u, unsigned char, unsigned int, int>::fill<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > > >(unsigned char*, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsign
  ed char const*, std::vector<unsigned char, std::allocator<unsigned char> > >) src/./prevector.h:204
          NEW_FUNC[0/1]: 0x5636ff8e48b0 in IsHex(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:61
          NEW_FUNC[0/2]: 0x5636fe6e1410 in prevector<28u, unsigned char, unsigned int, int>::change_capacity(unsigned int) src/./prevector.h:165
          NEW_FUNC[1/2]: 0x5636fe6e1f00 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) src/./prevector.h:161
          NEW_FUNC[0/1]: 0x5636fe6e0580 in void prevector<28u, unsigned char, unsigned int, int>::insert<unsigned char*>(prevector<28u, unsigned char, unsigned int, int>::iterator, unsigned char*, unsigned char*) src/./prevector.h:368
          NEW_FUNC[0/3]: 0x5636fe85f0d0 in CScript::push_int64(long) src/./script/script.h:394
          NEW_FUNC[1/3]: 0x5636fe85f520 in prevector<28u, unsigned char, unsigned int, int>::push_back(unsigned char const&) src/./prevector.h:422
          NEW_FUNC[2/3]: 0x5636ff8ed730 in atoi64(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:417
  stat::number_of_executed_units: 8153
  stat::average_exec_per_sec:     741
  stat::new_units_added:          296
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              237
  Number of unique code paths taken during fuzzing round: 98

  Testing fuzzer parse_univalue during 10 second(s)
  A subset of reached functions:
          NEW_FUNC[0/19]: 0x560db8655950 in tinyformat::detail::formatImpl(std::ostream&, char const*, tinyformat::detail::FormatArg const*, int) src/./tinyformat.h:791
          NEW_FUNC[4/19]: 0x560db86582b0 in tinyformat::detail::printFormatStringLiteral(std::ostream&, char const*) src/./tinyformat.h:564
          NEW_FUNC[5/19]: 0x560db8658690 in tinyformat::detail::streamStateFromFormat(std::ostream&, bool&, int&, char const*, tinyformat::detail::FormatArg const*, int&, int) src/./tinyformat.h:601
          NEW_FUNC[6/19]: 0x560db865f090 in tinyformat::detail::FormatArg::format(std::ostream&, char const*, char const*, int) const src/./tinyformat.h:513
          NEW_FUNC[12/19]: 0x560db8661ba0 in void tinyformat::detail::FormatArg::formatImpl<int>(std::ostream&, char const*, char const*, int, void const*) src/./tinyformat.h:530
          NEW_FUNC[13/19]: 0x560db8661d90 in void tinyformat::formatValue<int>(std::ostream&, char const*, char const*, int, int const&) src/./tinyformat.h:317
          NEW_FUNC[14/19]: 0x560db875c8b0 in void tinyformat::detail::FormatArg::formatImpl<unsigned int>(std::ostream&, char const*, char const*, int, void const*) src/./tinyformat.h:530
          NEW_FUNC[15/19]: 0x560db875caa0 in void tinyformat::formatValue<unsigned int>(std::ostream&, char const*, char const*, int, unsigned int const&) src/./tinyformat.h:317
          NEW_FUNC[16/19]: 0x560db9473ef0 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<int, unsigned int>(char const*, int const&, unsigned int const&) src/./tinyformat.h:976
          NEW_FUNC[17/19]: 0x560db94749a0 in void tinyformat::format<int, unsigned int>(std::ostream&, char const*, int const&, unsigned int const&) src/./tinyformat.h:968
          NEW_FUNC[18/19]: 0x560db9474cf0 in tinyformat::detail::FormatListN<2>::FormatListN<int, unsigned int>(int const&, unsigned int const&) src/./tinyformat.h:885
  stat::number_of_executed_units: 14089
  stat::average_exec_per_sec:     1280
  stat::new_units_added:          135
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              356
  Number of unique code paths taken during fuzzing round: 62

  Testing fuzzer psbt_input_deserialize during 10 second(s)
  A subset of reached functions:
          NEW_FUNC[0/46]: 0x557847ce3530 in prevector<28u, unsigned char, unsigned int, int>::~prevector() src/./prevector.h:456
          NEW_FUNC[3/46]: 0x557847cfdcf0 in prevector<28u, unsigned char, unsigned int, int>::size() const src/./prevector.h:277
          NEW_FUNC[4/46]: 0x557847cfe0c0 in prevector<28u, unsigned char, unsigned int, int>::change_capacity(unsigned int) src/./prevector.h:165
          NEW_FUNC[13/46]: 0x557847d3c890 in unsigned long ReadCompactSize<CDataStream>(CDataStream&) src/./serialize.h:290
          NEW_FUNC[14/46]: 0x557847d47b60 in prevector<28u, unsigned char, unsigned int, int>::resize(unsigned int) src/./prevector.h:311
          NEW_FUNC[16/46]: 0x557847d48800 in CTxOut::CTxOut() src/./primitives/transaction.h:140
          NEW_FUNC[17/46]: 0x557847d4b050 in CTxOut::SetNull() src/./primitives/transaction.h:155
          NEW_FUNC[18/46]: 0x557847d4b140 in CScript::clear() src/./script/script.h:563
          NEW_FUNC[19/46]: 0x557847d4ead0 in void Unserialize_impl<CDataStream, unsigned char, std::allocator<unsigned char> >(CDataStream&, std::vector<unsigned char, std::allocator<unsigned char> >&, unsigned char const&) src/./serialize.h:746
          NEW_FUNC[0/58]: 0x557847cfdf00 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295
          NEW_FUNC[1/58]: 0x557847cfe960 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) src/./prevector.h:196
          NEW_FUNC[2/58]: 0x557847cfebb0 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) src/./prevector.h:161
          NEW_FUNC[3/58]: 0x557847d03990 in uint256::uint256() src/./uint256.h:123
          NEW_FUNC[0/3]: 0x557847d47430 in void CScript::SerializationOp<CDataStream, CSerActionUnserialize>(CDataStream&, CSerActionUnserialize) src/./script/script.h:418
          NEW_FUNC[1/3]: 0x557847d47730 in void Unserialize_impl<CDataStream, 28u, unsigned char>(CDataStream&, prevector<28u, unsigned char, unsigned int, int>&, unsigned char const&) src/./serialize.h:666
          NEW_FUNC[2/3]: 0x557847d60dd0 in CDataStream& CDataStream::operator>><CScript&>(CScript&) src/./streams.h:460
          NEW_FUNC[1/78]: 0x557847cffae0 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) const src/./prevector.h:197
          NEW_FUNC[2/78]: 0x557847cffd30 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) const src/./prevector.h:162
          NEW_FUNC[0/1]: 0x557847d65f90 in OverrideStream<CDataStream>& OverrideStream<CDataStream>::operator>><unsigned char&>(unsigned char&) src/./streams.h:46
          NEW_FUNC[0/3]: 0x557847d470e0 in void SerReadWriteMany<CDataStream, CScript&>(CDataStream&, CSerActionUnserialize, CScript&) src/./serialize.h:989
          NEW_FUNC[1/3]: 0x557847d4ac50 in void CTxOut::SerializationOp<CDataStream, CSerActionUnserialize>(CDataStream&, CSerActionUnserialize) src/./primitives/transaction.h:149
          NEW_FUNC[2/3]: 0x557847d5f860 in void UnserializeFromVector<CDataStream, CTxOut>(CDataStream&, CTxOut&) src/./script/sign.h:90
          NEW_FUNC[0/1]: 0x557847d60840 in void UnserializeFromVector<CDataStream, int>(CDataStream&, int&) src/./script/sign.h:90
          NEW_FUNC[0/1]: 0x557847d41010 in CMutableTransaction::HasWitness() const src/./primitives/transaction.h:398
  stat::number_of_executed_units: 13615
  stat::average_exec_per_sec:     1237
  stat::new_units_added:          357
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              446
  Number of unique code paths taken during fuzzing round: 152

  Testing fuzzer psbt_output_deserialize during 10 second(s)
  A subset of reached functions:
          NEW_FUNC[0/27]: 0x55c9347e5940 in prevector<28u, unsigned char, unsigned int, int>::~prevector() src/./prevector.h:456
          NEW_FUNC[5/27]: 0x55c93483eca0 in unsigned long ReadCompactSize<CDataStream>(CDataStream&) src/./serialize.h:290
          NEW_FUNC[6/27]: 0x55c934850ee0 in void Unserialize_impl<CDataStream, unsigned char, std::allocator<unsigned char> >(CDataStream&, std::vector<unsigned char, std::allocator<unsigned char> >&, unsigned char const&) src/./serialize.h:746
          NEW_FUNC[14/27]: 0x55c934858500 in PSBTOutput::PSBTOutput() src/./psbt.h:281
          NEW_FUNC[15/27]: 0x55c934858870 in CDataStream& CDataStream::operator>><PSBTOutput&>(PSBTOutput&) src/./streams.h:460
          NEW_FUNC[0/1]: 0x55c934800100 in prevector<28u, unsigned char, unsigned int, int>::size() const src/./prevector.h:277
          NEW_FUNC[0/4]: 0x55c934849840 in void CScript::SerializationOp<CDataStream, CSerActionUnserialize>(CDataStream&, CSerActionUnserialize) src/./script/script.h:418
          NEW_FUNC[1/4]: 0x55c934849b40 in void Unserialize_impl<CDataStream, 28u, unsigned char>(CDataStream&, prevector<28u, unsigned char, unsigned int, int>&, unsigned char const&) src/./serialize.h:666
          NEW_FUNC[2/4]: 0x55c934849f70 in prevector<28u, unsigned char, unsigned int, int>::resize(unsigned int) src/./prevector.h:311
          NEW_FUNC[3/4]: 0x55c93485dc60 in CDataStream& CDataStream::operator>><CScript&>(CScript&) src/./streams.h:460
          NEW_FUNC[0/3]: 0x55c934800310 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295
          NEW_FUNC[1/3]: 0x55c934800d70 in prevector<28u, unsigned char, unsigned int, int>::item_ptr(int) src/./prevector.h:196
          NEW_FUNC[2/3]: 0x55c934849d40 in prevector<28u, unsigned char, unsigned int, int>::resize_uninitialized(unsigned int) src/./prevector.h:381
          NEW_FUNC[0/1]: 0x55c93485ddd0 in void DeserializeHDKeypaths<CDataStream>(CDataStream&, std::vector<unsigned char, std::allocator<unsigned char> > const&, std::map<CPubKey, KeyOriginInfo, std::less<CPubKey>, std::allocator<std::pair<CPubKey const, KeyOriginInfo> > >&) src/./script/sign.h:103
  stat::number_of_executed_units: 19130
  stat::average_exec_per_sec:     1739
  stat::new_units_added:          195
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              411
  Number of unique code paths taken during fuzzing round: 64

  Tested fuzz harnesses seem to work as expected.
  ```

Top commit has no ACKs.

Tree-SHA512: baf1630a6e438d02d33c77b9e602c99546b9e8d83705e67c2749e0600039c37707cdf419cee19282f069e8d787c536ed4960f9c47e93bd0f0251495b83780ada
2019-12-11 13:37:15 -05:00
MarcoFalke
14dafcbc13 Merge #17713: doc: Add release notes for 17447
fa4b656e97 doc: Add release notes for 17447 (MarcoFalke)

Pull request description:

  Stolen from https://github.com/bitcoin/bitcoin/pull/17447#issuecomment-553475914

ACKs for top commit:
  promag:
    ACK fa4b656e97.
  laanwj:
    ACK fa4b656e97

Tree-SHA512: 5d281c0a85e75c9fae8885faf0e4a2ca4e4f73788f3d214ca65c7c891203a7435cc77fe3046e2d7e3e2226d96c547005f1d970e768d6cd82423f575e07881431
2019-12-11 13:10:36 -05:00
MarcoFalke
f1d3d3430e Merge #17714: rpc: add missing newline in analyzepsbt RPCResult
7e8b4de059 rpc: add missing newline in analyzepsbt rpcresult (Jon Atack)

Pull request description:

  follow-up to 638e40c in #17524

  before
  ```
    "error" : "error"               (string) Error message if there is one}
  ```
  after
  ```
    "error" : "error"               (string) Error message if there is one
  }
  ```

ACKs for top commit:
  practicalswift:
    ACK 7e8b4de059
  promag:
    ACK 7e8b4de059.
  emilengler:
    ACK 7e8b4de

Tree-SHA512: 4cdd365e39d15b7925ea277b7ff3e9bfdc22f5845aa41ca547343b4dabdf319579843a1c7f11fb0edd6abbc31bae2ec96236b83e84f8872bd662848723725e4c
2019-12-11 10:40:49 -05:00
MarcoFalke
fab9d115cd Merge #17697: CI: GitHub Action workflow which duplicates AppVeyor job
b0b1531737 Adds GitHub Action workflow which duplicates AppVeyor job. (Aaron Clauson)

Pull request description:

  As discussed in #17594 this PR contains a GitHub Action workflow file that performs the same job as the current Appveyor CI task except for the Python functional tests. For the latter I've been unable to get them to execute successfully due to a Unicode error. I've tried on and off for a week to get it to work but with no joy.

  It may be that someone more proficient in Python will recognise the error and be able to provide a pointer on how to proceed. I've tried some obvious things like changing the Windows console code page.

  To run this job it should just be a matter of clicking on the GitHub `Actions` tab and enabling workflows. It's also not required that the file is on the `master` branch for the job to run. If anyone else wants to run the job they can pull this PR into their own fork and enable `Actions` (it's free).

Top commit has no ACKs.

Tree-SHA512: 8dce7509922ece3438b15ea371ec509a08b507e981a8fb705f1cf5a2b4a147a22ded599942aa95f3bd8d5e98cfc65b50cf3df6171f02dd863659160f1d77ef76
2019-12-11 09:44:56 -05:00
Wladimir J. van der Laan
4863a8ff16 Merge #17698: depends: don't configure xcb_proto
e97f5c1823 depends: don't configure xcb_proto (fanquake)

Pull request description:

  xcb_proto's configure doesn't understand `--disable-shared` or
  `--with-pic`. All the package does it put a stack of XML files into
  a directory to be used by libxcb.

  Probably enough to close #16354.

ACKs for top commit:
  dongcarl:
    ACK e97f5c1823

Tree-SHA512: 1a49fd7c8269405bbf312be33c1aeaac5f25ef8666829b01dc3c58f3a2a9281c23c42614a7f1cfc3ee260be4ea3e71285869b1cb9c2035dceda336296d9d9dea
2019-12-11 12:27:51 +01:00
Andrew Chow
f1ef7f0aa4 Don't calculate tx fees for PSBTs with invalid money values
In decodepsbt if an invalid amount is seen, don't calculate the fee
but still show the invalid value in the decode.

In analyze psbt, if an invalid amount is seen, set the next step to
be the creator as the creator needs to remake the transaction so that
it is valid.
2019-12-10 21:10:48 -05:00
MarcoFalke
facb416ad5 ci: Add valgrind run 2019-12-10 19:37:37 -05:00
Jon Atack
7e8b4de059 rpc: add missing newline in analyzepsbt rpcresult
follow-up to 638e40c
2019-12-10 19:48:53 +01:00
MarcoFalke
3d6752779f Merge #17633: tests: Add option --valgrind to run the functional tests under Valgrind
5db506ba59 tests: Add option --valgrind to run nodes under valgrind in the functional tests (practicalswift)

Pull request description:

  What is better than fixing bugs? Fixing entire bug classes of course! :)

  Add option `--valgrind` to run the functional tests under Valgrind.

  Regular functional testing under Valgrind would have caught many of the uninitialized reads we've seen historically.

  Let's kill this bug class once and for all: let's never use an uninitialized value ever again. Or at least not one that would be triggered by running the functional tests! :)

  My hope is that this addition will make it super-easy to run the functional tests under Valgrind and thus increase the probability of people making use of it :)

  Hopefully `test/functional/test_runner.py --valgrind` will become a natural part of the pre-release QA process.

  **Usage:**

  ```
  $ test/functional/test_runner.py --help
  …
    --valgrind            run nodes under the valgrind memory error detector:
                          expect at least a ~10x slowdown, valgrind 3.14 or
                          later required
  ```

  **Live demo:**

  First, let's re-introduce a memory bug by reverting the recent P2P uninitialized read bug fix from PR #17624 ("net: Fix an uninitialized read in ProcessMessage(…, "tx", …) when receiving a transaction we already have").

  ```
  $ git diff
  diff --git a/src/consensus/validation.h b/src/consensus/validation.h
  index 3401eb64c..940adea33 100644
  --- a/src/consensus/validation.h
  +++ b/src/consensus/validation.h
  @@ -114,7 +114,7 @@ inline ValidationState::~ValidationState() {};

   class TxValidationState : public ValidationState {
   private:
  -    TxValidationResult m_result = TxValidationResult::TX_RESULT_UNSET;
  +    TxValidationResult m_result;
   public:
       bool Invalid(TxValidationResult result,
                    const std::string &reject_reason="",
  ```

  Second, let's test as normal without Valgrind:

  ```
  $ test/functional/p2p_segwit.py -l INFO
  2019-11-28T09:30:42.810000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test__fc8q3qo
  …
  2019-11-28T09:31:57.187000Z TestFramework (INFO): Subtest: test_non_standard_witness_blinding (Segwit active = True)
  …
  2019-11-28T09:32:08.265000Z TestFramework (INFO): Tests successful
  ```

  Third, let's test with `--valgrind` and see if the test fail (as we expect) when the unitialized value is used:

  ```
  $ test/functional/p2p_segwit.py -l INFO --valgrind
  2019-11-28T09:32:33.018000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_gtjecx2l
  …
  2019-11-28T09:40:36.702000Z TestFramework (INFO): Subtest: test_non_standard_witness_blinding (Segwit active = True)
  2019-11-28T09:40:37.813000Z TestFramework (ERROR): Assertion failed
  ConnectionRefusedError: [Errno 111] Connection refused
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 5db506ba59
  jonatack:
    ACK 5db506ba59

Tree-SHA512: 2eaecacf4da166febad88b2a8ee6d7ac2bcd38d4c1892ca39516b6343e8f8c8814edf5eaf14c90f11a069a0389d24f0713076112ac284de987e72fc5f6cc3795
2019-12-10 13:30:37 -05:00
MarcoFalke
d5674c5f0f Merge #17703: build: Improve configure.ac formatting
3ab1824625 build: Use dnl for all comments in configure.ac, rather than # (fanquake)
8ddcbb4e41 build: Remove backticks from configure.ac (fanquake)

Pull request description:

  Use `dnl` for all comments, rather than `#`.
  Remove backticks - Their usage for the `bdb_prefix` and `qt5_prefix` commands may have improved backwards compatibility in some cases, however we now require recent versions of macOS. I'm not sure why they were being used in the `HAVE_STD__SYSTEM` and `HAVE_WSYSTEM` defines.

ACKs for top commit:
  dongcarl:
    ACK 3ab1824625
  hebasto:
    ACK 3ab1824625, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 2bcffb52c365acff87a0e6b9527ae31f36fdabb7ea095a8fd261f9a39b2c2848f5dfc148bc38d21e21e7bd761b1a2960e9a96f508c66be84d9569b8a401e812a
2019-12-10 13:16:17 -05:00
MarcoFalke
2126d6ce69 Merge #17561: doc: Changed MiniUPnPc link to https in dependencies.md
5ad4dd1ea1 doc: Changed MiniUPnPc link to https in dependencies.md (Marius Kjærstad)

Pull request description:

  doc: Changed MiniUPnPc link to https in dependencies.md

Top commit has no ACKs.

Tree-SHA512: 228ee98c877612468a34d09610999a47257ab1e060f3004a530639f0c29fb473b48e59588ff70297c53a3abeb2bb32bfedbb61e102a7fc10df4bb1b5d0d5893b
2019-12-10 13:09:05 -05:00
MarcoFalke
fa4b656e97 doc: Add release notes for 17447
Co-Authored-By: Russell Yanofsky <russ@yanofsky.org>
2019-12-10 13:03:28 -05:00
MarcoFalke
ea756bc48c Merge #17502: test: add unit test for non-standard bare multisig txs
1bb5d517aa test: add unit test for non-standard bare multisig txs (Sebastian Falbesoner)

Pull request description:

  Approaches another missing unit test of issue #17394: Checks that the function `IsStandardTx()` returns rejection reason `"bare-multisig"` if any one of the outputs' scriptPubKey has bare multisignature format (i.e. `M <PubKey1> <PubKey2> ... <PubKeyN> N OP_CHECKSIG`, not P2SH!) and the policy flag `fIsBareMultisigStd` is set to false.

ACKs for top commit:
  instagibbs:
    utACK 1bb5d517aa

Tree-SHA512: d7c95e35da16520d6dcd2b4278e2426fedd13f68d1f23c90e85e929774e123fbfcfbccc26df6ad1c0dd61780896fa4b4b3d4e8280c647bb06df2bfcf2ba572fb
2019-12-10 12:49:31 -05:00
MarcoFalke
fae94785d9 Merge #17524: psbt: handle unspendable psbts
773d4572a4 Mark PSBTs spending unspendable outputs as invalid in analysis (Andrew Chow)
638e40cb60 Have a PSBTAnalysis state that indicates invalid PSBT (Andrew Chow)

Pull request description:

  When analyzing an unspendable PSBT, report that it is unspendable and exit analysis early.

ACKs for top commit:
  Sjors:
    ACK 773d457
  instagibbs:
    After some thought ACK 773d4572a4

Tree-SHA512: 99b0cb2fa1ea37593fc65a20effe881639d69ddeeecf5197bc87bc7f2220cbeb40f1d429d517e4d27f2e9fb563a00cd845d2b4b1ce05246a75a6cb56fb9b0ba5
2019-12-10 12:12:10 -05:00
MarcoFalke
fab2f351f2 doc: Update release process with latest changes 2019-12-10 12:04:04 -05:00
practicalswift
a1308b7e12 tests: Add fuzzing harnesses for various JSON/univalue parsing functions 2019-12-10 16:39:40 +00:00
practicalswift
e3d2bcf5cf tests: Add fuzzing harnesses for various number parsing functions 2019-12-10 16:39:40 +00:00
practicalswift
fb8c12093a tests: Add ParseScript(...) (core_io) fuzzing harness 2019-12-10 16:39:40 +00:00
practicalswift
074cb6451b tests: Add ParseHDKeypath(...) (bip32) fuzzing harness 2019-12-10 16:39:40 +00:00
practicalswift
0dc5907d0f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-10 16:39:40 +00:00
Gregory Sanders
e1e1442f3e Activate no-privkey -> ISMINE_WATCH_ONLY behavior for LegacySPKM only 2019-12-10 09:27:15 -05:00
fanquake
3ab1824625 build: Use dnl for all comments in configure.ac, rather than # 2019-12-10 08:35:33 -05:00
fanquake
8ddcbb4e41 build: Remove backticks from configure.ac 2019-12-10 08:35:10 -05:00
Wladimir J. van der Laan
5f26855f10 test: Remove ubsan alignment suppressions
This can be done now that prevector is properly aligned.
2019-12-10 21:00:15 +10:00
Anthony Towns
9d933ef919 prevector: avoid misaligned member accesses
`#pragma pack(1)` prevents aligning the struct and its members to their
required alignment. This can result in code that performs non-aligned
reads and writes to integers and pointers, which is problematic on some
architectures.

It also triggers UBsan — see
  https://github.com/bitcoin/bitcoin/pull/17156#issuecomment-543123631
and #17510.
2019-12-10 18:11:40 +10:00
MarcoFalke
1189b6acab Merge #17109: tests: Add fuzzing harness for various functions consuming only integrals
597d10ceb9 tests: Add fuzzing harness for various functions consuming only integrals (practicalswift)
575383b3e1 tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)

Pull request description:

  Add fuzzing harness for various functions consuming only integrals.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/integer
  ```

Top commit has no ACKs.

Tree-SHA512: f0ccbd63671636f8e661385b682e16ad287fef8f92e7f91327ee2093afc36fcd424e1646fe90279388e28a760bcc795766eb80cf6375e0f873efff37fc7e2393
2019-12-09 15:22:27 -05:00
fanquake
b6f9e3576a test: re-enable CLI test support by using EncodeDecimal in json.dumps()
As mentioned in
https://github.com/bitcoin/bitcoin/pull/17675#issuecomment-563188648
2019-12-09 15:14:20 -05:00
MarcoFalke
347dd76ec8 Merge #17093: tests: Add fuzzing harness for various CTx{In,Out} related functions
d5766f223f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)
e75ecb91c7 tests: Add fuzzing harness for various CTxOut related functions (practicalswift)
ce935292c0 tests: Add fuzzing harness for various CTxIn related functions (practicalswift)

Pull request description:

  Add fuzzing harness for various `CTx{In,Out}` related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/tx_in
  …
  $ src/test/fuzz/tx_out
  …
  # And to to quickly verify that the relevant code regions are triggered, that the
  # fuzzing throughput seems reasonable, etc.
  $ contrib/devtools/test_fuzzing_harnesses.sh '^tx_'
  ```

  `test_fuzzing_harnesses.sh` can be found in PR #17000.

Top commit has no ACKs.

Tree-SHA512: f1374307a2581ebc3968d012ea2438061bbb84ece068e584fae9750669a6cd003723dde14db88e77c9579281ecd4eaa2a7ff0614f253d8c075e6dd16dd2e68d5
2019-12-09 15:12:08 -05:00
MarcoFalke
74c6ad3aab Merge #17225: tests: Test serialisation as part of deserialisation fuzzing. Test round-trip equality where possible.
709afb2a7d tests: Test serialisation as part of deserialisation fuzzing. Test round-trip equality where possible. Avoid code repetition. (practicalswift)

Pull request description:

  Test serialisation as part of deserialisation fuzzing. Test round-trip equality where possible.

ACKs for top commit:
  MarcoFalke:
    ACK 709afb2a7d 🍲

Tree-SHA512: b8c9c24538ee516607608ac685d2e9b01eca5c15213def3fd096b16516db84bfd45516fbee43e25b28cb3481a5d4ec3f7a34713e2da35b2902081ed42b85224d
2019-12-09 15:07:36 -05:00
MarcoFalke
5bf1909dd9 Merge #17682: util: Update tinyformat to upstream
978b25528c util: Update tinyformat to upstream (Wladimir J. van der Laan)

Pull request description:

  Last update was in 2017.
  Updates tinyformat to upstream commit  c42f/tinyformat@705e3f4e1d.
  Re-apply (and mark) bitcoin core specific changes.

  No changes that affect our use, as far as I can see, but this gets rid of the gcc `-Wimplicit-fallthrough` warnings, at least.

ACKs for top commit:
  MarcoFalke:
    ACK 978b25528c, extracted our patches based on the last update, did the update to v2.3.0 myself and re-applied the patches. Only diff is NULL/nullptr and explicit 🔝

Tree-SHA512: 2ba09e1095878d088520f379d545b40c7286ef199ecbbc17fdd5c85bca447d9b0c7a1829d4038bb6d432cd1ff92ad7bba75c0f2f96c71aeb6fa6031002f1ea1d
2019-12-09 14:38:28 -05:00
MarcoFalke
df2b743759 Merge #17675: tests: Enable tests which are incorrectly skipped when running test_runner.py --usecli
5ac804a9eb tests: Use a default of supports_cli=True (instead of supports_cli=False) (practicalswift)
993e38a4e2 tests: Mark functional tests not supporting bitcoin-cli (--usecli) as such (practicalswift)

Pull request description:

  Annotate functional tests supporting `bitcoin-cli` (`--usecli`) as such.

  Prior to this commit 74 tests were unnecessarily skipped when running `test_runner.py --usecli`.

  Before:

  ```
  $ test/functional/test_runner.py --usecli > /dev/null 2>&1
  $ echo $?
  0
  $ test/functional/test_runner.py --usecli 2>&1 | cut -f2 -d'|' | \
      grep -E ' (Passed|Skipped) *$' | sort | uniq -c
        9  ✓ Passed
      126  ○ Skipped
  ```

  After:

  ```
  $ test/functional/test_runner.py --usecli > /dev/null 2>&1
  $ echo $?
  0
  $ test/functional/test_runner.py --usecli 2>&1 | cut -f2 -d'|' | \
      grep -E ' (Passed|Skipped) *$' | sort | uniq -c
       83  ✓ Passed
       52  ○ Skipped
  ```

  Context: `--usecli` was introduced in f6ade9ce1a

ACKs for top commit:
  laanwj:
    Code review ACK 5ac804a9eb

Tree-SHA512: 249c0b691a74cf201c729df86c3db2b3faefa53b94703941e566943d252c6d14924e935a8da4f592951574235923fbb7cd22612a5e7e02ff6c762c55a2320ca3
2019-12-09 13:52:47 -05:00
MarcoFalke
03e781ff4f Merge #17702: gui: Move static placeholder texts to forms
a652dc5521 qt: Normalize placeholder to avoid using "address book" in sendcoinsentry (Wladimir J. van der Laan)
67f36e0b2c gui: Move static placeholder texts to forms (Wladimir J. van der Laan)

Pull request description:

  There was an issue around the time of Qt 4.6 when placeholder text was introduced, that caused a compile failure when it was specified in the form.

  As a workaround the placeholder texts were moved to the code.

  Qt 4 hasn't been relevant to us for ages. So move all (non-parametrized) placeholder texts to the form files instead.

  It's better to keep this kind of text content together. Translate/no-translate status is kept as it is.

  Proof that they still work:
  ![win1](https://user-images.githubusercontent.com/126646/70428014-0e80b300-1a76-11ea-9a6d-be78a0bf14ed.png)

  ![win2](https://user-images.githubusercontent.com/126646/70428019-10e30d00-1a76-11ea-8016-ffa0c4eafe34.png)

  ![win3](https://user-images.githubusercontent.com/126646/70428021-13456700-1a76-11ea-9449-9413487e39f6.png)

  ![win4](https://user-images.githubusercontent.com/126646/70428025-150f2a80-1a76-11ea-92ad-be5f3c171c43.png)

ACKs for top commit:
  hebasto:
    Re-ACK a652dc5521, `tooltip` and `placeholderText` are identical now.
  MarcoFalke:
    ACK a652dc5521 🚿
  fanquake:
    ACK a652dc5521 - checked that placeholder text still appears.

Tree-SHA512: 7d3c1faeef2eb5d4b195d9d78f2a3f161296d869e5059b5e8d308167e3c6c668a3ebabec93dc592762ba15bfc86d51985e20c4e17f1065c8dce84fec036ff5ee
2019-12-09 13:50:38 -05:00
MarcoFalke
1d6fda65dd Merge #17678: depends: Support for S390X and POWER targets
11113247c3 depends: Support for S390X targets (MarcoFalke)
989fd539d5 depends: Support for 64-bit POWER targets (Luke Dashjr)

Pull request description:

  Failure before:

  ```
  $ make -C depends HOST=powerpc64-linux-gnu
  ...
  ERROR: Feature 'system-zlib' was enabled, but the pre-condition 'libs.zlib' failed.

  ERROR: Feature 'xcb' was enabled, but the pre-condition 'libs.xcb' failed.

  ERROR: Feature 'system-freetype' was enabled, but the pre-condition 'features.freetype && libs.freetype' failed.

  ERROR: Feature 'fontconfig' was enabled, but the pre-condition '!config.win32 && !config.darwin && features.system-freetype && libs.fontconfig' failed.
  make: *** [funcs.mk:254: /bitcoin/depends/work/build/powerpc64-linux-gnu/qt/5.9.8-95548079095/qtbase/.stamp_configured] Error 3

  $ make -C depends HOST=s390x-linux-gnu
  ...
  ERROR: Feature 'system-zlib' was enabled, but the pre-condition 'libs.zlib' failed.

  ERROR: Feature 'xcb' was enabled, but the pre-condition 'libs.xcb' failed.

  ERROR: Feature 'system-freetype' was enabled, but the pre-condition 'features.freetype && libs.freetype' failed.

  ERROR: Feature 'fontconfig' was enabled, but the pre-condition '!config.win32 && !config.darwin && features.system-freetype && libs.fontconfig' failed.
  make: *** [funcs.mk:254: /bitcoin/depends/work/build/s390x-linux-gnu/qt/5.9.8-79c6d6ca6ec/qtbase/.stamp_configured] Error 3
  ```

ACKs for top commit:
  laanwj:
    Code review ACK 11113247c3
  dongcarl:
    tested ACK 11113247c3
  practicalswift:
    ACK 11113247c3 -- diff looks correct

Tree-SHA512: f990101ced0ed579168bb25762c1296c9b512c597bab924013af41832670a69ed786c6ec9b654c95fe064187797880a66c575395bc102a914c1bdb323ca7538a
2019-12-09 12:48:56 -05:00
Wladimir J. van der Laan
ac09180128 Merge #17680: gitian: fixed SC2001 regex
c966ff14c7 gitian: fixed SC2001 regex (willyk)

Pull request description:

  Currently the gitian-win-signer.yml produces OUTFILE names without `-unsigned` stripped out
  This is due to regex having an`%` in front of it
  ```
  $ INFILE="bitcoin-0.19.0-win64-setup-unsigned.exe"
  $ echo "${INFILE/%-unsigned}"
  bitcoin-0.19.0-win64-setup-unsigned.exe
  $ echo "${INFILE/-unsigned}"
  bitcoin-0.19.0-win64-setup.exe
  ```

  Fixes #17361

ACKs for top commit:
  laanwj:
    ACK c966ff14c7
  hebasto:
    ACK c966ff14c7

Tree-SHA512: 954547f9dfa4cab4def5f284d4837c21f0e6fed7454a04e83e6b1b7d3fd3f9661ea657047f0d8162f6591909d32ef2f72e801b2f3a44cbb1131ac344cb913a69
2019-12-09 16:16:51 +01:00
Aaron Clauson
b0b1531737 Adds GitHub Action workflow which duplicates AppVeyor job. 2019-12-09 15:06:00 +00:00
fanquake
05c23488c2 Merge #17694: ui: disable 3rd-party tx-urls when wallet disabled
48a5c92f9e ui: disable 3rd-party tx-urls when wallet disabled (Harris)

Pull request description:

  This PR closes #17683 by removing 3rd-party Url-Label and -TextBox from Display Options in wallet-disabled mode.

ACKs for top commit:
  laanwj:
    Code review ACK 48a5c92f9e
  fanquake:
    ACK 48a5c92f9e - tested with and without wallet (compiled out and `-disablewallet`).

Tree-SHA512: 3cc89825409fc0a3eec501c4dab5ff1caaa4ce410746a4b6ab200222fff986f4483eab90cda53a98a144be6acf1b6ca8650ab18242c39446f3335b3a9a537066
2019-12-09 09:55:09 -05:00
fanquake
c8e65ade09 Merge #17695: gui: disable File->CreateWallet during startup
d65fafc2f7 gui: disable File->CreateWallet during startup (fanquake)

Pull request description:

  Same as #16118. Early calls to Create Wallet will crash bitcoin-qt.

  ```bash
  lldb /Applications/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt -- --regtest -debug

  Process 18143 launched: '/Users/michael/github/bitcoin/src/qt/bitcoin-qt' (x86_64)
  2019-12-07 15:49:37.823867-0500 bitcoin-qt[18143:5696499] MessageTracer: Falling back to default whitelist
  Process 18143 stopped
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x18)
      frame #0: 0x00000001000d2d9d bitcoin-qt`CreateWalletActivity::createWallet() + 381
  bitcoin-qt`CreateWalletActivity::createWallet:
  ->  0x1000d2d9d <+381>: movq   0x18(%rax), %r14
      0x1000d2da1 <+385>: movq   %r15, -0xa8(%rbp)
      0x1000d2da8 <+392>: leaq   -0xa0(%rbp), %r12
      0x1000d2daf <+399>: leaq   -0x80(%rbp), %rsi
  Target 0: (bitcoin-qt) stopped.
  (lldb) bt
  * thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x18)
    * frame #0: 0x00000001000d2d9d bitcoin-qt`CreateWalletActivity::createWallet() + 381
      frame #1: 0x0000000100833e6f bitcoin-qt`QMetaObject::activate(QObject*, int, int, void**) + 1631
      frame #2: 0x0000000100a1fc47 bitcoin-qt`QDialog::done(int) + 247
      frame #3: 0x0000000100833ef5 bitcoin-qt`QMetaObject::activate(QObject*, int, int, void**) + 1765
      frame #4: 0x00000001009e04c2 bitcoin-qt`QDialogButtonBoxPrivate::_q_handleButtonClicked() + 786
  ```

ACKs for top commit:
  jonasschnelli:
    utACK d65fafc2f7
  promag:
    ACK d65fafc2f7.

Tree-SHA512: 12d7f9e8772508bffbb0163849d9eceec5b1c80068c5d377a4d0973c713dc5f8ad38be8f793fec843d7fb604f0e60a72398b0c95f0a8b775dab39d25b29ac046
2019-12-09 09:35:33 -05:00
MarcoFalke
1ebf6402fc Merge #17647: lcov: filter depends from coverage reports
f736f6920b lcov: filter /usr/lib64 from coverage report (nijynot)
a5a705b46d lcov: filter depends from coverage report (nijynot)

Pull request description:

  If you build the binaries with the `depends` folder and then generate coverage reports with `make cov`, `depends` will be included in the coverage reports. Coverage of the dependencies are not that interesting and should be filtered.

ACKs for top commit:
  laanwj:
    code review ACK f736f6920b
  MarcoFalke:
    ACK f736f6920b 🐇

Tree-SHA512: 57c3e09f32e71523afff6ddc4f92bc35ab7b783f26f7a7380ae7556222954111cccce4c6dbc99305c424818f91e15bf5fe3532a7dca1daaa8ad71315d1dd857c
2019-12-09 08:38:42 -05:00
Wladimir J. van der Laan
a652dc5521 qt: Normalize placeholder to avoid using "address book" in sendcoinsentry 2019-12-09 12:28:55 +01:00
Wladimir J. van der Laan
67f36e0b2c gui: Move static placeholder texts to forms
There was an issue around the time of Qt 4.6 when placeholder text was
introduced, that caused a compile failure when it was specified in the
form.

As a workaround the placeholder texts were moved to the code.

Qt 4 hasn't been relevant to us for ages. So move all (non-parametrized)
placeholder texts to the form files instead.

It's better to keep this kind of text content together. Makes sure
translate/no-translate status is kept as it is.
2019-12-09 11:12:20 +01:00
Harris
48a5c92f9e ui: disable 3rd-party tx-urls when wallet disabled 2019-12-09 10:57:39 +01:00
Wladimir J. van der Laan
91559da683 Merge #17676: depends: don't use OpenGL in Qt on macOS
2359a4790d depends: don't use OpenGL in Qt on macOS (fanquake)
ba0cad2702 build: pass -dead_strip_dylibs to ld on macOS (fanquake)

Pull request description:

  Based on #17663. OpenGL on macOS was also deprecated in 10.14.

  This also removes the `/System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL` dylib from `bitcoin-qt`.

ACKs for top commit:
  laanwj:
    ACK 2359a4790d
  jonasschnelli:
    utACK 2359a4790d
  hebasto:
    ACK 2359a4790d

Tree-SHA512: 39b0151832c829f6ebdc4910eb28ebbeba64539cd04eba6ce3ec75fc0f231569956ca51a1e0bffc76dd27e85643c65a155320b9b450c49e9841e12b108406d41
2019-12-09 10:06:32 +01:00
Wladimir J. van der Laan
b6fb899ead Merge #17699: Make env data logging optional
0ccad08fb2 Make env data logging optional (Pieter Wuille)

Pull request description:

  The dynamic env feeding logging is a bit chatty, make it dependent on `-debug=rand`.

ACKs for top commit:
  practicalswift:
    ACK 0ccad08fb2 -- less noise is good and diff looks correct
  laanwj:
    ACK 0ccad08fb2
  promag:
    ACK 0ccad08fb2.
  jonatack:
    ACK 0ccad08fb2, was considering to propose this.

Tree-SHA512: 01d7f9ac134852c2c0d5f66f96ee4395f0ff7a60573e648f3d01054073624042148c8e8b9f69a29c9a41c296e1f4be77c2015a642ee4113a2fd8779b62aa137d
2019-12-09 10:02:42 +01:00
Pieter Wuille
0ccad08fb2 Make env data logging optional 2019-12-08 18:34:02 -08:00
nijynot
f736f6920b lcov: filter /usr/lib64 from coverage report 2019-12-09 00:14:53 +01:00
fanquake
e97f5c1823 depends: don't configure xcb_proto
xcb_proto's configure doesn't understand --disable-shared or
--with-pic. All the package does it put a stack of xml files into
a directory to be used by libxcb.
2019-12-08 17:55:11 -05:00
MarcoFalke
bb03765e2d Merge #17685: tests: Fix bug in the descriptor parsing fuzzing harness (descriptor_parse)
6338c02034 tests: Fix fuzzing harness for descriptor parsing (descriptor_parse) (practicalswift)

Pull request description:

  Fix bug in the descriptor parsing fuzzing harness (`descriptor_parse`) by making sure `secp256k1_context_verify` is properly initialized (via `ECCVerifyHandle`).

  Background:

  When fuzzing `Parse(…)` with `libFuzzer` I eventually reached the test case `combo(020000000000000000000000000000000000000000000000000000000000000000)`. That input triggers a call to `CPubKey::IsFullyValid()` which in turns requires an initialized `secp256k1_context_verify`.

  The fuzzing harness did not fulfil that pre-condition prior to this commit (sorry, my fault!) :)

  Before:

  ```
  $ mkdir descriptors/
  $ echo -n 'combo(020000000000000000000000000000000000000000000000000000000000000000)' > descriptors/input
  $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" src/test/fuzz/descriptor_parse -runs=1 descriptors/
  …
  pubkey.cpp:210:38: runtime error: null pointer passed as argument 1, which is declared to never be null
  secp256k1/include/secp256k1.h:305:3: note: nonnull attribute specified here
      #0 0x561c032ccf25 in CPubKey::IsFullyValid() const src/pubkey.cpp:210:12
      #1 0x561c022139c3 in (anonymous namespace)::ParsePubkeyInner(Span<char const> const&, bool, FlatSigningProvider&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/script/descriptor.cpp:674:24
      #2 0x561c02207680 in (anonymous namespace)::ParsePubkey(Span<char const> const&, bool, FlatSigningProvider&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/script/descriptor.cpp:730:42
      #3 0x561c0220080e in (anonymous namespace)::ParseScript(Span<char const>&, (anonymous namespace)::ParseScriptContext, FlatSigningProvider&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/script/descriptor.cpp:774:23
      #4 0x561c021ffb07 in Parse(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, FlatSigningProvider&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool) src/script/descriptor.cpp:994:16
      #5 0x561c0218d5d4 in test_one_input(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/test/fuzz/descriptor_parse.cpp:20:9
  …
  $
  ```

  After:

  ```
  $ mkdir descriptors/
  $ echo -n 'combo(020000000000000000000000000000000000000000000000000000000000000000)' > descriptors/input
  $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1" src/test/fuzz/descriptor_parse -runs=1 descriptors/
  …
  Done 2 runs in 0 second(s)
  $
  ```

ACKs for top commit:
  paymog:
    ACK 6338c02034
  MarcoFalke:
    ACK 6338c02034 🕊

Tree-SHA512: bf24c404e1f64183761b057d2f210c3db85277f4415122977c315d7d6835acb5e897b5d64032615e9e44ad4a16dfe857e94481f6e4b57b6dfa8cb37adb2528a5
2019-12-08 13:17:23 -05:00
Hennadii Stepanov
a82bd8fa57 util: Replace magics with DEFAULT_PRUNE_TARGET_GB
This commit does not change behavior.
2019-12-08 12:07:01 +02:00
fanquake
d65fafc2f7 gui: disable File->CreateWallet during startup 2019-12-07 15:32:05 -05:00
MarcoFalke
5622d8f315 Merge #17671: script: fixed wget call in gitian-build.py
b11d35b5e2 Fixed wget call in gitian-build.py (willyk)

Pull request description:

  Missing comma makes the gitian-builder script to download osslsigncode-2.0.tar.gz as osslsigncode-2.0.tar.gz-N,  which makes the subsequent calls fail when building window binaries

ACKs for top commit:
  laanwj:
    ACK b11d35b5e2
  promag:
    ACK b11d35b5e2.
  hebasto:
    ACK b11d35b5e2, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: a74f6bec9e405fe57c2e53cc4ea0cb3546521b295478d02ae81a36306262e349e14395e46fd8106e72cc6a5bb80a88c8eff1ea38816af49c8b169c5715e84b0d
2019-12-07 09:56:10 -05:00
Wladimir J. van der Laan
a8f5d5c6b9 Merge #17658: depends: add ability to skip building qrencode
c8becb8280 depends: add ability to skip building qrencode (fanquake)

Pull request description:

  Similar to other depends packages, add the ability to skip building `qrencode` by passing `NO_QR=1`. Same as #16089.

ACKs for top commit:
  promag:
    ACK c8becb8280.
  hebasto:
    ACK c8becb8280, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 86c7a87a31b1b2e65be2b79f533ce49f8b0074cf31331411cb3d32bb542d0b99e69605482ad75e4d1be5f2c8c613f17ba9ff17195a6b48f45365f5eb35df8bf9
2019-12-07 11:46:36 +01:00
MarcoFalke
fc0da027e8 Merge #17635: ci: Add CentOS 7 build
711e0449cf ci: Remove trusty build (Hennadii Stepanov)
7f3ae22468 ci: Add CentOS 7 build (Hennadii Stepanov)

Pull request description:

  Arguably, CentOS is the most conservative distro of all the popular ones. Thus, it could be a good way to check the Bitcoin Core compatibility with aged dependencies.

  Currently, CentOS 7 has:
  - Berkeley DB == 4.8.30
  - Boost == 1.53.0
  - GCC == 4.8.5
  - libevent == 2.0.21 < minimum required [2.0.22](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md), but tests passed
  - MiniUPnPc == 2.0
  - Python == 3.6.8
  - qrencode == 3.4.1
  - Qt == 5.9.7
  - ZeroMQ == 4.1.4

  ~Please note that this PR is based on the bugfix #17634.~

  Also trusty build has been removed for the following reasons:
  - https://github.com/bitcoin/bitcoin/issues/17628#issuecomment-559448201:
  > Maybe it'd make sense to replace Ubuntu Trusty with Centos 7 as the "check ancient backward compatibililty" Travis run. It's supported until 2024, apparently.

  - https://github.com/bitcoin/bitcoin/pull/17635#discussion_r354811792:
  > Our travis is currently running at its limit and this doesn't seem like it is adding a lot new coverage compared to the other builds.

  Close #17628

ACKs for top commit:
  MarcoFalke:
    ACK 711e0449cf 🚠

Tree-SHA512: 614ec8394943f482a5867067f7119bffd052924a51e32ffda9a08e10c392c4a955a3539e2f8907cb65bfd9347dadf0ba62f6d1530bbc49927c347360a5a7f73c
2019-12-06 23:32:21 -05:00
Harris
034561f9cd cli: fix Fatal LevelDB error when specifying -blockfilterindex=basic twice 2019-12-06 21:47:55 +01:00
Andrew Chow
7cecf10ac3 Replace LegacyScriptPubKeyMan::IsCrypted with LegacyScriptPubKeyMan::HasEncryptionKeys 2019-12-06 15:05:48 -05:00
Andrew Chow
bf6417142f Remove SetCrypted() and fUseCrypto; Change IsCrypted()'s implementation
Removes SetCrypted() and fUseCrypto as we don't need them anymore.
SetCrypted calls in LegacyScriptPubKeyMan are replaced with mapKeys.empty()

IsCrypted() is changed to just call HasEncryptionKeys()
2019-12-06 15:05:48 -05:00
Andrew Chow
77a777118e Rename EncryptKeys to Encrypt and pass in the encrypted batch to use 2019-12-06 15:05:48 -05:00
Andrew Chow
35f962fcf0 Clear mapKeys before encrypting
Does not change behavior. Needed to make AddCryptedKeyInner() work
with SetCrypted() being gone.
2019-12-06 15:05:48 -05:00
Andrew Chow
14b5efd66f Move fDecryptionThoroughlyChecked from CWallet to LegacyScriptPubKeyMan 2019-12-06 15:05:48 -05:00
Andrew Chow
97c0374a46 Move Unlock implementation to LegacyScriptPubKeyMan
CWallet::Unlock is changed to call ScriptPubKeyMan::CheckDecryptionKey
and the original implementation of Unlock is renamed to CheckDecryptionKey.
2019-12-06 15:05:47 -05:00
Andrew Chow
e576b135d6 Replace LegacyScriptPubKeyMan::vMasterKey with GetDecryptionKey() 2019-12-06 15:05:08 -05:00
Andrew Chow
fd9d6eebc1 Add GetEncryptionKey() and HasEncryptionKeys() to WalletStorage
Adds functions in WalletStorage that allow ScriptPubKeyMans to check
and get encryption keys from the wallet.
2019-12-06 15:05:08 -05:00
Hennadii Stepanov
711e0449cf ci: Remove trusty build
Co-authored-by: Emil Engler <me@emilengler.com>
2019-12-06 21:54:37 +02:00
Hennadii Stepanov
7f3ae22468 ci: Add CentOS 7 build 2019-12-06 21:44:35 +02:00
fanquake
4ee8a58ce7 Merge #17373: wallet: Various fixes and cleanup to keypool handling in LegacyScriptPubKeyMan and CWallet
886f1731be Key pool: Fix omitted pre-split count in GetKeyPoolSize (Andrew Chow)
386a994b85 Key pool: Change ReturnDestination interface to take address instead of key (Andrew Chow)
ba41aa4969 Key pool: Move LearnRelated and GetDestination calls (Andrew Chow)
65833a7407 Add OutputType and CPubKey parameters to KeepDestination (Andrew Chow)
9fcf8ce7ae Rename Keep/ReturnKey to Keep/ReturnDestination and remove the wrapper (Andrew Chow)
596f6460f9 Key pool: Move CanGetAddresses call (Andrew Chow)

Pull request description:

  * The `pwallet->CanGetAddresses()` call in `ReserveDestination::GetReservedDestination` to `LegacyScriptPubKeyMan::GetReservedDestination` so that the sanity check results in a failure when a `ScriptPubKeyMan` individually cannot get a destination, not when any of the `ScriptPubKeyMan`s can't.
  * `ScriptPubKeyMan::GetReservedDestination` is changed to return the destination so that future `ScriptPubKeyMan`s can return destinations constructed in other ways. This is implemented for `LegacyScriptPubKeyMan` by moving key-to-destination code from `CWallet` to `LegacyScriptPubKeyMan`
  * In order for `ScriptPubKeyMan` to be generic and work with future `ScriptPubKeyMan`s, `ScriptPubKeyMan::ReturnDestination` is changed to take a `CTxDestination` instead of a `CPubKey`. Since `LegacyScriptPubKeyMan` still deals with keys internally, a new map `m_reserved_key_to_index` is added in order to track the keypool indexes that have been reserved.
  * A bug is fixed in how the total keypool size is calculated as it was omitting `set_pre_split_keypool` which is a bug.

  Split from #17261

ACKs for top commit:
  ryanofsky:
    Code review ACK 886f1731be. Only change is moving earlier fix to a better commit (same end result).
  promag:
    Code review ACK 886f1731be.
  instagibbs:
    code review re-ACK 886f1731be
  Sjors:
    Code review re-ACK 886f1731be

Tree-SHA512: f4be290759f63fdc920d5c02bd0d09acc4b06a5f053787d4afcd3c921b2e35d2bd97617fadae015da853dc189f559fb8d2c6e58d53e4cabfac9af151cd97ad19
2019-12-06 13:37:30 -05:00
MarcoFalke
da1af855f9 Merge #17275: pubkey: Assert CPubKey's ECCVerifyHandle precondition
d8daa8f371 pubkey: Assert CPubKey's ECCVerifyHandle precondition (practicalswift)

Pull request description:

  Assert `CPubKey`'s `ECCVerifyHandle` precondition.

  This makes it more clear for fuzzing harness writers and others that `ECCVerifyHandle` is expected to be held when interacting with `CPubKey`.

  Related PR #17274.

ACKs for top commit:
  sipa:
    ACK d8daa8f371

Tree-SHA512: 9e74086599799dc9b5c3fb8357445b662e5bf896d826af63d6d6b6ddb616612966f3bb5de3bd3ae0e692c47de85672f64b8ab6d3a1c45899dc25ba46990b5ec7
2019-12-06 13:35:37 -05:00
practicalswift
6338c02034 tests: Fix fuzzing harness for descriptor parsing (descriptor_parse) 2019-12-06 18:25:51 +00:00
nijynot
a5a705b46d lcov: filter depends from coverage report 2019-12-06 19:14:15 +01:00
practicalswift
5ac804a9eb tests: Use a default of supports_cli=True (instead of supports_cli=False) 2019-12-06 15:24:06 +00:00
practicalswift
993e38a4e2 tests: Mark functional tests not supporting bitcoin-cli (--usecli) as such 2019-12-06 14:40:28 +00:00
practicalswift
709afb2a7d tests: Test serialisation as part of deserialisation fuzzing. Test round-trip equality where possible. Avoid code repetition. 2019-12-06 09:15:56 +00:00
practicalswift
597d10ceb9 tests: Add fuzzing harness for various functions consuming only integrals 2019-12-06 09:14:17 +00:00
practicalswift
575383b3e1 tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-06 09:14:03 +00:00
practicalswift
d5766f223f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-06 09:11:02 +00:00
practicalswift
e75ecb91c7 tests: Add fuzzing harness for various CTxOut related functions 2019-12-06 09:10:44 +00:00
practicalswift
ce935292c0 tests: Add fuzzing harness for various CTxIn related functions 2019-12-06 09:10:31 +00:00
Wladimir J. van der Laan
978b25528c util: Update tinyformat to upstream
Last update was in 2017.
Updates tinyformat to upstream commit 705e3f4e1de922069bf715746d35bd2364b1f98f.
Re-apply bitcoin core specific changes.

No changes that affect our use, as far as I can see, but this gets rid
of the gcc `-Wimplicit-fallthrough` warnings, at least.
2019-12-06 10:02:08 +01:00
Wladimir J. van der Laan
cb11324a63 Merge #17051: tests: Add deserialization fuzzing harnesses
897849d8c2 tests: Add deserialization fuzzing harnesses (practicalswift)
16f0a186dc tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)

Pull request description:

  Add deserialization fuzzing harnesses.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ contrib/devtools/test_fuzzing_harnesses.sh 'addr_info|block_file_info|block_filter|block_header|ext_key|ext_pub_key|fee_rate|flat_file|key_origin|merkle_block|mutable_transaction|out_point|partial_merkle_tree|partially_signed_transaction|prefilled_transaction|psbt_input|psbt_output|pub_key|script_deserialize|sub_net|tx_in' 10
  ```

  `test_fuzzing_harnesses.sh` can be found in PR #17000.

ACKs for top commit:
  laanwj:
    thanks, ACK 897849d8c2

Tree-SHA512: 5a270a3002cc23b725f7b35476a43777b2b00b4d089cc006372e2fcc7afa430afaa3c1430f778ae08fc53dd85a13e7bd2fab0449c319f676423226e189a417f6
2019-12-06 09:45:26 +01:00
Wladimir J. van der Laan
c7c9c44278 Merge #17663: build: pass -dead_strip_dylibs to ld on macOS
bd44711e1b build: pass -dead_strip_dylibs to ld on macOS (fanquake)

Pull request description:

  This strips some unused dylibs from bitcoin-qt.

  ```diff
  otool -L src/qt/bitcoin-qt
    /usr/lib/libSystem.B.dylib
  - /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
    /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
    /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
    /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
    /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
    /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/Security.framework/Versions/A/Security
    /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
    /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
    /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
  -/System/Library/Frameworks/AGL.framework/Versions/A/AGL
    /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
    /usr/lib/libc++.1.dylib
    /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
    /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
    /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
    /usr/lib/libobjc.A.dylib
  ```

  `AGL` - ObjC wrapper for OpenGL.
  `DiskArbitration` - mount/unmount notifications and events.
  `Security` - low level security operations, authentication services.

  From `man ld`:
  ```
  Remove dylibs that are unreachable by the entry point or exported symbols.
  That is, suppresses the generation of load command commands for dylibs
  which supplied no symbols during the link. This option should not be
  used when linking against a dylib which is required at runtime for
  some indirect reason such as the dylib has an important initializer.
  ```

ACKs for top commit:
  theuni:
    ACK bd44711e1b.

Tree-SHA512: 9592ce2966d28cb6c58e01efd401f56a4baa5dc5be5313f4fe8454632b578608be65a23c8602772049cd4655a9cb020fdd40d6622a244c301920d8c3db43f99a
2019-12-06 09:11:14 +01:00
Pieter Wuille
5909bcd3bf Add bounds checks in key_io before DecodeBase58Check 2019-12-05 16:31:09 -08:00
willyk
c966ff14c7 gitian: fixed SC2001 regex
the `-` is not a special symbol and should not have `%` in front of it.
2019-12-05 13:00:34 -08:00
MarcoFalke
cf43f3f0a8 Merge #17674: tests: Add initialization order fiasco detection in Travis
1f9d5af4f1 tests: Add initialization order fiasco detection in Travis (practicalswift)

Pull request description:

  Add initialization order fiasco detection in Travis :)

  Context: https://github.com/bitcoin/bitcoin/pull/17670#issuecomment-562035813

  This would have caught the `events_hasher` initialization order issue introduced in #17573  and fixed in #17670.

  Output in case of an initialization order fiasco:

  ```
  ==7934==ERROR: AddressSanitizer: initialization-order-fiasco on address 0x557098d79200 at pc 0x55709796b9a3 bp 0x7ffde524dc30 sp 0x7ffde524dc28
  READ of size 8 at 0x557098d79200 thread T0
      #0 0x55709796b9a2 in CSHA256::Finalize(unsigned char*) src/crypto/sha256.cpp:667:25
      #1 0x5570978150e9 in SeedEvents(CSHA512&) src/random.cpp:462:19
      #2 0x5570978145e1 in SeedSlow(CSHA512&) src/random.cpp:482:5
      #3 0x5570978149a3 in SeedStartup(CSHA512&, (anonymous namespace)::RNGState&) src/random.cpp:527:5
      #4 0x55709781102d in ProcRand(unsigned char*, int, RNGLevel) src/random.cpp:571:9
      #5 0x557097810d19 in GetRandBytes(unsigned char*, int) src/random.cpp:576:59
      #6 0x557096c2f9d5 in (anonymous namespace)::CSignatureCache::CSignatureCache() src/script/sigcache.cpp:34:9
      #7 0x557096511977 in __cxx_global_var_init.7 src/script/sigcache.cpp:67:24
      #8 0x5570965119f8 in _GLOBAL__sub_I_sigcache.cpp src/script/sigcache.cpp
      #9 0x557097bba4ac in __libc_csu_init (src/bitcoind+0x18554ac)
      #10 0x7f214b1c2b27 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:266
      #11 0x5570965347d9 in _start (src/bitcoind+0x1cf7d9)

  0x557098d79200 is located 96 bytes inside of global variable 'events_hasher' defined in 'random.cpp:456:16' (0x557098d791a0) of size 104
    registered at:
      #0 0x557096545dfd in __asan_register_globals compiler-rt/lib/asan/asan_globals.cpp:360:3
      #1 0x557097817f8b in asan.module_ctor (src/bitcoind+0x14b2f8b)

  SUMMARY: AddressSanitizer: initialization-order-fiasco src/crypto/sha256.cpp:667:25 in CSHA256::Finalize(unsigned char*)
  ```

ACKs for top commit:
  promag:
    Tested ACK 1f9d5af4f1, got
  MarcoFalke:
    ACK 1f9d5af4f1 👔

Tree-SHA512: f24ac0a313df7549193bd7f4fcfdf9b72bdfc6a6ee31d0b08e6d0752e5108fbd532106b6c86377ae0641258c9adb4921872e5d9a0154c0284e03315e0777102c
2019-12-05 15:56:14 -05:00
MarcoFalke
5d0b7f9e3d Merge #17670: Move events_hasher into RNGState()
8bda0960f9 Move events_hasher into RNGState() (Pieter Wuille)

Pull request description:

  This moves `events_hasher` and `events_mutex` into `RNGState()` in random.cpp. This guarantees (through the existing `GetRNGState()` function) that the mutex is always created before any events are added, even when that happens inside global initializers.

  Fixes the issue reported here: https://github.com/bitcoin/bitcoin/pull/17573#issuecomment-561828251, and includes the annotation from #17666).

ACKs for top commit:
  MarcoFalke:
    re-ACK 8bda0960f9 🥈
  sipsorcery:
    re-ACK 8bda0960f9.

Tree-SHA512: 78702d668764df19e9d61d87d82eca71cceca87d5351b740e13e732a1c18a3d53d7fbaaf63245266da597370bfebec9fa6a4749c15ec5a78dcfe6122c33553ed
2019-12-05 15:55:27 -05:00
MarcoFalke
910a4301b1 Merge #17522: test: Wait until mempool is loaded in wallet_abandonconflict
dddd09eb33 test: Wait until mempool is loaded in wallet_abandonconflict (MarcoFalke)

Pull request description:

  This might or might not fix intermittent issues such as https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/28724018#L4091

  I believe the mempool was not loaded fully after the restart, in which case it was not dumped either on the next restart. Thus, the previous mempool was attempted to be loaded a second time, which succeeded and contained the txs.

ACKs for top commit:
  laanwj:
    ACK dddd09eb33

Tree-SHA512: ab7061f946b5e5388f825dddceadb125f5197b24af3a7fcf1e700235d106a323419a56bfb4d84a2e27442e0de63e540c623b704343d83a98deaab3c02fcbdcbe
2019-12-05 15:52:17 -05:00
MarcoFalke
fa660d65d7 node: Use mempool from node context instead of global 2019-12-05 14:22:05 -05:00
MarcoFalke
facbaf092f rpc: Use mempool from node context instead of global
Currently they are identical, but in the future we might want to turn
the mempool into a unique_ptr. Replacing the global with the mempool
pointer from the node context simplifies this step.
2019-12-05 13:45:49 -05:00
Pieter Wuille
8bda0960f9 Move events_hasher into RNGState() 2019-12-05 09:49:11 -08:00
MarcoFalke
11113247c3 depends: Support for S390X targets 2019-12-05 12:12:46 -05:00
Luke Dashjr
989fd539d5 depends: Support for 64-bit POWER targets 2019-12-05 12:06:28 -05:00
Andrew Chow
6e77a7b65c keypool: Add comment about TopUp and when to use it 2019-12-05 11:51:52 -05:00
Wladimir J. van der Laan
6fff333c9f Merge #17507: random: mark RandAddPeriodic and SeedPeriodic as noexcept
55b2cb199c random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake)
461e547877 doc: correct random.h docs after #17270 (fanquake)

Pull request description:

  The usage of `MilliSleep()` in SeedPeriodic (previously SeedSleep) was
  [removed](d61f2bb076) in #17270, meaning it, and its users can now be marked `noexcept`.

  This also corrects the docs in random.h for some of the changes in #17270.

ACKs for top commit:
  practicalswift:
    ACK 55b2cb199c
  laanwj:
    ACK 55b2cb199c
  sipa:
    ACK 55b2cb199c

Tree-SHA512: 672d369796e7c4f9b4d98dc545e5454999fa1bef373871994a26041d6163c58909e2255e4f820d3ef011679aa3392754eb57477306a89f5fd3d57e2bd7f0811a
2019-12-05 15:14:42 +01:00
fanquake
2359a4790d depends: don't use OpenGL in Qt on macOS 2019-12-05 09:07:03 -05:00
fanquake
ba0cad2702 build: pass -dead_strip_dylibs to ld on macOS
This strips some unused dylibs from bitcoin-qt.

From man ld:
Remove dylibs that are unreachable by the entry point or exported symbols. 
That is, suppresses the generation of load command commands for dylibs 
which supplied no symbols during the link. This option should not be 
used when linking against a dylib which is required at runtime for 
some indirect reason such as the dylib has an important initializer.
2019-12-05 08:41:15 -05:00
practicalswift
1f9d5af4f1 tests: Add initialization order fiasco detection in Travis 2019-12-05 10:54:18 +00:00
willyk
b11d35b5e2 Fixed wget call in gitian-build.py 2019-12-05 02:21:09 -08:00
fanquake
5aee0e2163 Merge #17650: util: remove unwanted fields from bitcoin-cli -getinfo
01c8701559 util: remove unwanted fields from bitcoin-cli -getinfo (malevolent)

Pull request description:

  Removed the following fields from -getinfo: protocolversion, walletversion and keypoololdest. This change closes #17314 .

ACKs for top commit:
  laanwj:
    ACK 01c8701559
  achow101:
    ACK 01c8701559
  practicalswift:
    ACK 01c8701559 -- diff looks correct

Tree-SHA512: c98f2e8a3fee04d46766f70cb88f4e49e892a4424eff8940a7d48e9e808597b702427225788f984f5c3641591fd8d86acee56774afde1d57a4259c31d971ea08
2019-12-04 14:59:12 -05:00
MarcoFalke
fee01bb053 Merge #17517: ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le
fa40e48c50 ci: Remove unparseable lines from supp file for old xenial clang tsan (MarcoFalke)
fa1bfc476c ci: ubsan report_error_type=1 and add suppressions (MarcoFalke)
fa69cef13e test: Print stderr when subprocess fails (MarcoFalke)
2222c30586 test: Use char instead of unsigned char (MarcoFalke)
faa8023ce9 ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le (MarcoFalke)

Pull request description:

  Use clang-8 instead of default clang (which is clang-6 on Bionic) to avoid spurious segfaults when running the ci system on ppc64le

ACKs for top commit:
  practicalswift:
    ACK fa40e48c50 assuming Travis is happy -- diff looks correct :)

Tree-SHA512: f4f26232d3a0ef38da245869340f723d279a3db9823befbc735fb5a00024dae041c7306d7ae55d2488e6f86aa96cdea155b007aefb561fba505141e8dbc717dc
2019-12-04 13:23:16 -05:00
MarcoFalke
fa40e48c50 ci: Remove unparseable lines from supp file for old xenial clang tsan 2019-12-04 12:16:52 -05:00
MarcoFalke
fa1bfc476c ci: ubsan report_error_type=1 and add suppressions 2019-12-04 12:16:38 -05:00
MarcoFalke
fa69cef13e test: Print stderr when subprocess fails 2019-12-04 11:25:32 -05:00
MarcoFalke
2222c30586 test: Use char instead of unsigned char 2019-12-04 09:32:19 -05:00
MarcoFalke
faa8023ce9 ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le 2019-12-04 09:32:15 -05:00
Wladimir J. van der Laan
41919631d5 Merge #17573: Seed RNG with precision timestamps on receipt of net messages.
02d8c56a18 Seed RNG with precision timestamps on receipt of net messages. (Matt Corallo)

Pull request description:

  See title. Exposes a generic dead-simple "SeedEvent" interface, but currently just used for net messages.

ACKs for top commit:
  sipa:
    utACK 02d8c56a18
  laanwj:
    ACK 02d8c56a18
  meshcollider:
    utACK 02d8c56a18

Tree-SHA512: 28eb39a201ee2b13393c5c64dbf7c1913f3482f095969ef5141bfe549ce77dd63bb5f14738f6eedb296c686ea36014aa157b9c5e8059710a318590f30e9caa14
2019-12-04 13:12:44 +01:00
Wladimir J. van der Laan
dcbe024f5e Merge #17648: doc: rename wallet-tool references to bitcoin-wallet
e7ad4a2f8c doc: rename wallet-tool references to bitcoin-wallet (Wilson Ccasihue S)

Pull request description:

  Fix. text reference to executable bitcoin-wallet instead of wallet-tool, there is not a wallet-tool at bin/ folder.

ACKs for top commit:
  fanquake:
    ACK e7ad4a2f8c - thanks for following up.

Tree-SHA512: aed41b08947728a4ff3a97a62858ee7c86e2e5d57dcbbd0aab492dae3d8a548bb60541924e68cf3a0aa3d53d7db0012b489462b466919cd83f05b2aa88b7fff7
2019-12-04 11:00:40 +01:00
malevolent
01c8701559 util: remove unwanted fields from bitcoin-cli -getinfo
In accordance with #17314, Removing noisy fields from -getinfo. Fields removed: protocolversion, walletversion and keypoololdest. In addition to changing bitcoin-cli -getinfo, there is another change to test/functional/interface_bitcoin_cli.py. This change deletes tests that utilize removed -getinfo calls.
2019-12-04 06:48:53 +00:00
fanquake
bd44711e1b build: pass -dead_strip_dylibs to ld on macOS
This strips some unused dylibs from bitcoin-qt.

From man ld:
Remove dylibs that are unreachable by the entry point or exported symbols. 
That is, suppresses the generation of load command commands for dylibs 
which supplied no symbols during the link. This option should not be 
used when linking against a dylib which is required at runtime for 
some indirect reason such as the dylib has an important initializer.
2019-12-03 17:50:30 -05:00
fanquake
bce44089db Merge #17547: build: Fix configure report about qr
651c636f9e build: Fix configure report about qr (Hennadii Stepanov)

Pull request description:

  On master (b7bc9b8330):
  ```
  $ apt list libqrencode-dev
  Listing... Done
  libqrencode-dev/bionic 3.4.4-1build1 amd64
  $ ./configure | grep -i qr
  checking for QR... no
  checking whether to build GUI with support for QR codes... no
      with qr     = auto
  ```

  With this PR:
  ```
  $ apt list libqrencode-dev
  Listing... Done
  libqrencode-dev/bionic 3.4.4-1build1 amd64
  $ ./configure | grep -i qr
  checking for QR... no
  checking whether to build GUI with support for QR codes... no
      with qr     = no
  ```

ACKs for top commit:
  laanwj:
    Concept and light code review ACK 651c636f9e
  fanquake:
    ACK 651c636f9e

Tree-SHA512: 8959b1c7da5b28d06affcdd27ff4e455f1f7d9c8363dbde8ef07aaf79139ec8bc7ce25610b28e1d90c7e168573ee90ac9ab359bf10c667d0254507f8a880a935
2019-12-03 16:35:50 -05:00
fanquake
c8becb8280 depends: add ability to skip building qrencode 2019-12-03 15:16:04 -05:00
MarcoFalke
e354db7877 Merge #17637: script: Add Keyserver to verify-commits README
bd6a243075 script: Add Keyserver to verify-commits README (Emil Engler)

Pull request description:

  When I use the option with the default keyserver on `gpg (GnuPG) 2.2.12` from the Debian repositories only the keys from meshcollider and fanquake are actually found. Using the ubuntu keyserver works without any problems and all keys are getting found.
  As this keyserver is also suggested on [https://bitcoincore.org/en/download/](), it would be good to have a common keyserver.

ACKs for top commit:
  laanwj:
    ACK bd6a243075, better to be explicit here

Tree-SHA512: d4127c42490390bed4e0074ebb1f8a6cc983f96bbea0ec1da011a93bed325a7f465ed9936e4bc2ef7c4b2e4501a2868d00b67ceb7bf85f2a902db9c6173a6c61
2019-12-03 13:41:56 -05:00
MarcoFalke
54b12e425b Merge #17555: test: add unit test for non-standard txs with wrong nVersion
76303f65f9 test: add unit test for non-standard txs with wrong nVersion (Dominik Spicher)

Pull request description:

  Takes care of one of the missing cases of #17394: nVersion must be within the allowed range.

ACKs for top commit:
  instagibbs:
    ACK 76303f65f9

Tree-SHA512: 94464f781cf70a5616f7cea2014ae0a97a338c34411cc989c60389de2ce00368374811db78c919bda30e0ebf341fb830998a5e97c124dd8afc8feb726cedfd3a
2019-12-03 13:22:36 -05:00
MarcoFalke
1fdaa04ceb Merge #17641: Add unit test for leveldb creation with unicode path
70ed2ab7ef Add unit test for DB creation with unicode path (Aaron Clauson)

Pull request description:

  An issue arose when attempting to switch back to the main repo version of leveldb when the bitcoin data directory uses a unicode path. The leveldb windows file IO wrapper was using the *A ANSI win32 calls instead of the Unicode *W ones. This unit test will catch if the path created by leveldb doesn't match what we're expecting. For more info see https://github.com/google/leveldb/issues/755.

ACKs for top commit:
  laanwj:
    ACK 70ed2ab7ef

Tree-SHA512: fc6dbd3aa26a439016e63e8d4d931f218ce99094fc7887a13b54562ad4133047020288ecbcd622a8309f422ee1eda5df50bcb8c8e44442af36ed57b22c069004
2019-12-03 11:06:59 -05:00
MarcoFalke
1705f1949a Merge #17549: ci: misc cleanups
fad88e6f86 ci: Remove use of cd (MarcoFalke)
fa2941bbf4 ci: Remove unmaintained extended_lint (MarcoFalke)
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b6 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)

Pull request description:

  Some minor cleanups:

  * Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
  * Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
  * Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
  * Fixes #17178 by removing `cd`

Top commit has no ACKs.

Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
2019-12-03 11:02:06 -05:00
MarcoFalke
69ca6cdd66 Merge #17599: ci: Run functional tests on s390x
fabd71076c ci: Print free disk space (MarcoFalke)
fad9fdbea5 test: Properly deserialize integers in little-endian (MarcoFalke)
fa94fc10c8 ci: Run functional tests on s390x (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 98ba77eb56f283131fdaeb393fda86cc308f1bf9781e1e0e5736b8d616528dc8ff2e494d55ba107c138083025c66a59e382fcfa9962d4349a5fd6cbbc52484c3
2019-12-03 10:34:11 -05:00
MarcoFalke
2b6575d989 Merge #17643: wallet: Fix origfee return for bumpfee with feerate arg
02afb0c550 Fix origfee return for bumpfee with feerate arg (Gregory Sanders)

Pull request description:

  fixes https://github.com/bitcoin/bitcoin/issues/17642 and adds a simple test that would have caught it

ACKs for top commit:
  achow101:
    ACK 02afb0c550

Tree-SHA512: 303e392e05407f204dffe360689b5bb5dc77fd462dd0e489bc0b6c8f94f89ab7fe2bd8cb47e4dc6dc5c23a619826d15f3bf6b02b2c8e96402fbb51953c462e2d
2019-12-03 10:25:34 -05:00
Wilson Ccasihue S
e7ad4a2f8c doc: rename wallet-tool references to bitcoin-wallet 2019-12-02 12:06:35 -05:00
Andrew Chow
886f1731be Key pool: Fix omitted pre-split count in GetKeyPoolSize
This is a bugfix: https://github.com/bitcoin/bitcoin/pull/16341#discussion_r330669214
2019-12-02 11:57:46 -05:00
Andrew Chow
386a994b85 Key pool: Change ReturnDestination interface to take address instead of key
In order for ScriptPubKeyMan to be generic and work with future
ScriptPubKeyMans, ScriptPubKeyMan::ReturnDestination is changed to
take a CTxDestination instead of a CPubKey. Since LegacyScriptPubKeyMan
still deals with keys internally, a new map m_reserved_key_to_index is
added in order to track the keypool indexes that have been reserved.

The CPubKey argument of KeepDestination is also  removed so that it is
more generic. Instead of taking a CPubKey or a CTxDestination, we just use
the nIndex given to find the pubkey.
2019-12-02 11:57:46 -05:00
Andrew Chow
ba41aa4969 Key pool: Move LearnRelated and GetDestination calls
Addresses are determined by LegacyScriptPubKeyMan::GetReservedDestination
instead of ReserveDestination::GetReservedDestination as other ScriptPubKeyMan
implementations may construct addresses differently

This does not change behavior.
2019-12-02 11:57:20 -05:00
fanquake
35eda631ed Merge #17634: qt: Fix comparison function signature
98fbd1cdff Use correct C++11 header for std::swap() (Hennadii Stepanov)
b66861e2e5 Fix comparison function signature (Hennadii Stepanov)

Pull request description:

  This PR fixes build on CentOS 7 with GCC 4.8.5:

  ```
  ...
  In file included from /usr/include/c++/4.8.2/algorithm:62:0,
                   from ./serialize.h:11,
                   from ./qt/sendcoinsrecipient.h:13,
                   from ./qt/recentrequeststablemodel.h:8,
                   from qt/recentrequeststablemodel.cpp:5:
  /usr/include/c++/4.8.2/bits/stl_algo.h: In instantiation of ‘_RandomAccessIterator std::__unguarded_partition(_RandomAccessIterator, _RandomAccessIterator, const _Tp&, _Compare) [with _RandomAccessIterator = QList<RecentRequestEntry>::iterator; _Tp = RecentRequestEntry; _Compare = RecentRequestEntryLessThan]’:
  /usr/include/c++/4.8.2/bits/stl_algo.h:2296:78:   required from ‘_RandomAccessIterator std::__unguarded_partition_pivot(_RandomAccessIterator, _RandomAccessIterator, _Compare) [with _RandomAccessIterator = QList<RecentRequestEntry>::iterator; _Compare = RecentRequestEntryLessThan]’
  /usr/include/c++/4.8.2/bits/stl_algo.h:2337:62:   required from ‘void std::__introsort_loop(_RandomAccessIterator, _RandomAccessIterator, _Size, _Compare) [with _RandomAccessIterator = QList<RecentRequestEntry>::iterator; _Size = int; _Compare = RecentRequestEntryLessThan]’
  /usr/include/c++/4.8.2/bits/stl_algo.h:5499:44:   required from ‘void std::sort(_RAIter, _RAIter, _Compare) [with _RAIter = QList<RecentRequestEntry>::iterator; _Compare = RecentRequestEntryLessThan]’
  qt/recentrequeststablemodel.cpp:208:82:   required from here
  /usr/include/c++/4.8.2/bits/stl_algo.h:2263:35: error: no match for call to ‘(RecentRequestEntryLessThan) (RecentRequestEntry&, const RecentRequestEntry&)’
      while (__comp(*__first, __pivot))
                                     ^
  In file included from qt/recentrequeststablemodel.cpp:5:0:
  ./qt/recentrequeststablemodel.h:43:7: note: candidate is:
   class RecentRequestEntryLessThan
         ^
  qt/recentrequeststablemodel.cpp:217:6: note: bool RecentRequestEntryLessThan::operator()(RecentRequestEntry&, RecentRequestEntry&) const
   bool RecentRequestEntryLessThan::operator()(RecentRequestEntry &left, RecentRequestEntry &right) const
        ^
  qt/recentrequeststablemodel.cpp:217:6: note:   no known conversion for argument 2 from ‘const RecentRequestEntry’ to ‘RecentRequestEntry&’
  In file included from /usr/include/c++/4.8.2/algorithm:62:0,
                   from ./serialize.h:11,
                   from ./qt/sendcoinsrecipient.h:13,
                   from ./qt/recentrequeststablemodel.h:8,
                   from qt/recentrequeststablemodel.cpp:5:
  /usr/include/c++/4.8.2/bits/stl_algo.h:2266:34: error: no match for call to ‘(RecentRequestEntryLessThan) (const RecentRequestEntry&, RecentRequestEntry&)’
      while (__comp(__pivot, *__last))
                                    ^
  In file included from qt/recentrequeststablemodel.cpp:5:0:
  ./qt/recentrequeststablemodel.h:43:7: note: candidate is:
   class RecentRequestEntryLessThan
         ^
  qt/recentrequeststablemodel.cpp:217:6: note: bool RecentRequestEntryLessThan::operator()(RecentRequestEntry&, RecentRequestEntry&) const
   bool RecentRequestEntryLessThan::operator()(RecentRequestEntry &left, RecentRequestEntry &right) const
        ^
  qt/recentrequeststablemodel.cpp:217:6: note:   no known conversion for argument 1 from ‘const RecentRequestEntry’ to ‘RecentRequestEntry&’
    CXX      qt/qt_libbitcoinqt_a-sendcoinsentry.o
  make[2]: *** [qt/qt_libbitcoinqt_a-recentrequeststablemodel.o] Error 1
  ```

  Also for `std::swap()` header `<algorithm>` is replaced with `<utility>` one.
  Refs:
  - [`std::swap()`](https://en.cppreference.com/w/cpp/algorithm/swap)
  - [standard library header `<utility>`](https://en.cppreference.com/w/cpp/header/utility)

ACKs for top commit:
  promag:
    Code review ACK 98fbd1cdff.
  jonasschnelli:
    utACK 98fbd1cdff
  fanquake:
    ACK 98fbd1cdff

Tree-SHA512: 91324490c1bdb98f186d233418e7e72ae7bee507876e94fb8c038bee031cea9e1046900f21156da4b7c33abcd726796867b124c4132d9ae3759877e90a8527db
2019-12-02 11:24:52 -05:00
Gregory Sanders
02afb0c550 Fix origfee return for bumpfee with feerate arg 2019-12-01 20:54:16 -05:00
practicalswift
5db506ba59 tests: Add option --valgrind to run nodes under valgrind in the functional tests 2019-12-01 19:53:30 +00:00
fanquake
19698ac6bc Merge #17568: wallet: fix when sufficient preset inputs and subtractFeeFromOutputs
eadd1304c8 tests: Add a test for funding with sufficient preset inputs and subtractFeeFromOutputs (Andrew Chow)
ff330badd4 Default to bnb_used = false as there are many cases where BnB is not used (Andrew Chow)

Pull request description:

  #17290 introduced a bug where, when we had preset inputs that covered the amount being sent and subtractFeeFrromOutputs was being used, transaction funding would result in a `Fee exceeds maximum configured by -maxtxfee` error. This was happening because we weren't setting `bnb_used = false` when the preset inputs were used as it should have been. This resulted in a too high fee because the change would go to fees accidentally.

  Apparently this particular case doesn't have a test, so I've added one as well.

ACKs for top commit:
  Sjors:
    ACK eadd130. I can't get this new test to fail on macOS (without this PR). It passes whether or not I compile with `--enable-debug`. It does fail on Ubuntu. Yay undefined behavior... Anyway, it's a useful test.
  fanquake:
    ACK eadd1304c8
  instagibbs:
    utACK eadd1304c8

Tree-SHA512: 7286c321f78666eea558cc591174630d210263594df41cab1065417510591ee514ade0e1d0cec8af09a785757da68de82592b013e8fe8d4966cec3254368706e
2019-12-01 12:23:44 -05:00
fanquake
2ecb7e1556 Merge #17521: depends: only use D-Bus with Qt on linux
6fdf5dab26 depends: only use dbus with qt on linux (fanquake)

Pull request description:

  Since #8210 we've been passing `-dbus-runtime` when configuring Qt, however D-Bus isn't used on macOS or Windows. So rather than blanket passing `-dbus-runtime`, only use D-Bus when building for linux, and disable it for Windows and macOS. This also saves some time building qt in depends (for windows or macOS).

  This gist contains a diff of the symbols in a macOS bitcoin-qt after applying this change: https://gist.github.com/fanquake/317e5c9c7d1b5e37a0c1ce8001af18c4.

ACKs for top commit:
  laanwj:
    ACK 6fdf5dab26

Tree-SHA512: 7c7df6036f27dae6adb807edf94cd26b4dafa3728976d219a68f7388b6477777b35acebd507320e4469c9f2fcf016b311c82e0b12d50546cb5ab66a1e955e464
2019-11-30 23:16:48 -05:00
Aaron Clauson
70ed2ab7ef Add unit test for DB creation with unicode path
An issue arose when attempting to switch back to the main repo version of leveldb when the bitcoin data directory uses a unicode path. The leveldb windows file IO wrapper was using the *A ANSI win32 calls instead of the Unicode *W ones. This unit test will catch if the path created by leveldb doesn't match what we're expecting. For more info see https://github.com/google/leveldb/issues/755.
2019-11-30 21:31:46 +00:00
Emil Engler
bd6a243075 script: Add Keyserver to verify-commits README 2019-11-30 04:41:06 +01:00
Hennadii Stepanov
98fbd1cdff Use correct C++11 header for std::swap() 2019-11-29 21:23:25 +02:00
Hennadii Stepanov
b66861e2e5 Fix comparison function signature
This commit fixes build on CentOS 7 with GCC 4.8.5
2019-11-29 21:11:00 +02:00
Samuel Dobson
abb30de63f Merge #17587: gui: show watch-only balance in send screen
4a96e459d7 [gui] send: show watch-only balance in send screen (Sjors Provoost)
2689c8fd71 [test] qt: add send screen balance test (Sjors Provoost)

Pull request description:

  Now that we can create a PSBT from a watch-only wallet (#16944), we should also display the watch-only balance on the send screen.

  Before:
  <img width="1008" alt="before" src="https://user-images.githubusercontent.com/10217/69533384-030e9180-0f78-11ea-9748-c32c957e822e.png">

  After:
  <img width="1009" alt="Schermafbeelding 2019-11-26 om 11 44 17" src="https://user-images.githubusercontent.com/10217/69622879-19811f80-1042-11ea-8279-091012f39b38.png">

  I added a test to check the balance on the send screen, but it only covers regular wallets. A better would add a watch-only only wallet.

ACKs for top commit:
  meshcollider:
    utACK 4a96e459d7
  jb55:
    utACK 4a96e459d7
  promag:
    reACK 4a96e45, rebased and label change since last review.
  instagibbs:
    code review and light test ACK 4a96e459d7

Tree-SHA512: 4213549888bd309f72bdbba1453218f4a2b07e809100d786a3791897c75468f9092b06fe4b971942b1c228aa75ee7c04971f262ca9a478b42756e056eb534620
2019-11-29 23:16:08 +13:00
Wladimir J. van der Laan
114e89e596 Merge #17624: net: Fix an uninitialized read in ProcessMessage(…, "tx", …) when receiving a transaction we already have
73b96c94cb net: Fix uninitialized read in ProcessMessage(...) (practicalswift)

Pull request description:

  Fix an uninitialized read in `ProcessMessage(…, "tx", …)` when receiving a transaction we already have.

  The uninitialized value is read and used on [L2526 in the case of `AlreadyHave(inv) == true`](d8a66626d6/src/net_processing.cpp (L2494-L2526)).

  Proof of concept being run against a `bitcoind` built with MemorySanitizer (`-fsanitize=memory`):

  ```
  $ ./p2p-uninit-read-in-conditional-poc.py
  Usage: ./p2p-uninit-read-in-conditional-poc.py <dstaddr> <dstport> <net>
  $ bitcoind -regtest &
  $ ./p2p-uninit-read-in-conditional-poc.py 127.0.0.1 18444 regtest
  SUMMARY: MemorySanitizer: use-of-uninitialized-value
  [1]+  Exit 77                 bitcoind -regtest
  $
  ```

  Proof of concept being run against a `bitcoind` running under Valgrind (`valgrind --exit-on-first-error`):

  ```
  $ valgrind -q --exit-on-first-error=yes --error-exitcode=1 bitcoind -regtest &
  $ ./p2p-uninit-read-in-conditional-poc.py 127.0.0.1 18444 regtest
  ==27351== Conditional jump or move depends on uninitialised value(s)
  [1]+  Exit 1                  valgrind -q --exit-on-first-error=yes --error-exitcode=1 bitcoind -regtest
  $
  ```

  Proof of concept script:

  ```
  #!/usr/bin/env python3

  import sys

  from test_framework.mininode import NetworkThread
  from test_framework.mininode import P2PDataStore
  from test_framework.messages import CTransaction, CTxIn, CTxOut, msg_tx

  def send_duplicate_tx(dstaddr="127.0.0.1", dstport=18444, net="regtest"):
      network_thread = NetworkThread()
      network_thread.start()

      node = P2PDataStore()
      node.peer_connect(dstaddr=dstaddr, dstport=dstport, net=net)()
      node.wait_for_verack()

      tx = CTransaction()
      tx.vin.append(CTxIn())
      tx.vout.append(CTxOut())
      node.send_message(msg_tx(tx))
      node.send_message(msg_tx(tx))
      node.peer_disconnect()
      network_thread.close()

  if __name__ == "__main__":
      if len(sys.argv) != 4:
          print("Usage: {} <dstaddr> <dstport> <net>".format(sys.argv[0]))
          sys.exit(0)
      send_duplicate_tx(sys.argv[1], int(sys.argv[2]), sys.argv[3])
  ```

  Note that the transaction in the proof of concept is the simplest possible, but really any transaction can be used. It does not have to be a valid transaction.

  This bug was introduced in #15921 ("validation: Tidy up ValidationState interface") which was merged in to `master` 28 days ago.

  Luckily this bug was caught before being part of any Bitcoin Core release :)

ACKs for top commit:
  jnewbery:
    utACK 73b96c94cb
  laanwj:
    ACK 73b96c94cb, thanks for discovering and reporting this before it ended up in a release.

Tree-SHA512: 7ce6b8f260bcdd9b2ec4ff4b941a891bbef578acf4456df33b7a8d42b248237ec4949e65e2445b24851d1639b10681c701ad500b1c0b776ff050ef8c3812c795
2019-11-28 12:58:56 +01:00
Wladimir J. van der Laan
23cecd6cd5 Merge #17604: util: make ScheduleBatchPriority advisory only
d2a3a5cadb util: make ScheduleBatchPriority advisory only (fanquake)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK d2a3a5cadb

Tree-SHA512: 14e44360bc6b0c0bfd794cb8a744af7d64fb01aa5602fdb392d6c54799a721ef04426e8379b157dd40f2a33c0b6a5248b09d59c865c453ff1f6e3abbafff524e
2019-11-28 12:49:32 +01:00
Wladimir J. van der Laan
1f59885d27 Merge #17361: script: Lint Gitian descriptors with ShellCheck
17f81e9648 script: Enable SC2001 rule for Gitian scripts (Hennadii Stepanov)
61bb21b418 script: Enable SC2155 rule for Gitian scripts (Hennadii Stepanov)
577682d9e8 script: Enable SC2006 rule for Gitian scripts (Hennadii Stepanov)
14aded46df script: Lint Gitian descriptors with ShellCheck (Hennadii Stepanov)

Pull request description:

  This PR extracts shell scripts from Gitian descriptors (`contrib/gitian-descriptors/`) and checks for ShellCheck warnings as any other one.

  Some non-controversial warnings are fixed.

ACKs for top commit:
  practicalswift:
    ACK 17f81e9648 -- diff looks correct

Tree-SHA512: bdfa3d35bbb65ff634c90835d75c3df63e958b558599771d21366724f5cf64da83a68957d926e926a99c3704b9529e96a17697dc8d9ff3adf7154d9cb1999a8d
2019-11-28 10:18:22 +01:00
practicalswift
73b96c94cb net: Fix uninitialized read in ProcessMessage(...) 2019-11-27 21:27:56 +00:00
Hennadii Stepanov
17f81e9648 script: Enable SC2001 rule for Gitian scripts 2019-11-27 19:27:56 +02:00
Hennadii Stepanov
61bb21b418 script: Enable SC2155 rule for Gitian scripts
Also pwd command is replaced with $PWD variable everywhere for
consistency.
2019-11-27 19:25:29 +02:00
Andrew Chow
eadd1304c8 tests: Add a test for funding with sufficient preset inputs and subtractFeeFromOutputs 2019-11-26 13:02:46 -05:00
Andrew Chow
ff330badd4 Default to bnb_used = false as there are many cases where BnB is not used 2019-11-26 13:02:46 -05:00
Andrew Chow
65833a7407 Add OutputType and CPubKey parameters to KeepDestination
These need to be added so that LearnRelatedScripts can be called
from within KeepDestination later.
2019-11-26 11:52:51 -05:00
Andrew Chow
9fcf8ce7ae Rename Keep/ReturnKey to Keep/ReturnDestination and remove the wrapper
There is no reason to have Keep/ReturnDestination to be a wrapper for
Keep/ReturnKey. Instead just make them the same function.
2019-11-26 11:46:40 -05:00
Wladimir J. van der Laan
d8a66626d6 Merge #17283: rpc: improve getaddressinfo test coverage, help, code docs
33f5fc32e5 test: add rpc getaddressinfo labels test coverage (Jon Atack)
0f3539ac6d test: add listlabels test in wallet_labels.py (Jon Atack)
1388de8390 rpc: add getaddressinfo code documentation (Jon Atack)
2ee0cb3330 rpc: update getaddressinfo RPCExamples to bech32 (Jon Atack)
8d1ed0c263 rpc: clarify label vs labels in getaddressinfo RPCHelpman (Jon Atack)
5a0ed85070 rpc: improve getaddressinfo RPCHelpman content (Jon Atack)
70cda342cd rpc: improve getaddressinfo RPCHelpman formatting (Jon Atack)

Pull request description:

  This PR is a continuation of the work in https://github.com/bitcoin/bitcoin/pull/12892.

  Main motivations:
  - There is currently no test coverage for the getaddressinfo `labels` response. Coverage here is a prerequisite before deprecating the `label` response or adding multiple labels per address.
  - `bitcoin-cli help getaddressinfo` returns a few content errors, difficult-to-read formatting, and no explanation why it returns both `label` and `labels` and how they relate, which can be confusing for application developers.

  Changes by order of commits:
  - [x] improve/fix getaddressinfo RPCHelpman layout formatting
  - [x] improve/fix getaddressinfo RPCHelpman content
  - [x] clarify the `label` and `labels` fields in getaddressinfo RPCHelpman
  - [x] update getaddressinfo RPCExamples addresses to bech32
  - [x] add getaddressinfo code docs
  - [x] add a `listlabels` test assertion in wallet_labels.py
  - [x] add missing getaddressinfo `labels` test coverage and improve the existing `label` tests

  Here are gists of the CLI help output:
  [`bitcoin-cli help getaddressinfo` before this PR](https://gist.github.com/jonatack/022af5221a85c069780359a22643c810)
  [`bitcoin-cli help getaddressinfo` after this PR](https://gist.github.com/jonatack/4ee5f6abc62a3d99269570206a5f90ba)

  It seems we ought to begin a deprecation process for the getaddressinfo `label` field? If yes, I have a follow-up ready. _--> EDIT: Deprecation follow-ups #17578 and #17585 now build on this PR._

ACKs for top commit:
  fjahr:
    Re-ACK 33f5fc32e5
  jnewbery:
    ACK 33f5fc32e5.

Tree-SHA512: a001aa863090ec2566a31059477945b1c303ebeb430b33472f8b150e420fa5742fc33bca9d95571746395b607f43f6078dd5b53e238ac1f3fc648b51c8f79a07
2019-11-26 17:11:16 +01:00
fanquake
4fb82e916b Merge #17567: gui: remove macOS start on login code
27d82b63fb gui: remove macOS start on login code (fanquake)

Pull request description:

  The macOS startup item code was disabled for builds targeting macOS >
  `10.11` in #15208. Now that we require macOS `10.12` as a minimum (#17550),
  we can remove the startup item code entirely. The API we were using, `LSSharedFileListItemCopyResolvedURL`, `LSSharedFileListCopySnapshot` etc,
  was removed in macOS `10.12` SDK.

ACKs for top commit:
  jonasschnelli:
    utACK 27d82b63fb
  jonasschnelli:
    Tested ACK 27d82b63fb - successfully compiled on 10.15.1

Tree-SHA512: 7420757b91c7820e6a63280887155394547134a9cebcf3721af0284da23292627f94cd431241e033075b3fd86d79ace3ebf1b25d17763acbf71e07a742395409
2019-11-26 10:53:40 -05:00
fanquake
d2a3a5cadb util: make ScheduleBatchPriority advisory only 2019-11-26 09:55:07 -05:00
fanquake
f2ab130961 Merge #17606: qt, refactor: Use proper classes for Ui::*
93352d261f qt: Use proper class for Ui::ReceiveCoinsDialog (Hennadii Stepanov)
8781904643 qt: Fix class name of Ui::ModalOverlay (Hennadii Stepanov)

Pull request description:

  Use proper classes for:
  - `Ui::ModalOverlay` to remove `<customwidget>` entry
  - `Ui::ReceiveCoinsDialog` to be consistent with the code base

  This PR does not change behavior.

ACKs for top commit:
  jonasschnelli:
    Tested ACK 93352d261f - ran this on top of master and tested the modal overlay on initial mainnet sync.
  laanwj:
    code review ACK 93352d261f

Tree-SHA512: faeed8e86dbf5355505defcdb7e1db07d6a6005ee5eb07367b00f6aa122dd8ad34f8372d4bae7b29c0eac87b538a33157e19328be2876135e8a6376a3197f1bc
2019-11-26 09:48:52 -05:00
Sjors Provoost
4a96e459d7 [gui] send: show watch-only balance in send screen 2019-11-26 11:43:53 +01:00
Sjors Provoost
2689c8fd71 [test] qt: add send screen balance test 2019-11-26 11:38:32 +01:00
Samuel Dobson
0ee914ba9e Merge #17584: wallet: replace raw pointer with const reference in AddrToPubKey
1a3a256d5e wallet: replace raw pointer with const reference in AddrToPubKey (Harris)

Pull request description:

  This PR replaces a redundant reference-to-pointer conversion in **addmultisigaddress** from *wallet/rpcwallet.cpp*. It also makes the API from *rpc/util.h* look more straightforward as **AddrToPubKey** now uses const references like other functions from there.

  I am not sure why there is a ref-to-ptr conversion in addmultisignatures, so I can only speculate that this is because of "historical reasons".

  The ref-to-ptr conversion happens here: https://github.com/bitcoin/bitcoin/blob/master/src/wallet/rpcwallet.cpp#L1001

  There, the address of LegacyScriptPubKeyMan& is given to AddrToPubKey.

  Later, in AddrToPubKey, it gets converted back to a reference, because GetKeyForDestination in rpc/util.cpp expects a const ref: https://github.com/bitcoin/bitcoin/blob/master/src/rpc/util.cpp#L140

  Regards,

ACKs for top commit:
  achow101:
    ACK 1a3a256d5e
  meshcollider:
    utACK 1a3a256d5e
  promag:
    Code review ACK 1a3a256d5e.
  hebasto:
    ACK 1a3a256d5e, I have not tested the code, but I have reviewed it and it looks OK, I agree it can be merged.

Tree-SHA512: 1a2b8ddab5694ef4c65fac69f011e38dd03a634e84a35857e13bd05ad99fe42af22ee0af6230865e3d2c725693512f3336acb055ede19c958424283e7a3856c4
2019-11-26 21:26:32 +13:00
Hennadii Stepanov
93352d261f qt: Use proper class for Ui::ReceiveCoinsDialog 2019-11-26 02:31:28 +02:00
Hennadii Stepanov
8781904643 qt: Fix class name of Ui::ModalOverlay 2019-11-26 02:22:29 +02:00
MarcoFalke
fabd71076c ci: Print free disk space 2019-11-25 16:23:36 -05:00
MarcoFalke
fad9fdbea5 test: Properly deserialize integers in little-endian 2019-11-25 15:27:50 -05:00
MarcoFalke
fa94fc10c8 ci: Run functional tests on s390x 2019-11-25 15:07:28 -05:00
MarcoFalke
2c1c43754b Merge #17591: ci: Add big endian platform - s390x
da1f153e5e Add s390x tests to travis (Elichai Turkel)
2fa65e0de9 Add ci script to install on s390x (Elichai Turkel)

Pull request description:

  Discovered this as part of #17402 and a conversation with gmaxwell.

  You can see here that the platform is indeed BE: https://travis-ci.org/elichai/bitcoin/jobs/616656410#L36

  This closes https://github.com/bitcoin/bitcoin/issues/6466

ACKs for top commit:
  MarcoFalke:
    ACK da1f153e5e

Tree-SHA512: e7e94e54e220257d91b24fddc79eab2bcaaadf0b2d1e7e6872d9757808ab2541728f00b1f3ab7e343305c0e7d91bb48a17a3f9621f6fff6c9fe6cde6682de408
2019-11-25 14:57:40 -05:00
fanquake
12230529e3 Merge #17596: doc: Change doxygen URL to doxygen.bitcoincore.org
41d7db0b60 doc: Change doxygen URL to doxygen.bitcoincore.org (Wladimir J. van der Laan)

Pull request description:

  The bitcoin core doxygen documentation has moved to https://doxygen.bitcoincore.org, see bitcoin-core/bitcoincore.org#681

  (the old URL still works as a redirect)

ACKs for top commit:
  Sjors:
    ACK 41d7db0b60 based on a short spot check.
  fanquake:
    ACK 41d7db0b60 - also checked the redirect.

Tree-SHA512: e6fa0477b7825e3557c1b3bed8c5a37c33188ddcba43e6a19f95d86618408f7d04bbaeb64bd79181930c5af1252ca5c462e3f7a850bfffa39a8f62bcccbb4260
2019-11-25 14:20:23 -05:00
Wladimir J. van der Laan
41d7db0b60 doc: Change doxygen URL to doxygen.bitcoincore.org
The bitcoin core doxygen documentation has moved to
https://doxygen.bitcoincore.org, see
bitcoin-core/bitcoincore.org#681

(the old URL still works as a redirect)
2019-11-25 19:20:49 +01:00
Martin Zumsande
78e283e656 [test] move wallet helper functions into test library 2019-11-25 16:40:09 +01:00
Martin Zumsande
f613e5dfda [test] move mining helper functions into test library 2019-11-25 16:40:03 +01:00
MarcoFalke
89a1f7a250 Merge #17592: Appveyor install libevent[thread] vcpkg
f13e274b6f Appveyor install libevent[thread] vcpkg (Aaron Clauson)

Pull request description:

  As per #17586 the default libevent vcpkg install now has thread disabled. This PR installs libevent with the thread feature enabled.

Top commit has no ACKs.

Tree-SHA512: 5448113f0444170777400fef3582719845ca50d61d9382dfaacc55c43d477dd714456e38a3094e9b6858d93e84def11c2efa46902b52648c9f0c9362cc909147
2019-11-25 10:20:00 -05:00
Aaron Clauson
f13e274b6f Appveyor install libevent[thread] vcpkg
As per #17586 the default libevent vcpkg install now has thread disabled. This PR installs libevent with the thread feature enabled.
2019-11-25 13:35:36 +00:00
Elichai Turkel
da1f153e5e Add s390x tests to travis 2019-11-25 14:34:49 +02:00
Elichai Turkel
2fa65e0de9 Add ci script to install on s390x 2019-11-25 14:34:47 +02:00
Martin Zumsande
2cb4e8bdc7 [test] move string helper functions into test library 2019-11-25 01:33:17 +01:00
MarcoFalke
e6f167bfdf Merge #17569: build: Allow export of environ symbols and work around rv64 toolchain issue
eafd259367 build: Add NX workaround for RV64 (Wladimir J. van der Laan)
f6e42256fe build: Allow export of environ symbols (Wladimir J. van der Laan)

Pull request description:

  This export was introduced in #17270 which added
  ```
  //! Necessary on some platforms
  extern char** environ;
  ```
  This should (finally) make the gitian build pass again (fix issue #17525.).

  Built on top of #17538 which should be merged first.

Top commit has no ACKs.

Tree-SHA512: 5c2054d52d0957aec3dc945b76d8e219187d22dc03889e7a88fb76049bf8e4a3e9f4da00dd1e9dd0351211f8e70d1a1b8ad7244f0348dab698e9d14b9d0c0bd4
2019-11-24 19:07:56 -05:00
MarcoFalke
7463181798 Merge #17538: build: Bump minimum libc to 2.17 for release binaries
8f15a31760 doc: add glibc 2.17 requirement to release-notes (fanquake)
16a7be1663 build: Bump minimum versions in symbol checker (Wladimir J. van der Laan)
b77d5ad59f build: Disallow dynamic linking against c++ library (Wladimir J. van der Laan)

Pull request description:

  Closes: #17525. Taken over from #17531.

  Debian 8 (Jessie) has:
  - g++ version 4.9.2
  - libc version 2.19

  CentOS 7 has:
  - g++ version 4.8.5
  - libc version 2.17

  Ubuntu 16.04.4 (Xenial, oldest supported Ubuntu) has:
  - g++ version 5.3.1
  - libc version 2.23.0

  Taking the minimum of these as our target. According to [GNU ABI document](https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html) this corresponds to:

  - GCC 4.8.5: GCC_4.8.0
  - (glibc)    GLIBC_2_17

  This also contains a (long needed) commit to disallow dynamic linking to stdc++, as our releases statically link against that.

ACKs for top commit:
  laanwj:
    re-ACK 8f15a31760

Tree-SHA512: a3cc92aa1c5de253b1531f4b854d6f5f4a15d614ba6290d9db293542a96994b55c4a8e33e03b601bae16eb65529630b4f94b48b010e0b66b7dc9ff0acf945107
2019-11-24 18:42:54 -05:00
Jon Atack
33f5fc32e5 test: add rpc getaddressinfo labels test coverage 2019-11-24 23:08:38 +01:00
Jon Atack
0f3539ac6d test: add listlabels test in wallet_labels.py 2019-11-24 23:07:10 +01:00
Jon Atack
1388de8390 rpc: add getaddressinfo code documentation
and separate the fields with a line break for readability.
2019-11-24 23:07:07 +01:00
Jon Atack
2ee0cb3330 rpc: update getaddressinfo RPCExamples to bech32 2019-11-24 23:07:05 +01:00
Jon Atack
8d1ed0c263 rpc: clarify label vs labels in getaddressinfo RPCHelpman 2019-11-24 23:06:54 +01:00
Jon Atack
5a0ed85070 rpc: improve getaddressinfo RPCHelpman content 2019-11-24 23:05:48 +01:00
Harris
1a3a256d5e wallet: replace raw pointer with const reference in AddrToPubKey 2019-11-24 22:53:42 +01:00
Jon Atack
70cda342cd rpc: improve getaddressinfo RPCHelpman formatting 2019-11-24 16:09:51 +01:00
Wladimir J. van der Laan
239d199667 Merge #17574: doc: Add historical release notes for 0.19.0.1
14feda0814 doc: Add historical release notes for 0.19.0.1 (Wladimir J. van der Laan)

Pull request description:

  Add historical release notes for 0.19.0.1. And replace 0.19.0's release notes with a short explanation.

Top commit has no ACKs.

Tree-SHA512: 3551250c8b0643a96b645af9088ef07a079452bad3abf2b5653563f5ecbc07fc1e1c6adcd56eb9fa6e3e7637719b3b99253b95bce409a9267a428323b559bfa3
2019-11-24 14:24:15 +01:00
Wladimir J. van der Laan
eafd259367 build: Add NX workaround for RV64
Work around https://bugs.launchpad.net/ubuntu/+source/gcc-8-cross-ports/+bug/1853740.
2019-11-24 13:40:11 +01:00
Wladimir J. van der Laan
2eeacdfe44 Merge #17527: Fix CPUID subleaf iteration
f93fc61c65 Put bounds on the number of CPUID leaves explored (Pieter Wuille)
ba2c5fe147 Fix CPUID subleaf iteration (Pieter Wuille)

Pull request description:

  This fixes #17523.

  The code to determine which CPUID subleaves to explore was incorrect in #17270. The new code here is based on Intel's reference documentation for CPUID (a document called "Intel® Processor Identification and the CPUID Instruction - Application Note 485", which I cannot actually find on their own website).

ACKs for top commit:
  laanwj:
    ACK f93fc61c65
  jonatack:
    ACK f93fc61c65 code review, tested rebased on current master bb862d7 with Debian 4.19 x86_64
  mzumsande:
    ACK f93fc61, reviewed code and compared with the intel doc, tested on an AMD and an Intel processor.

Tree-SHA512: 2790b326fa397b736c0f39f25807bea57de2752fdd58bf6693d044b8cb26df36c11cce165a334b471f8e33724f10e3b76edab5cc4e0e7776601aabda13277245
2019-11-24 11:19:54 +01:00
Wladimir J. van der Laan
14feda0814 doc: Add historical release notes for 0.19.0.1
Add historical release notes for 0.19.0.1. And replace 0.19.0's release
notes with a short explanation.
2019-11-24 11:15:03 +01:00
Samuel Dobson
9cbd87d8ee Merge #17518: refactor, wallet: Nuke coincontrol circular dependency
3ed5e6819a refactor: Nuke coincontrol circular dependency (Hennadii Stepanov)

Pull request description:

  This PR gets rid of `wallet/coincontrol` -> `wallet/wallet` -> `wallet/coincontrol` circular dependency.

ACKs for top commit:
  Sjors:
    re-ACK 3ed5e6819a
  meshcollider:
    utACK 3ed5e6819a

Tree-SHA512: 7fbceb74a9cd04157170df158d2deb979cf397df818376b478224d2423f1d8504a8688e3a9b8fc527da73e4a34ab6bc4a98be0cc2937e102a063ab2ac553e86d
2019-11-24 18:26:17 +13:00
Matt Corallo
02d8c56a18 Seed RNG with precision timestamps on receipt of net messages. 2019-11-23 16:06:34 -05:00
fanquake
2c98e2024d Merge #17571: Add libtest_util library to msvc build configuration
b509554433 Added libtest_util library to msvc build configuration. (Aaron Clauson)

Pull request description:

  libtest_util was introduced in #17542. This PR adds the msvc build configuration.

ACKs for top commit:
  fanquake:
    ACK b509554433 - Appveyor looks good.

Tree-SHA512: abd9f8427375ae0e75e8227d853cccc666fd9e906038d97b787d9185dec1024232a6c64301e26e66fcaf86492183328abe4a7abd7af3321f062cd8722eb83b4c
2019-11-23 12:39:16 -05:00
fanquake
33c103e2fe Merge #17539: doc: Update and improve Developer Notes
794fe91395 doc: Update and improve Developer Notes (Hennadii Stepanov)

Pull request description:

  This PR:
  - removes outdated things, e.g., global pointer `pwalletMain` etc
  - adds "Sanitizers" to the TOC
  - makes filenames, `peer.dat` and `debug.log`, monospaced
  - specifies that _compile-time_ constant names are all uppercase
  - rewords using `explicit` with constructors

ACKs for top commit:
  jamesob:
    lazy ACK 794fe91395
  practicalswift:
    ACK 794fe91395 -- nice improvements!

Tree-SHA512: 2c5f035b1627f5fac8dc2453199d9e46bd101f86771de567cd95698de3c61cc459444ec1a68710e1d280195e1e40b42d9f40906297d12f12bf37749eca58297d
2019-11-23 11:30:15 -05:00
Aaron Clauson
b509554433 Added libtest_util library to msvc build configuration. 2019-11-23 15:49:16 +00:00
Wladimir J. van der Laan
f6e42256fe build: Allow export of environ symbols
This export was introduced in #17270 which added
```
//! Necessary on some platforms
extern char** environ;
```
2019-11-23 10:30:41 +01:00
Hennadii Stepanov
3ed5e6819a refactor: Nuke coincontrol circular dependency 2019-11-23 08:30:03 +02:00
Andrew Chow
ea50e34b28 keypool: Move opportunistic TopUps from LegacyScriptPubKeyMan to CWallet and ReserveDestination
An opportunistic TopUp is moved from LegacyScriptPubKeyMan::GetNewDestination
to CWallet::GetNewDestination. Another opportunistic TopUp is moved from
LegacyScriptPubKeyMan::ReserveKeyFromKeyPool (called by LegacyScriptPubKeyMan::GetReservedDestination)
to ReserveDestination::GetReservedDestination.

Moving opportunistic TopUps ensures that ScriptPubKeyMans will always
be topped up before requesting Destinations from them as we cannot
always rely on future ScriptPubKeyMan implementaions topping up internally.
As such, it is also unnecessary to keep the TopUp calls in the
LegacyScriptPubKeyMan functions so they are moved.

This does not change behavior as TopUp calls are moved up the call stack.
2019-11-22 23:45:34 -05:00
Andrew Chow
bb2c8ce23c keypool: Remove superfluous topup from CWallet::GetNewChangeDestination
This does not change behavior. This TopUp() is unnecessary as currently
m_spk_man calls TopUp further down the call stack inside
LegacyScriptPubKeyMan::ReserveKeyFromKeyPool (called by LegacyScriptPubKeyMan::GetReservedDestination)

By removing this here, we also prepare for future changes where CWallet
has multiple ScriptPubKeyMans instead of m_spk_man.
2019-11-22 23:45:34 -05:00
Andrew Chow
596f6460f9 Key pool: Move CanGetAddresses call
Call LegacyScriptPubKeyMan::CanGetAddresses directly instead of calling
CWallet::CanGetAddresses to only query the relevant key manager

This is a minor change in behavior: call now only happens if a new key needs to
be reserved, since if a key is already reserved it might fail unnecessarily.

This change also serves as a sanity check
https://github.com/bitcoin/bitcoin/pull/16341#discussion_r331238394
2019-11-22 22:41:27 -05:00
fanquake
27d82b63fb gui: remove macOS start on login code
The macOS startup item code was disabled for builds targeting macOS >
10.11 in #15208. Now that we require macOS 10.12 as a minimum, #17550,
we can remove the startup item code entirely, as the API we were using
was removed in macOS 10.12.
2019-11-22 18:44:43 -05:00
Samuel Dobson
0b79caf658 Merge #17447: wallet: Make -walletdir network only
3c2c439dcd wallet: Make -walletdir network only (João Barbosa)

Pull request description:

  With this PR `bitcoind -regtest` doesn't run if bitcoin.conf has
  ```
  walletdir=/mnt/mydisk/wallets
  ```
  But works with
  ```
  [regtest]
  walletdir=/mnt/mydisk/wallets
  ```

  Doesn't change mainnet behavior.

  Closes #15630.

ACKs for top commit:
  ryanofsky:
    ACK 3c2c439dcd
  MarcoFalke:
    ACK 3c2c439dcd 🍈
  meshcollider:
    Tested ACK 3c2c439dcd

Tree-SHA512: 8ab3b2db5f3f9cab78b36baaf490c80f7330372cfd8f73fe6536c8fb4c6e55e09f62296feb70617075838b3bcd7101abebbef3b228b6c3dbd42ce8c7a5c372d9
2019-11-23 10:36:04 +13:00
Samuel Dobson
4effd67bf4 Merge #17387: wallet_importmulti: use addresses of the same type as being imported
b84e776fd1 wallet_importmulti: use addresses of the same type as being imported (Andrew Chow)

Pull request description:

  When constructing an import from the solving data of an address, make sure that the original address is the same type as the one that will be imported.

  See also: https://github.com/bitcoin/bitcoin/pull/17374#issuecomment-550036931

  Part of #17261

ACKs for top commit:
  Sjors:
    Code review ACK b84e776
  meshcollider:
    Tested re-ACK b84e776fd1

Tree-SHA512: 53c49c63af8cbade0116a62beddc77df1a411d8ed76571c3053f6aff096f41a5325421a188bab3dcacfda69bb28fdff6ba921ddd80f29c4abbadb3b58fda884c
2019-11-23 10:20:04 +13:00
fanquake
8f15a31760 doc: add glibc 2.17 requirement to release-notes 2019-11-22 15:57:44 -05:00
Wladimir J. van der Laan
16a7be1663 build: Bump minimum versions in symbol checker
Debian 8 (Jessie) has:
- g++ version 4.9.2
- libc version 2.19

Ubuntu 16.04.4 (Xenial) has:
- g++ version 5.3.1
- libc version 2.23.0

CentOS 7 has:
- g++ version 4.8.5
- libc version 2.17

Taking the minimum of these as our target.
According to the GNU ABI document this corresponds to:

- GCC 4.8.5: GCC_4.8.0
- (glibc)    GLIBC_2_17

Co-Authored-By: fanquake <fanquake@gmail.com>
2019-11-22 15:40:05 -05:00
Wladimir J. van der Laan
b77d5ad59f build: Disallow dynamic linking against c++ library
Ever since statically linking Qt, we've been linking the C++ library
statically too (-static-libstdc++). Take this into
account in the symbol checker.
2019-11-22 15:38:52 -05:00
fanquake
a6f5b6f47d Merge #17550: build: set minimum supported macOS to 10.12
7d7bf2ff4a build: set minimum supported macOS to 10.12 (fanquake)

Pull request description:

  Extracted from #16392 as this doesn't need to wait for the other build changes.

  Reasoning:
  * `10.10` has been unsupported since July 2017 (~3 years at `v0.20.0` release)
  * `10.11` has been unsupported since July 2018  (~2 years at `v0.20.0` release)
  * macOS users are consistent at upgrading to new releases.
  * Qt 5.12 LTS only supports [macOS > 10.12](https://doc-snapshots.qt.io/qt5-5.12/supported-platforms.html). As long as we're supporting macOS < 10.12 we would not be able to bump Qt in depends to 5.12 for the `v0.20.0` release.

  Once we drop support for 10.12 and start using the 10.15 SDK there are some other follow ups:
  * Enabling support for [`thread_local`](https://github.com/bitcoin/bitcoin/blob/master/configure.ac#L901).
  * Removing some of the macOS notification code
  * Removing macOS [startup item code](https://github.com/bitcoin/bitcoin/blob/master/src/qt/guiutil.cpp#L695).

  There was also some related discussion in the `#bitcoin-builds` channel yesterday arvo.

ACKs for top commit:
  laanwj:
    ACK 7d7bf2ff4a
  dongcarl:
    post-IRC-grilling-ACK 7d7bf2ff4a

Tree-SHA512: 30b0ed75e3b4df2d3f94db2091b7982c1ea7be8fb3a2732d8efd1a53ef1e492d7265a47e90c38e92f66e638d10e9400a6ecb56e5093688bb26d1621645b453e8
2019-11-22 15:35:58 -05:00
Samuel Dobson
2a97d2b1a5 Merge #17553: wallet: Remove out of date comments for CalculateMaximumSignedTxSize
6a2e6b0600 Remove out of date comments for CalculateMaximumSignedTxSize (Gregory Sanders)

Pull request description:

  These paths can be hit for probably a number of reasons, and ISMINE spendability is not a requirement to call it.

  For example: During watch-only transaction creation, previous transaction in wallet, pubkey imported, but not the witnessscript associated with the prevout.

  In this case I think no/minimal comment is better than specific and soon to be out of date.

ACKs for top commit:
  achow101:
    ACK 6a2e6b0600
  darosior:
    ACK 6a2e6b0600

Tree-SHA512: ad4c26fd2409eb5aed19d67c19cb5479d226bd11e9298630309c4344f6562ace2e10c2850ebe22770331d71e91320a606e79619b9fe52dd478ce1f589a740122
2019-11-23 09:33:41 +13:00
Samuel Dobson
7127c31020 Merge #17237: wallet: LearnRelatedScripts only if KeepDestination
3958295bc8 wallet: LearnRelatedScripts only if KeepDestination (João Barbosa)
55295fba4c wallet: Lock address type in ReserveDestination (João Barbosa)

Pull request description:

  Only mutates the wallet if the reserved key is kept.

  First commit is a refactor that makes the address type a class member.

  The second commit moves `LearnRelatedScripts` from `GetReservedDestination` to `KeepDestination` to avoid an unnecessary call to `AddCScript` - which in turn prevents multiple entries of the same script in the wallet DB.

ACKs for top commit:
  achow101:
    Re-ACK 3958295bc8
  Sjors:
    ACK 3958295bc8
  ryanofsky:
    Code review ACK 3958295bc8. I like this change. The new behavior makes more sense, and the change makes the code clearer, since the current LearnRelatedScripts call is hard to understand and explain. (Personally, I'd like it if this PR were merged before #17373 or that PR was rebased on top of this one so it would be less confusing.)
  meshcollider:
    utACK 3958295bc8

Tree-SHA512: 49a5f4b022b28042ad37ea309b28378a3983cb904e234a25795b5a360356652e0f8e60f15e3e64d85094ea63af9be01812d90ccfc08ca4f1dd927fdd8566e33f
2019-11-23 09:26:58 +13:00
Samuel Dobson
0aa72061e5 Merge #16944: gui: create PSBT with watch-only wallet
c6dd565c88 [gui] watch-only wallet: copy PSBT to clipboard (Sjors Provoost)
39465d545d [wallet] add fillPSBT to interface (Sjors Provoost)
848f889208 [gui] send: include watch-only (Sjors Provoost)
40537f0909 [wallet] ListCoins: include watch-only for wallets without private keys (Sjors Provoost)

Pull request description:

  For wallets with `WALLET_FLAG_DISABLE_PRIVATE_KEYS` this makes the watch-only balance available on the send screen (including coin selection). Instead of sending a transaction it generates a PSBT.

  The user can take this PSBT and process it with [HWI](https://github.com/bitcoin-core/HWI) or put it an SD card for hardware wallets that support that.

  The PSBT is copied to the clipboard. This was the easiest approach; we can add a dialog later to display it, as well as an option to save to disk.

ACKs for top commit:
  instagibbs:
    test and code review ACK c6dd565c88
  meshcollider:
    re-ACK c6dd565c88

Tree-SHA512: ebc3da0737e33b255ed926191b84569aedb6097d14868662bd5dce726ce3048e86e9a31eba987b10dffe1482b35c21ae1cd595c2caa4634bc4cf78a826a83852
2019-11-23 09:22:02 +13:00
Samuel Dobson
8aac85d71e Merge #17371: Refactor: Require scriptPubKey to get wallet SigningProvider
d0dab897af Refactor: Require scriptPubKey to get wallet SigningProvider (Andrew Chow)
4b0c718f8f Accumulate result UniValue in SignTransaction (Andrew Chow)

Pull request description:

  Easier to review ignoring whitespace:

      git log -p -n1 -w

  This commit does not change behavior. It passes new CScript arguments to
  signing functions, but the arguments aren't currently used.

  Split from #17261

ACKs for top commit:
  instagibbs:
    utACK d0dab897af
  ryanofsky:
    Code review ACK d0dab897af. Thanks for the SignTransaction update. No other changes since last review
  Sjors:
    Code review ACK d0dab897af
  promag:
    Code review ACK d0dab897af.
  meshcollider:
    Code review ACK d0dab897af

Tree-SHA512: c3f52df20fd9d6b3b5aa65562cf5f7dce7b7f44c148b0f988f8b578fce2a28e9b7bf010f5f04bb5bf60f5272b2899f1dbbfb8aee81579c21c9cba559d1d2bb70
2019-11-23 08:35:10 +13:00
Samuel Dobson
cef87f7a48 Merge #17290: Enable BnB coin selection for preset inputs and subtract fee from outputs
b007efdf19 Allow BnB when subtract fee from outputs (Andrew Chow)
db15e71e79 Use BnB when preset inputs are selected (Andrew Chow)

Pull request description:

  Currently we explicitly disable BnB when there are preset inputs selected or when the subtract fee from outputs option is enabled. This PR enables BnB for both cases.

  Kind of an alternative to #17246 (implements the subtract fee from outputs part of it) and borrows a test from there too.

ACKs for top commit:
  instagibbs:
    reACK b007efdf19
  Sjors:
    re-ACK b007efdf19

Tree-SHA512: 933276b09b2fa2ab43db7f0b98762f06f6f5fa8606195f96aca9fa1cb71ae4ee7156028dd482b1cada82ddd0996a9daf12ea5c152589fdf192cd96cbc51e99df
2019-11-23 08:06:35 +13:00
MarcoFalke
fad88e6f86 ci: Remove use of cd 2019-11-22 13:57:55 -05:00
Marius Kjærstad
5ad4dd1ea1 doc: Changed MiniUPnPc link to https in dependencies.md
doc: Changed MiniUPnPc link to https in dependencies.md
2019-11-22 17:48:49 +01:00
Wladimir J. van der Laan
bb862d7864 Merge #14384: Fire TransactionRemovedFromMempool callbacks from mempool
e20c72f9f0 Fire TransactionRemovedFromMempool from mempool (251)

Pull request description:

  This pull request fires TransactionRemovedFromMempool callbacks from the mempool and cleans up a bunch of code.

  It also resolves the `txmempool -> validation -> validationinterface -> txmempool` circular dependency.

  Ideally, `validationinterface` is a dumb component that doesn't have any knowledge of the sub-systems it sends its notifications to. The commit that aims to resolve this circular dependency by moving `txmempool` specific code out of `validationinterface` to `txmempool` where it belongs.

ACKs for top commit:
  jnewbery:
    ACK e20c72f9f0

Tree-SHA512: 354c3ff1113b21a0b511d80d604edfe3846dddae3355e43d1387f68906e54bf5dc01e7c029edc0b8e635b500b2ab97ee50362e2486eb4319f7347ee9a9e6cef3
2019-11-22 16:35:08 +01:00
fanquake
03f6f408ab Merge #17545: build: remove libanl.so.1 from ALLOWED_LIBRARIES
ec89d2882a build: remove libanl.so.1 from ALLOWED_LIBRARIES (fanquake)

Pull request description:

  It should no longer be needed after: 10ae7a7b23.

  Symbol checker output for the `0.19.0.1` gitian built Linux binaries:
  ```bash
  aarch64  arm  i686-pc  risvc  symbol-check.py  x86_64
  root@557096f567b5:/test# find aarch64/ -type f -executable | xargs python3 symbol-check.py
  ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  ['libpthread.so.0', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-aarch64.so.1']
  root@557096f567b5:/test# find arm -type f -executable | xargs python3 symbol-check.py
  ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  ['libpthread.so.0', 'librt.so.1', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-armhf.so.3']
  root@557096f567b5:/test# find i686-pc -type f -executable | xargs python3 symbol-check.py
  ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  ['libpthread.so.0', 'librt.so.1', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux.so.2']
  root@557096f567b5:/test# find risvc/ -type f -executable | xargs python3 symbol-check.py
  ['libpthread.so.0', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1']
  ['libpthread.so.0', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-riscv64-lp64d.so.1', 'libatomic.so.1']
  root@557096f567b5:/test# find x86_64/ -type f -executable | xargs python3 symbol-check.py
  ['libpthread.so.0', 'librt.so.1', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  ['libpthread.so.0', 'librt.so.1', 'libfontconfig.so.1', 'libfreetype.so.6', 'libxcb.so.1', 'libdl.so.2', 'libm.so.6', 'libgcc_s.so.1', 'libc.so.6', 'ld-linux-x86-64.so.2']
  ```

ACKs for top commit:
  laanwj:
    ACK, good catch ec89d2882a

Tree-SHA512: 6bc118da190a5c37d26f0dfad5d4661add2ef15525668efb93425423bddbddabf3d7d8809464e79691f517fbe2aab241678652b3dc55ec3f452cf0dcbc26057c
2019-11-22 09:52:58 -05:00
MarcoFalke
b983e7e172 Merge #17542: build: Create test utility library from src/test/util/
a2e581de94 build: Create test utility library from src/test/util/ (Harris)

Pull request description:

  This PR creates a static **test utility library** that replaces repetitive compilations of sources from *src/test/util* in **unit**, **gui** and **bench** **tests**.

  The original issue is here: https://github.com/bitcoin/bitcoin/issues/17401

  The changes are:

  * a new *Makefile.test_util.include*
  * a new entry in *Makefile.am* that includes *Makefile.test_util.include* when testing is enabled
  * removal of all *src/test/util* headers & sources from unit, gui and bench Makefiles
  * addition of *libtest_util.a* at LDADD's of every test

ACKs for top commit:
  MarcoFalke:
    ACK a2e581de94 🍞

Tree-SHA512: d172127a26ee70d16625e17d7d94337a65472c57bb97f910c357c52d3dc082ea478ee586ee9074d9ebfeb05b75027e5e15f5bcd2aa35962dadfd9ac6bfd55ab9
2019-11-22 09:21:02 -05:00
Wladimir J. van der Laan
a739d207a3 Merge #17519: rpc: Remove unused COINBASE_FLAGS
e9a27cf338 refactor: Remove unused COINBASE_FLAGS (Neha Narula)

Pull request description:

  Commit d449772cf6 stopped setting
  COINBASE_FLAGS, and it looks like it hasn't been used since P2SH.

  Following up on #17489, remove COINBASE_FLAGS which is unused. I verified that removing this did not change the contents of the coinbase's scriptSig.

ACKs for top commit:
  laanwj:
    ACK e9a27cf338
  MarcoFalke:
    ACK e9a27cf338 💻

Tree-SHA512: f9dac124ce7e3edcae974137764bb5039387b1b123b86af44486e398aa4a8d91a9ecf640e207b364ae303acbbaee7cca300d303ea3d6869ba9cae2bf555a6334
2019-11-22 09:37:01 +01:00
Dominik Spicher
76303f65f9 test: add unit test for non-standard txs with wrong nVersion 2019-11-21 21:30:01 +01:00
Harris
a2e581de94 build: Create test utility library from src/test/util/ 2019-11-21 21:13:08 +01:00
251
e20c72f9f0 Fire TransactionRemovedFromMempool from mempool
This commit fires TransactionRemovedFromMempool callbacks from the
mempool and cleans up a bunch of code.
2019-11-21 21:05:38 +01:00
Gregory Sanders
6a2e6b0600 Remove out of date comments for CalculateMaximumSignedTxSize 2019-11-21 14:37:26 -05:00
Wladimir J. van der Laan
69a6f1ad1f Merge #17513: refactor, qt: Nuke some circular dependencies
5f50599ae7 refactor: Cleanup headers from walletmodel.h (Hennadii Stepanov)
a53e9895db refactor: Nuke walletmodel circular dependency (Hennadii Stepanov)
49c4211c04 refactor: Nuke walletmodeltransaction circular dep (Hennadii Stepanov)
567cb44eb9 refactor: Nuke guiutil circular dependency (Hennadii Stepanov)
73b5505cfe refactor: Move SendCoinsRecipient in own header (Hennadii Stepanov)

Pull request description:

  This PR gets rid of the following circular dependencies:
  - `qt/guiutil` -> `qt/walletmodel` -> `qt/optionsmodel` -> `qt/guiutil`
  - `qt/walletmodel` -> `qt/walletmodeltransaction` -> `qt/walletmodel`
  - `qt/paymentserver` -> `qt/walletmodel` -> `qt/paymentserver`

ACKs for top commit:
  Sjors:
    ACK 5f50599ae7
  instagibbs:
    code review ACK 5f50599ae7
  practicalswift:
    ACK 5f50599ae7 -- diff looks correct
  promag:
    ACK 5f50599ae7.

Tree-SHA512: 070686ac82b5c68c3ef1b8b4c16b4b916b84d80d1e92e42287fdd9454671bea54779c0d2db4db623750aaaf180beaba212137190d6a427113905e2c4be5c60c5
2019-11-21 19:38:39 +01:00
practicalswift
897849d8c2 tests: Add deserialization fuzzing harnesses 2019-11-21 17:53:06 +00:00
practicalswift
16f0a186dc tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-11-21 17:52:35 +00:00
fanquake
7d7bf2ff4a build: set minimum supported macOS to 10.12 2019-11-21 11:08:47 -05:00
MarcoFalke
fa2941bbf4 ci: Remove unmaintained extended_lint 2019-11-21 11:03:05 -05:00
MarcoFalke
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR
-BEGIN VERIFY SCRIPT-
 # Use ci DEPENDS_DIR
 sed -i -e 's|BASE_BUILD_DIR/depends|DEPENDS_DIR|g' $(git grep -l depends ci)
 sed -i -e 's| depends/| ${DEPENDS_DIR}/|g'         $(git grep -l depends ci)

 # Remove redundant alias
 sed -i -e 's|BASE_BUILD_DIR|BASE_ROOT_DIR|g'       $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-21 11:01:34 -05:00
MarcoFalke
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders 2019-11-21 11:01:17 -05:00
MarcoFalke
ae6943620a Merge #17407: node: Add reference to mempool in NodeContext
fa538813b1 scripted-diff: Replace ::mempool with m_node.mempool in tests (MarcoFalke)
8888ad02e2 test: Replace recursive lock with locking annotations (MarcoFalke)
fac07f2038 node: Add reference to mempool in NodeContext (MarcoFalke)

Pull request description:

  This is the first step toward making the mempool a global that is not initialized before main.

  #### Motivation

  Currently the mempool is a global that is initialized before the `main` function. This is confusing and easy to get wrong. E.g. the mempool constructor queries state that has not been initialized, like randomness (fixed), or command line arguments (not an issue last time I checked). Also without having the chainstate (chain tip) initialized first, it doesn't make conceptually sense to have a mempool, since the mempool builds txs on top of the utxo set (chain tip).

  Finally, in the future someone might want to run a consensus-only full node (`-nowallet -noblockfilter -no... -nomempool` command line options) that only verifies blocks and updates the utxo set.

  This is conceptually the same change that has already been done for the connection manager `CConnman`.

ACKs for top commit:
  jnewbery:
    utACK fa538813b1
  ariard:
    Tested ACK fa53881.

Tree-SHA512: 2c446a8a51476354aad7126c2b833500d36b24490caa94f847b2bdc622054de0dae28980f23e3d91b1b492dc32931656d98dbd019af9e4e58f2f8c5375aac694
2019-11-21 10:18:02 -05:00
MarcoFalke
5ff798c39b Merge #17439: refactor: Use proper MAX_SCRIPT_ELEMENT_SIZE constants consistently
cb9d830a00 test: Use proper MAX_SCRIPT_ELEMENT_SIZE (Hennadii Stepanov)
402ee706d8 refactor: Use proper MAX_SCRIPT_ELEMENT_SIZE const (Hennadii Stepanov)

Pull request description:

  This PR replaces well-known "magic" numbers with proper `MAX_SCRIPT_ELEMENT_SIZE` constants.

ACKs for top commit:
  practicalswift:
    ACK cb9d830a00 -- diff looks correct and change appears to be complete
  instagibbs:
    utACK cb9d830a00

Tree-SHA512: 5fa033275d6df7e35962c38bfdf09a7b5cd7ef2ccdd5e30a39ba47d0c21ac779a5559c23f5ef5bfd4293be0fc639e836a308bbedf0e34717e1eead983b389bbd
2019-11-21 10:11:06 -05:00
Sebastian Falbesoner
1bb5d517aa test: add unit test for non-standard bare multisig txs
The function IsStandardTx() returns rejection reason "bare-multisig" if the
transaction has a bare multisig output and the policy flag fIsBareMultisigStd
is false (set by the boolean command-line argument "-permitbaremultisig" -- for
the unit test, we simply set the global flag variable directly).
2019-11-21 16:05:39 +01:00
Hennadii Stepanov
651c636f9e build: Fix configure report about qr 2019-11-21 13:52:41 +02:00
MarcoFalke
faeeca87b6 scripted-diff: Move various folders to ci scratch dir
-BEGIN VERIFY SCRIPT-
 # move ci sanitizer-output
 sed -i -e 's|BASE_BUILD_DIR}/sanitizer-output|BASE_SCRATCH_DIR}/sanitizer-output|g' $(git grep -l BASE_BUILD_DIR ci)
 # move qa-assets
 sed -i -e 's|BASE_BUILD_DIR}/qa-assets|BASE_SCRATCH_DIR}/qa-assets|g'               $(git grep -l BASE_BUILD_DIR ci)
 # move out dir
 sed -i -e 's|BASE_BUILD_DIR/out|BASE_SCRATCH_DIR/out|g'                             $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-20 19:56:26 -05:00
Neha Narula
e9a27cf338 refactor: Remove unused COINBASE_FLAGS
Commit d449772cf6 stopped setting
COINBASE_FLAGS, and it looks like it hasn't been used since P2SH.
Update the help string to remove "flags", which is not specified in
BIP 22.
2019-11-20 19:06:52 -05:00
MarcoFalke
b7bc9b8330 Merge #17444: wallet: Avoid showing GUI popups on RPC errors (take 2)
faffa7f0dc wallet: Avoid showing GUI popups on RPC errors (take 2) (MarcoFalke)

Pull request description:

  Commit 8b0d82bb42 claims "This commit does not change behavior." However, it re-introduced the bug I tried to fix in #17070

ACKs for top commit:
  ryanofsky:
    Code review ACK faffa7f0dc

Tree-SHA512: 99987f80c76414dca40c7d76b2fe4ea853debbe3c49e7acdeab2596c726a2935c468f4484d49212e65ecc9c8b0d861c0c2b83c1ddfc07670540699199dbfecb0
2019-11-20 16:53:18 -05:00
MarcoFalke
41162c0ba0 Merge #17423: ci: Make ci system read-only on the git work tree
fa7523d3aa ci: Extend docs (MarcoFalke)
fa493ef088 ci: Make ci system read-only on the git work tree (MarcoFalke)
fab1333292 ci: Remove git from required packages on host (MarcoFalke)
fa00393bce ci: Make all filesystem operations inside docker (MarcoFalke)

Pull request description:

  Running the ci completely in a docker, without leaving any traces on the host system is not possible right now because the ccache and depends dir needs to be propagated back and picked up by the host for caching.

  Fixes #17372

ACKs for top commit:
  JeremyRubin:
    tested ACK fa7523d3aa

Tree-SHA512: 4bce1a0f883bcbdb34abf409bdbc80d420c5da2045d2f9c5536ac433f9e5b490f23df084546c8c049f688b487572bbfc4f9c4029e9e672f4d9279739d066ed2e
2019-11-20 16:41:55 -05:00
Sebastian Falbesoner
1be0b1fb2a test: add functional test for non-standard bare multisig txs
A transaction is rejected by the mempool with reason "bare-multisig" if any of
the outputs' scriptPubKey has bare multisig format (M <PubKey1> <PubKey2> ...
<PubKeyN> N OP_CHECKSIG) and bitcoind is started with "-permitbaremultisig=0".
2019-11-20 21:40:53 +01:00
fanquake
ec89d2882a build: remove libanl.so.1 from ALLOWED_LIBRARIES 2019-11-20 15:05:35 -05:00
Pieter Wuille
f93fc61c65 Put bounds on the number of CPUID leaves explored 2019-11-20 10:54:08 -08:00
Hennadii Stepanov
794fe91395 doc: Update and improve Developer Notes 2019-11-20 20:30:39 +02:00
MarcoFalke
a8d9f7dfa7 Merge #17420: travis: Rework CACHE_ERR_MSG
9999a79074 travis: Rework CACHE_ERR_MSG (MarcoFalke)

Pull request description:

  This should fix the misconception that we can fix build problems in forked repos. See https://github.com/bitcoin/bitcoin/pull/17418#issuecomment-551917920

Top commit has no ACKs.

Tree-SHA512: e0c86f6613fa38f4d4984420fa1fdc999829407c71024c3b92fd1d35afa98f4b5073af72656c83f607d5f6fab15f9ea629ca8be0b4e892c2fb65a226036c470a
2019-11-20 12:47:27 -05:00
Andrew Chow
b007efdf19 Allow BnB when subtract fee from outputs 2019-11-20 12:12:01 -05:00
Andrew Chow
db15e71e79 Use BnB when preset inputs are selected 2019-11-20 12:12:01 -05:00
fanquake
3671c5721d Merge #17532: test: add functional test for non-standard txs with too large scriptSig
8f2d7737cc test: add functional test for non-standard txs with too large scriptSig (Sebastian Falbesoner)

Pull request description:

  Approaches another missing functional test of issue #17394 (counterpart to unit test in PR #17480, Commit 5e8a56348b): A transaction is rejected by the mempool with reason `"scriptsig-size"` if any of the inputs' scriptSig is larger than 1650 bytes.

ACKs for top commit:
  MarcoFalke:
    ACK 8f2d7737cc
  instagibbs:
    ACK 8f2d7737cc

Tree-SHA512: 7a45b8a4181158be3e3b91756783ddf032f132ca8780dc35fac91b2df2149268f784d28ac56005135c4d86a357c57805c5a54b8155f0d049932844b18dc03992
2019-11-20 11:29:27 -05:00
fanquake
6fdf5dab26 depends: only use dbus with qt on linux 2019-11-20 10:01:29 -05:00
fanquake
76e777df83 Merge #16161: util: Fix compilation errors in support/lockedpool.cpp
30fb598737 Fix segfault in allocator_tests/arena_tests (Jeffrey Czyz)
15c84f53f4 Define ARENA_DEBUG in Travis test runs (Jeffrey Czyz)
ad71548822 Fix compilation errors in support/lockedpool.cpp (Jeffrey Czyz)

Pull request description:

  Changes in #12048 cause a compilation error in Arena::walk() when
  ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
  changed to have a different value type.

  Additionally, missing includes cause other compilation errors when
  ARENA_DEBUG is defined.

  Reproduced with:

  make CPPFLAGS=-DARENA_DEBUG

ACKs for top commit:
  laanwj:
    ACK 30fb598737
  fanquake:
    ACK 30fb598737 - thanks for following up jkczyz.

Tree-SHA512: 4eec368a4e9c67e4e2a27bc05608a807c2892d50c60d06ed21490cd274c0369f9671bc05b3006acc2a193316caf4896454c9c299603bfed29bd488f1987ec446
2019-11-20 09:54:31 -05:00
Wladimir J. van der Laan
26744ae189 Merge #17446: gui: Changed tooltip for 'Label' & 'Message' text fields to be more clear
8944c1d340 Changed tooltips of receive form to highlight difference between Label and Message (dannmat)

Pull request description:

  I have changed the tooltips for 'Label' & 'Message' text fields to be more clear, stating the difference between the two (#17173)

ACKs for top commit:
  MarcoFalke:
    ACK 8944c1d340
  laanwj:
    ACK 8944c1d340

Tree-SHA512: 7fbea4d3c4416264ae6c146d51d29958c418a278bdd6744133db0b684ad7a9413178c005592aa21a81d127f3f3a8583fc5de00078239db08e6f101f657a5dd3a
2019-11-20 14:25:43 +01:00
Wladimir J. van der Laan
36191a8bb5 Merge #12461: scripted-diff: Rename key size consts to be relative to their class
0580f86bb4 Fixup whitespace (Ben Woosley)
47101bbb27 scripted-diff: Rename CPubKey and CKey::*_KEY_SIZE and COMPRESSED_*_KEY_SIZE (Ben Woosley)

Pull request description:

  ~~And introduce CPubKeySig to host code relative to key sigs.~~

ACKs for top commit:
  meshcollider:
    utACK 0580f86bb4

Tree-SHA512: 29aa0be54912358b138e391b9db78639786f56580493e590ec9f773c0e1b421740133d05a79be247c7ee57e71c9c9e41b9cb54088cb3c0e3f813f74f0895287b
2019-11-20 12:43:55 +01:00
Wladimir J. van der Laan
6496bb87af Merge #17008: build: bump libevent to 2.1.11 in depends
02ac445b2f bump libevent to 2.1.11 in depends (stefanwouldgo)

Pull request description:

  this doesn't need patches on Android anymore like 2.1.8 did.

ACKs for top commit:
  laanwj:
    ACK 02ac445b2f

Tree-SHA512: 1fbfe342ee15fa4c5cb417979bd6c443f7c7aa40a489accf8ccd7c919e5b08e859b3da6edeee3de484f6f156b35dd4e97c7e2c7971b59fc31029865585ccb296
2019-11-20 12:27:10 +01:00
Sebastian Falbesoner
8f2d7737cc test: add functional test for non-standard txs with too large scriptSig
A transaction is rejected by the mempool with reason "scriptsig-size" if any of
the inputs' scriptSig is larger than 1650 bytes.
2019-11-20 11:01:27 +01:00
Pieter Wuille
2bcf1fc444 Pass a maximum output length to DecodeBase58 and DecodeBase58Check
Also remove a needless loop in DecodeBase58 to prune zeroes in the base256
output of the conversion. The number of zeroes is implied by keeping track
explicitly of the length during the loop.
2019-11-19 15:38:27 -08:00
Pieter Wuille
ba2c5fe147 Fix CPUID subleaf iteration 2019-11-19 14:56:23 -08:00
Andrew Chow
773d4572a4 Mark PSBTs spending unspendable outputs as invalid in analysis 2019-11-19 14:54:13 -05:00
Andrew Chow
638e40cb60 Have a PSBTAnalysis state that indicates invalid PSBT
Invalid PSBTs need to be re-created, so the next role is the
Creator (new PSBTRole). Additionally, we need to know what went
wrong so an error field was added to PSBTAnalysis.

A PSBTAnalysis indicating invalid will have empty everything,
next will be set to PSBTRole::CREATOR, and an error message.
2019-11-19 14:54:08 -05:00
MarcoFalke
dddd09eb33 test: Wait until mempool is loaded in wallet_abandonconflict 2019-11-19 14:29:43 -05:00
fanquake
b4a1da9ef8 Merge #17515: Remove straggling OpenSSL references from doc and build
ea3c7e585c test: Remove libssl-dev packages from CI scripts (Wladimir J. van der Laan)
7ea55264b9 test: remove lsan suppression for libcrypto (Wladimir J. van der Laan)
2d7066527a build: remove libcrypto as internal dependency in libbitcoinconsensus.pc (Wladimir J. van der Laan)
278751ea11 doc: Remove ssl as a required dependency from build-unix (Wladimir J. van der Laan)

Pull request description:

  Some doc and build cleanups following #17265.

  I intentionally left the libssl-dev install in `gitian-win-signer.yml`, as it's necessary for the ossl signer.

ACKs for top commit:
  MarcoFalke:
    ACK ea3c7e585c 🗯
  jamesob:
    ACK ea3c7e585c
  practicalswift:
    ACK ea3c7e585c - nice!
  fanquake:
    ACK ea3c7e585c - thanks.

Tree-SHA512: 67ea35bdd6d6e512d69e6734713534c88cae033a2ed695677ea15c3e3d5ff570374e342775c88e60877fa43a19047853e7b2a433e2c9a4349a5c423726a7457e
2019-11-19 11:38:06 -05:00
MarcoFalke
e1df922132 Merge #17497: test: skip tests when utils haven't been compiled
a67352161c test: skip tool_wallet test when bitcoin-wallet isn't compiled (fanquake)
e9277baed6 test: skip wallet_listreceivedby test when the cli isn't compiled (fanquake)
621d398750 test: skip bitcoin_cli test when the cli isn't compiled (fanquake)

Pull request description:

  Don't try and run the `interface_bitcoin_cli.py` test when `bitcoin-cli` isn't available.

  ```bash
  stdout:
  2019-11-17T01:51:41.623000Z TestFramework (INFO): Initializing test directory /var/folders/z2/cn877pxd3czdfh47mfkmbwgm0000gn/T/test_runner_₿_🏃_20191116_205141/interface_bitcoin_cli_0
  2019-11-17T01:51:41.890000Z TestFramework (ERROR): Unexpected exception caught during testing
  Traceback (most recent call last):
    File "/Users/michael/github/bitcoin/test/functional/test_framework/test_framework.py", line 111, in main
      self.run_test()
    File "/Users/michael/github/bitcoin/test/functional/interface_bitcoin_cli.py", line 18, in run_test
      cli_response = self.nodes[0].cli("-version").send_cli()
    File "/Users/michael/github/bitcoin/test/functional/test_framework/test_node.py", line 528, in send_cli
      process = subprocess.Popen(p_args, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)
    File "/Users/michael/.pyenv/versions/3.5.6/lib/python3.5/subprocess.py", line 676, in __init__
      restore_signals, start_new_session)
    File "/Users/michael/.pyenv/versions/3.5.6/lib/python3.5/subprocess.py", line 1289, in _execute_child
      raise child_exception_type(errno_num, err_msg)
  FileNotFoundError: [Errno 2] No such file or directory: '/Users/michael/github/bitcoin/src/bitcoin-cli'
  ```

Top commit has no ACKs.

Tree-SHA512: de27513a615d9d21271a0948e012c3209351e7374efd19bfa1bb9cda77e8fffe15d99e3424e4dbfa8cf826084f8af1670726f4703bd2b6093e7d37df4bea64f0
2019-11-19 10:11:49 -05:00
Hennadii Stepanov
5f50599ae7 refactor: Cleanup headers from walletmodel.h 2019-11-19 17:05:35 +02:00
Hennadii Stepanov
a53e9895db refactor: Nuke walletmodel circular dependency 2019-11-19 17:05:35 +02:00
Hennadii Stepanov
49c4211c04 refactor: Nuke walletmodeltransaction circular dep 2019-11-19 17:05:35 +02:00
Hennadii Stepanov
567cb44eb9 refactor: Nuke guiutil circular dependency
Co-authored-by: Gregory Sanders <gsanders87@gmail.com>
2019-11-19 17:07:18 +02:00
Hennadii Stepanov
73b5505cfe refactor: Move SendCoinsRecipient in own header
Co-authored-by: Gregory Sanders <gsanders87@gmail.com>
2019-11-19 17:05:35 +02:00
Wladimir J. van der Laan
92db280817 Merge #17411: doc: Add some better examples for scripted diff
adbe155047 doc: Add some better examples for scripted diff (Wladimir J. van der Laan)

Pull request description:

  The current example isn't too great, for example it uses `find` instead of `git ls-files`. Add a subsection with suggestions and examples.

  Feel free to propose some other great examples to add.

ACKs for top commit:
  hebasto:
    re-ACK adbe155047

Tree-SHA512: 38f03716a122a1791c93abc052ea7572a3d2108b3d0d93dc95d3c4a7eb190c6b639d1cc66e4f74d378c4b11d6951dbd901d0973792f8f13cbeb9d9dcf4f8e037
2019-11-19 15:21:37 +01:00
Wladimir J. van der Laan
adbe155047 doc: Add some better examples for scripted diff
The current example isn't too great, for example it uses `find` instead
of `git ls-files`. Add a subsection with suggestions and examples.
2019-11-19 15:13:57 +01:00
Wladimir J. van der Laan
ea3c7e585c test: Remove libssl-dev packages from CI scripts 2019-11-19 15:04:43 +01:00
Wladimir J. van der Laan
7ea55264b9 test: remove lsan suppression for libcrypto 2019-11-19 15:03:44 +01:00
Wladimir J. van der Laan
2d7066527a build: remove libcrypto as internal dependency in libbitcoinconsensus.pc 2019-11-19 15:03:44 +01:00
Wladimir J. van der Laan
278751ea11 doc: Remove ssl as a required dependency from build-unix 2019-11-19 15:03:44 +01:00
Wladimir J. van der Laan
2065ef66ee Merge #17265: Remove OpenSSL
e5a0bece6e doc: add OpenSSL removal to release-notes.md (fanquake)
397dbae070 ci: remove OpenSSL installation (fanquake)
a4eb839619 doc: remove OpenSSL from build instructions and licensing info (fanquake)
648b2e3c32 depends: remove OpenSSL package (fanquake)
8983ee3e6d build: remove OpenSSL detection and libs (fanquake)
b49b6b0f70 random: Remove remaining OpenSSL calls and locking infrastructure (fanquake)
4fcfcc294e random: stop retrieving random bytes from OpenSSL (fanquake)
5624ab0b4f random: stop feeding RNG output back into OpenSSL (fanquake)

Pull request description:

  Now that #17165 has been merged, removing our remaining OpenSSL usage is possible.

  That remaining usage was a call to [`RAND_bytes`](https://www.openssl.org/docs/manmaster/man3/RAND_bytes.html) during the ::SLOW path of [ProcRand](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L616). As well as feeding output from our RNG back into OpenSSL via [`RAND_add`](https://www.openssl.org/docs/manmaster/man3/RAND_add.html) during the ::SLOW and ::SLEEP paths.

  Optimistically tagged for `0.20.0`. Needs discussion, potentially in an upcoming weekly meeting?

  Closes #12530.

ACKs for top commit:
  MarcoFalke:
    ACK e5a0bece6e
  laanwj:
    ACK e5a0bece6e

Tree-SHA512: 02fce08ec91d20e0da51e9314eec53dcf8699cded02f0a005417d627520c20b826332cb42bdae132af283d4903aa3088a9f613f3aea915d655a51532a4d4796c
2019-11-19 09:26:13 +01:00
fanquake
a67352161c test: skip tool_wallet test when bitcoin-wallet isn't compiled 2019-11-18 16:48:52 -05:00
fanquake
e9277baed6 test: skip wallet_listreceivedby test when the cli isn't compiled 2019-11-18 16:47:41 -05:00
fanquake
621d398750 test: skip bitcoin_cli test when the cli isn't compiled 2019-11-18 16:19:15 -05:00
Andrew Chow
d0dab897af Refactor: Require scriptPubKey to get wallet SigningProvider
Easier to review ignoring whitespace:

    git log -p -n1 -w

This commit does not change behavior. It passes new CScript arguments to
signing functions, but the arguments aren't currently used.
2019-11-18 15:42:01 -05:00
Andrew Chow
4b0c718f8f Accumulate result UniValue in SignTransaction
SignTransaction will be called multiple times in the future. Pass
it a result UniValue so that it can accumulate the results of multiple
SignTransaction passes.
2019-11-18 15:28:15 -05:00
MarcoFalke
30521302f9 Merge #17136: tests: Add fuzzing harness for various PSBT related functions
49f4c7f069 tests: Add fuzzing harness for various PSBT related functions (practicalswift)

Pull request description:

  Add fuzzing harness for various PSBT related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/psbt
  ```

ACKs for top commit:
  MarcoFalke:
    re-ACK 49f4c7f069 🐟

Tree-SHA512: 4cebe62bd8c244ee40a43e829f5bd175ab40e1dfbbab1affb1529374858225820d6c9fa9ba45862bf56c1522845422fd96d620cedbdec52a67ac1449dec4e1b2
2019-11-18 12:17:08 -05:00
practicalswift
49f4c7f069 tests: Add fuzzing harness for various PSBT related functions 2019-11-18 16:52:56 +00:00
fanquake
55b2cb199c random: mark RandAddPeriodic and SeedPeriodic as noexcept
The usage of MilliSleep() in SeedPeriodic (previously SeedSleep) was
removed in #17270, meaning it, and its users can now be marked noexcept.
2019-11-18 10:22:17 -05:00
fanquake
461e547877 doc: correct random.h docs after #17270 2019-11-18 10:22:08 -05:00
MarcoFalke
397c6d32c8 Merge #17503: doc: Remove bitness from bitcoin-qt help message and manpage
e161bc74d2 doc: Remove bitness from bitcoin-qt help message and manpage (Wladimir J. van der Laan)

Pull request description:

  Remove the `(64-bit)` from the bitcoin-qt help message.

  Since removing the Windows 32-bit builds, it is no longer information that is often useful for troubleshooting. This never worked for other architectures than x86, and the only 32-bit x86 build left is the Linux one. Linux users tend to know what architecture they are using.

  It also accidentally ends up in the bitcoin-qt manpage (if you happen to be generating them on a x86 machine), which gets checked in. See for example 1bc9988993 (diff-e4b84be382c8ea33b83203ceb8c85296)

ACKs for top commit:
  practicalswift:
    ACK e161bc74d2 -- rationale makes sense and diff looks correct :)
  MarcoFalke:
    Tested ACK e161bc74d2 🔮

Tree-SHA512: d38754903252896dc86fac6c12ad6615d322c2744db7c02b18574a08c69e8876b2c905e1f09b324002236b111ee93479f89769c562e7b3b2e6eb2992d76464ef
2019-11-18 09:35:07 -05:00
fanquake
e5a0bece6e doc: add OpenSSL removal to release-notes.md 2019-11-18 09:19:04 -05:00
fanquake
397dbae070 ci: remove OpenSSL installation 2019-11-18 08:56:48 -05:00
fanquake
a4eb839619 doc: remove OpenSSL from build instructions and licensing info 2019-11-18 08:56:48 -05:00
fanquake
648b2e3c32 depends: remove OpenSSL package 2019-11-18 08:56:47 -05:00
fanquake
8983ee3e6d build: remove OpenSSL detection and libs 2019-11-18 08:56:47 -05:00
fanquake
b49b6b0f70 random: Remove remaining OpenSSL calls and locking infrastructure 2019-11-18 08:56:47 -05:00
fanquake
4fcfcc294e random: stop retrieving random bytes from OpenSSL
On the ::SLOW path we would use OpenSSL as an additional source of
random bytes. This commit removes that functionality. Note that this was
always only an additional source, and that we never checked the return
value

RAND_bytes(): https://www.openssl.org/docs/manmaster/man3/RAND_bytes.html

RAND_bytes() puts num cryptographically strong pseudo-random bytes into buf.
2019-11-18 08:56:40 -05:00
fanquake
5624ab0b4f random: stop feeding RNG output back into OpenSSL
On the ::SLOW or ::SLEEP paths, we would feed our RNG output back into
OpenSSL using RAND_add. This commit removes that functionality.

RAND_add(): https://www.openssl.org/docs/manmaster/man3/RAND_add.html

RAND_add() mixes the num bytes at buf into the internal state of the
random generator. This function will not normally be needed, as
mentioned above. The randomness argument is an estimate of how much
randomness is contained in buf, in bytes, and should be a number
between zero and num.
2019-11-18 08:48:39 -05:00
Wladimir J. van der Laan
6fde676f64 Merge #16669: build: use new fork of osslsigncode for windows gitian signing
feb5075777 build: use osslsigncode 2.0 in gitian (fanquake)

Pull request description:

  The original osslsigncode project, https://sourceforge.net/projects/osslsigncode, has been marked as abandonware:
  > This is now - and has been for a long while - abandonware. Feel free to create your own forks etc.".

  However, a fork has emerged, https://github.com/mtrojnar/osslsigncode, that has incorporated
  theuni's patches ([add the -pem option in extract-signature mode ](36715c1183) & [add the attach-signature command](3be7eb1676)) as well as updated the tool to work with OpenSSL 1.1 and other improvements.

  This commit switches the windows signer descriptor to use this new version of `osslsigncode`.

  I've tested using this new version of `osslsigncode` while doing a 0.18.1 gitian build, and it "seems" to work. However this needs a look over from Cory, to check if the tool is still compatible with his usage in the [`detached-sig-create.sh`](https://github.com/bitcoin/bitcoin/blob/master/contrib/windeploy/detached-sig-create.sh) script, as well as some review of the changes to `osslsigncode` itself. Hence WIP and chasing Concept ACKs / NACKs.

ACKs for top commit:
  MarcoFalke:
    Concept ACK feb5075777 given that this upstream is now used in Ubuntu and Debian
  laanwj:
    ACK feb5075777

Tree-SHA512: c48de6dc32751d96dd04b920bfacca40af47a2883330ba0700371d56c580a7e45cedd8d8a913709d56be036762b63cb1825a98cff7aa77b6d7804fab11220850
2019-11-18 14:39:19 +01:00
Wladimir J. van der Laan
ca1eeba0b0 Merge #17488: test: fix "bitcoind already running" warnings on macOS
1c23ea5fe6 test: fix bitcoind already running warnings on macOS (fanquake)

Pull request description:

  On macOS, `pidof` installed via brew returns b'' rather than None.
  Account for this, to remove spurious warnings from the test_runner.

ACKs for top commit:
  laanwj:
    ACK 1c23ea5fe6

Tree-SHA512: 640f4323d4105eac5c7abb52daf80486d5d3b4a074720490ceeb97c3dd8d73a3de9a988d2550f1e2076c620bb10d452b2959d8b723d2ee64f499878909824e31
2019-11-18 14:23:41 +01:00
Wladimir J. van der Laan
63fac52f31 Merge #17328: GuessVerificationProgress: cap the ratio to 1
2f5f7d6b13 GuessVerificationProgress: cap the ratio to 1 (darosior)

Pull request description:

  Noticed `getblockchaininfo` would return a `verificationprogress` > 1, especially while generating. This caps the verification progress to `1`.

  Tried to append a check to functional tests but this would pass even without the patch, so it seems better to not add a superfluous check (but this can easily be reproduced by trying to generate blocks in the background and `watch`ing `getblockchainfo`).

ACKs for top commit:
  laanwj:
    ACK 2f5f7d6b13
  promag:
    ACK 2f5f7d6b13.

Tree-SHA512: fa3aca12acab9c14dab3b2cc94351082f548ea6e6c588987cd86e928a00feb023e8112433658a0e85084e294bfd940eaafa33fb46c4add94146a0901bc1c4f80
2019-11-18 14:14:03 +01:00
Wladimir J. van der Laan
24647a09e7 Merge #17470: ci: Use clang-8 for fuzzing to run on aarch64 ci systems
fa2ec9f451 fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs (MarcoFalke)
fa6e01b2f3 ci: Use clang-8 for fuzzing to run on aarch64 ci systems (MarcoFalke)

Pull request description:

  Ubuntu bionic clang is clang version 6, which does not come with libfuzzer. So the ci system breaks down when run on aarch64.

  Fix that by using clang-8

  For reference, the previous error on my ci system was:

  ```
  /usr/bin/ld: cannot find /usr/lib/llvm-6.0/lib/clang/6.0.0/lib/linux/libclang_rt.fuzzer-aarch64.a: No such file or directory

ACKs for top commit:
  laanwj:
    ACK fa2ec9f451

Tree-SHA512: 4954dbc36c444d1ae145290115eea6291753c9810c92003ab8d75433c3fe3bfee439d3a99dc394418275527157a8b89f04038c8b16e08c69ec9ded50fb869e70
2019-11-18 14:02:04 +01:00
Wladimir J. van der Laan
0bb37e437e Merge #17270: Feed environment data into RNG initializers
d1c02775aa Report amount of data gathered from environment (Pieter Wuille)
64e1e022ce Use thread-safe atomic in perfmon seeder (Pieter Wuille)
d61f2bb076 Run background seeding periodically instead of unpredictably (Pieter Wuille)
483b94292e Add information gathered through getauxval() (Pieter Wuille)
11793ea22e Feed CPUID data into RNG (Pieter Wuille)
a81c494b4c Use sysctl for seeding on MacOS/BSD (Pieter Wuille)
2554c1b81b Gather additional entropy from the environment (Pieter Wuille)
c2a262a78c Seed randomness with process id / thread id / various clocks (Pieter Wuille)
723c796667 [MOVEONLY] Move cpuid code from random & sha256 to compat/cpuid (Pieter Wuille)
cea3902015 [MOVEONLY] Move perfmon data gathering to new randomenv module (Pieter Wuille)
b51bae1a5a doc: minor corrections in random.cpp (fanquake)

Pull request description:

  This introduces a new `randomenv` module that queries varies non-cryptographic (and non-RNG) sources of entropy available on the system; things like user IDs, system configuration, time, statistics, CPUID data.

  The idea is that these provide a fallback in scenarios where system entropy is somehow broken (note that if system entropy *fails* we will abort regardless; this is only meant to function as a last resort against undetected failure). It includes some data sources OpenSSL currently uses, and more.

  The separation between random and randomenv is a bit arbitrary, but I felt that all this "non-essential" functionality deserved to be separated from the core random module.

ACKs for top commit:
  TheBlueMatt:
    utACK d1c02775aa. Certainly no longer measuring the time elapsed between a 1ms sleep (which got removed in the latest change) is a fair tradeoff for adding about 2 million other actually-higher-entropy bits :).
  laanwj:
    ACK d1c02775aa

Tree-SHA512: d290a8db6538a164348118ee02079e4f4c8551749ea78fa44b2aad57f5df2ccbc2a12dc7d80d8f3e916d68cdd8e204faf9e1bcbec15f9054eba6b22f17c66ae3
2019-11-18 13:33:43 +01:00
Wladimir J. van der Laan
1baf7d1819 Merge #17466: depends: qt: Fix C{,XX} pickup
bc728d12a9 depends: qt: Fix C{,XX} pickup (Carl Dong)

Pull request description:

  QT LOVES to do things its own way and simply ignore all environment variables. Even listing `QMAKE_CC=...` in `./configure` flags does't work reliably. This is why we gotta literally edit their config files for them. Thanks QT.

ACKs for top commit:
  laanwj:
    ACK bc728d12a9

Tree-SHA512: 3d6a0535174d70937d11b8f44ac2ab720870345a2d5a541c3e0181163ffa1988ab4f934788e9c841b646d6f53c6dc51a4fe39b294dd042c38976091c0fb96c96
2019-11-18 12:11:28 +01:00
Wladimir J. van der Laan
e161bc74d2 doc: Remove bitness from bitcoin-qt help message and manpage
Remove the `(64-bit)` from the bitcoin-qt help message.

Since removing the Windows 32-bit builds, it is no longer information
that is often useful for troubleshooting. This never worked for other
architectures than x86, and the only 32-bit x86 build left is the Linux
one. Linux users tend to know what architecture they are using.

It also accidentally ends up in the bitcoin-qt manpage.
2019-11-18 11:44:51 +01:00
Wladimir J. van der Laan
94f0749f91 Merge #17436: Add TheCharlatan's pgp key
0ec967164f Add TheCharlatan's pgp key (TheCharlatan)

Pull request description:

  Since I have submitted my pair of signed gitian assertions, I am now adding my key's fingerprint to the gitian keys list.

ACKs for top commit:
  jonasschnelli:
    ACK 0ec967164f - confirm key A8FC55F3B04BA3146F3492E79303B33A305224CB

Tree-SHA512: bbd5e637186ed1659432e4fcc96bdc57fdbdb608325805701e06a51689726e722d7abeb11b5c9de723d051976d9d7ac23602316403fa74029dceb3cf1e837aea
2019-11-18 10:54:49 +01:00
Jonas Schnelli
7dbc33f617 Merge #17499: refactor, qt: Remove unused signal from WalletView class
5fa28e9903 refactor: Remove unused signal (Hennadii Stepanov)

Pull request description:

  `WalletView::showNormalIfMinimized()` signal was introduced in #2872 (dbc0a6aba2).

  The only signal emit command was removed in #3144 (2384a2864b)

ACKs for top commit:
  promag:
    ACK 5fa28e9903.
  practicalswift:
    ACK 5fa28e9903: nice find
  emilengler:
    ACK 5fa28e9
  jonasschnelli:
    utACK 5fa28e9903

Tree-SHA512: 4714acf8c683594d3c00523c7b14bc6b94d469418f0cebe4f4b5266ca0e4c45c80d4caf358739eae9231ee4a69c9c902caeb35f3866b99443cf653f89d6d825b
2019-11-17 10:36:44 -10:00
dannmat
8944c1d340 Changed tooltips of receive form to highlight difference between Label and Message 2019-11-17 12:00:06 +00:00
Hennadii Stepanov
5fa28e9903 refactor: Remove unused signal 2019-11-17 05:45:07 +02:00
Jeffrey Czyz
30fb598737 Fix segfault in allocator_tests/arena_tests
The test uses reinterpret_cast<void*> on unallocated memory. Using this
memory in printchunk as char* causes a segfault, so have printchunk take
void* instead.
2019-11-16 10:43:37 -08:00
Jeffrey Czyz
15c84f53f4 Define ARENA_DEBUG in Travis test runs
The definition and uses of Arena::walk() are compiled only if
ARENA_DEBUG is defined. Configure Travis to define ARENA_DEBUG so
compilation errors do not go unnoticed.
2019-11-16 08:45:21 -08:00
Jeffrey Czyz
ad71548822 Fix compilation errors in support/lockedpool.cpp
Changes in #12048 cause a compilation error in Arena::walk() when
ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
changed to have a different value type.

Additionally, missing includes cause other compilation errors when
ARENA_DEBUG is defined.

Reproduced with:

make CPPFLAGS=-DARENA_DEBUG
2019-11-16 08:44:42 -08:00
fanquake
1c23ea5fe6 test: fix bitcoind already running warnings on macOS
On macOS, pidof installed via brew returns b'' rather than None.
Account for this, to remove spurious warnings from the test_runner.
2019-11-15 16:03:47 -05:00
MarcoFalke
b90dad5143 Merge #17486: build: make Travis catch unused variables
18b18f8e81 [build] ./configure --enable-werror: add unused-variable (Sjors Provoost)

Pull request description:

  The two macOS Travis machines run with `--enable-werror`. This PR adds `-Werror=unused-variable` to the existing `vla`, `switch` and `thread-safety-analysis` checks. This should prevent the need for fixes like b07b07cd87, 26a93bce29, dd777f3e12, 99be644966, fa39f674ae, 16bcc1b823, bb079a0e2c, bdaed47558 and ecf9b25a03 with minimal nuisance.

  Thoughts for followups:
  * Travis starts these macOS machines fairly late, so we should consider setting `--enable-werror` on earlier machines as well.
  * We should encourage the use of `--enable-werror` by developers. Maybe switch it on by default for `--enable-debug`?
  * See practicalswift's overview of other checks to consider in #17344

ACKs for top commit:
  MarcoFalke:
    ACK 18b18f8e81
  practicalswift:
    ACK 18b18f8e81 -- nice!

Tree-SHA512: 892b471ca5ea547f3c952ac88190cbebf8110cb7aec6f20466aeb312aeb0910bfe990f914e153c40ecb55709c03775ef30770412ad76f9d532ca77055596c582
2019-11-15 15:09:28 -05:00
MarcoFalke
1ed3e071df Merge #17455: tests: Update valgrind suppressions
d604b4cc8c tests: Update valgrind suppressions (practicalswift)

Pull request description:

  Update `valgrind` suppressions.

  To test this PR:

  ```
  $ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
  $ valgrind --suppressions=contrib/valgrind.supp src/bench/bench_bitcoin -evals=1 \
        -scaling=0.0
  ```

Top commit has no ACKs.

Tree-SHA512: 79cb318b5b9171e74d0bd0b89cc688ad4531b134182b06c2942c46058c19b45723c391b781e8ccd157a14fbf6a14588764c7728c5506c73ae237dde9f44db2f6
2019-11-15 14:43:50 -05:00
MarcoFalke
f92e750eb4 Merge #17480: test: add unit test for non-standard txs with too large scriptSig
5e8a56348b test: add unit test for non-standard txs with too large scriptSig (Sebastian Falbesoner)

Pull request description:

  Approaches the first missing test of issue #17394: Checks that the function `IsStandardTx()` returns rejection reason `"scriptsig-size"` if any one the inputs' scriptSig is larger than 1650 bytes.

ACKs for top commit:
  MarcoFalke:
    ACK 5e8a56348b
  instagibbs:
    ACK 5e8a56348b

Tree-SHA512: 79977b12ddea9438a37cefdbb48cc551e4ad02a8ccfaa2d2837ced9f3a185e2e07cc366c243b9e3c7736245e90e315d7b4110efc6b440c63dbef7ee2c9d78a73
2019-11-15 14:03:56 -05:00
MarcoFalke
422ec33d45 Merge #17322: Fix input size assertion in wallet_bumpfee.py
38516f9078 Fix input size assertion in wallet_bumpfee.py (Gregory Sanders)

Pull request description:

  I was investigating a curious error for https://github.com/bitcoin/bitcoin/pull/17290 and realized that this check should have caught that error earlier in the test.

  The loop is intended to ensure that only a single input exists the entire time until the change output disappears, a single additional bump occurs, then it leaves the loop.

Top commit has no ACKs.

Tree-SHA512: 1d2d6ef535ec2c55f516ee5de11352386ceac6bedaabc6842229a486d9f28d35310ad5f57bfcc1f1e654fc397ecff29ec33256f9b3da897500b7e1635004b63a
2019-11-15 14:02:01 -05:00
Gregory Sanders
38516f9078 Fix input size assertion in wallet_bumpfee.py 2019-11-15 13:58:51 -05:00
MarcoFalke
fa538813b1 scripted-diff: Replace ::mempool with m_node.mempool in tests
-BEGIN VERIFY SCRIPT-
 # tx pool member access (mempool followed by dot)
 sed --regexp-extended -i -e 's/(::)?\<mempool\>\.([a-zA-Z])/m_node.mempool->\2/g' $(git grep -l mempool ./src/test)
 # plain global (mempool not preceeded by dot, but followed by comma)
 sed --regexp-extended -i -e 's/([^\.])(::)?\<mempool\>,/\1*m_node.mempool,/g'     $(git grep -l mempool ./src/test)
-END VERIFY SCRIPT-
2019-11-15 13:40:14 -05:00
MarcoFalke
8888ad02e2 test: Replace recursive lock with locking annotations
Also, use m_node.mempool instead of the global
2019-11-15 13:40:08 -05:00
MarcoFalke
fac07f2038 node: Add reference to mempool in NodeContext
Currently it is an alias to the global ::mempool and should be used as
follows.

* Node code (validation and transaction relay) can use either ::mempool
  or node.mempool, whichever seems a better fit.
* RPC code should use the added convenience getter EnsureMempool, which
  makes sure the mempool exists before use. This prepares the RPC code
  to a future where the mempool might be disabled at runtime or compile
  time.
* Test code should use m_node.mempool directly, as the mempool is always
  initialized for tests.
2019-11-15 13:40:00 -05:00
MarcoFalke
c7709cbf4c Merge #17469: test: Remove fragile assert_memory_usage_stable
fac942ca57 test: Remove fragile assert_memory_usage_stable (MarcoFalke)

Pull request description:

  This test fails on arm64 and a fuzz tests seems inappropriate for the functional test suite anyway, so remove it.

  Example failures:

  * https://travis-ci.org/bitcoin/bitcoin/jobs/611497963#L14517
  * https://travis-ci.org/MarcoFalke/bitcoin-core/jobs/611029104#L3876

ACKs for top commit:
  jamesob:
    ACK fac942ca57

Tree-SHA512: 3577e7ce5891d221cb798454589ba796ed0c06621a26351bb919c23bc6bb46aafcd0b11cb02bbfde64b74d67cb2950da44959a7ecdc436491a34e8b045c1ccf4
2019-11-15 12:46:51 -05:00
Sjors Provoost
18b18f8e81 [build] ./configure --enable-werror: add unused-variable
Turn corresponding warning on by default (not always covered by -Wall).
2019-11-15 17:35:59 +01:00
MarcoFalke
fae75306ba scripted-diff: Set gitian arch back to amd64
-BEGIN VERIFY SCRIPT-
sed -i -e 's|"linux64"|"amd64"|g' $(git grep -l '"linux64"')
-END VERIFY SCRIPT-
2019-11-14 17:10:18 -05:00
fanquake
21ee676dd6 Merge #17449: fix uninitialized variable nMinerConfirmationWindow
edb6b768a4 fix uninitialized variable nMinerConfirmationWindow (NullFunctor)

Pull request description:

  It is used for the computation of `BIP9WarningHeight`, and by that time it isn't initialized.

ACKs for top commit:
  jnewbery:
    utACK edb6b768a
  promag:
    ACK edb6b768a4, commit description could be cleaned up though.
  MarcoFalke:
    ACK edb6b768a4, used python3 to do the addition locally 📍
  practicalswift:
    ACK edb6b768a4, used `clang++ -O2` on the previous version^W^W^W^W^W^W`bc` to verify the addition locally 🏓
  Sjors:
    Code review ACK  edb6b76. Nit: commit description has duplicate text.

Tree-SHA512: 6fa0be0ecfbfd5d537f2c5b4a9333c76530c1f3182f777330cc7939b0496e37b75d8f8810cdaf471a9bd3247b425f2e239578300dfa0d5a87cd14a6ccfafa619
2019-11-14 10:53:51 -08:00
Sebastian Falbesoner
5e8a56348b test: add unit test for non-standard txs with too large scriptSig
The function IsStandardTx() returns rejection reason "scriptsig-size" if any
one the inputs' scriptSig is larger than 1650 bytes.
2019-11-14 19:51:50 +01:00
MarcoFalke
fa2ec9f451 fuzz: Bump timeout in test_runner to accomodate for slow arm64 CPUs 2019-11-14 13:50:20 -05:00
MarcoFalke
fac942ca57 test: Remove fragile assert_memory_usage_stable 2019-11-14 10:56:57 -05:00
Luke Dashjr
4341bffb6e GUI: Refactor formatServicesStr to warn when a ServicesFlag is missing 2019-11-14 05:02:16 +00:00
Luke Dashjr
df77de8c21 Bugfix: GUI: Recognise NETWORK_LIMITED in formatServicesStr 2019-11-14 05:01:50 +00:00
MarcoFalke
fa6e01b2f3 ci: Use clang-8 for fuzzing to run on aarch64 ci systems 2019-11-13 17:53:04 -05:00
Russell Yanofsky
e9fd366044 refactor: Remove null setting check in GetSetting()
Also rename the "result_complete" variable in GetSettingsList() to "done" to be
more consistent with GetSetting().

This change doesn't affect current behavior but could be useful in the future
to support dynamically changing settings at runtime and adding new settings
sources, because it lets high priority sources reset settings back to default
(see test).

By removing a special case for null, this change also helps merge code treat
settings values more like black boxes, and interfere less with settings parsing
and retrieval.
2019-11-13 15:23:06 -05:00
Sjors Provoost
c6dd565c88 [gui] watch-only wallet: copy PSBT to clipboard 2019-11-13 20:03:42 +01:00
Carl Dong
bc728d12a9 depends: qt: Fix C{,XX} pickup 2019-11-13 13:12:01 -05:00
Sjors Provoost
39465d545d [wallet] add fillPSBT to interface 2019-11-13 18:54:40 +01:00
Sjors Provoost
848f889208 [gui] send: include watch-only
For wallets with WALLET_FLAG_DISABLE_PRIVATE_KEYS.
2019-11-13 18:54:40 +01:00
Sjors Provoost
40537f0909 [wallet] ListCoins: include watch-only for wallets without private keys
This makes them available in GUI coin selection.
2019-11-13 18:54:39 +01:00
practicalswift
d604b4cc8c tests: Update valgrind suppressions 2019-11-13 14:46:09 +00:00
Wladimir J. van der Laan
cd6cb9745e Merge #17409: build: Avoid hardcoded libfaketime dir in gitian
333362991c doc: Explain $LIB in LD_PRELOAD in gitian descriptors (MarcoFalke)
fab9850ef4 scripted-diff: Avoid hardcoded libfaketime dir in gitian (MarcoFalke)

Pull request description:

  Without this gitian prints warnings for me:

  ```
  ERROR: ld.so: object '/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
  ```

  ```
  $ ls /usr/lib/aarch64-linux-gnu/faketime/libfaketime.so.1
  /usr/lib/aarch64-linux-gnu/faketime/libfaketime.so.1
  ```

ACKs for top commit:
  laanwj:
    ACK 333362991c

Tree-SHA512: 3e7c4471b69c2ae38c29d0cc0db8b9eae0912085299d7f5ac67eeb4b6a2fdc7eb23d806eeeae0b0c2da22d6d1ba82513cab23652876b97aada9928b2c7d38e7e
2019-11-13 10:20:30 +01:00
Russell Yanofsky
cba2710220 scripted-diff: Remove unused ArgsManager type flags in tests
The bool/int/string flags were added speculatively in #16097 and trigger errors
when type checking is actually implemented in
https://github.com/bitcoin/bitcoin/pull/16545

-BEGIN VERIFY SCRIPT-
sed -i 's/ALLOW_\(BOOL\|INT\|STRING\)/ALLOW_ANY/g' src/test/util_tests.cpp src/test/getarg_tests.cpp
-END VERIFY SCRIPT-

This commit does not change behavior.
2019-11-13 04:20:30 -05:00
Russell Yanofsky
425bb30725 refactor: Add util_CheckValue test
Test GetSetting and GetArg type coercion, negation, and default value handling.
Test is expanded later to cover other flags besides ALLOW_ANY when they are
implemented in https://github.com/bitcoin/bitcoin/pull/16545

This commit does not change behavior.
2019-11-13 05:20:30 -04:00
Russell Yanofsky
0fa54358b0 refactor: Add ArgsManager::GetSettingsList method
Add for consistency with ArgsManager::GetSetting method and to make setting
types accessible to ArgsManager callers and tests (test added next commit).

This commit does not change behavior.
2019-11-13 04:20:30 -05:00
Russell Yanofsky
3e185522ac refactor: Get rid of ArgsManagerHelper class
Suggested by John Newbery <john@johnnewbery.com>
https://github.com/bitcoin/bitcoin/pull/15934#issuecomment-551969778

This commit does not change behavior.
2019-11-13 04:20:30 -05:00
Russell Yanofsky
dc0f148074 refactor: Replace FlagsOfKnownArg with GetArgFlags
Rename suggested by João Barbosa <joao.paulo.barbosa@gmail.com>
https://github.com/bitcoin/bitcoin/pull/16545#issuecomment-519048000

This also gets rid of ArgsManager::NONE constant, which was an implementation
detail not meant to be used by ArgsManager callers.

Finally this reverts a change from 7f40528cd5
https://github.com/bitcoin/bitcoin/pull/15934 adding "-" characters to argument
names. Better for GetArgFlags to require "-" prefixes for consistency with
other ArgsManager methods, and to be more efficient later when GetArg functions
need to call GetArgFlags (https://github.com/bitcoin/bitcoin/pull/16545)

This commit does not change behavior.
2019-11-13 04:20:30 -05:00
Russell Yanofsky
57e8b7a727 refactor: Clean up includeconf comments
Suggested by Antoine Riard <ariard@student.42.fr>
https://github.com/bitcoin/bitcoin/pull/15934#discussion_r344291875

and John Newbery <john@johnnewbery.com>
https://github.com/bitcoin/bitcoin/pull/15934#discussion_r344271224

This commit does not change behavior.
2019-11-13 04:20:30 -05:00
Russell Yanofsky
3f7dc9b808 refactor: Clean up long lines in settings code
Suggested by James O'Beirne <james.obeirne@gmail.com>
https://github.com/bitcoin/bitcoin/pull/15934#discussion_r344366743

This commit does not change behavior.
2019-11-13 04:20:30 -05:00
NullFunctor
edb6b768a4 fix uninitialized variable nMinerConfirmationWindow
fix uninitialized variable hard code the MinBIP9WarningHeight

fix uninitialized var hard code the MinBIP9WarningHeight instead
2019-11-12 17:59:52 -06:00
Pieter Wuille
d1c02775aa Report amount of data gathered from environment 2019-11-12 15:35:26 -08:00
Pieter Wuille
64e1e022ce Use thread-safe atomic in perfmon seeder
Also switch to chrono based types.
2019-11-12 15:35:26 -08:00
Pieter Wuille
d61f2bb076 Run background seeding periodically instead of unpredictably
* Instead of calling RandAddSeedSleep anytime the scheduler goes
  idle, call its replacement (RandAddSeedPeriodic) just once per
  minute. This has better guarantees of actually being run, and
  helps limit how frequently the dynamic env data is gathered.
* Since this code runs once per minute regardless now, we no
  longer need to keep track of the last time strengthening was
  run; just do it always.
* Make strengthening time context dependent (100 ms at startup,
  10 ms once per minute afterwards).
2019-11-12 15:35:26 -08:00
Pieter Wuille
483b94292e Add information gathered through getauxval()
Suggested by Wladimir van der Laan.
2019-11-12 15:35:26 -08:00
Pieter Wuille
11793ea22e Feed CPUID data into RNG 2019-11-12 15:35:26 -08:00
Pieter Wuille
a81c494b4c Use sysctl for seeding on MacOS/BSD 2019-11-12 15:35:22 -08:00
Pieter Wuille
2554c1b81b Gather additional entropy from the environment
This based on code by Gregory Maxwell.
2019-11-12 15:24:02 -08:00
Pieter Wuille
c2a262a78c Seed randomness with process id / thread id / various clocks
This sort of data is also used by OpenSSL.
2019-11-12 14:50:44 -08:00
Pieter Wuille
723c796667 [MOVEONLY] Move cpuid code from random & sha256 to compat/cpuid 2019-11-12 14:50:44 -08:00
Pieter Wuille
cea3902015 [MOVEONLY] Move perfmon data gathering to new randomenv module 2019-11-12 14:50:44 -08:00
fanquake
b51bae1a5a doc: minor corrections in random.cpp
This should have been part of #17151.
2019-11-12 14:50:44 -08:00
MarcoFalke
8237889e8d Merge #17435: test: check custom ancestor limit in mempool_packages.py
49997813a4 test: check custom ancestor limit in mempool_packages.py (Sebastian Falbesoner)

Pull request description:

  The functional test `mempool_packages.py` starts one node with default ancestor/descendant limit settings and one with a custom, reduced ancestor limit (currently `-limitancestorcount=5`). The effect of the latter had not been tested yet though. This is approached in this PR by checking on the expected mempool contents of node1 after the node0 ancestor tests are done, via the following three conditions:
  - the # of txs in the node1 mempool is equal to the the limit
  - all txs in node1 mempool are a subset of txs in node0 mempool
  - the node1 mempool txs match the start of the constructed tx-chain

  Note that this still doesn't *fully* check the expected mempool of node1 (e.g. that it isn't influenced by `prioritisetransaction` RPC on node0), hence I add another TODO. In the future it would make sense to also set a custom descendant limit when the second TODO about checking node1's mempool is approached: 89e93135ae/test/functional/mempool_packages.py (L228)

ACKs for top commit:
  MarcoFalke:
    ACK 49997813a4 👲

Tree-SHA512: d3a1d19fb49731238ad08ee7c02e2fa81a227e3b4ef3340d68598de42ddb62be9161134f6b8e08fa76b8c9faa02fecfa01111159642e20e9f358292a757b7608
2019-11-12 14:53:34 -05:00
MarcoFalke
1028882eea Merge #17437: rpc: Expose block height of wallet transactions
a5e77959c8 rpc: Expose block height of wallet transactions (João Barbosa)

Pull request description:

  Closes #17296.

ACKs for top commit:
  practicalswift:
    ACK a5e77959c8 -- diff looks correct now (good catch @theStack!)
  theStack:
    ACK a5e77959c8
  ryanofsky:
    Code review ACK a5e77959c8. Changes since last review getblockhash python test fixes, and removing the last hardcoded height

Tree-SHA512: 57dcd0e4e7083f34016bf9cf8ef578fbfde49e882b6cd8623dd1c64716e096e62f6177a4c2ed94f5de304e751fe23fb9d11cf107a86fbf0a3c5f539cd2844916
2019-11-12 14:44:13 -05:00
MarcoFalke
333362991c doc: Explain $LIB in LD_PRELOAD in gitian descriptors 2019-11-12 12:06:34 -05:00
MarcoFalke
6d4b97cb1c Merge #17450: util: Add missing headers to util/fees.cpp
b131524137 util: Add missing headers to util/fees.cpp (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  laanwj:
    code review ACK b131524137
  MarcoFalke:
    ACK b131524
  jnewbery:
    ACK b131524137

Tree-SHA512: a1ad36bff12219912c6aaacd7d9dcbeccf0fa3373280fa6e804d7a4d267b485433d6e1c01134cfa6732d2fb30ec1ab4629dff6e4bea2fe4c1976180064a3c6ca
2019-11-12 09:53:04 -05:00
Hennadii Stepanov
b131524137 util: Add missing headers to util/fees.cpp 2019-11-12 09:30:52 +02:00
João Barbosa
3c2c439dcd wallet: Make -walletdir network only 2019-11-12 00:16:17 +00:00
João Barbosa
a5e77959c8 rpc: Expose block height of wallet transactions 2019-11-11 22:32:44 +00:00
João Barbosa
3e730bf90a zmq: Fix due to invalid argument and multiple notifiers 2019-11-11 22:22:32 +00:00
Sebastian Falbesoner
49997813a4 test: check custom ancestor limit in mempool_packages.py
To test the custom ancestor limit on node1 (passed by the argument
-limitancestorcount), we check for three conditions:
    -> the # of txs in the node1 mempool is equal to the the limit
    -> all txs in node1 mempool are a subset of txs in node0 mempool
    -> the node1 mempool txs match the start of the constructed tx-chain
2019-11-11 22:37:00 +01:00
MarcoFalke
faffa7f0dc wallet: Avoid showing GUI popups on RPC errors (take 2) 2019-11-11 13:50:26 -05:00
MarcoFalke
80fdb6fad1 Merge #17442: fix Typo: "merkelRoot" -> "merkleRoot"
eb880f092b fix Typo: "merkelRoot" -> "merkleRoot" (ianliu)

Pull request description:

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  practicalswift:
    ACK eb880f092b but please change from `merkleRootofHashes` to `merkleRootOfHashes`

Tree-SHA512: ada9edceee19da5678bf35e1258163e7102fe176dc5cf40acaa1468fa8b2801494f8bf65d5359dcd0054fbc22f07fdc98d6208cfdb54dd9171fd45c89d71e098
2019-11-11 13:20:27 -05:00
ianliu
eb880f092b fix Typo: "merkelRoot" -> "merkleRoot" 2019-11-12 00:56:05 +08:00
Hennadii Stepanov
cb9d830a00 test: Use proper MAX_SCRIPT_ELEMENT_SIZE 2019-11-11 12:04:48 +02:00
Hennadii Stepanov
402ee706d8 refactor: Use proper MAX_SCRIPT_ELEMENT_SIZE const 2019-11-11 11:51:49 +02:00
MarcoFalke
fa7523d3aa ci: Extend docs 2019-11-10 22:30:13 -05:00
MarcoFalke
9999a79074 travis: Rework CACHE_ERR_MSG 2019-11-10 21:49:32 -05:00
MarcoFalke
a6f6333ba2 Merge #17416: Appveyor improvement - text file for vcpkg package list
29eb039252 Moves vcpkg list to a text file and updates the appveyor job and readme to use it. (Aaron Clauson)

Pull request description:

  #17364 attempted to save a couple of minutes by skipping the `vcpkg` steps if the vcpkg install directory was already cached.

  The discussion in #15382 highlights the approach used in #17364 does not accommodate adding a new package.

  ~~This PR improves the approach to individually check whether  each vcpg package is installed rather than checking for the existence of the vcpkg install directory.~~

  This PR moves the list of required vcpkg packages into a separate file and uses changes to that file to invalidate the appveyor cache. Whenever the cache is invalidated the vcpkg sources will be updated, the vcpkg binary built and the required packages installed from the latest port files.

ACKs for top commit:
  MarcoFalke:
    ACK 29eb039252

Tree-SHA512: 0c2a170f4e4b47ca0f9cef14f1e3892001b441a6d84f50bf5fd8a26bc4cdbd9358dfce7ef180d37150262e849650e9857d6b2bcd686964b963c3de6cd708a2f3
2019-11-10 21:42:10 -05:00
TheCharlatan
0ec967164f Add TheCharlatan's pgp key 2019-11-10 21:43:50 +01:00
Aaron Clauson
29eb039252 Moves vcpkg list to a text file and updates the appveyor job and readme to use it. 2019-11-10 13:49:28 +00:00
Wladimir J. van der Laan
89e93135ae Merge #17427: qt: Fix missing qRegisterMetaType for size_t
1828c6f05f refactor: Styling w/ clang-format, comment update (Hennadii Stepanov)
88a94f7bb8 qt: Fix missing qRegisterMetaType for size_t (Hennadii Stepanov)

Pull request description:

  On master (a7aec7ad97) this connection a7aec7ad97/src/qt/rpcconsole.cpp (L587) fails due to `ClientModel::mempoolSizeChanged()` signal has unregistered parameter type `size_t`: a7aec7ad97/src/qt/clientmodel.h (L102)

  More:
  ```
  $ QT_FATAL_WARNINGS=1 lldb src/qt/bitcoin-qt -- -debug=qt
  ...
  (lldb) bt
  * thread #17, name = 'QThread', stop reason = signal SIGABRT
    * frame #0: 0x00007ffff35fce97 libc.so.6`__GI_raise(sig=2) at raise.c:51
      frame #1: 0x00007ffff35fe801 libc.so.6`__GI_abort at abort.c:79
      frame #2: 0x00007ffff5901352 libQt5Core.so.5`QMessageLogger::warning(char const*, ...) const + 354
      frame #3: 0x00007ffff5b216fe libQt5Core.so.5`___lldb_unnamed_symbol2329$$libQt5Core.so.5 + 334
      frame #4: 0x00007ffff5b2456d libQt5Core.so.5`QMetaObject::activate(QObject*, int, int, void**) + 1933
      frame #5: 0x000055555566872e bitcoin-qt`ClientModel::mempoolSizeChanged(this=<unavailable>, _t1=<unavailable>, _t2=<unavailable>) at moc_clientmodel.cpp:260
  ...

  ```

  `debug.log`:
  ```
  [] GUI: QObject::connect: Cannot queue arguments of type 'size_t'
  (Make sure 'size_t' is registered using qRegisterMetaType().)
  ```

  This PR fixes it.

  Refs:
  - [Qt docs: qRegisterMetaType](https://doc.qt.io/qt-5/qmetatype.html#qRegisterMetaType)
  - #16348

  ---

  Side NOTE: Also I believe this line a7aec7ad97/src/qt/bitcoin.cpp (L63) is redundant since long `CAmount` is a `typedef`.

ACKs for top commit:
  laanwj:
    Tested ACK 1828c6f05f

Tree-SHA512: 2c7f9fe6a5ae70f2e1dd86b07f95d4b00c85c5706a9d722f063f80beb71880d012ec46556963fb1544c2af53d006936c2f7612eae60d9193f67db62ba3d86129
2019-11-10 11:38:53 +01:00
Wladimir J. van der Laan
ecc1a4ecd0 Merge #17431: Remove unnecessary forward declaration
3d133482b2 Remove unnecessary forward declaration (Mark Erhardt)

Pull request description:

  This removes an unnecessary forward declaration.

ACKs for top commit:
  jnewbery:
    Tested ACK 3d133482b2
  laanwj:
    ACK 3d133482b2

Tree-SHA512: 9e5b14e861c2b9fa2d7707ed67c4667540e9812a762e00f5039691eeca82390eb7462d20ad781d4e8c9111517e989da7aef60b112ab33abb774e32d9845b5459
2019-11-10 11:18:10 +01:00
Hennadii Stepanov
1828c6f05f refactor: Styling w/ clang-format, comment update 2019-11-10 12:11:07 +02:00
Hennadii Stepanov
88a94f7bb8 qt: Fix missing qRegisterMetaType for size_t
It is required in order to use size_t in QueuedConnections.
2019-11-10 12:03:53 +02:00
Wladimir J. van der Laan
cef7df37ce Merge #17410: Rename db log category to walletdb (like coindb)
e2c03c1156 doc: Add relase note for db→walletdb rename (Wladimir J. van der Laan)
4c1d263d93 scripted-diff: Change `BCLog::DB` to `BCLog::WALLETDB` (Wladimir J. van der Laan)
6b42b3ba90 Rename `db` log category to `walletdb` (like `coindb`) (Wladimir J. van der Laan)

Pull request description:

  Rename the `db` log category to `walletdb` (in the style of, and to distinguish from `coindb`). Deprecate (but still accept) '-debug=db'.

  Second commit is a scripted commit that changes the enum item name.

ACKs for top commit:
  hebasto:
    ACK e2c03c1156, tested on Linux Mint 19.2:

Tree-SHA512: a044de6f9a70e735cbb1caa4ed6bf75bc2269b2d5bc3241a25b6a6d69c1fc1d83456e252b431388ae61f4821e4fc06ecc1b634816ceadbe9a3c0e494bee6c11e
2019-11-10 10:50:58 +01:00
Mark Erhardt
3d133482b2 Remove unnecessary forward declaration 2019-11-09 22:08:29 -08:00
MarcoFalke
fa493ef088 ci: Make ci system read-only on the git work tree 2019-11-09 09:13:41 -05:00
MarcoFalke
fab1333292 ci: Remove git from required packages on host 2019-11-09 09:12:24 -05:00
MarcoFalke
fa00393bce ci: Make all filesystem operations inside docker 2019-11-09 09:11:32 -05:00
Wladimir J. van der Laan
a7aec7ad97 Merge #15934: Merge settings one place instead of five places
083c954b02 Add settings_tests (Russell Yanofsky)
7f40528cd5 Deduplicate settings merge code (Russell Yanofsky)
9dcb952fe5 Add util::Settings struct and helper functions. (Russell Yanofsky)
e2e37cfe8a Remove includeconf nested scope (Russell Yanofsky)
5a84aa880f Rename includeconf variables for clarity (Russell Yanofsky)
dc8e1e7548 Clarify emptyIncludeConf logic (Russell Yanofsky)

Pull request description:

  This is a refactoring-only change that makes it easier to add a new settings source.

  This PR doesn't change behavior. The [`util_ArgsMerge`](deb2327b43/src/test/util_tests.cpp (L626-L822)) and [`util_ChainMerge`](deb2327b43/src/test/util_tests.cpp (L843-L924)) tests added in #15869 and #15988 were written specifically to confirm that ArgsManager settings are parsed, merged, and returned the same way before and after this change.

  This change:

  - Makes it easier to add new settings sources that can get merged with existing sources (see 70675c3e4975203ad6222ba2b00c83b4e4213793 from #15935).
  - Separates parsing of settings from merging of settings, and deduplicates merging code so it doesn't happen five different places ([GetArg](c459c5f701/src/util/system.cpp (L221-L244)), [GetNetBoolArg](c459c5f701/src/util/system.cpp (L255-L261)), [GetArgs](c459c5f701/src/util/system.cpp (L460-L467)), [IsArgNegated](c459c5f701/src/util/system.cpp (L482-L491)), [GetUnsuitableSectionOnlyArgs](c459c5f701/src/util/system.cpp (L343-L352))) in inconsistent ways.
  - Documents and tests current strange merging behaviors, so they be cleaned up in the future if resulting code simplifications and UX improvements warrant loss of backwards compatibility. The newly documented behaviors are: command line [ignored arguments](69d44f3cc7/src/util/system.cpp (L323-L326)) and [more ignored arguments](69d44f3cc7/src/util/settings.cpp (L67-L72)), and config file [reverse precedence](69d44f3cc7/src/util/settings.cpp (L61-L65)), [inconsistently applied top-level settings](69d44f3cc7/src/util/settings.cpp (L55-L59)), and [zombie values](69d44f3cc7/src/util/settings.cpp (L101-L108)).

  The original motivation for this change was to make it easy to add a new persistent setting source without introducing more bugs and inconsistencies. Two commits building on top of this to add a persistent `-wallet` setting are pretty straightforward and show how the new code can be extended:

  * 70675c3e4975203ad6222ba2b00c83b4e4213793 from #15935 – _Add \<datadir>/settings.json persistent settings storage_
  * 04c80c40df9fc6f4734ba238ea7f65607cf88089 from #15937 – _Add loadwallet and createwallet RPC load_on_startup options_

ACKs for top commit:
  ariard:
    ACK 083c954
  jnewbery:
    ACK 083c954b02
  jamesob:
    ACK 083c954b02

Tree-SHA512: 5d106746a44d64d3963c4ef3f4a2fa668a4bedcc9018d3ea12c86beae2fda48a0b036241665837f68685712366f70f2e1faba84d193fa1f456013503097b7659
2019-11-08 23:23:08 +01:00
Wladimir J. van der Laan
adceca2ba5 Merge #17422: doc: add historical 0.19.0 release notes
fec230edcc doc: add historical 0.19.0 release notes (fanquake)

Pull request description:

ACKs for top commit:
  laanwj:
    ACK fec230edcc

Tree-SHA512: c0c83cd357e58acb7969b02a607fb6b40536f1c1072c25f58b0c6b6b9f50372f33403db51e846456b0f3f1cb3782a0b5c60604cb81e11a2cb99af664249c9069
2019-11-08 23:20:56 +01:00
Wladimir J. van der Laan
e2c03c1156 doc: Add relase note for db→walletdb rename 2019-11-08 23:05:47 +01:00
fanquake
fec230edcc doc: add historical 0.19.0 release notes 2019-11-08 16:20:37 -05:00
Wladimir J. van der Laan
9e0aabe50c Merge #17367: ci: Run non-cross-compile builds natively
faf757a125 ci: Guess the native host when not cross compiling (MarcoFalke)
fa8a60bce9 ci: Run non-cross-compile builds natively (MarcoFalke)
fa56bcbb01 ci: Run CI_WAIT only on travis (MarcoFalke)

Pull request description:

  non-cross-compile ci builds should not hardcode an architecture, so they can be run on any ci system

ACKs for top commit:
  laanwj:
    re-ACK faf757a125

Tree-SHA512: 97f86ad411e98c6317a62f829bee26c16dbe3fa54d8ac013018f7669b653d7d6d750740b2ecfb7175195d5fffc701ce503b0d11802b97af30904b51bb23f2073
2019-11-08 22:09:31 +01:00
Wladimir J. van der Laan
4c1d263d93 scripted-diff: Change BCLog::DB to BCLog::WALLETDB
-BEGIN VERIFY SCRIPT-
git grep -l "BCLog::DB" src | xargs sed -i "s/BCLog::DB/BCLog::WALLETDB/g"
sed -i "s/DB          =/WALLETDB    =/g" src/logging.h
-END VERIFY SCRIPT-
2019-11-08 18:45:38 +01:00
Wladimir J. van der Laan
6b42b3ba90 Rename db log category to walletdb (like coindb)
Deprecate (but still accept) '-debug=db'.
2019-11-08 18:28:26 +01:00
fanquake
8021392b82 Merge #17405: wallet: Remove unused boost::this_thread::interruption_point
fad1de66a2 wallet: Remove unused boost::this_thread::interruption_point (MarcoFalke)

Pull request description:

  `BerkeleyEnvironment::Open` is only called from the main thread (init) or an http rpc thread, neither of which can be interrupted, so remove the useless interruption point.

  `BerkeleyEnvironment{}` is only used in tests, which run in a single process/thread, so remove the useless interruption point.

ACKs for top commit:
  laanwj:
    ACK fad1de66a2
  fanquake:
    ACK fad1de66a2

Tree-SHA512: dacd8398e966e4a6ce5cf7d3ed821c9c267eff40b14c0635085441647cdb72d1642807f89355419f1710f814c7963e35a10d102d0b985c7198261dfc736256f8
2019-11-08 09:01:09 -05:00
fanquake
4a3b6f47cd Merge #17354: wallet: Tidy CWallet::SetUsedDestinationState
0b75a7f068 wallet: Reuse existing batch in CWallet::SetUsedDestinationState (João Barbosa)
01f45dd00e wallet: Avoid recursive lock in CWallet::SetUsedDestinationState (João Barbosa)

Pull request description:

  This PR makes 2 distinct changes around `CWallet::SetUsedDestinationState`:
   - 1st the recursive lock is removed and now it requires the lock to be held;
   - 2nd change is to support, in the best case, just a wallet database flush when transaction is added to the wallet.

ACKs for top commit:
  achow101:
    ACK 0b75a7f068
  MarcoFalke:
    ACK 0b75a7f068
  ryanofsky:
    Code review ACK 0b75a7f068. Code changes looks fine but PR description should be updated to say what benefits of the change are. I might have missed something, but I didn't see a place where multiple batches were used previously and a single batch was used now. So the main benefit of this change appears to be removing a recursive lock? And maybe moving toward a consistent convention for passing batch instances?

Tree-SHA512: abcf23a5850d29990668db20d6f624cca3e89629cc9ed003e0d05cde1b58ab2ff365034f156684ad13e55764b54c6c0c2bc7d5f96b8af7dc5e45a3be955d6b15
2019-11-08 08:44:49 -05:00
Samuel Dobson
99ab3a72c5 Merge #15931: Remove GetDepthInMainChain dependency on locked chain interface
36b68de5b2 Remove getBlockDepth method from Chain::interface (Antoine Riard)
b66c429c56 Remove locked_chain from GetDepthInMainChain and its callers (Antoine Riard)
0ff03871ad Use CWallet::m_last_block_processed_height in GetDepthInMainChain (Antoine Riard)
f77b1de16f Only return early from BlockUntilSyncedToCurrentChain if current tip is exact match (Antoine Riard)
769ff05e48 Refactor some importprunedfunds checks with guard clause (Antoine Riard)
5971d3848e Add block_height field in struct Confirmation (Antoine Riard)
9700fcb47f Replace CWalletTx::SetConf by Confirmation initialization list (Antoine Riard)
5aacc3eff1 Add m_last_block_processed_height field in CWallet (Antoine Riard)
10b4729e33 Pass block height in Chain::BlockConnected/Chain::BlockDisconnected (Antoine Riard)

Pull request description:

  Work starter to remove Chain::Lock interface by adding m_last_block_processed_height in CWallet and m_block_height in CMerkleTx to avoid GetDepthInMainChain having to keep a lock . Once this one done, it should ease work to wipe out more cs_main locks from wallet code.

  I think it's ready for a first round of review before to get further.

  - `BlockUntilSyncedToCurrent` : restrain isPotentialTip to isTip because we want to be sure that wallet see BlockDisconnected callbacks if its height differs from the Chain one. It means during a reorg, an RPC could return before the BlockDisconnected callback had been triggered. This could cause a tx that had been included in the disconnected block to be displayed as confirmed, for example.

  ~~- `AbandonTransaction` : in case of conflicted tx (nIndex = -1), we set its m_block_height to the one of conflicting blocks, but if this height is superior to CWallet::m_last_block_processed_height, that means tx isn't conflicted anymore so we return 0 as tx is again unconfirmed~~ After #16624, we instead rely on Confirmation.

  ~~- `AddToWalletIfInvolvingMe`: in case of block disconnected, transactions are added to mempool again, so we need to replace old txn in `mapWallet` with a height set to zero so we remove check on block_hash.IsNull~~ Already done in #16624

ACKs for top commit:
  jnewbery:
    @jkczyz you've ACKed an intermediate commit (github annoyingly orders commits in date order, not commit order). Did you mean to ACK the final commit in this branch (36b68de5b2).
  jkczyz:
    > @jkczyz you've ACKed an intermediate commit (github annoyingly orders commits in date order, not commit order). Did you mean to ACK the final commit in this branch ([36b68de](36b68de5b2)).
  meshcollider:
    utACK 36b68de5b2
  ryanofsky:
    Code review ACK 36b68de5b2. Changes since last review: new jkczyz refactor importprunedfunds commit, changed BlockUntilSyncedToCurrentChainChanges commit title and description, changed Confirmation struct field order and line-wrapped comment
  jnewbery:
    utACK 36b68de5b2
  promag:
    Code review ACK 36b68de5b2.

Tree-SHA512: 08b89a0bcc39f67c82a6cb6aee195e6a11697770c788ba737b90986b4893f44e90d1ab9ef87239ea3766508b7e24ea882b7199df41173ab27a3d000328c14644
2019-11-08 23:23:14 +13:00
MarcoFalke
fab9850ef4 scripted-diff: Avoid hardcoded libfaketime dir in gitian
-BEGIN VERIFY SCRIPT-
sed -i -e "s|'export LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1'|\"export LD_PRELOAD='/usr/\\\\\$LIB/faketime/libfaketime.so.1'\"|g" $(git grep -l 'x86_64-linux-gnu/faketime')
sed -i -e 's|"amd64"|"linux64"|g' $(git grep -l '"amd64"')
-END VERIFY SCRIPT-
2019-11-07 22:35:51 -05:00
Russell Yanofsky
083c954b02 Add settings_tests
Co-authored-by: James O'Beirne <james.obeirne@gmail.com>
2019-11-07 23:08:22 -04:00
Russell Yanofsky
7f40528cd5 Deduplicate settings merge code
Get rid of settings merging code in util/system.cpp repeated 5 places,
inconsistently:

- ArgsManagerHelper::GetArg
- ArgsManagerHelper::GetNetBoolArg
- ArgsManager::GetArgs
- ArgsManager::IsArgNegated
- ArgsManager::GetUnsuitableSectionOnlyArgs

Having settings merging code separated from parsing simplifies parsing somewhat
(for example negated values can simply be represented as false values instead
of partially cleared or emply placeholder lists).

Having settings merge happen one place instead of 5 makes it easier to add new
settings sources and harder to introduce new inconsistencies in the way
settings are merged.

This commit does not change behavior in any way.
2019-11-07 23:08:22 -04:00
Russell Yanofsky
9dcb952fe5 Add util::Settings struct and helper functions.
Implement merging of settings from different sources (command line and config
file) separately from parsing code in system.cpp, so it is easier to add new
sources.

Document current inconsistent merging behavior without changing it.

This commit only adds new settings code without using it. The next commit calls
the new code to replace existing code in system.cpp.

Co-authored-by: John Newbery <john@johnnewbery.com>
2019-11-07 22:08:22 -05:00
Russell Yanofsky
e2e37cfe8a Remove includeconf nested scope
Easier to review ignoring whitespace

Suggestion from John Newbery <john@johnnewbery.com> in
https://github.com/bitcoin/bitcoin/pull/15934#discussion_r343806860
2019-11-07 22:08:22 -05:00
Russell Yanofsky
5a84aa880f Rename includeconf variables for clarity
includeconf -> conf_file_names
to_include -> conf_file_name
include_config -> conf_file_stream

Suggestion from John Newbery <john@johnnewbery.com> in
https://github.com/bitcoin/bitcoin/pull/15934#discussion_r343905138
2019-11-07 22:04:42 -05:00
Russell Yanofsky
dc8e1e7548 Clarify emptyIncludeConf logic
Suggestion from John Newbery <john@johnnewbery.com> in
https://github.com/bitcoin/bitcoin/pull/15934#discussion_r343795528
2019-11-07 22:01:22 -05:00
MarcoFalke
fad1de66a2 wallet: Remove unused boost::this_thread::interruption_point 2019-11-07 16:01:34 -05:00
fanquake
2fb6140d58 Merge #17404: Remove redundant class file include from test_bitcoin msvc project
b80f7db832 Remove redundant class file includes from test_bitcoin project. (Aaron Clauson)

Pull request description:

  #17364 & #17384 overlapped and both added the same line of `..\..\src\test\util\*.cpp` to `test_bitcoin.vcxproj`. This didn't break the build but does result in duplicate symbol warnings. This PR cleans it up and removes the additional redundant line of `..\..\src\test\util\setup_common.cpp` which will also be covered by the wildcard include.

ACKs for top commit:
  MarcoFalke:
    ACK b80f7db832 🔅
  fanquake:
    ACK b80f7db832 - tested a build on a Windows machine. No longer see the warnings shown below:

Tree-SHA512: 55960821480483c517b475f2a6871cd7d4033d086db3fd679aa0de362e4f7e2c3ac7967ca278cc3728cc765ba23d4441ec769d83d7a47e7a3fa2f09de2bbc145
2019-11-07 15:42:25 -05:00
Aaron Clauson
b80f7db832 Remove redundant class file includes from test_bitcoin project. 2019-11-07 20:03:23 +00:00
John Newbery
3bd8db80d8 [validation] fix comments in CheckInputScripts() 2019-11-07 13:51:02 -05:00
John Newbery
6f6465cefc scripted-diff: [validation] Rename CheckInputs to CheckInputScripts
CheckInputs() used to check no double spends, scripts & sigs and amounts. Since
832e0744cb, the double spend and amount checks
have been moved to CheckTxInputs(), and CheckInputs() now just validates
input scripts. Rename the function to CheckInputScripts().

-BEGIN VERIFY SCRIPT-
sed -i -E -e 's/CheckInputs\b/CheckInputScripts/g' $(git grep -l CheckInputs | grep -v doc/)
-END VERIFY SCRIPT-
2019-11-07 13:50:58 -05:00
MarcoFalke
faf757a125 ci: Guess the native host when not cross compiling 2019-11-07 13:42:57 -05:00
Pieter Wuille
e6e622e5a0 Implement O(1) OP_IF/NOTIF/ELSE/ENDIF logic
This optimization was first suggested by Sergio Demian Lerner in
https://bitslog.wordpress.com/2017/04/17/new-quadratic-delays-in-bitcoin-scripts/.
The implementation follows the suggested approach there, but with a slightly
simpler representation.
2019-11-07 09:12:26 -08:00
Anthony Towns
d0e8f4d5d8 [refactor] interpreter: define interface for vfExec
Includes comments added by Pieter Wuille.
2019-11-07 09:12:26 -08:00
Pieter Wuille
89fb241c54 Benchmark script verification with 100 nested IFs 2019-11-07 09:12:26 -08:00
fanquake
270616228b Merge #17362: test: speed up wallet_avoidreuse, add logging
0e7c90eb37 test: speed up wallet_avoidreuse.py (Jon Atack)
6d50b2606e test: add logging to wallet_avoidreuse.py (Jon Atack)

Pull request description:

  Inspired by PRs #17340 and #15881.

  - add logging
  - pass -whitelist in `set_test_params` to speed up transaction relay

  `wallet_avoidreuse.py` is not intended to test P2P transaction relay/timing, so it should be fine to do this here. This reduces test run time variability and speeds up the test by 2-3 times on average.

  Test run times in seconds:

  - before: 20, 24, 22, 17, 27, 40, 30

  - after: 10, 10, 8, 9, 10, 7, 8

ACKs for top commit:
  MarcoFalke:
    ACK 0e7c90eb37 🐊
  fanquake:
    ACK 0e7c90eb37

Tree-SHA512: 6d954a0aaf402c9594201626b59d29263479059e68fa5155bb44ed973cd0c3347729dd78b78b4d5a2275e45da365dc1afb4cc7e3293dea33fcc2e3e83a39faf5
2019-11-07 11:59:51 -05:00
MarcoFalke
772673dfbe Merge #16978: test: Seed test RNG context for each test case, print seed
fae43a97ca test: Seed test RNG context for each test case, print seed (MarcoFalke)

Pull request description:

  Debugging failing unit tests is hard if the failure is non-deterministic and the seed is not known.

  Fix that by printing the seed and making it possible to set the seed from outside.

ACKs for top commit:
  davereikher:
    Tested ACK fae43a97ca

Tree-SHA512: 33d848dd1f4180d3664ecf60e9810c2a93590c05276b2c46b1e4fe6e376b45916a46b90c803bb602750ab666da3a05ce499e550024685a90b8cc38fab6667cb8
2019-11-07 10:18:40 -05:00
MarcoFalke
7d14e35f3f Merge #17342: refactor: Clean up nScriptCheckThreads
5506ecfe7a [refactor] Replace global int nScriptCheckThreads with bool (John Newbery)
d9957623b4 [tests] Don't use TestingSetup in the checkqueue_tests (John Newbery)

Pull request description:

  The meaning of this value is confusing. Refactor it and add comments.

ACKs for top commit:
  sipa:
    ACK 5506ecfe7a
  promag:
    ACK 5506ecfe7a, only change was addressing my nits.
  laanwj:
    Code review ACK 5506ecfe7a
  MarcoFalke:
    ACK 5506ecfe7a 🥐

Tree-SHA512: 78536727c98d2c23f3c0f3f169131474fef9a4486ae65029011caf06eab30f6f70ff73a65b2fb04a5d969fc1150858d1c6ea4767f04d48c1eea6b829316d0e63
2019-11-07 10:07:11 -05:00
MarcoFalke
46e0e27639 Merge #17364: Updates to appveyor config for VS2019 and Qt5.9.8 + msvc project fixes
3c84deebaa Updated appveyor config:  - Update build image from Visual Studio 2017 to Visual Studio 2019.  - Updated Qt static library from Qt5.9.7 to Qt5.9.8.  - Added commands to update vcpkg port files (this does not update already installed packages).  - Updated vcpkg package list as per #17309.  - Removed commands setting common project file options. Now done via common.init.vcxproj include.  - Changed msbuild verbosity from normal to quiet. Normal rights a LOT of logs and impacts appveyor job duration. Updated msvc project configs:  - Updated platform toolset from v141 to v142.  - Updated Qt static library from Qt5.9.7 to Qt5.9.8.  - Added ignore for linker warning building bitcoin-qt program.  - Added missing util/str.cpp class file to test_bitcoin project file. (Aaron Clauson)

Pull request description:

  Updates to appveyor config:
   - Update build image from Visual Studio 2017 to Visual Studio 2019.
   - Updated Qt static library from Qt5.9.7 to Qt5.9.8.
   - Added commands to update vcpkg port files (this does not update already installed packages).
   - Updated vcpkg package list as per #17309.
   - Removed commands setting common project file options. Now done via common.init.vcxproj include.
   - Changed msbuild verbosity from normal to quiet. Normal writes a LOT of logs and impacts appveyor job duration.

  Updates to msvc project configs:
   - Updated platform toolset from v141 to v142.
   - Updated Qt static library from Qt5.9.7 to Qt5.9.8.
   - Added ignore for linker warning building bitcoin-qt program.
   - Added missing util/str.cpp class file to test_bitcoin project file.

  In order for an existing appveyor job based on the new config to work the cache must be purged. The steps to do this are shown below. The specific appveyor project path will need to be adjusted.

  ````
  export APPVEYOR_TOKEN="<your-api-token>"
  curl -H "Authorization: Bearer $APPVEYOR_TOKEN" -X DELETE https://ci.appveyor.com/api/projects/bitcoin/bitcoin-9ql6k/buildcache
  ````

ACKs for top commit:
  ryanofsky:
    Non-expert code review ACK 3c84deebaa.

Tree-SHA512: 77063d4588c3499de78b0bcc4d8b638f36c70284485ae94ce5c718a3dacb6d28cc34f9443c54c4e98c07b446d26b59589259671c2f6bcc952344042b4a3baf8f
2019-11-07 09:45:47 -05:00
MarcoFalke
46fc4d1a24 Merge #17384: test: Create new test library
fa4c6fa9b1 doc: Add documentation for new test/lib (MarcoFalke)
faec28252c scripted-diff: test: Move setup_common to test library (MarcoFalke)

Pull request description:

  Sorry for clickbait, this is only a move-only scripted-diff commit and one documentation commit.

  Longer term, someone who knows something about build systems can make this an actual library. Motivation for this is that each module gets compiled for each target that includes it. For example, setup_common is compiled 27 times (for the fuzz suite) and another 3 times for the other tests (bench, unit test, gui)

ACKs for top commit:
  practicalswift:
    ACK fa4c6fa9b1 -- diff looks correct and Travis is happy
  jonatack:
    ACK fa4c6fa9b1 with the reserve that the commit messages (and PR description) contain the motivation for this change. Built, ran tests, light code review.
  ryanofsky:
    Code review ACK fa4c6fa9b1. I didn't realize `lib` was actually name of existing directory, not a new name. But in any case this looks good and nice to have one scripted diff instead of two.

Tree-SHA512: 2e176df90c60578276e4a6dc83ff57ff59d8e666ecf30c5ceacb8c326725da91baa4cac3dfa7a2e1605f58122a3e3e27e4938ff33e3a0ce7ea53afffebbf57a4
2019-11-07 08:02:25 -05:00
Jon Atack
0e7c90eb37 test: speed up wallet_avoidreuse.py
Use -whitelist to speed up transaction relay.

The wallet_avoidreuse.py test is not intended to test transaction relay/timing,
so it should be fine to do this here.

This greatly reduces test run time variability and speeds up the test by 2-3
times on average, e.g. on my system from 20-30 seconds down to 8-10 seconds.
2019-11-07 10:03:28 +01:00
Jon Atack
6d50b2606e test: add logging to wallet_avoidreuse.py 2019-11-07 10:03:26 +01:00
Gr0kchain
582e66b6e7 doc: Added regtest config for linearize script
Updated the example-linearize.cfg file to include support for the regtest chain network config which is used by the ./linearize-data.py

Problem:

Without the regtest magic, genesis hash and path config, the `linearize-data.py` script cannot generate a bootstrap.dat file.

Example of error:

./linearize-data.py ./linearize.cfg
Read 102 hashes
Genesis block not found in hashlist

Solution:

Added netmagic, genesis and input example parameters to file.

Resolution

1. Starting bitcoind in regtest mode
2. bitcoin-cli generatetoaddress 101 $(bitcoin-cli getnewaddress)
3. ./linearize-hashes.py ./linearize.cfg > ./hashlist.txt
4. ./linearize-data.py ./linearize.cfg

Example after fix:
$ ./linearize-data.py ./linearize.cfg
Read 102 hashes
Input file /Users/gr0kchain/.bitcoin/regtest/blocks/blk00000.dat
Output file /Users/gr0kchain/Downloads/bootstrap.dat
Done (102 blocks written)
2019-11-07 08:07:10 +02:00
Aaron Clauson
3c84deebaa Updated appveyor config:
- Update build image from Visual Studio 2017 to Visual Studio 2019.
 - Updated Qt static library from Qt5.9.7 to Qt5.9.8.
 - Added commands to update vcpkg port files (this does not update already installed packages).
 - Updated vcpkg package list as per #17309.
 - Removed commands setting common project file options. Now done via common.init.vcxproj include.
 - Changed msbuild verbosity from normal to quiet. Normal rights a LOT of logs and impacts appveyor job duration.
Updated msvc project configs:
 - Updated platform toolset from v141 to v142.
 - Updated Qt static library from Qt5.9.7 to Qt5.9.8.
 - Added ignore for linker warning building bitcoin-qt program.
 - Added missing util/str.cpp class file to test_bitcoin project file.
2019-11-06 22:02:16 +00:00
MarcoFalke
e65b4160e9 Merge #17340: Tests: speed up fundrawtransaction test
af7bae7340 [tests] Don't stop-start unnecessarily in rpc_fundrawtransaction.py (John Newbery)
9a8505299b [tests] Use -whitelist in rpc_fundrawtransaction.py (John Newbery)
646b593bbd [tests] Speed up rpc_fundrawtransaction.py (John Newbery)

Pull request description:

  Speed up rpc_fundrawtransaction.py

  Most of the time in rpc_fundrawtransaction.py is spent waiting for
  unconfirmed transactions to propagate. Net processing adds a poisson
  random delay to the time it will INV transactions with a mean interval
  of 5 seconds. Calls like the following:

  ```
  self.nodes[2].sendrawtransaction(signedTx['hex'])
  self.sync_all()
  self.nodes[1].generate(1)
  ````

  will therefore introduce a delay waiting for the mempools to sync.
  Instead just generate the block on the node that sent the transaction:

  ```
  self.nodes[2].sendrawtransaction(signedTx['hex'])
  self.nodes[2].generate(1)
  ```

  rpc_fundrawtransaction.py is not intended to be a test for transaction
  relay, so it's ok to do this.

ACKs for top commit:
  MarcoFalke:
    ACK af7bae7340 🛴

Tree-SHA512: db3407d871bfdc99a02e7304b07239dd3585ac47f27f020f1a70608b7f6386b134343c01f3e4d1c246ce734676755897671999695068d6388602fb042d178780
2019-11-06 15:18:41 -05:00
John Newbery
5506ecfe7a [refactor] Replace global int nScriptCheckThreads with bool
The global nScriptCheckThreads int is confusing and is only needed for
its int-ness in AppInitMain. Move all `-par` parsing logic there and
replace the int nScriptCheckThreads with a bool
g_parallel_script_checks.

Also tidy up logic and improve comments.
2019-11-06 15:04:50 -05:00
John Newbery
d9957623b4 [tests] Don't use TestingSetup in the checkqueue_tests
It's only needed for a hardcoded int, which we can define locally.
2019-11-06 15:03:59 -05:00
John Newbery
af7bae7340 [tests] Don't stop-start unnecessarily in rpc_fundrawtransaction.py
This was only added in c1dde3a949 to match
behaviour when `encryptwallet` would restart the node. It's not required
for the test (and slows things down).
2019-11-06 14:56:35 -05:00
John Newbery
9a8505299b [tests] Use -whitelist in rpc_fundrawtransaction.py
Makes tx relay faster
2019-11-06 14:56:29 -05:00
Antoine Riard
36b68de5b2 Remove getBlockDepth method from Chain::interface
Pass conflicting height in CWallet::MarkConflicted
2019-11-06 13:36:43 -05:00
Antoine Riard
b66c429c56 Remove locked_chain from GetDepthInMainChain and its callers
We don't remove yet Chain locks as we need to preserve lock
order with CWallet one until swapping at once to avoid
deadlock failures (spotted by --enable-debug)
2019-11-06 13:36:43 -05:00
Antoine Riard
0ff03871ad Use CWallet::m_last_block_processed_height in GetDepthInMainChain
Avoid to lock chain to query state thanks to tracking last block
height in CWallet.
2019-11-06 13:36:43 -05:00
Antoine Riard
f77b1de16f Only return early from BlockUntilSyncedToCurrentChain if current tip
is exact match

In the next commit, we start using BlockConnected/BlockDisconnected
callbacks to establish tx depth, rather than querying the chain
directly.

Currently, BlockUntilSyncedToCurrentChain will return early if
the best block processed by the wallet is a descendant of the node'tip.
That means that in the case of a re-org, it won't wait for the
BlockDisconnected callbacks that have been enqueued during the re-org
but have not yet been triggered in the wallet.

Change BlockUntilSyncedToCurrentChain to only return early if the
wallet's m_last_block_processed matches the tip exactly. This ensures
that there are no BlockDisconnected or BlockConnected callbacks
in-flight.
2019-11-06 13:36:43 -05:00
Antoine Riard
769ff05e48 Refactor some importprunedfunds checks with guard clause
Credit to jkczyz
2019-11-06 13:36:43 -05:00
Antoine Riard
5971d3848e Add block_height field in struct Confirmation
At wallet loading, we rely on chain state querying to retrieve
height of txn, to do so we ensure that lock order is respected
between cs_main and cs_wallet.

If wallet loaded is the wallet-tool one, all wallet txn will
show up with a height of zero. It doesn't matter as confirmation
height is not used by wallet-tool.

Reorder arguments and document Confirmation calls to avoid
ambiguity.

Fixes nits left from #16624
2019-11-06 13:29:53 -05:00
MarcoFalke
fa4c6fa9b1 doc: Add documentation for new test/lib 2019-11-06 11:56:53 -05:00
MarcoFalke
faec28252c scripted-diff: test: Move setup_common to test library
-BEGIN VERIFY SCRIPT-
 # Move files
 for f in $(git ls-files src/test/lib/); do git mv $f src/test/util/; done
 git mv src/test/setup_common.cpp                     src/test/util/
 git mv src/test/setup_common.h                       src/test/util/
 # Replace Windows paths
 sed -i -e 's|\\setup_common|\\util\\setup_common|g' $(git grep -l '\\setup_common')
 sed -i -e 's|src\\test\\lib\\|src\\test\\util\\|g'  build_msvc/test_bitcoin/test_bitcoin.vcxproj
 # Everything else
 sed -i -e 's|/setup_common|/util/setup_common|g'    $(git grep -l 'setup_common')
 sed -i -e 's|test/lib/|test/util/|g'                $(git grep -l 'test/lib/')
 # Fix include guard
 sed -i -e 's|BITCOIN_TEST_SETUP_COMMON_H|BITCOIN_TEST_UTIL_SETUP_COMMON_H|g' ./src/test/util/setup_common.h
 sed -i -e 's|BITCOIN_TEST_LIB_|BITCOIN_TEST_UTIL_|g'                     $(git grep -l 'BITCOIN_TEST_LIB_')
-END VERIFY SCRIPT-
2019-11-06 11:56:41 -05:00
Antoine Riard
9700fcb47f Replace CWalletTx::SetConf by Confirmation initialization list 2019-11-06 11:35:39 -05:00
Wladimir J. van der Laan
976cc766c4 Merge #17381: LegacyScriptPubKeyMan code cleanups
05b224a175 Add missing SetupGeneration error handling in EncryptWallet (Russell Yanofsky)
bfd826a675 Clean up nested scope in GetReservedDestination (Russell Yanofsky)
491a599b37 Get rid of confusing LegacyScriptPubKeyMan::TopUpKeyPool method (Russell Yanofsky)
4a0abf694e Pass CTxDestination to ScriptPubKeyMan::GetMetadata (Russell Yanofsky)
b07b07cd87 Add EnsureLegacyScriptPubKeyMan and use in rpcwallet.cpp (Russell Yanofsky)

Pull request description:

  This PR implements suggested code cleanups from #17300 and #17304 review comments

ACKs for top commit:
  Sjors:
    re-ACK 05b224a
  laanwj:
    Code review ACK 05b224a175

Tree-SHA512: 12fd86637088515b744c028e0501c5d21a9cf9ee9c9cfd70e9cb65d44611ea5643abd5f6f101105caa5aff015d74de606f074f08af7dae8429f929d21288ab45
2019-11-06 17:28:58 +01:00
Wladimir J. van der Laan
6f4e247357 Merge #17390: test: Add util_ArgParsing test
286f197704 Add util_ArgParsing test (Russell Yanofsky)

Pull request description:

  ArgsManager test coverage for parsing of integer and boolean values is
  currently very poor and doesn't give us a way of knowing whether changes to
  ArgsManager may unintentionally break backwards compatibility, so this adds a
  new test to catch regressions.

ACKs for top commit:
  promag:
    ACK 286f197, more surprising results 😱
  laanwj:
    ACK 286f197704

Tree-SHA512: 9e1db3ef87e55abbc280af60c088f35765a1f9e2ec20507ad0c1992027b875490016868dcb8cc287e6df279dd0e00f10550901af3de3d36287867249e0bd8207
2019-11-06 17:01:21 +01:00
Hennadii Stepanov
577682d9e8 script: Enable SC2006 rule for Gitian scripts 2019-11-06 15:11:54 +02:00
Hennadii Stepanov
14aded46df script: Lint Gitian descriptors with ShellCheck 2019-11-06 15:10:11 +02:00
Wladimir J. van der Laan
86771d4310 Merge #17339: doc: Add template for good first issues
7b78b8d3a6 doc: Add template for good first issues (Michael Folkson)

Pull request description:

  closes #17317

  Attempted to address everyone's suggestions in #17317 without making it too long. The first half is for the benefit of the individual opening the issue and the second half is for the benefit of the new contributor. Ideally we don't want the second half to be deleted by the individual opening the issue but whether they delete the first half or not isn't really a concern

ACKs for top commit:
  MarcoFalke:
    ACK 7b78b8d3a6
  jonatack:
    ACK 7b78b8d3a6

Tree-SHA512: 5874b244a52f432637600a73aac493972971568f8d8af10aa731b8a6b221566015827dd82c310c60a76fb01140c3bc56a691206c3442018611c820d4b98d104f
2019-11-06 14:00:42 +01:00
MarcoFalke
22a58811d4 Merge #17353: doc: Add ShellCheck to lint tests dependencies
80c9e66ab8 build: Remove install command samples (Hennadii Stepanov)
2ad74b78c6 doc: Add ShellCheck to lint tests dependencies (Hennadii Stepanov)

Pull request description:

  In master (9641366950) the lint tests dependencies list lacks ShellCheck. This PR fixes it.

  Also `lint-python.sh` is slightly improved.

ACKs for top commit:
  laanwj:
    ACK 80c9e66ab8
  promag:
    ACK 80c9e66ab8, verified internal and external links. Nice looking table.

Tree-SHA512: b63718a6c41be93137db70586465d84ca0b1ff33c0f2674147c928cb1bdf903ec7587861c09ad832841264285f99c7b171d5319eef3c989822a7cd01449222ae
2019-11-06 07:41:31 -05:00
Hennadii Stepanov
80c9e66ab8 build: Remove install command samples
test/README.md contains comprehensive install instructions.
2019-11-06 13:22:06 +02:00
Wladimir J. van der Laan
7967104aee Merge #17368: cli: fix -getinfo output when compiled with no wallet
3d05d33269 cli: fix -getinfo output when compiled with no wallet (fanquake)

Pull request description:

  master (33b155f287):
  ```bash
  src/bitcoin-cli -getinfo
  {
    "version": 199900,
    "protocolversion": 70015,
    "blocks": 602348,
    "headers": 602348,
    "verificationprogress": 0.9999995592310106,
    "timeoffset": 0,
    "connections": 10,
    "proxy": "",
    "difficulty": 13691480038694.45,
    "chain": "main",
    "walletversion": null,
    "balance": null,
    "keypoololdest": null,
    "keypoolsize": null,
    "paytxfee": null,
    "relayfee": 0.00001000,
    "warnings": "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
  }
  ```

  This PR (3d05d33269):
  ```bash
  {
    "version": 199900,
    "protocolversion": 70015,
    "blocks": 602348,
    "headers": 602348,
    "verificationprogress": 0.9999996313568186,
    "timeoffset": 0,
    "connections": 10,
    "proxy": "",
    "difficulty": 13691480038694.45,
    "chain": "main",
    "relayfee": 0.00001000,
    "warnings": "This is a pre-release test build - use at your own risk - do not use for mining or merchant applications"
  }
  ```

ACKs for top commit:
  MarcoFalke:
    ouch ACK 3d05d33269
  laanwj:
    ACK 3d05d33269
  darosior:
    ACK 3d05d33269

Tree-SHA512: 055424e122a082cbfea410da287d9ceb7ed405fd68d53e2f5bef62beea80bc374a7d00366de0479d23faecb7f063b232aca52e9fdbdb97c58ddf46e7749136a9
2019-11-06 11:07:12 +01:00
Wladimir J. van der Laan
224c19645f Merge #17388: Add missing newline in util_ChainMerge test
3645e4ca00 Add missing newline in util_ChainMerge test (Russell Yanofsky)

Pull request description:

  This was causing a lot of test cases not not be very meaningful because
  multiple configuration options were combined into one line.

  The changes in test output with this fix make sense and look like:

  ```diff
  - testnet=1 regtest=1 || test
  + testnet=1 regtest=1 || error: Invalid combination of -regtest, -testnet and -chain. Can use at most one.
  ```

  Issue was reported and debugged by
  Wladimir J. van der Laan <laanwj@protonmail.com> in
  https://github.com/bitcoin/bitcoin/pull/17385#issuecomment-550033222

  <!--
  *** Please remove the following help text before submitting: ***

  Pull requests without a rationale and clear improvement may be closed
  immediately.
  -->

  <!--
  Please provide clear motivation for your patch and explain how it improves
  Bitcoin Core user experience or Bitcoin Core developer experience
  significantly:

  * Any test improvements or new tests that improve coverage are always welcome.
  * All other changes should have accompanying unit tests (see `src/test/`) or
    functional tests (see `test/`). Contributors should note which tests cover
    modified code. If no tests exist for a region of modified code, new tests
    should accompany the change.
  * Bug fixes are most welcome when they come with steps to reproduce or an
    explanation of the potential issue as well as reasoning for the way the bug
    was fixed.
  * Features are welcome, but might be rejected due to design or scope issues.
    If a feature is based on a lot of dependencies, contributors should first
    consider building the system outside of Bitcoin Core, if possible.
  * Refactoring changes are only accepted if they are required for a feature or
    bug fix or otherwise improve developer experience significantly. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.
  -->

  <!--
  Bitcoin Core has a thorough review process and even the most trivial change
  needs to pass a lot of eyes and requires non-zero or even substantial time
  effort to review. There is a huge lack of active reviewers on the project, so
  patches often sit for a long time.
  -->

ACKs for top commit:
  laanwj:
    ACK 3645e4ca00
  practicalswift:
    ACK 3645e4ca00 -- diff looks correct

Tree-SHA512: ca5bde9b9f553811d4827113f4880d15d7b8f4f1455b95bbf34c9a1512fdd53062f1a2133c50d9b54f94160a1ee77a54bc82681a5f3bf25d2b0d01f8a8e95165
2019-11-06 09:53:38 +01:00
fanquake
4e21f72980 Merge #17370: doc: Update doc/bips.md with recent changes in master
fa7f5a4d2a doc: Update doc/bips.md with recent changes in master (MarcoFalke)

Pull request description:

  Follow-up to #17165

ACKs for top commit:
  jonatack:
    ACK fa7f5a4d2a. Verified markdown view at https://github.com/MarcoFalke/bitcoin-core/blob/1911-docBips/doc/bips.md and the urls in the links. Some of the PRs are indicated with # and some without, but this is the case over the whole document.
  laanwj:
    ACK fa7f5a4d2a
  fanquake:
    ACK fa7f5a4d2a

Tree-SHA512: 31782b5f1f2f10b1189f05f010f908c183dbe723477ca1c46ad1d3bee5ea483335847008a7fe48d72373ccd39b84e0b950d0d1b23e457cb70f34210c5f2dc6aa
2019-11-05 21:37:02 -05:00
Russell Yanofsky
286f197704 Add util_ArgParsing test
ArgsManager test coverage for parsing of integer and boolean values is
currently very poor and doesn't give us a way of knowing whether changes to
ArgsManager may unintentionally break backwards compatibility, so this adds a
new test to catch regressions.
2019-11-05 18:41:49 -05:00
Andrew Chow
b84e776fd1 wallet_importmulti: use addresses of the same type as being imported
When constructing an import from the solving data of an address,
make sure that the original address is the same type as the one that
will be imported.
2019-11-05 18:31:10 -05:00
Wladimir J. van der Laan
45e65376ac Merge #17382: rpc: Remove unused boost::this_thread::interruption_point
fa5facd3e7 rpc: Remove unused boost::this_thread::interruption_point (MarcoFalke)

Pull request description:

  There are predefined interruption points for `boost::thread`: https://www.boost.org/doc/libs/1_71_0/doc/html/thread/thread_management.html#interruption_points

  However, the rpc threads are `std::thread`, which does not have an `std::thread::interrupt` member function to request interruption: https://dev.visucore.com/bitcoin/doxygen/httpserver_8cpp.html#ae1a63374e18b9abd348eb74e4243ea34

  Thus, the interruption points can be removed.

ACKs for top commit:
  laanwj:
    ACK fa5facd3e7, this does nothing.
  practicalswift:
    ACK fa5facd3e7
  jamesob:
    ACK fa5facd3e7

Tree-SHA512: 4e29a44df1f2702cbd1ffdffa559440a8bb800baab64b4116e2c3d27cd64d8d1e8aafe1dc21b1a4e3988470d03be19cae294bd5669f7abf6d487685dc8fd8d7e
2019-11-06 00:04:38 +01:00
Wladimir J. van der Laan
40b6070ad7 Merge #16805: logs: add timing information to FlushStateToDisk()
dcef9a2922 logs: add timing information to FlushStateToDisk() (James O'Beirne)
41edaf227a logs: add BCLog::Timer and related macros (James O'Beirne)

Pull request description:

  It's currently annoying to detect FlushStateToDisk() calls when benchmarking since they have to be inferred from a drop in coins count from the `UpdateTip: ` log messages. This adds a new logging utility, `BCLog::Timer`, and some related macros that are generally useful for printing timing-related logging messages, and a message that is unconditionally written when the coins cache is flushed to disk.

  ```
  2019-09-04T20:17:51Z FlushStateToDisk: write block and undo data to disk completed (3ms)
  2019-09-04T20:17:51Z FlushStateToDisk: write block index to disk completed (370ms)
  2019-09-04T20:17:51Z FlushStateToDisk: write coins cache to disk (2068451 coins, 294967kB) completed (21481ms)
  ```

ACKs for top commit:
  laanwj:
    Thanks, ACK dcef9a2922
  ryanofsky:
    Code review ACK dcef9a2922. No changes since last review other than moving code to new timer.h header

Tree-SHA512: 6d61e48a062d3edb48d0e056a6f0b1f8031773cc99289ee4544f8349d24526b88519e1e304009d56e428f1eaf76c857bf8e7e1c0b6873a6f270306accb5edc3d
2019-11-05 23:45:30 +01:00
Russell Yanofsky
3645e4ca00 Add missing newline in util_ChainMerge test
This was causing a lot of test cases not not be very meaningful because
multiple configuration options were combined into one line.

The changes in test output with this fix make sense and look like:

```diff
- testnet=1 regtest=1 || test
+ testnet=1 regtest=1 || error: Invalid combination of -regtest, -testnet and -chain. Can use at most one.
```

Issue was reported and debugged by
Wladimir J. van der Laan <laanwj@protonmail.com> in
https://github.com/bitcoin/bitcoin/pull/17385#issuecomment-550033222
2019-11-05 17:25:16 -05:00
Wladimir J. van der Laan
d9a4550001 Merge #17360: gui: Improve "Hide" button tool-tip message
1c26c16065 Improve "Hide" button tool-tip message (Danny-Scott)

Pull request description:

  Cleaned up the tool tip text, it looks as though it just got included back in 2014 when the whole section was added.

  Changed hide button tool tip within transaction fee settings area from "collapse fee-settings" to "Hide transaction fee settings" to be more user friendly and fit with other tool tips.

  ![hide-transaction-fee-tool-tip](https://user-images.githubusercontent.com/17258195/68086415-b7b70680-fe43-11e9-82cb-567b9730c1b9.png)

ACKs for top commit:
  laanwj:
    ACK 1c26c16065

Tree-SHA512: e2c83271c273f785ac625da9f88e095076043e21a9c59792049c271747837d19483e0cae5466c26ef3231947b6245680c4c136a530ba6f1885f9ddc18f2560d6
2019-11-05 20:42:37 +01:00
MarcoFalke
fea532a5f2 Merge #16540: test: Add ASSERT_DEBUG_LOG to unit test framework
fa2c44c3cc test: Add ASSERT_DEBUG_LOG to unit test framework (MarcoFalke)
fa1936f57b logging: Add member for arbitrary print callbacks (MarcoFalke)

Pull request description:

  Similar to `assert_debug_log` in the functional test framework

Top commit has no ACKs.

Tree-SHA512: aa9eaeca386b61d806867c04a33275f6eb4624fa5bf50f2928d16c83f5634bac96bcac46f9e8eda3b00b4251c5f12d7b01d6ffd84ba8e05c09eeec810cc31251
2019-11-05 14:34:42 -05:00
Danny-Scott
1c26c16065 Improve "Hide" button tool-tip message 2019-11-05 19:15:42 +00:00
MarcoFalke
fa5facd3e7 rpc: Remove unused boost::this_thread::interruption_point 2019-11-05 14:00:03 -05:00
Wladimir J. van der Laan
b05b28183c Merge #16899: UTXO snapshot creation (dumptxoutset)
92b2f5306b test: add dumptxoutset RPC test (James O'Beirne)
c1ccbc3dde devtools: add utxo_snapshot.sh (James O'Beirne)
57cf74c991 rpc: add dumptxoutset (James O'Beirne)
92fafb3a7d coinstats: add coins_count (James O'Beirne)
707fde7b9b add unused SnapshotMetadata class (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This changeset defines the serialization format for UTXO snapshots and adds an RPC command for creating them, `dumptxoutset`. It also adds a convenience script for generating and verifying snapshots at a certain height, since that requires doing a hacky rewind of the chain via `invalidateblock`.

  All of this is unused at the moment.

ACKs for top commit:
  laanwj:
    ACK 92b2f5306b

Tree-SHA512: 200dff87767f157d627e99506ec543465d9329860a6cd49363081619c437163a640a46d008faa92b1f44fd403bfc7a7c9e851c658b5a4849efa9a34ca976bf31
2019-11-05 19:40:18 +01:00
James O'Beirne
92b2f5306b test: add dumptxoutset RPC test 2019-11-05 13:36:04 -05:00
James O'Beirne
c1ccbc3dde devtools: add utxo_snapshot.sh
to allow easy (if not time-consuming) generation and verification of
snapshots.
2019-11-05 13:36:04 -05:00
James O'Beirne
57cf74c991 rpc: add dumptxoutset
Allows the creation of a UTXO snapshot to disk.
2019-11-05 13:35:57 -05:00
Wladimir J. van der Laan
d35b12107e Merge #17044: init: Remove auto-import of bootstrap.dat and associated code
104f7de593 remove old bootstrap relevant code (tryphe)

Pull request description:

  This picks up #15954

  I fixed the code and added at a functional test utilizing the scripts in `contrib/linearize` as suggested by @MarcoFalke .

ACKs for top commit:
  laanwj:
    ACK 104f7de593

Tree-SHA512: acac9f285f9785fcbc3afc78118461e45bec2962f90ab90e9f82f3ad28adc90a44f0443b712458ccf486e46d891eb8a67f53e7bee5fa6d89e4387814fe03f117
2019-11-05 19:25:10 +01:00
MarcoFalke
22e7eea629 Merge #17363: test: add "diamond" unit test to MempoolAncestryTests
b2ff500fb3 test: add "diamond" unit test to MempoolAncestryTests (Sebastian Falbesoner)

Pull request description:

  Approaches #17271 (_Missing Unit Test for Ancestors "diamond"_).
  If ancestors are represented more than once (in this case `ta` and `tb`), check that those are not overcounted.

ACKs for top commit:
  laanwj:
    ACK b2ff500fb3

Tree-SHA512: 82a6573cc7f0e82bf6fcfe207d7ddecbf297d2a203d22e95b73d887e3cb280f45a3c5f649161561c1be1eb560ff81b9b385868f205d1c12284211c2377e5ad99
2019-11-05 13:19:50 -05:00
Antoine Riard
5aacc3eff1 Add m_last_block_processed_height field in CWallet
At BlockConnected/BlockDisconnected, we rely on height of block
itself to know current height of wallet
2019-11-05 12:59:16 -05:00
Antoine Riard
10b4729e33 Pass block height in Chain::BlockConnected/Chain::BlockDisconnected
To do so we update CValidationInterface::BlockDisconnect to take a
CBlockIndex pointing to the block being disconnected.

This new parameter will be use in the following commit to establish
wallet height.
2019-11-05 12:59:16 -05:00
MarcoFalke
50591f6ec6 Merge #17357: tests: Add fuzzing harness for Bech32 encoding/decoding
b7541705d0 tests: Add fuzzing harness for Bech32 encoding/decoding (practicalswift)
85a34b1683 tests: Move CaseInsensitiveEqual to test/util/str (practicalswift)

Pull request description:

  Add fuzzing harness for Bech32 encoding/decoding.

  **Testing this PR**

  Run:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/bech32 -max_total_time=60
  …
  ```

ACKs for top commit:
  jonatack:
    ACK b7541705d0

Tree-SHA512: ade01d30c6886a083b806dbfff08999cc0d08e687701c670c895e261ed242c789e8a0062d4ebbe8f82676b8f168dc37e83351a88822c9c0eab478572a9e1ec02
2019-11-05 12:54:50 -05:00
MarcoFalke
c7e6b3b343 Merge #17243: p2p: add PoissonNextSend method that returns mockable time
1a8f0d5a74 [tools] update nNextInvSend to use mockable time (Amiti Uttarwar)
4de630354f [tools] add PoissonNextSend method that returns mockable time (Amiti Uttarwar)

Pull request description:

  Introduce a Poisson helper method that wraps the existing method to return `std::chrono::duration` type, which is mockable.

  Needed for https://github.com/bitcoin/bitcoin/pull/16698.

ACKs for top commit:
  ajtowns:
    ACK 1a8f0d5a74
  MarcoFalke:
    re-ACK 1a8f0d5a74
  naumenkogs:
    ACK 1a8f0d5, and let's merge it and come back to it later.

Tree-SHA512: 7e2325d7c55fc0b4357cb86b83e0c218ba269f678c1786342d8bc380bfd9696373bc24ff124b9ff17a6e761c62b2b44ff5247c3911e2afdc7cc5c20417e8290b
2019-11-05 12:38:28 -05:00
MarcoFalke
54a337478d Merge #17378: TestShell: Fix typos & implement cleanups
2493770e36 TestShell: Return self from setup() (James Chiang)
a8dea45524 TestShell: Simplify default setting of num_nodes (James Chiang)
9c7806e4bf Doc: Remove backticks in test-shell.md code block (James Chiang)
d3ed06e2cd TestShell: Fix typo in TestShell warning printout (James Chiang)

Pull request description:

  This PR follows up on #17288 and fixes typos and implements code clean-ups suggested by reviewers of 19139ee.

  - Typo in `test_shell.py` warning
  - Typo in `test-shell.md` code block
  - Simplified default setting of `num_nodes` in `TestShell.setup()`
  - Enable initializer chaining: `TestShell().setup()`

ACKs for top commit:
  MarcoFalke:
    ACK 2493770e36
  instagibbs:
    tACK 2493770e36
  jnewbery:
    utACK 2493770e36

Tree-SHA512: 8fa7c2c550dbc3ec899de9dc328cd55cfa6daafe3b888aa5427e72fea69f064d938ec68e15bfa57109c0f6c3583e627ac4bd69303a11575d056941bd253adee0
2019-11-05 12:36:16 -05:00
Michael Folkson
7b78b8d3a6 doc: Add template for good first issues 2019-11-05 17:15:14 +00:00
James O'Beirne
92fafb3a7d coinstats: add coins_count
Also changes existing CCoinsStats attributes to be C++11 initialized.
2019-11-05 10:54:00 -05:00
Russell Yanofsky
05b224a175 Add missing SetupGeneration error handling in EncryptWallet
Suggested https://github.com/bitcoin/bitcoin/pull/17304#discussion_r341286026
by me
2019-11-05 10:53:07 -05:00
tryphe
104f7de593 remove old bootstrap relevant code
- only load blockfiles when we have paths
- add release notes for modified bootstrap functionality
- amend documentation on ThreadImport
2019-11-05 16:47:26 +01:00
Russell Yanofsky
bfd826a675 Clean up nested scope in GetReservedDestination
Suggested https://github.com/bitcoin/bitcoin/pull/17304#discussion_r341194391
by Gregory Sanders <gsanders87@gmail.com>

Reason for keeping the `return true` `return false` verbosity is that more code
will be added after the ReserveKeyFromKeyPool() call before returning.
2019-11-05 10:47:07 -05:00
Russell Yanofsky
491a599b37 Get rid of confusing LegacyScriptPubKeyMan::TopUpKeyPool method
Previous discussion https://github.com/bitcoin/bitcoin/pull/17304#discussion_r340307903
2019-11-05 10:43:36 -05:00
Russell Yanofsky
4a0abf694e Pass CTxDestination to ScriptPubKeyMan::GetMetadata
Pass CTxDestination instead of more ambiguous uint160 hash value. This is more
type safe and more efficient since it avoids doing map lookups that will always
fail and were not done previously before
a18edd7b38 from
https://github.com/bitcoin/bitcoin/pull/17304

Change suggested by Andrew Chow <achow101-github@achow101.com> in
https://github.com/bitcoin/bitcoin/pull/17304#discussion_r340345745 and
https://github.com/bitcoin/bitcoin/pull/17381#issuecomment-549994944
2019-11-05 10:36:55 -05:00
MarcoFalke
fa7f5a4d2a doc: Update doc/bips.md with recent changes in master 2019-11-05 10:20:16 -05:00
Russell Yanofsky
b07b07cd87 Add EnsureLegacyScriptPubKeyMan and use in rpcwallet.cpp
This also fixes unused variable warnings in rpcdump.cpp
2019-11-05 10:13:43 -05:00
MarcoFalke
e204dc11b5 Merge #17365: depends: update README.md with working Android targets and API levels
3fe1aba601 depends: move README.md Android instructions to a separate section (Igor Cota)
aa9b84acee depends: update README.md with working Android targets and API levels (Igor Cota)

Pull request description:

  Per @Sjors comments in https://github.com/bitcoin/bitcoin/pull/16110#pullrequestreview-310821810

ACKs for top commit:
  Sjors:
    ACK 3fe1aba

Tree-SHA512: 7a2e676070d51c7a4291b0d4b638f52321c08cc6ebe2bd2c02ba62f6cc3dd8a73227df4693c6ce9201863eb0bf26e0133805347b9016cb0f9a389a49cc9492aa
2019-11-05 09:42:57 -05:00
James Chiang
2493770e36 TestShell: Return self from setup()
This allows user to chain setup() to the initializer. test-shell.md code
examples have been updated to reflect this.
2019-11-05 12:55:52 +01:00
James Chiang
a8dea45524 TestShell: Simplify default setting of num_nodes 2019-11-05 12:55:52 +01:00
James Chiang
9c7806e4bf Doc: Remove backticks in test-shell.md code block 2019-11-05 12:55:52 +01:00
James Chiang
d3ed06e2cd TestShell: Fix typo in TestShell warning printout 2019-11-05 12:55:52 +01:00
Amiti Uttarwar
1a8f0d5a74 [tools] update nNextInvSend to use mockable time 2019-11-05 11:12:10 +01:00
Amiti Uttarwar
4de630354f [tools] add PoissonNextSend method that returns mockable time 2019-11-05 11:06:53 +01:00
Igor Cota
3fe1aba601 depends: move README.md Android instructions to a separate section 2019-11-05 10:35:53 +01:00
practicalswift
b7541705d0 tests: Add fuzzing harness for Bech32 encoding/decoding 2019-11-05 09:23:44 +00:00
practicalswift
85a34b1683 tests: Move CaseInsensitiveEqual to test/util/str 2019-11-05 09:23:44 +00:00
Samuel Dobson
bdda137878 Merge #16766: wallet: Make IsTrusted scan parents recursively
4671fc3d9e Expand on wallet_balance.py comment from https://github.com/bitcoin/bitcoin/pull/16766\#issuecomment-527563982 (Jeremy Rubin)
91f3073f08 Update release notes to mention changes to IsTrusted and impact on wallet (Jeremy Rubin)
8f174ef112 Systematize style of IsTrusted single line if (Jeremy Rubin)
b49dcbedf7 update variable naming conventions for IsTrusted (Jeremy Rubin)
5ffe0d1449 Update comment in test/functional/wallet_balance.py (Jeremy Rubin)
a550c58267 Update wallet_balance.py test to reflect new behavior (Jeremy Rubin)
5dd7da4ccd Reuse trustedParents in looped calls to IsTrusted (Jeremy Rubin)
595f09d6de Cache tx Trust per-call to avoid DoS (Jeremy Rubin)
dce032ce29 Make IsTrusted scan parents recursively (Jeremy Rubin)

Pull request description:

  This slightly modifies the behavior of IsTrusted to recursively check the parents of a transaction. Otherwise, it's possible that a parent is not IsTrusted but a child is. If a parent is not trusted, then a child should not be either.

  This recursive scan can be a little expensive, so ~it might be beneficial to have a way of caching IsTrusted state, but this is a little complex because various conditions can change between calls to IsTrusted (e.g., re-org).~ I added a cache which works per call/across calls, but does not store the results semi-permanently. Which reduces DoS risk of this change. There is no risk of untrusted parents causing a resource exploitation, as we immediately return once that is detected.

  This is a change that came up as a bug-fix esque change while working on OP_SECURETHEBAG. You can see the branch where this change is important here: https://github.com/bitcoin/bitcoin/compare/master...JeremyRubin:stb-with-rpc?expand=1. Essentially, without this change, we can be tricked into accepting an OP_SECURETHEBAG output because we don't properly check the parents. As this was a change which, on its own, was not dependent on OP_SECURETHEBAG, I broke it out as I felt the change stands on its own by fixing a long standing wallet bug.

  The test wallet_balance.py has been corrected to meet the new behavior. The below comment, reproduced, explains what the issue is and the edge cases that can arise before this change.

          # Before `test_balance()`, we have had two nodes with a balance of 50
          # each and then we:
          #
          # 1) Sent 40 from node A to node B with fee 0.01
          # 2) Sent 60 from node B to node A with fee 0.01
          #
          # Then we check the balances:
          #
          # 1) As is
          # 2) With transaction 2 from above with 2x the fee
          #
          # Prior to #16766, in this situation, the node would immediately report
          # a balance of 30 on node B as unconfirmed and trusted.
          #
          # After #16766, we show that balance as unconfirmed.
          #
          # The balance is indeed "trusted" and "confirmed" insofar as removing
          # the mempool transactions would return at least that much money. But
          # the algorithm after #16766 marks it as unconfirmed because the 'taint'
          # tracking of transaction trust for summing balances doesn't consider
          # which inputs belong to a user. In this case, the change output in
          # question could be "destroyed" by replace the 1st transaction above.
          #
          # The post #16766 behavior is correct; we shouldn't be treating those
          # funds as confirmed. If you want to rely on that specific UTXO existing
          # which has given you that balance, you cannot, as a third party
          # spending the other input would destroy that unconfirmed.
          #
          # For example, if the test transactions were:
          #
          # 1) Sent 40 from node A to node B with fee 0.01
          # 2) Sent 10 from node B to node A with fee 0.01
          #
          # Then our node would report a confirmed balance of 40 + 50 - 10 = 80
          # BTC, which is more than would be available if transaction 1 were
          # replaced.

  The release notes have been updated to note the new behavior.

ACKs for top commit:
  ariard:
    Code Review ACK 4671fc3, maybe extend DoS protection in a follow-up PR.
  fjahr:
    Code review ACK 4671fc3d9e
  ryanofsky:
    Code review ACK 4671fc3d9e. Changes since last review: 2 new commits adding suggested release note and python test comment, also a clean rebase with no changes to the earlier commits. The PR description is more comprehensive now, too. Looks good!
  promag:
    Code review ACK 4671fc3d9e.

Tree-SHA512: 6b183ff425304fef49724290053514cb2770f4a2350dcb83660ef24af5c54f7c4c2c345b0f62bba60eb2d2f70625ee61a7fab76a7f491bb5a84be5c4cc86b92f
2019-11-05 21:59:27 +13:00
Samuel Dobson
bfc4c896d6 Merge #17258: Fix issue with conflicted mempool tx in listsinceblock
436ad43643 Fix issue with conflicted mempool tx in listsinceblock (Adam Jonas)

Pull request description:

  Closes #8752 by bringing back abandoned #10470.

  This now checks that returned transactions are not conflicting with any transactions that are filtered out by the given blockhash and add a functional test to prevent this in the future.

  For more context, #8757 was closed in favor of #10470.

ACKs for top commit:
  instagibbs:
    utACK 436ad43643
  kallewoof:
    utACK 436ad43643
  jonatack:
    I'm not qualifed to give an ACK here but 436ad43643 appears reasonable. Built/ran tests/verified that this test fails without the change in rpcwallet.cpp:

Tree-SHA512: 63d75cd3d3f19fc84dc38899b200c96179b82b24db263cd0116ee5b715265be647157855c2e35912d2fbc49c7b37db9375d6aab0ac672f0f09bece8431de5ea9
2019-11-05 21:56:34 +13:00
MarcoFalke
bc38bb9a60 Merge #17288: Added TestShell class for interactive Python environments.
19139ee034 Add documentation for test_shell submodule (JamesC)
f5112369cf Add TestShell class (James Chiang)
5155602a63 Move argparse() to init() (JamesC)
2ab01462f4 Move assert num_nodes is set into main() (JamesC)
614c645643 Clear TestNode objects after shutdown (JamesC)
6f40820757 Add closing and flushing of logging handlers (JamesC)
6b71241291 Refactor TestFramework main() into setup/shutdown (JamesC)
ede8b7608e Remove network_event_loop instance in close() (JamesC)

Pull request description:

  This PR refactors BitcoinTestFramework to encapsulate setup and shutdown logic into dedicated methods, and adds a  ~~TestWrapper~~ TestShell child class. This wrapper allows the underlying BitcoinTestFramework to run _between user inputs_ in a REPL environment, such as a Jupyter notebook or any interactive Python3 interpreter.

  The ~~TestWrapper~~ TestShell is motivated by the opportunity to expose the test-framework as a prototyping and educational toolkit. Examples of code prototypes enabled by  ~~TestWrapper~~ TestShell can be found in the Optech [Taproot/Schnorr](https://github.com/bitcoinops/taproot-workshop) workshop repository.

  Usage example:
  ```
  >>> import sys
  >>> sys.path.insert(0, "/path/to/bitcoin/test/functional")
  ```
  ```
  >>> from test_framework.test_wrapper import TestShell
  >>> test = TestShell()
  >>> test.setup(num_nodes=2)
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Initializing test directory /path/to/bitcoin_func_test_XXXXXXX
  ```
  ```
  >>> test.nodes[0].generate(101)
  >>> test.nodes[0].getblockchaininfo()["blocks"]
  101
  ```
  ```
  >>> test.shutdown()
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Stopping nodes
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Cleaning up /path/to/bitcoin_func_test_XXXXXXX on exit
  20XX-XX-XXTXX:XX:XX.XXXXXXX TestFramework (INFO): Tests successful
  ```

  **Overview of changes to BitcoinTestFramework:**

  - Code moved to `setup()/shutdown()` methods.
  - Argument parsing logic encapsulated by `parse_args` method.
  - Success state moved to `BitcoinTestFramework.success`.

  _During Shutdown_

  - `BitcoinTestFramework` logging handlers are flushed and removed.
  - `BitcoinTestFrameowork.nodes` list is cleared.
  - `NetworkThread.network_event_loop` is reset. (NetworkThread class).

  **Behavioural changes:**
  - Test parameters can now also be set when overriding BitcoinTestFramework.setup() in addition to overriding `set_test_params` method.
  - Potential exceptions raised in BitcoinTestFramework.setup() will be handled in main().

  **Added files:**
  - ~~test_wrapper.py~~ `test_shell.py`
  - ~~test-wrapper.md~~ `test-shell.md`

ACKs for top commit:
  jamesob:
    ACK 19139ee034
  jonatack:
    ACK 19139ee034
  jnewbery:
    Rather than invalidate the three ACKs for a minor nit, can you force push back to 19139ee034 please? I think this PR was ready to merge before your last force push.
  jachiang:
    > Rather than invalidate the three ACKs for a minor nit, can you force push back to [19139ee](19139ee034) please? I think this PR was ready to merge before your last force push.
  jnewbery:
    ACK 19139ee034

Tree-SHA512: 0c24f405f295a8580a9c8f1b9e0182b5d753eb08cc331424616dd50a062fb773d3719db4d08943365b1f42ccb965cc363b4bcc5beae27ac90b3460b349ed46b2
2019-11-04 14:54:14 -05:00
fanquake
3d05d33269 cli: fix -getinfo output when compiled with no wallet 2019-11-04 14:39:34 -05:00
MarcoFalke
33b155f287 Merge #17366: test: Reset global args between test suites
fa07b8beb5 test: Reset global args between test suites (MarcoFalke)

Pull request description:

  Ideally there wouldn't be any globals in Bitcoin Core. However, as we still have globals, they need to be reset between runs of test cases. One way to do this is to run each suite in a different process. `make check` does that. However, `./src/test/test_bitcoin` when run manually or on appveyor is a single process, where all globals are preserved between test cases.

  This leads to hard to debug issues such as https://github.com/bitcoin/bitcoin/pull/15845#pullrequestreview-310852164.

  Fix that by resetting the global arg for each test suite. Note that this wont reset the arg between test cases, as the constructor/destructor is not called for them.

  Addendum: This is not a general fix, only for `-segwitheight`. I don't know if clearing all args can be done with today's argsmanager.  Nor do I know if it makes sense. Maybe we want datadir set to a temp path to not risk accidentally corrupting the default data dir?

ACKs for top commit:
  laanwj:
    ACK fa07b8beb5
  practicalswift:
    ACK fa07b8beb5
  mzumsande:
    ACK fa07b8beb5, I also tested that this fixes the issue in #15845.

Tree-SHA512: 1e30b06f0d2829144a61cc1bc9bdd6a694cbd911afff83dd3ad2a3f15b577fd30acdf9f1469f8cb724d0642ad5d297364fd5a8a2a9c8619a7a71fa9ae2837cdc
2019-11-04 14:20:53 -05:00
James O'Beirne
dcef9a2922 logs: add timing information to FlushStateToDisk() 2019-11-04 14:13:54 -05:00
James O'Beirne
41edaf227a logs: add BCLog::Timer and related macros
Makes logging timing information about a block of code easier.
2019-11-04 14:13:52 -05:00
MarcoFalke
fa8a60bce9 ci: Run non-cross-compile builds natively 2019-11-04 14:10:24 -05:00
MarcoFalke
fa56bcbb01 ci: Run CI_WAIT only on travis 2019-11-04 14:10:07 -05:00
John Newbery
646b593bbd [tests] Speed up rpc_fundrawtransaction.py
Most of the time in rpc_fundrawtransaction.py is spent waiting for
unconfirmed transactions to propagate. Net processing adds a poisson
random delay to the time it will INV transactions with a mean interval
of 5 seconds. Calls like the following:

```
self.nodes[2].sendrawtransaction(signedTx['hex'])
self.sync_all()
self.nodes[1].generate(1)
````

will therefore introduce a delay waiting for the mempools to sync.
Instead just generate the block on the node that sent the transaction:

```
self.nodes[2].sendrawtransaction(signedTx['hex'])
self.nodes[2].generate(1)
```

rpc_fundrawtransaction.py is not intended to be a test for transaction
relay, so it's ok to do this.
2019-11-04 13:20:23 -05:00
MarcoFalke
94a26b192f Merge #17318: replace asserts in RPC code with CHECK_NONFATAL and add linter
c98bd13e67 replace asserts in RPC code with CHECK_NONFATAL and add linter (Adam Jonas)

Pull request description:

  - Replace instances of assert in /rpc files and rpcwallet with CHECK_NONFATAL(condition)
  - Add a linter to prevent future usage of assert being used in RPC code

  ref https://github.com/bitcoin/bitcoin/pull/17192

ACKs for top commit:
  practicalswift:
    ACK c98bd13e67 -- diff looks correct

Tree-SHA512: a16036b6bbcca73a5334665f66e17e1756377d582317568291da1d727fc9cf8c84bac9d9bd099534e1be315345336e5f7b66b93793135155f320dc5862a2d875
2019-11-04 11:33:41 -05:00
MarcoFalke
6cb10c14c6 Merge #17199: test: use default address type (bech32) for wallet_bumpfee tests
8d8e5a79d0 test: use default address type (bech32) for wallet_bumpfee tests (Sebastian Falbesoner)

Pull request description:

  The use of native segwit addresses (pure p2wpkh instead of p2sh-p2wpkh) leads to smaller transaction sizes, needing adaption of some constants in the following test cases:
  - `test_dust_to_fee()`: adaption of dust calculation (p2wpkh spend estimate of 67 is taken from `src/policy/policy.cpp:GetDustThreshold()`)
  - `test_maxtxfee_fails()`: lowering `-maxtxfee` setting to trigger fail

Top commit has no ACKs.

Tree-SHA512: b4163700d56c11955f811bc5fe6edaf7aec69931d7db741c03b055fb518bb9825c031fb931c513b37a1968085cb8c2f263adf664b357aff8ee42795fd0f88d2d
2019-11-04 11:27:48 -05:00
MarcoFalke
8f9df2ed88 Merge #17164: p2p: Avoid allocating memory for addrKnown where we don't need it
b6d2183858 Minor refactoring to remove implied m_addr_relay_peer. (User)
a552e8477c added asserts to check m_addr_known when it's used (User)
090b75c14b p2p: Avoid allocating memory for addrKnown where we don't need it (User)

Pull request description:

  We should allocate memory for addrKnown filter only for those peers which are expected to participate in address relay.

  Currently, we do it for all peers (including SPV and block-relay-only),  which results in extra RAM where it's not needed.

  Upd:
  In future, we would still allow SPVs to ask for addrs, so allocation still will be done by default.
  However, they will be able to opt-out via [this proposal](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2019-October/017428.html) and then we could save some more memory.
  This PR still saves memory for block-relay-only peers immediately after merging.

Top commit has no ACKs.

Tree-SHA512: e84d93b2615556d466f5ca0e543580fde763911a3bfea3127c493ddfaba8f05c8605cb94ff795d165af542b594400995a2c51338185c298581408687e7812463
2019-11-04 11:17:20 -05:00
João Barbosa
3958295bc8 wallet: LearnRelatedScripts only if KeepDestination 2019-11-04 16:14:38 +00:00
João Barbosa
55295fba4c wallet: Lock address type in ReserveDestination 2019-11-04 16:13:51 +00:00
MarcoFalke
fa07b8beb5 test: Reset global args between test suites 2019-11-04 10:59:55 -05:00
stefanwouldgo
02ac445b2f bump libevent to 2.1.11 in depends
this doesn't need patches on Android anymore like 2.1.8 did.
2019-11-04 16:45:34 +01:00
MarcoFalke
fa2c44c3cc test: Add ASSERT_DEBUG_LOG to unit test framework 2019-11-04 10:42:33 -05:00
MarcoFalke
fa1936f57b logging: Add member for arbitrary print callbacks 2019-11-04 10:42:29 -05:00
Igor Cota
aa9b84acee depends: update README.md with working Android targets and API levels 2019-11-04 16:13:08 +01:00
JamesC
19139ee034 Add documentation for test_shell submodule 2019-11-04 16:02:28 +01:00
Wladimir J. van der Laan
bbc9e4133c Merge #17304: refactor: Move many functions into LegacyScriptPubKeyMan and further separate it from CWallet
152b0a00d8 Refactor: Move nTimeFirstKey accesses out of CWallet (Andrew Chow)
7ef47b88e6 Refactor: Move GetKeypoolSize code out of CWallet (Andrew Chow)
089e17d45c Refactor: Move RewriteDB code out of CWallet (Andrew Chow)
0eac7088ab Refactor: Move SetupGeneration code out of CWallet (Andrew Chow)
f45d12b36c Refactor: Move HavePrivateKeys code out of CWallet::CreateWalletFromFile (Andrew Chow)
8b0d82bb42 Refactor: Move Upgrade code out of CWallet::CreateWalletFromFile (Andrew Chow)
46865ec958 Refactor: Move MarkUnusedAddresses code out of CWallet::AddToWalletIfInvolvingMe (Andrew Chow)
a18edd7b38 Refactor: Move GetMetadata code out of getaddressinfo (Andrew Chow)
9716bbe0f8 Refactor: Move LoadKey LegacyScriptPubKeyMan method definition (Andrew Chow)
67be6b9e21 Refactor: Move SetAddressBookWithDB call out of LegacyScriptPubKeyMan::ImportScriptPubKeys (Andrew Chow)
fc2867fdf5 refactor: Replace UnsetWalletFlagWithDB with UnsetBlankWalletFlag in ScriptPubKeyMan (Andrew Chow)
78e7cbc7ba Refactor: Remove UnsetWalletFlag call from LegacyScriptPubKeyMan::SetHDSeed (Andrew Chow)
0391aba52d Remove SetWalletFlag from WalletStorage (Andrew Chow)
4c5491f99c Refactor: Move SetWalletFlag out of LegacyScriptPubKeyMan::UpgradeKeyMetadata (Andrew Chow)
769acef857 Refactor: Move SetAddressBook call out of LegacyScriptPubKeyMan::GetNewDestination (Andrew Chow)
acedc5b823 Refactor: Add new ScriptPubKeyMan virtual methods (Andrew Chow)
533d8b364f Refactor: Declare LegacyScriptPubKeyMan methods as virtual (Andrew Chow)
b4cb18bce3 MOVEONLY: Reorder LegacyScriptPubKeyMan methods (Andrew Chow)

Pull request description:

  Moves several more key management and metadata functions into LegacyScriptPubKeyMan from CWallet to further separate the two.

  Note to reviewers: All of the `if (auto spk_man = walletInstance->m_spk_man.get()) {` blocks will be replaced with for loops in the next PR so you may see some things in those blocks that don't necessarily make sense with an `if` but will with a `for`.

ACKs for top commit:
  laanwj:
    code review ACK 152b0a00d8
  Sjors:
    re-ACK 152b0a00d8
  promag:
    Code review ACK 152b0a00d8.

Tree-SHA512: ff9872a3ef818922166cb15d72363004ec184e1015a3928a66091bddf48995423602ccd7e55b814de85d25ad7c69058280b1fde2e633570c680dc7d6084b3122
2019-11-04 16:01:42 +01:00
Sebastian Falbesoner
b2ff500fb3 test: add "diamond" unit test to MempoolAncestryTests
Approaches #17271.
If ancestors are represented more than once, check that those are not
overcounted.
2019-11-04 15:05:47 +01:00
MarcoFalke
fba574c908 Merge #17349: Remove redundant copy constructors
fa8919889f bench: Remove redundant copy constructor in mempool_stress (MarcoFalke)
29f8434368 refactor: Remove redundant PSBT copy constructor (Hennadii Stepanov)

Pull request description:

  I fail to see why people add these copy constructors manually without explanation, when the compiler can generate them at least as good automatically with less code.

ACKs for top commit:
  promag:
    ACK fa8919889f.
  hebasto:
    ACK fa8919889f, nit s/constructor/operator/ in commit fa8919889f message, as @promag [mentioned](https://github.com/bitcoin/bitcoin/pull/17349#discussion_r341776389) above.
  jonatack:
    ACK fa8919889f

Tree-SHA512: ce024fdb894328f41037420b881169b8b1b48c87fbae5f432edf371a35c82e77e21468ef97cda6f54d34f1cf9bb010235d62904bb0669793457ed1c3b2a89723
2019-11-04 08:32:22 -05:00
MarcoFalke
def69e5a86 Merge #17233: travis: Run unit and functional tests on native arm
facc0da63a travis: Run unit and functional tests on native arm (MarcoFalke)
fafa064d2a ci: Remove ccache requirement on the host (MarcoFalke)

Pull request description:

  This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just `export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/"`.

ACKs for top commit:
  laanwj:
    LGTM ACK facc0da63a
  practicalswift:
    ACK facc0da63a -- diff looks correct and Travis seems happy

Tree-SHA512: 0dc1bc82eb93e2bd8b159e044f20fe3055f8cdfd73aaa238bd2e178397582144dfc0c6a87bd8270115dafea1a623e642bde5d5f30254f94140f1a2cdb12fc2da
2019-11-04 08:23:21 -05:00
MarcoFalke
5933c6d924 Merge #17228: test: Add RegTestingSetup to setup_common
fa0a731d00 test: Add RegTestingSetup to setup_common (MarcoFalke)
fa54b3e248 test: move-only ComputeFilter to src/test/lib/blockfilter (MarcoFalke)

Pull request description:

  The default chain for `TestingSetup` is the main chain. However, any test that wants to mine blocks on demand needs to switch to regtest. This is done manually and in-line right now.

  Fix that by creating an explicit `RegTestingSetup` and use it where appropriate.

  Also, add a move-only commit to move `ComputeFilter` into the newly created unit test library.

  Both commits are part of #15845, but split up because they are useful on their own.

ACKs for top commit:
  practicalswift:
    ACK fa0a731d00 -- diff looks correct

Tree-SHA512: 02b9765580b355ed8d1be555f8ae11fa6e3d575f5cb177bbdda0319378837e29de5555c126c477dc8a1e8a5be47335afdcff152cf2dea2fbdd1a988ddde3689b
2019-11-04 08:16:54 -05:00
MarcoFalke
73b26e38d7 Merge #17351: doc: Fix some misspellings
ac831339cb doc: Fix some misspellings (randymcmillan)

Pull request description:

  Here is a more thorough lint-spelling update.
  This PR takes care of easy to fix spelling errors to clean up the linting stages.
  There are misspellings coded into the functional tests.
  That is a whole separate job within itself.

ACKs for top commit:
  practicalswift:
    ACK ac831339cb -- diff looks correct

Tree-SHA512: d8fad83fed083715655f148263ddeffc6752c8007d568fcf3dc2c418ccd5db70089ce3ccfd3994fcbd78043171402eb9cca5bdd5125287e22c42ea305aaa6e9d
2019-11-04 08:03:48 -05:00
Wladimir J. van der Laan
91fbcf41b3 Merge #16110: depends: Add Android NDK support
f9af3ced1c Android: add all arch support (Block Mechanic)
d419ca7e32 depends: export dynamic JNI symbols from static qtforandroid.a (Igor Cota)
ed30684d03 Qt: patch androidjnimain.cpp to make sure JNI is initialised when statically compiled (Igor Cota)
e4c319e8a1 builds: remove superfluous config_opts_aarch64_android (Igor Cota)
24ffef0c27 Patch libevent when building for Android (fix arc4random_addrandom) (Igor Cota)
f1e40b3e71 Update bitcoin_qt.m4 (BlockMechanic)
b4057d8261 Define TARGET_OS when host is android (Igor Cota)
80b475f159 Fix Android zlib cross compilation issue (https://stackoverflow.com/questions/21396988/zlib-build-not-configuring-properly-with-cross-compiler-ignores-ar) (Igor Cota)
45f8219015 Add full Android build example command and instructions on getting SDK/NDK (Igor Cota)
b68f2a68c2 Add config opts and patch for aarch64_android build of Qt (Igor Cota)
9c4cb0166e Add ranlib to android.mk hosts file (fix OSX Android NDK build) (Igor Cota)
c2a749c9c1 Add example Android host-platform-triplet and options (Igor Cota)
0b0cff3c61 Add support for building Android dependencies (Igor Cota)

Pull request description:

  This allows one to build the dependencies with the Android SDK and goes towards fixing #11844. It has been tested to work with:
  `make HOST=aarch64-linux-android ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=/home/user/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin NO_QT=1 NO_WALLET=1`

ACKs for top commit:
  Sjors:
    ACK f9af3ce. I'm OK with merging and then improving later.

Tree-SHA512: cb805115ebe5c9e33db2bf3eab8628808fe3f50052053d8877d8b8e4406d6fea1ed9e5c4dff85d777fb99c81be6ffb9d95a0e6d32344e728e5e0da6c653e2ce7
2019-11-04 13:32:19 +01:00
Wladimir J. van der Laan
c4b8dd2060 Merge #17297: refactor: Remove addrdb.h dependency from node.h
f44abe4bed refactor: Remove addrdb.h dependency from node.h (Hennadii Stepanov)

Pull request description:

  `node.h` includes `addrdb.h` just for the sake of `banmap_t` type.
  This PR makes dependencies simpler and explicit.

  ~Also needless `typedef` has been removed from `enum BanReason`.~

ACKs for top commit:
  laanwj:
    ACK f44abe4bed
  practicalswift:
    ACK f44abe4bed

Tree-SHA512: 33a1be20e5c629daf4a61ebbf93ea6494b9256887cebd4974de4782f6d324404b6cc84909533d9502b2cc19902083f1f9307d4fb7231e67db5b412b842d13072
2019-11-04 13:18:27 +01:00
randymcmillan
ac831339cb doc: Fix some misspellings 2019-11-04 04:22:53 -05:00
James Chiang
f5112369cf Add TestShell class
A BitcoinTestFramework child class which can be imported by an external user or
project. TestShell.setup() initiates an underlying BitcoinTestFramework object
with bitcoind subprocesses, rpc interfaces and test logging.
TestShell.shutdown() safely tears down the BitcoinTestFramework object.
2019-11-04 08:56:56 +01:00
JamesC
5155602a63 Move argparse() to init()
This ensures TestFramework default parameters are set before setup is called. A
 child class will therefore have access to defaults when overriding setup.
2019-11-03 20:34:49 +01:00
JamesC
2ab01462f4 Move assert num_nodes is set into main()
This allows a BitcoinTestFramework child class to set test parameters in an
overridden setup() rather than in an overridden set_test_params().
2019-11-03 20:34:41 +01:00
JamesC
614c645643 Clear TestNode objects after shutdown
TestNode objects need to be removed during shutdown, as setup_nodes does not
remove previous TestNode objects from previous test runs during setup.
2019-11-03 20:34:27 +01:00
JamesC
6f40820757 Add closing and flushing of logging handlers
In order for BitcoinTestFramework to correctly restart after shutdown, the
previous logging handlers need to be removed, or else logging will continue in
the previous temp directory. "Flush" ensures buffers are emptied, and "close"
ensures file handler close logging file.
2019-11-03 20:34:18 +01:00
JamesC
6b71241291 Refactor TestFramework main() into setup/shutdown
Setup and shutdown code now moved into dedicated methods. Test "success" is
added as a BitcoinTestFramework member, which can be accessed outside of main.
Argument parsing also moved into separate method and called from main.
2019-11-03 20:34:07 +01:00
JamesC
ede8b7608e Remove network_event_loop instance in close()
The asyncio.new_event_loop() instance is now removed from the NetworkThread
class during shutdown. This enables a NetworkThread instance to be restarted
after being closed. The current NetworkThread class guards against an existing
new_event_loop during initialization.
2019-11-03 20:33:49 +01:00
João Barbosa
0b75a7f068 wallet: Reuse existing batch in CWallet::SetUsedDestinationState 2019-11-02 21:36:21 +00:00
João Barbosa
01f45dd00e wallet: Avoid recursive lock in CWallet::SetUsedDestinationState 2019-11-02 16:14:36 +00:00
Hennadii Stepanov
2ad74b78c6 doc: Add ShellCheck to lint tests dependencies 2019-11-02 17:43:07 +02:00
Wladimir J. van der Laan
463eab5e14 Merge #17285: doc: Bip70 removal follow-up
3ed8e3d079 doc: Remove explicit network name references (Fabian Jahr)
d6e493f0c2 wallet: Remove left-over BIP70 comment (Fabian Jahr)

Pull request description:

  A small follow-up to #17165 which removed BIP70 support.

  1. Removes one leftover mention of BIP70 in a comment.
  2. Removes BIP70 reference in comments on network/chain name strings. These can be removed as they are not really helpful and also incorrect: BIP70 only defines "main" and "test" but not "regtest". If/When signet gets merged we will add another name to the list that is not defined in BIP70. Mostly there is also an exhaustive list of the options included in the comment anyway.

  If we would like to keep an identifier for this naming scheme, I would suggest switching to something more generic, like 'short chain name'. Happy to implement that if that is preferred. Alternatively, we could add a reference to `CBaseChainParams`. That would also mean we don't have to change these lines again for signet.

ACKs for top commit:
  MarcoFalke:
    ACK 3ed8e3d079

Tree-SHA512: 9a7c0b9cacbb67bd31a089ffdc6f1ebc7f336493e2c8266eb697da34dce2b505a431d5639a3e4fc34f9287361343e861b55dc2662e0a1d2095cc1046db77d6ee
2019-11-02 14:47:41 +01:00
Wladimir J. van der Laan
9641366950 Merge #17293: Add assertion to randrange that input is not 0
a35b6824f3 Add assertion to randrange that input is not 0 (Jeremy Rubin)

Pull request description:

  From the comment in randrange, their is an implicit argument that randrange cannot accept an argument of 0. If the argument is 0, then we have to return {}, which is not possible in a uint64_t.

  The current code takes a very interesting approach, which is to return [0..std::numeric_limits<uint64_t>]. This can cause all sorts of fun problems, like allocating a lot of memory, accessing random memory (maybe with your private keys), and crashing the computer entirely.

  This gives us three choices of how to make it "safe":

  1) return Optional<uint64_t>
  2) Change the return type to [0..range]
  3) Return 0 if 0
  4) Assert(range)

  So which solution is best?

  1) seems a bit overkill, as it makes any code using randrange worse.
  2) Changing the return type as in 2 could be acceptable, but it imposes the potential overflow checking on the caller (which is what we want).
  3) An interesting option -- effective makes the return type in {0} U [0..range]. But this is a bad choice, because it leads to code like `vec[randrange(vec.size())]`, which is incorrect for an empty vector. Null set should mean null set.
  4) Assert(range) stands out as the best mitigation for now, with perhaps a future change to solution 2. It prevents the error from propagating at the earliest possible time, so the program crashes cleanly rather than by freezing the computer or accessing random memory.

ACKs for top commit:
  instagibbs:
    Seems reasonable for now, ACK a35b6824f3
  laanwj:
    ACK a35b6824f3
  promag:
    ACK a35b6824f3.

Tree-SHA512: 8fc626cde4b04b918100cb7af28753f25ec697bd077ce0e0c640be0357626322aeea233e3c8fd964ba1564b0fda830b7f5188310ebbb119c113513a4b89952dc
2019-11-02 11:40:56 +01:00
Andrew Chow
152b0a00d8 Refactor: Move nTimeFirstKey accesses out of CWallet
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
7ef47b88e6 Refactor: Move GetKeypoolSize code out of CWallet
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
089e17d45c Refactor: Move RewriteDB code out of CWallet
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
0eac7088ab Refactor: Move SetupGeneration code out of CWallet
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
f45d12b36c Refactor: Move HavePrivateKeys code out of CWallet::CreateWalletFromFile
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
8b0d82bb42 Refactor: Move Upgrade code out of CWallet::CreateWalletFromFile
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
46865ec958 Refactor: Move MarkUnusedAddresses code out of CWallet::AddToWalletIfInvolvingMe
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
a18edd7b38 Refactor: Move GetMetadata code out of getaddressinfo
Easier to review ignoring whitespace:

    git log -p -n1 -w

This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
9716bbe0f8 Refactor: Move LoadKey LegacyScriptPubKeyMan method definition
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
67be6b9e21 Refactor: Move SetAddressBookWithDB call out of LegacyScriptPubKeyMan::ImportScriptPubKeys
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
fc2867fdf5 refactor: Replace UnsetWalletFlagWithDB with UnsetBlankWalletFlag in ScriptPubKeyMan
ScriptPubKeyMan is only using UnsetWalletFlagWithDB to unset the blank
wallet flag. Just make that it's own function and not expose the flag
writing directly.

This does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
78e7cbc7ba Refactor: Remove UnsetWalletFlag call from LegacyScriptPubKeyMan::SetHDSeed
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
0391aba52d Remove SetWalletFlag from WalletStorage
SetWalletFlag is unused.

Does not change any behavior
2019-11-01 22:58:05 -04:00
Andrew Chow
4c5491f99c Refactor: Move SetWalletFlag out of LegacyScriptPubKeyMan::UpgradeKeyMetadata
This commit does not change behavior.
2019-11-01 22:58:05 -04:00
Andrew Chow
769acef857 Refactor: Move SetAddressBook call out of LegacyScriptPubKeyMan::GetNewDestination
This commit does not change behavior.
2019-11-01 22:56:37 -04:00
Andrew Chow
acedc5b823 Refactor: Add new ScriptPubKeyMan virtual methods
This commit does not change behavior.
2019-11-01 22:56:37 -04:00
Andrew Chow
533d8b364f Refactor: Declare LegacyScriptPubKeyMan methods as virtual
This commit does not change behavior.
2019-11-01 22:56:37 -04:00
Andrew Chow
b4cb18bce3 MOVEONLY: Reorder LegacyScriptPubKeyMan methods
Can verify move-only with:

    git log -p -n1 --color-moved

This commit is move-only and doesn't change code or affect behavior.
2019-11-01 22:56:37 -04:00
MarcoFalke
fa8919889f bench: Remove redundant copy constructor in mempool_stress 2019-11-01 18:15:53 -04:00
Hennadii Stepanov
29f8434368 refactor: Remove redundant PSBT copy constructor
The default (i.e., generated by a compiler) copy constructor does the 
same things.

Also this prevents -Wdeprecated-copy warning for implicitly declared 
operator= in GCC 9.
2019-11-01 18:12:57 -04:00
MarcoFalke
a5224be645 Merge #17292: Add new mempool benchmarks for a complex pool
b0c774b48a Add new mempool benchmarks for a complex pool (Jeremy Rubin)

Pull request description:

  This PR is related to #17268.

  It adds a mempool stress test which makes a really big complicated tx graph, and then, similar to mempool_eviction test, trims the size.

  The test setup is to make 100 original transactions with Rand(10)+2 outputs each.

  Then, 800 times:

  we create a new transaction with Rand(10) + 1 parents that are randomly sampled from all existing transactions (with unspent outputs). From each such parent, we then select Rand(remaining outputs) +1 50% of the time, or 1 outputs 50% of the time.

  Then, we trim the size to 3/4. Then we trim it to just a single transaction.

  This creates, hopefully, a big bundle of transactions with lots of complex structure, that should really put a strain on the mempool graph algorithms.

  This ends up testing both the descendant and ancestor tracking.

  I don't love that the test is "unstable". That is, in order to compare this test to another, you really can't modify any of the internal state because it will have a different order of invocations of the deterministic randomness. However, it certainly suffices for comparing branches.

Top commit has no ACKs.

Tree-SHA512: cabe96b849b9885878e20eec558915e921d49e6ed1e4b011b22ca191b4c99aa28930a8b963784c9adf78cc8b034a655513f7a0da865e280a1214ae15ebb1d574
2019-11-01 18:08:41 -04:00
MarcoFalke
5021ef8d7f Merge #17254: test: fix script_p2sh_tests OP_PUSHBACK2/4 missing
5710dadf9b test: fix script_p2sh_tests OP_PUSHBACK2/4 missing (kodslav)

Pull request description:

  Cleans up #15140 which fixes commit 6b25f29a91 where opcodes were lost in translation.

ACKs for top commit:
  laanwj:
    code review ACK 5710dadf9b

Tree-SHA512: 3f7fbcaf0dd199626d9ec9fdf3c5b5c5c2a91c4cfe81fae5b1d5662a48e52cf4bd27c94f8f42ebdfe7a076c5d600ada5661a6902b03eb5dc3dc953f4524345ac
2019-11-01 17:57:31 -04:00
MarcoFalke
c737839471 Merge #17309: doc: update MSVC instructions to remove Qt OpenSSL linking
162d0038e7 doc: compiling with Visual Studio is now supported on Windows (fanquake)
b1f1fb5f1d doc: update MSVC instructions to remove Qt configuration (fanquake)

Pull request description:

  Follow up from #17165. Flips `-openssl-linked` to `-no-openssl`. Also adds some missing packages to the vcpkg install instructions.

ACKs for top commit:
  sipsorcery:
    tACK 162d0038e7.

Tree-SHA512: 40577a3759a30170a14fd27e3eeac5a78a7852ae88daacecf584ad46c685708c167153d39357aa77a4f65bfd5a349f7589f20aa16fdf3d2895b4076b381e2c9c
2019-11-01 16:19:40 -04:00
fanquake
3f8d95500d Merge #17345: test: Do not instantiate CAddrDB for static call CAddrDB::Read()
59853c33f1 test: Do not instantiate CAddrDB for static call (Hennadii Stepanov)

Pull request description:

  Ref:
  6a7c40bee4/src/addrdb.h (L84-L94)

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 59853c33f1
  naumenkogs:
    ACK 59853c3

Tree-SHA512: 06b2e8e4f2b4a4f20454a4828a786878fc2bd441b0ee53f1128a7a0b6ad54dd3ca7598cee000ec60e5e4ef02570a09c01974d2d8260bd11fa9baf16b3ff9ba08
2019-11-01 15:30:07 -04:00
fanquake
162d0038e7 doc: compiling with Visual Studio is now supported on Windows 2019-11-01 15:25:52 -04:00
fanquake
b1f1fb5f1d doc: update MSVC instructions to remove Qt configuration
Also adds missing Boost packages. Installing only the currently listed
packages was not sufficient to complete a build.
2019-11-01 15:25:52 -04:00
Hennadii Stepanov
59853c33f1 test: Do not instantiate CAddrDB for static call 2019-11-01 19:28:15 +02:00
MarcoFalke
6a7c40bee4 Merge #15888: QA: Add wallet_implicitsegwit to test the ability to transform keys between address types
a6f6f77a86 QA: Add wallet_implicitsegwit to test the ability to transform keys between address types (Luke Dashjr)

Pull request description:

  This makes sure the wallet recognises payments to keys via address types they weren't created with.

  While we don't *want* this behaviour, it might make sense to explicitly test that it works until we remove it.

ACKs for top commit:
  adamjonas:
    utACK a6f6f77a86

Tree-SHA512: b208405729277e9ce06eb772b45e8d1683c4dc5703754448b8f19a590b37522abd7bb46d4dbd41513b3d46d7f9e8769ce4f15fa4114be600f31a1ebbc1157840
2019-11-01 13:24:50 -04:00
MarcoFalke
0e6cae0248 Merge #17284: build: update retry to current version
58d0393bec build: update retry to current version (randymcmillann)

Pull request description:

  This commit eliminates spelling and white space
  errors that are flagged in the linting process

ACKs for top commit:
  practicalswift:
    ACK 58d0393bec

Tree-SHA512: c241ed0775026c890dd29d1f7231c5540e9c9285867a99844605753a3007d08f0bd4f7a59f078e4c65b741301ff7fa8a871e2e3c64b9a9fe47b3ea74c4228498
2019-11-01 13:17:26 -04:00
MarcoFalke
bc3fcf3c0d Merge #17327: test: add rpc_fundrawtransaction logging
ff22751417 test: rm ascii art in rpc_fundrawtransaction (Jon Atack)
94fcc08541 test: add rpc_fundrawtransaction logging (Jon Atack)

Pull request description:

  `test/functional/rpc_fundrawtransaction.py` is fairly slow to run and has no logging, so it can appear to be stalled.

  This commit adds info logging at each test to provide feedback on the test run.

ACKs for top commit:
  instagibbs:
    utACK ff22751417
  jnewbery:
    tACK ff22751417

Tree-SHA512: f4fabad8ef51c29981351bb4e66fb0c0e0517418a4a15892ef804df11d16b2d2ae1a1abc958d2b121819850278de90a2003b0edb8d7098d00360b89fa76e9062
2019-11-01 11:06:20 -04:00
MarcoFalke
90a2341713 Merge #17218: Replace the LogPrint function with a macro
8734c856f8 Replace the LogPrint function with a macro (Jeffrey Czyz)

Pull request description:

  Calling `LogPrint` with a category that is not enabled results in
  evaluating the remaining function arguments, which may be arbitrarily
  complex (and possibly expensive) expressions. Defining `LogPrint` as a
  macro prevents this unnecessary expression evaluation.

  This is a partial revert of #14209. The decision to revert is discussed
  in #16688, which adds verbose logging for validation event notification.

ACKs for top commit:
  jnewbery:
    ACK 8734c856f8

Tree-SHA512: 19e995eaef0ff008a9f8c1fd6f3882f1fbf6794dd7e2dcf5c68056be787eee198d2956037d4ffba2b01e7658b47eba276cd7132feede78832373b3304203961e
2019-11-01 10:14:15 -04:00
MarcoFalke
ddd46293bd Merge #17324: Update univalue subtree
fa0b3da36c Squashed 'src/univalue/' changes from 7890db99d6..5a58a46671 (MarcoFalke)

Pull request description:

  Only change is a performance improvement. See https://github.com/bitcoin-core/univalue/pull/21#issue-333858474 and https://github.com/bitcoin-core/univalue/pull/15#issue-186748173

ACKs for top commit:
  jnewbery:
    ACK fa439e88af
  laanwj:
    ACK fa439e88af

Tree-SHA512: 35ea8f76ea4806182949c8eb5a8b652d1aaeec03ee023838e7cb29abcb81c61d59b38f15708564a78574722df57d13f61ef17d0f670a4056819705ef892321e0
2019-11-01 09:21:43 -04:00
Fabian Jahr
3ed8e3d079 doc: Remove explicit network name references 2019-11-01 12:06:35 +01:00
Jon Atack
ff22751417 test: rm ascii art in rpc_fundrawtransaction
Doc changes only to test/functional/rpc_fundrawtransaction.py:

- remove ascii art or convert to a docstring when sufficiently different from
the logging

- touch up other comments while here
2019-11-01 11:56:53 +01:00
Wladimir J. van der Laan
b54666c849 Merge #17286: Fix help-debug -checkpoints
a8b82867d5 Fix incorrect help-debug for -checkpoints (Antoine Riard)

Pull request description:

ACKs for top commit:
  jnewbery:
    ACK a8b82867d5 for improving the `-prune` help text.
  MarcoFalke:
    ACK a8b82867d5

Tree-SHA512: 973fa97436be09a9939386dc00023420a7296a9e268356bf26aa06468f9f0d2c822205a4f1ce8f44a0562aa64ad90a43dec5697af656ef28ba6829e4e4360e94
2019-11-01 11:41:45 +01:00
Jon Atack
94fcc08541 test: add rpc_fundrawtransaction logging
test/functional/rpc_fundrawtransaction.py is fairly long to run and has no
logging, so it can appear to be stalled.

This commit adds info logging at each test to provide feedback on the test run.
2019-11-01 11:27:11 +01:00
Wladimir J. van der Laan
462fbcd212 Merge #17325: log: Fix log message for -par=1
8a0ca5e9de log: Fix log message for -par=1 (Hennadii Stepanov)

Pull request description:

  Fix #17139

ACKs for top commit:
  jnewbery:
    Tested ACK 8a0ca5e9de
  laanwj:
    ACK 8a0ca5e9de

Tree-SHA512: 09f5416c00cd3e4f85cd9040267dc825d5c5623f8903e9d2373013686dce7f6b3ba573648787adc1d58e6f1d5012e26c78700d585273d4b508cb25312b3fa06b
2019-11-01 11:08:58 +01:00
MarcoFalke
222b7d0ca7 Merge #17330: test: Add shrinkdebugfile=0 to regtest bitcoin.conf
c5377ffbbb [qa] Add shrinkdebugfile=0 to regtest bitcoin.conf (Suhas Daftuar)

Pull request description:

  This helps avoid accidentally truncating the debug.log while manually
  debugging.

ACKs for top commit:
  MarcoFalke:
    ACK c5377ffbbb
  dongcarl:
    ACK c5377ffbbb

Tree-SHA512: a05d6a7c494ee2c300fa1a9dbaa48b7fed63a44b1fa823198cbf401cff1c4aa8e44eb431fa635b27a1987d0eb1a5f8e2712514dcea7c5dd05240a445c8bd505d
2019-10-31 14:57:44 -04:00
fanquake
811c381ab6 Merge #17329: linter: Strip trailing / in path for git-subtree-check
60582d6060 [linter] Strip trailing / in path for git-subtree-check (John Newbery)

Pull request description:

  git-subtree-check fails if the directory is given with a trailing slash,
  eg:

  ```
  > test/lint/git-subtree-check.sh src/univalue/
  ERROR: src/univalue/ is not a subtree
  ```

  Shell autocompletes will add the trailing slash when autofilling the
  path name, which will therefore cause the script to fail.

  Just ignore any trailing slash.

ACKs for top commit:
  laanwj:
    ACK 60582d6060
  dongcarl:
    ACK 60582d6060
  fanquake:
    ACK 60582d6060 - tested before and after.

Tree-SHA512: 5a91979b60e1d4b1310fd02a0ccc5465dbff57d9c94bba81e4758442a627cfa32217ab8f973990a17b5d961ecae61fb56b56ccf10f87e61dd03e88a1e0b8f99d
2019-10-31 14:51:36 -04:00
MarcoFalke
100fa0a62a Merge #17300: LegacyScriptPubKeyMan code cleanups
53fe0b70ad Fix missing strFailReason in CreateTransaction (Russell Yanofsky)
4b28a05f08 Fix misplaced AssertLockHeld (Russell Yanofsky)
2632b1f124 doc: Clarify WalletStorage / Wallet relation (Russell Yanofsky)
628d11b2ba Add back mistakenly removed AssertLockHeld (Russell Yanofsky)
52cf68f7ff Refactor: Add GetLegacyScriptPubKeyMan helper (Russell Yanofsky)

Pull request description:

  This PR implements suggested code cleanups from https://github.com/bitcoin/bitcoin/pull/17260 review comments

ACKs for top commit:
  Sjors:
    ACK 53fe0b70ad
  achow101:
    ACK 53fe0b70ad
  MarcoFalke:
    ACK 53fe0b70ad

Tree-SHA512: a577b96cb21a9aa7185d7d900e4db0665c302adcd12097957b9d8e838a8548c7de8f901bcb83e7c46d231b841221345c9264f5e29ed069f3d9236896430f959b
2019-10-31 14:40:39 -04:00
Suhas Daftuar
c5377ffbbb [qa] Add shrinkdebugfile=0 to regtest bitcoin.conf
This helps avoid accidentally truncating the debug.log while manually
debugging.
2019-10-31 13:54:07 -04:00
User
b6d2183858 Minor refactoring to remove implied m_addr_relay_peer.
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2019-10-31 13:42:02 -04:00
Hennadii Stepanov
8a0ca5e9de log: Fix log message for -par=1
Co-authored-by: Wladimir J. van der Laan <laanwj@protonmail.com>
2019-10-31 18:57:46 +02:00
darosior
2f5f7d6b13 GuessVerificationProgress: cap the ratio to 1
The getblockchaininfo RPC call could sometime return a
'validationprogress' > 1, but this is absurd.
2019-10-31 17:31:43 +01:00
John Newbery
60582d6060 [linter] Strip trailing / in path for git-subtree-check
git-subtree-check fails if the directory is given with a trailing slash,
eg:

```
> test/lint/git-subtree-check.sh src/univalue/
ERROR: src/univalue/ is not a subtree
```

Shell autocompletes will add the trailing slash when autofilling the
path name, which will therefore cause the script to fail.

Just ignore any trailing slash.
2019-10-31 11:53:46 -04:00
MarcoFalke
1c5e0ccaba Merge #17274: tests: Fix fuzzers eval_script and script_flags by re-adding ECCVerifyHandle dependency
9cae3d5e94 tests: Add fuzzer initialization (hold ECCVerifyHandle) (practicalswift)

Pull request description:

  The fuzzers `eval_script` and `script_flags` require holding `ECCVerifyHandle`.

  This is a follow-up to #17235 which accidentally broke those two fuzzers.

  Sorry about the temporary breakage my fuzzing friends: it took a while to fuzz before reaching these code paths. That's why this wasn't immediately caught. Sorry.

Top commit has no ACKs.

Tree-SHA512: 67ebb155ba90894c07eac630e33f2f985c97bdf96dc751f312633414abeccdca20315d7d8f2ec4ee3ac810b666a1e44afb4ea8bc28165151cd51b623f816cac2
2019-10-31 10:13:10 -04:00
Wladimir J. van der Laan
feb1a8c03a Merge #17308: nsis: Write to correct filename in first place
3b3b93174a nsis: Write to correct filename in first place (Carl Dong)

Pull request description:

  Per MarcoFalke's suggestion here https://github.com/bitcoin/bitcoin/pull/17029#discussion_r333216722

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 3b3b93174a, makes sense to name it that way because it will raise the "unsinged" error in Windows

Tree-SHA512: da72aae438505e162d0b3cd27d873b7ad8176178bb459a738e61b6e2ad0fa739d905b3109fab641bb1a3950fe59ad526c5568d12cf48a305166cdb7db6686543
2019-10-31 13:23:47 +01:00
randymcmillann
58d0393bec build: update retry to current version
from: 7d2caa94a7
2019-10-30 18:49:57 -04:00
Antoine Riard
a8b82867d5 Fix incorrect help-debug for -checkpoints 2019-10-30 18:29:16 -04:00
MarcoFalke
fa439e88af Update univalue subtree 2019-10-30 16:24:02 -04:00
MarcoFalke
fa0b3da36c Squashed 'src/univalue/' changes from 7890db99d6..5a58a46671
5a58a46671 Merge #21: Remove hand-coded UniValue destructor.
b4cdfc4f47 Remove hand-coded UniValue destructor.
7fba60b5ad Merge #17: [docs] Update readme
4577454e7e Merge #13: Fix typo
ac7e73cda8 [docs] Update readme
4a4964729b Fix typo

git-subtree-dir: src/univalue
git-subtree-split: 5a58a46671d3c1711e93d2fc7961085515c8c7a7
2019-10-30 16:22:42 -04:00
Luke Dashjr
a6f6f77a86 QA: Add wallet_implicitsegwit to test the ability to transform keys between address types 2019-10-30 18:56:25 +00:00
MarcoFalke
facc0da63a travis: Run unit and functional tests on native arm 2019-10-30 14:45:32 -04:00
fanquake
08e2947312 Merge #17316: refactor: Replace all uses of boost::optional with our own Optional type
d314e8a818 refactor: Replace all uses of boost::optional with our own Optional type (Wladimir J. van der Laan)

Pull request description:

  Replace all uses of boost::optional with our own Optional type. Luckily, there aren't so many.

  After this:

  - `boost::optional` is no longer used directly (only through `Optional` which is an alias for it)
  - `boost/optional.hpp` is only included in one place

ACKs for top commit:
  MarcoFalke:
    ACK d314e8a818
  practicalswift:
    ACK d314e8a818 -- diff looks correct + satisfying to see incremental progress towards the goal of a Boost free future :)
  jtimon:
    ACK d314e8a818
  fanquake:
    ACK d314e8a818

Tree-SHA512: b43e0017af81b07b5851377cd09624f114510ac5b9018d037664b58ad0fc8e893e30946b61f8f5e21e39125925bf9998a81f2226b468aab2df653ee57ed3213d
2019-10-30 14:20:17 -04:00
MarcoFalke
fa0a731d00 test: Add RegTestingSetup to setup_common 2019-10-30 14:12:45 -04:00
Matt Corallo
0a50019fde Walk pindexBestHeader back to ChainActive().Tip() if it is invalid
Instead of keeping pindexBestHeader set to the best header we've
ever seen, reset it back to our validated tip if we find an ancestor
of it turns out to be invalid. While the name is now a bit confusing,
this matches much better with how it is used in practice, see below.
Further, this opens up more use-cases for it in the future, namely
aggressively searching for new peers in case we have discovered
(possibly via some covert channel) headers which we do not know to be
invalid, but which we cannot find block data for.

Places pindexBestHeader is used:

 * Various GUI displays of the best header and getblockchaininfo["headers"],
   I don't think changing this is bad, and if anything this is less confusing
   in the presence of an invalid block.
 * IsCurrentForFeeEstimation(): If anything I think ensuring pindexBestHeader
   isn't some crazy invalid chain is better than the alternative, even in the
   case where you are rejecting the current chain due to hardware error (since
   hopefully in that case you won't get any new blocks anyway).
 * ConnectBlock assumevalid checks: We use pindexBestHeader to check that the
   block we're connecting leads to something with nMinimumChainWork (preventing
   a user-set assumevalid from having bogus work) and that the block we're
   connecting leads to pindexBestHeader (I'm not too worried about this one -
   it's nice to "disable" assumevalid if we have a long invalid headers chain,
   but I don't see it as a critical protection).
 * BlockRequestAllowed() uses pindexBestHeader as its target to ensure the
   requested block is within a month of the "current chain". I don't think this
   is a meaningful difference, if we're rejecting the current tip we're
   trivially fingerprintable anyway, and if the chain really does have a bunch
   of invalid crap near the tip, using the best not-invalid header is likely a
   better criteria.
 * ProcessGetBlockData uses pindexBestHeader as the "current chain" definition
   of whether a block request is "historical" for the purpose of bandwidth
   limiting. Similarly, I don't see why this is a meaningful change.
 * We use pindexBestHeader for requesting missing headers on receipt of a
   headers/compact block message or block inv as well as for initial getheaders.
   I think this is definitely wrong, using the best not-invalid header for such
   requests is much better.
 * We use pindexBestHeader to define the "current chain" for deciding when
   we're close to done with initial headers sync. I don't think this is a
   meaningful change.
 * We use pindexBestHeader to decide if initial headers sync has timed out. If
   we're rejecting the chain due to hardware error this may result in
   additional cases where we ban a peer, but this is already true, so I think
   its fine.
2019-10-30 13:33:48 -04:00
MarcoFalke
fa54b3e248 test: move-only ComputeFilter to src/test/lib/blockfilter 2019-10-30 13:19:30 -04:00
Wladimir J. van der Laan
a6abc94e93 Merge #17281: doc: Add developer note on c_str()
1cf9b35c0d doc: Add developer note on c_str() (Wladimir J. van der Laan)

Pull request description:

  Add a note when to use and when not to use `c_str()`.

ACKs for top commit:
  elichai:
    ACK 1cf9b35c0d
  MarcoFalke:
    Looking nice ACK 1cf9b35c0d

Tree-SHA512: 38cb5e54695782c23a82d03db214a8999b5bb52553f4fbe5322281686f42616981a217ba987feb6d87f3e6b95919cadd8484efe69ecc364ba1731aaf173626c9
2019-10-30 17:18:29 +01:00
Adam Jonas
c98bd13e67 replace asserts in RPC code with CHECK_NONFATAL and add linter 2019-10-30 12:03:07 -04:00
John Newbery
3ed772d221 [tests] remove bignum.py
It only contains one function and is only imported by one
other module (script.py). Just move the function to
script.py.
2019-10-30 11:02:51 -04:00
John Newbery
f950ec2520 [tests] remove bn2bin()
It's only called in one place.
2019-10-30 10:53:19 -04:00
John Newbery
3b9b38579c [tests] remove bn_bytes() function
It is one line and is called in one place.
2019-10-30 10:53:19 -04:00
John Newbery
a760aa14a9 [tests] remove mpi2vch() function
All it does is reverse the bytes order.
2019-10-30 10:53:16 -04:00
John Newbery
9a60bef50d [tests] don't encode the integer size in bignum
We just throw it away whenever we use the
result so don't add it.
2019-10-30 10:52:39 -04:00
John Newbery
1dc68aee66 [tests] add function comments to bignum 2019-10-30 10:52:39 -04:00
John Newbery
f31fc0e92e [tests] fix flake8 warnings in script.py and bignum.py 2019-10-30 10:52:33 -04:00
Wladimir J. van der Laan
3c40bc6726 Merge #15921: validation: Tidy up ValidationState interface
3004d5a12d [validation] Remove fMissingInputs from AcceptToMemoryPool() (John Newbery)
c428622a5b [validation] Remove unused first_invalid parameter from ProcessNewBlockHeaders() (John Newbery)
7204c6434b [validation] Remove useless ret parameter from Invalid() (John Newbery)
1a37de4b31 [validation] Remove error() calls from Invalid() calls (John Newbery)
067981e492 [validation] Tidy Up ValidationResult class (John Newbery)
a27a2957ed [validation] Add CValidationState subclasses (John Newbery)

Pull request description:

  Carries out some remaining tidy-ups remaining after PR 15141:

  - split ValidationState into TxValidationState and BlockValidationState (commit from ajtowns)
  - various minor code style tidy-ups to the ValidationState class
  - remove the useless `ret` parameter from `ValidationState::Invalid()`
  - remove the now unused `first_invalid` parameter from `ProcessNewBlockHeaders()`
  - remove the `fMissingInputs` parameter from `AcceptToMemoryPool()`, and deal with missing inputs the same way as other errors by using the `TxValidationState` object.

  Tip for reviewers (thanks ryanofsky!): The first commit ("[validation] Add CValidationState subclasses" ) is huge and can be easier to start reviewing if you revert the rote, mechanical changes:

  Substitute the commit hash of commit "[validation] Add CValidationState subclasses" for <CommitHash> in the commands below.

  ```sh
  git checkout <CommitHash>
  git grep -l ValidationState | xargs sed -i 's/BlockValidationState\|TxValidationState/CValidationState/g'
  git grep -l ValidationResult | xargs sed -i 's/BlockValidationResult\|TxValidationResult/ValidationInvalidReason/g'
  git grep -l MaybePunish | xargs sed -i 's/MaybePunishNode\(ForBlock\|ForTx\)/MaybePunishNode/g'
  git diff HEAD^
  ```

  After that it's possible to easily see the mechanical changes with:

  ```sh
  git log -p -n1 -U0 --word-diff-regex=. <CommitHash>
  ```

ACKs for top commit:
  laanwj:
    ACK 3004d5a12d
  amitiuttarwar:
    code review ACK 3004d5a12d. Also built & ran tests locally.
  fjahr:
    Code review ACK 3004d5a12d . Only nit style change and pure virtual destructor added since my last review.
  ryanofsky:
    Code review ACK 3004d5a12d. Just whitespace change and pure virtual destructor added since last review.

Tree-SHA512: 511de1fb380a18bec1944ea82b513b6192df632ee08bb16344a2df3c40811a88f3872f04df24bc93a41643c96c48f376a04551840fd804a961490d6c702c3d36
2019-10-30 15:37:34 +01:00
Wladimir J. van der Laan
cab94cc074 Merge #16943: test: Add generatetodescriptor RPC
fa144e6fde rpc: Add generatetodescriptor (MarcoFalke)

Pull request description:

  The existing `generatetoaddress` RPC can only generate to scriptPubKeys that can be represented by an address. However, raw scripts (such as `OP_TRUE`) or P2PK can not be represented by an address, which complicates testing.

ACKs for top commit:
  laanwj:
    ACK fa144e6fde

Tree-SHA512: aee934ab7e33f07c81f3b4c8ec23e7b6ddf63a1f4b86051af0bd76b75d8da1f51627cc682e5c6e42582340ca576bbf8ff724bdd43f87128ccecfa91e52d30ae7
2019-10-30 15:22:53 +01:00
MarcoFalke
341e8d355d Merge #17291: tests: Add fuzzing harness for ISO-8601 related functions
595cc9bcaf docs: Add undefined to --with-sanitizers=fuzzer,address (practicalswift)
d5dbb4898c tests: Add fuzzing harness for ISO-8601 related functions (practicalswift)

Pull request description:

  Add fuzzing harness for ISO-8601 related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/parse_iso8601
  …
  ```

Top commit has no ACKs.

Tree-SHA512: 8d4ad9e4eef546e97ea330cf518fdd7241c6f016d6c45c011369d5cdd832bbbc3564d1a990c953ffb33b0c05e58f5533e7b6fd77062f8484df36da1513567915
2019-10-30 10:17:54 -04:00
MarcoFalke
fa144e6fde rpc: Add generatetodescriptor 2019-10-30 10:01:32 -04:00
practicalswift
595cc9bcaf docs: Add undefined to --with-sanitizers=fuzzer,address 2019-10-30 13:34:10 +00:00
practicalswift
d5dbb4898c tests: Add fuzzing harness for ISO-8601 related functions 2019-10-30 13:32:29 +00:00
Wladimir J. van der Laan
d314e8a818 refactor: Replace all uses of boost::optional with our own Optional type
After this:

- `boost::optional` is no longer used directly (only through `Optional`
    which is an alias for it)
- `boost/optional.hpp` is only included in one place
2019-10-30 14:27:31 +01:00
Wladimir J. van der Laan
edd9d0781b Merge #17302: cli: Add "headers" and "verificationprogress" to -getinfo
31879345ee cli: Add "headers" and "verificationprogress" to -getinfo (Wladimir J. van der Laan)

Pull request description:

  These values are useful to know the current progress of initial sync, or of catching up, which is arguably the use of a quick `-getinfo` command.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 31879345ee
  jonasschnelli:
    utACK 31879345ee
  jonatack:
    Tested ACK 31879345ee on Debian 4.19.37-5+deb10u2 (2019-08-08) x86_64 GNU/Linux

Tree-SHA512: 185180ab426b4db5d99eb208ee88d1606f585361875ba3a92b6c28a74fe181d72ed710c8859b969ba49b1ca7d2385695932b79ff621c7a2a7cedd0df717a99ed
2019-10-30 12:38:31 +01:00
Wladimir J. van der Laan
471e5f8829 Merge #16839: Replace Connman and BanMan globals with NodeContext local
362ded410b Avoid using g_rpc_node global in wallet code (Russell Yanofsky)
8922d7f6b7 scripted-diff: Remove g_connman, g_banman globals (Russell Yanofsky)
e6f4f895d5 Pass NodeContext, ConnMan, BanMan references more places (Russell Yanofsky)
4d5448c76b MOVEONLY: Move NodeContext struct to node/context.h (Russell Yanofsky)
301bd41a2e scripted-diff: Rename InitInterfaces to NodeContext (Russell Yanofsky)

Pull request description:

  This change is mainly a naming / organization change intended to simplify #10102. It:

  - Renames struct InitInterfaces to struct NodeContext and moves it from
    src/init.h to src/node/context.h. This is a cosmetic change intended to make
    the point of the struct more obvious.

  - Gets rid of BanMan and ConnMan globals making them NodeContext members
    instead. Getting rid of these globals has been talked about in past as a way
    to implement testing and simulations. Making them NodeContext members is a
    way of keeping them accessible without the globals.

  - Splits g_rpc_interfaces global into g_rpc_node and g_rpc_chain globals. This
    better separates node and wallet rpc methods. Node RPC methods should have
    access NodeContext, while wallet RPC methods should only have indirect access
    to node functionality via interfaces::Chain.

  - Adds NodeContext& references to interfaces::Chain class and the
    interfaces::MakeChain() function. This is needed to access ConnMan and BanMan
    instances without the globals.

  - Gets rid of redundant Node and Chain instances in Qt tests. This is
    needed due to the previous MakeChain change, and also makes test setup a
    little more straightforward. More cleanup could be done in the future, but it
    will require deduplication of bitcoind, bitcoin-qt, and TestingSetup init
    code.

ACKs for top commit:
  laanwj:
    ACK 362ded410b

Tree-SHA512: 9ae6ff1e33423291d1e52056bac95e0874538390892a6e83c4c115b3c73155a8827c0191b46eb3d14e3b3f6c23ccb08095490880fbc3188026319c71739f7db2
2019-10-30 12:35:41 +01:00
Wladimir J. van der Laan
f129170b85 Merge #17306: refactor: Use name constants in chainparams initialization
37b8475dcf Chainparams: Use name constants in chainparams initialization (Jorge Timón)

Pull request description:

  I thought this wouldn't work for some reason, but it seems it does.
  Just a little bit more consistency. I'm still not able to use them in qt/networkstyle.cpp though, not sure why.

ACKs for top commit:
  MarcoFalke:
    ACK 37b8475dcf
  laanwj:
    ACK 37b8475dcf
  hebasto:
    ACK 37b8475dcf, I have reviewed the code and it looks OK, I agree it can be merged.
  fjahr:
    ACK 37b8475

Tree-SHA512: d9fa5df5650e10c645ac1f3afe831674a47f35d4a649e18a3d2aee1d04b08e6896aff6f1bbed0630d28775c51f989f9daaa9e405c9f3d7dca30e639a6f9008f0
2019-10-30 12:22:50 +01:00
Wladimir J. van der Laan
6daddc4495 Merge #17282: contrib: remove accounts from bash completion
dd94cc46e4 contrib: remove accounts from bash completion (fanquake)

Pull request description:

  Also removes `setgenerate`.

ACKs for top commit:
  jonasschnelli:
    utACK dd94cc46e4

Tree-SHA512: 218aa06841f4a347bb083c7f408c07fe80df7a30f7ce7a1126ec95971a9b66aff7c3c049d269da2c15f629bbde80e8c303bf6562bd5c1a36a6f6fd61ce9133e2
2019-10-30 11:53:25 +01:00
Wladimir J. van der Laan
1cf9b35c0d doc: Add developer note on c_str()
Add a note when to use and when not to use `c_str()`.
2019-10-30 10:53:27 +01:00
Wladimir J. van der Laan
5728f88d64 Merge #17280: refactor: Change occurences of c_str() used with size() to data()
f3b51eb935 Fix occurences of c_str() used with size() to data() (Wladimir J. van der Laan)

Pull request description:

  Using `data()` better communicates the intent here.

  ~~Also, depending on how `c_str()` is implemented, this fixes undefined behavior: The part of the string after the first NULL character might have undefined contents (or even be inaccessible, worst case).~~ Apparently [this is no longer an issue with C++11](https://github.com/bitcoin/bitcoin/pull/17281#discussion_r339742128).

ACKs for top commit:
  fjahr:
    Code review ACK f3b51eb
  practicalswift:
    ACK f3b51eb935 -- diff looks correct, `data()` more idiomatic
  ryanofsky:
    Code review ACK f3b51eb935. Most of these calls (including one in crypter.cpp) are passing text strings, not binary strings likely to contain `\0` and were probably safe before, but much better to avoid the possibility of bugs like this.

Tree-SHA512: 842e1bdd37efc4ece2ecb87ca34962aafef0a192180051def630607e349dc9c8b4e562481fff3de474515f493b4ee3ea53b00269a801a66e625326a38dfce5b8
2019-10-30 10:42:57 +01:00
MarcoFalke
ecad0a8019 Merge #17299: test: add reason checks for non-standard txs in test_IsStandard
c1c6c410a6 test: add reason checks for non-standard txs in test_IsStandard (Sebastian Falbesoner)

Pull request description:

  While taking a look at #17272 I noticed that for some reason the unit test `test_IsStandard` (which was not adapted to the policy change in the referenced PR commits) didn't fail as expected:
  6a97e8a060/src/test/transaction_tests.cpp (L758-L762)
  It turned out that `IsStandardTx()` returned `"dust"` as rejection reason (instead of the expected `"multi-op-return"`), leading to the conclusion that 5fe6f052bd erroneously performs the `IsDust()` check also for TX_NULL_DATA transactions. To avoid cases like this in the future, this PR makes the unit test `test_IsStandard` more strict by also checking for the concrete reason after each occurence of `IsStandardTx()` returning false.

ACKs for top commit:
  instagibbs:
    utACK c1c6c410a6

Tree-SHA512: c7419884cc52977c73f8f8c476eaebed80ba7bda4d03509d3f46dd977be911389f7b53daefa5ef31d2f7df9402243152e01e83f1b8a9fb300c19d1a0f69a89a9
2019-10-29 19:52:42 -04:00
John Newbery
3004d5a12d [validation] Remove fMissingInputs from AcceptToMemoryPool()
Handle this failure in the same way as all other failures: call Invalid()
with the reasons for the failure.
2019-10-29 15:46:45 -04:00
John Newbery
c428622a5b [validation] Remove unused first_invalid parameter from ProcessNewBlockHeaders()
No callers use the returned value in first_invalid. Remove it from the
function signature and don't set it in the function.
2019-10-29 15:46:45 -04:00
John Newbery
7204c6434b [validation] Remove useless ret parameter from Invalid()
ValidationState::Invalid() takes a parameter `ret` which is returned to
the caller. All call sites set this to false. Remove the `ret` parameter
and just return false always.
2019-10-29 15:46:45 -04:00
John Newbery
1a37de4b31 [validation] Remove error() calls from Invalid() calls
This is in preparation for the next commit, which removes the useless
`ret` parameter from ValidationState::Invalid().

error() is simply a convenience wrapper that calls LogPrintf and returns
false. Call LogPrintf explicitly and substitute the error() call for a
false bool literal.
2019-10-29 15:46:45 -04:00
John Newbery
067981e492 [validation] Tidy Up ValidationResult class
Minor style fixups and comment updates.

This is purely a style change. There is no change in behavior.
2019-10-29 15:46:45 -04:00
John Newbery
a27a2957ed [validation] Add CValidationState subclasses
Split CValidationState into TxValidationState and BlockValidationState
to store validation results for transactions and blocks respectively.
2019-10-29 15:46:45 -04:00
Jorge Timón
37b8475dcf Chainparams: Use name constants in chainparams initialization 2019-10-29 20:27:30 +01:00
Carl Dong
3b3b93174a nsis: Write to correct filename in first place 2019-10-29 15:12:52 -04:00
Wladimir J. van der Laan
31879345ee cli: Add "headers" and "verificationprogress" to -getinfo
These value are useful to know the current progress of
initial sync, or of catching up.
2019-10-29 20:00:58 +01:00
James O'Beirne
707fde7b9b add unused SnapshotMetadata class 2019-10-29 13:55:10 -04:00
Fabian Jahr
d6e493f0c2 wallet: Remove left-over BIP70 comment 2019-10-29 17:53:46 +01:00
Russell Yanofsky
53fe0b70ad Fix missing strFailReason in CreateTransaction
Suggested by MarcoFalke <falke.marco@gmail.com>
https://github.com/bitcoin/bitcoin/pull/17260#discussion_r340036269
2019-10-29 12:25:28 -04:00
Russell Yanofsky
4b28a05f08 Fix misplaced AssertLockHeld
Suggestion from MarcoFalke <falke.marco@gmail.com>
https://github.com/bitcoin/bitcoin/pull/17260#discussion_r340033021
2019-10-29 12:24:34 -04:00
Russell Yanofsky
2632b1f124 doc: Clarify WalletStorage / Wallet relation
Suggested by MarcoFalke <falke.marco@gmail.com>
https://github.com/bitcoin/bitcoin/pull/17260#discussion_r340031507
2019-10-29 12:23:47 -04:00
Russell Yanofsky
628d11b2ba Add back mistakenly removed AssertLockHeld
Suggestion from MarcoFalke <falke.marco@gmail.com>
https://github.com/bitcoin/bitcoin/pull/17260#discussion_r340029481
2019-10-29 12:21:57 -04:00
Russell Yanofsky
52cf68f7ff Refactor: Add GetLegacyScriptPubKeyMan helper
Suggested by João Barbosa <joao.paulo.barbosa@gmail.com>
https://github.com/bitcoin/bitcoin/pull/17260#discussion_r339505236
2019-10-29 12:20:19 -04:00
Sebastian Falbesoner
c1c6c410a6 test: add reason checks for non-standard txs in test_IsStandard 2019-10-29 16:02:19 +01:00
MarcoFalke
6a97e8a060 Merge #17260: Split some CWallet functions into new LegacyScriptPubKeyMan
f201ba59ff Refactor: Split up CWallet and LegacyScriptPubKeyMan and classes (Andrew Chow)
6702048f91 MOVEONLY: Move key handling code out of wallet to keyman file (Andrew Chow)
ab053ec6d1 Move wallet enums to walletutil.h (Andrew Chow)

Pull request description:

  Moves key management functions into a new class LegacyScriptPubKeyMan. First two commits are move-only commits which move stuff out of wallet.{h/cpp} and into newly created scriptpubkeyman.{h/cpp}. Third commit changes several things in CWallet to use LegacyScriptPubKeyMan.

  First step in the wallet boxes refactor. Note that LegacyScriptPubKeyMan and ScriptPubKeyMan cannot be used standalone yet and are still very much tied into CWallet with both accessing functions within each other. This PR is to help reduce review burden.

ACKs for top commit:
  Sjors:
    Code review ACK f201ba5.
  promag:
    Code review ACK f201ba59ff.
  ryanofsky:
    Code review ACK f201ba59ff
  MarcoFalke:
    ACK f201ba59ff

Tree-SHA512: bdc0d8595a06233fe003afcf968a38e0e8cc584a6a89c5bcd05309ac29dca852391802d46763ef81a108d146d0f40c79ea5438e87234ed12b4b8360c9aec94c0
2019-10-29 08:19:23 -04:00
Hennadii Stepanov
f44abe4bed refactor: Remove addrdb.h dependency from node.h 2019-10-29 11:30:12 +02:00
Jeremy Rubin
a35b6824f3 Add assertion to randrange that input is not 0 2019-10-28 16:42:39 -07:00
Jeremy Rubin
b0c774b48a Add new mempool benchmarks for a complex pool 2019-10-28 15:58:48 -07:00
MarcoFalke
4c1090c882 Merge #17279: refactor: Remove redundant c_str() calls in formatting
c72906dcc1 refactor: Remove redundant c_str() calls in formatting (Wladimir J. van der Laan)

Pull request description:

  Our formatter, tinyformat, *never* needs `c_str()` for strings. Still, many places call it redundantly, resulting in longer code and a slight overhead.

  Remove redundant `c_str()` calls for:

  - `strprintf`
  - `LogPrintf`
  - `tfm::format`

  (also, combined with #17095, I think this improves logging in case of unexpected embedded NULL characters)

ACKs for top commit:
  ryanofsky:
    Code review ACK c72906dcc1. Easy to review with `git log -p -n1 --word-diff-regex=. -U0 c72906dcc11a73fa06a0adf97557fa756b551bee`

Tree-SHA512: 9e21e7bed8aaff59b8b8aa11571396ddc265fb29608c2545b1fcdbbb36d65b37eb361db6688dd36035eab0c110f8de255375cfda50df3d9d7708bc092f67fefc
2019-10-28 15:10:06 -04:00
fanquake
dd94cc46e4 contrib: remove accounts from bash completion
Also removes setgenerate
2019-10-28 11:53:40 -04:00
practicalswift
d8daa8f371 pubkey: Assert CPubKey's ECCVerifyHandle precondition 2019-10-28 15:10:13 +00:00
MarcoFalke
cfec3e01b4 Merge #17266: util: Rename DecodeDumpTime to ParseISO8601DateTime
e7b02b54cc Add roundtrip and more tests to ParseISO8601DateTime and FormatISO8601DateTime (Elichai Turkel)
9e2c623be5 Rename DecodeDumpTime to ParseISO8601DateTime and move to time.cpp (Elichai Turkel)

Pull request description:

  As discussed in #17245.

  1. Renamed the function.
  2. Moved it from `rpcdump.cpp` to `time.cpp`.
  3. Added a check if the time is less then epoch return 0 to prevent an overflow.
  4. Added more edge cases tests and a roundtrip test.

ACKs for top commit:
  laanwj:
    ACK e7b02b54cc
  MarcoFalke:
    ACK e7b02b54cc
  promag:
    Code review ACK e7b02b54cc. Moved code is correct, left a comment regarding the test change.

Tree-SHA512: 703c21e09b2aabc992235149e67acba63d9d77a593ec8f6d2fec3eb63a7e5c406d56cbce6c6513ab32fba43367d073d2345f3b589843e3c5fe4f55ea3e00bf29
2019-10-28 10:30:51 -04:00
Russell Yanofsky
362ded410b Avoid using g_rpc_node global in wallet code
Wallet code should use interfaces::Chain and not directly access to node state.

Add a g_rpc_chain replacement global for wallet code to use, and move
g_rpc_node definition to a libbitcoin_server source file so there are link
errors if wallet code tries to access it.
2019-10-28 10:30:51 -04:00
Russell Yanofsky
8922d7f6b7 scripted-diff: Remove g_connman, g_banman globals
-BEGIN VERIFY SCRIPT-
sed -i 's:#include <interfaces/chain.h>:#include <banman.h>\n#include <interfaces/chain.h>\n#include <net.h>\n#include <net_processing.h>:' src/node/context.cpp
sed -i 's/namespace interfaces {/class BanMan;\nclass CConnman;\nclass PeerLogicValidation;\n&/' src/node/context.h
sed -i 's/std::unique_ptr<interfaces::Chain> chain/std::unique_ptr<CConnman> connman;\n    std::unique_ptr<PeerLogicValidation> peer_logic;\n    std::unique_ptr<BanMan> banman;\n    &/' src/node/context.h
sed -i '/std::unique_ptr<[^>]\+> \(g_connman\|g_banman\|peerLogic\);/d' src/banman.h src/net.h src/init.cpp
sed -i 's/g_connman/m_context.connman/g' src/interfaces/node.cpp
sed -i 's/g_banman/m_context.banman/g' src/interfaces/node.cpp
sed -i 's/g_connman/m_node.connman/g' src/interfaces/chain.cpp src/test/setup_common.cpp
sed -i 's/g_banman/m_node.banman/g' src/test/setup_common.cpp
sed -i 's/g_connman/node.connman/g' src/init.cpp src/node/transaction.cpp
sed -i 's/g_banman/node.banman/g' src/init.cpp
sed -i 's/peerLogic/node.peer_logic/g' src/init.cpp
sed -i 's/g_connman/g_rpc_node->connman/g' src/rpc/mining.cpp src/rpc/net.cpp src/rpc/rawtransaction.cpp
sed -i 's/g_banman/g_rpc_node->banman/g' src/rpc/net.cpp
sed -i 's/std::shared_ptr<CWallet> wallet =/node.context()->connman = std::move(test.m_node.connman);\n    &/' src/qt/test/wallettests.cpp
-END VERIFY SCRIPT-
2019-10-28 10:30:51 -04:00
Russell Yanofsky
e6f4f895d5 Pass NodeContext, ConnMan, BanMan references more places
So g_connman and g_banman globals can be removed next commit.
2019-10-28 10:30:51 -04:00
Russell Yanofsky
4d5448c76b MOVEONLY: Move NodeContext struct to node/context.h 2019-10-28 10:30:51 -04:00
Russell Yanofsky
301bd41a2e scripted-diff: Rename InitInterfaces to NodeContext
-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; }

s 'struct InitInterfaces'              'struct NodeContext'
s 'InitInterfaces interfaces'          'NodeContext node'
s 'InitInterfaces& interfaces'         'NodeContext\& node'
s 'InitInterfaces m_interfaces'        'NodeContext m_context'
s 'InitInterfaces\* g_rpc_interfaces'  'NodeContext* g_rpc_node'
s 'g_rpc_interfaces = &interfaces'     'g_rpc_node = \&node'
s 'g_rpc_interfaces'                   'g_rpc_node'
s 'm_interfaces'                       'm_context'
s 'interfaces\.chain'                  'node.chain'
s '\(AppInitMain\|Shutdown\|Construct\)(interfaces)' '\1(node)'
s 'init interfaces' 'chain clients'
-END VERIFY SCRIPT-
2019-10-28 10:30:51 -04:00
Adam Jonas
436ad43643 Fix issue with conflicted mempool tx in listsinceblock
listsinceblock now checks that returned transactions are not
conflicting with any transactions that are filtered out by
the given blockhash

Co-Authored-By: Michael Chrostowski <michael.chrostowski@gmail.com>
2019-10-28 10:26:46 -04:00
fanquake
1ab6c1267e Merge #16986: doc: Doxygen-friendly CuckooCache comments
7aad3b68e7 doc: Doxygen-friendly CuckooCache comments (Jon Layton)

Pull request description:

  Similar theme to #16947.

  - `invalid`, `contains` now appear in Doxygen docs
  - `setup` refers to correct argument name `b`
  - Argument references in `code blocks `
  - Lists markdown conformant, uniform line endings

  Tested with `make docs`

ACKs for top commit:
  laanwj:
    ACK 7aad3b68e7
  practicalswift:
    ACK 7aad3b68e7

Tree-SHA512: 70b38c10e534bad9c6ffcd88cc7a4797644afba5956d47a6c7cc655fcd5857a91f315d6da60e28ce9678d420ed4a51e22267eb8b89e26002b99cad63373dd349
2019-10-28 09:26:44 -04:00
fanquake
badca85e2c Merge #16202: p2p: Refactor network message deserialization
ed2dc5e48a Add override/final modifiers to V1TransportDeserializer (Pieter Wuille)
f342a5e61a Make resetting implicit in TransportDeserializer::Read() (Pieter Wuille)
6a91499496 Remove oversized message detection from log and interface (Pieter Wuille)
b0e10ff4df Force CNetMessage::m_recv to use std::move (Jonas Schnelli)
efecb74677 Use adapter pattern for the network deserializer (Jonas Schnelli)
1a5c656c31 Remove transport protocol knowhow from CNetMessage / net processing (Jonas Schnelli)
6294ecdb8b Refactor: split network transport deserializing from message container (Jonas Schnelli)

Pull request description:

  **This refactors the network message deserialization.**

  * It transforms the `CNetMessage` into a transport protocol agnostic message container.
  * A new class `TransportDeserializer` (unique pointer of `CNode`)  is introduced, handling the network buffer reading and the decomposing to a `CNetMessage`
  * **No behavioral changes** (in terms of disconnecting, punishing)
  * Moves the checksum finalizing into the `SocketHandler` thread (finalizing was in `ProcessMessages` before)

  The **optional last commit** makes the `TransportDeserializer` following an adapter pattern (polymorphic interface) to make it easier to later add a V2 transport protocol deserializer.

  Intentionally not touching the sending part.

  Pre-Requirement for BIP324 (v2 message transport protocol).
  Replacement for #14046 and inspired by a [comment](https://github.com/bitcoin/bitcoin/pull/14046#issuecomment-431528330) from sipa

ACKs for top commit:
  promag:
    Code review ACK ed2dc5e48a.
  marcinja:
    Code review ACK ed2dc5e48a
  ryanofsky:
    Code review ACK ed2dc5e48a. 4 cleanup commits added since last review. Unaddressed comments:
  ariard:
    Code review and tested ACK ed2dc5e.

Tree-SHA512: bab8d87464e2e8742529e488ddcdc8650f0c2025c9130913df00a0b17ecdb9a525061cbbbd0de0251b76bf75a8edb72e3ad0dbf5b79e26f2ad05d61b4e4ded6d
2019-10-28 09:15:59 -04:00
Wladimir J. van der Laan
f8cc2b967b Merge #17267: bench: Fix negative values and zero for -evals flag
3bb0a4674f bench: Fix negative values and zero for -evals flag (nijynot)

Pull request description:

  This PR makes `bench_bitcoin -evals=0` evaluate at once and throws when `-evals` is a negative integer.

  ---

  Currently when you run `bench_bitcoin -evals=0`, it'll get stuck at
  ```
  # Benchmark, evals, iterations, total, min, max, median
  ```
  . This is not intuitively expected and should instead evaluate instantly as it's set to zero. Negative integers for `-evals` does not make sense either and should throw if set.

ACKs for top commit:
  laanwj:
    ACK 3bb0a4674f

Tree-SHA512: 03cd4c7c55134c7ffd8cdb6ee993551ce41061a73e13c3c047247af9df1fd7ed07d798272b643ec864099036922aaadbdcd2b798d710406f48df60b9d5448c26
2019-10-28 14:14:11 +01:00
Wladimir J. van der Laan
f3b51eb935 Fix occurences of c_str() used with size() to data()
Using `data()` better communicates the intent here.

Also, depending on how `c_str()` is implemented, this fixes undefined
behavior: The part of the string after the first NULL character might
have undefined contents.
2019-10-28 13:41:45 +01:00
Wladimir J. van der Laan
c72906dcc1 refactor: Remove redundant c_str() calls in formatting
Our formatter, tinyformat, *never* needs `c_str()` for strings.
Remove redundant `c_str()` calls for:

- `strprintf`
- `LogPrintf`
- `tfm::format`
2019-10-28 13:31:33 +01:00
nijynot
3bb0a4674f bench: Fix negative values and zero for -evals flag 2019-10-28 13:07:38 +01:00
Wladimir J. van der Laan
a25945318f Merge #17250: Avoid unused call to GuessVerificationProgress in NotifyHeaderTip
fa398091b7 Avoid unused call to GuessVerificationProgress in NotifyHeaderTip (MarcoFalke)

Pull request description:

  `GuessVerificationProgress` for a header (not a block) is always 0 because the number of txs in the block can not be determined from the header alone. Anyway, this result was never used, so we can optimize this call by hardcoding 0.

  This is the next commit in a series of changes toward removing nChainTx (see #14863, #13875)

ACKs for top commit:
  promag:
    Code review ACK fa398091b7, missed that.
  laanwj:
    ACK fa398091b7

Tree-SHA512: 11016f8dbb1af1cf75241948d1ad35eac0c79d1311cd0db8c6ec806df2a9e3dc5f998dbd66ccbad5d84564e6cec7fe21ce7a2a13c2b34c746e2d3b31aa1db53a
2019-10-28 12:55:04 +01:00
Wladimir J. van der Laan
9ae468a6d5 Merge #17192: util: Add CHECK_NONFATAL and use it in src/rpc
faeb666536 util: Add CHECK_NONFATAL and use it in src/rpc (MarcoFalke)

Pull request description:

  Fixes #17181

  Currently, we use `assert` in RPC code to document logic and code assumptions. However, it seems a bit extreme to abort all of Bitcoin Core on an assert failure in one of the RPC threads. I suggest to replace all `assert`s with a macro `CHECK_NONFATAL(condition)` that throws a runtime error when the condition evaluates to `false`. That runtime error will then be returned to the rpc caller and will include instructions to report the error to our issue tracker.

ACKs for top commit:
  practicalswift:
    ACK faeb666536
  laanwj:
    ACK faeb666536
  ryanofsky:
    Code review ACK faeb666536

Tree-SHA512: 9b748715a5e0767ac11f1324a95a3a6ec672a0e0658013492219223bda83ce4b1b447fd8183bbb235f7df5ef7dddda7666ad569544b4d61cc65f232ca7a800ec
2019-10-28 12:00:36 +01:00
Rjected
317fb96de9 Add search for first blk file with pruned node 2019-10-27 23:08:10 -04:00
practicalswift
9cae3d5e94 tests: Add fuzzer initialization (hold ECCVerifyHandle) 2019-10-27 21:22:24 +00:00
Elichai Turkel
e7b02b54cc Add roundtrip and more tests to ParseISO8601DateTime and FormatISO8601DateTime 2019-10-27 01:00:13 +03:00
Elichai Turkel
9e2c623be5 Rename DecodeDumpTime to ParseISO8601DateTime and move to time.cpp 2019-10-27 01:00:05 +03:00
João Barbosa
6d6a7a8403 gui: Fix duplicate wallet showing up
The slot BitcoinGUI::addWallet can be invoked twice for the same
WalletModel due to a concurrent wallet being loaded after the first `connect()`:

```cpp
 connect(wallet_controller, &WalletController::walletAdded, this, &BitcoinGUI::addWallet);
 connect(wallet_controller, &WalletController::walletRemoved, this, &BitcoinGUI::removeWallet);

 for (WalletModel* wallet_model : m_wallet_controller->getOpenWallets()) {
     addWallet(wallet_model);
```
2019-10-26 14:55:31 +01:00
Russell Yanofsky
81ea66c30e Drop signal CClientUIInterface::LoadWallet 2019-10-26 14:55:30 +01:00
Jorge Timón
1abcecc40c Tests: Use self.chain instead of 'regtest' in almost all current tests 2019-10-26 13:24:39 +02:00
Wladimir J. van der Laan
be50469217 Merge #17257: gui: disable font antialiasing for QR image address
e156b9d8b9 gui: disable font antialiasing for QR image address (fanquake)

Pull request description:

  The address text inside the QR code is currently fairly blurry / unreadable. Explicitly disabling font antialiasing improves that somewhat.

  master (693e40090a):
  ![macOS_master](https://user-images.githubusercontent.com/863730/67591414-644e0580-f72b-11e9-8399-2cd0584e7d62.png)

  PR (e156b9d8b9):
  ![macOS_pr](https://user-images.githubusercontent.com/863730/67591424-6dd76d80-f72b-11e9-86b6-b3911f8e07e6.png)

ACKs for top commit:
  laanwj:
    ACK e156b9d8b9

Tree-SHA512: 32aeb2ffe8164a1006f80e76c6e413fcb88e32ced42d2b2af69cca908bd32673f3e379184be917f1870864b940db943e7f46a7ecb0779343d5d129b381660c38
2019-10-26 13:06:18 +02:00
Wladimir J. van der Laan
9bd109b78d Merge #17120: gui: Fix start timer from non QThread
a8f5026d6d gui: Fix start timer from non QThread (João Barbosa)

Pull request description:

  Fixes #16296.

ACKs for top commit:
  laanwj:
    code review ACK a8f5026d6d

Tree-SHA512: d7b05ac88e188de16cbbe80cb2f773b7976ee07ee876ac94a93f9351856c4f3a9d66a531d3f3748d2dccff8c8d77d9d8227433069ed5909c32be2efeaa32f655
2019-10-26 13:00:16 +02:00
Wladimir J. van der Laan
3f875744cb Merge #17249: rpc: Add missing deque include to fix build
a592913022 http: add missing header bootlegged by boost < 1.72 (Jan Beich)

Pull request description:

  Regressed by boostorg/filesystem@9a14c37d6f. See [error log](https://github.com/bitcoin/bitcoin/files/3772177/bitcoin-0.18.1.log).

  ```c++
  httpserver.cpp:74:10: error: no template named 'deque' in namespace 'std'
      std::deque<std::unique_ptr<WorkItem>> queue;
      ~~~~~^
  ```

ACKs for top commit:
  laanwj:
    ACK a592913022

Tree-SHA512: fb0aee6a698c7aaa6a73baad7adc4f891be573af0d3cf6f4f59bc825afe5c0bc439c668077ff1990a6135522a0533a1a867430eebad28f0ade93fd79a95e179b
2019-10-26 12:05:04 +02:00
Sjors Provoost
29a21c9061 [rpc] set default bip32derivs to true for psbt methods 2019-10-26 12:03:38 +02:00
Wladimir J. van der Laan
e9f73b839a Merge #17135: gui: Make polling in ClientModel asynchronous
6b6be41c36 gui: Make polling in ClientModel asynchronous (João Barbosa)

Pull request description:

  After #14193 `ClientModel::updateTimer` can take some time, as such the GUI hangs, like #17112.

  Fixes this by polling in a background thread and updating the GUI asynchronously.

ACKs for top commit:
  laanwj:
    ACK 6b6be41c36
  Sjors:
    Code review re-ACK 6b6be41; only replaced the scary cast with `{ timer->start(); }`

Tree-SHA512: fd98b0c6535441aee3ee03c48b58b4b1f9bdd172ec6b8150da883022f719df34cabfd4c133412bf410e7f709f7bf1e9ef16dca05ef1f3689d526ceaeee51de38
2019-10-26 11:49:51 +02:00
Wladimir J. van der Laan
d91af4768c Merge #17165: Remove BIP70 support
8c6081a884 compat: remove bswap_* check on macOS (fanquake)
2cba35ab38 build: skip building OpenSSL lib_ssl (fanquake)
45a2d3c552 build: remove OpenSSL from Qt build (fanquake)
befbc40eb5 build: remove EVP_MD_CTX_new detection (fanquake)
fcee10c2d0 build: remove SSL lib detection (fanquake)
c7f30dbca8 gui: Update BIP70 support message (fanquake)
a3e810326d build: remove BIP70 entries from macOS Info.plist (fanquake)
72fe13a58d gui: remove payment request file handling from OpenURI dialog (fanquake)
3548e4aac7 Remove BIP70 Support (fanquake)
1cb9a4e28c docs: remove protobuf from docs (fanquake)
67328bb7ca build: remove protobuf from depends (fanquake)

Pull request description:

  This removes [BIP70](https://github.com/bitcoin/bips/blob/master/bip-0070.mediawiki) support. It also removes OpenSSL linking from Qt and building OpenSSLs `lib_ssl` in depends, as well as SSL lib detection from the build system. It's something that I'd optimistically like to do for `0.20.0`.

ACKs for top commit:
  laanwj:
    Code review ACK 8c6081a884
  MarcoFalke:
    ACK 8c6081a884
  fjahr:
    ACK 8c6081a

Tree-SHA512: 9dd9153afa4eca1a795f983e5b31f5fee9fa9a064c2a95d2f98810689add3ad0bf221c4608282299e66e4d1ec31cd556d4b16eea55de7912c3b9931f64735883
2019-10-26 11:35:45 +02:00
Andrew Chow
f201ba59ff Refactor: Split up CWallet and LegacyScriptPubKeyMan and classes
This moves CWallet members and methods dealing with keys to a new
LegacyScriptPubKeyMan class, and updates calling code to reference the new
class instead of CWallet.

Most of the changes are simple text replacements and variable substitutions
easily verified with:

    git log -p -n1 -U0 --word-diff-regex=.

The only nontrivial chunk of code added is the new LegacyScriptPubKeyMan class
declaration, but this code isn't new and is just selectively copied and moved
from the previous CWallet class declaration. This can be verified with:

    git log -p -n1 --color-moved=dimmed_zebra src/wallet/scriptpubkeyman.h src/wallet/wallet.h

or

    git diff HEAD~1:src/wallet/wallet.h HEAD:src/wallet/scriptpubkeyman.h

This commit does not change behavior.
2019-10-25 19:20:24 -04:00
Andrew Chow
6702048f91 MOVEONLY: Move key handling code out of wallet to keyman file
Start moving wallet and ismine code to scriptpubkeyman.h, scriptpubkeyman.cpp

The easiest way to review this commit is to run:

   git log -p -n1 --color-moved=dimmed_zebra

And check that everything is a move (other than includes and copyrights comments).

This commit is move-only and doesn't change code or affect behavior.
2019-10-25 19:20:24 -04:00
Andrew Chow
ab053ec6d1 Move wallet enums to walletutil.h 2019-10-25 19:20:24 -04:00
User
a552e8477c added asserts to check m_addr_known when it's used 2019-10-25 16:28:14 -04:00
fanquake
25d7e2e781 Merge #17251: net: SocketHandler logs peer id for close and disconnect
04dbdd613f [net] SocketHandler: log peer id for close and disconnect (Sjors Provoost)

Pull request description:

  When combined with `-logips` this makes it easier to diagnose disconnects.

  To test on macOS, find a connection you want to disrupt:

  ```
  lsof -nP -iTCP:8333 -sTCP:ESTABLISHED
  ```

  To shut it down gracefully you can use tcpkill or this Python script: https://github.com/google/tcp_killer

  The log should say:

  ```
  2019-10-25T13:26:55Z socket closed for peer=1
  2019-10-25T13:26:55Z disconnecting peer=1
  2019-10-25T13:26:55Z Cleared nodestate for peer=1
  ```

  To shut it down ungracefully I made a patch to the above script, adding a `-force` argument. _Careful, this may result in data corruption_. Then the log should say:

  ```
  2019-10-25T13:39:57Z socket select error Bad file descriptor (9)
  2019-10-25T13:39:57Z socket recv error for peer=0: Bad file descriptor (9)
  2019-10-25T13:39:57Z disconnecting peer=0
  2019-10-25T13:39:57Z Socket close failed: 35. Error: Bad file descriptor (9)
  2019-10-25T13:39:57Z Cleared nodestate for peer=0
  ```

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 04dbdd613f
  TheBlueMatt:
    unsigned ACK 04dbdd613f LGTM!
  theuni:
    unsigned ACK 04dbdd613f.

Tree-SHA512: 415313908484f97ffe11a48b4ed6afab3ab0be660c788adb9ad975f88b69aa1cfd5ccbe5859350cdf19ef8fde191fd530fb22cef34e70638defdc9f3d761c71d
2019-10-25 15:07:58 -04:00
Sjors Provoost
04dbdd613f [net] SocketHandler: log peer id for close and disconnect 2019-10-25 20:05:30 +02:00
fanquake
e156b9d8b9 gui: disable font antialiasing for QR image address
More info available here: https://doc.qt.io/qt-5/qfont.html#StyleStrategy-enum
2019-10-25 13:20:32 -04:00
kodslav
5710dadf9b test: fix script_p2sh_tests OP_PUSHBACK2/4 missing
Fixes commit 6b25f29a91 where opcodes where lost in translation.
2019-10-25 12:13:19 -04:00
MarcoFalke
693e40090a Merge #17083: tests: Add fuzzing harness for various CScript related functions
dc2fdb9907 tests: Add fuzzing harness for various CScript related functions (practicalswift)

Pull request description:

  Add fuzzing harness for various `CScript` related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/script
  …
  # And to to quickly verify that the relevant code regions are triggered, that the
  # fuzzing throughput seems reasonable, etc.
  $ contrib/devtools/test_fuzzing_harnesses.sh '^script$'
  ```

  `test_fuzzing_harnesses.sh` can be found in PR #17000.

Top commit has no ACKs.

Tree-SHA512: a0c5dca3b64ae177020b2ca299a29015d70755231b6bf01edbfc67c8aac90c44b1b4d57350c3aebef6e031108e6ae8e5fa0987c67707831c314f5d3090e0cee8
2019-10-25 11:05:56 -04:00
João Barbosa
6b6be41c36 gui: Make polling in ClientModel asynchronous
With this change polling runs in a different thread to prevent
disturbing the event loop.
2019-10-25 14:53:37 +01:00
MarcoFalke
fa398091b7 Avoid unused call to GuessVerificationProgress in NotifyHeaderTip 2019-10-25 09:12:15 -04:00
Jan Beich
a592913022 http: add missing header bootlegged by boost < 1.72
httpserver.cpp:74:10: error: no template named 'deque' in namespace 'std'
    std::deque<std::unique_ptr<WorkItem>> queue;
    ~~~~~^
2019-10-25 13:11:09 +00:00
fanquake
48cb468ce3 Merge #17242: refactor: Remove unused cacheSigStore from CheckInputsFromMempooAndCache
0a433fc876 [validation] Remove unused cacheSigStore from CheckInputsFromMempoolAndCache (John Newbery)

Pull request description:

  CheckInputsFromMempoolAndCache() is only called in one place, and
  cacheSigStore is set to true in that call site. Remove the argument
  entirely.

  Also improve commenting.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 0a433fc876 Comment looks good
  jamesob:
    ACK 0a433fc876
  laanwj:
    ACK 0a433fc876
  fanquake:
    ACK 0a433fc876. Checked that `CheckInputsFromMempoolAndCache` is only called once, in `MemPoolAccept::ConsensusScriptChecks`, and that `cacheSigStore` is true.

Tree-SHA512: e4b4d2550e35df55c8f8fa4c539174cc2d3728112ddb937cb2ff759d8630a01566b5ec42a70a82e33994e6586f5a457a75a59f64b15d27c65331c723cbb097af
2019-10-25 08:14:11 -04:00
Wladimir J. van der Laan
37855ec9df Merge #17220: tests: Add unit testing for the CompressScript function
b05ec410f2 Add unit testing for the CompressScript functions (marcaiaf)

Pull request description:

  Salvaging #15104 which adds unit tests for CompressScript function in `compressor.cpp`

  Tested following cases for the CScript:
    - CKeyID
    - CScriptID
    - Uncompressed CPubKey (of size: 65)
    - Compressed CPubKey (of size: 32)

ACKs for top commit:
  theStack:
    ACK b05ec410f2

Tree-SHA512: 7e23ace39383122802dfe5f7d38190d772f5db4045a67b7a9bd4c06797a17e0cdc41d6fac92d448057eb7df50172155dc824587c16c68c79fd1a4de37b772001
2019-10-25 13:55:58 +02:00
Wladimir J. van der Laan
90ed98ae9a Merge #17080: consensus: Explain why fCheckDuplicateInputs can not be skipped and remove it
fa92813407 consensus: Explain why fCheckDuplicateInputs can not be skipped and remove it (MarcoFalke)

Pull request description:

  As a follow up to CVE-2018-17144, this removes the unused `fCheckDuplicateInputs` parameter and explains why the test can not be disabled. Apart from protecting against a dumb accident in the future, this should document the logic in the code. There is a technical write-up that explains how the underlying coins database behaves if this test is skipped: https://bitcoincore.org/en/2018/09/20/notice/#technical-details. However, it does not explicitly mention why the test can not be skipped. I hope my code comment does that.

ACKs for top commit:
  jnewbery:
    ACK fa92813407
  amitiuttarwar:
    utACK fa92813407
  Empact:
    Code review ACK fa92813407
  promag:
    ACK fa92813407.

Tree-SHA512: fc1ef670f1a467c543b84f704b9bd8cc7a59a9f707be048bd9b4e85fe70830702aa560a880efa2c840bb43818ab44dfdc611104df04db2ddc14ff92f46bfb28e
2019-10-25 13:47:28 +02:00
Wladimir J. van der Laan
366753e46e Merge #17231: depends: fix boost mac cross build with clang 9+
50037e97d1 depends: fix boost mac cross build with clang 9+ (Cory Fields)

Pull request description:

  The ancient "darwin-4.9.1" profile has long been used to match against clang, which prior to version 9, reported 4.9.1 as its version when invoking "clang++ -dumpversion". Presumably this was a historical compatibility quirk related to Apple's switch from gcc to clang.

  This was "fixed" in clang 9.0, so that -dumpversion reports the real version. Unfortunately that had the side-effect of breaking the (brittle) boost compiler detection.

  Move to the seemingly more-correct "clang-darwin" profile, which passes the checks and builds correctly.

  Also switch to using ar rather than libtool for archiving, as it's what the clang-darwin profile expects to be using.

  Note that because this is using a different profile, some of the final command-line arguments end up changing. Those changes look sane at a glance.

ACKs for top commit:
  fanquake:
    ACK 50037e97d1 - tested on on macOS, will wait for the gitian build.

Tree-SHA512: eac1f353513a445add6fbece7fc78dd3dbdde5e2219bfb7739b82f40bb14de449667a94d2e303d43c67d9b38e7ceb0ba5f0d8fe20b40be2017b1ca0875467c2c
2019-10-25 12:03:01 +02:00
practicalswift
dc2fdb9907 tests: Add fuzzing harness for various CScript related functions 2019-10-25 08:28:13 +00:00
MarcoFalke
fafa064d2a ci: Remove ccache requirement on the host
ccache is only needed to create the cache dir on the host, if it didn't
already exist. The same can be achieved with mkdir, so just use that
instead.
2019-10-24 18:37:38 -04:00
MarcoFalke
fce7c75422 Merge #16851: Continue relaying transactions after they expire from mapRelay
168b781fe7 Continue relaying transactions after they expire from mapRelay (Anthony Towns)

Pull request description:

  This change allows peers to request transactions even after they've expired from mapRelay and even if they're not doing mempool requests. This is intended to allow for CPFP of old transactions -- if parent tx P wasn't relayed due to low fees, then a higher fee rate child C is relayed, peers will currently request the parent P, but we prior to this patch, we will not relay it due to it not being in mapRelay.

ACKs for top commit:
  MarcoFalke:
    re-ACK 168b781fe7 (only change is comment fixup)
  sdaftuar:
    re-ACK 168b781fe7
  sipa:
    ACK 168b781fe7

Tree-SHA512: b206666dd1450cd0a161ae55fd1a7eda2c3d226842ba27d91fe463b551fd924b65b92551b14d6786692e15cf9a9a989666550dfc980b48ab0f8d4ca305bc7762
2019-10-24 17:50:42 -04:00
MarcoFalke
773026044f Merge #17212: refactor: Remove unused CExt{Pub,}Key (de)serialization methods
5b44a75493 refactor: Remove unused CExt{Pub,}Key (de)serialization methods (Sebastian Falbesoner)

Pull request description:

  As pointed out in issue #17130, the serialization/deserialization methods for the classes `CExtKey` and
  `CExtPubKey` are only used in the BIP32 unit tests and hence can be removed (see comments https://github.com/bitcoin/bitcoin/issues/17130#issuecomment-543750290, https://github.com/bitcoin/bitcoin/issues/17130#issuecomment-543794408 and https://github.com/bitcoin/bitcoin/issues/17130#issuecomment-543814727).

ACKs for top commit:
  practicalswift:
    ACK 5b44a75493 -- -60 LOC diff looks correct :)
  promag:
    ACK 5b44a75493.
  MarcoFalke:
    unsigned ACK 5b44a75493
  fjahr:
    ACK 5b44a75
  jonatack:
    Light ACK 5b44a75493. Built, ran tests and bitcoind. `git blame` shows most of the last changes are from commit 90604f16af in 2015 to add bip32 pubkey serialization.

Tree-SHA512: 6887573b76b9e54e117a076557407b6f7908719b2202fb9eea498522baf9f30198b3f78b87a62efcd17ad1ab0886196f099239992ce7cbbaee79979ffe9e5f2c
2019-10-24 17:25:08 -04:00
fanquake
8c6081a884 compat: remove bswap_* check on macOS
This was originally added in #9366 to fix the gui build, as
Protobuf would also define these macros. Now that we're no-longer
using Protobuf, remove the additional check.
2019-10-24 16:01:44 -04:00
fanquake
2cba35ab38 build: skip building OpenSSL lib_ssl 2019-10-24 16:01:44 -04:00
fanquake
45a2d3c552 build: remove OpenSSL from Qt build
More info available from:
https://doc.qt.io/qt-5/ssl.html#enabling-and-disabling-ssl-support
2019-10-24 16:01:44 -04:00
fanquake
befbc40eb5 build: remove EVP_MD_CTX_new detection
This was added in #9475 to fix LibreSSL compatibility for
BIP70, so is no longer required.
2019-10-24 16:01:43 -04:00
fanquake
fcee10c2d0 build: remove SSL lib detection 2019-10-24 16:01:43 -04:00
fanquake
c7f30dbca8 gui: Update BIP70 support message 2019-10-24 16:01:43 -04:00
fanquake
a3e810326d build: remove BIP70 entries from macOS Info.plist 2019-10-24 16:01:43 -04:00
fanquake
72fe13a58d gui: remove payment request file handling from OpenURI dialog 2019-10-24 16:01:43 -04:00
fanquake
3548e4aac7 Remove BIP70 Support 2019-10-24 16:01:43 -04:00
fanquake
1cb9a4e28c docs: remove protobuf from docs 2019-10-24 16:01:42 -04:00
fanquake
67328bb7ca build: remove protobuf from depends 2019-10-24 16:01:42 -04:00
MarcoFalke
fa92813407 consensus: Explain why fCheckDuplicateInputs can not be skipped and remove it 2019-10-24 14:58:34 -04:00
Wladimir J. van der Laan
4af0447169 Merge #17240: ci: Disable functional tests on mac host
fa710066b9 ci: Disable functional tests on mac host (MarcoFalke)

Pull request description:

  Judging from the lack of responses to https://github.com/bitcoin/bitcoin/issues/15400#issuecomment-543745053, no one can reproduce the failures locally. Thus, disable the tests on the ci mac host. Otherwise they cause ci failures to be ignored or overwritten by a blind re-run.

ACKs for top commit:
  laanwj:
    ok, ACK fa710066b9 in that case
  ryanofsky:
    Code review ACK fa710066b9. I've been blindly restarting these but can confirm it seemed like random test failures.

Tree-SHA512: 805f552124409f6e79b16b419826d334307202b8e3636edc8a91dadffe9949554d924c00cb16bb09473ec1aaaf2d8eaf7a97c7c2b8c11bf6372135d39bba802b
2019-10-24 20:27:08 +02:00
John Newbery
0a433fc876 [validation] Remove unused cacheSigStore from CheckInputsFromMempoolAndCache
CheckInputsFromMempoolAndCache() is only called in one place, and
cacheSigStore is set to true in that call site. Remove the argument
entirely.

Also improve commenting.
2019-10-24 13:14:03 -04:00
MarcoFalke
fa710066b9 ci: Disable functional tests on mac host 2019-10-24 12:43:55 -04:00
fanquake
d7134ead84 Merge #17239: doc: Changed miniupnp links to https
6f6852335f doc: Changed miniupnp links to https (Marius Kjærstad)

Pull request description:

  doc: Changed miniupnp links to https

ACKs for top commit:
  fjahr:
    ACK 6f68523
  fanquake:
    ACK 6f6852335f - as mentioned in this [comment](https://github.com/bitcoin/bitcoin/pull/17166#issuecomment-545906845), in future, there's no need to open a new PR instead than squashing. One of the requirements of contributing to Bitcoin Core is some basic `git` usage, and we have some documentation on that in our [CONTRIBUTING.md](https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md#squashing-commits) (which also mentions not to open multiple PRs).

Tree-SHA512: 1e806326552b94dc1c2f105bed822a1dd56127d42bbb0c15a16623ff084b2047331f2f1ad5303c99b0d68c70ad2caf8e8028eb354ba4379973b9e1198676ce72
2019-10-24 11:36:25 -04:00
Marius Kjærstad
6f6852335f doc: Changed miniupnp links to https
doc: Changed miniupnp links to https
2019-10-24 14:57:36 +02:00
MarcoFalke
d53828cb79 Merge #17235: tests: Skip unnecessary fuzzer initialisation. Hold ECCVerifyHandle only when needed.
c2f964a674 tests: Remove Cygwin WinMain workaround (practicalswift)
db4bd32cc3 tests: Skip unnecessary fuzzer initialisation. Hold ECCVerifyHandle only when needed. (practicalswift)

Pull request description:

  Skip unnecessary fuzzer initialisation. Hold `ECCVerifyHandle` only when needed.

  As suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/17018#discussion_r336645391.

Top commit has no ACKs.

Tree-SHA512: 598da44859d736e3fdc143b93e07f444d8ad19dfdab0cfe7c6ccff8644e862664d869337dfe6b49416ed09a0024e4a5f2220ca6246de568f9e9227d721baa28e
2019-10-24 08:32:07 -04:00
Wladimir J. van der Laan
205cffaf38 Merge #17226: gui: Fix payAmount tooltip in SendCoinsEntry
0fc81a1e87 gui: Fix payAmount tooltip in SendCoinsEntry (João Barbosa)

Pull request description:

  Before the tooltip shows in wrong places:

  ![Screenshot 2019-10-23 at 11 33 49](https://user-images.githubusercontent.com/3534524/67384904-f6b6a380-f589-11e9-832c-ec1643014b96.png)
  ![Screenshot 2019-10-23 at 11 33 23](https://user-images.githubusercontent.com/3534524/67384905-f74f3a00-f589-11e9-9944-a52fee097e02.png)

  Now only shows in the amount field:

  ![Screenshot 2019-10-23 at 11 35 30](https://user-images.githubusercontent.com/3534524/67384919-ff0ede80-f589-11e9-8ce4-c122e11fe885.png)

ACKs for top commit:
  laanwj:
    ACK 0fc81a1e87

Tree-SHA512: 0857e568c21d380a68c81e9be3212b1745d7d3199a1d5fdef9afc8feed0272f215726fa98bbf8a3fb332389c5454f2316bc1581f1a2ccd76cef46a0e3ac6f99f
2019-10-24 13:41:25 +02:00
Wladimir J. van der Laan
b688b859db Merge #17004: validation: Remove REJECT code from CValidationState
9075d13153 [docs] Add release notes for removal of REJECT reasons (John Newbery)
04a2f326ec [validation] Fix REJECT message comments (John Newbery)
e9d5a59e34 [validation] Remove REJECT code from CValidationState (John Newbery)
0053e16714 [logging] Don't log REJECT code when transaction is rejected (John Newbery)
a1a07cfe99 [validation] Fix peer punishment for bad blocks (John Newbery)

Pull request description:

  We no longer send BIP 61 REJECT messages, so there's no need to set
  a REJECT code in the CValidationState object.

  Note that there is a minor bug fix in p2p behaviour here. Because the
  call to `MaybePunishNode()` in `PeerLogicValidation::BlockChecked()` only
  previously happened if the REJECT code was > 0 and < `REJECT_INTERNAL`,
  then there are cases were `MaybePunishNode()` can get called where it
  wasn't previously:

  - when `AcceptBlockHeader()` fails with `CACHED_INVALID`.
  - when `AcceptBlockHeader()` fails with `BLOCK_MISSING_PREV`.

  Note that `BlockChecked()` cannot fail with an 'internal' reject code. The
  only internal reject code was `REJECT_HIGHFEE`, which was only set in
  ATMP.

  This reverts a minor bug introduced in 5d08c9c579.

ACKs for top commit:
  ariard:
    ACK 9075d13, changes since last reviewed are splitting them in separate commits to ease understanding and fix nits
  fjahr:
    ACK 9075d13153, confirmed diff to last review was fixing nits in docs/comments.
  ryanofsky:
    Code review ACK 9075d13153. Only changes since last review are splitting the main commit and updating comments

Tree-SHA512: 58e8a1a4d4e6f156da5d29fb6ad6a62fc9c594bbfc6432b3252e962d0e9e10149bf3035185dc5320c46c09f3e49662bc2973ec759679c0f3412232087cb8a3a7
2019-10-24 10:49:45 +02:00
Wladimir J. van der Laan
8a191148db Merge #17154: wallet: Remove return value from CommitTransaction
9e95931865 [wallet] Remove `state` argument from CWallet::CommitTransaction (John Newbery)
d1734f9a3b [wallet] Remove return value from CommitTransaction() (John Newbery)
b6f486a02b [wallet] Add doxygen comment to CWallet::CommitTransaction() (John Newbery)
8bba91b22d [wallet] Fix whitespace in CWallet::CommitTransaction() (John Newbery)

Pull request description:

  `CommitTransaction()` returns a bool to indicate success, but since commit
  b3a7410 (#9302) it only returns true, even if the transaction was not
  successfully broadcast. This commit changes CommitTransaction() to return
  void.

  All dead code in `if (!CommitTransaction())` branches has been removed.

  Two additional commits fix up the idiosyncratic whitespace in `CommitTransaction` and add a doxygen comment for the function.

ACKs for top commit:
  laanwj:
    ACK 9e95931865

Tree-SHA512: a55a2c20369a45222fc0e02d0891495655a926e71c4f52cb72624768dd7b9c1dca716ea67d38420afb90f40c6e0fd448caa60c18fd693bb10ecb110b641820e6
2019-10-24 10:16:12 +02:00
practicalswift
c2f964a674 tests: Remove Cygwin WinMain workaround 2019-10-24 08:07:59 +00:00
practicalswift
db4bd32cc3 tests: Skip unnecessary fuzzer initialisation. Hold ECCVerifyHandle only when needed. 2019-10-24 08:06:38 +00:00
MarcoFalke
c5ac7af779 Merge #17206: test: Add testcase to simulate bitcoin schema in leveldb
4896bacc00 Add testcase to simulate bitcoin schema in leveldb (MapleLaker)

Pull request description:

  Resurrecting #14125 with updates based on comments of closed PR

ACKs for top commit:
  laanwj:
    ACK 4896bacc00
  dongcarl:
    ACK 4896bacc00

Tree-SHA512: 3290ea7e1e998901d5ee8921d1d76cec399cae30ac1911a45b86826afed47cee1acf92bd6438f1fa11ed785a3b17abdcb1c169bc0419945eda9fe4c089d0b6eb
2019-10-23 15:49:43 -04:00
Cory Fields
50037e97d1 depends: fix boost mac cross build with clang 9+
The ancient "darwin-4.9.1" profile has long been used to match against
clang, which prior to version 9, reported 4.9.1 as its version when
invoking "clang++ -dumpversion". Presumably this was a historical
compatibility quirk related to Apple's switch from gcc to clang.

This was "fixed" in clang 9.0, so that -dumpversion reports the real
version. Unfortunately that had the side-effect of breaking the
(brittle) boost compiler detection.

Move to the seemingly more-correct "clang-darwin" profile, which passes
the checks and builds correctly.

Also switch to using ar rather than libtool for archiving, as it's what
the clang-darwin profile expects to be using.

Note that because this is using a different profile, some of the final
command-line arguments end up changing. The changes look sane at a
glance.
2019-10-23 14:02:39 -04:00
marcaiaf
b05ec410f2 Add unit testing for the CompressScript functions 2019-10-23 13:50:18 -04:00
MarcoFalke
deb2327b43 Merge #17018: tests: Add descriptor Parse(...) fuzzing harness
b5ffa9f3db tests: Add Parse(...) (descriptor) fuzzing harness (practicalswift)
fdef8bbf2f tests: Allow for using non-default fuzzing initialization (practicalswift)

Pull request description:

  Add `Parse(...)` (descriptor) fuzzing harness.

  To test this PR:

  We can run `test_fuzzing_harnesses.sh` (#17000) during ten seconds to quickly verify that the newly added  fuzz harness seem to hit relevant code regions, that the fuzzing throughput seems reasonable, etc.

  `test_fuzzing_harnesses.sh descriptor 10` runs all fuzzers matching the regexp `descriptor` giving them ten seconds of runtime each.

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ contrib/devtools/test_fuzzing_harnesses.sh descriptor 10
  Testing fuzzer descriptor_parse during 10 second(s)
  A subset of reached functions:
          NEW_FUNC[0/17]: 0x55ec8a240c90 in tinyformat::detail::formatImpl(std::ostream&, char const*, tinyformat::detail::FormatArg const*, int) src/./tinyformat.h:791
          NEW_FUNC[4/17]: 0x55ec8a2435f0 in tinyformat::detail::printFormatStringLiteral(std::ostream&, char const*) src/./tinyformat.h:564
          NEW_FUNC[5/17]: 0x55ec8a2439d0 in tinyformat::detail::streamStateFromFormat(std::ostream&, bool&, int&, char const*, tinyformat::detail::FormatArg const*, int&, int) src/./tinyformat.h:601
          NEW_FUNC[6/17]: 0x55ec8a24a3d0 in tinyformat::detail::FormatArg::format(std::ostream&, char const*, char const*, int) const src/./tinyformat.h:513
          NEW_FUNC[12/17]: 0x55ec8a29cd70 in void tinyformat::detail::FormatArg::formatImpl<long>(std::ostream&, char const*, char const*, int, void const*) src/./tinyformat.h:530
          NEW_FUNC[13/17]: 0x55ec8a29cf50 in void tinyformat::formatValue<long>(std::ostream&, char const*, char const*, int, long const&) src/./tinyformat.h:317
          NEW_FUNC[14/17]: 0x55ec8a2ea450 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<long>(char const*, long const&) src/./tinyformat.h:976
          NEW_FUNC[15/17]: 0x55ec8a346ac0 in void tinyformat::format<long>(std::ostream&, char const*, long const&) src/./tinyformat.h:968
          NEW_FUNC[16/17]: 0x55ec8a346d80 in tinyformat::detail::FormatListN<1>::FormatListN<long>(long const&) src/./tinyformat.h:885
          NEW_FUNC[0/16]: 0x55ec8a210c90 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/./tinyformat.h:976
          NEW_FUNC[2/16]: 0x55ec8a25c3e0 in void tinyformat::format<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::ostream&, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/./tinyformat.h:968
          NEW_FUNC[3/16]: 0x55ec8a25c6a0 in tinyformat::detail::FormatListN<1>::FormatListN<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/./tinyformat.h:885
          NEW_FUNC[4/16]: 0x55ec8a25c980 in void tinyformat::detail::FormatArg::formatImpl<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::ostream&, char const*, char const*, int, void const*) src/./tinyformat.h:530
          NEW_FUNC[6/16]: 0x55ec8b29cc60 in (anonymous namespace)::ParseScript(Span<char const>&, (anonymous namespace)::ParseScriptContext, FlatSigningProvider&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&) src/script/descriptor.cpp:810
          NEW_FUNC[8/16]: 0x55ec8b2a4710 in (anonymous namespace)::Expr(Span<char const>&) src/script/descriptor.cpp:657
          NEW_FUNC[9/16]: 0x55ec8b2a4d40 in (anonymous namespace)::Func(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, Span<char const>&) src/script/descriptor.cpp:647
          NEW_FUNC[15/16]: 0x55ec8b2d7dd0 in Span<char const>::subspan(long) const src/./span.h:33
          NEW_FUNC[0/1]: 0x55ec8b2d7830 in Span<char const>::operator[](long) const src/./span.h:31
          NEW_FUNC[0/10]: 0x55ec8a2ea090 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<char const*>(char const*, char const* const&) src/./tinyformat.h:976
          NEW_FUNC[1/10]: 0x55ec8a345d40 in void tinyformat::format<char const*>(std::ostream&, char const*, char const* const&) src/./tinyformat.h:968
          NEW_FUNC[2/10]: 0x55ec8a346000 in tinyformat::detail::FormatListN<1>::FormatListN<char const*>(char const* const&) src/./tinyformat.h:885
          NEW_FUNC[3/10]: 0x55ec8a3462e0 in void tinyformat::detail::FormatArg::formatImpl<char const*>(std::ostream&, char const*, char const*, int, void const*) src/./tinyformat.h:530
          NEW_FUNC[4/10]: 0x55ec8a3464b0 in void tinyformat::formatValue<char const*>(std::ostream&, char const*, char const*, int, char const* const&) src/./tinyformat.h:317
          NEW_FUNC[8/10]: 0x55ec8b438ef0 in ParsePrechecks(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:267
          NEW_FUNC[9/10]: 0x55ec8b4398b0 in ParseUInt32(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned int*) src/util/strencodings.cpp:309
          NEW_FUNC[0/3]: 0x55ec8a2e9430 in std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > tinyformat::format<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/./tinyformat.h:976
          NEW_FUNC[1/3]: 0x55ec8a33a6f0 in void tinyformat::format<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::ostream&, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/./tinyformat.h:968
          NEW_FUNC[2/3]: 0x55ec8a33aa40 in tinyformat::detail::FormatListN<2>::FormatListN<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/./tinyformat.h:885
          NEW_FUNC[1/2]: 0x55ec8b4331b0 in IsHex(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/util/strencodings.cpp:61
          NEW_FUNC[13/24]: 0x55ec8b126eb0 in Params() src/chainparams.cpp:384
          NEW_FUNC[14/24]: 0x55ec8b19a500 in DecodeDestination(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) src/key_io.cpp:217
          NEW_FUNC[15/24]: 0x55ec8b19a610 in (anonymous namespace)::DecodeDestination(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, CChainParams const&) src/key_io.cpp:74
          NEW_FUNC[18/24]: 0x55ec8b357160 in IsValidDestination(boost::variant<CNoDestination, PKHash, ScriptHash, WitnessV0ScriptHash, WitnessV0KeyHash, WitnessUnknown> const&) src/script/standard.cpp:325
          NEW_FUNC[19/24]: 0x55ec8b36fe40 in DecodeBase58(char const*, std::vector<unsigned char, std::allocator<unsigned char> >&) src/base58.cpp:36
  stat::number_of_executed_units: 54900
  stat::average_exec_per_sec:     4990
  stat::new_units_added:          421
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              412
  Number of unique code paths taken during fuzzing round: 93

  Tested fuzz harnesses seem to work as expected.
  ```

Top commit has no ACKs.

Tree-SHA512: f18d0a6798c55d2c85ef9e604af2c1d626da2b81c01ea3f77c5cecd4ce35b197030778b3cfebab4869dab84a022325dba94fd83290026bfbc59814938e1daa02
2019-10-23 12:42:48 -04:00
MarcoFalke
8f14d2002b Merge #17183: refactor: test/bench: dedup Build{Crediting,Spending}Transaction()
a0fc076476 refactor: test/bench: dedup Build{Crediting,Spending}Transaction() (Sebastian Falbesoner)

Pull request description:

  prototypes used in `src/test/script_tests.cpp`:
  - `CMutableTransaction BuildCreditingTransaction(const CScript& scriptPubKey, int nValue = 0);`
  - `CMutableTransaction BuildSpendingTransaction(const CScript& scriptSig, const CScriptWitness& scriptWitness, const CTransaction& txCredit);`

  prototypes used in `bench/verify_script.cpp`:
  - `CMutableTransaction BuildCreditingTransaction(const CScript& scriptPubKey);`
  - `CMutableTransaction BuildSpendingTransaction(const CScript& scriptSig, const CMutableTransaction& txCredit);`

  The more generic versions from the script tests are moved into `setup_common.cpp` and the calls are adapted accordingly in the verify_script benchmark (passing the nValue of 1 explicitely for `BuildCreditingTransaction()`, passing empty scriptWitness explicitely and converting txCredit parameter to CTransaction in `BuildSpendingTransaction()`).

Top commit has no ACKs.

Tree-SHA512: 8444f8a18f15070eeec1e5dfd255b55a851dfc2e6647c12b1995a6f7abd7196e830db2181d0e860bcd4cf4c815967584a3756dd450346bca70649dd1d4493e04
2019-10-23 09:24:07 -04:00
practicalswift
b5ffa9f3db tests: Add Parse(...) (descriptor) fuzzing harness 2019-10-23 11:10:10 +00:00
practicalswift
fdef8bbf2f tests: Allow for using non-default fuzzing initialization 2019-10-23 11:10:10 +00:00
João Barbosa
0fc81a1e87 gui: Fix payAmount tooltip in SendCoinsEntry 2019-10-23 11:38:30 +01:00
Wladimir J. van der Laan
a884b32854 Merge #16911: wallet: Only check the hash of transactions loaded from disk
cd68594dcd Only check the hash of transactions loaded from disk (Andrew Chow)

Pull request description:

  It feels unnecessary to do a full `CheckTransaction` for every transaction saved in the wallet. It should not be possible for an invalid transaction to get into the wallet in the first place, and if there is any disk corruption, the hash check will catch it.

ACKs for top commit:
  MarcoFalke:
    ACK cd68594dcd
  laanwj:
    ACK cd68594dcd
  promag:
    ACK cd68594dcd, AFAICT the check is not needed, hash comparison gives data integrity.

Tree-SHA512: 5b2e719f76097cfbf125392db6cc6c764355c81f0b7a5b60aee4b06af1afcca80cfd38a3cf5307fd9e2c1afc405f8321929a4552943099a8161e6762965451fb
2019-10-23 12:05:37 +02:00
Wladimir J. van der Laan
ceda6e8b74 Merge #17180: gui: Improved tooltip for send amount field
088a730fe6 static tooltip (JeremyCrookshank)

Pull request description:

  I noticed that on Bitcoin sends the tooltip wasn't very clear for new users and I hope my PR is more concise. If it needs changing more will happily change too 👍
  ![IMG_20191017_192739](https://user-images.githubusercontent.com/46864828/67036925-75d45380-f114-11e9-88bf-bab58161f80a.jpg)

ACKs for top commit:
  laanwj:
    ACK 088a730fe6

Tree-SHA512: 2b1103ac934d8f68d22333af3c0f5d4228b665b1e507378d4ae5b83cc2b6d6aeb46a3d68298cca93feb839db5caa560322c8df5261dc2f7db5abeed9f0dd9c69
2019-10-23 12:01:00 +02:00
Wladimir J. van der Laan
4258fd7377 Merge #17091: tests: Add test for loadblock option and linearize scripts
89339d1460 tests: Add test for loadblock option (Fabian Jahr)

Pull request description:

  Fixes #17019

  Was initially part of #17044 but as the test got larger it made sense to split it into its own commit as suggested in #17019 .

  This is testing the `-loadblock` option by using the scripts in `contrib/linearize` to generate a `bootstrap.dat` file and starting a disconnected node with it. So it is also testing the linearize scripts which were untested before and needed to be made available for the CI environment, hence they are added to `DIST_CONTRIB` in `Makefile.am`.

ACKs for top commit:
  laanwj:
    ACK 89339d1460

Tree-SHA512: aede0cd6e8b21194973f3633bc07fa2672d66a6f85dfe6a57cee2bb269a65d19ea49d5f9ed7914a173b3847c76e70257aa865f44bde170c1999d9655b4862d1c
2019-10-23 11:21:46 +02:00
Pieter Wuille
ed2dc5e48a Add override/final modifiers to V1TransportDeserializer 2019-10-23 09:27:35 +02:00
Pieter Wuille
f342a5e61a Make resetting implicit in TransportDeserializer::Read() 2019-10-23 09:27:32 +02:00
Pieter Wuille
6a91499496 Remove oversized message detection from log and interface 2019-10-23 09:27:25 +02:00
Anthony Towns
168b781fe7 Continue relaying transactions after they expire from mapRelay 2019-10-23 11:36:24 +10:00
Sebastian Falbesoner
a0fc076476 refactor: test/bench: dedup Build{Crediting,Spending}Transaction()
prototypes used in src/test/script_tests.cpp:
- CMutableTransaction BuildCreditingTransaction(const CScript& scriptPubKey, int nValue = 0);
- CMutableTransaction BuildSpendingTransaction(const CScript& scriptSig, const CScriptWitness& scriptWitness, const CTransaction& txCredit);

prototypes used in bench/verify_script.cpp:
- CMutableTransaction BuildCreditingTransaction(const CScript& scriptPubKey);
- CMutableTransaction BuildSpendingTransaction(const CScript& scriptSig, const CMutableTransaction& txCredit);

The more generic versions from the script tests are moved into a new file pair
transaction_utils.cpp/h and the calls are adapted accordingly in the
verify_script benchmark (passing the nValue of 1 explicitely for
BuildCreditingTransaction(), passing empty scriptWitness explicitely and
converting txCredit parameter to CTransaction in BuildSpendingTransaction()).
2019-10-23 01:59:32 +02:00
Jeffrey Czyz
8734c856f8 Replace the LogPrint function with a macro
Calling LogPrint with a category that is not enabled results in
evaluating the remaining function arguments, which may be arbitrarily
complex (and possibly expensive) expressions. Defining LogPrint as a
macro prevents this unnecessary expression evaluation.

This is a partial revert of #14209. The decision to revert is discussed
in #16688, which adds verbose logging for validation event notification.
2019-10-22 13:44:51 -07:00
JeremyCrookshank
088a730fe6 static tooltip 2019-10-22 20:36:38 +01:00
MarcoFalke
9dd6bbba61 Merge #17209: tests: Remove no longer needed UBSan suppressions (issues fixed). Add documentation.
0616138a07 tests: Remove no longer needed UBSan suppressions (issues fixed). Add documentation. (practicalswift)

Pull request description:

  Remove no longer needed UBSan suppressions (issues fixed). Add documentation.

  This PR is the CI-only subset of #17208 (which touches code).

  From a fuzzing perspective it would be really nice to be able to run UBSan with as few suppressions as possible :)

Top commit has no ACKs.

Tree-SHA512: a926ab3e80e12a805af110fbff470cdc61ef4db536919a5b8896ea8b70f761114a52d9b1c0f48b11c1d48338351bf2e003e01ce60c613612f26ba298dcc29cd9
2019-10-22 13:19:21 -04:00
MarcoFalke
4aaeb04497 Merge #17205: ci: Enable address sanitizer (ASan) stack-use-after-return checking
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking (practicalswift)

Pull request description:

  Enable address sanitizer (ASan) stack-use-after-return checking (`detect_stack_use_after_return=1`).

  Example:

  ```
  #include <iostream>
  #include <string>

  const std::string& get_string(int i) {
      return std::to_string(i);
  }

  int main() {
      std::cout << get_string(41) << "\n";
  }
  ```

  Without address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ./stack-use-after-return

  $
  ```

  With address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ASAN_OPTIONS="detect_stack_use_after_return=1" ./stack-use-after-return
  =================================================================
  ==10400==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f7fa0400030 at pc 0x00000049d2cc bp 0x7ffcbd617070 sp 0x7ffcbd616820
  READ of size 2 at 0x7f7abbecd030 thread T0
      #0 0x439781 in fwrite
      #1 0x7f7ac0504cb3 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x113cb3)
      #2 0x4f9b5f in main stack-use-after-return.cpp:9:15
      #3 0x7f7abf440b96 in __libc_start_main
      #4 0x41bbc9 in _start
  …
  $
  ```

Top commit has no ACKs.

Tree-SHA512: 6557a9ff184023380fd9aa433cdf413e01a928ea99dbc59ec138e5d69cb9e13592e8bb5951612f231ff17a37a895bec5c0940c8db5f328a5c840a5771bdeeba5
2019-10-22 13:14:16 -04:00
Sebastian Falbesoner
8d8e5a79d0 test: use default address type (bech32) for wallet_bumpfee tests
The use of native segwit addresses (pure p2wpkh instead of p2sh-p2wpkh) leads
to smaller transaction sizes, needing adaption of some constants in the
following test cases:
    - test_dust_to_fee(): adaption of dust calculation
          (p2wpkh spend estimate of 67 is taken from src/policy/policy.cpp:GetDustThreshold())
    - test_maxtxfee_fails(): lowering -maxtxfee setting to trigger fail
2019-10-22 16:38:48 +02:00
Jonas Schnelli
b0e10ff4df Force CNetMessage::m_recv to use std::move 2019-10-22 15:28:32 +02:00
fanquake
feb5075777 build: use osslsigncode 2.0 in gitian
The original osslsigncode project (https://sourceforge.net/projects/osslsigncode/) has been marked as abandonware,
"This is now - and has been for a long while - abandonware. Feel free to create your own forks etc.".

However, a fork at https://github.com/mtrojnar/osslsigncode has emerged that has incorporated
theuni's patches, updated the tool to work with OpenSSL 1.1 and made other improvements.

This commit switches the windows signer descriptor to use this new version of osslsigncode.
2019-10-21 18:32:53 -04:00
Sebastian Falbesoner
5b44a75493 refactor: Remove unused CExt{Pub,}Key (de)serialization methods
The serialization/deserialization methods for the classes CExtKey and
CExtPubKey were only used in the BIP32 unit tests, where the relevant parts are
removed as well.
2019-10-22 00:07:31 +02:00
Jeremy Rubin
4671fc3d9e Expand on wallet_balance.py comment from https://github.com/bitcoin/bitcoin/pull/16766\#issuecomment-527563982 2019-10-21 13:43:44 -07:00
Jeremy Rubin
91f3073f08 Update release notes to mention changes to IsTrusted and impact on wallet 2019-10-21 13:22:56 -07:00
Jeremy Rubin
8f174ef112 Systematize style of IsTrusted single line if 2019-10-21 13:16:22 -07:00
Jeremy Rubin
b49dcbedf7 update variable naming conventions for IsTrusted 2019-10-21 13:16:22 -07:00
Jeremy Rubin
5ffe0d1449 Update comment in test/functional/wallet_balance.py
Co-Authored-By: MarcoFalke <falke.marco@gmail.com>
2019-10-21 13:16:22 -07:00
Jeremy Rubin
a550c58267 Update wallet_balance.py test to reflect new behavior 2019-10-21 13:16:22 -07:00
Jeremy Rubin
5dd7da4ccd Reuse trustedParents in looped calls to IsTrusted 2019-10-21 13:16:22 -07:00
Jeremy Rubin
595f09d6de Cache tx Trust per-call to avoid DoS 2019-10-21 13:16:22 -07:00
Jeremy Rubin
dce032ce29 Make IsTrusted scan parents recursively 2019-10-21 13:16:22 -07:00
practicalswift
0616138a07 tests: Remove no longer needed UBSan suppressions (issues fixed). Add documentation. 2019-10-21 18:24:27 +00:00
MapleLaker
4896bacc00 Add testcase to simulate bitcoin schema in leveldb 2019-10-21 11:15:57 -04:00
Wladimir J. van der Laan
a22b62481a Merge #17070: wallet: Avoid showing GUI popups on RPC errors
facec1c643 wallet: Avoid showing GUI popups on RPC errors (MarcoFalke)

Pull request description:

  RPC errors and warnings are shown as popups in the GUI instead of being returned to the RPC caller. For example,

  ```
  $ ./src/bitcoin-cli loadwallet $(pwd)/./test/functional/data/wallets/high_minversion/
  error code: -4
  error message:
  Wallet loading failed.
  ```

  gives me a GUI popup and no reason why loading the wallet failed.

  After this pull request:

  ```
  $ ./src/bitcoin-cli loadwallet $(pwd)/./test/functional/data/wallets/high_minversion/
  error code: -4
  error message:
  Wallet loading failed: Error loading /home/marco/workspace/btc_bitcoin_core/./test/functional/data/wallets/high_minversion/wallet.dat: Wallet requires newer version of Bitcoin Core

ACKs for top commit:
  laanwj:
    Code review ACK facec1c643

Tree-SHA512: c8274bbb02cfcf71676eeec1e773e51fb3538cf93f82e7cb8536f4716d44ed819cdc162dfc039ac7386a4db381a734cdb27fd32567043a1180c02519fbcba194
2019-10-21 13:48:27 +02:00
Wladimir J. van der Laan
a75cb122ed Merge #17195: gui: send amount placeholder value
57e2edea0b Send amount shows minimum amount placeholder (JeremyCrookshank)

Pull request description:

  Noticed that there wasn't a default value for the send amount. However if you put a value in or click the up and down arrows you're unable to get it blank again, so it makes sense that it has a default value. I hope this also makes it more clear that users can send less than 1 BTC if it shows the 8 decimal places

  PR:
  ![Capture](https://user-images.githubusercontent.com/46864828/67132088-549c6180-f1ff-11e9-9ba5-67fdcd6db894.PNG)

ACKs for top commit:
  promag:
    ACK 57e2edea0b.
  GChuf:
    ACK 57e2edea0b
  laanwj:
    ACK 57e2edea0b, this is a surprisingly compact solution too

Tree-SHA512: 354590d2a88231b8649f7ae985c8a7864d74ca0e1f8603cb1730ba46747084de90ee6285ce4d39ee04b054fb9cd2d78ebc71146f3af694c37a8a3aff7f051800
2019-10-21 12:29:52 +02:00
Wladimir J. van der Laan
fc1040acc0 Merge #17176: ci: Cleanup macOS runs
fa677d1801 ci: Remove redundant check for TRAVIS_OS_NAME (MarcoFalke)
fadccb263b doc: Document that GNU tools are required for linters (MarcoFalke)
4444704ca9 ci: Cleanup macOS runs (MarcoFalke)

Pull request description:

  * Remove a commented out cleanup task in `before_cache`
  * Remove the linter run on macOS, and document that GNU tools are required to run the linters

ACKs for top commit:
  Sjors:
    Code review ACK fa677d1801
  laanwj:
    ACK fa677d1801
  ryanofsky:
    Code review ACK fa677d1801 for new third commit replacing TRAVIS_OS_NAME check with NO_DEPENDS setting

Tree-SHA512: 9122a63bbe7887d9e379123152ea4ba44324cb18033b9e6b45bfdb1af665c10ea598564b9fcd57330d208a08e4696e41b4d6175f05f0843a3a76530da114f8c6
2019-10-21 12:20:47 +02:00
practicalswift
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking 2019-10-21 10:04:28 +00:00
Wladimir J. van der Laan
0d6b6b7c65 Merge #17191: random: remove call to RAND_screen() (Windows only)
e892f9648a random: remove call to RAND_screen() (Windows only) (fanquake)

Pull request description:

  Follow up to https://github.com/bitcoin/bitcoin/pull/17151 where there were multiple calls to also remove our call to RAND_screen().

ACKs for top commit:
  MarcoFalke:
    unsigned ACK e892f9648a
  laanwj:
    ACK e892f9648a

Tree-SHA512: 1b846016d91e8113f90466b61fcaf0574edb6b4726eba1947549e2ac28907e1318d893f7b303e756f19730c8507c79b10e08d54b97153224b585ff1e0ac1953e
2019-10-21 09:53:32 +02:00
fanquake
0d7e8d66c4 Merge #17203: wallet: Remove unused GetLabelName
7ca68e1461 wallet: Remove unused GetLabelName (Sebastian Falbesoner)

Pull request description:

  While taking a look at #17198 I noticed that the method `CWallet::GetLabelName(...)` is not used anymore, since the `account` API was removed (c9c32e6b84).

ACKs for top commit:
  practicalswift:
    ACK 7ca68e1461
  promag:
    ACK 7ca68e1461.
  fanquake:
    ACK 7ca68e1461

Tree-SHA512: 6825d77a85934e2368a3fb44c8db0ed0872aa5606e3761decb0a6b7e3773277afa7021bf1f71009207c3961cdd0a1c448854ea2fd8be95c3afec466254faf82d
2019-10-20 19:50:20 -04:00
Sebastian Falbesoner
7ca68e1461 wallet: Remove unused GetLabelName 2019-10-20 21:00:33 +02:00
fanquake
b8f041af2d Merge #17157: doc: Added instructions for how to add an upsteam to forked repo
f09ba060ca doc: Added instructions for how to add an upsteam to forked repo (dannmat)

Pull request description:

  As a first time git developer, I struggled to understand whether to create a new fork for each pull request or not.

  After asking the IRC chat, I have added this to the documentation to further help new developers using git.

ACKs for top commit:
  fanquake:
    ACK f09ba060ca - For such a simple change, I think we've bike-shed this enough already. The `bitcoin/bitcoin` repo isn't really where anyone should be learning how to use `git` etc, but I think linking out here is ok.

Tree-SHA512: e0e9d655d0725e0128673afedb81dc5ba9387968fcbb681de7e50155a2cfa1a7f39fad040b596f4de9ad6727a1a8a90fd3d36eaa5242bc12186c3b82abd23fb2
2019-10-20 12:04:09 -04:00
dannmat
f09ba060ca doc: Added instructions for how to add an upsteam to forked repo
As a first time git developer, I struggled to understand whether to create a new fork for each pull request or not.

After asking the IRC chat, I have added this to the documentation to further help new developers using git.

Co-Authored-By: Michael <fanquake@gmail.com>
2019-10-20 11:47:42 +01:00
JeremyCrookshank
57e2edea0b Send amount shows minimum amount placeholder 2019-10-19 13:46:59 +01:00
fanquake
b6e34afe97 Merge #17184: util: Filter out macOS process serial number
b5f0be3879 util: Filter out macOS process serial number (Hennadii Stepanov)

Pull request description:

  Fix #17179

ACKs for top commit:
  laanwj:
    ACK b5f0be3879
  MarcoFalke:
    unsigned ACK b5f0be3879
  promag:
    ACK b5f0be3879.
  fanquake:
    ACK b5f0be3879 - Tested that this fixes #17179.

Tree-SHA512: 84ce859e53ebc7ad2d0a45e954243ef6efee640f1e0212322f68a317e4361a216ecb4b5a3a410ab31613adc285c8d3840fbf41fa9da9019be3d734db6b9427cd
2019-10-18 17:38:38 -04:00
MarcoFalke
faeb666536 util: Add CHECK_NONFATAL and use it in src/rpc 2019-10-18 17:19:36 -04:00
fanquake
fd3b4e422d Merge #16949: build: only pass --disable-dependency-tracking to packages that understand it
1ba49bcdc2 build: pass --enable-option-checking to applicable packages (fanquake)
bcff8e21b1 build: only pass --disable-dependency-tracking to packages that understand it (fanquake)

Pull request description:

  By blanket passing `--disable-dependency-tracking` to all depends packages we end up with warnings (i.e in `bdb` or `freetype`) like:
  ```bash
  configure: WARNING: unrecognized options: --disable-dependency-tracking
  ```
  Instead, only pass it to packages that actually understand it. Related to https://github.com/bitcoin/bitcoin/issues/16354.

  More info on `--disable-dependency-tracking` available [here](https://www.gnu.org/software/automake/manual/html_node/Dependency-Tracking.html).

  This PR also adds `--enable-option-checking` as a configure option to all applicable packages.

ACKs for top commit:
  laanwj:
    ACK 1ba49bcdc2
  theuni:
    ACK 1ba49bcdc2

Tree-SHA512: 6d3143ad5f5d1abed5e0a0b2ffbb4323f21c7bf24b0b8df26fb1b3cd16cf5309bbb830aa5aaec99164d5bbe8e9c62b97aa3e97ee1ddc2c7612bf8ff88a63885e
2019-10-18 16:57:24 -04:00
Block Mechanic
f9af3ced1c Android: add all arch support
Add support for armv7a, i686 and x86_64 archs to android.mk
Add -fPIC to depends file as anddroid requires it see https://stackoverflow.com/questions/30498776/position-independent-executables-and-android
2019-10-18 22:24:21 +02:00
fanquake
e892f9648a random: remove call to RAND_screen() (Windows only)
Follow up to https://github.com/bitcoin/bitcoin/pull/17151 where
there were multiple calls to also remove our call to RAND_screen().
2019-10-18 14:24:21 -04:00
fanquake
ddc3cf26ff Merge #17151: gui: remove OpenSSL PRNG seeding (Windows, Qt only)
cc3b5289ef gui: remove OpenSSL PRNG seeding (Windows, Qt only) (fanquake)

Pull request description:

  This removes the code introduced in [#4399](https://github.com/bitcoin/bitcoin/pull/4399) that attempts to add additional entroy to the OpenSSL PRNG using `RAND_event()`. This is specific to bitcoin-qt running on Windows.

  ```
  RAND_event() collects the entropy from Windows events such as mouse movements and other user interaction.
  It should be called with the iMsg, wParam and lParam arguments of all messages sent to the window procedure.
  It will estimate the entropy contained in the event message (if any), and add it to the PRNG.
  The program can then process the messages as usual.
  ```

  Besides BIP70, this is the last place we are directly using OpenSSL in the GUI code. All other OpenSSL usage is in [random.cpp](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp).

  Note that we are still also still doing other Windows specific gathering using [RandAddSeedPerfmon](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L268) and [RAND_screen()](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L600) on top of the other generation we do.

  Also note that if RAND_event returns `0` here (PRNG has **NOT** been seeded with enough data), we're just logging a single message and continuing, which also seems less than ideal.

ACKs for top commit:
  laanwj:
    ACK cc3b5289ef
  MarcoFalke:
    unsigned ACK cc3b5289ef
  theuni:
    ACK cc3b5289ef.

Tree-SHA512: 0bb18779cf37f6670e3e5ac6a6a38e5f95199491b2684f9e56391c76f030fe1621d6df064239c2a398f228129fdf3f2220fc8cd15b2b92ecf2ea6d98a79b2175
2019-10-18 14:23:47 -04:00
MarcoFalke
0ff7cd7d0c Merge #16889: Add some general std::vector utility functions
7d8d3e6a2a Add tests for util/vector.h's Cat and Vector (Pieter Wuille)
e65e61c812 Add some general std::vector utility functions (Pieter Wuille)

Pull request description:

  This is another general improvement extracted from #16800 .

  Two functions are added are:

  * Vector(arg1,arg2,arg3,...) constructs a vector with the specified arguments as elements. The vector's type is derived from the arguments. If some of the arguments are rvalue references, they will be moved into place rather than copied (which can't be achieved using list initialization).
  * Cat(vector1,vector2) returns a concatenation of the two vectors, efficiently moving elements when relevant.

  Vector generalizes (and replaces) the `Singleton` function in src/descriptor.cpp, and `Cat` replaces the function in bech32.cpp

ACKs for top commit:
  laanwj:
    ACK 7d8d3e6a2a
  MarcoFalke:
    ACK 7d8d3e6a2a (enjoyed reading the tests, but did not compile)

Tree-SHA512: 92325f14e90d7e7d9d920421979aec22bb0d730e0291362b4326cccc76f9c2d865bec33a797c5c0201773468c3773cb50ce52c8eee4c1ec1a4d10db5cf2b9d2a
2019-10-18 09:57:00 -04:00
John Newbery
9e95931865 [wallet] Remove state argument from CWallet::CommitTransaction
The `state` return argument has not been set since commit 611291c198.
Remove it (and the one place that it's used in a calling function).
2019-10-18 09:43:01 -04:00
John Newbery
d1734f9a3b [wallet] Remove return value from CommitTransaction()
CommitTransaction returns a bool to indicate success, but since commit
b3a74100b8 it only returns true, even if the transaction was not
successfully broadcast. This commit changes CommitTransaction() to return
void.

All dead code in `if (!CommitTransaction())` branches has been removed.
2019-10-18 09:26:32 -04:00
John Newbery
b6f486a02b [wallet] Add doxygen comment to CWallet::CommitTransaction() 2019-10-18 09:26:32 -04:00
fanquake
f2a094884d Merge #17186: gui: Add placeholder text to the sign message field
7005d6ab8f gui: Add placeholder text to the sign message field (Danny-Scott)

Pull request description:

  When using the sign message functionality I noticed the "message" field had no label or placeholder text to highlight what it's for.

  I've added the placeholder text to match the tool tip to help it be more user friendly.

ACKs for top commit:
  hebasto:
    Re-ACK 7005d6ab8f
  fanquake:
    ACK 7005d6ab8f

Tree-SHA512: 17fe51c134f6373d8d5f9ca98b15bd936da4e61aa5258ceb5d318575d49b43cbfde6f4c3f720eb5928206902e6ba52811ba08737a03c95224e45dabc947d9d11
2019-10-18 08:14:40 -04:00
Danny-Scott
7005d6ab8f gui: Add placeholder text to the sign message field 2019-10-18 12:21:42 +01:00
Jonas Schnelli
b9b58f8f68 Merge #15084: gui: don't disable the sync overlay when wallet is disabled
b3b6b6f62f gui: don't disable the sync overlay when wallet is disabled (Ben Carman)

Pull request description:

  Continuation of #13848.

  When running with `-disablewallet` the sync modal is now available by clicking on the progress bar or `syncing` icon.

  [Current Image of what the window looks like](https://imgur.com/6LsoT2l)

  Fixes #13828.

ACKs for top commit:
  jonasschnelli:
    Tested ACK b3b6b6f62f

Tree-SHA512: 325bc22a0b692bfb8fcc9d84e02dfc506146028b97b3609e23c2c45288c79b8aead1ad2e9b8d692f5f6771b4d2aee63fbe71bfaeaf17d260865da32ab3631e07
2019-10-18 09:15:20 +02:00
Jonas Schnelli
efecb74677 Use adapter pattern for the network deserializer 2019-10-18 08:56:08 +02:00
Jonas Schnelli
1a5c656c31 Remove transport protocol knowhow from CNetMessage / net processing 2019-10-18 08:56:08 +02:00
Jonas Schnelli
6294ecdb8b Refactor: split network transport deserializing from message container 2019-10-18 08:56:06 +02:00
Hennadii Stepanov
b5f0be3879 util: Filter out macOS process serial number 2019-10-18 09:12:15 +03:00
fanquake
4daadce36c Merge #17162: chain: Remove CBlockIndex::SetNull helper
fa0467326f chain: Set all CBlockIndex members to null, remove SetNull helper (MarcoFalke)

Pull request description:

  The first commit removes the `SetNull` helper and inlines the member initialization (C++11). See https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#c-data-structures for rationale.

  <strike>The second commit adds the `cs_main` lock annotation to `RaiseValidity`. See also #17161.</strike>

ACKs for top commit:
  promag:
    Code review ACK fa0467326f.
  practicalswift:
    ACK fa0467326f -- diff still looks correct :)
  laanwj:
    ACK fa0467326f, this makes it easy to see that all fields are initialized.

Tree-SHA512: 1b2b9fb0951c03c75b9cce322b89d4ecc9a364ae78b94d91b0b4669437824394dfada820ab6f74dfac3193f602899abfdc244ae2d9351ad293f555488f03470e
2019-10-17 14:53:33 -04:00
MarcoFalke
88eff969c2 Merge #17177: doc: Describe log files + consistent paths in test READMEs
9576614d2d doc: Describe log files + consistent paths in test READMEs (Martin Erlandsson)

Pull request description:

  picks up #15830

  I saw this was almost ready to merge but the test logging part was not 100% correct. I reworked that part, the rest is the same.

ACKs for top commit:
  GChuf:
    ACK 9576614d2d

Tree-SHA512: 3de7f1b0a1b0419df6e7b55964d00e715b6cb7874b1849ad6f120597610d7df4182c4b61b9c9691ce04f4e392ed3caead4c623374be2066ac31319e702d45d09
2019-10-17 14:35:21 -04:00
Martin Erlandsson
9576614d2d doc: Describe log files + consistent paths in test READMEs 2019-10-17 17:53:45 +02:00
MarcoFalke
fa677d1801 ci: Remove redundant check for TRAVIS_OS_NAME
Can be reviewed with
git diff --ignore-all-space --function-context
2019-10-17 11:30:36 -04:00
MarcoFalke
fadccb263b doc: Document that GNU tools are required for linters 2019-10-17 10:52:11 -04:00
MarcoFalke
4444704ca9 ci: Cleanup macOS runs 2019-10-17 10:47:02 -04:00
MarcoFalke
ec3ed5a448 Merge #16597: Travis: run full test suite on native macOS
1f6c650c99 travis: run tests on macOS native (Sjors Provoost)

Pull request description:

  Adds an additional Travis machine to run the functional test suite on native macOS

  Homebrew is not particularly Travis compatible, but I found some useful hints here: https://discourse.brew.sh/t/best-practice-for-homebrew-on-travis-brew-update-is-5min-to-build-time/5215/11

ACKs for top commit:
  MarcoFalke:
    re-ACK 1f6c650c99

Tree-SHA512: 3f19a1695fac53d4d6c2033a9c20be69294e3a798c84fd9bf6ae2aa7a6d92aa1dad1f62f4ee1ada9413fe7d05ee974050fa030fd2c547f33e0d5c0a3e74f64db
2019-10-17 10:37:09 -04:00
Wladimir J. van der Laan
048e456fc4 Merge #17119: doc: Fix broken bitcoin-cli examples
85016e52f6 [rpc] Fix broken bitcoin-cli examples (Andrew Toth)

Pull request description:

  This fixes the `bitcoin-cli` examples for `combinerawtransaction`, `combinepsbt` and `testmempoolaccept`. They currently return `Error parsing JSON`.

ACKs for top commit:
  laanwj:
    ACK 85016e52f6

Tree-SHA512: b561f68f7a188dc91dab1ceb98da3ac3e232143ab2b906c90f95c6b74b584599d0f3b51f067cdd3b1153931f95b3dc385e453b1a0dde86f9cb549b94560f219d
2019-10-17 15:40:26 +02:00
MarcoFalke
fcf1ebde3d Merge #17169: doc: correct function name in ReportHardwareRand()
5013171eaf doc: correct function name in ReportHardwareRand() (fanquake)

Pull request description:

  The function is `InitHardwareRand` not `HWRandInit`.

  46d6930f8c/src/random.cpp (L99)

ACKs for top commit:
  laanwj:
    ACK 5013171eaf
  theStack:
    ACK 5013171eaf

Tree-SHA512: c25e1bb56e923961fc8a9178d751222b60f5ca36be84abf8fd1ac971f3a9b79b587ed9d8a4a175981b66f3fd5ad7edd6697d343e4dc4852351a1510718745455
2019-10-17 08:26:09 -04:00
MarcoFalke
4f42284fc0 Merge #17140: test: Fix bug in blockfilter_index_tests.
f59bbb61af test: Fix bug in blockfilter_index_tests. (Jim Posen)

Pull request description:

  The test case tests a chain reorganization, however the two chains were generated in the same manner and thus produced the same blocks.

  This issue was [pointed out](https://github.com/bitcoin/bitcoin/pull/14121#discussion_r334282663) by MarcoFalke.

ACKs for top commit:
  MarcoFalke:
    Thanks! ACK f59bbb61af (looked at the diff on GitHub, didn't compile, nor run tests)

Tree-SHA512: a2f063ae9312051ffc2a3fcc1116a6a8ac09beeef261bc40aa3ff7270ff4de22a790eb19fec6b15ba1eb46e78f1f317bfd91472d8581b95bb9441a56b102554e
2019-10-17 08:22:05 -04:00
fanquake
5013171eaf doc: correct function name in ReportHardwareRand() 2019-10-16 20:20:59 -04:00
MarcoFalke
46d6930f8c Merge #16659: refactoring: Remove unused includes
084e17cebd Remove unused includes (practicalswift)

Pull request description:

  As requested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/16273#issuecomment-521332089:

  This PR removes unused includes.

  Please note that in contrast to #16273 I'm limiting the scope to the trivial cases of pure removals (i.e. no includes added) to make reviewing easier.

  I'm seeking "Concept ACK":s for this obviously non-urgent minor cleanup.

  Rationale:
  * Avoids unnecessary re-compiles in case of header changes.
  * Makes reasoning about code dependencies easier.
  * Reduces compile-time memory usage.
  * Reduces compilation time.
  * Warm fuzzy feeling of being lean :-)

ACKs for top commit:
  ryanofsky:
    Code review ACK 084e17cebd. PR only removes include lines and it still compiles. In the worst case someone might have to explicitly add an include later for something now included implicitly. But maybe some effort was taken to avoid this, and it wouldn't be a tragedy anyway.

Tree-SHA512: 89de56edc6ceea4696e9579bccff10c80080821685b9fb4e8c5ef593b6e43cf662f358788701bb09f84867693f66b2e4db035b92b522a0a775f50b7ecffd6a6d
2019-10-16 17:33:57 -04:00
User
090b75c14b p2p: Avoid allocating memory for addrKnown where we don't need it 2019-10-16 17:06:20 -04:00
MarcoFalke
fa0467326f chain: Set all CBlockIndex members to null, remove SetNull helper 2019-10-16 13:06:50 -04:00
Pieter Wuille
7d8d3e6a2a Add tests for util/vector.h's Cat and Vector 2019-10-16 08:57:26 -07:00
Pieter Wuille
e65e61c812 Add some general std::vector utility functions
Added are:

* Vector(arg1,arg2,arg3,...) constructs a vector with the specified
  arguments as elements. The vector's type is derived from the
  arguments. If some of the arguments are rvalue references, they
  will be moved into place rather than copied (which can't be achieved
  using list initialization).

* Cat(vector1,vector2) returns a concatenation of the two vectors,
  efficiently moving elements when relevant.

Vector generalizes (and replaces) the Singleton function in
src/descriptor.cpp, and Cat replaces the Cat function in bech32.cpp
2019-10-16 08:56:57 -07:00
John Newbery
8bba91b22d [wallet] Fix whitespace in CWallet::CommitTransaction()
Reviewer hint: use --ignore-all-space git diff option for review.
2019-10-16 10:57:27 -04:00
Wladimir J. van der Laan
c34b88620d Merge #17095: util: Filter control characters out of log messages
d7820a1250 util: Filter control characters out of log messages (Wladimir J. van der Laan)

Pull request description:

  Belts and suspenders: make sure outgoing log messages don't contain potentially suspicious characters, such as terminal control codes.

  This escapes control characters except newline ('\n') in C syntax. It escapes instead of removes them to still allow for troubleshooting issues where they accidentally end up in strings (it is a debug log, after all).

  (more checks could be added such as UTF-8 validity and unicode code-point range checking—this is substantially more involved and would need to keep track of state between characters and even `LogPrint` calls as they could end up split up—but escape codes seem to be the most common attack vector for terminals.)

ACKs for top commit:
  practicalswift:
    ACK d7820a1250 - tested and works as expected :)

Tree-SHA512: 0806265addebdcec1062a6def3e903555e62ba5e93967ce9ee6943d16462a222b3f41135a5bff0a76966ae9e7ed75f211d7785bceda788ae0b0654bf3fd891bf
2019-10-16 16:04:27 +02:00
Wladimir J. van der Laan
5a3dd93594 Merge #17131: rpc: fix -rpcclienttimeout 0 option
b3b26e149c rpc: fix -rpcclienttimeout 0 option (Fabian Jahr)

Pull request description:

  fixes #17117

  I understood the bug as the help string being wrong, rather than that this feature is missing and should be added. Let me know if it should be the other way around.

  It is notable that if 0 is given as an argument, the fallback that is being used is the libevent default of 50 seconds, rather than `DEFAULT_HTTP_CLIENT_TIMEOUT` (900 seconds). This is not intuitive for the user. I could handle this in this PR but I am unsure which would be the better solution then: Actually adding the feature as described in the help string or falling back to `DEFAULT_HTTP_CLIENT_TIMEOUT`? Happy to hear opinions.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK b3b26e149c

Tree-SHA512: 65e526a652c0adcdb4f895e8d78d60c7caa5904c9915b165a3ae95725c87d13af1f916359f80302452a2fcac1a80f4c58cd805ec8c28720fa4b91b3c8baa4155
2019-10-16 16:00:49 +02:00
MarcoFalke
1f6638630e Merge #17113: tests: Add fuzzing harness for descriptor Span-parsing helpers
58d67f1cc0 tests: Add fuzzing harness for descriptor Span-parsing helpers (practicalswift)

Pull request description:

  Add fuzzing harness for descriptor Span-parsing helpers (`spanparsing`).

  As suggested by a fuzz testing enthusiast in https://github.com/bitcoin/bitcoin/pull/16887#issuecomment-540655816.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz \
        --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/spanparsing
  ```

ACKs for top commit:
  MarcoFalke:
    re-ACK 58d67f1cc0

Tree-SHA512: 5eaca9fcda2856e0dcfeb4a98a2dc97051ae6251f7642b92fdae3ff96bb95ccb0377ee4e6c6b531e59061983b8d9485a5282467f2ab1d614861f60202a893b1c
2019-10-16 08:48:10 -04:00
Wladimir J. van der Laan
4cfb6738e8 Merge #17118: build: depends macOS: point --sysroot to SDK
a0daea459c [build] depends macOS: point --sysroot to SDK (Sjors Provoost)

Pull request description:

  Fixes errors like `fatal error: 'unistd.h' file not found` when building depends on macOS.

  Replaces #14352 (which doesn't work on Catalina).

ACKs for top commit:
  jonasschnelli:
    utACK a0daea459c

Tree-SHA512: 995b1e1e84e635b32d1d4038bc63730c94a7c318b7240f6d62825977e5c97fe52c5aa5a0f39070beb0df8271dd294b36d6b5cf7f09ad07494fb15d5bd4d77f68
2019-10-16 09:48:40 +02:00
practicalswift
084e17cebd Remove unused includes 2019-10-15 22:56:43 +00:00
practicalswift
58d67f1cc0 tests: Add fuzzing harness for descriptor Span-parsing helpers 2019-10-15 22:47:08 +00:00
fanquake
eb292af309 Merge #17105: gui: Make RPCConsole::TabTypes an enum class
8019b6b150 gui: Make RPCConsole::TabTypes an enum class (João Barbosa)

Pull request description:

  This change makes the compiler emit a warning/error if a missing enum value is not handled. See also #17134.

ACKs for top commit:
  MarcoFalke:
    unsigned ACK 8019b6b150
  hebasto:
    re-ACK 8019b6b150
  fanquake:
    ACK 8019b6b150

Tree-SHA512: 329161097f4d079f48d5fb33bf3a07e314fbb2ac325cafb08bafa9e76229ecff0f9010fe3c1c15ccd02d4539b5c93839c846b42bfeaffa897a917cea599bf811
2019-10-15 15:53:22 -04:00
MarcoFalke
a3af5b5c13 Merge #17138: Remove wallet access to some node arguments
b96ed03962 [wallet] Remove pruning check for -rescan option (John Newbery)
eea462de9c [wallet] Remove package limit config access from wallet (John Newbery)

Pull request description:

  Removes wallet access to `-limitancestorcount`, `-limitdescendantcount` and `-prune`:

  - `-limitancestorcount` and `-limitdescendantcount` are now accessed with a method `getPackageLimits` in the `Chain` interface.
  - `-prune` is not required. It was only used in wallet component initiation to prevent running `-rescan` when pruning was enabled. This check is not required.

  Partially addresses #17137.

ACKs for top commit:
  MarcoFalke:
    Tested ACK b96ed03962
  ryanofsky:
    Code review ACK b96ed03962
  promag:
    Code review ACK b96ed03962.
  ariard:
    ACK b96ed03, check there isn't left anymore wallet access to node arguments.

Tree-SHA512: 90c8e3e083acbd37724f1bccf63dab642cf9ae95cc5e684872a67443ae048b4fdbf57b52ea47c5a1da6489fd277278fe2d9bbe95e17f3d4965a1a0fbdeb815bf
2019-10-15 14:59:43 -04:00
MarcoFalke
e180be49d7 Merge #17098: refactor: Feebumper EstimateFeeRate follow-up
66b29848c7 change wallet pointers to references in feebumper (Adam Jonas)
9be6666a4e typo and unneccessary parentheses (Adam Jonas)

Pull request description:

  Picking up some of the suggestions in the comments of #16727 including:
  https://github.com/bitcoin/bitcoin/pull/16727#discussion_r330547321
  https://github.com/bitcoin/bitcoin/pull/16727#discussion_r330549766
  https://github.com/bitcoin/bitcoin/pull/16727#discussion_r333209674

ACKs for top commit:
  promag:
    Code review ACK 66b29848c7.
  MarcoFalke:
    ACK 66b29848c7 (looked at the diff on GitHub)
  fjahr:
    ACK 66b2984 reviewed code

Tree-SHA512: d118f7689970fe39d9f5318dc818f13283cce9194370b3ce4758f298172e4681ae119ddc809f5c0b7602677137ac0d38147b915422ff616531a76a570b766fa2
2019-10-15 14:32:03 -04:00
fanquake
137b7a2af1 Merge #17125: gui: Add toolTip and placeholderText to sign message fields
610d9384de gui: Added label & tooltip for Verify Message labels (dannmat)

Pull request description:

  When using the Verify Message functionality, I found the input boxes to be rather confusing as they had no guidance for their purpose.

  I have added tooltips and labels to aid users when verifying messages in future

ACKs for top commit:
  promag:
    Code review ACK 610d9384de. Nit, commit and title are a little weird. Suggestion: "gui: Add toolTip and placeholderText to sign message fields"
  MarcoFalke:
    ACK 610d9384de (looks good, didn't compile or tested the changes)
  fanquake:
    ACK - 610d9384de

Tree-SHA512: d6a1bc872ad270dce440e96a163ce72cdd4708913d87a0fea749fc8cf2d8163b791cbb96a82030e0cb7d239920ceb0e3f05e0eec113f45a1a8e1309fbd92b4b0
2019-10-15 12:03:12 -04:00
Fabian Jahr
b3b26e149c rpc: fix -rpcclienttimeout 0 option 2019-10-15 18:01:59 +02:00
dannmat
610d9384de gui: Added label & tooltip for Verify Message labels 2019-10-15 15:59:23 +01:00
fanquake
cc3b5289ef gui: remove OpenSSL PRNG seeding (Windows, Qt only)
This removes the code introduced in [#4399](https://github.com/bitcoin/bitcoin/pull/4399)
that attempts to add additional entroy to the OpenSSL PRNG using Windows messages.
Note that this is specific to bitcoin-qt running on Windows.

```
RAND_event() collects the entropy from Windows events such as mouse movements and other user interaction.
It should be called with the iMsg, wParam and lParam arguments of all messages sent to the window procedure.
It will estimate the entropy contained in the event message (if any), and add it to the PRNG.
The program can then process the messages as usual.
```

Besides BIP70, this is the last place we are directly using OpenSSL in the
GUI code. All other OpenSSL usage is in random.cpp.

Note that we are still also doing Windows specific entropy gathering in multiple
other places. Such as [RandAddSeedPerfmon](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L268)
and [RAND_screen()](https://github.com/bitcoin/bitcoin/blob/master/src/random.cpp#L600).

Also note that if RAND_event returns 0 (PRNG has NOT been seeded with enough data), we're
just logging a message and continuing on, which seems less than ideal.
2019-10-15 09:48:50 -04:00
fanquake
029c65e04c Merge #17141: Test: Rename SegwitVersion1SignatureHash()
eebcdfa86a [test] rename SegwitVersion1SignatureHash() (John Newbery)

Pull request description:

  The function implementing segwit v0 signature hash was originally named
  SegwitVersion1SignatureHash() (presumably before segwit v0 was named
  segwit v0). Rename it to SegwitV0SignatureHash().

  Also rename SignatureHash() to LegacySignatureHash() for disambiguation.

ACKs for top commit:
  laanwj:
    ACK eebcdfa86a
  elichai:
    ACK eebcdfa86a (Checked to see you didn't miss any renaming)
  theStack:
    ACK eebcdfa86a

Tree-SHA512: ae504ac33dc4fca38079a113beb5ebcaf509162aef121edec5368a460a24c2ac040ef84f0be1dfc6186c32d94d68f8129db049907f1d6449f92eea9d39a40dbd
2019-10-15 09:43:15 -04:00
fanquake
c787556403 Merge #17142: docs: Update macdeploy README to include all files produced by make deploy
4441e58497 Update macdeploy README to include correctly named `.dmg` file produced from `make deploy` (Zakk)

Pull request description:

  Fixes issue #16909 to update the `contrib/macdeploy/README.md` to match the files produced from `make deploy`

  The files produced from `make deploy` are as follows:

  - `Bitcoin-QT.dmg`
  - `Bitcoin Core.app`
  - `dist/Bitcoin Core.app`

ACKs for top commit:
  jonasschnelli:
    ACK 4441e58497
  fanquake:
    ACK 4441e58497 - checked that `Bitcoin-Qt.dmg` is produced.

Tree-SHA512: 99bfadab59c7c516005b051e4a369f330178313a284bb665c22c40f70a6159f175909c08db1b32976ad7b130b53b414f8ba96f8ff7cbd164f2724c0cc151704a
2019-10-15 09:25:54 -04:00
fanquake
e15ffec718 Merge #17146: github: Add warning for bug reports
5f40d2770a github: Add warning for bug reports (Wladimir J. van der Laan)

Pull request description:

  I've noticed the "Bug" label being added redundantly fairly frequently. I think this might be due to github's templates.

  All in all, the link in https://github.com/bitcoin/bitcoin/issues/new/choose to open a regular issue is a bit hidden from sight. Direct people's attention to it.

ACKs for top commit:
  practicalswift:
    ACK 5f40d2770a - currently it is very easy to miss the tiny "Open a regular issue" link :)
  jonasschnelli:
    ACK 5f40d2770a
  hebasto:
    ACK 5f40d2770a

Tree-SHA512: e6c94c02f9f7d00621b580d406d03f8754173150bf456409ccc474b76fb93ff857ff4a0c652bf5c03d4f1b97ecf29ae0ff7bf8b763207f9c8522b8dcecc20109
2019-10-15 09:20:07 -04:00
João Barbosa
8019b6b150 gui: Make RPCConsole::TabTypes an enum class 2019-10-15 13:46:34 +01:00
Wladimir J. van der Laan
ee47461ea5 Merge #17033: Disable _FORTIFY_SOURCE when enable-debug
44f7a8d7a7 Disable _FORTIFY_SOURCE when enable-debug (Andrew Chow)

Pull request description:

  The `_FORTIFY_SOURCE` macro is enabled by default when hardening is enabled, but it requires optimization in order to be used. Since we disable all optimization with `--enable-debug`, this macro doesn't actually do anything and instead just causes a lot of warnings to be printed. This PR explicitly disables `_FORTIFY_SOURCE` so that these useless warnings aren't printed.

ACKs for top commit:
  laanwj:
    Thanks. ACK 44f7a8d7a7

Tree-SHA512: e9302aef794dfd9ca9d0d032179ecc51d3212a9a0204454419f410011343b27c32e6be05f385051b5b594c607b91b8e0e588f644584d6684429a649a413077d9
2019-10-15 14:00:32 +02:00
Zakk
4441e58497 Update macdeploy README to include correctly named .dmg file produced from make deploy 2019-10-15 11:30:13 +01:00
Wladimir J. van der Laan
b544eb6370 Merge #17086: tests: Fix fs_tests for unknown locales
d48f664440 tests: Fix fs_tests for unknown locales (Daki Carnhof)

Pull request description:

  Fix by removing "L" as suggested by meeDamian in
  https://github.com/bitcoin/bitcoin/issues/14948#issuecomment-522355441

  ```
  # all in .../bitcoin/src/test
  $ uname -m
  x86_64
  $ export LC_ALL=randomnonexistentlocale
  $ ./test_bitcoin
  Running 369 test cases...
  unknown location(0): fatal error: in "fs_tests/fsbridge_fstream": boost::system::system_error: boost::filesystem::path codecvt to string: error
  test/fs_tests.cpp(13): last checkpoint: "fsbridge_fstream" test entry

  *** 1 failure is detected in the test module "Bitcoin Core Test Suite"
  ```

  After the patch is applied, the same test under the same conditions runs fine.

  ```
  $ export LC_ALL=randomnonexistentlocale
  $ ./test_bitcoin
  Running 369 test cases...

  *** No errors detected
  ```

  Co-Authored-By: bugs@meedamian.com

ACKs for top commit:
  laanwj:
    ACK d48f664440

Tree-SHA512: a9910252b8ce6a05cab5530874549c2999ca2c28e835fc18aa8e5468fb417bd7d245864ec71d9233dd53e02940a9f0691b247430257f27eb0d7c20745d1c846d
2019-10-15 11:34:11 +02:00
Wladimir J. van der Laan
a4a4964db1 Merge #17111: doc: update bips.md with buried BIP9 deployments
fa6ed82794 doc: update bips.md with buried BIP9 deployments (MarcoFalke)

Pull request description:

  Also, remove the activation heights, as they can be retrieved from `./src/chainparams.cpp` (if needed)

ACKs for top commit:
  laanwj:
    ACK fa6ed82794, needs backport to 0.19 I guess.

Tree-SHA512: 9c069cc14589a3e2309d76f042677c024a9e14d16dbfccef54c4a2963ca7853d01f042b0237e346538c557591b7553deed9dd811ba64bbd0ced88883d562c59a
2019-10-15 11:30:56 +02:00
Daki Carnhof
d48f664440 tests: Fix fs_tests for unknown locales
Fix by removing "L" as suggested by meeDamian in
https://github.com/bitcoin/bitcoin/issues/14948#issuecomment-522355441

Co-Authored-By: bugs@meedamian.com
2019-10-15 09:21:41 +00:00
Wladimir J. van der Laan
d7820a1250 util: Filter control characters out of log messages
Belts and suspenders: make sure outgoing log messages don't contain
potentially suspicious characters, such as terminal control codes.

This escapes control characters except newline ('\n') in C syntax.
It escapes instead of removes them to still allow for troubleshooting
issues where they accidentally end up in strings.
2019-10-15 10:53:17 +02:00
Wladimir J. van der Laan
5f40d2770a github: Add warning for bug reports
I've noticed the "Bug" label being added redundantly fairly frequently.
I think this might be due to github's templates.

All in all, the link in https://github.com/bitcoin/bitcoin/issues/new/choose
to open a regular issue is a bit hidden from sight. Direct people's
attention to it.
2019-10-15 08:53:42 +02:00
fanquake
dcc640811c Merge #17134: doc: Add switch on enum example to developer notes
c8961c7d9f doc: Add switch on enum example (Hennadii Stepanov)
11e3d5eb1d util: Add AllowShortCaseLabelsOnASingleLine option (Hennadii Stepanov)

Pull request description:

  This PR documents a recurring issue:
  - #15938
  - #17105

ACKs for top commit:
  laanwj:
    Seems like good advice to me. ACK c8961c7d9f
  practicalswift:
    ACK c8961c7d9f
  promag:
    ACK c8961c7d9f, no excuse now, thanks!

Tree-SHA512: 530da5117094ed1bfaa6e447089521bd2c86b0742758dbacec4e4f934dc07b0e24f15a1448c4d58e49905e8fd3797d87bcae5669a346d33ed4c2878a04891699
2019-10-14 18:33:26 -04:00
John Newbery
eebcdfa86a [test] rename SegwitVersion1SignatureHash()
The function implementing segwit v0 signature hash was originally named
SegwitVersion1SignatureHash() (presumably before segwit v0 was named
segwit v0). Rename it to SegwitV0SignatureHash().

Also rename SignatureHash() to LegacySignatureHash() for disambiguation.
2019-10-14 17:13:05 -04:00
Sjors Provoost
1f6c650c99 travis: run tests on macOS native
Review hint:
git show -w

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: keneanung <keneanung@googlemail.com>
Co-authored-by: Vadim Peretokin <vperetokin@gmail.com>
2019-10-14 23:00:49 +02:00
Jim Posen
f59bbb61af test: Fix bug in blockfilter_index_tests.
The test case tests a chain reorganization, however the two
chains were generated in the same manner and thus produced
the same blocks.
2019-10-14 22:04:19 +02:00
John Newbery
b96ed03962 [wallet] Remove pruning check for -rescan option
Prior to this PR, the wallet would not allow the `-rescan` option at
startup if pruning was enabled. This is unnecessarily restrictive. It
should be possible to rescan if pruning is enabled, as long as no blocks
have actually been pruned yet.

Remove the pruning check from WalletInit::ParameterInteraction(). If any
blocks have been pruned, that will be caught in CreateWalletFromFile().
2019-10-14 13:39:25 -04:00
John Newbery
eea462de9c [wallet] Remove package limit config access from wallet
The wallet should not be able to directly access global configuration
from the node. Remove access of "-limitancestorcount" and
"-limitdescendantcount".
2019-10-14 13:32:41 -04:00
Andrew Toth
85016e52f6 [rpc] Fix broken bitcoin-cli examples 2019-10-14 13:09:52 -04:00
MarcoFalke
b33c03b0cb Merge #17124: test: speed up wallet_address_types by whitelisting peers (immediate tx relay)
fba4baa4fa test: speed up wallet_address_types by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  approaches another part of #16613 ("Functional test suite bottlenecks")

  As for `wallet_backup.py` (Commit 581c9be0d8), the
  bottleneck is in relaying transactions. By whitelisting the peers, the
  inventory is transmissioned immediately rather than on average every 5 seconds,
  speeding up the test significantly:

  before:
  ```
  $ time ./wallet_address_types.py
  real    1m30.072s
  user    0m6.478s
  sys     0m2.298s
  ```

  with this PR:
  ```
  $ time ./wallet_address_types.py
  real    0m26.785s
  user    0m5.525s
  sys     0m1.888s
  ```

ACKs for top commit:
  fanquake:
    ACK - fba4baa4fa

Tree-SHA512: 6728ae44bd8839426fa943d06af884e40c2d88de5d7807269a1e78ff987077160aa7e8d395f4468e6ca1d6f2110c7a03cd346a3339b256702f4cdabd285f7f86
2019-10-14 10:23:20 -04:00
MarcoFalke
6c7da0736d Merge #17108: test: fix "tx-size-small" errors after default address change
32d665c265 test: fix "tx-size-small" errors after default address change (Sebastian Falbesoner)

Pull request description:

  Addresses #17043, affects RBF and BIP68 functional tests.

  The "tx-size-small" policy rule rejects transactions with a non-witness size of
  smaller than 82 bytes (see `src/validation.cpp:MemPoolAccept::PreChecks(...)`),
  which corresponds to a transaction with 1 segwit input and 1 P2WPKH output.

  Through the default address change, the created test transactions have segwit
  inputs now and sending to short scriptPubKeys might violate this rule. By
  bumping the dummy scriptPubKey size to 22 bytes (= the size of a P2WPKH
  scriptPubKey), on all occurences the problem is solved.

  The dummy scriptPubKey has the format:
      ```21 <21-byte-long string of 'a' or 1s>```

ACKs for top commit:
  instagibbs:
    reACK 32d665c265 just s/Bytes/bytes/
  MarcoFalke:
    ACK 32d665c265

Tree-SHA512: 80e0386ff3c3f462901ba5c1e5ef2cbf095d9c0a40c8c3cfeacd4a3ab676afe744aa95b9eed77b4b3eec88bed930b33aa718117ed0977f6374e858a2f3bd5c57
2019-10-14 10:14:46 -04:00
MarcoFalke
556820ee57 Merge #17009: tests: Add EvalScript(...) fuzzing harness
7e50abcc29 tests: Add EvalScript(...) fuzzing harness (practicalswift)
bebb637472 tests: Add FuzzedDataProvider fuzzing helper from the Chromium project (practicalswift)

Pull request description:

  Add `EvalScript(...)` fuzzing harness.

  To test this PR:

  We can run `contrib/devtools/test_fuzzing_harnesses.sh` (#17000) during five seconds to quickly verify that the newly added  fuzz harness seem to hit relevant code regions, that the fuzzing throughput seems reasonable, etc.

  `test_fuzzing_harnesses.sh eval 5` runs all fuzzers matching the regexp `eval` giving them five seconds of runtime each.

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ contrib/devtools/test_fuzzing_harnesses.sh eval 5
  Testing fuzzer eval_script during 5 second(s)
  A subset of reached functions:
          NEW_FUNC[1/24]: 0x557b808742e0 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) src/./prevector.h:161
          NEW_FUNC[2/24]: 0x557b80875460 in prevector<28u, unsigned char, unsigned int, int>::indirect_ptr(int) const src/./prevector.h:162
          NEW_FUNC[6/9]: 0x557b81acdaa0 in popstack(std::vector<std::vector<unsigned char, std::allocator<unsigned char> >, std::allocator<std::vector<unsigned char, std::allocator<unsigned char> > > >&) src/script/interpreter.cpp:57
          NEW_FUNC[5/16]: 0x557b809f1bf0 in CScriptNum::serialize(long const&) src/./script/script.h:326
          NEW_FUNC[4/6]: 0x557b817c93d0 in CScriptNum::CScriptNum(std::vector<unsigned char, std::allocator<unsigned char> > const&, bool, unsigned long) src/./script/script.h:225
          NEW_FUNC[5/6]: 0x557b817cbb80 in CScriptNum::set_vch(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/./script/script.h:360
          NEW_FUNC[0/11]: 0x557b80a88170 in CHash256::Write(unsigned char const*, unsigned long) src/./hash.h:34
          NEW_FUNC[1/11]: 0x557b80a88270 in CHash256::Finalize(unsigned char*) src/./hash.h:28
          NEW_FUNC[5/11]: 0x557b81affdb0 in CSHA256::CSHA256() src/crypto/sha256.cpp:644
          NEW_FUNC[6/11]: 0x557b81affe80 in (anonymous namespace)::sha256::Initialize(unsigned int*) src/crypto/sha256.cpp:66
          NEW_FUNC[7/11]: 0x557b81b00460 in CSHA256::Write(unsigned char const*, unsigned long) src/crypto/sha256.cpp:649
          NEW_FUNC[8/11]: 0x557b81b009a0 in CSHA256::Finalize(unsigned char*) src/crypto/sha256.cpp:675
          NEW_FUNC[9/11]: 0x557b81b015e0 in CSHA256::Reset() src/crypto/sha256.cpp:692
          NEW_FUNC[10/11]: 0x557b81b01d90 in (anonymous namespace)::sha256::Transform(unsigned int*, unsigned char const*, unsigned long) src/crypto/sha256.cpp:79
          NEW_FUNC[0/1]: 0x557b808cc180 in BaseSignatureChecker::CheckLockTime(CScriptNum const&) const src/./script/interpreter.h:153
          NEW_FUNC[0/2]: 0x557b81ab5640 in CastToBool(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/script/interpreter.cpp:36
          NEW_FUNC[0/1]: 0x557b817c9c30 in CScriptNum::getint() const src/./script/script.h:312
          NEW_FUNC[0/1]: 0x557b81ae1df0 in CScriptNum::operator-=(long const&) src/./script/script.h:298
          NEW_FUNC[0/5]: 0x557b81af5670 in CRIPEMD160::CRIPEMD160() src/crypto/ripemd160.cpp:243
          NEW_FUNC[1/5]: 0x557b81af5740 in (anonymous namespace)::ripemd160::Initialize(unsigned int*) src/crypto/ripemd160.cpp:25
          NEW_FUNC[2/5]: 0x557b81af5b00 in CRIPEMD160::Write(unsigned char const*, unsigned long) src/crypto/ripemd160.cpp:248
          NEW_FUNC[3/5]: 0x557b81af5fa0 in (anonymous namespace)::ripemd160::Transform(unsigned int*, unsigned char const*) src/crypto/ripemd160.cpp:55
          NEW_FUNC[4/5]: 0x557b81af8d60 in CRIPEMD160::Finalize(unsigned char*) src/crypto/ripemd160.cpp:274
          NEW_FUNC[0/16]: 0x557b80857a30 in CScript::operator<<(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/./script/script.h:462
          NEW_FUNC[1/16]: 0x557b80872670 in prevector<28u, unsigned char, unsigned int, int>::insert(prevector<28u, unsigned char, unsigned int, int>::iterator, unsigned char const&) src/./prevector.h:342
          NEW_FUNC[2/16]: 0x557b80872e00 in void prevector<28u, unsigned char, unsigned int, int>::insert<__gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > > >(prevector<28u, unsigned char, unsigned int, int>::iterator, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >, __gnu_cxx::__normal_iterator<unsigned char const*, std::vector<unsigned char, std::allocator<unsigned char> > >) src/./prevector.h:368
          NEW_FUNC[3/16]: 0x557b80873630 in prevector<28u, unsigned char, unsigned int, int>::capacity() const src/./prevector.h:295
          NEW_FUNC[4/16]: 0x557b80874ed0 in void prevector<28u, unsigned char, unsigned int, int>::fill<prevector<28u, unsigned char, unsigned int, int>::const_iterator>(unsigned char*, prevector<28u, unsigned char, unsigned int, int>::const_iterator, prevector<28u, unsigned char, unsigned int, int>::const_iterator) src/./prevector.h:204
          NEW_FUNC[5/16]: 0x557b808cc0f0 in BaseSignatureChecker::CheckSig(std::vector<unsigned char, std::allocator<unsigned char> > const&, std::vector<unsigned char, std::allocator<unsigned char> > const&, CScript const&, SigVersion) const src/./script/interpreter.h:148
          NEW_FUNC[6/16]: 0x557b809edb10 in CScript::operator=(CScript&&) src/./script/script.h:390
          NEW_FUNC[7/16]: 0x557b809f8ec0 in void prevector<28u, unsigned char, unsigned int, int>::insert<prevector<28u, unsigned char, unsigned int, int>::const_iterator>(prevector<28u, unsigned char, unsigned int, int>::iterator, prevector<28u, unsigned char, unsigned int, int>::const_iterator, prevector<28u, unsigned char, unsigned int, int>::const_iterator) src/./prevector.h:368
          NEW_FUNC[8/16]: 0x557b809f9260 in prevector<28u, unsigned char, unsigned int, int>::swap(prevector<28u, unsigned char, unsigned int, int>&) src/./prevector.h:451
          NEW_FUNC[9/16]: 0x557b81ab58c0 in CheckSignatureEncoding(std::vector<unsigned char, std::allocator<unsigned char> > const&, unsigned int, ScriptError_t*) src/script/interpreter.cpp:200
          NEW_FUNC[10/16]: 0x557b81ab6f30 in FindAndDelete(CScript&, CScript const&) src/script/interpreter.cpp:254
          NEW_FUNC[11/16]: 0x557b81acdc20 in CheckPubKeyEncoding(std::vector<unsigned char, std::allocator<unsigned char> > const&, unsigned int, SigVersion const&, ScriptError_t*) src/script/interpreter.cpp:217
          NEW_FUNC[12/16]: 0x557b81ad3890 in IsCompressedOrUncompressedPubKey(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/script/interpreter.cpp:63
          NEW_FUNC[13/16]: 0x557b81ad8830 in CScript::GetOp(prevector<28u, unsigned char, unsigned int, int>::const_iterator&, opcodetype&) const src/./script/script.h:505
          NEW_FUNC[14/16]: 0x557b81ae21a0 in prevector<28u, unsigned char, unsigned int, int>::prevector<prevector<28u, unsigned char, unsigned int, int>::const_iterator>(prevector<28u, unsigned char, unsigned int, int>::const_iterator, prevector<28u, unsigned char, unsigned int, int>::const_iterator) src/./prevector.h:246
          NEW_FUNC[0/1]: 0x557b81ae1a40 in CScriptNum::operator+=(long const&) src/./script/script.h:290
          NEW_FUNC[0/5]: 0x557b81af9760 in CSHA1::CSHA1() src/crypto/sha1.cpp:150
          NEW_FUNC[1/5]: 0x557b81af9830 in (anonymous namespace)::sha1::Initialize(unsigned int*) src/crypto/sha1.cpp:32
          NEW_FUNC[2/5]: 0x557b81af9bf0 in CSHA1::Write(unsigned char const*, unsigned long) src/crypto/sha1.cpp:155
          NEW_FUNC[3/5]: 0x557b81afa090 in (anonymous namespace)::sha1::Transform(unsigned int*, unsigned char const*) src/crypto/sha1.cpp:47
          NEW_FUNC[4/5]: 0x557b81afc5e0 in CSHA1::Finalize(unsigned char*) src/crypto/sha1.cpp:181
          NEW_FUNC[0/1]: 0x557b81ada4f0 in CScriptNum::operator-() const src/./script/script.h:278
          NEW_FUNC[0/1]: 0x557b808cc210 in BaseSignatureChecker::CheckSequence(CScriptNum const&) const src/./script/interpreter.h:158
          NEW_FUNC[0/1]: 0x557b81ab5c00 in IsValidSignatureEncoding(std::vector<unsigned char, std::allocator<unsigned char> > const&) src/script/interpreter.cpp:107
  stat::number_of_executed_units: 9728
  stat::average_exec_per_sec:     1621
  stat::new_units_added:          844
  stat::slowest_unit_time_sec:    0
  stat::peak_rss_mb:              326
  Number of unique code paths taken during fuzzing round: 583

  Tested fuzz harnesses seem to work as expected.
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 7e50abcc29

Tree-SHA512: 4874ab28efb4219c24a4cfc9be901a3297d1973f43acadec415c2e1d6843e4e661f90e8f9695849373775a4556884cdcc8862a092246ae0383b844c37c1627d5
2019-10-14 09:28:03 -04:00
Sebastian Falbesoner
32d665c265 test: fix "tx-size-small" errors after default address change
Addresses #17043, affects RBF and BIP68 functional tests.

The "tx-size-small" policy rule rejects transactions with a non-witness size of
smaller than 82 bytes (see src/validation.cpp:MemPoolAccept::PreChecks(...)),
which corresponds to a transaction with 1 segwit input and 1 P2WPKH output.

Through the default address change, the created test transactions have segwit
inputs now and sending to short scriptPubKeys might violate this rule. By
bumping the dummy scriptPubKey size to 22 bytes (= the size of a P2WPKH
scriptPubKey), on all occurences the problem is solved.

The dummy scriptPubKey has the format:
    21 <21-byte-long string of 'a' or 1s>

former commit messages, now squashed:
test: rbf, bip68: use constant DUMMY_P2WPKH_SCRIPT for bumped scriptPubKey
test: rbf, bip68: use constant DUMMY_P2WPKH_SCRIPT for dummy scriptPubKeys (b'a' * 35)
test: rbf, bip68: comment DUMMY_P2WPKH_SCRIPT constant, put into common (new) module
2019-10-14 15:03:11 +02:00
Hennadii Stepanov
c8961c7d9f doc: Add switch on enum example 2019-10-14 15:46:42 +03:00
MarcoFalke
4765b91f50 Merge #16667: build: remove mingw linker workaround from win gitian descriptor
bd3f5a90ec build: remove mingw linker workaround from win gitian descriptor (fanquake)

Pull request description:

  This workaround was added as part of the switch to gitian building using Ubuntu 14.04 (#6900).
  However, it should no longer be required, as we have switched to Bionic (#13171) and that
  has a far newer version of binutils.

  Original discussion: https://github.com/bitcoin/bitcoin/pull/6900
  binutils patch: https://sourceware.org/bugzilla/show_bug.cgi?id=16192

ACKs for top commit:
  MarcoFalke:
    ACK bd3f5a90ec
  theuni:
    ACK bd3f5a90ec
  laanwj:
    ACK bd3f5a90ec

Tree-SHA512: 01a5789994decf8cdedf7aaa0a449d2100a77e2e6b422d6b9dd5a4ac3e2e0b538c3d43aae4a1c3713614782f3c6b09d8d8bb21c20e86ce3c1734183dedd02d0c
2019-10-14 08:38:41 -04:00
Hennadii Stepanov
11e3d5eb1d util: Add AllowShortCaseLabelsOnASingleLine option 2019-10-14 13:24:54 +03:00
Wladimir J. van der Laan
d882f63589 Merge #17057: build: switch to upstream libdmg-hfsplus
9192ce5d63 depends: pull upstream libdmg-hfsplus changes (fanquake)
9b54f3436b depends: latest config.guess & config.sub (fanquake)

Pull request description:

  Pulled the first two commits off #16392, as they are standalone.

  Corys PRs (https://github.com/planetbeing/libdmg-hfsplus/pull/7, https://github.com/planetbeing/libdmg-hfsplus/pull/8) have been merged, and the author was active for a little while in 2017/18, so switch back to the upstream [`libdmg-hfsplus`](https://github.com/planetbeing/libdmg-hfsplus) repo. I've added the last of Corys patches into depends.

  [Switching back to upstream discussion](https://github.com/bitcoin/bitcoin/pull/16392#discussion_r303610767).

Top commit has no ACKs.

Tree-SHA512: a2313a731242251fef28921faf403c9bc354e6271cd7a26e9904ff5fae5766678136697fb5696006498a982ce319747b52b1df5151837388cda8f8145b45ab7c
2019-10-14 12:12:51 +02:00
João Barbosa
a8f5026d6d gui: Fix start timer from non QThread 2019-10-14 10:53:33 +01:00
Sebastian Falbesoner
fba4baa4fa test: speed up wallet_address_types by whitelisting peers (immediate tx relay)
approaches another part of #16613 ("Functional test suite bottlenecks")

As for wallet_backup.py (Commit 581c9be0d8), the
bottleneck is in relaying transactions. By whitelisting the peers, the
inventory is transmissioned immediately rather than on average every 5 seconds,
speeding up the test significantly:

before:
$ time ./wallet_address_types.py
real    1m30.072s
user    0m6.478s
sys     0m2.298s

with this PR:
$ time ./wallet_address_types.py
real    0m26.785s
user    0m5.525s
sys     0m1.888s
2019-10-13 21:11:37 +02:00
Ben Carman
9760293ce6 wallet: Fix for exported confirmation field in payment to self transactions 2019-10-13 11:10:36 -05:00
fanquake
e82f874a23 Merge #17121: test: speedup wallet_backup by whitelisting peers (immediate tx relay)
581c9be0d8 test: speedup wallet_backup by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  approaches part of #16613 ("Functional test suite bottlenecks")

  The majority of the test time is spent in `sync_mempools()` after sending to
  addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the
  peers via `-whitelist`, the inventory is transmissioned immediately rather than
  on average every 5 seconds, speeding up the test by at least a factor of two:

  before:
  ```
  $ time ./wallet_backup.py
  real    2m2.523s
  user    0m6.093s
  sys 0m2.454s
  ```
  with this PR:
  ```
  $ time ./wallet_backup_with_whitelist.py
  real    0m36.570s
  user    0m5.365s
  sys 0m1.696s
  ```
  Note that the test is not deterministic (the `sendtoaddress` RPC in function
  `one_send()` is executed with a probability of 50%), hence the times could vary
  between individual runs.

ACKs for top commit:
  MarcoFalke:
    ACK 581c9be0d8, this test is testing the backup behaviour, not the tx relay behaviour
  fanquake:
    ACK 581c9be0d8

Tree-SHA512: d016f39cdb85501e17a74a4c4db5a9f7404baa76fbcc3675a34d3cd7bf03d7a4cb4fd3e5f17cb0597248120bb5ac8b15d3db7663007b76b010902be72954bde0
2019-10-13 11:44:29 -04:00
Ben Carman
b3b6b6f62f gui: don't disable the sync overlay when wallet is disabled 2019-10-13 10:31:25 -05:00
Fabian Jahr
89339d1460 tests: Add test for loadblock option 2019-10-13 13:09:35 +02:00
Sebastian Falbesoner
581c9be0d8 test: speedup wallet_backup by whitelisting peers (immediate tx relay)
approaches part of #16613 ("Functional test suite bottlenecks")

The majority of the test time is spent in sync_mempools() after sending to
addresses, i.e. the bottleneck is in relaying transactions. By whitelisting the
peers via -whitelist, the inventory is transmissioned immediately rather than
on average every 5 seconds, speeding up the test by at least a factor of two:

before:
$ time ./wallet_backup.py
real    2m2.523s
user    0m6.093s
sys 0m2.454s

with this PR:
$ time ./wallet_backup_with_whitelist.py
real    0m36.570s
user    0m5.365s
sys 0m1.696s

Note that the test is not deterministic (the sendtoaddress RPC in function
one_send() is executed with a probability of 50%), hence the times could vary
between individual runs.
2019-10-13 02:50:08 +02:00
practicalswift
7e50abcc29 tests: Add EvalScript(...) fuzzing harness 2019-10-12 16:42:21 +00:00
Sjors Provoost
a0daea459c [build] depends macOS: point --sysroot to SDK 2019-10-12 16:05:33 +02:00
Igor Cota
d419ca7e32 depends: export dynamic JNI symbols from static qtforandroid.a 2019-10-12 13:25:06 +02:00
Igor Cota
ed30684d03 Qt: patch androidjnimain.cpp to make sure JNI is initialised when statically compiled 2019-10-12 12:49:25 +02:00
MarcoFalke
561a7d3047 Merge #16947: doc: Doxygen-friendly script/descriptor.h comments
15ac916642 doc: Doxygen-friendly descriptor.h comments (Jon Layton)

Pull request description:

  Closes #16942.

  - Make `Descriptor` overview subtext of `Interface for parsed descriptor objects.`
  - Conform to `@param[in, out] argname: Info` in parameter comments. Present in code: feb162d500/src/net_processing.cpp (L1001)
  - Remove redundant argument type, `in` vs `out` mentions
  - Removed unnecessary backticks around `IsSolvable()`, since Doxygen builds a link to the known function's docs
  - Add backticks to refer to `argname`s

  `descriptor.cpp` has more documentation, but Doxygen's output doesn't include anything inside unnamed namespaces for some reason. Tried to access them via searchbar.

Top commit has no ACKs.

Tree-SHA512: 587cc7596de46358a08b0321a7cf08a08785945715dbdce8945d837e1bee0664d1e11b1e47b7be85c4f35262f7ea173fb1f6202efcacc2023e2c6b0bd44133b3
2019-10-11 16:28:03 -04:00
MarcoFalke
fa6ed82794 doc: update bips.md with buried BIP9 deployments 2019-10-11 15:42:30 -04:00
MarcoFalke
f4d93078dd Merge #17097: build: Add README.md to DIST target
fa91590a74 build: Add README.md to DIST target (MarcoFalke)

Pull request description:

  This is required because our release tarball is generated by listing each needed file. See: #16734

  Should fix the failing builds after commit 9b4dfec831

ACKs for top commit:
  fanquake:
    ACK fa91590a74 - just checked that `README.md` is present inside the tarball after running `make dist`.

Tree-SHA512: 92e6145cbea4f7d0edfc88e4dcc11a34349bbf2f1948340531d2ff8832cfef94f979add8bd925b7e2c82a6bed4be40b8d41e94807d4a69983dcda3bccb206a31
2019-10-11 14:24:15 -04:00
fanquake
1ba49bcdc2 build: pass --enable-option-checking to applicable packages 2019-10-11 13:31:50 -04:00
fanquake
bcff8e21b1 build: only pass --disable-dependency-tracking to packages that understand it
By blanket passing --disable-dependency-tracking to all depends packages
we end up with some warnings like:

configure: WARNING: unrecognized options: --disable-dependency-tracking

So instead, only pass it to packages that understand it.

Related to https://github.com/bitcoin/bitcoin/issues/16354.
2019-10-11 11:54:39 -04:00
fanquake
8261587610 Merge #17102: doc: Add missing indexes/blockfilter/basic to doc/files.md
fa191c058f doc: Add missing indexes/blockfilter/basic/ to doc/files.md (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fa191c058f
  fanquake:
    ACK fa191c058f

Tree-SHA512: ea30776224e4a26bc6f60c9c65361f0b9d7b2b4f844c6d2aefd108f852dd7d813963df631a8ed20daea5acae7a23d0b405d12a162f90a6f61a3db236f5beed45
2019-10-11 08:59:34 -04:00
MarcoFalke
fa191c058f doc: Add missing indexes/blockfilter/basic/ to doc/files.md 2019-10-11 08:31:18 -04:00
Adam Jonas
66b29848c7 change wallet pointers to references in feebumper 2019-10-10 17:46:59 -04:00
practicalswift
bebb637472 tests: Add FuzzedDataProvider fuzzing helper from the Chromium project
Source: https://cs.chromium.org/chromium/src/third_party/libFuzzer/src/utils/FuzzedDataProvider.h?rcl=b9f51dc8c98065df0c8da13c051046f5bab833db
2019-10-10 21:13:33 +00:00
MarcoFalke
08ed87e887 Merge #16983: doc: Add detailed info about Bitcoin Core files
86b9f92da2 doc: Add detailed info about Bitcoin Core files (Hennadii Stepanov)

Pull request description:

  This PR:
  - provides detailed info about the Bitcoin Core files;
  - does not mention temporary files, e.g., `mempool.dat.new` and `peers.????`

ACKs for top commit:
  ch4ot1c:
    ACK 86b9f92
  laanwj:
    ACK 86b9f92da2
  MarcoFalke:
    ACK 86b9f92da2

Tree-SHA512: 9352119b08e3f6aaab4ce3797afc6533f90852e461957acb2bc73962fd4881403fabeaa5a371bd1218309f36f9b0f90fb147b80698e2e30a016634a62a160a15
2019-10-10 15:58:28 -04:00
Jon Layton
7aad3b68e7 doc: Doxygen-friendly CuckooCache comments 2019-10-10 15:47:39 -04:00
John Newbery
9075d13153 [docs] Add release notes for removal of REJECT reasons 2019-10-10 13:31:57 -04:00
John Newbery
04a2f326ec [validation] Fix REJECT message comments 2019-10-10 13:31:57 -04:00
John Newbery
e9d5a59e34 [validation] Remove REJECT code from CValidationState
We no longer send BIP 61 REJECT messages, so there's no need to set
a REJECT code in the CValidationState object.
2019-10-10 13:31:53 -04:00
MarcoFalke
d5a770b70d Merge #16973: test: Fix combine_logs.py for AppVeyor build
d478a472eb test: Fix combine_logs.py for AppVeyor build (Martin Zumsande)

Pull request description:

  Fixes #16894

  This fixes the problem of AppVeyor builds not showing `debug.log` if a functional test fails, because the windows separator `\` doesn't work together with the regex in `combine_logs.py`.

  A fix was already attempted in  #16896, however, that PR became inactive and was marked "up for grabs", plus it's a really small change.

  As suggested by jamesob, this PR uses `pathlib`: For the glob and to convert the path to a posix-style string, it leaves the regex as is (in contrast to #16896 which adjusted the regex).

  I tested this locally on Windows and Ubuntu.

Top commit has no ACKs.

Tree-SHA512: 603b4359b6009b6da874c30f69759acda03730ee5747898a0fe957a5fc37ee9ba07858c6aa2169bf4c40521f37e47138e8314d698652ea2760fa0a3f76b890bd
2019-10-10 13:19:10 -04:00
MarcoFalke
befdef8aee Merge #16887: Abstract out some of the descriptor Span-parsing helpers
bb36372b8f test: add unit tests for Span-parsing helpers (Sebastian Falbesoner)
5e69aeec3f Add documenting comments to spanparsing.h (Pieter Wuille)
230d43fdbc Abstract out some of the descriptor Span-parsing helpers (Pieter Wuille)

Pull request description:

  As suggested here: https://github.com/bitcoin/bitcoin/pull/16800#issuecomment-531605482.

  This moves the Span parsing functions out of the descriptor module, making them more easily usable for other parsers (in particular, in preparation for miniscript parsing).

ACKs for top commit:
  MarcoFalke:
    ACK bb36372b8f

Tree-SHA512: b5c5c11a9bc3f0a1c2c4cfa22755654ecfb8d4b69da0dc1fb9f04e1556dc0f6ffd87ad153600963279ac465d587d7971b53d240ced802d12693682411ac73deb
2019-10-10 12:33:03 -04:00
MarcoFalke
59f0687fea Merge #16786: test: add unit test for wallet watch-only methods involving PubKeys
a57a1d42d5 test: add unit test for wallet watch-only methods involving PubKeys (Sebastian Falbesoner)

Pull request description:

  The motivation for this addition was to unit test the function `wallet.cpp:ExtractPubKey()` (see recent change in commit 798a589aff) which is however static and only indirectly available via the public methods `AddWatchOnly()`, `LoadWatchOnly()` and `RemoveWatchOnly()`. Since the first of those methods also stores the addresses to the disk, the second, simpler one was chosen which only operates in memory.

ACKs for top commit:
  Sjors:
    ACK a57a1d4
  instagibbs:
    reACK a57a1d42d5
  Sjors:
    re-ACK a57a1d4

Tree-SHA512: 92a242204ab533022cd848662997372c41815b1265d07b3d96305697f801db29a5ba5668337faf4bea702bec1451972529afd6665927fb142aaf91700a338b26
2019-10-10 12:12:54 -04:00
Adam Jonas
9be6666a4e typo and unneccessary parentheses 2019-10-10 12:10:15 -04:00
John Newbery
0053e16714 [logging] Don't log REJECT code when transaction is rejected
Remove the BIP61 REJECT code from error messages and logs when a
transaction is rejected.

BIP61 support was removed from Bitcoin Core in
fa25f43ac5. The REJECT codes will be
removed from the codebase entirely in the following commit.
2019-10-10 11:19:42 -04:00
John Newbery
a1a07cfe99 [validation] Fix peer punishment for bad blocks
Because the call to MaybePunishNode() in
PeerLogicValidation::BlockChecked() only previously happened if the
REJECT code was > 0 and < REJECT_INTERNAL, then there are cases were
MaybePunishNode() can get called where it wasn't previously:

- when AcceptBlockHeader() fails with CACHED_INVALID.
- when AcceptBlockHeader() fails with BLOCK_MISSING_PREV.

Note that BlockChecked() cannot fail with an 'internal' reject code. The
only internal reject code was REJECT_HIGHFEE, which was only set in
ATMP.

This change restores the behaviour pre-commit
5d08c9c579 which did punish nodes that
sent us CACHED_INVALID and BLOCK_MISSING_PREV blocks.
2019-10-10 11:17:01 -04:00
MarcoFalke
fa91590a74 build: Add README.md to DIST target 2019-10-10 11:06:43 -04:00
MarcoFalke
2a0164085b Merge #16689: rpc: add missing fields to wallet rpc help output
f4c8953b00 Add missing fields in TransactionDescriptionString and others (Antoine Riard)
3530108491 MOVEONLY : move RPC wallets helpers to TransactionDescriptionString (Antoine Riard)

Pull request description:

  Knowledge of `walletconflicts` field existence is really nice when you're debugging conflicts. Was added in #3671 but never documented in RPC helps.

  Others were added after a quick skim, we may still have missing ones in wallet rpcs.

ACKs for top commit:
  MarcoFalke:
    re-ACK f4c8953b00 (only change is addressing my nits)
  kristapsk:
    ACK f4c8953b00

Tree-SHA512: 2bea4d8743399fb152f942df7454548b896b2ad5654fd4bf60253afec1a5387ef8797ced97776dc7ba4912291263c08abe7c2b608c6a28f9a0df67be4ebc4635
2019-10-10 10:04:36 -04:00
MarcoFalke
2352aec9fc Merge #17076: tests: Add fuzzing harness for CheckTransaction(...), IsStandardTx(...) and other CTransaction related functions
5c2987636f tests: Remove TRANSACTION_DESERIALIZE (replaced by transaction fuzzer) (practicalswift)
0a573682f2 tests: Add fuzzing harness for CheckTransaction(...), IsStandardTx(...) and other CTransaction related functions (practicalswift)

Pull request description:

  Add fuzzing harness for `CheckTransaction(...)`, `IsStandardTx(...)` and other `CTransaction` related functions.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ src/test/fuzz/transaction
  …
  # And to to quickly verify that the relevant code regions are triggered, that the
  # fuzzing throughput seems reasonable, etc.
  $ contrib/devtools/test_fuzzing_harnesses.sh '^transaction$'
  ```

  `test_fuzzing_harnesses.sh` can be found in PR #17000.

ACKs for top commit:
  MarcoFalke:
    ACK 5c2987636f

Tree-SHA512: 2f422df795c9dca13c98209ca9ce0fe5a0d4a71fb052fa33d599cc9c9f1d637fee27d58d02ed17b956b3e3d40931cbc1367fc99aa2e882473e54d95dee04d6b7
2019-10-10 09:18:34 -04:00
Wladimir J. van der Laan
b67ca4ee73 Merge #17085: init: Change fallback locale to C.UTF-8
facb9a1315 init: Change fallback locale to C.UTF-8 (Wladimir J. van der Laan)

Pull request description:

  Much of our code assumes file system UTF-8 support, and this is a more realistic guess for modern systems anyway than the default character set (which would be ASCII only). So change the assumed fallback locale (if no locale is defined by the user or OS) to `C.UTF-8`.

  related: https://github.com/bitcoin/bitcoin/issues/14948#issuecomment-488385462

ACKs for top commit:
  MarcoFalke:
    ACK facb9a1315

Tree-SHA512: 5075f9fe6791572d76ec38c58cd56f04ed8086c06a7d7f446d062dffc313c62466ba81f1a7d6b8c7e95791fcff82e4f76871c3534478fbfe5beb456dd8eea340
2019-10-10 13:09:50 +02:00
Jonas Schnelli
5cb1d938a1 Merge #15756: gui: Add shortcuts for tab tools
091747b46e gui: Add shortcuts for tab tools (João Barbosa)

Pull request description:

  This makes accessing the RPC console very fast/easy. It also improves accessibility.

  <img width="234" alt="Screenshot 2019-10-02 at 01 30 53" src="https://user-images.githubusercontent.com/3534524/66009867-50104300-e4b4-11e9-90b5-6b8dc961a8a1.png">

ACKs for top commit:
  jonasschnelli:
    Tested ACK 091747b46e - this is an improvment. Further solutions to solve the interference between the console and the shortcuts (if possible) can be done upstream (Qt) or with another PR.

Tree-SHA512: 6b8bc07e8a3a75e53c05f0fdb73458d75ef025f950569e885e655de53fdac8b91dcabfb1c6e643b1d23065420fa2701847c00cc1718bc188778640aefb5bcbd8
2019-10-10 09:03:44 +02:00
Jonas Schnelli
b265ffe323 Merge #15023: GUI: Restore RPC Console to non-wallet tray icon menu
f33efa8ec5 GUI: Restore RPC Console to non-wallet tray icon menu (Luke Dashjr)

Pull request description:

  #14383 moved the debug window's menu position, to make it conditional on wallet mode. The rationale given was to match the behaviour of the 'Help' menu.

  #14573 replaced the 'Help' menu's conditional debug window with an unconditional list of items in the new 'Window' menu.

  This PR reverts the no-longer-applicable part of #14383, putting the debug window back on the tray menu unconditionally, and in the position it previously had.

ACKs for top commit:
  jonasschnelli:
    Tested ACK f33efa8ec5 - the debug window is also accessible from the menu (though directly the subpages which counts IMO).

Tree-SHA512: c04a588fed37a8c31cb413baaa346e3c1c18724f9b40d64b8528c517f65290930d577bccf0a794180e968e84d3c52e9fa3fdc8a40bbc5fe3418eaddd73481271
2019-10-10 09:02:34 +02:00
Sebastian Falbesoner
bb36372b8f test: add unit tests for Span-parsing helpers
tests the following four functions:
    - Const() [parse constant]
    - Func()  [parse function]
    - Expr()  [parse expression]
    - Split() [split up a string]
2019-10-09 18:11:14 -07:00
fanquake
bc7e721d97 Merge #17087: build: Add variable printing target to Makefiles
181989f6c9 build: Add variable printing target to Makefiles (Carl Dong)

Pull request description:

  ```
  I kept finding myself needing these to debug our build system, since
  they are innocuous and are very helpful they probably belong in the
  codebase.

  Source: John Graham-Cumming
  https://www.cmcrossroads.com/article/printing-value-makefile-variable
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 181989f6c9
  fanquake:
    ACK 181989f6c9 - concise amount of useful code. Tested on macOS. Did not visit the link.

Tree-SHA512: 2139621e68a499c7347663ca9dc04e166ea6280e05986c27858df0156016ef2f9461262464d70c601419384f43a4ae3bcc67dfc0a05dbeef64f08386ab429cd8
2019-10-09 17:48:36 -04:00
MarcoFalke
42d0eca725 Merge #13728: lint: Run the CI lint stage on mac
cd82f75a43 lint: Install grep and git via brew on mac for --perl-regexp (Ben Woosley)
eafa747ca5 lint: Fix shellcheck SC2155 (Ben Woosley)
615ff4e7db lint: Run the linters against Mac OS on Travis (Ben Woosley)

Pull request description:

  This helps ensure ongoing compatibility with macOS-distributed version of GNU bash.

ACKs for top commit:
  MarcoFalke:
    ACK cd82f75a43

Tree-SHA512: 8d56d2303bbebedba8ea2291f4ab35b7fdf3245b7a4c3f04557eee4f19d83573798ad32facc92bfa060aaeb294e6d2c95e6d1c3b795fd7951dcf3aa1cccec107
2019-10-09 14:37:53 -04:00
fanquake
a2ae766a61 Merge #15098: qt: Show addresses for "SendToSelf" transactions
11fdfcf7f9 Show addresses for "SendToSelf" transactions (Hennadii Stepanov)

Pull request description:

  Fix #11464
  Fix #12688
  Ref: #11471 by jonasschnelli

  Note: change addresses are not recognized (ref: https://github.com/bitcoin/bitcoin/pull/11471#discussion_r180547041)

  Result:
  ![screenshot from 2019-01-04 09-06-12](https://user-images.githubusercontent.com/32963518/50677356-19c79500-1002-11e9-986c-999c366e4b2a.png)

ACKs for top commit:
  jonasschnelli:
    Tested ACK 11fdfcf7f9
  fanquake:
    ACK 11fdfcf7f9 - did the bare minimum testing.

Tree-SHA512: 2678a2fdf017c376750c73fdc751b7838b0d3a970ba02e9032e4c5824494362672036c3ebf87b425aefdfe197fb952b70e4b7b6011077abb39a8bfc1ae14dfd2
2019-10-09 14:35:29 -04:00
Antoine Riard
f4c8953b00 Add missing fields in TransactionDescriptionString and others
Fields involvesWatchonly, generated, walletconflicts were missing
in result description of listtransactions, listsinceblock,
gettransaction

Align getttransaction fields which were odd compare to other rpc
helpers
2019-10-09 14:26:50 -04:00
Antoine Riard
3530108491 MOVEONLY : move RPC wallets helpers to TransactionDescriptionString 2019-10-09 14:22:02 -04:00
MarcoFalke
fae43a97ca test: Seed test RNG context for each test case, print seed 2019-10-09 11:20:12 -04:00
Carl Dong
181989f6c9 build: Add variable printing target to Makefiles
I kept finding myself needing these to debug our build system, since
they are innocuous and are very helpful they probably belong in the
codebase.

Source: John Graham-Cumming
https://www.cmcrossroads.com/article/printing-value-makefile-variable
2019-10-09 11:12:55 -04:00
Ben Woosley
cd82f75a43 lint: Install grep and git via brew on mac for --perl-regexp
Particularly `--with-pcre2` is needed to run `git grep --perl-regexp` in
`test/link/check-doc.py`
2019-10-09 15:50:42 +01:00
Ben Woosley
eafa747ca5 lint: Fix shellcheck SC2155
Declare and assign separately to avoid masking return values.

https://github.com/koalaman/shellcheck/wiki/SC2155
2019-10-09 15:50:42 +01:00
Ben Woosley
615ff4e7db lint: Run the linters against Mac OS on Travis
This helps ensure ongoing compatibility with macOS-distributed version
of GNU bash.
2019-10-09 15:49:51 +01:00
fanquake
9192ce5d63 depends: pull upstream libdmg-hfsplus changes 2019-10-09 10:41:51 -04:00
fanquake
9b54f3436b depends: latest config.guess & config.sub 2019-10-09 10:40:42 -04:00
fanquake
1c11e53bb6 Merge #17041: ci: Run tests on arm
fa79dff624 ci: Run tests on arm (MarcoFalke)

Pull request description:

  Closes #16576

ACKs for top commit:
  laanwj:
    ACK fa79dff624
  fanquake:
    ACK fa79dff624 - assuming Travis is green again.

Tree-SHA512: c430db9852632567c6836981fb3c5922ccd7d3b6ab4a1c6405ffad75096b80433ba54785ffa4c5088c1a127689a945f0f86058a42de1d3efea3cc4967832d662
2019-10-09 10:30:48 -04:00
practicalswift
5c2987636f tests: Remove TRANSACTION_DESERIALIZE (replaced by transaction fuzzer) 2019-10-09 13:53:13 +00:00
practicalswift
0a573682f2 tests: Add fuzzing harness for CheckTransaction(...), IsStandardTx(...) and other CTransaction related functions 2019-10-09 13:53:13 +00:00
fanquake
ceecefe0b0 Merge #17081: doc: fix Makefile target in benchmarking.md
a54ab2104c [doc] fix Makefile target in benchmarking.md (Sebastian Falbesoner)

Pull request description:

  While the resulting binary is called `bench_bitcoin`, the Makefile target is
  named `bitcoin_bench` (see `src/Makefile.bench.include`)

ACKs for top commit:
  fanquake:
    ACK a54ab2104c - Tested on macOS and Debian 9.9, as this only [seemed to work there](https://github.com/bitcoin/bitcoin/pull/16536#discussion_r310366868) when these docs were added.

Tree-SHA512: bcf8d48ccba488f0533111a3be57ddc6c948b3a38beed129635e1c7e0b4608bc9ddf625e8469606bb31d4cedf3341c443564a197d6b1ab5268a9ed44ed5018a3
2019-10-09 09:51:36 -04:00
MarcoFalke
fa79dff624 ci: Run tests on arm 2019-10-09 09:33:07 -04:00
fanquake
b1de33d29a Merge #16821: Fix bug where duplicate PSBT keys are accepted
9743432034 Fix bug where duplicate PSBT keys are accepted (John L. Jegutanis)

Pull request description:

  As per the BIP 174 spec a PSBT key cannot be duplicated,
  however the current code accepts key duplication.
  The PSBT key/value entries can be duplicated when the value
  is `empty()` or `IsNull()` for `CScript` or `CTxOut` respectively
  and if those key/value entries are serialized before the non-empty ones.

  For example, the following PSBT, included in the test vectors,
  contains a duplicate field:

  ```
  // magic
  70736274ff

  // global tx
  //// key
  0100
  //// value
  2a02000000000140420f000000000017a9146e91b72d5593e7d4391e2ff44e91e985c31641f08700000000
  //// separator
  00

  // no inputs

  // outputs
  //// key PSBT_OUT_WITNESSSCRIPT
  0101
  //// value (empty script)
  00
  //// key PSBT_OUT_WITNESSSCRIPT (same as the above)
  0101
  //// value (an OP_RETURN script)
  016a
  //// separator
  00
  ```

ACKs for top commit:
  achow101:
    ACK 9743432034
  instagibbs:
    code review ACK 9743432034

Tree-SHA512: 34f4b34c8e6561c6a6ab745cdd319f6687eac6f7cecc735c94035eeca8c5157e17a27f2ae853dbaa6634fcd5a8f4e1c6cc13d1ebd7e563459665d72bb147cc1e
2019-10-09 09:18:27 -04:00
Wladimir J. van der Laan
97e05817d4 Merge #17084: build: Fix #include sys/poll.h to just poll.h (without sys/)
4de0bde7bc build: Fix #include sys/poll.h to just poll.h (without sys/) (Daki Carnhof)

Pull request description:

  http://pubs.opengroup.org/onlinepubs/009695399/functions/poll.html
  http://man7.org/linux/man-pages/man2/poll.2.html

ACKs for top commit:
  Empact:
    ACK 4de0bde7bc

Tree-SHA512: 01c22a62b5bc327b3a46f721312af2283f4e09cb314bc7a3f82dbc1f4eda6f018a394559c4370fee895532e768b8a9152783979d61cb8a0ed86de069f7c150fb
2019-10-09 15:01:58 +02:00
Wladimir J. van der Laan
facb9a1315 init: Change fallback locale to C.UTF-8
Much of our code assumes UTF-8 support, and this is a more realistic
guess for modern systems anyway than the default character set (which
would be ASCII only). So change the assumed fallback locale (if no
locale is defined by the user or OS) to `C.UTF-8`.
2019-10-09 14:53:18 +02:00
Ben Woosley
111880aaf7 [test] Add coverage to estimaterawfee and estimatesmartfee
This adds light functional coverage to estimaterawfee - a subset of
the testing applied to estimatesmartfee, and argument validation
testing to both estimaterawfee and estimatesmartfee.

One valid estimatesmartfee signature test is commented out because it
fails currently.
2019-10-09 12:49:30 +01:00
Daki Carnhof
4de0bde7bc build: Fix #include sys/poll.h to just poll.h (without sys/)
http://pubs.opengroup.org/onlinepubs/009695399/functions/poll.html
http://man7.org/linux/man-pages/man2/poll.2.html
2019-10-09 13:30:36 +02:00
Wladimir J. van der Laan
c08bf2b574 Merge #15437: p2p: Remove BIP61 reject messages
fa25f43ac5 p2p: Remove BIP61 reject messages (MarcoFalke)

Pull request description:

  Reject messages (BIP 61) appear in the following settings:

  * Parsing of reject messages (in case `-debug=net` is set, off by default). This has only been used for a single `LogPrint` call for several releases now. Such logging is completely meaningless to us and should thus be removed.

  * The sending of reject messages (in case `-enablebip61` is set, off by default). This can be used to debug a node that is under our control. Instead of hacking this debugging into the p2p protocol, it could be more easily achieved by parsing the debug log. (Use `-printtoconsole` to have it as stream, or read from the `debug.log` file like our python function `assert_debug_log` in the test framework does)

  Having to maintain all of this logic and code to accommodate debugging, which can be achieved by other means a lot easier, is a burden. It makes review on net processing changes a lot harder, since the reject message logic has to be carried around without introducing any errors or DOS vectors.

ACKs for top commit:
  jnewbery:
    utACK fa25f43ac5
  laanwj:
    I'm still not 100% convinced that I like getting rid of BIP61 conceptually, but apparently everyone wants it, code review ACK fa25f43ac5.
  ryanofsky:
    Code review ACK fa25f43ac5

Tree-SHA512: daf55254202925e56be3d6cfb3c1c804e7a82cecb1dd1e5bd7b472bae989fd68ac4f21ec53fc46751353056fd645f7f877bebcb0b40920257991423a3d99e0be
2019-10-09 11:51:58 +02:00
Wladimir J. van der Laan
1d9ac7fded Merge #17074: build: Added double quotes
dd85166b6c Added double quotes (Ava Barron)

Pull request description:

  Added double quotes to prevent globbing and word splitting.

ACKs for top commit:
  laanwj:
    ~~ACK dd85166b6c8200265ff222226305e6823149de19~~
  practicalswift:
    ACK dd85166b6c -- diff looks correct
  hebasto:
    ACK dd85166b6c, tested locally on Linux Mint 19.2.

Tree-SHA512: 503ed8c4a5dafd3868465fbb66b13e15862c114573609f681b69259703e76fe85260b782c45a1e0780b7d3fdf34d8fd2f627b59f4013b99eeb1b771a652e7b10
2019-10-09 09:45:38 +02:00
Wladimir J. van der Laan
9b4dfec831 Merge #17029: gitian: Various improvements for Windows descriptor
9d1f971c75 gitian: Put things in the right place to begin with (Carl Dong)
71949a97a7 gitian: Eliminate rename dependency (Carl Dong)
999a9a5f5b gitian: Smaller diff with gitian-linux.yml (Carl Dong)
c4a3c25ba1 gitian: Fix README inclusion in archives (Carl Dong)
93cb974980 gitian: Use split-debug.sh for Win builds (Carl Dong)

Pull request description:

  It would seem that our `gitian-win.yml` has not been keeping up with `gitian-linux.yml`, this PR:

  1. Minimizes the diff size between `gitian-{win,linux}.yml`
  2. Eliminates the `rename` dependency

ACKs for top commit:
  laanwj:
    ACK 9d1f971c75

Tree-SHA512: 84ed47c685e12d0064c02811907ae3d0fd3c47db8773d497dcc38f0defbfb3040fd82899fb026cf355f229b906d05a1c8038a95642bb90d044afbc2e0b239af2
2019-10-09 09:38:00 +02:00
Sebastian Falbesoner
a54ab2104c [doc] fix Makefile target in benchmarking.md
while the resulting binary is called `bench_bitcoin`, the Makefile target is
named `bitcoin_bench` (see `src/Makefile.bench.include`)
2019-10-08 22:14:04 +02:00
fanquake
520d140e6e Merge #17056: descriptors: Introduce sortedmulti descriptor
4bb660be90 Add release note (Andrew Chow)
ed96b295d7 Update descriptors.md to include sortedmulti (Andrew Chow)
80be78ea75 Test sortedmulti descriptor using BIP 67 tests (Andrew Chow)
6f588fd227 Add sortedmulti descriptor and unit tests (Andrew Chow)

Pull request description:

  Adds a `sortedmulti()` descriptor as mentioned in https://github.com/bitcoin/bitcoin/pull/17023#issuecomment-537596416.

  `sortedmulti()` works in the same way as `multi` does but sorts the pubkeys in the resulting scripts in lexicographic order as described in [BIP67](https://github.com/bitcoin/bips/blob/master/bip-0067.mediawiki). Note that this does not add support for BIP67 nor is BIP67 fully supported by this descriptor (which is why it is not named `multi67()`) as it does not require compressed pubkeys.

  Tests from BIP67 were added and documentation was updated.

ACKs for top commit:
  instagibbs:
    re-ACK 4bb660be90
  Sjors:
    re-ACK 4bb660be90

Tree-SHA512: 93b21112a74ebe0bf316d8f3e0291f69fd975cf0a29332f9728e7b880cad312b8b14007e86adcd7899f117b9303cbcf4cb35f3bb2f2f648d1a446f83f75a70a5
2019-10-08 14:26:26 -04:00
Andrew Chow
4bb660be90 Add release note 2019-10-08 13:56:56 -04:00
Andrew Chow
ed96b295d7 Update descriptors.md to include sortedmulti 2019-10-08 13:56:56 -04:00
Andrew Chow
80be78ea75 Test sortedmulti descriptor using BIP 67 tests 2019-10-08 13:56:56 -04:00
Andrew Chow
6f588fd227 Add sortedmulti descriptor and unit tests 2019-10-08 13:56:56 -04:00
Carl Dong
9d1f971c75 gitian: Put things in the right place to begin with 2019-10-08 13:39:48 -04:00
Carl Dong
71949a97a7 gitian: Eliminate rename dependency 2019-10-08 13:39:10 -04:00
Carl Dong
999a9a5f5b gitian: Smaller diff with gitian-linux.yml 2019-10-08 13:39:09 -04:00
Carl Dong
c4a3c25ba1 gitian: Fix README inclusion in archives
Linux:
The README was originally added in 8550f1fb2, but included the README
under the docs directory, which has a bunch of internal links that won't
make sense in a release tarball. In this patch, we include the root
level README instead, which makes more sense.

Windows:
.md files are inconvenient to open on windows and the line endings
differ, so we use README_windows.txt instead.
2019-10-08 13:36:50 -04:00
MarcoFalke
facec1c643 wallet: Avoid showing GUI popups on RPC errors 2019-10-08 13:02:14 -04:00
Wladimir J. van der Laan
e173d587e1 Merge #17068: qt: Always generate bitcoinstrings.cpp on make translate
c640ca24f9 Always generate `bitcoinstrings.cpp` on `make translate` (Franck Royer)

Pull request description:

  Resolves #16891.

  `bitcoinstrings.cpp` is to be generated at release time. Hence,
  it should not depend on whether the source files are younger as the
  releaser may proceed from a fresh checkout.

  More information on the investigation in the issue.

ACKs for top commit:
  jonasschnelli:
    Tested ACK c640ca24f9 - Tested master + this on macOS 10.14. With this PR, it always "runs" `GEN      qt/bitcoinstrings.cpp` and `GEN      translate`.

Tree-SHA512: f799fdc3ad16a2a6a59704bc2c50f5179e6a7e064d8f43354592f11857cc901cac99b2b90f3319d25d49c9d78378b8d119cc5f59b48ea7f1008f33dd26700877
2019-10-08 14:08:08 +02:00
Wladimir J. van der Laan
cc703a2398 Merge #17066: build: Remove workaround for ancient libtool
30fc1a3f54 build: Remove workaround for ancient libtool (Hennadii Stepanov)
6ca01b9a10 build: Ensure a minimal version of libtool (Hennadii Stepanov)

Pull request description:

  Since libtool 1.5.2, on Linux libtool no longer sets RPATH for any directories in the dynamic linker search path, so there is no longer an issue.

  This commit reverts a98356fee8.

  Refs:
  - https://wiki.debian.org/RpathIssue
  - [Debian jessie has libtool 2.4.2](https://packages.debian.org/jessie/libtool)

ACKs for top commit:
  laanwj:
    ACK 30fc1a3f54

Tree-SHA512: fab56265d4d2c96216a353cc076c6f510e15748d8134f97bae2f67b6d8c0b6a1a9f362d2ab23b19ccc3a8bba8eac3bb1668fc3e42037590f63a7ab4819c9ee15
2019-10-08 11:49:08 +02:00
Wladimir J. van der Laan
9f8cd0ac89 Merge #17059: util: Simplify path argument for CBlockTreeDB ctor
c2bb3919a8 util: Simplify path argument for CBlockTreeDB ctor (Hennadii Stepanov)

Pull request description:

  This PR:
  - simplifies path argument (`datadir/blocks/index`) for `CBlockTreeDB`  constructor
  - does not change behavior as `GetBlocksDir()` with unset "-blocksdir" returns the same path
  - improves code readability

ACKs for top commit:
  MarcoFalke:
    ACK c2bb3919a8
  laanwj:
    ACK c2bb3919a8
  promag:
    ACK c2bb3919a8.

Tree-SHA512: 646a0a3a31e2f419b05f696cbdfb7d8987f1d89ec0797b72464ae05680fd5f95f6469be0ea5b56f772434c49d48504cd9cf9760c05d4054d11349d502e157ee2
2019-10-08 11:47:24 +02:00
Wladimir J. van der Laan
99cebc922c Merge #16569: Increase init file stop timeout
7fb7acfc20 Set init stop timeout to 10 min (setpill)

Pull request description:

  `bitcoind` can take a long time to flush its db cache to disk upon
  shutdown. Systemd sends a `SIGKILL` after a timeout, causing unclean
  shutdowns and triggering a long "Rolling forward" at the next startup.
  Disabling the timeout should prevent this from happening, and does not
  break systemd's `restart` logic.

  Addresses #13736.

ACKs for top commit:
  instagibbs:
    utACK 7fb7acfc20

Tree-SHA512: 16e0ce5a9ecf0628f8d93d68db3f5a78ab36021d9bede05a90c84f144db2e87e17707a6eb910cb7c018c265ce2c81d43de2988bd79e4a2d8554515db8fb5aa36
2019-10-08 11:38:44 +02:00
Wladimir J. van der Laan
34919e0033 Merge #17011: ci: Use busybox utils for one build
ddddd8961b ci: Use busybox utils for one build (MarcoFalke)

Pull request description:

  To make sure Bitcoin Core can be built with BusyBox, see https://github.com/bitcoin/bitcoin/pull/16927#issuecomment-536483706

ACKs for top commit:
  laanwj:
    ACK ddddd8961b

Tree-SHA512: da3a4654ee7975206d04643675d309b4973a510ca344acaec97fb1ed19c43cf13489bdf236c92c4a90499ec5b3c18c3338fff096110b26abee5ffe955089f267
2019-10-08 11:31:48 +02:00
Wladimir J. van der Laan
2f174498d6 Merge #17073: move-only: PASTE macros to util/macros.h
0d86f4d3da refactor: consolidate PASTE macros (James O'Beirne)

Pull request description:

  Really dumb move-only stolen from #16805. Some of my pull requests also depend on this, so I split it up to not depend on #16805.

ACKs for top commit:
  practicalswift:
    ACK 0d86f4d3da -- diff looks correct
  hebasto:
    ACK 0d86f4d3da, I have reviewed the code and it looks OK, I agree it can be merged.
  promag:
    ACK 0d86f4d3da.

Tree-SHA512: 19208a8cbf83034b1ef25138d8f08d8f32ace7775f654b1597fc4599dd576f0758145f592f161cfdcaaa29d4907ac9aa5553f6f524e2b960205c760605a05901
2019-10-08 11:25:51 +02:00
Wladimir J. van der Laan
df50fd194f Merge #16802: scripts: In linearize, search for next position of magic bytes rather than fail
3284e6c09a scripts: search for next position of magic bytes rather than fail (Tim Akinbo)

Pull request description:

  When using the `linearize-data.py` contrib script to export block data, there are edge cases where the script fails with an `Invalid magic: 00000000` error. This error occurs due to the presence of padding bytes that occasionally appears between consecutive blocks in the block data file.

  There's an ongoing conversation about this in #14986. sipa also admitted that it is a bug in #5028. Fortunately, this is not an issue in bitcoin core as it handles this type of situation gracefully and so no fix in bitcoin core is required.

  This PR is an improvement on how the script handles these "invalid magic bytes". Rather than failing, this patch allows the script to search for the next occurrence of the magic bytes and then starts reading the block from there.

ACKs for top commit:
  laanwj:
    ACK 3284e6c09a

Tree-SHA512: 18067ae0b4b62e822dfc558a86439ad6acaf939b98479e38e8e4248536574643b26eb48e96ec7139375c88b42cbe7705a64deb13a3c239e16025a6aad3d69bfa
2019-10-08 10:42:04 +02:00
Wladimir J. van der Laan
866fd2888f Merge #17030: test: Fix Python Docstring to include all Args.
8acd58927a Fix Python Docstring to include all Args. (John Bampton)

Pull request description:

  Found a Python function that had incorrect and missing arguments in its Docstring.

ACKs for top commit:
  laanwj:
    ACK 8acd58927a

Tree-SHA512: 936f275f29a700d630bb479b5283e47b66f2df76d8b8c053f594e6aedf783cc98a29c924c3a46613f112dfc884acb50f21a0b18f96d939e887b12b921ef2e10f
2019-10-08 10:29:28 +02:00
John L. Jegutanis
9743432034 Fix bug where duplicate PSBT keys are accepted
As per the BIP 174 spec a PSBT key cannot be duplicated,
however the current code accepts key duplication.
The PSBT key/value entries can be duplicated when the value
is `empty()` or `IsNull()` for `CScript` or `CTxOut` respectively
and if those key/value entries are serialized before the non-empty ones.

For example, the following PSBT, included in the test vectors,
contains a duplicate field:

```
// magic
70736274ff

// global tx
//// key
0100
//// value
2a02000000000140420f000000000017a9146e91b72d5593e7d4391e2ff44e91e985c31641f08700000000
//// separator
00

// no inputs

// outputs
//// key PSBT_OUT_WITNESSSCRIPT
0101
//// value (empty script)
00
//// key PSBT_OUT_WITNESSSCRIPT (same as the above)
0101
//// value (an OP_RETURN script)
016a
//// separator
00
```
2019-10-08 01:45:36 +03:00
Ava Barron
dd85166b6c Added double quotes
Added double quotes to prevent globbing and word splitting.
2019-10-07 17:02:46 -04:00
Andrew Chow
44f7a8d7a7 Disable _FORTIFY_SOURCE when enable-debug
Since enable-debug disables optimization entirely, _FORTIFY_SOURCE
does nothing as it requires some kind of optimization enabled. It
instead produces a bunch of useless warnings. So explicitly disable
it when enable-debug so that those warnings are not produced.
2019-10-07 14:11:40 -04:00
MarcoFalke
febc31c641 Merge #17069: tests: Pass fuzzing inputs as constant references
ffa2221256 tests: Pass fuzzing inputs as constant references (practicalswift)

Pull request description:

  Pass fuzzing inputs as constant references.

  Split out from #17009 as suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/17009#discussion_r331502028.

ACKs for top commit:
  MarcoFalke:
    ACK ffa2221256

Tree-SHA512: c1e3d6658a0b45cece2ed7e7c2ba1b78cdb71a03767231de7e5c212575117a4e2b70079265c745963480d7fcd4d9706561a2325b8c993b94eec33268ad4b293c
2019-10-07 10:24:52 -04:00
practicalswift
ffa2221256 tests: Pass fuzzing inputs as constant references 2019-10-07 08:48:35 +00:00
Franck Royer
c640ca24f9 Always generate bitcoinstrings.cpp on make translate
`bitcoinstrings.cpp` is to be generated at release time. Hence,
it should not depend on whether the source files are younger as the
releaser may proceed from a fresh checkout.
2019-10-07 15:38:07 +11:00
Hennadii Stepanov
30fc1a3f54 build: Remove workaround for ancient libtool
Since libtool 1.5.2, on Linux libtool no longer sets RPATH for any
directories in the dynamic linker search path, so there is no longer an
issue.
This commit reverts a98356fee8.
2019-10-06 15:18:08 +03:00
Hennadii Stepanov
6ca01b9a10 build: Ensure a minimal version of libtool 2019-10-06 15:17:29 +03:00
John Bampton
8acd58927a Fix Python Docstring to include all Args. 2019-10-06 10:37:50 +10:00
Hennadii Stepanov
c2bb3919a8 util: Simplify path argument for CBlockTreeDB ctor
This commit does not change behavior as GetBlocksDir() with unset 
"-blocksdir" returns the same path.
2019-10-05 09:38:56 +03:00
fanquake
7b701fef58 Merge #17049: contrib: Bump gitian descriptors for 0.20
fa1ad8f06e build: Bump gitian descriptor versions (MarcoFalke)

Pull request description:

  Bump the gitian descriptor versions as a follow-up to #17007.

  Also fixes #17027 with a cherry-pick, and bump the manpages.

ACKs for top commit:
  fanquake:
    ACK fa1ad8f06e

Tree-SHA512: c3b669c3797e5febb51a8dd01e2621a7544a291e080d73c47a2a12ea9da84ff904533e68792e2e869ebbdc2226b2fee7517214549e6cc7e988f175098f7c412c
2019-10-04 14:39:59 -04:00
fanquake
917d3d9052 Merge #17052: scripts: update copyright_header script to include additional files
dffae5a5ad scripts: update copyright_header script to include additional files (gchuf)

Pull request description:

  Includes .sh and .bash-completion files in the script as well

ACKs for top commit:
  MarcoFalke:
    ACK dffae5a5ad
  fanquake:
    ACK dffae5a5ad

Tree-SHA512: b1372bc802b6613e3d6362819fefe7e2a9e656dc483238a626448e646ad57071297de108f89ecb7a71b0bcd49f8e6a2f7599f1dd7798a886872b6755de5d2ccf
2019-10-04 14:23:54 -04:00
fanquake
94d6a18f23 Merge #16507: feefilter: Compute the absolute fee rather than stored rate
eb7b781659 modify p2p_feefilter test to catch rounding error (Gregory Sanders)
6a51f79517 Disallow implicit conversion for CFeeRate constructor (Gregory Sanders)
8e59af55aa feefilter: Compute the absolute fee rather than stored rate to match mempool acceptance logic (Gregory Sanders)

Pull request description:

  This means we will use the rounding-down behavior in `GetFee` to match both mempool acceptance and wallet logic, with minimal changes.

  Fixes https://github.com/bitcoin/bitcoin/issues/16499

  Replacement PR for https://github.com/bitcoin/bitcoin/pull/16500

ACKs for top commit:
  ajtowns:
    ACK eb7b781659 code review only
  naumenkogs:
    utACK eb7b781659
  achow101:
    re ACK eb7b781659
  promag:
    ACK eb7b781659.

Tree-SHA512: 484a11c8f0e825f0c983b1f7e71cf6252b1bba6858194abfe4c088da3bae8a418ec539ef6c4181bf30940e277a95c08d493595d59dfcc6ddf77c65b05563dd7e
2019-10-04 13:46:45 -04:00
gchuf
dffae5a5ad scripts: update copyright_header script to include additional files
Includes .sh and .bash-completion files in the script as well
2019-10-04 17:47:55 +02:00
MarcoFalke
fa1ad8f06e build: Bump gitian descriptor versions 2019-10-04 09:17:57 -04:00
fanquake
94e6e9f38d Merge #17026: doc: Update bips.md for default bech32 addresses in 0.20.0
fad1dbda98 doc: Fix amount typo in release notes (MarcoFalke)
fa8d052da5 doc: Update bips.md for default bech32 addresses in 0.20.0 (MarcoFalke)
fab68a0772 doc: move-only release notes fragments for 0.20.0 (MarcoFalke)

Pull request description:

  * Restore release notes skeleton (headers)
  * Move-only all release notes fragments to the main file for 0.20.0
  * Update bips.md for the default bech32 address change

ACKs for top commit:
  fanquake:
    ACK fad1dbda98

Tree-SHA512: c3d5e124fe803e809d5af869d89de2d8312d4481d43745c1492c66f5ffc6ffc9c37581cc9d8773d2fb711a5a4a47a12aa1ea31292ac6c393ef8503c9db053fc9
2019-10-04 06:50:45 -04:00
fanquake
8ddc303931 Merge #17031: gui: Prevent processing duplicate payment requests
3f89e1eb23 Prevent processing duplicate payment requests (João Barbosa)

Pull request description:

  Considering the following from Qt [src/plugins/platforms/cocoa/qcocoaapplicationdelegate.mm#L267](13e0a36626/src/plugins/platforms/cocoa/qcocoaapplicationdelegate.mm (L267))

  ```cpp
  - (void)application:(NSApplication *)sender openFiles:(NSArray *)filenames
  {
      Q_UNUSED(filenames);
      Q_UNUSED(sender);

      for (NSString *fileName in filenames) {
          QString qtFileName = QString::fromNSString(fileName);
          if (inLaunch) {
              // We need to be careful because Cocoa will be nice enough to take
              // command line arguments and send them to us as events. Given the history
              // of Qt Applications, this will result in behavior people don't want, as
              // they might be doing the opening themselves with the command line parsing.
              if (qApp->arguments().contains(qtFileName))
                  continue;
          }
          QWindowSystemInterface::handleFileOpenEvent(qtFileName);
      }
  ```

  And that a2714a5c69 was merged, now Qt isn't able to filter out the above notifications, and then a [QFileOpenEvent](https://doc.qt.io/qt-5/qfileopenevent.html) event is delivered to `PaymentServer::eventFilter`, which in turn (re)adds the payment request.

  This change fixes #17025, but makes sense regardless of the issue.

ACKs for top commit:
  laanwj:
    Nah, this seems fine, utACK 3f89e1eb23
  Sjors:
    ACK 3f89e1e on macOS 10.14.6
  achow101:
    Code review ACK 3f89e1eb23

Tree-SHA512: dd1e0c73fd84953418173ca71f6f5a67ad74a5dc7e3b1d54915ef0545f513df6a24f27242a77bb094e2833a478e2f3bf30ecd50251f3c55b65e780097cb8ab4d
2019-10-04 06:19:08 -04:00
MarcoFalke
f1f284aa75 Merge #17042: contrib: fix minor typos in makeseeds.py
0b1dcd32bf contrib: fix minor typos in makeseeds.py (Brian Solon)

Pull request description:

  Fixes minor typos referenced in #17020

ACKs for top commit:
  laanwj:
    Thanks. ACK 0b1dcd32bf
  practicalswift:
    ACK 0b1dcd32bf

Tree-SHA512: 1b67358790b3e80e9229d42e399d9ddb21e9f9a0989e05fa6851925ef76289d123cbf648944ff2b60e1513c755023ddec3a0dd3b9ad26d3e105d1d88c937f968
2019-10-03 16:01:40 -04:00
Wladimir J. van der Laan
c90ce0f709 Merge #17022: doc: move-only: Steps for "before major release branch-off"
faca1c24f9 doc: move-only: Steps for "before major release branch-off" (MarcoFalke)

Pull request description:

  The chainparams are updated before branch-off, so that the master branch has the bumped values as well

ACKs for top commit:
  laanwj:
    ACK faca1c24f9

Tree-SHA512: ffc3ea49f0f6dc64dd9bea958e12ebc058496291c1c06d02994b3bf1751602e7c5000fd5eda166fcdbf9ba6d593e19731e93342dd8f2fe410f656a798bef459c
2019-10-03 20:56:13 +02:00
Wladimir J. van der Laan
4715037796 Merge #17038: Don't rename main thread at process level
07e4bdba3b Don't rename main thread at process level (Wladimir J. van der Laan)

Pull request description:

  Set only the internal name for the main threads.

  Fixes #17036 for both `bitcoind` and `bitcoin-qt`.

  After this, e.g. `killall` works again for either.

ACKs for top commit:
  promag:
    Tested ACK 07e4bdba3b, `killall bitcoind` and `killall bitcoin-qt` now just works!
  jonatack:
    ACK 07e4bdba3b `killall bitcoind` shuts down bitcoind mainnet/testnet/regtest, `killall bitcoin-qt` shuts down `./src/qt/bitcoin-qt`, tests pass, very light code review. Good idea to add the `@note` warning. Thanks!

Tree-SHA512: 8f310ae646c83a02de7cc6869aa9aca1d53613d8fb762d05e3dfa52e17ca82abeb99044564cf7ba45b3c4b320e65bf8315d0e8834a9e696f097be5af638c6fd9
2019-10-03 20:51:00 +02:00
Brian Solon
0b1dcd32bf contrib: fix minor typos in makeseeds.py
See #17020
2019-10-03 14:50:08 -04:00
MarcoFalke
fad1dbda98 doc: Fix amount typo in release notes
See https://github.com/bitcoin/bitcoin/pull/16524#issuecomment-538070291
2019-10-03 14:38:54 -04:00
Gregory Sanders
eb7b781659 modify p2p_feefilter test to catch rounding error 2019-10-03 14:03:27 -04:00
Gregory Sanders
6a51f79517 Disallow implicit conversion for CFeeRate constructor 2019-10-03 14:03:27 -04:00
Gregory Sanders
8e59af55aa feefilter: Compute the absolute fee rather than stored rate to match mempool acceptance logic 2019-10-03 14:03:27 -04:00
MarcoFalke
284cd3195a Merge #17040: ci: Make apt-get more verbose, to debug travis timeouts
fa95503d64 ci: Make apt-get more verbose, to debug travis timeouts (MarcoFalke)

Pull request description:

  See issue #16148

ACKs for top commit:
  practicalswift:
    ACK fa95503d64
  ryanofsky:
    utACK fa95503d64

Tree-SHA512: a81e38490dcc9f281f43a6d75ecf383a96374ddee3b4b0f909e485ed461159132d19d37cf9c9d92e95060aec75a5e856900083fd659baf711e438eb9719c96c6
2019-10-03 11:41:25 -04:00
MarcoFalke
fa95503d64 ci: Make apt-get more verbose, to debug travis timeouts 2019-10-03 10:50:33 -04:00
Wladimir J. van der Laan
07e4bdba3b Don't rename main thread at process level
Set only the internal name.

Fixes #17036 for both `bitcoind` and `bitcoin-qt`.
2019-10-03 06:58:47 +02:00
João Barbosa
3f89e1eb23 Prevent processing duplicate payment requests 2019-10-02 21:55:52 +01:00
Carl Dong
93cb974980 gitian: Use split-debug.sh for Win builds
Debug splitting was first introduced in 7e7eb2724, then gitian-linux.yml
changed to using split-debug.sh in 9d2536208. Here we change
gitian-win.yml to use split-debug.sh as well.
2019-10-02 16:21:22 -04:00
Hennadii Stepanov
86b9f92da2 doc: Add detailed info about Bitcoin Core files 2019-10-02 21:50:58 +03:00
MarcoFalke
fa8d052da5 doc: Update bips.md for default bech32 addresses in 0.20.0 2019-10-02 14:26:21 -04:00
MarcoFalke
fab68a0772 doc: move-only release notes fragments for 0.20.0 2019-10-02 14:25:07 -04:00
MarcoFalke
a689c11907 Merge #16524: Wallet: Disable -fallbackfee by default
ea4cc3a7b3 Truly decouple wallet from chainparams for -fallbackfee (Jorge Timón)

Pull request description:

  Before it was 0 by default for main and 20000 for test and regtest.
  Now it is 0 by default for all chains, thus there's no need to call Params().

  Also now the default for main is properly documented.

  Suggestion for release notes:

  -fallbackfee was 0 (disabled) by default for the main chain, but 20000 by default for the test chains. Now it is 0 by default for all chains. Testnet and regtest users will have to add fallbackfee=20000 to their configuration if they weren't setting it and they want it to keep working like before.

  Should I propose them to the wiki for the release notes or only after merge?

  For more context, see https://github.com/bitcoin/bitcoin/pull/16402#issuecomment-515701042

ACKs for top commit:
  MarcoFalke:
    ACK ea4cc3a7b3

Tree-SHA512: fdfaba5d813da4221e405e0988bef44f3856d10f897a94f9614386d14b7716f4326ab8a6646e26d41ef3f4fa61b936191e216b1b605e9ab0520b0657fc162e6c
2019-10-02 13:42:57 -04:00
Wladimir J. van der Laan
f4a0d27e85 Merge #13716: bitcoin-cli: -stdinwalletpassphrase and non-echo stdin passwords
50c4afa3c4 add newline after -stdin* (Karl-Johan Alm)
7f11fba2e3 cli: add -stdinwalletpassphrase for (slightly more) secure CLI (Karl-Johan Alm)
0da503e947 add stdin helpers for password input support (Karl-Johan Alm)

Pull request description:

  This PR
  * adds `-stdinwalletpassphrase` for use with `walletpasshprase(change)`
  * adds no-echo for passwords (`-stdinrpcpass` and above)

  It may not be ideal, but it's better than having to clear the screen whenever you unlock the wallet.

ACKs for top commit:
  laanwj:
    code review ACK 50c4afa3c4

Tree-SHA512: 473db8a303ff360ffaa36ac81a2f82be2136fa82696df0bc4f33cb44033a3ae258b5aa5bbcc1f101f88ae9abe9598ed564ce52877ab139bd5d709833f5275ec6
2019-10-02 18:37:14 +02:00
Jorge Timón
ea4cc3a7b3 Truly decouple wallet from chainparams for -fallbackfee
Before it was 0 by default for main and 20000 for test and regtest.
Now it is 0 by default for all chains, thus there's no need to call Params().

Also now the default for main is properly documented
2019-10-02 18:10:07 +02:00
Wladimir J. van der Laan
fecc1be231 Merge #16884: wallet: Change default address type to bech32
71d4eddf42 Add release note for bech32 by default in wallet (Gregory Sanders)
b34f0180e3 Revert "gui: Generate bech32 addresses by default (take 2, fixup)" (Gregory Sanders)
f50785ab56 Change default address type to bech32 (Gregory Sanders)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    re-ACK 71d4eddf42 (only change is restore mimick behavior)
  laanwj:
    ACK 71d4eddf42

Tree-SHA512: 3c49a1b51c49f3a762ad08985167ca1b89b0177ae20ab6d5883f1f74dde7a155921c1b855a842199bbf32f563c56b33f8b603bc842637bdcb121001023d454b6
2019-10-02 17:46:01 +02:00
Wladimir J. van der Laan
ccaef6c28b Merge #16908: txmempool: Make entry time type-safe (std::chrono)
faec689bed txmempool: Make entry time type-safe (std::chrono) (MarcoFalke)
faaa1f01da util: Add count_seconds time helper (MarcoFalke)
1111170f2f test: mempool entry time is persisted (MarcoFalke)

Pull request description:

  This changes the type of the entry time of txs into the mempool from `int64_t` to `std::chrono::seconds`.

  The benefits:
  * Documents the type for developers
  * Type violations result in compile errors
  * After compilation, the two are equivalent (at no run time cost)

ACKs for top commit:
  ajtowns:
    utACK faec689bed
  laanwj:
    ACK faec689bed

Tree-SHA512: d958e058755d1a1d54cef536a8b30a11cc502b7df0d6ecf84a0ab1d38bc8105a67668a99cd5087a444f6de2421238111c5fca133cdf8e2e2273cb12cb6957845
2019-10-02 16:55:36 +02:00
MarcoFalke
19ba43ae2d Merge #16964: gui: Change sendcoins dialogue Yes to Send
a649cc6a17 Change sendcoins dialogue Yes to Send (Gregory Sanders)

Pull request description:

  It's more self-explanatory, matches "cancel" better, and makes future extensions such as https://github.com/bitcoin/bitcoin/pull/16944 more directly understandable to the user.

ACKs for top commit:
  Sjors:
    Trivial code review ACK a649cc6. I also used Send in #16966 (`ui - make send a wizard`)
  laanwj:
    ACK a649cc6a17
  jonatack:
    Code review ACK a649cc6a17

Tree-SHA512: fe4993bc7ac653d28f3d399ade046bcfd405511aec06ff041bb5aef47e0736faf3e3112a6db660cd761af56392dc6b97f2c2341ed3eff4490079c5eb8a0d465a
2019-10-02 10:48:04 -04:00
MarcoFalke
fa25f43ac5 p2p: Remove BIP61 reject messages 2019-10-02 10:39:14 -04:00
Wladimir J. van der Laan
ecc0f272a0 Merge #17007: build: 0.19 release updates on master
434101875c doc: reset release notes after 0.19 split-off (Jon Atack)
c0859b7dac build: 0.19 release updates on master (Jon Atack)

Pull request description:

  Post split-off. As per https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-every-major-release and  issue #16996.

  Note: after split-off, the same changes should be made on the new 0.19.0 release branch, with also these additional changes to both files (configure.ac and build_msvc/bitcoin_config.h):

    - set `CLIENT_VERSION_REVISION` to `0`
    - set `CLIENT_VERSION_IS_RELEASE` to `true`

  The second commit resets the release notes after branch-off and proposes a few improvements.

ACKs for top commit:
  laanwj:
    ACK 434101875c

Tree-SHA512: 5a6afeb9cff6fa827865894cc7d3dc789db1c8b5d875ba49fdcfd9fd48af9d2d2864f49a992988136425744af74053cb57a4a92a1665a09b194eecb1a2972315
2019-10-02 16:20:02 +02:00
MarcoFalke
ddddd8961b ci: Use busybox utils for one build 2019-10-02 10:01:04 -04:00
MarcoFalke
faca1c24f9 doc: move-only: Steps for "before major release branch-off"
Can be reviewed with the git diff option

--color-moved=dimmed-zebra
2019-10-02 09:36:49 -04:00
Jon Atack
434101875c doc: reset release notes after 0.19 split-off
- empty release-notes.md and propose a few improvements

- delete release notes fragments
2019-10-02 11:17:20 +02:00
Jon Atack
c0859b7dac build: 0.19 release updates on master
as per doc/release-process.md.

Note: On branch-off, these same changes should be made on the release branch, with also these additional changes to both files:

  - set `CLIENT_VERSION_REVISION` to `0`
  - set `CLIENT_VERSION_IS_RELEASE` to `true`
2019-10-02 11:08:15 +02:00
João Barbosa
091747b46e gui: Add shortcuts for tab tools 2019-10-02 01:28:16 +01:00
Karl-Johan Alm
50c4afa3c4 add newline after -stdin* 2019-10-01 12:19:31 +09:00
Karl-Johan Alm
7f11fba2e3 cli: add -stdinwalletpassphrase for (slightly more) secure CLI 2019-10-01 12:19:31 +09:00
Karl-Johan Alm
0da503e947 add stdin helpers for password input support 2019-10-01 12:19:31 +09:00
Tim Akinbo
3284e6c09a scripts: search for next position of magic bytes rather than fail
document seek method for next position of magic bytes
2019-09-30 18:10:47 +00:00
Martin Zumsande
d478a472eb test: Fix combine_logs.py for AppVeyor build 2019-09-27 12:40:20 +02:00
Gregory Sanders
71d4eddf42 Add release note for bech32 by default in wallet 2019-09-26 16:23:32 -04:00
Gregory Sanders
b34f0180e3 Revert "gui: Generate bech32 addresses by default (take 2, fixup)"
This reverts commit fa5a4cd813.
2019-09-26 16:23:32 -04:00
Gregory Sanders
f50785ab56 Change default address type to bech32 2019-09-26 16:23:32 -04:00
James O'Beirne
0d86f4d3da refactor: consolidate PASTE macros 2019-09-25 10:56:00 -04:00
Gregory Sanders
a649cc6a17 Change sendcoins dialogue Yes to Send 2019-09-25 09:47:56 -04:00
Jon Layton
15ac916642 doc: Doxygen-friendly descriptor.h comments 2019-09-24 01:14:20 -04:00
Igor Cota
e4c319e8a1 builds: remove superfluous config_opts_aarch64_android 2019-09-23 14:44:49 +02:00
MarcoFalke
faec689bed txmempool: Make entry time type-safe (std::chrono) 2019-09-23 08:00:14 -04:00
fanquake
bd3f5a90ec build: remove mingw linker workaround from win gitian descriptor
This workaround was added as part of the switch to gitian building using Ubuntu 14.04 (#6900).
However, it should no longer be required, as we have switched to Bionic (#13171), and that
has a far newer version of binutils.

binutils patch: https://sourceware.org/bugzilla/show_bug.cgi?id=16192
2019-09-23 08:38:42 +08:00
Igor Cota
24ffef0c27 Patch libevent when building for Android (fix arc4random_addrandom) 2019-09-22 18:33:51 +02:00
BlockMechanic
f1e40b3e71 Update bitcoin_qt.m4
Fix to allow configure to detect at
2019-09-22 14:40:18 +02:00
Igor Cota
b4057d8261 Define TARGET_OS when host is android 2019-09-22 14:35:41 +02:00
Craig Andrews
9a19c9ada5 Always define the raii_event_tests test suite
The test suite must always be defined (even when EVENT_SET_MEM_FUNCTIONS_IMPLEMENTED is not defined) so that the test harness doesn't fail due to not being able to find the raii_event_tests test.

This improves upon 95f97f4 actually fixing https://github.com/bitcoin/bitcoin/issues/9493
2019-09-20 19:15:40 +00:00
Andrew Chow
cd68594dcd Only check the hash of transactions loaded from disk 2019-09-20 10:14:03 -04:00
MarcoFalke
faaa1f01da util: Add count_seconds time helper 2019-09-19 11:30:29 -04:00
MarcoFalke
1111170f2f test: mempool entry time is persisted
Also, remove the now unused "Mine a single block to get out of IBD".
This was fixed in commit 1111aecbb5.
2019-09-19 11:30:28 -04:00
Igor Cota
80b475f159 Fix Android zlib cross compilation issue (https://stackoverflow.com/questions/21396988/zlib-build-not-configuring-properly-with-cross-compiler-ignores-ar) 2019-09-19 11:43:10 +02:00
Igor Cota
45f8219015 Add full Android build example command and instructions on getting SDK/NDK 2019-09-19 11:27:18 +02:00
Igor Cota
b68f2a68c2 Add config opts and patch for aarch64_android build of Qt 2019-09-19 11:27:18 +02:00
Igor Cota
9c4cb0166e Add ranlib to android.mk hosts file (fix OSX Android NDK build) 2019-09-19 11:27:18 +02:00
Igor Cota
c2a749c9c1 Add example Android host-platform-triplet and options 2019-09-19 11:27:18 +02:00
Igor Cota
0b0cff3c61 Add support for building Android dependencies 2019-09-19 11:27:18 +02:00
Pieter Wuille
5e69aeec3f Add documenting comments to spanparsing.h 2019-09-18 12:25:55 -07:00
Pieter Wuille
230d43fdbc Abstract out some of the descriptor Span-parsing helpers 2019-09-18 12:12:13 -07:00
Sebastian Falbesoner
a57a1d42d5 test: add unit test for wallet watch-only methods involving PubKeys
The motivation for this addition was to unit test the function
wallet.cpp:ExtractPubKey() (see recent change in commit
798a589aff) which is however static and only
indirectly available via the public methods AddWatchOnly(), LoadWatchOnly() and
RemoveWatchOnly(). Since the first of those methods also stores the addresses
to the disk, the second, simpler one was chosen which only operates in memory.

test: add missing wallet lock for test case WatchOnlyPubKeys

test: test case WatchOnlyPubKeys, suggested review changes by instagibbs

test: test case WatchOnlyPubKeys, suggested review changes by achow101

test: test case WatchOnlyPubKeys, s/isPubKeyFullyValid/is_pubkey_fully_valid
2019-09-16 23:23:48 +02:00
setpill
7fb7acfc20 Set init stop timeout to 10 min
`bitcoind` can take a long time to flush its db cache to disk upon
shutdown. Most init files send a `SIGKILL` after a timeout of 1 minute,
causing unclean shutdowns and triggering a long "Rolling forward" at the
next startup. Increasing this timeout to 10 minutes should reduce how
often this occurs, especially during IBD.

fixup! Set ProtectHome in systemd service file
2019-09-06 17:03:04 +02:00
Matt Corallo
e80259f197 Additionally treat Tx.nVersion as unsigned in joinpsbts
This gets its own release note callout, though doesn't appear to
violate the BIP as the BIP appears to be underspecified. We
probably want to update BIP 174 to mention how version numbers are
combined.
2019-09-03 10:53:45 -04:00
Matt Corallo
970de70bdd Dump transaction version as an unsigned integer in RPC/TxToUniv
Consensus-wise we already treat it as an unsigned integer (the
only rules around it are in CSV/locktime handling), but changing
the underlying data type means touching consensus code for a
simple cleanup change, which isn't really worth it.

See-also, https://github.com/rust-bitcoin/rust-bitcoin/pull/299
2019-08-02 15:37:13 -04:00
Ben Woosley
0580f86bb4 Fixup whitespace 2019-03-04 01:30:36 -08:00
Ben Woosley
47101bbb27 scripted-diff: Rename CPubKey and CKey::*_KEY_SIZE and COMPRESSED_*_KEY_SIZE
To SIZE and COMPRESSED_SIZE

-BEGIN VERIFY SCRIPT-
sed -i 's/PRIVATE_KEY_SIZE/SIZE/g' src/*.h src/*.cpp src/**/*.h src/**/*.cpp
sed -i 's/COMPRESSED_PRIVATE_KEY_SIZE/COMPRESSED_SIZE/g' src/*.h src/**/*.cpp src/**/*.h src/**/*.cpp
sed -i 's/PUBLIC_KEY_SIZE/SIZE/g' src/*.h src/*.cpp src/**/*.h src/**/*.cpp
sed -i 's/COMPRESSED_PUBLIC_KEY_SIZE/COMPRESSED_SIZE/g' src/*.h src/*.cpp src/**/*.h src/**/*.cpp
-END VERIFY SCRIPT-
2019-03-04 01:30:29 -08:00
Michael Polzer
c119ba3c9b [doc] clarify getdata limit after #14897
GETDATA is limited to blocks and transactions now and can't be used for other non-block data
2019-02-20 21:00:04 +01:00
Gregory Sanders
ec30a79f1c Fix UB with bench on genesis block 2019-01-29 13:03:53 +00:00
Hennadii Stepanov
11fdfcf7f9 Show addresses for "SendToSelf" transactions
Change addresses are not recognized.
2019-01-06 11:52:37 +02:00
Luke Dashjr
f33efa8ec5 GUI: Restore RPC Console to non-wallet tray icon menu 2018-12-21 22:14:09 +00:00
mruddy
c276df7759 zmq: enable tcp keepalive 2018-11-15 08:16:25 -05:00
1927 changed files with 259449 additions and 176714 deletions

View File

@@ -1,70 +1,63 @@
version: '{branch}.{build}'
skip_tags: true
image: Visual Studio 2017
image: Visual Studio 2019
configuration: Release
platform: x64
clone_depth: 5
environment:
APPVEYOR_SAVE_CACHE_ON_ERROR: true
CLCACHE_SERVER: 1
PACKAGES: berkeleydb boost-filesystem boost-signals2 boost-test libevent openssl rapidcheck zeromq double-conversion
PATH: 'C:\Python37-x64;C:\Python37-x64\Scripts;%PATH%'
PYTHONUTF8: 1
QT_DOWNLOAD_URL: 'https://github.com/sipsorcery/qt_win_binary/releases/download/v1.0/Qt5.9.7_ssl_x64_static_vs2017.zip'
QT_DOWNLOAD_HASH: 'D4D35B8112302B67E5610A03421BB3E43FE13F14D9A5F637C22AE60DCEC0E0F5'
QT_LOCAL_PATH: 'C:\Qt5.9.7_ssl_x64_static_vs2017'
cache:
- C:\tools\vcpkg\installed
- C:\Users\appveyor\clcache -> .appveyor.yml, build_msvc\**, **\Makefile.am, **\*.vcxproj.in
- C:\Qt5.9.7_ssl_x64_static_vs2017
QT_DOWNLOAD_URL: 'https://github.com/sipsorcery/qt_win_binary/releases/download/qt51211x64_static_vs2019_16101/Qt5.12.11_x64_static_vs2019_16101.zip'
QT_DOWNLOAD_HASH: 'cf1b58107fadbf0d9a957d14dab16cde6b6eb6936a1908472da1f967dda34a3a'
QT_LOCAL_PATH: 'C:\Qt5.12.11_x64_static_vs2019_16101'
VCPKG_TAG: '75522bb1f2e7d863078bcd06322348f053a9e33f'
install:
- cmd: pip install --quiet git+https://github.com/frerich/clcache.git@v4.2.0
# Disable zmq test for now since python zmq library on Windows would cause Access violation sometimes.
# - cmd: pip install zmq
- cmd: echo set(VCPKG_BUILD_TYPE release) >> C:\tools\vcpkg\triplets\%PLATFORM%-windows-static.cmake
- cmd: vcpkg remove --outdated --recurse
- cmd: vcpkg install --triplet %PLATFORM%-windows-static %PACKAGES% > NUL
before_build:
- ps: clcache -M 536870912
# The powershell block below is to set up vcpkg to install the c++ dependencies. The pseudo code is:
# a. Checkout the vcpkg source (including port files) for the specific checkout and build the vcpkg binary,
# b. Append a setting to the vcpkg cmake config file to only do release builds of dependencies (skipping deubg builds saves ~5 mins).
# Note originally this block also installed the dependencies using 'vcpkg install'. Dependencies are now installed
# as part of the msbuild command using vcpkg mainfests.
- ps: |
if(!(Test-Path -Path ($env:QT_LOCAL_PATH))) {
Write-Host "Downloading Qt binaries.";
Invoke-WebRequest -Uri $env:QT_DOWNLOAD_URL -Out qtdownload.zip;
Write-Host "Qt binaries successfully downloaded, checking hash against $env:QT_DOWNLOAD_HASH...";
if((Get-FileHash qtdownload.zip).Hash -eq $env:QT_DOWNLOAD_HASH) {
Expand-Archive qtdownload.zip -DestinationPath $env:QT_LOCAL_PATH;
Write-Host "Qt binary download matched the expected hash.";
}
else {
Write-Host "ERROR: Qt binary download did not match the expected hash.";
Exit-AppveyorBuild;
}
cd c:\tools\vcpkg
$env:GIT_REDIRECT_STDERR = '2>&1' # git is writing non-errors to STDERR when doing git pull. Send to STDOUT instead.
git -c advice.detachedHead=false checkout $env:VCPKG_TAG
.\bootstrap-vcpkg.bat > $null
Add-Content "C:\tools\vcpkg\triplets\$env:PLATFORM-windows-static.cmake" "set(VCPKG_BUILD_TYPE release)"
cd "$env:APPVEYOR_BUILD_FOLDER"
before_build:
# Powershell block below is to download and extract the Qt static libraries. The pseudo code is:
# a. Download the zip file with the prebuilt Qt static libraries.
# b. Check that the downloaded file matches the expected hash.
# c. Extract the zip file to the specific destination path expected by the msbuild projects.
- ps: |
Write-Host "Downloading Qt binaries.";
Invoke-WebRequest -Uri $env:QT_DOWNLOAD_URL -Out qtdownload.zip;
Write-Host "Qt binaries successfully downloaded, checking hash against $env:QT_DOWNLOAD_HASH...";
if((Get-FileHash qtdownload.zip).Hash -eq $env:QT_DOWNLOAD_HASH) {
Expand-Archive qtdownload.zip -DestinationPath $env:QT_LOCAL_PATH;
Write-Host "Qt binary download matched the expected hash.";
}
else {
Write-Host "Qt binaries already present.";
Write-Host "ERROR: Qt binary download did not match the expected hash.";
Exit-AppveyorBuild;
}
- cmd: python build_msvc\msvc-autogen.py
- ps: $files = (Get-ChildItem -Recurse | where {$_.extension -eq ".vcxproj"}).FullName
- ps: for (${i} = 0; ${i} -lt ${files}.length; ${i}++) {
${content} = (Get-Content ${files}[${i}]);
${content} = ${content}.Replace("</RuntimeLibrary>", "</RuntimeLibrary><DebugInformationFormat>None</DebugInformationFormat>");
${content} = ${content}.Replace("<WholeProgramOptimization>true", "<WholeProgramOptimization>false");
Set-Content ${files}[${i}] ${content};
}
- ps: Start-Process clcache-server
- ps: fsutil behavior set disablelastaccess 0 # Enable Access time feature on Windows (for clcache)
build_script:
- cmd: msbuild /p:TrackFileAccess=false /p:CLToolExe=clcache.exe build_msvc\bitcoin.sln /m /v:n /nologo
- cmd: msbuild /p:TrackFileAccess=false build_msvc\bitcoin.sln /m /v:q /nologo
after_build:
- ps: fsutil behavior set disablelastaccess 1 # Disable Access time feature on Windows (better performance)
- ps: clcache -z
#- 7z a bitcoin-%APPVEYOR_BUILD_VERSION%.zip %APPVEYOR_BUILD_FOLDER%\build_msvc\%platform%\%configuration%\*.exe
test_script:
- cmd: src\test_bitcoin.exe -k stdout -e stdout 2> NUL
- cmd: src\bench_bitcoin.exe -evals=1 -scaling=0 > NUL
- cmd: src\test_bitcoin.exe -l test_suite
- cmd: src\bench_bitcoin.exe > NUL
- ps: python test\util\bitcoin-util-test.py
- cmd: python test\util\rpcauth-test.py
- cmd: python test\functional\test_runner.py --ci --quiet --combinedlogslen=4000 --failfast
# Fee estimation test failing on appveyor with: WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted.
# functional tests disabled for now. See
# https://github.com/bitcoin/bitcoin/pull/18626#issuecomment-613396202
# https://github.com/bitcoin/bitcoin/issues/18623
# - cmd: python test\functional\test_runner.py --ci --quiet --combinedlogslen=4000 --failfast --exclude feature_fee_estimation
artifacts:
#- path: bitcoin-%APPVEYOR_BUILD_VERSION%.zip
deploy: off

View File

@@ -1,55 +1,222 @@
task:
name: "FreeBsd 12.0 amd64 [GOAL: install] [no depends, only system libs]"
freebsd_instance:
image: freebsd-12-0-release-amd64
cpu: 8
memory: 8G
timeout_in: 60m
env:
MAKEJOBS: "-j9"
CONFIGURE_OPTS: "--disable-dependency-tracking"
GOAL: "install"
TEST_RUNNER_PORT_MIN: "14000" # Must be larger than 12321, which is used for the http cache. See https://cirrus-ci.org/guide/writing-tasks/#http-cache
CCACHE_SIZE: "200M"
CCACHE_COMPRESS: 1
CCACHE_DIR: "/tmp/ccache_dir"
ccache_cache:
folder: "/tmp/ccache_dir"
install_script:
- pkg install -y autoconf automake boost-libs git gmake libevent libtool openssl pkgconf python3 ccache
- ./contrib/install_db4.sh $(pwd)
- ccache --max-size=${CCACHE_SIZE}
configure_script:
- ./autogen.sh
- ./configure ${CONFIGURE_OPTS} BDB_LIBS="-L$(pwd)/db4/lib -ldb_cxx-4.8" BDB_CFLAGS="-I$(pwd)/db4/include" || ( cat config.log && false)
make_script:
- gmake ${MAKEJOBS} ${GOAL} || ( echo "Build failure. Verbose build follows." && gmake ${GOAL} V=1 ; false )
check_script:
- gmake check ${MAKEJOBS} VERBOSE=1
functional_test_script:
- ./test/functional/test_runner.py --jobs 9 --ci --extended --exclude feature_dbcrash --combinedlogslen=1000 --quiet --failfast
task:
name: "x86_64 Linux [GOAL: install] [bionic] [Using ./ci/ system]"
### Global defaults
env:
PACKAGE_MANAGER_INSTALL: "apt-get update && apt-get install -y"
MAKEJOBS: "-j4"
TEST_RUNNER_PORT_MIN: "14000" # Must be larger than 12321, which is used for the http cache. See https://cirrus-ci.org/guide/writing-tasks/#http-cache
CCACHE_SIZE: "200M"
CCACHE_DIR: "/tmp/ccache_dir"
CCACHE_NOHASHDIR: "1" # Debug info might contain a stale path if the build dir changes, but this is fine
cirrus_ephemeral_worker_template_env: &CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
DANGER_RUN_CI_ON_HOST: "1" # Containers will be discarded after the run, so there is no risk that the ci scripts modify the system
persistent_worker_template_env: &PERSISTENT_WORKER_TEMPLATE_ENV
RESTART_CI_DOCKER_BEFORE_RUN: "1"
persistent_worker_template: &PERSISTENT_WORKER_TEMPLATE
persistent_worker: {} # https://cirrus-ci.org/guide/persistent-workers/
# https://cirrus-ci.org/guide/tips-and-tricks/#sharing-configuration-between-tasks
base_template: &BASE_TEMPLATE
skip: $CIRRUS_REPO_FULL_NAME == "bitcoin-core/gui" && $CIRRUS_PR == "" # No need to run on the read-only mirror, unless it is a PR. https://cirrus-ci.org/guide/writing-tasks/#conditional-task-execution
merge_base_script:
- if [ "$CIRRUS_PR" = "" ]; then exit 0; fi
- bash -c "$PACKAGE_MANAGER_INSTALL git"
- git fetch $CIRRUS_REPO_CLONE_URL $CIRRUS_BASE_BRANCH
- git config --global user.email "ci@ci.ci"
- git config --global user.name "ci"
- git merge FETCH_HEAD # Merge base to detect silent merge conflicts
stateful: false # https://cirrus-ci.org/guide/writing-tasks/#stateful-tasks
global_task_template: &GLOBAL_TASK_TEMPLATE
<< : *BASE_TEMPLATE
timeout_in: 120m # https://cirrus-ci.org/faq/#instance-timed-out
container:
image: ubuntu:18.04
cpu: 8
memory: 8G
timeout_in: 60m
env:
MAKEJOBS: "-j9"
RUN_CI_ON_HOST: "1"
TEST_RUNNER_PORT_MIN: "14000" # Must be larger than 12321, which is used for the http cache. See https://cirrus-ci.org/guide/writing-tasks/#http-cache
CCACHE_SIZE: "200M"
CCACHE_DIR: "/tmp/ccache_dir"
# https://cirrus-ci.org/faq/#are-there-any-limits
# Each project has 16 CPU in total, assign 2 to each container, so that 8 tasks run in parallel
cpu: 2
memory: 8G # Set to 8GB to avoid OOM. https://cirrus-ci.org/guide/linux/#linux-containers
ccache_cache:
folder: "/tmp/ccache_dir"
depends_built_cache:
folder: "/tmp/cirrus-ci-build/depends/built"
depends_sdk_cache:
folder: "/tmp/cirrus-ci-build/depends/sdk-sources"
install_script:
- apt-get update
- apt-get -y install git bash ccache
- ccache --max-size=${CCACHE_SIZE}
folder: "depends/built"
ci_script:
- ./ci/test_run_all.sh
depends_sdk_cache_template: &DEPENDS_SDK_CACHE_TEMPLATE
depends_sdk_cache:
folder: "depends/sdk-sources"
compute_credits_template: &CREDITS_TEMPLATE
# https://cirrus-ci.org/pricing/#compute-credits
# Only use credits for pull requests to the main repo
use_compute_credits: $CIRRUS_REPO_FULL_NAME == 'bitcoin/bitcoin' && $CIRRUS_PR != ""
#task:
# name: "Windows"
# windows_container:
# image: cirrusci/windowsservercore:2019
# env:
# CIRRUS_SHELL: powershell
# PATH: 'C:\Python37;C:\Python37\Scripts;%PATH%'
# PYTHONUTF8: 1
# QT_DOWNLOAD_URL: 'https://github.com/sipsorcery/qt_win_binary/releases/download/v1.6/Qt5.9.8_x64_static_vs2019.zip'
# QT_DOWNLOAD_HASH: '9a8c6eb20967873785057fdcd329a657c7f922b0af08c5fde105cc597dd37e21'
# QT_LOCAL_PATH: 'C:\Qt5.9.8_x64_static_vs2019'
# VCPKG_INSTALL_PATH: 'C:\tools\vcpkg\installed'
# VCPKG_COMMIT_ID: 'ed0df8ecc4ed7e755ea03e18aaf285fd9b4b4a74'
# install_script:
# - choco install python --version=3.7.7 -y
task:
name: 'lint [bionic]'
<< : *BASE_TEMPLATE
container:
image: ubuntu:bionic # For python 3.6, oldest supported version according to doc/dependencies.md
cpu: 1
memory: 1G
# For faster CI feedback, immediately schedule the linters
<< : *CREDITS_TEMPLATE
lint_script:
- ./ci/lint_run_all.sh
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
task:
name: 'ARM [unit tests, no functional tests] [buster]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: debian:buster
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_arm.sh"
task:
name: 'Win64 [unit tests, no gui tests, no boost::process, no functional tests] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_win64.sh"
task:
name: '32-bit + dash [gui] [CentOS 8]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: centos:8
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
PACKAGE_MANAGER_INSTALL: "yum install -y"
FILE_ENV: "./ci/test/00_setup_env_i686_centos.sh"
task:
name: '[previous releases, uses qt5 dev package and some depends packages, DEBUG] [unsigned char] [bionic]'
previous_releases_cache:
folder: "releases"
<< : *GLOBAL_TASK_TEMPLATE
<< : *PERSISTENT_WORKER_TEMPLATE
env:
<< : *PERSISTENT_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_qt5.sh"
task:
name: '[depends, sanitizers: thread (TSan), no gui] [hirsute]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:hirsute
cpu: 6 # Increase CPU and Memory to avoid timeout
memory: 24G
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_tsan.sh"
task:
name: '[depends, sanitizers: memory (MSan)] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_msan.sh"
task:
name: '[no depends, sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer] [hirsute]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:hirsute
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_asan.sh"
task:
name: '[no depends, sanitizers: fuzzer,address,undefined,integer] [focal]'
only_if: $CIRRUS_BRANCH == $CIRRUS_DEFAULT_BRANCH || $CIRRUS_BASE_BRANCH == $CIRRUS_DEFAULT_BRANCH
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
cpu: 4 # Increase CPU and memory to avoid timeout
memory: 16G
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_fuzz.sh"
task:
name: '[multiprocess, DEBUG] [focal]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
cpu: 4
memory: 16G # The default memory is sometimes just a bit too small, so double everything
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
MAKEJOBS: "-j8"
FILE_ENV: "./ci/test/00_setup_env_native_multiprocess.sh"
task:
name: '[no wallet] [bionic]'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:bionic
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_nowallet.sh"
task:
name: 'macOS 10.14 [gui, no tests] [focal]'
<< : *DEPENDS_SDK_CACHE_TEMPLATE
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_mac.sh"
task:
name: 'macOS 11 native [gui] [no depends]'
brew_install_script:
- brew install boost libevent berkeley-db4 qt@5 miniupnpc libnatpmp ccache zeromq qrencode sqlite libtool automake pkg-config gnu-getopt
<< : *GLOBAL_TASK_TEMPLATE
osx_instance:
# Use latest image, but hardcode version to avoid silent upgrades (and breaks)
image: big-sur-xcode-12.5 # https://cirrus-ci.org/guide/macOS
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
CI_USE_APT_INSTALL: "no"
PACKAGE_MANAGER_INSTALL: "echo" # Nothing to do
FILE_ENV: "./ci/test/00_setup_env_mac_host.sh"
task:
name: 'ARM64 Android APK [focal]'
<< : *DEPENDS_SDK_CACHE_TEMPLATE
depends_sources_cache:
folder: "depends/sources"
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_android.sh"

26
.editorconfig Normal file
View File

@@ -0,0 +1,26 @@
# This is the top-most EditorConfig file.
root = true
# For all files.
[*]
charset = utf-8
end_of_line = lf
indent_style = space
insert_final_newline = true
trim_trailing_whitespace = true
# Source code files
[*.{h,cpp,py,sh}]
indent_size = 4
# .cirrus.yml, .appveyor.yml, .fuzzbuzz.yml, etc.
[*.yml]
indent_size = 2
# Makefiles
[{*.am,Makefile.*.include}]
indent_style = tab
# Autoconf scripts
[configure.ac]
indent_size = 2

View File

@@ -4,7 +4,11 @@ General bitcoin questions and/or support requests are best directed to the Bitco
For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/.
If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue! -->
If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue!
Any report, issue or feature request related to the GUI should be reported at
https://github.com/bitcoin-core/gui/issues/
-->
<!-- Describe the issue -->
<!--- What behavior did you expect? -->

View File

@@ -1,6 +1,6 @@
---
name: Bug report
about: Create a report to help us improve
about: Create a report to help us improve (use this for suspected bugs only, if not sure, open a regular issue below)
title: ''
labels: Bug
assignees: ''

View File

@@ -0,0 +1,22 @@
---
name: Good first issue
about: '(Regular devs only): Suggest a new good first issue'
title: ''
labels: ''
assignees: ''
---
<!-- Needs the label "good first issue" assigned manually before or after opening -->
<!-- A good first issue is an uncontroversial issue, that has a relatively unique and obvious solution -->
<!-- Motivate the issue and explain the solution briefly -->
#### Useful skills:
<!-- (For example, “C++11 std::thread”, “Qt5 GUI and async GUI design” or “basic understanding of Bitcoin mining and the Bitcoin Core RPC interface”.) -->
#### Want to work on this issue?
For guidance on contributing, please read [CONTRIBUTING.md](https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md) before opening your pull request.

11
.github/ISSUE_TEMPLATE/gui_issue.md vendored Normal file
View File

@@ -0,0 +1,11 @@
---
name: An issue or feature request related to the GUI
about: Any report, issue or feature request related to the GUI should be reported at https://github.com/bitcoin-core/gui/issues/
title: Any report, issue or feature request related to the GUI should be reported at https://github.com/bitcoin-core/gui/issues/
labels: GUI
assignees: ''
---
Any report, issue or feature request related to the GUI should be reported at
https://github.com/bitcoin-core/gui/issues/

View File

@@ -3,6 +3,10 @@
Pull requests without a rationale and clear improvement may be closed
immediately.
GUI-related pull requests should be opened against
https://github.com/bitcoin-core/gui
first. See CONTRIBUTING.md
-->
<!--

20
.gitignore vendored
View File

@@ -5,10 +5,13 @@
src/bitcoin
src/bitcoind
src/bitcoin-cli
src/bitcoin-gui
src/bitcoin-node
src/bitcoin-tx
src/bitcoin-util
src/bitcoin-wallet
src/test/fuzz/fuzz
src/test/test_bitcoin
src/test/test_bitcoin_fuzzy
src/qt/test/test_bitcoin-qt
# autoreconf
@@ -28,6 +31,7 @@ build-aux/m4/ltversion.m4
build-aux/missing
build-aux/compile
build-aux/test-driver
config.cache
config.log
config.status
configure
@@ -57,7 +61,7 @@ src/qt/bitcoin-qt.includes
.dirstamp
.libs
.*.swp
*.*~*
*~
*.bak
*.rej
*.orig
@@ -72,6 +76,7 @@ src/qt/bitcoin-qt.includes
*.log
*.trs
*.dmg
*.iso
*.json.h
*.raw.h
@@ -88,7 +93,7 @@ src/qt/bitcoin-qt.includes
*.qm
Makefile
!depends/Makefile
bitcoin-qt
src/qt/bitcoin-qt
Bitcoin-Qt.app
background.tiff*
@@ -106,13 +111,19 @@ qrc_*.cpp
.DS_Store
build
# Previous releases
releases
#lcov
*.gcno
*.gcda
/*.info
test_bitcoin.coverage/
total.coverage/
fuzz.coverage/
coverage_percent.txt
/cov_tool_wrapper.sh
qa-assets/
#build tests
linux-coverage-build
@@ -120,6 +131,7 @@ linux-build
win32-build
test/config.ini
test/cache/*
test/.mypy_cache/
!src/leveldb*/Makefile
@@ -137,3 +149,5 @@ db4/
osx_volname
dist/
*.background.tiff
/guix-build-*

View File

@@ -1 +1 @@
3.5.6
3.6.12

View File

@@ -1,140 +0,0 @@
# The test build matrix (stage: test) is constructed to test a wide range of
# configurations, rather than a single pass/fail. This helps to catch build
# failures and logic errors that present on platforms other than the ones the
# author has tested.
#
# Some builders use the dependency-generator in `./depends`, rather than using
# apt-get to install build dependencies. This guarantees that the tester is
# using the same versions as Gitian, so the build results are nearly identical
# to what would be found in a final release.
#
# In order to avoid rebuilding all dependencies for each build, the binaries
# are cached and re-used when possible. Changes in the dependency-generator
# will trigger cache-invalidation and rebuilds as necessary.
#
# These caches can be manually removed if necessary. This is one of the very
# few manual operations that is possible with Travis, and it can be done by a
# Bitcoin Core GitHub member via the Travis web interface [0].
#
# Travis CI uploads the cache after the script phase of the build [1].
# However, the build is terminated without saving the cache if it takes over
# 50 minutes [2]. Thus, if we spent too much time in early build stages, fail
# with an error and save the cache.
#
# [0] https://travis-ci.org/bitcoin/bitcoin/caches
# [1] https://docs.travis-ci.com/user/caching/#build-phases
# [2] https://docs.travis-ci.com/user/customizing-the-build#build-timeouts
dist: xenial
os: linux
language: minimal
cache:
ccache: true
directories:
- $TRAVIS_BUILD_DIR/depends/built
- $TRAVIS_BUILD_DIR/depends/sdk-sources
- $TRAVIS_BUILD_DIR/ci/scratch/.ccache
stages:
- lint
- test
- extended-lint
env:
global:
- CI_RETRY_EXE="travis_retry"
- CACHE_ERR_MSG="Error! Initial build successful, but not enough time remains to run later build stages and tests. Please manually re-run this job by using the travis restart button or asking a bitcoin maintainer to restart. The next run should not time out because the build cache has been saved."
before_install:
- set -o errexit; source ./ci/test/00_setup_env.sh
- set -o errexit; source ./ci/test/03_before_install.sh
install:
- set -o errexit; source ./ci/test/04_install.sh
before_script:
- set -o errexit; source ./ci/test/05_before_script.sh
script:
- export CONTINUE=1
- if [ $SECONDS -gt 1200 ]; then export CONTINUE=0; fi # Likely the depends build took very long
- if [ $TRAVIS_REPO_SLUG = "bitcoin/bitcoin" ]; then export CONTINUE=1; fi # Whitelisted repo (90 minutes build time)
- if [ $CONTINUE = "1" ]; then set -o errexit; source ./ci/test/06_script_a.sh; else set +o errexit; echo "$CACHE_ERR_MSG"; false; fi
- if [ $SECONDS -gt 2000 ]; then export CONTINUE=0; fi # Likely the build took very long; The tests take about 1000s, so we should abort if we have less than 50*60-1000=2000s left
- if [ $TRAVIS_REPO_SLUG = "bitcoin/bitcoin" ]; then export CONTINUE=1; fi # Whitelisted repo (90 minutes build time)
- if [ $CONTINUE = "1" ]; then set -o errexit; source ./ci/test/06_script_b.sh; else set +o errexit; echo "$CACHE_ERR_MSG"; false; fi
after_script:
- echo $TRAVIS_COMMIT_RANGE
jobs:
include:
- stage: lint
name: 'lint'
env:
cache: false
language: python
python: '3.5' # Oldest supported version according to doc/dependencies.md
install:
- set -o errexit; source ./ci/lint/04_install.sh
before_script:
- set -o errexit; source ./ci/lint/05_before_script.sh
script:
- set -o errexit; source ./ci/lint/06_script.sh
- stage: extended-lint
name: 'extended lint [runtime >= 60 seconds]'
env:
cache: false
language: python
python: '3.5'
install:
- set -o errexit; source ./ci/extended_lint/04_install.sh
before_script:
- set -o errexit; source ./ci/lint/05_before_script.sh
script:
- set -o errexit; source ./ci/extended_lint/06_script.sh
- stage: test
name: 'ARM [GOAL: install] [no unit or functional tests]'
env: >-
FILE_ENV="./ci/test/00_setup_env_arm.sh"
- stage: test
name: 'Win64 [GOAL: deploy] [no gui or functional tests]'
env: >-
FILE_ENV="./ci/test/00_setup_env_win64.sh"
- stage: test
name: '32-bit + dash [GOAL: install] [GUI: BIP70 enabled]'
env: >-
FILE_ENV="./ci/test/00_setup_env_i686.sh"
- stage: test
name: 'x86_64 Linux [GOAL: install] [bionic] [uses qt5 dev package instead of depends Qt to speed up build and avoid timeout] [unsigned char]'
env: >-
FILE_ENV="./ci/test/00_setup_env_amd64_qt5.sh"
- stage: test
name: 'x86_64 Linux [GOAL: install] [trusty] [no functional tests, no depends, only system libs]'
env: >-
FILE_ENV="./ci/test/00_setup_env_amd64_trusty.sh"
- stage: test
name: 'x86_64 Linux [GOAL: install] [xenial] [no depends, only system libs, sanitizers: thread (TSan), no wallet]'
env: >-
FILE_ENV="./ci/test/00_setup_env_amd64_tsan.sh"
TEST_RUNNER_EXTRA="--exclude feature_block" # Not enough memory on travis machines
- stage: test
name: 'x86_64 Linux [GOAL: install] [bionic] [no depends, only system libs, sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer]'
env: >-
FILE_ENV="./ci/test/00_setup_env_amd64_asan.sh"
- stage: test
name: 'x86_64 Linux [GOAL: install] [bionic] [no depends, only system libs, sanitizers: fuzzer,address,undefined]'
env: >-
FILE_ENV="./ci/test/00_setup_env_amd64_fuzz.sh"
- stage: test
name: 'x86_64 Linux [GOAL: install] [bionic] [no wallet]'
env: >-
FILE_ENV="./ci/test/00_setup_env_amd64_nowallet.sh"
- stage: test
name: 'macOS 10.10 [GOAL: deploy] [no functional tests]'
env: >-
FILE_ENV="./ci/test/00_setup_env_mac.sh"

View File

@@ -1,7 +1,7 @@
[main]
host = https://www.transifex.com
[bitcoin.qt-translation-019x]
file_filter = src/qt/locale/bitcoin_<lang>.ts
source_file = src/qt/locale/bitcoin_en.ts
[bitcoin.qt-translation-022x]
file_filter = src/qt/locale/bitcoin_<lang>.xlf
source_file = src/qt/locale/bitcoin_en.xlf
source_lang = en

View File

@@ -6,34 +6,69 @@ welcome to contribute towards development in the form of peer review, testing
and patches. This document explains the practical process and guidelines for
contributing.
Firstly in terms of structure, there is no particular concept of "Core
First, in terms of structure, there is no particular concept of "Bitcoin Core
developers" in the sense of privileged people. Open source often naturally
revolves around meritocracy where longer term contributors gain more trust from
the developer community. However, some hierarchy is necessary for practical
purposes. As such there are repository "maintainers" who are responsible for
merging pull requests as well as a "lead maintainer" who is responsible for the
release cycle, overall merging, moderation and appointment of maintainers.
revolves around a meritocracy where contributors earn trust from the developer
community over time. Nevertheless, some hierarchy is necessary for practical
purposes. As such, there are repository "maintainers" who are responsible for
merging pull requests, as well as a "lead maintainer" who is responsible for the
release cycle as well as overall merging, moderation and appointment of
maintainers.
Getting Started
---------------
New contributors are very welcome and needed.
Reviewing and testing is highly valued and the most effective way you can contribute
as a new contributor. It also will teach you much more about the code and
process than opening pull requests. Please refer to the [peer review](#peer-review)
section below.
Before you start contributing, familiarize yourself with the Bitcoin Core build
system and tests. Refer to the documentation in the repository on how to build
Bitcoin Core and how to run the unit tests, functional tests, and fuzz tests.
There are many open issues of varying difficulty waiting to be fixed.
If you're looking for somewhere to start contributing, check out the
[good first issue](https://github.com/bitcoin/bitcoin/issues?q=is%3Aopen+is%3Aissue+label%3A%22good+first+issue%22)
list.
list or changes that are
[up for grabs](https://github.com/bitcoin/bitcoin/issues?utf8=%E2%9C%93&q=label%3A%22Up+for+grabs%22).
Some of them might no longer be applicable. So if you are interested, but
unsure, you might want to leave a comment on the issue first.
You may also participate in the weekly
[Bitcoin Core PR Review Club](https://bitcoincore.reviews/) meeting.
### Good First Issue Label
The purpose of the `good first issue` label is to highlight which issues are
suitable for a new contributor without a deep understanding of the codebase.
However, good first issues can be solved by anyone. If they remain unsolved
for a longer time, a frequent contributor might address them.
You do not need to request permission to start working on an issue. However,
you are encouraged to leave a comment if you are planning to work on it. This
will help other contributors monitor which issues are actively being addressed
and is also an effective way to request assistance if and when you need it.
Communication Channels
----------------------
Most communication about Bitcoin Core development happens on IRC, in the
#bitcoin-core-dev channel on Freenode. The easiest way to participate on IRC is
with the web client, [webchat.freenode.net](https://webchat.freenode.net/). Chat
`#bitcoin-core-dev` channel on Libera Chat. The easiest way to participate on IRC is
with the web client, [web.libera.chat](https://web.libera.chat/#bitcoin-core-dev). Chat
history logs can be found
on [http://www.erisian.com.au/bitcoin-core-dev/](http://www.erisian.com.au/bitcoin-core-dev/)
and [http://gnusha.org/bitcoin-core-dev/](http://gnusha.org/bitcoin-core-dev/).
Discussion about code base improvements happens in GitHub issues and on pull
Discussion about codebase improvements happens in GitHub issues and pull
requests.
The developer
[mailing list](https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev)
should be used to discuss complicated or controversial changes before working on
should be used to discuss complicated or controversial consensus or P2P protocol changes before working on
a patch set.
@@ -41,22 +76,47 @@ Contributor Workflow
--------------------
The codebase is maintained using the "contributor workflow" where everyone
without exception contributes patch proposals using "pull requests". This
without exception contributes patch proposals using "pull requests" (PRs). This
facilitates social contribution, easy testing and peer review.
To contribute a patch, the workflow is as follows:
1. Fork repository
1. Fork repository ([only for the first time](https://help.github.com/en/articles/fork-a-repo))
1. Create topic branch
1. Commit patches
The project coding conventions in the [developer notes](doc/developer-notes.md)
must be adhered to.
For GUI-related issues or pull requests, the https://github.com/bitcoin-core/gui repository should be used.
For all other issues and pull requests, the https://github.com/bitcoin/bitcoin node repository should be used.
In general [commits should be atomic](https://en.wikipedia.org/wiki/Atomic_commit#Atomic_commit_convention)
and diffs should be easy to read. For this reason do not mix any formatting
The master branch for all monotree repositories is identical.
As a rule of thumb, everything that only modifies `src/qt` is a GUI-only pull
request. However:
* For global refactoring or other transversal changes the node repository
should be used.
* For GUI-related build system changes, the node repository should be used
because the change needs review by the build systems reviewers.
* Changes in `src/interfaces` need to go to the node repository because they
might affect other components like the wallet.
For large GUI changes that include build system and interface changes, it is
recommended to first open a pull request against the GUI repository. When there
is agreement to proceed with the changes, a pull request with the build system
and interfaces changes can be submitted to the node repository.
The project coding conventions in the [developer notes](doc/developer-notes.md)
must be followed.
### Committing Patches
In general, [commits should be atomic](https://en.wikipedia.org/wiki/Atomic_commit#Atomic_commit_convention)
and diffs should be easy to read. For this reason, do not mix any formatting
fixes or code moves with actual code changes.
Make sure each individual commit is hygienic: that it builds successfully on its
own without warnings, errors, regressions, or test failures.
Commit messages should be verbose by default consisting of a short subject line
(50 chars max), a blank line and detailed explanatory text as separate
paragraph(s), unless the title alone is self-explanatory (like "Corrected typo
@@ -68,7 +128,7 @@ If a particular commit references another issue, please add the reference. For
example: `refs #1234` or `fixes #4321`. Using the `fixes` or `closes` keywords
will cause the corresponding issue to be closed when the pull request is merged.
Commit messages should never contain any `@` mentions.
Commit messages should never contain any `@` mentions (usernames prefixed with "@").
Please refer to the [Git manual](https://git-scm.com/doc) for more information
about Git.
@@ -76,6 +136,8 @@ about Git.
- Push changes to your fork
- Create pull request
### Creating the Pull Request
The title of the pull request should be prefixed by the component or area that
the pull request affects. Valid areas as:
@@ -88,42 +150,54 @@ the pull request affects. Valid areas as:
- `refactor` for structural changes that do not change behavior
- `rpc`, `rest` or `zmq` for changes to the RPC, REST or ZMQ APIs
- `script` for changes to the scripts and tools
- `test` for changes to the bitcoin unit tests or QA tests
- `test`, `qa` or `ci` for changes to the unit tests, QA tests or CI code
- `util` or `lib` for changes to the utils or libraries
- `wallet` for changes to the wallet code
- `build` for changes to the GNU Autotools, reproducible builds or CI code
- `build` for changes to the GNU Autotools or reproducible builds
Examples:
consensus: Add new opcode for BIP-XXXX OP_CHECKAWESOMESIG
net: Automatically create hidden service, listen on Tor
net: Automatically create onion service, listen on Tor
qt: Add feed bump button
log: Fix typo in log message
Note that translations should not be submitted as pull requests, please see
The body of the pull request should contain sufficient description of *what* the
patch does, and even more importantly, *why*, with justification and reasoning.
You should include references to any discussions (for example, other issues or
mailing list discussions).
The description for a new pull request should not contain any `@` mentions. The
PR description will be included in the commit message when the PR is merged and
any users mentioned in the description will be annoyingly notified each time a
fork of Bitcoin Core copies the merge. Instead, make any username mentions in a
subsequent comment to the PR.
### Translation changes
Note that translations should not be submitted as pull requests. Please see
[Translation Process](https://github.com/bitcoin/bitcoin/blob/master/doc/translation_process.md)
for more information on helping with translations.
### Work in Progress Changes and Requests for Comments
If a pull request is not to be considered for merging (yet), please
prefix the title with [WIP] or use [Tasks Lists](https://help.github.com/articles/basic-writing-and-formatting-syntax/#task-lists)
in the body of the pull request to indicate tasks are pending.
The body of the pull request should contain enough description about what the
patch does together with any justification/reasoning. You should include
references to any discussions (for example other tickets or mailing list
discussions).
### Address Feedback
At this stage one should expect comments and review from other contributors. You
At this stage, one should expect comments and review from other contributors. You
can add more commits to your pull request by committing them locally and pushing
to your fork until you have satisfied all feedback.
Note: Code review is a burdensome but important part of the development process, and as such, certain types of pull requests are rejected. In general, if the **improvements** do not warrant the **review effort** required, the PR has a high chance of being rejected. It is up to the PR author to convince the reviewers that the changes warrant the review effort, and if reviewers are "Concept NAK'ing" the PR, the author may need to present arguments and/or do research backing their suggested changes.
Note: Code review is a burdensome but important part of the development process, and as such, certain types of pull requests are rejected. In general, if the **improvements** do not warrant the **review effort** required, the PR has a high chance of being rejected. It is up to the PR author to convince the reviewers that the changes warrant the review effort, and if reviewers are "Concept NACK'ing" the PR, the author may need to present arguments and/or do research backing their suggested changes.
Squashing Commits
---------------------------
If your pull request is accepted for merging, you may be asked by a maintainer
to squash and or [rebase](https://git-scm.com/docs/git-rebase) your commits
before it will be merged. The basic squashing workflow is shown below.
### Squashing Commits
If your pull request contains fixup commits (commits that change the same line of code repeatedly) or too fine-grained
commits, you may be asked to [squash](https://git-scm.com/docs/git-rebase#_interactive_mode) your commits
before it will be reviewed. The basic squashing workflow is shown below.
git checkout your_branch_name
git rebase -i HEAD~n
@@ -133,13 +207,13 @@ before it will be merged. The basic squashing workflow is shown below.
# Save and quit.
git push -f # (force push to GitHub)
Please update the resulting commit message if needed, it should read as a
coherent message. In most cases this means that you should not just list the
interim commits.
Please update the resulting commit message, if needed. It should read as a
coherent message. In most cases, this means not just listing the interim
commits.
If you have problems with squashing (or other workflows with `git`), you can
alternatively enable "Allow edits from maintainers" in the right GitHub
sidebar and ask for help in the pull request.
If you have problems with squashing or other git workflows, you can enable
"Allow edits from maintainers" in the right-hand sidebar of the GitHub web
interface and ask for help in the pull request.
Please refrain from creating several pull requests for the same change.
Use the pull request that is already open (or was created earlier) to amend
@@ -149,6 +223,20 @@ the respective change set.
The length of time required for peer review is unpredictable and will vary from
pull request to pull request.
### Rebasing Changes
When a pull request conflicts with the target branch, you may be asked to rebase it on top of the current target branch.
The `git rebase` command will take care of rebuilding your commits on top of the new base.
This project aims to have a clean git history, where code changes are only made in non-merge commits. This simplifies
auditability because merge commits can be assumed to not contain arbitrary code changes. Merge commits should be signed,
and the resulting git tree hash must be deterministic and reproducible. The script in
[/contrib/verify-commits](/contrib/verify-commits) checks that.
After a rebase, reviewers are encouraged to sign off on the force push. This should be relatively straightforward with
the `git range-diff` tool explained in the [productivity
notes](/doc/productivity.md#diff-the-diffs-with-git-range-diff). To avoid needless review churn, maintainers will
generally merge pull requests that received the most review attention first.
Pull Request Philosophy
-----------------------
@@ -173,9 +261,9 @@ in the future, they may be removed by the Repository Maintainer.
Refactoring is a necessary part of any software project's evolution. The
following guidelines cover refactoring pull requests for the project.
There are three categories of refactoring, code only moves, code style fixes,
code refactoring. In general refactoring pull requests should not mix these
three kinds of activity in order to make refactoring pull requests easy to
There are three categories of refactoring: code-only moves, code style fixes, and
code refactoring. In general, refactoring pull requests should not mix these
three kinds of activities in order to make refactoring pull requests easy to
review and uncontroversial. In all cases, refactoring PRs must not change the
behaviour of code within the pull request (bugs must be preserved as is).
@@ -209,8 +297,8 @@ In general, all pull requests must:
- Have a clear use case, fix a demonstrable bug or serve the greater good of
the project (for example refactoring for modularisation);
- Be well peer reviewed;
- Have unit tests and functional tests where appropriate;
- Be well peer-reviewed;
- Have unit tests, functional tests, and fuzz tests, where appropriate;
- Follow code style guidelines ([C++](doc/developer-notes.md), [functional tests](test/functional/README.md));
- Not break the existing test suite;
- Where bugs are fixed, where possible, there should be unit tests
@@ -237,7 +325,7 @@ spread out over GitHub, mailing list and IRC discussions).
#### Conceptual Review
A review can be a conceptual review, where the reviewer leaves a comment
* `Concept (N)ACK`, meaning "I do (not) agree in the general goal of this pull
* `Concept (N)ACK`, meaning "I do (not) agree with the general goal of this pull
request",
* `Approach (N)ACK`, meaning `Concept ACK`, but "I do (not) agree with the
approach of this change".
@@ -247,30 +335,28 @@ NACKs without accompanying reasoning may be disregarded.
#### Code Review
After conceptual agreement on the change, code review can be provided. It is
starting with `ACK BRANCH_COMMIT`, where `BRANCH_COMMIT` is the top of the
topic branch. The review is followed by a description of how the reviewer did
the review. The following
language is used within pull-request comments:
After conceptual agreement on the change, code review can be provided. A review
begins with `ACK BRANCH_COMMIT`, where `BRANCH_COMMIT` is the top of the PR
branch, followed by a description of how the reviewer did the review. The
following language is used within pull request comments:
- "I have tested the code", involving
change-specific manual testing in addition to running the unit and functional
tests, and in case it is not obvious how the manual testing was done, it should
be described;
- "I have tested the code", involving change-specific manual testing in
addition to running the unit, functional, or fuzz tests, and in case it is
not obvious how the manual testing was done, it should be described;
- "I have not tested the code, but I have reviewed it and it looks
OK, I agree it can be merged";
- Nit refers to trivial, often non-blocking issues.
- A "nit" refers to a trivial, often non-blocking issue.
Project maintainers reserve the right to weigh the opinions of peer reviewers
using common sense judgement and also may weight based on meritocracy: Those
that have demonstrated a deeper commitment and understanding towards the project
(over time) or have clear domain expertise may naturally have more weight, as
one would expect in all walks of life.
using common sense judgement and may also weigh based on merit. Reviewers that
have demonstrated a deeper commitment and understanding of the project over time
or who have clear domain expertise may naturally have more weight, as one would
expect in all walks of life.
Where a patch set affects consensus critical code, the bar will be set much
Where a patch set affects consensus-critical code, the bar will be much
higher in terms of discussion and peer review requirements, keeping in mind that
mistakes could be very costly to the wider community. This includes refactoring
of consensus critical code.
of consensus-critical code.
Where a patch set proposes to change the Bitcoin consensus, it must have been
discussed extensively on the mailing list and IRC, be accompanied by a widely
@@ -287,7 +373,7 @@ about:
- It may be because of a feature freeze due to an upcoming release. During this time,
only bug fixes are taken into consideration. If your pull request is a new feature,
it will not be prioritized until the release is over. Wait for release.
it will not be prioritized until after the release. Wait for the release.
- It may be because the changes you are suggesting do not appeal to people. Rather than
nits and critique, which require effort and means they care enough to spend time on your
contribution, thundering silence is a good sign of widespread (mild) dislike of a given change
@@ -297,18 +383,45 @@ about:
[developer notes](doc/developer-notes.md), is dangerous or insecure, is messily written, etc.
Identify and address any of the issues you find. Then ask e.g. on IRC if someone could give
their opinion on the concept itself.
- It may be because your code is too complex for all but a few people. And those people
- It may be because your code is too complex for all but a few people, and those people
may not have realized your pull request even exists. A great way to find people who
are qualified and care about the code you are touching is the
[Git Blame feature](https://help.github.com/articles/tracing-changes-in-a-file/). Simply
find the person touching the code you are touching before you and see if you can find
them and give them a nudge. Don't be incessant about the nudging though.
look up who last modified the code you are changing and see if you can find
them and give them a nudge. Don't be incessant about the nudging, though.
- Finally, if all else fails, ask on IRC or elsewhere for someone to give your pull request
a look. If you think you've been waiting an unreasonably long amount of time (month+) for
no particular reason (few lines changed, etc), this is totally fine. Try to return the favor
when someone else is asking for feedback on their code, and universe balances out.
a look. If you think you've been waiting for an unreasonably long time (say,
more than a month) for no particular reason (a few lines changed, etc.),
this is totally fine. Try to return the favor when someone else is asking
for feedback on their code, and the universe balances out.
- Remember that the best thing you can do while waiting is give review to others!
Backporting
-----------
Security and bug fixes can be backported from `master` to release
branches.
If the backport is non-trivial, it may be appropriate to open an
additional PR to backport the change, but only after the original PR
has been merged.
Otherwise, backports will be done in batches and
the maintainers will use the proper `Needs backport (...)` labels
when needed (the original author does not need to worry about it).
A backport should contain the following metadata in the commit body:
```
Github-Pull: #<PR number>
Rebased-From: <commit hash of the original commit>
```
Have a look at [an example backport PR](
https://github.com/bitcoin/bitcoin/pull/16189).
Also see the [backport.py script](
https://github.com/bitcoin-core/bitcoin-maintainer-tools#backport).
Release Policy
--------------

View File

@@ -1,7 +1,7 @@
The MIT License (MIT)
Copyright (c) 2009-2019 The Bitcoin Core developers
Copyright (c) 2009-2019 Bitcoin Developers
Copyright (c) 2009-2021 The Bitcoin Core developers
Copyright (c) 2009-2021 Bitcoin Developers
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -1,13 +1,18 @@
# Copyright (c) 2013-2016 The Bitcoin Core developers
# Copyright (c) 2013-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
# Pattern rule to print variables, e.g. make print-top_srcdir
print-%: FORCE
@echo '$*'='$($*)'
ACLOCAL_AMFLAGS = -I build-aux/m4
SUBDIRS = src
if ENABLE_MAN
SUBDIRS += doc/man
endif
.PHONY: deploy FORCE
.INTERMEDIATE: $(OSX_TEMP_ISO) $(COVERAGE_INFO)
export PYTHONPATH
@@ -20,8 +25,11 @@ BITCOIND_BIN=$(top_builddir)/src/$(BITCOIN_DAEMON_NAME)$(EXEEXT)
BITCOIN_QT_BIN=$(top_builddir)/src/qt/$(BITCOIN_GUI_NAME)$(EXEEXT)
BITCOIN_CLI_BIN=$(top_builddir)/src/$(BITCOIN_CLI_NAME)$(EXEEXT)
BITCOIN_TX_BIN=$(top_builddir)/src/$(BITCOIN_TX_NAME)$(EXEEXT)
BITCOIN_UTIL_BIN=$(top_builddir)/src/$(BITCOIN_UTIL_NAME)$(EXEEXT)
BITCOIN_WALLET_BIN=$(top_builddir)/src/$(BITCOIN_WALLET_TOOL_NAME)$(EXEEXT)
BITCOIN_WIN_INSTALLER=$(PACKAGE)-$(PACKAGE_VERSION)-win$(WINDOWS_BITS)-setup$(EXEEXT)
BITCOIN_NODE_BIN=$(top_builddir)/src/$(BITCOIN_MP_NODE_NAME)$(EXEEXT)
BITCOIN_GUI_BIN=$(top_builddir)/src/$(BITCOIN_MP_GUI_NAME)$(EXEEXT)
BITCOIN_WIN_INSTALLER=$(PACKAGE)-$(PACKAGE_VERSION)-win64-setup$(EXEEXT)
empty :=
space := $(empty) $(empty)
@@ -29,49 +37,49 @@ space := $(empty) $(empty)
OSX_APP=Bitcoin-Qt.app
OSX_VOLNAME = $(subst $(space),-,$(PACKAGE_NAME))
OSX_DMG = $(OSX_VOLNAME).dmg
OSX_TEMP_ISO = $(OSX_DMG:.dmg=).temp.iso
OSX_BACKGROUND_SVG=background.svg
OSX_BACKGROUND_IMAGE=background.tiff
OSX_BACKGROUND_IMAGE_DPIS=36 72
OSX_DSSTORE_GEN=$(top_srcdir)/contrib/macdeploy/custom_dsstore.py
OSX_DEPLOY_SCRIPT=$(top_srcdir)/contrib/macdeploy/macdeployqtplus
OSX_FANCY_PLIST=$(top_srcdir)/contrib/macdeploy/fancy.plist
OSX_INSTALLER_ICONS=$(top_srcdir)/src/qt/res/icons/bitcoin.icns
OSX_PLIST=$(top_builddir)/share/qt/Info.plist #not installed
OSX_QT_TRANSLATIONS = da,de,es,hu,ru,uk,zh_CN,zh_TW
DIST_DOCS = $(wildcard doc/*.md) $(wildcard doc/release-notes/*.md)
DIST_CONTRIB = $(top_srcdir)/contrib/bitcoin-cli.bash-completion \
$(top_srcdir)/contrib/bitcoin-tx.bash-completion \
$(top_srcdir)/contrib/bitcoind.bash-completion \
$(top_srcdir)/contrib/debian/copyright \
$(top_srcdir)/contrib/init \
$(top_srcdir)/contrib/install_db4.sh
DIST_CONTRIB = \
$(top_srcdir)/test/sanitizer_suppressions/lsan \
$(top_srcdir)/test/sanitizer_suppressions/tsan \
$(top_srcdir)/test/sanitizer_suppressions/ubsan \
$(top_srcdir)/contrib/linearize/linearize-data.py \
$(top_srcdir)/contrib/linearize/linearize-hashes.py
DIST_SHARE = \
$(top_srcdir)/share/genbuild.sh \
$(top_srcdir)/share/rpcauth
BIN_CHECKS=$(top_srcdir)/contrib/devtools/symbol-check.py \
$(top_srcdir)/contrib/devtools/security-check.py
$(top_srcdir)/contrib/devtools/security-check.py \
$(top_srcdir)/contrib/devtools/utils.py \
$(top_srcdir)/contrib/devtools/pixie.py
WINDOWS_PACKAGING = $(top_srcdir)/share/pixmaps/bitcoin.ico \
$(top_srcdir)/share/pixmaps/nsis-header.bmp \
$(top_srcdir)/share/pixmaps/nsis-wizard.bmp \
$(top_srcdir)/doc/README_windows.txt
OSX_PACKAGING = $(OSX_DEPLOY_SCRIPT) $(OSX_FANCY_PLIST) $(OSX_INSTALLER_ICONS) \
OSX_PACKAGING = $(OSX_DEPLOY_SCRIPT) $(OSX_INSTALLER_ICONS) \
$(top_srcdir)/contrib/macdeploy/$(OSX_BACKGROUND_SVG) \
$(OSX_DSSTORE_GEN) \
$(top_srcdir)/contrib/macdeploy/detached-sig-apply.sh \
$(top_srcdir)/contrib/macdeploy/detached-sig-create.sh
COVERAGE_INFO = baseline.info \
COVERAGE_INFO = $(COV_TOOL_WRAPPER) baseline.info \
test_bitcoin_filtered.info total_coverage.info \
baseline_filtered.info functional_test.info functional_test_filtered.info \
test_bitcoin_coverage.info test_bitcoin.info
test_bitcoin_coverage.info test_bitcoin.info fuzz.info fuzz_filtered.info fuzz_coverage.info
dist-hook:
-$(GIT) archive --format=tar HEAD -- src/clientversion.cpp | $(AMTAR) -C $(top_distdir) -xf -
if TARGET_WINDOWS
$(BITCOIN_WIN_INSTALLER): all-recursive
$(MKDIR_P) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIND_BIN) $(top_builddir)/release
@@ -79,17 +87,22 @@ $(BITCOIN_WIN_INSTALLER): all-recursive
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_CLI_BIN) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_TX_BIN) $(top_builddir)/release
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_WALLET_BIN) $(top_builddir)/release
@test -f $(MAKENSIS) && $(MAKENSIS) -V2 $(top_builddir)/share/setup.nsi || \
STRIPPROG="$(STRIP)" $(INSTALL_STRIP_PROGRAM) $(BITCOIN_UTIL_BIN) $(top_builddir)/release
@test -f $(MAKENSIS) && echo 'OutFile "$@"' | cat $(top_builddir)/share/setup.nsi - | $(MAKENSIS) -V2 - || \
echo error: could not build $@
@echo built $@
deploy: $(BITCOIN_WIN_INSTALLER)
endif
if TARGET_DARWIN
$(OSX_APP)/Contents/PkgInfo:
$(MKDIR_P) $(@D)
@echo "APPL????" > $@
$(OSX_APP)/Contents/Resources/empty.lproj:
$(MKDIR_P) $(@D)
@touch $@
@touch $@
$(OSX_APP)/Contents/Info.plist: $(OSX_PLIST)
$(MKDIR_P) $(@D)
@@ -116,7 +129,7 @@ osx_volname:
if BUILD_DARWIN
$(OSX_DMG): $(OSX_APP_BUILT) $(OSX_PACKAGING) $(OSX_BACKGROUND_IMAGE)
$(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) -add-qt-tr $(OSX_QT_TRANSLATIONS) -translations-dir=$(QT_TRANSLATION_DIR) -dmg -fancy $(OSX_FANCY_PLIST) -verbose 2 -volname $(OSX_VOLNAME)
$(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR) -dmg
$(OSX_BACKGROUND_IMAGE).png: contrib/macdeploy/$(OSX_BACKGROUND_SVG)
sed 's/PACKAGE_NAME/$(PACKAGE_NAME)/' < "$<" | $(RSVG_CONVERT) -f png -d 36 -p 36 -o $@
@@ -126,7 +139,7 @@ $(OSX_BACKGROUND_IMAGE): $(OSX_BACKGROUND_IMAGE).png $(OSX_BACKGROUND_IMAGE)@2x.
tiffutil -cathidpicheck $^ -out $@
deploydir: $(OSX_DMG)
else
else !BUILD_DARWIN
APP_DIST_DIR=$(top_builddir)/dist
APP_DIST_EXTRAS=$(APP_DIST_DIR)/.background/$(OSX_BACKGROUND_IMAGE) $(APP_DIST_DIR)/.DS_Store $(APP_DIST_DIR)/Applications
@@ -136,8 +149,11 @@ $(APP_DIST_DIR)/Applications:
$(APP_DIST_EXTRAS): $(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt
$(OSX_DMG): $(APP_DIST_EXTRAS)
$(GENISOIMAGE) -no-cache-inodes -D -l -probe -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -apple -o $@ dist
$(OSX_TEMP_ISO): $(APP_DIST_EXTRAS)
$(XORRISOFS) -D -l -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -o $@ $(APP_DIST_DIR) -- $(if $(SOURCE_DATE_EPOCH),-volume_date all_file_dates =$(SOURCE_DATE_EPOCH))
$(OSX_DMG): $(OSX_TEMP_ISO)
$(DMG) dmg "$<" "$@"
dpi%.$(OSX_BACKGROUND_IMAGE): contrib/macdeploy/$(OSX_BACKGROUND_SVG)
sed 's/PACKAGE_NAME/$(PACKAGE_NAME)/' < "$<" | $(RSVG_CONVERT) -f png -d $* -p $* | $(IMAGEMAGICK_CONVERT) - $@
@@ -146,22 +162,15 @@ $(APP_DIST_DIR)/.background/$(OSX_BACKGROUND_IMAGE): $(OSX_BACKGROUND_IMAGE_DPIF
$(MKDIR_P) $(@D)
$(TIFFCP) -c none $(OSX_BACKGROUND_IMAGE_DPIFILES) $@
$(APP_DIST_DIR)/.DS_Store: $(OSX_DSSTORE_GEN)
$(PYTHON) $< "$@" "$(OSX_VOLNAME)"
$(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt: $(OSX_APP_BUILT) $(OSX_PACKAGING)
INSTALLNAMETOOL=$(INSTALLNAMETOOL) OTOOL=$(OTOOL) STRIP=$(STRIP) $(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) -translations-dir=$(QT_TRANSLATION_DIR) -add-qt-tr $(OSX_QT_TRANSLATIONS) -verbose 2
INSTALLNAMETOOL=$(INSTALLNAMETOOL) OTOOL=$(OTOOL) STRIP=$(STRIP) $(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR)
deploydir: $(APP_DIST_EXTRAS)
endif
endif !BUILD_DARWIN
if TARGET_DARWIN
appbundle: $(OSX_APP_BUILT)
deploy: $(OSX_DMG)
endif
if TARGET_WINDOWS
deploy: $(BITCOIN_WIN_INSTALLER)
endif
$(BITCOIN_QT_BIN): FORCE
$(MAKE) -C src qt/$(@F)
@@ -175,19 +184,54 @@ $(BITCOIN_CLI_BIN): FORCE
$(BITCOIN_TX_BIN): FORCE
$(MAKE) -C src $(@F)
$(BITCOIN_UTIL_BIN): FORCE
$(MAKE) -C src $(@F)
$(BITCOIN_WALLET_BIN): FORCE
$(MAKE) -C src $(@F)
if USE_LCOV
LCOV_FILTER_PATTERN=-p "/usr/include/" -p "/usr/lib/" -p "src/leveldb/" -p "src/bench/" -p "src/univalue" -p "src/crypto/ctaes" -p "src/secp256k1"
$(BITCOIN_NODE_BIN): FORCE
$(MAKE) -C src $(@F)
baseline.info:
$(BITCOIN_GUI_BIN): FORCE
$(MAKE) -C src $(@F)
if USE_LCOV
LCOV_FILTER_PATTERN = \
-p "/usr/local/" \
-p "/usr/include/" \
-p "/usr/lib/" \
-p "/usr/lib64/" \
-p "src/leveldb/" \
-p "src/crc32c/" \
-p "src/bench/" \
-p "src/univalue" \
-p "src/crypto/ctaes" \
-p "src/secp256k1" \
-p "depends"
DIR_FUZZ_SEED_CORPUS ?= qa-assets/fuzz_seed_corpus
$(COV_TOOL_WRAPPER):
@echo 'exec $(COV_TOOL) "$$@"' > $(COV_TOOL_WRAPPER)
@chmod +x $(COV_TOOL_WRAPPER)
baseline.info: $(COV_TOOL_WRAPPER)
$(LCOV) -c -i -d $(abs_builddir)/src -o $@
baseline_filtered.info: baseline.info
$(abs_builddir)/contrib/filter-lcov.py $(LCOV_FILTER_PATTERN) $< $@
$(LCOV) -a $@ $(LCOV_OPTS) -o $@
fuzz.info: baseline_filtered.info
@TIMEOUT=15 test/fuzz/test_runner.py $(DIR_FUZZ_SEED_CORPUS) -l DEBUG
$(LCOV) -c $(LCOV_OPTS) -d $(abs_builddir)/src --t fuzz-tests -o $@
$(LCOV) -z $(LCOV_OPTS) -d $(abs_builddir)/src
fuzz_filtered.info: fuzz.info
$(abs_builddir)/contrib/filter-lcov.py $(LCOV_FILTER_PATTERN) $< $@
$(LCOV) -a $@ $(LCOV_OPTS) -o $@
test_bitcoin.info: baseline_filtered.info
$(MAKE) -C src/ check
$(LCOV) -c $(LCOV_OPTS) -d $(abs_builddir)/src -t test_bitcoin -o $@
@@ -206,12 +250,19 @@ functional_test_filtered.info: functional_test.info
$(abs_builddir)/contrib/filter-lcov.py $(LCOV_FILTER_PATTERN) $< $@
$(LCOV) -a $@ $(LCOV_OPTS) -o $@
fuzz_coverage.info: fuzz_filtered.info
$(LCOV) -a $(LCOV_OPTS) baseline_filtered.info -a fuzz_filtered.info -o $@ | $(GREP) "\%" | $(AWK) '{ print substr($$3,2,50) "/" $$5 }' > coverage_percent.txt
test_bitcoin_coverage.info: baseline_filtered.info test_bitcoin_filtered.info
$(LCOV) -a $(LCOV_OPTS) baseline_filtered.info -a test_bitcoin_filtered.info -o $@
total_coverage.info: test_bitcoin_filtered.info functional_test_filtered.info
$(LCOV) -a $(LCOV_OPTS) baseline_filtered.info -a test_bitcoin_filtered.info -a functional_test_filtered.info -o $@ | $(GREP) "\%" | $(AWK) '{ print substr($$3,2,50) "/" $$5 }' > coverage_percent.txt
fuzz.coverage/.dirstamp: fuzz_coverage.info
$(GENHTML) -s $(LCOV_OPTS) $< -o $(@D)
@touch $@
test_bitcoin.coverage/.dirstamp: test_bitcoin_coverage.info
$(GENHTML) -s $(LCOV_OPTS) $< -o $(@D)
@touch $@
@@ -220,13 +271,15 @@ total.coverage/.dirstamp: total_coverage.info
$(GENHTML) -s $(LCOV_OPTS) $< -o $(@D)
@touch $@
cov_fuzz: fuzz.coverage/.dirstamp
cov: test_bitcoin.coverage/.dirstamp total.coverage/.dirstamp
endif
dist_noinst_SCRIPTS = autogen.sh
EXTRA_DIST = $(DIST_SHARE) $(DIST_CONTRIB) $(DIST_DOCS) $(WINDOWS_PACKAGING) $(OSX_PACKAGING) $(BIN_CHECKS)
EXTRA_DIST = $(DIST_SHARE) $(DIST_CONTRIB) $(WINDOWS_PACKAGING) $(OSX_PACKAGING) $(BIN_CHECKS)
EXTRA_DIST += \
test/functional \
@@ -281,6 +334,8 @@ EXTRA_DIST += \
test/util/data/txcreatescript3.json \
test/util/data/txcreatescript4.hex \
test/util/data/txcreatescript4.json \
test/util/data/txcreatescript5.hex \
test/util/data/txcreatescript6.hex \
test/util/data/txcreatesignv1.hex \
test/util/data/txcreatesignv1.json \
test/util/data/txcreatesignv2.hex \
@@ -288,8 +343,6 @@ EXTRA_DIST += \
CLEANFILES = $(OSX_DMG) $(BITCOIN_WIN_INSTALLER)
.INTERMEDIATE: $(COVERAGE_INFO)
DISTCHECK_CONFIGURE_FLAGS = --enable-man
doc/doxygen/.stamp: doc/Doxyfile FORCE
@@ -308,7 +361,20 @@ clean-docs:
rm -rf doc/doxygen
clean-local: clean-docs
rm -rf coverage_percent.txt test_bitcoin.coverage/ total.coverage/ test/tmp/ cache/ $(OSX_APP)
rm -rf coverage_percent.txt test_bitcoin.coverage/ total.coverage/ fuzz.coverage/ test/tmp/ cache/ $(OSX_APP)
rm -rf test/functional/__pycache__ test/functional/test_framework/__pycache__ test/cache share/rpcauth/__pycache__
rm -rf osx_volname dist/ dpi36.background.tiff dpi72.background.tiff
test-security-check:
if TARGET_DARWIN
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_MACHO
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-symbol-check.py TestSymbolChecks.test_MACHO
endif
if TARGET_WINDOWS
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_PE
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-symbol-check.py TestSymbolChecks.test_PE
endif
if TARGET_LINUX
$(AM_V_at) CC='$(CC)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-security-check.py TestSecurityChecks.test_ELF
$(AM_V_at) CC='$(CC)' CPPFILT='$(CPPFILT)' $(PYTHON) $(top_srcdir)/contrib/devtools/test-symbol-check.py TestSymbolChecks.test_ELF
endif

View File

@@ -3,6 +3,11 @@ Bitcoin Core integration/staging tree
https://bitcoincore.org
For an immediately usable, binary version of the Bitcoin Core software, see
https://bitcoincore.org/en/download/.
Further information about Bitcoin Core is available in the [doc folder](/doc).
What is Bitcoin?
----------------
@@ -12,9 +17,7 @@ with no central authority: managing transactions and issuing money are carried
out collectively by the network. Bitcoin Core is the name of open source
software which enables the use of this currency.
For more information, as well as an immediately useable, binary version of
the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the
[original whitepaper](https://bitcoincore.org/bitcoin.pdf).
For more information read the original Bitcoin whitepaper.
License
-------
@@ -25,9 +28,14 @@ information or see https://opensource.org/licenses/MIT.
Development Process
-------------------
The `master` branch is regularly built and tested, but is not guaranteed to be
The `master` branch is regularly built (see `doc/build-*.md` for instructions) and tested, but it is not guaranteed to be
completely stable. [Tags](https://github.com/bitcoin/bitcoin/tags) are created
regularly to indicate new official, stable release versions of Bitcoin Core.
regularly from release branches to indicate new official, stable release versions of Bitcoin Core.
The https://github.com/bitcoin-core/gui repository is used exclusively for the
development of the GUI. Its master branch is identical in all monotree
repositories. Release branches and tags do not exist, so please do not fork
that repository unless it is for development reasons.
The contribution workflow is described in [CONTRIBUTING.md](CONTRIBUTING.md)
and useful hints for developers can be found in [doc/developer-notes.md](doc/developer-notes.md).
@@ -48,10 +56,11 @@ submit new unit tests for old code. Unit tests can be compiled and run
and extending unit tests can be found in [/src/test/README.md](/src/test/README.md).
There are also [regression and integration tests](/test), written
in Python, that are run automatically on the build server.
in Python.
These tests can be run (if the [test dependencies](/test) are installed) with: `test/functional/test_runner.py`
The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.
The CI (Continuous Integration) systems make sure that every pull request is built for Windows, Linux, and macOS,
and that unit/sanity tests are run automatically.
### Manual Quality Assurance (QA) Testing
@@ -71,5 +80,3 @@ Translations are periodically pulled from Transifex and merged into the git repo
**Important**: We do not accept translation changes as GitHub pull requests because the next
pull from Transifex would automatically overwrite them again.
Translators should also subscribe to the [mailing list](https://groups.google.com/forum/#!forum/bitcoin-translators).

138
REVIEWERS Normal file
View File

@@ -0,0 +1,138 @@
# ==============================================================================
# Bitcoin Core REVIEWERS
# ==============================================================================
# Configuration of automated review requests for the bitcoin/bitcoin repo
# via DrahtBot.
# Order is not important; if a modified file or directory matches a fnmatch,
# the reviewer will be mentioned in a PR comment requesting a review.
# Regular contributors are free to add their names to specific directories or
# files provided that they are willing to provide a review.
# Absence from this list should not be interpreted as a discouragement to
# review a pull request. Peer review is always welcome and is a critical
# component of the progress of the codebase. Information on peer review
# guidelines can be found in the CONTRIBUTING.md doc.
# Maintainers
# @fanquake
# @hebasto
# @jonasschnelli
# @laanwj
# @marcofalke
# @meshcollider
# @sipa
# Docs
/doc/*[a-zA-Z-].md @harding
/doc/Doxyfile.in @fanquake
/doc/REST-interface.md @jonasschnelli
/doc/benchmarking.md @ariard
/doc/bitcoin-conf.md @hebasto
/doc/build-freebsd.md @fanquake
/doc/build-netbsd.md @fanquake
/doc/build-openbsd.md @laanwj
/doc/build-osx.md @fanquake
/doc/build-unix.md @laanwj
/doc/build-windows.md @sipsorcery
/doc/dependencies.md @fanquake
/doc/developer-notes.md @laanwj
/doc/files.md @hebasto
/doc/gitian-building.md @laanwj
/doc/reduce-memory.md @fanquake
/doc/reduce-traffic.md @jonasschnelli
/doc/release-process.md @laanwj
/doc/translation_strings_policy.md @laanwj
# Build aux
/build-aux/m4/bitcoin_qt.m4 @hebasto
# MSVC build system
/build_msvc/ @sipsorcery
# Settings
/src/util/settings.* @ryanofsky
# Fuzzing
# Tests
/src/test/net_peer_eviction_tests.cpp @jonatack
/test/functional/mempool_updatefromblock.py @hebasto
/test/functional/feature_asmap.py @jonatack
/test/functional/interface_bitcoin_cli.py @jonatack
# Backwards compatibility tests
*_compatibility.py @sjors
/test/functional/wallet_upgradewallet.py @sjors @achow101
/test/get_previous_releases.py @sjors
# Translations
/src/util/translation.h @hebasto
# Dev Tools
/contrib/devtools/security-check.py @fanquake
/contrib/devtools/test-security-check.py @fanquake
/contrib/devtools/symbol-check.py @fanquake
# Gitian/Guix
/contrib/gitian-build.py @hebasto
/contrib/guix/ @dongcarl
# Compatibility
/src/compat/glibc_* @fanquake
# GUI
/src/qt/forms/ @hebasto
# Wallet
/src/wallet/ @achow101
# CLI
/src/bitcoin-cli.cpp @jonatack
# Coinstats
/src/node/coinstats.* @fjahr
# Index
/src/index/ @fjahr
# Descriptors
*descriptor* @achow101 @sipa
# External signer
*external_signer* @sjors
/doc/external-signer.md @sjors
*signer.py @sjors
# Interfaces
/src/interfaces/ @ryanofsky
# DB
/src/txdb.* @jamesob
/src/dbwrapper.* @jamesob
# Linter
/test/lint/lint-shell.sh @hebasto
# Bech32
/src/bech32.* @sipa
/src/bench/bech32.* @sipa
# PSBT
/src/psbt* @achow101
/src/node/psbt* @achow101
/doc/psbt.md @achow101
# P2P
/src/net_processing.* @sipa
/src/protocol.* @sipa
# Consensus
/src/coins.* @sipa @jamesob
/src/script/script.* @sipa
/src/script/interpreter.* @sipa
/src/validation.* @sipa
/src/consensus/ @sipa

View File

@@ -1,13 +1,13 @@
#!/bin/sh
# Copyright (c) 2013-2016 The Bitcoin Core developers
# Copyright (c) 2013-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
set -e
srcdir="$(dirname $0)"
srcdir="$(dirname "$0")"
cd "$srcdir"
if [ -z ${LIBTOOLIZE} ] && GLIBTOOLIZE="$(command -v glibtoolize)"; then
if [ -z "${LIBTOOLIZE}" ] && GLIBTOOLIZE="$(command -v glibtoolize)"; then
LIBTOOLIZE="${GLIBTOOLIZE}"
export LIBTOOLIZE
fi

View File

@@ -11,7 +11,7 @@
# Test for the Boost C++ libraries of a particular version (or newer)
#
# If no path to the installed boost library is given the macro searchs
# under /usr, /usr/local, /opt and /opt/local and evaluates the
# under /usr, /usr/local, /opt, /opt/local and /opt/homebrew and evaluates the
# $BOOST_ROOT environment variable. Further documentation is available at
# <http://randspringer.de/boost/index.html>.
#
@@ -33,7 +33,7 @@
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 47
#serial 48
# example boost program (need to pass version)
m4_define([_AX_BOOST_BASE_PROGRAM],
@@ -123,6 +123,7 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
dnl are almost assuredly the ones desired.
AS_CASE([${host_cpu}],
[i?86],[multiarch_libsubdir="lib/i386-${host_os}"],
[armv7l],[multiarch_libsubdir="lib/arm-${host_os}"],
[multiarch_libsubdir="lib/${host_cpu}-${host_os}"]
)
@@ -150,7 +151,7 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
else
search_libsubdirs="$multiarch_libsubdir $libsubdirs"
fi
for _AX_BOOST_BASE_boost_path_tmp in /usr /usr/local /opt /opt/local ; do
for _AX_BOOST_BASE_boost_path_tmp in /usr /usr/local /opt /opt/local /opt/homebrew/; do
if test -d "$_AX_BOOST_BASE_boost_path_tmp/include/boost" && test -r "$_AX_BOOST_BASE_boost_path_tmp/include/boost" ; then
for libsubdir in $search_libsubdirs ; do
if ls "$_AX_BOOST_BASE_boost_path_tmp/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi
@@ -226,7 +227,7 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
fi
else
if test "x$cross_compiling" != "xyes" ; then
for _AX_BOOST_BASE_boost_path in /usr /usr/local /opt /opt/local ; do
for _AX_BOOST_BASE_boost_path in /usr /usr/local /opt /opt/local /opt/homebrew ; do
if test -d "$_AX_BOOST_BASE_boost_path" && test -r "$_AX_BOOST_BASE_boost_path" ; then
for i in `ls -d $_AX_BOOST_BASE_boost_path/include/boost-* 2>/dev/null`; do
_version_tmp=`echo $i | sed "s#$_AX_BOOST_BASE_boost_path##" | sed 's/\/include\/boost-//' | sed 's/_/./'`

View File

@@ -1,118 +0,0 @@
# ===========================================================================
# https://www.gnu.org/software/autoconf-archive/ax_boost_chrono.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_BOOST_CHRONO
#
# DESCRIPTION
#
# Test for Chrono library from the Boost C++ libraries. The macro requires
# a preceding call to AX_BOOST_BASE. Further documentation is available at
# <http://randspringer.de/boost/index.html>.
#
# This macro calls:
#
# AC_SUBST(BOOST_CHRONO_LIB)
#
# And sets:
#
# HAVE_BOOST_CHRONO
#
# LICENSE
#
# Copyright (c) 2012 Xiyue Deng <manphiz@gmail.com>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 5
AC_DEFUN([AX_BOOST_CHRONO],
[
AC_ARG_WITH([boost-chrono],
AS_HELP_STRING([--with-boost-chrono@<:@=special-lib@:>@],
[use the Chrono library from boost - it is possible to specify a certain library for the linker
e.g. --with-boost-chrono=boost_chrono-gcc-mt ]),
[
if test "$withval" = "no"; then
want_boost="no"
elif test "$withval" = "yes"; then
want_boost="yes"
ax_boost_user_chrono_lib=""
else
want_boost="yes"
ax_boost_user_chrono_lib="$withval"
fi
],
[want_boost="yes"]
)
if test "x$want_boost" = "xyes"; then
AC_REQUIRE([AC_PROG_CC])
AC_REQUIRE([AC_CANONICAL_BUILD])
CPPFLAGS_SAVED="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS_SAVED="$LDFLAGS"
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_CACHE_CHECK(whether the Boost::Chrono library is available,
ax_cv_boost_chrono,
[AC_LANG_PUSH([C++])
CXXFLAGS_SAVE=$CXXFLAGS
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[@%:@include <boost/chrono.hpp>]],
[[boost::chrono::system_clock::time_point* time = new boost::chrono::system_clock::time_point; delete time;]])],
ax_cv_boost_chrono=yes, ax_cv_boost_chrono=no)
CXXFLAGS=$CXXFLAGS_SAVE
AC_LANG_POP([C++])
])
if test "x$ax_cv_boost_chrono" = "xyes"; then
AC_SUBST(BOOST_CPPFLAGS)
AC_DEFINE(HAVE_BOOST_CHRONO,,[define if the Boost::Chrono library is available])
BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'`
LDFLAGS_SAVE=$LDFLAGS
if test "x$ax_boost_user_chrono_lib" = "x"; then
for libextension in `ls $BOOSTLIBDIR/libboost_chrono*.so* $BOOSTLIBDIR/libboost_chrono*.dylib* $BOOSTLIBDIR/libboost_chrono*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^lib\(boost_chrono.*\)\.so.*$;\1;' -e 's;^lib\(boost_chrono.*\)\.dylib.*$;\1;' -e 's;^lib\(boost_chrono.*\)\.a.*$;\1;'` ; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break],
[link_chrono="no"])
done
if test "x$link_chrono" != "xyes"; then
for libextension in `ls $BOOSTLIBDIR/boost_chrono*.dll* $BOOSTLIBDIR/boost_chrono*.a* 2>/dev/null | sed 's,.*/,,' | sed -e 's;^\(boost_chrono.*\)\.dll.*$;\1;' -e 's;^\(boost_chrono.*\)\.a.*$;\1;'` ; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break],
[link_chrono="no"])
done
fi
else
for ax_lib in $ax_boost_user_chrono_lib boost_chrono-$ax_boost_user_chrono_lib; do
AC_CHECK_LIB($ax_lib, exit,
[BOOST_CHRONO_LIB="-l$ax_lib"; AC_SUBST(BOOST_CHRONO_LIB) link_chrono="yes"; break],
[link_chrono="no"])
done
fi
if test "x$ax_lib" = "x"; then
AC_MSG_ERROR(Could not find a version of the Boost::Chrono library!)
fi
if test "x$link_chrono" = "xno"; then
AC_MSG_ERROR(Could not link against $ax_lib !)
fi
fi
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
fi
])

View File

@@ -1,163 +0,0 @@
# ===========================================================================
# https://www.gnu.org/software/autoconf-archive/ax_boost_thread.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_BOOST_THREAD
#
# DESCRIPTION
#
# Test for Thread library from the Boost C++ libraries. The macro requires
# a preceding call to AX_BOOST_BASE. Further documentation is available at
# <http://randspringer.de/boost/index.html>.
#
# This macro calls:
#
# AC_SUBST(BOOST_THREAD_LIB)
#
# And sets:
#
# HAVE_BOOST_THREAD
#
# LICENSE
#
# Copyright (c) 2009 Thomas Porschberg <thomas@randspringer.de>
# Copyright (c) 2009 Michael Tindal
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 32
AC_DEFUN([AX_BOOST_THREAD],
[
AC_ARG_WITH([boost-thread],
AS_HELP_STRING([--with-boost-thread@<:@=special-lib@:>@],
[use the Thread library from boost -
it is possible to specify a certain library for the linker
e.g. --with-boost-thread=boost_thread-gcc-mt ]),
[
if test "$withval" = "yes"; then
want_boost="yes"
ax_boost_user_thread_lib=""
else
want_boost="yes"
ax_boost_user_thread_lib="$withval"
fi
],
[want_boost="yes"]
)
if test "x$want_boost" = "xyes"; then
AC_REQUIRE([AC_PROG_CC])
AC_REQUIRE([AC_CANONICAL_BUILD])
CPPFLAGS_SAVED="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS_SAVED="$LDFLAGS"
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_CACHE_CHECK(whether the Boost::Thread library is available,
ax_cv_boost_thread,
[AC_LANG_PUSH([C++])
CXXFLAGS_SAVE=$CXXFLAGS
if test "x$host_os" = "xsolaris" ; then
CXXFLAGS="-pthreads $CXXFLAGS"
elif test "x$host_os" = "xmingw32" ; then
CXXFLAGS="-mthreads $CXXFLAGS"
else
CXXFLAGS="-pthread $CXXFLAGS"
fi
AC_COMPILE_IFELSE([
AC_LANG_PROGRAM(
[[@%:@include <boost/thread/thread.hpp>]],
[[boost::thread_group thrds;
return 0;]])],
ax_cv_boost_thread=yes, ax_cv_boost_thread=no)
CXXFLAGS=$CXXFLAGS_SAVE
AC_LANG_POP([C++])
])
if test "x$ax_cv_boost_thread" = "xyes"; then
if test "x$host_os" = "xsolaris" ; then
BOOST_CPPFLAGS="-pthreads $BOOST_CPPFLAGS"
elif test "x$host_os" = "xmingw32" ; then
BOOST_CPPFLAGS="-mthreads $BOOST_CPPFLAGS"
else
BOOST_CPPFLAGS="-pthread $BOOST_CPPFLAGS"
fi
AC_SUBST(BOOST_CPPFLAGS)
AC_DEFINE(HAVE_BOOST_THREAD,,
[define if the Boost::Thread library is available])
BOOSTLIBDIR=`echo $BOOST_LDFLAGS | sed -e 's/@<:@^\/@:>@*//'`
LDFLAGS_SAVE=$LDFLAGS
case "x$host_os" in
*bsd* )
LDFLAGS="-pthread $LDFLAGS"
break;
;;
esac
if test "x$ax_boost_user_thread_lib" = "x"; then
for libextension in `ls -r $BOOSTLIBDIR/libboost_thread* 2>/dev/null | sed 's,.*/lib,,' | sed 's,\..*,,'`; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
if test "x$link_thread" != "xyes"; then
for libextension in `ls -r $BOOSTLIBDIR/boost_thread* 2>/dev/null | sed 's,.*/,,' | sed 's,\..*,,'`; do
ax_lib=${libextension}
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
fi
else
for ax_lib in $ax_boost_user_thread_lib boost_thread-$ax_boost_user_thread_lib; do
AC_CHECK_LIB($ax_lib, exit,
[link_thread="yes"; break],
[link_thread="no"])
done
fi
if test "x$ax_lib" = "x"; then
AC_MSG_ERROR(Could not find a version of the Boost::Thread library!)
fi
if test "x$link_thread" = "xno"; then
AC_MSG_ERROR(Could not link against $ax_lib !)
else
BOOST_THREAD_LIB="-l$ax_lib"
case "x$host_os" in
*bsd* )
BOOST_LDFLAGS="-pthread $BOOST_LDFLAGS"
break;
;;
xsolaris )
BOOST_THREAD_LIB="$BOOST_THREAD_LIB -lpthread"
break;
;;
xmingw32 )
break;
;;
* )
BOOST_THREAD_LIB="$BOOST_THREAD_LIB -lpthread"
break;
;;
esac
AC_SUBST(BOOST_THREAD_LIB)
fi
fi
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
fi
])

View File

@@ -1,5 +1,5 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx.html
# https://www.gnu.org/software/autoconf-archive/ax_cxx_compile_stdcxx.html
# ===========================================================================
#
# SYNOPSIS
@@ -33,21 +33,23 @@
# Copyright (c) 2014, 2015 Google Inc.; contributed by Alexey Sokolov <sokolov@google.com>
# Copyright (c) 2015 Paul Norman <penorman@mac.com>
# Copyright (c) 2015 Moritz Klammler <moritz@klammler.eu>
# Copyright (c) 2016, 2018 Krzesimir Nowak <qdlacz@gmail.com>
# Copyright (c) 2019 Enji Cooper <yaneurabeya@gmail.com>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 4
#serial 11
dnl This macro is based on the code from the AX_CXX_COMPILE_STDCXX_11 macro
dnl (serial version number 13).
AC_DEFUN([AX_CXX_COMPILE_STDCXX], [dnl
m4_if([$1], [11], [],
[$1], [14], [],
[$1], [17], [m4_fatal([support for C++17 not yet implemented in AX_CXX_COMPILE_STDCXX])],
m4_if([$1], [11], [ax_cxx_compile_alternatives="11 0x"],
[$1], [14], [ax_cxx_compile_alternatives="14 1y"],
[$1], [17], [ax_cxx_compile_alternatives="17 1z"],
[m4_fatal([invalid first argument `$1' to AX_CXX_COMPILE_STDCXX])])dnl
m4_if([$2], [], [],
[$2], [ext], [],
@@ -57,26 +59,13 @@ AC_DEFUN([AX_CXX_COMPILE_STDCXX], [dnl
[$3], [mandatory], [ax_cxx_compile_cxx$1_required=true],
[$3], [optional], [ax_cxx_compile_cxx$1_required=false],
[m4_fatal([invalid third argument `$3' to AX_CXX_COMPILE_STDCXX])])
m4_if([$4], [], [ax_cxx_compile_cxx$1_try_default=true],
[$4], [default], [ax_cxx_compile_cxx$1_try_default=true],
[$4], [nodefault], [ax_cxx_compile_cxx$1_try_default=false],
[m4_fatal([invalid fourth argument `$4' to AX_CXX_COMPILE_STDCXX])])
AC_LANG_PUSH([C++])dnl
ac_success=no
m4_if([$4], [nodefault], [], [dnl
AC_CACHE_CHECK(whether $CXX supports C++$1 features by default,
ax_cv_cxx_compile_cxx$1,
[AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_testbody_$1])],
[ax_cv_cxx_compile_cxx$1=yes],
[ax_cv_cxx_compile_cxx$1=no])])
if test x$ax_cv_cxx_compile_cxx$1 = xyes; then
ac_success=yes
fi])
m4_if([$2], [noext], [], [dnl
if test x$ac_success = xno; then
for switch in -std=gnu++$1 -std=gnu++0x; do
for alternative in ${ax_cxx_compile_alternatives}; do
switch="-std=gnu++${alternative}"
cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx$1_$switch])
AC_CACHE_CHECK(whether $CXX supports C++$1 features with $switch,
$cachevar,
@@ -102,22 +91,27 @@ AC_DEFUN([AX_CXX_COMPILE_STDCXX], [dnl
dnl HP's aCC needs +std=c++11 according to:
dnl http://h21007.www2.hp.com/portal/download/files/unprot/aCxx/PDF_Release_Notes/769149-001.pdf
dnl Cray's crayCC needs "-h std=c++11"
for switch in -std=c++$1 -std=c++0x +std=c++$1 "-h std=c++$1"; do
cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx$1_$switch])
AC_CACHE_CHECK(whether $CXX supports C++$1 features with $switch,
$cachevar,
[ac_save_CXX="$CXX"
CXX="$CXX $switch"
AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_testbody_$1])],
[eval $cachevar=yes],
[eval $cachevar=no])
CXX="$ac_save_CXX"])
if eval test x\$$cachevar = xyes; then
CXX="$CXX $switch"
if test -n "$CXXCPP" ; then
CXXCPP="$CXXCPP $switch"
for alternative in ${ax_cxx_compile_alternatives}; do
for switch in -std=c++${alternative} +std=c++${alternative} "-h std=c++${alternative}"; do
cachevar=AS_TR_SH([ax_cv_cxx_compile_cxx$1_$switch])
AC_CACHE_CHECK(whether $CXX supports C++$1 features with $switch,
$cachevar,
[ac_save_CXX="$CXX"
CXX="$CXX $switch"
AC_COMPILE_IFELSE([AC_LANG_SOURCE([_AX_CXX_COMPILE_STDCXX_testbody_$1])],
[eval $cachevar=yes],
[eval $cachevar=no])
CXX="$ac_save_CXX"])
if eval test x\$$cachevar = xyes; then
CXX="$CXX $switch"
if test -n "$CXXCPP" ; then
CXXCPP="$CXXCPP $switch"
fi
ac_success=yes
break
fi
ac_success=yes
done
if test x$ac_success = xyes; then
break
fi
done
@@ -154,6 +148,11 @@ m4_define([_AX_CXX_COMPILE_STDCXX_testbody_14],
_AX_CXX_COMPILE_STDCXX_testbody_new_in_14
)
m4_define([_AX_CXX_COMPILE_STDCXX_testbody_17],
_AX_CXX_COMPILE_STDCXX_testbody_new_in_11
_AX_CXX_COMPILE_STDCXX_testbody_new_in_14
_AX_CXX_COMPILE_STDCXX_testbody_new_in_17
)
dnl Tests for new features in C++11
@@ -191,11 +190,13 @@ namespace cxx11
struct Base
{
virtual ~Base() {}
virtual void f() {}
};
struct Derived : public Base
{
virtual ~Derived() override {}
virtual void f() override {}
};
@@ -524,7 +525,7 @@ namespace cxx14
}
namespace test_digit_seperators
namespace test_digit_separators
{
constexpr auto ten_million = 100'000'000;
@@ -566,3 +567,385 @@ namespace cxx14
#endif // __cplusplus >= 201402L
]])
dnl Tests for new features in C++17
m4_define([_AX_CXX_COMPILE_STDCXX_testbody_new_in_17], [[
// If the compiler admits that it is not ready for C++17, why torture it?
// Hopefully, this will speed up the test.
#ifndef __cplusplus
#error "This is not a C++ compiler"
#elif __cplusplus < 201703L
#error "This is not a C++17 compiler"
#else
#include <initializer_list>
#include <utility>
#include <type_traits>
namespace cxx17
{
namespace test_constexpr_lambdas
{
constexpr int foo = [](){return 42;}();
}
namespace test::nested_namespace::definitions
{
}
namespace test_fold_expression
{
template<typename... Args>
int multiply(Args... args)
{
return (args * ... * 1);
}
template<typename... Args>
bool all(Args... args)
{
return (args && ...);
}
}
namespace test_extended_static_assert
{
static_assert (true);
}
namespace test_auto_brace_init_list
{
auto foo = {5};
auto bar {5};
static_assert(std::is_same<std::initializer_list<int>, decltype(foo)>::value);
static_assert(std::is_same<int, decltype(bar)>::value);
}
namespace test_typename_in_template_template_parameter
{
template<template<typename> typename X> struct D;
}
namespace test_fallthrough_nodiscard_maybe_unused_attributes
{
int f1()
{
return 42;
}
[[nodiscard]] int f2()
{
[[maybe_unused]] auto unused = f1();
switch (f1())
{
case 17:
f1();
[[fallthrough]];
case 42:
f1();
}
return f1();
}
}
namespace test_extended_aggregate_initialization
{
struct base1
{
int b1, b2 = 42;
};
struct base2
{
base2() {
b3 = 42;
}
int b3;
};
struct derived : base1, base2
{
int d;
};
derived d1 {{1, 2}, {}, 4}; // full initialization
derived d2 {{}, {}, 4}; // value-initialized bases
}
namespace test_general_range_based_for_loop
{
struct iter
{
int i;
int& operator* ()
{
return i;
}
const int& operator* () const
{
return i;
}
iter& operator++()
{
++i;
return *this;
}
};
struct sentinel
{
int i;
};
bool operator== (const iter& i, const sentinel& s)
{
return i.i == s.i;
}
bool operator!= (const iter& i, const sentinel& s)
{
return !(i == s);
}
struct range
{
iter begin() const
{
return {0};
}
sentinel end() const
{
return {5};
}
};
void f()
{
range r {};
for (auto i : r)
{
[[maybe_unused]] auto v = i;
}
}
}
namespace test_lambda_capture_asterisk_this_by_value
{
struct t
{
int i;
int foo()
{
return [*this]()
{
return i;
}();
}
};
}
namespace test_enum_class_construction
{
enum class byte : unsigned char
{};
byte foo {42};
}
namespace test_constexpr_if
{
template <bool cond>
int f ()
{
if constexpr(cond)
{
return 13;
}
else
{
return 42;
}
}
}
namespace test_selection_statement_with_initializer
{
int f()
{
return 13;
}
int f2()
{
if (auto i = f(); i > 0)
{
return 3;
}
switch (auto i = f(); i + 4)
{
case 17:
return 2;
default:
return 1;
}
}
}
namespace test_template_argument_deduction_for_class_templates
{
template <typename T1, typename T2>
struct pair
{
pair (T1 p1, T2 p2)
: m1 {p1},
m2 {p2}
{}
T1 m1;
T2 m2;
};
void f()
{
[[maybe_unused]] auto p = pair{13, 42u};
}
}
namespace test_non_type_auto_template_parameters
{
template <auto n>
struct B
{};
B<5> b1;
B<'a'> b2;
}
namespace test_structured_bindings
{
int arr[2] = { 1, 2 };
std::pair<int, int> pr = { 1, 2 };
auto f1() -> int(&)[2]
{
return arr;
}
auto f2() -> std::pair<int, int>&
{
return pr;
}
struct S
{
int x1 : 2;
volatile double y1;
};
S f3()
{
return {};
}
auto [ x1, y1 ] = f1();
auto& [ xr1, yr1 ] = f1();
auto [ x2, y2 ] = f2();
auto& [ xr2, yr2 ] = f2();
const auto [ x3, y3 ] = f3();
}
namespace test_exception_spec_type_system
{
struct Good {};
struct Bad {};
void g1() noexcept;
void g2();
template<typename T>
Bad
f(T*, T*);
template<typename T1, typename T2>
Good
f(T1*, T2*);
static_assert (std::is_same_v<Good, decltype(f(g1, g2))>);
}
namespace test_inline_variables
{
template<class T> void f(T)
{}
template<class T> inline T g(T)
{
return T{};
}
template<> inline void f<>(int)
{}
template<> int g<>(int)
{
return 5;
}
}
} // namespace cxx17
#endif // __cplusplus < 201703L
]])

View File

@@ -1,223 +0,0 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_gcc_func_attribute.html
# ===========================================================================
#
# SYNOPSIS
#
# AX_GCC_FUNC_ATTRIBUTE(ATTRIBUTE)
#
# DESCRIPTION
#
# This macro checks if the compiler supports one of GCC's function
# attributes; many other compilers also provide function attributes with
# the same syntax. Compiler warnings are used to detect supported
# attributes as unsupported ones are ignored by default so quieting
# warnings when using this macro will yield false positives.
#
# The ATTRIBUTE parameter holds the name of the attribute to be checked.
#
# If ATTRIBUTE is supported define HAVE_FUNC_ATTRIBUTE_<ATTRIBUTE>.
#
# The macro caches its result in the ax_cv_have_func_attribute_<attribute>
# variable.
#
# The macro currently supports the following function attributes:
#
# alias
# aligned
# alloc_size
# always_inline
# artificial
# cold
# const
# constructor
# constructor_priority for constructor attribute with priority
# deprecated
# destructor
# dllexport
# dllimport
# error
# externally_visible
# flatten
# format
# format_arg
# gnu_inline
# hot
# ifunc
# leaf
# malloc
# noclone
# noinline
# nonnull
# noreturn
# nothrow
# optimize
# pure
# unused
# used
# visibility
# warning
# warn_unused_result
# weak
# weakref
#
# Unsuppored function attributes will be tested with a prototype returning
# an int and not accepting any arguments and the result of the check might
# be wrong or meaningless so use with care.
#
# LICENSE
#
# Copyright (c) 2013 Gabriele Svelto <gabriele.svelto@gmail.com>
#
# Copying and distribution of this file, with or without modification, are
# permitted in any medium without royalty provided the copyright notice
# and this notice are preserved. This file is offered as-is, without any
# warranty.
#serial 3
AC_DEFUN([AX_GCC_FUNC_ATTRIBUTE], [
AS_VAR_PUSHDEF([ac_var], [ax_cv_have_func_attribute_$1])
AC_CACHE_CHECK([for __attribute__(($1))], [ac_var], [
AC_LINK_IFELSE([AC_LANG_PROGRAM([
m4_case([$1],
[alias], [
int foo( void ) { return 0; }
int bar( void ) __attribute__(($1("foo")));
],
[aligned], [
int foo( void ) __attribute__(($1(32)));
],
[alloc_size], [
void *foo(int a) __attribute__(($1(1)));
],
[always_inline], [
inline __attribute__(($1)) int foo( void ) { return 0; }
],
[artificial], [
inline __attribute__(($1)) int foo( void ) { return 0; }
],
[cold], [
int foo( void ) __attribute__(($1));
],
[const], [
int foo( void ) __attribute__(($1));
],
[constructor_priority], [
int foo( void ) __attribute__((__constructor__(65535/2)));
],
[constructor], [
int foo( void ) __attribute__(($1));
],
[deprecated], [
int foo( void ) __attribute__(($1("")));
],
[destructor], [
int foo( void ) __attribute__(($1));
],
[dllexport], [
__attribute__(($1)) int foo( void ) { return 0; }
],
[dllimport], [
int foo( void ) __attribute__(($1));
],
[error], [
int foo( void ) __attribute__(($1("")));
],
[externally_visible], [
int foo( void ) __attribute__(($1));
],
[flatten], [
int foo( void ) __attribute__(($1));
],
[format], [
int foo(const char *p, ...) __attribute__(($1(printf, 1, 2)));
],
[format_arg], [
char *foo(const char *p) __attribute__(($1(1)));
],
[gnu_inline], [
inline __attribute__(($1)) int foo( void ) { return 0; }
],
[hot], [
int foo( void ) __attribute__(($1));
],
[ifunc], [
int my_foo( void ) { return 0; }
static int (*resolve_foo(void))(void) { return my_foo; }
int foo( void ) __attribute__(($1("resolve_foo")));
],
[leaf], [
__attribute__(($1)) int foo( void ) { return 0; }
],
[malloc], [
void *foo( void ) __attribute__(($1));
],
[noclone], [
int foo( void ) __attribute__(($1));
],
[noinline], [
__attribute__(($1)) int foo( void ) { return 0; }
],
[nonnull], [
int foo(char *p) __attribute__(($1(1)));
],
[noreturn], [
void foo( void ) __attribute__(($1));
],
[nothrow], [
int foo( void ) __attribute__(($1));
],
[optimize], [
__attribute__(($1(3))) int foo( void ) { return 0; }
],
[pure], [
int foo( void ) __attribute__(($1));
],
[unused], [
int foo( void ) __attribute__(($1));
],
[used], [
int foo( void ) __attribute__(($1));
],
[visibility], [
int foo_def( void ) __attribute__(($1("default")));
int foo_hid( void ) __attribute__(($1("hidden")));
int foo_int( void ) __attribute__(($1("internal")));
int foo_pro( void ) __attribute__(($1("protected")));
],
[warning], [
int foo( void ) __attribute__(($1("")));
],
[warn_unused_result], [
int foo( void ) __attribute__(($1));
],
[weak], [
int foo( void ) __attribute__(($1));
],
[weakref], [
static int foo( void ) { return 0; }
static int bar( void ) __attribute__(($1("foo")));
],
[
m4_warn([syntax], [Unsupported attribute $1, the test may fail])
int foo( void ) __attribute__(($1));
]
)], [])
],
dnl GCC doesn't exit with an error if an unknown attribute is
dnl provided but only outputs a warning, so accept the attribute
dnl only if no warning were issued.
[AS_IF([test -s conftest.err],
[AS_VAR_SET([ac_var], [no])],
[AS_VAR_SET([ac_var], [yes])])],
[AS_VAR_SET([ac_var], [no])])
])
AS_IF([test yes = AS_VAR_GET([ac_var])],
[AC_DEFINE_UNQUOTED(AS_TR_CPP(HAVE_FUNC_ATTRIBUTE_$1), 1,
[Define to 1 if the system has the `$1' function attribute])], [])
AS_VAR_POPDEF([ac_var])
])

View File

@@ -1,5 +1,5 @@
# ===========================================================================
# http://www.gnu.org/software/autoconf-archive/ax_pthread.html
# https://www.gnu.org/software/autoconf-archive/ax_pthread.html
# ===========================================================================
#
# SYNOPSIS
@@ -55,6 +55,7 @@
#
# Copyright (c) 2008 Steven G. Johnson <stevenj@alum.mit.edu>
# Copyright (c) 2011 Daniel Richard G. <skunk@iSKUNK.ORG>
# Copyright (c) 2019 Marc Stevens <marc.stevens@cwi.nl>
#
# This program is free software: you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by the
@@ -67,7 +68,7 @@
# Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program. If not, see <http://www.gnu.org/licenses/>.
# with this program. If not, see <https://www.gnu.org/licenses/>.
#
# As a special exception, the respective Autoconf Macro's copyright owner
# gives unlimited permission to copy, distribute and modify the configure
@@ -82,7 +83,7 @@
# modified version of the Autoconf Macro, you may extend this special
# exception to the GPL to apply to your modified version as well.
#serial 23
#serial 27
AU_ALIAS([ACX_PTHREAD], [AX_PTHREAD])
AC_DEFUN([AX_PTHREAD], [
@@ -123,10 +124,12 @@ fi
# (e.g. DEC) have both -lpthread and -lpthreads, where one of the
# libraries is broken (non-POSIX).
# Create a list of thread flags to try. Items starting with a "-" are
# C compiler flags, and other items are library names, except for "none"
# which indicates that we try without any flags at all, and "pthread-config"
# which is a program returning the flags for the Pth emulation library.
# Create a list of thread flags to try. Items with a "," contain both
# C compiler flags (before ",") and linker flags (after ","). Other items
# starting with a "-" are C compiler flags, and remaining items are
# library names, except for "none" which indicates that we try without
# any flags at all, and "pthread-config" which is a program returning
# the flags for the Pth emulation library.
ax_pthread_flags="pthreads none -Kthread -pthread -pthreads -mthreads pthread --thread-safe -mt pthread-config"
@@ -194,14 +197,47 @@ case $host_os in
# that too in a future libc.) So we'll check first for the
# standard Solaris way of linking pthreads (-mt -lpthread).
ax_pthread_flags="-mt,pthread pthread $ax_pthread_flags"
ax_pthread_flags="-mt,-lpthread pthread $ax_pthread_flags"
;;
esac
# Are we compiling with Clang?
AC_CACHE_CHECK([whether $CC is Clang],
[ax_cv_PTHREAD_CLANG],
[ax_cv_PTHREAD_CLANG=no
# Note that Autoconf sets GCC=yes for Clang as well as GCC
if test "x$GCC" = "xyes"; then
AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
[/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
# if defined(__clang__) && defined(__llvm__)
AX_PTHREAD_CC_IS_CLANG
# endif
],
[ax_cv_PTHREAD_CLANG=yes])
fi
])
ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
# GCC generally uses -pthread, or -pthreads on some platforms (e.g. SPARC)
# Note that for GCC and Clang -pthread generally implies -lpthread,
# except when -nostdlib is passed.
# This is problematic using libtool to build C++ shared libraries with pthread:
# [1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25460
# [2] https://bugzilla.redhat.com/show_bug.cgi?id=661333
# [3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=468555
# To solve this, first try -pthread together with -lpthread for GCC
AS_IF([test "x$GCC" = "xyes"],
[ax_pthread_flags="-pthread -pthreads $ax_pthread_flags"])
[ax_pthread_flags="-pthread,-lpthread -pthread -pthreads $ax_pthread_flags"])
# Clang takes -pthread (never supported any other flag), but we'll try with -lpthread first
AS_IF([test "x$ax_pthread_clang" = "xyes"],
[ax_pthread_flags="-pthread,-lpthread -pthread"])
# The presence of a feature test macro requesting re-entrant function
# definitions is, on some systems, a strong hint that pthreads support is
@@ -224,25 +260,86 @@ AS_IF([test "x$ax_pthread_check_macro" = "x--"],
[ax_pthread_check_cond=0],
[ax_pthread_check_cond="!defined($ax_pthread_check_macro)"])
# Are we compiling with Clang?
AC_CACHE_CHECK([whether $CC is Clang],
[ax_cv_PTHREAD_CLANG],
[ax_cv_PTHREAD_CLANG=no
# Note that Autoconf sets GCC=yes for Clang as well as GCC
if test "x$GCC" = "xyes"; then
AC_EGREP_CPP([AX_PTHREAD_CC_IS_CLANG],
[/* Note: Clang 2.7 lacks __clang_[a-z]+__ */
# if defined(__clang__) && defined(__llvm__)
AX_PTHREAD_CC_IS_CLANG
# endif
],
[ax_cv_PTHREAD_CLANG=yes])
fi
])
ax_pthread_clang="$ax_cv_PTHREAD_CLANG"
if test "x$ax_pthread_ok" = "xno"; then
for ax_pthread_try_flag in $ax_pthread_flags; do
case $ax_pthread_try_flag in
none)
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
*,*)
PTHREAD_CFLAGS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\1/"`
PTHREAD_LIBS=`echo $ax_pthread_try_flag | sed "s/^\(.*\),\(.*\)$/\2/"`
AC_MSG_CHECKING([whether pthreads work with "$PTHREAD_CFLAGS" and "$PTHREAD_LIBS"])
;;
-*)
AC_MSG_CHECKING([whether pthreads work with $ax_pthread_try_flag])
PTHREAD_CFLAGS="$ax_pthread_try_flag"
;;
pthread-config)
AC_CHECK_PROG([ax_pthread_config], [pthread-config], [yes], [no])
AS_IF([test "x$ax_pthread_config" = "xno"], [continue])
PTHREAD_CFLAGS="`pthread-config --cflags`"
PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`"
;;
*)
AC_MSG_CHECKING([for the pthreads library -l$ax_pthread_try_flag])
PTHREAD_LIBS="-l$ax_pthread_try_flag"
;;
esac
ax_pthread_save_CFLAGS="$CFLAGS"
ax_pthread_save_LIBS="$LIBS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
LIBS="$PTHREAD_LIBS $LIBS"
# Check for various functions. We must include pthread.h,
# since some functions may be macros. (On the Sequent, we
# need a special flag -Kthread to make this header compile.)
# We check for pthread_join because it is in -lpthread on IRIX
# while pthread_create is in libc. We check for pthread_attr_init
# due to DEC craziness with -lpthreads. We check for
# pthread_cleanup_push because it is one of the few pthread
# functions on Solaris that doesn't have a non-functional libc stub.
# We try pthread_create on general principles.
AC_LINK_IFELSE([AC_LANG_PROGRAM([#include <pthread.h>
# if $ax_pthread_check_cond
# error "$ax_pthread_check_macro must be defined"
# endif
static void *some_global = NULL;
static void routine(void *a)
{
/* To avoid any unused-parameter or
unused-but-set-parameter warning. */
some_global = a;
}
static void *start_routine(void *a) { return a; }],
[pthread_t th; pthread_attr_t attr;
pthread_create(&th, 0, start_routine, 0);
pthread_join(th, 0);
pthread_attr_init(&attr);
pthread_cleanup_push(routine, 0);
pthread_cleanup_pop(0) /* ; */])],
[ax_pthread_ok=yes],
[])
CFLAGS="$ax_pthread_save_CFLAGS"
LIBS="$ax_pthread_save_LIBS"
AC_MSG_RESULT([$ax_pthread_ok])
AS_IF([test "x$ax_pthread_ok" = "xyes"], [break])
PTHREAD_LIBS=""
PTHREAD_CFLAGS=""
done
fi
ax_pthread_clang_warning=no
# Clang needs special handling, because older versions handle the -pthread
# option in a rather... idiosyncratic way
@@ -261,11 +358,6 @@ if test "x$ax_pthread_clang" = "xyes"; then
# -pthread does define _REENTRANT, and while the Darwin headers
# ignore this macro, third-party headers might not.)
PTHREAD_CFLAGS="-pthread"
PTHREAD_LIBS=
ax_pthread_ok=yes
# However, older versions of Clang make a point of warning the user
# that, in an invocation where only linking and no compilation is
# taking place, the -pthread option has no effect ("argument unused
@@ -320,78 +412,7 @@ if test "x$ax_pthread_clang" = "xyes"; then
fi # $ax_pthread_clang = yes
if test "x$ax_pthread_ok" = "xno"; then
for ax_pthread_try_flag in $ax_pthread_flags; do
case $ax_pthread_try_flag in
none)
AC_MSG_CHECKING([whether pthreads work without any flags])
;;
-mt,pthread)
AC_MSG_CHECKING([whether pthreads work with -mt -lpthread])
PTHREAD_CFLAGS="-mt"
PTHREAD_LIBS="-lpthread"
;;
-*)
AC_MSG_CHECKING([whether pthreads work with $ax_pthread_try_flag])
PTHREAD_CFLAGS="$ax_pthread_try_flag"
;;
pthread-config)
AC_CHECK_PROG([ax_pthread_config], [pthread-config], [yes], [no])
AS_IF([test "x$ax_pthread_config" = "xno"], [continue])
PTHREAD_CFLAGS="`pthread-config --cflags`"
PTHREAD_LIBS="`pthread-config --ldflags` `pthread-config --libs`"
;;
*)
AC_MSG_CHECKING([for the pthreads library -l$ax_pthread_try_flag])
PTHREAD_LIBS="-l$ax_pthread_try_flag"
;;
esac
ax_pthread_save_CFLAGS="$CFLAGS"
ax_pthread_save_LIBS="$LIBS"
CFLAGS="$CFLAGS $PTHREAD_CFLAGS"
LIBS="$PTHREAD_LIBS $LIBS"
# Check for various functions. We must include pthread.h,
# since some functions may be macros. (On the Sequent, we
# need a special flag -Kthread to make this header compile.)
# We check for pthread_join because it is in -lpthread on IRIX
# while pthread_create is in libc. We check for pthread_attr_init
# due to DEC craziness with -lpthreads. We check for
# pthread_cleanup_push because it is one of the few pthread
# functions on Solaris that doesn't have a non-functional libc stub.
# We try pthread_create on general principles.
AC_LINK_IFELSE([AC_LANG_PROGRAM([#include <pthread.h>
# if $ax_pthread_check_cond
# error "$ax_pthread_check_macro must be defined"
# endif
static void routine(void *a) { a = 0; }
static void *start_routine(void *a) { return a; }],
[pthread_t th; pthread_attr_t attr;
pthread_create(&th, 0, start_routine, 0);
pthread_join(th, 0);
pthread_attr_init(&attr);
pthread_cleanup_push(routine, 0);
pthread_cleanup_pop(0) /* ; */])],
[ax_pthread_ok=yes],
[])
CFLAGS="$ax_pthread_save_CFLAGS"
LIBS="$ax_pthread_save_LIBS"
AC_MSG_RESULT([$ax_pthread_ok])
AS_IF([test "x$ax_pthread_ok" = "xyes"], [break])
PTHREAD_LIBS=""
PTHREAD_CFLAGS=""
done
fi
# Various other checks:
if test "x$ax_pthread_ok" = "xyes"; then
@@ -438,7 +459,8 @@ if test "x$ax_pthread_ok" = "xyes"; then
AC_CACHE_CHECK([for PTHREAD_PRIO_INHERIT],
[ax_cv_PTHREAD_PRIO_INHERIT],
[AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <pthread.h>]],
[[int i = PTHREAD_PRIO_INHERIT;]])],
[[int i = PTHREAD_PRIO_INHERIT;
return i;]])],
[ax_cv_PTHREAD_PRIO_INHERIT=yes],
[ax_cv_PTHREAD_PRIO_INHERIT=no])
])

View File

@@ -6,7 +6,9 @@ AC_DEFUN([BITCOIN_FIND_BDB48],[
AC_ARG_VAR(BDB_CFLAGS, [C compiler flags for BerkeleyDB, bypasses autodetection])
AC_ARG_VAR(BDB_LIBS, [Linker flags for BerkeleyDB, bypasses autodetection])
if test "x$BDB_CFLAGS" = "x"; then
if test "x$use_bdb" = "xno"; then
use_bdb=no
elif test "x$BDB_CFLAGS" = "x"; then
AC_MSG_CHECKING([for Berkeley DB C++ headers])
BDB_CPPFLAGS=
bdbpath=X
@@ -44,25 +46,30 @@ AC_DEFUN([BITCOIN_FIND_BDB48],[
],[])
done
if test "x$bdbpath" = "xX"; then
use_bdb=no
AC_MSG_RESULT([no])
AC_MSG_ERROR([libdb_cxx headers missing, ]AC_PACKAGE_NAME[ requires this library for wallet functionality (--disable-wallet to disable wallet functionality)])
AC_MSG_ERROR([libdb_cxx headers missing, ]AC_PACKAGE_NAME[ requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)])
elif test "x$bdb48path" = "xX"; then
BITCOIN_SUBDIR_TO_INCLUDE(BDB_CPPFLAGS,[${bdbpath}],db_cxx)
AC_ARG_WITH([incompatible-bdb],[AS_HELP_STRING([--with-incompatible-bdb], [allow using a bdb version other than 4.8])],[
AC_MSG_WARN([Found Berkeley DB other than 4.8; wallets opened by this build will not be portable!])
AC_MSG_WARN([Found Berkeley DB other than 4.8; BDB wallets opened by this build will not be portable!])
],[
AC_MSG_ERROR([Found Berkeley DB other than 4.8, required for portable wallets (--with-incompatible-bdb to ignore or --disable-wallet to disable wallet functionality)])
AC_MSG_ERROR([Found Berkeley DB other than 4.8, required for portable BDB wallets (--with-incompatible-bdb to ignore or --without-bdb to disable BDB wallet support)])
])
use_bdb=yes
else
BITCOIN_SUBDIR_TO_INCLUDE(BDB_CPPFLAGS,[${bdb48path}],db_cxx)
bdbpath="${bdb48path}"
use_bdb=yes
fi
else
BDB_CPPFLAGS=${BDB_CFLAGS}
fi
AC_SUBST(BDB_CPPFLAGS)
if test "x$BDB_LIBS" = "x"; then
if test "x$use_bdb" = "xno"; then
use_bdb=no
elif test "x$BDB_LIBS" = "x"; then
# TODO: Ideally this could find the library version and make sure it matches the headers being used
for searchlib in db_cxx-4.8 db_cxx db4_cxx; do
AC_CHECK_LIB([$searchlib],[main],[
@@ -71,8 +78,12 @@ AC_DEFUN([BITCOIN_FIND_BDB48],[
])
done
if test "x$BDB_LIBS" = "x"; then
AC_MSG_ERROR([libdb_cxx missing, ]AC_PACKAGE_NAME[ requires this library for wallet functionality (--disable-wallet to disable wallet functionality)])
AC_MSG_ERROR([libdb_cxx missing, ]AC_PACKAGE_NAME[ requires this library for BDB wallet support (--without-bdb to disable BDB wallet support)])
fi
fi
AC_SUBST(BDB_LIBS)
if test "x$use_bdb" != "xno"; then
AC_SUBST(BDB_LIBS)
AC_DEFINE([USE_BDB], [1], [Define if BDB support should be compiled in])
use_bdb=yes
fi
])

View File

@@ -64,6 +64,13 @@ AC_DEFUN([BITCOIN_QT_INIT],[
],
[bitcoin_qt_want_version=auto])
AS_IF([test "x$with_gui" = xqt5_debug],
[AS_CASE([$host],
[*darwin*], [qt_lib_suffix=_debug],
[*mingw*], [qt_lib_suffix=d],
[qt_lib_suffix= ]); bitcoin_qt_want_version=qt5],
[qt_lib_suffix= ])
AC_ARG_WITH([qt-incdir],[AS_HELP_STRING([--with-qt-incdir=INC_DIR],[specify qt include path (overridden by pkgconfig)])], [qt_include_path=$withval], [])
AC_ARG_WITH([qt-libdir],[AS_HELP_STRING([--with-qt-libdir=LIB_DIR],[specify qt lib path (overridden by pkgconfig)])], [qt_lib_path=$withval], [])
AC_ARG_WITH([qt-plugindir],[AS_HELP_STRING([--with-qt-plugindir=PLUGIN_DIR],[specify qt plugin path (overridden by pkgconfig)])], [qt_plugin_path=$withval], [])
@@ -72,41 +79,39 @@ AC_DEFUN([BITCOIN_QT_INIT],[
AC_ARG_WITH([qtdbus],
[AS_HELP_STRING([--with-qtdbus],
[enable DBus support (default is yes if qt is enabled and QtDBus is found)])],
[enable DBus support (default is yes if qt is enabled and QtDBus is found, except on Android)])],
[use_dbus=$withval],
[use_dbus=auto])
dnl Android doesn't support D-Bus and certainly doesn't use it for notifications
case $host in
*android*)
if test "x$use_dbus" != xyes; then
use_dbus=no
fi
;;
esac
AC_SUBST(QT_TRANSLATION_DIR,$qt_translation_path)
])
dnl Find the appropriate version of Qt libraries and includes.
dnl Inputs: $1: Whether or not pkg-config should be used. yes|no. Default: yes.
dnl Inputs: $2: If $1 is "yes" and --with-gui=auto, which qt version should be
dnl tried first.
dnl Outputs: See _BITCOIN_QT_FIND_LIBS_*
dnl Find Qt libraries and includes.
dnl
dnl BITCOIN_QT_CONFIGURE([MINIMUM-VERSION])
dnl
dnl Outputs: See _BITCOIN_QT_FIND_LIBS
dnl Outputs: Sets variables for all qt-related tools.
dnl Outputs: bitcoin_enable_qt, bitcoin_enable_qt_dbus, bitcoin_enable_qt_test
AC_DEFUN([BITCOIN_QT_CONFIGURE],[
use_pkgconfig=$1
if test "x$use_pkgconfig" = x; then
use_pkgconfig=yes
fi
if test "x$use_pkgconfig" = xyes; then
BITCOIN_QT_CHECK([_BITCOIN_QT_FIND_LIBS_WITH_PKGCONFIG])
else
BITCOIN_QT_CHECK([_BITCOIN_QT_FIND_LIBS_WITHOUT_PKGCONFIG])
fi
qt_version=">= $1"
qt_lib_prefix="Qt5"
BITCOIN_QT_CHECK([_BITCOIN_QT_FIND_LIBS])
dnl This is ugly and complicated. Yuck. Works as follows:
dnl For Qt5, we can check a header to find out whether Qt is build
dnl statically. When Qt is built statically, some plugins must be linked into
dnl the final binary as well.
dnl With Qt5, languages moved into core and the WindowsIntegration plugin was
dnl added.
dnl _BITCOIN_QT_CHECK_STATIC_PLUGINS does a quick link-check and appends the
dnl results to QT_LIBS.
dnl We check a header to find out whether Qt is built statically.
dnl When Qt is built statically, some plugins must be linked into
dnl the final binary as well. _BITCOIN_QT_CHECK_STATIC_PLUGIN does
dnl a quick link-check and appends the results to QT_LIBS.
BITCOIN_QT_CHECK([
TEMP_CPPFLAGS=$CPPFLAGS
TEMP_CXXFLAGS=$CXXFLAGS
@@ -114,28 +119,59 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
CXXFLAGS="$PIC_FLAGS $CXXFLAGS"
_BITCOIN_QT_IS_STATIC
if test "x$bitcoin_cv_static_qt" = xyes; then
_BITCOIN_QT_FIND_STATIC_PLUGINS
_BITCOIN_QT_CHECK_STATIC_LIBS
if test "x$qt_plugin_path" != x; then
if test -d "$qt_plugin_path/platforms"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms"
fi
if test -d "$qt_plugin_path/styles"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/styles"
fi
if test -d "$qt_plugin_path/accessible"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/accessible"
fi
if test -d "$qt_plugin_path/platforms/android"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms/android -lqtfreetype -lEGL"
fi
fi
AC_DEFINE(QT_STATICPLUGIN, 1, [Define this symbol if qt plugins are static])
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QMinimalIntegrationPlugin)],[-lqminimal])
AC_DEFINE(QT_QPA_PLATFORM_MINIMAL, 1, [Define this symbol if the minimal qt platform exists])
if test "x$TARGET_OS" != xandroid; then
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QMinimalIntegrationPlugin], [-lqminimal])
AC_DEFINE(QT_QPA_PLATFORM_MINIMAL, 1, [Define this symbol if the minimal qt platform exists])
fi
if test "x$TARGET_OS" = xwindows; then
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QWindowsIntegrationPlugin)],[-lqwindows])
dnl Linking against wtsapi32 is required. See #17749 and
dnl https://bugreports.qt.io/browse/QTBUG-27097.
AX_CHECK_LINK_FLAG([-lwtsapi32], [QT_LIBS="$QT_LIBS -lwtsapi32"], [AC_MSG_ERROR([could not link against -lwtsapi32])])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QWindowsIntegrationPlugin], [-lqwindows])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QWindowsVistaStylePlugin], [-lqwindowsvistastyle])
AC_DEFINE(QT_QPA_PLATFORM_WINDOWS, 1, [Define this symbol if the qt platform is windows])
elif test "x$TARGET_OS" = xlinux; then
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QXcbIntegrationPlugin)],[-lqxcb -lxcb-static])
dnl workaround for https://bugreports.qt.io/browse/QTBUG-74874
AX_CHECK_LINK_FLAG([-lxcb-shm], [QT_LIBS="$QT_LIBS -lxcb-shm"], [AC_MSG_ERROR([could not link against -lxcb-shm])])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QXcbIntegrationPlugin], [-lqxcb])
AC_DEFINE(QT_QPA_PLATFORM_XCB, 1, [Define this symbol if the qt platform is xcb])
elif test "x$TARGET_OS" = xdarwin; then
AX_CHECK_LINK_FLAG([[-framework IOKit]],[QT_LIBS="$QT_LIBS -framework IOKit"],[AC_MSG_ERROR(could not iokit framework)])
_BITCOIN_QT_CHECK_STATIC_PLUGINS([Q_IMPORT_PLUGIN(QCocoaIntegrationPlugin)],[-lqcocoa])
AX_CHECK_LINK_FLAG([[-framework Carbon]],[QT_LIBS="$QT_LIBS -framework Carbon"],[AC_MSG_ERROR(could not link against Carbon framework)])
AX_CHECK_LINK_FLAG([[-framework IOSurface]],[QT_LIBS="$QT_LIBS -framework IOSurface"],[AC_MSG_ERROR(could not link against IOSurface framework)])
AX_CHECK_LINK_FLAG([[-framework Metal]],[QT_LIBS="$QT_LIBS -framework Metal"],[AC_MSG_ERROR(could not link against Metal framework)])
AX_CHECK_LINK_FLAG([[-framework QuartzCore]],[QT_LIBS="$QT_LIBS -framework QuartzCore"],[AC_MSG_ERROR(could not link against QuartzCore framework)])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QCocoaIntegrationPlugin], [-lqcocoa])
_BITCOIN_QT_CHECK_STATIC_PLUGIN([QMacStylePlugin], [-lqmacstyle])
AC_DEFINE(QT_QPA_PLATFORM_COCOA, 1, [Define this symbol if the qt platform is cocoa])
elif test "x$TARGET_OS" = xandroid; then
QT_LIBS="-Wl,--export-dynamic,--undefined=JNI_OnLoad -lqtforandroid -ljnigraphics -landroid -lqtfreetype $QT_LIBS"
AC_DEFINE(QT_QPA_PLATFORM_ANDROID, 1, [Define this symbol if the qt platform is android])
fi
fi
CPPFLAGS=$TEMP_CPPFLAGS
CXXFLAGS=$TEMP_CXXFLAGS
])
if test "x$use_pkgconfig$qt_bin_path" = xyes; then
qt_bin_path="`$PKG_CONFIG --variable=host_bins Qt5Core 2>/dev/null`"
if test "x$qt_bin_path" = x; then
qt_bin_path="`$PKG_CONFIG --variable=host_bins ${qt_lib_prefix}Core 2>/dev/null`"
fi
if test "x$use_hardening" != xno; then
@@ -190,13 +226,14 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
BITCOIN_QT_PATH_PROGS([RCC], [rcc-qt5 rcc5 rcc], $qt_bin_path)
BITCOIN_QT_PATH_PROGS([LRELEASE], [lrelease-qt5 lrelease5 lrelease], $qt_bin_path)
BITCOIN_QT_PATH_PROGS([LUPDATE], [lupdate-qt5 lupdate5 lupdate],$qt_bin_path, yes)
BITCOIN_QT_PATH_PROGS([LCONVERT], [lconvert-qt5 lconvert5 lconvert], $qt_bin_path, yes)
MOC_DEFS='-DHAVE_CONFIG_H -I$(srcdir)'
case $host in
*darwin*)
BITCOIN_QT_CHECK([
MOC_DEFS="${MOC_DEFS} -DQ_OS_MAC"
base_frameworks="-framework Foundation -framework ApplicationServices -framework AppKit"
base_frameworks="-framework Foundation -framework AppKit"
AX_CHECK_LINK_FLAG([[$base_frameworks]],[QT_LIBS="$QT_LIBS $base_frameworks"],[AC_MSG_ERROR(could not find base frameworks)])
])
;;
@@ -208,7 +245,7 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
dnl enable qt support
AC_MSG_CHECKING(whether to build ]AC_PACKAGE_NAME[ GUI)
AC_MSG_CHECKING([whether to build ]AC_PACKAGE_NAME[ GUI])
BITCOIN_QT_CHECK([
bitcoin_enable_qt=yes
bitcoin_enable_qt_test=yes
@@ -223,13 +260,16 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
AC_MSG_ERROR([libQtDBus not found. Install libQtDBus or remove --with-qtdbus.])
fi
if test "x$LUPDATE" = x; then
AC_MSG_WARN([lupdate is required to update qt translations])
AC_MSG_WARN([lupdate tool is required to update Qt translations.])
fi
if test "x$LCONVERT" = x; then
AC_MSG_WARN([lconvert tool is required to update Qt translations.])
fi
],[
bitcoin_enable_qt=no
])
if test x$bitcoin_enable_qt = xyes; then
AC_MSG_RESULT([$bitcoin_enable_qt ($QT_LIB_PREFIX)])
AC_MSG_RESULT([$bitcoin_enable_qt ($qt_lib_prefix)])
else
AC_MSG_RESULT([$bitcoin_enable_qt])
fi
@@ -246,61 +286,20 @@ AC_DEFUN([BITCOIN_QT_CONFIGURE],[
AC_SUBST(MOC_DEFS)
])
dnl All macros below are internal and should _not_ be used from the main
dnl configure.ac.
dnl ----
dnl All macros below are internal and should _not_ be used from configure.ac.
dnl Internal. Check included version of Qt against minimum specified in doc/dependencies.md
dnl Requires: INCLUDES must be populated as necessary.
dnl Output: bitcoin_cv_qt5=yes|no
AC_DEFUN([_BITCOIN_QT_CHECK_QT5],[
AC_CACHE_CHECK(for Qt 5, bitcoin_cv_qt5,[
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <QtCore/qconfig.h>
#ifndef QT_VERSION
# include <QtCore/qglobal.h>
#endif
]],
[[
#if QT_VERSION < 0x050501
choke
#endif
]])],
[bitcoin_cv_qt5=yes],
[bitcoin_cv_qt5=no])
])])
dnl Internal. Check if the included version of Qt is greater than Qt58.
dnl Requires: INCLUDES must be populated as necessary.
dnl Output: bitcoin_cv_qt58=yes|no
AC_DEFUN([_BITCOIN_QT_CHECK_QT58],[
AC_CACHE_CHECK(for > Qt 5.7, bitcoin_cv_qt58,[
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <QtCore/qconfig.h>
#ifndef QT_VERSION
# include <QtCore/qglobal.h>
#endif
]],
[[
#if QT_VERSION_MINOR < 8
choke
#endif
]])],
[bitcoin_cv_qt58=yes],
[bitcoin_cv_qt58=no])
])])
dnl Internal. Check if the linked version of Qt was built as static libs.
dnl Requires: Qt5.
dnl Internal. Check if the linked version of Qt was built statically.
dnl
dnl _BITCOIN_QT_IS_STATIC
dnl ---------------------
dnl
dnl Requires: INCLUDES and LIBS must be populated as necessary.
dnl Output: bitcoin_cv_static_qt=yes|no
dnl Output: Defines QT_STATICPLUGIN if plugins are static.
AC_DEFUN([_BITCOIN_QT_IS_STATIC],[
AC_CACHE_CHECK(for static Qt, bitcoin_cv_static_qt,[
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <QtCore/qconfig.h>
#ifndef QT_VERSION OR QT_VERSION_STR
#ifndef QT_VERSION
# include <QtCore/qglobal.h>
#endif
]],
@@ -312,200 +311,88 @@ AC_DEFUN([_BITCOIN_QT_IS_STATIC],[
[bitcoin_cv_static_qt=yes],
[bitcoin_cv_static_qt=no])
])
if test "x$bitcoin_cv_static_qt" = xyes; then
AC_DEFINE(QT_STATICPLUGIN, 1, [Define this symbol for static Qt plugins])
fi
])
dnl Internal. Check if the link-requirements for static plugins are met.
dnl Internal. Check if the link-requirements for a static plugin are met.
dnl
dnl _BITCOIN_QT_CHECK_STATIC_PLUGIN(PLUGIN, LIBRARIES)
dnl --------------------------------------------------
dnl
dnl Requires: INCLUDES and LIBS must be populated as necessary.
dnl Inputs: $1: A series of Q_IMPORT_PLUGIN().
dnl Inputs: $1: A static plugin name.
dnl Inputs: $2: The libraries that resolve $1.
dnl Output: QT_LIBS is prepended or configure exits.
AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_PLUGINS],[
AC_MSG_CHECKING(for static Qt plugins: $2)
AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_PLUGIN], [
AC_MSG_CHECKING([for $1 ($2)])
CHECK_STATIC_PLUGINS_TEMP_LIBS="$LIBS"
LIBS="$2 $QT_LIBS $LIBS"
LIBS="$2${qt_lib_suffix} $QT_LIBS $LIBS"
AC_LINK_IFELSE([AC_LANG_PROGRAM([[
#define QT_STATICPLUGIN
#include <QtPlugin>
$1]],
[[return 0;]])],
[AC_MSG_RESULT(yes); QT_LIBS="$2 $QT_LIBS"],
[AC_MSG_RESULT(no); BITCOIN_QT_FAIL(Could not resolve: $2)])
#include <QtPlugin>
Q_IMPORT_PLUGIN($1)
]])],
[AC_MSG_RESULT([yes]); QT_LIBS="$2${qt_lib_suffix} $QT_LIBS"],
[AC_MSG_RESULT([no]); BITCOIN_QT_FAIL([$1 not found.])])
LIBS="$CHECK_STATIC_PLUGINS_TEMP_LIBS"
])
dnl Internal. Find paths necessary for linking qt static plugins
dnl Inputs: qt_plugin_path. optional.
dnl Outputs: QT_LIBS is appended
AC_DEFUN([_BITCOIN_QT_FIND_STATIC_PLUGINS],[
if test "x$qt_plugin_path" != x; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/platforms"
if test -d "$qt_plugin_path/accessible"; then
QT_LIBS="$QT_LIBS -L$qt_plugin_path/accessible"
fi
if test "x$use_pkgconfig" = xyes; then
: dnl
m4_ifdef([PKG_CHECK_MODULES],[
if test x$bitcoin_cv_qt58 = xno; then
PKG_CHECK_MODULES([QTPLATFORM], [Qt5PlatformSupport], [QT_LIBS="$QTPLATFORM_LIBS $QT_LIBS"])
else
PKG_CHECK_MODULES([QTFONTDATABASE], [Qt5FontDatabaseSupport], [QT_LIBS="-lQt5FontDatabaseSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTEVENTDISPATCHER], [Qt5EventDispatcherSupport], [QT_LIBS="-lQt5EventDispatcherSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTTHEME], [Qt5ThemeSupport], [QT_LIBS="-lQt5ThemeSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTDEVICEDISCOVERY], [Qt5DeviceDiscoverySupport], [QT_LIBS="-lQt5DeviceDiscoverySupport $QT_LIBS"])
PKG_CHECK_MODULES([QTACCESSIBILITY], [Qt5AccessibilitySupport], [QT_LIBS="-lQt5AccessibilitySupport $QT_LIBS"])
PKG_CHECK_MODULES([QTFB], [Qt5FbSupport], [QT_LIBS="-lQt5FbSupport $QT_LIBS"])
fi
if test "x$TARGET_OS" = xlinux; then
PKG_CHECK_MODULES([QTXCBQPA], [Qt5XcbQpa], [QT_LIBS="$QTXCBQPA_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xdarwin; then
PKG_CHECK_MODULES([QTCLIPBOARD], [Qt5ClipboardSupport], [QT_LIBS="-lQt5ClipboardSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTGRAPHICS], [Qt5GraphicsSupport], [QT_LIBS="-lQt5GraphicsSupport $QT_LIBS"])
PKG_CHECK_MODULES([QTCGL], [Qt5CglSupport], [QT_LIBS="-lQt5CglSupport $QT_LIBS"])
fi
])
else
if test "x$TARGET_OS" = xwindows; then
AC_CACHE_CHECK(for Qt >= 5.6, bitcoin_cv_need_platformsupport,[
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <QtCore/qconfig.h>
#ifndef QT_VERSION
# include <QtCore/qglobal.h>
#endif
]],
[[
#if QT_VERSION < 0x050600 || QT_VERSION_MINOR < 6
choke
#endif
]])],
[bitcoin_cv_need_platformsupport=yes],
[bitcoin_cv_need_platformsupport=no])
])
if test "x$bitcoin_cv_need_platformsupport" = xyes; then
if test x$bitcoin_cv_qt58 = xno; then
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}PlatformSupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXPlatformSupport not found)))
else
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}FontDatabaseSupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXFontDatabaseSupport not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}EventDispatcherSupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXEventDispatcherSupport not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}ThemeSupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXThemeSupport not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}FbSupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXFbSupport not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}DeviceDiscoverySupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXDeviceDiscoverySupport not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}AccessibilitySupport],[main],,BITCOIN_QT_FAIL(lib$QT_LIB_PREFIXAccessibilitySupport not found)))
QT_LIBS="$QT_LIBS -lversion -ldwmapi -luxtheme"
fi
fi
fi
fi
fi
dnl Internal. Check Qt static libs with PKG_CHECK_MODULES.
dnl
dnl _BITCOIN_QT_CHECK_STATIC_LIBS
dnl -----------------------------
dnl
dnl Outputs: QT_LIBS is prepended.
AC_DEFUN([_BITCOIN_QT_CHECK_STATIC_LIBS], [
PKG_CHECK_MODULES([QT_ACCESSIBILITY], [${qt_lib_prefix}AccessibilitySupport${qt_lib_suffix}], [QT_LIBS="$QT_ACCESSIBILITY_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_DEVICEDISCOVERY], [${qt_lib_prefix}DeviceDiscoverySupport${qt_lib_suffix}], [QT_LIBS="$QT_DEVICEDISCOVERY_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_EDID], [${qt_lib_prefix}EdidSupport${qt_lib_suffix}], [QT_LIBS="$QT_EDID_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_EVENTDISPATCHER], [${qt_lib_prefix}EventDispatcherSupport${qt_lib_suffix}], [QT_LIBS="$QT_EVENTDISPATCHER_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_FB], [${qt_lib_prefix}FbSupport${qt_lib_suffix}], [QT_LIBS="$QT_FB_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_FONTDATABASE], [${qt_lib_prefix}FontDatabaseSupport${qt_lib_suffix}], [QT_LIBS="$QT_FONTDATABASE_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_THEME], [${qt_lib_prefix}ThemeSupport${qt_lib_suffix}], [QT_LIBS="$QT_THEME_LIBS $QT_LIBS"])
if test "x$TARGET_OS" = xlinux; then
PKG_CHECK_MODULES([QT_INPUT], [${qt_lib_prefix}XcbQpa], [QT_LIBS="$QT_INPUT_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_SERVICE], [${qt_lib_prefix}ServiceSupport], [QT_LIBS="$QT_SERVICE_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_XCBQPA], [${qt_lib_prefix}XcbQpa], [QT_LIBS="$QT_XCBQPA_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xdarwin; then
PKG_CHECK_MODULES([QT_CLIPBOARD], [${qt_lib_prefix}ClipboardSupport${qt_lib_suffix}], [QT_LIBS="$QT_CLIPBOARD_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_GRAPHICS], [${qt_lib_prefix}GraphicsSupport${qt_lib_suffix}], [QT_LIBS="$QT_GRAPHICS_LIBS $QT_LIBS"])
PKG_CHECK_MODULES([QT_SERVICE], [${qt_lib_prefix}ServiceSupport${qt_lib_suffix}], [QT_LIBS="$QT_SERVICE_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xwindows; then
PKG_CHECK_MODULES([QT_WINDOWSUIAUTOMATION], [${qt_lib_prefix}WindowsUIAutomationSupport${qt_lib_suffix}], [QT_LIBS="$QT_WINDOWSUIAUTOMATION_LIBS $QT_LIBS"])
elif test "x$TARGET_OS" = xandroid; then
PKG_CHECK_MODULES([QT_EGL], [${qt_lib_prefix}EglSupport], [QT_LIBS="$QT_EGL_LIBS $QT_LIBS"])
fi
])
dnl Internal. Find Qt libraries using pkg-config.
dnl Inputs: bitcoin_qt_want_version (from --with-gui=). The version to check
dnl first.
dnl Inputs: $1: If bitcoin_qt_want_version is "auto", check for this version
dnl first.
dnl
dnl _BITCOIN_QT_FIND_LIBS
dnl ---------------------
dnl
dnl Outputs: All necessary QT_* variables are set.
dnl Outputs: have_qt_test and have_qt_dbus are set (if applicable) to yes|no.
AC_DEFUN([_BITCOIN_QT_FIND_LIBS_WITH_PKGCONFIG],[
m4_ifdef([PKG_CHECK_MODULES],[
QT_LIB_PREFIX=Qt5
qt5_modules="Qt5Core Qt5Gui Qt5Network Qt5Widgets"
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT5], [$qt5_modules], [QT_INCLUDES="$QT5_CFLAGS"; QT_LIBS="$QT5_LIBS" have_qt=yes],[have_qt=no])
if test "x$have_qt" != xyes; then
have_qt=no
BITCOIN_QT_FAIL([Qt dependencies not found])
fi
])
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_TEST], [${QT_LIB_PREFIX}Test], [QT_TEST_INCLUDES="$QT_TEST_CFLAGS"; have_qt_test=yes], [have_qt_test=no])
if test "x$use_dbus" != xno; then
PKG_CHECK_MODULES([QT_DBUS], [${QT_LIB_PREFIX}DBus], [QT_DBUS_INCLUDES="$QT_DBUS_CFLAGS"; have_qt_dbus=yes], [have_qt_dbus=no])
fi
])
])
true; dnl
])
dnl Internal. Find Qt libraries without using pkg-config. Version is deduced
dnl from the discovered headers.
dnl Inputs: bitcoin_qt_want_version (from --with-gui=). The version to use.
dnl If "auto", the version will be discovered by _BITCOIN_QT_CHECK_QT5.
dnl Outputs: All necessary QT_* variables are set.
dnl Outputs: have_qt_test and have_qt_dbus are set (if applicable) to yes|no.
AC_DEFUN([_BITCOIN_QT_FIND_LIBS_WITHOUT_PKGCONFIG],[
TEMP_CPPFLAGS="$CPPFLAGS"
TEMP_CXXFLAGS="$CXXFLAGS"
CXXFLAGS="$PIC_FLAGS $CXXFLAGS"
TEMP_LIBS="$LIBS"
AC_DEFUN([_BITCOIN_QT_FIND_LIBS],[
BITCOIN_QT_CHECK([
if test "x$qt_include_path" != x; then
QT_INCLUDES="-I$qt_include_path -I$qt_include_path/QtCore -I$qt_include_path/QtGui -I$qt_include_path/QtWidgets -I$qt_include_path/QtNetwork -I$qt_include_path/QtTest -I$qt_include_path/QtDBus"
CPPFLAGS="$QT_INCLUDES $CPPFLAGS"
fi
PKG_CHECK_MODULES([QT_CORE], [${qt_lib_prefix}Core${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_CORE_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_CORE_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Core${qt_lib_suffix} $qt_version not found])])
])
BITCOIN_QT_CHECK([AC_CHECK_HEADER([QtPlugin],,BITCOIN_QT_FAIL(QtCore headers missing))])
BITCOIN_QT_CHECK([AC_CHECK_HEADER([QApplication],, BITCOIN_QT_FAIL(QtGui headers missing))])
BITCOIN_QT_CHECK([AC_CHECK_HEADER([QLocalSocket],, BITCOIN_QT_FAIL(QtNetwork headers missing))])
BITCOIN_QT_CHECK([
if test "x$bitcoin_qt_want_version" = xauto; then
_BITCOIN_QT_CHECK_QT5
_BITCOIN_QT_CHECK_QT58
fi
QT_LIB_PREFIX=Qt5
PKG_CHECK_MODULES([QT_GUI], [${qt_lib_prefix}Gui${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_GUI_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_GUI_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Gui${qt_lib_suffix} $qt_version not found])])
])
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_WIDGETS], [${qt_lib_prefix}Widgets${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_WIDGETS_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_WIDGETS_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Widgets${qt_lib_suffix} $qt_version not found])])
])
BITCOIN_QT_CHECK([
PKG_CHECK_MODULES([QT_NETWORK], [${qt_lib_prefix}Network${qt_lib_suffix} $qt_version], [QT_INCLUDES="$QT_NETWORK_CFLAGS $QT_INCLUDES" QT_LIBS="$QT_NETWORK_LIBS $QT_LIBS"],
[BITCOIN_QT_FAIL([${qt_lib_prefix}Network${qt_lib_suffix} $qt_version not found])])
])
BITCOIN_QT_CHECK([
LIBS=
if test "x$qt_lib_path" != x; then
LIBS="$LIBS -L$qt_lib_path"
fi
if test "x$TARGET_OS" = xwindows; then
AC_CHECK_LIB([imm32], [main],, BITCOIN_QT_FAIL(libimm32 not found))
fi
])
BITCOIN_QT_CHECK(AC_CHECK_LIB([z] ,[main],,AC_MSG_WARN([zlib not found. Assuming qt has it built-in])))
if test x$bitcoin_cv_qt58 = xno; then
BITCOIN_QT_CHECK(AC_SEARCH_LIBS([png_error] ,[qtpng png],,AC_MSG_WARN([libpng not found. Assuming qt has it built-in])))
BITCOIN_QT_CHECK(AC_SEARCH_LIBS([pcre16_exec], [qtpcre pcre16],,AC_MSG_WARN([libpcre16 not found. Assuming qt has it built-in])))
else
BITCOIN_QT_CHECK(AC_SEARCH_LIBS([png_error] ,[qtlibpng png],,AC_MSG_WARN([libpng not found. Assuming qt has it built-in])))
BITCOIN_QT_CHECK(AC_SEARCH_LIBS([pcre2_match_16], [qtpcre2 libqtpcre2],,AC_MSG_WARN([libqtpcre2 not found. Assuming qt has it built-in])))
fi
BITCOIN_QT_CHECK(AC_SEARCH_LIBS([hb_ot_tags_from_script] ,[qtharfbuzzng qtharfbuzz harfbuzz],,AC_MSG_WARN([libharfbuzz not found. Assuming qt has it built-in or support is disabled])))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Core] ,[main],,BITCOIN_QT_FAIL(lib${QT_LIB_PREFIX}Core not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Gui] ,[main],,BITCOIN_QT_FAIL(lib${QT_LIB_PREFIX}Gui not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Network],[main],,BITCOIN_QT_FAIL(lib${QT_LIB_PREFIX}Network not found)))
BITCOIN_QT_CHECK(AC_CHECK_LIB([${QT_LIB_PREFIX}Widgets],[main],,BITCOIN_QT_FAIL(lib${QT_LIB_PREFIX}Widgets not found)))
QT_LIBS="$LIBS"
LIBS="$TEMP_LIBS"
BITCOIN_QT_CHECK([
LIBS=
if test "x$qt_lib_path" != x; then
LIBS="-L$qt_lib_path"
fi
AC_CHECK_LIB([${QT_LIB_PREFIX}Test], [main],, have_qt_test=no)
AC_CHECK_HEADER([QTest],, have_qt_test=no)
QT_TEST_LIBS="$LIBS"
PKG_CHECK_MODULES([QT_TEST], [${qt_lib_prefix}Test${qt_lib_suffix} $qt_version], [QT_TEST_INCLUDES="$QT_TEST_CFLAGS"; have_qt_test=yes], [have_qt_test=no])
if test "x$use_dbus" != xno; then
LIBS=
if test "x$qt_lib_path" != x; then
LIBS="-L$qt_lib_path"
fi
AC_CHECK_LIB([${QT_LIB_PREFIX}DBus], [main],, have_qt_dbus=no)
AC_CHECK_HEADER([QtDBus],, have_qt_dbus=no)
QT_DBUS_LIBS="$LIBS"
PKG_CHECK_MODULES([QT_DBUS], [${qt_lib_prefix}DBus $qt_version], [QT_DBUS_INCLUDES="$QT_DBUS_CFLAGS"; have_qt_dbus=yes], [have_qt_dbus=no])
fi
])
CPPFLAGS="$TEMP_CPPFLAGS"
CXXFLAGS="$TEMP_CXXFLAGS"
LIBS="$TEMP_LIBS"
])

View File

@@ -12,8 +12,17 @@ dnl warranty.
m4_define([_CHECK_ATOMIC_testbody], [[
#include <atomic>
#include <cstdint>
#include <chrono>
using namespace std::chrono_literals;
int main() {
std::atomic<bool> lock{true};
std::atomic_exchange(&lock, false);
std::atomic<std::chrono::seconds> t{0s};
t.store(2s);
std::atomic<int64_t> a{};
int64_t v = 5;

36
build-aux/m4/l_socket.m4 Normal file
View File

@@ -0,0 +1,36 @@
# Illumos/SmartOS requires linking with -lsocket if
# using getifaddrs & freeifaddrs
m4_define([_CHECK_SOCKET_testbody], [[
#include <sys/types.h>
#include <ifaddrs.h>
int main() {
struct ifaddrs *ifaddr;
getifaddrs(&ifaddr);
freeifaddrs(ifaddr);
}
]])
AC_DEFUN([CHECK_SOCKET], [
AC_LANG_PUSH(C++)
AC_MSG_CHECKING([whether ifaddrs funcs can be used without link library])
AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_SOCKET_testbody])],[
AC_MSG_RESULT([yes])
],[
AC_MSG_RESULT([no])
LIBS="$LIBS -lsocket"
AC_MSG_CHECKING([whether getifaddrs needs -lsocket])
AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_SOCKET_testbody])],[
AC_MSG_RESULT([yes])
],[
AC_MSG_RESULT([no])
AC_MSG_FAILURE([cannot figure out how to use getifaddrs])
])
])
AC_LANG_POP
])

15
build_msvc/.gitignore vendored
View File

@@ -8,7 +8,20 @@ packages/*
*/Release
*/x64
*.vcxproj.user
*.vcxproj
# .vcxproj files that are auto-generated by the msvc-autogen.py script.
libbitcoin_cli/libbitcoin_cli.vcxproj
libbitcoin_common/libbitcoin_common.vcxproj
libbitcoin_crypto/libbitcoin_crypto.vcxproj
libbitcoin_server/libbitcoin_server.vcxproj
libbitcoin_util/libbitcoin_util.vcxproj
libbitcoin_wallet_tool/libbitcoin_wallet_tool.vcxproj
libbitcoin_wallet/libbitcoin_wallet.vcxproj
libbitcoin_zmq/libbitcoin_zmq.vcxproj
bench_bitcoin/bench_bitcoin.vcxproj
libtest_util/libtest_util.vcxproj
*/Win32
libbitcoin_qt/QtGeneratedFiles/*
test_bitcoin-qt/QtGeneratedFiles/*
vcpkg_installed

View File

@@ -3,7 +3,7 @@ Building Bitcoin Core with Visual Studio
Introduction
---------------------
Solution and project files to build the Bitcoin Core applications `msbuild` or Visual Studio can be found in the build_msvc directory. The build has been tested with Visual Studio 2017 and 2019.
Solution and project files to build the Bitcoin Core applications `msbuild` or Visual Studio can be found in the `build_msvc` directory. The build has been tested with Visual Studio 2019 (building with earlier versions of Visual Studio should not be expected to work).
Building with Visual Studio is an alternative to the Linux based [cross-compiler build](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md).
@@ -12,9 +12,9 @@ Quick Start
The minimal steps required to build Bitcoin Core with the msbuild toolchain are below. More detailed instructions are contained in the following sections.
```
vcpkg install --triplet x64-windows-static boost-filesystem boost-signals2 boost-test libevent openssl zeromq berkeleydb rapidcheck double-conversion
py -3 build_msvc\msvc-autogen.py
msbuild /m build_msvc\bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
cd build_msvc
py -3 msvc-autogen.py
msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
```
Dependencies
@@ -27,61 +27,43 @@ Options for installing the dependencies in a Visual Studio compatible manner are
- Download the source code, build each dependency, add the required include paths, link libraries and binary tools to the Visual Studio project files.
- Use [nuget](https://www.nuget.org/) packages with the understanding that any binary files have been compiled by an untrusted third party.
The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are:
The [external dependencies](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md) required for building are listed in the `build_msvc/vcpkg.json` file. To ensure `msbuild` project files automatically install the `vcpkg` dependencies use:
- Berkeley DB
- Boost
- DoubleConversion
- libevent
- OpenSSL
- Qt5
- RapidCheck
- ZeroMQ
```
vcpkg integrate install
```
Qt
---------------------
All the Bitcoin Core applications are configured to build with static linking. In order to build the Bitcoin Core Qt applications a static build of Qt is required.
In order to build Bitcoin Core a static build of Qt is required. The runtime library version (e.g. v142) and platform type (x86 or x64) must also match.
The runtime library version (e.g. v141, v142) and platform type (x86 or x64) must also match. OpenSSL must also be linked into the Qt binaries in order to provide full functionality of the Bitcoin Core Qt programs. An example of the configure command to build Qtv5.9.7 locally to link with Bitcoin Core is shown below (adjust paths accordingly), note it can be expected that the configure and subsequent build will fail numerous times until dependency issues are resolved.
Some prebuilt x64 versions of Qt can be downloaded from [here](https://github.com/sipsorcery/qt_win_binary/releases). Please be aware these downloads are NOT officially sanctioned by Bitcoin Core and are provided for developer convenience only. They should NOT be used for builds that will be used in a production environment or with real funds.
````
..\Qtv5.9.7_src\configure -developer-build -confirm-license -debug-and-release -opensource -platform win32-msvc -opengl desktop -no-shared -static -no-static-runtime -mp -qt-zlib -qt-pcre -qt-libpng -ltcg -make libs -make tools -no-libjpeg -nomake examples -no-compile-examples -no-dbus -no-libudev -no-qml-debug -no-icu -no-gtk -no-opengles3 -no-angle -no-sql-sqlite -no-sql-odbc -no-sqlite -no-libudev -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcanvas3d -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -nomake tests -openssl-linked -IC:\Dev\github\vcpkg\installed\x64-windows-static\include -LC:\Dev\github\vcpkg\installed\x64-windows-static\lib OPENSSL_LIBS="-llibeay32 -lssleay32 -lgdi32 -luser32 -lwsock32 -ladvapi32" -prefix C:\Qt5.9.7_ssl_x64_static_vs2017
````
To determine which Qt prebuilt version to download open the `.appveyor.yml` file and note the `QT_DOWNLOAD_URL`. When extracting the zip file the destination path must be set to `C:\`. This is due to the way that Qt includes, libraries and tools use internal paths.
A prebuilt version for x64 and Visual C++ runtime v141 (Visual Studio 2017) can be downloaded from [here](https://github.com/sipsorcery/qt_win_binary/releases). Please be aware this download is NOT an officially sanctioned Bitcoin Core distribution and is provided for developer convenience. It should NOT be used for builds that will be used in a production environment or with real funds.
To build Bitcoin Core without Qt unload or disable the bitcoin-qt, libbitcoin_qt and test_bitcoin-qt projects.
To build Bitcoin Core without Qt unload or disable the `bitcoin-qt`, `libbitcoin_qt` and `test_bitcoin-qt` projects.
Building
---------------------
The instructions below use `vcpkg` to install the dependencies.
- Clone `vcpkg` from the [github repository](https://github.com/Microsoft/vcpkg) and install as per the instructions in the main README.md.
- Install the required packages (replace x64 with x86 as required):
- Install [`vcpkg`](https://github.com/Microsoft/vcpkg).
- Use Python to generate `*.vcxproj` from Makefile
```
PS >.\vcpkg install --triplet x64-windows-static boost-filesystem boost-signals2 boost-test libevent openssl zeromq berkeleydb rapidcheck double-conversion
PS >py -3 msvc-autogen.py
```
- Use Python to generate *.vcxproj from Makefile
- An optional step is to adjust the settings in the `build_msvc` directory and the `common.init.vcxproj` file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set.
```
PS >py -3 msvc-autogen.py
```
- An optional step is to adjust the settings in the build_msvc directory and the common.init.vcxproj file. This project file contains settings that are common to all projects such as the runtime library version and target Windows SDK version. The Qt directories can also be set.
- Build with Visual Studio 2017 or msbuild.
- To build from the command line with the Visual Studio 2019 toolchain use:
```
msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /t:build
```
- Build with Visual Studio 2019 or msbuild.
```
msbuild /m bitcoin.sln /p:Platform=x64 /p:Configuration=Release /p:PlatformToolset=v142 /t:build
```
- Alternatively, open the `build_msvc/bitcoin.sln` file in Visual Studio 2019.
AppVeyor
---------------------
@@ -93,3 +75,25 @@ For safety reasons the Bitcoin Core .appveyor.yml file has the artifact options
#- 7z a bitcoin-%APPVEYOR_BUILD_VERSION%.zip %APPVEYOR_BUILD_FOLDER%\build_msvc\%platform%\%configuration%\*.exe
#- path: bitcoin-%APPVEYOR_BUILD_VERSION%.zip
```
Security
---------------------
[Base address randomization](https://docs.microsoft.com/en-us/cpp/build/reference/dynamicbase-use-address-space-layout-randomization?view=msvc-160) is used to make Bitcoin Core more secure. When building Bitcoin using the `build_msvc` process base address randomization can be disabled by editing `common.init.vcproj` to change `RandomizedBaseAddress` from `true` to `false` and then rebuilding the project.
To check if `bitcoind` has `RandomizedBaseAddress` enabled or disabled run
```
.\dumpbin.exe /headers src/bitcoind.exe
```
If is it enabled then in the output `Dynamic base` will be listed in the `DLL characteristics` under `OPTIONAL HEADER VALUES` as shown below
```
8160 DLL characteristics
High Entropy Virtual Addresses
Dynamic base
NX compatible
Terminal Server Aware
```
This may not disable all stack randomization as versions of windows employ additional stack randomization protections. These protections must be turned off in the OS configuration.

View File

@@ -42,6 +42,9 @@
<ProjectReference Include="..\libleveldb\libleveldb.vcxproj">
<Project>{18430fef-6b61-4c53-b396-718e02850f1b}</Project>
</ProjectReference>
<ProjectReference Include="..\libtest_util\libtest_util.vcxproj">
<Project>{1e065f03-3566-47d0-8fa9-daa72b084e7d}</Project>
</ProjectReference>
</ItemGroup>
<Target Name="RawBenchHeaderGen" BeforeTargets="PrepareForBuild">
<PropertyGroup>

View File

@@ -50,12 +50,14 @@
</ProjectReference>
</ItemGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ItemDefinitionGroup Condition="'$(Configuration)'=='Release'">
<ClCompile>
<AdditionalIncludeDirectories>$(QtIncludes);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Windows</SubSystem>
<AdditionalDependencies>$(QtReleaseLibraries);%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalOptions>/ignore:4206 /LTCG:OFF</AdditionalOptions>
</Link>
<ResourceCompile>
<AdditionalIncludeDirectories>..\..\src;</AdditionalIncludeDirectories>
@@ -63,12 +65,13 @@
</ResourceCompile>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ItemDefinitionGroup Condition="'$(Configuration)'=='Debug'">
<ClCompile>
<AdditionalIncludeDirectories>$(QtIncludes);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<AdditionalDependencies>$(QtDebugLibraries);%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalOptions>/ignore:4206</AdditionalOptions>
</Link>
<ResourceCompile>
<AdditionalIncludeDirectories>..\..\src;</AdditionalIncludeDirectories>

View File

@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\common.init.vcxproj" />
<PropertyGroup Label="Globals">
<ProjectGuid>{D3022AF6-AD33-4CE3-B358-87CB6A1B29CF}</ProjectGuid>
</PropertyGroup>
<PropertyGroup Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\bitcoin-util.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">
<Project>{7c87e378-df58-482e-aa2f-1bc129bc19ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_crypto\libbitcoin_crypto.vcxproj">
<Project>{6190199c-6cf4-4dad-bfbd-93fa72a760c1}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_util\libbitcoin_util.vcxproj">
<Project>{b53a5535-ee9d-4c6f-9a26-f79ee3bc3754}</Project>
</ProjectReference>
<ProjectReference Include="..\libunivalue\libunivalue.vcxproj">
<Project>{5724ba7d-a09a-4ba8-800b-c4c1561b3d69}</Project>
</ProjectReference>
<ProjectReference Include="..\libsecp256k1\libsecp256k1.vcxproj">
<Project>{bb493552-3b8c-4a8c-bf69-a6e7a51d2ea6}</Project>
</ProjectReference>
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<Import Project="..\common.vcxproj" />
</Project>

View File

@@ -32,6 +32,8 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bench_bitcoin", "bench_bitc
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-tx", "bitcoin-tx\bitcoin-tx.vcxproj", "{D3022AF6-AD33-4CE3-B358-87CB6A1B29CF}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-util", "bitcoin-util\bitcoin-util.vcxproj", "{D3022AF6-AD33-4CE3-B358-87CB6A1B29CF}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-wallet", "bitcoin-wallet\bitcoin-wallet.vcxproj", "{84DE8790-EDE3-4483-81AC-C32F15E861F4}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbitcoin_wallet_tool", "libbitcoin_wallet_tool\libbitcoin_wallet_tool.vcxproj", "{F91AC55E-6F5E-4C58-9AC5-B40DB7DEEF93}"
@@ -44,6 +46,10 @@ Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbitcoin_qt", "libbitcoin
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoin-qt", "bitcoin-qt\bitcoin-qt.vcxproj", "{7E99172D-7FF2-4CB6-B736-AC9B76ED412A}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libtest_util", "libtest_util\libtest_util.vcxproj", "{868474FD-35F6-4400-8EED-30A33E7521D4}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "test_bitcoin-qt", "test_bitcoin-qt\test_bitcoin-qt.vcxproj", "{51201D5E-D939-4854-AE9D-008F03FF518E}"
EndProject
Global
GlobalSection(SolutionConfigurationPlatforms) = preSolution
Debug|x64 = Debug|x64
@@ -220,11 +226,29 @@ Global
{7E99172D-7FF2-4CB6-B736-AC9B76ED412A}.Release|x64.Build.0 = Release|x64
{7E99172D-7FF2-4CB6-B736-AC9B76ED412A}.Release|x86.ActiveCfg = Release|Win32
{7E99172D-7FF2-4CB6-B736-AC9B76ED412A}.Release|x86.Build.0 = Release|Win32
{868474FD-35F6-4400-8EED-30A33E7521D4}.Debug|x64.ActiveCfg = Debug|x64
{868474FD-35F6-4400-8EED-30A33E7521D4}.Debug|x64.Build.0 = Debug|x64
{868474FD-35F6-4400-8EED-30A33E7521D4}.Debug|x86.ActiveCfg = Debug|Win32
{868474FD-35F6-4400-8EED-30A33E7521D4}.Debug|x86.Build.0 = Debug|Win32
{868474FD-35F6-4400-8EED-30A33E7521D4}.Release|x64.ActiveCfg = Release|x64
{868474FD-35F6-4400-8EED-30A33E7521D4}.Release|x64.Build.0 = Release|x64
{868474FD-35F6-4400-8EED-30A33E7521D4}.Release|x86.ActiveCfg = Release|Win32
{868474FD-35F6-4400-8EED-30A33E7521D4}.Release|x86.Build.0 = Release|Win32
{51201D5E-D939-4854-AE9D-008F03FF518E}.Debug|x64.ActiveCfg = Debug|x64
{51201D5E-D939-4854-AE9D-008F03FF518E}.Debug|x64.Build.0 = Debug|x64
{51201D5E-D939-4854-AE9D-008F03FF518E}.Debug|x86.ActiveCfg = Debug|Win32
{51201D5E-D939-4854-AE9D-008F03FF518E}.Debug|x86.Build.0 = Debug|Win32
{51201D5E-D939-4854-AE9D-008F03FF518E}.Release|x64.ActiveCfg = Release|x64
{51201D5E-D939-4854-AE9D-008F03FF518E}.Release|x64.Build.0 = Release|x64
{51201D5E-D939-4854-AE9D-008F03FF518E}.Release|x86.ActiveCfg = Release|Win32
{51201D5E-D939-4854-AE9D-008F03FF518E}.Release|x86.Build.0 = Release|Win32
EndGlobalSection
GlobalSection(SolutionProperties) = preSolution
HideSolutionNode = FALSE
EndGlobalSection
GlobalSection(ExtensibilityGlobals) = postSolution
SolutionGuid = {8AA72EDA-2CD4-4564-B1E4-688B760EEEE9}
SolutionGuid = {8AA72EDA-2CD4-4564-B1E4-688B760EEEE9}
SolutionGuid = {8607C0F4-F33D-41B8-8D51-18E366A0F8DF}
SolutionGuid = {58AAB032-7274-49BD-845E-5EF4DBB69B70}
EndGlobalSection
EndGlobal

View File

@@ -1,3 +1,7 @@
// Copyright (c) 2018-2020 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_BITCOIN_CONFIG_H
#define BITCOIN_BITCOIN_CONFIG_H
@@ -8,16 +12,13 @@
#define CLIENT_VERSION_BUILD 0
/* Version is release */
#define CLIENT_VERSION_IS_RELEASE false
#define CLIENT_VERSION_IS_RELEASE true
/* Major version */
#define CLIENT_VERSION_MAJOR 0
#define CLIENT_VERSION_MAJOR 22
/* Minor version */
#define CLIENT_VERSION_MINOR 18
/* Build revision */
#define CLIENT_VERSION_REVISION 99
#define CLIENT_VERSION_MINOR 0
/* Copyright holder(s) before %s replacement */
#define COPYRIGHT_HOLDERS "The %s developers"
@@ -29,35 +30,32 @@
#define COPYRIGHT_HOLDERS_SUBSTITUTION "Bitcoin Core"
/* Copyright year */
#define COPYRIGHT_YEAR 2019
#define COPYRIGHT_YEAR 2021
/* Define to 1 to enable wallet functions */
#define ENABLE_WALLET 1
/* Define to 1 to enable BDB wallet */
#define USE_BDB 1
/* Define to 1 to enable SQLite wallet */
#define USE_SQLITE 1
/* Define to 1 to enable ZMQ functions */
#define ENABLE_ZMQ 1
/* parameter and return value type for __fdelt_chk */
/* #undef FDELT_TYPE */
/* define if the Boost library is available */
#define HAVE_BOOST /**/
/* define if the Boost::Chrono library is available */
#define HAVE_BOOST_CHRONO /**/
/* define if the Boost::Filesystem library is available */
#define HAVE_BOOST_FILESYSTEM /**/
/* define if the Boost::PROGRAM_OPTIONS library is available */
#define HAVE_BOOST_PROGRAM_OPTIONS /**/
/* define if external signer support is enabled (requires Boost::Process) */
#define ENABLE_EXTERNAL_SIGNER /**/
/* define if the Boost::System library is available */
#define HAVE_BOOST_SYSTEM /**/
/* define if the Boost::Thread library is available */
#define HAVE_BOOST_THREAD /**/
/* define if the Boost::Unit_Test_Framework library is available */
#define HAVE_BOOST_UNIT_TEST_FRAMEWORK /**/
@@ -94,13 +92,9 @@
don't. */
#define HAVE_DECL_BSWAP_64 0
/* Define to 1 if you have the declaration of `daemon', and to 0 if you don't.
/* Define to 1 if you have the declaration of `fork', and to 0 if you don't.
*/
#define HAVE_DECL_DAEMON 0
/* Define to 1 if you have the declaration of `EVP_MD_CTX_new', and to 0 if
you don't. */
//#define HAVE_DECL_EVP_MD_CTX_NEW 1
#define HAVE_DECL_FORK 0
/* Define to 1 if you have the declaration of `htobe16', and to 0 if you
don't. */
@@ -138,6 +132,10 @@
don't. */
#define HAVE_DECL_LE64TOH 0
/* Define to 1 if you have the declaration of `setsid', and to 0 if you don't.
*/
#define HAVE_DECL_SETSID 0
/* Define to 1 if you have the declaration of `strerror_r', and to 0 if you
don't. */
#define HAVE_DECL_STRERROR_R 0
@@ -146,18 +144,6 @@
don't. */
#define HAVE_DECL_STRNLEN 1
/* Define to 1 if you have the declaration of `__builtin_clz', and to 0 if you
don't. */
//#define HAVE_DECL___BUILTIN_CLZ 1
/* Define to 1 if you have the declaration of `__builtin_clzl', and to 0 if
you don't. */
//#define HAVE_DECL___BUILTIN_CLZL 1
/* Define to 1 if you have the declaration of `__builtin_clzll', and to 0 if
you don't. */
//#define HAVE_DECL___BUILTIN_CLZLL 1
/* Define to 1 if you have the <dlfcn.h> header file. */
/* #undef HAVE_DLFCN_H */
@@ -183,75 +169,6 @@
/* Define to 1 if you have the <inttypes.h> header file. */
#define HAVE_INTTYPES_H 1
/* Define to 1 if you have the `advapi32' library (-ladvapi32). */
#define HAVE_LIBADVAPI32 1
/* Define to 1 if you have the `comctl32' library (-lcomctl32). */
#define HAVE_LIBCOMCTL32 1
/* Define to 1 if you have the `comdlg32' library (-lcomdlg32). */
#define HAVE_LIBCOMDLG32 1
/* Define to 1 if you have the `crypt32' library (-lcrypt32). */
#define HAVE_LIBCRYPT32 1
/* Define to 1 if you have the `gdi32' library (-lgdi32). */
#define HAVE_LIBGDI32 1
/* Define to 1 if you have the `imm32' library (-limm32). */
#define HAVE_LIBIMM32 1
/* Define to 1 if you have the `iphlpapi' library (-liphlpapi). */
#define HAVE_LIBIPHLPAPI 1
/* Define to 1 if you have the `kernel32' library (-lkernel32). */
#define HAVE_LIBKERNEL32 1
/* Define to 1 if you have the `mingwthrd' library (-lmingwthrd). */
#define HAVE_LIBMINGWTHRD 1
/* Define to 1 if you have the `mswsock' library (-lmswsock). */
#define HAVE_LIBMSWSOCK 1
/* Define to 1 if you have the `ole32' library (-lole32). */
#define HAVE_LIBOLE32 1
/* Define to 1 if you have the `oleaut32' library (-loleaut32). */
#define HAVE_LIBOLEAUT32 1
/* Define to 1 if you have the `rpcrt4' library (-lrpcrt4). */
#define HAVE_LIBRPCRT4 1
/* Define to 1 if you have the `rt' library (-lrt). */
/* #undef HAVE_LIBRT */
/* Define to 1 if you have the `shell32' library (-lshell32). */
#define HAVE_LIBSHELL32 1
/* Define to 1 if you have the `shlwapi' library (-lshlwapi). */
#define HAVE_LIBSHLWAPI 1
/* Define to 1 if you have the `ssp' library (-lssp). */
#define HAVE_LIBSSP 1
/* Define to 1 if you have the `user32' library (-luser32). */
#define HAVE_LIBUSER32 1
/* Define to 1 if you have the `uuid' library (-luuid). */
#define HAVE_LIBUUID 1
/* Define to 1 if you have the `winmm' library (-lwinmm). */
#define HAVE_LIBWINMM 1
/* Define to 1 if you have the `winspool' library (-lwinspool). */
#define HAVE_LIBWINSPOOL 1
/* Define to 1 if you have the `ws2_32' library (-lws2_32). */
#define HAVE_LIBWS2_32 1
/* Define to 1 if you have the `z ' library (-lz ). */
#define HAVE_LIBZ_ 1
/* Define this symbol if you have malloc_info */
/* #undef HAVE_MALLOC_INFO */
@@ -264,9 +181,6 @@
/* Define to 1 if you have the <miniupnpc/miniupnpc.h> header file. */
#define HAVE_MINIUPNPC_MINIUPNPC_H 1
/* Define to 1 if you have the <miniupnpc/miniwget.h> header file. */
#define HAVE_MINIUPNPC_MINIWGET_H 1
/* Define to 1 if you have the <miniupnpc/upnpcommands.h> header file. */
#define HAVE_MINIUPNPC_UPNPCOMMANDS_H 1
@@ -330,12 +244,6 @@
/* Define if the visibility attribute is supported. */
#define HAVE_VISIBILITY_ATTRIBUTE 1
/* Define this symbol if boost sleep works */
/* #undef HAVE_WORKING_BOOST_SLEEP */
/* Define this symbol if boost sleep_for works */
#define HAVE_WORKING_BOOST_SLEEP_FOR 1
/* Define to the sub-directory where libtool stores uninstalled libraries. */
#define LT_OBJDIR ".libs/"
@@ -346,7 +254,7 @@
#define PACKAGE_NAME "Bitcoin Core"
/* Define to the full name and version of this package. */
#define PACKAGE_STRING "Bitcoin Core 0.18.99"
#define PACKAGE_STRING "Bitcoin Core 22.0.0"
/* Define to the one symbol short name of this package. */
#define PACKAGE_TARNAME "bitcoin"
@@ -355,7 +263,7 @@
#define PACKAGE_URL "https://bitcoincore.org/"
/* Define to the version of this package. */
#define PACKAGE_VERSION "0.18.99"
#define PACKAGE_VERSION "22.0.0"
/* Define to necessary symbol if this constant uses a non-standard name on
your system. */
@@ -385,9 +293,6 @@
/* Define this symbol to build in assembly routines */
//#define USE_ASM 1
/* Define this symbol if coverage is enabled */
/* #undef USE_COVERAGE */
/* Define if dbus support should be compiled in */
/* #undef USE_DBUS */

View File

@@ -10,6 +10,9 @@
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\bitcoind.cpp" />
<ClCompile Include="..\..\src\init\bitcoind.cpp">
<ObjectFileName>$(IntDir)init_bitcoind.obj</ObjectFileName>
</ClCompile>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
@@ -62,6 +65,10 @@
Replace="@EXEEXT@" By=".exe"></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@ENABLE_WALLET_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@USE_BDB_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@USE_SQLITE_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"
Replace="@BUILD_BITCOIN_CLI_TRUE@" By=""></ReplaceInFile>
<ReplaceInFile FilePath="$(ConfigIniOut)"

View File

@@ -4,9 +4,19 @@
<PropertyGroup Label="Globals">
<VCProjectVersion>16.0</VCProjectVersion>
<UseNativeEnvironment>true</UseNativeEnvironment>
</PropertyGroup>
<PropertyGroup Label="Vcpkg">
<VcpkgEnabled>true</VcpkgEnabled>
<VcpkgEnableManifest>true</VcpkgEnableManifest>
<VcpkgManifestInstall>true</VcpkgManifestInstall>
<VcpkgUseStatic>true</VcpkgUseStatic>
<VcpkgAutoLink>true</VcpkgAutoLink>
<VcpkgConfiguration>$(Configuration)</VcpkgConfiguration>
<VcpkgTriplet Condition="'$(Platform)'=='Win32'">x86-windows-static</VcpkgTriplet>
<VcpkgTriplet Condition="'$(Platform)'=='x64'">x64-windows-static</VcpkgTriplet>
</PropertyGroup>
</PropertyGroup>
<PropertyGroup Condition="'$(WindowsTargetPlatformVersion)'=='' and !Exists('$(WindowsSdkDir)\DesignTime\CommonConfiguration\Neutral\Windows.props')">
<WindowsTargetPlatformVersion_10 Condition="'$(WindowsTargetPlatformVersion_10)' == ''">$(Registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v10.0@ProductVersion)</WindowsTargetPlatformVersion_10>
@@ -35,66 +45,46 @@
</ProjectConfiguration>
</ItemGroup>
<PropertyGroup Condition="'$(Configuration)'=='Debug'" Label="Configuration">
<LinkIncremental>true</LinkIncremental>
<WholeProgramOptimization>false</WholeProgramOptimization>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v141</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\$(ProjectName)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\$(ProjectName)\</IntDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)'=='Release'" Label="Configuration">
<LinkIncremental>false</LinkIncremental>
<WholeProgramOptimization>true</WholeProgramOptimization>
<UseDebugLibraries>false</UseDebugLibraries>
<PlatformToolset>v141</PlatformToolset>
<PlatformToolset>v142</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
<GenerateManifest>No</GenerateManifest>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\$(ProjectName)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\$(ProjectName)\</IntDir>
</PropertyGroup>
<PropertyGroup Condition="'$(Configuration)'=='Debug'" Label="Configuration">
<LinkIncremental>true</LinkIncremental>
<UseDebugLibraries>true</UseDebugLibraries>
<PlatformToolset>v142</PlatformToolset>
<CharacterSet>Unicode</CharacterSet>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\$(ProjectName)\</OutDir>
<IntDir>$(Platform)\$(Configuration)\$(ProjectName)\</IntDir>
</PropertyGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<ItemDefinitionGroup Condition="'$(Configuration)'=='Release'">
<ClCompile>
<Optimization>MaxSpeed</Optimization>
<Optimization>Disabled</Optimization>
<WholeProgramOptimization>false</WholeProgramOptimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<SDLCheck>true</SDLCheck>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
<DebugInformationFormat>None</DebugInformationFormat>
</ClCompile>
<Link>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
<EnableCOMDATFolding>false</EnableCOMDATFolding>
<OptimizeReferences>false</OptimizeReferences>
<AdditionalOptions>/LTCG:OFF</AdditionalOptions>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ClCompile>
<Optimization>Disabled</Optimization>
<PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<SDLCheck>true</SDLCheck>
<RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
<AdditionalOptions>/bigobj %(AdditionalOptions)</AdditionalOptions>
</ClCompile>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|Win32'">
<ClCompile>
<Optimization>MaxSpeed</Optimization>
<FunctionLevelLinking>true</FunctionLevelLinking>
<IntrinsicFunctions>true</IntrinsicFunctions>
<SDLCheck>true</SDLCheck>
<RuntimeLibrary>MultiThreaded</RuntimeLibrary>
</ClCompile>
<Link>
<EnableCOMDATFolding>true</EnableCOMDATFolding>
<OptimizeReferences>true</OptimizeReferences>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|Win32'">
<ItemDefinitionGroup Condition="'$(Configuration)'=='Debug'">
<ClCompile>
<Optimization>Disabled</Optimization>
<WholeProgramOptimization>false</WholeProgramOptimization>
<PreprocessorDefinitions>_DEBUG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<SDLCheck>true</SDLCheck>
<RuntimeLibrary>MultiThreadedDebug</RuntimeLibrary>
@@ -106,16 +96,16 @@
<ClCompile>
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<AdditionalOptions>/utf-8 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4018;4221;4244;4267;4334;4715;4805;</DisableSpecificWarnings>
<AdditionalOptions>/utf-8 /Zc:__cplusplus /std:c++17 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4018;4244;4267;4334;4715;4805;4834</DisableSpecificWarnings>
<TreatWarningAsError>true</TreatWarningAsError>
<PreprocessorDefinitions>ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING;_SILENCE_CXX17_OLD_ALLOCATOR_MEMBERS_DEPRECATION_WARNING;ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;_WIN32_IE=0x0501;WIN32_LEAN_AND_MEAN;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src;..\..\src\univalue\include;..\..\src\secp256k1\include;..\..\src\leveldb\include;..\..\src\leveldb\helpers\memenv;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<SubSystem>Console</SubSystem>
<GenerateDebugInformation>true</GenerateDebugInformation>
<AdditionalDependencies>crypt32.lib;Iphlpapi.lib;ws2_32.lib;Shlwapi.lib;kernel32.lib;user32.lib;gdi32.lib;winspool.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalDependencies>Iphlpapi.lib;ws2_32.lib;Shlwapi.lib;kernel32.lib;user32.lib;gdi32.lib;comdlg32.lib;advapi32.lib;shell32.lib;ole32.lib;oleaut32.lib;uuid.lib;odbc32.lib;odbccp32.lib;%(AdditionalDependencies)</AdditionalDependencies>
<RandomizedBaseAddress>true</RandomizedBaseAddress>
</Link>
<Lib>
<AdditionalOptions>/ignore:4221</AdditionalOptions>

View File

@@ -2,15 +2,15 @@
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PropertyGroup Label="QtGlobals">
<QtBaseDir>C:\Qt5.9.7_ssl_x64_static_vs2017</QtBaseDir>
<QtBaseDir>C:\Qt5.12.11_x64_static_vs2019_16101</QtBaseDir>
<QtPluginsLibraryDir>$(QtBaseDir)\plugins</QtPluginsLibraryDir>
<QtLibraryDir>$(QtBaseDir)\lib</QtLibraryDir>
<QtIncludeDir>$(QtBaseDir)\include</QtIncludeDir>
<QtIncludes>$(QtIncludeDir);$(QtIncludeDir)\QtNetwork;$(QtIncludeDir)\QtCore;$(QtIncludeDir)\QtWidgets;$(QtIncludeDir)\QtGui;</QtIncludes>
<GeneratedFilesOutDir>.\QtGeneratedFiles\qt</GeneratedFilesOutDir>
<QtToolsDir>$(QtBaseDir)\bin</QtToolsDir>
<QtReleaseLibraries>$(QtPluginsLibraryDir)\platforms\qminimal.lib;$(QtPluginsLibraryDir)\platforms\qwindows.lib;$(QtLibraryDir)\qtfreetype.lib;$(QtLibraryDir)\qtharfbuzz.lib;$(QtLibraryDir)\qtlibpng.lib;$(QtLibraryDir)\qtpcre2.lib;$(QtLibraryDir)\Qt5AccessibilitySupport.lib;$(QtLibraryDir)\Qt5Core.lib;$(QtLibraryDir)\Qt5Concurrent.lib;$(QtLibraryDir)\Qt5EventDispatcherSupport.lib;$(QtLibraryDir)\Qt5FontDatabaseSupport.lib;$(QtLibraryDir)\Qt5Gui.lib;$(QtLibraryDir)\Qt5Network.lib;$(QtLibraryDir)\Qt5PlatformCompositorSupport.lib;$(QtLibraryDir)\Qt5ThemeSupport.lib;$(QtLibraryDir)\Qt5Widgets.lib;$(QtLibraryDir)\Qt5WinExtras.lib;$(QtLibraryDir)\qtmain.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtReleaseLibraries>
<QtDebugLibraries>$(QtPluginsLibraryDir)\platforms\qwindowsd.lib;$(QtPluginsLibraryDir)\platforms\qminimald.lib;$(QtLibraryDir)\*d.lib;crypt32.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtDebugLibraries>
<QtReleaseLibraries>$(QtPluginsLibraryDir)\platforms\qminimal.lib;$(QtPluginsLibraryDir)\platforms\qwindows.lib;$(QtPluginsLibraryDir)\styles\qwindowsvistastyle.lib;$(QtLibraryDir)\Qt5WindowsUIAutomationSupport.lib;$(QtLibraryDir)\qtfreetype.lib;$(QtLibraryDir)\qtharfbuzz.lib;$(QtLibraryDir)\qtlibpng.lib;$(QtLibraryDir)\qtpcre2.lib;$(QtLibraryDir)\Qt5AccessibilitySupport.lib;$(QtLibraryDir)\Qt5Core.lib;$(QtLibraryDir)\Qt5Concurrent.lib;$(QtLibraryDir)\Qt5EventDispatcherSupport.lib;$(QtLibraryDir)\Qt5FontDatabaseSupport.lib;$(QtLibraryDir)\Qt5Gui.lib;$(QtLibraryDir)\Qt5Network.lib;$(QtLibraryDir)\Qt5PlatformCompositorSupport.lib;$(QtLibraryDir)\Qt5ThemeSupport.lib;$(QtLibraryDir)\Qt5Widgets.lib;$(QtLibraryDir)\Qt5WinExtras.lib;$(QtLibraryDir)\qtmain.lib;Wtsapi32.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtReleaseLibraries>
<QtDebugLibraries>$(QtPluginsLibraryDir)\platforms\qwindowsd.lib;$(QtPluginsLibraryDir)\platforms\qminimald.lib;$(QtPluginsLibraryDir)\styles\qwindowsvistastyled.lib;$(QtLibraryDir)\*d.lib;Wtsapi32.lib;crypt32.lib;userenv.lib;netapi32.lib;imm32.lib;Dwmapi.lib;version.lib;winmm.lib;UxTheme.lib</QtDebugLibraries>
</PropertyGroup>
</Project>

View File

@@ -4,7 +4,7 @@
<Target Name="CopyBuildArtifacts" Condition="'$(ConfigurationType)' != 'StaticLibrary'">
<ItemGroup>
<BuildArtifacts Include="$(OutDir)$(TargetName)$(TargetExt)"></BuildArtifacts>
<BuildArtifacts Include="$(OutDir)$(TargetName).pdb"></BuildArtifacts>
<BuildArtifacts Include="$(OutDir)$(TargetName).pdb" Condition="Exists('$(OutDir)$(TargetName).pdb')"></BuildArtifacts>
</ItemGroup>
<Copy SourceFiles="@(BuildArtifacts)" SkipUnchangedFiles="true" DestinationFolder="..\..\src\" Condition="'$(OutDir)' != ''"></Copy>
</Target>

View File

@@ -34,7 +34,9 @@
<ClCompile Include="..\..\src\qt\overviewpage.cpp" />
<ClCompile Include="..\..\src\qt\paymentserver.cpp" />
<ClCompile Include="..\..\src\qt\peertablemodel.cpp" />
<ClCompile Include="..\..\src\qt\peertablesortproxy.cpp" />
<ClCompile Include="..\..\src\qt\platformstyle.cpp" />
<ClCompile Include="..\..\src\qt\psbtoperationsdialog.cpp" />
<ClCompile Include="..\..\src\qt\qrimagewidget.cpp" />
<ClCompile Include="..\..\src\qt\qvalidatedlineedit.cpp" />
<ClCompile Include="..\..\src\qt\qvaluecombobox.cpp" />
@@ -86,7 +88,9 @@
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_overviewpage.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_paymentserver.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_peertablemodel.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_peertablesortproxy.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_platformstyle.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_psbtoperationsdialog.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_qrimagewidget.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_qvalidatedlineedit.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_qvaluecombobox.cpp" />
@@ -102,6 +106,7 @@
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactiondesc.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactiondescdialog.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactionfilterproxy.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactionoverviewwidget.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactionrecord.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactiontablemodel.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_transactionview.cpp" />

View File

@@ -8,6 +8,7 @@
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\util\url.cpp" />
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -8,6 +8,9 @@
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\wallet\bdb.cpp" />
<ClCompile Include="..\..\src\wallet\salvage.cpp" />
<ClCompile Include="..\..\src\wallet\sqlite.cpp" />
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -24,8 +24,6 @@
<ClCompile Include="..\..\src\leveldb\db\version_set.cc" />
<ClCompile Include="..\..\src\leveldb\db\write_batch.cc" />
<ClCompile Include="..\..\src\leveldb\helpers\memenv\memenv.cc" />
<ClCompile Include="..\..\src\leveldb\port\port_posix_sse.cc" />
<ClCompile Include="..\..\src\leveldb\port\port_win.cc" />
<ClCompile Include="..\..\src\leveldb\table\block.cc" />
<ClCompile Include="..\..\src\leveldb\table\block_builder.cc" />
<ClCompile Include="..\..\src\leveldb\table\filter_block.cc" />
@@ -42,7 +40,7 @@
<ClCompile Include="..\..\src\leveldb\util\comparator.cc" />
<ClCompile Include="..\..\src\leveldb\util\crc32c.cc" />
<ClCompile Include="..\..\src\leveldb\util\env.cc" />
<ClCompile Include="..\..\src\leveldb\util\env_win.cc" />
<ClCompile Include="..\..\src\leveldb\util\env_windows.cc" />
<ClCompile Include="..\..\src\leveldb\util\filter_policy.cc" />
<ClCompile Include="..\..\src\leveldb\util\hash.cc" />
<ClCompile Include="..\..\src\leveldb\util\histogram.cc" />
@@ -51,11 +49,11 @@
<ClCompile Include="..\..\src\leveldb\util\status.cc" />
</ItemGroup>
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>_CRT_NONSTDC_NO_DEPRECATE;LEVELDB_PLATFORM_WINDOWS;LEVELDB_ATOMIC_PRESENT;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<DisableSpecificWarnings>4244;4267;4312;</DisableSpecificWarnings>
<AdditionalIncludeDirectories>..\..\src\leveldb;..\..\src\leveldb\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<ClCompile>
<PreprocessorDefinitions>HAVE_CRC32C=0;HAVE_SNAPPY=0;__STDC_LIMIT_MACROS;LEVELDB_IS_BIG_ENDIAN=0;_UNICODE;UNICODE;_CRT_NONSTDC_NO_DEPRECATE;LEVELDB_PLATFORM_WINDOWS;LEVELDB_ATOMIC_PRESENT;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<DisableSpecificWarnings>4244;4267</DisableSpecificWarnings>
<AdditionalIncludeDirectories>..\..\src\leveldb;..\..\src\leveldb\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />

View File

@@ -10,13 +10,14 @@
<ItemGroup>
<ClCompile Include="..\..\src\secp256k1\src\secp256k1.c" />
</ItemGroup>
<ItemDefinitionGroup>
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>ENABLE_MODULE_ECDH;ENABLE_MODULE_RECOVERY;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src\secp256k1;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<PreprocessorDefinitions>ENABLE_MODULE_ECDH;ENABLE_MODULE_RECOVERY;ENABLE_MODULE_EXTRAKEYS;ENABLE_MODULE_SCHNORRSIG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src\secp256k1;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<DisableSpecificWarnings>4146;4244;4267;4334</DisableSpecificWarnings>
</ClCompile>
</ItemDefinitionGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<Import Project="..\common.vcxproj" />
</Project>
</Project>

View File

@@ -26,4 +26,7 @@
#define USE_FIELD_10X26 1
#define USE_SCALAR_8X32 1
#define ECMULT_GEN_PREC_BITS 4
#define ECMULT_WINDOW_SIZE 15
#endif /* BITCOIN_LIBSECP256K1_CONFIG_H */

View File

@@ -0,0 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\common.init.vcxproj" />
<PropertyGroup Label="Globals">
<ProjectGuid>{868474FD-35F6-4400-8EED-30A33E7521D4}</ProjectGuid>
</PropertyGroup>
<PropertyGroup Label="Configuration">
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<Import Project="..\common.vcxproj" />
</Project>

4
build_msvc/msvc-autogen.py Normal file → Executable file
View File

@@ -1,4 +1,7 @@
#!/usr/bin/env python3
# Copyright (c) 2016-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
import os
import re
@@ -18,6 +21,7 @@ libs = [
'libbitcoin_wallet',
'libbitcoin_zmq',
'bench_bitcoin',
'libtest_util',
]
ignore_list = [

View File

@@ -1,19 +1,16 @@
<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Build" ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Project DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<Import Project="..\common.init.vcxproj" />
<Import Project="..\common.qt.init.vcxproj" />
<PropertyGroup Label="Globals">
<ProjectGuid>{51201D5E-D939-4854-AE9D-008F03FF518E}</ProjectGuid>
</PropertyGroup>
<PropertyGroup Label="Configuration">
<ConfigurationType>Application</ConfigurationType>
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\test\setup_common.cpp" />
<ClCompile Include="..\..\src\test\util\setup_common.cpp" />
<ClCompile Include="..\..\src\qt\test\addressbooktests.cpp" />
<ClCompile Include="..\..\src\qt\test\apptests.cpp" />
<ClCompile Include="..\..\src\qt\test\compattests.cpp" />
<ClCompile Include="..\..\src\qt\test\rpcnestedtests.cpp" />
<ClCompile Include="..\..\src\qt\test\test_main.cpp" />
<ClCompile Include="..\..\src\qt\test\uritests.cpp" />
@@ -22,7 +19,6 @@
<ClCompile Include="..\..\src\wallet\test\wallet_test_fixture.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_addressbooktests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_apptests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_compattests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_rpcnestedtests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_uritests.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_wallettests.cpp" />
@@ -66,29 +62,31 @@
</ProjectReference>
</ItemGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Release|x64'">
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<ItemDefinitionGroup Condition="'$(Configuration)'=='Release'">
<ClCompile>
<AdditionalIncludeDirectories>..\libbitcoin_qt\$(GeneratedFilesOutDir)\..\;$(QtIncludeDir)\QtTest;$(QtIncludes);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<AdditionalDependencies>$(QtReleaseLibaries);%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalDependencies>$(QtLibraryDir)\Qt5Test.lib;$(QtReleaseLibraries);%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalOptions>/ignore:4206 /LTCG:OFF</AdditionalOptions>
</Link>
</ItemDefinitionGroup>
<ItemDefinitionGroup Condition="'$(Configuration)|$(Platform)'=='Debug|x64'">
<ItemDefinitionGroup Condition="'$(Configuration)'=='Debug'">
<ClCompile>
<AdditionalIncludeDirectories>..\libbitcoin_qt\$(GeneratedFilesOutDir)\..\;$(QtIncludeDir)\QtTest;$(QtIncludes);%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>
<AdditionalDependencies>$(QtDebugLibraries);%(AdditionalDependencies)</AdditionalDependencies>
<AdditionalOptions>/ignore:4206</AdditionalOptions>
</Link>
</ItemDefinitionGroup>
<ItemGroup>
<MocTestFiles Include="..\..\src\qt\test\addressbooktests.h" />
<MocTestFiles Include="..\..\src\qt\test\apptests.h" />
<MocTestFiles Include="..\..\src\qt\test\compattests.h" />
<MocTestFiles Include="..\..\src\qt\test\paymentservertests.h" />
<MocTestFiles Include="..\..\src\qt\test\rpcnestedtests.h" />
<MocTestFiles Include="..\..\src\qt\test\uritests.h" />
<MocTestFiles Include="..\..\src\qt\test\wallettests.h" />
@@ -106,8 +104,6 @@
<RemoveDir Directories="$(GeneratedFilesOutDir)\moc\*" />
<RemoveDir Directories="$(GeneratedFilesOutDir)\moc" />
</Target>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />
<PropertyGroup>
<BuildDependsOn>
moccode;
@@ -120,4 +116,4 @@
$(CleanDependsOn);
</CleanDependsOn>
</PropertyGroup>
</Project>
</Project>

View File

@@ -9,13 +9,13 @@
<OutDir>$(SolutionDir)$(Platform)\$(Configuration)\</OutDir>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\test\*_tests.cpp" />
<ClCompile Include="..\..\src\test\*_properties.cpp" />
<ClCompile Include="..\..\src\test\*_tests.cpp" />
<ClCompile Include="..\..\src\test\gen\*_gen.cpp" />
<ClCompile Include="..\..\src\wallet\test\*_tests.cpp" />
<ClCompile Include="..\..\src\test\setup_common.cpp" />
<ClCompile Include="..\..\src\test\main.cpp" />
<ClCompile Include="..\..\src\test\util\*.cpp" />
<ClCompile Include="..\..\src\wallet\test\*_fixture.cpp" />
<ClCompile Include="..\..\src\wallet\test\*_tests.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
@@ -42,6 +42,9 @@
<ProjectReference Include="..\libbitcoin_zmq\libbitcoin_zmq.vcxproj">
<Project>{792d487f-f14c-49fc-a9de-3fc150f31c3f}</Project>
</ProjectReference>
<ProjectReference Include="..\libtest_util\libtest_util.vcxproj">
<Project>{1e065f03-3566-47d0-8fa9-daa72b084e7d}</Project>
</ProjectReference>
<ProjectReference Include="..\libunivalue\libunivalue.vcxproj">
<Project>{5724ba7d-a09a-4ba8-800b-c4c1561b3d69}</Project>
</ProjectReference>
@@ -58,7 +61,9 @@
</PropertyGroup>
<ItemGroup>
<JsonTestFile Include="..\..\src\test\data\*.json" />
<RawTestFile Include="..\..\src\test\data\*.raw" />
</ItemGroup>
<HeaderFromHexdump RawFilePath="%(RawTestFile.FullPath)" HeaderFilePath="%(RawTestFile.FullPath).h" SourceHeader="static unsigned const char %(RawTestFile.Filename)_raw[] = {" SourceFooter="};" />
<HeaderFromHexdump RawFilePath="%(JsonTestFile.FullPath)" HeaderFilePath="%(JsonTestFile.FullPath).h" SourceHeader="namespace json_tests{ static unsigned const char %(JsonTestFile.Filename)[] = {" SourceFooter="};}" />
</Target>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -1,3 +1,7 @@
// Copyright (c) 2018-2020 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <iostream>
// bitcoin includes.
@@ -41,7 +45,7 @@ int main()
stream << vanillaSpendTx;
bitcoinconsensus_error err;
auto op0Result = bitcoinconsensus_verify_script_with_amount(pubKeyScript.data(), pubKeyScript.size(), amount, (const unsigned char*)&stream[0], stream.size(), 0, bitcoinconsensus_SCRIPT_FLAGS_VERIFY_ALL, &err);
auto op0Result = bitcoinconsensus_verify_script_with_amount(pubKeyScript.data(), pubKeyScript.size(), amount, stream.data(), stream.size(), 0, bitcoinconsensus_SCRIPT_FLAGS_VERIFY_ALL, &err);
std::cout << "Op0 result: " << op0Result << ", error code " << err << std::endl;
getchar();

19
build_msvc/vcpkg.json Normal file
View File

@@ -0,0 +1,19 @@
{
"name": "bitcoin-core",
"version-string": "1",
"dependencies": [
"berkeleydb",
"boost-filesystem",
"boost-multi-index",
"boost-process",
"boost-signals2",
"boost-test",
"sqlite3",
"double-conversion",
{
"name": "libevent",
"features": ["thread"]
},
"zeromq"
]
}

View File

@@ -1,18 +1,24 @@
## ci scripts
## CI Scripts
This directory contains scripts for each build step in each build stage.
Currently three stages `lint`, `extended_lint` and `test` are defined. Each stage has its own lifecycle, similar to the
[Travis CI lifecycle](https://docs.travis-ci.com/user/job-lifecycle#the-job-lifecycle). Every script in here is named
and numbered according to which stage and lifecycle step it belongs to.
### Running a Stage Locally
### Running a stage locally
Be aware that the tests will be built and run in-place, so please run at your own risk.
If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory.
While most of the actions are done inside a docker container, this is not possible for all. Thus, cache directories,
such as the depends cache, previous release binaries, or ccache, are mounted as read-write into the docker container. While it should be fine to run
the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and
testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci
system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires `docker` to be installed. To install all requirements on Ubuntu, run
```
sudo apt install docker.io ccache bash git
sudo apt install docker.io bash
```
To run the default test stage,
@@ -27,5 +33,33 @@ To run the test stage with a specific configuration,
FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh
```
Be aware that the tests will be build and run in-place, so please run at your own risk.
If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
### Configurations
The test files (`FILE_ENV`) are constructed to test a wide range of
configurations, rather than a single pass/fail. This helps to catch build
failures and logic errors that present on platforms other than the ones the
author has tested.
Some builders use the dependency-generator in `./depends`, rather than using
the system package manager to install build dependencies. This guarantees that
the tester is using the same versions as the release builds, which also use
`./depends`.
If no `FILE_ENV` has been specified or values are left out, `00_setup_env.sh`
is used as the default configuration with fallback values.
It is also possible to force a specific configuration without modifying the
file. For example,
```
MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh
```
The files starting with `0n` (`n` greater than 0) are the scripts that are run
in order.
### Cache
In order to avoid rebuilding all dependencies for each build, the binaries are
cached and re-used when possible. Changes in the dependency-generator will
trigger cache-invalidation and rebuilds as necessary.

View File

@@ -1,12 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
CPPCHECK_VERSION=1.86
curl -s https://codeload.github.com/danmar/cppcheck/tar.gz/${CPPCHECK_VERSION} | tar -zxf - --directory /tmp/
(cd /tmp/cppcheck-${CPPCHECK_VERSION}/ && make CFGDIR=/tmp/cppcheck-${CPPCHECK_VERSION}/cfg/ > /dev/null)
export PATH="$PATH:/tmp/cppcheck-${CPPCHECK_VERSION}/"

View File

@@ -1,9 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
test/lint/extended-lint-all.sh

View File

@@ -1,14 +1,22 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
travis_retry pip3 install codespell==1.15.0
travis_retry pip3 install flake8==3.7.8
${CI_RETRY_EXE} apt-get update
${CI_RETRY_EXE} apt-get install -y clang-format-9 python3-pip curl git gawk jq
update-alternatives --install /usr/bin/clang-format clang-format $(which clang-format-9 ) 100
update-alternatives --install /usr/bin/clang-format-diff clang-format-diff $(which clang-format-diff-9) 100
SHELLCHECK_VERSION=v0.6.0
curl -s "https://storage.googleapis.com/shellcheck/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | tar --xz -xf - --directory /tmp/
${CI_RETRY_EXE} pip3 install codespell==2.0.0
${CI_RETRY_EXE} pip3 install flake8==3.8.3
${CI_RETRY_EXE} pip3 install yq
${CI_RETRY_EXE} pip3 install mypy==0.781
${CI_RETRY_EXE} pip3 install vulture==2.3
SHELLCHECK_VERSION=v0.7.2
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | tar --xz -xf - --directory /tmp/
export PATH="/tmp/shellcheck-${SHELLCHECK_VERSION}:${PATH}"

View File

@@ -1,9 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
git fetch --unshallow

View File

@@ -1,25 +1,33 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
if [ "$TRAVIS_EVENT_TYPE" = "pull_request" ]; then
test/lint/commit-script-check.sh $TRAVIS_COMMIT_RANGE
GIT_HEAD=$(git rev-parse HEAD)
if [ -n "$CIRRUS_PR" ]; then
COMMIT_RANGE="$CIRRUS_BASE_SHA..$GIT_HEAD"
test/lint/commit-script-check.sh $COMMIT_RANGE
fi
export COMMIT_RANGE
# This only checks that the trees are pure subtrees, it is not doing a full
# check with -r to not have to fetch all the remotes.
test/lint/git-subtree-check.sh src/crypto/ctaes
test/lint/git-subtree-check.sh src/secp256k1
test/lint/git-subtree-check.sh src/univalue
test/lint/git-subtree-check.sh src/leveldb
test/lint/git-subtree-check.sh src/crc32c
test/lint/check-doc.py
test/lint/check-rpc-mappings.py .
test/lint/lint-all.sh
if [ "$TRAVIS_REPO_SLUG" = "bitcoin/bitcoin" ] && [ "$TRAVIS_EVENT_TYPE" = "cron" ]; then
if [ "$CIRRUS_REPO_FULL_NAME" = "bitcoin/bitcoin" ] && [ -n "$CIRRUS_CRON" ]; then
git log --merges --before="2 days ago" -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
travis_retry gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
${CI_RETRY_EXE} gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys $(<contrib/verify-commits/trusted-keys) &&
./contrib/verify-commits/verify-commits.py --clean-merge=2;
fi
echo
git log --no-merges --oneline $COMMIT_RANGE

11
ci/lint_run_all.sh Executable file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/lint/04_install.sh
set -o errexit; source ./ci/lint/06_script.sh

View File

@@ -32,8 +32,8 @@ Help:
-v, --verbose Verbose output
-t, --tries=# Set max retries: Default 10
-s, --sleep=secs Constant sleep amount (seconds)
-m, --min=secs Exponenetial Backoff: minimum sleep amount (seconds): Default 0.3
-x, --max=secs Exponenetial Backoff: maximum sleep amount (seconds): Default 60
-m, --min=secs Exponential Backoff: minimum sleep amount (seconds): Default 0.3
-x, --max=secs Exponential Backoff: maximum sleep amount (seconds): Default 60
-f, --fail="script +cmds" Fail Script: run in case of final failure
### Examples

View File

@@ -17,7 +17,7 @@ __log_out() {
echo "$1" 1>&2
}
# Paramters: max_tries min_sleep max_sleep constant_sleep fail_script EXECUTION_COMMAND
# Parameters: max_tries min_sleep max_sleep constant_sleep fail_script EXECUTION_COMMAND
retry()
{
local max_tries="$1"; shift
@@ -83,8 +83,8 @@ Usage: $retry [options] -- execute command
-v, --verbose Verbose output
-t, --tries=# Set max retries: Default 10
-s, --sleep=secs Constant sleep amount (seconds)
-m, --min=secs Exponenetial Backoff: minimum sleep amount (seconds): Default 0.3
-x, --max=secs Exponenetial Backoff: maximum sleep amount (seconds): Default 60
-m, --min=secs Exponential Backoff: minimum sleep amount (seconds): Default 0.3
-x, --max=secs Exponential Backoff: maximum sleep amount (seconds): Default 60
-f, --fail="script +cmds" Fail Script: run in case of final failure
EOF
}

View File

@@ -1,11 +1,23 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
# The root dir.
# The ci system copies this folder.
# This is where the depends build is done.
BASE_ROOT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )"/../../ >/dev/null 2>&1 && pwd )
export BASE_ROOT_DIR
# The depends dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
export DEPENDS_DIR=${DEPENDS_DIR:-$BASE_ROOT_DIR/depends}
# A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
# This folder only exists on the ci host.
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch}
echo "Setting specific values in env"
if [ -n "${FILE_ENV}" ]; then
set -o errexit;
@@ -13,34 +25,48 @@ if [ -n "${FILE_ENV}" ]; then
source "${FILE_ENV}"
fi
BASE_ROOT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )"/../../ >/dev/null 2>&1 && pwd )
export BASE_ROOT_DIR
echo "Fallback to default values in env (if not yet set)"
# The number of parallel jobs to pass down to make and test_runner.py
export MAKEJOBS=${MAKEJOBS:--j4}
# A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch/}
export HOST=${HOST:-x86_64-unknown-linux-gnu}
# What host to compile for. See also ./depends/README.md
# Tests that need cross-compilation export the appropriate HOST.
# Tests that run natively guess the host
export HOST=${HOST:-$("$BASE_ROOT_DIR/depends/config.guess")}
# Whether to prefer BusyBox over GNU utilities
export USE_BUSY_BOX=${USE_BUSY_BOX:-false}
export RUN_UNIT_TESTS=${RUN_UNIT_TESTS:-true}
export RUN_FUNCTIONAL_TESTS=${RUN_FUNCTIONAL_TESTS:-true}
export RUN_SECURITY_TESTS=${RUN_SECURITY_TESTS:-false}
# By how much to scale the test_runner timeouts (option --timeout-factor).
# This is needed because some ci machines have slow CPU or disk, so sanitizers
# might be slow or a reindex might be waiting on disk IO.
export TEST_RUNNER_TIMEOUT_FACTOR=${TEST_RUNNER_TIMEOUT_FACTOR:-40}
export TEST_RUNNER_ENV=${TEST_RUNNER_ENV:-}
export RUN_FUZZ_TESTS=${RUN_FUZZ_TESTS:-false}
export DOCKER_NAME_TAG=${DOCKER_NAME_TAG:-ubuntu:18.04}
export EXPECTED_TESTS_DURATION_IN_SECONDS=${EXPECTED_TESTS_DURATION_IN_SECONDS:-1000}
export CONTAINER_NAME=${CONTAINER_NAME:-ci_unnamed}
export DOCKER_NAME_TAG=${DOCKER_NAME_TAG:-ubuntu:20.04}
# Randomize test order.
# See https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/random.html
export BOOST_TEST_RANDOM=${BOOST_TEST_RANDOM:-1}
# See man 7 debconf
export DEBIAN_FRONTEND=noninteractive
export CCACHE_SIZE=${CCACHE_SIZE:-100M}
export CCACHE_TEMPDIR=${CCACHE_TEMPDIR:-/tmp/.ccache-temp}
export CCACHE_COMPRESS=${CCACHE_COMPRESS:-1}
# The cache dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
export CCACHE_DIR=${CCACHE_DIR:-$BASE_SCRATCH_DIR/.ccache}
# Folder where the build is done (depends and dist). Can not be changed and is equal to the root of the git repo
export BASE_BUILD_DIR=${BASE_BUILD_DIR:-$BASE_ROOT_DIR}
# Folder where the build is done (bin and lib). Can not be changed.
export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_BUILD_DIR/out/$HOST}
# Folder where the build result is put (bin and lib).
export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_SCRATCH_DIR/out/$HOST}
# Folder where the build is done (dist and out-of-tree build).
export BASE_BUILD_DIR=${BASE_BUILD_DIR:-$BASE_SCRATCH_DIR/build}
export PREVIOUS_RELEASES_DIR=${PREVIOUS_RELEASES_DIR:-$BASE_ROOT_DIR/releases/$HOST}
export SDK_URL=${SDK_URL:-https://bitcoincore.org/depends-sources/sdks}
export WINEDEBUG=${WINEDEBUG:-fixme-all}
export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3}
export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps bison}
export GOAL=${GOAL:-install}
export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_BUILD_DIR}/qa-assets}
export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_SCRATCH_DIR}/qa-assets}
export PATH=${BASE_ROOT_DIR}/ci/retry:$PATH
export CI_RETRY_EXE=${CI_RETRY_EXE:retry}
export CI_RETRY_EXE=${CI_RETRY_EXE:-"retry --"}

View File

@@ -1,13 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-unknown-linux-gnu
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libssl1.0-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev"
export NO_DEPENDS=1
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS=-DDEBUG_LOCKORDER --with-sanitizers=address,integer,undefined CC=clang CXX=clang++"

View File

@@ -1,13 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-unknown-linux-gnu
export PACKAGES="python3-zmq"
export DEP_OPTS="NO_WALLET=1"
export GOAL="install"
export BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports"

View File

@@ -1,14 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-unknown-linux-gnu
export PACKAGES="python3-zmq qtbase5-dev qttools5-dev-tools libdbus-1-dev libharfbuzz-dev"
export DEP_OPTS="NO_QT=1 NO_UPNP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1"
export TEST_RUNNER_EXTRA="--coverage --extended --exclude feature_dbcrash" # Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports --enable-debug CFLAGS=\"-g0 -O2 -funsigned-char\" CXXFLAGS=\"-g0 -O2 -funsigned-char\""

View File

@@ -1,15 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-unknown-linux-gnu
export DOCKER_NAME_TAG=ubuntu:14.04
export PACKAGES="python3-zmq qtbase5-dev qttools5-dev-tools libicu-dev libpng-dev libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.1++-dev libzmq3-dev libqrencode-dev"
export NO_DEPENDS=1
export RUN_FUNCTIONAL_TESTS=false
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no"

View File

@@ -1,14 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-unknown-linux-gnu
export DOCKER_NAME_TAG=ubuntu:16.04
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libssl-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev libdb5.3++-dev libminiupnpc-dev libzmq3-dev libqrencode-dev"
export NO_DEPENDS=1
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --disable-wallet --with-gui=qt5 CPPFLAGS=-DDEBUG_LOCKORDER --with-sanitizers=thread --disable-hardening --disable-asm CC=clang CXX=clang++"

25
ci/test/00_setup_env_android.sh Executable file
View File

@@ -0,0 +1,25 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=aarch64-linux-android
export PACKAGES="clang llvm unzip openjdk-8-jdk gradle"
export CONTAINER_NAME=ci_android
export DOCKER_NAME_TAG="ubuntu:focal"
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export ANDROID_API_LEVEL=28
export ANDROID_BUILD_TOOLS_VERSION=28.0.3
export ANDROID_NDK_VERSION=21.1.6352462
export ANDROID_TOOLS_URL=https://dl.google.com/android/repository/commandlinetools-linux-6609375_latest.zip
export ANDROID_HOME="${DEPENDS_DIR}/SDKs/android"
export ANDROID_NDK_HOME="${ANDROID_HOME}/ndk/${ANDROID_NDK_VERSION}"
export DEP_OPTS="ANDROID_SDK=${ANDROID_HOME} ANDROID_NDK=${ANDROID_NDK_HOME} ANDROID_API_LEVEL=${ANDROID_API_LEVEL} ANDROID_TOOLCHAIN_BIN=${ANDROID_NDK_HOME}/toolchains/llvm/prebuilt/linux-x86_64/bin/"
export BITCOIN_CONFIG="--disable-ccache"

18
ci/test/00_setup_env_arm.sh Normal file → Executable file
View File

@@ -1,14 +1,26 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=arm-linux-gnueabihf
export PACKAGES="python3 g++-arm-linux-gnueabihf"
export RUN_UNIT_TESTS=false
# The host arch is unknown, so we run the tests through qemu.
# If the host is arm and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
if [ -z ${QEMU_USER_CMD+x} ]; then export QEMU_USER_CMD="${QEMU_USER_CMD:-"qemu-arm -L /usr/arm-linux-gnueabihf/"}"; fi
export DPKG_ADD_ARCH="armhf"
export PACKAGES="python3-zmq g++-arm-linux-gnueabihf busybox libc6:armhf libstdc++6:armhf libfontconfig1:armhf libxcb1:armhf"
if [ -n "$QEMU_USER_CMD" ]; then
# Likely cross-compiling, so install the needed gcc and qemu-user
export PACKAGES="$PACKAGES qemu-user"
fi
export CONTAINER_NAME=ci_arm_linux
# Use debian to avoid 404 apt errors when cross compiling
export DOCKER_NAME_TAG="debian:buster"
export USE_BUSY_BOX=true
export RUN_UNIT_TESTS=true
export RUN_FUNCTIONAL_TESTS=false
export GOAL="install"
# -Wno-psabi is to disable ABI warnings: "note: parameter passing for argument of type ... changed in GCC 7.1"

View File

@@ -1,14 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=i686-pc-linux-gnu
export DEP_OPTS="PROTOBUF=1"
export PACKAGES="g++-multilib python3-zmq"
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-bip70 --enable-glibc-back-compat --enable-reduce-exports LDFLAGS=-static-libstdc++"
export CONFIG_SHELL="/bin/dash"

View File

@@ -0,0 +1,16 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=i686-pc-linux-gnu
export CONTAINER_NAME=ci_i686_centos_8
export DOCKER_NAME_TAG=centos:8
export DOCKER_PACKAGES="gcc-c++ glibc-devel.x86_64 libstdc++-devel.x86_64 glibc-devel.i686 libstdc++-devel.i686 ccache libtool make git python3 python3-zmq which patch lbzip2 dash rsync coreutils bison"
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-reduce-exports"
export CONFIG_SHELL="/bin/dash"
export TEST_RUNNER_ENV="LC_ALL=en_US.UTF-8"

13
ci/test/00_setup_env_mac.sh Normal file → Executable file
View File

@@ -1,15 +1,18 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-apple-darwin14
export PACKAGES="cmake imagemagick libcap-dev librsvg2-bin libz-dev libbz2-dev libtiff-tools python3-dev python3-setuptools"
export OSX_SDK=10.11
export CONTAINER_NAME=ci_macos_cross
export DOCKER_NAME_TAG=ubuntu:20.04 # Check that Focal can cross-compile to macos (Focal is used in the gitian build as well)
export HOST=x86_64-apple-darwin18
export PACKAGES="cmake imagemagick librsvg2-bin libz-dev libtiff-tools libtinfo5 python3-setuptools xorriso"
export XCODE_VERSION=12.1
export XCODE_BUILD_ID=12A7403
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export GOAL="deploy"
export BITCOIN_CONFIG="--enable-gui --enable-reduce-exports --enable-werror"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports"

View File

@@ -0,0 +1,17 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-apple-darwin18
export PIP_PACKAGES="zmq lief"
export GOAL="install"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports"
export CI_OS_NAME="macos"
export NO_DEPENDS=1
export OSX_SDK=""
export CCACHE_SIZE=300M
export RUN_SECURITY_TESTS="true"

View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_asan
export PACKAGES="clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev"
export DOCKER_NAME_TAG=ubuntu:hirsute
export NO_DEPENDS=1
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=address,integer,undefined CC=clang CXX=clang++"

View File

@@ -1,16 +1,18 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-unknown-linux-gnu
export PACKAGES="clang llvm python3 libssl1.0-dev libevent-dev bsdmainutils libboost-system-dev libboost-filesystem-dev libboost-chrono-dev libboost-test-dev libboost-thread-dev"
export DOCKER_NAME_TAG="ubuntu:20.04"
export CONTAINER_NAME=ci_native_fuzz
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export RUN_FUZZ_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,address,undefined CC=clang CXX=clang++"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,address,undefined,integer CC=clang CXX=clang++"
export CCACHE_SIZE=200M

View File

@@ -0,0 +1,24 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
LIBCXX_DIR="${BASE_SCRATCH_DIR}/msan/build/"
export MSAN_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls"
LIBCXX_FLAGS="-nostdinc++ -stdlib=libc++ -L${LIBCXX_DIR}lib -lc++abi -I${LIBCXX_DIR}include -I${LIBCXX_DIR}include/c++/v1 -lpthread -Wl,-rpath,${LIBCXX_DIR}lib -Wno-unused-command-line-argument"
export MSAN_AND_LIBCXX_FLAGS="${MSAN_FLAGS} ${LIBCXX_FLAGS}"
export CONTAINER_NAME="ci_native_msan"
export PACKAGES="clang-9 llvm-9 cmake"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' boost_cxxflags='-std=c++17 -fvisibility=hidden -fPIC ${MSAN_AND_LIBCXX_FLAGS}' libevent_cflags='${MSAN_FLAGS}' zeromq_cxxflags='-std=c++17 ${MSAN_AND_LIBCXX_FLAGS}'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,memory --with-asm=no --prefix=${DEPENDS_DIR}/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export USE_MEMORY_SANITIZER="true"
export RUN_UNIT_TESTS="false"
export RUN_FUNCTIONAL_TESTS="false"
export RUN_FUZZ_TESTS=true
export CCACHE_SIZE=250M

View File

@@ -0,0 +1,19 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
export CONTAINER_NAME=ci_native_fuzz_valgrind
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev valgrind"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export RUN_FUZZ_TESTS=true
export FUZZ_TESTS_CONFIG="--valgrind"
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer CC=clang CXX=clang++"
export CCACHE_SIZE=200M

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
LIBCXX_DIR="${BASE_SCRATCH_DIR}/msan/build/"
export MSAN_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls"
LIBCXX_FLAGS="-nostdinc++ -stdlib=libc++ -L${LIBCXX_DIR}lib -lc++abi -I${LIBCXX_DIR}include -I${LIBCXX_DIR}include/c++/v1 -lpthread -Wl,-rpath,${LIBCXX_DIR}lib -Wno-unused-command-line-argument"
export MSAN_AND_LIBCXX_FLAGS="${MSAN_FLAGS} ${LIBCXX_FLAGS}"
export BDB_PREFIX="${BASE_ROOT_DIR}/db4"
export CONTAINER_NAME="ci_native_msan"
export PACKAGES="clang-9 llvm-9 cmake"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' boost_cxxflags='-std=c++17 -fvisibility=hidden -fPIC ${MSAN_AND_LIBCXX_FLAGS}' libevent_cflags='${MSAN_FLAGS}' sqlite_cflags='${MSAN_FLAGS}' zeromq_cxxflags='-std=c++17 ${MSAN_AND_LIBCXX_FLAGS}'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-wallet --with-sanitizers=memory --with-asm=no --prefix=${DEPENDS_DIR}/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' BDB_LIBS='-L${BDB_PREFIX}/lib -ldb_cxx-4.8' BDB_CFLAGS='-I${BDB_PREFIX}/include'"
export USE_MEMORY_SANITIZER="true"
export RUN_FUNCTIONAL_TESTS="false"
export CCACHE_SIZE=250M

View File

@@ -0,0 +1,16 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_multiprocess
export DOCKER_NAME_TAG=ubuntu:20.04
export PACKAGES="cmake python3 python3-pip llvm clang"
export DEP_OPTS="DEBUG=1 MULTIPROCESS=1"
export GOAL="install"
export BITCOIN_CONFIG="--enable-debug CC=clang CXX=clang++" # Use clang to avoid OOM
export TEST_RUNNER_ENV="BITCOIND=bitcoin-node"
export PIP_PACKAGES="lief"

View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_nowallet
export DOCKER_NAME_TAG=ubuntu:18.04 # Use bionic to have one config run the tests in python3.6, see doc/dependencies.md
export PACKAGES="python3-zmq clang-5.0 llvm-5.0" # Use clang-5 to test C++17 compatibility, see doc/dependencies.md
export DEP_OPTS="NO_WALLET=1"
export GOAL="install"
export BITCOIN_CONFIG="--enable-glibc-back-compat --enable-reduce-exports CC=clang-5.0 CXX=clang++-5.0"

View File

@@ -0,0 +1,19 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_qt5
export DOCKER_NAME_TAG=ubuntu:18.04 # Check that bionic gcc-7 can compile our c++17 and run our functional tests in python3, see doc/dependencies.md
export PACKAGES="python3-zmq qtbase5-dev qttools5-dev-tools libdbus-1-dev libharfbuzz-dev"
export DEP_OPTS="NO_QT=1 NO_UPNP=1 NO_NATPMP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1"
export TEST_RUNNER_EXTRA="--previous-releases --coverage --extended --exclude feature_dbcrash" # Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
export RUN_UNIT_TESTS_SEQUENTIAL="true"
export RUN_UNIT_TESTS="false"
export GOAL="install"
export PREVIOUS_RELEASES_TO_DOWNLOAD="v0.15.2 v0.16.3 v0.17.2 v0.18.1 v0.19.1 v0.20.1"
export BITCOIN_CONFIG="--enable-zmq --with-libs=no --with-gui=qt5 --enable-glibc-back-compat --enable-reduce-exports
--enable-debug --disable-fuzz-binary CFLAGS=\"-g0 -O2 -funsigned-char\" CXXFLAGS=\"-g0 -O2 -funsigned-char\""

View File

@@ -0,0 +1,14 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_tsan
export DOCKER_NAME_TAG=ubuntu:hirsute
export PACKAGES="clang llvm libc++abi-dev libc++-dev python3-zmq"
export DEP_OPTS="CC=clang CXX='clang++ -stdlib=libc++'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-gui=no CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' CXXFLAGS='-g' --with-sanitizers=thread CC=clang CXX='clang++ -stdlib=libc++'"

View File

@@ -0,0 +1,15 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_valgrind
export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-dev libboost-system-dev libboost-filesystem-dev libboost-test-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libsqlite3-dev"
export USE_VALGRIND=1
export NO_DEPENDS=1
export TEST_RUNNER_EXTRA="--exclude rpc_bind" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no CC=clang CXX=clang++" # TODO enable GUI

26
ci/test/00_setup_env_s390x.sh Executable file
View File

@@ -0,0 +1,26 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=s390x-linux-gnu
# The host arch is unknown, so we run the tests through qemu.
# If the host is s390x and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
if [ -z ${QEMU_USER_CMD+x} ]; then export QEMU_USER_CMD="${QEMU_USER_CMD:-"qemu-s390x"}"; fi
export PACKAGES="python3-zmq"
if [ -n "$QEMU_USER_CMD" ]; then
# Likely cross-compiling, so install the needed gcc and qemu-user
export DPKG_ADD_ARCH="s390x"
export PACKAGES="$PACKAGES g++-s390x-linux-gnu qemu-user libc6:s390x libstdc++6:s390x libfontconfig1:s390x libxcb1:s390x"
fi
# Use debian to avoid 404 apt errors
export CONTAINER_NAME=ci_s390x
export DOCKER_NAME_TAG="debian:buster"
export RUN_UNIT_TESTS=true
export TEST_RUNNER_ENV="LC_ALL=C"
export RUN_FUNCTIONAL_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="--enable-reduce-exports --with-incompatible-bdb"

13
ci/test/00_setup_env_win64.sh Normal file → Executable file
View File

@@ -1,13 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_win64
export DOCKER_NAME_TAG=ubuntu:20.04 # Check that Focal can cross-compile to win64 (Focal is used in the gitian build as well)
export HOST=x86_64-w64-mingw32
export PACKAGES="python3 nsis g++-mingw-w64-x86-64 wine-binfmt wine64"
export DPKG_ADD_ARCH="i386"
export PACKAGES="python3 nsis g++-mingw-w64-x86-64 wine-binfmt wine64 wine32 file"
export RUN_FUNCTIONAL_TESTS=false
export GOAL="deploy"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests --disable-external-signer"
# Compiler for MinGW-w64 causes false -Wreturn-type warning.
# See https://sourceforge.net/p/mingw-w64/bugs/306/
export NO_WERROR=1

View File

@@ -1,26 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
# Add llvm-symbolizer directory to PATH. Needed to get symbolized stack traces from the sanitizers.
PATH=$PATH:/usr/lib/llvm-6.0/bin/
export PATH
BEGIN_FOLD () {
echo ""
CURRENT_FOLD_NAME=$1
echo "travis_fold:start:${CURRENT_FOLD_NAME}"
}
END_FOLD () {
RET=$?
echo "travis_fold:end:${CURRENT_FOLD_NAME}"
if [ $RET != 0 ]; then
echo "${CURRENT_FOLD_NAME} failed with status code ${RET}"
fi
}

View File

@@ -1,50 +1,122 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
mkdir -p "${BASE_SCRATCH_DIR}"
ccache echo "Creating ccache dir if it didn't already exist"
if [ ! -d ${DIR_QA_ASSETS} ]; then
git clone https://github.com/bitcoin-core/qa-assets ${DIR_QA_ASSETS}
if [[ $QEMU_USER_CMD == qemu-s390* ]]; then
export LC_ALL=C
fi
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
mkdir -p "${BASE_BUILD_DIR}/sanitizer-output/"
export ASAN_OPTIONS=""
export LSAN_OPTIONS="suppressions=${BASE_BUILD_DIR}/test/sanitizer_suppressions/lsan"
export TSAN_OPTIONS="suppressions=${BASE_BUILD_DIR}/test/sanitizer_suppressions/tsan:log_path=${BASE_BUILD_DIR}/sanitizer-output/tsan"
export UBSAN_OPTIONS="suppressions=${BASE_BUILD_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1"
env | grep -E '^(BITCOIN_CONFIG|CCACHE_|WINEDEBUG|LC_ALL|BOOST_TEST_RANDOM|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS)' | tee /tmp/env
if [[ $HOST = *-mingw32 ]]; then
DOCKER_ADMIN="--cap-add SYS_ADMIN"
elif [[ $BITCOIN_CONFIG = *--with-sanitizers=*address* ]]; then # If ran with (ASan + LSan), Docker needs access to ptrace (https://github.com/google/sanitizers/issues/764)
if [ "$CI_OS_NAME" == "macos" ]; then
sudo -H pip3 install --upgrade pip
IN_GETOPT_BIN="/usr/local/opt/gnu-getopt/bin/getopt" ${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
# Create folders that are mounted into the docker
mkdir -p "${CCACHE_DIR}"
mkdir -p "${PREVIOUS_RELEASES_DIR}"
export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1"
export LSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/lsan"
export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:halt_on_error=1:log_path=${BASE_SCRATCH_DIR}/sanitizer-output/tsan"
export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"
env | grep -E '^(BITCOIN_CONFIG|BASE_|QEMU_|CCACHE_|LC_ALL|BOOST_TEST_RANDOM|DEBIAN_FRONTEND|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS|PREVIOUS_RELEASES_DIR)' | tee /tmp/env
if [[ $BITCOIN_CONFIG = *--with-sanitizers=*address* ]]; then # If ran with (ASan + LSan), Docker needs access to ptrace (https://github.com/google/sanitizers/issues/764)
DOCKER_ADMIN="--cap-add SYS_PTRACE"
fi
if [ -z "$RUN_CI_ON_HOST" ]; then
export P_CI_DIR="$PWD"
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
echo "Creating $DOCKER_NAME_TAG container to run in"
${CI_RETRY_EXE} docker pull "$DOCKER_NAME_TAG"
DOCKER_ID=$(docker run $DOCKER_ADMIN -idt --mount type=bind,src=$BASE_BUILD_DIR,dst=$BASE_BUILD_DIR --mount type=bind,src=$CCACHE_DIR,dst=$CCACHE_DIR -w $BASE_BUILD_DIR --env-file /tmp/env $DOCKER_NAME_TAG)
if [ -n "${RESTART_CI_DOCKER_BEFORE_RUN}" ] ; then
echo "Restart docker before run to stop and clear all containers started with --rm"
systemctl restart docker
fi
DOCKER_EXEC () {
docker exec $DOCKER_ID bash -c "cd $PWD && $*"
}
DOCKER_ID=$(docker run $DOCKER_ADMIN --rm --interactive --detach --tty \
--mount type=bind,src=$BASE_ROOT_DIR,dst=/ro_base,readonly \
--mount type=bind,src=$CCACHE_DIR,dst=$CCACHE_DIR \
--mount type=bind,src=$DEPENDS_DIR,dst=$DEPENDS_DIR \
--mount type=bind,src=$PREVIOUS_RELEASES_DIR,dst=$PREVIOUS_RELEASES_DIR \
-w $BASE_ROOT_DIR \
--env-file /tmp/env \
--name $CONTAINER_NAME \
$DOCKER_NAME_TAG)
export DOCKER_CI_CMD_PREFIX="docker exec $DOCKER_ID"
else
echo "Running on host system without docker wrapper"
DOCKER_EXEC () {
bash -c "cd $PWD && $*"
}
fi
DOCKER_EXEC free -m -h
DOCKER_EXEC echo "Number of CPUs \(nproc\): $(nproc)"
DOCKER_EXEC () {
$DOCKER_CI_CMD_PREFIX bash -c "export PATH=$BASE_SCRATCH_DIR/bins/:\$PATH && cd $P_CI_DIR && $*"
}
export -f DOCKER_EXEC
${CI_RETRY_EXE} DOCKER_EXEC apt-get update
${CI_RETRY_EXE} DOCKER_EXEC apt-get install --no-install-recommends --no-upgrade -qq $PACKAGES $DOCKER_PACKAGES
if [ -n "$DPKG_ADD_ARCH" ]; then
DOCKER_EXEC dpkg --add-architecture "$DPKG_ADD_ARCH"
fi
if [[ $DOCKER_NAME_TAG == centos* ]]; then
${CI_RETRY_EXE} DOCKER_EXEC dnf -y install epel-release
${CI_RETRY_EXE} DOCKER_EXEC dnf -y --allowerasing install $DOCKER_PACKAGES $PACKAGES
elif [ "$CI_USE_APT_INSTALL" != "no" ]; then
${CI_RETRY_EXE} DOCKER_EXEC apt-get update
${CI_RETRY_EXE} DOCKER_EXEC apt-get install --no-install-recommends --no-upgrade -y $PACKAGES $DOCKER_PACKAGES
if [ -n "$PIP_PACKAGES" ]; then
${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
fi
if [ "$CI_OS_NAME" == "macos" ]; then
top -l 1 -s 0 | awk ' /PhysMem/ {print}'
echo "Number of CPUs: $(sysctl -n hw.logicalcpu)"
else
DOCKER_EXEC free -m -h
DOCKER_EXEC echo "Number of CPUs \(nproc\):" \$\(nproc\)
DOCKER_EXEC echo $(lscpu | grep Endian)
fi
DOCKER_EXEC echo "Free disk space:"
DOCKER_EXEC df -h
if [ "$RUN_FUZZ_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
if [ ! -d ${DIR_QA_ASSETS} ]; then
DOCKER_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets ${DIR_QA_ASSETS}
fi
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
export DIR_UNIT_TEST_DATA=${DIR_QA_ASSETS}/unit_test_data/
fi
DOCKER_EXEC mkdir -p "${BASE_SCRATCH_DIR}/sanitizer-output/"
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
DOCKER_EXEC "update-alternatives --install /usr/bin/clang++ clang++ \$(which clang++-9) 100"
DOCKER_EXEC "update-alternatives --install /usr/bin/clang clang \$(which clang-9) 100"
DOCKER_EXEC "mkdir -p ${BASE_SCRATCH_DIR}/msan/build/"
DOCKER_EXEC "git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-12.0.0 ${BASE_SCRATCH_DIR}/msan/llvm-project"
DOCKER_EXEC "cd ${BASE_SCRATCH_DIR}/msan/build/ && cmake -DLLVM_ENABLE_PROJECTS='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=Memory -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_TARGETS_TO_BUILD=X86 ../llvm-project/llvm/"
DOCKER_EXEC "cd ${BASE_SCRATCH_DIR}/msan/build/ && make $MAKEJOBS cxx"
fi
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
echo "Create $BASE_ROOT_DIR"
DOCKER_EXEC rsync -a /ro_base/ $BASE_ROOT_DIR
fi
if [ "$USE_BUSY_BOX" = "true" ]; then
echo "Setup to use BusyBox utils"
DOCKER_EXEC mkdir -p $BASE_SCRATCH_DIR/bins/
# tar excluded for now because it requires passing in the exact archive type in ./depends (fixed in later BusyBox version)
# find excluded for now because it does not recognize the -delete option in ./depends (fixed in later BusyBox version)
# ar excluded for now because it does not recognize the -q option in ./depends (unknown if fixed)
# shellcheck disable=SC1010
DOCKER_EXEC for util in \$\(busybox --list \| grep -v "^ar$" \| grep -v "^tar$" \| grep -v "^find$"\)\; do ln -s \$\(command -v busybox\) $BASE_SCRATCH_DIR/bins/\$util\; done
# Print BusyBox version
DOCKER_EXEC patch --help
fi

View File

@@ -1,24 +1,60 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
DOCKER_EXEC echo \> \$HOME/.bitcoin # Make sure default datadir does not exist and is never read by creating a dummy file
mkdir -p depends/SDKs depends/sdk-sources
if [ -n "$OSX_SDK" ] && [ ! -f depends/sdk-sources/MacOSX${OSX_SDK}.sdk.tar.gz ]; then
curl --location --fail $SDK_URL/MacOSX${OSX_SDK}.sdk.tar.gz -o depends/sdk-sources/MacOSX${OSX_SDK}.sdk.tar.gz
# Make sure default datadir does not exist and is never read by creating a dummy file
if [ "$CI_OS_NAME" == "macos" ]; then
echo > $HOME/Library/Application\ Support/Bitcoin
else
DOCKER_EXEC echo \> \$HOME/.bitcoin
fi
if [ -n "$OSX_SDK" ] && [ -f depends/sdk-sources/MacOSX${OSX_SDK}.sdk.tar.gz ]; then
tar -C depends/SDKs -xf depends/sdk-sources/MacOSX${OSX_SDK}.sdk.tar.gz
DOCKER_EXEC mkdir -p ${DEPENDS_DIR}/SDKs ${DEPENDS_DIR}/sdk-sources
OSX_SDK_BASENAME="Xcode-${XCODE_VERSION}-${XCODE_BUILD_ID}-extracted-SDK-with-libcxx-headers.tar.gz"
OSX_SDK_PATH="${DEPENDS_DIR}/sdk-sources/${OSX_SDK_BASENAME}"
if [ -n "$XCODE_VERSION" ] && [ ! -f "$OSX_SDK_PATH" ]; then
DOCKER_EXEC curl --location --fail "${SDK_URL}/${OSX_SDK_BASENAME}" -o "$OSX_SDK_PATH"
fi
if [ -n "$ANDROID_TOOLS_URL" ]; then
ANDROID_TOOLS_PATH=$DEPENDS_DIR/sdk-sources/android-tools.zip
DOCKER_EXEC curl --location --fail "${ANDROID_TOOLS_URL}" -o "$ANDROID_TOOLS_PATH"
DOCKER_EXEC mkdir -p "${ANDROID_HOME}/cmdline-tools"
DOCKER_EXEC unzip -o "$ANDROID_TOOLS_PATH" -d "${ANDROID_HOME}/cmdline-tools"
DOCKER_EXEC "yes | ${ANDROID_HOME}/cmdline-tools/tools/bin/sdkmanager --install \"build-tools;${ANDROID_BUILD_TOOLS_VERSION}\" \"platform-tools\" \"platforms;android-${ANDROID_API_LEVEL}\" \"ndk;${ANDROID_NDK_VERSION}\""
fi
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
# Use BDB compiled using install_db4.sh script to work around linking issue when using BDB
# from depends. See https://github.com/bitcoin/bitcoin/pull/18288#discussion_r433189350 for
# details.
DOCKER_EXEC "contrib/install_db4.sh \$(pwd) --enable-umrw CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
fi
if [ -n "$XCODE_VERSION" ] && [ -f "$OSX_SDK_PATH" ]; then
DOCKER_EXEC tar -C "${DEPENDS_DIR}/SDKs" -xf "$OSX_SDK_PATH"
fi
if [[ $HOST = *-mingw32 ]]; then
DOCKER_EXEC update-alternatives --set $HOST-g++ \$\(which $HOST-g++-posix\)
fi
if [ -z "$NO_DEPENDS" ]; then
DOCKER_EXEC CONFIG_SHELL= make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS
if [[ $DOCKER_NAME_TAG == centos* ]]; then
# CentOS has problems building the depends if the config shell is not explicitly set
# (i.e. for libevent a Makefile with an empty SHELL variable is generated, leading to
# an error as the first command is executed)
SHELL_OPTS="LC_ALL=en_US.UTF-8 CONFIG_SHELL=/bin/bash"
else
SHELL_OPTS="CONFIG_SHELL="
fi
DOCKER_EXEC $SHELL_OPTS make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS
fi
if [ -n "$PREVIOUS_RELEASES_TO_DOWNLOAD" ]; then
DOCKER_EXEC test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR" "${PREVIOUS_RELEASES_TO_DOWNLOAD}"
fi

View File

@@ -1,46 +1,55 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
BITCOIN_CONFIG_ALL="--disable-dependency-tracking --prefix=$BASE_BUILD_DIR/depends/$HOST --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
if [ -z "$NO_DEPENDS" ]; then
DOCKER_EXEC ccache --max-size=$CCACHE_SIZE
if [ -n "$ANDROID_TOOLS_URL" ]; then
DOCKER_EXEC make distclean || true
DOCKER_EXEC ./autogen.sh
DOCKER_EXEC ./configure $BITCOIN_CONFIG --prefix=$DEPENDS_DIR/aarch64-linux-android || ( (DOCKER_EXEC cat config.log) && false)
DOCKER_EXEC "cd src/qt && make $MAKEJOBS && ANDROID_HOME=${ANDROID_HOME} ANDROID_NDK_HOME=${ANDROID_NDK_HOME} make apk"
exit 0
fi
BEGIN_FOLD autogen
BITCOIN_CONFIG_ALL="--enable-suppress-external-warnings --disable-dependency-tracking --prefix=$DEPENDS_DIR/$HOST --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
if [ -z "$NO_WERROR" ]; then
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-werror"
fi
DOCKER_EXEC "ccache --zero-stats --max-size=$CCACHE_SIZE"
if [ -n "$CONFIG_SHELL" ]; then
DOCKER_EXEC "$CONFIG_SHELL" -c "./autogen.sh"
else
DOCKER_EXEC ./autogen.sh
fi
END_FOLD
mkdir -p build
cd build || (echo "could not enter build directory"; exit 1)
DOCKER_EXEC mkdir -p "${BASE_BUILD_DIR}"
export P_CI_DIR="${BASE_BUILD_DIR}"
BEGIN_FOLD configure
DOCKER_EXEC ../configure --cache-file=config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG || ( cat config.log && false)
END_FOLD
DOCKER_EXEC "${BASE_ROOT_DIR}/configure" --cache-file=config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG || ( (DOCKER_EXEC cat config.log) && false)
BEGIN_FOLD distdir
DOCKER_EXEC make distdir VERSION=$HOST
END_FOLD
cd "bitcoin-$HOST" || (echo "could not enter distdir bitcoin-$HOST"; exit 1)
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST"
BEGIN_FOLD configure
DOCKER_EXEC ./configure --cache-file=../config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG || ( cat config.log && false)
END_FOLD
DOCKER_EXEC ./configure --cache-file=../config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG || ( (DOCKER_EXEC cat config.log) && false)
set -o errtrace
trap 'DOCKER_EXEC "cat ${BASE_BUILD_DIR}/sanitizer-output/* 2> /dev/null"' ERR
trap 'DOCKER_EXEC "cat ${BASE_SCRATCH_DIR}/sanitizer-output/* 2> /dev/null"' ERR
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
# MemorySanitizer (MSAN) does not support tracking memory initialization done by
# using the Linux getrandom syscall. Avoid using getrandom by undefining
# HAVE_SYS_GETRANDOM. See https://github.com/google/sanitizers/issues/852 for
# details.
DOCKER_EXEC 'grep -v HAVE_SYS_GETRANDOM src/config/bitcoin-config.h > src/config/bitcoin-config.h.tmp && mv src/config/bitcoin-config.h.tmp src/config/bitcoin-config.h'
fi
BEGIN_FOLD build
DOCKER_EXEC make $MAKEJOBS $GOAL || ( echo "Build failure. Verbose build follows." && DOCKER_EXEC make $GOAL V=1 ; false )
END_FOLD
cd ${BASE_BUILD_DIR} || (echo "could not enter travis build dir $BASE_BUILD_DIR"; exit 1)
DOCKER_EXEC "ccache --version | head -n 1 && ccache --show-stats"
DOCKER_EXEC du -sh "${DEPENDS_DIR}"/*/
DOCKER_EXEC du -sh "${PREVIOUS_RELEASES_DIR}"

View File

@@ -1,29 +1,45 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018 The Bitcoin Core developers
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
cd "build/bitcoin-$HOST" || (echo "could not enter distdir build/bitcoin-$HOST"; exit 1)
if [[ $HOST = *-mingw32 ]]; then
# Generate all binaries, so that they can be wrapped
DOCKER_EXEC make $MAKEJOBS -C src/secp256k1 VERBOSE=1
DOCKER_EXEC make $MAKEJOBS -C src/univalue VERBOSE=1
DOCKER_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-wine.sh"
fi
if [ -n "$QEMU_USER_CMD" ]; then
# Generate all binaries, so that they can be wrapped
DOCKER_EXEC make $MAKEJOBS -C src/secp256k1 VERBOSE=1
DOCKER_EXEC make $MAKEJOBS -C src/univalue VERBOSE=1
DOCKER_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-qemu.sh"
fi
if [ -n "$USE_VALGRIND" ]; then
DOCKER_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-valgrind.sh"
fi
if [ "$RUN_UNIT_TESTS" = "true" ]; then
BEGIN_FOLD unit-tests
DOCKER_EXEC LD_LIBRARY_PATH=$BASE_BUILD_DIR/depends/$HOST/lib make $MAKEJOBS check VERBOSE=1
END_FOLD
DOCKER_EXEC ${TEST_RUNNER_ENV} DIR_UNIT_TEST_DATA=${DIR_UNIT_TEST_DATA} LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib make $MAKEJOBS check VERBOSE=1
fi
if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
DOCKER_EXEC ${TEST_RUNNER_ENV} DIR_UNIT_TEST_DATA=${DIR_UNIT_TEST_DATA} LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib "${BASE_BUILD_DIR}/bitcoin-*/src/test/test_bitcoin*" --catch_system_errors=no -l test_suite
fi
if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
BEGIN_FOLD functional-tests
DOCKER_EXEC test/functional/test_runner.py --ci $MAKEJOBS --tmpdirprefix "${BASE_SCRATCH_DIR}/test_runner/" --ansi --combinedlogslen=4000 ${TEST_RUNNER_EXTRA} --quiet --failfast
END_FOLD
DOCKER_EXEC LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib ${TEST_RUNNER_ENV} test/functional/test_runner.py --ci $MAKEJOBS --tmpdirprefix "${BASE_SCRATCH_DIR}/test_runner/" --ansi --combinedlogslen=4000 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA} --quiet --failfast
fi
if [ "$RUN_SECURITY_TESTS" = "true" ]; then
DOCKER_EXEC make test-security-check
fi
if [ "$RUN_FUZZ_TESTS" = "true" ]; then
BEGIN_FOLD fuzz-tests
DOCKER_EXEC test/fuzz/test_runner.py -l DEBUG ${DIR_FUZZ_IN}
END_FOLD
DOCKER_EXEC LD_LIBRARY_PATH=$DEPENDS_DIR/$HOST/lib test/fuzz/test_runner.py ${FUZZ_TESTS_CONFIG} $MAKEJOBS -l DEBUG ${DIR_FUZZ_IN}
fi
cd ${BASE_BUILD_DIR} || (echo "could not enter travis build dir $BASE_BUILD_DIR"; exit 1)

18
ci/test/wrap-qemu.sh Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/univalue/{no_nul,test_json,unitester,object}}; do
# shellcheck disable=SC2044
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name $(basename $b_name)); do
echo "Wrap $b ..."
mv "$b" "${b}_orig"
echo '#!/usr/bin/env bash' > "$b"
echo "$QEMU_USER_CMD \"${b}_orig\" \"\$@\"" >> "$b"
chmod +x "$b"
done
done

18
ci/test/wrap-valgrind.sh Executable file
View File

@@ -0,0 +1,18 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
for b_name in "${BASE_OUTDIR}/bin"/*; do
# shellcheck disable=SC2044
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name $(basename $b_name)); do
echo "Wrap $b ..."
mv "$b" "${b}_orig"
echo '#!/usr/bin/env bash' > "$b"
echo "valgrind --gen-suppressions=all --quiet --error-exitcode=1 --suppressions=${BASE_ROOT_DIR}/contrib/valgrind.supp \"${b}_orig\" \"\$@\"" >> "$b"
chmod +x "$b"
done
done

20
ci/test/wrap-wine.sh Executable file
View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/univalue/{no_nul,test_json,unitester,object}}.exe; do
# shellcheck disable=SC2044
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename $b_name)"); do
if (file "$b" | grep "Windows"); then
echo "Wrap $b ..."
mv "$b" "${b}_orig"
echo '#!/usr/bin/env bash' > "$b"
echo "( wine \"${b}_orig\" \"\$@\" ) || ( sleep 1 && wine \"${b}_orig\" \"\$@\" )" >> "$b"
chmod +x "$b"
fi
done
done

View File

@@ -1,13 +1,12 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019 The Bitcoin Core developers
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/test/03_before_install.sh
set -o errexit; source ./ci/test/04_install.sh
set -o errexit; source ./ci/test/05_before_script.sh
set -o errexit; source ./ci/test/06_script_a.sh

File diff suppressed because it is too large Load Diff

View File

@@ -29,8 +29,8 @@ All other packaging related files can be found in the [bitcoin-core/packaging](h
### [Gitian-descriptors](/contrib/gitian-descriptors) ###
Files used during the gitian build process. For more information about gitian, see the [the Bitcoin Core documentation repository](https://github.com/bitcoin-core/docs).
### [Gitian-keys](/contrib/gitian-keys)
PGP keys used for signing Bitcoin Core [Gitian release](/doc/release-process.md) results.
### [Builder keys](/contrib/builder-keys)
PGP keys used for signing Bitcoin Core [release](/doc/release-process.md) results.
### [MacDeploy](/contrib/macdeploy) ###
Scripts and notes for Mac builds.

View File

@@ -1,5 +1,5 @@
# bash programmable completion for bitcoin-cli(1)
# Copyright (c) 2012-2016 The Bitcoin Core developers
# Copyright (c) 2012-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -17,13 +17,6 @@ _bitcoin_rpc() {
$bitcoin_cli "${rpcargs[@]}" "$@"
}
# Add wallet accounts to COMPREPLY
_bitcoin_accounts() {
local accounts
accounts=$(_bitcoin_rpc listaccounts | awk -F '"' '{ print $2 }')
COMPREPLY=( "${COMPREPLY[@]}" $( compgen -W "$accounts" -- "$cur" ) )
}
_bitcoin_cli() {
local cur prev words=() cword
local bitcoin_cli
@@ -60,10 +53,9 @@ _bitcoin_cli() {
if ((cword > 3)); then
case ${words[cword-3]} in
addmultisigaddress)
_bitcoin_accounts
return 0
;;
getbalance|gettxout|importaddress|importpubkey|importprivkey|listreceivedbyaccount|listreceivedbyaddress|listsinceblock)
getbalance|gettxout|importaddress|importpubkey|importprivkey|listreceivedbyaddress|listsinceblock)
COMPREPLY=( $( compgen -W "true false" -- "$cur" ) )
return 0
;;
@@ -80,14 +72,10 @@ _bitcoin_cli() {
COMPREPLY=( $( compgen -W "add remove" -- "$cur" ) )
return 0
;;
fundrawtransaction|getblock|getblockheader|getmempoolancestors|getmempooldescendants|getrawtransaction|gettransaction|listaccounts|listreceivedbyaccount|listreceivedbyaddress|sendrawtransaction)
fundrawtransaction|getblock|getblockheader|getmempoolancestors|getmempooldescendants|getrawtransaction|gettransaction|listreceivedbyaddress|sendrawtransaction)
COMPREPLY=( $( compgen -W "true false" -- "$cur" ) )
return 0
;;
move|setaccount)
_bitcoin_accounts
return 0
;;
esac
fi
@@ -96,12 +84,11 @@ _bitcoin_cli() {
_filedir
return 0
;;
getaddednodeinfo|getrawmempool|lockunspent|setgenerate)
getaddednodeinfo|getrawmempool|lockunspent)
COMPREPLY=( $( compgen -W "true false" -- "$cur" ) )
return 0
;;
getaccountaddress|getaddressesbyaccount|getbalance|getnewaddress|getreceivedbyaccount|listtransactions|move|sendfrom|sendmany)
_bitcoin_accounts
getbalance|getnewaddress|listtransactions|sendmany)
return 0
;;
esac

View File

@@ -1,22 +0,0 @@
FORMS += \
../src/qt/forms/aboutdialog.ui \
../src/qt/forms/addressbookpage.ui \
../src/qt/forms/askpassphrasedialog.ui \
../src/qt/forms/coincontroldialog.ui \
../src/qt/forms/editaddressdialog.ui \
../src/qt/forms/helpmessagedialog.ui \
../src/qt/forms/intro.ui \
../src/qt/forms/openuridialog.ui \
../src/qt/forms/optionsdialog.ui \
../src/qt/forms/overviewpage.ui \
../src/qt/forms/receivecoinsdialog.ui \
../src/qt/forms/receiverequestdialog.ui \
../src/qt/forms/debugwindow.ui \
../src/qt/forms/sendcoinsdialog.ui \
../src/qt/forms/sendcoinsentry.ui \
../src/qt/forms/signverifymessagedialog.ui \
../src/qt/forms/transactiondescdialog.ui \
../src/qt/forms/createwalletdialog.ui
RESOURCES += \
../src/qt/bitcoin.qrc

View File

@@ -1,5 +1,5 @@
# bash programmable completion for bitcoind(1) and bitcoin-qt(1)
# Copyright (c) 2012-2016 The Bitcoin Core developers
# Copyright (c) 2012-2019 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -15,7 +15,7 @@ _bitcoind() {
_get_comp_words_by_ref -n = cur prev words cword
case "$cur" in
-conf=*|-pid=*|-loadblock=*|-rootcertificates=*|-rpccookiefile=*|-wallet=*)
-conf=*|-pid=*|-loadblock=*|-rpccookiefile=*|-wallet=*)
cur="${cur#*=}"
_filedir
return 0

View File

@@ -1,10 +1,10 @@
## PGP keys of Gitian builders and Developers
## PGP keys of builders and Developers
The file `keys.txt` contains fingerprints of the public keys of Gitian builders
and active developers.
The file `keys.txt` contains fingerprints of the public keys of builders and
active developers.
The associated keys are mainly used to sign git commits or the build results
of Gitian builds.
of Guix builds.
The most recent version of each pgp key can be found on most pgp key servers.
@@ -16,12 +16,12 @@ To fetch the latest version of all pgp keys in your gpg homedir,
gpg --refresh-keys
```
To fetch keys of Gitian builders and active developers, feed the list of
fingerprints of the primary keys into gpg:
To fetch keys of builders and active developers, feed the list of fingerprints
of the primary keys into gpg:
```sh
while read fingerprint keyholder_name; do gpg --keyserver hkp://subset.pool.sks-keyservers.net --recv-keys ${fingerprint}; done < ./keys.txt
```
Add your key to the list if you provided Gitian signatures for two major or
Add your key to the list if you provided Guix attestations for two major or
minor releases of Bitcoin Core.

View File

@@ -0,0 +1,51 @@
9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C Aaron Clauson (sipsorcery)
617C90010B3BD370B0AC7D424BB42E31C79111B8 Akira Takizawa (akx20000)
E944AE667CF960B1004BC32FCA662BE18B877A60 Andreas Schildbach (aschildbach)
152812300785C96444D3334D17565732E08E5E41 Andrew Chow (achow101)
590B7292695AFFA5B672CBB2E13FC145CD3F4304 Antoine Poinsot (darosior)
0AD83877C1F0CD1EE9BD660AD7CC770B81FD22A8 Ben Carman (benthecarman)
912FD3228387123DC97E0E57D5566241A0295FA9 BtcDrak (btcdrak)
C519EBCF3B926298946783EFF6430754120EC2F4 Christian Decker (cdecker)
18AE2F798E0D239755DA4FD24B79F986CBDF8736 Chun Kuan Le (ken2812221)
101598DC823C1B5F9A6624ABA5E0907A0380E6C3 CoinForensics (CoinForensics)
F20F56EF6A067F70E8A5C99FFF95FAA971697405 centaur (centaur)
C060A6635913D98A3587D7DB1C2491FFEB0EF770 Cory Fields (cfields)
BF6273FAEF7CC0BA1F562E50989F6B3048A116B5 Dev Random (devrandom)
6D3170C1DC2C6FD0AEEBCA6743811D1A26623924 Douglas Roark (droark)
1C6621605EC50319C463D56C7F81D87985D61612 Emanuele Cisbani (cisba)
9A1689B60D1B3CCE9262307A2F40A9BF167FBA47 Erik Mossberg (erkmos)
D35176BE9264832E4ACA8986BF0792FBE95DC863 fivepiece (fivepiece)
6F993B250557E7B016ADE5713BDCDA2D87A881D9 Fuzzbawls (Fuzzbawls)
01CDF4627A3B88AAE4A571C87588242FBE38D3A8 Gavin Andresen (gavinandresen)
D1DBF2C4B96F2DEBF4C16654410108112E7EA81F Hennadii Stepanov (hebasto)
A2FD494D0021AA9B4FA58F759102B7AE654A4A5A Ilyas Ridhuan (IlyasRidhuan)
D3F22A3A4C366C2DCB66D3722DA9C5A7FA81EA35 Jarol Rodriguez (jarolrod)
7480909378D544EA6B6DCEB7535B12980BB8A4D3 Jeffri H Frontz (jhfrontz)
D3CC177286005BB8FF673294C5242A1AB3936517 jl2012 (jl2012)
82921A4B88FD454B7EB8CE3C796C4109063D4EAF Jon Atack (jonatack)
32EE5C4C3FA15CCADB46ABE529D4BCB6416F53EC Jonas Schnelli (jonasschnelli)
4B4E840451149DD7FB0D633477DFAB5C3108B9A8 Jorge Timon (jtimon)
C42AFF7C61B3E44A1454CD3557AF762DB3353322 Karl-Johan Alm (kallewoof)
30DE693AE0DE9E37B3E7EB6BBFF0F67810C1EED1 Lisa Neigut (niftynei)
E463A93F5F3117EEDE6C7316BD02942421F4889F Luke Dashjr (luke-jr)
B8B3F1C0E58C15DB6A81D30C3648A882F4316B9B Marco Falke (marco)
07DF3E57A548CCFB7530709189BBB8663E2E65CE Matt Corallo (BlueMatt)
CA03882CB1FC067B5D3ACFE4D300116E1C875A3D MeshCollider (meshcollider)
E777299FC265DD04793070EB944D35F9AC3DB76A Michael Ford (fanquake)
AD5764F4ADCE1B99BDFD179E12335A271D4D62EC Michael Tidwell (miketwenty1)
9692B91BBF0E8D34DFD33B1882C5C009628ECF0C Michagogo (michagogo)
C57E4B42223FDE851D4F69DD28DF2724F241D8EE midnightmagic (midnightmagic)
F4FC70F07310028424EFC20A8E4256593F177720 Oliver Gugger (guggero, Oliver Gugger)
D62A803E27E7F43486035ADBBCD04D8E9CCCAC2A Paul Rabahy (prab)
37EC7D7B0A217CDB4B4E007E7FAB114267E4FA04 Peter Todd (petertodd)
D762373D24904A3E42F33B08B9A408E71DAAC974 Pieter Wuille [Location: Leuven, Belgium] (sipa)
133EAC179436F14A5CF1B794860FEB804E669320 Pieter Wuille (sipa)
6A8F9C266528E25AEB1D7731C2371D91CB716EA7 Sebastian Falbesoner (theStack)
A8FC55F3B04BA3146F3492E79303B33A305224CB Sebastian Kung (TheCharlatan)
ED9BDF7AD6A55E232E84524257FF9BDBCC301009 Sjors Provoost (sjors)
867345026B6763E8B07EE73AB6737117397F5C4F Stephan Oeste (Emzy)
9EDAFF80E080659604F4A76B2EBB056FD847F8A7 Stephan Oeste (Emzy)
6DEEF79B050C4072509B743F8C275BC595448867 Tomas Kanocz (KanoczTomas)
AEC1884398647C47413C1C3FB1179EB7347DC10D Warren Togami (wtogami)
79D00BAC68B56D422F945A8F8E3A8F3247DBCBBF Willy Ko (willyko)
71A3B16735405025D447E8F274810B012346C9A6 Wladimir J. van der Laan (laanwj)

Some files were not shown because too many files have changed in this diff Show More