Compare commits

...

3869 Commits

Author SHA1 Message Date
glozow
7b7041019b Merge bitcoin/bitcoin#30376: 26.2 final changes
eef5dbc21b [doc] update release notes 26.2 (glozow)
00f0267ac0 [doc] update manpages 26.2 (glozow)
29cbec45dd [build] bump copyright year (glozow)
fe8dee826d [build] bump version to 26.2 (glozow)

Pull request description:

  bins were uploaded 2 weeks ago on June 18
  website PR: https://github.com/bitcoin-core/bitcoincore.org/pull/1039

ACKs for top commit:
  stickies-v:
    ACK eef5dbc21b
  fanquake:
    ACK eef5dbc21b If you want, you could also backport the changes to get the ASAN job running again, but that isn't blocking here.
  theStack:
    ACK eef5dbc21b

Tree-SHA512: 7c9e97231fd51784f1cc78a9b7b07b8a201ad7f54715fab6dd3243244e9f52831f57883966f2061cbff7a68018b4345de27e3953c50d7ec801d1a61f216907d1
2024-07-03 16:49:43 +01:00
glozow
eef5dbc21b [doc] update release notes 26.2 2024-07-02 10:41:25 +01:00
glozow
00f0267ac0 [doc] update manpages 26.2 2024-07-02 10:41:25 +01:00
glozow
29cbec45dd [build] bump copyright year 2024-07-02 10:34:39 +01:00
glozow
fe8dee826d [build] bump version to 26.2 2024-07-02 10:33:47 +01:00
merge-script
6bbfdcfb6d Merge bitcoin/bitcoin#30319: [26.x] upnp: fix build with miniupnpc 2.2.8
10413ac46c doc: add 30283 cherry-pick to rel notes (fanquake)
391ce775f4 upnp: add compatibility for miniupnpc 2.2.8 (Cory Fields)

Pull request description:

  Backports https://github.com/bitcoin/bitcoin/pull/30283 to the 26.x branch.

ACKs for top commit:
  edilmedeiros:
    ACK 10413ac46c
  theuni:
    ACK 10413ac46c

Tree-SHA512: 4a0f4eceefd5bbf9c97d19c4890b85963d56449856a56e6fe24161d3d6f37332de719da342d6c00ee67f2cd9434d849809a3cdc51719dc93219ec218c35a9f97
2024-06-24 13:37:23 +01:00
fanquake
10413ac46c doc: add 30283 cherry-pick to rel notes 2024-06-21 15:45:56 +01:00
Cory Fields
391ce775f4 upnp: add compatibility for miniupnpc 2.2.8
See: c0a50ce33e

The return value of 2 now indicates:
"A valid connected IGD has been found but its IP address is reserved (non routable)"

We continue to ignore any return value other than 1.

Github-Pull: #30283
Rebased-From: 8acdf66540
2024-06-21 15:42:38 +01:00
glozow
d10cf127e5 Merge bitcoin/bitcoin#30260: [26.x] backports and final changes for 26.2rc1
7ca424f8e6 [doc] update release notes for 26.2rc1 (glozow)
03a91a0142 [doc] manpages for 26.2rc1 (glozow)
4c824bbe03 [build] bump to version 26.2rc1 (glozow)
b99229737d qt: 26.2rc1 translations update (Hennadii Stepanov)
7b2489f57f depends: Fetch miniupnpc sources from an alternative website (Hennadii Stepanov)

Pull request description:

  Backports + release notes for:
  - #30151

  And final changes for 26.2rc1:
  - build changes
  - manpages
  - non-removal translations updates

ACKs for top commit:
  stickies-v:
    ACK 7ca424f8e6
  fanquake:
    ACK 7ca424f8e6

Tree-SHA512: 881697eb6fb78261e3deb5064d6f38227b449049add810c7da999a1f22fad60b9dd32057c626bbfaa7e461d53635cd2f7a9fb225eba0f6fb503afd4f1ef6c46d
2024-06-12 13:06:12 +01:00
glozow
7ca424f8e6 [doc] update release notes for 26.2rc1 2024-06-10 14:53:16 +01:00
glozow
03a91a0142 [doc] manpages for 26.2rc1 2024-06-10 14:50:34 +01:00
glozow
4c824bbe03 [build] bump to version 26.2rc1 2024-06-10 14:50:34 +01:00
Hennadii Stepanov
b99229737d qt: 26.2rc1 translations update
This change is a result if pulling the recent translations
from the Transifex website using the
bitcoin-maintainer-tools/update-translations.py tool.

A few manual adjustments were made:
- skipped removing of `bitcoin_af.ts`
- skipped removing of `bitcoin_ar.ts`
- skipped removing of `bitcoin_fa.ts`
- skipped removing of `bitcoin_la.ts`
- skipped removing of `bitcoin_ru.ts`
- skipped removing of `bitcoin_th.ts`
- skipped removing of `bitcoin_uk.ts`
present)
2024-06-10 14:50:28 +01:00
Hennadii Stepanov
7b2489f57f depends: Fetch miniupnpc sources from an alternative website
The https://miniupnp.tuxfamily.org website is unavailable now.

Rebased-From: 21b8a14d37
Github-Pull: #30151
2024-06-10 14:08:42 +01:00
glozow
6d7a1e3670 Merge bitcoin/bitcoin#29899: [26.x] archive 26.1 release notes + backports
aa7e876953 [doc] add draft release notes for 26.2rc1 (glozow)
21d9aaae32 p2p, bugfix: detect addnode cjdns peers in GetAddedNodeInfo() (Jon Atack)
ec5ce2fb2b windeploy: Renew certificate (Ava Chow)
96d0e818c5 rpc: Reword SighashFromStr error message (MarcoFalke)
6685affe92 rpc: move UniValue in blockToJSON (willcl-ark)
7f45e00174 depends: Fix build of Qt for 32-bit platforms (laanwj)
f9b76bae57 ci: Pull in qtbase5-dev instead of seperate low-level libraries (laanwj)
c587753f48 doc: Suggest installing dev packages for debian/ubuntu qt5 build (laanwj)
7ecdb0846d ci: Bump s390x to ubuntu:24.04 (MarcoFalke)
d9ef6cf934 sign: don't assume we are parsing a sane Miniscript (Antoine Poinsot)
e4859c82c7 depends: fix mingw-w64 Qt DEBUG=1 build (fanquake)
bb46b90b2e Fix #29767, set m_synced = true after Commit() (nanlour)
bf5b6fc8a7 Throw error if invalid parameters passed to getnetworkhashps RPC endpoint (Jameson Lopp)
a81a9228fb [rpc, bugfix] Enforce maximum value for setmocktime (dergoegge)
d39ea519c7 Change Luke Dashjr seed to dashjr-list-of-p2p-nodes.us (Luke Dashjr)
c21bbcc898 [doc] archive 26.1 release notes (glozow)

Pull request description:

  Archives 26.1 release notes and adds draft release notes for 26.2rc1

  Also backports:
  - #29691
  - #29869
  - #28554
  - #29747
  - #29853
  - #29856
  - #29764
  - #29776
  - #29985
  - #30094
  - #29870
  - #30149
  - #30085

ACKs for top commit:
  stickies-v:
    re-ACK aa7e876953, only changes are fixing commit msg and transifex reference
  willcl-ark:
    ACK aa7e876953

Tree-SHA512: b81ba6092640de696d782114cdf43e7ed1d63ea0a3231cade30653c2743d87700e0f852a1b1fcc42ae313b2d4f004e6026ddbad87d58c2fde0a660e90026ed98
2024-05-24 11:22:00 +01:00
glozow
aa7e876953 [doc] add draft release notes for 26.2rc1 2024-05-23 17:06:26 +01:00
Jon Atack
21d9aaae32 p2p, bugfix: detect addnode cjdns peers in GetAddedNodeInfo()
Addnode (manual) peers connected to us via the cjdns network are currently not
detected by CConnman::GetAddedNodeInfo(), i.e. fConnected is always false.

This causes the following issues:

- RPC `getaddednodeinfo` incorrectly shows them as not connected

- CConnman::ThreadOpenAddedConnections() continually retries to connect them

Github-Pull: #30085
Rebased-From: 684da97070
2024-05-23 17:05:57 +01:00
Ava Chow
ec5ce2fb2b windeploy: Renew certificate
Github-Pull: #30149
Rebased-From: 9f4ff1e965
2024-05-23 17:05:57 +01:00
MarcoFalke
96d0e818c5 rpc: Reword SighashFromStr error message
Github-Pull: #29870
Rebased-From: fa6ab0d020
2024-05-23 17:05:57 +01:00
willcl-ark
6685affe92 rpc: move UniValue in blockToJSON
Without explicitly declaring the move, these UniValues get copied,
causing increased memory usage. Fix this by explicitly moving the
UniValue objects.

Used by `rest_block` and `getblock` RPC.

Github-Pull: #30094
Rebased-From: b77bad309e
2024-05-23 17:05:57 +01:00
laanwj
7f45e00174 depends: Fix build of Qt for 32-bit platforms
The 32 to 64-bit time_t transition causes a build failure in the built-in
zlib about conflicting _TIME_BITS and _FILE_OFFSET_BITS.

Note that zlib doesn't use time_t at all, so it is a false alarm.

Take the following patch from upstream zlib:
a566e156b3.patch

Closes #29980.

Github-Pull: #29985
Rebased-From: 2e266f33b5
2024-05-23 17:05:57 +01:00
laanwj
f9b76bae57 ci: Pull in qtbase5-dev instead of seperate low-level libraries
Fix CI build for t64 migration.

Github-Pull: #29764
Rebased-From: 6c2990416e
2024-05-23 17:05:57 +01:00
laanwj
c587753f48 doc: Suggest installing dev packages for debian/ubuntu qt5 build
Pretty much all library packages were renamed in the 64-bit time_t
migration to add `t64` (even on 64-bit platforms).

Instead of complicating the doc with conditional package names, suggest
installing the `-dev` packages which still have the same name, and
besides that, are the right way to go about it as they contain the
"user facing" C++ headers needed to build against Qt5.

For Fedora, devel packages are already suggested.

This affects Ubuntu 24.04 and Debian Testing.

Github-Pull: #29764
Rebased-From: a3c6a13cb2
2024-05-23 17:05:57 +01:00
MarcoFalke
7ecdb0846d ci: Bump s390x to ubuntu:24.04
Re-enable feature_init

Github-Pull: #29856
Rebased-From: fadf7e90dc
2024-05-23 17:05:57 +01:00
Antoine Poinsot
d9ef6cf934 sign: don't assume we are parsing a sane Miniscript
The script provided for signature might be externally provided, for
instance by way of 'finalizepsbt'. Therefore the script might be
ill-crafted, so don't assume pubkeys are always 32 bytes.

Thanks to Niklas for finding this.

Github-Pull: #29853
Rebased-From: 4d8d21320e
2024-05-23 17:05:57 +01:00
fanquake
e4859c82c7 depends: fix mingw-w64 Qt DEBUG=1 build
The issue is that compilation is done with `x86_64-w64-mingw32-g++-posix`,
but then linking is done with `x86_64-w64-mingw32-g++`.

I'm guessing this has been broken since #24131
(01d1845a80), but have not checked.

Fixes #29734.
Unblocks #29527 (now DEBUG=1 builds can be tested).

Github-Pull: #29747
Rebased-From: b7e7e727ab
2024-05-23 17:05:57 +01:00
nanlour
bb46b90b2e Fix #29767, set m_synced = true after Commit()
Github-Pull: #29776
Rebased-From: bbe82c116e
2024-05-23 17:05:22 +01:00
Jameson Lopp
bf5b6fc8a7 Throw error if invalid parameters passed to getnetworkhashps RPC endpoint
Github-Pull: #28554
Rebased-From: 9ac114e5cd
2024-04-17 13:47:50 +01:00
dergoegge
a81a9228fb [rpc, bugfix] Enforce maximum value for setmocktime
Github-Pull: #29869
Rebased-From: c2e0489b71
2024-04-17 13:28:13 +01:00
Luke Dashjr
d39ea519c7 Change Luke Dashjr seed to dashjr-list-of-p2p-nodes.us
To avoid issues with DNS blacklisting, I've setup a separate domain for my DNS seed.

Github-Pull: #29691
Rebased-From: 4f273ab436
2024-04-17 13:28:13 +01:00
glozow
c21bbcc898 [doc] archive 26.1 release notes 2024-04-17 13:28:13 +01:00
fanquake
0b4aa31c34 Merge bitcoin/bitcoin#29719: [26.x] final changes for 26.1
cc0553d0d6 [doc] add manual pages for 26.1 (glozow)
785242dd4c [doc] update release notes 26.1 (glozow)
5f06dcf9c9 [build] bump version to 26.1 final (glozow)
b53bf22c72 ci, macos: Use `--break-system-packages` with Homebrew's python (Hennadii Stepanov)
324e562399 ci: Add workaround for Homebrew's python link error (Hennadii Stepanov)

Pull request description:

  Final changes for `v26.1`.

  Bins for rc2 have been available for 10 days and I haven't seen any bug reports or new things to add.
  Includes #29610 backport for the CI, which has no effect on what goes into the release.
  Website PR: https://github.com/bitcoin-core/bitcoincore.org/pull/1009

ACKs for top commit:
  hebasto:
    ACK cc0553d0d6.
  fanquake:
    ACK cc0553d0d6
  stickies-v:
    ACK cc0553d0d6 (modulo CI passing)

Tree-SHA512: d032157c7cdf07a474e40b947f7e51bfc6a8e280e43345522bad67b6ad449d473f29bf03ee845b2e403693c1c81078589d042337c895eceb8a59cb4340432887
2024-03-25 16:51:08 +00:00
glozow
cc0553d0d6 [doc] add manual pages for 26.1 2024-03-25 10:01:44 +00:00
glozow
785242dd4c [doc] update release notes 26.1 2024-03-25 10:01:43 +00:00
glozow
5f06dcf9c9 [build] bump version to 26.1 final 2024-03-25 10:01:13 +00:00
Hennadii Stepanov
b53bf22c72 ci, macos: Use --break-system-packages with Homebrew's python
Homebrew's python@3.12 is marked as externally managed (PEP 668),
necessitating different approaches for installing Python packages.

For more details, please refer to https://github.com/orgs/Homebrew/discussions/3404.

Github-Pull: #29610
Rebased-From: acc06bc91f
2024-03-25 09:59:31 +00:00
Hennadii Stepanov
324e562399 ci: Add workaround for Homebrew's python link error
Promoting Homebrew's python@3.12 to the default python3 breaks symbolic
links on macOS x86_64.

This change adds a workaround for that issue.

Also see: https://github.com/actions/runner-images/issues/9471 etc.

Github-Pull: #29610
Rebased-From: ae5f72027f
2024-03-25 09:59:07 +00:00
glozow
c33e83a53a Merge bitcoin/bitcoin#29509: [26.x] backports and final changes for v26.1rc2
c68d4d0ac5 [doc] update manual pages for v26.1rc2 (glozow)
bd715bfb30 [build] bump version to v26.1rc2 (glozow)
b6d006d2a2 update release notes 26.1 (glozow)
fce992b38e fuzz: restrict fopencookie usage to Linux & FreeBSD (fanquake)
40c56a4d13 test: make sure keypool sizes do not change on `getrawchangeaddress`/`getnewaddress` failures (UdjinM6)
7c82b2758c wallet: Avoid updating `ReserveDestination::nIndex` when `GetReservedDestination` fails (UdjinM6)
b5419ce6b6 p2p: Don't consider blocks mutated if they don't connect to known prev block (Greg Sanders)
0535c253fe [test] IsBlockMutated unit tests (dergoegge)
8141498f3a [validation] Cache merkle root and witness commitment checks (dergoegge)
0c5c5962cb [test] Add regression test for #27608 (dergoegge)
24736350e9 [net processing] Don't process mutated blocks (dergoegge)
50c0b61a9d [validation] Merkle root malleation should be caught by IsBlockMutated (dergoegge)
aff368fa81 [validation] Introduce IsBlockMutated (dergoegge)
076c67c3aa [refactor] Cleanup merkle root checks (dergoegge)
97a1d0a459 [validation] Isolate merkle root checks (dergoegge)
4ac0eb543d test: Drop `x` modifier in `fsbridge::fopen` call for mingw builds (Hennadii Stepanov)

Pull request description:

  Includes:
  - #29357
  - #29412
  - #29524
  - #29510
  - #29529

  Also does:
  - update to release notes
  - bump to rc2
  - manpages
  - (no changes to bitcoin.conf)

ACKs for top commit:
  achow101:
    ACK c68d4d0ac5

Tree-SHA512: 2f8c3dd705e3f9f33403b3cc17e8006510ff827d7dbd609b09732a1669964e9b001cfecdc63d8d8daeb8f39c652e1e4ad0aac873d44d259c21803de85688ed2b
2024-03-11 09:54:18 +00:00
glozow
c68d4d0ac5 [doc] update manual pages for v26.1rc2 2024-03-07 10:51:31 +00:00
glozow
bd715bfb30 [build] bump version to v26.1rc2 2024-03-07 10:47:19 +00:00
glozow
b6d006d2a2 update release notes 26.1 2024-03-07 10:45:40 +00:00
fanquake
fce992b38e fuzz: restrict fopencookie usage to Linux & FreeBSD
Should fix the GCC compilation portion of #29517:
https://github.com/bitcoin/bitcoin/issues/29517#issuecomment-1973573314.

See also:
https://www.gnu.org/software/gnulib/manual/html_node/fopencookie.html
but note that FreeBSD has supported this function since 11.x.

Github-Pull: #29529
Rebased-From: 312f3381a2
2024-03-07 10:45:14 +00:00
UdjinM6
40c56a4d13 test: make sure keypool sizes do not change on getrawchangeaddress/getnewaddress failures
Github-Pull: #29510
Rebased-From: e073f1dfda
2024-03-05 10:50:03 +00:00
UdjinM6
7c82b2758c wallet: Avoid updating ReserveDestination::nIndex when GetReservedDestination fails
Github-Pull: #29510
Rebased-From: 367bb7a80c
2024-03-05 10:49:41 +00:00
Greg Sanders
b5419ce6b6 p2p: Don't consider blocks mutated if they don't connect to known prev block
Github-Pull: #29524
Rebased-From: a1fbde0ef7
2024-03-05 10:47:50 +00:00
dergoegge
0535c253fe [test] IsBlockMutated unit tests
Github-Pull: #29412
Rebased-From: d8087adc7e
2024-03-05 10:46:15 +00:00
dergoegge
8141498f3a [validation] Cache merkle root and witness commitment checks
Slight performance improvement by avoiding duplicate work.

Github-Pull: #29412
Rebased-From: 1ec6bbeb8d
2024-03-05 10:24:35 +00:00
dergoegge
0c5c5962cb [test] Add regression test for #27608
Github-Pull: #29412
Rebased-From: 5bf4f5ba32
2024-03-05 10:24:27 +00:00
dergoegge
24736350e9 [net processing] Don't process mutated blocks
We preemptively perform a block mutation check before further processing
a block message (similar to early sanity checks on other messsage
types). The main reasons for this change are as follows:

- `CBlock::GetHash()` is a foot-gun without a prior mutation check, as
  the hash returned only commits to the header but not to the actual
  transactions (`CBlock::vtx`) contained in the block.
- We have observed attacks that abused mutated blocks in the past, which
  could have been prevented by simply not processing mutated blocks
  (e.g. https://github.com/bitcoin/bitcoin/pull/27608).

Github-Pull: #29412
Rebased-From: 49257c0304
2024-03-05 10:24:17 +00:00
dergoegge
50c0b61a9d [validation] Merkle root malleation should be caught by IsBlockMutated
Github-Pull: #29412
Rebased-From: 2d8495e080
2024-03-05 10:24:16 +00:00
dergoegge
aff368fa81 [validation] Introduce IsBlockMutated
Github-Pull: #29412
Rebased-From: 66abce1d98
2024-03-05 10:21:16 +00:00
dergoegge
076c67c3aa [refactor] Cleanup merkle root checks
Github-Pull: #29412
Rebased-From: e7669e1343
2024-03-01 16:28:19 +00:00
dergoegge
97a1d0a459 [validation] Isolate merkle root checks
Github-Pull: #29412
Rebased-From: 95bddb930a
2024-03-01 16:27:43 +00:00
Hennadii Stepanov
4ac0eb543d test: Drop x modifier in fsbridge::fopen call for mingw builds
The MinGW-w64 toolchain links executables to the old msvcrt C Runtime
Library that does not support the `x` modifier for the _wfopen()
function.

Github-Pull: #29357
Rebased-From: d2fe90571e
2024-02-28 22:21:25 +00:00
glozow
a718bfafe7 Merge bitcoin/bitcoin#29503: 26.x: backport #28784 ("rpc: keep .cookie if it was not generated")
1e956439eb rpc: keep .cookie if it was not generated (Roman Zeyde)

Pull request description:

  v26 introduced a regression in that starting a `bitcoind` twice may have the second instance delete the cookie file of the first, making it impossible to communicate with it.

  Not a big deal but it's annoying, only an issue for 26.0, and the patch is trivial.

ACKs for top commit:
  glozow:
    lgtm ACK 1e956439eb

Tree-SHA512: 0e4b18aebaaf284944f1709b238c8c0acce5e8997409e0c278a5a30ac221ac1ff1d3ad31fbf2ac15b03bf7582891e07a7a2cf00f13cb596aa9512566b9320c23
2024-02-28 11:00:19 -05:00
Roman Zeyde
1e956439eb rpc: keep .cookie if it was not generated
Otherwise, starting bitcoind twice may cause the `.cookie`
file generated by the first instance to be deleted by the
second instance shutdown (after failing to obtain a lock).

Github-Pull: bitcoin/bitcoin#28784
Rebased-From: 7cb9367157
2024-02-28 11:19:27 +01:00
fanquake
a4690485d1 Merge bitcoin/bitcoin#29440: [26.1] final changes for 26.1rc1
1e7fb270d3 [doc] add release notes for 26.1rc1 (glozow)
0ffeaf237a [doc] update manual pages for 26.1rc1 (glozow)
d4aeb4a5a0 [build] bump version to 26.1rc1 (glozow)
c27793891a qt: 26.1rc1 translations update (Hennadii Stepanov)
40e7693fa1 ci: Switch native macOS CI job to Xcode 15.0 (Hennadii Stepanov)
72eab1ca64 build: Fix `-Xclang -internal-isystem` option (Hennadii Stepanov)

Pull request description:

  Final changes to tag a 26.1rc1.
  Bumps version numbers, man pages, adds release notes etc.

ACKs for top commit:
  hebasto:
    re-ACK 1e7fb270d3
  fanquake:
    ACK 1e7fb270d3

Tree-SHA512: 0f4146590d61f4f9322f3147f672ff2af48fc1d85c708b959adcbdb50b8f280570cb732d9f4a424c014046d07e9aa6e408fc3a0337dbaf8cbaea5283697da35a
2024-02-19 15:31:06 +00:00
glozow
1e7fb270d3 [doc] add release notes for 26.1rc1 2024-02-16 17:30:26 +00:00
glozow
0ffeaf237a [doc] update manual pages for 26.1rc1 2024-02-16 17:29:46 +00:00
glozow
d4aeb4a5a0 [build] bump version to 26.1rc1 2024-02-16 17:29:46 +00:00
Hennadii Stepanov
c27793891a qt: 26.1rc1 translations update
This change is a result if pulling the recent translations
from the Transifex website using the
bitcoin-maintainer-tools/update-translations.py tool.

A few manual adjustments were made:
- skipped removing of `bitcoin_af.ts`
- skipped removing of `bitcoin_ar.ts`
- skipped adding of `bitcoin_ru_RU.ts` (`bitcoin_ru.ts` is already
present)
2024-02-16 17:29:46 +00:00
Hennadii Stepanov
40e7693fa1 ci: Switch native macOS CI job to Xcode 15.0
Github-Pull: #29195
Rebased-From: d742be3d3f
2024-02-16 17:29:31 +00:00
Hennadii Stepanov
72eab1ca64 build: Fix -Xclang -internal-isystem option
LLVM Clang >=16.0 and Apple Clang >=15.0 do not recognize
`-Xclang -internal-isystem/usr/local/include` anymore.

For example, see: cbbe1d4454

Github-Pull: #29195
Rebased-From: 8decc5c726
2024-02-16 17:28:33 +00:00
fanquake
74df372750 Merge bitcoin/bitcoin#29209: [26.x] more backports
11f3a7e6ba Use hardened runtime on macOS release builds. (Mark Friedenbach)
ac1b9a51db [test] import descriptor wallet with reorged parent + IsFromMe child in mempool (glozow)
ecb8ebc660 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool (Gloria Zhao)
438ac2947d snapshots: don't core dump when running -checkblockindex after `loadtxoutset` (Mark Friedenbach)
7ec34554af [log] mempool loading (glozow)
fe0f8fe8aa net: create I2P sessions with both ECIES-X25519 and ElGamal encryption (Jon Atack)
fc62271015 [refactor] Add helper for iterating through mempool entries (stickies-v)

Pull request description:

  Backports for 26.x. Includes:
  - 453b481 from #28391
    - #29179
  - #29200
  - #29227
  - #28791
  - #29127

ACKs for top commit:
  stickies-v:
    ACK 11f3a7e6ba

Tree-SHA512: 20ef871ec768f2328056d83f958e595b36ae9b8baa8a6e8b0e1f02c3df4b16965a8e05dcb4323afbcc9ecc4bdde10931232512022c39ee7e12baf9795bf25bf1
2024-02-16 13:37:21 +00:00
Mark Friedenbach
11f3a7e6ba Use hardened runtime on macOS release builds.
The Apple notary service requires submitted app bundles to be configured to use the hardened runtime libraries.  This is configured at signing time, and supported by the signapple tool Bitcoin Core uses for reproduceable signed binaries.  We simply need to pass "--hardened-runtime" when the signature is created.  Once attached to the bundle, the resulting codesigned binary can be successfully submitted to the Apple binary notarization service by any Apple Developer.

Github-Pull: #29127
Rebased-From: 4fdd836db9
2024-01-19 16:14:36 +00:00
glozow
ac1b9a51db [test] import descriptor wallet with reorged parent + IsFromMe child in mempool
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>

Github-Pull: #29179
Rebased-From: df30247705
2024-01-19 16:13:22 +00:00
Gloria Zhao
ecb8ebc660 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.

Github-Pull: #29179
Rebased-From: c3d02be536
2024-01-19 16:13:14 +00:00
Mark Friedenbach
438ac2947d snapshots: don't core dump when running -checkblockindex after loadtxoutset
Github-Pull: #28791
Rebased-From: cdc6ac4126
2024-01-19 16:13:05 +00:00
glozow
7ec34554af [log] mempool loading
Log at the top before incrementing so that this log isn't printed when
there's only 1 tx.

Github-Pull: #29227
Rebased-From: eb78ea4eeb
2024-01-19 16:12:57 +00:00
Jon Atack
fe0f8fe8aa net: create I2P sessions with both ECIES-X25519 and ElGamal encryption
A Bitcoin Core node may only connect to a peer destination via I2P if both sides
have sessions with the same encryption type.  The encryption type is a property
of the session, not the destination.  Sessions may support multiple encryption
types.

As Bitcoin Core is not currently setting the I2P encryption type when creating
sessions, it is using the older default, ElGamal (type 0).

This pull updates Bitcoin Core to use both ECIES-X25519 and ElGamal (types 4 and
0, respectively).  This allows to connect to I2P peers with either type, and the
newer, faster ECIES-X25519 will be preferred.

See also the recently updated section "Signature and Encryption Types" in
https://geti2p.net/en/docs/api/samv3

Thanks and credit to zzzi2p (https://github.com/zzzi2p) for reporting.

Closes https://github.com/bitcoin/bitcoin/issues/29197.

Github-Pull: #29200
Rebased-From: 9d728916b2
2024-01-19 16:12:47 +00:00
stickies-v
fc62271015 [refactor] Add helper for iterating through mempool entries
Instead of reaching into the mapTx data structure, use a helper method
that provides the required vector of CTxMemPoolEntry pointers.

Github-Pull: #28391
Rebased-From: 453b4813eb
2024-01-19 16:12:23 +00:00
glozow
04edf9f586 Merge bitcoin/bitcoin#29011: [26.x] Backports
7b79e54474 doc: update release notes for 26.x (fanquake)
ccf00b1e6e wallet: Fix use-after-free in WalletBatch::EraseRecords (MarcoFalke)
40252e184e ci: Set `HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK` to avoid failures (Hennadii Stepanov)
b06b14e68d rpc: getwalletinfo, return wallet 'birthtime' (furszy)
12834012c2 test: coverage for wallet birth time interaction with -reindex (furszy)
0fa47e2569 wallet: fix legacy spkm default birth time (furszy)
84f4a6c145 wallet: birth time update during tx scanning (furszy)
074296dd60 refactor: rename FirstKeyTimeChanged to MaybeUpdateBirthTime (furszy)
35039ac3cc fuzz: disable BnB when SFFO is enabled (furszy)
903b4623d3 test: add coverage for BnB-SFFO restriction (furszy)
05d0576d3c wallet: create tx, log resulting coin selection info (furszy)
5493ebbe74 wallet: skip BnB when SFFO is active (Murch)
b15e2e2cec test: add regression test for the getrawtransaction segfault (Martin Zumsande)
5097bb3389 rpc: fix getrawtransaction segfault (Martin Zumsande)
81e744a9a6 ci: Use Ubuntu 24.04 Noble for asan (MarcoFalke)
69e53d1e47 ci: Use Ubuntu 24.04 Noble for tsan,tidy,fuzz (MarcoFalke)
d2c80b6f52 doc: Missing additions to 26.0 release notes (fanquake)
8dc2c753ff doc: add historical release notes for 26.0 (fanquake)

Pull request description:

  Backports for `26.x`. Currently:
  * https://github.com/bitcoin/bitcoin/pull/28920
  * https://github.com/bitcoin/bitcoin/pull/28992
  * https://github.com/bitcoin/bitcoin/pull/28994
  * https://github.com/bitcoin/bitcoin/pull/29003
  * https://github.com/bitcoin/bitcoin/pull/29023
  * https://github.com/bitcoin/bitcoin/pull/29080
  * https://github.com/bitcoin/bitcoin/pull/29176

ACKs for top commit:
  TheCharlatan:
    ACK 7b79e54474
  glozow:
    ACK 7b79e54474, matches mine

Tree-SHA512: 898aec76ed3ad35e0edd0980af5bcc21bd60003bbf69e0b4f473ed2aa38c4e3b360b930bc3747cf798195906a8f9fe66417524f5e5ef40fa68f1c1aaceebdeb0
2024-01-09 15:39:19 +00:00
fanquake
7b79e54474 doc: update release notes for 26.x 2024-01-04 16:21:37 +00:00
MarcoFalke
ccf00b1e6e wallet: Fix use-after-free in WalletBatch::EraseRecords
Github-Pull: #29176
Rebased-From: faebf1df2a
2024-01-04 16:21:37 +00:00
Hennadii Stepanov
40252e184e ci: Set HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK to avoid failures
Homebrew attempts to check for outdated dependents or those with broken
linkage. Such behavior might lead to failures when Homebrew updates them
on old macOS images.

This change prevents such behavior.

Github-Pull: #29080
Rebased-From: 43c3246af7
2024-01-04 16:21:37 +00:00
furszy
b06b14e68d rpc: getwalletinfo, return wallet 'birthtime'
And add coverage for it

Github-Pull: #28920
Rebased-From: 1ce45baed7
2024-01-04 16:21:37 +00:00
furszy
12834012c2 test: coverage for wallet birth time interaction with -reindex
Verifying the wallet updates the birth time accordingly when it
detects a transaction with a time older than the oldest descriptor
timestamp.
This could happen when the user blindly imports a descriptor with
'timestamp=now'.

Github-Pull: #28920
Rebased-From: 83c66444d0
2024-01-04 16:21:37 +00:00
furszy
0fa47e2569 wallet: fix legacy spkm default birth time
To avoid scanning blocks, as assumed by a wallet with no
generated keys or imported scripts, the default value for
the birth time needs to be set to the maximum int64_t value.

Once the first key is generated or the first script is imported,
the legacy SPKM will update the birth time automatically.

Github-Pull: #28920
Rebased-From: 6f497377aa
2024-01-04 16:21:37 +00:00
furszy
84f4a6c145 wallet: birth time update during tx scanning
As the user could have imported a descriptor with
a newer timestamp (by blindly setting 'timestamp=now'),
the wallet needs to update the birth time when it detects
a transaction older than the oldest descriptor timestamp.

Github-Pull: #28920
Rebased-From: 75fbf444c1
2024-01-04 16:21:37 +00:00
furszy
074296dd60 refactor: rename FirstKeyTimeChanged to MaybeUpdateBirthTime
In the following-up commit, the wallet birth time will also
be modified by the transactions scanning process. When a tx
older than all descriptor's timestamp is detected.

Github-Pull: #28920
Rebased-From: b4306e3c8d
2024-01-04 16:21:37 +00:00
furszy
35039ac3cc fuzz: disable BnB when SFFO is enabled
Github-Pull: #28994
Rebased-From: 576bee88fd
2024-01-04 16:21:36 +00:00
furszy
903b4623d3 test: add coverage for BnB-SFFO restriction
Verify the transaction creation process does not produce
a BnB solution when SFFO is enabled.
This is currently problematic because it could require a
change output. And BnB is specialized on changeless solutions.

Co-authored-by: Andrew Chow <achow101@gmail.com>
Co-authored-by: Murch <murch@murch.one>

Github-Pull: #28994
Rebased-From: 05e5ff194c
2024-01-04 16:21:36 +00:00
furszy
05d0576d3c wallet: create tx, log resulting coin selection info
Useful for understanding what is going on internally
when the software is running. Debug issues, and provide
more accurate feedback to users.

Github-Pull: #28994
Rebased-From: 0c5755761c
2024-01-04 16:21:36 +00:00
Murch
5493ebbe74 wallet: skip BnB when SFFO is active
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>

Github-Pull: #28994
Rebased-From: 5cea25ba79
2024-01-04 16:21:36 +00:00
Martin Zumsande
b15e2e2cec test: add regression test for the getrawtransaction segfault
This fails on master without the previous commit.

Github-Pull: #29003
Rebased-From: 9075a44646
2024-01-04 16:21:36 +00:00
Martin Zumsande
5097bb3389 rpc: fix getrawtransaction segfault
The crash would happen when querying a mempool transaction with verbosity=2, while pruning.

Github-Pull: #29003
Rebased-From: 494a926d05
2024-01-04 16:21:36 +00:00
MarcoFalke
81e744a9a6 ci: Use Ubuntu 24.04 Noble for asan
Github-Pull: #28992
Rebased-From: fad2392c58
2024-01-04 16:21:36 +00:00
MarcoFalke
69e53d1e47 ci: Use Ubuntu 24.04 Noble for tsan,tidy,fuzz
Github-Pull: #28992
Rebased-From: fa83b65ef8
2024-01-04 16:21:22 +00:00
fanquake
d2c80b6f52 doc: Missing additions to 26.0 release notes
Github-Pull: #29023
Rebased-From: ca5937553b
2023-12-08 09:18:28 +00:00
fanquake
8dc2c753ff doc: add historical release notes for 26.0
Github-Pull: #29023
Rebased-From: 7d4e47d184
2023-12-08 09:18:24 +00:00
fanquake
44d8b13c81 Merge bitcoin/bitcoin#28959: [26.0] Finalize or rc4
b1d350c78b doc: update release notes for 26.0 (fanquake)
b0546bc907 doc: update manual pages for 26.0 (fanquake)
9ce1766d20 build: bump version to v26.0 final (fanquake)

Pull request description:

  Final changes for 26.0. Assuming no further backports. rc3 was done in #28872.

ACKs for top commit:
  achow101:
    ACK b1d350c78b
  hebasto:
    ACK b1d350c78b, I have reviewed the code and it looks OK.

Tree-SHA512: 8b1bfa9e9d6c5ccf8305335eba503c02a76043b2752e2302da84cb574078889ddb761b9efd14ef97f68bbae154b00ac54f531e2e33eba6baf8d703aa98ef5175
2023-12-04 10:45:21 +00:00
fanquake
b1d350c78b doc: update release notes for 26.0 2023-11-28 16:13:36 +00:00
fanquake
b0546bc907 doc: update manual pages for 26.0 2023-11-28 16:12:43 +00:00
fanquake
9ce1766d20 build: bump version to v26.0 final 2023-11-28 16:09:47 +00:00
fanquake
e4fef4ae65 Merge bitcoin/bitcoin#28872: [26.x] Changes for rc3
2f86d30533 doc: update release notes for v26.0rc3 (fanquake)
3b6c7f240c doc: update manual pages for v26.0rc3 (fanquake)
3db4d1cff2 build: bump version to v26.0rc3 (fanquake)
6045f38dc8 build: Fix regression in "ARMv8 CRC32 intrinsics" test (Hennadii Stepanov)
5eaa179f27 ci: Avoid toolset ambiguity that MSVC can't handle (Hennadii Stepanov)
55af112565 p2p: do not make automatic outbound connections to addnode peers (Jon Atack)
5e0bcc1977 ci: Switch from `apt` to `apt-get` (Hennadii Stepanov)
437a5316e5 ci: Update apt cache (Hennadii Stepanov)
1488648104 pool: change memusage_test to use int64_t, add allocation check (Martin Leitner-Ankerl)
bcc183ccce pool: make sure PoolAllocator uses the correct alignment (Martin Leitner-Ankerl)
7dda4991a8 doc: regenerate example bitcoin.conf (fanquake)
5845331a6c doc: rewrite explanation for -par= (fanquake)

Pull request description:

  Currently backports:
  * https://github.com/bitcoin/bitcoin/pull/28858
  * https://github.com/bitcoin/bitcoin/pull/28895 (partial)
  * https://github.com/bitcoin/bitcoin/pull/28913
  * https://github.com/bitcoin/bitcoin/pull/28905
  * https://github.com/bitcoin/bitcoin/pull/28919
  * https://github.com/bitcoin/bitcoin/pull/28925

  Also includes changes for rc3, and reintegrating the release-notes.

ACKs for top commit:
  hebasto:
    re-ACK 2f86d30533, only https://github.com/bitcoin/bitcoin/pull/28919 backported since my [recent](https://github.com/bitcoin/bitcoin/pull/28872#pullrequestreview-1744950215) review.
  TheCharlatan:
    ACK 2f86d30533

Tree-SHA512: 43c91b344d37f582081ac184ac59cf76c741317b2b69a24fcd4287eefa8333e20c545e150798f4057d6f4ac8e70ed9cba1c8dd9777b11c1cf8992cce09108727
2023-11-22 18:25:34 +00:00
fanquake
2f86d30533 doc: update release notes for v26.0rc3
Few further changes are expected, so reintegrate the release-notes.
2023-11-22 17:22:06 +00:00
fanquake
3b6c7f240c doc: update manual pages for v26.0rc3 2023-11-22 17:22:06 +00:00
fanquake
3db4d1cff2 build: bump version to v26.0rc3 2023-11-22 17:22:06 +00:00
Hennadii Stepanov
6045f38dc8 build: Fix regression in "ARMv8 CRC32 intrinsics" test
The `vmull_p64` is a part of the Crypto extensions from the ACLE. They
are optional extensions, so they get enabled with a `+crypto` for
architecture flags.

Github-Pull: #28919
Rebased-From: 228d6a2969
2023-11-22 17:22:05 +00:00
Hennadii Stepanov
5eaa179f27 ci: Avoid toolset ambiguity that MSVC can't handle
This change is required to work with the new windows-2022 image version
20231115 properly.

Github-Pull: #28905
Rebased-From: 91d5bd8ac9
2023-11-22 12:01:34 +00:00
Jon Atack
55af112565 p2p: do not make automatic outbound connections to addnode peers
to allocate our limited outbound slots correctly, and to ensure addnode
connections benefit from their intended protections.

Our addnode logic usually connects the addnode peers before the automatic
outbound logic does, but not always, as a connection race can occur.  If an
addnode peer disconnects us and if it was the only one from its network, there
can be a race between reconnecting to it with the addnode thread, and it being
picked as automatic network-specific outbound peer.  Or our internet connection
or router, or the addnode peer, could be temporarily offline, and then return
online during the automatic outbound thread.  Or we could add a new manual peer
using the addnode RPC at that time.

The race can be more apparent when our node doesn't know many peers, or with
networks like cjdns that currently have few bitcoin peers.

When an addnode peer is connected as an automatic outbound peer and is the only
connection we have to a network, it can be protected by our new outbound
eviction logic and persist in the "wrong role".

Examples on mainnet using logging added in the same pull request:

2023-08-12T14:51:05.681743Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic network-specific outbound-full-relay connection
to i2p peer selected for manual (addnode) connection: [geh...odq.b32.i2p]:0

2023-08-13T03:59:28.050853Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic block-relay-only connection to onion peer
selected for manual (addnode) connection: kpg...aid.onion:8333

2023-08-13T16:21:26.979052Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic network-specific outbound-full-relay connection
to cjdns peer selected for manual (addnode) connection: [fcc...8ce]:8333

2023-08-14T20:43:53.401271Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic network-specific outbound-full-relay connection
to cjdns peer selected for manual (addnode) connection: [fc7...59e]:8333

2023-08-15T00:10:01.894147Z [opencon] [net.cpp:1949] [ThreadOpenConnections]
[net:debug] Not making automatic feeler connection to i2p peer selected for
manual (addnode) connection: geh...odq.b32.i2p:8333

Finally, there does not seem to be a reason to make block-relay or short-lived
feeler connections to addnode peers, as the addnode logic will ensure we connect
to them if they are up, within the addnode connection limit.

Fix these issues by checking if the address is an addnode peer in our automatic
outbound connection logic.

Github-Pull: #28895
Rebased-From: cc62716920
2023-11-22 11:53:27 +00:00
Hennadii Stepanov
5e0bcc1977 ci: Switch from apt to apt-get
Github-Pull: #28925
Rebased-From: 710da28c72
2023-11-22 11:36:55 +00:00
Hennadii Stepanov
437a5316e5 ci: Update apt cache
Github-Pull: #28925
Rebased-From: a6cc059ea5
2023-11-22 11:36:45 +00:00
Martin Leitner-Ankerl
1488648104 pool: change memusage_test to use int64_t, add allocation check
If alignment of the PoolAllocator would be insufficient, then the test would fail. This also catches the issue with ARM 32bit,
where int64_t is aligned to 8 bytes but void* is aligned to 4 bytes. The test adds a check to ensure the pool has allocated
a minimum number of chunks

Github-Pull: #28913
Rebased-From: d5b4c0b69e
2023-11-22 11:33:12 +00:00
Martin Leitner-Ankerl
bcc183ccce pool: make sure PoolAllocator uses the correct alignment
This changes the PoolAllocator to default the alignment to the given type. This makes the code simpler, and most importantly
fixes a bug on ARM 32bit that caused OOM: The class CTxOut has a member CAmount which is an int64_t and on ARM 32bit int64_t
are 8 byte aligned which is larger than the pointer alignment of 4 bytes. So for CCoinsMap to be able to use the pool, we
need to use the alignment of the member instead of just alignof(void*).

Github-Pull: #28913
Rebased-From: ce881bf9fc
2023-11-22 11:33:01 +00:00
fanquake
7dda4991a8 doc: regenerate example bitcoin.conf 2023-11-22 11:21:11 +00:00
fanquake
5845331a6c doc: rewrite explanation for -par=
The negative bound for script threads comes from the machine which
generates the man pages, so may only be correct for that machine. Any
other placeholder value will also be wrong for some machines. Fix this
be removing the value. This also fixes help2man incorrectly bolding the
value, as if it were a paramater.

Closes #28850.

Github-Pull: #28858
Rebased-From: d799ea26ed
2023-11-14 15:46:10 +00:00
fanquake
67b2512560 Merge bitcoin/bitcoin#28754: [26.x] Backports for rc2
e4e84790f6 doc: update manual pages for v26.0rc2 (fanquake)
0b189a9092 build: bump version to v26.0rc2 (fanquake)
e097d4cb53 gui: fix crash on selecting "Mask values" in transaction view (Sebastian Falbesoner)
05e8874554 guix: update signapple (fanquake)
deccc50631 guix: Zip needs to include all files with time as SOURCE_DATE_EPOCH (Andrew Chow)
fe57abd7e9 test: add coverage for snapshot chainstate not matching AssumeUTXO parameters (pablomartin4btc)
b761a58171 assumeutxo, blockstorage: prevent core dump on invalid hash (pablomartin4btc)
d3ebf6e9fc [test] Test i2p private key constraints (Vasil Dimov)
1f11784aac [net] Check i2p private key constraints (dergoegge)
6544ffa01f bugfix: Mark CNoDestination and PubKeyDestination constructor explicit (MarcoFalke)

Pull request description:

  Backports for v26.0rc2:
  * #28695
  * #28698
  * #28728
  * #28757
  * #28759
  * https://github.com/bitcoin-core/gui/pull/774

ACKs for top commit:
  josibake:
    ACK e4e84790f6
  hebasto:
    re-ACK e4e84790f6, only a backport of https://github.com/bitcoin-core/gui/pull/774 added since my [recent](https://github.com/bitcoin/bitcoin/pull/28754#pullrequestreview-1707143194) review.
  TheCharlatan:
    Re-ACK e4e84790f6

Tree-SHA512: 4b95afd26b8bf91250cb883423de8b274cefa48dc474734f5900aeb756eee3a6c656116efcfa2caff3c250678c16b70cc6b7a5d840018dc7e2c1e8161622cd61
2023-11-01 10:44:55 +00:00
fanquake
e4e84790f6 doc: update manual pages for v26.0rc2 2023-11-01 10:01:05 +00:00
fanquake
0b189a9092 build: bump version to v26.0rc2 2023-11-01 10:01:05 +00:00
Sebastian Falbesoner
e097d4cb53 gui: fix crash on selecting "Mask values" in transaction view
This commits fixes a crash bug that can be caused with the following steps:
- change to the "Transactions" view
- right-click on an arbitrary transaction -> "Show transaction details"
- close the transaction detail window again
- select "Settings" -> "Mask values"

The problem is that the list of opened dialogs, tracked in the member
variable `m_opened_dialogs`, is only ever appended with newly opened
transaction detail dialog pointers, but never removed. This leads to
dangling pointers in the list, and if the "Mask values" menu item is
selected, a crash is caused in the course of trying to close the opened
transaction detail dialogs (see `closeOpenedDialogs()` method). Fix this
by removing the pointer from the list if the corresponding widget is
destroyed.

Github-Pull: https://github.com/bitcoin-core/gui/pull/774
Rebased-From: e26e665f9f
2023-11-01 10:01:05 +00:00
fanquake
05e8874554 guix: update signapple
Fixes #28449

Github-Pull: #28759
Rebased-From: 79539fbfbf
2023-10-31 17:10:20 +00:00
Andrew Chow
deccc50631 guix: Zip needs to include all files with time as SOURCE_DATE_EPOCH
The zip for codesigned MacOS distribution needs to have all files have
the same timestamp. These files also need to be included in the zip as
zip is not automatically recursive. We use the same pattern for zip as
is done for the other zip files produced by guix.

Github-Pull: #28757
Rebased-From: f6f18eeaa8
2023-10-31 17:07:52 +00:00
pablomartin4btc
fe57abd7e9 test: add coverage for snapshot chainstate not matching AssumeUTXO parameters
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>

Github-Pull: #28698
Reabsed-From: 811067ca1c
2023-10-31 17:07:52 +00:00
pablomartin4btc
b761a58171 assumeutxo, blockstorage: prevent core dump on invalid hash
Github-Pull: #28698
Rebased-from: 4a5be10b92
2023-10-31 17:07:52 +00:00
Vasil Dimov
d3ebf6e9fc [test] Test i2p private key constraints
Github-Pull: #28695
Rebased-From: 5cf4d266d9
2023-10-31 17:07:52 +00:00
dergoegge
1f11784aac [net] Check i2p private key constraints
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>

GitHub-Pull: #28695
Rebased-From: cf70a8d565
2023-10-31 17:07:52 +00:00
MarcoFalke
6544ffa01f bugfix: Mark CNoDestination and PubKeyDestination constructor explicit
This should fix the bug reported in
https://github.com/bitcoin/bitcoin/pull/28246#discussion_r1371640502,
which caused the GUI to not detect the destination type of recipients,
thus picking the wrong change destination type.

Also, add missing lifetimebound attribute to a getter method.

GitHub-Pull: #28728
Rebased-From: 1111475b41
2023-10-31 17:07:52 +00:00
fanquake
7d0e5b099c Merge bitcoin/bitcoin#28763: [26.x] qt: 26.0rc2 translations update
1695c4801c qt: 26.0rc2 translations update (Hennadii Stepanov)

Pull request description:

  This PR pulls the recent translations from the [Transifex.com](https://www.transifex.com/bitcoin/bitcoin) using the [`bitcoin-maintainer-tools/update-translations.py`](https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/main/update-translations.py) tool.

  According to our [Release Process docs](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-every-release-candidate), it is supposed to be merged before `v26.0rc2` tagging.

ACKs for top commit:
  stickies-v:
    ACK 1695c4801c

Tree-SHA512: 778b3ebdcd7567b2ed4bac63ae4abcb25562c502266e597845c436fce7db8829fdb2902d087051f5fac76785d516374d54883045f5025d728fe001476dc73b55
2023-10-31 17:04:20 +00:00
Hennadii Stepanov
1695c4801c qt: 26.0rc2 translations update 2023-10-31 16:53:32 +00:00
fanquake
11254a34d7 Merge bitcoin/bitcoin#28717: [26.x] Bump version to v26.0rc1
958ee5db63 doc: generate example bitcoin.conf for v26.0rc1 (fanquake)
d1fe90c423 doc: generate manual pages for v26.0rc1 (fanquake)
c4fa45c2c3 build: bump version to v26.0rc1 (fanquake)

Pull request description:

  Bump the version number.
  Generate the man pages.
  Generate example bitcoin.conf.
  Release-notes.md already points to the wiki: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/26.0%E2%80%90Release%E2%80%90Notes%E2%80%90Draft

ACKs for top commit:
  josibake:
    ACK 958ee5db63 🚀
  hebasto:
    ACK 958ee5db63

Tree-SHA512: eb13253dd5dbae2723a1a89a42d832c9c817cd00ecb51414313f15c8a988504dd1940babb42bf65cc97a4e37eb861a1ea6df19e6c763eebe4c8734c1cdf99e07
2023-10-24 16:38:39 +01:00
fanquake
958ee5db63 doc: generate example bitcoin.conf for v26.0rc1 2023-10-24 12:00:42 +01:00
fanquake
d1fe90c423 doc: generate manual pages for v26.0rc1 2023-10-24 12:00:42 +01:00
fanquake
c4fa45c2c3 build: bump version to v26.0rc1 2023-10-24 12:00:41 +01:00
fanquake
f40d7fc180 Merge bitcoin/bitcoin#28715: [26.x] qt: 26.0rc1 translations update
74604662f3 qt: 26.0rc1 translations update (Hennadii Stepanov)

Pull request description:

  This PR pulls the recent translations from the [Transifex.com](https://www.transifex.com/bitcoin/bitcoin) using the [`bitcoin-maintainer-tools/update-translations.py`](https://github.com/bitcoin-core/bitcoin-maintainer-tools/blob/main/update-translations.py) tool.

  According to our [Release Process docs](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#before-every-release-candidate), it is supposed to be merged before `v26.0rc1` tagging.

ACKs for top commit:
  stickies-v:
    ACK 74604662f3

Tree-SHA512: 707a45f7df513352bf5e76d4fb39a74d77ea907921f0b543623e6f723ac0cf65d9bc94c975b3a6e650e4c88a554b13a4701dd9f0009cf4374104cb743ec6c2cc
2023-10-24 11:59:22 +01:00
fanquake
df28880ec0 Merge bitcoin/bitcoin#28713: [26.x] doc: consolidate release note fragments
b226e275b3 doc: consolidate release note fragments (fanquake)

Pull request description:

  Consolidate the 26.x release note fragments to the wiki: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/26.0%E2%80%90Release%E2%80%90Notes%E2%80%90Draft.

ACKs for top commit:
  dergoegge:
    ACK b226e275b3

Tree-SHA512: 189ccb45ed5cc213dd71cce0366a7086dc688422bcfbf28836f212c2a59628cac4f9379883ea23106585ba6ded4cce9995020b787add4ac7437be7cd868d0210
2023-10-24 11:16:28 +01:00
Hennadii Stepanov
74604662f3 qt: 26.0rc1 translations update 2023-10-24 11:01:03 +01:00
fanquake
b226e275b3 doc: consolidate release note fragments 2023-10-24 10:47:42 +01:00
fanquake
96ec3b67a7 Merge bitcoin/bitcoin#28707: doc: add historical release notes for 24.2
3f482ac231 doc: add historical release notes for 24.2 (fanquake)

Pull request description:

  v24.2 has [been tagged](https://github.com/bitcoin/bitcoin/releases/tag/v24.2).

ACKs for top commit:
  stickies-v:
    ACK 3f482ac231

Tree-SHA512: 71773832910ecda7ed34c6545d184ecbc743d9a36aadd8e4bd367ff60ef5b8048d39335b2347878c4a1a076cc691f12f0e36e8db542757c883d7f83d1161625d
2023-10-24 10:33:26 +01:00
fanquake
ab61087a7e Merge bitcoin/bitcoin#28660: test: enable reindex readonly test on *BSD
5a0688a20d test: enable reindex readonly test on *BSD and macOS as root (Matthew Zipkin)

Pull request description:

  see https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1349505585

  OpenBSD and FreeBSD don't have `chattr` but they do have `chflags`, use that method to make the block file immutable for the reindex_readonly test.

  Written and tested on a VPS running FreeBSD:
  ```
  FreeBSD freebsd-13-1 13.2-RELEASE-p4 FreeBSD 13.2-RELEASE-p4 GENERIC amd64
  ```

ACKs for top commit:
  maflcko:
    re-cr-lgtm-ACK 5a0688a20d
  jonatack:
    ACK 5a0688a20d tested on macOS only
  theStack:
    ACK 5a0688a20d

Tree-SHA512: 8c88d282d09c00355d22c4c504b779f60e420327a5e07bcf80fa77b97fefcb04952af9ceaf439d9033a0a2448cb26a02663fe6bddcd4a74792857cfbaf1c5162
2023-10-24 10:32:48 +01:00
Ryan Ofsky
d724bb5291 Merge bitcoin/bitcoin#28609: wallet: Reload watchonly and solvables wallets after migration
4814e4063e test: Check tx metadata is migrated to watchonly (Andrew Chow)
d616d30ea5 wallet: Reload watchonly and solvables wallets after migration (Andrew Chow)
118f2d7d70 wallet: Copy all tx metadata to watchonly wallet (Andrew Chow)
9af87cf348 test: Check that a failed wallet migration is cleaned up (Andrew Chow)

Pull request description:

  Some incomplete/incorrect state as a result of migration can be mitigated/cleaned up by simply restarting the migrated wallets. We already do this for a wallet when it is migrated, but we do not for the new watchonly and solvables wallets that may be created. This PR introduces this behavior, in addition to creating those wallets initially without an attached chain.

  While implementing this, I noticed that not all `CWalletTx` metadata was being copied over to the watchonly wallet and so some data, such as time received, was being lost. This PR fixes this as a side effect of not having a chain attached to the watchonly wallet. A test has also been added.

ACKs for top commit:
  ishaanam:
    light code review ACK 4814e4063e
  ryanofsky:
    Code review ACK 4814e4063e. Just implemented the suggested orderpos, copyfrom, and path set comments since last review
  furszy:
    ACK 4814e406

Tree-SHA512: 0b992430df9f452cb252c2212df8e876613f43564fcd1dc00c6c31fa497adb84dfff6b5ef597590f9b288c5f64cb455f108fcc9b6c9d1fe9eb2c39e7f2c12a89
2023-10-23 17:35:36 -04:00
Andrew Chow
da8e397e4a Merge bitcoin/bitcoin#28685: coinstats, assumeutxo: fix hash_serialized2 calculation
4bfaad4eca chainparams, assumeutxo: Fix signet txoutset hash (Fabian Jahr)
a503cd0f0b chainparams, assumeutxo: Fix testnet txoutset hash (Fabian Jahr)
f6213929c5 assumeutxo: Check deserialized coins for out of range values (Fabian Jahr)
66865446a7 docs: Add release notes for #28685 (Fabian Jahr)
cb0336817e scripted-diff: Rename hash_serialized_2 to hash_serialized_3 (Fabian Jahr)
351370a1d2 coinstats: Fix hash_serialized2 calculation (Fabian Jahr)

Pull request description:

  Closes #28675

  The last commit demonstrates that theStack's analysis [here](https://github.com/bitcoin/bitcoin/issues/28675#issuecomment-1770389468) seems to be correct. There will be more changes needed for the rest of the test suite but the `feature_assumeutxo.py` with my additional tests pass.

ACKs for top commit:
  achow101:
    ACK 4bfaad4eca
  theStack:
    Code-review ACK 4bfaad4eca
  ryanofsky:
    Code review ACK 4bfaad4eca

Tree-SHA512: 2f6abc92b282f7c5da46391803cf0804d13978d191d541f2509b532c538abccd0a081e46cda23d80d47206a05fa2b5d41b7ab246e6a263db7a7461d6292116ef
2023-10-23 15:16:08 -04:00
Ryan Ofsky
5c32c5971c Merge bitcoin/bitcoin#28618: doc: assumeutxo prune and index notes
03f82087f6 doc: assumeutxo prune and index notes (Sjors Provoost)

Pull request description:

  Based on recent comments on #27596.

ACKs for top commit:
  pablomartin4btc:
    re ACK 03f82087f6
  ryanofsky:
    ACK 03f82087f6. Nice changes, these seem like very helpful notes

Tree-SHA512: fe651b49f4d667400a3655899f27a96dd1eaf67cf9215fb35db5f44fb8c0313e7d541518be6791fec93392df24b909793f3886adb808e53228ed2a291165639d
2023-10-23 12:41:15 -04:00
Matthew Zipkin
5a0688a20d test: enable reindex readonly test on *BSD and macOS as root 2023-10-23 10:58:54 -04:00
fanquake
d90eea989a Merge bitcoin/bitcoin#28708: ci: Add missing --external to podman image prune
fa65887377 ci: Add missing --external to podman image prune (MarcoFalke)

Pull request description:

  This should fix the out-of-space issues seen in CI. For example:

  https://cirrus-ci.com/task/6208410429947904?logs=ci#L8613

  ```
  Error: committing container for step {Env:[FILE_ENV=./ci/test/00_setup_env_native_msan.sh PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin] Command:run Args:[bash -c cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh] Flags:[] Attrs:map[json:true] Message:RUN bash -c cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh Original:RUN ["bash", "-c", "cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh"]}: copying layers and metadata for container "8d882455cc157be6a85d9779b45cacf4dd92a37cfb16fad38213f758a830827d": writing blob: adding layer with blob "sha256:371f657e226fef20f4af6fb88a288dd6248c82c2088daca2d53aaacb51b4303a": processing tar file(write /usr/lib/x86_64-linux-gnu/perl/5.34.0/auto/Unicode/Collate/Collate.so: no space left on device): exit status 1

ACKs for top commit:
  stickies-v:
    utACK fa65887377

Tree-SHA512: a949b957654272b9c9450feebb64397542029019f031a18444b5b403aca899c972b3b163be716bf1bfbd5820430c70a6fec008771c6e13d3d8281ed100df575b
2023-10-23 15:38:24 +01:00
Hennadii Stepanov
565c55119b Merge bitcoin-core/gui#770: Revert "gui: provide wallet controller context to wallet actions"
f09bfab4af Revert "gui: provide wallet controller context to wallet actions" (Hennadii Stepanov)

Pull request description:

  The commit 7066e8996d from https://github.com/bitcoin-core/gui/pull/765 breaks "Open Wallet", "Close Wallet" and "Close All Wallets" items in the File menu (at least on Ubuntu 23.10 + Wayland).

  Reverting it to avoid this regression in the 26.0 release.

ACKs for top commit:
  furszy:
    ACK f09bfab4 for including it in v26.
  jarolrod:
    ACK f09bfab4af

Tree-SHA512: fedc621c8e9bf84a263b0c28da53225febe0267d0123830a6192297f38e40726e1613e003b634215e7d16791ba6eab52fb4baab3da9637f6660b6ae1ae98462b
2023-10-23 15:15:48 +01:00
MarcoFalke
fa65887377 ci: Add missing --external to podman image prune 2023-10-23 14:10:12 +02:00
Hennadii Stepanov
f09bfab4af Revert "gui: provide wallet controller context to wallet actions"
This reverts commit 7066e8996d.
2023-10-23 12:14:37 +01:00
fanquake
0046f3dc27 Merge bitcoin/bitcoin#28693: build: Include config/bitcoin-config.h explicitly in util/trace.h
6bdff429ec build: Include `config/bitcoin-config.h` explicitly in `util/trace.h` (Hennadii Stepanov)

Pull request description:

  The `ENABLE_TRACING` macro is expected to be defined in the `config/bitcoin-config.h` header.

  Therefore, the current code is error-prone as it depends on whether the `config/bitcoin-config.h` header was included before or not.

  This bug was noticed while working on CMake [stuff](https://github.com/hebasto/bitcoin/pull/37).

ACKs for top commit:
  fanquake:
    ACK 6bdff429ec

Tree-SHA512: 22c4fdeb51628814050eb99a83db4268a4f3106207eeef918a07214bbc52f2b22490f6b05fcb96216f147afa4197c51102503738131e2583e750b6d195747a49
2023-10-23 11:32:43 +01:00
fanquake
f4e96c29a6 Merge bitcoin/bitcoin#28691: refactor: Remove CBlockFileInfo::SetNull
fac36b94ef refactor: Remove CBlockFileInfo::SetNull (MarcoFalke)

Pull request description:

  Seems better to use C++11 member initializers and then let the compiler figure out how to construct objects of this class.

ACKs for top commit:
  stickies-v:
    ACK fac36b94ef
  pablomartin4btc:
    ACK fac36b94ef
  theStack:
    LGTM ACK fac36b94ef

Tree-SHA512: aee741c8f668f0e5b658fc83f4ebd196b43fead3dd437afdb0a2dafe092ae3d559332b3d9d61985c92e1a59982d8f24942606e6a98598c6ef7ff43697e858725
2023-10-23 10:37:27 +01:00
fanquake
3f482ac231 doc: add historical release notes for 24.2 2023-10-23 10:21:00 +01:00
fanquake
0f15db0ec9 Merge bitcoin/bitcoin#28697: fuzz: Increase merge -rss_limit_mb
fa21535551 fuzz: Increase merge -rss_limit_mb (MarcoFalke)

Pull request description:

  For some reason, the limit is hit. (Presumably due to `-set_cover_merge=1` eating more memory, or by simply having more fuzz inputs).

  Fix it by increasing it for the merge operation.

ACKs for top commit:
  dergoegge:
    ACK fa21535551
  hebasto:
    ACK fa21535551, considering the discussion in https://github.com/bitcoin-core/qa-assets/pull/155.

Tree-SHA512: 4fed0f254eccc6fe0b53656bc345ff898b13811dc39387387317d34b521ab77cee03d82b0896dd92d253b7546b6a7e4bdcd478749f47064374ab44ad759ab9ff
2023-10-23 10:10:53 +01:00
Fabian Jahr
4bfaad4eca chainparams, assumeutxo: Fix signet txoutset hash
Review hint: You can use devtools/utxo_snapshot.sh to validate this.

./contrib/devtools/utxo_snapshot.sh 160000 signet-utxo.dat ./src/bitcoin-cli
2023-10-20 22:53:07 +02:00
Fabian Jahr
a503cd0f0b chainparams, assumeutxo: Fix testnet txoutset hash
Review hint: You can use devtools/utxo_snapshot.sh to validate this.

./contrib/devtools/utxo_snapshot.sh 2500000 testnet-utxo.dat ./src/bitcoin-cli
2023-10-20 22:53:07 +02:00
Fabian Jahr
f6213929c5 assumeutxo: Check deserialized coins for out of range values 2023-10-20 22:53:07 +02:00
Fabian Jahr
66865446a7 docs: Add release notes for #28685 2023-10-20 22:53:06 +02:00
Fabian Jahr
cb0336817e scripted-diff: Rename hash_serialized_2 to hash_serialized_3
-BEGIN VERIFY SCRIPT-
sed -i 's/hash_serialized_2/hash_serialized_3/g' $( git grep -l 'hash_serialized_2' ./src ./contrib ./test )
-END VERIFY SCRIPT-
2023-10-20 22:53:06 +02:00
Fabian Jahr
351370a1d2 coinstats: Fix hash_serialized2 calculation
The legacy serialization was vulnerable to maleation and is fixed by
adopting the same serialization procedure as was already in use for
MuHash.

This also includes necessary test fixes where the hash_serialized2 was
hardcoded as well as correction of the regtest chainparams.

Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-10-20 22:53:05 +02:00
Andrew Chow
c1106cfef5 Merge bitcoin/bitcoin#28669: test: check assumeutxo file for changed outpoint index + de-duplications
d3223685b1 test: De-dublicate/optimize assumeutxo test for further extensions (Fabian Jahr)
0a576d62fe test: check au file with changed outpoint index (Fabian Jahr)

Pull request description:

  Also doing some de-duplications. I kept the second commit separate for now as I am not 100% if this is overdoing it and makes it harder to reason about. But it also makes it easier to add more cases where we change more data.

ACKs for top commit:
  maflcko:
    lgtm ACK d3223685b1
  achow101:
    ACK d3223685b1

Tree-SHA512: be950a34b0ed50cb58459df47cff6513df19d834bf81815572cd26b10dee26e6f80866f0c44023cf246aafbbd256e62d23ce903e8b07fdff2297bc7065799bb8
2023-10-20 16:24:16 -04:00
MarcoFalke
fa21535551 fuzz: Increase merge -rss_limit_mb 2023-10-20 18:16:59 +02:00
fanquake
abfc8c901d Merge bitcoin/bitcoin#28692: fuzz: Delete i2p fuzz test
dd4dcbd4cd [fuzz] Delete i2p target (dergoegge)

Pull request description:

  closes #28665

  The target is buggy and doesn't reach basic coverage.

ACKs for top commit:
  maflcko:
    lgtm ACK dd4dcbd4cd
  glozow:
    ACK dd4dcbd4cd, agree it's better to delete this test until somebody wants to write a better one

Tree-SHA512: b6ca6cad1773b1ceb6e5ac0fd501ea615f66507ef811745799deaaa4460f1700d96ae03cf55b740a96ed8cd2283b3d6738cd580ba97f2af619197d6c4414ca21
2023-10-20 15:30:16 +01:00
MarcoFalke
fac36b94ef refactor: Remove CBlockFileInfo::SetNull 2023-10-20 16:29:02 +02:00
Hennadii Stepanov
6bdff429ec build: Include config/bitcoin-config.h explicitly in util/trace.h
The `ENABLE_TRACING` macro is expected to be defined in the
`config/bitcoin-config.h` header.

Therefore, the current code is error-prone as it depends on whether the
`config/bitcoin-config.h` header was included before or not.
2023-10-20 14:40:26 +01:00
fanquake
3c856e2fe8 Merge bitcoin/bitcoin#28569: log: Don't log cache rebalancing in absense of a snapshot chainstate
ec84f999f1 log: Don't log cache rebalancing in absense of a snapshot chainstate (Fabian Jahr)

Pull request description:

  I have noticed that this log now is always printed, even if there is no snapshot chainstate present or even was present. I think this is confusing to users that have never even thought about using assumeutxo since in that case the rebalancing is just ensuring the normal environment with one chainstate. So I suggest we don't log in absence of a snapshot chainstate. We could also think about rewording the message instead but I think this is simpler.

ACKs for top commit:
  stickies-v:
    utACK ec84f999f1
  glozow:
    concept ACK ec84f999f1, don't have opinions other than removing confusing log
  theStack:
    utACK ec84f999f1

Tree-SHA512: 30bbfc648e7c788106f78d52e47a3aa1e1874f65d13743643dc50bcf7f450d8330711ff9fdeac361722542da6051533153829c6d49033227ed315e111afc899f
2023-10-20 14:39:34 +01:00
dergoegge
dd4dcbd4cd [fuzz] Delete i2p target 2023-10-20 14:03:34 +01:00
Fabian Jahr
ec84f999f1 log: Don't log cache rebalancing in absense of a snapshot chainstate 2023-10-20 14:53:44 +02:00
Fabian Jahr
d3223685b1 test: De-dublicate/optimize assumeutxo test for further extensions 2023-10-20 14:39:23 +02:00
Fabian Jahr
0a576d62fe test: check au file with changed outpoint index 2023-10-20 14:39:21 +02:00
fanquake
fc1073bb45 Merge bitcoin/bitcoin#28643: ci: Add missing CI_RETRY_EXE before git clone
6889a80766 ci: Add missing CI_RETRY_EXE before git clone (MarcoFalke)
b705bade44 ci: Export `IN_GETOPT_BIN` on macOS (Hennadii Stepanov)
1c2132ddd9 Revert "ci: Upgrading pip version in macos environment" (Hennadii Stepanov)

Pull request description:

  This PR is a resurrection of https://github.com/bitcoin/bitcoin/pull/28623:

  > This should fix [bitcoin/bitcoin/actions/runs/6457002476/job/17527598426#step:7:240](https://github.com/bitcoin/bitcoin/actions/runs/6457002476/job/17527598426#step:7:240):
  >
  > ```
  > + git clone --depth=1 https://github.com/bitcoin-core/qa-assets /Users/runner/work/bitcoin/bitcoin/ci/scratch/qa-assets
  > Cloning into '/Users/runner/work/bitcoin/bitcoin/ci/scratch/qa-assets'...
  > error: RPC failed; curl 18 HTTP/2 stream 5 was reset
  > error: 54975 bytes of body are still expected
  > fetch-pack: unexpected disconnect while reading sideband packet
  > fatal: early EOF
  > fatal: fetch-pack: invalid index-pack output
  > Error: Process completed with exit code 128.
  > ```

ACKs for top commit:
  maflcko:
    lgtm ACK 6889a80766

Tree-SHA512: cd50102061a5c57fcf53ecbf552e7eae09995395e91651d00037e3f101c4906c0a994b2f6b1ccf984f2fc902ca59f0db66206ace56a2b6e2810dfa4167f3e115
2023-10-20 11:23:04 +01:00
Andrew Chow
4814e4063e test: Check tx metadata is migrated to watchonly 2023-10-19 18:06:44 -04:00
Andrew Chow
d616d30ea5 wallet: Reload watchonly and solvables wallets after migration
When migrating, create the watchonly and solvables wallets without a
context. Then unload and reload them after migration completes, as we do
for the actual wallet.

There is also additional handling for a failed reload.
2023-10-19 18:06:43 -04:00
Andrew Chow
118f2d7d70 wallet: Copy all tx metadata to watchonly wallet
When moving a tx to the watchonly wallet during migration, make sure
that all of the CWalletTx data follows it.
2023-10-19 18:06:43 -04:00
Andrew Chow
77f0ceb717 Merge bitcoin/bitcoin#28077: I2P: also sleep after errors in Accept() & destroy the session if we get an unexpected error
5c8e15c451 i2p: destroy the session if we get an unexpected error from the I2P router (Vasil Dimov)
762404a68c i2p: also sleep after errors in Accept() (Vasil Dimov)

Pull request description:

  ### Background

  In the `i2p::sam::Session` class:

  `Listen()` does:
  * if the session is not created yet
    * create the control socket and on it:
    * `HELLO`
    * `SESSION CREATE ID=sessid`
    * leave the control socked opened
  * create a new socket and on it:
  * `HELLO`
  * `STREAM ACCEPT ID=sessid`
  * read reply (`STREAM STATUS`), `Listen()` only succeeds if it contains `RESULT=OK`

  Then a wait starts, for a peer to connect. When connected,

  `Accept()` does:
  * on the socket from `STREAM ACCEPT` from `Listen()`: read the Base64 identification of the connecting peer

  ### Problem

  The I2P router may be in such a state that this happens in a quick succession (many times per second, see https://github.com/bitcoin/bitcoin/issues/22759#issuecomment-1609907115): `Listen()`-succeeds, `Accept()`-fails.

  `Accept()` fails because the I2P router sends something that is not Base64 on the socket: `STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed"`

  We only sleep after failed `Listen()` because the assumption was that if `Accept()` fails then the next `Listen()` will also fail.

  ### Solution

  Avoid filling the log with "Error accepting:" messages and sleep also after a failed `Accept()`.

  ### Extra changes

  * Reset the error waiting time after one successful connection. Otherwise the timer will remain high due to problems that have been solved long time in the past.

  * Increment the wait time less aggressively.

  * Handle the unexpected "Session was closed" message more gracefully (don't log stupid messages like `Cannot decode Base64: "STREAM STATUS...`) and destroy the session right way.

ACKs for top commit:
  achow101:
    ACK 5c8e15c451
  jonatack:
    re-ACK 5c8e15c451

Tree-SHA512: 1d47958c50eeae9eefcb668b8539fd092adead93328e4bf3355267819304b99ab41cbe1b5dbedbc3452c2bc389dc8330c0e27eb5ccb880e33dc46930a1592885
2023-10-19 16:08:06 -04:00
Andrew Chow
0655e9dd92 Merge bitcoin/bitcoin#27071: Handle CJDNS from LookupSubNet()
0e6f6ebc06 net: remove unused CConnman::FindNode(const CSubNet&) (Vasil Dimov)
9482cb780f netbase: possibly change the result of LookupSubNet() to CJDNS (Vasil Dimov)
53afa68026 net: move MaybeFlipIPv6toCJDNS() from net to netbase (Vasil Dimov)
6e308651c4 net: move IsReachable() code to netbase and encapsulate it (Vasil Dimov)
c42ded3d9b fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNS (Vasil Dimov)
64d6f77907 net: put CJDNS prefix byte in a constant (Vasil Dimov)

Pull request description:

  `LookupSubNet()` would treat addresses that start with `fc` as IPv6 even if `-cjdnsreachable` is set. This creates the following problems where it is called:

  * `NetWhitelistPermissions::TryParse()`: otherwise `-whitelist=` fails to white list CJDNS addresses: when a CJDNS peer connects to us, it will be matched against IPv6 `fc...` subnet and the match will never succeed.

  * `BanMapFromJson()`: CJDNS bans are stored as just IPv6 addresses in `banlist.json`. Upon reading from disk they have to be converted back to CJDNS, otherwise, after restart, a ban entry like (`fc00::1`, IPv6) would not match a peer (`fc00::1`, CJDNS).

  * `RPCConsole::unbanSelectedNode()`: in the GUI the ban entries go through `CSubNet::ToString()` and back via `LookupSubNet()`. Then it must match whatever is stored in `BanMan`, otherwise it is impossible to unban via the GUI.

  These were uncovered by https://github.com/bitcoin/bitcoin/pull/26859.

  Thus, flip the result of `LookupSubNet()` to CJDNS if the network base address starts with `fc` and `-cjdnsreachable` is set. Since subnetting/masking does not make sense for CJDNS (the address is "random" bytes, like Tor and I2P, there is no hierarchy) treat `fc.../mask` as an invalid `CSubNet`.

  To achieve that, `MaybeFlipIPv6toCJDNS()` has to be moved from `net` to `netbase` and thus also `IsReachable()`. In the process of moving `IsReachable()`, `SetReachable()` and `vfLimited[]` encapsulate those in a class.

ACKs for top commit:
  jonatack:
    Code review ACK 0e6f6ebc06
  achow101:
    ACK 0e6f6ebc06
  mzumsande:
    re-ACK 0e6f6ebc06

Tree-SHA512: 4767a60dc882916de4c8b110ce8de208ff3f58daaa0b560e6547d72e604d07c4157e72cf98b237228310fc05c0a3922f446674492e2ba02e990a272d288bd566
2023-10-19 12:48:39 -04:00
fanquake
6e721c923c Merge bitcoin/bitcoin#28667: doc: add historical release notes for 25.1
2338715506 doc: add historical release notes for 25.1 (fanquake)

Pull request description:

  v25.1 has [been tagged](https://github.com/bitcoin/bitcoin/releases/tag/v25.1).
  For merge post-bin-availability.

ACKs for top commit:
  stickies-v:
    ACK 2338715506

Tree-SHA512: 799937b8bacab7908d86c91855bfcdfd1f8f806d82df0d04cf984d957d9bf1bbdde6103975358360ef35b0cfeba3ce894bc7dce412ea6790d495b78b3cebee0d
2023-10-19 15:45:43 +01:00
fanquake
9e616baec0 Merge bitcoin/bitcoin#22764: build: Include qt sources for parsing with extract_strings.py
b59b31ae0b build: Drop redundant qt/bitcoin.cpp (Hennadii Stepanov)
d90ad5a42e build: Include qt sources for parsing with extract_strings.py (Hennadii Stepanov)

Pull request description:

  On master (4fc15d1566) some strings are still untranslated.

  This PR fixes this issue.

  To verify:
  1) `./autogen.sh && ./configure && make -C src translate` _before_ applying this change
  2) apply this change
  3) `./autogen.sh && ./configure && make -C src translate` _after_ applying this change

  The result of `git diff src/qt/bitcoinstrings.cpp`:
  ```diff
  --- a/src/qt/bitcoinstrings.cpp
  +++ b/src/qt/bitcoinstrings.cpp
  @@ -126,6 +126,7 @@ QT_TRANSLATE_NOOP("bitcoin-core", ""
   "You need to rebuild the database using -reindex to go back to unpruned "
   "mode.  This will redownload the entire blockchain"),
   QT_TRANSLATE_NOOP("bitcoin-core", "%s is set very high!"),
  +QT_TRANSLATE_NOOP("bitcoin-core", "(press q to shutdown and continue later)"),
   QT_TRANSLATE_NOOP("bitcoin-core", "-maxmempool must be at least %d MB"),
   QT_TRANSLATE_NOOP("bitcoin-core", "A fatal internal error occurred, see debug.log for details"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Cannot resolve -%s address: '%s'"),
  @@ -204,6 +205,8 @@ QT_TRANSLATE_NOOP("bitcoin-core", "SQLiteDatabase: Failed to prepare statement t
   QT_TRANSLATE_NOOP("bitcoin-core", "SQLiteDatabase: Failed to read database verification error: %s"),
   QT_TRANSLATE_NOOP("bitcoin-core", "SQLiteDatabase: Unexpected application id. Expected %u, got %u"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Section [%s] is not recognized."),
  +QT_TRANSLATE_NOOP("bitcoin-core", "Settings file could not be read"),
  +QT_TRANSLATE_NOOP("bitcoin-core", "Settings file could not be written"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Signing transaction failed"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Specified -walletdir \"%s\" does not exist"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Specified -walletdir \"%s\" is a relative path"),
  @@ -242,4 +245,5 @@ QT_TRANSLATE_NOOP("bitcoin-core", "User Agent comment (%s) contains unsafe chara
   QT_TRANSLATE_NOOP("bitcoin-core", "Verifying blocks…"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Verifying wallet(s)…"),
   QT_TRANSLATE_NOOP("bitcoin-core", "Wallet needed to be rewritten: restart %s to complete"),
  +QT_TRANSLATE_NOOP("bitcoin-core", "press q to shutdown"),
   };
  ```

ACKs for top commit:
  ryanofsky:
    Code review ACK b59b31ae0b. Being able to use `_()` macro in qt would allow simplifying some code, for example replacing repetitive:
  TheCharlatan:
    ACK b59b31ae0b

Tree-SHA512: 13d9d86b487a1b6e718ae96c198a0a927c881bf33df318412793ec9efba3a7e59cfa836204f73f5b53ff4c99edce778c11bffaa88138b80e37b71e36df6b816f
2023-10-19 13:25:49 +01:00
fanquake
106ab20f12 Merge bitcoin/bitcoin#28673: docs: Add reference to total.coverage report
2ce7e31d2d docs: Add reference to total.coverage report (Greg Sanders)

Pull request description:

  I couldn't get fuzz coverage to work, so couldn't verify what it would look like if made.

ACKs for top commit:
  darosior:
    ACK 2ce7e31d2d
  dergoegge:
    ACK 2ce7e31d2d

Tree-SHA512: c773ef28f7cc630cb094740805c0d235255e7b0559ac7fed4e8952ade4005335f3787bdd2f8786979e341e914b41c4f3eef02bb4773b15f92d618f1040f53c54
2023-10-19 10:32:04 +01:00
fanquake
091d29c495 Merge bitcoin/bitcoin#28617: test: Add Wallet Unlock Context Manager
004903ebad test: Add Wallet Unlock Context Manager (Brandon Odiwuor)

Pull request description:

  Fixes #28601, see https://github.com/bitcoin/bitcoin/pull/28403#discussion_r1325426430

  Add Context Manager to manage the locking and unlocking of locked wallets with a passphrase during testing.

ACKs for top commit:
  kevkevinpal:
    lgtm ACK [004903e](004903ebad)
  maflcko:
    lgtm ACK 004903ebad

Tree-SHA512: ab234c167e71531df0d974ff9a31d444f7ce2a1d05aba5ea868cc9452f139845eeb24ca058d88f058bc02482b762adf2d99e63a6640b872cc71a57a0068abfe8
2023-10-19 10:23:44 +01:00
fanquake
5eb82d5706 Merge bitcoin/bitcoin#28672: build: move -fstack-reuse=none to CORE_CXXFLAGS
8cfa22a846 build: move -fstack-reuse=none to CORE_CXXFLAGS (fanquake)

Pull request description:

  This is not a hardening specific flag, it should be used at all times, regardless of if hardening is enabled or not. Note that this was still the case here, but having this exist in the hardening flags is confusing, and may lead someone to move it inside one of the `use_hardening` blocks, where it would become unused, with `--disable-hardening`.

  Noticed while reviewing https://github.com/hebasto/bitcoin/pull/32#discussion_r1363564161.

ACKs for top commit:
  theuni:
    ACK 8cfa22a846. Agree it's confusing as-is and this better matches the intent.
  hebasto:
    ACK 8cfa22a846
  luke-jr:
    utACK 8cfa22a846
  TheCharlatan:
    ACK 8cfa22a846

Tree-SHA512: 74c3219301398361d06b1ef2257fc9ec18055b1661f8733ee909adefee61e458d70991c32adf0e0450905a7ffbddc99799f5fdac894f4896cfade19f961818df
2023-10-19 10:07:04 +01:00
fanquake
f4049eaf08 Merge bitcoin/bitcoin#28671: test: Fix failing time check in rpc_net.py
fa4c6836c9 test: Fix failing time check in rpc_net.py (MarcoFalke)

Pull request description:

  This check fails on slow runners, such as s390x qemu.

  Fix it by using mocktime.

  See https://github.com/bitcoin/bitcoin/pull/28523#discussion_r1357980527

ACKs for top commit:
  0xB10C:
    ACK fa4c6836c9
  pinheadmz:
    ACK fa4c6836c9
  brunoerg:
    crACK fa4c6836c9

Tree-SHA512: 83fb534682e11e97537dc89de8c16f206f38af1a892a2d5970c02684c05eaea8fc9adba3159f16b2440ca0b3871d513a0562a6f3a38f19a5574a47be0919e42f
2023-10-19 09:46:28 +01:00
fanquake
8cfa22a846 build: move -fstack-reuse=none to CORE_CXXFLAGS
This is not a hardening specific flag, it should be used at all times,
regardless of if hardening is enabled or not. Note that this was
still the case here, but having this exist in the hardening flags is
confusing, and may lead someone to move it inside one of the `use_hardening`
blocks, where it would become unused, with `--disable-hardening`.
2023-10-18 15:20:30 +01:00
Greg Sanders
2ce7e31d2d docs: Add reference to total.coverage report 2023-10-18 09:27:50 -04:00
Ryan Ofsky
655dc716aa Merge bitcoin/bitcoin#28666: test: assumeutxo file with unknown block hash
621db2f004 test: assumeutxo file with unknown block hash (Fabian Jahr)

Pull request description:

  Takes care of one of the open Todos in the assumeutxo functional test. Since an unknown block could be any hash, I simply chose one placeholder, it could also be a random string though.

ACKs for top commit:
  maflcko:
    lgtm ACK 621db2f004
  pablomartin4btc:
    cr ACK 621db2f004
  theStack:
    ACK 621db2f004
  ryanofsky:
    Code review ACK 621db2f004

Tree-SHA512: ee0438ce619f7348c6f88e39b0ea7ddddb8832956d9034ecc795c6033d5d905c09d11b7d0d5afc38231b2fd091ea7c1bd0a0be99d9c32c4e6357a25d76294142
2023-10-18 08:18:48 -04:00
glozow
1803fee1cf Merge bitcoin/bitcoin#28637: doc: add release note for #27460 (new importmempool RPC)
1b672eb766 doc: add release note for #27460 (new `importmempool` RPC) (Sebastian Falbesoner)

Pull request description:

  This PR adds a missing release note for #27460.

ACKs for top commit:
  glozow:
    ACK 1b672eb766

Tree-SHA512: 89deadbfd6779e6eb19801c9fe7459a9876b920d44e09df102774c1eb8b3c0716462613dc99d1711eda4bd959ea61595b33f4528424ac02cf1af6cb4e5f1f0e9
2023-10-18 10:32:45 +01:00
MarcoFalke
fa4c6836c9 test: Fix failing time check in rpc_net.py 2023-10-18 10:35:08 +02:00
Andrew Chow
c2d4e40e45 Merge bitcoin/bitcoin#28651: Make miniscript GetWitnessSize accurate for tapscript
b22810887b miniscript: make GetWitnessSize accurate for tapscript (Pieter Wuille)
8be9851408 test: add tests for miniscript GetWitnessSize (Pieter Wuille)
7ed2b2d430 test: remove mutable global contexts in miniscript fuzzer/test (Pieter Wuille)

Pull request description:

  So far, the same algorithm is used to compute an (upper bound on) the maximum witness size for both P2WSH and P2TR miniscript. That's unfortunate, because it means fee estimations for P2TR miniscript will miss out on the generic savings brought by P2TR witnesses (smaller signatures and public keys, specifically).

  Fix this by making the algorithm use script context specification calculations, and add tests for it. Also included is a cleanup for the tests to avoid mutable globals, as I found it hard to reason about what exactly was being tested.

ACKs for top commit:
  achow101:
    ACK b22810887b
  darosior:
    ACK b22810887b

Tree-SHA512: e4bda7376628f3e91cfc74917cefc554ca16eb5f2a0e1adddc33eb8717c4aaa071e56a40f85a2041ae74ec445a7bd0129bba48994c203e0e6e4d25af65954d9e
2023-10-17 18:27:52 -04:00
Hennadii Stepanov
9c30f5ef9d Merge bitcoin-core/gui#766: Fix coin control input size accounting for taproot spends
00a52e6394 gui: fix coin control input size accounting for taproot spends (Sebastian Falbesoner)

Pull request description:

  If manual coin control is used in the GUI, the input size accounting for P2TR is currently overshooting, as it still assumes P2WPKH (segwitv0) spends which have a larger witness, as ECDSA signatures are longer and the pubkey also has to be provided. Fix that by adding sizes depending on the witness version. Note that the total accounting including outputs is still off and there is some weird logic involved depending on whether SFFO is used, but it's (hopefully) a first step into the right direction.

ACKs for top commit:
  maflcko:
    lgtm ACK 00a52e6394
  furszy:
    utACK 00a52e6394

Tree-SHA512: 9633642f8473247cc3d8e6e0ef502fd515e1dde0e2939d28d6754d0cececedd6a328df22a3d4c85eb2846fd0417cf224b92594613f6e84ada82d2d7d84fc455f
2023-10-17 22:26:25 +01:00
Andrew Chow
fbcf1029a7 Merge bitcoin/bitcoin#28544: wallet: Add TxStateString function for debugging and logging
8a553c9409 wallet: Add TxStateString function for debugging and logging (Ryan Ofsky)

Pull request description:

  I found this useful while debugging silent conflict between #10102 and #27469 recently

ACKs for top commit:
  ishaanam:
    utACK 8a553c9409
  achow101:
    ACK 8a553c9409
  furszy:
    Code ACK 8a553c9

Tree-SHA512: 87965c66bcb59a21e7639878bb567e583a0e624735721ff7ad1104eed6bb9fba60607d0e3de7be3304232b3a55f48bab7039ea9c26b0e81963e59f9acd94f666
2023-10-17 15:28:05 -04:00
Andrew Chow
9af87cf348 test: Check that a failed wallet migration is cleaned up 2023-10-17 14:57:04 -04:00
Ryan Ofsky
ff6be778f9 Merge bitcoin/bitcoin#28647: test: Add assumeutxo test for wrong hash
fa68571566 test: Add assumeutxo test for wrong hash (MarcoFalke)

Pull request description:

  Also:
  * Update test TODOs
  * Fix off-by-4 typo in test, remove `struct` import

ACKs for top commit:
  fjahr:
    utACK fa68571566
  theStack:
    Code-review re-ACK fa68571566
  pablomartin4btc:
    re ACK fa68571566
  ryanofsky:
    Code review ACK fa68571566

Tree-SHA512: 877653010efe4e20018827e8ec2801d036e1344457401f0c9e5d55907b817724201dd2e3f0f29505bbff619882c0c2cd731ecdcd209258bcefe11b86ff0205dd
2023-10-17 12:05:42 -04:00
fanquake
2338715506 doc: add historical release notes for 25.1 2023-10-17 16:39:25 +01:00
Fabian Jahr
621db2f004 test: assumeutxo file with unknown block hash 2023-10-17 17:12:29 +02:00
MarcoFalke
fa68571566 test: Add assumeutxo test for wrong hash 2023-10-17 13:24:07 +02:00
fanquake
738ef44abb Merge bitcoin/bitcoin#28652: assumeutxo: fail early if snapshot block hash doesn't match AssumeUTXO parameters
9620cb4493 assumeutxo: fail early if snapshot block hash doesn't match AssumeUTXO parameters (Sebastian Falbesoner)

Pull request description:

  Right now the `loadtxoutset` RPC call treats literally all files with a minimum size of 40 bytes (=size of metadata) as potential valid snapshot candidates and the waiting loop for seeing the metadata block hash in the headers chain is always entered, e.g.:
  ```
  $ ./src/bitcoin-cli loadtxoutset ~/.vimrc
  <wait>

  bitcoind log:
  ...
  2023-10-15T14:55:45Z [snapshot] waiting to see blockheader 626174207465730a7265626d756e207465730a656c62616e65207861746e7973 in headers chain before snapshot activation
  ...
  ```
  There is no point in doing any further action though if we already know from the start that the UTXO snapshot loading won't be successful. This PR adds an assumeutxo parameter check immediately after the metadata is read in, so we can fail immediately on a mismatch:
  ```
  $ ./src/bitcoin-cli loadtxoutset ~/.vimrc
  error code: -32603
  error message:
  Unable to load UTXO snapshot, assumeutxo block hash in snapshot metadata not recognized (626174207465730a7265626d756e207465730a656c62616e
  65207861746e7973)
  ```
  This way, users who mistakenly try to load files that are not snapshots don't have to wait 10 minutes (=the block header waiting timeout) anymore to get a negative response. If a file is loaded which is a valid snapshot (referencing to an existing block hash), but one which doesn't match the parameters, the feedback is also faster, as we don't have to wait anymore to see the hash in the headers chain before getting an error.

  This is also partially fixes #28621.

ACKs for top commit:
  maflcko:
    lgtm ACK 9620cb4493
  ryanofsky:
    Code review ACK 9620cb4493. This should fix an annoyance and bad UX.
  pablomartin4btc:
    tACK 9620cb4493

Tree-SHA512: f88b865e9d46254858e57c024463f389cd9d8760a7cb30c190aa1723a931e159987dfc2263a733825d700fa612e7416691e4d8aab64058f1aeb0a7fa9233ac9c
2023-10-17 10:20:08 +01:00
fanquake
e6c30834b4 Merge bitcoin/bitcoin#28656: fuzz: Allow multiple --m_dir args
faa5e061c2 fuzz: Allow multiple --m_dir args (MarcoFalke)

Pull request description:

  This allows to merge the result from several servers (or just several folders) at the same time, instead of having to iterate over them.

  This should also allow the fuzz engine (libFuzzer) to optimize the final merge result more, because all fuzz inputs from all folders are available at the same time.

ACKs for top commit:
  dergoegge:
    tACK faa5e061c2

Tree-SHA512: bf0da418b1f7b8a8af16bb7cc1e148b1ccd0f17062ce70758d1ca5b35c3eee77c0c30377d376befdd55480adfd1f1a1073cfc47118e7a710e6760e020abe24bb
2023-10-17 09:51:51 +01:00
fanquake
4caa10b580 Merge bitcoin/bitcoin#28628: lint: Include test_utxo_snapshots in lint_shell, fix linter errors
348e79f7c6 lint: Include test_utxo_snapshots in lint_shell (Fabian Jahr)

Pull request description:

  jamesob excluded `test_utxo_snapshots.sh` from the shell linter with this explanation: "Add the script to the shellcheck exception list since the quoted variables rule needs to be violated in order to get bitcoind to pick up on $EARLY_IBD_FLAGS." However, macrofake pointed out that single lines can be excluded from linting.

  This fixes one fixable rule violation, excludes the rest of the offending lines from the linter and then removes the exclusion of the `test_utxo_snapshots.sh` file. Also adds documentation.

ACKs for top commit:
  Empact:
    ACK 348e79f7c6
  maflcko:
    lgtm ACK 348e79f7c6
  pablomartin4btc:
    tACK 348e79f7c6

Tree-SHA512: a904cc1cc3c94488dfbd39ea69a3ef17259f991708a797009001669448fef81eed086ecbce1ec433988d88baef293849698e2e0eb86a969b949cc7ef93af7b4b
2023-10-17 09:48:50 +01:00
Andrew Chow
76d89579b5 Merge bitcoin/bitcoin#28630: depends: update LD64_VERSION to 711
092daa2f95 contrib: add test for macOS linker version to symbol-check (fanquake)
cefbf0bc20 depends: update LD64_VERSION to 711 (fanquake)

Pull request description:

  I forgot to do this in 7d5815293e.
  Add a test so it's impossible to forget.

ACKs for top commit:
  TheCharlatan:
    utACK 092daa2f95
  achow101:
    ACK 092daa2f95
  jarolrod:
    ACK 092daa2f95
  hebasto:
    ACK 092daa2f95.
  laanwj:
    ACK 092daa2f95

Tree-SHA512: 37f0bdfd6607a7760eabe5efe279532ba0c59c0915161e08d5e3b9a0b7705839d62537d6e17406062f6a0a1db5407575da7cd671e9cb916e422e77f5649c6e2b
2023-10-16 13:29:26 -04:00
Andrew Chow
90f7d8a7f9 Merge bitcoin/bitcoin#28539: lib: add taproot support to libconsensus
ff8e2fc2e2 fuzz: add coverage for `bitcoinconsensus_verify_script_with_spent_outputs` (brunoerg)
c5f2a757d7 docs: add release notes for #28539 (brunoerg)
de54882348 docs: add docs for additional libconsensus functions (Jake Rawsthorne)
70106e0689 docs: link to rust-bitcoinconsensus (Jake Rawsthorne)
fb0db07e41 lib: add Taproot support to libconsensus (Jake Rawsthorne)

Pull request description:

  Grabbed from #21158. Closes #21133.

ACKs for top commit:
  achow101:
    ACK ff8e2fc2e2
  theStack:
    ACK ff8e2fc2e2
  darosior:
    re-ACK ff8e2fc2e2

Tree-SHA512: bf6f500c7e8c9ff6884137c2cd9b4522c586e52848dd639b774b94d998b0516b877498d24f3a6cc7425aedf81d18b0d30c1ccf19e2d527fdfdfa3955ca49b6e7
2023-10-16 12:59:39 -04:00
Sebastian Falbesoner
9620cb4493 assumeutxo: fail early if snapshot block hash doesn't match AssumeUTXO parameters 2023-10-16 17:20:59 +02:00
fanquake
08ea835220 Merge bitcoin/bitcoin#28583: refactor: [tidy] modernize-use-emplace
fa05a726c2 tidy: modernize-use-emplace (MarcoFalke)

Pull request description:

  Constructing a temporary unnamed object only to copy or move it into a container seems both verbose in code and a strict performance penalty.

  Fix both issues via the `modernize-use-emplace` tidy check.

ACKs for top commit:
  Sjors:
    re-utACK fa05a726c2
  hebasto:
    ACK fa05a726c2.
  TheCharlatan:
    ACK fa05a726c2

Tree-SHA512: 4408a094f406e7bf6c1468c2b0798f68f4d952a1253cf5b20bdc648ad7eea4a2c070051fed46d66fd37bce2ce6f85962484a1d32826b7ab8c9baba431eaa2765
2023-10-16 15:35:50 +01:00
MarcoFalke
faa5e061c2 fuzz: Allow multiple --m_dir args 2023-10-16 16:04:02 +02:00
Hennadii Stepanov
92704535f6 Merge bitcoin-core/gui#765: Fix wallet list hover crash on shutdown
8b6470a906 gui: disable top bar menu actions during shutdown (furszy)
7066e8996d gui: provide wallet controller context to wallet actions (furszy)

Pull request description:

  Small follow-up to #751.

  Fixes another crash cause during shutdown. Which occurs when the user hovers over the wallets list.

  Future Note:
  This surely happen in other places as well, we should re-work the way we connect signals. Register
  lambas without any precaution can leave dangling pointers.

ACKs for top commit:
  hebasto:
    ACK 8b6470a906, I've tested each commit separately on macOS Sonoma 14.0 (Apple M1).

Tree-SHA512: 6fbd1bcd6717a8c1633beb9371463ed22422f929cccf9b791ee292c5364134c501e099329cf77a06b74a84c64c1c3d22539199ec49ccd74b3950036316c0dab3
2023-10-16 13:37:35 +01:00
Vasil Dimov
0e6f6ebc06 net: remove unused CConnman::FindNode(const CSubNet&) 2023-10-16 12:59:47 +02:00
Vasil Dimov
9482cb780f netbase: possibly change the result of LookupSubNet() to CJDNS
All callers of `LookupSubNet()` need the result to be of CJDNS type if
`-cjdnsreachable` is set and the address begins with `fc`:

* `NetWhitelistPermissions::TryParse()`: otherwise `-whitelist=` fails
  to white list CJDNS addresses: when a CJDNS peer connects to us, it
  will be matched against IPv6 `fc...` subnet and the match will never
  succeed.

* `BanMapFromJson()`: CJDNS bans are stored as just IPv6 addresses in
  `banlist.json`. Upon reading from disk they have to be converted back
  to CJDNS, otherwise, after restart, a ban entry like (`fc00::1`, IPv6)
  would not match a peer (`fc00::1`, CJDNS).

* `setban()` (in `rpc/net.cpp`): otherwise `setban fc.../mask add` would
  add an IPv6 entry to BanMan. Subnetting does not make sense for CJDNS
  addresses, thus treat `fc.../mask` as invalid `CSubNet`. The result of
  `LookupHost()` has to be converted for the case of banning a single
  host.

* `InitHTTPAllowList()`: not necessary since before this change
  `-rpcallowip=fc...` would match IPv6 subnets against IPv6 peers even
  if they started with `fc`. But because it is necessary for the above,
  `HTTPRequest::GetPeer()` also has to be adjusted to return CJDNS peer,
  so that now CJDNS peers are matched against CJDNS subnets.
2023-10-16 12:57:49 +02:00
fanquake
22fa1f4702 Merge bitcoin/bitcoin#28565: rpc: getaddrmaninfo followups
e6e444c06c refactor: add and use EnsureAnyAddrman in rpc (stratospher)
bf589a50a0 doc: add release notes for #27511 (stratospher)
3931e6abc3 rpc: `getaddrmaninfo` followups (stratospher)

Pull request description:

  - make `getaddrmaninfo` RPC public since it's not for development purposes only and regular users might find it useful. [#26988 (comment)](https://github.com/bitcoin/bitcoin/pull/26988#issuecomment-1738371584)
  - add missing `all_networks` key to RPC help. [#27511 (comment)](https://github.com/bitcoin/bitcoin/pull/27511#discussion_r1335084087)
  - fix clang format spacing
  - add and use `EnsureAddrman` in RPC code. [#27511 (comment)](https://github.com/bitcoin/bitcoin/pull/27511#discussion_r1331501491)

ACKs for top commit:
  0xB10C:
    Code Review re-ACK e6e444c06c
  theStack:
    Code-review ACK e6e444c06c
  pablomartin4btc:
    tested ACK e6e444c06c

Tree-SHA512: c14090d5c64ff15e92d252578de2437bb2ce2e1e431d6698580241a29190f0a3528ae5b013c0ddb76a9ae538507191295c37cab7fd93469941cadbde44587072
2023-10-16 11:21:45 +02:00
Sebastian Falbesoner
1b672eb766 doc: add release note for #27460 (new importmempool RPC)
Co-authored-by: glozow <gloriajzhao@gmail.com>
2023-10-15 18:14:43 +02:00
fanquake
ab2f531b78 Merge bitcoin/bitcoin#28650: fuzz: Merge with -set_cover_merge=1
fa858d63a0 fuzz: Merge with -set_cover_merge=1 (MarcoFalke)

Pull request description:

  This should be less controversial than commit 151a2b189c. The overall size of the qa-assets repo is reduced further from 1.9GB to 1.6GB. Also, the runtime to iterate on the resulting folder is reduced further from ~1699s to ~1149s (N=1).

ACKs for top commit:
  murchandamus:
    crACK fa858d63a0
  dergoegge:
    ACK fa858d63a0

Tree-SHA512: e23fa93bd48f01d11c551b035004c678bd6d76bc24ac7d0d0a7883060804e6711763cbd0cd0ded3aad3e4c40da764decae81c2703388cc11961def3c89a4f9ba
2023-10-15 11:08:31 +02:00
fanquake
cee39d0628 Merge bitcoin/bitcoin#27793: ci: label docker images and prune dangling images selectively
e44c574650 ci: always prune all dangling bitcoin-ci-test images (stickies-v)
ce1699706e ci: add label to docker images (stickies-v)

Pull request description:

  Follow-up from https://github.com/bitcoin/bitcoin/pull/27777#discussion_r1210209382.

  Labeling the docker images produced by the CI allows us/the user to apply batch operations to all images (including dangling ones) produced by the ci without affecting other, non-bitcoin-ci images. With labeling, we can safely always prune dangling bitcoin-ci-test images without checking for `RESTART_CI_DOCKER_BEFORE_RUN`, which we enable on our persistent runners.

ACKs for top commit:
  fanquake:
    utACK e44c574650

Tree-SHA512: 1009fb1be78fbc80b5341ba92eac2991e77d050e1ab6048d1d9a65af73413a6be7afc1e1c764eb3f347f363af31245b93fdb38f6ac016d775aad4a0f36e4c98f
2023-10-15 10:48:37 +02:00
furszy
8b6470a906 gui: disable top bar menu actions during shutdown
Opening the top bar menu when the app is being destroyed
freezes the GUI shutdown process for no reason. No menu
action can be executed.

Note:
This behavior is consistent with how the tray icon menu
is cleared too.
2023-10-13 17:40:32 -03:00
furszy
7066e8996d gui: provide wallet controller context to wallet actions
Addressing potential crashes during shutdown. The most
noticeable one can be triggered by hovering over the
wallet list as the app shuts down.
2023-10-13 17:40:32 -03:00
Pieter Wuille
b22810887b miniscript: make GetWitnessSize accurate for tapscript 2023-10-13 15:28:38 -04:00
Pieter Wuille
8be9851408 test: add tests for miniscript GetWitnessSize 2023-10-13 14:57:03 -04:00
Pieter Wuille
7ed2b2d430 test: remove mutable global contexts in miniscript fuzzer/test 2023-10-13 14:34:02 -04:00
Andrew Chow
78b7e95518 Merge bitcoin/bitcoin#28645: test: fix assert_debug_log call-site bugs, add type checks
ac4caf3366 test: fix `assert_debug_log` call-site bugs, add type checks (Sebastian Falbesoner)

Pull request description:

  Two recently added tests (PR #28625 / commit 2e31250027 and PR #28634 / commit 3bb51c29df) introduced bugs by wrongly using the `assert_debug_log` helper:

  5ea4fc05ed/test/functional/feature_assumeutxo.py (L84-L85) (already fixed in https://github.com/bitcoin/bitcoin/pull/28639)

  5ea4fc05ed/test/functional/p2p_v2_transport.py (L148)
  5ea4fc05ed/test/functional/p2p_v2_transport.py (L159)

  Instead of passing the expected debug string in a list as expected, it was passed as bare string, which is then interpretered as a list of characters, very likely leading the debug log assertion pass even if the intended message is not appearing. Thanks to maflcko for discovering: https://github.com/bitcoin/bitcoin/pull/28625#discussion_r1356489861

  In order to avoid bugs like this in the future, enforce that the `{un}expected_msgs` parameters are lists, as discussed in https://github.com/bitcoin/bitcoin/pull/28625#discussion_r1356864233. Using mypy might be an alternative, but I guess it takes quite a bit of effort to properly integrate this into CI for the whole functional test suite (including taking care of false-positives), so I decided to go with the simpler "manual asserts" hack. Suggestions are very welcome of course.

ACKs for top commit:
  achow101:
    ACK ac4caf3366
  maflcko:
    lgtm ACK ac4caf3366
  dergoegge:
    ACK ac4caf3366

Tree-SHA512: a9677af76a0c370e71f0411339807b1dc6b2a81763db4ec049cd6d766404b916e2bdd002883db5a79c9c388d7d8ebfcbd5f31d43d50be868eeb928e3c906a746
2023-10-13 12:32:01 -04:00
MarcoFalke
fa858d63a0 fuzz: Merge with -set_cover_merge=1 2023-10-13 18:09:29 +02:00
Fabian Jahr
348e79f7c6 lint: Include test_utxo_snapshots in lint_shell 2023-10-13 14:06:06 +02:00
brunoerg
ff8e2fc2e2 fuzz: add coverage for bitcoinconsensus_verify_script_with_spent_outputs
Co-authored-by: Antonie Poinsot <darosior@protonmail.com>
2023-10-13 08:55:12 -03:00
brunoerg
c5f2a757d7 docs: add release notes for #28539 2023-10-13 08:55:12 -03:00
Jake Rawsthorne
de54882348 docs: add docs for additional libconsensus functions
Co-authored-by: Bruno Garcia <brunoely.gc@gmail.com>
2023-10-13 08:55:12 -03:00
Jake Rawsthorne
70106e0689 docs: link to rust-bitcoinconsensus 2023-10-13 08:55:12 -03:00
Jake Rawsthorne
fb0db07e41 lib: add Taproot support to libconsensus
Co-authored-by: Bruno Garcia <brunoely.gc@gmail.com>
2023-10-13 08:55:07 -03:00
Sebastian Falbesoner
ac4caf3366 test: fix assert_debug_log call-site bugs, add type checks
Two recently added tests (PR #28625 / commit 2e31250027
and PR #28634 / commit 3bb51c29df)
introduced a bug by wrongly using the `assert_debug_log` helper.
Instead of passing the expected debug string in a list as expected, it
was passed as bare string, which is then interpretered as a list of
characters, very likely leading the debug log assertion pass even if the
intended message is not appearing.

In order to avoid bugs like this in the future, enforce that the
`{un}expected_msgs` parameters are lists.
2023-10-13 12:42:49 +02:00
stickies-v
e44c574650 ci: always prune all dangling bitcoin-ci-test images
Since all bitcoin-ci-test images are now labeled, we can always
prune all dangling images, regardless of whether we are in
RESTART_CI_DOCKER_BEFORE_RUN.

To be safe, still prune all images if RESTART_CI_DOCKER_BEFORE_RUN
in case the filtering doesn't work, or if images were created on
an earlier version that did not assign labels.
2023-10-13 11:23:30 +01:00
MarcoFalke
6889a80766 ci: Add missing CI_RETRY_EXE before git clone 2023-10-13 10:37:34 +01:00
Hennadii Stepanov
b705bade44 ci: Export IN_GETOPT_BIN on macOS
This variable is required for the `retry` script.
2023-10-13 10:37:34 +01:00
Hennadii Stepanov
1c2132ddd9 Revert "ci: Upgrading pip version in macos environment"
This reverts commit 057750c09d.

It is not needed anymore in the GHA CI.
This change will make the code much simpler in the following commit.
2023-10-13 10:37:28 +01:00
stickies-v
ce1699706e ci: add label to docker images
This allows us or the user to perform batch operations on all
images produced by the ci, e.g. to prune all dangling images,
without affecting non-ci images.
2023-10-13 10:17:21 +01:00
fanquake
73dfa6da08 Merge bitcoin/bitcoin#28631: devtools: test_utxo_snapshots.sh sleep cleanup and documentation
36a3004a41 devtools: test_utxo_snapshots.sh sleep cleanup and documentation (Fabian Jahr)

Pull request description:

  There were bare sleep statements in the script where it was unclear why they were needed and I think they could also be replaced by existing helpers.

ACKs for top commit:
  Sjors:
    utACK 36a3004a41
  pablomartin4btc:
    utACK 36a3004a41

Tree-SHA512: b6e2fc69cd7babcfa7f413f11304f4d648d6d64b3b526862664ccedb0016ad182b9e886aa4e8c33315e4c18824512e11a4fd6365f9c3c95093967d3ef7687e62
2023-10-13 11:16:33 +02:00
fanquake
448790c00a Merge bitcoin/bitcoin#28639: refactor: Remove unused nchaintx from SnapshotMetadata constructor, fix test, add test
fafde92f84 test: Check snapshot file with wrong number of coins (MarcoFalke)
faa90f6e7b refactor: Remove unused nchaintx from SnapshotMetadata constructor (MarcoFalke)

Pull request description:

  See commit messages

ACKs for top commit:
  Sjors:
    utACK fafde92f84
  theStack:
    ACK fafde92f84

Tree-SHA512: 9ed2720b50d1c0938f30543ba143e1a4c6af3a0ff166f8b3eb452e1d99ddee6e3443a4c99f77efe94b8c3eb2feff984bf5259807ee8085e1e0e1e0d1de98227e
2023-10-13 11:14:54 +02:00
fanquake
9e068f9612 Merge bitcoin/bitcoin#28459: build: add -mbranch-protection=bti (aarch64) to hardening flags
61a6c3b0e9 build: add `-mbranch-protection=bti` to aarch64 hardening flags (fanquake)

Pull request description:

  This is a simpler (less hardening) version of https://github.com/bitcoin/bitcoin/pull/24123.

  You can inspect binaries using `readelf -n`, and look for BTI in a `.note.gnu.property`. i.e
  ```bash
  readelf -n src/bitcoin-cli

  Displaying notes found in: .note.gnu.property
    Owner                Data size Description
    GNU                  0x00000010NT_GNU_PROPERTY_TYPE_0
        Properties: AArch64 feature: BTI
  ```

  Related to https://github.com/bitcoin/bitcoin/issues/19075.

ACKs for top commit:
  TheCharlatan:
    utACK 61a6c3b0e9

Tree-SHA512: 64504de44e91d853165daf4111dca905d8eb9ef3f4bfb0d447c677b02c9100dbd56f13e6fe6539fb06c2343a094229591ac5d1bd9e184b32b512c0ac3f9bac36
2023-10-13 10:59:57 +02:00
fanquake
504f1f30ed Merge bitcoin/bitcoin#28547: ci: Work around podman stop intermittent failure
fa2c894cbb ci: move-only CI_CONTAINER_ID to 02_run_container.sh (MarcoFalke)
fa695b4df0 ci: Work around podman stop bug (MarcoFalke)
fa09a031c1 ci: Add set -ex to 02_run_container.sh (MarcoFalke)
fac9abbf47 ci: Rename 04_install to 02_run_container (MarcoFalke)

Pull request description:

  Sometimes, it seems that `podman stop` does not work. Presumably, it falls back to `podman kill`, which is async.

  Try to work around this intermittent issue by using the `rm --force` over `stop`.

  Example failing log https://cirrus-ci.com/task/4549784611061760?logs=ci#L238:

  ```
  Restart docker before run to stop and clear all containers started with --rm
  ++ podman container stop --all
  e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009
  ++ echo 'Prune all dangling images'
  Prune all dangling images
  ++ docker image prune --force
  Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
  +++ docker run --cap-add LINUX_IMMUTABLE --rm --interactive --detach --tty --mount type=bind,src=/tmp/cirrus-build-1970593815,dst=/tmp/cirrus-build-1970593815,readonly --mount type=volume,src=ci_macos_cross_ccache,dst=/tmp/ccache_dir --mount type=volume,src=ci_macos_cross_depends,dst=/ci_container_base/depends --mount type=volume,src=ci_macos_cross_previous_releases,dst=/ci_container_base/prev_releases --env-file /tmp/env --name ci_macos_cross ci_macos_cross
  Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
  time="2023-09-27T20:55:39Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly"
  Error: creating container storage: the container name "ci_macos_cross" is already in use by e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009. You have to remove that container to be able to reuse that name: that name is already in use

ACKs for top commit:
  hebasto:
    ACK fa2c894cbb, I have reviewed the code and tested it locally.

Tree-SHA512: 31fca340c6bedaadf4dd51fa745d9b3969042cebc0c7c904ef18af3f2f986039ec4354ccdff1422fbf77cf223e4423857368dce53cfa67ef15c76b78d007eace
2023-10-13 10:24:07 +02:00
fanquake
151a2b189c Merge bitcoin/bitcoin#28644: test: Fuzz merge with -use_value_profile=0 for now
faa190b1ef test: Fuzz merge with -use_value_profile=0 for now (MarcoFalke)

Pull request description:

  Seems odd that this has to be done, but for now there are (unknown) size limits on the qa-assets repo. Also, a larger size means that cloning and iterating over the files takes a longer time.

  Not sure how to measure the net impact of this, but with some backups reverting this commit, it can be limited on the downside?

ACKs for top commit:
  dergoegge:
    ACK faa190b1ef

Tree-SHA512: 9f8b3f4526f60e4ff6fca97859a725d145a8339c216bd15c92fad7e53f84308745fee47727527de459c0245ef9d474a9dc836fee599ab2b556b519bd900b9a33
2023-10-13 10:16:38 +02:00
fanquake
5ea4fc05ed Merge bitcoin/bitcoin#28640: ci: Use clang-17 in Asan task
fa2843eba4 ci: Bump asan (MarcoFalke)

Pull request description:

  Needed to bump the EOL date and unlock clang-17.

ACKs for top commit:
  fanquake:
    ACK fa2843eba4

Tree-SHA512: 7d8b3b30ed65bd26f9640db8d06de8c27a9ad8a8160f9e645a97b99df867aa30508491dd8957a54edad724f8672ef1054041106d90ee826ba3e856176ab52afc
2023-10-12 17:18:00 +02:00
fanquake
bf8c41d251 Merge bitcoin/bitcoin#28641: ci: Drop no longer needed NOLINTNEXTLINE
79789ccafe ci: Drop no longer needed `NOLINTNEXTLINE` (Hennadii Stepanov)

Pull request description:

  After recent tool updates in the "tidy" CI task, the one instance of `NOLINTNEXTLINE` is not required anymore.

ACKs for top commit:
  fanquake:
    ACK 79789ccafe

Tree-SHA512: 382f3e0570f36a481498ac118ee30a34aabfd6285b21a67c19c74b240305315c5a118fb70f172b843a086262433462d45998e78306697be0ddfb20094733ee98
2023-10-12 17:04:24 +02:00
MarcoFalke
faa190b1ef test: Fuzz merge with -use_value_profile=0 for now 2023-10-12 16:44:41 +02:00
MarcoFalke
fa2843eba4 ci: Bump asan 2023-10-12 13:07:06 +02:00
Hennadii Stepanov
79789ccafe ci: Drop no longer needed NOLINTNEXTLINE 2023-10-12 11:01:47 +01:00
fanquake
06d469c26b Merge bitcoin/bitcoin#28629: test: fix usdt undeclared function errors on mantis
4077e43bf6 test: fix usdt undeclared function errors on mantis (willcl-ark)

Pull request description:

  This is one way to fix #28600

  Recently usage of undeclared functions became an error rather than a warning, in C2x. https://reviews.llvm.org/D122983?id=420290

  This change has migrated into the build tools of Ubuntu 23.10 which now causes the USDT tests to fail to compile, see
  https://github.com/bitcoin/bitcoin/issues/28600

  I think there are various potential fixes:

  1. Manually declare the functions we use
  2. Fix imports so that manual declarations aren't needed
  3. Revert the new C2X behaviour and don't error on implicit function declarations

  I would have preferred solution 2, but I believe this will require changes to the upstream bcc package. Having played with the imports I can get things working in a standalone C program, using system headers, but when building the program from a python context as we do in the test it uses its own headers (bundled with the python lib) rather than the system ones, and manually importing (some) system headers results in definition mismatches. I also investigated explicitly importing required headers from the package, which use paths like `#import </virtual/bcc/bcc_helpers.h>`, but this seems more obtuse and brittle than simply ignoring the warning.

  Therefore I think that until the upstream python pacakge fixes their declarations, we should fix this by setting `-Wno-error=implicit-function-declaration` for the tracing programs.

  cc maflcko 0xB10C

ACKs for top commit:
  maflcko:
    lgtm ACK 4077e43bf6

Tree-SHA512: 8368bb1155e920a95db128dc893267f8dab64f1ae53f6d63c6d9294e2e4e92bef8515e3697e9113228bedc51c0afdbc5bbcf558c119bf0eb3293dc2ced86b435
2023-10-12 12:00:12 +02:00
fanquake
a927d5cabe Merge bitcoin/bitcoin#27228: test: exempt previous release binaries from valgrind
850670e3d6 test: don't run old binaries under valgrind (Sjors Provoost)

Pull request description:

  Some, but not all, backward compatibility tests fail for me and it seems useless to run old release binaries under valgrind anyway.

  Can be tested by running `test/functional/feature_txindex_compatibility.py --valgrind --timeout-factor=10` with and without this PR.

  —
  The previous version of this PR disabled these test entirely under valgrind. The current version does run the test, but starts the old binaries without valgrind.

ACKs for top commit:
  maflcko:
    lgtm ACK 850670e3d6

Tree-SHA512: ebdf461083f1292528e6619963b910f486b60b4f6b183f0aea2c8bfcafa98caeb204d138700cd288450643bcec5e49e12b89f2f7537fccdf495a2a33acd9cea0
2023-10-12 11:53:15 +02:00
MarcoFalke
fa05a726c2 tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
MarcoFalke
fafde92f84 test: Check snapshot file with wrong number of coins
Also, fix a bug in an assert_debug_log call.
2023-10-12 11:14:56 +02:00
MarcoFalke
faa90f6e7b refactor: Remove unused nchaintx from SnapshotMetadata constructor
Also, remove wrong nChainTx comment and cast.
2023-10-12 11:14:32 +02:00
fanquake
4a5aae9330 Merge bitcoin/bitcoin#28634: test: BIP324: add check for detection of missing garbage terminator
3bb51c29df test: BIP324: add check for missing garbage terminator detection (Sebastian Falbesoner)

Pull request description:

  This PR adds test coverage for the "missing garbage terminator" detection on incoming v2 transport (BIP324) connections:
  04265ba937/src/net.cpp (L1205-L1209)

  Note that this always happens at the same exact amount of bytes sent in (after 64 + 4095 + 16 = 4175 bytes), if at no point, the last 16 bytes of potential authentication data match the garbage, i.e. all the previous bytes after the ellswift pubkey. To keep it simple, we just send in zero-value bytes here and verify that the detection hits exactly after the last bytes is sent.

  AFAICT, with this PR all the v2 transport errors that can be triggered in this simple way of "just open a socket and send in a fixed byte-string" are covered. For more advanced test, we need BIP324 cryptography in the test framework in order to perform a v2 handshake etc. (PRs #28374, #24748).

ACKs for top commit:
  sipa:
    utACK 3bb51c29df
  laanwj:
    ACK 3bb51c29df

Tree-SHA512: f88275061c7c377a3d9f2608452671afc26deb6d5bd5be596de987c7e5042555153ffe681760c33bce2b921ae04e50f349ea0128a677e6443a95a079e52cdc5f
2023-10-12 09:35:07 +02:00
Sjors Provoost
850670e3d6 test: don't run old binaries under valgrind
This is unnecessary and caused test failures. The backward
compatibility tests are meant to find regressions in the
current codebase, not to detect bugs in older releases.
2023-10-12 09:25:44 +02:00
fanquake
ccf7895e17 Merge bitcoin/bitcoin#28633: ci: Install Qt's default Android API platform
78d3062b68 ci: Install Android API 31 platform as Qt expects (Hennadii Stepanov)

Pull request description:

  When building the `qt` package, it expects that the default (in Qt's view) Android API platform is installed.

  During the recent Qt version [update](https://github.com/bitcoin/bitcoin/pull/28561), it has been changed:
  ```diff
  --- a/mkspecs/features/android/sdk.prf
  +++ b/mkspecs/features/android/sdk.prf
  @@ -1,6 +1,6 @@
   API_VERSION_TO_USE = $$(ANDROID_API_VERSION)
   isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = $$API_VERSION
  -isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-28
  +isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-31

   ANDROID_JAR_FILE = $$ANDROID_SDK_ROOT/platforms/$$API_VERSION_TO_USE/android.jar
   !exists($$ANDROID_JAR_FILE) {
  ```

  This PR fixes the CI for the Android task and addresses https://github.com/bitcoin/bitcoin/pull/28561#issuecomment-1749180177.

  Qt [docs](https://doc.qt.io/qt-5/android.html) still claim that Android API Level 21 and up are supported, however, I did not test every possible configuration.

  NOTE: https://github.com/bitcoin/bitcoin/pull/28611 is still valid.

ACKs for top commit:
  maflcko:
    lgtm ACK 78d3062b68
  jarolrod:
    tACK 78d3062b68

Tree-SHA512: 781fba6d80aae7e6500854de14af0d30169c258e395b9e482a5430a7b4a2211a6181f8c9ee58543c896b431abf09e3e7c5573b9672ed128658f11f98a2006e7e
2023-10-12 09:11:49 +02:00
Andrew Chow
e3eb3aae30 Merge bitcoin/bitcoin#28625: test: check that loading snapshot not matching AssumeUTXO parameters fails
2e31250027 test: check that loading snapshot not matching AssumeUTXO parameters fails (Sebastian Falbesoner)

Pull request description:

  This PR adds test coverage for the failed loading of an AssumeUTXO snapshot in case the referenced block hash doesn't match the parameters in the chainparams. Right now, I expect this would be the most common error-case for `loadtxoutset` out in the wild, as for mainnet the `m_assumeutxo_data` map is empty and this error condition would obviously always be triggered for any (otherwise valid, correctly encoded) snapshot. Note that this test-case is the simplest scenario and doesn't cover any of the TODO ideas mentioned at the top of the functional test yet.

ACKs for top commit:
  jamesob:
    ACK 2e31250027
  Sjors:
    utACK 2e31250027
  achow101:
    ACK 2e31250027

Tree-SHA512: 8bcb2d525c95fbc95f87d3e978ad717d95bddb1ff67cbe7d3b06e4783f0f1ffba32b17ef451468c39c23bc1b3ef1150baa71148c145275c386f2d4822d790d39
2023-10-11 14:25:23 -04:00
fanquake
d98d88c779 Merge bitcoin/bitcoin#28392: test: Use pathlib over os path
bfa0bd632a  test: Use pathlib over os.path #28362 (ns-xvrn)

Pull request description:

  In reference to issue  #28362  refactoring of functional tests to use pathlib over os.path to reduce verbosity and increase the intuitiveness of managing file access.

ACKs for top commit:
  maflcko:
    re-ACK  bfa0bd632a  🐨
  willcl-ark:
    ACK bfa0bd632a

Tree-SHA512: fb0833c4039d09758796514e47567a93ac831cb0776ff1a7ed8299792ad132e83282ef80bea098a88ae1551d906f2a56093d3e8de240412f9080735d00d496d9
2023-10-11 20:06:36 +02:00
fanquake
744157ef1a Merge bitcoin/bitcoin#28602: descriptors: Disallow hybrid and uncompressed keys when inferring
74c77825e5 test: Unit test for inferring scripts with hybrid and uncompressed keys (Andrew Chow)
f895f97014 test: Scripts with hybrid pubkeys are migrated to watchonly (Andrew Chow)
37b9b73477 descriptors: Move InferScript's pubkey validity checks to InferPubkey (Andrew Chow)
b7485f11ab descriptors: Check result of InferPubkey (Andrew Chow)

Pull request description:

  `InferDescriptor` was not always checking that the pubkey it was placing into the descriptor was an allowed pubkey. For example, given a P2WPKH script that uses an uncompressed pubkey, it would produce a `wpkh()` with the uncompressed key. Additionally, the hybrid key check was only being done for `pk()` scripts, where it should've been done for all scripts.

  This PR moves the key checking into `InferPubkey`. If the key is not valid for the context, then `nullptr` is returned and the inferring will fall through to the defaults of either `raw()` or `addr()`.

  This also resolves an issue with migrating legacy wallets that contain hybrid pubkeys as such watchonly scripts will become `raw()` or `addr()` and go to the watchonly wallet. Note that a legacy wallet cannot sign for hybrid pubkeys. A test has been added for the migration case.

  Also added unit tests for `InferDescriptor` itself as the edge cases with that function are not covered by the descriptor roundtrip test.

ACKs for top commit:
  furszy:
    ACK 74c77825
  Sjors:
    utACK 74c77825e5

Tree-SHA512: ed5f63e42a2e46120245a6b0288b90d2a6912860814c6c08fe393332add1cb364dc5eca72f16980352143570aef0c07bf1a91acd294099463bd028b6ce2fe40c
2023-10-11 12:50:43 +02:00
fanquake
154404e33f Merge bitcoin/bitcoin#28624: docs: fix typo
57131bfa3c docs: fix typo (vuittont60)

Pull request description:

ACKs for top commit:
  maflcko:
    lgtm ACK 57131bfa3c
  hebasto:
    ACK 57131bfa3c, the `codespell` is powerless to catch this typo.
  jarolrod:
    ACK 57131bfa3c

Tree-SHA512: 816dfc5ff64531ea92acf35feca2286a71c75344df2524ff003a3d375e60100b8531e1678be0ed11863d03ab522d5733d8a0bf4b6f5f79c495a65246fe0b697f
2023-10-11 12:35:48 +02:00
fanquake
bf32ae3969 Merge bitcoin/bitcoin#28482: ci: use LLVM/Clang 17 in tidy job
8735e2c136 ci: use LLVM/Clang 17 in tidy job (fanquake)
ce46b68941 ci: use LLVM 17.0.2 in MSAN jobs (fanquake)

Pull request description:

  Also update MSAN to use 17.0.2.

  Related to #28465.

ACKs for top commit:
  maflcko:
    lgtm ACK 8735e2c136

Tree-SHA512: 74452b95326cf065afe8332dc1b5b8e5ac12c8fe05c278a1cee017f87a7f7e0cdb8cac5e39d718c8ef587c8ee229bbaadd847df9f191313d41c5cdcab45e7c76
2023-10-11 12:29:18 +02:00
Sebastian Falbesoner
3bb51c29df test: BIP324: add check for missing garbage terminator detection 2023-10-11 11:58:07 +02:00
Sjors Provoost
03f82087f6 doc: assumeutxo prune and index notes 2023-10-11 09:20:48 +02:00
Hennadii Stepanov
78d3062b68 ci: Install Android API 31 platform as Qt expects 2023-10-10 19:29:18 +01:00
Brandon Odiwuor
004903ebad test: Add Wallet Unlock Context Manager
Add Context Manager to manage wallet locking/unlocking with passphrase
2023-10-10 18:12:31 +03:00
ns-xvrn
bfa0bd632a test: Use pathlib over os.path #28362
revert netutil chgs py3.8 compliant

fixes based on PR review
2023-10-10 09:26:56 -04:00
fanquake
61a6c3b0e9 build: add -mbranch-protection=bti to aarch64 hardening flags
This is a simpler (less hardening) version of #24123.
Scoped to aarch64 to avoid unused command line option warnings when
building on x86_64.

Related to #19075.
2023-10-10 14:08:27 +01:00
Fabian Jahr
36a3004a41 devtools: test_utxo_snapshots.sh sleep cleanup and documentation 2023-10-10 12:57:54 +02:00
fanquake
092daa2f95 contrib: add test for macOS linker version to symbol-check 2023-10-10 11:22:14 +01:00
fanquake
cefbf0bc20 depends: update LD64_VERSION to 711
I forgot to do this in 7d5815293e.
2023-10-10 11:12:23 +01:00
fanquake
8735e2c136 ci: use LLVM/Clang 17 in tidy job 2023-10-10 11:02:10 +01:00
fanquake
ce46b68941 ci: use LLVM 17.0.2 in MSAN jobs 2023-10-10 10:23:18 +01:00
Sebastian Falbesoner
2e31250027 test: check that loading snapshot not matching AssumeUTXO parameters fails 2023-10-10 09:43:10 +02:00
vuittont60
57131bfa3c docs: fix typo 2023-10-10 15:06:47 +08:00
willcl-ark
4077e43bf6 test: fix usdt undeclared function errors on mantis
Recently usage of undeclared functions became an error rather than a
warning, in C2x. https://reviews.llvm.org/D122983?id=420290

This change has migrated into the build tools of Ubuntu 23.10 which now
causes the USDT tests to fail to compile, see
https://github.com/bitcoin/bitcoin/issues/28600

Fix this by setting `-Wno-error=implicit-function-declaration` for the
tracing programs.
2023-10-09 21:01:38 +01:00
Andrew Chow
04265ba937 Merge bitcoin/bitcoin#26331: Implement CCoinsViewErrorCatcher::HaveCoin and check disk space periodically
ed52e71176 Periodically check disk space to avoid corruption (Aurèle Oulès)
7fe537f7a4 Implement CCoinsViewErrorCatcher::HaveCoin (Aurèle Oulès)

Pull request description:

  Attempt to fix #26112.

  As suggested by sipa in https://github.com/bitcoin/bitcoin/issues/26112#issuecomment-1249683401:
  > CCoinsViewErrorCatcher, the wrapper class used around CCoinsViewDB that's supposed to detect these problems and forcefully exit the application, has an override for GetCoins. But in CheckTxInputs, HaveInputs is first invoked, which on its turn calls HaveCoin. HaveCoin is implemented in CCoinsViewDB, but not in CCoinsViewErrorCatcher, and thus the disk read exception escapes.
  > A solution may be to just add an override for HaveCoin in CCoinsViewErrorCatcher.

  I implemented `CCoinsViewErrorCatcher::HaveCoin` and also added a periodic disk space check that shutdowns the node if there is not enough space left on disk, the minimum here is 50MB.

  For reviewers, it's possible to saturate disk space to test the PR by creating large files with `fallocate -l 50G test.bin`

ACKs for top commit:
  achow101:
    ACK ed52e71176
  w0xlt:
    Code Review ACK ed52e71176
  sipa:
    utACK ed52e71176

Tree-SHA512: 456aa7b996023df42b4fbb5158ee429d9abf7374b7b1ec129b21aea1188ad19be8da4ae8e0edd90b85b7a3042b8e44e17d3742e33808a4234d5ddbe9bcef1b78
2023-10-09 14:52:00 -04:00
Andrew Chow
74c77825e5 test: Unit test for inferring scripts with hybrid and uncompressed keys 2023-10-09 14:07:37 -04:00
Andrew Chow
f895f97014 test: Scripts with hybrid pubkeys are migrated to watchonly
Descriptors disallows hybrid pubkeys. Anything with hybrid pubkeys
should becomes a raw() descriptor that shows up in the watchonly wallet.
2023-10-09 14:07:37 -04:00
Andrew Chow
37b9b73477 descriptors: Move InferScript's pubkey validity checks to InferPubkey 2023-10-09 14:07:37 -04:00
Andrew Chow
b7485f11ab descriptors: Check result of InferPubkey
InferPubkey can return a nullptr, so check it's result before continuing
with creating the inferred descriptor.
2023-10-09 14:07:37 -04:00
MarcoFalke
fa2c894cbb ci: move-only CI_CONTAINER_ID to 02_run_container.sh
This limits the scope of the CI_CONTAINER_ID symbol.

Can be reviewed with --color-moved=dimmed-zebra
2023-10-09 16:17:04 +02:00
MarcoFalke
fa695b4df0 ci: Work around podman stop bug
Force remove any containers, pontentially leaving dangling processes,
which should be fine.
2023-10-09 16:16:55 +02:00
MarcoFalke
fa09a031c1 ci: Add set -ex to 02_run_container.sh
The same is done by the 06 script.
2023-10-09 16:16:41 +02:00
MarcoFalke
fac9abbf47 ci: Rename 04_install to 02_run_container
This reflects what the script does (docker run ...).
2023-10-09 16:16:22 +02:00
fanquake
4e7442e743 Merge bitcoin/bitcoin#28573: github actions: Fix test-one-commit when parent of head is merge commit
88c8e3a0e4 github actions: Fix test-one-commit when parent of head is merge commit (Ryan Ofsky)

Pull request description:

  Instead of figuring out the commit *after* the last merge and rebasing on that with a ~1 suffix, just figure out the last merge commit directly and rebase on it. This way, if HEAD happens to be a merge commit, the rebase just succeeds immediately without blank variables or errors.

  Explanation of the problem from https://github.com/bitcoin/bitcoin/pull/28497#issuecomment-1743430631:

  > The problem is that the PR only contains a one commit after the last merge, so the job _should_ be skipped, but the `pull_request.commits != 1` check is not smart enough to skip it because the PR is based on another PR and has merge ancestor commits. So specifically what happens is that after HEAD~ is checked out, the new HEAD  is a merge commit, so the range `$(git log --merges -1 --format=%H)..HEAD` is equivalent to HEAD..HEAD, which is empty, so the `COMMIT_AFTER_LAST_MERGE` variable is empty and the rebase command fails.

  Note: In the current version of this PR, the "test each commit" job is skipped, because this PR only contains a single commit. But I manually verified the code works in earlier versions of the PR that included dummy commits.

ACKs for top commit:
  maflcko:
    lgtmrecr ACK 88c8e3a0e4
  RandyMcMillan:
    utACK 88c8e3a

Tree-SHA512: a6865b5c8b96eb0b622b3255971a3cf050dd0f5a356cdfcf7f0cbb659e4a363612e8e62b3ae4fd6b5d9a40bc29176891bc4690659b026c5ef8feea25c8e263cc
2023-10-09 15:03:13 +01:00
fanquake
3003861e78 Merge bitcoin/bitcoin#28591: kernel: chainparams updates for 26.x
bd71f03df7 doc: update example pulls in release-process.md (fanquake)
b2ede22395 headerssync: update params for 26.x (fanquake)
f12f92b813 kernel: update m_assumed_* chain params for 26.x (fanquake)
a8c2e5e556 kernel: update chainTxData for 26.x (fanquake)
a9d070a6f8 kernel: update nMinimumChainWork & defaultAssumeValid for 26.x (fanquake)

Pull request description:

  Update chainparams pre `26.x` branch off.

  Note: Remember that some variance is expected in the m_assumed_* sizes.

  Closes #28572.

ACKs for top commit:
  Sjors:
    re-ACK bd71f03df7
  ajtowns:
    reACK bd71f03df7
  darosior:
    re-ACK bd71f03df7 for assumevalid and minchainwork params.

Tree-SHA512: c586b82711477bfc1fd9ac4c9c1130ccde29f75d652492c66ef5730f4a49b7da822a3ad2d9090468ea0c9f0e77d00c25f6a800600c81878d8141ce2ffb3724af
2023-10-09 14:54:08 +01:00
fanquake
f4674444b2 Merge bitcoin/bitcoin#28595: ci: Avoid cache depends/work/build
fac88a874f ci: Avoid cache depends/build (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/28365 (maybe?)

ACKs for top commit:
  hebasto:
    ACK fac88a874f

Tree-SHA512: c121f335f4b7d1f5f8f0508a17cee4d33dcf43c3928f9003a01dcc9ed0c45d1d912f7ff976e605b06c2f293ed21371d2dcace5562fe466bc444144790353bdd3
2023-10-09 14:13:29 +01:00
fanquake
66f8ee2969 Merge bitcoin/bitcoin#28611: Adjust Gradle properties to fix apk build
5f50406554 Adjust Gradle properties (Hennadii Stepanov)

Pull request description:

  On the master branch @ d2b8c5e123, building the `apk` target fails:
  ```
  $ make -C src/qt apk
  ...
  > Task :compileDebugJavaWithJavac FAILED
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/QtActivityDelegate.java:690: error: cannot find symbol
                  Display display = (Build.VERSION.SDK_INT < Build.VERSION_CODES.R)
                                                                                ^
    symbol:   variable R
    location: class VERSION_CODES
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/QtActivityDelegate.java:692: error: cannot find symbol
                          : m_activity.getDisplay();
                                      ^
    symbol:   method getDisplay()
    location: variable m_activity of type Activity
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/QtActivityDelegate.java:833: error: cannot find symbol
          float refreshRate = (Build.VERSION.SDK_INT < Build.VERSION_CODES.R)
                                                                          ^
    symbol:   variable R
    location: class VERSION_CODES
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/QtActivityDelegate.java:835: error: cannot find symbol
                  : m_activity.getDisplay().getRefreshRate();
                              ^
    symbol:   method getDisplay()
    location: variable m_activity of type Activity
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/QtLayout.java:95: error: cannot find symbol
          Display display = (Build.VERSION.SDK_INT < Build.VERSION_CODES.R)
                                                                        ^
    symbol:   variable R
    location: class VERSION_CODES
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/QtLayout.java:97: error: cannot find symbol
                  : ((Activity)getContext()).getDisplay();
                                            ^
    symbol:   method getDisplay()
    location: class Activity
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/ExtractStyle.java:418: error: cannot find symbol
              if (Build.VERSION.SDK_INT < Build.VERSION_CODES.Q)
                                                             ^
    symbol:   variable Q
    location: class VERSION_CODES
  /home/hebasto/git/gui/src/qt/android/src/org/qtproject/qt5/android/ExtractStyle.java:421: error: cannot find symbol
                  numStates = stateList.getStateCount();
                                       ^
    symbol:   method getStateCount()
    location: variable stateList of type StateListDrawable
  Note: Some input files use or override a deprecated API.
  Note: Recompile with -Xlint:deprecation for details.
  8 errors

  FAILURE: Build failed with an exception.

  * What went wrong:
  Execution failed for task ':compileDebugJavaWithJavac'.
  > Compilation failed; see the compiler error output for details.

  * Try:
  Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

  * Get more help at https://help.gradle.org

  Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0.
  Use '--warning-mode all' to show the individual deprecation warnings.
  See https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings

  BUILD FAILED in 827ms
  ...
  ```

  Fixing it by updating the Gradle tool's properties.

ACKs for top commit:
  fanquake:
    ACK 5f50406554 - seems fine.

Tree-SHA512: 52e59fe1c69841370ce2eb670f3618182bf2843582074af4895b8ecb6e5f70dc3fe4eecbffa212efaa534b423ced5b75020f6f09917b52f452121c1e55fbcaac
2023-10-09 14:12:00 +01:00
Hennadii Stepanov
c976657a1b Merge bitcoin-core/gui#767: Update translation source file for v26.0 string freeze
5c9513ece9 qt: Update translation source file for v26.0 string freeze (Hennadii Stepanov)

Pull request description:

  This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to [Release schedule for 26.0](https://github.com/bitcoin/bitcoin/issues/27758).

  Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally.

ACKs for top commit:
  stickies-v:
    ACK 5c9513ece9

Tree-SHA512: 137c636c84525cbfe58d519d416b1f2931c3a56dc212128cf23bd04534ed588f90d38cd5030e3ae239ffccd81f0437aab1a5ebf65a77017561444f3df7becea9
2023-10-09 12:32:39 +01:00
Hennadii Stepanov
5c9513ece9 qt: Update translation source file for v26.0 string freeze
The diff is produced by running `make -C src translate`.
2023-10-09 10:08:52 +01:00
fanquake
62346bc394 Merge bitcoin/bitcoin#28604: test: Use feerate higher than minrelay fee in wallet_fundraw
05af4dfa50 test: Use feerate higher than minrelay fee in wallet_fundraw (Andrew Chow)

Pull request description:

  The external input weight test in wallet_fundrawtransaction.py made transactions at the minimum relay fee. However due to ECDSA sometimes making a shorter signature than expected, the size estimate (and therefore the funded fee) ends up being a little bit too low, which results in the final transaction being under the min relay fee. We can compensate for this by just using a feerate higher than the minrelayfee as the actual feerate itself does not matter in this test.

  Fixes #28437

ACKs for top commit:
  glozow:
    utACK 05af4dfa50, seems right to me

Tree-SHA512: 3e08f052db32d891515d32b27b2d7c5bcbfc77d5ea457eefc75e8aa6d40960278e537c1e8bffe7ddc211949c78e14145a671a8d34e7e1bb15438773cb0d9e89d
2023-10-09 09:53:04 +01:00
Andrew Chow
db283a6b6f Merge bitcoin/bitcoin#27255: MiniTapscript: port Miniscript to Tapscript
ec0fc14a22 miniscript: remove P2WSH-specific part of GetStackSize doc comment (Antoine Poinsot)
128bc104ef qa: bound testing for TapMiniscript (Antoine Poinsot)
117927bd5f miniscript: have a custom Node destructor (Antoine Poinsot)
b917c715ac qa: Tapscript Miniscript signing functional tests (Antoine Poinsot)
5dc341dfe6 qa: list descriptors in Miniscript signing functional tests (Antoine Poinsot)
4f473ea515 script/sign: Miniscript support in Tapscript (Antoine Poinsot)
febe2abc0e MOVEONLY: script/sign: move Satisfier declaration above Tapscript signing (Antoine Poinsot)
bd4b11ee06 qa: functional test Miniscript inside Taproot descriptors (Antoine Poinsot)
8571b89a7f descriptor: parse Miniscript expressions within Taproot descriptors (Antoine Poinsot)
8ff9489422 descriptor: Tapscript-specific Miniscript key serialization / parsing (Antoine Poinsot)
5e76f3f0dd fuzz: miniscript: higher sensitivity for max stack size limit under Tapscript (Antoine Poinsot)
6f529cbaaf qa: test Miniscript max stack size tracking (Antoine Poinsot)
770ba5b519 miniscript: check maximum stack size during execution (Antoine Poinsot)
574523dbe0 fuzz: adapt Miniscript targets to Tapscript (Antoine Poinsot)
84623722ef qa: Tapscript-Miniscript unit tests (Antoine Poinsot)
fcb6f13f44 pubkey: introduce a GetEvenCorrespondingCPubKey helper (Antoine Poinsot)
ce8845f5dd miniscript: account for keys as being 32 bytes under Taproot context (Antoine Poinsot)
f4f978d38e miniscript: adapt resources checks depending on context (Antoine Poinsot)
9cb4c68b89 serialize: make GetSizeOfCompactSize constexpr (Antoine Poinsot)
892436c7d5 miniscript: sanity asserts context in ComputeType (Antoine Poinsot)
e5aaa3d77a miniscript: make 'd:' have the 'u' property under Tapscript context (Antoine Poinsot)
687a0b0fa5 miniscript: introduce a multi_a fragment (Antoine Poinsot)
9164c2eca1 miniscript: restrict multi() usage to P2WSH context (Antoine Poinsot)
91b4db8590 miniscript: store the script context within the Node structure (Antoine Poinsot)
c3738d0344 miniscript: introduce a MsContext() helper to contexts (Antoine Poinsot)
bba9340a94 miniscript: don't anticipate signature presence in CalcStackSize() (Antoine Poinsot)
a3793f2d1a miniscript: add a missing dup key check bypass in Parse() (Antoine Poinsot)

Pull request description:

  Miniscript was targeting P2WSH, and as such can currently only be used in `wsh()` descriptors. This pull request introduces support for Tapscript in Miniscript and makes Miniscript available inside `tr()` descriptors. It adds support for both watching *and* signing TapMiniscript descriptors.

  The main changes to Miniscript for Tapscript are the following:
  - A new `multi_a` fragment is introduced with the same semantics as `multi`. Like in other descriptors `multi` and `multi_a` can exclusively be used in respectively P2WSH and Tapscript.
  - The `d:` fragment has the `u` property under Tapscript, since the `MINIMALIF` rule is now consensus. See also https://github.com/bitcoin/bitcoin/pull/24906.
  - Keys are now serialized as 32 bytes. (Note this affects the key hashes.)
  - The resource consumption checks and calculation changed. Some limits were lifted in Tapscript, and signatures are now 64 bytes long.

  The largest amount of complexity probably lies in the last item. Scripts under Taproot can now run into the maximum stack size while executing a fragment. For instance if you've got a stack size of `999` due to the initial witness plus some execution that happened before and try to execute a `hash256` it would `DUP` (increasing the stack size `1000`), `HASH160` and then push the hash on the stack making the script fail.
  To make sure this does not happen on any of the spending paths of a sane Miniscript, we introduce a tracking of the maximum stack size during execution of a fragment. See the commits messages for details. Those commits were separated from the resource consumption change, and the fuzz target was tweaked to sometimes pad the witness so the script runs on the brink of the stack size limit to make sure the stack size was not underestimated.

  Existing Miniscript unit, functional and fuzz tests are extended with Tapscript logic and test cases. Care was taken for seed stability in the fuzz targets where we cared more about them.

  The design of Miniscript for Tapscript is the result of discussions between various people over the past year(s). To the extent of my knowledge at least Pieter Wuille, Sanket Kanjalkar, Andrew Poelstra and Andrew Chow contributed thoughts and ideas.

ACKs for top commit:
  sipa:
    ACK ec0fc14a22
  achow101:
    ACK ec0fc14a22

Tree-SHA512: f3cf98a3ec8e565650ccf51b7ee7e4b4c2b3949a1168bee16ec03d2942b4d9f20dedc2820457f67a3216161022263573d08419c8346d807a693169ad3a436e07
2023-10-08 12:10:12 -04:00
Hennadii Stepanov
5f50406554 Adjust Gradle properties
This change fixes the `apk` target build after bumping Qt version from
5.15.5 to 5.15.10.
2023-10-08 14:01:59 +01:00
Antoine Poinsot
ec0fc14a22 miniscript: remove P2WSH-specific part of GetStackSize doc comment 2023-10-08 02:43:27 +02:00
Antoine Poinsot
128bc104ef qa: bound testing for TapMiniscript
Make sure we can spend a maximum-sized Miniscript under Tapscript
context.
2023-10-08 02:43:26 +02:00
Antoine Poinsot
117927bd5f miniscript: have a custom Node destructor
To avoid recursive calls in shared_ptr's destructor that could lead to a
stack overflow.
2023-10-08 02:43:26 +02:00
Antoine Poinsot
b917c715ac qa: Tapscript Miniscript signing functional tests 2023-10-08 02:43:25 +02:00
Antoine Poinsot
5dc341dfe6 qa: list descriptors in Miniscript signing functional tests
This makes it more generalistic than just having the miniscripts since
we are going to have Taproot descriptors with (multiple) miniscripts in
them too.
2023-10-08 02:43:24 +02:00
Antoine Poinsot
4f473ea515 script/sign: Miniscript support in Tapscript
We make the Satisfier a base in which to store the common methods
between the Tapscript and P2WSH satisfier, and from which they both
inherit.

A field is added to SignatureData to be able to satisfy pkh() under
Tapscript context (to get the pubkey hash preimage) without wallet data.
For instance in `finalizepsbt` RPC. See also the next commits for a
functional test that exercises this.
2023-10-08 02:43:24 +02:00
Antoine Poinsot
febe2abc0e MOVEONLY: script/sign: move Satisfier declaration above Tapscript signing
We'll need the Miniscript satisfier for Tapscript too.
2023-10-08 02:43:23 +02:00
Antoine Poinsot
bd4b11ee06 qa: functional test Miniscript inside Taproot descriptors 2023-10-08 02:43:23 +02:00
Antoine Poinsot
8571b89a7f descriptor: parse Miniscript expressions within Taproot descriptors 2023-10-08 02:43:22 +02:00
Antoine Poinsot
8ff9489422 descriptor: Tapscript-specific Miniscript key serialization / parsing
64-hex-characters public keys are valid in Miniscript key expressions
within a Tapscript context.

Keys under a Tapscript context always serialize as 32-bytes x-only
public keys (and that's what get hashed by OP_HASH160 on the stack too).
2023-10-08 02:43:22 +02:00
Antoine Poinsot
5e76f3f0dd fuzz: miniscript: higher sensitivity for max stack size limit under Tapscript
In order to exacerbate a mistake in the stack size tracking logic,
sometimes pad the witness to make the script execute at the brink of the
stack size limit. This way if the stack size is underestimated for a
script it would immediately fail `VerifyScript`.
2023-10-08 02:43:21 +02:00
Antoine Poinsot
6f529cbaaf qa: test Miniscript max stack size tracking 2023-10-08 02:43:21 +02:00
Antoine Poinsot
770ba5b519 miniscript: check maximum stack size during execution
Under Tapscript, due to the lifting of some standardness and consensus
limits, scripts can now run into the maximum stack size during
execution. Any Miniscript that may hit the limit on any of its spending
paths must be marked as unsafe.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-10-08 02:43:20 +02:00
Antoine Poinsot
574523dbe0 fuzz: adapt Miniscript targets to Tapscript
We introduce another global that dictates the script context under which
to operate when running the target.

For miniscript_script, just consume another byte to set the context.
This should only affect existing seeds to the extent they contain a
CHECKMULTISIG. However it would not invalidate them entirely as they may
contain a NUMEQUAL or a CHECKSIGADD, and this still exercises a bit of
the parser.

For miniscript_string, reduce the string size by one byte and use the
last byte to determine the context. This is the change that i think
would invalidate the lowest number of existing seeds.

For miniscript_stable, we don't want to invalidate any seed. Instead of
creating a new miniscript_stable_tapscript, simply run the target once
for P2WSH and once for Tapscript (with the same seed).

For miniscript_smart, consume one byte before generating a pseudo-random
node to set the context. We have less regard for seed stability for this
target anyways.
2023-10-08 02:43:20 +02:00
Antoine Poinsot
84623722ef qa: Tapscript-Miniscript unit tests
Adapt the test data and the parsing context to support x-only keys.
Adapt the Test() helper to test existing cases under both Tapscript and
P2WSH context, asserting what needs to be valid or not in each.
Finally, add more cases that exercise the logic that was added in the
previous commits (multi_a, different resource checks and keys
serialization under Tapscript, different properties for 'd:' fragment,
..).
2023-10-08 02:43:19 +02:00
Antoine Poinsot
fcb6f13f44 pubkey: introduce a GetEvenCorrespondingCPubKey helper
We'll need to get a compressed key out of an x-only one in other places.
Avoid duplicating the code.
2023-10-08 02:43:19 +02:00
Antoine Poinsot
ce8845f5dd miniscript: account for keys as being 32 bytes under Taproot context 2023-10-08 02:43:18 +02:00
Antoine Poinsot
f4f978d38e miniscript: adapt resources checks depending on context
Under Tapscript, there is:
- No limit on the number of OPs
- No limit on the script size, it's implicitly limited by the maximum
  (standard) transaction size.
- No standardness limit on the number of stack items, it's limited by
  the consensus MAX_STACK_SIZE. This requires tracking the maximum stack
  size at all times during script execution, which will be tackled in
  its own commit.

In order to avoid any Miniscript that would not be spendable by a
standard transaction because of the size of the witness, we limit the
script size under Tapscript to the maximum standard transaction size
minus the maximum possible witness and Taproot control block sizes. Note
this is a conservative limit but it still allows for scripts more than a
hundred times larger than under P2WSH.
2023-10-08 02:43:17 +02:00
Antoine Poinsot
9cb4c68b89 serialize: make GetSizeOfCompactSize constexpr 2023-10-08 02:43:17 +02:00
Antoine Poinsot
892436c7d5 miniscript: sanity asserts context in ComputeType 2023-10-08 02:43:16 +02:00
Antoine Poinsot
e5aaa3d77a miniscript: make 'd:' have the 'u' property under Tapscript context
In Tapscript MINIMALIF is a consensus rule, so we can rely on the fact
that the `DUP IF [X] ENDIF` will always put an exact 1 on the stack upon
satisfaction.
2023-10-08 02:43:16 +02:00
Antoine Poinsot
687a0b0fa5 miniscript: introduce a multi_a fragment
It is the equivalent of multi() but for Tapscript, using CHECKSIGADD
instead of CHECKMULTISIG.

It shares the same properties as multi() but for 'n', since a threshold
multi_a() may have an empty vector as the top element of its
satisfaction. It could also have the 'o' property when it only has a
single key, but in this case a 'pk()' is always preferable anyways.
2023-10-08 02:43:15 +02:00
Antoine Poinsot
9164c2eca1 miniscript: restrict multi() usage to P2WSH context
CHECKMULTISIG is disabled for Tapscript. Instead, we'll introduce
a multi_a() fragment with the same semantic as multi().
2023-10-08 02:43:15 +02:00
Antoine Poinsot
91b4db8590 miniscript: store the script context within the Node structure
Some checks will be different depending on the script context (for
instance the maximum script size).
2023-10-08 02:43:14 +02:00
Antoine Poinsot
c3738d0344 miniscript: introduce a MsContext() helper to contexts
We are going to introduce Tapscript support in Miniscript, for which
some of Miniscript rules and properties change (new or modified
fragments, different typing rules, different resources consumption, ..).
2023-10-08 02:43:14 +02:00
Antoine Poinsot
bba9340a94 miniscript: don't anticipate signature presence in CalcStackSize()
It's true that for any public key there'll be a signature check in a
valid Miniscript. The code would previously, when computing the size of
a satisfaction, account for the signature when it sees a public key
push. Instead, account for it when it is required (ie when encountering
the `c:` wrapper). This has two benefits:
- Allows to accurately compute the net effect of a fragment on the stack
  size. This is necessary to track the size of the stack during the
  execution of a Script.
- It also just makes more sense, making the code more accessible to
  future contributors.
2023-10-08 02:43:13 +02:00
Antoine Poinsot
a3793f2d1a miniscript: add a missing dup key check bypass in Parse()
This was calling the wrong constructor.
2023-10-08 02:43:13 +02:00
Hennadii Stepanov
d2b8c5e123 Merge bitcoin-core/gui#764: Remove legacy wallet creation
b442580ed2 gui: remove legacy wallet creation (furszy)

Pull request description:

  Fixes #763

  Preventing users from creating a legacy wallet prior to its deprecation in the upcoming releases.

  Note:
  This is still available using the `createwallet` RPC command.

  Future Note:
  Would be nice to re-write this modal as a wizard. And improve the design.

  <details><summary> Pre-Changes Screenshot </summary>
  <img width="611" alt="Screenshot 2023-10-06 at 11 30 14" src="https://github.com/bitcoin-core/gui/assets/5377650/ca10c97d-46e8-4aed-82da-068f2afbe25c">
  </details>

  <details><summary> Post-Changes  Screenshot </summary>
  <img width="729" alt="Screenshot 2023-10-06 at 11 32 58" src="https://github.com/bitcoin-core/gui/assets/5377650/f6bdcb57-646a-43d8-86a7-476e3cca683f">
  </details>

ACKs for top commit:
  achow101:
    ACK b442580ed2
  hebasto:
    re-ACK b442580ed2
  pablomartin4btc:
    tACK b442580ed2

Tree-SHA512: f5d26ffbb0962648b9edf273b325e89425a318e136df26a26acb21b88730fd7d6499c68a705680539dc1b40862fbf413a1e0c8572436a0cfc665e2d08a3cf97d
2023-10-07 11:53:16 +01:00
fanquake
38f4b0d9d1 Merge bitcoin/bitcoin#28562: AssumeUTXO follow-ups
5d227a6862 rpc: Use Ensure(Any)Chainman in assumeutxo related RPCs (Fabian Jahr)
710e5db61b doc: Drop references to assumevalid in assumeutxo docs (Fabian Jahr)
1ff1c34656 test: Rename wait_until_helper to wait_until_helper_internal (Fabian Jahr)
a482f86779 chain: Rename HaveTxsDownloaded to HaveNumChainTxs (Fabian Jahr)
82e48d20f1 blockstorage: Let FlushChainstateBlockFile return true in case of missing cursor (Fabian Jahr)
73700fb554 validation, test: Improve and document nChainTx check for testability (Fabian Jahr)
2c9354facb doc: Add snapshot chainstate removal warning to reindexing documentation (Fabian Jahr)
4e915e926b test: Improvements of feature_assumeutxo (Fabian Jahr)
a47fbe7d49 doc: Add and edit some comments around assumeutxo (Fabian Jahr)
0a39b8cbd8 validation: remove unused mempool param in DetectSnapshotChainstate (Fabian Jahr)

Pull request description:

  Addressing what I consider to be non- or not-too-controversial comments from #27596.

  Let me know if I missed anything among the many comments that can be easily included here.

ACKs for top commit:
  ryanofsky:
    Code review ACK 5d227a6862. Just suggested doc change and new EnsureChainman RPC cleanup commit since last review.

Tree-SHA512: 6f7c762100e18f82946b881676db23e67da7dc3a8bf04e4999a183e90b4f150a0b1202bcb95920ba937a358867bbf2eca300bd84b9b1776c7c490410e707c267
2023-10-07 11:22:40 +01:00
Sebastian Falbesoner
00a52e6394 gui: fix coin control input size accounting for taproot spends 2023-10-07 01:07:21 +02:00
furszy
b442580ed2 gui: remove legacy wallet creation 2023-10-06 17:20:54 -03:00
Fabian Jahr
5d227a6862 rpc: Use Ensure(Any)Chainman in assumeutxo related RPCs 2023-10-06 19:43:32 +02:00
Fabian Jahr
710e5db61b doc: Drop references to assumevalid in assumeutxo docs 2023-10-06 19:43:32 +02:00
Fabian Jahr
1ff1c34656 test: Rename wait_until_helper to wait_until_helper_internal
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2023-10-06 19:43:32 +02:00
Fabian Jahr
a482f86779 chain: Rename HaveTxsDownloaded to HaveNumChainTxs
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2023-10-06 19:43:32 +02:00
Fabian Jahr
82e48d20f1 blockstorage: Let FlushChainstateBlockFile return true in case of missing cursor
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-10-06 19:43:32 +02:00
Fabian Jahr
73700fb554 validation, test: Improve and document nChainTx check for testability
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-10-06 19:43:31 +02:00
Fabian Jahr
2c9354facb doc: Add snapshot chainstate removal warning to reindexing documentation 2023-10-06 19:43:29 +02:00
Fabian Jahr
4e915e926b test: Improvements of feature_assumeutxo
- Remove usage of the internal wait_until_helper function
- Use framework self.no_op instead of new no_sync function

co-authored-by: Andrew Chow <github@achow101.com>
2023-10-06 18:39:02 +02:00
Fabian Jahr
a47fbe7d49 doc: Add and edit some comments around assumeutxo
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-10-06 18:12:31 +02:00
Fabian Jahr
0a39b8cbd8 validation: remove unused mempool param in DetectSnapshotChainstate 2023-10-06 18:11:24 +02:00
fanquake
1472df63f7 Merge bitcoin/bitcoin#28253: test: display abrupt shutdown errors in console output
0f83ab407e test: display abrupt shutdown errors in console output (furszy)

Pull request description:

  Making it easier to debug errors in the CI environment,
  particularly in scenarios where it's not immediately clear
  what happened nor which node crashed (or shutdown abruptly).

  A bit of context:
  Currently, the test framework redirects each node's stderr output
  stream to a different temporary file inside each node's data directory.
  While this is sufficient for storing the error, it isn't very helpful for
  understanding what happened just by reading the CI console output.

  Most of the time, reading the stderr file in the CI environment is not
  possible, because people don't have access to it.

  Testing Note:
  The displayed error difference can be observed by cherry-picking this
  commit 9cc5393c0f on top of this branch and running any
  functional test.

ACKs for top commit:
  maflcko:
    lgtm ACK 0f83ab407e
  theStack:
    ACK 0f83ab407e

Tree-SHA512: 83ce4d21d5316e8cb16a17d3fbe77b8649fced9e09410861d9674c233f6e9c34bcf573504e387e4f439c2841b2ee9855d0d35607fa13aa89eafe0080c45ee82d
2023-10-06 13:51:44 +01:00
fanquake
634b68f0dc Merge bitcoin/bitcoin#28532: qt: enable -ltcg for windows under LTO
f0cebbdb2a qt: enable -ltcg for windows HOST (fanquake)

Pull request description:

  Patch around multiple definition issues in Qt, and enable `-ltcg` when using `LTO=1`.

  Split from #25391.

ACKs for top commit:
  hebasto:
    ACK f0cebbdb2a

Tree-SHA512: 2d6e34779f360bf6dfea4f70fc9004a16e95da79716fcb3046afbf2b01317b7e16965cb51b967b7b5fb64549306c5f48cf59082884289c52016bc1e86949e062
2023-10-06 10:46:11 +01:00
Andrew Chow
05af4dfa50 test: Use feerate higher than minrelay fee in wallet_fundraw
The external input weight test in wallet_fundrawtransaction.py made
transactions at the minimum relay fee. However due to ECDSA sometimes
making a shorter signature than expected, the size estimate (and
therefore the funded fee) ends up being a little bit too low, which
results in the final transaction being under the min relay fee. We can
compensate for this by just using a feerate higher than the minrelayfee
as the actual feerate itself does not matter in this test.
2023-10-05 19:20:30 -04:00
Andrew Chow
54bdb6e074 Merge bitcoin/bitcoin#27609: rpc: allow submitpackage to be called outside of regtest
5b878be742 [doc] add release note for submitpackage (glozow)
7a9bb2a2a5 [rpc] allow submitpackage to be called outside of regtest (glozow)
5b9087a9a7 [rpc] require package to be a tree in submitpackage (glozow)
e32ba1599c [txpackages] IsChildWithParentsTree() (glozow)
b4f28cc345 [doc] parent pay for child in aggregate CheckFeeRate (glozow)

Pull request description:

  Permit (restricted topology) submitpackage RPC outside of regtest. Suggested in https://github.com/bitcoin/bitcoin/pull/26933#issuecomment-1510851570

  This RPC should be safe but still experimental - interface may change, not all features (e.g. package RBF) are implemented, etc. If a miner wants to expose this to people, they can effectively use "package relay" before the p2p changes are implemented. However, please note **this is not package relay**; transactions submitted this way will not relay to other nodes if the feerates are below their mempool min fee. Users should put this behind some kind of rate limit or permissions.

ACKs for top commit:
  instagibbs:
    ACK 5b878be742
  achow101:
    ACK 5b878be742
  dergoegge:
    Code review ACK 5b878be742
  ajtowns:
    ACK 5b878be742
  ariard:
    Code Review ACK  5b878be742. Though didn’t manually test the PR.

Tree-SHA512: 610365c0b2ffcccd55dedd1151879c82de1027e3319712bcb11d54f2467afaae4d05dca5f4b25f03354c80845fef538d3938b958174dda8b14c10670537a6524
2023-10-05 19:08:19 -04:00
Andrew Chow
cf553e3ab7 Merge bitcoin/bitcoin#28597: wallet: No BDB creation, unless -deprecatedrpc=create_bdb
fa071aeb61 wallet: No BDB creation, unless -deprecatedrpc=create_bdb (MarcoFalke)

Pull request description:

  With BDB being removed soon, it seems confusing and harmful to allow users to create fresh BDB wallets going forward, as it would load them with an additional burden of having to migrate them soon after.

  Also, it would be good to allow for one release for test (and external) scripts to adapt.

  Fix all issues by introducing the `-deprecatedrpc=create_bdb` setting.

ACKs for top commit:
  Sjors:
    tACK fa071aeb61
  achow101:
    ACK fa071aeb61
  furszy:
    utACK fa071aeb

Tree-SHA512: 37a4c3e4ba659e0ebe2382e71d9c80e42a895d9ad743f5dda7c110fbbb7d2a36f46769982552a9ac0c3a57203379ef164be97aa8033eb7674d6b4da030ba8f9b
2023-10-05 15:35:54 -04:00
Andrew Chow
0b2c93bc56 Merge bitcoin/bitcoin#28590: assumeutxo: change getchainstates RPC to return a list of chainstates
a9ef702a87 assumeutxo: change getchainstates RPC to return a list of chainstates (Ryan Ofsky)

Pull request description:

  Current `getchainstates` RPC returns "normal" and "snapshot" fields which are not ideal because it requires new "normal" and "snapshot" terms to be defined, and the definitions are not really consistent with internal code. (In the RPC interface, the "snapshot" chainstate becomes the "normal" chainstate after it is validated, while in internal code there is no "normal chainstate" and the "snapshot chainstate" is still called that temporarily after it is validated).

  The current `getchainstates` RPC is also awkward to use if you to want information about the most-work chainstate, because you have to look at the "snapshot" field if it exists, and otherwise fall back to the "normal" field.

  Fix these issues by having `getchainstates` just return a flat list of chainstates ordered by work, and adding a new chainstate "validated" field alongside the existing "snapshot_blockhash" field so it is explicit if a chainstate was originally loaded from a snapshot, and whether the snapshot has been validated.

  This change was motivated by comment thread in https://github.com/bitcoin/bitcoin/pull/28562#discussion_r1344154808

ACKs for top commit:
  Sjors:
    re-ACK a9ef702a87
  jamesob:
    re-ACK a9ef702
  achow101:
    ACK a9ef702a87

Tree-SHA512: b364e2e96675fb7beaaee60c4dff4b69e6bc2d8a30dea1ba094265633d1cddf9dbf1c5ce20c07d6e23222cf1e92a195acf6227e4901f3962e81a1e53a43490aa
2023-10-05 14:19:34 -04:00
Andrew Chow
6e5cf8e953 Merge bitcoin/bitcoin#28587: descriptors: disallow hybrid public keys
c1e6c542af descriptors: disallow hybrid public keys (Pieter Wuille)

Pull request description:

  Fixes #28511

  The descriptor documentation (`doc/descriptors.md`) and [BIP380](https://github.com/bitcoin/bips/blob/master/bip-0380.mediawiki) explicitly require that hex-encoded public keys start with 02 or 03 (compressed) or 04 (uncompressed). However, the current parsing/inference code permit 06 and 07 (hybrid) encoding as well. Fix this.

ACKs for top commit:
  darosior:
    ACK c1e6c542af
  achow101:
    ACK c1e6c542af

Tree-SHA512: 23b674fb420619b2536d12da10008bb87cf7bc0333ec59e618c0d02c3574b468cc71248475ece37f76658d743ef51e68566948e903bca79fda5f7d75416fea4d
2023-10-05 11:58:07 -04:00
fanquake
d9c1cc5f1f Merge bitcoin/bitcoin#28027: test: Fixes and updates to wallet_backwards_compatibility.py for 25.0 and descriptor wallets
afd9a673c4 test: roundtrip wallet backwards compat downgrade (Andrew Chow)
bbf43c63b9 test: Add 25.0 to wallet backwards compatibiilty test (Andrew Chow)
538939ec39 test: Run upgrade test on all nodes (Andrew Chow)
6d4699028b test: Run downgrade test on descriptor wallets (Andrew Chow)
f158573be1 test: Add 0.21 tr() incompatibility test (Andrew Chow)
f41215c3f0 test: add logging 0.17 incompatibilities in wallet back compat (Andrew Chow)
71c03aeff7 test: Refactor v19 addmultisigaddress test to be distinct (Andrew Chow)
53f35d02cb test: Remove w1_v18 from wallet backwards compatibility (Andrew Chow)
313d665437 test: Fix 0.16 wallet paths and downgrade test (Andrew Chow)
5d8469362a test: Add helper functions for checking node versions (Andrew Chow)

Pull request description:

  It was somewhat surprising to me that wallet_backwards_compatibility.py did not catch #27915 since the purpose of the test is to find downgrade issues such as that. It turns out the test was deficient in several places when it came to testing descriptor wallets, as well as deficient in addition to failing to correctly test some releases.

  This PR fixes these test cases, adds more informative logging, slightly refactors the entire test in order to better test future versions, and adds a 25.0 node to the test.

  Notable changes:
  * The compatibility test with 0.16 should not have been passing. The wallets were being copied incorrectly for 0.16 and resulting in 0.16 creating new wallets rather than testing the target wallets.
  * The downgrade test will actually be run on descriptor wallets and it will test that downgrades are successful, and a subsequent upgrade is also successful. This catches #27915.
  * The upgrade and downgrade test will be run on all versions up to master, rather than just 0.16, 0.17, and 0.19.

ACKs for top commit:
  Sjors:
    re-ACK afd9a673c4
  furszy:
    ACK afd9a67

Tree-SHA512: dd2d85cab29a636da93020681c533534af4a9cda18d8550c9db9d8937719b3a225025966981c5d4d2f30486448a772b760f0e723a25ea6bc49df80387dc7b8b0
2023-10-05 16:51:15 +01:00
Andrew Chow
db19a7e89d Merge bitcoin/bitcoin#28403: test: Bump walletpassphrase timeouts to avoid intermittent issues
fa28f5a381 test: Bump walletpassphrase timeouts to avoid intermittent issues (MarcoFalke)

Pull request description:

  This bumps all timeouts for all `walletpassphrase` to avoid intermittent issues in `valgrind` (or other sanitizers).

  As an idea for a follow-up, `walletpassphrase` could be changed to treat `0` as "no timeout" instead of "instant timeout".

  Example failure:

  ```
   node0 2023-09-03T22:44:38.374955Z [httpworker.3] [rpc/server.cpp:594] [RPCRunLater] [rpc] queue run of timer lockwallet(w6) in 60 seconds (using HTTP)
   test  2023-09-03T22:44:40.173000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'getnewaddress', '', 'legacy']
   node0 2023-09-03T22:44:59.810893Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:48928
   node0 2023-09-03T22:44:59.813132Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getnewaddress user=__cookie__
   node0 2023-09-03T22:44:59.837183Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   node0 2023-09-03T22:44:59.929735Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   node0 2023-09-03T22:44:59.934484Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   node0 2023-09-03T22:44:59.935467Z [httpworker.1] [wallet/sqlite.cpp:53] [TraceSqlCallback] [/ci_container_base/ci/scratch/test_runner/test_runner_₿_🏃_20230903_183350/wallet_createwallet_171/node0/regtest/w6/wallet.dat] SQLite Statement: INSERT or REPLACE into main values(?, ?)
   test  2023-09-03T22:45:02.328000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'signmessage', 'mqatqH4VQmrZ81nxUfrnfcLnxgbzhZb4PC', 'test']
   node0 2023-09-03T22:45:20.269375Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:44618
   node0 2023-09-03T22:45:20.270670Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=signmessage user=__cookie__
   test  2023-09-03T22:45:23.490000Z TestFramework.bitcoincli (DEBUG): Running bitcoin-cli ['-rpcwallet=w6', 'keypoolrefill', '1']
   node0 2023-09-03T22:45:40.244603Z [http] [httpserver.cpp:255] [http_request_cb] [http] Received a POST request for /wallet/w6 from 127.0.0.1:32854
   node0 2023-09-03T22:45:40.293021Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=keypoolrefill user=__cookie__
   test  2023-09-03T22:45:41.852000Z TestFramework (ERROR): JSONRPC error
                                     Traceback (most recent call last):
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 131, in main
                                         self.run_test()
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_createwallet.py", line 156, in run_test
                                         w6.keypoolrefill(1)
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 732, in __call__
                                         return self.cli.send_cli(self.command, *args, **kwargs)
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                       File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_node.py", line 795, in send_cli
                                         raise JSONRPCException(dict(code=int(code), message=message))
                                     test_framework.authproxy.JSONRPCException: Error: Please enter the wallet passphrase with walletpassphrase first. (-13)

ACKs for top commit:
  achow101:
    ACK fa28f5a381

Tree-SHA512: 58caa569cec39acc121d4cc038a4190937af34e85d2696272ed4f2792fd386469b0cfefd2cb564438fedded97b21b23d8bf46ba27b5633671a277ed4679f0d5d
2023-10-05 11:18:31 -04:00
fanquake
bd71f03df7 doc: update example pulls in release-process.md 2023-10-05 16:04:23 +01:00
Ryan Ofsky
88c8e3a0e4 github actions: Fix test-one-commit when parent of head is merge commit
Instead of figuring out the commit *after* the last merge and rebasing on that
with a ~1 suffix, just figure out the last merge commit directly and rebase on
it. This way, if HEAD happens to be a merge commit, the rebase just succeeds
immediately without blank variables or errors.

From https://github.com/bitcoin/bitcoin/pull/28497#issuecomment-1743430631:

    The problem is that the PR only contains a one commit after the last merge,
    so the job _should_ be skipped, but the `pull_request.commits != 1` check
    is not smart enough to skip it because the PR is based on another PR and
    has merge ancestor commits. So specifically what happens is that after
    HEAD~ is checked out, the new HEAD is a merge commit, so the range `$(git
    log --merges -1 --format=%H)..HEAD` is equivalent to HEAD..HEAD, which is
    empty, so the `COMMIT_AFTER_LAST_MERGE` variable is empty and the rebase
    command fails.
2023-10-05 11:00:15 -04:00
MarcoFalke
fa071aeb61 wallet: No BDB creation, unless -deprecatedrpc=create_bdb 2023-10-05 15:47:44 +02:00
Ryan Ofsky
a9ef702a87 assumeutxo: change getchainstates RPC to return a list of chainstates
Current getchainstates RPC returns "normal" and "snapshot" fields which are not
ideal because it requires new "normal" and "snapshot" terms to be defined, and
the definitions are not really consistent with internal code. (In the RPC
interface, the "snapshot" chainstate becomes the "normal" chainstate after it
is validated, while in internal code there is no "normal chainstate" and the
"snapshot chainstate" is still called that temporarily after it is validated).

The current getchainstatees RPC is also awkward to use if you to want
information about the most-work chainstate because you have to look at the
"snapshot" field if it exists, and otherwise fall back to the "normal" field.

Fix these issues by having getchainstates just return a flat list of
chainstates ordered by work, and adding new chainstate "validated" field
alongside the existing "snapshot_blockhash" so it is explicit if a chainstate
was originally loaded from a snapshot, and whether the snapshot has been
validated.
2023-10-05 09:41:43 -04:00
Vasil Dimov
53afa68026 net: move MaybeFlipIPv6toCJDNS() from net to netbase
It need not be in the `net` module and we need to call it from
`LookupSubNet()`, thus move it to `netbase`.
2023-10-05 15:10:34 +02:00
Vasil Dimov
6e308651c4 net: move IsReachable() code to netbase and encapsulate it
`vfLimited`, `IsReachable()`, `SetReachable()` need not be in the `net`
module. Move them to `netbase` because they will be needed in
`LookupSubNet()` to possibly flip the result to CJDNS (if that network
is reachable).

In the process, encapsulate them in a class.

`NET_UNROUTABLE` and `NET_INTERNAL` are no longer ignored when adding
or removing reachable networks. This was unnecessary.
2023-10-05 15:10:34 +02:00
Vasil Dimov
c42ded3d9b fuzz: ConsumeNetAddr(): avoid IPv6 addresses that look like CJDNS
The fuzz testing framework runs as if `-cjdnsreachable` is set and in
this case addresses like `{net=IPv6, addr=fc...}` are not possible.
2023-10-05 15:10:33 +02:00
Vasil Dimov
64d6f77907 net: put CJDNS prefix byte in a constant 2023-10-05 15:10:32 +02:00
fanquake
52c6904c78 Merge bitcoin/bitcoin#28558: Make PeerManager own a FastRandomContext
4cafe9f176 [test] Make PeerManager's rng deterministic in tests (dergoegge)
fecec3e1c6 [net processing] FeeFilterRounder doesn't own a FastRandomContext (dergoegge)
47520ed209 [net processing] Make fee filter rounder non-global (dergoegge)
77506f4ac6 [net processing] Addr shuffle uses PeerManager's rng (dergoegge)
a648dd79e5 [net processing] PushAddress uses PeerManager's rng (dergoegge)
87c706713e [net processing] PeerManager holds a FastRandomContext (dergoegge)

Pull request description:

  This lets us avoid some non-determinism in tests (also see #28537).

ACKs for top commit:
  MarcoFalke:
    re-ACK 4cafe9f176  🕗
  glozow:
    concept && light code review ACK 4cafe9f176

Tree-SHA512: 3c18700773d0bc547ccb6442c41567e6f26b0b50fab5b79620da417ec91b9c0ae1395d15258da3aa4a91447b8ce560145dd135e39fbbd0610749e528e665b111
2023-10-05 14:06:39 +01:00
furszy
0f83ab407e test: display abrupt shutdown errors in console output
Making it easier to debug errors in the CI environment,
particularly in scenarios where it's not immediately clear
what happened nor which node crashed (or shutdown abruptly).
2023-10-05 09:45:09 -03:00
Vasil Dimov
5c8e15c451 i2p: destroy the session if we get an unexpected error from the I2P router
From https://geti2p.net/en/docs/api/samv3:

  If SILENT=false was passed, which is the default value, the SAM bridge
  sends the client a ASCII line containing the base64 public destination
  key of the requesting peer

So, `Accept()` is supposed to receive a Base64 encoded destination of
the connecting peer, but if it receives something like this instead:

  STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed"

then destroy the session.
2023-10-05 14:11:13 +02:00
Vasil Dimov
762404a68c i2p: also sleep after errors in Accept()
Background:

`Listen()` does:
* if the session is not created yet
  * create the control socket and on it:
  * `HELLO`
  * `SESSION CREATE ID=sessid`
  * leave the control socked opened
* create a new socket and on it:
* `HELLO`
* `STREAM ACCEPT ID=sessid`
* read reply (`STREAM STATUS`)

Then a wait starts, for a peer to connect. When connected,

`Accept()` does:
* on the socket from `STREAM ACCEPT` from `Listen()`: read the
  Base64 identification of the connecting peer

Problem:

The I2P router may be in such a state that this happens in a quick
succession (many times per second, see https://github.com/bitcoin/bitcoin/issues/22759#issuecomment-1609907115):
`Listen()`-succeeds, `Accept()`-fails.

`Accept()` fails because the I2P router sends something that is
not Base64 on the socket:
STREAM STATUS RESULT=I2P_ERROR MESSAGE="Session was closed"

We only sleep after failed `Listen()` because the assumption was that
if `Accept()` fails then the next `Listen()` will also fail.

Solution:

Avoid filling the log with "Error accepting:" messages and sleep also
after a failed `Accept()`.

Extra changes:

* Reset the error waiting time after one successful connection.
  Otherwise the timer will remain high due to problems that have
  vanished long time ago.

* Increment the wait time less aggressively.
2023-10-05 14:10:43 +02:00
MarcoFalke
fac88a874f ci: Avoid cache depends/build 2023-10-05 13:45:04 +02:00
fanquake
78fd3c2672 Merge bitcoin/bitcoin#28588: test: BIP324: add checks for v1 prefix matching / wrong network magic detection
e1308967e1 test: BIP324: add checks for v1 prefix matching / wrong network magic detection (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the detection of incoming v1 connections and wrong network magic on BIP324-enabled (i.e. running with `-v2transport=1`) nodes. Both checks are using prefix sizes of 16 bytes (previously only 12 bytes were used for the v1 prefix matching, which was fixed by PR #28577).

ACKs for top commit:
  Sjors:
    utACK e1308967e1
  MarcoFalke:
    lgtm ACK e1308967e1

Tree-SHA512: d4d1567277297f42c543b9638a6c64d14b17ff0ddbf85a7efff22f45c619478139dbedcb9dc4f449b4814b00856ee43644f15df1aa20c8931d5496a607ca2fd4
2023-10-05 12:02:51 +01:00
MarcoFalke
fa28f5a381 test: Bump walletpassphrase timeouts to avoid intermittent issues 2023-10-05 12:57:33 +02:00
fanquake
b2ede22395 headerssync: update params for 26.x 2023-10-05 11:36:03 +01:00
fanquake
f12f92b813 kernel: update m_assumed_* chain params for 26.x 2023-10-05 11:29:42 +01:00
fanquake
a8c2e5e556 kernel: update chainTxData for 26.x 2023-10-05 11:29:41 +01:00
fanquake
a9d070a6f8 kernel: update nMinimumChainWork & defaultAssumeValid for 26.x 2023-10-05 11:29:41 +01:00
fanquake
2eacc61ad7 Merge bitcoin/bitcoin#25970: Add headerssync tuning parameters optimization script to repo
3d420d8f28 Add instructions for headerssync-params.py to release-process.md (Pieter Wuille)
53d7d35b58 Update parameters in headerssync.cpp (Pieter Wuille)
7899402cff Add headerssync-params.py script to the repository (Pieter Wuille)

Pull request description:

  Builds upon #25946, as it incorporates changes based on the selected values there.

  This adds the headerssync tuning parameters optimization script from https://gist.github.com/sipa/016ae445c132cdf65a2791534dfb7ae1 to the repository, updates the parameters based on its output, and adds release process instructions for doing this update in the future.

  A few considerations:
  * It would be a bit cleaner to have these parameters be part of `CChainParams`, but due to the nature of the approach, it really only applies to chains with unforgeable proof-of-work, which we really can only reasonably expect from mainnet, so I think it's fine to keep them local to `headerssync.cpp`. Keeping them as compile-time evaluatable constants also has a (likely negligible) performance impact (avoiding runtime modulo operations).
  * If we want to make sure the chainparams and headerssync params don't go out of date, it could be possible to run the script in CI, and and possibly even have the parameters be generated automatically at build time. I think that's overkill for how unfrequently these need to change, and running the script has non-trivial cost (~minutes in the normal python interpreter).
  * A viable alternative is just leaving this out-of-repo entirely, and just do ad-hoc updating from time to time. Having it in the repo and release notes does make sure it's not forgotten, though adds a cost to contributors/maintainers who follow the process.

ACKs for top commit:
  ajtowns:
    reACK 3d420d8f28

Tree-SHA512: 03188301c20423c72c1cbd008ccce89b93e2898edcbeecc561b2928a0d64e9a829ab0744dc3b017c23de8b02f3c107ae31e694302d3931f4dc3540e184de1963
2023-10-05 11:28:29 +01:00
fanquake
f0cebbdb2a qt: enable -ltcg for windows HOST
Patch around multiple definition issues in Qt.

Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-10-05 10:26:34 +01:00
Hennadii Stepanov
0e3de3b83e Merge bitcoin-core/gui#754: Add BIP324-specific labels to peer details
d9c4e344d7 qt: Add "Session id" label to peer details (Hennadii Stepanov)
f08adec886 qt: Add "Transport" label to peer details (Hennadii Stepanov)

Pull request description:

  This PR adds BIP324-specific labels to the peer details widget:
  -  a transport version
  - a session id

  See: https://github.com/bitcoin/bitcoin/pull/28331#issuecomment-1693239025.

  ![image](https://github.com/bitcoin-core/gui/assets/32963518/0e0b4c92-dde0-4b2e-b285-a2c69ef09efc)

ACKs for top commit:
  achow101:
    ACK d9c4e344d7
  RandyMcMillan:
    ACK d9c4e34
  theStack:
    Tested re-ACK d9c4e344d7
  MarnixCroes:
    tACK d9c4e344d7

Tree-SHA512: 524e634b1eedcae535d5c2a10dd8dea90d33d65d6790614f86ab6387a0ec9ee4bbc7646b8c20a5b3f1bccbb69bc52a46514e2b28cb4588979834d945b1f89b3a
2023-10-05 08:36:49 +01:00
Sebastian Falbesoner
e1308967e1 test: BIP324: add checks for v1 prefix matching / wrong network magic detection 2023-10-05 00:00:09 +02:00
Andrew Chow
5b4478418b Merge bitcoin/bitcoin#28577: net: raise V1_PREFIX_LEN from 12 to 16
ba2e5bfc67 net: raise V1_PREFIX_LEN from 12 to 16 (Pieter Wuille)

Pull request description:

  A "version" message in the V1 protocol starts with a fixed 16 bytes:
  * The 4-byte network magic
  * The 12-byte command string: "version" plus 5 0x00 bytes

  The current code detects incoming V1 connections by just looking at the first 12 bytes (matching an [earlier version](https://github.com/bitcoin/bips/pull/1496) of BIP324), but 16 bytes is more precise. This isn't an observable difference right now, as a 12 byte prefix ought to be negligible already, but it may become observable with future extensions to the protocol, so make the code match the specification.

ACKs for top commit:
  achow101:
    ACK ba2e5bfc67
  theStack:
    re-ACK ba2e5bfc67
  mzumsande:
    Code review ACK ba2e5bfc67

Tree-SHA512: 64876b03613bd1c5dda82f4ca1b367014365f9ae4cfa30f45c5758a563c68cbea81a98d02ba616c264674c204517aac8b7de94da10f32e77b56267a43710c651
2023-10-04 16:57:32 -04:00
Hennadii Stepanov
8369467c94 Merge bitcoin-core/gui#757: Add wallet name to address book page title
58c9b50a95 gui: Add wallet name to address book page (pablomartin4btc)

Pull request description:

  It fixes bitcoin-core/gui#756.

  Each address book page window it's now labeled with the wallet name they were opened with, so the user can easily identify which addresses belong to which wallet even when there are many windows opened. It's a helpful enhancement for users managing multiple wallets.

  ![image](https://github.com/bitcoin-core/gui/assets/110166421/628e37bb-87b9-42fb-9158-bffdd8428bcb)

ACKs for top commit:
  hebasto:
    ACK 58c9b50a95, tested on Ubuntu 22.04.

Tree-SHA512: 82febc020653560281da144cd35c672c49ca9f48b23d173eb19395e9ab4d045500295a9b5f24c82efdbf6e7ea70c87e733207cb6a31d3f84828b27e3b2df558b
2023-10-04 21:36:57 +01:00
pablomartin4btc
58c9b50a95 gui: Add wallet name to address book page
Extend addresstablemodel to return the display name from the wallet and
set it to the addressbookpage window title when its  model is set.
2023-10-04 17:01:49 -03:00
Andrew Chow
ab163b0fb5 Merge bitcoin/bitcoin#27823: init: return error when block index is non-contiguous, fix feature_init.py file perturbation
d27b9a2248 test: fix feature_init.py file perturbation (Martin Zumsande)
ad66ca1e47 init: abort loading of blockindex in case of missing height. (Martin Zumsande)

Pull request description:

  When the block index database is non-contiguous due to file corruption (i.e. it contains indexes of height `x-1` and `x+1`, but not `x`), bitcoind can currently crash with an assert in `BuildSkip()` / `GetAncestor()` during `BlockManager::LoadBlockIndex()`:
  ```
  bitcoind: chain.cpp:112: const CBlockIndex* CBlockIndex::GetAncestor(int) const: Assertion `pindexWalk->pprev' failed.
  ```
  This PR changes it such that we instead return an `InitError` to the user.

  I stumbled upon this because I noticed that the file perturbation in `feature_init.py`  wasn't working as intended, which is fixed in the second commit:
  * Opening the file twice in one `with` statement would lead to `tf_read` being empty, so the test wouldn't perturb anything but replace the file with a new one. Fixed by first opening for read, then for write.
  * We need to restore the previous state after perturbations, so that only the current perturbation is active and not a mix of the current and previous ones.
  * I also added `checkblocks=200` to the startup parameters so that corruption in earlier blocks of `blk00000.dat` is detected during init verification and not ignored.

  After fixing `feature_init.py` like that I'd run into the `assert` mentioned above (so running the testfix from the second commit without the first one is a way to reproduce it).

ACKs for top commit:
  achow101:
    ACK d27b9a2248
  furszy:
    Code ACK d27b9a224
  fjahr:
    Code review ACK d27b9a2248

Tree-SHA512: 2e54da6030c5813c86bd58f816401e090bb43c5b834764a5e3c0e55dbfe09e423f88042cab823db3742088204b274d4ad2abf58a3832a4b18328b11a30bf7094
2023-10-04 15:36:57 -04:00
fanquake
30b3477507 Merge bitcoin/bitcoin#28561: build: Update qt package up to 5.15.10
6988a2f097 build: Update qt package up to 5.15.10 (Hennadii Stepanov)

Pull request description:

  The Qt 5.15.10 contains at least three important for us fixes:
  - 8bb90ab760, which allows us to drop the [`dont_hardcode_x86_64.patch`](fd8ab08558/depends/patches/qt/dont_hardcode_x86_64.patch)
  - 8467beddb7, which allows us to drop the [`fix_montery_include.patch`](fd8ab08558/depends/patches/qt/fix_montery_include.patch)
  - df08a21fa4, which addresses https://github.com/bitcoin/bitcoin/pull/28349#issuecomment-1743519614

ACKs for top commit:
  fanquake:
    ACK 6988a2f097.

Tree-SHA512: 838c44cf0d7508714f35887bb1f0983e59d96764f08f467bbd03d102d7152e797daeedc178afb03141115cc3be623e4ef7c4c4c65c0a8d1f0bb9702c9a6f8fc6
2023-10-04 18:37:31 +01:00
Andrew Chow
cc68a3b9dd Merge bitcoin/bitcoin#28589: test: assumeutxo func test race fixes
7e40032260 tests: assumeutxo: accept final height from either chainstate (James O'Beirne)
5bd2010f02 test: assumeutxo: avoid race in functional test (James O'Beirne)
7005a01c19 test: add wait_for_connect to BitcoinTestFramework.connect_nodes (James O'Beirne)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/28585.

  Fixes a few races within the assumeutxo tests:
  - In general, `-stopatheight` can't be used with `connect_nodes` safely because the latter performs blocking assertions that are racy with the stopatheight triggering.
  - Now that the snapshot chainstate is listed as `normal` after background validation, accept the final height from either chainstate.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 7e40032260
  fjahr:
    Code review ACK 7e40032260
  achow101:
    ACK 7e40032260
  ryanofsky:
    Code review ACK 7e40032260

Tree-SHA512: 8cbd2a0ca8643f94baa0ae3561dcf68c3519d5ba851c6049e1768f28cae6434f47ffc28d404bf38ed11030ce3f00aae0a8be3f6d563e6ae6680d83c928a173d8
2023-10-04 13:32:40 -04:00
fanquake
3cd02806ec Merge bitcoin/bitcoin#28567: ci: Only run functional tests on native windows in master
aba4a5887b ci: Only run functional tests on windows in master (Fabian Jahr)

Pull request description:

  This idea was discussed [here](https://github.com/bitcoin/bitcoin/pull/28509#issuecomment-1740841988).

ACKs for top commit:
  hebasto:
    ACK aba4a5887b

Tree-SHA512: 89fd6352b585bae3538d5350b0404c216a8225fe356d408c1ebe3394e7b9a190d65639f4eef310056e020909928d7a1f2de25585c97d2ac087d1a9f72af281eb
2023-10-04 17:13:15 +01:00
Fabian Jahr
aba4a5887b ci: Only run functional tests on windows in master 2023-10-04 17:48:13 +02:00
James O'Beirne
7e40032260 tests: assumeutxo: accept final height from either chainstate 2023-10-04 11:32:36 -04:00
Pieter Wuille
c1e6c542af descriptors: disallow hybrid public keys
The descriptor documentation (doc/descriptors.md) and BIP380 explicitly
require that hex-encoded public keys start with 02 or 03 (compressed) or
04 (uncompressed). However, the current parsing/inference code permit 06
and 07 (hybrid) encoding as well. Fix this.
2023-10-04 11:28:13 -04:00
James O'Beirne
5bd2010f02 test: assumeutxo: avoid race in functional test 2023-10-04 11:05:27 -04:00
James O'Beirne
7005a01c19 test: add wait_for_connect to BitcoinTestFramework.connect_nodes 2023-10-04 11:05:03 -04:00
Pieter Wuille
ba2e5bfc67 net: raise V1_PREFIX_LEN from 12 to 16
A "version" message in the V1 protocol starts with a fixed 16 bytes:
* The 4-byte network magic
* The 12-byte zero-padded command "version" plus 5 0x00 bytes

The current code detects incoming V1 connections by just looking at the
first 12 bytes (matching an earlier version of BIP324), but 16 bytes is
more precise. This isn't an observable difference right now, as a 12 byte
prefix ought to be negligible already, but it may become observable with
future extensions to the protocol, so make the code match the
specification.
2023-10-04 11:04:43 -04:00
fanquake
058488276f Merge bitcoin/bitcoin#27598: bench: Add SHA256 implementation specific benchmarks
ce6df7df9b bench: Add SHA256 implementation specific benchmarks (Hennadii Stepanov)
5f72417176 Add ability to specify SHA256 implementation for benchmark purposes (Hennadii Stepanov)

Pull request description:

  On the master branch, only the best available `SHA256` implementation is being benchmarked. This PR makes `bench_bitcoin` benchmark all `SHA256` implementations that are available on the system.

  For  example:
  - on Linux:
  ```
  $ ./src/bench/bench_bitcoin -filter=SHA.*
  Using the 'x86_shani(1way,2way)' SHA256 implementation

  |             ns/byte |              byte/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |                1.00 |    1,002,545,462.93 |    0.4% |      0.01 | `SHA1`
  |                2.91 |      344,117,991.18 |    0.1% |      0.03 | `SHA256 using the 'standard' SHA256 implementation`
  |                2.21 |      453,081,794.40 |    0.1% |      0.02 | `SHA256 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
  |                2.21 |      453,396,506.58 |    0.1% |      0.02 | `SHA256 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
  |                0.53 |    1,870,520,687.49 |    0.1% |      0.01 | `SHA256 using the 'x86_shani(1way,2way)' SHA256 implementation`
  |                7.90 |      126,627,134.33 |    0.0% |      0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation`
  |                3.94 |      253,850,206.07 |    0.0% |      0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way)' SHA256 implementation`
  |                1.40 |      716,247,553.38 |    0.4% |      0.01 | `SHA256D64_1024 using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
  |                1.26 |      792,706,270.13 |    0.9% |      0.01 | `SHA256D64_1024 using the 'x86_shani(1way,2way)' SHA256 implementation`
  |                6.75 |      148,172,097.64 |    0.2% |      0.01 | `SHA256_32b using the 'standard' SHA256 implementation`
  |                4.90 |      204,156,289.96 |    0.1% |      0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way)' SHA256 implementation`
  |                4.90 |      204,101,274.22 |    0.1% |      0.01 | `SHA256_32b using the 'sse4(1way),sse41(4way),avx2(8way)' SHA256 implementation`
  |                1.70 |      589,052,595.35 |    0.4% |      0.01 | `SHA256_32b using the 'x86_shani(1way,2way)' SHA256 implementation`
  |                2.21 |      453,441,736.14 |    1.0% |      0.02 | `SHA3_256_1M`
  |                1.92 |      521,807,101.48 |    1.0% |      0.02 | `SHA512`
  ```

  - on macOS (M1):
  ```
  % ./src/bench/bench_bitcoin -filter=SHA.\*
  Using the 'arm_shani(1way,2way)' SHA256 implementation

  |             ns/byte |              byte/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |                1.36 |      737,644,274.00 |    0.6% |      0.02 | `SHA1`
  |                3.08 |      324,556,777.15 |    0.2% |      0.03 | `SHA256 using the 'standard' SHA256 implementation`
  |                0.45 |    2,198,104,135.18 |    0.3% |      0.01 | `SHA256 using the 'arm_shani(1way,2way)' SHA256 implementation`
  |                8.84 |      113,131,299.18 |    0.0% |      0.01 | `SHA256D64_1024 using the 'standard' SHA256 implementation`
  |                0.94 |    1,059,406,239.36 |    0.0% |      0.01 | `SHA256D64_1024 using the 'arm_shani(1way,2way)' SHA256 implementation`
  |                6.17 |      162,050,659.51 |    0.2% |      0.01 | `SHA256_32b using the 'standard' SHA256 implementation`
  |                1.15 |      866,637,155.98 |    0.0% |      0.01 | `SHA256_32b using the 'arm_shani(1way,2way)' SHA256 implementation`
  |                1.69 |      592,636,491.59 |    0.2% |      0.02 | `SHA3_256_1M`
  |                1.89 |      528,785,775.66 |    0.0% |      0.02 | `SHA512`
  ```

  Found it useful, while working on https://github.com/bitcoin/bitcoin/pull/24773.

ACKs for top commit:
  martinus:
    ACK ce6df7df9b. I would have created a helper function in the test to avoid the code duplication for each test, but that's just me nitpicking. Here are results from my Ryzen 7950X, with `./src/bench/bench_bitcoin -filter="SHA256.*" -min-time=1000`:
  MarcoFalke:
    review ACK ce6df7df9b 🏵
  sipa:
    ACK ce6df7df9b

Tree-SHA512: e3de50e11b9a3a0d1e05583786041d4dc9afa2022e2115d75d6d1f63b11f62f6336f093001e53a631431d558c4dae29c596755c9e2d6aa78c382270116cc1f7f
2023-10-04 16:04:07 +01:00
Hennadii Stepanov
6988a2f097 build: Update qt package up to 5.15.10 2023-10-04 14:00:57 +01:00
dergoegge
4cafe9f176 [test] Make PeerManager's rng deterministic in tests 2023-10-04 13:16:53 +01:00
dergoegge
fecec3e1c6 [net processing] FeeFilterRounder doesn't own a FastRandomContext 2023-10-04 13:16:52 +01:00
dergoegge
47520ed209 [net processing] Make fee filter rounder non-global 2023-10-04 13:14:05 +01:00
Hennadii Stepanov
d9c4e344d7 qt: Add "Session id" label to peer details 2023-10-04 12:07:21 +01:00
fanquake
db7b5dfcc5 Merge bitcoin/bitcoin#28551: http: bugfix: allow server shutdown in case of remote client disconnection
68f23f57d7 http: bugfix: track closed connection (stickies-v)
084d037231 http: log connection instead of request count (stickies-v)
41f9027813 http: refactor: use encapsulated HTTPRequestTracker (stickies-v)

Pull request description:

  #26742 significantly increased the http server shutdown speed, but also introduced a bug (#27722 - see https://github.com/bitcoin/bitcoin/issues/27722#issuecomment-1559453982 for steps to reproduce on master) that causes http server shutdown to halt in case of a remote client disconnection. This happens because `evhttp_request_set_on_complete_cb` is never called and thus the request is never removed from `g_requests`.

  This PR fixes that bug, and improves robustness of the code by encapsulating the request tracking logic. Earlier approaches (#27909, #27245, #19434) attempted to resolve this but [imo are fundamentally unsafe](https://github.com/bitcoin/bitcoin/pull/27909#discussion_r1265614783) because of differences in lifetime between an `evhttp_request` and `evhttp_connection`.

  We don't need to keep track of open requests or connections, we just [need to ensure](https://github.com/bitcoin/bitcoin/pull/19420#issue-648067169) that there are no active requests on server shutdown. Because a connection can have multiple requests, and a request can be completed in various ways (the request actually being handled, or the client performing a remote disconnect), keeping a counter per connection seems like the approach with the least overhead to me.

  Fixes #27722

ACKs for top commit:
  vasild:
    ACK 68f23f57d7
  theStack:
    ACK 68f23f57d7

Tree-SHA512: dfa711ff55ec75ba44d73e9e6fac16b0be25cf3c20868c2145a844a7878ad9fc6998d9ff62d72f3a210bfa411ef03d3757b73d68a7c22926e874c421e51444d6
2023-10-04 10:09:03 +01:00
fanquake
2f835d2ae4 Merge bitcoin/bitcoin#28576: docs: fix typo in packages.md
bdee858964 typo: in packages.md (Erik McKelvey)

Pull request description:

  Removed extra word `the` in packages.md

ACKs for top commit:
  fanquake:
    ACK bdee858964

Tree-SHA512: 14a745e5f8ad97f38c21c7b80e88592f84f50d87bc71930c1212fb9ba5a46b129ffec0aa1dd53780f872c0067b58bd2a65ed9def4d9f5f50dc8c7d8e34a429d4
2023-10-04 09:53:44 +01:00
stratospher
e6e444c06c refactor: add and use EnsureAnyAddrman in rpc 2023-10-04 08:53:51 +05:30
stratospher
bf589a50a0 doc: add release notes for #27511 2023-10-04 08:34:32 +05:30
stratospher
3931e6abc3 rpc: getaddrmaninfo followups
- make `getaddrmaninfo` RPC public since it's not for development
  purposes only and regular users might find it useful
- add missing `all_networks` key to RPC help
- use clang format spacing
2023-10-04 08:34:30 +05:30
Erik McKelvey
bdee858964 typo: in packages.md 2023-10-03 11:22:46 -07:00
fanquake
97f756b12c Merge bitcoin/bitcoin#28575: ci: Print Linux kernel info
fac054d24c ci: Print Linux kernel info (MarcoFalke)

Pull request description:

  Required to debug issues like https://github.com/bitcoin/bitcoin/pull/28487#issuecomment-1729717923. For example:

  ```
  FATAL: ThreadSanitizer: unexpected memory mapping 0x57cf8f031000-0x57cf8f173000

ACKs for top commit:
  hebasto:
    ACK fac054d24c

Tree-SHA512: 7eb158e52daffe5cbcdfa3ed1efb45e1930b80a2672558fe400c8d72ce59a8cbeb02296dfc2032721d511410885a1f057672fe8086ba1c72a494aef541bf7eb4
2023-10-03 16:57:11 +01:00
Andrew Chow
01bd9d7b99 Merge bitcoin/bitcoin#28523: rpc: add hidden getrawaddrman RPC to list addrman table entries
352d5eb2a9 test: getrawaddrman RPC (0xb10c)
da384a286b rpc: getrawaddrman for addrman entries (0xb10c)

Pull request description:

  Inspired by `getaddrmaninfo` (#27511), this adds a hidden/test-only `getrawaddrman` RPC. The RPC returns information on all addresses in the address manager new and tried tables. Addrman table contents can be used in tests and during development.

  The RPC result encodes the `bucket` and `position`, the internal location of addresses in the tables, in the address object's string key. This allows users to choose to consume or to ignore the location information. If the internals of the address manager implementation change, the location encoding might change too.

  ```
  getrawaddrman

  EXPERIMENTAL warning: this call may be changed in future releases.

  Returns information on all address manager entries for the new and tried tables.

  Result:
  {                                  (json object)
    "table" : {                      (json object) buckets with addresses in the address manager table ( new, tried )
      "bucket/position" : {          (json object) the location in the address manager table (<bucket>/<position>)
        "address" : "str",           (string) The address of the node
        "port" : n,                  (numeric) The port number of the node
        "network" : "str",           (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the address
        "services" : n,              (numeric) The services offered by the node
        "time" : xxx,                (numeric) The UNIX epoch time when the node was last seen
        "source" : "str",            (string) The address that relayed the address to us
        "source_network" : "str"     (string) The network (ipv4, ipv6, onion, i2p, cjdns) of the source address
      },
      ...
    },
    ...
  }

  Examples:
  > bitcoin-cli getrawaddrman
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "getrawaddrman", "params": []}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  willcl-ark:
    reACK 352d5eb2a9
  amitiuttarwar:
    reACK 352d5eb2a9
  stratospher:
    reACK 352d5eb.
  achow101:
    ACK 352d5eb2a9

Tree-SHA512: cc462666b5c709617c66b0e3e9a17c4c81e9e295f91bdd9572492d1cb6466fc9b6d48ee805ebe82f9f16010798370effe5c8f4db15065b8c7c0d8637675d615e
2023-10-03 11:38:20 -04:00
fanquake
5a4eb56e3f Merge bitcoin/bitcoin#28571: depends: fix unusable memory_resource in macos qt build
848eec0936 depends: fix unusable memory_resource in macos qt build (fanquake)

Pull request description:

  See https://codereview.qt-project.org/c/qt/qtbase/+/482392.

  Fixes #28566.

ACKs for top commit:
  hebasto:
    ACK 848eec0936.

Tree-SHA512: dd902f7abb09bda3800d78fe58937b4426d974c24ba321b979eba0d6da30fa0c661b4ed629afab827df8f9ab599efc7a288e9f381ec2b3c69d1063d4d4f73f9e
2023-10-03 15:54:10 +01:00
MarcoFalke
fac054d24c ci: Print Linux kernel info 2023-10-03 16:37:43 +02:00
Ryan Ofsky
d0b928b29d Merge bitcoin/bitcoin#26312: Remove Sock::Get() and Sock::Sock()
7df4508369 test: improve sock_tests/move_assignment (Vasil Dimov)
5086a99b84 net: remove Sock default constructor, it's not necessary (Vasil Dimov)
7829272f78 net: remove now unnecessary Sock::Get() (Vasil Dimov)
944b21b70a net: don't check if the socket is valid in ConnectSocketDirectly() (Vasil Dimov)
aeac68d036 net: don't check if the socket is valid in GetBindAddress() (Vasil Dimov)
5ac1a51ee5 i2p: avoid using Sock::Get() for checking for a valid socket (Vasil Dimov)

Pull request description:

  _This is a piece of #21878, chopped off to ease review._

  Peeking at the underlying socket file descriptor of `Sock` and checkig if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of testing/mocking/fuzzing.

  Instead use an empty `unique_ptr` to denote that there is no valid socket where appropriate or outright remove such checks where they are not necessary.

  The default constructor `Sock::Sock()` is unnecessary now after recent changes, thus remove it.

ACKs for top commit:
  ajtowns:
    ACK 7df4508369
  jonatack:
    ACK 7df4508369

Tree-SHA512: 9742aeeeabe8690530bf74caa6ba296787028c52f4a3342afd193b05dbbb1f6645935c33ba0a5230199a09af01c666bd3c7fb16b48692a0d185356ea59a8ddbf
2023-10-03 09:57:46 -04:00
Hennadii Stepanov
88e5a02b8b Merge bitcoin-core/gui#751: macOS, do not process actions during shutdown
bae209e387 gui: macOS, make appMenuBar part of the main app window (furszy)
e14cc8fc69 gui: macOS, do not process dock icon actions during shutdown (furszy)

Pull request description:

  As the 'QMenuBar' is created without a parent window in MacOS, the app crashes when the user presses the shutdown button and, right after it, triggers any action in the menu bar.

  This happens because the QMenuBar is manually deleted in the BitcoinGUI destructor but the events attached to it children actions are not disconnected, so QActions events such us the 'QMenu::aboutToShow' could try to access null pointers.

  Instead of guarding every single QAction pointer inside the QMenu::aboutToShow slot, or manually disconnecting all registered events in the destructor, we can check if a shutdown was requested and discard the event.

  The 'node' field is a ref whose memory is held by the main application class, so it is safe to use here. Events are disconnected prior destructing the main application object.

  Furthermore, the 'MacDockIconHandler::dockIconClicked' signal can make the app crash during shutdown for the very same reason. The 'show()' call triggers the 'QApplication::focusWindowChanged' event, which is connected to the 'minimize_action' QAction, which is also part of the app menu bar, which could no longer exist.

  Another cause of crashes stems from the shortcuts provided by the `appMenuBar` submenus during shutdown. For instance, executing actions like opening the information dialog (command + I) or the console dialog (command + T) lead to access null pointers. The second commit addresses and resolves these issues.
  Basically, in the present setup, we create a parentless `appMenuBar` whose submenus `QActions` are connected to `qApp` events (the app's global instance). However, at the `BitcoinGUI` destructor, we manually destruct this object without properly disconnecting the events. This leaves `qApp` events, such as `focusWindowChanged`, tied to submenus' `QAction` pointers, which causes the application to crash when it attempts to access them.

  Important Note:
  This happened to me few times. The worst consequence was an inconsistent chain state during IBD. Which triggered a full "replay blocks" process on the next startup. Which was painfully slow.

ACKs for top commit:
  RandyMcMillan:
    utACK bae209e
  hebasto:
    ACK bae209e387.

Tree-SHA512: 432e19c5f7e02c3165b7e7bd7f96f2a902bae5b5e439c2594db1c69d74ab6e0d4509d90f02db8c076f616e567e6a07492ede416ef651b5f749637398291b92fd
2023-10-03 13:56:41 +01:00
stickies-v
68f23f57d7 http: bugfix: track closed connection
It is possible that the client disconnects before the request is
handled. In those cases, evhttp_request_set_on_complete_cb is never
called, which means that on shutdown the server we'll keep waiting
endlessly.

By adding evhttp_connection_set_closecb, libevent automatically
cleans up those dead connections at latest when we shutdown, and
depending on the libevent version already at the moment of remote
client disconnect. In both cases, the bug is fixed.
2023-10-03 13:35:46 +01:00
stickies-v
084d037231 http: log connection instead of request count
There is no significant benefit in logging the request count instead
of the connection count. Reduces amount of code and computational
complexity.
2023-10-03 13:35:44 +01:00
stickies-v
41f9027813 http: refactor: use encapsulated HTTPRequestTracker
Introduces and uses a HTTPRequestTracker class to keep track of
how many HTTP requests are currently active, so we don't stop the
server before they're all handled.

This has two purposes:
1. In a next commit, allows us to untrack all requests associated
with a connection without running into lifetime issues of the
connection living longer than the request
(see https://github.com/bitcoin/bitcoin/pull/27909#discussion_r1265614783)

2. Improve encapsulation by making the mutex and cv internal members,
and exposing just the WaitUntilEmpty() method that can be safely
used.
2023-10-03 13:34:15 +01:00
fanquake
848eec0936 depends: fix unusable memory_resource in macos qt build
See https://codereview.qt-project.org/c/qt/qtbase/+/482392.
2023-10-03 13:31:19 +01:00
dergoegge
77506f4ac6 [net processing] Addr shuffle uses PeerManager's rng 2023-10-03 11:23:24 +01:00
dergoegge
a648dd79e5 [net processing] PushAddress uses PeerManager's rng 2023-10-03 11:23:24 +01:00
dergoegge
87c706713e [net processing] PeerManager holds a FastRandomContext 2023-10-03 11:23:24 +01:00
fanquake
693a7cfc6c Merge bitcoin/bitcoin#28507: contrib/bash-completions: use package naming conventions
360b917674 contrib/bash-completions: use package naming conventions (Erik Arvstedt)

Pull request description:

  #### Copy of commit msg

  This naming scheme supports auto-detection and on-demand loading of completions.

  See
  ba109693ee/README.md (faq),
  section "Where should I put it to be sure that interactive bash shells will find it and source it", keyword `foo.bash`.

  Previously, distro package maintainers had to rename these files manually.

ACKs for top commit:
  willcl-ark:
    ACK 360b917674

Tree-SHA512: 6dd1f62309e877402fac2f7aec785d3053a1fd3fdeae38abc961c3f61269eb664a2489b6fa8294bd706d8ac387c7af3ac66cd7a852810f69371fa8ae991afacb
2023-10-03 10:58:34 +01:00
fanquake
4e78834ec1 Merge bitcoin/bitcoin#28304: doc: Remove confusing assert linter
fa6e6a3f03 doc: Remove confusing assert linter (MarcoFalke)

Pull request description:

  The `assert()` documentation and linter are redundant and confusing:

  * The source code already refuses to compile with `assert()` disabled.
  * They violate the assumptions about `Assert()`, which *requires* side effects.
  * The existing linter doesn't enforce the guideline, only checking for `++` and `--` side effects.

  Fix all issues by removing the docs and the linter. See also https://github.com/bitcoin/bitcoin/pull/26684#discussion_r1287370102

  Going forward everyone is free to use whatever code in this regard they think is the easiest to read. Also, everyone is still free to share style-nits, if they think it is a good use of their time and of the pull request author. Finally, the author is still free to dismiss or ignore this style-nit, or any other style-nit.

ACKs for top commit:
  hebasto:
    ACK fa6e6a3f03, I have reviewed the code and it looks OK.
  theStack:
    ACK fa6e6a3f03

Tree-SHA512: 686738d71e1316cc95e5d3f71869b55a02bfb137c795cc0875057f4410e564bc8eff03c985a2087b007fb08fc84551c7da1e8b30c7a9c3f2b14e5e44a5970236
2023-10-03 10:44:21 +01:00
fanquake
8113ac0ab4 Merge bitcoin/bitcoin#28543: build, macos: Fix qt package build with new Xcode 15 linker
79ef528511 build, macos: Fix `qt` package build with new Xcode 15 linker (Hennadii Stepanov)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/28541 by backporting an upstream [patch](cdf64b0e47).

  Guix build:
  ```
  x86_64
  b37713bc8a526662eac3d9535924f4a4d2893c58f9c12d3c7599e761e6ff677c  guix-build-79ef528511f0/output/arm64-apple-darwin/SHA256SUMS.part
  0befb524181aa10e1635a2616a8bed53f51beafa4f0d495d3bf52a64cbd2d977  guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin-unsigned.tar.gz
  9cba170f2ffe542c33fdd1ac52b7684dd6301e91d32aa45af7b4ce8769d88d4a  guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin-unsigned.zip
  04556309266c791ae4d7409359222c88cd7aeb569566f7ef4d29816148a5b7e4  guix-build-79ef528511f0/output/arm64-apple-darwin/bitcoin-79ef528511f0-arm64-apple-darwin.tar.gz
  51229df8e104a2ffcd5c5b3f81f7585e1258ef10461d136948ea2a2d690a920d  guix-build-79ef528511f0/output/dist-archive/bitcoin-79ef528511f0.tar.gz
  3fe216a05561f2fe7229ddf186ff495b29a5cc31b6f35f407187573d072c5743  guix-build-79ef528511f0/output/x86_64-apple-darwin/SHA256SUMS.part
  961d71104e61a2baf727576eb2da630697bb4f109f66e73be5c96add25378d12  guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin-unsigned.tar.gz
  5598f514d065756ac376e2f3c4f8e758bfba53a43ddef778f106456de1536073  guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin-unsigned.zip
  5360ae1f1b7d96a44a33b2c87708b466e4a7bf3f9de0fc58bccbbcdb21ee254e  guix-build-79ef528511f0/output/x86_64-apple-darwin/bitcoin-79ef528511f0-x86_64-apple-darwin.tar.gz
  ```

Top commit has no ACKs.

Tree-SHA512: e3a0f7a578b30a216cc84c8ac6a0eeac3f59b02525e1eb5a9f5512bc9a049a1b17d3feb140259ffe5d2197279c74594126b85112aa596df9013f74bb1047c298
2023-10-03 10:29:21 +01:00
Hennadii Stepanov
f08adec886 qt: Add "Transport" label to peer details 2023-10-03 10:18:51 +01:00
fanquake
6f882e6f86 Merge bitcoin/bitcoin#28331: BIP324 integration
75a3291035 doc: mention BIP324 support in bips.md (Pieter Wuille)
64ca7210f0 test: enable v2 transport between nodes in some functional tests (Pieter Wuille)
05d19fbcc1 test: Functional test for opportunistic encryption (dhruv)
b815cce50e net: expose transport types/session IDs of connections in RPC and logs (Pieter Wuille)
432a62c4dc net: reconnect with V1Transport under certain conditions (Pieter Wuille)
4d265d0342 sync: modernize CSemaphore / CSemaphoreGrant (Pieter Wuille)
c73cd42363 rpc: addnode arg to use BIP324 v2 p2p (dhruv)
62d21ee097 net: use V2Transport when NODE_P2P_V2 service flag is present (Pieter Wuille)
a4706bc877 rpc: don't report v2 handshake bytes in the per-type sent byte statistics (Sebastian Falbesoner)
abf343b320 net: advertise NODE_P2P_V2 if CLI arg -v2transport is on (Pieter Wuille)

Pull request description:

  Part of #27634.

  This makes BIP324 support feature complete, through a (default off) `-v2transport` option for enabling V2 connections. If it is enabled:
  * The `NODE_P2P_V2` service flag (*1 << 11*) is advertized.
  * Inbound connections can use V1 or V2 (automatically detected based on the protocol used by the peer)
  * V2 connections are used on outbound when the `NODE_P2P_V2` service is available (or the new `use_v2` parameter is set on the `addnode` RPC).
  * V2 outbound connections that instantly fail get retried as V1.

  There are two new RPC fields, `"transport_protocol_type"` and `"session_id"`, in `getpeerinfo`.

ACKs for top commit:
  mzumsande:
    re-ACK 75a3291035
  theStack:
    re-ACK 75a3291035

Tree-SHA512: 90ea1cd37f3dce410a59ff5de1c2405891e8aa62318d0e06dcb68b21603fb0c061631526633f3d4fb630e63d2b8db407eed48e246befcbef3503bea893a4ff15
2023-10-03 10:12:50 +01:00
Pieter Wuille
75a3291035 doc: mention BIP324 support in bips.md 2023-10-02 18:11:11 -04:00
Pieter Wuille
64ca7210f0 test: enable v2 transport between nodes in some functional tests 2023-10-02 18:11:11 -04:00
dhruv
05d19fbcc1 test: Functional test for opportunistic encryption
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-10-02 18:11:11 -04:00
Pieter Wuille
b815cce50e net: expose transport types/session IDs of connections in RPC and logs
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:11:11 -04:00
Pieter Wuille
432a62c4dc net: reconnect with V1Transport under certain conditions
When an outbound v2 connection is disconnected without receiving anything, but at
least 24 bytes of our pubkey were sent out (enough to constitute an invalid v1
header), add them to a queue of reconnections to be tried.

The reconnections are in a queue rather than performed immediately, because we should
not block the socket handler thread with connection creation (a blocking operation
that can take multiple seconds).
2023-10-02 18:11:11 -04:00
Pieter Wuille
4d265d0342 sync: modernize CSemaphore / CSemaphoreGrant 2023-10-02 18:11:11 -04:00
dhruv
c73cd42363 rpc: addnode arg to use BIP324 v2 p2p
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-10-02 18:10:30 -04:00
Pieter Wuille
62d21ee097 net: use V2Transport when NODE_P2P_V2 service flag is present
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:09:53 -04:00
Sebastian Falbesoner
a4706bc877 rpc: don't report v2 handshake bytes in the per-type sent byte statistics
This matches the behavior for per-type received bytes.
2023-10-02 18:09:53 -04:00
Pieter Wuille
abf343b320 net: advertise NODE_P2P_V2 if CLI arg -v2transport is on
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-10-02 18:09:53 -04:00
Andrew Chow
e7b0004b37 Merge bitcoin/bitcoin#27596: assumeutxo (2)
edbed31066 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost)
b8cafe3871 chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost)
99839bbfa7 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne)
7ee46a755f contrib: add script to demo/test assumeutxo (James O'Beirne)
42cae39356 test: add feature_assumeutxo functional test (James O'Beirne)
0f64bac603 rpc: add getchainstates (James O'Beirne)
bb05857794 refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne)
ce585a9a15 rpc: add loadtxoutset (James O'Beirne)
62ac519e71 validation: do not activate snapshot if behind active chain (James O'Beirne)
9511fb3616 validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne)
7fcd21544a blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne)
4c3b8ca35c validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne)
49ef778158 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne)
1019c39982 validation: pruning for multiple chainstates (James O'Beirne)
373cf91531 validation: indexing changes for assumeutxo (James O'Beirne)
1fffdd76a1 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne)
fbe0a7d7ca wallet: validationinterface: only handle active chain notifications (James O'Beirne)
f073917a9e validationinterface: only send zmq notifications for active (James O'Beirne)
4d8f4dcb45 validation: pass ChainstateRole for validationinterface calls (James O'Beirne)
1e59acdf17 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne)
c6af23c517 validation: add ChainstateRole (James O'Beirne)
9f2318c76c validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne)
434495a8c1 chainparams: add blockhash to AssumeutxoData (James O'Beirne)
c711ca186f assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne)
c93ef43e4f bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne)
b73d3bbd23 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar)

Pull request description:

  - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal
  - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11
  - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there.

  ---

  This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background.

  This may look like a lot to review, but note that
  - ~200 lines are a (non-essential) demo shell script
  - Many lines are functional test, documentation, and relatively dilute RPC code.

  So it shouldn't be as burdensome to review as the linecount might suggest.

  - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work.
  - **Pruning**: implement correct pruning behavior when using a background chainstate
  - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning.
  - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially.
  - Have `-reindex` properly wipe snapshot chainstates.
  - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`.
  - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release.
    - This will complete the project and allow use of UTXO snapshots for faster node bootstrap.

  The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network.

  ---

  ### UTXO snapshots

  Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g.
  ```shell
  ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`)
  ```
  or use the pre-generated ones listed below.

  - Testnet: **2'500'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388`
  - Signet: **160'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c`
  - Mainnet: **800'000** (Sjors):
    - Note: this needs the following commit cherry-picked in: 24deb2022b
    - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`

  ### Testing

  #### For fun (~5min)

  If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though.

  #### For real (longer)

  If you'd like to experience a real usage of assumeutxo, you can do that too.
  I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing:
  ```sh
  $ cd ~/src/bitcoin  # or whatever

  # get the snapshot
  $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat

  # you'll want to do this if you like copy/pasting
  $ export AU_DATADIR=/home/${USER}/au-test # or wherever

  $ mkdir ${AU_DATADIR}
  $ vim ${AU_DATADIR}/bitcoin.conf

  dbcache=8000  # or, you know, something high
  blockfilterindex=1
  coinstatsindex=1
  prune=3000
  logthreadnames=1
  ```
  Obtain this branch, build it, and then start bitcoind:
  ```sh
  $ git remote add jamesob https://github.com/jamesob/bitcoin
  $ git fetch jamesob assumeutxo
  $ git checkout jamesob/assumeutxo

  $ ./configure $conf_args && make  # (whatever you like to do here)

  # start 'er up and watch the logs
  $ ./src/bitcoind -datadir=${AU_DATADIR}
  ```
  Then, in some other window, load the snapshot
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat
  ```

  You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by.

  In yet another window, you can check out chainstate status with
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates
  ```
  as well as usual favorites like `getblockchaininfo`.

ACKs for top commit:
  achow101:
    ACK edbed31066

Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02 17:09:44 -04:00
Sjors Provoost
edbed31066 chainparams: add signet assumeutxo param at height 160_000 2023-10-02 16:56:48 -04:00
Sjors Provoost
b8cafe3871 chainparams: add testnet assumeutxo param at height 2_500_000 2023-10-02 16:56:46 -04:00
glozow
fd8ab08558 Merge bitcoin/bitcoin#28557: bench: remove NO_THREAD_SAFETY_ANALYSIS usage
d67aa25eb2 bench: drop NO_THREAD_SAFETY_ANALYSIS from disconnected_txs (fanquake)

Pull request description:

  Followup to https://github.com/bitcoin/bitcoin/pull/28385#discussion_r1339964480.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d67aa25eb2
  hebasto:
    ACK d67aa25eb2, tested on Ubuntu 22.04 with clang 18.0.
  glozow:
    utACK d67aa25eb2

Tree-SHA512: a9a9a8cc70a50d4fbd51779a3203bbc2a29d354b557e8a99cfd649d5998b71ff1087f5bae7170471bed9a917a93c8f3351ae90c9a6e87d88928c35912d007b64
2023-10-02 16:20:36 +01:00
0xb10c
352d5eb2a9 test: getrawaddrman RPC
Test that the getrawaddrman returns the addresses in the new and tried
tables. We can't check the buckets and positions as these are not
deterministic (yet).
2023-10-02 15:45:14 +02:00
0xb10c
da384a286b rpc: getrawaddrman for addrman entries
Exposing address manager table entries in a hidden RPC allows to introspect
addrman tables in tests and during development.

As response JSON object the following FORMAT1 is choosen:
{
  "table": {
    "<bucket>/<position>": { "address": "..", "port": .., ... },
    "<bucket>/<position>": { "address": "..", "port": .., ... },
    "<bucket>/<position>": { "address": "..", "port": .., ... },
    ...
  }
}

An alternative would be FORMAT2
{
  "table": {
    "bucket": {
      "position": { "address": "..", "port": .., ... },
      "position": { "address": "..", "port": .., ... },
      ..
    },
    "bucket": {
      "position": { "address": "..", "port": .., ... },
      ..
    },
  }
}

FORMAT1 and FORMAT2 have different encodings for the location of the
address in the address manager. While FORMAT2 might be easier to process
for downstream tools, it also mimics internal addrman mappings, which
might change at some point. Users not interested in the address location
can ignore the location key. They don't have to adapt to a new RPC
response format, when the internal addrman layout changes. Additionally,
FORMAT1 is also slightly easier to to iterate in downstream tools. The
RPC response-building implemenation complexcity is lower with FORMAT1
as we can more easily build a "<bucket>/<position>" key than a multiple
"bucket" objects with multiple "position" objects (FORMAT2).
2023-10-02 15:34:28 +02:00
fanquake
d67aa25eb2 bench: drop NO_THREAD_SAFETY_ANALYSIS from disconnected_txs 2023-10-02 14:27:21 +01:00
fanquake
8909667ab8 Merge bitcoin/bitcoin#28176: tests: add coverage to feature_addrman.py
380130d9d7 test: add coverage to feature_addrman.py (kevkevin)

Pull request description:

  I added two new tests that will cover the nNew and nTried tests which add coverage to the if block by checking values larger than our range since we only check for negative values now

  adding coverage to these lines
  https://github.com/bitcoin/bitcoin/blob/master/src/addrman.cpp#L273
  https://github.com/bitcoin/bitcoin/blob/master/src/addrman.cpp#L280

  our test seem to only cover the `nTried < 0` and `nNew < 0` scenarios

ACKs for top commit:
  ismaelsadeeq:
    ACK 380130d9d7, code looks good to me 🍃 .
  0xB10C:
    Re-ACK 380130d9d7

Tree-SHA512: a063bd9ca4d2d536a27c8c22a28fb13759a96f19cd8ba6cb8879cf7f65046d4ff6e8f70df17feaffd0d0d08ef914cb18a11258d313a4841c811a7e7ae4df6d5b
2023-10-02 14:07:20 +01:00
fanquake
50f250a67d Merge bitcoin/bitcoin#28542: wallet: Check for uninitialized last processed and conflicting heights in MarkConflicted
782701ce7d test: Test loading wallets with conflicts without a chain (Andrew Chow)
4660fc82a1 wallet: Check last block and conflict height are valid in MarkConflicted (Andrew Chow)

Pull request description:

  `MarkConflicted` assumes that `m_last_block_processed_height` is always valid. However it may not be valid when a chain is not attached, as happens in the wallet tool and during migration. In such situations, when the conflicting height is also negative (which occurs on loading when no chain is available), the calculation of the number of conflict confirms results in a non-negative value which passes the existing check for valid values. This will subsequently hit an assertion in `GetTxDepthInMainChain`.

  Furthermore, `MarkConflicted` is also only called on loading a transaction whose parent has a stored state of `TxStateConflicted` and was loaded before the child transaction. This depends on the loading order, which for both sqlite and bdb depends on the txids.

  We can avoid this by explicitly checking that both `m_last_block_processed_height` and `conflicting_height` are non-negative. Both `tool_wallet.py` and `wallet_migration.py` are updated to create wallets with a state that triggers the assertion.

  Fixes #28510

ACKs for top commit:
  ryanofsky:
    Code review ACK 782701ce7d. Nice catch, and clever test (grinding the txid)
  furszy:
    ACK 782701ce

Tree-SHA512: 1344e0279ec5413a43a2819d101fb571fbf4821de2d13958a0fdffc99f57082ef3243ec454c8343f97dc02ed1fce8c8b0fd89388420ab2e55618af42ad5630a9
2023-10-02 13:27:41 +01:00
fanquake
dcf6230f92 Merge bitcoin/bitcoin#28556: doc: fix link to developer-notes.md file in multiprocess.md
d9b172cd00 doc: fix link to developer-notes.md file in multiprocess.md (David Álvarez Rosa)

Pull request description:

  Fix link to `developer-notes.md` file in `multiprocess.md`.

ACKs for top commit:
  fanquake:
    ACK d9b172cd00

Tree-SHA512: 55fffefb37c4d67acb1fa8b0660216ec1c7f2c2314d11e4d319cae40480ed59ef448909fa2ca334167c86d60d41932220dce4186e28fa300f4d03eb0b3c769d0
2023-10-02 12:49:26 +02:00
fanquake
48b8910d12 Merge bitcoin/bitcoin#28508: refactor: Remove SER_GETHASH, hard-code client version in CKeyPool serialize
fac29a0ab1 Remove SER_GETHASH, hard-code client version in CKeyPool serialize (MarcoFalke)
fa72f09d6f Remove CHashWriter type (MarcoFalke)
fa4a9c0f43 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader (MarcoFalke)

Pull request description:

  Removes a bunch of redundant, dead or duplicate code.

  Uses the idea from and finishes the idea https://github.com/bitcoin/bitcoin/pull/28428 by theuni

ACKs for top commit:
  ajtowns:
    ACK fac29a0ab1
  kevkevinpal:
    added one nit but otherwise ACK [fac29a0](fac29a0ab1)

Tree-SHA512: cc805e2f38e73869a6691fdb5da09fa48524506b87fc93f05d32c336ad3033425a2d7608e317decd3141fde3f084403b8de280396c0c39132336fe0f7510af9e
2023-10-02 12:33:54 +02:00
fanquake
0f9307c4cb Merge bitcoin/bitcoin#28500: Prevent default/invalid CKey objects from allocating secure memory
6ef405ddb1 key: don't allocate secure mem for null (invalid) key (Pieter Wuille)
d9841a7ac6 Add make_secure_unique helper (Anthony Towns)

Pull request description:

  Bitcoin Core has `secure_allocator`, which allocates inside special "secure" (non-swappable) memory pages, which may be limited in availability. Currently, every `CKey` object uses 32 such secure bytes, even when the `CKey` object contains the (invalid) value zero.

  Change this to not use memory when the `CKey` is invalid. This is particularly relevant for `BIP324Cipher` which briefly holds a `CKey`, but after receiving the remote's public key and initializing the encryption ciphers, the key is wiped. In case secure memory usage is in high demand, it'd be silly to waste it on P2P encryption keys instead of wallet keys.

ACKs for top commit:
  ajtowns:
    ACK 6ef405ddb1
  john-moffett:
    ACK 6ef405ddb1

Tree-SHA512: 987f4376ed825daf034ea4d7c4b4952fe664b25b48f1c09fbcfa6257a40b06c4da7c2caaafa35c346c86bdf298ae21f16c68ea4b1039836990d1a205de2034fd
2023-10-02 12:16:20 +02:00
David Álvarez Rosa
d9b172cd00 doc: fix link to developer-notes.md file in multiprocess.md 2023-10-02 12:12:36 +02:00
fanquake
e3b052800f Merge bitcoin/bitcoin#28227: test: check for specific bip157 disconnect reasons, add test coverage
2ab7952bda test: add bip157 coverage for (start height > stop height) disconnect (Sebastian Falbesoner)
63e90e1d3f test: check for specific disconnect reasons in p2p_blockfilters.py (Sebastian Falbesoner)

Pull request description:

  This PR checks for specific disconnect reasons using `assert_debug_log` in the functional test `p2p_blockfilters.py`. With that we ensure that the disconnect happens for the expected reason and also makes it easier to navigate between implementation and test code, i.e. both the questions "do we have test coverage for this disconnect cause?" (from an implementation reader's perspective) and "where is the code handling this disconnect cause?" (from a test reader's perspective) can be answered simply by grep-ping the corresponding debug message.

  Also, based on that, missing coverage for the (start height > stop height) disconnect case is added:
  b7138252ac/src/net_processing.cpp (L3050-L3056)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 2ab7952bda
  furszy:
    Looks good, code ACK 2ab7952b

Tree-SHA512: 0581cb569d5935aaa004a95a6f16eeafe628b9d816ebb89232f2832e377049df878a1e74c369fb46931b94e1a3a5e3f4aaa21a007c0a488f4ad2cda0919c605d
2023-10-02 11:57:56 +02:00
fanquake
8b44d01118 Merge bitcoin/bitcoin#28184: lint: fix custom mypy cache dir setting
f9047771d6 lint: fix custom mypy cache dir setting (Fabian Jahr)

Pull request description:

  fixes #28183

  The custom cache dir for `mypy` can only be set via an environment variable, setting the `MYPY_CACHE_DIR` variable in the program is not sufficient. This error was introduced while translating the shell script to python.

  See also the mypy documentation: https://mypy.readthedocs.io/en/stable/config_file.html#confval-cache_dir

ACKs for top commit:
  MarcoFalke:
    lgtm ACK f9047771d6

Tree-SHA512: 7e8fb0cd06688129bd46d1afb8647262eb53d0f60b1ef6f288fedaa122d906fb62c9855e8bb0d6c6297d41a87a47d3cec7a00df55a7d033947937dfe23d07ba7
2023-10-02 11:19:08 +02:00
glozow
5b878be742 [doc] add release note for submitpackage 2023-10-02 10:15:22 +01:00
glozow
7a9bb2a2a5 [rpc] allow submitpackage to be called outside of regtest 2023-10-02 10:13:38 +01:00
glozow
5b9087a9a7 [rpc] require package to be a tree in submitpackage 2023-10-02 10:13:38 +01:00
glozow
e32ba1599c [txpackages] IsChildWithParentsTree()
Many edge cases exist when parents in a child-with-parents package can
spend each other. However, this pattern should also be uncommon in
normal use cases.
2023-10-02 10:13:38 +01:00
glozow
b4f28cc345 [doc] parent pay for child in aggregate CheckFeeRate 2023-10-02 10:13:38 +01:00
fanquake
f66af92f1a Merge bitcoin/bitcoin#28545: test: Avoid test failure on Linux root without cap-add LINUX_IMMUTABLE
fa40b3ee22 test: Avoid test failure on Linux root without cap-add LINUX_IMMUTABLE (MarcoFalke)

Pull request description:

  This turns a test failure on Linux when running the test as `root`, but without the `LINUX_IMMUTABLE` capability, into an early return, with a suggestion to turn on `LINUX_IMMUTABLE` next time (if possible).

ACKs for top commit:
  pinheadmz:
    utACK fa40b3ee22
  jonatack:
    ACK fa40b3ee22

Tree-SHA512: d986ff8aeae5f8267c21a23d5be16f7c5a4d4d3be045a6999d8b39c7b8672cfe915dedde762cc9965cdc4970940bffc4b0d1412833d8036d4425450eb6181f67
2023-10-02 11:08:56 +02:00
kevkevin
380130d9d7 test: add coverage to feature_addrman.py
I added two new tests that will cover the nNew and nTried tests which
add coverage to the if block by checking values larger than our range
since we only check for negative values now

Co-authored-by: ismaelsadeeq <ask4ismailsadiq@gmail.com>
2023-10-01 22:43:49 -05:00
James O'Beirne
99839bbfa7 doc: add note about confusing HaveTxsDownloaded name 2023-09-30 06:41:23 -04:00
James O'Beirne
7ee46a755f contrib: add script to demo/test assumeutxo
Add the script to the shellcheck exception list since the
quoted variables rule needs to be violated in order to get
bitcoind to pick up on $CHAIN_HACK_FLAGS.
2023-09-30 06:41:23 -04:00
James O'Beirne
42cae39356 test: add feature_assumeutxo functional test
Most ideas for test improvements (TODOs) provided by Russ Yanofsky.
2023-09-30 06:41:23 -04:00
James O'Beirne
0f64bac603 rpc: add getchainstates
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30 06:41:23 -04:00
James O'Beirne
bb05857794 refuse to activate a UTXO snapshot if mempool not empty
This ensures that we avoid any unexpected conditions inherent in
transferring non-empty mempools across chainstates.

Note that this should never happen in practice given that snapshot
activation will not occur outside of IBD, based upon the height checks
in `loadtxoutset`.
2023-09-30 06:41:23 -04:00
James O'Beirne
ce585a9a15 rpc: add loadtxoutset
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2023-09-30 06:41:23 -04:00
James O'Beirne
62ac519e71 validation: do not activate snapshot if behind active chain
Most easily reviewed with

  git show --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30 06:41:21 -04:00
James O'Beirne
9511fb3616 validation: assumeutxo: swap m_mempool on snapshot activation
Otherwise we will not receive transactions during background sync until
restart.
2023-09-30 06:40:17 -04:00
James O'Beirne
7fcd21544a blockstorage: segment normal/assumedvalid blockfiles
When using an assumedvalid (snapshot) chainstate along with a background
chainstate, we are syncing two very different regions of the chain
simultaneously. If we use the same blockfile space for both of these
syncs, wildly different height blocks will be stored alongside one
another, making pruning ineffective.

This change implements a separate blockfile cursor for the assumedvalid
chainstate when one is in use.
2023-09-30 06:40:17 -04:00
James O'Beirne
4c3b8ca35c validation: populate nChainTx value for assumedvalid chainstates
Use the expected AssumeutxoData in order to bootstrap nChainTx values
for assumedvalid blockindex entries in the snapshot chainstate. This
is necessary because nChainTx is normally built up from nTx values,
which are populated using blockdata which the snapshot chainstate
does not yet have.
2023-09-30 06:40:17 -04:00
James O'Beirne
49ef778158 test: adjust chainstate tests to use recognized snapshot base
In future commits, loading the block index while making use of a
snapshot is contingent on the snapshot being recognized by chainparams.

Ensure all existing unittests that use snapshots use a recognized
snapshot (at height 110).

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-09-30 06:40:17 -04:00
James O'Beirne
1019c39982 validation: pruning for multiple chainstates
Introduces ChainstateManager::GetPruneRange().

The prune budget is split evenly between the number of chainstates,
however the prune budget may be exceeded if the resulting shares are
beneath `MIN_DISK_SPACE_FOR_BLOCK_FILES`.
2023-09-30 06:40:16 -04:00
James O'Beirne
373cf91531 validation: indexing changes for assumeutxo
When using an assumedvalid chainstate, only process validationinterface
callbacks from the background chainstate within indexes. This ensures
that all indexes are built in-order.

Later, we can possibly designate indexes which can be built out of order
and continue their operation during snapshot use.

Once the background sync has completed, restart the indexes so that
they continue to index the now-validated snapshot chainstate.
2023-09-30 06:38:47 -04:00
James O'Beirne
1fffdd76a1 net_processing: validationinterface: ignore some events for bg chain 2023-09-30 06:38:47 -04:00
James O'Beirne
fbe0a7d7ca wallet: validationinterface: only handle active chain notifications 2023-09-30 06:38:47 -04:00
James O'Beirne
f073917a9e validationinterface: only send zmq notifications for active 2023-09-30 06:38:47 -04:00
James O'Beirne
4d8f4dcb45 validation: pass ChainstateRole for validationinterface calls
This allows consumers to decide how to handle events from background or
assumedvalid chainstates.
2023-09-30 06:38:47 -04:00
James O'Beirne
1e59acdf17 validation: only call UpdatedBlockTip for active chainstate
This notification isn't needed for background chainstates.

`kernel::Notifications::blockTip` are also skipped.
2023-09-30 06:38:47 -04:00
James O'Beirne
c6af23c517 validation: add ChainstateRole 2023-09-30 06:38:47 -04:00
James O'Beirne
9f2318c76c validation: MaybeRebalanceCaches when chain leaves IBD
Check to see if we need to rebalance caches across chainstates when
a chain leaves IBD.
2023-09-30 06:38:47 -04:00
James O'Beirne
434495a8c1 chainparams: add blockhash to AssumeutxoData
This allows us to reference assumeutxo configuration by blockhash as
well as height; this is helpful in future changes when we want to
reference assumeutxo configurations before the block index is loaded.
2023-09-30 06:38:47 -04:00
James O'Beirne
c711ca186f assumeutxo: remove snapshot during -reindex{-chainstate}
Removing a snapshot chainstate from disk (and memory) is consistent with
existing reindex operations.
2023-09-30 06:38:43 -04:00
James O'Beirne
c93ef43e4f bugfix: correct is_snapshot_cs in VerifyDB 2023-09-30 05:45:40 -04:00
Suhas Daftuar
b73d3bbd23 net_processing: Request assumeutxo background chain blocks
Add new PeerManagerImpl::TryDownloadingHistoricalBlocks method and use it to
request background chain blocks in addition to blocks normally requested by
FindNextBlocksToDownload.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: James O'Beirne <james.obeirne@gmail.com>
2023-09-30 05:45:37 -04:00
Andrew Chow
5bbf735def Merge bitcoin/bitcoin#27653: test: add unit test coverage for Python ECDSA implementation
96b3f2dbe4 test: add unit test coverage for Python ECDSA implementation (Sebastian Falbesoner)

Pull request description:

  This PR adds missing unit test coverage for the Python ECDSA implementation, which should be useful for detecting potential problems early whenever changes in the test framework's Python implementation of secp256k1 are made (e.g. #26222). Note that right now we don't call `ECPubKey.verify_ecdsa` anywhere in our tests, so we wouldn't notice if it is broken at some point.

  To keep it simple, the already existing unit test for Schnorr signatures is extended to also check ECDSA signatures. For that purpose, the dictionary storing private-key/public-key entries use their legacy types `ECKey/ECPubKey` instead of bare byte-arrays, and for Schnorr signing/verification the necessary conversions (ECKey -> bare private key, ECPubKey -> x-only pubkey) is done later when needed. To avoid code duplication, a helper function `random_bitflip` for damaging signatures is introduced.

  The unit test can be run by either calling it for this single module:
  `$ python3 -m unittest ./test/functional/test_framework/key.py`
  or simply running `$ ./test/functional/test_runner.py` which calls all test framework module's unit tests at the start (see TEST_FRAMEWORK_MODULES list).

ACKs for top commit:
  achow101:
    ACK 96b3f2dbe4
  sipa:
    utACK 96b3f2dbe4
  stratospher:
    tested ACK 96b3f2d.

Tree-SHA512: b993f25b843fa047376addda4ce4b0f15750ffba926528b5cca4c5f99b9af456206f4e8af885d25a017dddddf382ddebf38765819b3d16a3f28810d03b010808
2023-09-29 14:49:58 -04:00
Ryan Ofsky
f562856d02 Merge bitcoin/bitcoin#27866: blockstorage: Return on fatal flush errors
d8041d4e04 blockstorage: Return on fatal undo file flush error (TheCharlatan)
f0207e0030 blockstorage: Return on fatal block file flush error (TheCharlatan)
5671c15f45 blockstorage: Mark FindBlockPos as nodiscard (TheCharlatan)

Pull request description:

  The goal of this PR is to establish that fatal blockstorage flush errors should be treated as errors at their call site.

  Prior to this patch `FlushBlockFile` may have failed without returning in `Chainstate::FlushStateToDisk`, leading to a potential write from `WriteBlockIndexDB` that may refer to a block that is not fully flushed to disk yet. By returning if either `FlushUndoFile` or `FlushBlockFile` fail, we ensure that no further write operations take place that may lead to an inconsistent database when crashing. Add `[[nodiscard]]` annotations to them such that they are not ignored in future.

  Functions that call either `FlushUndoFile` or `FlushBlockFile`, need to handle these extra abort cases properly. Since `Chainstate::FlushStateToDisk` already produces an abort error in case of `WriteBlockIndexDB` failing, no extra logic for functions calling `Chainstate::FlushStateToDisk` is required.

  Besides `Chainstate::FlushStateToDisk`, `FlushBlockFile` is also called by `FindBlockPos`, while `FlushUndoFile` is only called by `FlushBlockFile` and `WriteUndoDataForBlock`. For both these cases, the flush error is not further bubbled up. Instead, the error is logged and a comment is provided why bubbling up an error would be less desirable in these cases.

  ---

  This pull request is part of a larger effort towards improving the shutdown / abort / fatal error handling in validation code. It is a first step towards implementing proper fatal error return type enforcement similar as proposed by theuni in this pull request [comment](https://github.com/bitcoin/bitcoin/pull/27711#issuecomment-1563561502). For ease of review of these critical changes, a first step would be checking that `AbortNode` leads to early and error-conveying returns at its call site. Further work for enforcing returns when `AbortNode` is called is done in https://github.com/bitcoin/bitcoin/pull/27862.

ACKs for top commit:
  stickies-v:
    re-ACK d8041d4
  ryanofsky:
    Code review ACK d8041d4e04

Tree-SHA512: 47ade9b873b15e567c8f60ca538d5a0daf32163e1031be3212a3a45eb492b866664b225f2787c9e40f3e0c089140157d8fd1039abc00c7bdfeec1b52ecd7e219
2023-09-29 13:29:51 -04:00
Andrew Chow
d18a8f6f69 Merge bitcoin/bitcoin#28525: net: Drop v2 garbage authentication packet
e3720bca39 net: Simplify v2 recv logic by decoupling AAD from state machine (Tim Ruffing)
b0f5175c04 net: Drop v2 garbage authentication packet (Tim Ruffing)

Pull request description:

  Note that this is a breaking change, see also https://github.com/bitcoin/bips/pull/1498

  The benefit is a simpler implementation:
   - The protocol state machine does not need separate states for garbage authentication and version phases.
   - The special case of "ignoring the ignore bit" is removed.
   - The freedom to choose the contents of the garbage authentication packet is removed. This simplifies testing.

ACKs for top commit:
  naumenkogs:
    ACK e3720bca39
  sipa:
    ACK e3720bca39. Re-ran the v2 transport fuzzer overnight.
  ajtowns:
    ACK e3720bca39 - simpler and more flexible, nice
  achow101:
    ACK e3720bca39
  Sjors:
    utACK e3720bca39
  theStack:
    Code-review ACK e3720bca39

Tree-SHA512: 16600ed868c8a346828de075c4072e37cf86440751d08ab099fe8b58ff71d8b371a90397d6a4247096796db68794275e7e0403f218859567d04838e0411dadd6
2023-09-29 09:50:02 -04:00
Pieter Wuille
3d420d8f28 Add instructions for headerssync-params.py to release-process.md 2023-09-28 12:11:30 -04:00
Pieter Wuille
53d7d35b58 Update parameters in headerssync.cpp 2023-09-28 12:07:54 -04:00
Pieter Wuille
7899402cff Add headerssync-params.py script to the repository 2023-09-28 12:05:52 -04:00
Andrew Chow
9d5150ac47 Merge bitcoin/bitcoin#28540: tests: Fix wallet_resendwallettransactions.py intermittent failure by using manual bumps instead of bumpfee
b5a962564e tests: Use manual bumps instead of bumpfee for resendwallettransactions (Andrew Chow)

Pull request description:

  Bumpfee will try to increase the entire package to the target feerate, which causes repeated bumpfees to quickly shoot up in fees, causing intermittent failures when the fee is too large. We don't care about this property, just that the child is continuously replaced until we observe it's position in mapWallet is before its parent. Instead of using bumpfee, we can create raw transactions which have only pay (just above) the additional incremental relay fee, thus avoiding this problem.

  Fixes #28491

ACKs for top commit:
  kevkevinpal:
    ACK [b5a9625](b5a962564e)
  mzumsande:
    Code review ACK b5a962564e
  pablomartin4btc:
    ACK b5a962564e -> adding the `try_rpc` to avoid (skip) any possible failure around the manual bump fee (if we ever reach it as [explained](https://github.com/bitcoin/bitcoin/pull/28540#issuecomment-1737648048)) makes a lot of sense as the spirit of the test is the tx (child before parent) sort in the `mapWallet` (as also [explained](https://github.com/bitcoin/bitcoin/issues/28491#issuecomment-1736161363)).
  MarcoFalke:
    lgtm ACK b5a962564e

Tree-SHA512: f184f11c73be0c30753181901f51a3b4b9c4135e0c4681e9f4ca94692c49bac15c91683c85266a2124333c8593e9919bfd9102724616faab299740f2eb98741f
2023-09-28 11:22:10 -04:00
Fabian Jahr
f9047771d6 lint: fix custom mypy cache dir setting 2023-09-28 13:20:25 +02:00
glozow
6619d6a8dc Merge bitcoin/bitcoin#28450: Add package evaluation fuzzer
262ab8ef78 Add package evaluation fuzzer (Greg Sanders)

Pull request description:

  This fuzzer target caught the issue in https://github.com/bitcoin/bitcoin/pull/28251 within 5 minutes on master branch, and an additional issue which I've applied a preliminary patch to cover.

  Fuzzer target does the following:

  1) Picks mempool confgs, including max package size, count, mempool size, etc
  2) Generates 1 to 26 transactions with arbitrary coins/fees, the first N-1 spending only confirmed outpoints
  3) Nth transaction, if >1, sweeps all unconfirmed outpoints in mempool
  4) If N==1, it may submit it through single-tx submission path, to allow for more interesting topologies
  5) Otherwise submits through package submission interface
  6) Repeat 1-5  a few hundred times per mempool instance

  In other words, it ends up building chains of txns in the mempool using parents-and-children packages, which is currently the topology supported on master.

  The test itself is a direct rip of tx_pool.cpp, with a number of assertions removed because they were failing for unknown reasons, likely due to the notification changes of single tx submission to package, which is used to track addition/removal of transactions in the test. I'll continue working on re-adding these assertions for further invariant testing.

ACKs for top commit:
  murchandamus:
    ACK 262ab8ef78
  glozow:
    reACK 262ab8ef78
  dergoegge:
    tACK 262ab8ef78

Tree-SHA512: 190784777d0f2361b051b3271db8f79b7927e3cab88596d2c30e556da721510bd17f6cc96f6bb03403bbf0589ad3f799fa54e63c1b2bd92a2084485b5e3e96a5
2023-09-28 12:05:24 +01:00
Andrew Chow
19a7e608f9 Merge bitcoin/bitcoin#28505: rpc: bumpfee, improve doc for 'reduce_output' arg
b3db8c9d5c rpc: bumpfee, improve doc for 'reduce_output' arg (furszy)

Pull request description:

  Fixes #28180. Resulted from discussions with S3RK, achow101, and Murch.

  The current argument name and description are dangerous as it don't
  describe the case where the user selects the recipient output as the
  change address. This one could end up been increased by the inputs
  minus outputs remainder. Which, when `bumpfee` adds new inputs
  to the transaction, leads the process to send more coins to the
  recipient. Which is not what the user would expect from a
  'reduce_output' param naming.

ACKs for top commit:
  S3RK:
    ACK b3db8c9d5c
  achow101:
    ACK b3db8c9d5c
  murchandamus:
    ACK b3db8c9d5c

Tree-SHA512: 91f607e2f5849041d7c099afdddae11af8bed5b1ac90c9d22921267f272e21b44e107d6968e037f05f958a61fe29e94e5fb44b224fb3606f197f83ec4ba3b1e7
2023-09-27 16:46:27 -04:00
Greg Sanders
262ab8ef78 Add package evaluation fuzzer 2023-09-27 16:27:05 -04:00
Pieter Wuille
6ef405ddb1 key: don't allocate secure mem for null (invalid) key
Instead of storing the key material as an std::vector (with secure allocator),
use a secure_unique_ptr to a 32-byte array, and use nullptr for invalid keys.
This means a smaller CKey type, and no secure/dynamic memory usage for invalid
keys.
2023-09-27 15:05:26 -04:00
Anthony Towns
d9841a7ac6 Add make_secure_unique helper
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-09-27 15:05:15 -04:00
Ryan Ofsky
8a553c9409 wallet: Add TxStateString function for debugging and logging
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-09-27 13:47:38 -04:00
Andrew Chow
b5a962564e tests: Use manual bumps instead of bumpfee for resendwallettransactions
Bumpfee will try to increase the entire package to the target feerate,
which causes repeated bumpfees to quickly shoot up in fees, causing
intermittent failures when the fee is too large. We don't care about
this property, just that the child is continuously replaced until we
observe it's position in mapWallet is before its parent. Instead of
using bumpfee, we can create raw transactions which have only pay the
additional incremental relay fee, thus avoiding this problem.
2023-09-27 11:39:07 -04:00
MarcoFalke
fa40b3ee22 test: Avoid test failure on Linux root without cap-add LINUX_IMMUTABLE 2023-09-27 16:47:44 +02:00
Hennadii Stepanov
79ef528511 build, macos: Fix qt package build with new Xcode 15 linker 2023-09-27 12:19:57 +01:00
Tim Ruffing
e3720bca39 net: Simplify v2 recv logic by decoupling AAD from state machine 2023-09-27 12:19:54 +02:00
Tim Ruffing
b0f5175c04 net: Drop v2 garbage authentication packet
See also https://github.com/bitcoin/bips/pull/1498

The benefit is a simpler implementation:
 - The protocol state machine does not need separate states for garbage
   authentication and version phases.
 - The special case of "ignoring the ignore bit" is removed.
 - The freedom to choose the contents of the garbage authentication
   packet is removed. This simplifies testing.
2023-09-27 12:19:54 +02:00
Andrew Chow
782701ce7d test: Test loading wallets with conflicts without a chain
Loading a wallet with conflicts without a chain (e.g. wallet tool and
migration) would previously result in an assertion due to -1 being both
a valid number of conflict confirmations, and the indicator that that
member has not been set yet.
2023-09-26 22:28:53 -04:00
Andrew Chow
4660fc82a1 wallet: Check last block and conflict height are valid in MarkConflicted
MarkConflicted calculates conflict confirmations incorrectly when both
the last block processed height and the conflicting height are negative
(i.e. uninitialized). If either are negative, we should not be marking
conflicts and should exit early.
2023-09-26 21:28:20 -04:00
furszy
b3db8c9d5c rpc: bumpfee, improve doc for 'reduce_output' arg
The current argument name and description are dangerous as it don't
describe the case where the user selects the recipient output as the
change address. This one could end up been increased by the inputs
minus outputs remainder. Which, when bumpfee adds new inputs
to the transaction, leads the process to send more coins to the
recipient. Which is not what the user would expect from a
'reduce_output' param naming.

Co-authored-by: Murch <murch@murch.one>
2023-09-26 20:17:02 -03:00
fanquake
c9f288244b Merge bitcoin/bitcoin#28483: refactor: Return CAutoFile from BlockManager::Open*File()
fa56c421be Return CAutoFile from BlockManager::Open*File() (MarcoFalke)
9999b89cd3 Make BufferedFile to be a CAutoFile wrapper (MarcoFalke)
fa389d902f refactor: Drop unused fclose() from BufferedFile (MarcoFalke)

Pull request description:

  This is required for https://github.com/bitcoin/bitcoin/pull/28052, but makes sense on its own, because offloading logic to `CAutoFile` instead of re-implementing it allows to delete code and complexity.

ACKs for top commit:
  TheCharlatan:
    Re-ACK fa56c421be
  willcl-ark:
    tACK fa56c421be

Tree-SHA512: fe4638f3a6bd3f9d968cfb9ae3259c9d6cd278fe2912cbc90289851311c8c781099db4c160e775960975c4739098d9af801a8d2d12603f371f8edfe134d8f85a
2023-09-26 14:01:44 +01:00
fanquake
dcfbf3c210 Merge bitcoin/bitcoin#28512: doc: Be vague instead of wrong about MALLOC_ARENA_MAX
12f7257b8f doc: Be vague instead of wrong about MALLOC_ARENA_MAX (Tim Ruffing)

Pull request description:

  Before this commit, we claim that glibc's malloc implementation uses 2 arenas by default. But that's true only on 32-bit systems, and even there, it uses *up* to 2 arenas.

  This commit fixes the wrong statement. The new statement is intentionally vague to reduce our maintenance burden.

  For details, see:
  https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html#index-glibc_002emalloc_002earena_005fmax

  Noticed in:
  https://github.com/bitcoin/bitcoin/pull/27642#issuecomment-1728103427

ACKs for top commit:
  fanquake:
    ACK 12f7257b8f

Tree-SHA512: c0ff1e35b682a841e366a1cad26e18ff79a93d97103529be35a972c7dcbb95f5354e7a7b98a86731f491434d64685bb58cc3cc9100f0577d8f75db05e951b09a
2023-09-24 18:54:16 +01:00
fanquake
ac9fa6ec78 Merge bitcoin/bitcoin#28385: [refactor] rewrite DisconnectedBlockTransactions to not use boost
4313c77400 make DisconnectedBlockTransactions responsible for its own memory management (glozow)
cf5f1faa03 MOVEONLY: DisconnectedBlockTransactions to its own file (glozow)
2765d6f343 rewrite DisconnectedBlockTransactions as a list + map (glozow)
79ce9f0aa4 add std::list to memusage (glozow)
59a35a7398 [bench] DisconnectedBlockTransactions (glozow)
925bb723ca [refactor] batch-add transactions to DisconnectedBlockTransactions (glozow)

Pull request description:

  Motivation
  - I think it's preferable to use stdlib data structures instead of depending on boost if we can achieve the same thing.
  - Also see #28335 for further context/motivation. This PR simplifies that one.

  Things done in this PR:
  - Add a bench for `DisconnectedBlockTransactions` where we reorg and the new chain has {100%, 90%, 10%} of the same transactions. AFAIU in practice, it's usually close to 100%.
  - Rewrite `DisconnectedBlockTransactions` as a `std::list` + `unordered_map` instead of a boost multi index container.
    - On my machine, the bench suggests the performance is very similar.
  - Move `DisconnectedBlockTransactions` from txmempool.h to its own kernel/disconnected_transactions.h. This struct isn't used by txmempool and doesn't have much to do with txmempool. My guess is that it's been living there for convenience since the boost includes are there.

ACKs for top commit:
  ismaelsadeeq:
    Tested ACK 4313c77400
  stickies-v:
    ACK 4313c77400
  TheCharlatan:
    ACK 4313c77400

Tree-SHA512: 273c80866bf3acd39b2a039dc082b7719d2d82e0940e1eb6c402f1c0992e997256722b85c7e310c9811238a770cfbdeb122ea4babbc23835d17128f214a1ef9e
2023-09-23 18:42:36 +01:00
Andrew Chow
719cb301e6 Merge bitcoin/bitcoin#28492: RPC: descriptorprocesspsbt returns hex encoded tx if complete
a99e9e655a doc: add release note (ismaelsadeeq)
2b4edf889a test: check `descriptorprocesspsbt` return hex encoded tx (ismaelsadeeq)
c405207a18 rpc: `descriptorprocesspsbt` return hex encoded tx (ismaelsadeeq)

Pull request description:

  Coming from [#28414 comment](https://github.com/bitcoin/bitcoin/pull/28414#pullrequestreview-1618684391) Same thing also for `descriptorprocesspsbt`.

  Before this PR `descriptorprocesspsbt` returns a boolean `complete` which indicates that the psbt is final, users then have to call `finalizepsbt` to get the hex encoded network transaction.

  In this PR if the psbt is complete the return object also has the hex encoded network transaction ready for broadcast with `sendrawtransaction`.

  This save users calling `finalizepsbt` with the descriptor, if it is already complete.

ACKs for top commit:
  achow101:
    ACK a99e9e655a
  pinheadmz:
    ACK a99e9e655a
  ishaanam:
    ACK a99e9e655a

Tree-SHA512: c3f1b1391d4df05216c463127cd593f8703840430a99febb54890bc66fadabf9d9530860605f347ec54c1694019173247a0e7a9eb879d3cbb420f9e8d9839b75
2023-09-23 11:55:38 -04:00
Hennadii Stepanov
b000ed5ee5 Merge bitcoin-core/gui#119: Replace send-to-self with dual send+receive entries
099dbe4224 GUI: TransactionRecord: When time/index/etc match, sort send before receive (Luke Dashjr)
2d182f77cd Bugfix: Ignore ischange flag when we're not the sender (Luke Dashjr)
71fbdb7f40 GUI: Remove SendToSelf TransactionRecord type (Luke Dashjr)
f3fbe99fcf GUI: TransactionRecord: Refactor to turn send-to-self into send+receive pairs (Luke Dashjr)
b9765ba1d6 GUI: TransactionRecord: Use "any from me" as the criteria for deciding whether a transaction is a send or receive (Luke Dashjr)

Pull request description:

  Makes the GUI transaction list more like the RPC, and IMO clearer in general.

  As a side effect, this also fixes the GUI entries when a transaction is a net profit to us, but some inputs were also from us.

  Originally https://github.com/bitcoin/bitcoin/pull/15115

  Has Concept ACKs from @*Empact @*jonasschnelli

ACKs for top commit:
  hebasto:
    ACK 099dbe4224.

Tree-SHA512: 7d581add2f59431aa019126d54232a1f15723def5147d7a1b672e9b6d525b6e5a944cc437701aa1bd5bd0fbe557a3d1f4b239337f42bdba4fe1d3960442d0e3b
2023-09-22 18:29:51 +01:00
Hennadii Stepanov
bce7b087cb Merge bitcoin-core/gui#739: Disable and uncheck blank when private keys are disabled
9ea31eba04 gui: Disable and uncheck blank when private keys are disabled (Andrew Chow)

Pull request description:

  Unify the GUI's create wallet with the RPC createwallet so that the blank flag is not set when private keys are disabled.

ACKs for top commit:
  S3RK:
    Code review ACK 9ea31eba04
  jarolrod:
    ACK 9ea31eba04
  pablomartin4btc:
    tACK 9ea31eba04

Tree-SHA512: 0c90dbd77e66f088c6a57711a4b91e254814c4ee301ab703807f281cacd4b08712d2dfeac7661f28bc0e93acc55d486a17b8b4a53ffa57093d992e7a3c51f4e8
2023-09-22 15:26:26 +01:00
Hennadii Stepanov
2e1d4bf444 Merge bitcoin-core/gui#755: Silence -Wcast-function-type warning
befb42f146 qt: Silence `-Wcast-function-type` warning (Hennadii Stepanov)

Pull request description:

  On Fedora 38 @ 8f7b9eb871:
  ```
  $ x86_64-w64-mingw32-g++ --version | head -1
  x86_64-w64-mingw32-g++ (GCC) 12.2.1 20221121 (Fedora MinGW 12.2.1-8.fc38)
  $ ./configure CONFIG_SITE=$PWD/depends/x86_64-w64-mingw32/share/config.site CXXFLAGS="-Wno-return-type -Wcast-function-type"
  $ make > /dev/null
  qt/winshutdownmonitor.cpp: In static member function 'static void WinShutdownMonitor::registerShutdownBlockReason(const QString&, HWND__* const&)':
  qt/winshutdownmonitor.cpp:46:42: warning: cast between incompatible function types from 'FARPROC' {aka 'long long int (*)()'} to 'PSHUTDOWNBRCREATE' {aka 'int (*)(HWND__*, const wchar_t*)'} [-Wcast-function-type]
     46 |     PSHUTDOWNBRCREATE shutdownBRCreate = (PSHUTDOWNBRCREATE)GetProcAddress(GetModuleHandleA("User32.dll"), "ShutdownBlockReasonCreate");
        |                                          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ```

  [Required](https://github.com/bitcoin/bitcoin/pull/25972#issuecomment-1713999563) for https://github.com/bitcoin/bitcoin/pull/25972.

  Picked from https://trac.nginx.org/nginx/ticket/1865.

ACKs for top commit:
  MarcoFalke:
    review ACK befb42f146

Tree-SHA512: b37b2c5dd8702caf84d1833c3511bc46ee14f23b84678b8de0fd04e24e5ecc5fd4d27ba38be0d0b08de91299369f70d4924c895a71fd8e0b6feffcfb7407574a
2023-09-22 12:41:28 +01:00
fanquake
b66f6dcb26 Merge bitcoin/bitcoin#28513: ci: Install Homebrew's pkg-config package
43cd8029fa ci: Install Homebrew's `pkg-config` package (Hennadii Stepanov)

Pull request description:

  Some versions of macOS images lack the `pkg-config` package.

  For example, https://github.com/bitcoin/bitcoin/actions/runs/6248032071/job/16961797066:
  ```
  Runner Image
    Image: macos-13
    Version: 20230417.1
  ```

  ```
  + ./autogen.sh
  configure.ac:16: error: PKG_PROG_PKG_CONFIG macro not found. Please install pkg-config and re-run autogen.sh
  ```

  This PR makes Homebrew install the `pkg-config` package explicitly.

  Also please refer to [macOS Build Guide](https://github.com/bitcoin/bitcoin/blob/master/doc/build-osx.md).

ACKs for top commit:
  kevkevinpal:
    ACK [43cd802](43cd8029fa)
  MarcoFalke:
    lgtm ACK 43cd8029fa
  RandyMcMillan:
    ACK 43cd802

Tree-SHA512: 2b934b22e5f6748634089e0525b92219484e37b2afc11e9fd4c53faf112b33ca1c8deb5b4aa36939bf5c4807e7599d4aabae6335317ecc5d4a4d562bbd7dbdf2
2023-09-21 16:32:59 +00:00
fanquake
f29091410d Merge bitcoin/bitcoin#28379: Refactor: Remove m_is_test_chain
78c2707b2a Refactor: Replace 'isMockableChain' with inline 'ChainType' check for 'submitpackage' (Tim Neubauer)
27b4084e16 Refactor: Remove m_is_test_chain (Tim Neubauer)

Pull request description:

  Remove the m_is_test_chain bool
  Compiled and run tests locally

  #28376

ACKs for top commit:
  MarcoFalke:
    re-ACK 78c2707b2a
  ajtowns:
    ACK 78c2707b2a

Tree-SHA512: 2eedd855c379dd12b7ff28b0e03414680cc892313f16502f36e09906513df9c222e8cc2cea3ff4d9a4f47c9efdfa00d017f38398021b0c96d4543711206d6ff8
2023-09-21 16:31:05 +00:00
Andrew Chow
2303fd2f43 Merge bitcoin/bitcoin#28471: Fix virtual size limit enforcement in transaction package context
eb8f58f5e4 Add functional test to catch too large vsize packages (Greg Sanders)
1a579f9d01 Handle over-sized (in virtual bytes) packages with no in-mempool ancestors (Greg Sanders)
bc013fe8e3 Bugfix: Pass correct virtual size to CheckPackageLimits (Luke Dashjr)
533660c58a Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT to avoid vbyte confusion (Greg Sanders)

Pull request description:

  (Alternative) Minimal subset of https://github.com/bitcoin/bitcoin/pull/28345 to:

  1) Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT which accounts for additional WU necessary to not exclude default chain limit transactions that would have been accepted individually. Avoids sigops vbyte confusion.
  2) pass correct vsize to chain limit evaluations in package context
  3) stop overly-large packages that have no existing mempool ancestors (also a bugfix by itself if someone sets non-standard chain limits)

  This should fix the known issues while not blocking additional refactoring later.

ACKs for top commit:
  achow101:
    ACK eb8f58f5e4
  ariard:
    Re-Code ACK eb8f58f5e
  glozow:
    reACK eb8f58f5e4

Tree-SHA512: 1b5cca1a526207e25d387fcc29a776a3198c3a013dc2b35c6275b9d5a64db2476c154ebf52e3a1aed0b9924c75613f21a946577aa760de28cadf0c9c7f68dc39
2023-09-21 12:13:52 -04:00
fanquake
cf0711cac3 Merge bitcoin/bitcoin#27934: test: added coverage to estimatefee
d05be124db test: added coverage to estimatefee (kevkevin)

Pull request description:

  Added a assert for an rpc error when we try to estimate fee for the max conf_target

  Line I am adding coverage to https://github.com/bitcoin/bitcoin/blob/master/src/rpc/fees.cpp#LL71C52-L71C52

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d05be124db

Tree-SHA512: dfab075989446e33d1a5ff1a308f1ba1b9f80cce3848fbe4231f69212ceef456a3f2b19365a42123e0397c31893fd9f1fd9973cc00cfbb324386e12ed0e6bccc
2023-09-21 15:50:15 +00:00
Andrew Chow
41cb17fdb6 Merge bitcoin/bitcoin#28078: net, refactor: remove unneeded exports, use helpers over low-level code, use optional
4ecfd3eaf4 Inline short, often-called, rarely-changed basic CNetAddr getters (Jon Atack)
5316ae5dd8 Convert GetLocal() to std::optional and remove out-param (Jon Atack)
f1304db136 Use higher-level CNetAddr and CNode helpers in net.cpp (Jon Atack)
07f5891588 Add CNetAddr::IsPrivacyNet() and CNode::IsConnectedThroughPrivacyNet() (Jon Atack)
df488563b2 GetLocal() type-safety, naming, const, and formatting cleanups (stickies-v)
fb4265747c Add and use CNetAddr::HasCJDNSPrefix() helper (Jon Atack)
5ba73cd0ee Move GetLocal() declaration from header to implementation (Jon Atack)
11426f6557 Move CaptureMessageToFile() declaration from header to implementation (Jon Atack)
deccf1c484 Move IsPeerAddrLocalGood() declaration from header to implementation (Jon Atack)

Pull request description:

  and other improvements noticed while reviewing #27411.

  Addresses https://github.com/bitcoin/bitcoin/pull/27411#discussion_r1263969104 and https://github.com/bitcoin/bitcoin/pull/27411#discussion_r1263967598.

  See commit messages for details.

ACKs for top commit:
  achow101:
    ACK 4ecfd3eaf4
  vasild:
    ACK 4ecfd3eaf4
  stickies-v:
    ACK 4ecfd3eaf4

Tree-SHA512: d792bb2cb24690aeae9bedf97e92b64fb6fd080c39385a4bdb8ed05f37f3134d85bda99da025490829c03017fd56382afe7951cdd039aede1c08ba98fb1aa7f9
2023-09-21 11:26:16 -04:00
Andrew Chow
5027d41988 Merge bitcoin/bitcoin#26366: rpc, test: addnode improv + add test coverage for invalid command
f52cb02f70 doc: make it clear that `node` in `addnode` refers to the node's address (brunoerg)
effd1efefb test: `addnode` with an invalid command should throw an error (brunoerg)
56b27b8487 rpc, refactor: clean-up `addnode` (brunoerg)

Pull request description:

  This PR:

  - Adds test coverage for an invalid `command` in `addnode`.
  - Rename `test_getaddednodeinfo` to `test_addnode_getaddednodeinfo` and its log since this function also tests `addnode` and it doesn't worth to split into 2 ones.
  - Makes it clear in docs that `node` in `addnode` refers to the node's address. It seemed a little weird for me "The node (see getpeerinfo for nodes)", it could mean a lot of things e.g. the node id.
  - Some small improv/clean-up: use `const` where possible, rename some vars, and remove the check for nullance for `command` since it's a non-optional field.

ACKs for top commit:
  achow101:
    ACK f52cb02f70
  jonatack:
    ACK f52cb02f70
  theStack:
    re-ACK f52cb02f70

Tree-SHA512: e4a69e58b784e233463945b4d55a401957f9fe4562c129f59216a44f44fb3221d3449ac578fb35e665ca654c6ade2e741b72c3df78040f7527229c77b6c5b82e
2023-09-21 06:35:16 -04:00
Hennadii Stepanov
1d4846a844 Merge bitcoin-core/gui#738: Add menu option to migrate a wallet
48aae2cffe gui: Add File > Migrate Wallet (Andrew Chow)
577be889cd gui: Optionally return passphrase after unlocking (Andrew Chow)
5b3a85b4c6 interfaces, wallet: Expose migrate wallet (Andrew Chow)

Pull request description:

  GUI users need to be able to migrate wallets without going to the RPC console.

ACKs for top commit:
  jarolrod:
    ACK 48aae2cffe
  pablomartin4btc:
    tACK 48aae2cffe
  hebasto:
    ACK 48aae2cffe

Tree-SHA512: 2d02b1e85e7d6cfbf503f417f150cdaa0c63822942e9a6fe28c0ad3e7f40a957bb01a375c909a60432dc600e84574881aa446c7ec983b56f0bb23f07ef15de54
2023-09-20 22:32:31 +01:00
Hennadii Stepanov
43cd8029fa ci: Install Homebrew's pkg-config package
Some versions of macOS images lack the 'pkg-config' package, which is
required for the build process.
2023-09-20 21:49:57 +01:00
Hennadii Stepanov
ce6df7df9b bench: Add SHA256 implementation specific benchmarks 2023-09-20 21:12:00 +01:00
Hennadii Stepanov
5f72417176 Add ability to specify SHA256 implementation for benchmark purposes 2023-09-20 21:11:55 +01:00
Andrew Chow
8247a8db69 Merge bitcoin/bitcoin#28154: test: refactor: deduplicate segwitv0 ECDSA signing for tx inputs
83d7cfd542 test: refactor: deduplicate segwitv0 ECDSA signing for tx inputs (Sebastian Falbesoner)

Pull request description:

  This PR is a simple follow-up for #28025. It introduces a `signing_input_segwitv0` helper in order to deduplicate the following steps needed to create a segwitv0 ECDSA signature:
  1. calculate the `SegwitV0SignatureHash` with the desired sighash type
  2. create the actual digital signature by calling ECKey.sign_ecdsa on the signature message hash calculated above
  3. put the DER-encoded result (plus sighash byte) at the bottom of the witness stack

ACKs for top commit:
  achow101:
    ACK 83d7cfd542
  pinheadmz:
    code review ACK at 83d7cfd542

Tree-SHA512: b8e55409ddc9ddb14cfc06daeb4730d7750a4632f175f88dcac6ec4d216e71fd4a7eee325a64d6ebba3b33be50bcd30c2de7400f834c01abb67e52840d9823b6
2023-09-20 13:50:15 -04:00
Tim Ruffing
12f7257b8f doc: Be vague instead of wrong about MALLOC_ARENA_MAX
Before this commit, we claim that glibc's malloc implementation uses 2
arenas by default. But that's true only on 32-bit systems, and even
there, it uses *up* to 2 arenas.

This commit fixes the wrong statement. The new statement is
intentionally vague to reduce our maintenance burden.

For details, see:
https://www.gnu.org/software/libc/manual/html_node/Memory-Allocation-Tunables.html#index-glibc_002emalloc_002earena_005fmax

Noticed in:
https://github.com/bitcoin/bitcoin/pull/27642#issuecomment-1728103427
2023-09-20 17:12:24 +00:00
Sebastian Falbesoner
96b3f2dbe4 test: add unit test coverage for Python ECDSA implementation 2023-09-20 18:19:29 +02:00
fanquake
99ce8366ed Merge bitcoin/bitcoin#28504: ci: Use nproc over MAKEJOBS in 01_base_install
fa3b5e5e57 ci: Use nproc over MAKEJOBS in 01_base_install (MarcoFalke)

Pull request description:

  Currently `$MAKEJOBS` is the default value in `01_base_install.sh` when building the container image.

  This problem can't be fixed (see below), so just use `nproc` for now.

  Other solutions would be bad:

  * Passing in the `MAKEJOBS` as a dockerfile env would create a new image if the number of tasks are changed, seems verbose and confusing.
  * Leaving `master` as-is would leave CPUs unused if there are more than `4`.

ACKs for top commit:
  fanquake:
    ACK - I think this is fine as-is, it's an improvement over the current state fa3b5e5e57

Tree-SHA512: 15014eb7b548945737b0fed5cd46f0cd4b72b1d54d044f60fce628f914053a2de6518878428a54822d236d08d6f257d8c464d3fb589400f4be56022d2ec8676d
2023-09-20 16:14:30 +00:00
Greg Sanders
eb8f58f5e4 Add functional test to catch too large vsize packages 2023-09-20 11:33:53 -04:00
Greg Sanders
1a579f9d01 Handle over-sized (in virtual bytes) packages with no in-mempool ancestors 2023-09-20 10:34:09 -04:00
Andrew Chow
ff564c75e7 Merge bitcoin/bitcoin#27511: rpc: Add test-only RPC getaddrmaninfo for new/tried table address count
28bac81a34 test: add functional test for getaddrmaninfo (stratospher)
c8eb8dae51 rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table (stratospher)

Pull request description:

  implements https://github.com/bitcoin/bitcoin/issues/26907. split off from #26988 to keep RPC, CLI discussions separate.

  This PR introduces a new RPC `getaddrmaninfo`which returns the count of addresses in the new/tried table of a node's addrman broken down by network type. This would be useful for users who want to see the distribution of addresses from different networks across new/tried table in the addrman.

  ```jsx
  $ getaddrmaninfo

  Result:
  {                   (json object) json object with network type as keys
    "network" : {     (json object) The network (ipv4, ipv6, onion, i2p, cjdns)
      "new" : n,      (numeric) number of addresses in new table
      "tried" : n,    (numeric) number of addresses in tried table
      "total" : n     (numeric) total number of addresses in both new/tried tables from a network
    },
    ...
  }
  ```

  ### additional context from [original PR](https://github.com/bitcoin/bitcoin/pull/26988)

  1. network coverage tests were skipped because there’s a small chance that addresses from different networks could hash to the same bucket and cause count of different network addresses in the tests to fail. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1137596851.
  2. #26988 uses this RPC in -addrinfo CLI. Slight preference for keeping the RPC hidden since this info will mostly be useful to only super users. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1173964808.

ACKs for top commit:
  0xB10C:
    ACK 28bac81a34
  willcl-ark:
    reACK 28bac81a34
  achow101:
    ACK 28bac81a34
  brunoerg:
    reACK 28bac81a34
  theStack:
    Code-review ACK 28bac81a34

Tree-SHA512: 346390167e1ebed7ca5c79328ea452633736aff8b7feefea77460e04d4489059334ae78a3f757f32f5fb7827b309d7186bebab3c3760b3dfb016d564a647371a
2023-09-20 08:25:20 -04:00
Luke Dashjr
bc013fe8e3 Bugfix: Pass correct virtual size to CheckPackageLimits 2023-09-20 08:13:18 -04:00
Greg Sanders
533660c58a Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT to avoid vbyte confusion
While allowing submitted packages to be slightly larger than what
may be allowed in the mempool to allow simpler reasoning
about contextual-less checks vs chain limits.
2023-09-20 08:10:30 -04:00
Andrew Chow
3966b0a0b6 Merge bitcoin/bitcoin#28472: Remove MemPoolAccept::m_limits to avoid mutating it in package evaluation
ee589d4466 Add regression test for m_limit mutation (Greg Sanders)
275579d8c1 Remove MemPoolAccept::m_limits, only have local copies for carveouts (Greg Sanders)

Pull request description:

  Without remoing it, if we ever call `PreChecks()` multiple times for any reason during any one `MempoolAccept`, subsequent invocations may have incorrect limits, allowing longer/larger chains than should be allowed.

  Currently this is only an issue with `submitpackage`, so this is not exposed on mainnet.

ACKs for top commit:
  achow101:
    ACK ee589d4466
  glozow:
    ACK ee589d4466, nits can be ignored
  ariard:
    Code Review ACK ee589d446

Tree-SHA512: 14cf8edc73e014220def82563f5fb4192d1c2c111829712abf16340bfbfd9a85e2148d723af6fd4995d503dd67232b48dcf8b1711668d25b5aee5eab1bdb578c
2023-09-20 07:49:13 -04:00
fanquake
e9a4793b82 Merge bitcoin/bitcoin#28432: build: Produce a .zip for macOS distribution
b5790c35f7 build: remove dmg dependencies (fanquake)
33ae0bd1e4 macdeploy: remove DMG generation from deploy script (fanquake)
a128111c29 build: produce a .zip for macOS distribution (Hennadii Stepanov)
c38561d6b1 build: add -zip option to macdeployqtplus (fanquake)

Pull request description:

  It is https://github.com/bitcoin/bitcoin/pull/27099 revived with addressed [comments](https://github.com/bitcoin/bitcoin/pull/27099#issuecomment-1708705686).

  From https://github.com/bitcoin/bitcoin/pull/27099#issue-1584429885:
  > Reviving the discussion around using a `.zip` for the distributed macOS binaries, as opposed to a `.dmg`.
  >
  > Given we only had a single report of the "no finder window" issue (#26176), I wonder if that means macOS users were able to figure it out, they gave up/didn't report, or, we just have very few macOS users.
  >
  > Related to #18128.

  That's how it looks on macOS:

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/baa637bb-256b-4b24-8645-8c2754c2ae64)

ACKs for top commit:
  Sjors:
    tACK b5790c35f7
  jarolrod:
    ACK b5790c35f7
  TheCharlatan:
    utACK b5790c35f7

Tree-SHA512: 6e9cb3ab0f60f8a92bfec50577e8d096c5b23ec09ebbb334826415609140ddc96d470aea37379495c1c6bb1beec0d306b09460f62e1543bb0f4396c10a1dfbe2
2023-09-20 11:40:47 +00:00
fanquake
1bf915db99 Merge bitcoin/bitcoin#28470: fuzz: Rework addr fuzzing
fad52baf1e fuzz: Rework addr fuzzing (MarcoFalke)
fa5b6d29ee fuzz: Drop unused params from serialize helpers (MarcoFalke)

Pull request description:

  Some minor fixups to addr fuzzing

ACKs for top commit:
  dergoegge:
    utACK fad52baf1e

Tree-SHA512: 6a2b07fb1a65cf855d5e7c0a52bfcb81d46dbc5d4b3e72cef359987cbd28dbfeb2fc54f210e9737cb131b40ac5f88a90e9af284e441e0b37196121590bbaf015
2023-09-20 11:38:56 +00:00
Andrew Chow
abe4fedab7 Merge bitcoin/bitcoin#28125: wallet: bugfix, disallow migration of invalid scripts
8e7e3e6149 test: wallet, verify migration doesn't crash for an invalid script (furszy)
1de8a2372a wallet: disallow migration of invalid or not-watched scripts (furszy)

Pull request description:

  Fixing #28057.

  The legacy wallet allows to import any raw script (#28126), without
  checking if it was valid or not. Appending it to the watch-only set.

  This causes a crash in the migration process because we are only
  expecting to find valid scripts inside the legacy spkm.

  These stored scripts internally map to `ISMINE_NO` (same as if they
  weren't stored at all..).

  So we need to check for these special case, and take into account that
  the legacy spkm could be storing invalid not watched scripts.

  Which, in code words, means `IsMineInner()` returning
  `IsMineResult::INVALID` for them.

  Note:
  To verify this, can run the test commit on top of master.
  `wallet_migration.py` will crash without the bugfix commit.

ACKs for top commit:
  achow101:
    ACK 8e7e3e6149

Tree-SHA512: c2070e8ba78037a8f573b05bf6caa672803188f05429adf5b93f9fc1493faedadecdf018dee9ead27c656710558c849c5da8ca5f6f3bc9c23b3c4275d2fb50c7
2023-09-19 13:10:57 -04:00
stratospher
28bac81a34 test: add functional test for getaddrmaninfo 2023-09-19 22:38:56 +05:30
fanquake
53313c49d6 Merge bitcoin/bitcoin#28246: wallet: Use CTxDestination in CRecipient instead of just scriptPubKey
ad0c469d98 wallet: Use CTxDestination in CRecipient rather than scriptPubKey (Andrew Chow)
07d3bdf4eb Add PubKeyDestination for P2PK scripts (Andrew Chow)
1a98a51c66 Allow CNoDestination to represent a raw script (Andrew Chow)
8dd067088d Make WitnessUnknown members private (Andrew Chow)

Pull request description:

  For silent payments, we want to provide a `SilentPaymentsDestination` to be used as the recipient, which requires `CRecipient` to use something other than just the `scriptPubKey` as we cannot know the output script for a silent payment prior to transaction creation. `CTxDestination` seems like the obvious place to add a `SilentPaymentsDestination` as it is our internal representation of an address.

  In order to still allow paying to arbitrary scriptPubKeys (e.g. for data carrier outputs, or the user hand crafted a raw transaction that they have given to `fundrawtransaction`), `CNoDestination` is changed to contain raw scripts.

  Additionally, P2PK scripts are now interpreted as a new `PubKeyDestination` rather than `PKHash`. This results in some things that would have given an address for P2PK scripts to no longer do so. This is arguably more correct.

  `ExtractDestination`'s behavior is slightly changed for the above. It now returns `true` for those destinations that have addresses, so P2PK scripts now result in `false`. Even though it returns false for `CNoDestination`, the script will now be included in that `CNoDestination`.

  Builds on #28244

ACKs for top commit:
  josibake:
    ACK ad0c469d98

Tree-SHA512: ef3f8f3c7284779d9806c77c85b21caf910a79a1f7e7f1b51abcc0d7e074f14e00abf30f625a13075e41d94dad6202c10ddff462c0ee74c2ca4aab585b145a52
2023-09-19 16:48:43 +00:00
MarcoFalke
fa3b5e5e57 ci: Use nproc over MAKEJOBS in 01_base_install 2023-09-19 16:13:27 +00:00
MarcoFalke
fac29a0ab1 Remove SER_GETHASH, hard-code client version in CKeyPool serialize
It was never set, so it can be removed along with any code reading it.
2023-09-19 16:11:59 +00:00
fanquake
737aac8cc8 Merge bitcoin/bitcoin#28497: ci: Reintroduce fixed "test-each-commit" job
27b636a921 ci: Reintroduce fixed "test-each-commit" job (Hennadii Stepanov)

Pull request description:

  This is a fixed version of https://github.com/bitcoin/bitcoin/pull/28279:
  > Currently, if a pull request has more than one commit, previous commits may fail to compile, or may fail the tests. This is problematic, because it breaks git-bisect, or worse.
  >
  > Fix this by adding a CI task for this.

  The new job checks at most 6 commits of a pull request, excluding the top one.

  The maximum number of tested commits is 6, which derives from the time [constrains](https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes).

  For historical context, please see:
  - https://github.com/bitcoin/bitcoin/pull/28279
  - https://github.com/bitcoin/bitcoin/pull/28477
  - https://github.com/bitcoin/bitcoin/pull/28478

  **A note for reviewers:** To test scripts locally, ensure that you works with a _shallow_ copy of the repo.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 27b636a921

Tree-SHA512: 0c69ced13509fa0ed2dd6ef13f4c710d678e31b294b6318b59ab1ba899086a71b5c893aaf70e143036349329167bf8e16bdca319b2c761e2aef6222d0db1470c
2023-09-19 16:09:12 +00:00
fanquake
a5979a8d4a Merge bitcoin/bitcoin#28506: fuzz: Add missing PROVIDE_FUZZ_MAIN_FUNCTION guard to __AFL_FUZZ_INIT
fa33b2c889 fuzz: Add missing PROVIDE_FUZZ_MAIN_FUNCTION guard to __AFL_FUZZ_INIT (MarcoFalke)

Pull request description:

  Should fix https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=62455

ACKs for top commit:
  dergoegge:
    utACK fa33b2c889

Tree-SHA512: 735926f7f94ad1c3c5dc0fc62a2ef3a85abae25f4fe1e7654c2857ce3e867667ed28da58ab36281d730d3e206a0728cb429671ea5d3ccd11519e637eb191f70d
2023-09-19 15:13:26 +00:00
MarcoFalke
fa72f09d6f Remove CHashWriter type
The type is only ever set, but never read via GetType(), so remove it.
Also, remove SerializeHash to avoid silent merge conflicts and use the
already existing GetHash() boilerplate consistently.
2023-09-19 16:38:08 +02:00
MarcoFalke
fa4a9c0f43 Remove unused GetType() from OverrideStream, CVectorWriter, SpanReader
GetType() is never called, so it is completely unused and can be
removed.
2023-09-19 14:19:57 +00:00
Greg Sanders
ee589d4466 Add regression test for m_limit mutation 2023-09-19 09:30:58 -04:00
Erik Arvstedt
360b917674 contrib/bash-completions: use package naming conventions
This naming scheme supports auto-detection and on-demand loading of completions.

See
ba109693ee/README.md (faq),
section "Where should I put it to be sure that interactive bash shells will find it and source it".

Previously, distro package maintainers had to rename these files manually.
2023-09-19 13:45:22 +02:00
MarcoFalke
fa33b2c889 fuzz: Add missing PROVIDE_FUZZ_MAIN_FUNCTION guard to __AFL_FUZZ_INIT 2023-09-19 13:41:24 +02:00
stratospher
c8eb8dae51 rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table 2023-09-19 16:02:55 +05:30
Hennadii Stepanov
27b636a921 ci: Reintroduce fixed "test-each-commit" job
The new job checks at most 6 commits of a pull request, excluding the
top one.
2023-09-19 09:36:53 +01:00
fanquake
f01416e23c Merge bitcoin/bitcoin#28479: build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS
4a825039a5 build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS (fanquake)

Pull request description:

  `_LIBCPP_ENABLE_ASSERTIONS` is deprecated, and will be removed. [See (from libc++ __config in main)](b57df9fe9a/libcxx/include/__config (L205-L209)):

  > TODO(hardening): remove this in LLVM 19.
  > This is for backward compatibility -- make enabling `_LIBCPP_ENABLE_ASSERTIONS` (which predates hardening modes)
  > equivalent to setting the safe mode.
  > ifdef _LIBCPP_ENABLE_ASSERTIONS
  > warning "_LIBCPP_ENABLE_ASSERTIONS is deprecated, please use _LIBCPP_ENABLE_SAFE_MODE instead."

  From LLVM 17, `_LIBCPP_ENABLE_DEBUG_MODE` can be used instead, which also performs more checks than safe mode:

  > Enables the debug mode which contains all the checks from the hardened mode and additionally more expensive checks that may affect the complexity of algorithms. The debug mode is intended to be used for testing, not in production. Mutually exclusive with `_LIBCPP_ENABLE_HARDENED_MODE` and `_LIBCPP_ENABLE_SAFE_MODE`.

  See https://libcxx.llvm.org/Hardening.html.

  Related to #28476.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4a825039a5 🙏

Tree-SHA512: ca52603f86214e8e9350bd2b2baa44fbde0f72f1b186da7aecd8690256dff5b2be75fe89383158298a6f683bbd6ae0dff528d2ba4cc5ece1f56cfbdee0e1dc5d
2023-09-16 12:16:46 +01:00
fanquake
372e7b6510 Merge bitcoin/bitcoin#28489: tests: fix incorrect assumption in v2transport_test
3f4e1bb9ae tests: fix incorrect assumption in v2transport_test (Pieter Wuille)

Pull request description:

  One part of the current `v2transport_test` introduced in #28196 assumes that if a bit gets modified in a message, failure should instantly be detected after sending that message. This is not correct in case the length descriptor is modified, as that may cause the receiver to need more data first. Fix this by sending more messages until failure actually occurs.

  Discovered in https://github.com/bitcoin/bitcoin/pull/27495#issuecomment-1719934041.

ACKs for top commit:
  theStack:
    ACK 3f4e1bb9ae

Tree-SHA512: faa90bf91996cbaaef62d764e746cb222eaf6796316b0d0e13709e528750b7c0ef09172f7fecfe814dbb8c136c5259f65ca1ac79318e6768a0bfc4e626a63249
2023-09-16 12:15:16 +01:00
ismaelsadeeq
a99e9e655a doc: add release note 2023-09-15 16:53:59 +01:00
ismaelsadeeq
2b4edf889a test: check descriptorprocesspsbt return hex encoded tx
Test that if the processed psbt is complete the hex encoded tx
is returned and remove unneccessary rpc call to finalize the
psbt.
2023-09-15 16:48:36 +01:00
ismaelsadeeq
c405207a18 rpc: descriptorprocesspsbt return hex encoded tx
If processed psbt is complete return hex encoded network
transaction in the output.
2023-09-15 16:46:09 +01:00
Tim Neubauer
78c2707b2a Refactor: Replace 'isMockableChain' with inline 'ChainType' check for 'submitpackage' 2023-09-15 16:32:11 +02:00
fanquake
5c7cdda992 Merge bitcoin/bitcoin#28473: refactor: Serialization parameter cleanups
fb6a2ab63e scripted-diff: use SER_PARAMS_OPFUNC (Anthony Towns)
5e5c8f86b6 serialize: add SER_PARAMS_OPFUNC (Anthony Towns)
33203f59b4 serialize: specify type for ParamsWrapper not ref (Anthony Towns)
bf147bfffa serialize: move ser_action functions out of global namespace (Anthony Towns)

Pull request description:

  Cleanups after #25284:

   * ser_action namespacing - https://github.com/bitcoin/bitcoin/pull/25284#discussion_r1316189977
   * make reference implicit - https://github.com/bitcoin/bitcoin/pull/25284#discussion_r1316277030
   * function notation - https://github.com/bitcoin/bitcoin/pull/25284#issuecomment-1710714821

ACKs for top commit:
  MarcoFalke:
    lgtm ACK fb6a2ab63e 💨
  TheCharlatan:
    ACK fb6a2ab63e

Tree-SHA512: aacca2ee9cfec360ade6b394606e13d1dfe05bc29c5fbdd48a4e6992bd420312d4ed0d32218d95c560646af326e9977728dc2e759990636298e326947f6f9526
2023-09-15 14:27:20 +01:00
fanquake
b5790c35f7 build: remove dmg dependencies 2023-09-15 13:47:50 +01:00
fanquake
33ae0bd1e4 macdeploy: remove DMG generation from deploy script 2023-09-15 13:47:50 +01:00
Hennadii Stepanov
a128111c29 build: produce a .zip for macOS distribution
Instead of a .dmg.

Co-authored-by: fanquake <fanquake@gmail.com>
2023-09-15 13:47:50 +01:00
fanquake
c38561d6b1 build: add -zip option to macdeployqtplus
This zips the app bundle in /dist.
2023-09-15 13:47:49 +01:00
MarcoFalke
fa56c421be Return CAutoFile from BlockManager::Open*File()
This is a refactor.
2023-09-15 14:34:24 +02:00
MarcoFalke
9999b89cd3 Make BufferedFile to be a CAutoFile wrapper
This refactor allows to forward some calls to the underlying CAutoFile,
instead of re-implementing the logic in the buffered file.
2023-09-15 14:34:17 +02:00
MarcoFalke
fa389d902f refactor: Drop unused fclose() from BufferedFile
This was only explicitly used in the tests, where it can be replaced by
wrapping the original raw file pointer into a CAutoFile on creation and
then calling CAutoFile::fclose().

Also, it was used in LoadExternalBlockFile(), where it can also be
replaced by the (implicit call to the) CAutoFile destructor after
wrapping the original raw file pointer in a CAutoFile.
2023-09-15 14:33:51 +02:00
Pieter Wuille
3f4e1bb9ae tests: fix incorrect assumption in v2transport_test 2023-09-15 07:18:13 -04:00
fanquake
717a4d8944 Merge bitcoin/bitcoin#28476: ci: LLVM 17 for MSAN jobs
a241d6069c ci: use LLVM 17.0.0 in MSAN jobs (fanquake)

Pull request description:

  See https://libcxx.llvm.org/Hardening.html as well as https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.

ACKs for top commit:
  MarcoFalke:
    review ACK a241d6069c

Tree-SHA512: c374dabf307fe762be0da96f63695a150f6018c1468fe9414fad23f74f5818bbf7a5a699e109084e31467482a900cfebf1d5835821e4da94aa310b2c9570749c
2023-09-15 11:49:29 +01:00
fanquake
f608a409f7 Merge bitcoin/bitcoin#28480: fuzz: Don't use afl++ deferred forkserver mode
508d05f8a7 [fuzz] Don't use afl++ deferred forkserver mode (dergoegge)

Pull request description:

  Fixes #28469

  This makes our afl++ harness essentially behave like libFuzzer, with the exception that the whole program does fully reset every 100000 iterations. 100000 is somewhat arbitrary and we could also go with `std::numeric_limits<unsigned in>::max()` but a smaller limit does allow for the occasional reset to counter act some amount of instability in the fuzzing loop (e.g. non-determinism, statefulness).

  It's a bit of a shame to do this just for the targets whose initial state can't be forked (e.g. threads) because other targets do benefit from not having to redo the state setup. An alternative would be https://github.com/bitcoin/bitcoin/issues/28469#issuecomment-1717526774:
  ```
  If the goal is to be maximally performant, the fork would need to happen for each fuzz target specifically.
  I guess it can be achieved by wrapping __AFL_INIT(); into a helper function and then require all fuzz
  target initialize() to call it?
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 508d05f8a7

Tree-SHA512: d9fe94e2e3198795f8fb58f67eb383531a534bcd4ec75a1f0ae6ccb5531863dbc09800bb7d77536417745c4c8bc49a4f84dcc959918b27d4997a270eeacb0e7e
2023-09-15 10:16:26 +01:00
fanquake
8ef672937e Merge bitcoin/bitcoin#28452: Do not use std::vector = {} to release memory
3fcd7fc7ff Do not use std::vector = {} to release memory (Pieter Wuille)

Pull request description:

  It appears that invoking `v = {};` for an `std::vector<...> v` is equivalent to `v.clear()`, which does not release its allocated memory. There are a number of places in the codebase where it appears to be used for that purpose however (mostly written by me). Replace those with `std::vector<...>{}.swap(v);` (using a helper function `ClearShrink` in util/vector.h).

  To explain what is going on: `v = {...};` is equivalent in general to `v.operator=({...});`. For many types, the `{}` is converted to the type of `v`, and then assigned to `v` - which for `std::vector` would ordinarily have the effect of clearing its memory (constructing a new empty vector, and then move-assigning it to `v`). However, since `std::vector<T>` has an `operator=(std::initializer_list<T>)` defined, it has precedence (since no implicit conversion is needed), and with an empty list, that is equivalent to `clear()`.

  I did consider using `v = std::vector<T>{};` as replacement for `v = {};` instances where memory releasing is desired, but it appears that it does not actually work universally either. `V{}.swap(v);` does.

ACKs for top commit:
  ajtowns:
    utACK 3fcd7fc7ff
  stickies-v:
    ACK 3fcd7fc7ff
  theStack:
    Code-review ACK 3fcd7fc7ff

Tree-SHA512: 6148558126ec3c8cfd6daee167ec1c67b360cf1dff2cbc132bd71768337cf9bc4dda3e5a9cf7da4f7457d2123288eeba77dd78f3a17fa2cfd9c6758262950cc5
2023-09-15 10:04:41 +01:00
Andrew Chow
459272d639 Merge bitcoin/bitcoin#26152: Bump unconfirmed ancestor transactions to target feerate
f18f9ef4d3 Amend bumpfee for inputs with overlapping ancestry (Murch)
2e35e944da Bump unconfirmed parent txs to target feerate (Murch)
3e3e052411 coinselection: Move GetSelectionWaste into SelectionResult (Andrew Chow)
c57889da66 [node] interface to get bump fees (glozow)
c24851be94 Make MiniMinerMempoolEntry fields private (Murch)
ac6030e4d8 Remove unused imports (Murch)
d2f90c31ef Fix calculation of ancestor set feerates in test (Murch)
a1f7d986e0 Match tx names to index in miniminer overlap test (Murch)

Pull request description:

  Includes some commits to address follow-ups from #27021: https://github.com/bitcoin/bitcoin/pull/27021#issuecomment-1554675156

  Reduces the effective value of unconfirmed UTXOs by the fees necessary to bump their ancestor transactions to the same feerate.

  While the individual UTXOs always account for their full ancestry before coin-selection, we can correct potential overestimates with a second pass where we establish the ancestry and bump fee for the whole input set collectively.

  Fixes #9645
  Fixes #9864
  Fixes #15553

ACKs for top commit:
  S3RK:
    ACK f18f9ef4d3
  ismaelsadeeq:
    ACK f18f9ef4d3
  achow101:
    ACK f18f9ef4d3
  brunoerg:
    crACK f18f9ef4d3
  t-bast:
    ACK f18f9ef4d3, I reviewed the latest changes and run e2e tests against eclair, everything looks good 👍

Tree-SHA512: b65180c4243b1f9d13c311ada7a1c9f2f055d530d6c533b78c2068b50b8c29ac1321e89e85675b15515760d4f1b653ebd9da77b37c7be52d9bc565a3538f0aa6
2023-09-14 16:08:37 -04:00
Greg Sanders
275579d8c1 Remove MemPoolAccept::m_limits, only have local copies for carveouts 2023-09-14 13:32:01 -04:00
Andrew Chow
541976b42e Merge bitcoin/bitcoin#27850: test: Add unit & functional test coverage for blockstore
de8f9123af test: cover read-only blockstore (Matthew Zipkin)
5c2185b3b6 ci: enable chattr +i capability inside containers (Matthew Zipkin)
e573f24202 unit test: add coverage for BlockManager (Matthew Zipkin)

Pull request description:

  This PR adds unit and functional tests to cover the behavior described in #2039. In particular, that bitcoind will crash on startup if a reindex is requested but the `blk` files are read-only. Eventually this behavior can be updated with https://github.com/bitcoin/bitcoin/pull/27039. This PR just commits the test coverage from #27039 as suggested in https://github.com/bitcoin/bitcoin/pull/27039#issuecomment-1584915782

ACKs for top commit:
  jonatack:
    ACK de8f9123af modulo suggestions in https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1319010039, tested on macOS, but not on Linux for the Linux-related change in the last push
  achow101:
    ACK de8f9123af
  MarcoFalke:
    lgtm ACK de8f9123af 📶

Tree-SHA512: b9bd684035dcea11c901b649fc39f397a2155a9a8459f3348e67947e387e45312fddeccb52981aef486f8a31deebb5356a7901c1bb94b78f82c24192a369af73
2023-09-14 13:21:14 -04:00
MarcoFalke
fa6e6a3f03 doc: Remove confusing assert linter 2023-09-14 18:59:52 +02:00
Matthew Zipkin
de8f9123af test: cover read-only blockstore
Co-authored-by: Andrew Chow <github@achow101.com>
2023-09-14 12:02:01 -04:00
dergoegge
508d05f8a7 [fuzz] Don't use afl++ deferred forkserver mode
Deferring the forkserver initialization doesn't make sense for some of
our targets since they involve state that can't be forked (e.g.
threads). We therefore remove the use of __AFL_INIT entirely.

We also increase the __AFL_LOOP count to 100000. Our fuzz targets are
meant to all be deterministic and stateless therefore this should be
fine.
2023-09-14 16:58:19 +01:00
fanquake
4a825039a5 build: use _LIBCPP_ENABLE_DEBUG_MODE over ENABLE_ASSERTIONS
`_LIBCPP_ENABLE_ASSERTIONS` is deprecated, and will be removed. [See (from libc++ __config in main)](b57df9fe9a/libcxx/include/__config (L205-L209)):

> TODO(hardening): remove this in LLVM 19.
> This is for backward compatibility -- make enabling `_LIBCPP_ENABLE_ASSERTIONS` (which predates hardening modes)
> equivalent to setting the safe mode.
> ifdef _LIBCPP_ENABLE_ASSERTIONS
> warning "_LIBCPP_ENABLE_ASSERTIONS is deprecated, please use _LIBCPP_ENABLE_SAFE_MODE instead."

From LLVM 17, `_LIBCPP_ENABLE_DEBUG_MODE` can be used instead.

See https://libcxx.llvm.org/Hardening.html.

Related to #28476.
2023-09-14 14:16:49 +01:00
fanquake
f5c5ddafbc Merge bitcoin/bitcoin#28478: ci: Temporarily disable test-each-commit
fa2cb2f5d3 Revert "Merge bitcoin/bitcoin#28279: ci: Add test-each-commit task" (MarcoFalke)

Pull request description:

  This should unbreak the GHA CI for now, and allow someone to fix the task in a follow-up. The issue is https://github.com/bitcoin/bitcoin/pull/28279#issuecomment-1719324530 .

  If no one fixes it, it can be replaced by a Cirrus CI self-hosted runner.

ACKs for top commit:
  sipa:
    ACK fa2cb2f5d3
  dergoegge:
    ACK fa2cb2f5d3

Tree-SHA512: d9c70915b3fb676f44054cee8033286910682ff6819d4ee71e432f7efb3043a0a9507f0052b6c24e3b0c431875f6337b1adccc5b48432970d7c29c6d6b00a2d7
2023-09-14 13:59:05 +01:00
fanquake
858d3138bb Merge bitcoin/bitcoin#28460: fuzz: Use afl++ shared-memory fuzzing
97e2e1d641 [fuzz] Use afl++ shared-memory fuzzing (dergoegge)

Pull request description:

  Using shared-memory is faster than reading from stdin, see 7d2122e059/instrumentation/README.persistent_mode.md

ACKs for top commit:
  MarcoFalke:
    review ACK 97e2e1d641

Tree-SHA512: 7e71b5f84835e41531c19ee959be2426da245869757de8e5dd1c730ae83ead650e2ef75f4d594d7965f661821a4ffbd27be84d3ce623702991501b34a8d02fc3
2023-09-14 13:58:35 +01:00
MarcoFalke
fa2cb2f5d3 Revert "Merge bitcoin/bitcoin#28279: ci: Add test-each-commit task"
This reverts commit 744e0e3670, reversing
changes made to 8209e86eeb.
2023-09-14 14:24:04 +02:00
fanquake
9e9206f52a Merge bitcoin/bitcoin#28465: ci: clang-17 for fuzz and tsan
fa23c9aa7c ci: clang-17 for fuzz and tsan (MarcoFalke)

Pull request description:

  Bump clang in CI from 16 to 17, to:

  * Bump the CI "EOL" from Jan 2024 to July 2024, by bumping from Ubuntu lunar to mantic
  * Test, ensure compatibility, and make use of any new sanitizer features in clang-17

ACKs for top commit:
  dergoegge:
    utACK fa23c9aa7c

Tree-SHA512: 25b625b98341e6e9c56e0b0c080347c2225ea1b0b7bf0e91068f4fc369eaa53fa380b636ffd8ecf1fc7426a1082539a493e176afa3531f1ec86059080a2646de
2023-09-14 11:13:48 +01:00
fanquake
a241d6069c ci: use LLVM 17.0.0 in MSAN jobs
See https://libcxx.llvm.org/Hardening.html as well as
https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
2023-09-14 11:12:24 +01:00
fanquake
1e9d367d0d Merge bitcoin/bitcoin#28423: kernel: Remove protocol.h/netaddress.h/compat.h from kernel headers
d506765199 [refactor] Remove compat.h from kernel headers (TheCharlatan)
36193af47c [refactor] Remove netaddress.h from kernel headers (TheCharlatan)
2b08c55f01 [refactor] Add CChainParams member to CConnman (TheCharlatan)
f0d1d8b35c [refactor] Add missing includes for next commit (TheCharlatan)
534b314a74 kernel: Move MessageStartChars to its own file (TheCharlatan)
9be330b654 [refactor] Define MessageStartChars as std::array (TheCharlatan)
37e2b01113 [refactor] Allow std::array<std::byte, N> in serialize.h (MarcoFalke)

Pull request description:

  This removes the non-consensus critical `protocol.h` and `netaddress.h` headers from the kernel headers. With this patch, they are no longer required to include in order to use the libbitcoinkernel library. This also allows for the removal of the `compat.h` header from the kernel headers.

  As an added future benefit it also reduces the number of of kernel headers that include the platform specific `bitcoin-config.h`.

  For those interested, the currently required kernel headers can be inspected visually with the [sourcetrail](https://github.com/CoatiSoftware/Sourcetrail) tool by looking at the required includes of `bitcoin-chainstate.cpp`.

  ---

  This is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), namely its stage 1 step 3: Decouple most non-consensus headers from libbitcoinkernel.

ACKs for top commit:
  stickies-v:
    re-ACK d506765
  hebasto:
    ACK d506765199.
  ajtowns:
    utACK d506765199
  MarcoFalke:
    lgtm ACK d506765199 🍛

Tree-SHA512: 6f90ea510a302c2927e84d16900e89997c39b8ff3ce9d4effeb8a134bd29cc52bd9e81e51aaa11f7496bad00025b78a58b88c5a9e0bb3f4ebbe9a76309215fb7
2023-09-14 11:11:38 +01:00
fanquake
744e0e3670 Merge bitcoin/bitcoin#28279: ci: Add test-each-commit task
fa5356cd49 ci: Limit test-each-commit to --max-count=6 (MarcoFalke)
fafcd2e9ef ci: Add test-each-commit task (MarcoFalke)

Pull request description:

  Currently, if a pull request has more than one commit, previous commits may fail to compile, or may fail the tests. This is problematic, because it breaks git-bisect, or worse.

  Fix this by adding a CI task for this.

ACKs for top commit:
  jonatack:
    ACK fa5356cd49
  dergoegge:
    utACK fa5356cd49
  hebasto:
    ACK fa5356cd49

Tree-SHA512: 304eff5545501ee499b881f0b0a0c1fe32a7c9f00d96b45bba731af08ac5ca917cef223f5c3d346e30c11a3e6e12e1da297929d3caea9644f3ec1de25dd97c37
2023-09-14 10:14:52 +01:00
fanquake
8209e86eeb Merge bitcoin/bitcoin#28458: refactor: Remove unused GetType() from CBufferedFile and CAutoFile
fa19c914f7 scripted-diff: Rename CBufferedFile to BufferedFile (MarcoFalke)
fa2f2413b8 Remove unused GetType() from CBufferedFile and CAutoFile (MarcoFalke)
5c2b3cd4b8 dbwrapper: Use DataStream for batch operations (TheCharlatan)

Pull request description:

  This refactor is required for https://github.com/bitcoin/bitcoin/pull/28052 and https://github.com/bitcoin/bitcoin/pull/28451

  Thus, split it out.

ACKs for top commit:
  ajtowns:
    utACK fa19c914f7
  TheCharlatan:
    ACK fa19c914f7

Tree-SHA512: d9c232324702512e45fd73ec3e3170f1e8a8c8f9c49cb613a6b693a9f83358914155527ace2517a2cd626a0cedcada56ef70a2a7812edafb1888fd6765eebba2
2023-09-14 09:56:10 +01:00
Anthony Towns
fb6a2ab63e scripted-diff: use SER_PARAMS_OPFUNC
-BEGIN VERIFY SCRIPT-
sed -i 's/WithParams(\(CAddress::V[12]_[A-Z]*\) *, */\1(/g' $(git grep -l 'WithParams' src/)
sed -i 's/WithParams(\(CNetAddr::V[12]\) *, */\1(/g' $(git grep -l 'WithParams' src/)
sed -i 's@\(CNetAddr::V1.CService{}.*\)    //@\1                //@' src/test/util/net.cpp
-END VERIFY SCRIPT-
2023-09-14 10:25:26 +10:00
Anthony Towns
5e5c8f86b6 serialize: add SER_PARAMS_OPFUNC 2023-09-14 10:25:26 +10:00
Anthony Towns
33203f59b4 serialize: specify type for ParamsWrapper not ref 2023-09-14 10:25:20 +10:00
Anthony Towns
bf147bfffa serialize: move ser_action functions out of global namespace 2023-09-14 10:00:45 +10:00
Murch
f18f9ef4d3 Amend bumpfee for inputs with overlapping ancestry
At the end of coin selection reduce the fees by the difference between
the individual bump fee estimates and the collective bump fee estimate.
2023-09-13 15:46:59 -04:00
Murch
2e35e944da Bump unconfirmed parent txs to target feerate
When a transaction uses an unconfirmed input, preceding this commit it
would not consider the feerate of the parent transaction. Given a parent
transaction with a lower ancestor feerate, this resulted in the new
transaction's ancestor feerate undershooting the target feerate.

This commit changes how we calculate the effective value of unconfirmed UTXOs.
The effective value of unconfirmed UTXOs is decreased by the fee
necessary to bump its ancestry to the target feerate. This also impacts
the calculation of the waste metric: since the estimate for the current
fee is increased by the bump fees, unconfirmed UTXOs current fees appear less
favorable compared to their unchanged long term fees.

This has one caveat: if multiple UTXOs have overlapping ancestries, each
of their individual estimates will account for bumping all ancestors.
2023-09-13 14:33:58 -04:00
Andrew Chow
3e3e052411 coinselection: Move GetSelectionWaste into SelectionResult
GetSelectionWaste will need to access more context within a selection
result, and so should be a private member function rather than a static
function. It's only use outside of SelectionResult was for tests which
have now been updated to just make a SelectionResult.

Co-authored-by: Murch <murch@murch.one>
2023-09-13 14:33:57 -04:00
glozow
c57889da66 [node] interface to get bump fees 2023-09-13 14:33:55 -04:00
Murch
c24851be94 Make MiniMinerMempoolEntry fields private
Follow-up from #27021: accessing of fields in MiniMinerMempoolEntry was
done inconsistently. Even though we had a getter, we would directly
write to the fields when we needed to update them.
This commits sets the fields to private and introduces a method for
updating the ancestor information in transactions using the same method
name as used for Mempool Entries.
2023-09-13 14:33:54 -04:00
Murch
ac6030e4d8 Remove unused imports
Follow-up from #27021
2023-09-13 14:33:53 -04:00
Murch
d2f90c31ef Fix calculation of ancestor set feerates in test
Follow-up from #27021.
Also included is an ASCII art visualization of the test’s transaction
topology by theStack.

Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
2023-09-13 14:33:51 -04:00
Murch
a1f7d986e0 Match tx names to index in miniminer overlap test
Follow-up from #27021: In the prior commit, the vector started counting
at 0, but the transaction names started with 1. This commit matches the
names to the transactions’ vector indices for better readability.

Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
2023-09-13 14:33:38 -04:00
fanquake
f1a9fd627b Merge bitcoin/bitcoin#28251: validation: fix coins disappearing mid-package evaluation
32c1dd1ad6 [test] mempool coins disappearing mid-package evaluation (glozow)
a67f460c3f [refactor] split setup in mempool_limit test (glozow)
d08696120e [test framework] add ability to spend only confirmed utxos (glozow)
3ea71feb11 [validation] don't LimitMempoolSize in any subpackage submissions (glozow)
d227b7234c [validation] return correct result when already-in-mempool tx gets evicted (glozow)
9698b81828 [refactor] back-fill results in AcceptPackage (glozow)
8ad7ad3392 [validation] make PackageMempoolAcceptResult members mutable (glozow)
03b87c11ca [validation] add AcceptSubPackage to delegate Accept* calls and clean up m_view (glozow)
3f01a3dab1 [CCoinsViewMemPool] track non-base coins and allow Reset (glozow)
7d7f7a1189 [policy] check for duplicate txids in package (glozow)

Pull request description:

  While we are evaluating a package, we split it into "subpackages" for evaluation (currently subpackages all have size 1 except the last one). If a subpackage has size 1, we may add a tx to mempool and call `LimitMempoolSize()`, which evicts transactions if the mempool gets full. We handle the case where the just-submitted transaction is evicted immediately, but we don't handle the case in which a transaction from a previous subpackage (either just submitted or already in mempool) is evicted. Mainly, since the coins created by the evicted transaction are cached in `m_view`, we don't realize the UTXO has disappeared until `CheckInputsFromMempoolAndCache` asserts that they exist. Also, the returned `PackageMempoolAcceptResult` reports that the transaction is in mempool even though it isn't anymore.

  Fix this by not calling `LimitMempoolSize()` until the very end, and editing the results map with "mempool full" if things fall out.

  Pointed out by instagibbs in faeed687e5 on top of the v3 PR.

ACKs for top commit:
  instagibbs:
    reACK 32c1dd1ad6

Tree-SHA512: 61e7f69db4712e5e5bfa27d037ab66bdd97f1bf60a8d9ffb96adb1f0609af012c810d681102ee5c7baec7b5fe8cb7c304a60c63ccc445d00d86a2b7f0e7ddb90
2023-09-13 17:51:00 +01:00
glozow
32c1dd1ad6 [test] mempool coins disappearing mid-package evaluation
Test for scenario(s) outlined in PR 28251.
Test what happens when a package transaction spends a mempool coin which
is fetched and then disappears mid-package evaluation due to eviction or
replacement.
2023-09-13 16:14:18 +01:00
glozow
a67f460c3f [refactor] split setup in mempool_limit test
We want to be able to re-use fill_mempool so that none of the tests
affect each other.

Change the logs from info to debug because they are otherwise repeated
many times in the test output.
2023-09-13 16:14:18 +01:00
glozow
d08696120e [test framework] add ability to spend only confirmed utxos
Useful to ensure that the topologies of packages/transactions are as
expected, preventing bugs caused by having unexpected mempool ancestors.
2023-09-13 16:14:18 +01:00
glozow
3ea71feb11 [validation] don't LimitMempoolSize in any subpackage submissions
Don't do any mempool evictions until package validation is done,
preventing the mempool minimum feerate from changing. Whether we submit
transactions separately or as a package depends on whether they meet the
mempool minimum feerate threshold, so it's best that the value not
change while we are evaluating a package.
This avoids a situation where we have a CPFP package in which
the parents meet the mempool minimum feerate and are submitted by
themselves, but they are evicted before we have submitted the child.
2023-09-13 16:14:18 +01:00
glozow
d227b7234c [validation] return correct result when already-in-mempool tx gets evicted
Bug fix: a transaction may be in the mempool when package evaluation
begins (so it is added to results_final with MEMPOOL_ENTRY or
DIFFERENT_WITNESS), but get evicted due to another transaction
submission.
2023-09-13 16:14:17 +01:00
glozow
9698b81828 [refactor] back-fill results in AcceptPackage
Instead of populating the last PackageMempoolAcceptResult with stuff
from results_final and individual_results_nonfinal, fill results_final
and create a PackageMempoolAcceptResult using that one.

A future commit will add LimitMempoolSize() which may change the status
of each of these transactions from "already in mempool" or "submitted to
mempool" to "no longer in mempool". We will change those transactions'
results here.

A future commit also gets rid of the last AcceptSubPackage outside of
the loop. It makes more sense to use results_final as the place where
all results end up.
2023-09-13 16:14:17 +01:00
glozow
8ad7ad3392 [validation] make PackageMempoolAcceptResult members mutable
After the PackageMempoolAcceptResult is returned from
AcceptMultipleTransactions, leave room for results to change due to
LimitMempool() eviction.
2023-09-13 16:14:17 +01:00
glozow
03b87c11ca [validation] add AcceptSubPackage to delegate Accept* calls and clean up m_view
(1) Call AcceptSingleTransaction when there is only 1 transaction in the
  subpackage. This avoids calling PackageMempoolChecks() which enforces
rules that don't need to be applied for a single transaction, i.e.
disabling CPFP carve out.
There is a slight change in the error type returned, as shown in the
txpackage_tests change.  When a transaction is the last one left in the
package and its fee is too low, this returns a PCKG_TX instead of
PCKG_POLICY. This interface is clearer; "package-fee-too-low" for 1
transaction would be a bit misleading.

(2) Clean up m_view and m_viewmempool so that coins created in this
sub-package evaluation are not available for other sub-package
evaluations. The contents of the mempool may change, so coins that are
available now might not be later.
2023-09-13 16:14:17 +01:00
glozow
3f01a3dab1 [CCoinsViewMemPool] track non-base coins and allow Reset
Temporary coins should not be available in separate subpackage submissions.
Any mempool coins that are cached in m_view should be removed whenever
mempool contents change, as they may be spent or no longer exist.
2023-09-13 16:14:17 +01:00
glozow
7d7f7a1189 [policy] check for duplicate txids in package
Duplicates of normal transactions would be found by looking for
conflicting inputs, but this doesn't catch identical empty transactions.
These wouldn't be valid but exiting early is good and AcceptPackage's
result sanity checks assume non-duplicate transactions.
2023-09-13 16:14:17 +01:00
MarcoFalke
fad52baf1e fuzz: Rework addr fuzzing
* Replace ConsumeDeserializationParams with V1, because V2 is
  unconditionally checked as well.
* Also fuzz CAddress::Format::Disk in the address_deserialize fuzz
  target.
2023-09-13 16:12:51 +02:00
MarcoFalke
fa5b6d29ee fuzz: Drop unused params from serialize helpers
With the ser-type and ser-version going away, it seems unlikely that
there is need for them in the future, so just remove them.
2023-09-13 16:09:23 +02:00
glozow
4313c77400 make DisconnectedBlockTransactions responsible for its own memory management
Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-09-13 13:03:38 +01:00
glozow
cf5f1faa03 MOVEONLY: DisconnectedBlockTransactions to its own file
This struct is only used in validation + tests and has very little to do
with txmempool.
2023-09-13 13:01:59 +01:00
glozow
2765d6f343 rewrite DisconnectedBlockTransactions as a list + map
And encapsulate underlying data structures to avoid misuse.
It's better to use stdlib instead of boost when we can achieve the same thing.

Behavior change: the number returned by DynamicMemoryUsage for the same
transactions is higher (due to change in usage or more accurate
accounting), which effectively decreases the maximum amount of
transactions kept for resubmission in a reorg.

Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-09-13 13:01:39 +01:00
MarcoFalke
fa23c9aa7c ci: clang-17 for fuzz and tsan 2023-09-13 13:35:39 +02:00
Pieter Wuille
3fcd7fc7ff Do not use std::vector = {} to release memory 2023-09-13 07:20:36 -04:00
glozow
79ce9f0aa4 add std::list to memusage 2023-09-13 11:37:45 +01:00
glozow
59a35a7398 [bench] DisconnectedBlockTransactions 2023-09-13 11:37:13 +01:00
TheCharlatan
d506765199 [refactor] Remove compat.h from kernel headers
This commit makes compat.h no longer a required include for users of the
libbitcoinkernel. Including compat.h imports a bunch of
platform-specific definitions.

This commit is part of the libbitcoinkernel project, namely its stage 1
step 3: Decouple most non-consensus headers from libbitcoinkernel.
2023-09-12 22:51:48 +02:00
TheCharlatan
36193af47c [refactor] Remove netaddress.h from kernel headers
Move functions requiring the netaddress.h include out of
libbitcoinkernel source files.

The netaddress.h file contains many non-consensus related definitions
and should thus not be part of the libbitcoinkernel. This commit makes
netaddress.h no longer a required include for users of the
libbitcoinkernel.

This commit is part of the libbitcoinkernel project, namely its stage 1
step 3: Decouple most non-consensus headers from libbitcoinkernel.
2023-09-12 22:51:46 +02:00
TheCharlatan
2b08c55f01 [refactor] Add CChainParams member to CConnman
This is done in preparation to the next commit, but has the nice
effect of removing one further data structure relying on the global
`Params()`.
2023-09-12 22:51:45 +02:00
TheCharlatan
f0d1d8b35c [refactor] Add missing includes for next commit 2023-09-12 22:51:42 +02:00
TheCharlatan
534b314a74 kernel: Move MessageStartChars to its own file
The protocol.h file contains many non-consensus related definitions and
should thus not be part of the libbitcoinkernel. This commit makes
protocol.h no longer a required include for users of the
libbitcoinkernel.

This commit is part of the libbitcoinkernel project, namely its stage 1
step 3: Decouple most non-consensus headers from libbitcoinkernel.

Co-Authored-By: Cory Fields <cory-nospam-@coryfields.com>
2023-09-12 22:51:38 +02:00
TheCharlatan
9be330b654 [refactor] Define MessageStartChars as std::array 2023-09-12 22:49:49 +02:00
MarcoFalke
37e2b01113 [refactor] Allow std::array<std::byte, N> in serialize.h
This is already possible for C-style arrays, so allow it for C++11
std::array as well.
2023-09-12 22:44:28 +02:00
furszy
bae209e387 gui: macOS, make appMenuBar part of the main app window
By moving the appMenuBar destruction responsibility to the QT
framework, we ensure the disconnection of the submenus signals
prior to the destruction of the main app window.

The standalone menu bar may have served a purpose in earlier
versions when it didn't contain actions that directly open
specific screens within the main application window. However,
at present, all the actions within the appMenuBar lead to the
opening of screens within the main app window. So, the absence
of a main app window makes these actions essentially pointless.
2023-09-12 14:36:16 -03:00
Andrew Chow
adc0921ea1 Merge bitcoin/bitcoin#28101: doc, refactor: changing -torcontrol help to specify that a default port is used
9a84200cfc doc, refactor: Changing -torcontrol help to specify that a default port is used (kevkevin)

Pull request description:

  Right now when we get the help for -torcontrol it says that there is a default ip and port we dont specify if there is a specified ip that we would also use port 9051 as default

  Also I create a new const instead of using 9051 directly in the function

  linking this PR because this was discussed here https://github.com/bitcoin/bitcoin/pull/28018

ACKs for top commit:
  jonatack:
    re-ACK 9a84200cfc
  achow101:
    ACK 9a84200cfc
  MarnixCroes:
    utACK 9a84200cfc
  kristapsk:
    utACK 9a84200cfc

Tree-SHA512: 21d9e65f3c280a2853a9cf60d4e93e8d72caccea106206d1862c19535bde7ea6ada7f55e6ea19a1fc0f59dbe791ec6fc4084fdbe7fa6d6991fa89c62070db637
2023-09-12 12:41:30 -04:00
Andrew Chow
8f9c74cb11 Merge bitcoin/bitcoin#28414: wallet rpc: return final tx hex from walletprocesspsbt if complete
2e249b9227 doc: add release note for PR #28414 (Matthew Zipkin)
4614332fc4 test: remove unnecessary finalizepsbt rpc calls (ismaelsadeeq)
e3d484b603 wallet rpc: return final tx hex from walletprocesspsbt if complete (Matthew Zipkin)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/28363#discussion_r1315753887

  `walletprocesspsbt` currently returns a base64-encoded PSBT and a boolean indicating if the tx is "complete". If it is complete, the base64 PSBT can be finalized with `finalizepsbt` which returns the hex-encoded transaction suitable for `sendrawtransaction`.

  With this patch, `walletprocesspsbt` return object will ALSO include the broadcast-able hex string if the tx is already final. This saves users the extra step of calling `finalizepsbt` assuming they have already inspected and approve the transaction from earlier steps.

ACKs for top commit:
  ismaelsadeeq:
    re ACK 2e249b9227
  BrandonOdiwuor:
    re ACK 2e249b9
  Randy808:
    Tested ACK 2e249b9227
  achow101:
    ACK 2e249b9227
  ishaanam:
    ACK 2e249b9227

Tree-SHA512: 229c1103265a9b4248f080935a7ad5607c3be3f9a096a9ab6554093b2cd8aa8b4d1fa55b1b97d3925ba208dbc3ccba4e4d37c40e1491db0d27ba3d9fe98f931e
2023-09-12 12:28:13 -04:00
Andrew Chow
ad0c469d98 wallet: Use CTxDestination in CRecipient rather than scriptPubKey 2023-09-12 12:14:31 -04:00
Andrew Chow
07d3bdf4eb Add PubKeyDestination for P2PK scripts
P2PK scripts are not PKHash destinations, they should have their own
type.

This also results in no longer showing a p2pkh address for p2pk outputs.
However for backwards compatibility, ListCoinst will still do this
conversion.
2023-09-12 12:14:31 -04:00
Andrew Chow
1a98a51c66 Allow CNoDestination to represent a raw script
Even if a script is not a standard destination type, it can still be
useful to have a CTxDestination that stores the script.
2023-09-12 12:14:31 -04:00
Andrew Chow
8dd067088d Make WitnessUnknown members private
Make sure that nothing else can change WitnessUnknown's data members by
making them private. Also change the program to use a vector rather than
C-style array.
2023-09-12 12:14:31 -04:00
dergoegge
97e2e1d641 [fuzz] Use afl++ shared-memory fuzzing
Using shared-memory is faster than reading from stdin, see
7d2122e059/instrumentation/README.persistent_mode.md
2023-09-12 15:07:07 +01:00
fanquake
7649431637 Merge bitcoin/bitcoin#28105: doc: Clarify that -fstack-reuse=all bugs exist on all versions of GCC
fabb419a3c doc: Clarify that -fstack-reuse=all bugs exist on all versions of GCC (MarcoFalke)

Pull request description:

  This is a follow-up to commit 7b850bc2a1. While the test case no longer reproduces, the general class of `-fstack-reuse` bugs still exists in all versions of GCC. The workaround can never be removed, unless the whole class of bugs is fixed.

ACKs for top commit:
  fanquake:
    ACK fabb419a3c

Tree-SHA512: 566e14fe82d13dda4f7b8cca90c6de75006d14828906b936780716d5b5b31de9b36a904aa7cfc9820ccdfb4d3224a8437f502f25f7230da5abe87c927123f0c8
2023-09-12 15:00:25 +01:00
fanquake
578f50fc48 Merge bitcoin/bitcoin#28448: rpc: Deprecate rpcserialversion=0
971bae9174 rpc: Deprecate rpcserialversion=0 (Anthony Towns)

Pull request description:

  This option was introduced in #9194 to ease the transition to segwit; now that most libraries and apps have been updated it should no longer be necessary.

ACKs for top commit:
  MarcoFalke:
    review ACK 971bae9174
  Randy808:
    Code Review ACK 971bae9174
  glozow:
    ACK 971bae9174, seems appropriate to remove. Thanks for looking at usage in https://github.com/bitcoin/bitcoin/pull/28448#issuecomment-1714699556

Tree-SHA512: 6880314504281e9d7c288bd159f8cadefb3e653ac2dd148396810f7f5a27ba352ecfe720eb2dbc6172b57820cb9a2a254dcb2585881abae43811013505f0e09a
2023-09-12 14:26:57 +01:00
MarcoFalke
fabb419a3c doc: Clarify that -fstack-reuse=all bugs exist on all versions of GCC 2023-09-12 13:41:19 +02:00
MarcoFalke
fa19c914f7 scripted-diff: Rename CBufferedFile to BufferedFile
While touching all constructors in the previous commit, the class name
can be adjusted to comply with the style guide.

-BEGIN VERIFY SCRIPT-
 sed -i 's/CBufferedFile/BufferedFile/g' $( git grep -l CBufferedFile )
-END VERIFY SCRIPT-
2023-09-12 12:55:29 +02:00
MarcoFalke
fa2f2413b8 Remove unused GetType() from CBufferedFile and CAutoFile
GetType() is only called in tests, so it is unused and can be removed.
2023-09-12 12:35:13 +02:00
TheCharlatan
5c2b3cd4b8 dbwrapper: Use DataStream for batch operations 2023-09-12 12:07:39 +02:00
fanquake
fd69ffbbfb Merge bitcoin/bitcoin#28427: index: coinstats reorg, fail when block cannot be reversed
c0bf667912 index: add [nodiscard] attribute to functions writing to the db (furszy)
eef595560e index: coinstats reorg, fail when block cannot be reversed (furszy)

Pull request description:

  Found it while reviewing https://github.com/bitcoin/bitcoin/pull/24230#discussion_r1310863359.

  During a reorg, continuing execution when a block cannot be reversed leaves the
  coinstats index in an inconsistent state.
  This was surely overlooked when 'CustomRewind' was implemented.

ACKs for top commit:
  ryanofsky:
    Code review ACK c0bf667912. Only change since last review is new commit adding [[nodiscard]]

Tree-SHA512: f4fc8522508d23e4fff09a29c935971819b1bd3b2a260e08e2e2b72f9340980d74fbec742a58fe216baf61d27de057c7c8300e8fa075f8507cd1227f128af909
2023-09-12 09:44:55 +01:00
Hennadii Stepanov
befb42f146 qt: Silence -Wcast-function-type warning
Required for https://github.com/bitcoin/bitcoin/pull/25972.
Picked from https://trac.nginx.org/nginx/ticket/1865.
2023-09-11 16:30:58 +01:00
fanquake
8f7b9eb871 Merge bitcoin/bitcoin#28433: Follow-up to BIP324 connection support
64704386b2 doc: fix typos and mistakes in BIP324 code comments (Pieter Wuille)
9bde93df2c net: do not use send buffer to store/cache garbage (Pieter Wuille)
b6934fd03f net: merge V2Transport constructors, move key gen (Pieter Wuille)

Pull request description:

  This addresses a few remaining comments on #28196:

  * Deduplicate the `V2Transport` constructors (https://github.com/bitcoin/bitcoin/pull/28196#discussion_r1318573111)
  * Do not use the send buffer to store garbage (https://github.com/bitcoin/bitcoin/pull/28196#discussion_r1319134141)
  * Fix typo (https://github.com/bitcoin/bitcoin/pull/28196#discussion_r1315179378)

  In addition, also fix an incorrect description in `V2Transport::SendState` (it claimed garbage was sent in the `READY` state, but it's in the `AWAITING_KEY` state).

ACKs for top commit:
  naumenkogs:
    ACK 64704386b2
  theStack:
    Code-review ACK 64704386b2

Tree-SHA512: 4bf6d2fe73c8054502d0b60e9de1722f8b3dd269c2dd6bf67197c3fb6eabcf047b6360cdab3c1fd5504215c2ac4ac2890a022780efc30ff583776242c8112451
2023-09-11 09:44:38 +01:00
Anthony Towns
971bae9174 rpc: Deprecate rpcserialversion=0 2023-09-11 17:21:53 +10:00
Pieter Wuille
64704386b2 doc: fix typos and mistakes in BIP324 code comments 2023-09-10 16:12:30 -04:00
Pieter Wuille
9bde93df2c net: do not use send buffer to store/cache garbage
Before this commit the V2Transport::m_send_buffer is used to store the
garbage:
* During MAYBE_V1 state, it's there despite not being sent.
* During AWAITING_KEY state, while it is being sent.
* At the end of the AWAITING_KEY state it cannot be wiped as it's still
  needed to compute the garbage authentication packet.

Change this by introducing a separate m_send_garbage field, taking over
the first and last role listed above. This means the garbage is only in
the send buffer when it's actually being sent, removing a few special
cases related to this.
2023-09-10 16:12:27 -04:00
Pieter Wuille
b6934fd03f net: merge V2Transport constructors, move key gen
This removes the ability for BIP324Cipher to generate its own key, moving that
responsibility to the caller (mostly, V2Transport). This allows us to write
the random-key V2Transport constructor by delegating to the explicit-key one.
2023-09-10 16:11:52 -04:00
fanquake
c5a63ea56f Merge bitcoin/bitcoin#27944: test: various USDT functional test cleanups (27831 follow-ups)
9f55773a37 test: refactor: usdt_mempool: store all events (stickies-v)
bc43270450 test: refactor: remove unnecessary nonlocal (stickies-v)
326db63a68 test: log sanity check assertion failures (stickies-v)
f5525ad680 test: store utxocache events (stickies-v)
f1b99ac94f test: refactor: deduplicate handle_utxocache_* logic (stickies-v)
ad90ba36bd test: refactor:  rename inbound to is_inbound (stickies-v)
afc0224cdb test: refactor: remove unnecessary blocks_checked counter (stickies-v)

Pull request description:

  Various cleanups to the USDT functional tests, largely (but not exclusively) follow-ups to https://github.com/bitcoin/bitcoin/pull/27831#pullrequestreview-1491438045. Except for slightly different logging behaviour in "test: store utxocache events" and "test: log sanity check assertion failures", this is a refactor PR, removing unnecessary code and (imo) making it more readable and maintainable.

  The rationale for each change is in the corresponding commit message.

  Note: except for "test: store utxocache events" (which relies on its parent, and I separated into two commits because we may want the parent but not the child), all commits are stand-alone and I'm okay with dropping one/multiple commits if they turn out to be controversial or undesired.

ACKs for top commit:
  0xB10C:
    ACK 9f55773a37. Reviewed the code and ran the USDT interface tests. I stepped through the commits and think all changes are reasonable.

Tree-SHA512: 6c37a0265b6c26d4f9552a056a690b8f86f7304bd33b4419febd8b17369cf6af799cb87c16df35d0c2a1b839ad31de24661d4384eafa88816c2051c522fd3bf5
2023-09-10 14:10:16 +01:00
fanquake
e25af11225 Merge bitcoin/bitcoin#28431: Remove needless GetTransactionOutputWeight helper
8d6228fc1f consensus/validation.h: remove needless GetTransactionOutputWeight helper (Antoine Poinsot)

Pull request description:

  Introduced in #26567. My bad. Thanks AJ for noticing.

ACKs for top commit:
  ajtowns:
    utACK 8d6228fc1f

Tree-SHA512: cf13647b4aac82fb6a54ae0338e3928e9bdf226ed4f5e91d529996328471744132db2bee9676e0b3f40a8bbe0e0ca51a9e5f91560a84e0f33597290551a1ee18
2023-09-09 11:45:15 +01:00
fanquake
579c49b3a6 Merge bitcoin/bitcoin#28428: Hard-code version number value for CBlockLocator and CDiskBlockIndex
e73d2a8018 refactor: remove clientversion include from dbwrapper.h (Cory Fields)
4240a082b8 refactor: Use DataStream now that version/type are unused (Cory Fields)
f15f790618 Remove version/hashing options from CBlockLocator/CDiskBlockIndex (Cory Fields)

Pull request description:

  This is also a much simpler replacement for #28327.

  There are version fields in `CBlockLocator` and `CDiskBlockIndex` that have always been written but discarded when read.

  I intended to convert them to use SerParams as introduced by #25284, which [ended up looking like this](3e3af45165). However because we don't currently have any definition of what a hash value would mean for either one of those, and we've never assigned the version field any meaning, I think it's better to just not worry about them.

  If we ever need to assign meaning in the future, we can introduce `SerParams` as was done for `CAddress`.

  As for the dummy values chosen:

  `CDiskBlockIndex::DUMMY_VERSION` was easy as the highest ever client version, and I don't expect any objection there.

  `CBlockLocator::DUMMY_VERSION` is hard-coded to the higest _PROTOCOL_ version ever used. This is to avoid a sudden bump that would be visible on the network if CLIENT_VERSION were used instead. In the future, if we ever need to use the value, we can discard anything in the CLIENT_VERSION range (for a few years as needed), as it's quite a bit higher.

  While reviewing, I suggest looking at the throwaway `SerParams` commit above as it shows where the call-sites are. I believe that should be enough to convince one's self that hashing is never used.

ACKs for top commit:
  TheCharlatan:
    Re-ACK e73d2a8018
  ajtowns:
    reACK e73d2a8018

Tree-SHA512: 45b0dd7c2e918493e2ee92a8e35320ad17991cb8908cb811150a96c5fd584ce177c775baeeb8675a602c90b9ba9203b8cefc0a2a0c6a71078b1d9c2b41e1f3ba
2023-09-09 11:30:57 +01:00
furszy
e14cc8fc69 gui: macOS, do not process dock icon actions during shutdown
As the 'QMenuBar' is created without a parent window in MacOS, the
app crashes when the user presses the shutdown button and, right
after it, triggers any action in the menu bar.

This happens because the QMenuBar is manually deleted in the
BitcoinGUI destructor but the events attached to it children
actions are not disconnected, so QActions events such us the
'QMenu::aboutToShow' could try to access null pointers.

Instead of guarding every single QAction pointer inside the
QMenu::aboutToShow slot, or manually disconnecting all
registered events in the destructor, we can check if a
shutdown was requested and discard the event.

The 'node' field is a ref whose memory is held by the
main application class, so it is safe to use here. Events
are disconnected prior destructing the main application object.

Furthermore, the 'MacDockIconHandler::dockIconClicked' signal
can make the app crash during shutdown for the very same
reason. The 'show()' call triggers the 'QApplication::focusWindowChanged'
event, which is connected to the 'minimize_action' QAction,
which is also part of the app menu bar, which could no longer exist.
2023-09-08 11:33:52 -03:00
Cory Fields
e73d2a8018 refactor: remove clientversion include from dbwrapper.h 2023-09-08 13:40:15 +00:00
Cory Fields
4240a082b8 refactor: Use DataStream now that version/type are unused 2023-09-08 13:40:15 +00:00
Cory Fields
f15f790618 Remove version/hashing options from CBlockLocator/CDiskBlockIndex 2023-09-08 13:40:15 +00:00
furszy
c0bf667912 index: add [nodiscard] attribute to functions writing to the db 2023-09-08 10:04:14 -03:00
fanquake
4e1a38c6df Merge bitcoin/bitcoin#28196: BIP324 connection support
db9888feec net: detect wrong-network V1 talking to V2Transport (Pieter Wuille)
91e1ef8684 test: add unit tests for V2Transport (Pieter Wuille)
297c888997 net: make V2Transport preallocate receive buffer space (Pieter Wuille)
3ffa5fb49e net: make V2Transport send uniformly random number garbage bytes (Pieter Wuille)
0be752d9f8 net: add short message encoding/decoding support to V2Transport (Pieter Wuille)
8da8642062 net: make V2Transport auto-detect incoming V1 and fall back to it (Pieter Wuille)
13a7f01557 net: add V2Transport class with subset of BIP324 functionality (Pieter Wuille)
dc2d7eb810 crypto: Spanify EllSwiftPubKey constructor (Pieter Wuille)
5f4b2c6d79 net: remove unused Transport::SetReceiveVersion (Pieter Wuille)
c3fad1f29d net: add have_next_message argument to Transport::GetBytesToSend() (Pieter Wuille)

Pull request description:

  This is part of #27634.

  This implements the BIP324 v2 transport (which implements all of what the BIP calls transport layer *and* application layer), though in a non-exposed way. It is tested through an extensive fuzz test, which verifies that v2 transports can talk to v2 transports, and v1 transports can talk to v2 transports, and a unit test that exercises a number of unusual scenarios. The transport is functionally complete, including:
  * Autodetection of incoming V1 connections.
  * Garbage, both sending and receiving.
  * Short message type IDs, both sending and receiving.
  * Ignore packets (receiving only, but tested in a unit test).
  * Session IDs are visible in `getpeerinfo` output (for manual comparison).

  Things that are not included, left for future PRs, are:
  * Actually using the v2 transport for connections.
  * Support for the `NODE_P2P_V2` service flag.
  * Retrying downgrade to V1 when attempted outbound V2 connections immediately fail.
  * P2P functional and unit tests

ACKs for top commit:
  naumenkogs:
    ACK db9888feec
  theStack:
    re-ACK db9888feec
  mzumsande:
    Code Review ACK db9888feec

Tree-SHA512: 8906ac1e733a99e1f31c9111055611f706d80bbfc2edf6a07fa6e47b21bb65baacd1ff17993cbbf588063b2f5ad30b3af674a50c7bc8e8ebf4671483a21bbfeb
2023-09-08 10:24:03 +01:00
Antoine Poinsot
8d6228fc1f consensus/validation.h: remove needless GetTransactionOutputWeight helper
Introduced in 9b7ec393b8. This copied the format of the other Get.*Weight helpers but it's useless for a CTxOut.
2023-09-08 11:16:06 +02:00
glozow
925bb723ca [refactor] batch-add transactions to DisconnectedBlockTransactions
No behavior change.
In a future commit, we can optimize by reserving vtx.size().
2023-09-07 18:55:44 +01:00
fanquake
238d29aff9 Merge bitcoin/bitcoin#28361: fuzz: add ConstructPubKeyBytes util function
1580e3be83 fuzz: add ConstructPubKeyBytes function (josibake)

Pull request description:

  In https://github.com/bitcoin/bitcoin/pull/28246 and https://github.com/bitcoin/bitcoin/pull/28122 , we add a `PubKeyDestination` and a `V0SilentPaymentsDestination`. Both of these PRs update `fuzz/util.cpp` and need a way to create well-formed pubkeys. Currently in `fuzz/util.cpp`, we have some logic for creating pubkeys in the multisig data provider. This logic is duplicated in #28246 and duplicated again in #28122. Seems much better to have a `ConstructPubKeyBytes` function that both PRs (and any future work) can reuse.

  This PR introduces a function to do this and has the existing code use it. While the purpose is to introduce a utility function, the previous multisig code used `ConsumeIntegralInRange(4, 7)` which would have created some uncompressed pubkeys with the prefix 0x05, which is incorrect (see https://bitcoin.stackexchange.com/questions/57855/c-secp256k1-what-do-prefixes-0x06-and-0x07-in-an-uncompressed-public-key-signif)

  tldr; using `PickValueFromArray` is more correct as it limits to the set of defined prefixes for compressed and uncompressed pubkeys.

ACKs for top commit:
  Sjors:
    ACK 1580e3be83

Tree-SHA512: c87c8bcd1f6b3a97ef772be93102efb912811c59f32211cfd531a116f1da8a57c8c6ff106b34f2a2b88d8b34fb5bc30d9f9ed6d2720113ffcaaa2f8d5dc9eb27
2023-09-07 16:22:16 +01:00
furszy
eef595560e index: coinstats reorg, fail when block cannot be reversed
During a reorg, continuing execution when a block cannot be
reversed leaves the coinstats index in an inconsistent state,
which was surely overlooked when 'CustomRewind' was implemented.
2023-09-07 12:15:34 -03:00
fanquake
0354206a30 Merge bitcoin/bitcoin#28412: test: remove unused variables in p2p_invalid_block
3eb03803c4 test: remove unused variables in `p2p_invalid_block` (brunoerg)

Pull request description:

ACKs for top commit:
  stickies-v:
    ACK 3eb03803c4

Tree-SHA512: eadae1eb323e5562d1ea0aed43ebf0145f0fdbb6cd6d4646bbf1ca89f384820e7b9cb69f0bb04a949e9f8983a879aee8387d6f7ac3d4e4ea027f8892e656fb98
2023-09-07 16:10:14 +01:00
fanquake
b097a689ed Merge bitcoin/bitcoin#28422: depends: cctools 986, ld64 711 & libtapi 1300.0.6.5
7d5815293e depends: cctools 986 & ld64 711 (fanquake)
1ed1183f47 depends: libtapi 1300.0.6.5 (fanquake)

Pull request description:

  Update [libtapi to `1300.0.6.5`](https://github.com/tpoechtrager/apple-libtapi/tree/1300.6.5), [cctools to `986` & ld64 to `711`](https://github.com/tpoechtrager/cctools-port/commits/986-ld64-711).

  Updating these dependencies fixes the macOS build failures in #27897, when using a native GCC 12 toolchain, which means after this change, we can use use GCC 12 for all HOSTS, rather than splitting between 12 & 10 (for macOS).

  Guix Build (x86_64):
  ```bash
  a684a3e7668250610d653e5160115fb50b477339423335b185bd4716c8420e14  guix-build-7d5815293ed8/output/aarch64-linux-gnu/SHA256SUMS.part
  9c66b82e2c5b6dbdcabdd962481485f71af5770e8bb573eaefb55d45e85e2875  guix-build-7d5815293ed8/output/aarch64-linux-gnu/bitcoin-7d5815293ed8-aarch64-linux-gnu-debug.tar.gz
  63f82714a93e7a3c0e922cdcce7d3b27789d63cdf136dc3a71b74d61b0910ce8  guix-build-7d5815293ed8/output/aarch64-linux-gnu/bitcoin-7d5815293ed8-aarch64-linux-gnu.tar.gz
  dcc79295462e0f585753959247ec02b273f5a933fadc5d528c266d56bbdaaf44  guix-build-7d5815293ed8/output/arm-linux-gnueabihf/SHA256SUMS.part
  044fc0ad2a8615dd004579b5003fa981ebc4e3bfaacf5eba8a33cea02a9ba518  guix-build-7d5815293ed8/output/arm-linux-gnueabihf/bitcoin-7d5815293ed8-arm-linux-gnueabihf-debug.tar.gz
  a032899be14fa2987dfd5a21865567d3beea703e778a2f91ec17eddb099d0c46  guix-build-7d5815293ed8/output/arm-linux-gnueabihf/bitcoin-7d5815293ed8-arm-linux-gnueabihf.tar.gz
  27d43a97ea6c4ed2e84d9a9dd6c240e8e199174f4491ee1fb62abb1d950a96bd  guix-build-7d5815293ed8/output/arm64-apple-darwin/SHA256SUMS.part
  029347186cc55c15152e8c255493ed457e58127ffa536982f3a632fc416b6662  guix-build-7d5815293ed8/output/arm64-apple-darwin/bitcoin-7d5815293ed8-arm64-apple-darwin-unsigned.dmg
  791399447c33f7d3ed4d77fd22fde69aa7c87adda9291fca9508237b69b05b80  guix-build-7d5815293ed8/output/arm64-apple-darwin/bitcoin-7d5815293ed8-arm64-apple-darwin-unsigned.tar.gz
  b9c72b7f37068889a83384598b6de97c8e39e7a097b661152084a5e69236541d  guix-build-7d5815293ed8/output/arm64-apple-darwin/bitcoin-7d5815293ed8-arm64-apple-darwin.tar.gz
  6a249cc6d02d7398b9ed70b251422bf4bb1c2acdaafb5a885870e8ef20d43a89  guix-build-7d5815293ed8/output/dist-archive/bitcoin-7d5815293ed8.tar.gz
  064e5e0df21aa177eaf8d67254ad5166b190559f69da68d34174a323d8c975c6  guix-build-7d5815293ed8/output/powerpc64-linux-gnu/SHA256SUMS.part
  6aee00585bdd680d42f3341df523b7173dc06defa8f51f98af38c8518ddf4532  guix-build-7d5815293ed8/output/powerpc64-linux-gnu/bitcoin-7d5815293ed8-powerpc64-linux-gnu-debug.tar.gz
  d6e51b521c84276d5c9606a96ee36b94bc235370064cf75148d15d3a4011ed9d  guix-build-7d5815293ed8/output/powerpc64-linux-gnu/bitcoin-7d5815293ed8-powerpc64-linux-gnu.tar.gz
  e0a133afe0e0e7894d3c6110b7b1895efff8f45abbe473d95100d3c5615dc855  guix-build-7d5815293ed8/output/powerpc64le-linux-gnu/SHA256SUMS.part
  8d62884b81fca76720284ae4d5b2a0d0ebfcbcf533d9936c8aa8ab04739540e9  guix-build-7d5815293ed8/output/powerpc64le-linux-gnu/bitcoin-7d5815293ed8-powerpc64le-linux-gnu-debug.tar.gz
  c4c90dd3c83a84aa074ab5dc77b82fd2a26d872547037ef2a8a2d11abba1db98  guix-build-7d5815293ed8/output/powerpc64le-linux-gnu/bitcoin-7d5815293ed8-powerpc64le-linux-gnu.tar.gz
  dd28d196e89f74cf7666b347e1cbdd16eb90bfe0d5b054ac2edbf0e4f755bd9a  guix-build-7d5815293ed8/output/riscv64-linux-gnu/SHA256SUMS.part
  8d3ace8a17d4c1a13786adb3669b1d929485b164c59efb1fe7cf8fe3aad90f73  guix-build-7d5815293ed8/output/riscv64-linux-gnu/bitcoin-7d5815293ed8-riscv64-linux-gnu-debug.tar.gz
  bf530b7944dfdd9ffd03b2be541fa349e8598cb39ee5e1bd4be0b5b48bdfcc31  guix-build-7d5815293ed8/output/riscv64-linux-gnu/bitcoin-7d5815293ed8-riscv64-linux-gnu.tar.gz
  6172224f94d52eb22a3749b1e3c0639476dfdaff47d15c6a8ec3e6dbf8763f9b  guix-build-7d5815293ed8/output/x86_64-apple-darwin/SHA256SUMS.part
  f59f68ad4783ff1ecb09cd84f15bbfc803c856971de5a131fbb1777cf8f1b79a  guix-build-7d5815293ed8/output/x86_64-apple-darwin/bitcoin-7d5815293ed8-x86_64-apple-darwin-unsigned.dmg
  ac31df20d1e5f3a73aadb5455e804af053b179ca6471b5a57afe3c7df0f09128  guix-build-7d5815293ed8/output/x86_64-apple-darwin/bitcoin-7d5815293ed8-x86_64-apple-darwin-unsigned.tar.gz
  a33f661386d415f621ab17edf7fe3efa61f0c2b0c3003fecd4f5f6898129992d  guix-build-7d5815293ed8/output/x86_64-apple-darwin/bitcoin-7d5815293ed8-x86_64-apple-darwin.tar.gz
  03104057b86de994258587ff5a8de16f48354a7493527e5bb2559714c0e2bf39  guix-build-7d5815293ed8/output/x86_64-linux-gnu/SHA256SUMS.part
  a9c3f233629038978b9fa11c1353b4e686406851c0e84a5539cf3bc90722d98b  guix-build-7d5815293ed8/output/x86_64-linux-gnu/bitcoin-7d5815293ed8-x86_64-linux-gnu-debug.tar.gz
  61db7ec4aa306a6f79628b217c05387b69002d123fcf07d6784cce03d9a8574d  guix-build-7d5815293ed8/output/x86_64-linux-gnu/bitcoin-7d5815293ed8-x86_64-linux-gnu.tar.gz
  7f5b42d02d8c026492376b644a94a45613155a0ccc1d3ef9f067b3370d45c4d2  guix-build-7d5815293ed8/output/x86_64-w64-mingw32/SHA256SUMS.part
  de23de0fb32c77f9ff91ffe554347c7ddc77fae22212d0e609389b4b7a1d6e29  guix-build-7d5815293ed8/output/x86_64-w64-mingw32/bitcoin-7d5815293ed8-win64-debug.zip
  2131ce905aa3452883a9f47b6618506a6cb90c2a1725818084bc4e08856ff8d3  guix-build-7d5815293ed8/output/x86_64-w64-mingw32/bitcoin-7d5815293ed8-win64-setup-unsigned.exe
  5555d28d9ea14a50ab296c2b1f1db7f7acde933009334999b6be29a2ed469bc0  guix-build-7d5815293ed8/output/x86_64-w64-mingw32/bitcoin-7d5815293ed8-win64-unsigned.tar.gz
  7dc29c317c140ae8a8f487d2d66761c3fae90a5521dd4bfd0d071adbfafb30ef  guix-build-7d5815293ed8/output/x86_64-w64-mingw32/bitcoin-7d5815293ed8-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 7d5815293e
  hebasto:
    ACK 7d5815293e

Tree-SHA512: 98873733abb2ebc318129d9fa6627d39f751495776338f539bd58985d08246c1fb85d6bb9b5911f0cd692c26e519713b93d496ffbae513a45942357b82497a91
2023-09-07 15:30:18 +01:00
Pieter Wuille
db9888feec net: detect wrong-network V1 talking to V2Transport 2023-09-07 09:04:55 -04:00
Pieter Wuille
91e1ef8684 test: add unit tests for V2Transport 2023-09-07 09:04:55 -04:00
Pieter Wuille
297c888997 net: make V2Transport preallocate receive buffer space 2023-09-07 09:04:55 -04:00
Pieter Wuille
3ffa5fb49e net: make V2Transport send uniformly random number garbage bytes 2023-09-07 09:04:55 -04:00
Pieter Wuille
0be752d9f8 net: add short message encoding/decoding support to V2Transport 2023-09-07 09:04:51 -04:00
Pieter Wuille
8da8642062 net: make V2Transport auto-detect incoming V1 and fall back to it 2023-09-07 09:01:01 -04:00
Pieter Wuille
13a7f01557 net: add V2Transport class with subset of BIP324 functionality
This introduces a V2Transport with a basic subset of BIP324 functionality:
* no ability to send garbage (but receiving is supported)
* no ability to send decoy packets (but receiving them is supported)
* no support for short message id encoding (neither encoding or decoding)
* no waiting until 12 non-V1 bytes have been received
* (and thus) no detection of V1 connections on the responder side
  (on the sender side, detecting V1 is not supported either, but that needs
  to be dealt with at a higher layer, by reconnecting)
2023-09-07 09:00:58 -04:00
Pieter Wuille
dc2d7eb810 crypto: Spanify EllSwiftPubKey constructor 2023-09-07 08:53:45 -04:00
Pieter Wuille
5f4b2c6d79 net: remove unused Transport::SetReceiveVersion 2023-09-07 08:53:45 -04:00
Pieter Wuille
c3fad1f29d net: add have_next_message argument to Transport::GetBytesToSend()
Before this commit, there are only two possibly outcomes for the "more" prediction
in Transport::GetBytesToSend():
* true: the transport itself has more to send, so the answer is certainly yes.
* false: the transport has nothing further to send, but if vSendMsg has more message(s)
         left, that still will result in more wire bytes after the next
         SetMessageToSend().

For the BIP324 v2 transport, there will arguably be a third state:
* definitely not: the transport has nothing further to send, but even if vSendMsg has
                  more messages left, they can't be sent (right now). This happens
                  before the handshake is complete.

To implement this, we move the entire decision logic to the Transport, by adding a
boolean to GetBytesToSend(), called have_next_message, which informs the transport
whether more messages are available. The return values are still true and false, but
they mean "definitely yes" and "definitely no", rather than "yes" and "maybe".
2023-09-07 08:53:45 -04:00
fanquake
8e0d9796da Merge bitcoin/bitcoin#25284: net: Use serialization parameters for CAddress serialization
fa626af3ed Remove unused legacy CHashVerifier (MarcoFalke)
fafa3fc5a6 test: add tests that exercise WithParams() (MarcoFalke)
fac81affb5 Use serialization parameters for CAddress serialization (MarcoFalke)
faec591d64 Support for serialization parameters (MarcoFalke)
fac42e9d35 Rename CSerAction* to Action* (MarcoFalke)
aaaa3fa947 Replace READWRITEAS macro with AsBase wrapping function (MarcoFalke)

Pull request description:

  It seems confusing that picking a wrong value for `ADDRV2_FORMAT` could have effects on consensus. (See the docstring of `ADDRV2_FORMAT`).

  Fix this by implementing https://github.com/bitcoin/bitcoin/issues/19477#issuecomment-1147421608 .

  This may also help with libbitcoinkernel, see https://github.com/bitcoin/bitcoin/pull/28327

ACKs for top commit:
  TheCharlatan:
    ACK fa626af3ed
  ajtowns:
    ACK fa626af3ed

Tree-SHA512: 229d379da27308890de212b1fd2b85dac13f3f768413cb56a4b0c2da708f28344d04356ffd75bfcbaa4cabf0b6cc363c4f812a8f1648cff9e436811498278318
2023-09-07 11:34:34 +01:00
fanquake
5ad4eb3365 Merge bitcoin/bitcoin#28426: doc: s/--no-substitute/--no-substitutes in guix/INSTALL
8f541023b9 doc: s/--no-substitute/--no-substitutes in guix/INSTALL (fanquake)

Pull request description:

  https://guix.gnu.org/manual/en/html_node/Common-Build-Options.html

ACKs for top commit:
  hebasto:
    ACK 8f541023b9.
  TheCharlatan:
    ACK 8f541023b9

Tree-SHA512: 4d79791a8fa772d35723d35e85aaf9ab86304f2a1238e883bbb85051d0cc5f9841dab99ef0fe1e9d67f6259798e98942be00ae7c4320328987234293d9bf3012
2023-09-07 11:24:19 +01:00
fanquake
d98180a969 Merge bitcoin/bitcoin#28419: fuzz: introduce and use ConsumePrivateKey helper
583af18fd1 fuzz: introduce and use `ConsumePrivateKey` helper (Sebastian Falbesoner)

Pull request description:

  In the course of reviewing BIP324 related PRs I noticed a frequent pattern of creating private keys (`CKey` instances) with data consumed from the fuzz data provider:
  ```
      auto key_data = provider.ConsumeBytes<unsigned char>(32);
      key_data.resize(32);
      CKey key;
      key.Set(key_data.begin(), key_data.end(), /*fCompressedIn=*/true);
  ```
  This PR introduces a corresponding helper `ConsumePrivateKey` in order to deduplicate code. The compressed flag can either be set to a fixed value, or, if `std::nullopt` is passed (=default), is also consumed from the fuzz data provider via `.ConsumeBool()`.

  Note that this is not a pure refactor, as some of the replaced call-sites previously consumed a random length (`ConsumeRandomLengthByteVector`) instead of a fixed size of 32 bytes for key data. As far as I can see, there is not much value in using a random size, as in all those cases we can only proceed or do something useful with a valid private key, and key data with sizes other than 32 bytes always lead to invalid keys.

ACKs for top commit:
  sipa:
    utACK 583af18fd1
  brunoerg:
    crACK 583af18fd1

Tree-SHA512: 58a178432ba1eb0a2f7597b6700e96477e8b10f429ef642445a52db12e74d81aec307888315b772bfda9610f90df3e1d556cf024c2bef1d520303b12584feaaa
2023-09-07 11:20:12 +01:00
fanquake
8f541023b9 doc: s/--no-substitute/--no-substitutes in guix/INSTALL 2023-09-07 09:51:12 +01:00
Andrew Chow
d2ccca253f Merge bitcoin/bitcoin#26567: Wallet: estimate the size of signed inputs using descriptors
10546a569c wallet: accurately account for the size of the witness stack (Antoine Poinsot)
9b7ec393b8 wallet: use descriptor satisfaction size to estimate inputs size (Antoine Poinsot)
8d870a9873 script/signingprovider: introduce a MultiSigningProvider (Antoine Poinsot)
fa7c46b503 descriptor: introduce a method to get the satisfaction size (Antoine Poinsot)
bdba7667d2 miniscript: introduce a helper to get the maximum witness size (Antoine Poinsot)
4ab382c2cd miniscript: make GetStackSize independent of P2WSH context (Antoine Poinsot)

Pull request description:

  The wallet currently estimates the size of a signed input by doing a dry run of the signing logic. This is unnecessary since all outputs we can sign for can be represented by a descriptor, and we can derive the size of a satisfaction ("signature") directly from the descriptor itself.
  In addition, the current approach does not generalize well: dry runs of the signing logic are only possible for the most basic scripts. See for instance the discussion in #24149 around that.

  This introduces a method to get the maximum size of a satisfaction from a descriptor, and makes the wallet use that instead of the dry-run.

ACKs for top commit:
  sipa:
    utACK 10546a569c
  achow101:
    re-ACK 10546a569c

Tree-SHA512: 43ed1529fbd30af709d903c8c5063235e8c6a03b500bc8f144273d6184e23a53edf0fea9ef898ed57d8a40d73208b5d935cc73b94a24fad3ad3c63b3b2027174
2023-09-06 13:31:03 -04:00
fanquake
cf421820f5 Merge bitcoin/bitcoin#28409: test: Combine sync_send_with_ping and sync_with_ping
fae0b21e6c test: Combine sync_send_with_ping and sync_with_ping (MarcoFalke)

Pull request description:

  This reduces bloat, complexity, and makes tests less fragile to intermittent failures, see https://github.com/bitcoin/bitcoin/pull/27675#discussion_r1315648343.

  This should not cause any noticeable slowdown, or may even be faster, because active polling will be done at most once.

ACKs for top commit:
  glozow:
    Concept ACK fae0b21e6c
  theStack:
    ACK fae0b21e6c 🏓

Tree-SHA512: 6c543241a7b85458dc7ff6a6203316b80a6227d83d38427e74f53f4c666a882647a8a221e5259071ee7bb5dfd63476fb03c9b558a1ea546734b14728c3c619ba
2023-09-06 17:08:33 +01:00
Matthew Zipkin
2e249b9227 doc: add release note for PR #28414 2023-09-06 11:56:19 -04:00
ismaelsadeeq
4614332fc4 test: remove unnecessary finalizepsbt rpc calls 2023-09-06 11:52:19 -04:00
fanquake
7d5815293e depends: cctools 986 & ld64 711 2023-09-06 16:36:40 +01:00
fanquake
1ed1183f47 depends: libtapi 1300.0.6.5 2023-09-06 16:36:33 +01:00
Matthew Zipkin
5c2185b3b6 ci: enable chattr +i capability inside containers 2023-09-06 10:40:09 -04:00
Matthew Zipkin
e573f24202 unit test: add coverage for BlockManager 2023-09-06 10:39:54 -04:00
Sebastian Falbesoner
583af18fd1 fuzz: introduce and use ConsumePrivateKey helper 2023-09-06 13:59:12 +02:00
Matthew Zipkin
e3d484b603 wallet rpc: return final tx hex from walletprocesspsbt if complete 2023-09-05 09:14:32 -04:00
fanquake
9d3b216e00 Merge bitcoin/bitcoin#28151: build: use -muse-unaligned-vector-move for Windows builds
96f2cf8d2c build: use -muse-unaligned-vector-move for Windows (fanquake)

Pull request description:

  We currently work around a longstanding GCC issue with aligned vector instructions, by patching the behaviour we want into GCC (see discussion in #24736). Possibly in response to the GCC thread (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412#c40), a new option was [introduced into the binutils assembler](https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c8480b58e1968f209b6365af7422678f348222c2) with the 2.38 release:
  ```
  x86: Add -muse-unaligned-vector-move to assembler

  Unaligned load/store instructions on aligned memory or register are as
  fast as aligned load/store instructions on modern Intel processors.  Add
  a command-line option, -muse-unaligned-vector-move, to x86 assembler to
  encode encode aligned vector load/store instructions as unaligned
  vector load/store instructions.
  ```

ACKs for top commit:
  theuni:
    ACK 96f2cf8d2c.

Tree-SHA512: f5aaa125922bb17bbb51454103b3394d293184214b0dea554c36c2f130488a3ede2f39678044ec846fa0fdf4cd441d4227f4565c29d380f5a73b50bf6f3b9a67
2023-09-05 13:44:59 +01:00
brunoerg
3eb03803c4 test: remove unused variables in p2p_invalid_block 2023-09-05 09:09:45 -03:00
fanquake
ecab855838 Merge bitcoin/bitcoin#28195: blockstorage: Drop legacy -txindex check
fae405556d scripted-diff: Rename CBlockTreeDB -> BlockTreeDB (MarcoFalke)
faf63039cc Fixup style of moved code (MarcoFalke)
fa65111b99 move-only: Move CBlockTreeDB to node/blockstorage (MarcoFalke)
fa8685597e index: Drop legacy -txindex check (MarcoFalke)
fa69148a0a scripted-diff: Use blocks_path where possible (MarcoFalke)

Pull request description:

  The only reason for the check was to print a warning about an increase in storage use. Now that 22.x is EOL and everyone should have migrated (or decided to not care about storage use), remove the check.

  Also, a move-only commit is included. (Rebased from https://github.com/bitcoin/bitcoin/pull/22242)

ACKs for top commit:
  TheCharlatan:
    ACK fae405556d, though I lack historical context to really judge the second commit fa8685597e.
  stickies-v:
    ACK fae405556d

Tree-SHA512: 9da8f48767ae52d8e8e21c09a40c949cc0838794f1856cc5f58a91acd3f00a3bca818c8082242b3fdc9ca5badb09059570bb3870850d3807b75a8e23b5222da1
2023-09-05 11:37:35 +01:00
MarcoFalke
fae0b21e6c test: Combine sync_send_with_ping and sync_with_ping 2023-09-05 12:11:10 +02:00
fanquake
8c7e735456 Merge bitcoin/bitcoin#28404: Update libsecp256k1 subtree to release 0.4.0
c0da4f60e2 Squashed 'src/secp256k1/' changes from c545fdc374..199d27cea3 (Pieter Wuille)

Pull request description:

  We had previously pulled in a non-released commit along with #27479. The necessary changes have now been released in version 0.4.0, so update to that.

ACKs for top commit:
  hebasto:
    ACK 0e0fc18c3c, having a zero diff with my local branch that updates the `secp256k1` subtree up to v0.4.0.
  fanquake:
    ACK 0e0fc18c3c

Tree-SHA512: 8b771e7da89b9cdb7a680b9dd4eb99a6f737b32914b0b62c485b3c484e5438f9f60942030d3072243aaa196da22d2b1fdb3b6a668d75a46e6ac78c9d86b4bd8b
2023-09-05 10:38:21 +01:00
fanquake
260445bee8 Merge bitcoin/bitcoin#28359: ci: Asan with -ftrivial-auto-var-init=pattern
fa07ac48d8 ci: Asan with -ftrivial-auto-var-init=pattern (MarcoFalke)

Pull request description:

  This makes memory bugs deterministic. `-ftrivial-auto-var-init=pattern` is incompatible with other memory sanitizers (like valgrind and msan), but that is irrelevant here, because the address sanitizer in this fuzz CI config is already incompatible with them.

  `-ftrivial-auto-var-init=pattern` goes well with `-fsanitize=bool` and `-fsanitize=enum`, but those are already enabled via `-fsanitize=undefined`. See https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html#available-checks

ACKs for top commit:
  fanquake:
    ACK fa07ac48d8 - going to get back to fixing up the cxxflags usage in CI, but not a blocker here:

Tree-SHA512: 2ea6c5128a9cd262bdd1b1475c7e1be23ce2c520fad05f6c2aace6c29e203573323c0564d272e25da35ad5ff46fde488a5eae1ed14d3349e793d14a5e2533fb1
2023-09-05 10:07:50 +01:00
fanquake
c004ba48f4 Merge bitcoin/bitcoin#28386: test: remove fixed timeouts from feature_config_args
fbcacd4cf0 test: remove fixed timeouts from feature_config_args (Martin Zumsande)

Pull request description:

  Fixes #28290

  These fixed timeouts aren't affected by the `timeout_factor` option and can therefore cause timeouts in slow environments.
  They are also unnecessary for the test because they measure the wrong thing:
  While there is an internal waiting time of 60s within `ThreadOpenConnections` (beginning only when that thread is started) for fixed seeds querying, the timeouts here don't measure that but the time from startup until a debug log message is encountered, during which many other things happen in init, so they don't make much sense to me in the first place.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK fbcacd4cf0

Tree-SHA512: 7bb3b7db2f9666b1929ffb7773c838ee98b0845569428e5d00ecf5234973d534c4f474e213896c71baabd6096a79347bd21b41a17b130053049714eb8a447c79
2023-09-05 09:59:33 +01:00
fanquake
fb619e1bc3 Merge bitcoin/bitcoin#28396: test: p2p: check that getaddr msgs are only responded once per connection
668aa6af8d test: p2p: check that `getaddr` msgs are only responded once per connection (Sebastian Falbesoner)

Pull request description:

  This simple PR adds missing test coverage for ignoring repeated `getaddr` requests (introduced in #7856, commit 66b07247a7):
  6f03c45f6b/src/net_processing.cpp (L4642-L4648)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 668aa6af8d
  brunoerg:
    crACK 668aa6af8d

Tree-SHA512: edcdc6501c684fb41911e393f55ded9b044cd2f92918877eca152edd5a4287d1a9d57ae999f1cb42185eae00c3a0af411fcb9bcd5b990ef48849c3834b141584
2023-09-05 09:58:50 +01:00
fanquake
818bfd6dfe Merge bitcoin/bitcoin#28402: ci: Bump actions/checkout version
f10acecd43 ci: Bump `actions/checkout` version (Hennadii Stepanov)

Pull request description:

  See: https://github.com/actions/checkout/releases/tag/v4.0.0.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK f10acecd43

Tree-SHA512: fc71876ee1df881a61295fc3b3cd3dac0c17b3651ad725076d3ff0b8975bab0d37c03cd421e6e88ddcf2e28910c96e3f1fa8fd5d5280e06e5a7a7ec683f7b930
2023-09-05 11:43:51 +03:00
fanquake
337d6f35a2 Merge bitcoin/bitcoin#28291: rpc: removed StrFormatInternalBug quote delimitation
6e8f6468cb removed StrFormatInternalBug quote delimitation (Reese Russell)

Pull request description:

  This PR rectifies an unnecessary set of quotes delimiting the contents of  ```StrFormatInternalBug```. This is a follow up to MarcoFalke https://github.com/bitcoin/bitcoin/pull/28123#discussion_r1297191493.  The method of action was to remove the escaped quotes that were a part of strprintf. A single functional test case was modified to reflect the new output format.

  ```STR_INTERNAL_BUG``` was applied to https://github.com/bitcoin/bitcoin/pull/28123 in ```std::string RPCArg::ToString(const bool oneline)``` in ```rpc/util.cpp```

  The results can be seen below.

  Previously
  ![image](https://github.com/bitcoin/bitcoin/assets/3104223/53f9ea59-317f-4c62-9fc1-04255eeb4641)

  This PR
  ![image](https://github.com/bitcoin/bitcoin/assets/3104223/5c6a3110-f1f3-4b3c-8e8a-9c8f1c3176e7)

  Additional context can be found here.
  https://github.com/bitcoin/bitcoin/pull/28123#discussion_r1271871716

  Thank you.

ACKs for top commit:
  MarcoFalke:
    review ACK 6e8f6468cb
  stickies-v:
    ACK 6e8f6468cb

Tree-SHA512: 35317e31a527630495b566407e37db9941dab7f81cfaeb1ea3309683c48e4273284645ad615f73e646a137b4f2ae35933603e9182a7dbdd22cac98d038c491dc
2023-09-05 11:41:02 +03:00
MarcoFalke
fa626af3ed Remove unused legacy CHashVerifier 2023-09-05 10:13:50 +02:00
MarcoFalke
fafa3fc5a6 test: add tests that exercise WithParams()
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2023-09-05 10:13:41 +02:00
MarcoFalke
fac81affb5 Use serialization parameters for CAddress serialization
This also cleans up the addrman (de)serialization code paths to only
allow `Disk` serialization. Some unit tests previously forced a
`Network` serialization, which does not make sense, because Bitcoin Core
in production will always `Disk` serialize.
This cleanup idea was suggested by Pieter Wuille and implemented by Anthony
Towns.

Co-authored-by: Pieter Wuille <pieter@wuille.net>
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2023-09-05 10:13:25 +02:00
Pieter Wuille
0e0fc18c3c Update secp256k1 subtree to upstream release 0.4.0 2023-09-04 12:51:20 -04:00
Pieter Wuille
c0da4f60e2 Squashed 'src/secp256k1/' changes from c545fdc374..199d27cea3
199d27cea3 Merge bitcoin-core/secp256k1#1415: release: Prepare for 0.4.0
16339804c9 release: Prepare for 0.4.0
d9a85065a9 changelog: Catch up in preparation of release
0b4640aedd Merge bitcoin-core/secp256k1#1413: ci: Add `release` job
8659a01714 ci: Add `release` job
f9b38894ba ci: Update `actions/checkout` version
727bec5bc2 Merge bitcoin-core/secp256k1#1414: ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot
2635068abf ci/gha: Let MSan continue checking after errors in all jobs
e78c7b68eb ci/Dockerfile: Reduce size of Docker image further
2f0d3bbffb ci/Dockerfile: Warn if `ulimit -n` is too high when running Docker
4b8a647ad3 ci/gha: Add ARM64 QEMU jobs for clang and clang-snapshot
6ebe7d2bb3 ci/Dockerfile: Always use versioned clang packages
65c79fe2d0 Merge bitcoin-core/secp256k1#1412: ci: Switch macOS from Ventura to Monterey and add Valgrind
c223d7e33d ci: Switch macOS from Ventura to Monterey and add Valgrind
ea26b71c3a Merge bitcoin-core/secp256k1#1411: ci: Make repetitive command the default one
cce0456304 ci: Make repetitive command the default one
317a4c48f0 ci: Move `git config ...` to `run-in-docker-action`
4d7fe60905 Merge bitcoin-core/secp256k1#1409: ci: Move remained task from Cirrus to GitHub Actions
676ed8f9cf ci: Move "C++ (public headers)" from Cirrus to GitHub Actions
61fc3a2dc8 ci: Move "C++ -fpermissive..." from Cirrus to GitHub Actions
d51fb0a533 ci: Move "MSan" from Cirrus to GitHub Actions
c22ac27529 ci: Move sanitizers task from Cirrus to GitHub Actions
26a989924b Merge bitcoin-core/secp256k1#1410: ci: Use concurrency for pull requests only
ee1be62d84 ci: Use concurrency for pull requests only
6ee14550c8 Merge bitcoin-core/secp256k1#1406: ci, gha: Move more non-x86_64 tasks from Cirrus CI to GitHub Actions
fc3dea29ea ci: Move "ppc64le: Linux..." from Cirrus to GitHub Actions
7782dc8276 ci: Move "ARM64: Linux..." from Cirrus to GitHub Actions
0a16de671c ci: Move "ARM32: Linux..." from Cirrus to GitHub Actions
ea33914e00 ci: Move "s390x (big-endian): Linux..." from Cirrus to GitHub Actions
880be8af99 ci: Move "i686: Linux (Debian stable)" from Cirrus to GiHub Actions
2e6cf9bae5 Merge bitcoin-core/secp256k1#1396: ci, gha: Add "x86_64: Linux (Debian stable)" GitHub Actions job
5373693e45 Merge bitcoin-core/secp256k1#1405: ci: Drop no longer needed workaround
ef9fe959de ci: Drop no longer needed workaround
e10878f58e ci, gha: Drop `driver-opts.network` input for `setup-buildx-action`
4ad4914bd1 ci, gha: Add `retry_builder` Docker image builder
6617a620d9 ci: Remove "x86_64: Linux (Debian stable)" task from Cirrus CI
03c9e6508c ci, gha: Add "x86_64: Linux (Debian stable)" GitHub Actions job
ad3e65d9fe ci: Remove GCC build files and sage to reduce size of Docker image
6b9507adf6 Merge bitcoin-core/secp256k1#1398: ci, gha: Add Windows jobs based on Linux image
87d35f30c0 ci: Rename `cirrus.sh` to more general `ci.sh`
d6281dd008 ci: Remove Windows tasks from Cirrus CI
2b6f9cd546 ci, gha: Add Windows jobs based on Linux image
48b1d939b5 Merge bitcoin-core/secp256k1#1403: ci, gha: Ensure only a single workflow processes `github.ref` at a time
0ba2b94551 Merge bitcoin-core/secp256k1#1373: Add invariant checking for scalars
060e32cb60 Merge bitcoin-core/secp256k1#1401: ci, gha: Run all MSVC tests on Windows natively
de657c2044 Merge bitcoin-core/secp256k1#1062: Removes `_fe_equal_var`, and unwanted `_fe_normalize_weak` calls (in tests)
bcffeb14bc Merge bitcoin-core/secp256k1#1404: ci: Remove "arm64: macOS Ventura" task from Cirrus CI
c2f6435802 ci: Add comment about switching macOS to M1 on GHA later
4a24fae0bc ci: Remove "arm64: macOS Ventura" task from Cirrus CI
b0886fd35c ci, gha: Ensure only a single workflow processes `github.ref` at a time
3d05c86d63 Merge bitcoin-core/secp256k1#1394: ci, gha: Run "x86_64: macOS Ventura" job on GitHub Actions
d78bec7001 ci: Remove Windows MSVC tasks from Cirrus CI
3545dc2b9b ci, gha: Run all MSVC tests on Windows natively
5d8fa825e2 Merge bitcoin-core/secp256k1#1274: test: Silent noisy clang warnings about Valgrind code on macOS x86_64
8e54a346d2 ci, gha: Run "x86_64: macOS Ventura" job on GitHub Actions
b327abfcea Merge bitcoin-core/secp256k1#1402: ci: Use Homebrew's gcc in native macOS task
d62db57427 ci: Use Homebrew's gcc in native macOS task
54058d16fe field: remove `secp256k1_fe_equal_var`
bb4efd6404 tests: remove unwanted `secp256k1_fe_normalize_weak` call
eedd781085 Merge bitcoin-core/secp256k1#1348: tighten group magnitude limits, save normalize_weak calls in group add methods (revival of #1032)
b2f6712dd3 Merge bitcoin-core/secp256k1#1400: ctimetests: Use new SECP256K1_CHECKMEM macros also for ellswift
9c91ea41b1 ci: Enable ellswift module where it's missing
db32a24761 ctimetests: Use new SECP256K1_CHECKMEM macros also for ellswift
ce765a5b8e Merge bitcoin-core/secp256k1#1399: ci, gha: Run "SageMath prover" job on GitHub Actions
8408dfdc4c Revert "ci: Run sage prover on CI"
c8d9914fb1 ci, gha: Run "SageMath prover" job on GitHub Actions
8d2960c8e2 Merge bitcoin-core/secp256k1#1397: ci: Remove "Windows (VS 2022)" task from Cirrus CI
f1774e5ec4 ci, gha: Make MSVC job presentation more explicit
5ee039bb58 ci: Remove "Windows (VS 2022)" task from Cirrus CI
96294c00fb Merge bitcoin-core/secp256k1#1389: ci: Run "Windows (VS 2022)" job on GitHub Actions
a2f7ccdecc ci: Run "Windows (VS 2022)" job on GitHub Actions
374e2b54e2 Merge bitcoin-core/secp256k1#1290: cmake: Set `ENVIRONMENT` property for examples on Windows
1b13415df9 Merge bitcoin-core/secp256k1#1391: refactor: take use of `secp256k1_scalar_{zero,one}` constants (part 2)
a1bd4971d6 refactor: take use of `secp256k1_scalar_{zero,one}` constants (part 2)
b7c685e74a Save _normalize_weak calls in group add methods
c83afa66e0 Tighten group magnitude limits
26392da2fb Merge bitcoin-core/secp256k1#1386: ci: print $ELLSWIFT in cirrus.sh
d23da6d557 use secp256k1_scalar_verify checks
4692478853 ci: print $ELLSWIFT in cirrus.sh
c7d0454932 add verification for scalars
c734c64278 Merge bitcoin-core/secp256k1#1384: build: enable ellswift module via SECP_CONFIG_DEFINES
ad152151b0 update max scalar in scalar_cmov_test and fix schnorrsig_verify exhaustive test
78ca880788 build: enable ellswift module via SECP_CONFIG_DEFINES
0e00fc7d10 Merge bitcoin-core/secp256k1#1383: util: remove unused checked_realloc
b097a466c1 util: remove unused checked_realloc
2bd5f3e618 Merge bitcoin-core/secp256k1#1382: refactor: Drop unused cast
4f8c5bd761 refactor: Drop unused cast
173e8d061a Implement current magnitude assumptions
49afd2f5d8 Take use of _fe_verify_magnitude in field_impl.h
4e9661fc42 Add _fe_verify_magnitude (no-op unless VERIFY is enabled)
690b0fc05a add missing group element invariant checks
175db31149 ci: Drop no longer needed `PATH` variable update on Windows
116d2ab3df cmake: Set `ENVIRONMENT` property for examples on Windows
cef373997c cmake, refactor: Use helper function instead of interface library
747ada3587 test: Silent noisy clang warnings about Valgrind code on macOS x86_64

git-subtree-dir: src/secp256k1
git-subtree-split: 199d27cea32203b224b208627533c2e813cd3b21
2023-09-04 12:51:20 -04:00
MarcoFalke
fa5356cd49 ci: Limit test-each-commit to --max-count=6 2023-09-04 18:04:49 +02:00
MarcoFalke
fafcd2e9ef ci: Add test-each-commit task 2023-09-04 18:04:47 +02:00
Hennadii Stepanov
f10acecd43 ci: Bump actions/checkout version
See: https://github.com/actions/checkout/releases/tag/v4.0.0
2023-09-04 15:25:41 +01:00
Sebastian Falbesoner
668aa6af8d test: p2p: check that getaddr msgs are only responded once per connection 2023-09-03 14:19:41 +02:00
fanquake
6f03c45f6b Merge bitcoin/bitcoin#28383: Update translations for 26.0 soft translation string freeze
162ce4e5dd qt: Update translation source file (Hennadii Stepanov)
19520bff09 qt: Bump Transifex slug for 26.x (Hennadii Stepanov)
69821b27f5 qt: Translation updates from Transifex (Hennadii Stepanov)

Pull request description:

  This PR follows our [Release Process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md).

  Required to open Transifex translations for 26.0 on 2023-09-01 as it's [planned](https://github.com/bitcoin/bitcoin/issues/27758).

  The previous similar PR: https://github.com/bitcoin/bitcoin/pull/27169.

ACKs for top commit:
  stickies-v:
    ACK 162ce4e5dd
  jarolrod:
    ACK 162ce4e5dd

Tree-SHA512: a810c727e5c9fb3ffe4ff88ca7c5484178c483b38adb4b67758fb1f3e75b0d0ffb755fdc43de3fa59607f45833ec3594d87649fd6f0a40ed8a249ee5fa6979dd
2023-09-02 20:46:33 +03:00
Martin Zumsande
fbcacd4cf0 test: remove fixed timeouts from feature_config_args
They cannot be scaled by the timeout_factor option and can
therefore cause timeouts in slow environments.
They are also not necessary for the test, since they measure time
frome startup until a debug message is encountered, which
is not restricted to 1 minute by any internal logic within bitcoind.
2023-09-01 14:48:55 -04:00
Andrew Chow
df98a12fc2 Merge bitcoin/bitcoin#28350: Log explicit error message when coindb is found in inconsistent state
df60de770d log: Print error message when coindb is in inconsistent state (Fabian Jahr)

Pull request description:

  While doing manual testing on assumeutxo this week I managed to put the coindb into an inconsistent state twice. For a normal user, this can also happen if their computer crashes during a flush or if they try to stop their node during a flush and then get tired of waiting and just shut their computer down or kill the process. It's an edge case but I wouldn't be surprised if this does happen more often when assumeutxo gets used more widely because there might be multiple flushes happening during loading of the UTXO set in the beginning and users may think something is going wrong because of the unexpected wait or they forgot some configs and want to start over quickly.

  The problem is, when this happens at first the node starts up normally until it's time to flush again and then it hits an assert that the user can not understand.

  ```
  2023-08-25T16:31:09Z [httpworker.0] [snapshot] 52000000 coins loaded (43.30%, 6768 MB)
  2023-08-25T16:31:16Z [httpworker.0] Cache size (7272532192) exceeds total space (7256510300)
  2023-08-25T16:31:16Z [httpworker.0] FlushSnapshotToDisk: flushing coins cache (7272 MB) started
  Assertion failed: (old_heads[0] == hashBlock), function BatchWrite, file txdb.cpp, line 126.
  Abort trap: 6
  ```

  We should at least log an error message that gives users a hint of what the problem is and what they can do to resolve it. I am keeping this separate from the assumeutxo project since this issue can also happen during any regular flush.

ACKs for top commit:
  jonatack:
    ACK df60de770d
  achow101:
    ACK df60de770d
  ryanofsky:
    Code review ACK df60de770d
  jamesob:
    Code review ACK df60de770d

Tree-SHA512: b546aa0b0323ece2962867a29c38e014ac83ae8f1ded090da2894b4ff2450c05229629c7e8892f7b550cf7def4038a0b4119812e548e11b00c60b1dc3d4276d2
2023-09-01 13:25:01 -04:00
glozow
b2f1d732cb Merge bitcoin/bitcoin#28384: ci: Avoid oversubscription in functional tests on Windows
f2d4e510b3 ci: Avoid saving the same Ccache cache (Hennadii Stepanov)
14e5de6d02 ci: Avoid oversubscription in functional tests on Windows (Hennadii Stepanov)

Pull request description:

  This PR aims to reduce the frequency of functional test failures on Windows like this [one](https://github.com/bitcoin/bitcoin/actions/runs/6040229997):
  ```

  2023-09-01T01:05:01.850000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "D:\a\bitcoin\bitcoin\test\functional\test_framework\test_framework.py", line 552, in start_nodes
      node.wait_for_rpc_connection()
    File "D:\a\bitcoin\bitcoin\test\functional\test_framework\test_node.py", line 296, in wait_for_rpc_connection
      self._raise_assertion_error("Unable to connect to bitcoind after {}s".format(self.rpc_timeout))
    File "D:\a\bitcoin\bitcoin\test\functional\test_framework\test_node.py", line 177, in _raise_assertion_error
      raise AssertionError(self._node_msg(msg))
  AssertionError: [node 1] Unable to connect to bitcoind after 2400s
  ```

  This code has had zero failures in my personal repository in more than 25 runs (and is still counting).

  ---

  The second commit is a minor improvement to avoid "Cache save failed." warnings during job re-runs. For [example](https://github.com/bitcoin/bitcoin/actions/runs/5998688759):

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/d8a049df-fccd-4395-99c9-4be01d0ea706)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK f2d4e510b3 🐾

Tree-SHA512: 0c92817d37325a114886900e49a4d644201397d98d6ac9f2dcd41170c7e7ea2cb1873f7e51b5cb3ad3cc2e59554ad1c8f87d439ea6c1c960bf5c339153be7040
2023-09-01 13:55:11 +01:00
Hennadii Stepanov
f2d4e510b3 ci: Avoid saving the same Ccache cache
This occurred when a job was being rerun.
2023-09-01 11:08:20 +01:00
Hennadii Stepanov
14e5de6d02 ci: Avoid oversubscription in functional tests on Windows 2023-09-01 11:08:05 +01:00
Hennadii Stepanov
162ce4e5dd qt: Update translation source file
The diff is generated by executing `make -C src translate`.
2023-09-01 08:08:36 +01:00
Hennadii Stepanov
19520bff09 qt: Bump Transifex slug for 26.x 2023-09-01 07:49:31 +01:00
Hennadii Stepanov
69821b27f5 qt: Translation updates from Transifex
The diff is generated by executing the `update-translations.py` script.
2023-09-01 07:39:00 +01:00
TheCharlatan
d8041d4e04 blockstorage: Return on fatal undo file flush error
By returning an error code if either `FlushUndoFile` or `FlushBlockFile`
fail, the caller now has to explicitly handle block undo file flushing
errors. Before this change such errors were non-explicitly ignored
without a clear rationale.

Besides the call to `FlushUndoFile` in `FlushBlockFile`, ignore its
return code at its call site in `WriteUndoDataForBlock`. There, a failed
flush of the undo data should not be indicative of a failed write.

Add [[nodiscard]] annotations to `FlushUndoFile` such that its return
value is not just ignored in the future.
2023-08-31 23:26:51 +02:00
TheCharlatan
f0207e0030 blockstorage: Return on fatal block file flush error
By returning an error code if `FlushBlockFile` fails, the caller now has
to explicitly handle block file flushing errors. Before this change
such errors were non-explicitly ignored without a clear rationale.

Prior to this patch `FlushBlockFile` may have failed silently in
`Chainstate::FlushStateToDisk`. Improve this with a log line. Also add a
TODO comment to flesh out whether returning early in the case of an
error is appropriate or not. Returning early might be appropriate to
prohibit `WriteBlockIndexDB` from writing a block index entry that does
not refer to a fully flushed block.

Besides `Chainstate::FlushStateToDisk`, `FlushBlockFile` is also called
by `FindBlockPos`. Don't change the abort behavior there, since we don't
want to fail the function if the flushing of already written blocks
fails. Instead, just document it.
2023-08-31 23:26:44 +02:00
TheCharlatan
5671c15f45 blockstorage: Mark FindBlockPos as nodiscard
A false return value indicates a fatal error (disk space being too low),
so make sure we always consume this error code.

This commit is part of an ongoing process for making the handling of
fatal errors more transparent and easier to understand.
2023-08-31 22:53:08 +02:00
Tim Neubauer
27b4084e16 Refactor: Remove m_is_test_chain 2023-08-31 18:59:43 +02:00
Andrew Chow
5666966dff Merge bitcoin/bitcoin#28364: log: log wtxids when possible, add TXPACKAGES category
a3b55c94b9 [doc] move comment about AlreadyHaveTx DoS score to the right place (glozow)
3b8c17838a [log] add more logs related to orphan handling (glozow)
51b3275cd1 [log] add category TXPACKAGES for orphanage and package relay (glozow)
a33dde1e41 [log] include wtxid in tx {relay,validation,orphanage} logging (glozow)

Pull request description:

  This was taken from #28031 (see #27463 for project tracking).

  - Log wtxids in addition to txids when possible. This allows us to track the fate of a transaction from inv to mempool accept/reject through logs.
  - Additional orphan-related logging to make testing and debugging easier. Suggested in https://github.com/bitcoin/bitcoin/pull/28031#pullrequestreview-1531022386 and https://github.com/bitcoin/bitcoin/pull/28031#discussion_r1269622220
  - Add `TXPACKAGES` category for logging.
  - Move a nearby comment block that was in the wrong place.

ACKs for top commit:
  instagibbs:
    reACK a3b55c94b9
  achow101:
    ACK a3b55c94b9
  brunoerg:
    crACK a3b55c94b9
  mzumsande:
    Code review ACK a3b55c94b9

Tree-SHA512: 21884ef7c2ea2fd006e715574a9dd3e6cbbe8f82d62c6187fe1d39aad5a834051203fda5f355a06ca40c3e2b9561aec50d7c922a662b1edc96f7b552c9f4b24d
2023-08-31 12:55:38 -04:00
fanquake
a4e0bcb6c9 Merge bitcoin/bitcoin#28369: doc: Fill in the required skills in the good_first_issue template
fa3b816240 doc: Fill in the required skills in the good_first_issue template (MarcoFalke)

Pull request description:

  Compiling and running the tests is always required, so fill it in to avoid having to type it manually every time.

ACKs for top commit:
  willcl-ark:
    ACK fa3b816

Tree-SHA512: 1bcb93aaff235dd62513cda05547db90d12ad7638c050ee125845d20df1e1bc457bf4ec590677a0875fae8729dcc58842398e637e517997b35e3b3adffc34a72
2023-08-31 13:37:09 +01:00
Hennadii Stepanov
9561e497e9 Merge bitcoin-core/gui#749: make '-min' minimize wallet loading dialog
32db15450a gui: make '-min' minimize wallet loading dialog (furszy)

Pull request description:

  Simple fix for #748.

  When '-min' is enabled, no loading dialog should
  be presented on screen during startup.

ACKs for top commit:
  hebasto:
    ACK 32db15450a, tested on Debian 11 + XFCE.

Tree-SHA512: d08060b044938c67e8309db77b49ca645850fc21fdd7d78d5368d336fb9f602dcc66ea398a7505b00bf7d43afa07108347c7260480319fad3ec84cb41332f780
2023-08-31 12:29:24 +01:00
josibake
1580e3be83 fuzz: add ConstructPubKeyBytes function
Today, this code only has one spot where it needs well-formed pubkeys,
but future PRs will want to reuse this code.

Add a function which creates a well-formed byte array that can be turned
into a pubkey. It is not required that the pubkey is valid, just that it
can be recognized as a compressed or uncompressed pubkey.

Note: while the main intent of this commit is to wrap the existing
logic into a function, it also switches to `PickValueFromArray` so that
we are only choosing one of 0x04, 0x06, or 0x07. The previous code,
`ConsumeIntegralInRange` would have also picked 0x05, which is not
definied in the context of compressed vs uncompressed keys.

See https://bitcoin.stackexchange.com/questions/57855/c-secp256k1-what-do-prefixes-0x06-and-0x07-in-an-uncompressed-public-key-signif
for more details.
2023-08-30 17:45:51 +02:00
fanquake
96f2cf8d2c build: use -muse-unaligned-vector-move for Windows
We currently work around a longstanding GCC issue with aligned vector
instructions, in our release builds, by patching the behaviour we want
into GCC (see discussion in #24736).

A new option now exists in the binutils assembler,
`-muse-unaligned-vector-move`, which should also achieve the behaviour
we want (at least for our code). This was added in the 2.38 release,
see
https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=c8480b58e1968f209b6365af7422678f348222c2.
```bash
x86: Add -muse-unaligned-vector-move to assembler

Unaligned load/store instructions on aligned memory or register are as
fast as aligned load/store instructions on modern Intel processors.  Add
a command-line option, -muse-unaligned-vector-move, to x86 assembler to
encode encode aligned vector load/store instructions as unaligned
vector load/store instructions.
```

Even if we introduce this option into our build system, we'll have to
maintain our GCC patching, as we want all code that ends up in the
binary, to avoid these instructions. However, there may be some value in
adding the option, as it could be an improvement for someone building
(bitcoind.exe) with an unpatched compiler.
2023-08-30 16:34:37 +01:00
fanquake
df5af114df Merge bitcoin/bitcoin#28353: ci: Remove /ro_base bind mount, Remove LC_ALL=C from s390x task
fa70cbd969 ci: Remove unused TEST_RUNNER_ENV="LC_ALL=C" from s390x task (MarcoFalke)
fa33354dcb ci: Remove /ro_base bind mount (MarcoFalke)
fa0df9d4c4 doc: Remove sudo from command that is already run as root (MarcoFalke)

Pull request description:

  Remove some CI stuff no longer needed.

ACKs for top commit:
  fanquake:
    ACK  fa70cbd969 - did not test the s390x job.

Tree-SHA512: 3a6ed0cfc855a92c2f834e59494c0a19a5647510247aece5e40a1aa78074894fe7454e684a1ea1f8f0662c50ac1caf2e390398b0fcfbf81544e6488fa9b8915e
2023-08-30 16:23:39 +01:00
fanquake
505ea30b47 Merge bitcoin/bitcoin#28370: guix: remove GCC 10 workaround from NSIS
588068d5a1 guix: remove GCC 10 workaround from NSIS (fanquake)

Pull request description:

  Fixed upstream in 3.06.
  See 229b6136c4 and https://sourceforge.net/p/nsis/bugs/1248/.

  Split from #27897, as that is still WIP, and GCC 12 isn't required to make this change.
  This commit could have gone in with #28328.

  Guix Build (same for x86_64 & aarch64):
  ```bash
  81a565bc5c719d04cb6640cf7593d7fdd6e37935dc2ac9280f638e625c1c5f87  guix-build-588068d5a12b/output/aarch64-linux-gnu/SHA256SUMS.part
  6f1ed3a2a053c3e90f1511fe33239378fc2631f46d795a955807ab39e2b93dfc  guix-build-588068d5a12b/output/aarch64-linux-gnu/bitcoin-588068d5a12b-aarch64-linux-gnu-debug.tar.gz
  c8a2904c651b862d9dce3f4094b59b66be5e745111b75c420452bd04ab9cdbe3  guix-build-588068d5a12b/output/aarch64-linux-gnu/bitcoin-588068d5a12b-aarch64-linux-gnu.tar.gz
  654bb5d89e12f936e63e2c910c78842c43db7048a895052295f613a50b639857  guix-build-588068d5a12b/output/arm-linux-gnueabihf/SHA256SUMS.part
  8a1b67f3c12b579b374c559f59d709bd83bcd3cff3ec5a4274119c0e5f6c388f  guix-build-588068d5a12b/output/arm-linux-gnueabihf/bitcoin-588068d5a12b-arm-linux-gnueabihf-debug.tar.gz
  7d8a3327599702a94be4b8d49eda7857f732c290773a775c6de507da656e271b  guix-build-588068d5a12b/output/arm-linux-gnueabihf/bitcoin-588068d5a12b-arm-linux-gnueabihf.tar.gz
  215786257409612206db7ff4008dbcb6a60dcf6cc92a1a437124a1483612f699  guix-build-588068d5a12b/output/arm64-apple-darwin/SHA256SUMS.part
  e36b41c531d34ca843028b778163bed4ce4c52310ed8056eff8a1a1b47b61fb7  guix-build-588068d5a12b/output/arm64-apple-darwin/bitcoin-588068d5a12b-arm64-apple-darwin-unsigned.dmg
  ecb488625bdf566e7c63781795f296ec4f630ef90edf7e6073591ed1d9b90bda  guix-build-588068d5a12b/output/arm64-apple-darwin/bitcoin-588068d5a12b-arm64-apple-darwin-unsigned.tar.gz
  d8f9759a903cd052ea3e562d1ef186c9dad38089205650d1bcb218e7506c378f  guix-build-588068d5a12b/output/arm64-apple-darwin/bitcoin-588068d5a12b-arm64-apple-darwin.tar.gz
  385084fe6d80d0b563b1b20d558b29681fd5bf624bc70c403e831cd8aae45dc0  guix-build-588068d5a12b/output/dist-archive/bitcoin-588068d5a12b.tar.gz
  89504c8be28482d4f70740d6865aa26db0e1854d0b35598c43462cd35f0b49c4  guix-build-588068d5a12b/output/powerpc64-linux-gnu/SHA256SUMS.part
  5836bfb4c9a11318362a89db25e4cd9363688ae6be18a6abfd1974a52c0b2759  guix-build-588068d5a12b/output/powerpc64-linux-gnu/bitcoin-588068d5a12b-powerpc64-linux-gnu-debug.tar.gz
  3e6f4481a594316ab08ea946c5fac6055ca8a81a91ce0a20c44976b3bc4f8d0d  guix-build-588068d5a12b/output/powerpc64-linux-gnu/bitcoin-588068d5a12b-powerpc64-linux-gnu.tar.gz
  4e753efe5a20cf6f1e83e604d9f8ba0d79b3c96fc5d5487ec7ed0f33c819b99a  guix-build-588068d5a12b/output/powerpc64le-linux-gnu/SHA256SUMS.part
  9d29e49b519ce1d683531699dd15e72495052151b21404affc50e7b228155eb1  guix-build-588068d5a12b/output/powerpc64le-linux-gnu/bitcoin-588068d5a12b-powerpc64le-linux-gnu-debug.tar.gz
  511179d90bbf6beb45b6168575a78600d38a060891dc1aa8f11f0adc5ba51f87  guix-build-588068d5a12b/output/powerpc64le-linux-gnu/bitcoin-588068d5a12b-powerpc64le-linux-gnu.tar.gz
  d46184105524d59a75066b067b63df6bfd2203d8003c343a5898ee7d4e4f78e5  guix-build-588068d5a12b/output/riscv64-linux-gnu/SHA256SUMS.part
  b6434ec5a4a046b9448f36f5596f6db0f8cd71a3e4383827869fcbaa25e005e4  guix-build-588068d5a12b/output/riscv64-linux-gnu/bitcoin-588068d5a12b-riscv64-linux-gnu-debug.tar.gz
  237e37ccae4095a1d8aa34d6b967655561ba0a524dfb79e3dcd4c18ad8f8a222  guix-build-588068d5a12b/output/riscv64-linux-gnu/bitcoin-588068d5a12b-riscv64-linux-gnu.tar.gz
  db28897936262e344da8011d37afde97ed53ba4f5ce1a3f935d258284b22e649  guix-build-588068d5a12b/output/x86_64-apple-darwin/SHA256SUMS.part
  8d71763fc748cde1f68785eef4b1a498c1ec4c13cf94f02ad8727debe0d94a1d  guix-build-588068d5a12b/output/x86_64-apple-darwin/bitcoin-588068d5a12b-x86_64-apple-darwin-unsigned.dmg
  1e0bf2ef0f65f2e7fe5439fe0a15a4803e06d7c42dae06133be5d1b67fa30657  guix-build-588068d5a12b/output/x86_64-apple-darwin/bitcoin-588068d5a12b-x86_64-apple-darwin-unsigned.tar.gz
  2d71083593583f3a3ed2e83d4a05f67338e9cb51cb730d238827eef7d3e9007e  guix-build-588068d5a12b/output/x86_64-apple-darwin/bitcoin-588068d5a12b-x86_64-apple-darwin.tar.gz
  cc3cc2c1077a04d5643ca21a43b8b7f233b8c743eb73f066643ddfd4a1e1ce27  guix-build-588068d5a12b/output/x86_64-linux-gnu/SHA256SUMS.part
  e18c1c0c1cf836c9e540edd969d2657afd3847345b1e0a6e5a9bebfd35782c47  guix-build-588068d5a12b/output/x86_64-linux-gnu/bitcoin-588068d5a12b-x86_64-linux-gnu-debug.tar.gz
  4c88aa50227ccd7bb9069369cfc2196de42212f5d0553902155346ae39ef2401  guix-build-588068d5a12b/output/x86_64-linux-gnu/bitcoin-588068d5a12b-x86_64-linux-gnu.tar.gz
  51f9f0d9c21598d9d37132e21c786378468c50c393b46e084d81c504186238cd  guix-build-588068d5a12b/output/x86_64-w64-mingw32/SHA256SUMS.part
  a4a0b55a40ba17b70c93eaba132e31c94cb2380cad0fe4e2ebe199c68a49c293  guix-build-588068d5a12b/output/x86_64-w64-mingw32/bitcoin-588068d5a12b-win64-debug.zip
  75495c9a00ee7401b96afa51c93fe56d8b10a531ce0526f5b4713b9e8aaab9a5  guix-build-588068d5a12b/output/x86_64-w64-mingw32/bitcoin-588068d5a12b-win64-setup-unsigned.exe
  a6e8416846ebea253d2b3c5c40f60a227bd5cfb2272a477cf364a085fea140e4  guix-build-588068d5a12b/output/x86_64-w64-mingw32/bitcoin-588068d5a12b-win64-unsigned.tar.gz
  2bbd6cd49ad0297a5b3279a5cfbcbb35586cfdd72388e5e1ef2d87e1c8a9cf97  guix-build-588068d5a12b/output/x86_64-w64-mingw32/bitcoin-588068d5a12b-win64.zip
  ```

ACKs for top commit:
  hebasto:
    ACK 588068d5a1.

Tree-SHA512: e5876d64ee02fef8a7152b9f649949933b356347de713e00a58aba278243b7de999c3fc50f3cc88e625d07a38e5a6ec2afcb6e836d256b6ac91e91f90f473e64
2023-08-30 16:19:59 +01:00
fanquake
1348454d82 Merge bitcoin/bitcoin#28352: test: Support powerpc64le in get_previous_releases.py
faf7e69862 test: Support powerpc64le in get_previous_releases.py (MarcoFalke)

Pull request description:

  To test: `test/get_previous_releases.py -b -t /tmp/prev_releases v22.0`

  On master: `Not sure which binary to download for powerpc64le-unknown-linux-gnu`
  Here: (pass)

ACKs for top commit:
  fanquake:
    ACK faf7e69862

Tree-SHA512: 33d9348f99e0d3924a6a5cba8833ec9e413e80167012b557922f3628069dabd555b02f98a6bfd0eb80e2bbbcdb50865b7bca216e1d080b1546ee4812abda4bc2
2023-08-30 16:16:44 +01:00
fanquake
13e169a0c6 Merge bitcoin/bitcoin#28360: ci, windows: Do not run extended functional tests for pull requests
62ab3e98ff ci, windows: Do not run extended functional tests for pull requests (Hennadii Stepanov)

Pull request description:

  This PR is intended to speed up the CI feedback for pull requests:

  - a [PR](https://github.com/bitcoin/bitcoin/actions/runs/6019964104?pr=28196) opened against the current master branch:
  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/481a70eb-13f3-40c9-8f6a-ca2f06350158)

  - this PR:
  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/2582307f-7b72-4816-b5be-e84d5e4a3016)

  Suggested in https://github.com/bitcoin/bitcoin/pull/28173#discussion_r1302929493:

  > An alternative would be to run them on non-pr pushes only. Failures should be rare enough to deal with them post-merge.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 62ab3e98ff if https://github.com/hebasto/bitcoin/actions/runs/6023862001/job/16341417883 is green

Tree-SHA512: e937efc5c03290f5d246ce1b0638dc72f39ef1d509ba5d2063f92bfe9157c602e6a952a9558dfc6413bbc5209fe55280b3278a0e4079773b8cc9ff236c8f9246
2023-08-30 16:15:46 +01:00
MarcoFalke
fa3b816240 doc: Fill in the required skills in the good_first_issue template 2023-08-30 14:02:08 +02:00
Hennadii Stepanov
62ab3e98ff ci, windows: Do not run extended functional tests for pull requests
This change is intended to speed up the CI feedback for pull requests.
2023-08-30 11:46:47 +01:00
fanquake
588068d5a1 guix: remove GCC 10 workaround from NSIS
Fixed upstream in 3.06, see
229b6136c4.
https://sourceforge.net/p/nsis/bugs/1248/
2023-08-30 11:10:53 +01:00
MarcoFalke
fa07ac48d8 ci: Asan with -ftrivial-auto-var-init=pattern 2023-08-30 10:52:27 +02:00
glozow
a3b55c94b9 [doc] move comment about AlreadyHaveTx DoS score to the right place
This comment isn't in the right place, as detection of a tx in
recent_rejects would cause the function to exit much earlier.
Move the comment to the right place and tweak the first sentence for
accuracy.
2023-08-29 16:41:22 +01:00
glozow
3b8c17838a [log] add more logs related to orphan handling
- Whenever a tx is erased. Allows somebody to see which transactions
  have been erased due to expiry/overflow, not just how many.
- Whenever a tx is added to a peer's workset.
- AcceptToMemoryPool when a tx is accepted, mirroring the one logged for
  a tx received from a peer. This allows someone to see all of the
  transactions that are accepted to mempool just by looking for ATMP logs.
- MEMPOOLREJ when a tx is rejected, mirroring the one logged for
  a tx received from a peer. This allows someone to see all of the
  transaction rejections by looking at MEMPOOLREJ logs.
2023-08-29 16:41:22 +01:00
glozow
51b3275cd1 [log] add category TXPACKAGES for orphanage and package relay 2023-08-29 16:41:22 +01:00
glozow
a33dde1e41 [log] include wtxid in tx {relay,validation,orphanage} logging 2023-08-29 16:41:22 +01:00
fanquake
ab42b2ebdb Merge bitcoin/bitcoin#28347: lint: replace deprecated pkg_resources with importlib.metadata
6c008a2006 script: replace deprecated pkg_resources with importlib.metadata (Jon Atack)

Pull request description:

  Running our python linter with a recent python and the latest release of setuptools [v68.1.2](https://setuptools.pypa.io/en/stable/history.html):

  ```
  $ python3 --version
  Python 3.11.5
  $ ./test/lint/lint-python.py:12: DeprecationWarning: pkg_resources is deprecated as an API.
    See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources
  ```

  Using `pkg_resources` was [deprecated](https://github.com/pypa/setuptools/pull/3843) earlier in [v67.5.0](https://setuptools.pypa.io/en/stable/history.html#id55): "Although pkg_resources has been discouraged for use, some projects still consider pkg_resources viable for usage. This change makes it clear that pkg_resources should not be used, emitting a DeprecationWarning when imported."

  The `importlib.metadata` library requires Python 3.8, which is currently our minimum-supported Python version.

  For more details about `importlib.metadata` and the two methods imported and used here, see:

  - https://docs.python.org/3/library/importlib.metadata.html
  - https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.metadata
  - https://importlib-metadata.readthedocs.io/en/latest/api.html#importlib_metadata.PackageNotFoundError

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 6c008a2006 (review only, did not test)

Tree-SHA512: f5258d37043fcc9744f85641a60a3395ad43822c72d030dea8c39fa7f48ec3d7790cdeeb832f96e8f38046adb7c62fbc577c975ef0c77c8047c0c8f2353ce540
2023-08-29 09:42:31 +01:00
fanquake
5175ae482e Merge bitcoin/bitcoin#28354: test: default acceptnonstdtxn=0 on all chains
13eb8aa572 doc: Release notes for testnet defaulting to -acceptnonstdtxn=0 (Anthony Towns)
e1dc15d690 config: default acceptnonstdtxn=0 on all chains (Anthony Towns)

Pull request description:

  Changes `-acceptnonstxtxn` to default to 0 on testnet, matching the other chains. Allowing non-standard txs on testnet by default contributed to the difficulties RSK described in #26348: "We see that there are two script paths and, to reduce the script size, a single CHECKMULTISIG is used for the two paths, separating the signer count from the CHECKMULTISIG opcode. This script worked on testnet, because it lacks the standard checks performed in Mainnet."

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 13eb8aa572
  sipa:
    utACK 13eb8aa572
  instagibbs:
    utACK 13eb8aa572
  theStack:
    Code-review ACK 13eb8aa572

Tree-SHA512: eff7a3f9fc9b94003a730beb96e6f3399bc8b8e93fde4b15f20a11eda61d9a3e076f4423989f98b794b32681abecbc3756a54cd0d37b136e2fb2ffbb47ee7774
2023-08-29 09:27:19 +01:00
MarcoFalke
faec591d64 Support for serialization parameters
The moved part can be reviewed with the git options
 --ignore-all-space --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space

(Modified by Marco Falke)

Co-authored-by: Pieter Wuille <pieter@wuille.net>
2023-08-28 17:48:26 +02:00
MarcoFalke
fac42e9d35 Rename CSerAction* to Action*
This allows new code, added in the next commit, to conform to the coding
guideline: No C-prefix for class names.
2023-08-28 17:48:15 +02:00
MarcoFalke
aaaa3fa947 Replace READWRITEAS macro with AsBase wrapping function
Co-authored-by: Pieter Wuille <pieter@wuille.net>
2023-08-28 17:48:15 +02:00
fanquake
1c1a02bbd2 Merge bitcoin/bitcoin#28341: refactor: Use HashWriter over legacy CHashWriter
99995cfe8d refactor: Use HashWriter over legacy CHashWriter (via SerializeHash) (MarcoFalke)
5555aa2d0d refactor: Use HashWriter over legacy CHashWriter (MarcoFalke)

Pull request description:

  `HashWriter` is a slim and less confusing version of `CHashWriter`, so use it in all places where it compiles.

  This should be correct, if it compiles.

ACKs for top commit:
  sipa:
    That said, code review ACK 99995cfe8d
  theuni:
    ACK 99995cfe8d
  TheCharlatan:
    ACK 99995cfe8d

Tree-SHA512: fc967a18379bd00bd334ac3d50beb5435b65ca66a48f72623f1dcdbbce3292fd91839160cd0e69b8f4f3d98e258dcbbc6f73f5e91345f938898ee39c903a442b
2023-08-28 16:20:50 +01:00
Vasil Dimov
7df4508369 test: improve sock_tests/move_assignment
Use also a socket for the moved-to object and check which one is closed when.
2023-08-28 14:32:17 +02:00
Anthony Towns
13eb8aa572 doc: Release notes for testnet defaulting to -acceptnonstdtxn=0 2023-08-28 22:09:39 +10:00
Anthony Towns
e1dc15d690 config: default acceptnonstdtxn=0 on all chains
Previously, the default for acceptnonstdtxn defaulted to 0 on all
chains except testnet. Change this to be consistent across all
chains, and remove the parameter from chainparams entirely.
2023-08-28 22:09:39 +10:00
fanquake
db57574aa8 Merge bitcoin/bitcoin#28173: ci: Run Windows native task on GitHub Actions
6a43372980 ci: Run "Win64 native" job on GitHub Actions (Hennadii Stepanov)

Pull request description:

  From https://github.com/bitcoin/bitcoin/issues/28098:
  > Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.

  > If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.

  Historical context:
  - https://github.com/bitcoin/bitcoin/pull/17697
  - https://github.com/bitcoin/bitcoin/issues/17803
  - https://github.com/bitcoin/bitcoin/pull/18031

  Security concerns:
  - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
  - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197

  `GITHUB_TOKEN` permissions (from the build log in my personal repo):
  ```
  2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
  2023-07-27T07:30:17.8314113Z Contents: read
  2023-07-27T07:30:17.8314608Z Metadata: read
  2023-07-27T07:30:17.8314957Z Packages: read
  2023-07-27T07:30:17.8315233Z ##[endgroup]
  ```

  Comparison of resources:

  | Resource | Current, Cirrus CI | Suggested, GitHub Actions |
  |---|:-:|:-:|
  | CPU | 6 | 2 |
  | RAM, GB | 12 | 7 |

  The `TEST_RUNNER_TIMEOUT_FACTOR` variable is set to the current default value for all CI tasks: 64440bb733/ci/test/00_setup_env.sh (L48)

Top commit has no ACKs.

Tree-SHA512: ddfdaf7a1e4793a64ac0cd20f116b29608dd06f15b062769ac70b3ea2fb82775aa96aa79c7b768efefec4338aaa5b57d267b592f62d0e8d5d94ecc11001a165d
2023-08-28 11:27:29 +01:00
MarcoFalke
fa70cbd969 ci: Remove unused TEST_RUNNER_ENV="LC_ALL=C" from s390x task 2023-08-27 22:50:42 +02:00
MarcoFalke
fa33354dcb ci: Remove /ro_base bind mount
Just set the bind mount to BASE_READ_ONLY_DIR, which allows to drop one
line of code and makes the code easier to understand.
2023-08-27 22:50:07 +02:00
MarcoFalke
fa0df9d4c4 doc: Remove sudo from command that is already run as root 2023-08-27 22:49:59 +02:00
MarcoFalke
faf7e69862 test: Support powerpc64le in get_previous_releases.py 2023-08-27 22:28:57 +02:00
Fabian Jahr
df60de770d log: Print error message when coindb is in inconsistent state 2023-08-27 20:16:43 +02:00
fanquake
8371914604 Merge bitcoin/bitcoin#28337: build: use Clang 15 for macOS cross-compilation
ff42d81383 guix: use clang-toolchain-15 for macOS compilation (fanquake)
94955b4b1d depends: use LLVM/Clang 15.0.6 for macOS cross-compile (fanquake)

Pull request description:

  This will end up being a blocker for #28210, and is already part of #21778, even though an even newer LLVM/Clang combination is required (and still missing from upstream Guix). Seems straight-forward enough to just bump the macOS compiler to a more modern Clang.

ACKs for top commit:
  TheCharlatan:
    re-ACK ff42d81383

Tree-SHA512: 8af4b54c3a56abb3825c6470444a28e14e9c69820c09ec4a33acebb8ae434df9ae18163c088a582130cc68755293a7e2bde5d065763919d94064ff9b3f83730f
2023-08-27 14:54:48 +01:00
fanquake
ff42d81383 guix: use clang-toolchain-15 for macOS compilation 2023-08-27 10:03:00 +01:00
fanquake
94955b4b1d depends: use LLVM/Clang 15.0.6 for macOS cross-compile
There is no x86_64 binaries for 15.0.7.
2023-08-27 10:02:55 +01:00
Jon Atack
6c008a2006 script: replace deprecated pkg_resources with importlib.metadata
in our python linter:

```
./test/lint/lint-python.py:12: DeprecationWarning: pkg_resources is deprecated as an API.
  See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
```

The importlib.metadata library was added in Python 3.8, which is currently our
minimum-supported Python version.

For more details about importlib.metadata, see https://docs.python.org/3/library/importlib.metadata.html
2023-08-26 16:42:02 -06:00
fanquake
03675b2ba3 Merge bitcoin/bitcoin#28328: guix: update time-machine
6cbedc3d9b guix: drop Windows broken-longjmp.patch (fanquake)
a1d4a42993 guix: drop NSIS patch now that we use 3.09 (fanquake)
1f6c75e82b guix: use cross-* keyword arguments (fanquake)
ea4387e1d0 guix: update time-machine to 160f78a4d92205df986ed9efcce7d3aac188cb24 (fanquake)
f1a4afb26f guix: update python-oscrypto to 1.3.0 (fanquake)
a8a7b75a01 guix: backport glibc patch to fix powerpc build (fanquake)

Pull request description:

  Bumps our Guix time-machine to [160f78a4d92205df986ed9efcce7d3aac188cb24](https://git.savannah.gnu.org/cgit/guix.git/commit/?id=160f78a4d92205df986ed9efcce7d3aac188cb24), which includes:

  GCC 10.3.0 -> 10.4.0
  Binutils 2.37 -> 2.38
  Automake 1.16.3 -> 1.16.5
  Coreutils 8.32 -> 9.1
  Findutils 4.8.2 -> 4.9.0
  Libtool 2.4.6 -> 2.4.7
  Linux Libre Headers 5.15.37 -> 5.15.127
  Git 2.36.0 -> 2.41.0
  Mingw-w64 8.0.0 -> 11.0.1
  NSIS 3.05 -> 3.09
  Xorriso 1.5.2 -> 1.5.6.pl02
  Python 3.9 -> 3.10.7
  Python-asn1crypto 1.4.0 -> 1.5.1

  GCC 12.3.0 becomes available. See #27897.
  LLVM 15.0.7 becomes available. Sadly not quite new enough for #21778.

  Split from #27897 for easier review, and to make it clear which changes are part of the time-machine bump vs changing to GCC 12.

  Guix Build:
  ```bash
  1e4aab93de36b47cbbfc5882d3d2a598e54a9c3604b3be1891ddac70cdc46547  guix-build-6cbedc3d9b67/output/aarch64-linux-gnu/SHA256SUMS.part
  278206bddf166ce507ab2ae4742ffbe6ec4256dbc6beee7e42ea040ec3d5b679  guix-build-6cbedc3d9b67/output/aarch64-linux-gnu/bitcoin-6cbedc3d9b67-aarch64-linux-gnu-debug.tar.gz
  261974753869297a480db24733f057aa58e33a0ce08754bea74984a00f8c376f  guix-build-6cbedc3d9b67/output/aarch64-linux-gnu/bitcoin-6cbedc3d9b67-aarch64-linux-gnu.tar.gz
  13bde7d903665f7699e2aa42a23ddc5ece9fab450437ae641d8ebfe6f6b084e1  guix-build-6cbedc3d9b67/output/arm-linux-gnueabihf/SHA256SUMS.part
  df951e4b23db34d1248f7ee9837594645acea8e2de1ed03b23ee002579a4c653  guix-build-6cbedc3d9b67/output/arm-linux-gnueabihf/bitcoin-6cbedc3d9b67-arm-linux-gnueabihf-debug.tar.gz
  5839bb71cca6aafa662f3c7e1c09393dce1c3fd13241726796bf3e681a0a34f0  guix-build-6cbedc3d9b67/output/arm-linux-gnueabihf/bitcoin-6cbedc3d9b67-arm-linux-gnueabihf.tar.gz
  637b1aa49db0cab1bc35064d068f1a5fdee5ab2ceba30b7f6d2c80bbf174c107  guix-build-6cbedc3d9b67/output/arm64-apple-darwin/SHA256SUMS.part
  be87c1ad41f06bcaa8c940fc8d881eca41c531d250ce114325b0e042e603c257  guix-build-6cbedc3d9b67/output/arm64-apple-darwin/bitcoin-6cbedc3d9b67-arm64-apple-darwin-unsigned.dmg
  632956e64d2baf9329ee086781b9268786f37b7c3c5f5232e2394ef2839d6d40  guix-build-6cbedc3d9b67/output/arm64-apple-darwin/bitcoin-6cbedc3d9b67-arm64-apple-darwin-unsigned.tar.gz
  1b23548a94aee5e03a8aec1e63938d9c8505005b87fa832ed112af60861c42a6  guix-build-6cbedc3d9b67/output/arm64-apple-darwin/bitcoin-6cbedc3d9b67-arm64-apple-darwin.tar.gz
  e4056df68bac0bdc0a54899af8f8178e78bfb7d170750f2296fd512d0c6c20db  guix-build-6cbedc3d9b67/output/dist-archive/bitcoin-6cbedc3d9b67.tar.gz
  9f149c82f30eed35e2c217dc57cab37745db27f0a3205e66b4535c247dffc500  guix-build-6cbedc3d9b67/output/powerpc64-linux-gnu/SHA256SUMS.part
  ef91dce0a735ce0633fd273d9e1d6599d74c18280a866c1d6b71c2c3434af9c3  guix-build-6cbedc3d9b67/output/powerpc64-linux-gnu/bitcoin-6cbedc3d9b67-powerpc64-linux-gnu-debug.tar.gz
  dfb60f13b9a7435f608f6d09585de37f789b055c77a0ee729e5723b0ef0550fe  guix-build-6cbedc3d9b67/output/powerpc64-linux-gnu/bitcoin-6cbedc3d9b67-powerpc64-linux-gnu.tar.gz
  0ae916b95617042330166e20a27399c14abbc20638ae3e5e5f9f812146d4a07e  guix-build-6cbedc3d9b67/output/powerpc64le-linux-gnu/SHA256SUMS.part
  180d9c29da1efd21fba6be8a670b6ed01d71a5e90991da3c543f6720273831a2  guix-build-6cbedc3d9b67/output/powerpc64le-linux-gnu/bitcoin-6cbedc3d9b67-powerpc64le-linux-gnu-debug.tar.gz
  9753408e3b4c10b19dba30ecb12ee2ae065c08e7c88be25599e1fdffe81bddbe  guix-build-6cbedc3d9b67/output/powerpc64le-linux-gnu/bitcoin-6cbedc3d9b67-powerpc64le-linux-gnu.tar.gz
  b1f09baec636a3bbaff1be629c95e75767c4ae3d96f0ebe63dcefb6fdb4953fb  guix-build-6cbedc3d9b67/output/riscv64-linux-gnu/SHA256SUMS.part
  09149283880c39c76f73ce83848f266f93777f6d9d389c2448ef0a9af9339b30  guix-build-6cbedc3d9b67/output/riscv64-linux-gnu/bitcoin-6cbedc3d9b67-riscv64-linux-gnu-debug.tar.gz
  788ce14925fec9861c78e16a57a1fd25d0f2ddf3f4e10f3c795bac534d339197  guix-build-6cbedc3d9b67/output/riscv64-linux-gnu/bitcoin-6cbedc3d9b67-riscv64-linux-gnu.tar.gz
  f357a0522c779d1953f37f4cab0160622cbf89ed3fe49cb5abea2509dba1d76f  guix-build-6cbedc3d9b67/output/x86_64-apple-darwin/SHA256SUMS.part
  be5c797a3e4dc969fd9116dede0cb27bda5180e653bb63d1ef3dbcc3de983bf0  guix-build-6cbedc3d9b67/output/x86_64-apple-darwin/bitcoin-6cbedc3d9b67-x86_64-apple-darwin-unsigned.dmg
  7364b2d0223b39debd386b521a5c5d1b26a446331f347b4d4e8fc58bf5b4439e  guix-build-6cbedc3d9b67/output/x86_64-apple-darwin/bitcoin-6cbedc3d9b67-x86_64-apple-darwin-unsigned.tar.gz
  7e873165d4ed785381b6e461a81a947814d14fdd41ae65bc82134c03b3ac0471  guix-build-6cbedc3d9b67/output/x86_64-apple-darwin/bitcoin-6cbedc3d9b67-x86_64-apple-darwin.tar.gz
  bf03ca15a885330ed0c91b3137dcc4ec1102570898360cc3f1073f03b792d729  guix-build-6cbedc3d9b67/output/x86_64-linux-gnu/SHA256SUMS.part
  0707b0e4d8529fc5f7567d1ff029b4b6fdf0bb46f1eb227d70c3aaa2018d65e5  guix-build-6cbedc3d9b67/output/x86_64-linux-gnu/bitcoin-6cbedc3d9b67-x86_64-linux-gnu-debug.tar.gz
  5f1e3713cf856f884d305ad12f43254be3871d9020cc514194fc0a6efd0867b1  guix-build-6cbedc3d9b67/output/x86_64-linux-gnu/bitcoin-6cbedc3d9b67-x86_64-linux-gnu.tar.gz
  c0d5e284903dbb55e4ad3d43db9b1e5d211c5259c3fae43bf4cddedea2e02c01  guix-build-6cbedc3d9b67/output/x86_64-w64-mingw32/SHA256SUMS.part
  d16f61c78b784b664d558a9a537e15da432ef9c18de0754dd7fd19682ab0569f  guix-build-6cbedc3d9b67/output/x86_64-w64-mingw32/bitcoin-6cbedc3d9b67-win64-debug.zip
  617010c1ac295bc692e0a4074acc19ebbc9561a01c8e5365428cfe61dc50fe3c  guix-build-6cbedc3d9b67/output/x86_64-w64-mingw32/bitcoin-6cbedc3d9b67-win64-setup-unsigned.exe
  a48961c49edf9f8988b902d874f47918f6716053be6c3bc263932834cb896510  guix-build-6cbedc3d9b67/output/x86_64-w64-mingw32/bitcoin-6cbedc3d9b67-win64-unsigned.tar.gz
  0cd95e8f919ac694dfd88a14fda37c57f0534cb890acc73f70aff71937fa1a8e  guix-build-6cbedc3d9b67/output/x86_64-w64-mingw32/bitcoin-6cbedc3d9b67-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 6cbedc3d9b

Tree-SHA512: 48950535febab3865a8899b0ce590a7da6dbd0a89d2d2987a467da457e9ff0687d5afc65f952386fa562aa2a3fc6d8c85b1c221c73998d522076486a9fd845c4
2023-08-26 08:49:34 +01:00
MarcoFalke
99995cfe8d refactor: Use HashWriter over legacy CHashWriter (via SerializeHash) 2023-08-25 17:09:32 +02:00
MarcoFalke
5555aa2d0d refactor: Use HashWriter over legacy CHashWriter 2023-08-25 17:09:13 +02:00
Hennadii Stepanov
6a43372980 ci: Run "Win64 native" job on GitHub Actions 2023-08-25 13:50:04 +01:00
Vasil Dimov
5086a99b84 net: remove Sock default constructor, it's not necessary 2023-08-25 14:42:07 +02:00
Vasil Dimov
7829272f78 net: remove now unnecessary Sock::Get()
`Sock::Get()` was used only in `sock.{cpp,h}`. Remove it and access
`Sock::m_socket` directly.

Unit tests that used `Get()` to test for equality still verify that the
behavior is correct by using the added `operator==()`.
2023-08-25 14:41:31 +02:00
Antoine Poinsot
10546a569c wallet: accurately account for the size of the witness stack
When estimating the maximum size of an input, we were assuming the
number of elements on the witness stack could be encode in a single
byte. This is a valid approximation for all the descriptors we support
(including P2WSH Miniscript ones), but may not hold anymore once we
support Miniscript within Taproot descriptors (since the max standard
witness stack size of 100 gets lifted).

It's a low-hanging fruit to account for it correctly, so just do it now.
2023-08-25 12:40:12 +02:00
Antoine Poinsot
9b7ec393b8 wallet: use descriptor satisfaction size to estimate inputs size
Instead of using the dummysigner to compute a placeholder satisfaction,
infer a descriptor on the scriptPubKey of the coin being spent and use
the estimation of the satisfaction size given by the descriptor
directly.

Note this (almost, see next paragraph) exactly conserves the previous
behaviour. For instance CalculateMaximumSignedInputSize was previously
assuming the input to be spent in a transaction that spends at least one
Segwit coin, since it was always accounting for the serialization of the
number of witness elements.

In this commit we use a placeholder for the size of the serialization of
the witness stack size (1 byte). Since the logic in this commit is
already tricky enough to review, and that it is only a very tiny
approximation not observable through the existing tests, it is addressed
in the next commit.
2023-08-25 12:40:12 +02:00
Antoine Poinsot
8d870a9873 script/signingprovider: introduce a MultiSigningProvider
It is sometimes useful to interface with multiple signing providers at
once. For instance when inferring a descriptor with solving information
being provided from multiple sources (see next commit).

Instead of inneficiently copying the information from one provider into
the other, introduce a new signing provider that takes a list of
pointers to existing providers.
2023-08-25 12:40:11 +02:00
Antoine Poinsot
fa7c46b503 descriptor: introduce a method to get the satisfaction size
In the wallet code, we are currently estimating the size of a signed
input by doing a dry run of the signing logic. This is unnecessary as
all outputs we are able to sign for can be represented by a descriptor,
and we can derive the size of a satisfaction ("signature") from the
descriptor itself directly.
In addition, this approach does not scale: getting the size of a
satisfaction through a dry run of the signing logic is only possible for
the most basic scripts.

This commit introduces the computation of the size of satisfaction per
descriptor. It's a bit intricate for 2 main reasons:
- We want to conserve the behaviour of the current dry-run logic used by
  the wallet that sometimes assumes ECDSA signatures will be low-r,
  sometimes not (when we don't create them).
- We need to account for the witness discount. A single descriptor may
  sometimes benefit of it, sometimes not (for instance `pk()` if used as
  top-level versus if used inside `wsh()`).
2023-08-25 12:40:11 +02:00
Antoine Poinsot
bdba7667d2 miniscript: introduce a helper to get the maximum witness size
Similarly to how we compute the maximum stack size.

Also note how it would be quite expensive to recompute it recursively
by accounting for different ECDSA signature sizes. So we just assume
high-R everywhere. It's only a trivial difference anyways.
2023-08-25 11:56:10 +02:00
Antoine Poinsot
4ab382c2cd miniscript: make GetStackSize independent of P2WSH context
It was taking into account the P2WSH script push in the number of stack
elements.
2023-08-25 11:56:09 +02:00
Andrew Chow
c9273f68f6 Merge bitcoin/bitcoin#28287: rpc, test: add sendmsgtopeer rpc and a test for net-level deadlock situation
b3a93b409e test: add functional test for deadlock situation (Martin Zumsande)
3557aa4d0a test: add basic tests for sendmsgtopeer to rpc_net.py (Martin Zumsande)
a9a1d69391 rpc: add test-only sendmsgtopeer rpc (Martin Zumsande)

Pull request description:

  This adds a `sendmsgtopeer` rpc (for testing only) that allows a node to send a message (provided in hex) to a peer.
  While we would usually use a `p2p` object instead of a node for this in the test framework, that isn't possible in situations where this message needs to trigger an actual interaction of multiple nodes.

  Use this rpc to add test coverage for the bug fixed in #27981 (that just got merged):
  The test lets two nodes (almost) simultaneously send a single large (4MB) p2p message to each other, which would have caused a deadlock previously (making this test fail), but succeeds now.

  As can be seen from the discussion in #27981, it was not easy to reproduce this bug without `sendmsgtopeer`. I would imagine that `sendmsgtopeer` could also be helpful in various other test constellations.

ACKs for top commit:
  ajtowns:
    ACK b3a93b409e
  sipa:
    ACK b3a93b409e
  achow101:
    ACK b3a93b409e

Tree-SHA512: 6e22e72402f3c4dd70cddb9e96ea988444720f7a164031df159fbdd48056c8ac77ac53def045d9208a3ca07437c7c8e34f8b4ebc7066c0a84d81cd53f2f4fa5f
2023-08-24 17:34:57 -04:00
Andrew Chow
5ce200dda2 Merge bitcoin/bitcoin#27480: doc: Improve documentation of rpcallowip
c8e066461b doc: Improve documentation of rpcallowip rpchelp (willcl-ark)

Pull request description:

  Closes #21070

  v21.0 introduced a behaviour changed noted in #21070 where using a config value `rpcallowip=::0` no longer also permitted ipv4 ip addresses.

  The rpc_bind.py functional test covers this new behaviour already by checking that the list of bind addresses exactly matches what is expected so this commit only updates the documentation.

ACKs for top commit:
  achow101:
    ACK c8e066461b
  pinheadmz:
    ACK c8e066461b
  jonatack:
    ACK c8e066461b

Tree-SHA512: 332060cf0df0427c6637a9fd1e0783ce0b0940abdb41b0df13f03bfbdc28af067cec8f0b1bbc4e47b3d54fa1b2f110418442b05b39d5e7c7e0b96744ddd7c003
2023-08-24 16:31:38 -04:00
Andrew Chow
74d66359da Merge bitcoin/bitcoin#27585: fuzz: improve coinselection
bf26f978ff fuzz: coinselection, fix `m_cost_of_change` (brunoerg)
6d9b26d56a fuzz: coinselection, BnB should never produce change (brunoerg)
b2eb558407 fuzz: coinselection, compare `GetSelectedValue` with target (brunoerg)
0df0438c60 fuzz: coinselection, improve `ComputeAndSetWaste` (brunoerg)
1e351e5db1 fuzz: coinselection, add coverage for `Merge` (brunoerg)
f0244a8614 fuzz: coinselection, add coverage for `GetShuffledInputVector`/`GetInputSet` (brunoerg)
808618b8a2 fuzz: coinselection, add coverage for `AddInputs` (brunoerg)
90c4e6a241 fuzz: coinselection, add coverage for `EligibleForSpending` (brunoerg)
2a031cb2c2 fuzz: coinselection, add `CreateCoins` (brunoerg)

Pull request description:

  This PR:

  - Moves coin creation to its own function called `CreateCoins`.
  - Add coverage for `EligibleForSpending`
  - Add coverage for `AddInputs`: get result of each algorithm (srd, knapsack and bnb), call `CreateCoins` and add into them.
  - Add coverage for `GetShuffledInputVector` and `GetInputSet` using the result of each algorithm (srd, knapsack and bnb).
  - Add coverage for `Merge`: Call SRD with the new utxos and, if successful, try to merge with the previous SRD result.

ACKs for top commit:
  murchandamus:
    reACK with some minimal fuzzing bf26f978ff
  achow101:
    ACK bf26f978ff
  furszy:
    re-ACK bf26f97

Tree-SHA512: bdd2b0a39de37be0a9b21a7c51260b6b8abe538cc0ea74312eb658b90a121a1ae07306c09fb0e75e93b531ce9ea2402feb041b0d852902d07739257f792e64ab
2023-08-24 16:11:20 -04:00
fanquake
1fa6411dde Merge bitcoin/bitcoin#28165: net: transport abstraction
8a3b6f3387 refactor: make Transport::ReceivedBytes just return success/fail (Pieter Wuille)
bb4aab90fd net: move message conversion to wire bytes from PushMessage to SocketSendData (Pieter Wuille)
a1a1060fd6 net: measure send buffer fullness based on memory usage (Pieter Wuille)
009ff8d650 fuzz: add bidirectional fragmented transport test (Pieter Wuille)
fb2c5edb79 net: make V1Transport implicitly use current chainparams (Pieter Wuille)
0de48fe858 net: abstract sending side of transport serialization further (Pieter Wuille)
649a83c7f7 refactor: rename Transport class receive functions (Pieter Wuille)
27f9ba23ef net: add V1Transport lock protecting receive state (Pieter Wuille)
93594e42c3 refactor: merge transport serializer and deserializer into Transport class (Pieter Wuille)

Pull request description:

  This PR furthers the P2P message serialization/deserialization abstraction introduced in #16202 and #16562, in preparation for introducing the BIP324 v2 transport (making this part of #27634). However, nothing in this PR is BIP324-specific, and it contains a number of independently useful improvements.

  The overall idea is to have a single object in every `CNode` (called `m_transport`) that is responsible for converting sent messages to wire bytes, and for converting received wire bytes back to messages, while having as little as possible knowledge about this conversion process in higher-level net code. To accomplish that, there is an abstract `Transport` class with (currently) a single `V1Transport` implementation.

  Structurally, the above is accomplished by:
  * Merging the `TransportDeserializer` and `TransportSerializer` classes into a single `Transport` class, which encompasses both the sending and receiving side. For `V1Transport` these two sides are entirely separate, but this assumption doesn't hold for the BIP324 transport where e.g. the sending encryption key depends on the DH key negotiation data received from the other side. Merging the two means a future `V2Transport` can handle all this interaction without callers needing to be aware.
  * Removing the assumption that each message is sent using a computed header followed by (unmodified) data bytes. To achieve that, the sending side of `Transport` mirrors what the receiver side does: callers can set a message to be sent, then ask what bytes must be sent out, and then allowing them to transition to the next message.
  * Adding internal locks to protect the sending and receiving state of the `V1Transport` implementation. I believe these aren't strictly needed (opinions welcome) as there is no real way to use `Transport` objects in a multi-threaded fashion without some form of external synchronization (e.g. "get next bytes to send" isn't meaningful to call from multiple threads at the same time without mechanism to control the order they'll actually get sent). Still, I feel it's cleaner to make the object responsible for its own consistency (as we definitely do not want the entire object to be under a single external GUARDED_BY, as that'd prevent simultaneous sending and receiving).
  * Moving the conversion of messages to bytes on the sending side from `PushMessage` to `SocketSendData`, which is needed to deal with the fact that a transport may not immediately be able to send messages.

  This PR is not a refactor, though some commits are. Among the semantic changes are:
  * Changing the send buffer pushback mechanism to trigger based on the memory usage of the buffer rather than the amount of bytes to be sent. This is both closer to the desired behavior, and makes the buffering independent from transport details (which is why it's included here).
  * When optimistic send is not applicable, the V1 message checksum calculation now runs in the net thread rather than the message handling thread. I believe that's generally an improvement, as the message handling thread is far more computationally bottlenecked already.
  * The checksum calculation now runs under the `CNode::cs_vSend` lock, which does mean no two checksum calculations for messages sent to the same node can run in parallel, even if running in separate threads. Despite that limitation, having the checksum for non-optimistic sends moved in the net thread is still an improvement, I believe.
  * Statistics for per-message-type sent bytes are now updated when the bytes are actually handed to the OS rather than in `PushMessage`. This is because the actual serialized sizes aren't known until they've gone through the transport object.

  A fuzz test of the entire `V1Transport` is included. More elaborate rationale for each of the changes can be found in the commit messages.

ACKs for top commit:
  theStack:
    re-ACK 8a3b6f3387
  vasild:
    ACK 8a3b6f3387
  dergoegge:
    Code review ACK 8a3b6f3387

Tree-SHA512: 26e9a6df47f1dd3e3f3edb4874edf365728e5a8bbc9d0d4d71fb6000cb2dfde5574902c47ffcf825af6743922f2ff9d31a5a38942a196f4ca6669122e15e42e4
2023-08-24 13:55:41 +01:00
Vasil Dimov
944b21b70a net: don't check if the socket is valid in ConnectSocketDirectly()
The socket is always valid (the underlying file descriptor is not
`INVALID_SOCKET`) when `ConnectSocketDirectly()` is called.
2023-08-24 14:40:03 +02:00
Vasil Dimov
aeac68d036 net: don't check if the socket is valid in GetBindAddress()
The socket is always valid (the underlying file descriptor is not
`INVALID_SOCKET`) when `GetBindAddress()` is called.
2023-08-24 14:40:02 +02:00
Vasil Dimov
5ac1a51ee5 i2p: avoid using Sock::Get() for checking for a valid socket
Peeking at the underlying socket file descriptor of `Sock` and checkig
if it is `INVALID_SOCKET` is bad encapsulation and stands in the way of
testing/mocking/fuzzing.

Instead use an empty unique_ptr to denote that there is no valid socket.
2023-08-24 14:39:58 +02:00
fanquake
083316c4fe Merge bitcoin/bitcoin#28230: rpc: Add MaybeArg() and Arg() default helper
c00000df16 rpc: Add MaybeArg() and Arg() default helper (MarcoFalke)

Pull request description:

  Currently the RPC method implementations have many issues:

  * Default RPC argument values (and their optionality state) are duplicated in the documentation and the C++ code, with no checks to prevent them from going out of sync.
  * Getting an optional RPC argument is verbose, using a ternary operator, or worse, a multi-line `if`.

  Fix all issues by adding default helper that can be called via `self.Arg<int>(0)`. The helper needs a few lines of code in the `src/rpc/util.h` header file. Everything else will be implemented in the cpp file once and if an RPC method needs it.

  There is also an `self.MaybeArg<int>(0)` helper that works on any arg to return the argument, the default, or a falsy value.

ACKs for top commit:
  ajtowns:
    reACK c00000df16
  stickies-v:
    re-ACK c00000df16
  TheCharlatan:
    re-ACK c00000df16

Tree-SHA512: e7ddcab3faa319bc53edbdf3f89ce83389d2c4e571d5db42401620ff105e522a4a0669dad08e08cde5fd05c790aec3b806f63261a9100c2778865a489e57381e
2023-08-24 11:57:12 +01:00
fanquake
27101d0f0c Merge bitcoin/bitcoin#28330: ci: Add missing docker.io prefix for native tasks to CI_IMAGE_NAME_TAG
fab7f5c01d ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAG (MarcoFalke)

Pull request description:

  Currently, the CI system may pick the wrong (non-native) architecture due to the missing prefix.

  For example, assuming the CI_IMAGE_NAME_TAG is `debian:bookworm` and the user has previously pulled an s390x image:

  ```
  $ podman run --rm 'docker.io/s390x/debian:bookworm' dpkg --print-architecture
  exec /usr/bin/dpkg: exec format error
  ```

  Now, `debian:bookworm` will refer to the same image:

  ```
  $ podman run --rm 'debian:bookworm' dpkg --print-architecture
  exec /usr/bin/dpkg: exec format error
  ```

  However, `docker.io/debian:bookworm` works fine:

  ```
   $ podman run --rm 'docker.io/debian:bookworm' dpkg --print-architecture
  arm64

ACKs for top commit:
  hebasto:
    ACK fab7f5c01d.

Tree-SHA512: c423c5cd454a95fa3e67081411ca08d316b8c680a5bba49196c514b91df65d9cc46a47700cc00d9579327842615f98146d0ac50abb016616a9b17d042598dab6
2023-08-24 10:56:03 +01:00
fanquake
7ee7c6d2c3 Merge bitcoin/bitcoin#28332: test: previous releases: speed up fetching sources with shallow clone
360ac64b90 test: previous releases: speed up fetching sources with shallow clone (Sebastian Falbesoner)

Pull request description:

  For the sake of building previous releases, fetching the whole history of the repository for each version seems to be overkill as it takes much more time, bandwidth and disk space than necessary. Create a shallow clone instead with history truncated to the one commit of the version tag, which is directly checked out in the same command. This has the nice side-effect that we can remove the extra `git checkout` step after as it's not needed anymore.

  Note that it might look confusing to pass a _tag_ to a parameter named `--branch`, but the git-clone manpage explicitly states that this is supported.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 360ac64b90

Tree-SHA512: c885a695c1ea90895cf7a785540c24e8ef8d1d9ea78db28143837240586beb6dfb985b8b0b542d2f64e2f0ffdca7c65fc3d55f44b5e1b22cc5535bc044566f86
2023-08-24 10:37:41 +01:00
fanquake
e8989f295d Merge bitcoin/bitcoin#28325: test: wallet_backup.py, fix intermittent failure in "restore using dumped wallet"
c4929cfa50 test: wallet_backup.py, fix intermittent failure in "restore using dumped wallet" (furszy)

Pull request description:

  Aiming to fix #25652.

  The failure arises because the test expects `init_wallet()` (the test framework function) to create a wallet with no keys. However, the function also imports the deterministic private key used to receive the coinbase coins.

  This causes a race within the "restore using dumped wallet" case, where we intend to have a new wallet (with no existing keys) to test the 'importwallet()' RPC result.
  The reason why this failure is intermittent is that it depends on other peers delivering the chain right after node2 startup and prior to the test 'node2.getbalance()' call and also the synchronization of the validation queue.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK c4929cfa50

Tree-SHA512: 80faa590439305576086a7d6e328f2550c97b218771fc5eba0567feff78732a2605d028a30a368d50944ae3d25fdbd6d321fb97321791a356416f2b790999613
2023-08-24 10:30:58 +01:00
fanquake
94af164e5e Merge bitcoin/bitcoin#28322: ci: Use concurrency for pull requests only
e7d67efd13 ci: Use concurrency for pull requests only (Hennadii Stepanov)

Pull request description:

  This PR is an amendment for https://github.com/bitcoin/bitcoin/pull/28282.

  It avoids skipping builds when some pushes were done consequentially:

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/977e9ead-1856-4020-82eb-d16dbead5753)

  From GitHub Actions [docs](https://docs.github.com/en/actions/using-jobs/using-concurrency):

  > When a concurrent ... workflow is queued, if another ... workflow using the same concurrency group in the repository is in progress, the queued ... workflow will be pending. **Any previously pending ... workflow in the concurrency group will be canceled.**

  No behavior change for pull requests:

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/4865ae04-fc42-4028-b91e-500c0b36bce6)

ACKs for top commit:
  MarcoFalke:
    review only ACK e7d67efd13
  dergoegge:
    lgtm ACK e7d67efd13

Tree-SHA512: 360be9dbb46354d988935643e8793ea630c3416609c45aab4c39075a00a427487446a34a4fb93b490ece43cabe43640fcc6fb2ac97760e2c6ee54b95241da826
2023-08-24 10:17:14 +01:00
fanquake
e0ad847073 Merge bitcoin/bitcoin#28321: test: Fix intermittent issue in mempool_reorg
fa5cc3ccfb test: Fix intermittent issue in mempool_reorg (MarcoFalke)

Pull request description:

  Currently the test case may fail intermittently, see https://github.com/bitcoin/bitcoin/issues/28313

  Fix this by changing a number and reducing the failure rate a bit.

ACKs for top commit:
  glozow:
    ACK fa5cc3ccfb

Tree-SHA512: ff552111b434ca712c7dbdc5ba32a986a6fa4512cba5a756234eae69428063bf6ecfdc8f350ee84226ed4d3e4262b4639dbe49162a722e8da85f0d61e5690c51
2023-08-24 10:16:13 +01:00
MarcoFalke
c00000df16 rpc: Add MaybeArg() and Arg() default helper 2023-08-24 10:44:45 +02:00
fanquake
cd5d2f5f09 Merge bitcoin/bitcoin#27976: ci: Start with clean env
fa15f7e082 ci: Remove no longer applicable section (MarcoFalke)
fa378bed56 ci: Start with clean env (MarcoFalke)
fa8c250c2f ci: Limit scope of some env vars (MarcoFalke)

Pull request description:

  Starting with a clean `env` should help to avoid non-determinism, such as the one fixed in https://github.com/bitcoin/bitcoin/pull/27739#issuecomment-1564529747

ACKs for top commit:
  dergoegge:
    utACK fa15f7e082

Tree-SHA512: 716b264217557b6524dab92d5a2a8d61ecb982dff475bd0cf5a763070b4c5916cd5995e764eb5d67d9cf2428c29d5fc2f42b32941b54c7c3053123ce448171e5
2023-08-24 09:28:49 +01:00
fanquake
6cbedc3d9b guix: drop Windows broken-longjmp.patch
This is no-longer required, now that we are building using GCC 10.4.0.
2023-08-24 09:21:50 +01:00
fanquake
a1d4a42993 guix: drop NSIS patch now that we use 3.09
See https://sourceforge.net/p/nsis/bugs/1283/.
2023-08-24 09:21:49 +01:00
fanquake
1f6c75e82b guix: use cross-* keyword arguments
Using the new time-machine results in warnings about consistently using
keyword arguments:
```bash
guix environment: warning: 'cross-kernel-headers' must be used with keyword arguments
guix environment: warning: 'cross-libc' must be used with keyword arguments
```
2023-08-24 09:21:49 +01:00
fanquake
ea4387e1d0 guix: update time-machine to 160f78a4d92205df986ed9efcce7d3aac188cb24
In our time-machine environment this changes the following:

GCC 10.3.0 -> 10.4.0
Binutils 2.37 -> 2.38
Linux Libre Headers 5.15.37 -> 5.15.127
git 2.36.0 -> 2.41.0
mingw-w64 8.0.0 -> 11.0.1
NSIS 3.05 -> 3.09
xorriso 1.5.2 -> 1.5.6.pl02
Python 3.9 -> 3.10.7
Python-asn1crypto 1.4.0 -> 1.5.1

GCC 12.3.0 becomes available.
LLVM 15.0.7 becomes available.
2023-08-24 09:21:49 +01:00
fanquake
f1a4afb26f guix: update python-oscrypto to 1.3.0
This is required for bumping the time-machine, for compatibility with
OpenSSL:

oscrypto: openssl backend, 1.2.1, /tmp/guix-build-python-oscrypto-1.2.1.drv-0/source/oscrypto
Traceback (most recent call last):
  File "/tmp/guix-build-python-oscrypto-1.2.1.drv-0/source/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
    libcrypto.EVP_PKEY_size.argtypes = [
  File "/gnu/store/9dkl9fnidcdpw19ncw5pk0p7dljx7ijb-python-3.10.7/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
    func = self.__getitem__(name)
  File "/gnu/store/9dkl9fnidcdpw19ncw5pk0p7dljx7ijb-python-3.10.7/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /gnu/store/2hr7w64zhr6jjznidyc2xi40d5ynhj9c-openssl-3.0.8/lib/libcrypto.so.3: undefined symbol: EVP_PKEY_size. Did you mean: 'EVP_PKEY_free'?
2023-08-24 09:20:31 +01:00
fanquake
a8a7b75a01 guix: backport glibc patch to fix powerpc build
Do this prior to bumping the time-machine, to avoid the following build
failure:
```bash
 /tmp/guix-build-glibc-cross-powerpc64le-linux-gnu-2.27.drv-0/build/string/memset-power8.o.dt -MT /tmp/guix-build-glibc-cross-powerpc64le-linux-gnu-2.27.drv-0/build/string/memset-power8.o
../sysdeps/powerpc/powerpc64/power4/memcmp.S: Assembler messages:
../sysdeps/powerpc/powerpc64/power4/memcmp.S:87: Error: unrecognized opcode: `ldbrx'
../sysdeps/powerpc/powerpc64/power4/memcmp.S:88: Error: unrecognized opcode: `ldbrx'
../sysdeps/powerpc/powerpc64/power4/memcmp.S:112: Error: unrecognized opcode: `ldbrx'
```

See:
https://sourceware.org/git/?p=glibc.git;a=commit;h=9250e6610fdb0f3a6f238d2813e319a41fb7a810.
e154242724.
2023-08-24 09:20:31 +01:00
fanquake
f3cc29fc5f Merge bitcoin/bitcoin#28324: guix: pre time-machine bump changes (Linux)
806b75b213 guix: consolidate Linux GCC package (fanquake)
4415275f96 guix: consolidate glibc 2.27 package (fanquake)

Pull request description:

  This is some refactoring to the Linux Guix build that facilitates bumping our Guix time-machine. Namely, avoiding `package-with-extra-configure-variable`, which is non-functional in the newer time-machine, see https://issues.guix.gnu.org/64436.

  At the same time, consolidate our Linux GCC build into `linux-base-gcc`. Now that we only use `building-on`, remove `explicit-cross-configure`.

  Split out of https://github.com/bitcoin/bitcoin/pull/27897. Most of the [[WIP] Linux commit](8335fc4775), minus anything GCC 12 related.

  I'll also be splitting out the other changes we can do pre-timemachine bump, for easier review.

  Similar/followup to #28294. Requirement for #28328.

  Guix Build:
  ```bash
  17463110d4b4721a7c188e71b1fc00c9b5b82227aa8342471390c17678e04a9a  guix-build-806b75b21340/output/aarch64-linux-gnu/SHA256SUMS.part
  0ca919ce568e7d4ffe44dda871d48963ca2988516068e75b1f30ca342d853d20  guix-build-806b75b21340/output/aarch64-linux-gnu/bitcoin-806b75b21340-aarch64-linux-gnu-debug.tar.gz
  134afa263e4da6c8d7df79a7dd8e59911c1e643b53b7c285de9418d97fb06d5f  guix-build-806b75b21340/output/aarch64-linux-gnu/bitcoin-806b75b21340-aarch64-linux-gnu.tar.gz
  22ce318241084829e57f064bf47de57752151863aac545e643eea5dd8eee96fe  guix-build-806b75b21340/output/arm-linux-gnueabihf/SHA256SUMS.part
  a26fecfeb821040704ca70ea056bff796789ed9500d9575d8fa13a85b32143f6  guix-build-806b75b21340/output/arm-linux-gnueabihf/bitcoin-806b75b21340-arm-linux-gnueabihf-debug.tar.gz
  213c84494835c81a40ebc5d38a62bb19cbee5b214b2a8aaed6d28746b245108e  guix-build-806b75b21340/output/arm-linux-gnueabihf/bitcoin-806b75b21340-arm-linux-gnueabihf.tar.gz
  ce1efcf6d3ca0e7422c5ce35f5e45e0770a3ae48173e061137daa7dc551e9d48  guix-build-806b75b21340/output/arm64-apple-darwin/SHA256SUMS.part
  fc01aaeb4e4722d21fd60c78f1b5322c9875ec6fb4d244f4547a354e91a33ed7  guix-build-806b75b21340/output/arm64-apple-darwin/bitcoin-806b75b21340-arm64-apple-darwin-unsigned.dmg
  632e4a243d3e4144313f53047499f91b7c9380a1a50f5846e1635d0a00fd202a  guix-build-806b75b21340/output/arm64-apple-darwin/bitcoin-806b75b21340-arm64-apple-darwin-unsigned.tar.gz
  8e694e4cd1bf45e6a586a0d8c19b675014f168f342f881a9ae0c4fbbda796914  guix-build-806b75b21340/output/arm64-apple-darwin/bitcoin-806b75b21340-arm64-apple-darwin.tar.gz
  dad6e8475f13dac6c4f0b182f53dc330997e9e1e5cf4d46413655f319dcd9bff  guix-build-806b75b21340/output/dist-archive/bitcoin-806b75b21340.tar.gz
  32e8b6c7e7a7561e132c5f15e2151a51aad1c5004ab90a36a7e80f92c200ef6f  guix-build-806b75b21340/output/powerpc64-linux-gnu/SHA256SUMS.part
  9033e85e03bd12a3a19599735cfd44fcfdfb1bf1b632733341cec6a4f75ff86f  guix-build-806b75b21340/output/powerpc64-linux-gnu/bitcoin-806b75b21340-powerpc64-linux-gnu-debug.tar.gz
  72698691b27ec0ac17f21dce8551de0ca683dd00b5b9442ea7616fb56cca8c6b  guix-build-806b75b21340/output/powerpc64-linux-gnu/bitcoin-806b75b21340-powerpc64-linux-gnu.tar.gz
  ef7c6d7184249eb59fa67d6df91d1a567570b9fb026dbb8682763029decaacca  guix-build-806b75b21340/output/powerpc64le-linux-gnu/SHA256SUMS.part
  fc6bb5619ee76899a88c1dd62640b429ad8957bbdf821238038b41fc87d18eca  guix-build-806b75b21340/output/powerpc64le-linux-gnu/bitcoin-806b75b21340-powerpc64le-linux-gnu-debug.tar.gz
  0eceb969f41f6b8dba88f641e268590de7edf0008318c8051d9cb208fb15e7f7  guix-build-806b75b21340/output/powerpc64le-linux-gnu/bitcoin-806b75b21340-powerpc64le-linux-gnu.tar.gz
  6f51a4791d87a610abd81cee83efa7f469e905829797bc2edac4fb95a2e0f3e4  guix-build-806b75b21340/output/riscv64-linux-gnu/SHA256SUMS.part
  c978706988f31e65a7991ff7582d79b3d1df44249c14d9807d93c01bf3f5080d  guix-build-806b75b21340/output/riscv64-linux-gnu/bitcoin-806b75b21340-riscv64-linux-gnu-debug.tar.gz
  93aefe058025067550692adae59ead14228ac252a9e7cf8b55c8fb4189ece545  guix-build-806b75b21340/output/riscv64-linux-gnu/bitcoin-806b75b21340-riscv64-linux-gnu.tar.gz
  862a53f6023bd1ca98a078ea540bba8ca9bfa335a9560f3d8d62ac873c2d5848  guix-build-806b75b21340/output/x86_64-apple-darwin/SHA256SUMS.part
  8f632b42c94d061fa30364087e75bb8b04eb2ac5a0a988eacc37c5983669f01b  guix-build-806b75b21340/output/x86_64-apple-darwin/bitcoin-806b75b21340-x86_64-apple-darwin-unsigned.dmg
  ce62e76ca446a6316b31490e12463c0a641e15beef9bdae7acc8e5db057b433a  guix-build-806b75b21340/output/x86_64-apple-darwin/bitcoin-806b75b21340-x86_64-apple-darwin-unsigned.tar.gz
  f57b014818e3b1ec07d27c8224ec4ac0e5786dacd00639513b599c6138790ece  guix-build-806b75b21340/output/x86_64-apple-darwin/bitcoin-806b75b21340-x86_64-apple-darwin.tar.gz
  70e75f6f13795f968f91719d221673b687bf747f90d77912cbcb2c1ee45ec623  guix-build-806b75b21340/output/x86_64-linux-gnu/SHA256SUMS.part
  30dec525364bb21a26cfe8bfff061d013c4ce849165aa67b06eb154019444862  guix-build-806b75b21340/output/x86_64-linux-gnu/bitcoin-806b75b21340-x86_64-linux-gnu-debug.tar.gz
  d8b3a996f25fb948b3555d5750852aaf82f7051848586b9ba0f4d0d223226e4b  guix-build-806b75b21340/output/x86_64-linux-gnu/bitcoin-806b75b21340-x86_64-linux-gnu.tar.gz
  4259adec77912bab6494f71a2a95d98093b116c05fc9ad03069e92de4ce0248c  guix-build-806b75b21340/output/x86_64-w64-mingw32/SHA256SUMS.part
  0a2d5cab3fe94a86def0cc1b6efe9ac871839cbcdc05ad92686df1d2bdd154ea  guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64-debug.zip
  d2a1876333bdb1cd5b8b1d4a52bccd756ea2e992c291dac233e65beeb0c905fd  guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64-setup-unsigned.exe
  192ea38d70e12c23327ff811ea930b50ac31c9fb2bc8dcc9391ad585112322ff  guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64-unsigned.tar.gz
  474f88a1f4cc8900a7d8967909336d4122e449ce98cacaf2cacec340780ede0b  guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    Nice cleanups, ACK 806b75b213

Tree-SHA512: cede797c3b9b88cc1588d0ff7ff9b2908316a8ba384d9087b16466aceeb2e0c194aa56e3023f6b6ce7ca8896a1b87ef56b966db198cc1712cb6ddc37fe684567
2023-08-24 09:19:57 +01:00
Sebastian Falbesoner
360ac64b90 test: previous releases: speed up fetching sources with shallow clone
For the sake of building previous releases, fetching the whole history
of the repository for each version seems to be overkill as it takes much
more time, bandwidth and disk space than necessary. Create a shallow
clone instead with history truncated to the one commit of the version
tag, which is directly checked out in the same command. This has the
nice side-effect that we can remove the extra `git checkout` step after
as it's not needed anymore.

Note that it might look confusing to pass a _tag_ to a parameter named
`--branch`, but the git-clone manpage explicitly states that this is
supported.
2023-08-24 02:37:36 +02:00
Pieter Wuille
8a3b6f3387 refactor: make Transport::ReceivedBytes just return success/fail 2023-08-23 20:13:49 -04:00
Pieter Wuille
bb4aab90fd net: move message conversion to wire bytes from PushMessage to SocketSendData
This furthers transport abstraction by removing the assumption that a message
can always immediately be converted to wire bytes. This assumption does not hold
for the v2 transport proposed by BIP324, as no messages can be sent before the
handshake completes.

This is done by only keeping (complete) CSerializedNetMsg objects in vSendMsg,
rather than the resulting bytes (for header and payload) that need to be sent.
In SocketSendData, these objects are handed to the transport as permitted by it,
and sending out the bytes the transport tells us to send. This also removes the
nSendOffset member variable in CNode, as keeping track of how much has been sent
is now a responsability of the transport.

This is not a pure refactor, and has the following effects even for the current
v1 transport:

* Checksum calculation now happens in SocketSendData rather than PushMessage.
  For non-optimistic-send messages, that means this computation now happens in
  the network thread rather than the message handler thread (generally a good
  thing, as the message handler thread is more of a computational bottleneck).
* Checksum calculation now happens while holding the cs_vSend lock. This is
  technically unnecessary for the v1 transport, as messages are encoded
  independent from one another, but is untenable for the v2 transport anyway.
* Statistics updates about per-message sent bytes now happen when those bytes
  are actually handed to the OS, rather than at PushMessage time.
2023-08-23 20:13:49 -04:00
Pieter Wuille
a1a1060fd6 net: measure send buffer fullness based on memory usage
This more accurately captures the intent of limiting send buffer size, as
many small messages can have a larger overhead that is not counted with the
current approach.

It also means removing the dependency on the header size (which will become
a function of the transport choice) from the send buffer calculations.
2023-08-23 20:13:49 -04:00
Pieter Wuille
009ff8d650 fuzz: add bidirectional fragmented transport test
This adds a simulation test, with two V1Transport objects, which send messages
to each other, with sending and receiving fragmented into multiple pieces that
may be interleaved. It primarily verifies that the sending and receiving side
are compatible with each other, plus a few sanity checks.
2023-08-23 20:13:45 -04:00
Pieter Wuille
fb2c5edb79 net: make V1Transport implicitly use current chainparams
The rest of net.cpp already uses Params() to determine chainparams in many
places (and even V1Transport itself does so in some places).

Since the only chainparams dependency is through the message start characters,
just store those directly in the transport.
2023-08-23 19:56:24 -04:00
Pieter Wuille
0de48fe858 net: abstract sending side of transport serialization further
This makes the sending side of P2P transports mirror the receiver side: caller provides
message (consisting of type and payload) to be sent, and then asks what bytes must be
sent. Once the message has been fully sent, a new message can be provided.

This removes the assumption that P2P serialization of messages follows a strict structure
of header (a function of type and payload), followed by (unmodified) payload, and instead
lets transports decide the structure themselves.

It also removes the assumption that a message must always be sent at once, or that no
bytes are even sent on the wire when there is no message. This opens the door for
supporting traffic shaping mechanisms in the future.
2023-08-23 19:56:24 -04:00
Pieter Wuille
649a83c7f7 refactor: rename Transport class receive functions
Now that the Transport class deals with both the sending and receiving side
of things, make the receive side have function names that clearly indicate
they're about receiving.

* Transport::Read() -> Transport::ReceivedBytes()
* Transport::Complete() -> Transport::ReceivedMessageComplete()
* Transport::GetMessage() -> Transport::GetReceivedMessage()
* Transport::SetVersion() -> Transport::SetReceiveVersion()

Further, also update the comments on these functions to (among others) remove
the "deserialization" terminology. That term is better reserved for just the
serialization/deserialization between objects and bytes (see serialize.h), and
not the conversion from/to wire bytes as performed by the Transport.
2023-08-23 19:56:24 -04:00
Pieter Wuille
27f9ba23ef net: add V1Transport lock protecting receive state
Rather than relying on the caller to prevent concurrent calls to the
various receive-side functions of Transport, introduce a private m_cs_recv
inside the implementation to protect the lock state.

Of course, this does not remove the need for callers to synchronize calls
entirely, as it is a stateful object, and e.g. the order in which Receive(),
Complete(), and GetMessage() are called matters. It seems impossible to use
a Transport object in a meaningful way in a multi-threaded way without some
form of external synchronization, but it still feels safer to make the
transport object itself responsible for protecting its internal state.
2023-08-23 19:56:24 -04:00
Pieter Wuille
93594e42c3 refactor: merge transport serializer and deserializer into Transport class
This allows state that is shared between both directions to be encapsulated
into a single object. Specifically the v2 transport protocol introduced by
BIP324 has sending state (the encryption keys) that depends on received
messages (the DH key exchange). Having a single object for both means it can
hide logic from callers related to that key exchange and other interactions.
2023-08-23 19:56:24 -04:00
Andrew Chow
afd9a673c4 test: roundtrip wallet backwards compat downgrade
Test that old nodes don't mess up new wallets by loading a downgraded
wallet in master again.
2023-08-23 16:49:41 -04:00
Andrew Chow
bbf43c63b9 test: Add 25.0 to wallet backwards compatibiilty test 2023-08-23 16:49:41 -04:00
Andrew Chow
538939ec39 test: Run upgrade test on all nodes 2023-08-23 16:49:41 -04:00
Andrew Chow
6d4699028b test: Run downgrade test on descriptor wallets 2023-08-23 16:49:41 -04:00
Andrew Chow
f158573be1 test: Add 0.21 tr() incompatibility test 2023-08-23 16:49:41 -04:00
Andrew Chow
f41215c3f0 test: add logging 0.17 incompatibilities in wallet back compat 2023-08-23 16:49:41 -04:00
Andrew Chow
71c03aeff7 test: Refactor v19 addmultisigaddress test to be distinct
This specific test is distinct from the rest of the backwards
compatibility tests as it is checking a specific failure.
2023-08-23 16:49:41 -04:00
Andrew Chow
53f35d02cb test: Remove w1_v18 from wallet backwards compatibility
This wallet is no longer used in the test
2023-08-23 16:49:41 -04:00
Andrew Chow
313d665437 test: Fix 0.16 wallet paths and downgrade test
The test for 0.16 wallet downgrading was using the wrong wallet path and
thus incorrectly finding that 0.16 could open wallets created in master.
2023-08-23 16:49:41 -04:00
Andrew Chow
23f3f402fc Merge bitcoin/bitcoin#27829: rpc: fix data optionality for RPC calls.
27b168b81f Update help text for spend and rawtransaction rpcs (Michael Tidwell)

Pull request description:

  The "data" field without outputs was marked as "required" in the help docs when using bitcoin-cli. This field when left off worked as an intended optional OP_RETURN. closes #27828.

  Motivation: It is hard to understand that "data" is actually optional for commands like `createpsbt` and `walletcreatefundedpsbt`.

ACKs for top commit:
  achow101:
    ACK 27b168b81f
  Sjors:
    tACK 27b168b81f

Tree-SHA512: 235e7ed4af69880880c04015b3f7de72c8f31ae035485c4c64c483e282948f3ea3f1eef16f15e260a1aaf21114150713516ba6a99967ccad9ecd91ff67cb0450
2023-08-23 16:37:28 -04:00
MarcoFalke
fab7f5c01d ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAG 2023-08-23 22:33:02 +02:00
Andrew Chow
8ff90d9dcf Merge bitcoin/bitcoin#26291: Update MANDATORY_SCRIPT_VERIFY_FLAGS
1b09cc5959 Make post-p2sh consensus rules mandatory for tx relay (Anthony Towns)
69c31bc748 doc, policy: Clarify comment on STANDARD_SCRIPT_VERIFY_FLAGS (Anthony Towns)

Pull request description:

  The `MANDATORY_SCRIPT_VERIFY_FLAGS` constant was introduced in #3843 to distinguish between block consensus rules and relay standardness rules. However it was not actually used in the consensus code path: instead it only differentiates between the failure being reported as `TX_CONSENSUS` and `mandatory-script-verify-flag-failed` vs `TX_NOT_STANDARD` and `non-mandatory-script-verify-flag`.

  This updates the list of mandatory flags to include the post-p2sh soft forks that are enforced as consensus rules via `GetBlockScriptFlags()`. The effect of this change is that validation.cpp will report `TX_CONSENSUS` failures for txs that fail dersig/csv/cltv/nulldummy/witness/taproot checks, instead of `TX_NOT_STANDARD`, which in turn adds `Misbehaving(100)` via `MaybePunishNodeForTx` in `net_processing`.

ACKs for top commit:
  Sjors:
    Code review ACK 1b09cc5959
  darosior:
    ACK 1b09cc5959
  achow101:
    ACK 1b09cc5959
  theStack:
    Concept and code-review ACK 1b09cc5959

Tree-SHA512: d3e5868e8cece478f2e934956ba0c231d8bb9c2daefd0df1f817774e292049902cfc1d0cd76dbd2e7722627a93eab2d7046ff678199aac70a2b01642e69349f1
2023-08-23 16:19:39 -04:00
brunoerg
bf26f978ff fuzz: coinselection, fix m_cost_of_change
`m_cost_of_change` must not be generated randomly
independent from m_change_fee. This commit changes
it to set it up according to `wallet/spend`.
2023-08-23 14:48:27 -03:00
brunoerg
6d9b26d56a fuzz: coinselection, BnB should never produce change 2023-08-23 14:48:27 -03:00
brunoerg
b2eb558407 fuzz: coinselection, compare GetSelectedValue with target
The valid results should have a target below the sum of
the selected inputs amounts. Also, it increases the
minimum value for target to make it more realistic.
2023-08-23 14:48:27 -03:00
brunoerg
0df0438c60 fuzz: coinselection, improve ComputeAndSetWaste
Instead of using `cost_of_change` for `min_viable_change`
and `change_cost`, and 0 for `change_fee`, use values from
`coin_params`. The previous values don't generate any effects
that is relevant for that context.
2023-08-23 14:48:27 -03:00
brunoerg
1e351e5db1 fuzz: coinselection, add coverage for Merge 2023-08-23 14:48:27 -03:00
brunoerg
f0244a8614 fuzz: coinselection, add coverage for GetShuffledInputVector/GetInputSet 2023-08-23 14:48:04 -03:00
brunoerg
808618b8a2 fuzz: coinselection, add coverage for AddInputs 2023-08-23 14:47:11 -03:00
MarcoFalke
fa15f7e082 ci: Remove no longer applicable section
This fails with:

"Error: determining starting point for build: no FROM statement found"
2023-08-23 15:40:21 +02:00
MarcoFalke
fa378bed56 ci: Start with clean env
This should help to avoid non-determinism.
2023-08-23 15:40:05 +02:00
MarcoFalke
fa8c250c2f ci: Limit scope of some env vars
No need to have a larger scope than needed.

Can be reviewed with --color-moved=dimmed-zebra
2023-08-23 15:39:45 +02:00
fanquake
33da5d0eb1 Merge bitcoin/bitcoin#21652: ci: Switch remaining Linux tasks to self-hosted
fa8e89d5e4 ci: Remove distro-name from task name (MarcoFalke)
fad006fa0a ci: Switch remaining tasks to self-hosted (MarcoFalke)

Pull request description:

  Cirrus CI will be capping the free compute soon. For now, switch more tasks to persistent worker, as recommended by Cirrus CI.

  (See slightly related discussion in https://github.com/bitcoin/bitcoin/issues/28098)

ACKs for top commit:
  pinheadmz:
    concept ACK fa8e89d5e4
  dergoegge:
    ACK fa8e89d5e4
  hebasto:
    ACK fa8e89d5e4.

Tree-SHA512: f6b172fee14856021b7a219b2490c8a163ad0137567c34a383080c68f8319b1d846923e508a968f43fb63ed6ce536dcb0611905fa288271f2267764b07bf9ecb
2023-08-23 14:27:35 +01:00
furszy
c4929cfa50 test: wallet_backup.py, fix intermittent failure in "restore using dumped wallet"
The failure arises because the test expects 'init_wallet()' (the test
framework function) creating a wallet with no keys. However, the function
also imports the deterministic private key used to receive the coinbase coins.

This causes a race within the "restore using dumped wallet" case, where we
intend to have a new wallet (with no existing keys) to test the
'importwallet()' RPC result.
The reason behind the intermittent failures might be other peers delivering
the chain right after node2 startup (sync of the validation queue included)
and prior to the 'node2.getbalance()' check.
2023-08-23 09:41:24 -03:00
Michael Tidwell
27b168b81f Update help text for spend and rawtransaction rpcs
fixing typo
2023-08-22 22:29:08 -04:00
Andrew Chow
5aa67eb365 Merge bitcoin/bitcoin#28199: test: tx orphan handling
9eac5a0529 [functional test] transaction orphan handling (glozow)
61e77bb901 [test framework] make it easier to fast-forward setmocktime (glozow)

Pull request description:

  I was doing some mutation testing (through reckless refactoring) locally and found some specific behaviors in orphan handling that weren't picked up by tests. Adding some of these test cases now can maybe help with reviewing refactors like #28031.

  - Parent requests aren't sent immediately. A delay is added and the requests are filtered by AlreadyHaveTx before they are sent, which means you can't use fake orphans to probe precise arrival timing of a tx.
  - Parent requests include all that are not AlreadyHaveTx. This means old confirmed parents may be requested.
  - The node does not give up on orphans if the peer responds to a parent request with notfound. This means that if a parent is an old confirmed transaction (in which notfound is expected), the orphan should still be resolved.
  - Rejected parents can cause an orphan to be dropped, but it depends on the reason and only based on txid.
  - Rejected parents can cause an orphan to be rejected too, by both wtxid and txid.
  - Requests for orphan parents should be de-duplicated with "regular" txrequest. If a missing parent has the same hash as an in-flight request, it shouldn't be requested.
  - Multiple orphans with overlapping parents should not cause duplicated parent requests.

ACKs for top commit:
  instagibbs:
    reACK 9eac5a0529
  dergoegge:
    reACK 9eac5a0529
  achow101:
    ACK 9eac5a0529
  fjahr:
    Code review ACK 9eac5a0529

Tree-SHA512: 85488dc6a3f62cf0c38e7dfe7839c01215b44b172d1755b18164d41d01038f3a749451241e4eba8b857fd344a445740b21d6382c45977234b21460e3f53b1b2a
2023-08-22 17:03:37 -04:00
Martin Zumsande
b3a93b409e test: add functional test for deadlock situation 2023-08-22 13:45:26 -04:00
brunoerg
90c4e6a241 fuzz: coinselection, add coverage for EligibleForSpending 2023-08-22 14:41:57 -03:00
brunoerg
2a031cb2c2 fuzz: coinselection, add CreateCoins
Move coins creation for a specific function. It
allows us to use it in other parts of the code.
2023-08-22 14:41:57 -03:00
Martin Zumsande
3557aa4d0a test: add basic tests for sendmsgtopeer to rpc_net.py 2023-08-22 13:28:15 -04:00
Martin Zumsande
a9a1d69391 rpc: add test-only sendmsgtopeer rpc
This rpc can be used when we want a node to send a message, but
cannot use a python P2P object, for example for testing of low-level
net transport behavior.
2023-08-22 13:28:15 -04:00
fanquake
806b75b213 guix: consolidate Linux GCC package
Refactor our Linux GCC to be a single 'package', and avoid the use of
`package-with-extra-configure-variable`.
2023-08-22 15:01:15 +01:00
fanquake
4415275f96 guix: consolidate glibc 2.27 package
Refactor our glibc 2.27 to be a single 'package', and avoid the use of
`package-with-extra-configure-variable`. This also lets us drop the
`enable_werror` workaround, and just use --disable-werror directly.

Employ the same workaround as the Guix glibc, to avoid a "permission
denied" failure during build:
```bash
make  subdir=sunrpc -C sunrpc ..=../ subdir_install
make[2]: Entering directory '/tmp/guix-build-glibc-cross-x86_64-linux-gnu-2.27.drv-0/source/sunrpc'
.././scripts/mkinstalldirs /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/rpc
mkdir -p -- /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/rpc
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 rpc/netdb.h /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/rpc/netdb.h
.././scripts/mkinstalldirs /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/nfs
mkdir -p -- /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/nfs
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 ../sysdeps/unix/sysv/linux/nfs/nfs.h /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/nfs/nfs.h
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 /tmp/guix-build-glibc-cross-x86_64-linux-gnu-2.27.drv-0/build/gnu/lib-names-64.h /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/gnu/lib-names-64.h
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 etc.rpc /etc/rpc
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install: cannot create regular file '/etc/rpc': Permission denied
make[2]: *** [Makefile:197: /etc/rpc] Error 1
```
2023-08-22 15:01:15 +01:00
fanquake
8372ab0ea3 Merge bitcoin/bitcoin#28294: guix: pre time-machine bump changes (Windows)
51324c9517 guix: pre time-machine bump changes (Windows) (fanquake)

Pull request description:

  This is some refactoring to the Windows Guix build that facilitates bumping our Guix time-machine. Namely, avoiding `package-with-extra-configure-variable`, which is non-functional in the newer time-machine, see https://issues.guix.gnu.org/64436.

  At the same time, consolidate our Windows GCC build into `mingw-w64-base-gcc`.
  Rename `gcc-10-remap-guix-store.patch` to avoid changing it whenever GCC changes.

  We move the old `building-on` inside `explicit-cross-configure`, so that non-windows builds continue to work. Note that `explicit-cross-configure` will be going away entirely (see #27897), so this is only temporary duplication.

  Split out of #27897. Most of the [[WIP] Windows commit](e50d5f8953), minus the gcov change, as it's not-yet needed, and any gexp changes, which need the newer time-machine.

  I'll also be splitting out the other changes we can do pre-timemachine bump, for easier review.

  Guix Build:
  ```bash
  9cff4df245df0b8e92d102bda786eb13a1f10be58940b98a16e119d9d4859d7f  guix-build-51324c9517a2/output/aarch64-linux-gnu/SHA256SUMS.part
  dbe6a5fe2e6152cd114b1d549a383384cd3325ed4625ad93f8d824ba457ae9cd  guix-build-51324c9517a2/output/aarch64-linux-gnu/bitcoin-51324c9517a2-aarch64-linux-gnu-debug.tar.gz
  e36409cd7287e4da62806168ee21ef764e607a11685bb36b7642381060291789  guix-build-51324c9517a2/output/aarch64-linux-gnu/bitcoin-51324c9517a2-aarch64-linux-gnu.tar.gz
  0721a4b61ad991ec97d09281eda223ed7bad32cb2b4812e05bc5e64772756be7  guix-build-51324c9517a2/output/arm-linux-gnueabihf/SHA256SUMS.part
  f2aa9412ff77bfc3092404533ec669a610c3fa95711b514abd6cd8ae6918aca1  guix-build-51324c9517a2/output/arm-linux-gnueabihf/bitcoin-51324c9517a2-arm-linux-gnueabihf-debug.tar.gz
  cda70b7df940dcc1fdeb25e2d3cc6ecce45732fdd741881e2b0384e35897b6d5  guix-build-51324c9517a2/output/arm-linux-gnueabihf/bitcoin-51324c9517a2-arm-linux-gnueabihf.tar.gz
  15b5d30336bcaf75db4138c59716af93b9660511222c5f1ad814fc9d7c48c852  guix-build-51324c9517a2/output/arm64-apple-darwin/SHA256SUMS.part
  a29b327e4d869629af3625d6b7fe44e029e98ac4fedb265133e0d88e045ef933  guix-build-51324c9517a2/output/arm64-apple-darwin/bitcoin-51324c9517a2-arm64-apple-darwin-unsigned.dmg
  b1af35290051da64462987e32f215464c5e2376afbe5b5bff272f74f160a3461  guix-build-51324c9517a2/output/arm64-apple-darwin/bitcoin-51324c9517a2-arm64-apple-darwin-unsigned.tar.gz
  6c3815392377a8497cab15f534cfdb749dbe31e39ca2ae7e53a55b37acfe069d  guix-build-51324c9517a2/output/arm64-apple-darwin/bitcoin-51324c9517a2-arm64-apple-darwin.tar.gz
  efff276305c36ccf2e6eeca56928e6284965d0d08b05565b0bfc94a3dc30cec4  guix-build-51324c9517a2/output/dist-archive/bitcoin-51324c9517a2.tar.gz
  0d70106907ac873bd31603b58183ef2cd3b0820ce20b27e3a8d00044b876835c  guix-build-51324c9517a2/output/powerpc64-linux-gnu/SHA256SUMS.part
  9a6cbc010d08c26c5bc587ed9c42383a6d004ac20426508e67364848aa099154  guix-build-51324c9517a2/output/powerpc64-linux-gnu/bitcoin-51324c9517a2-powerpc64-linux-gnu-debug.tar.gz
  e39ca22d186bad00119a740279b046a0040628847bea76db0b5d5345efa2312d  guix-build-51324c9517a2/output/powerpc64-linux-gnu/bitcoin-51324c9517a2-powerpc64-linux-gnu.tar.gz
  945ab751f65d5333b259fd169b280a5fabd35cae89a8f777bc6019f2ac011073  guix-build-51324c9517a2/output/powerpc64le-linux-gnu/SHA256SUMS.part
  1c33b4385d5759e4a38cd42e6af99c1224788d03e15b8abd3df48caef3ebcb0a  guix-build-51324c9517a2/output/powerpc64le-linux-gnu/bitcoin-51324c9517a2-powerpc64le-linux-gnu-debug.tar.gz
  3b4d0f6d874d600095b90eb19ad568ab08c105762c4ba285c6b1a4dc8d9fc2db  guix-build-51324c9517a2/output/powerpc64le-linux-gnu/bitcoin-51324c9517a2-powerpc64le-linux-gnu.tar.gz
  6d4f508256747841f9b099370cc7da2a97687f7b182683fce0369b524388e952  guix-build-51324c9517a2/output/riscv64-linux-gnu/SHA256SUMS.part
  34a474abaf833664448f3643d572eef284037535685af3f3928d9f859e7353e6  guix-build-51324c9517a2/output/riscv64-linux-gnu/bitcoin-51324c9517a2-riscv64-linux-gnu-debug.tar.gz
  b91a2209d360fe97d3d04cee4ff52271d45c2e47044e4f196dde4585cf61a273  guix-build-51324c9517a2/output/riscv64-linux-gnu/bitcoin-51324c9517a2-riscv64-linux-gnu.tar.gz
  88a40ee878a6340a4c5dd4bb82effaff814328e6b6f1345c5824170ba7498279  guix-build-51324c9517a2/output/x86_64-apple-darwin/SHA256SUMS.part
  0714092f76e32a1e8484651aed349ccdb5acc38f54dc2ac3ae3031111b5050eb  guix-build-51324c9517a2/output/x86_64-apple-darwin/bitcoin-51324c9517a2-x86_64-apple-darwin-unsigned.dmg
  ab66d04ea3e9304155e66b0e2899b8325fd609c3fed27c74fe1b53f1b9efea15  guix-build-51324c9517a2/output/x86_64-apple-darwin/bitcoin-51324c9517a2-x86_64-apple-darwin-unsigned.tar.gz
  222b688ef72b657c0701c8b4660240833e7c516fbbdcc4df8e9a006f5aa9b3bb  guix-build-51324c9517a2/output/x86_64-apple-darwin/bitcoin-51324c9517a2-x86_64-apple-darwin.tar.gz
  340e73e1697ae5cc1984771ce6e5caf22361ba4d063cbc4073236b210f1754dc  guix-build-51324c9517a2/output/x86_64-linux-gnu/SHA256SUMS.part
  a967753c192b8cb28cc644ce9d239da88ef8faaebc2094e21b7580616a0b8415  guix-build-51324c9517a2/output/x86_64-linux-gnu/bitcoin-51324c9517a2-x86_64-linux-gnu-debug.tar.gz
  7d77b14a2c94091a2d68dd9cdb0d878348da323c67eeeb5600273814db1b0e10  guix-build-51324c9517a2/output/x86_64-linux-gnu/bitcoin-51324c9517a2-x86_64-linux-gnu.tar.gz
  fbfb10b13b7bc5a1ed33578dcdd9483360cad47fd744ac99a9aecdf2a8ac14c2  guix-build-51324c9517a2/output/x86_64-w64-mingw32/SHA256SUMS.part
  925bf5f88e907ddc4567acecfc634ef3570b0576499c6427225a52e1f6209d1e  guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64-debug.zip
  b3d19a25df2dbd913814e74471da585e629187a19a6afede5890a7df2dd0101f  guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64-setup-unsigned.exe
  96b03fa66e294e95c0cee5d4c9869ac377e739d2da2e87c0c9d35eddd25e6ceb  guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64-unsigned.tar.gz
  95d7f52622c534e40dceb89f150b3f2e0061c56768bcf7399aac3f24b937983b  guix-build-51324c9517a2/output/x86_64-w64-mingw32/bitcoin-51324c9517a2-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 51324c9517

Tree-SHA512: 4d26ea2b587e4ea2bf76861925437b178376cda7373012f4a6d48ccb3606c20b035b94cba6248ac8d9e0c9e974e9eace818043d312c67d5c5e5771300a839c44
2023-08-22 15:00:48 +01:00
fanquake
03a536f1ed Merge bitcoin/bitcoin#28284: refactor: Remove confusing static_cast in address types
fadf671fa5 Refactor: Remove confusing static_cast (MarcoFalke)
faeea1ab58 refactor: Add missing includes (MarcoFalke)

Pull request description:

  It seems confusing to use `static_cast<uint160>(bla)` to call the constructor of `uint160`. The normal and common way to call a constructor is by simply calling it. (`uint160{bla}`).

  Do this, and also drop the constructor completely where the existing `const&` reference is enough.

  Also, add missing includes while touching the file.

ACKs for top commit:
  vincenzopalazzo:
    ACK fadf671fa5
  TheCharlatan:
    ACK fadf671fa5

Tree-SHA512: 8fb9a72203a6461b1f4b38bb90943ca25a92b218fc87da2022b90802e7747350e3668a13db3189201ad30e2e39a51d6658fed4aad176fd52cecc1c7f972c3134
2023-08-22 14:46:10 +01:00
Hennadii Stepanov
e7d67efd13 ci: Use concurrency for pull requests only
Otherwise, any previously pending workflow will be canceled on the
following push.
2023-08-22 11:26:01 +01:00
fanquake
38db2bd4e1 Merge bitcoin/bitcoin#28320: test: Support riscv64 in get_previous_releases.py
2222e15771 test: Support riscv64 in get_previous_releases.py (MarcoFalke)

Pull request description:

  To test: `test/get_previous_releases.py -b -t /tmp/prev_releases v0.18.1`

  On master: `Not sure which binary to download for riscv64-unknown-linux-gnu`
  Here: (pass)

ACKs for top commit:
  fanquake:
    ACK 2222e15771

Tree-SHA512: 18dc9a6c65f78adb5f7fc09e57db34c6b544071cb7bb3fa2846c86a23202e37d6ea1c5aca9acc1c2040b7d2b97bb93840a8a949f81f71fe6f01c395d2894739d
2023-08-22 10:35:13 +01:00
fanquake
00fc7cdc25 Merge bitcoin/bitcoin#28200: refactor: Remove unused includes from wallet.cpp
fa6286891f Remove unused includes from wallet.cpp (MarcoFalke)
fa8fdbe229 Remove unused includes from blockfilter.h (MarcoFalke)
fad8c36aa9 move-only: Create src/kernel/mempool_removal_reason.h (MarcoFalke)
fa57608800 Remove unused includes from txmempool.h (MarcoFalke)

Pull request description:

  This makes compilation of wallet.cpp use a few % less memory and time, locally.

  Created in the context of https://github.com/bitcoin/bitcoin/issues/28109, but I don't think it is enough to actually fix this problem.

ACKs for top commit:
  hebasto:
    ACK fa6286891f, I have reviewed the code and it looks OK.

Tree-SHA512: 06f1120af2a8ef3368dbd9ae747acda88ace2507bd261bcc10341d476a0b3d71c8485377ea6c108b47df3e4c13b7f75a15f486bafa6a8466303168dde16ebbc8
2023-08-22 10:34:10 +01:00
fanquake
c00bc63061 Merge bitcoin/bitcoin#28288: test: fix 'unknown named parameter' test in wallet_basic
452c094449 test: fix 'unknown named parameter' test in `wallet_basic` (brunoerg)

Pull request description:

  This PR removes loop when testing an unknown named parameter. They don't have any effect.

ACKs for top commit:
  jonatack:
    ACK 452c094449
  theStack:
    re-ACK 452c094449

Tree-SHA512: cf1a37d738bb6fdf9817e7b1d33bc69643dae61e3dbfae5c1e9f26220c55db6f134018dd9a1c65c13869ee58bcb6f3337c5999aabf2614d3126fbc01270705e8
2023-08-22 10:15:37 +01:00
MarcoFalke
fa5cc3ccfb test: Fix intermittent issue in mempool_reorg 2023-08-22 11:08:46 +02:00
MarcoFalke
2222e15771 test: Support riscv64 in get_previous_releases.py 2023-08-22 10:32:33 +02:00
glozow
a84dade1f9 Merge bitcoin/bitcoin#28157: test doc: tests acceptstalefeeestimates option is only supported on regtest chain
ee5a0369cc test: ensure acceptstalefeeestimates is supported only on regtest chain (ismaelsadeeq)
22d5d4b2b2 tx fees, policy: doc: update and delete unnecessary comment (ismaelsadeeq)

Pull request description:

  This PR Follow up comments from [#27622](https://github.com/bitcoin/bitcoin/pull/27622)

  It test that the new `regtest-only` option `acceptstalefeeestimates` is not supported on [main, signet and test chains](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235218268), removes an unnecessary [comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1235204323), and update fee estimator  `MAXFILEAGE` [description comment](https://github.com/bitcoin/bitcoin/pull/27622/files#r1233887314).

ACKs for top commit:
  jonatack:
    ACK ee5a0369cc
  glozow:
    utACK ee5a0369cc

Tree-SHA512: 4755f25b08db62f37614ea768272b12580ee0d481fb7fa339379901a6132c66828777c6747d3fe67490ceace3a6ff248bf13bdf65720f6e5ba8642eb762acd3c
2023-08-22 09:17:12 +01:00
brunoerg
452c094449 test: fix 'unknown named parameter' test in wallet_basic
Fixes loop when testing an unknown named parameter.
2023-08-21 20:59:15 -03:00
fanquake
ded6873340 Merge bitcoin/bitcoin#28292: ci: Disable cache save for pull requests in GitHub Actions
241d6ca34c ci: Disable cache save for pull requests in GitHub Actions (Hennadii Stepanov)

Pull request description:

  This PR disable cache save for pull requests in GitHub Actions.

  Otherwise, multiple pull requests fill GitHub Actions cache quota shortly.

  See a discussion [here](https://github.com/bitcoin/bitcoin/pull/28187#discussion_r1295459732).

  ---

  **NOTE** for the maintainers with "owner" permissions.

  This PR needs the `actions/cache/restore@*` and `actions/cache/save@*` acrions to be explicitly allowed in the repository's Actions permissions.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 241d6ca34c

Tree-SHA512: a7786c7ec99bfa6991bf6ae08fd7ed3546e8c5d083a1b2bae7638f6f31e77fdf2cf4fc69d85834faf87f98db1f4a82026ce1dc5fc1bc6650e8bf1c09bf7e90f5
2023-08-21 16:59:33 +01:00
Hennadii Stepanov
241d6ca34c ci: Disable cache save for pull requests in GitHub Actions
Otherwise, multiple pull requests fill GitHub Actions cache quota
shortly.
2023-08-21 11:26:11 +01:00
fanquake
723f1c669f Merge bitcoin/bitcoin#28218: refactor: Make IsInitialBlockDownload & NotifyHeaderTip not require a Chainstate
94a98fbd1d assumeutxo cleanup: Move IsInitialBlockDownload & NotifyHeaderTip to ChainstateManager (Ryan Ofsky)

Pull request description:

  This change makes `IsInitialBlockDownload` and `NotifyHeaderTip` functions no longer tied to individual `Chainstate` objects. It makes them work with the `ChainstateManager` object instead so code is simpler and it is no longer possible to call them incorrectly with an inactive `Chainstate`.

  This change also makes `m_cached_finished_ibd` caching easier to reason about, because now there is only one cached value instead of two (for background and snapshot chainstates) so the cached IBD state now no longer gets reset when a snapshot is loaded.

  There should be no change in behavior because these functions were always called on the active `ChainState` objects.

  These changes were discussed previously https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as possible followups for that PR.

ACKs for top commit:
  MarcoFalke:
    re-ACK 94a98fbd1d 🐺
  naumenkogs:
    ACK 94a98fbd1d
  dergoegge:
    reACK 94a98fbd1d

Tree-SHA512: 374d6e5c9bbc7564c143f634bd709a4e8f4a42c8d77e7a8554c832acdcf60fa2a134f3ea10827db1a1e0191006496329c0ebf5c64f3ab868398c3722bb7ff56f
2023-08-21 10:55:35 +01:00
ismaelsadeeq
ee5a0369cc test: ensure acceptstalefeeestimates is supported only on regtest chain 2023-08-21 07:21:34 +01:00
Ryan Ofsky
94a98fbd1d assumeutxo cleanup: Move IsInitialBlockDownload & NotifyHeaderTip to ChainstateManager
This change makes IsInitialBlockDownload and NotifyHeaderTip functions no
longer tied to individual Chainstate objects. It makes them work with the
ChainstateManager object instead so code is simpler and it is no longer
possible to call them incorrectly with an inactive Chainstate.

This change also makes m_cached_finished_ibd caching easier to reason about,
because now there is only one cached value instead of two (for background and
snapshot chainstates) so the cached IBD state now no longer gets reset when a
snapshot is loaded.

There should be no change in behavior because these functions were always
called on the active ChainState objects.

These changes were discussed previously
https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1246868905 and
https://github.com/bitcoin/bitcoin/pull/27746#discussion_r1237552792 as
possible followups for that PR.
2023-08-18 12:52:30 -04:00
MarcoFalke
fa8e89d5e4 ci: Remove distro-name from task name
The exact distro name should not be important. Also, it is easy to find
out, if needed. Thus, remove it to avoid bloat and maintenance overhead
having to keep it in sync.
2023-08-18 18:06:04 +02:00
MarcoFalke
fad006fa0a ci: Switch remaining tasks to self-hosted
This allows to drop unused templates, such as
cirrus_ephemeral_worker_template_env, or container_depends_template.

Also, ccache_cache, previous_releases_cache, and
base_depends_built_cache can be dropped, because the caching is done in
container volumes on the self-hosted runners.
2023-08-18 18:05:59 +02:00
fanquake
9b066da8af Merge bitcoin/bitcoin#28295: ci: Add missing amd64 to win64-cross task
fa56d17a4b ci: Add missing amd64 to win64-cross task (MarcoFalke)

Pull request description:

  Currently the task will fail if run on non-`x86_64`.

  Fix this by adding the missing `amd64`, similar to

  7bf078f2b7/ci/test/00_setup_env_i686_multiprocess.sh (L11)

ACKs for top commit:
  hebasto:
    ACK fa56d17a4b

Tree-SHA512: faab1c5b945283b7e8d080bbcc8e9379c480cf6973506149ace5990cb4d04673f83f4bc36d08d5b4e9cb17a86fdbe23ac97ef4eab0e842616b367b8138229c58
2023-08-18 14:42:01 +01:00
fanquake
93e8bc22bf Merge bitcoin/bitcoin#28296: ci: Add missing ${CI_RETRY_EXE} before curl
fa968ef6a3 ci: Add missing ${CI_RETRY_EXE} before curl (MarcoFalke)

Pull request description:

  GitHub is frequently down and this is causing many intermittent issues. For example, from today: https://cirrus-ci.com/task/5740122163904512?logs=ci#L398

  Try to fix it with a retry.

ACKs for top commit:
  hebasto:
    ACK fa968ef6a3

Tree-SHA512: e9a1e51af37ec718ca776f20d8b5394680a9b059fb2a22505ccb6781472e4b072694e7d01661ede5a87ef1f58a9143e29d032b6a5d13d8be3b7d46eeff061563
2023-08-18 13:59:18 +01:00
MarcoFalke
fa56d17a4b ci: Add missing amd64 to win64-cross task
Also, do the same for android, which also fails.
2023-08-18 14:23:17 +02:00
MarcoFalke
fa968ef6a3 ci: Add missing ${CI_RETRY_EXE} before curl 2023-08-18 14:11:40 +02:00
fanquake
51324c9517 guix: pre time-machine bump changes (Windows)
Split out of #27897. This is some refactoring to the Windows Guix build
that facilitates bumping our Guix time-machine. Namely, avoiding
`package-with-extra-configure-variable`, which is non-functional in the
newer time-machine, see https://issues.guix.gnu.org/64436.

At the same time, consolidate our Windows GCC build into mingw-w64-base-gcc.
Rename `gcc-10-remap-guix-store.patch` to avoid changing it whenever GCC changes.

We move the old `building-on` inside `explicit-cross-configure`, so that
non-windows builds continue to work. Note that `explicit-cross-configure`
will be going away entirely (see #27897).
2023-08-18 12:02:26 +01:00
fanquake
7bf078f2b7 Merge bitcoin/bitcoin#28237: refactor: Enforce C-str fmt strings in WalletLogPrintf()
fa60fa3b0c bitcoin-tidy: Apply bitcoin-unterminated-logprintf to spkm as well (MarcoFalke)
faa11434fe refactor: Enable all clang-tidy plugin bitcoin tests (MarcoFalke)
fa6dc57760 refactor: Enforce C-str fmt strings in WalletLogPrintf() (MarcoFalke)
fa244f3321 doc: Fix bitcoin-unterminated-logprintf tidy comments (MarcoFalke)

Pull request description:

  All fmt functions only accept a raw C-string as argument.

  There should never be a need to pass a format string that is not a compile-time string literal, so disallow it in `WalletLogPrintf()` to avoid accidentally introducing it.

  Apart from consistency, this also fixes the clang-tidy plugin bug https://github.com/bitcoin/bitcoin/pull/26296#discussion_r1286821141.

ACKs for top commit:
  theuni:
    ACK fa60fa3b0c

Tree-SHA512: fa6f4984c50f9b34e850bdfee7236706af586e512d866cc869cf0cdfaf9aa707029c210ca72d91f85e75fcbd8efe0d77084701de8c3d2004abfd7e46b6fa9072
2023-08-18 11:38:38 +01:00
fanquake
5eb669024f Merge bitcoin/bitcoin#28100: crypto: more Span<std::byte> modernization & follow-ups
57cc136282 crypto: make ChaCha20::SetKey wipe buffer (Pieter Wuille)
da0ec62e34 tests: miscellaneous hex / std::byte improvements (Pieter Wuille)
bdcbc8594c fuzz: support std::byte in Consume{Fixed,Variable}LengthByteVector (Pieter Wuille)
7d1cd93234 crypto: require key on ChaCha20 initialization (Pieter Wuille)
44c11769a8 random: simplify FastRandomContext::randbytes using fillrand (Pieter Wuille)
3da636e08b crypto: refactor ChaCha20 classes to use Span<std::byte> interface (Pieter Wuille)

Pull request description:

  This modernizes the ChaCha20 and ChaCha20Aligned interfaces to be `Span<std::byte>` based, and other improvements.

  * Modifies all functions and constructors of `ChaCha20` and `ChaCha20Aligned` to be `Span<std::byte>` based (aligning them with `FSChaCha20`, `AEADChaCha20Poly1305`, and `FSChaCha20Poly1305`)
  * Remove default constructors, to make sure all call sites provide a key (suggested in https://github.com/bitcoin/bitcoin/pull/26153#discussion_r1129313162)
  * Wipe key material on rekey for security (suggested in https://github.com/bitcoin/bitcoin/pull/26153#discussion_r1267164605)
  * Use `HexStr` on byte vectors in tests (suggested in https://github.com/bitcoin/bitcoin/pull/27993#discussion_r1262023316)
  * Support `std::byte` vectors in `ConsumeRandomLengthByteVector` and `ConsumeFixedLengthByteVector`, and use it (suggested in https://github.com/bitcoin/bitcoin/pull/27993#discussion_r1265337111)
  * And a few more.

  While related, I don't see this as a necessary for BIP324.

ACKs for top commit:
  stratospher:
    ACK 57cc136.
  theStack:
    re-ACK 57cc136282

Tree-SHA512: 361da4ff003c8465a32eeac0983a8a6f047dbbf5b400168b409c8e3234e79d577fc854e0764389446585da3e12b964c94dd67fc0c9c1d1d092cec296121e05d4
2023-08-18 11:19:34 +01:00
fanquake
e4a855c4e0 Merge bitcoin/bitcoin#28289: rpc: remove one more quote from non-string oneline description
2394314442 rpc: remove one more quote from non-string oneline description (Martin Zumsande)

Pull request description:

  This fixes a silent conflict between https://github.com/bitcoin/bitcoin/pull/28123 (which removed all `\"options\"`) and https://github.com/bitcoin/bitcoin/pull/27460 (which added a new one).

  It should fix the current CI failures.

ACKs for top commit:
  ajtowns:
    utACK 2394314442
  MarcoFalke:
    lgtm ACK 2394314442
  jonatack:
    ACK 2394314442
  hebasto:
    ACK 2394314442

Tree-SHA512: feb0c2b936a77be45d9c65aa7d738277b2266b5153665fee3b1413045de521195dc7d5efa2fc8b37b22f16e9b8d0ee8de25bfd151a428666122b31f64056557a
2023-08-18 10:01:22 +01:00
MarcoFalke
fa6286891f Remove unused includes from wallet.cpp
This removes unused includes, such as undo.h or txmempool.h from
wallet.cpp.

Also, add missing ones, according to IWYU.
2023-08-18 08:20:43 +02:00
kevkevin
9a84200cfc doc, refactor: Changing -torcontrol help to specify that a default port is used
Right now when we get the help for -torcontrol it says that there is a
default ip and port we dont specify if there is a specified ip that we
would also use port 9051 as default
2023-08-17 23:58:47 -05:00
Reese Russell
6e8f6468cb removed StrFormatInternalBug quote delimitation 2023-08-18 04:04:06 +00:00
Martin Zumsande
2394314442 rpc: remove one more quote from non-string oneline description
This fixes a silent conflict betwen #28123 and #27460
2023-08-17 16:18:56 -04:00
Pieter Wuille
57cc136282 crypto: make ChaCha20::SetKey wipe buffer 2023-08-17 15:37:41 -04:00
Pieter Wuille
da0ec62e34 tests: miscellaneous hex / std::byte improvements 2023-08-17 15:31:56 -04:00
Pieter Wuille
bdcbc8594c fuzz: support std::byte in Consume{Fixed,Variable}LengthByteVector 2023-08-17 15:31:56 -04:00
Pieter Wuille
7d1cd93234 crypto: require key on ChaCha20 initialization 2023-08-17 15:31:27 -04:00
Pieter Wuille
44c11769a8 random: simplify FastRandomContext::randbytes using fillrand 2023-08-17 15:26:38 -04:00
Pieter Wuille
3da636e08b crypto: refactor ChaCha20 classes to use Span<std::byte> interface 2023-08-17 15:26:34 -04:00
MarcoFalke
fa8fdbe229 Remove unused includes from blockfilter.h
This removes unused includes, primitives/block found manually, and the
others by iwyu:

blockfilter.h should remove these lines:
- #include <serialize.h>  // lines 16-16
- #include <undo.h>  // lines 18-18
2023-08-17 18:28:15 +02:00
Anthony Towns
1b09cc5959 Make post-p2sh consensus rules mandatory for tx relay 2023-08-18 00:59:27 +10:00
Anthony Towns
69c31bc748 doc, policy: Clarify comment on STANDARD_SCRIPT_VERIFY_FLAGS 2023-08-18 00:57:59 +10:00
MarcoFalke
fad8c36aa9 move-only: Create src/kernel/mempool_removal_reason.h
This is needed for a future commit. Can be reviewed with:
--color-moved=dimmed-zebra
2023-08-17 16:26:20 +02:00
MarcoFalke
fa57608800 Remove unused includes from txmempool.h
... and move them to where they are really needed.

This was found by IWYU:

txmempool.h should remove these lines:
- #include <random.h>  // lines 29-29
- class CBlockIndex;  // lines 43-43
- class Chainstate;  // lines 45-45

Also, move the stdlib section to the right place. Can be reviewed with:
--color-moved=dimmed-zebra
2023-08-17 16:25:31 +02:00
MarcoFalke
fadf671fa5 Refactor: Remove confusing static_cast 2023-08-17 15:55:07 +02:00
MarcoFalke
faeea1ab58 refactor: Add missing includes 2023-08-17 15:55:01 +02:00
fanquake
6ce5e8f475 Merge bitcoin/bitcoin#28278: ci: Refactor: Remove CI_USE_APT_INSTALL
fa26387769 ci: Refactor: Remove CI_USE_APT_INSTALL (MarcoFalke)

Pull request description:

  Seems odd to use `CI_USE_APT_INSTALL == no` as an alias for `CI_OS_NAME == macos`. Fix this by removing the alias.

  Also, for github CI:
  * restore MAKEJOBS to the same value as in cirrus.yml.
  * remove cirrus-only PACKAGE_MANAGER_INSTALL.
  * remove redundant TEST_RUNNER_TIMEOUT_FACTOR
  * Add M1 link

ACKs for top commit:
  hebasto:
    ACK fa26387769.

Tree-SHA512: e235aa70abd60738a9ad1531284a94e2122c9c7a22c2514ede437b49da5c06b2597fba7fccf615541fb3adb4e1f8076aa8c6047f926393191a629713554ab000
2023-08-17 14:17:40 +01:00
fanquake
de197c19e4 Merge bitcoin/bitcoin#28282: ci: Ensure that only a single workflow processes github.ref at a time
0080b5650e ci: Ensure that only a single workflow processes `github.ref` at a time (Hennadii Stepanov)

Pull request description:

  This PR ensures that only a single workflow processes any push or pull request at a time.

  A new push will be queued (including the master branch).

  For a new pull request update, the previous in-progress one will be cancelled.

  Address https://github.com/bitcoin/bitcoin/pull/28187#discussion_r1295144563.

ACKs for top commit:
  dergoegge:
    utACK 0080b5650e

Tree-SHA512: d03f1678c93c817c1c3be5e75171bfe85d494f99a9aab7113c9438e0c820e950edd5c10199cccd54868e6dc50217bb3aa5601b23bc88ad4927c001031e917513
2023-08-17 14:03:11 +01:00
fanquake
ecb20563b6 Merge bitcoin/bitcoin#28123: Bugfix: RPC: Remove quotes from non-string oneline descriptions
5e3e83b005 RPC/Mining: Document template_request better for getblocktemplate (Luke Dashjr)
de319c6175 RPC/rpcdoccheck: Error if a oneline_description has a quote for a non-string (Luke Dashjr)
7c61e9df90 Bugfix: RPC: Remove quotes from non-string oneline descriptions (Luke Dashjr)

Pull request description:

  Various JSON Object parameters had a `oneline_description` with quote characters. Fix those, and extend `rpcdoccheck` to detect them.

  Also, slightly improve GBT's oneline description for template_request.

ACKs for top commit:
  MarcoFalke:
    review ACK 5e3e83b005

Tree-SHA512: 363d1669a661d0acfc19fddb57e777d781c7246f330cf62160e77dde10a6adcb0249db748127067da1afe1b7d17c71cf611d9fdc3664d6bf5b3f30105637769a
2023-08-17 13:58:31 +01:00
fanquake
6d473bad22 Merge bitcoin/bitcoin#27941: test: Fix intermittent issue in mining_getblocktemplate_longpoll.py
fa748c6f2a test: Fix intermittent issue in mining_getblocktemplate_longpoll.py (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/26962

  Wait for the thread to have started and the RPC to have reached the node before continuing. Otherwise the test may run into a race.

  For example:

  ```
   test  2023-06-23T13:10:29.245000Z TestFramework (INFO): Test that introducing a new transaction into the mempool will terminate the longpoll
   node0 2023-06-23T13:10:29.245712Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.245915Z [httpworker.3] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__
   node0 2023-06-23T13:10:29.252594Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.254545Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblockchaininfo user=__cookie__
   node0 2023-06-23T13:10:29.256530Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.256741Z [httpworker.1] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=sendrawtransaction user=__cookie__
   node0 2023-06-23T13:10:29.258033Z [httpworker.1] [validationinterface.cpp:213] [TransactionAddedToMempool] [validation] Enqueuing TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778
   node0 2023-06-23T13:10:29.258263Z [httpworker.1] [txmempool.cpp:660] [check] [mempool] Checking mempool with 1 transactions and 1 inputs
   node0 2023-06-23T13:10:29.258542Z [scheduler] [validationinterface.cpp:213] [operator()] [validation] TransactionAddedToMempool: txid=38335600f2465c0f8bb2b86d5830a34851d86fa879800c0e1434ddfc78c42898 wtxid=c033cd3efd301c369d66cf759769159609471bd4f9efb3ee30e7209e57b74778
   node0 2023-06-23T13:10:29.259549Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.259745Z [httpworker.0] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=decoderawtransaction user=__cookie__
   node0 2023-06-23T13:10:29.261066Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:52690
   node0 2023-06-23T13:10:29.261803Z [http] [httpserver.cpp:254] [http_request_cb] [http] Received a POST request for / from 127.0.0.1:43568
   node0 2023-06-23T13:10:29.262770Z [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getblocktemplate user=__cookie__
  ```

  (`sendrawtransaction` is called before `getblocktemplate`)

ACKs for top commit:
  jamesob:
    Github ACK fa748c6f2a
  theStack:
    ACK fa748c6f2a

Tree-SHA512: c67d9ec7c56e8a22c1a26a3c3d4d4a4bcc17e4282cad0d66561ba2abd6e92240cb028369b4edc6077ea34e8736c0294f6066381979aee22a6166580cea43729a
2023-08-17 13:30:49 +01:00
fanquake
0a55bcd299 Merge bitcoin/bitcoin#27981: Fix potential network stalling bug
3388e523a1 Rework receive buffer pushback (Pieter Wuille)

Pull request description:

  See https://github.com/ElementsProject/elements/issues/1233. There, it has been observed that if both sides of a P2P connection have a significant amount of data to send, a stall can occur, where both try to drain their own send queue before trying to receive. The same issue seems to apply to the current Bitcoin Core codebase, though I don't know whether it's a frequent issue for us.

  The core issue is that whenever our optimistic send fails to fully send a message, we do subsequently not even select() for receiving; if it then turns out that sending is not possible either, no progress is made at all. To address this, the solution used in this PR is to still select() for both sending and receiving when an optimistic send fails, but skip receiving if sending succeeded, and (still) doesn't fully drain the send queue.

  This is a significant reduction in how aggressive the "receive pushback" mechanism is, because now it will only mildly push back while sending progress is made; if the other side stops receiving entirely, the pushback disappears. I don't think that's a serious problem though:
  * We still have a pushback mechanism at the application buffer level (when the application receive buffer overflows, receiving is paused until messages in the buffer get processed; waiting on our own net_processing thread, not on the remote party).
  * There are cases where the existing mechanism is too aggressive; e.g. when the send queue is non-empty, but tiny, and can be sent with a single send() call. In that case, I think we'd prefer to still receive within the same processing loop of the network thread.

ACKs for top commit:
  ajtowns:
    ACK 3388e523a1
  naumenkogs:
    ACK 3388e523a1
  mzumsande:
    Tested ACK 3388e523a1

Tree-SHA512: 28960feb3cd2ff3dfb39622510da62472612f88165ea98fc9fb844bfcb8fa3ed3633f83e7bd72bdbbbd37993ef10181b2e1b34836ebb8f0d83fd1c558921ec17
2023-08-17 13:15:42 +01:00
MarcoFalke
fa26387769 ci: Refactor: Remove CI_USE_APT_INSTALL 2023-08-17 13:55:18 +02:00
fanquake
7ef2d4ee4d Merge bitcoin/bitcoin#28244: Break up script/standard.{h/cpp}
91d924ede1 Rename script/standard.{cpp/h} to script/solver.{cpp/h} (Andrew Chow)
bacdb2e208 Clean up script/standard.{h/cpp} includes (Andrew Chow)
f3c9078b4c Clean up things that include script/standard.h (Andrew Chow)
8bbe257bac MOVEONLY: Move datacarrier defaults to policy.h (Andrew Chow)
7a172c76d2 Move CTxDestination to its own file (Andrew Chow)
145f36ec81 Move Taproot{SpendData/Builder} to signingprovider.{h/cpp} (Andrew Chow)
86ea8bed54 Move CScriptID to script.{h/cpp} (Andrew Chow)
b81ebff0d9 Remove ScriptHash from CScriptID constructor (Andrew Chow)
cba69dda3d Move MANDATORY_SCRIPT_VERIFY_FLAGS from script/standard.h to policy/policy.h (Anthony Towns)

Pull request description:

  Some future work needs to touch things in script/standard.{h/cpp}, however it is unclear if it is safe to do so as they are included in several different places that could effect standardness and consensus. It contains a mix of policy parameters, consensus parameters, and utilities only used by the wallet. This PR breaks up the various components and renames the files to clearly separate everything.

  * `CTxDestination` is moved to a new file `src/addresstype.{cpp/h}`
  * `TaprootSpendData` and `TaprootBuilder` (and their utility functions and structs) are moved to `SigningProvider` as these are used only during signing.
  * `CScriptID` is moved to `script/script.h` to be next to `CScript`.
  * `MANDATORY_SCRIPT_VERIFY_FLAGS` is moved to `interpreter.h`
  * The parameters `DEFAULT_ACCEPT_DATACARRIER` and `MAX_OP_RETURN_RELAY` are moved to `policy.h`
  * `standard.{cpp/h}` is renamed to `solver.{cpp/h}` since that's all that's left in the file after the above moves

ACKs for top commit:
  Sjors:
    ACK 91d924ede1
  ajtowns:
    ACK 91d924ede1
  MarcoFalke:
    ACK 91d924ede1 😇
  murchandamus:
    ACK 91d924ede1
  darosior:
    Code review ACK 91d924ede1.
  theStack:
    Code-review ACK 91d924ede1

Tree-SHA512: d347439890c652081f6a303d99b2bde6c371c96e7f4127c5db469764a17d39981f19884679ba883e28b733fde6142351dd8288c7bc61c379b7eefe7fa7acca1a
2023-08-17 12:54:16 +01:00
Hennadii Stepanov
0080b5650e ci: Ensure that only a single workflow processes github.ref at a time 2023-08-17 11:49:14 +01:00
fanquake
d78ff380a2 Merge bitcoin/bitcoin#28214: ci: Move tidy to persistent worker
faaa0794b2 refactor: Remove PERSISTENT_WORKER_* yaml templates (MarcoFalke)
fa1d8955f6 ci: Move tidy to persistent worker (MarcoFalke)

Pull request description:

  Cirrus CI will be capping the free compute soon. For now, switch more tasks to persistent worker, as recommended by Cirrus CI.

  (See slightly related discussion in https://github.com/bitcoin/bitcoin/issues/28098)

  Also, add more docs.

ACKs for top commit:
  hebasto:
    re-ACK faaa0794b2

Tree-SHA512: d83032eeeda7869969aa8504ed5e88089f896da850f97dfb799c4d4f64e6cb9da7973eec9a97b07f646613d1dabd2308dc0055ab6e1062d18bd34a201fcaf6db
2023-08-17 11:21:04 +01:00
ismaelsadeeq
22d5d4b2b2 tx fees, policy: doc: update and delete unnecessary comment 2023-08-17 11:09:14 +01:00
fanquake
a62f5ee86c Merge bitcoin/bitcoin#27675: p2p: Drop m_recently_announced_invs bloom filter
fb02ba3c5f mempool_entry: improve struct packing (Anthony Towns)
1a118062fb net_processing: Clean up INVENTORY_BROADCAST_MAX constants (Anthony Towns)
6fa49937e4 test: Check tx from disconnected block is immediately requestable (glozow)
e4ffabbffa net_processing: don't add txids to m_tx_inventory_known_filter (Anthony Towns)
6ec1809d33 net_processing: drop m_recently_announced_invs bloom filter (Anthony Towns)
a70beafdb2 validation: when adding txs due to a block reorg, allow immediate relay (Anthony Towns)
1e9684f39f mempool_entry: add mempool entry sequence number (Anthony Towns)

Pull request description:

  This PR replaces the `m_recently_announced_invs` bloom filter with a simple sequence number tracking the mempool state when we last considered sending an INV message to a node. This saves 33kB per peer (or more if we raise the rate at which we relay transactions over the network, in which case we would need to increase the size of the bloom filter proportionally).

  The philosophy here (compare with #18861 and #19109) is that we consider the rate limiting on INV messages to only be about saving bandwidth and not protecting privacy, and therefore after you receive an INV message, it's immediately fair game to request any transaction that was in the mempool at the time the INV message was sent. We likewise consider the BIP 133 feefilter and BIP 37 bloom filters to be bandwidth optimisations here, and treat transactions as requestable if they would have been announced without those filters. Given that philosophy, tracking the timestamp of the last INV message and comparing that against the mempool entry time allows removal of each of `m_recently_announced_invs`, `m_last_mempool_req` and `UNCONDITIONAL_RELAY_DELAY` and associated logic.

ACKs for top commit:
  naumenkogs:
    ACK fb02ba3c5f
  amitiuttarwar:
    review ACK fb02ba3c5f
  glozow:
    reACK fb02ba3c5f

Tree-SHA512: cbba5ee04c86df26b6057f3654c00a2b45ec94d354f4f157a769cecdaa0b509edaac02b3128afba39b023e82473fc5e28c915a787f84457ffe66638c6ac9c2d4
2023-08-17 10:52:06 +01:00
MarcoFalke
fa60fa3b0c bitcoin-tidy: Apply bitcoin-unterminated-logprintf to spkm as well 2023-08-16 14:56:37 +02:00
MarcoFalke
faa11434fe refactor: Enable all clang-tidy plugin bitcoin tests
This makes it easier to add new ones without having to modify this file
every time.
2023-08-16 14:48:06 +02:00
fanquake
60d3e4b0cd Merge bitcoin/bitcoin#28273: ci: Fix macOS-cross SDK rsync
fa6e5d3eef ci: Avoid error on macOS native (MarcoFalke)
fa193f5dfc ci: Fix macOS-cross SDK rsync (MarcoFalke)

Pull request description:

  This should fix the macOS-cross build on Cirrus CI containers.

  Locally this was already working, because the SDK was cached in
  `/ci_container_base/` in the image, which is also the folder used for a
  later CI run.

  However, on Cirrus CI, when using an image *and* a custom `BASE_ROOT_DIR`,
  the SDK will not be found in `/ci_base_install/`, nor in `BASE_ROOT_DIR`.

  Fix this by normalizing *all* folders to `/ci_container_base/`.

ACKs for top commit:
  hebasto:
    ACK fa6e5d3eef

Tree-SHA512: 8312f7e72c3638caa6804e39206d3563ba1703204d53ce63de22e0a16a71e1e143ec00fac6b43ebfc0653c7b74160472c04e95e2d694c8c0965e7dc39e627d39
2023-08-16 13:10:58 +01:00
fanquake
72304ccf1e Merge bitcoin/bitcoin#28257: test: check backup from migratewallet can be successfully restored
769f5b15f2 test: check backup from `migratewallet` can be successfully restored (brunoerg)

Pull request description:

  `migratewallet` migrates the wallet to a descriptor one. During the process, it generates a backup file of the wallet in case of an incorrect migration. This PR adds test to check if the backup file can be successfully restored.

ACKs for top commit:
  achow101:
    ACK 769f5b15f2
  MarcoFalke:
    lgtm ACK 769f5b15f2

Tree-SHA512: 94c50b34fbd47c4d3cc34b94e9e7903bc233608c7f50f45c161669996fd5f5b7d8f9a4e6a3437b9151d66a76af833f3f1ca28e44ecb63b5a8f391f6d6be0e39f
2023-08-16 12:56:09 +01:00
MarcoFalke
fa6e5d3eef ci: Avoid error on macOS native
This avoids "mkdir: /ci_container_base: Read-only file system"
2023-08-16 10:30:51 +02:00
MarcoFalke
fa193f5dfc ci: Fix macOS-cross SDK rsync
This should fix the macOS-cross build on Cirrus CI containers.

Locally this was already working, because the SDK was cached in
/ci_container_base/ in the image, which is also the folder used for a
later CI run.

However, on Cirrus CI, when using an image *and* a custom BASE_ROOT_DIR,
the SDK will not be found in /ci_base_install/, nor in BASE_ROOT_DIR.

Fix this by normalizing *all* folders to /ci_container_base/.
2023-08-16 10:30:50 +02:00
Andrew Chow
b8ee2fa02e Merge bitcoin/bitcoin#28240: refactor: Remove unused boost signals2 from torcontrol
faaba770e1 Sort includes in compat.h (MarcoFalke)
fa91a23d63 remove unused limits.h include in compat.h (MarcoFalke)
fa32af22b3 Replace LocaleIndependentAtoi with ToIntegral (MarcoFalke)
faab76c1c0 iwyu on torcontrol (MarcoFalke)
fa0a60dd93 Remove unused boost signals2 from torcontrol (MarcoFalke)

Pull request description:

  Remove unused boost, and other includes, and other legacy functions from torcontrol.

ACKs for top commit:
  TheCharlatan:
    Re-ACK faaba770e1
  achow101:
    ACK faaba770e1
  dergoegge:
    utACK faaba770e1

Tree-SHA512: 440f8d3ae9c3cf4dcc368e35b29459b5fcec8c6d233e8f9be3a854e7624b8633d6ccdde10cb0c6f74f86278e06557c4e9e24de30c3c692826237939265c6160a
2023-08-15 17:21:54 -04:00
Andrew Chow
b97b05048d Merge bitcoin/bitcoin#28187: ci: Run "macOS native x86_64" job on GitHub Actions
9658d0dc17 ci: Run "macOS native x86_64" job on GitHub Actions (Hennadii Stepanov)

Pull request description:

  From https://github.com/bitcoin/bitcoin/issues/28098:
  > Thus, someone would have to sponsor an amount of roughly 5kUSD/mo for those two tasks.

  > If the goal is to stay on a free plan, I think the only option is GitHub Actions CI.

  ---

  **IMPORTANT NOTE**. We currently ship macOS release binaries for both architectures: `x86_64` and `arm64`. If this PR gets merged, only `x86_64` architecture will be tested on CI, which implies some [drawbacks](https://github.com/bitcoin/bitcoin/pull/28187#issuecomment-1658077549).

  However, it has never been the case that our CI tested both architectures simultaneously. And we hope that GitHub Actions will soon host macOS `arm64` runners.

  Historically, we moved from `x86_64` to `arm64` in https://github.com/bitcoin/bitcoin/pull/26388 less than a year ago.

  ---

  Security concerns:
  - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651432106
  - https://github.com/bitcoin/bitcoin/issues/28098#issuecomment-1651688197

  `GITHUB_TOKEN` permissions (from the build log in my personal repo):
  ```
  2023-07-27T07:30:17.8313534Z ##[group]GITHUB_TOKEN Permissions
  2023-07-27T07:30:17.8314113Z Contents: read
  2023-07-27T07:30:17.8314608Z Metadata: read
  2023-07-27T07:30:17.8314957Z Packages: read
  2023-07-27T07:30:17.8315233Z ##[endgroup]
  ```

  Comparison of resources:

  | Resource | Current, Cirrus CI | Suggested, GitHub Actions |
  |---|:-:|:-:|
  | CPU | 4 | 4 \*\* |
  | RAM, GB | 8 | 14 |

  **\*\* NOTE**: However, [docs](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners#supported-runners-and-hardware-resources) are mentioning:
  > 3-core CPU (x86_64)

ACKs for top commit:
  MarcoFalke:
    re-ACK 9658d0dc17 🏂
  achow101:
    ACK 9658d0dc17
  jarolrod:
    ACK 9658d0dc17

Tree-SHA512: 6123e68e6784cdf4e53c3e77b435709261db21f09091af2c22e667d3816a305fffb9d617297a5bc1bda18aaba84a6e210cec6a75c52afa7746a3780a67b69865
2023-08-15 17:03:13 -04:00
Andrew Chow
cd43a8444b Merge bitcoin/bitcoin#27460: rpc: Add importmempool RPC
fa776e61cd Add importmempool RPC (MarcoFalke)
fa20d734a2 refactor: Add and use kernel::ImportMempoolOptions (MarcoFalke)
fa8866990d doc: Clarify the getmempoolinfo.loaded RPC field documentation (MarcoFalke)
6888886cec Remove Chainstate::LoadMempool (MarcoFalke)

Pull request description:

  Currently it is possible to import a mempool by placing it in the datadir and starting the node. However this has many issues:

  * Users aren't expected to fiddle with the datadir, possibly corrupting it
  * An existing mempool file in the datadir may be overwritten
  * The node needs to be restarted
  * Importing an untrusted file this way is dangerous, because it can corrupt the mempool

  Fix all issues by adding a new RPC.

ACKs for top commit:
  ajtowns:
    utACK fa776e61cd
  achow101:
    ACK fa776e61cd
  glozow:
    reACK fa776e61cd

Tree-SHA512: fcb1a92d6460839283c546c47a2d930c363ac1013c4c50dc5215ddf9fe5e51921d23fe0abfae0a5a7631983cfc7e2fff3788b70f95937d0a989a203be4d67546
2023-08-15 10:15:22 -04:00
fanquake
80d70cb6b0 Merge bitcoin/bitcoin#28185: ci: Use hard-coded root path for CI containers (bugfix)
fafa17c00b ci: Use hard-coded root path for CI containers (MarcoFalke)
fa084f5ba5 ci: Only create folders when needed (MarcoFalke)
fab27127f4 ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIR (MarcoFalke)

Pull request description:

  Currently the CI system will fail if the git folder that holds the Bitcoin Core source is moved from one location to another.

  Fix this by using a single hard-coded root path *inside* the CI system containers.

  Steps to test:

  * Run the CI system: `MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_win64.sh" ./ci/test_run_all.sh`
  * Move the git folder: `pwd && cd .. && mv bitcoin_core_folder_1 bitcoin_core_folder_2 && cd ./bitcoin_core_folder_2 && pwd`
  * Run the CI system again: (same cmd as above)

  On master (error):

  ```
  STRIPPROG="x86_64-w64-mingw32-strip" /bin/bash /bitcoin_core_folder_2/ci/scratch/build/bitcoin-x86_64-w64-mingw32/build-aux/install-sh -c -s ./src/qt/bitcoin-qt.exe ./release
  /bitcoin_core_folder_2/ci/scratch/build/bitcoin-x86_64-w64-mingw32/build-aux/install-sh: ./src/qt/bitcoin-qt.exe does not exist.
  make: *** [Makefile:1258: bitcoin-25.99.0-win64-setup.exe] Error 1
  ```

  On this pull: (pass).

ACKs for top commit:
  fanquake:
    ACK fafa17c00b - somewhat tested. MSAN changes are the same as what we did for tidy.

Tree-SHA512: 2ce693a3773c70fcfca062c2a6f0e5a16b94960b34a6145d10cee1a28f79154829d59d014465ccbb80e1cb9dcd5aa043729cee9afd2c4175b05e9bc945364b79
2023-08-15 11:17:39 +01:00
fanquake
85e672ab3d Merge bitcoin/bitcoin#28269: ci: Drop no longer needed macos_sdk_cache
c2a87bd302 ci: Drop no longer needed `macos_sdk_cache` (Hennadii Stepanov)

Pull request description:

  It has been cached in the Docker image since https://github.com/bitcoin/bitcoin/pull/27028.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK c2a87bd302

Tree-SHA512: fba888d132910f9600db0acccf633400e699f7d5ca802ef109a792546f60f3a04791c503b43d156c8debc3bd0bce2ad911f6209eabc47cd63fce7996a6ae3cfc
2023-08-15 11:13:44 +01:00
fanquake
5606d7f5a8 Merge bitcoin/bitcoin#28267: crypto: BIP324 ciphersuite follow-up
93cb8f0380 refactor: add missing headers for BIP324 ciphersuite (stratospher)
d22d5d925c crypto: BIP324 ciphersuite follow-up (stratospher)

Pull request description:

  follow-up to #28008.
  * move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests outside of the loop to be reused every time
  * use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()`
  * comment for initiator in `BIP324Cipher::Initialize()`
  * systematically damage ciphertext with bit positions in bip324_tests
  * use 4095 max bytes for `aad` in bip324 fuzz test

ACKs for top commit:
  fanquake:
    ACK 93cb8f0380 - thanks for following up here.

Tree-SHA512: 361f3e226d3168fdef69a2eebe6092cfc04ba14ce009420222e762698001eaf8be69a1138dab0be237964509c2b96a41a0b4db5c1df43ef75062f143c5aa741a
2023-08-15 11:11:55 +01:00
fanquake
e38c225261 Merge bitcoin/bitcoin#28215: fuzz: fix a couple incorrect assertions in the coins_view target
e417c988f6 fuzz: coins_view: remove an incorrect assertion (Antoine Poinsot)
c5f6b1db56 fuzz: coins_view: correct an incorrect assertion (Antoine Poinsot)

Pull request description:

  The `coins_view` fuzz target would assert in two places that the cache is consistent with the backend. But it's never the case (that's the whole point of using a cache).

  The only reason this didn't result in a crash was that we would never actually hit these assertions. I ran into this while introducing a new target with an in-memory `CCoinsViewDB` as the backend view (see https://github.com/bitcoin/bitcoin/pull/28216) which made the code paths with those assertions actually reachable.

ACKs for top commit:
  dergoegge:
    Code review ACK e417c988f6

Tree-SHA512: 5847bb2744a2f2831dace62d32b79cc491bf54e2af4ce425411d245d566622d9aff816d9be5ec8e830d10851c13f2500bf4f0c004d88b4d7cca1d483ef8960a6
2023-08-15 11:05:42 +01:00
MarcoFalke
faaa0794b2 refactor: Remove PERSISTENT_WORKER_* yaml templates
* PERSISTENT_WORKER_TEMPLATE_ENV is not needed at all, because
  RESTART_CI_DOCKER_BEFORE_RUN is already set on the persistent worker.
* PERSISTENT_WORKER_TEMPLATE can be replaced by pinning the
  previous_releases task to a type of worker. This should make the CI
  performance more consistent.
2023-08-15 11:59:30 +02:00
stratospher
93cb8f0380 refactor: add missing headers for BIP324 ciphersuite 2023-08-15 07:30:48 +05:30
Andrew Chow
91d924ede1 Rename script/standard.{cpp/h} to script/solver.{cpp/h}
Since script/standard only contains things that are used by the Solver
and its callers, rename the files to script/solver.
2023-08-14 17:39:49 -04:00
Andrew Chow
bacdb2e208 Clean up script/standard.{h/cpp} includes 2023-08-14 17:38:27 -04:00
Andrew Chow
f3c9078b4c Clean up things that include script/standard.h
Remove standard.h from files that don't use anything in it, and include
it in files that do.
2023-08-14 17:38:27 -04:00
Andrew Chow
8bbe257bac MOVEONLY: Move datacarrier defaults to policy.h 2023-08-14 17:38:27 -04:00
Andrew Chow
7a172c76d2 Move CTxDestination to its own file
CTxDestination is really our internal representation of an address and
doesn't really have anything to do with standard script types, so move
them to their own file.
2023-08-14 17:38:27 -04:00
Andrew Chow
145f36ec81 Move Taproot{SpendData/Builder} to signingprovider.{h/cpp}
TaprootSpendData and TaprootBuilder are used in signing in
SigningProvider contexts, so they should live near that.
2023-08-14 17:38:27 -04:00
Andrew Chow
86ea8bed54 Move CScriptID to script.{h/cpp}
CScriptID should be next to CScript just as CKeyID is next to CPubKey
2023-08-14 17:38:27 -04:00
Andrew Chow
b81ebff0d9 Remove ScriptHash from CScriptID constructor
Replaces the constructor in CScriptID that converts a ScriptHash with a
function ToScriptID that does the same. This prepares for a move of
CScriptID to avoid a circular dependency.
2023-08-14 17:38:27 -04:00
Anthony Towns
cba69dda3d Move MANDATORY_SCRIPT_VERIFY_FLAGS from script/standard.h to policy/policy.h 2023-08-14 17:38:27 -04:00
Hennadii Stepanov
c2a87bd302 ci: Drop no longer needed macos_sdk_cache
It has been cached in the Docker image since https://github.com/bitcoin/bitcoin/pull/27028.
2023-08-14 17:02:06 +01:00
fanquake
aadaa5625e Merge bitcoin/bitcoin#28232: test: locked_wallet, skip default fee estimation
5364dd8666 test: locked_wallet, skip default fee estimation (furszy)

Pull request description:

  Coming from https://github.com/bitcoin/bitcoin/pull/28139#discussion_r1284563239.

  No test case in this file is meant to exercise fee estimation. All default wallets have a
  custom tx fee set [here](b7138252ac/test/functional/wallet_fundrawtransaction.py (L100)). The only one missing is the one created for `locked_wallet`.

ACKs for top commit:
  theStack:
    ACK 5364dd8666

Tree-SHA512: 514c02708081d18330d759d10e306cee16c6350de243c68f0973777d2582f5d81968a237393c1f59aba245297e03f3f98d3ae5249a042469d0d016255f568719
2023-08-14 16:18:10 +01:00
glozow
9eac5a0529 [functional test] transaction orphan handling 2023-08-14 15:54:31 +01:00
glozow
61e77bb901 [test framework] make it easier to fast-forward setmocktime
Have each TestNode keep track of the last timestamp it called
setmocktime with, and add a bumpmocktime() function to bump by a
number of seconds. Makes it easy to fast forward n seconds without
keeping track of what the last timestamp was.
2023-08-14 15:53:33 +01:00
fanquake
6c508ac3ff Merge bitcoin/bitcoin#28258: bitcoin-tidy: fix macOS build
bb3263d3e3 bitcoin-tidy: fix macOS build (Cory Fields)

Pull request description:

  [LLVM uses these options](https://github.com/llvm/llvm-project/blob/main/llvm/cmake/modules/HandleLLVMOptions.cmake#L178) for building as well, so there's precedent.

  Also fix the shared library extension which was incorrectly being set to dylib.

  Thanks to jonatack for reporting and debugging.

ACKs for top commit:
  jonatack:
    ACK bb3263d3e3 tested with arm64 macos 13.5, llvm 16.0.6 and cmake 3.27.2

Tree-SHA512: de7bfd497f38f1565a14d217d0b057cbfa788bdda702b5942b7f0b55947ae5e1c05af13e7d6a073ed036bc4db57035868f180034508b6e084ab9b901a5baaf2f
2023-08-14 13:10:13 +01:00
stratospher
d22d5d925c crypto: BIP324 ciphersuite follow-up
follow-up to #28008.
* move `dummy_tag` variable in FSChaCha20Poly1305 crypto_tests
outside of the loop to be reused every time
* use easy to read `cipher.last()` in `AEADChaCha20Poly1305::Decrypt()`
* comment for initiator in `BIP324Cipher::Initialize()`
* systematically damage ciphertext with bit positions in bip324_tests
* use 4095 max bytes for aad in bip324 fuzz test
2023-08-14 09:03:21 +05:30
furszy
32db15450a gui: make '-min' minimize wallet loading dialog
When '-min' is enabled, no loading dialog should
be presented on screen during startup.
2023-08-13 20:38:07 -03:00
brunoerg
769f5b15f2 test: check backup from migratewallet can be successfully restored 2023-08-11 16:40:23 -03:00
Antoine Poinsot
e417c988f6 fuzz: coins_view: remove an incorrect assertion
Again, this was not hit because the default implementation of
`CCoinsView` return `false` for `GetCoin`.
2023-08-11 18:11:07 +02:00
fanquake
3654d84c6f Merge bitcoin/bitcoin#28245: doc: use llvm-config for bitcoin-tidy example
d82bb90a5b doc: use llvm-config for bitcoin-tidy example (fanquake)

Pull request description:

  An LLVM installation will have `llvm-config` available to query for info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy example, rather than listing multiple different (potential) paths per distro/OS etc.

ACKs for top commit:
  theuni:
    ACK d82bb90a5b.
  jonatack:
    ACK d82bb90a5b
  TheCharlatan:
    Nice, Re-ACK d82bb90a5b

Tree-SHA512: e07e979231f8f000deafce0751bed4b73ff0eff995bec49e90f579c9051cf5859dac5e49554b8219d33b00c81192db979eed98fee1c643a9205ea8babfce2c5d
2023-08-11 11:36:38 +02:00
Cory Fields
bb3263d3e3 bitcoin-tidy: fix macOS build
LLVM uses these options for building as well, so there's precedent.

Also fix the shared library extension which was incorrectly being set to dylib
2023-08-10 21:14:50 +00:00
furszy
8e7e3e6149 test: wallet, verify migration doesn't crash for an invalid script
The migration process must skip any invalid script inside the legacy
spkm and all the addressbook records linked to them.

These scripts are not being watched by the current wallet, nor should
be watched by the migrated one.

IsMine() returns ISMINE_NO for them.
2023-08-10 10:38:03 -03:00
furszy
1de8a2372a wallet: disallow migration of invalid or not-watched scripts
The legacy wallet allowed to import any raw script, without checking if
it was valid or not. Appending it to the watch-only set.

This causes a crash in the migration process because we are only
expecting to find valid scripts inside the legacy spkm.

These stored scripts internally map to `ISMINE_NO` (same as if they
weren't stored at all..).

So we need to check for these special case, and take into account that
the legacy spkm could be storing invalid not watched scripts.

Which, in code words, means IsMineInner() returning IsMineResult::INVALID
for them.
2023-08-10 10:35:29 -03:00
furszy
5364dd8666 test: locked_wallet, skip default fee estimation
Same as we do with the nodes default wallets.
No test case on this file is meant to exercise fee estimation.
2023-08-10 09:50:53 -03:00
fanquake
d82bb90a5b doc: use llvm-config for bitcoin-tidy example
An LLVM installation will have `llvm-config` available to query for
info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy
example, rather than listing multiple different (potential) paths per
distro/OS etc.
2023-08-10 12:39:35 +02:00
fanquake
b2ec0326fd Merge bitcoin/bitcoin#28008: BIP324 ciphersuite
1c7582ead6 tests: add decryption test to bip324_tests (Pieter Wuille)
990f0f8da9 Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers (Pieter Wuille)
c91cedf281 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt (Pieter Wuille)
af2b44c76e bench: add benchmark for FSChaCha20Poly1305 (Pieter Wuille)
aa8cee9334 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305 (Pieter Wuille)
0fee267792 crypto: add FSChaCha20, a rekeying wrapper around ChaCha20 (Pieter Wuille)
9ff0768bdc crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439 (Pieter Wuille)
9fd085a1a4 crypto: remove outdated variant of ChaCha20Poly1305 AEAD (Pieter Wuille)

Pull request description:

  Depends on #27985 and #27993, based on and partially replaces #25361, part of #27634. Draft while dependencies are not merged.

  This adds implementations of:
  * The ChaCha20Poly1305 AEAD from [RFC8439 section 2.8](https://datatracker.ietf.org/doc/html/rfc8439#section-2.8), including test vectors.
  * The FSChaCha20 stream cipher as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20.
  * The FSChaCha20Poly1305 AEAD as specified in [BIP324](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#rekeying-wrappers-fschacha20poly1305-and-fschacha20), a rekeying wrapper around ChaCha20Poly1305.
  * A BIP324Cipher class that encapsulates key agreement, key derivation, and stream ciphers and AEADs for [BIP324 packet encoding](https://github.com/bitcoin/bips/blob/master/bip-0324.mediawiki#overall-packet-encryption-and-decryption-pseudocode).

  The ChaCha20Poly1305 and FSChaCha20Poly1305 implementations are new, taking advance of the improvements in #27993.

ACKs for top commit:
  jamesob:
    reACK 1c7582e
  theStack:
    ACK 1c7582ead6
  stratospher:
    tested ACK 1c7582e.

Tree-SHA512: 06728b4b95b21c5b732ed08faf40e94d0583f9d86ff4db3b92dd519dcd9fbfa0f310bc66ef1e59c9e49dd844ba8c5ac06e2001762a804fb5aa97027816045a46
2023-08-10 11:58:59 +02:00
willcl-ark
c8e066461b doc: Improve documentation of rpcallowip rpchelp
Closes #21070

v21.0 introduced a behaviour changed noted in #21070 where using a config value
`rpcallowip=::0` no longer also permitted ipv4 ip addresses.

The rpc_bind.py functional test covers this new behaviour already by checking
that the list of bind addresses exactly matches what is expected so this
commit only updates the documentation.
2023-08-10 08:09:32 +01:00
fanquake
ef3f9f389f Merge bitcoin/bitcoin#28189: doc: diversify network outbounds release note
7463d259e1 doc: Add release note (Amiti Uttarwar)

Pull request description:

  release notes for #27213

ACKs for top commit:
  mzumsande:
    ACK 7463d259e1

Tree-SHA512: 16c479774ed9242d8d044d08cc919550ccd07020423a3dcd99f07dad36e4dafd8243dc47f9f7f0c8eedcb53efd85ec65afedba56422452f637d313ec7c901520
2023-08-09 19:11:51 +02:00
glozow
0d9a13ddd8 Merge bitcoin/bitcoin#28149: net processing: clamp PeerManager::Options user input
547fa52443 net processing: clamp -blockreconstructionextratxn to uint32_t bounds (stickies-v)
e451d1e3c6 net processing: clamp -maxorphantx to uint32_t bounds (stickies-v)
aa89e04e07 doc: document PeerManager::Options members (stickies-v)

Pull request description:

  Avoid out-of-bounds user input for `PeerManager::Options` by safely clamping `-maxorphantx` and `-blockreconstructionextratxn`, and avoid platform-specific behaviour by changing `PeerManager::Options::max_extra_txs` from `size_t` to a `uint32_t`. Addresses https://github.com/bitcoin/bitcoin/pull/27499#pullrequestreview-1544114932.

  Also documents all `PeerManager::Options` members, addressing https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1272302469.

ACKs for top commit:
  dergoegge:
    Code review ACK 547fa52443
  glozow:
    reACK 547fa52443

Tree-SHA512: 042d47b35bb8a7b29ef3dadd4c0c5d26f13a8f174f33687855d603c19f8de0fcbbda94418453331e149885412d4edd5f402d640d938f6d94b4dcf54e2fdbbcc9
2023-08-09 14:26:03 +02:00
MarcoFalke
fa1d8955f6 ci: Move tidy to persistent worker 2023-08-09 12:50:21 +02:00
MarcoFalke
fafa17c00b ci: Use hard-coded root path for CI containers 2023-08-09 12:32:44 +02:00
MarcoFalke
fa084f5ba5 ci: Only create folders when needed
Now that container volumes are used, the folders are no longer mounted.
They are only needed when running without a container engine (docker,
podman).
2023-08-09 12:32:30 +02:00
MarcoFalke
fab27127f4 ci: Drop BASE_SCRATCH_DIR from LIBCXX_DIR
Using a hard-coded path avoids non-determinism issues and improves CI
UX.
2023-08-09 12:32:15 +02:00
fanquake
492257019d Merge bitcoin/bitcoin#28087: ci: Use qemu-user through container engine
fad0b67c21 ci: Use qemu-user through container engine (MarcoFalke)

Pull request description:

  Currently the CI containers always run on the host architecture, and only wrap `bitcoind` into `qemu-user` when needed. This has many issues:

  * The `i386` tasks can not be run on non-x86 hosts.
  * `config.guess` isn't present when building the CI image, which is fine. But it prints a warning, see https://github.com/bitcoin/bitcoin/pull/27739#pullrequestreview-1446580353
  * The python tests are run on the host architecture, making it harder to find architecture specific bugs. See for example https://github.com/bitcoin/bitcoin/pull/27529#issuecomment-1521387810
  * All modern container engines support automatic dispatch to qemu-user, so it seems redundant to re-invent the wheel.

  Fix all issues by:
  * removing `HOST` from `ci/test/00_setup_env.sh`.
  * removing `QEMU_USER_CMD` and `ci/test/wrap-qemu.sh`.
  * removing `DPKG_ADD_ARCH` where possible, and pruning `PACKAGES` where possible.
  * specifying the architecture in `CI_IMAGE_NAME_TAG` to be used by the container engine.

ACKs for top commit:
  fanquake:
    ACK fad0b67c21 - this seems ok to me, and removes complexity from our CI system.

Tree-SHA512: 85e79f9f570e292d70a629d112fd4a6e6217d96226a1b665ed13485f616d84720ad2126b7d4b22fc603049f72fa7f2163b56a6bc276319fcd8b0496304ea4157
2023-08-09 12:09:00 +02:00
Hennadii Stepanov
9658d0dc17 ci: Run "macOS native x86_64" job on GitHub Actions
Also, the "macOS native arm64" task has been removed from Cirrus CI.
2023-08-09 10:59:43 +01:00
MarcoFalke
faaba770e1 Sort includes in compat.h
Can be reviewed with:
--color-moved=blocks  --color-moved-ws=ignore-all-space --ignore-all-space
2023-08-08 17:50:41 +02:00
MarcoFalke
fa91a23d63 remove unused limits.h include in compat.h 2023-08-08 17:47:23 +02:00
MarcoFalke
fa32af22b3 Replace LocaleIndependentAtoi with ToIntegral
No need for saturating behavior when the int is composed of 3 digits.
2023-08-08 16:04:22 +02:00
MarcoFalke
faab76c1c0 iwyu on torcontrol 2023-08-08 16:03:40 +02:00
MarcoFalke
fa0a60dd93 Remove unused boost signals2 from torcontrol 2023-08-08 14:39:14 +02:00
MarcoFalke
fa6dc57760 refactor: Enforce C-str fmt strings in WalletLogPrintf() 2023-08-08 10:55:11 +02:00
MarcoFalke
fa244f3321 doc: Fix bitcoin-unterminated-logprintf tidy comments
* Move module description from test to LogPrintfCheck
* Add test doc
* Remove unused comment, see https://github.com/bitcoin/bitcoin/pull/26296/files#r1279351539
2023-08-08 09:11:05 +02:00
fanquake
b565485c24 Merge bitcoin/bitcoin#28186: kernel: Prune leveldb headers
d8f1222ac5 refactor: Correct dbwrapper key naming (TheCharlatan)
be8f159ac5 build: Remove leveldb from BITCOIN_INCLUDES (TheCharlatan)
c95b37d641 refactor: Move CDBWrapper leveldb members to their own context struct (TheCharlatan)
c534a615e9 refactor: Split dbwrapper CDBWrapper::EstimateSize implementation (TheCharlatan)
586448888b refactor: Move HandleError to dbwrapper implementation (TheCharlatan)
dede0eef7a refactor: Split dbwrapper CDBWrapper::Exists implementation (TheCharlatan)
a5c2eb5748 refactor: Fix logging.h includes (TheCharlatan)
84058e0eed refactor: Split dbwrapper CDBWrapper::Read implementation (TheCharlatan)
e4af2408f2 refactor: Pimpl leveldb::Iterator for CDBIterator (TheCharlatan)
ef941ff128 refactor: Split dbwrapper CDBIterator::GetValue implementation (TheCharlatan)
b7a1ab5cb4 refactor: Split dbwrapper CDBIterator::GetKey implementation (TheCharlatan)
d7437908cd refactor: Split dbwrapper CDBIterator::Seek implementation (TheCharlatan)
ea8135de7e refactor: Pimpl leveldb::batch for CDBBatch (TheCharlatan)
b9870c920d refactor: Split dbwrapper CDBatch::Erase implementation (TheCharlatan)
532ee812a4 refactor: Split dbwrapper CDBBatch::Write implementation (TheCharlatan)
afc534df9a refactor: Wrap DestroyDB in dbwrapper helper (TheCharlatan)

Pull request description:

  Leveldb headers are currently included in the `dbwrapper.h` file and thus available to many of Bitcoin Core's source files. However, leveldb-specific functionality should be abstracted by the `dbwrapper` and does not need to be available to the rest of the code. Having leveldb included in a widely-used header such as `dbwrapper.h` bloats the entire project's header tree.

  The `dbwrapper` is a key component of the libbitcoinkernel library. Future users of this library would not want to contend with  having the leveldb headers exposed and potentially polluting their project's namespace.

  For these reasons, the leveldb headers are removed from the `dbwrapper` by moving leveldb-specific code to the implementation file and creating a [pimpl](https://en.cppreference.com/w/cpp/language/pimpl) where leveldb member variables are indispensable. As a final step, the leveldb include flags are removed from the `BITCOIN_INCLUDES` and moved to places where the dbwrapper is compiled.

  ---

  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587), and more specifically its stage 1 step 3 "Decouple most non-consensus headers from libbitcoinkernel".

ACKs for top commit:
  stickies-v:
    re-ACK d8f1222ac5
  MarcoFalke:
    ACK d8f1222ac5  🔠

Tree-SHA512: 0f58309be165af0162e648233451cd80fda88726fc10c0da7bfe4ec2ffa9afe63fbf7ffae9493698d3f39653b4ad870c372eee652ecc90ab1c29d86c387070f3
2023-08-07 22:31:46 +02:00
fanquake
064919e00d Merge bitcoin/bitcoin#28231: doc: remove Fedora libdb4-*-devel install docs
11a499eb4d doc: remove Fedora libdb4-*-devel install docs (fanquake)

Pull request description:

  These are no-longer installable on any recent Fedora (last working version was 32).
  Remove the install instructions, and consolidate this section to be the same as the
  Ubuntu & Debian BDB install instructions.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 11a499eb4d

Tree-SHA512: 11e3c92d6dcf475a6f5529a2e41dc9f79eeae8f8d3600087ce5ae083264f999782a2c04a4c4c70073e96d4053daa23037a344224197ee5f15a3d635172c201e2
2023-08-07 18:55:43 +02:00
MarcoFalke
fad0b67c21 ci: Use qemu-user through container engine 2023-08-07 17:36:14 +02:00
fanquake
624333455a Merge bitcoin/bitcoin#26296: ci: Integrate bitcoin-tidy clang-tidy plugin
1c976c691c tidy: Integrate bicoin-tidy clang-tidy plugin (fanquake)
7de23cceb8 refactor: fix unterminated LogPrintf()s (fanquake)
0a1029aa29 lint: remove  /* Continued */ markers from codebase (fanquake)
910007995d lint: remove lint-logs.py (fanquake)
d86a83d6b8 lint: drop DIR_IWYU global (fanquake)

Pull request description:

  Demo of integrating the [bitcoin-tidy](https://github.com/theuni/bitcoin-tidy-plugin), [clang-tidy plugin](https://clang.llvm.org/extra/clang-tidy/) written by theuni into our tidy CI job.

  The plugin currently has a single check, `bitcoin-unterminated-logprintf`. This would replace our current Python driven, `git-grep`-based, `.cpp` file only, lint-logs linter.

ACKs for top commit:
  TheCharlatan:
    ACK 1c976c691c
  theuni:
    ACK 1c976c691c
  MarcoFalke:
    re-ACK 1c976c691c  👠

Tree-SHA512: 725b45c70e431d48e6f276671e05c694e10b6047cae1a31906ac3ee9093bc8105fb226b36a5bac6709557526ca6007222112d66aecec05a574434edc4897e4b8
2023-08-07 17:14:07 +02:00
fanquake
97ba72117c Merge bitcoin/bitcoin#27401: tracepoints: Disables -Wgnu-zero-variadic-macro-arguments to compile without warnings
5197660e94 tracepoints: Disables `-Wgnu-zero-variadic-macro-arguments` to compile without warnings (Martin Leitner-Ankerl)

Pull request description:

  Fixes #26916 by disabling the warning `-Wgnu-zero-variadic-macro-arguments` when clang is used as the compiler.

  Also see the comments
  * Proposed changes in the bug  https://github.com/bitcoin/bitcoin/issues/26916#issuecomment-1480997053
  * Proposed changes when moving to a variadic maro: https://github.com/bitcoin/bitcoin/pull/26593#discussion_r1155488768

ACKs for top commit:
  hebasto:
    ACK 5197660e94, I've reconsidered my [comment](https://github.com/bitcoin/bitcoin/pull/27401#issuecomment-1507142439) and I think the current localized approach is optimal.
  fanquake:
    ACK 5197660e94 - checked that this fixes the warnings under Clang.

Tree-SHA512: c3dda3bcbb2540af6283ffff65885a9937bfdaaef3b00dc7d60b9f9740031d5c36ac9cb3d3d8756dbadce4812201a9754f5b8770df0d5e0d5ee690ba8a7135d2
2023-08-07 16:03:55 +02:00
fanquake
11a499eb4d doc: remove Fedora libdb4-*-devel install docs
These are no-longer installable on any recent Fedora (33+).
Remove the install instructions.
Fix the typo in the Ubuntu/Debian instructions.
2023-08-07 14:48:35 +02:00
Anthony Towns
fb02ba3c5f mempool_entry: improve struct packing 2023-08-07 20:24:33 +10:00
Sebastian Falbesoner
2ab7952bda test: add bip157 coverage for (start height > stop height) disconnect 2023-08-07 12:20:22 +02:00
Sebastian Falbesoner
63e90e1d3f test: check for specific disconnect reasons in p2p_blockfilters.py
This ensures that the disconnect happens for the expected reason and
also makes it easier to navigate between implementation and test code,
i.e. both the questions "do we have test coverage for this disconnect?"
(from an implementation reader's perspective) and "where is the code
handling this disconnect?" (from a test reader's perspective) can be
answered simply by grep-ping the corresponding debug message.

Can be easiest reviewed with `-w` (to ignore whitespace changes).
2023-08-07 12:20:16 +02:00
MarcoFalke
fa776e61cd Add importmempool RPC
test_importmempool_union contributed by glozow

Co-authored-by: glozow <gloriajzhao@gmail.com>
2023-08-07 11:33:34 +02:00
MarcoFalke
fa20d734a2 refactor: Add and use kernel::ImportMempoolOptions
This allows optional named arguments with default values.
2023-08-07 11:32:34 +02:00
MarcoFalke
fa8866990d doc: Clarify the getmempoolinfo.loaded RPC field documentation
Also, clarify the LoadMempool doxygen.
2023-08-07 11:32:29 +02:00
MarcoFalke
6888886cec Remove Chainstate::LoadMempool
The 3-line function is only called once outside of tests, so it is
clearer to inline it.
2023-08-07 10:59:15 +02:00
fanquake
be44332803 Merge bitcoin/bitcoin#28191: refactor: Remove unused MessageStartChars parameters from BlockManager methods
fa69e3a95c Remove unused MessageStartChars parameters from BlockManager methods (MarcoFalke)

Pull request description:

  Seems odd to expose these for mocking, when it is not needed.

  Fix this by removing the the unused parameters and use the already existing member field instead.

ACKs for top commit:
  Empact:
    utACK fa69e3a95c
  dergoegge:
    utACK fa69e3a95c

Tree-SHA512: 7814e9560abba8d9c0926bcffc70f92e502d22f543af43671248f6fcd1433f35238553c0f05123fde6d8e0f80261af0ab0500927548115153bd68d57fe2da746
2023-08-07 10:57:39 +02:00
fanquake
b7138252ac Merge bitcoin/bitcoin#27213: p2p: Diversify automatic outbound connections with respect to networks
1b52d16d07 p2p: network-specific management of outbound connections (Martin Zumsande)
65cff00cee test: Add test for outbound protection by network (Martin Zumsande)
034f61f83b p2p: Protect extra full outbound peers by network (Martin Zumsande)
654d9bc276 p2p: Introduce data struct to track connection counts by network (Amiti Uttarwar)

Pull request description:

  This is joint work with mzumsande.

  This is a proposal to diversify outbound connections with respect to reachable networks. The existing logic evaluates peers for connection based purely on the frequency of available addresses in `AddrMan`. This PR adds logic to automatically connect to alternate reachable networks and adds eviction logic that protects one existing connection to each network.

  For instance, if `AddrMan` is populated primarily with IPv4 and IPv6 addresses and only a handful of onion addresses, it is likely that we won't establish any automatic outbound connections to Tor, even if we're capable of doing so. For smaller networks like CJDNS, this is even more of an issue and often requires adding manual peers to ensure regularly being connected to the network.

  Connecting to multiple networks improves resistance to eclipse attacks for individual nodes. It also benefits the entire p2p network by increasing partition resistance and privacy in general.

  The automatic connections to alternate networks is done defensively, by first filling all outbound slots with random addresses (as in the status quo) and then adding additional peers from reachable networks the node is currently not connected to. This approach ensures that outbound slots are not left unfilled while attempting to connect to a network that may be unavailable due to a technical issue or misconfiguration that bitcoind cannot detect.

  Once an additional peer is added and we have one more outbound connection than we want, outbound eviction ensures that peers are protected if they are the only ones for their network.

  Manual connections are also taken into account: If a user already establishes manual connections to a trusted peer from a network, there is no longer a need to make extra efforts to ensure we also have an automatic connection to it (although this may of course happen by random selection).

ACKs for top commit:
  naumenkogs:
    ACK 1b52d16d07
  vasild:
    ACK 1b52d16d07

Tree-SHA512: 5616c038a5fbb868d4c46c5963cfd53e4599feee25db04b0e18da426d77d22e0994dc4e1da0b810f5b457f424ebbed3db1704f371aa6cad002b3565b20170ec0
2023-08-06 18:44:42 +02:00
TheCharlatan
d8f1222ac5 refactor: Correct dbwrapper key naming
The ss- prefix should connotate a DataStream variable. Now that these
variables are byte spans, drop the prefix.
2023-08-05 10:45:19 +02:00
TheCharlatan
be8f159ac5 build: Remove leveldb from BITCOIN_INCLUDES
Since leveldb is no longer in our header tree, move its include flags to
whereever dbwrapper.cpp is built.
2023-08-05 10:45:17 +02:00
TheCharlatan
c95b37d641 refactor: Move CDBWrapper leveldb members to their own context struct
The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:45:12 +02:00
TheCharlatan
c534a615e9 refactor: Split dbwrapper CDBWrapper::EstimateSize implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

Since CharCast is no longer needed in the header, move it to the
implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:43:01 +02:00
TheCharlatan
586448888b refactor: Move HandleError to dbwrapper implementation
Make it a static function in dbwrapper.cpp, since it is not used
elsewhere and when left in the header, would expose a leveldb type.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:59 +02:00
TheCharlatan
dede0eef7a refactor: Split dbwrapper CDBWrapper::Exists implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:58 +02:00
TheCharlatan
a5c2eb5748 refactor: Fix logging.h includes
These were uncovered as missing by the next commit.
2023-08-05 10:42:56 +02:00
TheCharlatan
84058e0eed refactor: Split dbwrapper CDBWrapper::Read implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:55 +02:00
TheCharlatan
e4af2408f2 refactor: Pimpl leveldb::Iterator for CDBIterator
Hide the leveldb::Iterator member variable with a pimpl in order not to
expose it directly in the header.

Also, move CDBWrapper::NewIterator to the dbwrapper implementation to
use the pimpl for CDBIterator initialziation.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:53 +02:00
TheCharlatan
ef941ff128 refactor: Split dbwrapper CDBIterator::GetValue implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:51 +02:00
TheCharlatan
b7a1ab5cb4 refactor: Split dbwrapper CDBIterator::GetKey implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:48 +02:00
TheCharlatan
d7437908cd refactor: Split dbwrapper CDBIterator::Seek implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:45 +02:00
TheCharlatan
ea8135de7e refactor: Pimpl leveldb::batch for CDBBatch
Hide the leveldb::WriteBatch member variable with a pimpl in order not
to expose it directly in the header.

Also move CDBBatch::Clear to the dbwrapper implementation to use the new
impl_batch.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:42:38 +02:00
TheCharlatan
b9870c920d refactor: Split dbwrapper CDBatch::Erase implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:27:53 +02:00
TheCharlatan
532ee812a4 refactor: Split dbwrapper CDBBatch::Write implementation
Keep the generic serialization in the header, while moving
leveldb-specifics to the implementation file.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-05 10:27:47 +02:00
fanquake
d096743150 Merge bitcoin/bitcoin#28213: scripted-diff: Specify Python major version explicitly on Windows
6a7686b446 scripted-diff: Specify Python major version explicitly on Windows (Hennadii Stepanov)

Pull request description:

  On Windows, it is the accepted practice to use `py.exe` launcher:
  - https://learn.microsoft.com/en-us/windows/python/faqs#what-is-py-exe-
  - https://docs.python.org/3/using/windows.html#python-launcher-for-windows

  One of its features is the correct handling of shebang lines like the one we use: `#!/usr/bin/env python3`.

  However, Windows OS app execution aliases might [interfere](https://learn.microsoft.com/en-us/windows/python/faqs#why-does-running-python-exe-open-the-microsoft-store-) with the launcher's behaviour. Such aliases are enabled on Windows 11 by default:

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/407837ec-e89a-4bc1-98b1-db983002065a)

  For example, on a fresh Windows 11 Pro installation with the Python installed from the [Chocolatey](https://community.chocolatey.org/packages/python/3.11.4) package manager, one will get the following error:
  ```
  >py -3 test\functional\rpc_signer.py
  2023-08-03T19:41:13.353000Z TestFramework (INFO): PRNG seed is: 2694758731106548661
  2023-08-03T19:41:13.353000Z TestFramework (INFO): Initializing test directory C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3
  2023-08-03T19:41:14.538000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\util.py", line 140, in try_rpc
      fun(*args, **kwds)
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\coverage.py", line 50, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\authproxy.py", line 129, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: RunCommandParseJSON error: process(py C:\Users\hebasto\bitcoin\test\functional\mocks\signer.py enumerate) returned 9009: Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
   (-1)

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\test_framework.py", line 131, in main
      self.run_test()
    File "C:\Users\hebasto\bitcoin\test\functional\rpc_signer.py", line 72, in run_test
      assert_raises_rpc_error(-1, 'fingerprint not found',
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\util.py", line 131, in assert_raises_rpc_error
      assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "C:\Users\hebasto\bitcoin\test\functional\test_framework\util.py", line 146, in try_rpc
      raise AssertionError(
  AssertionError: Expected substring not found in error message:
  substring: 'fingerprint not found'
  error message: 'RunCommandParseJSON error: process(py C:\Users\hebasto\bitcoin\test\functional\mocks\signer.py enumerate) returned 9009: Python was not found; run without arguments to install from the Microsoft Store, or disable this shortcut from Settings > Manage App Execution Aliases.
  '.
  2023-08-03T19:41:14.592000Z TestFramework (INFO): Stopping nodes
  2023-08-03T19:41:14.799000Z TestFramework (WARNING): Not cleaning up dir C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): Test failed. Test logging available at C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3/test_framework.log
  2023-08-03T19:41:14.799000Z TestFramework (ERROR):
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): Hint: Call C:\Users\hebasto\bitcoin\test\functional\combine_logs.py 'C:\Users\hebasto\AppData\Local\Temp\bitcoin_func_test_mldbzzw3' to consolidate all logs
  2023-08-03T19:41:14.799000Z TestFramework (ERROR):
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): If this failure happened unexpectedly or intermittently, please file a bug and provide a link or upload of the combined log.
  2023-08-03T19:41:14.799000Z TestFramework (ERROR): https://github.com/bitcoin/bitcoin/issues
  2023-08-03T19:41:14.799000Z TestFramework (ERROR):

  ```

  This PR resolves this issue by explicitly specifying the Python major version and makes testing of self-compiled binaries more straightforward.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 6a7686b446
  stickies-v:
    utACK 6a7686b446

Tree-SHA512: 5681141e222bc833c6250cb79fe3a1c8e02255eb2c86010bc0f8239afcdfed784ed7788c8579209d931bd357f58d5655cf33ffeb2f46b1879f37cdc30e7a7c91
2023-08-04 15:09:34 +02:00
fanquake
f138422d37 Merge bitcoin/bitcoin#28203: refactor: serialization simplifications
f054bd072a refactor: use "if constexpr" in std::vector's Unserialize() (Martin Leitner-Ankerl)
088caa68fb refactor: use "if constexpr" in std::vector's Serialize() (Martin Leitner-Ankerl)
0fafaca4d3 refactor: use "if constexpr" in prevector's Unserialize() (Martin Leitner-Ankerl)
c8839ec5cd refactor: use "if constexpr" in prevector's Serialize() (Martin Leitner-Ankerl)
1403d181c1 refactor: use fold expressions instead of recursive calls in UnserializeMany() (Martin Leitner-Ankerl)
bd08a008b4 refactor: use fold expressions instead of recursive calls in SerializeMany() (Martin Leitner-Ankerl)

Pull request description:

  This simplifies the serialization code a bit and should also make it a bit faster.

  * use fold expressions instead of recursive calls. This simplifies the code, makes it most likely faster because it reduces the number of function calls, and compiles faster because there are fewer template instantiations.

  * use `if constexpr` instead of unnecessarily creating a temporary object only to call the right overload. This is used for `std::vector` and `prevector` serialization.

ACKs for top commit:
  MarcoFalke:
    only change is to add a missing `&`. lgtm, re-ACK f054bd072a 📦
  jonatack:
    ACK f054bd072a
  sipa:
    utACK f054bd072a
  john-moffett:
    ACK f054bd072a

Tree-SHA512: 0417bf2d6be486c581732297945449211fc3481bac82964e27628b38ef55a47dfa58d730148aeaf1b19fa8eb1076489cc646ceebb178162a9afa59034601501d
2023-08-04 14:50:49 +02:00
Antoine Poinsot
c5f6b1db56 fuzz: coins_view: correct an incorrect assertion
It is incorrect to assert that `cache.HaveCoin()` will always be `true`
if `backend.HaveCoin()` is. The coin could well have been marked as
spent in the cache but not yet flushed, in which case `cache.HaveCoin()`
would return `false`.

Note this was never hit because `exists_using_have_coin_in_backend` is
currently never `true` (it's the default implementation of `CCoinsView`.
However this might change if we were to add a target where the backend
is a `CCoinsViewDB`.
2023-08-04 13:51:30 +02:00
Martin Zumsande
1b52d16d07 p2p: network-specific management of outbound connections
Diversify outbound connections with respect to
networks: Every ~5 minutes, try to add an extra connection
to a reachable network which we currently don't have a connection to.
This is done defensively - only try management with respect to networks
after all existing outbound slots are filled.
The resulting situation with an extra outbound peer will be handled
by the extra outbound eviction logic, which protects peers from
eviction if they are the only ones for their network.

Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-08-03 19:27:23 -06:00
Martin Zumsande
65cff00cee test: Add test for outbound protection by network
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-08-03 19:27:23 -06:00
Martin Zumsande
034f61f83b p2p: Protect extra full outbound peers by network
If a peer is the only one of its network, protect it from eviction.
This improves the diversity of outbound connections with respect to
reachable networks.

Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-08-03 19:27:23 -06:00
Amiti Uttarwar
7463d259e1 doc: Add release note 2023-08-03 13:16:38 -06:00
Amiti Uttarwar
654d9bc276 p2p: Introduce data struct to track connection counts by network
Connman uses this new map to keep a count of active OUTBOUND_FULL_RELAY and
MANUAL connections. Unused until next commit.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-08-03 12:46:24 -06:00
fanquake
a4ca497588 Merge bitcoin/bitcoin#28161: ci: Move ASan USDT to persistent_worker
fa474397b5 ci: Add missing linux-headers package to ASan task (MarcoFalke)
fabaa85c01 ci: Move ASan USDT to persistent_worker (MarcoFalke)

Pull request description:

  To run the USDT functional tests, the ASan task currently requires the container host to run the Ubuntu Lunar Linux kernel (or later). Cirrus CI is the only provider that allows to spin up full VMs with Ubuntu Lunar, however they will start to charge for all tasks (See slightly related discussion in https://github.com/bitcoin/bitcoin/issues/28098).

  Since it is cheaper and recommended by Cirrus CI to just run a persistent worker, do that.

  Also, using a persistent worker allows to make use of the docker image cache.

ACKs for top commit:
  hebasto:
    ACK fa474397b5, I have reviewed the code and it looks OK.

Tree-SHA512: afd084ab1b56cbc3fa44d4611aaa01ec21c1d80aedf1f5f1bc4b8b3d1bd08095e0c7fcea7a3e6ec4b6cd97d01e97ee86061eb84a5e2c7e7195ce02a186254900
2023-08-03 18:03:18 +01:00
fanquake
1c976c691c tidy: Integrate bicoin-tidy clang-tidy plugin
Enable `bitcoin-unterminated-logprintf`.

Co-authored-by: Cory Fields <cory-nospam-@coryfields.com>
2023-08-03 17:52:24 +01:00
fanquake
7de23cceb8 refactor: fix unterminated LogPrintf()s 2023-08-03 17:52:24 +01:00
fanquake
0a1029aa29 lint: remove /* Continued */ markers from codebase 2023-08-03 17:52:24 +01:00
fanquake
910007995d lint: remove lint-logs.py 2023-08-03 17:52:24 +01:00
fanquake
d86a83d6b8 lint: drop DIR_IWYU global 2023-08-03 17:52:24 +01:00
fanquake
da3816e4e8 Merge bitcoin/bitcoin#27832: doc: Clarify -datacarriersize, add -datacarriersize=2 tests
faafc35a77 doc: Clarify that -datacarriersize applies to the full raw scriptPubKey, not the data push (MarcoFalke)
55550e7fe7 test: Add -datacarriersize=2 tests (MarcoFalke)

Pull request description:

  Clarify with a test that `-datacarriersize` applies to the serialized size of the scriptPubKey, not the size of the pushed data. So for example,

  * `-datacarriersize=2` will reject a `raw(6a01aa)`, even though only one byte is pushed
  * `-datacarriersize=0` (or `-datacarrier=0`) will reject a `raw(6a)`, even though no byte is pushed
  * `-datacarriersize=0` (or `-datacarrier=0`) will reject a `raw(6a00)`, even though zero bytes are pushed

ACKs for top commit:
  ajtowns:
    ACK faafc35a77
  instagibbs:
    ACK faafc35a77

Tree-SHA512: f01ace02798f596ac2a02461e9f2a6ef91b3b37c976ea0b3bc860e2d3efb0ace0fd8b779dd18249cee7f84ebbe5fd21d8506afd3a15edadc00b843ff3b4aacc7
2023-08-03 17:46:43 +01:00
fanquake
61849f0464 Merge bitcoin/bitcoin#27918: fuzz: addrman, avoid ConsumeDeserializable when possible
025fda0a76 fuzz: addrman, avoid `ConsumeDeserializable` when possible (brunoerg)

Pull request description:

  Using specific functions like `ConsumeService`, `ConsumeAddress` and `ConsumeNetAddr` may be more effective than using `ConsumeDeserializable`. They always return some value while `ConsumeDeserializable` may return `std::nullopt`.

  E.g.: In this part of the code, if `op_net_addr` is `std::nullopt`,  we basically generated the addresses (if so) unnecessarily, because we won't be able to use them:
  ```cpp
  std::vector<CAddress> addresses;
  LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 10000) {
      const std::optional<CAddress> opt_address = ConsumeDeserializable<CAddress>(fuzzed_data_provider);
      if (!opt_address) {
          break;
      }
      addresses.push_back(*opt_address);
  }
  const std::optional<CNetAddr> opt_net_addr = ConsumeDeserializable<CNetAddr>(fuzzed_data_provider);
  if (opt_net_addr) {
      addr_man.Add(addresses, *opt_net_addr, std::chrono::seconds{ConsumeTime(fuzzed_data_provider, 0, 100000000)});
  }
  ```

  Also, if we are not calling `Add` effectively, it would also be affect other functions that may "depend" on it.

ACKs for top commit:
  dergoegge:
    Code review ACK 025fda0a76

Tree-SHA512: 02450bec0b084c15ba0cd1cbdfbac067c8fea4ccf27be0c86d54e020f029a6c749a16d8e0558f9d6d35a7ca9db8916f180c872f09474702b5591129e9be0d192
2023-08-03 17:32:46 +01:00
Hennadii Stepanov
6a7686b446 scripted-diff: Specify Python major version explicitly on Windows
Using `py.exe` launcher might by fragile depending on how Python was
installed. Specifying the Python version explicitly fixes test errors
like this:
```
RunCommandParseJSON error: process(py C:\Users\hebasto\bitcoin\test\functional\mocks\signer.py enumerate) returned 9009: Python was not found...
```

-BEGIN VERIFY SCRIPT-
sed -i 's|"py "|"py -3 "|g' $(git grep -l '"py "' -- test/functional)
-END VERIFY SCRIPT-
2023-08-03 14:57:45 +01:00
Anthony Towns
1a118062fb net_processing: Clean up INVENTORY_BROADCAST_MAX constants 2023-08-03 21:33:22 +10:00
glozow
6fa49937e4 test: Check tx from disconnected block is immediately requestable
Check that peers can immediately request txs from blocks that have been
reorged out and are now in our mempool.
2023-08-03 21:33:22 +10:00
Anthony Towns
e4ffabbffa net_processing: don't add txids to m_tx_inventory_known_filter
We no longer have m_recently_announced_invs, so there is no need to add
txids to m_tx_inventory_known_filter to dedupe that filter.
2023-08-03 20:56:41 +10:00
Anthony Towns
6ec1809d33 net_processing: drop m_recently_announced_invs bloom filter
Rather than using a bloom filter to track announced invs, simply allow
a peer to request any tx that entered the mempool prior to the last INV
message we sent them. This also obsoletes the UNCONDITIONAL_RELAY_DELAY.
2023-08-03 20:56:41 +10:00
glozow
7c66a4b610 Merge bitcoin/bitcoin#28059: refactor: Make more transaction size variables signed
92de74ef18 refactor: Make more transaction size variables signed (Hennadii Stepanov)

Pull request description:

  This PR is a continuation of https://github.com/bitcoin/bitcoin/pull/23962 and it:
  - gets rid of two static casts,
  - addresses https://github.com/bitcoin/bitcoin/pull/23962#issuecomment-1593289706,
  - is useful for https://github.com/bitcoin/bitcoin/pull/25972, see the failed ARM and multiprocess CI jobs.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 92de74ef18  🥔
  glozow:
    ACK 92de74ef18

Tree-SHA512: 84225961af8e08439664e75661b98fe86560217e891e5633a28316bf248d88df317a0c6b5a5f6b03feb2b0e0fd40a1f91dd4a85a0610d567470805bf47a84487
2023-08-03 11:45:51 +01:00
fanquake
532bd1f2e7 Merge bitcoin/bitcoin#28204: qa: Close SQLite connection properly
703b758e18 qa: Close SQLite connection properly (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up for https://github.com/bitcoin/bitcoin/pull/26462 that introduced a bug on Windows:
  ```
  >test\functional\wallet_descriptor.py
  ...
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
  ...
  ```

  From `sqlite3` Python module [docs](https://docs.python.org/3/library/sqlite3.html#how-to-use-the-connection-context-manager):
  > `Connection` object used as context manager only commits or rollbacks transactions, so the connection object should be closed manually.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 703b758e18
  theStack:
    utACK 703b758e18

Tree-SHA512: 35b1403507be06d1fc04e7e07ff56af5bcfe5013024671f0c1d9f3c41aacc4c777bcc6376ce82d720394e27450415d50ff5d5834ed388ec3f21503f86f1a42a5
2023-08-03 10:02:19 +01:00
Martin Leitner-Ankerl
f054bd072a refactor: use "if constexpr" in std::vector's Unserialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:35:22 +02:00
Martin Leitner-Ankerl
088caa68fb refactor: use "if constexpr" in std::vector's Serialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:34:42 +02:00
Martin Leitner-Ankerl
0fafaca4d3 refactor: use "if constexpr" in prevector's Unserialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:34:09 +02:00
Martin Leitner-Ankerl
c8839ec5cd refactor: use "if constexpr" in prevector's Serialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:32:25 +02:00
Martin Leitner-Ankerl
1403d181c1 refactor: use fold expressions instead of recursive calls in UnserializeMany()
Instead of recursively calling `UnserializeMany` and peeling off one
argument at a time, use a fold expression. This simplifies the code,
makes it most likely faster because it reduces the number of function
calls, and compiles faster because there are fewer template
instantiations.
2023-08-03 10:31:33 +02:00
Martin Leitner-Ankerl
bd08a008b4 refactor: use fold expressions instead of recursive calls in SerializeMany()
Instead of recursively calling `SerializeMany` and peeling off one
argument at a time, use a fold expression. This simplifies the code,
makes it most likely faster because it reduces the number of function
calls, and compiles faster because there are fewer template
instantiations.
2023-08-03 10:30:02 +02:00
Anthony Towns
a70beafdb2 validation: when adding txs due to a block reorg, allow immediate relay 2023-08-03 13:42:46 +10:00
Anthony Towns
1e9684f39f mempool_entry: add mempool entry sequence number 2023-08-03 13:42:45 +10:00
Hennadii Stepanov
703b758e18 qa: Close SQLite connection properly
Connection object used as context manager only commits or rollbacks
transactions, so the connection object should be closed manually.

Fixes the following error on Windows:
```
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: ...
```
2023-08-02 19:29:01 +01:00
brunoerg
f52cb02f70 doc: make it clear that node in addnode refers to the node's address 2023-08-02 10:28:28 -03:00
brunoerg
effd1efefb test: addnode with an invalid command should throw an error 2023-08-02 10:28:28 -03:00
brunoerg
56b27b8487 rpc, refactor: clean-up addnode
1. Use const where possible;
2. Rename variables to make them clearer;
3. There is no need to check whether `command` is null since it's a non-optional field.
2023-08-02 10:28:28 -03:00
fanquake
2fa60f0b68 Merge bitcoin/bitcoin#27452: test: cover addrv2 anchors by adding TorV3 to CAddress in messages.py
ba8ab4fc54 test: cover addrv2 support in anchors.dat with a TorV3 address (Matthew Zipkin)
b4bee4bbf4 test: add keep_alive option to socks5 proxy in test_framework (Matthew Zipkin)
5aaf988ccc test: cover TorV3 address in p2p_addrv2_relay (Matthew Zipkin)
80f64a3d40 test: add support for all networks in CAddress in messages.py (brunoerg)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27140

  Adds test coverage for https://github.com/bitcoin/bitcoin/pull/20516 to ensure that https://github.com/bitcoin/bitcoin/issues/20511 is completed and may be closed.

  This PR adds a test case to `feature_anchors.py` where an onion v3 address is set as a blocks-only relay peer and then shutdown, ensuring that the address is saved to anchors.dat in addrv2 format. We then ensure that bitcoin attempts to reconnect to that anchor address on restart.

  To compute the addrv2 serialization of the onion v3 address, I added logic to `CAddress` in `messages.py`. This new logic is covered by extending `p2p_addrv2_relay.py` to include an onion v3 address. Future work will be adding coverage for ipv6, torv2 and cjdns in these modules and also `feature_proxy.py`

  Also includes de/serialization unit test for `CAddress` in test framework.

ACKs for top commit:
  jonatack:
    ACK ba8ab4fc54
  brunoerg:
    crACK ba8ab4fc54
  willcl-ark:
    ACK ba8ab4fc54

Tree-SHA512: 7220e30d7cb975903d9ac575a7215a08e8f784c24c5741561affcbde12fb92cbf8704cb42e66494b788ba6ed4bb255fb0cc327e4f2190fae50c0ed9f336c0ff0
2023-08-02 12:57:30 +01:00
fanquake
2dea6c5ca0 Merge bitcoin/bitcoin#27572: test: dedup file hashing using sha256sum_file helper
2c0c6f4477 test: dedup file hashing using `sha256sum_file` helper (Sebastian Falbesoner)

Pull request description:

  Rather than doing the open/read/hash-steps manually in the affected functional tests, we can just use the `sha256sum_file` helper from the utils module instead.

  Note that for the tool_wallet.py test, the used hash is changed from sha1 to sha256, but as the only purpose is to detect file content changes, this doesn't matter. Also, the optimization using `memoryview` is overkill here, as the opened file has only a size of 24KiB and determining the hash via the helper doesn't take longer than a few hundred micro-seconds on my machine.

ACKs for top commit:
  kristapsk:
    ACK 2c0c6f4477

Tree-SHA512: 64fe21650b56a50e9f1a95f6ef27d88d8bfbb621e5be456f327ef8dbb5596b529d03976c200f3fd68da48cc427de9f257b403f3228e38cf1df918006674fac68
2023-08-02 11:53:35 +01:00
MarcoFalke
fae405556d scripted-diff: Rename CBlockTreeDB -> BlockTreeDB
-BEGIN VERIFY SCRIPT-
 sed -i 's|CBlockTreeDB|BlockTreeDB|g' $( git grep -l CBlockTreeDB )
-END VERIFY SCRIPT-
2023-08-02 07:49:32 +02:00
TheCharlatan
afc534df9a refactor: Wrap DestroyDB in dbwrapper helper
Wrap leveldb::DestroyDB in a helper function without exposing
leveldb-specifics.

Also, add missing optional include.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-01 22:14:15 +02:00
fanquake
1b5cbf71df Merge bitcoin/bitcoin#28144: test: fix intermittent failure in p2p_getaddr_caching.py
8a20f765cc test: drop duplicate getaddrs from p2p_getaddr_caching (Martin Zumsande)
feb0096139 test: fix intermittent failure in p2p_getaddr_caching (Martin Zumsande)

Pull request description:

  Fixes #28133

  In the consistency check, it's not enough to check that our address/port is unique, only the combination of source and target must be unique. Otherwise, the OS may reuse ports for connections to different `-addrbind`, which was happening in the failed runs.

  While at it, the second commit cleans up duplicate `getaddr` messages in `p2p_getaddr_caching.py` that do nothing but generate `Ignoring repeated "getaddr"` log messages (and cleans up some whitespace the python linter complains about).

ACKs for top commit:
  vasild:
    ACK 8a20f765cc

Tree-SHA512: eabe4727d7887f729074076f6333a918bba8cb34b8e3baaa83f167b441b0daa24f7c4824abcf03a9538a2ef14b2d826ff19aeffcb93a6c20735253a9678aac9c
2023-08-01 16:59:37 +01:00
fanquake
eb95368e0c Merge bitcoin/bitcoin#28166: test, rpc: invalid sighashtype coverage
90c8f79e94 test: remove redundant test values (Jon Atack)
c3f203387d test: use common assert_signing_completed_successfully helper (Jon Atack)
647d95aae9 test: add coverage for passing an invalid sighashtype (Jon Atack)

Pull request description:

  Add test coverage for passing an invalid sighashtype to RPCs signrawtransactionwithwallet, signrawtransactionwithkey, walletprocesspsbt, and descriptorprocesspsbt.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 90c8f79e94 🎥
  brunoerg:
    light crACK 90c8f79e94

Tree-SHA512: 3861658487edd0d9a377390acf3d43f45c3dd9e324894f0fdb8f5312b618301a55479b1f70c61daee0b20785e768ffde6fa5abe6af190b73c0d0e017f3976704
2023-08-01 16:56:15 +01:00
fanquake
ceda819886 Merge bitcoin/bitcoin#28060: util: Teach AutoFile how to XOR
fa633aa690 streams: Teach AutoFile how to XOR (MarcoFalke)
000019e158 Add AutoFile::detail_fread member function (MarcoFalke)
fa7724bc9d refactor: Modernize AutoFile (MarcoFalke)
fa8d227d58 doc: Remove comments that just repeat what the code does (MarcoFalke)
fafe2ca0ce refactor: Remove redundant file check from AutoFile shift operators (MarcoFalke)
9999a49b32 Extract util::Xor, Add key_offset option, Add bench (MarcoFalke)

Pull request description:

  This allows `AutoFile` to roll an XOR pattern while reading or writing to the underlying file.

  This is needed for https://github.com/bitcoin/bitcoin/pull/28052, but can also be used in any other place.

  Also, there are tests, so I've split this up from the larger pull to make review easier, hopefully.

ACKs for top commit:
  Crypt-iQ:
    crACK fa633aa
  willcl-ark:
    Lightly tested ACK fa633aa690
  jamesob:
    reACK fa633aa690 ([`jamesob/ackr/28060.4.MarcoFalke.util_add_xorfile`](https://github.com/jamesob/bitcoin/tree/ackr/28060.4.MarcoFalke.util_add_xorfile))

Tree-SHA512: 6d66cad0a089a096d3f95e4f2b28bce80b349d4b76f53d09dc9a0bea4fc1b7c0652724469c37971ba27728c7d46398a4c1d289c252af4c0f83bb2fcbc6f8e90b
2023-08-01 16:38:06 +01:00
MarcoFalke
fa474397b5 ci: Add missing linux-headers package to ASan task
Otherwise the task will throw in skip_if_no_python_bcc.

Also, adjust CI_CONTAINER_CAP for all needed permissions.
2023-08-01 17:33:36 +02:00
MarcoFalke
fabaa85c01 ci: Move ASan USDT to persistent_worker 2023-08-01 17:33:33 +02:00
MarcoFalke
faf63039cc Fixup style of moved code
Can be reviewed with --word-diff-regex=.
2023-08-01 15:27:51 +02:00
MarcoFalke
fa65111b99 move-only: Move CBlockTreeDB to node/blockstorage
The block index (CBlockTreeDB) is required to write and read blocks, so
move it to blockstorage. This allows to drop the txdb.h include from
`node/blockstorage.h`.

Can be reviewed with:

--color-moved=dimmed-zebra  --color-moved-ws=ignore-all-space
2023-08-01 15:27:33 +02:00
MarcoFalke
fa8685597e index: Drop legacy -txindex check 2023-08-01 15:27:23 +02:00
MarcoFalke
fa69148a0a scripted-diff: Use blocks_path where possible
-BEGIN VERIFY SCRIPT-
  sed -i 's|].chain_path, .blocks.|].blocks_path|g' $(git grep -l chain_path)
-END VERIFY SCRIPT-
2023-08-01 15:26:55 +02:00
fanquake
e5a9f2fb62 Merge bitcoin/bitcoin#28194: test: python E721 and flake8 updates
bee2d57a65 script: update flake8 to 6.1.0 (Jon Atack)
38c3fd846b test: python E721 updates (Jon Atack)

Pull request description:

  Update our functional tests per [E721](https://www.flake8rules.com/rules/E721.html) enforced by [flake8 6.1.0](https://flake8.pycqa.org/en/latest/release-notes/6.1.0.html), and update our CI lint task to use that release.  This makes the following linter output on current master with flake8 6.1.0 green.

  ```
  $ ./test/lint/lint-python.py ; ./test/lint/lint-spelling.py
  test/functional/p2p_invalid_locator.py:35:16: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:34:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:64:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  src/test/fuzz/descriptor_parse.cpp:88: occurences ==> occurrences
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/spelling.ignore-words.txt
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bee2d57a65

Tree-SHA512: f3788a543ca98e44eeeba1d06c32f1b11eec95d4aef068aa1b6b5c401261adfa3fb6c6d6c769f3fe6839d78e74a310d5c926867e7c367d6513a53d580fd376f3
2023-08-01 09:42:07 +01:00
fanquake
fadad10126 Merge bitcoin/bitcoin#28131: test: Add UBSan -fsanitize=integer suppressions for src/secp256k1 subtree
a7477744c5 Add UBSan `-fsanitize=integer` suppressions for `src/secp256k1` subtree (Hennadii Stepanov)

Pull request description:

  Required for https://github.com/bitcoin/bitcoin/pull/27991 (see the [comment](https://github.com/bitcoin/bitcoin/pull/27991#issuecomment-1611472816)) and for the upcoming CMake-based build system.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK a7477744c5

Tree-SHA512: 602fa3ad22d3b0f6981a51358677d2347c92c4c9f59626b497af10f7ba828ede37227d8ee717f089bf33bde5efe0854d53acc89bea46f0955e62b7f22c454d05
2023-08-01 09:40:36 +01:00
fanquake
b6c66f3091 Merge bitcoin/bitcoin#28003: doc: cleanup release process doc
bd5ae6c663 doc: misc changes in release-process (fanquake)
d99ba3cc01 doc: filter out merge-script from list of authors (josibake)
472d6f79b9 doc: remove generate changelog section from release-process.md (fanquake)
5555ecb80e doc: remove note to update bips.md version number (fanquake)

Pull request description:

  Collection of changes to to simplify / correct the release-process documentation.
  I think we could still simplify this further.
  For example, we could remove the guix building docs, and defer to `contrib/guix`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bd5ae6c663

Tree-SHA512: 44bd12dd4f09380daee03fa79f01f9f7e8e2d5cc2fd5ff8c9e85ab54b4ea2b8a35df30ce3bdecfb4ff056cf52822be771ed3419613f68929c122750b1f8c89f9
2023-08-01 09:40:07 +01:00
fanquake
8535802f1d Merge bitcoin/bitcoin#28070: test: Drop 22.x node from TxindexCompatibilityTest
fafe43cb6c scripted-diff: Use blocks_path where possible (MarcoFalke)
fa060c15fb test: Add blocks_path property to TestNode (MarcoFalke)
faba4fc325 test: Drop 22.x node from TxindexCompatibilityTest (MarcoFalke)
fa7f65b0f8 test: Use clean chain in MempoolCompatibilityTest (MarcoFalke)

Pull request description:

  The node in this test was never really needed, because the compatibility tests shouldn't be used to test previous releases. (The test suite of the previous release itself should be used for that). So remove it.

  Also, other test changes. (See individual commits)

ACKs for top commit:
  theStack:
    Code-review ACK fafe43cb6c

Tree-SHA512: 289f54695bf5310663ab38ebf1aa457f53d0aafae56e6657be0e75bf96b303165bad417dc7eaf4c40f0639aa92ce139e5bacb318a2eabab1f8e23a811cabe0cc
2023-08-01 09:38:49 +01:00
Ryan Ofsky
f4f1d6d230 Merge bitcoin/bitcoin#27746: Rework validation logic for assumeutxo
a733dd79e2 Remove unused function `reliesOnAssumedValid` (Suhas Daftuar)
d4a11abb19 Cache block index entry corresponding to assumeutxo snapshot base blockhash (Suhas Daftuar)
3556b85022 Move CheckBlockIndex() from Chainstate to ChainstateManager (Suhas Daftuar)
0ce805b632 Documentation improvements for assumeutxo (Ryan Ofsky)
768690b7ce Fix initialization of setBlockIndexCandidates when working with multiple chainstates (Suhas Daftuar)
d43a1f1a2f Tighten requirements for adding elements to setBlockIndexCandidates (Suhas Daftuar)
d0d40ea9a6 Move block-storage-related logic to ChainstateManager (Suhas Daftuar)
3cfc75366e test: Clear block index flags when testing snapshots (Suhas Daftuar)
272fbc370c Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot (Suhas Daftuar)
10c05710ce Add wrapper for adding entries to a chainstate's block index candidates (Suhas Daftuar)
471da5f6e7 Move block-arrival information / preciousblock counters to ChainstateManager (Suhas Daftuar)
1cfc887d00 Remove CChain dependency in node/blockstorage (Suhas Daftuar)
fe86a7cd48 Explicitly track maximum block height stored in undo files (Suhas Daftuar)

Pull request description:

  This PR proposes a clean up of the relationship between block storage and the chainstate objects, by moving the decision of whether to store a block on disk to something that is not chainstate-specific.  Philosophically, the decision of whether to store a block on disk is related to validation rules that do not require any UTXO state; for anti-DoS reasons we were using some chainstate-specific heuristics, and those have been reworked here to achieve the proposed separation.

  This PR also fixes a bug in how a chainstate's `setBlockIndexCandidates` was being initialized; it should always have all the HAVE_DATA block index entries that have more work than the chain tip.  During startup, we were not fully populating `setBlockIndexCandidates` in some scenarios involving multiple chainstates.

  Further, this PR establishes a concept that whenever we have 2 chainstates, that we always know the snapshotted chain's base block and the base block's hash must be an element of our block index. Given that, we can establish a new invariant that the background validation chainstate only needs to consider blocks leading to that snapshotted block entry as potential candidates for its tip. As a followup I would imagine that when writing net_processing logic to download blocks for the background chainstate, that we would use this concept to only download blocks towards the snapshotted entry as well.

ACKs for top commit:
  achow101:
    ACK a733dd79e2
  jamesob:
    reACK a733dd79e2 ([`jamesob/ackr/27746.5.sdaftuar.rework_validation_logic`](https://github.com/jamesob/bitcoin/tree/ackr/27746.5.sdaftuar.rework_validation_logic))
  Sjors:
    Code review ACK a733dd79e2.
  ryanofsky:
    Code review ACK a733dd79e2. Just suggested changes since the last review. There are various small things that could be followed up on, but I think this is ready for merge.

Tree-SHA512: 9ec17746f22b9c27082743ee581b8adceb2bd322fceafa507b428bdcc3ffb8b4c6601fc61cc7bb1161f890c3d38503e8b49474da7b5ab1b1f38bda7aa8668675
2023-07-31 16:18:20 -04:00
Jon Atack
bee2d57a65 script: update flake8 to 6.1.0
and touch up the spelling returned by lint-spelling.py
2023-07-31 12:14:06 -06:00
Jon Atack
38c3fd846b test: python E721 updates 2023-07-31 12:13:46 -06:00
MarcoFalke
fa69e3a95c Remove unused MessageStartChars parameters from BlockManager methods 2023-07-31 14:32:57 +02:00
fanquake
44b05bf3fe Merge bitcoin/bitcoin#28091: fuzz: use ConnmanTestMsg in connman
ecfe507e07 fuzz: use `ConnmanTestMsg` in `connman` (brunoerg)

Pull request description:

  Fixes #27980

  Using `ConnmanTestMsg` we can add nodes and be
  more effective fuzzing functions like `DisconnectNode`,
  `FindNode`, `GetNodeStats` and other ones.

ACKs for top commit:
  MarcoFalke:
    review ACK ecfe507e07
  dergoegge:
    utACK ecfe507e07

Tree-SHA512: 97c363b422809f2e9755c082d1102237347abfab72c7baca417bd8975f8a595ddf3a085f8353dbdb9f17fb98fbfe830792bfc0b83451168458018faf6c239efa
2023-07-31 11:43:39 +01:00
fanquake
e92013e178 Merge bitcoin/bitcoin#28188: ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE
79ceb161db ci: Use documented `CCACHE_MAXSIZE` instead of `CCACHE_SIZE` (Hennadii Stepanov)

Pull request description:

  This PR aims to:
  1) Remove our own `CCACHE_SIZE` environment variable that violates Ccache's `CCACHE_*` namespace.
  2) Introduce the `CCACHE_MAXSIZE` environment variable that is documented since [v3.3](https://ccache.dev/manual/3.3.html), which makes its usage consistent with other ones, such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 79ceb161db

Tree-SHA512: 13c8a3a3b2337191cab32a3e1c21c19dc465cd4fa9267b2999ca2fde5cca0c03811f520cd3940959aa57ea9cf0251db325df56a90fca85069d04ce2141ec7044
2023-07-31 11:19:55 +01:00
fanquake
80800361b1 Merge bitcoin/bitcoin#28181: qa, doc: Fix comment
ab498d913c qa, doc: Fix comment (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up for:
  - https://github.com/bitcoin/bitcoin/pull/9956
  - https://github.com/bitcoin/bitcoin/pull/10096

ACKs for top commit:
  RandyMcMillan:
    ACK ab498d913c

Tree-SHA512: 267ae52c961ee79e172f27cb1587282ac5cf7ec929a136db6feed3de4f319a74b462befd9b99711081db8a5c30a513f72366364068700418b273a31958b31b1d
2023-07-31 10:51:50 +01:00
Hennadii Stepanov
79ceb161db ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE
This change aims to:
1) Remove our own `CCACHE_SIZE` environment variable that violates
Ccache's `CCACHE_*` namespace.
2) Introduce the `CCACHE_MAXSIZE` environment variable that is
documented since v3.3, which makes its usage consistent with other ones,
such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.
2023-07-30 21:36:56 +01:00
fanquake
64440bb733 Merge bitcoin/bitcoin#28118: test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC
fabef121b0 refactor: Use EnsureAnyNodeContext (MarcoFalke)
fa1640617e test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC (MarcoFalke)

Pull request description:

  There should be no risk or downside in adding a call to `SyncWithValidationInterfaceQueue` here. In fact, it will make tests less brittle. For example,

  * If one sets the timeouts in `test/functional/feature_fee_estimation.py` to `0`, on `master` the test will fail and here it will pass.
  * It may avoid a rare (theoretic) intermittent issue in https://github.com/bitcoin/bitcoin/pull/28108/files#r1268966663

ACKs for top commit:
  TheCharlatan:
    ACK fabef121b0
  furszy:
    Code review ACK fabef121. Convinced by checking all current tests usages.

Tree-SHA512: c9e9a536a8721d1b3f267a66b40578b34948892301affdcad121ef8e02bf17037305d0dd53aa94b1b064753e66f9cfb31823b916b707a9d812627f502b818003
2023-07-30 11:21:40 +01:00
Hennadii Stepanov
ab498d913c qa, doc: Fix comment
This change is a follow-up for:
- https://github.com/bitcoin/bitcoin/pull/9956
- https://github.com/bitcoin/bitcoin/pull/10096
2023-07-30 11:10:59 +01:00
fanquake
4c57e53a61 Merge bitcoin/bitcoin#28138: ci: Keep system env vars as-is (bugfix)
fabc04a4d9 ci: Keep system env vars as-is (MarcoFalke)
fa8dcdcc8b ci: Set PATH inside the CI env (MarcoFalke)
fac229ab1f ci: Remove P_CI_DIR and --workdir (MarcoFalke)

Pull request description:

  This fixes a bug where the `$PATH` from the host is used inside the container. This will lead to bugs when the `$PATH` is different. For example on a host of Fedora 38, and a container of `debian:bullseye`.

  This can be tested with the `FILE_ENV=./ci/test/00_setup_env_arm.sh` CI env. On master:

  ```
  Error: crun: executable file `bash` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
  ```

  On this pull:

  (everything passes)

ACKs for top commit:
  TheCharlatan:
    lgtm ACK fabc04a4d9

Tree-SHA512: 51d2affed91624d0e5b43a3ee1e696313f934e05fde6b5a19e8ac4e6666c1e7b667a444bf3de3f77190bcd00e81efb7576196afb0f6b5e788d1a50e7ddb28d7e
2023-07-28 15:56:14 +01:00
fanquake
42a9110899 Merge bitcoin/bitcoin#28162: refactor: Revert additional univalue check in ParseSighashString
06199a995f refactor: Revert addition of univalue sighash string check (TheCharlatan)
0b47c16215 doc: Correct release-notes for sighashtype exceptions (TheCharlatan)

Pull request description:

  This is a follow up for #28113.

  The string type check is already done by the rpc parser / RPCHelpMan. Re-doing it is adding dead code. Instead, throwing an exception when the assumption does not hold is the already correct behavior. Pointed out in this [comment](https://github.com/bitcoin/bitcoin/pull/28113/files#r1274568557).

  Also correct the release note for the correct sighashtype exception change. There is no change in the handling of non-string sighashtype arugments. Pointed out in this [comment](https://github.com/bitcoin/bitcoin/pull/28113/files#r1274567555).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 06199a995f
  jonatack:
    Tested ACK 06199a995f
  stickies-v:
    ACK 06199a995f

Tree-SHA512: 3faa6b3d2247624c0973df8d79c09fbf1f90ffb99f1be484e359b528f485c31affea45976759bd206e4c81cbb54ebba5ad0ef4127d1deacbfe2a58153fcc94ee
2023-07-28 12:29:55 +01:00
fanquake
8aa77a77e6 Merge bitcoin/bitcoin#28168: refactor: Remove unused raw-pointer read helper from univalue
fa940f41ea Remove unused raw-pointer read helper from univalue (MarcoFalke)

Pull request description:

  The helpers are unused outside of tests and redundant with the existing `bool read(std::string_view raw);`.

  Fix both issues by removing them.

  Also, simplify the tests code by removing a `std::string` constructor where possible.

ACKs for top commit:
  stickies-v:
    utACK fa940f41ea
  TheCharlatan:
    tACK fa940f41ea

Tree-SHA512: 60c154c1046f01551335af79bf820a6104844f63e89977271b4336b3cd06ac3bab1379e18b7bc61d12bef7446029e91c16541ddecf9e88bc8bc897fc1f6ee2c8
2023-07-28 12:04:49 +01:00
Andrew Chow
cbf385058b Merge bitcoin/bitcoin#27888: Fuzz: a more efficient descriptor parsing target
131314b62e fuzz: increase coverage of the descriptor targets (Antoine Poinsot)
90a24741e7 fuzz: add a new, more efficient, descriptor parsing target (Antoine Poinsot)
d60229ede5 fuzz: make the parsed descriptor testing into a function (Antoine Poinsot)

Pull request description:

  The current descriptor parsing fuzz target requires valid public or private keys to be provided. This is unnecessary as we are only interested in fuzzing the descriptor parsing logic here (other targets are focused on fuzzing keys serializations). And it's pretty inefficient, especially for formats that need a checksum (`xpub`, `xprv`, WIF).

  This introduces a new target that mocks the keys as an index in a list of precomputed keys. Keys are represented as 2 hex characters in the descriptor. The key type (private, public, extended, ..) is deterministically based on this one-byte value. Keys are deterministically generated at target initialization. This is much more efficient and also largely reduces the size of the seeds.
  TL;DR: for instance instead of requiring the fuzzer to generate a `pk(xpub6DdBu7pBoyf7RjnUVhg8y6LFCfca2QAGJ39FcsgXM52Pg7eejUHLBJn4gNMey5dacyt4AjvKzdTQiuLfRdK8rSzyqZPJmNAcYZ9kVVEz4kj)` to parse a valid descriptor, it just needs to generate a `pk(03)`.

  Note we only mock the keys themselves, not the entire descriptor key expression. As we want to fuzz the real code that parses the rest of the key expression (origin, derivation paths, ..).

  This is a target i used for reviewing #17190 and #27255, and figured it was worth PR'ing on its own since the added complexity for mocking the keys is minimal and it could help prevent introducing bugs to the descriptor parsing logic much more efficiently.

ACKs for top commit:
  MarcoFalke:
    re-ACK 131314b62e  🐓
  achow101:
    ACK 131314b62e

Tree-SHA512: 485a8d6a0f31a3a132df94dc57f97bdd81583d63507510debaac6a41dbbb42fa83c704ff3f2bd0b78c8673c583157c9a3efd79410e5e79511859e1470e629118
2023-07-27 13:48:12 -04:00
Andrew Chow
272c4f3f10 Merge bitcoin/bitcoin#28148: refactor: consistently use ApplyArgsManOptions for PeerManager::Options
8a3159728a refactor: deduplicate ignores_incoming_txs (stickies-v)
5f41afcc46 refactor: set ignore_incoming_txs in ApplyArgsManOptions (stickies-v)

Pull request description:

  Consistently use `ApplyArgsManOptions` for `PeerManager::Options`, and initialize `PeerManager::Options` early to avoid reading `"-blocksonly"` twice. Suggested in https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1268400386 and also requested in https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1273346189.

  No behaviour change, but the [`TestingSetup`](e35fb7bc48/src/test/util/setup_common.cpp (L255-L256)) is now also able to access `"-blocksonly"`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 8a3159728a
  achow101:
    ACK 8a3159728a
  TheCharlatan:
    ACK 8a3159728a
  dergoegge:
    utACK 8a3159728a

Tree-SHA512: 6cb489d79ac2a87e8faedb76c96973ab3fc597426f274a90a3ffd0bc5fe3f2b25db9c7ec2e55a0c806c2bcbc0fdded6e228adb43d2cd81f14fd6552863847698
2023-07-27 11:33:47 -04:00
Jon Atack
90c8f79e94 test: remove redundant test values
as they are parsed identically.

See AmountFromValue() / ParseFixedPoint() / UniValue#getValStr()
2023-07-27 06:54:32 -06:00
Jon Atack
c3f203387d test: use common assert_signing_completed_successfully helper 2023-07-27 06:37:55 -06:00
Jon Atack
647d95aae9 test: add coverage for passing an invalid sighashtype
in RPCs descriptorprocesspbst, walletprocesspbst, signrawtransactionwithkey,
and signrawtransactionwithwallet.
2023-07-27 06:37:54 -06:00
MarcoFalke
fa940f41ea Remove unused raw-pointer read helper from univalue 2023-07-27 14:24:52 +02:00
fanquake
dfe2dc1d84 Merge bitcoin/bitcoin#28164: test: remove unused code in wallet_fundrawtransaction
108c6255bc test: remove unused `totalOut` code (brunoerg)
0fc3deee9a test: remove unecessary `decoderawtransaction` calls (brunoerg)

Pull request description:

  This PR removes in `wallet_fundrawtransaction`:
  - unecessary variables/calls to `decoderawtransaction`
  - unused `totalOut` variable and its related code (`totalOut` is used in some functions to test change, in other ones its value is not used)

ACKs for top commit:
  kevkevinpal:
    utACK [108c625](108c6255bc)
  MarcoFalke:
    lgtm ACK 108c6255bc

Tree-SHA512: c352524f3633146117534c79bd1a24523a7068f13a17d0b8a425cc3c85d62cb769a79ea60db8b075b137da2a0cc43142c43a23ca5af89246ff86cd824e37cf17
2023-07-27 12:25:18 +01:00
fanquake
04f66ce500 Merge bitcoin/bitcoin#28092: ci: document that -Wreturn-type has been fixed upstream (mingw-w64)
08eb5f1b67 ci: document that -Wreturn-type has been fixed upstream (Windows) (fanquake)

Pull request description:

  `noreturn` attributes have been added to the mingw-w64 headers, 1690994f51, meaning that [from 11.0.0 onwards](https://www.mingw-w64.org/changelog/), you'll no-longer see `-Wreturn-type` warnings when using `assert(false)`.

  Add -Wno-return-type to the Windows CI, where is should have been all
  along, and document why it's required. This can be dropped when we are
  using the fixed version of the mingw-w64 headers there.

  Drop the -Werror -Wno-return-type special case from our build system.
  -Wreturn-type is on by default in Clang and GCC.

  The new mingw-w64 header behaviour can be checked on Ubuntu mantic, [which ships with 11.0.0](https://packages.ubuntu.com/mantic/mingw-w64), using:
  ```cpp
  #include <cassert>

  int f(){ assert(false); }

  int main() {
  return 0;
  }
  ```

  On Mantic (with 11.0.0):
  ```bash
  x86_64-w64-mingw32-g++ test.cpp -Wreturn-type
  # nada
  ```

  On Lunar ([with 10.0.0](https://packages.ubuntu.com/lunar/mingw-w64)):
  ```bash
  x86_64-w64-mingw32-g++ test.cpp -Wreturn-type
  test.cpp: In function 'int f()':
  test.cpp:3:25: warning: no return statement in function returning non-void [-Wreturn-type]
      3 | int f(){ assert(false); }
        |                         ^
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 08eb5f1b67

Tree-SHA512: 9cd4310a96abd87bf8ceb37949ad0259fe4adee3367c604f4c4ad521a0cf09bdcc5dd305db19a0f45ce74c85178b0d739e2fca5ad0fc841ac935523a23b28a7f
2023-07-27 11:21:49 +01:00
TheCharlatan
06199a995f refactor: Revert addition of univalue sighash string check
This check is already done by the rpc parser. Re-doing it is adding dead
code. Instead, throwing an exception when the assumption does not hold
is the already correct behavior.

To make the fuzz test more accurate and not swallow all runtime errors,
add a check that the passed in UniValue sighash argument is either a
string or null.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2023-07-27 09:36:05 +02:00
TheCharlatan
0b47c16215 doc: Correct release-notes for sighashtype exceptions 2023-07-27 09:16:11 +02:00
Pieter Wuille
1c7582ead6 tests: add decryption test to bip324_tests 2023-07-26 17:09:27 -04:00
Pieter Wuille
990f0f8da9 Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers
Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-07-26 17:09:23 -04:00
Pieter Wuille
c91cedf281 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt 2023-07-26 17:00:22 -04:00
Pieter Wuille
af2b44c76e bench: add benchmark for FSChaCha20Poly1305
Add a benchmark for FSChaCha20Poly1305 encryption, so the overhead of key
generation and authentication can be observed for various message sizes.
2023-07-26 16:55:05 -04:00
Pieter Wuille
aa8cee9334 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305
This adds the FSChaCha20Poly1305 AEAD as specified in BIP324, a wrapper
around the ChaCha20Poly1305 AEAD (as specified in RFC8439 section 2.8) which
automatically rekeys every N messages, and automatically increments the nonce
every message.
2023-07-26 16:55:05 -04:00
Pieter Wuille
0fee267792 crypto: add FSChaCha20, a rekeying wrapper around ChaCha20
This adds the FSChaCha20 stream cipher as specified in BIP324, a
wrapper around the ChaCha20 stream cipher (specified in RFC8439
section 2.4) which automatically rekeys every N messages, and
manages the nonces used for encryption.

Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-07-26 16:55:05 -04:00
Pieter Wuille
9ff0768bdc crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439
This adds an implementation of the ChaCha20Poly1305 AEAD exactly matching
the version specified in RFC8439 section 2.8, including tests and official
test vectors.
2023-07-26 16:55:00 -04:00
Pieter Wuille
9fd085a1a4 crypto: remove outdated variant of ChaCha20Poly1305 AEAD
Remove the variant of ChaCha20Poly1305 AEAD that was previously added in
anticipation of BIP324 using it. BIP324 was updated to instead use rekeying
wrappers around otherwise unmodified versions of the ChaCha20 stream cipher
and the ChaCha20Poly1305 AEAD as specified in RFC8439.
2023-07-26 16:51:51 -04:00
brunoerg
108c6255bc test: remove unused totalOut code
In `wallet_fundrawtransaction`, `totalOut` is used in
some functions to check if the change is correct. In
other ones, it has been created but never used.
2023-07-26 16:05:37 -03:00
brunoerg
0fc3deee9a test: remove unecessary decoderawtransaction calls
In `wallet_fundrawtransaction`, there are some unecessary
variables/calls to `decoderawtransaction`. They have not
been used.
2023-07-26 15:57:43 -03:00
fanquake
f57e724a80 Merge bitcoin/bitcoin#28127: refactor: Remove C-style const-violating cast, Use reinterpret_cast
fa9108f85a refactor: Use reinterpret_cast where appropriate (MarcoFalke)
3333f950d4 refactor: Avoid casting away constness (MarcoFalke)
fa6394dd10 refactor: Remove unused C-style casts (MarcoFalke)

Pull request description:

  Using a C-style cast to convert pointer types to a byte-like pointer type has many issues:

  * It may accidentally and silently throw away `const`.
  * It forces reviewers to check that it doesn't accidentally throw away `const`.

  For example, on current master a `const char*` is cast to `unsigned char*` (without `const`), see d23fda0584/src/span.h (L273) . This can lead to UB, and the only reason why it didn't lead to UB is because the return type added back the `const`. (Obviously this would break if the return type was deduced via `auto`)

  Fix all issues by adding back the `const` and using `reinterpret_cast` where appropriate.

ACKs for top commit:
  darosior:
    re-utACK fa9108f85a
  hebasto:
    re-ACK fa9108f85a.
  john-moffett:
    ACK fa9108f85a

Tree-SHA512: 87f6e4b574f9bd96d4e0f2a0631fd0a9dc6096e5d4f1b95042fe9f197afc2fe9a24e333aeb34fed11feefcdb184a238fe1ea5aff10d580bb18d76bfe48b76a10
2023-07-26 16:03:39 +01:00
fanquake
f033a981ed Merge bitcoin/bitcoin#28139: test: create wallet specific for test_locked_wallet case
c648bdbda2 test: create wallet specific for test_locked_wallet case (furszy)

Pull request description:

  Coming from https://github.com/bitcoin/bitcoin/pull/28089#discussion_r1265478128.

  Several test cases are relying on the node1 default wallet, which thanks to 'test_locked_wallet' is encrypted.
  And can be only accessed within a specific timeframe (100ms), a duration internally set by the same test.

  This situation introduces a potential race condition, where other tests must complete their operations within
  the specified 100ms window to pass (otherwise the wallet gets re-locked and they fail).

  This can be seen running the test in valgrind (https://github.com/bitcoin/bitcoin/pull/28089), where other test cases fail due the wallet re-locking
  itself after the 100ms.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK c648bdbda2
  ishaanam:
    utACK c648bdbda2

Tree-SHA512: 01cde5a4a0cb3405adb9ea3c1f73841f3fa237d1162268ed06f0d49ca38541006b423a029e0b5e5955e1aa7e018c4600d894e555a68cf17ff60a4b8be58f4aa9
2023-07-26 10:19:56 +01:00
fanquake
c2ff87e1fa Merge bitcoin/bitcoin#28150: test: Avoid intermittent issues due to async events in validationinterface_tests
faca9a3d5a test: Avoid intermittent issues due to async events in validationinterface_tests (MarcoFalke)

Pull request description:

  Currently the tests have many issues:

  * They setup the genesis block, even though it is not needed
  * They queue an async `UpdatedBlockTip` even, which causes intermittent issues: https://github.com/bitcoin/bitcoin/issues/28146#issuecomment-1650064645

  Fix all issues by trimming down the setup to just `ChainTestingSetup`.

ACKs for top commit:
  Crypt-iQ:
    tACK faca9a3d5a

Tree-SHA512: 4449040330f89bbaf5ce5b2052417c160b451c373987fdf1069596c07834ed81f0aea1506d53c7d2cd21062b27332d30679285dae194b272fd0cb9ce5ded32cf
2023-07-26 09:58:33 +01:00
fanquake
4517e2f4d4 Merge bitcoin/bitcoin#28147: suppressions: note that type:ClassName::MethodName should be used
d0c6cc4abe suppressions: note that 'type:ClassName::MethodName' should be used (fanquake)

Pull request description:

  Now that the symbolizer is back in play, suppressions can once-again be targeted to functions, rather than file-wide.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d0c6cc4abe
  hebasto:
    ACK d0c6cc4abe

Tree-SHA512: fb65398eae18a6ebc5f8414275c568cf2664ab5357c2b3160f3bf285b67bc3af788225c5dba3c824c0e098627789450bec775375f52529d71c6ef700a9632d65
2023-07-26 09:42:50 +01:00
fanquake
8fba5dfc10 Merge bitcoin/bitcoin#27529: test: fix feature_addrman.py on big-endian systems
53c990ad34 test: fix `feature_addrman.py` on big-endian systems (Sebastian Falbesoner)

Pull request description:

  The test `feature_addrman.py` currently serializes the addrdb without specifying endianness for `int`s, so the machine's native byte order is used (see https://docs.python.org/3/library/struct.html#byte-order-size-and-alignment) and the generated `peers.dat` would be invalid on big-endian systems (our internal (de)serializers always use little-endian, see `ser_{read,write}data32`). Fix this by explicitly specifying little-endian serialization via the `<` character in `struct.pack(...)`.

  This is not detected by CI as we unfortunately don't run functional tests on big-endian systems there (I think we definitely should!).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 53c990ad34   🔚

Tree-SHA512: 513af6f1f785a713e7a8ef3a57fcd3fe2520a7d537f63a9c8e1f4bdea4c2f605fd4c35001623d6b13458883dbc256f24943684ab8f224055c22bf8d8eeee5fe2
2023-07-26 09:42:20 +01:00
fanquake
95d523fabb Merge bitcoin/bitcoin#28145: valgrind: add suppression for bug 472219
50f7214e09 valgrind: add suppression for bug 472219 (fanquake)

Pull request description:

  Now that https://bugs.kde.org/show_bug.cgi?id=472219 has been fixed upstream in:

  https://sourceware.org/git/?p=valgrind.git;a=commit;h=6ce0979884a8f246c80a098333ceef1a7b7f694d

  Add a supression to ignore the bug until we are using a fixed version of Valgrind.

  Related to #28072.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 50f7214e09

Tree-SHA512: 1030f3709195250350fd9c558420a9b1773fb54fdb323e0452a46eeb69ec6d60b5df50bde617c12d917e16dde07db64dee1b0101ddd4eda6161261fc7f6d4474
2023-07-26 09:36:21 +01:00
fanquake
54fe963a53 Merge bitcoin/bitcoin#28035: test: Ignore UTF-8 errors in assert_debug_log
fa3d72960b lint: Ignore check_fileopens failure on **kwargs (MarcoFalke)
fa6bb85cd2 test: Ignore UTF-8 errors in assert_debug_log (MarcoFalke)
fa63326fbc test: Fix debug_log_size helper (MarcoFalke)

Pull request description:

  Fix two bugs, see commit messages.

ACKs for top commit:
  theStack:
    utACK fa3d72960b

Tree-SHA512: 4a29bdf954bf62bb7676c2a41b03ad017bc86d535b2bd912c96bd41d1621beb06d840b53c211480ad51974e8b293bbae620060d2528d269159f32c0b44e47712
2023-07-26 09:35:51 +01:00
Andrew Chow
32c15237b6 Merge bitcoin/bitcoin#27930: util: Don't derive secure_allocator from std::allocator
07c59eda00 Don't derive secure_allocator from std::allocator (Casey Carter)

Pull request description:

  Giving the C++ Standard Committee control of the public interface of your type means they will break it. C++23 adds a new `allocate_at_least` member to `std::allocator`. Very bad things happen when, say, `std::vector` uses `allocate_at_least` from `secure_allocator`'s base to allocate memory which it then tries to free with `secure_allocator::deallocate`.

  (Discovered by microsoft/STL#3712, which will be reverted by microsoft/STL#3819 before it ships.)

ACKs for top commit:
  jonatack:
    re-ACK 07c59eda00 no change since my previous ACK apart from squashing the commits
  achow101:
    ACK 07c59eda00
  john-moffett:
    ACK 07c59eda00 Reviewed and tested. Performance appears unaffected in my environment.

Tree-SHA512: 23606c40414d325f5605a9244d4dd50907fdf5f2fbf70f336accb3a2cb98baa8acd2972f46eab1b7fdec1d28a843a96b06083cd2d09791cda7c90ee218e5bbd5
2023-07-25 18:54:29 -04:00
Andrew Chow
1ed8a0f8d2 Merge bitcoin/bitcoin#28113: kernel: Remove UniValue from kernel library
6960c81cbf kernel: Remove Univalue from kernel library (TheCharlatan)
10eb3a9faa kernel: Split ParseSighashString (TheCharlatan)

Pull request description:

  Besides the build system changes, this is a mostly move-only change for moving the few UniValue-related functions out of kernel files.

  UniValue is not required by any of the kernel components and a JSON library should not need to be part of a consensus library.

ACKs for top commit:
  achow101:
    ACK 6960c81cbf
  theuni:
    Re-ACK 6960c81cbf
  stickies-v:
    re-ACK 6960c81cbf

Tree-SHA512: d92e4cb4e12134c94b517751bd746d39f9b8da528ec3a1c94aaedcce93274a3bae9277832e8a7c0243c13df0397ca70ae7bbb24ede200018c569f8d81103c1da
2023-07-25 18:13:16 -04:00
stickies-v
547fa52443 net processing: clamp -blockreconstructionextratxn to uint32_t bounds
Also changes max_extra_txs into a uint32_t to avoid platform-specific
behaviour
2023-07-25 21:51:20 +01:00
stickies-v
e451d1e3c6 net processing: clamp -maxorphantx to uint32_t bounds 2023-07-25 21:50:37 +01:00
stickies-v
aa89e04e07 doc: document PeerManager::Options members 2023-07-25 21:49:40 +01:00
Sebastian Falbesoner
83d7cfd542 test: refactor: deduplicate segwitv0 ECDSA signing for tx inputs
Follow-up for #28025.
2023-07-25 22:15:56 +02:00
TheCharlatan
6960c81cbf kernel: Remove Univalue from kernel library
It is not required by any of the kernel components.
A JSON library should not need to be part of a consensus library.
2023-07-25 17:40:07 +02:00
TheCharlatan
10eb3a9faa kernel: Split ParseSighashString
This split is done in preparation for the next commit where the
dependency on UniValue in the kernel library is removed.
2023-07-25 17:40:02 +02:00
MarcoFalke
faca9a3d5a test: Avoid intermittent issues due to async events in validationinterface_tests 2023-07-25 17:32:16 +02:00
Martin Zumsande
8a20f765cc test: drop duplicate getaddrs from p2p_getaddr_caching
python p2p instances will automatically send a getaddr msg after
connecting, the explicit message was a duplicate that was being ignored.
2023-07-25 10:34:29 -04:00
Martin Zumsande
feb0096139 test: fix intermittent failure in p2p_getaddr_caching
Only the combined addr:port of source and destination
must be unique. If the destination is different, the same addr:port
for the source may be used by the OS.
2023-07-25 10:34:29 -04:00
MarcoFalke
fafe43cb6c scripted-diff: Use blocks_path where possible
-BEGIN VERIFY SCRIPT-
  sed -i 's|].chain_path / "blocks"|].blocks_path|g' $(git grep -l chain_path)
-END VERIFY SCRIPT-
2023-07-25 15:51:36 +02:00
MarcoFalke
fa060c15fb test: Add blocks_path property to TestNode 2023-07-25 15:51:22 +02:00
MarcoFalke
faba4fc325 test: Drop 22.x node from TxindexCompatibilityTest
* The node was only used to migrate the legacy txindex. But now that it
  is known to be working and that 22.x is EOL, it can be dropped.
* Also, fix a typo to properly check the txindex of node [1], not [2].
2023-07-25 15:51:19 +02:00
stickies-v
8a3159728a refactor: deduplicate ignores_incoming_txs
Initialize PeerManager::Options early to avoid reading -blocksonly twice.
2023-07-25 14:34:15 +01:00
stickies-v
5f41afcc46 refactor: set ignore_incoming_txs in ApplyArgsManOptions
Refactor to consistently use ApplyArgsManOptions to set all PeerManager::Options,
including ignore_incoming_txs.
2023-07-25 14:34:06 +01:00
MarcoFalke
fa7f65b0f8 test: Use clean chain in MempoolCompatibilityTest
The test creates enough blocks itself, so there is no need to have more.

Also, remove os import.
2023-07-25 15:24:04 +02:00
fanquake
bd5ae6c663 doc: misc changes in release-process
Nobody is pushing direct to guix.sigs, nor should they, as that
bypasses CI.
Use a newer example for the testing issue.
Don't duplicate the bitcoincore.org doc instructions.
2023-07-25 13:34:20 +01:00
fanquake
d0c6cc4abe suppressions: note that 'type:ClassName::MethodName' should be used
Now that the symbolizer is back in play, suppressions can once-again be
targeted to functions, rather than file-wide.
2023-07-25 13:25:55 +01:00
josibake
d99ba3cc01 doc: filter out merge-script from list of authors 2023-07-25 11:54:02 +01:00
fanquake
472d6f79b9 doc: remove generate changelog section from release-process.md
We haven't done this since 22.0. It's not clear why dumping out a
version of git log into the release-notes is that useful.
2023-07-25 11:54:02 +01:00
fanquake
5555ecb80e doc: remove note to update bips.md version number
It has been removed from the file.
2023-07-25 11:54:02 +01:00
fanquake
e35fb7bc48 Merge bitcoin/bitcoin#28124: fuzz: Re-enable symbolize=1 in ASAN_OPTIONS
faa8c1be26 fuzz: Re-enable symbolize=1 in ASAN_OPTIONS (MarcoFalke)

Pull request description:

  Looks like this fixed itself somehow and is no longer reproducible?

ACKs for top commit:
  fanquake:
    ACK faa8c1be26

Tree-SHA512: 67d2d6349cc7485f32bebabc18869ab101ae66a778a40ff9ddb037980997e600d7c6d1e0a17a011fa2a4ba07c73594b087dd781248cb8351f2688bc4cf6e587d
2023-07-25 10:47:06 +01:00
fanquake
c97270d722 Merge bitcoin/bitcoin#27499: net processing, refactor: Decouple PeerManager from gArgs
23c7b51ddd [net processing] Move -capturemessages to PeerManager::Options (dergoegge)
bd59bda26b [net processing] Move -blockreconstructionextratxn to PeerManager::Options (dergoegge)
567c4e0b6a [net processing] Move -maxorphantx to PeerManager::Options (dergoegge)
fa9e6d80d1 [net processing] Move -txreconciliation to PeerManager::Options (dergoegge)
4cfb7b925f [net processing] Use ignore_incoming_txs from m_opts (dergoegge)
8b87725921 [net processing] Introduce PeerManager options (dergoegge)

Pull request description:

  This PR decouples `PeerManager` from our global args manager by introducing `PeerManager::Options`.

ACKs for top commit:
  stickies-v:
    re-ACK 23c7b51ddd
  TheCharlatan:
    ACK 23c7b51ddd

Tree-SHA512: cd807b36ec018010e11935d3539fa7ed5015fdfb531d13a042a65b54ee8533a35a919a6a6c5fa293b5cba76000e9403c64dfd790fb9c649b7838544929b1fee8
2023-07-25 10:42:20 +01:00
fanquake
50f7214e09 valgrind: add suppression for bug 472219
Now that https://bugs.kde.org/show_bug.cgi?id=472219 has been fixed
upstream in:

https://sourceware.org/git/?p=valgrind.git;a=commit;h=6ce0979884a8f246c80a098333ceef1a7b7f694d

Add a supression to ignore the bug until we are using a fixed version of
Valgrind.

Related to #28072.
2023-07-25 10:23:18 +01:00
Casey Carter
07c59eda00 Don't derive secure_allocator from std::allocator
Affects both secure_allocator and zero_after_free_allocator.

Giving the C++ Standard Committee control of the public interface of your type means they will break it. C++23 adds a new `allocate_at_least` member to `std::allocator`. Very bad things happen when, say, `std::vector` uses `allocate_at_least` from `secure_allocator`'s base to allocate memory which it then tries to free with `secure_allocator::deallocate`.

Drive-by: Aggressively remove facilities unnecessary since C++11 from both allocators to keep things simple.
2023-07-24 22:33:40 -07:00
Suhas Daftuar
a733dd79e2 Remove unused function reliesOnAssumedValid 2023-07-24 16:27:04 -04:00
Suhas Daftuar
d4a11abb19 Cache block index entry corresponding to assumeutxo snapshot base blockhash
This is to (a) avoid repeated lookups into the block index for an entry that
should never change and (b) emphasize that the snapshot base should always
exist when set and not change during the runtime of the program.

Thanks to Russ Yanofsky for suggesting this approach.
2023-07-24 16:23:38 -04:00
Suhas Daftuar
3556b85022 Move CheckBlockIndex() from Chainstate to ChainstateManager
Also rewrite CheckBlockIndex() to perform tests on all chainstates.

This increases sanity-check coverage, as any place in our code where we were
invoke CheckBlockIndex() on a single chainstate will now invoke the sanity
checks on all chainstates.

This change also tightens up the checks on setBlockIndexCandidates and
mapBlocksUnlinked, to more precisely match what we aim for even in the presence
of assumed-valid blocks.
2023-07-24 16:23:38 -04:00
Ryan Ofsky
0ce805b632 Documentation improvements for assumeutxo 2023-07-24 16:23:38 -04:00
Suhas Daftuar
768690b7ce Fix initialization of setBlockIndexCandidates when working with multiple chainstates
When using assumeutxo and multiple chainstates are active, the background
chainstate should consider all HAVE_DATA blocks that are ancestors of the
snapshotted block and that have more work than the tip as potential candidates.
2023-07-24 16:23:38 -04:00
Suhas Daftuar
d43a1f1a2f Tighten requirements for adding elements to setBlockIndexCandidates
When using assumeutxo, we only need the background chainstate to consider
blocks that are on the chain leading to the snapshotted block.

Note that this introduces the new invariant that we can only have an assumeutxo
snapshot where the snapshotted blockhash is in our block index. Unknown block
hashes that are somehow passed in will cause assertion failures when processing
new blocks.

Includes test fixes and improvements by Andrew Chow and Fabian Jahr.
2023-07-24 16:23:38 -04:00
dergoegge
23c7b51ddd [net processing] Move -capturemessages to PeerManager::Options 2023-07-24 18:35:30 +02:00
dergoegge
bd59bda26b [net processing] Move -blockreconstructionextratxn to PeerManager::Options 2023-07-24 18:35:30 +02:00
dergoegge
567c4e0b6a [net processing] Move -maxorphantx to PeerManager::Options 2023-07-24 18:35:30 +02:00
dergoegge
fa9e6d80d1 [net processing] Move -txreconciliation to PeerManager::Options 2023-07-24 18:35:28 +02:00
dergoegge
4cfb7b925f [net processing] Use ignore_incoming_txs from m_opts 2023-07-24 18:31:16 +02:00
dergoegge
8b87725921 [net processing] Introduce PeerManager options 2023-07-24 18:30:59 +02:00
MarcoFalke
fabc04a4d9 ci: Keep system env vars as-is 2023-07-24 16:14:24 +02:00
MarcoFalke
fa8dcdcc8b ci: Set PATH inside the CI env
This is needed for the next commit.

This also requires dropping CI_RETRY from the docker build step, which
is fine, because CI_RETRY should be called inside the build script, not
outside.

Also, fix a doc typo.
2023-07-24 16:14:24 +02:00
MarcoFalke
fac229ab1f ci: Remove P_CI_DIR and --workdir
The --workdir setting to the docker run command is not needed. And
P_CI_DIR/PWD is equal to BASE_ROOT_DIR, so just use that directly.
2023-07-24 16:10:47 +02:00
furszy
c648bdbda2 test: create wallet specific for test_locked_wallet case
Other tests are also relying on the node1 default wallet,
which thanks to 'test_locked_wallet' is encrypted.
And can only be accessed within a specific timeframe (100ms)
set internally by the same test.

This make other tests susceptible to races. They can only
perform their operations successfully within the specified
time.

This can be seen running the test in valgrind, where other
test cases fail due the wallet re-locking itself after the
100ms.
2023-07-24 11:03:05 -03:00
MarcoFalke
fa9108f85a refactor: Use reinterpret_cast where appropriate
Also, wrap reinterpret_cast into a CharCast to ensure it is only called
on byte pointers.
2023-07-24 15:32:35 +02:00
MarcoFalke
3333f950d4 refactor: Avoid casting away constness
Seems confusing and brittle to remove const and then add it back in the
return type.
2023-07-24 15:32:27 +02:00
MarcoFalke
fa6394dd10 refactor: Remove unused C-style casts 2023-07-24 15:32:00 +02:00
Hennadii Stepanov
a7477744c5 Add UBSan -fsanitize=integer suppressions for src/secp256k1 subtree 2023-07-23 14:56:51 +01:00
brunoerg
ecfe507e07 fuzz: use ConnmanTestMsg in connman
Using `ConnmanTestMsg` we can add nodes and be
more effective fuzzing functions like `DisconnectNode`,
`FindNode`, `GetNodeStats` and other ones.
2023-07-22 13:42:17 -03:00
Hennadii Stepanov
92de74ef18 refactor: Make more transaction size variables signed
This change gets rid of `static_cast`s and compiler warnings.
2023-07-22 07:46:49 +01:00
MarcoFalke
faa8c1be26 fuzz: Re-enable symbolize=1 in ASAN_OPTIONS 2023-07-22 08:26:34 +02:00
Luke Dashjr
5e3e83b005 RPC/Mining: Document template_request better for getblocktemplate 2023-07-22 01:29:11 +00:00
Luke Dashjr
de319c6175 RPC/rpcdoccheck: Error if a oneline_description has a quote for a non-string 2023-07-22 01:03:56 +00:00
Luke Dashjr
7c61e9df90 Bugfix: RPC: Remove quotes from non-string oneline descriptions 2023-07-22 01:03:18 +00:00
Antoine Poinsot
131314b62e fuzz: increase coverage of the descriptor targets
Once a descriptor is successfully parsed, execute more of its methods.
There is probably still room for improvements by checking for some
invariants, but this is a low hanging fruit that significantly increases
the code coverage of these targets.
2023-07-21 19:14:36 +02:00
Antoine Poinsot
90a24741e7 fuzz: add a new, more efficient, descriptor parsing target
This new target focuses on fuzzing the actual descriptor parsing logic
by not requiring the fuzzer to produce valid keys (nor a valid checksum
for that matter).
This should make it much more efficient to find bugs we could introduce
moving forward.

Using a character as a marker (here '%') to be able to search and
replace in the string without having to mock the actual descriptor
parsing logic was an insight from Pieter Wuille.
2023-07-21 19:14:30 +02:00
Suhas Daftuar
d0d40ea9a6 Move block-storage-related logic to ChainstateManager
Separate the notion of which blocks are stored on disk, and what data is in our
block index, from what tip a chainstate might be able to get to. We can use
chainstate-agnostic data to determine when to store a block on disk (primarily,
an anti-DoS set of criteria) and let the chainstates figure out for themselves
when a block is of interest for being a candidate tip.

Note: some of the invariants in CheckBlockIndex are modified, but more work is
needed (ie to move CheckBlockIndex to ChainstateManager, as most of what
CheckBlockIndex is doing is checking the consistency of the block index, which
is outside of Chainstate).
2023-07-21 10:09:44 -04:00
MarcoFalke
fabef121b0 refactor: Use EnsureAnyNodeContext
node_context is never null, but if it was, it would lead to a nullptr
dereference in node_context->scheduler. Just use EnsureAnyNodeContext
everywhere for more robust, consistent, and correct code.
2023-07-21 15:05:07 +02:00
MarcoFalke
fa1640617e test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC
This makes existing tests less brittle, see
https://github.com/bitcoin/bitcoin/pull/28108/files#r1268966663
2023-07-21 14:44:30 +02:00
Antoine Poinsot
d60229ede5 fuzz: make the parsed descriptor testing into a function
We'll be reusing it in the new target.
2023-07-21 10:40:13 +02:00
fanquake
d23fda0584 Merge bitcoin/bitcoin#28103: test: Add missing set -ex to ci/lint/06_script.sh
ffff4b5dc5 lint: Add missing `set -ex` to ci/lint/06_script.sh (MarcoFalke)
fadc5232f4 doc: Add doc comment to ci/test_imagefile (MarcoFalke)

Pull request description:

  Requested in https://github.com/bitcoin/bitcoin/pull/28083#pullrequestreview-1535304219.

  Also, one doc commit.

ACKs for top commit:
  fanquake:
    ACK ffff4b5dc5
  jamesob:
    ACK ffff4b5dc5 ([`jamesob/ackr/28103.1.MarcoFalke.test_add_missing_set_ex`](https://github.com/jamesob/bitcoin/tree/ackr/28103.1.MarcoFalke.test_add_missing_set_ex))

Tree-SHA512: 99e67aeaae460319c2c428eab5297dbe1f1dc7f162f6592380bc5d2005308300c391cc187959cb2ace486dfe3411a8b0477f703ff11b5fe33944942c210a2d32
2023-07-20 17:03:27 +01:00
fanquake
e0c8294f29 Merge bitcoin/bitcoin#28110: doc: correct Fedora systemtap dep
12edf7b155 doc: correct Fedora systemtap dep (fanquake)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK 12edf7b155, tested on Fedora 38:

Tree-SHA512: c7995fed4bb7091691181ca92efa0f1892ae141f8f950798cfc2f446c082261ded1c4aab9ef397e15a6f1e896f1adee91e81b95c84542051ba3474f702850dc0
2023-07-20 16:54:41 +01:00
Andrew Chow
b3022af0e2 Merge bitcoin/bitcoin#28108: test: fix intermittent failure in wallet_resendwallettransactions.py
e667bd68a1 test: fix intermittent failure in wallet_resendwallettransactions.py (Martin Zumsande)

Pull request description:

  Fixes #28094

  The test bumps the mocktime for ~2 weeks and then triggers eviction from the mempool. But this bump will also cause a new resubmit, and if the timing is such that this resubmit happens right after the eviction and before the check that the tx was evicted, the test can fail as in #28094:

  ```
  node0 2023-07-17T21:31:23.809483Z (mocktime: 2023-08-02T09:46:27Z) [httpworker.1] [validation.cpp:267] [LimitMempoolSize] [mempool] Expired 2 transactions from the memory pool
  node0 2023-07-17T21:31:23.810079Z (mocktime: 2023-08-02T09:46:27Z) [scheduler] [wallet/wallet.h:895] [WalletLogPrintf] [default wallet] ResubmitWalletTransactions: resubmit 2 unconfirmed transactions
  node0 2023-07-17T21:31:23.810474Z (mocktime: 2023-08-02T09:46:27Z) [httpworker.2] [rpc/request.cpp:181] [parse] [rpc] ThreadRPCServer method=getmempoolentry user=__cookie__
  2023-07-17T21:31:23.811000Z TestFramework (ERROR): Assertion failed (...) AssertionError: No exception raised
  ```
  Fix this by flushing out the current resubmit call before triggering mempool eviction.

ACKs for top commit:
  MarcoFalke:
    Nice. lgtm ACK e667bd68a1
  achow101:
    ACK e667bd68a1
  jonatack:
    Light "this looks like the other tests in this file" ACK e667bd68a1

Tree-SHA512: 027c2177ecd8bea80ec388ec2564f8fcbc717efd2722304b16fc0e9fa7ad216af61977c4e360b8135de68586cf13b0aa729ffa4fa27bad655092c3a55f73933c
2023-07-20 11:39:24 -04:00
Andrew Chow
7edce77ff3 Merge bitcoin/bitcoin#28067: descriptors: do not return top-level only funcs as sub descriptors
dd9633b516 test: wallet, add coverage for watch-only raw sh script migration (furszy)
cc781a2180 descriptor: InferScript, do not return top-level only func as sub descriptor (furszy)
286e0c7d5e wallet: loading, log descriptor parsing error details (furszy)

Pull request description:

  Linked to #28057.

  Currently, the `InferScript` function returns an invalid descriptor when it tries to infer a p2sh-p2pkh script whose pubkey is not known by the wallet.

  This behavior occurs because the inference process bypasses the `pkh` subscript when the pubkey is not contained by the wallet (no pubkey provider), interpreting it as a `sh(addr(ADDR))` descriptor. Then, the failure arises because the `addr()` function is restricted to being used only at the top level.

  For reviewers, would recommend to start by examining the functional test to understand the context and the circumstances on which this can result in a fatal error (e.g. during the migration process).

ACKs for top commit:
  achow101:
    ACK dd9633b516
  darosior:
    utACK dd9633b516

Tree-SHA512: 61e763206c604c372019d2c36e31684f3dddf81f8b154eb9aba5cd66d8d61bda457ed4e591613eb6ce6c76cf7c3f11764abc6cd727a7c2b6414f1065783be032
2023-07-20 11:16:45 -04:00
Pieter Wuille
3388e523a1 Rework receive buffer pushback
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2023-07-20 10:36:22 -04:00
fanquake
79954903b2 Merge bitcoin/bitcoin#27620: test: miner: add coverage for -blockmintxfee setting
bbbb89d238 test: miner: add coverage for `-blockmintxfee` setting (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `-blockmintxfee` option, which can be used by miners to specify the lowest fee-rate for transactions to be included in blocks. The setting was introduced in PR #9380 (commit daec955fd6), with the rationale to decouple different minimum fees from `-minrelaytxfee`. According to the PR description it _"should be set by miners to reflect their marginal cost of transmitting extra bytes."_.

  On each iteration, the test creates and submits two txs using MiniWallet: one with the the minimum fee-rate as specified for `-blockmintxfee` and a second one with a fee-rate a little below that (-0.01 sats/vbyte). Then it checks that  only the first one is picked for the block template and accordingly also only exists in the block that is mined after. This is repeatedly done for a fixed (but obviously somewhat arbitrary) list of different `-blockmintxfee` settings on a single node, including the default and zero (i.e. no minimum fee a.k.a. "include even zero-fee txs") settings.

ACKs for top commit:
  ryanofsky:
    Code review ACK bbbb89d238, nice test
  brunoerg:
    reACK bbbb89d238
  glozow:
    ACK bbbb89d238, sorry for the late re-review!

Tree-SHA512: 7b72612971e6a1667b4b3913ec27109953fd17a1020a4bde6941a93666b2e10a23fb6fe7df23471a5671ffe31e42cd992d2efb8b31903915b3dfc1d6478df757
2023-07-20 15:33:54 +01:00
furszy
dd9633b516 test: wallet, add coverage for watch-only raw sh script migration 2023-07-20 11:04:52 -03:00
furszy
cc781a2180 descriptor: InferScript, do not return top-level only func as sub descriptor
e.g. sh(addr(ADDR)) or sh(raw(HEX)) are invalid descriptors.

Making sh and wsh top level functions to return addr/raw descriptors when
the subscript inference fails.
2023-07-20 11:04:52 -03:00
fanquake
ac7c1772f9 Merge bitcoin/bitcoin#26654: util: Show descriptive error messages when FileCommit fails
5408a55fc8 Consolidate Win32-specific error formatting (John Moffett)
c95a4432d7 Show descriptive error messages when FileCommit fails (John Moffett)

Pull request description:

  Only raw [`errno`](https://en.cppreference.com/w/cpp/error/errno) int values are logged if `FileCommit` fails. These values are implementation-specific, so it makes it harder to debug based on user reports. For instance, https://github.com/bitcoin/bitcoin/issues/26455#issue-1436654238 and [another](https://bitcointalk.org/index.php?topic=5182526.0#:~:text=FileCommit%3A%20FlushFileBuffers%20failed%3A%205).

  Instead, use `SysErrorString` (or the refactored Windows equivalent `Win32ErrorString`) to display both the raw int value and the descriptive message. All other instances in the code I could find where `errno` or (Windows-only) `GetLastError()`/`WSAGetLastError()` are logged use the full descriptive string. For example:

  1b680948d4/src/util/sock.cpp (L390)

  1b680948d4/src/util/sock.cpp (L272)

  7e1007a3c6/src/netbase.cpp (L515-L516)

  8ccab65f28/src/init.cpp (L164)

  I refactored the Windows formatting code to put it in `syserror.cpp`, as it's applicable to all Win32 API system errors, not just networking errors. To be clear, the Windows API functions `WSAGetLastError()` and `GetLastError()` are currently [equivalent](https://stackoverflow.com/questions/15586224/is-wsagetlasterror-just-an-alias-for-getlasterror).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 5408a55fc8 💡

Tree-SHA512: 3921cbac98bd9edaf84d3dd7a43896c7921f144c8ca2cde9bc96d5fb05281f7c55e7cc99db8debf6203b5f916f053025e4fa741f51458fe2c53bb57b0a781027
2023-07-20 13:37:21 +01:00
fanquake
1fde20faf8 Merge bitcoin/bitcoin#28099: contrib: move user32.dll from bitcoind.exe libs
8c38509233 contrib: move user32.dll from bitcoind.exe libs (fanquake)

Pull request description:

  The user interface library is no-longer needed by `bitcoind.exe`, or utils, only `bitcoin-qt.exe`.
  Add missing doc.

ACKs for top commit:
  hebasto:
    ACK 8c38509233, I've verified imported libraries on a Windows machine with the `dumpbin /imports` command.

Tree-SHA512: f752a5b807341c87320523f4e7c564c8acdbfc1313054a684844035102a7c4695d34cfefb0c6904f3151b2dfdcb54d6ea243c570deceeda30345944251e4c513
2023-07-20 13:16:54 +01:00
fanquake
12edf7b155 doc: correct Fedora systemtap dep 2023-07-20 11:36:02 +01:00
fanquake
355bbcba01 Merge bitcoin/bitcoin#28066: fuzz: Generate process_message targets individually
fa6245da60 fuzz: Generate process_message targets individually (MarcoFalke)
fa1471e575 refactor: Remove duplicate allNetMessageTypesVec (MarcoFalke)

Pull request description:

  Now that `LIMIT_TO_MESSAGE_TYPE` is a runtime setting after commit 927b001502, it shouldn't hurt to also generate each message type individually. Something similar was done for the `rpc` target in commit cf4da5ec29.

ACKs for top commit:
  stickies-v:
    re-crACK fa6245da60
  brunoerg:
    reACK fa6245da60

Tree-SHA512: 8f3ec71bab89781f10820a0e027fcde8949f3333eb19a30315aaad6f90f5167028113cea255b2d60b700da817c7eaac20b7b4c92f931052d7f5c2f148d33aa5a
2023-07-20 10:17:08 +01:00
fanquake
04afe55e29 Merge bitcoin/bitcoin#26467: bumpfee: Allow the user to choose which output is change
e8c31f135c tests: Test for bumping single output transaction (Andrew Chow)
4f4d4407e3 test: Test bumpfee reduce_output (Andrew Chow)
7d83502d3d bumpfee: Allow original change position to be specified (Andrew Chow)

Pull request description:

  When bumping the transaction fee, we will try to find the change output of the transaction in order to have an output whose value we can modify so that we can meet the target feerate. However we do not always find the change output which can cause us to unnecessarily add an additional output to the transaction. We can avoid this issue by outsourcing the determination of change to the user if they so desire.

  This PR adds a `orig_change_pos` option to bumpfee which the user can use to specify the index of the change output.

  Fixes #11233
  Fixes #20795

ACKs for top commit:
  ismaelsadeeq:
    re ACK e8c31f135c
  pinheadmz:
    re-ACK e8c31f135c
  furszy:
    Code review ACK e8c31f13

Tree-SHA512: 3a230655934af17f7c1a5953fafb5ef0d687c21355cf284d5e98fece411f589cd69ea505f06d6bdcf82836b08d268c366ad2dd30ae3d71541c9cdf94d1f698ee
2023-07-20 09:55:04 +01:00
MarcoFalke
fa3d72960b lint: Ignore check_fileopens failure on **kwargs
This fixes a bug in the linter:

"""
Python's open(...) seems to be used to open text files without explicitly specifying encoding='utf8':
test/functional/test_framework/test_node.py:        with open(self.debug_log_path, **kwargs) as dl:
"""
2023-07-20 09:15:43 +02:00
MarcoFalke
fa6bb85cd2 test: Ignore UTF-8 errors in assert_debug_log
read() fails in text mode when the unicode hasn't been fully written
yet. Fixes: "wallet_importdescriptors.py: can't decode bytes in position
228861-228863: unexpected end of data"
(https://github.com/bitcoin/bitcoin/issues/28030)
2023-07-20 09:15:29 +02:00
MarcoFalke
fa63326fbc test: Fix debug_log_size helper
debug_log_bytes returned "an opaque number when in text mode"
(https://docs.python.org/3/tutorial/inputoutput.html#methods-of-file-objects),
not the number of bytes.

Fix this by using binary mode or text mode (with the same encoding)
consistently when opening the file for ftell() and read().
2023-07-20 09:15:04 +02:00
Ryan Ofsky
5608e1d3b4 Merge bitcoin/bitcoin#27928: test: Add more tests for the BIP21 implementation
f1d807e383 Add more tests for the BIP21 implementation (Kiminuo)

Pull request description:

  This PR is an attempt to make it clear how the current BIP21 implementation behaves in Bitcoin Core. Especially, I'm interested whether one can specify multiple `amount` (`message`, etc.) parameters.

  My primary end goal is to answer [this question of mine](https://bitcoin.stackexchange.com/questions/118654/how-to-interpret-bip21-uri-with-amount-specified-twice/) but I figured that maybe it's worth a PR. If not, I'll close the PR.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK f1d807e383
  kevkevinpal:
    ACK [f1d807e](f1d807e383)

Tree-SHA512: d287809d47c5cfc667f850927bfd969bd345a996d3d53a4c26ef0ffd29eb75ef53358692a15f9a0493ec9e1c101123b6584572e25f87bcb98ff67f6b6c166de4
2023-07-19 16:43:12 -04:00
Andrew Chow
4d828ef427 Merge bitcoin/bitcoin#28085: refactor: use Span for SipHash::Write
7d92b1430a refactor: use Span for SipHash::Write (Sebastian Falbesoner)

Pull request description:

  This simple refactoring PR changes the interface for the `SipHash` arbitrary-data `Write` method to take a `Span<unsigned char>` instead of having to pass data and length. (`Span<std::byte>` seems to be more modern, but vectors of `unsigned char` are still used prety much everywhere where SipHash is called, and I didn't find it very appealing having to clutter the code with `Make(Writable)ByteSpan` helpers).

ACKs for top commit:
  sipa:
    utACK 7d92b1430a
  MarcoFalke:
    lgtm ACK 7d92b1430a
  achow101:
    ACK 7d92b1430a

Tree-SHA512: f17a27013c942aead4b09f5a64e0c3ff8dbc7e83fe63eb9a2e3ace8be9921c9cbba3ec67e3e83fbe3332ca941c42370efd059e702c060f9b508307e9657c66f2
2023-07-19 16:27:08 -04:00
Jon Atack
4ecfd3eaf4 Inline short, often-called, rarely-changed basic CNetAddr getters
and make them nodiscard.

Member functions containing a few lines of code are usually inlined, either
implicitly by defining them in the declaration as done here, or declared inline.

References
https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Rf-inline
https://google.github.io/styleguide/cppguide#Inline_Functions
https://www.ibm.com/docs/en/i/7.1?topic=only-inline-member-functions-c
2023-07-19 12:43:05 -06:00
Jon Atack
5316ae5dd8 Convert GetLocal() to std::optional and remove out-param
Co-authored-by: stickies-v <stickies-v@protonmail.com>
2023-07-19 12:43:05 -06:00
Jon Atack
f1304db136 Use higher-level CNetAddr and CNode helpers in net.cpp
rather than low-level comparisons with Network enum values.
2023-07-19 12:42:48 -06:00
Jon Atack
07f5891588 Add CNetAddr::IsPrivacyNet() and CNode::IsConnectedThroughPrivacyNet()
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2023-07-19 12:40:45 -06:00
Martin Zumsande
e667bd68a1 test: fix intermittent failure in wallet_resendwallettransactions.py
Before, it was possible that a resend was triggered right between
eviction the txns from the mempool and the check that they were evicted.
2023-07-19 14:14:59 -04:00
stickies-v
df488563b2 GetLocal() type-safety, naming, const, and formatting cleanups
Co-authored-by: Jon Atack <jon@atack.com>
2023-07-19 11:49:27 -06:00
Jon Atack
fb4265747c Add and use CNetAddr::HasCJDNSPrefix() helper 2023-07-19 11:49:27 -06:00
Jon Atack
5ba73cd0ee Move GetLocal() declaration from header to implementation 2023-07-19 11:49:21 -06:00
Jon Atack
11426f6557 Move CaptureMessageToFile() declaration from header to implementation 2023-07-19 11:49:16 -06:00
Jon Atack
deccf1c484 Move IsPeerAddrLocalGood() declaration from header to implementation 2023-07-19 11:48:49 -06:00
Matthew Zipkin
ba8ab4fc54 test: cover addrv2 support in anchors.dat with a TorV3 address 2023-07-19 13:24:44 -04:00
MarcoFalke
fa633aa690 streams: Teach AutoFile how to XOR 2023-07-19 18:12:42 +02:00
fanquake
78a983f597 Merge bitcoin/bitcoin#28097: depends: xcb-proto 1.15.2
7cb88c8b46 depends: xcb-proto 1.15.2 (fanquake)

Pull request description:

  Resolves build failures with Python 3.12, i.e building on rawhide:
  ```bash
  make -C depends -j9
  ...
  make[3]: Nothing to be done for 'install-exec-am'.
   /usr/bin/mkdir -p '/bitcoin/depends/work/staging/aarch64-unknown-linux-gnu/xcb_proto/1.14.1-4a91ac9dc41/bitcoin/depends/aarch64-unknown-linux-gnu/lib/python3.12/site-packages/xcbgen'
   /usr/bin/install -c -m 644 __init__.py error.py expr.py align.py matcher.py state.py xtypes.py '/bitcoin/depends/work/staging/aarch64-unknown-linux-gnu/xcb_proto/1.14.1-4a91ac9dc41/bitcoin/depends/aarch64-unknown-linux-gnu/lib/python3.12/site-packages/xcbgen'
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
  ModuleNotFoundError: No module named 'imp'
  make[3]: *** [Makefile:271: install-pkgpythonPYTHON] Error 1
  ```

  `imp` was removed in 3.12: https://docs.python.org/3/library/imp.html.

  Guix Build:
  ```bash
  6d1725c9346bdf04e1eeec2deffda90957bd081700ba896834a143756410fb0e  guix-build-7cb88c8b4672/output/aarch64-linux-gnu/SHA256SUMS.part
  34486c39daea8a3ce8d92e9c1501f26e5f5e300d9612a25bd56c48ab56353329  guix-build-7cb88c8b4672/output/aarch64-linux-gnu/bitcoin-7cb88c8b4672-aarch64-linux-gnu-debug.tar.gz
  cb0080d75d0fc4fc5c7fc022b5771470c60609c35836e7b66c4f479a2edcd098  guix-build-7cb88c8b4672/output/aarch64-linux-gnu/bitcoin-7cb88c8b4672-aarch64-linux-gnu.tar.gz
  395c332f83206125819aabf9a5dad5c906bdcd78ec13269fdfb4215bb5cc3247  guix-build-7cb88c8b4672/output/arm-linux-gnueabihf/SHA256SUMS.part
  f9e72693dd59f1b4dce6b6efa7bf325b2b589075a904c57b430dbe5c39add4ff  guix-build-7cb88c8b4672/output/arm-linux-gnueabihf/bitcoin-7cb88c8b4672-arm-linux-gnueabihf-debug.tar.gz
  14cfcebcd557d30d5a07837ab38ca217346f9b97e238a96376d6467d0db39e1a  guix-build-7cb88c8b4672/output/arm-linux-gnueabihf/bitcoin-7cb88c8b4672-arm-linux-gnueabihf.tar.gz
  1aaa7c04156a78628060a2ec47077589fb5dfb7a2e558fe604aaaa916b8a7f51  guix-build-7cb88c8b4672/output/arm64-apple-darwin/SHA256SUMS.part
  94585898babc8fdce7660c3160fe7b26c6f6749ac53a177f91a8b4226856ab7b  guix-build-7cb88c8b4672/output/arm64-apple-darwin/bitcoin-7cb88c8b4672-arm64-apple-darwin-unsigned.dmg
  9bcc693e5fad668e6c7beb8280c9c3e6988318569e0efa7d24f8cd990a3c1dd4  guix-build-7cb88c8b4672/output/arm64-apple-darwin/bitcoin-7cb88c8b4672-arm64-apple-darwin-unsigned.tar.gz
  3e427a7c2530cc3bb2f87125997bafa94234c21ebb7004cc36dc8408c0b0c9ff  guix-build-7cb88c8b4672/output/arm64-apple-darwin/bitcoin-7cb88c8b4672-arm64-apple-darwin.tar.gz
  988909c91077cb83b8d5ee77933192525d1fa1fd0bba0874138433314cb64927  guix-build-7cb88c8b4672/output/dist-archive/bitcoin-7cb88c8b4672.tar.gz
  b13c8b1f4f75505d2356f9e702db384dbb58ee3f7f0af3b110df4b7bc4c80f52  guix-build-7cb88c8b4672/output/powerpc64-linux-gnu/SHA256SUMS.part
  6f2f204401c36ee59564c8d56f9adf4f2587471ea0705e0f6a89ade4a8decf55  guix-build-7cb88c8b4672/output/powerpc64-linux-gnu/bitcoin-7cb88c8b4672-powerpc64-linux-gnu-debug.tar.gz
  c1af63ec54fcbfd267a047d2dca41bdfcace1ae6a5746df11bd710217b17b205  guix-build-7cb88c8b4672/output/powerpc64-linux-gnu/bitcoin-7cb88c8b4672-powerpc64-linux-gnu.tar.gz
  2407bc6d0beaa12dd7601ff14ef2c077346752a7b57fdf27df0a512f2af825bd  guix-build-7cb88c8b4672/output/powerpc64le-linux-gnu/SHA256SUMS.part
  ff033b4b2a3c2816555ee3a443e4087cba47d4eb05b00e092603892ff91acc30  guix-build-7cb88c8b4672/output/powerpc64le-linux-gnu/bitcoin-7cb88c8b4672-powerpc64le-linux-gnu-debug.tar.gz
  ebd9c3ab462290e2c3264dd4184a1d132b90cc93d8359ae28bc23a120b918716  guix-build-7cb88c8b4672/output/powerpc64le-linux-gnu/bitcoin-7cb88c8b4672-powerpc64le-linux-gnu.tar.gz
  57afcfe859903b960ef792447d34a17d556e44c98c021f5a35bfbd963f9e7be5  guix-build-7cb88c8b4672/output/riscv64-linux-gnu/SHA256SUMS.part
  105e6ef4fbac91e1bbbea3e98c80915654894d247cbfe0e4d2ff89b633ace10e  guix-build-7cb88c8b4672/output/riscv64-linux-gnu/bitcoin-7cb88c8b4672-riscv64-linux-gnu-debug.tar.gz
  e893e321ddd966c29fa949c3d7866fd0dca6ea87f9185e702a17c62896acf90b  guix-build-7cb88c8b4672/output/riscv64-linux-gnu/bitcoin-7cb88c8b4672-riscv64-linux-gnu.tar.gz
  80e7610cdb2ea9f1a503fa7c9ac7e29600e98fdac020951a76525494822416d9  guix-build-7cb88c8b4672/output/x86_64-apple-darwin/SHA256SUMS.part
  8fe050ca40af1d2830f7efe928f8ea8e30800dfa286f5d22549255d7ed893a30  guix-build-7cb88c8b4672/output/x86_64-apple-darwin/bitcoin-7cb88c8b4672-x86_64-apple-darwin-unsigned.dmg
  51a06c78edb9b29008deab11f5e98acac04de64080804d22b5a4f30c4b195ee5  guix-build-7cb88c8b4672/output/x86_64-apple-darwin/bitcoin-7cb88c8b4672-x86_64-apple-darwin-unsigned.tar.gz
  5cb44b69b2315fd2bb4122fa252158a9460269f24121226d88f6146e52aafc5b  guix-build-7cb88c8b4672/output/x86_64-apple-darwin/bitcoin-7cb88c8b4672-x86_64-apple-darwin.tar.gz
  d57f8ad1041a8112b012472ff00593a214f55215a29c5e5c3bc724d5eb645b1e  guix-build-7cb88c8b4672/output/x86_64-linux-gnu/SHA256SUMS.part
  76f1865d49cedf4b03f9ac5c4cb968968e54f8b19e96391340d82669a9395e93  guix-build-7cb88c8b4672/output/x86_64-linux-gnu/bitcoin-7cb88c8b4672-x86_64-linux-gnu-debug.tar.gz
  63cca6f0cbfd74148b20093bc2494b9abf50594cfb7055c5b5d702583d6c37a4  guix-build-7cb88c8b4672/output/x86_64-linux-gnu/bitcoin-7cb88c8b4672-x86_64-linux-gnu.tar.gz
  8d2afd4156525bc4684271092b5017e12f01fcc3dad29608886d10c0ef01b09d  guix-build-7cb88c8b4672/output/x86_64-w64-mingw32/SHA256SUMS.part
  6ebb038dc1f589bf92bcf3766dfcb0310f7df9ff4f75c943c2eb6fcced1a2580  guix-build-7cb88c8b4672/output/x86_64-w64-mingw32/bitcoin-7cb88c8b4672-win64-debug.zip
  d73aad8495174a49dad172278fbe590b86104bca9fcec8301a548a9fc8b1118f  guix-build-7cb88c8b4672/output/x86_64-w64-mingw32/bitcoin-7cb88c8b4672-win64-setup-unsigned.exe
  7697fffdb9b2c2f0c6042f56daf9940c561ed4c950692a2adbb039ecd885e047  guix-build-7cb88c8b4672/output/x86_64-w64-mingw32/bitcoin-7cb88c8b4672-win64-unsigned.tar.gz
  4483876d0e5709b25bd2e238958a820f7996eaebe21c1c3f3a8c3f490a0c8562  guix-build-7cb88c8b4672/output/x86_64-w64-mingw32/bitcoin-7cb88c8b4672-win64.zip
  ```

ACKs for top commit:
  hebasto:
    ACK 7cb88c8b46, tested on Ubuntu Mantic with:

Tree-SHA512: 7e01c20f15864c29ada6719051e683fbf7a533aaed810aa74763f50f6810fa49e8d3773d13d18ba88a20404305fc92d3c95190e799ebaf25a82301e97422e7a8
2023-07-19 12:34:09 +01:00
fanquake
0be2f5481c Merge bitcoin/bitcoin#27986: test: remove race in the user-agent reception check
20b49460b3 test: remove race in the user-agent reception check (Vasil Dimov)

Pull request description:

  In `add_p2p_connection()` we connect to `bitcoind` from the Python client and check that it has received our version string.

  This check looked up the last/newest entry from `getpeerinfo` RPC, assuming that it must be the connection we have just opened. But this will not be the case if a new inbound or outbound connection is made to/from `bitcoind` in the meantime.

  Instead of the last entry in `getpeerinfo`, check all and find the one which corresponds to our connection using our outgoing address:port tuple which is unique.

ACKs for top commit:
  jonatack:
    re-ACK 20b49460b3
  MarcoFalke:
    Concept ACK 20b49460b3

Tree-SHA512: 61fd3359ef11ea830021ede0e745497a7b60690c32d21c47cd12ff79f78907bb45e922c9d01e5d7ff614a8cd5e4560d39a3fc86b01b200429773a23ace3917e4
2023-07-19 12:32:30 +01:00
fanquake
3648a9b4f7 Merge bitcoin/bitcoin#28056: rpc: doc: Added longpollid and data params to template_request
f6a26196cf Added `longpollid` and `data` params to `template_request` #27998 (Rhythm Garg)

Pull request description:

  This PR will add the optional parameters `longpollid` and `data` to `template_request` as they were missing when calling `help getblocktemplate` in RPCHelpMan.

  I request the maintainers to review this and let me know about any mistakes in the descriptions of the parameters.

  This PR refers to the issue #27998

ACKs for top commit:
  ItIsOHM:
    > tACK [f6a2619](f6a26196cf)
  russeree:
    tACK f6a26196cf
  stickies-v:
    tACK f6a26196cf

Tree-SHA512: 6c592db59cb11b2d031ce5265c547fa296266278f6c25f96afe18a420e0d547f4d483e0f66de75d52c0c319ac1585f3558b9f70c12ef208c96ec96a51f786c6a
2023-07-19 12:26:09 +01:00
fanquake
84c5416b03 Merge bitcoin/bitcoin#28093: subtree: update libsecp256k1 to latest master
5080c9c25f build: adapt Windows builds for libsecp256k1 build changes (fanquake)
ff061fde18 Squashed 'src/secp256k1/' changes from 705ce7ed8c..c545fdc374 (fanquake)

Pull request description:

  Includes https://github.com/bitcoin-core/secp256k1/pull/1378. Which fixes #28079.
  Adapts Windows build for https://github.com/bitcoin-core/secp256k1/pull/1367.

ACKs for top commit:
  hebasto:
    ACK 5080c9c25f, I've made the `src/secp256k1` subtree update locally and got zero diff with this PR branch.
  jonasnick:
    ACK 5080c9c25f

Tree-SHA512: 37915d420ebacefc6bc82c2511bff3d6884e01d5c92795f19cd61862f96b30aa1fe768aeabec128c9d25c1d8bc62b46b46969626067266074b39566ad9e2f5ba
2023-07-19 11:36:30 +01:00
MarcoFalke
ffff4b5dc5 lint: Add missing set -ex to ci/lint/06_script.sh
This is needed for the container-entrypoint.sh

Also, remove unused `source` from ci/lint_run_all.sh, since it is the
last step.
2023-07-19 11:39:50 +02:00
MarcoFalke
fadc5232f4 doc: Add doc comment to ci/test_imagefile
(Similar to the doc comment in ci/lint_imagefile)

Also, rename docker-entrypoint.sh to container-entrypoint.sh

Also, add copyright header to touched files.
2023-07-19 11:27:28 +02:00
fanquake
24d5cf9314 Merge bitcoin/bitcoin#27425: test: move remaining rand code from util/setup_common to util/random
1cd45d4e08 test: move random.h include header from setup_common.h to cpp (Jon Atack)
1b246fdd14 test: move remaining random test util code from setup_common to random (jonatack)

Pull request description:

  and drop the `util/random` dependency on `util/setup_common`.  This improves code separation and allows `util/setup_common` to call `util/random` functions without creating a circular dependency, thereby addressing https://github.com/bitcoin/bitcoin/pull/26940#issuecomment-1497266140 by glozow (thanks!)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 1cd45d4e08 🌂

Tree-SHA512: 6ce63d9103ba9b04eebbd8ad02fe9aa79e356296533404034a1ae88e9b7ca0bc9a5c51fd754b71cf4e7b55b18bcd4d5474b2d588edee3851e3b3ce0e4d309a93
2023-07-19 10:26:11 +01:00
Martin Zumsande
d27b9a2248 test: fix feature_init.py file perturbation
Simultaneously opening the file in read and write mode would
lead to opening of an empty file instead of perturbing the existing
file.
Also, revert to the previous state after each perturbation so that
each perturbation is applied in isolation.
2023-07-18 15:54:23 -04:00
fanquake
c6a338b67e Merge bitcoin/bitcoin#28083: ci: Use DOCKER_BUILDKIT for lint image
fa2f18ad8e ci: Use DOCKER_BUILDKIT for lint image (MarcoFalke)

Pull request description:

  Currently the lint docker/podman image has many issues:

  * It relies on an EOL debian version.
  * It relies on a debian version different from the one used in the CI lint task.
  * It relies on the legacy docker build command, which requires the user to make `cd ./ci/lint/` before the build step.
  * It doesn't use the `.python-version` file, but a hardcoded version.

  Fix all issues by using the recommended `DOCKER_BUILDKIT=1` to generate the image.

  Also:
  * Rename `/tmp/python` to `/python_build`.
  * Compress all `pip install` commands into one.
  * Bump `.python-version`.

ACKs for top commit:
  jamesob:
    ACK fa2f18ad8e

Tree-SHA512: 804b384904ad753845667998841cc7825f4229933ca2c42af021384713486ec3cca80ba58612d37557fba7ee1921439dacca5e1236aac0557dd75bd9a2f1875d
2023-07-18 16:40:39 +01:00
fanquake
8c38509233 contrib: move user32.dll from bitcoind.exe libs
The user interface library is no-longer needed by `bitcoind.exe`,
or utils, only `bitcoin-qt.exe`.
Add missing lib doc.
2023-07-18 16:39:22 +01:00
Martin Zumsande
ad66ca1e47 init: abort loading of blockindex in case of missing height.
If a height is missing we are facing a non-contiguous block index db, and could previously
hit an assert in GetAncestor() called from BuildSkip() instead of returning an error.
2023-07-18 11:29:40 -04:00
fanquake
5080c9c25f build: adapt Windows builds for libsecp256k1 build changes
See https://github.com/bitcoin-core/secp256k1/pull/1367.
2023-07-18 15:26:51 +01:00
fanquake
ff061fde18 Squashed 'src/secp256k1/' changes from 705ce7ed8c..c545fdc374
c545fdc374 Merge bitcoin-core/secp256k1#1298: Remove randomness tests
b40e2d30b7 Merge bitcoin-core/secp256k1#1378: ellswift: fix probabilistic test failure when swapping sides
c424e2fb43 ellswift: fix probabilistic test failure when swapping sides
907a67212e Merge bitcoin-core/secp256k1#1313: ci: Test on development snapshots of GCC and Clang
0f7657d59c Merge bitcoin-core/secp256k1#1366: field: Use `restrict` consistently in fe_sqrt
cc55757552 Merge bitcoin-core/secp256k1#1340: clean up in-comment Sage code (refer to secp256k1_params.sage, update to Python3)
600c5adcd5 clean up in-comment Sage code (refer to secp256k1_params.sage, update to Python3)
981e5be38c ci: Fix typo in comment
e9e9648219 ci: Reduce number of macOS tasks from 28 to 8
609093b387 ci: Add x86_64 Linux tasks for gcc and clang snapshots
1deecaaf3b ci: Install development snapshots of gcc and clang
b79ba8aa4c field: Use `restrict` consistently in fe_sqrt
c9ebca95f9 Merge bitcoin-core/secp256k1#1363: doc: minor ellswift.md updates
afd7eb4a55 Merge bitcoin-core/secp256k1#1371: Add exhaustive tests for ellswift (with create+decode roundtrip)
2792119278 Add exhaustive test for ellswift (create+decode roundtrip)
c7d900ffd1 doc: minor ellswift.md updates
332af315fc Merge bitcoin-core/secp256k1#1344: group: save normalize_weak calls in `secp256k1_ge_is_valid_var`/`secp256k1_gej_eq_x_var`
9e6d1b0e9b Merge bitcoin-core/secp256k1#1367: build: Improvements to symbol visibility logic on Windows (attempt 3)
0aacf64352 Merge bitcoin-core/secp256k1#1370: Corrected some typos
b6b9834e8d small fixes
07c0e8b82e group: remove unneeded normalize_weak in `secp256k1_gej_eq_x_var`
3fc1de5c55 Merge bitcoin-core/secp256k1#1364: Avoid `-Wmaybe-uninitialized` when compiling with `gcc -O1`
fb758fe8d6 Merge bitcoin-core/secp256k1#1323: tweak_add: fix API doc for tweak=0
c6cd2b15a0 ci: Add task for static library on Windows + CMake
020bf69a44 build: Add extensive docs on visibility issues
0196e8ade1 build: Introduce `SECP256k1_DLL_EXPORT` macro
9f1b1904a3 refactor: Replace `SECP256K1_API_VAR` with `SECP256K1_API`
ae9db95cea build: Introduce `SECP256K1_STATIC` macro for Windows users
7966aee31d Merge bitcoin-core/secp256k1#1369: ci: Print commit in Windows container
a7bec34231 ci: Print commit in Windows container
249c81eaa3 Merge bitcoin-core/secp256k1#1368: ci: Drop manual checkout of merge commit
98579e297b ci: Drop manual checkout of merge commit
5b9f37f136 ci: Add `CFLAGS: -O1` to task matrix
a6ca76cdf2 Avoid `-Wmaybe-uninitialized` when compiling with `gcc -O1`
0fa84f869d Merge bitcoin-core/secp256k1#1358: tests: introduce helper for non-zero `random_fe_test()` results
5a95a268b9 tests: introduce helper for non-zero `random_fe_test` results
304421d57b tests: refactor: remove duplicate function `random_field_element_test`
3aef6ab8e1 Merge bitcoin-core/secp256k1#1345: field: Static-assert that int args affecting magnitude are constant
4494a369b6 Merge bitcoin-core/secp256k1#1357: tests: refactor: take use of `secp256k1_ge_x_on_curve_var`
799f4eec27 Merge bitcoin-core/secp256k1#1356: ci: Adjust Docker image to Debian 12 "bookworm"
c862a9fb49 ci: Adjust Docker image to Debian 12 "bookworm"
a1782098a9 ci: Force DWARF v4 for Clang when Valgrind tests are expected
7d8d5c86df tests: refactor: take use of `secp256k1_ge_x_on_curve_var`
8a7273465b Help the compiler prove that a loop is entered
fd491ea1bb Merge bitcoin-core/secp256k1#1355: Fix a typo in the error message
ac43613d25 Merge bitcoin-core/secp256k1#1354: Add ellswift to CHANGELOG
67887ae65c Fix a typo in the error message
926dd3e962 Merge bitcoin-core/secp256k1#1295: abi: Use dllexport for mingw builds
10836832e7 Merge bitcoin-core/secp256k1#1336: Use `__shiftright128` intrinsic in `secp256k1_u128_rshift` on MSVC
7c7467ab7f Refer to ellswift.md in API docs
c32ffd8d8c Add ellswift to CHANGELOG
3c1a0fd37f Merge bitcoin-core/secp256k1#1347: field: Document return value of fe_sqrt()
5779137457 field: Document return value of fe_sqrt()
be8ff3a02a field: Static-assert that int args affecting magnitude are constant
efa76c4bf7 group: remove unneeded normalize_weak in `secp256k1_ge_is_valid_var`
5b7bf2e9d4 Use `__shiftright128` intrinsic in `secp256k1_u128_rshift` on MSVC
05873bb6b1 tweak_add: fix API doc for tweak=0
6ec3731e8c Simplify test PRNG implementation
fb5bfa4eed Add static test vector for Xoshiro256++
723e8ca8f7 Remove randomness tests
bc7c8db179 abi: Use dllexport for mingw builds

git-subtree-dir: src/secp256k1
git-subtree-split: c545fdc374964424683d9dac31a828adedabe860
2023-07-18 15:25:05 +01:00
fanquake
8ee662984f Update secp256k1 subtree to latest upstream master 2023-07-18 15:25:05 +01:00
fanquake
08eb5f1b67 ci: document that -Wreturn-type has been fixed upstream (Windows)
`noreturn` attributes have been added to the mingw-w64 headers, meaning
that from 11.0.0 onwards, you'll no-longer see `-Wreturn-type` warnings
when using assert(false):
1690994f51.

Add -Wno-return-type to the Windows CI, where is should have been all
along, and document why it's required. This can be dropped when we are
using the fixed version of the mingw-w64 headers there.

Drop the -Werror -Wno-return-type special case from our build system.
-Wreturn-type is on by default in Clang and GCC.
2023-07-18 14:27:30 +01:00
fanquake
7cb88c8b46 depends: xcb-proto 1.15.2
Resolves build failures under Python 3.12, i.e building on rawhide:
```bash
make[3]: Nothing to be done for 'install-exec-am'.
 /usr/bin/mkdir -p '/bitcoin/depends/work/staging/aarch64-unknown-linux-gnu/xcb_proto/1.14.1-4a91ac9dc41/bitcoin/depends/aarch64-unknown-linux-gnu/lib/python3.12/site-packages/xcbgen'
 /usr/bin/install -c -m 644 __init__.py error.py expr.py align.py matcher.py state.py xtypes.py '/bitcoin/depends/work/staging/aarch64-unknown-linux-gnu/xcb_proto/1.14.1-4a91ac9dc41/bitcoin/depends/aarch64-unknown-linux-gnu/lib/python3.12/site-packages/xcbgen'
Traceback (most recent call last):
  File "<string>", line 2, in <module>
ModuleNotFoundError: No module named 'imp'
make[3]: *** [Makefile:271: install-pkgpythonPYTHON] Error 1
```

`imp` was removed in 3.12: https://docs.python.org/3/library/imp.html.
2023-07-18 11:27:24 +01:00
fanquake
673acab223 Merge bitcoin/bitcoin#28090: validation: use noexcept instead of deprecated throw()
047daad4f5 clang-tidy: turn on modernize-use-noexcept (fanquake)
85e9e1f802 validation: use noexcept instead of deprecated throw() (fanquake)

Pull request description:

  We fixed this once before in https://github.com/bitcoin/bitcoin/pull/10965.
  Turn on https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-noexcept.html#modernize-use-noexcept.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 047daad4f5
  sipa:
    utACK 047daad4f5
  Empact:
    utACK 047daad4f5
  stickies-v:
    ACK 047daad4f5

Tree-SHA512: 949b0fe598d66583747853094db13f196b402000e601f8634e5a708b55454d29c5aa18eaf1f2420d3ccf10e3e524b7414ff3a6fe4cb431420bf749c22b2b8bab
2023-07-18 10:24:13 +01:00
Andrew Chow
bc88f3ab90 Merge bitcoin/bitcoin#27997: Descriptors: rule out unspendable miniscript descriptors
c7db88af71 descriptor: assert we never parse a sane miniscript with no pubkey (Antoine Poinsot)
a49402a9ec qa: make sure we don't let unspendable Miniscript descriptors be imported (Antoine Poinsot)
639e3b6c97 descriptor: refuse to parse unspendable miniscript descriptors (Antoine Poinsot)
e3280eae1b miniscript: make GetStackSize() and GetOps() return optionals (Antoine Poinsot)

Pull request description:

  `IsSane()` in Miniscript does not ensure a Script is actually spendable. This is an issue as we would accept any sane Miniscript when parsing a descriptor. Fix this by explicitly checking a Miniscript descriptor is both sane and spendable when parsing it.

  This bug was exposed due to a check added in #22838 (https://github.com/bitcoin/bitcoin/pull/22838#discussion_r1226859880) that triggered a fuzz crash (https://github.com/bitcoin/bitcoin/pull/22838#issuecomment-1612510057).

ACKs for top commit:
  sipa:
    utACK c7db88af71
  achow101:
    ACK c7db88af71

Tree-SHA512: e79bc9f7842e98a4e8f358f05811fca51b15b4b80a171c0d2b17cf4bb1f578a18e4397bc2ece9817d392e0de0196ee6a054b7318441fd3566dd22e1f03eb64a5
2023-07-17 19:16:09 -04:00
Andrew Chow
306157ae92 Merge bitcoin/bitcoin#27993: Make poly1305 support incremental computation + modernize
4e5c933f6a Switch all callers from poly1305_auth to Poly1305 class (Pieter Wuille)
8871f7d1ae tests: add more Poly1305 test vectors (Pieter Wuille)
40e6c5b9fc crypto: add Poly1305 class with std::byte Span interface (Pieter Wuille)
50269b391f crypto: switch poly1305 to incremental implementation (Pieter Wuille)

Pull request description:

  Our current Poly1305 code (src/crypto/poly1305.*) only supports computing the entire tag in one go (the `poly1305_auth` function takes a key and message, and outputs the tag). However, the RFC8439 authenticated encryption (as used in BIP324, see #27634) scheme makes use of Poly1305 in a way where the message consists of 3 different pieces:
  * The additionally authenticated data (AAD), padded to 16 bytes.
  * The ciphertext, padded to 16 bytes.
  * The length of the AAD and the length of the ciphertext, together another 16 bytes.

  Implementing RFC8439 using the existing `poly1305_auth` function requires creating a temporary copy with all these pieces of data concatenated just for the purpose of computing the tag (the approach used in #25361).

  This PR replaces the poly1305 code with new code from https://github.com/floodyberry/poly1305-donna (with minor adjustments to make it match our coding style and use our utility functions, documented in the commit) which supports incremental operation, and then adds a C++ wrapper interface using std::byte Spans around it, and adds tests that incremental and all-at-once computation match.

ACKs for top commit:
  achow101:
    ACK 4e5c933f6a
  theStack:
    ACK 4e5c933f6a
  stratospher:
    tested ACK 4e5c933.

Tree-SHA512: df6e9a2a4a38a480f9e4360d3e3def5311673a727a4a85b008a084cf6843b260dc82cec7c73e1cecaaccbf10f3521a0ae7dba388b65d0b086770f7fbc5223e2a
2023-07-17 18:30:39 -04:00
fanquake
d09c8bc730 Merge bitcoin/bitcoin#28088: test: Disable known broken USDT test
faf8be7c32 test: Disable known broken USDT test (MarcoFalke)

Pull request description:

  The failure is known and running into more failures doesn't help anyone. Not disabling the test would be a waste of CPU and developer time.

  https://github.com/bitcoin/bitcoin/issues/27380

Top commit has no ACKs.

Tree-SHA512: d0469153b00d6b30e10a21bcd52d508fcf9f796ff2468f59aff75020a82c718bcae85caf4b58397dea6fd9e210b501353fd51567f979c6b57d3b1bb23d318216
2023-07-17 15:04:49 +01:00
fanquake
047daad4f5 clang-tidy: turn on modernize-use-noexcept 2023-07-17 14:45:42 +01:00
fanquake
85e9e1f802 validation: use noexcept instead of deprecated throw()
```bash
  CXX      libbitcoin_node_a-validation.o
validation.cpp:5164:30: warning: dynamic exception specifications are deprecated [-Wdeprecated-dynamic-exception-spec]
    const char* what() const throw() override
                             ^~~~~~~
validation.cpp:5164:30: note: use 'noexcept' instead
    const char* what() const throw() override
                             ^~~~~~~
                             noexcept
```
2023-07-17 14:44:46 +01:00
fanquake
c123e1d215 Merge bitcoin/bitcoin#28069: guix: Remove librt usage from release binaries
8f6f0d81ee guix: backport glibc patch to prevent redundant librt link (fanquake)
e14473299c contrib: remove librt from release deps (fanquake)
e64e17830a build: remove check for gettimeofday & librt (fanquake)

Pull request description:

  Our release binaries currently have a runtime dependency on `librt`. However this is redundant, and only the case due to  a bug in glibc. The `clock_*` suit of funcs were absorbed into libc long ago, however an issue with compatibility code meant that librt would still be linked against / used redundantly:
  > But the forwarders were not marked as compatibility symbols.
  > As a result, on older architectures, historic configure checks such as
  > AC_CHECK_LIB(rt, clock_gettime)
  > still cause linking against librt, even though this is completely
  > unnecessary.  It also creates a needless porting hazard because
  > architectures behave differently when it comes to symbol availability.

  This PR drops our configure check for librt (which is redundant, and could be PR'd standalone), and backports [the relevant patch](https://sourceware.org/git/?p=glibc.git;a=commit;h=f289e656ec8221756519a601042bc9fbe1b310fb) into our glibc, so we can drop librt from our runtime dependencies.

  Guix Build:
  ```bash
  67078bddd5dc32801b8c916c3bc12f1404da572312f0158a89b9603c1f753969  guix-build-8f6f0d81ee3a/output/aarch64-linux-gnu/SHA256SUMS.part
  794dd00009860fd67d7e51463ee1c5ea9677dfff1c739dd0b91cf73136deb655  guix-build-8f6f0d81ee3a/output/aarch64-linux-gnu/bitcoin-8f6f0d81ee3a-aarch64-linux-gnu-debug.tar.gz
  eb9cf3f472ffbc37446fe4d80fe81dc62cf1c28c4d57dd8a7b7176e65487aeeb  guix-build-8f6f0d81ee3a/output/aarch64-linux-gnu/bitcoin-8f6f0d81ee3a-aarch64-linux-gnu.tar.gz
  e775a9e9b23be44b5c7e7121e88124746836d5bdeda1cd9ba693080d9f3a52a8  guix-build-8f6f0d81ee3a/output/arm-linux-gnueabihf/SHA256SUMS.part
  8289f0770333d800e414747026c0fb105d95f389f6c8d901c1041cc65272fb02  guix-build-8f6f0d81ee3a/output/arm-linux-gnueabihf/bitcoin-8f6f0d81ee3a-arm-linux-gnueabihf-debug.tar.gz
  e40256c5fb1b9a137845a50fc051f92c3e4cc013b0875a71c62af32f7024af9d  guix-build-8f6f0d81ee3a/output/arm-linux-gnueabihf/bitcoin-8f6f0d81ee3a-arm-linux-gnueabihf.tar.gz
  c8db222e54e78b27a8a5d3a373a9bbafa51ed29a1fda5c19e8b0eac819b002f2  guix-build-8f6f0d81ee3a/output/arm64-apple-darwin/SHA256SUMS.part
  52d4063af628467605fcf533205705b38237a0cc60cafbec224ca8cf4a644738  guix-build-8f6f0d81ee3a/output/arm64-apple-darwin/bitcoin-8f6f0d81ee3a-arm64-apple-darwin-unsigned.dmg
  103d80180a9f38e7c903d0b6581e4bb5130c640fac1fd5019eee7fa90e303c1d  guix-build-8f6f0d81ee3a/output/arm64-apple-darwin/bitcoin-8f6f0d81ee3a-arm64-apple-darwin-unsigned.tar.gz
  a8f0a89c4d4b1d05e6ea968dde3b13368999dfc1c3ea765e81fd3c4db46197b3  guix-build-8f6f0d81ee3a/output/arm64-apple-darwin/bitcoin-8f6f0d81ee3a-arm64-apple-darwin.tar.gz
  726d2671bbed2355c083b8516faa5d8e0422fab6cb38a135f68ee011f9e09af5  guix-build-8f6f0d81ee3a/output/dist-archive/bitcoin-8f6f0d81ee3a.tar.gz
  955fff1c9998bb04bcf1afe9b467590960206e9c512b3446ecdd701e251bb419  guix-build-8f6f0d81ee3a/output/powerpc64-linux-gnu/SHA256SUMS.part
  e95cdeda727d641c002755c4a3e3b69049a35f1bff4867ac14320585d65595c4  guix-build-8f6f0d81ee3a/output/powerpc64-linux-gnu/bitcoin-8f6f0d81ee3a-powerpc64-linux-gnu-debug.tar.gz
  21bda341cd8af44bc731cf7e3637322a92032e7a956acdde25ea6e59989c67b9  guix-build-8f6f0d81ee3a/output/powerpc64-linux-gnu/bitcoin-8f6f0d81ee3a-powerpc64-linux-gnu.tar.gz
  6f90c38998696f61c373c3546bcc03e6b5ecfbe3b9fec9a7c75d601b3175b698  guix-build-8f6f0d81ee3a/output/powerpc64le-linux-gnu/SHA256SUMS.part
  7166c2354b8777464bf8c5c3d7e4a171d00b5e0617635fa8b12c4d47ad619e84  guix-build-8f6f0d81ee3a/output/powerpc64le-linux-gnu/bitcoin-8f6f0d81ee3a-powerpc64le-linux-gnu-debug.tar.gz
  8c879a3ae9fefc1071d0b6ea3b0cf858295386860b10079b472b526abfdcd2b5  guix-build-8f6f0d81ee3a/output/powerpc64le-linux-gnu/bitcoin-8f6f0d81ee3a-powerpc64le-linux-gnu.tar.gz
  7dc7153d3c180308d873cb20320e8a6221cec81d8018da85683870168380eef7  guix-build-8f6f0d81ee3a/output/riscv64-linux-gnu/SHA256SUMS.part
  c37b79e33b9a318d3acee9114cdf057ee518abaa09736bd63e015d924d2c3ffb  guix-build-8f6f0d81ee3a/output/riscv64-linux-gnu/bitcoin-8f6f0d81ee3a-riscv64-linux-gnu-debug.tar.gz
  d25abfb09d12e74bffd7f42e95eba211317acefa4718dbea27055d905f5b6999  guix-build-8f6f0d81ee3a/output/riscv64-linux-gnu/bitcoin-8f6f0d81ee3a-riscv64-linux-gnu.tar.gz
  5ffc5c97012d8ae85cb56e635760029b774ea4f57a64e41cd4bdade4ed93e619  guix-build-8f6f0d81ee3a/output/x86_64-apple-darwin/SHA256SUMS.part
  ecf96275016e82af2c1a4842578feac286de9db8b7f5e4266cf877cb29da1da8  guix-build-8f6f0d81ee3a/output/x86_64-apple-darwin/bitcoin-8f6f0d81ee3a-x86_64-apple-darwin-unsigned.dmg
  50bee378ed88471dc326730564ca24cea2625ce1477b82881cda572f0a8913cc  guix-build-8f6f0d81ee3a/output/x86_64-apple-darwin/bitcoin-8f6f0d81ee3a-x86_64-apple-darwin-unsigned.tar.gz
  f4215a018f18e3639c50f10909af3ceff6982abf8b292fd88fa5d690b06d704a  guix-build-8f6f0d81ee3a/output/x86_64-apple-darwin/bitcoin-8f6f0d81ee3a-x86_64-apple-darwin.tar.gz
  ee5278c8afc7ead80853aff69c1bbd624ef078428076f0e92b0ad35931036b3f  guix-build-8f6f0d81ee3a/output/x86_64-linux-gnu/SHA256SUMS.part
  daed3889107ffe8b3ec2c59abff93d4b92a4dff382457485d29489a0e9421965  guix-build-8f6f0d81ee3a/output/x86_64-linux-gnu/bitcoin-8f6f0d81ee3a-x86_64-linux-gnu-debug.tar.gz
  f1acd6b1d296f2de5ff838fe3fb82035f2774485b06678ecdd461e631ebbe092  guix-build-8f6f0d81ee3a/output/x86_64-linux-gnu/bitcoin-8f6f0d81ee3a-x86_64-linux-gnu.tar.gz
  3e9f9f92e4de995c9029f17962c33e317f7000df9c1afa2a447b65ac98c27f4b  guix-build-8f6f0d81ee3a/output/x86_64-w64-mingw32/SHA256SUMS.part
  4b50a73917450770c793bfc787a6785c7389ce02bd25368db9a1445da07bb7b1  guix-build-8f6f0d81ee3a/output/x86_64-w64-mingw32/bitcoin-8f6f0d81ee3a-win64-debug.zip
  832ddec19b8c5698cc3497f93fc59f0f72b0d7a3f313d46c2c1c52b5badf19fd  guix-build-8f6f0d81ee3a/output/x86_64-w64-mingw32/bitcoin-8f6f0d81ee3a-win64-setup-unsigned.exe
  d9bc2dabd0cff8e9ee6ccb309bee34a6faa1298771c0cc9bff8f948d34ec047e  guix-build-8f6f0d81ee3a/output/x86_64-w64-mingw32/bitcoin-8f6f0d81ee3a-win64-unsigned.tar.gz
  55cc5607d3fdf113fde463d87c5dd895c305ba0313e56bba1b0875a8a78c65a7  guix-build-8f6f0d81ee3a/output/x86_64-w64-mingw32/bitcoin-8f6f0d81ee3a-win64.zip
  ```

ACKs for top commit:
  hebasto:
    ACK 8f6f0d81ee

Tree-SHA512: f6fd4b9ed37ad93c7a5df4ca17f1ae5b8705f5dc4a377c8e01c6376b1818980534a233a08f2a20c4ff851a25f660ebb89c7416b93f6f039747194661b00c75ed
2023-07-17 13:43:45 +01:00
fanquake
bf03fed2c7 Merge bitcoin/bitcoin#28065: fuzz: Flatten all FUZZ_TARGET macros into one
fa6dfaaf45 scripted-diff: Use new FUZZ_TARGET macro everywhere (MarcoFalke)
fa36ad8b09 fuzz: Accept options in FUZZ_TARGET macro (MarcoFalke)

Pull request description:

  The `FUZZ_TARGET` macros have many issues:
  * The developer will have to pick the right macro to pass the wanted option.
  * Adding a new option requires doubling the number of existing macros in the worst case.

  Fix all issues by using only a single macro.

  This refactor does not change behavior.

ACKs for top commit:
  dergoegge:
    ACK fa6dfaaf45

Tree-SHA512: 49a34553867a1734ce89e616b2d7c29b784a67cd8990db6573f0c7b18957636ef0c81d3d0d444a04c12cdc98bc4c4aa7a2ec94e6232dc363620a746e28416444
2023-07-17 13:36:53 +01:00
fanquake
275b3e47cc Merge bitcoin/bitcoin#28084: doc: update windows -fstack-clash-protection doc
05ef059a33 doc: update windows -fstack-clash-protection doc (fanquake)

Pull request description:

  Now that changes have been made in GCC, to fix the build failures.
  See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458.

ACKs for top commit:
  TheCharlatan:
    ACK 05ef059a33
  hebasto:
    ACK 05ef059a33, I've verified that the fix commit is present in all branches starting from `gcc-11`.

Tree-SHA512: 96b79d65b46e6b9d939c8e6079e984da86987503210106d5155dbe5a6fd82d56d9983694656e27156b01bab795c766b85fc60c799813bc676bba5f3b73f9be22
2023-07-17 13:14:35 +01:00
fanquake
7d8af44356 Merge bitcoin/bitcoin#28086: fuzz: Bump FuzzedDataProvider.h
fa367422ef fuzz: Bump FuzzedDataProvider.h (MarcoFalke)

Pull request description:

  Also, remove suppression.

ACKs for top commit:
  dergoegge:
    utACK fa367422ef

Tree-SHA512: 1d960cbedc4f516ef3dcec05b158164eb9673bcb02793c39d4b345be6d767aded1569289175701bc7382afd00ca41a2409831877f100ab9324969de9045ab6fc
2023-07-17 13:13:34 +01:00
MarcoFalke
faf8be7c32 test: Disable known broken USDT test 2023-07-17 13:49:00 +02:00
Sebastian Falbesoner
7d92b1430a refactor: use Span for SipHash::Write
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2023-07-17 13:27:46 +02:00
MarcoFalke
fa367422ef fuzz: Bump FuzzedDataProvider.h
From fa8401f9bf/compiler-rt/include/fuzzer/FuzzedDataProvider.h
2023-07-17 09:39:52 +02:00
Hennadii Stepanov
57b8336dfe Merge bitcoin-core/gui#740: Show own outputs on PSBT signing window
4da243ba02 qt: show own outputs on PSBT signing window (Hernan Marino)

Pull request description:

  This fixes https://github.com/bitcoin-core/gui/issues/732 .
  It allows you to identify your own addresses in the outputs of a transaction in the PSBT signing window. This enables easy identification of change outputs, and prevents certain attacks where someone (co-signers of a multisig, or others ) might trick you into signing a transaction while they are stealing the change, since prior to this modification there was no easy way of knowing this.

  The identification of the output is similar to the way this is done in the transaction details window.

  A sample output is :

  ![image](https://github.com/bitcoin-core/gui/assets/87907936/48b8a652-7570-466b-9a34-cc0303c86d8c)

ACKs for top commit:
  achow101:
    ACK 4da243ba02
  jarolrod:
    ACK 4da243ba02

Tree-SHA512: fa9901d2acc84472c11afcd0a59a859db598cdf5cea755b492178d3e7434b70d9bd8f554928938a2ff9920c8f397fef814ce14b416556c30fba0c3c1f62cd722
2023-07-16 19:43:11 +01:00
fanquake
05ef059a33 doc: update windows -fstack-clash-protection doc
Now that changes have been made in GCC, to fix the build failures.
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458.
2023-07-16 15:19:44 +01:00
MarcoFalke
fa2f18ad8e ci: Use DOCKER_BUILDKIT for lint image
Can be reviewed with:
--color-moved=dimmed-zebra  --ignore-all-space
2023-07-16 13:18:18 +02:00
Suhas Daftuar
3cfc75366e test: Clear block index flags when testing snapshots
When simulating a snapshot, remove the HAVE_DATA status for blocks below the
snapshot height, to simulate never having downloaded them at all. This makes
tests more realistic (and more closely match what will happen when using
assumeutxo).
2023-07-14 17:10:49 -04:00
Suhas Daftuar
272fbc370c Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot 2023-07-14 17:09:06 -04:00
Suhas Daftuar
10c05710ce Add wrapper for adding entries to a chainstate's block index candidates 2023-07-14 17:09:06 -04:00
Suhas Daftuar
471da5f6e7 Move block-arrival information / preciousblock counters to ChainstateManager
Block arrival information (and the preciousblock RPC, a related concept) are
both chainstate-agnostic, so these are moved to ChainstateManager. This should
just be a refactor, without any observable behavior changes.
2023-07-14 17:09:06 -04:00
Andrew Chow
5d8469362a test: Add helper functions for checking node versions 2023-07-14 17:03:30 -04:00
Suhas Daftuar
1cfc887d00 Remove CChain dependency in node/blockstorage 2023-07-14 14:54:57 -04:00
Suhas Daftuar
fe86a7cd48 Explicitly track maximum block height stored in undo files
When writing a new block to disk, if we have filled up the current block file,
then we flush and truncate that block file (to free allocated but unused
space) before advancing to the next one. When this happens, we have to
determine whether to also flush and truncate the corresponding undo file.

Undo data is only written when blocks are connected, not when blocks are
received. Thus it's possible that the corresponding undo file already has all
the data it will ever have, and we should flush/truncate it as we advance
files; or it's possible that there is more data we expect to write, and should
therefore defer flush/truncation until undo data is later written.

Prior to this commit, we made the determination of whether the undo file was
full of all requisite data by comparing against the chain tip. This patch
replaces that dependence on validation data structures by instead just tracking
the highest height of any block written in the undo file as we go.
2023-07-14 14:47:00 -04:00
Andrew Chow
01e5d6b105 Merge bitcoin/bitcoin#28048: kernel: Remove StartShutdown calls from validation code
31eca93a9e kernel: Remove StartShutdown calls from validation code (Ryan Ofsky)

Pull request description:

  This change drops the last kernel dependency on shutdown.cpp. It also adds new hooks for libbitcoinkernel applications to be able to interrupt kernel operations when the chain tip changes.

  This change is mostly a refactoring, but does slightly change `-stopatheight` behavior (see release note and commit message)

ACKs for top commit:
  TheCharlatan:
    ACK 31eca93a9e
  furszy:
    Concept and light review ACK 31eca93a
  hebasto:
    ACK 31eca93a9e, I have reviewed the code and it looks OK.
  MarcoFalke:
    lgtm ACK 31eca93a9e 🕷

Tree-SHA512: e26928436bcde658e842b1f92e9c24b1ce91031fb63b41aafccf3130bfff532b75338a269a2bb7558bff2973913f17b97a00fec3e7e0588e2ce44de097142047
2023-07-14 13:12:30 -04:00
Rhythm Garg
f6a26196cf Added longpollid and data params to template_request #27998
Added `longpollid` and `data` params to `template_request` in `getblocktemplate` #27998
2023-07-14 20:48:16 +05:30
fanquake
4a1aae6749 Merge bitcoin/bitcoin#28071: ci: Add missing -O2 to valgrind tasks
fa4ccf1511 ci: Add missing -O2 to valgrind tasks (MarcoFalke)

Pull request description:

  Currently the tasks have nothing (`-O0`) set, which makes them slow.

  Fix this by falling back to the default (`-O2`).

ACKs for top commit:
  recursive-rat4:
    utACK fa4ccf1511
  dergoegge:
    utACK fa4ccf1511

Tree-SHA512: 44d803000d883cfa534f2c76d793d7d7f840e114fc377d20fc36d008b471d69ec9f0170358ed1f3567d49e3ff63682244062c954cd0b963df31ca39c08d2d5b9
2023-07-14 10:54:30 +01:00
Andrew Chow
ee467b8238 Merge bitcoin/bitcoin#27549: fuzz: addrman, add coverage for network field in Select(), Size() and GetAddr()
35a2175ad8 fuzz: addrman, add coverage for `network` field in `Select()`, `Size()` and `GetAddr()` (brunoerg)

Pull request description:

  This PR adds fuzz coverage for `network` field in `Select()`, `Size()` and `GetAddr()`, there was only call to them without passing a network.
  https://marcofalke.github.io/b-c-cov/fuzz.coverage/src/addrman.cpp.gcov.html

ACKs for top commit:
  amitiuttarwar:
    for the record, ACK 35a2175ad8 - only small changes from the version (previously) proposed in 27213
  achow101:
    ACK 35a2175ad8
  mzumsande:
    Code Review ACK 35a2175ad8, haven't tested this yet, but I will let the fuzzer run for a while now.

Tree-SHA512: dddb8322298d6c373c8e68d57538470b11825a9a310a355828c351d5c0b19ff6779d024a800e3ea90126d0c050e86f71fd22cd23d1a306c784cef0f82c45e3ca
2023-07-13 19:07:15 -04:00
MarcoFalke
fa6dfaaf45 scripted-diff: Use new FUZZ_TARGET macro everywhere
-BEGIN VERIFY SCRIPT-

  ren() { sed --regexp-extended -i "s|$1|$2|g" $(git grep -l --extended-regexp "$1"); }

  # Replace FUZZ_TARGET_INIT
  ren 'FUZZ_TARGET_INIT\((.+), (.+)\)' 'FUZZ_TARGET(\1, .init = \2)'

  # Delete unused FUZZ_TARGET_INIT
  sed -i -e '37,39d' src/test/fuzz/fuzz.h

-END VERIFY SCRIPT-
2023-07-13 20:37:14 +02:00
MarcoFalke
fa36ad8b09 fuzz: Accept options in FUZZ_TARGET macro
* This allows to reduce the number of total macros.
* Also, adding a new option no longer requires doubling the number of
  macros in the worst case.
2023-07-13 20:37:05 +02:00
Andrew Chow
05ad4de158 Merge bitcoin/bitcoin#27411: p2p: Restrict self-advertisements with privacy networks to avoid fingerprinting
e7cf8657e1 test: add unit test for local address advertising (Martin Zumsande)
f4754b9dfb net: restrict self-advertisements with privacy networks (Martin Zumsande)
e4d541c7cf net, refactor: pass reference for peer address in GetReachabilityFrom (Martin Zumsande)
62d73f5370 net, refactor: pass CNode instead of CNetAddr to GetLocalAddress (Martin Zumsande)

Pull request description:

  The current logic for self-advertisements works such that we detect as many local addresses as we can, and then, using the scoring matrix from `CNetAddr::GetReachabilityFrom()`, self-advertise with the address that fits best to our peer.
  It is in general not hard for our peers to distinguish our self-advertisements from other addrs we send them, because we self-advertise every ~24h and because the first addr we send over a connection is likely our self-advertisement.

  `GetReachabilityFrom()` currently only takes into account actual reachability, but not whether we'd _want_ to announce our identity for one network to peers from other networks, which is not straightforward in connection with privacy networks.

  While the general approach is to prefer self-advertising with the address for the network our peer is on, there are several special situations in which we don't have one, and as a result could allow self-advertise other local addresses, for example:

  A) We run i2p and clearnet, use `-i2pacceptincoming=0` (so we have no local i2p address), and we have a local ipv4 address. In this case, we'd advertise the ipv4 address to our outbound i2p peers.

  B) Our `-discover` logic cannot detect any local clearnet addresses in our network environment, but we are actually reachable over clearnet. If we ran bitcoind clearnet-only, we'd always advertise the address our peer sees us with instead, and could get inbound peers this way. Now, if we also have an onion service running (but aren't using tor as a proxy for clearnet connections), we could advertise our onion address to clearnet peers, so that they would be able to connect our clearnet and onion identities.

  This PR tries to avoid these situations by
  1.) never advertising our local Tor or I2P address to peers from other networks.
  2.) never advertising local addresses from non-anonymity networks to peers from Tor or I2P

  Note that this affects only our own self-advertisements, the rules to forward other people's addrs are not changed.

  [Edit] after Initial [discussion](https://github.com/bitcoin/bitcoin/pull/27411#issuecomment-1497176155): CJDNS is not being treated like Tor and I2P at least for now, because it has different privacy properties and for the practical reason that it has still very few bitcoin nodes.

ACKs for top commit:
  achow101:
    ACK e7cf8657e1
  vasild:
    ACK e7cf8657e1
  luke-jr:
    utACK e7cf8657e1

Tree-SHA512: 3db8415dea6f82223d11a23bd6cbb3b8cf68831321280e926034a1f110cbe22562570013925f6fa20d8f08e41d0202fd69c733d9f16217318a660d2a1a21b795
2023-07-13 13:50:58 -04:00
MarcoFalke
000019e158 Add AutoFile::detail_fread member function
New code can call the method without having first to retrieve the raw
FILE* pointer via Get().

Also, move implementation to the cpp file. Can be reviewed with:
--color-moved=dimmed-zebra  --color-moved-ws=ignore-all-space
2023-07-13 11:51:07 +02:00
MarcoFalke
fa7724bc9d refactor: Modernize AutoFile
* Add m_ prefix to the std::FILE member variable
* Add std namespace where possible, to avoid confusion with member
  functions of the same name.
* Add AutoFile::feof() member function, to be used in place of
  std::feof(AutoFile::Get())
* Simplify fclose() in terms of release()
* Fix typo in the error message in the ignore member function.
2023-07-13 11:50:55 +02:00
Pieter Wuille
4e5c933f6a Switch all callers from poly1305_auth to Poly1305 class
This also removes the old poly1305_auth interface, as it no longer serves any
function. The new Poly1305 class based interface is more modern and safe.
2023-07-12 22:43:55 -04:00
Pieter Wuille
8871f7d1ae tests: add more Poly1305 test vectors 2023-07-12 22:43:52 -04:00
Pieter Wuille
40e6c5b9fc crypto: add Poly1305 class with std::byte Span interface 2023-07-12 22:40:55 -04:00
Pieter Wuille
50269b391f crypto: switch poly1305 to incremental implementation
This code is taken from poly1305-donna-32.h, poly1305-donna.h, poly1305-donna.c
from https://github.com/floodyberry/poly1305-donna, commit
e6ad6e091d30d7f4ec2d4f978be1fcfcbce72781, with the following modifications:

* Coding style (braces around one-line indented if/for loops).
* Rename unsigned long (long) to uint32_t and uint64_t.
* Rename poly1305_block_size to POLY1305_BLOCK_SIZE.
* Adding noexcept to functions.
* Merging poly1305_state_internal_t and poly1305_context types.
* Merging code from multiple files.
* Place all imported code in the poly1305_donna namespace.
2023-07-12 14:47:37 -04:00
Andrew Chow
b4794740f8 Merge bitcoin/bitcoin#27985: Add support for RFC8439 variant of ChaCha20
0bf87476f5 test: add ChaCha20 test triggering 32-bit block counter overflow (Sebastian Falbesoner)
7f2a985147 tests: improve ChaCha20 unit tests (Pieter Wuille)
511a8d406e crypto: Implement RFC8439-compatible variant of ChaCha20 (Pieter Wuille)

Pull request description:

  Based on and replaces part of #25361, part of the BIP324 project (#27634). See also #19225 for background.

  There are two variants of ChaCha20 in use. The currently implemented one uses a 64-bit nonce and a 64-bit block counter, while the one used in RFC8439 (and thus BIP324) uses a 96-bit nonce and 32-bit block counter. This PR changes the logic to use the 96-bit nonce variant, though in a way that's compatible with >256 GiB output (by automatically incrementing the first 32-bit part of the nonce when the block counter overflows).

  For those who reviewed the original PR, the biggest change is here that the 96-bit nonce is passed as a Nonce96 type (pair of 32-bit + 64-bit integer) rather than a 12-byte array.

ACKs for top commit:
  achow101:
    ACK 0bf87476f5
  theStack:
    Code-review ACK 0bf87476f5

Tree-SHA512: 62e4cbd5388b8d50ef1a0dc99b6f4ad36c7b4419032035f8e622dda63a62311dd923032217e20054bcd836865d4be5c074f9e5538ca158f94f08eab75c5519c1
2023-07-12 12:58:44 -04:00
MarcoFalke
fa6245da60 fuzz: Generate process_message targets individually
Also, add an "rpc" target without LIMIT_TO_RPC_COMMAND set.
2023-07-12 15:52:14 +02:00
MarcoFalke
fa4ccf1511 ci: Add missing -O2 to valgrind tasks 2023-07-12 15:41:49 +02:00
fanquake
8f6f0d81ee guix: backport glibc patch to prevent redundant librt link 2023-07-12 11:22:02 +01:00
fanquake
e14473299c contrib: remove librt from release deps 2023-07-12 09:09:14 +01:00
fanquake
e64e17830a build: remove check for gettimeofday & librt 2023-07-12 09:09:14 +01:00
MarcoFalke
fa8d227d58 doc: Remove comments that just repeat what the code does
No need to artificially bloat the code and waste space.
2023-07-12 10:01:51 +02:00
MarcoFalke
fafe2ca0ce refactor: Remove redundant file check from AutoFile shift operators
The shift operators will call the write or read member function, which
already does the check. Also, call sites are free to directly call
::(Un)Serialize(s, obj) to skip this check, so removing it increases
consistency.
2023-07-12 10:00:56 +02:00
MarcoFalke
9999a49b32 Extract util::Xor, Add key_offset option, Add bench 2023-07-12 09:59:55 +02:00
Andrew Chow
357e3f6aa4 Merge bitcoin/bitcoin#28025: test: refactor: deduplicate legacy ECDSA signing for tx inputs
5cf44275c8 test: refactor: deduplicate legacy ECDSA signing for tx inputs (Sebastian Falbesoner)

Pull request description:

  There are several instances in functional tests and the framework (MiniWallet, feature_block.py, p2p_segwit.py) where we create a legacy ECDSA signature for a certain transaction's input by doing the following steps:

  1. calculate the `LegacySignatureHash` with the desired sighash type
  2. create the actual digital signature by calling `ECKey.sign_ecdsa` on the signature message hash calculated above
  3. put the DER-encoded result as CScript data push into tx input's scriptSig

  Create a new helper `sign_input_legacy` which hides those details and takes only the necessary parameters (tx, input index, relevant scriptPubKey, private key, sighash type [SIGHASH_ALL by default]). For further convenience, the signature is prepended to already existing data-pushes in scriptSig, in order to avoid rehashing the transaction after calling the new signing function.

ACKs for top commit:
  dimitaracev:
    ACK `5cf4427`
  achow101:
    ACK 5cf44275c8
  pinheadmz:
    ACK 5cf44275c8

Tree-SHA512: 8f0e4fb2c3e0f84fac5dbc4dda87973276242b0f628034272a7f3e45434c1e17dd1b26a37edfb302dcaf380dbfe98b0417391ace5e0ac9720155d8fba702031e
2023-07-11 17:25:40 -04:00
MarcoFalke
fa1471e575 refactor: Remove duplicate allNetMessageTypesVec
With C++11 (and later), the duplicate variable is no longer needed.

Also, run clang-format on the namespace, as the script in the next
commit relies on a specific format. This prevents a clang-format run in
the future from breaking the script. (Review hint: --ignore-all-space).
2023-07-11 19:26:52 +02:00
Ryan Ofsky
31eca93a9e kernel: Remove StartShutdown calls from validation code
This change drops the last kernel dependency on shutdown.cpp. It also adds new
hooks for libbitcoinkernel applications to be able to interrupt kernel
operations when the chain tip changes.

This is a refactoring that does not affect behavior. (Looking at the code it
can appear like the new break statement in the ActivateBestChain function is a
change in behavior, but actually the previous StartShutdown call was indirectly
triggering a break before, because it was causing m_chainman.m_interrupt to be
true. The new code just makes the break more obvious.)
2023-07-11 12:30:56 -04:00
Ryan Ofsky
99b3af78bd Merge bitcoin/bitcoin#28044: test: indexes, fix on error infinite loop
89ba8905f5 test: indexes, fix on error infinite loop (furszy)

Pull request description:

  Coming from https://github.com/bitcoin/bitcoin/pull/28036#issuecomment-1623813703, I thought that we were going to fix it there but seems that got merged without it for some reason.

  As index sync failures trigger a shutdown request without notifying `BaseIndex::BlockUntilSyncedToCurrentChain` in any way, we also need to check whether a shutdown was requested or not inside 'IndexWaitSynced'.

  Otherwise, any error inside the index sync process will hang the test forever.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 89ba8905f5
  jamesob:
    ACK 89ba890
  ryanofsky:
    Code review ACK 89ba8905f5. Just comment update since last review

Tree-SHA512: 1f6daf34e51d3fbc802799bfa4ac0ef0d8f774db5f9e2f5d35df18a77679778475c94efc3da1fb723ebaf3583e4075e4a5cbe4a5104ad0c50e2b32076e247b29
2023-07-11 12:30:56 -04:00
furszy
286e0c7d5e wallet: loading, log descriptor parsing error details
The `UNKNOWN_DESCRIPTOR` error comes from the
`WalletDescriptor::DeserializeDescriptor` std::ios_base
exception, which contains further information about the
parsing error.
2023-07-11 11:34:25 -03:00
Ryan Ofsky
e253568da8 Merge bitcoin/bitcoin#28053: refactor: Move stopafterblockimport option out of blockstorage
462390c85f refactor: Move stopafterblockimport handling out of blockstorage (TheCharlatan)

Pull request description:

  This has the benefit of moving this StartShutdown call out of the blockstorage file and thus out of the kernel's responsibility. The user can now decide if he wants to start shutdown / interrupt after a block import or not.

  This also simplifies https://github.com/bitcoin/bitcoin/pull/28048, making it one fewer shutdown call to handle.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 462390c85f  🗝
  ryanofsky:
    Code review ACK 462390c85f. Just has been rebased and is a simpler change after #27607

Tree-SHA512: 84e58256b1c61f10e7ec5ecf32916f40a2ab1ea7cce703de0fa1c61ee2be94bd45ed32718bc99903b6eff3e6d3d5b506470bf567ddbb444a58232913918e8ab8
2023-07-11 09:47:06 -04:00
TheCharlatan
462390c85f refactor: Move stopafterblockimport handling out of blockstorage
This has the benefit of moving the StartShutdown call out of the
blockstorage file and thus out of the kernel's responsibility. The user
can now decide if he wants to start shutdown / interrupt after a block
import or not.
2023-07-11 12:00:57 +02:00
fanquake
21ed784614 Merge bitcoin/bitcoin#28028: test: Check expected_stderr after stop
faf902858d test: Check expected_stderr after stop (MarcoFalke)

Pull request description:

  This fixes a bug where stderr wasn't checked for the shutdown sequence.

  Fix that by waiting for the shutdown to finish and then check stderr.

ACKs for top commit:
  theStack:
    ACK faf902858d

Tree-SHA512: a70cd1e6cda84d542782e41e8b59741dbcd472c0d0575bcef5cbfd1418473ce94efe921481d557bae3fbbdd78f1c49c09c48872883c052d87c5c9a9a51492692
2023-07-11 10:14:48 +01:00
furszy
89ba8905f5 test: indexes, fix on error infinite loop
As index sync failures trigger a shutdown request without notifying
BaseIndex::BlockUntilSyncedToCurrentChain in any way, we also need
to check whether a shutdown was requested or not inside 'IndexWaitSynced'.

Otherwise, any error inside the index sync process will hang the test
forever.
2023-07-10 15:27:13 -03:00
Ryan Ofsky
ef29d5d7e2 Merge bitcoin/bitcoin#27607: index: make startup more efficient
ca91c244ef index: verify blocks data existence only once (furszy)
fcbdaeef4d init: don't start indexes sync thread prematurely (furszy)
2ec89f1970 refactor: simplify pruning violation check (furszy)
c82ef91eae make GetFirstStoredBlock assert that 'start_block' always has data (furszy)
430e7027a1 refactor: index, decouple 'Init' from 'Start' (furszy)
225e213110 refactor: init indexes, decouple 'Start()' from the creation step (furszy)
2ebc7e68cc doc: describe 'init load' thread actions (Martin Zumsande)
04575106b2 scripted-diff: rename 'loadblk' thread name to 'initload' (furszy)
ed4462cc78 init: start indexes sync earlier (furszy)

Pull request description:

  Simplifies index startup code, eliminating the `g_indexes_ready_to_sync` variable,
  deduplicating code and moving the prune violation check out of the `BaseIndex` class.

  Also makes startup more efficient by running the prune violation check once for all indexes
  instead of once for each index, and by delaying the prune violation check and moving it off
  of the main thread so the node can start up faster and perform the block data availability
  verification even when the '-reindex" or the "-reindex-chainstate" flags are enabled (which
  hasn't  being possible so far).

ACKs for top commit:
  ryanofsky:
    Code review ACK ca91c244ef. Just rebase and suggested changes since last review (Start return check, and code simplification)
  TheCharlatan:
    re-ACK ca91c244ef

Tree-SHA512: e9c98ce89aeb29e8d0f505f17b34aa54fe44efefbf017f4746e3b446ab4de25ade4f707254a0bbe4b99b69731b04a4067ce529eb7aa834ced196784b694cf7ce
2023-07-10 11:56:11 -04:00
fanquake
c464e67e0b Merge bitcoin/bitcoin#28050: test: make assumeUTXO test capture the expected fatal error
3e8bf2e10c test: make assumeUTXO test capture the expected fatal error (furszy)

Pull request description:

  The test is exercising the error, so it can capture it before the
  test framework displays it on the console as an unforeseen
  fatal error.

  It is odd to observe a fatal error after executing the complete
  test suite and seeing it pass successfully.

  Reproduction Steps:
  Run the unit test suite. A long AssumeUTXO fatal error will be
  printed even when all tests pass successfully.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 3e8bf2e10c
  theStack:
    Tested ACK 3e8bf2e10c
  TheCharlatan:
    ACK 3e8bf2e10c

Tree-SHA512: 820a5a4db52085ed72cbe7eb433b8c0f2d283ac6f5d456bc2b3e3f0305301022b2729e32e5fd9002859e4491ae7ac6de568a4c20557c7b249b0e7694ab8bd177
2023-07-10 16:22:45 +01:00
Matthew Zipkin
b4bee4bbf4 test: add keep_alive option to socks5 proxy in test_framework
The Socks5 server we use in the test framework would disconnect
by default immediately after the handshake and sometimes would
not register as a connected peer by bitcoind.
2023-07-10 10:07:45 -04:00
Matthew Zipkin
5aaf988ccc test: cover TorV3 address in p2p_addrv2_relay 2023-07-10 10:07:45 -04:00
brunoerg
80f64a3d40 test: add support for all networks in CAddress in messages.py
Also removes TorV2 from messages.py
See https://github.com/bitcoin/bitcoin/pull/22050

Co-authored-by: Matthew Zipkin <pinheadmz@gmail.com>
2023-07-10 10:07:40 -04:00
furszy
ca91c244ef index: verify blocks data existence only once
At present, during init, we traverse the chain (once per index)
to confirm that all necessary blocks to sync each index up to
the current tip are present.

To make the process more efficient, we can fetch the oldest block
from the indexers and perform the chain data existence check from
that point only once.

This also moves the pruning violation check to the end of the
'loadinit' thread, which is where the reindex, block loading and
chain activation processes happen.

Making the node's startup process faster, allowing us to remove
the global g_indexes_ready_to_sync flag, and enabling the
execution of the pruning violation verification even when the
reindex or reindex-chainstate flags are enabled (which has being
skipped so far).
2023-07-10 10:50:50 -03:00
furszy
fcbdaeef4d init: don't start indexes sync thread prematurely
By moving the 'StartIndexes()' call into the 'initload'
thread, we can remove the threads active wait. Optimizing
the available resources.

The only difference with the current state is that now the
indexes threads will only be started when they can process
work and not before it.
2023-07-10 10:50:50 -03:00
furszy
2ec89f1970 refactor: simplify pruning violation check
By generalizing 'GetFirstStoredBlock' and implementing
'CheckBlockDataAvailability' we can dedup code and
avoid repeating work when multiple indexes are enabled.
E.g. get the oldest block across all indexes and
perform the pruning violation check from that point
up to the tip only once (this feature is being introduced
in a follow-up commit).

This commit shouldn't change behavior in any way.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-07-10 10:50:50 -03:00
furszy
c82ef91eae make GetFirstStoredBlock assert that 'start_block' always has data
And transfer the responsibility of verifying whether 'start_block'
has data or not to the caller.

This is because the 'GetFirstStoredBlock' function responsibility
is to return the first block containing data. And the current
implementation can return 'start_block' when it has no data!. Which
is misleading at least.

Edge case behavior change:
Previously, if the block tip lacked data but all preceding blocks
contained data, there was no prune violation. And now, such
scenario will result in a prune violation.
2023-07-10 10:47:17 -03:00
furszy
430e7027a1 refactor: index, decouple 'Init' from 'Start'
So indexes can be initialized without spawning
the sync thread.

This makes asynchronous indexes startup
possible in the following commits.
2023-07-10 10:47:17 -03:00
MarcoFalke
faf902858d test: Check expected_stderr after stop 2023-07-10 13:45:50 +02:00
Sebastian Falbesoner
0bf87476f5 test: add ChaCha20 test triggering 32-bit block counter overflow
Verify that our ChaCha20 implementation using the 96/32 split interface
is compatible with >256 GiB outputs by triggering a 32-bit block counter
overflow and checking that the keystream matches one created with an
alternative implementation using a 64/64 split interface with the
corresponding input data. The test case data was generated with the
following Python script using the PyCryptodome library (version 3.15.0):

----------------------------------------------------------------------------------------------
from Crypto.Cipher import ChaCha20
key = bytes(list(range(32))); nonce = 0xdeadbeef12345678; pos = 2**32 - 1
c = ChaCha20.new(key=key, nonce=nonce.to_bytes(8, 'little'))
c.seek(pos * 64); stream = c.encrypt(bytes([0])*128)
print(f"Key: {key.hex()}\nNonce: {hex(nonce)}\nPos: {hex(pos)}\nStream: {stream.hex()}")
----------------------------------------------------------------------------------------------
2023-07-09 10:08:41 -04:00
furszy
3e8bf2e10c test: make assumeUTXO test capture the expected fatal error
The test is exercising the error, so it can capture it before
the test framework displays it on the console as an unforeseen
fatal error.
2023-07-08 11:45:34 -03:00
furszy
225e213110 refactor: init indexes, decouple 'Start()' from the creation step
No behavior change.

The goal here is to group indexes, so we can perform the same
initialization and verification process equally for all of them.

The checks performed inside `StartIndexes` will be expanded
in the subsequent commits.
2023-07-07 19:31:27 -03:00
Martin Zumsande
2ebc7e68cc doc: describe 'init load' thread actions 2023-07-07 19:31:27 -03:00
furszy
04575106b2 scripted-diff: rename 'loadblk' thread name to 'initload'
The thread does not only load blocks, it loads the mempool and,
in a future commit, will start the indexes as well.

Also, renamed the 'ThreadImport' function to 'ImportBlocks'
And the 'm_load_block' class member to 'm_thread_load'.

-BEGIN VERIFY SCRIPT-

sed -i "s/ThreadImport/ImportBlocks/g" $(git grep -l ThreadImport -- ':!/doc/')
sed -i "s/loadblk/initload/g" $(git grep -l loadblk -- ':!/doc/release-notes/')
sed -i "s/m_load_block/m_thread_load/g" $(git grep -l m_load_block)

-END VERIFY SCRIPT-
2023-07-07 19:31:27 -03:00
furszy
ed4462cc78 init: start indexes sync earlier
The mempool load can take a while, and it is not
needed for the indexes' synchronization.

Also, having the mempool load function call
inside 'blockstorage.cpp' wasn't structurally
correct.
2023-07-07 19:31:26 -03:00
Pieter Wuille
7f2a985147 tests: improve ChaCha20 unit tests 2023-07-07 17:17:20 -04:00
Pieter Wuille
511a8d406e crypto: Implement RFC8439-compatible variant of ChaCha20
There are two variants of ChaCha20 in use. The original one uses a 64-bit
nonce and a 64-bit block counter, while the one used in RFC8439 uses a
96-bit nonce and 32-bit block counter. This commit changes the interface
to use the 96/32 split (but automatically incrementing the first 32-bit
part of the nonce when the 32-bit block counter overflows, so to retain
compatibility with >256 GiB output).

Simultaneously, also merge the SetIV and Seek64 functions, as we almost
always call both anyway.

Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-07-07 17:16:27 -04:00
Andrew Chow
79e8247ddb Merge bitcoin/bitcoin#28039: wallet: don't include bdb files from our headers
8b5397c00e wallet: bdb: include bdb header from our implementation files only (Cory Fields)
6e010626af wallet: bdb: don't use bdb define in header (Cory Fields)
004b184b02 wallet: bdb: move BerkeleyDatabase constructor to cpp file (Cory Fields)
b3582baa3a wallet: bdb: move SafeDbt to cpp file (Cory Fields)
e5e5aa1da2 wallet: bdb: move SpanFromDbt to below SafeDbt's implementation (Cory Fields)
4216f69250 wallet: bdb: move TxnBegin to cpp file since it uses a bdb function (Cory Fields)
43369f3706 wallet: bdb: drop default parameter (Cory Fields)

Pull request description:

  Only `#include` upstream bdb headers from our cpp files.

  It's generally good practice to avoid including 3rd party deps in headers as otherwise they tend to sneak into new compilation units. IMO this makes for a nice cleanup.

  There's a good bit of code movement here, but each commit is small and _should_ be obviously correct.

  Note: in the future, the buildsystem can add the bdb include path for `bdb.cpp` and `salvage.cpp` only, rather than all wallet sources.

ACKs for top commit:
  achow101:
    reACK 8b5397c00e
  hebasto:
    ACK 8b5397c00e

Tree-SHA512: 0ef6e8a9c4c6e2d1e5d6a3534495f91900e4175143911a5848258c56da54535b85fad67b6d573da5f7b96e7881299b5a8ca2327e708f305b317b9a3e85038d66
2023-07-07 13:43:28 -04:00
fanquake
87e19b047c Merge bitcoin/bitcoin#28038: wallet: address book migration bug fixes
7ecc29a0b7 test: wallet, add coverage for addressbook migration (furszy)
a277f8357a wallet: migration bugfix, persist empty labels (furszy)
1b64f6498c wallet: migration bugfix, clone 'send' record label to all wallets (furszy)

Pull request description:

  Addressing two specific bugs encountered during the wallet migration process, related to the address book, and improves the test coverage for it.

  Bug 1: Non-Cloning of External 'Send' Records
  The external 'send' records were not being correctly cloned to all wallets.

  Bug 2: Persistence of Empty Labels
  As address book entries without associated db label records can be treated as change (the `label` field inside the `CAddressBookData` class is optional, `nullopt` labels make `CAddressBookData ::IsChange()` return true), we must persist empty labels during the migration process.
  The user might have called `setlabel` with an "" string for an external address and that must be retained during migration.

ACKs for top commit:
  achow101:
    ACK 7ecc29a0b7

Tree-SHA512: b8a8483a4178a37c49af11eb7ba8a82ca95e54a6cd799e155e33f9fbe7f37b259e28372c77d6944d46b6765f9eaca6b8ca8d1cdd9d223120a3653e4e41d0b6b7
2023-07-07 17:30:07 +01:00
Cory Fields
8b5397c00e wallet: bdb: include bdb header from our implementation files only
This way the dependency can't sneak into other files without being noticed.

Forward-declare bdb classes as necessary.
2023-07-07 14:14:20 +00:00
Cory Fields
6e010626af wallet: bdb: don't use bdb define in header 2023-07-07 14:14:20 +00:00
Cory Fields
004b184b02 wallet: bdb: move BerkeleyDatabase constructor to cpp file
Else some compilers/stdlibs may not be able to construct
std::unique_ptr<Db> without Db defined.
2023-07-07 14:14:20 +00:00
Cory Fields
b3582baa3a wallet: bdb: move SafeDbt to cpp file
Dbt requires including bdb headers.
2023-07-07 14:14:20 +00:00
Cory Fields
e5e5aa1da2 wallet: bdb: move SpanFromDbt to below SafeDbt's implementation
No functional change, just simplifies the code move in the next commit.
2023-07-07 14:14:20 +00:00
Sebastian Falbesoner
bbbb89d238 test: miner: add coverage for -blockmintxfee setting
Co-authored-by: glozow <gloriajzhao@gmail.com>
2023-07-07 15:56:24 +02:00
fanquake
cf4da5ec29 Merge bitcoin/bitcoin#28015: fuzz: Generate rpc fuzz targets individually
fa1e27fe8e fuzz: Generate rpc fuzz targets individually (MarcoFalke)

Pull request description:

  The `rpc` fuzz target was added more than two years ago in e45863166f. However, the bug https://github.com/bitcoin/bitcoin/issues/27913 was only found recently. Thus, it is pretty clear that fuzz engines can't deal with a search space that is too broad and can be extended in too many directions.

  Fix that by limiting the search space to each RPC method name and then iterate over all names, instead of letting the fuzz engine do the iteration.

  With this, the bug can be found in seconds, as opposed to years of CPU time (or never).

ACKs for top commit:
  brunoerg:
    ACK fa1e27fe8e
  dergoegge:
    ACK fa1e27fe8e

Tree-SHA512: 45ccba842367650d010320603153276b1b303deda9ba8c6bb31a4d2473b00aa5bca866db95f541485d65efd8276e2575026968c037872ef344fa33cf45bcdcd7
2023-07-07 11:26:22 +01:00
fanquake
a9dde841e1 Merge bitcoin/bitcoin#28012: util: Allow FastRandomContext::randbytes for std::byte, Allow std::byte serialization
fac6af16f4 Allow std::byte serialization (MarcoFalke)
fade43edc4 Allow FastRandomContext::randbytes for all byte types (MarcoFalke)

Pull request description:

  I need this for some stuff, but it should also be useful by itself for other developers that need it.

ACKs for top commit:
  sipa:
    utACK fac6af16f4
  dergoegge:
    Code review ACK fac6af16f4

Tree-SHA512: db4b1bbd6bf6ef6503d59b0b4ed1681db8d935d2d10f8d89f071978ea59b49a1d319bccb4e9717c0c88a4908bbeca4fd0cbff6c655d8a443554fd14146fe16de
2023-07-07 10:48:09 +01:00
fanquake
299f17a69e Merge bitcoin/bitcoin#28036: test: Restore unlimited timeout in IndexWaitSynced
fabed7eb79 test: Restore unlimited timeout in IndexWaitSynced (MarcoFalke)

Pull request description:

  The timeout was unlimited before, so just restore that value for now: https://github.com/bitcoin/bitcoin/pull/27988#issuecomment-1619218007 .

  (Strictly speaking, this is a behavior change for the blockfilterindex and txindex tests, because it only restores the coinstatsindex behavior.)

ACKs for top commit:
  ajtowns:
    utACK fabed7eb79
  mzumsande:
    ACK fabed7eb79
  furszy:
    ACK fabed7eb

Tree-SHA512: 66a878be58bbe53ad8e0c23f05569dd42df688be747551fbd202ada22d20a8285714e58fa2a71664deadb070ddf86cfad88c01042ff95ed26f6b40e4a10cec0a
2023-07-07 09:57:37 +01:00
fanquake
334f45fe62 Merge bitcoin/bitcoin#28040: wallet: sqlite: don't include sqlite files from our headers
bea9fc2600 wallet: sqlite: force sqlite3.h to be included by the cpp files (Cory Fields)

Pull request description:

  Only `#include` upstream sqlite headers from our cpp files.

  Like #28039 but simpler :)

ACKs for top commit:
  achow101:
    ACK bea9fc2600
  TheCharlatan:
    Nice, ACK bea9fc2600
  kristapsk:
    utACK bea9fc2600
  hebasto:
    ACK bea9fc2600, I have reviewed the code and it looks OK.

Tree-SHA512: cb83ac51eed7e0740f1c75ee87c7849fa7e535bc4836c499290041eb995ccfd82533e3babfe83a164257b62b180f206112d6a1bae7ea290ad0ec7f55d62432da
2023-07-07 09:39:48 +01:00
Ryan Ofsky
75135c673e Merge bitcoin/bitcoin#27861: kernel: Rm ShutdownRequested and AbortNode from validation code.
6eb33bd0c2 kernel: Add fatalError method to notifications (TheCharlatan)
7320db96f8 kernel: Add flushError method to notifications (TheCharlatan)
3fa9094b92 scripted-diff: Rename FatalError to FatalErrorf (TheCharlatan)
edb55e2777 kernel: Pass interrupt reference to chainman (TheCharlatan)
e2d680a32d util: Add SignalInterrupt class and use in shutdown.cpp (TheCharlatan)

Pull request description:

  Get rid of all `ShutdownRequested` calls in validation code by introducing an interrupt object that applications can use to cancel long-running kernel operations.

  Replace all `AbortNode` calls in validation code with new fatal error and flush error notifications so kernel applications can be notified about failures and choose how to handle them.

  ---

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/27587 https://github.com/orgs/bitcoin/projects/3 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel".

  The pull request mostly allows dropping the kernel dependency on shutdown.cpp. The only dependency left after this is a `StartShutdown` call which will be removed in followup PR https://github.com/bitcoin/bitcoin/pull/27711. This PR also drops the last reference to the `uiInterface` global in kernel code. The process of moving the `uiInterface` out of the kernel was started in https://github.com/bitcoin/bitcoin/pull/27636.

  This pull request contains a subset of patches originally proposed in #27711. It will be part of a series of changes required to make handling of interrupts (or in other words the current shutdown procedure) in the kernel library more transparent and less reliable on global mutable state. The set of patches contained here was originally proposed by @ryanofsky [here](https://github.com/bitcoin/bitcoin/pull/27711#issuecomment-1580779869).

ACKs for top commit:
  achow101:
    light ACK 6eb33bd0c2
  hebasto:
    ACK 6eb33bd0c2, I have reviewed the code and it looks OK.
  ryanofsky:
    Code review ACK 6eb33bd0c2. No changes since last review other than rebase.

Tree-SHA512: 7d2d05fa4805428a09466d43c11ae32946cbb25aa5e741b1eec9cd142e4de4bb311e13ebf1bb125ae490c9d08274f2d56c93314e10f3d69e7fec7445e504987c
2023-07-06 17:07:27 -04:00
Cory Fields
4216f69250 wallet: bdb: move TxnBegin to cpp file since it uses a bdb function 2023-07-06 20:16:55 +00:00
Cory Fields
43369f3706 wallet: bdb: drop default parameter 2023-07-06 20:16:53 +00:00
Cory Fields
bea9fc2600 wallet: sqlite: force sqlite3.h to be included by the cpp files
This way sqlite usage is explicit.
2023-07-06 19:53:44 +00:00
furszy
7ecc29a0b7 test: wallet, add coverage for addressbook migration 2023-07-06 16:11:55 -03:00
furszy
a277f8357a wallet: migration bugfix, persist empty labels
addressbook records with no associated label could be
treated as change. And we don't want that for external
addresses.
2023-07-06 16:11:55 -03:00
furszy
1b64f6498c wallet: migration bugfix, clone 'send' record label to all wallets 2023-07-06 13:48:11 -03:00
Vasil Dimov
20b49460b3 test: remove race in the user-agent reception check
In `add_p2p_connection()` we connect to `bitcoind` from the Python
client and check that it has received our version string.

This check looked up the last/newest entry from `getpeerinfo` RPC,
assuming that it must be the connection we have just opened. But this
will not be the case if a new inbound or outbound connection is made
to/from `bitcoind` in the meantime.

Instead of the last entry in `getpeerinfo`, check all and find the one
which corresponds to our connection using our outgoing address:port
tuple which is unique.

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Co-authored-by: Jon Atack <jon@atack.com>
2023-07-06 17:48:15 +02:00
fanquake
c325f0fbae Merge bitcoin/bitcoin#28034: ci: Print full lscpu output
fa956d2048 ci: Print full lscpu output (MarcoFalke)

Pull request description:

  Seems odd to withhold the other output, given that it may be useful to debug issues?

ACKs for top commit:
  fanquake:
    ACK fa956d2048

Tree-SHA512: d93a79734a594c2ee180107e3ed1d1c07c1b6324b4b1e239d3f263e72490ca641f60a4e80793229523e2d52059958a896cb210014e3aa747a19871be62f5a961
2023-07-06 13:21:18 +01:00
MarcoFalke
fabed7eb79 test: Restore unlimited timeout in IndexWaitSynced
The timeout was unlimited before, so just restore that value for now:
https://github.com/bitcoin/bitcoin/pull/27988#issuecomment-1619218007
2023-07-06 14:19:59 +02:00
glozow
ecf3baffc0 Merge bitcoin/bitcoin#27869: wallet: Give deprecation warning when loading a legacy wallet
8fbb6e99bf wallet: Give deprecation warning when loading a legacy wallet (Andrew Chow)

Pull request description:

  Next step in legacy wallet deprecation.

ACKs for top commit:
  S3RK:
    reACK 8fbb6e99bf
  jonatack:
    re-ACK 8fbb6e99bf

Tree-SHA512: 902984b09452926cf199f06e5fb56e4985325cdd5e0dcc829992158488f42d5fbc33e9a30a29303feac24c8315193e8d31712022e2a0503abd6b67169a0027f4
2023-07-06 10:47:41 +01:00
MarcoFalke
fa956d2048 ci: Print full lscpu output 2023-07-06 11:13:05 +02:00
fanquake
bc4f6b13fe Merge bitcoin/bitcoin#28024: ci: Remove deprecated container.greedy
fac14c4e49 ci: Remove deprecated container.greedy (MarcoFalke)

Pull request description:

  The option is to be phased out, so remove it to avoid relying on it. Update container.cpu where needed.

ACKs for top commit:
  hebasto:
    ACK fac14c4e49.

Tree-SHA512: 0440b710e607aaa2f78f811f9d5ae786a59af4a44861d7905a25c742ff7f5b4558518b939539455309503266757182a1c8fce92c8d3430983b2103a613fe01d7
2023-07-04 17:43:48 +01:00
Hennadii Stepanov
c71a96c431 Merge bitcoin-core/gui#696: Switch RPCConsole wallet selection to the one most recently opened/restored/created
99c0eb9701 Fix RPCConsole wallet selection (John Moffett)

Pull request description:

  If a user opens multiple wallets in the GUI from the menu bar, the last one opened is the active one in the main window. However, For the RPC Console window, the  _first_ one opened is active. This can be confusing, as wallet RPC commands may be sent to a wallet the user didn't intend.

  This PR makes the RPC Console switch to the wallet just opened / restored / created from the menu bar, which is how the main GUI now works.

  Similar to https://github.com/bitcoin-core/gui/pull/665 and specifically requested [in a comment](https://github.com/bitcoin-core/gui/pull/665#issuecomment-1270003660).

ACKs for top commit:
  luke-jr:
    utACK 99c0eb9701
  hebasto:
    ACK 99c0eb9701, tested on Ubuntu 23.04.

Tree-SHA512: d5e5acdaa114130ad4d27fd3f25393bc8d02d92b5001cd39352601d04283cdad3bd62c4da6d369c69764e3b188e9cd3e83152c00b09bd42966082ad09037c328
2023-07-04 16:45:54 +01:00
Hennadii Stepanov
7446cb186c Merge bitcoin-core/gui#719: Remove confusing "Dust" label from coincontrol / sendcoins dialog
a582b4141f gui: send, left alignment for "bytes" and "change" label (furszy)
210ef1e980 qt: remove confusing "Dust" label from coincontrol / sendcoins dialog (Sebastian Falbesoner)

Pull request description:

  In contrast to to all other labels on the coin selection dialog, the displayed dust information has nothing to do with the selected coins. All that this label shows is whether at least one of the _outputs_ qualify as dust, but the outputs are set in a different dialog. (Even worse, the dust check is currently simply wrong because it only looks at an output's nValue and just assumes a P2PKH script size.)

  As the label clearly doesn't help the user and is, quite the contrary, rather increasing confusion/misguidance, it seems sensible to remove it. The label from the sendcoins dialog is also removed with the same rationale. Additionally, the "bytes" and "change" labels are aligned to the left (second commit).

  Closes https://github.com/bitcoin-core/gui/issues/699.

ACKs for top commit:
  furszy:
    ACK a582b41
  hebasto:
    Looks good. ACK a582b4141f.

Tree-SHA512: ebc00b68bdeab69f6ab643e4b89301a7e3d04a8a4027b50813314ddddb1387bc97a83313851e375dfbce97751c234686c82af7f4e55fa5ef29f4fed4e8fc11d9
2023-07-04 16:03:30 +01:00
MarcoFalke
fac14c4e49 ci: Remove deprecated container.greedy
The option is to be phased out, so remove it to avoid relying on it.
Update container.cpu and timeouts where needed.
2023-07-04 10:41:25 +02:00
Andrew Chow
f08d914a67 Merge bitcoin/bitcoin#27920: wallet: bugfix, always use apostrophe for spkm descriptor ID
5df988b534 test: add coverage for descriptor ID (furszy)
6a9510d2da wallet: bugfix, always use apostrophe for spkm descriptor ID (furszy)
97a965d98f refactor: extract descriptor ID calculation from spkm GetID() (furszy)
1d207e3931 wallet: do not allow loading descriptor with an invalid ID (furszy)

Pull request description:

  Aiming to fix #27915.

  As we re-write the descriptor's db record every time that
  the wallet is loaded (at `TopUp` time), if the spkm ID differs
  from the one in db, the wallet will enter in an unrecoverable
  corruption state (due to the storage of a descriptor with an ID
  that is not linked to any other descriptor record in DB), and
  no soft version will be able to open it anymore.

  Because we cannot change the past, to stay compatible between
  releases, we need to always use the apostrophe version for the
  spkm IDs.

ACKs for top commit:
  achow101:
    ACK 5df988b534
  Sjors:
    tACK 5df988b534

Tree-SHA512: f63fc4aac7d21a4e515657471758d28857575e751865bfa359298f8b89b2568970029ca487a873c1786a5716325f453f06cd417ed193f3366417f6e8c2987332
2023-07-03 21:42:01 -04:00
furszy
a582b4141f gui: send, left alignment for "bytes" and "change" label 2023-07-03 18:40:56 +02:00
John Moffett
99c0eb9701 Fix RPCConsole wallet selection
If a user opens multiple wallets in the GUI from the
menu bar, the last one opened is the active one in
the main window. However, For the RPC Console window,
the  _first_ one opened is active. This can be
confusing, as wallet RPC commands may be sent to a
wallet the user didn't intend.

This commit makes the RPC Console switch to the wallet
opened from the menu bar.
2023-07-03 12:07:30 -04:00
Sebastian Falbesoner
210ef1e980 qt: remove confusing "Dust" label from coincontrol / sendcoins dialog
In contrast to to all other labels on the coin selection dialog, the
displayed dust information has nothing to do with the selected coins.
All that this label shows is whether at least one of the _outputs_
qualify as dust, but the outputs are set in a different dialog.
(Even worse, the dust check is currently simply wrong because it only
looks at an output's nValue and just assumes a P2PKH script size.)

As the label clearly doesn't help the user and is, quite the contrary,
rather increasing confusion/misguidance, it seems sensible to remove it.

Also, remove the label from the sendcoins dialog with the same rationale.
2023-07-03 18:06:48 +02:00
Sebastian Falbesoner
5cf44275c8 test: refactor: deduplicate legacy ECDSA signing for tx inputs
There are several instances in functional tests and the framework
(MiniWallet, feature_block.py, p2p_segwit.py) where we create a legacy
ECDSA signature for a certain transaction's input by doing the following
steps:
    1) calculate the `LegacySignatureHash` with the desired sighash type
    2) create the actual digital signature by calling `ECKey.sign_ecdsa`
       on the signature message hash calculated above
    3) put the DER-encoded result as CScript data push into
       tx input's scriptSig

Create a new helper `sign_input_legacy` which hides those details and
takes only the necessary parameters (tx, input index, relevant
scriptPubKey, private key, sighash type [SIGHASH_ALL by default]). For
further convenience, the signature is prepended to already existing
data-pushes in scriptSig, in order to avoid rehashing the transaction
after calling the new signing function.
2023-07-03 17:33:41 +02:00
fanquake
600c595b8d Merge bitcoin/bitcoin#28021: docs: fixup honggfuzz fuzz patch
c1247c3746 docs: fixup honggfuzz patch (fanquake)

Pull request description:

  Closes #28019.

ACKs for top commit:
  brunoerg:
    ACK c1247c3746

Tree-SHA512: 3f2d146d3d0c24fd25458f6a41e2d20bf6024fc0ea9942ee6254a1b6d0e3c017c55fe79dfbf90652cad64a4d6f026f463b0011dfab065b3d9754ca7047018084
2023-07-03 11:44:55 +01:00
fanquake
c1247c3746 docs: fixup honggfuzz patch
Closes #28019.
2023-07-03 11:00:57 +01:00
Antoine Poinsot
c7db88af71 descriptor: assert we never parse a sane miniscript with no pubkey 2023-07-01 12:12:29 +02:00
Antoine Poinsot
a49402a9ec qa: make sure we don't let unspendable Miniscript descriptors be imported 2023-07-01 12:12:26 +02:00
Antoine Poinsot
639e3b6c97 descriptor: refuse to parse unspendable miniscript descriptors
It's possible for some unsatisfiable miniscripts to be considered sane.
Make sure we refuse to import those, as they would be unspendable.
2023-07-01 12:02:06 +02:00
fanquake
61d59fed74 Merge bitcoin/bitcoin#24005: test: add python implementation of Elligator swift
4f4d039a98 test: add ellswift test vectors from BIP324 (stratospher)
a31287718a test: Add ellswift unit tests (stratospher)
714fb2c02a test: Add python ellswift implementation to test framework (stratospher)

Pull request description:

  Built on top of https://github.com/bitcoin/bitcoin/pull/26222.

  This PR introduces Elligator swift encoding and decoding in the functional test framework. It's used in #24748 for writing p2p encryption tests.

ACKs for top commit:
  sipa:
    ACK 4f4d039a98
  theStack:
    ACK 4f4d039a98 🐊

Tree-SHA512: 32bc8e88f715f2cd67dc04cd38db92680872072cb3775478e2c30da89aa2da2742992779ea14da2f1faca09228942cfbd86d6957402b24bf560244b389e03540
2023-06-30 19:30:49 +01:00
Andrew Chow
6744d840df Merge bitcoin/bitcoin#27745: addrman: select addresses by network follow-up
cd8ef5b3e6 test: ensure addrman test is finite (Amiti Uttarwar)
b9f1e86f12 addrman: change asserts to Assumes (Amiti Uttarwar)
768770771f doc: update `Select` function description (Amiti Uttarwar)
2b6bd12eea refactor: de-duplicate lookups (Amiti Uttarwar)

Pull request description:

  this PR addresses outstanding review comments from #27214

ACKs for top commit:
  achow101:
    ACK cd8ef5b3e6
  mzumsande:
    Code Review ACK cd8ef5b3e6
  brunoerg:
    crACK cd8ef5b3e6

Tree-SHA512: 669f67904263e3f51c39b175eabf5fa1b1e7b6841e889656afec33d0bd93fb446de9403f0a91b186ddeaf29498c8938484a0547b1188256c4e7c90db6f30bb55
2023-06-30 13:29:04 -04:00
fanquake
a8bd0fef25 Merge bitcoin/bitcoin#27999: contrib: add macOS test for fixup_chains usage
7f96638723 contrib: add macOS fixup_chains check to security-check (fanquake)
3dca683cb7 build: support -no_fixup_chains in ld64 (fanquake)

Pull request description:

  Followup to #27676, adding the check for chained fixups.

  Somewhat annoyingly, we have to patch support for `-no_fixup_chains` into ld64. As it doesn't seem to have been added [until a later version](59a99ab603/src/ld/Options.cpp (L4172)).

  Guix Build:
  ```bash
  0e17d462808f86aa7157e27a957da88fd1adeb491ad6c01138aca93e5ad1d018  guix-build-7f96638723a0/output/arm64-apple-darwin/SHA256SUMS.part
  ceb208e6374f5d7367b73128e90ca6eaeea15d50c69e49c8cf75b47212525ad7  guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.dmg
  e31663554cfde8a37a9f3438c9c895dde94b90ff87e28f12f78be71ef6421d93  guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.tar.gz
  68a7bbc42418641eab391a85725b5c2f3c46d38a7acc07e7a8cef98909be07ec  guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin.tar.gz
  38d966ad93e7384f4f1ce16faded003a675ecce7be1987e6c4eee8e4b82c0432  guix-build-7f96638723a0/output/dist-archive/bitcoin-7f96638723a0.tar.gz
  9d314f595d897a715a321a9fba0d552220fbd4bf69aff84eb8c0001cdb48234f  guix-build-7f96638723a0/output/x86_64-apple-darwin/SHA256SUMS.part
  c218ebfd0e96348c4912e6d522492b621bb043ef45b75105ff1fde979d1004d0  guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.dmg
  1c5ff7fa82f5c76d7d8b9582ad5202f4a82a917102ecafdc3c1fb7b783f6bc3e  guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.tar.gz
  15fb01e5afcc842db6a3e793b42c70c05ce07bec79e0d2d605e241901ff9f639  guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin.tar.gz
  ```

ACKs for top commit:
  theuni:
    utACK 7f96638723.
  hebasto:
    ACK 7f96638723, I have reviewed the code and the patch, and they look OK.
  TheCharlatan:
    ACK 7f96638723

Tree-SHA512: 7f94710460f54b2afe3c9f5d57107b71436c59b799b15f78e5e3011c3c4f6b23a3acc1008eccea9c22226a200774c82900bad6c6236ab6c5c48a17dec3f2d5a2
2023-06-30 16:35:09 +01:00
fanquake
3367e1c850 Merge bitcoin/bitcoin#28009: script, test: python typing and linter updates
6c97757a48 script: appease spelling linter (Jon Atack)
1316119ce7 script: update ignored-words.txt (Jon Atack)
146c861da2 script: update linter dependencies (Jon Atack)
92408224a4 test: fix PEP484 no implicit optional argument types errors (Jon Atack)
f86a301433 script, test: add missing python type annotations (Jon Atack)

Pull request description:

  With these updates, `./test/lint/lint-python.py` and `./test/lint/lint-spelling.py` should be green again for developers using relatively recent Python dependencies, in particular mypy 0.991 (released 11/2022) and later. Please see the commit messages for details.

ACKs for top commit:
  fanquake:
    ACK 6c97757a48

Tree-SHA512: 8a46a4d36d5978affdcecf4f2ace20ca1b52d483e098304911a2169afe60ccb9b042fa90c04b762d94f3ce53d2cafe6f24476ae839867a770c7f31e7e7242d99
2023-06-30 16:20:37 +01:00
fanquake
f8a71f3fc0 Merge bitcoin/bitcoin#28014: ci: re-enable gui tests for s390x
9be4565c2d ci: re-enable gui tests for s390x (fanquake)

Pull request description:

  These work for me now. If they still don't work in other setups, maybe we can better document the issues.

  ```bash
  time FILE_ENV="./ci/test/00_setup_env_s390x.sh" ./ci/test_run_all.sh
  ...
  Running tests: coins_tests from test/coins_tests.cpp
  PASS: qt/test/test_bitcoin-qt
  Running tests: coinstatsindex_tests from test/coinstatsindex_tests.cpp
  ...
  Stop and remove CI container by ID
  + docker container kill 617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf
  617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf

  real51m37.809s
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 9be4565c2d , didn't test
  hebasto:
    ACK 9be4565c2d, tested on Ubuntu 23.04.

Tree-SHA512: 2ab708013fab7f9bf06cfc106cdffd3b7733cc7b2fe5b2122e915895735452c9b4a70ccc5a8435b28655369d54820fcf62ecb89791d5ee2e349724857e944796
2023-06-30 15:09:40 +01:00
John Moffett
5408a55fc8 Consolidate Win32-specific error formatting
GetErrorReason()'s Win32 implementation does the same thing as
Win32ErrorString(int err) from syserror.cpp, so call the latter.

Also remove now-unnecessary headers from sock.cpp and less verbose
handling of #ifdefs.
2023-06-30 09:48:21 -04:00
John Moffett
c95a4432d7 Show descriptive error messages when FileCommit fails
Only raw errno codes are logged if FileCommit fails. These are
implementation-specific, so it makes it harder to debug based on
user reports. Instead, use SysErrorString to display both the
raw int value and the descriptive message.
2023-06-30 09:48:21 -04:00
fanquake
76240ce55d Merge bitcoin/bitcoin#28011: test: Rename EncodeDecimal to serialization_fallback
fabd34873c test: Rename EncodeDecimal to serialization_fallback (MarcoFalke)

Pull request description:

  The new name better explains that the function handles fallbacks, without listing all in the function name.

ACKs for top commit:
  theStack:
    ACK fabd34873c

Tree-SHA512: a0405aab2bfb2fd10c61b51b4eb767053b25b0d914d2dac006dd3eaf360fbc6f3a444bc7b580ab8469ec492fe4358cfad5943adde4a7c8f783032ceef5cc5383
2023-06-30 12:12:40 +01:00
fanquake
ca8bfbd980 Merge bitcoin/bitcoin#28013: doc: Fix verify-binaries link in contrib README
ab8f673357 doc: Fix verify-binaries link in contrib README (TheCharlatan)

Pull request description:

ACKs for top commit:
  Zero-1729:
    crACK ab8f673357
  theStack:
    ACK ab8f673357

Tree-SHA512: 901d99b897d0b4b1af612e5650e84530252d96b5b49bc4f87c512af993abe32e6494e7248bb7ce3eb7c5ff7377a949980f143007446e3720450146367e346519
2023-06-30 11:57:08 +01:00
fanquake
9be4565c2d ci: re-enable gui tests for s390x
These work for me now. If they still don't work in other setups,
maybe we can better document the issues.

```bash
time FILE_ENV="./ci/test/00_setup_env_s390x.sh" ./ci/test_run_all.sh
...
Running tests: coins_tests from test/coins_tests.cpp
PASS: qt/test/test_bitcoin-qt
Running tests: coinstatsindex_tests from test/coinstatsindex_tests.cpp
...
Stop and remove CI container by ID
+ docker container kill 617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf
617bef8accb87530e5fbb03ff07b3b9f0aa9e3030d4da424c9612d153ab98dbf

real	51m37.809s
```
2023-06-30 11:15:57 +01:00
TheCharlatan
ab8f673357 doc: Fix verify-binaries link in contrib README 2023-06-30 12:12:24 +02:00
MarcoFalke
fac6af16f4 Allow std::byte serialization 2023-06-30 12:09:45 +02:00
MarcoFalke
fade43edc4 Allow FastRandomContext::randbytes for all byte types 2023-06-30 12:09:44 +02:00
fanquake
2cd71d3a13 Merge bitcoin/bitcoin#27937: doc: i2p documentation updates
11900e5a8a doc: simplify the router options in doc/i2p.md (Jon Atack)
b505d59326 doc: clarify when and how to launch the SAM bridge in doc/i2p.md (Jon Atack)

Pull request description:

  1. Clarify when and how to launch the SAM application bridge to address user questions and https://github.com/bitcoin/bitcoin/issues/22759#issuecomment-1599449753. The bridge is not enabled by default in the Java I2P Router, and the relevant info is somewhat difficult to find in its documentation.

  2. Remove a duplicate sentence and link (the preceding paragraph begins with the same sentence and link).

  3. Simplify the router options:

      - the Java I2P router and i2pd are the two routers have been heavily tested with Bitcoin Core and are what node operators and node software packages are using

      - [i2p-zero](https://github.com/i2p-zero/i2p-zero) hasn't been updated since July 2021 and its last release was in December 2020

      - the other routers in the wikipedia page are niche and I haven't heard anyone report using them

ACKs for top commit:
  fanquake:
    ACK 11900e5a8a

Tree-SHA512: add5f40004feddc7829d658a5de30422f144ab8fa09ff1396d5d263ee86e39803595a2c50e2aa601d0df6da5eff4358c3f7f17e039dcd4952cb306596c6c0397
2023-06-30 10:08:31 +01:00
fanquake
47ab00666e Merge bitcoin/bitcoin#27988: test: Use same timeout for all index sync
fa086248e5 test: Use same timeout for all index sync (MarcoFalke)

Pull request description:

  Seems odd to use different timeouts.

  Fix this by using the same timeout for all syncs.

  May also fix https://github.com/bitcoin/bitcoin/issues/27355 or at least make it less frequent?

ACKs for top commit:
  mzumsande:
    code review ACK fa086248e5

Tree-SHA512: a61619247c97f3a88dd19eb3f200adedd120e6da8c4e4f2cf83621545b8c289dbad77e16f13cf7973a090f7b2c3391cb0297f09b0cc95fe4f55de21ae247670f
2023-06-30 10:06:58 +01:00
fanquake
b5ebeb376d Merge bitcoin/bitcoin#28002: refactor: remove in-code warning suppression
3210f224db refactor: remove in-code warning suppression (fanquake)

Pull request description:

  Should no-longer be needed post #27872. If it is, then suppress-external-warnings should be fixed.

ACKs for top commit:
  hebasto:
    ACK 3210f224db

Tree-SHA512: 2405250b7308779d576f13ce9144944abd5b2293499a0c0fe940398dae951cb871246a55c0e644a038ee238f9510b5845c3e39f9658d9f10225a076d8122f078
2023-06-30 09:45:42 +01:00
Andrew Chow
54ba330f79 Merge bitcoin/bitcoin#27863: net: do not break when addr is not from a distinct network group
5fa4055452 net: do not `break` when `addr` is not from a distinct network group (brunoerg)

Pull request description:

  When the address is from a network group we already caught,
  do a `continue` and try to find another address until conditions
  are met or we reach the limit (`nTries`).

ACKs for top commit:
  amitiuttarwar:
    utACK 5fa4055452
  achow101:
    ACK 5fa4055452
  mzumsande:
    utACK 5fa4055452

Tree-SHA512: 225bb6df450b46960db934983c583e862d1a17bacfc46d3657a0eb25a0204e106e8cd18de36764e210e0a92489ab4b5773437e4a641c9b455bde74ff8a041787
2023-06-29 19:42:47 -04:00
Jon Atack
6c97757a48 script: appease spelling linter 2023-06-29 16:14:07 -06:00
Jon Atack
1316119ce7 script: update ignored-words.txt 2023-06-29 16:14:07 -06:00
Jon Atack
146c861da2 script: update linter dependencies 2023-06-29 16:14:07 -06:00
Jon Atack
92408224a4 test: fix PEP484 no implicit optional argument types errors
Fix warnings for these files when ./test/lint/lint-python.py is run using
mypy 0.991 (released 11/2022) and later:

$ test/lint/lint-python.py
test/functional/test_framework/coverage.py:23: error: Incompatible default for argument "coverage_logfile" (default has type "None", argument has type "str")  [assignment]
test/functional/test_framework/coverage.py:23: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
test/functional/test_framework/util.py:318: error: Incompatible default for argument "timeout" (default has type "None", argument has type "int")  [assignment]
test/functional/test_framework/util.py:318: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True
test/functional/test_framework/util.py:318: error: Incompatible default for argument "coveragedir" (default has type "None", argument has type "str")  [assignment]
test/functional/interface_rest.py:67: error: Incompatible default for argument "query_params" (default has type "None", argument has type "dict[str, Any]")  [assignment]
test/functional/interface_rest.py:67: note: PEP 484 prohibits implicit Optional. Accordingly, mypy has changed its default to no_implicit_optional=True

Verified using https://github.com/hauntsaninja/no_implicit_optional

For details, see:

https://mypy-lang.blogspot.com/2022/11/mypy-0990-released.html
2023-06-29 16:14:07 -06:00
Jon Atack
f86a301433 script, test: add missing python type annotations
Fix warnings for these files when ./test/lint/lint-python.py is run using
mypy 0.991 (released 11/2022) and later:

"By default the bodies of untyped functions are not checked, consider using
--check-untyped-defs [annotation-unchecked]"

For details, see:

https://mypy-lang.blogspot.com/2022/11/mypy-0990-released.html
2023-06-29 16:13:51 -06:00
Andrew Chow
561915f35f Merge bitcoin/bitcoin#27978: refactor: Drop unsafe AsBytePtr function
7c853619ee refactor: Drop unsafe AsBytePtr function (Ryan Ofsky)

Pull request description:

  Replace calls to `AsBytePtr` with calls to `AsBytes` or `reinterpret_cast`. `AsBytePtr` is just a wrapper around `reinterpret_cast`. It accepts any type of pointer as an argument and uses `reinterpret_cast` to cast the argument to a `std::byte` pointer.

  Despite taking any type of pointer as an argument, it is not useful to call `AsBytePtr` on most types of pointers, because byte representations of most types will be platform specific or undefined. Also, because it is named similarly to the `AsBytes` function, `AsBytePtr` looks safer than it actually is. Both `AsBytes` and `AsBytePtr` call reinterpret_cast internally and may be unsafe to use with certain types, but AsBytes at least has some type checking and can only be called on `Span` objects, while `AsBytePtr` can be called on any pointer argument.

  The change was motivated by discussion on #27973 and #27927 and is compatible with those PRs

ACKs for top commit:
  jonatack:
    re-ACK 7c853619ee
  sipa:
    utACK 7c853619ee
  achow101:
    ACK 7c853619ee

Tree-SHA512: 200d858b1d4d579f081a7f9a14d488a99713b4918b4564ac3dd5c18578d927dbd6426e62e02f49f04a3fa73ca02ff7109c495cb0b92bec43c27d9b74e2f95757
2023-06-29 17:29:40 -04:00
stratospher
4f4d039a98 test: add ellswift test vectors from BIP324
The test vector input file is taken from:
1. https://github.com/bitcoin/bips/blob/master/bip-0324/xswiftec_inv_test_vectors.csv
2. https://github.com/bitcoin/bips/blob/master/bip-0324/ellswift_decode_test_vectors.csv

Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
2023-06-29 23:32:56 +05:30
stratospher
a31287718a test: Add ellswift unit tests
remove util also since unit tests there were removed in #27538

Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
2023-06-29 23:32:56 +05:30
stratospher
714fb2c02a test: Add python ellswift implementation to test framework
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2023-06-29 23:32:54 +05:30
MarcoFalke
fabd34873c test: Rename EncodeDecimal to serialization_fallback
The new name better explains that the function handles fallbacks,
without listing all in the function name.
2023-06-29 19:51:43 +02:00
fanquake
3210f224db refactor: remove in-code warning suppression
Should no-longer be needed post #27872. If it is, then
suppress-external-warnings should be fixed.
2023-06-29 14:41:33 +01:00
fanquake
c6287faae4 Merge bitcoin/bitcoin#27996: ci: filter all subtrees from tidy output
62633b5046 ci: filter all subtrees from tidy output (fanquake)

Pull request description:

  We are currently dumping output for some. i.e:
  ```bash
  diff --git a/src/minisketch/src/fields/clmul_1byte.cpp b/src/minisketch/src/fields/clmul_1byte.cpp
  index 8826af9..7fd6f2a 100644
  --- a/src/minisketch/src/fields/clmul_1byte.cpp
  +++ b/src/minisketch/src/fields/clmul_1byte.cpp
  @@ -4,21 +4,16 @@
    * file LICENSE or http://www.opensource.org/licenses/mit-license.php.*
    **********************************************************************/

  -/* This file was substantially auto-generated by doc/gen_params.sage. */
  -#include "../fielddefines.h"
  -
  +class Sketch;
   #if defined(ENABLE_FIELD_BYTES_INT_1)
  ```

ACKs for top commit:
  hebasto:
    re-ACK 62633b5046

Tree-SHA512: fd0a17af6b37fc7641547dab329c2d14ec784941c4d100db1e80d232aff39e45ad9c588982810a2cfc54b4fe820bfe0d50638b53209fec6774fd556b9b0ae180
2023-06-29 13:35:47 +01:00
fanquake
3d51f7c9a8 Merge bitcoin/bitcoin#27932: test: Fuzz on macOS
fae7c50d20 test: Run fuzz tests on macOS (MarcoFalke)

Pull request description:

  Any reason not to?

ACKs for top commit:
  jamesob:
    Github ACK fae7c50d20
  dergoegge:
    utACK fae7c50d20

Tree-SHA512: e45122d73fafb17cea312258314b826cb0745e08daadd28465f687ec02d4c127d2f8cbe20179a9fff5712038850c02c968abb4838fa088b7555e28709317d3a3
2023-06-29 13:08:58 +01:00
fanquake
7f96638723 contrib: add macOS fixup_chains check to security-check
Followup to #27676.
2023-06-29 12:08:08 +01:00
fanquake
62633b5046 ci: filter all subtrees from tidy output
We are currently dumping output for some. i.e:
```bash
diff --git a/src/minisketch/src/fields/clmul_1byte.cpp b/src/minisketch/src/fields/clmul_1byte.cpp
index 8826af9..7fd6f2a 100644
--- a/src/minisketch/src/fields/clmul_1byte.cpp
+++ b/src/minisketch/src/fields/clmul_1byte.cpp
@@ -4,21 +4,16 @@
  * file LICENSE or http://www.opensource.org/licenses/mit-license.php.*
  **********************************************************************/

-/* This file was substantially auto-generated by doc/gen_params.sage. */
-#include "../fielddefines.h"
-
+class Sketch;
 #if defined(ENABLE_FIELD_BYTES_INT_1)
```
2023-06-29 11:59:31 +01:00
fanquake
3dca683cb7 build: support -no_fixup_chains in ld64
Patch in suport for using -no_fixup_chains, with ld64. This option just
seems to be missing from our version, as it exists in later releases.

This is needed so we can disable fixup_chains in our security checks.
2023-06-29 11:55:40 +01:00
fanquake
d4018f0b6c Merge bitcoin/bitcoin#27811: guix: Clean up manifest
a51d7abf1e guix: Specify symbols in modules explicitly (Hennadii Stepanov)
47d51fb048 guix: Drop unneeded modules (Hennadii Stepanov)
57fdedd0e9 guix: Unify fetch methods (Hennadii Stepanov)

Pull request description:

  This PR cleans up the `contrib/guix/manifest.scm` in the following way:
  - Unneeded for a successful build modules have be dropped.
  - Some modules have been enhanced with `#:select` clauses, which improves maintainability (see the commit message for details).

ACKs for top commit:
  TheCharlatan:
    ACK a51d7abf1e

Tree-SHA512: 380a36d03ec303ff8700893cfaad75ca09d84a77fd08d6c6a1679ac96409014b36f0698eb071e09af25ad36f1bc62aec0eec1092146d879251c6a8cce586169b
2023-06-29 10:43:33 +01:00
Antoine Poinsot
e3280eae1b miniscript: make GetStackSize() and GetOps() return optionals
The value is only set for satisfiable nodes, so it was undefined for
non-satisfiable nodes. Make it clear in the interface by returning
std::nullopt if the node isn't satisfiable instead of an undefined
value.
2023-06-29 11:35:42 +02:00
fanquake
e0cd7458e6 Merge bitcoin/bitcoin#27530: Remove now-unnecessary poll, fcntl includes from net(base).cpp
8d9b90a61e Remove now-unnecessary poll, fcntl includes from net(base).cpp (Ben Woosley)

Pull request description:

  As far as I can tell, the code calling for these includes was removed in:
  6e68ccbefe #24356
  82d360b5a8 #21387

ACKs for top commit:
  fanquake:
    ACK 8d9b90a61e

Tree-SHA512: e536d60f4cf204a10a5b461eca20c8329aa6b0fd3b27651ac9490ed42d3e22e31d652cd991ddc84c96e4758df15aefa7e7f84c710f2feb6d2e0fcfbda9ad4356
2023-06-29 10:32:48 +01:00
fanquake
e8543629ae Merge bitcoin/bitcoin#27884: test: Use TestNode datadir_path or chain_path where possible
aaaa3aefbd test: Use TestNode *_path properties where possible (MarcoFalke)
dddd89962b test: Allow pathlib.Path as RPC argument via authproxy (MarcoFalke)
fa41614a0a scripted-diff: Use wallets_path and chain_path where possible (MarcoFalke)
fa493fadfb test: Use wallet_dir lambda in wallet_multiwallet test where possible (MarcoFalke)

Pull request description:

  It seems inconsistent, fragile and verbose to:

  * Call `get_datadir_path` to recreate the path that already exists as field in TestNode
  * Call `os.path.join` with the hardcoded chain name or `self.chain` to recreate the TestNode `chain_path` property
  * Sometimes even use the hardcoded node dir name (`"node0"`)

  Fix all issues by using the TestNode properties.

ACKs for top commit:
  willcl-ark:
    re-ACK aaaa3aefbd
  theStack:
    Code-review ACK aaaa3aefbd 🌊

Tree-SHA512: e4720278085beb8164e1fe6c1aa18f601558a9263494ce69a83764c1487007de63ebb51d1b1151862dc4d5b49ded6162a5c1553cd30ea1c28627d447db4d8e72
2023-06-29 09:51:53 +01:00
Ben Woosley
8d9b90a61e Remove now-unnecessary poll, fcntl includes from net(base).cpp
As far as I can tell, the code calling for these includes was removed in:
6e68ccbefe #24356
82d360b5a8 #21387
2023-06-28 16:35:45 -05:00
Andrew Chow
626d346469 Merge bitcoin/bitcoin#26222: Introduce secp256k1 module with field and group classes to test framework
d4fb58ae8a test: EC: optimize scalar multiplication of G by using lookup table (Sebastian Falbesoner)
1830dd8820 test: add secp256k1 module with FE (field element) and GE (group element) classes (Pieter Wuille)

Pull request description:

  This PR rewrites a portion of `test_framework/key.py`, in a compatible way, by introducing classes that encapsulate field element and group element logic, in an attempt to be more readable and reusable.

  To maximize readability, the group element logic does not use Jacobian coordinates. Instead, group elements just store (affine) X and Y coordinates directly. To compensate for the performance loss this causes, field elements are represented as fractions. This undoes most, but not all, of the performance loss, and there is a few % slowdown (as measured in `feature_taproot.py`, which heavily uses this).

  The upside is that the implementation for group laws (point doubling, addition, subtraction, ...) is very close to the mathematical description of elliptic curves, and this extends to potential future extensions (e.g. ElligatorSwift as needed by #27479).

ACKs for top commit:
  achow101:
    ACK d4fb58ae8a
  theStack:
    re-ACK d4fb58ae8a
  stratospher:
    tested ACK d4fb58a. really liked how this PR makes the secp256k1 code in the tests more intuitive and easier to follow!

Tree-SHA512: 9e0d65d7de0d4fb35ad19a1c19da7f41e5e1db33631df898c6d18ea227258a8ba80c893dab862b0fa9b0fb2efd0406ad4a72229ee26d7d8d733dee1d56947f18
2023-06-28 16:27:55 -04:00
Ryan Ofsky
7c853619ee refactor: Drop unsafe AsBytePtr function
Replace calls to AsBytePtr with direct calls to AsBytes or reinterpret_cast.
AsBytePtr is just a wrapper around reinterpret_cast. It accepts any type of
pointer as an argument and uses reinterpret_cast to cast the argument to a
std::byte pointer.

Despite taking any type of pointer as an argument, it is not useful to call
AsBytePtr on most types of pointers, because byte representations of most types
will be implmentation-specific. Also, because it is named similarly to the
AsBytes function, AsBytePtr looks safer than it actually is. Both AsBytes and
AsBytePtr call reinterpret_cast internally and may be unsafe to use with
certain types, but AsBytes at least has some type checking and can only be
called on Span objects, while AsBytePtr can be called on any pointer argument.

Co-authored-by: Pieter Wuille <pieter@wuille.net>
2023-06-28 15:14:45 -04:00
Andrew Chow
7952a5934a Merge bitcoin/bitcoin#27927: util: Allow std::byte and char Span serialization
fa38d86235 Use only Span{} constructor for byte-like types where possible (MarcoFalke)
fa257bc831 util: Allow std::byte and char Span serialization (MarcoFalke)

Pull request description:

  Seems odd to require developers to cast all byte-like spans passed to serialization to `unsigned char`-spans. Fix that by passing and accepting byte-like spans as-is. Finally, add tests and update the code to use just `Span` where possible.

ACKs for top commit:
  sipa:
    utACK fa38d86235
  achow101:
    ACK fa38d86235
  ryanofsky:
    Code review ACK fa38d86235. This looks great. The second commit really removes a lot of boilerplate and shows why the first commit is useful.

Tree-SHA512: 788592d9ff515c3ebe73d48f9ecbb8d239f5b985af86f09974e508cafb0ca6d73a959350295246b4dfb496149bc56330a0b5d659fc434ba6723dbaba0b7a49e5
2023-06-28 15:12:12 -04:00
MarcoFalke
fa1e27fe8e fuzz: Generate rpc fuzz targets individually 2023-06-28 17:50:45 +02:00
furszy
5df988b534 test: add coverage for descriptor ID
Tests vectors were calculated by running the same tests on
v25. Which was the last release prior to introducing the
diff in the descriptor's string representation ('h' format).

Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
2023-06-28 09:37:16 -03:00
furszy
6a9510d2da wallet: bugfix, always use apostrophe for spkm descriptor ID
As we update the descriptor's db record every time that
the wallet is loaded (at `TopUp` time), if the spkm ID differs
from the one in db, the wallet will enter in an unrecoverable
corruption state, and no soft version will be able to open
it anymore.

Because we cannot change the past, to stay compatible between
releases, we need to always use the apostrophe version for the
spkm IDs.
2023-06-28 09:37:16 -03:00
furszy
97a965d98f refactor: extract descriptor ID calculation from spkm GetID()
This allows us to verify the descriptor ID on the descriptors
unit tests in different software versions without requiring to
use the entire DescriptorScriptPubKeyMan machinery.

Note:
The unit test changes are introduced after the bugfix commit
but this commit + the unit test commit can be cherry-picked
on top of the v25 branch to verify IDs correctness. IDs must
be the same for v25 and after the bugfix commit.
2023-06-28 09:37:15 -03:00
furszy
1d207e3931 wallet: do not allow loading descriptor with an invalid ID
If the computed descriptor's ID doesn't match the wallet's
DB spkm ID, return early from the loading process to prevent
DB data from being modified in any post-loading procedure
(e.g 'TopUp' updates the descriptor's data).
2023-06-28 09:37:15 -03:00
fanquake
d6ee03507f Merge bitcoin/bitcoin#27987: ci: remove duplicate bsdmainutils from CI configs
248a17addf ci: remove duplicate python3 from CI configs (fanquake)
b50767fdde ci: remove duplicate bsdmainutils from CI configs (fanquake)

Pull request description:

  `bsdmainutils` and `python3` are included in `CI_BASE_PACKAGES`.

ACKs for top commit:
  hebasto:
    ACK 248a17addf

Tree-SHA512: 1e5cddd8a37128690ef3110891549cb9a4c69c6bca558137c97031bc0e494e1053063923d3ccee8b1d9f05d3432765ee10f9ce872e88959b802ba64b6e2d300c
2023-06-28 13:25:14 +01:00
Hennadii Stepanov
a51d7abf1e guix: Specify symbols in modules explicitly
This change improves the maintainability of the manifest:
(1) It allows to remove the module when the specified symbols are no
    longer used.
(2) It prevents accidental use of other symbols, such as `bash`
    instead of `bash-minimal`.
2023-06-28 12:36:26 +01:00
Hennadii Stepanov
47d51fb048 guix: Drop unneeded modules 2023-06-28 12:35:20 +01:00
Hennadii Stepanov
57fdedd0e9 guix: Unify fetch methods 2023-06-28 12:35:14 +01:00
fanquake
a15388c606 Merge bitcoin/bitcoin#27949: http: update libevent workaround to correct version
79d343a642 http: update libevent workaround to correct version (stickies-v)

Pull request description:

  The libevent bug described in 5ff8eb2637 was already patched in [release-2.1.9-beta](https://github.com/libevent/libevent/releases/tag/release-2.1.9-beta), with cherry-picked commits [5b40744d1581447f5b4496ee8d4807383e468e7a](5b40744d15) and [b25813800f97179b2355a7b4b3557e6a7f568df2](b25813800f).

  There should be no side-effects by re-applying the workaround on an already patched version of libevent (as is currently done in master for people running libevent between 2.1.9 and 2.1.12), but it is best to just set the correct version number to avoid confusion.

  This will prevent situations like e.g. in https://github.com/bitcoin/bitcoin/pull/27909#discussion_r1238858604, where a reverse workaround was incorrectly applied to the wrong version range.

ACKs for top commit:
  fanquake:
    ACK 79d343a642

Tree-SHA512: 56d2576411cf38e56d0976523fec951e032a48e35af293ed1ef3af820af940b26f779b9197baaed6d8b79bd1c7f7334646b9d73f80610d63cffbc955958ca8a0
2023-06-28 12:20:25 +01:00
MarcoFalke
fa086248e5 test: Use same timeout for all index sync 2023-06-28 12:45:36 +02:00
fanquake
ec822e88dd Merge bitcoin/bitcoin#27813: guix: Update python-lief package to 0.13.2
529c92e837 guix: Update `python-lief` package to 0.13.2 (Hennadii Stepanov)

Pull request description:

  The Guix's `python-lief` package is going to move to using external deps, rather than the bundled ones (https://lists.gnu.org/archive/html/guix-patches/2023-05/msg01302.html). We want to continue using our own package indefinitely, to keep the build simpler, and allow for easier updating.

  Changes in `contrib/devtools/security-check.py` are caused by 6357c6370b.

  Also see: https://github.com/bitcoin/bitcoin/pull/27507.

ACKs for top commit:
  fanquake:
    ACK 529c92e837

Tree-SHA512: ad81111b090a39b380fe25bb27b54a339e78a158f462c7adda25d5ee55f0d654107b1486b29b9687ad0808e27b01e04f53a0e8ffc6600b79103d6bd0dfec64ef
2023-06-28 11:41:35 +01:00
fanquake
248a17addf ci: remove duplicate python3 from CI configs 2023-06-28 11:10:51 +01:00
fanquake
b50767fdde ci: remove duplicate bsdmainutils from CI configs 2023-06-28 10:07:51 +01:00
TheCharlatan
6eb33bd0c2 kernel: Add fatalError method to notifications
FatalError replaces what previously was the AbortNode function in
shutdown.cpp.

This commit is part of the libbitcoinkernel project and further removes
the shutdown's and, more generally, the kernel library's dependency on
interface_ui with a kernel notification method. By removing interface_ui
from the kernel library, its dependency on boost is reduced to just
boost::multi_index. At the same time it also takes a step towards
de-globalising the interrupt infrastructure.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-06-28 09:52:33 +02:00
TheCharlatan
7320db96f8 kernel: Add flushError method to notifications
This is done in addition with the following commit. Both have the goal
of getting rid of direct calls to AbortNode from kernel code. This extra
flushError method is added to notify specifically about errors that
arrise when flushing (syncing) block data to disk. Unlike other
instances, the current calls to AbortNode in the blockstorage flush
functions do not report an error to their callers.

This commit is part of the libbitcoinkernel project and further removes
the shutdown's and, more generally, the kernel library's dependency on
interface_ui with a kernel notification method. By removing interface_ui
from the kernel library, its dependency on boost is reduced to just
boost::multi_index. At the same time it also takes a step towards
de-globalising the interrupt infrastructure.
2023-06-28 09:52:32 +02:00
TheCharlatan
3fa9094b92 scripted-diff: Rename FatalError to FatalErrorf
This is done in preparation for the next commit where a new FatalError
function is introduced. FatalErrorf follows common convention to append
'f' for functions accepting format arguments.

-BEGIN VERIFY SCRIPT-
sed -i 's/FatalError/FatalErrorf/g' $( git grep -l 'FatalError')
-END VERIFY SCRIPT-
2023-06-28 09:52:30 +02:00
TheCharlatan
edb55e2777 kernel: Pass interrupt reference to chainman
This and the following commit seek to decouple the libbitcoinkernel
library from the shutdown code. As a library, it should it should have
its own flexible interrupt infrastructure without relying on node-wide
globals.

The commit takes the first step towards this goal by de-globalising
`ShutdownRequested` calls in kernel code.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-06-28 09:52:27 +02:00
TheCharlatan
e2d680a32d util: Add SignalInterrupt class and use in shutdown.cpp
This change helps generalize shutdown code so an interrupt can be
provided to libbitcoinkernel callers. This may also be useful to
eventually de-globalize all of the shutdown code.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-06-28 09:49:28 +02:00
Ryan Ofsky
d9c7c2fd3e Merge bitcoin/bitcoin#24914: wallet: Load database records in a particular order
3c83b1d884 doc: Add release note for wallet loading changes (Andrew Chow)
2636844f53 walletdb: Remove loading code where the database is iterated (Andrew Chow)
cd211b3b99 walletdb: refactor decryption key loading (Andrew Chow)
31c033e5ca walletdb: refactor defaultkey and wkey loading (Andrew Chow)
c978c6d39c walletdb: refactor active spkm loading (Andrew Chow)
6fabb7fc99 walletdb: refactor tx loading (Andrew Chow)
abcc13dd24 walletdb: refactor address book loading (Andrew Chow)
405b4d9147 walletdb: Refactor descriptor wallet records loading (Andrew Chow)
30ab11c497 walletdb: Refactor legacy wallet record loading into its own function (Andrew Chow)
9e077d9b42 salvage: Remove use of ReadKeyValue in salvage (Andrew Chow)
ad779e9ece walletdb: Refactor hd chain loading to its own function (Andrew Chow)
72c2a54ebb walletdb: Refactor encryption key loading to its own function (Andrew Chow)
3ccde4599b walletdb: Refactor crypted key loading to its own function (Andrew Chow)
7be10adff3 walletdb: Refactor key reading and loading to its own function (Andrew Chow)
52932c5adb walletdb: Refactor wallet flags loading (Andrew Chow)
01b35b55a1 walletdb: Refactor minversion loading (Andrew Chow)

Pull request description:

  Currently when we load a wallet, we just iterate through all of the records in the database and add them completely statelessly. However we have some records which do rely on other records being loaded before they are. To deal with this, we use `CWalletScanState` to hold things temporarily until all of the records have been read and then we load the stateful things.

  However this can be slow, and with some future improvements, can cause some pretty drastic slowdowns to retain this pattern. So this PR changes the way we load records by choosing to load the records in a particular order. This lets us do things such as loading a descriptor record, then finding and loading that descriptor's cache and key records. In the future, this will also let us use `IsMine` when loading transactions as then `IsMine` will actually be working as we now always load keys and descriptors before transactions.

  In order to get records of a specific type, this PR includes some refactors to how we do database cursors. Functionality is also added to retrieve a cursor that will give us records beginning with a specified prefix.

  Lastly, one thing that iterating the entire database let us do was to find unknown records. However even if unknown records were found, we would not do anything with this information except output a number in a log line. With this PR, we would no longer be aware of any unknown records. This does not change functionality as we don't do anything with unknown records, and having unknown records is not an error. Now we would just not be aware that unknown records even exist.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3c83b1d884 🍤
  furszy:
    reACK 3c83b1d8
  ryanofsky:
    Code review ACK 3c83b1d884. Just Marco's suggested error handling fixes since last review

Tree-SHA512: 15fa56332fb2ce4371db468a0c674ee7a3a8889c8cee9f428d06a7d1385d17a9bf54bcb0ba885c87736841fe6a5c934594bcf4476a473616510ee47862ef30b4
2023-06-27 19:03:15 -04:00
Andrew Chow
caff95a023 Merge bitcoin/bitcoin#27896: Remove the syscall sandbox
32e2ffc393 Remove the syscall sandbox (fanquake)

Pull request description:

  After initially being merged in #20487, it's no-longer clear that an internal syscall sandboxing mechanism is something that Bitcoin Core should have/maintain, especially when compared to better maintained/supported alterantives, i.e [firejail](https://github.com/netblue30/firejail).

  There is more related discussion in #24771.

  Note that given where it's used, the sandbox also gets dragged into the kernel.

  If it's removed, this should not require any sort of deprecation, as this was only ever an opt-in, experimental feature.

  Closes #24771.

ACKs for top commit:
  davidgumberg:
     crACK 32e2ffc393
  achow101:
    ACK 32e2ffc393
  dergoegge:
    ACK 32e2ffc393

Tree-SHA512: 8cf71c5623bb642cb515531d4a2545d806e503b9d57bfc15a996597632b06103d60d985fd7f843a3c1da6528bc38d0298d6b8bcf0be6f851795a8040d71faf16
2023-06-27 18:19:21 -04:00
Andrew Chow
5cce4d293e Merge bitcoin/bitcoin#27334: util: implement noexcept move assignment & move ctor for prevector
bfb9291a86 util: implement prevector's move ctor & move assignment (Martin Leitner-Ankerl)
fffc86f49f test: CScriptCheck is used a lot in std::vector, make sure that's efficient (Martin Leitner-Ankerl)
81f67977f5 util: prevector's move ctor and move assignment is `noexcept` (Martin Leitner-Ankerl)
d380d2877e bench: Add benchmark for prevector usage in std::vector (Martin Leitner-Ankerl)

Pull request description:

  `prevector`'s move assignment and move constructor were not `noexcept`, which makes it inefficient to use inside STL containers like `std::vector`. That's the case e.g. for `CScriptCheck`. This PR adds `noexcept`, and also implements the move assignment & ctor, which makes it quite a bit more efficient to use prevector in an std::vector.

  The PR also adds a benchmark which grows an `std::vector` by adding `prevector` objects to it.

  merge-base:
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |            6,440.29 |          155,272.42 |    0.2% |       40,713.01 |       20,473.84 |  1.989 |       7,132.01 |    0.2% |      0.44 | `PrevectorFillVectorDirectNontrivial`
  |            3,213.19 |          311,217.35 |    0.7% |       35,373.01 |       10,214.07 |  3.463 |       6,945.00 |    0.2% |      0.43 | `PrevectorFillVectorDirectTrivial`
  |           34,749.70 |           28,777.23 |    0.1% |      364,396.05 |      110,521.94 |  3.297 |      78,568.37 |    0.1% |      0.43 | `PrevectorFillVectorIndirectNontrivial`
  |           32,535.05 |           30,736.09 |    0.4% |      353,823.31 |      103,464.53 |  3.420 |      79,871.80 |    0.2% |      0.40 | `PrevectorFillVectorIndirectTrivial`

  util: prevector's move ctor and move assignment is `noexcept`:
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |            6,603.87 |          151,426.40 |    0.2% |       23,734.01 |       21,009.63 |  1.130 |       2,445.01 |    0.3% |      0.44 | `PrevectorFillVectorDirectNontrivial`
  |            1,980.93 |          504,813.15 |    0.1% |       13,784.00 |        6,304.32 |  2.186 |       2,258.00 |    0.3% |      0.44 | `PrevectorFillVectorDirectTrivial`
  |           19,110.54 |           52,327.15 |    0.1% |      139,816.41 |       51,987.72 |  2.689 |      28,512.18 |    0.1% |      0.43 | `PrevectorFillVectorIndirectNontrivial`
  |           12,334.37 |           81,074.27 |    0.7% |      125,655.12 |       39,253.58 |  3.201 |      27,854.46 |    0.2% |      0.44 | `PrevectorFillVectorIndirectTrivial`

  util: implement prevector's move ctor & move assignment
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |            5,262.66 |          190,018.01 |    0.2% |       20,157.01 |       16,745.26 |  1.204 |       2,445.01 |    0.3% |      0.44 | `PrevectorFillVectorDirectNontrivial`
  |            1,687.07 |          592,744.35 |    0.2% |       12,742.00 |        5,368.02 |  2.374 |       2,258.00 |    0.3% |      0.44 | `PrevectorFillVectorDirectTrivial`
  |           17,930.80 |           55,769.95 |    0.1% |      136,237.69 |       47,903.31 |  2.844 |      28,512.02 |    0.2% |      0.42 | `PrevectorFillVectorIndirectNontrivial`
  |           11,893.75 |           84,077.78 |    0.2% |      126,182.02 |       37,852.91 |  3.333 |      28,152.01 |    0.1% |      0.44 | `PrevectorFillVectorIndirectTrivial`

  As can be seen, mostly thanks to just `noexcept` the benchmark becomes about 2 times faster because `std::vector` can now use move operations instead of having to fall back to copy everything

  I had a look at how this change affects the other benchmarks, and they are all pretty much the same, the only noticable difference is `CCheckQueueSpeedPrevectorJob` goes from 364.56ns down to 346.21ns.

ACKs for top commit:
  achow101:
    ACK bfb9291a86
  jonatack:
    > Tested Approach ACK [bfb9291](bfb9291a86), ~ACK modulo re-verifying the implementation of the last commit.
  john-moffett:
    Approach ACK bfb9291a86
  theStack:
    Tested and light code-review ACK bfb9291a86

Tree-SHA512: 242995d7cb2f8ebfa73177aa690a505f189d91edeb8cea3e34a41ca507c8cb17c65fe2a4e196fdafc5c6e89b2b2222627bfc9f5c316517de0857b7b5e9c58225
2023-06-27 15:42:51 -04:00
Jon Atack
11900e5a8a doc: simplify the router options in doc/i2p.md
- the Java I2P router and i2pd are the two routers have been heavily tested
  with Bitcoin Core and are what people and node software packages use

- i2p-zero (https://github.com/i2p-zero/i2p-zero) hasn't been updated since
  July 2021 and its last release was in December 2020

- the other routers in the wikipedia page are niche
2023-06-27 10:26:57 -06:00
Jon Atack
b505d59326 doc: clarify when and how to launch the SAM bridge in doc/i2p.md
The SAM application bridge is not enabled by default in the Java I2P Router,
and the relevant info is somewhat difficult to find in its documentation.

Also, remove a duplicate sentence; the preceding paragraph begins with the same.
2023-06-27 10:26:06 -06:00
Andrew Chow
3c83b1d884 doc: Add release note for wallet loading changes
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
2023-06-27 11:08:06 -04:00
Andrew Chow
2636844f53 walletdb: Remove loading code where the database is iterated
Instead of iterating the database to load the wallet, we now load
particular kinds of records in an order that we want them to be loaded.
So it is no longer necessary to iterate the entire database to load the
wallet.
2023-06-27 11:08:05 -04:00
Andrew Chow
cd211b3b99 walletdb: refactor decryption key loading
Instead of loading decryption keys as we iterate the database, load them
explicitly.
2023-06-27 11:08:00 -04:00
Andrew Chow
31c033e5ca walletdb: refactor defaultkey and wkey loading
Instead of dealing with these records when iterating the entire
database, find and handle them explicitly.

Loading of OLD_KEY records is bumped up to a LOAD_FAIL error as we will
not be able to use these types of keys which can lead to users missing
funds.
2023-06-27 11:07:51 -04:00
Andrew Chow
c978c6d39c walletdb: refactor active spkm loading
Instead of loading active spkm records as we come across them when
iterating the database, load them explicitly.

Due to exception handling changes, deserialization errors are now
treated as critical.
2023-06-27 11:07:46 -04:00
Andrew Chow
6fabb7fc99 walletdb: refactor tx loading
Instead of loading tx records as we come across them when iterating the
database, load them explicitly.
2023-06-27 11:07:38 -04:00
Andrew Chow
abcc13dd24 walletdb: refactor address book loading
Instead of loading address book records as we come across them when
iterating the database, load them explicitly

Due to exception handling changes, deserialization errors are now
treated as critical.

The error message for noncritical errors has also been updated to
reflect that there's more data that could be missing than just address
book entries and tx data.
2023-06-27 11:04:18 -04:00
Andrew Chow
405b4d9147 walletdb: Refactor descriptor wallet records loading
Instead of loading descriptor wallet records as we come across them when
iterating the database, loading them explicitly.

Exception handling for these records changes to a per-record type basis,
rather than globally. This results in some records now failing with a
critical error rather than a non-critical one.
2023-06-27 11:04:18 -04:00
Andrew Chow
30ab11c497 walletdb: Refactor legacy wallet record loading into its own function
Instead of loading legacy wallet records as we come across them when
iterating the database, load them explicitly.

Exception handling for these records changes to a per-record type basis,
rather than globally. This results in some records now failing with a
critical error rather than a non-critical one.
2023-06-27 11:00:47 -04:00
fanquake
7ee41217b3 Merge bitcoin/bitcoin#27940: test: Add implicit-signed-integer-truncation:*/include/c++/ suppression
fae55f989e test: Add implicit-signed-integer-truncation:*/include/c++/ suppression (MarcoFalke)

Pull request description:

  Needed for aarch64. Steps to test on aarch64:

  ```
  lscpu | grep Arch
  FILE_ENV="./ci/test/00_setup_env_native_fuzz.sh" ./ci/test_run_all.sh
  ```

ACKs for top commit:
  fanquake:
    ACK fae55f989e  - reproduced the failure:

Tree-SHA512: b5058873118d285cc5d678a572cf4b890f8d68a24e1ac0987490f1b4123469a2b4456b08474f372e6aa49bb0d69e16f2c8277208b1cde3222a317f000beb5056
2023-06-27 15:52:33 +01:00
Sebastian Falbesoner
d4fb58ae8a test: EC: optimize scalar multiplication of G by using lookup table
On my machine, this speeds up the functional test feature_taproot.py by
a factor of >1.66x (runtime decrease from 1m16.587s to 45.334s).

Co-authored-by: Pieter Wuille <pieter@wuille.net>
2023-06-27 09:34:52 -04:00
Pieter Wuille
1830dd8820 test: add secp256k1 module with FE (field element) and GE (group element) classes
These are primarily designed for ease of understanding, not performance.
2023-06-27 09:34:48 -04:00
fanquake
b741a62a2f Merge bitcoin/bitcoin#27929: Added static_assert to check that base_blob is using whole bytes.
5fc4939e17 Added static_assert to check that base_blob is using whole bytes. (Brotcrunsher)

Pull request description:

  Prior to this commit it was possible to create base_blobs with any arbitrary amount of bits, like base_blob<9>. One could assume that this would be a valid way to create a bit field that guarantees to have at least 9 bits. However, in such a case, base_blob would not behave as expected because the WIDTH is rounded down to the closest whole byte (simple integer division by 8). This commit makes sure that this oddity is detected and blocked by the compiler.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 5fc4939e17
  theStack:
    ACK 5fc4939e17
  stickies-v:
    ACK 5fc4939e17

Tree-SHA512: 6a06760f09d4a9e6f0b9338d4dddd4091f2ac59a843a443d9302959936d72c55f7cccd55a51ec3a5a799921f68be1b87968ef3c9c11d3389cbd369b5045bb50a
2023-06-27 12:54:20 +01:00
fanquake
35b01e8504 Merge bitcoin/bitcoin#27676: macOS: Bump minimum required runtime version and prepare for building with upstream LLVM
3df6070466 contrib: remove macOS lazy_bind check (fanquake)
9bc357e205 build: explicitly opt-in to new fixup_chains functionality for darwin (Cory Fields)
fb61bc0c02 depends: Bump MacOS minimum runtime requirement to 11.0 (Cory Fields)
c2cd47280c depends: bump darwin clang to 11.1 (Cory Fields)

Pull request description:

  This (I believe) resolves the last of the blockers for [switching us away from cctools and instead using out-of-the-box llvm and lld](https://github.com/bitcoin/bitcoin/pull/21778) for building Darwin binaries.

  For now, we continue building with a pre-packaged llvm and cctools, but after this PR the clang+lld combo should just work for anyone trying it. Additionally after this PR, the new runtime `fixup_chains` behavior will be in-use, as ld64 uses it as well.

  The commits may seem unrelated, so in detail:

  lld (llvm's linker) has been a work-in-progress for Darwin for years. Recently though, it has gained nearly all of the features we require. The last missing feature from ld64, `-Wl,-bind_at_load`, is not implemented in lld; as far as I can tell [lazy loading has conceptually been replaced by fixup chains](https://www.emergetools.com/blog/posts/iOS15LaunchTime).

  So that means we don't need ld64's `bind_at_load` as long as lld can handle `-Wl,-fixup_chains` (which it can). I've added it to our configure as a linker option mostly so that we can see it in the logs; it's default-on as long as the minimum version is >11.0.

  About that: the runtime functionality required for `-Wl,-fixup_chains` [requires macOS >=11.0](https://github.com/llvm/llvm-project/blob/release/16.x/lld/MachO/Driver.cpp#L1021). Hence the commit that bumps the minimum version. Our current min runtime of `10.15` has been unsupported since September 2022, so I don't expect this bump to be controversial.

  Lastly, with the minimum runtime version bumped to 11.0, our current version of pre-compiled clang we use for macOS is too old to understand `-mmacosx-version-min=11.0` because it expects `=10.x`. So I've made the smallest possible bump (from 10.0.1 to 11.1.0) to a version that understands. This bump is arbitrary and unfortunate, but likely to be short-lived as we may end up replacing it with llvm+lld for v26 anyway. I've held off on bumping the SDK as I think that makes sense to do as part of the lld switch instead.

ACKs for top commit:
  hebasto:
    ACK 3df6070466
  gruve-p:
    ACK 3df6070466
  fanquake:
    ACK 3df6070466
  TheCharlatan:
    ACK 3df6070466

Tree-SHA512: 0200ec4a3b88df33877ae82c15b5c04d745852550923f491a354b391cac65f88e4df116a40055c23a8cbcfcdfb9a376c6ae8fdd0e898e7b966bc213dcb5857cf
2023-06-27 09:57:21 +01:00
MarcoFalke
fa38d86235 Use only Span{} constructor for byte-like types where possible
This removes bloat that is not needed.
2023-06-27 10:13:37 +02:00
MarcoFalke
fa257bc831 util: Allow std::byte and char Span serialization 2023-06-27 10:13:29 +02:00
Andrew Chow
e8c31f135c tests: Test for bumping single output transaction 2023-06-26 17:56:50 -04:00
Andrew Chow
4f4d4407e3 test: Test bumpfee reduce_output 2023-06-26 17:56:50 -04:00
Andrew Chow
7d83502d3d bumpfee: Allow original change position to be specified
If the user used a custom change address, it may not be detected as a
change output, resulting in an additional change output being added to
the bumped transaction. We can avoid this issue by allowing the user to
specify the position of the change output.
2023-06-26 17:49:09 -04:00
Andrew Chow
679f825ba3 Merge bitcoin/bitcoin#27479: BIP324: ElligatorSwift integrations
3168b08043 Bench test for EllSwift ECDH (Pieter Wuille)
42d759f239 Bench tests for CKey->EllSwift (dhruv)
2e5a8a437c Fuzz test for Ellswift ECDH (dhruv)
c3ac9f5cf4 Fuzz test for CKey->EllSwift->CPubKey creation/decoding (dhruv)
aae432a764 Unit test for ellswift creation/decoding roundtrip (dhruv)
eff72a0dff Add ElligatorSwift key creation and ECDH logic (Pieter Wuille)
42239f8390 Enable ellswift module in libsecp256k1 (dhruv)
901336eee7 Squashed 'src/secp256k1/' changes from 4258c54f4e..705ce7ed8c (Pieter Wuille)

Pull request description:

  This replaces #23432 and part of #23561.

  This PR introduces all of the ElligatorSwift-related changes (libsecp256k1 updates, generation, decoding, ECDH, tests, fuzzing, benchmarks) needed for BIP324.

  ElligatorSwift is a special 64-byte encoding format for public keys introduced in libsecp256k1 in https://github.com/bitcoin-core/secp256k1/pull/1129. It has the property that *every* 64-byte array is a valid encoding for some public key, and every key has approximately $2^{256}$ encodings. Furthermore, it is possible to efficiently generate a uniformly random encoding for a given public key or private key. This is used for the key exchange phase in BIP324, to achieve a byte stream that is entirely pseudorandom, even before the shared encryption key is established.

ACKs for top commit:
  instagibbs:
    reACK 3168b08043
  achow101:
    ACK 3168b08043
  theStack:
    re-ACK 3168b08043

Tree-SHA512: 308ac3d33e9a2deecb65826cbf0390480a38de201918429c35c796f3421cdf94c5501d027a043ae8f012cfaa0584656da1de6393bfba3532ab4c20f9533f06a6
2023-06-26 17:08:03 -04:00
fanquake
296735f763 Merge bitcoin/bitcoin#27947: MaybePunishNodeForTx: Remove unused message arg and logging
9fe5f6d5d1 MaybePunishNodeForTx: Remove unused message arg and logging (Greg Sanders)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 9fe5f6d5d1 🕚
  dergoegge:
    utACK 9fe5f6d5d1

Tree-SHA512: c40e39b0f6bd738675dadc707a2f5823aad8ddf532afa956579899f2b76edc6fb0bdc1ed78b4ed2f659ffe02c85d1d9fc16667f131e92562c2c0541bd8eee9bd
2023-06-26 10:39:32 +01:00
fanquake
80f04febbc Merge bitcoin/bitcoin#27957: net: remove unused CConnmanTest
9f0d129565 net: remove unused `CConnmanTest` (brunoerg)

Pull request description:

  `CConnmanTest` was removed in fa72fce7c9.

ACKs for top commit:
  theStack:
    ACK 9f0d129565

Tree-SHA512: 9ad974e8db700e6914a3ed5c936bfe0077cb7dcac915f4efccfe14ecf1917a0eafb37ad5ce6903ed81194bd99359ab300dababa3407f6f7b4d888d459782ad58
2023-06-26 10:05:54 +01:00
fanquake
931ac6f836 Merge bitcoin/bitcoin#27914: feerate: For GetFeePerK() return nSatoshisPerK instead of round trip through GetFee
11d650060a feerate: For GetFeePerK() return nSatoshisPerK instead of round trip through GetFee (Andrew Chow)

Pull request description:

  Returning the sats/kvb does not need to round trip through GetFee(1000) since the feerate is already stored as sats/kvb.

  Fixes #27913, although this does bring up a larger question of how we should handle such large feerates in fuzzing.

ACKs for top commit:
  furszy:
    Code ACK 11d65006

Tree-SHA512: bec1a0d4b572a0c810cf7eb4e97d729d67e96835c2d576a909f755b053a9707c2f1b3df9adb8f08a9c4d310cdbb8b1e1b42b9c004bd1ade02a07d8ce9e902138
2023-06-26 09:43:38 +01:00
Andrew Chow
8847497161 Merge bitcoin/bitcoin#27631: test: avoid sporadic MINIMALDATA failure in feature_taproot.py (fixes #27595)
54877253c8 test: avoid sporadic MINIMALDATA failure in feature_taproot.py (fixes #27595) (Sebastian Falbesoner)

Pull request description:

  The functional test feature_taproot.py fails in some rare cases on the execution of the following `"branched_codesep"` spending script (can be reproduced via `$ ./test/functional/feature_taproot.py --randomseed 9048710178866422833` on master / 137a98c5a2):

  9d85c03620/test/functional/feature_taproot.py (L741)

  The problem occurs if the first data-push (having random content with a random length in the range [0, 510]) has a length of 1 and the single byte has value of [1...16] or [-1]; in this case, the data-push is not minimally encoded by test framework's CScript class (i.e. doesn't use the special op-codes OP_1...OP_16 or OP_1NEGATE) and the script interpreter throws an SCRIPT_ERR_MINIMALDATA error:

  ```
  test_framework.authproxy.JSONRPCException: non-mandatory-script-verify-flag (Data push larger than necessary) (-26)
  ```

  Background: the functional test framework's CScript class translates passed bytes/bytearrays always to data pushes using OP_PUSHx/OP_PUSHDATA{1,2,4} op-codes (see `CScript.__coerce_instance(...)`). E.g. the expression `CScript(bytes([1]))` yields `bytes([OP_PUSH1, 1])` instead of the minimal-encoded `bytes([OP_1])`.

  Fix this by adapting the random-size range to [2,...], i.e. never pass byte-arrays below length two to be pushed.

  Closes #27595.

ACKs for top commit:
  instagibbs:
    ACK 54877253c8
  sipa:
    utACK 54877253c8
  achow101:
    ACK 54877253c8

Tree-SHA512: 3ffad89b2c3985c20702242192e744c9b10188bff880efaf3c38424a00fa07bd4608d8c948678ff9cdbb4e1e5b06696c7f55407ee10bb05edbb3ee03aa599cdc
2023-06-23 18:54:06 -04:00
Andrew Chow
50a664aceb Merge bitcoin/bitcoin#26969: net, refactor: net_processing, add ProcessCompactBlockTxns
77d6d89d43 net: net_processing, add `ProcessCompactBlockTxns` (brunoerg)

Pull request description:

  When processing `CMPCTBLOCK` message, at some moments we can need to process compact block txns / `BLOCKTXN`, since all messages are handled by `ProcessMessage`, so we call `ProcessMessage` all over again.
  ab98673f05/src/net_processing.cpp (L4331-L4348)

  This PR creates a function called `ProcessCompactBlockTxns` to process it to avoid calling `ProcessMessage` for it - this function is also called when processing `BLOCKTXN` msg.

ACKs for top commit:
  instagibbs:
    reACK 77d6d89d43
  ajtowns:
    utACK 77d6d89d43
  achow101:
    ACK 77d6d89d43

Tree-SHA512: 4b73c189487b999a04a8f15608a2ac1966d0f5c6db3ae0782641e68b9e95cb0807bd065d124c1f316b25b04d522a765addcd7d82c541702695113d4e54db4fda
2023-06-23 18:18:39 -04:00
Andrew Chow
035ae61c5a Merge bitcoin/bitcoin#27577: p2p: give seednodes time before falling back to fixed seeds
30778124b8 net: Give seednodes time before falling back to fixed seeds (Martin Zumsande)

Pull request description:

  `-seednode` is an alternative bootstrap mechanism - when choosing it, we make a `AddrFetch` connection to the specified peer, gather addresses from them, and then disconnect. Presumably, if users specify a seednode they prefer addresses from that node over fixed seeds.

  However, when disabling dns seeds and specifiying `-seednode`, `CConnman::ProcessAddrFetch()`  immediately removes the entry from `m_addr_fetches` (before the seednode could give us addresses) - and once `m_addr_fetches`  is empty, `ThreadOpenConnections` will add fixed seeds, resulting in a "race" between the fixed seeds and seednodes filling up AddrMan.

  This PR suggests to check for any provided `-seednode` arg instead of using the size of `m_addr_fetches`, thus delaying the querying of fixed seeds for 1 minute when specifying any seednode (as we already do for `addnode` peers).
  That way, we actually give the seednodes a chance for  to provide us with addresses before falling back to fixed seeds.

  This can be tested with `bitcoind -debug=net -dnsseed=0 -seednode=(...)` on a node without `peers.dat` and observing the debug log.

ACKs for top commit:
  ajtowns:
    utACK 30778124b8
  achow101:
    ACK 30778124b8
  dergoegge:
    Code review ACK 30778124b8
  sr-gi:
    ACK [3077812](30778124b8) with a tiny nit, feel free to ignore it

Tree-SHA512: 96446eb34c0805f10ee158a00a3001a07029e795ac40ad5638228d426e30e9bb836c64ac05d145f2f9ab23ec5a528f3a416e3d52ecfdfb0b813bd4b1ebab3c01
2023-06-23 17:39:58 -04:00
brunoerg
9f0d129565 net: remove unused CConnmanTest 2023-06-23 18:03:06 -03:00
Andrew Chow
8e0cf4f90c Merge bitcoin/bitcoin#27846: [coinselection] Increase SRD target by change_fee
1771daa815 [fuzz] Show that SRD budgets for non-dust change (Murch)
941b8c6539 [bug] Increase SRD target by change_fee (Murch)

Pull request description:

  I discovered via fuzzing of another coin selection approach that at extremely high feerates SRD may find input sets that lead to transactions without change outputs. This is an unintended outcome since SRD is meant to always produce a transaction with a change output—we use other algorithms to specifically search for changeless solutions.

  The issue occurs when the flat allowance of 50,000 ṩ for change is insufficient to pay for the creation of a change output with a non-dust amount, at and above 1,613 ṩ/vB. Increasing the change budget by `change_fee` makes SRD behave as expected at any feerates.

  Note: The intermittent failures of `test/functional/interface_usdt_mempool.py` are a known issue: https://github.com/bitcoin/bitcoin/issues/27380

ACKs for top commit:
  achow101:
    ACK 1771daa815
  S3RK:
    ACK 1771daa815

Tree-SHA512: 3f36a3e317ef0a711d0e409069c05032bff1d45403023f3728bf73dfd55ddd9e0dc2a9969d4d69fe0a426807ebb0bed1f54abfc05581409bfe42c327acf766d4
2023-06-23 16:57:11 -04:00
Andrew Chow
8fbb6e99bf wallet: Give deprecation warning when loading a legacy wallet 2023-06-23 16:37:22 -04:00
Andrew Chow
2c2150aa04 Merge bitcoin/bitcoin#26828: assumeutxo: catch and log fs::remove error instead of two exist checks
0e21b56a44 assumeutxo: catch and log fs::remove error instead of two exist checks (Andrew Toth)

Pull request description:

  Fixes a block of code which seems to be incorrectly performing two existence checks instead of catching and logging errors. `fs::remove` returns `false` only if the file being removed does not exist, so it is redundant with the `fs::exists` check. If an error does occur when trying to remove an existing file, `fs::remove` will throw. See https://en.cppreference.com/w/cpp/filesystem/remove.

  Also see https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L326-L332 for a similar pattern.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 0e21b56a44
  jamesob:
    ACK 0e21b56a44
  achow101:
    ACK 0e21b56a44

Tree-SHA512: 137d0be5266cfd947e5e50ec93b895ac659adadf9413bef3468744bfdacee8dbe7d9bdfaf91784c45708610325d2241a114f4be4e622a108a639b3672b618fd2
2023-06-23 16:21:43 -04:00
Andrew Chow
48aae2cffe gui: Add File > Migrate Wallet 2023-06-23 14:24:33 -04:00
Pieter Wuille
3168b08043 Bench test for EllSwift ECDH
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-06-23 14:24:32 -04:00
Andrew Chow
577be889cd gui: Optionally return passphrase after unlocking
AskPassphraseDialog has an optional parameter for the caller to get the
passphrase. Make this available for Unlocking.
2023-06-23 14:24:31 -04:00
dhruv
42d759f239 Bench tests for CKey->EllSwift
Co-Authored-By: Pieter Wuille <bitcoin-dev@wuille.net>
2023-06-23 14:24:28 -04:00
dhruv
2e5a8a437c Fuzz test for Ellswift ECDH
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-06-23 14:22:39 -04:00
dhruv
c3ac9f5cf4 Fuzz test for CKey->EllSwift->CPubKey creation/decoding
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-06-23 14:22:39 -04:00
dhruv
aae432a764 Unit test for ellswift creation/decoding roundtrip
Co-authored-by: Pieter Wuille <bitcoin-dev@wuille.net>
2023-06-23 14:22:39 -04:00
Pieter Wuille
eff72a0dff Add ElligatorSwift key creation and ECDH logic
Co-authored-by: Dhruv Mehta <856960+dhruv@users.noreply.github.com>
2023-06-23 14:22:33 -04:00
dhruv
42239f8390 Enable ellswift module in libsecp256k1 2023-06-23 14:15:49 -04:00
Andrew Chow
9ea31eba04 gui: Disable and uncheck blank when private keys are disabled
Unify the GUI's create wallet with the RPC createwallet so that the
blank flag is not set when private keys are disabled.
2023-06-23 13:40:41 -04:00
stickies-v
79d343a642 http: update libevent workaround to correct version
The libevent bug described in 5ff8eb2637
was already patched in release-2.1.9-beta, with cherry-picked
commits 5b40744d1581447f5b4496ee8d4807383e468e7a and
b25813800f97179b2355a7b4b3557e6a7f568df2.

There should be no side-effects by re-applying the workaround on
an already patched version of libevent, but it is best to set the
correct version number to avoid confusion.
2023-06-23 17:21:27 +01:00
Greg Sanders
9fe5f6d5d1 MaybePunishNodeForTx: Remove unused message arg and logging 2023-06-23 12:20:54 -04:00
Hennadii Stepanov
529c92e837 guix: Update python-lief package to 0.13.2 2023-06-23 17:02:49 +01:00
stickies-v
9f55773a37 test: refactor: usdt_mempool: store all events
Even though we expect these functions to only produce one event,
we still keep a counter to check if that's true. By simply storing
all the events, we can remove the counters and make debugging
easier, by allowing pdb to access the events.
2023-06-23 16:47:43 +01:00
stickies-v
bc43270450 test: refactor: remove unnecessary nonlocal
Since we're only mutating, and not reassigning, we don't need to
declare `events` as `nonlocal`.
2023-06-23 16:47:43 +01:00
stickies-v
326db63a68 test: log sanity check assertion failures 2023-06-23 16:47:43 +01:00
stickies-v
f5525ad680 test: store utxocache events
By storing the events instead of doing the comparison inside the
handle_utxocache_* functions, we simplify the overall logic and
potentially making debugging easier, by allowing pdb to access the
events.

Mostly a refactor, but changes logging behaviour slightly by not
raising and not calling self.log.exception("Assertion failed")
2023-06-23 16:46:46 +01:00
stickies-v
f1b99ac94f test: refactor: deduplicate handle_utxocache_* logic
Carve out the comparison logic into a helper function to avoid
code duplication.
2023-06-23 16:46:32 +01:00
stickies-v
ad90ba36bd test: refactor: rename inbound to is_inbound
Makes it easier to recognize this variable represents a flag.
2023-06-23 16:46:15 +01:00
stickies-v
afc0224cdb test: refactor: remove unnecessary blocks_checked counter
Since we already store all the blocks in `events`, keeping an
additional counter is redundant.
2023-06-23 16:46:02 +01:00
MarcoFalke
fa748c6f2a test: Fix intermittent issue in mining_getblocktemplate_longpoll.py 2023-06-23 15:21:29 +02:00
MarcoFalke
fae55f989e test: Add implicit-signed-integer-truncation:*/include/c++/ suppression 2023-06-23 12:03:51 +02:00
Luke Dashjr
099dbe4224 GUI: TransactionRecord: When time/index/etc match, sort send before receive 2023-06-23 02:06:54 +00:00
Luke Dashjr
2d182f77cd Bugfix: Ignore ischange flag when we're not the sender
If we didn't send it, it can't possibly be change, even if that's the key's purpose
2023-06-23 02:06:32 +00:00
Luke Dashjr
71fbdb7f40 GUI: Remove SendToSelf TransactionRecord type 2023-06-23 02:06:02 +00:00
Luke Dashjr
f3fbe99fcf GUI: TransactionRecord: Refactor to turn send-to-self into send+receive pairs 2023-06-23 02:05:24 +00:00
Luke Dashjr
b9765ba1d6 GUI: TransactionRecord: Use "any from me" as the criteria for deciding whether a transaction is a send or receive
This changes behaviour (IMO for the better) in the case where some but not all inputs are from us, and the net amount is positive.
2023-06-23 02:05:19 +00:00
Andrew Chow
6a473373d4 Merge bitcoin/bitcoin#27862: validation: Stricter assumeutxo error handling when renaming chainstates
1c7d08b9ac validation: Stricter assumeutxo error handling in InvalidateCoinsDBOnDisk (Ryan Ofsky)
9047337d36 validation: Stricter assumeutxo error handling in LoadChainstate (Ryan Ofsky)

Pull request description:

  There are two places in assumeutxo code where it is calling `AbortNode` to trigger asynchronous shutdowns without returning errors to calling functions.

  One case, in `LoadChainstate`, happens when snapshot validation succeeds, and there is an error trying to replace the background chainstate with the snapshot chainstate.

  The other case, in `InvalidateCoinsDBOnDisk`, happens when snapshot validatiion fails, and there is an error trying to remove the snapshot chainstate.

  In both cases the node is being forced to shut down, so it makes sense for these functions to raise errors so callers can know that an error happened without having to infer it from the shutdown state.

  Noticed these cases while reviewing #27861, which replaces the `AbortNode` function with a `FatalError` function.

ACKs for top commit:
  achow101:
    ACK 1c7d08b9ac
  TheCharlatan:
    ACK 1c7d08b9ac
  jamesob:
    ACK 1c7d08b9ac ([`jamesob/ackr/27862.1.ryanofsky.validation_stricter_assu`](https://github.com/jamesob/bitcoin/tree/ackr/27862.1.ryanofsky.validation_stricter_assu))

Tree-SHA512: fb1dcde3fa0e77b4ba0c48507d289552b939c2866781579c8e994edc209abc3cd29cf81c89380057199323a8eec484956abb1fd3a43c957ecd0e7f7bbfd63fd8
2023-06-22 13:20:36 -04:00
fanquake
3df6070466 contrib: remove macOS lazy_bind check
In future, this will be replaced by a check for fixup_chains usage.
2023-06-22 15:28:47 +00:00
Cory Fields
9bc357e205 build: explicitly opt-in to new fixup_chains functionality for darwin
This replaces (but does not collide with) the previous bind_on_load. There
is technically no need to opt-in to this functionality as long as >= MacOS 11.0
is being targetted, but it will be helpful to see in the logs.
2023-06-22 15:28:47 +00:00
Cory Fields
fb61bc0c02 depends: Bump MacOS minimum runtime requirement to 11.0
This is necessary as the new fixup_chains linker behavior is only valid
when the runtime target is >=11.0.
2023-06-22 15:28:47 +00:00
Cory Fields
c2cd47280c depends: bump darwin clang to 11.1
Unfortunately clang 10 does not understand "-mmacosx-version-min=11.0",
as it expects to see only 10.x.

Bump minimally to 11.1 to fix that problem. This will likely be our last
binary toolchain bump, as it will soon be replaced with usage of upstream
vanilla llvm.
2023-06-22 15:28:47 +00:00
fanquake
a7261da479 Merge bitcoin/bitcoin#27831: test: handle failed assert_equal() assertions in bcc callback functions
61f4b9b7ad Manage exceptions in bcc callback functions (virtu)

Pull request description:

  Address #27380 (and similar future issues) by handling failed `assert_equal()` assertions in bcc callback functions

  ### Problem

  Exceptions are not propagated in ctype callback functions used by bcc. This means an AssertionError exception raised by `assert_equal()` to signal a failed assertion is not getting caught and properly logged. Instead, the error is logged to stdout and execution of the callback stops.

  The current workaround to check whether all `assert_equal()` assertions in a callback succeeded is to increment a success counter after the assertions (which only gets incremented if none exception is raised and stops execution). Then, outside the callback, the success counter can be used to check whether a callback executed successfully.

  One issue with the described workaround is that when an exception occurs, there is no way of telling which of the `assert_equal()` statements caused the exception; moreover, there is no way of inspecting how the pieces of data that got compared in `assert_equal()` differed (often a crucial clue when debugging what went wrong).

  This problem is happening in #27380: Sporadically, in the `mempool:rejected` test, execution does not reach the end of the callback function and the success counter is not incremented. Thus, the test fails when comparing the counter to its expected value of one. Without knowing which of the asserts failed any why it failed, this issue is hard to debug.

  ### Solution

  Two fixes come to mind. The first involves having the callback function make event data accessible outside the callback and inspecting the event using `assert_equal()` outside the callback. This solution still requires a counter in the callback in order  to tell whether a callback was actually executed or if instead the call to perf_buffer_poll() timed out.

  The second fix entails wrapping all relevant `assert_equal()` statements inside callback functions into try-catch blocks and manually logging AssertionErrors. While not as elegant in terms of design, this approach can be more pragmatic for more complex tests (e.g., ones involving multiple events, events of different types, or the order of events).

  The solution proposed here is to select the most pragmatic fix on a case-by-case basis: Tests in `interface_usdt_net.py`, `interface_usdt_mempool.py` and `interface_usdt_validation.py` have been refactored to use the first approach, while the second approach was chosen for `interface_usdt_utxocache.py` (partly to provide a reference for the second approach, but mainly because the utxocache tests are the most intricate tests, and refactoring them to use the first approach would negatively impact their readability). Lastly, `interface_usdt_coinselection.py` was kept unchanged because it does not use `assert_equal()` statements inside callback functions.

ACKs for top commit:
  0xB10C:
    Reviewed the changes since my last review. ACK 61f4b9b7ad. I've tested that the combined log contains both exceptions by modifying `interface_usdt_utxocache.py`.
  willcl-ark:
    utACK 61f4b9b
  stickies-v:
    utACK 61f4b9b7a

Tree-SHA512: 85cdaabf370d4f09a9eab6af9ce7c796cd9d08cb91f38f021f71adda34c5f643331022dd09cadb95be2185dad6016c95cbb8942e41e4fbd566a49bf431c5141a
2023-06-22 16:08:15 +01:00
kevkevin
d05be124db test: added coverage to estimatefee
Added a assert for an rpc error when we try to estimate fee for the max
conf_target
2023-06-22 08:27:38 -05:00
MarcoFalke
fae7c50d20 test: Run fuzz tests on macOS
Also, fix a few bugs:

* Error: RPC command "enumeratesigners" not found in RPC_COMMANDS_SAFE_FOR_FUZZING or RPC_COMMANDS_NOT_SAFE_FOR_FUZZING. Please update test/fuzz/rpc.cpp.
* in run_once: ...format(" ".join(result.args), ... TypeError: sequence item 2: expected str instance, PosixPath found
2023-06-22 13:54:17 +02:00
fanquake
2880bb588a Merge bitcoin/bitcoin#27889: test: Kill BOOST_ASSERT and update the linter
28fff06afe test: Make linter to look for `BOOST_ASSERT` macros (Hennadii Stepanov)
47fe551e52 test: Kill `BOOST_ASSERT` (Hennadii Stepanov)

Pull request description:

  One of the goals of https://github.com/bitcoin/bitcoin/pull/27783 was to get rid of the `BOOST_ASSERT` macros instead of including the `boost/assert.hpp` headers. See https://github.com/bitcoin/bitcoin/pull/27783#discussion_r1210612717.

  It turns out that a couple of those macros sneaked into the codebase in https://github.com/bitcoin/bitcoin/pull/27790.

  This PR makes the linter guard against new instances of the `BOOST_ASSERT` macros and replaces the current ones.

ACKs for top commit:
  kevkevinpal:
    ACK [28fff06](28fff06afe)
  stickies-v:
    ACK 28fff06af
  TheCharlatan:
    ACK 28fff06afe

Tree-SHA512: 371f613592cf677afe0196d18c83943c6c8f1e998f57b4ff3ee58bfeff8636e4dac1357840d8611b4f7b197def94df10fe1a8ca3282b00b7b4eff4624552dda8
2023-06-22 12:33:35 +01:00
fanquake
0c84a0e484 Merge bitcoin/bitcoin#27798: depends: modernize clang flags for Darwin
cbee1d7091 depends: modernize clang flags (Cory Fields)
2a85857ce5 ci: disable false-positive warnings for now (Cory Fields)

Pull request description:

  This is a cleaner and simpler alternative to #25098. Inspired by [this conversation](https://github.com/bitcoin/bitcoin/pull/27737#issuecomment-1562543301). The diff is large but the change itself is quite small.

  Fixes builds with llvm >= 11 in guix by working around the problem. As a bonus, this is much cleaner and more maintainable than what we had before.

  See the updated comment for more info. At a high level: rather than playing tricks and trying to work around clang's default includes, disable them and re-add what we want.

ACKs for top commit:
  fanquake:
    ACK cbee1d7091 - tested Guix and the depends cross-compile. Would like to move this along, to unblock #27676, which itself might be a blocker for #27897. Note that macOS might seem somewhat in flux for the moment, but once we finish the migration to LLVM Clang + LLD, things will be must simpler, and ultimately more maintainable.
  TheCharlatan:
    ACK cbee1d7091

Tree-SHA512: 5a8300be528f550e15ab23d869e77df7a62201c6d40c0384795a9eecee38118a676e0b79b2b76c5e597597181443caada54a01b75a544dbcde76da1deba8e3a4
2023-06-22 09:47:30 +01:00
Brotcrunsher
5fc4939e17 Added static_assert to check that base_blob is using whole bytes.
Prior to this commit it was possible to create base_blobs with any arbitrary amount of bits, like base_blob<9>. One could assume that this would be a valid way to create a bit field that guarantees to have at least 9 bits. However, in such a case, base_blob would not behave as expected because the WIDTH is rounded down to the closest whole byte (simple integer division by 8). This commit makes sure that this oddity is detected and blocked by the compiler.
2023-06-22 01:31:06 +02:00
Murch
1771daa815 [fuzz] Show that SRD budgets for non-dust change
Adding this assert to the fuzz test without increasing the change target
by the change_fee resulted in a crash within a few seconds.
2023-06-21 16:19:26 -04:00
Murch
941b8c6539 [bug] Increase SRD target by change_fee
I discovered via fuzzing of another coin selection approach that at
extremely high feerates SRD may find input sets that lead to
transactions without change outputs. This is an unintended outcome since
SRD is meant to always produce a transaction with a change output—we use
other algorithms to specifically search for changeless solutions.

The issue occures when the flat allowance of 50,000 ṩ for change is
insufficient to pay for the creation of a change output with a non-dust
amount, at and above 1,613 ṩ/vB. Increasing the change budget by
change_fees makes SRD behave as expected at any feerates.
2023-06-21 16:19:19 -04:00
Kiminuo
f1d807e383 Add more tests for the BIP21 implementation 2023-06-21 21:34:05 +02:00
Martin Zumsande
30778124b8 net: Give seednodes time before falling back to fixed seeds
Before, we'd remove a seednode from the list right after connecting
to it, leading to a race with loading the fixed seed and connecting
to them.
2023-06-21 15:11:00 -04:00
fanquake
f1b4975461 Merge bitcoin/bitcoin#27921: fuzz: Avoid OOM in transaction fuzz target
fa31c4daac fuzz: Avoid OOM in transaction fuzz target (MarcoFalke)

Pull request description:

  To test: `FUZZ=transaction /usr/bin/time -f '%Us %MkB' ./src/test/fuzz/fuzz ../btc_qa_assets/fuzz_seed_corpus/transaction/9dc22b51df0af05ee5a595beefb0ce291feb6b99`

  Before: `0.72s 249636kB`
  After: `0.30s 92128kB`

ACKs for top commit:
  dergoegge:
    utACK fa31c4daac

Tree-SHA512: 958fc54e7af31af7db3e3e1fb37553ae24de251c7fdeea3d68ec168f03db48de6aa54a96bf971f9cc804e94ff8a02fda9c56d7e85869d62962f6f020568e3a7b
2023-06-21 16:40:25 +01:00
Pieter Wuille
901336eee7 Squashed 'src/secp256k1/' changes from 4258c54f4e..705ce7ed8c
705ce7ed8c Merge bitcoin-core/secp256k1#1129: ElligatorSwift + integrated x-only DH
0702ecb061 Merge bitcoin-core/secp256k1#1338: Drop no longer needed `#include "../include/secp256k1.h"`
90e360acc2 Add doc/ellswift.md with ElligatorSwift explanation
4f091847c2 Add ellswift testing to CI
1bcea8c57f Add benchmarks for ellswift module
2d1d41acf8 Add ctime tests for ellswift module
df633cdeba Add _prefix and _bip324 ellswift_xdh hash functions
9695deb351 Add tests for ellswift module
c47917bbd6 Add ellswift module implementing ElligatorSwift
79e5b2a8b8 Add functions to test if X coordinate is valid
a597a5a9ce Add benchmark for key generation
30574f22ea Merge bitcoin-core/secp256k1#1349: Normalize ge produced from secp256k1_pubkey_load
45c5ca7675 Merge bitcoin-core/secp256k1#1350: scalar: introduce and use `secp256k1_{read,write}_be64` helpers
f1652528be Normalize ge produced from secp256k1_pubkey_load
7067ee54b4 tests: add tests for `secp256k1_{read,write}_be64`
740528caad scalar: use newly introduced `secp256k1_{read,write}_be64` helpers (4x64 impl.)
67214f5f7d Merge bitcoin-core/secp256k1#1339: scalar: refactor: use `secp256k1_{read,write}_be32` helpers
cb1a59275c Merge bitcoin-core/secp256k1#1341: docs: correct `pubkey` param descriptions for `secp256k1_keypair_{xonly_,}pub`
f3644287b1 docs: correct `pubkey` param descriptions for `secp256k1_keypair_{xonly_,}pub`
887183e7de scalar: use `secp256k1_{read,write}_be32` helpers (4x64 impl.)
52b84238de scalar: use `secp256k1_{read,write}_be32` helpers (8x32 impl.)
e449af6872 Drop no longer needed `#include "../include/secp256k1.h"`
60556c9f49 Merge bitcoin-core/secp256k1#1337: ci: Fix error D8037 in `cl.exe` (attempt 2)
db29bf220c ci: Remove quirk that runs dummy command after wineserver
c7db4942b3 ci: Fix error D8037 in `cl.exe`
7dae115861 Revert "ci: Move wine prefix to /tmp to avoid error D8037 in cl.exe"
bf29f8d0a6 Merge bitcoin-core/secp256k1#1334: fix input range comment for `secp256k1_fe_add_int`
605e07e365 fix input range comment for `secp256k1_fe_add_int`
debf3e5c08 Merge bitcoin-core/secp256k1#1330: refactor: take use of `secp256k1_scalar_{zero,one}` constants
d75dc59b58 Merge bitcoin-core/secp256k1#1333: test: Warn if both `VERIFY` and `COVERAGE` are defined
ade5b36701 tests: add checks for scalar constants `secp256k1_scalar_{zero,one}`
e83801f5db test: Warn if both `VERIFY` and `COVERAGE` are defined
654246c635 refactor: take use of `secp256k1_scalar_{zero,one}` constants
908e02d596 Merge bitcoin-core/secp256k1#1328: build: Bump MSVC warning level up to W3
1549db0ca5 build: Level up MSVC warnings
20a5da5fb1 Merge bitcoin-core/secp256k1#1310: Refine release process
ad84603297 release process: clarify change log updates
6348bc7eee release process: fix process for maintenance release
79fa50b082 release process: mention targeted release schedule
165206789b release process: add sanity checks
09df0bfb23 Merge bitcoin-core/secp256k1#1327: ci: Move wine prefix to /tmp to avoid error D8037 in cl.exe
27504d5c94 ci: Move wine prefix to /tmp to avoid error D8037 in cl.exe
d373a7215b Merge bitcoin-core/secp256k1#1316: Do not invoke fe_is_zero on failed set_b32_limit
6433175ffe Do not invoke fe_is_zero on failed set_b32_limit
5f7903c73c Merge bitcoin-core/secp256k1#1318: build: Enable -DVERIFY for precomputation binaries
e9e4526a4e Merge bitcoin-core/secp256k1#1317: Make fe_cmov take max of magnitudes
5768b50229 build: Enable -DVERIFY for precomputation binaries
31b4bbee1e Make fe_cmov take max of magnitudes
83186db34a Merge bitcoin-core/secp256k1#1314: release cleanup: bump version after 0.3.2
95448ef2f8 release cleanup: bump version after 0.3.2
acf5c55ae6 Merge bitcoin-core/secp256k1#1312: release: Prepare for 0.3.2
d490ca2046 release: Prepare for 0.3.2
3e3d125b83 Merge bitcoin-core/secp256k1#1309: changelog: Catch up
e8295d07ab Merge bitcoin-core/secp256k1#1311: Revert "Remove unused scratch space from API"
697e1ccf4a changelog: Catch up
3ad1027a40 Revert "Remove unused scratch space from API"
76b43f3443 changelog: Add entry for #1303
7d4f86d242 Merge bitcoin-core/secp256k1#1307: Mark more assembly outputs as early clobber
b54a0672ef Merge bitcoin-core/secp256k1#1304: build: Rename arm to arm32 and check if it's really supported
c6bb29b303 build: Rename `64bit` to `x86_64`
8c9ae37a5a Add release note
03246457a8 autotools: Add `SECP_ARM32_ASM_CHECK` macro
ed4ba238e2 cmake: Add `check_arm32_assembly` function
350b4bd6e6 Mark stack variables as early clobber for technical correctness
0c729ba70d Bugfix: mark outputs as early clobber in scalar x86_64 asm
3353d3c753 Merge bitcoin-core/secp256k1#1207: Split fe_set_b32 into reducing and normalizing variants
5b32602295 Split fe_set_b32 into reducing and normalizing variants
006ddc1f42 Merge bitcoin-core/secp256k1#1306: build: Make tests work with external default callbacks
1907f0f166 build: Make tests work with external default callbacks
fb3a806365 Merge bitcoin-core/secp256k1#1133: schnorrsig: Add test vectors for variable-length messages
cd54ac7c1c schnorrsig: Improve docs of schnorrsig_sign_custom
28687b0312 schnorrsig: Add BIP340 varlen test vectors
97a98bed1e schnorrsig: Refactor test vector code to allow varlen messages
ab5a917128 Merge bitcoin-core/secp256k1#1303: ct: Use more volatile
9eb6934f69 Merge bitcoin-core/secp256k1#1305: Remove unused scratch space from API
073d98a076 Merge bitcoin-core/secp256k1#1292: refactor: Make 64-bit shift explicit
17fa21733a ct: Be cautious and use volatile trick in more "conditional" paths
5fb336f9ce ct: Use volatile trick in scalar_cond_negate
712e7f8722 Remove unused scratch space from API
54d34b6c24 Merge bitcoin-core/secp256k1#1300: Avoid normalize conditional on VERIFY
c63ec88ebf Merge bitcoin-core/secp256k1#1066: Abstract out and merge all the magnitude/normalized logic
7fc642fa25 Simplify secp256k1_fe_{impl_,}verify
4e176ad5b9 Abstract out verify logic for fe_is_square_var
4371f98346 Abstract out verify logic for fe_add_int
89e324c6b9 Abstract out verify logic for fe_half
283cd80ab4 Abstract out verify logic for fe_get_bounds
d5aa2f0358 Abstract out verify logic for fe_inv{,_var}
3167646072 Abstract out verify logic for fe_from_storage
76d31e5047 Abstract out verify logic for fe_to_storage
1e6894bdd7 Abstract out verify logic for fe_cmov
be82bd8e03 Improve comments/checks for fe_sqrt
6ab35082ef Abstract out verify logic for fe_sqr
4c25f6efbd Abstract out verify logic for fe_mul
e179e651cb Abstract out verify logic for fe_add
7e7ad7ff57 Abstract out verify logic for fe_mul_int
65d82a3445 Abstract out verify logic for fe_negate
144670893e Abstract out verify logic for fe_get_b32
f7a7666aeb Abstract out verify logic for fe_set_b32
ce4d2093e8 Abstract out verify logic for fe_cmp_var
7d7d43c6dd Improve comments/check for fe_equal{,_var}
c5e788d672 Abstract out verify logic for fe_is_odd
d3f3fe8616 Abstract out verify logic for fe_is_zero
c701d9a471 Abstract out verify logic for fe_clear
19a2bfeeea Abstract out verify logic for fe_set_int
864f9db491 Abstract out verify logic for fe_normalizes_to_zero{,_var}
6c31371120 Abstract out verify logic for fe_normalize_var
e28b51f522 Abstract out verify logic for fe_normalize_weak
b6b6f9cb97 Abstract out verify logic for fe_normalize
7fa5195559 Bugfix: correct SECP256K1_FE_CONST mag/norm fields
e5cf4bf3ff build: Rename `arm` to `arm32`
b29566c51b Merge magnitude/normalized fields, move/improve comments
97c63b9039 Avoid normalize conditional on VERIFY
341cc19726 Merge bitcoin-core/secp256k1#1299: Infinity handling: ecmult_const(infinity) works, and group verification
bbc834467c Avoid secp256k1_ge_set_gej_zinv with uninitialized z
0a2e0b2ae4 Make secp256k1_{fe,ge,gej}_verify work as no-op if non-VERIFY
f20266722a Add invariant checking to group elements
a18821d5b1 Always initialize output coordinates in secp256k1_ge_set_gej
3086cb90ac Expose secp256k1_fe_verify to other modules
a0e696fd4d Make secp256k1_ecmult_const handle infinity
24c768ae09 Merge bitcoin-core/secp256k1#1301: Avoid using bench_verify_data as bench_sign_data; merge them
2e65f1fdbc Avoid using bench_verify_data as bench_sign_data; merge them
1cf15ebd94 Merge bitcoin-core/secp256k1#1296: docs: complete interface description for `secp256k1_schnorrsig_sign_custom`
149c41cee1 docs: complete interface description for `secp256k1_schnorrsig_sign_custom`
f30c74866b Merge bitcoin-core/secp256k1#1270: cmake: Fix library ABI versioning
d1e48e5474 refactor: Make 64-bit shift explicit
b2e29e43d0 ci: Treat all compiler warnings as errors in "Windows (VS 2022)" task
3c81838856 Merge bitcoin-core/secp256k1#1289: cmake: Use full signature of `add_test()` command
755629bc03 cmake: Use full signature of `add_test()` command
bef448f9af cmake: Fix library ABI versioning
4b0f711d46 Merge bitcoin-core/secp256k1#1277: autotools: Clean up after adding Wycheproof
222ecaf661 Merge bitcoin-core/secp256k1#1284: cmake: Some improvements using `PROJECT_IS_TOP_LEVEL` variable
71f746c057 cmake: Include `include` directory for subtree builds
024a409484 Merge bitcoin-core/secp256k1#1240: cmake: Improve and document compiler flag checks
a8d059f76c cmake, doc: Document compiler flags
6ece1507cb cmake, refactor: Rename `try_add_compile_option` to `try_append_cflags`
19516ed3e9 cmake: Use `add_compile_options()` in `try_add_compile_option()`
4b84f4bf0f Merge bitcoin-core/secp256k1#1239: cmake: Bugfix and other improvements after bumping CMake up to 3.13
596b336ff6 Merge bitcoin-core/secp256k1#1234: cmake: Add dev-mode
6b7e5b717d Merge bitcoin-core/secp256k1#1275: build: Fix C4005 "macro redefinition" MSVC warnings in examples
1c89536718 Merge bitcoin-core/secp256k1#1286: tests: remove extra semicolon in macro
c4062d6b5d debug: move helper for printing buffers into util.h
7e977b3c50 autotools: Take VPATH builds into account when generating testvectors
2418d3260a autotools: Create src/wycheproof dir before creating file in it
8764034ed5 autotools: Make all "pregenerated" targets .PHONY
e1b9ce8811 autotools: Use same conventions for all pregenerated files
3858bad2c6 tests: remove extra semicolon in macro
1f33bb2b1c Merge bitcoin-core/secp256k1#1205: field: Improve docs +tests of secp256k1_fe_set_b32
162da73e9a tests: Add debug helper for printing buffers
e9fd3dff76 field: Improve docs and tests of secp256k1_fe_set_b32
f6bef03c0a Merge bitcoin-core/secp256k1#1283: Get rid of secp256k1_fe_const_b
5431b9decd cmake: Make `SECP256K1_INSTALL` default depend on `PROJECT_IS_TOP_LEVEL`
5ec1333d4f Merge bitcoin-core/secp256k1#1285: bench: Make sys/time.h a system include
68b16a1662 bench: Make sys/time.h a system include
162608cc98 cmake: Emulate `PROJECT_IS_TOP_LEVEL` for CMake<3.21
69e1ec0331 Get rid of secp256k1_fe_const_b
ce5ba9e24d gitignore: Add CMakeUserPresets.json
0a446a312f cmake: Add dev-mode CMake preset
a6f4bcf6e1 Merge bitcoin-core/secp256k1#1231: Move `SECP256K1_INLINE` macro definition out from `include/secp256k1.h`
a273d74b2e cmake: Improve version comparison
6a58b483ef cmake: Use `if(... IN_LIST ...)` command
2445808c02 cmake: Use dedicated `GENERATOR_IS_MULTI_CONFIG` property
9f8703ef17 cmake: Use dedicated `CMAKE_HOST_APPLE` variable
8c2017035a cmake: Use recommended `add_compile_definitions` command
04d4cc071a cmake: Add `DESCRIPTION` and `HOMEPAGE_URL` options to `project` command
8a8b6536ef cmake: Use `SameMinorVersion` compatibility mode
5b0444a3b5 Merge bitcoin-core/secp256k1#1263: cmake: Make installation optional
47ac3d63cd cmake: Make installation optional
2e035af251 Merge bitcoin-core/secp256k1#1273: build: Make `SECP_VALGRIND_CHECK` preserve `CPPFLAGS`
5be353d658 Merge bitcoin-core/secp256k1#1279: tests: lint wycheproof's python script
08f4b1632d autotools: Move code around to tidy Makefile
04bf3f6778 Merge bitcoin-core/secp256k1#1230: Build: allow static or shared but not both
9ce9984f32 Merge bitcoin-core/secp256k1#1265: Remove bits argument from secp256k1_wnaf_const{_xonly}
566faa17d3 Merge bitcoin-core/secp256k1#1267: doc: clarify process for patch releases
ef49a11d29 build: allow static or shared but not both
35ada3b954 tests: lint wycheproof's python script
529b54d922 autotools: Move Wycheproof header from EXTRA_DIST to noinst_HEADERS
dc0657c762 build: Fix C4005 "macro redefinition" MSVC warnings in examples
1ecb94ebe9 build: Make `SECP_VALGRIND_CHECK` preserve `CPPFLAGS`
1b6fb5593c doc: clarify process for patch releases
a575339c02 Remove bits argument from secp256k1_wnaf_const (always 256)
36b0adf1b9 build: remove warning until it's reproducible
8e142ca410 Move `SECP256K1_INLINE` macro definition out from `include/secp256k1.h`
77445898a5 Remove `SECP256K1_INLINE` usage from examples
ca92a35d01 field: Simplify code in secp256k1_fe_set_b32
d93f62e369 field: Verify field element even after secp256k1_fe_set_b32 fails

git-subtree-dir: src/secp256k1
git-subtree-split: 705ce7ed8c1557a31e1bfc99be06082c5098d9f5
2023-06-21 11:04:00 -04:00
Pieter Wuille
a143a12d44 Update src/secp256k1 subtree to version with ElligatorSwift support 2023-06-21 11:04:00 -04:00
fanquake
d23cdf659e Merge bitcoin/bitcoin#27905: validation: add missing insert to m_dirty_blockindex
e639364495 validation: add missing insert to m_dirty_blockindex (Martin Zumsande)

Pull request description:

  When the status of a block index is changed, we must add it to `m_dirty_blockindex` or the change might not get persisted to disk.
  This is missing from one spot in `FindMostWorkChain()`, where `BLOCK_FAILED_CHILD` is set.
  Since we have [code](f0758d8a66/src/node/blockstorage.cpp (L284-L287)) that later sets missing `BLOCK_FAILED_CHILD` during the next startup, I don't think that this can lead to bad block indexes in practice, but I still think it's worth fixing.

ACKs for top commit:
  TheCharlatan:
    ACK e639364495
  stickies-v:
    ACK e639364495

Tree-SHA512: a97af9c173e31b90b677a1f95de822e08078d78013de5fa5fe4c3bec06f45d6e1823b7694cdacb887d031329e4b4afc6a2003916e0ae131279dee71f43e1f478
2023-06-21 13:40:28 +01:00
fanquake
e410fb711c Merge bitcoin/bitcoin#27822: Renamed UniValue::__pushKV to UniValue::pushKVEnd.
bdea2bb114 scripted-diff: Following the C++ Standard rules for identifiers with _. (Brotcrunsher)

Pull request description:

  Any identifier starting with 2 _ is reserved for the compiler and thus must not be used.

  See: https://stackoverflow.com/a/228797/7130273

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bdea2bb114

Tree-SHA512: 74c8e676449f3f61476d846bfd2c514103c8914e13c4a0db841203abdc0267c25ddc6ed57d6791459efe3edea17753a1b53c3795071ddfe8aba8662521063407
2023-06-21 11:22:40 +01:00
fanquake
7d65e3372f Merge bitcoin/bitcoin#27733: test: refactor: introduce generate_keypair helper with WIF support
1a572ce7d6 test: refactor: introduce `generate_keypair` helper with WIF support (Sebastian Falbesoner)

Pull request description:

  In functional tests it is a quite common scenario to generate fresh elliptic curve keypairs, which is currently a bit cumbersome as it involves multiple steps, e.g.:

      privkey = ECKey()
      privkey.generate()
      privkey_wif = bytes_to_wif(privkey.get_bytes())
      pubkey = privkey.get_pubkey().get_bytes()

  Simplify this by providing a new `generate_keypair` helper function that returns the private key either as `ECKey` object or as WIF-string (depending on the boolean `wif` parameter) and the public key as byte-string; these formats are what we mostly need (currently we don't use `ECPubKey` objects from generated keypairs anywhere).

  With this, most of the affected code blocks following the pattern above can be replaced by one-liners, e.g.:

      privkey, pubkey = generate_keypair(wif=True)

  Note that after this commit, the only direct uses of `ECKey` remain in situations where we want to set the private key explicitly, e.g. in MiniWallet (test/functional/test_framework/wallet.py) or the test for the signet miner script (test/functional/tool_signet_miner.py).

ACKs for top commit:
  instagibbs:
    ACK 1a572ce7d6
  kevkevinpal:
    reACK [1a572ce](1a572ce7d6)
  stratospher:
    ACK 1a572ce7. neat to have this since keypair generation is done in lots of places.

Tree-SHA512: ceb695ba7b34dc9f65357b55be03e67609e7e13a178083d405284eff4d8d3c5cea4fb0b6632658604a533f38ebfefc33e0c375995cc21ebc7843442ad764287b
2023-06-21 10:45:25 +01:00
fanquake
a596bdf3e9 Merge bitcoin/bitcoin#27919: ci: Run fuzz target even if input folder is empty
0000f55293 ci: Run fuzz target even if input folder is empty (MarcoFalke)

Pull request description:

  This should catch trivial integer sanitizer bugs if the author and all reviewers forget to look for them.

ACKs for top commit:
  brunoerg:
    reACK 0000f55293
  dergoegge:
    reACK 0000f55293

Tree-SHA512: f139b9d56f0cf1aae339c2890721c77c88d1fea77b73d492c1386ec99b4f393c5b664029919ff4a22e4e8a2929f085699a148c6acc2cc3e40df8a72fd39ff474
2023-06-21 10:08:53 +01:00
fanquake
8d5b93cf54 Merge bitcoin/bitcoin#27922: ci: fix llvm-symbolizer in MSAN jobs
682274aab0 ci: install llvm-symbolizer in MSAN jobs (fanquake)
96527cd51e ci: use LLVM 16.0.6 in MSAN jobs (fanquake)

Pull request description:

  Fixes: https://github.com/bitcoin/bitcoin/pull/27737#issuecomment-1599007233.

  Tested (locally) with #27495 that it produces a symbolized backtrace:
  ```bash
  2023-06-20T17:5Uninitialized bytes in __interceptor_strlen at offset 113 inside [0x719000006908, 114)
  ==35429==WARNING: MemorySanitizer: use-of-uninitialized-value
      #0 0x56060fae8c4b in sqlite3Strlen30 /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:32670:28
      #1 0x56060fb0fcf4 in sqlite3PagerOpen /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:57953:17
      #2 0x56060fb0f48b in sqlite3BtreeOpen /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:68679:10
      #3 0x56060fb01384 in openDatabase /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:171911:8
      #4 0x56060fb016ca in sqlite3_open_v2 /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:172034:10
      #5 0x56060e8a94db in wallet::SQLiteDatabase::Open() src/wallet/sqlite.cpp:250:19
      #6 0x56060e8a30fd in wallet::SQLiteDatabase::SQLiteDatabase(fs::path const&, fs::path const&, wallet::DatabaseOptions const&, bool) src/wallet/sqlite.cpp:133:9
      #7 0x56060e8b78f5 in std::__1::__unique_if<wallet::SQLiteDatabase>::__unique_single std::__1::make_unique[abi:v160006]<wallet::SQLiteDatabase, std::__1::__fs::filesystem::path, fs::path&, wallet::DatabaseOptions const&>(std::__1::__fs::filesystem::path&&, fs::path&, wallet::DatabaseOptions const&) /home/ubuntu/ci_scratch/ci/scratch/msan/cxx_build/include/c++/v1/__memory/unique_ptr.h:686:30
      #8 0x56060e8b5240 in wallet::MakeSQLiteDatabase(fs::path const&, wallet::DatabaseOptions const&, wallet::DatabaseStatus&, bilingual_str&) src/wallet/sqlite.cpp:641:19
      #9 0x56060e83560b in wallet::MakeDatabase(fs::path const&, wallet::DatabaseOptions const&, wallet::DatabaseStatus&, bilingual_str&) src/wallet/walletdb.cpp:1261:16
      #10 0x56060e7546e9 in wallet::MakeWalletDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, wallet::DatabaseOptions const&, wallet::DatabaseStatus&, bilingual_str&) src/wallet/wallet.cpp:2905:12
      #11 0x56060e4bc03f in wallet::TestLoadWallet(wallet::WalletContext&) src/wallet/test/util.cpp:68:21
      #12 0x56060e349ad4 in wallet::wallet_tests::ZapSelectTx::test_method() src/wallet/test/wallet_tests.cpp:897:19
      #13 0x56060e348598 in wallet::wallet_tests::ZapSelectTx_invoker() src/wallet/test/wallet_tests.cpp:891:1
      #14 0x56060cfec325 in boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:117:11
      #15 0x56060ced3a7e in boost::function0<void>::operator()() const /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:763:14
      #16 0x56060ced3a7e in boost::detail::forward::operator()() /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/execution_monitor.ipp:1388:32
      #17 0x56060ced3a7e in boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:137:18
      #18 0x56060cda71c2 in boost::function0<int>::operator()() const /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:763:14
      #19 0x56060cda71c2 in int boost::detail::do_invoke<boost::shared_ptr<boost::detail::translator_holder_base>, boost::function<int ()>>(boost::shared_ptr<boost::detail::translator_holder_base> const&, boost::function<int ()> const&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/execution_monitor.ipp:301:30
      #20 0x56060cda71c2 in boost::execution_monitor::catch_signals(boost::function<int ()> const&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/execution_monitor.ipp:903:16
      #21 0x56060cda784a in boost::execution_monitor::execute(boost::function<int ()> const&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/execution_monitor.ipp:1301:16
      #22 0x56060cd9ec3a in boost::execution_monitor::vexecute(boost::function<void ()> const&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/execution_monitor.ipp:1397:5
      #23 0x56060cd9ec3a in boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned long) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/unit_test_monitor.ipp:49:9
      #24 0x56060ce1a07b in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/framework.ipp:815:44
      #25 0x56060ce1ad8b in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/framework.ipp:784:58
      #26 0x56060ce1ad8b in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/framework.ipp:784:58
      #27 0x56060cd9b8de in boost::unit_test::framework::run(unsigned long, bool) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/framework.ipp:1722:29
      #28 0x56060cdd4fac in boost::unit_test::unit_test_main(boost::unit_test::test_suite* (*)(int, char**), int, char**) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/unit_test_main.ipp:250:9
      #29 0x56060cdd6094 in main /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/unit_test_main.ipp:306:12
      #30 0x7f7379691d8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
      #31 0x7f7379691e3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
      #32 0x56060cce2e24 in _start (/home/ubuntu/ci_scratch/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/test_bitcoin+0x188e24)

    Uninitialized value was created by a heap allocation
      #0 0x56060cd163f2 in malloc /ci_base_install/ci/scratch/msan/llvm-project/compiler-rt/lib/msan/msan_interceptors.cpp:934:3
      #1 0x56060fc10069 in sqlite3MemMalloc /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:25163:7
      #2 0x56060fb063bc in mallocWithAlarm /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:28846:7
      #3 0x56060fae4eb9 in sqlite3Malloc /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:28876:5
      #4 0x56060faf9e19 in sqlite3DbMallocRaw /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:29176:7
      #5 0x56060fb0fc67 in sqlite3PagerOpen /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:57938:17
      #6 0x56060fb0f48b in sqlite3BtreeOpen /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:68679:10
      #7 0x56060fb01384 in openDatabase /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:171911:8
      #8 0x56060fb016ca in sqlite3_open_v2 /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:172034:10
      #9 0x56060e8a94db in wallet::SQLiteDatabase::Open() src/wallet/sqlite.cpp:250:19
      #10 0x56060e8a30fd in wallet::SQLiteDatabase::SQLiteDatabase(fs::path const&, fs::path const&, wallet::DatabaseOptions const&, bool) src/wallet/sqlite.cpp:133:9
      #11 0x56060e8b78f5 in std::__1::__unique_if<wallet::SQLiteDatabase>::__unique_single std::__1::make_unique[abi:v160006]<wallet::SQLiteDatabase, std::__1::__fs::filesystem::path, fs::path&, wallet::DatabaseOptions const&>(std::__1::__fs::filesystem::path&&, fs::path&, wallet::DatabaseOptions const&) /home/ubuntu/ci_scratch/ci/scratch/msan/cxx_build/include/c++/v1/__memory/unique_ptr.h:686:30
      #12 0x56060e8b5240 in wallet::MakeSQLiteDatabase(fs::path const&, wallet::DatabaseOptions const&, wallet::DatabaseStatus&, bilingual_str&) src/wallet/sqlite.cpp:641:19
      #13 0x56060e83560b in wallet::MakeDatabase(fs::path const&, wallet::DatabaseOptions const&, wallet::DatabaseStatus&, bilingual_str&) src/wallet/walletdb.cpp:1261:16
      #14 0x56060e7546e9 in wallet::MakeWalletDatabase(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>> const&, wallet::DatabaseOptions const&, wallet::DatabaseStatus&, bilingual_str&) src/wallet/wallet.cpp:2905:12
      #15 0x56060e4bc03f in wallet::TestLoadWallet(wallet::WalletContext&) src/wallet/test/util.cpp:68:21
      #16 0x56060e349ad4 in wallet::wallet_tests::ZapSelectTx::test_method() src/wallet/test/wallet_tests.cpp:897:19
      #17 0x56060e348598 in wallet::wallet_tests::ZapSelectTx_invoker() src/wallet/test/wallet_tests.cpp:891:1
      #18 0x56060cfec325 in boost::detail::function::void_function_invoker0<void (*)(), void>::invoke(boost::detail::function::function_buffer&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:117:11
      #19 0x56060ced3a7e in boost::function0<void>::operator()() const /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:763:14
      #20 0x56060ced3a7e in boost::detail::forward::operator()() /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/test/impl/execution_monitor.ipp:1388:32
      #21 0x56060ced3a7e in boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /home/ubuntu/ci_scratch/depends/x86_64-pc-linux-gnu/include/boost/function/function_template.hpp:137:18

  SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/ubuntu/ci_scratch/depends/work/build/x86_64-pc-linux-gnu/sqlite/3380500-f816a3e2d52/sqlite3.c:32670:28 in sqlite3Strlen30
  ```

  as opposed to unsymbolized: https://cirrus-ci.com/task/6005512018329600?logs=ci#L3245.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 682274aab0

Tree-SHA512: 8f3e7636761c956537a472989bf07529f5afbd988c5e7e1f07ece8b2599608fa4fe9e1efdc6e302cf0f7f44dec3cf9a3c1e68b758af81a8a8b476a43d3220807
2023-06-21 09:41:05 +01:00
MarcoFalke
aaaa3aefbd test: Use TestNode *_path properties where possible
Seems odd to place the burden on test writers to hardcode the chain or
datadir path for the nodes under test.
2023-06-21 08:49:18 +02:00
MarcoFalke
dddd89962b test: Allow pathlib.Path as RPC argument via authproxy
Also, add datadir_path property to TestNode
2023-06-21 08:48:52 +02:00
MarcoFalke
fa41614a0a scripted-diff: Use wallets_path and chain_path where possible
Instead of passing the datadir and chain name to os.path.join, just use
the existing properties, which are the same.

-BEGIN VERIFY SCRIPT-
 sed -i --regexp-extended 's|\.datadir, self\.chain, .wallets.|.wallets_path|g' $(git grep -l '\.datadir, self\.chain,')
 sed -i --regexp-extended 's|\.datadir, self\.chain,|.chain_path,|g'            $(git grep -l '\.datadir, self\.chain,')
-END VERIFY SCRIPT-
2023-06-21 08:48:34 +02:00
MarcoFalke
fa493fadfb test: Use wallet_dir lambda in wallet_multiwallet test where possible
Seems odd to hardcode all parent directory names in the path for no good
reason.

Also, add wallet_path property to TestNode.

Also, rework wallet_backup.py test for scripted-diff in the next commit.
2023-06-21 08:47:54 +02:00
MarcoFalke
fa31c4daac fuzz: Avoid OOM in transaction fuzz target
Also fix bug where the json object is reused between two calls.
2023-06-21 07:51:29 +02:00
Hernan Marino
4da243ba02 qt: show own outputs on PSBT signing window 2023-06-21 02:48:55 -03:00
glozow
d1ae96755a Merge bitcoin/bitcoin#27890: refactor: Make m_count_with_* in CTxMemPoolEntry int64_t, drop UBSAN supp
fa76f0d0ef refactor: Make m_count_with_* in CTxMemPoolEntry int64_t, drop UBSAN supp (MarcoFalke)

Pull request description:

  This is a refactor as long as no signed integer overflow appears. In normal operation and absent bugs, signed integer overflow should never happen in the touched code paths.

  The main benefit of this refactor is to drop the file-wide ubsan suppression `unsigned-integer-overflow:txmempool.cpp`.

  For now, this only changes the internal private representation and the publicly returned type remains `uint64_t`.

ACKs for top commit:
  glozow:
    ACK fa76f0d0ef
  ryanofsky:
    Code review ACK fa76f0d0ef

Tree-SHA512: a09e33a915d60c65d369d44ba1a45ce4a6a76e6dc2bea43216ba02b5eab0b74e214b2c7cc44360493f2c483d18d96e4636b7a75b23050976efc80e38de852c39
2023-06-20 21:38:28 +01:00
Ryan Ofsky
ee22ca59a2 Merge bitcoin/bitcoin#26740: wallet: Migrate wallets that are not in a wallet dir
a1e653828b test: Add test for migrating default wallet and plain file wallet (Andrew Chow)
bdbe3fd76b wallet: Generated migrated wallet's path from walletdir and name (Andrew Chow)

Pull request description:

  This PR fixes an assertion error that is hit during the setup of the new database during migration of a wallet that was not contained in a wallet dir. Also added a test for this case as well as one for migrating the default wallet.

ACKs for top commit:
  ryanofsky:
    Code review ACK a1e653828b
  furszy:
    ACK a1e65382

Tree-SHA512: 96b218c0de8567d8650ec96e1bf58b0f8ca4c4726f5efc6362453979b56b9d569baea0bb09befb3a5aed8d16d29bf75ed5cd8ffc432bbd4cbcad3ac5574bc479
2023-06-20 16:10:44 -04:00
Cory Fields
cbee1d7091 depends: modernize clang flags
Fixes builds with llvm >= 11 in guix by working around the problem. As a bonus,
this is much cleaner and more maintainable than what we had before.
2023-06-20 19:55:02 +00:00
Cory Fields
2a85857ce5 ci: disable false-positive warnings for now
clang <=17 warns on -nostdlibinc, which causes an error on our -Werror builds.

Note that this breaks the "-fPIE" check in configure because it relies on
catching warnings, but that is not a problem for macOS.
2023-06-20 19:55:02 +00:00
Andrew Chow
e4bbfb2d49 Merge bitcoin/bitcoin#27632: Raise on invalid -debug and -loglevel config options
daa5a658c0 refactor: rename BCLog::BLOCKSTORE to BLOCKSTORAGE (Jon Atack)
cf622b214b doc: release note re raising on invalid -debug/debugexclude/loglevel (Jon Atack)
6cb1c66041 init: remove config option names from translated -loglevel strings (Jon Atack)
2547829272 test: -loglevel raises on invalid values (Jon Atack)
a9c295888b init: raise on invalid loglevel config option (Jon Atack)
b0c3995393 test: -debug and -debugexclude raise on invalid values (Jon Atack)
4c3c19d943 init: raise on invalid debug/debugexclude config options (Jon Atack)

Pull request description:

  and rename BCLog::BLOCKSTORE to BLOCKSTORAGE so the enum is the same as its value like the other BCLog enums.

  Per discussion in bitcoin-core-dev IRC today from https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2023-05-11#921458.

ACKs for top commit:
  achow101:
    ACK daa5a658c0
  ryanofsky:
    Code review ACK daa5a658c0. Just translated string template cleanup since last review
  pinheadmz:
    re-ACK daa5a658c0

Tree-SHA512: 4c107a93d8e8ce4e2ee81d44aec672526ca354ec390b241221067f68204beac8b4ba7a65748bcfa124ff2245c4307fa9243ec4fe0b464d0fa69c787fb322c3cc
2023-06-20 13:55:18 -04:00
Andrew Chow
688c61303b Merge bitcoin/bitcoin#27907: bench: bugfix, disable birth time block skip for wallet_create_tx.cpp
a72af2e833 bench: disable birth time block skip for wallet_create_tx.cpp (furszy)

Pull request description:

  As the  benchmarks inside `wallet_create_tx.cpp` assert the wallet
  balance at the end, they require all blocks to be scanned by the wallet.
  So, we need to ensure that no blocks are skipped by the recently added
  wallet birth time functionality.

  This just means setting the wallet birth time to the genesis block time.
  So the wallet is always older than any new block.

ACKs for top commit:
  achow101:
    ACK a72af2e833
  hernanmarino:
    ACK a72af2e833
  TheCharlatan:
    ACK a72af2e833

Tree-SHA512: d3148659bd633d20978736e1292e3456a2c6dd2b6c8f60625a4160e16818d923487c889237eb3f34693f7dd78b7d124b89afdc56e4c9fad370026d0733ef1e08
2023-06-20 13:40:56 -04:00
MarcoFalke
0000f55293 ci: Run fuzz target even if input folder is empty 2023-06-20 18:19:01 +02:00
fanquake
682274aab0 ci: install llvm-symbolizer in MSAN jobs 2023-06-20 17:16:22 +01:00
fanquake
96527cd51e ci: use LLVM 16.0.6 in MSAN jobs 2023-06-20 17:14:06 +01:00
fanquake
c2316b1e34 Merge bitcoin/bitcoin#27917: fuzz: Fix implicit-integer-sign-change in wallet/fees fuzz target
faa05d1965 fuzz: Fix implicit-integer-sign-change in wallet/fees fuzz target (MarcoFalke)

Pull request description:

  This fixes a bug in the fuzz target.

  ```
  echo 'OiAAAPr//wAAAAAAAAA=' | base64  --decode > /tmp/a
  UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1" FUZZ=wallet_fees ./src/test/fuzz/fuzz  /tmp/a
  ```

  ```
  wallet/fees.cpp:58:58: runtime error: implicit conversion from type 'unsigned int' of value 4294574080 (32-bit, unsigned) to type 'int' changed the value to -393216 (32-bit, signed)
      #0 0x5625ef46a094 in wallet::GetMinimumFeeRate(wallet::CWallet const&, wallet::CCoinControl const&, FeeCalculation*) src/wallet/fees.cpp:58:58
      #1 0x5625eedd467f in wallet::(anonymous namespace)::wallet_fees_fuzz_target(Span<unsigned char const>) src/wallet/test/fuzz/fees.cpp:64:11
  ...

  SUMMARY: UndefinedBehaviorSanitizer: implicit-integer-sign-change wallet/fees.cpp:58:58 in

ACKs for top commit:
  dergoegge:
    tACK faa05d1965
  brunoerg:
    ACK faa05d1965

Tree-SHA512: 66a4020d6a4153a92c7023e9f94ec6279862566db7236ce3cf6951b7fbee616dc88a56fe9502de4099d74f9840439b20a984b0733fb432e43129e774bcc2a6e6
2023-06-20 16:50:32 +01:00
glozow
f80db62b2d Merge bitcoin/bitcoin#27622: Fee estimation: avoid serving stale fee estimate
d2b39e09bc test: ensure old fee_estimate.dat not read on restart and flushed (ismaelsadeeq)
cf219f29f3 tx fees, policy: read stale fee estimates with a regtest-only option (ismaelsadeeq)
3eb241a141 tx fees, policy: do not read estimates of old fee_estimates.dat (ismaelsadeeq)
5b886f2b43 tx fees, policy: periodically flush fee estimates to fee_estimates.dat (ismaelsadeeq)

Pull request description:

  Fixes #27555

  The issue arises when an old `fee_estimates.dat` file is sometimes read during initialization.
  Or after an unclean shutdown, the latest fee estimates are not flushed to `fee_estimates.dat`.
  If the fee estimates in the old file are old, they can cause transactions to become stuck in the mempool.
  This  PR ensures that nodes do not use stale estimates from the old file during initialization. If  `fee_estimates.dat`
  has not been updated for 60 hours or more, it is considered stale and will not be read during initialization. To avoid
  having old estimates, the `fee_estimates.dat` file will be flushed periodically every hour. As mentioned #27555

  > "The immediate improvement would be to store fee estimates to disk once an hour or so to reduce the chance of having an old file. From there, this case could probably be detected, and refuse to serve estimates until we sync."

  In addition, I will follow-up PR to persist the `mempoolminfee` across restarts.

ACKs for top commit:
  willcl-ark:
    ACK d2b39e09bc
  instagibbs:
    reACK d2b39e09bc
  glozow:
    ACK d2b39e09bc. One nit if you follow up.

Tree-SHA512: 4f6e0c296995d0eea5cf80c6aefdd79b7295a6a0ba446f2166f32afc105fe4f831cfda1ad3abd13c5c752b4fbea982cf4b97eaeda2af1fd7184670d41edcfeec
2023-06-20 16:48:29 +01:00
MarcoFalke
faa05d1965 fuzz: Fix implicit-integer-sign-change in wallet/fees fuzz target 2023-06-20 12:05:09 +02:00
Brotcrunsher
bdea2bb114 scripted-diff: Following the C++ Standard rules for identifiers with _.
Any identifier starting with two _, or one _ followed by a capital letter is reserved for the compiler and thus must not be used. See: https://stackoverflow.com/a/228797/7130273

-BEGIN VERIFY SCRIPT-
s() { git grep -l "$1" src | xargs sed -i "s/$1/$2/g"; }

s '__pushKV' 'pushKVEnd'
s '_EraseTx' 'EraseTxNoLock'
s '_Other' 'Other'
-END VERIFY SCRIPT-
2023-06-20 10:23:08 +02:00
brunoerg
025fda0a76 fuzz: addrman, avoid ConsumeDeserializable when possible
`ConsumeDeserializable` may return `std::nullopt`, prefer
to call specific functions such as `ConsumeService`and
`ConsumeNetAddr` which always return a value.
2023-06-19 18:21:43 -03:00
Andrew Chow
9e077d9b42 salvage: Remove use of ReadKeyValue in salvage
To prepare to remove ReadKeyValue, change salvage to not use it
2023-06-19 16:46:39 -04:00
Andrew Chow
11d650060a feerate: For GetFeePerK() return nSatoshisPerK instead of round trip through GetFee
Returning the sats/kvb does not need to round trip through
GetFee(1000) since the feerate is already stored as sats/kvb.
2023-06-19 14:36:14 -04:00
Andrew Chow
8f40271037 Merge bitcoin/bitcoin#27902: fuzz: wallet, add target for CoinControl
40b333e21f fuzz: wallet, add target for CoinControl (Ayush Singh)

Pull request description:

  This PR adds fuzz coverage for `wallet/coincontrol`.

  Motivation: Issue [#27272](https://github.com/bitcoin/bitcoin/issues/27272#issue-1628327906)

  The idea is to create different/unique instances of `COutPoint` by placing it inside the `CallOneOf` function, which may or may not be consumed by all of the `CoinControl` file's methods.

  This is my first PR on Bitcoin Core, and I will try my best to address any reviews/changes ASAP. I'm also working on fuzz harness files for other files in the wallet and plan to open PR for them soon.

ACKs for top commit:
  kevkevinpal:
    reACK [40b333e](40b333e21f)
  MarcoFalke:
    lgtm ACK 40b333e21f
  achow101:
    ACK 40b333e21f
  brunoerg:
    crACK 40b333e21f
  dergoegge:
    ACK 40b333e21f

Tree-SHA512: 174769f4e86df8590b532b85480fd620082587e84e50e49ca9b52f0588a219355362cefd66250dd9942e86019d27af4ca599b45e871e9f147d2cc0ba97c4aa7b
2023-06-19 13:07:37 -04:00
Sebastian Falbesoner
1a572ce7d6 test: refactor: introduce generate_keypair helper with WIF support
In functional tests it is a quite common scenario to generate fresh
elliptic curve keypairs, which is currently a bit cumbersome as it
involves multiple steps, e.g.:

    privkey = ECKey()
    privkey.generate()
    privkey_wif = bytes_to_wif(privkey.get_bytes())
    pubkey = privkey.get_pubkey().get_bytes()

Simplify this by providing a new `generate_keypair` helper function that
returns the private key either as `ECKey` object or as WIF-string
(depending on the boolean `wif` parameter) and the public key as
byte-string; these formats are what we mostly need (currently we don't
use `ECPubKey` objects from generated keypairs anywhere).

With this, most of the affected code blocks following the pattern above
can be replaced by one-liners, e.g.:

    privkey, pubkey = generate_keypair(wif=True)

Note that after this commit, the only direct uses of `ECKey` remain in
situations where we want to set the private key explicitly, e.g. in
MiniWallet (test/functional/test_framework/wallet.py) or the test for
the signet miner script (test/functional/tool_signet_miner.py).
2023-06-19 17:38:14 +02:00
Andrew Chow
ad779e9ece walletdb: Refactor hd chain loading to its own function 2023-06-19 11:38:01 -04:00
Andrew Chow
72c2a54ebb walletdb: Refactor encryption key loading to its own function 2023-06-19 11:36:52 -04:00
Andrew Chow
3ccde4599b walletdb: Refactor crypted key loading to its own function 2023-06-19 11:35:15 -04:00
Andrew Chow
7be10adff3 walletdb: Refactor key reading and loading to its own function 2023-06-19 11:29:14 -04:00
virtu
61f4b9b7ad Manage exceptions in bcc callback functions
Exceptions are not propagated in ctype callback functions used by bcc.
This means an AssertionError exception raised by check_equal() to signal
a failed assertion is not getting caught and properly logged. Instead,
the error is logged to stdout and execution of the handler stops.

The current workaround to check whether all check_equal() assertions in
a callback succeeded is to increment a success counter after the
assertions (which only gets incremented if none exception is raised and
stops execution). Then, outside the callback, the success counter can be
used to check whether a callback executed successfully.

One issue with the described workaround is that when an exception
occurs, there is no way of telling which of the check_equal() statements
caused the exception; moreover, there is no way of inspecting how the
pieces of data that got compared in check_equal() differed (often
a crucial clue when debugging what went wrong).

Two fixes to this problem come to mind. The first involves having the
callback function make event data accessible outside the callback and
inspecting the event using check_equal() outside the callback. This
solution still requires a counter in the callback to tell whether
a callback was actually executed or if instead the call to
perf_buffer_poll() timed out.

The second fix entails wrapping all relevant check_equal() statements
inside callback functions into try-catch blocks and manually logging
AssertionErrors. While not as elegant in terms of design, this approach
can be more pragmatic for more complex tests (e.g., ones involving
multiple events, events of different types, or the order of events).

The solution proposed here is to select the most pragmatic fix on
a case-by-case basis: Tests in interface_usdt_net.py,
interface_usdt_mempool.py and interface_usdt_validation.py have been
refactored to use the first approach, while the second approach was
chosen for interface_usdt_utxocache.py (partly to provide a reference
for the second approach, but mainly because the utxocache tests are the
most intricate tests, and refactoring them to use the first approach
would negatively impact their readability). Lastly,
interface_usdt_coinselection.py was kept unchanged because it does not
use check_equal() statements inside callback functions.
2023-06-19 14:38:32 +02:00
fanquake
7f0b79ea13 Merge bitcoin/bitcoin#27906: doc: test: update TestShell instructions
14405e8d4d doc: test: update TestShell instructions (ismaelsadeeq)

Pull request description:

  Fixes  #27904

  From  #27904 and IRC.
  Update [Testshell instructions ](https://github.com/bitcoin/bitcoin/blob/master/test/functional/test-shell.md#2-importing-testshell-from-the-bitcoin-core-repository)

  E.g `TestShell.setup()` throws
  ```
  AttributeError: type object 'TestShell' has no attribute 'setup'
  ```
  Parentheses are missing, it should be `TestShell().setup()`

ACKs for top commit:
  Sjors:
    utACK 14405e8d4d
  brunoerg:
    crACK 14405e8d4d
  hernanmarino:
    utACK 14405e8d4d

Tree-SHA512: ffe5fa1103a3b00ef0ee99879adae967b0da07cb8f8451c4c261b0a70b3b666af7aeaacd6f46f85a84ee5e9c7c7ed49700209b5b1f124d7a76efc420ad5c9cd9
2023-06-18 12:48:46 +02:00
Ayush Singh
40b333e21f fuzz: wallet, add target for CoinControl 2023-06-17 23:55:16 +05:30
furszy
a72af2e833 bench: disable birth time block skip for wallet_create_tx.cpp
As the benchmarks inside wallet_create_tx.cpp assert the
wallet balance at the end, they require all
blocks to be scanned by the wallet. So, we need
to ensure that no blocks are skipped by the recently
added wallet birth time functionality.

This just means setting the wallet birthtime to the
genesis block time. So the wallet is always older than
any new block.
2023-06-16 21:00:20 -03:00
ismaelsadeeq
14405e8d4d doc: test: update TestShell instructions
add missing parentheses `TestShell.method` should be `TestShell().method`.
2023-06-16 22:55:36 +01:00
Martin Zumsande
e639364495 validation: add missing insert to m_dirty_blockindex
...in FindMostWorkChain(). Before this, it was possible that the change
to the block index wouldn't be persisted to disk.
2023-06-16 17:23:03 -04:00
Andrew Chow
f0758d8a66 Merge bitcoin/bitcoin#27757: rpc: remove deprecated "warning" field from {create,load,restore,unload}wallet
5524fa00fa doc: add release note about removal of `deprecatedrpc=walletwarningfield` flag (Sebastian Falbesoner)
5c77db7354 Restorewallet/createwallet help documentation fixups/improvements (Jon Atack)
a00ae31fcc rpc: remove deprecated "warning" field from {create,load,restore,unload}wallet (Sebastian Falbesoner)

Pull request description:

  The "warning" string field for wallet creating/loading RPCs (`createwallet`, `loadwallet`, `unloadwallet` and `restorewallet`) has been deprecated with the configuration option `-deprecatedrpc=walletwarningfield` in PR #27279 (released in v25.0). For the next release v26.0, the field and the configuration option can be removed.

ACKs for top commit:
  achow101:
    ACK 5524fa00fa
  jonatack:
    ACK 5524fa00fa

Tree-SHA512: 8212f72067d08095304018b8a95d2ebef630004b65123483fbbfb078cc5709c2d825bbc35b16ea5f6b28ae7377347382d7e9afaf7bdbf0575d2c229d970784de
2023-06-16 15:11:44 -04:00
fanquake
1ecdf6ea8f Merge bitcoin/bitcoin#27875: build: make sure we can overwrite config.{guess,sub} before doing so
fc6c17b838 build: make sure we can overwrite config.{guess,sub} (0xb10c)

Pull request description:

  Since ea7b8528 (#26422), `autogen.sh` overwrites the `build-aux/config.{guess, sub}` files (installed there by `autoreconf`) with the `depends/config.{guess, sub}` files if these are newer.

  The `autoreconf` tool copies them from it's `share/autoconf/build-aux/` directory. Specifically on NixOS, the `share/autoconf/build-aux/` files are located in the nix-store and are read-only. `autoreconf` preserves the read-only permissions when copying. Overwriting them with our `depends/config.{guess, sub}` files subsequently fails.

  To make sure we can overwrite the files, set write permissions to the current user and group before overwriting. This fixes the problem on NixOS.

  fixes #27873

ACKs for top commit:
  dergoegge:
    tACK fc6c17b838
  fanquake:
    ACK fc6c17b838

Tree-SHA512: e8a31f739d5b598b2fe9fe6fc3d02303c117a6adccc49b8d0fea4980027a64f915a0e1e00e4788dce6113ef1b9ec9acf9e4164486f6e4904bad405f20b6746a0
2023-06-16 11:03:07 +01:00
fanquake
32e2ffc393 Remove the syscall sandbox
After initially being merged in #20487, it's no-longer clear that an
internal syscall sandboxing mechanism is something that Bitcoin Core
should have/maintain, especially when compared to better
maintained/supported alterantives, i.e firejail.

Note that given where it's used, the sandbox also gets dragged into the
kernel.

There is some related discussion in #24771.

This should not require any sort of deprecation, as this was only ever
an opt-in, experimental feature.

Closes #24771.
2023-06-16 10:38:19 +01:00
Ryan Ofsky
1c7d08b9ac validation: Stricter assumeutxo error handling in InvalidateCoinsDBOnDisk
Currently InvalidateCoinsDBOnDisk is calling AbortNode without an error to the
caller if it fails. Change it to return just return util::Result, and update
the caller to handle the error itself.

This causes the secondary error to be shown below the main error instead of the
other way around.
2023-06-15 15:11:32 -04:00
Andrew Chow
b3db18a012 Merge bitcoin/bitcoin#27712: test: p2p: check misbehavior for non-continuous headers messages
a97c59f12d test: p2p: check misbehavior for non-continuous headers messages (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for a peer sending a `headers` message where the headers don't connect to each other, which should be treated as misbehaving (not disconnecting though, as the score increase is only 20). The relevant code path is `PeerManagerImpl::ProcessHeadersMessage` -> `PeerManagerImpl::CheckHeadersPoW` -> `PeerManagerImpl::CheckHeadersAreContinuous`:

  17acb2782a/src/net_processing.cpp (L2415-L2419)

  17acb2782a/src/net_processing.cpp (L2474-L2484)

ACKs for top commit:
  sr-gi:
    ACK a97c59f12d
  achow101:
    ACK a97c59f12d
  instagibbs:
    ACK a97c59f12d

Tree-SHA512: 3f8d6a2492e5c8b63c7b11be2e4ec455f83581b2c58f2d4e705baadfe8d7c6377296d6cd0eda679d291a13d8930b09443f8e3d183795df34b780c703d5d3aeb3
2023-06-15 15:11:32 -04:00
Ryan Ofsky
9047337d36 validation: Stricter assumeutxo error handling in LoadChainstate
Make LoadChainstate return an explicit error when snapshot validation succeeds,
but there is an error trying to replace the background chainstate with the
snapshot chainstate. Previously in this case LoadChainstate would trigger a
shutdown and return INTERRUPTED, now it will return an actual error code.

There's no real change to behavior other than error message being formatted a
little differently.

Motivation for this change is to replace error handling via callbacks with
error handling via return value ahead of
https://github.com/bitcoin/bitcoin/pull/27861
2023-06-15 15:11:32 -04:00
Andrew Chow
5b8e07725d Merge bitcoin/bitcoin#27892: refactor: Avoid copy of bilingual_str when formatting, Fix ADL violation
fa8ef7d138 refactor: Avoid copy of bilingual_str when formatting, Fix ADL violation (MarcoFalke)

Pull request description:

  This refactor shouldn't change behavior, but may fix compile errors such as https://github.com/bitcoin/bitcoin/pull/27862#issuecomment-1592516184

ACKs for top commit:
  achow101:
    ACK fa8ef7d138
  ryanofsky:
    Code review ACK fa8ef7d138. Looks great! Thanks for updating
  hebasto:
    ACK fa8ef7d138, I have reviewed the code and it looks OK.

Tree-SHA512: 903019962f27b5432b8e3af052b472238ef68d3ee165148c9d2232bf290309075f9f17d8d06c9b5c7fddb89c1a9c3a4c09c6310af01e8561adc0244a30db0857
2023-06-15 14:29:55 -04:00
Jon Atack
daa5a658c0 refactor: rename BCLog::BLOCKSTORE to BLOCKSTORAGE
so the enum name is the same as its value, like the other BCLog enums.
2023-06-15 10:27:56 -06:00
Jon Atack
cf622b214b doc: release note re raising on invalid -debug/debugexclude/loglevel 2023-06-15 10:27:56 -06:00
Jon Atack
6cb1c66041 init: remove config option names from translated -loglevel strings 2023-06-15 10:27:56 -06:00
brunoerg
77d6d89d43 net: net_processing, add ProcessCompactBlockTxns
When processing `CMPCTBLOCK` message, at some moments
we can need to process cmpct block txns, since all messages
are handled by ProcessMessage, we call ProcessMessage
all over again. For this reason, it creates a function called
`ProcessCompactBlockTxns` to process it.
2023-06-15 12:08:10 -03:00
fanquake
c454395115 Merge bitcoin/bitcoin#27895: test: clean up is node stopped
6779e6ed7f test: clean up is node stopped (dimitaracev)

Pull request description:

  Fixes #27893

  Use f'strings for the message when asserting `expected_ret_code` and `return_code`. Change the `expected_ret_code` from an optional to have a default value of `0`.

  cc MarcoFalke

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 6779e6ed7f
  stickies-v:
    ACK 6779e6ed7f
  brunoerg:
    ACK 6779e6ed7f

Tree-SHA512: af84e7ffe467ced29236dee9206687786a2efb89ab8b039c3ebfb93ea23fc273206cd51f20c9fb6bee4135770e9a649538942571d9c0be83ba9535fa8e59cb28
2023-06-15 15:39:51 +01:00
MarcoFalke
fa8ef7d138 refactor: Avoid copy of bilingual_str when formatting, Fix ADL violation
The return type of TranslateArg is std::string, which creates a copy.
Fix this by moving everything into a lambda that takes a reference and
returns a reference.

Also, the format function is called without specifying the namespace it
lives in. Fix this by specifying the namespace. See also:
7a59865793/doc/developer-notes.md (L117-L137).
2023-06-15 16:21:29 +02:00
Andrew Chow
9372ec71e8 Merge bitcoin/bitcoin#27872: build: suppress external warnings by default
3b2acfcfec build: suppress external warnings by default (fanquake)

Pull request description:

  I think we are at the point where it make more sense to make this the default, than not. It's already used in the CI, and I assume most building locally are also utilising it.

ACKs for top commit:
  achow101:
    ACK 3b2acfcfec
  hebasto:
    ACK 3b2acfcfec
  stickies-v:
    ACK 3b2acfcfec

Tree-SHA512: be20203381c03dea8b5d64876c56bf8bb8defdfd6fc6d5398b71d3f28d0209c4bd1374f108df708aaa8867fda818a9bc611d1908c9fbb74f8cccdfbc5aff05af
2023-06-15 09:54:23 -04:00
fanquake
3b2acfcfec build: suppress external warnings by default 2023-06-15 14:12:10 +01:00
dimitaracev
6779e6ed7f test: clean up is node stopped 2023-06-15 14:14:22 +02:00
fanquake
7a59865793 Merge bitcoin/bitcoin#27647: fuzz: wallet, add target for fees
162602b208 fuzz: wallet, add target for `fees` (brunoerg)

Pull request description:

  This PR adds fuzz coverage for `wallet/fees`. Some functions may use or not (non default) values from `wallet`, `CCoinControl` or `FeeCalculation`. So the logic is to make the test sometimes fill up some attributes and others no.

  Obs: As soon as this PR gets some reviews, I can open the proper PR to `qa-assets` as well.

ACKs for top commit:
  Xekyo:
    ACK 162602b208
  MarcoFalke:
    lgtm ACK 162602b208
  dergoegge:
    Code review ACK 162602b208

Tree-SHA512: 6545802f27aafb60bf5a119af514e9425b643780dea6650bba766bb5be813f2aaddb7afc7f0efa2943ceb26f5ea08b42c95a3c0df897493c71f2d2f99e9e4236
2023-06-15 11:44:02 +01:00
ismaelsadeeq
d2b39e09bc test: ensure old fee_estimate.dat not read on restart and flushed
This commit adds tests to ensure that old fee_estimates.dat files
are not read and that fee_estimates are periodically flushed to the
fee_estimates.dat file.

Additionaly it tests the -regtestonly option -acceptstalefeeestimates.
2023-06-14 22:40:20 +01:00
ismaelsadeeq
cf219f29f3 tx fees, policy: read stale fee estimates with a regtest-only option
If -acceptstalefeeestimates option is passed stale fee estimates can now
be read when operating in regtest environments.

Additionally, this commit updates all declarations of the CBlockPolicyEstimator
class to include a the second constructor variable.
2023-06-14 22:39:26 +01:00
ismaelsadeeq
3eb241a141 tx fees, policy: do not read estimates of old fee_estimates.dat
Old fee estimates could cause transactions to become stuck in the
mempool. This commit prevents the node from using stale estimates
from an old file.
2023-06-14 22:32:27 +01:00
MarcoFalke
fa76f0d0ef refactor: Make m_count_with_* in CTxMemPoolEntry int64_t, drop UBSAN supp
This is a refactor as long as no signed integer overflow appears. In
normal operation and absent bugs, signed integer overflow should never
happen in the touched code paths.

The main benefit of this refactor is to drop the file-wide ubsan
suppression unsigned-integer-overflow:txmempool.cpp.

For now, this only changes the internal private representation and the
publicly returned type remains uint64_t.
2023-06-14 23:15:20 +02:00
ismaelsadeeq
5b886f2b43 tx fees, policy: periodically flush fee estimates to fee_estimates.dat
This reduces chances of having old estimates in fee_estimates.dat.
2023-06-14 21:42:41 +01:00
Hennadii Stepanov
28fff06afe test: Make linter to look for BOOST_ASSERT macros
The `BOOST_ASSERT` macro requires to `#include boost/assert.hpp`.
2023-06-14 16:26:11 +01:00
Hennadii Stepanov
47fe551e52 test: Kill BOOST_ASSERT 2023-06-14 16:00:27 +01:00
Jon Atack
1cd45d4e08 test: move random.h include header from setup_common.h to cpp 2023-06-14 08:28:33 -06:00
jonatack
1b246fdd14 test: move remaining random test util code from setup_common to random
and drop the util/random dependency on util/setup_common.

This improves code separation and avoids creating a circular dependency if
setup_common needs to call the util/random functions.
2023-06-14 08:28:33 -06:00
Jon Atack
2547829272 test: -loglevel raises on invalid values 2023-06-14 08:27:46 -06:00
Jon Atack
a9c295888b init: raise on invalid loglevel config option 2023-06-14 08:27:46 -06:00
Jon Atack
b0c3995393 test: -debug and -debugexclude raise on invalid values 2023-06-14 08:27:46 -06:00
Jon Atack
4c3c19d943 init: raise on invalid debug/debugexclude config options 2023-06-14 08:27:46 -06:00
brunoerg
162602b208 fuzz: wallet, add target for fees 2023-06-14 11:20:39 -03:00
fanquake
681ecac5c2 Merge bitcoin/bitcoin#27881: ci: Use latest macos-ventura-xcode:14.3.1 image
a13c3f3177 ci: Use latest `macos-ventura-xcode:14.3.1` image (Hennadii Stepanov)

Pull request description:

  As documented: 427853ab49/.cirrus.yml (L339)

  Last time, the image was updated in https://github.com/bitcoin/bitcoin/pull/26388.

  The `check_clang` script diff:
  ```diff
  --- master
  +++ pr
  @@ -1,5 +1,5 @@
   clang --version
  -Apple clang version 14.0.0 (clang-1400.0.29.202)
  -Target: arm64-apple-darwin22.1.0
  +Apple clang version 14.0.3 (clang-1403.0.22.14.1)
  +Target: arm64-apple-darwin22.5.0
   Thread model: posix
  -InstalledDir: /Applications/Xcode-14.1.0.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
  +InstalledDir: /Applications/Xcode-14.3.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

ACKs for top commit:
  fanquake:
    ACK a13c3f3177

Tree-SHA512: 18ffa97cc5900a5b35d0ecda79a55d7427610c6e799742b5811e1c470ef320fd98f9400168c4c517f8668f1bf76e57d68cd373a41fa346d15084855aa2c409b3
2023-06-14 15:10:55 +01:00
fanquake
ff17b28b02 Merge bitcoin/bitcoin#27883: ci: Bump macOS cross task to ubuntu:jammy
fa70e85e00 ci: Bump macOS cross task to ubuntu:jammy (MarcoFalke)

Pull request description:

  It shouldn't matter what underlying image is used for the task, because the compiler is fully provided by `./depends/`.

  So just use the latest Ubuntu LTS, which is also most likely the OS that is used by people cross-compiling, if there are any at all.

ACKs for top commit:
  fanquake:
    ACK fa70e85e00
  hebasto:
    ACK fa70e85e00

Tree-SHA512: ab2831a8182ca382b8af37d5395c35b5341b8f55b0ce05f4787c627cbec306cefad66713ad053228862eeac01fb8b79be7e168c41e6ec4615fbcb4ef106125b8
2023-06-14 15:09:18 +01:00
fanquake
a8d0f6c863 Merge bitcoin/bitcoin#27886: ci: Switch to amd64 container in "ARM" task
016fe6d828 ci: Switch to `amd64` container in "ARM" task (Hennadii Stepanov)

Pull request description:

  The `arm_container` does not support 32-bit mode anymore.

  Fixes https://github.com/bitcoin/bitcoin/issues/27879.

  Also, the `arm_container` could be used for testing `aarch64` binaries, which are the part of our releases. Leaving that for another PR.

ACKs for top commit:
  MarcoFalke:
    review ACK 016fe6d828 if CI is green

Tree-SHA512: cdcc034938f4c101e211fceca0dcd7f50258f1085b74d6abe50bb0f45f1b92b99e2763436271583fdbad194cb9ed7e266d5597274555110ceaa7d762fe5a49d9
2023-06-14 14:36:03 +01:00
Ryan Ofsky
6663c802fe Merge bitcoin/bitcoin#25634: wallet, tests: Expand and test when the blank wallet flag should be un/set
cdba23db35 wallet: Document blank flag use in descriptor wallets (Ryan Ofsky)
43310200dc wallet: Ensure that the blank wallet flag is unset after imports (Andrew Chow)
e9379f1ffa rpc, wallet: Include information about blank flag (Andrew Chow)

Pull request description:

  The `blank` wallet flag is used to indicate that the wallet intentionally does not have any keys, scripts, or descriptors, and it prevents the automatic generation of those things for such a wallet. Once the wallet contains any of those data, it is unnecessary, and possibly incorrect, to have `blank` set. This PR fixes a few places where this was not properly happening. It also adds a test for this unset behavior.

ACKs for top commit:
  S3RK:
    reACK cdba23db35
  ryanofsky:
    Code review ACK cdba23db35. Only change since last review is dropping the commit which makes createwallet RPC set BLANK flag automatically when DISABLE_PRIVATE_KEYS flag is set

Tree-SHA512: 85bc2a9754df0531575d5c8f4ad7e8f38dcd50083dc29b3283dacf56feae842e81f34654c5e1781f2dadb0560ff80e454bbc8ca3b2d1fab1b236499ae9abd7da
2023-06-14 09:30:39 -04:00
Hennadii Stepanov
016fe6d828 ci: Switch to amd64 container in "ARM" task
Tee `arm_container` does not support 32-bit mode anymore.
See: https://github.com/bitcoin/bitcoin/issues/27879
2023-06-14 13:32:22 +01:00
MarcoFalke
fa70e85e00 ci: Bump macOS cross task to ubuntu:jammy 2023-06-14 10:49:27 +02:00
Andrew Chow
5b3a85b4c6 interfaces, wallet: Expose migrate wallet 2023-06-13 19:00:58 -04:00
Hennadii Stepanov
a13c3f3177 ci: Use latest macos-ventura-xcode:14.3.1 image 2023-06-13 23:00:33 +01:00
Andrew Chow
427853ab49 Merge bitcoin/bitcoin#27876: test: (refactor) Use datadir from options in chainstatemanager test
d54819d74e scripted-diff: Use datadir from options in chainstatemanager test (TheCharlatan)

Pull request description:

  This should make the test less reliant on argument state from the test setup. This is a follow-up PR as requested in https://github.com/bitcoin/bitcoin/pull/27576#discussion_r1224638890.

ACKs for top commit:
  achow101:
    ACK d54819d74e
  MarcoFalke:
    lgtm ACK d54819d74e
  kevkevinpal:
    ACK d54819d74e
  ryanofsky:
    Code review ACK d54819d74e

Tree-SHA512: 939fde2505c5585d993545a3d05d3a00caec40f860c74fa002caebdf4c1b70e774cfb028a8a8f780525f8968844157d2c568d9f2c8dd5ec32b093173d8644c34
2023-06-13 16:28:16 -04:00
Ryan Ofsky
cdba23db35 wallet: Document blank flag use in descriptor wallets 2023-06-13 15:11:41 -04:00
Andrew Chow
43310200dc wallet: Ensure that the blank wallet flag is unset after imports 2023-06-13 15:11:41 -04:00
fanquake
da494186f2 Merge bitcoin/bitcoin#27806: fuzz: Fix mini_miner_selection running out of coin
76c5ea703e fuzz: Fix mini_miner_selection running out of coin (Murch)

Pull request description:

  Fixes a bug in the mini_miner_selection fuzz test found by fuzzing: It was possible for the mini_miner_selection fuzz test to generated transactions that created fewer new outputs than the two inputs they each spent. If the fuzz seed did so consistently, eventually it would cause a `pop_front()` on an empty available_coins which resulted in undefined behavior.

  Fixed per belt-suspender approach:
  - assert that available_coins is not empty before generating tx
  - generate at least two coins per new tx
  - allow building tx with a single input if only one coin is available

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 76c5ea703e
  dergoegge:
    reACK 76c5ea703e

Tree-SHA512: 5b7ffd1905a712733ad5364958ad79874dd8c31bd50069b0d3e6f734da0f2d496cb08cbe0afa47115674313e1cb7166a6087f2ccbce289774caddc790583e241
2023-06-13 17:08:07 +01:00
Andrew Chow
58b36fc303 Merge bitcoin/bitcoin#23962: Use int32_t type for most transaction size/weight values
3ef756a5b5 Remove txmempool implicit-integer-sign-change sanitizer suppressions (Hennadii Stepanov)
d2f6d2a95a Use `int32_t` type for most transaction size/weight values (Hennadii Stepanov)

Pull request description:

  From bitcoin/bitcoin#23957 which has been incorporated into this PR:
  > A file-wide suppression is problematic because it will wave through future violations, potentially bugs.
  >
  > Fix that by using per-statement casts.
  >
  > This refactor doesn't change behavior because the now explicit casts were previously done implicitly.
  >
  > Similar to commit 8b5a4de904

ACKs for top commit:
  achow101:
    ACK 3ef756a5b5
  0xB10C:
    ACK 3ef756a5b5. I've focused my testing and code review on the tracepoint related changes. The docs, the test, and the mempool_monitor.py demo script are updated. I ran the `interface_usdt_mempool.py` test and the `mempool_monitor.py` script. The `mempool_monitor.py` output looks correct.
  Xekyo:
    codereview ACK 3ef756a5b5
  ryanofsky:
    Code review ACK 3ef756a5b5. Since last review, just rebased with more type changes in test and tracing code

Tree-SHA512: 397407f72165b6fb85ff1794eb1447836c4f903efed1a05d7a9704c88aa9b86f330063964370bbd59f6b5e322e04e7ea8e467805d58dce381e68f7596433330f
2023-06-13 10:37:25 -04:00
0xb10c
fc6c17b838 build: make sure we can overwrite config.{guess,sub}
Since ea7b8528 (#26422), autogen.sh overwrites the
build-aux/config.{guess, sub} files (installed there by autoreconf)
with the depends/config.{guess, sub} files if these are newer.

The autoreconf tool copies them from it's share/autoconf/build-aux/
directory. Specifically on NixOS, the share/autoconf/build-aux/
files are located in the nix-store and are read-only. autoreconf
preserves the read-only permissions when copying. Overwriting them
with our depends/config.{guess, sub} subsequently fails.

To make sure we can overwrite the files, we set write permissions to
the current user and group before overwriting. This fixes the problem
on NixOS.

fixes #27873: Can't copy to 'build-aux/config.guess' in autoconf.sh: Permission denied
2023-06-13 14:58:43 +02:00
TheCharlatan
d54819d74e scripted-diff: Use datadir from options in chainstatemanager test
This should make the test less reliant on details of the test setup

-BEGIN VERIFY SCRIPT-
sed -i 's/m_args.GetDataDirNet()/chainman.m_options.datadir/g' src/test/validation_chainstatemanager_tests.cpp
-END VERIFY SCRIPT-
2023-06-13 13:52:42 +02:00
fanquake
8de9bb7a5a Merge bitcoin/bitcoin#27864: test: fix intermittent failure in p2p_leak_tx.py
ee2417ed61 test: fix intermittent failure in p2p_leak_tx.py (Martin Zumsande)

Pull request description:

  Fixes #27860

  The problem was that the replacement tx `tx_b` would sometimes be sent out to the inbound peer after the `notfound`, so that threre  would be an unexpected `tx` message and the test fails.

  ```
   node0 2023-06-12T12:48:24.903204Z [msghand] [net.cpp:2856] [PushMessage] [net] sending notfound (73 bytes) peer=1
   node0 2023-06-12T12:48:24.903916Z [msghand] [net.cpp:2856] [PushMessage] [net] sending tx (133 bytes) peer=1
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_leak_tx.py", line 74, in test_notfound_on_replaced_tx
                                         assert "tx" not in inbound_peer.last_message

  ```

  Fix this by letting the peer wait for the initial broadcast of the replacement tx before continuing with the test.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK ee2417ed61

Tree-SHA512: ecc8fb44cac6097a949e4ee622f6f654f49851d7966359532ab3af4c5ed9d587bf08110820b473a616cde3ae6fc8d0da9bb3cee39347655a8c433e819d4d1065
2023-06-13 09:43:53 +01:00
Andrew Chow
d80348ccb6 Merge bitcoin/bitcoin#27853: rest: bugfix, fix crash error when calling /deploymentinfo
7d452d826a test: add coverage for `/deploymentinfo` passing a blockhash (brunoerg)
ce887eaf49 rest: bugfix, fix crash error when calling `/deploymentinfo` (brunoerg)

Pull request description:

  Calling `/deploymentinfo` passing a valid blockhash makes bitcoind to crash. It happens because we're pushing a JSON value of type array when it expects type object. See:
  ```cpp
  jsonRequest.params = UniValue(UniValue::VARR);
  ```
  ```cpp
  jsonRequest.params.pushKV("blockhash", hash_str);
  ```

  This PR fixes it by changing `pushKV` to `push_back` and adds more test coverage.

ACKs for top commit:
  achow101:
    ACK 7d452d826a
  stickies-v:
    ACK 7d452d826a

Tree-SHA512: f01551e556aba2380c3eaed0bc59057304302c202d317d7c1eec5f7ef839851f672aed80819a8719cb1cbbad2aad735d6d44314ac7d6d98bff8217f5a16c312b
2023-06-12 18:34:42 -04:00
Andrew Chow
a1e653828b test: Add test for migrating default wallet and plain file wallet 2023-06-12 15:14:42 -04:00
Andrew Chow
bdbe3fd76b wallet: Generated migrated wallet's path from walletdir and name
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
2023-06-12 15:13:50 -04:00
Hennadii Stepanov
3ef756a5b5 Remove txmempool implicit-integer-sign-change sanitizer suppressions 2023-06-12 19:48:47 +01:00
Hennadii Stepanov
d2f6d2a95a Use int32_t type for most transaction size/weight values
This change gets rid of a few casts and makes the following commit diff
smaller.
2023-06-12 19:47:19 +01:00
Martin Zumsande
ee2417ed61 test: fix intermittent failure in p2p_leak_tx.py 2023-06-12 14:46:15 -04:00
Murch
76c5ea703e fuzz: Fix mini_miner_selection running out of coin
Fixes a bug in the mini_miner_selection fuzz test found by fuzzing:
It was possible for the mini_miner_selection fuzz test to generated
transactions that created fewer new spendable outputs than the two
inputs they each spend. If the fuzz seed did so consistently, eventually
it would cause a `pop_front()` on an empty available_coins.

Fixed by:
- asserting that available_coins is not empty before generating tx
- allowing to build tx with a single coin if only one is available
2023-06-12 14:19:53 -04:00
brunoerg
5fa4055452 net: do not break when addr is not from a distinct network group
When the address is from a network group we already caught,
do a `continue` and try to find another address until conditions
are met or we reach the limit (`nTries`).
2023-06-12 13:58:45 -03:00
Ryan Ofsky
c92fd63886 Merge bitcoin/bitcoin#27708: Return EXIT_FAILURE on post-init fatal errors
61c569ab60 refactor: decouple early return commands from AppInit (furszy)
4927167f85 gui: return EXIT_FAILURE on post-init fatal errors (furszy)
3b2c61e819 Return EXIT_FAILURE on post-init fatal errors (furszy)
3c06926cf2 refactor: index: use `AbortNode` in fatal error helper (Sebastian Falbesoner)
9ddf7e03a3 move ThreadImport ABC error to use AbortNode (furszy)

Pull request description:

  It seems odd to return `EXIT_SUCCESS` when the node aborted execution due a fatal internal error
  or any post-init problem that triggers an unrequested shutdown.

  e.g. blocks or coins db I/O errors, disconnect block failure, failure during thread import (external
  blocks loading process error), among others.

ACKs for top commit:
  TheCharlatan:
    ACK 61c569ab60
  ryanofsky:
    Code review ACK 61c569ab60
  pinheadmz:
    ACK 61c569ab60
  theStack:
    Code-review ACK 61c569ab60

Tree-SHA512: 18a59c3acc1c6d12cbc74a20a401e89659740c6477fccb59070c9f97922dfe588468e9e5eef56c5f395762187c34179a5e3954aa5b844787fa13da2e666c63d3
2023-06-12 12:54:49 -04:00
brunoerg
7d452d826a test: add coverage for /deploymentinfo passing a blockhash 2023-06-12 13:30:42 -03:00
fanquake
361a0c00b3 Merge bitcoin/bitcoin#27783: Add public Boost headers explicitly
2484cacb7a Add public Boost headers explicitly (Hennadii Stepanov)
fade2adb5b test: Avoid `BOOST_ASSERT` macro (Hennadii Stepanov)

Pull request description:

  To check symbols in the code base, run:
  ```
  git grep boost::multi_index::identity
  git grep boost::multi_index::indexed_by
  git grep boost::multi_index::tag
  git grep boost::make_tuple
  ```

  Hoping on the absence of conflicts with top-prio PRs :)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 2484cacb7a
  TheCharlatan:
    ACK 2484cacb7a

Tree-SHA512: d122ab028eee76ee1c4609ed51ec8db0c8c768edcc2ff2c0e420a48e051aa71e99748cdb5d22985ae6d97c808c77c1a27561f0715f77b256f74c1c310b37694c
2023-06-12 16:53:16 +01:00
brunoerg
ce887eaf49 rest: bugfix, fix crash error when calling /deploymentinfo 2023-06-12 10:24:14 -03:00
fanquake
6f5f37eefd Merge bitcoin/bitcoin#27357: validation: Move warningcache to ChainstateManager and rename to m_warningcache
552684976b validation: Move warningcache to ChainstateManager (dimitaracev)

Pull request description:

  Removes `warningcache`  and moves it to `ChainstateManager`. Also removes the respective `TODO`  completely.

ACKs for top commit:
  ajtowns:
    ACK 552684976b
  dimitaracev:
    > ACK [5526849](552684976b)
  TheCharlatan:
    ACK 552684976b
  ryanofsky:
    Code review ACK 552684976b

Tree-SHA512: 6869bd7aa4f0b59324e12eb8e3df47f2c9a3f3b0d9b7d45857426ec9e8b71c5573bdcf71db822f8c10aff7d8679a00a4bedc7a256c28f325e744e5d7267b41e9
2023-06-12 13:20:18 +01:00
fanquake
fbe48f97df Merge bitcoin/bitcoin#27625: p2p: Stop relaying non-mempool txs
faa2976a56 Remove mapRelay (MarcoFalke)
fccecd75fe net_processing: relay txs from m_most_recent_block (Anthony Towns)

Pull request description:

  `mapRelay` (used to relay announced transactions that are no longer in the mempool) has issues:

  * It doesn't have an absolute memory limit, only an implicit one based on the rate of transaction announcements
  * <strike>It doesn't have a use-case</strike> EDIT: see below

  Fix all issues by removing `mapRelay`.

  For more context, on why a transaction may have been removed from the mempool, see c2f2abd0a4/src/txmempool.h (L228-L238)

  For my rationale on why it is fine to not relay them:

  Reason | | Rationale
  -- | -- | --
  `EXPIRY` | Expired from mempool | Mempool expiry is by default 2 weeks and can not be less than 1 hour, so a transaction can not be in `mapRelay` while expiring, unless a re-broadcast happened. This should be fine, because the transaction will be re-added to the mempool and potentially announced/relayed on the next re-broadcast.
  `SIZELIMIT` | Removed in size limiting | A low fee transaction, which will be relayed by a different peer after `GETDATA_TX_INTERVAL` or after we sent a `notfound` message. Assuming it ever made it to another peer, otherwise it will happen on re-broadcast (same as with `EXPIRY` above).
  `REORG` | Removed for reorganization | Block races are rare, so reorgs should be rarer. Also, the transaction is likely to be re-accepted via the `disconnectpool` later on. If not, it seems fine to let the originating wallet deal with rebroadcast in this case.
  `BLOCK` | Removed for block | EDIT: Needed for compact block relay, see https://github.com/bitcoin/bitcoin/pull/27625#issuecomment-1544047433
  `CONFLICT` | Removed for conflict with in-block transaction | The peer won't be able to add the tx to the mempool anyway, unless it is on a different block, in which case it seems fine to let the originating wallet take care of the rebroadcast (if needed).
  `REPLACED` | Removed for replacement | EDIT: Also needed for compact block relay, see https://github.com/bitcoin/bitcoin/pull/27625#issuecomment-1544171255 ?

ACKs for top commit:
  sdaftuar:
    ACK faa2976a56
  ajtowns:
    ACK faa2976a56
  glozow:
    code review ACK faa2976a56

Tree-SHA512: 64ae3e387b001bf6bd5b6c938e7317f4361f9bc0b8cc5d8f63a16cda2408d2f634a22f8157dfcd8957502ef358208292ec91e7d70c9c2d8a8c47cc0114ecfebd
2023-06-12 10:50:27 +01:00
fanquake
5111d8e02f Merge bitcoin/bitcoin#27844: ci: Use podman stop over podman kill
faaa62754e ci: Use podman stop over podman kill (MarcoFalke)

Pull request description:

  This should avoid a race where the kill is not done when spinning up the new container. podman stop waits 10 seconds by default.

  Fixes https://github.com/bitcoin/bitcoin/pull/27777#discussion_r1217942753

ACKs for top commit:
  dimitaracev:
    ACK [faaa627](faaa62754e)

Tree-SHA512: d46a32429629dcfa711a2d0abe79100f5593d72f8e5eded7b505b0f270e28abfeba0a96bec43e9951a76a96bb23fe2c7092433e5e0c66510e3e3b6c3cb58f4db
2023-06-12 09:53:07 +01:00
fanquake
bc80b2df1e Merge bitcoin/bitcoin#27840: contrib: docs fix --import-keys flag on verify.py
ceb0168935 contrib: docs fix --import-keys flag on verify.py (Bufo)

Pull request description:

  When trying to run `./contrib/verify-binaries/verify.py` with the --import-keys flag, I figured that there was a little mistake in the docs. It stated that the `--import-keys` flag has to be provided after the arguments, instead of before. It was stated correctly in the rest of the README, but not in this particular case.

  I tested this on macOS 13.4 as well as on Debian 10.

ACKs for top commit:
  willcl-ark:
    ACK ceb0168
  Tguntenaar:
    I ran into this together with bufo24, therefore ACK [ceb0168](ceb0168935)
  MarcoFalke:
    lgtm ACK ceb0168935

Tree-SHA512: a8acdcc7f92c43e731ae8b6c86dba8df06481e9765118b5b2b63caf3cdf0dd34333e48c848602003082f49d971d7127373aaf9ff7a8572d41285869efa06b0f4
2023-06-12 09:52:09 +01:00
fanquake
62140b5e10 Merge bitcoin/bitcoin#27834: ci: Nuke Android APK task, Use credits for tsan
fa22538e48 ci: Nuke Android APK task, Use credits for tsan (MarcoFalke)

Pull request description:

  The Android task has many issues:

  * It runs into more network timeouts (intermittent failures) than other tasks
  * It never failed since its introduction years ago in a scenario where all other tasks passed, thus it is useless (so far)

  Fix all issues by removing the task. Note that the CI env file is kept, so anyone can still run the Android CI.

  Also, use the compute credits to promote tsan, a more useful task.

ACKs for top commit:
  dergoegge:
    ACK fa22538e48 - nuke it
  glozow:
    ACK fa22538e48

Tree-SHA512: e2aa1bd2d0288a769d48412d00cef50d385dca86c1090ba2155f4776da69f34f5b2735b33526bbf845f33f4b55677578c7680f16ef67218b6f73b17d4be7c836
2023-06-12 09:49:08 +01:00
furszy
61c569ab60 refactor: decouple early return commands from AppInit
Cleaned up the init flow to make it more obvious when
the 'exit_status' value will and won't be returned.

This is because it was confusing that `AppInit` was
returning true under two different circumstances:

1) When bitcoind was launched only to retrieve the "-help"
or "-version" information. In this case, the app was
not initialized.

2) When the user triggers a shutdown. In this case,
the app was fully initialized.
2023-06-10 11:10:29 -03:00
furszy
4927167f85 gui: return EXIT_FAILURE on post-init fatal errors 2023-06-10 11:10:29 -03:00
furszy
3b2c61e819 Return EXIT_FAILURE on post-init fatal errors
It seems odd to return `EXIT_SUCCESS` when the node aborted
execution due a fatal internal error or any post-init problem
that triggers an unrequested shutdown.

e.g. blocks or coins db I/O errors, disconnect block failure,
failure during thread import (external blocks loading process
error), among others.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-06-09 17:52:23 -03:00
Ryan Ofsky
153a6882f4 Merge bitcoin/bitcoin#27576: kernel: Remove args, settings, chainparams, chainparamsbase from kernel library
db77f87c63 scripted-diff: move settings to common namespace (TheCharlatan)
c27e4bdc35 move-only: Move settings to the common library (TheCharlatan)
c2dae5d7d8 kernel: Remove chainparams, chainparamsbase, args, settings from kernel library (TheCharlatan)
05870b1c92 refactor: Remove gArgs access from validation.cpp (TheCharlatan)
8789b11114 refactor: Add path argument to FindSnapshotChainstateDir (TheCharlatan)
ef95be334f refactor: Add stop_at_height option in ChainstateManager (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/27587 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel".

  ---

  This completes the removal of the node's chainparams, chainparamsbase, args and settings files and their respective classes from the kernel library. This is the last pull request in a long series working towards decoupling the `ArgsManager` and the `gArgs` global from kernel code. These prior pull requests are: https://github.com/bitcoin/bitcoin/pull/26177 https://github.com/bitcoin/bitcoin/pull/27125 https://github.com/bitcoin/bitcoin/pull/25527 https://github.com/bitcoin/bitcoin/pull/25487 https://github.com/bitcoin/bitcoin/pull/25290

ACKs for top commit:
  MarcoFalke:
    lgtm ACK db77f87c63 🍄
  hebasto:
    ACK db77f87c63, I have reviewed the code and it looks OK.
  ryanofsky:
    Code review ACK db77f87c63. Looks great!

Tree-SHA512: cbfbd705d056f2f10f16810d4f869eb152362fff2c5ddae5e1ac6785deae095588e52ad48b29d921962b085e51de1e0ecab6e50f46149ffe3c16250608a2c93a
2023-06-09 14:58:49 -04:00
Ryan Ofsky
456af7a955 Merge bitcoin/bitcoin#27467: p2p: skip netgroup diversity follow-up
11bb31c1c4 p2p: "skip netgroup diversity of new connections for tor/i2p/cjdns" follow-up (Jon Atack)

Pull request description:

  In #27374 the role of the `setConnected` data structure in `CConnman::ThreadOpenConnections` changed from the set of outbound peer netgroups to those of outbound IPv4/6 peers only.

  In accordance with the changed semantics, this pull fixes a code comment regarding feeler connections and updates the naming of `setConnected` to `outbound_ipv46_peer_netgroups`.

  Addresses https://github.com/bitcoin/bitcoin/pull/27374#discussion_r1167172725.

ACKs for top commit:
  mzumsande:
    Code Review ACK 11bb31c1c4
  vasild:
    ACK 11bb31c1c4
  ryanofsky:
    Code review ACK 11bb31c1c4

Tree-SHA512: df9151a6cce53c279e549683a9f30fdc23d513dc664cfee1cf0eb8ec80b2848d32c80a92cc0a9f47d967f305864975ffb339fe0eaa80bc3bef1b28406419eb96
2023-06-09 14:21:19 -04:00
MarcoFalke
faaa62754e ci: Use podman stop over podman kill
This should avoid a race where the kill is not done when spinning up the
new container. podman stop waits 10 seconds by default.
2023-06-09 16:58:38 +02:00
Bufo
ceb0168935 contrib: docs fix --import-keys flag on verify.py 2023-06-08 22:26:09 +02:00
Sebastian Falbesoner
3c06926cf2 refactor: index: use AbortNode in fatal error helper
Deduplicates code in the `FatalError` template function by using
`AbortNode` which does the exact same thing if called without any user
message (i.e. without second parameter specified). The template is still
kept for ease-of-use w.r.t. not having to call `tfm::format(...)` at the
call-side each time, and also to keep the diff minimal.
2023-06-08 16:38:36 -03:00
furszy
9ddf7e03a3 move ThreadImport ABC error to use AbortNode
'StartShutdown' should only be used for user requested
shutdowns. Internal errors that cause a shutdown should
use 'AbortNode'.
2023-06-08 16:38:36 -03:00
Andrew Chow
e9379f1ffa rpc, wallet: Include information about blank flag
This allows us to test that the blank flag is being set appropriately.
2023-06-08 07:29:00 -04:00
MarcoFalke
faafc35a77 doc: Clarify that -datacarriersize applies to the full raw scriptPubKey, not the data push 2023-06-08 11:53:50 +02:00
MarcoFalke
55550e7fe7 test: Add -datacarriersize=2 tests 2023-06-08 11:53:45 +02:00
MarcoFalke
faa2976a56 Remove mapRelay 2023-06-08 11:52:30 +02:00
Anthony Towns
fccecd75fe net_processing: relay txs from m_most_recent_block 2023-06-08 11:52:08 +02:00
Andrew Chow
52932c5adb walletdb: Refactor wallet flags loading
Move wallet flags loading to its own function in WalletBatch

The return value is changed to be TOO_NEW rather than CORRUPT when
unknown flags are found.
2023-06-08 05:43:21 -04:00
Andrew Chow
01b35b55a1 walletdb: Refactor minversion loading
Move minversion loading to its own function in WalletBatch
2023-06-08 05:43:21 -04:00
Andrew Chow
a36134fcc7 Merge bitcoin/bitcoin#27838: ci: Invalidate Cirrus CI docker cache
fac7f4ab5e ci: Invalidate Cirrus CI docker cache (MarcoFalke)

Pull request description:

  Currently the Cirrus CI seems to fail for some reason. No idea why, but maybe invalidating the Docker image cache fixes it?

  The failure is:

  ```
      Failed to start an instance! Failed to pull null image! Repository does not exist or may require authentication.
      Container errored with 'ImagePullBackOff: Back-off pulling image "gcr.io/cirrus-ci-community/bitcoin/bitcoin/ci/test_imagefile:b3e086572130d8954f84bb90778d02e2cfbb6dc624c01e2f74ee17335a9c453e"'
  ```

  https://cirrus-ci.com/task/5983593860694016

ACKs for top commit:
  achow101:
    ACK fac7f4ab5e

Tree-SHA512: 3e8ed4f51ba29109e508f269b4281c81dfa844dc0dd5658a83388da586fc4234ae5c076c174122834114fb7c9e0026c042ea38deac53e7d796cc59526fbb7c46
2023-06-08 05:41:21 -04:00
MarcoFalke
fac7f4ab5e ci: Invalidate Cirrus CI docker cache 2023-06-08 10:28:38 +02:00
MarcoFalke
fa22538e48 ci: Nuke Android APK task, Use credits for tsan 2023-06-07 16:05:26 +02:00
fanquake
2026301405 Merge bitcoin/bitcoin#27810: fuzz: Partially revert #27780
71200ac390 [fuzz] Only check duplicate coinbase script when block was valid (dergoegge)

Pull request description:

  Partially revert #27780, because moving the duplicate coinbase check out of the `was_valid` branch leads to non-bug crashes in the fuzz target.

  For context and further explanation see: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=59516

ACKs for top commit:
  MarcoFalke:
    nice lgtm ACK 71200ac390

Tree-SHA512: 8c38e5ff9de6331016b9a0c5e435d007d46186151b04c09085f617bb31627a28ad56678066fe152372a3ad8656f026439e3e2f9ee61d7ef588072aef8124eaa3
2023-06-07 15:55:36 +02:00
fanquake
6cba698a59 Merge bitcoin/bitcoin#27824: ci: enable AArch64 target in MSAN jobs
2ebeb421dd ci: enable AArch64 target in MSAN jobs (fanquake)
c93bfc54e8 ci: use LLVM 16.0.5 in MSAN jobs (fanquake)

Pull request description:

  Make it possible to run the MSAN jobs on aarch64, as it was previously.

ACKs for top commit:
  dergoegge:
    utACK 2ebeb421dd

Tree-SHA512: 9c2e9800f24258fd0f4be5e337178ff473158b2e8f1c431c825465b4f3bd27802422d540d0e7a3b84878c5936f8c302a2fd1c428f41fcd992e12bcc3685698e3
2023-06-07 13:29:36 +01:00
Andrew Chow
1af72e728d Merge bitcoin/bitcoin#27501: mempool / rpc: add getprioritisedtransactions, delete a mapDeltas entry when delta==0
67b7fecacd [mempool] clear mapDeltas entry if prioritisetransaction sets delta to 0 (glozow)
c1061acb9d [functional test] prioritisation is not removed during replacement and expiry (glozow)
0e5874f0b0 [functional test] getprioritisedtransactions RPC (glozow)
99f8046829 [rpc] add getprioritisedtransactions (glozow)
9e9ca36c80 [mempool] add GetPrioritisedTransactions (glozow)

Pull request description:

  Add an RPC to get prioritised transactions (also tells you whether the tx is in mempool or not), helping users clean up `mapDeltas` manually. When `CTxMemPool::PrioritiseTransaction` sets a delta to 0, remove the entry from `mapDeltas`.

  Motivation / Background
  - `mapDeltas` entries are never removed from mapDeltas except when the tx is mined in a block or conflicted.
  - Mostly it is a feature to allow `prioritisetransaction` for a tx that isn't in the mempool {yet, anymore}. A user can may resbumit a tx and it retains its priority, or mark a tx as "definitely accept" before it is seen.
  - Since #8448, `mapDeltas` is persisted to mempool.dat and loaded on restart. This is also good, otherwise we lose prioritisation on restart.
  - Note the removal due to block/conflict is only done when `removeForBlock` is called, i.e. when the block is received. If you load a mempool.dat containing `mapDeltas` with transactions that were mined already (e.g. the file was saved prior to the last few blocks), you don't delete them.
  - Related: #4818 and #6464.
  - There is no way to query the node for not-in-mempool `mapDeltas`. If you add a priority and forget what the value was, the only way to get that information is to inspect mempool.dat.
  - Calling `prioritisetransaction` with an inverse value does not remove it from `mapDeltas`, it just sets the value to 0. It disappears on a restart (`LoadMempool` checks if delta is 0), but that might not happen for a while.

  Added together, if a user calls `prioritisetransaction` very regularly and not all those transactions get mined/conflicted, `mapDeltas` might keep lots of entries of delta=0 around. A user should clean up the not-in-mempool prioritisations, but that's currently difficult without keeping track of what those txids/amounts are.

ACKs for top commit:
  achow101:
    ACK 67b7fecacd
  theStack:
    Code-review ACK 67b7fecacd
  instagibbs:
    code review ACK 67b7fecacd
  ajtowns:
    ACK 67b7fecacd code review only, some nits

Tree-SHA512: 9df48b622ef27f33db1a2748f682bb3f16abe8172fcb7ac3c1a3e1654121ffb9b31aeaad5570c4162261f7e2ff5b5912ddc61a1b8beac0e9f346a86f5952260a
2023-06-07 03:29:05 -04:00
fanquake
8cc65f093c Merge bitcoin/bitcoin#27779: guix: remove cURL from build env
641897a83d guix: remove cURL from build env (fanquake)

Pull request description:

  Remove cURL & osslsigncode option.

ACKs for top commit:
  hebasto:
    ACK 641897a83d
  TheCharlatan:
    ACK 641897a83d

Tree-SHA512: f917afe5aaffa8436009c63ace4a78ed3bc8a13fffeb12db2c12204f603fbd05f975f798c1bccaefa22b6131c91415477c115921dfe89f8fa064aab82bcd4a6f
2023-06-06 10:39:58 +01:00
Martin Zumsande
e7cf8657e1 test: add unit test for local address advertising 2023-06-05 13:01:29 -04:00
Martin Zumsande
f4754b9dfb net: restrict self-advertisements with privacy networks
Stop advertising
1) our i2p/onion address to peers from other networks
2) Local addresses of non-privacy networks to i2p/onion peers
Doing so could lead to fingerprinting ourselves.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2023-06-05 11:02:47 -04:00
Martin Zumsande
e4d541c7cf net, refactor: pass reference for peer address in GetReachabilityFrom
The address of the peer always exists (because addr is a member of
CNode), so it was not possible to pass a nullptr before.
Also remove NET_UNKNOWN, which is unused now.
2023-06-05 11:02:47 -04:00
Martin Zumsande
62d73f5370 net, refactor: pass CNode instead of CNetAddr to GetLocalAddress
Access to CNode will be needed in the following commits.
2023-06-05 11:02:47 -04:00
fanquake
2ebeb421dd ci: enable AArch64 target in MSAN jobs
Use Native.
2023-06-05 15:39:51 +01:00
fanquake
c93bfc54e8 ci: use LLVM 16.0.5 in MSAN jobs 2023-06-05 11:23:25 +01:00
fanquake
f4a8269dfc Merge bitcoin/bitcoin#27801: wallet: Add tracing for sqlite statements
ff9d961bf3 wallet: Add tracing for sqlite statements (Ryan Ofsky)

Pull request description:

  I found sqlite tracing was useful for debugging a test in #27790, and thought it might be helpful in other contexts too, so this PR adds an option to enable it. Tracing is still disabled by default and only shown with `-debug=walletdb -loglevel=walletdb:trace` options.

ACKs for top commit:
  achow101:
    ACK ff9d961bf3
  kevkevinpal:
    ACK ff9d961bf3
  theStack:
    ACK ff9d961bf3

Tree-SHA512: 592fabfab3218cec36c2d00a21cd535fa840daa126ee8440c384952fbb3913180aa3796066c630087e933d6517f19089b867f158e0b737f25283a14799eefb05
2023-06-05 10:51:08 +01:00
Sebastian Falbesoner
5524fa00fa doc: add release note about removal of deprecatedrpc=walletwarningfield flag
Co-authored-by: Jon Atack <jon@atack.com>
2023-06-04 23:53:26 +02:00
Jon Atack
5c77db7354 Restorewallet/createwallet help documentation fixups/improvements 2023-06-04 23:53:26 +02:00
Sebastian Falbesoner
a00ae31fcc rpc: remove deprecated "warning" field from {create,load,restore,unload}wallet
Co-authored-by: Jon Atack <jon@atack.com>
2023-06-04 23:53:26 +02:00
Sebastian Falbesoner
54877253c8 test: avoid sporadic MINIMALDATA failure in feature_taproot.py (fixes #27595)
The functional test feature_taproot.py fails in some rare cases on the
execution of the `"branched_codesep"` spending script. The problem
occurs if the first data-push (having random content with a random
length in the range [0, 510]) has a length of 1 and the single byte has
value of [1...16] or [-1]; in this case, the data-push is not minimally
encoded by test framework's CScript class (i.e. doesn't use the special
op-codes OP_1...OP_16 or OP_1NEGATE) and the script interpreter throws
an SCRIPT_ERR_MINIMALDATA error:

```
test_framework.authproxy.JSONRPCException: non-mandatory-script-verify-flag (Data push larger than necessary) (-26)
```

Background:
The functional test framework's CScript class translates passed
bytes/bytearrays always to data pushes using OP_PUSHx/OP_PUSHDATA{1,2,4}
op-codes. E.g. the expression `CScript(bytes([1]))` yields
`bytes([OP_PUSH1, 1])` instead of the minimal-encoded `bytes([OP_1])`.

Fix this by adapting the random-size range to [2,...], i.e. never pass
byte-arrays below length two to be pushed.

Closes #27595.
2023-06-04 23:50:30 +02:00
dergoegge
71200ac390 [fuzz] Only check duplicate coinbase script when block was valid 2023-06-03 15:37:11 +02:00
Ryan Ofsky
ff9d961bf3 wallet: Add tracing for sqlite statements
I found sqlite tracing was useful for debugging a test in #27790, and thought
it might be helpful in other contexts too, so this PR adds an option to enable
it. Tracing is still disabled by default and only shown with `-debug=walletdb
-loglevel=walletdb:trace` options.
2023-06-02 16:47:33 -04:00
fanquake
7f2019755d Merge bitcoin/bitcoin#27790: walletdb: Add PrefixCursor
ba616b932c wallet: Add GetPrefixCursor to DatabaseBatch (Andrew Chow)
1d858b055d walletdb: Handle when database keys are empty (Ryan Ofsky)
84b2f353bb walletdb: Consistently clear key and value streams before writing (Andrew Chow)

Pull request description:

  Split from #24914 as suggested in https://github.com/bitcoin/bitcoin/pull/24914#pullrequestreview-1442091917

  This PR adds a wallet database cursor that gives a view over all of the records beginning with the same prefix.

ACKs for top commit:
  ryanofsky:
    Code review ACK ba616b932c. Just suggested changes since last review
  furszy:
    ACK ba616b93

Tree-SHA512: 38a61849f108d8003d28c599b1ad0421ac9beb3afe14c02f1253e7b4efc3d4eef483e32647a820fc6636bca3f9efeff9fe062b6b602e0cded69f21f8b26af544
2023-06-02 17:00:19 +01:00
fanquake
641897a83d guix: remove cURL from build env 2023-06-02 16:32:47 +01:00
fanquake
e43fdfd9ad Merge bitcoin/bitcoin#27225: doc: document json rpc endpoints
65e3abcbf2 doc: document json rpc endpoints (willcl-ark)

Pull request description:

  fixes #20246

  This documents the two JSON-RPC endpoints available, details when they are active, specifies when they can or must be used, and outlines some known behaviour quirks.

ACKs for top commit:
  fanquake:
    ACK 65e3abcbf2 - Seems fine. Can be improved if need be.

Tree-SHA512: d557c2caf000a1bdd7b46c9da38afe63dc28446ba4a961526f1af3cec81d994a9da663e02c81ebdc4f609b794585349cfca77a582dc1e788c120de1d3b4c7db6
2023-06-02 16:27:27 +01:00
fanquake
436c185b05 Merge bitcoin/bitcoin#27256: refactor: rpc: Remove unnecessary uses of ParseNonRFCJSONValue() and rename it
cfbc8a623b refactor: rpc: hide and rename ParseNonRFCJSONValue() (stickies-v)
6c8bde6d54 test: move coverage on ParseNonRFCJSONValue() to UniValue::read() (stickies-v)

Pull request description:

  Follow-up to https://github.com/bitcoin/bitcoin/pull/26612#issuecomment-1453623741. As per https://github.com/bitcoin/bitcoin/pull/26506#pullrequestreview-1211984059, `ParseNonRFCJSONValue()` is no longer necessary and we can use `UniValue::read()` directly:

  > IIRC before that PR UniValue::read could only parse JSON object and array values, but now it will parse string/number/boolean/null values as well. laanwj pointed this out in https://github.com/bitcoin/bitcoin/issues/9028#issuecomment-257885368

  The implementation of `ParseNonRFCJSONValue()` was already [simplified in #26612](https://github.com/bitcoin/bitcoin/pull/26612/files#diff-84c7a7f36362b9724c31e5dec9879b2f81eae0d0addbc9c0933c3558c577de65R259-R263)  and [test coverage updated](https://github.com/bitcoin/bitcoin/pull/26612/files#diff-fc0f86b6c3bb23b0e983bcf79d7546d1c9eaa15d6e4d8a7b03b5b85955f585abR292-R312) to ensure behaviour didn't change.

  To avoid code duplication, we keep the function to throw on invalid input data but rename it to `Parse()` and remove it from the header.

  The existing test coverage we had on `ParseNonRFCJSONValue()` is moved over to `UniValue::read()`.

ACKs for top commit:
  ryanofsky:
    Code review ACK cfbc8a623b. Only change since last review is adding a new test

Tree-SHA512: 89be959d2353af7ace0c1348ba1600b9ac1d3c7b3cf7f0b59f6e005b7fb9d695ce3e8720e1be3cf77fe7e318a4017c880df108928e7179ec50447583d13bc781
2023-06-02 16:18:11 +01:00
glozow
b22408df16 Merge bitcoin/bitcoin#27603: test: added coverage to mining_basic.py
a7b46a1fea test: added coverage to mining_basic.py (kevkevin)

Pull request description:

  Included a test that checks if we call submitblock with block.vtx.empty() then it throws an rpc deserialization error, currently we only test if !block.vtx->IsCoinBase() throws an rpc deserialization error

  I've tested to make sure this actually doing what I intended by breaking up this if block into two if blocks with different error messages and running the functional test
  322ec63b01/src/rpc/mining.cpp (L963)

  This change should increase the test coverage for the `submitblock()` rpc in `./src/rpc/mining.cpp`

ACKs for top commit:
  theStack:
    ACK a7b46a1fea

Tree-SHA512: 4078cb1fa879cc9e34438319f73085b521b90a5a95348b23e494cf8e5ac792ec426bc0e1a63e949645e16afebe54c5f35a194f02e20b7273871163d89a5c44e6
2023-06-02 14:21:52 +01:00
glozow
6a560aceb7 Merge bitcoin/bitcoin#27803: Fuzz: Mitigate timeout in CalculateTotalBumpFees
5d718f6913 Mitigate timeout in CalculateTotalBumpFees (Murch)

Pull request description:

  The slow fuzz seed described in #27799 was just slower than expected, not an endless loop. Ensuring that every anscestor is only processed once speeds up the termination of the graph traversal.

  Fixes #27799

ACKs for top commit:
  glozow:
    ACK 5d718f6913

Tree-SHA512: f3c7cd2ef6716332136c75b43f6d54ce920be6f546a11bbf92b1fd65575607c42cc24b319691d86d0db038335636ba12b6387383a184f1589a8d71d1180f194f
2023-06-02 10:59:35 +01:00
fanquake
8a972813ba Merge bitcoin/bitcoin#27737: ci: compile Clang and compiler-rt in msan jobs
5763b232e6 ci: return to using Ubuntu 22.04 in MSAN jobs (fanquake)
d3cbcbf626 ci: compile clang and compiler-rt in MSAN jobs (fanquake)
796bd1d0d1 ci: use LLVM 16.0.4 in MSAN jobs (fanquake)
883bc9f561 ci: remove extra CC & CXX from MSAN jobs (fanquake)
2d4f4b8f29 ci: standardize custom libc++ usage in MSAN jobs (fanquake)

Pull request description:

  This reworks the MSAN CIs, to first compile Clang and compiler-rt (using GCC 12), and then, compile an MSAN instrumented libc++ using the just-built Clang 16. This fixes the `native_fuzz_with_msan` job, working around https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1005341, by not using the Debian provided Clang/LLVM.

  Also included are changes to streamline how we use our "custom libc++", according to upstream: https://releases.llvm.org/16.0.0/projects/libcxx/docs/UsingLibcxx.html#using-a-custom-built-libc, as well as other minor cleanups in the CI configs.

  An example job is currently running in the qa-assets repo: https://github.com/bitcoin-core/qa-assets/pull/129 (https://cirrus-ci.com/task/4632561431871488).

ACKs for top commit:
  dergoegge:
    utACK 5763b232e6

Tree-SHA512: 4f2a6e0b796bb1830b8346dd1e55eaa86a79037b8b4f16a336c1e29f4fc460acca2ecba076635459370bcbb4009333cb79d27ef1521c1fb5db7599cd5bdf558c
2023-06-02 10:42:05 +01:00
fanquake
83c7269965 Merge bitcoin/bitcoin#27800: streams: Drop confusing DataStream::Serialize method and << operator
5cd0717a54 streams: Drop confusing DataStream::Serialize method and << operator (Ryan Ofsky)

Pull request description:

  DataStream Serialize method has surprising behavior because it just serializes raw bytes without a length prefix. When you serialize a string or vector, a length prefix is serialized before the raw object contents so the object can be unambiguously deserialized later. But DataStreams don't support deserializing at all and just dump the raw bytes.

  Having this inconsistency is not necessary and could be confusing (see https://github.com/bitcoin/bitcoin/pull/27790#discussion_r1212315030) so this PR just drops the DataStream::Serialize method.

ACKs for top commit:
  furszy:
    lgtm ACK 5cd0717a
  MarcoFalke:
    lgtm ACK 5cd0717a54 🌙

Tree-SHA512: 49dd117de266f091a5336b13a91c5d8658abe1b3a0a9c51c8b5f6a2e0e814781b73afc39256353e79dade603a8a2761e8536716d1a48499720c266f4500477e2
2023-06-02 10:28:01 +01:00
fanquake
dba757fb35 Merge bitcoin/bitcoin#27802: Update .style.yapf
bc70072de1 Update .style.yapf (Ari)

Pull request description:

  Corrected a minor typo

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bc70072de1

Tree-SHA512: 04146d17dc034a275be59d75d45977ff99a0c911a0b53df1c50bc874dc20268faa9bd93d62715a4f629e4cd9ce42d6a5ae1d4d99a2325143affbebccfb8e0602
2023-06-02 10:08:16 +01:00
Murch
5d718f6913 Mitigate timeout in CalculateTotalBumpFees
The slow fuzz seed described in #27799 was just slower than expected,
not an endless loop. Ensuring that every anscestor is only processed
once speeds up the termination of the graph traversal.

Fixes #27799
2023-06-01 18:04:44 -04:00
Andrew Chow
34ac3f438a Merge bitcoin/bitcoin#26485: RPC: Accept options as named-only parameters
2cd28e9fef rpc: Add check for unintended option/parameter name clashes (Ryan Ofsky)
95d7de0964 test: Update python tests to use named parameters instead of options objects (Ryan Ofsky)
96233146dd RPC: Allow RPC methods accepting options to take named parameters (Ryan Ofsky)
702b56d2a8 RPC: Add add OBJ_NAMED_PARAMS type (Ryan Ofsky)

Pull request description:

  Allow RPC methods which take an `options` parameter (`importmulti`, `listunspent`, `fundrawtransaction`, `bumpfee`, `send`, `sendall`, `walletcreatefundedpsbt`, `simulaterawtransaction`), to accept the options as named parameters, without the need for nested JSON objects.

  This makes it possible to make calls like:

  ```sh
  src/bitcoin-cli -named bumpfee txid fee_rate=10
  ```

  instead of

  ```sh
  src/bitcoin-cli -named bumpfee txid options='{"fee_rate": 10}'
  ```

  RPC help is also updated to show options as top level named arguments instead of as nested objects.

  <details><summary>diff</summary>
  <p>

  ```diff
  @@ -15,16 +15,17 @@

   Arguments:
   1. txid                           (string, required) The txid to be bumped
  -2. options                        (json object, optional)
  +2. options                        (json object, optional) Options object that can be used to pass named arguments, listed below.
  +
  +Named Arguments:
  -     {
  -       "conf_target": n,          (numeric, optional, default=wallet -txconfirmtarget) Confirmation target in blocks
  +conf_target                       (numeric, optional, default=wallet -txconfirmtarget) Confirmation target in blocks

  -       "fee_rate": amount,        (numeric or string, optional, default=not set, fall back to wallet fee estimation)
  +fee_rate                          (numeric or string, optional, default=not set, fall back to wallet fee estimation)
                                     Specify a fee rate in sat/vB instead of relying on the built-in fee estimator.
                                     Must be at least 1.000 sat/vB higher than the current transaction fee rate.
                                     WARNING: before version 0.21, fee_rate was in BTC/kvB. As of 0.21, fee_rate is in sat/vB.

  -       "replaceable": bool,       (boolean, optional, default=true) Whether the new transaction should still be
  +replaceable                       (boolean, optional, default=true) Whether the new transaction should still be
                                     marked bip-125 replaceable. If true, the sequence numbers in the transaction will
                                     be left unchanged from the original. If false, any input sequence numbers in the
                                     original transaction that were less than 0xfffffffe will be increased to 0xfffffffe
  @@ -32,11 +33,10 @@
                                     still be replaceable in practice, for example if it has unconfirmed ancestors which
                                     are replaceable).

  -       "estimate_mode": "str",    (string, optional, default="unset") The fee estimate mode, must be one of (case insensitive):
  +estimate_mode                     (string, optional, default="unset") The fee estimate mode, must be one of (case insensitive):
                                     "unset"
                                     "economical"
                                     "conservative"
  -     }

   Result:
   {                    (json object)
  ```

  </p>
  </details>

  **Review suggestion:** To understand this PR, it is probably easiest to review the commits in reverse order because the last commit shows the external API changes, the middle commit shows the internal API changes, and the first commit contains the low-level implementation.

ACKs for top commit:
  achow101:
    ACK 2cd28e9fef

Tree-SHA512: 50f6e78fa622826dab3f810400d8c1a03a98a090b1f2fea79729c58ad8cff955554bd44c2a5975f62a526b900dda68981862fd7d7d05c17f94f5b5d847317436
2023-06-01 15:30:31 -04:00
Ari
bc70072de1 Update .style.yapf
Corrected a minor typo
2023-06-01 23:35:10 +05:30
Andrew Chow
ba616b932c wallet: Add GetPrefixCursor to DatabaseBatch
In order to get records beginning with a prefix, we will need a cursor
specifically for that prefix. So add a GetPrefixCursor function and
DatabaseCursor classes for dealing with those prefixes.

Tested on each supported db engine.

1) Write two different key->value elements to db.
2) Create a new prefix cursor and walk-through every returned element,
   verifying that it gets parsed properly.
3) Try to move the cursor outside the filtered range: expect failure
   and flag complete=true.

Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
Co-Authored-By: furszy <matiasfurszyfer@protonmail.com>
2023-06-01 13:09:08 -04:00
Ryan Ofsky
5cd0717a54 streams: Drop confusing DataStream::Serialize method and << operator
DataStream Serialize method has surprising behavior because it just serializes
raw bytes without a length prefix. When you serialize a string or vector, a
length prefix is serialized before the raw object contents so the object can be
unambiguously deserialized later. But DataStreams don't support deserializing
at all and just dump the raw bytes.

Having this inconsistency is not necessary and could be confusing (see
https://github.com/bitcoin/bitcoin/pull/27790#discussion_r1212315030) so this
PR just drops the DataStream::Serialize method.
2023-06-01 10:27:33 -04:00
fanquake
9e54dde04c Merge bitcoin/bitcoin#27719: doc: remove Tor link & generalize onion getnodeaddresses RPC
6fce5ddc17 doc: update getnodeaddresses for CJDNS, I2P and Tor and rm link (Marnix)

Pull request description:

  - remove broken link about how to properly configure tor
  - generalize getnodeaddresses RPC in doc

ACKs for top commit:
  fanquake:
    ACK 6fce5ddc17

Tree-SHA512: 3a077a0724c57a5c6182d40fbf34a84d2515bf1bf06ea0ce717174d0a27f5b19b9521c1ed1995adfdf4d43c2ce978a81e2ec9e3c8faf83f5188571fa75ea5314
2023-06-01 15:24:23 +01:00
Marnix
6fce5ddc17 doc: update getnodeaddresses for CJDNS, I2P and Tor and rm link
- unify bitcoin-cli getnodeaddresses for CJDNS, I2P and Tor
- remove outdated link to Tor project
2023-06-01 15:52:51 +02:00
Ryan Ofsky
1d858b055d walletdb: Handle when database keys are empty 2023-05-31 15:24:06 -04:00
Andrew Chow
84b2f353bb walletdb: Consistently clear key and value streams before writing
Before writing data to the output key and value streams, make sure they
are cleared.
2023-05-31 15:17:05 -04:00
Andrew Chow
3a83d4417b Merge bitcoin/bitcoin#27720: index: prevent race by calling 'CustomInit' prior setting 'synced' flag
3126454dcf index: prevent race by calling 'CustomInit' prior setting 'synced' flag (furszy)

Pull request description:

  Decoupled from #27607.

  Fixed a potential race condition in master (not possible so far) that could become an actual issue soon.
  Where the index's  `CustomAppend` method could be called (from `BlockConnected`) before its
  `CustomInit` method, causing the index to try to update itself before it is initialized.

  This could happen because we set the index `m_synced` flag (which enables `BlockConnected` events)
  before calling to the child class init function (`CustomInit`). So, for example, the block filter index could
  process a block before initialize the next filter position field and end up overwriting the first stored filter.

  This race was introduced in bef4e405f3 from https://github.com/bitcoin/bitcoin/pull/25494.

ACKs for top commit:
  achow101:
    ACK 3126454dcf
  mzumsande:
    Code review ACK 3126454dcf
  TheCharlatan:
    Nice, ACK 3126454dcf

Tree-SHA512: 7a53fed1d2035cb4c1f331d6ee9f92d499b6cbb618ea534c6440f5a45ff9b3ac4dcff5fb4b88937f45a0be249e3a9c6dc6c3ac77180f12ae25fc56856ba39735
2023-05-31 13:56:28 -04:00
Hennadii Stepanov
2484cacb7a Add public Boost headers explicitly 2023-05-31 15:43:01 +01:00
Hennadii Stepanov
fade2adb5b test: Avoid BOOST_ASSERT macro
The `BOOST_ASSERT` macro is defined in the `boost/assert.hpp` header.
This change allows to skip `#include <boost/assert.hpp>`.
2023-05-31 15:42:40 +01:00
fanquake
f08bde7f71 Merge bitcoin/bitcoin#27778: ci: Enable float-divide-by-zero check
fa3ab45203 ci: Enable float-divide-by-zero check (MarcoFalke)

Pull request description:

  Enable it, because

  * It is enabled on OSS-Fuzz, so to be able to catch bugs earlier, enable it here as well.
  * It makes sense to enable, because when a float is divided by zero, it may be a logic bug in our code, so it should be suppressed in the suppressions file.

ACKs for top commit:
  willcl-ark:
    utACK fa3ab45203
  dergoegge:
    ACK fa3ab45203

Tree-SHA512: 2c2c025af4fe3ec267b3cfa38f25495e9da678cf6c529a6438ec923ef09a06ad37fa4503c30cbacc83578ac2856a7f729ef70a24befffd61d10ec075132d1ee0
2023-05-31 14:42:46 +01:00
fanquake
30d6c7d8c0 Merge bitcoin/bitcoin#27657: doc: Remove unused NO_BLOOM_VERSION constant
facbcd3742 doc: Remove unused NO_BLOOM_VERSION constant (MarcoFalke)

Pull request description:

  This source code is the wrong place to document historic and now irrelevant details. Also, while touching the docs, clarify that the BIP 35 `mempool` message type is currently also guarded by the BIP 111 `NODE_BLOOM` flag, even though BIP 111 does not mention the `mempool` message type.

ACKs for top commit:
  0xB10C:
    ACK facbcd3
  dergoegge:
    ACK facbcd3742

Tree-SHA512: e69cf5cc2a4e6b061e8996bd9afc0c3e3c4e8174c086ecde425e0e9350b918f1c6612ce273ea1722d30e856049b83dada8782e7e96f676ae0112af85b22f868f
2023-05-31 11:42:40 +01:00
fanquake
2a786ea349 Merge bitcoin/bitcoin#27780: fuzz: Avoid timeout in utxo_total_supply
fafb4da121 fuzz: Avoid timeout in utxo_total_supply (MarcoFalke)

Pull request description:

  Looks like for high block counts it may be better to mock the chain, otherwise a high limit will lead to fuzz input bloat and timeouts, see https://github.com/bitcoin/bitcoin/pull/17860#issuecomment-1538252773.

  It can be checked that the fuzz target can still find the CVE, see https://github.com/bitcoin/bitcoin/pull/17860#pullrequestreview-1410594057 with a diff of:

  ```diff
  diff --git a/src/consensus/tx_check.cpp b/src/consensus/tx_check.cpp
  index f949655909..6f4cfb5f51 100644
  --- a/src/consensus/tx_check.cpp
  +++ b/src/consensus/tx_check.cpp
  @@ -39,8 +39,6 @@ bool CheckTransaction(const CTransaction& tx, TxValidationState& state)
       // the underlying coins database.
       std::set<COutPoint> vInOutPoints;
       for (const auto& txin : tx.vin) {
  -        if (!vInOutPoints.insert(txin.prevout).second)
  -            return state.Invalid(TxValidationResult::TX_CONSENSUS, "bad-txns-inputs-duplicate");
       }

       if (tx.IsCoinBase())
  ```

  Also, fix a nit, see https://github.com/bitcoin/bitcoin/pull/17860#discussion_r1186451948

ACKs for top commit:
  dergoegge:
    ACK fafb4da121

Tree-SHA512: a28fe9cd6ebb4c9bed5a5b35be76c1c436a87586c8fc3b3c4c8559a4a77ac08098324370da421d794c99579882c0872b6b29415de47ade6a05a08504a3d494c4
2023-05-31 11:24:57 +01:00
fanquake
1b8b28d83b Merge bitcoin/bitcoin#27786: fuzz: fix wallet notifications.cpp
a10f032115 fuzz: fix wallet notifications.cpp (furszy)

Pull request description:

  Fixing https://github.com/bitcoin/bitcoin/pull/27469#issuecomment-1568815816.

  As the fuzzing test requires all blocks to be scanned by the wallet
  (because it is asserting the wallet balance at the end), we need to
  ensure that no blocks are skipped by the recently added wallet
  birth time functionality.

  This just means setting the chain accumulated time to the maximum
  value, so the wallet birth time is always below it, and the block is
  always processed by the wallet.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK a10f032115, thanks

Tree-SHA512: c9b38c52917cc36674415470752625b8161fc6b878b0b87d6926b462ba9666be3c225d396604c7e944a4c268fc35fc624807777aa0ed94bddbe18d8f8436de3c
2023-05-31 11:10:57 +01:00
fanquake
433f17bc3f Merge bitcoin/bitcoin#27784: test: fix intermittent error in getblockfrompeer.py
9fe9074266 test: add block sync to getblockfrompeer.py (Martin Zumsande)

Pull request description:

  This adds an additional `sync_blocks` call, fixing an intermittent error caused by blocks arriving out of order due to how compact block relay may revert to headers processing when the tip hasn't caught up, and resulting in slightly different pruning behavior.
  Making sure that all blocks from the previous tests are synced before generating more blocks makes this impossible.

  See https://github.com/bitcoin/bitcoin/issues/27749#issuecomment-1566354933 and https://github.com/bitcoin/bitcoin/issues/27749#issuecomment-1566554075 for a more detailed analysis.

  #27770 is a more long-term approach to avoid having to deal with magic pruneheight numbers in the first place, but that PR introduces a new RPC and needs more discussion.

  Fixes #27749.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 9fe9074266
  theStack:
    ACK 9fe9074266

Tree-SHA512: f3de1ea68725429aeef448c351ea812b805fa216912b112d7db9aceeddb1f2381b705c2577734b0d308e78ec5e0c4d26dc65fc2171f6e21f13061fc71d48216c
2023-05-31 10:33:42 +01:00
fanquake
08722f20c2 Merge bitcoin/bitcoin#27777: ci: Prune dangling images on RESTART_CI_DOCKER_BEFORE_RUN
fa123077bc ci: Use podman for persistent workers (MarcoFalke)
fa9c65a74c ci: Prune dangling images on RESTART_CI_DOCKER_BEFORE_RUN (MarcoFalke)

Pull request description:

  This should prevent the persistent workers from running out of disk space. Containers are already removed, but not images. This is required since CI images are built and cached.

ACKs for top commit:
  hebasto:
    ACK fa123077bc

Tree-SHA512: 07c4faec57d659d1762e4e6d776c882ee48d4bac6ce6d438d56d9ab13277be3e39d6aa38816165a5a3e0938ac5d47674ee2921b6e115a4bb54e3e4910b34c4b6
2023-05-31 09:56:31 +01:00
furszy
a10f032115 fuzz: fix wallet notifications.cpp
As the fuzzer test requires all blocks to be
scanned by the wallet (because it is asserting
the wallet balance at the end), we need to
ensure that no blocks are skipped by the recently
added wallet birth time functionality.

This just means setting the chain accumulated time
to the maximum value, so the wallet birth time is
always below it, and the block is always processed.
2023-05-30 17:46:46 -03:00
Martin Zumsande
9fe9074266 test: add block sync to getblockfrompeer.py
This fixes an intermittent error, caused by blocks arriving
out of order due to how compact block relay may revert to headers
processing when the tip hasn't caught up, and resulting in slightly
different pruning behavior.

Making sure that all blocks from the previous tests are synced before
generating more blocks makes this impossible.
See Issue #27749 for more details.
2023-05-30 14:17:17 -04:00
Andrew Chow
71300489af Merge bitcoin/bitcoin#26261: p2p: cleanup LookupIntern, Lookup and LookupHost
5c832c3820 p2p, refactor: return `std::optional<CNetAddr>` in `LookupHost` (brunoerg)
34bcdfc6a6 p2p, refactor: return vector/optional<CService> in `Lookup` (brunoerg)
7799eb125b p2p, refactor: return `std::vector<CNetAddr>` in `LookupHost` (brunoerg)
5c1774a563 p2p, refactor: return `std::vector<CNetAddr>` in `LookupIntern` (brunoerg)

Pull request description:

  Continuation of #26078.

  To improve readability instead of returning a bool and passing stuff by reference, this PR changes:

  - `LookupHost` to return `std::vector<CNetAddr>`
  - `LookupHost` to return `std::optional<CNetAddr>`
  - `Lookup` to return `std::vector<CService>`
  - `Lookup` to return `std::optional<CService>`.
  - `LookupIntern` to return `std::vector<CNetAddr>`

  As discussed in #26078, it would be better to avoid using `optional` in some cases, but for specific `Lookup` and `LookupHost` functions it's necessary to use `optional` to verify if they were able to catch some data from their overloaded function.

ACKs for top commit:
  achow101:
    ACK 5c832c3820
  stickies-v:
    re-ACK 5c832c3820 - just addressing two nits, no other changes
  theStack:
    re-ACK 5c832c3820

Tree-SHA512: ea346fdc54463999646269bd600cd4a1590ef958001d2f0fc2be608ca51e1b4365efccca76dd4972b023e12fcc6e67d226608b0df7beb901bdeadd19948df840
2023-05-30 11:39:59 -04:00
TheCharlatan
db77f87c63 scripted-diff: move settings to common namespace
-BEGIN VERIFY SCRIPT-
sed -i 's/namespace\ util/namespace\ common/g' src/common/settings.cpp src/common/settings.h
sed -i 's/util\:\:GetSetting/common\:\:GetSetting/g' $( git grep -l 'util\:\:GetSetting')
sed -i 's/util\:\:Setting/common\:\:Setting/g' $( git grep -l 'util\:\:Setting')
sed -i 's/util\:\:FindKey/common\:\:FindKey/g' $( git grep -l 'util\:\:FindKey')
sed -i 's/util\:\:ReadSettings/common\:\:ReadSettings/g' $( git grep -l 'util\:\:ReadSettings')
sed -i 's/util\:\:WriteSettings/common\:\:WriteSettings/g' $( git grep -l 'util\:\:WriteSettings')
-END VERIFY SCRIPT-
2023-05-30 17:26:51 +02:00
TheCharlatan
c27e4bdc35 move-only: Move settings to the common library
The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from code that is not strictly required by it.
The settings code belongs into the common library and namespace, since
the kernel library should not depend on it. See doc/design/libraries.md
for more information on this rationale.

Changing the namespace of the moved functions is scripted in the
following commit.
2023-05-30 17:26:47 +02:00
fanquake
05ec664632 Merge bitcoin/bitcoin#27666: wallet, bench: Move commonly used functions to their own file and fix a bug
7379a54ec4 bench: Remove incorrect LoadWallet call in WalletBalance (Andrew Chow)
846b2fe67e tests: Move ADDRESS_BCRT1_UNSPENDABLE to wallet/test/util.h (Andrew Chow)
c61d3f02f5 tests, bench: Consolidate {Test,Bench}Un/LoadWallet helper (Andrew Chow)

Pull request description:

  I have a few PRs and branches that use these two commits, probably makes sense to split them into a separate PR to be merged sooner.

  The first commit contains some things that end up being commonly used in new wallet benchmarks. These are moved into `wallet_common.{h/cpp}`.

  The second commit contains a bugfix for the wallet_balance benchmark where it calls `LoadWallet` in the wrong place. It's unnecessary to call that function in this benchmark. Although this does not cause any issues currently, it ends up causing issues in some PRs and branches that I'm working on.

ACKs for top commit:
  Sjors:
    utACK 7379a54ec4
  furszy:
    ACK 7379a54

Tree-SHA512: 47773887a16c69ac7121c699d3446a8c399bd792a6a31714998b7b7a19fea179c6d3b29cb898b04397b2962c1b4120d57009352b8460b8283e188d4cb480c9ba
2023-05-30 16:20:47 +01:00
TheCharlatan
c2dae5d7d8 kernel: Remove chainparams, chainparamsbase, args, settings from kernel library 2023-05-30 17:15:22 +02:00
TheCharlatan
05870b1c92 refactor: Remove gArgs access from validation.cpp
This is done in the context of the libbitcoinkernel project, wherein
reliance of libbitcoinkernel code on the global gArgs is incrementally
removed.
2023-05-30 16:52:50 +02:00
TheCharlatan
8789b11114 refactor: Add path argument to FindSnapshotChainstateDir
Remove access to the global gArgs for getting the directory in
utxo_snapshot.

This is done in the context of the libbitcoinkernel project, wherein
reliance of libbitcoinkernel code on the global gArgs is incrementally
removed.
2023-05-30 16:52:48 +02:00
TheCharlatan
ef95be334f refactor: Add stop_at_height option in ChainstateManager
Remove access to the global gArgs for the stopatheight argument and
replace it by adding a field to the existing ChainstateManager Options
struct.

This should eventually allow users of the ChainstateManager to not rely
on the global gArgs and instead pass in their own options.
2023-05-30 16:52:47 +02:00
fanquake
214f8f18b3 Merge bitcoin/bitcoin#27774: refactor: Add [[nodiscard]] where ignoring a Result return type is an error
fa5680b752 fix includes for touched header files (iwyu) (MarcoFalke)
dddde27f6f Add [[nodiscard]] where ignoring a Result return type is an error (MarcoFalke)

Pull request description:

  Only add it for those where it is an error to ignore. Also, fix the gcc compile warning https://github.com/bitcoin/bitcoin/pull/25977#issuecomment-1564350880. Also, fix iwyu for touched header files.

ACKs for top commit:
  TheCharlatan:
    ACK fa5680b752
  stickies-v:
    ACK fa5680b752

Tree-SHA512: c3509103bfeae246e2cf565bc561fcd68d8118515bac5431ba5304c3a63c8254b9c4f40e268b6f6d6b79405675c5a960db9b4eb3bdd14aedca333dc1c9e76415
2023-05-30 15:32:19 +01:00
fanquake
9564f98fee Merge bitcoin/bitcoin#27636: kernel: Remove util/system from kernel library, interface_ui from validation.
7d3b35004b refactor: Move system from util to common library (TheCharlatan)
7eee356c0a refactor: Split util::AnyPtr into its own file (TheCharlatan)
44de325d95 refactor: Split util::insert into its own file (TheCharlatan)
9ec5da36b6 refactor: Move ScheduleBatchPriority to its own file (TheCharlatan)
f871c69191 kernel: Add warning method to notifications (TheCharlatan)
4452707ede kernel: Add progress method to notifications (TheCharlatan)
84d71457e7 kernel: Add headerTip method to notifications (TheCharlatan)
447761c822 kernel: Add notification interface (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/27587 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel".

  ---

  It removes the kernel library's dependency on `util/system` and `interface_ui`. `util/system` contains networking and shell-related code that should not be part of the kernel library. The following pull requests prepared `util/system` for this final step: https://github.com/bitcoin/bitcoin/pull/27419 https://github.com/bitcoin/bitcoin/pull/27254 https://github.com/bitcoin/bitcoin/pull/27238.

  `interface_ui` defines functions for a more general node interface and has a dependency on `boost/signals2`. After applying the patches from this pull request, the kernel's reliance on boost is down to `boost::multiindex`.

  The approach implemented here introduces some indirection, which makes the code a bit harder to read. Any suggestions for improving or reworking this pull request to make it more concise, or even reworking it into a more proper interface, are appreciated.

ACKs for top commit:
  MarcoFalke:
    re-ACK 7d3b35004b (no change) 🎋
  stickies-v:
    Code Review ACK 7d3b35004b
  hebasto:
    re-ACK 7d3b35004b, only last two commits dropped since my [recent](https://github.com/bitcoin/bitcoin/pull/27636#pullrequestreview-1435394620) review.

Tree-SHA512: c8cfc698dc9d78e20191c444708f2d957501229abe95e5806106d1126fb9c5fbcee686fb55645658c0107ce71f10646f37a2fdf7fde16bbf22cbf1ac885dd08d
2023-05-30 14:57:22 +01:00
MarcoFalke
fafb4da121 fuzz: Avoid timeout in utxo_total_supply 2023-05-30 14:17:20 +02:00
MarcoFalke
fa3ab45203 ci: Enable float-divide-by-zero check 2023-05-30 12:01:38 +02:00
fanquake
f467b28ac3 Merge bitcoin/bitcoin#27673: log: don't log total disk read time in ConnectTip bench
bc862fad29 ConnectTip: don't log total disk read time in bench (Sjors Provoost)

Pull request description:

  The " Load block from disk" log introduced in #24216 incorrectly assumed `num_blocks_total` would be greater than 0. This is not guaranteed until the `ConnectBlock` call right below it.

  The total and average metric is not very useful because it does not distinguish between blocks read from disk and those loaded from memory. So rather than fixing the divide by zero issue, we just drop the metric.

  Fixes #27635

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bc862fad29 🐓
  willcl-ark:
    tACK bc862fad29

Tree-SHA512: ff52ff8a8a93f1c82071ca84c57ce5839e14271943393deac0aa5555d63383708777ed96e7226be6dd71b63ed07dc27bad1634ee848e88e4d0b95d511a8267e7
2023-05-30 10:49:14 +01:00
MarcoFalke
fa123077bc ci: Use podman for persistent workers 2023-05-30 10:27:23 +02:00
MarcoFalke
fa9c65a74c ci: Prune dangling images on RESTART_CI_DOCKER_BEFORE_RUN 2023-05-30 08:51:53 +02:00
fanquake
5763b232e6 ci: return to using Ubuntu 22.04 in MSAN jobs
We no-longer need to use 23.04, now that we aren't installing clang-16
and friends.
2023-05-29 17:20:50 +01:00
fanquake
d3cbcbf626 ci: compile clang and compiler-rt in MSAN jobs
This works around https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1005341.
2023-05-29 17:20:50 +01:00
fanquake
796bd1d0d1 ci: use LLVM 16.0.4 in MSAN jobs 2023-05-29 17:20:49 +01:00
fanquake
883bc9f561 ci: remove extra CC & CXX from MSAN jobs
This is passed through from depends.
2023-05-29 17:20:47 +01:00
fanquake
2d4f4b8f29 ci: standardize custom libc++ usage in MSAN jobs
Use `-isystem` & `-nostd*` flags, which is the preferred way to use a
custom libc++ (ours is libc++ build with MSAN) with Clang, as opposed to
our current ad-hoc flags.

See: https://releases.llvm.org/16.0.0/projects/libcxx/docs/UsingLibcxx.html#using-a-custom-built-libc
for more info.
2023-05-29 17:19:42 +01:00
fanquake
6cf47a8f44 Merge bitcoin/bitcoin#27507: lint: stop ignoring LIEF imports
015cc5e588 lint: stop ignoring LIEF imports (fanquake)

Pull request description:

  Type stubs are now available as of 0.13.0.
  See https://github.com/lief-project/LIEF/issues/650.

ACKs for top commit:
  TheCharlatan:
    ACK 015cc5e588

Tree-SHA512: ebb754f293c2a61a0ef64c3552f7c700ceb3054b50fd3f1573e4a9e87773ddeba47bd9875f6ab055043012dbc20aeb71e4d76cd3da535c76651dfb1fbfc66e89
2023-05-29 17:11:31 +01:00
fanquake
fb4f047686 Merge bitcoin/bitcoin#27724: build: disable boost multi index safe mode in debug mode
59c8944749 build: disable boost multi index safe mode (willcl-ark)

Pull request description:

  Fixes #27586

  Disable boost multi index safe mode by default when configuring with
  --enable-debug.

  This option can cause transactions to take a long time to be accepted
  into the mempool under certain conditions; iterator destruction takes
  O(n) time vs O(1) as they are stored in a singly linked list. See
  27586 and the [boost docs](https://www.boost.org/doc/libs/1_58_0/boost/multi_index/detail/safe_mode.hpp) for more information.

  Re-enable it on the CI builds which previously had it enabled.

  Re-enable it on the msan fuzz task so that we have fuzz tasks testing
  with it enabled and disabled in this repo.

ACKs for top commit:
  hebasto:
    ~ACK 59c89447499bd9d6202269879555b8bc37373aa2~
  fanquake:
    ACK 59c8944749

Tree-SHA512: ed654f63dbebdd02e4414d1f81147d92a4d490dbb5a2e0376858e3129097645f3a2df45191d6b40c410a76e803b0d28796d1a01c1d2fd995b94e8b7eb3949027
2023-05-29 17:09:47 +01:00
fanquake
dfe658009d Merge bitcoin/bitcoin#27759: Fix #includes in src/wallet
1f97572b9c Fix `#include`s in `src/wallet` (Hennadii Stepanov)

Pull request description:

  This PR is a minimum required changes to fix https://github.com/bitcoin/bitcoin/pull/27571#discussion_r1195497290.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 1f97572b9c

Tree-SHA512: de885210076d23f3394c42ca50e6ae2470c0ae6523399a2fa3ebb7c06383bdacef9c26166fa19747200396bed796c8772165e24416eb30ed8edd024e3394b2fe
2023-05-29 16:33:14 +01:00
fanquake
769dd1e826 Merge bitcoin/bitcoin#25975: contrib/init: Better systemd integration
689a65d878 contrib/init: Better systemd integration (Carl Dong)

Pull request description:

  ```
  1. Make logs available to journalctl (systemd's logging system) by not
     specifying -daemonwait, which rightfully has its own set of stdout
     and stderr descriptors (a user invoking with -daemonwait on the
     command line should not see any logs). It makes more sense not to
     daemonize in the systemd context anyway.

  2. Make systemd aware of when bitcoind is started and in steady state by
     specifying -startupnotify='systemd-notify --ready' and Type=notify.
     NotifyAccess=all is necessary so that the spawned thread for
     startupnotify is allowed to inform systemd of bitcoind's readiness.

     Note that NotifyAccess=exec won't work because it only allows
     sd_notify readiness signalling from Exec*= declarations in the
     .service file.

  Note that we currently don't allow multiple startupnotify commands, but
  users can override it in systemd via:

    # systemctl edit bitcoind

  By specifying something like:

    [Service]
    ExecStart=/usr/bin/bitcoind -pid=/run/bitcoind/bitcoind.pid \
                                -conf=/etc/bitcoin/bitcoin.conf \
                                -datadir=/var/lib/bitcoind \
                                -startupnotify='systemd-notify --ready; mycommandhere'
  ```

ACKs for top commit:
  real-or-random:
    ACK 689a65d878 tested this service file with 25.0

Tree-SHA512: 9a52ad5cf25886c0d8dabc986d8920602a056db25875b5edd910b387043b78bb78c76d6df82e6e322e3be3bfd5c35c80721cbc8308cec946060bd7586820e9c6
2023-05-29 13:43:42 +01:00
MarcoFalke
fa5680b752 fix includes for touched header files (iwyu) 2023-05-29 13:26:31 +02:00
MarcoFalke
dddde27f6f Add [[nodiscard]] where ignoring a Result return type is an error 2023-05-29 13:12:45 +02:00
fanquake
a2e111b8a3 Merge bitcoin/bitcoin#27765: test: Throw error when -signetchallenge is non-hex
fa6b11a556 test: Throw error when -signetchallenge is non-hex (MarcoFalke)

Pull request description:

  Instead of silently parsing non-hex to an empty challenge, throw an error.

  Also, add missing includes while touching the file.

ACKs for top commit:
  kevkevinpal:
    ACK [fa6b11a](fa6b11a556)
  kallewoof:
    ACK fa6b11a
  TheCharlatan:
    Nice, ACK fa6b11a556

Tree-SHA512: 018ebbbf819ba7cdf0c6dd294fdfaa5ddb81b87058a8b9c57b96066d5b07e1656fd78f18e3cef375aebefa191fa515c2c70bc764880fa05f98f526334431a616
2023-05-29 10:48:53 +01:00
fanquake
b5ed656c3b Merge bitcoin/bitcoin#27739: ci: Add missing set -e to 01_base_install.sh
fa12558d21 ci: Avoid leaking HOME var into CI pod (MarcoFalke)
aaaa432603 ci: Remove "default" test env (MarcoFalke)
fa7a87bc7c ci: Add missing set -e to 01_base_install.sh (MarcoFalke)

Pull request description:

  Otherwise errors are silently ignored

ACKs for top commit:
  TheCharlatan:
    ACK [fa12558](fa12558d21)
  hebasto:
    ACK fa12558d21

Tree-SHA512: dbf3f16302c83973b78f3a5e7793090bc9ac44fdf20d51a26b30a99a97369971661e9aed1cd810d80d49d60009651ca0a8aeb2bdc24198a143bf4fff0ec89901
2023-05-29 10:34:08 +01:00
fanquake
015cc5e588 lint: stop ignoring LIEF imports
Type stubs are now available as of 0.13.0.
See https://github.com/lief-project/LIEF/issues/650.
2023-05-29 10:23:52 +01:00
MarcoFalke
fa12558d21 ci: Avoid leaking HOME var into CI pod
This will lead to a duplicate install, see https://github.com/bitcoin/bitcoin/pull/27739#issuecomment-1564122573
2023-05-29 09:16:43 +02:00
MarcoFalke
aaaa432603 ci: Remove "default" test env
It is unclear what the point is of maintaining a "default", the meaning
of which is unclear.
2023-05-29 09:16:21 +02:00
MarcoFalke
fa7a87bc7c ci: Add missing set -e to 01_base_install.sh
Also, set -x for easier debugging.

Also, do the same for ci/test/00_setup_env.sh
2023-05-29 09:16:10 +02:00
Carl Dong
689a65d878 contrib/init: Better systemd integration
1. Make logs available to journalctl (systemd's logging system) by not
   specifying -daemonwait, which rightfully has its own set of stdout
   and stderr descriptors (a user invoking with -daemonwait on the
   command line should not see any logs). It makes more sense not to
   daemonize in the systemd context anyway.

2. Make systemd aware of when bitcoind is started and in steady state by
   specifying -startupnotify='systemd-notify --ready' and Type=notify.
   NotifyAccess=all is necessary so that the spawned thread for
   startupnotify is allowed to inform systemd of bitcoind's readiness.

   Note that NotifyAccess=exec won't work because it only allows
   sd_notify readiness signalling from Exec*= declarations in the
   .service file.

3. Also make systemd aware of when bitcoind is stopping by specifying
   -shutdownnotify='systemd-notify --stopping'

Note that we currently don't allow multiple *notify commands, but users
can override it in systemd via:

  # systemctl edit bitcoind

By specifying something like:

  [Service]
  ExecStart=/usr/bin/bitcoind -pid=/run/bitcoind/bitcoind.pid \
                              -conf=/etc/bitcoin/bitcoin.conf \
                              -datadir=/var/lib/bitcoind \
                              -startupnotify='systemd-notify --ready; mystartupcommandhere' \
                              -shutdownnotify='systemd-notify --stopping; myshutdowncommandhere'
2023-05-28 13:10:30 -04:00
Andrew Chow
7d33ae755d Merge bitcoin/bitcoin#27145: wallet: when a block is disconnected, update transactions that are no longer conflicted
89df7987c2 Add wallets_conflicts (Antoine Riard)
dced203162 wallet, tests: mark unconflicted txs as inactive (ishaanam)
096487c4dc wallet: introduce generic recursive tx state updating function (ishaanam)

Pull request description:

  This implements a fix for #7315. Previously when a block was disconnected any transactions that were conflicting with transactions mined in that block were not updated to be marked as inactive. The fix implemented here is described on the [Bitcoin DevWiki](https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking#idea-refresh-conflicted). A test which tested the previous behavior has also been updated.

  Second attempt at #17543

ACKs for top commit:
  achow101:
    ACK 89df7987c2
  rajarshimaitra:
    tACK 89df7987c2.
  glozow:
    ACK 89df7987c2
  furszy:
    Tested ACK 89df7987

Tree-SHA512: 3133b151477e8818302fac29e96de30cd64c09a8fe3a7612074a34ba1a332e69148162e6cb3f1074d9d9c9bab5ef9996967d325d8c4c99ba42b5fe3b66a60546
2023-05-27 13:07:09 -04:00
fanquake
927b001502 Merge bitcoin/bitcoin#27766: fuzz: Change LIMIT_TO_MESSAGE_TYPE from a compile-time to a run-time setting
1111c9ac97 fuzz: Change LIMIT_TO_MESSAGE_TYPE from a compile-time to a run-time setting (MarcoFalke)

Pull request description:

  The `process_message_${msg_type}` fuzz targets have many issues:

  * In a context where each fuzz target must be a separate binary, this bloats the storage requirements by the number of message types.
  * The qa-assets repo for fuzz inputs also bloats, because each input in the type specific folder (`./process_message_${msg_type}`) is accompanied by a similar input in the general folder (`./process_message`) or a in another specific folder. The size seems to be ~3GB for the sum of all folders vs 0.3GB for the general folder.
  * Handling of different folders for each message type and one general folder for all message types (and unknown message types) is undocumented and unclear. Cross-pollination is encouraged, I guess, but who does it?
  * It is unclear if the fuzz target has any value at all, given that any bug that is found here should also be found by the `process_messages` fuzz target, and historically always has been? So maybe it can even be removed completely in the future?
  * (minor nit): When adding a new message type, the message type has to be added to this fuzz target as well.

  Fix all issues by turning the compile-time setting into a run-time setting, thus removing the extra executables and fuzz folders. The same approach is also taken by the `rpc` fuzz target.

  If someone wants to limit their fuzzing to a specific message type, they can still do it. For example,

  ```
  LIMIT_TO_MESSAGE_TYPE=inv FUZZ=process_message ./src/test/fuzz/fuzz

ACKs for top commit:
  dergoegge:
    ACK 1111c9ac97

Tree-SHA512: 9495538d9bc83b24671a44e9311a4e82ce5b2fa89e431e42772dcfa19f675a9fe2dd8cd3d3b15b154c8b7f400e57ee08a976d37e55a5425778ced0ee85fb984c
2023-05-27 10:23:21 +01:00
Andrew Chow
10c4a4613f Merge bitcoin/bitcoin#27469: wallet: improve IBD sync time by skipping block scanning prior birth time
82bb7831fa wallet: skip block scan if block was created before wallet birthday (furszy)
a082434d12 refactor: single method to append new spkm to the wallet (furszy)

Pull request description:

  During initial block download, the node's wallet(s) scans every arriving block looking for data that it owns.
  This process can be resource-intensive, as it involves sequentially scanning all transactions within each
  arriving block.

  To avoid wasting processing power, we can skip blocks that occurred before the wallet's creation time,
  since these blocks are guaranteed not to contain any relevant wallet data.

  This has direct implications (an speed improvement) on the underlying blockchain synchronization process
  as well. The reason is that the validation interface queue is limited to 10 tasks per time. This means that no
  more than 10 blocks can be waiting for the wallet(s) to be processed while we are synchronizing the chain
  (activating the best chain to be more precise).
  Which can be a bottleneck if blocks arrive and are processed faster from the network than what they are
  processed by the wallet(s).

  So, by skipping not relevant blocks in the wallet's IBD scanning process, we will also improve the chain
  synchronization time.

ACKs for top commit:
  ishaanam:
    re-ACK 82bb7831fa
  achow101:
    re-ACK 82bb7831fa
  pinheadmz:
    ACK 82bb7831fa

Tree-SHA512: 70158c9657f1fcc396badad2c4410b7b7f439466142640b31a9b1a8cea4555e45ea254e48043c9b27f783d5e4d24d91855f0d79d42f0484b8aa83cdbf3d6c50b
2023-05-26 21:35:28 -04:00
Amiti Uttarwar
cd8ef5b3e6 test: ensure addrman test is finite
Add a counter to ensure that the error case is bounded rather than leading to a
CI timeout
2023-05-26 15:47:55 -07:00
Amiti Uttarwar
b9f1e86f12 addrman: change asserts to Assumes
`Assume` is safer since the checks are non-fatal- errors in these functions
should provide feedback in debug builds, but do not need to deter further node
operations in production.
2023-05-26 15:47:55 -07:00
brunoerg
5c832c3820 p2p, refactor: return std::optional<CNetAddr> in LookupHost 2023-05-26 13:41:07 -03:00
brunoerg
34bcdfc6a6 p2p, refactor: return vector/optional<CService> in Lookup 2023-05-26 13:40:02 -03:00
brunoerg
7799eb125b p2p, refactor: return std::vector<CNetAddr> in LookupHost 2023-05-26 13:38:22 -03:00
brunoerg
5c1774a563 p2p, refactor: return std::vector<CNetAddr> in LookupIntern 2023-05-26 13:38:21 -03:00
fanquake
8b59231641 Merge bitcoin/bitcoin#27761: p2p: Log addresses of stalling peers
fb02a3cd1a p2p: Log addresses of stalling peers (Martin Zumsande)

Pull request description:

  This was suggested in #27705 by ArmchairCryptologist.
  It allows node operators that have the `-logips` option enabled to better identify potentially misbehaving peers and maybe ban them.
  This is especially helpful in case of inbound peers for which (dis)connections aren't logged per default, so it's impossible to use the debug log to connect their `nodeId` to an address unless the very noisy `net` debugging is enabled.
  In case of outbound peers for which the address is potentially logged when establishing the connection, this just adds some convenience.

ACKs for top commit:
  stratospher:
    tACK fb02a3c.
  jamesob:
    github ACK fb02a3cd1a
  0xB10C:
    Untested ACK fb02a3cd1a
  instagibbs:
    utACK fb02a3cd1a

Tree-SHA512: 2080f794c715bd36143405828b4b0e1be859095caf8f8a0c20dd2a4b64d192d78fee0fa350a2bb7c39848718332c4dd4d8edb2cc8d22095b65afe710591f7ccb
2023-05-26 17:12:28 +01:00
MarcoFalke
1111c9ac97 fuzz: Change LIMIT_TO_MESSAGE_TYPE from a compile-time to a run-time setting 2023-05-26 17:14:23 +02:00
fanquake
7794d9d93f Merge bitcoin/bitcoin#27735: test: Move test_chain_listunspent wallet check from mempool_packages to wallet_basic
ffffe622e9 test: Move test_chain_listunspent wallet check from mempool_packages to wallet_basic (MarcoFalke)

Pull request description:

  This fixes a bug.

  On master:

  ```
  $ ./test/functional/mempool_packages.py  --legacy-wallet
    File "./test/functional/mempool_packages.py", line 52, in run_test
      self.nodes[0].importaddress(self.wallet.get_address())
  test_framework.authproxy.JSONRPCException: Bech32m addresses cannot be imported into legacy wallets (-5)
  ```

  On this pull, all tests pass.

ACKs for top commit:
  glozow:
    ACK ffffe622e9, thanks for changing! Nice to remove wallet from another non-wallet test.

Tree-SHA512: 842c3b7c2e90285a155b8ed9924ef0c99f7773892be4f1847e5d7ece79c914ea5acee0d71de2ce46c354ee95fb74a03c20c0afb5e49c0b8e1c0ce406df963650
2023-05-26 14:53:23 +01:00
fanquake
66b08e7822 Merge bitcoin/bitcoin#27302: init: Error if ignored bitcoin.conf file is found
eefe56967b bugfix: Fix incorrect debug.log config file path (Ryan Ofsky)
3746f00be1 init: Error if ignored bitcoin.conf file is found (Ryan Ofsky)
398c3719b0 lint: Fix lint-format-strings false positives when format specifiers have argument positions (Ryan Ofsky)

Pull request description:

  Show an error on startup if a bitcoin datadir that is being used contains a `bitcoin.conf` file that is ignored. There are two cases where this could happen:

  - One case reported in [#27246 (comment)](https://github.com/bitcoin/bitcoin/issues/27246#issuecomment-1470006043) happens when a `bitcoin.conf` file in the default datadir (e.g. `$HOME/.bitcoin/bitcoin.conf`) has a `datadir=/path` line that sets different datadir containing a second `bitcoin.conf` file. Currently the second `bitcoin.conf` file is ignored with no warning.

  - Another way this could happen is if a `-conf=` command line argument points to a configuration file with a `datadir=/path` line and that path contains a `bitcoin.conf` file, which is currently ignored.

  This change only adds an error message and doesn't change anything about way settings are applied. It also doesn't trigger errors if there are redundant `-datadir` or `-conf` settings pointing at the same configuration file, only if they are pointing at different files and one file is being ignored.

ACKs for top commit:
  pinheadmz:
    re-ACK eefe56967b
  willcl-ark:
    re-ACK eefe56967b
  TheCharlatan:
    ACK eefe56967b

Tree-SHA512: 939a98a4b271b5263d64a2df3054c56fcde94784edf6f010d78693a371c38aa03138ae9cebb026b6164bbd898d8fd0845a61a454fd996e328fd7bcf51c580c2b
2023-05-26 13:33:42 +01:00
fanquake
4d13fe47be Merge bitcoin/bitcoin#25977: refactor: Replace std::optional<bilingual_str> with util::Result
8aa8f73adc refactor: Replace std::optional<bilingual_str> with util::Result (Ryan Ofsky)
5f49cb1bc8 util: Add void support to util::Result (MarcoFalke)

Pull request description:

  Replace uses of `std::optional<bilingual_str>` with `util::Result` as suggested https://github.com/bitcoin/bitcoin/pull/25648#discussion_r936311768, https://github.com/bitcoin/bitcoin/pull/27632#discussion_r1192007516, https://github.com/bitcoin/bitcoin/pull/27632#discussion_r1194858242, https://github.com/bitcoin/bitcoin/pull/24313#discussion_r1204047087

ACKs for top commit:
  MarcoFalke:
    very nice ACK 8aa8f73adc 🏏
  TheCharlatan:
    ACK 8aa8f73adc
  hebasto:
    ACK 8aa8f73adc, I have reviewed the code and it looks OK.

Tree-SHA512: 6c2f218bc445184ce93ec2b907e61666a05f26f2c9447f69fcb504aa8291b0c693d913f659dfd19813a9e24615546c72cbe2ca419218fd867ff0694c4a1b6a30
2023-05-26 13:09:26 +01:00
fanquake
aa6cc5bec9 Merge bitcoin/bitcoin#27751: doc: Add doc/release-notes/release-notes-25.0.md
034cb5ad4d doc: Fix broken link in release notes (MacrabFalke)
fab19a8ae3 doc: Fix typo in doc/release-process.md URL (MarcoFalke)
faaa97bb38 doc: Add doc/release-notes/release-notes-25.0.md (MarcoFalke)

Pull request description:

  Also, fix a typo in another doc.

ACKs for top commit:
  fanquake:
    ACK 034cb5ad4d

Tree-SHA512: f487fadcefdf0257ab6be1550faaad36825be551d6ec83abd32d69784ce537144714b3a023f4e13003f5f1b5e6a944f6d63f278cdeca2299139c9043af1c726f
2023-05-26 11:10:43 +01:00
MacrabFalke
034cb5ad4d doc: Fix broken link in release notes
Also, add missing unit "bytes"

Co-authored-by: stickies-v <69010457+stickies-v@users.noreply.github.com>
2023-05-26 09:47:15 +02:00
MarcoFalke
ffffe622e9 test: Move test_chain_listunspent wallet check from mempool_packages to wallet_basic 2023-05-26 09:02:55 +02:00
Martin Zumsande
fb02a3cd1a p2p: Log addresses of stalling peers
This allows node operators that have the -logips option enabled
to better identify potentially misbehaving peers and maybe
ban them.
2023-05-26 00:57:37 -04:00
Andrew Chow
7379a54ec4 bench: Remove incorrect LoadWallet call in WalletBalance
The WalletBalance benchmarks would incorrectly call LoadWallet after the
wallet has been setup. LoadWallet expects to be the first thing that is
called and for the CWallet to be in a fresh state. When it is not, it
results in bogus pointers which can cause segfaults during this
benchmark.
2023-05-25 14:40:42 -04:00
Andrew Chow
846b2fe67e tests: Move ADDRESS_BCRT1_UNSPENDABLE to wallet/test/util.h
This static address is usable for other wallet tests and benchmarks, so
make it available to them.
2023-05-25 14:40:42 -04:00
Andrew Chow
c61d3f02f5 tests, bench: Consolidate {Test,Bench}Un/LoadWallet helper
The wallet tests and benchmarks both had helper functions for loading
and unloading the wallet for the test that were almost identical.
These functions are consolidated and reused.
2023-05-25 14:40:26 -04:00
MarcoFalke
fa6b11a556 test: Throw error when -signetchallenge is non-hex 2023-05-25 19:24:05 +02:00
Hennadii Stepanov
1f97572b9c Fix #includes in src/wallet 2023-05-25 15:52:08 +01:00
fanquake
25202cace9 Merge bitcoin/bitcoin#27721: depends: remove redundant stdlib option
4fe5f3c467 depends: remove redundant stdlib option (Cory Fields)

Pull request description:

  Like #27628, this is another dependency of #21778, though it doesn't become obvious until used with a newer clang.

  This should be a no-op.

  Use of -stdlib++-isystem gets rid of any system c++ header include paths and negates the need for this option. In newer versions of clangs the combo produces an annoying warning that actually causes problems during configure.

ACKs for top commit:
  fanquake:
    ACK 4fe5f3c467

Tree-SHA512: 904072f2b13dffbbeab2bc9ff20a74969888502fa1ea35d9030784dd397c6751e31233e6ec7dc34e9fd42574950be733b25d6653c2a93e214cc5e4eef2e0133a
2023-05-25 15:36:23 +01:00
furszy
82bb7831fa wallet: skip block scan if block was created before wallet birthday
To avoid wasting processing power, we can skip blocks that occurred
before the wallet's creation time,  since these blocks are guaranteed
not to contain any relevant wallet data.

This has direct implications (an speed improvement) on the underlying
blockchain synchronization process as well.

The reason is that the validation interface queue is limited to
10 tasks per time. This means that no more than 10 blocks can be
waiting for the wallet(s) to be processed while we are synchronizing
the chain (activating the best chain to be more precise).
Which can be a bottleneck if blocks arrive and are processed faster
from the network than what they are  processed by the wallet(s).
2023-05-25 10:45:38 -03:00
furszy
a082434d12 refactor: single method to append new spkm to the wallet 2023-05-25 10:38:20 -03:00
MarcoFalke
fab19a8ae3 doc: Fix typo in doc/release-process.md URL 2023-05-25 13:17:17 +02:00
MarcoFalke
faaa97bb38 doc: Add doc/release-notes/release-notes-25.0.md 2023-05-25 13:16:44 +02:00
fanquake
9d098af5a9 Merge bitcoin/bitcoin#27747: rpc: Use 'byte'/'bytes' for bech32(m) validation error message
3d0a5c37e9 use 'byte'/'bytes' for bech32(m) validation error (Reese Russell)

Pull request description:

  This PR rectifies a linguistic inconsistency found in merged PR [27727](https://github.com/bitcoin/bitcoin/pull/27727). It addresses the improper usage of the term 'byte' in error reports. As it stands, PR [27727](https://github.com/bitcoin/bitcoin/pull/27727) exclusively utilizes 'byte' in error messages, regardless of the context, as demonstrated below:

  Currently: ```Invalid Bech32 v0 address program size (16 byte), per BIP141```

  This modification enhances the accuracy of error reporting in most scenarios users are likely to encounter by checking for a plural or singular number of bytes.

  This PR

  **16 Bytes program size error** :

  ```
  (
      "BC1QR508D6QEJXTDG4Y5R3ZARVARYV98GJ9P",
      "Invalid Bech32 v0 address program size (16 bytes), per BIP141",
      [],
  )
  ```

  **1 Byte program size error**

  ```
  (
      "bc1pw5dgrnzv",
      "Invalid Bech32 address program size (1 byte)",
      []
  ),
  ```
  Thank you

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 3d0a5c37e9

Tree-SHA512: 55069194a6a33a37559cf14b59b6ac05b1160f57f14d1415aef8e76c916c7c7f343310916ae85b3fa895937802449c1dddb2f653340d0f39203f06aee10f6fce
2023-05-25 12:02:49 +01:00
fanquake
e43432086a Merge bitcoin/bitcoin#27743: p2p: Unconditionally return when compact block status == READ_STATUS_FAILED
d972695797 Unconditionally return when compact block status == READ_STATUS_FAILED (Greg Sanders)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/pull/27626#discussion_r1204491894 which would have resulted in wasted bandwidth every once in a while.

ACKs for top commit:
  Sjors:
    utACK d972695797
  mzumsande:
    ACK d972695797

Tree-SHA512: 2483cae03093fc61b36279bbc455829822258703ae608ff2797a20449410b12bfe8d40de63c02fc79ed4ef4d91b34c63281b71b81a1c691cab9647b65761586f
2023-05-25 10:05:55 +01:00
Reese Russell
3d0a5c37e9 use 'byte'/'bytes' for bech32(m) validation error
changed from std::string -> std::string_view

applied snake case to byteStr -> byte_str
2023-05-25 06:30:10 +00:00
Amiti Uttarwar
768770771f doc: update Select function description
Capture potential performance slow down for `Select` by network & clarify
return values.
2023-05-24 12:03:18 -07:00
Amiti Uttarwar
2b6bd12eea refactor: de-duplicate lookups
retain the values needed to prevent redundant node lookups
2023-05-24 11:39:31 -07:00
Greg Sanders
d972695797 Unconditionally return when compact block status == READ_STATUS_FAILED 2023-05-24 13:59:49 -04:00
Andrew Chow
a13f3746dc Merge bitcoin/bitcoin#27727: rpc: Fix invalid bech32 address handling
eeee55f928 rpc: Fix invalid bech32 handling (MarcoFalke)

Pull request description:

  Currently the handling of invalid bech32(m) addresses over RPC has many issues:

  * No error for invalid addresses is reported, leading to internal bugs via `CHECK_NONFATAL`, see https://github.com/bitcoin/bitcoin/issues/27723
  * The error messages use "data size" (the meaning of which is unclear to the user, because the witness program data and bech32 section data are related but different) when they mean "program size"

  Fix all issues. Also, use the BIP 173 and BIP 350 test vectors.

ACKs for top commit:
  achow101:
    ACK eeee55f928
  brunoerg:
    crACK eeee55f928

Tree-SHA512: c8639ee49e2a54b740b72d66bc4a40352dd553a6e3220dea9f94e48e33124f21f597a2817cb405d0a4c88d21df1013c0a4877a01370a2d326aa2cff1f9c381a8
2023-05-24 12:10:55 -04:00
Ryan Ofsky
8aa8f73adc refactor: Replace std::optional<bilingual_str> with util::Result 2023-05-24 08:55:47 -04:00
MarcoFalke
5f49cb1bc8 util: Add void support to util::Result
A minimal (but hacky) way to add support for void to Result
originally posted https://github.com/bitcoin/bitcoin/pull/27632#discussion_r1195604095
2023-05-24 08:55:47 -04:00
fanquake
51c050787f Merge bitcoin/bitcoin#27626: Parallel compact block downloads, take 3
d7f359b35e Add tests for parallel compact block downloads (Greg Sanders)
03423f8bd1 Support up to 3 parallel compact block txn fetchings (Greg Sanders)
13f9b20b4c Only request full blocks from the peer we thought had the block in-flight (Greg Sanders)
cce96182ba Convert mapBlocksInFlight to a multimap (Greg Sanders)
a90595478d Remove nBlocksInFlight (Greg Sanders)
86cff8bf18 alias BlockDownloadMap for mapBlocksInFlight (Greg Sanders)

Pull request description:

  This is an attempt at mitigating https://github.com/bitcoin/bitcoin/issues/25258 , which is a revival of https://github.com/bitcoin/bitcoin/pull/10984, which is a revival of https://github.com/bitcoin/bitcoin/pull/9447.

  This PR attempts to mitigate a single case, where high bandwidth peers can bail us out of a flakey
  peer not completing blocks for us. We allow up to 2 additional getblocktxns requests per unique block.
  This would hopefully allow the chance for an honest high bandwidth peer to hand us the transactions
  even if the first in flight peer stalls out.

  In contrast to previous effort:

  1) it will not help if subsequent peers send block headers only, so only high-bandwidth peers this time. See: https://github.com/bitcoin/bitcoin/pull/10984/files#diff-6875de769e90cec84d2e8a9c1b962cdbcda44d870d42e4215827e599e11e90e3R1411
  2)  `MAX_GETBLOCKTXN_TXN_AFTER_FIRST_IN_FLIGHT` is removed, in favor of aiding recovery during turbulent mempools
  3) We require one of the 3 block fetching slots to be an outbound peer. This can be the original offering peer, or subsequent compact blocks given by high bandwidth peers.

ACKs for top commit:
  sdaftuar:
    ACK d7f359b35e
  mzumsande:
    Code Review ACK d7f359b35e

Tree-SHA512: 54980eac179e30f12a0bd49df147b2c3d63cd8f9401abb23c7baf02f76eeb59f2cfaaa155227990d0d39384de9fa38663f88774e891600a3837ae927f04f0db3
2023-05-24 10:09:09 +01:00
Andrew Chow
3132ec64d9 Merge bitcoin/bitcoin#27177: test: fix intermittent issue in feature_bip68_sequence
272eb55616 test: fix `include_immature_coinbase` logic in `get_utxos` (brunoerg)
a951c34f17 test: fix `interface_usdt_mempool` by mining a block after each test (brunoerg)
1557bf1196 test: fix mature utxos addition to wallet in `mempool_package_limits` (brunoerg)
60ced9007d test: fix intermittent issue in `feature_bip68_sequence` (brunoerg)

Pull request description:

  Fixes #27129

  To avoid `bad-txns-premature-spend-of-coinbase` error,
  when getting a utxo (using `get_utxo`) to create a new
  transaction `get_utxo` shouldn't return (if possible)
  by default immature coinbase.

ACKs for top commit:
  achow101:
    ACK 272eb55616
  pinheadmz:
    re-ACK 272eb55616

Tree-SHA512: eae821c7833bf084d8b907c94876ed010a7925d2177c3013a0c61b69d9571df006da83397a19487d93b0d1fa415951152f0b8ad0de2a55d86c39f6917934f050
2023-05-23 15:48:07 -04:00
Greg Sanders
d7f359b35e Add tests for parallel compact block downloads 2023-05-23 13:07:49 -04:00
Greg Sanders
03423f8bd1 Support up to 3 parallel compact block txn fetchings
A single outbound slot is required, so if the first two slots
are taken by inbound in-flights, the node will reject additional
unless they are coming from outbound.

This means in the case where a fast sybil peer is attempting to
stall out a node, a single high bandwidth outbound peer can
mitigate the attack.
2023-05-23 13:07:49 -04:00
MarcoFalke
eeee55f928 rpc: Fix invalid bech32 handling 2023-05-23 15:10:00 +02:00
willcl-ark
59c8944749 build: disable boost multi index safe mode
Disable boost multi index safe mode by default when configuring with
--enable-debug.

This option can cause transactions to take a long time to be accepted
into the mempool under certain conditions; iterator destruction takes
O(n) time vs O(1) as they are stored in a singly linked list. See
27586 for more information.

Re-enable it on the CI builds which previously had it enabled.

Re-enable it on the msan fuzz target so that we have fuzz tasks testing
with it enabeld and disabled in this repo.
2023-05-23 13:44:07 +01:00
fanquake
5ef2c1ee7a Merge bitcoin/bitcoin#27717: test: Make util/test_runner.py honor BITCOINUTIL and BITCOINTX
4f2f615d13 test: Make `util/test_runner.py` honor `BITCOINUTIL` and `BITCOINTX` (Hennadii Stepanov)

Pull request description:

  This PR is a continuation of changes to our testing frameworks (https://github.com/bitcoin/bitcoin/pull/27554, https://github.com/bitcoin/bitcoin/pull/27561) that allow them to work correctly in a multi-config build environment that is possible for [upcoming](https://github.com/bitcoin/bitcoin/pull/25797) CMake-based build system. That means that built for different configurations binaries (e.g., "Debug" and "Release") can coexist in separated directories.

  The commit has been pulled from https://github.com/hebasto/bitcoin/pull/15 and it seems [useful](https://github.com/hebasto/bitcoin/pull/15#discussion_r1200251404) by itself as:
  > I believe the rationale for allowing to drop in the executables via env var is to allow to test the guix-produced, or other third-party-produced executables...

  The current implementation of the `test/functional/test_framework/test_framework.py` script uses the same approach: 09351f51d2/test/functional/test_framework/test_framework.py (L231-L246)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4f2f615d13
  TheCharlatan:
    ACK 4f2f615d13
  stickies-v:
    ACK 4f2f615d13

Tree-SHA512: 99ee9a727b266700649d8f2ec528dfaeb04a1e48f0cb1d4eeaece65917165be647c10c4548429a9e8b30d094597f67e860c1db03ac689ebb409b223ce1b63aa9
2023-05-23 13:23:58 +01:00
Cory Fields
4fe5f3c467 depends: remove redundant stdlib option
Use of -stdlib++-isystem gets rid of any system c++ header include paths and
negates the need for this option. In newer versions of clangs the combo
produces a warning.
2023-05-22 16:02:44 +00:00
furszy
3126454dcf index: prevent race by calling 'CustomInit' prior setting 'synced' flag
The 'm_synced' flag enables 'BlockConnected' events to be processed by
the index. If we set the flag before calling 'CustomInit', we could be
dispatching a block connected event to an uninitialized index child
class.

e.g. BlockFilterIndex, initializes the next filter position
inside 'CustomInit'. So, if `CustomInit` is not called prior receiving
the block event, the index will use 'next_filter_position=0' which
overwrites the first filter in disk.
2023-05-22 12:56:16 -03:00
Andrew Chow
22139f6e83 Merge bitcoin/bitcoin#25796: rpc: add descriptorprocesspsbt rpc
1bce12acd3 test: add test for `descriptorprocesspsbt` RPC (ishaanam)
fb2a3a70e8 rpc: add descriptorprocesspsbt rpc (ishaanam)

Pull request description:

  This PR implements an RPC called `descriptorprocesspsbt`. This RPC is based off of `walletprocesspsbt`, but instead of interacting with the wallet to update, sign and finalize a psbt, it instead accepts an array of output descriptors and uses that information along with information from the mempool, txindex, and the utxo set to do so. `utxoupdatepsbt` also updates a psbt in this manner, but doesn't sign or finalize it. Because of this overlap, a helper function that is added in this PR is called by both `utxoupdatepsbt` and `descriptorprocesspsbt`. Whether or not the helper function signs a psbt is dictated by if the HidingSigningProvider passed to it contains any private information. There is also a test added in this PR for this new RPC that uses p2wsh, p2wpkh, and legacy outputs.
  Edit: see https://github.com/bitcoin/bitcoin/pull/25796#issuecomment-1228830963

ACKs for top commit:
  achow101:
    re-ACK 1bce12acd3
  instagibbs:
    reACK 1bce12acd3

Tree-SHA512: e1d0334739943e71f2ee68b4db7637ebe725da62e7aa4be071f71c7196d2a5970a31ece96d91e372d34454cde8509e95ab0eebd2c8edb94f7d5a781a84f8fc5d
2023-05-22 11:28:11 -04:00
fanquake
456701420b Merge bitcoin/bitcoin#27672: fuzz: Print error message when FUZZ is missing
fa1b3abc83 ci: Log qa-assets repo last commit (MarcoFalke)
fa22966f33 fuzz: Print error message when FUZZ is missing (MarcoFalke)

Pull request description:

  Some trivial UX improvements.

  * Change the exit code for `PRINT_ALL_FUZZ_TARGETS_AND_ABORT` and `WRITE_ALL_FUZZ_TARGETS_AND_ABORT` to `EXIT_SUCCESS` instead of `Aborted (core dumped)`.
  * Print readable error message when `FUZZ` is missing instead of `Aborted (core dumped)`.
  * Clarify that a fuzz target needs to be compiled into the executable.

ACKs for top commit:
  dergoegge:
    ACK fa1b3abc83

Tree-SHA512: 065ef8920449c64b3516f89a61cb397b505eccf531318c4f3830895d5ff6cd7ae2525cb857320481e3d0ed0b2f8a522cd8f7835e69f021241b6ec297a6102fc8
2023-05-22 12:55:18 +01:00
fanquake
09351f51d2 Merge bitcoin/bitcoin#27699: random: drop syscall wrapper usage for getrandom()
5228223e1f ci: remove MSAN getrandom syscall workaround (fanquake)
d5e06919db random: switch to using getrandom() directly (fanquake)
c2ba3f5b0c random: add [[maybe_unused]] to GetDevURandom (fanquake)
c13c97dbf8 random: getentropy on macOS does not need unistd.h (fanquake)

Pull request description:

  This requires a linux kernel of `3.17`+, which seems entirely
  reasonable. `3.17` went EOL in 2015, and the last supported `3.x` kernel
  (`3.16`) went EOL > 4 years ago, in 2020. For reference, the current
  oldest maintained kernel is `4.14` (released 2017, going EOL Jan 2024).

  Support for `getrandom()` (and `getentropy()`) was added to
  glibc `2.25` https://sourceware.org/legacy-ml/libc-alpha/2017-02/msg00079.html:
  > * The getentropy and getrandom functions, and the <sys/random.h> header
    file have been added.

  and we already require `2.27` or later.

  All that being said, I don't think you would encounter a current day (+~6 months from now)
  system, running with kernel headers older than 3.17 (released 2014) but also having a
  glibc of 2.27+ (released 2018)?

  Removing this (our only) use of `syscall()` also means we can drop a workaround in our MSAN jobs.
  If this is merged, I'll drop the [same workaround in oss-fuzz](25946a5448/projects/bitcoin-core/build.sh (L49-L56)).

ACKs for top commit:
  josibake:
    ACK 5228223e1f
  hebasto:
    ACK 5228223e1f, I've tested build system changes on Ubuntu 22.04 and macOS Monterey 12.6.6 (x86_64).

Tree-SHA512: cc978e08510c461b875ca8c08ae176b4519fa1108f0efd74dcb7474518945357e0184e54423282c9a496de195e4ddc3e221ee78623bd63e24c50cc86acdf32e2
2023-05-22 11:34:58 +01:00
Hennadii Stepanov
4f2f615d13 test: Make util/test_runner.py honor BITCOINUTIL and BITCOINTX
This change allows to drop in the executables via environment variables
in the same way as for functional tests.
2023-05-22 11:23:24 +01:00
fanquake
ad7819d2f8 Merge bitcoin/bitcoin#27670: guix: remove redundant glibc patches
3cfe366ec3 guix: remove redundant glibc patches (fanquake)

Pull request description:

  These should only be relevant for a glibc that is built as part of a Guix system, and should not be required for a glibc that is just being built to compile our binaries against. A x86_64 linux bitcoind produced with Guix using master vs this change has no difference. i.e:
  #### Comparing `bitcoind` & `../../../../../guix-build-d7700d3a2647/output/x86_64-linux-gnu/bitcoin-d7700d3a2647/bin/bitcoind`:

  ## strings --all --bytes=8 {}

  ```diff
  @@ -20311,15 +20311,15 @@
   This is experimental software.
   The source code is available from %s.
   Please contribute if you find %s useful. Visit %s for further information about the software.
   The %s developers
   The Bitcoin Core developers
   <https://bitcoincore.org/>
   Copyright (C) %i-%i
  -v25.99.0-gda0bf1d07639b0490791bbd6aec71bbea8aa2aThe %s developer<https://github.com/bitcoin/bitcDistributed under the MIT software license, see the accompanyingThis is experimeThe source code is available froPlease contribute if you find %s useful. Visit %s for further information about Copyright (C) %ibool BCLog::Logger::StartLogging()
  +v25.99.0-gd7700d3a26478d9b1648463c188648c7047b1cThe %s developer<https://github.com/bitcoin/bitcDistributed under the MIT software license, see the accompanyingThis is experimeThe source code is available froPlease contribute if you find %s useful. Visit %s for further information about Copyright (C) %ibool BCLog::Logger::StartLogging()
   std::string BCLog::Logger::LogLevelToStr(BCLog::Level) const
   std::string LogCategoryToStr(BCLog::LogFlags)
   void BCLog::Logger::LogPrintStr(const string&, const string&, const string&, int, BCLog::LogFlags, BCLog::Level)
   void BCLog::Logger::ShrinkDebugFile()
   Failed to shrink debug log file: fseek(...) failed
   logging.cpp
   m_buffering
  ```

  #### objdump --line-numbers --disassemble --demangle --reloc --no-show-raw-insn --section=.text {}

  ```diff
  @@ -1505889,15 +1505889,15 @@
   call   aa3380 <malloc@plt+0xa4edb0>
   mov    (%rsp),%rdx
   movdqa 0x465540(%rip),%xmm0
   mov    %rax,0x7a0559(%rip)
   lea    0x7a0552(%rip),%rsi
   lea    0x3957bb(%rip),%rdi
   mov    %rdx,0x7a0554(%rip)
  -mov    $0x3038,%edx
  +mov    $0x3036,%edx
   movups %xmm0,(%rax)
   movdqa 0x465524(%rip),%xmm0
   mov    %dx,0x30(%rax)
   mov    0x7a0529(%rip),%rdx
   movups %xmm0,0x10(%rax)
   movdqa 0x46551d(%rip),%xmm0
   movups %xmm0,0x20(%rax)
  ```

  #### readelf --wide --decompress --hex-dump=.rodata {}

  ```diff
  @@ -37238,17 +37238,17 @@
     0x00b73730 65202573 20646576 656c6f70 65727300 e %s developers.
     0x00b73740 54686520 42697463 6f696e20 436f7265 The Bitcoin Core
     0x00b73750 20646576 656c6f70 65727300 434f5059  developers.COPY
     0x00b73760 494e4700 3c687474 70733a2f 2f626974 ING.<https://bit
     0x00b73770 636f696e 636f7265 2e6f7267 2f3e0043 coincore.org/>.C
     0x00b73780 6f707972 69676874 20284329 2025692d opyright (C) %i-
     0x00b73790 25690053 61746f73 68690000 00000000 %i.Satoshi......
  -  0x00b737a0 7632352e 39392e30 2d676461 30626631 v25.99.0-gda0bf1
  -  0x00b737b0 64303736 33396230 34393037 39316262 d07639b0490791bb
  -  0x00b737c0 64366165 63373162 62656138 61613261 d6aec71bbea8aa2a
  +  0x00b737a0 7632352e 39392e30 2d676437 37303064 v25.99.0-gd7700d
  +  0x00b737b0 33613236 34373864 39623136 34383436 3a26478d9b164846
  +  0x00b737c0 33633138 38363438 63373034 37623163 3c188648c7047b1c
     0x00b737d0 54686520 25732064 6576656c 6f706572 The %s developer
     0x00b737e0 3c687474 70733a2f 2f676974 6875622e <https://github.
     0x00b737f0 636f6d2f 62697463 6f696e2f 62697463 com/bitcoin/bitc
     0x00b73800 44697374 72696275 74656420 756e6465 Distributed unde
     0x00b73810 72207468 65204d49 5420736f 66747761 r the MIT softwa
     0x00b73820 7265206c 6963656e 73652c20 73656520 re license, see
     0x00b73830 74686520 6163636f 6d70616e 79696e67 the accompanying
  ```

  #### readelf --wide --decompress --hex-dump=.gnu_debuglink {}

  ```diff
  @@ -1,5 +1,5 @@

   Hex dump of section '.gnu_debuglink':
     0x00000000 62697463 6f696e64 2e646267 00000000 bitcoind.dbg....
  -  0x00000010 6b6e8eda                            kn..
  +  0x00000010 345cb865                            4\.e
  ```

  Guix Build:
  ```bash
  3d180219536b4ae2b4ea012a2e2afc8dcc76a79a7f55a36418a6e5a83f5adf90  guix-build-3cfe366ec35e/output/aarch64-linux-gnu/SHA256SUMS.part
  c25fbd84b7791d5bd3cab36d26828bf2b1063fadc4e944096e65597b66aba867  guix-build-3cfe366ec35e/output/aarch64-linux-gnu/bitcoin-3cfe366ec35e-aarch64-linux-gnu-debug.tar.gz
  bee8bf6f100912a0548cee798abb1ee9ac1ee17c065259a2410950e71eb3ff13  guix-build-3cfe366ec35e/output/aarch64-linux-gnu/bitcoin-3cfe366ec35e-aarch64-linux-gnu.tar.gz
  caa17fa9ba8b731c903a96211b2c17e8a1e2600bd9df8abd79eac4a89bfff72d  guix-build-3cfe366ec35e/output/arm-linux-gnueabihf/SHA256SUMS.part
  27829fab271cca459e2d037c42ccbefbbbbb1eb4463d5895d5a40220d737ecd9  guix-build-3cfe366ec35e/output/arm-linux-gnueabihf/bitcoin-3cfe366ec35e-arm-linux-gnueabihf-debug.tar.gz
  6b3eba0d9518dce3a6b7d88a32ae2a5b5ab943126e2a105d4ee6a861d44bea6f  guix-build-3cfe366ec35e/output/arm-linux-gnueabihf/bitcoin-3cfe366ec35e-arm-linux-gnueabihf.tar.gz
  855ab932aa0cc6d583a0f0422b1373afd44bff244e0022f29ce45305e5c8e8e2  guix-build-3cfe366ec35e/output/arm64-apple-darwin/SHA256SUMS.part
  02aabfdfe730400550bfc01e45055f6bc5b643511f08e314634c06b462a00c9e  guix-build-3cfe366ec35e/output/arm64-apple-darwin/bitcoin-3cfe366ec35e-arm64-apple-darwin-unsigned.dmg
  310722826ba985c58d800135f7ba9c73489e138cbf1b84a50be4f13453918ce1  guix-build-3cfe366ec35e/output/arm64-apple-darwin/bitcoin-3cfe366ec35e-arm64-apple-darwin-unsigned.tar.gz
  52d4f1af1e2608da4fa28ed446301d5c516e492c760db03c05c2a421c0a64ab9  guix-build-3cfe366ec35e/output/arm64-apple-darwin/bitcoin-3cfe366ec35e-arm64-apple-darwin.tar.gz
  2055c29fcde9aba8274d3649ea0c34ef0dac207d9d6f6a76fd9df9b010cdd7a8  guix-build-3cfe366ec35e/output/dist-archive/bitcoin-3cfe366ec35e.tar.gz
  99feae7ee4bfaf818efe49fbc9de81575a1e087593059bd630da70f5c6b8a7c8  guix-build-3cfe366ec35e/output/powerpc64-linux-gnu/SHA256SUMS.part
  7a55a6287eec3cfe598378684293b077791da234f1d5fcfe6f368e42f8a52428  guix-build-3cfe366ec35e/output/powerpc64-linux-gnu/bitcoin-3cfe366ec35e-powerpc64-linux-gnu-debug.tar.gz
  c2ba080a26b4bbfa443113d0044d07b97cc08f55df6bec90f162232f3f934c58  guix-build-3cfe366ec35e/output/powerpc64-linux-gnu/bitcoin-3cfe366ec35e-powerpc64-linux-gnu.tar.gz
  a670349367e671e73317476795eb7317559bf171d3facdfc2086031eb9dca264  guix-build-3cfe366ec35e/output/powerpc64le-linux-gnu/SHA256SUMS.part
  f9c0683f515bfd072ad18a780ad35ace7e4f5529d5bd9fffc06490d55bac402e  guix-build-3cfe366ec35e/output/powerpc64le-linux-gnu/bitcoin-3cfe366ec35e-powerpc64le-linux-gnu-debug.tar.gz
  608cce8d989b6cbfa723d57744a7e0ceac6a8668b12b4e223fe3de7833fe73ac  guix-build-3cfe366ec35e/output/powerpc64le-linux-gnu/bitcoin-3cfe366ec35e-powerpc64le-linux-gnu.tar.gz
  bbdebd22afc49c66c70738f68e3beea363c4a03701ccbb729d6f0eb0a0eaf150  guix-build-3cfe366ec35e/output/riscv64-linux-gnu/SHA256SUMS.part
  a84871c91a9b9d3423e9b86ffd46eb926672a1a88a3a3df1a5e8288a1fe6d98b  guix-build-3cfe366ec35e/output/riscv64-linux-gnu/bitcoin-3cfe366ec35e-riscv64-linux-gnu-debug.tar.gz
  21a89eb023113398bc1968284cbea86c6630284cb09325b9cee9669348206683  guix-build-3cfe366ec35e/output/riscv64-linux-gnu/bitcoin-3cfe366ec35e-riscv64-linux-gnu.tar.gz
  10f4ef77a97420490bc4494797d0acf8278f5bd4998b6c32881e611cc2faf237  guix-build-3cfe366ec35e/output/x86_64-apple-darwin/SHA256SUMS.part
  6e47a3676e76cd7175a08b6da81dcf7186849aba7c2ee95f12e998fdf1d4596d  guix-build-3cfe366ec35e/output/x86_64-apple-darwin/bitcoin-3cfe366ec35e-x86_64-apple-darwin-unsigned.dmg
  9ca8cd648a464e4e0bef107e23876d4588866eb12b844484a16fe93e4cd2f3b3  guix-build-3cfe366ec35e/output/x86_64-apple-darwin/bitcoin-3cfe366ec35e-x86_64-apple-darwin-unsigned.tar.gz
  89c7a5040683b63f58667f4eea6827af2874fc0962ddba3a158ad3aa78b8a407  guix-build-3cfe366ec35e/output/x86_64-apple-darwin/bitcoin-3cfe366ec35e-x86_64-apple-darwin.tar.gz
  1577c7f6c5eb7cb073c0ba32cfe7347df5aeaf62508d0ba1936506b1cb8a739e  guix-build-3cfe366ec35e/output/x86_64-linux-gnu/SHA256SUMS.part
  8703d39ce218216ee43502e030d3b3fbe6a00bdab82e8cd0706fa597fc6e11b7  guix-build-3cfe366ec35e/output/x86_64-linux-gnu/bitcoin-3cfe366ec35e-x86_64-linux-gnu-debug.tar.gz
  3e4d44d3cddfe2e34c12f55a704f791834385e1a867856e8a1c05f4f4fb3482a  guix-build-3cfe366ec35e/output/x86_64-linux-gnu/bitcoin-3cfe366ec35e-x86_64-linux-gnu.tar.gz
  87186fbcc7f0580ef3a347603c868f96bba31a987cad86991fa79b740d41f654  guix-build-3cfe366ec35e/output/x86_64-w64-mingw32/SHA256SUMS.part
  ceefefe1eb1d518f1534e0e3d51347332874016ce6adeba691fbbfc0b561437a  guix-build-3cfe366ec35e/output/x86_64-w64-mingw32/bitcoin-3cfe366ec35e-win64-debug.zip
  7bf2736457431bbba5c64b5320dd1c72d0d13fae59127fcc92805946de83908a  guix-build-3cfe366ec35e/output/x86_64-w64-mingw32/bitcoin-3cfe366ec35e-win64-setup-unsigned.exe
  721838ac437db5764c22c90d9c0a0b51283d6a50da8c60a6bccb394090380195  guix-build-3cfe366ec35e/output/x86_64-w64-mingw32/bitcoin-3cfe366ec35e-win64-unsigned.tar.gz
  dd58422fc4fd89353002bdb6a546b997fe31546c348a9b4a87bc697913abd382  guix-build-3cfe366ec35e/output/x86_64-w64-mingw32/bitcoin-3cfe366ec35e-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 3cfe366ec3

Tree-SHA512: b1f30f8775acd69e897784c2168887eedc008db80f6d2d0d68390716965fbd3ddfd70fd1560ef30a8cc70941e9010c395c7feed9386ca92b2c9148d063d64724
2023-05-22 10:49:07 +01:00
fanquake
5421dc3244 Merge bitcoin/bitcoin#27561: test: Explicitly specify directory where to search tests for
c44f3f2319 test: Explicitly specify directory where to search tests for (Hennadii Stepanov)

Pull request description:

  For out-of-source builds, the `test/functional/test_runner.py` is supposed to be run from the build directory which allows it to pick the `test/config.ini` file generated by the build system. Currently, it works accidently for the following reasons:
  - on POSIX systems, when running a created by Autoconf symlink to the `test/functional/test_runner.py` in the source directory, it actually has the source directory location in the `sys.path`.
  - on Windows (the `build_msvc` directory) VS project puts and copies every build artifact into the source tree (which is wrong and ugly).

  This PR makes `test/functional/test_runner.py` work from a build directory in any form (a symbolic link, a hard link, a copy) on _all_ supported platforms, which is highly desirable in the upcoming [CMake-based build system](https://github.com/bitcoin/bitcoin/pull/25797).

  For the current master branch, this PR has no behaviour change.

  Required for https://github.com/hebasto/bitcoin/pull/15.

  ---

  **Steps to reproduce the issue**

  While the issue is mostly specific to Windows and CMake builds, it is still possible to reproduce it on the current master branch.

  1. Make an out-of-source build:
  ```
  $ ./autogen.sh
  $ mkdir ../build && cd ../build
  $ ../bitcoin/configure
  $ make
  ```

  2. Note that Autoconf created a symbolic link `test/functional/test_runner.py` in the `../build` directory:
  ```
  $ ls -l test/functional/test_runner.py
  lrwxrwxrwx 1 hebasto hebasto 47 May  5 17:40 test/functional/test_runner.py -> ../../../bitcoin/test/functional/test_runner.py
  ```
  which works flawlessly.

  3. However, replacing this symbolic link with a hard link or a copy of `test/functional/test_runner.py` from the source tree will cause the following error:
  ```
  $ cp ../bitcoin/test/functional/test_runner.py test/functional/test_runner.py
  $ ls -l test/functional/test_runner.py
  $ ./test/functional/test_runner.py
  Temporary test directory at /tmp/test_runner_₿_🏃_20230505_175104
  Running Unit Tests for Test Framework Modules
  E
  ======================================================================
  ERROR: test_framework (unittest.loader._FailedTest)
  ----------------------------------------------------------------------
  ImportError: Failed to import test module: test_framework
  Traceback (most recent call last):
    File "/usr/lib/python3.10/unittest/loader.py", line 154, in loadTestsFromName
      module = __import__(module_name)
  ModuleNotFoundError: No module named 'test_framework'

  ----------------------------------------------------------------------
  Ran 1 test in 0.000s

  FAILED (errors=1)
  Early exiting after failure in TestFramework unit tests
  ```

ACKs for top commit:
  stickies-v:
    re-ACK c44f3f2319
  MarcoFalke:
    lgtm ACK c44f3f2319 💸

Tree-SHA512: 622ff629080a55f76dd4c1dab6699de0e9f06b75da3315cd3b31b972ef4bde746458bf3e8a95e879b3c6a63be2368af70005a83f6a3c85c4f1ba5be51e91a61d
2023-05-22 10:21:08 +01:00
fanquake
a106a86c46 Merge bitcoin/bitcoin#27696: build: Do not define ENABLE_ZMQ when ZMQ is not available
fa5831bd6f build: Do not define `ENABLE_ZMQ` when ZMQ is not available (Hennadii Stepanov)

Pull request description:

  A new behavior is consistent with the other optional dependencies.

  The source code contains `#if ENABLE_ZMQ` lines only:
  ```
  $ git grep ENABLE_ZMQ -- src/*.cpp
  src/init.cpp:#if ENABLE_ZMQ
  src/init.cpp:#if ENABLE_ZMQ
  src/init.cpp:#if ENABLE_ZMQ
  src/init.cpp:#if ENABLE_ZMQ
  src/init.cpp:#if ENABLE_ZMQ
  ```

  Change in description line -- "Define to 1..." -->  "Define this symbol.." -- is motivated by the fact that the actual value of the defined `ENABLE_ZMQ` macro does not matter at all.

  Related to:
  - https://github.com/bitcoin/bitcoin/issues/16419
  - https://github.com/bitcoin/bitcoin/pull/25302

ACKs for top commit:
  TheCharlatan:
    ACK fa5831bd6f
  jarolrod:
    ACK fa5831bd6f

Tree-SHA512: 5e72ff0d34c4b33205338daea0aae8d7aa0e48fd633e21af01af32b7ddb0532ef68dd3dd74deb2c1d2599691929617e8c09676bcbaaf7d669b88816f866f1db2
2023-05-22 10:00:15 +01:00
fanquake
f998eb7662 Merge bitcoin/bitcoin#27683: ci: remove RUN_SECURITY_TESTS
6a936580d1 ci: remove RUN_SECURITY_TESTS (fanquake)

Pull request description:

  We no-longer run any security/symbol checks in the CI, and doubt we will in future (if we do, it'll be via Guix, where this var would be redundant in any case). The CI environment doesn't (exactly) match the release build environment (and is semi-regularly changing), and the binaries produced in the CI don't match how we build release binaries, so there is no point trying to run these checks, especially as we add more involved tests, i.e #26953.

ACKs for top commit:
  josibake:
    code review ACK 6a936580d1
  TheCharlatan:
    ACK 6a936580d1

Tree-SHA512: c0eec61a4b873bac487ba9321b50116a215b4796bd7d416d98ffcd09969dbf635c2cb5aeb225c89d1e6462838fa2a48565048ebe730f48d76d3db46b64855a91
2023-05-22 09:52:27 +01:00
fanquake
9a8318f30b Merge bitcoin/bitcoin#27707: ci, iwyu: Double maximum line length for includes
98ea798411 ci, iwyu: Double maximum line length for includes (Hennadii Stepanov)

Pull request description:

  This PR makes the IWYU output in the CI 'tidy' task more useful by avoiding most cases where a comment ends with an ellipsis like that:
  ```
  #include "primitives/transaction.h"  // for CTxIn, CMutableTransaction, CTra...
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 98ea798411

Tree-SHA512: 25195ccb2095884b23586416b86999ebc42577c6d777abdbd176a704fa75c64deb91fa61cd91d570a5408dd459e930e53bc70d963b76c73fca7a800e74b1bdbf
2023-05-22 09:42:42 +01:00
MarcoFalke
fa1b3abc83 ci: Log qa-assets repo last commit
This documents the state in the CI output and may help debugging in case
of failure.
2023-05-22 10:02:40 +02:00
MarcoFalke
fa22966f33 fuzz: Print error message when FUZZ is missing
Also, add missing includes.
2023-05-22 10:02:29 +02:00
Sebastian Falbesoner
a97c59f12d test: p2p: check misbehavior for non-continuous headers messages 2023-05-21 15:21:35 +02:00
fanquake
5228223e1f ci: remove MSAN getrandom syscall workaround
The corresponding workaround will also be dropped in oss-fuzz:
25946a5448/projects/bitcoin-core/build.sh (L49).
2023-05-20 17:20:06 +01:00
fanquake
d5e06919db random: switch to using getrandom() directly
This requires a linux kernel of 3.17.0+, which seems entirely
reasonable. 3.17 went EOL in 2015, and the last supported 3.x kernel
(3.16) went EOL > 4 years ago, in 2020. For reference, the current
oldest maintained kernel is 4.14 (released 2017, EOL Jan 2024).

Support for `getrandom()` (and `getentropy()`) was added to
glibc 2.25, https://sourceware.org/legacy-ml/libc-alpha/2017-02/msg00079.html,
and we already require 2.27+.

All that being said, I don't think you would encounter a current day
system, running with kernel headers older than 3.17 (released 2014) but
also having a glibc of 2.27+ (released 2018).
2023-05-20 17:20:01 +01:00
fanquake
c2ba3f5b0c random: add [[maybe_unused]] to GetDevURandom
Rather than multiple instances of (void)GetDevURandom to silence
compiler warnings.
2023-05-20 17:09:48 +01:00
fanquake
c13c97dbf8 random: getentropy on macOS does not need unistd.h
Remove it. Make this change, so in a future commit, we can
combine #ifdefs, and avoid duplicate <sys/random.h> includes once we
switch to using getrandom directly.

Also remove the comment about macOS 10.12. We already require macOS >
10.15, so it is redundant.
2023-05-20 17:09:47 +01:00
Hennadii Stepanov
98ea798411 ci, iwyu: Double maximum line length for includes 2023-05-20 13:16:26 +01:00
fanquake
17acb2782a Merge bitcoin/bitcoin#27688: doc: remove Security section from build-unix.md
4bfcbbfd4a doc: remove Security section from build-unix.md (fanquake)

Pull request description:

  Our compile documentation isn't the right place for generic binary hardening notes, which are neither particularly Bitcoin-Core specific, or as relevant as they might have once been, i.e non-executable stacks are now just the norm.

  Just remove the notes for now, if someone has something more interesting/Bitcoin Core specific, it could be added in separate documentation in the future (maybe into the devwiki or similar).

  Split from https://github.com/bitcoin/bitcoin/pull/27685#discussion_r1196517868.

ACKs for top commit:
  jarolrod:
    ACK 4bfcbbfd4a

Tree-SHA512: 01b523ba40353d6cafb5dbd6540b514d83a2dc4e462a068fb390ca7de45b78e4a329367f70527f1824006ebe43f8caeea4ad05ec60556d5a5bd0143e27bf6581
2023-05-20 11:29:12 +01:00
TheCharlatan
7d3b35004b refactor: Move system from util to common library
Since the kernel library no longer depends on the system file, move it
to the common library instead in accordance to the diagram in
doc/design/libraries.md.
2023-05-20 12:08:13 +02:00
TheCharlatan
7eee356c0a refactor: Split util::AnyPtr into its own file 2023-05-20 12:03:33 +02:00
TheCharlatan
44de325d95 refactor: Split util::insert into its own file 2023-05-20 12:03:31 +02:00
TheCharlatan
9ec5da36b6 refactor: Move ScheduleBatchPriority to its own file
With the previous move of AlertNotify out of the validation file, and
thus out of the kernel library, ScheduleBatchPriority is the last
remaining function used by the kernel library from util/system. Move it
to its own file, such that util/system can be moved out of the util
library in the following few commits.

Moving util/system out of the kernel library removes further networking
as well as shell related code from it.
2023-05-20 12:03:30 +02:00
TheCharlatan
f871c69191 kernel: Add warning method to notifications
This commit is part of the libbitcoinkernel project and seeks to remove
the ChainstateManager's and, more generally, the kernel library's
dependency on interface_ui with options methods in this and the following
few commits. By removing interface_ui from the kernel library, its
dependency on boost is reduced to just boost::multi_index.

The DoWarning and AlertNotify functions are moved out of the
validation.cpp file, which removes its dependency on interface_ui as
well as util/system.
2023-05-20 12:03:28 +02:00
TheCharlatan
4452707ede kernel: Add progress method to notifications
This commit is part of the libbitcoinkernel project and seeks to remove
the ChainstateManager's and, more generally, the kernel library's
dependency on interface_ui with options methods in this and the
following few commits. By removing interface_ui from the kernel library,
its dependency on boost is reduced to just boost::multi_index.
2023-05-20 12:03:26 +02:00
TheCharlatan
84d71457e7 kernel: Add headerTip method to notifications
This commit is part of the libbitcoinkernel project and seeks to remove
the ChainstateManager's and, more generally, the kernel library's
dependency on interface_ui with options methods in this and the following
few commits. By removing interface_ui from the kernel library, its
dependency on boost is reduced to just boost::multi_index.
2023-05-20 12:03:24 +02:00
TheCharlatan
447761c822 kernel: Add notification interface
This commit is part of the libbitcoinkernel project and seeks to remove
the ChainstateManager's and, more generally, the kernel library's
dependency on interface_ui with options methods in this and the following
few commits. By removing interface_ui from the kernel library, its
dependency on boost is reduced to just boost::multi_index.

Define a new kernel notification class with virtual methods for
notifying about internal kernel events. Create a new file in the node
library for defining a function creating the default set of notification
methods such that these do not need to be re-defined all over the
codebase. As a first step, add a `blockTip` method, wrapping
`uiInterface.NotifyBlockTip`.
2023-05-20 12:03:22 +02:00
Hennadii Stepanov
c44f3f2319 test: Explicitly specify directory where to search tests for
This change allows `test_runner.py` to work from an out-of-source build
directory using a symlink, a hard link or a copy on any platform.
2023-05-19 19:21:10 +01:00
glozow
0f8c95dccd Merge bitcoin/bitcoin#27021: Implement Mini version of BlockAssembler to calculate mining scores
6b605b91c1 [fuzz] Add MiniMiner target + diff fuzz against BlockAssembler (glozow)
3f3f2d59ea [unit test] GatherClusters and MiniMiner unit tests (glozow)
59afcc8354 Implement Mini version of BlockAssembler to calculate mining scores (glozow)
56484f0fdc [mempool] find connected mempool entries with GatherClusters(…) (glozow)

Pull request description:

  Implement Mini version of BlockAssembler to calculate mining scores

  Run the mining algorithm on a subset of the mempool, only disturbing the
  mempool to copy out fee information for relevant entries. Intended to be
  used by wallet to calculate amounts needed for fee-bumping unconfirmed
  transactions.

  From comments of sipa and glozow below:

  > > In what way does the code added here differ from the real block assembly code?
  >
  >    * Only operates on the relevant transactions rather than full mempool
  >    * Has the ability to remove transactions that will be replaced so they don't impact their ancestors
  >    * Does not hold mempool lock outside of the constructor, makes copies of the entries it needs instead (though I'm not sure if this has an effect in practice)
  >    * Doesn't do the sanity checks like keeping weight within max block weight and `IsFinalTx()`
  >    * After the block template is built, additionally calculates fees to bump remaining ancestor packages to target feerate

ACKs for top commit:
  achow101:
    ACK 6b605b91c1
  Xekyo:
    > ACK [6b605b9](6b605b91c1) modulo `miniminer_overlap` test.
  furszy:
    ACK 6b605b91 modulo `miniminer_overlap` test.
  theStack:
    Code-review ACK 6b605b91c1

Tree-SHA512: f86a8b4ae0506858a7b15d90f417ebceea5038b395c05c825e3796123ad3b6cb8a98ebb948521316802a4c6d60ebd7041093356b1e2c2922a06b3b96b3b8acb6
2023-05-19 10:26:19 -04:00
brunoerg
272eb55616 test: fix include_immature_coinbase logic in get_utxos
Use current block height to compute the confirmation count
instead of using the value from utxo object
2023-05-19 09:13:30 -03:00
brunoerg
a951c34f17 test: fix interface_usdt_mempool by mining a block after each test
Co-authored-by: josibake <josibake@protonmail.com>
2023-05-19 09:13:30 -03:00
brunoerg
1557bf1196 test: fix mature utxos addition to wallet in mempool_package_limits 2023-05-19 09:13:30 -03:00
brunoerg
60ced9007d test: fix intermittent issue in feature_bip68_sequence
To avoid `bad-txns-premature-spend-of-coinbase` error,
when getting a utxo (using `get_utxo`) to create a new
transaction `get_utxo` shouldn't return by default
immature coinbase.
2023-05-19 09:13:17 -03:00
fanquake
fc4bee3f19 Merge bitcoin/bitcoin#27687: msvc: Provide ObjectFileName explicitly
b8ed95127b msvc: Provide `ObjectFileName` explicitly (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up to https://github.com/bitcoin/bitcoin/pull/26715.

  Fixes intermittent MSVC link [errors](https://cirrus-ci.com/task/6646912535756800).

ACKs for top commit:
  sipsorcery:
    ACK b8ed95127b.

Tree-SHA512: 4319ecf61b578ce66d240998d089b9bb0a42f89dcfcb1a73f648cd3915f566c773721dcff1feba27d393a743d121334ccb890b1a519173e35a156d6135821ef4
2023-05-19 12:32:22 +01:00
fanquake
7be7e62fdf Merge bitcoin/bitcoin#27695: test: Add test to check tx in the last block can be downloaded
fa4c16b186 test: Add test to check tx in the last block can be downloaded (MarcoFalke)
fadc8490ab test: Split up test_notfound_on_unannounced_tx test case (MarcoFalke)

Pull request description:

  If a peer received an `inv` about a transaction, which was included in a block before receiving the corresponding `getdata`, it can be beneficial to send this transaction to the peer to aid compact block relay.

  Add a test for this to avoid breaking it in the future.

ACKs for top commit:
  sdaftuar:
    ACK fa4c16b186
  instagibbs:
    ACK fa4c16b186

Tree-SHA512: 1ec16dcc216dd29c849928e753158d45c409612e9ac528db16e5af465bb5b69cd42241ac6f67e5a4583b8e558eeba49fa0a0889a4247b3fb0053b2758eec9490
2023-05-19 10:49:08 +01:00
fanquake
2f1403ae53 Merge bitcoin/bitcoin#27458: build: Detect USDT the same way how it is used in the code
b53cab0083 build: Detect USDT the same way how it is used in the code (Hennadii Stepanov)

Pull request description:

  In the code we do not use string literals.

  Also a check for `DTRACE_PROBE7` macro has been added as not all systems define`DTRACE_PROBE{6,7,8,9,10,11,12}` macros (e.g., FreeBSD).

ACKs for top commit:
  0xB10C:
    ACK b53cab0083

Tree-SHA512: 74f49424d57bf1929f2b09edba1449cef5a1a2448161952da35302343f3003d5bedeab1417e166b656c5f629303e2de888550b1219e886a1b991b12b9c880794
2023-05-19 10:05:15 +01:00
fanquake
edd6d8395f Merge bitcoin/bitcoin#27662: build: Bump minimum supported GCC to g++-9
fa953f15bf build: Bump minimum supported GCC to g++-9 (MarcoFalke)
fa69955e74 ci: Bump centos:stream8 to centos:stream9 (MarcoFalke)
fa6a755d9f ci: Document the false positive error for g++-9 (MarcoFalke)

Pull request description:

  It is a bit frustrating to write valid C++ code only to realize that g++-8 fails to parse it later on.

  The only non-EOL operating system still shipping with g++-8 is CentOS Stream 8. I think it is reasonable for users of affected Linux distributions to:

  * Upgrade their operating system, or compiler to a supported version.
  * Alternatively, stay with a previous release of Bitcoin Core as long as it is supported.

  Fixes https://github.com/bitcoin/bitcoin/issues/27537

ACKs for top commit:
  hebasto:
    ACK fa953f15bf
  fanquake:
    ACK fa953f15bf

Tree-SHA512: b9cf7e763d3071e1e008c5010de19601d4773afe46d58cf869d3f59285c53240c739a1cd7235a5525ede1bbdf6b6cb6fb091c8fc314864a28d5b27a400bb7632
2023-05-19 09:57:36 +01:00
Andrew Chow
6cc136bbd3 Merge bitcoin/bitcoin#27556: wallet: fix deadlock in bdb read write operation
69d43905b7 test: add coverage for wallet read write db deadlock (furszy)
12daf6fcdc walletdb: scope bdb::EraseRecords under a single db txn (furszy)
043fcb0b05 wallet: bugfix, GetNewCursor() misses to provide batch ptr to BerkeleyCursor (furszy)

Pull request description:

  Decoupled from #26644 so it can closed in favor of #26715.

  Basically, with bdb, we can't make a write operation while we are traversing the db with the same db handler. These two operations are performed in different txn contexts and cause a deadlock.

  Added coverage by using `EraseRecords()` which is the simplest function that executes this process.

  To replicate it, need bdb support and drop the first commit. The test will run forever.

ACKs for top commit:
  achow101:
    ACK 69d43905b7
  hebasto:
    re-ACK 69d43905b7

Tree-SHA512: b3773be78925f674e962f4a5c54b398a9d0cfe697148c01c3ec0d68281cc5c1444b38165960d219ef3cf1a57c8ce6427f44a876275958d49bbc0808486e19d7d
2023-05-18 11:10:05 -04:00
Andrew Chow
ce2440e680 Merge bitcoin/bitcoin#27698: doc: add historical release notes for 24.1 & 23.2
e0f69a7081 doc: add 24.1 release notes (fanquake)
f1e6f2d681 doc: add 23.2 release notes (fanquake)

Pull request description:

  Also for linking to from `Releases`.

ACKs for top commit:
  achow101:
    ACK e0f69a7081

Tree-SHA512: d00d692ab477408842d2ae8575c3355433fb7f236a354317075a6de06f8fbe3b038ee8ccd349789ce1fca24c5fae63fad78521480c14425ce679cdee3307e210
2023-05-18 10:46:02 -04:00
fanquake
e0f69a7081 doc: add 24.1 release notes 2023-05-18 15:38:31 +01:00
fanquake
f1e6f2d681 doc: add 23.2 release notes 2023-05-18 15:38:19 +01:00
Greg Sanders
13f9b20b4c Only request full blocks from the peer we thought had the block in-flight
This is a change in behavior so that if for some reason we request a block from a peer, we don't allow an unsolicited CMPCT_BLOCK announcement for that same block to cause a request for a full block from the uninvited peer (as some type of request is already outstanding from the original peer)
2023-05-18 10:03:37 -04:00
Greg Sanders
cce96182ba Convert mapBlocksInFlight to a multimap 2023-05-18 10:03:37 -04:00
Greg Sanders
a90595478d Remove nBlocksInFlight 2023-05-18 10:03:37 -04:00
Greg Sanders
86cff8bf18 alias BlockDownloadMap for mapBlocksInFlight 2023-05-18 10:03:37 -04:00
Hennadii Stepanov
b53cab0083 build: Detect USDT the same way how it is used in the code 2023-05-18 14:59:14 +01:00
fanquake
ccc431d53e Merge bitcoin/bitcoin#27640: test: Return dict in MiniWallet::send_to
faf4315c88 test: Return dict in MiniWallet::send_to (MarcoFalke)

Pull request description:

  Returning a tuple has many issues:

  * If only one value is needed, it can not be indexed by name
  * If another value is added to the return value, all call sites need to be updated

  Bite the bullet now and update all call sites to fix the above issues.

ACKs for top commit:
  brunoerg:
    crACK faf4315c88
  theStack:
    Code-review ACK faf4315c88
  stickies-v:
    Code review ACK faf4315c88

Tree-SHA512: 8ce1aca237df21f04b3990d0e5fcb49cc408fe6404399d3769a64eae1b5218941157d9785fce1bd9e45140cf70e06c3aa42646ee8f7b57855beb784fc3ef0261
2023-05-18 14:26:13 +01:00
fanquake
4bfcbbfd4a doc: remove Security section from build-unix.md
Our compile documentation isn't the right place for genric binary
hardening notes, which are neither particularly Bitcoin-Core specific,
or as relevant as they might have once been, i.e non-executable stacks
are now just the norm.

Just remove the notes for now, if someone has
something more interesting/Bitcoin Core specific, it could be added in
separate documentation in the future (maybe into the devwiki or
similar).

Split from
https://github.com/bitcoin/bitcoin/pull/27685#discussion_r1196517868.
2023-05-18 12:09:02 +01:00
fanquake
5f70cd3997 Merge bitcoin/bitcoin#27685: doc: Rework build-unix.md
fa29651c3f doc: Rework build-unix.md (MarcoFalke)

Pull request description:

  The doc has many issues:

  * The fist section contains outdated non-existing and confusing configure flags like `--enable-cxx` and `--disable-shared`, as well as edge-case expert options such as `BDB_PREFIX`. Fix that by removing the section and adding notes elsewhere, if applicable.
  * There are links to the depends system before instructions on how to simply build from system packages. Fix that by moving that later.
  * Also, remove sections that are duplicate with other depends READMEs.

ACKs for top commit:
  fanquake:
    ACK fa29651c3f
  TheCharlatan:
    ACK fa29651c3f

Tree-SHA512: 5348ddf3fa094c630d80b63033ca7b40ec0356427856f9a1075b31244f6bf8ec65cb2a738366e1174ef2fe7e0bf5cc249a62c58f458bbaf50668aceeac954820
2023-05-18 12:07:57 +01:00
fanquake
87d6f6c29d Merge bitcoin/bitcoin#27615: msvc: Rename libbitcoinconsensus to libbitcoin_consensus and other adjustments
a94d75fa81 msvc: Do not define `HAVE_CONSENSUS_LIB` (Hennadii Stepanov)
cf6ff1031b msvc: Clean up `libbitcoin_consensus` source files (Hennadii Stepanov)
30aee016f1 scripted-diff: Rename `libbitcoinconsensus` to `libbitcoin_consensus` (Hennadii Stepanov)

Pull request description:

  The current Autotools-based build system operates with two build artifacts:
  - [`LIBBITCOIN_CONSENSUS`](3777c75d14/src/Makefile.am (L31)) which is [defined as](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) "Stable, backwards-compatible consensus functionality used by _libbitcoin_node_ and _libbitcoin_wallet_"
  - [`LIBBITCOINCONSENSUS`](3777c75d14/src/Makefile.am (L42)) which is [defined as](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) "Shared library build of static _libbitcoin_consensus_ library"

  The way how the `libbitcoinconsensus.vcxproj` project is used in the MSVC build system obviously shows that it is the former use case.

  This PR makes the related adjustments to the MSVC build system.

ACKs for top commit:
  sipsorcery:
    ACK a94d75fa81.

Tree-SHA512: 1144e13ee2b428ce14be8f76729744830c502a07814eb03e2aa6b8e009d8936fd13743e3f36ef3f31fac0e3979eb9af23e6a1364f151df49b3ae18dbd14cbf99
2023-05-18 11:34:34 +01:00
fanquake
8146f2a035 Merge bitcoin/bitcoin#27689: doc: remove mention of glibc 2.10+
7014e08015 doc: remove mention of glibc 2.10+ (fanquake)

Pull request description:

  We already require glibc 2.27+, so mentioning a much older version here is redundant.

ACKs for top commit:
  TheCharlatan:
    ACK 7014e08015

Tree-SHA512: 883a566a80cabe34bfb5d902990f3eca08d0e11438e6c128d311e558f373ec232b0934deb85d12d796baacfeae590af8c73aa1b2faef07f27ffa9011270ffd96
2023-05-18 11:33:35 +01:00
MarcoFalke
fa953f15bf build: Bump minimum supported GCC to g++-9
Also, update the code to use constexpr, which does not work in g++-8.

Also, drop the no longer needed build-aux/m4/l_filesystem.m4.
2023-05-18 12:24:40 +02:00
MarcoFalke
fa69955e74 ci: Bump centos:stream8 to centos:stream9
This is required for the next commit. Also, drop CI_RETRY_EXE before
"dnf install", because it requires getopt, which will only be installed
later on via util-linux
2023-05-18 12:24:38 +02:00
MarcoFalke
fa6a755d9f ci: Document the false positive error for g++-9 2023-05-18 12:24:28 +02:00
fanquake
4bca26e744 Merge bitcoin/bitcoin#27690: ci: Use credits for ARM task
fa3761d19d ci: Reduce tsan CPU and memory for faster scheduling (MarcoFalke)
aaaa07bc84 ci: Use credits for ARM task (MarcoFalke)

Pull request description:

  After https://github.com/bitcoin/bitcoin/pull/27562 the task should finish in less than 10 minutes, so also using credits for it will be cheap and improve dev experience.

ACKs for top commit:
  fanquake:
    ACK fa3761d19d

Tree-SHA512: 98ba76eaa63bcbab674076bb9877b3a20d1b90e6d51e2fd7b97ae245414e984d734006b15ea60228a1f4f9bc72fbe2a1a8910fd05e7c52dd2a8b223bfaa25b50
2023-05-18 11:17:08 +01:00
fanquake
77b0a80ce7 Merge bitcoin/bitcoin#27682: build: Bump minimum supported Clang to clang-10
fa199ee614 ci: Drop NO_WERROR=1 for clang-10 build (MarcoFalke)
fad2c200f4 build: Bump minimum Clang to clang-10 (MarcoFalke)
fad7cfee8d doc: Remove outdated CentOS comment (MarcoFalke)

Pull request description:

  It doesn't make sense to support a minimum clang version that is difficult to install on all supported operating systems, which generally ship a later version:

  * Ubuntu Focal 20.04: https://packages.ubuntu.com/focal/clang-10 and https://packages.ubuntu.com/focal/clang-12
  * Debian Bullseye: https://packages.debian.org/bullseye/clang-13
  * CentOS 8 Stream: All Clang versions from 11.0 to 15.0

  Also, it allows to drop build code, which means it won't waste review when rolling over into cmake (`cmake/module/CheckStdFilesystem.cmake`).

ACKs for top commit:
  hebasto:
    ACK fa199ee614
  fanquake:
    ACK fa199ee614

Tree-SHA512: c1a0e8f191a6db866b8be3c9d254dc3f576fa021e2eaaeb68f3354554a8b38eaa90bbf9871ff92351b715e62a6b7b98cf94eba6dc53d7c951bddb6ad49ba7716
2023-05-18 11:05:08 +01:00
Hennadii Stepanov
fa5831bd6f build: Do not define ENABLE_ZMQ when ZMQ is not available
A new behavior is consistent with the other optional dependencies.

The source code contains `#if ENABLE_ZMQ` lines only.
2023-05-18 10:48:48 +01:00
MarcoFalke
fa4c16b186 test: Add test to check tx in the last block can be downloaded 2023-05-18 11:28:24 +02:00
MarcoFalke
fadc8490ab test: Split up test_notfound_on_unannounced_tx test case 2023-05-18 09:57:02 +02:00
Ryan Ofsky
4e8a7654f6 Merge bitcoin/bitcoin#25193: indexes: Read the locator's top block during init, allow interaction with reindex-chainstate
97844d9268 index: Enable reindex-chainstate with active indexes (Martin Zumsande)
60bec3c82d index: Use first block from locator instead of looking for fork point (Martin Zumsande)

Pull request description:

  This makes two improvements to the index init phase:

  **1) Prevent index corruption in case a reorg happens when the index was switched off**:
  This is done by reading in the top block stored in the locator instead of looking for a fork point already in `BaseIndex::Init()`.
  Before, we'd just go back to the fork point by calling `FindForkInGlobalIndex()`, which would have corrupted the coinstatsindex because its saved muhash needs to be reverted step by step by un-applying all blocks in between, which wasn't done before. This is now being done a bit later in  `ThreadSync()`, which has existing logic to call the custom `Rewind()` method when going back along the chain to the forking point (thanks ryanofsky for pointing this out to me!).

  **2) Allow using the `-reindex-chainstate` option without needing to disabling indexes**:
  With `BaseIndex::Init()` not calling `FindForkInGlobalIndex()` anymore, we can allow `reindex-chainstate` with active indexes. `reindex-chainstate` deletes the chain and rebuilds it later in `ThreadImport`, so there is no chain available during `BaseIndex::Init()`, which would lead to problems (see #24789).
  But now we'll only need the chain a bit later in `BaseIndex::ThreadSync`, which will wait for the reindex-chainstate in `ThreadImport` to finish and will continue syncing after that.

ACKs for top commit:
  ryanofsky:
    Code review ACK 97844d9268. Just simple rebase since last review

Tree-SHA512: e24973fc22e0b87a49026f4820aecb0a4e415f4d381bade9969dd31cf97afecfea0449dce7fcc797343b792199cc8287276d1f5ffa4433dcb54fb24a808db6fb
2023-05-17 13:27:31 -04:00
Martin Zumsande
97844d9268 index: Enable reindex-chainstate with active indexes
This is achieved by letting the index sync thread wait until
reindex-chainstate is finished.

This also disables the pruning check when reindexing the chainstate (which is
incompatible with prune mode) because there would be no chain at this point
in init.
2023-05-17 11:14:28 -04:00
Martin Zumsande
60bec3c82d index: Use first block from locator instead of looking for fork point
The index sync code has logic to go back the chain to the forking point, while
also updating index-specific state, which is necessary to prevent
possible corruption of the coinstatsindex.

Also add a test for this (a reorg happens while the index is deactivated)
that would not pass before this change.
2023-05-17 11:14:10 -04:00
MarcoFalke
fa3761d19d ci: Reduce tsan CPU and memory for faster scheduling 2023-05-17 16:29:58 +02:00
MarcoFalke
aaaa07bc84 ci: Use credits for ARM task 2023-05-17 16:17:00 +02:00
Hennadii Stepanov
b8ed95127b msvc: Provide ObjectFileName explicitly
Should fix intermittent MSVC link errors.
2023-05-17 15:13:11 +01:00
fanquake
7014e08015 doc: remove mention of glibc 2.10+
We already require glibc 2.27+, so mentioning a much older version here
is redundant.
2023-05-17 15:02:51 +01:00
MarcoFalke
fa29651c3f doc: Rework build-unix.md 2023-05-17 14:31:03 +02:00
fanquake
594f05db19 Merge bitcoin/bitcoin#27668: guix: document when certain patches can be dropped
a09269a146 guix: document when certain guix patches can be dropped (fanquake)

Pull request description:

  Additional notes for when patches can be dropped.

ACKs for top commit:
  hebasto:
    ACK a09269a146, I have reviewed the changes and they look OK.
  jarolrod:
    ACK a09269a146

Tree-SHA512: c1876b9a4e3cf73645d25c9077cef19a9b6b7fe2eda5dc9d82fd3ca3f9105453406c1b197e6635035b6ce19c9f255c070bebed5563f68913033d04627202155a
2023-05-17 13:08:04 +01:00
fanquake
4c3d67a2d0 Merge bitcoin/bitcoin#27571: ci: Run iwyu on all src files
ddddf4957b ci: Run iwyu on all src files (MarcoFalke)

Pull request description:

  This makes it easier to look at the CI output of a file without having to manually add it first to the list.

ACKs for top commit:
  hebasto:
    ACK ddddf4957b

Tree-SHA512: 342b52838ae45ea343731c30058cdd5595d5ea5601a1f396de4466ccdd63f7ab07b3a193df3669e4dca7cb535557dcc98f866b3cf986b98176b20ecead123868
2023-05-17 13:06:23 +01:00
fanquake
6a936580d1 ci: remove RUN_SECURITY_TESTS
We no-longer run any security/syymbol checks in the CI, and doubt we
will in future (if we do, it'll be via Guix, where this var would be
redundant in any case). The CI environment doesn't (exactly) match the
release build environment (and is semi-regularly changing), and the
binaries produced in the CI don't match how we build release binaries,
so there is no point trying to run these checks, especially as we add
more involved tests, i.e #26953.
2023-05-17 10:57:39 +01:00
MarcoFalke
fa199ee614 ci: Drop NO_WERROR=1 for clang-10 build
This partially reverts commit 71383f2fad.
This should be fine, because if warnings are issues again in the future,
it can be disabled again, along with a list of the false warnings.
2023-05-17 10:55:03 +02:00
Hennadii Stepanov
a75c77ea90 Merge bitcoin-core/gui#729: test: Add missed header
36e2d51b8f qt, test: Add missed header (Hennadii Stepanov)

Pull request description:

  Should fix MSVC link errors like [that](https://api.cirrus-ci.com/v1/task/4870882892447744/logs/build.log):
  ```
  addressbooktests.obj : error LNK2019: unresolved external symbol "void __cdecl ConfirmMessage(class QString *,class std::chrono::duration<__int64,struct std::ratio<1,1000> >)" (?ConfirmMessage@@YAXPEAVQString@@V?$duration@_JU?$ratio@$00$0DOI@@std@@@chrono@std@@@Z) referenced in function "void __cdecl `anonymous namespace'::EditAddressAndSubmit(class EditAddressDialog *,class QString const &,class QString const &,class QString)" (?EditAddressAndSubmit@?A0x2e52698e@@YAXPEAVEditAddressDialog@@AEBVQString@@1V3@@Z) [C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\build_msvc\test_bitcoin-qt\test_bitcoin-qt.vcxproj]
  wallettests.obj : error LNK2001: unresolved external symbol "void __cdecl ConfirmMessage(class QString *,class std::chrono::duration<__int64,struct std::ratio<1,1000> >)" (?ConfirmMessage@@YAXPEAVQString@@V?$duration@_JU?$ratio@$00$0DOI@@std@@@chrono@std@@@Z) [C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\build_msvc\test_bitcoin-qt\test_bitcoin-qt.vcxproj]
  C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\build_msvc\x64\Release\test_bitcoin-qt.exe : fatal error LNK1120: 1 unresolved externals [C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\build_msvc\test_bitcoin-qt\test_bitcoin-qt.vcxproj]
  ```

ACKs for top commit:
  fanquake:
    ACK 36e2d51b8f

Tree-SHA512: 84685598fbf8857c0284ff660d953b93da3c2f47ba4ac0d3591b5009a6bcdb76898031fd70f289c4256ce389e485bd259ca145f9f862f085795e374dfa88705d
2023-05-17 09:39:12 +01:00
MarcoFalke
fad2c200f4 build: Bump minimum Clang to clang-10 2023-05-17 10:30:41 +02:00
MarcoFalke
fad7cfee8d doc: Remove outdated CentOS comment
No longer applicable after CONFIG_SHELL must be explicitly set to dash
after commit fafc55a489.
2023-05-17 09:54:31 +02:00
fanquake
904631e0fc Merge bitcoin/bitcoin#27667: ci: Remove unused errtrace trap ERR
fad09b703f ci: Remove unused errtrace trap ERR (MarcoFalke)

Pull request description:

  This was added in commit 069752b726, presumably at a time when the functional tests wouldn't capture stderr.

  Now that all tests capture and print stderr on failure, it can be removed. Reference:

  * Unit tests capture via `2>&1`:

  d7700d3a26/src/Makefile.test.include (L421)

  * Functional tests capture as well:

  d7700d3a26/test/functional/test_framework/test_node.py (L356)

ACKs for top commit:
  fanquake:
    ACK fad09b703f
  hebasto:
    ACK fad09b703f, tested on Ubuntu 22.04: I can still see warnings from the sanitizers in both unit and functional tests.

Tree-SHA512: 1e786eee432a7a50eb9f78b06b2b157321cc16f91b613e3b476e9e51572592fe4bcf4dc15df176e5f019f24497ac68cf332d2037b55b57498c93f4e19613163c
2023-05-16 15:28:04 +01:00
Hennadii Stepanov
36e2d51b8f qt, test: Add missed header
Should fix MSVC link errors.
2023-05-16 15:27:43 +01:00
fanquake
03a4e8dd4c Merge bitcoin/bitcoin#27674: ci: Fix "Number of CPUs" output
5d49d98731 ci: Fix "Number of CPUs" output (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up to https://github.com/bitcoin/bitcoin/pull/27616:

  - on [master](https://api.cirrus-ci.com/v1/task/5809898840129536/logs/ci.log):
  ```
  Number of CPUs \(nproc\): $(nproc)
  ```

  - this [PR](https://api.cirrus-ci.com/v1/task/6495994095861760/logs/ci.log):
  ```
  Number of CPUs (nproc): 32
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 5d49d98731

Tree-SHA512: d97ee3587dbadb604a381aa9990b58d75441307fc98e7ae674436f8318200c8faef7171348655cdcc3ed360c8ca22eacf063cb430b826a40cb0952a436c511f3
2023-05-16 13:38:51 +01:00
Hennadii Stepanov
5d49d98731 ci: Fix "Number of CPUs" output 2023-05-16 12:45:32 +01:00
Sjors Provoost
bc862fad29 ConnectTip: don't log total disk read time in bench
This incorrectly assumed num_blocks_total would be greater than 0. This is not guaranteed until the ConnectBlock call right below it.

The total and average metric is not very useful because it does not distinguish between blocks read from disk and those loaded from memory. So rather than fixing the divide by zero issue, we just drop the metric.
2023-05-16 13:15:37 +02:00
fanquake
3cfe366ec3 guix: remove redundant glibc patches
These should only be relevant for a glibc that is built as part of a
Guix system, and should not be required for a glibc that is just being
built to compile our binaries against. A x86_64 linux bitcoind produced
with Guix using master vs this change has no difference. i.e:

```diff
@@ -20311,15 +20311,15 @@
 This is experimental software.
 The source code is available from %s.
 Please contribute if you find %s useful. Visit %s for further information about the software.
 The %s developers
 The Bitcoin Core developers
 <https://bitcoincore.org/>
 Copyright (C) %i-%i
-v25.99.0-gda0bf1d07639b0490791bbd6aec71bbea8aa2aThe %s developer<https://github.com/bitcoin/bitcDistributed under the MIT software license, see the accompanyingThis is experimeThe source code is available froPlease contribute if you find %s useful. Visit %s for further information about Copyright (C) %ibool BCLog::Logger::StartLogging()
+v25.99.0-gd7700d3a26478d9b1648463c188648c7047b1cThe %s developer<https://github.com/bitcoin/bitcDistributed under the MIT software license, see the accompanyingThis is experimeThe source code is available froPlease contribute if you find %s useful. Visit %s for further information about Copyright (C) %ibool BCLog::Logger::StartLogging()
 std::string BCLog::Logger::LogLevelToStr(BCLog::Level) const
 std::string LogCategoryToStr(BCLog::LogFlags)
 void BCLog::Logger::LogPrintStr(const string&, const string&, const string&, int, BCLog::LogFlags, BCLog::Level)
 void BCLog::Logger::ShrinkDebugFile()
 Failed to shrink debug log file: fseek(...) failed
 logging.cpp
 m_buffering
```

```diff
@@ -1505889,15 +1505889,15 @@
 	call   aa3380 <malloc@plt+0xa4edb0>
 	mov    (%rsp),%rdx
 	movdqa 0x465540(%rip),%xmm0
 	mov    %rax,0x7a0559(%rip)
 	lea    0x7a0552(%rip),%rsi
 	lea    0x3957bb(%rip),%rdi
 	mov    %rdx,0x7a0554(%rip)
-	mov    $0x3038,%edx
+	mov    $0x3036,%edx
 	movups %xmm0,(%rax)
 	movdqa 0x465524(%rip),%xmm0
 	mov    %dx,0x30(%rax)
 	mov    0x7a0529(%rip),%rdx
 	movups %xmm0,0x10(%rax)
 	movdqa 0x46551d(%rip),%xmm0
 	movups %xmm0,0x20(%rax)
```

```diff
@@ -37238,17 +37238,17 @@
   0x00b73730 65202573 20646576 656c6f70 65727300 e %s developers.
   0x00b73740 54686520 42697463 6f696e20 436f7265 The Bitcoin Core
   0x00b73750 20646576 656c6f70 65727300 434f5059  developers.COPY
   0x00b73760 494e4700 3c687474 70733a2f 2f626974 ING.<https://bit
   0x00b73770 636f696e 636f7265 2e6f7267 2f3e0043 coincore.org/>.C
   0x00b73780 6f707972 69676874 20284329 2025692d opyright (C) %i-
   0x00b73790 25690053 61746f73 68690000 00000000 %i.Satoshi......
-  0x00b737a0 7632352e 39392e30 2d676461 30626631 v25.99.0-gda0bf1
-  0x00b737b0 64303736 33396230 34393037 39316262 d07639b0490791bb
-  0x00b737c0 64366165 63373162 62656138 61613261 d6aec71bbea8aa2a
+  0x00b737a0 7632352e 39392e30 2d676437 37303064 v25.99.0-gd7700d
+  0x00b737b0 33613236 34373864 39623136 34383436 3a26478d9b164846
+  0x00b737c0 33633138 38363438 63373034 37623163 3c188648c7047b1c
   0x00b737d0 54686520 25732064 6576656c 6f706572 The %s developer
   0x00b737e0 3c687474 70733a2f 2f676974 6875622e <https://github.
   0x00b737f0 636f6d2f 62697463 6f696e2f 62697463 com/bitcoin/bitc
   0x00b73800 44697374 72696275 74656420 756e6465 Distributed unde
   0x00b73810 72207468 65204d49 5420736f 66747761 r the MIT softwa
   0x00b73820 7265206c 6963656e 73652c20 73656520 re license, see
   0x00b73830 74686520 6163636f 6d70616e 79696e67 the accompanying
```

```diff
@@ -1,5 +1,5 @@

 Hex dump of section '.gnu_debuglink':
   0x00000000 62697463 6f696e64 2e646267 00000000 bitcoind.dbg....
-  0x00000010 6b6e8eda                            kn..
+  0x00000010 345cb865                            4\.e
```
2023-05-16 11:28:49 +01:00
fanquake
edd2a86445 Merge bitcoin/bitcoin#26422: build: Use newest config.{guess,sub} available
ea7b852849 build: Use newest `config.{guess,sub}` available (Hennadii Stepanov)

Pull request description:

  Fixes cross-compiling on old systems. See https://github.com/bitcoin/bitcoin/issues/26420#issuecomment-1295886891.

  Guix builds:
  ```
  ab58f8db4fb50c2af08be646e4a57491ff853bf41319fe3e962bb928ace2c1be  guix-build-ea7b8528490d/output/aarch64-linux-gnu/SHA256SUMS.part
  c4a42a7dbfb15aa910ce269c3f3158df758eba01dd855cf6529c960d9b8bd76b  guix-build-ea7b8528490d/output/aarch64-linux-gnu/bitcoin-ea7b8528490d-aarch64-linux-gnu-debug.tar.gz
  add5f80b0c8ee216c1d8db31e13656c45f11fe5c967324b8fcedd64d0d408938  guix-build-ea7b8528490d/output/aarch64-linux-gnu/bitcoin-ea7b8528490d-aarch64-linux-gnu.tar.gz
  ba807ae5b29aca611a35700a968a4f9010528d6cdb6412f98c707aa0e9510e8b  guix-build-ea7b8528490d/output/arm-linux-gnueabihf/SHA256SUMS.part
  fe127eff1fc8389ef00ae39d1446640512120d2f247a2ee64c06290116265729  guix-build-ea7b8528490d/output/arm-linux-gnueabihf/bitcoin-ea7b8528490d-arm-linux-gnueabihf-debug.tar.gz
  ac75f0aba2e0801fc6b59405bff09884837e0ff8b6fa9fc1225e8325a4ec78a4  guix-build-ea7b8528490d/output/arm-linux-gnueabihf/bitcoin-ea7b8528490d-arm-linux-gnueabihf.tar.gz
  dee402db94829bd5ce029ca02d75e008f8a18a44fd88e8ecd00b3ce586f223f4  guix-build-ea7b8528490d/output/arm64-apple-darwin/SHA256SUMS.part
  bbf985dbd1ca875fbb0a03c21698d4b6797c27be45e3a31a3229daee0ecbcbf4  guix-build-ea7b8528490d/output/arm64-apple-darwin/bitcoin-ea7b8528490d-arm64-apple-darwin-unsigned.dmg
  3cca89e532a2b637185ca316168b0f6fac2fce8795b486d530ee966a844716f1  guix-build-ea7b8528490d/output/arm64-apple-darwin/bitcoin-ea7b8528490d-arm64-apple-darwin-unsigned.tar.gz
  487c68e1a71f6254d48f8d322a46c94e671e00b5930fbfe689804cd8e379c8e9  guix-build-ea7b8528490d/output/arm64-apple-darwin/bitcoin-ea7b8528490d-arm64-apple-darwin.tar.gz
  db663cf3216871f4c87b687a839cb5cf18b5d9906b74d4c19e8ad422d904aa87  guix-build-ea7b8528490d/output/dist-archive/bitcoin-ea7b8528490d.tar.gz
  09121bcd36d72ad26364182c678f2edaaf5b5e635ed43a6964ad7e4be1335075  guix-build-ea7b8528490d/output/powerpc64-linux-gnu/SHA256SUMS.part
  e8638959b501c3d70421dcdc6579c5d77c5786abed7ad3d6051c628fd8f17c6b  guix-build-ea7b8528490d/output/powerpc64-linux-gnu/bitcoin-ea7b8528490d-powerpc64-linux-gnu-debug.tar.gz
  b1aee39067d5010b4e63dc916232eb1582299d7acb56afacf5c9a5b75dcd8b82  guix-build-ea7b8528490d/output/powerpc64-linux-gnu/bitcoin-ea7b8528490d-powerpc64-linux-gnu.tar.gz
  9ed3d0e710847068cb2064e51e3f4c14166ad86f5c297adcf0a4d24cd4e2a723  guix-build-ea7b8528490d/output/powerpc64le-linux-gnu/SHA256SUMS.part
  19f99e41f41d9525a412872e61687452ab5ac23c5c964ef34b8c31a9d71adabc  guix-build-ea7b8528490d/output/powerpc64le-linux-gnu/bitcoin-ea7b8528490d-powerpc64le-linux-gnu-debug.tar.gz
  bd669150ce6830ffbd8523acf75b6daf894c9449c6ce3073a9ce89156d9268f9  guix-build-ea7b8528490d/output/powerpc64le-linux-gnu/bitcoin-ea7b8528490d-powerpc64le-linux-gnu.tar.gz
  5e42cb1fee7b5b046da37f62674a1662a6135f9644f5ea0288876334c3e8d34a  guix-build-ea7b8528490d/output/riscv64-linux-gnu/SHA256SUMS.part
  8767a84e04e0cd27318305d31a5a897cdc5f56d8505416dd01753bb35c8980b0  guix-build-ea7b8528490d/output/riscv64-linux-gnu/bitcoin-ea7b8528490d-riscv64-linux-gnu-debug.tar.gz
  00f1abcc64a7010f194dab847c04481e10991128b919b513aa9eb2911d03a243  guix-build-ea7b8528490d/output/riscv64-linux-gnu/bitcoin-ea7b8528490d-riscv64-linux-gnu.tar.gz
  ae7add2e3ea0663fdd9d2c61c737e4bb4155d42932a0a5ce8cbbbcee627f01f3  guix-build-ea7b8528490d/output/x86_64-apple-darwin/SHA256SUMS.part
  ba6b4a2f6a97e382a10ffec605f025d8724ac2dc0a441dbb42be79e915e98738  guix-build-ea7b8528490d/output/x86_64-apple-darwin/bitcoin-ea7b8528490d-x86_64-apple-darwin-unsigned.dmg
  67914f29d9e81ee50b340c3ac05b336b98766b59b8cce153493786b04a51f18a  guix-build-ea7b8528490d/output/x86_64-apple-darwin/bitcoin-ea7b8528490d-x86_64-apple-darwin-unsigned.tar.gz
  1d813d0e905a324f48beb2ce1da980e372c529b0eadb521662063f283fbc9bf6  guix-build-ea7b8528490d/output/x86_64-apple-darwin/bitcoin-ea7b8528490d-x86_64-apple-darwin.tar.gz
  4e3d176ad9059e59e9b21919001ced2710c74229536b27ef65acef514287f7e5  guix-build-ea7b8528490d/output/x86_64-linux-gnu/SHA256SUMS.part
  cbe31f4a60aac1a72bcdedccf39c18da21a5a4e257e77c0832ad93ce722923dd  guix-build-ea7b8528490d/output/x86_64-linux-gnu/bitcoin-ea7b8528490d-x86_64-linux-gnu-debug.tar.gz
  d5f3cf53adf1d964cded3461b66199c4917715af0d84cac78c97acc92432b059  guix-build-ea7b8528490d/output/x86_64-linux-gnu/bitcoin-ea7b8528490d-x86_64-linux-gnu.tar.gz
  9b1877d00fd447222839615cfeb089a8237f1d5d80cd2b819dd66621df8f2375  guix-build-ea7b8528490d/output/x86_64-w64-mingw32/SHA256SUMS.part
  2fb88e40dd399a2dedb1a44c5bd1091899b00a6d3b8d99149f622a0069205560  guix-build-ea7b8528490d/output/x86_64-w64-mingw32/bitcoin-ea7b8528490d-win64-debug.zip
  074ce203494dc15c916c151426438f09982a270f389cd139d3f28a75aea51af8  guix-build-ea7b8528490d/output/x86_64-w64-mingw32/bitcoin-ea7b8528490d-win64-setup-unsigned.exe
  2aa8ff735ce96580fb915757ccce79dc2a19afb043a6c93d3749023ca647666d  guix-build-ea7b8528490d/output/x86_64-w64-mingw32/bitcoin-ea7b8528490d-win64-unsigned.tar.gz
  0ac19e1901506c47d5b20297d468dbaa93e8572232671a16d08355988df8e7c3  guix-build-ea7b8528490d/output/x86_64-w64-mingw32/bitcoin-ea7b8528490d-win64.zip
  ```

ACKs for top commit:
  fanquake:
    ACK ea7b852849

Tree-SHA512: 75c154d2195e2e8cf2e2786c08bc5096b07027715f559fc81c1ce221ce74c2732e745b74809d7c4a9f909a1018399a07329abd993e50ecec91dd95518fce0b39
2023-05-16 11:28:05 +01:00
fanquake
d819840f38 Merge bitcoin/bitcoin#27041: Build: Improve handling of suppressed logging in Makefiles
1b1ffbd014 Build: Log when test -f fails in Makefile (TheCharlatan)
541012e621 Build: Use AM_V_GEN in Makefiles where appropriate (TheCharlatan)

Pull request description:

  This PR triages some behavior around Makefile recipe echoing suppression.

  When generating new files as part of the Makefile the recipe is sometimes suppressed with $(AM_V_GEN) and sometimes with `@`. We should prefer $(AM_V_GEN), since this also prints the lines in silent mode. This is arguably more in style with the current recipe echoing.

  Before:
  `Generated test/data/script_tests.json.h`
  Now:
  `  GEN      test/data/script_tests.json.h`

  A side effect of this change is that the recipe for generating build.h is now echoed on each make run. Arguably this makes its generation more transparent.

  Sometimes the error emitted by `test -f` is currently thrown without any logging. This makes it a bit harder to debug. Instead, print a helpful log message to point the developer in the right direction.

  Alternatively this could have been implemented by just removing the recipe echo suppression (@), but the subsequent make output became too noisy.

ACKs for top commit:
  fanquake:
    ACK 1b1ffbd014

Tree-SHA512: e31869fab25e72802b692ce6735f9561912caea903c1577101b64c9cb115c98de01a59300e8ffe7b05b998345c1b64a79226231d7d1453236ac338c62dc9fbb3
2023-05-16 11:13:11 +01:00
fanquake
a09269a146 guix: document when certain guix patches can be dropped 2023-05-16 10:10:28 +01:00
fanquake
f7036a4745 Merge bitcoin/bitcoin#27664: docs: fix spelling errors
e9dcac1ec7 add `lief` to `spelling.ignore-words` (brunoerg)
258f93000b test: fix spelling in `interface_usdt_utxocache` (brunoerg)

Pull request description:

  Add `lief` to `spelling.ignore-words` since it's the name of a Python lib and fix spelling error in `interface_usdt_utxocache` (s/eariler/earlier)

ACKs for top commit:
  fanquake:
    ACK e9dcac1ec7

Tree-SHA512: f42cdbb74144c5d289d70bb9bac1179650bb32594678e15f4e18e4b2f68009999d60ff69494d4e68650d82fc1838e67515ed2f922ee84db98735ef906911ec40
2023-05-16 10:09:51 +01:00
fanquake
b34e19a2bf Merge bitcoin/bitcoin#27665: walletdb: Remove unused CreateMockWalletDatabase
0282b2126d walletdb: Remove unused CreateMockWalletDatabase (Andrew Chow)

Pull request description:

  This has been superseded by the MockableDatabase. Remove to avoid confusion as to which type of mock database to use for testing.

  I thought this was included in #26715, maybe it got lost in a rebase.

ACKs for top commit:
  furszy:
    utACK 0282b212
  brunoerg:
    crACK 0282b2126d

Tree-SHA512: 18445c4d8a4e2609ef7471c6d75297f43694b79e768f6c993a5addb1dc0e88bd12bac263c9d8e903d828ddf6bf50434f9e2f72048f4fc528e98fed8ee65123ca
2023-05-16 10:02:45 +01:00
MarcoFalke
fad09b703f ci: Remove unused errtrace trap ERR 2023-05-16 10:00:47 +02:00
Andrew Chow
0282b2126d walletdb: Remove unused CreateMockWalletDatabase
This has been superseded by the MockableDatabase. Remove to avoid
confusion as to which type of mock database to use for testing.
2023-05-15 16:14:43 -04:00
brunoerg
e9dcac1ec7 add lief to spelling.ignore-words 2023-05-15 16:31:13 -03:00
brunoerg
258f93000b test: fix spelling in interface_usdt_utxocache 2023-05-15 16:27:41 -03:00
furszy
69d43905b7 test: add coverage for wallet read write db deadlock 2023-05-15 12:29:38 -03:00
furszy
12daf6fcdc walletdb: scope bdb::EraseRecords under a single db txn
so we erase all the records atomically or abort the entire
procedure.

and, at the same time, we can share the same db txn context
for the db cursor and the erase functionality.

extra note from the Db.cursor doc:
"If transaction protection is enabled, cursors must be
opened and closed within the context of a transaction"

thus why added a `CloseCursor` call before calling to
`TxnAbort/TxnCommit`.
2023-05-15 12:23:15 -03:00
furszy
043fcb0b05 wallet: bugfix, GetNewCursor() misses to provide batch ptr to BerkeleyCursor
If the batch ptr is not passed, the cursor will not use the db active
txn context which could lead to a deadlock if the code tries to modify
the db while it is traversing it.

E.g. the 'EraseRecords()' function.
2023-05-15 12:23:15 -03:00
fanquake
d7700d3a26 Merge bitcoin/bitcoin#27661: doc, test: Document steps to reproduce TSan warning for libdb
f03a708c11 doc, test: Document steps to reproduce TSan warning for `libdb` (Hennadii Stepanov)

Pull request description:

  Requested [here](https://github.com/bitcoin/bitcoin/pull/27658#issuecomment-1547767101).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK f03a708c11

Tree-SHA512: 0c61c05d75d074df0686502739341fdbef8dd5a2d2f6cdfdd85bd0014ac43efb6fab112ee66d8d0f33f8f4695aeffc12a05923181260d81511d4e4d53b7686f2
2023-05-15 13:56:12 +01:00
Hennadii Stepanov
f03a708c11 doc, test: Document steps to reproduce TSan warning for libdb 2023-05-15 13:28:25 +01:00
fanquake
d02df7db6b Merge bitcoin/bitcoin#26715: Introduce MockableDatabase for wallet unit tests
33e2b82a4f wallet, bench: Remove unused database options from WalletBenchLoading (Andrew Chow)
80ace042d8 tests: Modify records directly in wallet ckey loading test (Andrew Chow)
b3bb17d5d0 tests: Update DuplicateMockDatabase for MockableDatabase (Andrew Chow)
f0eecf5e40 scripted-diff: Replace CreateMockWalletDB with CreateMockableWalletDB (Andrew Chow)
075962bc25 wallet, tests: Include wallet/test/util.h (Andrew Chow)
14aa4cb1e4 wallet: Move DummyDatabase to salvage (Andrew Chow)
f67a385556 wallet, tests: Replace usage of dummy db with mockable db (Andrew Chow)
33c6245ac1 Introduce MockableDatabase for wallet unit tests (Andrew Chow)

Pull request description:

  For the wallet's unit tests, we currently use either `DummyDatabase` or memory-only versions of either BDB or SQLite. The tests that use `DummyDatabase` just need a `WalletDatabase` so that the `CWallet` can be constructed, while the tests using the memory-only databases just need a backing data store. There is also a `FailDatabase` that is similar to `DummyDatabase` except it fails be default or can have a configured return value. Having all of these different database types can make it difficult to write tests, particularly tests that work when either BDB or SQLite is disabled.

  This PR unifies all of these different unit test database classes into a single `MockableDatabase`. Like `DummyDatabase`, most functions do nothing and just return true. Like `FailDatabase`, the return value of some functions can be configured on the fly to test various failure cases. Like the memory-only databases, records can actually be written to the `MockableDatabase` and be retrieved later, but all of this is still held in memory. Using `MockableDatabase` completely removes the need for having BDB or SQLite backed wallets in the unit tests for the tests that are not actually testing specific database behaviors.

  Because `MockableDatabase`s can be created by each unit test, we can also control what records are stored in the database. Records can be added and removed externally from the typical database modification functions. This will give us greater ability to test failure conditions, particularly those involving corrupted records.

  Possible alternative to #26644

ACKs for top commit:
  furszy:
    ACK 33e2b82
  TheCharlatan:
    ACK 33e2b82a4f

Tree-SHA512: c2b09eff9728d063d2d4aea28a0f0e64e40b76483e75dc53f08667df23bd25834d52656cd4eafb02e552db0b9e619cfdb1b1c65b26b5436ee2c971d804768bcc
2023-05-15 11:39:43 +01:00
fanquake
b2c85bd82f Merge bitcoin/bitcoin#27656: build, doc: Adjust comment after PR27254
3ece0ebf62 build, doc: Adjust comment after PR27254 (Hennadii Stepanov)

Pull request description:

  This is a follow up for https://github.com/bitcoin/bitcoin/pull/27254.

ACKs for top commit:
  TheCharlatan:
    ACK 3ece0ebf62

Tree-SHA512: 36c627524304f0ea964383488acb9b16d0b553cc9cf3bce7a12aec65a7905b13e4582b7b7ec6d1efa32cd3c623969bc00f805ff31d9e6eec86a614e75796c8ff
2023-05-15 11:18:17 +01:00
fanquake
3a63ef5020 Merge bitcoin/bitcoin#27616: ci: Remove CI_EXEC bloat
fa01c3c59c ci: Remove CI_EXEC bloat (MarcoFalke)
fa8a428c92 move-only: Move almost all CI_EXEC code to 06_script_b.sh (MarcoFalke)

Pull request description:

  `CI_EXEC` has many issues:

  * It is roughly equivalent to `bash -c "$*"`, meaning that the full command will be treated as a single string, ignoring tokens.
  * It must be put in front of (almost) every command, making it easy to forget, hard to debug the resulting failure, and the code verbose.

  Fix all issues by removing it almost completely.

ACKs for top commit:
  TheCharlatan:
    ACK fa01c3c59c

Tree-SHA512: 4a65d61f5c35ca945d31f270dba3e96305fd83333a7713f0452c67f02a78e1901113e9f18d21e1dc016403c0033eb32038a9308d0a0ded7ee6b970d18381a1c2
2023-05-15 11:15:25 +01:00
MarcoFalke
facbcd3742 doc: Remove unused NO_BLOOM_VERSION constant 2023-05-15 10:18:58 +02:00
Hennadii Stepanov
3ece0ebf62 build, doc: Adjust comment after PR27254 2023-05-14 16:29:23 +01:00
Antoine Riard
89df7987c2 Add wallets_conflicts
Test the case of a tx being conflicted by multiple
txs with different depths. The conflicted tx is also spent by
a child tx for which confirmation status is tied to the parent's.
After a reorg of conflicting txs, the conflicted status should be
undone properly.

Co-authored-by: furszy <mfurszy@protonmail.com>
2023-05-14 10:45:27 -04:00
ishaanam
dced203162 wallet, tests: mark unconflicted txs as inactive
In `blockDisconnected`, for each transaction in the block, look
for any wallet transactions spending the same inputs. If any of
these transactions were marked conflicted, they are now marked as
inactive.

Co-authored-by: ariard <antoine.riard@gmail.com>
2023-05-14 10:45:21 -04:00
fanquake
29c36f0706 Merge bitcoin/bitcoin#27493: depends: no-longer nuke libc++abi.so* in native_clang package
9ae854da19 depends: no-longer nuke libc++abi.so* in native_clang package (fanquake)

Pull request description:

  We weren't copying it over in any case.

ACKs for top commit:
  hebasto:
    ACK 9ae854da19
  theuni:
    Sure. utACK no-op 9ae854da19.

Tree-SHA512: 3cc7f18f27c1b498f930bc1a09283aa04ba673d3c1a5220d8462213f0a06b74bc34989f23404402850de518cba35ddab900a54f7f0fac112fc86664e4155f8cb
2023-05-14 11:15:28 +01:00
fanquake
8bfe503ec6 Merge bitcoin/bitcoin#27628: build: Fix shared lib linking for darwin with lld
67aacc73ea build: cleanup comments after adding yet another libtool hack (Cory Fields)
283d95516a build: Fix shared lib linking for darwin with lld (Cory Fields)

Pull request description:

  Solves one of the last remaining blockers for #21778. Fixes lld linking shared libs for macos via libtool.

  lld fails one of libtool's earliest checks [because it happens to output a warning that contains a specific string](https://git.savannah.gnu.org/cgit/libtool.git/tree/m4/libtool.m4#n999):

  >     # If there is a non-empty error log, and "single_module"
  >     # appears in it, assume the flag caused a linker warning

  And here is the test being run:
  > x86_64-apple-darwin-ld: warning: Option `-single_module' is deprecated in ld64:
  > x86_64-apple-darwin-ld: warning: Unnecessary option: this is already the default

  Because the warning is printed the test fails. So libtool falls back to a very primitive and broken link-line for shared libs.

  Arguably this should be worked-around in upstream lld by changing the warning string, as otherwise every libtool project will fail to link with it.

  Like many other libtool hacks, the solution is to simply disable the check and hard-code the answer we know to be correct.

ACKs for top commit:
  hebasto:
    re-ACK 67aacc73ea

Tree-SHA512: 792e4d208a3a4921edb5f267f43ecd052b5b650df0db5cb2788ee1e4f3c4087413f354b22e407ff5fa2f99a22a16154ec6826d14c6654a57c00aae3b3e744bca
2023-05-14 11:13:32 +01:00
MarcoFalke
ddddf4957b ci: Run iwyu on all src files
This makes it easier to look at the CI output of a file without having
to manually add it first.
2023-05-13 10:45:52 +02:00
Cory Fields
67aacc73ea build: cleanup comments after adding yet another libtool hack 2023-05-12 19:16:24 +00:00
Cory Fields
283d95516a build: Fix shared lib linking for darwin with lld
libtool gets a false-positive from the warning produced by lld -single_module
because it is already the default and unneeded.

Skip the check unconditionally for Darwin linkers.
2023-05-12 19:13:57 +00:00
MarcoFalke
faf4315c88 test: Return dict in MiniWallet::send_to 2023-05-12 15:26:50 +02:00
Andrew Chow
9d85c03620 Merge bitcoin/bitcoin#27629: doc: remove version number from bips.md
308caf326d doc: remove version number from bips.md (fanquake)

Pull request description:

  This always just needs "bumping" (see previous rc type pulls), and the version number is already whichever version of the code you acquired bips.md with.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 308caf326d
  achow101:
    ACK 308caf326d
  theStack:
    ACK 308caf326d
  hebasto:
    ACK 308caf326d

Tree-SHA512: fcb98e7cdc0c1f8960bfba86be09c2badb36b613060fae394a56e1561c69d28f433434f573c8b1ae1d71ae326277dea2a4841d5c08ad39f8e8848300743146e7
2023-05-11 13:43:32 -04:00
fanquake
308caf326d doc: remove version number from bips.md
This always just needs "bumping", and the version number is already
whichever version of the code you acquired bips.md with.
2023-05-11 17:49:16 +01:00
fanquake
137a98c5a2 Merge bitcoin/bitcoin#27610: Improve performance of p2p inv to send queues
5b3406094f net_processing: Boost inv trickle rate (Anthony Towns)
228e9201ef txmempool: have CompareDepthAndScore sort missing txs first (Anthony Towns)

Pull request description:

  Couple of performance improvements when draining the inventory-to-send queue:

   * drop txs that have already been evicted from the mempool (or included in a block) immediately, rather than at the end of processing
   * marginally increase outgoing trickle rate during spikes in tx volume

ACKs for top commit:
  willcl-ark:
    ACK 5b34060
  instagibbs:
    ACK 5b3406094f
  darosior:
    utACK 5b3406094f
  glozow:
    code review ACK 5b3406094f
  dergoegge:
    utACK 5b3406094f

Tree-SHA512: 155cd3b5d150ba3417c1cd126f2be734497742e85358a19c9d365f4f97c555ff9e846405bbeada13c3575b3713c3a7eb2f780879a828cbbf032ad9a6e5416b30
2023-05-11 14:20:30 +01:00
fanquake
9ae854da19 depends: no-longer nuke libc++abi.so* in native_clang package
We weren't copying it over in any case.
2023-05-11 12:29:52 +01:00
fanquake
c2f2abd0a4 Merge bitcoin/bitcoin#27125: refactor, kernel: Decouple ArgsManager from blockstorage
5ff63a09a9 refactor, blockstorage: Replace stopafterblockimport arg (TheCharlatan)
18e5ba7c80 refactor, blockstorage: Replace blocksdir arg (TheCharlatan)
02a0899527 refactor, BlockManager: Replace fastprune from arg with options (TheCharlatan)
a498d699e3 refactor/iwyu: Complete includes for blockmanager_args (TheCharlatan)
f0bb1021f0 refactor: Move functions to BlockManager methods (TheCharlatan)
cfbb212493 zmq: Pass lambda to zmq's ZMQPublishRawBlockNotifier (TheCharlatan)
8ed4ff8e05 refactor: Declare g_zmq_notification_interface as unique_ptr (TheCharlatan)

Pull request description:

  The libbitcoin_kernel library should not rely on the `ArgsManager`, but rather use option structs that can be passed to the various classes it uses. This PR removes reliance on the `ArgsManager` from the `blockstorage.*` files. Like similar prior work, it uses the options struct in the `BlockManager` that can be populated with `ArgsManager` values.

  Some related prior work: https://github.com/bitcoin/bitcoin/pull/26889 https://github.com/bitcoin/bitcoin/pull/25862 https://github.com/bitcoin/bitcoin/pull/25527 https://github.com/bitcoin/bitcoin/pull/25487

  Related PR removing blockstorage globals: https://github.com/bitcoin/bitcoin/pull/25781

ACKs for top commit:
  ryanofsky:
    Code review ACK 5ff63a09a9. Since last ACK just added std::move and fixed commit title. Sorry for the noise!
  mzumsande:
    Code Review ACK 5ff63a09a9

Tree-SHA512: 4bde8fd140a40b97eca923e9016d85dcea6fad6fd199731f158376294af59c3e8b163a0725aa47b4be3519b61828044e0a042deea005e0c28de21d8b6c3e1ea7
2023-05-11 10:28:51 +01:00
Andrew Chow
3ff67f7783 Merge bitcoin/bitcoin#19690: util: improve FindByte() performance
72efc26439 util: improve streams.h:FindByte() performance (Larry Ruane)
604df63f6c [bench] add streams findbyte (gzhao408)

Pull request description:

  This PR is strictly a performance improvement; there is no functional change. The `CBufferedFile::FindByte()` method searches for the next occurrence of the given byte in the file. Currently, this is done by explicitly inspecting each byte in turn. This PR takes advantage of `std::find()` to do the same more efficiently, improving its CPU runtime by a factor of about 25 in typical use.

ACKs for top commit:
  achow101:
    re-ACK 72efc26439
  stickies-v:
    re-ACK 72efc26439

Tree-SHA512: ddf0bff335cc8aa34f911aa4e0558fa77ce35d963d602e4ab1c63090b4a386faf074548daf06ee829c7f2c760d06eed0125cf4c34e981c6129cea1804eb3b719
2023-05-10 17:50:42 -04:00
glozow
67b7fecacd [mempool] clear mapDeltas entry if prioritisetransaction sets delta to 0
It's unnecessary to keep the data around, as it doesn't do anything. If
prioritisetransaction is called again, we'll make a new entry in
mapDeltas.

These entries are only deleted when the transaction is mined or conflicted
from a block (i.e. not in replacement or eviction), are persisted in
mempool.dat, and never expire. If node operators use the RPC to
regularly prioritise/de-prioritise transactions, these (meaningless)
map entries may hang around forever and take up valuable mempool memory.
2023-05-10 21:10:44 +01:00
glozow
c1061acb9d [functional test] prioritisation is not removed during replacement and expiry 2023-05-10 21:10:44 +01:00
glozow
0e5874f0b0 [functional test] getprioritisedtransactions RPC 2023-05-10 21:10:44 +01:00
glozow
99f8046829 [rpc] add getprioritisedtransactions
This allows the user to see prioritisation for not-in-mempool
transactions.
2023-05-10 21:10:44 +01:00
glozow
9e9ca36c80 [mempool] add GetPrioritisedTransactions 2023-05-10 21:10:44 +01:00
TheCharlatan
5ff63a09a9 refactor, blockstorage: Replace stopafterblockimport arg
Add a stop_after_block_import field to the BlockManager options. Use
this field instead of the global gArgs.

This should allow users of the BlockManager to not rely on the global
Args.
2023-05-10 19:07:46 +02:00
TheCharlatan
18e5ba7c80 refactor, blockstorage: Replace blocksdir arg
Add a blocks_dir field to the BlockManager options. Move functions
relying on the global gArgs to get the blocks_dir into the BlockManager
class.

This should eventually allow users of the BlockManager to not rely on
the global Args and instead pass in their own options.
2023-05-10 19:07:44 +02:00
TheCharlatan
02a0899527 refactor, BlockManager: Replace fastprune from arg with options
Remove access to the global gArgs for the fastprune argument and
replace it by adding a field to the existing BlockManager Options
struct.

When running `clang-tidy-diff` on this commit, there is a diagnostic
error: `unknown type name 'uint64_t' [clang-diagnostic-error] uint64_t
prune_target{0};`, which is fixed by including cstdint.

This should eventually allow users of the BlockManager to not rely on
the global gArgs and instead pass in their own options.
2023-05-10 19:07:42 +02:00
TheCharlatan
a498d699e3 refactor/iwyu: Complete includes for blockmanager_args 2023-05-10 19:07:30 +02:00
TheCharlatan
f0bb1021f0 refactor: Move functions to BlockManager methods
This is a commit in preparation for the next few commits. The functions
are moved to methods to avoid their re-declaration for the purpose of
passing in BlockManager options.

The functions that were now moved into the BlockManager should no longer
use the params as an argument, but instead use the member variable.

In the moved ReadBlockFromDisk and UndoReadFromDisk, change
the function signature to accept a reference to a CBlockIndex instead of
a raw pointer. The pointer is expected to be non-null, so reflect that
in the type.

To allow for the move of functions to BlockManager methods all call
sites require an instantiated BlockManager, or a callback to one.
2023-05-10 19:06:53 +02:00
TheCharlatan
cfbb212493 zmq: Pass lambda to zmq's ZMQPublishRawBlockNotifier
The lambda captures a reference to the chainman unique_ptr to retrieve
block data. An assert is added on the chainman to ensure that the lambda
is not used while the chainman is uninitialized.

This is done in preparation for the following commits where blockstorage
functions are made BlockManager methods.
2023-05-10 19:06:45 +02:00
Andrew Chow
e0a70c5b4f Merge bitcoin/bitcoin#27605: refactor: Replace global find_value function with UniValue::find_value method
fa266c4bbf Temporarily work around gcc-13 warning bug in interfaces_tests (MarcoFalke)
fa28850562 Fix clang-tidy performance-unnecessary-copy-initialization warnings (MarcoFalke)
faaa60a30e Remove unused find_value global function (MarcoFalke)
fa422aeec2 scripted-diff: Use UniValue::find_value method (MarcoFalke)
fa548ac872 Add UniValue::find_value method (MarcoFalke)

Pull request description:

  The global function has issues:

  * It causes gcc-13 warnings, see https://github.com/bitcoin/bitcoin/issues/26926
  * There is no rationale for it being a global function, when it acts like a member function
  * `performance-unnecessary-copy-initialization` clang-tidy isn't run on it

  Fix all issues by making it a member function.

ACKs for top commit:
  achow101:
    ACK fa266c4bbf
  hebasto:
    re-ACK fa266c4bbf

Tree-SHA512: 6c4e25da3122cd3b91c376bef73ea94fb3beb7bf8ef5cb3853c5128d95bfbacbcbfb16cc843eb7b1a7ebd350c2b6311f8085eeacf9aeeab3366987037d209e44
2023-05-10 12:56:37 -04:00
MarcoFalke
fa01c3c59c ci: Remove CI_EXEC bloat 2023-05-10 15:28:19 +02:00
kevkevin
a7b46a1fea test: added coverage to mining_basic.py
Included a test that checks if we call submitblock with
block.vtx.empty() then it throws an rpc deserialization error, currently
we only test if !block.vtx->IsCoinBase() throws an rpc deserialization
error
2023-05-10 07:50:46 -05:00
MarcoFalke
fa8a428c92 move-only: Move almost all CI_EXEC code to 06_script_b.sh
[WARN] The commit is obviously broken and will not run the CI system. In
the rare case this is hit in a git bisect, just skip the commit.

The goal here was to make it trivial to review with the git option:
--color-moved=dimmed-zebra

It is required to move everything into one file because "exit 0" will
otherwise stop working as intended when the containing bash script is no
longer executed with "source ...".

If there is desire to split up 06_script_b.sh into logical chunks in the
future, it will also be easier after the following commit.
2023-05-10 14:10:38 +02:00
fanquake
104eed1166 Merge bitcoin/bitcoin#27611: refactor: Use ChainType enum exhaustively
e23088707b refactor: Use ChainType enum exhaustively (TheCharlatan)

Pull request description:

  This is a follow up of https://github.com/bitcoin/bitcoin/pull/27491, more concretely https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188847896, for not using default cases (as per the style guide), and https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188852707 and https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188851857 for avoiding dead code.

ACKs for top commit:
  fanquake:
    ACK e23088707b - deals with almost all follow up comments out of #27491.

Tree-SHA512: 1794190b03b91d3ca349a4da08e9610dbb3432983eee7cb21ecc758d1d7d710560c97661de14cdf493c28c00ebe8977511b4696055c0940e7f815b622dbacd16
2023-05-10 12:27:18 +01:00
TheCharlatan
8ed4ff8e05 refactor: Declare g_zmq_notification_interface as unique_ptr
Ensures better memory safety for this global. This came up during
discussion of the following commit, but is not strictly required for its
implementation.
2023-05-10 12:56:46 +02:00
fanquake
883766fa45 Merge bitcoin/bitcoin#27573: ci: Remove CI_EXEC bloat in test/06_script_b.sh
fa1dbd04ca ci: Remove CI_EXEC bloat in test/06_script_b.sh (MarcoFalke)
fae8de926a ci: Move CI container kill out of 06_script_b.sh (MarcoFalke)
fa7d75540e ci: Pass full env to CI pod to avoid missing a var (MarcoFalke)

Pull request description:

  `CI_EXEC` has many issues:

  * It is roughly equivalent to `bash -c "$*"`, meaning that the full command will be treated as a single string, ignoring tokens.
  * It must be put in front of (almost) every command, making it easy to forget, hard to debug the resulting failure, and the code verbose.

  Fix all issues in one script by removing it.

ACKs for top commit:
  fanquake:
    ACK fa1dbd04ca - this conflicts with #27125, but that is going to be rebased soon, and this could be merged in the interim. cc TheCharlatan
  TheCharlatan:
    ACK fa1dbd04ca

Tree-SHA512: e5ab5503a05a787f2bc6ca25e71ad3dc166aade57e25d9677e72b1ca4e5fb6045c058dfd55f47ac93f710538e62d57c12cd7eb9d1260c6f55f3c8091908dc70d
2023-05-10 11:55:29 +01:00
Hennadii Stepanov
a94d75fa81 msvc: Do not define HAVE_CONSENSUS_LIB
The `HAVE_CONSENSUS_LIB` symbol is supposed to be defined when a
user-exposed shared library (DLL) is built which is not the case here.
2023-05-10 11:26:51 +01:00
Hennadii Stepanov
cf6ff1031b msvc: Clean up libbitcoin_consensus source files
See `libbitcoin_consensus_a_SOURCES` in the `src/Makefile.am`.
2023-05-10 11:26:44 +01:00
Hennadii Stepanov
30aee016f1 scripted-diff: Rename libbitcoinconsensus to libbitcoin_consensus
This build artifact is not a user-exposed shared library (DLL) but a
convenience static library.
See: https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md

-BEGIN VERIFY SCRIPT-
sed -i 's/libbitcoinconsensus/libbitcoin_consensus/g' $(git grep -l "libbitcoinconsensus" -- build_msvc)
git mv build_msvc/libbitcoinconsensus build_msvc/libbitcoin_consensus
git mv build_msvc/libbitcoin_consensus/libbitcoinconsensus.vcxproj build_msvc/libbitcoin_consensus/libbitcoin_consensus.vcxproj
-END VERIFY SCRIPT-
2023-05-10 11:26:35 +01:00
fanquake
3777c75d14 Merge bitcoin/bitcoin#27604: add ryanofsky to trusted-keys
59ebee3fb4 add ryanofsky to trusted-keys (Ryan Ofsky)

Pull request description:

  For maintaining interfaces and other areas of the codebase. Some previous discussion in IRC meeting https://gnusha.org/bitcoin-core-dev/2023-05-04.log

ACKs for top commit:
  darosior:
    ACK 59ebee3fb4 for adding ryanofsky as a maintainer.
  hebasto:
    ACK 59ebee3fb4
  glozow:
    ACK 59ebee3fb4
  pinheadmz:
    ACK 59ebee3fb4
  stickies-v:
    utACK 59ebee3fb4
  brunoerg:
    ACK 59ebee3fb4
  theStack:
    ACK 59ebee3fb4
  mzumsande:
    ACK 59ebee3fb4
  theuni:
    ACK 59ebee3fb4.
  jarolrod:
    ACK 59ebee3fb4

Tree-SHA512: e3d2815d8950e419316ee49ec70f01cb1939de61b3017b8140c0194b519b5b523a618d3ad2ab9fe3fd32543649c1465fdd6baf52ad68da48b680bd4898186ff4
2023-05-10 10:19:05 +01:00
fanquake
dbfc748d3d Merge bitcoin/bitcoin#27608: p2p: Avoid prematurely clearing download state for other peers
52e52071e0 p2p: Avoid prematurely clearing download state for other peers (Suhas Daftuar)

Pull request description:

  Avoid letting one peer send us data that clears out the download request (and related timers etc) from another peer.

  The exception is if a block is definitely stored to disk, in which case we'll clear the download state (just as we do for compact blocks).

ACKs for top commit:
  jamesob:
    ACK 52e52071e0 ([`jamesob/ackr/27608.1.sdaftuar.p2p_avoid_prematurely_cl`](https://github.com/jamesob/bitcoin/tree/ackr/27608.1.sdaftuar.p2p_avoid_prematurely_cl))
  instagibbs:
    ACK 52e52071e0
  fjahr:
    Code review ACK 52e52071e0
  mzumsande:
    Code Review ACK 52e52071e0

Tree-SHA512: 3ee92507edc3303c16c70ca44ba6c28c104afe95196e4b9167032590ed23d4f569f654f8eb8758940bd6536bc9ca810d2a77d2739db386b927e8b3f3cf55cb16
2023-05-10 10:12:55 +01:00
TheCharlatan
e23088707b refactor: Use ChainType enum exhaustively
This is a follow up of https://github.com/bitcoin/bitcoin/pull/27491,
more concretely
https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188847896,
for not using default cases (as per the style guide), and
https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188852707 and
https://github.com/bitcoin/bitcoin/pull/27491#discussion_r1188851857 for
avoiding dead code.

Also change chain name to chain type in docstrings
2023-05-10 10:39:58 +02:00
Anthony Towns
5b3406094f net_processing: Boost inv trickle rate
If transactions are being added to the mempool at a rate faster than 7tx/s
(INVENTORY_BROADCAST_PER_SECOND) then peers' inventory_to_send queue can
become relatively large. If this happens, increase the number of txids
we include in an INV message (normally capped at 35) by 5 for each 1000
txids in the queue.

This will tend to clear a temporary excess out reasonably quickly; an
excess of 4000 invs to send will be cleared down to 1000 in about 30
minutes, while an excess of 20000 invs would be cleared down to 1000 in
about 60 minutes.
2023-05-10 10:51:26 +10:00
Anthony Towns
228e9201ef txmempool: have CompareDepthAndScore sort missing txs first
We use CompareDepthAndScore to choose an order of txs to inv. Rather
than sorting txs that have been evicted from the mempool at the end
of the list, sort them at the beginning so they are removed from
the queue immediately.
2023-05-10 10:51:26 +10:00
MarcoFalke
fa266c4bbf Temporarily work around gcc-13 warning bug in interfaces_tests
This can be reverted once gcc excludes lambdas with decltype(auto)
return type from its -Wdangling-reference analysis.
2023-05-09 20:27:05 +02:00
Suhas Daftuar
52e52071e0 p2p: Avoid prematurely clearing download state for other peers 2023-05-09 13:52:13 -04:00
MarcoFalke
fa28850562 Fix clang-tidy performance-unnecessary-copy-initialization warnings 2023-05-09 18:48:52 +02:00
MarcoFalke
faaa60a30e Remove unused find_value global function 2023-05-09 18:48:10 +02:00
MarcoFalke
fa422aeec2 scripted-diff: Use UniValue::find_value method
-BEGIN VERIFY SCRIPT-
 sed --regexp-extended -i 's/find_value\(([^ ,]+), /\1.find_value(/g' $(git grep -l find_value)
-END VERIFY SCRIPT-
2023-05-09 18:47:14 +02:00
MarcoFalke
fa548ac872 Add UniValue::find_value method 2023-05-09 18:47:14 +02:00
fanquake
fc06881f13 Merge bitcoin/bitcoin#27491: refactor: Move chain constants to the util library
d168458d1f scripted-diff: Remove unused chainparamsbase includes (TheCharlatan)
e9ee8aaf3a Add missing definitions in prep for scripted diff (TheCharlatan)
ba8fc7d788 refactor: Replace string chain name constants with ChainTypes (TheCharlatan)
401453df41 refactor: Introduce ChainType getters for ArgsManager (TheCharlatan)
bfc21c31b2 refactor: Create chaintype files (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is also a follow up to #26177.

  It replaces pull request https://github.com/bitcoin/bitcoin/pull/27294, which just moved the constants to a new file, but did not re-declare them as enums.

  The code move of the chain name constants out of the `chainparamsbase` to their own separate header allows the kernel `chainparams` to no longer include `chainparamsbase`. The `chainparamsbase` contain references to the `ArgsManager` and networking related options that should not belong to the kernel library. Besides this move, the constants are re-declared as enums with helper functions facilitating string conversions.

ACKs for top commit:
  ryanofsky:
    Code review ACK d168458d1f. Just suggested changes since last review.

Tree-SHA512: ac2fbe5cbbab4f52eae1e30af1f16700b6589eb4764c328a151a712adfc37f326cc94a65c385534c57d4bc92cc1a13bf1777d92bc924a20dbb30440e7380b316
2023-05-09 15:42:21 +01:00
fanquake
d5ff96f920 Merge bitcoin/bitcoin#27594: refactor: Remove unused GetTimeMillis
fae1d9cded refactor: Remove unused GetTimeMillis (MarcoFalke)

Pull request description:

  The function is unused, not type-safe, and does not denote the underlying clock type. So remove it.

ACKs for top commit:
  willcl-ark:
    tACK fae1d9cded

Tree-SHA512: 41ea7125d1964192b85a94265be974d02bf1e79b1feb61bff11486dc0ac811745156940ec5cad2ad1f94b653936f8ae563c959c1c4142203a55645fcb83203e8
2023-05-09 15:21:56 +01:00
TheCharlatan
d168458d1f scripted-diff: Remove unused chainparamsbase includes
This is a follow-up to previous commits moving the chain constants out
of chainparamsbase.

The script removes the chainparamsbase header in all files where it is
included, but not used. This is done by filtering against all defined
symbols of the header as well as its respective .cpp file.

The kernel chainparams now no longer relies on chainparamsbase.

-BEGIN VERIFY SCRIPT-
sed -i '/#include <chainparamsbase.h>/d' $( git grep -l 'chainparamsbase.h' | xargs grep -L 'CBaseChainParams\|CreateBaseChainParams\|SetupChainParamsBaseOptions\|BaseParams\|SelectBaseParams\|chainparamsbase.cpp' )
-END VERIFY SCRIPT-
2023-05-09 15:49:19 +02:00
TheCharlatan
e9ee8aaf3a Add missing definitions in prep for scripted diff
The missing include and ArgsManager were found after applying the
scripted diff in the following commit.
2023-05-09 15:49:17 +02:00
TheCharlatan
ba8fc7d788 refactor: Replace string chain name constants with ChainTypes
This commit effectively moves the definition of these constants
out of the chainparamsbase to their own file.

Using the ChainType enums provides better type safety compared to
passing around strings.

The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
2023-05-09 15:49:14 +02:00
fanquake
b13830eff6 Merge bitcoin/bitcoin#27575: Introduce platform-agnostic ALWAYS_INLINE macro
3f19875d66 scripted-diff: Use platform-agnostic `ALWAYS_INLINE` macro (Hennadii Stepanov)
e16c22fe02 Introduce platform-agnostic `ALWAYS_INLINE` macro (Hennadii Stepanov)

Pull request description:

  Split from https://github.com/bitcoin/bitcoin/pull/24773 as requested in https://github.com/bitcoin/bitcoin/pull/24773#issuecomment-1534954977.

ACKs for top commit:
  theuni:
    utACK 3f19875d66
  fanquake:
    ACK 3f19875d66

Tree-SHA512: a19b713433bb4d3c5fff1ddb4d1413837823a400c1d46363a8181e7632b059846ba92264be1c867f35f532af90945ed20887103471b09c07623e0f3905b4098b
2023-05-09 14:45:52 +01:00
TheCharlatan
401453df41 refactor: Introduce ChainType getters for ArgsManager
These are introduced for the next commit where the usage of the
ChainType is adopted throughout the code.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-05-09 15:38:37 +02:00
TheCharlatan
bfc21c31b2 refactor: Create chaintype files
This is the first of a number of commits with the goal of moving the
chain type definitions out of chainparamsbase to their own file and
implementing them as enums instead of constant strings. The goal is to
allow the kernel chainparams to no longer include chainparamsbase.

The commit is part of an ongoing effort to decouple the libbitcoinkernel
library from the ArgsManager and other functionality that should not be
part of the kernel library.
2023-05-09 11:33:09 +02:00
Ryan Ofsky
59ebee3fb4 add ryanofsky to trusted-keys 2023-05-08 23:30:56 -04:00
Andrew Chow
fa53611cf1 Merge bitcoin/bitcoin#26076: Switch hardened derivation marker to h
fe49f06c0e doc: clarify PR 26076 release note (Sjors Provoost)
bd13dc2f46 Switch hardened derivation marker to h in descriptors (Sjors Provoost)

Pull request description:

  This makes it easier to handle descriptor strings manually, especially when importing from another Bitcoin Core wallet.

  For example the `importdescriptors` RPC call is easiest to use `h` as the marker: `'["desc": ".../0h/..."]'`, avoiding the need for escape characters. With this change `listdescriptors` will use `h`, so you can copy-paste the result, without having to add escape characters or switch `'` to 'h' manually.

  Both markers can still be parsed.

  The `hdkeypath` field in `getaddressinfo` is also impacted by this change, except for legacy wallets. The latter is to prevent accidentally breaking ancient software that uses our legacy wallet.

  See discussion in #15740

ACKs for top commit:
  achow101:
    ACK fe49f06c0e
  darosior:
    re-ACK fe49f06c0e

Tree-SHA512: f78bc873b24a6f7a2bf38f5dd58f2b723e35e6b10e4d65c36ec300e2d362d475eeca6e5afa04b3037ab4bee0bf8ebc93ea5fc18102a2111d3d88fc873c08dc89
2023-05-08 13:31:28 -04:00
fanquake
26cb32c02d Merge bitcoin/bitcoin#27580: msvc: Cleanup after upgrading libsecp256k1 up to 0.3.0
d9b54c46cc msvc: Cleanup after upgrading libsecp256k1 up to 0.3.0 (Hennadii Stepanov)

Pull request description:

  libsecp256k1 [v0.3.0](https://github.com/bitcoin-core/secp256k1/blob/master/CHANGELOG.md#030---2023-03-08):
  > Removed the configuration header `src/libsecp256k1-config.h`.

  This PR removed the code that has been unused since https://github.com/bitcoin/bitcoin/pull/27230.

  The `USE_ASM_X86_64` is now undefined explicitly (but actually it seems a bit redundant).

  The `ECMULT_GEN_PREC_BITS` and `ECMULT_WINDOW_SIZE` macros are defined by the source code to their defaults.

  ---

  Considering the upcoming CMake-based build system, these changes have a low-priority.

ACKs for top commit:
  fanquake:
    ACK d9b54c46cc

Tree-SHA512: f279aeee1da57af5fdc4bd4f2000f1fea4180895f0e5b576545092a8318c756d36192f09a0cb0929cef74ed384c46777d5e6b6f92f4542b308e984e4abf473dc
2023-05-08 15:12:58 +01:00
Sjors Provoost
fe49f06c0e doc: clarify PR 26076 release note 2023-05-08 16:07:15 +02:00
MarcoFalke
fae1d9cded refactor: Remove unused GetTimeMillis
The function is unused, not type-safe, and does not denote the
underlying clock type. So remove it.
2023-05-08 12:40:48 +02:00
ishaanam
1bce12acd3 test: add test for descriptorprocesspsbt RPC 2023-05-06 10:03:25 -04:00
fanquake
322ec63b01 Merge bitcoin/bitcoin#17860: fuzz: BIP 30, CVE-2018-17144
fa2d8b61f9 fuzz: BIP 42, BIP 30, CVE-2018-17144 (MarcoFalke)
faae7d5c00 Move LoadVerifyActivateChainstate to ChainTestingSetup (MarcoFalke)
fa26e3462a Avoid dereferencing interruption_point if it is nullptr (MarcoFalke)
fa846ee074 test: Add util to mine invalid blocks (MarcoFalke)

Pull request description:

  Add a validation fuzz test for BIP 30 and CVE-2018-17144

ACKs for top commit:
  dergoegge:
    Code review ACK fa2d8b61f9
  mzumsande:
    Tested ACK fa2d8b61f9

Tree-SHA512: 1f4620cc078709487abff24b304a6bb4eeab2e7628b392e2bc6de9cc0ce6745c413388ede6e93025d0c56eec905607ba9786633ef183e5779bf5183cc9ff92c0
2023-05-06 12:13:06 +01:00
fanquake
e460c0a24a Merge bitcoin/bitcoin#27405: util: Use steady clock instead of system clock to measure durations
fa83fb3161 wallet: Use steady clock to calculate number of derive iterations (MarcoFalke)
fa2c099cec wallet: Use steady clock to measure scanning duration (MarcoFalke)
fa97621804 qt: Use steady clock to throttle GUI notifications (MarcoFalke)
fa1d8044ab test: Use steady clock in index tests (MarcoFalke)
fa454dcb20 net: Use steady clock in InterruptibleRecv (MarcoFalke)

Pull request description:

  `GetTimeMillis` has multiple issues:

  * It doesn't denote the underlying clock type
  * It isn't type-safe
  * It is used incorrectly in places that should use a steady clock

  Fix all issues here.

ACKs for top commit:
  willcl-ark:
    ACK fa83fb3161
  martinus:
    Code review ACK fa83fb3161, also ran all tests. All usages of the steady_clock are just for duration measurements, so the change to a different epoch is ok.

Tree-SHA512: 5ec4fede8c7f97e2e08863c011856e8304f16ba30a68fdeb42f96a50a04961092cbe46ccf9ea6ac99ff5203c09f9e0924eb483eb38d7df0759addc85116c8a9f
2023-05-06 12:03:50 +01:00
fanquake
4a72af9ca4 Merge bitcoin/bitcoin#27584: ci: fix asan task name
bf07e3a47e ci: fix asan task name (fanquake)

Pull request description:

  Pointed out in https://github.com/bitcoin/bitcoin/issues/27380#issuecomment-1536434598.

Top commit has no ACKs.

Tree-SHA512: 82e83443844b9ddad039fc4d3eda5a6c84ce924ea703cdd9d0ef3af3d44e45c24cdc085fdae9b48ff22dc116b700459043ec179173ddae14bef434342cdadaa9
2023-05-06 11:49:04 +01:00
fanquake
ccd4db7d62 Merge bitcoin/bitcoin#27570: refactor: Remove need to pass chainparams to BlockManager methods
fa5d7c39eb Remove unused chainparams from BlockManager methods (MarcoFalke)
fa3f74a40e Replace pindex pointer with block reference (MarcoFalke)
facdb8b331 Add BlockManagerOpts::chainparams reference (MarcoFalke)

Pull request description:

  Seems confusing to pass chainparams to each method individually, when the params can't change anyway for the whole lifetime of the block manager, and also must be equal to the ones used by the chainstate manager.

  Fix this issue by removing them from the methods and instead storing a reference once in a member field.

ACKs for top commit:
  dergoegge:
    Code review ACK fa5d7c39eb
  TheCharlatan:
    ACK fa5d7c39eb

Tree-SHA512: b44e2466b70a2a39a46625d618ce3173897ef30418db4efb9ff73d0eb2c082633204a5586c34b95f227e6711e64f19f12d5ac0f9f34692d40cb372e98389324b
2023-05-05 17:45:09 +01:00
fanquake
bf07e3a47e ci: fix asan task name
Pointed out in
https://github.com/bitcoin/bitcoin/issues/27380#issuecomment-1536434598.
2023-05-05 16:58:23 +01:00
fanquake
5d1014d5a1 Merge bitcoin/bitcoin#27574: doc: Add post branch-off note about fuzz input pruning
9143b6988b [doc] Add post branch-off note about fuzz input pruning (dergoegge)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 82658faaa31668591853703604edb45ce24ee703b8f4077ab690865f3674e154f76c55c3b523f543a862aab9707d70a46c8bf4d41b51d0002635806413921017
2023-05-05 16:36:37 +01:00
fanquake
5566405a95 Merge bitcoin/bitcoin#27554: test: Treat bitcoin-wallet binary in the same way as others
f6d7636be4 test: Treat `bitcoin-wallet` binary in the same way as others (Hennadii Stepanov)
dda961cec5 test, refactor: Add `set_binary_paths` function (Hennadii Stepanov)

Pull request description:

  This PR makes the `bitcoin-wallet` binary path customizable in the same way how it can be done now with other ones, including `bitcoind`, `bitcoin-cli` and `bitcoin-util`.

ACKs for top commit:
  stickies-v:
    re-ACK f6d7636be4

Tree-SHA512: 480fae14c5440e530ba78a2be19eaaf642260070435e533fc7ab98ddcc2fcac7ad83f2c7e7c6706db3167e8391d7d4abf8784889796c218c2d5bba043144e787
2023-05-05 16:34:37 +01:00
fanquake
b11bd045e4 Merge bitcoin/bitcoin#26653: test, init: perturb file to ensure failure instead of only deleting them
c371cae07a test, init: perturb file to ensure failure instead of only deleting them (brunoerg)

Pull request description:

  In `feature_init.py` there is a TODO about perturbing the files instead of only testing by deleting them.
  ```py
              # TODO: at some point, we should test perturbing the files instead of removing
              # them, e.g.
              #
              # contents = target_file.read_bytes()
              # tweaked_contents = bytearray(contents)
              # tweaked_contents[50:250] = b'1' * 200
              # target_file.write_bytes(bytes(tweaked_contents))
              #
              # At the moment I can't get this to work (bitcoind loads successfully?) so
              # investigate doing this later.
  ```

  This PR adds it by writing into the file random bytes and checking whether it throws an error when starting.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK c371cae07a

Tree-SHA512: d691eee60b91dd9d1b200588608f56b0a10dccd9761a75254b69e0ba5e5866cae14d2f90cb2bd7ec0f95b0617c2562cd33f20892ffd16355b6df770d3806a0ff
2023-05-05 14:13:03 +01:00
Hennadii Stepanov
f6d7636be4 test: Treat bitcoin-wallet binary in the same way as others
This change makes the `bitcoin-wallet` binary path customizable in the
same way how it can be done now with other ones, including `bitcoind`,
`bitcoin-cli` and `bitcoin-util`.
2023-05-05 13:35:30 +01:00
Hennadii Stepanov
dda961cec5 test, refactor: Add set_binary_paths function
This change factors out the repeated code into a new `set_binary_paths`
function.
2023-05-05 13:35:06 +01:00
Larry Ruane
72efc26439 util: improve streams.h:FindByte() performance
Avoid use of the expensive mod operator (%) when calculating the
buffer offset. No functional difference.

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2023-05-05 06:03:17 -06:00
gzhao408
604df63f6c [bench] add streams findbyte 2023-05-05 06:03:14 -06:00
MarcoFalke
fa2d8b61f9 fuzz: BIP 42, BIP 30, CVE-2018-17144 2023-05-05 13:31:01 +02:00
MarcoFalke
faae7d5c00 Move LoadVerifyActivateChainstate to ChainTestingSetup 2023-05-05 13:19:09 +02:00
Hennadii Stepanov
d9b54c46cc msvc: Cleanup after upgrading libsecp256k1 up to 0.3.0 2023-05-05 10:58:15 +01:00
MarcoFalke
fa1dbd04ca ci: Remove CI_EXEC bloat in test/06_script_b.sh 2023-05-05 08:51:43 +02:00
MarcoFalke
fae8de926a ci: Move CI container kill out of 06_script_b.sh
This cleans up 06_script_b.sh to only contain code to be executed inside
the CI pod, which avoids confusion and is needed for the next commit.
2023-05-05 08:50:39 +02:00
MarcoFalke
fa7d75540e ci: Pass full env to CI pod to avoid missing a var
Instead of enumerating each passed env var, just pass all. This avoids
the risk of missing to enumerate one. Also, it is less code.

The risk could be that an env var causes non-deterministic behavior, but
this can be fixed by explicitly excluding it once the issue is known.

Values with newlines can not be stored in the file and parsed by
docker/podman, so they are excluded.
2023-05-05 08:46:35 +02:00
Hennadii Stepanov
3f19875d66 scripted-diff: Use platform-agnostic ALWAYS_INLINE macro
-BEGIN VERIFY SCRIPT-
sed -i 's/ inline __attribute__((always_inline)) / ALWAYS_INLINE /g' $(git grep -l "inline __attribute__((always_inline))")
sed -i 's/ inline  __attribute__((always_inline)) / ALWAYS_INLINE /g' $(git grep -l "inline  __attribute__((always_inline))")
-END VERIFY SCRIPT-
2023-05-04 20:58:01 +01:00
Hennadii Stepanov
e16c22fe02 Introduce platform-agnostic ALWAYS_INLINE macro
`<attributes.h>` has been included in anticipation of the following
commit.
2023-05-04 20:57:51 +01:00
dergoegge
9143b6988b [doc] Add post branch-off note about fuzz input pruning 2023-05-04 19:39:57 +02:00
MarcoFalke
fa5d7c39eb Remove unused chainparams from BlockManager methods
Also, replace pointer with reference while touching the signature.
2023-05-04 19:27:23 +02:00
MarcoFalke
fa3f74a40e Replace pindex pointer with block reference
pindex can not be nullptr, so document that, and clear it up in the next
commit.
2023-05-04 19:26:48 +02:00
MarcoFalke
facdb8b331 Add BlockManagerOpts::chainparams reference
and use it in blockstorage.cpp
2023-05-04 19:26:43 +02:00
ishaanam
fb2a3a70e8 rpc: add descriptorprocesspsbt rpc
This RPC can be the Updater, Signer, and optionally the Input
Finalizer for a psbt, and has no interaction with the Bitcoin
Core wallet.
2023-05-04 13:06:39 -04:00
fanquake
6c7ebcc14b Merge bitcoin/bitcoin#27422: test: add coverage to rpc_scantxoutset.py
7e3d4f8e86 test: add coverage to ensure the first arg of scantxoutset is needed (ismaelsadeeq)

Pull request description:

  Include a test that checks whether the first argument of scantxoutset RPC call "start" is required.
  The rpc call should fail if the "start" argument is not provided.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 7e3d4f8e86

Tree-SHA512: 6a456af9f3ccd5437be2edcd61936eb9f9c21ab926a6056c2c11b6b5121d1caca4e1f2ffd09015f9414af152c635a20e1da041eefdef980afbe8a0e8ccce07bd
2023-05-04 17:19:26 +01:00
Andrew Chow
30bf70c8b6 Merge bitcoin/bitcoin#27325: test: various converttopsbt check cleanups in rpc_psbt.py
afc2dd5484 test: various `converttopsbt` check cleanups in rpc_psbt.py (Sebastian Falbesoner)

Pull request description:

  In the functional test rpc_psbt.py, some comments around the `converttopsbt` RPC checks are wrong or outdated and can be removed:

  > _Error could be either "TX decode failed" (segwit inputs causes
  > parsing to fail) or "Inputs must not have scriptSigs and
  > scriptWitnesses"_

  Decoding a valid TX with at least one input always succeeds with the [heuristic](e352f5ab6b/src/core_read.cpp (L126)), i.e. this comment is not right and we can assert for the error string "Inputs must not have scriptSigs and scriptWitnesses" on the calls below.

  > _We must set iswitness=True because the serialized transaction has
  > inputs and is therefore a witness transaction_

  This is also unneeded (and confusing, w.r.t. "is therefore a witness transaction"?), for a TX with one input there is no need to set the `iswitness` parameter. For sake of completeness, we still keep one variant where iswitness is explicitly set to true.

  Lastly, there is a superflous `converttopsbt` call on the raw tx which is the same as just [about ~10 lines above](https://github.com/bitcoin/bitcoin/blob/master/test/functional/rpc_psbt.py#L393-L397), so it can be removed.

ACKs for top commit:
  ismaelsadeeq:
    tested ACK afc2dd5484
  achow101:
    ACK afc2dd5484

Tree-SHA512: 467efefdb3f61efdb79145044b90fc8dc2f0c425f078117a99112b0074e7d4a32c34e464f665fbf8de70d06caaa5d4ad5908c1d75d2e7607eccb0837480afab3
2023-05-04 11:08:16 -04:00
Sebastian Falbesoner
2c0c6f4477 test: dedup file hashing using sha256sum_file helper
Rather than doing the open/read/hash-steps manually in the affected
functional tests, we can just use the `sha256sum_file` helper from the
utils module instead.

Note that for the tool_wallet.py test, the used hash is changed from
sha1 to sha256, but as the only purpose is to detect file content
changes, this doesn't matter. Also, the optimization using `memoryview`
is overkill here, as the opened file has only a size of 24KiB and
determining the hash doesn't take longer than a few hundred
micro-seconds on my machine.
2023-05-04 14:18:13 +02:00
Andrew Chow
aebcd18c65 Merge bitcoin/bitcoin#24957: prune, import: allow pruning to work during loadblock import
c4981e7f63 prune, import: fixes #23852 (mruddy)

Pull request description:

  Fixes #23852

  This allows pruning to work during the `-loadblock` import process.

  An example use case is where you have a clean set of block files and you want to create a pruned node from them, but you don't want to alter the input set of block files.

  #23852 noted that pruning was not working reliably during the loadblock import process. The reason why the loadblock process was not pruning regularly as it progressed is that the pruning process (`BlockManager::FindFilesToPrune`) checks the tip height of the active chainstate, and `CChainState::ActivateBestChain` was not called (which updates that tip height) in `ThreadImport` until after all the import files were processed.

  An example bash command line that makes it easy to import a bunch of block files:
  ```
  ./src/qt/bitcoin-qt -debug -logthreadnames -datadir=/tmp/btc -prune=550 -loadblock=/readonly/btc/main/blk{00000..00043}.dat
  ```

  One interesting side note is that `CChainState::ActivateBestChain` can be called while the import process is running (in the `loadblk` thread) by concurrent network message processing activity in the `msghand` thread. For example, one way to reproduce this easily is with the `getblockfrompeer` RPC (requesting a block with height greater than 100000) run from a node connected to an importing node. There are other ways too, but this is an easy way. I only mention this to explain how the `max_prune_height=225719` log message in the original issue came to occur.

ACKs for top commit:
  achow101:
    re-ACK c4981e7f63

Tree-SHA512: d287c7753952c22f598ba782914c47f45ad44ce60b0fbce9561354e701f1a2a98bafaaaa106c8428690b814e281305ca3622b177ed3cb2eb7559f07c958ab537
2023-05-03 17:49:57 -04:00
Ryan Ofsky
2cd28e9fef rpc: Add check for unintended option/parameter name clashes
Also add flag to allow RPC methods that intendionally accept options and
parameters with the same name bypass the check.

Check and flag were suggested by ajtowns
https://github.com/bitcoin/bitcoin/pull/26485#issuecomment-1507916357

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2023-05-03 12:27:51 -04:00
fanquake
1d7f1ada48 Merge bitcoin/bitcoin#27562: ci: Use arm_container.dockerfile
fa6e2bfd05 ci: Use arm_container.dockerfile (MarcoFalke)

Pull request description:

  This allows to cache the image and thus speed up the CI task

ACKs for top commit:
  fanquake:
    nice ACK fa6e2bfd05
  hebasto:
    ACK fa6e2bfd05

Tree-SHA512: 3b7f734799bd8a3ca9166e85d2cd8f68c134c32a496757eafb118250c0b158d0d76ead0c461e307d6440a622fe51bc599aa13e67766ce232b750ed0a66d4b450
2023-05-03 17:27:51 +01:00
Ryan Ofsky
95d7de0964 test: Update python tests to use named parameters instead of options objects 2023-05-03 11:27:51 -05:00
Ryan Ofsky
96233146dd RPC: Allow RPC methods accepting options to take named parameters
Co-authored-by: Andrew Chow <github@achow101.com>
2023-05-03 11:27:51 -05:00
Ryan Ofsky
702b56d2a8 RPC: Add add OBJ_NAMED_PARAMS type
OBJ_NAMED_PARAMS type works the same as OBJ type except it registers the object
keys to be accepted as top-level named-only RPC parameters. Generated
documentation also lists the object keys seperately in a new "Named arguments"
section of help text.

Named-only RPC parameters have the same semantics as python keyword-only
arguments (https://peps.python.org/pep-3102/). They are always required to be
passed by name, so they don't affect interpretation of positional arguments,
and aren't affected when positional arguments are added or removed.

The new OBJ_NAMED_PARAMS type is used in the next commit to make it easier to
pass options values to various RPC methods.

Co-authored-by: Andrew Chow <github@achow101.com>
2023-05-03 11:27:51 -05:00
Andrew Chow
0e70a1b625 Merge bitcoin/bitcoin#26066: wallet: Refactor and document CoinControl
daba95700b refactor: Make ListSelected return vector (Sebastian Falbesoner)
94776621ba wallet: Move CoinCointrol definitions to .cpp (Aurèle Oulès)
1db23da6e1 wallet: Use std::optional for GetExternalOutput and fixups (Aurèle Oulès)
becc45b589 scripted-diff: Rename setSelected->m_selected_inputs (Aurèle Oulès)

Pull request description:

  - Moves CoinControl function definitions from `coincontrol.h` to `coincontrol.cpp`
  - Adds more documentation
  - Renames class member for an improved comprehension
  - Use `std::optional` for `GetExternalOutput`

ACKs for top commit:
  achow101:
    ACK daba95700b
  Xekyo:
    ACK daba95700b

Tree-SHA512: 3bf2dc834a3246c2f53f8c55154258e605fcb169431d3f7b156931f33c7e3b1ae28e03e16b37f9140a827890eb7798be485b2c36bfc23ff29bb01763f289a07c
2023-05-03 11:17:28 -04:00
Andrew Chow
33e2b82a4f wallet, bench: Remove unused database options from WalletBenchLoading 2023-05-03 11:05:56 -04:00
Andrew Chow
80ace042d8 tests: Modify records directly in wallet ckey loading test
In the wallet ckey loading test, we modify various ckey records to test
corruption handling. As the database is now a mockable database, we can
modify the records that the database will be initialized with. This
avoids having to use the verbose database reading and writing functions.
2023-05-03 10:45:10 -04:00
Andrew Chow
b3bb17d5d0 tests: Update DuplicateMockDatabase for MockableDatabase 2023-05-03 10:45:10 -04:00
Andrew Chow
f0eecf5e40 scripted-diff: Replace CreateMockWalletDB with CreateMockableWalletDB
Since we have a mockable wallet database, we don't really need to be
using BDB or SQLite's in-memory database capabilities. It doesn't really
help us to be using those as we aren't doing anything that requires one
type of db over the other, and will just prefer SQLite if it's
available.

MockableDatabase is suitable for these uses, so use
CreateMockableWalletDatabase to use that.

-BEGIN VERIFY SCRIPT-
sed -i "s/CreateMockWalletDatabase(options)/CreateMockableWalletDatabase()/" $(git grep -l "CreateMockWalletDatabase(options)" -- ":(exclude)src/wallet/walletdb.*")
sed -i "s/CreateMockWalletDatabase/CreateMockableWalletDatabase/" $(git grep -l "CreateMockWalletDatabase" -- ":(exclude)src/wallet/walletdb.*")
-END VERIFY SCRIPT-
2023-05-03 10:45:10 -04:00
Andrew Chow
075962bc25 wallet, tests: Include wallet/test/util.h
This will be needed for the following scripted-diff to work.
2023-05-03 10:45:10 -04:00
Andrew Chow
14aa4cb1e4 wallet: Move DummyDatabase to salvage
It's only used by salvage, so make it local to that only.
2023-05-03 10:45:10 -04:00
Andrew Chow
f67a385556 wallet, tests: Replace usage of dummy db with mockable db 2023-05-03 10:45:10 -04:00
Andrew Chow
33c6245ac1 Introduce MockableDatabase for wallet unit tests
MockableDatabase is a WalletDatabase that allows us to interact with the
records to change them independently from the wallet, as well as
changing the return values from within the tests. This will give us
greater flexibility in testing the wallet.
2023-05-03 10:45:10 -04:00
MarcoFalke
fa6e2bfd05 ci: Use arm_container.dockerfile 2023-05-03 15:31:40 +02:00
glozow
8f5da89625 Merge bitcoin/bitcoin#27559: doc: clarify processing of mempool-msgs when NODE_BLOOM
4581a682d2 clarify processing of mempool-msgs when NODE_BLOOM (0xb10c)

Pull request description:

  Under which circumstances we process received 'mempool' P2P messages caused confusion in #27426. Rather than bike-shedding the formulation of the IF-statement, this adds a comment clarifying when we process the message. Also, correcting the `m_send_mempool` description.

ACKs for top commit:
  dergoegge:
    ACK 4581a682d2
  willcl-ark:
    ACK 4581a682d2
  glozow:
    ACK 4581a682d2

Tree-SHA512: 51ec673c3446b67c26f6c715430d0708b998b256260f5f5d0c034f271be8447d0bb8540dfd3879aa51904512fb26c9411766786c86287acff62d037a1df88855
2023-05-03 07:56:43 -04:00
fanquake
49d543dcaf Merge bitcoin/bitcoin#26953: contrib: add ELF OS ABI check to symbol-check.py
65ba8a79a2 contrib: add ELF ABI check to symbol-check.py (fanquake)

Pull request description:

  Check that the operating system ABI version embedded into the release binaries, is the version we expect it to be.

ACKs for top commit:
  laanwj:
    Code review ACK 65ba8a79a2
  TheCharlatan:
    ACK 65ba8a79a2

Tree-SHA512: 798d7c3b05183becf113a2ea13d889e18f1cec01d3cc279e64dbddede4d57f87444978f3f52c44bc5fdf0ba93d77c7c0be37aa815f93f348c35da45dc3d30ac2
2023-05-03 10:03:34 +01:00
fanquake
067a835adb Merge bitcoin/bitcoin#27553: test: Simplify feature_fastprune.py
fa17767154 test: Simplify feature_fastprune.py (MarcoFalke)

Pull request description:

  The goal of the test is a single regression check to see if a RPC times out. It shouldn't do more than calling the RPC (and the minimum work needed to get there).

  Fix that by removing all blocktools imports and a `for` loop.

ACKs for top commit:
  pinheadmz:
    ACK fa17767154
  theStack:
    ACK fa17767154

Tree-SHA512: c9c0154102199b250015ece53005a14d52d857dfa986f3b02a2cb899f16ac8e040d24eb826f35ba15e5ee22ee6a59bf8f74bb8d576b9a12ac6e888beeaaf81cc
2023-05-03 09:47:19 +01:00
0xb10c
4581a682d2 clarify processing of mempool-msgs when NODE_BLOOM
Under which circumstances we process received 'mempool' P2P messages
caused confusion in #27426. Rather than bikeshedding the formulation
of the IF-statement, this adds a comment clarifing when we process
the message. Also, correcting the comment of `m_send_mempool`.

Co-authored-by: willcl-ark <will8clark@gmail.com>
2023-05-03 00:24:22 +02:00
MarcoFalke
fa26e3462a Avoid dereferencing interruption_point if it is nullptr 2023-05-02 20:19:28 +02:00
fanquake
65ba8a79a2 contrib: add ELF ABI check to symbol-check.py 2023-05-02 16:54:36 +01:00
Andrew Chow
da9f62f912 Merge bitcoin/bitcoin#26094: rpc: Return block hash & height in getbalances, gettransaction and getwalletinfo
710b83938a rpc: return block hash & height in getbalances, gettransaction & getwalletinfo JSONs (Harris)

Pull request description:

  Reopens #18570 and closes #18567.
  I have rebased the original PR.
  Not sure why the original got closed as it was about to get merged.

ACKs for top commit:
  achow101:
    ACK 710b83938a

Tree-SHA512: d4478d990be98b1642e9ffb2930987f4a224e8bd64e2e35a5dda927a54c509ec9d712cd0eac35dc2bb89f00a1678e530ce14d7445f1dd93aa3a4cce2bc9b130d
2023-05-02 11:50:45 -04:00
MarcoFalke
fa846ee074 test: Add util to mine invalid blocks
With the current utils it is only possible to mine valid blocks. This
commit adds new util methods to mine invalid blocks.
2023-05-02 17:17:06 +02:00
brunoerg
35a2175ad8 fuzz: addrman, add coverage for network field in Select(), Size() and GetAddr()
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-05-02 11:05:36 -03:00
fanquake
7b45d171f5 Merge bitcoin/bitcoin#27360: ci: use LLVM/clang-16 in native_asan job
f952e679cd ci: remove usage of untrusted bpfcc-tools (fanquake)
1232c2f6b9 ci: use LLVM/clang-16 in native_asan job (fanquake)

Pull request description:

  Similar to #27298. Working for me on `x86_64` and solves the issue I currently see with TSAN on `aarch64` with master (68828288e5):
  ```bash
  crc32c/src/crc32c_arm64.cc:101:26: runtime error: load of misaligned address 0xffff84400406 for type 'uint64_t' (aka 'unsigned long'), which requires 8 byte alignment
  0xffff84400406: note: pointer points here
   b9 c5 22 00 01 01  1a 6c 65 76 65 6c 64 62  2e 42 79 74 65 77 69 73  65 43 6f 6d 70 61 72 61  74 6f
               ^
      #0 0xaaaaaddaf0b4 in crc32c::ExtendArm64(unsigned int, unsigned char const*, unsigned long) src/./src/crc32c/src/crc32c_arm64.cc:101:26
      #1 0xaaaaadd2c838 in leveldb::crc32c::Value(char const*, unsigned long) src/./leveldb/util/crc32c.h:20:60
      #2 0xaaaaadd2c838 in leveldb::log::Reader::ReadPhysicalRecord(leveldb::Slice*) src/./src/leveldb/db/log_reader.cc:246:29
      #3 0xaaaaadd2ba9c in leveldb::log::Reader::ReadRecord(leveldb::Slice*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) src/./src/leveldb/db/log_reader.cc:72:38
      #4 0xaaaaadd41710 in leveldb::VersionSet::Recover(bool*) src/./src/leveldb/db/version_set.cc:910:19
      #5 0xaaaaadcf9fec in leveldb::DBImpl::Recover(leveldb::VersionEdit*, bool*) src/./src/leveldb/db/db_impl.cc:320:18
      #6 0xaaaaadd12068 in leveldb::DB::Open(leveldb::Options const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, leveldb::DB**) src/./src/leveldb/db/db_impl.cc:1487:20
      #7 0xaaaaad314e80 in CDBWrapper::CDBWrapper(DBParams const&) src/./src/dbwrapper.cpp:156:30
      #8 0xaaaaace94880 in CBlockTreeDB::CBlockTreeDB(DBParams const&) src/./txdb.h:89:23
      #9 0xaaaaace94880 in std::_MakeUniq<CBlockTreeDB>::__single_object std::make_unique<CBlockTreeDB, DBParams>(DBParams&&) /usr/bin/../lib/gcc/aarch64-linux-gnu/11/../../../../include/c++/11/bits/unique_ptr.h:962:34
      #10 0xaaaaace94880 in ChainTestingSetup::ChainTestingSetup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&) src/./src/test/util/setup_common.cpp:188:51
      #11 0xaaaaace95da0 in TestingSetup::TestingSetup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&, bool, bool) src/./src/test/util/setup_common.cpp:243:7
      #12 0xaaaaace96730 in TestChain100Setup::TestChain100Setup(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<char const*, std::allocator<char const*> > const&, bool, bool) src/./src/test/util/setup_common.cpp:274:7
      #13 0xaaaaac1ddbc8 in blockfilter_index_tests::BuildChainTestingSetup::BuildChainTestingSetup() src/./src/test/blockfilter_index_tests.cpp:26:8
      #14 0xaaaaac1ddbc8 in blockfilter_index_tests::blockfilter_index_initial_sync::blockfilter_index_initial_sync() src/./src/test/blockfilter_index_tests.cpp:112:1
      #15 0xaaaaac1ddbc8 in blockfilter_index_tests::blockfilter_index_initial_sync_invoker() src/./src/test/blockfilter_index_tests.cpp:112:1
      #16 0xaaaaabf08f7c in boost::function0<void>::operator()() const /usr/include/boost/function/function_template.hpp:763:14
      #17 0xaaaaabf95468 in boost::detail::forward::operator()() /usr/include/boost/test/impl/execution_monitor.ipp:1388:32
      #18 0xaaaaabf95468 in boost::detail::function::function_obj_invoker0<boost::detail::forward, int>::invoke(boost::detail::function::function_buffer&) /usr/include/boost/function/function_template.hpp:137:18
      #19 0xaaaaabf8e12c in boost::function0<int>::operator()() const /usr/include/boost/function/function_template.hpp:763:14
      #20 0xaaaaabe7be14 in boost::execution_monitor::catch_signals(boost::function<int ()> const&) /usr/include/boost/test/impl/execution_monitor.ipp:903:16
      #21 0xaaaaabe7c1c0 in boost::execution_monitor::execute(boost::function<int ()> const&) /usr/include/boost/test/impl/execution_monitor.ipp:1301:16
      #22 0xaaaaabe6f47c in boost::execution_monitor::vexecute(boost::function<void ()> const&) /usr/include/boost/test/impl/execution_monitor.ipp:1397:5
      #23 0xaaaaabe75124 in boost::unit_test::unit_test_monitor_t::execute_and_translate(boost::function<void ()> const&, unsigned long) /usr/include/boost/test/impl/unit_test_monitor.ipp:49:9
      #24 0xaaaaabed19fc in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /usr/include/boost/test/impl/framework.ipp:815:44
      #25 0xaaaaabed0f6c in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /usr/include/boost/test/impl/framework.ipp:784:58
      #26 0xaaaaabed0f6c in boost::unit_test::framework::state::execute_test_tree(unsigned long, unsigned long, boost::unit_test::framework::state::random_generator_helper const*) /usr/include/boost/test/impl/framework.ipp:784:58
      #27 0xaaaaabe73878 in boost::unit_test::framework::run(unsigned long, bool) /usr/include/boost/test/impl/framework.ipp:1721:29
      #28 0xaaaaabe9d244 in boost::unit_test::unit_test_main(boost::unit_test::test_suite* (*)(int, char**), int, char**) /usr/include/boost/test/impl/unit_test_main.ipp:250:9
      #29 0xffff8f0773f8  (/lib/aarch64-linux-gnu/libc.so.6+0x273f8) (BuildId: f37f3aa07c797e333fd106472898d361f71798f5)
      #30 0xffff8f0774c8 in __libc_start_main (/lib/aarch64-linux-gnu/libc.so.6+0x274c8) (BuildId: f37f3aa07c797e333fd106472898d361f71798f5)
      #31 0xaaaaabda55ac in _start (/home/fedora/ci_scratch/ci/scratch/build/bitcoin-aarch64-unknown-linux-gnu/src/test/test_bitcoin+0x10e55ac) (BuildId: b7909adaefd9db6cd6a7c4d3d40207cf6bdaf4b3)

  SUMMARY: UndefinedBehaviorSanitizer: misaligned-pointer-use crc32c/src/crc32c_arm64.cc:101:26 in
  ```

ACKs for top commit:
  dergoegge:
    utACK f952e679cd
  MarcoFalke:
    lgtm ACK f952e679cd

Tree-SHA512: 9dee2abf73d3f23bb9979bfb453b48e39f0b7a5f58d43824ecf053a53e9800ed413b915382b274d1a84baf2999683e3b485463e377e0455b3f0ead65ed1d1916
2023-05-02 14:03:59 +01:00
fanquake
d654c762c8 Merge bitcoin/bitcoin#27453: test: added coverage to rpc_scantxoutset.py
24d55fb9cf test: added coverage to rpc_scantxoutset.py (kevkevin)

Pull request description:

  Included a test that checks if an invalid first argument is entered we receive a rpc error. The rpc should fail if "start", "status" or "abort" is not the first command.

  Relavant: mentioned in https://github.com/bitcoin/bitcoin/pull/27422

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 24d55fb9cf
  theStack:
    ACK 24d55fb9cf

Tree-SHA512: 4b804235d3fa17c7bf492068ab47c1f87cb6cfc1a428c51e273ec059d3c41f581bcc467bb5d6d8bbf2fab14c60cd1c52a30c50009efe1c9b5adee70c88897ad9
2023-05-02 13:45:09 +01:00
fanquake
cfe5da4c90 Merge bitcoin/bitcoin#27542: test: add ripemd160 to test framework modules list
82e6e3cae5 test: add ripemd160 to test framework modules list (Sebastian Falbesoner)

Pull request description:

  Currently test runner doesn't execute the unit tests of the ripemd160 module, so add it to the list. All other framework modules that contain unit tests are included, as can be easily checked via
  `$ git grep unittest.TestCase ./test/functional/test_framework/`

  This is a late follow-up to PR #23716 (commit ad3e9e1f21).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 82e6e3cae5

Tree-SHA512: 10940e215f728291c7149931a356bfc42795c098bda76d760dfa68f86443a3755e1cd35cb9a8a7b2f48880beb53f3bee3842de2d74bcadd45c7b05c13ff04203
2023-05-02 13:30:18 +01:00
MarcoFalke
fa17767154 test: Simplify feature_fastprune.py 2023-05-02 14:14:00 +02:00
Sebastian Falbesoner
82e6e3cae5 test: add ripemd160 to test framework modules list
Currently test runner doesn't execute the unit tests of the ripemd160
module, so add it to the list. All other framework modules that contain
unit tests are included, as can be easily checked via
`git grep unittest.TestCase ./test/functional/test_framework/`

This is a late follow-up to PR #23716 (commit
ad3e9e1f21).
2023-05-02 14:12:04 +02:00
kevkevin
24d55fb9cf test: added coverage to rpc_scantxoutset.py
Included a test that checks if an invalid first argument is entered we
receive a rpc error. The rpc should fail if "start", "status" or "abort"
is not the first command.
2023-05-02 06:46:45 -05:00
fanquake
f952e679cd ci: remove usage of untrusted bpfcc-tools
We've migrated this job to Ubuntu 23.04, which
ships with newer versions of the tools:
https://packages.ubuntu.com/lunar/bpfcc-tools.
2023-05-02 12:02:45 +01:00
fanquake
1232c2f6b9 ci: use LLVM/clang-16 in native_asan job
Similar to #27298.
2023-05-02 12:02:23 +01:00
fanquake
8a373a5c7f Merge bitcoin/bitcoin#27191: blockstorage: Adjust fastprune limit if block exceeds blockfile size
8f14fc8622 test: cover fastprune with excessive block size (Matthew Zipkin)
271c23e87f blockstorage: Adjust fastprune limit if block exceeds blockfile size (Martin Zumsande)

Pull request description:

  The debug-only `-fastprune` option used in several tests is not always safe to use:
  If a `-fastprune` node receives a block larger than the maximum blockfile size of `64kb` bad things happen: The while loop in `BlockManager::FindBlockPos` never terminates, and the node runs oom because memory for `m_blockfile_info` is allocated in each iteration of the loop.
  The same would happen if a naive user used `-fastprune` on anything other than regtest (so this can be tested by syncing on signet for example, the first block that crashes the node is at height 2232).

  Change the approach by raising the blockfile size to the size of the block, if that block otherwise wouldn't fit (idea by TheCharlatan).

ACKs for top commit:
  ryanofsky:
    Code review ACK 8f14fc8622. Added new assert, test, and comment since last review
  TheCharlatan:
    ACK 8f14fc8622
  pinheadmz:
    ACK 8f14fc8622

Tree-SHA512: df2fea30613ef9d40ebbc2416eacb574f6d7d96847db5c33dda22a29a2c61a8db831aa9552734ea4477e097f253dbcb6dcb1395d43d2a090cc0588c9ce66eac3
2023-05-02 10:04:34 +01:00
fanquake
be0325c6a6 Merge bitcoin/bitcoin#27538: test: Remove modinv python util helper function
dc14ba08e6 test: remove modinv python util helper function (Fabian Jahr)

Pull request description:

  Since #27483 was merged the `modinv()` body is just one line calling pythons own implementation of `pow()`. We can just remove the function as it doesn't seem to add any value. Additionally the comment in the function is now outdated and the test is only testing two ways of doing modular inverse but both using python's `pow()` function.

ACKs for top commit:
  theStack:
    ACK dc14ba08e6

Tree-SHA512: e8b470c72dc3f9fd53699d0684650517b1ea35ad1d4c01cf9472c80d3e4474c0c72e429c0bd201eb99d204c87eee0d68285e6a388e4c506f30e14b2bff9c1c32
2023-05-01 14:31:01 +01:00
Andrew Chow
539452242e Merge bitcoin/bitcoin#26733: test: Add test for sendmany rpc that uses subtractfeefrom parameter
057057a2d7 Add test for `sendmany` rpc that uses `subtractfeefrom` parameter (Yusuf Sahin HAMZA)

Pull request description:

  This PR adds test that uses `sendmany` rpc to send **BTC** to multiple addresses using `subtractfeefrom` parameter, then checks receiver addresses balances to make sure fees are subtracted correctly.

ACKs for top commit:
  achow101:
    ACK 057057a2d7

Tree-SHA512: 51167120d489f0ff7b8b9855424d07cb55a8965984f904643cddf45e7a08c350eaded498c350ec9c660edf72c2f128ec142347c9c79d5043d9f6cd481b15cd7e
2023-05-01 09:28:06 -04:00
fanquake
ab99b95b00 Merge bitcoin/bitcoin#26604: test: add coverage for -bantime
9c18992bba test: add coverage for `-bantime` (brunoerg)

Pull request description:

  This PR adds test coverage for `-bantime`. This flag sets the time in seconds how long the IP is banned (in the case you don't explicitly set `bantime` when using `setban`).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 9c18992bba

Tree-SHA512: e95f8608aa5df9b09cc5577daae662ed79ef5d5c69ee5e704d7c69520b9b51cc142e9e6be69d80356eda25a5215c4770b1a208638560c48cd3bc8f6d195a371f
2023-05-01 14:21:06 +01:00
Andrew Chow
0eae93e65f Merge bitcoin/bitcoin#26780: rpc: simplify scan blocks
b922f6b526 rpc: scanblocks, add "completed" flag to the result obj (furszy)
ce50acc54f rpc: scanblocks, do not traverse the whole chain block by block (furszy)

Pull request description:

  Coming from https://github.com/bitcoin/bitcoin/pull/23549#pullrequestreview-1105712566

  The current `scanblocks` flow walks-through every block in the active chain
  until hits the chain tip or processes 10k blocks, then calls `lookupFilterRange`
  function to obtain all filters from that particular range.

  This is only done to obtain the heights range to look up the block
  filters. Which is unneeded.

  As `scanblocks` only lookup block filters in the active chain, we can
  directly calculate the lookup range heights, by using the chain tip,
  without requiring to traverse the chain block by block.

ACKs for top commit:
  achow101:
    ACK b922f6b526
  TheCharlatan:
    ACK b922f6b526

Tree-SHA512: 0587e6d9cf87a59184adb2dbc26a4e2bce3a16233594c6c330f69feb49bf7dc63fdacf44fc20308e93441159ebc604c63eb7de19204d3e745a2ff16004892b45
2023-05-01 09:10:11 -04:00
Andrew Chow
3497df4c75 Merge bitcoin/bitcoin#27195: bumpfee: allow send coins back to yourself
be72663a15 test: bumpfee, add coverage for "send coins back to yourself" (furszy)
7bffec6715 bumpfee: enable send coins back to yourself (furszy)

Pull request description:

  Simple example:

  1) User_1 sends 0.1 btc to user_2 on a low fee transaction.
  2) After few hours, the tx is still in the mempool, user_2
     is not interested anymore, so user_1 decides to cancel
     it by sending coins back to himself.
  3) User_1 has the bright idea of opening the explorer and
     copy the change output address of the transaction. Then
     call bumpfee providing such output (in the "outputs" arg).

  Currently, this is not possible. The wallet fails with
  "Unable to create transaction. Transaction must have at least
  one recipient" error.
  The error reason is because we discard the provided output
  from the recipients list and set it inside the coin control
  so the process adds it later (when the change is calculated).
  But.. there is no later if the tx has no outputs.

ACKs for top commit:
  ishaanam:
    reACK be72663a15
  achow101:
    ACK be72663a15

Tree-SHA512: c2c38290a998f9b426a830d9624c7feb730158980ac186f8fb0138d5e200935d6538307bc60a2c3d0b7b6ee2b4ffb77a1e98baf8feb1d20a7d825f6055ac377f
2023-05-01 08:38:50 -04:00
Andrew Chow
071308860a Merge bitcoin/bitcoin#25680: rpc, docs: Add note for commands that supports only legacy wallets
9141e4395a rpc, docs: Add note for commands that supports only legacy wallets (Yusuf Sahin HAMZA)

Pull request description:

  Refs #25363, apparently issue is not updated since over a month, so i decided to put the same `importaddress` note in #25368 to other rpc commands that needs this note.

  Note is added for following commands:

  - `importprivkey`
  - `importpubkey`
  - `importwallet`
  - `dumpprivkey`
  - `dumpwallet`
  - `importmulti`
  - `addmultisigaddress`
  - `sethdseed`

ACKs for top commit:
  achow101:
    ACK 9141e4395a

Tree-SHA512: f3dc05d26577fd8dbe2bd69cb5c14ffccebacd6010402af44427b3d01be8484895dfcf33d55dfa766eadb7f9f3bae5cc4c2add3ac816a2ac60e8beb5a97527f3
2023-05-01 08:24:42 -04:00
Andrew Chow
5325a61167 Merge bitcoin/bitcoin#27224: refactor: Remove CAddressBookData::destdata
a5986e82dd refactor: Remove CAddressBookData::destdata (Ryan Ofsky)
5938ad0bdb wallet: Add DatabaseBatch::ErasePrefix method (Ryan Ofsky)

Pull request description:

  This is cleanup that doesn't change external behavior. Benefits of the cleanup are:

  - Removes awkward `StringMap` intermediate representation for wallet address metadata.
  - Simplifies `CWallet`, deals with used address and received request serialization in `walletdb.cpp` instead of higher level wallet code
  - Adds test coverage and documentation

  This PR doesn't change externally observable behavior. Internally, the only change in behavior is that `EraseDestData` deletes rows directly from the database because they are no longer stored in memory. This is more direct and efficient because it uses a single lookup and scan instead of multiple lookups.

  Motivation for this cleanup is making changes like #18550, #18192, #13756 easier to reason about and less likely to result in unintended behavior and bugs

  ---

  This PR is a rebased copy of #18608. For some reason that PR is locked and couldn't be reopened or commented on.

  This PR is an alternative to #27215 with differences described in https://github.com/bitcoin/bitcoin/pull/27215#pullrequestreview-1329028143

ACKs for top commit:
  achow101:
    ACK a5986e82dd
  furszy:
    Code ACK a5986e82

Tree-SHA512: 6bd3e402f1f60263fbd433760bdc29d04175ddaf8307207c4a67d59f6cffa732e176ba57886e02926f7a1615dce0ed9cda36c8cbc6430aa8e5b56934c23f3fe7
2023-05-01 08:16:54 -04:00
furszy
b922f6b526 rpc: scanblocks, add "completed" flag to the result obj
To tell the user whether the process was aborted or not.

Plus, as the process can be aborted prior to the end range,
have also changed the "to_height" result value to return the
last scanned block instead of the end range block.
2023-04-30 19:26:11 +01:00
furszy
ce50acc54f rpc: scanblocks, do not traverse the whole chain block by block
The current flow walks-through every block in the active chain until
hits the chain tip or processes 10k blocks, then calls
`lookupFilterRange()` to obtain all the filters from that
particular range.

This is only done to obtain the heights range to look up the block
filters. Which is unneeded.

As `scanblocks` only lookup block filters in the active chain, we can
directly calculate the lookup range heights, by using the chain tip,
without requiring to traverse the chain block by block.
2023-04-30 19:14:20 +01:00
Fabian Jahr
dc14ba08e6 test: remove modinv python util helper function 2023-04-28 14:19:18 +02:00
fanquake
d89aca1bdb Merge bitcoin/bitcoin#27483: Bump python minimum version to 3.8
fac395e5eb ci: Bump ci/lint/Dockerfile (MarcoFalke)
fa6eb65167 test: Use python3.8 pow() (MarcoFalke)
88881cf7ac Bump python minimum version to 3.8 (MarcoFalke)

Pull request description:

  There is no pressing reason to drop support for 3.7, however there are several maintenance issues:

  * There is no supported operating system that ships 3.7 by default. (debian:buster is EOL and unmaintained to the extent that it doesn't run in the CI environment. See https://github.com/bitcoin/bitcoin/pull/27340#issuecomment-1484988445)
  * Compiling python 3.7 from source is also unsupported on at least macos, according to https://github.com/bitcoin/bitcoin/pull/24017#issuecomment-1107820790
  * Recent versions of lief require 3.8, see https://github.com/bitcoin/bitcoin/pull/27507#issuecomment-1517561645

  Fix all maintenance issues by bumping the minimum.

ACKs for top commit:
  RandyMcMillan:
    ACK fac395e
  fjahr:
    ACK fac395e5eb
  fanquake:
    ACK fac395e5eb

Tree-SHA512: c198decdbbe29d186d73ea3f6549d8a38479383495d14a965a2f9211ce39637b43f13a4c2a5d3bf56e2d468be4bbe49b4ee8e8e19ec69936ff43ddf2b714c712
2023-04-28 10:22:20 +01:00
fanquake
904a98702e Merge bitcoin/bitcoin#26314: test: perturb anchors.dat to test error during initialization
33fdfc7986 test: perturb anchors.dat to test it doesn't throw an error during initialization (brunoerg)

Pull request description:

  Got some inspiration from `feature_init`. This PR tests whether perturbing `anchors.dat` doesn't throw any error during initialization.

  3f1f5f6f1e/src/addrdb.cpp (L223-L235)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 33fdfc7986

Tree-SHA512: e6584debb37647677581fda08366b45b42803022cc4c4f1d5a7bd5e9e04d64da77656dad2b804855337487bdcfc891f300a2e03668d6122de769dd14f39af9ed
2023-04-27 10:33:35 +01:00
fanquake
03cb2fce4a Merge bitcoin/bitcoin#26794: test: test banlist database recreation
4bdcf57158 test: test banlist database recreation (brunoerg)

Pull request description:

  This PR adds test coverage for 'banlist database recreation'. If it wasn't able to read ban db (in `LoadBanlist`), so it should create a new (an empty, ofc) one.
  d8bdee0fc8/src/banman.cpp (L28-L45)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4bdcf57158

Tree-SHA512: d9d0cd0c4b3797189dff00d3a634878188e7cf51e78346601fc97e2bf78c495561705214062bb42ab8e491e0d111f8bfcf74dbc801768bc02cf2b45f162aad85
2023-04-27 10:18:57 +01:00
fanquake
ba4076d26f Merge bitcoin/bitcoin#25937: test: add coverage for rpc error when trying to rescan beyond pruned data
cca4f82b82 test: add coverage for rpc error when trying to rescan beyond pruned data (brunoerg)

Pull request description:

  This PR adds test coverage for the following rpc error:
  15692e2641/src/wallet/rpc/transactions.cpp (L896-L899)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK cca4f82b82
  aureleoules:
    ACK cca4f82b82

Tree-SHA512: 724a055e9f6cddf1935699e8769015115f24f6485a0bd87e8660072ee44a15c1bddfdda848acc101ea7184b7e65a33b5b0d80b563d2ba3ecdab7a631378d6476
2023-04-27 06:20:20 +01:00
Harris
710b83938a rpc: return block hash & height in getbalances, gettransaction & getwalletinfo JSONs
Co-authored-by: Aurèle Oulès <aurele@oules.com>
2023-04-26 16:07:47 +02:00
Andrew Chow
91ccb62faa Merge bitcoin/bitcoin#25158: rpc, wallet: add abandoned field for all categories of transaction in ListTransaction
0c520679ab doc: add release notes for `abandoned` field in `gettransaction` and `listtransactions` (brunoerg)
a1aaa7f51f rpc, wallet: add `abandoned` field for all categories of transactions in ListTransactions (brunoerg)

Pull request description:

  Fixes #25130

ACKs for top commit:
  achow101:
    re-ACK 0c520679ab

Tree-SHA512: 1864460d76decab7898737c96517d722055eb8f81ca52248fe1035723258c6cd4a93251e06a86ecbbb0b0a80af1466b2c86fb142ace4ccb74cc40d5dc3967d7f
2023-04-26 08:50:56 -04:00
glozow
bdfe27c9d2 Merge bitcoin/bitcoin#26933: mempool: disallow txns under min relay fee, even in packages
bf77fc9cb4 [test] mempool full in package accept (glozow)
b51ebccc28 [validation] set PackageValidationState when mempool full (glozow)
563a2ee4f5 [policy] disallow transactions under min relay fee, even in packages (glozow)
c4554fe894 [test] package cpfp bumps parents <mempoolminfee but >=minrelaytxfee (glozow)
ac463e87df [test util] mock mempool minimum feerate (glozow)

Pull request description:

  Part of package relay, see #27463.

  Note that this still allows packages to bump transactions that are below the dynamic mempool minimum feerate, which means this still solves the "mempool is congested and my presigned 1sat/vB tx is screwed" problem for all transactions.

  On master, the package policy (only accessible through regtest-only RPC submitpackage) allows 0-fee (or otherwise below min relay feerate) transactions if they are bumped by a child. However, with default package limits, we don't yet have a DoS-resistant way of ensuring these transactions remain bumped throughout their time in the mempool. Primarily, the fee-bumping child may later be replaced by another transaction that doesn't bump the parent(s). The parent(s) could potentially stay bumped by other transactions, but not enough to ever be selected by the `BlockAssembler` (due to `blockmintxfee`).

  For example, (tested [here](https://github.com/glozow/bitcoin/commits/26933-motivation)):
  - The mempool accepts 24 below-minrelayfeerate transactions ("0-fee parents"), all bumped by a single high-fee transaction ("the fee-bumping child"). The fee-bumping child also spends a confirmed UTXO.
  - Two additional children are added to each 0-fee parent. These children each pay a feerate slightly above the minimum relay feerate (e.g. 1.9sat/vB) such that, for each 0-fee parent, the total fees of its two children divided by the total size of the children and parent is above the minimum relay feerate.
  - If a block template is built now, all transactions would be selected.
  - A transaction replaces the the fee-bumping child, spending only the confirmed UTXO and not any of the outputs from the 0-fee parents.
   - The 0-fee parents now each have 2 children. Their descendant feerates are above minrelayfeerate, which means that they remain in the mempool, even if the mempool evicts all below-minrelayfeerate packages.
   - If a block template is built now, none of the 0-fee parents or their children would be selected.
   - Even more low-feerate descendants can be added to these below-minrelayfeerate packages and they will not be evicted until they expire or the mempool reaches capacity.

  Unless we have a DoS-resistant way of ensuring package CPFP-bumped transactions are always bumped, allowing package CPFP to bump below-minrelayfeerate transactions can result in these problematic situations. See #27018 which proposes a partial solution with some limitations, and contains discussion about potential improvements to eviction strategy. While no adequate solution exists, for now, avoid these situations by requiring all transactions to meet min relay feerate.

ACKs for top commit:
  ajtowns:
    reACK bf77fc9cb4
  instagibbs:
    re-ACK bf77fc9cb4

Tree-SHA512: 28940f41493a9e280b010284316fb8caf1ed7b2090ba9a4ef8a3b2eafc5933601074b142f4f7d4e3c6c4cce99d3146f5c8e1393d9406c6f2070dd41c817985c9
2023-04-26 11:18:09 +01:00
Sebastian Falbesoner
daba95700b refactor: Make ListSelected return vector 2023-04-26 10:41:10 +02:00
Aurèle Oulès
94776621ba wallet: Move CoinCointrol definitions to .cpp
Move definitions to coincontrol.cpp and add documentation.
2023-04-26 10:16:16 +02:00
Aurèle Oulès
1db23da6e1 wallet: Use std::optional for GetExternalOutput and fixups 2023-04-26 10:16:16 +02:00
Aurèle Oulès
becc45b589 scripted-diff: Rename setSelected->m_selected_inputs
-BEGIN VERIFY SCRIPT-
sed -i 's/setSelected/m_selected_inputs/g' src/wallet/coincontrol.h src/wallet/coincontrol.cpp
-END VERIFY SCRIPT-
2023-04-26 10:16:16 +02:00
fanquake
2cc43de69b Merge bitcoin/bitcoin#27516: test: simplify uint256 (de)serialization routines
96bf0bca4a test: simplify uint256 (de)serialization routines (Sebastian Falbesoner)

Pull request description:

  These routines look fancy, but do nothing more than converting between byte objects of length 32 to/from integers in little endian byte order and can be replaced by simple one-liners, using the `int.{from,to}_bytes` methods (available since Python 3.2).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 96bf0bca4a
  brunoerg:
    crACK 96bf0bca4a

Tree-SHA512: f3031502d61a936147867ad8a0efa841a9bbdd2acf8781653036889a38524f4f1a5c86b1e07157bf2d9663097e7b84be6846678d0883d2a334beafd87e9510f0
2023-04-25 11:34:33 +01:00
Sebastian Falbesoner
53c990ad34 test: fix feature_addrman.py on big-endian systems
The test `feature_addrman.py` currently serializes the addrdb without
specifying endianness for `int`s, so the machine's native byte order is used (see
https://docs.python.org/3/library/struct.html#byte-order-size-and-alignment)
and the generated `peers.dat` would be invalid on big-endian systems.
Fix this by explicitly specifying little-endian serialization via the
`<` character in `struct.pack(...)`.
2023-04-25 08:39:13 +02:00
Hennadii Stepanov
ea7b852849 build: Use newest config.{guess,sub} available 2023-04-23 11:26:11 +01:00
fanquake
397ed22162 Merge bitcoin/bitcoin#27508: build: use latest config.{guess,sub} in depends
4a3f1db4ea depends: latest config.sub (fanquake)
ac462c58f9 depends: latest config.guess (fanquake)

Pull request description:

  Been a few years since we last updated these.
  Also related to https://github.com/bitcoin/bitcoin/pull/26422#issuecomment-1421178967.

ACKs for top commit:
  TheCharlatan:
    ACK 4a3f1db4ea
  hebasto:
    ACK 4a3f1db4ea, I've got zero diff with files from the [upstream](https://git.savannah.gnu.org/gitweb/?p=config.git;a=tree).

Tree-SHA512: 8f1af0813c56289c796a6e74965632dd6fa6dd135409250b2d5ebf7c1c2bfb4001195d35e5d7ecc0cad2a049468193b9fefc2b26beb7669afe6bba4d9c3ffa33
2023-04-23 11:10:43 +01:00
Sebastian Falbesoner
96bf0bca4a test: simplify uint256 (de)serialization routines
These routines look fancy, but do nothing more than converting between
byte objects of length 32 to/from integers in little endian byte order
and can be replaced by simple one-liners, using the int.{from,to}_bytes
methods (available since Python 3.2).
2023-04-23 02:47:20 +02:00
ishaanam
096487c4dc wallet: introduce generic recursive tx state updating function
This commit also changed `MarkConflicted` and `AbandonTransaction`
to use this new function

Co-authored-by: ariard <antoine.riard@gmail.com>
2023-04-22 09:19:24 -04:00
fanquake
49d07ea9a1 Merge bitcoin/bitcoin#27506: test: prevent intermittent failures
10a354f174 test: prevent intermittent failures (Amiti Uttarwar)

Pull request description:

  Follow up to #27214 - add an address to the tried table before the new table to make sure a new table collision is not possible.

ACKs for top commit:
  mzumsande:
    Code review ACK 10a354f174 - the fix is what I suggested [here](https://github.com/bitcoin/bitcoin/pull/27214#discussion_r1169169601) and should make these intermittent failures impossible.

Tree-SHA512: 24099f02e1915395130065af0ef6a2a1893955d222517d156d928765541d9c427da00172a9b5a540163f4d6aae93ca3882e8267eeb35ecc595d42178abc6191c
2023-04-21 19:31:01 +01:00
Andrew Chow
2755aa5121 Merge bitcoin/bitcoin#25939: rpc: In utxoupdatepsbt also look for the tx in the txindex
6e9f8bb050 rpc, tests: in `utxoupdatepsbt` also look for the transaction in the txindex (ishaanam)
a5b4883fb4 rpc: extract psbt updating logic into ProcessPSBT (ishaanam)

Pull request description:

  Previously the `utxoupdatepsbt` RPC, added in #13932, only updated the inputs spending segwit outputs with the `witness_utxo`, because the full transaction is needed for legacy inputs. Before this RPC looked for just the utxos in the utxo set and the mempool. This PR makes it so that if the user has txindex enabled, then the full transaction is looked for there for all inputs. If it is not found in the txindex or  txindex isn't enabled, then the mempool is checked for the full transaction. If the transaction an input is spending from is still not found at that point, then the utxo set is searched and the inputs spending segwit outputs are updated with just the utxo.

ACKs for top commit:
  achow101:
    ACK 6e9f8bb050
  Xekyo:
    ACK 6e9f8bb050

Tree-SHA512: 078db3c37a1ecd5816d80a42e8bd1341e416d661f508fa5fce0f4e1249fefd7b92a0d45f44957781cb69d0953145ef096ecdd4545ada39062be27742402dac6f
2023-04-21 14:06:12 -04:00
fanquake
f3f5c97126 Merge bitcoin/bitcoin#27496: depends: reuse _config_opts for CMake options
63c0c4ff10 depends: reuse _config_opts for CMake options (Cory Fields)

Pull request description:

  Context: I'm [currently experimenting with building our depends with CMake when possible](https://github.com/theuni/bitcoin/commits/depends-cmake) as part of our future transition to CMake. Specifically zmq, libevent, libnatpmp, and miniupnpc all have existing CMake buildsystems. Building them with CMake will allow us to drop several deps that we currently have (autoconf, automake, pkg-config, etc) which would be unfortunate to carry over after the switch-over.

  But that's not relevant for this PR. This is just a very simple change that makes the above work easier to experiment with as it [adds a needed feature for CMake packages](5733dc2000 (diff-e6ed342a25092e0a6d0308e0bfd826044578847132cc6726ac4afa2ca767b61aR20)). It's a no-op for the current builds.

  ---

  From commit description:

  This will allow us to use the existing machinery for filtering by arch, os, debug/release, etc.

  For example, the following becomes possible for libevent when building with CMake:

  `$(package)_config_opts_release=-DEVENT__DISABLE_DEBUG_MODE`

  Now the define is only set when not building depends with DEBUG=1

ACKs for top commit:
  hebasto:
    ACK 63c0c4ff10

Tree-SHA512: 17d123219d2f98c017880196966ffebd5d09d3e2db8e443e227eb7e49da46e9d971fbe7fd2b99a324fc367e1bbe0ac3cd15b399bce98dd87fbb144d767e15fe7
2023-04-21 14:29:03 +01:00
MarcoFalke
fac395e5eb ci: Bump ci/lint/Dockerfile
This bump should not be needed, see discussion starting at
https://github.com/bitcoin/bitcoin/pull/27483#issuecomment-1517739626
2023-04-21 14:31:13 +02:00
Ryan Ofsky
eefe56967b bugfix: Fix incorrect debug.log config file path
Currently debug.log will show the wrong bitcoin.conf config file path when
bitcoind is invoked without -conf or -datadir arguments, and there's a default
bitcoin.conf file which specifies another datadir= location. When this happens,
the debug.log will include an incorrect "Config file:" line referring to a
bitcoin.conf file in the other datadir, instead of the referring to the actual
configuration file in the default datadir which was parsed.

The bad log print was reported and originally fixed in
https://github.com/bitcoin/bitcoin/pull/27303 by
Matthew Zipkin <pinheadmz@gmail.com>

This PR takes a slightly different approach to fixing the bug, trying to avoid
future bugs by not allowing the GetConfigFilePath function to be called before
the the configuration is parsed, and deleting GetConfigFile function which
could be confused with GetConfigFilePath. It also includes a test for the bug
which the original fix did not have.

Co-authored-by: Matthew Zipkin <pinheadmz@gmail.com>
2023-04-21 06:53:23 -04:00
fanquake
c63c8a1590 Merge bitcoin/bitcoin#27464: fuzz: re-enable prioritisetransaction & analyzepsbt RPC
faa7144d3c fuzz: re-enable prioritisetransaction & analyzepsbt RPC (MarcoFalke)

Pull request description:

  The linked issue seems fixed, so it should be fine to re-enable

ACKs for top commit:
  dergoegge:
    utACK faa7144d3c

Tree-SHA512: a681c726fceacc27ab5a03d455c7808d33f3cb11fe7d253d455526568af840b29f0c3c1d97c54785ef9277e7891a3aa742ac73ccd3cf115b7606eba50864aaa9
2023-04-21 11:53:23 +01:00
Ryan Ofsky
3746f00be1 init: Error if ignored bitcoin.conf file is found
Show an error on startup if a bitcoin datadir that is being used contains a
`bitcoin.conf` file that is ignored. There are two cases where this could
happen:

- One case reported in
  https://github.com/bitcoin/bitcoin/issues/27246#issuecomment-1470006043
  happens when a bitcoin.conf file in the default datadir (e.g.
  $HOME/.bitcoin/bitcoin.conf) has a "datadir=/path" line that sets different
  datadir containing a second bitcoin.conf file. Currently the second
  bitcoin.conf file is ignored with no warning.

- Another way this could happen is if a -conf= command line argument points
  to a configuration file with a "datadir=/path" line and that specified path
  contains a bitcoin.conf file, which is currently ignored.

This change only adds an error message and doesn't change anything about way
settings are applied. It also doesn't trigger errors if there are redundant
-datadir or -conf settings pointing at the same configuration file, only if
they are pointing at different files and one file is being ignored.
2023-04-21 06:53:23 -04:00
Ryan Ofsky
398c3719b0 lint: Fix lint-format-strings false positives when format specifiers have argument positions
Do not error on valid format specifications like strprintf("arg2=%2$s arg1=%1$s arg2=%2$s", arg1, arg2);

Needed to avoid lint error in upcoming commit: https://cirrus-ci.com/task/4755032734695424?logs=lint#L221

Additionally tested with python -m doctest test/lint/run-lint-format-strings.py
2023-04-21 06:53:23 -04:00
fanquake
4a3f1db4ea depends: latest config.sub 2023-04-21 11:48:26 +01:00
fanquake
ac462c58f9 depends: latest config.guess 2023-04-21 11:47:46 +01:00
fanquake
cfcea12b1f Merge bitcoin/bitcoin#27498: test: Remove unused sanitizer suppressions
fa15a9934e test: Remove unused sanitizer suppressions (MarcoFalke)

Pull request description:

ACKs for top commit:
  hebasto:
    ACK fa15a9934e
  fanquake:
    ACK fa15a9934e

Tree-SHA512: 414dcddb1b2b515b19ec81926876512868bbb05f8ee7d7432c3dacb61dd0b0221c005a687651aa6dd8b8a856cf8391621ce2db0418ba56e81f6af08056f2af1a
2023-04-21 11:33:30 +01:00
fanquake
669af32632 Merge bitcoin/bitcoin#27419: move-only: Extract common/args from util/system
be55f545d5 move-only: Extract common/args and common/config.cpp from util/system (TheCharlatan)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". It is part of a series of patches splitting up the `util/system` files. Its preceding pull request is https://github.com/bitcoin/bitcoin/pull/27254.

  The pull request contains an extraction of ArgsManager related functions from util/system into their own common/ file.

  The background of this commit is an ongoing effort to decouple the libbitcoinkernel library from the ArgsManager. The ArgsManager belongs into the common library, since the kernel library should not depend on it. See [doc/design/libraries.md](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) for more information on this rationale.

ACKs for top commit:
  MarcoFalke:
    re-ACK be55f545d5  🚲
  ryanofsky:
    Code review ACK be55f545d5. Just small cleanups since the last review.
  hebasto:
    ACK be55f545d5, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 90eb03334af0155b823030b4f2ecf286d35058d700ee2ddbbaa445be19e31eb0fe982656f35bd14ecee3ad2c3d0db3746855cb8f3777eff7253713e42873e111
2023-04-21 11:19:08 +01:00
MarcoFalke
fa6eb65167 test: Use python3.8 pow() 2023-04-21 10:19:04 +02:00
MarcoFalke
88881cf7ac Bump python minimum version to 3.8
Also, switch ci_native_qt5 to g++-9 (from g++-8) to work around bugs.
This should be fine, because the i686_centos task still checks for g++-8
compatibility.

See
https://github.com/bitcoin/bitcoin/pull/27483#issuecomment-1513477050
for the list of bugs.
2023-04-21 10:18:19 +02:00
MarcoFalke
fa15a9934e test: Remove unused sanitizer suppressions
* The GCC suppression was fixed in gcc-11, which is available on all LTS
  releases of Linux distros.
* The feerate suppression was likely fixed and does not trigger anymore.
  If it was to trigger again, the underlying bug should be fixed instead
  of suppressing it.
* The bench suppression does not trigger anymore.

Also, add comments to tsan suppressions on how to reproduce.
2023-04-21 09:52:24 +02:00
Amiti Uttarwar
10a354f174 test: prevent intermittent failures
Add to the tried table before the new table to make sure a new table collision
is not possible

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-04-20 19:16:36 -07:00
Andrew Chow
4c40837a45 Merge bitcoin/bitcoin#27412: logging, net: add ASN from peers on logs
0076bed45e logging: log ASN when using `-asmap` (brunoerg)
9836c76ae0 net: add `GetMappedAS` in `CConnman` (brunoerg)

Pull request description:

  When using `-asmap`, you can check the ASN assigned to the peers only with the RPC command `getpeerinfo` (check `mapped_as` field), however, it's not possible to check it in logs (e.g. see in logs the ASN of the peers when a new outbound peer has been connected). This PR includes the peers' ASN in debug output when using `-asmap`.

  Obs: Open this primarily to chase some Concept ACK, I've been using this on my node to facilitate to track the peers' ASN especially when reading the logs.

ACKs for top commit:
  Sjors:
    tACK 0076bed45e
  jamesob:
    ACK 0076bed45e ([`jamesob/ackr/27412.1.brunoerg.logging_net_add_asn_from`](https://github.com/jamesob/bitcoin/tree/ackr/27412.1.brunoerg.logging_net_add_asn_from))
  achow101:
    ACK 0076bed45e

Tree-SHA512: c19cd11e8ab49962021f390459aadf6d33d221ae9a2c3df331a25d6865a8df470e2c8828f6e5219b8a887d6ab5b3450d34be9e26c00cca4d223b4ca64d51111b
2023-04-20 17:20:29 -04:00
Andrew Chow
395b932807 Merge bitcoin/bitcoin#26720: wallet: coin selection, don't return results that exceed the max allowed weight
25ab14712b refactor: coinselector_tests, unify wallet creation code (furszy)
ba9431c505 test: coverage for bnb max weight (furszy)
5a2bc45ee0 wallet: clean post coin selection max weight filter (furszy)
2d112584e3 coin selection: BnB, don't return selection if exceeds max allowed tx weight (furszy)
d3a1c098e4 test: coin selection, add coverage for SRD (furszy)
9d9689e5a6 coin selection: heap-ify SRD, don't return selection if exceeds max tx weight (furszy)
6107ec2229 coin selection: knapsack, select closest UTXO above target if result exceeds max tx size (furszy)
1284223691 wallet: refactor coin selection algos to return util::Result (furszy)

Pull request description:

  Coming from the following comment https://github.com/bitcoin/bitcoin/pull/25729#discussion_r1029324367.

  The reason why we are adding hundreds of UTXO from different sources when the target
  amount is covered only by one of them is because only SRD returns a usable result.

  Context:
  In the test, we create 1515 UTXOs with 0.033 BTC each, and 1 UTXO with 50 BTC. Then
  perform Coin Selection to fund 49.5 BTC.

  As the selection of the 1515 small UTXOs exceeds the max allowed tx size, the
  expectation here is to receive a selection result that only contain the big UTXO.
  Which is not happening for the following reason:

  Knapsack returns a result that exceeds the max allowed transaction size, when
  it should return the closest utxo above the target, so we fallback to SRD who
  selects coins randomly up until the target is met. So we end up with a selection
  result with lot more coins than what is needed.

ACKs for top commit:
  S3RK:
    ACK 25ab14712b
  achow101:
    ACK 25ab14712b
  Xekyo:
    reACK 25ab14712b
  theStack:
    Code-review ACK 25ab14712b

Tree-SHA512: 2425de4cc479b4db999b3b2e02eb522a2130a06379cca0418672a51c4076971a1d427191173820db76a0f85a8edfff100114e1c38fb3b5dc51598d07cabe1a60
2023-04-20 16:33:39 -04:00
Andrew Chow
5aa0c82ccd Merge bitcoin/bitcoin#25325: Add pool based memory resource
9f947fc3d4 Use PoolAllocator for CCoinsMap (Martin Leitner-Ankerl)
5e4ac5abf5 Call ReallocateCache() on each Flush() (Martin Leitner-Ankerl)
1afca6b663 Add PoolResource fuzzer (Martin Leitner-Ankerl)
e19943f049 Calculate memory usage correctly for unordered_maps that use PoolAllocator (Martin Leitner-Ankerl)
b8401c3281 Add pool based memory resource & allocator (Martin Leitner-Ankerl)

Pull request description:

  A memory resource similar to `std::pmr::unsynchronized_pool_resource`, but optimized for node-based containers. The goal is to be able to cache more coins with the same memory usage, and allocate/deallocate faster.

  This is a reimplementation of #22702. The goal was to implement it in a way that is simpler to review & test

  * There is now a generic `PoolResource` for allocating/deallocating memory. This has practically the same API as `std::pmr::memory_resource`. (Unfortunately I cannot use std::pmr because libc++ simply doesn't implement that API).
  * Thanks to sipa there is now a fuzzer for PoolResource! On a fast machine I ran it for ~770 million executions without finding any issue.

  * The estimation of the correct node size is now gone, PoolResource now has multiple pools and just needs to be created large enough to have space for the unordered_map nodes.

  I ran benchmarks with #22702, mergebase, and this PR. Frequency locked Intel i7-8700, clang++ 13.0.1 to reindex up to block 690000.

  ```sh
  bitcoind -dbcache=5000 -assumevalid=00000000000000000002a23d6df20eecec15b21d32c75833cce28f113de888b7 -reindex-chainstate -printtoconsole=0 -stopatheight=690000
  ```

  The performance is practically identical with #22702, just 0.4% slower. It's ~21% faster than master:

  ![Progress in Million Transactions over Time(2)](https://user-images.githubusercontent.com/14386/173288685-91952ade-f304-4825-8bfb-0725a71ca17b.png)

  ![Size of Cache in MiB over Time](https://user-images.githubusercontent.com/14386/173291421-e6b410be-ac77-479b-ad24-5fafcebf81eb.png)
  Note that on cache drops mergebase's memory doesnt go so far down because it does not free the `CCoinsMap` bucket array.

  ![Size of Cache in Million tx over Time(1)](https://user-images.githubusercontent.com/14386/173288703-a80c9c9e-93c8-4a16-9df8-610c89c61cc4.png)

ACKs for top commit:
  LarryRuane:
    ACK 9f947fc3d4
  achow101:
    re-ACK 9f947fc3d4
  john-moffett:
    ACK 9f947fc3d4
  jonatack:
    re-ACK 9f947fc3d4

Tree-SHA512: 48caf57d1775875a612b54388ef64c53952cd48741cacfe20d89049f2fb35301b5c28e69264b7d659a3ca33d4c714d47bafad6fd547c4075f08b45acc87c0f45
2023-04-20 16:20:15 -04:00
Andrew Chow
3a93957a5d Merge bitcoin/bitcoin#27214: addrman: Enable selecting addresses by network
17e705428d doc: clarify new_only param for Select function (Amiti Uttarwar)
b0010c83a1 bench: test select for a new table with only one address (Amiti Uttarwar)
9b91aae085 bench: add coverage for addrman select with network parameter (Amiti Uttarwar)
22a4d1489c test: increase coverage of addrman select (without network) (Amiti Uttarwar)
a98e542e0c test: add addrman test for special case (Amiti Uttarwar)
5c8b4baff2 tests: add addrman_select_by_network test (Amiti Uttarwar)
6b229284fd addrman: add functionality to select by network (Amiti Uttarwar)
26c3bf11e2 scripted-diff: rename local variables to match modern conventions (Amiti Uttarwar)
48806412e2 refactor: consolidate select logic for new and tried tables (Amiti Uttarwar)
ca2a9c5f8f refactor: generalize select logic (Amiti Uttarwar)
052fbcd5a7 addrman: Introduce helper to generalize looking up an addrman entry (Amiti Uttarwar)
9bf078f66c refactor: update Select_ function (Amiti Uttarwar)

Pull request description:

  For the full context & motivation of this patch, see #27213

  This is joint work with mzumsande.

  This PR adds functionality to `AddrMan::Select` to enable callers to specify a network they are interested in.

  Along the way, it refactors the function to deduplicate the logic, updates the local variables to match modern conventions, adds test coverage for both the new and existing `Select` logic, and adds bench tests for the worst case performance of both the new and existing `Select` logic.

  This functionality is used in the parent PR.

ACKs for top commit:
  vasild:
    ACK 17e705428d
  brunoerg:
    re-ACK 17e705428d
  ajtowns:
    ACK 17e705428d
  mzumsande:
    Code Review ACK 17e705428d

Tree-SHA512: e99d1ce0c44a15601a3daa37deeadfc9d26208a92969ecffbea358d57ca951102d759734ccf77eacd38db368da0bf5b6fede3cd900d8a77b3061f4adc54e52d8
2023-04-20 16:07:06 -04:00
fanquake
bbbf89a9de Merge bitcoin/bitcoin#27503: Bump to 25.99 and remove release note fragments
9c24826e7b doc: Remove 25.0 release note fragments (Andrew Chow)
088a93dce8 build: Bump to 25.99 (Andrew Chow)

Pull request description:

  Pre-25.x branch off version bump and release note fragments removal.

  The 25.0 draft release notes are in the dev wiki: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/25.0-Release-Notes-Draft

ACKs for top commit:
  fanquake:
    ACK 9c24826e7b

Tree-SHA512: f7c7b04aa904e946bc672b5b07082a819b9d76ebccda0838bc27d0e6179cfb88b8f110500d5ea815f711580916bcfa0275774ec50a7298a4c66e645647111125
2023-04-20 20:51:20 +01:00
Andrew Chow
9c24826e7b doc: Remove 25.0 release note fragments 2023-04-20 14:01:06 -04:00
Andrew Chow
088a93dce8 build: Bump to 25.99 2023-04-20 13:58:00 -04:00
Andrew Chow
6db0a3002b Merge bitcoin/bitcoin#27488: p2p: update hardcoded mainnet seeds for 25.x
31b1798d2c p2p: update hardcoded mainnet seeds for 25.x (Jon Atack)
04dd1d3926 contrib: make-seeds updates for 25.x (Jon Atack)
f5c8788628 p2p: update manual tor/i2p/cjdns mainnet seeds for 25.x (Jon Atack)

Pull request description:

  Update the hardcoded P2P network seeds for 25.x after updating the manual seeds and the generation script as necessary. Previous update was #25911.

  The manual seeds are selected for reachability, uptime and service bit 1 and/or curated trusted peers. We need more Tor and CJDNS seeds and some of the current Tor and I2P seeds are no longer reachable.

  Can be tested by following the steps in `contrib/seeds/README.md` and verifying the manual seeds by checking their presence and services in getnodeaddresses and/or connecting to them and checking their services with getpeerinfo and behavior with -netinfo.

  Tool output:

  ```
  $ python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt

  Loading asmap database "asmap-filled.dat"…Done.
  Loading and parsing DNS seeds…Done.
    IPv4   IPv6  Onion Pass
    3972   1118      0 Initial
    3972   1118      0 Skip entries with invalid address
    3972   1118      0 After removing duplicates
    3946   1112      0 Enforce minimal number of blocks
    3946   1112      0 Require service bit 1
    2791    798      0 Require minimum uptime
    2757    788      0 Require a known and recent user agent
    2757    788      0 Filter out hosts with multiple bitcoin ports
     512    289      0 Look up ASNs and limit results per ASN and per net```

ACKs for top commit:
  achow101:
    ACK 31b1798d2c
  mzumsande:
    reACK 31b1798d2c

Tree-SHA512: 40cdd0ac74e3d26975ab688ee4af09bedcf15f6e02311757b27c91aafdc4da16cca2be90fee767207bb4ccd542ad19197e4d5e00011185018239de35da19c174
2023-04-20 13:42:01 -04:00
Jon Atack
31b1798d2c p2p: update hardcoded mainnet seeds for 25.x 2023-04-20 06:08:22 -07:00
Jon Atack
04dd1d3926 contrib: make-seeds updates for 25.x
and make the steps in /contrib/seeds/README.md easier to copy-paste
2023-04-20 06:08:22 -07:00
Jon Atack
f5c8788628 p2p: update manual tor/i2p/cjdns mainnet seeds for 25.x
selected for reachability, uptime, and service bit 1
2023-04-20 06:08:22 -07:00
fanquake
3133d935ce Merge bitcoin/bitcoin#27482: kernel: chainparams updates for 25.x
a2bef805c1 kernel: update m_assumed_* chain params for 25.x (fanquake)
4128e01dba kernel: update chainTxData for 25.x (fanquake)
00b2b114b4 kernel: update nMinimumChainWork & defaultAssumeValid for 25.x (fanquake)
07fcc0a82c doc: update references to kernel/chainparams.cpp (fanquake)

Pull request description:

  Update chainparams pre `25.x` branch off.
  Co-Author in the commits as a PR (#27223) had previously been opened too-early to do the same.

  Note: Remember that some variance is expected in the `m_assumed_*` sizes.

ACKs for top commit:
  achow101:
    ACK a2bef805c1
  josibake:
    ACK a2bef805c1
  gruve-p:
    ACK a2bef805c1
  dergoegge:
    ACK a2bef805c1 on the new mainnet params

Tree-SHA512: 0b19c2ef15c6b15863d6a560a1053ee223057c7bfb617ffd3400b1734cee8f75bc6fd7f04d8f8e3f5af6220659a1987951a1b36945d6fe17d06972004fd62610
2023-04-20 11:23:13 +01:00
fanquake
b627924300 Merge bitcoin/bitcoin#26681: contrib: Bugfix for checking bad dns seeds without casting in makeseeds.py
3cc989da5c Fix checking bad dns seeds without casting (Yusuf Sahin HAMZA)

Pull request description:

  - Since seed lines comes with `str` type, comparing `good` column directly with **0** (`int` type) in the if statement was not working at all. This is fixed by casting `int` type to the values in the `good` column of seeds text file.
  - Lines that starts with comment in the seeds text file are now ignored.
  - If statement for checking bad seeds are moved to the top of the `parseline` function as if a seed is bad; there is no point of going forward from there.

  Since this bug-fix eliminates bad seeds over **550k** in the first place, in my case; particular job for parsing all seeds speed is up by **600%** and whole script's speed is up by **%30**.

  Note that **stats** in the terminal are not going to include bad seeds after this fix, which would be the same if this bug were never there before.

ACKs for top commit:
  achow101:
    ACK 3cc989da5c
  jonatack:
    ACK 3cc989da5c

Tree-SHA512: 13c82681de4d72de07293f0b7f09721ad8514a2ad99b0584d1c94fa5f2818821df2000944f9514d6a222a5dccc82856d16c8c05aa36d905cfa7d4610c629fd38
2023-04-20 10:04:47 +01:00
Cory Fields
63c0c4ff10 depends: reuse _config_opts for CMake options
This will allow us to use the existing machinery for filtering by
arch, os, debug/release, etc.

For example, the following becomes possible for libevent:

$(package)_config_opts_release=-DEVENT__DISABLE_DEBUG_MODE

Now the define is only set when not building depends with DEBUG=1
2023-04-19 17:19:51 -04:00
fanquake
d26a71a94a Merge bitcoin/bitcoin#27448: ci: build libc++ in DEBUG mode in MSAN jobs
4de9c2a65f ci: build libc++ with assertions in MSAN jobs (fanquake)
23b8b2026a ci: build libc++ in DEBUG mode in MSAN jobs (fanquake)

Pull request description:

  Followup to #27447.

  See https://releases.llvm.org/16.0.0/projects/libcxx/docs/DesignDocs/DebugMode.html:
  > Libc++ provides a debug mode that enables special debugging checks meant to detect incorrect usage of the standard library. These checks are disabled by default, but they can be enabled by vendors when building the library by using LIBCXX_ENABLE_DEBUG_MODE.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4de9c2a65f

Tree-SHA512: 788c7f031ccf7a6ac96a87758e57f604cf4d9db0144f0ecc4931823111d2396e64ab260825d74f06b2770d0ac3e4e2c21c46f4def046cf3e1a44d705921ab6d2
2023-04-19 18:18:39 +01:00
Matthew Zipkin
8f14fc8622 test: cover fastprune with excessive block size 2023-04-19 11:25:07 -04:00
Martin Zumsande
271c23e87f blockstorage: Adjust fastprune limit if block exceeds blockfile size
If the added block exceeds the blockfile size in test-only
-fastprune mode, the node would get stuck in an infinite loop and
run out of memory.

Avoid this by raising the blockfile size to the size of the added block
in this situation.

Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2023-04-19 11:25:07 -04:00
fanquake
4de9c2a65f ci: build libc++ with assertions in MSAN jobs 2023-04-19 11:54:44 +01:00
fanquake
23b8b2026a ci: build libc++ in DEBUG mode in MSAN jobs 2023-04-19 11:54:44 +01:00
fanquake
d908877c47 Merge bitcoin/bitcoin#27447: depends: Remove _LIBCPP_DEBUG from depends DEBUG mode
bc4fd49d09 depends: add _LIBCPP_ENABLE_ASSERTIONS to DEBUG mode (fanquake)
cf266b2270 depends: Remove _LIBCPP_DEBUG from depends DEBUG mode (fanquake)

Pull request description:

  It was deprecated in LLVM 15, turned into compile-time error in LLVM 16:
  ```bash
  In file included from /usr/lib/llvm-16/bin/../include/c++/v1/cassert:19:
  /usr/lib/llvm-16/bin/../include/c++/v1/__assert:22:5: error: "Defining _LIBCPP_DEBUG is not supported anymore.
  Please use _LIBCPP_ENABLE_DEBUG_MODE instead."
      ^
  1 error generated.
  ```

  and has been removed entirely in LLVM 17 (main): ff573a42cd.

  [Building libc++ in debug mode](https://releases.llvm.org/16.0.0/projects/libcxx/docs/DesignDocs/DebugMode.html), will also automatically set
  `_LIBCPP_ENABLE_DEBUG_MODE` (the new define), so adding it to depends
  doesn't seem useful, and would just result in redefinition errors.

  I'm wondering if as a followup, we could enable a DEBUG build of libc++
  in our MSAN CI job? i.e https://github.com/fanquake/bitcoin/tree/msan_with_enable_debug_mode.

  Somewhat related to https://github.com/google/oss-fuzz/pull/9828, where
  it looks like we'll have to sort out getting a DEBUG build of LLVM, and can drop the commentary about re-enabling DEBUG=1.

ACKs for top commit:
  MarcoFalke:
    lgtm Approach ACK bc4fd49d09

Tree-SHA512: 9c0f48fc428278fbf34fbb8f81e761e232506d7ab28e971cb9a9b9a81d549b4d8bbe51e2f7608d56e489428679231da5b7431443849b238a8a993ad241740282
2023-04-19 11:53:28 +01:00
TheCharlatan
be55f545d5 move-only: Extract common/args and common/config.cpp from util/system
This is an extraction of ArgsManager related functions from util/system
into their own common file.

Config file related functions are moved to common/config.cpp.

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager. The ArgsManager belongs
into the common library, since the kernel library should not depend on
it. See doc/design/libraries.md for more information on this rationale.
2023-04-19 10:48:30 +02:00
Andrew Chow
2fa7344aa9 Merge bitcoin/bitcoin#27484: doc: remove outdated version number usage from release-process
fde224a661 doc: remove outdated version number usage from release-process (fanquake)

Pull request description:

  We no-longer use the leading `0.` version number, and having a mixture is both in the release-process examples is needlessly confusing.

ACKs for top commit:
  achow101:
    ACK fde224a661
  stickies-v:
    ACK fde224a661

Tree-SHA512: c6693b01741921f6870244243640fe857e024bb8581ce57b2d870abf20d257309b97af61fe464427e31a60cda78cf23c1012c6b6a2246250f3670c4b813417d9
2023-04-18 09:56:28 -04:00
fanquake
fde224a661 doc: remove outdated version number usage from release-process
We no-longer use the leading 0. version number, and having a mixture is
both in the release-process examples is needlessly confusing.
2023-04-18 12:36:19 +01:00
fanquake
a2bef805c1 kernel: update m_assumed_* chain params for 25.x
Co-authored-by: johnny9 <985648+johnny9@users.noreply.github.com>
2023-04-18 11:57:29 +01:00
fanquake
4128e01dba kernel: update chainTxData for 25.x
Co-authored-by: johnny9 <985648+johnny9@users.noreply.github.com>
2023-04-18 11:48:16 +01:00
fanquake
00b2b114b4 kernel: update nMinimumChainWork & defaultAssumeValid for 25.x
Co-authored-by: johnny9 <985648+johnny9@users.noreply.github.com>
2023-04-18 11:48:13 +01:00
fanquake
07fcc0a82c doc: update references to kernel/chainparams.cpp 2023-04-18 11:02:05 +01:00
fanquake
bc4fd49d09 depends: add _LIBCPP_ENABLE_ASSERTIONS to DEBUG mode
See
https://releases.llvm.org/16.0.0/projects/libcxx/docs/UsingLibcxx.html#assertions-mode
for more info.
2023-04-18 10:43:20 +01:00
fanquake
cf266b2270 depends: Remove _LIBCPP_DEBUG from depends DEBUG mode
It was deprecated in LLVM 15, turned into a compile-time error in LLVM 16:
```bash
In file included from /usr/lib/llvm-16/bin/../include/c++/v1/cassert:19:
/usr/lib/llvm-16/bin/../include/c++/v1/__assert:22:5: error: "Defining _LIBCPP_DEBUG is not supported anymore.
Please use _LIBCPP_ENABLE_DEBUG_MODE instead."
    ^
1 error generated.
```

and has been removed entirely in LLVM 17 (main),
ff573a42cd.

Building libc++ in debug mode, will also automatically set
`_LIBCPP_ENABLE_DEBUG_MODE` (the new define), so adding it to depends
doesn't seem useful, and would just result in redefinition errors.

I'm wondering if as a followup, we could enable a DEBUG build of libc++
in our MSAN CI job?

Somewhat related to https://github.com/google/oss-fuzz/pull/9828, where
it looks like we'll have to sort out getting a DEBUG build of LLVM.
2023-04-18 10:43:20 +01:00
fanquake
5165984afc Merge bitcoin/bitcoin#27340: ci: Use Cirrus CI dockerfile env
fa4a46de0b ci: Bump nowallet_libbitcoinkernel task to ubuntu:focal (MarcoFalke)
fabc7d90a9 ci: Use credits in more tasks (MarcoFalke)
facae3b149 ci: Use Cirrus CI dockerfile env (MarcoFalke)

Pull request description:

  Currently the CI env has many intermittent issues:

  * The Ubuntu package servers are frequently down
  * Occasionally other stuff is down, such as dnf, pip, or the android sdk
  * Installing packages is slower than downloading them, at least on Cirrus, which has a fast download speed

  Fix all issues by using the Cirrus CI dockerfile env.

ACKs for top commit:
  josibake:
    code review ACK fa4a46de0b

Tree-SHA512: fea5663f7b6dc1c4ea9f87188026ec542b9269bac8ee3398cd58d4df6c86a0af9d275f1876e03f92fb1f6166ec49b817d9e588e6fe1ed54b77592502c2eccd9d
2023-04-18 10:39:39 +01:00
fanquake
467fa89438 Merge bitcoin/bitcoin#27477: test: add regression tests for #27468 (invalid URI segfaults)
6a77d290da test: add regression tests for #27468 (invalid URI segfaults) (Sebastian Falbesoner)

Pull request description:

  Prior to PR #27468 (commit 11422cc572) all call-sites of `GetQueryParameter(...)` in the REST module could trigger a crash. Add missing test cases for all possible code-paths as a regression test, as a foundation for possible follow-up fixes (which aim to resolve this issue in a more general and robust way).

ACKs for top commit:
  stickies-v:
    ACK 6a77d290da
  vasild:
    ACK 6a77d290da

Tree-SHA512: b5dd22d7d448f92236575ea950287259795a957a3f8e364682510c7c1ede5f9d67e7daccc5146c8d0817bcb71742d49273801574bd2bb96e44a9ae5a006ac2a7
2023-04-18 09:19:46 +01:00
Sebastian Falbesoner
6a77d290da test: add regression tests for #27468 (invalid URI segfaults)
Prior to PR #27468 (commit 11422cc572) all
call-sites of `GetQueryParameter(...)` in the REST module could trigger
a crash. Add missing test cases for all possible code-paths as a
regression test, as a foundation for possible follow-up fixes (which aim
to resolve this issue in a more general and robust way).
2023-04-17 18:40:58 +02:00
fanquake
54e07a05b2 Merge bitcoin/bitcoin#27471: test: fix bumpfee 'spend_one_input' occasional failure
e07dd5fff9 test: fix bumpfee 'spend_one_input' occasional failure (furszy)

Pull request description:

  CI test failure, in master: https://cirrus-ci.com/task/5975232842825728.
  In #27469 https://cirrus-ci.com/task/6452468402356224

  Most of the subtests in `wallet_bumpfee.py` expect to find spendable UTXOs of 0.001 btc in the rbf wallet. They use the `spend_one_input()` method which fails if none of them exist.

  The sporadic failure comes from the recently added `test_feerate_checks_replaced_outputs` subtest that can spend all them. Leaving the next subtests with no 0.001 UTXOs to spend.

  To solve it, this PR moves the recently added case into a "context independent subtests" section.
  Which is placed at the end to not affect other cases.

ACKs for top commit:
  achow101:
    ACK e07dd5fff9
  pablomartin4btc:
    ACK e07dd5fff9.

Tree-SHA512: c150ed6fcfbb6f1502eaf6370a57aae0da991c0fc48e8bb3d446805f4336abba5d80ff0de26094914da95432dd0255030fe527001af4510dfdcefbc7230f14d6
2023-04-17 16:31:31 +01:00
fanquake
5d9d6f7fbc Merge bitcoin/bitcoin#27462: depends: fix compiling bdb with clang-16 on aarch64
f8b8458276 depends: fix compiling bdb with clang-16 on aarch64 (fanquake)

Pull request description:

  Compiling bdb with clang-16 on aarch64 (hardware) currently fails:
  ```bash
  make -C depends/ bdb CC=clang CXX=clang++
  ...
  checking for mutexes... UNIX/fcntl
  configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
  configure: error: Unable to find a mutex implementation
  ```

  Looking at config.log we've got:
  ```bash
  configure:18704: checking for mutexes
  configure:18815: clang -o conftest -pipe -std=c11 -O2 -Wno-error=implicit-function-declaration -Wno-error=format-security    -I/bitcoin/depends/aarch64-unknown-linux-gnu/include -D_GNU_SOURCE -D_REENTRANT   -L/bitcoin/depends/aarch64-unknown-linux-gnu/lib conftest.c  -lpthread >&5
  conftest.c:45:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
  main() {
  ^
  int
  conftest.c:50:2: warning: call to undeclared library function 'exit' with type 'void (int) __attribute__((noreturn))'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
          exit (
          ^
  conftest.c:50:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
  1 warning and 1 error generated.
  ```

  Clang-16 changed `-Wimplicit-function-declaration` and `-Wimplicit-int`
  warnings into errors, see:
  https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#potentially-breaking-changes.

  > The -Wimplicit-function-declaration and -Wimplicit-int warnings now
  > default to an error in C99, C11, and C17. As of C2x, support for implicit
  > function declarations and implicit int has been removed, and the
  > warning options will have no effect. Specifying -Wimplicit-int in
  > C89 mode will now issue warnings instead of being a noop.

ACKs for top commit:
  hebasto:
    ACK f8b8458276, tested on Ubuntu Lunar (`aarch64`) with:

Tree-SHA512: 5ca078b1c00915446e9f0f2ecaa4342295a2097996554345753315d1c81c23000c57be14e1ac5506a87820f5114aba748456f2c2b6426b0810504d62d761a787
2023-04-17 16:19:14 +01:00
Andrew Chow
4ad20a2258 Merge bitcoin/bitcoin#27473: bugfix: Properly handle "unknown" Address Type
0d6383fda0 Don't return OutputType::UNKNOWN in ParseOutputType (Pttn)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/27472 by also handling at the relevant places the case where ParseOutputType returns `OutputType::UNKNOWN`, and not just when it returns `std::nullopt`.

ACKs for top commit:
  achow101:
    ACK 0d6383fda0
  MarcoFalke:
    lgtm ACK 0d6383fda0
  furszy:
    ACK 0d6383fda0

Tree-SHA512: 776793027b926283d3162e69fb9c8883c814b19bcce4574ccdf8e3140a1ec4ebc4aa8ccd1abae7ef3571f942d2e6c35305fd1244259540d90605106e01afc34c
2023-04-17 10:18:04 -04:00
fanquake
e054b7390c Merge bitcoin/bitcoin#27468: bugfix: rest: avoid segfault for invalid URI
11422cc572 bugfix: rest: avoid segfault for invalid URI (pablomartin4btc)

Pull request description:

  Minimal fix to get it promptly into 25.0 release (suggested by  [stickies-v](https://github.com/bitcoin/bitcoin/pull/27253#pullrequestreview-1385130381) and supported by [vasild](https://github.com/bitcoin/bitcoin/pull/27253#pullrequestreview-1385842606)  )

  Please check #27253 for reviewers comments and acks regarding this PR and read the commit comment message body for more details about the fix.

ACKs for top commit:
  achow101:
    ACK 11422cc572
  stickies-v:
    re-ACK 11422cc

Tree-SHA512: 5af6b53fb266a12b463f960910556d5e97bc88b3c2a4f437ffa343886b38749e1eb058cf7bc64d62e82e1acf6232a186bddacd8f3b4500c87bf9e550a0153386
2023-04-17 15:11:15 +01:00
Jon Atack
11bb31c1c4 p2p: "skip netgroup diversity of new connections for tor/i2p/cjdns" follow-up
In PR 27374, the semantics of the `setConnected` data structure in
CConnman::ThreadOpenConnections changed from the set of outbound peer
netgroups to those of outbound IPv4/6 peers only.

This commit updates a code comment in this regard about feeler connections and
updates the naming of `setConnected` to `outbound_ipv46_peer_netgroups` to
reflect its new role.
2023-04-17 06:27:24 -07:00
pablomartin4btc
11422cc572 bugfix: rest: avoid segfault for invalid URI
`evhttp_uri_parse` can return a nullptr, for example when the URI
contains invalid characters (e.g. "%").
`GetQueryParameterFromUri` passes the output of `evhttp_uri_parse`
straight into `evhttp_uri_get_query`, which means that anyone calling
a REST endpoint in which query parameters are used (e.g. `rest_headers`)
can cause a segfault.

This bugfix is designed to be minimal and without additional behaviour change.
Follow-up work should be done to resolve this in a more general and robust way,
so not every endpoint has to handle it individually.
2023-04-17 10:13:34 -03:00
glozow
bf77fc9cb4 [test] mempool full in package accept 2023-04-17 12:35:23 +01:00
glozow
b51ebccc28 [validation] set PackageValidationState when mempool full 2023-04-17 10:52:04 +01:00
fanquake
f8b8458276 depends: fix compiling bdb with clang-16 on aarch64
Compiling bdb with clang-16 on aarch64 (hardware) currently fails:
```bash
make -C depends/ bdb CC=clang CXX=clang++
...
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
```

Looking at config.log we've got:
```bash
configure:18704: checking for mutexes
configure:18815: clang -o conftest -pipe -std=c11 -O2 -Wno-error=implicit-function-declaration -Wno-error=format-security    -I/bitcoin/depends/aarch64-unknown-linux-gnu/include -D_GNU_SOURCE -D_REENTRANT   -L/bitcoin/depends/aarch64-unknown-linux-gnu/lib conftest.c  -lpthread >&5
conftest.c:45:1: error: type specifier missing, defaults to 'int'; ISO C99 and later do not support implicit int [-Wimplicit-int]
main() {
^
int
conftest.c:50:2: warning: call to undeclared library function 'exit' with type 'void (int) __attribute__((noreturn))'; ISO C99 and later do not support implicit function declarations [-Wimplicit-function-declaration]
        exit (
        ^
conftest.c:50:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
1 warning and 1 error generated.
```

Clang-16 changed `-Wimplicit-function-declaration` and `-Wimplicit-int`
warnings into errors, see:
https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#potentially-breaking-changes.

> The -Wimplicit-function-declaration and -Wimplicit-int warnings now
> default to an error in C99, C11, and C17. As of C2x, support for implicit
> function declarations and implicit int has been removed, and the
> warning options will have no effect. Specifying -Wimplicit-int in
> C89 mode will now issue warnings instead of being a noop.
2023-04-17 10:14:25 +01:00
glozow
563a2ee4f5 [policy] disallow transactions under min relay fee, even in packages
Avoid adding transactions below min relay feerate because, even if they
were bumped through CPFP when entering the mempool, we do not have a
DoS-resistant way of ensuring they always remain bumped.  In the future,
this rule can be relaxed (e.g. to allow packages to bump 0-fee
transactions) if we find a way to do so.
2023-04-17 09:53:59 +01:00
glozow
c4554fe894 [test] package cpfp bumps parents <mempoolminfee but >=minrelaytxfee 2023-04-17 09:52:25 +01:00
glozow
ac463e87df [test util] mock mempool minimum feerate 2023-04-17 09:52:25 +01:00
Pttn
0d6383fda0 Don't return OutputType::UNKNOWN in ParseOutputType
Fixes https://github.com/bitcoin/bitcoin/issues/27472

Signed-off-by: Pttn <28868425+Pttn@users.noreply.github.com>
2023-04-16 23:48:05 +02:00
furszy
e07dd5fff9 test: fix bumpfee 'spend_one_input' occasional failure
Most of the subtests in wallet_bumpfee.py expect to
find spendable UTXOs of 0.001 btc in the rbf wallet
(they use the 'spend_one_input()' method that tries
to spend one of them and if it doesn't find any, it
throws an exception).

The sporadic failure comes from the recently added
'test_feerate_checks_replaced_outputs' subtest that
can spend all them. Leaving the next subtests with
no 0.001 UTXOs to spend.

To solve it, this PR moves the recently added case
into a "context independent subtests" section.
Which is placed at the end to not affect other cases.
2023-04-15 23:01:45 -03:00
furszy
be72663a15 test: bumpfee, add coverage for "send coins back to yourself" 2023-04-15 10:44:51 -03:00
furszy
7bffec6715 bumpfee: enable send coins back to yourself
Simple example:

1) User_1 sends 0.1 btc to user_2 on a low fee transaction.
2) After few hours, the tx is still in the mempool, user_2
   is not interested anymore, so user_1 decides to cancel
   it by sending coins back to himself.
3) User_1 has the bright idea of opening the explorer and
   copy the change output address of the transaction. Then
   call bumpfee providing such output (in the "outputs" arg).

Currently, this is not possible. The wallet fails with
"Unable to create transaction. Transaction must have at least
one recipient" error.
The error reason is that we discard the provided output from
the recipients list and set it inside the coin control
so the process adds it later (when the change is calculated).
But.. there is no later if the tx has no outputs.
2023-04-15 10:44:50 -03:00
fanquake
b22c275582 Merge bitcoin/bitcoin#27465: doc: fix typo in developer-notes.md
f24f4fa3f1 Update developer-notes.md (Riahiamirreza)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK f24f4fa3f1

Tree-SHA512: 10301170dff6f2f7b47a229ba99f4a4f4953c361be24996b6dc70343ad118879cd90ebb54d78cd31c852f577fb17f9726582fdd02ed5b6fd7b71566942e8b408
2023-04-15 12:56:49 +01:00
fanquake
90bfa9d2d7 Merge bitcoin/bitcoin#27308: bumpfee: avoid making bumped transactions with too low fee when replacing outputs
d52fa1b0a5 tests: Make sure that bumpfee feerate checks work when replacing outputs (Andrew Chow)
be177c15a4 bumpfee: Check the correct feerate when replacing outputs (Andrew Chow)

Pull request description:

  When replacing the outputs of a transaction during `bumpfee`, it is possible to accidentally create a transaction that will not be accepted into the mempool as it does not meet the incremental relay fee requirements. This occurs because the size estimation used for checking the provided feerate does not account for the replaced outputs; it instead uses the original outputs. When the replaced outputs is significantly different from the original, there can be a large difference in estimated transaction sizes that can make a transaction miss the absolute fee requirements for the incremental relay fee. Unfortunately we do not currently inform the user when the bumped transaction fails to relay, so they could use `bumpfee` and think the transaction has been bumped when it actually has not.

  This issue is resolved by replacing the outputs before doing the size estimation, and also updating the feerate checker to use the actual fee values when calculating the required minimum fee.

  Also added a test for this scenario.

ACKs for top commit:
  ishaanam:
    reACK d52fa1b0a5
  Xekyo:
    reACK d52fa1b0a5

Tree-SHA512: d18301b587465322dd3fb1bb86496c3675265a56072047576e2baa5cf907dd3b54778f30721f662f0c235709a5568427c18542eb7efbfb6fdd9f481fe676c66b
2023-04-15 12:55:10 +01:00
fanquake
3650e74808 Merge bitcoin/bitcoin#27445: Update src/secp256k1 subtree to release v0.3.1
621c17869d Respect and update FILES_ARGS in test/lint/lint-python.py (Pieter Wuille)
719a74989b Disable Python lint in src/secp256k1 (Pieter Wuille)
c981671e9b Squashed 'src/secp256k1/' changes from bdf39000b9..4258c54f4e (Pieter Wuille)

Pull request description:

  There is no strict need for any of the changes in v0.3.1 (compared to the v0.3.0 that's currently subtreed) for Bitcoin Core release builds, but if anyone may compile Bitcoin Core from source using Clang v14+, this will prevent known timing leaks in the signing/keygen logic.

  This also includes a CI fix from libsecp256k1 master (on top of 0.3.1) which fixes Wycheproof test vector generation.

  I also had to amend some of the linters to avoid enforcing their rules on the .py files in the secp256k1 subtree.

ACKs for top commit:
  real-or-random:
    utACK 621c17869d subtree matches. diff to linter script looks good
  fanquake:
    ACK 621c17869d

Tree-SHA512: 059722540a4fd387d9e231036e59685db373c085a346c7a9d2b87eac3ffe538099356b5f06fc2112a1df80e3818d80fe380f27a47901496e8092c836ea3ee14d
2023-04-15 12:42:11 +01:00
Riahiamirreza
f24f4fa3f1 Update developer-notes.md 2023-04-14 20:11:51 +03:30
MarcoFalke
faa7144d3c fuzz: re-enable prioritisetransaction & analyzepsbt RPC 2023-04-14 17:34:22 +02:00
Pieter Wuille
621c17869d Respect and update FILES_ARGS in test/lint/lint-python.py 2023-04-14 10:52:33 -04:00
Pieter Wuille
719a74989b Disable Python lint in src/secp256k1 2023-04-14 10:36:36 -04:00
Pieter Wuille
c981671e9b Squashed 'src/secp256k1/' changes from bdf39000b9..4258c54f4e
4258c54f4e Merge bitcoin-core/secp256k1#1276: autotools: Don't regenerate Wycheproof header automatically
06c67dea9f autotools: Don't regenerate Wycheproof header automatically
3bab71cf05 Merge bitcoin-core/secp256k1#1268: release cleanup: bump version after 0.3.1
656c6ea8d8 release cleanup: bump version after 0.3.1
346a053d4c Merge bitcoin-core/secp256k1#1269: changelog: Fix link
6a37b2a5ea changelog: Fix link
ec98fcedd5 Merge bitcoin-core/secp256k1#1266: release: Prepare for 0.3.1
898e1c676e release: Prepare for 0.3.1
1d9a13fc26 changelog: Remove inconsistent newlines
0e091669a1 changelog: Catch up in preparation of 0.3.1
7b7503dac5 Merge bitcoin-core/secp256k1#1245: tests: Add Wycheproof ECDSA vectors
145078c418 Merge bitcoin-core/secp256k1#1118: Add x-only ecmult_const version with x specified as n/d
e5de454609 tests: Add Wycheproof ECDSA vectors
0f8642079b Add exhaustive tests for ecmult_const_xonly
4485926ace Add x-only ecmult_const version for x=n/d
a0f4644f7e Merge bitcoin-core/secp256k1#1252: Make position of * in pointer declarations in include/ consistent
4e682626a3 Merge bitcoin-core/secp256k1#1226: Add CMake instructions to release process
2d51a454fc Merge bitcoin-core/secp256k1#1257: ct: Use volatile "trick" in all fe/scalar cmov implementations
4a496a36fb ct: Use volatile "trick" in all fe/scalar cmov implementations
3d1f430f9f Make position of * in pointer declarations in include/ consistent
2bca0a5cbf Merge bitcoin-core/secp256k1#1241: build: Improve `SECP_TRY_APPEND_DEFAULT_CFLAGS` macro
afd8b23b27 Merge bitcoin-core/secp256k1#1244: Suppress `-Wunused-parameter` when building for coverage analysis
1d8f367515 Merge bitcoin-core/secp256k1#1250: No need to subtract 1 before doing a right shift
3e43041be6 No need to subtract 1 before doing a right shift
3addb4c1e8 build: Improve `SECP_TRY_APPEND_DEFAULT_CFLAGS` macro
0c07c82834 Add CMake instructions to release process
464a9115b4 Merge bitcoin-core/secp256k1#1242: Set ARM ASM symbol visibility to `hidden`
f16a709fd6 Merge bitcoin-core/secp256k1#1247: Apply Checks only in VERIFY mode.
70be3cade5 Merge bitcoin-core/secp256k1#1246: Typo
4ebd82852d Apply Checks only in VERIFY mode.
d1e7ca192d Typo
5bb03c2911 Replace `SECP256K1_ECMULT_TABLE_VERIFY` macro by a function
9c8c4f443c Merge bitcoin-core/secp256k1#1238: build: bump CMake minimum requirement to 3.13
0cf2fb91ef Merge bitcoin-core/secp256k1#1243: build: Ensure no optimization when building for coverage analysis
fd2a408647 Set ARM ASM symbol visibility to `hidden`
4429a8c218 Suppress `-Wunused-parameter` when building for coverage analysis
8e79c7ed11 build: Ensure no optimization when building for coverage analysis
96dd062511 build: bump CMake minimum requirement to 3.13
427bc3cdcf Merge bitcoin-core/secp256k1#1236: Update comment for secp256k1_modinv32_inv256
647f0a5cb1 Update comment for secp256k1_modinv32_inv256
5658209459 Merge bitcoin-core/secp256k1#1228: release cleanup: bump version after 0.3.0
28e63f7ea7 release cleanup: bump version after 0.3.0

git-subtree-dir: src/secp256k1
git-subtree-split: 4258c54f4ebfc09390168e8a43306c46b315134b
2023-04-14 10:35:51 -04:00
Pieter Wuille
f5fdd4e279 Update src/secp256k1 to latest upstream master (v0.3.1 + CI fix) 2023-04-14 10:35:51 -04:00
Andrew Chow
69460bd8bc Merge bitcoin/bitcoin#27461: verify-commits: error and exit cleanly when git is too old.
1fefcf27ed verify-commits: error and exit cleanly when git is too old. (Cory Fields)

Pull request description:

  Requested by fanquake. Rather than failing with a cryptic error with older git, fail gracefully and mention why.

  The new option semantics [are explained here](1f0c3a29da).

  Note: my local git versions are currently too old to test the new functionality, so I've only verified the failure case.

ACKs for top commit:
  josibake:
    ACK 1fefcf27ed
  achow101:
    ACK 1fefcf27ed

Tree-SHA512: f3dc583edf6ff6ff9bf06f33de967e10b8423ce62e7370912ffdca8a4ca4bfe4c2e783e9ad76281ce9e6748a4643d187aa5cb4a6b9ec4c1582910f02b94b6e3c
2023-04-14 09:27:11 -04:00
Andrew Chow
2bfe43db16 Merge bitcoin/bitcoin#27374: p2p: skip netgroup diversity of new connections for tor/i2p/cjdns
b5585ba5f9 p2p: skip netgroup diversity of new connections for tor/i2p/cjdns networks (stratospher)

Pull request description:

  Follow up for #27264.

  In order to make sure that our persistent outbound slots belong to different netgroups, distinct net groups of our peers are added to `setConnected`. We’d only open a persistent outbound connection to peers which have a different netgroup compared to those netgroups present in `setConnected`.

  Current `GetGroup()` logic assumes route-based diversification behaviour for tor/i2p/cjdns addresses (addresses are public key based and not route-based). Distinct netgroups possible (according to the current `GetGroup()` logic) for:
  1. tor => 030f, 031f, .. 03ff (16 possibilities)
  2. i2p => 040f, 041f, .. 04ff (16 possibilities)
  3. cjdns => 05fc0f, 05fc1f, ... 05fcff (16 possibilities)

  `setConnected` is used in `ThreadOpenConnections()` before making [outbound](84f4ac39fd/src/net.cpp (L1846)) and [anchor](84f4ac39fd/src/net.cpp (L1805)) connections to new peers so that they belong to distinct netgroups.

  **behaviour on master**

  - if we run a node only on tor/i2p/cjdns
  - we wouldn't be able to open more than 16 outbound connections(manual, block-relay-only anchor, outbound full relay, block-relay-only connections) because we run out of possible netgroups.
  - see https://github.com/bitcoin/bitcoin/pull/27264#issuecomment-1481322628
  - tested by changing `MAX_OUTBOUND_FULL_RELAY_CONNECTIONS` to 17 with `onlynet=onion` and observed how node wouldn't make more than 16 outbound connections.

  **behaviour on PR**

  - netgroup diversity checks are skipped for tor/i2p/cjdns addresses.
  - we don't insert tor/i2p/cjdns address in `setConnected` and `GetGroup` doesn't get called on tor/i2p/cjdns(see #27369)

ACKs for top commit:
  achow101:
    ACK b5585ba5f9
  mzumsande:
    ACK b5585ba5f9
  vasild:
    ACK b5585ba5f9

Tree-SHA512: c120b3f9ca7f0be3f29ea665cd2f7dfb40cd1d7ec7058984252fb6e0295e414f736c5b4fba03c31188188a5ae4f543fb2654f6ee9776bad745c7ca72d23d5b9b
2023-04-13 18:21:54 -04:00
Cory Fields
1fefcf27ed verify-commits: error and exit cleanly when git is too old. 2023-04-13 21:07:06 +00:00
Hennadii Stepanov
19764dc143 Merge bitcoin-core/gui#726: Register wallet::AddressPurpose type
a45b54406d qt: Register `wallet::AddressPurpose` type (Hennadii Stepanov)

Pull request description:

  This PR is a follow up of bitcoin/bitcoin#27217.

  Fixes #725.

ACKs for top commit:
  achow101:
    ACK a45b54406d
  furszy:
    Tested ACK a45b54406d

Tree-SHA512: c670f4bf56442613d3fe038b0ba21acfcd4c69aa5340072e9a77d83f5fab1bf2facd87a9e1f42d88f496d277b27b79e7090444d59a9b9e71f3b486e171daa669
2023-04-13 14:41:28 +01:00
fanquake
cd59bb2f52 Merge bitcoin/bitcoin#27459: ci: explicitly install libclang-rt-dev in valgrind jobs
2c60826b50 ci: explicitly install libclang-rt-dev in valgrind jobs (fanquake)

Pull request description:

  This fixes some cases, i.e under --no-install-recommends, where libclang-rt-dev wouldn't be installed, and configuring would then fail.

  Followup to #27444.

Top commit has no ACKs.

Tree-SHA512: d1ab0050731df47c21f6ac4f575a728b045b4617beaa1fa8b878050e07e5ddda18fb7d066c7b32bee5ed0ac0e878958a812d4c6b5bd704612755ccb3c172d7e2
2023-04-13 12:16:29 +01:00
Hennadii Stepanov
a45b54406d qt: Register wallet::AddressPurpose type 2023-04-13 12:03:06 +01:00
fanquake
2c60826b50 ci: explicitly install libclang-rt-dev in valgrind jobs
This fixes some cases, i.e under --no-install-recommends, where
libclang-rt-dev wouldn't be installed, and configuring would then fail.

Followup to #27444.
2023-04-13 11:22:39 +01:00
fanquake
cd603edeef Merge bitcoin/bitcoin#27444: ci: use Debian Bookworm and Valgrind 3.19 in Valgrind jobs
e047ae84d2 valgrind: update supps for Debian Bookworm. (fanquake)
ba29143d98 ci: use Debian Bookworm and Valgrind 3.19 in Valgrind jobs (fanquake)

Pull request description:

  Switch to using Debian Bookworm and [valgrind 3.19](https://packages.debian.org/bookworm/valgrind) in the Valgrind jobs. Also update the suppressions file.

  This originally contained a changed to build valgrind 3.20 from source (for improved aarch64 support), but I'll split that into it's own change.

Top commit has no ACKs.

Tree-SHA512: 73ec162d6e07f8a6767d15c0fc298ec6e1a2ba8ec8f9ea902dbfd0a1e3c491411781beec2f6de66fd15006475dbc024bc512f09aa94e2615b713ba873fac14de
2023-04-13 10:23:24 +01:00
Andrew Chow
6a167325f0 Merge bitcoin/bitcoin#27279: Add "warnings", deprecate "warning" in {create,load,unload,restore}wallet
7ccdd741fe test: fix importmulti/importdescriptors assertion (Jon Atack)
19d888ce40 rpc: move WALLET_FLAG_CAVEATS to the compilation unit of its caller (Jon Atack)
01df011ca2 doc: release note for wallet RPCs "warning" field deprecation (Jon Atack)
9ea8b3739a test: createwallet "warning" field deprecation test (Jon Atack)
645d7f75ac rpc: deprecate "warning" field in {create,load,unload,restore}wallet (Jon Atack)
2f4a926e95 test: add test coverage for "warnings" field in createwallet (Jon Atack)
4a1e479ca6 rpc: add "warnings" field to RPCs {create,load,unload,restore}wallet (Jon Atack)
079d8cdda8 rpc: extract wallet "warnings" fields to a util helper (Jon Atack)
f73782a903 doc: fix/improve warning helps in {create,load,unload,restore}wallet (Jon Atack)

Pull request description:

  Based on discussion and concept ACKed in #27138, add a `warnings` field to RPCs createwallet, loadwallet, unloadwallet, and restorewallet as a JSON array of strings to replace the `warning` string field in these 4 RPCs. The idea is to more gracefully handle multiple warning messages and for consistency with other wallet RPCs.  Then, deprecate the latter fields, which represent all the remaining RPC `warning` fields.

  The first commit f73782a903 implements https://github.com/bitcoin/bitcoin/pull/27138#issuecomment-1474789198 as an alternative to #27138. One of those two could potentially be backported to our currently supported releases.

ACKs for top commit:
  achow101:
    ACK 7ccdd741fe
  1440000bytes:
    utACK 7ccdd741fe
  vasild:
    ACK 7ccdd741fe
  pinheadmz:
    re-ACK 7ccdd741fe

Tree-SHA512: 314e0a4c41fa383d95e2817bfacf359d449e460529d235c3eb902851e2f4eacbabe646d9a5a4beabc4964cdfabf6397ed8301366a58d344a2f787f83b75e9d64
2023-04-12 13:09:23 -04:00
fanquake
e047ae84d2 valgrind: update supps for Debian Bookworm.
Remove no-longer-required libstdc++ suppression.
Remove unused (and versioned) GUI suppression.
2023-04-12 17:45:32 +01:00
fanquake
ba29143d98 ci: use Debian Bookworm and Valgrind 3.19 in Valgrind jobs
https://packages.debian.org/bookworm/valgrind
2023-04-12 17:45:14 +01:00
fanquake
7f4ab67e7b Merge bitcoin/bitcoin#27449: doc: update OpenBSD build docs for 7.3 (external signer support available)
6b17994ede doc: update OpenBSD build docs for 7.3 (external signer support available) (Sebastian Falbesoner)

Pull request description:

  With OpenBSD 7.3, the waitid(2) system call is implemented (see 8112871f19, first mentioned kernel improvement at https://www.openbsd.org/73.html).

  This means Boost.Process finally doesn't fail to compile anymore and we can remove the build hint about missing external signer support. Tested on my amd64 machine by reconfiguring / rebuilding master branch and successfully running the functional test wallet_signer.py. ✔️

ACKs for top commit:
  fanquake:
    ACK 6b17994ede - haven't tested, but looks good to me.

Tree-SHA512: 5bbcecce4ced38d8221f2c906a54667e50317e9ded182554cf73bb7f2fce55a38e53730eca25f813cff1d2d65c94141eb158d40f83228d12dcf859c16a1798b9
2023-04-12 15:50:09 +01:00
Ryan Ofsky
a5986e82dd refactor: Remove CAddressBookData::destdata
This is cleanup that doesn't change external behavior.

- Removes awkward `StringMap` intermediate representation
- Simplifies CWallet code, deals with used address and received request
  serialization in walletdb.cpp
- Adds test coverage and documentation
- Reduces memory usage

This PR doesn't change externally observable behavior. Internally, only change
in behavior is that EraseDestData deletes directly from database because the
`StringMap` is gone. This is more direct and efficient because it uses a single
btree lookup and scan instead of multiple lookups

Motivation for this cleanup is making changes like #18550, #18192, #13756
easier to reason about and less likely to result in unintended behavior and
bugs

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-04-12 05:30:43 -04:00
fanquake
cae0608ad4 Merge bitcoin/bitcoin#27217: wallet: Replace use of purpose strings with an enum
18fc71a3ad doc: Release note for purpose string restriction (Andrew Chow)
e83babe3b8 wallet: Replace use of purpose strings with an enum (Andrew Chow)
2f80005136 wallet: add AddressPurpose enum to replace string values (Ryan Ofsky)
8741522e6c wallet: Add wallet/types.h for simple public enum and struct types (Ryan Ofsky)

Pull request description:

  Instead of storing and passing around fixed strings for the purpose of an address, use an enum.

ACKs for top commit:
  josibake:
    reACK 18fc71a3ad

Tree-SHA512: 82034f020e96b99b29da34dfdd7cfe58f8b7d2afed1409ea4a290c2cac69fc43e449e8b7b2afd874a9facf8f4cd6ebb80d17462317e60a6f011ed8f9eab5d4c5
2023-04-12 10:30:43 +01:00
Ryan Ofsky
5938ad0bdb wallet: Add DatabaseBatch::ErasePrefix method
This new function is not used yet this commit, but next commit adds usages and
test coverage for both BDB and sqlite.
2023-04-12 05:30:43 -04:00
Sebastian Falbesoner
6b17994ede doc: update OpenBSD build docs for 7.3 (external signer support available)
With OpenBSD 7.3, the waitid(2) system call is implemented (see
8112871f19).

This means Boost.Process finally doesn't fail to compile anymore and we
can remove the build hint about missing external signer support. Tested
on my amd64 machine by reconfiguring / rebuilding master branch and
successfully running the functional test wallet_signer.py.
2023-04-11 22:59:08 +02:00
Andrew Chow
18fc71a3ad doc: Release note for purpose string restriction 2023-04-11 15:55:32 -04:00
Andrew Chow
e83babe3b8 wallet: Replace use of purpose strings with an enum
Instead of storing and passing around fixed strings for the purpose of
an address, use an enum.

This also rationalizes the CAddressBookData struct, documenting all fields and
making them public, and simplifying the representation to avoid bugs like
https://github.com/bitcoin/bitcoin/pull/26761#discussion_r1134615114 and make
it not possible to invalid address data like change addresses with labels.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-04-11 15:55:31 -04:00
Ryan Ofsky
2f80005136 wallet: add AddressPurpose enum to replace string values 2023-04-11 15:52:25 -04:00
Ryan Ofsky
8741522e6c wallet: Add wallet/types.h for simple public enum and struct types
Move isminetype and isminefilter there this commit, add WalletPurpose type next
commit.
2023-04-11 15:52:25 -04:00
Andrew Chow
27dcc07c08 Merge bitcoin/bitcoin#26699: wallet, gui: bugfix, getAvailableBalance skips selected coins
68eed5df86 test,gui: add coverage for PSBT creation on legacy watch-only wallets (furszy)
306aab5bb4 test,gui: decouple widgets and model into a MiniGui struct (furszy)
2f76ac0383 test,gui: decouple chain and wallet initialization from test case (furszy)
cd98b71739 gui: 'getAvailableBalance', include watch only balance (furszy)
74eac3a82f test: add coverage for 'useAvailableBalance' functionality (furszy)
dc1cc1c359 gui: bugfix, getAvailableBalance skips selected coins (furszy)

Pull request description:

  Fixes https://github.com/bitcoin-core/gui/issues/688 and https://github.com/bitcoin/bitcoin/issues/26687.

  First Issue Description (https://github.com/bitcoin-core/gui/issues/688):

  The previous behavior for `getAvailableBalance`, when the coin control had selected coins, was to return the sum of them. Instead, we are currently returning the wallet's available total balance minus the selected coins total amount.

  Reason:
  Missed to update the `GetAvailableBalance` function to include the coin control selected coins on #25685.

  Context:
  Since #25685 we skip the selected coins inside `AvailableCoins`, the reason is that there is no need to waste resources walking through the entire wallet's txes map just to get coins that could have gotten by just doing a simple `mapWallet.find`).

  Places Where This Generates Issues (only when the user manually select coins via coin control):
  1) The GUI balance check prior the transaction creation process.
  2) The GUI "useAvailableBalance" functionality.

  Note 1:
  As the GUI uses a balance cache since https://github.com/bitcoin-core/gui/pull/598, this issue does not affect the regular spending process. Only arises when the user manually select coins.

  Note 2:
  Added test coverage for the `useAvailableBalance` functionality.

  ----------------------------------

  Second Issue Description (https://github.com/bitcoin/bitcoin/issues/26687):

  As we are using a cached balance on `WalletModel::getAvailableBalance`,
  the function needs to include the watch-only available balance for wallets
  with private keys disabled.

ACKs for top commit:
  Sjors:
    tACK 68eed5df86
  achow101:
    ACK 68eed5df86
  theStack:
    ACK 68eed5df86

Tree-SHA512: 674f3e050024dabda2ff4a04b9ed3750cf54a040527204c920e1e38bd3d7f5fd4d096e4fd08a0fea84ee6abb5070f022b5c0d450c58fd30202ef05ebfd7af6d3
2023-04-11 14:05:55 -04:00
fanquake
c17d4d3b6b Merge bitcoin/bitcoin#26662: fuzz: Add HeadersSyncState target
3153e7d779 [fuzz] Add HeadersSyncState target (dergoegge)
53552affca [headerssync] Make m_commit_offset protected (dergoegge)

Pull request description:

  This adds a fuzz target for the `HeadersSyncState` class.

  I am unsure how well this is able to cover the logic since it is just processing unserialized CBlockHeaders straight from the fuzz input (headers are sometimes made continuous). However, it does manage to get to the redownload phase so i thought it is better then not having fuzzing at all.

  It would also be nice to fuzz the p2p logic that is using `HeadersSyncState` (e.g. `TryLowWorkHeadersSync`, `IsContinuationOfLowWorkHeadersSync`) but that likely requires some more work (refactoring👻).

ACKs for top commit:
  mzumsande:
    ACK 3153e7d779

Tree-SHA512: 8a4630ceeeb30e4eeabaa8eb5491d98f0bf900efe7cda07384eaac9f2afaccfbcaa979cc1cc7f0b6ca297a8f5c17a7759f94809dd87eb87d35348d847c83e8ab
2023-04-11 16:17:04 +01:00
MarcoFalke
fa4a46de0b ci: Bump nowallet_libbitcoinkernel task to ubuntu:focal
This is needed to work around
https://github.com/bitcoin/bitcoin/pull/27340#issuecomment-1484988445

The only change should be that python3.7 is bumped to 3.8, but this is
fine because ci/test/00_setup_env_native_qt5.sh still checks for
python3.7 compatibility.
2023-04-11 14:12:37 +02:00
MarcoFalke
fabc7d90a9 ci: Use credits in more tasks
This should give faster feedback about the CI result, while still
keeping expenses reasonable.
2023-04-11 14:12:30 +02:00
MarcoFalke
facae3b149 ci: Use Cirrus CI dockerfile env 2023-04-11 14:11:48 +02:00
fanquake
53eb4b7a21 Merge bitcoin/bitcoin#27270: refactor, net processing: Avoid CNode::m_relays_txs usage
55c4795c57 [net processing] Use TxRelay::m_relay_txs over CNode::m_relays_txs (dergoegge)

Pull request description:

  `CNode::m_relays_txs` is meant to only be used for the eviction logic in `net`. `TxRelay::m_relay_txs` will hold the same value and is meant to be used on the application layer to determine if we will/should relay transactions to a peer.

  (Shameless plug: we should really better specify the interface for updating eviction data to avoid refactors like this in the future -> #25572)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 55c4795c57

Tree-SHA512: 59cfd23e32568fd96cda5570790e518242a6c76d4edf5b7d1a2a7f9724d590d2a38395504e05be0af4e98dd5c0056fc0be6568eab2818934692483a186e5181d
2023-04-11 11:43:21 +01:00
fanquake
b24553c04c Merge bitcoin/bitcoin#27436: test: LLVM/Clang 16 for MSAN jobs
676671527f test: LLVM/Clang 16 for MSAN jobs (fanquake)

Pull request description:

  Similar to other CI infra changes we've made recently. Move to LLVM/Clang 16 for the MSAN jobs (which is currently using LLVM 12).

  See also: https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNotes.html#sanitizers:
  > `-fsanitize-memory-param-retval` is turned on by default. With `-fsanitize=memory`, passing uninitialized variables to functions and returning uninitialized variables from functions is more aggressively reported. `-fno-sanitize-memory-param-retval` restores the previous behavior.

ACKs for top commit:
  dergoegge:
    utACK 676671527f

Tree-SHA512: a105bd1bf7f4e3ede50bb119fd8ab7f308919dc46e093eb3e94351484d65a13220e2449c40d80b8103b9ac0f4b1c8ca29576ab83e2083c26b9d8060c5802b64d
2023-04-11 11:10:42 +01:00
Andrew Chow
a3eea2a27d Merge bitcoin/bitcoin#27441: doc: correct sqlite & qrencode versions used in depenendencies.md
a12d9cfa46 doc: correct sqlite & qrencode versions used in depenendencies.md (fanquake)

Pull request description:

  Followup to https://github.com/bitcoin/bitcoin/pull/27312 & https://github.com/bitcoin/bitcoin/pull/25378.

ACKs for top commit:
  achow101:
    ACK a12d9cfa46
  hebasto:
    ACK a12d9cfa46, I have reviewed the code and it looks OK, I agree it can be merged.
  jarolrod:
    ACK a12d9cfa46

Tree-SHA512: 29e1fe4c31089fce6acbadb14aa7619fdd55738a882b490f1a0835d7648798a68b4f0d62e213c60d92f8e021ea856a4d1759578da07413265fef2338840da506
2023-04-10 21:20:53 -04:00
Andrew Chow
9270a56662 Merge bitcoin/bitcoin#27440: contrib: followups to #27358 (verify-binaries)
ad841608d4 contrib: minor doc improvements in verify-binaries (fanquake)
e2e5683afe contrib: fixup verifybinaries example docs (fanquake)
663a89cfed contrib: move verify scripts to verify-binaries (fanquake)

Pull request description:

  Followup to #27358, fixing up the example command docs and other requests. See https://github.com/bitcoin/bitcoin/pull/27358#issuecomment-1500389847.

ACKs for top commit:
  josibake:
    ACK ad841608d4
  achow101:
    ACK ad841608d4
  theuni:
    ACK ad841608d4. Thanks for doing these.

Tree-SHA512: 14c47b5a1b231d5116a1e5ddc78cb3a32ca1d4e86f7e18a0c63d5caac95a5272b3eddcc531052e130970a694dd1bc721bfcb29092755e306c37abc0b9f6c9dfd
2023-04-10 21:19:33 -04:00
Jon Atack
7ccdd741fe test: fix importmulti/importdescriptors assertion
as these RPCs have a "warnings" field, not a "warning" one.
2023-04-10 10:42:25 -07:00
Jon Atack
19d888ce40 rpc: move WALLET_FLAG_CAVEATS to the compilation unit of its caller
and add the walletutil.h include header for WALLET_FLAG_AVOID_REUSE that was
already missing before this change.

WALLET_FLAG_CAVEATS is only used in one RPC, so no need to encumber wallet.h and
wallet.cpp with it, along with all of the files that include wallet.h during
their compilation. Also apply clang-format per:

git diff -U0 HEAD~1.. | ./contrib/devtools/clang-format-diff.py -p1 -i -v
2023-04-10 10:41:56 -07:00
Jon Atack
01df011ca2 doc: release note for wallet RPCs "warning" field deprecation 2023-04-10 10:41:56 -07:00
Jon Atack
9ea8b3739a test: createwallet "warning" field deprecation test 2023-04-10 10:41:56 -07:00
Jon Atack
645d7f75ac rpc: deprecate "warning" field in {create,load,unload,restore}wallet
This string field has been replaced in these four RPCs by a "warnings" field
returning a JSON array of strings.
2023-04-10 10:41:56 -07:00
Jon Atack
2f4a926e95 test: add test coverage for "warnings" field in createwallet
and clarify the "warning" field behavior.
2023-04-10 10:41:56 -07:00
Jon Atack
4a1e479ca6 rpc: add "warnings" field to RPCs {create,load,unload,restore}wallet
This new "warnings" field is a JSON array of strings intended to replace the
"warning" string field in these four RPCs, to better handle returning multiple
warning messages and for consistency with other wallet RPCs.
2023-04-10 10:41:35 -07:00
Jon Atack
079d8cdda8 rpc: extract wallet "warnings" fields to a util helper 2023-04-10 10:41:35 -07:00
Jon Atack
f73782a903 doc: fix/improve warning helps in {create,load,unload,restore}wallet
- clarify that there can be multiple warning messages
- specify the correct wallet action
- describe the use of newlines as delimiters
2023-04-10 10:41:06 -07:00
Andrew Chow
d52fa1b0a5 tests: Make sure that bumpfee feerate checks work when replacing outputs
When replacing the outputs of a transaction, we can end up with
fees that are drastically different from the original. This tests that
the feerate checks we perform will properly detect when the bumping tx
will have an insufficient feerate.
2023-04-10 10:02:41 -04:00
Andrew Chow
be177c15a4 bumpfee: Check the correct feerate when replacing outputs
When doing the feerate check for bumped transactions that replace the
outputs, we need to consider that the size of the new outputs may be
different from the old outputs and calculate the minimum feerate accordingly.
2023-04-10 09:56:25 -04:00
fanquake
676671527f test: LLVM/Clang 16 for MSAN jobs
Sync up with other CI infra.
2023-04-10 10:57:05 +01:00
fanquake
d544d03ba6 Merge bitcoin/bitcoin#26741: doc: FreeBSD DataDirectoryGroupReadable Setting
499c464394 doc: update DataDirectoryGroupReadable 1 in tor.md (Jesse Barton)

Pull request description:

  Updating tor.md doc to include mention of FreeBSD requiring the DataDirectoryGroupReadable be set to 1.
  Default per the FreeBSD man page is 0.

         DataDirectoryGroupReadable 0|1
     If this option is set to 0, don't allow the filesystem groupto
     readthe DataDirectory. If the option is setto 1, make the
     DataDirectory readable by the default GID. (Default:0)

ACKs for top commit:
  vasild:
    ACK 499c464394

Tree-SHA512: 8750b49cd04e900435c7991d1a24641fd1171227c1f14ed59afb157f24c1ca60380d30aecfb174ca46fd5b4b99dcdb3a1cfd019aafc343362e8103abf7c17e6a
2023-04-09 22:11:43 +02:00
fanquake
ad841608d4 contrib: minor doc improvements in verify-binaries 2023-04-09 13:08:46 +02:00
fanquake
e2e5683afe contrib: fixup verifybinaries example docs
Followup to #27358, fixing up the example command docs.
2023-04-09 13:08:38 +02:00
fanquake
a12d9cfa46 doc: correct sqlite & qrencode versions used in depenendencies.md 2023-04-09 12:55:47 +02:00
fanquake
663a89cfed contrib: move verify scripts to verify-binaries 2023-04-09 12:43:59 +02:00
Jesse Barton
499c464394 doc: update DataDirectoryGroupReadable 1 in tor.md
Move DataDirectoryGroupReadable 1 up a few lines to more clearly
communicate that it is required for the filesystem group to read the
DataDirectory.

Per the Tor documentation
https://2019.www.torproject.org/docs/tor-manual.html.en#DataDirectoryGroupReadable
"If this option is set to 0, don’t allow the filesystem group to read
the DataDirectory. If the option is set to 1, make the DataDirectory
readable by the default GID. (Default: 0)"
2023-04-07 12:31:58 -05:00
fanquake
db720b5a70 Merge bitcoin/bitcoin#27358: contrib: allow multi-sig binary verification v2
754fb6bb81 verifybinaries: fix argument type error pointed out by mypy (Cory Fields)
8a65e5145c verifybinaries: catch the correct exception (Cory Fields)
4b23b488d2 verifybinaries: fix OS download filter (Cory Fields)
8cdadd1729 verifybinaries: use recommended keyserver by default (Cory Fields)
4e0396835d verifybinaries: remove unreachable code (Cory Fields)
5668c6473a verifybinaries: Don't delete shasums file (Cory Fields)
46c73b57c6 verifybinaries: README cleanups (Cory Fields)
6d11830265 verifybinaries: remove awkward bitcoin-core prefix handling (Cory Fields)
c44323a717 verifybinaries: move all current examples to the pub subcommand (Cory Fields)
7a6e7ffd06 contrib: Use machine parseable GPG output in verifybinaries (Andrew Chow)
6b2cebfa2f contrib: Add verifybinaries command for specifying files to verify (Andrew Chow)
e4d5778228 contrib: Specify to GPG the SHA256SUMS file that is detached signed (Andrew Chow)
17575c0efa contrib: Refactor verifbinaries to support subcommands (Andrew Chow)
37c9fb7a59 contrib: verifybinaries: allow multisig verification (James O'Beirne)

Pull request description:

  Following up on #23020 from jamesob with achow101's additional features on top.

  Both mentioned that they will be away for the next few weeks, so this is intended to keep review going.

  All credit to the jamesob and achow101. See #23020 for the original description and [here](https://github.com/bitcoin/bitcoin/pull/23020#issuecomment-1480603300) for the added features.

  I squashed the last commit from https://github.com/achow101/bitcoin/tree/pr23020-direct-bins-gpg-parse into the first commit here.

  Fetching and local verification seem to work as intended for me.

ACKs for top commit:
  josibake:
    ACK 754fb6bb81

Tree-SHA512: b310c57518daa690a00126308a3e7e94b978ded56d13da15d5189e9e90b71c93888d854f64179150586b0a915db8dadd43c92b716613913c198128db8867257b
2023-04-07 08:26:45 +01:00
fanquake
d6c2a46a4b Merge bitcoin/bitcoin#27429: ci: Run base install at most once
fa5af94de6 ci: Run base install at most once (MarcoFalke)

Pull request description:

  This should avoid errors when running it twice. For example, network errors on the second invocation of 'apt update'; or unguarded modifications such as APPEND_APT_SOURCES_LIST, which will append the same string repeatedly.

  The base install may be run twice in Cirrus CI with dockerfiles, or locally when running twice with DANGER_RUN_CI_ON_HOST specified.

ACKs for top commit:
  josibake:
    utACK fa5af94de6

Tree-SHA512: 020ba747e282076ddf10547025bb576ce626378d9f8313e09b1a5f99af1cf8237e44957aa4a0b8d64015bc730a090c83af0b3908a8eda15d3fb1ca7dae69dd7c
2023-04-07 08:14:08 +01:00
Cory Fields
754fb6bb81 verifybinaries: fix argument type error pointed out by mypy 2023-04-06 19:57:25 +00:00
Cory Fields
8a65e5145c verifybinaries: catch the correct exception 2023-04-06 19:57:25 +00:00
Cory Fields
4b23b488d2 verifybinaries: fix OS download filter
Co-authored-by: Reproducibility Matters <seb.kung@gmail.com>
2023-04-06 19:32:18 +00:00
Cory Fields
8cdadd1729 verifybinaries: use recommended keyserver by default 2023-04-06 19:32:18 +00:00
Cory Fields
4e0396835d verifybinaries: remove unreachable code 2023-04-06 19:32:18 +00:00
Cory Fields
5668c6473a verifybinaries: Don't delete shasums file
It may be useful for local validation.
2023-04-06 19:32:18 +00:00
Cory Fields
46c73b57c6 verifybinaries: README cleanups
- Use correct name for verify.py
- Add usage examples for verifybinaries bin
- Document proper use of new cleanup option
- Fixup broken example
2023-04-06 19:32:18 +00:00
glozow
6b605b91c1 [fuzz] Add MiniMiner target + diff fuzz against BlockAssembler
Co-authored-by: dergoegge <n.goeggi@gmail.com>
Co-authored-by: mzumsande <mzumsande@gmail.com>
Co-authored-by: Murch <murch@murch.one>
2023-04-06 15:15:39 -04:00
glozow
3f3f2d59ea [unit test] GatherClusters and MiniMiner unit tests
Co-authored-by: Murch <murch@murch.one>
Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-04-06 15:15:34 -04:00
Cory Fields
6d11830265 verifybinaries: remove awkward bitcoin-core prefix handling 2023-04-06 19:13:35 +00:00
Cory Fields
c44323a717 verifybinaries: move all current examples to the pub subcommand 2023-04-06 19:13:35 +00:00
stratospher
b5585ba5f9 p2p: skip netgroup diversity of new connections for tor/i2p/cjdns networks
Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2023-04-07 00:13:15 +05:30
MarcoFalke
fa5af94de6 ci: Run base install at most once
This should avoid errors when running it twice. For example, network
errors on the second invocation of 'apt update'; or unguarded
modifications such as APPEND_APT_SOURCES_LIST, which will append the
same string repeatedly.

The base install may be run twice in Cirrus CI with dockerfiles, or
locally when running twice with DANGER_RUN_CI_ON_HOST specified.
2023-04-06 11:52:30 +02:00
fanquake
5a8bd45056 Merge bitcoin/bitcoin#27423: ci: fix git dubious permissions error
ed4a8339b8 ci: fix git dubious permissions error (josibake)

Pull request description:

  fixes https://github.com/bitcoin/bitcoin/pull/27376#issuecomment-1496449588

  this appears to be caused by a more recent version of git being sensitive to mismatched permissions on directories. we didn't notice this before because we were using two separate user accounts to fix up dir permissions in the container , but the second account was removed in #27376

  there might be a more elegant way to do this, but this does the trick and seems to be the way others are fixing this issue around the internets.

ACKs for top commit:
  RandyMcMillan:
    ACK ed4a833
  hebasto:
    re-ACK ed4a8339b8

Tree-SHA512: dad467deca101a24f3ed34b3e26a1db5099a5bd5c3e9c9a22771c59848f7d7e7843c7386348e6fdf86d5a556e4706e5e20005d7a6637193e1c8aef7a5ff7fb19
2023-04-06 10:04:01 +01:00
fanquake
06fb95b51b Merge bitcoin/bitcoin#27335: Fixes compile errors in MSVC build #27332
6a9a4d13b2 Fixes compile errors in MSVC build #27332 (Ethan Heilman)

Pull request description:

  This PR is designed to address the issue https://github.com/bitcoin/bitcoin/issues/27332. The MSVC build is failing because of two bugs in how the build is configured.

  The issue
  ====

  When running `msbuild build_msvc\bitcoin.sln -property:Configuration=Release -maxCpuCount -verbosity:minima`l the build fails with following two errors.

  * `C:\Users\e0\Documents\GitHub\bitcoin\src\httpserver.cpp(637,9): error C2664: 'void evhttp_connection_get_peer(evhttp_connection *,const char **,uint16_t *)': cannot convert argument 2 from 'char **' to 'const char **' [C:\Users\e0\Documents\GitHub\bitcoin\build_msvc\libbitcoin_node\libbitcoin_node .vcxproj]`

  This error is occurs because bitcoin is using the wrong function signature for `evhttp_connection_get_peer` in libevent. In automake builds, configure.ac inspects the version of libevent it is building against and then defines `HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR` to flag the source code to use the correct signature. In MSVC build there does not appear to be a mechanism to do this. So it uses the wrong signature and fails. See the PR https://github.com/bitcoin/bitcoin/pull/23607 for when this logic was added to automake builds.

  * `event.lib(evutil_rand.c.obj) : error LNK2019: unresolved external symbol BCryptGenRandom referenced in function arc4_seed [C:\Users\e0\Documents\GitHub\bitcoin\build_msvc\bitcoin-cli\bitcoin-cli.vcxproj]
  C:\Users\e0\Documents\GitHub\bitcoin\build_msvc\x64\Release\bitcoin-cli.exe : fatal error LNK1120: 1 unresolved externals [C:\Users\e0\Documents\GitHub\bitcoin\build_msvc\bitcoin-cli\bitcoin-cli.vcxproj]`

  This error is caused by msbuild not being able to find the library bcrypt.lib because it has not been configured to use bcrypt.lib.

  Fixes
  ====

  While for automake builds a macro is being define to configure the current function signature for `evhttp_connection_get_peer` in libevent, this macro is not being defined for MSVC builds.

  1.  This PR addresses this issue by assuming more recent version of libevent is installed and always defining `HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR`. This logic is more brittle the automake logic, but someone following the MSVC build instructions should only get the more recent version of libevent.

  2. This PR fixes the bcrypt.lib errors this by setting this library as a dependency in build_msvc/common.init.vcxproj.in.

ACKs for top commit:
  hebasto:
    re-ACK 6a9a4d13b2

Tree-SHA512: 69d2cb6a62ecca976540a39e5f83a4e5386d920a564761cedc5127d82e5fa66ced7fcde3e5e5eb3bd6cde1cc78f843e94aa2789f02bd3e3414118030017d0387
2023-04-06 09:36:45 +01:00
Ethan Heilman
6a9a4d13b2 Fixes compile errors in MSVC build #27332
+ Pins the compatible version of libevent in vcpkg
2023-04-05 16:49:53 -04:00
fanquake
04595484d9 Merge bitcoin/bitcoin#27404: ci: use clang-16 in tidy task
a56c96507a ci: use clang-16 in tidy task (fanquake)

Pull request description:

  Follow up to https://github.com/bitcoin/bitcoin/pull/27311#issuecomment-1481020371, as IWYU now has a [clang_16 branch](https://github.com/include-what-you-use/include-what-you-use/tree/clang_16).

  This also removes some workarounds for (now fixed) clang-tidy issues, and simplifies the IWYU install steps.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK a56c96507a
  josibake:
    ACK a56c96507a
  hebasto:
    ACK a56c96507a

Tree-SHA512: 5bbec6cc196c3305302895c77986f3695fc6f4024363ee57503654d54e0ebf108719a7a1d7908817f84115dcaa13377493eb764b00bdf574f1290c73251426fa
2023-04-05 14:03:01 +01:00
furszy
25ab14712b refactor: coinselector_tests, unify wallet creation code
same lines of code repeated across the entire file over and over.
2023-04-05 09:32:39 -03:00
furszy
ba9431c505 test: coverage for bnb max weight
Basic positive and negative scenarios
2023-04-05 09:32:39 -03:00
furszy
5a2bc45ee0 wallet: clean post coin selection max weight filter
Now the coin selection algorithms contemplate the
maximum allowed weight internally and return
std::nullopt if their result exceeds it.
2023-04-05 09:32:39 -03:00
furszy
2d112584e3 coin selection: BnB, don't return selection if exceeds max allowed tx weight 2023-04-05 09:32:39 -03:00
furszy
d3a1c098e4 test: coin selection, add coverage for SRD 2023-04-05 09:32:39 -03:00
furszy
9d9689e5a6 coin selection: heap-ify SRD, don't return selection if exceeds max tx weight
Uses a min-effective-value heap, so we can remove the least valuable input/s
while the selected weight exceeds the maximum allowed weight.

Co-authored-by: Murch <murch@murch.one>
2023-04-05 09:32:39 -03:00
furszy
6107ec2229 coin selection: knapsack, select closest UTXO above target if result exceeds max tx size
The simplest scenario where this is useful is on the 'check_max_weight' unit test
already:

We create 1515 UTXOs with 0.033 BTC each, and 1 UTXO with 50 BTC. Then perform
Coin Selection.

As the selection of the 1515 small UTXOs exceeds the max allowed tx size, the
expectation here is to receive a selection result that only contain the big
UTXO (which is not happening for the reasons stated below).

As knapsack returns a result that exceeds the max allowed transaction size, we
fallback to SRD, which selects coins randomly up until the target is met. So
we end up with a selection result with lot more coins than what is needed.
2023-04-05 09:32:39 -03:00
josibake
ed4a8339b8 ci: fix git dubious permissions error 2023-04-05 14:25:13 +02:00
fanquake
75d807ac9a Merge bitcoin/bitcoin#27406: depends: add NO_HARDEN= option
436df1e826 depends: add NO_HARDEN option (fanquake)

Pull request description:

  Add an option that when passed, will disable hardening options, and pass `--disable-hardening` through to configure. Due to the way we link `libssp` for Windows builds, they now fail (after #27118), if building with depends, and configuring with `--disable-hardening` (Windows is the odd build out here). See: https://github.com/bitcoin/bitcoin/pull/27118#issuecomment-1492606272.

  This change would add a depends option such that, if someone wants to build with depends, for Windows, without hardening, they can do so. This may also be useful when building for debugging.

ACKs for top commit:
  hebasto:
    re-ACK 436df1e826

Tree-SHA512: 5a3ef5ec87b10a5ad0a284201988ce94789451735c7c7e20d337f7232955b0b9a0addab1c3b5725755f00d8ce6741aa9c8cb5e3d48d926515b7dde46acdbcaa0
2023-04-05 12:29:32 +01:00
fanquake
9ec30db888 Merge bitcoin/bitcoin#27418: test: Remove windows workaround in authproxy
fa584b4d01 test: Remove windows workaround in authproxy (MarcoFalke)

Pull request description:

  I wonder if the windows issues have also been fixed by bumping the server timeout in commit 88134fcee9.

  I guess the only way to find out and try.

  Note that even with the workaround, the issue would still happen occasionally: https://github.com/bitcoin/bitcoin/issues/18623

ACKs for top commit:
  fanquake:
    ACK fa584b4d01

Tree-SHA512: 1c633d7d8b54fa47bb851457d48b7d80bf6d6ea353db80610231e0bd385061b795f97ea2b64022fa24e79b4d7dcb5e947e0d637da228f10998e950d84b920878
2023-04-05 12:12:58 +01:00
fanquake
27ad26de2f Merge bitcoin/bitcoin#27317: log: Check that the timestamp string is non-empty to avoid undefined behavior
73f4eb511c Check that the Timestamp String is valid (John Moffett)

Pull request description:

  Follow-up to https://github.com/bitcoin/bitcoin/pull/27233

  The current `FormatISO8601DateTime` function will return an empty string if it encounters an error when converting the `int64_t` seconds-since-epoch to a formatted date time. In the unlikely case that happens, here `strStamped.pop_back()` would be undefined behavior.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 73f4eb511c
  stickies-v:
    ACK 73f4eb511c

Tree-SHA512: 089ed639c193deb98870a8385039b31b4baed821ea907937bfc6f65a5b0981bbf8284b2afec81b2d0a922e2340716b48cf55349640eb6b8c311ef7af25abc361
2023-04-05 11:50:27 +01:00
fanquake
a56c96507a ci: use clang-16 in tidy task 2023-04-05 11:43:42 +01:00
fanquake
8c3cc4cad3 Merge bitcoin/bitcoin#27420: build: remove ancient unused define
9fbc5fcd28 build: remove ancient unused define (Cory Fields)

Pull request description:

  The generic define was removed in [upstream miniupnpc in 2014](f6774e3316).

  Noticed while reviewing hebasto's new CMake buildsystem: https://github.com/hebasto/bitcoin/pull/12#discussion_r1156267350.

ACKs for top commit:
  RandyMcMillan:
    utACK 9fbc5fc
  hebasto:
    ACK 9fbc5fcd28
  TheCharlatan:
    ACK 9fbc5fcd28

Tree-SHA512: 2075a2e63441e32d939c23f30f811c73dd1ab1ddcc9e6bf079cc8060e77d9a2a6d707add7b72e9251939a53b53e28a134e9bb330ccf3556244497728f1b091ae
2023-04-05 11:37:01 +01:00
fanquake
23a899b27c Merge bitcoin/bitcoin#26832: compat: move (win) S_* defines into bdb
54e4061189 refactor: don't avoid sys/types.h on when building for Windows (fanquake)

Pull request description:

  This is the only place these defines are used. They may also be available when building for Windows. `sys/stat.h` is available, and we already use it unguarded in other code. So move the defines into bdb, after the stat.h include, and remove compat from bdb.cpp.

ACKs for top commit:
  TheCharlatan:
    re-ACK 54e4061189
  hebasto:
    ACK 54e4061189, I have reviewed the code and it looks OK.

Tree-SHA512: b75bb120654b4dec9ccc83aa407ee1a50969fec92f196a3722ec51282b91ac50e455af04f07211f3e93270ab83660f1efdeef43928b44b1e4296f6b06ea807c8
2023-04-05 11:36:19 +01:00
ismaelsadeeq
7e3d4f8e86 test: add coverage to ensure the first arg of scantxoutset is needed
Include a test that checks whether the first argument of
scantxoutset RPC call is required. The rpc call should fail if
the "start" argument is not provided.
2023-04-05 11:03:06 +01:00
brunoerg
0076bed45e logging: log ASN when using -asmap
When using `-asmap`, it will log the ASN from the peer on some logs (e.g. when a new outbound peer has been connected).
2023-04-04 16:53:25 -03:00
Martin Leitner-Ankerl
5197660e94 tracepoints: Disables -Wgnu-zero-variadic-macro-arguments to compile without warnings
Fixes #26916 by disabling the warning `-Wgnu-zero-variadic-macro-arguments` when clang is used as the compiler.
2023-04-04 20:21:38 +02:00
Cory Fields
9fbc5fcd28 build: remove ancient unused define
The generic define was removed in upstream miniupnpc in 2014:
f6774e3316

Noticed while reviewing hebasto's new CMake buildsystem:
https://github.com/hebasto/bitcoin/pull/12#discussion_r1156267350
2023-04-04 17:13:40 +00:00
Sjors Provoost
bd13dc2f46 Switch hardened derivation marker to h in descriptors
This makes it easier to handle descriptor strings manually. E.g. an RPC call that takes an array of descriptors can now use '["desc": ".../0h/..."]'.

Both markers can still be parsed. The default for new descriptors is changed to h. In normalized form h is also used. For private keys the chosen marker is preserved in a round trip.

The hdkeypath field in getaddressinfo is also impacted by this change.
2023-04-04 18:33:08 +02:00
MarcoFalke
fa584b4d01 test: Remove windows workaround in authproxy 2023-04-04 13:23:53 +02:00
MarcoFalke
fa83fb3161 wallet: Use steady clock to calculate number of derive iterations 2023-04-04 12:34:29 +02:00
MarcoFalke
fa2c099cec wallet: Use steady clock to measure scanning duration 2023-04-04 12:34:06 +02:00
MarcoFalke
fa97621804 qt: Use steady clock to throttle GUI notifications 2023-04-04 12:34:05 +02:00
MarcoFalke
fa1d8044ab test: Use steady clock in index tests 2023-04-04 12:33:59 +02:00
MarcoFalke
fa454dcb20 net: Use steady clock in InterruptibleRecv 2023-04-04 12:33:49 +02:00
fanquake
49b87bfe7e Merge bitcoin/bitcoin#27389: test: refactor: replace unnecessary BytesIO uses
f842ed9a40 test: refactor: replace unnecessary `BytesIO` uses (Sebastian Falbesoner)

Pull request description:

  Rather than needing to create intermediate stream variables, we can use helper functions like `tx_from_hex` instead or access the result directly, leading both to increased readability and less code.

ACKs for top commit:
  stickies-v:
    ACK f842ed9a40
  brunoerg:
    crACK f842ed9a40
  aureleoules:
    ACK f842ed9a40 - It seems that these are the only instances that can be changed and it simplifies test code.

Tree-SHA512: 7f4fd7a26720d1988bf27f66c817ff6cd7060350a3be62d28e7848c768fd43578719ca475193d4057ccf4f2458af18564fd513fe3a1d458a11c799927c12bd65
2023-04-04 10:36:45 +01:00
fanquake
436df1e826 depends: add NO_HARDEN option
Add an option that when passed, will disable hardening options, and
pass `--disable-hardening` through to configure. Due to the way
we link libssp for Windows builds, they now fail (after #27118),
if building with depends, and configuring with --disable-hardening.
See:
https://github.com/bitcoin/bitcoin/pull/27118#issuecomment-1492606272.

This change would add a depends opiton such that, if someone wants to
build with, for windows, without hardening, they can do so. This may
also be useful when building for debugging.
2023-04-04 10:07:41 +01:00
furszy
68eed5df86 test,gui: add coverage for PSBT creation on legacy watch-only wallets 2023-04-03 17:23:43 -03:00
furszy
306aab5bb4 test,gui: decouple widgets and model into a MiniGui struct
So it can be reused across tests.
2023-04-03 17:23:43 -03:00
furszy
2f76ac0383 test,gui: decouple chain and wallet initialization from test case
Prepare ground for legacy watch-only test.
2023-04-03 17:23:43 -03:00
furszy
cd98b71739 gui: 'getAvailableBalance', include watch only balance
Only for wallets with private keys disabled.

The returned amount need to include the watch-only
available balance too.

Solves #26687.
2023-04-03 17:23:43 -03:00
furszy
74eac3a82f test: add coverage for 'useAvailableBalance' functionality
The following cases were covered:

Case 1: No coin control selected coins.
  - 'useAvailableBalance' should fill the amount edit box with the total available balance.

Case 2: With coin control selected coins.
  - 'useAvailableBalance' should fill the amount edit box with the sum of the selected coins values.
2023-04-03 17:23:42 -03:00
furszy
dc1cc1c359 gui: bugfix, getAvailableBalance skips selected coins
The previous behavior for getAvailableBalance when coin control
has selected coins was to return the sum of them. Instead, we
are currently returning the wallet's available total balance minus
the selected coins total amount.

This turns into a GUI-only issue for the "use available balance"
button when the user manually select coins in the send screen.

Reason:
We missed to update the GetAvailableBalance function to include
the coin control selected coins on #25685.

Context:
Since #25685 we skip the selected coins inside `AvailableCoins`,
the reason is that there is no need to traverse the wallet's
txes map just to get coins that can directly be fetched by
their id.
2023-04-03 17:23:42 -03:00
brunoerg
9836c76ae0 net: add GetMappedAS in CConnman 2023-04-03 15:42:15 -03:00
fanquake
54e4061189 refactor: don't avoid sys/types.h on when building for Windows
We've already used it unguarded in `httpserver.cpp` for years, with no
build issues.
2023-04-03 14:44:48 +01:00
fanquake
369d4c03b7 Merge bitcoin/bitcoin#27254: refactor: Extract util/fs from util/system
00e9b97f37 refactor: Move fs.* to util/fs.* (TheCharlatan)
106b46d9d2 Add missing fs.h includes (TheCharlatan)
b202b3dd63 Add missing cstddef include in assumptions.h (TheCharlatan)
18fb36367a refactor: Extract util/fs_helpers from util/system (Ben Woosley)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". This commit was originally authored by empact and is taken from its parent PR #25152.

  #### Context

  There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527, https://github.com/bitcoin/bitcoin/pull/25862, https://github.com/bitcoin/bitcoin/pull/26177, and https://github.com/bitcoin/bitcoin/pull/27125). The `ArgsManager` is defined in `system.h`. A similar pull request extracting functionality from `system.h` has been merged in https://github.com/bitcoin/bitcoin/pull/27238.

  #### Changes

  Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving filesystem related functions out of the `system.*` files.

  There is already a pair of `fs.h` / `fs.cpp` in the top-level `src/` directory. They were not combined with the files introduced here, to keep the patch cleaner and more importantly because they are often included without the utility functions. The new files are therefore named `fs_helpers` and the existing `fs` files are moved into the util directory.

  Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed.

ACKs for top commit:
  hebasto:
    ACK 00e9b97f37

Tree-SHA512: 31422f148d14ba3c843b99b1550a6fd77c77f350905ca324f93d4f97b652246bc58fa9696c64d1201979cf88733e40be02d262739bb7d417cf22bf506fdb7666
2023-04-03 14:41:22 +01:00
fanquake
5150e28010 Merge bitcoin/bitcoin#27382: miniscript: explicit cast instead of comparing integers of different signs
9a54d88c8c miniscript: explicit cast instead of comparing integers of different signs (Antoine Poinsot)

Pull request description:

  Fixes #27381

ACKs for top commit:
  stickies-v:
    ACK 9a54d88c8c

Tree-SHA512: beba35ecf3325b3b0e87f3288af4522de455c2983f7f6dd64a34a1636e351afcd1e00d249dfe2b3cde4f65254d62b610daaebfbda4e4af76f1fef4dd168c631d
2023-04-02 13:06:10 +01:00
fanquake
a8649984d8 Merge bitcoin/bitcoin#27387: depends, doc: Document NO_USDT option
c9aace2bfd depends, doc: Document `NO_USDT` option (Hennadii Stepanov)

Pull request description:

  A follow-up for https://github.com/bitcoin/bitcoin/pull/23724.

  This also removes a stray `</dd>` from the `NO_NATPMP` docs.

ACKs for top commit:
  0xB10C:
    ACK c9aace2bfd

Tree-SHA512: 44426f5b2bf9eaa3b8a6c54020d23c97ab04f3d75bfa93469c2c5d3bd512fc214d82652e4558d9ad33e2b2e3605991cdbf7779ba47b10a50385d4c962a598b60
2023-04-02 12:57:21 +01:00
fanquake
88134fcee9 Merge bitcoin/bitcoin#27378: test: Remove python3.5 workaround
fae66fceb3 test: Remove python3.5 workaround in authproxy (MarcoFalke)

Pull request description:

  Remove workaround for a bug that is long fixed in a EOL python version, that isn't used by us.

  If the workaround is still needed, it should at least log the exception before silently discarding it, so that debugging is possible/easier.

ACKs for top commit:
  fanquake:
    ACK fae66fceb3

Tree-SHA512: 9da28e495d530b9f9c5c75eff4982ef23b3775309e1f8d509722a9e7fd8b3535942c9a9cbd2d5e43e6487d46fdec4a63114aaa104e258c261cb98cb58560872a
2023-04-02 12:54:37 +01:00
fanquake
8e9e2b4cb3 Merge bitcoin/bitcoin#27379: net processing: #26140 follow-ups
3fa4c54ac5 [net processing] Pass TxRelay to FindTxForGetData instead of Peer (dergoegge)
c85ee76a36 [net processin] Don't take cs_main in FindTxForGetData (dergoegge)

Pull request description:

  Addresses left over feedback from #26140.

  * https://github.com/bitcoin/bitcoin/pull/26140#discussion_r1153498543
  * https://github.com/bitcoin/bitcoin/pull/26140#discussion_r1153499627

  `mapRelay` is only accessed from the message processing thread and does not need to be kept in sync with anything validation specific, it is therfore perfectly fine to have it guarded by `g_msgproc_mutex`.

ACKs for top commit:
  jnewbery:
    utACK 3fa4c54ac5
  hebasto:
    ACK 3fa4c54ac5, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 3ef84bfe4abfa8d991a7e65d9184221294d80e0df0bbb47f0270ab6ca1593266c98abf83c610f9f86b4d16c7a4b62bcf83f8856c68d3c2e10894bff6ed3e88cd
2023-04-02 12:42:06 +01:00
Hennadii Stepanov
bc49a477de Merge bitcoin-core/gui#724: Update translation source file for v25.0 string freeze
e414edd8fc qt: Update translation source file (Hennadii Stepanov)
b780095091 qt: Adjust plural forms for translations (Hennadii Stepanov)
6ae8a24009 GUI: Send: Make feerates translatable (Luke Dashjr)
bd42f5e1cd Bugfix: GUI: Send/PSBT: Correct virtual size unit and make translatable (Luke Dashjr)
1b0407f5f1 Bugfix: GUI: transactiondesc: Translate outlier "own address" and "watch-only" (Luke Dashjr)
170f3126f2 GUI: Use translated external signer errors for messagebox text (Luke Dashjr)
96989599d6 GUI: Make messages for copying unsigned PSBTs translatable (Luke Dashjr)
08b8b287d3 Bugfix: GUI: Debug info: Use correct "kB" case for small mempool sizes, and make translation-friendly (Luke Dashjr)
dacc322be1 GUI: PSBTOperationsDialog: Support translating window title (Luke Dashjr)
5a4fe55270 GUI: Intro: Support translating caption of data directory chooser (Luke Dashjr)
3868ba3a27 GUI: Support translating peer network names (Luke Dashjr)
f1f9811198 GUI: Support translating address type dropdown entries (Luke Dashjr)

Pull request description:

  This PR updates the `src/qt/locale/bitcoin_en.xlf` translation source file according to [Release schedule for 25.0](https://github.com/bitcoin/bitcoin/issues/26549).

  Some translation-related fixes have been picked from https://github.com/bitcoin-core/gui/pull/599 and https://github.com/bitcoin-core/gui/pull/716.

  Note for reviewers: it is expected to get a zero diff after running `make -C src translate` locally.

ACKs for top commit:
  jarolrod:
    ACK e414edd8fc

Tree-SHA512: 5b0c70db1e2f5871067e84f43ebea4ee4f0027fc5f2be49bbcb1d04e162ae76607b2b038e9d0622bcb5b3658d0ede8c10c4421ddaa3343f0e0be54315ca7a4f5
2023-04-01 22:05:37 +01:00
Sebastian Falbesoner
f842ed9a40 test: refactor: replace unnecessary BytesIO uses
Rather than needing to create intermediate stream variables, we can use
helper functions like `tx_from_hex` instead or access the result
directly, leading both to increased readability and less code.
2023-04-01 14:15:17 +02:00
Hennadii Stepanov
c9aace2bfd depends, doc: Document NO_USDT option 2023-04-01 10:26:10 +01:00
fanquake
a0d37d1d23 Merge bitcoin/bitcoin#27274: refactor: remove unused param from legacy pubkey interface
1869310f3c refactor: remove unused param from legacy pubkey (Bushstar)

Pull request description:

  Unused param present in legacy pubkey manager interface. This param will not be used and should be removed to prevent unintended usage.

ACKs for top commit:
  Sjors:
    ACK 1869310f3c
  furszy:
    ACK 1869310f3c

Tree-SHA512: 0fb41fc8f481f859262f2e8e9a93c990c1b4637e74fd9191ccc0b3c523d0e7d94217a3074bb357276e1941a10d29326f850f9b27eccc1eca57cf6b549353400c
2023-03-31 17:03:32 +01:00
Antoine Poinsot
9a54d88c8c miniscript: explicit cast instead of comparing integers of different signs 2023-03-31 17:27:19 +02:00
MarcoFalke
fae66fceb3 test: Remove python3.5 workaround in authproxy
Also, move the burden of checking for a timeout to the client and
disable the timeout on the server. This should avoid intermittent issues
in slow tests (for example mining_getblocktemplate_longpoll.py, or
feature_dbcrash.py), or possibly when the server is running slow (for
example in valgrind).  There shouldn't be any downside in tests caused
by a high rpcservertimeout.
2023-03-31 14:26:08 +02:00
fanquake
5c2bb2b54c Merge bitcoin/bitcoin#27376: ci: Remove second user account
fafe3a8e38 ci: Remove second user account (MarcoFalke)

Pull request description:

  The rationale for the second (nonroot) account no longer applies. See also https://github.com/bitcoin/bitcoin/pull/27333#discussion_r1148898438

ACKs for top commit:
  josibake:
    utACK fafe3a8e38

Tree-SHA512: 4464e50bc6499e012ff33c591f20c5989b99e223c476cefea669aedb79a0e731734db9469d4c7d800642191b43b57ea32cda187093395f100dec719fea17724d
2023-03-31 13:14:43 +01:00
Hennadii Stepanov
e414edd8fc qt: Update translation source file
The diff is produced by running `make -C src translate`.
2023-03-31 12:23:03 +01:00
Hennadii Stepanov
b780095091 qt: Adjust plural forms for translations 2023-03-31 12:22:50 +01:00
Luke Dashjr
6ae8a24009 GUI: Send: Make feerates translatable 2023-03-31 12:22:21 +01:00
Luke Dashjr
bd42f5e1cd Bugfix: GUI: Send/PSBT: Correct virtual size unit and make translatable 2023-03-31 12:22:16 +01:00
Luke Dashjr
1b0407f5f1 Bugfix: GUI: transactiondesc: Translate outlier "own address" and "watch-only" 2023-03-31 12:22:11 +01:00
Luke Dashjr
170f3126f2 GUI: Use translated external signer errors for messagebox text 2023-03-31 12:22:05 +01:00
Luke Dashjr
96989599d6 GUI: Make messages for copying unsigned PSBTs translatable 2023-03-31 12:22:00 +01:00
Luke Dashjr
08b8b287d3 Bugfix: GUI: Debug info: Use correct "kB" case for small mempool sizes, and make translation-friendly 2023-03-31 12:21:51 +01:00
Luke Dashjr
dacc322be1 GUI: PSBTOperationsDialog: Support translating window title 2023-03-31 12:21:42 +01:00
Luke Dashjr
5a4fe55270 GUI: Intro: Support translating caption of data directory chooser 2023-03-31 12:21:35 +01:00
Luke Dashjr
3868ba3a27 GUI: Support translating peer network names 2023-03-31 12:21:23 +01:00
Luke Dashjr
f1f9811198 GUI: Support translating address type dropdown entries 2023-03-31 12:21:11 +01:00
dergoegge
3fa4c54ac5 [net processing] Pass TxRelay to FindTxForGetData instead of Peer 2023-03-31 13:19:20 +02:00
dergoegge
c85ee76a36 [net processin] Don't take cs_main in FindTxForGetData
Taking cs_main is no longer necessary since we moved
`m_recently_announced_invs` to `Peer` and `mapRelay` is actually only
accessed from the message processing thread.
2023-03-31 13:18:39 +02:00
fanquake
47184cfa2f Merge bitcoin/bitcoin#27362: test: remove GetRNGState lsan suppression
71b3e9b0ad sanitizers: remove GetRNGState lsan suppression (fanquake)

Pull request description:

  I am no-longer seeing this, testing with the native_asan job over `x86_64` (Ubuntu 22.04) and `aarch64` (Fedora 37).

  Can anyone recreate the false-positive?

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 71b3e9b0ad
  hebasto:
    ACK 71b3e9b0ad, tested on Ubuntu 22.04 x86_64.

Tree-SHA512: 63020327d61acd6c94c6c278c9c4d72aedc10253fa172bcf9353bcad4c28d068bee824969eb3ce92152244831df8fe92cffae536453c8073a4fda74dfdfbcefa
2023-03-31 11:16:35 +01:00
MarcoFalke
fafe3a8e38 ci: Remove second user account 2023-03-31 08:28:38 +02:00
glozow
59afcc8354 Implement Mini version of BlockAssembler to calculate mining scores
Rewrite the same algo instead of reusing BlockAssembler because we have
a few extra requirements that would make the changes invasive and
difficult to review:

- Only operate on the relevant transactions rather than full mempool
- Remove transactions that will be replaced so they can't bump their ancestors
- Don't hold mempool lock outside of the constructor
- Skip things like max block weight and IsFinalTx
- Additionally calculate fees to bump remaining ancestor packages to target feerate

Co-authored-by: Murch <murch@murch.one>
2023-03-30 17:03:04 -04:00
fanquake
6b9fedd221 Merge bitcoin/bitcoin#27361: guix: use python-minimal (3.9)
d0e571ebb1 guix: use python-minimal (3.9) (fanquake)

Pull request description:

  This further minifies the Guix release build environment.

ACKs for top commit:
  TheCharlatan:
    ACK d0e571ebb1
  hebasto:
    ACK d0e571ebb1

Tree-SHA512: 0a8aa9ae861107f106c3b9c41f78ffbaf0e71e3c61f6d96e5c82415b4570b8ac85d6578d37cd0df0ec315c1c9f35fc90b281f139271ccfd15a1495ba76166789
2023-03-30 18:56:19 +01:00
fanquake
45be6af694 Merge bitcoin/bitcoin#27333: ci: cleanup of CI_EXEC & CI_EXEC_ROOT
b5ef1419ec ci: cleanup of CI_EXEC & CI_EXEC_ROOT (refs #27321) (Vasil Stoyanov)

Pull request description:

  Basically it removes the above-mentioned env-vars as per MarcoFalke's instructions. The only deviation from the plan laid out there was that I double-quoted the last instance of $ANDROID_HOME for the sake of consistency and future-proofing and the rest of the non-quoted vars due to lint failing the build.

  Fixes #27321.

ACKs for top commit:
  josibake:
    ACK b5ef1419ec
  hernanmarino:
    untested ACK b5ef1419ec. LGTM

Tree-SHA512: a79776bf64a2fa8b38195cc84445e171fd689f156aac5a1e5d39040300567eb9f4c2ebd00fbf3fa0e55b68793f8f752d94f7d817f6097ed9dd3a8ea57651b981
2023-03-30 18:55:27 +01:00
glozow
328087d16f Merge bitcoin/bitcoin#27350: test: refactor: dedup mempool_package_limits.py subtests via decorator
e669833943 test: dedup package limit checks via decorator in mempool_package_limits.py (Sebastian Falbesoner)
72f25e238c test: refactor: use Satoshis for fees in mempool_package_limits.py (Sebastian Falbesoner)

Pull request description:

  The subtests in the functional test mempool_package_limits.py all follow the same pattern:
  1. first, check that the mempool is currently empty
  2. create and submit certain single txs to the mempool, prepare list of hex transactions
  3. check that `testmempoolaccept` on the package hex fails with a "package-mempool-limits" error on each tx result
  4. after mining a block, check that submitting the package succeeds

  Note that steps 1,3,4 are identical for each of the subtests and only step 2 varies, so this might be a nice opportunity to deduplicate code by using a newly introduced decorator which executes the necessary before and after the essential part of the subtest. This also makes it easier to add new subtests without having to copy-paste those parts once again.

  In addition, the first commit switches the fee unit from BTC to Satoshis, which allows to get rid of some imports (`COIN` and `Decimal`) and a comment for the `test_desc_size_limits` subtest is fixed (s/25KvB/21KvB/).

ACKs for top commit:
  ismaelsadeeq:
    ACK e669833943
  glozow:
    utACK e669833943

Tree-SHA512: 84a85e739de7387391c13bd46aeb015a74302ea7c6f0ca3d4e2b1b487d38df390dc118eb5b1c11d3e4206bff316a4dab60ef6b25d8feced672345d4e36ffd205
2023-03-30 18:47:17 +01:00
glozow
d4833e9c49 Merge bitcoin/bitcoin#26140: refactor: Move CNodeState members guarded by g_msgproc_mutex to Peer
3a060ae7b6 scripted-diff: Rename nUnconnectingHeaders and fPreferHeaders (dergoegge)
279c53d7e4 [net processing] Move m_recently_announced_invs from CNodeState to Peer (dergoegge)
938a8e2566 [net processing] Annotate m_recently_announced_invs as guarded by g_msgproc_mutex (dergoegge)
8a2cb1f749 [net processing] Move fPreferHeaders from CNodeState to Peer (dergoegge)
3605011e79 [net processing] Annotate fPreferHeaders as guarded by g_msgproc_mutex (dergoegge)
4b84e502f5 [net processing] Move m_headers_sync_timeout from CNodeState to Peer (dergoegge)
689b747fc3 [net processing] Annotate m_headers_sync_timeout as guarded by g_msgproc_mutex (dergoegge)
d8c0d1c345 [net processing] Move nUnconnectingHeaders from CNodeState to Peer (dergoegge)
5f80d8d1ee [net processing] Annotate nUnconnectingHeaders as guarded by g_msgproc_mutex (dergoegge)
1d87137227 [validation] Annotate ChainstateManager::m_best_header as guarded by cs_main (dergoegge)

Pull request description:

  `nUnconnectingHeaders`, `m_headers_sync_timeout`, `fPreferHeaders` and  `m_recently_announced_headers` are currently all `CNodeState` members even though they are only ever accessed from the message processing thread (therefore sufficiently guarded exclusively by `g_msgproc_mutex`). `CNodeState` exists purely to hold validation-specific state guarded by `cs_main` that is accessed by multiple threads.

  This PR adds thread-safety annotations for the above mentioned `CNodeState` members and moves them to `Peer`.

ACKs for top commit:
  glozow:
    code review ACK 3a060ae7b6, as in I am convinced these members shouldn't be guarded by cs_main and belong in Peer/TxRelay. clang checked the annotations for me.
  hebasto:
    ACK 3a060ae7b6

Tree-SHA512: 2db27c03f2c6ed36ad7dfbb4f862eeed3c3e57f845cf8abb9e7cada36f976257311892020bbcff513fbe662a881c93270e3a126946ceb0c3f94213b546bcaa81
2023-03-30 17:10:40 +01:00
fanquake
71b3e9b0ad sanitizers: remove GetRNGState lsan suppression
I am no-longer seeing this. Can anyone recreate the false-positive?
2023-03-30 14:10:35 +01:00
dergoegge
3a060ae7b6 scripted-diff: Rename nUnconnectingHeaders and fPreferHeaders
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren nUnconnectingHeaders     m_num_unconnecting_headers_msgs
ren fPreferHeaders           m_prefers_headers
ren MAX_UNCONNECTING_HEADERS MAX_NUM_UNCONNECTING_HEADERS_MSGS

-END VERIFY SCRIPT-
2023-03-30 14:56:34 +02:00
dergoegge
279c53d7e4 [net processing] Move m_recently_announced_invs from CNodeState to Peer 2023-03-30 14:56:19 +02:00
dergoegge
938a8e2566 [net processing] Annotate m_recently_announced_invs as guarded by g_msgproc_mutex 2023-03-30 14:56:19 +02:00
dergoegge
8a2cb1f749 [net processing] Move fPreferHeaders from CNodeState to Peer 2023-03-30 14:56:19 +02:00
dergoegge
3605011e79 [net processing] Annotate fPreferHeaders as guarded by g_msgproc_mutex 2023-03-30 14:56:19 +02:00
dergoegge
4b84e502f5 [net processing] Move m_headers_sync_timeout from CNodeState to Peer 2023-03-30 14:56:19 +02:00
dergoegge
689b747fc3 [net processing] Annotate m_headers_sync_timeout as guarded by g_msgproc_mutex 2023-03-30 14:56:19 +02:00
dergoegge
d8c0d1c345 [net processing] Move nUnconnectingHeaders from CNodeState to Peer 2023-03-30 14:56:17 +02:00
dergoegge
5f80d8d1ee [net processing] Annotate nUnconnectingHeaders as guarded by g_msgproc_mutex 2023-03-30 14:55:28 +02:00
dergoegge
1d87137227 [validation] Annotate ChainstateManager::m_best_header as guarded by cs_main 2023-03-30 14:55:28 +02:00
fanquake
84f4ac39fd Merge bitcoin/bitcoin#27363: ci: use LLVM/clang-16 in native_fuzz (ASAN) job
a634c288c3 ci: use LLVM/clang-16 in native_fuzz (ASAN) job (fanquake)

Pull request description:

  Similar to #27298.

ACKs for top commit:
  dergoegge:
    utACK a634c288c3

Tree-SHA512: 7a2625a3ac83710063d941dcbca42431b3b79a1380872fd2c566c0ab3041d8123d7dcddeb8a4972efd0ef6496b15bbe0b39b6d2de84df81fcdd8d68e1248fbc5
2023-03-30 13:49:06 +01:00
fanquake
5241b8bdff Merge bitcoin/bitcoin#27368: refactor: Drop no longer used CNetMsgMaker instances
ea7ec78087 refactor: Drop no longer used `CNetMsgMaker` instances (Hennadii Stepanov)

Pull request description:

  The removed lines have been unused since the abf5d16c24 commit from https://github.com/bitcoin/bitcoin/pull/25454.

ACKs for top commit:
  dergoegge:
    utACK ea7ec78087
  Sjors:
    ACK ea7ec78087
  TheCharlatan:
    ACK ea7ec78087

Tree-SHA512: 9a2a9ff3f124b68a8cd20a637e90885096996c3aa354a4d8adbec98f5761e9e826c1c064ccd90aaf6d72beac61dd9e22c8b76d089e18bba6e0ad51e59a9c7df8
2023-03-30 13:45:24 +01:00
Hennadii Stepanov
ea7ec78087 refactor: Drop no longer used CNetMsgMaker instances 2023-03-30 11:00:44 +01:00
fanquake
a634c288c3 ci: use LLVM/clang-16 in native_fuzz (ASAN) job
Similar to #27298.
2023-03-29 16:05:59 +01:00
fanquake
d0e571ebb1 guix: use python-minimal (3.9)
This further minifies the Guix release build environment.
2023-03-29 15:47:04 +01:00
dimitaracev
552684976b validation: Move warningcache to ChainstateManager 2023-03-29 13:40:42 +02:00
fanquake
8e7179aa13 Merge bitcoin/bitcoin#27345: guix: use GCC tool wrappers
4133c8104f guix: use gcc tool wrappers (fanquake)

Pull request description:

  This way, correct `--plugin` arguments are passed through.

  This is a prerequisite for LTO (see #25391). Split out, to try move things along, as this change is isolated, and should be straight-forward.

ACKs for top commit:
  TheCharlatan:
    ACK [4133c81](4133c8104f)
  hebasto:
    ACK 4133c8104f

Tree-SHA512: 4311a72a613cf027bd4490caa29604c985ed455589acd972285f13cbdf4806d2184a4dc6f20cb6f47c3fa751d58bfd0bacc257b87d4a804bf5ecf5b240e4a757
2023-03-29 12:09:23 +01:00
fanquake
c0311b1dda Merge bitcoin/bitcoin#27349: test: use address_to_scriptpubkey instead of RPC call
e47ce42f67 refactor: use address_to_scriptpubkey to retrieve addresses scriptpubkey (ismaelsadeeq)
4142d19d74 refactor: move address_to_scriptpubkey to address.py (ismaelsadeeq)

Pull request description:

  PR #27269 enables the function address_to_scriptpubkey() to decode all address types and return their corresponding scriptpubkeys. As a result, there is no longer any need to call getaddressinfo or validateaddress RPCs in order to retrieve an address scriptpubkey, as explained in the comments on this pull request (see https://github.com/bitcoin/bitcoin/pull/27269#pullrequestreview-1353681933 and https://github.com/bitcoin/bitcoin/pull/27269#issuecomment-1481016118).

  Instead of using RPC calls, this update replaces the process of obtaining an address scriptPubkey with the address_to_scriptpubkey method, resulting in improved performance for functional tests.

ACKs for top commit:
  josibake:
    re-ACK e47ce42f67
  theStack:
    ACK e47ce42f67 🌱

Tree-SHA512: 05285349a7d5ce7097b8f2582e573a5135c6deef85ea9936f68f6ce94e9ebb1d84d94f7fc7e5ed833a698e01585addd80deb52e6338f8aee985bf14db45417d2
2023-03-29 12:05:15 +01:00
Andrew Chow
7a6e7ffd06 contrib: Use machine parseable GPG output in verifybinaries
GPG has an option to provide machine parseable output. Use that instead
of trying to parse the human readable output.
2023-03-28 22:16:05 +00:00
Andrew Chow
6b2cebfa2f contrib: Add verifybinaries command for specifying files to verify
In addition to verifying the published releases with the `pub` command,
the verifybinaries script is updated to take a `bin` command where the
user specifies the local files, sums, and sigs to verify.
2023-03-28 22:16:05 +00:00
Andrew Chow
e4d5778228 contrib: Specify to GPG the SHA256SUMS file that is detached signed 2023-03-28 22:16:05 +00:00
Andrew Chow
17575c0efa contrib: Refactor verifbinaries to support subcommands
Prepares for the option to provide local binaries, sha256sums, and
signatures directly.
2023-03-28 22:16:05 +00:00
James O'Beirne
37c9fb7a59 contrib: verifybinaries: allow multisig verification
This commit adds the functionality necessary to transition from
doing binary verification on the basis of a single signature to
requiring a minimum threshold of trusted signatures.

A signature can appear as "good" from GPG output, but it may not come
from an identity the user trusts. We call these "good, untrusted"
signatures.

We report bad signatures but do not necessarily fail in their presence,
since a bad signature might coexist with enough good, trusted signatures
to fulfill our criteria.

If "--import-keys" is enabled, we will prompt the user to
optionally try to retrieve unknown keys. Marking them as trusted locally
is a WIP, but keys which are retrieved successfully and appear on the
builder-keys list will immediately count as being useful towards
fulfilling the threshold.

Logging is improved and an option to output JSON that summarizes the
whole sum signature and binary verification processes has been added.

Co-authored-by: Russ Yanofsky <russ@yanofsky.org>
Co-authored-by: willcl-ark <will8clark@gmail.com>
2023-03-28 22:16:05 +00:00
Sebastian Falbesoner
e669833943 test: dedup package limit checks via decorator in mempool_package_limits.py 2023-03-28 22:03:31 +02:00
Sebastian Falbesoner
72f25e238c test: refactor: use Satoshis for fees in mempool_package_limits.py
This avoids having to convert from BTC to Sats and needs less imports.
Also specify the tx's target size in vsize rather than in weight, which
allows us to specify the fee-rate by a simple multiplication, rather
than having another magic number for it.
2023-03-28 22:03:27 +02:00
ismaelsadeeq
e47ce42f67 refactor: use address_to_scriptpubkey to retrieve addresses scriptpubkey
This commit updates the code by replacing the RPC call used to
decode an address and retrieve its corresponding scriptpubkey
with the address_to_scriptpubkey function. address_to_scriptpubkey
function can now decode all addresses formats, which makes
it more efficient to use.
2023-03-28 16:58:16 +01:00
ismaelsadeeq
4142d19d74 refactor: move address_to_scriptpubkey to address.py
The COINBASE_MATURITY constant in blocktools.py is imported in wallet.py.
However, importing address_to_scriptpubkey to blocktools.py will
generate a circular import error. Since the method is related to
addresses, it is best to move it to address.py, which will also
fix the circular import error.

Update imports of address_to_scriptpubkey accordingly.
2023-03-28 16:54:28 +01:00
fanquake
68828288e5 Merge bitcoin/bitcoin#27318: test: wallet_create_tx.py fix race
8aab5157c5 test: wallet_create_tx.py fix race (furszy)

Pull request description:

  Fixes #27316

  Because wallets are internally synchronized through the validation interface,
  and the interface dispatches events on a worker thread, it is possible for a
  transaction created by the first wallet to not arrive to the second wallet
  before the second wallet attempts to use one of its outputs. This is because
  we do not wait for the `BroadcastTransaction` callback during the wallet's
  "submit to mempool" process. To address this in the tests, we need to
  manually sync the validation queue.

ACKs for top commit:
  josibake:
    ACK 8aab5157c5
  theStack:
    ACK 8aab5157c5

Tree-SHA512: 76364370ab292a5c3ea1ed61cd353fc626a9e9cd6ce18464c24da1b3dcb34b65006e2bc42b84bbd25af03f9449231990bf789504728972db3217b569099eb309
2023-03-28 15:41:15 +01:00
fanquake
220008604f Merge bitcoin/bitcoin#27298: ci: Use TSan new runtime (llvm-16, take 3)
faf4aca15a ci: Use TSan new runtime (llvm-16, take 3) (MarcoFalke)

Pull request description:

  The previous two attempts failed:
  * llvm-14: Failed in https://github.com/bitcoin/bitcoin/pull/24572
  * llvm-15: Failed in https://github.com/bitcoin/bitcoin/pull/26775

  However, now that the bug is known and fixed, it should be good to go. See also https://github.com/bitcoin/bitcoin/pull/26775#issuecomment-1380590669

ACKs for top commit:
  fanquake:
    ACK faf4aca15a - I still see [this](https://github.com/bitcoin/bitcoin/pull/27298#issuecomment-1480041089) failure on aarch64, but that isn't really a regression, as running this tests was already broken for me. I'll open a separate issue, and we can follow up.

Tree-SHA512: 372b53c4d42ca7f527dae4a2b5bc5ab33c816930daf7a3479d20ea7749159a0b19cfd8d76244b95b03130e4a3d12ddbbb74668b8f7e9fc272cf1084f53b7ff9b
2023-03-28 14:17:30 +01:00
fanquake
8d31d769b7 Merge bitcoin/bitcoin#27344: fuzz: Remove legacy int parse fuzz tests
faf8dc496e fuzz: Remove legacy int parse fuzz tests (MarcoFalke)

Pull request description:

  The fuzz tests checked that the result of the new function was equal to the legacy function. (Side note: The checks were incomplete, as evident by the follow-up fix in commit b5c9bb5cb9).

  Given that they haven't found any issues in years (beside missing the above issue, that they couldn't catch), it seems time to remove them.

  They may come in handy in the rare case that someone would want to modify `LocaleIndependentAtoi()` or `Parse*Int*()`, however that seems unlikely. Also, appropriate checks can be added then.

ACKs for top commit:
  fanquake:
    ACK faf8dc496e
  dergoegge:
    ACK faf8dc496e

Tree-SHA512: 4ec88b9fa8ba49a923b0604016f0f471b3c9b9e0ba6c5c3dc4e20503c6994789921e7221d9ec467a2a37a73f21a70ba51ba3370ed5ad311dee989e218290b29a
2023-03-28 12:03:39 +01:00
fanquake
d254f942a5 Merge bitcoin/bitcoin#27324: net: #27257 follow-ups
cd0c8eeb09 [net] Pass nRecvFloodSize to CNode (dergoegge)
860402ef2e [net] Remove trivial GetConnectionType() getter (dergoegge)
b5a85b365a [net] Delete CNetMessage copy constructor/assignment op (dergoegge)

Pull request description:

  Follow-up PR for #27257

  * Deletes the copy constructor/assignment operator of `CNetMessage`
  * Removes trivial getter for the connection type
  * Avoids passing `nRecvFloodSize` to CNode methods by passing it to `CNode` on creation

ACKs for top commit:
  jnewbery:
    utACK cd0c8eeb09
  theStack:
    ACK cd0c8eeb09

Tree-SHA512: 673a758668617f69fba77e61f0eaa1538da27a4849c82c98742436692baa2d7f001129af3e7a66b160e599d12109dac08137a146f10ff9b9ebdc5c2237311d41
2023-03-28 11:48:02 +01:00
fanquake
86e7410b22 Merge bitcoin/bitcoin#27348: test: fix intermittent failure in ChainStateManager tests
f8abcb3e3b test: Fix intermittent failure in ChainStateManager tests (Martin Zumsande)

Pull request description:

  Before wiping a `ChainStateManager` and creating a new one, the `validationinterface` queue should be emptied to avoid the possibility of accessing deleted memory.
  This could lead to very rare CI failures reported in #26613 and #27320 (see [here](https://github.com/bitcoin/bitcoin/issues/27320#issuecomment-1485694691) for a more detailed explanation).

  Fixes #27320

ACKs for top commit:
  jamesob:
    crACK f8abcb3e3b
  MarcoFalke:
    lgtm ACK f8abcb3e3b

Tree-SHA512: 953fcca46ffee039ae1c8c98eed5464ba7ee0007f54e18989caab3f645f5f45e64407a21f614984fd2843a3d56219f44603086b0c305a9a5cec64a8bf76c110a
2023-03-28 11:14:02 +01:00
MarcoFalke
faf4aca15a ci: Use TSan new runtime (llvm-16, take 3) 2023-03-28 10:01:47 +02:00
glozow
56484f0fdc [mempool] find connected mempool entries with GatherClusters(…)
We limit GatherClusters’s result to a maximum of 500 transactions as
clusters can be made arbitrarily large by third parties.

Co-authored-by: Murch <murch@murch.one>
2023-03-27 17:35:12 -04:00
Martin Zumsande
f8abcb3e3b test: Fix intermittent failure in ChainStateManager tests
Before wiping the ChainStateManager, the validationinterface
queue must be drained to avoid accessing deleted memory.
2023-03-27 15:47:32 -04:00
Vasil Stoyanov
b5ef1419ec ci: cleanup of CI_EXEC & CI_EXEC_ROOT (refs #27321) 2023-03-27 20:45:17 +03:00
fanquake
b759cefe93 Merge bitcoin/bitcoin#27301: depends: make fontconfig build under clang-16
9cbc1c2792 depends: make fontconfig build under clang-16 (fanquake)

Pull request description:

  Use the same workaround we've applied to qrencode, and other packages. Fontconfig not building is currently a blocker for fuzz/sanitizer infra upgrades (#27298).

  For now, this is also more straightforward than bumping the package, which introduces more complexity/usage of gperf.

  Closes: #27299.

ACKs for top commit:
  hebasto:
    ACK 9cbc1c2792

Tree-SHA512: 387ea1a73e3429f166ef5278305a56cb3c69b6e3fc8a21a66521738e313e3fe783f042759b396cd88e28c10918a4427fb836a8dfecc5a846723b6f6c6a7ade51
2023-03-27 17:49:45 +01:00
fanquake
9cbc1c2792 depends: make fontconfig build under clang-16
Use the same workaround we've applied to qrencode, and other packages.
Fontconfig not building is currently a blocker for fuzz/sanitizer infra
upgrades.

For now, this is also more straightforward than bumping the package,
which introduces more complexity/usage of gperf.
2023-03-27 16:33:15 +01:00
Hennadii Stepanov
ff26406b2b Merge bitcoin-core/gui#693: Fix segfault when shutdown during wallet open
9a1d73fdff Fix segfault when shutdown during wallet open (John Moffett)

Pull request description:

  Fixes #689

  ## Summary

  If you open a wallet and send a shutdown signal during that process, you'll get a segfault when the wallet finishes opening. That's because the `WalletController` object gets deleted manually in bitcoin.cpp during shutdown, but copies of the pointer (and pointers to child objects) are dangling in various places and are accessed in queued events after the deletion.

  ## Details

  The issue in #689 is caused by the following sequence of events:

  1. Wallet open modal dialog is shown and worker thread does the actual work.
  2. Every 200ms, the main event loop checks to see if a shutdown has been requested, but only if a modal is not being shown.
  3. Request a shutdown while the modal window is shown.
  4. The wallet open process completes, the modal window is dismissed, and various `finish` signals are sent.
  5. During handling of one of the `finish` signals, `qApp->processEvents()` is [called](e9262ea32a/src/qt/sendcoinsdialog.cpp (L603)), which causes the main event loop to detect the shutdown (now that the modal window has been dismissed). The `WalletController` and all the `WalletModel`s are [deleted](65de8eeeca/src/qt/bitcoin.cpp (L394-L401)).
  6. Control returns to the `finish` method, which eventually tries to send a [signal](e9262ea32a/src/qt/sendcoinsdialog.cpp (L167)) from a wallet model, but it's been deleted already (and the signal is sent from a now-[dangling](d8bdee0fc8/src/qt/walletview.cpp (L65)) pointer).

  The simplest fix for that is to change the `qApp->processEvents()` into a `QueuedConnection` call. (The `qApp->processEvents() was a [workaround](https://github.com/bitcoin/bitcoin/pull/593#issuecomment-3050699) to get the GUI to scroll to the last item in a list that just got added, and this is just a safer way of doing that).

  However, once that segfault is fixed, another segfault occurs due to some queued wallet events happening after the wallet controller object is deleted here:

  65de8eeeca/src/qt/bitcoin.cpp (L394-L401)

  Since `m_wallet_controller` is a copy of that pointer in `bitcoingui.cpp`, it's now dangling and `if(null)` checks won't work correctly. For instance, this line:

  65de8eeeca/src/qt/bitcoingui.cpp (L413)

  sets up a `QueuedConnection` to `setCurrentWallet`, but by the time control reaches that method (one event cycle after shutdown deleted `m_wallet_controller` in `bitcoin.cpp`), the underlying objects have been destroyed (but the pointers are still dangling).

  Ideally, we'd use a `QPointer` or `std::shared_ptr / std::weak_ptr`s for these, but the changes would be more involved.

  This is a minimal fix for the issues. Just set `m_wallet_controller` to `nullptr` in `bitcoingui.cpp`, check its value in a couple places, and avoid a use of `qApp->processEvents`.

ACKs for top commit:
  hebasto:
    ACK 9a1d73fdff, I have reviewed the code and it looks OK.
  furszy:
    ACK 9a1d73fdff

Tree-SHA512: a1b94676eb2fcb7606e68fab443b1565b4122aab93c35382b561842a049f4b43fecc459535370d67a64d6ebc4bcec0ebcda981fff633ebd41bdba6f7093ea540
2023-03-27 15:53:42 +01:00
MarcoFalke
faf8dc496e fuzz: Remove legacy int parse fuzz tests 2023-03-27 16:37:31 +02:00
fanquake
4133c8104f guix: use gcc tool wrappers
This way, correct `--plugin` argument are passed through.

This is a prerequisite for LTO (see #25391).
2023-03-27 15:25:32 +01:00
fanquake
20bd591345 Merge bitcoin/bitcoin#27314: build, qt: Fix handling of CXX=clang++ when building qt package
25e8fe70c6 build, qt: Fix handling of `CXX=clang++` when building `qt` package (Hennadii Stepanov)

Pull request description:

  On the master branch (f380bb93e8):
  ```
  $ cd depends
  $ make qt CC=clang CXX=clang++
  ...
  Project ERROR: failed to parse default search paths from compiler output
  make: *** [funcs.mk:292: /home/hebasto/git/bitcoin/depends/x86_64-pc-linux-gnu/.qt_stamp_configured] Error 3
  ```

  Fixes https://github.com/bitcoin/bitcoin/pull/27301#issuecomment-1479622034.

  Guix builds:
  ```
  daa94946aae7d4826d6f329337791a6bda0f01ac73f27003d1677dc8de031071  guix-build-25e8fe70c6e6/output/aarch64-linux-gnu/SHA256SUMS.part
  38e9dcb99e329ad02837fd08559a8408bd7ba698f65163d0834d52dc9eefceae  guix-build-25e8fe70c6e6/output/aarch64-linux-gnu/bitcoin-25e8fe70c6e6-aarch64-linux-gnu-debug.tar.gz
  36d48b93724b112e5553a8d048c79df0532da6d271598a105038c6a5dfbce630  guix-build-25e8fe70c6e6/output/aarch64-linux-gnu/bitcoin-25e8fe70c6e6-aarch64-linux-gnu.tar.gz
  6fe9697b6c73f75b1d1ce1bf89176e4afb77a0f876cfbf72b1298660c8060c17  guix-build-25e8fe70c6e6/output/arm-linux-gnueabihf/SHA256SUMS.part
  dffe564732625f7697274f5cec89c4340327fd3094c6e22dcda39edce064b249  guix-build-25e8fe70c6e6/output/arm-linux-gnueabihf/bitcoin-25e8fe70c6e6-arm-linux-gnueabihf-debug.tar.gz
  d2fe5bb056b5d12363b2b7a2bf6dca95f2fc3734b61c770fe4919ecbf6570338  guix-build-25e8fe70c6e6/output/arm-linux-gnueabihf/bitcoin-25e8fe70c6e6-arm-linux-gnueabihf.tar.gz
  ea63144558e7f9546b771f04fec5d59814d6dc5159fdc82c6da960fdf3de328f  guix-build-25e8fe70c6e6/output/arm64-apple-darwin/SHA256SUMS.part
  579d33fb5d53789849551b25de0323714fe7d282b7506d1bcdb96690dc357911  guix-build-25e8fe70c6e6/output/arm64-apple-darwin/bitcoin-25e8fe70c6e6-arm64-apple-darwin-unsigned.dmg
  439628b325a72806d2a598c9c40811f9506b3d0ca3cd56ac327ed7f25d018f22  guix-build-25e8fe70c6e6/output/arm64-apple-darwin/bitcoin-25e8fe70c6e6-arm64-apple-darwin-unsigned.tar.gz
  8434b75c17c3e3fae9f28cfa8236191134327118c83e603783d700b0ebcf07a6  guix-build-25e8fe70c6e6/output/arm64-apple-darwin/bitcoin-25e8fe70c6e6-arm64-apple-darwin.tar.gz
  61dc9b083d63fc955f4c064348d9c49937e6cc0e969c48538b60281e0824a0f4  guix-build-25e8fe70c6e6/output/dist-archive/bitcoin-25e8fe70c6e6.tar.gz
  e31df50d868ec4ab6ecbd35ca0057a053eedd474dcc9c6287e4d225e8c0e8bee  guix-build-25e8fe70c6e6/output/powerpc64-linux-gnu/SHA256SUMS.part
  086cabfda322c5f86c128d52212ccc7276a4f6273d6acaedd949634bd211f8f4  guix-build-25e8fe70c6e6/output/powerpc64-linux-gnu/bitcoin-25e8fe70c6e6-powerpc64-linux-gnu-debug.tar.gz
  a245643fdad052b7fcc798bd1b7760540dcf47c54a9cdcc1e2f6bdb1fcc3f206  guix-build-25e8fe70c6e6/output/powerpc64-linux-gnu/bitcoin-25e8fe70c6e6-powerpc64-linux-gnu.tar.gz
  cf8f81932f8132bc26f10599cd318310b6207aea9e31a1ac41393ffc7df51801  guix-build-25e8fe70c6e6/output/powerpc64le-linux-gnu/SHA256SUMS.part
  f2fee686528ef507b5de03eb7f74f4d8a6d32338dbd7c6758795c963c6e9ce41  guix-build-25e8fe70c6e6/output/powerpc64le-linux-gnu/bitcoin-25e8fe70c6e6-powerpc64le-linux-gnu-debug.tar.gz
  84a3b3c3b76eff95f8a6f49d1e4a3c7922722c1a202d982e9e4b83f9797d1e8d  guix-build-25e8fe70c6e6/output/powerpc64le-linux-gnu/bitcoin-25e8fe70c6e6-powerpc64le-linux-gnu.tar.gz
  42b8d6fa5756e200fcd99fdad07186437403ec85098d6d3d3246d7750b6f8361  guix-build-25e8fe70c6e6/output/riscv64-linux-gnu/SHA256SUMS.part
  0e588eb81dfc7dcc472b9ea18825a6dee5967dd7f5a0f90a59d0ea2ac54bac8d  guix-build-25e8fe70c6e6/output/riscv64-linux-gnu/bitcoin-25e8fe70c6e6-riscv64-linux-gnu-debug.tar.gz
  6afcc482debcec8d3a7e32ca1216bfcd9ad4f653ace88ef39113f38bfb76e447  guix-build-25e8fe70c6e6/output/riscv64-linux-gnu/bitcoin-25e8fe70c6e6-riscv64-linux-gnu.tar.gz
  f58997be8b2dde088c80534977ad26c485c687a1e55783131da259d35b477e81  guix-build-25e8fe70c6e6/output/x86_64-apple-darwin/SHA256SUMS.part
  e7e17caa9035ba92216b591c9225b72316a65b1b6f68c85f85831d4c067e8d14  guix-build-25e8fe70c6e6/output/x86_64-apple-darwin/bitcoin-25e8fe70c6e6-x86_64-apple-darwin-unsigned.dmg
  21df826825dd3b737f373513974e5e075470b6372b5ef00f0bfb69eaa806837c  guix-build-25e8fe70c6e6/output/x86_64-apple-darwin/bitcoin-25e8fe70c6e6-x86_64-apple-darwin-unsigned.tar.gz
  81544073e667222d60591316571444b19c72be5b2e41007098b17a45e63c9691  guix-build-25e8fe70c6e6/output/x86_64-apple-darwin/bitcoin-25e8fe70c6e6-x86_64-apple-darwin.tar.gz
  b1e644cc1ce3ba5bb91ca9fa22bd7abf25b2b537c9f041416555eb4327fb6a7b  guix-build-25e8fe70c6e6/output/x86_64-linux-gnu/SHA256SUMS.part
  6942721e65ae40604e83957d19b12a6d320fa5676a5668e890db13a7b2df5102  guix-build-25e8fe70c6e6/output/x86_64-linux-gnu/bitcoin-25e8fe70c6e6-x86_64-linux-gnu-debug.tar.gz
  c0596a5625d8cd2daf3c361290be6fc58642293d44944c10fde38bff203d1225  guix-build-25e8fe70c6e6/output/x86_64-linux-gnu/bitcoin-25e8fe70c6e6-x86_64-linux-gnu.tar.gz
  ebc7bab758d37dc7a16154d24b2335aa1d261345f94eeab1ed2664bc8679f2f0  guix-build-25e8fe70c6e6/output/x86_64-w64-mingw32/SHA256SUMS.part
  38f31a70a630bf9db2616eb6732635d6b9e929ebe79a5e75c0179a5dfd9c076f  guix-build-25e8fe70c6e6/output/x86_64-w64-mingw32/bitcoin-25e8fe70c6e6-win64-debug.zip
  b84591a290e8d1d246eb56f29125bedd7b0e66eeeb875dee9c377c16ba3f7029  guix-build-25e8fe70c6e6/output/x86_64-w64-mingw32/bitcoin-25e8fe70c6e6-win64-setup-unsigned.exe
  f3b50aee206c012c663d758d5a31d0efa7186805b70c8ec79962e1d7f977de6b  guix-build-25e8fe70c6e6/output/x86_64-w64-mingw32/bitcoin-25e8fe70c6e6-win64-unsigned.tar.gz
  6d7da31b00adc5005a42983c3444f44ac13a1e0028f1db7ba07d253baf40bf9e  guix-build-25e8fe70c6e6/output/x86_64-w64-mingw32/bitcoin-25e8fe70c6e6-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    tACK 25e8fe70c6

Tree-SHA512: a773d3d4454fefd37df1c05f2227e9ff66090536af3f9ff1d5373844fbcad9666ff4d199bc990ab0190f54f34059218830c1b3472a022f88d0a4971f77f82aaf
2023-03-27 15:24:33 +01:00
dergoegge
cd0c8eeb09 [net] Pass nRecvFloodSize to CNode 2023-03-27 16:00:02 +02:00
dergoegge
860402ef2e [net] Remove trivial GetConnectionType() getter 2023-03-27 16:00:02 +02:00
dergoegge
b5a85b365a [net] Delete CNetMessage copy constructor/assignment op 2023-03-27 16:00:01 +02:00
fanquake
b968424c25 Merge bitcoin/bitcoin#27326: guix: combine and document enable_werror
4becee396f guix: combine and document enable_werror (fanquake)

Pull request description:

  Combine into `hardened-glibc`.
  Document why we don't use `--disable-werror` directly.

  https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html
  > By default, the GNU C Library is built with -Werror. If you wish
  > to build without this option (for example, if building with a
  > newer version of GCC than this version of the GNU C Library was
  > tested with, so new warnings cause the build with -Werror to fail),
  > you can configure with --disable-werror.

ACKs for top commit:
  hebasto:
    ACK 4becee396f, the diff is correct.
  TheCharlatan:
    ACK 4becee396f

Tree-SHA512: 8724415f51b4d72d40c4e797faf52c93a81147fb629332b9388ffd7f113f2b16db3b7496bf3063dd978ac629fd5bde3ec7df4f1ff1ed714cb56f316a9334d119
2023-03-27 14:55:27 +01:00
fanquake
7fbc70395c Merge bitcoin/bitcoin#27296: guix: import/sync python-lief (0.12.3) package definition from upstream
24f26e08cc guix: use cmake-minimal for python-lief (fanquake)
43d8173f99 guix: import LIEF from upstream (0.12.3) (fanquake)

Pull request description:

  Update to version 0.12.3.
  Retain our PPC64 patch.
  Mention when we can drop our local definition.

  Also switch to using cmake-minimal (see #27172), which fixes atleast one build failure I've seen on aarch64, where cmake dependencies fail to build. Fix that by using the cmake without all the dependencies we don't actually need:
  ```bash
  The following derivations will be built:
    /gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv
    /gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv
    /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv

  building /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv...
  / 'check' phasenote: keeping build directory `/tmp/guix-build-python-sphinx-4.2.0.drv-5'
  builder for `/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv' failed with exit code 1
  build of /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv failed
  View build log at '/var/log/guix/drvs/3w/g6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv.gz'.
  cannot build derivation `/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv': 1 dependencies couldn't be built
  cannot build derivation `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv': 1 dependencies couldn't be built
  guix environment: error: build of `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv' failed
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 24f26e08cc

Tree-SHA512: d4260cdf5121686fd2fa36c1fc85687848eeb26cabaad2c6566feb71a18ea7fb013cfc6353c99f6f74bc89108a9505adce513c1cfa22a0a67450e6a1c451d209
2023-03-27 14:37:16 +01:00
fanquake
3963067555 Merge bitcoin/bitcoin#26642: clang-tidy: Add more performance-* checks and related fixes
03ec5b6f9c clang-tidy: Exclude `performance-*` checks rather including them (Hennadii Stepanov)
2400437230 clang-tidy: Add `performance-type-promotion-in-math-fn` check (Hennadii Stepanov)
7e975e6cf8 clang-tidy: Add `performance-inefficient-vector-operation` check (Hennadii Stepanov)
516b75f66e clang-tidy: Add `performance-faster-string-find` check (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  martinus:
    ACK 03ec5b6f9c
  TheCharlatan:
    re-ACK [03ec5b6](03ec5b6f9c)

Tree-SHA512: 2dfa52f9131da88826f32583bfd534a56a998477db9804b7333c0e7ac0b6b36141009755c7163b9f95d0ecbf5c2cb63f8a69ce4b114bb83423faed21b50cec67
2023-03-27 14:34:52 +01:00
fanquake
3e835ca958 Merge bitcoin/bitcoin#27328: depends: fix osx build with clang 16
87afcb0029 depends: fix osx build with clang 16 (Cory Fields)

Pull request description:

  Current build (using forced system clang as a test) results in:

  > error: unknown argument: '-internal-externc-isystem/opt/clang+llvm-16.0.0-x86_64-linux-gnu-ubuntu-18.04/lib/clang/16/include'

  For some reason the previous syntax worked with clang 15 and below, but clang 16 requires that the option and value are properly separated.

  See [here for an example of upstream using this syntax](https://github.com/llvm/llvm-project/blob/main/clang/test/Driver/crash-report-with-asserts.c#L9).

  There is no change in behavior for previous versions.

  I'm seeing an additional unrelated problem with linking with system clang, but I'll PR the solution to that separately as it's not as straightforward as this.

ACKs for top commit:
  TheCharlatan:
    ACK 87afcb0029
  hebasto:
    ACK 87afcb0029

Tree-SHA512: 127037c888c37c6ccd9679e96da34037cc43ccdc07915865a0a5494edb62633e83fc1bd6b1c4bb7a0322f5b59622e10090a31987f38496fb6b306488e9941594
2023-03-27 12:32:53 +01:00
fanquake
ad62bd5d6a Merge bitcoin/bitcoin#27312: depends: qrencode 4.1.1
eb1c3adf38 depends: qrencode 4.1.1 (fanquake)

Pull request description:

  Upgrade to the latest qrencode, and disable some warnings that cause compile failures with newer compilers (clang-15+).

  I haven't tested this (from a GUI perspective) at all. This is just "good enough" to keep things compiling, and uses some similar work-arounds as we have with other older packages, i.e bdb.

  Note that upstream, libqrencode is effectively unmaintained. No code changes for > 2 years. No responses to issues/PRs. Seems like the author has mostly dropped off of GitHub as well.

  This fixes part of #27299.

ACKs for top commit:
  TheCharlatan:
    Code review ACK eb1c3adf38

Tree-SHA512: 898eaac3e9915dfcdc0a011b736fff685a3b46990bd27f6038ef4d3e7cb6a276206438ea50d45908a051ce55c9b0779347d4be1d35271b67f76f409a7dc21fed
2023-03-27 12:13:28 +01:00
Hennadii Stepanov
03ec5b6f9c clang-tidy: Exclude performance-* checks rather including them 2023-03-26 20:18:13 +01:00
Hennadii Stepanov
2400437230 clang-tidy: Add performance-type-promotion-in-math-fn check
https://clang.llvm.org/extra/clang-tidy/checks/performance/type-promotion-in-math-fn.html
2023-03-26 20:18:03 +01:00
Hennadii Stepanov
7e975e6cf8 clang-tidy: Add performance-inefficient-vector-operation check
https://clang.llvm.org/extra/clang-tidy/checks/performance/inefficient-vector-operation.html
2023-03-26 20:17:55 +01:00
Hennadii Stepanov
516b75f66e clang-tidy: Add performance-faster-string-find check
https://clang.llvm.org/extra/clang-tidy/checks/performance/faster-string-find.html
2023-03-26 20:17:46 +01:00
Martin Leitner-Ankerl
bfb9291a86 util: implement prevector's move ctor & move assignment
Using swap() was rather wasteful because it had to copy the whole direct
memory data twice. Also, due to the swap() in move assignment the moved-from
object might hold on to unused memory for longer than necessary.
2023-03-26 15:49:52 +02:00
Martin Leitner-Ankerl
fffc86f49f test: CScriptCheck is used a lot in std::vector, make sure that's efficient
Adds a few static_asserts so CScriptCheck stays is_nothrow_move_assignable,
is_nothrow_move_constructible, and is_nothrow_destructible
2023-03-26 15:49:52 +02:00
Martin Leitner-Ankerl
81f67977f5 util: prevector's move ctor and move assignment is noexcept
Move operations already are `noexcept`, so add the keyword to the methods.
This makes the `PrevectorFillVectorIndirect...` benchmarks about twice
as fast on my machine, because otherwise `std::vector` has to use a copy
when the vector resizes.
2023-03-26 15:49:52 +02:00
Martin Leitner-Ankerl
d380d2877e bench: Add benchmark for prevector usage in std::vector 2023-03-26 15:49:41 +02:00
Cory Fields
87afcb0029 depends: fix osx build with clang 16
For some reason the previous syntax worked with clang 15 and below, but
clang 16 requires that the option and value are properly separated.
2023-03-24 20:41:22 +00:00
Sebastian Falbesoner
afc2dd5484 test: various converttopsbt check cleanups in rpc_psbt.py
In the functional test rpc_psbt.py, some comments around the
`converttopsbt` RPC checks are wrong or outdated and can be
removed:

> Error could be either "TX decode failed" (segwit inputs causes
> parsing to fail) or "Inputs must not have scriptSigs and
> scriptWitnesses"

Decoding a valid TX with at least one input always succeeds with the
heuristic, i.e. this comment is not right and we can assert for the
error string "Inputs must not have scriptSigs and scriptWitnesses"
on the calls below.

> We must set iswitness=True because the serialized transaction has
> inputs and is therefore a witness transaction

This is also unneeded (and confusing, w.r.t. "is therefore a witness
transaction"?), for a TX with one input there is no need to set the
`iswitness` parameter. For sake of completeness, we still keep one
variant where iswitness is explicitly set to true.

Lastly, there is a superflous `converttopsbt` call on the raw tx which
is the same as just about ~10 lines above, so it can be removed.
2023-03-24 17:54:09 +01:00
fanquake
4becee396f guix: combine and document enable_werror
Combine into hardened-glibc.
Document why we don't use --disable-werror directly.

https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html
> By default, the GNU C Library is built with -Werror. If you wish
> to build without this option (for example, if building with a
> newer version of GCC than this version of the GNU C Library was
> tested with, so new warnings cause the build with -Werror to fail),
> you can configure with --disable-werror.
2023-03-24 15:35:09 +00:00
fanquake
e352f5ab6b Merge bitcoin/bitcoin#27028: ci: Cache more stuff in the ci images: msan, iwyu, pip, sdks
faa0839837 ci: Cache more stuff in the ci images: msan, iwyu, pip, sdks (MarcoFalke)

Pull request description:

  Now that `apt` packages are cached in the ci images, it makes sense to think about caching all other packages as well.

ACKs for top commit:
  TheCharlatan:
    re-ACK faa0839837

Tree-SHA512: e2ea491570c6cdcc8522585ae7669c51ab2c0b680ff34067b58727994aa8f2e5c45ba7b76ed27a9c76d788ed155d7aade554dc164f7552fa713c00cc47b722f1
2023-03-24 12:19:18 +00:00
fanquake
873a5062db Merge bitcoin/bitcoin#27269: test: Support decoding segwit address in address_to_scriptpubkey()
d178082996 test: add bech32 decoding support to address_to_scriptpubkey() (ismaelsadeeq)
aac8793c7a test: test_bech32_decode in address.py (ismaelsadeeq)

Pull request description:

  [rpc_scantxoutset.py](e695d8536e/test/functional/rpc_scantxoutset.py (L26))  sendtodestination only sends to legacy addresses and scriptPubkeys because  [wallet.py](e695d8536e/test/functional/test_framework/wallet.py (L415)) address_to_scriptpubkey does not support conversion of segwit address.

  This update enables address_to_scriptpubkey to support the conversion of testnet segwit addresses to scriptPubkeys.

  This change will enable [rpc_scantxoutset.py](e695d8536e/test/functional/rpc_scantxoutset.py (L22)) ScantxoutsetTest to have more test coverage by adding more sendtodestination calls with bech32 and bech32m testnet addresses, then test the bech32 and bech32m  derivation subsets UTXO amount in [Test extended key derivation](e695d8536e/test/functional/rpc_scantxoutset.py (L84)).

  I will add the test coverage in a subsequent Pull request.

ACKs for top commit:
  josibake:
    ACK d178082996
  theStack:
    ACK d178082996 ✔️
  willcl-ark:
    ACK d17808299

Tree-SHA512: 312c20ce192c648faf7dd178622700c9b871d755db56c246250e25508c3c19e7b02c0ae901dda11a1794629b9a9429c877168c05e1c4c1dbf41493316e30e7e9
2023-03-24 12:17:38 +00:00
MarcoFalke
faa0839837 ci: Cache more stuff in the ci images: msan, iwyu, pip, sdks 2023-03-24 10:19:25 +01:00
Andrew Chow
630756cac0 Merge bitcoin/bitcoin#26957: bench: update logging benchmarks
8c47d599b8 doc: improve -debuglogfile help to be a bit clearer (jonatack)
20d89d6802 bench: document expected results in logging benchmarks (jonatack)
d8deba8c36 bench: add LogPrintfCategory and LogPrintLevel benchmarks (Jon Atack)
102b203349 bench: order the logging benchmark code by output (Jon Atack)
4b3fdbf6fe bench: update logging benchmark naming for clarity (Jon Atack)
4684aa8733 bench: allow logging benchmarks to be order-independent (Larry Ruane)

Pull request description:

  Update our logging benchmarks for evaluating ongoing work like #25203 and refactoring proposals like #26619 and #26697.

  - make the logging benchmarks order-independent (Larry Ruane)
  - add missing benchmarks for the `LogPrintLevel` and `LogPrintfCategory` macros that our logging is migrating to; at some later point it should be feasible to drop some of the previous logging benchmarks
  - update the logging benchmark naming to be clear which benchmark corresponds to which log macro, and update the ordering to be the same as the output
  - add clarifying documentation to the logging benchmarks
  - improve the `-debuglogfile` config option help to be clearer; can be tested by running `./src/bitcoind -help | grep -A4 '\-debuglogfile'`

  Reviewers can run the logging benchmarks with:
  ```bash
  ./src/bench/bench_bitcoin -filter='LogP*.*'
  ```

ACKs for top commit:
  LarryRuane:
    ACK 8c47d599b8
  martinus:
    code review & tested ACK 8c47d599b8, here are my benchmark results:
  achow101:
    ACK 8c47d599b8

Tree-SHA512: 705f8720c9ceaf14a1945039c7578a0c17a12215cbc44908099af4ac444561c3f95d833c5a91b325cdd4470737d8a01e2da64db2d542dd7c9a3747fbfdbf213e
2023-03-23 17:03:39 -04:00
furszy
8aab5157c5 test: wallet_create_tx.py fix race
Because wallets are internally synchronized
through the validation interface, and the
interface dispatches events on a worker thread,
it is possible for a transaction created by the
first wallet to not arrive at the second wallet
before the second wallet attempts to use one of
its outputs. This is because we do not wait for
the BroadcastTransaction callback during the wallet's
"submit to mempool" process. To address this in the
tests, we need to sync the validation queue.
2023-03-23 17:06:54 -03:00
John Moffett
73f4eb511c Check that the Timestamp String is valid
The current `FormatISO8601DateTime` function will
return an empty string if it encounters an error
when converting the `int64_t` seconds since epoch
to a formatted date time. In the unlikely case that happens,
`strStamped.pop_back()` would be undefined behavior.
2023-03-23 16:00:34 -04:00
Martin Leitner-Ankerl
9f947fc3d4 Use PoolAllocator for CCoinsMap
In my benchmarks, using this pool allocator for CCoinsMap gives about
20% faster `-reindex-chainstate` with -dbcache=5000 with practically the
same memory usage. The change in max RSS changed was 0.3%.

The `validation_flush_tests` tests need to be updated because
memory allocation is now done in large pools instead of one node at a
time, so the limits need to be updated accordingly.
2023-03-23 19:38:38 +01:00
Martin Leitner-Ankerl
5e4ac5abf5 Call ReallocateCache() on each Flush()
This frees up all associated memory with the map, not only the nodes.
This is necessary in preparation for using the PoolAllocator for
CCoinsMap, which does not actually free any memory on clear().
2023-03-23 19:38:38 +01:00
Martin Leitner-Ankerl
1afca6b663 Add PoolResource fuzzer
Fuzzes PoolResource with random allocations/deallocations, and multiple
asserts.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-03-23 19:38:38 +01:00
Martin Leitner-Ankerl
e19943f049 Calculate memory usage correctly for unordered_maps that use PoolAllocator
Extracts the resource from a PoolAllocator and uses it for
calculation of the node's memory usage.
2023-03-23 19:38:38 +01:00
Martin Leitner-Ankerl
b8401c3281 Add pool based memory resource & allocator
A memory resource similar to std::pmr::unsynchronized_pool_resource, but
optimized for node-based containers.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-03-23 19:38:38 +01:00
stickies-v
cfbc8a623b refactor: rpc: hide and rename ParseNonRFCJSONValue()
As per https://github.com/bitcoin/bitcoin/pull/26506#pullrequestreview-1211984059,
this function is no longer necessary and we can use UniValue::read() directly.

To avoid code duplication, we keep the function to throw on invalid input data
but rename it to Parse() and remove it from the header.
2023-03-23 18:18:46 +00:00
stickies-v
6c8bde6d54 test: move coverage on ParseNonRFCJSONValue() to UniValue::read()
Preparation to deprecate ParseNonRFCJSONValue() but keep test coverage
on the underlying UniValue::read() unaffected. The test coverage on
AmountFromValue is no longer included, since that is already tested
in the rpc_parse_monetary_values test case.

Fuzzing coverage on ParseNonRFCJSONValue() was duplicated between string.cpp
and parse_univalue.cpp, only the one in parse_univalue.cpp is kept.
2023-03-23 18:18:46 +00:00
fanquake
2305643646 Merge bitcoin/bitcoin#27257: refactor, net: End friendship of CNode, CConnman and ConnmanTestMsg
3566aa7d49 [net] Remove CNode friends (dergoegge)
3eac5e7cd1 [net] Add CNode helper for send byte accounting (dergoegge)
60441a3432 scripted-diff: [net] Rename CNode process queue members (dergoegge)
6693c499f7 [net] Make cs_vProcessMsg a non-recursive mutex (dergoegge)
23d9352654 [net] Make CNode msg process queue members private (dergoegge)
897e342d6e [net] Encapsulate CNode message polling (dergoegge)
cc5cdf8776 [net] Deduplicate marking received message for processing (dergoegge)
ad44aa5c64 [net] Add connection type getter to CNode (dergoegge)

Pull request description:

  We should define clear interfaces between CNode, CConnman and PeerManager. This PR makes a small step in that direction by ending the friendship of CNode, CConnman and ConnmanTestMsg. CNode's message processing queue is made private in the process and its mutex is turned into a non-recursive mutex.

ACKs for top commit:
  jnewbery:
    utACK 3566aa7d49
  vasild:
    ACK 3566aa7d49
  theStack:
    re-ACK 3566aa7d49
  brunoerg:
    re-ACK 3566aa7d49

Tree-SHA512: 26b87da5054e32401b693b2904e9c5f40e35a53937c0b6cf44b8597034ad07bacf27d87cdffc54d3e7ccfebde4231ef30a38d326f88cc18133bbb34688ead567
2023-03-23 17:31:52 +00:00
glozow
381593c906 Merge bitcoin/bitcoin#24845: wallet: return error msg for "too-long-mempool-chain"
f3221d373a test: add wallet too-long-mempool-chain error coverage (furszy)
acf0119d24 wallet: return error msg for too-long-mempool-chain failure (furszy)

Pull request description:

  Fixes #23144.

  We currently return a general "Insufficient funds" from Coin
  Selection when we actually skipped unconfirmed UTXOs that
  surpassed the mempool ancestors limit.

  This PR make the error clearer by returning:
  "Unconfirmed UTXOs are available, but spending them creates
  a chain of transactions that will be rejected by the mempool"

  Also, added an early return from Coin Selection if the sum of
  the discarded coins decreases the available balance below the
  target amount.

ACKs for top commit:
  achow101:
    ACK f3221d373a
  S3RK:
    Code review ACK f3221d373a
  Xekyo:
    ACK f3221d373a

Tree-SHA512: 13e5824b75ac302280ff894560a4ebf32a74f32fe49ef8281f2bc99c0104b92cef33d3b143c6e131f3a07eafe64533af7fc60abff585142c134b9d6e531a6a66
2023-03-23 15:53:56 +00:00
fanquake
483fb8d216 Merge bitcoin/bitcoin#27287: test: Replace threading with concurrent.futures
fa0696e786 test: Replace threading with concurrent.futures (MarcoFalke)

Pull request description:

  `threading` has no easy way to get the return value or exception once the target function stops. Not checking the return value or exception can make tests more fragile and failures harder to debug.

  Fix this by checking the return value (or exception) by wrapping the function execution into a future and calling `result()` on it.

  Can be reviewed with `--ignore-all-space`.

  (There are still some uses of `threading` around, because some tests do expect an exception to be thrown and caught in the target function)

ACKs for top commit:
  ishaanam:
    ACK fa0696e786
  stickies-v:
    ACK fa0696e786

Tree-SHA512: d9ddf6b3c530cd8c485a030a3c84d4e03d3e9f9ea8240b050afcd566a884f5cabe816ac56910cec9ea9fa299239e5abb99e672dda05a74974f61bb68dc3c1d65
2023-03-23 15:09:50 +00:00
Hennadii Stepanov
25e8fe70c6 build, qt: Fix handling of CXX=clang++ when building qt package 2023-03-23 14:38:02 +00:00
fanquake
8acfb1f8e0 Merge bitcoin/bitcoin#18933: rpc: Add submit option to generateblock
fa18504d57 rpc: Add submit option to generateblock (MarcoFalke)
fab9a08e14 refactor: Replace block_hash with block_out (MarcoFalke)

Pull request description:

  When submit is turned off, a block can be generated and returned as hex, to be used for further tests. For example, it can be submitted on a different node, on a different interface (like p2p), or just never submitted and be used for other testing purposes.

ACKs for top commit:
  instagibbs:
    ACK fa18504d57
  TheCharlatan:
    tACK fa18504d57

Tree-SHA512: 1b2ab6b71bb7e155c6482d75f5373f4e77de6446cb16bc2dfd19e7a4075b3a6ad87d7ad7a049a9eed934cb71574acfd27202f54c8bb3b03fac869f2e95db7ee5
2023-03-23 13:40:30 +00:00
fanquake
f380bb93e8 Merge bitcoin/bitcoin#27311: ci: Use clang-15 in "tidy" task
8fe27fbed8 ci: Use clang-15 in "tidy" task (Hennadii Stepanov)

Pull request description:

  Newer tools usually are better in terms of features and bug fixes.

  Requested in https://github.com/bitcoin/bitcoin/pull/26642#issuecomment-1440230390.

  Split from https://github.com/bitcoin/bitcoin/pull/26766.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 8fe27fbed8

Tree-SHA512: 62be3307d488fc4f75c40c0fa095aaa091aade2d5fe85296b56751e006c801f9d58c72c5cee8c0a0b1ba1a43804e315a3301c03e6e394bb3f3eb9b763fbb6271
2023-03-23 13:27:50 +00:00
TheCharlatan
00e9b97f37 refactor: Move fs.* to util/fs.*
The fs.* files are already part of the libbitcoin_util library. With the
introduction of the fs_helpers.* it makes sense to move fs.* into the
util/ directory as well.
2023-03-23 12:55:18 +01:00
TheCharlatan
106b46d9d2 Add missing fs.h includes
The inclusion of this header should not depend on the inclusion of other
headers that include fs.h themselves.
2023-03-23 12:54:27 +01:00
TheCharlatan
b202b3dd63 Add missing cstddef include in assumptions.h
The inclusion of this header should not depend on the inclusion of other
headers that include cstddef themselves.
2023-03-23 12:52:08 +01:00
Ben Woosley
18fb36367a refactor: Extract util/fs_helpers from util/system
This is an extraction of filesystem related functions from util/system
into their own utility file.

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving these functions out of system.h allows including them from a
separate source file without including the ArgsManager definitions from
system.h.
2023-03-23 12:52:00 +01:00
ismaelsadeeq
d178082996 test: add bech32 decoding support to address_to_scriptpubkey()
This permits functional tests to decode bech32 addresses to scriptpubkeys.
2023-03-23 12:00:54 +01:00
ismaelsadeeq
aac8793c7a test: test_bech32_decode in address.py
Adds bech32_to_bytes() which can decode a bech32 address and return the
version as an `int` and the payload in bytes.

bech32_to_bytes() is used by the test_bech32_decode unit test to test
decoding of segwit addresses.
2023-03-23 11:59:29 +01:00
fanquake
eb1c3adf38 depends: qrencode 4.1.1
Upgrade to the latest qrencode, and disable some warnings that cause
compile failures with newer compilers (clang-15+).

Fixes part of #27299.
2023-03-23 10:31:25 +00:00
fanquake
2fadb261b6 Merge bitcoin/bitcoin#27233: refactor: Replace GetTimeMicros by SystemClock
faf3f12424 refactor: Replace GetTimeMicros by SystemClock (MarcoFalke)

Pull request description:

  It is unclear from the name that `GetTimeMicros` returns the system time. Also, it is not using the type-safe `std::chrono` types.

  Fix both issues by replacing it with `SystemClock` in the only place it is used.

  This refactor should not change behavior.

ACKs for top commit:
  willcl-ark:
    tACK faf3f1242
  john-moffett:
    ACK faf3f12424 changes, but left a comment for the existing code.

Tree-SHA512: 069e6ef26467a469f128b98a4aeb334f141742befd7880cb3a7d280480e9f0684dc0686fa6a828cdcb3d11943ae5c7f8ad5d9d9dab4c668be85e5d28c78cd489
2023-03-23 10:17:23 +00:00
fanquake
4c6b7d330a Merge bitcoin/bitcoin#27297: test: Remove unused Check* default constructors
fae349076d test: Remove unused Check* default constructors (MarcoFalke)

Pull request description:

  They are no longer needed after the removal of `swap`, see https://github.com/bitcoin/bitcoin/pull/26749#discussion_r1144532693

  Also, flatten a redundant `if` check.

ACKs for top commit:
  hebasto:
    ACK fae349076d

Tree-SHA512: c0bc0c16b5df0f16fc25e18d2414a2a3c4769da1aa30d53f8d267bc2e97dd79a0296db94c1e49cd1ca89bd42275d8c462f7bf47f03f105dfe867ebea6563454b
2023-03-22 17:58:35 +00:00
Andrew Chow
fc7c21f664 Merge bitcoin/bitcoin#27271: RPC: Fix fund transaction crash when at 0-value, 0-fee
d7cc503843 Fix fund transaction case at 0-value, 0-fee (Greg Sanders)

Pull request description:

  and when no inputs are pre-selected.

  triggered via:

  walletcreatefundedpsbt '[]' '[{"data": "deadbeef"}]' 0 '{"fee_rate": "0"}'

ACKs for top commit:
  achow101:
    ACK d7cc503843
  josibake:
    ACK d7cc503843
  furszy:
    Crashes sucks code ACK d7cc5038

Tree-SHA512: 3f5e10875666aaf52c11d6a38b951aa75d0cbe684cc7f904e199f7a864923bf31d03a654687f8b746cae0eebb886a799bff2c6d200699438480d4c0ff8785f3a
2023-03-22 12:54:26 -04:00
Hennadii Stepanov
8fe27fbed8 ci: Use clang-15 in "tidy" task 2023-03-22 15:21:39 +00:00
dergoegge
3566aa7d49 [net] Remove CNode friends
Both `CConnman` and `ConnmanTestMsg` no longer access private members of
`CNode`, we can therefore remove the friend relationship.
2023-03-22 13:18:57 +01:00
dergoegge
3eac5e7cd1 [net] Add CNode helper for send byte accounting 2023-03-22 13:18:57 +01:00
dergoegge
60441a3432 scripted-diff: [net] Rename CNode process queue members
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren cs_vProcessMsg    m_msg_process_queue_mutex
ren vProcessMsg       m_msg_process_queue
ren nProcessQueueSize m_msg_process_queue_size

-END VERIFY SCRIPT-
2023-03-22 13:18:56 +01:00
dergoegge
6693c499f7 [net] Make cs_vProcessMsg a non-recursive mutex 2023-03-22 13:18:32 +01:00
dergoegge
23d9352654 [net] Make CNode msg process queue members private
Now that all access to the process queue members is handled by methods
of `CNode` we can make these members private.
2023-03-22 13:18:32 +01:00
dergoegge
897e342d6e [net] Encapsulate CNode message polling 2023-03-22 13:18:30 +01:00
MarcoFalke
fae349076d test: Remove unused Check* default constructors 2023-03-22 12:37:07 +01:00
fanquake
a70911492f Merge bitcoin/bitcoin#26749: refactor: Use move semantics instead of custom swap functions
95ad70ab65 test: Default initialize `should_freeze` to `true` (Hennadii Stepanov)
cea50521fe refactor: Drop no longer used `swap` member functions (Hennadii Stepanov)
a87fb6bee5 clang-tidy: Fix modernize-use-default-member-init in `CScriptCheck` (Hennadii Stepanov)
b4bed5c1f9 refactor: Drop no longer used `CScriptCheck()` default constructor (Hennadii Stepanov)
d8427cc28e refactor: Use move semantics in `CCheckQueue::Loop` (Hennadii Stepanov)
9a0b524139 clang-tidy, test: Fix bugprone-use-after-move in `Correct_Queue_range()` (Hennadii Stepanov)
04831fee6d refactor: Make move semantics explicit for callers (Hennadii Stepanov)
6c2d5972f3 refactor: Use move semantics in `CCheckQueue::Add` (Hennadii Stepanov)
0682003214 test, refactor: Avoid `CScriptCheck::swap` in `transaction_tests` (Hennadii Stepanov)
15209d97c6 consensus, refactor: Avoid `CScriptCheck::swap` in `CheckInputScripts` (Hennadii Stepanov)

Pull request description:

  This PR makes code more succinct and readable by using move semantics.

ACKs for top commit:
  martinus:
    re-ACK 95ad70ab65
  achow101:
    ACK 95ad70ab65
  TheCharlatan:
    re-ACK 95ad70ab65
  MarcoFalke:
    re-ACK 95ad70ab65 🚥

Tree-SHA512: adda760891b12d252dc9b823fe7c41eed660364b6fb1a69f17607d7a31eb0bbb82a80d154a7acfaa241b5de37d42a293c2b6e059f26a8e92d88d3a87c99768fb
2023-03-22 11:16:56 +00:00
fanquake
6e69fead2b Merge bitcoin/bitcoin#27280: test: Fix TypeError (expected str instance, bytes found) in wait_for_debug_log
33337eb860 test: Fix TypeError in wait_for_debug_log (MarcoFalke)

Pull request description:

ACKs for top commit:
  davidgumberg:
    tACK 33337eb860

Tree-SHA512: e641f23f0adc074d12b0ee10cab5845c16f3ac2858e42f895c69857c375fcb15c31bc1c9476bf2b6e2b49d0d2db4944687733da16d4a464152ae3323cbc6ca68
2023-03-22 10:01:01 +00:00
fanquake
24f26e08cc guix: use cmake-minimal for python-lief
This also fixes atleast one --no-substitues build failure I've seen,
where cmake dependencies wouldn't build:
```bash
The following derivations will be built:
  /gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv
  /gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv
  /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv

building /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-python-sphinx-4.2.0.drv-5'
builder for `/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv' failed with exit code 1
build of /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv failed
View build log at '/var/log/guix/drvs/3w/g6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv.gz'.
cannot build derivation `/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv': 1 dependencies couldn't be built
guix environment: error: build of `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv' failed
```
2023-03-22 09:53:13 +00:00
fanquake
43d8173f99 guix: import LIEF from upstream (0.12.3)
Updates to version 0.12.3.
Retain our PPC64 patch.
Mention when we can drop our local definition.
2023-03-22 09:52:03 +00:00
fanquake
34551cb97a Merge bitcoin/bitcoin#27289: Refactor: Remove unused FlatFilePos::SetNull
fa67b8181c Refactor: Remove unused FlatFilePos::SetNull (MarcoFalke)

Pull request description:

  This is unused outside of tests and the default constructor. With C++11, it can be replaced by C++11 member initializers in the default constructor.

  Beside removing unused code, this also makes it less fragile in light of uninitialized memory. (See also https://github.com/bitcoin/bitcoin/pull/26296#issuecomment-1477801767)

  If new code needs to set this to null, it can use `std::optional`, or in the worst case re-introduce this method.

ACKs for top commit:
  dergoegge:
    Code review ACK fa67b8181c
  TheCharlatan:
    ACK fa67b8181c
  john-moffett:
    ACK fa67b8181c

Tree-SHA512: 465c5e3eb4625405c445695d33e09a1fc5185c7dd1e766ba06034fb093880bfc65441d5334f7d9b20e2e417c2075557d86059f59d9648ca0e62a54c699c029b9
2023-03-22 09:43:43 +00:00
Andrew Chow
664500fc71 Merge bitcoin/bitcoin#27278: Log new headers
2c3a90f663 log: on new valid header (James O'Beirne)
e5ce857634 log: net: new header over cmpctblock (James O'Beirne)

Pull request description:

  Alternate to #27276.

  Devs were [suprised to realize](https://twitter.com/jamesob/status/1637237917201383425) last night that we don't have definitive logging for when a given header was first received.

  This logs to the main stream when new headers are received outside of IBD, as well as when headers come in over cmpctblocks. The rationale of not hiding these under log categories is that they may be useful to have widely available when debugging strange network activity, and the marginal volume is modest.

ACKs for top commit:
  dergoegge:
    Code review ACK 2c3a90f663
  achow101:
    ACK 2c3a90f663
  Sjors:
    tACK 2c3a90f663
  josibake:
    ACK 2c3a90f663

Tree-SHA512: 49fdcbe07799c8adc24143d7e5054a0c93fef120d2e9d5fddbd3b119550d895e2985be6ac10dd1825ea23a6fa5479c1b76d5518c136fbd983fa76c0d39dc354f
2023-03-21 13:48:12 -04:00
Hennadii Stepanov
95ad70ab65 test: Default initialize should_freeze to true
It is safe now, when move semantics is used instead of a custom swap
function.
2023-03-21 13:05:00 +00:00
Hennadii Stepanov
cea50521fe refactor: Drop no longer used swap member functions 2023-03-21 13:04:53 +00:00
Hennadii Stepanov
a87fb6bee5 clang-tidy: Fix modernize-use-default-member-init in CScriptCheck 2023-03-21 13:04:44 +00:00
Hennadii Stepanov
b4bed5c1f9 refactor: Drop no longer used CScriptCheck() default constructor 2023-03-21 13:04:35 +00:00
Hennadii Stepanov
d8427cc28e refactor: Use move semantics in CCheckQueue::Loop
Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
2023-03-21 13:04:21 +00:00
Hennadii Stepanov
9a0b524139 clang-tidy, test: Fix bugprone-use-after-move in Correct_Queue_range() 2023-03-21 13:04:12 +00:00
Hennadii Stepanov
04831fee6d refactor: Make move semantics explicit for callers 2023-03-21 13:04:01 +00:00
Hennadii Stepanov
6c2d5972f3 refactor: Use move semantics in CCheckQueue::Add
Co-authored-by: Martin Leitner-Ankerl <martin.ankerl@gmail.com>
2023-03-21 13:03:41 +00:00
Hennadii Stepanov
0682003214 test, refactor: Avoid CScriptCheck::swap in transaction_tests 2023-03-21 13:03:30 +00:00
Hennadii Stepanov
15209d97c6 consensus, refactor: Avoid CScriptCheck::swap in CheckInputScripts 2023-03-21 13:03:16 +00:00
MarcoFalke
fa67b8181c Refactor: Remove unused FlatFilePos::SetNull 2023-03-21 13:54:11 +01:00
MarcoFalke
fa0696e786 test: Replace threading with concurrent.futures 2023-03-21 09:49:39 +01:00
James O'Beirne
2c3a90f663 log: on new valid header 2023-03-20 13:05:55 -04:00
Andrew Chow
f4e42a78c7 Merge bitcoin/bitcoin#27179: guix: use osslsigncode 2.5
285edfadca guix: use osslsigncode 2.5 (fanquake)

Pull request description:

  Switches to using a newer version of [osslsigncode](https://github.com/mtrojnar/osslsigncode) in our Guix environment.

  achow101 can you test this with some sort of WIndows code-signing dry-run (no-rush).

ACKs for top commit:
  achow101:
    ACK 285edfadca

Tree-SHA512: 2ab8f65e506bd97e74e76f24e791ae20694e567a751cc57d3a27f31f0733e3530d058ef19825a35dc21d1342e3fffc52d8d643258198c669cc68b6db41bda629
2023-03-20 12:50:11 -04:00
Andrew Chow
b7edd55c22 Merge bitcoin/bitcoin#26899: p2p: set -dnsseed and -listen false if maxconnections=0
fabb95e7bf doc: add release note for 26899 (brunoerg)
c84c5f6e89 p2p: set `-dnsseed` and `-listen` false if `maxconnections=0` (brunoerg)

Pull request description:

  If `maxconnections=0`, it means our possible connections are going to be manual (e.g via `addnode`). For this reason, we can skip DNS seeds and set `listen` false.

ACKs for top commit:
  achow101:
    ACK fabb95e7bf
  vasild:
    ACK fabb95e7bf
  1440000bytes:
    reACK fabb95e7bf

Tree-SHA512: 33919a784723a32450f39ee4f6de3e27cc7c7f4c6ab4b8ce673981d461df334197deaf43e3f882039fa1ac36b2fddc6c6ab4413512d6c393d4a6865302dd05e7
2023-03-20 12:49:10 -04:00
Andrew Chow
60f142e395 Merge bitcoin/bitcoin#26531: mempool: Add mempool tracepoints
4b7aec2951 Add mempool tracepoints (virtu)

Pull request description:

  This PR adds multiple mempool tracepoints.

  | tracepoint  | description |
  | ------------- | ------------- |
  | `mempool:added`  | Is called when a transaction enters the mempool  |
  | `mempool:removed`  | ... when a transaction is removed from the mempool |
  | `mempool:replaced`  | ... when a transaction is replaced in the mempool |
  | `mempool:rejected`  | ... when a transaction is rejected from entering the mempool |

  The tracepoints are further documented in `docs/tracing.md`. Usage is demonstrated in the example script `contrib/tracing/mempool_monitor.py`. Interface tests are provided in `test/functional/interface_usdt_mempool.py`.

  The rationale for passing the removal reason as a string instead of numerically is that the benefits of not having to maintain a redundant enum-string mapping seem to outweigh the small cost of string generation. The reject reason is passed as string as well, although in this instance the string does not have to be generated but is readily available.

ACKs for top commit:
  0xB10C:
    ACK 4b7aec2951
  achow101:
    ACK 4b7aec2951

Tree-SHA512: 6deb3ba2d1a061292fb9b0f885f7a5c4d11b109b838102d8a8f4828cd68f5cd03fa3fc64adc6fdf54a08a1eaccce261b0aa90c2b8c33cd5fd3828c8f74978958
2023-03-20 12:42:24 -04:00
virtu
4b7aec2951 Add mempool tracepoints
Tracepoints for added, removed, replaced, and rejected transactions.

The removal reason is passed as string instead of a numeric value, since
the benefits of not having to maintain a redundant enum-string mapping
seem to outweigh the small cost of string generation.  The reject reason
is passed as string as well, although here the string does not have to
be generated but is readily available.

So far, tracepoint PRs typically included two demo scripts: a naive
bpftrace script to show raw tracepoint data and a bcc script for a more
refined view. However, as some of the ongoing changes to bpftrace
introduce a certain degree of unreliability (running some of the
existing bpftrace scripts was not possible with standard kernels and
bpftrace packages on latest stable Ubuntu, Debian, and NixOS), this PR
includes only a single bcc script that fuses the functionality of former
bpftrace and bcc scripts.
2023-03-20 15:57:31 +01:00
James O'Beirne
e5ce857634 log: net: new header over cmpctblock 2023-03-20 08:32:40 -04:00
Bushstar
1869310f3c refactor: remove unused param from legacy pubkey 2023-03-20 11:41:52 +00:00
MarcoFalke
33337eb860 test: Fix TypeError in wait_for_debug_log
Traceback:

print_log = " - " + "\n - ".join(log.splitlines())
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, bytes found
2023-03-20 11:54:06 +01:00
dergoegge
cc5cdf8776 [net] Deduplicate marking received message for processing 2023-03-19 14:34:37 +01:00
dergoegge
ad44aa5c64 [net] Add connection type getter to CNode 2023-03-19 14:34:36 +01:00
fanquake
40e1c4d402 Merge bitcoin/bitcoin#25666: refactor: wallet, do not translate init options names
e43a547a36 refactor: wallet, do not translate init arguments names (furszy)

Pull request description:

  Simple, and not interesting, refactor that someone has to do sooner or later. We are translating some init arguments names when those shouldn't be translated.

ACKs for top commit:
  achow101:
    ACK e43a547a36
  MarcoFalke:
    lgtm ACK e43a547a36
  ryanofsky:
    Code review ACK e43a547a36. Just rebased since last review.

Tree-SHA512: c6eca98fd66d54d5510de03ab4e63c00ba2838af4237d2bb135d01c47f8ad8ca9aa7ae1e45cf668afcfb9dd958b075a1756cc887b3beef2cb494933d4d83eab0
2023-03-19 12:24:21 +00:00
fanquake
0973018067 Merge bitcoin/bitcoin#27265: test: check that sigop limit also affects ancestor/descendant size (27171 follow-up)
6d24d1ef2b test: check that sigop limit also affects ancestor/descendant size (Sebastian Falbesoner)

Pull request description:

  This is a follow-up to #27171, adding a check that the sigop-limit vsize logic is also respected for {ancestor,descendant}size calculation (as suggested in https://github.com/bitcoin/bitcoin/pull/27171#pullrequestreview-1331143909). For simplicity, we use a one-parent-one-child cluster here and only check for the case that the sigop-limit equivalent size is larger than the serialized vsize.

ACKs for top commit:
  glozow:
    code review ACK 6d24d1ef2b, thanks for taking!

Tree-SHA512: dc65e455d06cfef1f1d6a53b959f99ec1ca3fe51c98dc1ed5826614b5619773d34aff0171c43a0ede4fd45605b2eb7a9278e027196128bb7ad8586b859f1cf70
2023-03-19 12:11:47 +00:00
fanquake
053b2d3377 Merge bitcoin/bitcoin#27264: p2p: Improve diversification of new connections
72e8ffd7f8 p2p: Account for MANUAL conns when diversifying persistent outbound conns (Gleb Naumenko)
3faae99c3d p2p: Diversify connections only w.r.t *persistent* outbound peers (Gleb Naumenko)

Pull request description:

  Revives #19860.

  In order to make sure that our persistent outbound slots belong to different netgroups, distinct net groups of our peers are added to [`setConnected`](8c4958bd4c/src/net.cpp (L1716)). We’d only open a persistent outbound connection to peers which have a different netgroup compared to those netgroups present in `setConnected`.

  **behaviour on master**

  we open persistent outbound connections to peers which have different netgroups compared to outbound full relay, block relay, addrfetch and feeler connection peers.

  **behaviour on PR**

  netgroup diversity is based on outbound full relay, block relay and manual connection peers.

  **rationale**

  - addrfetch and feeler connections are short lived connections and shouldn’t affect how we select outbound peers from addrman.
  - manual connections are like regular connections when viewed from addrman’s netgroup diversity point of view and should affect how we select outbound peers from addrman

ACKs for top commit:
  amitiuttarwar:
    code review ACK 72e8ffd7f8
  vasild:
    ACK 72e8ffd7f8
  mzumsande:
    Code Review ACK 72e8ffd7f8
  brunoerg:
    crACK 72e8ffd7f8

Tree-SHA512: 359451945a707b312ef6c2696a3a9d4256ab14dab9bd461cca4a52dae034db099012df6de3faef2f3fb38184b05996402ac280b681959483824419b6deb4db1a
2023-03-19 12:08:53 +00:00
ishaanam
6e9f8bb050 rpc, tests: in utxoupdatepsbt also look for the transaction in the txindex
Previously only the segwit utxos being spent by the psbt were looked for and
added to the psbt. Now, the full transaction corresponding to each of these
utxos (legacy and segwit) is looked for in the txindex and mempool and added
to the psbt. If txindex is disabled and the transaction is not in the mempool,
then we fall back to getting just the utxo (if segwit) from the utxo set.
2023-03-18 20:58:15 -04:00
Amiti Uttarwar
17e705428d doc: clarify new_only param for Select function
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 18:02:40 -07:00
Amiti Uttarwar
b0010c83a1 bench: test select for a new table with only one address
the addrman select function will demonstrate it's worst case performance when
it is almost empty, because it might have to linearly search several buckets.
add a bench test to cover this case

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 18:02:40 -07:00
Amiti Uttarwar
9b91aae085 bench: add coverage for addrman select with network parameter
to evaluate the worst case performance with the network parameter passed
through, fill the new table with addresses then add a singular I2P address to
retrieve

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 18:02:40 -07:00
Amiti Uttarwar
22a4d1489c test: increase coverage of addrman select (without network)
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 18:02:40 -07:00
Amiti Uttarwar
a98e542e0c test: add addrman test for special case
if an addr matching the network requirements is only on the new table and
select is invoked with new_only = false, ensure that the code selects the new
table.

in order to test this case, we use a non deterministic addrman. this means we
cannot have more than one address in any addrman table, or risk sporadic
failures when the second address happens to conflict.

if the code chose a table at random, the test would fail 50% of the time

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 18:02:40 -07:00
Amiti Uttarwar
5c8b4baff2 tests: add addrman_select_by_network test
this adds coverage for the 7 different cases of which table should be selected
when the network is specified. the different cases are the result of new_only
being true or false and whether there are network addresses on both, neither,
or one of new vs tried tables. the only case not covered is when new_only is
false and the only network addresses are on the new table.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2023-03-17 18:02:40 -07:00
Amiti Uttarwar
6b229284fd addrman: add functionality to select by network
Add an optional parameter to the addrman Select function that allows callers to
specify which network the returned address should be on. Ensure that the proper
table is selected with different cases of whether the new or tried table has
network addresses that match.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 17:59:02 -07:00
Amiti Uttarwar
26c3bf11e2 scripted-diff: rename local variables to match modern conventions
-BEGIN VERIFY SCRIPT-
sed -i 's/fChanceFactor/chance_factor/g' src/addrman.cpp
sed -i 's/nBucketPos/initial_position/g' src/addrman.cpp
sed -i 's/nBucket/bucket/g' src/addrman.cpp src/addrman_impl.h
sed -i 's/newOnly/new_only/g' src/addrman.cpp src/addrman_impl.h src/addrman.h src/test/addrman_tests.cpp
-END VERIFY SCRIPT-

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 17:59:02 -07:00
Amiti Uttarwar
48806412e2 refactor: consolidate select logic for new and tried tables
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 17:59:02 -07:00
Amiti Uttarwar
ca2a9c5f8f refactor: generalize select logic
in preparation for consolidating the logic for searching the new and tried
tables, generalize the call paths for both

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 17:59:02 -07:00
Amiti Uttarwar
052fbcd5a7 addrman: Introduce helper to generalize looking up an addrman entry
Unused until later commit.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-17 17:59:02 -07:00
fanquake
50171df26c Merge bitcoin/bitcoin#27212: test: Make the unlikely race in p2p_invalid_messages impossible
fa1eb0ecae test: Make the unlikely race in p2p_invalid_messages impossible (MarcoFalke)

Pull request description:

  After `add_p2p_connection` both sides have the verack processed.
  However the pong from conn in reply to the ping from the node has not
  been processed and recorded in totalbytesrecv.
  Flush the pong from conn by sending a ping from conn.

  This should make the unlikely race impossible.

ACKs for top commit:
  mzumsande:
    ACK fa1eb0ecae
  pinheadmz:
    ACK fa1eb0ecae

Tree-SHA512: 44166587572e8c0c758cac460fcfd5cf403b2883880128b13dc62e7f74ca5cb8f145bb68a903df177ff0e62faa360f913fd409b009d4cd1360f1f4403ade39ae
2023-03-17 14:55:48 +00:00
MarcoFalke
fa1eb0ecae test: Make the unlikely race in p2p_invalid_messages impossible 2023-03-17 09:25:32 +01:00
Andrew Chow
db03248070 Merge bitcoin/bitcoin#27199: test: fix race condition in encrypted wallet rescan tests
dbeca792a9 test: fix race condition in encrypted wallet rescan tests (ishaanam)

Pull request description:

  This fixes https://github.com/bitcoin/bitcoin/pull/26347#discussion_r1123340738

ACKs for top commit:
  MarcoFalke:
    nice re-ACK dbeca792a9  🚜
  achow101:
    ACK dbeca792a9

Tree-SHA512: 7127254ac0274b5bc8ba0242736e77464acbf1f6e3f6af098b4e47742124c336cd67dffdb385e1e8dbd3a8ae74acd073c99e82fa35c44a615fd7d22b29a0daf7
2023-03-16 17:28:39 -04:00
Greg Sanders
d7cc503843 Fix fund transaction case at 0-value, 0-fee 2023-03-16 14:58:41 -04:00
Andrew Chow
09e86d7a1a Merge bitcoin/bitcoin#27200: test: psbt: check non-witness UTXO removal for segwit v1 input
3dd2f6461b test: psbt: check non-witness UTXO removal for segwit v1 input (Sebastian Falbesoner)
dd78e3fa43 test: speedup rpc_psbt.py by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
e194e3e93d test: PSBT: eliminate magic numbers for global unsigned tx key (0) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for dropping non-witness UTXOs from PSBTs for segwit v1+ inputs (see commit 103c6fd279). The formerly [disabled](4600479058) method `test_utxo_conversion` is re-enabled and adapted to spend a Taproot (`bech32m`) instead of a wrapped SegWit (`p2sh-segwit`) output. Note that in contrast to the original test, we have to add the non-witness UTXO manually here using the test framework's PSBT module, since the constructing node knows that the output is segwit v1 and hence doesn't add the non-witness UTXO in the first place (see also [BIP371]( https://github.com/bitcoin/bips/blob/master/bip-0371.mediawiki#user-content-UTXO_Types)).

  I strongly assume that most wallets would behave the same as Bitcoin Core here and wouldn't create PSBTs with non-witness UTXOs for Taproot inputs, but it's still good to test everything works as expected if it's still done and that the non-witness UTXO is simply dropped in that case.

  The first two commits contain a small refactor (magic number elimination in PSBT module) and test speedup of ~2-3x (using whitelisting peers / immediate tx relay).

ACKs for top commit:
  achow101:
    ACK 3dd2f6461b
  instagibbs:
    ACK 3dd2f6461b

Tree-SHA512: b8d7f7ea5d7d21def024b70dfca61991cc96a4193be8857018b4d7cf3ca1465d185619fd4a77623803d9da309aa489c53273e9b7683d970ce12e2399b5b50031
2023-03-16 14:48:10 -04:00
dergoegge
55c4795c57 [net processing] Use TxRelay::m_relay_txs over CNode::m_relays_txs 2023-03-16 18:23:49 +01:00
fanquake
e695d8536e Merge bitcoin/bitcoin#26177: refactor / kernel: Move non-gArgs chainparams functionality to kernel
b3e78dc91d refactor: Don't use global chainparams in chainstatemanager method (TheCharlatan)
382b692a50 Split non/kernel chainparams (Carl Dong)
edabbc78a3 Add factory functions for Main/Test/Sig/Reg chainparams (Carl Dong)
d938098398 Remove UpdateVersionBitsParameters (Carl Dong)
84b85786f0 Decouple RegTestChainParams from ArgsManager (Carl Dong)
76cd4e7c96 Decouple SigNetChainParams from ArgsManager (Carl Dong)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". dongcarl is the original author of this patchset, these commits were taken from https://github.com/dongcarl/bitcoin/tree/2022-03-libbitcoinkernel-chainparams-args-only.

  #### Context

  The bitcoin kernel library currently relies on code containing user configurations through the `ArgsManager`. This is not optimal, since as a stand-alone library it should not rely on bitcoind's argument parsing logic. Instead, its interfaces should accept control and options structs that control the kernel library's desired configuration.

  Similar work towards decoupling the `ArgsManager` from the kernel has been done in
  https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527 and https://github.com/bitcoin/bitcoin/pull/25862.

  #### Changes

  By moving the `CChainParams` class definition into the kernel and giving it new factory functions `CChainParams::{RegTest,SigNet,Main,TestNet}`it can be constructed without an `ArgsManager` reference, unlike the current factory function `CreateChainParams`.

  The first few commits remove uses of `ArgsManager` within `CChainParams`. Then the `CChainParams` definition is moved to a new file in the `kernel/` subdirectory.

ACKs for top commit:
  MarcoFalke:
    re-ACK b3e78dc91d 🛁
  ryanofsky:
    Code review ACK b3e78dc91d. Only changes since last review were recent review suggestions.
  ajtowns:
    ACK b3e78dc91d

Tree-SHA512: 3835aca1d3e3c75cc3303dd584bab3a77e58f6c678724a5e359fe4b0e17e0763a00931ee6191f516b9fde50496f59cc691f0709c0254206db3863bbf7ab2cacd
2023-03-16 13:56:35 +00:00
Andrew Chow
ebb15ea75a Merge bitcoin/bitcoin#26207: rest: add verbose and mempool_sequence query params for mempool/contents
1ff5d61dfd doc: add mempool/contents rest verbose and mempool_sequence args (Andrew Toth)
52a31dccc9 tests: mempool/contents verbose and mempool_sequence query params tests (Andrew Toth)
a518fff0f2 rest: add verbose and mempool_sequence query params for mempool/contents (Andrew Toth)

Pull request description:

  The verbose mempool json response can get very large. This adds an option to return the non-verbose response of just the txids. It is identical to the rpc response so the diff here is minimal. This also adds the mempool_sequence parameter for rpc consistency. Verbose defaults to true to remain backwards compatible.

  It uses query parameters to be compatible with the efforts in https://github.com/bitcoin/bitcoin/issues/25752.

ACKs for top commit:
  achow101:
    ACK 1ff5d61dfd
  stickies-v:
    re-ACK [1ff5d61](1ff5d61dfd)
  pablomartin4btc:
    tested ACK 1ff5d61dfd.

Tree-SHA512: 1bf08a7ffde2e7db14dc746e421feedf17d84c4b3f1141e79e36feb6014811dfde80e1d8dbc476c15ff705de2d3c967b3081dcd80536d76b7edf888f1a92e9d1
2023-03-15 19:39:30 -04:00
Andrew Chow
609c95d4a8 Merge bitcoin/bitcoin#27227: wallet: 25806 follow-up
475c20aa56 wallet: remove coin control arg from AutomaticCoinSelection (furszy)
8a5583131c wallet: remove unused methods (furszy)
8471967d7b wallet: GroupOutput, remove unneeded "spendable" check (furszy)
a9aa04183c wallet: OutputGroup, remove unused effective_feerate member (furszy)
99034b2b72 wallet: APS, don't create empty groups (furszy)
805f399b17 wallet: do not make two COutputs, use shared_ptr (furszy)

Pull request description:

  Few small findings post-#25806 and extra cleanups, nothing biggie.

ACKs for top commit:
  S3RK:
    Code review ACK 475c20aa56
  Xekyo:
    utACK 475c20aa56
  achow101:
    ACK 475c20aa56

Tree-SHA512: df45efebd6e2e4ecac619d6ecef794979c328a2d6ef532e25124d0dc1c72b55ccf13498f61fb65958b907bfba6a72ed569bf34eb5fbe35419632fe0406e78798
2023-03-15 19:07:19 -04:00
Andrew Chow
cbfbf46e57 Merge bitcoin/bitcoin#25781: Remove almost all blockstorage globals
fadf8b8182 refactor: Add and use PRUNE_TARGET_MANUAL constexpr (MarcoFalke)
fa9bd7be47 Move ::fImporting to BlockManager (MarcoFalke)
fa442b1377 Pass fImporting to ImportingNow helper class (MarcoFalke)
fa177d7b6b Move ::fPruneMode into BlockManager (MarcoFalke)
fa721f1cab Move ::nPruneTarget into BlockManager (MarcoFalke)

Pull request description:

  It seems preferable to assign globals to a class (in this case `BlockManager`), than to leave them dangling. This should clarify scope for code-readers, as well as clarifying unit test behaviour.

ACKs for top commit:
  TheCharlatan:
    Code review ACK fadf8b8182
  achow101:
    ACK fadf8b8182
  dergoegge:
    Code review ACK fadf8b8182

Tree-SHA512: d261b69257560c9f460bbe85944ca478d0390b498a5af514bafcb4f6444841e5ea58c2e8982f38c48685d6f649039234aec853a934e24ebf23e20d975991a5dc
2023-03-15 18:52:35 -04:00
ishaanam
dbeca792a9 test: fix race condition in encrypted wallet rescan tests 2023-03-15 17:27:57 -04:00
Sebastian Falbesoner
6d24d1ef2b test: check that sigop limit also affects ancestor/descendant size 2023-03-15 19:39:25 +01:00
TheCharlatan
b3e78dc91d refactor: Don't use global chainparams in chainstatemanager method
The chainstatemanager m_options.chainparams member variable gets its
value from the global chainparams in init.cpp. This allows
validation.cpp to only include the the kernel chainparams file.
2023-03-15 16:43:33 +01:00
Carl Dong
382b692a50 Split non/kernel chainparams
Moves chainparams code not using the ArgsManager to the kernel.

Subsequently use the kernel chainparams header now where possible in
order to further decouple chainparams call sites from gArgs.
2023-03-15 16:43:31 +01:00
Carl Dong
edabbc78a3 Add factory functions for Main/Test/Sig/Reg chainparams
This normalizes the behavior of initializing Main/Test/Sig/Reg
chainparams with RegTest/SigNet chainparams. These factory functions can
also easily be used from a context without an instantiated ArgsManager,
e.g. from libbitcoin kernel code, unlike the existing CreateChainParams
method.
2023-03-15 16:43:29 +01:00
Carl Dong
d938098398 Remove UpdateVersionBitsParameters
Moves setting struct member fields from a function to its call site.
This improves readability by surfacing the code.
2023-03-15 16:43:27 +01:00
Carl Dong
84b85786f0 Decouple RegTestChainParams from ArgsManager
RegTest chain params can now be initialized by configuring a
RegTestOptions struct, or with ArgsManager. This offers an interface for
creating RegTestChainParams without a gArgs object.
2023-03-15 16:42:42 +01:00
brunoerg
9c18992bba test: add coverage for -bantime 2023-03-15 12:26:50 -03:00
Carl Dong
76cd4e7c96 Decouple SigNetChainParams from ArgsManager
SigNet chain params can now be initialized by configuring a
SigNetOptions struct, or with ArgsManager. This offers an interface for
creating SigNetChainParams without a gArgs object.
2023-03-15 16:10:09 +01:00
MarcoFalke
fadf8b8182 refactor: Add and use PRUNE_TARGET_MANUAL constexpr 2023-03-15 16:02:47 +01:00
MarcoFalke
fa9bd7be47 Move ::fImporting to BlockManager 2023-03-15 15:48:44 +01:00
MarcoFalke
fa442b1377 Pass fImporting to ImportingNow helper class 2023-03-15 15:47:48 +01:00
MarcoFalke
fa177d7b6b Move ::fPruneMode into BlockManager 2023-03-15 15:47:42 +01:00
Gleb Naumenko
72e8ffd7f8 p2p: Account for MANUAL conns when diversifying persistent outbound conns
Previously, we would make connections to peer from the netgroups to which
our MANUAL outbound connections belong.
However, they should be seen as regular connections from Addrman when it comes to netgroup diversity check, since the same rationale can be applied.

Note, this has nothing to do with how we connect to MANUAL connections:
we connect to them unconditionally.
2023-03-15 20:12:05 +05:30
Gleb Naumenko
3faae99c3d p2p: Diversify connections only w.r.t *persistent* outbound peers
ADDR_FETCH and FEELER are short-lived connections,
and they should not affect our choice of peers.

Also, improve comments.
2023-03-15 20:12:03 +05:30
MarcoFalke
fa721f1cab Move ::nPruneTarget into BlockManager 2023-03-15 15:33:12 +01:00
fanquake
8c4958bd4c Merge bitcoin/bitcoin#27153: guix: pass --enable-initfini-array to release GCC
127c637cf0 guix: pass --enable-initfini-array to release GCC (fanquake)

Pull request description:

  This returns us to pre-Guix behaviour, where the compilers we were using to build releases, were configured with this option.

  > [--enable-initfini-array](https://gcc.gnu.org/install/configure.html)
  > Force the use of sections .init_array and .fini_array (instead of .init and .fini) for constructors and destructors. Option --disable-initfini-array has the opposite effect. If neither option is specified, the configure script will try to guess whether the .init_array and .fini_array sections are supported and, if they are, use them.

ACKs for top commit:
  TheCharlatan:
    ACK 127c637cf0
  vincenzopalazzo:
    utACK 127c637cf0

Tree-SHA512: fa61227054d52d4dfb4524af3888203a501f680661bdef00bb0970d4e8f7c96cf7f592686c4795be5a0debca267b8e564a4960859297c31f6b261c0729238382
2023-03-15 08:41:21 +01:00
Hennadii Stepanov
460e394625 Merge bitcoin-core/gui#708: Mask values on Transactions View
4492de1be1 qt: mask values on transactions view (pablomartin4btc)

Pull request description:

  Currently the mask values option (Settings menu->Mask values) hides the wallet balances shown on the Overview page including the recent transactions list from the right panel but it doesn't hide the amounts from the transaction view.

  ![mask values - hiding wallet balances on overview tab but not on transactions tab](https://user-images.githubusercontent.com/110166421/216876325-56a68006-1be0-4b3f-b1e2-a0575c377cf5.gif)

  This enhancement has been mentioned on PR #701 as a [desirable follow-up](https://github.com/bitcoin-core/gui/pull/701#issuecomment-1401350037).

  First approach was to hide the amounts on the  transactions view when mask values option is checked:

  ![mask values - hiding amounts on transactions tab](https://user-images.githubusercontent.com/110166421/216876440-0ff1a2ec-2ef2-405c-8b62-e4a94b9221cc.gif)

  But later on as reviewer **furszy** recommended, I've disabled the Transaction tab directly and switch to the Overview tab if the mask values option is set, check the new screenshots in the [comment below](https://github.com/bitcoin-core/gui/pull/708#issuecomment-1449025828).

ACKs for top commit:
  achow101:
    ACK 4492de1be1
  furszy:
    ACK 4492de1
  hernanmarino:
    ACK 4492de1be1

Tree-SHA512: 94c04064cde456ab916cb91fc4619353cf4c8ef0e92aa1522a6b2c18a8d0fa641db9fddac04c2e1a290128879db598a9dfc4a0003dcf6e3413543908ada95afb
2023-03-14 10:03:31 +00:00
fanquake
b175bdb9b2 Merge bitcoin/bitcoin#27238: refactor: Split logging utilities from system.h
aaced5633b refactor: Move error() from util/system.h to logging.h (Ben Woosley)
e7333b420e refactor: Extract util/exception from util/system (Ben Woosley)

Pull request description:

  This pull request is part of the `libbitcoinkernel` project https://github.com/bitcoin/bitcoin/issues/24303 https://github.com/bitcoin/bitcoin/projects/18 and more specifically its "Step 2: Decouple most non-consensus code from libbitcoinkernel". These commits were originally authored by empact and are taken from their parent PR #25152.

  #### Context

  There is an ongoing effort to decouple the `ArgsManager` used for command line parsing user-provided arguments from the libbitcoinkernel library (https://github.com/bitcoin/bitcoin/pull/25290, https://github.com/bitcoin/bitcoin/pull/25487, https://github.com/bitcoin/bitcoin/pull/25527, https://github.com/bitcoin/bitcoin/pull/25862, https://github.com/bitcoin/bitcoin/pull/26177, and https://github.com/bitcoin/bitcoin/pull/27125). The `ArgsManager` is defined in `system.h`.

  #### Changes

  Next to providing better code organization, this PR removes some reliance of the tree of libbitcoinkernel header includes on `system.h` (and thus the `ArgsManager` definition) by moving some logging functions out of the `system.*` files.

  Further commits splitting more functionality out of `system.h` are still in #25152 and will be submitted in separate PRs once this PR has been processed.

ACKs for top commit:
  MarcoFalke:
    re-ACK aaced5633b 🐍

Tree-SHA512: cb39f4cb7a77e7dc1887b1cbf340d53decab8880fc00878a2f12dc627fe67245b4aafd4cc31a9eab0fad1e5bb5d0eb4cdb8d501323ca200fa6ab7b201ae34aea
2023-03-14 09:56:40 +01:00
glozow
f50fb178c3 Merge bitcoin/bitcoin#27235: Avoid integer overflow in CheckDiskSpace
05eeba2c5f [test] Add manual prune startup test case (dergoegge)
4517419628 [util] Avoid integer overflow in CheckDiskSpace (dergoegge)

Pull request description:

  Starting a fresh node with `-prune=1` causes an integer overflow to happen in `CheckDiskSpace` ([here](f7bdcfc83f/src/init.cpp (L1633-L1648))) because `nPruneTarget` is to the max `uint64_t` value.
  ```
   node1 stderr util/system.cpp:138:51: runtime error: unsigned integer overflow: 52428800 + 18446744073709551615 cannot be represented in type 'unsigned long'
      #0 0x564a482b5088 in CheckDiskSpace(fs::path const&, unsigned long) src/./src/util/system.cpp:138:51
      #1 0x564a4728dc59 in AppInitMain(node::NodeContext&, interfaces::BlockAndHeaderTipInfo*) src/./src/init.cpp:1639:14
      #2 0x564a47256e6a in AppInit(node::NodeContext&, int, char**) src/./src/bitcoind.cpp:221:43
      #3 0x564a47256087 in main src/./src/bitcoind.cpp:265:13
      #4 0x7fcb7cbffd8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
      #5 0x7fcb7cbffe3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
      #6 0x564a471957f4 in _start (/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bitcoind+0xca07f4) (BuildId: 035cb22302d37317a630900a15a26ecb326d395c)
  SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow util/system.cpp:138:51 in
  ```

  I think side stepping the overflow for this specific case, is better than adding an exception to the UB suppresions file.

ACKs for top commit:
  MarcoFalke:
    ACK 05eeba2c5f 🥝
  john-moffett:
    ACK 05eeba2c5f

Tree-SHA512: 1d8e6bcb49818139f04b5ab2cbef7f9b422bf0c38a804cd532b6bd0ba4c4fd07f959ba977e59896343f213086c8ecc48180f50d006638dc84649c66ec379d58a
2023-03-13 17:01:48 +00:00
glozow
73a9892bce Merge bitcoin/bitcoin#26514: Improve address decoding errors
962a0930e6 Improve address decoding errors (Aurèle Oulès)

Pull request description:

  Attempt to fix #21741.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 962a0930e6
  davidgumberg:
    ACK 962a0930e6
  1440000bytes:
    utACK 962a0930e6

Tree-SHA512: 6f216eeaeccf6bfdf0730d38835fdf26c935a5e1fc35006660393a9ad76bf38c85340f0f20e92f87840463d83d891d9714cfad313aab301a16bb8efa4490df06
2023-03-13 16:57:31 +00:00
Ben Woosley
aaced5633b refactor: Move error() from util/system.h to logging.h
error is a low-level function with a sole dependency on LogPrintf, which
is defined in logging.h

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving the function out of system.h allows including it from a separate
source file without including the ArgsManager definitions from system.h.
2023-03-13 17:09:54 +01:00
Ben Woosley
e7333b420e refactor: Extract util/exception from util/system
This is a minimal extraction of a single function, but also the only use
of std::exception in util/system.

The background of this commit is an ongoing effort to decouple the
libbitcoinkernel library from the ArgsManager defined in system.h.
Moving the function out of system.h allows including it from a separate
source file without including the ArgsManager definitions from system.h.
2023-03-13 17:09:47 +01:00
fanquake
f088949fcf Merge bitcoin/bitcoin#27221: test: Default timeout factor to 4 under --valgrind
fa27cf4cc7 test: Default timeout factor to 4 under --valgrind (MarcoFalke)

Pull request description:

  valgrind will incur a slowdown of at least 2, so increase the default timeout factor.

  This should reduce the number of reported issues. See also https://github.com/bitcoin/bitcoin/issues/27112#issuecomment-1455762739

ACKs for top commit:
  fanquake:
    ACK fa27cf4cc7 - I still see at least one actual issue when running the functional tests under `--valgrind` (outside the CI system), but will follow up separately with that. Increasing the timeout here seems fine.

Tree-SHA512: 4467559a3bfd98f5735f300f6ed54c68f951191d65a2b1294d71d72cc5d0864964de562d5dfa0a4855fc541ccb269a538b7aeb3d408d2d012a5369513397c395
2023-03-13 16:15:04 +01:00
dergoegge
05eeba2c5f [test] Add manual prune startup test case 2023-03-13 13:10:11 +01:00
fanquake
285edfadca guix: use osslsigncode 2.5
Co-authored-by: Andrew Chow <github@achow101.com>
2023-03-12 19:37:44 +01:00
fanquake
1884b71b1d Merge bitcoin/bitcoin#27230: Update src/secp256k1 subtree to upstream release v0.3.0
763079a3f1 Squashed 'src/secp256k1/' changes from 21ffe4b22a9..bdf39000b9c (Pieter Wuille)

Pull request description:

  This updates the libsecp256k1 subtree to [v0.3.0](https://github.com/bitcoin-core/secp256k1/releases/tag/v0.3.0). I don't believe there are code changes that are particularly important to Bitcoin Core, apart from the added CMake build system support.

ACKs for top commit:
  jonasnick:
    ACK e5c7fcb361
  fanquake:
    ACK e5c7fcb361

Tree-SHA512: eda42e44d6d4ae43e9fab8a15854e41c8d9e14b645945039dbc35402bec501d73caa5d293264bd03ec6a7fe4919b9a725560f1831a58a6364dc6edaf259145a0
2023-03-12 16:35:57 +01:00
fanquake
40d0b0a524 Merge bitcoin/bitcoin#27239: refactor: Consistently use context args over gArgs in node/interfaces
fa3e9b420f refactor: Consistently use args over gArgs in init.cpp (MarcoFalke)
fa891120c8 refactor: Consistently use context args over gArgs in node/interfaces (MarcoFalke)

Pull request description:

  Currently `node/interfaces.cpp` uses a mix of `gArgs` vs `m_context->args`. This is fine, because outside of tests those should be identical. However, it makes the code inconsistent and harder to use in tests.

  Fix that by using `args` from the context consistently. Do the same in `init.cpp`, where `gArgs` and `args` are inconsistently used in the same scope or even line.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa3e9b420f

Tree-SHA512: bcd52f176794ebb1ecb9e1922411f7b84d212ae13bad314a1961b85f3077f645fca71fb0124c0889ebfdd8b59a0903b99b9985b1a4fb8f152aa6d7f0126fe5c7
2023-03-11 11:20:01 +01:00
fanquake
87af64a1a1 Merge bitcoin/bitcoin#27236: util: fix argsman dupe key error
8fcbdadfad util: fix argsman dupe key error (willcl-ark)

Pull request description:

  fixes #22638

  Make GUI "Settings file could not be read. Do you want to reset settings to default values?" dialog actually clear all settings instead of partially keeping them when `settings.json` contains duplicate keys. This change has no effect on `bitcoind` because it treats a corrupt `settings.json` file as a hard error and doesn't attempt to modify it.

  If we find a duplicate key and error, clear `values` before returning so that `WriteSettings()` will write an empty file, therefore clearing it.

  This aligns with GUI behaviour added in 1ee6d0b.

  The test added only checks that `values` is empty after a duplicate key is detected. This paves the way for the `abort` option in the GUI to properly clear `settings.json`, if the user selects the option, but the test does not currently check this entire mechanism (e.g. the file contents).

ACKs for top commit:
  TheCharlatan:
    Code review ACK 8fcbdadfad
  ryanofsky:
    Code review ACK 8fcbdadfad. Thanks for the fix! I would maybe update the PR description or add to it to describe behavior change of this PR:

Tree-SHA512: a5fd49b30ede0a24188623192825bccb952e427cc35f96ff9bfdc737361dcc35ac6480589ddf7f0ddeaebd34361bdaee31e7a91f2c0d857e4ff682614bb6bc04
2023-03-11 11:09:47 +01:00
fanquake
3d53a85ddf Merge bitcoin/bitcoin#27220: doc: update broken str util reference links on developer-notes
da347de530 doc: update broken links (pablomartin4btc)

Pull request description:

  References to `utilstrencodings` and `lint-locale-dependence.sh` where incorrect, updating them accordingly.

  Also, adding another reference to util function [`LocaleIndependentAtoi`](https://github.com/bitcoin/bitcoin/blob/master/src/util/strencodings.h#L108-L118), which is related with the updated section of the guide:

  ```
  // LocaleIndependentAtoi is provided for backwards compatibility reasons.
  //
  // New code should use ToIntegral or the ParseInt* functions
  // which provide parse error feedback.
  //
  // The goal of LocaleIndependentAtoi is to replicate the defined behaviour of
  // std::atoi as it behaves under the "C" locale, and remove some undefined
  // behavior. If the parsed value is bigger than the integer type's maximum
  // value, or smaller than the integer type's minimum value, std::atoi has
  // undefined behavior, while this function returns the maximum or minimum
  // values, respectively.
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK da347de530

Tree-SHA512: c8f4cd9cff1fb3ea367ac9dbe5aa45dc187fc60114f2e2106e02e0e17fea4ee34d6e0c408fe920c2d8765e06b4dc30c231f0454fa35469c4399e0cadbcd341ba
2023-03-11 11:02:03 +01:00
pablomartin4btc
4492de1be1 qt: mask values on transactions view 2023-03-10 14:52:01 -03:00
brunoerg
fabb95e7bf doc: add release note for 26899 2023-03-10 13:42:32 -03:00
brunoerg
c84c5f6e89 p2p: set -dnsseed and -listen false if maxconnections=0
If `maxconnections=0`, it means our possible connections are
going to be manual (e.g via `addnode`). For this reason, we
can skip DNS seeds and set `listen` false.
2023-03-10 13:42:32 -03:00
fanquake
c7f1d95f52 Merge bitcoin/bitcoin#27205: doc: Show how less noisy clang-tidy output can be achieved
54c4d03578 doc: Show how less noisy clang-tidy output can be achieved (TheCharlatan)

Pull request description:

  Adds a paragraph to the clang-tidy section explaining how to de-noise its output. By default clang-tidy will print errors arrising from included headers in leveldb and other dependencies. By passing `--enable-suppress-external-warnings` flag to configure, errors arising from external dependencies are suppressed. Additional errors arrising from internal dependencies such as leveldb are suppressed by passing the `src/.bear-tidy-config` configuration file to bear. This file includes exclusionary rules for leveldb.

ACKs for top commit:
  MarcoFalke:
    utACK 54c4d03578

Tree-SHA512: c3dd8fb0600157582a38365a587e02e1d249fb246d6b8b4949a800fd05d3473dee49e2a4a556c60e51d6508feff810024e55fe09f5a0875f560fde30f3b6817c
2023-03-10 17:38:56 +01:00
MarcoFalke
fa3e9b420f refactor: Consistently use args over gArgs in init.cpp 2023-03-10 17:29:51 +01:00
MarcoFalke
fa891120c8 refactor: Consistently use context args over gArgs in node/interfaces 2023-03-10 17:29:04 +01:00
TheCharlatan
54c4d03578 doc: Show how less noisy clang-tidy output can be achieved 2023-03-10 16:47:41 +01:00
dergoegge
4517419628 [util] Avoid integer overflow in CheckDiskSpace 2023-03-10 15:50:26 +01:00
furszy
f3221d373a test: add wallet too-long-mempool-chain error coverage 2023-03-10 11:29:37 -03:00
furszy
acf0119d24 wallet: return error msg for too-long-mempool-chain failure
We currently return "Insufficient funds" which doesn't really
describe what went wrong; the tx creation failed because of
a long-mempool-chain, not because of a lack of funds.

Also, return early from Coin Selection if the sum of the
discarded coins decreases the available balance below the
target amount.
2023-03-10 11:29:37 -03:00
fanquake
99b64eec1b Merge bitcoin/bitcoin#27232: Use string interpolation for default value of -listen
5c938e74cf Use string interpolation for default value of -listen (ekzyis)

Pull request description:

  This is a refactoring change. So I have read the following and will try to answer why this change should be accepted

  > * Refactoring changes are only accepted if they are required for a feature or
    bug fix or **_otherwise improve developer experience significantly_**. For example,
    most "code style" refactoring changes require a thorough explanation why they
    are useful, what downsides they have and why they *significantly* improve
    developer experience or avoid serious programming bugs. Note that code style
    is often a subjective matter. Unless they are explicitly mentioned to be
    preferred in the [developer notes](/doc/developer-notes.md), stylistic code
    changes are usually rejected.

  I have noticed in https://github.com/bitcoin/bitcoin/pull/26899#discussion_r1086731856 that the helper message for `-listen` does not use string interpolation.

  That confused me and I wasn't sure what the reasons for that are. So it could be argued this confusion (by possibly many people in the past and in the future) may already be enough to accept this change.

  However, not accepting this means that if `DEFAULT_LISTEN` is ever changed, this helper message will still use the old value (however unlikely that may be).

  Therefore, this PR makes the helper message consistent with how other helper messages are implemented (using string interpolation) which leads to less confusion and prevents possibly wrong documentation in the future.

ACKs for top commit:
  stratospher:
    ACK 5c938e7.
  vasild:
    ACK 5c938e74cf
  kristapsk:
    cr utACK 5c938e74cf

Tree-SHA512: 8905f548ff399967966e67b29962821c28fd2620ff788c2b2bdfd088bbca75457dc63e933ad1985f93580508a65b91930fe4b2d97a2e0a7d83a3748b9ea2c26f
2023-03-10 14:57:15 +01:00
fanquake
3e7dd4ff33 Merge bitcoin/bitcoin#27171: test: add coverage for sigop limit policy (-bytespersigop setting)
89cd20cbed test: add coverage for sigop limit policy (`-bytespersigop` setting) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `-bytespersigop` option, which determines how pre-taproot signature operations (OP_CHECKSIG{VERIFY}, OP_CHECKMULTIGSIG{VERIFY}) affect fee handling calculations. The setting was introduced in PR #7081 for mitigating the [sigop spam attack](https://bitcointalk.org/index.php?topic=1166928.0); the initial implementation rejected txs exceeding the limit, but was changed in #8365 later to account for higher sizes in the mempool (i.e. exceeding the sigop limit is possible, but has to be compensated by higher fees).

  For each combination of `-bytespersigop` setting and sigops count, the test first creates a P2WSH spending transaction with a witness script that puts sigops in a non-executing branch (OP_FALSE OP_IF OP_CHECKMULTISIG ... OP_CHECKSIG ... OP_ENDIF). This tx is then bumped up to reach exactly the _sig-op limit equivalent vsize_ by padding its datacarrier output. Based on that, increasing the tx's vsize should still reflect a vsize increase in the mempool, while a decrease of the tx's vsize should lead to the mempool treating the tx's vsize to be the _sig-op limit equivalent vsize_, since the limit was exceeded.

  I assume that this parameter is almost never set explicitly by users (also it is not relevant for taproot spends), but it doesn't hurt to have a test for it. See also https://bitcoin.stackexchange.com/a/87958 for another explanation.

ACKs for top commit:
  glozow:
    light review ACK 89cd20cbed
  MarcoFalke:
    nice ACK 89cd20cbed  📁

Tree-SHA512: 06998ce93bf9d5ce6143db2996a43f13990c415f97afe684227ad469349e73952bf4f6c871c1e6349e07606f4d45db64408848873a86a89481cdca5a134e5e60
2023-03-10 14:34:34 +01:00
fanquake
6f5eb7a39e Merge bitcoin/bitcoin#27226: test: Use self.wait_until over wait_until_helper
faa671591f test: Use self.wait_until over wait_until_helper (MarcoFalke)

Pull request description:

  `wait_until_helper` is a "private" helper, not intended to be used directly, because it doesn't scale the timeout with the timeout factor. Fix this by replacing it with a call to `self.wait_until`, which does the scaling.

ACKs for top commit:
  theStack:
    Code-review ACK faa671591f

Tree-SHA512: 70705f309f83ffd6ea5d090218195d05b868624d909106863372f861138b5a70887070b25beb25044ae1b44250345e45c9cc11191ae7aeca2ad37801a0f62f61
2023-03-10 14:26:06 +01:00
fanquake
6e662a8985 Merge bitcoin/bitcoin#23813: Add test and docs for getblockfrompeer with pruning
fe329dc936 test: Add test for getblockfrompeer on pruned nodes (Fabian Jahr)
cd761e6b2c rpc: Add note on guarantees to getblockfrompeer (Fabian Jahr)

Pull request description:

  These are additions to `getblockfrompeer` that I already [suggested on the original PR](https://github.com/bitcoin/bitcoin/pull/20295#pullrequestreview-817157738).

  The two commits do the following:
  1. Add a test for `getblockfrompeer` usage on pruned nodes. This is important because many use-cases for `getblockfrompeer` are in a context of a pruned node.
  2. Add some information on how long the users of pruned nodes can expect the block to be available after they have used the RPC. I think the behavior is not very intuitive for users and I would not be surprised if users expect the block to be available indefinitely.

ACKs for top commit:
  Sjors:
    re-utACK fe329dc936
  MarcoFalke:
    review ACK fe329dc936 🍉
  stratospher:
    ACK  fe329dc.
  brunoerg:
    re-ACK fe329dc936

Tree-SHA512: a686bd8955d9c3baf365db384e497d6ee1aa9ce2fdb0733fe6150f7e3d94bae19d55bc1b347f1c9f619e749e18b41a52b9f8c0aa2042dd311a968a4b5d251fac
2023-03-10 14:25:00 +01:00
MarcoFalke
fa18504d57 rpc: Add submit option to generateblock 2023-03-10 10:39:14 +01:00
MarcoFalke
fab9a08e14 refactor: Replace block_hash with block_out 2023-03-10 10:39:08 +01:00
willcl-ark
8fcbdadfad util: fix argsman dupe key error
fixes #22638

If we find a duplicate key and error, clear `values` before returning so that
WriteSettings will write an empty file, therefore clearing it.

This aligns with GUI behaviour added in 1ee6d0b.
2023-03-09 23:24:06 +00:00
glozow
f7bdcfc83f Merge bitcoin/bitcoin#27025: github: Switch to yaml issue templates
3fa1185dda github: Switch to yaml issue templates (willcl-ark)

Pull request description:

  The new YAML templates provide more flexibility and can be designed to extract more information from users when submitting issues, avoiding initial back-and-forth when reports do not include enough background information to begin with.

  Key differences:

  * YAML format
  * Allows us to require responses to certain questions
  * Not currently compatible with GitLab (.md only)

  This does keep the "Blank Issue" option at the bottom.

  Testing this must be done with the master branch of the repo, which is slightly annoying for this repo. I have therefore pushed this to my own fork so that you can see the new templates, along with how the output is rendered in newly-created issues:

  [github.com/willcl-ark/bitcoin/issues](https://github.com/willcl-ark/bitcoin/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc)

  I did make some minor changes to some of the template wording, but this change could also be a good time to add/remove additional questions.

  This seems like a net-positive for me, setting aside the issue that if we ever migrated away from GitHub these might have to be ported back to *.md (or something else), but that seems easy-enough that this change would be worth it.

  Curious to know what others think of this, and whether they would suggest adding any other questions to any of the templates as part of this update?

ACKs for top commit:
  achow101:
    ACK 3fa1185dda
  glozow:
    ACK 3fa1185dda

Tree-SHA512: ce7990cd5f951e3839bc54022ce9f4b0ff9ffb8b19754d657d79acf9118bbdc4aba196f872cd5511b81e03993e54dfe4fcb85e89deade024e5a65a336adb638b
2023-03-09 17:08:53 +00:00
MarcoFalke
faf3f12424 refactor: Replace GetTimeMicros by SystemClock 2023-03-09 15:05:11 +01:00
Hennadii Stepanov
9985013350 Merge bitcoin-core/gui#717: Use the stored CSubNet entry when unbanning
4be57a5df1 gui: fix comments for BanTableModel and BanTablePriv::refreshBanlist() (Vasil Dimov)
a981af4e6f gui: use the stored CSubNet entry when unbanning (Vasil Dimov)

Pull request description:

  The previous code visualized the `CSubNet` object as string, then parsed that string back to `CSubNet`. This is sub-optimal given that the original `CSubNet` object can be used directly instead.

  This avoids calling `LookupSubNet()` from the GUI.

ACKs for top commit:
  furszy:
    utACK 4be57a5d
  mzumsande:
    Tested ACK 4be57a5df1

Tree-SHA512: b783c18c9d676aa9486cff2d27039dd5c5ef3f1cc67e5056a2be68e35930926f368f26dacdf4f3d394a1f73e3e28f42dc8a6936cd1765c6e6e60695c7b4d78af
2023-03-09 13:49:22 +00:00
ekzyis
5c938e74cf Use string interpolation for default value of -listen 2023-03-09 10:46:30 +01:00
willcl-ark
65e3abcbf2 doc: document json rpc endpoints
fixes #20246

Document both JSON-RPC endpoints, when they are active and which types
of requests they are able to service.

Adds two example curl requests, one for each endpoint.
2023-03-09 08:41:59 +00:00
Andrew Chow
23e2bfcbc4 Merge bitcoin/bitcoin#25696: build: Re-enable external signer on Windows
1a0d8e178c build: Re-enable external signer on Windows (Hennadii Stepanov)
989451d068 configure: Detect compatibility of Boost.Process rather than hardcode non-Windows (Luke Dashjr)

Pull request description:

  As https://github.com/boostorg/process/issues/207 has been resolved, it is possible now to re-enable external signer on Windows when cross-compiling.

  Guix build hashes:
  ```
  78f69ea7e0dbc8338981a92c0352220ccd7c2272d8cbff6a3b082a1412a935c5  guix-build-1a0d8e178c7b/output/aarch64-linux-gnu/SHA256SUMS.part
  ee17456ec818ddf5a175182508966e622573ccb518807cca43a40fa1dceda092  guix-build-1a0d8e178c7b/output/aarch64-linux-gnu/bitcoin-1a0d8e178c7b-aarch64-linux-gnu-debug.tar.gz
  5080551bde379c746cc67b10429aef33b9f9e49d2d4e21ee1c3bfd9c1c845d46  guix-build-1a0d8e178c7b/output/aarch64-linux-gnu/bitcoin-1a0d8e178c7b-aarch64-linux-gnu.tar.gz
  dfab220ce76a40bf7dcf07aab352a616a91b516503639455fe7e1b137bad3e85  guix-build-1a0d8e178c7b/output/arm-linux-gnueabihf/SHA256SUMS.part
  516ceb822571a8bd88fe107dca434ef596b1e4328ccbda1d51e1d482d3050396  guix-build-1a0d8e178c7b/output/arm-linux-gnueabihf/bitcoin-1a0d8e178c7b-arm-linux-gnueabihf-debug.tar.gz
  21325380638f817107c203b9a1aedb808d1a4a2b4041493753ca4cbf19aa4f2c  guix-build-1a0d8e178c7b/output/arm-linux-gnueabihf/bitcoin-1a0d8e178c7b-arm-linux-gnueabihf.tar.gz
  cf48ed78fcfceaeb3610ccf22326d735a129dcbf9d50b557b3de359169aefdfd  guix-build-1a0d8e178c7b/output/arm64-apple-darwin/SHA256SUMS.part
  d4d51e136148bac6a20bb3adb402c499967647736acb420bfdeb71603aba57da  guix-build-1a0d8e178c7b/output/arm64-apple-darwin/bitcoin-1a0d8e178c7b-arm64-apple-darwin-unsigned.dmg
  95bb62d24f860e08a392ddb74d5860ccf27e8baa183e6749af877d26a3bd6b0b  guix-build-1a0d8e178c7b/output/arm64-apple-darwin/bitcoin-1a0d8e178c7b-arm64-apple-darwin-unsigned.tar.gz
  68da4c92f37bb802df37141af194f47c16da1d84f77a0fbb1016013ae0338502  guix-build-1a0d8e178c7b/output/arm64-apple-darwin/bitcoin-1a0d8e178c7b-arm64-apple-darwin.tar.gz
  6704e38c2d3f11321403797598d05f062648fec6f2d76900ba250dab481e29da  guix-build-1a0d8e178c7b/output/dist-archive/bitcoin-1a0d8e178c7b.tar.gz
  64b936bc90d1e01fe8f276511edc9bb945dcebe70332aa37d3a786348443b8e7  guix-build-1a0d8e178c7b/output/powerpc64-linux-gnu/SHA256SUMS.part
  3d03532e54b6e42498ea240c86b8567e94fd462f56087b869c3d6f09e2dde878  guix-build-1a0d8e178c7b/output/powerpc64-linux-gnu/bitcoin-1a0d8e178c7b-powerpc64-linux-gnu-debug.tar.gz
  c5843d79a58b0a864fe723458dab4eee54ad11f4b1f7960975b086eeedc0d541  guix-build-1a0d8e178c7b/output/powerpc64-linux-gnu/bitcoin-1a0d8e178c7b-powerpc64-linux-gnu.tar.gz
  f861ff519bd5e3d6d5ce1646ee0a06bcef1288ddb804a4a600e4dbfe5d5be521  guix-build-1a0d8e178c7b/output/powerpc64le-linux-gnu/SHA256SUMS.part
  5f477da21980dbcf9696081903dc1ba8a3f79ce3579641d208e69a6f598c8eb9  guix-build-1a0d8e178c7b/output/powerpc64le-linux-gnu/bitcoin-1a0d8e178c7b-powerpc64le-linux-gnu-debug.tar.gz
  b3757b11c614136934158acea5139e8abd0c5c9cdfda72ae44db436f21716b33  guix-build-1a0d8e178c7b/output/powerpc64le-linux-gnu/bitcoin-1a0d8e178c7b-powerpc64le-linux-gnu.tar.gz
  1c21bdb17fe3436e685e88c62423e630fe2b3c41dd00025a99fd80d97817ac2f  guix-build-1a0d8e178c7b/output/riscv64-linux-gnu/SHA256SUMS.part
  f36ae98473f086ae8f0dc66223b5ec407d57dc4d8d45ae284401520ff5c0b273  guix-build-1a0d8e178c7b/output/riscv64-linux-gnu/bitcoin-1a0d8e178c7b-riscv64-linux-gnu-debug.tar.gz
  1603e4d0e869eb47a1dc2d26b67772d0016d90f7ba5e50d2009365cc02cb8169  guix-build-1a0d8e178c7b/output/riscv64-linux-gnu/bitcoin-1a0d8e178c7b-riscv64-linux-gnu.tar.gz
  f86ef652102f022827b70477bffa0a44008c6300cf62ca7b3595146cf2ed91ba  guix-build-1a0d8e178c7b/output/x86_64-apple-darwin/SHA256SUMS.part
  f84d435d8e4709bf29bc7ac7ed8dc6b8af4077cef05e520b468b2896ce10876a  guix-build-1a0d8e178c7b/output/x86_64-apple-darwin/bitcoin-1a0d8e178c7b-x86_64-apple-darwin-unsigned.dmg
  af2aab969b7ed7aeea0e02adbcc9e3b438086bf76b6bfc36146c53e05a27bd57  guix-build-1a0d8e178c7b/output/x86_64-apple-darwin/bitcoin-1a0d8e178c7b-x86_64-apple-darwin-unsigned.tar.gz
  32a5109ba28ab74ff66238e6a8f8a04e455ebce382a3be287df92a227818fe72  guix-build-1a0d8e178c7b/output/x86_64-apple-darwin/bitcoin-1a0d8e178c7b-x86_64-apple-darwin.tar.gz
  377462e9a96f4aba72c915dd5df5159a4301a1fa8ed0ee48faa6c71573de80c3  guix-build-1a0d8e178c7b/output/x86_64-linux-gnu/SHA256SUMS.part
  a3bf62e828d2350a483b2d16205014f66e8884597b0b72e178042a958c548336  guix-build-1a0d8e178c7b/output/x86_64-linux-gnu/bitcoin-1a0d8e178c7b-x86_64-linux-gnu-debug.tar.gz
  66cda980188ea1941a7d66c8b03c447580af33db55abe3bbe3581823ae0534a3  guix-build-1a0d8e178c7b/output/x86_64-linux-gnu/bitcoin-1a0d8e178c7b-x86_64-linux-gnu.tar.gz
  2117f0dd9baeb4d585f841592e94c088f4487bf2008b8f281d0c3ceee92ff6cc  guix-build-1a0d8e178c7b/output/x86_64-w64-mingw32/SHA256SUMS.part
  d40d5dec3287f467c42232c05d82f7fb538cda34bd2e63ff7e1876f471c3a790  guix-build-1a0d8e178c7b/output/x86_64-w64-mingw32/bitcoin-1a0d8e178c7b-win64-debug.zip
  92dcc92765fbc07b1cc8258bfa69280541e1b4553cc41fed18672c2c6931d5c0  guix-build-1a0d8e178c7b/output/x86_64-w64-mingw32/bitcoin-1a0d8e178c7b-win64-setup-unsigned.exe
  a6dd9b4d29f21d3a18cf64556cb03446ef17bf801eb6ac257b65d27cbd95080f  guix-build-1a0d8e178c7b/output/x86_64-w64-mingw32/bitcoin-1a0d8e178c7b-win64-unsigned.tar.gz
  a4022e595d955198f73530473ef8e90a708746089ee2dd27de794176873330c1  guix-build-1a0d8e178c7b/output/x86_64-w64-mingw32/bitcoin-1a0d8e178c7b-win64.zip
  ```

ACKs for top commit:
  Sjors:
    tACK 1a0d8e178c
  achow101:
    ACK 1a0d8e178c

Tree-SHA512: db7319259b1e1571cfab4bb3b99ae10a2f744e62757cae5059fd6f4dd6d5586eb09feb63a0c4bb07f7128b283f1dc281ed435224bc8e40da577fd4f04cde489a
2023-03-08 21:01:53 -05:00
Pieter Wuille
763079a3f1 Squashed 'src/secp256k1/' changes from 21ffe4b22a9..bdf39000b9c
bdf39000b9c Merge bitcoin-core/secp256k1#1223: release: prepare for 0.3.0
b40adf23604 release: prepare for 0.3.0
90b513aadad Merge bitcoin-core/secp256k1#1229: cmake: Rename project to "libsecp256k1"
8be82d43628 cmake: Rename project to "libsecp256k1"
ef4f8bd0259 Merge bitcoin-core/secp256k1#1227: readme: Use correct build type in CMake/Windows build instructions
756b61d451d readme: Use correct build type in CMake/Windows build instructions
3295aa149bd Merge bitcoin-core/secp256k1#1225: changelog: Add entry for CMake
92098d84cf7 changelog: Add entry for CMake
df323b5c146 Merge bitcoin-core/secp256k1#1113: build: Add CMake-based build system
e1eb33724c2 ci: Add "x86_64: Windows (VS 2022)" task
10602b0030e cmake: Export config files
5468d709644 build: Add CMake-based build system
6048e6c03e4 Merge bitcoin-core/secp256k1#1222: Remove redundant checks.
eb8749fcd0f Merge bitcoin-core/secp256k1#1221: Update Changelog
5d8f53e3129 Remove redudent checks.
9d1b458d5fb Merge bitcoin-core/secp256k1#1217: Add secp256k1_fe_add_int function
d232112fa7e Update Changelog
8962fc95bb0 Merge bitcoin-core/secp256k1#1218: Update overflow check
2ef1c9b3870 Update overflow check
57573187826 Merge bitcoin-core/secp256k1#1212: Prevent dead-store elimination when clearing secrets in examples
b081f7e4cbf Add secp256k1_fe_add_int function
5660c137552 prevent optimization in algorithms
09b1d466db7 Merge bitcoin-core/secp256k1#979: Native jacobi symbol algorithm
ce3cfc78a60 doc: Describe Jacobi calculation in safegcd_implementation.md
6be01036c8a Add secp256k1_fe_is_square_var function
1de2a01c2b2 Native jacobi symbol algorithm
04c6c1b1816 Make secp256k1_modinv64_det_check_pow2 support abs val
5fffb2c7af5 Make secp256k1_i128_check_pow2 support -(2^n)
cbd25559343 Merge bitcoin-core/secp256k1#1209: build: Add SECP256K1_API_VAR to fix importing variables from DLLs
1b21aa51752 Merge bitcoin-core/secp256k1#1078: group: Save a normalize_to_zero in gej_add_ge
e4330341bd6 ci: Shutdown wineserver whenever CI script exits
9a5a611a21f build: Suppress stupid MSVC linker warning
739c53b19a2 examples: Extend sig examples by call that uses static context
914276e4d27 build: Add SECP256K1_API_VAR to fix importing variables from DLLs
1cca7c1744b Merge bitcoin-core/secp256k1#1206: build: Add -Wreserved-identifier supported by clang
8c7e0fc1de0 build: Add -Wreserved-identifier supported by clang
8ebe5c52050 Merge bitcoin-core/secp256k1#1201: ci: Do not set git's `user.{email,name}` config options
5596ec5c2cf Merge bitcoin-core/secp256k1#1203: Do not link `bench` and `ctime_tests` to `COMMON_LIB`
ef39721ccce Do not link `bench` and `ctime_tests` to `COMMON_LIB`
9b60e3148d8 ci: Do not set git's `user.{email,name}` config options
e1817a6f54f Merge bitcoin-core/secp256k1#1199: ci: Minor improvements inspired by Bitcoin Core
1bff2005885 Merge bitcoin-core/secp256k1#1200: Drop no longer used Autoheader macros
9b7d18669dc Drop no longer used Autoheader macros
c2415866c7a ci: Don't fetch git history
0ecf3188515 ci: Use remote pull/merge ref instead of local git merge
2b77240b3ba Merge bitcoin-core/secp256k1#1172: benchmarks: fix bench_scalar_split
eb6bebaee39 scalar: restrict split_lambda args, improve doc and VERIFY_CHECKs
7f49aa7f2dc ci: add test job with -DVERIFY
620ba3d74be benchmarks: fix bench_scalar_split
5fbff5d348f Merge bitcoin-core/secp256k1#1170: contexts: Forbid destroying, cloning and randomizing the static context
233822d849d Merge bitcoin-core/secp256k1#1195: ctime_tests: improve output when CHECKMEM_RUNNING is not defined
ad7433b1409 Merge bitcoin-core/secp256k1#1196: Drop no longer used variables from the build system
e39d954f118 tests: Add CHECK_ILLEGAL(_VOID) macros and use in static ctx tests
2cd4e3c0a97 Drop no longer used `SECP_{LIBS,INCLUDE}` variables
613626f94c7 Drop no longer used `SECP_TEST_{LIBS,INCLUDE}` variables
61841fc9ee5 contexts: Forbid randomizing secp256k1_context_static
4b6df5e33e1 contexts: Forbid cloning/destroying secp256k1_context_static
b1579cf5fb4 Merge bitcoin-core/secp256k1#1194: Ensure safety of ctz_debruijn implementation.
8f51229e034 ctime_tests: improve output when CHECKMEM_RUNNING is not defined
d6ff738d5bb Ensure safety of ctz_debruijn implementation.
a01a7d86dc2 Merge bitcoin-core/secp256k1#1192: Switch to exhaustive groups with small B coefficient
a7a7bfaf3dc Merge bitcoin-core/secp256k1#1190: Make all non-API functions (except main) static
f29a3270923 Merge bitcoin-core/secp256k1#1169: Add support for msan instead of valgrind (for memcheck and ctime test)
ff8edf89e2e Merge bitcoin-core/secp256k1#1193: Add `noverify_tests` to `.gitignore`
ce60785b265 Introduce SECP256K1_B macro for curve b coefficient
4934aa79958 Switch to exhaustive groups with small B coefficient
d4a6b58df74 Add `noverify_tests` to `.gitignore`
88e80722d2a Merge bitcoin-core/secp256k1#1160: Makefile: add `-I$(top_srcdir)/{include,src}` to `CPPFLAGS` for precomputed
0f088ec1126 Rename CTIMETEST -> CTIMETESTS
74b026f05d5 Add runtime checking for DECLASSIFY flag
5e2e6fcfc0e Run ctime test in Linux MSan CI job
18974061a3f Make ctime tests building configurable
5048be17e93 Rename valgrind_ctime_test -> ctime_tests
6eed6c18ded Update error messages to suggest msan as well
8e11f89a685 Add support for msan integration to checkmem.h
8dc64079eb1 Add compile-time error to valgrind_ctime_test
0db05a770eb Abstract interactions with valgrind behind new checkmem.h
4f1a54e41d8 Move valgrind CPPFLAGS into SECP_CONFIG_DEFINES
cc3b8a4f404 Merge bitcoin-core/secp256k1#1187: refactor: Rename global variables in tests
9a93f48f502 refactor: Rename STTC to STATIC_CTX in tests
3385a2648d7 refactor: Rename global variables to uppercase in tests
e03ef865593 Make all non-API functions (except main) static
cbe41ac138b Merge bitcoin-core/secp256k1#1188: tests: Add noverify_tests which is like tests but without VERIFY
203760023c6 tests: Add noverify_tests which is like tests but without VERIFY
e862c4af0c5 Makefile: add -I$(top_srcdir)/src to CPPFLAGS for precomputed
0eb3000417f Merge bitcoin-core/secp256k1#1186: tests: Tidy context tests
39e8f0e3d7b refactor: Separate run_context_tests into static vs proper contexts
a4a09379b1a tests: Clean up and improve run_context_tests() further
fc90bb56956 refactor: Tidy up main()
f32a36f620e tests: Don't use global context for context tests
ce4f936c4fa tests: Tidy run_context_tests() by extracting functions
18e0db30cb4 tests: Don't recreate global context in scratch space test
b19806122e9 tests: Use global copy of secp256k1_context_static instead of clone
2a39ac162e0 Merge bitcoin-core/secp256k1#1185: Drop `SECP_CONFIG_DEFINES` from examples
2f9ca284e2a Drop `SECP_CONFIG_DEFINES` from examples
31ed5386e84 Merge bitcoin-core/secp256k1#1183: Bugfix: pass SECP_CONFIG_DEFINES to bench compilation
c0a555b2ae3 Bugfix: pass SECP_CONFIG_DEFINES to bench compilation
01b819a8c7d Merge bitcoin-core/secp256k1#1158: Add a secp256k1_i128_to_u64 function.
eacad90f699 Merge bitcoin-core/secp256k1#1171: Change ARG_CHECK_NO_RETURN to ARG_CHECK_VOID which returns (void)
3f57b9f7749 Merge bitcoin-core/secp256k1#1177: Some improvements to the changelog
c30b889f17e Clarify that the ABI-incompatible versions are earlier
881fc33d0c1 Consistency in naming of modules
665ba77e793 Merge bitcoin-core/secp256k1#1178: Drop `src/libsecp256k1-config.h`
75d7b7f5bae Merge bitcoin-core/secp256k1#1154: ci: set -u in cirrus.sh to treat unset variables as an error
7a746882013 ci: add missing CFLAGS & CPPFLAGS variable to print_environment
c2e0fdadebd ci: set -u in cirrus.sh to treat unset variables as an error
9c5a4d21bbe Do not define unused `HAVE_VALGRIND` macro
ad8647f548c Drop no longer relevant files from `.gitignore`
b627ba7050b Remove dependency on `src/libsecp256k1-config.h`
9ecf8149a19 Reduce font size in changelog
2dc133a67ff Add more changelog entries
ac233e181a5 Add links to diffs to changelog
cee8223ef6d Mention semantic versioning in changelog
9a8d65f07f1 Merge bitcoin-core/secp256k1#1174: release cleanup: bump version after 0.2.0
02ebc290f74 release cleanup: bump version after 0.2.0
b6b360efafc doc: improve message of cleanup commit
a49e0940ad6 docs: Fix typo
2551cdac903 tests: Fix code formatting
c635c1bfd54 Change ARG_CHECK_NO_RETURN to ARG_CHECK_VOID which returns (void)
cf66f2357c6 refactor: Add helper function secp256k1_context_is_proper()
d2164752053 test secp256k1_i128_to_i64
4bc429019dc Add a secp256k1_i128_to_u64 function.
e089eecc1e5 group: Further simply gej_add_ge
ac71020ebe0 group: Save a normalize_to_zero in gej_add_ge

git-subtree-dir: src/secp256k1
git-subtree-split: bdf39000b9c6a0818e7149ccb500873d079e6e85
2023-03-08 17:41:24 -05:00
Pieter Wuille
e5c7fcb361 Update src/secp256k1 subtree to upstream libsecp256k1 v0.3.0 2023-03-08 17:41:24 -05:00
furszy
475c20aa56 wallet: remove coin control arg from AutomaticCoinSelection
we only need the "include unsafe" flag, not all what coin
control stores.
2023-03-08 19:03:40 -03:00
pablomartin4btc
da347de530 doc: update broken links 2023-03-08 18:26:41 -03:00
fanquake
710fd571ff Merge bitcoin/bitcoin#26996: test: Flatten miniwallet array and remove random fee in longpoll
fa0abcdafe test: Flatten miniwallet array and remove random fee in longpoll (MarcoFalke)

Pull request description:

  * Using a single MiniWallet is enough.
  * A random fee isn't needed either.

ACKs for top commit:
  theStack:
    re-ACK fa0abcdafe

Tree-SHA512: 77b99885b3f0d325d067838122114be57ec999ebc82912de6a22c33e2ba28a341c5e053c5bbc424b9922c2616562289a57c7156bd3b431d779182c2e472da59c
2023-03-08 18:32:43 +01:00
Andrew Chow
1ff135ca7f Merge bitcoin/bitcoin#26194: rpc, wallet: use the same next_index key in listdescriptors and importdescriptors
b082f28101 rpc, wallet: use the same `next_index` in listdescriptors and importdescriptors (w0xlt)

Pull request description:

  Currently `listdescriptors` RPC uses `next` key to represent `WalletDescriptor::next_index` while `importdescriptors` uses `next_index`. This creates two different descriptor formats.

  This  PR changes `listdescriptors` to use the same key as `importdescriptors`.

ACKs for top commit:
  achow101:
    ACK b082f28101
  aureleoules:
    reACK b082f28101

Tree-SHA512: c29ec59051878e614d749ed6dc85e5c14ad00db0e8fcbce3f5066d1aae85ef07ca70f02920299e48d191b7387024fe224b0054c4191a5951cb805106f7b8e37b
2023-03-08 12:15:31 -05:00
furszy
8a5583131c wallet: remove unused methods
CWallet::DummySignTx, OutputGroupTypeMap::find
2023-03-08 10:32:30 -03:00
furszy
8471967d7b wallet: GroupOutput, remove unneeded "spendable" check
`AvailableCoins` already filters non-spendable coins.
2023-03-08 10:32:30 -03:00
furszy
a9aa04183c wallet: OutputGroup, remove unused effective_feerate member 2023-03-08 10:32:30 -03:00
furszy
99034b2b72 wallet: APS, don't create empty groups
By moving the "positive-only" flag out of
the lambda function.
2023-03-08 10:32:30 -03:00
furszy
805f399b17 wallet: do not make two COutputs, use shared_ptr 2023-03-08 10:15:06 -03:00
MarcoFalke
faa671591f test: Use self.wait_until over wait_until_helper 2023-03-08 11:31:56 +01:00
fanquake
8d12127a9c Merge bitcoin/bitcoin#26968: doc: Fixup remove 'omitted...' doc for rpc getrawtransaction when verbose is 2
3e947d7117 doc: remove 'omitted...' doc for rpc getrawtransaction when verbose is 2 (dougEfish)

Pull request description:

  Remove optional rpc doc for getrawtransaction when verbose is 2

ACKs for top commit:
  stickies-v:
    ACK 3e947d7117

Tree-SHA512: b9e970d6ef4a47ec7ca32f5ff1028cc901f1bfdc1571668208505d42f4160733530601b78e469de82a854d3b298a55a81d0a7916bc5db4a43ad6d6a299c55c9e
2023-03-08 08:55:20 +01:00
fanquake
69ba5727d5 Merge bitcoin/bitcoin#27180: doc: DummySignInput mention external signer
6fc5f4fdb6 doc: DummySignInput mention external signer (Sjors Provoost)

Pull request description:

  Followups for #26032. So far nothing major.

ACKs for top commit:
  ishaanam:
    ACK 6fc5f4fdb6
  S3RK:
    ACK 6fc5f4fdb6

Tree-SHA512: e27edde9853487fe3eef8213f991aae3724f318bbbe0b11da23759879adaf9a31771e6ea0c30baaebca149032780b89b32aa540ff456ca3d5ec6adb0371749c6
2023-03-08 08:49:25 +01:00
fanquake
2de0559f2c Merge bitcoin/bitcoin#27189: util: Use steady clock in SeedStrengthen, FindBestImplementation, FlushStateToDisk
fa1b4e5c32 Use steady clock in FlushStateToDisk (MarcoFalke)
1111e2f8b4 Use steady clock in SeedStrengthen and FindBestImplementation (MarcoFalke)

Pull request description:

  There may be a theoretical deadlock for the duration of the offset when the system clock is adjusted into a past time while executing `SeedStrengthen`.

  Fix this by using steady clock.

  Do the same in `FindBestImplementation`, which shouldn't be affected, because it discards outlier measurements. However, doing the same there for consistency seems fine.

  Do the same in `FlushStateToDisk`, which should make the flushes more steady, if the system clock is adjusted by a large offset.

ACKs for top commit:
  john-moffett:
    ACK fa1b4e5c32
  willcl-ark:
    ACK fa1b4e5c3

Tree-SHA512: cc625e796b186accd53222bd64eb57d0512bc7e588312d254349b542bbc5e5daac348ff2b3b3f7dc5ae0bbbae2ec11fdbf3022cf2164211633765a4b0108e83e
2023-03-08 08:48:41 +01:00
Andrew Chow
d5e4f9a439 Merge bitcoin/bitcoin#25740: assumeutxo: background validation completion
2b373fe49d docs: update assumeutxo.md (James O'Beirne)
87a1108c81 test: add snapshot completion unittests (James O'Beirne)
d70919a88f refactor: make MempoolMutex() public (James O'Beirne)
7300ced9de log: add LoadBlockIndex() message for assumedvalid blocks (James O'Beirne)
d96c59cc5c validation: add ChainMan logic for completing UTXO snapshot validation (James O'Beirne)
f2a4f3376f move-only-ish: init: factor out chainstate initialization (James O'Beirne)
637a90b973 add Chainstate::HasCoinsViews() (James O'Beirne)
c29f26b47b validation: add CChainState::m_disabled and ChainMan::isUsable (James O'Beirne)
5ee22cdafd add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}() (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606)

  Part two of replacing https://github.com/bitcoin/bitcoin/pull/24232.

  ---

  When a user activates a snapshot, the serialized UTXO set data is used to create an "assumed-valid" chainstate, which becomes active in an attempt to get the node to network tip as quickly as possible. Simultaneously in the background, the already-existing chainstate continues "conventional" IBD to both accumulate full block data and serve as a belt-and-suspenders to validate the assumed-valid chainstate.

  Once the background chainstate's tip reaches the base block of the snapshot used, we set `m_stop_use` on that chainstate and immediately take the hash of its UTXO set; we verify that this matches the assumeutxo value in the source code. Note that while we ultimately want to remove this background chainstate, we don't do so until the following initialization process, when we again check the UTXO set hash of the background chainstate, and if it continues to match, we remove the (now unnecessary) background chainstate, and move the (previously) assumed-valid chainstate into its place. We then reinitialize the chainstate in the normal way.

  As noted in previous comments, we could do the filesystem operations "inline" immediately when the background validation completes, but that's basically just an optimization that saves disk space until the next restart. It didn't strike me as worth the risk of moving chainstate data around on disk during runtime of the node, though maybe my concerns are overblown.

  The final result of this completion process is a fully-validated chain, where the only evidence that the user synced using assumeutxo is the existence of a `base_blockhash` file in the `chainstate` directory.

ACKs for top commit:
  achow101:
    ACK 2b373fe49d

Tree-SHA512: a204e1d6e6932dd83c799af3606b01a9faf893f04e9ee1a36d63f2f1ccfa9118bdc1c107d86976aa0312814267e6a42074bf3e2bf1dead4b2513efc6d955e13d
2023-03-07 18:54:59 -05:00
James O'Beirne
2b373fe49d docs: update assumeutxo.md
Include notes about the `chainstate_snapshot` rename as well as
updates for the included code.
2023-03-07 16:06:20 -05:00
James O'Beirne
87a1108c81 test: add snapshot completion unittests
Also adjusts the previous snapshot chainstate init tests
to account for the fact that the init process is now attempting to
validate and complete background chainstates whose tip is at the
snapshot base block. We use a DisconnectTip() hack to preserve the
nature of the test.
2023-03-07 16:06:20 -05:00
James O'Beirne
d70919a88f refactor: make MempoolMutex() public
for use in the following unittests.
2023-03-07 16:06:20 -05:00
James O'Beirne
7300ced9de log: add LoadBlockIndex() message for assumedvalid blocks
I found this useful during unittest debugging.
2023-03-07 16:06:20 -05:00
James O'Beirne
d96c59cc5c validation: add ChainMan logic for completing UTXO snapshot validation
Trigger completion when a background validation chainstate reaches the
same height as a UTXO snapshot, and handle cleaning up the chainstate
on subsequent startup.
2023-03-07 16:06:17 -05:00
Andrew Chow
fc037c8c83 Merge bitcoin/bitcoin#27150: Deduplicate bitcoind and bitcoin-qt init code
802cc1ef53 Deduplicate bitcoind and bitcoin-qt init code (Ryan Ofsky)
d172b5c671 Add InitError(error, details) overload (Ryan Ofsky)
3db2874bd7 Extend bilingual_str support for tinyformat (Ryan Ofsky)
c361df90b9 scripted-diff: Remove double newlines after some init errors (Ryan Ofsky)

Pull request description:

  Add common InitConfig function to deduplicate bitcoind and bitcoin-qt code reading config files and creating the datadir.

  Noticed the duplicate code while reviewing #27073 and want to remove it because difference in bitcoin-qt and bitcoind behavior make it hard to evaluate changes like #27073

  There are a few minor changes in behavior:

  - In bitcoin-qt, when there is a problem reading the configuration file, the GUI error text has changed from "Error: Cannot parse configuration file:" to "Error reading configuration file:" to be consistent with bitcoind.
  - In bitcoind, when there is a problem reading the settings.json file, the error text has changed from "Failed loading settings file" to "Settings file could not be read" to be consistent with bitcoin-qt.
  - In bitcoind, when there is a problem writing the settings.json file, the error text has changed from "Failed saving settings file" to "Settings file could not be written" to be consistent with bitcoin-qt.
  - In bitcoin-qt, if there datadir is not accessible (e.g. no permission to read), there is an normal error dialog showing "Error: filesystem error: status: Permission denied [.../settings.json]", instead of an uncaught exception.

ACKs for top commit:
  Sjors:
    Light review ACK 802cc1ef53
  TheCharlatan:
    ACK 802cc1ef53
  achow101:
    ACK 802cc1ef53

Tree-SHA512: 9c78d277e9ed595fa8ce286b97d2806e1ec06ddbbe7bd3434bd9dd7b456faf8d989f71231e97311f36edb9caaec645a50c730bd7514b8e0fe6e6f7741b13d981
2023-03-07 13:05:01 -05:00
jonatack
8c47d599b8 doc: improve -debuglogfile help to be a bit clearer 2023-03-07 09:33:02 -08:00
jonatack
20d89d6802 bench: document expected results in logging benchmarks
and clarify the intention behind the -nodebuglogfile bench.

Co-authored-by: "kouloumos <kouloumosa@gmail.com>"
Co-authored-by: "Larry Ruane <larryruane@gmail.com>"
2023-03-07 09:32:55 -08:00
Jon Atack
d8deba8c36 bench: add LogPrintfCategory and LogPrintLevel benchmarks
for these new macros that our logging is planned to migrate to.  At some
point it may be feasible to drop some of the previous logging benchmarks.
2023-03-07 08:47:40 -08:00
MarcoFalke
fa0abcdafe test: Flatten miniwallet array and remove random fee in longpoll 2023-03-07 17:47:28 +01:00
Jon Atack
102b203349 bench: order the logging benchmark code by output 2023-03-07 08:45:50 -08:00
Jon Atack
4b3fdbf6fe bench: update logging benchmark naming for clarity
to better track which benchmark corresponds to which log macro.
2023-03-07 08:45:29 -08:00
MarcoFalke
fa27cf4cc7 test: Default timeout factor to 4 under --valgrind 2023-03-07 17:04:27 +01:00
fanquake
d4ebdceaef Merge bitcoin/bitcoin#27218: util: Work around ParseHex gcc cross compiler bug
fa8481b05f util: Work around ParseHex gcc cross compiler bug (MarcoFalke)

Pull request description:

  I fail to see how an explicit `ParseHex` template instantiation fails to also instantiate `TryParseHex`.

  Nonetheless, to work around a compiler bug, change the explicit instantiation from `ParseHex` to `TryParseHex`. (`ParseHex` is inline anyway and will be instantiated by the compiler either way).

  Fixes https://github.com/bitcoin/bitcoin/pull/25227#issuecomment-1456009757 :

  ```
    CXXLD    bitcoind
  /usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_node.a(libbitcoin_node_a-net_processing.o): in function `(anonymous namespace)::PeerManagerImpl::ProcessMessage(CNode&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, CDataStream&, std::chrono::duration<long, std::ratio<1l, 1000000l> >, std::atomic<bool> const&)':
  net_processing.cpp:(.text+0x29660): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
  /usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_node.a(libbitcoin_node_a-rest.o): in function `rest_getutxos(std::any const&, HTTPRequest*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)':
  rest.cpp:(.text+0x83b4): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
  /usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_node.a(libbitcoin_node_a-torcontrol.o): in function `std::vector<unsigned char, std::allocator<unsigned char> > ParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)':
  torcontrol.cpp:(.text._Z8ParseHexIhESt6vectorIT_SaIS1_EESt17basic_string_viewIcSt11char_traitsIcEE[_Z8ParseHexIhESt6vectorIT_SaIS1_EESt17basic_string_viewIcSt11char_traitsIcEE]+0x2c): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
  /usr/lib/gcc-cross/powerpc64le-linux-gnu/11/../../../../powerpc64le-linux-gnu/bin/ld: libbitcoin_common.a(libbitcoin_common_a-external_signer.o): in function `ExternalSigner::SignTransaction(PartiallySignedTransaction&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&)':
  external_signer.cpp:(.text+0x8d84): undefined reference to `std::optional<std::vector<unsigned char, std::allocator<unsigned char> > > TryParseHex<unsigned char>(std::basic_string_view<char, std::char_traits<char> >)'
  collect2: error: ld returned 1 exit status

ACKs for top commit:
  gruve-p:
    ACK fa8481b05f
  hebasto:
    ACK fa8481b05f, tested on Ubuntu 22.04, gcc 11.3 for the `riscv64-linux-gnu` host.,

Tree-SHA512: 53efa424e7e18d85a2c9ac2267b9370ae3594d9be73da5135a3a79bf07ab50fcc5357cbde09dc0b2a9eb78d78ec37beae0c9f876333b568e678b9d0067bc9e4e
2023-03-07 15:42:33 +01:00
furszy
1284223691 wallet: refactor coin selection algos to return util::Result
so the selection processes can retrieve different errors and not
uninformative std::nullopt
2023-03-07 09:01:57 -03:00
MarcoFalke
fa8481b05f util: Work around ParseHex gcc cross compiler bug 2023-03-07 11:33:42 +01:00
Sebastian Falbesoner
89cd20cbed test: add coverage for sigop limit policy (-bytespersigop setting) 2023-03-07 04:23:33 +01:00
Andrew Chow
86bacd75e7 Merge bitcoin/bitcoin#26742: http: Track active requests and wait for last to finish - 2nd attempt
60978c8080 test: Reduce extended timeout on abortnode test (Fabian Jahr)
660bdbf785 http: Release server before waiting for event base loop exit (João Barbosa)
8c6d007c80 http: Track active requests and wait for last to finish (João Barbosa)

Pull request description:

  This revives #19420. Since promag is not so active at the moment, I can support this to finally get it merged.

  The PR is rebased and comments by jonatack have been addressed.

  Once this is merged, I will also reopen #19434.

ACKs for top commit:
  achow101:
    ACK 60978c8080
  stickies-v:
    re-ACK [60978c8](60978c8080)
  hebasto:
    ACK 60978c8080

Tree-SHA512: eef0fe1081e9331b95cfafc71d82f2398abd1d3439dac5b2fa5c6d9c0a3f63ef19adde1c38c88d3b4e7fb41ce7c097943f1815c10e33d165918ccbdec512fe1c
2023-03-06 19:35:59 -05:00
Andrew Chow
4ea3a8b71d Merge bitcoin/bitcoin#25806: wallet: group outputs only once, decouple it from Coin Selection
6a302d40df wallet: single output groups filtering and grouping process (furszy)
bd91ed1cb2 wallet: unify outputs grouping process (furszy)
55962001da test: coinselector_tests refactor, use CoinsResult instead of plain std::vector (furszy)
34f54a0a3a wallet: decouple outputs grouping process from each ChooseSelectionResult (furszy)
461f0821a2 refactor: make OutputGroup::m_outputs field a vector of shared_ptr (furszy)
d8e749bb84 test: wallet, add coverage for outputs grouping process (furszy)
06ec8f9928 wallet: make OutputGroup "positive_only" filter explicit (furszy)

Pull request description:

  The idea originates from https://github.com/bitcoin/bitcoin/pull/24845#issuecomment-1130310321.

  Note:
  For clarity, it's recommended to start reviewing from the end result to understand the structure of the flow.

  #### GroupOutputs function rationale:
  If "Avoid Partial Spends" is enabled, the function gathers outputs with the same script together inside a container. So Coin Selection can treats them as if them were just one possible input and either select them all or not select them.

  #### How the Inputs Fetch + Selection process roughly works:

  ```
  1. Fetch user’s manually selected inputs.
  2. Fetch wallet available coins (walks through the entire wallet txes map) and insert them into a set of vectors (each vector store outputs from a single type).
  3. Coin Selection Process:
     Call `AttemptSelection` 8 times. Each of them expands the coin eligibility filter (accepting a larger subset of coins in the calculation) until it founds a solutions or completely fails if no solutions gets founds after the 8 rounds.

     Each `AttemptSelection` call performs the following actions:
       - For each output type supported by the wallet (P2SH, P2PK, P2WPKH, P2WSH and a combination of all of them):
         Call ‘ChooseSelectionResult’ providing the respective, filtered by type, coins vector. Which:
             I. Groups the outputs vector twice (one for positive only and a second one who includes the negative ones as well).
                - GroupOutputs walks-through the entire inputted coins vector one time at least, + more if we are avoiding partial spends, to generate a vector of OutputGroups.
             II. Then performs every coin selection algorithm using the recently created vector of OutputGroup: (1) BnB, (2) knapsack and (3) SRD.
             III. Then returns the best solution out of them.
  ```

  We perform the general operation of gathering outputs, with the same script, into a single container inside:
  Each coins selection attempt (8 times —> each coin eligibility filter), for each of the outputs vector who were filtered by type (plus another one joining all the outputs as well if needed), twice (one for the positive only outputs effective value and a second one for all of them).

  So, in the worst case scenario where no solution is found after the 8 Coin Selection attempts, the `GroupOutputs` function is called 80 times (8 * 5 * 2).

  #### Improvements:

  This proposal streamlines the process so that the output groups, filtered by coin eligibility and type, are created in a single loop outside of the Coin Selection Process.

  The new process is as follows:

  ```
  1. Fetch user’s manually selected inputs.
  2. Fetch wallet available coins.
  3. Group outputs by each coin eligibility filter and each different output type found.
  4. Coin Selection Process:
     Call AttemptSelection 8 times. Each of them expands the coin eligibility filter (accepting different output groups) until it founds a solutions or completely fails if no solutions gets founds after the 8 rounds.

     Each ‘AttemptSelection’ call performs the following actions:
        - For each output type supported by the wallet (P2SH, P2PK, P2WPKH, P2WSH and all of them):
            A. Call ‘ChooseSelectionResult’ providing the respective, filtered by type, output group. Which:
               I. Performs every coin selection algorithm using the provided vector of OutputGroup: (1) BnB, (2) knapsack and (3) SRD.
               II. Then returns the best solution out of them.
  ```

  Extra Note:
  The next steps after this PR will be to:
  1) Merge `AvailableCoins` and `GroupOutputs` processes.
  2) Skip entire coin selection rounds if no new coins are added into the subsequent round.
  3) Remove global feerates from the OutputGroup class.
  4) Remove secondary "grouped" tx creation from `CreateTransactionInternal` by running Coin Selection results over the aps grouped outputs vs non-aps ones.

ACKs for top commit:
  S3RK:
    ReACK 6a302d4
  achow101:
    ACK 6a302d40df
  theStack:
    re-ACK 6a302d40df 🥥

Tree-SHA512: dff849063be328e7d9c358ec80239a6db2cd6131963b511b83699b95b337d3106263507eaba0119eaac63e6ac21c6c42d187ae23d79d9220b90c323d44b01d24
2023-03-06 18:51:34 -05:00
Larry Ruane
4684aa8733 bench: allow logging benchmarks to be order-independent
The global logging object instance is not re-created for each run, so when
multiple logging benchmarks are run, each one after the first one still has
the logging categories enabled from the previous ones.  This commit disables
all categories at the start of each benchmark.
2023-03-06 12:45:28 -08:00
dougEfish
3e947d7117 doc: remove 'omitted...' doc for rpc getrawtransaction when verbose is 2 2023-03-06 21:52:43 +02:00
Andrew Chow
5e1aab2334 Merge bitcoin/bitcoin#27155: doc: Expand scantxoutset help text to cover tr() and miniscript
e4ede64fe8 Expand scantxoutset help text to cover tr() and miniscript (Greg Sanders)

Pull request description:

ACKs for top commit:
  achow101:
    ACK e4ede64fe8
  darosior:
    webACK e4ede64fe8

Tree-SHA512: 6b5d9e7fccc8242f4534861c1b438ec40fb03fbf5968c5a4af5ddbced73df6d666812053c2e12a1e0a34057f6f4fe11987c8c59d8cdfa48ca7ab7d2720b51ef9
2023-03-06 11:15:16 -05:00
Andrew Chow
dddc936d83 Merge bitcoin/bitcoin#25491: wallet: use Mutex for g_sqlite_mutex instead of GlobalMutex
4163093d63 wallet: use Mutex for g_sqlite_mutex instead of GlobalMutex (Vasil Dimov)

Pull request description:

  Using `Mutex` provides stronger guarantee than `GlobalMutex` wrt Clang's
  thread safety analysis. Thus it is better to reduce the usage of
  `GlobalMutex` in favor of `Mutex`.

  Using `Mutex` for `g_sqlite_mutex` is ok because its usage is limited in
  `wallet/sqlite.cpp` and it does not require propagating the negative
  annotations to not relevant code.

ACKs for top commit:
  achow101:
    ACK 4163093d63
  hebasto:
    re-ACK 4163093d63
  TheCharlatan:
    ACK 4163093d63

Tree-SHA512: 4913bcb8437ecf0e6b6cb781d02a6d24ffb4bf3e2e1899fa60785eab41c4c65dbdd9600bcb696290c873661b873ad61e5a4c4f205b7e66fdef2ae17c676cd12f
2023-03-06 10:50:10 -05:00
Greg Sanders
e4ede64fe8 Expand scantxoutset help text to cover tr() and miniscript 2023-03-06 10:49:43 -05:00
Vasil Dimov
4be57a5df1 gui: fix comments for BanTableModel and BanTablePriv::refreshBanlist() 2023-03-06 16:07:08 +01:00
Vasil Dimov
a981af4e6f gui: use the stored CSubNet entry when unbanning
The previous code visualized the `CSubNet` object as string, then
parsed that string back to `CSubNet`. This is sub-optimal given that
the original `CSubNet` object can be used directly instead.

This avoids calling `LookupSubNet()` from the GUI.

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2023-03-06 16:06:05 +01:00
glozow
2a0c05defd Merge bitcoin/bitcoin#27209: ci: Remove unused EXPECTED_TESTS_DURATION_IN_SECONDS env var
3fffff50f6 ci: Remove unused EXPECTED_TESTS_DURATION_IN_SECONDS env var (MarcoFalke)

Pull request description:

  Remove long unused travis leftover

ACKs for top commit:
  fanquake:
    ACK 3fffff50f6
  dergoegge:
    ACK 3fffff50f6

Tree-SHA512: 34b85a18c0d34f54bbc6ff5c2454307318e4747145e11f52f08baddefef9e1b80d8f6c85efcad97e575380162fd193f2aa837e99b156ed7e3e95370b635ddf4e
2023-03-06 14:00:36 +00:00
furszy
6a302d40df wallet: single output groups filtering and grouping process
Optimizes coin selection by performing the "group outputs"
procedure only once, outside the "attempt selection" process.

Avoiding the repeated execution of the 'GroupOutputs' operation
that occurs on each coin eligibility filters (up to 8 of them);
then for every coin vector type plus one for all the coins together.

This also let us not perform coin selection over coin eligibility
filtered groups that don't add new elements.
(because, if the previous round failed, and the subsequent one has
the same coins, then this new round will fail again).
2023-03-06 09:45:40 -03:00
furszy
bd91ed1cb2 wallet: unify outputs grouping process
The 'GroupOutputs()' function performs the same
calculations for only-positive and mixed groups,
the only difference is that when we look for
only-positive groups, we discard negative utxos.

So, instead of wasting resources calling GroupOutputs()
for positive-only first, then call it again to include
the negative ones in the result, we can execute
GroupOutputs() only once, including in the response
both group types (positive-only and mixed).
2023-03-06 09:45:40 -03:00
furszy
55962001da test: coinselector_tests refactor, use CoinsResult instead of plain std::vector
No functional changes. Only cosmetic changes to simplify the follow-up commit.
2023-03-06 09:45:40 -03:00
furszy
34f54a0a3a wallet: decouple outputs grouping process from each ChooseSelectionResult
Another step towards the single OutputGroups calculation goal
2023-03-06 09:45:40 -03:00
furszy
461f0821a2 refactor: make OutputGroup::m_outputs field a vector of shared_ptr
Initial steps towards sharing COutput instances across all possible
OutputGroups (instead of copying them time after time).
2023-03-06 09:45:40 -03:00
furszy
d8e749bb84 test: wallet, add coverage for outputs grouping process
The following scenarios are covered:

1) 10 UTXO with the same script:
   partial spends is enabled --> outputs must not be grouped.

2) 10 UTXO with the same script:
   partial spends disabled --> outputs must be grouped.

3) 20 UTXO, 10 one from scriptA + 10 from scriptB:
   a) if partial spends is enabled --> outputs must not be grouped.
   b) if partial spends is not enabled --> 2 output groups expected (one per script).

3) Try to add a negative output (value - fee < 0):
   a) if "positive_only" is enabled --> negative output must be skipped.
   b) if "positive_only" is disabled --> negative output must be added.

4) Try to add a non-eligible UTXO (due not fulfilling the min depth target for
 "not mine" UTXOs) --> it must not be added to any group

5) Try to add a non-eligible UTXO (due not fulfilling the min depth target for
 "mine" UTXOs) --> it must not be added to any group

6) Surpass the 'OUTPUT_GROUP_MAX_ENTRIES' size and verify that a second partial
group gets created.
2023-03-06 09:45:40 -03:00
Amiti Uttarwar
9bf078f66c refactor: update Select_ function
Extract the logic that decides whether the new or the tried table is going to
be searched to the beginning of the function.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-03-05 17:34:15 -08:00
Sebastian Falbesoner
3dd2f6461b test: psbt: check non-witness UTXO removal for segwit v1 input 2023-03-05 04:07:11 +01:00
Sebastian Falbesoner
dd78e3fa43 test: speedup rpc_psbt.py by whitelisting peers (immediate tx relay)
master branch:
    0m36.86s real     0m03.26s user     0m01.69s system
    0m35.71s real     0m03.78s user     0m01.64s system
    0m45.76s real     0m03.12s user     0m01.27s system

PR branch:
    0m13.04s real     0m02.66s user     0m00.93s system
    0m14.08s real     0m02.81s user     0m00.82s system
    0m14.05s real     0m02.50s user     0m00.93s system
2023-03-05 01:46:57 +01:00
Sebastian Falbesoner
e194e3e93d test: PSBT: eliminate magic numbers for global unsigned tx key (0) 2023-03-04 12:43:38 +01:00
fanquake
40c6c85c05 Merge bitcoin/bitcoin#27192: util: add missing include and fix function signature
8847ce44e0 util: add missing include and fix function signature (Cory Fields)

Pull request description:

  ping hebasto

  Discovered while testing pre-compiled header support with CMake: https://github.com/theuni/bitcoin/commits/cmake-pch-poc. Compilation of that branch fails without this fix and succeeds with it.

  Similar to the fix in #27144.

  The problem of having a default argument in the definition was masked by the missing include. Using PCH forces that include, so we end up with the compiler error we should've been getting all along.

ACKs for top commit:
  fanquake:
    ACK 8847ce44e0

Tree-SHA512: 5eb9a6691ee37cbc5033a48aedcbf5c93af3b234614ae14c3fcc858f1ee505f630ad68f8bbb69ffa280080c8d0f91451362cb3819290b741ce906b2b3224a622
2023-03-04 08:17:37 +01:00
fanquake
236cd231d0 Merge bitcoin/bitcoin#27197: Fix typos in comments to make linter happy
987f1bb41c Fixed a couple of typos in comments to make linter happy (hernanmarino)

Pull request description:

  While working on a different PR, I stumbled upon a couple of typos being reported by the linter and fixed them.

ACKs for top commit:
  kristapsk:
    utACK 987f1bb41c

Tree-SHA512: bf53e150bb7c6b59a9dc270bdf748bf8fb0430f35d1d0a68ef70e9345d322470c4a0b5cd7f2c3cfc81ce087bb6624d820d5f4501f58aaa5806f358ddd4e0b551
2023-03-04 08:10:46 +01:00
Cory Fields
8847ce44e0 util: add missing include and fix function signature 2023-03-03 22:19:00 +00:00
hernanmarino
987f1bb41c Fixed a couple of typos in comments to make linter happy 2023-03-03 19:06:02 -03:00
furszy
06ec8f9928 wallet: make OutputGroup "positive_only" filter explicit
And not hide it inside the `OutputGroup::Insert` method.
This method does not return anything if insertion fails.

We can know before calling `Insert` whether the coin
will be accepted or not.
2023-03-03 18:18:03 -03:00
fanquake
3b88c85025 Merge bitcoin/bitcoin#26612: refactor: RPC: pass named argument value as string_view
545ff924ab refactor: use string_view for RPC named argument values (stickies-v)
7727603e44 refactor: reduce unnecessary complexity in ParseNonRFCJSONValue (stickies-v)
1d02e59901 test: add cases to JSON parsing (stickies-v)

Pull request description:

  Inspired by MarcoFalke's [comment](https://github.com/bitcoin/bitcoin/pull/26506#discussion_r1036149426). Main purpose of this PR is to minimize copying (potentially large) RPC named arguments when calling `.substr()` by using `std::string_view` instead of `std::string`. Furthermore, cleans up the code by removing unnecessary complexity in `ParseNonRFCJSONValue()` (done first to avoid refactoring required to concatenate `string` and `string_view`), updates some naming and adds a few test cases. Should not introduce any behaviour change.

  ## Questions
  - ~Was there actually any merit to `ParseNonRFCJSONValue()` surrounding the value with brackets and then parsing it as an array? I don't see it, and the new approach doesn't fail any tests. Still a bit suspicious about it though.~
    - Cleared up by https://github.com/bitcoin/bitcoin/pull/26506#pullrequestreview-1211984059
    - If there are no objections to 7727603e44, I think we should follow up with a PR to rename `ParseNonRFCJSONValue()` to a local `Parse()` helper function (that throws if invalid), remove it from `client.h` and merge the test coverage we currently have on `ParseNonRFCJSONValue()` with the coverage we have on `UniValue::read()`.

ACKs for top commit:
  ryanofsky:
    Code review ACK 545ff924ab
  MarcoFalke:
    review ACK 545ff924ab 📻

Tree-SHA512: b1c89fb010ac9c3054b023cac1acbba2a539a09cf39a7baffbd7f7571ee268d5a6d98701c7ac10d68a814526e8fd0fe96ac1d1fb072f272033e415b753f64a5c
2023-03-03 15:23:43 +01:00
fanquake
a12b27a2a6 Merge bitcoin/bitcoin#27183: doc: Update Transifex links and slug format in Release Process
9c371e50a2 doc: Update Transifex links and slug format in Release Process (Hennadii Stepanov)

Pull request description:

  Reflected the recent changes in Transifex's [workflow](https://github.com/bitcoin/bitcoin/pull/26321) and on its website.

ACKs for top commit:
  jarolrod:
    ACK 9c371e50a2

Tree-SHA512: d11f2fa197bba2e3198ca255d80edd7178e8074dc84b03262054d01fa33c8160ca492a1358e9da1e7d32cec21a298593322d10e4cb045ef99ccec2cbe901cb63
2023-03-02 22:02:58 +01:00
MarcoFalke
fa1b4e5c32 Use steady clock in FlushStateToDisk 2023-03-02 15:05:17 +01:00
MarcoFalke
1111e2f8b4 Use steady clock in SeedStrengthen and FindBestImplementation 2023-03-02 14:48:28 +01:00
fanquake
127c637cf0 guix: pass --enable-initfini-array to release GCC
This returns us to pre-Guix behaviour, where the compilers we were using
to build releases, were configured with this option.
2023-03-01 21:23:36 +01:00
Andrew Chow
74981aa02d Merge bitcoin/bitcoin#27172: guix: switch to some minimal versions of packages in our manifest
2c9eb4afe1 guix: use cmake-minimal over cmake (fanquake)
1475515312 guix: use coreutils-minimal over coreutils (fanquake)
4445621415 guix: use bash-minimal over bash (fanquake)

Pull request description:

  Minimal versions of the same packages, that should still be sufficient for our use:

  > (define-public bash-minimal
    ;; A stripped-down Bash for non-interactive use.

  > (define-public coreutils-minimal
    ;; Coreutils without its optional dependencies.

  > ;;; This minimal variant of CMake does not include the documentation.  It is
  ;;; used by the cmake-build-system.
  (define-public cmake-minimal

ACKs for top commit:
  TheCharlatan:
    ACK 2c9eb4afe1
  Sjors:
    tACK 2c9eb4afe1
  achow101:
    ACK 2c9eb4afe1
  hebasto:
    ACK 2c9eb4afe1,

Tree-SHA512: f91ca9e088b8346b20c2affc80870c31640de3aedcfcc0fb98a5e82c77ef64537870b88552f26759d31d8d0956b1fd685e6c25d5acbc92f5feaececd1a7dd37e
2023-03-01 11:07:04 -05:00
Hennadii Stepanov
9c371e50a2 doc: Update Transifex links and slug format in Release Process 2023-03-01 15:01:16 +00:00
fanquake
4d24e9c571 Merge bitcoin/bitcoin#27169: Update translations for 25.0 soft translation string freeze
9172cc672e qt: Update translation source file (Hennadii Stepanov)
7b0cbf444d qt: Bump Transifex slug for 25.x (Hennadii Stepanov)
369023d22d qt: Periodic translation updates from Transifex (Hennadii Stepanov)

Pull request description:

  This PR follows our [Release Process](https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md).

  Required to open Transifex translations for 25.0 on 2023-03-01 as it's [planned](https://github.com/bitcoin/bitcoin/issues/26549).

  **NOTE.** Translations for the following languages for the latest 24.x Transifex resource have been effectively cancelled/damaged/vandalized:
  - German (de) by [nesbonk83](https://www.transifex.com/user/profile/nesbonk83/) on 2023-01-27
  - Dutch (nl) by [bram00767](https://www.transifex.com/user/profile/bram00767/) on 2022-12-17
  - Spanish, Mexico (es_MX) by [VCFNFT](https://www.transifex.com/user/profile/VCFNFT/) on 2022-08-08

  The first commit ignores changes to translations mentioned above.

ACKs for top commit:
  jarolrod:
    ACK 9172cc672e

Tree-SHA512: 85641facecd11526bbcde934b43629aba1b856c4f97272a956c2ce194af8a1723325a160a0a518fc052af9373f853204848b58d3c0a3bea09788fccfc5d9f557
2023-03-01 14:31:06 +01:00
Sjors Provoost
6fc5f4fdb6 doc: DummySignInput mention external signer 2023-03-01 11:44:53 +00:00
fanquake
cb40639bdf Merge bitcoin/bitcoin#27165: Make miniscript_{stable,smart} fuzzers avoid too large scripts
56e37e71a2 Make miniscript fuzzers avoid script size limit (Pieter Wuille)
bcec5ab4ff Make miniscript fuzzers avoid ops limit (Pieter Wuille)
213fffa513 Enforce type consistency in miniscript_stable fuzz test (Pieter Wuille)
e1f30414c6 Simplify miniscript fuzzer NodeInfo struct (Pieter Wuille)
5abb0f5ac3 Do base type propagation in miniscript_stable fuzzer (Pieter Wuille)

Pull request description:

  This adds a number of improvements to the miniscript fuzzers that all amount to rejecting invalid or overly big miniscripts early on:
  * Base type propagation in the miniscript_stable fuzzers prevents constructing a large portion of miniscripts that would be illegal, with just a little bit of type logic in the fuzzer. The fuzzer input format is unchanged.
  * Ops and script size tracking in GenNode means that too-large scripts (either due to script size limit or ops limit) will be detected on the fly during fuzz input processing, before actually constructing the scripts.

  Closes #27147.

ACKs for top commit:
  darosior:
    re-ACK 56e37e71a2
  dergoegge:
    tACK 56e37e71a2

Tree-SHA512: 245584adf9a6644a35fe103bc81b619e5b4f5d467571a761b5809d08b1dec48f7ceaf4d8791ccd8208b45c6b309d2ccca23b3d1ec5399df76cd5bf88f2263280
2023-02-28 17:04:47 +00:00
Ryan Ofsky
802cc1ef53 Deduplicate bitcoind and bitcoin-qt init code
Add common InitConfig function to deduplicate bitcoind and bitcoin-qt code
reading config files and creating the datadir.

There are a few minor changes in behavior:

- In bitcoin-qt, when there is a problem reading the configuration file, the
  GUI error text has changed from "Error: Cannot parse configuration file:" to
  "Error reading configuration file:" to be consistent with bitcoind.
- In bitcoind, when there is a problem reading the settings.json file, the
  error text has changed from "Failed loading settings file" to "Settings
  file could not be read" to be consistent with bitcoin-qt.
- In bitcoind, when there is a problem writing the settings.json file, the
  error text has changed from "Failed saving settings file" to "Settings file
  could not be written" to be consistent with bitcoin-qt.
- In bitcoin-qt, if there datadir is not accessible (e.g. no permission to read),
  there is an normal error dialog showing "Error: filesystem error: status:
  Permission denied [.../settings.json]", instead of an uncaught exception
2023-02-28 12:04:47 -05:00
Ryan Ofsky
d172b5c671 Add InitError(error, details) overload
This is only used in the current PR to avoid ugly
`strprintf(Untranslated("%s:\n%s"), str, MakeUnorderedList(details)`
boilerplate in init code. But in the future the function could be extended and
more widely used to include more details in GUI error messages or display them
in a more readable way, see code comment.
2023-02-28 12:04:47 -05:00
Ryan Ofsky
3db2874bd7 Extend bilingual_str support for tinyformat
Previous bilingual_str tinyformat::format accepted bilingual format strings,
but not bilingual arguments. Extend it to accept both. This is useful when
embedding one translated string inside another translated string, for example:
`strprintf(_("Error: %s"), message)` which would fail previously if `message`
was a bilingual_str.
2023-02-28 12:04:47 -05:00
Ryan Ofsky
c361df90b9 scripted-diff: Remove double newlines after some init errors
Some InitError calls had trailing \n characters, causing double newlines in
error output. After this change InitError calls consistently output one newline
instead of two. Appearance of messages in the GUI does not seem to be affected.
Can be tested with:

  src/bitcoind -regtest -datadir=noexist
  src/qt/bitcoin-qt -regtest -datadir=noexist

-BEGIN VERIFY SCRIPT-
git grep -l InitError src/ | xargs sed -i 's/\(InitError(.*\)\\n"/\1"/'
-END VERIFY SCRIPT-
2023-02-28 12:04:47 -05:00
fanquake
4398cfb22b Merge bitcoin/bitcoin#27173: valgrind: remove libsecp256k1 suppression
29b62c01c8 valgrind: remove libsecp256k1 suppression (fanquake)

Pull request description:

  I am no-longer been able to recreate this issue, atleast after the most recent libsecp256k1 changes. Can someone else confirm?

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 29b62c01c8
  sipa:
    utACK 29b62c01c8

Tree-SHA512: e61e5b88ac2af3c779f23d999938bec4497f6433a029c07dd57a9481fe9cc104d8d8f63586910b29f41e66bbf0ed094bc7539c0df84754a1783c4b1b15af072e
2023-02-28 16:56:29 +00:00
glozow
a8080c0def Merge bitcoin/bitcoin#23897: refactor: Move calculation logic out from CheckSequenceLocksAtTip()
75db62ba4c refactor: Move calculation logic out from `CheckSequenceLocksAtTip()` (Hennadii Stepanov)
3bc434f459 refactor: Add `CalculateLockPointsAtTip()` function (Hennadii Stepanov)

Pull request description:

  This PR is follow up for bitcoin/bitcoin#22677 and bitcoin/bitcoin#23683.

  On master (013daed9ac) it is not obvious that `CheckSequenceLocksAtTip()` function can modify its `LockPoints* lp` parameter which leads to https://github.com/bitcoin/bitcoin/pull/22677#discussion_r762040101.

  This PR:
  - separates the lockpoint calculate logic from `CheckSequenceLocksAtTip()` function into a new `CalculateLockPointsAtTip()` one
  - cleans up the `CheckSequenceLocksAtTip()` function interface
  - makes code easier to reason about (hopefully)

ACKs for top commit:
  achow101:
    ACK 75db62ba4c
  stickies-v:
    re-ACK 75db62b

Tree-SHA512: 072c3fd9cd1e1b0e0bfc8960a67b01c80a9f16d6778f374b6944ade03a020415ce8b8ab2593b0f5e787059c8cf90af798290b4c826785d41955092f6e12e7486
2023-02-28 16:53:02 +00:00
Andrew Chow
8303f11e10 Merge bitcoin/bitcoin#27170: refactor: Stop using gArgs global in system.cpp
9a9d5da11f refactor: Stop using gArgs global in system.cpp (Ryan Ofsky)
b20b34f5b3 refactor: Use new GetConfigFilePath function (Ryan Ofsky)

Pull request description:

  Most of the code in `util/system.cpp` that was hardcoded to use the global `ArgsManager` instance `gArgs` has been changed to stop using it (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a few hardcoded references to `gArgs` remain. This commit removes the last ones so these functions aren't reading or writing global state.

  Noticed these `gArgs` references while reviewing #27073

ACKs for top commit:
  achow101:
    ACK 9a9d5da11f
  stickies-v:
    ACK 9a9d5da11
  willcl-ark:
    tACK 9a9d5da11

Tree-SHA512: 2c74b0d5fc83e9ed2ec6562eb26ec735512f75db8876a11a5d5f04e6cdbe0cd8beec19894091aa2cbf29319194d2429ccbf8036f5520ecc394f6fe89a0079a7b
2023-02-28 11:01:21 -05:00
fanquake
9384536eb3 Merge bitcoin/bitcoin#27174: ci: bump lint task to bookworm for git v2.38
a984beeca1 [ci] change lint to bookworm for git v2.38 (glozow)

Pull request description:

  Since 5497c14, verify-commits.py uses `git merge-tree` which requires git v2.38 or later. Fix the lint jobs on master (e.g. https://cirrus-ci.com/task/4971007513985024).

ACKs for top commit:
  achow101:
    ACK a984beeca1
  hebasto:
    re-ACK a984beeca1

Tree-SHA512: dd50598aefb6f9c86cf221dea27fcc521e335cb182f7a3abcb3215d3991794354085be3e07c133ab74ca8b957e3a6acc43722899165957b3d898867d6253ebdc
2023-02-28 15:44:19 +00:00
fanquake
c37fb251f5 Merge bitcoin/bitcoin#27176: docs: GetDataDirNet and GetDataDirBase don't create datadir
fb0dbe9423 docs: GetDataDirNet and GetDataDirBase don't create datadir (stickies-v)

Pull request description:

  Since #27073, the behaviour of `GetDataDir()` [changed](https://github.com/bitcoin/bitcoin/pull/27073/files#diff-19427b0dd1a791adc728c82e88f267751ba4f1c751e19262cac03cccd2822216L435-L443) to only return the datadir path, but not create it if non-existent. This also changed the behaviour of `GetDataDirNet()` and `GetDataDirBase()` but the docs do not yet reflect that.

ACKs for top commit:
  TheCharlatan:
    ACK fb0dbe9423
  theStack:
    ACK fb0dbe9423
  willcl-ark:
    ACK fb0dbe942

Tree-SHA512: 3f10f4871df59882f3649c6d3b2362cae2f8a01ad0bd0c636c5608b0d177d279a2e8712930b819d6d3912e91fa6447b9e54507c33d8afe427f7f39002b013bfb
2023-02-28 15:34:23 +00:00
Andrew Chow
bb136aaf2c Merge bitcoin/bitcoin#26533: prune: scan and unlink already pruned block files on startup
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
77557dda4a prune: scan and unlink already pruned block files on startup (Andrew Toth)

Pull request description:

  There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.
  1. If we call `FindFilesToPrune` or `FindFilesToPruneManual` and crash before `UnlinkPrunedFiles`.
  2. If on Windows there is an open file handle to the file somewhere else when calling `fs::remove` in `UnlinkPrunedFiles` (https://en.cppreference.com/w/cpp/filesystem/remove, https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew#remarks). This could be from another process, or if we are calling `ReadBlockFromDisk`/`ReadRawBlockFromDisk` without having a lock on `cs_main` (which has been allowed since ccd8ef65f9).

  This PR mitigates this by scanning all pruned block files on startup after `LoadBlockIndexDB` and unlinking them again.

ACKs for top commit:
  achow101:
    ACK 3141eab9c6
  pablomartin4btc:
    re-ACK with added functional test 3141eab9c6.
  furszy:
    Code review ACK 3141eab9
  theStack:
    Code-review ACK 3141eab9c6

Tree-SHA512: 6c73bc57838ad1b7e5d441af3c4d6bf4c61c4382e2b86485e57fbb74a61240710c0ceeceb8b4834e610ecfa3175c6955c81ea4b2285fee11ca6383f472979d8d
2023-02-28 09:54:10 -05:00
glozow
a984beeca1 [ci] change lint to bookworm for git v2.38
Since 5497c14, verify-commits.py requires git merge-tree which is only
available in git v2.38 or later.
2023-02-28 14:42:48 +00:00
Pieter Wuille
56e37e71a2 Make miniscript fuzzers avoid script size limit
Use the same technique as is using in the FromString miniscript parser to
predict the final script size of the miniscript being generated in the
miniscript_stable and miniscript_smart fuzzers (by counting every unexplored
sub node as 1 script byte, which is possible because every leaf node always
adds at least 1 byte). This allows bailing out early if the script being
generated would exceed the maximum allowed size (before actually constructing
the miniscript, as that may happen only significantly later potentially).

Also add a self-check to make sure this predicted script size matches that
of generated scripts.
2023-02-28 09:42:33 -05:00
Pieter Wuille
bcec5ab4ff Make miniscript fuzzers avoid ops limit
Keep track of the total number of ops the constructed script will have
during miniscript_stable and miniscript_smart fuzzers' GenNode, so it
can abort early if the 201 ops limit would be exceeded.

Also add a self-check that the final constructed node has the predicted
ops size limit, so we know the fuzzer's logic for keeping track of this
is correct.
2023-02-28 09:22:42 -05:00
Pieter Wuille
213fffa513 Enforce type consistency in miniscript_stable fuzz test
Add a self-check to the fuzzer that the constructed types match the expected
types in the miniscript_stable fuzzer too.
2023-02-28 09:22:42 -05:00
Pieter Wuille
e1f30414c6 Simplify miniscript fuzzer NodeInfo struct
Since we now keep track of all expected child node types (even if rudimentary)
in both miniscript_stable and miniscript_smart fuzzers, there is no need anymore
for the former shortcut NodeInfo constructors without sub types.
2023-02-28 09:22:42 -05:00
Pieter Wuille
5abb0f5ac3 Do base type propagation in miniscript_stable fuzzer
Keep track of which base type (B, K, V, or W) is desired in the miniscript_stable
ConsumeStableNode function. This allows aborting early if the constructed node
won't have the right type.

Note that this does not change the fuzzer format; the meaning of inputs in
ConsumeStableNode is unmodified. The only change is that often the fuzzer will
abort early.

The direct motivation is preventing recursing v: wrappers, which are the only
fragment type that does not otherwise increase the overall minimum possible script
size. In a later commit this will be exploited to prevent overly-large scripts from
being constructed.
2023-02-28 09:22:42 -05:00
stickies-v
fb0dbe9423 docs: GetDataDirNet and GetDataDirBase don't create datadir
Since #27073, the behaviour of GetDataDir changed to only return
the datadir path, but not create it. This also changed the behaviour
of GetDataDirNet and GetDataDirBase but the docs do not yet reflect
that.
2023-02-28 12:52:42 +00:00
fanquake
2c9eb4afe1 guix: use cmake-minimal over cmake 2023-02-28 12:15:18 +00:00
fanquake
1475515312 guix: use coreutils-minimal over coreutils 2023-02-28 12:14:52 +00:00
fanquake
4445621415 guix: use bash-minimal over bash 2023-02-28 12:14:51 +00:00
fanquake
29b62c01c8 valgrind: remove libsecp256k1 suppression 2023-02-28 10:45:57 +00:00
fanquake
519ec2650e Merge bitcoin/bitcoin#27157: init: Return ChainstateLoadStatus::INTERRUPTED when verification was interrupted.
c5825e14f8 doc: add explanation for fail_on_insufficient_dbcache (Ryan Ofsky)
7dff7da4f5 init: Return more fitting ChainStateLoadStatus if verification was interrupted (Martin Zumsande)

Pull request description:

  This addresses two outstanding comments by ryanofsky from #25574:
  * return `ChainstateLoadStatus::INTERRUPTED` instead of `ChainstateLoadStatus::SUCCESS`  if verification was stopped by an interrupt. This would coincide with straightforward expectation, and it avoids a misleading [log entry](c5825e14f8/src/init.cpp (L1526)) in `init` for the block index load time (because that would include the verificiation, which didn't complete). It shouldn't affect node behavior otherwise because the shutdown signal would be caught in init anyway. In test, this would lead to an assert ([link](c5825e14f8/src/test/util/setup_common.cpp (L230))), which also makes more sense because benign interrupts are not expected there during init.
  This can be tested by setting a large value for `-checkblocks`, interrupting the node during block verification and observing the log.
   https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110050930
  * add documentation for `require_full_verification` https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110031541

ACKs for top commit:
  MarcoFalke:
    thanks lgtm ACK c5825e14f8

Tree-SHA512: ca1c71a1b046d30083337dd9ef6d52e66fa1ac8c4ecd807716e4aa6a894179a81df41caee916fa30997fd6e0b284412a3c8f2919d19c29d826fb580ffb89fd73
2023-02-28 10:40:24 +00:00
fanquake
e60a58f191 Merge bitcoin/bitcoin#27118: depends: harden libevent
ff4a73aea2 depends: use FORTIFY_SOURCE=3 with libevent (fanquake)

Pull request description:

  Use `FORTIFY_SOURCE=3` when building libevent in depends. I've upstreamed a change to switch libevent from using =2 to =3 as well: https://github.com/libevent/libevent/pull/1418.

  Solves half of #27038, by giving us some fortified funcs in `bitcoin-cli`.

ACKs for top commit:
  TheCharlatan:
    ACK ff4a73aea2

Tree-SHA512: eaf692ec92b288f0cb524c011fc81529f58efa4c43d418a7b3ae7108eba2bccba708a81a28ac6d063267be80ca615637c6e3fccc02497d7367af2eaae0e8d812
2023-02-28 10:28:34 +00:00
Ryan Ofsky
9a9d5da11f refactor: Stop using gArgs global in system.cpp
Most of the code in util/system.cpp that was hardcoded to use the global
ArgsManager instance `gArgs` has been changed to work with explicit ArgsManager
instances (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a
few hardcoded references to `gArgs` remain. This commit removes the last ones
so these functions aren't reading or writing global state.
2023-02-27 14:21:13 -05:00
Ryan Ofsky
b20b34f5b3 refactor: Use new GetConfigFilePath function
New function was introduced by willcl-ark <will@256k1.dev> in commit
56e370fbb9 from
https://github.com/bitcoin/bitcoin/pull/27073 and removes some duplicate code.
2023-02-27 14:14:58 -05:00
Andrew Chow
710cab1d43 Merge bitcoin/bitcoin#26032: wallet: skip R-value signature grinding for external signers
807de2cebd wallet: skip R-value grinding for external signers (Sjors Provoost)
72b763e452 wallet: annotate bools in descriptor SPKM FillPSBT() (Sjors Provoost)

Pull request description:

  When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.

  In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.

  Suggested testing:
  1. On master, launch with `-signet` and create an external signer wallet using e.g. a Trezor and HWI, see [guide](https://github.com/bitcoin/bitcoin/blob/master/doc/external-signer.md#example-usage) (with the GUI it should "just work" once you have the HWI path configured).
  2. Create a few addresses and fund them from the faucet: https://signet.bc-2.jp/ (wait for confirmation)
  3. Create another address, and now send the entire wallet to it, set the fee to 1 sat/byte
  4. Most likely this transaction never gets broadcast and you won't see it on the [signet explorer](https://explorer.bc-2.jp)

  5. With this PR, try again.
  6. Check the explorer and inspect the transaction. Each input witness starts with either `30440220` (R has 32 bytes) or `30440221` (R has 33 bytes). See this explainer for [DER encoding](https://bitcoin.stackexchange.com/questions/92680/what-are-the-der-signature-and-sec-format).

  Fixes #26030

ACKs for top commit:
  S3RK:
    ACK 807de2cebd
  achow101:
    ACK 807de2cebd
  furszy:
    ACK 807de2ce
  ishaanam:
    utACK 807de2cebd

Tree-SHA512: 64f626a3030ef0ab1e43af86d8fba113151512561baf425e6e5182af53df3a64fa9e85c7f67bf4ed15b5ad6e5d5afc7fbba8b6e1f3bad388e48db51cb9446074
2023-02-27 12:37:46 -05:00
fanquake
82793f1984 Merge bitcoin/bitcoin#27146: Fix various libbitcoinkernel DLL build problems
5da7c0b3e3 build: allow libitcoinkernel dll builds now that exports are fixed (Cory Fields)
130490aef9 build: always build bitcoin-chainstate against static libbitcoinkernel (Cory Fields)
545a74ef32 build: fix bitcoin-chainstate when libbitcoinkernel is static (Cory Fields)
9c253d2398 build: don't define DLL_EXPORT for windows (Cory Fields)

Pull request description:

  Fixes #25008.
  Fixes #19772.

  1. Fixup the build defines so that exports are clean.
  2. Work around a libtool issue wrt dependency calculation
  3. Simplify everything by only ever building in-tree bitcoin-chainstate against a static libbitcoinkernel
  4. Remove Windows-only hack that disabled dll creation

ACKs for top commit:
  TheCharlatan:
    ACK 5da7c0b3e3

Tree-SHA512: 61bab457e13842946387240da703d313509af30d4ca3371a19a26a5ef1716e4d7107b09567323041b549ab1fc97a064aa1d6992406936ab9c491a616bc7f4e7f
2023-02-27 14:41:47 +00:00
fanquake
a2877f7ad3 Merge bitcoin/bitcoin#25227: Handle invalid hex encoding in ParseHex
faab273e06 util: Return empty vector on invalid hex encoding (MarcoFalke)
fa3549a77b test: Add hex parse unit tests (MarcoFalke)

Pull request description:

  Seems a bit confusing to happily accept random bytes and pretend they are hex encoded strings.

ACKs for top commit:
  stickies-v:
    re-ACK faab273e06

Tree-SHA512: a808135f744f50aece03d4bf5a71481c7bdca1fcdd0d5b113abdb0c8b382bf81cafee6d17c239041fb49b59f4e19970f24a475378e7f711c3a47d6438de2bdab
2023-02-27 14:27:50 +00:00
Hennadii Stepanov
9172cc672e qt: Update translation source file
The diff is produced by running `make -C src translate`.
2023-02-27 14:07:19 +00:00
Hennadii Stepanov
7b0cbf444d qt: Bump Transifex slug for 25.x 2023-02-27 14:01:14 +00:00
Hennadii Stepanov
369023d22d qt: Periodic translation updates from Transifex
Pulled from 24.x resource.
Changes to "de", "es_MX" and "nl" have been ignored as they remove
translations altogether.
2023-02-27 13:53:29 +00:00
glozow
873dcc1910 Merge bitcoin/bitcoin#27058: contrib: Improve verify-commits.py to work with maintainers leaving
14fac808bd verify-commits: Mention git v2.38.0 requirement (Andrew Chow)
bb86887527 verify-commits: Skip checks for commits older than trusted roots (Andrew Chow)
5497c14830 verify-commits: Use merge-tree in clean merge check (Andrew Chow)
76923bfa09 verify-commits: Remove all allowed commit exceptions (Andrew Chow)
53b07b2b47 verify-commits: Move trusted-keys valid sig check into verify-commits itself (Andrew Chow)

Pull request description:

  Currently the `verify-commits.py` script does not work well with maintainers giving up their commit access. If a key is removed from `trusted-keys`, any commits it signed previously will fail to verify, however keys cannot be kept in the list as it would allow that person to continue to push new commits. Furthermore, the `trusted-keys` used depends on the working tree which `verify-commits.py` itself may be modifying. When the script is run, the `trusted-keys` may be the one that is intended to be used, but the script may change the tree to a different commit with a different `trusted-keys` and use that instead!

  To resolve these issues, I've updated `verify-commits.py` to load the `trusted-keys` file and check the keys itself rather than delegating that to `gpg.sh` (which previously read in `trusted-keys`). This avoids the issue with the tree changing.

  I've also updated the script so that it stops modifying the tree. It would do this for the clean merge check where it would checkout each individual commit and attempt to reapply the merges, and then checking out the commit given as a cli arg. `git merge-tree` lets us do basically that but without modifying the tree. It will give us the object id for the resulting tree which we can compare against the object id of the tree in the merge commit in question. This also appears to be quite a bit faster.

  Lastly I've removed all of the exception commits in `allow-revsig-commits`, `allow-incorrect-sha512-commits`, and `allow-unclean-merge-commits` since all of these predate the commits in `trusted-git-root` and `trusted-sha512-root`. I've also updated the script to skip verification of commits that predate `trusted-git-root`, and skip sha512 verification for those that predate `trusted-sha512-root`.

ACKs for top commit:
  Sjors:
    ACK 14fac808bd
  glozow:
    Concept ACK 14fac808bd

Tree-SHA512: f9b0c6e1f1aecb169cdd6c833b8871b15e31c2374dc589858df0523659b294220d327481cc36dd0f92e9040d868eee6a8a68502f3163e05fa751f9fc2fa8832a
2023-02-27 13:17:48 +00:00
MarcoFalke
faab273e06 util: Return empty vector on invalid hex encoding 2023-02-27 13:39:55 +01:00
MarcoFalke
fa3549a77b test: Add hex parse unit tests 2023-02-27 13:35:51 +01:00
Ryan Ofsky
c5825e14f8 doc: add explanation for fail_on_insufficient_dbcache 2023-02-24 15:11:27 -05:00
Martin Zumsande
7dff7da4f5 init: Return more fitting ChainStateLoadStatus if verification was interrupted
This also avoids a misleading block index loadtime log entry in init.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-02-24 15:09:24 -05:00
glozow
be2e748f37 Merge bitcoin/bitcoin#27154: doc: mention sanitizer suppressions in developer docs
84ca5b349e doc: mention sanitizer suppressions in developer docs (fanquake)

Pull request description:

  Should be enough to close #17834.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 84ca5b349e

Tree-SHA512: 233c688a3cef1006c9a00f7b7a52fd6ee0ec150367e5e56904b6f1bbdca21b9217c69f8fcf653a4943613d12c3178a39f761b25eb24fc1954a563cfb1f832f5e
2023-02-24 13:58:04 +00:00
fanquake
84ca5b349e doc: mention sanitizer suppressions in developer docs
Should be enough to close #17834.
2023-02-24 12:16:37 +00:00
glozow
6758bd7de7 Merge bitcoin/bitcoin#27135: Remove MarcoFalke fingerprint, update trusted-git-root
fab17f08e2 Revert "[contrib] verify-commits: Add MarcoFalke fingerprint" (MarcoFalke)

Pull request description:

  This reverts commit fa24329334.

  The commit may be signed by my key, but I haven't checked it. Also, I haven't checked the new `contrib/verify-commits/trusted-git-root`.

ACKs for top commit:
  achow101:
    ACK fab17f08e2
  glozow:
    ACK fab17f08e2

Tree-SHA512: 485fb302f7e42704412afffd6c09a031f63df18f259b27282b8373d5bf95b0ec72426cec476d88bf23e793a6e1dae4c1df2059645961806e34b50448ebf1862a
2023-02-24 10:27:25 +00:00
fanquake
cb25c21921 Merge bitcoin/bitcoin#27151: util: Remove duplicate include
e8462690a9 util: Remove duplicate include (Andrew Chow)

Pull request description:

  Duplicate `#include <utility>` is upsetting the linter.

ACKs for top commit:
  davidgumberg:
    ACK e8462690a9
  theStack:
    ACK e8462690a9
  john-moffett:
    ACK e8462690a9

Tree-SHA512: 9e45d8f6a2dd5efcb8eb1c3c440d94b16490dbd63255784cb39863767fa07227e06da112a150ef337ef89e2e305b60b00d5b1c12ff7e1e9c02f6648ed97fac8c
2023-02-24 09:03:02 +00:00
Andrew Chow
e8462690a9 util: Remove duplicate include
Duplicate `#include <utility>` is upsetting the linter.
2023-02-23 17:58:40 -05:00
Andrew Chow
1258af40c0 Merge bitcoin/bitcoin#27073: Convert ArgsManager::GetDataDir to a read-only function
64c105442c util: make GetDataDir read-only & create datadir.. (willcl-ark)
56e370fbb9 util: add ArgsManager datadir helper functions (willcl-ark)

Pull request description:

  Fixes #20070

  Currently `ArgsManager::GetDataDir()` ensures it will always return a datadir by creating one if necessary. The function is shared between `bitcoind` `bitcoin-qt` and `bitcoin-cli` which results in the undesirable behaviour described in #20070.

  This PR splits out the part of the function which creates directories and adds it as a standalone function, only called as part of `bitcoind` and `bitcoin-qt` init, but not `bitcoin-cli`.

  `ReadConfigFiles`' behavior is changed to use the absolute path of the config file in error and warning messages instead of a relative path.

  This was inadvertantly the form being tested [here](73966f75f6/test/functional/feature_config_args.py (L287)), whilst we were _not_ testing that a relative path was returned by the message even though we passed a relative path in as argument.

ACKs for top commit:
  achow101:
    ACK 64c105442c
  hebasto:
    re-ACK 64c105442c, only comments have been adjusted as requsted since my previous [review](https://github.com/bitcoin/bitcoin/pull/27073#pullrequestreview-1307435890).
  TheCharlatan:
    Re-ACK 64c105442c
  ryanofsky:
    Code review ACK 64c105442c. Only comment changes since last review

Tree-SHA512: b129501346071ad62551c9714492b21536d0558a94117d97218e255ef4e948d00df899a4bc2788faea27d3b1f20fc6136ef9d03e6a08498d926d9ad8688d6c96
2023-02-23 16:41:14 -05:00
Andrew Chow
c033720b2b Merge bitcoin/bitcoin#16195: util: Use void* throughout support/lockedpool.h
f36d1d5b89 Use void* throughout support/lockedpool.h (Jeffrey Czyz)

Pull request description:

  Replace uses of char* with void* in Arena's member variables. Instead,
  cast to char* where needed in the implementation.

  Certain compiler environments disallow std::hash<char*> specializations
  to prevent hashing the pointer's value instead of the string contents.
  Thus, compilation fails when std::unordered_map is keyed by char*.

  Explicitly using void* is a workaround in such environments. For
  consistency, void* is used throughout all member variables similarly to
  the public interface.

  Changes to this code are covered by src/test/allocator_tests.cpp.

ACKs for top commit:
  achow101:
    ACK f36d1d5b89
  theStack:
    Code-review ACK f36d1d5b89
  jonatack:
    ACK f36d1d5b89 review, debug build, unit tests, checked clang 15 raises "error: arithmetic on a pointer to void"  without the conversions here from the generic void* pointer back to char*

Tree-SHA512: f9074e6d29ef78c795a512a6e00e9b591e2ff34165d09b73eae9eef25098c59e543c194346fcd4e83185a39c430d43744b6f7f9d1728a132843c67bd27ea5189
2023-02-23 15:44:42 -05:00
Andrew Chow
b7702bd546 Merge bitcoin/bitcoin#25943: rpc: Add a parameter to sendrawtransaction which sets a maximum value for unspendable outputs.
7013da07fb Add release note for PR#25943 (David Gumberg)
04f270b435 Add test for unspendable transactions and parameter 'maxburnamount' to sendrawtransaction. (David Gumberg)

Pull request description:

  This PR adds a user configurable, zero by default parameter — `maxburnamount` — to `sendrawtransaction`. This PR makes bitcoin core reject transactions that contain unspendable outputs which exceed `maxburnamount`.  closes #25899.

  As a result of this PR, `sendrawtransaction` will by default block 3 kinds of transactions:

  1. Those that begin with `OP_RETURN` - (datacarriers)
  2. Those whose lengths exceed the script limit.
  3. Those that contain invalid opcodes.

  The user is able to configure a `maxburnamount` that will override this check and allow a user to send a potentially unspendable output into the mempool.

  I see two legitimate use cases for this override:
  1. Users that deliberately use `OP_RETURN` for datacarrier transactions that embed data into the blockchain.
  2.  Users that refuse to update, or are unable to update their bitcoin core client would be able to make use of new opcodes that their client doesn't know about.

ACKs for top commit:
  glozow:
    reACK 7013da07fb
  achow101:
    re-ACK 7013da07fb

Tree-SHA512: f786a796fb71a587d30313c96717fdf47e1106ab4ee0c16d713695e6c31ed6f6732dff6cbc91ca9841d66232166eb058f96028028e75c1507324426309ee4525
2023-02-23 13:57:38 -05:00
Sjors Provoost
807de2cebd wallet: skip R-value grinding for external signers
When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.

In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.

This commit also  drops the nullptr default for CCoinControl arguments for functions that it touches. This is because having a boolean argument right next to an optional pointer is error prone.

Co-Authored-By: S3RK <1466284+S3RK@users.noreply.github.com>
2023-02-23 18:30:12 +01:00
Sjors Provoost
72b763e452 wallet: annotate bools in descriptor SPKM FillPSBT() 2023-02-23 11:46:29 +01:00
fanquake
32f9ce0f52 Merge bitcoin/bitcoin#27124: docs: add ramdisk guide for running tests on OSX
2f84ad7b9e docs: add ramdisk guide for running tests on OSX (Matthew Zipkin)

Pull request description:

  Using a ramdisk on OSX sped up the test suite by about 5x (using default `jobs=4`) on my M1 macbook pro running macOS Monterey 12.3.1. This PR adds the relevant OSX commands following the Linux directions.

  Default:
  ```
  8204 s (accumulated)
  Runtime: 2104 s
  ```

  following commands from the PR:
  ```
  1606 s (accumulated)
  Runtime: 421 s
  ```

  ramdisk + `jobs=32`:
  ```
  2090 s (accumulated)
  Runtime: 85 s
  ```

ACKs for top commit:
  jonatack:
    ACK 2f84ad7b9e
  willcl-ark:
    ACK 2f84ad7b9e
  brunoerg:
    utACK 2f84ad7b9e

Tree-SHA512: 37a9903c8ac2cbfaa91e7e73fc96ef65042ff4b15763d452af7b8615255adf03429ad01cf85265a99dd569290c1d69c05a393d616868c05c190b60b053820786
2023-02-23 10:04:37 +00:00
willcl-ark
64c105442c util: make GetDataDir read-only & create datadir..
.. only in bitcoind and bitcoin-qt

This changes behaviour of GetConfigFilePath which now always returns the
absolute path of the provided -conf argument.
2023-02-23 08:38:35 +00:00
willcl-ark
56e370fbb9 util: add ArgsManager datadir helper functions
* Add ArgsManager::EnsureDataDir()
  Creates data directory if it doesn't exist
* Add ArgsManager::GetConfigFilePath()
  Return config file path (read-only)
2023-02-23 08:37:27 +00:00
Cory Fields
5da7c0b3e3 build: allow libitcoinkernel dll builds now that exports are fixed
Symbol visibility issues are not actually fixed yet because we have not yet
defined an api and exported symbols, but everything is now in place for that.
2023-02-22 21:23:10 +00:00
Cory Fields
130490aef9 build: always build bitcoin-chainstate against static libbitcoinkernel
Building binaries against our uninstalled shared libs is impractical. Instead,
to test them, we'll need to work on a runtime shared-lib execution harness.
2023-02-22 21:23:10 +00:00
Cory Fields
545a74ef32 build: fix bitcoin-chainstate when libbitcoinkernel is static
Libtool is unable to calculate dependencies correctly so give it some help.
2023-02-22 20:56:11 +00:00
Andrew Chow
832fa2d238 Merge bitcoin/bitcoin#25574: validation: Improve error handling when VerifyDB dosn't finish successfully
0af16e7134 doc: add release note for #25574 (Martin Zumsande)
57ef2a4812 validation: report if pruning prevents completion of verification (Martin Zumsande)
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache (Martin Zumsande)
d6f781f1cf validation: return VerifyDBResult::INTERRUPTED if verification was interrupted (Martin Zumsande)
6360b5302d validation: Change return value of VerifyDB to enum type (Martin Zumsande)

Pull request description:

  `VerifyDB()` can fail to complete due to insufficient dbcache at the level 3 checks. This PR improves the error handling in this case in the following ways:
  - The rpc `-verifychain` now returns false if the check can't be completed due to insufficient cache
  - During init, we only log a warning if the default values for `-checkblocks` and `-checklevel` are taken and the check doesn't complete. However, if the user actively specifies one of these args, we return with an InitError if we can't complete the check.

  This PR also changes `-verifychain` RPC to return `false` if the verification didn't finish due to missing block data (pruning) or due to being interrupted by the node being shutdown.

  Previously, this PR also included a fix for a possible assert during verification - this was done in #27009 (now merged).

ACKs for top commit:
  achow101:
    ACK 0af16e7134
  ryanofsky:
    Code review ACK 0af16e7134. Only small suggested changes since the last review, like renaming some of the enum values. I did leave more suggestions, but they are not very important and could be followups
  john-moffett:
    ACK 0af16e7134
  MarcoFalke:
    lgtm re-ACK 0af16e7134 🎚

Tree-SHA512: 84b4f767cf9bfbafef362312757c9bf765b41ae3977f4ece840e40c52a2266b1457832df0cdf70440be0aac2168d9b58fc817238630b0b6812f3836ca950bc0e
2023-02-22 14:19:44 -05:00
Cory Fields
9c253d2398 build: don't define DLL_EXPORT for windows
This fixes libbitcoinkernel dll linking.
2023-02-22 19:04:21 +00:00
fanquake
9f6ef0c156 Merge bitcoin/bitcoin#27143: test: Replace 0xC0 constant
c3b4b5a142 test: Replace 0xC0 constant (roconnor-blockstream)

Pull request description:

  Instead it should be the named constant `LEAF_VERSION_TAPSCRIPT`.

ACKs for top commit:
  instagibbs:
    ACK c3b4b5a142
  theStack:
    ACK c3b4b5a142

Tree-SHA512: c00be584ea2d0e7c01bf5620da0da1f37e5b5298ef95df48d91d137c8c542f5d91be158d45392cf2ba8874bf27bd12924e2eed395773b49d091e3028de3356a2
2023-02-22 18:11:26 +00:00
Matthew Zipkin
2f84ad7b9e docs: add ramdisk guide for running tests on OSX 2023-02-22 13:04:23 -05:00
Andrew Chow
5e55534586 Merge bitcoin/bitcoin#27068: wallet: SecureString to allow null characters
4bbf5ddd44 Detailed error message for passphrases with null chars (John Moffett)
b4bdabc223 doc: Release notes for 27068 (John Moffett)
4b1205ba37 Test case for passphrases with null characters (John Moffett)
00a0861181 Pass all characters to SecureString including nulls (John Moffett)

Pull request description:

  `SecureString` is a `std::string` specialization with a secure allocator. However, in practice it's treated like a C- string (no explicit length and null-terminated). This can cause unexpected and potentially insecure behavior. For instance, if a user enters a passphrase with embedded null characters (which is possible through Qt and the JSON-RPC), it will ignore any characters after the first null, potentially giving the user a false sense of security.

  Instead of assigning to `SecureString` via `std::string::c_str()`, assign it via a `std::string_view` of the original. This explicitly captures the size and still doesn't make any extraneous copies in memory.

  Note to reviewers, the following all compile identically in recent `GCC` (x86-64 and ARM64) with `-O2` (and `-std=c++17`):

  ```C++
  std::string orig_string;
  std::cin >> orig_string;
  SecureString s;
  s.reserve(100);
  // The following all compile identically
  s = orig_string;
  s = std::string_view{orig_string};
  s.assign(std::string_view{orig_string});
  s.assign(orig_string.data(), orig_string.size());
  ```

  So it's largely a matter of preference. However, one thing to keep in mind is that we want to avoid making unnecessary copies of any sensitive data in memory.

  Something like `SecureString s{orig_string};` is still invalid and probably unwanted in our case, since it'd get treated as a short string and optimized away from the secure allocator. I presume that's the reason for the `reserve()` calls.

  Fixes #27067.

ACKs for top commit:
  achow101:
    re-ACK 4bbf5ddd44
  stickies-v:
    re-ACK [4bbf5dd](4bbf5ddd44)
  furszy:
    utACK 4bbf5ddd

Tree-SHA512: 47a96905a82ca674b18076a20a388123beedf70e9de73e42574ea68afbb434734e56021835dd9b148cdbf61709926b487cc95e9021d9bc534a7c93b3e143d2f7
2023-02-22 13:02:16 -05:00
fanquake
174f022f68 Merge bitcoin/bitcoin#27144: kernel: add missing include
49d01f32c9 kernel: add missing include (Cory Fields)

Pull request description:

  This syncs the cs_main definition/declaration.

  Noticed when experimenting with the external visibility of `cs_main`.

  Specifically, this is needed for the following to work as intended:
  ```c++
  __attribute__ ((visibility ("default"))) extern RecursiveMutex cs_main;
  ```

ACKs for top commit:
  fanquake:
    ACK 49d01f32c9

Tree-SHA512: ea0dbcf81959566f949d76c7dcd1e33de53e613519500c863bfb0ac8209665b1c12cff2daa7890d03b76debc4d046339ee7b3231adb71b128e9d5a8fa3132b6c
2023-02-22 18:01:48 +00:00
fanquake
30874a7cc9 Merge bitcoin/bitcoin#26837: I2P network optimizations
3c1de032de i2p: use consistent number of tunnels with i2pd and Java I2P (Vasil Dimov)
801b405f85 i2p: lower the number of tunnels for transient sessions (Vasil Dimov)
b906b64eb7 i2p: reuse created I2P sessions if not used (Vasil Dimov)

Pull request description:

  * Reuse an I2P transient session instead of discarding it if we failed to connect to the desired peer. This means we never used the generated address (destination), whose creation is not cheap. This does not mean that we will use the same address for more than one peer.
  * Lower the number of tunnels for transient sessions.
  * Explicitly specify the number of tunnels for persistent sessions instead of relying on the defaults which differ between I2P routers. This way we get consistent behavior with all routers.

  Alleviates: https://github.com/bitcoin/bitcoin/issues/26754

  (I have not tested this with i2pd, yet)

ACKs for top commit:
  jonatack:
    ACK 3c1de032de
  mzumsande:
    Light ACK 3c1de032de

Tree-SHA512: 477b4b9a5755e6a9a46bc0f7b268fa419dff4414e25445c750ae913f7552d9e2313f2aca4e3b70067b8390c2d0c2d68ec459f331765e939fc84139e454031cd4
2023-02-22 17:58:41 +00:00
fanquake
c6e65a102c Merge bitcoin/bitcoin#27137: test: Raise PRNG seed log to INFO
4d84eaec82 Raise PRNG seed log to INFO. (roconnor-blockstream)

Pull request description:

  Some build infrastructure, such as Nix, will delete failed builds by default, keeping only the log (stdout/stderr) of the failed build.

  For flaky tests, it would be very helpful to have the PRNG seed in the default log in order to redo the failed test.

  By simply raising the PRNG seed logging to INFO, we can, by default, record the seed in the log of every build.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4d84eaec82
  theStack:
    ACK 4d84eaec82

Tree-SHA512: 3ccb4a4e7639a3babc3b2a6456a6d0bffc090da34e4545b317f7bfbed4e9950d1b38ea5b2a90c37ccb49b3454bdeff03a6aaf86770b9c4dd14b26320aba50b94
2023-02-22 17:51:39 +00:00
fanquake
63893d5eab Merge bitcoin/bitcoin#26595: wallet: be able to specify a wallet name and passphrase to migratewallet
9486509be6 wallet, rpc: Update migratewallet help text for encrypted wallets (Andrew Chow)
aaf02b5721 tests: Tests for migrating wallets by name, and providing passphrase (Andrew Chow)
7fd125b27d wallet: Be able to unlock the wallet for migration (Andrew Chow)
6bdbc5ff59 rpc: Allow users to specify wallet name for migratewallet (Andrew Chow)
dbfa345403 wallet: Allow MigrateLegacyToDescriptor to take a wallet name (Andrew Chow)

Pull request description:

  `migratewallet` currently operates on wallets that are already loaded, however this is not necessarily required, and in the future, not possible once the legacy wallet is removed. So we need to also be able to give the wallet name to migrate.

  Additionally, the passphrase is required when migrating a wallet. Since a wallet may not be loaded when we migrate, and as we currently unload wallets when migrating, we need the passphrase to be given to `migratewallet` in order to migrate encrypted wallets.

  Fixes #27048

ACKs for top commit:
  john-moffett:
    reACK 9486509be6
  pinheadmz:
    ACK 9486509be6
  furszy:
    ACK 9486509b

Tree-SHA512: 35e2ba69a148e129a41e20d7fb99c4cab7947b1b7e7c362f4fd06ff8ac6e79e476e07207e063ba5b80e1a33e2343f4b4f1d72d7930ce80c34571c130d2f5cff4
2023-02-22 17:48:23 +00:00
James O'Beirne
f2a4f3376f move-only-ish: init: factor out chainstate initialization
Moves chainstate initialization into its own function. This is
necessary to later support a more readable way of handling
background-validation chainstate cleanup during init, since the
chainstate initialization functions may need to be repeated after
moving leveldb filesystem content around.

This commit isn't strictly necessary, but the alternative is to (ab)use
the `while` loop in init.cpp with a `continue` on the basis of a
specific ChainstateLoadingError return value from LoadChainstate. Not
only is this harder to read, but it can't be unittested.

The approach here lets us consolidate background-validation cleanup to
LoadChainstate, and therefore exercise it within tests.

This commit is most easily reviewed with

  git diff --color-moved=dimmed-zebra
  --color-moved-ws=ignore-space-change
2023-02-22 12:13:26 -05:00
James O'Beirne
637a90b973 add Chainstate::HasCoinsViews()
Used in subsequent commits. Also cleans up asserts in
coins_views-related convenience methods to be more exact.
2023-02-22 12:13:26 -05:00
James O'Beirne
c29f26b47b validation: add CChainState::m_disabled and ChainMan::isUsable
and remove m_snapshot_validated. This state can now be inferred by the
number of isUsable chainstates.

m_disabled is used to signal that a chainstate should no longer be used
by validation logic; it is used as a sentinel when background validation
completes or if the snapshot chainstate is found to be invalid.

isUsable is a convenience method that incorporates m_disabled.
2023-02-22 12:13:11 -05:00
James O'Beirne
5ee22cdafd add ChainstateManager.GetSnapshot{BaseHeight,BaseBlock}()
For use in later commits.
2023-02-22 12:07:25 -05:00
Cory Fields
49d01f32c9 kernel: add missing include
This syncs the cs_main definition/declaration.

Noticed when experimenting with the external visibility of cs_main.
2023-02-22 15:46:21 +00:00
roconnor-blockstream
c3b4b5a142 test: Replace 0xC0 constant
Instead it should be the named constant `LEAF_VERSION_TAPSCRIPT`.
2023-02-22 10:26:07 -05:00
mruddy
c4981e7f63 prune, import: fixes #23852
allows pruning to work during the loadblock import process.
2023-02-22 05:16:28 -05:00
fanquake
8b4dc94734 Merge bitcoin/bitcoin#27117: fuzz: avoid redundant dup key checks when creating Miniscript nodes
c1b7bd047f fuzz: avoid redundant dup key checks when creating Miniscript nodes (Antoine Poinsot)

Pull request description:

  I thought i had done that already in #24149, but it must have slipped through the rebase. It's a 2x speed improvement against the existing corpora and will probably be much more as we extend them with larger nodes.

ACKs for top commit:
  sipa:
    ACK c1b7bd047f

Tree-SHA512: 9e6ceb6254183964b6c5538e21ba6321df95a68acb343a15a6ecfef5c51a1980d2627df5aeef9aef1db41656e18cc4f3bc96e6f24314d12fa60368b04a350001
2023-02-22 09:37:07 +00:00
fanquake
0c579203d2 Merge bitcoin/bitcoin#25867: lint: enable E722 do not use bare except
61bb4e783b lint: enable E722 do not use bare except (Leonardo Lazzaro)

Pull request description:

  Improve test code and enable E722 lint check.

   If you want to catch all exceptions that signal program errors, use except Exception: (bare except is equivalent to except BaseException:).

  Reference: https://peps.python.org/pep-0008/#programming-recommendations

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 61bb4e783b

Tree-SHA512: c7497769d5745fa02c78a20f4a0e555d8d3996d64af6faf1ce28e22ac1d8be415b98e967294679007b7bda2a9fd04031a9d140b24201e00257ceadeb5c5d7665
2023-02-22 09:28:09 +00:00
Andrew Chow
14fac808bd verify-commits: Mention git v2.38.0 requirement 2023-02-21 17:00:13 -05:00
Andrew Chow
9486509be6 wallet, rpc: Update migratewallet help text for encrypted wallets 2023-02-21 15:51:31 -05:00
Andrew Chow
aaf02b5721 tests: Tests for migrating wallets by name, and providing passphrase 2023-02-21 15:51:31 -05:00
John Moffett
4bbf5ddd44 Detailed error message for passphrases with null chars
Since users may have thought the null characters in their
passphrases were actually evaluated prior to this change,
they may be surprised to learn that their passphrases no
longer work. Give them feedback to explain how to remedy
the issue.
2023-02-21 14:53:54 -05:00
John Moffett
b4bdabc223 doc: Release notes for 27068
To reflect the change in behavior.
2023-02-21 14:40:59 -05:00
John Moffett
4b1205ba37 Test case for passphrases with null characters
Add a functional test to make sure the system
properly accepts passphrases with null characters.
2023-02-21 14:40:59 -05:00
John Moffett
00a0861181 Pass all characters to SecureString including nulls
`SecureString` is a `std::string` specialization with
a secure allocator. However, it's treated like a C-
string (no explicit length and null-terminated). This
can cause unexpected behavior. For instance, if a user
enters a passphrase with an embedded null character
(which is possible through Qt and the JSON-RPC), it will
ignore any characters after the null, giving the user
a false sense of security.

Instead of assigning `SecureString` via `std::string::c_str()`,
assign it via a `std::string_view` of the original. This
explicitly captures the size and doesn't make any extraneous
copies in memory.
2023-02-21 14:40:59 -05:00
Andrew Chow
80f4979322 Merge bitcoin/bitcoin#26347: wallet: ensure the wallet is unlocked when needed for rescanning
6a5b348f2e test: test rescanning encrypted wallets (ishaanam)
493b813e17 wallet: ensure that the passphrase is not deleted from memory when being used to rescan (ishaanam)
66a86ebabb wallet: keep track of when the passphrase is needed when rescanning (ishaanam)

Pull request description:

  Wallet passphrases are needed to top up the keypool of encrypted wallets
  during a rescan. The following RPCs need the passphrase when rescanning:
      - `importdescriptors`
      - `rescanblockchain`

  The following RPCs use the information about whether or not the
  passphrase is being used to ensure that full rescans are able to
  take place (meaning the following RPCs should not be able to run
  if a rescan requiring the wallet to be unlocked  is taking place):
      - `walletlock`
      - `encryptwallet`
      - `walletpassphrasechange`

  `m_relock_mutex` is also introduced so that the passphrase is not
  deleted from memory when the timeout provided in
  `walletpassphrase` is up and the wallet is still rescanning.
  Fixes #25702, #11249

  Thanks to achow101 for coming up with the idea of using a new mutex to solve this issue and for answering related questions.

ACKs for top commit:
  achow101:
    ACK 6a5b348f2e
  hernanmarino:
    ACK 6a5b348f2e
  furszy:
    Tested ACK 6a5b348f

Tree-SHA512: 0b6db692714f6f94594fa47249f5ee24f85713bfa70ac295a7e84b9ca6c07dda65df7b47781a2dc73e5b603a8725343a2f864428ae20d3e126c5b4802abc4ab5
2023-02-21 14:02:49 -05:00
Andrew Chow
ad46141602 Merge bitcoin/bitcoin#27122: script: BIP341 txdata cannot be precomputed without spent outputs
95f12de925 BIP341 txdata cannot be precomputed without spent outputs (Pieter Wuille)

Pull request description:

  In `PrecomputedTransactionData::Init`, if `force` is set to `true`, `m_bip341_taproot_ready` is always set to true, suggesting that all its BIP341-relevant members (including `m_spent_amounts_single_hash`) are correct. If however no `spent` array of spent previous `CTxOut`s is provided, some of these members will be incorrect. This option was introduced in #21365.

  That doesn't actually hurt, as without prevout data, it's fundamentally impossible to generate correct BIP341 signatures anyway, and f722a9bd13/src/script/sign.cpp (L71) should prevent the logic from being used anyway.

  Still, don't set `m_bip341_taproot_ready` variable when we clearly don't have enough data to compute it.

  Discovered by Russell O'Connor.

ACKs for top commit:
  ajtowns:
    ACK 95f12de925
  achow101:
    ACK 95f12de925
  instagibbs:
    ACK 95f12de925

Tree-SHA512: 90acd2bfa50a7a0bde75a15a9f6c1f5c40f48fb5b870b1bbc4082777e24a482c8282463ef7d1245e53201dbcb5c196ef0386352f8e380e68cdf00c2111633b77
2023-02-21 13:55:03 -05:00
roconnor-blockstream
4d84eaec82 Raise PRNG seed log to INFO.
Some build infrastructure, such as Nix, will delete failed builds by default, keeping only the log of the failed build.

For flaky tests, it would be very helpful to have the PRNG seed in the default log in order to redo the failed test.

By simply raising the PRNG seed logging to INFO, we can, by default, record the seed in the log of every build.
2023-02-21 12:01:13 -05:00
MarcoFalke
fab17f08e2 Revert "[contrib] verify-commits: Add MarcoFalke fingerprint"
This reverts commit fa24329334.
2023-02-21 14:00:13 +01:00
willcl-ark
3fa1185dda github: Switch to yaml issue templates
These provide more flexibility and can be designed to extract more
information from users when submitting issues.
2023-02-21 11:31:16 +00:00
David Gumberg
7013da07fb Add release note for PR#25943
Co-authored-by: glozow <gloriajzhao@gmail.com>
2023-02-20 11:47:20 -07:00
David Gumberg
04f270b435 Add test for unspendable transactions and parameter 'maxburnamount' to sendrawtransaction.
'maxburnamount' sets a maximum value for outputs heuristically deemed unspendable including datacarrier scripts that begin with `OP_RETURN`.
2023-02-20 11:38:52 -07:00
fanquake
94070029fb Merge bitcoin/bitcoin#27053: wallet: reuse change dest when re-creating TX with avoidpartialspends
14b4921a91 wallet: reuse change dest when recreating TX with avoidpartialspends (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27051

  When the wallet creates a transaction internally, it will also create an alternative that spends using destination groups and see if the fee difference is negligible. If it costs the user the same to send the grouped version, we send it (even if the user has `avoidpartialspends` set to `false` which is default). This patch ensures that the second transaction creation attempt re-uses the change destination selected by the first attempt. Otherwise, the first change address remains reserved, will not be used in the second attempt, and then will never be used by the wallet, leaving gaps in the BIP44 chain.

  If the user had `avoidpartialspends` set to true, there is no second version of the created transaction and the change addresses are not affected.

  I believe this behavior was introduced in https://github.com/bitcoin/bitcoin/pull/14582

ACKs for top commit:
  achow101:
    ACK 14b4921a91

Tree-SHA512: a3d56f251ff4b333fc11325f30d05513e34ab0a2eb703fadd0ad98d167ae074493df1a24068298336c6ed2da6b31aa2befa490bc790bbc260ed357c8f2397659
2023-02-20 17:20:37 +00:00
fanquake
0f670e0eae Merge bitcoin/bitcoin#27127: rpc: fix successful broadcast count in submitpackage error msg
7554b1fd66 rpc: fix successful broadcast count in `submitpackage` error msg (Sebastian Falbesoner)

Pull request description:

  If a `submitpackage` RPC call errors due to any of the individual tx broadcasts failing, the returned error message is supposed to contain the number of successful broadcasts so far:

  4395b7f084/src/rpc/mempool.cpp (L848-L849)

  Right now this is wrongly always shown as zero. Fix this by adding the missing increment of the counter. While touching that area, the variable is also renamed to better reflect its purpose (s/num_submitted/num_broadcast/; the submission has already happened at that point) and named arguments for the `BroadcastTransaction` call are added.

  (Note that the error should be really rare, as all txs have already been submitted succesfully to the mempool. IIUC this code-path could only hit if somehow a tx is being removed from the mempool between `ProcessNewPackage` and the `BroadcastTransaction` calls, e.g. if a new block is received which confirms any of the package's txs.)

ACKs for top commit:
  glozow:
    utACK 7554b1fd66, thanks!

Tree-SHA512: e362e93b443109888e28d6facf6f52e67928e8baaa936e355bfdd324074302c4832e2fa0bd8745309a45eb729866d0513b928ac618ccc9432b7befc3aa2aac66
2023-02-20 16:54:15 +00:00
fanquake
e996219f9a Merge bitcoin/bitcoin#27113: rpc: Use a FlatSigningProvider in decodescript to allow inferring descriptors for scripts larger than 520 bytes
73ec4b2a83 tests: decodescript can infer descriptors for scripts >520 bytes (Andrew Chow)
7cc7822371 rpc: Use FlatSigningProvider in decodescript (Andrew Chow)

Pull request description:

  `FillableSigningProvider` limits scripts to 520 bytes even though segwit allows scripts to be larger than that. We can avoid this limit by using a `FlatSigningProvider` so that such larger scripts can be decoded.

  Fixes #27111

ACKs for top commit:
  instagibbs:
    ACK 73ec4b2a83

Tree-SHA512: c0e6d21025e2da864471989ac94c54e127d05459b9b048f34a0da8d76d8e372d5472a2e667ba2db74d6286e3e6faa55486ffa9232a068b519afa676394031d5a
2023-02-20 16:41:46 +00:00
fanquake
ff4a73aea2 depends: use FORTIFY_SOURCE=3 with libevent 2023-02-20 16:36:36 +00:00
fanquake
0561f344e0 Merge bitcoin/bitcoin#27027: build: use _FORTIFY_SOURCE=3
4faa4e37a6 build: use _FORTIFY_SOURCE=3 (fanquake)

Pull request description:

  [glibc 2.33](https://sourceware.org/pipermail/libc-alpha/2021-February/122207.html) introduced a new fortification level, `_FORTIFY_SOURCE=3`. It improves the coverage of cases where `_FORTIFY_SOURCE` can use `_chk` functions.

  For example, using GCC 13 and glibc 2.36 (Fedora Rawhide), compiling master:
  ```bash
  nm -C src/bitcoind | grep _chk
                   U __fprintf_chk@GLIBC_2.17
                   U __memcpy_chk@GLIBC_2.17
                   U __snprintf_chk@GLIBC_2.17
                   U __sprintf_chk@GLIBC_2.17
                   U __stack_chk_fail@GLIBC_2.17
                   U __stack_chk_guard@GLIBC_2.17
                   U __vsnprintf_chk@GLIBC_2.17

  objdump -d src/bitcoind | grep "_chk@plt" | wc -l
  33
  ```

  vs this branch:
  ```bash
  nm -C src/bitcoind | grep _chk
                   U __fprintf_chk@GLIBC_2.17
                   U __memcpy_chk@GLIBC_2.17
                   U __memset_chk@GLIBC_2.17
                   U __snprintf_chk@GLIBC_2.17
                   U __sprintf_chk@GLIBC_2.17
                   U __stack_chk_fail@GLIBC_2.17
                   U __stack_chk_guard@GLIBC_2.17
                   U __vsnprintf_chk@GLIBC_2.17

  objdump -d src/bitcoind | grep "_chk@plt" | wc -l
  61
  ```

  Usage of level 3 requires LLVM/Clang 9+, or GCC 12+. Older compilers/glibc will still use _FORTIFY_SOURCE=2. For example, in the glibc we currently use for Linux release builds (2.24), `__USE_FORTIFY_LEVEL` is determined using the following:
  ```c
  #if defined _FORTIFY_SOURCE && _FORTIFY_SOURCE > 0
  # if !defined __OPTIMIZE__ || __OPTIMIZE__ <= 0
  #  warning _FORTIFY_SOURCE requires compiling with optimization (-O)
  # elif !__GNUC_PREREQ (4, 1)
  #  warning _FORTIFY_SOURCE requires GCC 4.1 or later
  # elif _FORTIFY_SOURCE > 1
  #  define __USE_FORTIFY_LEVEL 2
  # else
  #  define __USE_FORTIFY_LEVEL 1
  # endif
  #endif
  #ifndef __USE_FORTIFY_LEVEL
  # define __USE_FORTIFY_LEVEL 0
  #endif
  ```
  so any value > 1 will turn on `_FORTIFY_SOURCE=2`. This value detection logic has become slightly more complex in later versions of glibc.

  https://sourceware.org/pipermail/libc-alpha/2021-February/122207.html
  https://developers.redhat.com/blog/2021/04/16/broadening-compiler-checks-for-buffer-overflows-in-_fortify_source

ACKs for top commit:
  theuni:
    ACK 4faa4e37a6. After playing with this quite a bit I didn't observe any noticeable pitfalls.

Tree-SHA512: e84ba49e3872c29fed1e2aea237b0d6bdff0d1274fa3297e2e08317cb62004396ee97b1cd6addb7c8b582498f3fa857a6d84c8e8f5ca97791b93985b47ff7faa
2023-02-20 16:35:38 +00:00
fanquake
150cc8ef42 Merge bitcoin/bitcoin#27128: test: fix intermittent issue in p2p_disconnect_ban
1819564c21 test: fix intermittent issue in `p2p_disconnect_ban` (brunoerg)

Pull request description:

  Fixes #26808

  When `node0` calls `disconnectnode` to disconnect `node1`, we should check in `node1` if it worked, because for `node0` the informations in `getpeerinfo` may be updated before really completing the disconnection.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 1819564c21

Tree-SHA512: 53a386fc38e2faa6f6da3536e76857ff4b6f55e2590d73fe857b3fe5d0f3ff92c5c7e4abd50ab4be250cb2106a4d14ad95d4809ea60c6e00ed3ac0e71255b0b0
2023-02-20 16:28:02 +00:00
fanquake
446c8f581c Merge bitcoin/bitcoin#25950: test: fix test abort for high timeout values (and --timeout-factor 0)
14302a4802 test: fix test abort for high timeout values (and `--timeout-factor 0`) (Sebastian Falbesoner)

Pull request description:

  On master, the functional tests's option `--timeout-factor 0` (which according to the test docs and parameter description should disable the RPC timeouts) currently fails, same as high values like `--timeout-factor 999999`:
  ```
  $ ./test/functional/wallet_basic.py --timeout-factor 0
  2022-08-29T01:26:39.561000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_f24yxzp5
  2022-08-29T01:26:40.262000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/honey/bitcoin/test/functional/test_framework/test_framework.py", line 549, in start_nodes
      node.wait_for_rpc_connection()
    File "/home/honey/bitcoin/test/functional/test_framework/test_node.py", line 234, in wait_for_rpc_connection
      rpc.getblockcount()
    File "/home/honey/bitcoin/test/functional/test_framework/coverage.py", line 49, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/home/honey/bitcoin/test/functional/test_framework/authproxy.py", line 142, in __call__
      response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File "/home/honey/bitcoin/test/functional/test_framework/authproxy.py", line 107, in _request
      self.__conn.request(method, path, postdata, headers)
    File "/usr/local/lib/python3.9/http/client.py", line 1285, in request
      self._send_request(method, url, body, headers, encode_chunked)
    File "/usr/local/lib/python3.9/http/client.py", line 1331, in _send_request
      self.endheaders(body, encode_chunked=encode_chunked)
    File "/usr/local/lib/python3.9/http/client.py", line 1280, in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
    File "/usr/local/lib/python3.9/http/client.py", line 1040, in _send_output
      self.send(msg)
    File "/usr/local/lib/python3.9/http/client.py", line 980, in send
      self.connect()
    File "/usr/local/lib/python3.9/http/client.py", line 946, in connect
      self.sock = self._create_connection(
    File "/usr/local/lib/python3.9/socket.py", line 844, in create_connection
      raise err
    File "/usr/local/lib/python3.9/socket.py", line 832, in create_connection
      sock.connect(sa)
  OSError: [Errno 22] Invalid argument
  ```
  This is caused by a high timeout value that Python's HTTP(S) client library can't cope with. Fix this by clamping down the connection's set timeout value in AuthProxy. The change can easily be tested by running an arbitrary test with `--timeout-factor 0` on master (should fail), on this PR (should pass) and on this PR with the clamping value increased by 1 (should fail).

  // EDIT: The behaviour was observed on OpenBSD 7.1 and Python 3.9.12.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 14302a4802

Tree-SHA512: 6469e8ac699f1bb7dea11d5fb8b3ae54d895bb908570587c5631144cd41fe980ca0b1e6d0b7bfa07983307cba15fb26ae92e6766375672bf5be838d8e5422dbc
2023-02-20 16:20:55 +00:00
brunoerg
1819564c21 test: fix intermittent issue in p2p_disconnect_ban
When `node0` calls `disconnectnode` to disconnect `node1`, we should check in `node1` if it worked, because for `node0` the informations in `getpeerinfo` may be updated before really completing the disconnection.
2023-02-20 10:36:35 -03:00
glozow
08b65df1bb Merge bitcoin/bitcoin#26883: src/node/miner cleanups, follow-ups for #26695
6a5e88e5cf miner: don't re-apply default Options value if argument is unset (stickies-v)
ea72c3d9d5 refactor: avoid duplicating BlockAssembler::Options members (stickies-v)
cba749a9b7 refactor: rename local gArgs to args (stickies-v)

Pull request description:

  Two follow-ups for #26695, both refactoring and no observed (*) behaviour change:
  - Rename `gArgs` to `args` because it's not actually a global
  - Add `BlockAssembler::Options` as a (private) member to `BlockAssembler` to avoid having to assign all the options individually, essentially duplicating them

  Reduces LoC and makes the code more readable, in my opinion.

  ---

  (*) as [pointed out by ajtowns](https://github.com/bitcoin/bitcoin/pull/26883#discussion_r1068247937), this PR changes the interface of `ApplyArgsManOptions()`, making this not a pure refactoring PR. In practice, `ApplyArgsManOptions()` is never called in such a way that this leads to observed behaviour change. Regardless, I've carved out the potential behaviour change into a separate commit and would be okay with dropping it, should it turn out to be controversial.

ACKs for top commit:
  glozow:
    ACK 6a5e88e5cf
  TheCharlatan:
    Light code review ACK 6a5e88e5cf

Tree-SHA512: 15c30442ff0e070b1a58dc4c9615550d619ce35b4a2596b2c0a9d790259bbf987cab708f7cbb1057a8cf8b4c3226f3ad981282d3499ac442094806492a5f68ce
2023-02-20 11:32:43 +00:00
Sebastian Falbesoner
7554b1fd66 rpc: fix successful broadcast count in submitpackage error msg
If a `submitpackage` RPC call errors due to any of the individual tx
broadcasts failing, the returned error message is supposed to contain
the number of successful broadcasts so far. Right now this is wrongly
always shown as zero. Fix this by adding the missing counting.
(Note though that the error should be really rare, as all txs have
already been submitted succesfully to the mempool.)
2023-02-20 00:34:48 +01:00
fanquake
4395b7f084 Merge bitcoin/bitcoin#26814: refactor: remove windows-only compat.h usage in random
621cfb7722 random: consolidate WIN32 #ifdefs (fanquake)
75ec6275e6 random: remove compat.h include (fanquake)
4dc12816ac random: use int for MAX_TRIES (fanquake)

Pull request description:

  This change is related to removing the use of `compat.h` as a miscellaneous catch-all for unclear/platform specific includes. Somewhat prompted by IWYU-related discussion here: https://github.com/bitcoin/bitcoin/pull/26763/files#r1058861693.

  The only reason `compat.h` is required in random.cpp for Windows (note the `#ifdef WIN32`), is for `ssize_t` and an "indirect" inclusion of `windows.h`. I say indirect, because `windows.h` isn't actually included in compat.h either, it's dragged in as a side-effect of other windows includes there, i.e `winsock2.h`.

  Remove this coupling by replacing `ssize_t` with int, just including `windows.h` and removing compat.h.

ACKs for top commit:
  hebasto:
    re-ACK 621cfb7722, rebased only since my [recent](https://github.com/bitcoin/bitcoin/pull/26814#pullrequestreview-1237312144) review. Verified with:
  john-moffett:
    ACK 621cfb7722

Tree-SHA512: 31e1ed2e7ff7daf6c3ee72e6a908def52f7addf8305ba371c5032f1927cbb8ef5d302785e8de42b5c04a123052f04688cc9fd80decceb04738b5d9153f3d32d7
2023-02-19 13:55:17 +00:00
Leonardo Lazzaro
61bb4e783b lint: enable E722 do not use bare except 2023-02-18 11:24:09 +00:00
Sebastian Falbesoner
14302a4802 test: fix test abort for high timeout values (and --timeout-factor 0) 2023-02-17 23:30:59 +01:00
Andrew Chow
a245429d68 Merge bitcoin/bitcoin#26940: test: create random and coins utils, add amount helper, dedupe add_coin
4275195606 De-duplicate add_coin methods to a test util helper (Jon Atack)
9d92c3d7f4 Create InsecureRandMoneyAmount() test util helper (Jon Atack)
81f5ade2a3 Move random test util code from setup_common to random (Jon Atack)

Pull request description:

  - Move random test utilities from `setup_common` to a new `random` file, as many tests don't use this code.

  - Create a helper to generate semi-random CAmounts up to `MONEY_RANGE` rather than only uint32, and use the helper in the unit tests.

  - De-duplicate a shared `add_coin` method by extracting it to a `coins` test utility.

ACKs for top commit:
  pinheadmz:
    ACK 4275195606
  achow101:
    ACK 4275195606
  john-moffett:
    ACK 4275195606

Tree-SHA512: 3ed974251149c7417f935ef2f8865aa0dcc33b281b47522b0f96f1979dff94bb8527957f098fe4d210f40d715c00f29512f2ffe189097102229023b7284a3a27
2023-02-17 17:28:14 -05:00
Andrew Chow
9321df4487 Merge bitcoin/bitcoin#25862: refactor, kernel: Remove gArgs accesses from dbwrapper and txdb
aadd7c5b9b refactor, validation: Add ChainstateManagerOpts db options (Ryan Ofsky)
0352258148 refactor, txdb: Use DBParams struct in CBlockTreeDB (Ryan Ofsky)
c00fa1a734 refactor, txdb: Add CoinsViewOptions struct (Ryan Ofsky)
2eaeded37f refactor, dbwrapper: Add DBParams and DBOptions structs (Ryan Ofsky)

Pull request description:

  Code in the libbitcoin_kernel library should not be calling `ArgsManager` methods or trying to read options from the command line. Instead it should just get options values from simple structs and function arguments that are passed in externally. This PR removes `gArgs` accesses from `dbwrapper` and `txdb` modules by defining appropriate options structs, and is a followup to PR's #25290 #25487 #25527 which remove other `ArgsManager` calls from kernel modules.

  This PR does not change behavior in any way. It is a simpler alternative to #25623 because the only thing it does is remove `gArgs` references from kernel code. It avoids other unnecessary changes like adding options to the kernel API (they can be added separately later).

ACKs for top commit:
  TheCharlatan:
    Code review ACK aadd7c5b9b
  achow101:
    ACK aadd7c5b9b
  furszy:
    diff ACK aadd7c5b

Tree-SHA512: 46dfd5d99ab3110492e7bba97a87122c831b8344caaf7dd2ebdb6e0ad6aa9174d4d1832d6f3a7465eda9294fe50defaa3c000afbbddc4e72838687df09a63ffd
2023-02-17 16:54:55 -05:00
Pieter Wuille
95f12de925 BIP341 txdata cannot be precomputed without spent outputs 2023-02-17 16:29:49 -05:00
Andrew Chow
f722a9bd13 Merge bitcoin/bitcoin#20018: p2p: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified
2555a3950f p2p: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified (Dhruv Mehta)

Pull request description:

  If the user runs: `bitcoind -connect=X -seednode=Y`, I _think_ it is safe to ignore `-seednode`. A more populated `addrman` (via `getaddr` calls to peers in `-seednode`) is not useful in this configuration: `addrman` entries are used to initiate new outbound connections when slots are open, or to open feeler connections and keep `addrman` from getting stale. This is all done in a part of `ThreadOpenConnections` (below [this line](https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1803)) which is never executed when `-connect` is supplied. With `-connect`, `ThreadOpenConnections` will run [this loop](https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1785) and exit thread execution when interrupted.

  Reviewers may also find it relevant that when `-connect` is used, we [soft disable](https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L800) `-dnsseed` in init.cpp perhaps for the same reason i.e. seeding is not useful with `-connect`.

  Running `ProcessAddrFetch` does not seem to have downside except developer confusion AFAICT. I was confused by this and felt it might affect other new bitcoiners too. If there is strong preference to not remove the line, I'd also be happy to just leave a comment there mentioning `ADDR_FETCH`/`-seednode` is irrelevant when used with `-connect`.

  If this change is accepted, the node will still make `getaddr` calls to peers in `-connect` and expand `addrman`. However, disabling those `getaddr` calls would leak information about the node's configuration.

ACKs for top commit:
  mzumsande:
    Code Review ACK 2555a3950f
  achow101:
    ACK 2555a3950f
  vasild:
    ACK 2555a3950f

Tree-SHA512: 9187a0cff58db8edeca7e15379b1c121e7ebe8c38fb82f69e3dae8846ee94c92a329d79025e0f023c7579b2d86e7dbf756e4e30e90a72236bfcd2c00714180b3
2023-02-17 14:21:06 -05:00
Andrew Chow
35fbc97208 Merge bitcoin/bitcoin#25619: net: avoid overriding non-virtual ToString() in CService and use better naming
c9d548c91f net: remove CService::ToStringPort() (Vasil Dimov)
fd4f0f41e9 gui: simplify OptionsDialog::updateDefaultProxyNets() (Vasil Dimov)
96c791dd20 net: remove CService::ToString() use ToStringAddrPort() instead (Vasil Dimov)
944a9de08a net: remove CNetAddr::ToString() and use ToStringAddr() instead (Vasil Dimov)
043b9de59a scripted-diff: rename ToStringIP[Port]() to ToStringAddr[Port]() (Vasil Dimov)

Pull request description:

  Before this PR we had the somewhat confusing combination of methods:

  `CNetAddr::ToStringIP()`
  `CNetAddr::ToString()` (duplicate of the above)
  `CService::ToStringIPPort()`
  `CService::ToString()` (duplicate of the above, overrides a non-virtual method from `CNetAddr`)
  `CService::ToStringPort()`

  Avoid [overriding non-virtual methods](https://github.com/bitcoin/bitcoin/pull/25349/#issuecomment-1185226396).

  "IP" stands for "Internet Protocol" and while sometimes "IP addresses" are called just "IPs", it is incorrect to call Tor or I2P addresses "IPs". Thus use "Addr" instead of "IP".

  Change the above to:

  `CNetAddr::ToStringAddr()`
  `CService::ToStringAddrPort()`

  The changes touch a lot of files, but are mostly mechanical.

ACKs for top commit:
  sipa:
    utACK c9d548c91f
  achow101:
    ACK c9d548c91f
  jonatack:
    re-ACK c9d548c91f only change since my previous reviews is rebase, but as a sanity check rebased to current master and at each commit quickly re-reviewed and re-verified clean build and green unit tests
  LarryRuane:
    ACK c9d548c91f

Tree-SHA512: 633fb044bdecf9f551b5e3314c385bf10e2b78e8027dc51ec324b66b018da35e5b01f3fbe6295bbc455ea1bcd1a3629de1918d28de510693afaf6a52693f2157
2023-02-17 13:34:40 -05:00
Andrew Chow
27772d8009 Merge bitcoin/bitcoin#26889: refactor: wallet, remove global 'ArgsManager' dependency
52f4d567d6 refactor: remove <util/system.h> include from wallet.h (furszy)
6c9b342c30 refactor: wallet, remove global 'ArgsManager' access (furszy)
d8f5fc4462 wallet: set '-walletnotify' script instead of access global args manager (furszy)
3477a28dd3 wallet: set keypool_size instead of access global args manager (furszy)

Pull request description:

  Structurally, the wallet class shouldn't access the global `ArgsManager` class, its internal behavior shouldn't be coupled to a global command line args parsing object.

  So this PR migrates the only two places where we depend on it: (1) the keypool size, and (2) the "-walletnotify" script. And cleans up the, now unneeded, wallet `ArgsManager` ref member.

  Extra note:
  In the process of removing the args ref member, discovered and fixed files that were invalidly depending on the wallet header including `util/system.h`.

ACKs for top commit:
  achow101:
    ACK 52f4d567d6
  TheCharlatan:
    Re-ACK 52f4d567d6
  hebasto:
    re-ACK 52f4d567d6

Tree-SHA512: 0cffd99b4dd4864bf618aa45aeaabbef2b6441d27b6dbb03489c4e013330877682ff17b418d07aa25fbe1040bdf2c67d7559bdeb84128c5437bf0e6247719016
2023-02-17 12:47:52 -05:00
fanquake
621cfb7722 random: consolidate WIN32 #ifdefs
Order includes
Remove // for xyz comments
2023-02-17 15:01:50 +00:00
fanquake
75ec6275e6 random: remove compat.h include
We no-longer need ssize_t.

Add windows.h, which was being indirectly included via compat.h. It isn't
actually included in compat.h itself, but was being included as a side-effect
of other includes, like winsock2.h.
2023-02-17 15:01:49 +00:00
fanquake
4dc12816ac random: use int for MAX_TRIES
Removing the use of ssize_t, removes the need to include compat.h, just 
to make Windows happy.
2023-02-17 15:01:49 +00:00
Antoine Poinsot
c1b7bd047f fuzz: avoid redundant dup key checks when creating Miniscript nodes
Check it only once on the top level node.

Running libfuzzer with -runs=0 against the qa-assets corpus (1b9ddc96586769d92b1b62775f397b7f1a63f142).
Without this patch:
	miniscript_stable: Done 6616 runs in 118 second(s)
	miniscript_smart: Done 13182 runs in 253 second(s)
With this patch:
	miniscript_stable: Done 6616 runs in 57 second(s)
	miniscript_smart: Done 13182 runs in 124 second(s)
2023-02-17 12:41:04 +01:00
fanquake
4faa4e37a6 build: use _FORTIFY_SOURCE=3
glibc 2.33 introduced a new fortification level, _FORTIFY_SOURCE=3.
Which improves the coverage of cases where _FORTIFY_SOURCE can use _chk
functions. For example, using GCC 13 and glibc 2.36 (Fedora Rawhide),
compiling master:
```bash
nm -C src/bitcoind | grep _chk
                 U __fprintf_chk@GLIBC_2.17
                 U __memcpy_chk@GLIBC_2.17
                 U __snprintf_chk@GLIBC_2.17
                 U __sprintf_chk@GLIBC_2.17
                 U __stack_chk_fail@GLIBC_2.17
                 U __stack_chk_guard@GLIBC_2.17
                 U __vsnprintf_chk@GLIBC_2.17

objdump -d src/bitcoind | grep "_chk@plt" | wc -l
33
```

vs this branch:
```bash
nm -C src/bitcoind | grep _chk
                 U __fprintf_chk@GLIBC_2.17
                 U __memcpy_chk@GLIBC_2.17
                 U __memset_chk@GLIBC_2.17
                 U __snprintf_chk@GLIBC_2.17
                 U __sprintf_chk@GLIBC_2.17
                 U __stack_chk_fail@GLIBC_2.17
                 U __stack_chk_guard@GLIBC_2.17
                 U __vsnprintf_chk@GLIBC_2.17

objdump -d src/bitcoind | grep "_chk@plt" | wc -l
61
```

Usage of level 3 requires LLVM/Clang 9+, or GCC 12+. Older
compilers/glibc will still use _FORTIFY_SOURCE=2. For example, in the
glibc we currently use for Linux release builds (2.24), FORTIFY_LEVEL is
determined using the following:
```c
```
so any value > 1 will turn on _FORTIFY_SOURCE=2.

https://sourceware.org/pipermail/libc-alpha/2021-February/122207.html
https://developers.redhat.com/blog/2021/04/16/broadening-compiler-checks-for-buffer-overflows-in-_fortify_source
2023-02-17 10:49:17 +00:00
fanquake
fe1b325688 Merge bitcoin/bitcoin#27029: guix: consolidate to glibc 2.27 for Linux builds
d5d4b75840 guix: combine glibc hardening options into hardened-glibc (fanquake)
c49f2b8eb5 guix: remove no-longer needed powerpc workaround (fanquake)
74c9893989 guix: use glibc 2.27 for all Linux builds (fanquake)

Pull request description:

  Build against glibc 2.27 for all Linux builds (previously only used for RISC-V), and at the same time, increase our minimum required glibc to 2.27 (2018). This would drop support for Ubuntu Xenial (16.04) & Debian Stretch (9), from the produced release binaries. Compiling from source on those systems may be possible, assuming you can install a recent enough compiler/toolchain etc.

ACKs for top commit:
  hebasto:
    ACK d5d4b75840, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 910f0ef45b4558f2a45d35a5c1c39aaac97e8aff086dc4fc1eddbb80c0b6e4bd23667d64e21d0fd42e4db37b6f26f447ca5d1150bb861128af7e71fb42835cf8
2023-02-17 10:40:57 +00:00
fanquake
bc35c4f58c Merge bitcoin/bitcoin#27106: net: remove orphaned CSubNet::SanityCheck()
30a3230e86 script: remove out-of-date snprintf TODO (Jon Atack)
0e015146bd net: remove orphaned CSubNet::SanityCheck() (Jon Atack)

Pull request description:

  `CSubNet::SanityCheck()` was added in #20140, and not removed in #22570 when it became orphaned code.

  Also, remove an out-of-date `snprintf` TODO that was resolved in #27036, and fix up 2 words to make the spelling linter green again.

ACKs for top commit:
  fanquake:
    ACK 30a3230e86
  pinheadmz:
    ACK 30a3230e86
  brunoerg:
    crACK 30a3230e86

Tree-SHA512: f91a2a5af902d3b82ab496f19deeac17d58dbf72a8016e880ea61ad858b66e7ea0ae70b964c4032018eb3252cc34ac5fea163131c6a7f1baf87fc9ec9b5833d8
2023-02-17 10:31:24 +00:00
Martin Zumsande
0af16e7134 doc: add release note for #25574 2023-02-16 17:58:52 -05:00
Martin Zumsande
57ef2a4812 validation: report if pruning prevents completion of verification
Now the verifychain RPC returns false if the checks didn't
finish because the blocks requested to be queried have been pruned.
2023-02-16 17:58:52 -05:00
Martin Zumsande
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache
The rpc command verifychain now fails if the dbcache was not sufficient
to complete the verification at the specified level and depth.

In the same situation, the VerifyDB check during Init will now fail (and lead to
an early shutdown) if the user has explicitly specified -checkblocks or
-checklevel but the check couldn't be executed because of the limited
cache. If the user didn't change any of the two and is using the defaults, log a warning
but don't prevent the node from starting up.
2023-02-16 17:58:52 -05:00
Martin Zumsande
d6f781f1cf validation: return VerifyDBResult::INTERRUPTED if verification was interrupted
This means that the -verifydb RPC will now return false if it
cannot finish due to the node being shutdown.
2023-02-16 17:32:15 -05:00
Martin Zumsande
6360b5302d validation: Change return value of VerifyDB to enum type
This does not change behavior. It is in preparation for
special handling of the case where VerifyDB doesn't finish
for various reasons, but doesn't fail.
2023-02-16 17:29:34 -05:00
Andrew Chow
73966f75f6 Merge bitcoin/bitcoin#25344: New outputs argument for bumpfee/psbtbumpfee
4c8ecccdcd test: add tests for `outputs` argument to `bumpfee`/`psbtbumpfee` (Seibart Nedor)
c0ebb98382 wallet: add `outputs` arguments to `bumpfee` and `psbtbumpfee` (Seibart Nedor)
a804f3cfc0 wallet: extract and reuse RPC argument format definition for outputs (Seibart Nedor)

Pull request description:

  This implements a modification of the proposal in #22007: instead of **adding** outputs to the set of outputs in the original transaction, the outputs given by `outputs` argument **completely replace** the outputs in the original transaction.

  As noted below, this makes it easier to "cancel" a transaction or to reduce the amounts in the outputs, which is not the case with the original proposal in #22007, but it seems from the discussion in this PR that the **replace** behavior is more desirable than **add** one.

ACKs for top commit:
  achow101:
    ACK 4c8ecccdcd
  1440000bytes:
    Code Review ACK 4c8ecccdcd
  ishaanam:
    reACK 4c8ecccdcd

Tree-SHA512: 31361f4a9b79c162bda7929583b0a3fd200e09f4c1a5378b12007576d6b14e02e9e4f0bab8aa209f08f75ac25a1f4805ad16ebff4a0334b07ad2378cc0090103
2023-02-16 13:47:41 -05:00
Andrew Chow
7fd125b27d wallet: Be able to unlock the wallet for migration
Since migration reloads the wallet, the wallet will always be locked
unless the passphrase is given. migratewallet can now take the
passphrase in order to unlock the wallet for migration.
2023-02-16 13:39:03 -05:00
Andrew Chow
6bdbc5ff59 rpc: Allow users to specify wallet name for migratewallet 2023-02-16 13:39:02 -05:00
Andrew Chow
dbfa345403 wallet: Allow MigrateLegacyToDescriptor to take a wallet name
An overload of MigrateLegacyToDescriptor is added which takes the wallet
name. The original that took a wallet pointer is still available, it
just gets the name, closes the wallet, and calls the new overload.
2023-02-16 13:39:02 -05:00
Andrew Chow
bb86887527 verify-commits: Skip checks for commits older than trusted roots 2023-02-16 12:47:00 -05:00
Andrew Chow
5497c14830 verify-commits: Use merge-tree in clean merge check 2023-02-16 12:47:00 -05:00
Andrew Chow
76923bfa09 verify-commits: Remove all allowed commit exceptions
These commits predate the current trusted root.
2023-02-16 12:46:59 -05:00
Andrew Chow
53b07b2b47 verify-commits: Move trusted-keys valid sig check into verify-commits itself
Instead of having gpg.sh check against the trusted keys for a valid
signature, do it inside of verify-commits itself.

This also allows us to use the same trusted-keys throughout the
verify-commits.py check rather than it possibly being modified during
the clean merge check.
2023-02-16 12:46:40 -05:00
Andrew Chow
73ec4b2a83 tests: decodescript can infer descriptors for scripts >520 bytes 2023-02-16 11:42:28 -05:00
Andrew Chow
7cc7822371 rpc: Use FlatSigningProvider in decodescript
Using a FillableSigningProvider results in decodescript being unable to
infer descriptors for scripts larger than 520 bytes. Using a
FlatSigningProvider resolves this.
2023-02-16 11:37:31 -05:00
fanquake
75f0e0b607 Merge bitcoin/bitcoin#26773: doc: FreeBSD build doc updates to reflect removal of install_db4.sh
c572eae989 update the freebsd build doc to reflect recent changes to DB4 install process (Murray Nesbitt)

Pull request description:

  This PR introduces documentation changes needed to keep up with #26834.

ACKs for top commit:
  fanquake:
    ACK c572eae989 - have not tested, but looks ok.

Tree-SHA512: 42a79e7b45834916b1b738db524b51b9ff4fde8348ba66fc331ff6603532dd9fce73ea392eef97d31112326c6d60ec2c5c7c29e66aab33aaf846aab8aea1d1aa
2023-02-16 14:50:00 +00:00
merge-script
437dfe1c26 Merge bitcoin/bitcoin#26714: test: add coverage for unparsable -maxuploadtarget
7a83aa0982 test: add coverage for unparsable `-maxuploadtarget` (brunoerg)

Pull request description:

  This PR adds test coverage for the following error:
  7386da7a0b/src/init.cpp (L1096-L1099)

Top commit has no ACKs.

Tree-SHA512: c115b2b4d2d0eb2316bf9fafd7e0046aa18c9650062779b3a82d6145d188765bff5317f4ca5f79607732fde6d83e1f67756ac20a12c98d060ee68d8acc20c76e
2023-02-16 15:17:54 +01:00
merge-script
98042a0456 Merge bitcoin/bitcoin#27107: doc: remove mention of "proper signing key"
304ae6dc8e doc: remove mention of "proper signing key" (fanquake)

Pull request description:

  This key is no-longer in use: https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2023-February/000115.html
  > Please remove it from verification pipelines.

ACKs for top commit:
  hebasto:
    ACK 304ae6dc8e

Tree-SHA512: 3dfd221a48f69ac56b4568db06b5d5b5d6a60b7d027a26157912219a2073589a0a3934cb30e11a161d48db55d3a637338f96617e3f3b92cb9e60e0d1d1dd372a
2023-02-16 15:10:41 +01:00
merge-script
3a68e194f5 Merge bitcoin/bitcoin#26586: test: previous releases: add v24.0.1
741908afc1 test: previous releases: add v24.0.1 (Sebastian Falbesoner)

Pull request description:

  The same procedure as every release (see dba1231672 [v23.0] and d8b705f1ca [v22.0]), only a little simpler now: thanks to #25650, the previous release fetch script defaults to downloading/building the necessary tags, i.e. we don't need to extend the tag list in the CI scripts and test/README.md anymore.

ACKs for top commit:
  Sjors:
    tACK 741908afc1

Tree-SHA512: a5426e989bd0bba42aa13e7d4cf60f792bf36bd9a6cdb6ef5799f7574d9a8a20979244627bbd0c6219630367e7fd73bac9e677814bc50233f64592ad035e713e
2023-02-16 15:02:02 +01:00
fanquake
3995c88e43 Merge bitcoin/bitcoin#25898: util: remove WSL 1 workaround in fs
5669afb80e fs: drop old WSL1 hack. (sinetek)

Pull request description:

  Following discussion, the WSL1 patch will be removed, as WSL1 is no longer being developed by Microsoft. Instead, please upgrade to a mainstream WSL2 version. More information can be found on [the official website](https://docs.microsoft.com/en-us/windows/wsl/).

ACKs for top commit:
  1440000bytes:
    ACK 5669afb80e
  fanquake:
    ACK 5669afb80e - seems ok as-is.

Tree-SHA512: 256c13985f6dd3453caf39c7ef1c951dbdfa8457a18cd05e4624db36d8ed8a4f809bb78a7b3c82c72997e9ed3823d5566a5c2d0812d2501aba2e54bc5e6eec79
2023-02-16 12:33:26 +00:00
fanquake
304ae6dc8e doc: remove mention of "proper signing key"
This key is no-longer in use:
https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2023-February/000115.html
2023-02-16 10:27:19 +00:00
fanquake
fb82d91a9c Merge bitcoin/bitcoin#24149: Signing support for Miniscript Descriptors
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction (Antoine Poinsot)
840a396029 qa: add a "smart" Miniscript fuzz target (Antoine Poinsot)
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding (Antoine Poinsot)
611e12502a qa: functional test Miniscript signing with key and timelocks (Antoine Poinsot)
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable (Antoine Poinsot)
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins (Antoine Poinsot)
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges (Antoine Poinsot)
a2f81b6a8f script/sign: signing support for Miniscript with timelocks (Antoine Poinsot)
61c6d1a844 script/sign: basic signing support for Miniscript descriptors (Antoine Poinsot)
4242c1c521 Align 'e' property of or_d and andor with website spec (Pieter Wuille)
f5deb41780 Various additional explanations of the satisfaction logic from Pieter (Pieter Wuille)
22c5b00345 miniscript: satisfaction support (Antoine Poinsot)

Pull request description:

  This makes the Miniscript descriptors solvable.

  Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.

ACKs for top commit:
  achow101:
    ACK 6c7a17a8e0
  sipa:
    utACK 6c7a17a8e0 (to the extent that it's not my own code).

Tree-SHA512: a71ec002aaf66bd429012caa338fc58384067bcd2f453a46e21d381ed1bacc8e57afb9db57c0fb4bf40de43b30808815e9ebc0ae1fbd9e61df0e7b91a17771cc
2023-02-16 10:01:33 +00:00
fanquake
cbf511b3f4 Merge bitcoin/bitcoin#27054: Remove laanwj from trusted-keys
aafa5e945c Remove laanwj from trusted-keys (laanwj)

Pull request description:

  allow-revsig-commits list generated using:

      git log --format="%H %ce" --merges 577bd51a4b8de066466a445192c1c653872657e2..master | grep laanwj | cut -c -40 >> allow-revsig-commits

ACKs for top commit:
  Sjors:
    tACK aafa5e945c 😢
  achow101:
    ACK aafa5e945c
  fanquake:
    ACK aafa5e945c

Tree-SHA512: 5e38ac8101f948030f9577480bfba14674351a7d697d7f6985966d98a0200fa110cee13fb331a1ff0c05874d92d9d03402c540f063155e7eea093accb5f4590e
2023-02-16 09:42:11 +00:00
fanquake
424fc603d8 Merge bitcoin/bitcoin#26997: psbt: s/transcation/transaction/
906631450d s/transcation/transaction/ (Greg Sanders)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 906631450d - looks like other comments are being addressed elsewhere.

Tree-SHA512: c835a14db2e0cf5e0317c95c8c7441df1f7c6cb14be7809fd947e07ea9d23f1f171f111429aabd0509b7f17601bc742041316b18e1135e547a966961f2c65038
2023-02-16 09:39:50 +00:00
Andrew Chow
54742532ce Merge bitcoin-core/gui#711: refactor: Disable unused special members functions in UnlockContext
9fa43b5af6 refactor: Disable unused special members functions in `UnlockContext` (Hennadii Stepanov)

Pull request description:

  Also `UnlockContext::valid` and `UnlockContext::relock` are `const` now.

ACKs for top commit:
  achow101:
    ACK 9fa43b5af6
  john-moffett:
    ACK 9fa43b5af6
  furszy:
    ACK 9fa43b5a

Tree-SHA512: 6d9fa8208676b9bd5d85b73cb2d3136e7f28ef59e68ee34915ec598458868e302a80b9ef1384c0bf7a4c42f936830c3add9662ca0bae73860a55a25cc374b699
2023-02-15 18:41:44 -05:00
Jon Atack
30a3230e86 script: remove out-of-date snprintf TODO
that was resolved in PR27036 "test: Remove last uses of snprintf and simplify"
and while here, fix up 2 words in docs to make the spelling linter green again.
2023-02-15 14:42:28 -08:00
Jon Atack
0e015146bd net: remove orphaned CSubNet::SanityCheck()
CSubNet::SanityCheck() was added in #20140, and not removed in #22570
when it became orphaned code.
2023-02-15 14:41:58 -08:00
furszy
52f4d567d6 refactor: remove <util/system.h> include from wallet.h
Since we no longer store a ref to the global `ArgsManager`
inside the wallet, we can move the util/system.h
include to the cpp.

This dependency removal opened a can of worms, as few
other places were, invalidly, depending on the wallet's
header including it.
2023-02-15 15:49:45 -03:00
furszy
6c9b342c30 refactor: wallet, remove global 'ArgsManager' access
we are not using it anymore
2023-02-15 15:49:45 -03:00
furszy
d8f5fc4462 wallet: set '-walletnotify' script instead of access global args manager 2023-02-15 15:49:44 -03:00
furszy
3477a28dd3 wallet: set keypool_size instead of access global args manager 2023-02-15 15:49:44 -03:00
fanquake
5ecd14a31c Merge bitcoin/bitcoin#26844: Net: Pass MSG_MORE flag when sending non-final network messages (round 2)
691eaf8873 Pass MSG_MORE flag when sending non-final network messages (Matt Whitlock)

Pull request description:

  **N.B.:** This is my second attempt at introducing this optimization. #12519 (2018) was closed in deference to switching to doing gathering socket writes using `sendmsg(2)`, which I agree would have superior performance due to fewer syscalls, but that work was apparently abandoned in late 2018. Ever since, Bitcoin Core has continued writing tons of runt packets to the wire. Can we proceed with my halfway solution for now?

  ----

  Since Nagle's algorithm is disabled, each and every call to `send(2)` can potentially generate a separate TCP segment on the wire. This is especially inefficient when sending the tiny header preceding each message payload.

  Linux implements a `MSG_MORE` flag that tells the kernel not to push the passed data immediately to the connected peer but rather to collect it in the socket's internal transmit buffer where it can be combined with data from successive calls to `send(2)`. Where available, specify this flag when calling `send(2)` in `CConnman::SocketSendData(CNode &)` if the data buffer being sent is not the last one in `node.vSendMsg`.

ACKs for top commit:
  sipa:
    ACK 691eaf8873
  vasild:
    ACK 691eaf8873

Tree-SHA512: 9a7f46bc12edbf78d488f05d1c46760110a24c95af74b627d2604fcd198fa3f511c5956bac36d0034e88c632d432f7d394147e667a11b027af0a30f70a546d70
2023-02-15 16:10:46 +00:00
merge-script
a65d2259f1 Merge bitcoin/bitcoin#27035: test: simplify and speedup mempool_updatefromblock.py by using MiniWallet
dee8549be3 test: simplify and speedup mempool_updatefromblock.py by using MiniWallet (Sebastian Falbesoner)

Pull request description:

  This PR simplifies the functional test mempool_updatefromblock.py by using MiniWallet in order to avoid manual low-level tx creation (signing, outputs selection, fee calculation). Most of the tedious work is done by the method `MiniWallet.send_self_transfer_multi` (calling `create_self_transfer_multi` internally) which supports spending a given set of UTXOs and creating a certain number of outputs.

  As a nice side-effect, the test's performance increases significantly (~3.5x on my system):

  ```
  master
      1m56.80s real     1m50.10s user     0m06.36s system

  PR
      0m32.34s real     0m30.26s user     0m01.41s system
  ```

  The arguments `start_input_txid` and `end_address` have been removed from the `transaction_graph_test` method, as they are currently unused and I don't see them being needed for future tests.

ACKs for top commit:
  brunoerg:
    crACK dee8549be3
  MarcoFalke:
    lgtm ACK dee8549be3 🚏

Tree-SHA512: 9f6da634bdc8c272f9a2af1cddaa364ee371d4e95554463a066249eecebb668d8c6cb123ec8a5404c41b3291010c0c8806a8a01dd227733cec03e73aa93b0103
2023-02-15 16:26:00 +01:00
Matthew Zipkin
14b4921a91 wallet: reuse change dest when recreating TX with avoidpartialspends 2023-02-15 10:14:30 -05:00
fanquake
1e0198b6c1 Merge bitcoin/bitcoin#26153: Reduce wasted pseudorandom bytes in ChaCha20 + various improvements
511aa4f1c7 Add unit test for ChaCha20's new caching (Pieter Wuille)
fb243d25f7 Improve test vectors for ChaCha20 (Pieter Wuille)
93aee8bbda Inline ChaCha20 32-byte specific constants (Pieter Wuille)
62ec713961 Only support 32-byte keys in ChaCha20{,Aligned} (Pieter Wuille)
f21994a02e Use ChaCha20Aligned in MuHash3072 code (Pieter Wuille)
5d16f75763 Use ChaCha20 caching in FastRandomContext (Pieter Wuille)
38eaece67b Add fuzz test for testing that ChaCha20 works as a stream (Pieter Wuille)
5f05b27841 Add xoroshiro128++ PRNG (Martin Leitner-Ankerl)
12ff72476a Make unrestricted ChaCha20 cipher not waste keystream bytes (Pieter Wuille)
6babf40213 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 (Pieter Wuille)
e37bcaa0a6 Split ChaCha20 into aligned/unaligned variants (Pieter Wuille)

Pull request description:

  This is an alternative to #25354 (by my benchmarking, somewhat faster), subsumes #25712, and adds additional test vectors.

  It separates the multiple-of-64-bytes-only "core" logic (which becomes simpler) from a layer around which performs caching/slicing to support arbitrary byte amounts. Both have their uses (in particular, the MuHash3072 code can benefit from multiple-of-64-bytes assumptions), plus the separation results in more readable code. Also, since FastRandomContext effectively had its own (more naive) caching on top of ChaCha20, that can be dropped in favor of ChaCha20's new built-in caching.

  I thought about rebasing #25712 on top of this, but the changes before are fairly extensive, so redid it instead.

ACKs for top commit:
  ajtowns:
    ut reACK 511aa4f1c7
  dhruv:
    tACK crACK 511aa4f1c7

Tree-SHA512: 3aa80971322a93e780c75a8d35bd39da3a9ea570fbae4491eaf0c45242f5f670a24a592c50ad870d5fd09b9f88ec06e274e8aa3cefd9561d623c63f7198cf2c7
2023-02-15 14:58:47 +00:00
fanquake
2b0cd7679f Merge bitcoin/bitcoin#27076: verify-commits: Bump trusted git root to after most recent laanwj merge
6ada37d44c verify-commits: Bump trusted git root to after most recent laanwj merge (Andrew Chow)

Pull request description:

  To prepare for the removal of laanwj's key from trusted key (#27054), the trusted git root needs to be newer than the most recent merge commit signed by his key.

  This can be tested by removing the laanwj's key from trusted keys (e.g. by merging with #27054) and running `verify-commits.py` with `--clean-merge 0`: `./contrib/verify-commits/verify-commits.py --clean-merge 0 HEAD~`. (`--clean-merge 0` disables the clean merge check which will checkout some commits, which results in the `trusted-keys` used in checking of subsequent commits to be different than expected).

ACKs for top commit:
  fanquake:
    ACK 6ada37d44c
  hebasto:
    ACK 6ada37d44c, I've verified the history of laanwj's merge commits.

Tree-SHA512: 55cafeddd54aa2b62d7b7cd41c542f4fd974b322a0405de546600d88658575714ebc893b087eb31f28c205559a7b213f88d9038de431271fca00be866610df74
2023-02-15 12:30:38 +00:00
Hennadii Stepanov
e43ff4eab2 Merge bitcoin-core/gui#603: Add settings.json prune-prev, proxy-prev, onion-prev settings
9d3127b11e Add settings.json prune-prev, proxy-prev, onion-prev settings (Ryan Ofsky)

Pull request description:

  With #602, if proxy and pruning settings are disabled in the GUI and the GUI is restarted, proxy and prune values are not stored anywhere. So if these settings are enabled in the future, default values will be shown, not previous values.

  This PR stores previous values so they will preserved across restarts. I'm not sure I like this behavior because showing default values seems simpler and safer to me. Previous values may just have been set temporarily and may have never actually worked, and it adds some code complexity to store them.

  This PR is one way of resolving #596. Other solutions are possible and could be implemented as alternatives.

ACKs for top commit:
  hebasto:
    ACK 9d3127b11e, tested on Ubuntu 22.04.
  vasild:
    ACK 9d3127b11e
  jarolrod:
    tACK 9d3127b11e

Tree-SHA512: 1778d1819443490c880cfd5c1711d9c5ac75ea3ee8440e2f0ced81d293247163a78ae8aba6027215110aec6533bd7dc6472aeead6796bfbd51bf2354e28f24a9
2023-02-15 12:21:31 +00:00
Murray Nesbitt
c572eae989 update the freebsd build doc to reflect recent changes to DB4 install process 2023-02-15 01:12:00 -08:00
merge-script
68e484afbb Merge bitcoin/bitcoin#26584: cli: include local ("unroutable") peers in -netinfo table
77192c9598 cli: include local ("unreachable") peers in -netinfo table (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/26579

  The `-netinfo` dashboard did not list peers that were connected via "unroutable" networks. This included local peers including local-network peers. Personally, I run one bitcoind instance on my network that is used by other services like Wasabi Wallet and LND running on other machines.

  This PR adds an "npr" (not publicly routable) column to the table of networks (ipv4, ipv6, onion, etc) so that every connection to the node is listed, and the totals are accurate as they relate to max inbound and max outbound limits.

  Example connecting in regtest mode to one local and one remote peer:

  ```
  Bitcoin Core client v24.99.0-151ce099ea8f-dirty regtest - server 70016/Satoshi:24.99.0/

  <->   type   net  mping   ping send recv  txn  blk  hb addrp addrl  age id address         version
   in          npr      0      0   90   90                              1  1 127.0.0.1:59180 70016/Satoshi:24.99.0/
  out manual  ipv4     63     63   84   84         3                    3  0 143.244.175.41  70016/Satoshi:24.0.1/
                       ms     ms  sec  sec  min  min                  min

           ipv4    ipv6     npr   total   block  manual
  in          0       0       1       1
  out         1       0       0       1       0       1
  total       1       0       1       2

  Local addresses: n/a

  ```

ACKs for top commit:
  jonatack:
    Re-tested ACK 77192c9598

Tree-SHA512: 78aa68bcff0dbaadb5f0604bf023fe8fd921313bd8276d12581f7655c089466a48765f9e123cb31d7f1d294d5ca45fdefdf8aa220466ff738f32414f41099c06
2023-02-15 09:18:57 +01:00
ishaanam
6a5b348f2e test: test rescanning encrypted wallets 2023-02-14 23:32:43 -05:00
ishaanam
493b813e17 wallet: ensure that the passphrase is not deleted from memory when being used to rescan
`m_relock_mutex` is introduced so that the passphrase is not
deleted from memory when the timeout provided in
`walletpassphrase` is up, but the wallet is still rescanning.
2023-02-14 23:32:40 -05:00
ishaanam
66a86ebabb wallet: keep track of when the passphrase is needed when rescanning
Wallet passphrases are needed to top up the keypool during a
rescan. The following RPCs need the passphrase when rescanning:
    - `importdescriptors`
    - `rescanblockchain`

The following RPCs use the information about whether or not the
passphrase is being used to ensure that full rescans are able to
take place:
    - `walletlock`
    - `encryptwallet`
    - `walletpassphrasechange`
2023-02-14 23:31:26 -05:00
Andrew Chow
576e16e702 Merge bitcoin/bitcoin#26184: test: p2p: check that headers message with invalid proof-of-work disconnects peer
772671245d test: p2p: check that headers message with invalid proof-of-work disconnects peer (Sebastian Falbesoner)

Pull request description:

  One of the earliest anti-DoS checks done after receiving and deserializing a `headers` message from a peer is verifying whether the proof-of-work is valid (called in method `PeerManagerImpl::ProcessHeadersMessage`):
  f227e153e8/src/net_processing.cpp (L2752-L2762)
  The called method `PeerManagerImpl::CheckHeadersPoW` calls `Misbehaving` with a score of 100, i.e. leading to an immediate disconnect of the peer:
  f227e153e8/src/net_processing.cpp (L2368-L2372)

  This PR adds a simple test for both the misbehaving log and the resulting disconnect. For creating a block header with invalid proof-of-work, we first create one that is accepted by the node (the difficulty field `nBits` is copied from the genesis block) and based on that the nonce is modified until we have block header hash prefix that is too high to fulfill even the minimum difficulty.

ACKs for top commit:
  Sjors:
    ACK 772671245d
  achow101:
    ACK 772671245d
  brunoerg:
    crACK 772671245d
  furszy:
    Code review ACK 77267124 with a non-blocking speedup.

Tree-SHA512: 680aa7939158d1dc672b90aa6554ba2b3a92584b6d3bcb0227776035858429feb8bc66eed18b47de0fe56df7d9b3ddaee231aaeaa360136603b9ad4b19e6ac11
2023-02-14 18:45:35 -05:00
Hennadii Stepanov
9fa43b5af6 refactor: Disable unused special members functions in UnlockContext 2023-02-14 17:55:57 +00:00
fanquake
fb2f093479 Merge bitcoin/bitcoin#27097: descriptors: fix docstring (param [in] vs [out])
588fad868d descriptors: fix docstring (param [in] vs [out]) (SomberNight)

Pull request description:

  As in title, these docstrings look incorrect.

ACKs for top commit:
  john-moffett:
    ACK 588fad868d

Tree-SHA512: 1ab343a1b1fc57a7d6bd8363b84db9d96e8ea11a4cec85bcf79885c9df53da889fe2fb10b1fa92d824ddf0dee800c07353f46f1fea9887d2ad518bed0afebe3d
2023-02-14 17:02:29 +00:00
fanquake
af49d86dd7 Merge bitcoin/bitcoin#27093: test: Fix intermittent sync issue in wallet_pruning
fa9ec7b0fe test: Fix intermittent sync issue in wallet_pruning (MarcoFalke)

Pull request description:

  The `sync_fun=self.no_op` has no motivation or rationale, and seems to be causing issues.

  Fix that by removing it.

  Actually fixes https://github.com/bitcoin/bitcoin/issues/27065, see https://github.com/bitcoin/bitcoin/pull/27066#issuecomment-1428249997

ACKs for top commit:
  fanquake:
    ACK fa9ec7b0fe

Tree-SHA512: 3c67da6705d6698fcabb29de169a2b4723f74705c979380d1fddce5fe9595b4595445fd7d9790a6b2a89f10ce8ec3c64ce45248f58fd920b72b7b6fba8afb09f
2023-02-14 16:52:18 +00:00
SomberNight
588fad868d descriptors: fix docstring (param [in] vs [out])
As in title, these docstrings look incorrect.
2023-02-14 14:28:08 +00:00
fanquake
d6ef44cccb Merge bitcoin/bitcoin#27081: Modernize rpcauth.py
e4e17907b6 Modernize rpcauth.py and its tests (Pieter Wuille)

Pull request description:

  Use Python3 constructions, and f-strings.

ACKs for top commit:
  jamesob:
    Github ACK e4e17907b6

Tree-SHA512: 005573d967e04400fec727f45739f138879be703e692745c0a639272d37d221d230f388de23f2615cb954bb47179fb46e53da0410ae9f0865319b91bb2dc01f4
2023-02-14 11:26:17 +00:00
Pieter Wuille
e4e17907b6 Modernize rpcauth.py and its tests 2023-02-13 17:11:15 -05:00
Andrew Chow
2c1fe27bf3 Merge bitcoin/bitcoin#27080: Wallet: Zero out wallet master key upon locking so it doesn't persist in memory
3a11adc700 Zero out wallet master key upon lock (John Moffett)

Pull request description:

  When an encrypted wallet is locked (for instance via the RPC `walletlock`), the documentation indicates that the key is removed from memory:

  b92d609fb2/src/wallet/rpc/encrypt.cpp (L157-L158)

  However, the vector (a `std::vector<unsigned char, secure_allocator<unsigned char>>`) is merely _cleared_. As it is a member variable, it also stays in scope as long as the wallet is loaded, preventing the secure allocator from deallocating. This allows the key to persist indefinitely in memory. I confirmed this behavior on my macOS machine by using an open-source third party memory inspector ("Bit Slicer"). I was able to find my wallet's master key in Bit Slicer after unlocking and re-locking my encrypted wallet. I then confirmed the key data was at the address in LLDB.

  This PR manually fills the bytes with zeroes before calling `clear()` by using our `memory_cleanse` function, which is designed to prevent the compiler from optimizing it away. I confirmed that it does remove the data from memory on my machine upon locking.

  Note: An alternative approach could be to call `vMasterKey.shrink_to_fit()` after the `clear()`, which would trigger the secure allocator's deallocation. However, `shrink_to_fit()` is not _guaranteed_ to actually change the vector's capacity, so I think it's unwise to rely on it.

  ## Edit: A little more clarity on why this is an improvement.

  Since `mlock`ed memory is guaranteed not to be swapped to disk and our threat model doesn't consider a super-user monitoring the memory in realtime, why is this an improvement? Most importantly, consider hibernation. Even `mlock`ed memory may get written to disk. From the `mlock` [manpage](https://man7.org/linux/man-pages/man2/mlock.2.html):

  > (But be aware that the suspend mode on laptops and some desktop computers will save a copy of the system's RAM to disk, regardless of memory locks.)

  As far as I can tell, this is true of [Windows](https://web.archive.org/web/20190127110059/https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=1833#:~:text=%5BThere%20does%20not%20appear%20to%20be%20any%20guarantee%20that%20the%20memory%20won%27t%20be%20written%20to%20disk%20while%20locked.%20As%20you%20noted%2C%20the%20machine%20may%20be%20hibernated%2C%20or%20it%20may%20be%20running%20in%20a%20VM%20that%20gets%20snapshotted.%20%2DRaymond%5D) and macOS as well.

  Therefore, a user with a strong OS password and a strong wallet passphrase could still have their keys stolen if a thief takes their (hibernated) machine and reads the permanent storage.

ACKs for top commit:
  S3RK:
    Code review ACK 3a11adc700
  achow101:
    ACK 3a11adc700

Tree-SHA512: c4e3dab452ad051da74855a13aa711892c9b34c43cc43a45a3b1688ab044e75d715b42843c229219761913b4861abccbcc8d5cb6ac54957d74f6e357f04e8730
2023-02-13 15:18:16 -05:00
MarcoFalke
fa9ec7b0fe test: Fix intermittent sync issue in wallet_pruning 2023-02-13 17:32:42 +01:00
fanquake
1ad0711d7c Merge bitcoin/bitcoin#27016: mapport: require miniupnpc API version 17 or later
b3b673f704 mapport: require miniupnpc API version 17 or later (fanquake)

Pull request description:

  Version 17 is currently the latest version, see: https://github.com/miniupnp/miniupnp/blob/master/miniupnpc/apiversions.txt, and has been available since the release of 2.1. 2.1 or newer is readily available across all distros, see https://repology.org/project/miniupnpc/versions, so drop support for the older API versions.

  Split out of #22644.

ACKs for top commit:
  hebasto:
    ACK b3b673f704, tested on Ubuntu 20.04 w/ and w/o [`libminiupnpc-dev`](https://packages.ubuntu.com/focal/libminiupnpc-dev) package.
  TheCharlatan:
    ACK b3b673f704

Tree-SHA512: f53b36b82462c4ea83d9b83413dca8097885d1620f7ca0a53a79d6b3d3cf37c7773828b23f4278ccfcc3b14fcb0faffa35f60191b519b04570f3d2783d0303e2
2023-02-13 16:25:09 +00:00
Antoine Poinsot
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction
Co-Authored-By: Andrew Chow <github@achow101.com>
2023-02-13 15:39:25 +01:00
merge-script
8126551d54 Merge bitcoin/bitcoin#27011: Add simulation-based CCoinsViewCache fuzzer
561848aaf2 Exercise non-DIRTY spent coins in caches in fuzz test (Pieter Wuille)
59e6828bb5 Add deterministic mode to CCoinsViewCache (Pieter Wuille)
b0ff310840 Add CCoinsViewCache::SanityCheck() and use it in fuzz test (Pieter Wuille)
3c9cea1340 Add simulation-based CCoinsViewCache fuzzer (Pieter Wuille)

Pull request description:

  The fuzzer goes through a sequence of operations that get applied to both a real stack of `CCoinsViewCache` objects, and to simulation data, comparing the two at the end.

ACKs for top commit:
  jamesob:
    re-ACK 561848aaf2
  dergoegge:
    Code review ACK 561848aaf2

Tree-SHA512: 68634f251fdb39436b128ecba093f651bff12ac11508dc9885253e57fd21efd44edf3b22b0f821c228175ec507df7d46c7f9f5404fc1eb8187fdbd136a5d5ee2
2023-02-13 15:31:50 +01:00
fanquake
d5d4b75840 guix: combine glibc hardening options into hardened-glibc 2023-02-13 14:16:59 +00:00
fanquake
c49f2b8eb5 guix: remove no-longer needed powerpc workaround 2023-02-13 14:16:58 +00:00
fanquake
74c9893989 guix: use glibc 2.27 for all Linux builds
Also point to the latest commit on the glibc 2.27 releases branch.

https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.27/master
2023-02-13 14:16:24 +00:00
merge-script
141115a060 Merge bitcoin/bitcoin#27033: ci: Cache stuff in volumes, not host folders
fa8e92c022 doc: Update ci docs (721217.xyz)
5fffff54e9 ci: Cache stuff in volumes, not host folders (MarcoFalke)

Pull request description:

  Storing cached stuff in host system folders may lead to unexpected issues when the ci-built stuff is used for a non-ci build or a ci task leaks into another ci task.

ACKs for top commit:
  john-moffett:
    ACK fa8e92c022

Tree-SHA512: 8b0c9019452fbe507a272c1037c3dce3c178c21f85ab1096ed3372ad9d4b3c7aa27d89e5bf80c9a6260ea652e0268be0cbe61d6a4fcb3add569fa38076d32287
2023-02-13 11:53:50 +01:00
merge-script
a6316590d5 Merge bitcoin/bitcoin#26970: test: fix immediate tx relay in wallet_groups.py
ab4efad51b test: fix immediate tx relay in wallet_groups.py (Sebastian Falbesoner)

Pull request description:

  In the functional test wallet_groups.py we whitelist peers on all nodes (`-whitelist=noban@127.0.0.1`) to enable immediate tx relay for fast mempool synchronization. However, considering that this setting only applies to inbound peers and the default test topology looks like this:
  ```
      node0 <--- node1 <---- node2 <--- ... <-- nodeN
  ```

  txs propagate fast only from lower- to higher-numbered nodes (i.e. "left to right" in the above diagram) and take long from higher- to lower-numbered nodes ("right to left") since in the latter direction we only have outbound peers, where the trickle relay is still active. As a consequence, if a tx is submitted from any node other than node0, the mempool synchronization can take quite long.

  This PR fixes this by simply adding another connection from node0 to the last node, leading to a ~2-3x speedup (5 runs measured via `time ./test/functional/wallet_groups.py` are shown):

  ```
  master:
      0m53.31s real     0m08.22s user     0m05.60s system
      0m32.85s real     0m07.44s user     0m04.08s system
      0m46.40s real     0m09.18s user     0m04.23s system
      0m46.96s real     0m11.10s user     0m05.74s system
      0m57.23s real     0m10.53s user     0m05.59s system

  PR:
      0m19.64s real     0m09.58s user     0m05.50s system
      0m18.05s real     0m07.77s user     0m04.03s system
      0m18.99s real     0m07.90s user     0m04.25s system
      0m17.49s real     0m07.56s user     0m03.92s system
      0m18.11s real     0m07.74s user     0m03.88s system
  ```
  Note that in most tests this is not a problem since txs very often originate from node0.

ACKs for top commit:
  brunoerg:
    utACK ab4efad51b

Tree-SHA512: 12675357e6eb5a18383f2bfe719a184c0790863b37a98749d8e757dd5dc3a36212e16a81f0a192340c11b793eda00db359c7011f46f7c27e3a093af4f5b62147
2023-02-13 11:51:03 +01:00
Antoine Poinsot
840a396029 qa: add a "smart" Miniscript fuzz target
At the expense of more complexity, this target generates a valid
Miniscript node at every iteration.

This target will at first run populate a list of recipe (a map from
desired type to possible ways of creating such type) and curate it
(remove the unavailable or redundant recipes).
Then, at each iteration it will pick a type, choose a manner to create a
node of such type from the available recipes, and then
pseudo-recursively do the same for the type constraints of the picked
recipe.

For instance, if it is instructed based on the fuzzer output to create a
Miniscript node of type 'Bd', it could choose to create an 'or_i(subA, subB)'
nodes with type constraints 'B' for subA and 'Bd' for subB. It then
consults the recipes for creating subA and subB, etc...

Here is the list of all the existing recipes, by type constraint:

B: 0()
B: 1()
B: older()
B: after()
B: sha256()
B: hash256()
B: ripemd160()
B: hash160()
B: c:(K)
B: d:(Vz)
B: j:(Bn)
B: n:(B)
B: and_v(V,B)
B: and_b(B,W)
B: or_b(Bd,Wd)
B: or_d(Bdu,B)
B: or_i(B,B)
B: andor(Bdu,B,B)
B: thresh(Bdu)
B: thresh(Bdu,Wdu)
B: thresh(Bdu,Wdu,Wdu)
B: multi()

V: v:(B)
V: and_v(V,V)
V: or_c(Bdu,V)
V: or_i(V,V)
V: andor(Bdu,V,V)

K: pk_k()
K: pk_h()
K: and_v(V,K)
K: or_i(K,K)
K: andor(Bdu,K,K)

W: a:(B)
W: s:(Bo)

Bz: 0()
Bz: 1()
Bz: older()
Bz: after()
Bz: n:(Bz)
Bz: and_v(Vz,Bz)
Bz: or_d(Bzdu,Bz)
Bz: andor(Bzdu,Bz,Bz)
Bz: thresh(Bzdu)

Vz: v:(Bz)
Vz: and_v(Vz,Vz)
Vz: or_c(Bzdu,Vz)
Vz: andor(Bzdu,Vz,Vz)

Bo: sha256()
Bo: hash256()
Bo: ripemd160()
Bo: hash160()
Bo: c:(Ko)
Bo: d:(Vz)
Bo: j:(Bon)
Bo: n:(Bo)
Bo: and_v(Vz,Bo)
Bo: and_v(Vo,Bz)
Bo: or_d(Bodu,Bz)
Bo: or_i(Bz,Bz)
Bo: andor(Bzdu,Bo,Bo)
Bo: andor(Bodu,Bz,Bz)
Bo: thresh(Bodu)

Vo: v:(Bo)
Vo: and_v(Vz,Vo)
Vo: and_v(Vo,Vz)
Vo: or_c(Bodu,Vz)
Vo: or_i(Vz,Vz)
Vo: andor(Bzdu,Vo,Vo)
Vo: andor(Bodu,Vz,Vz)

Ko: pk_k()
Ko: and_v(Vz,Ko)
Ko: andor(Bzdu,Ko,Ko)

Bn: sha256()
Bn: hash256()
Bn: ripemd160()
Bn: hash160()
Bn: c:(Kn)
Bn: d:(Vz)
Bn: j:(Bn)
Bn: n:(Bn)
Bn: and_v(Vz,Bn)
Bn: and_v(Vn,B)
Bn: and_b(Bn,W)
Bn: multi()

Vn: v:(Bn)
Vn: and_v(Vz,Vn)
Vn: and_v(Vn,V)

Kn: pk_k()
Kn: pk_h()
Kn: and_v(Vz,Kn)
Kn: and_v(Vn,K)

Bon: sha256()
Bon: hash256()
Bon: ripemd160()
Bon: hash160()
Bon: c:(Kon)
Bon: d:(Vz)
Bon: j:(Bon)
Bon: n:(Bon)
Bon: and_v(Vz,Bon)
Bon: and_v(Von,Bz)

Von: v:(Bon)
Von: and_v(Vz,Von)
Von: and_v(Von,Vz)

Kon: pk_k()
Kon: and_v(Vz,Kon)

Bd: 0()
Bd: sha256()
Bd: hash256()
Bd: ripemd160()
Bd: hash160()
Bd: c:(Kd)
Bd: d:(Vz)
Bd: j:(Bn)
Bd: n:(Bd)
Bd: and_b(Bd,Wd)
Bd: or_b(Bd,Wd)
Bd: or_d(Bdu,Bd)
Bd: or_i(B,Bd)
Bd: or_i(Bd,B)
Bd: andor(Bdu,B,Bd)
Bd: thresh(Bdu)
Bd: thresh(Bdu,Wdu)
Bd: thresh(Bdu,Wdu,Wdu)
Bd: multi()

Kd: pk_k()
Kd: pk_h()
Kd: or_i(K,Kd)
Kd: or_i(Kd,K)
Kd: andor(Bdu,K,Kd)

Wd: a:(Bd)
Wd: s:(Bod)

Bzd: 0()
Bzd: n:(Bzd)
Bzd: or_d(Bzdu,Bzd)
Bzd: andor(Bzdu,Bz,Bzd)
Bzd: thresh(Bzdu)

Bod: sha256()
Bod: hash256()
Bod: ripemd160()
Bod: hash160()
Bod: c:(Kod)
Bod: d:(Vz)
Bod: j:(Bon)
Bod: n:(Bod)
Bod: or_d(Bodu,Bzd)
Bod: or_i(Bz,Bzd)
Bod: or_i(Bzd,Bz)
Bod: andor(Bzdu,Bo,Bod)
Bod: andor(Bodu,Bz,Bzd)
Bod: thresh(Bodu)

Kod: pk_k()
Kod: andor(Bzdu,Ko,Kod)

Bu: 0()
Bu: 1()
Bu: sha256()
Bu: hash256()
Bu: ripemd160()
Bu: hash160()
Bu: c:(K)
Bu: d:(Vz)
Bu: j:(Bnu)
Bu: n:(B)
Bu: and_v(V,Bu)
Bu: and_b(B,W)
Bu: or_b(Bd,Wd)
Bu: or_d(Bdu,Bu)
Bu: or_i(Bu,Bu)
Bu: andor(Bdu,Bu,Bu)
Bu: thresh(Bdu)
Bu: thresh(Bdu,Wdu)
Bu: thresh(Bdu,Wdu,Wdu)
Bu: multi()

Bzu: 0()
Bzu: 1()
Bzu: n:(Bz)
Bzu: and_v(Vz,Bzu)
Bzu: or_d(Bzdu,Bzu)
Bzu: andor(Bzdu,Bzu,Bzu)
Bzu: thresh(Bzdu)

Bou: sha256()
Bou: hash256()
Bou: ripemd160()
Bou: hash160()
Bou: c:(Ko)
Bou: d:(Vz)
Bou: j:(Bonu)
Bou: n:(Bo)
Bou: and_v(Vz,Bou)
Bou: and_v(Vo,Bzu)
Bou: or_d(Bodu,Bzu)
Bou: or_i(Bzu,Bzu)
Bou: andor(Bzdu,Bou,Bou)
Bou: andor(Bodu,Bzu,Bzu)
Bou: thresh(Bodu)

Bnu: sha256()
Bnu: hash256()
Bnu: ripemd160()
Bnu: hash160()
Bnu: c:(Kn)
Bnu: d:(Vz)
Bnu: j:(Bnu)
Bnu: n:(Bn)
Bnu: and_v(Vz,Bnu)
Bnu: and_v(Vn,Bu)
Bnu: and_b(Bn,W)
Bnu: multi()

Bonu: sha256()
Bonu: hash256()
Bonu: ripemd160()
Bonu: hash160()
Bonu: c:(Kon)
Bonu: d:(Vz)
Bonu: j:(Bonu)
Bonu: n:(Bon)
Bonu: and_v(Vz,Bonu)
Bonu: and_v(Von,Bzu)

Bdu: 0()
Bdu: sha256()
Bdu: hash256()
Bdu: ripemd160()
Bdu: hash160()
Bdu: c:(Kd)
Bdu: d:(Vz)
Bdu: j:(Bnu)
Bdu: n:(Bd)
Bdu: and_b(Bd,Wd)
Bdu: or_b(Bd,Wd)
Bdu: or_d(Bdu,Bdu)
Bdu: or_i(Bu,Bdu)
Bdu: or_i(Bdu,Bu)
Bdu: andor(Bdu,Bu,Bdu)
Bdu: thresh(Bdu)
Bdu: thresh(Bdu,Wdu)
Bdu: thresh(Bdu,Wdu,Wdu)
Bdu: multi()

Wdu: a:(Bdu)
Wdu: s:(Bodu)

Bzdu: 0()
Bzdu: n:(Bzd)
Bzdu: or_d(Bzdu,Bzdu)
Bzdu: andor(Bzdu,Bzu,Bzdu)
Bzdu: thresh(Bzdu)

Bodu: sha256()
Bodu: hash256()
Bodu: ripemd160()
Bodu: hash160()
Bodu: c:(Kod)
Bodu: d:(Vz)
Bodu: j:(Bonu)
Bodu: n:(Bod)
Bodu: or_d(Bodu,Bzdu)
Bodu: or_i(Bzu,Bzdu)
Bodu: or_i(Bzdu,Bzu)
Bodu: andor(Bzdu,Bou,Bodu)
Bodu: andor(Bodu,Bzu,Bzdu)
Bodu: thresh(Bodu)

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2023-02-11 16:51:18 +01:00
Antoine Poinsot
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding
This is a "dumb" way of randomly generating a Miniscript node from
fuzzer input. It defines a strict binary encoding and will always generate
a node defined from the encoding without "helping" to create valid nodes.
It will cut through as soon as it encounters an invalid fragment so
hopefully the fuzzer can tend to learn the encoding and generate valid
nodes with a higher probability.

On a valid generated node a number of invariants are checked, especially
around the satisfactions and testing them against the Script
interpreter.

The node generation and testing is modular in order to later introduce
other ways to generate nodes from fuzzer inputs with minimal code.

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-02-11 16:51:17 +01:00
Antoine Poinsot
611e12502a qa: functional test Miniscript signing with key and timelocks
We'll need a better integration of the hash preimages PSBT fields to
satisfy Miniscript with such challenges from the RPC.

Thanks to Greg Sanders for his examples and suggestions to improve this
test.
2023-02-11 14:12:13 +01:00
Antoine Poinsot
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable
We'll add more of them in the next commit, let's keep it bearable.
2023-02-11 14:12:13 +01:00
Antoine Poinsot
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins
This is a workaround for Miniscript descriptors containing hash
challenges. For those we can't mock the signature creator without making
OP_EQUAL mockable in the interpreter, so CalculateMaximumInputSize will
always return -1 and outputs for these descriptors would appear
unsolvable while they actually are.
2023-02-11 14:12:12 +01:00
Antoine Poinsot
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges
Preimages must be externally provided (typically, via a PSBT).
2023-02-11 14:12:12 +01:00
Antoine Poinsot
a2f81b6a8f script/sign: signing support for Miniscript with timelocks 2023-02-11 14:12:11 +01:00
Antoine Poinsot
61c6d1a844 script/sign: basic signing support for Miniscript descriptors
Try to solve a script using the Miniscript satisfier if the legacy
solver fails under P2WSH context. Only solve public key and public key
hash challenges for now.

We don't entirely replace the raw solver and especially rule out trying to
solve CHECKMULTISIG-based multisigs with the Miniscript satisfier since
some features, such as the transaction input combiner, rely on the
specific behaviour of the former.
2023-02-11 14:12:10 +01:00
Pieter Wuille
4242c1c521 Align 'e' property of or_d and andor with website spec 2023-02-11 14:12:10 +01:00
Pieter Wuille
f5deb41780 Various additional explanations of the satisfaction logic from Pieter
Cherry-picked and squashed from
https://github.com/sipa/bitcoin/commits/202302_miniscript_improve.

- Explain thresh() and multi() satisfaction algorithms
- Comment on and_v dissatisfaction
- Mark overcomplete thresh() dissats as malleable and explain
- Add comment on unnecessity of Malleable() in and_b dissat
2023-02-11 14:12:09 +01:00
Antoine Poinsot
22c5b00345 miniscript: satisfaction support
This introduces the logic to "sign for" a Miniscript.

Co-Authored-By: Pieter Wuille <pieter.wuille@gmail.com>
2023-02-11 14:12:09 +01:00
John Moffett
3a11adc700 Zero out wallet master key upon lock
When an encrypted wallet is locked (for instance via the
RPC `walletlock`), the docs indicate that the key is
removed from memory. However, the vector (with a secure
allocator) is merely cleared. This allows the key to persist
indefinitely in memory. Instead, manually fill the bytes with
zeroes before clearing.
2023-02-10 20:21:23 -05:00
Fabian Jahr
60978c8080 test: Reduce extended timeout on abortnode test
This was made obsolete by tracking the active requests and explicitly waiting for them to finish before shutdown.
2023-02-10 20:35:02 +01:00
João Barbosa
660bdbf785 http: Release server before waiting for event base loop exit 2023-02-10 20:35:01 +01:00
João Barbosa
8c6d007c80 http: Track active requests and wait for last to finish 2023-02-10 20:34:58 +01:00
Andrew Chow
6ada37d44c verify-commits: Bump trusted git root to after most recent laanwj merge
To prepare for the removal of laanwj's key from trusted key, the trusted
git root needs to be newer than the most recent merge commit signed by
his key.
2023-02-10 11:36:06 -05:00
721217.xyz
fa8e92c022 doc: Update ci docs 2023-02-10 17:05:39 +01:00
merge-script
b92d609fb2 Merge bitcoin/bitcoin#27072: doc: Remove unused REVIEWERS file
fa8e3aa60d doc: Remove unused REVIEWERS file (721217.xyz)

Pull request description:

  Unused for way more than two months after https://github.com/bitcoin/bitcoin/pull/25613#issuecomment-1200113115

  See also bb5ebadeaa

ACKs for top commit:
  sipa:
    ACK fa8e3aa60d
  john-moffett:
    ACK fa8e3aa60d

Tree-SHA512: def650d34f04abd18e37cf19b5df34463157a99fbb537c4c98092c8312eb2a572ccde13ec5374f3b9e7cb90cef220cfe07f4522ffbfeb9407b20b3ada1d7706c
2023-02-10 16:57:53 +01:00
merge-script
e0d8378f2d Merge bitcoin/bitcoin#27069: net: add Ensure{any}Banman
2d955ff006 net: add `Ensure{any}Banman` (brunoerg)

Pull request description:

  This PR adds `Ensure{any}Banman` functions to avoid code repetition and make it cleaner. Same approach as done with argsman, chainman, connman and others.

ACKs for top commit:
  davidgumberg:
    ACK [2d955ff](2d955ff006)

Tree-SHA512: 0beb7125312168a3df130c1793a1412ab423ef0f46023bfe2a121630c79df7e55d3d143fcf053bd09e2d96e9385a7a04594635da3e5c6be0c5d3a9cafbe3b631
2023-02-10 15:10:21 +01:00
721217.xyz
fa8e3aa60d doc: Remove unused REVIEWERS file 2023-02-10 14:11:31 +01:00
merge-script
4f841cbb81 Merge bitcoin/bitcoin#27070: ci: Fix fingerprint_script for depends subdir caches
d66efa30cd ci: Fix `fingerprint_script` for `depends` subdir caches (Hennadii Stepanov)

Pull request description:

  https://github.com/bitcoin/bitcoin/pull/26977 made current `git rev-list -1 HEAD ./depends` [not working](https://github.com/bitcoin/bitcoin/pull/26977#issuecomment-1424614490).

  This PR fixes this issue with an idea from https://github.com/bitcoin/bitcoin/pull/26977#issuecomment-1424636503.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d66efa30cd

Tree-SHA512: e6dbb1a80439bf8c044e50afc44b8b08b0903d20da5ef9dcbb265f2d64b6810ea5f4cb8abb44e00b96673082f1dbdd350627627d9d28f362bdb662c92ef257d8
2023-02-10 09:39:11 +01:00
Ryan Ofsky
aadd7c5b9b refactor, validation: Add ChainstateManagerOpts db options
Use ChainstateManagerOpts struct to remove ArgsManager uses from validation.cpp.

This commit does not change behavior.
2023-02-10 04:39:11 -04:00
Ryan Ofsky
0352258148 refactor, txdb: Use DBParams struct in CBlockTreeDB
Use DBParams struct to remove ArgsManager uses from txdb.

To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in chainstate.cpp. But these moves are temporary. The
gArgs references in chainstate.cpp are moved out to calling code in init.cpp in
later commits.

This commit does not change behavior.
2023-02-10 04:39:11 -04:00
Ryan Ofsky
c00fa1a734 refactor, txdb: Add CoinsViewOptions struct
Add CoinsViewOptions struct to remove ArgsManager uses from txdb.

To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in validation.cpp. But these moves are temporary. The
gArgs references in validation.cpp are moved out to calling code in init.cpp in
later commits.

This commit does not change behavior.
2023-02-10 04:39:11 -04:00
Ryan Ofsky
2eaeded37f refactor, dbwrapper: Add DBParams and DBOptions structs
Add DBParams and DBOptions structs to remove ArgsManager uses from dbwrapper.

To reduce size of this commit, this moves references to gArgs variable out of
dbwrapper.cpp to calling code in txdb.cpp. But these moves are temporary. The
gArgs references in txdb.cpp are moved out to calling code in init.cpp in later
commits.

This commit does not change behavior.
2023-02-10 04:39:11 -04:00
merge-script
ffdf630790 Merge bitcoin/bitcoin#27066: test: Fix intermittent sync issue in wallet_pruning
fa6f67837b test: Fix intermittent sync issue in wallet_pruning (721217.xyz)

Pull request description:

  Setting the mocktime on each loop iteration will make net processing racy and cause a disconnect due to timeout.

  Fix that by setting the mocktime only once.

  Fixes https://github.com/bitcoin/bitcoin/issues/27065

ACKs for top commit:
  brunoerg:
    crACK fa6f67837b

Tree-SHA512: 128b962c05a6fa3caf3ce392e870fff6609ce2206a43bbae6661ecb45291df93bed77fe362a514d4472056f83fb6631df39a5170fa34e41a7577b9685dd26b1f
2023-02-10 09:35:40 +01:00
Jon Atack
4275195606 De-duplicate add_coin methods to a test util helper 2023-02-09 15:03:36 -08:00
Jon Atack
9d92c3d7f4 Create InsecureRandMoneyAmount() test util helper
to generate semi-random CAmounts up to MAX_MONEY rather
than only uint32, and use it in the unit tests.
2023-02-09 15:03:36 -08:00
Sebastian Falbesoner
772671245d test: p2p: check that headers message with invalid proof-of-work disconnects peer 2023-02-10 00:01:04 +01:00
Hennadii Stepanov
d66efa30cd ci: Fix fingerprint_script for depends subdir caches 2023-02-09 22:20:42 +00:00
brunoerg
2d955ff006 net: add Ensure{any}Banman
it adds `Ensure{any}Banman` functions to avoid
code repetition and make it cleaner. Similar
approach as done with argsman, chainman, connman
and others.
2023-02-09 17:14:01 -03:00
Hennadii Stepanov
1313b90735 Merge bitcoin-core/gui#701: Persist Mask Values option
4de02def84 qt: Persist Mask Values option (Andrew Chow)

Pull request description:

  The mask values option is memory only. If a user has enabled this option, it's reasonable to expect that they would want to have it enabled on the next start.

ACKs for top commit:
  RandyMcMillan:
    tACK 4de02def84
  jarolrod:
    tACK 4de02def84
  pablomartin4btc:
    > tACK [4de02de](4de02def84)
  john-moffett:
    tACK 4de02def84

Tree-SHA512: 247deb78df4911516625bf8b25d752feb480ce30eb31335cf9baeb07b7c6c225fcc37d5c45de62d6e6895ec10c7eefabb15527e3c9723a3b8ddda1e12ebbf46b
2023-02-09 20:11:11 +00:00
merge-script
af2bd07587 Merge bitcoin/bitcoin#27062: ci: Re-introduce depends_built cache back in macOS and Android tasks
73a3b161b7 ci: Inline `MACOS_NATIVE_TASK_TEMPLATE` (Hennadii Stepanov)
8a61527cf6 ci: Re-introduce `depends_built` cache back in macOS and Android tasks (Hennadii Stepanov)

Pull request description:

  This PR brings a `depends_built` cache back to the "macOS 10.15" and "ARM64 Android APK" CI tasks.

  Fixes #27031.

ACKs for top commit:
  MarcoFalke:
    reACK 73a3b161b7 🌻

Tree-SHA512: 2eb845f865ee2ee453c1fd284d5eeddbebb2653586b17946822fec03d46e73c5eb483499761a0de6c3c466b06623957664e22dee01f7312ad18e212f1c9c6439
2023-02-09 19:05:49 +01:00
Hennadii Stepanov
73a3b161b7 ci: Inline MACOS_NATIVE_TASK_TEMPLATE 2023-02-09 12:11:47 +00:00
Hennadii Stepanov
8a61527cf6 ci: Re-introduce depends_built cache back in macOS and Android tasks 2023-02-09 12:11:26 +00:00
721217.xyz
fa6f67837b test: Fix intermittent sync issue in wallet_pruning 2023-02-09 12:28:56 +01:00
merge-script
dc905f6c2a Merge bitcoin/bitcoin#27063: ci: Use the latest Ubuntu LTS for "ARM64 Android APK" task
887bb53b67 ci: Use the latest Ubuntu LTS for "ARM64 Android APK" task (Hennadii Stepanov)

Pull request description:

  Suggested in https://github.com/bitcoin/bitcoin/pull/25797#discussion_r1100172227:

  >  I don't expect that anyone is building for android, and if they did, it should be fine to just require the latest Ubuntu LTS, which is Jammy

ACKs for top commit:
  fanquake:
    ACK 887bb53b67 - but I'd also suggest we remove this task entirely, and either replacing it with another task, maybe a *BSD, or delegating the resources to other jobs.

Tree-SHA512: 1f4b6155e5bbb8ca3580809c5999e3abf6b15b409d164a719b0a89205ca48c178aa6401039a22151ce464009adc48ba272a5a2ff05dc3ca06d3b2d64c99e3e22
2023-02-09 10:44:48 +01:00
fanquake
835af48e03 Merge bitcoin/bitcoin#27057: build: set boost cppflags with --enable-fuzz
b03a98291b build: set boost cppflags with --enable-fuzz (fanquake)

Pull request description:

  Even though all other targets are disabled, we still need Boost CPPFLAGS (`use_boost`) to compile. This currently works everywhere, except on arm macOS (where the include path is non-standard), because generally, the Boost include path is generic, i.e `/usr/include`.

ACKs for top commit:
  hebasto:
    ACK b03a98291b

Tree-SHA512: 7544a903ce641fd4b994ae51423a7007de85628ae29be36362a5cbdd62f9b16ac0a62e9edadaaa998ad4c1e82c0fde0d8c53aba41f94ad30ffa9f10ba0984521
2023-02-08 17:09:40 +00:00
fanquake
b03a98291b build: set boost cppflags with --enable-fuzz
Even though all other targets are disabled, we still need Boost CPPFLAGS
(use_boost) to compile. This currently works everywhere, except on arm
macOS (where the include path is pretty non-standard), because
generally, the Boost include path is generic, i.e `/usr/include`.
2023-02-08 16:10:34 +00:00
merge-script
07a23b4946 Merge bitcoin/bitcoin#26994: depends: define __BSD_VISIBLE for FreeBSD bdb build
0e02f72548 depends: define `__BSD_VISIBLE` for FreeBSD bdb build (fanquake)

Pull request description:

  Required for additional definitions (`IPC_R` & friends), to be available, when compiling under C11, which would otherwise cause compile fails.

  See: https://github.com/MarcoFalke/btc_nightly/pull/4.

ACKs for top commit:
  hebasto:
    ACK 0e02f72548, tested on FreeBSD 13.1:

Tree-SHA512: 885d4aa341d9668da360cf6dfafb97ce816803c54e76c0a06e448db39a723666d42cd14b3e713d17ecbe33163f5af69924567cf449d679a2db95b36357005d43
2023-02-08 16:37:09 +01:00
fanquake
7b850bc2a1 Merge bitcoin/bitcoin#27061: doc: Document affected gcc versions for -fstack-reuse=none workaround
fa83005a26 doc: Document affected gcc versions for -fstack-reuse=none workaround (MarcoFalke)

Pull request description:

  gcc version(s) 11 and prior won't be fixed, looking at the activity in the bug report. So it seems best to just document gcc 12.1+ as fixed, so that in the future the workaround can be removed once the minimum compiler is gcc12.1.

ACKs for top commit:
  fanquake:
    ACK fa83005a26
  hebasto:
    re-ACK fa83005a26

Tree-SHA512: a19723457eb1925196828a5fafd4e7f75a04f86ffae63cb86679d732c662fd1a22e17fe3c69195a97438ff189ba3ff681be3650cf99aa195d7a3e89cd8ee137c
2023-02-08 14:59:43 +00:00
Hennadii Stepanov
887bb53b67 ci: Use the latest Ubuntu LTS for "ARM64 Android APK" task 2023-02-08 14:15:13 +00:00
MarcoFalke
fa83005a26 doc: Document affected gcc versions for -fstack-reuse=none workaround 2023-02-08 13:06:11 +01:00
MarcoFalke
de1d189534 Merge bitcoin/bitcoin#27056: doc: use arch agnostic clang path in fuzzing doc (macOS)
b49e19ccd9 doc: use arch agnostic clang path in fuzzing doc (macOS) (fanquake)

Pull request description:

  The current path will only work for clang installed via brew on x86_64 macOS.

ACKs for top commit:
  hebasto:
    ACK b49e19ccd9, similar to 702836530f.

Tree-SHA512: 8ae4845e1953d5a7178f2b422e2241af1057d8cce1ab79da65df0cd068456dbf85da3489355f81fc4ee09ba602a4b53e989e2dc02476b4abf6c5b3bc3e96473b
2023-02-08 12:49:49 +01:00
MarcoFalke
51d51d3082 Merge bitcoin/bitcoin#26507: test: remove unused vars in feature_block
741c215b5f test: remove unused vars in `feature_block` (brunoerg)

Pull request description:

  There is no need to assign `self.next_block` to variables if we're not using its return value. Most cases touched here, we're reassigning it right after with the value from `self.update_block`.

Top commit has no ACKs.

Tree-SHA512: 25bbea2a09f38c3a3483fa363f024d2a8edd06a00cccc93cef99e489b9a3485d58bbd6a1ed2dddc00f1cebec7e63aed8ad95701a2645ce20a0db9b69573c20a7
2023-02-08 11:42:22 +01:00
fanquake
8d69b614cc Merge bitcoin/bitcoin#23810: docs: avoid C-style casts; use modern C++ casts
75347236f2 docs: document c-style cast prohibition (Pasta)

Pull request description:

  In the words of practicalswift:
  ```
  A C-style cast is equivalent to try casting in the following order:

      const_cast(...)
      static_cast(...)
      const_cast(static_cast(...))
      reinterpret_cast(...)
      const_cast(reinterpret_cast(...))

  By using static_cast<T>(...) explicitly we avoid the possibility of an unintentional and
  dangerous reinterpret_cast. Furthermore static_cast<T>(...) allows for easier grepping of casts.

  For a more thorough discussion, see "ES.49: If you must use a cast, use a named cast"
  in the C++ Core Guidelines (Stroustrup & Sutter).
  ```

  Modern tooling, specifically `-Wold-style-cast` can enable us to enforce never using C-style casts. I believe this is especially important due to the number of C-style casts the codebase is currently being used as a reinterpret_cast. reinterpret_casts are especially dangerous, and should never be done via C-style casts.

  Update the docs to suggest the use of named cast or functional casts.

Top commit has no ACKs.

Tree-SHA512: 29a98de396f0c78e32d8a1831319162203c4405a670da5add5da956fcc7df200a1cec162ef1cfac4ddfb02714b66406081d40ed435c7f0f28581cfa24d94fac1
2023-02-08 10:36:43 +00:00
MarcoFalke
5fffff54e9 ci: Cache stuff in volumes, not host folders 2023-02-08 11:21:23 +01:00
fanquake
b49e19ccd9 doc: use arch agnostic clang path in fuzzing doc (macOS) 2023-02-08 09:45:04 +00:00
MarcoFalke
1bcabe6f2a Merge bitcoin-core/gui#697: Remove reindex special case from the progress bar label
faff2ba4f8 Remove reindex special case from the progress bar label (MarcoFalke)

Pull request description:

  The user knows which option they passed to the program, so it seems overly verbose to offer the user feedback whether or not they passed `-reindex`. Treat it as `DISK`, like all other cases that are treated as `DISK`:

  * `-reindex-chainstate`
  * `-loadblock`

ACKs for top commit:
  john-moffett:
    Re-ACK faff2ba4f8
  hebasto:
    ACK faff2ba4f8, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 7f110c4beb1451d26f32da3a60150dac91c8a7b8d1c01749017204712b73cc1b77578af492930e4b6704097a73ed051f77bc39d8f60e0ff15a797a201805312e
2023-02-07 16:49:13 +01:00
fanquake
fe86616bb4 Merge bitcoin/bitcoin#26421: build: copy config.{guess,sub} post autogen in zmq package
1914e470e3 build: copy config.{guess,sub} post autogen in zmq package (fanquake)

Pull request description:

  Otherwise our config.guess and config.sub will be copied over. This problem has been masked by the fact that modern systems ship with versions that recognise all the triplets we use (namely arm64-apple-darwin). However building on ubuntu 20.04 surfaces the issue.

  Fixes #26420.

ACKs for top commit:
  hebasto:
    ACK 1914e470e3, tested on Ubuntu 18.04.

Tree-SHA512: dff64c3c62d9f8fc205e5a4dffe8befd58838418d073a15dfe304a0f64b182dfffd9dcf98b53df44bfab905c12a62d03cd5c0f91fa7c4b246ac21ae5f20540fd
2023-02-07 12:01:30 +00:00
fanquake
ab5b26f072 Merge bitcoin/bitcoin#21995: build: Make dependency package archive timestamps deterministic
6ebe57622c build: Make dependency package archive timestamps deterministic (Hennadii Stepanov)

Pull request description:

  This PR makes testing changes like bitcoin/bitcoin#20641, bitcoin/bitcoin#21593, bitcoin/bitcoin#22142, bitcoin/bitcoin#24279, bitcoin/bitcoin#24285 as easy as comparing hashes.

  With this PR:
  ```
  $ make -C depends clean
  $ make -C depends HOST=x86_64-w64-mingw32
  $ find depends/built/x86_64-w64-mingw32 -name '*.hash' | sort | xargs cat
  1f685a61cbf205f81977ecf88cba91fa1ccdfbe77ab4ec3405dcd33ceb778af4  bdb-4.8.30-ca950bd6d13.tar.gz
  08a9acde276e6e5e5c8913e3ad07eeecda184a996882ae226b3ed056c7ec1b01  boost-1.80.0-b537c466dcb.tar.gz
  144c6d92e4108fcc90740bee27007db58a88336a97be6367f9c8ba4cc208af27  libevent-2.1.12-stable-e13b2bdd8b8.tar.gz
  e3c9c9609bf32bfd460432c6ab99a64e9f8750ed775a193925ff4f5aed363e4c  libnatpmp-07004b97cf691774efebe70404cf22201e4d330d-82255b84667.tar.gz
  62c6a089a4b24a413eccd2f389bf4c8b0716423b0ace5e87e984069635da9f83  miniupnpc-2.2.2-c43fc4cf2f6.tar.gz
  78762700066273e597698a78479a506b33532ea565d18ef561614b9fc3820cf5  qrencode-3.4.4-663de0dc628.tar.gz
  5e2183faf91838510a48e6dbb4b65ae74a7d48ba1abc070b82767c4076582360  qt-5.15.5-986926343e2.tar.gz
  9f8459f8d27fc3af9146712be6ba6577f15741429936504a950cc51c17da1ba8  sqlite-3380500-bec6a4d3299.tar.gz
  0eca5d01d427de50be4bd57c8bb100ab69b017792c32b8733e2b20443f4c9c28  zeromq-4.3.4-8ae81bab6f4.tar.gz
  ```

  As an example, here is an evidence that bitcoin/bitcoin#24279 is a strict refactoring change:
  ```
  $ git fetch origin pull/24279/head
  $ git cherry-pick 706026838d917a3d853e03e83db040f1fd4aeb74
  $ git cherry-pick 3f90ddea8a6a2061cfb347a1d77df2c0a6fa238c
  $ make -C depends clean
  $ make -C depends HOST=x86_64-w64-mingw32
  $ find depends/built/x86_64-w64-mingw32 -name '*.hash' | sort | xargs cat
  1f685a61cbf205f81977ecf88cba91fa1ccdfbe77ab4ec3405dcd33ceb778af4  bdb-4.8.30-c7faf31d5ca.tar.gz
  08a9acde276e6e5e5c8913e3ad07eeecda184a996882ae226b3ed056c7ec1b01  boost-1.80.0-1af3dd1d99e.tar.gz
  144c6d92e4108fcc90740bee27007db58a88336a97be6367f9c8ba4cc208af27  libevent-2.1.12-stable-6228a9f8534.tar.gz
  e3c9c9609bf32bfd460432c6ab99a64e9f8750ed775a193925ff4f5aed363e4c  libnatpmp-07004b97cf691774efebe70404cf22201e4d330d-41aa6194ecc.tar.gz
  62c6a089a4b24a413eccd2f389bf4c8b0716423b0ace5e87e984069635da9f83  miniupnpc-2.2.2-6a93027769c.tar.gz
  78762700066273e597698a78479a506b33532ea565d18ef561614b9fc3820cf5  qrencode-3.4.4-d40cb2d45c9.tar.gz
  5e2183faf91838510a48e6dbb4b65ae74a7d48ba1abc070b82767c4076582360  qt-5.15.5-120c3cb745d.tar.gz
  9f8459f8d27fc3af9146712be6ba6577f15741429936504a950cc51c17da1ba8  sqlite-3380500-bbd4d813c69.tar.gz
  0eca5d01d427de50be4bd57c8bb100ab69b017792c32b8733e2b20443f4c9c28  zeromq-4.3.4-df0858a19d2.tar.gz
  ```

ACKs for top commit:
  TheCharlatan:
    Code review ACK 6ebe57622c

Tree-SHA512: 20e0222781f5dcb50126c11677d0671bcdd7be144b2e528c75a02983acc494206552fb35039697ccd094de27a21b3fb439e9965c34feb8a6d74627fa20a9a5e7
2023-02-07 11:59:11 +00:00
fanquake
6e08e5cb5c Merge bitcoin/bitcoin#17127: util: Set safe permissions for data directory and wallets/ subdir
c9ba4f9ecb test: Add test for file system permissions (Hennadii Stepanov)
581f16ef34 Apply default umask in `SetupEnvironment()` (Hennadii Stepanov)
8a6219e543 Remove `-sysperms` option (Hennadii Stepanov)

Pull request description:

  On master (1e7564eca8) docs say:
  ```
  $ ./src/bitcoind -help | grep -A 3 sysperms
    -sysperms
         Create new files with system default permissions, instead of umask 077
         (only effective with disabled wallet functionality)

  ```

  Basing on that, one could expect that running `bitcoind` first time will create data directory and `wallets/` subdirectory with safe 0700 permissions.

  But that is not the case:
  ```
  $ stat .bitcoin | grep id
  Access: (0775/drwxrwxr-x)  Uid: ( 1000/ hebasto)   Gid: ( 1000/ hebasto)
  $ stat .bitcoin/wallets | grep id
  Access: (0775/drwxrwxr-x)  Uid: ( 1000/ hebasto)   Gid: ( 1000/ hebasto)
  ```

  Both directories, in fact, are created with system default permissions.

  With this PR:
  ```
  $ stat .bitcoin/wallets | grep id
  Access: (0700/drwx------)  Uid: ( 1000/ hebasto)   Gid: ( 1000/ hebasto)
  $ stat .bitcoin/wallets | grep id
  Access: (0700/drwx------)  Uid: ( 1000/ hebasto)   Gid: ( 1000/ hebasto)
  ```

  ---

  This PR:
  - is alternative to bitcoin/bitcoin#13389
  - fixes bitcoin/bitcoin#15902
  - fixes bitcoin/bitcoin#22595
  - closes bitcoin/bitcoin#13371
  - reverts bitcoin/bitcoin#4286

  Changes in behavior: removed `-sysperms` command-line argument / configure option. The related discussions are here:
  - https://github.com/bitcoin/bitcoin/pull/13389#issuecomment-395306690
  - https://github.com/bitcoin/bitcoin/pull/13389#issuecomment-539906114
  - https://github.com/bitcoin/bitcoin/pull/13389#discussion_r279160472

  If users rely on non-default access permissions, they could use `chmod`.

ACKs for top commit:
  john-moffett:
    ACK c9ba4f9ecb
  willcl-ark:
    ACK c9ba4f9ecb

Tree-SHA512: 96c745339e6bd0e4d7bf65daf9a721e2e1945b2b0ab74ca0f66576d0dc358b5de8eb8cdb89fe2160f3b19c39d2798bb8b291784316085dc73a27102d3415bd57
2023-02-07 10:44:40 +00:00
MarcoFalke
faff2ba4f8 Remove reindex special case from the progress bar label 2023-02-07 11:02:01 +01:00
laanwj
aafa5e945c Remove laanwj from trusted-keys
allow-revsig-commits list generated using:

    git log --format="%H %ce" --merges 577bd51a4b8de066466a445192c1c653872657e2..master | grep laanwj | cut -c -40 >> allow-revsig-commits

Tree-SHA512: e665d1f3f6ae45ad435cb2802d49988f5133d695b145aa2dc65af95c052e562e0afaf585c351a41529985b4229965cf555f7197a44c90ba7daaea7a28975648d
2023-02-07 10:12:49 +01:00
MarcoFalke
5a80086ec2 Merge bitcoin/bitcoin#26701: contrib: make DNS seeds file an argument in CLI (makeseeds)
1c07500dbb contrib: make DNS seeds file an argument in CLI (brunoerg)

Pull request description:

  Instead of using `makeseeds.py` this way:
  ```sh
  python3 makeseeds.py -a asmap-filled.dat < seeds_main.txt > nodes_main.txt
  ```

  We could use the DNS seeds file as an argument since it is a required one. It improves the way the script handles it when that file is missing as well as makes this script more friendly.
  E.g:
  ```sh
  python3 makeseeds.py -a asmap-filled.dat -s seeds_main.txt > nodes_main.txt
  ```

ACKs for top commit:
  vincenzopalazzo:
    ACK  1c07500dbb

Tree-SHA512: bddf728d5d376659155f5bbeb1fa0d42aa273ec4a0cf5687f4d3f3be85625f541d392f30008e3c9d2c65967cb882deb36af34330994727771be73c9adeb521e0
2023-02-07 10:03:36 +01:00
Jon Atack
81f5ade2a3 Move random test util code from setup_common to random
as many of the unit tests don't use this code
2023-02-06 12:26:04 -08:00
Andrew Chow
52ddbd52f9 Merge bitcoin/bitcoin#26345: refactor: modernize the implementation of uint256.*
935acdcc79 refactor: modernize the implementation of uint256.* (pasta)

Pull request description:

  - Constructors of uint256 to utilize Span instead of requiring a std::vector
  - converts m_data into a std::array
  - Prefers using `WIDTH` instead of `sizeof(m_data)`
  - make all the things constexpr
  - replace C style functions with c++ equivalents
      - memset -> std::fill
          This may also be replaced by std::memset, but I think that std::fill is more idiomatic of modern c++ and readable.
      - memcpy -> std::copy
          Note: In practice, implementations of std::copy avoid multiple assignments and use bulk copy functions such as std::memmove if the value type is TriviallyCopyable and the iterator types satisfy LegacyContiguousIterator. (https://en.cppreference.com/w/cpp/algorithm/copy)
          This could also likely be replaced by std::memcpy, but as said above, I believe the using std::copy is the more c++ way to do anything and is almost guaranteed to compile to the same asm
      - memcmp -> std::memcmp

ACKs for top commit:
  achow101:
    ACK 935acdcc79
  hebasto:
    Approach ACK 935acdcc79.
  aureleoules:
    reACK 935acdcc79
  john-moffett:
    ACK 935acdcc79
  stickies-v:
    Approach ACK 935acdcc7

Tree-SHA512: 4f1ba54ff2198eea0e505d41e73d552c84c60f6878d5c85a94a8ab57f39afc94ef8d79258e7afd01fa84ec2a99f4404bb877eecd671f65e1ee9273f3129fc650
2023-02-06 13:56:51 -05:00
Hennadii Stepanov
c9ba4f9ecb test: Add test for file system permissions 2023-02-06 11:08:57 +00:00
Hennadii Stepanov
581f16ef34 Apply default umask in SetupEnvironment()
This change makes all filesystem artifacts--files and directories--being
created with the default umask.
2023-02-06 11:08:03 +00:00
MarcoFalke
aff75463e2 Merge bitcoin/bitcoin#27036: test: Remove last uses of snprintf and simplify
b8032293e6 Remove use of snprintf and simplify (John Moffett)

Pull request description:

  These are the only remaining uses of `snprintf` in our project, and they can cause unexpected issues -- for example, see https://github.com/bitcoin/bitcoin/issues/27014. Change them to use our `ToString` (which uses a locale-independent version of `std::to_string`) to convert an `int` to `std::string`. Also remove resulting unused parts of `StringContentsSerializer`.

  Closes https://github.com/bitcoin/bitcoin/issues/27014

ACKs for top commit:
  Sjors:
    tACK b8032293e6, fixes #27014.

Tree-SHA512: c903977e654711929decafe8887d0de13b38a340d7082875acc5d41950d834dcfde074e9cabecaf5f9a760f62c34322297b4b156af29761650ef5803b1a54b59
2023-02-06 10:32:55 +01:00
fanquake
d8f9826037 Merge bitcoin/bitcoin#27030: Update nanobench to version v4.3.10
82f895d7b5 Update nanobench to version v4.3.10 (Martin Leitner-Ankerl)

Pull request description:

  Nothing has changed that would affect Bitcoin's usage of nanobench.

   Here is a detailed list of the changes
  * Plenty of clang-tidy updates
  * documentation updates
  * faster Rng::shuffle
  * Enable perf counters on older kernels
  * Raise default minimum epoch time to 1ms (doesn't effect bitcoin's usage)
  * Add support for custom information per benchmark

ACKs for top commit:
  hebasto:
    ACK 82f895d7b5, I've reviewed the code, all related changes from #26642 have been implemented.

Tree-SHA512: 942518398809a2794617a347ab8182b784a8e822e84de5af078b2531eabb438412d687cac22a21936585e60e07138a89b41c28c9750744c05a3d1053f55cad01
2023-02-05 15:16:16 +00:00
fanquake
8f4ae65818 Merge bitcoin/bitcoin#27009: validation: Skip VerifyDB checks of level >=3 if dbcache is too small
fe683f3524 log: Log VerifyDB Progress over multiple lines (Martin Zumsande)
61431e3a57 validation: Skip VerifyDB checks of level >=3 if dbcache is too small (Martin Zumsande)

Pull request description:

  This is the first two commits from #25574, leaving out all changes to `-verifychain` error-handling :

  - The Problem of [25563](https://github.com/bitcoin/bitcoin/issues/25563) is that when we skip blocks at level 3 due to an insufficient dbcache (skipping some `DisconnectBlock()` calls), we would still attempt the level 4 checks, attempting to reconnect a block that was never disconnected, leading to an assert in `ConnectBlock()`.
  Fix this by not attempting level 4 checks in this case.
  - Logging of verification progress is now split over multiple lines. This is more verbose, but now each update has its own timestamp, and other threads logging concurrently will no longer lead to mangled output.

  This can be tested with a small `dbcache` value, for example:
  `bitcoind -signet -dbcache=10`
  `bitcoin-cli -signet verifychain 4 1000`

  Fixes #25563

ACKs for top commit:
  MarcoFalke:
    review ACK fe683f3524 🗄
  john-moffett:
    ACK fe683f3524

Tree-SHA512: 3e2e0f8b73cbc518a0fa17912c1956da437787aab95001c110b01048472e0dfe4783c44df22bd903d198069dd2f6b02bfdf74e0b934c7a776f144c2e86cb818a
2023-02-05 13:28:05 +00:00
Hennadii Stepanov
8a6219e543 Remove -sysperms option
This change effectively reverts commits from
https://github.com/bitcoin/bitcoin/pull/4286.

Users, who rely on non-default access permissions, should use `chmod`
command.
2023-02-05 08:09:16 +00:00
Andrew Chow
d71b0e78eb Merge bitcoin/bitcoin#25966: test: Remove redundant test
fb1c6c14c1 test: Remove redundant test (yancy)

Pull request description:

  I can't think of any reason to keep this test case around labeled [fix me](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L242).  The test was originally added [here](4566ab75f2) however there was never an assertion about the coins that should be selected, only that a solution is found (which is a redundant solution to the test [above](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L222)).  The comment was later added here to [fix](384273260a) it, however it's unclear what exactly it's testing.  A test was later added [here](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L366) where if the [long term fee](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L357) is less than the current [fee](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L356), then select fewer UTXOs, which may have been the original intent.

ACKs for top commit:
  S3RK:
    ACK fb1c6c14c1
  Zero-1729:
    Concept ACK fb1c6c14c1
  achow101:
    ACK fb1c6c14c1

Tree-SHA512: bce2cdae669c144ffaa130237a1643e3b6728e13d603cebf5d9493c4c7c68b3635868e4d93d210783c2ded2a871f185ca09a2053168c05b26a1e056ff6edf68f
2023-02-03 17:32:46 -05:00
TheCharlatan
1b1ffbd014 Build: Log when test -f fails in Makefile
Silently emitting an error makes it a bit harder to debug. Instead,
print a helpful log message to point the developer in the right
direction.

Alternatively this could have been implemented by just removing the
recipe echo suppression (@), but the subsequent make output became too
noisy.
2023-02-03 22:33:14 +01:00
TheCharlatan
541012e621 Build: Use AM_V_GEN in Makefiles where appropriate
When generating new files as part of the Makefile the recipe is
sometimes suppressed with $(AM_V_GEN) and sometimes with `@`. We should
prefer $(AM_V_GEN), since this also prints the lines in silent mode.
This is arguably more in style with the current recipe echoing.

Before:
Generated test/data/script_tests.json.h
Now:
  GEN      test/data/script_tests.json.h

A side effect of this change is that the recipe for generating build.h
is now echoed on each make run. Arguably this makes its generation more
transparent.
2023-02-03 22:26:00 +01:00
Andrew Chow
e2ae5c349c Merge bitcoin/bitcoin#27037: rpc: decode Miniscript descriptor when possible in decodescript
6699d850e4 doc: release notes for #27037 (Antoine Poinsot)
dfc9acbf01 rpc: decode Miniscript descriptor when possible in decodescript (Antoine Poinsot)

Pull request description:

  The descriptor inference logic would previously always use a dummy signing provider and would never analyze the witness script of a P2WSH scriptPubKey.

  It's often not possible to infer a Miniscript only from the onchain Script, but it was such a low hanging fruit that it's probably worth having it?

  Fixes https://github.com/bitcoin/bitcoin/issues/27007. I think it also closes https://github.com/bitcoin/bitcoin/issues/25606.

ACKs for top commit:
  instagibbs:
    ACK 6699d850e4
  achow101:
    ACK 6699d850e4
  sipa:
    utACK 6699d850e4

Tree-SHA512: e592bf1ad45497e7bd58c26b33cd9d05bb3007f1e987bee773d26013c3824e1b394fe4903809d80997d5ba66616cc79d77850cd7e7f847a0efb2211c59466982
2023-02-03 15:34:38 -05:00
Hennadii Stepanov
2ccd7be26f Merge bitcoin-core/gui#653: Show watchonly balance only for Legacy wallets
fdb8dc8a5a gui: Show watchonly balance only for Legacy wallets (Andrew Chow)

Pull request description:

  Descriptor wallets do not have a watchonly balance as wallets are designated watchonly or not. Thus we should not be displaying the empty watchonly balance for descriptor wallets.

  The result is that instead of the send page showing "Watch-only balance: 0.00000000 BTC" for watchonly descriptor wallets, we see the actual balance as "Balance: 10.00000000 BTC"

ACKs for top commit:
  johnny9:
    tACK fdb8dc8a5a
  furszy:
    ACK fdb8dc8a
  hebasto:
    ACK fdb8dc8a5a

Tree-SHA512: e5c0703a62d25c881c8dadfb9cffd482791f3d437a4ec5ae0088ce1a2069c2455ad6d3ec6c95a4404a3b55fbd727f92694529c35052236951553ca90c4ed31b5
2023-02-03 19:18:30 +00:00
Hennadii Stepanov
daebf9ebb0 Merge bitcoin-core/gui#705: doc: Fix comment about how wallet txs are sorted
c497a198db Fix comment about how wallet txs are sorted (John Moffett)

Pull request description:

  The wallet transactions in the node are not sorted by txid (or any hash) since https://github.com/bitcoin/bitcoin/pull/24699.

  This is how they're stored in memory now:

  835212cd1d/src/wallet/wallet.h (L397-L399)

ACKs for top commit:
  achow101:
    ACK c497a198db
  jarolrod:
    ACK c497a198db

Tree-SHA512: e72559991688452ef254474d4235dc75fac655bce04909c3a0eece907360f4c6f57707db9b4373a4bd2271b23c57e863684c33e0728adf48e477c5499cdfdad7
2023-02-03 19:03:46 +00:00
John Moffett
b8032293e6 Remove use of snprintf and simplify
One test case uses snprintf to convert an
int to a string. Change it to use ToString
(which uses a locale-independent version of
std::to_string). Also remove unnecessary
parts of StringContentsSerializer.
2023-02-03 12:35:54 -05:00
Antoine Poinsot
6699d850e4 doc: release notes for #27037 2023-02-03 18:18:39 +01:00
Antoine Poinsot
dfc9acbf01 rpc: decode Miniscript descriptor when possible in decodescript
The descriptor inference logic would previously always use a dummy
signing provider and would never analyze the witness script of a P2WSH
scriptPubKey.

Note even a valid Miniscript might not always be decodable from Script
without more contextual information (for instance the key preimage for a
pk_h).
2023-02-03 18:15:42 +01:00
Sebastian Falbesoner
dee8549be3 test: simplify and speedup mempool_updatefromblock.py by using MiniWallet 2023-02-03 17:23:26 +01:00
Pieter Wuille
561848aaf2 Exercise non-DIRTY spent coins in caches in fuzz test 2023-02-03 10:33:31 -05:00
MarcoFalke
aaa55971f6 Merge bitcoin/bitcoin#26875: Tests: Fill out dust limit unit test for known types except bare multisig
b093f5619f Fill out dust limit unit test for known types except bare multisig (Greg Sanders)

Pull request description:

  Having the constants checked explicitly in a single spot helps with possible regressions and also useful for documentation.

  In addition, add a check for undefined v1 witness programs.

ACKs for top commit:
  theStack:
    Code-review ACK b093f5619f
  MarcoFalke:
    review ACK b093f5619f  🥉

Tree-SHA512: 1421f75471739d29b9ef59b0a925b6b07e4e9af92822dbe56eedfb590be9a00fb0c34312146c7c1b5211906461ed00bfa2eb53c88595c6e5a27694b2dc21df38
2023-02-03 13:57:08 +01:00
Martin Leitner-Ankerl
82f895d7b5 Update nanobench to version v4.3.10
Nothing has changed that would affect Bitcoin's usage of nanobench. Here is a detailed list of the changes
* Plenty of clang-tidy updates
* documentation updates
* faster Rng::shuffle
* Enable perf counters on older kernels
* Raise default minimum epoch time to 1ms (doesn't effect bitcoin's usage)
* Add support for custom information per benchmark
2023-02-03 07:08:28 +01:00
Matthew Zipkin
77192c9598 cli: include local ("unreachable") peers in -netinfo table 2023-02-02 13:14:48 -05:00
fanquake
7753efcbcf Merge bitcoin/bitcoin#27004: test: Use std::unique_ptr over manual delete in coins_tests
fab9f7d1bd test: Use std::unique_ptr over manual delete in coins_tests (MarcoFalke)

Pull request description:

  Makes the code smaller and easier to read

ACKs for top commit:
  stickies-v:
    ACK fab9f7d1bd
  john-moffett:
    ACK fab9f7d1bd

Tree-SHA512: 30d2d2097906e61fdef47a52fc6a0c5ce2417bc41c3c82dafc1b216c655f31dabf9c1c13759575a696f61bbdfdba3f442be032d5e5145b7a54fae2a927824621
2023-02-02 16:53:51 +00:00
fanquake
c2028f98ae Merge bitcoin/bitcoin#27012: ci: Print iwyu patch in git diff format
fa6986a66b ci: Print iwyu patch in git diff format (MarcoFalke)

Pull request description:

  Seems more dev friendly to also have a patch to copy-paste

ACKs for top commit:
  hebasto:
    ACK fa6986a66b, tested on Ubuntu 22.04 locally.
  fanquake:
    ACK fa6986a66b - did not test but example CI output looks ok.
  stickies-v:
    utACK fa6986a66b

Tree-SHA512: 7cfd8584bf12e03c28af23f4712c6bcafd648d87ddb92788b9cd35455b2db49f4bd4aef8ad4711f75c7f11ad2bb2492c2eb6044007086c20e36016575c060603
2023-02-02 15:48:20 +00:00
MarcoFalke
3fffff50f6 ci: Remove unused EXPECTED_TESTS_DURATION_IN_SECONDS env var 2023-02-02 16:26:16 +01:00
MarcoFalke
b3ef329199 Merge bitcoin/bitcoin#26976: ci: Cache package manager install step
fa486de212 ci: Cache package manager install step (MarcoFalke)

Pull request description:

  Use the local podman or docker image cache to skip the slow `apt` step

ACKs for top commit:
  jamesob:
    ACK fa486de212 ([`jamesob/ackr/26976.1.MarcoFalke.ci_cache_package_manager`](https://github.com/jamesob/bitcoin/tree/ackr/26976.1.MarcoFalke.ci_cache_package_manager))

Tree-SHA512: 3495346c6c862b63296d2691cc492bf52a0a99ee7fae798887c792609904546013eba788045cd508a5f669f2c52e3479c122c18a5275c87af38237a1b5c9da17
2023-02-02 16:09:23 +01:00
Pieter Wuille
59e6828bb5 Add deterministic mode to CCoinsViewCache 2023-02-02 09:00:15 -05:00
Hennadii Stepanov
ea41abade4 Merge bitcoin-core/gui#695: Fix misleading RPC console wallet message
576f7b8614 Fix misleading RPC console wallet message (John Moffett)

Pull request description:

  ## Misleading message from RPCConsole window ##

  In certain circumstances, the GUI console will display the message 'Executing command without any wallet' when it is, in fact, using the currently loaded wallet. For instance:

  ![scr3](https://user-images.githubusercontent.com/116917595/211404066-d49a6cbf-d3c3-4e89-8720-3583c6acf521.gif)

  In RPC calls, if no wallet is explicitly selected and there is exactly one wallet loaded, the [default](39363a4b94/src/wallet/rpc/util.cpp (L71-L93)) is to act on that loaded wallet.

  The GUI console acts that way in reality, but sometimes erroneously reports that it's not acting on any particular wallet. The root issue is due to the logic that prevents changing the selected wallet if the RPCConsole is visible:

  39363a4b94/src/qt/rpcconsole.cpp (L783-L786)

  This PR removes that unnecessary logic. This does have some ramifications. Prior to this PR, if a user opened the console window without any wallets loaded, then opened two or more wallets, the RPC console would select "None" of the wallets and any wallet-specific RPCs would fail. However, the behavior was different if the user hadn't had the console window open. In that case, if they opened the RPC Console window _after_ loading at least the first wallet, it would select the first-loaded wallet. This context-dependent behavior is (IMO) undesirable, and this PR changes it to be consistent.

ACKs for top commit:
  hebasto:
    ACK 576f7b8614, tested on Ubuntu 22.04 (Qt 5.15.3).

Tree-SHA512: 627da186025ba4f4e8df7fdd1b10363f923c4ecc50f023bbf2aece6e2593da65c45147c933effaca9040f813a6e46f034fc2d1ee2fb0f401000a3a6221a0e36e
2023-02-02 12:18:36 +00:00
Hennadii Stepanov
526f67a5ca Merge bitcoin-core/gui#704: Correctly limit overview transaction list
08209c039f Correctly limit overview transaction list (John Moffett)

Pull request description:

  Fixes #703

  The way the main overview page limits the number of transactions displayed (currently 5) is not an appropriate use of Qt. Our subclassed transaction sort/filter proxy model returns a maximum of `5` in `rowCount()`. However, the model itself actually may hold significantly more. While this has _worked_, it breaks the contract of `rowCount()`.

  If `bitcoin-qt` is run with a DEBUG build of Qt, it'll result in an assert-crash in certain relatively common situations (see #703 for details). Instead of artificially limiting the `rowCount()` in the subclassed filter, we can hide/unhide the rows in the displaying `QListView` upon any changes in the sorted proxy filter.

  I loaded a wallet with 20,000 transactions and did not notice any performance differences between master and this branch.

  For reference, this is the list I'm referring to:

  <img width="934" alt="image" src="https://user-images.githubusercontent.com/116917595/214947304-3f289380-3510-487b-80e8-d19428cf2f0f.png">

ACKs for top commit:
  Sjors:
    tACK 08209c039f
  hebasto:
    ACK 08209c039f, tested on Ubuntu 22.04.

Tree-SHA512: c2a7b1a2a6e6ff30694830d7c722274c4c47494a81ce9ef25f8e5587c24871b02343969f4437507693d4fd40ba7a212702b159cf54b3357d8d76c02bc8245113
2023-02-02 11:45:26 +00:00
fanquake
21138fe377 Merge bitcoin/bitcoin#26992: refactor: Remove unused CDataStream SerializeMany constructor
fa47b28dfc refactor: Remove unused CDataStream SerializeMany constructor (MarcoFalke)

Pull request description:

  Seems odd to have an unused method. Moreover, the function is fragile and dangerous, because one could have a `std::vector vec_a` and type `CDataStream{vec_a, 0, 0}.size()` and `CDataStream{0, 0, vec_a}.size()`, assuming they are the same thing, when in fact they are not. (The first takes over the memory as is, the second serializes the vector).

  So my suggestion would be to remove the unused method and introduce a new method when this functionality is needed. For example: `static DataStream FromMany(Args&&... args)`.

ACKs for top commit:
  stickies-v:
    ACK fa47b28dfc

Tree-SHA512: 9593a034b997e33a0794f779f76f02425b1097b218cf8cb1facb7f874fa69da328ce567a79138015baeebe004ae7d103dda4f64f83e8ad375b6dae6b66d3d950
2023-02-02 10:47:37 +00:00
fanquake
9dc50a5a07 Merge bitcoin/bitcoin#27005: util: Use steady clock for logging timer
fad7af700e Use steady clock for logging timer (MarcoFalke)

Pull request description:

  The logging timer has many issues:

  * The underlying clock is mockable, meaning that benchmarks are useless when mocktime was set at the beginning or end of the benchmark.
  * The underlying clock is not monotonic, meaning that benchmarks are useless when the system time was changed during the benchmark.

  Fix all issues in this patch.

ACKs for top commit:
  stickies-v:
    Approach ACK fad7af700e
  john-moffett:
    ACK fad7af700e

Tree-SHA512: bec8da0f338ed4611e1807937575e1b2afda25139d88015b1c29fa7d13946fbfbc4ee589b576c0508d505df5e5fafafcbc07d63ce4bab4b01475260d9d5d2107
2023-02-02 10:30:29 +00:00
MarcoFalke
102645280b Merge bitcoin/bitcoin#27013: ci: avoid using -Werror for older compilers
71383f2fad ci: avoid using -Werror for older compilers (fanquake)

Pull request description:

  Don't enable `-Werror` (in the CI) for compilers at least older than our current release compiler (GCC 10). It provides little-to-no value, other than turning compiler bugs & false positives into build failures, and we aren't going to mutate perfectly fine/working code, for the sake of avoid a warning that shouldn't even exist.

  I also do not see the point of playing whack-a-mole and turning off various warnings/trying to further work around the broken compiler, just to acheive warningless builds for the sake of warningless builds.

  One anecdote from ["How SQLite Is Tested"](https://www.sqlite.org/testing.html):
  > Static analysis has found a few bugs in SQLite, but those are the
  > exceptions. More bugs have been introduced into SQLite while trying
  > to get it to compile without warnings than have been found by static
  > analysis.

ACKs for top commit:
  hebasto:
    ACK 71383f2fad
  jarolrod:
    ACK 71383f2fad

Tree-SHA512: 20ed3dcf54fb17a7d9f0d8ca68c0ad2ee8f171f8bd61673a428f3123ab322c24cd9833f65915489bc8cebeffc37fd683a30e9669684b219960e69ddc7adae5bd
2023-02-02 10:40:40 +01:00
Pieter Wuille
b0ff310840 Add CCoinsViewCache::SanityCheck() and use it in fuzz test 2023-02-01 23:14:12 -05:00
Pieter Wuille
3c9cea1340 Add simulation-based CCoinsViewCache fuzzer
The fuzzer goes through a sequence of operations that get applied to both a
real stack of CCoinsViewCache objects, and to simulation data, comparing
the two at the end.
2023-02-01 18:28:41 -05:00
Andrew Chow
fdd363ebd9 Merge bitcoin/bitcoin#26910: wallet: migrate wallet, exit early if no legacy data exist
6d31900e52 wallet: migrate wallet, exit early if no legacy data exist (furszy)

Pull request description:

  The process first creates a backup file then return an error,
  without removing the recently created file, when notices that
  the db is already running sqlite.

ACKs for top commit:
  john-moffett:
    ACK 6d31900e52
  achow101:
    ACK 6d31900e52
  ishaanam:
    crACK 6d31900e52

Tree-SHA512: 9fb52e80de96e129487ab91bef13647bc4570a782003b1e37940e2a00ca26283fd24ad39bdb63a984ae0a56140b518fd0d74aa2fc59ab04405b2c179b7d3c54a
2023-02-01 17:14:13 -05:00
fanquake
b3b673f704 mapport: require miniupnpc API version 17 or later
Version 17 is currently the latest version, and has been available since
the release of 2.1.
See: https://github.com/miniupnp/miniupnp/blob/master/miniupnpc/apiversions.txt.
2023-02-01 15:57:26 +00:00
fanquake
2d5acc901d Merge bitcoin/bitcoin#27015: p2p: 26847 fixups (AddrMan totals)
dc70c1eb08 addrman: Use std::nullopt instead of {} (Martin Zumsande)
59cc66abb9 test: Remove AddrMan unit test that fails consistency checks (Martin Zumsande)

Pull request description:

  Two fixups for #26847:
  * Now that `AddrMan::Size()` performs internal consistency tests (it didn't before), we can't call it in the `load_addrman_corrupted` unit tests, where we deal with an artificially corrupted `AddrMan`. This would fail the test when using `-checkaddrman=1` (leading to spurious CI fails). Therefore remove the tests assertion, which is not particularly helpful anyway (in production we abort init when peers.dat is corrupted instead of querying AddrMan in its corrupted state).
   (See https://github.com/bitcoin/bitcoin/pull/26847#issuecomment-1411458339)
  * Use `std::nullopt` instead of `{}` for default args (suggested in https://github.com/bitcoin/bitcoin/pull/26847#discussion_r1090643603)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK dc70c1eb08

Tree-SHA512: dd8a988e23d71a66d3dd30560bb653c9ad17db6915abfa5f722818b0ab18921051ec9223bfbc75d967df8bcd204dfe473d680bf68e8a8e4e4998fbb91dc973c5
2023-02-01 15:56:48 +00:00
fanquake
550e6bd227 Merge bitcoin/bitcoin#26935: refactor: Fix clang-tidy readability-const-return-type violations
fa451d4b60 Fix clang-tidy readability-const-return-type violations (MarcoFalke)

Pull request description:

  This comes up during review, so instead of wasting review cycles on this, just enforce it via CI

ACKs for top commit:
  stickies-v:
    utACK fa451d4b6
  hebasto:
    ACK fa451d4b60.

Tree-SHA512: 144a85612f00ec43f7ea1fdaa11901ca981a9f0465a8849745712d741b201b9c3307118172ee0b8efd12bebf25bc6f32a6e2c908495e371f9ada0a917994f44e
2023-02-01 15:53:35 +00:00
fanquake
71383f2fad ci: avoid using -Werror for older compilers
Don't enable `-Werror` (in the CI) for compilers at least older than
our current release compiler (GCC 10). It provides little-to-no value,
other than turning compiler bugs & false positives into build failures,
and we aren't going to mutate perfectly fine/working code, for the sake
of avoid a warning that shouldn't even exist.

I also do not see the point of playing whack-a-mole and turning off various
warnings/trying to further work around the broken compiler, just to
acheive warningless builds for the sake of warningless builds.

One anecdote from "How SQLite Is Tested":
> Static analysis has found a few bugs in SQLite, but those are the
> exceptions. More bugs have been introduced into SQLite while trying
> to get it to compile without warnings than have been found by static
> analysis.

https://www.sqlite.org/testing.html.
2023-02-01 15:22:24 +00:00
Martin Zumsande
dc70c1eb08 addrman: Use std::nullopt instead of {} 2023-02-01 10:18:08 -05:00
Martin Zumsande
59cc66abb9 test: Remove AddrMan unit test that fails consistency checks
Now that Size() performs internal consistency checks,
it will rightfully fail (and assert) when dealing with
a corrupted AddrMan. Therefore remove this check.
2023-02-01 10:14:20 -05:00
MarcoFalke
8fc3bcf93d Merge bitcoin/bitcoin#27010: refactor: use Hash helpers for double-SHA256 calculations
87f11ef47f refactor: use `Hash` helper for double-SHA256 calculations (Sebastian Falbesoner)

Pull request description:

  We have two helper templates `Hash(const T& in1)` and `Hash(const T& in1, const T& in2)` available for calculating the double-SHA256 hash of one object or two concatenated objects, respectively:

  b5868f4b1f/src/hash.h (L74-L89)

  This PR uses them in order to increase readability and simplify the code. As in #15294 (which inspired this PR, doing the same for RIPEMD160), the helper is not utilized in validation.cpp and  script/interpreter.cpp to avoid touching consensus-relevant code.

ACKs for top commit:
  john-moffett:
    ACK 87f11ef47f
  stickies-v:
    ACK 87f11ef47f
  MarcoFalke:
    review ACK 87f11ef47f  😬

Tree-SHA512: 11d7e3d00c89685107784010fbffb33ccafb4d1b6a76c4dceb937b29bb234ef4d54581b16bd0737c8d2994a90cf4fe10a9738c7cc5b6d085c6a819f06176dab9
2023-02-01 15:56:30 +01:00
MarcoFalke
fa6986a66b ci: Print iwyu patch in git diff format 2023-02-01 14:11:48 +01:00
glozow
22ccf4e360 Merge bitcoin/bitcoin#26991: doc: followups to #26471
47c174d8ce doc: NetPermissionFlags for tx relay in blocksonly (willcl-ark)
e325e0fccb doc: Fix comment syntax error (willcl-ark)

Pull request description:

  Fix syntax error and specify `NetPermissionFlags` for whitelisted tx relay

ACKs for top commit:
  w0xlt:
    ACK 47c174d8ce

Tree-SHA512: eb579dc599a96a3ea79c01ac3e76160ec59cf71c2486c9401da8fbbd96ae756ba647aa9ba874835946bc76ba02782729da788617f982ae5a852139e10e7dfd75
2023-02-01 11:46:22 +00:00
fanquake
17acbc1a5a Merge bitcoin/bitcoin#25974: test, build: Separate read_json function into its own module
7a820cee0e test, build: Separate `read_json` function into its own module (Hennadii Stepanov)

Pull request description:

  Currently, 4 source files rely on the definition of the `read_json` function provided in `src/test/script_tests.cpp`.

  This PR breaks this entanglement, improves code structure and maintainability.

ACKs for top commit:
  fanquake:
    ACK 7a820cee0e

Tree-SHA512: f1567989f76cb54ab86cc48927851a8c424b08a9483d02d4918b629e0c792108bad4ccf7fa341d57b0921d91e84bf8fa3b9c07e5fdf12c64d9d5da83e4e464fb
2023-02-01 11:43:42 +00:00
MarcoFalke
fa451d4b60 Fix clang-tidy readability-const-return-type violations 2023-02-01 11:33:35 +01:00
MarcoFalke
e1bf5470f9 Merge bitcoin/bitcoin#26705: clang-tidy: Fix modernize-use-default-member-init in headers and force to check all headers
b0e916913c clang-tidy: Force to check all headers (Hennadii Stepanov)
96ee992ac3 clang-tidy: Fix `modernize-use-default-member-init` in headers (Hennadii Stepanov)

Pull request description:

  This PR:
  - fixes the only [remained](https://github.com/bitcoin/bitcoin/pull/26705#issuecomment-1353742082) check in headers, i.e., `modernize-use-default-member-init`
  - forces `clang-tidy` check all headers

  Closes bitcoin/bitcoin#26703.

ACKs for top commit:
  MarcoFalke:
    review ACK b0e916913c 🍹

Tree-SHA512: 4d33fe873094914541ae81968cdb4e7a7a01b3fdd4f25bc6daa8a53f45dab80565a5b3607ddc338f122369ca5a0a2d0d09c8e78cabe1beb6bd50c115bc5c5210
2023-02-01 10:38:45 +01:00
MarcoFalke
ba39ffe938 Merge bitcoin/bitcoin#26888: net: simplify the call to vProcessMsg.splice()
dfc01ccd73 net: simplify the call to vProcessMsg.splice() (Vasil Dimov)

Pull request description:

  At the time when

  ```cpp
  pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), it);
  ```

  is called, `it` is certainly `pnode->vRecvMsg.end()` which makes the call equivalent to:

  ```cpp
  pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), pnode->vRecvMsg.end());
  ```

  which is equivalent to:

  ```cpp
  pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg);
  ```

  Thus, use the latter. Further, maybe irrelevant, but the latter has constant complexity while the original code is `O(length of vRecvMsg)`.

ACKs for top commit:
  theStack:
    Code-review ACK dfc01ccd73
  MarcoFalke:
    review ACK dfc01ccd73   🐑
  jonatack:
    Light review ACK dfc01ccd73

Tree-SHA512: 9f4eb61d1caf4af9a61ba2f54b915fcfe406db62c58ab1ec42f736505b6792e9379a83d0458d6cc04f289edcec070b7c962f94a920ab51701c3cab103152866f
2023-02-01 09:42:46 +01:00
Andrew Chow
ba3d32715f Merge bitcoin/bitcoin#26847: p2p: track AddrMan totals by network and table, improve precision of adding fixed seeds
80f39c99ef addrman, refactor: combine two size functions (Amiti Uttarwar)
4885d6f197 addrman, refactor: move count increment into Create() (Martin Zumsande)
c77c877a8e net: Load fixed seeds from reachable networks for which we don't have addresses (Martin Zumsande)
d35595a78a addrman: add function to return size by network and table (Martin Zumsande)

Pull request description:

  AddrMan currently doesn't track the number of its entries by network, it only knows the total number of addresses. This PR makes AddrMan keep track of these numbers, which would be helpful for multiple things:

  1. Allow to specifically add fixed seeds to AddrMan of networks where we don't have any addresses yet - even if AddrMan as a whole is not empty (partly fixing #26035). This is in particular helpful if the user abruptly changes `-onlynet` settings (such that addrs that used to be reachable are no longer and vice versa), in which case they currently could get stuck and not find any outbound peers. The second commit of this PR implements this.
  1. (Future work): Add logic for automatic connection management with respect to networks - such as making attempts to have at least one connection to each reachable network as suggested [here](https://github.com/bitcoin/bitcoin/issues/26035#issuecomment-1249420209). This would involve requesting an address from a particular network from AddrMan, and expanding its corresponding function `AddrMan::Select()`  to do this requires internal knowledge of the current number of addresses for each network and table to avoid getting stuck in endless loops.
  1. (Future work): Perhaps display the totals to users. At least I would find this helpful to debug, the existing option (`./bitcoin-cli -addrinfo`) is rather indirect by doing the aggregation itself in each call, doesn't distinguish between new and tried, and being based on `AddrMan::GetAddr()` it's also subject to a quality filter which we probably don't want in this spot.

ACKs for top commit:
  naumenkogs:
    utACK 80f39c9
  stratospher:
    ACK 80f39c9
  achow101:
    ACK 80f39c99ef
  vasild:
    ACK 80f39c99ef

Tree-SHA512: 6359f2e3f4db7c120c0789d92d74cb7d87a2ceedb7d6a34b5eff20c7f55c5c81092d10ed94efe29afc1c66947820a0d9c14876ee0c8d1f8e068a6df4e1131927
2023-01-31 16:08:44 -05:00
Sebastian Falbesoner
87f11ef47f refactor: use Hash helper for double-SHA256 calculations 2023-01-31 19:34:35 +01:00
fanquake
b5868f4b1f Merge bitcoin/bitcoin#23670: build: Build minisketch test in make check, not in make
6d58117a31 build: Build minisketch test in `make check`, not in `make` (Hennadii Stepanov)

Pull request description:

  On master (d1e42659bb):
  ```
  $ ./autogen.sh && ./configure --without-gui --disable-wallet && make clean
  $ make 2>&1 | grep LD | grep -v .la
    CXXLD    bitcoind
    CXXLD    bitcoin-cli
    CXXLD    bitcoin-tx
    CXXLD    bitcoin-util
    CXXLD    test/test_bitcoin
    CXXLD    bench/bench_bitcoin
    CXXLD    minisketch/test
    CXXLD    test/fuzz/fuzz
    CXXLD    univalue/test/object
    CXXLD    univalue/test/unitester
  $ make check 2>&1 | grep LD
    CCLD     exhaustive_tests
    CCLD     tests
  ```

  With this PR:
  ```
  $ ./autogen.sh && ./configure --without-gui --disable-wallet && make clean
  $ make 2>&1 | grep LD | grep -v .la
    CXXLD    bitcoind
    CXXLD    bitcoin-cli
    CXXLD    bitcoin-tx
    CXXLD    bitcoin-util
    CXXLD    test/test_bitcoin
    CXXLD    bench/bench_bitcoin
    CXXLD    test/fuzz/fuzz
    CXXLD    univalue/test/object
    CXXLD    univalue/test/unitester
  $ make check 2>&1 | grep LD
    CXXLD    minisketch/test
    CCLD     exhaustive_tests
    CCLD     tests
  ```

  In fact, this PR restores behavior that was before bitcoin/bitcoin#22646, and that behavior looks more optimal.

  As an outcome, the `contrib/guix/libexec/build.sh` does not spend resources to build binaries which are not a part of the release package.

ACKs for top commit:
  TheCharlatan:
    ACK 6d58117a31

Tree-SHA512: 4957c8f88a01aca005813bf4c1c26f433756bf68ea0c958481c638ead229fa8e23ecae3a8ac31ea555876ba6f2cc10ecd91caf2e2f664de5cb529ec05fb38fa7
2023-01-31 17:55:44 +00:00
MarcoFalke
fad7af700e Use steady clock for logging timer 2023-01-31 18:48:50 +01:00
MarcoFalke
1ff254e45c Merge bitcoin/bitcoin#26974: refactor: rpc: set TxToJSON default verbosity to SHOW_DETAILS
a24e633339 refactor: rpc: set TxToJSON default verbosity to SHOW_DETAILS (stickies-v)

Pull request description:

  `TxToJSON()` and `TxToUniv()` are only to be called when we want to decode the transaction (i.e. its details) into JSON. If `TxVerbosity` is `SHOW_TXID`, the function should not have been (and currently is not) called in the first place.

  There is no behaviour change, current logic simply assumes anything less than `TxVerbosity::SHOW_DETAILS_AND_PREVOUT` equals `TxVerbosity::SHOW_DETAILS`. With this change, the assumptions and intent become more explicit.

ACKs for top commit:
  w0xlt:
    ACK a24e633339

Tree-SHA512: b97235adae49b972bdbe10aca1438643fb35ec66a4e57166b1975b3015bc5a06a711feebe4453a8fefe71781e484b21ef80847d8e8a33694a3abcc863accd4d7
2023-01-31 18:22:19 +01:00
Martin Zumsande
fe683f3524 log: Log VerifyDB Progress over multiple lines
This allows to log a timestamp for each entry,
and avoids potential interference with other
threads that could log concurrently.
2023-01-31 10:43:39 -05:00
Martin Zumsande
61431e3a57 validation: Skip VerifyDB checks of level >=3 if dbcache is too small
The previous behavior, skipping some L3 DisconnectBlock calls,
but still attempting to reconnect these blocks at L4, makes
ConnectBlock assert.

The variable skipped_l3_checks is introduced because even with an
insufficient cache for the L3 checks, the L1/L2 checks in the same
loop should still be completed.

Fixes #25563.
2023-01-31 10:43:39 -05:00
fanquake
2b211b41e3 Merge bitcoin/bitcoin#26952: build: Avoid BOOST_NO_CXX98_FUNCTION_BASE macro redefinition
d4c59da8d6 build: Avoid `BOOST_NO_CXX98_FUNCTION_BASE` macro redefinition (Hennadii Stepanov)

Pull request description:

  With GCC 12 and Boost 1.81 (from depends) having multiple warnings:
  ```
  In file included from /home/hebasto/bitcoin/depends/x86_64-pc-linux-gnu/include/boost/config.hpp:48:
  /home/hebasto/bitcoin/depends/x86_64-pc-linux-gnu/include/boost/config/stdlib/libstdcpp3.hpp:397:9: warning: 'BOOST_NO_CXX98_FUNCTION_BASE' macro redefined [-Wmacro-redefined]
  #define BOOST_NO_CXX98_FUNCTION_BASE
          ^
  <command line>:8:9: note: previous definition is here
  #define BOOST_NO_CXX98_FUNCTION_BASE 1
          ^
  1 warning generated.
  ```

  This PR fixes those warnings.

  Defining of the `BOOST_NO_CXX98_FUNCTION_BASE` macro was introduced in https://github.com/bitcoin/bitcoin/pull/25436, but since https://github.com/boostorg/config/pull/430, it is required to check it before adding.

ACKs for top commit:
  fanquake:
    ACK d4c59da8d6 - it works now.

Tree-SHA512: 53b9ddcf8dad729638ed41251e30c80f2d7d1ae3ffe47466865834f1f10184fe0881abeb339b3e46c270c3eb11fb63d19ab12cc9461bf5c2be12b4763c1b1c34
2023-01-31 14:58:52 +00:00
Hennadii Stepanov
75db62ba4c refactor: Move calculation logic out from CheckSequenceLocksAtTip() 2023-01-31 13:26:54 +00:00
Hennadii Stepanov
3bc434f459 refactor: Add CalculateLockPointsAtTip() function 2023-01-31 13:26:45 +00:00
Hennadii Stepanov
b0e916913c clang-tidy: Force to check all headers 2023-01-31 11:50:24 +00:00
Hennadii Stepanov
96ee992ac3 clang-tidy: Fix modernize-use-default-member-init in headers
See https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-default-member-init.html
2023-01-31 11:50:10 +00:00
MarcoFalke
fab9f7d1bd test: Use std::unique_ptr over manual delete in coins_tests 2023-01-31 12:09:01 +01:00
MarcoFalke
357d750cab Merge bitcoin/bitcoin#26956: test: refactor: introduce replace_in_config helper
b530d9605d test: refactor: introduce `replace_in_config` helper (Sebastian Falbesoner)

Pull request description:

  Currently two functional tests (p2p_permissions.py and wallet_crosschain.py) include quite similar code for substituting strings in a TestNode's bitcoind configuration file, so refactoring that out to a dedicated helper method seems to make sense (probably other tests could need that too in the future).

ACKs for top commit:
  kouloumos:
    ACK b530d9605d

Tree-SHA512: 5ca65a2ef3292460e5720d5c6acf7326335001858e8f71ab054560117f9479dbadb1dacd8c9235f67f3fcfd08dbc761b62704f379cbf619bba8804f16a25bc7b
2023-01-31 10:23:37 +01:00
Pieter Wuille
511aa4f1c7 Add unit test for ChaCha20's new caching 2023-01-30 19:12:55 -05:00
Pieter Wuille
fb243d25f7 Improve test vectors for ChaCha20 2023-01-30 18:12:21 -05:00
Pieter Wuille
93aee8bbda Inline ChaCha20 32-byte specific constants 2023-01-30 18:12:21 -05:00
Pieter Wuille
62ec713961 Only support 32-byte keys in ChaCha20{,Aligned} 2023-01-30 18:12:21 -05:00
Pieter Wuille
f21994a02e Use ChaCha20Aligned in MuHash3072 code 2023-01-30 18:12:21 -05:00
Pieter Wuille
5d16f75763 Use ChaCha20 caching in FastRandomContext 2023-01-30 18:12:21 -05:00
Pieter Wuille
38eaece67b Add fuzz test for testing that ChaCha20 works as a stream 2023-01-30 18:12:21 -05:00
Martin Leitner-Ankerl
5f05b27841 Add xoroshiro128++ PRNG
Xoroshiro128++ is a fast non-cryptographic random generator.
Reference implementation is available at https://prng.di.unimi.it/

Co-Authored-By: Pieter Wuille <pieter@wuille.net>
2023-01-30 18:12:21 -05:00
Pieter Wuille
12ff72476a Make unrestricted ChaCha20 cipher not waste keystream bytes
Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-01-30 18:12:21 -05:00
Pieter Wuille
6babf40213 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 2023-01-30 18:12:21 -05:00
Pieter Wuille
e37bcaa0a6 Split ChaCha20 into aligned/unaligned variants 2023-01-30 18:12:21 -05:00
Andrew Chow
ceb74b844c Merge bitcoin/bitcoin#26998: depends: ensure we are appending to sqlite cflags
56a03f1834 depends: ensure we are appending to sqlite cflags (fanquake)

Pull request description:

  Otherwise we'll just override other flags passed in (i.e msan).

  Should fix https://cirrus-ci.com/task/6598922274078720?logs=ci#L3661.

ACKs for top commit:
  achow101:
    ACK 56a03f1834
  TheCharlatan:
    ACK 56a03f1834

Tree-SHA512: 5890018cfc5deaef18b0f01a3a0396f803e97f9a9785bf6873ef48bc13b74b644315f0f29cf11d3522964a6396f74e1f080bb4e412bc302956a651fed28b27df
2023-01-30 17:23:58 -05:00
Andrew Chow
c8cb62272e Merge bitcoin/bitcoin#26999: A few follow-ups to #17487 (coins write without cache drop)
2e16054a66 Add assertions that BatchWrite(erase=true) erases (Pieter Wuille)
941feb6ca2 Avoid unclear {it = ++it;} (Pieter Wuille)
98db35c2f8 Follow coding style for named arguments (Pieter Wuille)
bb00357add Make test/fuzz/coins_view exercise CCoinsViewCache::Sync() (Pieter Wuille)

Pull request description:

  This addresses a few nits left open in #17487.

ACKs for top commit:
  jonatack:
    ACK 2e16054a66
  Sjors:
    utACK 2e16054a66
  achow101:
    ACK 2e16054a66
  jamesob:
    ACK 2e16054a66 ([`jamesob/ackr/26999.1.sipa.a_few_follow_ups_to_1748`](https://github.com/jamesob/bitcoin/tree/ackr/26999.1.sipa.a_few_follow_ups_to_1748))

Tree-SHA512: 5e64807b850fa12ce858e87470420555ad081f2f63d53649d9904034ed5311592005eb979a8a4df2b70ecb56cf022db9750cd6999e5480bc8226184d4be22ce4
2023-01-30 17:16:46 -05:00
Pieter Wuille
2e16054a66 Add assertions that BatchWrite(erase=true) erases 2023-01-30 13:13:54 -05:00
Pieter Wuille
941feb6ca2 Avoid unclear {it = ++it;} 2023-01-30 13:13:24 -05:00
Pieter Wuille
98db35c2f8 Follow coding style for named arguments 2023-01-30 13:13:20 -05:00
Pieter Wuille
bb00357add Make test/fuzz/coins_view exercise CCoinsViewCache::Sync() 2023-01-30 13:11:53 -05:00
fanquake
7241b936c5 Merge bitcoin/bitcoin#26965: refactor: Remove stray cs_main redundant declaration
faba08b5b4 refactor: Remove stray cs_main redundant declaration (MarcoFalke)
fa02591edf doc: Export threadsafety.h from sync.h (MarcoFalke)

Pull request description:

  Looks like this was forgotten when introducing kernel/cs_main ?

  Also, there is a commit to export threadsafety.h from sync.h.

ACKs for top commit:
  hebasto:
    ACK faba08b5b4

Tree-SHA512: 0aa58e7693b6fcd504f9da7339f8baa463a6407f67b27f68002db705f4642321ac3765f16c3d906c925ee24085591b79160a62fa5f4aaf6f2e5dcc788411800d
2023-01-30 17:27:44 +00:00
fanquake
56a03f1834 depends: ensure we are appending to sqlite cflags
Otherwise we'll just override other flags passed in (i.e msan).
2023-01-30 17:15:01 +00:00
fanquake
82903a7a8d Merge bitcoin/bitcoin#17487: coins: allow write to disk without cache drop
1d7935b45a test: add test for coins view flush behavior using Sync() (James O'Beirne)
2c3cbd6c00 test: add use of Sync() to coins tests (James O'Beirne)
6d8affca96 test: refactor: clarify the coins simulation (James O'Beirne)
79cedc36af coins: add Sync() method to allow flush without cacheCoins drop (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  In certain circumstances, we may want to flush chainstate data to disk without
  emptying `cacheCoins`, which affects performance. UTXO snapshot
  activation is one such case, as we populate `cacheCoins` with the snapshot
  contents and want to persist immediately afterwards but also enter IBD.

  See also #15265, which makes the case that under normal operation a
  flush-without-erase doesn't necessarily add much benefit. I open this PR
  even in light of the previous discussion because (i) flush-without-erase
  almost certainly provides benefit in the case of snapshot activation (especially
  on spinning disk hardware) and (ii) this diff is fairly small and gives us convenient
  options for more granular cache management without changing existing policy.

  See also #15218.

ACKs for top commit:
  sipa:
    ACK 1d7935b45a
  achow101:
    ACK 1d7935b45a
  Sjors:
    tACK 1d7935b45a

Tree-SHA512: 897583963e98661767d2d09c9a22f6019da24125558cd88770bfe2d017d924f23a9075b729e4b1febdec5b0709a38e8fa1ef94d62aa88650556b06cb4826c845
2023-01-30 16:01:16 +00:00
fanquake
0a1d372ad0 Merge bitcoin/bitcoin#26649: refactor: Use AutoFile and HashVerifier (without ser-type and ser-version) where possible
eeee61065f Use AutoFile and HashVerifier where possible (MarcoFalke)
fa961141f7 Add HashVerifier (MarcoFalke)

Pull request description:

  This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone.

  The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version.

  So do this here for `AutoFile` and `HashVerifier`. `CAutoFile` and `CHashVerifier` remain in places where it is not yet possible.

ACKs for top commit:
  stickies-v:
    ACK eeee61065f

Tree-SHA512: 93786778c309ecfdc1ed43552d24ff9d966954d69a47f66faaa6de24daacd25c651f3f62bde5abbb362700298fb3c04ffbd3207a0dd13d0bd8bff7fd6d07dcf8
2023-01-30 15:57:12 +00:00
Greg Sanders
906631450d s/transcation/transaction/ 2023-01-30 10:46:15 -05:00
fanquake
228edafc66 Merge bitcoin/bitcoin#23619: build: Propagate user-defined flags to host packages
a3a2bd9e8a ci: Drop no longer needed package-specific flags (Hennadii Stepanov)
071eef1e97 build: Propagate user-defined flags to host packages (Hennadii Stepanov)

Pull request description:

  On master (4f8b1f8759) `{CPP,C,CXX,LD}FLAGS` that are specified in the command line are not propagated to packages:
  ```
  $ make --no-print-directory -C depends print-libevent_cxxflags CXXFLAGS=-some-fancy-flag
  libevent_cxxflags=-pipe -O2
  ```

  This PR:
  - propagates `{CPP,C,CXX,LD}FLAGS` to host packages:
  ```
  $ make --no-print-directory -C depends print-libevent_cxxflags CXXFLAGS=-some-fancy-flag
  libevent_cxxflags= -some-fancy-flag
  ```
  - does not propagate `{CPP,C,CXX,LD}FLAGS` to native packages:
  ```
  $ make --no-print-directory -C depends print-native_b2_cxxflags CXXFLAGS=-some-fancy-flag
  native_b2_cxxflags=
  ```
  - actually addresses the https://github.com/bitcoin/bitcoin/pull/23551#issuecomment-973896518

ACKs for top commit:
  TheCharlatan:
    Code review ACK a3a2bd9e8a

Tree-SHA512: 243d6b1b0e9c5de46debc36de62a77b6b4d6f638940fd530040c219956ec624e321b0c25290fed164e3a8c88befa7b97b20f765d7b9a428c269b3720f21da099
2023-01-30 14:32:32 +00:00
fanquake
0e02f72548 depends: define __BSD_VISIBLE for FreeBSD bdb build
Required for additional definitions (IPC_R & friends), to be available,
when compiling under C11.

See: https://github.com/MarcoFalke/btc_nightly/pull/4.
2023-01-30 13:43:24 +00:00
MarcoFalke
fa47b28dfc refactor: Remove unused CDataStream SerializeMany constructor 2023-01-30 13:04:50 +01:00
willcl-ark
47c174d8ce doc: NetPermissionFlags for tx relay in blocksonly
Detail which permission type enables transaction relay for nodes
connected in blockonly mode
2023-01-30 11:21:49 +00:00
fanquake
79e18ebc81 Merge bitcoin/bitcoin#26896: build: Remove port-forwarding runtime setting options from configure
d51f0fa4b7 doc: add release notes for 26896 (fanquake)
2b248798d9 build: remove --enable-upnp-default from configure (fanquake)
02f5a5e7b5 build: remove --enable-natpmp-default from configure (fanquake)
25a0e8ba0b Remove configure-time setting of DEFAULT_UPNP (fanquake)
06562e5fa7 Remove configure-time setting of DEFAULT_NATPMP (fanquake)

Pull request description:

  This PR removes the `--enable-upnp-default` and `--enable-natpmp-default` options from configure.

  It's odd to me that we maintain configure-time options for setting the default port-forwarding runtime state (but no other similar options), and I'm not sure what use-case it satisfies, that can't be achieved by multiple other means. I also doubt that we'll ever restart using these in release builds, or turning on any of this by default.

  I think the only scenario these options would be used is when you want to compile your own binaries (we don't use them in Guix), with port-forwarding on by default, but otherwise can't or don't want to use a `.conf` file, can't or don't want to pass command line options at runtime, and also don't want to modify the source code?

ACKs for top commit:
  hebasto:
    ACK d51f0fa4b7, rebased and comments have been addressed since my recent [review](https://github.com/bitcoin/bitcoin/pull/26896#pullrequestreview-1273910740).
  TheCharlatan:
    ACK d51f0fa4b7

Tree-SHA512: 481decd8bddd8b03b7319591e3acf189f7b6b96c9a9a8c5bc1a3f8ec00d0b8f9b52d2f5c28a298a2ec947cfe9611cfd184e393ccb2e4e21bfce86ca7d4de60d3
2023-01-30 11:18:07 +00:00
willcl-ark
e325e0fccb doc: Fix comment syntax error 2023-01-30 10:26:02 +00:00
Vasil Dimov
dfc01ccd73 net: simplify the call to vProcessMsg.splice()
At the time when

```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), it);
```

is called, `it` is certainly `pnode->vRecvMsg.end()` which makes the
call equivalent to:

```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg, pnode->vRecvMsg.begin(), pnode->vRecvMsg.end());
```

which is equivalent to:

```cpp
pnode->vProcessMsg.splice(pnode->vProcessMsg.end(), pnode->vRecvMsg);
```

Thus, use the latter. Further, maybe irrelevant, but the latter has
constant complexity while the original code is `O(length of vRecvMsg)`.
2023-01-30 11:21:21 +01:00
glozow
b1329b7523 Merge bitcoin/bitcoin#26499: wallet: Abandon descendants of orphaned coinbases
b0fa5989e1 test: Check that orphaned coinbase unconf spend is still abandoned (Andrew Chow)
9addbd7890 wallet: Automatically abandon orphaned coinbases and their children (Andrew Chow)

Pull request description:

  When a block is reorged out of the main chain, any descendants of the coinbase will no longer be valid. Currently they are only marked as inactive, which means that our balance calculations will still include them. In order to be excluded from the balance calculation, they need to either be abandoned or conflicted. This PR goes with the abandoned method.

  Note that even when they are included in balance calculations, coin selection will not select outputs belonging to these transactions because they are not in the mempool.

  Fixes #14148

ACKs for top commit:
  furszy:
    ACK b0fa5989 with a not-blocking nit.
  aureleoules:
    reACK b0fa5989e1
  ishaanam:
    ACK b0fa5989e1

Tree-SHA512: 68f12e7aa8df392d8817dc6ac0becce8dbe83837bfa538f47027e6730e5b2e1b1a090cfcea2dc598398fdb66090e02d321d799f087020d7e1badcf96e598c3ac
2023-01-30 10:09:41 +00:00
MarcoFalke
37fea41bbf Merge bitcoin/bitcoin#26982: p2p: 25880 fixups (stalling timeout)
b2a1e47744 net_processing: simplify logging statement (Martin Zumsande)
6548ba68e8 test: fix intermittent errors in p2p_ibd_stalling.py (Martin Zumsande)

Pull request description:

  Two small fixups to #25880:

  - Use `is_connected` instead of `num_test_p2p_connections` to avoid intermittent failures where the p2p MiniNode got disconnected but this info hasn't made it to python yet, so it fails a ping. (https://github.com/bitcoin/bitcoin/pull/25880#discussion_r1089217720)

  - Simplify a logging statement (suggested in https://github.com/bitcoin/bitcoin/pull/25880#discussion_r1013738635)

ACKs for top commit:
  MarcoFalke:
    review ACK b2a1e47744 🕧

Tree-SHA512: 337f0883bf1c94cc26301a80dfa649093ed1e211ddda1acad8449a2add5be44e5c12d6073c209df9c7aa1edb9da33ec1cfdcb0deafd76178ed78785843e80bc7
2023-01-30 10:54:11 +01:00
MarcoFalke
1c8b80f440 Merge bitcoin/bitcoin#15294: refactor: Extract RipeMd160
6879be691b refactor: Extract RIPEMD160 (Ben Woosley)

Pull request description:

  To directly return a CRIPEMD160 hash from data.

  Simplifies the call sites.

ACKs for top commit:
  achow101:
    ACK 6879be691b
  theStack:
    re-ACK 6879be691b
  MarcoFalke:
    review ACK 6879be691b  🏔

Tree-SHA512: 6ead85d8060c2ac6afd43ec716ff5a82d6754c4132fe7df3b898541fa19f1dfd8b301b2b66ae7cb7594b1b1a8c7f68bce3790a8c610d4a1164e995d89bc5ae34
2023-01-30 09:49:01 +01:00
Martin Zumsande
b2a1e47744 net_processing: simplify logging statement
Also use count_seconds() instead of count() for type safety.
2023-01-29 17:35:15 -05:00
Sebastian Falbesoner
b530d9605d test: refactor: introduce replace_in_config helper 2023-01-28 23:49:55 +01:00
Hennadii Stepanov
d4c59da8d6 build: Avoid BOOST_NO_CXX98_FUNCTION_BASE macro redefinition 2023-01-28 17:07:52 +00:00
MarcoFalke
fa486de212 ci: Cache package manager install step 2023-01-28 17:59:07 +01:00
fanquake
114c5eca77 Merge bitcoin/bitcoin#25465: build: remove boost library detection
db648e8ccc build: remove unneeded var exporting (fanquake)
14afc71a61 build: remove Boost lib detection from ax_boost_base (fanquake)

Pull request description:

  We no longer link against any Boost libraries, so don't need to detect them, or set any Boost related LDFLAGS. Removing this from the macro also allows cleaning up some code in our configure.

  Guix Build:
  ```bash
  e1ca070d085115767415121f3be8d8fa29547c9df633f9782b168a00753e995a  guix-build-db648e8ccc69/output/aarch64-linux-gnu/SHA256SUMS.part
  fb4d229a71d64aef9de4a0301fad3d9ee5937025807a0e101f4cc0e20cf942b2  guix-build-db648e8ccc69/output/aarch64-linux-gnu/bitcoin-db648e8ccc69-aarch64-linux-gnu-debug.tar.gz
  5241bb543617df5e64584741b402117a3b9e7015a423507fd6c529ad397b13a5  guix-build-db648e8ccc69/output/aarch64-linux-gnu/bitcoin-db648e8ccc69-aarch64-linux-gnu.tar.gz
  eb4ba0914e9a38b3804c062ffd47f4d49dc996ea249c5cdbf64a349f73f59555  guix-build-db648e8ccc69/output/arm-linux-gnueabihf/SHA256SUMS.part
  497d1c0d03d52cc6469acf4c61f224e7567c2601b64df3fe5888374e3416d868  guix-build-db648e8ccc69/output/arm-linux-gnueabihf/bitcoin-db648e8ccc69-arm-linux-gnueabihf-debug.tar.gz
  86daab76f41581b752a5a786c5be1b094c14b25cdc8f4090f323b914118884ff  guix-build-db648e8ccc69/output/arm-linux-gnueabihf/bitcoin-db648e8ccc69-arm-linux-gnueabihf.tar.gz
  e8e2c58948a29b0286d17d7dfb89c81234ab28b98f44c2e1d70034da5ce9b8fc  guix-build-db648e8ccc69/output/arm64-apple-darwin/SHA256SUMS.part
  d8f0e791cce42db0741afbb7ec23e251a40ec30f8e1e7fef0d1afc364ce32e9b  guix-build-db648e8ccc69/output/arm64-apple-darwin/bitcoin-db648e8ccc69-arm64-apple-darwin-unsigned.dmg
  ee85b5b7942dc3361a3b05a395e271d54215f76fb67fc3c30144b089da374cd7  guix-build-db648e8ccc69/output/arm64-apple-darwin/bitcoin-db648e8ccc69-arm64-apple-darwin-unsigned.tar.gz
  72efbe40e3d033165871a8e8b4719c73a085ffc17a3198bf1191e5d22dec8c3a  guix-build-db648e8ccc69/output/arm64-apple-darwin/bitcoin-db648e8ccc69-arm64-apple-darwin.tar.gz
  0d86d3a303669235b91b4edbd6a5248dff65e31af3f3830cc8bdc116a2637e64  guix-build-db648e8ccc69/output/dist-archive/bitcoin-db648e8ccc69.tar.gz
  0f2371e331d97df50c65714ada6918565d8698637a78c7c97ba254dd5b3cc4b3  guix-build-db648e8ccc69/output/powerpc64-linux-gnu/SHA256SUMS.part
  1447777f7adababc6fddad3349ac435744abef35cd30b673c62621718441ca01  guix-build-db648e8ccc69/output/powerpc64-linux-gnu/bitcoin-db648e8ccc69-powerpc64-linux-gnu-debug.tar.gz
  3bf1e90df4d7fab18159ffd039dd1fa7e5251b0bdca020afd851fb7cd189cfd2  guix-build-db648e8ccc69/output/powerpc64-linux-gnu/bitcoin-db648e8ccc69-powerpc64-linux-gnu.tar.gz
  67ebc32843eb83a08468496c24750ab56bfdb259eb9004732be13427da5dbbc1  guix-build-db648e8ccc69/output/powerpc64le-linux-gnu/SHA256SUMS.part
  0da5fad78ab854f62db57a2f44b6ffecbf2b3e9de34b81681c66e28100a209ae  guix-build-db648e8ccc69/output/powerpc64le-linux-gnu/bitcoin-db648e8ccc69-powerpc64le-linux-gnu-debug.tar.gz
  e96a41287040d13da4738dde43dd7283356b8af10b535b15b212da702aa7f0af  guix-build-db648e8ccc69/output/powerpc64le-linux-gnu/bitcoin-db648e8ccc69-powerpc64le-linux-gnu.tar.gz
  b54a48e03e76672008445a174078e92a149dd9e4ce0c896b013162a6221b3abe  guix-build-db648e8ccc69/output/riscv64-linux-gnu/SHA256SUMS.part
  b7fea8fe4c1baecff28c45fa514ec34338a7f0e04a7f58b8cb5493932242c221  guix-build-db648e8ccc69/output/riscv64-linux-gnu/bitcoin-db648e8ccc69-riscv64-linux-gnu-debug.tar.gz
  775d406d4a2b1fa1750ae1924cf2f67de7d89dbf0044dc7ab08d3908812fff2e  guix-build-db648e8ccc69/output/riscv64-linux-gnu/bitcoin-db648e8ccc69-riscv64-linux-gnu.tar.gz
  a1dcd8c95517e8b98d8584eed00b561582b74a68a1b3d06efd86d5322186b21e  guix-build-db648e8ccc69/output/x86_64-apple-darwin/SHA256SUMS.part
  952ea47428b9bcadb809fb02a56347a9f8b29f09eba4edf67a157f7cc4ec9a57  guix-build-db648e8ccc69/output/x86_64-apple-darwin/bitcoin-db648e8ccc69-x86_64-apple-darwin-unsigned.dmg
  adb226a702e24962d263edf2e95100508d6728b4dd93a5d6098c37a5721c2bcc  guix-build-db648e8ccc69/output/x86_64-apple-darwin/bitcoin-db648e8ccc69-x86_64-apple-darwin-unsigned.tar.gz
  fd389c11cce919c53e4aeccb0e5d1ee1d12b1d9f6987a3317c4b254c3ca03387  guix-build-db648e8ccc69/output/x86_64-apple-darwin/bitcoin-db648e8ccc69-x86_64-apple-darwin.tar.gz
  c76314a9194733790533bbfefa7e6a234307aaa252989c2b3fd19191ae286c57  guix-build-db648e8ccc69/output/x86_64-linux-gnu/SHA256SUMS.part
  5e2b023b62b77709f30d545705a61826e96ddcfea4c24cde83fe2b98010262e1  guix-build-db648e8ccc69/output/x86_64-linux-gnu/bitcoin-db648e8ccc69-x86_64-linux-gnu-debug.tar.gz
  a3e9dcb58aa8554cdeb211461976087b724cd81157517a9f1e00b1a73d74ab9b  guix-build-db648e8ccc69/output/x86_64-linux-gnu/bitcoin-db648e8ccc69-x86_64-linux-gnu.tar.gz
  d6733313b8f262b214c28d44d8ee644a2435cb4da90555b30e20dbf3807b6660  guix-build-db648e8ccc69/output/x86_64-w64-mingw32/SHA256SUMS.part
  bc4b269f5b89200537d7175492ae31ab87eb4a1da86c1ade968a7bbdc472c5bf  guix-build-db648e8ccc69/output/x86_64-w64-mingw32/bitcoin-db648e8ccc69-win64-debug.zip
  ccd83eb4c9ab0df288935ab655cd71b00525b57f82c8b9d7a4d1a08325d26aac  guix-build-db648e8ccc69/output/x86_64-w64-mingw32/bitcoin-db648e8ccc69-win64-setup-unsigned.exe
  91ee20dfbaa923066379ea22f3e3a85a52f5b323877b44c19ccec6300d25cd41  guix-build-db648e8ccc69/output/x86_64-w64-mingw32/bitcoin-db648e8ccc69-win64-unsigned.tar.gz
  595baf193e0955436c4f2e5047e6842434bb4273b03c9d74e5c90972dde812f4  guix-build-db648e8ccc69/output/x86_64-w64-mingw32/bitcoin-db648e8ccc69-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    ACK db648e8ccc

Tree-SHA512: 940a148cf2db40484084cacf40e8ba4425fcbf5d6856a57be5af6f9f66d8adc0eb601c911513902bee120e7ddf97c7802e2d21b448beb4226f737164e5656b8a
2023-01-28 15:55:23 +00:00
fanquake
69f35d20a6 Merge bitcoin/bitcoin#22811: build: Fix depends build system when working with subtargets
978852aad8 build: Fix depends build system when working with subtargets (Hennadii Stepanov)

Pull request description:

  On master (f3e0ace8ec), the depends build system does _not_ guarantee that dependencies packages are available for `$(package)_built` target because these dependencies being prepared in `$(host_prefix)` at `$(package)_configured` target can be wiped out during building other package.

  Please consider:
  ```
  $ cd depends
  $ make clean
  $ make fontconfig_configured
  $ make
  ...
    CC       fcdir.lo
  In file included from fcftint.h:26,
                   from fcdir.c:26:
  ../fontconfig/fcfreetype.h:27:10: fatal error: ft2build.h: No such file or directory
     27 | #include <ft2build.h>
        |          ^~~~~~~~~~~~
  compilation terminated.
  make[4]: *** [Makefile:642: fcdir.lo] Error 1
  make[4]: *** Waiting for unfinished jobs....
  make[4]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/src'
  make[3]: *** [Makefile:503: all] Error 2
  make[3]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/src'
  make[2]: *** [Makefile:581: all-recursive] Error 1
  make[2]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94'
  make[1]: *** [Makefile:465: all] Error 2
  make[1]: Leaving directory '/home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94'
  make: *** [funcs.mk:288: /home/hebasto/GitHub/bitcoin/depends/work/build/x86_64-pc-linux-gnu/fontconfig/2.12.6-7daa5620c94/./.stamp_built] Error 2
  ```

  The following commands:
  ```
  $ cd depends
  $ make clean
  $ make qt_configured
  $ make
  ```
  also fail.

  The similar issue was reported earlier: #21381.

  This PR guarantees that dependencies packages are available for `$(package)_built` target.

  Guix builds:
  ```
  accef9ffccfe280fec1114c0440092ed5d792e9c53d95abc7fe65435aa136656  guix-build-978852aad8e2/output/aarch64-linux-gnu/SHA256SUMS.part
  a75f1250975525a21d2e213e23f1f0dab516d2b28d0c7d747de292fe5c906013  guix-build-978852aad8e2/output/aarch64-linux-gnu/bitcoin-978852aad8e2-aarch64-linux-gnu-debug.tar.gz
  d20787af2e7a14a3b7b1d21e0d8784aa6ebad1e916f02aebfa25afe9229ba43c  guix-build-978852aad8e2/output/aarch64-linux-gnu/bitcoin-978852aad8e2-aarch64-linux-gnu.tar.gz
  11c94a39c084763858c6de31b221868e52554f5500c0dc5589def429bb6b54c8  guix-build-978852aad8e2/output/arm-linux-gnueabihf/SHA256SUMS.part
  3935b0e14d78800977dc813a3824215797096b6fb29c84b5996f48338908a65f  guix-build-978852aad8e2/output/arm-linux-gnueabihf/bitcoin-978852aad8e2-arm-linux-gnueabihf-debug.tar.gz
  c828c3f87a453db443a385a266331661f623f8f4684d88eb006290c83bfa8150  guix-build-978852aad8e2/output/arm-linux-gnueabihf/bitcoin-978852aad8e2-arm-linux-gnueabihf.tar.gz
  b6ff14e1cc36e568fc726b50300f77498560322b3582738eb70e7144784f7e63  guix-build-978852aad8e2/output/arm64-apple-darwin/SHA256SUMS.part
  2a3e6ba5843eaf9562e9dcfdb4595024a71738079cea00e391558feca4d5bde1  guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin-unsigned.dmg
  ffd2ad39e8b9f95dd714513ba1e1c77666b0f3cc4b67be4ab763ebd431fe9276  guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin-unsigned.tar.gz
  5c9f8acd1777effc1e860b64143ba9d06ba5e3d0330e7341529eeae5cc6b3c5e  guix-build-978852aad8e2/output/arm64-apple-darwin/bitcoin-978852aad8e2-arm64-apple-darwin.tar.gz
  e34c693ecef6159c57fdedabff9dc3d69ec20387966083b828532c58e1e6e30b  guix-build-978852aad8e2/output/dist-archive/bitcoin-978852aad8e2.tar.gz
  8de4681114d96425bf9b0ccc47d49f696293ead514faa3fa83ddcccfdca2eeb2  guix-build-978852aad8e2/output/powerpc64-linux-gnu/SHA256SUMS.part
  d780330a105931eb4c66a536c332eb09e4b6d8c288832bb5a74b931c4600c0b3  guix-build-978852aad8e2/output/powerpc64-linux-gnu/bitcoin-978852aad8e2-powerpc64-linux-gnu-debug.tar.gz
  af036a6d714ef362a2facfe4e5c63deaa9dfa391d20542ead9ffb4a43b2b7ca2  guix-build-978852aad8e2/output/powerpc64-linux-gnu/bitcoin-978852aad8e2-powerpc64-linux-gnu.tar.gz
  dc6b4365632e7de386ead512b1cdcdd3c985623f63cff8b62974bd9ed8c05d5d  guix-build-978852aad8e2/output/powerpc64le-linux-gnu/SHA256SUMS.part
  5543a6dec58515186b71139a4a5c603d85638672f205e7c9fabb281c98018461  guix-build-978852aad8e2/output/powerpc64le-linux-gnu/bitcoin-978852aad8e2-powerpc64le-linux-gnu-debug.tar.gz
  dd791be2e61ce6cbd3e14c165ce2f8c2d22881992e0df72bd338423d092cc467  guix-build-978852aad8e2/output/powerpc64le-linux-gnu/bitcoin-978852aad8e2-powerpc64le-linux-gnu.tar.gz
  2b740db8e5b9c435be3e7b186c3b4a40885302243326ec990e24fe4ba4f777da  guix-build-978852aad8e2/output/riscv64-linux-gnu/SHA256SUMS.part
  de899c89874d4f34afeca179a6c7c8f49b0a975983ab2b31afd9f2d365674578  guix-build-978852aad8e2/output/riscv64-linux-gnu/bitcoin-978852aad8e2-riscv64-linux-gnu-debug.tar.gz
  9052b1db22c56692d99a61c3783b36c6f76537d9aec14f17d87a155beac82532  guix-build-978852aad8e2/output/riscv64-linux-gnu/bitcoin-978852aad8e2-riscv64-linux-gnu.tar.gz
  5e79ddf57a94c5978ad819896786107f735d5742bbd042c2c64ae2d0681ce53a  guix-build-978852aad8e2/output/x86_64-apple-darwin/SHA256SUMS.part
  96443ad839f87c723db1c0a96d8ead0afc69e9d96ad45b5814344866da2dae73  guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin-unsigned.dmg
  14b0a3081772e81a463398a2702aca039d2f276e301dee9f5a0ccffbb09e2749  guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin-unsigned.tar.gz
  6bfb8252524202028308267f5e96bc30f284052f5feaa58ed3697dde27a3130f  guix-build-978852aad8e2/output/x86_64-apple-darwin/bitcoin-978852aad8e2-x86_64-apple-darwin.tar.gz
  5f8ea6297e246b08ffd806913897cc863feeec6522fcfb4456a59c5f154e0c2d  guix-build-978852aad8e2/output/x86_64-linux-gnu/SHA256SUMS.part
  40d1bcf491660d54fe20b2db24828ebf61be848eefdc38fba09ed43f6bdba4b1  guix-build-978852aad8e2/output/x86_64-linux-gnu/bitcoin-978852aad8e2-x86_64-linux-gnu-debug.tar.gz
  3200e67a4dea115e8e341b4d71d84dc5e8bd2ae35e550cde6aef88d120c65eae  guix-build-978852aad8e2/output/x86_64-linux-gnu/bitcoin-978852aad8e2-x86_64-linux-gnu.tar.gz
  0b0bf7effc493ecc68398f23fc81647f64fdee115e8ccd7ae91e7881804ec328  guix-build-978852aad8e2/output/x86_64-w64-mingw32/SHA256SUMS.part
  e2064c9ddeb4af18468f37ba8cf70004062c31e1387b4cc0fe4b445fae518e8d  guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-debug.zip
  be347a901b896e0a1dc2f0f5a7f84614075805cccf1f2af8ec8df678d086fdbc  guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-setup-unsigned.exe
  bab8700e9e266970e8c7cad494902058ad12d1f2a6462e0039daa637b1a0ce0d  guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64-unsigned.tar.gz
  c8e55e64b248fd7c9056fe811a1eba992bbb92e44857204e3024416d9ba6307d  guix-build-978852aad8e2/output/x86_64-w64-mingw32/bitcoin-978852aad8e2-win64.zip
  ```

ACKs for top commit:
  TheCharlatan:
    tACK 978852aad8

Tree-SHA512: c195484274433039e327d44b1949afa296e09e7470a2b138b7a8476c8bf9c1302bc21284cd5436f09aa97824aae9f362b7932ff2937b78f79df0b43e50f3dfaa
2023-01-28 15:45:53 +00:00
fanquake
d51f0fa4b7 doc: add release notes for 26896 2023-01-28 15:27:27 +00:00
fanquake
2b248798d9 build: remove --enable-upnp-default from configure 2023-01-28 15:27:23 +00:00
fanquake
02f5a5e7b5 build: remove --enable-natpmp-default from configure 2023-01-28 15:26:12 +00:00
fanquake
25a0e8ba0b Remove configure-time setting of DEFAULT_UPNP
Default to false.
2023-01-28 15:24:13 +00:00
fanquake
06562e5fa7 Remove configure-time setting of DEFAULT_NATPMP
Default to false.
2023-01-28 15:24:12 +00:00
Martin Zumsande
6548ba68e8 test: fix intermittent errors in p2p_ibd_stalling.py
Using is_connected instead of num_test_p2p_connections
ensures that python has taken notice that the p2p was
disconnected.
2023-01-27 15:28:21 -05:00
fanquake
4b51290f71 Merge bitcoin/bitcoin#26977: ci: Fetch no git history, unless lint
faa65f12fc ci: Fetch no git history, unless lint (MarcoFalke)

Pull request description:

  Should cut 20s from each build, with no downside?

  This is possible since commit fad7281d78

ACKs for top commit:
  real-or-random:
    ACK faa65f12fc

Tree-SHA512: 1912d6a2c494de0ac2f69ac1dad568b7b2b9eb52e261d98e05f1184852281cd7fa9bc065ffe632b29ef204c3c7208034ef7b3b516aab3f3eba6b7a81db96ddee
2023-01-27 18:46:20 +00:00
Andrew Chow
483a4bb819 Merge bitcoin/bitcoin#26834: contrib: remove install_db4.sh
44f3c7de21 contrib: remove install_db4.sh (fanquake)
14ce84388f doc: add new NO_* options from #26833 (fanquake)

Pull request description:

  Now that we can build a bdb-only depends prefix (#26833), there is no need to
  maintain a bdb-building bash script, that does the same thing as
  depends, except worse, as it's missing patches and workarounds. i.e #26623.

  Someone that wants to compile bdb themselves, but doesn't want to use other depends built libs, can do:
  ```bash
  make -C depends NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_SQLITE=1 NO_NATPMP=1 NO_UPNP=1 NO_ZMQ=1 NO_USDT=1
  ...
  to: /path/to/bitcoin/depends/x86_64-pc-linux-gnu
  ```

  which gives them a BDB only prefix, and then compile using:
  ```bash
  export BDB_PREFIX="/path/to/bitcoin/depends/x86_64-pc-linux-gnu"
  ./autogen.sh
  ./configure \
      BDB_LIBS="-L${BDB_PREFIX}/lib -ldb_cxx-4.8" \
      BDB_CFLAGS="-I${BDB_PREFIX}/include"
  ```

  Wondering if we should extract the build bdb/legacy wallet docs somewhere, to avoid the repetition?

ACKs for top commit:
  TheCharlatan:
    ACK 44f3c7de21
  achow101:
    ACK 44f3c7de21
  hebasto:
    ACK 44f3c7de21
  jarolrod:
    ACK 44f3c7de21

Tree-SHA512: 50b33ae9df2ab94a1bd114e846cec16f647a61023b72f0d3e547a18db09c01d60bb7b42a04758212f4930314df03016feb6ebc96962dd8a8e26eb8cd4e0d167d
2023-01-27 12:42:16 -05:00
MarcoFalke
9a288430df Merge bitcoin/bitcoin#26900: refactor: Add BlockManager getters
faf7b4f1fc Add BlockManager::IsPruneMode() (MarcoFalke)
fae71fe27e Add BlockManager::GetPruneTarget() (MarcoFalke)
fa0f0436d8 Add BlockManager::LoadingBlocks() (MarcoFalke)

Pull request description:

  Requested in https://github.com/bitcoin/bitcoin/pull/25781#discussion_r1061323795, but adding getters seems unrelated from removing globals, so I split it out for now.

ACKs for top commit:
  dergoegge:
    Code review ACK faf7b4f1fc
  brunoerg:
    crACK faf7b4f1fc

Tree-SHA512: 204d0e9a0e8b78175482f89b4ce620fba0e65d8e49ad845d187af44d3843f4c733a01bac1ffe5a5319f524d8346123693a456778b69d6c75268c447eb8839642
2023-01-27 17:33:11 +01:00
John Moffett
c497a198db Fix comment about how wallet txs are sorted
The wallet transactions in the node are not sorted
by hash. Also make comment Doxygen compatible.
2023-01-27 10:04:48 -05:00
MarcoFalke
faa65f12fc ci: Fetch no git history, unless lint 2023-01-27 15:05:29 +01:00
Hennadii Stepanov
7a820cee0e test, build: Separate read_json function into its own module 2023-01-27 09:26:29 +00:00
Andrew Chow
835212cd1d Merge bitcoin/bitcoin#25880: p2p: Make stalling timeout adaptive during IBD
39b93649c4 test: add functional test for IBD stalling logic (Martin Zumsande)
0565951f34 p2p: Make block stalling timeout adaptive (Martin Zumsande)

Pull request description:

  During IBD, there is the following stalling mechanism if we can't proceed with assigning blocks from a 1024 lookahead window because all of these blocks are either already downloaded or in-flight: We'll mark the peer from which we expect the current block that would allow us to advance our tip (and thereby move the 1024 window ahead) as a possible staller. We then give this peer 2 more seconds to deliver a block (`BLOCK_STALLING_TIMEOUT`) and if it doesn't, disconnect it and assign the critical block we need to another peer.

  Now the problem is that this second peer is immediately marked as a potential staller using the same mechanism and given 2 seconds as well - if our own connection is so slow that it simply takes us more than 2 seconds to download this block, that peer will also be disconnected (and so on...), leading to repeated disconnections and no progress in IBD. This has been described in #9213, and I have observed this when doing IBD  on slower connections or with Tor - sometimes there would be several minutes without progress, where all we did was disconnect peers and find new ones.

  The `2s` stalling timeout was introduced in #4468, when blocks weren't full and before Segwit increased the maximum possible physical size of blocks - so I think it made a lot of sense back then.
  But it would be good to revisit this timeout now.

  This PR makes the timout adaptive (idea by sipa):
  If we disconnect a peer for stalling, we now double the timeout for the next peer (up to a maximum of 64s). If we connect a block, we half it again up to the old value of 2 seconds. That way, peers that are comparatively slower will still get disconnected, but long phases of disconnecting all peers shouldn't happen anymore.

  Fixes #9213

ACKs for top commit:
  achow101:
    ACK 39b93649c4
  RandyMcMillan:
    Strong Concept ACK 39b93649c4
  vasild:
    ACK 39b93649c4
  naumenkogs:
    ACK 39b93649c4

Tree-SHA512: 85bc57093b2fb1d28d7409ed8df5a91543909405907bc129de7c6285d0810dd79bc05219e4d5aefcb55c85512b0ad5bed43a4114a17e46c35b9a3f9a983d5754
2023-01-27 01:53:21 -05:00
Amiti Uttarwar
80f39c99ef addrman, refactor: combine two size functions
The functionality of the old size() is covered by the new Size()
when no arguments are specified, so this does not change behavior.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2023-01-26 18:11:13 -05:00
Martin Zumsande
4885d6f197 addrman, refactor: move count increment into Create()
Create() is only called in one spot, so this doesn't
change behavior.

Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2023-01-26 18:11:13 -05:00
Martin Zumsande
c77c877a8e net: Load fixed seeds from reachable networks for which we don't have addresses
Previously, we'd only load fixed seeds if we'd not
know any addresses at all. This change makes it possible
to change -onlynet abruptly, e.g. from -onlynet=onion to
-onlynet=i2p and still find peers.
2023-01-26 18:11:13 -05:00
Martin Zumsande
d35595a78a addrman: add function to return size by network and table
For now, the new functionality will be used in the context of
querying fixed seeds. Other possible applications for
future changes is the use in the context of making automatic
connections to specific networks, or making more detailed info
about addrman accessible via rpc.
2023-01-26 18:11:13 -05:00
Ben Woosley
6879be691b refactor: Extract RIPEMD160
To directly return a CRIPEMD160 hash from data.

Incidentally, decoding this acronym:
* RIPEMD -> RIPE Message Digest
* RIPE -> RACE Integrity Primitives Evaluation
* RACE -> Research and Development in Advanced Communications Technologies in Europe
2023-01-26 15:48:49 -06:00
John Moffett
08209c039f Correctly limit overview transaction list
The way that the main overview page limits the number
of transactions displayed (currently 5) is not
an appropriate use of Qt. If it's run with a DEBUG
build of Qt, it'll result in a segfault in certain
relatively common situations. Instead of artificially
limiting the rowCount() in the subclassed proxy
filter, we hide/unhide the rows in the displaying
QListView upon any changes in the sorted proxy filter.
2023-01-26 14:56:12 -05:00
MarcoFalke
ffc22b7d42 Merge bitcoin/bitcoin#26923: test: refactor: simplify p2p_{tx_download,eviction}.py by using MiniWallet
8609f24be2 test: refactor: simplify p2p_eviction.py by using MiniWallet (Sebastian Falbesoner)
7aa4b32cd4 test: refactor: simplify p2p_tx_download.py by using MiniWallet (Sebastian Falbesoner)

Pull request description:

  Similar to #26892, this PR simplies the functional tests p2p_tx_download.py and p2p_eviction.py by using MiniWallet in order to avoid manual low-level tx creation. For the latter, rather than mining 100 blocks manually, the pre-mined chain of the test framework is used.

  These instances were found via `$ git grep signrawtransactionwithkey ./test/functional`. AFAICT, there are no other instances where MiniWallet could replace tx creation trivially.

ACKs for top commit:
  MarcoFalke:
    review ACK 8609f24be2

Tree-SHA512: dfb0103fe7f0625d125e8e4408baed8bfc1ff579954af17d0ead5277e05f933b2c2d98a0093e8109e947635f1718d5c58d837ab825f26077fac0a40575bd3e6f
2023-01-26 15:54:06 +01:00
fanquake
eee2c28985 Merge bitcoin/bitcoin#26945: depends: systemtap 4.8
df7ae8b7ca depends: systemtap: remove variadic params that trigger compiler warnings (Cory Fields)
a66d82e25f depends: systemtap 4.8 (fanquake)

Pull request description:

  Updates systemtap to 4.8.
  Includes acc2895a66a4b654e9a0a05ed0927f67f48c75b2 from #25972.
  Will half (depends) fix #26916.

  Release notes etc: https://lwn.net/Articles/913908/.

ACKs for top commit:
  0xB10C:
    ACK df7ae8b7ca
  hebasto:
    re-ACK df7ae8b7ca.

Tree-SHA512: 0f64fba87888058380183f38b6ace7b71f4a8b5503eb32b82b283a40c99d92c449c60deecc2386df5084235cfd760af6c1e7f432fa1bd30f97bb42f100f23d62
2023-01-26 13:37:48 +00:00
stickies-v
a24e633339 refactor: rpc: set TxToJSON default verbosity to SHOW_DETAILS
`TxToJSON()` and `TxToUniv()` are only to be called when we want to
decode the transaction (i.e. its details) into JSON. If `TxVerbosity` is
`SHOW_TXID`, the function should not have been (and currently is not)
called in the first place.

There is no behaviour change, current logic simply assumes anything less
than `TxVerbosity::SHOW_DETAILS_AND_PREVOUT` equals
`TxVerbosity::SHOW_DETAILS`. With this change, the assumptions and
intent become more explicit.
2023-01-26 11:51:33 +00:00
fanquake
79e007d1d6 Merge bitcoin/bitcoin#25296: Add DataStream without ser-type and ser-version and use it where possible
fa035fe2d6 Remove unused CDataStream::SetType (MarcoFalke)
fa29e73cda Use DataStream where possible (MarcoFalke)
fa9becfe1c streams: Add DataStream without ser-type and ser-version (MarcoFalke)

Pull request description:

  This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone.

  The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version.

  So do this here for `DataStream`. `CDataStream` remains in places where it is not yet possible.

ACKs for top commit:
  stickies-v:
    re-ACK [fa035fe](fa035fe2d6)
  aureleoules:
    diff re-ACK fa035fe2d6 fa0e6640ba..fa035fe2d6

Tree-SHA512: cb5e53d0df7c94319ffadc6ea1d887fc38516decaf43f0673396d79cc62d450a1a61173654a91b8c2b52d2cecea53fe4a500b8f6466596f35731471163fb051c
2023-01-26 11:30:34 +00:00
glozow
77a36033b5 Merge bitcoin/bitcoin#26551: p2p: Track orphans by who provided them
c58c249a5b net_processing: indicate more work to do when orphans are ready to reconsider (Anthony Towns)
ecb0a3e425 net_processing: Don't process tx after processing orphans (Anthony Towns)
c583775706 net_processing: only process orphans before messages (Anthony Towns)
be2304676b txorphange: Drop redundant originator arg from GetTxToReconsider (Anthony Towns)
a4fe09973a txorphanage: index workset by originating peer (Anthony Towns)

Pull request description:

  We currently process orphans by assigning them to the peer that provided a missing parent; instead assign them to the peer that provided the orphan in the first place. This prevents a peer from being able to marginally delay another peer's transactions and also simplifies the internal API slightly. Because we're now associating orphan processing with the peer that provided the orphan originally, we no longer process orphans immediately after receiving the parent, but defer until a future call to `ProcessMessage`.

  Based on #26295

ACKs for top commit:
  naumenkogs:
    utACK c58c249a5b
  glozow:
    ACK c58c249a5b
  mzumsande:
    Code Review ACK c58c249a5b

Tree-SHA512: 3186c346f21e60440266a2a80a9d23d7b96071414e14b2b3bfe50457c04c18b1eab109c3d8c2a7726a6b10a2eda1f0512510a52c102da112820a26f5d96f12de
2023-01-26 10:36:18 +00:00
MarcoFalke
fa035fe2d6 Remove unused CDataStream::SetType
The last use was removed in the previous commit.
2023-01-26 10:45:02 +01:00
MarcoFalke
fa29e73cda Use DataStream where possible 2023-01-26 10:44:05 +01:00
MarcoFalke
d4c180ecc9 Merge bitcoin/bitcoin#26960: refactor: Remove c_str from util/check
fab958290b refactor: Remove c_str from util/check (MarcoFalke)

Pull request description:

  Seems confusing and fragile to require calling code to call `c_str()` when passing a read-only view of a std::string.

  Fix that by using std::string_view, which can be constructed from string literals and std::string.

  Also, remove the now unused `c_str()` from `src/wallet/bdb.cpp`.

ACKs for top commit:
  stickies-v:
    ACK fab958290b
  aureleoules:
    ACK fab958290b
  theStack:
    ACK fab958290b

Tree-SHA512: ae39812c6bb8e2ef095e1b843774af2718f48404cb848c3e43b16d3c22240557d69d54a13a038a4a9c48b3ba0e4523e1f87abdd60f91486092f50fd43c0e8483
2023-01-26 09:02:36 +01:00
Sebastian Falbesoner
ab4efad51b test: fix immediate tx relay in wallet_groups.py 2023-01-26 02:56:34 +01:00
fanquake
ab98673f05 Merge bitcoin/bitcoin#26929: rpc: Throw more user friendly arg type check error (1.5/2)
fafeddfe0e rpc: Throw more user friendly arg type check error (MarcoFalke)

Pull request description:

  The arg type check error doesn't list which arg (position or name) failed. Fix that.

ACKs for top commit:
  stickies-v:
    ACK fafeddfe0e - although I think the functional test isn't in a logical place (but not blocking)

Tree-SHA512: 17425aa145aab5045940ec74fff28f0e3b2b17ae55f91c4bb5cbcdff0ef13732f8e31621d85964dc2c04333ea37dbe528296ac61be27541384b44e37957555c8
2023-01-25 15:25:49 +00:00
MarcoFalke
0486148f75 Merge bitcoin/bitcoin#26829: init: Remove unnecessary sensitive flag from rpcbind
b9d5674541 init: Remove sensitive flag from rpcbind (Andrew Chow)

Pull request description:

  `-rpcbind` is currently flagged as a sensitive option which means that its value will be masked when the command line args are written to the debug.log file. However this is not useful as if `rpcbind` is actually activated, the bound IP addresses will be written to the log anyways. The test `feature_config_args.py` did not catch this contradiction as the test node was not started with `rpcallowip` and so `rpcbind` was not acted upon.

  This also brings `rpcbind` inline with `bind` as that is not flagged as sensitive either.

ACKs for top commit:
  Sjors:
    re-utACK b9d5674541
  willcl-ark:
    ACK b9d5674
  theStack:
    ACK b9d5674541

Tree-SHA512: 50ab5ad2e18ae70649deb1ac429d404b5f5c41f32a4943b2041480580152df22e72d4aae493379d0b23fcb649ab342376a82119760fbf6dfdcda659ffd3e244a
2023-01-25 15:32:41 +01:00
MarcoFalke
f703c79ad7 Merge bitcoin/bitcoin#26961: ci: Fix APPEND_APT_SOURCES_LIST trying to modify the host system
fa88d42123 ci: Fix APPEND_APT_SOURCES_LIST trying to modify the host system (MarcoFalke)

Pull request description:

  `>>` will be redirected to the host system, unless the CI system is already running in docker.

  This shouldn't lead to any issues, unless someone is running the CI as root, I guess.

  Still, fix it to avoid problems.

ACKs for top commit:
  fanquake:
    ACK fa88d42123

Tree-SHA512: 6c501fd69a62e3cea27d24647e1a02f0f63dad45737dbfed23e3f70d89aacfbe477c71ca8812a47ab4641a31288df5919512542d5d8b49fc57dc3ef2aa0cde33
2023-01-25 15:05:20 +01:00
MarcoFalke
faba08b5b4 refactor: Remove stray cs_main redundant declaration 2023-01-25 09:56:26 +01:00
MarcoFalke
fa02591edf doc: Export threadsafety.h from sync.h
All places that include sync.h will likely need threadsafety
annotations, so export them.
2023-01-25 09:33:26 +01:00
Anthony Towns
c58c249a5b net_processing: indicate more work to do when orphans are ready to reconsider
When PR#15644 made orphan processing interruptible, it also introduced a
potential 100ms delay between processing of the first and second newly
reconsiderable orphan, because it didn't check if the orphan work set
was non-empty after invoking ProcessMessage(). This adds that check, so
that ProcessMessages() will return true if there are orphans to process,
usually avoiding the 100ms delay in CConnman::ThreadMessageHandler().
2023-01-25 18:15:26 +10:00
Anthony Towns
ecb0a3e425 net_processing: Don't process tx after processing orphans
If we made progress on orphans, consider that enough work for this peer
for this round of ProcessMessages. This also allows cleaning up the api
for TxOrphange:GetTxToReconsider().
2023-01-25 18:15:12 +10:00
Anthony Towns
c583775706 net_processing: only process orphans before messages
Previously, when we processed a new tx we would attempt to ATMP any
orphans that considered the new tx a parent immediately, but would only
accept at most one such tx, leaving any others to be considered on a
future run of ProcessMessages(). With this patch, we don't attempt any
orphan processing immediately after receiving a tx, instead deferring
all of them until the next call to ProcessMessages().
2023-01-25 18:13:42 +10:00
Anthony Towns
be2304676b txorphange: Drop redundant originator arg from GetTxToReconsider 2023-01-25 18:13:42 +10:00
MarcoFalke
fa88d42123 ci: Fix APPEND_APT_SOURCES_LIST trying to modify the host system 2023-01-24 20:33:55 +01:00
Andrew Chow
50ac8f5774 Merge bitcoin/bitcoin#25987: build: compile depends sqlite with more recommended options
dee690257c build: pass --enable-debug to sqlite when DEBUG=1 (fanquake)
807b61fcca build: use more recommended sqlite3 compile options (fanquake)

Pull request description:

  https://www.sqlite.org/compile.html

  [SQLITE_DQS](https://www.sqlite.org/compile.html#dqs)
  > This setting disables the double-quoted string literal misfeature.

  [SQLITE_DEFAULT_MEMSTATUS](https://www.sqlite.org/compile.html#default_memstatus)
  > This setting causes the sqlite3_status() interfaces that track
  > memory usage to be disabled.
  > This helps the sqlite3_malloc() routines run much faster, and since
  > SQLite uses sqlite3_malloc() internally, this helps to make the
  > entire library faster.

  [SQLITE_OMIT_DEPRECATED](https://www.sqlite.org/compile.html#omit_deprecated)
  > Omitting deprecated interfaces and features will not help SQLite
  > to run any faster.
  > It will reduce the library footprint, however. And it is the
  > right thing to do.

  [SQLITE_OMIT_SHARED_CACHE](https://www.sqlite.org/compile.html#omit_shared_cache)
  > Omitting the possibility of using shared cache allows many
  > conditionals in performance-critical sections of the code to be
  > eliminated. This can give a noticeable improvement in performance.

  Also: https://www.sqlite.org/sharedcache.html
  > Shared-cache mode is an obsolete feature.
  > The use of shared-cache mode is discouraged.
  > Most use cases for shared-cache are better served by WAL mode.
  > Applications that build their own copy of SQLite from source code
  > are encouraged to use the -DSQLITE_OMIT_SHARED_CACHE compile-time
  > option, as the resulting binary will be both smaller and faster.

  [SQLITE_OMIT_JSON](https://www.sqlite.org/compile.html#omit_json)
  > Starting with sqlite 3.38.0 the JSON extension became opt-out rather
  than opt-in, so we disable it here.

  --disable-rtree
  > An R-Tree is a special index that is designed for doing range queries.
  > R-Trees are most commonly used in geospatial systems...
  https://www.sqlite.org/rtree.html

  --disable-fts4 --disable-fts5
  > FTS5 is an SQLite virtual table module that provides full-text
  > search functionality to database applications.

  DSQLITE_LIKE_DOESNT_MATCH_BLOBS
  > simplifies the implementation of the LIKE optimization and allows
  > queries that use the LIKE optimization to run faster.

  DSQLITE_OMIT_DECLTYPE
  > By omitting the (seldom-needed) ability to return the declared type of
  > columns from the result set of query, prepared statements can be made
  > to consume less memory.

  DSQLITE_OMIT_PROGRESS_CALLBACK
  > By omitting this interface, a single conditional is removed from the
  > inner loop of the bytecode engine, helping SQL statements to run slightly
  > faster.

  DSQLITE_OMIT_AUTOINIT
  > with the SQLITE_OMIT_AUTOINIT option, the automatic initialization is omitted.
  > This helps many API calls to run a little faster
  > it also means that the application must call sqlite3_initialize()
  manually.

  On my Linux box this results in a `libsqlite3.a` that shrinks from ~1.7mb to ~1.3mb.
  On macOS, `libsqlite3.a` shrinks from ~2.2mb to ~1.3mb.

  Guix Build:
  ```bash
  04626ec7b64086103bfb5537397d7953d0699b31da1b1bc100003c7d30991fdf  guix-build-dee690257c79/output/aarch64-linux-gnu/SHA256SUMS.part
  a77c72052266788e890a0d3d89cc6eee9dd72db6a1b6fdc8aab882b35a315284  guix-build-dee690257c79/output/aarch64-linux-gnu/bitcoin-dee690257c79-aarch64-linux-gnu-debug.tar.gz
  86d782f185ee0fae8e8733ab8a511de2ca2500f02eb3d4aaa20d65b4434dcd77  guix-build-dee690257c79/output/aarch64-linux-gnu/bitcoin-dee690257c79-aarch64-linux-gnu.tar.gz
  d81874ecd378470b01a429f2a4be07fcb4824b56ace146226b77088cc3c41baa  guix-build-dee690257c79/output/arm-linux-gnueabihf/SHA256SUMS.part
  8f24076800e5a19ac9005af8c6e4987b32e0a857a83af84931958505228ef58d  guix-build-dee690257c79/output/arm-linux-gnueabihf/bitcoin-dee690257c79-arm-linux-gnueabihf-debug.tar.gz
  b913f2b0d8f0e32a176bc0e9330106cf1973afe6e8ee955f3eeb89918ab9d565  guix-build-dee690257c79/output/arm-linux-gnueabihf/bitcoin-dee690257c79-arm-linux-gnueabihf.tar.gz
  034b47d8a60a19391e0e18f6fcc8fb3a87be78a9707e3ac3e7ee45eb37687bb7  guix-build-dee690257c79/output/arm64-apple-darwin/SHA256SUMS.part
  9efc5c0ceac05f6efce03fcde0cbe8b79bae43fe0d98aecb817908e76f5c39b7  guix-build-dee690257c79/output/arm64-apple-darwin/bitcoin-dee690257c79-arm64-apple-darwin-unsigned.dmg
  a549cdcbd311d9b025971b13987290f6cd2b93e16c8389e8a7a2ccd3608326ed  guix-build-dee690257c79/output/arm64-apple-darwin/bitcoin-dee690257c79-arm64-apple-darwin-unsigned.tar.gz
  3dd9e26a41596151022aa4288d778dc335a2b38bd391e3f65c0575dd6eb47b54  guix-build-dee690257c79/output/arm64-apple-darwin/bitcoin-dee690257c79-arm64-apple-darwin.tar.gz
  a71685a3ff62fbf456fc1d3c0f78ab7ab1d5a4a094c1f43d3829730d65a3ec51  guix-build-dee690257c79/output/dist-archive/bitcoin-dee690257c79.tar.gz
  d5c74167b2260b7de72fdbac77eedacbbb929214e279e07f41c3236ea20116b6  guix-build-dee690257c79/output/powerpc64-linux-gnu/SHA256SUMS.part
  03c043daa971f70886805c9e4174acd600e5cda03ae0e827043c4df81a6a6426  guix-build-dee690257c79/output/powerpc64-linux-gnu/bitcoin-dee690257c79-powerpc64-linux-gnu-debug.tar.gz
  3884423f67d48f671a90169962f9568dcbd8df582cd269df29a4399dc9c70e99  guix-build-dee690257c79/output/powerpc64-linux-gnu/bitcoin-dee690257c79-powerpc64-linux-gnu.tar.gz
  cee2847d4feeadebfdf4d37110faafa250824c68cfefc3bfa289ae8b09599a20  guix-build-dee690257c79/output/powerpc64le-linux-gnu/SHA256SUMS.part
  ca9f1085184b50e7bd377b68a58afd42ac4055ffe214eede8c6ca7291e00b45a  guix-build-dee690257c79/output/powerpc64le-linux-gnu/bitcoin-dee690257c79-powerpc64le-linux-gnu-debug.tar.gz
  b9cadff5b30244f6cb4f277e1ac610841332cb0d732637fe438fd51c88adf5a4  guix-build-dee690257c79/output/powerpc64le-linux-gnu/bitcoin-dee690257c79-powerpc64le-linux-gnu.tar.gz
  db6e45d9fc8868e5cb25fa3d0789247db10a6bab2a6f3866a27d641ae00bd03b  guix-build-dee690257c79/output/riscv64-linux-gnu/SHA256SUMS.part
  751e90c0aefa249f175c0c1707dc5eee3637e2b9657a9ee1b2068479e9830ff0  guix-build-dee690257c79/output/riscv64-linux-gnu/bitcoin-dee690257c79-riscv64-linux-gnu-debug.tar.gz
  8dd87bb24f1fbdba67fb6b72ab5fc93532452eada8c2942463100b8ed8118ae3  guix-build-dee690257c79/output/riscv64-linux-gnu/bitcoin-dee690257c79-riscv64-linux-gnu.tar.gz
  b53fe650f431e394a5af6bd809401551e3c36e4f7d90c31f39f51c5fe236efaf  guix-build-dee690257c79/output/x86_64-apple-darwin/SHA256SUMS.part
  b35b4ae345a625e42b8315acd9b75bb651f60265f67ec6f7d78a41c8d21c635e  guix-build-dee690257c79/output/x86_64-apple-darwin/bitcoin-dee690257c79-x86_64-apple-darwin-unsigned.dmg
  6748e4c64ae069632624cbec20de7f03eca1a7578fcef96607f3e8cb0ac4106e  guix-build-dee690257c79/output/x86_64-apple-darwin/bitcoin-dee690257c79-x86_64-apple-darwin-unsigned.tar.gz
  6415cbecfa18002a1e4865b996268558b4f26121118fd73b0ef4ca44967f34c7  guix-build-dee690257c79/output/x86_64-apple-darwin/bitcoin-dee690257c79-x86_64-apple-darwin.tar.gz
  f0a8d8837e80e42b5d77c3efc9d746c11ffc9909aece16dcba0a10726e1be7e0  guix-build-dee690257c79/output/x86_64-linux-gnu/SHA256SUMS.part
  418f4607159387ee17657120beda1c741fa6060e4457b304b857aa6c33859e6a  guix-build-dee690257c79/output/x86_64-linux-gnu/bitcoin-dee690257c79-x86_64-linux-gnu-debug.tar.gz
  d384fe8f2d6515c256a371ad84ae76ac9727ffd49887c0e22bb5ada4cd890092  guix-build-dee690257c79/output/x86_64-linux-gnu/bitcoin-dee690257c79-x86_64-linux-gnu.tar.gz
  3f441eb2d68ddd530b32f56a16ff639b27dcc58b5ba395e40f4c2be805f6cd8b  guix-build-dee690257c79/output/x86_64-w64-mingw32/SHA256SUMS.part
  d3d22479adc554ced965c4bde79f4c919e6b216acfca598fac4ebfc3b0e2d39d  guix-build-dee690257c79/output/x86_64-w64-mingw32/bitcoin-dee690257c79-win64-debug.zip
  89de153f5d839df6cd1266f8b3cb3ad455b2f6c43cee7644889da1e3efc7b2f1  guix-build-dee690257c79/output/x86_64-w64-mingw32/bitcoin-dee690257c79-win64-setup-unsigned.exe
  ba30708b9551b3794e4d2726aff27775dd4f1b347ea71b3fe1b00e39e8b7a594  guix-build-dee690257c79/output/x86_64-w64-mingw32/bitcoin-dee690257c79-win64-unsigned.tar.gz
  ab6a2b7a3e899fc58030a44b555e449d64aa69b5215d761bc550946b14f93515  guix-build-dee690257c79/output/x86_64-w64-mingw32/bitcoin-dee690257c79-win64.zip
  ```

ACKs for top commit:
  Sjors:
    tACK dee690257c
  achow101:
    ACK dee690257c
  hebasto:
    ACK dee690257c, I have reviewed the code and it looks OK.

Tree-SHA512: e0605f049404905db94ea61473b2b68df008ceb86b9a09d8562ca3acc1f3a3be3893149fc62d189a6fbf24cfc76c393f2d1a1215292e9ae5dc4afc199e876821
2023-01-24 13:34:01 -05:00
MarcoFalke
30f553d457 Merge bitcoin/bitcoin#26707: clang-tidy: Fix performance-*move* warnings in headers
1308b837dc clang-tidy: Fix `performance-no-automatic-move` in headers (Hennadii Stepanov)
0a5dc030b9 clang-tidy: Fix `performance-move-const-arg` in headers (Hennadii Stepanov)

Pull request description:

  Split from bitcoin/bitcoin#26705 as was requested in https://github.com/bitcoin/bitcoin/pull/26705#issuecomment-1353293405.

  To test this PR, consider applying a diff as follows:
  ```diff
  --- a/src/.clang-tidy
  +++ b/src/.clang-tidy
  @@ -1,16 +1,7 @@
   Checks: '
   -*,
  -bugprone-argument-comment,
  -bugprone-use-after-move,
  -misc-unused-using-decls,
  -modernize-use-default-member-init,
  -modernize-use-nullptr,
  -performance-for-range-copy,
   performance-move-const-arg,
   performance-no-automatic-move,
  -performance-unnecessary-copy-initialization,
  -readability-redundant-declaration,
  -readability-redundant-string-init,
   '
   WarningsAsErrors: '
   bugprone-argument-comment,
  @@ -28,4 +19,4 @@ readability-redundant-string-init,
   CheckOptions:
    - key: performance-move-const-arg.CheckTriviallyCopyableMove
      value: false
  -HeaderFilterRegex: './qt'
  +HeaderFilterRegex: '.'
  ```

ACKs for top commit:
  fanquake:
    ACK 1308b837dc

Tree-SHA512: b7ef9a3e789846130ab4c3fd6fbe8d887bdbcd438e4cbc78e2b1ac01f819ae13d7f69c2a25f480bd36e3e7f58886a7d5a8609a3c3275c315e0697cd4010474bd
2023-01-24 16:28:08 +01:00
MarcoFalke
fa9becfe1c streams: Add DataStream without ser-type and ser-version
The moved parts can be reviewed with "--color-moved=dimmed-zebra".
The one-char changes can be reviewed with "--word-diff-regex=.".
2023-01-24 13:18:09 +01:00
MarcoFalke
3ce7b27124 Merge bitcoin/bitcoin#26930: fuzz: Actually use mocked mempool in tx_pool target
9ab62d71fb [fuzz] Actually use mocked mempool in tx_pool target (dergoegge)

Pull request description:

  The current tx_pool target uses the default mempool, making the target non-deterministic. This PR replaces the active chainstate's mempool (i.e. the node's default mempool) with the already present mocked mempool in the target.

ACKs for top commit:
  fanquake:
    ACK 9ab62d71fb

Tree-SHA512: fe9af3dbdd13cb569fdc2ddbb4290b5ce94206ae83d94267c6365ed0ee9bbe072fcfe7fd632a1a8522dce44608e89aba2f398c1e20bd250484bbadb78143320c
2023-01-24 12:54:48 +01:00
fanquake
f1b5d6be57 Merge bitcoin/bitcoin#26955: wallet: permit mintxfee=0
f11eb1fe27 wallet: permit mintxfee=0 (willcl-ark)

Pull request description:

  Fixes #26797

  Permit nodes to use `-mintxfee=0`. Values below 0 are handled by the ParseMoney() check.

ACKs for top commit:
  MarcoFalke:
    review ACK f11eb1fe27
  john-moffett:
    ACK f11eb1fe27

Tree-SHA512: 3bf50362bced4fee8e3a846cfb46f1c65dd607c9c824aa3f8c52294371b0646d167a04772d5302bdbee35bbaf407ef0aa634228f70e522c3e423f4213b4ae071
2023-01-24 11:49:28 +00:00
MarcoFalke
837e9ed611 Merge bitcoin/bitcoin#26898: fuzz: Add PartiallyDownloadedBlock target
a1c36275b5 [fuzz] Assert that omitting missing transactions always fails block reconstruction (dergoegge)
a8ac61ab5e [fuzz] Add PartiallyDownloadedBlock target (dergoegge)
42bd4c7468 [block encodings] Avoid fuzz blocking asserts in PartiallyDownloadedBlock (dergoegge)
1429f83770 [block encodings] Make CheckBlock mockable for PartiallyDownloadedBlock (dergoegge)

Pull request description:

  This PR adds a fuzz target for `PartiallyDownloadedBlock`, which we currently do not have any coverage for.

ACKs for top commit:
  mzumsande:
    Code Review ACK a1c36275b5
  MarcoFalke:
    re-ACK a1c36275b5  🎼

Tree-SHA512: 01ae452fe457da0c8f2b28c72091d40807c56a9e5d0f80b55f166b67be50baf80a02f53d4cbe9736bb22424cca1758b87e4e471b8a24e756c22563a2640e9a5f
2023-01-24 12:38:26 +01:00
MarcoFalke
75e752f134 Merge bitcoin/bitcoin#26954: test: Avoid rpc timeout in p2p_headers_sync_with_minchainwork
fa952fad2f test: Avoid rpc timeout in p2p_headers_sync_with_minchainwork (MarcoFalke)

Pull request description:

  When running a lot of tests in parallel, I get `JSONRPCException: 'generatetoaddress' RPC took longer than 30.000000 seconds.`

  The general recommendation, if running into timeouts, is to increase the `--timeout-factor`. However, I think that the default timeout values should be suitable to run the tests out of the box on reasonable hardware.

ACKs for top commit:
  fanquake:
    ACK fa952fad2f

Tree-SHA512: b7eeda54f8db900f077417c0431f659c67e686e2fc078f8c713e37ed75b8bc862814ce20e8400741638e35e224d7284ad16172bf5f82168f803376d0c9ec4524
2023-01-24 12:24:09 +01:00
MarcoFalke
fab958290b refactor: Remove c_str from util/check 2023-01-24 12:09:29 +01:00
MarcoFalke
fcff639af1 Merge bitcoin/bitcoin#26958: build: fix usage of -Wloop-analysis
166e0c057c build: fix usage of -Wloop-analysis (fanquake)

Pull request description:

  Looks like I introduced this in
  5ced925283.

ACKs for top commit:
  hebasto:
    ACK 166e0c057c, tested on Ubuntu 22.04:
  jarolrod:
    ACK 166e0c057c

Tree-SHA512: ad1e7f39207da232dd7065e91b3a856c20d88df43908a4bf327fba1afc424f5dd84b546bf89c23da52765839aa8e5e278ee6ed0033ee8fae760a64a800c2dd42
2023-01-24 09:10:23 +01:00
Andrew Chow
4de02def84 qt: Persist Mask Values option
The mask values option is memory only. If a user has enabled this
option, it's reasonable to expect that they would want to have it
enabled on the next start.
2023-01-23 20:59:05 -05:00
Andrew Chow
b9d5674541 init: Remove sensitive flag from rpcbind 2023-01-23 17:25:02 -05:00
Cory Fields
df7ae8b7ca depends: systemtap: remove variadic params that trigger compiler warnings 2023-01-23 18:12:26 +00:00
fanquake
166e0c057c build: fix usage of -Wloop-analysis
Looks like I introduced this in
5ced925283.
2023-01-23 17:57:40 +00:00
fanquake
a62231bca6 Merge bitcoin/bitcoin#26690: wallet: Refactor database cursor into its own object with proper return codes
4aebd832a4 db: Change DatabaseCursor::Next to return status enum (Andrew Chow)
d79e8dcf29 wallet: Have cursor users use DatabaseCursor directly (Andrew Chow)
7a198bba0a wallet: Introduce DatabaseCursor RAII class for managing cursor (Andrew Chow)
69efbc011b Move SafeDbt out of BerkeleyBatch (Andrew Chow)

Pull request description:

  Instead of having database cursors be tied to a particular `DatabaseBatch` object and requiring its setup and teardown be separate functions in that batch, we can have cursors be separate RAII classes. This makes it easier to create and destroy cursors as well as having cursors that have slightly different behaviors.

  Additionally, since reading data from a cursor is a tri-state, this PR changes the return value of the `Next` function (formerly `ReadAtCursor`) to return an Enum rather than the current system of 2 booleans. This greatly simplifies and unifies the code that deals with cursors as now there is no confusion as to what the function returns when there are no records left to be read.

  Extracted from #24914

ACKs for top commit:
  furszy:
    diff ACK 4aebd83
  theStack:
    Code-review ACK 4aebd832a4

Tree-SHA512: 5d0be56a18de5b08c777dd5a73ba5a6ef1e696fdb07d1dca952a88ded07887b7c5c04342f9a76feb2f6fe24a45dc31f094f1f5d9500e6bdf4a44f4edb66dcaa1
2023-01-23 17:56:16 +00:00
dergoegge
a1c36275b5 [fuzz] Assert that omitting missing transactions always fails block reconstruction 2023-01-23 17:29:41 +01:00
dergoegge
a8ac61ab5e [fuzz] Add PartiallyDownloadedBlock target 2023-01-23 17:29:41 +01:00
dergoegge
42bd4c7468 [block encodings] Avoid fuzz blocking asserts in PartiallyDownloadedBlock 2023-01-23 17:18:35 +01:00
dergoegge
1429f83770 [block encodings] Make CheckBlock mockable for PartiallyDownloadedBlock 2023-01-23 17:18:35 +01:00
MarcoFalke
5271c77f83 Merge bitcoin/bitcoin#26826: refactor: remove windows-only compat.h usage in randomenv
b358bde020 randomenv: consolidate WIN32 #ifdefs (fanquake)
fff80cd248 random: remove windows-only compat.h include in randomenv (fanquake)

Pull request description:

  Similar to #26814.

  Having a windows-only include of compat.h is confusing, not-only because it's already included globally via util/time.h, but also because it's unclear why compat.h is included (neither of the required headers are included there).

  This change is related to removing the use of compat.h as a miscellaneous catch-all for unclear/platform specific includes. Somewhat prompted by IWYU-related discussion here: https://github.com/bitcoin/bitcoin/pull/26763/files#r1058861693.

ACKs for top commit:
  hebasto:
    ACK b358bde020.
  TheCharlatan:
    ACK b358bde020

Tree-SHA512: d46dffe36a17ad0f9374a55e0ecaf2d60d0b473c8fc9ad6f3005142014c08a7c10bae4948856531abb443f5e0bd6062958fe574197e282dad22ae50134d71e5f
2023-01-23 16:36:27 +01:00
MarcoFalke
fa952fad2f test: Avoid rpc timeout in p2p_headers_sync_with_minchainwork 2023-01-23 16:35:13 +01:00
willcl-ark
f11eb1fe27 wallet: permit mintxfee=0
Fixes #26797

Permit nodes to use a mintxfee of `0` if they choose.
Values below 0 are handled by the ParseMoney() check.
2023-01-23 13:35:04 +00:00
fanquake
dee690257c build: pass --enable-debug to sqlite when DEBUG=1 2023-01-23 10:21:08 +00:00
fanquake
807b61fcca build: use more recommended sqlite3 compile options
See https://www.sqlite.org/compile.html.

DSQLITE_DQS
> This setting disables the double-quoted string literal misfeature.

DSQLITE_DEFAULT_MEMSTATUS
> This setting causes the sqlite3_status() interfaces that track
> memory usage to be disabled.
> This helps the sqlite3_malloc() routines run much faster, and since
> SQLite uses sqlite3_malloc() internally, this helps to make the
> entire library faster.

DSQLITE_OMIT_DEPRECATED
> Omitting deprecated interfaces and features will not help SQLite
> to run any faster.
> It will reduce the library footprint, however. And it is the
> right thing to do.

DSQLITE_OMIT_SHARED_CACHE
> Omitting the possibility of using shared cache allows many
> conditionals in performance-critical sections of the code to be
> eliminated. This can give a noticeable improvement in performance.

Also: https://www.sqlite.org/sharedcache.html
> Shared-cache mode is an obsolete feature.
> The use of shared-cache mode is discouraged.
> Most use cases for shared-cache are better served by WAL mode.
> Applications that build their own copy of SQLite from source code
> are encouraged to use the -DSQLITE_OMIT_SHARED_CACHE compile-time
> option, as the resulting binary will be both smaller and faster.

DSQLITE_OMIT_JSON
Starting with sqlite 3.38.0 the JSON extension became opt-out rather
than opt-in, so we disable it here.

--disable-rtree
> An R-Tree is a special index that is designed for doing range queries.
> R-Trees are most commonly used in geospatial systems...
https://www.sqlite.org/rtree.html

--disable-fts4 --disable-fts5
> FTS5 is an SQLite virtual table module that provides full-text
> search functionality to database applications.

DSQLITE_LIKE_DOESNT_MATCH_BLOBS
> simplifies the implementation of the LIKE optimization and allows
> queries that use the LIKE optimization to run faster.

DSQLITE_OMIT_DECLTYPE
> By omitting the (seldom-needed) ability to return the declared type of
> columns from the result set of query, prepared statements can be made
> to consume less memory.

DSQLITE_OMIT_PROGRESS_CALLBACK
> By omitting this interface, a single conditional is removed from the
> inner loop of the bytecode engine, helping SQL statements to run slightly
> faster.

DSQLITE_OMIT_AUTOINIT
> with the SQLITE_OMIT_AUTOINIT option, the automatic initialization is omitted.
> This helps many API calls to run a little faster
> it also means that the application must call sqlite3_initialize()
manually.
2023-01-23 10:17:48 +00:00
fanquake
a66d82e25f depends: systemtap 4.8 2023-01-23 10:09:18 +00:00
MarcoFalke
a8c1ea50c7 Merge bitcoin/bitcoin#26919: scripted-diff: use RPCArg::Optional::OMITTED over OMITTED_NAMED_ARG
83f70c8e86 doc: improve doc for RPCArg::Optional::OMITTED (fanquake)
ea8c7daf7a scripted-diff: use RPCArg::Optional::OMITTED over OMITTED_NAMED_ARG (fanquake)

Pull request description:

  Remove deprecated `RPCArg::Optional::OMITTED_NAMED_ARG` in favour of `OMITTED`.

ACKs for top commit:
  kouloumos:
    re-ACK 83f70c8e86
  aureleoules:
    ACK 83f70c8e86

Tree-SHA512: caba82c0f9f7969d7534bdcdcdfd80b94c16750598343ce63af28e0d579bfd49405bf28acd66dbc6c9a720f040b0e146a5efdff79ebc5ac7364dab54ab16c4a6
2023-01-23 10:12:58 +01:00
MarcoFalke
f99b85642f Merge bitcoin/bitcoin#26944: depends: fix systemtap download URL
d81ca6619a depends: fix systemtap download URL (fanquake)

Pull request description:

  The URL has changed, and the current one 404s.

ACKs for top commit:
  hebasto:
    ACK d81ca6619a, verified each link: the old one returns 404, the new one is OK.
  theStack:
    ACK d81ca6619a
  jarolrod:
    ACK d81ca6619a

Tree-SHA512: e3240efd97003b4063c84bf72638d005f1629d0753359520353e249745fde185ef8e23fcd504037486bce4c4453dcb86f972e33111486ace8ad65746636e1499
2023-01-23 09:23:40 +01:00
fanquake
d81ca6619a depends: fix systemtap download URL 2023-01-22 15:58:04 +00:00
fanquake
83f70c8e86 doc: improve doc for RPCArg::Optional::OMITTED 2023-01-22 15:05:14 +00:00
fanquake
ea8c7daf7a scripted-diff: use RPCArg::Optional::OMITTED over OMITTED_NAMED_ARG
-BEGIN VERIFY SCRIPT-
sed -i -e "/Deprecated alias for OMITTED, can be removed/d" src/rpc/util.h src/rpc/util.cpp
sed -i -e "s/OMITTED_NAMED_ARG/OMITTED/g" $(git grep -l "OMITTED_NAMED_ARG" src/)
-END VERIFY SCRIPT-
2023-01-22 15:01:48 +00:00
fanquake
ad09b76275 Merge bitcoin/bitcoin#26471: Reduce default mempool size in -blocksonly mode
8e85164e7d doc: release note on mempool size in -blocksonly (willcl-ark)
ae797463dc doc: Update blocksonly behaviour in reduce-memory (willcl-ark)
1134686ef9 mempool: Don't share mempool with dbcache in blocksonly (willcl-ark)

Pull request description:

  Fixes #9526

  When `-blocksonly` has been set reduce default mempool size to avoid surprising resource usage via sharing un-used mempool cache space with dbcache.

  In comparison to https://github.com/bitcoin/bitcoin/pull/9569 which either set `maxmempool` size to 0 when `-blocksonly` was set or else errored on startup, this change will permit `maxmempool` options being set.

  This preserves the current (surprising?) behaviour of having a functional mempool in `-blocksonly` mode, to permit whitelisted peer transaction relay, whilst reducing average runtime memory usage for blocksonly nodes which either use the default settings or have otherwise configured a `maxmempool` size.

  To use the previous old defaults node operators can configure their node with: `-blocksonly -maxmempool=300`.

ACKs for top commit:
  ajtowns:
    ACK 8e85164e7d
  stickies-v:
    re-ACK 8e85164e7d

Tree-SHA512: 1c461c24b6f14ba02cfe4e2cde60dc629e47485db5701bca3003b8df79e3aa311c0c967979f6a1dca3ba69f5b1e45fa2db6ff83352fdf2d4349d5f8d120e740d
2023-01-22 14:57:16 +00:00
fanquake
bf9361d375 Merge bitcoin/bitcoin#26941: test: Fix intermittent feature_rbf issue
fa88c043d1 test: Fix intermittent feature_rbf issue (MarcoFalke)

Pull request description:

  The miniwallet will rescan the chain and mempool on construction. If the mempools are still in sync, it may lead to crashes. Fix that by moving the sync first.

  Fixes #26937

ACKs for top commit:
  theStack:
    Code-review ACK fa88c043d1

Tree-SHA512: 5ffcd5e91118b57811b62f12454da8ae3ca98ffad175cd895cd41b63d7cf420906b1e15a4d4489d223d6b21ab796f9839676af8a5f340c606868bc249f4ea340
2023-01-22 14:17:49 +00:00
MarcoFalke
fa88c043d1 test: Fix intermittent feature_rbf issue 2023-01-21 11:18:12 +01:00
James O'Beirne
1d7935b45a test: add test for coins view flush behavior using Sync()
Thanks to Marco Falke for help with move semantics.
2023-01-20 10:39:50 -05:00
James O'Beirne
2c3cbd6c00 test: add use of Sync() to coins tests 2023-01-20 10:36:52 -05:00
James O'Beirne
6d8affca96 test: refactor: clarify the coins simulation
Adds comments, slight refactor clarifications to make the code
easier to follow.
2023-01-20 10:36:52 -05:00
James O'Beirne
79cedc36af coins: add Sync() method to allow flush without cacheCoins drop
In certain circumstances, we may want to flush to disk without
emptying `cacheCoins`, which affects performance. UTXO snapshot
activation is one such case.

This method is currently unused and this commit does not
change any behavior.

Incorporates feedback from John Newbery.

Co-authored-by: Suhas Daftuar <sdaftuar@gmail.com>
2023-01-20 10:36:48 -05:00
MarcoFalke
c0b6c40bb0 Merge bitcoin/bitcoin#26934: test: Add unit test for ComputeTapleafHash
f34ada89fd Add unit test for ComputeTapleafHash (Greg Sanders)

Pull request description:

  Quick follow-up to https://github.com/bitcoin/bitcoin/pull/25877

ACKs for top commit:
  sipa:
    ACK f34ada89fd

Tree-SHA512: ebec658c9b33859874a3e5d13ca0a00a2484233f00f2da09c7d3fb47ed7f56fc6d476ddd0473fe1396a514dffd6ea6a200f26c6dbca45bac2473e729ffef04c2
2023-01-20 16:25:21 +01:00
Greg Sanders
f34ada89fd Add unit test for ComputeTapleafHash 2023-01-20 09:36:51 -05:00
willcl-ark
8e85164e7d doc: release note on mempool size in -blocksonly
Adds a release note detailing the new mempool sizing behaviour when
running in blocksonly mode, and instruction on how to override the new
defaults.
2023-01-20 13:54:05 +00:00
willcl-ark
ae797463dc doc: Update blocksonly behaviour in reduce-memory
Changes to the default mempool allocation size now documented.

Provides users with guidance on the mempool implications of -blocksonly
mode, along with instructions on how to re-enable old behaviour.
2023-01-20 13:27:07 +00:00
willcl-ark
1134686ef9 mempool: Don't share mempool with dbcache in blocksonly
When -blockonly is set, reduce mempool size to 5MB unless -maxmempool
is also set.

See #9569
2023-01-20 12:53:13 +00:00
MarcoFalke
fafeddfe0e rpc: Throw more user friendly arg type check error 2023-01-20 13:26:47 +01:00
dergoegge
9ab62d71fb [fuzz] Actually use mocked mempool in tx_pool target 2023-01-20 12:15:01 +01:00
fanquake
392dc68e37 Merge bitcoin/bitcoin#26924: refactor: Add missing includes to fix gcc-13 compile error
fadeb6b103 Add missing includes to fix gcc-13 compile error (MarcoFalke)

Pull request description:

  On current master:

  ```
    CXX      support/libbitcoin_util_a-lockedpool.o
  support/lockedpool.cpp: In member function ‘void Arena::free(void*)’:
  support/lockedpool.cpp:99:20: error: ‘runtime_error’ is not a member of ‘std’
     99 |         throw std::runtime_error("Arena: invalid or double free");
        |                    ^~~~~~~~~~~~~
  support/lockedpool.cpp:22:1: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; did you forget to ‘#include <stdexcept>’?
     21 | #include <algorithm>
    +++ |+#include <stdexcept>
     22 | #ifdef ARENA_DEBUG
  support/lockedpool.cpp: In member function ‘void LockedPool::free(void*)’:
  support/lockedpool.cpp:320:16: error: ‘runtime_error’ is not a member of ‘std’
    320 |     throw std::runtime_error("LockedPool: invalid address not pointing to any arena");
        |                ^~~~~~~~~~~~~
  support/lockedpool.cpp:320:16: note: ‘std::runtime_error’ is defined in header ‘<stdexcept>’; did you forget to ‘#include <stdexcept>’?

ACKs for top commit:
  hebasto:
    ACK fadeb6b103.
  fanquake:
    ACK fadeb6b103 - tested this fixes compilation with GCC 13. I don't think theres a need to do anything else here, and that'd also just potentially complicate backporting.

Tree-SHA512: 99f79cf385c913138a9cf9fc23be0a3a067b0a28518b8bdc033a7220b85bbc5d18f5356c5bdad2f628c22abb87c18b232724f606eba6326c031518559054be31
2023-01-20 10:26:36 +00:00
MarcoFalke
eebc24bfc6 Merge bitcoin/bitcoin#26887: RPC: make RPCResult::MatchesType return useful errors
3d1a4d8a45 RPC: make RPCResult::MatchesType return useful errors (Anthony Towns)

Pull request description:

  Currently if you don't correctly update the description of the return value for an RPC call, you essentially just get an assertion failure with no useful information; this generates a description of the problems instead.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3d1a4d8a45 🌷

Tree-SHA512: cf0580b7046faab0128672a74f8cc5a1655dfdca6646a2e38b51f0fb5f672c98aad6cd4c5769454a2d644a67da639ccb1c8ff5d24d3d6b4446a082398a643722
2023-01-20 10:37:23 +01:00
Andrew Chow
58da1619be Merge bitcoin/bitcoin#25877: refactor: Do not use CScript for tapleaf scripts until the tapleaf version is known
dee89438b8 Abstract out ComputeTapbranchHash (Russell O'Connor)
8e3fc99427 Do not use CScript for tapleaf scripts until the tapleaf version is known (Russell O'Connor)

Pull request description:

  While BIP-341 calls the contents of tapleaf a "script", only in the case that the tapleaf version is `0xc0` is this script known to be a tapscript.  Otherwise the tapleaf "script" is simply an uninterpreted string of bytes.

  This PR corrects the issue where the type `CScript` is used prior to the tapleaf version being known to be a tapscript.  This prevents `CScript` methods from erroneously being called on non-tapscript data.

  A second commit abstracts out the TapBranch hash computation in the same manner that the TapLeaf computation is already abstracted.  These two abstractions ensure that the TapLeaf and TapBranch tagged hashes are always constructed properly.

ACKs for top commit:
  ajtowns:
    ACK dee89438b8
  instagibbs:
    ACK dee89438b8
  achow101:
    ACK dee89438b8
  sipa:
    ACK dee89438b8
  aureleoules:
    reACK dee89438b8 - I verified that there is no behavior change.

Tree-SHA512: 4a1d37f3e9a1890e7f5eadcf65562688cc451389581fe6e2da0feb2368708edacdd95392578d8afff05270d88fc61dce732d83d1063d84d12cf47b5f4633ec7e
2023-01-19 17:51:21 -05:00
Anthony Towns
3d1a4d8a45 RPC: make RPCResult::MatchesType return useful errors 2023-01-20 06:24:15 +10:00
MarcoFalke
fadeb6b103 Add missing includes to fix gcc-13 compile error 2023-01-19 19:30:10 +01:00
MarcoFalke
250598a905 Merge bitcoin/bitcoin#26906: test: add an easy way to run linters locally
b68e5a7fef lint: specify the right commit range when running locally (James O'Beirne)
dff7ed5732 test: add an easy way to run linters locally (James O'Beirne)

Pull request description:

  Adds a Dockerfile configuration ~~(originally written mostly by fanquake)~~ that allows straightforward running of linters with compatible versions locally. This removes a ton of annoyance when trying to appease CI, because many of the linter versions are quite old and difficult to maintain locally.

  I realize that people may not be thrilled to add more ancillary tooling to the repo, but I think this makes a lot of sense given the linter versions listed in this container configuration are dictated by this repo (within the CI configuration), so having these things live in two separate places is a recipe for version mismatches.

  Eventually we can likely just use this container on CI directly to avoid any chance of inconsistencies between local dev experience and CI.

ACKs for top commit:
  aureleoules:
    ACK b68e5a7fef
  stickies-v:
    ACK b68e5a7fe
  john-moffett:
    ACK b68e5a7fef

Tree-SHA512: 7ef7a5dae023d81fdb6296d5d92dfa074ee321c7993e607c9f014d0f21c91558611aa00fc3ce1edc7b5e68371aea0d27fa1931291a79bb867a6c783bb536775f
2023-01-19 19:01:28 +01:00
Sebastian Falbesoner
8609f24be2 test: refactor: simplify p2p_eviction.py by using MiniWallet
Also, use the pre-mined chain of the test framework rather than
mining 100 blocks manually on each run.
2023-01-19 16:11:27 +01:00
Sebastian Falbesoner
7aa4b32cd4 test: refactor: simplify p2p_tx_download.py by using MiniWallet 2023-01-19 16:08:55 +01:00
MarcoFalke
b5c88a5479 Merge bitcoin/bitcoin#26909: net: prevent peers.dat corruptions by only serializing once
5eabb61b23 addrdb: Only call Serialize() once (Martin Zumsande)
da6c7aeca3 hash: add HashedSourceWriter (Martin Zumsande)

Pull request description:

  There have been various reports of corruption of `peers.dat` recently, see #26599.
  As explained in [this post](https://github.com/bitcoin/bitcoin/issues/26599#issuecomment-1381082886) in more detail, the underlying issue is likely that we currently serialize `AddrMan` twice - once for the file stream, once for the hasher that helps create the checksum - and if `AddrMan` changes in between these two calls, the checksum doesn't match the data and the resulting `peers.dat` is corrupted.

  This PR attempts to fix this by introducing and using `HashedSourceWriter` - a class that keeps a running hash while serializing data, similar to the existing `CHashVerifier` which does the analogous thing while unserializing data. Something like this was suggested before, see https://github.com/bitcoin/bitcoin/pull/10248#discussion_r120694343.

  Fixes #26599 (not by changing the behavior in case of a crash, but by hopefully fixing the underlying cause of these corruptions).

ACKs for top commit:
  sipa:
    utACK 5eabb61b23
  naumenkogs:
    utACK 5eabb61b23

Tree-SHA512: f19ad37c37088d3a9825c60de2efe85cc2b7a21b79b9156024d33439e021443ef977a5f8424a7981bcc13d73d11e30eaa82de60e14d88b3568a67b03e02b153b
2023-01-19 16:03:08 +01:00
MarcoFalke
05e3468fb3 Merge bitcoin/bitcoin#26686: fuzz: Enable erlay setting in process_message(s) targets
58c2bbdb55 [fuzz] Enable erlay in process_message(s) targets (dergoegge)

Pull request description:

  The process_message(s) targets can't exercise the Erlay logic at the moment as the config setting is off by default and not switched on in the fuzz targets.

  This PR enables the `-txreconciliation` setting in both targets.

ACKs for top commit:
  fanquake:
    ACK 58c2bbdb55

Tree-SHA512: a2754fd04549bdcac94d8225244c5c83fe4c26114c0c2fdf316257480625e05e4e6b1b791974e1f1021451d3f81cb59a109261fb73178ad03911f0a3db963077
2023-01-19 15:56:58 +01:00
fanquake
2343886217 Merge bitcoin/bitcoin#26920: doc: add release note for #25957 (fast wallet rescan)
783288334c doc: add release note for #25957 (fast wallet rescan) (Sebastian Falbesoner)

Pull request description:

  This PR adds a missing release note for #25957.

ACKs for top commit:
  Sjors:
    ACK 783288334c

Tree-SHA512: 817aa3d27b3f839de3975ace7c8ec59bcc4dbe4b5628bf64153e503cd143599d8923bd7e181ad5b196dacf1a9078347825bc40d4de5c6e2df9ed12e752217094
2023-01-19 13:40:23 +00:00
Sebastian Falbesoner
783288334c doc: add release note for #25957 (fast wallet rescan) 2023-01-19 13:40:26 +01:00
MarcoFalke
92dcbe9cc3 Merge bitcoin/bitcoin#23395: util: Add -shutdownnotify option
d96d97ad30  doc: Add release note for shutdownnotify. (klementtan)
0bd73e2c45 util: Add -shutdownnotify option. (klementtan)

Pull request description:

  **Description**: Similar to `-startupnotify`, this PR adds a new option to allow users to specify a command to be executed when Bitcoin Core shuts down.

  **Note**: The `shutdownnotify` commands will not be executed if bitcoind shut down due to *unexpected* reasons (ie `killall -9 bitcoind`).

  ### Testing:
  **Normal shutdown commands**
  ```
  # start bitcoind with shutdownnotify optioin
  ./src/bitcoind -signet -shutdownnotify="touch foo.txt"

  # shutdown bitcoind
  ./src/bitcoin-cli -signet stop

  # check that foo.txt has been created
  ```

  **Final RPC call**
  Commands:
  ```
  $  ./src/bitcoind -signet -nolisten -noconnect -shutdownnotify="./src/bitcoin-cli -signet getblockchaininfo > tmp.txt"
  $ ./src/bitcoin-cli stop
  $ cat tmp.txt
  ```
  <details>
  <summary>Screen Shot</summary>

  ![image](https://user-images.githubusercontent.com/49265907/141186183-cbc6f82c-400d-4a8b-baba-27c0346c2c8a.png)
  </details>

ACKs for top commit:
  achow101:
    ACK d96d97ad30
  1440000bytes:
    ACK d96d97ad30
  theStack:
    re-ACK d96d97ad30

Tree-SHA512: 16f7406fd232e8b97aea5e58854c84755b0c35c88cb3ef9ee123b29a1475a376122b1e100da860cc336d4d657e6046a70e915fdb9b70c9fd097c6eef1b028161
2023-01-19 10:34:54 +01:00
ishaanam
a5b4883fb4 rpc: extract psbt updating logic into ProcessPSBT
This function is called from utxoupdatepsbt and will be modified
in a following commit to allow for updating inputs with the
`non_witness_utxo` as well.
2023-01-18 20:38:23 -05:00
Andrew Chow
8ae2808a43 Merge bitcoin/bitcoin#25659: wallet: simplify ListCoins implementation
a2ac6f9582 wallet: unify FindNonChangeParentOutput functions (furszy)
b3f4e82737 wallet: simplify ListCoins implementation (furszy)

Pull request description:

  Focused on the following changes:

  1) Removed the entire locked coins lookup that was inside `ListCoins` by including them directly on the `AvailableCoins` result (where we were skipping them before).
  2) Unified both `FindNonChangeParentOutput` functions (only called from `ListCoins`)

ACKs for top commit:
  achow101:
    ACK a2ac6f9582
  aureleoules:
    ACK a2ac6f9582, LGTM
  theStack:
    Code-review ACK a2ac6f9582

Tree-SHA512: f72b21ee1600c5992559b5dcd8ff260527afac2ec696737f998343a0850b84d439e7f86ea52a14cc0cddabf132a30bf5b52fb34950578ac323083d4375b937f1
2023-01-18 14:26:39 -05:00
stickies-v
545ff924ab refactor: use string_view for RPC named argument values
Minimize copying RPC named argument values when calling .substr() by
using std::string_view instead of std::string.
2023-01-18 17:08:58 +00:00
stickies-v
7727603e44 refactor: reduce unnecessary complexity in ParseNonRFCJSONValue
Since https://github.com/jgarzik/univalue/pull/31, UniValue::read() can now
parse raw literals directly, so there is no more need to wrap them into an
array first.
2023-01-18 17:01:21 +00:00
fanquake
44f3c7de21 contrib: remove install_db4.sh
Now that we can build a bdb-only depends prefix, there is no need to
maintain a bdb-building bash script, that does the same things as
depends, except worse, as it's missing patches and workarounds. i.e #26623.
2023-01-18 16:59:02 +00:00
fanquake
14ce84388f doc: add new NO_* options from #26833 2023-01-18 16:59:01 +00:00
furszy
6d31900e52 wallet: migrate wallet, exit early if no legacy data exist
otherwise the process will create a backup file then return
an error when notices that the db is already running sqlite.
2023-01-18 13:47:31 -03:00
Hennadii Stepanov
1308b837dc clang-tidy: Fix performance-no-automatic-move in headers
See https://clang.llvm.org/extra/clang-tidy/checks/performance/no-automatic-move.html
2023-01-18 15:47:06 +00:00
Hennadii Stepanov
0a5dc030b9 clang-tidy: Fix performance-move-const-arg in headers
See https://clang.llvm.org/extra/clang-tidy/checks/performance/move-const-arg.html
2023-01-18 15:47:06 +00:00
MarcoFalke
aef8b4f43b Merge bitcoin/bitcoin#26226: Bump minimum python version to 3.7
fa8fe5b696 scripted-diff: Use new python 3.7 keywords (MarcoFalke)
fa2a23548a Revert "contrib: Fix capture_output in getcoins.py" (MarcoFalke)
dddd462137 Bump minimum python version to 3.7 (MarcoFalke)

Pull request description:

  While there is nothing that requires a bump, it may require less maintenance to drop python3.6 support. Python3.7 is available through the package manager on all currently supported operating systems.

ACKs for top commit:
  jamesob:
    ACK fa8fe5b696
  hebasto:
    ACK fa8fe5b696

Tree-SHA512: f6e080d8751948bb0e01c87be601363158f345e8037b70ce7e1bc507c611eb61600e4f24f1d2f8a6e7e44877ab09319302869e33ce8118c4c4f71fc89c0a1198
2023-01-18 16:46:12 +01:00
fanquake
3fef2944ad Merge bitcoin/bitcoin#26706: doc: Properly report optional RPC args
fad56f7dd6 doc: Properly report optional RPC args (MarcoFalke)
fa09cb6086 refactor: Introduce is_top_level_arg (MarcoFalke)
fab92a5a5a refactor: Remove const to fix performance-move-const-arg clang-tidy errors (MarcoFalke)

Pull request description:

  `OMITTED_NAMED_ARG` and `OMITTED` are a confusing burden:

  * It puts the burden on developers to pick the right one of the two
  * They can be interchanged without introducing a compile failure or other error
  * Picking the wrong one is leading to incorrect docs
  * They are redundant, because the correct one can already be determined by the surrounding type

  Fix all issues by making them an alias of each other; Pick the right one based on the outer type.

ACKs for top commit:
  fanquake:
    ACK fad56f7dd6

Tree-SHA512: 6e7193a05a852ba1618a9cb3261220c7ad3282bc5595325c04437aa811f529a88e2851e9c7dbf9878389b8aa42e98f8817b7eb0260fbb9e123da0893cbae6ca2
2023-01-18 15:43:00 +00:00
fanquake
b52a6c0cf9 Merge bitcoin/bitcoin#26873: doc: add databases/py-sqlite3 to FreeBSD test suite deps
376e01b382 doc: add databases/py-sqlite3 to FreeBSD test suite deps (fanquake)

Pull request description:

  Adds missing documentation. See also https://cirrus-ci.com/task/5639240319500288.

ACKs for top commit:
  john-moffett:
    ACK 376e01b382

Tree-SHA512: a7b8d0bae00c3538934d23eae207b7927a64e748eb228ac6c5754aee0e9b6796c0f39066c7d81cc009bf442c8b1a0c31739adde87d1ebc3445aed54dda47c9ce
2023-01-18 15:38:02 +00:00
James O'Beirne
b68e5a7fef lint: specify the right commit range when running locally
When running lints on Cirrus, a special envvar is set ($CIRRUS_PR);
emulate this when running linters locally by setting $LOCAL_BRANCH
to any value.
2023-01-18 09:48:14 -05:00
James O'Beirne
dff7ed5732 test: add an easy way to run linters locally
Adds a Dockerfile configuration
that allows straightforward running of linters with compatible versions
locally. This removes a ton of annoyance when trying to appease CI,
because many of the linter versions are quite old and difficult to
maintain locally.

I realize that people may not be thrilled to more ancillary tooling to
the repo, but I think this makes a lot of sense given the linter
versions listed in this container configuration are dictated by this
repo (within the CI configuration), so having these things live in
two separate places is a recipe for version mismatches.

Eventually we can likely just use this container on CI directly to avoid
any chance of inconsistencies between local dev experience and CI.
2023-01-18 09:48:08 -05:00
MarcoFalke
8741cd88b4 Merge bitcoin/bitcoin#26913: doc: Clarify debian copyright comment
faa05cd8ce doc: Clarify debian copyright comment (MarcoFalke)

Pull request description:

  Seems fragile to link to an external site for a list of "current" devs. Also, current devs shouldn't matter in this context. It might be better to explain where *all* contributors are found, so do that instead.

ACKs for top commit:
  fanquake:
    ACK faa05cd8ce
  john-moffett:
    ACK faa05cd8ce

Tree-SHA512: 0695c8da86b7c3efbd13e7c1d645528dbd402c18ddd63d8bc532d42de9aa4e40d0f04f3c1e1208806e74ca9fbe3cb967d5206ca59fec20c35cafd22271deecf4
2023-01-18 13:24:53 +01:00
stickies-v
1d02e59901 test: add cases to JSON parsing 2023-01-18 12:24:06 +00:00
MarcoFalke
78c30814f9 Merge bitcoin/bitcoin#26506: refactor: rpc: use convenience fn to auto parse non-string parameters
6d0ab07e81 refactor: use convenience fn to auto parse non-string parameters (stickies-v)

Pull request description:

  Minimizes code duplication and improves function naming by having a single (overloaded) convenience function `ParseIfNonString` that both checks if the parameter is a non-string parameter and automatically parses the value if so.

ACKs for top commit:
  aureleoules:
    ACK 6d0ab07e81

Tree-SHA512: 8cbf68a17cfbdce1e31a19916f447a2965c139fdef00c19e32a9b679f4a4015dfe69ceea0bbe1723711e1c5033ea8d4005d1f4485dfbeea22226140f8cbe8aa3
2023-01-18 13:12:09 +01:00
MarcoFalke
fa8fe5b696 scripted-diff: Use new python 3.7 keywords
-BEGIN VERIFY SCRIPT-
 sed -i 's/universal_newlines/text/g' $(git grep -l universal_newlines)
-END VERIFY SCRIPT-
2023-01-18 13:00:34 +01:00
MarcoFalke
fa2a23548a Revert "contrib: Fix capture_output in getcoins.py"
This reverts commit be59bd17ec
because the changes are no longer needed.
2023-01-18 13:00:29 +01:00
MarcoFalke
dddd462137 Bump minimum python version to 3.7 2023-01-18 12:59:11 +01:00
MarcoFalke
500f25d880 Merge bitcoin/bitcoin#26727: rpc: remove optional from fStateStats fields
1dc0e4bc6f rpc: remove optional from fStateStats fields (fanquake)

Pull request description:

  These are no-longer optional after #26515, so remove the documentation, and no-op `fStateStats` checks.

ACKs for top commit:
  dergoegge:
    Code review ACK 1dc0e4bc6f

Tree-SHA512: 06d4550e866341b379bfdbc72d67d71a3b7ceceec06ebd4c5e6f178b75fe40cbf4aff51adba1bc52590e69e818cbdecb0366bf1528c59c5c3dff5bbdba8eac68
2023-01-18 12:48:22 +01:00
fanquake
c9db788a11 Merge bitcoin/bitcoin#26908: ci: Add missing lint dependency (gpg)
fa87d71872 ci: Add missing lint dependency (MarcoFalke)

Pull request description:

  Also, document each dependency.

  Adding `gpg` avoids errors when running a release or dev branch in the CI:

  01ec5308bf/ci/lint/06_script.sh (L30-L42)

  ```
  bash: line 1: gpg: command not found
  ```

  https://cirrus-ci.com/task/4582854860996608?logs=lint#L185

ACKs for top commit:
  hebasto:
    ACK fa87d71872

Tree-SHA512: 869a3d2feab764b2c8d47d481359680a1d2c54a33b13ca26c5f8ce56cf2f368d4c74637dcbc53fdbf323f10940965c1c0e592e2fb4ce725d5cd467e77e62b6e5
2023-01-18 11:41:12 +00:00
MarcoFalke
d91364fdd9 Merge bitcoin/bitcoin#26912: ci: Bump --combinedlogslen to debug intermittent issues
faddb7373a ci: Bump --combinedlogslen to debug intermittent issues (MarcoFalke)

Pull request description:

  May help to debug intermittent issues such as https://github.com/bitcoin/bitcoin/issues/26808

ACKs for top commit:
  fanquake:
    ACK faddb7373a - if this is going to improve the chance of tracking down intermittent failures.

Tree-SHA512: f844856ede71b9fb816c39bfba6241e35480db71bdc2e534d4746a666114bfc82f9dea804f70201fbf8af32eb579b7eab3c164a0bb2f77268b5554467ff6e97d
2023-01-18 12:03:50 +01:00
MarcoFalke
faa05cd8ce doc: Clarify debian copyright comment 2023-01-18 10:29:24 +01:00
MarcoFalke
faddb7373a ci: Bump --combinedlogslen to debug intermittent issues 2023-01-18 09:35:47 +01:00
MarcoFalke
52d6509497 Merge bitcoin/bitcoin#26904: build: move rpc/request from util lib to common
87a08cba43 build: move rpc/request from util lib to common (fanquake)

Pull request description:

  This is JSON RPC related code that doesn't need to be in util, and should not be required by the kernel.

ACKs for top commit:
  TheCharlatan:
    ACK 87a08cba43

Tree-SHA512: 5f335be9f0f9ff02eff073af47558ecf505c1392c05f18ca24a065b12b8d92529ec3942d84978cc5028c38369c496ed0243653e1fa26d4db2fae26dfe55c3d65
2023-01-18 09:27:29 +01:00
MarcoFalke
e6ff110820 Merge bitcoin/bitcoin#26810: doc: remove nonexistent files from copyright
4bb91be124 debian: remove nonexistent files from copyright (fanquake)

Pull request description:

  The removed files were dropped during a secp256k1 subtree update.

Top commit has no ACKs.

Tree-SHA512: 19ef1cf76908b5468265cc25b76abf8cf3a1dd0d5f7390f9cf4c5cd4c421c8cb04b5991ded7102add896d06555696a8059df37fd1d8f7374487a12dfa594c9cd
2023-01-18 09:12:00 +01:00
Martin Zumsande
5eabb61b23 addrdb: Only call Serialize() once
The previous logic would call it once for serializing into the filestream,
and then again for serializing into the hasher. If AddrMan was changed
in between these calls by another thread, the resulting peers.dat would
be corrupt with non-matching checksum and data.
Fix this by using HashedSourceWriter, which writes the data
to the underlying stream and keeps track of the hash in one go.
2023-01-17 17:20:03 -05:00
Martin Zumsande
da6c7aeca3 hash: add HashedSourceWriter
This class is the counterpart to CHashVerifier, in that it
writes data to an underlying source stream,
while keeping a hash of the written data.
2023-01-17 17:19:56 -05:00
MarcoFalke
fa87d71872 ci: Add missing lint dependency 2023-01-17 19:09:44 +01:00
MarcoFalke
01ec5308bf Merge bitcoin/bitcoin#26716: ci: Build python from source in "lint" task
123043e99c ci: Bump lint task image to Ubuntu Jammy (Hennadii Stepanov)
9b86114058 ci: Use pyenv's `python-build` to install Python in lint task (Hennadii Stepanov)

Pull request description:

  This PR:
  - is an alternative to bitcoin/bitcoin#26581 and bitcoin/bitcoin#26637
  - closes bitcoin/bitcoin#26548

  Key advantages of this PR over others:
  - it uses pyenv's `python-build` [standalone](https://github.com/pyenv/pyenv/tree/master/plugins/python-build#using-python-build-standalone)
  - requires no additional computational resources

  Note for testing. The lint task must success regardless of whether the `python_cache` is populated or invalidated.

ACKs for top commit:
  MarcoFalke:
    ACK 123043e99c
  fanquake:
    ACK 123043e99c

Tree-SHA512: ba0fcdd4f2939a59692b173dcd1f5704444cfcfbb8111538c6f8160056d0536bba250e4f9b0f8c66f8b454e52034bf36ffe6afae76cdc0f7cc5b58b576d790ba
2023-01-17 18:35:47 +01:00
Aurèle Oulès
962a0930e6 Improve address decoding errors 2023-01-17 18:31:51 +01:00
fanquake
89fb354f28 Merge bitcoin/bitcoin#26625: test: Run mempool_packages.py with MiniWallet
fa6b402114 test: Run mempool_packages.py with MiniWallet (MarcoFalke)
fa448c27d2 test: Return fee from MiniWallet (MarcoFalke)
faec09f240 test: Return chain of MiniWallet txs from MiniWallet chain method (MarcoFalke)
faa12d4ccd test: Refactor MiniWallet sign_tx (MarcoFalke)
fa2d82103f test: Return wtxid from create_self_transfer_multi (MarcoFalke)

Pull request description:

  This allows to run the test even when no wallet is compiled in.

  Also, it is a lot nicer to read now.

ACKs for top commit:
  glozow:
    reACK fa6b402

Tree-SHA512: de0338068fd51db01d64ce270f94fd2982a63a6de597325cd1e1f11127e9075bd4aeacace0ed76d09a2db8b962b27237cf11edb4c1fe1a01134d397f8a11bd05
2023-01-17 16:39:45 +00:00
MarcoFalke
f41252f19d Merge bitcoin/bitcoin#26905: refactor: Remove duplication of clang-tidy's check names
06fc29326b refactor: Remove duplication of clang-tidy's check names (Hennadii Stepanov)

Pull request description:

  This PR removes duplication of `clang-tidy`'s check names.

  No behavior change.

  Split up from https://github.com/bitcoin/bitcoin/pull/26642 as [requested](https://github.com/bitcoin/bitcoin/pull/26642#issuecomment-1385351923).

ACKs for top commit:
  fanquake:
    ACK 06fc29326b

Tree-SHA512: a21bef3d7d7201e14565b526af2eae7a90cf0f792803704a80a70a4c78f07ef2a2eef6a8dced80361efbf13291ecccb0977378b9532fc30970a2070426e4d82c
2023-01-17 16:46:24 +01:00
Hennadii Stepanov
06fc29326b refactor: Remove duplication of clang-tidy's check names 2023-01-17 15:40:29 +00:00
fanquake
ccd3d8d2c0 Merge bitcoin/bitcoin#25093: doc: Fix incorrect sendmany RPC doc
fa95f2033a doc: Fix incorrect sendmany RPC doc (MarcoFalke)
fa96f93f05 test: Add test for missing and omitted required arg (MarcoFalke)

Pull request description:

  This enables the skipped type check for `sendmany` and fixes the resulting error.

  Also, there is an unrelated test-only commit.

ACKs for top commit:
  fanquake:
    ACK fa95f2033a

Tree-SHA512: 6f9992786472d3927485a34e918db76824cfb60fa96f42cc9c3cdba7074fe08c657bd77cb3e748432161a290f2dcf90bb0ece279904bd274c529119e65fa0959
2023-01-17 15:19:47 +00:00
MarcoFalke
fad56f7dd6 doc: Properly report optional RPC args 2023-01-17 13:14:15 +01:00
MarcoFalke
fa09cb6086 refactor: Introduce is_top_level_arg 2023-01-17 13:13:46 +01:00
MarcoFalke
fab92a5a5a refactor: Remove const to fix performance-move-const-arg clang-tidy errors
The warnings look like:

src/rpc/util.h:192:19: error: std::move of the const variable 'name' has no effect; remove std::move() or make the variable non-const [performance-move-const-arg,-warnings-as-errors]
        : m_names{std::move(name)},
                  ^~~~~~~~~~    ~
2023-01-17 13:13:01 +01:00
MarcoFalke
fa95f2033a doc: Fix incorrect sendmany RPC doc
This enables the type check and fixes the wrong docs.

Otherwise the enabled check would lead to test errors, such as:

> "wallet_labels.py", line 96, in run_test
>     node.sendmany(
>
> test_framework.authproxy.JSONRPCException:
>  JSON value of type null is not of expected type string (-3)
2023-01-17 12:59:20 +01:00
MarcoFalke
fa96f93f05 test: Add test for missing and omitted required arg 2023-01-17 12:31:59 +01:00
Seibart Nedor
4c8ecccdcd test: add tests for outputs argument to bumpfee/psbtbumpfee 2023-01-17 13:28:53 +02:00
Seibart Nedor
c0ebb98382 wallet: add outputs arguments to bumpfee and psbtbumpfee 2023-01-17 13:28:53 +02:00
Seibart Nedor
a804f3cfc0 wallet: extract and reuse RPC argument format definition for outputs 2023-01-17 13:28:53 +02:00
fanquake
87a08cba43 build: move rpc/request from util lib to common
This is JSON RPC request code that doesn't need to be in util, and
should not be required by the kernel.
2023-01-17 11:02:08 +00:00
MarcoFalke
635f1900d0 Merge bitcoin/bitcoin#26884: test: wallet: add coverage for -spendzeroconfchange setting
603d295199 test: wallet: add coverage for `-spendzeroconfchange` setting (Sebastian Falbesoner)
50112034bc test: remove `-spendzeroconfchange` setting from mempool_limit.py (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `-spendzeroconfchange` setting (in particular the non-default case `=0`). Note that in contrast to the name, the setting does not only apply to change outputs, but in fact to _all_ unconfirmed outputs that we sent to ourselves, i.e. we can trigger the testing path simply with a single recipient address. The first commit removes the setting from the functional test mempool_limit.py, where it doesn't have any effect, since the test was changed to use MiniWallet in commit dddca3899c.

ACKs for top commit:
  brunoerg:
    crACK 603d295199

Tree-SHA512: 15d9c8bd5eb37c6b228bf887eb27debee0a391c82356662785da4553ee2558e611834c3936ef7136812b46f877bab7aa5f3088bbd278b81f296bdda96cc8e1c3
2023-01-17 11:13:50 +01:00
Hennadii Stepanov
b7f6a89a3e Merge bitcoin-core/gui#686: clang-tidy: Force checks for headers in src/qt
7b7cd11244 clang-tidy, qt: Force checks for headers in `src/qt` (Hennadii Stepanov)
69eacf2c5e clang-tidy, qt: Fix `modernize-use-default-member-init` in headers (Hennadii Stepanov)

Pull request description:

  This PR split from bitcoin/bitcoin#26705 and contains only changes in `src/qt`.

  Effectively, it fixes the clang-tidy's `modernize-use-default-member-init` errors, and forces clang-tidy checks for all headers in the `src/qt` directory.

ACKs for top commit:
  jarolrod:
    ACK 7b7cd11244

Tree-SHA512: 79525bb0f31ae7cad88c781e55091a21467c0485ddc1ed03ad62e051480fda3b3710619ea11af480437edba3c6e038f7c40edc6b373e3a37408c006d11b34686
2023-01-17 09:54:56 +00:00
fanquake
10a5f19037 Merge bitcoin/bitcoin#26833: build: allow NO_BOOST=1 & NO_LIBEVENT=1 in depends
7fdeb80441 build: allow NO_LIBEVENT=1 in depends (fanquake)
0cee156eee build: allow NO_BOOST=1 in depends (fanquake)

Pull request description:

  Prerequisite for removing `install_db4.sh`. So we can invoke `make -C depends/ NO_BOOST=1 NO_LIBEVENT=1 NO_QT=1 NO_SQLITE=1 NO_NATPMP=1 NO_ZMQ=1 NO_UPNP=1 NO_USDT=1` and get a prefix with only bdb headers/libs.

ACKs for top commit:
  hebasto:
    ACK 7fdeb80441, tested on Ubuntu 22.04:

Tree-SHA512: 9dd595a61b3178f0296b98d780be2ac37a0d498a75ed222fb1408b84c3e6d179ea18fd96025ea6ce250216be6843dfa46621021e8bc4601a4d1bb8b20387b9e4
2023-01-17 09:40:50 +00:00
fanquake
7799f53542 Merge bitcoin/bitcoin#26039: refactor: Run type check against RPCArgs (1/2)
fa9f6d7bcd rpc: Run type check against RPCArgs (MarcoFalke)
faf96721a6 test: Fix wrong types passed to RPCs (MarcoFalke)

Pull request description:

  It seems brittle to require `RPCTypeCheck` being called inside the code logic. Without compile-time enforcement this will lead to places where it is forgotten and thus to inconsistencies and bugs. Fix this by removing the calls to `RPCTypeCheck` and doing the check internally.

  The changes should be reviewed as refactoring changes. However, if they change behavior, it will be a bugfix. For example the changes here happen to also detect/fix bugs like the one fixed in commit 3b5fb6e77a.

ACKs for top commit:
  ajtowns:
    ACK fa9f6d7bcd

Tree-SHA512: fb2c0981fe6e24da3ca7dbc06898730779ea4e02ea485458505a281cf421015e44dad0221a04023fc547ea2c660d94657909843fc85d92b847611ec097532439
2023-01-17 09:39:26 +00:00
MarcoFalke
fa6b402114 test: Run mempool_packages.py with MiniWallet 2023-01-17 10:33:45 +01:00
MarcoFalke
fa448c27d2 test: Return fee from MiniWallet 2023-01-17 10:33:22 +01:00
MarcoFalke
faec09f240 test: Return chain of MiniWallet txs from MiniWallet chain method 2023-01-17 10:32:36 +01:00
MarcoFalke
faa12d4ccd test: Refactor MiniWallet sign_tx
To make the code less verbose and easier to read.
2023-01-17 10:32:23 +01:00
MarcoFalke
fa2d82103f test: Return wtxid from create_self_transfer_multi
This is not used right now, but may be in the future. Also, it
simplifies the create_self_transfer return logic
2023-01-17 10:30:00 +01:00
MarcoFalke
8339f3cea8 Merge bitcoin/bitcoin#26886: test: add rescan utxos inside MiniWallet's initialization
6bd098a838 test: simplify tests by using the pre-mined chain (kouloumos)
42029a7fd4 test: remove redundant blocks generation logic (kouloumos)
0377d6bb42 test: add `rescan_utxos` in MiniWallet's initialization (kouloumos)

Pull request description:

  When a pre-mined blockchain is used (default behavior), it [contains coinbase outputs in blocks 76-10](07c54de550/test/functional/test_framework/test_framework.py (L809-L813)) to [the MiniWallet's default address](07c54de550/test/functional/test_framework/wallet.py (L99-L101)). That's why we always* `rescan_utxos()` after initializing the MiniWallet, in order for the MiniWallet to account for those mature UTXOs.

  > The tests following this usage pattern can be seen with:
  > ```git grep -n "MiniWallet(" $(git grep -le "rescan_utxos()" $(git grep -Le "self.setup_clean_chain = True"))```

  **This PR adds `rescan_utxos()` inside MiniWallet's initialization to simplify usage when the MiniWallet is used with a pre-mined chain.**

  ### secondary changes

  - *There are a few tests that use the pre-mined blockchain but do not `rescan_utxos()`, they instead generate new blocks to create mature UTXOs.

    > Those were written before the `rescan_utxos()` method was introduced with https://github.com/bitcoin/bitcoin/pull/22955 (fac66d0a39) and can be seen with:
    > `git grep -n "MiniWallet(" $(git grep -Le "rescan_utxos()" $(git grep -Le "self.setup_clean_chain = True"))`
    >

    After including `rescan_utxos()` inside MiniWallets initilization, this blocks generation logic is not needed as the MiniWallet already accounts for enough mature UTXOs to perform the tests. **Therefore the now redundant blocks generation logic is removed from those tests with the second commit.**

  - The rest of the MiniWallet tests use a clean chain (`self.setup_clean_chain = True`)  and can be seen with
    `git grep -n "MiniWallet(" $(git grep -le "self.setup_clean_chain = True")`

    From those, there are a few that start from a clean chain and then create enough mature UTXOs for the MiniWallet with this kind of logic:
   07c54de550/test/functional/mempool_expiry.py (L36-L40)

    **Those tests are simplified in the third commit to instead utilize the mature UTXOs of the pre-mined chain.**

ACKs for top commit:
  MarcoFalke:
    ACK 6bd098a838 🕷
  theStack:
    re-ACK 6bd098a838

Tree-SHA512: 7f9361e36910e4000c33a32efdde4449f4a8a763bb42df96de826fcde469f9362f701b8c99e2a2c482d2d5a42a83ae5ae3844fdbed187ed0ff231f386c222493
2023-01-17 09:38:32 +01:00
Andrew Chow
04e54fd21f Merge bitcoin/bitcoin#26325: rpc: Return accurate results for scanblocks
5ca7a7be76 rpc: Return accurate results for scanblocks (Aurèle Oulès)

Pull request description:

  Implements #26322.
  Adds a `filter_false_positives` mode to `scanblocks` to accurately verify results from blockfilters.

  If the option is enabled, pre-results given by blockfilters will be filtered out again by checking vouts and vins of all transactions of the relevant blocks against the given descriptors.

  ### Master
  ```bash
  ./src/bitcoin-cli -testnet -named scanblocks action=start scanobjects='["addr(tb1qcxf2gv93c26s6mqz7y6etpqdf70zmn67dualgr)"]'
  {
    "from_height": 0,
    "to_height": 2376055,
    "relevant_blocks": [
      "000000000001bc35077dec4104e0ab1f667ae27059bd907f9a8fac55c802ae36",
      "00000000000120a9c50542d73248fb7c37640c252850f0cf273134ad9febaf61",
      "0000000000000082f7af3835da8b6146b0bfb243b8842f09c495fa1e74d454ed",
      "0000000000000094c32651728193bfbe91f6789683b8d6ac6ae2d22ebd3cb5d3"
    ]
  }
  ```

  ### PR (without `filter_false_positives` mode)
  Same as master
  ```bash
  ./src/bitcoin-cli -testnet -named scanblocks action=start scanobjects='["addr(tb1qcxf2gv93c26s6mqz7y6etpqdf70zmn67dualgr)"]' filter_false_positives=false
  {
    "from_height": 0,
    "to_height": 2376055,
    "relevant_blocks": [
      "000000000001bc35077dec4104e0ab1f667ae27059bd907f9a8fac55c802ae36",
      "00000000000120a9c50542d73248fb7c37640c252850f0cf273134ad9febaf61",
      "0000000000000082f7af3835da8b6146b0bfb243b8842f09c495fa1e74d454ed",
      "0000000000000094c32651728193bfbe91f6789683b8d6ac6ae2d22ebd3cb5d3"
    ]
  }
  ```

  ### PR (with `filter_false_positives` mode)
  ```bash
  ./src/bitcoin-cli -testnet -named scanblocks action=start scanobjects='["addr(tb1qcxf2gv93c26s6mqz7y6etpqdf70zmn67dualgr)"]' filter_false_positives=true
  {
    "from_height": 0,
    "to_height": 2376058,
    "relevant_blocks": [
      "0000000000000082f7af3835da8b6146b0bfb243b8842f09c495fa1e74d454ed",
      "0000000000000094c32651728193bfbe91f6789683b8d6ac6ae2d22ebd3cb5d3"
    ]
  }
  ```

  Also adds a test to check that the blockhash of a transaction will be included in the `relevant_blocks` whether the `filter_false_positives` mode is enabled or not.

ACKs for top commit:
  achow101:
    ACK 5ca7a7be76
  theStack:
    re-ACK 5ca7a7be76
  furszy:
    Code review ACK 5ca7a7be

Tree-SHA512: e8f3cceddddd66f59509717b6314d89e2fef241e13cee81b18fd95e8362cbb95cc40f884342ce6cf892a86febd9e2d434afce05d51892240e67f72ae991852e7
2023-01-16 17:39:51 -05:00
Andrew Chow
b55b11f92a Merge bitcoin/bitcoin#25375: rpc: add minconf/maxconf options to sendall and fund transaction calls
cfe5aebc79 rpc: add minconf and maxconf options to sendall (ishaanam)
a07a413466 Wallet/RPC: Allow specifying min & max chain depth for inputs used by fund calls (Juan Pablo Civile)

Pull request description:

  This PR adds a "minconf" option to `fundrawtransaction`, `walletcreatefundedpsbt`,  and `sendall`.
  Alternative implementation of #14641
  Fixes #14542

  Edit: This PR now also adds this option to `send`

ACKs for top commit:
  achow101:
    ACK cfe5aebc79
  Xekyo:
    ACK cfe5aebc79
  furszy:
    diff ACK cfe5aebc, only a non-blocking nit.

Tree-SHA512: 836e610926eec3a62308fba88ddbd6a13d8f4dac37352d0309599f893cde9c1df5e9c298fda6e076493068e4d213e4afa7290a9e3bdb5a95a5d507da3f7b59e8
2023-01-16 17:23:51 -05:00
stickies-v
6a5e88e5cf miner: don't re-apply default Options value if argument is unset
ApplyArgsManOptions does not need to set default values for missing
arguments, these are already defined in the BlockAssembler::Options.

This commit changes the interface of ApplyArgsManOptions(). If
ApplyArgsManOptions() is called again after a option is changed,
this option will no longer be reset to the default value.

There is no observed behaviour change due to how
ApplyArgsManOptions() is currently used, and the new interface is
consistent with e.g. ValidationCacheSizes and MemPoolLimits.
2023-01-16 18:58:11 +00:00
stickies-v
ea72c3d9d5 refactor: avoid duplicating BlockAssembler::Options members
Add Options as a member to BlockAssembler to avoid having to assign
all the options individually.

Additionally brings the struct more in line with how we typically
define default and ArgManager values, as e.g. with
ChainstateManager::Options and and CTxMemPool::Options
2023-01-16 18:58:10 +00:00
kouloumos
6bd098a838 test: simplify tests by using the pre-mined chain 2023-01-16 19:14:21 +02:00
kouloumos
42029a7fd4 test: remove redundant blocks generation logic
those tests already have enough mature utxos from the pre-mined chain.
2023-01-16 19:14:11 +02:00
kouloumos
0377d6bb42 test: add rescan_utxos in MiniWallet's initialization
this simplifies usage when MiniWallet is used with a pre-mined chain.
2023-01-16 19:01:09 +02:00
MarcoFalke
faf7b4f1fc Add BlockManager::IsPruneMode() 2023-01-16 17:31:32 +01:00
MarcoFalke
fae71fe27e Add BlockManager::GetPruneTarget() 2023-01-16 17:16:30 +01:00
MarcoFalke
fa0f0436d8 Add BlockManager::LoadingBlocks() 2023-01-16 16:38:11 +01:00
MarcoFalke
599e941c19 Merge bitcoin/bitcoin#26657: test: Run feature_bip68_sequence.py with MiniWallet
4159ccd031 test: Run feature_bip68_sequence.py with MiniWallet (Miles Liu)
fc0caaf4aa test: Add "include mempool" flag to MiniWallet rescan_utxos (Miles Liu)
d0a909ae54 test: Add "include immature coinbase" flag to MiniWallet get_utxos (Miles Liu)
e5b9127d9e test: Add signs P2TR and RAWSCRIPT to MiniWallet sign_tx (Miles Liu)

Pull request description:

  This PR enables one more of the non-wallet functional tests (feature_bip68_sequence.py) to be run even when no wallet is compiled in by using the MiniWallet instead, as proposed in #20078.

ACKs for top commit:
  achow101:
    ACK 4159ccd031
  MarcoFalke:
    review ACK 4159ccd031 🤸

Tree-SHA512: e891b189381e961c840b45fc30d058363707fd54c1c4bdc3d29623b03309981f1d3ebfac27e6aecf621bdbcd7e31754a3ef7c53f86346f7dd241c137e64c92bd
2023-01-16 16:26:12 +01:00
Hennadii Stepanov
123043e99c ci: Bump lint task image to Ubuntu Jammy 2023-01-16 13:30:00 +00:00
Hennadii Stepanov
9b86114058 ci: Use pyenv's python-build to install Python in lint task 2023-01-16 13:29:44 +00:00
MarcoFalke
6b7ccb98a5 Merge bitcoin/bitcoin#26251: refactor: add kernel/cs_main.h
282019cd3d refactor: add kernel/cs_main.* (fanquake)

Pull request description:

  One place to find / include `cs_main`.
  No more:
  > // Actually declared in validation.cpp; can't include because of circular dependency.
  > extern RecursiveMutex cs_main;

  Ultimately, no more need to include `validation.h` (which also includes (heavy/boost filled) `txmempool.h`) everywhere for `cs_main`. See #26087 for another example of why that is useful.

ACKs for top commit:
  ajtowns:
    ACK 282019cd3d

Tree-SHA512: 142835b794873e7a09c3246d6101843ae81ec0c6295e6873130c98a2abfa5f7282748d0f1a37237a779cc71c3bc0a75d03b20313ef5398c83d4814215cbc8287
2023-01-16 13:44:56 +01:00
MarcoFalke
2182149dc5 Merge bitcoin/bitcoin#26631: test: add coverage for dust mempool policy (-dustrelayfee setting)
d6fc1d6a33 test: add coverage for dust mempool policy (`-dustrelayfee` setting) (Sebastian Falbesoner)
8a5dbe2879 test: add `CScript` method for checking for witness program (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `-dustrelayfee` setting, which specifies the fee-rate used to define dust. Output scripts for all common types that are treated as standard by default (P2PK, P2(W)PKH, P2(W)SH, P2TR, bare multisig, null data, unknown witness versions v2+) are created and then checked for dust-mempool-policy each via the `testmempoolaccept` RPC: a tx with an output's nValue equal to the dust threshold should be accepted, one with an nValue of just one 1 satoshi below that should be rejected with reason `dust`. This is repeatedly done for a fixed (but obviously somewhat arbitrary) list of different `-dustrelayfee` settings on a single node, including the default and zero (i.e. no dust limit) settings.

  Note that the first commit introduces a necessary `CScript` helper method `IsWitnessProgram` (using PascalCase in Python is likely controversial; in this case the style for the already existing method `GetSigOpCount` was followed, which also refers to a method in the core `CScript` class).

  Some historical information about dust, contributed by pablomartin4btc:
  "The concept of dust was first introduced in https://github.com/bitcoin/bitcoin/pull/2577. This [commit](eb30d1a5b2) from https://github.com/bitcoin/bitcoin/pull/9380 introduced the -dustrelayfee option. Previous to that PR, the dust feerate was whatever -minrelaytxfee was set to."

ACKs for top commit:
  LarryRuane:
    ACK d6fc1d6a33
  glozow:
    ACK d6fc1d6a33
  kouloumos:
    ACK d6fc1d6a33

Tree-SHA512: 35ea2b2497dfb466395af5665bb217f7250aa7cab9dc43539a5658ab69a454e3623ff58fce7489fcc1105b37f8cb4840a93cec658c5df1de611732bc6439ccad
2023-01-16 11:36:13 +01:00
MarcoFalke
08d2a3ab4b Merge bitcoin/bitcoin#26891: ci: Update "Win64 native" task
46c31eea8c ci: Bump `ccache` version to the latest 4.7.4 in "Win64 native" task (Hennadii Stepanov)
1039ed44fb ci: Bump vcpkg to the latest version 2023.01.09 (Hennadii Stepanov)

Pull request description:

  This PR bumps `vcpkg` and `ccache` versions.

  Dependency changes in `vcpkg` ([2022.09.27](https://github.com/microsoft/vcpkg/releases/tag/2022.09.27) - [2023.01.09](https://github.com/microsoft/vcpkg/releases/tag/2023.01.09)):
   - boost 1.80.0#0 -> 1.81.0#0
   - libevent 2.1.12#6 -> libevent 2.1.12#7
   - sqlite3 3.39.2#0 -> 3.40.0#1

  Also see https://github.com/bitcoin/bitcoin/pull/26866#issuecomment-1378591258.

Top commit has no ACKs.

Tree-SHA512: 9f8fda9192f76526201098c0300261c37deb8f17be19f4bfec52c5d8c78f348a11df76dc6f2d23f556cfb30e7ae5ff750ef824f2b3e133d892fe24642b802033
2023-01-16 11:13:38 +01:00
MarcoFalke
ac4c79a267 Merge bitcoin/bitcoin#26892: test: refactor: simplify p2p_permissions.py by using MiniWallet
8cbd926a2c test: refactor: simplify p2p_permissions.py by using MiniWallet (Sebastian Falbesoner)

Pull request description:

  This PR simplies the functional test p2p_permissions.py by using MiniWallet in order to avoid manual low-level tx creation. Also, rather than mining 100 blocks manually, the pre-mined chain of the test framework is used, which speeds up the test a little (~2-3 seconds faster on my machine).

ACKs for top commit:
  MarcoFalke:
    ACK 8cbd926a2c

Tree-SHA512: 36cbc2a0f6fb0251c8696cd017163ed30529690736bafd36e80b53007bd02b9030b68fe93b90dc50323526c8b7d8e0abd8f20890d46ff6015d5c632b64a08535
2023-01-16 10:37:14 +01:00
MarcoFalke
53ae1022ea Merge bitcoin/bitcoin#26894: test: Remove redundant key_to_p2pkh call
61360e0cf9 test: Remove redundant function call (Kolby ML)

Pull request description:

  Removed unnecessary function call and assignment `get_generate_key()` already calls `key_to_p2pkh()` and stores it in the object as p2pkh_addr.

  key.p2pkh_addr is already used for most testcases as well, so it is just a redundant call

ACKs for top commit:
  MarcoFalke:
    ACK 61360e0cf9

Tree-SHA512: d39117310d6630bad7a78c051e683dbfc77b27f6182014557fb900c1adaf5f1e12039c5a6c0ea50e908d4a8688da05f2deae701f12e06086d17dde86ae275ec5
2023-01-16 10:23:50 +01:00
Kolby ML
61360e0cf9 test: Remove redundant function call 2023-01-15 18:34:01 -07:00
Hennadii Stepanov
3dd2762cf8 Merge bitcoin-core/gui#690: Catch invalid networks combination crash
f4a11d7baf gui: bugfix, catch invalid networks combination crash (furszy)

Pull request description:

  The app currently crashes if a network is set inside bitcoin.conf and
  another one is provided as param.
  The reason is an uncaught runtime_error.

ACKs for top commit:
  jarolrod:
    tACK f4a11d7baf
  johnny9:
    tACK f4a11d7baf
  john-moffett:
    ACK f4a11d7baf
  pablomartin4btc:
    Tested ACK f4a11d7baf.
  hebasto:
    ACK f4a11d7baf, tested on Ubuntu 22.04 (Qt 5.15.3).

Tree-SHA512: fc5e26ae0a361e37d53d904cc122d07f064f261b309629c6386cb046ab1b3d2c805cbfe0db8ed3e934af52c6cf0ebb0bef9df9117b4330d9b0ea40c76f9270f9
2023-01-15 18:55:59 +00:00
Sebastian Falbesoner
8cbd926a2c test: refactor: simplify p2p_permissions.py by using MiniWallet
Also, use the pre-mined chain of the test framework rather than mining
100 blocks manually on each run.
2023-01-15 01:11:15 +01:00
fanquake
0c2a1288a2 Merge bitcoin/bitcoin#26835: contrib: add PE Canary check to security-check
6ba17d4955 scripts: add PE Canary check to security-check (fanquake)

Pull request description:

  We should be checking this, same as ELF & MACHO.

  Guix Build:
  ```bash
  6334c001b276ca5f0278092be68bf6d49d9b755bcac893bbd4aa58df57356e40  guix-build-6ba17d4955b7/output/dist-archive/bitcoin-6ba17d4955b7.tar.gz
  e27ad7fffb377bc6264477933859ab47c7283a68fbf86124d3801bc4c8b790dd  guix-build-6ba17d4955b7/output/x86_64-w64-mingw32/SHA256SUMS.part
  ef7b61bd854f0d3c39f356ef85ac18d37c5740874111f5ce46f7ce3381e714ca  guix-build-6ba17d4955b7/output/x86_64-w64-mingw32/bitcoin-6ba17d4955b7-win64-debug.zip
  c419324597487f248143a076d6eb2a56b0dbf5ce690ca89afaaee5c6b352e1a1  guix-build-6ba17d4955b7/output/x86_64-w64-mingw32/bitcoin-6ba17d4955b7-win64-setup-unsigned.exe
  a18ff1e3026cd9fc08dd7b500c06a343462aef4a37538608d940d1845bcdb94a  guix-build-6ba17d4955b7/output/x86_64-w64-mingw32/bitcoin-6ba17d4955b7-win64-unsigned.tar.gz
  7e4ee0669940f4b8c1a12dab836898511a60f06a62057ac03beaca8bb693bfb4  guix-build-6ba17d4955b7/output/x86_64-w64-mingw32/bitcoin-6ba17d4955b7-win64.zip
  ```

ACKs for top commit:
  sipsorcery:
    ACK 6ba17d4955.

Tree-SHA512: 1acc24c0cb36dbc30311f4eee64e3d4737c828b97039be0f72cfe061bcb8c4d5c830d7792f503e711e219a62d85b7e07cdff3510cbd4f8d46895a7cb66b88219
2023-01-14 12:38:01 +00:00
fanquake
7fdeb80441 build: allow NO_LIBEVENT=1 in depends 2023-01-14 12:31:22 +00:00
fanquake
0cee156eee build: allow NO_BOOST=1 in depends 2023-01-14 12:30:42 +00:00
Sebastian Falbesoner
603d295199 test: wallet: add coverage for -spendzeroconfchange setting 2023-01-13 21:56:17 +01:00
Hennadii Stepanov
46c31eea8c ci: Bump ccache version to the latest 4.7.4 in "Win64 native" task 2023-01-13 20:50:38 +00:00
Hennadii Stepanov
1039ed44fb ci: Bump vcpkg to the latest version 2023.01.09
Dependency changes (2022.09.27 - 2023.01.09):
 - boost 1.80.0#0 -> 1.81.0#0
 - libevent 2.1.12#6 -> libevent 2.1.12#7
 - sqlite3 3.39.2#0 -> 3.40.0#1
2023-01-13 20:50:15 +00:00
fanquake
8915e4d9f5 Merge bitcoin/bitcoin#26824: build: fix configuring with only bitcoin-util
0f883df7a5 build: fix configuring with only bitcoin-util (fanquake)

Pull request description:

  Fixes the issue presented in #25037 in a single (easily backportable) diff, with no additional refactoring/changes.

  Can be tested with:
  ```bash
  ./configure \
    --disable-tests \
    --disable-bench \
    --without-libs \
    --without-daemon \
    --without-gui \
    --disable-fuzz-binary \
    --without-utils \
    --enable-util-util
  ```

ACKs for top commit:
  TheCharlatan:
    tACK 0f883df7a5
  hebasto:
    ACK 0f883df7a5, tested on Ubuntu 22.04.

Tree-SHA512: 3682712405c360852c4edd90c171e21302154bf8789252c64083974a5c873cf04d97e8721c7916d5b2dafa6acd2b8dc32deecf550e90e03bcbbabbbbf75ce959
2023-01-13 15:56:16 +00:00
brunoerg
0c520679ab doc: add release notes for abandoned field in gettransaction and listtransactions 2023-01-13 10:29:05 -03:00
brunoerg
a1aaa7f51f rpc, wallet: add abandoned field for all categories of transactions in ListTransactions 2023-01-13 10:29:00 -03:00
MarcoFalke
32834034a3 Merge bitcoin/bitcoin#26882: test: skip sqlite3 tests if it isn't available
eea73d465e test: skip sqlite3 tests if it isn't available (fanquake)

Pull request description:

  Fixes #26819. Related too #26873, which adds the missing documentation.

ACKs for top commit:
  brunoerg:
    ACK eea73d465e

Tree-SHA512: 9bcf5c642695c39a58d56ef8bf4d62601ab0acd4ad71c9400b9d8a2a336a1cd55b69408a02c583d9a67c5de0286364c9f38debd03ec8316725ad69450d980f0d
2023-01-13 13:52:08 +01:00
fanquake
db648e8ccc build: remove unneeded var exporting 2023-01-13 10:42:15 +00:00
fanquake
14afc71a61 build: remove Boost lib detection from ax_boost_base
We don't use / link against Boost libraries, so there is no reason for
us to try and detect them / muddle with LDFLAGS.
2023-01-13 10:41:33 +00:00
fanquake
07c54de550 Merge bitcoin/bitcoin#26691: Update secp256k1 subtree to libsecp256k1 version 0.2.0
2022917223 Add secp256k1_selftest call (Pieter Wuille)
3bfca788b0 Remove explicit enabling of default modules (Pieter Wuille)
4462cb0498 Adapt to libsecp256k1 API changes (Pieter Wuille)
9d47e7b71b Squashed 'src/secp256k1/' changes from 44c2452fd3..21ffe4b22a (Pieter Wuille)

Pull request description:

  Now that libsecp256k1 has a release (https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-December/021271.html), update the subtree to match it.

  The changes themselves are not very impactful for Bitcoin Core, but include:
  * It's no longer needed to specify whether contexts are for signing or verification or both (all contexts support everything), so make use of that in this PR.
  * Verification operations can use the static context now, removing the need for some infrastructure in pubkey.cpp to make sure a context exists.
  * Most modules are now enabled by default, so we can drop explicit enabling for them.
  * CI improvements (in particular, MSVC and more recent MacOS)
  * Introduction of an internal int128 type, which has no effect for GCC/Clang builds, but enables 128-bit multiplication in MSVC, giving a ~20% speedup there (but still slower than GCC/Clang).
  * Release process changes (process documentation, changelog, ...).

ACKs for top commit:
  Sjors:
    ACK 2022917223, but 4462cb0498 could use more eyes on it.
  achow101:
    ACK 2022917223
  jonasnick:
    utACK 2022917223

Tree-SHA512: 8a9fe28852abe74abd6f96fef16a94d5a427b1d99bff4caab1699014d24698aab9b966a5364a46ed1001c07a7c1d825154ed4e6557c7decce952b77330a8616b
2023-01-13 09:40:57 +00:00
MarcoFalke
dcae3c19b8 Merge bitcoin/bitcoin#26867: doc: Mention restoring wallet via GUI
dc9bad5192 Change dots to an ellipsis and fix capitalization (John Moffett)
9b158ae73f Update to mention restoring wallet via GUI (John Moffett)

Pull request description:

  f9783b0f07 Recently added the ability to restore wallets via the GUI, but the current wallet guide says backups must be restored via RPC.

ACKs for top commit:
  kouloumos:
    ACK dc9bad5192
  jarolrod:
    re-ACK dc9bad5
  hebasto:
    re-ACK dc9bad5192

Tree-SHA512: 325a0023ef10c75073b0288f69c99f01b029b0b7b64ae91e7ef72d4ab1fa4da60fe4cd1b4528c1c0d34617122d9aee3cd9cb32aef05a25493fc01e9ec2e6cc10
2023-01-13 09:42:08 +01:00
Sebastian Falbesoner
50112034bc test: remove -spendzeroconfchange setting from mempool_limit.py
Since this test was changed to use MiniWallet instead of the Bitcoin
Core wallet (see commit d447ded6ba),
the setting doesn't have any effect and hence can be removed.
2023-01-13 03:41:42 +01:00
MarcoFalke
f4ef856375 Merge bitcoin/bitcoin#26850: ci: Stop and remove CI container
fa0584eb82 ci: Stop and remove CI container (MarcoFalke)
fa5dccba32 scripted-diff: ci: Rework docker naming (MarcoFalke)

Pull request description:

  This should not affect CI runs that have `DANGER_RUN_CI_ON_HOST=1` set, for example `.cirrus.yml`.

  However, when running CI locally with podman or docker, the container is stopped and thus deleted when all tests have passed. This feature was requested in https://github.com/bitcoin/bitcoin/pull/26843#issuecomment-1374445512 and can help to reduce used disk space when running several CI tasks subsequently.

ACKs for top commit:
  fanquake:
    ACK fa0584eb82
  hebasto:
    ACK fa0584eb82, tested on Ubuntu 22.04.

Tree-SHA512: 8bc1fabd15dad06d9ab3535d2743556137d512332b377b333ab13dbd7c5911b9295549511c6f3836b3acb7f8a39fb67e5f5249e45e1178d16ad4ed0593d05f04
2023-01-12 20:43:28 +01:00
Greg Sanders
b093f5619f Fill out dust limit unit test for known types except bare multisig 2023-01-12 13:45:35 -05:00
John Moffett
dc9bad5192 Change dots to an ellipsis and fix capitalization
Matches ellipsis usage in the "Restore" section.
2023-01-12 11:15:02 -05:00
John Moffett
9b158ae73f Update to mention restoring wallet via GUI 2023-01-12 11:12:25 -05:00
MarcoFalke
fcd1a57be0 Merge bitcoin/bitcoin#26827: doc: use "std lib clock" over "C++11 clock"
672f7ad747 doc: remove usages of C++11 (fanquake)

Pull request description:

  These were new in C++11, and now they are just our standard library.

ACKs for top commit:
  jarolrod:
    re-ACK 672f7ad747
  hebasto:
    re-ACK 672f7ad747

Tree-SHA512: 7e3b8b0346ba29b19e6d8536700ca510e2b543cdeecd9e740bba71ea6d0133dd96cdaeaa00f371f8ef85913ff5aaabe12878255f393dac7d354a8b89b58d050a
2023-01-12 16:49:58 +01:00
fanquake
672f7ad747 doc: remove usages of C++11
Now it's just the standard library.
2023-01-12 13:42:44 +00:00
fanquake
eea73d465e test: skip sqlite3 tests if it isn't available
Fixes #26819. Related too #26873.
2023-01-12 13:37:15 +00:00
MarcoFalke
edc3d1b296 Merge bitcoin/bitcoin#26854: test: Fix intermittent timeout in p2p_permissions.py
fa1bf4e705 test: Fix intermittent timeout in p2p_permissions.py (MarcoFalke)

Pull request description:

  The sync is based on `bytesrecv_per_msg["verack"]`. However, the bytes are counted before processing the message, so they are not sufficient to ensure the connection is fully up.

ACKs for top commit:
  mzumsande:
    ACK fa1bf4e705
  aureleoules:
    ACK fa1bf4e705

Tree-SHA512: eb1ed537032c76a449b1ed5e42ff062e9b8b3c7e11fde2a5b8183ae0d6fbe31dba39e2c758836160cd8157d9ac5cc1f5d1916415861b8d711b7370c88f5e9790
2023-01-12 12:51:34 +01:00
stickies-v
cba749a9b7 refactor: rename local gArgs to args
Avoid confusion with the global gArgs
2023-01-12 11:16:17 +00:00
MarcoFalke
bd74004532 Merge bitcoin/bitcoin#26877: doc: move errant release note to doc/
c28d461834 doc: move errant release note to doc/ (fanquake)

Pull request description:

  Release note from #26646 should be in doc/.

Top commit has no ACKs.

Tree-SHA512: 741323c0bc526163d65e441c9e677ee3613ed0c55b1880b99fb63b0f3306aeca0cad83dadef05c23def8d0fc6a224956b27ae1bf4b1ff980a1be2d31feef7e3d
2023-01-12 10:56:38 +01:00
fanquake
535adce272 Merge bitcoin/bitcoin#26825: build: remove already tested headers from AC_CHECK_HEADERS
94d0c418c5 build: remove already tested headers from AC_CHECK_HEADERS (fanquake)

Pull request description:

  These headers are already included in a default set which are checked early during configure.

  We already use at least sys/types.h and unistd.h unconditionally in configure.

ACKs for top commit:
  TheCharlatan:
    Code review ACK 94d0c418c5

Tree-SHA512: d81ce01d46ef1c226b19a4b906e5b622baeee62f04d75eebd02094585ad4712a6c830314227d3a1c03a5f76be5c659b456bb065c175e3cf391e0ece37fd99ead
2023-01-12 09:42:09 +00:00
fanquake
c28d461834 doc: move errant release note to doc/ 2023-01-12 09:28:34 +00:00
Andrew Chow
fbe5e1220a Merge bitcoin/bitcoin#26675: wallet: For feebump, ignore abandoned descendant spends
f9ce0eadf4 For feebump, ignore abandoned descendant spends (John Moffett)

Pull request description:

  Closes #26667

  To be eligible for fee-bumping, a transaction must not have any of its outputs (eg - change) spent in other unconfirmed transactions in the wallet. This behavior is currently [enforced](9e229a542f/src/wallet/feebumper.cpp (L25-L28)) and [tested](9e229a542f/test/functional/wallet_bumpfee.py (L270-L286)).

  However, this check shouldn't apply to spends in abandoned descendant transactions, as explained by #26667.

  `CWallet::IsSpent` already carves out an exception for abandoned transactions, so we can just use that.

  I've also added a new test to cover this case.

ACKs for top commit:
  Sjors:
    re-utACK f9ce0eadf4
  achow101:
    ACK f9ce0eadf4
  furszy:
    ACK f9ce0ead

Tree-SHA512: 19d957d1cf6747668bb114e27a305027bfca5a9bed2b1d9cc9e1b0bd4666486c7c4b60b045a7fe677eb9734d746f5de76390781fb1e9e0bceb4a46d20acd1749
2023-01-11 18:24:53 -05:00
Andrew Chow
2f6a8e5e02 Merge bitcoin/bitcoin#26695: bench: BlockAssembler on a mempool with packages
04528054fc [bench] BlockAssembler with mempool packages (glozow)
6ce265acf4 [test util] lock cs_main before pool.cs in PopulateMempool (glozow)
8791410662 [test util] randomize fee in PopulateMempool (glozow)
cba5934eb6 [miner] allow bypassing TestBlockValidity (glozow)
c058852308 [refactor] parameterize BlockAssembler::Options in PrepareBlock (glozow)
a2de971ba1 [refactor] add helper to apply ArgsManager to BlockAssembler::Options (glozow)

Pull request description:

  Performance of block template building matters as miners likely want to be able to start mining on a block with transactions asap after a block is found. We would want to know if a mempool PR accidentally caused, for example, a 100x slowdown. An `AssembleBlock()` bench exists, but it operates on a mempool with 101 transactions, each with 0 ancestors or descendants and with the same fee. Adding a bench with a more complex mempool is useful because (1) it's more realistic (2) updating packages can potentially cause the algorithm to take a long time.

ACKs for top commit:
  kevkevinpal:
    Tested ACK [0452805](04528054fc)
  achow101:
    ACK 04528054fc
  stickies-v:
    ACK 04528054f

Tree-SHA512: 38c138d6a75616651f9b1faf4e3a1cd833437a486f4e84308fbee958e8462bb570582c88f7ba7ab99d80191e97855ac2cf27c43cc21585d3e4b0e227effe2fb5
2023-01-11 18:11:11 -05:00
ishaanam
cfe5aebc79 rpc: add minconf and maxconf options to sendall 2023-01-11 17:08:35 -05:00
Juan Pablo Civile
a07a413466 Wallet/RPC: Allow specifying min & max chain depth for inputs used by fund calls
Enables users to craft BIP-125 replacements with changes to the output
list, ensuring that if additional funds are needed they will be added.
2023-01-11 17:08:23 -05:00
Andrew Chow
908212506d Merge bitcoin/bitcoin#26821: refactor: Make ThreadHTTP return void
45553e11c9 refactor: Make `ThreadHTTP` return void (Hennadii Stepanov)

Pull request description:

  The `bool` return value was introduced in 755aa05174 (https://github.com/bitcoin/bitcoin/pull/8421).

  It has been not used since 8d3f46ec39 (https://github.com/bitcoin/bitcoin/pull/14670).

  No behavior change.

ACKs for top commit:
  achow101:
    ACK 45553e11c9
  brunoerg:
    crACK 45553e11c9
  w0xlt:
    ACK 45553e11c9
  stickies-v:
    ACK 45553e11c

Tree-SHA512: 1593a5740e729967fbe1363235cd5b77ecf431b29bc740a89a6c70fc838ad97a2e4a2cd7cd63aa482f7c50bc2ffabc8cd53e8f64d6032603cb3b662229bc3dc2
2023-01-11 16:46:45 -05:00
fanquake
376e01b382 doc: add databases/py-sqlite3 to FreeBSD test suite deps 2023-01-11 16:55:56 +00:00
MarcoFalke
fa9f6d7bcd rpc: Run type check against RPCArgs 2023-01-11 17:42:09 +01:00
MarcoFalke
faf96721a6 test: Fix wrong types passed to RPCs 2023-01-11 17:41:34 +01:00
fanquake
329d7e379d Merge bitcoin/bitcoin#26328: doc: fix -netinfo relaytxes help
0f5fc4f656 doc: fix up -netinfo relaytxes help (Jon Atack)

Pull request description:

  Addresses https://github.com/bitcoin/bitcoin/pull/26109#discussion_r995502563 by Marco Falke (thanks!)

ACKs for top commit:
  mzumsande:
    Code Review ACK 0f5fc4f656

Tree-SHA512: d7345d1a94b15c4ec1a2bb0be5c04c472411d90cefb4c16ed524933d2bfc36816bb7519c2e109b2e41ff451b039dd2ddaa6d5db917ad54745332f2a1d8b85570
2023-01-11 16:39:11 +00:00
MarcoFalke
9887fc7898 Merge bitcoin/bitcoin#26758: refactor: Add performance-no-automatic-move clang-tidy check
9567bfeab9 clang-tidy: Add `performance-no-automatic-move` check (Hennadii Stepanov)

Pull request description:

  Split from bitcoin/bitcoin#26642 as [requested](https://github.com/bitcoin/bitcoin/pull/26642#discussion_r1054673201).

  For the problem description see https://clang.llvm.org/extra/clang-tidy/checks/performance/no-automatic-move.html.

  The following types are affected:
  - `std::pair<CAddress, NodeSeconds>`
  - `std::vector<CAddress>`
  - `UniValue`, also see bitcoin/bitcoin#25429
  - `QColor`
  - `CBlock`
  - `MempoolAcceptResult`
  - `std::shared_ptr<CWallet>`
  - `std::optional<SelectionResult>`
  - `CTransactionRef`, which is `std::shared_ptr<const CTransaction>`

ACKs for top commit:
  andrewtoth:
    ACK 9567bfeab9
  aureleoules:
    ACK 9567bfeab9

Tree-SHA512: 9b6a5d539205b41d2c86402d384318ed2e1d89e66333ebd200a48fd7df3ce6f6c60a3e989eda5cc503fb34b8d82526f95e56776e1af51e63b49e3a1fef72dbcb
2023-01-11 16:18:34 +01:00
MarcoFalke
fa1bf4e705 test: Fix intermittent timeout in p2p_permissions.py 2023-01-11 15:20:17 +01:00
MarcoFalke
b168b71a5d Merge bitcoin/bitcoin#26730: test: add coverage for purpose arg in listlabels
c467cfffce test: add coverage for `purpose` arg in `listlabels` (brunoerg)

Pull request description:

  This PR adds test coverage for `listlabels` command when specifying the `purpose` (send and receive).

  dcdfd72861/src/wallet/rpc/addresses.cpp (L698-L704)

ACKs for top commit:
  kristapsk:
    ACK c467cfffce

Tree-SHA512: 7e7143c1264692f7b22952e7c70dbe9ed3f5dcd2e3b69962a47be9f9c21b3f4a9089ca87962fbc8ff9116e7d2dbeb7f36d6a132c9ac13724a255cfe1b32373a8
2023-01-11 14:55:37 +01:00
glozow
26002570ab Merge bitcoin/bitcoin#26646: validation, bugfix: provide more info in *MempoolAcceptResult
264f9ef17f [validation] return MempoolAcceptResult for every tx on PCKG_TX failure (glozow)
dae81e01e8 [refactor] rename variables in AcceptPackage for clarity (glozow)
da484bc738 [doc] release note effective-feerate and effective-includes RPC results (glozow)
5eab397b98 [validation] remove PackageMempoolAcceptResult::m_package_feerate (glozow)
601bac88cb [rpc] return effective-includes in testmempoolaccept and submitpackage (glozow)
1691eaa818 [rpc] return effective-feerate in testmempoolaccept and submitpackage (glozow)
d6c7b78ef2 [validation] return wtxids of other transactions whose fees were used (glozow)
1605886380 [validation] return effective feerate from mempool validation (glozow)
5d35b4a7de [test] package validation quits early due to non-policy, non-missing-inputs failure (glozow)
be2e4d94e5 [validation] when quitting early in AcceptPackage, set package_state and tx result (glozow)

Pull request description:

  This PR fixes a bug and improves the mempool accept interface to return information more predictably.

  Bug: In package validation, we first try the transactions individually (see doc/policy/packages.md for more explanation) and, if they all failed for missing inputs and policy-related (i.e. fee) reasons, we'll try package validation. Otherwise, we'll just "quit early" since, for example, if a transaction had an invalid signature, adding a child will not help make it valid. Currently, when we quit early, we're not setting the `package_state` to be invalid, so the caller might think it succeeded. Also, we're returning no results - it makes more sense to return the individual transaction failure. Thanks instagibbs for catching https://github.com/bitcoin/bitcoin/pull/25038#discussion_r1013293248!

  Also, make the package results interface generally more useful/predictable:
  - Always return the feerate at which a transaction was considered for `CheckFeeRate` in `MempoolAcceptResult::m_effective_feerate` when it was successful. This can replace the current `PackageMempoolAcceptResult::m_package_feerate`, which only sometimes exists.
  - Always provide an entry for every transaction in `PackageMempoolAcceptResult::m_tx_results` when the error is `PCKG_TX`.

ACKs for top commit:
  instagibbs:
    reACK 264f9ef17f
  achow101:
    ACK 264f9ef17f
  naumenkogs:
    reACK 264f9ef17f

Tree-SHA512: ce7fd9927a80030317cc6157822596e85a540feff5dbf5eea7c62da2eb50c917cdddc9da1e2ff62cc18b98b27d360151811546bd9d498859679a04bbee090837
2023-01-11 13:25:39 +00:00
Vasil Dimov
3c1de032de i2p: use consistent number of tunnels with i2pd and Java I2P
The default number of tunnels in the Java implementation is 2 and in the
C++ i2pd it is 5. Pick a mid-number (3) and explicitly set it in order
to get a consistent behavior with both routers. Do this for persistent
sessions which are created once at startup and can be used to open up
to ~10 outbound connections and can accept up to ~125 incoming
connections. Transient sessions already set number of tunnels to 1.

Suggested in:
https://github.com/bitcoin/bitcoin/issues/26754#issuecomment-1367356129
https://geti2p.net/en/docs/api/samv3

Alleviates: https://github.com/bitcoin/bitcoin/issues/26754
2023-01-11 13:56:14 +01:00
Vasil Dimov
801b405f85 i2p: lower the number of tunnels for transient sessions
This will lower the load on the I2P network. Since we use one transient
session for connecting to just one peer, a higher number of tunnels is
unnecessary.

This was suggested in:
https://github.com/bitcoin/bitcoin/issues/26754#issuecomment-1365449401
https://github.com/bitcoin/bitcoin/issues/26754#issuecomment-1367356129

The options are documented in:
https://geti2p.net/en/docs/protocol/i2cp#options

A tunnel is unidirectional, so even if we make a single outbound
connection we still need an inbound tunnel to receive the messages sent
to us over that connection.

Alleviates: https://github.com/bitcoin/bitcoin/issues/26754
2023-01-11 13:56:13 +01:00
Vasil Dimov
b906b64eb7 i2p: reuse created I2P sessions if not used
In the case of `i2pacceptincoming=0` we use transient addresses
(destinations) for ourselves for each outbound connection. It may
happen that we
* create the session (and thus our address/destination too)
* fail to connect to the particular peer (e.g. if they are offline)
* dispose the unused session.

This puts unnecessary load on the I2P network because session creation
is not cheap. Is exaggerated if `onlynet=i2p` is used in which case we
will be trying to connect to I2P peers more often.

To help with this, save the created but unused sessions and pick them
later instead of creating new ones.

Alleviates: https://github.com/bitcoin/bitcoin/issues/26754
2023-01-11 13:56:12 +01:00
MarcoFalke
dbca00ef76 Merge bitcoin/bitcoin#26838: doc: I2P documentation updates
3e1d2941e9 doc: remove recommended I2P router versions (jonatack)
295849abb5 doc: update/clarify/de-emphasize I2P transient address section (jonatack)
dffa319457 doc: update bandwidth section of I2P documentation (jonatack)
0ed9cc5892 doc: clarify -i2pacceptincoming help documentation (jonatack)

Pull request description:

  Address the documentation updates requested in issue #26754, clarify/simplify the -i2pacceptincoming help, and a few other fixups.

ACKs for top commit:
  willcl-ark:
    ACK 3e1d294
  1440000bytes:
    ACK 3e1d2941e9
  w0xlt:
    ACK 3e1d2941e9
  vasild:
    ACK 3e1d2941e9

Tree-SHA512: e647221884af34646b99150617f4d4cc8d5fce325a769294f49047b9d8c9c8ab2b365cfdd9f56b3bd0303da706233f03d24cececf6e161c53f04ed947751052a
2023-01-11 13:03:54 +01:00
MarcoFalke
fa0584eb82 ci: Stop and remove CI container 2023-01-11 10:49:39 +01:00
MarcoFalke
fa5dccba32 scripted-diff: ci: Rework docker naming
DOCKER in names is confusingly used as synonym for "image", "container",
and "ci". Fix the confusion by picking the term that fits the context.

-BEGIN VERIFY SCRIPT-
 ren() { sed -i "s:$1:$2:g" $( git grep -l "$1" ) ; }

 ren DOCKER_PACKAGES CI_BASE_PACKAGES
 # This better reflects that they are the common base for all CI
 # containers.

 ren DOCKER_ID CI_CONTAINER_ID
 # This is according to the documentation of "--detach , -d: Run
 # container in background and print container ID".

 ren DOCKER_NAME_TAG CI_IMAGE_NAME_TAG
 # This avoids confusing with CONTAINER_NAME and clarifies that it is an
 # image.

 ren DOCKER_ADMIN CI_CONTAINER_CAP
 # This clarifies that it is a capability added to the container.

 ren DOCKER_CI_CMD_PREFIX CI_EXEC_CMD_PREFIX
 # This brings it in line with the CI_EXEC naming.

-END VERIFY SCRIPT-
2023-01-11 10:49:18 +01:00
Andrew Chow
4586ae2da1 Merge bitcoin/bitcoin#26679: wallet: Skip rescanning if wallet is more recent than tip
3784009534 wallet: Skip rescanning if wallet is more recent than tip (Andrew Chow)

Pull request description:

  If a wallet has key birthdates that are more recent than the currrent chain tip, or a bestblock height higher than the current tip, we should not attempt to rescan as there is nothing to scan for.

  Fixes #26655

ACKs for top commit:
  ishaanam:
    re-utACK 3784009534
  w0xlt:
    utACK 3784009534
  furszy:
    Code review ACK 37840095

Tree-SHA512: f0d90b62940d97d50f21e1e01fa6dcb54409fad819cea4283612825c4d93d733df323cd92787fed43956b0a8e386a5bf88218f1f5749c913398667a5c8f54470
2023-01-10 19:56:32 -05:00
Andrew Chow
b0fa5989e1 test: Check that orphaned coinbase unconf spend is still abandoned
When an orphaned coinbase is reorged back into the main chain, any
unconfirmed ancestors should still be marked as abandoned due to the
original reorg that orphaned that coinbase.
2023-01-10 18:25:20 -05:00
Andrew Chow
9addbd7890 wallet: Automatically abandon orphaned coinbases and their children 2023-01-10 18:23:45 -05:00
Andrew Chow
68f88bc03f Merge bitcoin/bitcoin#26186: rpc: Sanitize label name in various RPCs with tests
65e78bda7c test: Invalid label name coverage (Aurèle Oulès)
552b51e682 refactor: Add sanity checks in LabelFromValue (Aurèle Oulès)
67e7ba8e1a rpc: Sanitize label name in various RPCs (Aurèle Oulès)

Pull request description:

  The following RPCs did not sanitize the optional label name:
  - importprivkey
  - importaddress
  - importpubkey
  - importmulti
  - importdescriptors
  - listsinceblock

  Thus is was possible to import an address with a label `*` which should not be possible.
  The wildcard label is used for backwards compatibility in the `listtransactions` rpc.
  I added test coverage for these RPCs.

ACKs for top commit:
  ajtowns:
    ACK 65e78bda7c
  achow101:
    ACK 65e78bda7c
  furszy:
    diff ACK 65e78bd
  stickies-v:
    re-ACK 65e78bda7c
  theStack:
    re-ACK 65e78bda7c

Tree-SHA512: ad99f2824d4cfae352166b76da4ca0069b7c2eccf81aaa0654be25bbb3c6e5d6b005d93960f3f4154155f80e12be2d0cebd5529922ae3d2a36ee4eed82440b31
2023-01-10 17:31:19 -05:00
Jon Atack
0f5fc4f656 doc: fix up -netinfo relaytxes help
Co-authored-by: "MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>"
2023-01-10 12:55:04 -08:00
MarcoFalke
b264410e01 Merge bitcoin/bitcoin#26864: doc: net: fix link to onion address encoding scheme [ONIONADDRESS]
3076f1815d doc: net: fix link to onion address encoding scheme [ONIONADDRESS] (Sebastian Falbesoner)

Pull request description:

  Instead of referring to a fixed line number to a file in master (which is obviously always quickly outdated), use a permalink tied to the latest commit.

ACKs for top commit:
  vasild:
    ACK 3076f1815d

Tree-SHA512: 7070a7e47d683b1539f33daa4c67093a87d6121a84430a3b24afee139a7f2b3cab32fcdf0bc561f8e177b5ba864a98e4491e08dac90cdd4bd2e4e6b8fa7e4b14
2023-01-10 17:12:40 +01:00
Sebastian Falbesoner
3076f1815d doc: net: fix link to onion address encoding scheme [ONIONADDRESS]
Instead of referring to a fixed line number to a file in master (which
is obviously always quickly outdated), use a permalink tied to the
latest commit.
2023-01-10 14:23:27 +01:00
MarcoFalke
1c9457bc8b Merge bitcoin/bitcoin#26856: ci: Run one task with all tests on credits
fae885b98f ci: Run one task with all tests on credits (MarcoFalke)

Pull request description:

  This should allow to detect any obvious issues in the tests within 10 minutes of opening a pull request, regardless of the current scheduling load on the Cirrus CI community cluster.

ACKs for top commit:
  fanquake:
    ACK fae885b98f

Tree-SHA512: 384651364648cd38a126f64a5cc650dd396735c05f7d8b5d5c9a8ee7b3b98e36f5dfb3b1c0563ba4d9f554b5e0e82e13aa38e78092f11856a82fc76270fe6f3e
2023-01-10 12:33:54 +01:00
glozow
264f9ef17f [validation] return MempoolAcceptResult for every tx on PCKG_TX failure
This makes the interface more predictable and useful. The caller
understands one or more transactions failed, and can learn what happened
with each transaction. We already have this information, so we might as
well return it.

It doesn't make sense to do this for other PackageValidationResult
values because:
- PCKG_RESULT_UNSET: this means everything succeeded, so the individual
  failures are no longer accurate.
- PCKG_MEMPOOL_ERROR: something went wrong with the mempool logic;
  transaction failures might not be meaningful.
- PCKG_POLICY: this means something was wrong with the package as a
  whole. The caller should use the PackageValidationState to find the
  error, rather than looking at individual MempoolAcceptResults.
2023-01-10 11:10:50 +00:00
glozow
dae81e01e8 [refactor] rename variables in AcceptPackage for clarity 2023-01-10 11:09:03 +00:00
glozow
da484bc738 [doc] release note effective-feerate and effective-includes RPC results
No release note for submitpackage because it is regtest-only.
2023-01-10 11:09:03 +00:00
glozow
5eab397b98 [validation] remove PackageMempoolAcceptResult::m_package_feerate
This value creates an extremely confusing interface as its existence is
dependent upon implementation details (whether something was submitted
on its own, etc). MempoolAcceptResult::m_effective_feerate is much more
helpful, as it always exists for submitted transactions.
2023-01-10 11:09:03 +00:00
glozow
601bac88cb [rpc] return effective-includes in testmempoolaccept and submitpackage 2023-01-10 11:07:38 +00:00
glozow
1691eaa818 [rpc] return effective-feerate in testmempoolaccept and submitpackage 2023-01-10 11:06:10 +00:00
glozow
d6c7b78ef2 [validation] return wtxids of other transactions whose fees were used 2023-01-10 10:36:57 +00:00
MarcoFalke
0e81d119e0 Merge bitcoin/bitcoin#26851: ci: Remove unused busybox workaround
fadfae42f1 ci: Remove unused busybox workaround (MarcoFalke)
fac424fce7 ci: Create named symbol for BINS_SCRATCH_DIR (MarcoFalke)

Pull request description:

  It can be re-added when it is needed again. But it may be more likely that the other workarounds can be removed as well, in a follow-up.

ACKs for top commit:
  fanquake:
    ACK fadfae42f1

Tree-SHA512: 39984d02ddcba907a5b780fd9cb4dab7f1abbc32468968b4862d226de2e4f95c71708c7361db97347e1ac65196dc40d379e68f34bef37d42b5464d343e7466d8
2023-01-10 09:54:32 +01:00
Andrew Chow
1aedc3b6c8 Merge bitcoin/bitcoin#26618: rpc: Prevent unloading a wallet when rescanning
109cbb819d doc: Add release notes for #26618 (Aurèle Oulès)
b13902d2e4 rpc: Prevent unloading a wallet when rescanning (Aurèle Oulès)

Pull request description:

  Fixes #26463.

  This PR prevents a user from unloading a wallet if it is currently rescanning.

  To test:

  ```bash
  ./src/bitcoin-cli -testnet -named createwallet wallet_name=wo disable_private_keys=true
  ./src/bitcoin-cli -testnet -rpcwallet=wo importdescriptors '[{
    "desc": "addr(mmcuW74MyJUZuLnWXGQLoNXPrS9RbFz6gD)#tpnrahgc",
        "timestamp": 0,
        "active": false,
        "internal": false,
        "next": 0
  }]'
  ./src/bitcoin-cli -testnet unloadwallet wo
  error code: -4
  error message:
  Wallet is currently rescanning. Abort existing rescan or wait.

ACKs for top commit:
  achow101:
    ACK 109cbb819d
  w0xlt:
    ACK 109cbb819d
  kouloumos:
    ACK 109cbb819d
  promag:
    ACK 109cbb819d

Tree-SHA512: 15fdddf4cf9f3fa08f52069fe4a25a76e04a55bb2586b031bfb0393dce7f175dcdb52823e132a7dff6a894539beeb980a1aad2a792790be036be6977628149b2
2023-01-09 16:56:40 -05:00
John Moffett
576f7b8614 Fix misleading RPC console wallet message
In certain circumstances, the GUI console will display
the message 'Executing command without any wallet' when
it is, in fact, using the default wallet.

In RPC calls, if no wallet is explicitly selected and
there is exactly one wallet loaded, the default is to
act on that loaded wallet.

The GUI console acts that way in reality, but
erroneously reports that it's not acting on any
particular wallet.
2023-01-09 16:02:25 -05:00
jonatack
3e1d2941e9 doc: remove recommended I2P router versions
as these go stale and users will generally install the current versions available.
2023-01-09 08:18:58 -08:00
jonatack
295849abb5 doc: update/clarify/de-emphasize I2P transient address section 2023-01-09 08:18:58 -08:00
jonatack
dffa319457 doc: update bandwidth section of I2P documentation 2023-01-09 08:18:58 -08:00
jonatack
0ed9cc5892 doc: clarify -i2pacceptincoming help documentation
and also hoist the default setting to a constexpr and
remove unused f-string operators in a related functional test.
2023-01-09 08:18:47 -08:00
MarcoFalke
fae885b98f ci: Run one task with all tests on credits 2023-01-09 16:31:14 +01:00
MarcoFalke
39363a4b94 Merge bitcoin/bitcoin#26822: p2p, rpc: don't allow past absolute timestamp in setban
abccb27466 test: add coverage for absolute timestamp in `setban` (brunoerg)
b99f1f20f7 p2p, rpc: don't allow past absolute timestamp in `setban` (brunoerg)

Pull request description:

  We shouldn't allow call `setban` with past absolute timestamp. First, because doesn't make sense to ban a node until ~ past ~. Besides that, it could make an unnecessary write to the DB since `BanMan::Ban` calls `DumpBanlist` and it calls `SweepBanned` which will remove this new ban (because of the timestamp) of the array.

ACKs for top commit:
  1440000bytes:
    ACK abccb27466

Tree-SHA512: 6d0cadf99fc4f78d77d3bafd6f5c85ac56e243ebc376210fdb2bee751e7b139ec7d6f5f346317fd0b10051b685f2d0ee1d8e40f4bc10f4dbdbbddd5e1ee84de5
2023-01-09 13:00:42 +01:00
MarcoFalke
fadfae42f1 ci: Remove unused busybox workaround
The find workaround is no longer needed after commit
d3d547c545
2023-01-09 12:40:45 +01:00
MarcoFalke
fac424fce7 ci: Create named symbol for BINS_SCRATCH_DIR
Also, create the dir a bit earlier, right after it was put in the PATH.
2023-01-09 11:53:20 +01:00
MarcoFalke
49aefc2c2e Merge bitcoin/bitcoin#26843: ci: Fix ci_native_fuzz_msan CONTAINER_NAME
fa4e98c77f ci: Fix ci_native_fuzz_msan CONTAINER_NAME (MarcoFalke)

Pull request description:

  This avoids a duplicate name with the other msan task, which will lead to errors when running locally:

  > Error: creating container storage: the container name "ci_native_msan" is already in use by 77350e26f9c36abbb601140cd0b485ead093ff118803c720ca8b10f6bdfa37d2. You have to remove that container to be able to reuse that name: that name is already in use

ACKs for top commit:
  fanquake:
    ACK fa4e98c77f
  hebasto:
    ACK fa4e98c77f, I've verified that there are no other duplicated `CONTAINER_NAME`'s values.

Tree-SHA512: f1b28b21302c0947912d642c12c2ccad236af6824fd27e68341baddedec24087af738f3226028a0eeb6e0fc7e9f90713fc680855eeb07adc113c4f6e8b03a545
2023-01-09 09:46:17 +01:00
John Moffett
9a1d73fdff Fix segfault when shutdown during wallet open
If you open a wallet and send a shutdown signal during
that process, the GUI will segfault due to some queued
wallet events happening after the wallet controller
is deleted. This is a minimal fix for those issues.
2023-01-08 19:31:56 -05:00
Matt Whitlock
691eaf8873 Pass MSG_MORE flag when sending non-final network messages
Since Nagle's algorithm is disabled, each and every call to send(2) can potentially generate a separate TCP segment on the wire. This is especially inefficient when sending the tiny header preceding each message payload.

Linux implements a MSG_MORE flag that tells the kernel not to push the passed data immediately to the connected peer but rather to collect it in the socket's internal transmit buffer where it can be combined with data from successive calls to send(2). Where available, specify this flag when calling send(2) in CConnman::SocketSendData(CNode &) if the data buffer being sent is not the last one in node.vSendMsg.
2023-01-07 14:11:07 -05:00
MarcoFalke
fa4e98c77f ci: Fix ci_native_fuzz_msan CONTAINER_NAME 2023-01-07 11:47:38 +01:00
glozow
1605886380 [validation] return effective feerate from mempool validation 2023-01-06 17:37:01 +00:00
glozow
5d35b4a7de [test] package validation quits early due to non-policy, non-missing-inputs failure 2023-01-06 17:37:01 +00:00
glozow
be2e4d94e5 [validation] when quitting early in AcceptPackage, set package_state and tx result
Bug: not setting package_state means package_state.IsValid() == true and
the caller does not know that this failed.

We won't be validating this transaction again, so it makes sense to return this
failure to the caller.

Rename package_state to package_state_quit_early to make it more clear
what this variable is used for and what its scope is.

Co-authored-by: Greg Sanders <gsanders87@gmail.com>
2023-01-06 17:37:01 +00:00
brunoerg
abccb27466 test: add coverage for absolute timestamp in setban 2023-01-06 13:33:38 -03:00
brunoerg
b99f1f20f7 p2p, rpc: don't allow past absolute timestamp in setban 2023-01-06 13:33:38 -03:00
MarcoFalke
adc41cf3b2 Merge bitcoin/bitcoin#26805: tests: Use unique port for ZMQ tests to allow for multiple test instances
c6119f4788 tests: Use unique port for ZMQ tests (Andrew Chow)

Pull request description:

  The ZMQ interface tests should use unique ports as we do for the p2p and rpc ports so that multiple instances of the test can be run at the same time.

  Without this, the test may hang until killed, or fail.

ACKs for top commit:
  MarcoFalke:
    ACK c6119f4788

Tree-SHA512: 2ca3ed2f35e5a83d7ab83740674fed362a8d146dc751156cfe100133a591347cd1ac9d164046f1744d65451a57c52cb22d3bb2161105f421f8f655c4a2512c59
2023-01-06 16:31:34 +01:00
Aurèle Oulès
5ca7a7be76 rpc: Return accurate results for scanblocks
This makes use of undo data to accurately verify results
from blockfilters.
2023-01-06 12:01:22 +01:00
Hennadii Stepanov
1a0d8e178c build: Re-enable external signer on Windows 2023-01-06 10:53:36 +00:00
Luke Dashjr
989451d068 configure: Detect compatibility of Boost.Process rather than hardcode non-Windows 2023-01-06 10:51:01 +00:00
fanquake
6ba17d4955 scripts: add PE Canary check to security-check 2023-01-06 10:49:18 +00:00
fanquake
911a40ead2 Merge bitcoin/bitcoin#26557: build: Update Boost to 1.81.0 in depends
2427468f27 doc: Update Boost version in doc/dependencies.md (Hennadii Stepanov)
e8b4201ba2 build: Update Boost to 1.81.0 in depends (Hennadii Stepanov)

Pull request description:

  Required for https://github.com/bitcoin/bitcoin/pull/25696, also see https://github.com/bitcoin/bitcoin/pull/25696#discussion_r1005600540.

  Guix build:
  ```
  0aade9c6a91d8550dea6cb07d4657c299a78d4434c1e78ef487e30e53239da64  guix-build-e8b4201ba2fd/output/aarch64-linux-gnu/SHA256SUMS.part
  f21964d25f96bca46ced3286b00e7e605d6517a6de7b00aa7ebf3bb6ee63d0d2  guix-build-e8b4201ba2fd/output/aarch64-linux-gnu/bitcoin-e8b4201ba2fd-aarch64-linux-gnu-debug.tar.gz
  65e991407232714455a4bb5fb072b3b7f58f8f4696ca78b6756aae6e6497540b  guix-build-e8b4201ba2fd/output/aarch64-linux-gnu/bitcoin-e8b4201ba2fd-aarch64-linux-gnu.tar.gz
  92535c8b7ead8f3319858e55c7d076b3db174efd10b4b8dce8efbbd5e5b98819  guix-build-e8b4201ba2fd/output/arm-linux-gnueabihf/SHA256SUMS.part
  9cd199415fbfe6ee6fdf5c57def2591327c97e7e31c4c7b323a738761fe6a285  guix-build-e8b4201ba2fd/output/arm-linux-gnueabihf/bitcoin-e8b4201ba2fd-arm-linux-gnueabihf-debug.tar.gz
  e13dacef38e68bb30283fda82bbf73864e5977b4cc6f82be41476246c7f51e90  guix-build-e8b4201ba2fd/output/arm-linux-gnueabihf/bitcoin-e8b4201ba2fd-arm-linux-gnueabihf.tar.gz
  e4c57c126927c2b40a3bd7c697e8b524140f989094c2a9ff216e1c03e90810da  guix-build-e8b4201ba2fd/output/arm64-apple-darwin/SHA256SUMS.part
  90b15bd6e8eab3c541a2a057402cf41630e5c72d07bf62b3d2d4d54698b3f03b  guix-build-e8b4201ba2fd/output/arm64-apple-darwin/bitcoin-e8b4201ba2fd-arm64-apple-darwin-unsigned.dmg
  848c7928d633f19912f340fa6df8b77756d291991dbd849b3d95761bcd445bad  guix-build-e8b4201ba2fd/output/arm64-apple-darwin/bitcoin-e8b4201ba2fd-arm64-apple-darwin-unsigned.tar.gz
  9880bfc57770372478c56b22b513d91d25a799591a345069e04c3f3a88bfc9ec  guix-build-e8b4201ba2fd/output/arm64-apple-darwin/bitcoin-e8b4201ba2fd-arm64-apple-darwin.tar.gz
  e61cb6337dc1480292300fcd45e9b128c567222911cac3a9209955433e1f413b  guix-build-e8b4201ba2fd/output/dist-archive/bitcoin-e8b4201ba2fd.tar.gz
  fd685ffcd1c5e32c25a84638fd47c5e14d0247dcf7d51e5317fea356f8c8347a  guix-build-e8b4201ba2fd/output/powerpc64-linux-gnu/SHA256SUMS.part
  b710e9123684e459732553b1d6df2912257ac43b315db7f2e9288d95c76ce2b6  guix-build-e8b4201ba2fd/output/powerpc64-linux-gnu/bitcoin-e8b4201ba2fd-powerpc64-linux-gnu-debug.tar.gz
  ea7878ea044dbfaf9c022de5bf7532bf00b028c8582d24ce67bec917f34869e9  guix-build-e8b4201ba2fd/output/powerpc64-linux-gnu/bitcoin-e8b4201ba2fd-powerpc64-linux-gnu.tar.gz
  737d9e054356ea50c7efe964a7b223eb81a7fcc10b235647ffcb588784e7e212  guix-build-e8b4201ba2fd/output/powerpc64le-linux-gnu/SHA256SUMS.part
  46de2537ea90ddaced620a7dbfa1379b2d0cc80ae87d6225fef94a7196ff5166  guix-build-e8b4201ba2fd/output/powerpc64le-linux-gnu/bitcoin-e8b4201ba2fd-powerpc64le-linux-gnu-debug.tar.gz
  7e10be7f494bd4635477919ca9b9968b62054e0448f2963d1c25f0215361feae  guix-build-e8b4201ba2fd/output/powerpc64le-linux-gnu/bitcoin-e8b4201ba2fd-powerpc64le-linux-gnu.tar.gz
  cfb90022ea23526d8ca26259a06d2ec06a278e8fbcb05d6ba37fa3ef33ba1f77  guix-build-e8b4201ba2fd/output/riscv64-linux-gnu/SHA256SUMS.part
  ba45d57e19758682991fc51cc04c0e13e7e586b74ae1adae0273cd15bdd5cdb2  guix-build-e8b4201ba2fd/output/riscv64-linux-gnu/bitcoin-e8b4201ba2fd-riscv64-linux-gnu-debug.tar.gz
  970941bee4c342b7b23065ce515d5e340b3aed989210a569faa33f638a3575f9  guix-build-e8b4201ba2fd/output/riscv64-linux-gnu/bitcoin-e8b4201ba2fd-riscv64-linux-gnu.tar.gz
  3943c224a3fa0554ca75292f301ae02514c7ec100d9e13e185ed6ad4db932194  guix-build-e8b4201ba2fd/output/x86_64-apple-darwin/SHA256SUMS.part
  4c5f14af1961e33336423a2021b5da76b550ae1bc2284003d6009b1ced940eea  guix-build-e8b4201ba2fd/output/x86_64-apple-darwin/bitcoin-e8b4201ba2fd-x86_64-apple-darwin-unsigned.dmg
  3ab7fd5623b414c4e0504c2943b84048524374150e9626b3a0b6931adc46109a  guix-build-e8b4201ba2fd/output/x86_64-apple-darwin/bitcoin-e8b4201ba2fd-x86_64-apple-darwin-unsigned.tar.gz
  00c4c2da392eaee439984ab2bd614d2df6b48ba775350b2c6d48c3061e7667c5  guix-build-e8b4201ba2fd/output/x86_64-apple-darwin/bitcoin-e8b4201ba2fd-x86_64-apple-darwin.tar.gz
  df20f8a3c8836ca5b7bac4c5104a31f4be76da75a67846d69cc8cbdf42339743  guix-build-e8b4201ba2fd/output/x86_64-linux-gnu/SHA256SUMS.part
  68f72876397bfbf6cde73b8303bd081235407211d684cdb78fc932ffb0752942  guix-build-e8b4201ba2fd/output/x86_64-linux-gnu/bitcoin-e8b4201ba2fd-x86_64-linux-gnu-debug.tar.gz
  769556d6a4a8baed5dfb81ab63afd1c0999101a5afeb2f7a1bab9b94ce6635bc  guix-build-e8b4201ba2fd/output/x86_64-linux-gnu/bitcoin-e8b4201ba2fd-x86_64-linux-gnu.tar.gz
  4b8487e9b16e0ec6645cb63ddd25149011797af14d596ed31a23130f201d8f57  guix-build-e8b4201ba2fd/output/x86_64-w64-mingw32/SHA256SUMS.part
  6fbad9caf9da6041f0d119be293d0895b50535604f86dda45a7b8345fb581a39  guix-build-e8b4201ba2fd/output/x86_64-w64-mingw32/bitcoin-e8b4201ba2fd-win64-debug.zip
  42e84737cac5ec2d2052e5632f68bc6d211d5f02a171cb7347c067025edce13f  guix-build-e8b4201ba2fd/output/x86_64-w64-mingw32/bitcoin-e8b4201ba2fd-win64-setup-unsigned.exe
  9205f92b1ecfcfa649552e27bb0942611dfac2a5f99c326b7de89bbeb46ad564  guix-build-e8b4201ba2fd/output/x86_64-w64-mingw32/bitcoin-e8b4201ba2fd-win64-unsigned.tar.gz
  098d98161393e663da794cc336de8a4676fd2c8bc14dbc8b1cac1aaf6b4098f5  guix-build-e8b4201ba2fd/output/x86_64-w64-mingw32/bitcoin-e8b4201ba2fd-win64.zip
  ```

ACKs for top commit:
  fanquake:
    ACK 2427468f27

Tree-SHA512: 31d268e4608ab75f843c0cb1874d060cea762ebbd09b220a7f84054fb837f60b41b8452c181a4de7aa3f8aa27fd584d0a84e770209e02adfb978d2b5f625d7a6
2023-01-06 10:37:10 +00:00
Hennadii Stepanov
2427468f27 doc: Update Boost version in doc/dependencies.md 2023-01-06 08:48:48 +00:00
MarcoFalke
2cfe379623 Merge bitcoin/bitcoin#26823: refactor: Work around Werror=free-nonheap-object in AssumeCalculateMemPoolAncestors
faa86eeb41 refactor: Work around Werror=free-nonheap-object in AssumeCalculateMemPoolAncestors (MarcoFalke)

Pull request description:

  This works around the s390x gcc bug mentioned in https://github.com/bitcoin/bitcoin/issues/26820

ACKs for top commit:
  achow101:
    ACK faa86eeb41

Tree-SHA512: 041d5daa157ea1856b0a8027181085d70624f5f8822049ace9963e90c653bbb8c91d1f16b8a5bf460687eb4ed13f1db72e3885a511aadbad6dede93d9f9ccd6d
2023-01-06 08:04:44 +01:00
Miles Liu
4159ccd031 test: Run feature_bip68_sequence.py with MiniWallet 2023-01-06 09:52:39 +08:00
Miles Liu
fc0caaf4aa test: Add "include mempool" flag to MiniWallet rescan_utxos 2023-01-06 09:52:39 +08:00
Miles Liu
d0a909ae54 test: Add "include immature coinbase" flag to MiniWallet get_utxos 2023-01-06 09:52:39 +08:00
Miles Liu
e5b9127d9e test: Add signs P2TR and RAWSCRIPT to MiniWallet sign_tx 2023-01-06 09:52:39 +08:00
Andrew Toth
0e21b56a44 assumeutxo: catch and log fs::remove error instead of two exist checks 2023-01-05 17:35:14 -05:00
Andrew Toth
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles 2023-01-05 17:04:28 -05:00
brunoerg
4bdcf57158 test: test banlist database recreation 2023-01-05 17:42:27 -03:00
fanquake
b358bde020 randomenv: consolidate WIN32 #ifdefs
Order includes.
Remove // for xyz comments
2023-01-05 19:45:03 +00:00
fanquake
fff80cd248 random: remove windows-only compat.h include in randomenv
Note that this was probably only here to indirectly receive windows.h
via another include in compat.h (windows.h or winreg.h aren't included
there).

Also note that compat.h is already pulled in here for everyone via
util/time.h, so including inside a windows only ifdef is secondarily
redundant.
2023-01-05 19:38:49 +00:00
fanquake
0f883df7a5 build: fix configuring with only bitcoin-util
Fixes the issue presented in #25037 in a single (easily backportable)
diff, with no additional refactoring/changes.

Can be tested with:
```bash
./configure \
  --disable-tests \
  --disable-bench \
  --without-libs \
  --without-daemon \
  --without-gui \
  --disable-fuzz-binary \
  --without-utils \
  --enable-util-util
```
2023-01-05 19:05:40 +00:00
MarcoFalke
faa86eeb41 refactor: Work around Werror=free-nonheap-object in AssumeCalculateMemPoolAncestors 2023-01-05 19:48:14 +01:00
fanquake
94d0c418c5 build: remove already tested headers from AC_CHECK_HEADERS
These headers are already included in a default set which are checked
early during configure.

We already use at least sys/types.h and unistd.h unconditionally in
configure.
2023-01-05 18:33:23 +00:00
Hennadii Stepanov
45553e11c9 refactor: Make ThreadHTTP return void
The `bool` return value was introduced in 755aa05174.

It has been not used since 8d3f46ec39.

No behavior change.
2023-01-05 17:54:08 +00:00
Andrew Chow
b4fb0a3255 Merge bitcoin/bitcoin#26761: wallet: fully migrate address book entries for watchonly/solvable wallets
730e14a317 test: wallet: check that labels are migrated to watchonly wallet (Sebastian Falbesoner)
d5f4ae7fac wallet: fully migrate address book entries for watchonly/solvable wallets (Sebastian Falbesoner)

Pull request description:

  Currently `migratewallet` migrates the address book (i.e. labels and purposes) for watchonly and solvable wallets only in RAM, but doesn't persist them on disk. Fix this by adding another loop for both of the special wallet types after which writes the corresponding NAME and PURPOSE entries to the database in a single batch. Also adds a corresponding test that checks if labels were migrated correctly for a watchonly wallet.

ACKs for top commit:
  achow101:
    ACK 730e14a317
  furszy:
    code ACK 730e14a3, left a non-blocking nit.
  aureleoules:
    ACK 730e14a317

Tree-SHA512: 159487e11e858924ef762e0190ccaea185bdff239e3d2280c8d63c4ac2649ec71714dc4d53dec644f03488f91c3b4bbbbf3434dad23bc0fcecb6657f353ea766
2023-01-05 12:22:51 -05:00
MarcoFalke
3212d104f4 Merge bitcoin/bitcoin#23829: refactor: use braced init for integer literals instead of c style casts
f2fc03ec85 refactor: use braced init for integer constants instead of c style casts (Pasta)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/23810 for more context. This is broken out from that PR, as it is less breaking, and should be trivial to review and merge.

  EDIT: Long term, the intention is to remove all C-style casts, as they can dangerously introduce reinterpret_casts. This is one step which removes a number of trivially removable C-style casts

ACKs for top commit:
  aureleoules:
    ACK f2fc03ec85

Tree-SHA512: 2fd11b92c9147e3f970ec3e130e3b3dce70e707ff02950a8c697d4b111ddcbbfa16915393db20cfc8f384bc76f13241c9b994a187987fcecd16a61f8cc0af14c
2023-01-05 17:30:52 +01:00
MarcoFalke
61f35159ff Merge bitcoin/bitcoin#26818: test: Fix feature_startupnotify intermittent issue
fac810bb0a test: Fix feature_startupnotify intermittent issue (MarcoFalke)

Pull request description:

  Might fix #25644

ACKs for top commit:
  aureleoules:
    ACK fac810bb0a
  brunoerg:
    ACK fac810bb0a

Tree-SHA512: 870bf65da8120b6897d02e3bb70eea018d4761396abe64c3533bbc5237e65be9f77d35f62cd5d08cf7132dd53b504bf58229c33e18833c191495ad229c84d7c2
2023-01-05 17:22:28 +01:00
MarcoFalke
fac810bb0a test: Fix feature_startupnotify intermittent issue 2023-01-05 14:10:07 +01:00
fanquake
282019cd3d refactor: add kernel/cs_main.*
Co-authored-by: Anthony Towns <aj@erisian.com.au>
2023-01-05 09:05:14 +00:00
MarcoFalke
296e882250 Merge bitcoin/bitcoin#26598: contrib: remove builder keys
e6864fa157 contrib: remove builder keys (fanquake)

Pull request description:

  This has been superseded by adding a builder-keys/ directory in
  guix.sigs, where the presence of keys, and validity of signatures
  is checked. Preventing issues like missing keys or invalid signatures.

  New (or exisiting) Guix builders can add their key in the next PR
  they open adding attestations.

  Related to issues like #26566, #26563.

  Also follows up with the comment here: https://github.com/bitcoin/bitcoin/pull/26565#issuecomment-1326053939.

ACKs for top commit:
  hebasto:
    ACK e6864fa157, modulo s/update/remove/ in the PR tittle.

Tree-SHA512: 095b4cf12ed0baeaf0ee7b8edcb3e2647e9c0f812e8fd63915ddb454f81dacc9c2d2b409de2773b7adb5ff643893d614d8aad1bc44c26da648e1bbbe19e11e05
2023-01-05 09:18:16 +01:00
Andrew Chow
360e047a71 Merge bitcoin/bitcoin#26747: wallet: fix confusing error / GUI crash on cross-chain legacy wallet restore
21ad4e26ec test: add coverage for cross-chain wallet restore (Sebastian Falbesoner)
8c7222bda3 wallet: fix GUI crash on cross-chain legacy wallet restore (Sebastian Falbesoner)

Pull request description:

  Restoring a wallet backup from another chain should result in a dedicated error message (we have _"Wallet files should not be reused across chains. Restart bitcoind with -walletcrosschain to override."_ for that). Unfortunately this is currently not the case for legacy wallet restores, as in the course of cleaning up the newly created wallet directory a `filesystem_error` exception is thrown due to the directory not being empty; the wallet database did indeed load successfully (otherwise we wouldn't know that the chain doesn't match) and hence BDB-related files and directories are already created in the wallet directory.

  For bitcoind, this leads to a very confusing error message:
  ```
  $ ./src/bitcoin-cli restorewallet test123 ~/.bitcoin/regtest/wallets/regtest_wallet/wallet.dat
  error code: -1
  error message: filesystem error: in remove: Directory not empty ["/home/thestack/.bitcoin/wallets/test123"]
  ```

  Even worse, the GUI crashes in such a scenario:
  ```
  libc++abi: terminating with uncaught exception of type std::__1::__fs::filesystem::filesystem_error: filesystem error: in remove: Directory not empty ["/home/thestack/.bitcoin/wallets/foobar"]
  Abort trap (core dumped)
  ```

  Fix this by simply deleting the whole folder via `fs::remove_all`. With this, the expected error message appears both for the `restorewallet` RPC call and in the GUI (as a message-box):

  ```
  $ ./src/bitcoin-cli restorewallet test123 ~/.bitcoin/regtest/wallets/regtest_wallet/wallet.dat
  error code: -4
  error message:
  Wallet loading failed. Wallet files should not be reused across chains. Restart bitcoind with -walletcrosschain to override.
  ```

ACKs for top commit:
  achow101:
    ACK 21ad4e26ec
  aureleoules:
    ACK 21ad4e26ec
  furszy:
    utACK 21ad4e26

Tree-SHA512: 313f6494c2fbe823bff9b975cb2d9410bb518977a1e59a5159ee9836bc012947fa50b56be0e41b1a2f50d9c0c7f4fddfdf4fbe479d8a59a6ee44bb389c804abc
2023-01-04 17:46:37 -05:00
Andrew Chow
cabeae43ea Merge bitcoin/bitcoin#26809: compat: use STDIN_FILENO over 0
585c672212 compat: use STDIN_FILENO over 0 (fanquake)

Pull request description:

  This is already used throughout this file, and is self-documenting.

ACKs for top commit:
  john-moffett:
    ACK 585c672212
  achow101:
    ACK 585c672212
  hebasto:
    ACK 585c672212, I have reviewed the code and it looks OK, I agree it can be merged.
  kristapsk:
    utACK 585c672212
  aureleoules:
    ACK 585c672212

Tree-SHA512: c0114ae896ba5404be70b804ee9f454d213f1d789c8f5a578c422dd15a308a214e6851fee76c0ec736a212bc86fb33ec17af1b22e5d23422c375ca4458251356
2023-01-04 17:30:47 -05:00
glozow
196a43eddb Merge bitcoin/bitcoin#26603: doc: CalculateSequenceLocks: prevHeights entries are set to 0, not removed
f537127271 doc: fix: prevHeights entries are set to 0, not removed (stickies-v)

Pull request description:

  In [`CalculateSequenceLocks`](a035b6a0c4/src/consensus/tx_verify.h (L69)) no items are removed from `prevHeights`, they are just set to 0:

  a035b6a0c4/src/consensus/tx_verify.cpp (L69-L73)

  This PR updates the docs to reflect the actual implementation. Seems to have been wrongly documented since introduction in #7184 already ([implementation](https://github.com/bitcoin/bitcoin/pull/7184/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R742-R749) and [documentation](https://github.com/bitcoin/bitcoin/pull/7184/files#diff-34d21af3c614ea3cee120df276c9c4ae95053830d7f1d3deaf009a4625409ad2R712-R713))

ACKs for top commit:
  hebasto:
    ACK f537127271

Tree-SHA512: 3661501660f6832b2116fd83466ffe95a60b341c14cb09a37489e2a587bea3290b0528690120a0f644c3eea02177aa1fb8968258482fa43b0303e016abb17418
2023-01-04 18:07:31 +00:00
glozow
65ecf24b5c Merge bitcoin/bitcoin#26752: wallet: Remove mempool_sequence from interface methods
55696a0ac3 wallet: remove `mempool_sequence` from `transactionRemovedFromMempool` (w0xlt)
bf19069c53 wallet: remove `mempool_sequence` from `transactionAddedToMempool` (w0xlt)

Pull request description:

  This PR removes `mempool_sequence` from `transactionRemovedFromMempool` and `transactionAddedToMempool`.

  `mempool_sequence` is  not used in these methods, only in ZMQ notifications.

ACKs for top commit:
  instagibbs:
    ACK 55696a0ac3

Tree-SHA512: 621e89230bcb6edfed83e2758601a2b093822fc2dc4e9bfb00487e340f2bc4c5ac3bf6df3ca00b7fe55bb3df15858820f2bf698f403d2e48b915dd9eb47b63e0
2023-01-04 17:53:58 +00:00
Andrew Chow
a273241480 Merge bitcoin/bitcoin#26020: test: Change coinselection parameter location to make tests independent
b942c94d15 test: Change coinselection parameter location to make tests independent (yancy)

Pull request description:

  the `subtract_fee_outputs` param is expected to be `true` for all subsequent tests.  It should be defined outside of a single test so that if it's removed or changed, all subsequent tests won't fail.  Currently if you remove this [test](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L304:L325) the following [test](https://github.com/bitcoin/bitcoin/blob/master/src/wallet/test/coinselector_tests.cpp#L327:L345) fails.  This change makes the tests independent.

ACKs for top commit:
  achow101:
    ACK b942c94d15
  aureleoules:
    ACK b942c94d15.
  rajarshimaitra:
    tACK b942c94d15
  theStack:
    ACK b942c94d15

Tree-SHA512: 461e19d15351318102ef9f96c68442365d8ca238c48ad7aefe23e8532b33b91dadf6c7840c7894574bccede6da162a55ad7a6f6a330d61a11ce804e68ddc5e9c
2023-01-04 12:41:47 -05:00
Andrew Chow
139ba2bf12 Merge bitcoin/bitcoin#25234: bench: add benchmark for wallet 'AvailableCoins' function.
3a4f8bc242 bench: add benchmark for wallet 'AvailableCoins' function. (furszy)

Pull request description:

  #### Rationale

  `AvailableCoins` is part of several important flows for the wallet; from RPC commands that create transactions like `fundrawtransaction`, `send`, `walletcreatefundedpsbt`, get the available balance, list the available coins with `listunspent` etc. to GUI connected processes that perform the same or similar actions: tx creation, available balance calculation, present the spendable coins in the coin control dialog.

  As we are improving this process in #24699, #25005 and there are more structural changes coming on the way. This benchmark aims to ensure us that, at least, there are no regressions (obviously performance improvements are great but, at least for me, this heads into the direction of having a base metric to compare future structural changes).

  #### Implementation Notes

  There are 5 new benchmarks, one per wallet supported output type (LEGACY, P2SH_SEGWIT, BECH32, BECH32M), plus a multi-output-type wallet benchmark which contains outputs from all the descriptor types.

  The test, by default, fills-up the wallet with 1k transactions, 2k outputs. Mainly to not consume much time if the user just want to verify that no substantial regressions were introduced. But, my expectation for those who are focused on this process is to use a much higher number locally to really note the differences across commits.

ACKs for top commit:
  achow101:
    ACK 3a4f8bc242
  hernanmarino:
    ACK 3a4f8bc242
  aureleoules:
    ACK 3a4f8bc242

Tree-SHA512: d0bb4c165f1efa181b47cb31561e6217eff9135bcd1b6761a7292f9018e456d13d18a1b886c2e2268d35c52f9e1fd8e0f252972424e5c5f00c280620b79c5a1b
2023-01-04 12:11:44 -05:00
stickies-v
6d0ab07e81 refactor: use convenience fn to auto parse non-string parameters
Minimizes code duplication and improves function naming by having
a single (overloaded) convenience function that both checks if
the parameter is a non-string parameter and automatically parses the
value if so.
2023-01-04 16:06:34 +00:00
MarcoFalke
bf3b589413 Merge bitcoin/bitcoin#26791: ci: Properly set COMMIT_RANGE in lint task
fa5cbf2290 ci: Properly set COMMIT_RANGE in lint task (MarcoFalke)

Pull request description:

  Currently the variable holds (apart from the commits in the pull request) all commits to master since the pull was opened.

  This is problematic, because already merged commits are linted in unrelated pulls, leading to:

  * Wasted resources. For example, currently the lint task may take 9 minutes, when it should take 1. See https://cirrus-ci.com/task/6032782770569216?logs=lint#L1449
  * False failures. For example, when a "wrong" commit is in master it can lead to some pulls failing unrelatedly, and others not.

  Now that the CI has the `/merge` commit (since commit fad7281d78), `COMMIT_RANGE` can simply be set to `HEAD~..HEAD` to only hold the changes in the pull.

ACKs for top commit:
  fanquake:
    ACK fa5cbf2290

Tree-SHA512: e85fca4ca9d2615ddd2544403485e06885769a3f70bca297e23eefda2a1d28f47c5271f6adfa6ce0e5e972335c78098b76e0db4b109f59d0986bf508cef7528f
2023-01-04 13:56:45 +01:00
MarcoFalke
4bb840a453 Merge bitcoin/bitcoin#26802: test: Use same Python executable for subprocesses as for all-lint.py
f6eadaa413 Use same Python executable for subprocesses as for all-lint.py (Kristaps Kaupe)

Pull request description:

  Before this all linters were ran by `/usr/bin/env python3`, no matter what was used to run `test/lint/all-lint.py`. This change allows to use non-default Python executable for `test/lint/all-lint.py` and then all subprocesses will also use same Python interpreter (for example, `python3.10 ./test/lint/all-lint.py`). See https://github.com/bitcoin/bitcoin/issues/26792#issuecomment-1369558866 as use case.

ACKs for top commit:
  fanquake:
    ACK f6eadaa413 - did not test

Tree-SHA512: 4da3b5581a0dd8ab9a6387829495019091a93a7ceaf2135d65d40a1983fd11a0b92b20891ef30d2a132abb0a690cd9b2f7eb5fcc38df06a340394ef449d640af
2023-01-04 13:53:27 +01:00
Aurèle Oulès
65e78bda7c test: Invalid label name coverage 2023-01-04 13:45:05 +01:00
Aurèle Oulès
552b51e682 refactor: Add sanity checks in LabelFromValue 2023-01-04 13:45:03 +01:00
fanquake
4bb91be124 debian: remove nonexistent files from copyright 2023-01-04 12:32:20 +00:00
fanquake
585c672212 compat: use STDIN_FILENO over 0
This is already used throughout this file, and is self-documenting.
2023-01-04 12:00:25 +00:00
fanquake
2ec97825e7 Merge bitcoin/bitcoin#26771: doc: Correct linked Microsoft URLs
f84e445dee doc: Correct linked Microsoft URLs (Suriyaa Sundararuban)

Pull request description:

  Update Microsoft-related links.

Top commit has no ACKs.

Tree-SHA512: 40c7b25a96772259fb04da1946d52f6aac9562262aef472ae75807bfbd246de47d72118140a12f7553037b94b89f95d69dea6ce30e611ac3d71a32d102355150
2023-01-04 11:40:56 +00:00
Aurèle Oulès
67e7ba8e1a rpc: Sanitize label name in various RPCs
- importprivkey
- importaddress
- importpubkey
- listtransactions
- listsinceblock
- importmulti
- importdescriptors
2023-01-04 12:31:28 +01:00
fanquake
4717a5aa31 Merge bitcoin/bitcoin#26772: contrib: fix sha256 check in install_db4.sh for FreeBSD
22e9afe40d use sha256 command instead of sha256sum on FreeBSD (Murray Nesbitt)

Pull request description:

  The FreeBSD version of `sha256sum` takes different arguments than the GNU version.

  The `sha256_check` function in `contrib/install_db4.sh` has code specific to FreeBSD, however it doesn't get reached because while the `sha256sum` command does exist on FreeBSD, it is incompatible and results in an error:

  ```
  sha256sum: option requires an argument -- c
  usage: sha256sum [-pqrtx] [-c file] [-s string] [files ...]
  ```

  This change moves the FreeBSD-specific code before the check for the `sha256sum` command.

  Fixes: #26774

Top commit has no ACKs.

Tree-SHA512: 2485e2e7d8fdca3b072b29fb22bbdfd69e520740537b331b33c64cc645b63da712cfa63a23bdf039bbc92a6558fc7bf03323a51784bf601ff360ff0ef59506c8
2023-01-04 10:24:00 +00:00
MarcoFalke
53653060c1 Merge bitcoin/bitcoin#26795: rpc: Correct RPCHelpMan for fundrawtransaction's input_weights field
927b8d4e0c rpc: Correct RPCHelpMan for fundrawtransaction's input_weights field (jdjkelly@gmail.com)

Pull request description:

  `input_weights` is incorrectly documented as a fixed length JSON array, but it is actually a JSON array of JSON objects - this commit changes `input_weights` to use `RPCArg::Type::OBJ`

  The behavior of `input_weights` as an object exists as a functional test in [wallet_fundrawtransaction.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/wallet_fundrawtransaction.py).

ACKs for top commit:
  achow101:
    ACK 927b8d4e0c

Tree-SHA512: 384f5e16be36dba670d64d96f16f1fde2d0d51357e1094ae13eb71d004af0f4dc8bac965b4d2d724ccf64fb671faad37b73055152a9882af24f65dfceaf1e5fb
2023-01-04 11:09:57 +01:00
glozow
03254c2229 Merge bitcoin/bitcoin#19909: refactor: Remove unused CTxMemPool::clear() helper
fa818e103c txmempool: Remove unused clear() member function (MarcoFalke)

Pull request description:

  Seems odd to have code in Bitcoin Core that is unused.

  Moreover the function was broken (see https://github.com/bitcoin/bitcoin/pull/24145) and is brittle, as there is nothing that prevents similar bugs from re-appearing.

  Fix both issues by replacing it with C++11 member initializers.

ACKs for top commit:
  glozow:
    ACK fa818e103c

Tree-SHA512: e79e44cac7d5a84d9ecc8e3f3b0b9a50e1e3ebec358b20ba5dac175ef07d1fbe338a20f83ee80f746f7c726c79e77f8be49e14bca57a41063da8a5302123c3a9
2023-01-04 08:44:26 +00:00
Pasta
f2fc03ec85 refactor: use braced init for integer constants instead of c style casts 2023-01-03 19:31:29 -06:00
Andrew Chow
3f8591d46b Merge bitcoin/bitcoin#26661: wallet: Coin Selection, return accurate error messages
76dc547ee7 gui: create tx, launch error dialog if backend throws runtime_error (furszy)
f4d79477ff wallet: coin selection, add duplicated inputs checks (furszy)
0aa065b14e wallet: return accurate error messages from Coin Selection (furszy)
7e8340ab1a wallet: make SelectCoins flow return util::Result (furszy)
e5e147fe97 wallet: refactor eight consecutive 'AttemptSelection' calls into a loop (furszy)

Pull request description:

  Work decoupled from #25806, which cleanup and improves the Coin Selection flow further.

  Adding the capability to propagate specific error messages from the Coin Selection process to the user.
  Instead of always returning the general "Insufficient funds" message which is not always accurate to what happened internally.
  Letting us instruct the user how to proceed under certain circumstances.

  The following error messages were added:

  1) If the selection result exceeds the maximum transaction weight,
     we now will return:
  -> "The inputs size exceeds the maximum weight. Please try sending
  a smaller amount or manually consolidating your wallet's UTXOs".

  2) If the user pre-selected inputs and disallowed the automatic coin
     selection process (no other inputs are allowed), we now will
     return:
  -> "The preselected coins total amount does not cover the transaction
  target. Please allow other inputs to be automatically selected or include
  more coins manually".

  3) The double-counted preset inputs during Coin Selection error will now
  throw an "internal bug detected" message instead of crashing the node.

  The essence of this work comes from several comments:
  1. https://github.com/bitcoin/bitcoin/pull/26560#discussion_r1037395665
  2. https://github.com/bitcoin/bitcoin/pull/25729#discussion_r940619491
  3. https://github.com/bitcoin/bitcoin/pull/25269#pullrequestreview-1135240825
  4. https://github.com/bitcoin/bitcoin/issues/23144 (which is connected to #24845)

ACKs for top commit:
  ishaanam:
    crACK 76dc547ee7
  achow101:
    ACK 76dc547ee7
  aureleoules:
    ACK 76dc547ee7
  theStack:
    ACK 76dc547ee7 🌇

Tree-SHA512: 9de30792d7a5849cae77747aa978e70390b66ee9d082779a56088a024f82e725b0af050e6603aece0ac8229f6d73bc471ba97b4ab69dc7eddf419f5f56ae89a5
2023-01-03 18:53:36 -05:00
Sebastian Falbesoner
21ad4e26ec test: add coverage for cross-chain wallet restore 2023-01-04 00:06:05 +01:00
Andrew Chow
c6119f4788 tests: Use unique port for ZMQ tests
The ZMQ interface tests should use unique ports as we do for the p2p and
rpc ports so that multiple instances of the test can be run at the same
time.
2023-01-03 17:30:26 -05:00
Andrew Chow
80fc1af096 Merge bitcoin/bitcoin#26289: Use util::Result in for calculating mempool ancestors
47c4b1f52a mempool: log/halt when CalculateMemPoolAncestors fails unexpectedly (stickies-v)
5481f65849 mempool: add AssumeCalculateMemPoolAncestors helper function (stickies-v)
f911bdfff9 mempool: use util::Result for CalculateMemPoolAncestors (stickies-v)
66e028f739 mempool: use util::Result for CalculateAncestorsAndCheckLimits (stickies-v)

Pull request description:

  Upon reviewing the documentation for `CTxMemPool::CalculateMemPoolAncestors`, I noticed `setAncestors` was meant to be an `out` parameter but actually is an `in,out` parameter, as can be observed by adding `assert(setAncestors.empty());` as the first line in the function and running `make check`. This PR fixes this unexpected behaviour and introduces refactoring improvements to make intents and effects of the code more clear.

  ## Unexpected behaviour
  This behaviour occurs only in the package acceptance path, currently only triggered by `testmempoolaccept` and `submitpackage` RPCs.

  In `MemPoolAccept::AcceptMultipleTransactions()`, we first call `PreChecks()` and then `SubmitPackage()` with the same `Workspace ws` reference. `PreChecks` leaves `ws.m_ancestors` in a potentially non-empty state, before it is passed on to `MemPoolAccept::SubmitPackage`. `SubmitPackage` is the only place where `setAncestors` isn't guaranteed to be empty before calling `CalculateMemPoolAncestors`. The most straightforward fix is to just forcefully clear `setAncestors` at the beginning of CalculateMemPoolAncestors, which is done in the first bugfix commit.

  ## Improvements
  ### Return value instead of out-parameters
  This PR updates the function signatures for `CTxMemPool::CalculateMemPoolAncestors` and `CTxMemPool::CalculateAncestorsAndCheckLimits` to use a `util::Result` return type and eliminate both the `setAncestors` `in,out`-parameter as well as the error string. It simplifies the code and makes the intent and effects more explicit.

  ### Observability
  There are 7 instances where we currently call `CalculateMemPoolAncestors` without actually checking if the function succeeded because we assume that it can't fail, such as in [miner.cpp](69b10212ea/src/node/miner.cpp (L399)). This PR adds a new wrapper `AssumeCalculateMemPoolAncestors` function that logs such unexpected failures, or in case of debug builds even halts the program. It's not crucial to the objective, more of an observability improvement that seems sensible to add on here.

ACKs for top commit:
  achow101:
    ACK 47c4b1f52a
  w0xlt:
    ACK 47c4b1f52a
  glozow:
    ACK 47c4b1f52a
  furszy:
    light code review ACK 47c4b1f5
  aureleoules:
    ACK 47c4b1f52a

Tree-SHA512: d908dad00d1a5645eb865c4877cc0bae74b9cd3332a3641eb4a285431aef119f9fc78172d38b55c592168a73dae83242e6af3348815f7b37cbe2d448a3a58648
2023-01-03 16:30:55 -05:00
Kristaps Kaupe
f6eadaa413 Use same Python executable for subprocesses as for all-lint.py 2023-01-03 23:23:07 +02:00
MarcoFalke
f301bf52ab Merge bitcoin/bitcoin#26257: script, test: python linter flake8 E275 fixup, update dependencies
1e5e87cec3 script: update python linter dependencies (Jon Atack)
459cb637ac script, test: fix python linter E275 errors with flake8 5.0.4 (Jon Atack)

Pull request description:

  It is helpful to be able to run the python linter locally to review PRs and check local diffs and work.  Fix the errors raised by `./test/lint/lint-python.py` when run locally with flake8 5.0.4, which enforces rule E275 more strictly than previous versions, and update our python linter CI dependencies.

ACKs for top commit:
  kristapsk:
    ACK 1e5e87cec3

Tree-SHA512: c7da09396144b9fed4ce6405c0763b2e3e5651d49a5220b053da465aceca09f754fb70a8ab9647278ce2028dde803bea461a3cd93fb39868ead39d06187cd8af
2023-01-03 22:08:52 +01:00
furszy
a2ac6f9582 wallet: unify FindNonChangeParentOutput functions
The function is only used in ListCoins.
2023-01-03 17:25:36 -03:00
furszy
b3f4e82737 wallet: simplify ListCoins implementation
Can remove the locked coins lookup if we include them directly
inside the AvailableCoins result
2023-01-03 17:25:35 -03:00
Jon Atack
1e5e87cec3 script: update python linter dependencies 2023-01-03 11:05:09 -08:00
Jon Atack
459cb637ac script, test: fix python linter E275 errors with flake8 5.0.4 2023-01-03 10:59:56 -08:00
Andrew Chow
cb552c5f21 Merge bitcoin/bitcoin#26192: rpc: Improve error when wallet is already loaded
04609284ad rpc: Improve error when wallet is already loaded (Aurèle Oulès)

Pull request description:

  Currently, trying to load a descriptor (sqlite) wallet that is already loaded throws the following error:
  > error code: -4
  > error message:
  > Wallet file verification failed. SQLiteDatabase: Unable to obtain an exclusive lock on the database, is it being used by another instance of Bitcoin Core?

  I don't think it is very clear what it means for a user.

  While a legacy wallet would throw:
  > error code: -35
  > error message:
  > Wallet file verification failed. Refusing to load database. Data file '/home/user/.bitcoin/signet/wallets/test_wallet/wallet.dat' is already loaded.

  This PR changes the error message for both types of wallet to:
  > error code: -35
  > error message:
  > Wallet file verification failed. Wallet "test_wallet" is already loaded.

ACKs for top commit:
  achow101:
    ACK 04609284ad
  hernanmarino:
    ACK  0460928
  theStack:
    Tested ACK 04609284ad

Tree-SHA512: a8f3d5133bfaef7417a6c05d160910ea08f32ac62bfdf7f5ec305ff5b62e9113b55f385abab4d5a4ad711aabcb1eb7ef746eb41f841b196e8fb5393ab3ccc01e
2023-01-03 13:02:20 -05:00
Andrew Chow
65d7c31b3f Merge bitcoin/bitcoin#25789: test: clean and extend availablecoins_tests coverage
9622fe64b8 test: move coins result test to wallet_tests.cpp (furszy)
f69347d058 test: extend and simplify availablecoins_tests (furszy)
212ccdf2c2 wallet: AvailableCoins, add arg to include/skip locked coins (furszy)

Pull request description:

  Negative PR with extended test coverage :).

  1) Cleaned duplicated code and added coverage for the 'AvailableCoins' incremental result.

  2) The class `AvailableCoinsTestingSetup` inside `availablecoins_tests.cpp` is a plain copy
  of `ListCoinsTestingSetup` that is inside `wallet_tests.cpp`.

      So, deleted the file and moved the `BasicOutputTypesTest` test case to `wallet_tests.cpp`.

  3) Added arg to include/skip locked coins from the `AvailableCoins` result. This is needed for point (1) as otherwise the wallet will spend the coins that we recently created due its closeness to the recipient amount.
  Note: this last point comes from #25659 where I'm using the same functionality to clean/speedup another flow as well.

ACKs for top commit:
  achow101:
    ACK 9622fe64b8
  theStack:
    ACK 9622fe64b8
  aureleoules:
    reACK 9622fe64b8, nice cleanup!

Tree-SHA512: 1ed9133120bfe8815455d1ad317bb0ff96e11a0cc34ee8098716ab9b001749168fa649212b2fa14b330c1686cb1f29039ff1f88ae306db68881b0428c038f388
2023-01-03 12:52:40 -05:00
Andrew Chow
7bb07bf8bd Merge bitcoin/bitcoin#25932: refactor: Simplify backtrack logic
81d4a2b14f refactor: Move feerate comparison invariant outside of the loop (yancy)
365aca4045 refactor: Simplify feerate comparison statement (yancy)

Pull request description:

  This is a small nit, however I think it's more understandable to write:

  `utxo_pool.at(0).fee > utxo_pool.at(0).long_term_fee`

  vs

  `(utxo_pool.at(0).fee - utxo_pool.at(0).long_term_fee) > 0`

ACKs for top commit:
  Xekyo:
    ACK 81d4a2b14f
  achow101:
    ACK 81d4a2b14f
  aureleoules:
    ACK 81d4a2b14f

Tree-SHA512: 3e89377989c36716b53114fe40178261671dde5688075fab1c21ec173ac310f8c84ed6af90354d7c329176cb7262dfcaa7191fd19847d3b7147a9a10c3e31176
2023-01-03 12:26:19 -05:00
Andrew Chow
1e6b384d59 Merge bitcoin/bitcoin#26702: refactor: walletdb: drop unused FindWalletTx parameter and rename
f496528556 walletdb: refactor: drop unused `FindWalletTx` parameter and rename (Sebastian Falbesoner)

Pull request description:

  Since commit 3340dbadd3 ("Remove -zapwallettxes"), the `FindWalletTx` helper is only needed to read tx hashes, so drop the other parameter and rename the method accordingly.

ACKs for top commit:
  S3RK:
    code review ACK f496528556
  achow101:
    ACK f496528556
  vincenzopalazzo:
    ACK f496528556

Tree-SHA512: ead85bc724462f9e920f9d7fe89679931361187579ffd6e63427c8bf5305cd5f71da24ed84f3b1bd22a12be46b5abec13f11822e71a3e1a63bf6cf49de950ab5
2023-01-03 11:54:51 -05:00
MarcoFalke
eeee61065f Use AutoFile and HashVerifier where possible 2023-01-03 12:55:29 +01:00
MarcoFalke
fa961141f7 Add HashVerifier
It is similar to CHashVerifier, but HashVerifier does not need a
serialize type and version
2023-01-03 12:54:45 +01:00
jdjkelly@gmail.com
927b8d4e0c rpc: Correct RPCHelpMan for fundrawtransaction's input_weights field
input_weights is incorrectly documented as a fixed length JSON array,
but it is actually a JSON array of JSON objects - this commit changes
input_weights to use RPCArg::Type::OBJ
2023-01-02 14:31:49 -05:00
MarcoFalke
fa5cbf2290 ci: Properly set COMMIT_RANGE in lint task 2023-01-02 14:05:23 +01:00
MarcoFalke
d8bdee0fc8 Merge bitcoin/bitcoin#26775: ci: Revert tsan task changes
fabb6af850 ci: Remove duplicate CC and CXX from tsan task (MarcoFalke)
fa5d9a0e24 Revert "ci: Use clang-15 in tsan task" (MarcoFalke)
faa835e7e5 Revert "test: Drop no longer needed `race:epoll_ctl` TSan suppression" (MarcoFalke)

Pull request description:

  Looks like there are still bugs in clang-15, so we need to roll back all the way to the previously used version (clang-13).

ACKs for top commit:
  hebasto:
    ACK fabb6af850, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: d62203049847ab9095ee3fc89e18bdd721d1d9d5a7ef7a9f524c80e6be58d1d9f6aa2f14533df1ea77eb59597fba6fa9b987b17eb03b2c3f7cb577ab59cd59c0
2023-01-01 10:26:02 +01:00
Yusuf Sahin HAMZA
057057a2d7 Add test for sendmany rpc that uses subtractfeefrom parameter 2023-01-01 02:53:16 +03:00
Suriyaa Sundararuban
f84e445dee doc: Correct linked Microsoft URLs 2022-12-31 16:54:13 +01:00
MarcoFalke
8575d5d842 Merge bitcoin/bitcoin#26777: rpc: Remove duplicate field in RPCHelpMan for gettransactions
090ad51c80 rpc: Remove duplicate field in RPCHelpMan for gettransactions (Joshua Kelly)

Pull request description:

  The field 'comment' appears twice in `TransactionDescriptionString`, incorrectly - this commit removes the instance of the comment field without a description, preserving the one with a description.

  On master, the duplicate fields can be be viewed here: https://github.com/bitcoin/bitcoin/blob/master/src/wallet/rpc/transactions.cpp#L419-L423

  `TransactionDescriptionString` is included in RPC calls such as `listtransactions` which have functional tests.

ACKs for top commit:
  w0xlt:
    ACK 090ad51c80

Tree-SHA512: 4bacdafdb517dda2af6d1c193f331b634ae74bd62ac6289c0c288957f39f98a73d07aeab72fbe5bf1ece5532406d4a40a5b8a2277be50115f76c92bb938e21fa
2022-12-31 13:40:05 +01:00
Joshua Kelly
090ad51c80 rpc: Remove duplicate field in RPCHelpMan for gettransactions
The field 'comment' appears twice in TransactionDescriptionString,
incorrectly - this commit removes the instance of the comment field
without a description, preserving the one with a description
2022-12-30 15:46:30 -05:00
MarcoFalke
fabb6af850 ci: Remove duplicate CC and CXX from tsan task 2022-12-30 09:50:22 +01:00
MarcoFalke
fa5d9a0e24 Revert "ci: Use clang-15 in tsan task"
This reverts commit faa00ca78e.
2022-12-30 09:49:51 +01:00
MarcoFalke
faa835e7e5 Revert "test: Drop no longer needed race:epoll_ctl TSan suppression"
This reverts commit a3f5e54152.
2022-12-30 09:47:52 +01:00
Murray Nesbitt
22e9afe40d use sha256 command instead of sha256sum on FreeBSD 2022-12-29 22:23:44 -08:00
MarcoFalke
65de8eeeca Merge bitcoin/bitcoin#26770: ci: Move --enable-c++20 from "tidy" task back to "ASan..." one
afc6052430 ci: Move `--enable-c++20` from "tidy" task back to "ASan..." one (Hennadii Stepanov)

Pull request description:

  This PR reverts cc7335edc8 from https://github.com/bitcoin/bitcoin/pull/25528 partially.

  C++20 has introduced some new headers, and it is premature to consider them when using the IWYU tool.

  Required for https://github.com/bitcoin/bitcoin/pull/26763 and https://github.com/bitcoin/bitcoin/pull/26766.

  Related discussions:
  - https://github.com/bitcoin/bitcoin/pull/25528#discussion_r1058906785
  - https://github.com/bitcoin/bitcoin/pull/26763#discussion_r1058860943

ACKs for top commit:
  MarcoFalke:
    review only ACK afc6052430

Tree-SHA512: 9c1d3317d0f7a94d46d1e442da1a91669cd9ed1e62a41579edc96e1f5447551b536d32eeb222caf277e85228482753be545e0a874208cb24f9a8491fce3b6d9f
2022-12-29 20:49:21 +01:00
MarcoFalke
e9e2e87c85 Merge bitcoin/bitcoin#26768: ci: Use clang-15 in tsan task
faa00ca78e ci: Use clang-15 in tsan task (MarcoFalke)

Pull request description:

  Generally it is best to use the latest clang version for sanitizers, because it comes with the most features and bugfixes.

  So bump to clang-15, the latest release, for the tsan task.

  The task was using clang-13 (instead of 14) due to a bug, see https://github.com/bitcoin/bitcoin/pull/24572#issue-1169970859. Bumping to 15 will hopefully fix this bug, as well as https://github.com/bitcoin/bitcoin/pull/26759#issuecomment-1367360491

ACKs for top commit:
  hebasto:
    ACK faa00ca78e

Tree-SHA512: adb2386bb9615a3e1185e0624b0b68cd2738309530185819714a26e63bdf1c79461c4b4d3aa9cbe2fe08cc412349d7453f192abbbe9fb5adca74cf4b148ae7b7
2022-12-29 20:27:07 +01:00
Hennadii Stepanov
afc6052430 ci: Move --enable-c++20 from "tidy" task back to "ASan..." one
This change reverts cc7335edc8 partially.

C++20 has introduced some new headers, and it is premature to consider
them when using the IWYU tool.
2022-12-29 18:39:00 +00:00
MarcoFalke
faa00ca78e ci: Use clang-15 in tsan task 2022-12-29 16:30:13 +01:00
MarcoFalke
3b6e0f0345 Merge bitcoin/bitcoin#26738: test: add coverage for unknown wallet flag in setwalletflag
3666a06730 test: add coverage for unknown wallet flag in `setwalletflag` (brunoerg)

Pull request description:

  This PR adds test coverage for the following error:
  6d40a1a7e7/src/wallet/rpc/wallet.cpp (L275-L277)

  https://marcofalke.github.io/btc_cov/total.coverage/src/wallet/rpc/wallet.cpp.gcov.html

ACKs for top commit:
  aureleoules:
    ACK 3666a06730

Tree-SHA512: b6b2415442dfbc2404aed9720cc899995686007d6ba222dae461d064e4454d5af1326d3d527770b51b1005721ac42a49972f1eabf21108f656c30d3584790747
2022-12-29 11:46:56 +01:00
MarcoFalke
b9028b2e26 Merge bitcoin/bitcoin#26481: bench: Suppress output when running with -sanity-check option
f1e89597c8 test: Drop no longer required bench output redirection (Hennadii Stepanov)
4dbcdf26a3 bench: Suppress output when running with `-sanity-check` option (Hennadii Stepanov)

Pull request description:

  This change allows to simplify CI tests, and makes it easier to integrate the `bench_bitcoin` binary into CMake custom [targets](https://cmake.org/cmake/help/latest/command/add_custom_target.html) or [commands](https://cmake.org/cmake/help/latest/command/add_custom_command.html), as `COMMAND` does not support output redirection.

ACKs for top commit:
  aureleoules:
    tACK f1e89597c8. Ran as expected and is more practical than using an output redirection.

Tree-SHA512: 29086d428cccedcfd031c0b4514213cbc1670e35f955e8fd35cee212bc6f9616cf9f20d0cb984495390c4ae2c50788ace616aea907d44e0d6a905b9dda1685d8
2022-12-29 11:42:03 +01:00
MarcoFalke
4654506c30 Merge bitcoin/bitcoin#26759: test: Drop no longer needed race:epoll_ctl TSan suppression
a3f5e54152 test: Drop no longer needed `race:epoll_ctl` TSan suppression (Hennadii Stepanov)

Pull request description:

  The removed suppression seems no needed.

  I cannot point the exact commit/PR which makes this change possible.

Top commit has no ACKs.

Tree-SHA512: 8ee79cbdb2bc62796d72c69be4a818379132eae47be33951e8b9d224b049ff77e867004801c7cb0cc564a5374f318dafd9142b5231e9bd428f80acc75253933e
2022-12-28 18:02:50 +01:00
Sebastian Falbesoner
730e14a317 test: wallet: check that labels are migrated to watchonly wallet 2022-12-28 13:51:08 +01:00
Sebastian Falbesoner
d5f4ae7fac wallet: fully migrate address book entries for watchonly/solvable wallets
Currently `migratewallet` migrates the address book (i.e. labels and
purposes) for watchonly and solvable wallets only in RAM, but doesn't
persist them on disk. Fix this by adding another loop for both of the
special wallet types after which writes the corresponding NAME and
PURPOSE entries to the database in a single batch.
2022-12-28 13:44:22 +01:00
Hennadii Stepanov
a3f5e54152 test: Drop no longer needed race:epoll_ctl TSan suppression 2022-12-27 18:33:34 +00:00
Hennadii Stepanov
9567bfeab9 clang-tidy: Add performance-no-automatic-move check
https://clang.llvm.org/extra/clang-tidy/checks/performance/no-automatic-move.html
2022-12-27 15:25:51 +00:00
furszy
f4a11d7baf gui: bugfix, catch invalid networks combination crash
We shouldn't crash if a network is set inside
bitcoin.conf and another one is provided as param.
2022-12-26 11:08:52 -03:00
w0xlt
55696a0ac3 wallet: remove mempool_sequence from transactionRemovedFromMempool 2022-12-26 06:17:05 -03:00
w0xlt
bf19069c53 wallet: remove mempool_sequence from transactionAddedToMempool 2022-12-26 06:14:24 -03:00
MarcoFalke
e9262ea32a Merge bitcoin/bitcoin#26750: Insert and bump copyright headers
3ae76ea6dd scripted-diff: Insert missed copyright header (Hennadii Stepanov)
306ccd4927 scripted-diff: Bump copyright headers (Hennadii Stepanov)

Pull request description:

  This PR bumps the existing copyright headers, as we did every year, and adds a missed one.

Top commit has no ACKs.

Tree-SHA512: 5f6b02e2baad21750e3dd8f0612bb6e7e2cfa6a743c669f26baf5a39c168b2d3a92afae1ce2dad59b70492175186c38f172c4ee68fc7ac87a4d85330429ca054
2022-12-25 10:50:16 +01:00
MarcoFalke
06dd571534 Merge bitcoin/bitcoin#26748: doc: Update license year range to 2023
e9abb13ecb doc: Update license year range to 2023 (Suriyaa Sundararuban)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/23945, https://github.com/bitcoin/bitcoin/pull/20805, https://github.com/bitcoin/bitcoin/pull/17801, https://github.com/bitcoin/bitcoin/pull/15061
  The same procedure as every year. Happy new year to all of you! 😄

Top commit has no ACKs.

Tree-SHA512: 2c80ae41a827e3fa6980a4406ed3518238f953465d7672b730a77c121cee6758c7cc1aa615ce9b988019aacb253c681dcc3c312bd982366de7038a0200d27bc4
2022-12-25 10:45:54 +01:00
Hennadii Stepanov
3ae76ea6dd scripted-diff: Insert missed copyright header
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py insert src/policy/fees_args.cpp
-END VERIFY SCRIPT-
2022-12-24 23:59:12 +00:00
Hennadii Stepanov
306ccd4927 scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
- 2021: f47dda2c58
- 2020: fa0074e2d8
- 2019: aaaaad6ac9
2022-12-24 23:49:50 +00:00
Suriyaa Sundararuban
e9abb13ecb doc: Update license year range to 2023 2022-12-24 11:40:16 +01:00
Sebastian Falbesoner
8c7222bda3 wallet: fix GUI crash on cross-chain legacy wallet restore
Restoring a wallet backup from another chain should obviously result
in a dedicated error message (we have "Wallet files should not be
reused across chains. Restart bitcoind with -walletcrosschain to
override." for that). Unfortunately this is currently not the case
for legacy wallet restores, as in the course of cleaning up the
newly created wallet directory a `filesystem_error` exception is
thrown due to the directory not being empty; the wallet database did
indeed load successfully (otherwise we wouldn't know that the chain doesn't
match) and hence BDB-related files and directories are created in the wallet
directory.

For bitcoind, this leads to a very confusing error message:
```
$ ./src/bitcoin-cli restorewallet test123 ~/.bitcoin/regtest/wallets/regtest_wallet/wallet.dat
error code: -1
error message: filesystem error: in remove: Directory not empty ["/home/thestack/.bitcoin/wallets/test123"]
```

Even worse, the GUI crashes in such a scenario:
```
libc++abi: terminating with uncaught exception of type std::__1::__fs::filesystem::filesystem_error: filesystem error: in remove: Directory not empty ["/home/thestack/.bitcoin/wallets/foobar"]
Abort trap (core dumped)
```

Fix this by simply deleting the whole folder via `fs::remove_all`.
2022-12-23 03:24:36 +01:00
Fabian Jahr
fe329dc936 test: Add test for getblockfrompeer on pruned nodes 2022-12-22 20:01:47 +01:00
glozow
04528054fc [bench] BlockAssembler with mempool packages
The current BlockAssembler bench only tests on a mempool where all
transactions have 0 ancestors or descendants, which does not exercise
any of the package-handling logic in BlockAssembler
2022-12-22 11:33:46 +00:00
glozow
6ce265acf4 [test util] lock cs_main before pool.cs in PopulateMempool 2022-12-22 11:33:44 +00:00
glozow
8791410662 [test util] randomize fee in PopulateMempool
This makes the contents of the mempool more realistic and iterating by
ancestor feerate order more meaningful. If transactions have varying
feerates, it's also more likely that packages will need to be updated
during block template assembly.
2022-12-22 11:33:42 +00:00
glozow
cba5934eb6 [miner] allow bypassing TestBlockValidity
Allows us to test BlockAssembler on transactions without signatures or
mature coinbases (which is what PopulateMempool creates). Also means
that `TestBlockValidity()` is not included in the bench timing.
2022-12-22 11:33:39 +00:00
glozow
c058852308 [refactor] parameterize BlockAssembler::Options in PrepareBlock 2022-12-22 11:33:37 +00:00
glozow
a2de971ba1 [refactor] add helper to apply ArgsManager to BlockAssembler::Options
This allows us to both manually manipulate options and grab values from
ArgsManager (i.e. -blockmaxweight and -blockmintxfee config options)
when constructing BlockAssembler::Options. Prior to this change, the
only way to apply the config options is by ctoring BlockAssembler with
no options, which calls DefaultOptions().
2022-12-22 11:33:28 +00:00
furszy
76dc547ee7 gui: create tx, launch error dialog if backend throws runtime_error
only will ever happen if something unexpected happened.
2022-12-21 23:20:17 -03:00
furszy
f4d79477ff wallet: coin selection, add duplicated inputs checks
As no process should be able to trigger this error
using the regular transaction creation process, throw
a runtime_error if happens to tell users/devs to
report the bug if happens.
2022-12-21 23:20:16 -03:00
furszy
0aa065b14e wallet: return accurate error messages from Coin Selection
and not the general "Insufficient funds" when the wallet
actually have funds.

Two new error messages:

1) If the selection result exceeds the maximum transaction weight,
   we now will return: "The inputs size exceeds the maximum weight".

2) If the user preselected inputs and disallowed the automatic coin
   selection process (no other inputs are allowed), we now will
   return: "The preselected coins total amount does not cover the
   transaction target".
2022-12-21 23:14:50 -03:00
furszy
7e8340ab1a wallet: make SelectCoins flow return util::Result 2022-12-21 23:14:50 -03:00
furszy
e5e147fe97 wallet: refactor eight consecutive 'AttemptSelection' calls into a loop
and remove 'CoinEligibilityFilter' default constructor to prevent
mistakes.
2022-12-21 23:14:50 -03:00
Andrew Chow
f3bc1a7282 Merge bitcoin/bitcoin#26265: POLICY: Relax MIN_STANDARD_TX_NONWITNESS_SIZE to 65 non-witness bytes
b2aa9e8528 Add release note for MIN_STANDARD_TX_NONWITNESS_SIZE relaxation (Greg Sanders)
8c5b3646b5 Relax MIN_STANDARD_TX_NONWITNESS_SIZE to 65 non-witness bytes (Greg Sanders)

Pull request description:

  Since the original fix was set to be a "reasonable" transaction to reduce allocations and the true motivation later revealed, it makes sense to relax this check to something more principled.

  There are more exotic transaction patterns that could take advantage of a relaxed requirement, such as 1 input, 1 output OP_RETURN to burn a utxo to fees for CPFP purposes when change isn't practical.

  Two changes could be accomplished:

  1) Anything not 64 bytes could be allowed

  2) Anything above 64 bytes could be allowed

  In the Great Consensus Cleanup, suggestion (2)
  was proposed as a consensus change, and is the simpler of the two suggestions. It would not allow an "empty" OP_RETURN but would reduce the required padding from 22 bytes to 5.

  The functional test is also modified to test the actual case
  we care about: 64 bytes

  Related mailing list discussions here:
  https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-October/020995.html
  And a couple years earlier:
  https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2020-May/017883.html

ACKs for top commit:
  achow101:
    reACK b2aa9e8528
  glozow:
    reACK b2aa9e8528
  pablomartin4btc:
    re-ACK b2aa9e8528
  jonatack:
    ACK b2aa9e8528 with some suggestions

Tree-SHA512: c1ec1af9ddcf31b2272209a4f1ee0c5607399f8172e5a1dfd4604cf98bfb933810dd9369a5917ad122add003327c9fcf6ee26995de3aca41d5c42dba527991ad
2022-12-21 12:58:46 -05:00
brunoerg
3666a06730 test: add coverage for unknown wallet flag in setwalletflag 2022-12-21 11:03:24 -03:00
MarcoFalke
6d40a1a7e7 Merge bitcoin/bitcoin#26694: test: get_previous_releases.py: M1/M2 macs can't run unsigned arm64 binaries; self-sign when needed
dc12f2e212 test: improve error msg on previous release tarball extraction failure (kdmukai)
7121fd8fa7 test: self-sign previous release binaries for arm64 macOS (kdmukai)

Pull request description:

  ## The Problem
  If you run `test/get_previous_releases.py -b` on an M1 or M2 mac, you'll get an unsigned v23.0 binary in the arm64 tarball. macOS [sets stricter requirements on ARM binaries](https://news.ycombinator.com/item?id=26996578) so the unsigned arm64 binary is apparently completely unusable without being signed/notarized(?).

  This means that any test that depends on a previous release (e.g. `wallet_backwards_compatibility.py`) will fail because the v23.0 node cannot launch:

  ```
  TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/Users/kdmukai/dev/bitcoin-core/test/functional/test_framework/test_framework.py", line 563, in start_nodes
      node.wait_for_rpc_connection()
    File "/Users/kdmukai/dev/bitcoin-core/test/functional/test_framework/test_node.py", line 231, in wait_for_rpc_connection
      raise FailedToStartError(self._node_msg(
  test_framework.test_node.FailedToStartError: [node 2] bitcoind exited with status -9 during initialization
  ```

  This can also be confirmed by downloading bitcoin-23.0-arm64-apple-darwin.tar.gz (https://bitcoincore.org/bin/bitcoin-core-23.0/) and trying to run any of the binaries manually on an M1 or M2 mac.

  ## Solution in this PR
  (UPDATED) Per @ hebasto, we can self-sign the arm64 binaries. This PR checks each binary in the previous release's "bin/" and verifies if the arm64 binary is signed. If not, attempt to self-sign and confirm success.

  (note: an earlier version of this PR downloaded the x86_64 binary as a workaround but this approach has been discarded)

  ## Longer term solution
  If possible, produce signed arm64 binaries in a future v23.x tarball?

  Note that this same problem affects the new v24.0.1 arm64 tarball so perhaps a signed v24.x.x tarball would also be ideal?

  That being said, this PR will check all current and future arm64 binaries and self-sign as needed, so perhaps we need not worry about pre-signing the tarball binaries. And I did test a version of `get_previous_releases.py` that includes the new v24.0.1 binaries and it successfully self-signed both v23.0 and v24.0.1, as expected.

  ## Further info:
  Somewhat related to: https://github.com/bitcoin/bitcoin/issues/15774#issuecomment-1265164753

  And @ fanquake noted on IRC that you can confirm which binaries are or are not signed via:
  ```
  $ codesign -v -d bitcoin-qt
  bitcoin-qt: code object is not signed at all
  ```

ACKs for top commit:
  hebasto:
    ACK dc12f2e212

Tree-SHA512: 644895f8e97f5ffb3c4754c1db2c48abd77fa100c2058e3c896af04806596fc2b9c807a3f3a2add5be53301ad40ca2b8171585bd254e691f6eb38714d938396b
2022-12-21 11:02:20 +01:00
fanquake
3e536ea9ac Merge bitcoin/bitcoin#26683: ci: Use CONFIG_SITE variable and --prefix option properly
d3a84347e8 ci: remove --prefix from msan job (fanquake)
574e50addf ci: Use `CONFIG_SITE` variable and `--prefix` option properly (Hennadii Stepanov)

Pull request description:

  When running CI scripts locally, they attempt to use a `$DEPENDS_DIR/$HOST` directory even `NO_DEPENDS=1` is provided.

  This PR fixes this broken behavior.

Top commit has no ACKs.

Tree-SHA512: 5e83b921763e6d463e520bbee2ed1599e9f4de36668d19b23dd9d2d7e4441c415e275f588c585b72cadda8bfab5a938979acc1ee4963230aa47081785c741e98
2022-12-21 09:22:57 +00:00
fanquake
dd7d82bec0 Merge bitcoin/bitcoin#26734: doc: Fixup getrawtransaction RPC docs
97115de183 doc: Refactor/Format getrawtransaction RPC docs and add ScriptPubKeyDoc function (Douglas Chimento)

Pull request description:

  Added  `ScriptPubKeyDoc` function

ACKs for top commit:
  MarcoFalke:
    ACK 97115de183
  kristapsk:
    cr utACK 97115de183

Tree-SHA512: 1371375986177862e8c99923eb7f1800fef8da7a7ac9f0ec9037bf5b23681c3348d5afe913aab7457f029ee1774d160ac10d7f57238500a03c6385cc0c7013fc
2022-12-21 08:57:14 +00:00
MarcoFalke
4cd6b3b557 Merge bitcoin-core/gui#687: Load PSBTs using istreambuf_iterator rather than istream_iterator
bb5ea1d9a9 qt: Load PSBTs using istreambuf_iterator rather than istream_iterator (Andrew Chow)

Pull request description:

  `istream_iterator` eats whitespace charactesr which causes parsing failures for PSBTs that contain the bytes corresponding to those characters. `istreambuf_iterator` is the correct thing to use here.

  This is a regression in 24.0. https://github.com/bitcoin/bitcoin/pull/25001 accidentally changed the original `istreambuf_iterator` to `istream_iterator`.

ACKs for top commit:
  furszy:
    Tested ACK bb5ea1d9
  MarcoFalke:
    review ACK bb5ea1d9a9   🍇

Tree-SHA512: 35d90eee3efdcb6a360af69ac1727f9f2837ea621297196de3136299f5de6d9975df4e425e1fc5b8813c1ddb2a4d60c3969e1d5d968953a4628ca45e37d3bf05
2022-12-21 09:47:56 +01:00
MarcoFalke
0139a0d5c0 Merge bitcoin/bitcoin#26722: test: speed up the two slowest functional tests by 18-35% via keypoolrefill()
31fdc54dba test: speed up wallet_fundrawtransaction.py and wallet_sendall.py (kdmukai)

Pull request description:

  ## Problem
  `wallet_fundrawtransaction.py` and `wallet_sendall.py` are the two slowest functional tests *when running without a RAM disk*.
  ```
  # M1 MacBook Pro timings
  wallet_fundrawtransaction.py --descriptors   | ✓ Passed  | 55 s
  wallet_fundrawtransaction.py --legacy-wallet | ✓ Passed  | 381 s

  wallet_sendall.py --descriptors   | ✓ Passed  | 43 s
  wallet_sendall.py --legacy-wallet | ✓ Passed  | 327 s
  ```

  In each case, the majority of the time is spent iterating through 1500 to 1600 `getnewaddress()` calls. This is particularly slow in the `--legacy-wallet` runs.

  see:  https://github.com/bitcoin/bitcoin/blob/master/test/functional/wallet_fundrawtransaction.py#L986-L987
  see:  https://github.com/bitcoin/bitcoin/blob/master/test/functional/wallet_sendall.py#L324

  ## Solution
  Pre-fill the keypool before iterating through those `getnewaddress()` calls.

  With this change, the execution time drops to:
  ```
  wallet_fundrawtransaction.py --descriptors   | ✓ Passed  | 52 s     # -3s diff
  wallet_fundrawtransaction.py --legacy-wallet | ✓ Passed  | 291 s    # -90s diff

  wallet_sendall.py --descriptors   | ✓ Passed  | 27 s     # -16s diff
  wallet_sendall.py --legacy-wallet | ✓ Passed  | 228 s    # -99s diff
  ```

  ---

  Tagging @ Sjors as he had encouraged me to take a look at speeding up the tests.

ACKs for top commit:
  achow101:
    ACK 31fdc54dba

Tree-SHA512: e8dd89323551779832a407d068977c827c09dff55c1079d3c19aab39fcce6957df22b1da797ed7aa3bc2f6dd22fdf9e6f5e1a9a0200fdb16ed6042fc5f6dd992
2022-12-21 09:06:24 +01:00
Andrew Chow
8456bfac6b Merge bitcoin/bitcoin#26638: test: prefer sqlite for wallet tests
17554efb60 test: prefer sqlite for wallet tests (S3RK)
8e0fabaabf test: make wallet_migration.py pass with both wallet flags (S3RK)

Pull request description:

  Fixes #26511

ACKs for top commit:
  MarcoFalke:
    review ACK 17554efb60
  achow101:
    ACK 17554efb60

Tree-SHA512: 97cae275998f07032feffe1b533d4747b8ff03c3c1fb830af69ee38cadb75fd58532956f66f79c0d275b00620ce53b0b5240f885e4f29b8bd4d0b6e6cbc683fa
2022-12-20 18:12:08 -05:00
Douglas Chimento
97115de183 doc: Refactor/Format getrawtransaction RPC docs and add ScriptPubKeyDoc function 2022-12-21 00:46:16 +02:00
Andrew Chow
cbcad79eef Merge bitcoin/bitcoin#21576: rpc, gui: bumpfee signer support
2c07cfacd1 gui: bumpfee signer support (Sjors Provoost)
7e02a33297 rpc: bumpfee signer support (Sjors Provoost)
304ece9945 rpc: document bools in FillPSBT() calls (Sjors Provoost)

Pull request description:

  The `bumpfee` RPC call and GUI fee bump interface now work with an external signer.

ACKs for top commit:
  achow101:
    ACK 2c07cfacd1
  furszy:
    code review ACK 2c07cfac
  jarolrod:
    tACK 2c07cfa

Tree-SHA512: 0c7b931f76fac67c9e33b9b935f29af6f69ac67a5ffcc586ed2f1676feac427735b1d971723b29ef332bb6fb5762949598ebbf728587e8f0ded95a9bfbb3e7a4
2022-12-20 15:30:17 -05:00
Hennadii Stepanov
497f26552b Merge bitcoin-core/gui#605: Delete splash screen widget early
1b228497fa qt: Drop no longer used `SplashScreen::finish()` slot (Hennadii Stepanov)
10811afff4 qt: Drop no longer used `BitcoinApplication::splashFinished()` signal (Hennadii Stepanov)
5299cfe371 qt: Delete splash screen widget explicitly (Hennadii Stepanov)

Pull request description:

  Fixes bitcoin-core/gui#604.
  Fixes bitcoin/bitcoin#25146.
  Fixes bitcoin/bitcoin#26340.

  `SplashScreen::deleteLater()` [does not guarantee](https://doc.qt.io/qt-5/qobject.html#deleteLater) deletion of the `m_splash` object prior to the wallet context deletion. If the latter happens first, the [segfault](https://github.com/bitcoin-core/gui/issues/604#issuecomment-1133907013) follows.

ACKs for top commit:
  dooglus:
    ACK 1b228497fa
  furszy:
    ACK 1b228497
  john-moffett:
    ACK 1b228497fa

Tree-SHA512: bb01d0bf2051f5b184dc415c4f5d32dfb7b8bd772feff7ec7754ded4c6482de27f004b9712df7d53c5ee82e153f48aef4372e4a49d7bcbbb137f73e9b4947962
2022-12-20 20:13:12 +00:00
Andrew Toth
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles 2022-12-20 12:25:54 -05:00
Andrew Toth
77557dda4a prune: scan and unlink already pruned block files on startup 2022-12-20 12:25:36 -05:00
fanquake
d3a84347e8 ci: remove --prefix from msan job 2022-12-20 17:17:35 +00:00
brunoerg
c467cfffce test: add coverage for purpose arg in listlabels 2022-12-20 11:15:28 -03:00
fanquake
dcdfd72861 Merge bitcoin/bitcoin#26721: test, lint: add crypted to ignore-words
a4defcdd57 test, lint: add `crypted` to `ignore-words` (brunoerg)

Pull request description:

  Fixes #26719

  "Crypted" is used in some comments at `walletload_tests` because it refers to `DBKeys::CRYPTED_KEY`, it's not necessary
  a mistake.

  Obs: I can change the approach (changing `walletload_tests` comments to use `encrypted` word instead of adding it to the `ignore_words`) if reviewers think it makes more sense.

ACKs for top commit:
  achow101:
    ACK a4defcdd57

Tree-SHA512: 49f38eed30ffb0fda4e792566591c3455629379619eb9a5c4240c5b00e14cd27ba1faa36337192233752e642f0998373b86fcb8ca586508bbf15900d68b17950
2022-12-20 11:46:07 +00:00
Fabian Jahr
cd761e6b2c rpc: Add note on guarantees to getblockfrompeer 2022-12-19 23:23:57 +01:00
kdmukai
dc12f2e212 test: improve error msg on previous release tarball extraction failure 2022-12-19 11:25:33 -06:00
fanquake
e6864fa157 contrib: remove builder keys
This has been superseded by adding a builder-keys/ directory in
guix.sigs, where the presence of keys, and validity of signatures
is checked. Preventing issues like missing keys or invalid signatures.

New (or exisiting) Guix builders can add their key in the next PR
they open adding attestations.
2022-12-19 17:21:35 +00:00
kdmukai
7121fd8fa7 test: self-sign previous release binaries for arm64 macOS 2022-12-19 11:18:24 -06:00
kdmukai
31fdc54dba test: speed up wallet_fundrawtransaction.py and wallet_sendall.py 2022-12-19 11:12:40 -06:00
fanquake
1dc0e4bc6f rpc: remove optional from fStateStats fields
These are no-longer optional after #26515, so remove the documentation,
and no-op fStateStats checks.
2022-12-19 15:15:41 +00:00
Greg Sanders
b2aa9e8528 Add release note for MIN_STANDARD_TX_NONWITNESS_SIZE relaxation 2022-12-19 10:03:51 -05:00
Greg Sanders
8c5b3646b5 Relax MIN_STANDARD_TX_NONWITNESS_SIZE to 65 non-witness bytes
Since the original fix was set to be a "reasonable" transaction
to reduce allocations and the true motivation later revealed,
it makes sense to relax this check to something more principled.

There are more exotic transaction patterns that could take advantage
of a relaxed requirement, such as 1 input, 1 output OP_RETURN to burn
a utxo to fees for CPFP purposes when change isn't practical.

Two changes could be accomplished:

1) Anything not 64 bytes could be allowed

2) Anything above 64 bytes could be allowed

In the Great Consensus Cleanup, suggestion (2) was the route taken.
It would not allow an "empty" OP_RETURN
but would reduce the required padding from 22 bytes to 5.

The functional test is also modified to test the actual case
we care about: 64 bytes
2022-12-19 10:03:51 -05:00
MarcoFalke
8ab19237e1 Merge bitcoin/bitcoin#26723: test: call keypoolrefill with priv key disabled should throw an error
ec63a4892e test: call `keypoolrefill` with private keys disabled should throw an error (brunoerg)

Pull request description:

  This PR adds test coverage for the following error:
  cb32328d1b/src/wallet/rpc/addresses.cpp (L332-L334)

ACKs for top commit:
  aureleoules:
    ACK ec63a4892e

Tree-SHA512: b5deda8981ff472f290e6e16c8723a58e02cbe099afd1f672c099f4add0a1d9b192b11a2c3f0e11b96794671f6b9efa75812b7a174248d7c58d7fd7d3310e7b9
2022-12-19 15:17:44 +01:00
MarcoFalke
3d974960d3 Merge bitcoin/bitcoin#26515: rpc: skip getpeerinfo for a peer without CNodeStateStats
6fefd49527 rpc: Require NodeStateStats object in getpeerinfo (Martin Zumsande)

Pull request description:

  The objects `CNode`, `CNodeState` and `Peer` store different info about a peer - `InitializeNode()` and `FinalizeNode()` make sure that for the duration of a connection, we should always have one of each for a peer.

  Therefore, there is no situation in which, as part of getpeerinfo RPC,  `GetNodeStateStats()` (which requires a `CNodeState` and a `Peer` entry for a `NodeId` to succeed)  could fail for a legitimate reason while the peer is connected - this can only happen if there is a race condition between peer disconnection and the `getpeerinfo` processing (see also a more detailed description of this in https://github.com/bitcoin/bitcoin/pull/26457#pullrequestreview-1181641835).

  But in this case I think it's better to just not include the newly disconnected peer in the response instead of returning just parts of its data.

  An earlier version of this PR also made the affected `CNodeStateStats` fields non-optional (see 5f900e27d0). Since this conflicts with #25923 and should be a separate discussion, I removed that commit from this PR.

ACKs for top commit:
  dergoegge:
    Approach ACK 6fefd49527
  MarcoFalke:
    review ACK 6fefd49527 👒

Tree-SHA512: 89c8f7318df4634c1630415de9c8350e6dc2d14d9d07e039e5b180c51bfd3ee2ce99eeac4f9f858af7de846f7a6b48fcae96ebac08495b30e431a5d2d4660532
2022-12-19 13:59:17 +01:00
brunoerg
1c07500dbb contrib: make DNS seeds file an argument in CLI 2022-12-19 07:20:14 -03:00
fanquake
65f5cfda65 Merge bitcoin/bitcoin#25311: refactor: remove CBlockIndex copy construction
36c201feb7 remove CBlockIndex copy construction (James O'Beirne)

Pull request description:

  Copy construction of CBlockIndex objects is a footgun because of the
  wide use of equality-by-pointer comparison in the code base. There are
  also potential lifetime confusions of using copied instances, since
  there are recursive pointer members (e.g. pprev).

  (See also https://github.com/bitcoin/bitcoin/pull/24008#discussion_r891949166)

  We can't just delete the copy constructors because they are used for
  derived classes (CDiskBlockIndex), so we mark them protected.

ACKs for top commit:
  ajtowns:
    ACK 36c201feb7 - code review only
  MarcoFalke:
    re-ACK 36c201feb7  🏻

Tree-SHA512: b1cf9a1cb992464a4377dad609713eea63cc099435df374e4553bfe62d362a4eb5e3c6c6649177832f38c0905b23841caf9d62196cef8e3084bfea0bfc26374b
2022-12-19 09:34:39 +00:00
MarcoFalke
bd13d6b369 Merge bitcoin/bitcoin#26656: tests: Improve runtime of some tests when --enable-debug
1647a11f39 tests: Reorder longer running tests in test_runner (Andrew Chow)
ff6c9fe027 tests: Whitelist test p2p connection in rpc_packages (Andrew Chow)
8c20796aac tests: Use waitfornewblock for work queue test in interface_rpc (Andrew Chow)
6c872d5e65 tests: Initialize sigops draining script with bytes in feature_taproot (Andrew Chow)
544cbf776c tests: Use batched RPC in feature_fee_estimation (Andrew Chow)
4ad7272f8b tests: reduce number of generated blocks for wallet_import_rescan (Andrew Chow)

Pull request description:

  When configured with `--enable-debug`, many tests become dramatically slower. These slow downs are particularly noticed in tests that generate a lot of blocks in separate calls, make a lot of RPC calls, or send a lot of data from the test framework's P2P connection. This PR aims to improve the runtime of some of the slower tests and improve the overall runtime of the test runner. This has improved the runtime of the test runner from ~400s to ~140s on my computer.

  The slowest test by far was `wallet_import_rescan.py`. This was taking ~320s. Most of that time was spent waiting for blocks to be mined and then synced to the other nodes. It was generating a new block for every new transaction it was creating in a setup loop. However it is not necessary to have one tx per block. By mining a block only every 10 txs, the runtime is improved to ~61s.

  The second slowest test was `feature_fee_estimation.py`. This test spends most of its time waiting for RPCs to respond. I was able to improve its runtime by batching RPC requests. This has improved the runtime from ~201s to ~140s.

  In `feature_taproot.py`, the test was constructing a Python `CScript` using a very large list of `OP_CHECKSIG`s. The constructor for the Python implementation of `CScript` was iterating this list in order to create a `bytes` from it even though a `bytes` could be created from it without iterating. By making the `bytes` before passing it into the constructor, we are able to improve this test's runtime from ~131s to ~106s.

  Although `interface_rpc.py` was not typically a slow test, I found that it would occasionally have a super long runtime. It typically takes ~7s, but I have observed it taking >400s to run on occasion. This longer runtime occurs more often when `--enable-debug`. This long runtime was caused by the "exceeding work queue" test which is really just trying to trigger a race condition. In this test, it would create a few threads and try an RPC in a loop in the hopes that eventually one of the RPCs would be added to the work queue while another was processing. It used `getrpcinfo` for this, but this function is fairly fast. I believe what was happening was that with `--enable-debug`, all of the code for receiving the RPC would often take longer to run than the RPC itself, so the majority of the requests would succeed, until we got lucky after 10's of thousands of requests. By changing this to use a slow RPC, the race condition can be triggered more reliably, and much sooner as well. I've used `waitfornewblock` with a 500ms timeout. This improves the runtime to ~3s consistently.

  The last test I've changed was `rpc_packages.py`. This test was one of the higher runtime variability tests. The main source of this variation appears to be waiting for the test node to relay a transaction to the test framework's P2P connection. By whitelisting that peer, the variability is reduced to nearly 0.

  Lastly, I've reordered the tests in `test_runner.py` to account for the slower runtimes when configured with `--enable-debug`. Some of the slow tests I've looked at were listed as being fast which was causing overall `test_runner.py` runtime to be extended. This change makes the test runner's runtime be bounded by the slowest test (currently `feature_fee_estimation.py` with my usual config (`-j 60`).

ACKs for top commit:
  willcl-ark:
    ACK 1647a11

Tree-SHA512: 529e0da4bc51f12c78a40d6d70b3a492b97723c96a3526148c46943d923c118737b32d2aec23d246392e50ab48013891ef19fe6205bf538b61b70d4f16a203eb
2022-12-19 10:14:35 +01:00
MarcoFalke
8b05f13ddb Merge bitcoin/bitcoin#26717: test: Improve check-doc.py pattern
2b77a33e5b test: Improve `check-doc.py` pattern (Hennadii Stepanov)

Pull request description:

  On master (cb32328d1b):
  ```
  $ ./test/lint/check-doc.py
  Args used        : 158
  Args documented  : 219
  Args undocumented: 0
  set()
  Args unknown     : 61
  {'-stopatheight', '-maxtipage', '-maxreceivebuffer', '-txconfirmtarget', '-maxconnections', '-maxsigcachesize', '-peertimeout', '-limitancestorsize', '-output-csv', '-blockmaxweight', '-par', '-rpcclienttimeout', '-dbcrashratio', '-zmqpubsequence', '-zmqpubhashtxhwm', '-zmqpubrawblock', '-dbbatchsize', '-zmqpubrawtxhwm', '-includeconf', '-checkblocks', '-limitancestorcount', '-zmqpubrawtx', '-checklevel', '-checkmempool', '-rpcthreads', '-rpcworkqueue', '-zmqpubsequencehwm', '-zmqpubrawblockhwm', '-rpcservertimeout', '-testnet', '-zmqpubhashtx', '-signet', '-rpcwaittimeout', '-limitdescendantcount', '-output-json', '-maxmempool', '-mocktime', '-datacarriersize', '-rpcport', '-dbcache', '-zmqpubhashblockhwm', '-mempoolexpiry', '-settings', '-min-time', '-maxtimeadjustment', '-bytespersigop', '-blockversion', '-limitdescendantsize', '-maxorphantx', '-rpccookiefile', '-rpcserialversion', '-bantime', '-blockreconstructionextratxn', '-checkaddrman', '-debuglogfile', '-pid', '-dblogsize', '-timeout', '-zmqpubhashblock', '-maxsendbuffer', '-regtest'}
  ```

  With this PR:
  ```
  $ ./test/lint/check-doc.py
  Args used        : 208
  Args documented  : 219
  Args undocumented: 0
  set()
  Args unknown     : 11
  {'-zmqpubrawblock', '-zmqpubhashblockhwm', '-zmqpubsequencehwm', '-zmqpubrawtx', '-zmqpubhashblock', '-zmqpubhashtx', '-includeconf', '-zmqpubhashtxhwm', '-zmqpubrawblockhwm', '-zmqpubrawtxhwm', '-zmqpubsequence'}
  ```

ACKs for top commit:
  vincenzopalazzo:
    ACK 2b77a33e5b

Tree-SHA512: 6cf4ccc4e8319aad8006ae915f0d25637ac12974fbc1f81808f26b72fbe2649e2b6ff993bc2c1894f81bd6756bff77491b3d56382c034a84fd50325a3c807d8b
2022-12-19 09:22:09 +01:00
brunoerg
ec63a4892e test: call keypoolrefill with private keys disabled should throw an error 2022-12-18 22:00:38 -03:00
Andrew Chow
bb5ea1d9a9 qt: Load PSBTs using istreambuf_iterator rather than istream_iterator
istream_iterator eats whitespace charactesr which causes parsing
failures for PSBTs that contain the bytes corresponding to those
characters.
2022-12-18 13:20:20 -05:00
brunoerg
a4defcdd57 test, lint: add crypted to ignore-words 2022-12-18 11:46:32 -03:00
Hennadii Stepanov
2b77a33e5b test: Improve check-doc.py pattern 2022-12-17 16:25:17 +00:00
MarcoFalke
cb32328d1b Merge bitcoin/bitcoin#26710: refactor: Fix performance-for-range-copy in headers
48033d43dc clang-tidy: Fix `performance-for-range-copy` in headers (Hennadii Stepanov)

Pull request description:

  Split from bitcoin/bitcoin#26705 as was requested in https://github.com/bitcoin/bitcoin/pull/26705#issuecomment-1353293405.

  To test this PR, consider applying a diff as follows:
  ```diff
  --- a/src/.clang-tidy
  +++ b/src/.clang-tidy
  @@ -12,17 +12,9 @@ readability-redundant-declaration,
   readability-redundant-string-init,
   '
   WarningsAsErrors: '
  -bugprone-argument-comment,
  -bugprone-use-after-move,
  -misc-unused-using-decls,
  -modernize-use-default-member-init,
  -modernize-use-nullptr,
   performance-for-range-copy,
  -performance-move-const-arg,
  -performance-unnecessary-copy-initialization,
  -readability-redundant-declaration,
  -readability-redundant-string-init,
   '
   CheckOptions:
    - key: performance-move-const-arg.CheckTriviallyCopyableMove
      value: false
  +HeaderFilterRegex: '.'
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 48033d43dc

Tree-SHA512: eaf7a0e9b4fdc4ce788f78e5675632f3c278fc24bee2434874cbabc3e25ad7059b0c53ab7834908e901872d5afee08acba860542b03454c09fe129be6ad03f09
2022-12-17 12:52:41 +01:00
MarcoFalke
6c01323d9d Merge bitcoin/bitcoin#26708: clang-tidy: Fix modernize-use-nullptr in headers
adb7dba9de clang-tidy: Fix `modernize-use-nullptr` in headers (Hennadii Stepanov)

Pull request description:

  Split from bitcoin/bitcoin#26705 as was requested in https://github.com/bitcoin/bitcoin/pull/26705#issuecomment-1353293405.

  To test this PR, consider applying a diff as follows:
  ```diff
  --- a/src/.clang-tidy
  +++ b/src/.clang-tidy
  @@ -12,17 +12,9 @@ readability-redundant-declaration,
   readability-redundant-string-init,
   '
   WarningsAsErrors: '
  -bugprone-argument-comment,
  -bugprone-use-after-move,
  -misc-unused-using-decls,
  -modernize-use-default-member-init,
   modernize-use-nullptr,
  -performance-for-range-copy,
  -performance-move-const-arg,
  -performance-unnecessary-copy-initialization,
  -readability-redundant-declaration,
  -readability-redundant-string-init,
   '
   CheckOptions:
    - key: performance-move-const-arg.CheckTriviallyCopyableMove
      value: false
  +HeaderFilterRegex: '.'
  ```

ACKs for top commit:
  john-moffett:
    ACK adb7dba9de

Tree-SHA512: 67241fb212d837157a0a26f0d59e7f30a9d270d5b0ebfeb6ad9631e460fc7fba8c9a9dcd4c0520789353f68025a9f090f40f17176472a93cce1411e6d56f930b
2022-12-17 11:55:16 +01:00
MarcoFalke
caa2240680 Merge bitcoin/bitcoin#26120: refactor: Make bitcoin-util grind_task tsan friendly
fafcc94398 Make bitcoin-util grind_task tsan friendly (MacroFake)

Pull request description:

  While there is no issue with the current code, `libtsan-12.2.1` on my machine does not seem to like it. This is understandable, because the nonce isn't protected by a mutex that the sanitizer can see (only by an atomic, which achieves the same).

  Fix this by guarding the nonce by the existing atomic bool, which tsan seems to understand.

ACKs for top commit:
  ajtowns:
    ACK fafcc94398
  hebasto:
    ACK fafcc94398, I have reviewed the code and it looks OK, I agree it can be merged. Confirming that initial bug has been fixed.

Tree-SHA512: 4e67fab5833ec7d91678b85a300368892ee9f7cd89a52cc5e15a7df65b2da813b24eaffd8362d0d8a3c8951e024041d69ebddf25101b11d0a1a62c1208ddc9a5
2022-12-17 11:46:12 +01:00
Andrew Chow
66c08e741d Merge bitcoin/bitcoin#24865: rpc: Enable wallet import on pruned nodes and add test
564b580bf0 test: Introduce MIN_BLOCKS_TO_KEEP constant (Aurèle Oulès)
71d9a7c03b test: Wallet imports on pruned nodes (Aurèle Oulès)
e6906fcf9e rpc: Enable wallet import on pruned nodes (Aurèle Oulès)

Pull request description:

  Reopens #16037

  I have rebased the PR, addressed the comments of the original PR and added a functional test.

  > Before this change importwallet fails if any block is pruned. This PR makes it possible to importwallet if all required blocks aren't pruned. This is possible because the dump format includes key timestamps.

  For reviewers:
  `python test/functional/wallet_pruning.py --nocleanup` will generate a large blockchain (~700MB) that can be used to manually test wallet imports on a pruned node. Node0 is not pruned, while node1 is.

ACKs for top commit:
  kouloumos:
    ACK 564b580bf0
  achow101:
    reACK 564b580bf0
  furszy:
    ACK 564b580
  w0xlt:
    ACK 564b580bf0

Tree-SHA512: b345a6c455fcb6581cdaa5f7a55d79e763a55cb08c81d66be5b12794985d79cd51b9b39bdcd0f7ba0a2a2643e9b2ddc49310ff03d16b430df2f74e990800eabf
2022-12-16 17:30:57 -05:00
brunoerg
7a83aa0982 test: add coverage for unparsable -maxuploadtarget 2022-12-16 14:57:19 -03:00
Andrew Chow
4aebd832a4 db: Change DatabaseCursor::Next to return status enum
Next()'s result is a tri-state - failed, more to go, complete. Replace
the way that this is returned with an enum with values FAIL, MORE, and
DONE rather than with two booleans.
2022-12-16 12:35:54 -05:00
Andrew Chow
d79e8dcf29 wallet: Have cursor users use DatabaseCursor directly
Instead of having the DatabaseBatch manage the cursor, having the
consumer handle it directly
2022-12-16 12:35:54 -05:00
fanquake
7386da7a0b Merge bitcoin/bitcoin#26709: doc: add 23.1 release notes
a2724808ab doc: add 23.1 release notes (fanquake)

Pull request description:

  Same as past releases / https://github.com/bitcoin/bitcoin/pull/26524 etc.

ACKs for top commit:
  stickies-v:
    ACK a2724808ab

Tree-SHA512: e9f7ad72c23c8621e8a98ffa0dc0d08ebe30ad0bc8d23e25fabda5b1a9318ff74c65344821267c6af5a8d94c26c775ce83a67cbe0c4922eac07a4319fd94eb49
2022-12-16 14:13:35 +00:00
Sebastian Falbesoner
d6fc1d6a33 test: add coverage for dust mempool policy (-dustrelayfee setting) 2022-12-16 15:03:22 +01:00
Hennadii Stepanov
7b7cd11244 clang-tidy, qt: Force checks for headers in src/qt 2022-12-16 11:58:46 +00:00
Hennadii Stepanov
69eacf2c5e clang-tidy, qt: Fix modernize-use-default-member-init in headers
See https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-default-member-init.html
2022-12-16 11:58:38 +00:00
Hennadii Stepanov
48033d43dc clang-tidy: Fix performance-for-range-copy in headers
See https://clang.llvm.org/extra/clang-tidy/checks/performance/for-range-copy.html
2022-12-16 10:53:55 +00:00
fanquake
a2724808ab doc: add 23.1 release notes 2022-12-16 09:43:56 +00:00
MacroFake
fafcc94398 Make bitcoin-util grind_task tsan friendly
This does not change behavior of the bitcoin-util binary.
2022-12-16 09:56:06 +01:00
MarcoFalke
5055d07edf Merge bitcoin-core/gui#685: clang-tidy: Fix readability-redundant-string-init in headers
c39619eeb4 clang-tidy: Fix `readability-redundant-string-init` in headers (Hennadii Stepanov)

Pull request description:

  Split from bitcoin/bitcoin#26705 as was requested in https://github.com/bitcoin/bitcoin/pull/26705#issuecomment-1353293405.

  To test this PR, consider applying a diff as follows:
  ```diff
  --- a/src/.clang-tidy
  +++ b/src/.clang-tidy
  @@ -12,17 +12,9 @@ readability-redundant-declaration,
   readability-redundant-string-init,
   '
   WarningsAsErrors: '
  -bugprone-argument-comment,
  -bugprone-use-after-move,
  -misc-unused-using-decls,
  -modernize-use-default-member-init,
  -modernize-use-nullptr,
  -performance-for-range-copy,
  -performance-move-const-arg,
  -performance-unnecessary-copy-initialization,
  -readability-redundant-declaration,
   readability-redundant-string-init,
   '
   CheckOptions:
    - key: performance-move-const-arg.CheckTriviallyCopyableMove
      value: false
  +HeaderFilterRegex: '.'
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK c39619eeb4

Tree-SHA512: d7b61be17737f68b8bb40b084cf03f89eae86f4951da2aa000fde0c5245491a01dbb83e5d6e870c6bab4de2dbb5c0eb0dd6613da71592b3a27cf2000a45eaeeb
2022-12-16 09:15:37 +01:00
Hennadii Stepanov
c39619eeb4 clang-tidy: Fix readability-redundant-string-init in headers
See https://clang.llvm.org/extra/clang-tidy/checks/readability/redundant-string-init.html
2022-12-15 21:24:14 +00:00
Hennadii Stepanov
adb7dba9de clang-tidy: Fix modernize-use-nullptr in headers
https://clang.llvm.org/extra/clang-tidy/checks/modernize/use-nullptr.html
2022-12-15 20:58:19 +00:00
James O'Beirne
36c201feb7 remove CBlockIndex copy construction
Copy construction of CBlockIndex objects is a footgun because of the
wide use of equality-by-pointer comparison in the code base. There are
also potential lifetime confusions of using copied instances, since
there are recursive pointer references (e.g. pprev).

We can't just delete the copy constructors because they are used for
derived classes (CDiskBlockIndex), so we mark them protected.

Delete move constructors and declare the destructor to satisfy the
"rule of 5."
2022-12-15 14:52:28 -05:00
John Moffett
f9ce0eadf4 For feebump, ignore abandoned descendant spends
To be eligible for fee-bumping, a transaction must not have any
of its outputs (eg - change) spent in other unconfirmed transactions
in the wallet. However, this check should not apply to abandoned
transactions.

A new test case is added to cover this case.
2022-12-15 14:38:25 -05:00
furszy
3a4f8bc242 bench: add benchmark for wallet 'AvailableCoins' function. 2022-12-15 15:42:39 -03:00
MarcoFalke
03708dac0a Merge bitcoin/bitcoin#26704: doc: add 22.1 release notes
062e4e9fe9 doc: add 22.1 release notes (fanquake)

Pull request description:

  Same as past releases / #26524 etc.

Top commit has no ACKs.

Tree-SHA512: e41b1eaff1aacd89260f070650044629de5673020e0e70bdceb0749981403aad380e5595c494fa5ebaaa7c87e0ebea0def5f5bbd433a4b3b810e40c2de6dc448
2022-12-15 13:33:53 +01:00
MarcoFalke
9a72119e7e Merge bitcoin/bitcoin#26651: test: Avoid intermittent timeout in feature_assumevalid.py
fa34e5f3d3 test: Avoid intermittent timeout in feature_assumevalid.py (MarcoFalke)

Pull request description:

  Currently the test will spin up p2p connections in the beginning, then announce the headers to all nodes, but only send the blocks sequentially. This takes a long time, so when getting to the last node, it will have already timed out, while node1 is busy eating blocks. Example:

  ```
   node2 2022-12-06T19:31:35.419291Z [msghand] [net_processing.cpp:5783] [SendMessages] [net] Requesting block 2cfdb317b3b901f79e2d4f96339d0c0dffd8ef2685d324f62ab0e2fa3402430e (1) peer=0
   node2 2022-12-06T19:31:35.424784Z [msghand] [net.cpp:2776] [PushMessage] [net] sending getdata (577 bytes) peer=0
  [...]
   node2 2022-12-06T19:41:35.423257Z [msghand] [net_processing.cpp:5729] [SendMessages] Timeout downloading block 2cfdb317b3b901f79e2d4f96339d0c0dffd8ef2685d324f62ab0e2fa3402430e from peer=0, disconnecting
   node1 2022-12-06T19:41:35.438706Z [msghand] [net_processing.cpp:5783] [SendMessages] [net] Requesting block 6575919043306ed309014d0bd79814b4fab8afaa281e026d8cc3a1c4c2336fbc (1748) peer=0
   node2 2022-12-06T19:41:35.521253Z [net] [net.cpp:573] [CloseSocketDisconnect] [net] disconnecting peer=0
   node2 2022-12-06T19:41:35.630417Z [net] [net_processing.cpp:1532] [FinalizeNode] [net] Cleared nodestate for peer=0
  ```

  Fix this by only spinning up the p2p connection right before they are needed.

ACKs for top commit:
  jamesob:
    ACK fa34e5f3d3 ([`jamesob/ackr/26651.1.MarcoFalke.test_avoid_intermittent`](https://github.com/jamesob/bitcoin/tree/ackr/26651.1.MarcoFalke.test_avoid_intermittent))

Tree-SHA512: 7a1b114c07dfa30237c8cd8637dd6646c5c2dc2530c9de61db231738fddc800b620c31dc664237e33d35e951cf161f015fda593162efc9d85c5f68c6e37217d4
2022-12-15 13:09:06 +01:00
fanquake
062e4e9fe9 doc: add 22.1 release notes
Same as past releases / #26524 etc.
2022-12-15 10:42:06 +00:00
Aurèle Oulès
564b580bf0 test: Introduce MIN_BLOCKS_TO_KEEP constant 2022-12-15 09:53:51 +01:00
Aurèle Oulès
71d9a7c03b test: Wallet imports on pruned nodes
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Andreas Kouloumos <kouloumosa@gmail.com>
2022-12-15 09:53:50 +01:00
Sebastian Falbesoner
f496528556 walletdb: refactor: drop unused FindWalletTx parameter and rename
Since commit 3340dbadd3 ("Remove
-zapwallettxes"), the `FindWalletTx` helper is only needed to read tx
hashes, so drop the other parameter and rename the method accordingly.
2022-12-15 00:58:12 +01:00
Andrew Chow
ba47a4ba97 Merge bitcoin/bitcoin#26668: wallet: if only have one output type, don't perform "mixed" coin selection
89c1491d35 wallet: if only have one output type, don't perform "mixed" coin selection (furszy)

Pull request description:

  For wallets that only have one output type, we are currently performing the same
  selection process over the same coins twice.

  The "mixed coin selection" doesn't add any value to the result
  (there is nothing to mix if the available coins struct has only one type).

ACKs for top commit:
  achow101:
    ACK 89c1491d35
  john-moffett:
    ACK 89c1491d35
  kristapsk:
    cr utACK 89c1491d35

Tree-SHA512: 672eaeed3ba911d13fa61a46f719c8fe1ebe4d2dc7d723040e71937c693659411bc99cdbd9f0014e836b70eebeff1b8ca861f4d81d39e6f79f437364a526edbe
2022-12-14 16:16:03 -05:00
Andrew Chow
7a198bba0a wallet: Introduce DatabaseCursor RAII class for managing cursor
Instead of having DatabaseBatch deal with opening and closing database
cursors, have a separate RAII class that deals with those.

For now, DatabaseBatch manages DatabaseCursor, but this will change
later.
2022-12-14 12:41:41 -05:00
Hennadii Stepanov
e8b4201ba2 build: Update Boost to 1.81.0 in depends
This update includes https://github.com/boostorg/process/pull/264
2022-12-14 15:25:02 +00:00
furszy
9622fe64b8 test: move coins result test to wallet_tests.cpp
The class `AvailableCoinsTestingSetup` inside `availablecoins_tests.cpp` is a plain copy
of `ListCoinsTestingSetup` that is inside wallet_tests.cpp.
2022-12-14 11:16:01 -03:00
furszy
f69347d058 test: extend and simplify availablecoins_tests
Clean redundant code and add coverage for 'AvailableCoins' incremental result.
2022-12-14 11:16:01 -03:00
MarcoFalke
678889e6c6 Merge bitcoin/bitcoin#26689: test: add add_wallet_options to TestShell
bcb7123406 test: add add_wallet_options to TestShell (josibake)

Pull request description:

  following 555519d082, `TestShell` now always runs with `-disablewallet`. simple fix is to add `add_wallet_options` to `add_options`; looks like testshell was overlooked when adding in the `add_wallet_options` call to the functional tests in #26480

ACKs for top commit:
  amitiuttarwar:
    ACK bcb7123406

Tree-SHA512: db554a8b3c8ff5bd10cab9592b316035a92f86a0a0ae8ff914de9687bbbb6fc2235bdf82c4cd40e4071782f8b6edf91aad372e82ed3b826c9d8ae39dbe3dbf57
2022-12-14 09:16:02 +01:00
Andrew Chow
daf881de9d Merge bitcoin/bitcoin#23319: rpc: Return fee and prevout (utxos) to getrawtransaction
f86697163e rpc: Return fee and prevout(s) to getrawtransaction (Douglas Chimento)

Pull request description:

  Add fee response in BTC to getrawtransaction #23264

  ### For Reviewers

  * Verbose arg is now an int
  * Verbose = 2 includes a `fee` field and `prevout`
  * [./test/functional/rpc_rawtransaction.py](./test/functional/rpc_rawtransaction.py) contains a new test to validate fields of new verbosity 2 (not the values)

  ```
  bitcoin-cli -chain=test getrawtransaction 9ae533f7da9be4a34997db78343a8d8d6d6186b6bba3959e56f416a5c70e7de4 2 000000000000001d442e556146d5f2841d85150c200e8d8b8a4b5005b13878f6
  ```
  ```
    "in_active_chain": true,
    "txid": "9ae533f7da9be4a34997db78343a8d8d6d6186b6bba3959e56f416a5c70e7de4",
    "hash": "7f23e3f3a0a256ddea1d35ffd43e9afdd67cc68389ef1a804bb20c76abd6863e",
   ....
    "vin": [
      {
        "txid": "23fc75d6d74f6f97e225839af69ff36a612fe04db58a4414ec4828d1749a05a0",
        "vout": 0,
        "scriptSig": {
          "asm": "",
          "hex": ""
        },
        "prevout": {
          "generated": false,
          "height": 2099486,
          "value": 0.00017764,
          "scriptPubKey": {
            "asm": "0 7846ce1ced3253d8bd43008db2ca364cc722f5a2",
            "hex": "00147846ce1ced3253d8bd43008db2ca364cc722f5a2",
            "address": "tb1q0prvu88dxffa302rqzxm9j3kfnrj9adzk49mlp",
            "type": "witness_v0_keyhash"
          }
        },
        "sequence": 4294967295
      },
  ...
   "fee": 0.00000762
  }
  ```

ACKs for top commit:
  achow101:
    ACK f86697163e
  aureleoules:
    ACK f86697163e
  hernanmarino:
    re ACK f86697163e
  pablomartin4btc:
    re-tACK f86697163e

Tree-SHA512: 591fdc285d74fa7803e04ad01c7b70bc20fac6b1369e7bd5b8e2cde9b750ea52d6c70d79225b74bef4f4bbc0fb960877778017184e146119da4a55f9593d1224
2022-12-13 18:09:09 -05:00
Hennadii Stepanov
ffa32ab108 Merge bitcoin-core/gui#682: Don't directly delete abandoned txes from GUI
e75d227632 Minor fix: Don't directly delete abandoned txes (John Moffett)

Pull request description:

  This fully closes bitcoin/bitcoin#12179. Currently, when a user abandons a transaction by clicking "Abandon Transaction" in the context menu, a call is made to remove it from the GUI view:

  `model->getTransactionTableModel()->updateTransaction(hashQStr, CT_UPDATED, false);`

  (The `false` parameter is for `bool showTransaction`)

  This behavior is probably unwanted, as the transaction is not actually removed from the wallet and would show up again if the node is restarted.

  However, the previous line, `model->wallet().abandonTransaction(hash);`, changes the underlying model and calls `NotifyTransactionChanged(wtx.GetHash(), CT_UPDATED);`, which queues a signal that eventually calls back to `updateTransaction`, this time with `showTransaction` set to `true`. This runs on a separate thread, so it gets called *after* the 'subsequent' `updateTransaction`. The transaction gets removed from the GUI and immediately added back.

  In a nutshell, `updateTransaction` gets called twice. The first (direct) call deletes the transaction from the GUI. The second (sent via a queued signal) brings it back to the GUI. The first direct call is redundant and unwanted. Worse, if the `abandonTransaction` call fails for any reason, the transaction still gets removed from the GUI. (This is what caused bitcoin#12179. It can still be triggered if, eg., a user clicks "Abandon Transaction" the moment after a new block is found.)

  There are no conditions (to my knowledge) where an abandoned transaction should be directly removed from the GUI. If the underlying model changes, the deletion should be reflected anyway by the queued signal to `updateTransaction`.

  The behavior is borne out by the QT logs. To reproduce, send a transaction with RBF enabled, then bump the fee, then 'abandon transaction' on the first transaction. The logs will show something like this:

  ```
  2022-11-28T14:48:00Z [qt] GUI: "NotifyTransactionChanged: 2c5811484f1adec92a739a5e70b453b03eaed0f7cc0538fbd0ee1589e586b951 status= 1"
  2022-11-28T14:48:00Z [qt] GUI: "TransactionTablePriv::updateWallet: 2c5811484f1adec92a739a5e70b453b03eaed0f7cc0538fbd0ee1589e586b951 1"
  2022-11-28T14:48:00Z [qt] GUI: "    inModel=1 Index=381-382 showTransaction=0 derivedStatus=2"
  2022-11-28T14:48:00Z [qt] GUI: "TransactionTablePriv::updateWallet: 2c5811484f1adec92a739a5e70b453b03eaed0f7cc0538fbd0ee1589e586b951 1"
  2022-11-28T14:48:00Z [qt] GUI: "    inModel=0 Index=381-381 showTransaction=1 derivedStatus=0"
  ```

  Notice the duplicate `updateWallet` calls with different `showTransaction` values.

ACKs for top commit:
  hebasto:
    ACK e75d227632
  jarolrod:
    tACK e75d227632

Tree-SHA512: 00f150f747c2ee1605af861a21d5c3b9773a4a9985e8dab62e48bd32885b1bfa4e8cbf805ad61af77aec9d3ccefaed3f4311a29086aa8c22d55d5326ba68ece6
2022-12-13 21:51:06 +00:00
Andrew Chow
3784009534 wallet: Skip rescanning if wallet is more recent than tip
If a wallet has key birthdates that are more recent than the currrent
chain tip, or a bestblock height higher than the current tip, we should
not attempt to rescan as there is nothing to scan for.
2022-12-13 15:55:35 -05:00
Pieter Wuille
2022917223 Add secp256k1_selftest call 2022-12-13 15:08:26 -05:00
Pieter Wuille
3bfca788b0 Remove explicit enabling of default modules 2022-12-13 15:08:26 -05:00
Pieter Wuille
4462cb0498 Adapt to libsecp256k1 API changes
* Use SECP256K1_CONTEXT_NONE when creating signing context, as
  SECP256K1_CONTEXT_SIGN is deprecated and unnecessary.
* Use secp256k1_static_context where applicable.
2022-12-13 15:08:24 -05:00
Andrew Chow
8f3021155e Merge bitcoin/bitcoin#26643: wallet: Move fee underpayment check to after all fee has been set
798430d127 wallet: Sanity check fee paid cannot be negative (Andrew Chow)
c1a84f108e wallet: Move fee underpayment check to after fee setting (Andrew Chow)
e5daf976d5 wallet: Rename nFeeRet in CreateTransactionInternal to current_fee (Andrew Chow)

Pull request description:

  Currently the fee underpayment check occurs right after we calculate what the transaction's fee should be. However the fee paid by the transaction at that time does not always match. Notably, when doing SFFO, the fee paid at that time will almost always be less than the fee required, which then required having a bypass of the underpayment check that results in SFFO payments going through when they should not.

  This PR moves the underpayment check to after fees have been finalized so that we always check whether the fee is being underpaid. This removes the exception for SFFO and unifies this behavior for both SFFO and non-SFFO txs.

ACKs for top commit:
  S3RK:
    Code review ACK 798430d127
  furszy:
    Code review ACK 798430d
  glozow:
    utACK 798430d127, code looks correct to me

Tree-SHA512: 720e8a3dbdc9937b12ee7881eb2ad58332c9584520da87ef3080e6f9d6220ce8d3bd8b9317b4877e56a229113437340852976db8f64df0d5cc50723fa04b02f0
2022-12-13 14:19:00 -05:00
MarcoFalke
a4baf3f177 Merge bitcoin/bitcoin#26628: RPC: Reject RPC requests with same named parameter specified multiple times
8c3ff7d52a test: Suggested cleanups for rpc_namedparams test (Ryan Ofsky)
d1ca563825 bitcoin-cli: Make it an error to specify the "args" parameter two different ways (Ryan Ofsky)
6bd1d20b8c rpc: Make it an error server-side to specify same named parameter multiple times (Ryan Ofsky)
e2c3b18e67 test: Add RPC tests for same named parameter specified more than once (Ryan Ofsky)

Pull request description:

  Make the JSON-RPC server reject requests with the same named parameter specified multiple times, instead of silently overwriting earlier parameter values with later ones.

  Generally JSON keys are supposed to unique, and their order isn't supposed to be significant, so having the server silently discard duplicate keys is error-prone. Most likely if an RPC client is sending a request with duplicate keys it means something is wrong with the request and there should be an error.

  After this change, named parameters are still allowed to specified multiple times on the `bitcoin-cli` command line, since `bitcoin-cli` automatically replaces earlier values with later values before sending the JSON-RPC request. This makes sense, since it's not unusual for the order of command line options to be significant or for later command line options to override earlier ones.

ACKs for top commit:
  MarcoFalke:
    review ACK 8c3ff7d52a 🗂
  kristapsk:
    ACK 8c3ff7d52a
  stickies-v:
    ACK 8c3ff7d52

Tree-SHA512: 2d1357dcc2c171da287aeefc7b333ba4e67babfb64fc14d7fa0940256e18010a2a65054f3bf7fa1571b144d2de8b82d53076111b5f97ba29320cfe84b6ed986f
2022-12-13 17:57:23 +01:00
stickies-v
47c4b1f52a mempool: log/halt when CalculateMemPoolAncestors fails unexpectedly
When CalculateMemPoolAncestors fails unexpectedly (e.g. it exceeds
ancestor/descendant limits even though we expect no limits to be applied),
add an error log entry for increased visibility. For debug builds,
the application will even halt completely since this is not supposed
to happen.
2022-12-13 15:44:45 +00:00
stickies-v
5481f65849 mempool: add AssumeCalculateMemPoolAncestors helper function
There are quite a few places that assume CalculateMemPoolAncestors
will return a value without raising an error. This helper function
adds logging (and Assume for debug builds) that ensures robustness
but increases visibility in case of unexpected failures
2022-12-13 15:43:17 +00:00
stickies-v
f911bdfff9 mempool: use util::Result for CalculateMemPoolAncestors
Avoid using setAncestors outparameter, simplify function signatures
and avoid creating unused dummy strings.
2022-12-13 15:42:49 +00:00
MarcoFalke
fa818e103c txmempool: Remove unused clear() member function 2022-12-13 12:57:17 +01:00
fanquake
968f03e65c Merge bitcoin/bitcoin#26477: validation: fix broken maxtipage comparison
e4be0e9b06 test: add -maxtipage test for the maximum allowable value (James O'Beirne)
a451e832b4 fix: validation: cast now() to seconds for maxtipage comparison (James O'Beirne)

Pull request description:

  Since faf44876db, the maxtipage comparison in IsInitialBlockDownload() has been broken, since the NodeClock::now() time_point is in the system's native denomination (nanoseconds).

  Without this patch, specifying the maximum allowable -maxtipage (9223372036854775807) results in a SIGABRT crash:

  ```
  % gdb --args ./src/bitcoind -maxtipage=9223372036854775207 -minimumchainwork=0x00 -stopatheight=30000
  ...
  2022-11-09T15:55:17Z [dnsseed] dnsseed thread exit
  [Thread 0x7fff937fe640 (LWP 69883) exited]

  Thread 29 "b-msghand" received signal SIGABRT, Aborted.
  [Switching to Thread 0x7fff91ffb640 (LWP 69886)]
  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
  44      ./nptl/pthread_kill.c: No such file or directory.
  (gdb) bt
  #0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at ./nptl/pthread_kill.c:44
  #1  0x00007ffff768989f in __pthread_kill_internal (signo=6, threadid=<optimized out>) at ./nptl/pthread_kill.c:78
  #2  0x00007ffff763da52 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
  #3  0x00007ffff7628469 in __GI_abort () at ./stdlib/abort.c:79
  #4  0x00007ffff7cf79a4 in __mulvdi3 () from /lib/x86_64-linux-gnu/libgcc_s.so.1
  #5  0x00005555558d13ab in std::chrono::__duration_cast_impl<std::chrono::duration<long, std::ratio<1l, 1000000000l> >, std::ratio<1000000000l, 1l>, long, false, true>::__cast<long, std::ratio<1l, 1l> > (__d=...) at /usr/include/c++/12/bits/chrono.h:521
  #6  std::chrono::duration_cast<std::chrono::duration<long, std::ratio<1l, 1000000000l> >, long, std::ratio<1l, 1l> > (__d=...)
      at /usr/include/c++/12/bits/chrono.h:260
  #7  std::chrono::duration<long, std::ratio<1l, 1000000000l> >::duration<long, std::ratio<1l, 1l>, void> (__d=..., this=<optimized out>)
      at /usr/include/c++/12/bits/chrono.h:514
  #8  std::chrono::operator-<long, std::ratio<1l, 1000000000l>, long, std::ratio<1l, 1l> > (__rhs=..., __lhs=...)
      at /usr/include/c++/12/bits/chrono.h:650
  #9  std::chrono::operator-<NodeClock, std::chrono::duration<long, std::ratio<1l, 1000000000l> >, long, std::ratio<1l, 1l> > (__rhs=...,
      __lhs=...) at /usr/include/c++/12/bits/chrono.h:1020
  #10 Chainstate::IsInitialBlockDownload (this=0x555556071940) at ./src/validation.cpp:1545
  #11 0x00005555556efd1e in operator() (__closure=<optimized out>) at ./src/net_processing.cpp:3369
  #12 (anonymous namespace)::PeerManagerImpl::ProcessMessage (this=0x555556219be0, pfrom=..., msg_type=..., vRecv=..., time_received=...,
      interruptMsgProc=...) at ./src/net_processing.cpp:3369
  #13 0x00005555556f75cc in (anonymous namespace)::PeerManagerImpl::ProcessMessages (this=0x555556219be0, pfrom=<optimized out>,
      interruptMsgProc=std::atomic<bool> = { false }) at ./src/net_processing.cpp:4985
  #14 0x00005555556a83c9 in CConnman::ThreadMessageHandler (this=0x5555560ebc70) at ./src/net.cpp:2014
  #15 0x0000555555c4d5d6 in std::function<void ()>::operator()() const (this=0x7fff91ffadb0) at /usr/include/c++/12/bits/std_function.h:591
  #16 util::TraceThread(std::basic_string_view<char, std::char_traits<char> >, std::function<void ()>) (
      thread_name="0\255\377\221\377\177\000\000\v\000\000\000\000\000\000\000TraceThread\000\000\000\000\000P\255\377\221\377\177\000\000\017\000\000\000\000\000\000\000util/thread.cpp\000\000\000\000\000\000\000\000\000\000ihB鵿6\000\000\000\000\000\000\000\000\260\255\377\221\377\177\000\000\277\211\321UUU\000\000p\324\304UUU\000\000\002\000\000\000\000\000\000\000\240xh\367\377\177\000\000\000\000\000\000\000\000\000\000]\340iUUU\000\000p\274\016VUU\000\000\000\000\000\000\000\000\000\000\300\303iUUU\000\000p\206jUUU", '\000' <repeats 11 times>, "ihB鵿6\200\251!VUU\000\000"..., thread_func=...) at util/thread.cpp:21
  #17 0x000055555569e05d in std::__invoke_impl<void, void (*)(std::basic_string_view<char>, std::function<void()>), char const*, CConnman::Start(CScheduler&, const Options&)::<lambda()> > (__f=<optimized out>) at /usr/include/c++/12/bits/invoke.h:61
  #18 std::__invoke<void (*)(std::basic_string_view<char>, std::function<void()>), char const*, CConnman::Start(CScheduler&, const Options&)::<lambda()> > (__fn=<optimized out>) at /usr/include/c++/12/bits/invoke.h:96
  #19 std::thread::_Invoker<std::tuple<void (*)(std::basic_string_view<char, std::char_traits<char> >, std::function<void()>), char const*, CConnman::Start(CScheduler&, const Options&)::<lambda()> > >::_M_invoke<0, 1, 2> (this=<optimized out>) at /usr/include/c++/12/bits/std_thread.h:252
  #20 std::thread::_Invoker<std::tuple<void (*)(std::basic_string_view<char, std::char_traits<char> >, std::function<void()>), char const*, CConnman::Start(CScheduler&, const Options&)::<lambda()> > >::operator() (this=<optimized out>) at /usr/include/c++/12/bits/std_thread.h:259
  #21 std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)(std::basic_string_view<char, std::char_traits<char> >, std::function<void()>), char const*, CConnman::Start(CScheduler&, const Options&)::<lambda()> > > >::_M_run(void) (this=<optimized out>)
      at /usr/include/c++/12/bits/std_thread.h:210
  #22 0x00007ffff7ad43d3 in ?? () from /lib/x86_64-linux-gnu/libstdc++.so.6
  #23 0x00007ffff7687b27 in start_thread (arg=<optimized out>) at ./nptl/pthread_create.c:435
  #24 0x00007ffff770a78c in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
  (gdb)
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK e4be0e9b06 🏽

Tree-SHA512: d892d6264a284d952a68a8631a6301277373b8df939dafd9e2652f2f22ab60712cde63b90c27c67ea2d05f02443452e3e4e1b9f25479bfaca00d4c4de13b9fbd
2022-12-13 10:07:37 +00:00
Pieter Wuille
9d47e7b71b Squashed 'src/secp256k1/' changes from 44c2452fd3..21ffe4b22a
21ffe4b22a Merge bitcoin-core/secp256k1#1055: Prepare initial release
e025ccdf74 release: prepare for initial release 0.2.0
6d1784a2e2 build: add missing files to EXTRA_DIST
8c949f56da Merge bitcoin-core/secp256k1#1173: Don't use compute credits for now
13bf1b6b32 changelog: make order of change types match keepachangelog.com
b1f992a552 doc: improve release process
7e5b22684f Don't use compute credits for now
ad39e2dc41 build: change package version to 0.1.0-dev
5c789dcd73 Merge bitcoin-core/secp256k1#1168: Replace deprecated context flags with NONE in benchmarks and tests
d6dc0f4ae3 tests: Switch to NONE contexts in module tests
0c8a5caddd tests: Switch to NONE contexts in tests.c
86540e9e1f tests: add test for deprecated flags and rm them from run_context
caa0ad631e group: add gej_eq_var
37ba744f5b tests: Switch to NONE contexts in exhaustive and ctime tests
8d7a9a8eda benchmarks: Switch to NONE contexts
90618e9263 doc: move CHANGELOG from doc/ to root directory
e3f84777eb Merge bitcoin-core/secp256k1#1126: API cleanup with respect to contexts
4386a2306c examples: Switch to NONE contexts
7289b51d31 docs: Use doxygen style if and only if comment is user-facing
e7d0185c90 docs: Get rid of "initialized for signing" terminology
06126364ad docs: Tidy and improve docs about contexts and randomization
e02d6862bd selftest: Expose in public API
e383fbfa66 selftest: Rename internal function to make name available for API
d2c6d48de3 tests: Use new name of static context
53796d2b24 contexts: Rename static context
72fedf8a6c docs: Improve docs for static context
316ac7625a contexts: Deprecate all context flags except SECP256K1_CONTEXT_NONE
477f02c4de Merge bitcoin-core/secp256k1#1165: gitignore: Add *.sage.py files autogenerated by sage [skip ci]
092be61c5e gitignore: Add *.sage.py files autogenerated by sage
1a553ee8be docs: Change signature "validation" to "verification"
ee7341fbac docs: Never require a verification context
751c4354d5 Merge bitcoin-core/secp256k1#1152: Update macOS image for CI
2286f80902 Merge bitcoin-core/secp256k1#993: Enable non-experimental modules by default
e40fd277b7 Merge bitcoin-core/secp256k1#1156: Followups to int128_struct arithmetic
99bd335599 Make int128 overflow test use secp256k1_[ui]128_mul
a8494b02bf Use compute credits for macOS jobs
3afce0af7c Avoid signed overflow in MSVC AMR64 secp256k1_mul128
c0ae48c995 Update macOS image for CI
9b5f589d30 Heuristically decide whether to use int128_struct
63ff064d2f int128: Add test override for testing __(u)mulh on MSVC X64
f2b7e88768 Add int128 randomized tests
6138d73be4 Merge bitcoin-core/secp256k1#1155: Add MSan CI jobs
ddf2b2910e Merge bitcoin-core/secp256k1#1000: Synthetic int128 type.
86e3b38a4a Merge bitcoin-core/secp256k1#1149: Remove usage of CHECK from non-test file
00a42b91b3 Add MSan CI job
44916ae915 Merge bitcoin-core/secp256k1#1147: ci: print env to allow reproducing the job outside of CI
c2ee9175e9 Merge bitcoin-core/secp256k1#1146: ci: prevent "-v/--version: not found" irrelevant error
e13fae487e Merge bitcoin-core/secp256k1#1150: ci: always cat test_env.log
a340d9500a ci: add int128_struct tests
dceaa1f579 int128: Tidy #includes of int128.h and int128_impl.h
2914bccbc0 Simulated int128 type.
6a965b6b98 Remove usage of CHECK from non-test file
5c9f1a5c37 ci: always cat all logs_snippets
49ae843592 ci: mostly prevent "-v/--version: not found" irrelevant error
4e54c03153 ci: print env to allow reproducing the job outside of CI
a43e982bca Merge bitcoin-core/secp256k1#1144: Cleanup `.gitignore` file
f5039cb66c Cleanup `.gitignore` file
798727ae1e Revert "Add test logs to gitignore"
41e8704b48 build: Enable some modules by default
694ce8fb2d Merge bitcoin-core/secp256k1#1131: readme: Misc improvements
88b00897e7 readme: Fix line break
78f5296da4 readme: Sell "no runtime dependencies"
ef48f088ad readme: Add IRC channel
9f8a13dc8e Merge bitcoin-core/secp256k1#1128: configure: Remove pkgconfig macros again (reintroduced by mismerge)
cabe085bb4 configure: Remove pkgconfig macros again (reintroduced by mismerge)
3efeb9da21 Merge bitcoin-core/secp256k1#1121: config: Set preprocessor defaults for ECMULT_* config values
6a873cc4a9 Merge bitcoin-core/secp256k1#1122: tests: Randomize the context with probability 15/16 instead of 1/4
17065f48ae tests: Randomize the context with probability 15/16 instead of 1/4
c27ae45144 config: Remove basic-config.h
da6514a04a config: Introduce DEBUG_CONFIG macro for debug output of config
63a3565e97 Merge bitcoin-core/secp256k1#1120: ecmult_gen: Skip RNG when creating blinding if no seed is available
d0cf55e13a config: Set preprocessor defaults for ECMULT_* config values
55f8bc99dc ecmult_gen: Improve comments about projective blinding
7a86955800 ecmult_gen: Simplify code (no observable change)
4cc0b1b669 ecmult_gen: Skip RNG when creating blinding if no seed is available
af65d30cc8 Merge bitcoin-core/secp256k1#1116: build: Fix #include "..." paths to get rid of further -I arguments
40a3473a9d build: Fix #include "..." paths to get rid of further -I arguments
43756da819 Merge bitcoin-core/secp256k1#1115: Fix sepc256k1 -> secp256k1 typo in group.h
069aba8125 Fix sepc256k1 -> secp256k1 typo in group.h
accadc94df Merge bitcoin-core/secp256k1#1114: `_scratch_destroy`: move `VERIFY_CHECK` after invalid scrach space check
cd47033335 Merge bitcoin-core/secp256k1#1084: ci: Add MSVC builds
1827c9bf2b scratch_destroy: move VERIFY_CHECK after invalid scrach space check
49e2acd927 configure: Improve rationale for WERROR_CFLAGS
8dc4b03341 ci: Add a C++ job that compiles the public headers without -fpermissive
51f296a46c ci: Run persistent wineserver to speed up wine
3fb3269c22 ci: Add 32-bit MinGW64 build
9efc2e5221 ci: Add MSVC builds
2be6ba0fed configure: Convince autotools to work with MSVC's archiver lib.exe
bd81f4140a schnorrsig bench: Suppress a stupid warning in MSVC
09f3d71c51 configure: Add a few CFLAGS for MSVC
3b4f3d0d46 build: Reject C++ compilers in the preprocessor
1cc0941414 configure: Don't abort if the compiler does not define __STDC__
cca8cbbac8 configure: Output message when checking for valgrind
1a6be5745f bench: Make benchmarks compile on MSVC

git-subtree-dir: src/secp256k1
git-subtree-split: 21ffe4b22a9683cf24ae0763359e401d1284cc7a
2022-12-12 23:40:36 -05:00
Pieter Wuille
3d8a6ae283 Update secp256k1 subtree to upstream libsecp256k1 version 0.2.0 2022-12-12 23:40:36 -05:00
dergoegge
3153e7d779 [fuzz] Add HeadersSyncState target 2022-12-12 21:06:04 +00:00
dergoegge
53552affca [headerssync] Make m_commit_offset protected 2022-12-12 21:06:04 +00:00
Andrew Chow
69efbc011b Move SafeDbt out of BerkeleyBatch 2022-12-12 14:52:04 -05:00
stickies-v
66e028f739 mempool: use util::Result for CalculateAncestorsAndCheckLimits
Avoid using setAncestors outparameter, simplify function signatures
and avoid creating unused dummy strings.
2022-12-12 18:05:25 +00:00
josibake
bcb7123406 test: add add_wallet_options to TestShell
without this, testShell runs with -disablewallet
2022-12-12 17:58:15 +01:00
dergoegge
58c2bbdb55 [fuzz] Enable erlay in process_message(s) targets 2022-12-12 11:19:02 +00:00
Vasil Dimov
c9d548c91f net: remove CService::ToStringPort()
It is used only internally in `CService::ToStringAddrPort()`.
2022-12-12 11:54:23 +01:00
Vasil Dimov
fd4f0f41e9 gui: simplify OptionsDialog::updateDefaultProxyNets()
Do not create strings and compare them to check if one `addr:port`
equals another. Use `CService::operator==()` instead.

`strDefaultProxyGUI` was assigned the same value 3 times. Instead save
it in `const CService ui_proxy` at the beginning of the function.
2022-12-12 11:54:22 +01:00
Vasil Dimov
96c791dd20 net: remove CService::ToString() use ToStringAddrPort() instead
Both methods do the same thing, so simplify to having just one.

`ToString()` is too generic in this case and it is unclear what it does,
given that there are similar methods:
`ToStringAddr()` (inherited from `CNetAddr`),
`ToStringPort()` and
`ToStringAddrPort()`.
2022-12-12 11:54:20 +01:00
Vasil Dimov
944a9de08a net: remove CNetAddr::ToString() and use ToStringAddr() instead
Both methods do the same thing, so simplify to having just one.

Further, `CService` inherits `CNetAddr` and `CService::ToString()`
overrides `CNetAddr::ToString()` but the latter is not virtual which
may be confusing. Avoid such a confusion by not having non-virtual
methods with the same names in inheritance.
2022-12-12 11:48:31 +01:00
Vasil Dimov
043b9de59a scripted-diff: rename ToStringIP[Port]() to ToStringAddr[Port]()
"IP" stands for "Internet Protocol".

"IP address" is sometimes shortened to just "IP" or "address".

However, Tor or I2P addresses are not "IP addresses", nor "IPs".

Thus, use "Addr" instead of "IP" for addresses that could be IP, Tor or
I2P addresses:

`CService::ToStringIPPort()` -> `CService::ToStringAddrPort()`
`CNetAddr::ToStringIP()` -> `CNetAddr::ToStringAddr()`

-BEGIN VERIFY SCRIPT-
sed -i 's/ToStringIPPort/ToStringAddrPort/g' -- $(git grep -l ToStringIPPort src)
sed -i 's/ToStringIP/ToStringAddr/g' -- $(git grep -l ToStringIP src)
-END VERIFY SCRIPT-
2022-12-12 11:48:30 +01:00
MarcoFalke
6061eb6564 Merge bitcoin/bitcoin#26199: p2p: Don't self-advertise during version processing
956c67059c refactor, doc: Improve SetupAddressRelay call in version processing (Martin Zumsande)
3c43d9db1e p2p: Don't self-advertise during VERSION processing (Gleb Naumenko)

Pull request description:

  This picks up the last commit from #19843.

  Previously, we would prepare to self-announce to a new peer while parsing a `version` message from that peer.
  This is redundant, because we do something very similar in `MaybeSendAddr()`, which is called from `SendMessages()` after
  the version handshake is finished.

  There are a couple of differences:

  1) `MaybeSendAddr()` self-advertises to all peers we do address relay with, not just outbound ones.
  2) `GetLocalAddrForPeer()` called from `MaybeSendAddr()` makes a probabilistic decision to either advertise what they think we are or what we think we are, while `PushAddress()` on `version` deterministically only does the former if the address from the latter is unroutable.
  3) During `version` processing, we haven't received a potential sendaddrv2 message from our peer yet, so self-advertisements with addresses from addrV2-only networks would always be dropped in `PushAddress()`.

  Since it's confusing to have two slightly different mechanisms for self-advertising, and the one in `MaybeSendAddr()` is better, remove the one in `version`.

ACKs for top commit:
  stratospher:
    ACK  956c670
  naumenkogs:
    ACK 956c67059c
  amitiuttarwar:
    reACK 956c67059c

Tree-SHA512: 933d40615289f055c022170dde7bad0ac0a1d4be377538bfe9ba64375cfeb03bcd803901591f0739ac4850c880e8475a68fd1ab0330800030ab7f19e38c00274
2022-12-12 10:12:09 +01:00
Sebastian Falbesoner
741908afc1 test: previous releases: add v24.0.1 2022-12-11 03:39:57 +01:00
pasta
935acdcc79 refactor: modernize the implementation of uint256.*
- Constructors of uint256 to utilize Span instead of requiring a std::vector
- converts m_data into a std::array
- Prefers using `WIDTH` instead of `sizeof(m_data)`
- make all the things constexpr
- replace C style functions with c++ equivalents
    - memset -> std::fill
    - memcpy -> std::copy
        Note: In practice, implementations of std::copy avoid multiple assignments and use bulk copy functions such as std::memmove if the value type is TriviallyCopyable and the iterator types satisfy LegacyContiguousIterator. (https://en.cppreference.com/w/cpp/algorithm/copy)
    - memcmp -> std::memcmp
2022-12-10 14:34:44 -06:00
Hennadii Stepanov
574e50addf ci: Use CONFIG_SITE variable and --prefix option properly
This change fixes scripts when they are being run locally with a pre-
existed `$DEPENDS_DIR/$HOST` directory.
2022-12-10 19:13:20 +00:00
Yusuf Sahin HAMZA
3cc989da5c Fix checking bad dns seeds without casting
Since seed lines comes with 'str' type, comparing it directly with 0
('int' type) in the if statement was not working at all. This is fixed
by casting 'int' type to the values in the 'good' column of seeds text file.

Lines that starts with comment in the seeds text file are now ignored.

If statement for checking bad seeds are moved to the top of the 'parseline'
function as if seed is bad, there is no point of going forward from there.
2022-12-10 19:30:28 +03:00
MarcoFalke
1ea02791f3 Merge bitcoin/bitcoin#26666: refactor: Deleted unreachable code in httpserver.cpp
8f5c560e11 refactor: Refactored RequestMethodString function to follow developer notes (JoaoAJMatos)
7fd3b9491b refactor: Deleted unreachable code in httpserver.cpp (JoaoAJMatos)

Pull request description:

  Some of the code in httpserver.cpp was unreachable, and didn't follow the developer notes.
  Continuation of [#26570 ](https://github.com/bitcoin/bitcoin/pull/26570)

ACKs for top commit:
  stickies-v:
    re-ACK [8f5c560](8f5c560e11)

Tree-SHA512: ba8cf4c6dde9e2bb0ca9d63a0de86dfa37b070803dde71ac8384c261045835697a2335652cf5894511b3af8fd99f30e1cbda4e4234815b8b39538ade90fab3f9
2022-12-10 13:03:22 +01:00
fanquake
e1fb7381be Merge bitcoin/bitcoin#26672: build: Update libmultiprocess library
1986f129c6 build: Update `libmultiprocess` library (Hennadii Stepanov)

Pull request description:

  This update in particular includes:
  - https://github.com/chaincodelabs/libmultiprocess/pull/78 which is [required](https://github.com/bitcoin/bitcoin/pull/25972#issuecomment-1246812573) for https://github.com/bitcoin/bitcoin/pull/25972
  - https://github.com/chaincodelabs/libmultiprocess/pull/74
  - https://github.com/chaincodelabs/libmultiprocess/pull/70

ACKs for top commit:
  ryanofsky:
    Code review ACK 1986f129c6

Tree-SHA512: 2d9fa72df5de7d5be37d77d479702cba36c45e9fa9d9fc27e58aac0437c39e4a1054c0ac53b612cb43e830982e444d98c7d3e651d093ac68344e66f4734227bb
2022-12-10 10:40:40 +00:00
fanquake
a28fb36c47 Merge bitcoin/bitcoin#26673: univalue: Remove confusing getBool method
293849a260 univalue: Remove confusing getBool method (Ryan Ofsky)

Pull request description:

  Drop `UniValue::getBool` method because it is easy to confuse with the `UniValue::get_bool` method, and could potentially cause bugs. Unlike `get_bool`, `getBool` doesn't ensure that the value is a boolean and returns false for all integer, string, array, and object values instead of throwing an exception.

  The `getBool` method is also redundant because it is an alias for `isTrue`. There were only 5 `getBool()` calls in the codebase, so this commit replaces them with `isTrue()` or `get_bool()` calls as appropriate.

  These changes were originally made by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/26213 but were dropped to limit the scope of that PR.

ACKs for top commit:
  justinpickering:
    ACK 293849a260
  sipa:
    utACK 293849a260
  w0xlt:
    ACK 293849a260
  hebasto:
    ACK 293849a260, also verified that the removed `getBool` method is not mentioned in any docs:
  furszy:
    ACK 293849a2

Tree-SHA512: 9fbfe5e2083410f123b18703a0cc0161ecbbb4958f331c9ff808dcfcc6ad499b0e896abd16fb8ea200c53ba29878db9812ce141e59cc5e0fd174741b0bcb192d
2022-12-10 10:18:18 +00:00
fanquake
3b5fb6e77a Merge bitcoin/bitcoin#26213: rpc: Strict type checking for RPC boolean parameters
fa0153e609 refactor: Replace isTrue with get_bool (MarcoFalke)
fa2cc5d1d6 bugfix: Strict type checking for RPC boolean parameters (MarcoFalke)

Pull request description:

ACKs for top commit:
  ryanofsky:
    Code review ACK fa0153e609
  furszy:
    Code ACK fa0153e6

Tree-SHA512: b221f823c69d90c94447fd491071ff3659cfd512872b495ebc3e711f50633351974102c9ef7e50fa4a393c4131d349adea8fd41cc9d66f1f31e1f5e7a5f78757
2022-12-10 09:58:33 +00:00
Andrew Chow
798430d127 wallet: Sanity check fee paid cannot be negative
We need to check that the fee is not negative even before it is
finalized. The setting of fees for SFFO may adjust the fee to be
"correct" and no longer negative, but erroneously reduce the amounts too
far. So we need to check this condition before we do those adjustments.
2022-12-09 14:52:43 -05:00
Andrew Chow
c1a84f108e wallet: Move fee underpayment check to after fee setting
It doesn't make sense to be checking whether the fee paid is underpaying
before we've finished setting the fees. So do that after we have done
the reduction for SFFO and change adjustment for fee overpayment.
2022-12-09 14:52:26 -05:00
Andrew Chow
1647a11f39 tests: Reorder longer running tests in test_runner
The logest running tests should be at the front of the list in
test_runner.py. Since compiling with --enable-debug can have a
significant effect on test runtime, the order is based on the runtime
with that option configured.
2022-12-09 13:57:55 -05:00
Andrew Chow
ff6c9fe027 tests: Whitelist test p2p connection in rpc_packages
test_submit_child_with_parents creates a p2p connection which waits for
the node to announce transactions to it. By whitelisting this
connection, we can reduce the amount of time spent waiting for this
announcement which improves the test runtime and runtime variance.
2022-12-09 13:57:01 -05:00
Andrew Chow
8c20796aac tests: Use waitfornewblock for work queue test in interface_rpc
The work queue exceeded test in interface_rpc.py would repeatedly call
an RPC until the error was achieved. However hitting this error is
dependent on the processing speed of the computer and the optimization
level of the binary. Configurations that result in slower processing
would result in the RPC used being processed before the error could be
hit, resulting the test's runtime having a high variance.

Switching the RPC to waitfornewblock allows it to run in a much more
consistent time that is still fairly fast. waitfornewblock forces
the RPC server to allocate a thread and wait, occupying a spot in the
work queue. This is perfect for this test because the slower the RPC,
the more likely we will achieve the race condition necessary to pass the
test. Using a timeout of 500 ms appears to work reliably without causing
the test to take too long.
2022-12-09 13:57:01 -05:00
Andrew Chow
6c872d5e65 tests: Initialize sigops draining script with bytes in feature_taproot
The sigops draining script in feature_taproot's block_submit was
initialized with a list that would end up always being iterated by
CScript's constructor. Since this list is very large, a lot of time
would be wasted. By creating and passing a bytes object initialized from
that list, we can avoid this iteration and dramatically improve the
runtime of feature_taproot.
2022-12-09 13:57:01 -05:00
Andrew Chow
544cbf776c tests: Use batched RPC in feature_fee_estimation
feature_fee_estimation has a lot of loops that hit the RPC many times in
succession in order to setup scenarios. Using batched requests for these
can reduce the test's runtime without effecting the test's behavior.
2022-12-09 13:57:01 -05:00
Andrew Chow
4ad7272f8b tests: reduce number of generated blocks for wallet_import_rescan
Generating blocks is slow, especially when --enable-debug. There is no
need to generate a new block for each transaction, so avoid doing that
to improve this test's runtime.
2022-12-09 13:57:01 -05:00
JoaoAJMatos
8f5c560e11 refactor: Refactored RequestMethodString function to follow developer notes
Removed the default case in the switch statement in order to comply with the Developer Notes
2022-12-09 16:14:27 +00:00
JoaoAJMatos
7fd3b9491b refactor: Deleted unreachable code in httpserver.cpp
Removed all break statements from both RequestMethodString and GetRequestMethod functions as they were unreachable
2022-12-09 16:13:57 +00:00
MarcoFalke
9e229a542f Merge bitcoin/bitcoin#26601: test: Move wallet tests to wallet_*.py
fa7d71accc test: Move rpc_fundrawtransaction.py to wallet_fundrawtransaction.py (MarcoFalke)
fa933d6985 test: Move feature_backwards_compatibility.py to wallet_backwards_compatibility.py (MarcoFalke)

Pull request description:

  The tests only tests the wallet and it doesn't make sense to extend it for other stuff, so clarify that.

ACKs for top commit:
  fanquake:
    ACK fa7d71accc
  pablomartin4btc:
    re-ACK fa7d71accc

Tree-SHA512: 9dc131ed8ff119bd6d43d04ecc5c96d02f2d6fdc3d0805492774a414c0fbe6a984da7631154122a080c54ddd25fc5a5bdd52b282c918fce4932057ba72c2bf71
2022-12-09 16:34:48 +01:00
Ryan Ofsky
8c3ff7d52a test: Suggested cleanups for rpc_namedparams test
No changes in behavior, just implements review suggestions from

https://github.com/bitcoin/bitcoin/pull/19762#discussion_r1025573943
https://github.com/bitcoin/bitcoin/pull/19762#discussion_r1035955247
https://github.com/bitcoin/bitcoin/pull/26628#discussion_r1038765894
2022-12-09 10:34:28 -05:00
Hennadii Stepanov
1986f129c6 build: Update libmultiprocess library
Replacing `install` with `install-lib` and `install-bin` is not strictly
necessary just to update the library, but it takes advantage of recent
changes in the new version, and makes the build more minimal.
2022-12-09 15:26:58 +00:00
Ryan Ofsky
293849a260 univalue: Remove confusing getBool method
Drop UniValue::getBool method because it is easy to confuse with the
UniValue::get_bool method, and could potentially cause bugs. Unlike get_bool,
getBool doesn't ensure that the value is a boolean and returns false for all
integer, string, array, and object values instead of throwing an exceptions.

The getBool method is also redundant because it is an alias for isTrue. There
were only 5 getBool() calls in the codebase, so this commit replaces them with
isTrue() or get_bool() calls as appropriate.

These changes were originally made by MarcoFalke in
https://github.com/bitcoin/bitcoin/pull/26213 but were dropped to limit the
scope of that PR.

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2022-12-09 10:03:26 -05:00
Hennadii Stepanov
978852aad8 build: Fix depends build system when working with subtargets 2022-12-09 13:36:46 +00:00
MarcoFalke
fa7d71accc test: Move rpc_fundrawtransaction.py to wallet_fundrawtransaction.py 2022-12-09 11:54:29 +01:00
MarcoFalke
fa933d6985 test: Move feature_backwards_compatibility.py to wallet_backwards_compatibility.py 2022-12-09 11:54:17 +01:00
MarcoFalke
16624e6ff3 Merge bitcoin/bitcoin#26660: test: Use last release in compatibility tests
fabb24cbef test: Use last release in compatibility tests (MarcoFalke)

Pull request description:

  In compatibility tests it makes sense to always use the last release without the new feature, as it is likely more in use than any even older previous release.

ACKs for top commit:
  Sjors:
    utACK fabb24c

Tree-SHA512: beb854f4d28ba313282e1e0303abb0e09377828b138bde5a3e209337210b6b4c24855ab90a68f8789387001e4ca33b15cc37dbc9b7809929f4e7d1b69833a527
2022-12-09 09:25:52 +01:00
MarcoFalke
6d11f19cf5 Merge bitcoin/bitcoin#26658: test: Fix backwards compatibility intermittent failure
c29bff5b91 test: Fix backwards compatibility intermittent failure (Aurèle Oulès)

Pull request description:

  Fixes #24400.

  See https://github.com/bitcoin/bitcoin/issues/24400#issuecomment-1341531696 to reproduce the failure.

  As MarcoFalke suggested in https://github.com/bitcoin/bitcoin/issues/24400#issuecomment-1342282165, it can happen that the wallet is not fully flushed before being copied over to the other node. Fixed by unloading the wallet before copying the file.

ACKs for top commit:
  Sjors:
    utACK c29bff5

Tree-SHA512: ea93b859878873d07fa44ccb1c5b4c1b1014be8568dc2b9bbef02bb7a9230fefa7a71b149eb553870d3f4e986263342ea4f996fa0221098a2244f38952100471
2022-12-09 09:21:54 +01:00
furszy
89c1491d35 wallet: if only have one output type, don't perform "mixed" coin selection
there is nothing to mix.
2022-12-08 15:56:36 -03:00
fanquake
3eaf7be6ad Merge bitcoin/bitcoin#24279: build: Make $(package)_*_env available to all $(package)_*_cmds
affbf58a1e build: Move environment variables into `$(package)_config_env` (Hennadii Stepanov)
d44fcd3c97 build: Make $(package)_*_env available to all $(package)_*_cmds (Hennadii Stepanov)

Pull request description:

  On master (1e7564eca8) the depends build system, which is based on pure GNU Make, works, but it lacks robustness, and in some corner cases it fails. For example, see bitcoin/bitcoin#22552.

  Another [bug](https://github.com/bitcoin/bitcoin/issues/22719) in the depends build system has already become a problem at least two times in the past (https://github.com/bitcoin/bitcoin/pull/16883#issuecomment-683817472 and https://github.com/bitcoin/bitcoin/pull/24134). Each time the problem was solved with other means.

  The initial [solution](https://github.com/bitcoin/bitcoin/pull/19882) had some discussion. Also it was discussed on the IRC meeting in #bitcoin-core-builds channel. This PR, actually, is a resurrection of it, as the bug silently struck pretty [recently](https://github.com/bitcoin/bitcoin/pull/24134).

  The bug is well described in bitcoin/bitcoin#22719.

  Here is another, a bit simpler description, which requires only basic shell (bash, dash etc) experience.
  After creating targets by this code:1e7564eca8/depends/funcs.mk (L280) a "draft" line of recipe like `$($(1)_config_env) $(call $(1)_config_cmds, $(1))` becomes a shell command sequence `VAR1=foo VAR2=bar command1 && command2` which is supposed to be executed in a [new sub-shell](https://www.gnu.org/software/make/manual/html_node/Execution.html#Execution).

  Please note that `VAR1=foo VAR2=bar` part is visible for the first `command1` only (for details see shell docs). Example:
  ```sh
  $ VAR1="foo" VAR2="bar" echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
  begin
  $ echo $?
  1
  ```

  Using the `export` command is a trivial solution:
  ```sh
  $ export VAR1="foo" VAR2="bar"; echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
  begin
  foo
  bar
  end
  $ echo $?
  0
  ```

  As a [new sub-shell](https://www.gnu.org/software/make/manual/html_node/Execution.html#Execution) is invoked for each line of the recipe, there are no side effects of using `export`. It means this solution should not be considered invasive.

  Fixes bitcoin/bitcoin#22719.

  ---

  Also this PR removes no longer needed crutch from `qt.mk`.

ACKs for top commit:
  fanquake:
    ACK affbf58a1e

Tree-SHA512: 0ce2cf82870a7774bdf1592fac50857126ae47da902e349f1092d50109223be9d6a8efd5e92ec08c2ca775b17516482aabaf232378950ade36484a883acc177b
2022-12-08 16:41:40 +00:00
MarcoFalke
5126e625cb Merge bitcoin/bitcoin#26378: refactor: Pass reference to last header, not pointer
fa579f3063 refactor: Pass reference to last header, not pointer (MacroFake)

Pull request description:

  It is never a nullptr, otherwise an assertion would fire in UpdatePeerStateForReceivedHeaders.

  Passing a reference makes the code easier to read and less brittle.

ACKs for top commit:
  john-moffett:
    ACK fa579f3
  aureleoules:
    ACK fa579f3063

Tree-SHA512: 9725195663a31df57ae46bb7b11211cc4963a8f3d100f60332bfd4a3f3327a73ac978b3172e3007793cfc508dfc7c3a81aab57a275a6963a5ab662ce85743fd0
2022-12-08 17:04:05 +01:00
Aurèle Oulès
109cbb819d doc: Add release notes for #26618 2022-12-08 16:45:38 +01:00
Aurèle Oulès
b13902d2e4 rpc: Prevent unloading a wallet when rescanning 2022-12-08 16:45:21 +01:00
fanquake
07ac7a2dbf Merge bitcoin/bitcoin#26513: Make static nLastFlush and nLastWrite Chainstate members
07dfbb5bb8 Make static nLastFlush and nLastWrite Chainstate members (Aurèle Oulès)

Pull request description:

  Fixes #22189.

  The `static std::multimap<uint256, FlatFilePos> mapBlocksUnknownParent; ` referenced in the issue was already fixed by #25571. I don't believe Chainstate references any other static variables.

ACKs for top commit:
  jamesob:
    ACK 07dfbb5bb8 ([`jamesob/ackr/26513.1.aureleoules.make_static_nlastflush_a`](https://github.com/jamesob/bitcoin/tree/ackr/26513.1.aureleoules.make_static_nlastflush_a))
  theStack:
    Concept and code-review ACK 07dfbb5bb8

Tree-SHA512: 0f26463c079bbc5e0e62707d4ca4c8c9bbb99edfa3391d48d4915d24e2a1190873ecd4f9f11da25b44527671cdc82c41fd8234d56a4592a246989448d34406b0
2022-12-08 15:35:28 +00:00
MarcoFalke
fabb24cbef test: Use last release in compatibility tests 2022-12-08 14:57:25 +01:00
MarcoFalke
fa34e5f3d3 test: Avoid intermittent timeout in feature_assumevalid.py 2022-12-08 14:24:46 +01:00
Aurèle Oulès
e6906fcf9e rpc: Enable wallet import on pruned nodes
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2022-12-08 12:23:39 +01:00
Aurèle Oulès
c29bff5b91 test: Fix backwards compatibility intermittent failure 2022-12-08 11:13:34 +01:00
MarcoFalke
1801d8c3c9 Merge bitcoin/bitcoin#26308: rpc/rest/zmq: reduce LOCK(cs_main) scope: ~6 times as many requests per second
d7f61e7d59 rpc: reduce LOCK(cs_main) scope in gettxoutproof (Andrew Toth)
4d92b5aaba rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats (Andrew Toth)
efd82aec8a rpc: reduce LOCK(cs_main) scope in blockToJSON (Andrew Toth)
f00808e932 rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock (Andrew Toth)
7d253c943f zmq: remove LOCK(cs_main) from NotifyBlock (Andrew Toth)
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block (Andrew Toth)

Pull request description:

  Picking up from #21006.

  After commit ccd8ef65f9 it is no longer required to hold `cs_main` when calling `ReadBlockFromDisk`. This can be verified in `master` at https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L755. Same can be seen for `UndoReadFromDisk` https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L485.

  The first commit moves `ReadBlockFromDisk` outside the lock scope in `rest_block`, where we can see a huge performance improvement when fetching blocks with multiple threads.

  My test setup, on an Intel i7 with 8 cores (16 threads):

  1. Start a fully synced bitcoind, with this `bitcoin.conf`:
  ```
      rest=1
      rpcthreads=16
      rpcworkqueue=64
      rpcuser=user
      rpcpassword=password
  ```
  2. Run ApacheBench: 10000 requests, 16 parallel threads, fetching block nr. 750000 in binary:
  ```
      ab -n 10000 -c 16 "http://127.0.0.1:8332/rest/block/0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e.bin"
  ```

  Time per request (mean)
      183 ms on master
      30 ms this branch

  So this can process 6.1 times as many requests, and saturates all the cores instead of keeping them partly idle waiting in the lock. With 8 threads the mean times were 90 ms on master and 19 ms on this branch, a speedup of 4.7x.

  Big thanks to martinus for finding this and the original PR.

  The second commit is from a suggestion on the original PR by jonatack to remove the unnecessary `LOCK(cs_main)` in the zmq notifier's `NotifyBlock`.

  I also found that this approach could be applied to rpcs `getblock` (including `verbosity=3`), `getblockstats`, and `gettxoutproof` with similar very good results. The above benchmarks steps need to be modified slightly for RPC. Run the following ApacheBench command with different request data in a file named `data.json`:
  ```
  ab -p data.json -n 10000 -c 16 -A user:password "http://127.0.0.1:8332/"
  ```
  For `getblock`, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e"]}
  ```
  master - 184 ms mean request time
  branch - 28 ms mean request time

  For `getblock` with verbosity level 3, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", 3]}
  ```
  This verbosity level fetches an undo file from disk, so it benefits from this approach as well. However, a lot of time is spent serializing to JSON so the performance gain is not as severe.
  master - 818 ms mean request time
  branch - 505 ms mean request time

  For `getblockstats`, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblockstats", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", ["minfeerate","avgfeerate"]]}
  ```
  This request used a lock on reading both a block and undo file, so the results are very good.
  master - 244 ms mean request time
  branch - 28 ms mean request time

ACKs for top commit:
  MarcoFalke:
    re-ACK d7f61e7d59 💫
  hebasto:
    ACK d7f61e7d59, I have reviewed the code and it looks OK. Did not make benchmarking though.

Tree-SHA512: 305ac945b4571c5f47646d4f0e78180d7a3d40b2f70ee43e4b3e00c96a465f6d0b9c750b8e85c89ed833e557e2cdb5896743f07ef90e4e53d4ad85452b545886
2022-12-08 10:48:02 +01:00
Andrew Chow
a653f4bb1f Merge bitcoin/bitcoin#25934: wallet, rpc: add label to listsinceblock
4e362c2b72 doc: add release note for 25934 (brunoerg)
fe488b4c4b test: add coverage for `label` in `listsinceblock` (brunoerg)
722e9a418d wallet, rpc: add `label` to `listsinceblock` (brunoerg)
852891ff98 refactor, wallet: use optional for `label` in `ListTransactions` (brunoerg)

Pull request description:

  This PR adds `label` parameter to `listsinceblock` to be able to fetch all incoming transactions having the specified label since a specific block.

  It's possible to use it in `listtransactions`, however, it's only possible to set the number of transactions to return, not a specific block to fetch from. `getreceivedbylabel` only returns the total amount received, not the txs info. `listreceivedbylabel` doesn't list all the informations about the transactions and it's not possible to fetch since a block.

ACKs for top commit:
  achow101:
    ACK 4e362c2b72
  w0xlt:
    ACK 4e362c2b72
  aureleoules:
    ACK 4e362c2b72

Tree-SHA512: fbde5db8cebf7a27804154fa61997b5155ad512e978cebb78c17acab9efcb624ea5f39d649899d12e5e675f80d4d0064cae8132b864de0d93a8d1e6fbcb9a737
2022-12-07 18:42:41 -05:00
Andrew Chow
bbfcbcfa0c Merge bitcoin/bitcoin#24611: Add fish completions
ccba4fe7e3 doc: Add completion subdir to contrib/README.md (willcl-ark)
7075848f96 script: Add fish completions (willcl-ark)
a27a445b71 refactor: Sub-folder bash completions (willcl-ark)

Pull request description:

  The completions are dynamically generated from the respective binary
  help pages.

  Completions should be sourced into the shell or added to
  `$XDG_CONFIG/fish/completions`. See [where to put completions](https://fishshell.com/docs/current/completions.html#where-to-put-completions) for more information.

  As the completions are auto-generated they should only require as much maintenance as the bash equivalents, which is to say very little!

ACKs for top commit:
  achow101:
    ACK ccba4fe7e3
  josibake:
    ACK ccba4fe7e3

Tree-SHA512: fe6ed899ea1fe90f82970bde7739db11dd0c845ccd70b65f28ad5212f75b57d9105a3a7f70ccdff552d5b21fa3fe9c697d128fb10740bae31fe1854e716b4b8b
2022-12-07 18:30:14 -05:00
brunoerg
c371cae07a test, init: perturb file to ensure failure instead of only deleting them 2022-12-07 15:01:31 -03:00
MarcoFalke
fa0153e609 refactor: Replace isTrue with get_bool
This makes the code more robust, see previous commit.

In general replacing isTrue with get_bool is not equivalent because
get_bool can throw exceptions, but in this case, exceptions won't happen
because of RPCTypeCheck() and isNull() checks in the preceding code.
2022-12-07 17:56:49 +01:00
MarcoFalke
fa2cc5d1d6 bugfix: Strict type checking for RPC boolean parameters 2022-12-07 17:55:58 +01:00
Hennadii Stepanov
affbf58a1e build: Move environment variables into $(package)_config_env 2022-12-07 16:51:48 +00:00
Hennadii Stepanov
d44fcd3c97 build: Make $(package)_*_env available to all $(package)_*_cmds 2022-12-07 16:51:35 +00:00
MarcoFalke
9052d869c9 Merge bitcoin/bitcoin#26517: test: Changed small_txpuzzle_randfee to return the virtual size instead of the transaction hex for feerate calculation.
6fb102c9f3 test: Changed small_txpuzzle_randfee to return the virtual size instead of the transaction hex for feerate calculation. (Randall Naar)

Pull request description:

  The fee rates used in feature_fee_estimation.py are calculated using the raw transaction size instead of the virtual transaction size (which is used in 'CBlockPolicyEstimator::processBlockTx' and 'CBlockPolicyEstimator::processBlock'). This leads to inconsistencies as the fee rates used in check_raw_estimates are incorrect and can cause assertions to fail.

  refs #25179

ACKs for top commit:
  MarcoFalke:
    ACK 6fb102c9f3

Tree-SHA512: b2bca85fffa605aeb17574f050736d4556506d782dd7fd969e165e48e108fd95ef4c4e2abbae83cce05ca777a00f4459cabfa0932694fa8bb93ddfba09d84357
2022-12-07 17:33:37 +01:00
fanquake
7d51560003 Merge bitcoin/bitcoin#26298: refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a
b19c4124b3 refactor: Rename ambiguous interfaces::MakeHandler functions (Ryan Ofsky)
dd6e8bd71c build: remove BOOST_CPPFLAGS from libbitcoin_util (fanquake)
82e272a109 refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a (Ryan Ofsky)

Pull request description:

  These belong in `libbitcoin_common.a`, not `libbitcoin_util.a`, because they aren't general-purpose utilities, they just contain some common glue code that is used by both the node and the wallet. Another reason not to include these in `libbitcoin_util.a` is to prevent them from being used by the kernel library.

  Also rename ambiguous `MakeHandler` functions to `MakeCleanupHandler` and `MakeSignalHandler`. Cleanup function handler was introduced after boost signals handler, so original naming didn't make much sense.

  This just contains a move-only commit, and a rename commit. There are no actual code or behavior changes.

  This PR is an alternative to #26293, and solves the same issue of removing a boost dependency from the _util_ library. The advantages of this PR compared to #26293 are that it keeps the source directory structure more flat, and it avoids having to change #includes all over the codebase.

ACKs for top commit:
  hebasto:
    ACK b19c4124b3

Tree-SHA512: b3a1d33eedceda7ad852c6d6f35700159d156d96071e59acae2bc325467fef81476f860a8855ea39cf3ea706a1df2a341f34fb2dcb032c31a3b0e9cf14103b6a
2022-12-07 14:54:23 +00:00
MarcoFalke
272fb0a5cf Merge bitcoin/bitcoin#26645: util: Include full version id in bug reports
fa825bd227 util: Include full version id in bug reports (MarcoFalke)

Pull request description:

  This will show the unique id of the full source code when the bug occurred, which can help debugging

ACKs for top commit:
  1440000bytes:
    utACK fa825bd227
  theStack:
    ACK fa825bd227
  john-moffett:
    ACK fa825bd227

Tree-SHA512: a7a775718f5f9796b5cffafbb3ace8adb5c163414ec584a57143157fc9dfb86f799e3b9c8365fcb831ee1e9eafc59d699d1653d772c68392de421b3de74dcd61
2022-12-07 08:47:32 +01:00
Andrew Chow
e5daf976d5 wallet: Rename nFeeRet in CreateTransactionInternal to current_fee
nFeeRet represents the fee that the transaction currently pays. Update
it's name to reflect that.
2022-12-06 15:18:18 -05:00
Andrew Toth
d7f61e7d59 rpc: reduce LOCK(cs_main) scope in gettxoutproof 2022-12-06 15:07:04 -05:00
Andrew Toth
4d92b5aaba rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats 2022-12-06 15:07:04 -05:00
Andrew Toth
efd82aec8a rpc: reduce LOCK(cs_main) scope in blockToJSON 2022-12-06 15:07:04 -05:00
Andrew Toth
f00808e932 rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock 2022-12-06 15:07:04 -05:00
Andrew Toth
7d253c943f zmq: remove LOCK(cs_main) from NotifyBlock 2022-12-06 15:07:04 -05:00
Andrew Toth
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block 2022-12-06 15:07:04 -05:00
Hennadii Stepanov
0596aa40f7 Merge bitcoin-core/gui#683: doc: Drop no longer relevant comment
5d332da2cf doc: Drop no longer relevant comment (Hennadii Stepanov)

Pull request description:

  The comment was introduced in 4cf3411056, and since 7e4bd19785 it has been no longer relevant.

ACKs for top commit:
  jarolrod:
    ACK 5d332da2cf

Tree-SHA512: 6d32561336993b1ff7d7c524d090ac52aefb40078ed706ca4c6d5026cc3f63244c49c0e00e45ff192ba0e9f1527faf63249aa18bc8aa677b9e053d387e0f4027
2022-12-06 18:58:08 +00:00
brunoerg
4e362c2b72 doc: add release note for 25934 2022-12-06 15:27:50 -03:00
brunoerg
fe488b4c4b test: add coverage for label in listsinceblock 2022-12-06 15:27:50 -03:00
brunoerg
722e9a418d wallet, rpc: add label to listsinceblock 2022-12-06 15:27:50 -03:00
brunoerg
852891ff98 refactor, wallet: use optional for label in ListTransactions 2022-12-06 15:27:50 -03:00
fanquake
0abb5b2dd6 Merge bitcoin/bitcoin#26648: doc: move release notes to 24.0.1 and add notice
896fca16a3 doc: move release notes to 24.0.1 and add notice (fanquake)

Pull request description:

  This mirrors what was done with 0.19.0.1.

ACKs for top commit:
  instagibbs:
    ACK 896fca16a3
  gruve-p:
    ACK 896fca16a3
  w0xlt:
    ACK 896fca16a3

Tree-SHA512: 590462555422c0f96895152d1a2f9f9cf0ebf2c61dc342094d747f4d48b878e5d91840b9c2ac6825bb7214239f035789f1765a907fb614017205377ed89631fd
2022-12-06 18:14:52 +00:00
MarcoFalke
1ff79292e3 Merge bitcoin/bitcoin#26609: refactor: Move txmempool_entry.h --> kernel/mempool_entry.h
38941a703e refactor: Move `txmempool_entry.h` --> `kernel/mempool_entry.h` (Hennadii Stepanov)

Pull request description:

  This PR addresses the https://github.com/bitcoin/bitcoin/pull/17786#discussion_r1027818360:
  > why not move it to the right place, that is to `kernel/txmempool_entry.h`?

ACKs for top commit:
  MarcoFalke:
    review ACK 38941a703e 📊

Tree-SHA512: 0145974b63b67ca1d9d89af2dd9d4438beca480c16a563f330da05fec49b8394d7ba20ed83cf7d50b2e19454e006978ebed42b0e07887b98d00210f3201ce9ba
2022-12-06 19:04:31 +01:00
Andrew Chow
ef744c03e5 Merge bitcoin/bitcoin#25729: wallet: Check max transaction weight in CoinSelection
c7c7ee9d0b test: Check max transaction weight in CoinSelection (Aurèle Oulès)
6b563cae92 wallet: Check max tx weight in coin selector (Aurèle Oulès)

Pull request description:

  This PR is an attempt to fix #5782.

  I have added 4 test scenarios, 3 of them provided here https://github.com/bitcoin/bitcoin/issues/5782#issuecomment-73819058 (slightly modified to use a segwit wallet).

  Here are my benchmarks :
  ## PR
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        1,466,341.00 |              681.97 |    0.6% |   11,176,762.00 |    3,358,752.00 |  3.328 |   1,897,839.00 |    0.3% |      0.02 | `CoinSelection`

  ## Master

  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        1,526,029.00 |              655.30 |    0.5% |   11,142,188.00 |    3,499,200.00 |  3.184 |   1,994,156.00 |    0.2% |      0.02 | `CoinSelection`

ACKs for top commit:
  achow101:
    reACK c7c7ee9d0b
  w0xlt:
    ACK c7c7ee9d0b
  furszy:
    diff ACK c7c7ee9d

Tree-SHA512: ef0b28576ff845174651ba494aa9adee234c96e6f886d0e032eceb7050296e45b099dda0039d1dfb9944469f067627b2101f3ff855c70353cf39d1fc7ee81828
2022-12-06 12:08:58 -05:00
w0xlt
b082f28101 rpc, wallet: use the same next_index in listdescriptors and importdescriptors 2022-12-06 11:38:07 -03:00
fanquake
896fca16a3 doc: move release notes to 24.0.1 and add notice
This mirrors what was done with 0.19.0.1.
2022-12-06 14:10:45 +00:00
MarcoFalke
8ccab65f28 Merge bitcoin/bitcoin#26238: clang-tidy: fixup named argument comments
203886c443 Fixup clang-tidy named argument comments (fanquake)

Pull request description:

  Fix comments so they are checked/consistent.
  Fix incorrect comments.

ACKs for top commit:
  hebasto:
    ACK 203886c443, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: e1257840f91fe39842e2b19299c1633604697b8584fe44b1977ada33cdde5433c877ed0b669fa334e20b04971dc89cd47d58b2783b6f7004521f01d05a1245da
2022-12-06 12:05:09 +01:00
MarcoFalke
fa825bd227 util: Include full version id in bug reports 2022-12-06 11:14:47 +01:00
MarcoFalke
edbe4f808a Merge bitcoin/bitcoin#26611: wallet: Change coin selection fee assert to error
3eb041f014 wallet: Change coin selection fee assert to error (Andrew Chow)
c6e7f224c1 util: Add StrFormatInternalBug and STR_INTERNAL_BUG (MarcoFalke)

Pull request description:

  Returning an error instead of asserting for the low fee check will be better as it does not crash the node and instructs users to report the bug.

ACKs for top commit:
  S3RK:
    ACK 3eb041f014
  aureleoules:
    ACK 3eb041f014
  furszy:
    ACK 3eb041f0

Tree-SHA512: 118c13d7cdfce492080edd4cb12e6d960695377b978c7573f9c58b6d918664afd0e8e591eed0605d08ac756fa8eceed456349de5f3a025174069abf369bb5a5f
2022-12-06 10:31:02 +01:00
S3RK
17554efb60 test: prefer sqlite for wallet tests 2022-12-06 09:17:25 +01:00
S3RK
8e0fabaabf test: make wallet_migration.py pass with both wallet flags 2022-12-06 09:17:24 +01:00
Andrew Chow
5d9b5305af Merge bitcoin/bitcoin#19888: rpc, test: Improve getblockstats for unspendables
d885bb2f6e test: Test exclusion of OP_RETURN from getblockstats (Fabian Jahr)
ba9d288b24 test: Fix getblockstats test data generator (Fabian Jahr)
2ca5a496c2 rpc: Improve getblockstats (Fabian Jahr)
cb94db119f validation, index: Add unspendable coinbase helper functions (Fabian Jahr)

Pull request description:

  Fixes #19885

  The genesis block does not have undo data saved to disk so the RPC errored because of that.

ACKs for top commit:
  achow101:
    ACK d885bb2f6e
  aureleoules:
    ACK d885bb2f6e
  stickies-v:
    ACK d885bb2f6

Tree-SHA512: f37bda736ed605b7a41a81eeb4bfbb5d2b8518f847819e5d6a090548a61caf1455623e15165d72589ab3f4478252b00e7b624f9313ad6708cac06dd5edb62e9a
2022-12-05 17:46:54 -05:00
Andrew Chow
2ce3d26757 Merge bitcoin/bitcoin#26462: wallet: fix crash on loading descriptor wallet containing legacy key type entries
3198e4239e test: check that loading descriptor wallet with legacy entries throws error (Sebastian Falbesoner)
349ed2a0ee wallet: throw error if legacy entries are present on loading descriptor wallets (Sebastian Falbesoner)

Pull request description:

  Loading a descriptor wallet currently leads to a segfault if a legacy key type entry is present that can be deserialized successfully and needs SPKman-interaction. To reproduce with a "cscript" entry (see second commit for details):

  ```
  $ ./src/bitcoin-cli createwallet crashme
  $ ./src/bitcoin-cli unloadwallet crashme
  $ sqlite3 ~/.bitcoin/wallets/crashme/wallet.dat
  SQLite version 3.38.2 2022-03-26 13:51:10
  Enter ".help" for usage hints.
  sqlite> INSERT INTO main VALUES(x'07637363726970740000000000000000000000000000000000000000', x'00');
  $ ./src/bitcoin-cli loadwallet crashme

  --- bitcoind output: ---
  2022-11-06T13:51:01Z Using SQLite Version 3.38.2
  2022-11-06T13:51:01Z Using wallet /home/honey/.bitcoin/wallets/crashme
  2022-11-06T13:51:01Z init message: Loading wallet…
  2022-11-06T13:51:01Z [crashme] Wallet file version = 10500, last client version = 249900

  Segmentation fault (core dumped)
  ```

  Background: In the wallet key-value-loading routine, most legacy type entries require a `LegacyScriptPubKeyMan` instance after successful deserialization. On a descriptor wallet, creating that (via method `GetOrCreateLegacyScriptPubKeyMan`) fails and then leads to a null-pointer dereference crash. E.g. for CSCRIPT: 50422b770a/src/wallet/walletdb.cpp (L589-L594)

  ~~This PR fixes this by simply ignoring legacy entries if the wallet flags indicate that we have a descriptor wallet. The second commits adds a regression test to the descriptor wallet's functional test (fortunately Python includes sqlite3 support in the standard library).~~

  ~~Probably it would be even better to throw a warning to the user if unexpected legacy entries are found in descriptor wallets, but I think as a first mitigation everything is obvisouly better than crashing. As far as I'm aware, descriptor wallets created/migrated by Bitcoin Core should never end up in a state containing legacy type entries though.~~

  This PR fixes this by throwing an error if legacy entries are found in descriptor wallets on loading.

ACKs for top commit:
  achow101:
    ACK 3198e4239e
  aureleoules:
    ACK 3198e4239e

Tree-SHA512: ee43da3f61248e0fde55d9a705869202cb83df678ebf4816f0e77263f0beac0d7bae9490465d1753159efb093ee37182931d76b2e2b6e8c6f8761285700ace1c
2022-12-05 17:37:48 -05:00
Aurèle Oulès
c7c7ee9d0b test: Check max transaction weight in CoinSelection
Co-authored-by: Andrew Chow <github@achow101.com>
2022-12-05 19:32:11 +01:00
Aurèle Oulès
6b563cae92 wallet: Check max tx weight in coin selector
Co-authored-by: Andrew Chow <github@achow101.com>
2022-12-05 19:32:11 +01:00
Andrew Chow
3eb041f014 wallet: Change coin selection fee assert to error
Returning an error instead of asserting for the low fee check will be
better as it does not crash the node and instructs users to report the
bug.
2022-12-05 12:59:22 -05:00
Andrew Chow
7734a0160d Merge bitcoin/bitcoin#26640: test: Run mempool_compatibility.py with MiniWallet
fa43f60a0c test: Run mempool_compatibility.py with MiniWallet (MarcoFalke)

Pull request description:

  By using the already existing miniwallet, the test can be run even when no wallet is compiled.

ACKs for top commit:
  glozow:
    ACK fa43f60a0c
  achow101:
    ACK fa43f60a0c

Tree-SHA512: 6877b3f2f364663f04c28ab9f3d69780de6d1b77cc862379bba8c8242bbcfb0d26eb84c56cf721141407c393f1f3b49f667ae4fb32b3566108d71250e8b5d7bc
2022-12-05 12:39:15 -05:00
Andrew Chow
f0c4807a6a Merge bitcoin/bitcoin#26560: wallet: bugfix, invalid CoinsResult cached total amount
7362f8e5e2 refactor: make CoinsResult total amounts members private (furszy)
3282fad599 wallet: add assert to SelectionResult::Merge for safety (S3RK)
c4e3b7d6a1 wallet: SelectCoins, return early if wallet's UTXOs cannot cover the target (furszy)
cac2725fd0 test: bugfix, coinselector_test, use 'CoinsResult::Erase/Add' instead of direct member access (furszy)
cf79384697 test: Coin Selection, duplicated preset inputs selection (furszy)
341ba7ffd8 test: wallet, coverage for CoinsResult::Erase function (furszy)
f930aefff9 wallet: bugfix, 'CoinsResult::Erase' is erasing only one output of the set (furszy)

Pull request description:

  This comes with #26559.

  Solving few bugs inside the wallet's transaction creation
  process and adding test coverage for them.
  Plus, making use of the `CoinsResult::total_amount` cached value
  inside the Coin Selection process to return early if we don't have
  enough funds to cover the target amount.

  ### Bugs

  1) The `CoinsResult::Erase` method removes only one
  output from the available coins vector (there is a [loop break](c1061be14a/src/wallet/spend.cpp (L112))
  that should have never been there) and not all the preset inputs.

     Which on master is not a problem, because since [#25685](https://github.com/bitcoin/bitcoin/pull/25685)
     we are no longer using the method. But, it's a bug on v24
     (check [#26559](https://github.com/bitcoin/bitcoin/pull/26559)).

     This method it's being fixed and not removed because I'm later using it to solve
     another bug inside this PR.

  2) As we update the total cached amount of the `CoinsResult` object inside
     `AvailableCoins` and we don't use such function inside the coin selection
     tests (we manually load up the `CoinsResult` object), there is a discrepancy
     between the outputs that we add/erase and the total amount cached value.

  ### Improvements

  * This makes use of the `CoinsResult` total amount field to early return
    with an "Insufficient funds" error inside Coin Selection if the tx target
    amount is greater than the sum of all the wallet available coins plus the
    preset inputs amounts (we don't need to perform the entire coin selection
    process if we already know that there aren't enough funds inside our wallet).

  ### Test Coverage

  1) Adds test coverage for the duplicated preset input selection bug that we have in v24.
    Where the wallet invalidly selects the preset inputs twice during the Coin Selection
    process. Which ends up with a "good" Coin Selection result that does not cover the
    total tx target amount. Which, alone, crashes the wallet due an insane fee.
    But.. to make it worst, adding the subtract fee from output functionality
    to this mix ends up with the wallet by-passing the "insane" fee assertion,
    decreasing the output amount to fulfill the insane fee, and.. sadly,
    broadcasting the tx to the network.

  2) Adds test coverage for the `CoinsResult::Erase` method.

  ------------------------------------

  TO DO:
  * [ ] Update [#26559 ](https://github.com/bitcoin/bitcoin/pull/26559) description.

ACKs for top commit:
  achow101:
    ACK 7362f8e5e2
  glozow:
    ACK 7362f8e5e2, I assume there will be a followup PR to add coin selection sanity checks and we can discuss the best way to do that there.
  josibake:
    ACK [7362f8e](7362f8e5e2)

Tree-SHA512: 37a6828ea10d8d36c8d5873ceede7c8bef72ae4c34bef21721fa9dad83ad6dba93711c3170a26ab6e05bdbc267bb17433da08ccb83b82956d05fb16090328cba
2022-12-05 12:00:45 -05:00
fanquake
203886c443 Fixup clang-tidy named argument comments
Fix comments so they are checked/consistent.
Fix incorrect arguments.
2022-12-05 15:51:46 +00:00
MarcoFalke
38cbf43dee Merge bitcoin/bitcoin#26414: test: Move tx creation to create_self_transfer_multi
0b78110f73 test: Move tx creation to create_self_transfer_multi (kouloumos)

Pull request description:

  Two birds with one stone: replacement of https://github.com/bitcoin/bitcoin/pull/26278 with simplification of the MiniWallet's transaction creation logic.

  Currently the MiniWallet creates simple txns (1 input, 1 output) with `create_self_transfer`.  https://github.com/bitcoin/bitcoin/pull/24637 introduced `create_self_transfer_multi` **which uses** `create_self_transfer` to create a "transaction template" which then adjusts (copy and mutate inputs and outputs) in order to create more complex multi-input multi-output transactions.
  This can more easily lead to issues such as https://github.com/bitcoin/bitcoin/pull/26278 and is more of a maintenance burden.

  This PR simplifies the logic by going the other way around. Now `create_self_transfer` **uses** `create_self_transfer_multi`.
  The transaction creation logic has been moved to `create_self_transfer_multi` which is being called by `create_self_transfer` to construct the simple case of 1 input 1 output transaction.

ACKs for top commit:
  MarcoFalke:
    ACK 0b78110f73 👒

Tree-SHA512: 147e577ed5444bee57865bd375b37c9b49d6539e9875c30c2667e70fcba27fe80bcb4552a4e6efb42760d34b40d5dad826883b778eaeefe29425ec081787b4bd
2022-12-05 16:22:42 +01:00
MarcoFalke
5b3f05b7eb Merge bitcoin/bitcoin#24226: rpc: warn that nodes ignore requests for old stale blocks
f39d9269eb rpc: warn that nodes ignore requests for old stale blocks (Sjors Provoost)

Pull request description:

  Adds warning to RPC help that `getblockfrompeer` is of little use for stale blocks that are more than a month old.

  This is an anti-fingerprinting measure. See `BlockRequestAllowed` in `net_processing`.

  It's been in Bitcoin Core since 2014, introduced in #2910 and later improved to not rely on checkpoints.
  Older and alternative clients might still serve these blocks, so not throwing an error.

  Allowing whitelisted nodes to fetch these blocks anyway might be nice.

ACKs for top commit:
  fjahr:
    Code review ACK f39d9269eb

Tree-SHA512: db88f9f7521289640c5e629c840dda1c2c3ab70d458e9e7136c60fbaeb02acfb36dc093502d83d4c098c331e22aab81bf8f4c4961d805e3bde0f8f3cfe68d968
2022-12-05 14:01:59 +01:00
MarcoFalke
fa43f60a0c test: Run mempool_compatibility.py with MiniWallet 2022-12-05 13:13:00 +01:00
fanquake
71abee86db Merge bitcoin/bitcoin#25993: doc: Add I2P guidance related to bandwidth and i2pd software version
874c861885 doc: Add I2P bandwidth guidance to i2p.md (willcl-ark)

Pull request description:

  Add some general guidance on lowering bandwidth usage when using I2P routers.

ACKs for top commit:
  jonatack:
    ACK 874c861885
  hernanmarino:
    ACK 874c861885
  pablomartin4btc:
    ACK 874c861885.

Tree-SHA512: 3990375b23c01a2ad8e15a51e1b1a0275df8747ecd789ddf1888fbc88c20cde5a3813615982669af5e8d021dc995f6c7b1f080167b33f48574d6f50fc4851498
2022-12-05 11:45:23 +00:00
glozow
8b796866b4 Merge bitcoin/bitcoin#26624: refactor: Rename local variable to distinguish it from type alias
1984db1d50 refactor: Rename local variable to distinguish it from type alias (Hennadii Stepanov)

Pull request description:

  The `txiter` type alias is declared in the `txmempool.h`: 9e59d21fbe/src/txmempool.h (L406)

ACKs for top commit:
  stickies-v:
    ACK 1984db1d5
  vasild:
    ACK 1984db1d50
  jarolrod:
    ACK 1984db1d50

Tree-SHA512: 127bfb62627e2d79d8cdb0bd0ac11b3737568c3631b54b2d1e37984f673a1f60edf7bc102a269f7eb40e4bb124b910b924a89475c6a6ea978b2171219fa30685
2022-12-05 11:15:41 +00:00
fanquake
fe8d15c907 Merge bitcoin/bitcoin#26630: test: Fix intermittent issue in rpc_net.py
fadf7b8fef test: Fix intermittent issue in rpc_net.py (MarcoFalke)

Pull request description:

  Both nodes must be aware of the closed connections before re-connecting, otherwise the test will fail.

  Fixes #25741

ACKs for top commit:
  jarolrod:
    ACK fadf7b8fef

Tree-SHA512: 0492dd59a46516007e903fe8f6288982e305d4fd152a0297dd60e10ac663efc9bdc0bc6d1d2c6ec5f239196dac08319f37b079401fe057e4864b117b0fd9dcbc
2022-12-04 15:04:20 +00:00
Sebastian Falbesoner
8a5dbe2879 test: add CScript method for checking for witness program
This is needed in the next commit to calculate the dust threshold
for a given output script and min feerate for defining dust.
2022-12-04 03:07:42 +01:00
MarcoFalke
fadf7b8fef test: Fix intermittent issue in rpc_net.py 2022-12-03 17:42:05 +01:00
MarcoFalke
cac29f5cd6 Merge bitcoin/bitcoin#26622: test: Add test for sendall min-fee setting
cb44c5923a test: Add sendall test for min-fee setting (Aurèle Oulès)

Pull request description:

  While experimenting with mutation testing it appeared that the minimum fee-rate check was not tested for the `sendall` RPC.

  https://bcm-ui.aureleoules.com/mutations/3581479318544ea6b97f788cec6e6ef1

ACKs for top commit:
  0xB10C:
    ACK cb44c5923a
  ishaanam:
    ACK cb44c5923a
  stickies-v:
    re-ACK [cb44c59](cb44c5923a)

Tree-SHA512: 31978436e1f01cc6abf44addc62b6887e65611e9a7ae7dc72e6a73cdfdb3a6a4f0a6c53043b47ecd1b10fc902385a172921e68818a7f5061c96e5e1ef5280b48
2022-12-03 12:28:13 +01:00
Ryan Ofsky
d1ca563825 bitcoin-cli: Make it an error to specify the "args" parameter two different ways
MarcoFalke reported the case of positional arguments silently overwriting the
named "args" parameter in bitcoin-cli
https://github.com/bitcoin/bitcoin/pull/19762#discussion_r1035761471 and this
behavior is confusing and was not intended when support for "args" parameters
was added to bitcoin-cli in #19762.

Instead of letting one "args" value overwrite the other in the client, just
pass the values to the server verbatim, and let the error be handled server
side.
2022-12-02 17:53:58 -05:00
Ryan Ofsky
6bd1d20b8c rpc: Make it an error server-side to specify same named parameter multiple times
Specifying same named parameter multiple times is still allowed by bitcoin-cli.
The client implementation overwrites earlier option values with later ones
before sending to server. This is tested by interface_bitcoin_cli.py

Rationale for allowing client parameters to be specified multiple times in
bitcoin-cli is that this behavior has been supported for a long time, and that
when using the command line interactively, it can be convenient to override
earlier option values with new values without having to go back and remove the
old value.

But for the RPC server, there isn't really a good use-case for earlier values
to be discarded if multiple values are specified. JSON keys are generally
supposed to be unique and if they aren't it's probably an indication of some
problem generating the RPC request.
2022-12-02 17:53:16 -05:00
Ryan Ofsky
e2c3b18e67 test: Add RPC tests for same named parameter specified more than once
Current behavior isn't ideal and will be changed in upcoming commits, but it's
useful to have test coverage regardless.

MarcoFalke reported the case of bitcoin-cli positional arguments overwriting
the named "args" parameter in
https://github.com/bitcoin/bitcoin/pull/19762#discussion_r1035761471
2022-12-02 17:37:08 -05:00
Hennadii Stepanov
5d332da2cf doc: Drop no longer relevant comment
The comment was introduced in 4cf3411056,
and since 7e4bd19785 it has been no longer
relevant.
2022-12-02 15:44:34 +00:00
furszy
7362f8e5e2 refactor: make CoinsResult total amounts members private 2022-12-02 12:39:16 -03:00
S3RK
3282fad599 wallet: add assert to SelectionResult::Merge for safety 2022-12-02 12:39:15 -03:00
furszy
c4e3b7d6a1 wallet: SelectCoins, return early if wallet's UTXOs cannot cover the target
The CoinsResult class will now count the raw total amount and the effective
total amount internally (inside the 'CoinsResult::Add' and 'CoinsResult::Erase'
methods).
So there is no discrepancy between what we add/erase and the total values.
(which is what was happening on the coinselector_test because the 'CoinsResult'
object is manually created there, and we were not keeping the total amount
in sync with the outputs being added/removed).
2022-12-02 12:39:15 -03:00
furszy
cac2725fd0 test: bugfix, coinselector_test, use 'CoinsResult::Erase/Add' instead of direct member access
Aside from the cleanup, this solves a bug in the following-up commit. Because, in these
tests, we are manually adding/erasing outputs from the CoinsResult object but never
updating the internal total amount field.
2022-12-02 12:39:15 -03:00
furszy
cf79384697 test: Coin Selection, duplicated preset inputs selection
This exercises the bug inside CoinsResult::Erase that
ends up on (1) a wallet crash or (2) a created and
broadcasted tx that contains a reduced recipient's amount.

This is covered by making the wallet selects the preset
inputs twice during the coin selection process.

Making the wallet think that the selection process result covers
the entire tx target when it does not. It's actually creating
a tx that sends more coins than what inputs are covering for.

Which, combined with the SFFO option, makes the wallet
incorrectly reduce the recipient's amount by the difference
between the original target and the wrongly counted inputs.
Which means, a created and relayed tx sending less coins to
the destination than what the user inputted.
2022-12-02 12:39:15 -03:00
furszy
341ba7ffd8 test: wallet, coverage for CoinsResult::Erase function 2022-12-02 12:39:15 -03:00
fanquake
78aee0fe2c Merge bitcoin/bitcoin#26569: p2p: Ensure transaction announcements are only queued for fully connected peers
8f2dac5409 [test] Add p2p_tx_privacy.py (dergoegge)
ce63fca13e [net processing] Assume that TxRelay::m_tx_inventory_to_send is empty pre-verack (dergoegge)
845e3a34c4 [net processing] Ensure transaction announcements are only queued for fully connected peers (dergoegge)

Pull request description:

  `TxRelay::m_next_inv_send_time` is initialized to 0, which means that any txids in `TxRelay::m_tx_inventory_to_send` will be announced on the first call to `PeerManagerImpl::SendMessages` for a fully connected peer (i.e. it completed the version handshake).

  Prior to #21160, `TxRelay::m_tx_inventory_to_send` was guaranteed to be empty on the first `SendMessages` call, as transaction announcements were only queued for fully connected peers. #21160 replaced a `CConnman::ForEachNode` call with a loop over `PeerManagerImpl::m_peer_map`, in which the txid for a transaction to be relayed is added to `TxRelay::m_tx_inventory_to_send` for all peers. Even for those peers that have not completed the version handshake. Prior to the PR this was not the case as `ForEachNode` has a "fully connected check" before calling a function for each node.

ACKs for top commit:
  MarcoFalke:
    ACK 8f2dac5409 🔝
  jnewbery:
    utACK 8f2dac5409

Tree-SHA512: e9eaccf7e00633ee0806fff1068b0e413a69a5e389d96c9659f68079915a6381ad5040c61f716cfcde77931d1b563b1049da97a232a95c6cd8355bd3d13404b9
2022-12-02 15:13:31 +00:00
Hennadii Stepanov
6ebe57622c build: Make dependency package archive timestamps deterministic 2022-12-02 12:44:07 +00:00
Aurèle Oulès
cb44c5923a test: Add sendall test for min-fee setting 2022-12-02 13:30:40 +01:00
Hennadii Stepanov
1984db1d50 refactor: Rename local variable to distinguish it from type alias
The `txiter` type alias is declared in the `txmempool.h`.
2022-12-02 11:25:35 +00:00
MarcoFalke
9e59d21fbe Merge bitcoin/bitcoin#26592: ci: only run USDT interface tests on CirrusCI
2811f40f30 ci: only run USDT interface tests on CirrusCI (0xb10c)

Pull request description:

  As mentioned in #26571, the task running the USDT interface tests fail when run in docker. cc7335edc8 in #25528 added that the tests are run in a **VM** in Cirrus CI. Running them locally in docker containers might not work:

  - We use [bcc] as tracing toolkit which requires the kernel headers to compile the BPF bytecode. As docker containers use the hosts kernel and don't run their own, there is a potential for mismatches between kernel headers available in the container and the host kernel. This results in a failure loading the BPF byte code.
  - Privilges are required to load the BPF byte code into the kernel. Normally, the docker containers aren't run with these.
  - We currently use an untrusted third-party PPA to install the bpfcc-tools package on Ubuntu 22.04. Using this on a local dev system could be a security risk.

  To not hinder the ASan + LSan + UBSan part of the CI task, the USDT tests are disabled on non-CirrusCI runs.

  [bcc]: https://github.com/iovisor/bcc

Top commit has no ACKs.

Tree-SHA512: 7c159b59630b36d5ed927b501fa0ad6f54ff3d98d0902f975cc8122b4147a7f828175807d40a470a85f0f3b6eeb79307d3465a287dbd2f3d75b803769ad0b6e7
2022-12-02 10:52:52 +01:00
MarcoFalke
4037478114 Merge bitcoin/bitcoin#26610: test: Remove unused blocktools imports from wallet_bumpfee
fa15c671f7 test: Remove unused blocktools imports from wallet_bumpfee (MarcoFalke)

Pull request description:

  Seems bloaty and confusing to use "tools" when a single RPC can already achieve the same.

ACKs for top commit:
  theStack:
    ACK fa15c671f7

Tree-SHA512: 87f9c31bbb286fee5e479ae54a1f9131f4d4294d665a985df8b14a0cc837a2a2e145ccd3660612768d88cfa0827a3eef392f85519b6cb7df365ba9fadafb0a41
2022-12-02 09:43:26 +01:00
MarcoFalke
02515117dc Merge bitcoin/bitcoin#26620: test: refactor: eliminate genesis block timestamp magic numbers
dbed28968a test: refactor: eliminate genesis block timestamp magic numbers (Sebastian Falbesoner)

Pull request description:

  This tiny PR replaces all occurences of the regtest/testnet genesis block timestamp (found via `git grep 1296688602`) with the constant `TIME_GENESIS_BLOCK` to increase the readability.

ACKs for top commit:
  aureleoules:
    ACK dbed28968a

Tree-SHA512: be39d5c2631ad20eb775c2a077b1b1f056a1a4930aa44e6fdec73b974fd4bdf8da0103a3a38e3514b68fcf6a6316e007a371c523da5076a315545c9bf3091aee
2022-12-01 18:37:59 +01:00
MarcoFalke
c6e7f224c1 util: Add StrFormatInternalBug and STR_INTERNAL_BUG 2022-12-01 12:22:47 -05:00
MarcoFalke
a04121bdf9 Merge bitcoin/bitcoin#26617: test: add extra_args to BitcoinTestFramework class
150340aeac test: remove unneeded extra_args code (josibake)
989a52e0a5 test: add extra_args to BTF class (josibake)

Pull request description:

  ## problem
  If you try to add `extra_args` when using `TestShell`, you will get the following error:

  ```python
  >>> import sys
  >>>
  >>> sys.path.insert(0, "/home/josibake/bitcoin/test/functional")
  >>>
  >>> from test_framework.test_shell import TestShell
  >>> test = TestShell().setup(num_nodes=2, extra_args=[[],['-fallbackfee=0.0002']])
  Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "/home/josibake/bitcoin/test/functional/test_framework/test_shell.py", line 41, in setup
      raise KeyError(key + " not a valid parameter key!")
  KeyError: 'extra_args not a valid parameter key!'
  >>>
  ```

  ## solution

  add `self.extra_args = None` so that `extra_args` is recognized as a valid parameter to be passed to `BitcoinTestFramework`

  ```python
  >>> import sys
  >>>
  >>> sys.path.insert(0, "/home/josibake/bitcoin/test/functional")
  >>>
  >>> from test_framework.test_shell import TestShell
  >>> test = TestShell().setup(num_nodes=2, extra_args=[[],['-fallbackfee=0.0002']])
  2022-12-01T11:23:23.765000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_sbwthbb_
  ```

ACKs for top commit:
  willcl-ark:
    re-ACK 150340aeac

Tree-SHA512: e6fa2a780a8f2d3472c322e8cdb00ec35cb220c3b4d6ca02291eb8b41c0d8676a635fbc79c6be80e3bb71d700a2501a4b73f762478f533ae453d492d449307bb
2022-12-01 17:50:46 +01:00
josibake
150340aeac test: remove unneeded extra_args code 2022-12-01 16:55:43 +01:00
Sebastian Falbesoner
dbed28968a test: refactor: eliminate genesis block timestamp magic numbers 2022-12-01 12:59:59 +01:00
josibake
989a52e0a5 test: add extra_args to BTF class
this allows us to pass extra_args when using TestShell
2022-12-01 12:14:10 +01:00
fanquake
e334f7a545 Merge bitcoin/bitcoin#26594: wallet: Avoid a segfault in migratewallet failure cleanup
5e65a216d1 wallet: Explicitly say migratewallet on encrypted wallets is unsupported (Andrew Chow)
88afc73ae0 tests: Test for migrating encrypted wallets (Andrew Chow)
86ef7b3c7b wallet: Avoid null pointer deref when cleaning up migratewallet (Andrew Chow)

Pull request description:

  When `migratewallet` fails, we do an automatic cleanup in order to reset everything so that the user does not experience any interruptions. However, this apparently has a segfault in it, caused by the the pointers to the watchonly and solvables wallets being nullptr. If those wallets are not created (either not needed, or failed early on), we will accidentally attempt to dereference these nullptrs, which causes a segfault.

  This failure can be easily reached by trying to migrate an encrypted wallet. Currently, we can't migrate encrypted wallets because of how we unload wallets before migrating, and therefore forget the encryption key if the wallet was unlocked. So any encrypted wallets will fail, entering the cleanup, and because watchonly and solvables wallets don't exist yet, the segfault is reached.

  This PR fixes this by not putting those nullptrs in a place that we will end up dereferencing them later. It also adds a test that uses the encrypted wallet issue.

ACKs for top commit:
  S3RK:
    reACK 5e65a216d1
  stickies-v:
    ACK [5e65a21](5e65a216d1)
  furszy:
    diff ACK 5e65a21

Tree-SHA512: f75643797220d4232ad3ab8cb4b46d0f3667f00486e910ca748c9b6d174d446968f1ec4dd7f907da1be9566088849da7edcd8cd8f12de671c3241b513deb8e80
2022-12-01 10:17:09 +00:00
dergoegge
8f2dac5409 [test] Add p2p_tx_privacy.py 2022-11-30 16:36:16 +00:00
Andrew Chow
e2bfd41f83 Merge bitcoin/bitcoin#25942: test: add ismine test for descriptor ScriptPubKeyMan
1b77db2653 test: add `ismine` test for descriptor scriptpubkeyman (w0xlt)

Pull request description:

  Currently `src/wallet/test/ismine_tests.cpp` has tests for the legacy ScriptPubKeyMan only.
  This PR adds tests for the descriptor ScriptPubKeyMan.

ACKs for top commit:
  ishaanam:
    ACK 1b77db2653
  achow101:
    ACK 1b77db2653
  furszy:
    ACK 1b77db26 with a non-blocking comment.

Tree-SHA512: 977b5d1e71f9468331aeb4ebaf3708dd651f9f3018d4544a395b87ca6d7fb8bfa6d20acc1a4f6e096e240e81d30fb7a6e8add190e52536e7a3cb5a80f392883f
2022-11-30 11:28:32 -05:00
MarcoFalke
fa15c671f7 test: Remove unused blocktools imports from wallet_bumpfee 2022-11-30 16:59:21 +01:00
Andrew Chow
5e65a216d1 wallet: Explicitly say migratewallet on encrypted wallets is unsupported 2022-11-30 10:30:57 -05:00
dergoegge
ce63fca13e [net processing] Assume that TxRelay::m_tx_inventory_to_send is empty pre-verack
This commit documents our assumption about
TxRelay::m_tx_inventory_to_send being empty prior to version handshake
completion.

The added Assume acts as testing oracle for our fuzzing tests to
potentially detect if the assumption is violated.
2022-11-30 12:22:05 +00:00
fanquake
bcee94d107 Merge bitcoin/bitcoin#26359: p2p: Erlay support signaling follow-ups
46339d29b1 test, refactor: Reorder sendtxrcncl tests for better readability (Gleb Naumenko)
14263c13f1 p2p, refactor: Extend logs for unexpected sendtxrcncl (Gleb Naumenko)
87493e112e p2p, test, refactor: Minor code improvements (Gleb Naumenko)
00c5dec818 p2p: Clarify sendtxrcncl policies (Gleb Naumenko)
ac6ee5ba21 test: Expand unit and functional tests for txreconciliation (Gleb Naumenko)
bc84e24a4f p2p, refactor: Switch to enum class for ReconciliationRegisterResult (Gleb Naumenko)
a60f729e29 p2p: Drop roles from sendtxrcncl (Gleb Naumenko)
6772cbf69c tests: stabilize sendtxrcncl test (Gleb Naumenko)

Pull request description:

  Non-trivial changes include:
  - Getting rid of roles in `sendtxrcncl` message (summarized in the [BIP PR](https://github.com/bitcoin/bips/pull/1376));
  - Disconnect the peer if it send `sendtxrcncl` although we are in `blocksonly` and notified the peer with `fRelay=0`;
  - Don't send `sendtxrcncl` to feeler connections.

ACKs for top commit:
  vasild:
    ACK 46339d29b1
  ariard:
    ACK 46339d2
  mzumsande:
    Code Review ACK 46339d29b1

Tree-SHA512: b5cc6934b4670c12b7dbb3189e739ef747ee542ec56678bf4e4355bfb481b746d32363c173635685b71969b3fe4bd52b1c8ebd3ea3b35c82044bba69220f6417
2022-11-30 10:52:44 +00:00
Hennadii Stepanov
38941a703e refactor: Move txmempool_entry.h --> kernel/mempool_entry.h 2022-11-30 10:37:57 +00:00
MarcoFalke
3be21060d6 Merge bitcoin/bitcoin#26600: test: Set wallet type in test_runner when only one type is allowed
fadb8696dd test: Set wallet type in test_runner when only one type is allowed (MarcoFalke)

Pull request description:

  Currently devs are free to set or not set the wallet type in the test_runner when only one type is allowed to be set.

  This is inconsistent and causes review comments such as:

  * https://github.com/bitcoin/bitcoin/pull/24865#discussion_r1009752111

ACKs for top commit:
  achow101:
    ACK fadb8696dd

Tree-SHA512: 1ca0946df07b5bf6778fea957d74393757781c324d554fec2f7d03bf1915033e644d9a4c3d77e0b24090ab593d7ed3cb3c9169666bc39fff423706fceaa1af80
2022-11-30 10:23:05 +01:00
Yusuf Sahin HAMZA
9141e4395a rpc, docs: Add note for commands that supports only legacy wallets
Note is added for following rpc commands:

importprivkey, importpubkey, importwallet, dumpprivkey,
dumpwallet, importmulti, addmultisigaddress, sethdseed
2022-11-30 04:21:28 +03:00
Andrew Chow
88afc73ae0 tests: Test for migrating encrypted wallets
Due to an oversight, we cannot currently migrate encrypted wallets,
regardless of whether they are unlocked. Migrating such wallets will
trigger an error, and result in the cleanup being run. This conveniently
allows us to check some parts of the cleanup code.
2022-11-29 19:31:27 -05:00
Andrew Chow
86ef7b3c7b wallet: Avoid null pointer deref when cleaning up migratewallet
If migratewallet fails, we do a cleanup which removes the watchonly and
solvables wallets if they were created. However, if they were not, their
pointers are nullptr and we don't check for that, which causes a
segfault during the cleanup. So check that they aren't nullptr before
cleaning them up.
2022-11-29 19:31:27 -05:00
Andrew Chow
5690848dfb Merge bitcoin/bitcoin#26532: wallet: bugfix, invalid crypted key "checksum_valid" set
13d9760829 test: load wallet, coverage for crypted keys (furszy)
373c99633e refactor: move DuplicateMockDatabase to wallet/test/util.h (furszy)
ee7a984f85 refactor: unify test/util/wallet.h with wallet/test/util.h (furszy)
cc5a5e8121 wallet: bugfix, invalid crypted key "checksum_valid" set (furszy)

Pull request description:

  At wallet load time, the crypted key "checksum_valid" variable is always set to false. Which, on every wallet decryption call, forces the process to re-write all the ckeys to db when it's not needed.

  Note:
  The first commit fixes the issue, the two commits in the middle are cleanups so `DuplicateMockDatabase`
  can be used without duplicating code. And, the last one is pure test coverage for the crypted keys loading
  process.

  Includes test coverage for the following scenarios:

  1) "All ckeys checksums valid" test:
  Loads an encrypted wallet with all the crypted keys with a valid checksum and
  verifies that 'CWallet::Unlock' doesn't force an entire crypted keys re-write.

      (we force a complete ckeys re-write if we find any missing crypted key checksum
  during the wallet loading process)

  2) "Missing checksum in one ckey" test:
  Verifies that loading up a wallet with, at least one, 'ckey' with no checksum
  triggers a complete re-write of the crypted keys.

  3) "Invalid ckey checksum error" test:
  Verifies that loading up a ckey with an invalid checksum stops the wallet loading
  process with a corruption error.

  4) "Invalid ckey pubkey error" test:
  Verifies that loading up a ckey with an invalid pubkey stops the wallet loading
  process with a corruption error.

ACKs for top commit:
  achow101:
    ACK 13d9760829
  aureleoules:
    ACK 13d9760829

Tree-SHA512: 9ea630ee4a355282fbeee61ca04737294382577bb4b2631f50e732568fdab8f72491930807fbda58206446c4f26200cdc34d8afa14dbe1241aec713887d06a0b
2022-11-29 18:54:57 -05:00
Andrew Chow
a63192afb8 Merge bitcoin/bitcoin#19762: rpc: Allow named and positional arguments to be used together
d8b12a75db rpc: Allow named and positional arguments to be used together (Ryan Ofsky)

Pull request description:

  It's nice to be able to use named options and positional arguments together.

  Most shell tools accept both, and python functions combine options and arguments allowing them to be passed with even more flexibility. This change adds support for python's approach so as a motivating example:

  ```sh
  bitcoin-cli -named createwallet wallet_name=mywallet load_on_startup=1
  ```

  Can be shortened to:

  ```sh
  bitcoin-cli -named createwallet mywallet load_on_startup=1
  ```

  JSON-RPC standard doesn't have a convention for passing named and positional parameters together, so this implementation makes one up and interprets any unused `"args"` named parameter as a positional parameter array.

  This change is backwards compatible. It doesn't change the interpretation of any previously valid calls, just treats some previously invalid calls as valid.

  Another use case even if you only occasionally use named arguments is that you can define an alias:

  ```
  alias bcli='bitcoin-cli -named'
  ```

  And now use both named named and unnamed arguments from the same alias without having to manually add `-named` option for named arguments or see annoying error "No '=' in named argument... this needs to be present for every argument (even if it is empty)`" for unnamed arguments

ACKs for top commit:
  achow101:
    ACK d8b12a75db
  stickies-v:
    re-ACK d8b12a75d
  aureleoules:
    re-ACK d8b12a75db

Tree-SHA512: 0cff8b50f584bcbbd376624adccf40536566ed8d1bcd6c88ad565dbc208f19d5e7a48c994efd6329d42b560149340d330397278f08a2912af5f3418d8c8837a9
2022-11-29 18:37:55 -05:00
stickies-v
f537127271 doc: fix: prevHeights entries are set to 0, not removed 2022-11-29 17:58:36 +00:00
furszy
f930aefff9 wallet: bugfix, 'CoinsResult::Erase' is erasing only one output of the set
The loop break shouldn't have being there.
2022-11-29 12:30:31 -03:00
dergoegge
845e3a34c4 [net processing] Ensure transaction announcements are only queued for fully connected peers 2022-11-29 13:54:50 +00:00
MarcoFalke
fadb8696dd test: Set wallet type in test_runner when only one type is allowed 2022-11-29 14:26:55 +01:00
Ryan Ofsky
b19c4124b3 refactor: Rename ambiguous interfaces::MakeHandler functions 2022-11-29 08:12:24 -04:00
fanquake
a035b6a0c4 Merge bitcoin/bitcoin#26565: contrib/builder-keys/keys.txt: remove unavailable key
dddfa028ff contrib/builder-keys/keys.txt: remove unavailable key (@RandyMcMillan)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK dddfa028ff

Tree-SHA512: e7bd2d9d828b44c4343478c68d01530621677f5c66912a4181815e66ef1926dfc66e6a7652789677ae263684d30a39da038b8ca669ad24833c1f104244c2a2b1
2022-11-29 12:12:24 +00:00
fanquake
dd6e8bd71c build: remove BOOST_CPPFLAGS from libbitcoin_util 2022-11-29 20:12:24 +08:00
Ryan Ofsky
82e272a109 refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a
These belong in libbitcoin_common.a, not libbitcoin_util.a, because they aren't
general-purpose utilities, they just contain common code that is used by both
the node and the wallet. Another reason to reason to not include these in
libbitcoin_util.a is to prevent them from being used by the kernel library.
2022-11-29 08:12:24 -04:00
Anthony Towns
a4fe09973a txorphanage: index workset by originating peer 2022-11-29 09:03:57 +10:00
willcl-ark
ccba4fe7e3 doc: Add completion subdir to contrib/README.md 2022-11-28 20:39:20 +00:00
willcl-ark
7075848f96 script: Add fish completions
Completions are dynamically generated from the respective binary help
pages.

Completions should be sourced into the shell or added to
$XDG_CONFIG/fish/completions.
2022-11-28 20:39:19 +00:00
0xb10c
2811f40f30 ci: only run USDT interface tests on CirrusCI
As mentioned in #26571, the task running the USDT interface tests
fail when run in docker. cc7335edc8
in #25528 added that the tests are run in a **VM** in Cirrus CI.
Running them locally in docker containers might not work:

- We use [bcc] as tracing toolkit which requires the kernel headers
  to compile the BPF bytecode. As docker containers use the hosts
  kernel and don't run their own, there is a potential for mismatches
  between kernel headers available in the container and the host
  kernel. This results in a failure loading the BPF byte code.
- Privilges are required to load the BPF byte code into the kernel.
  Normally, the docker containers aren't run with these.
- We currently use an untrusted third-party PPA to install the
  bpfcc-tools package on Ubuntu 22.04. Using this on a local dev
  system could be a security risk.

To not hinder the ASan + LSan + UBSan part of the CI task, the USDT
tests are disabled on non-CirrusCI runs.

[bcc]: https://github.com/iovisor/bcc
2022-11-28 21:26:26 +01:00
Martin Zumsande
6fefd49527 rpc: Require NodeStateStats object in getpeerinfo
There is no situation in which CNodeStateStats could be
missing for a legitimate reason - this can only happen if
there is a race condition between peer disconnection and
the getpeerinfo call, in which case the disconnected peer
doesn't need to be included in the response.
2022-11-28 13:45:26 -05:00
MarcoFalke
d415b7261c Merge bitcoin/bitcoin#26588: ci: Skip COMMIT_RANGE if no CIRRUS_PR
fad1c55301 lint: Skip COMMIT_RANGE if no CIRRUS_PR (MarcoFalke)

Pull request description:

  It doesn't make sense to run this for non-PRs, because:

  * There are known whitespace "violations" in previous commits, so the lint may fail
  * Once the changes are merged, it is too late to fix them up (force pushes are illegal)
  * It isn't possible to determine which commits to run on if there is no reference branch (target branch of the pull request)

  Moreover, the test fails on non-master:

  * https://github.com/bitcoin/bitcoin/runs/8664441400

  Fix all issues by skipping it.

ACKs for top commit:
  hebasto:
    ACK fad1c55301, also tested in my personal Cirrus account.

Tree-SHA512: be15f00e2b2a9069583833545883e0e5968a33d2455dad59e6fb47c1102b4dd16ef932e9ba945e29e9d941e6c17bd531a02c66b0491097801be6bda476875537
2022-11-28 17:18:29 +01:00
Andrew Chow
8597260872 Merge bitcoin/bitcoin#26480: test: Remove wallet option from non-wallet tests
fa10f193b5 test: Set default in add_wallet_options if only one type can be chosen (MacroFake)
555519d082 test: Remove wallet option from non-wallet tests (MacroFake)
fac8d59d31 test: Set -disablewallet when no wallet has been compiled (MacroFake)
fa68937b89 test: Make requires_wallet private (MacroFake)

Pull request description:

  The tests have several issues:

  * Some tests that are wallet-type specific offer the option to run the test with the incompatible type

  For example, `wallet_dump.py` offers `--descriptors` and on current master fails with `JSONRPCException: Invalid public key`. After the changes here, it fails with a clear error: `unrecognized arguments: --descriptors`.

  * Tests that don't use the wallet at all offer the option to run it with a wallet type. This is confusing and wastes developers time if they are "tricked" into running the test for both wallet types, even though no wallet code is executed at all.

  For example, `feature_addrman.py` will happily accept and run with `--descriptors` or `--legacy-wallet`. After the changes here, it no longer silently ignores the flag, but reports a clear error: `unrecognized arguments`.

ACKs for top commit:
  achow101:
    ACK fa10f193b5

Tree-SHA512: a5784da7305f4ec58c0013f433289000d94fc3d434b00fc329ffa37b812e2cd1da0071e34c3462bf79d904808564f2ae6d3d582f6b86b26215f9b07391b58460
2022-11-28 11:16:49 -05:00
fanquake
5488dc1eb4 Merge bitcoin/bitcoin#26073: build: fix depends bdb compilation for BSDs
5b2529b269 build: fix depends bdb compilation for BSDs (fanquake)

Pull request description:

  Currently, building bdb for *BSD HOSTs in depends fails with:
  ```bash
  libtool: compile:  clang -m64 -c -I. -I../dist/./.. -I/home/ubuntu/bitcoin/depends/x86_64-unknown-freebsd/include -D_THREAD_SAFE -pipe -std=c11 -O2 -Wno-error=implicit-function-declaration -Wno-error=format-security ../dist/./../mutex/mut_tas.c  -fPIC -DPIC -o mut_tas.o
  In file included from ../dist/./../mutex/mut_tas.c:11:
  In file included from ./db_int.h:884:
  In file included from ../dist/./../dbinc/mutex.h:15:
  ../dist/./../dbinc/mutex_int.h:932:2: error: unknown type name 'pthread_rwlock_t'
          MUTEX_FIELDS                    /* Opaque thread mutex structures. */
          ^
  ../dist/./../dbinc/mutex_int.h:65:3: note: expanded from macro 'MUTEX_FIELDS'
                  pthread_rwlock_t rwlock;        /* Read/write lock */   \
                  ^
  1 error generated.
  ```

  Defining `_XOPEN_SOURCE` >=600 fixes access to the missing `pthread_rwlock_t` definitions.

ACKs for top commit:
  jarolrod:
    ACK 5b2529b269

Tree-SHA512: 76584bfef7a86c69a095eb722657aa681b95658a031ef2da79d91a616e5370da292a65e7d67cdf641711791422b8a407b647b630ad497ffb9908683479e6cfb6
2022-11-28 13:27:48 +00:00
MarcoFalke
5939794940 Merge bitcoin/bitcoin#26589: test: small fixups/improvements for get_previous_releases.py
9b5feb76bc script: small fixups/improvements for get_previous_releases.py (Sebastian Falbesoner)

Pull request description:

  This is a small follow-up to #25650 (commit 614d4682ba) with three fixes/improvements:

  - fix "Checksum did not match" detection, which was not adapted to the new `SHA256_SUMS` structure and hence never executed (the list of tarball names isn't directly in the dictionary's values anymore, but has to be extracted from the `'tarball'` field of each value)
  - make both help text and default tag download order deterministic by sorting default tags
  - `--tags` argument help text: add missing space between "for" and "backwards"

ACKs for top commit:
  Sjors:
    tACK 9b5feb76bc. Tested that if I change a checksum, or remove a release, it catches that.
  josibake:
    tested ACK 9b5feb76bc

Tree-SHA512: 791fa693477eebbda7fd41f3f5ec78fe7eab57df06979aa907ab258a6945534bdc3b931ddfce0fb440c9666b98c88ce5e1b6dc353ed39e129e87d3634855165c
2022-11-28 13:59:54 +01:00
glozow
00c323610a Merge bitcoin/bitcoin#25986: test: refactor RPCPackagesTest to use MiniWallet
17cad44851 test: refactor `RPCPackagesTest` to use `MiniWallet` (w0xlt)

Pull request description:

  This PR refactors `RPCPackagesTest` to use `MiniWallet` and removes `create_child_with_parents`, `make_chain`, and `create_raw_chain` from `test_framework/wallet`, as requested in https://github.com/bitcoin/bitcoin/issues/25965.

  Close https://github.com/bitcoin/bitcoin/issues/25965.

ACKs for top commit:
  glozow:
    ACK 17cad44851
  pablomartin4btc:
    tested ACK 17cad44; went thru all changes and recommendations from @kouloumos & @glozow; also went up to #20833 to get a bit of background of the origin and purpose of these tests.
  kouloumos:
    ACK 17cad44851

Tree-SHA512: 9228c532afaecedd577019dbc56f8749046d66f904dd69eb23e7ca3d7806e2132d90af29be276c7635fefb37ef348ae781eb3b225cd6741b20300e6f381041c3
2022-11-28 11:54:52 +00:00
willcl-ark
a27a445b71 refactor: Sub-folder bash completions
Move bash completions to
contrib/completions/bash/*

Precursor to adding fish completions
2022-11-28 11:44:08 +00:00
glozow
a79b720092 Merge bitcoin/bitcoin#26295: Replace global g_cs_orphans lock with local
7082ce3e88 scripted-diff: rename and de-globalise g_cs_orphans (Anthony Towns)
733d85f79c Move all g_cs_orphans locking to txorphanage (Anthony Towns)
a936f41a5d txorphanage: make m_peer_work_set private (Anthony Towns)
3614819864 txorphange: move orphan workset to txorphanage (Anthony Towns)
6f8e442ba6 net_processing: Localise orphan_work_set handling to ProcessOrphanTx (Anthony Towns)
0027174b39 net_processing: move ProcessOrphanTx docs to declaration (Anthony Towns)
9910ed755c net_processing: Pass a Peer& to ProcessOrphanTx (Anthony Towns)
89e2e0da0b net_processing: move extra transactions to msgproc mutex (Anthony Towns)
ff8d44d196 Remove unnecessary includes of txorphange.h (Anthony Towns)

Pull request description:

  Moves extra transactions to be under the `m_msgproc_mutex` lock rather than `g_cs_orphans` and refactors orphan handling so that the lock can be internal to the `TxOrphange` class.

ACKs for top commit:
  dergoegge:
    Code review ACK 7082ce3e88
  glozow:
    ACK 7082ce3e88 via code review and some [basic testing](https://github.com/glozow/bitcoin/blob/review-26295/src/test/orphanage_tests.cpp#L150). I think putting txorphanage in charge of handling peer work sets is the right direction.

Tree-SHA512: 1ec454c3a69ebd45ff652770d6a55c6b183db71aba4d12639ed70f525f0035e069a81d06e9b65b66e87929c607080a1c5e5dcd2ca91eaa2cf202dc6c02aa6818
2022-11-28 10:59:02 +00:00
fanquake
9c2854cda4 Merge bitcoin/bitcoin#26578: doc: install binutils, not binutils-gold in depends
0953c622f9 doc: install binutils, not binutils-gold in depends (fanquake)

Pull request description:

  We don't use the gold linker.
  binutils-gold just installs binutils (and the gold linker) in any case.

ACKs for top commit:
  hebasto:
    ACK 0953c622f9.

Tree-SHA512: c6e44c8d2fe9ef184aeb3b8aec87bac9c49ed0c3ef67de630b12da2945552b3dc4b9b3e75203a7d24a94d9586d59bb638e98f712dbc477681c95a793ee2327d9
2022-11-28 10:29:19 +00:00
MarcoFalke
fad1c55301 lint: Skip COMMIT_RANGE if no CIRRUS_PR 2022-11-28 11:09:30 +01:00
MarcoFalke
c239d3dac9 Merge bitcoin/bitcoin#26574: ci: use ci_exec_root for clang install
54dd8f51ce ci: use ci_exec_root for clang install (josibake)

Pull request description:

  fixes a bug introduced in #25900 ; see https://github.com/bitcoin/bitcoin/pull/25900#issuecomment-1327311069

  the general idea of #25900 was to use a non-root user as much as possible to avoid modifying the user's local filesystem. however, it appears the root user is needed to correctly install clang.

ACKs for top commit:
  hebasto:
    ACK 54dd8f51ce, tested on Ubuntu 22.04.

Tree-SHA512: beb01d4b6127fbba3c8d18e85cf7ec7d1b2ec93ea05c475ab51bcaa04ef1b0591d886f1a7e0732c5ae86806013f022c0b44027380d2b0cfb1bfdc843e40f99b4
2022-11-28 10:20:21 +01:00
MarcoFalke
19baf0178b Merge bitcoin/bitcoin#26498: doc: Mention required workload when building with MSVC
f97892d21a doc: Mention required workload when building with MSVC (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  sipsorcery:
    ACK f97892d21a.

Tree-SHA512: c3a77c478cefb262c9c18d1d7820d2ffee61fc8f2d2943eee7cfcbc581d9665f95f0f9242d1bf23f012cc4f5104458f2257f492bc3aac8b3f6c02082308dda76
2022-11-28 10:13:33 +01:00
Sebastian Falbesoner
9b5feb76bc script: small fixups/improvements for get_previous_releases.py
This is a small follow-up to #25650 (commit
614d4682ba) with three fixes/improvements:

- fix "Checksum did not match" detection, which was not adapted to the new
  SHA256_SUMS structure and hence never executed (the list of tarball
  names isn't directly in the dictionary's values anymore, but has to be
  extracted from the 'tarball' field of each value)
- make both help text and default tag download order deterministic by
  sorting default tags
- "--tags" argument help text: add missing space between "for" and
  "backwards"
2022-11-28 03:22:01 +01:00
John Moffett
e75d227632 Minor fix: Don't directly delete abandoned txes
This fully closes bitcoin#12179. Currently, in the GUI, when a user
abandons a transaction, a call is made to remove it from the list,
and another signal fires (eventually) that adds it back to the GUI
with a trash can icon.

There are no conditions where the abandoned transaction should be
directly removed from the GUI. If the underlying model changes, the
deletion will be reflected anyway.
2022-11-25 15:56:40 -05:00
fanquake
9c47eb4503 Merge bitcoin/bitcoin#26446: build: Drop unneeded linking of contrib/devtools/ scripts
29ef26ae25 build: Drop unneeded linking of `contrib/devtools/` scripts (Hennadii Stepanov)
77779c3717 script: Improve `test-{security,symbol}-check.py` robustness (Hennadii Stepanov)

Pull request description:

  The build system targets `make test-security-check`, `make -C src check-security` and `make -C src check-symbols` run `contrib/devtools/{test-,}{security,symbol}-check.py` scripts from the top source directory, i.e. `$(top_srcdir)` in the current Autotools-based build system.

  This renders needless of linking of those scripts into the build directory.

  Both build systems, the current Autotools-based and the future CMake-based, benefit from this simplification.

ACKs for top commit:
  fanquake:
    ACK 29ef26ae25

Tree-SHA512: 442b6aa116615d01eabc58b6ded67d9c6993033a071bb7008afdb956c468b65bb2b51705aeaed60fd68211dd2b9c8b8e2234babd45abd022daff391c00091165
2022-11-25 17:32:18 +00:00
fanquake
0953c622f9 doc: install binutils, not binutils-gold in depends
We don't use the gold linker.
binutils-gold just installs binutils (and the gold linker) in any case.
2022-11-25 17:20:05 +00:00
fanquake
0fe225e378 Merge bitcoin/bitcoin#26470: guix: Clean up libexec/build.sh
d3e5779f11 guix: Drop non-existent directories (Hennadii Stepanov)
43c4afc407 guix: Drop no longer used `prepend_to_search_env_var()` (Hennadii Stepanov)

Pull request description:

  1. The `prepend_to_search_env_var()` function was introduced in c1ae726a13, and it has no longer been used since 1dd8cbfbc6.

  2. The `${NATIVE_GCC}/lib64` and `${NATIVE_GCC_STATIC}/lib64` do not exist at all.

  Guix builds:
  ```
  0a1ed84ba127853fdeb14e6104e97d853116bf3ac9fa4cdd4b72c428ca738f7a  guix-build-d3e5779f110a/output/aarch64-linux-gnu/SHA256SUMS.part
  c69fddab6384ece4061606109acce2f4f991129b5ad2005e9977bc74ad88f60c  guix-build-d3e5779f110a/output/aarch64-linux-gnu/bitcoin-d3e5779f110a-aarch64-linux-gnu-debug.tar.gz
  b8a0968e7057ed6b1703cc5b5213178b06c51c7649c2639d32523f049b082a69  guix-build-d3e5779f110a/output/aarch64-linux-gnu/bitcoin-d3e5779f110a-aarch64-linux-gnu.tar.gz
  9624c5f331d6e2e4f6bb63a3573ff8f688e2662bf8a1959068511ee76fea1b31  guix-build-d3e5779f110a/output/arm-linux-gnueabihf/SHA256SUMS.part
  4b2fecfac94aa55489a9e7316944a1c0216750bfc8658e8dbe2266ad6abed7cd  guix-build-d3e5779f110a/output/arm-linux-gnueabihf/bitcoin-d3e5779f110a-arm-linux-gnueabihf-debug.tar.gz
  db413d3ed5e4917a236dea10d121953c01812d7f3202bf503fb3a7bb40e9655e  guix-build-d3e5779f110a/output/arm-linux-gnueabihf/bitcoin-d3e5779f110a-arm-linux-gnueabihf.tar.gz
  f4952abe163f536c085a3ad9249840f940f5a2670b77042e2c355886301174a0  guix-build-d3e5779f110a/output/arm64-apple-darwin/SHA256SUMS.part
  3f9b76ed00e90547e980062114de435c10b0253a3fa3bca867758c413f68d465  guix-build-d3e5779f110a/output/arm64-apple-darwin/bitcoin-d3e5779f110a-arm64-apple-darwin-unsigned.dmg
  ec30943bf796c7c55f9d5fb186054070483b888c87d1c79a02ba2828c85d9643  guix-build-d3e5779f110a/output/arm64-apple-darwin/bitcoin-d3e5779f110a-arm64-apple-darwin-unsigned.tar.gz
  31fccaf6ff6f70ef3fe09c592f2630f5b2bc39f5c8eadf4f7660e379389442be  guix-build-d3e5779f110a/output/arm64-apple-darwin/bitcoin-d3e5779f110a-arm64-apple-darwin.tar.gz
  6df55d847ad0f3c4cdd35c3fc9603ab58e44bf76a4a942c4f2f268fa02e710e1  guix-build-d3e5779f110a/output/dist-archive/bitcoin-d3e5779f110a.tar.gz
  1e264b6cd129e1353c271049fd3c8338a1d2183aea847cb4ea9d4095fba56a54  guix-build-d3e5779f110a/output/powerpc64-linux-gnu/SHA256SUMS.part
  03d325d8bffa72e79638ae0e5cd7d6c761cdc5e9d664158c83a5f9ff9f393634  guix-build-d3e5779f110a/output/powerpc64-linux-gnu/bitcoin-d3e5779f110a-powerpc64-linux-gnu-debug.tar.gz
  8d459c2fca6e6712cde54ce030cddae9c3fa0607dd9fb4a951f8d62f3e456162  guix-build-d3e5779f110a/output/powerpc64-linux-gnu/bitcoin-d3e5779f110a-powerpc64-linux-gnu.tar.gz
  0ef58af43e18dc7490ea8f01d392e6645b7f978817a8fb0727d90559680a0105  guix-build-d3e5779f110a/output/powerpc64le-linux-gnu/SHA256SUMS.part
  fe4be346506e5f9cc3c5f8a952f4d4b2ac13a2562854d26deee536f411847189  guix-build-d3e5779f110a/output/powerpc64le-linux-gnu/bitcoin-d3e5779f110a-powerpc64le-linux-gnu-debug.tar.gz
  52c0961bfca3985899f872145c566b5220245096a3edc94347800e1de199a17e  guix-build-d3e5779f110a/output/powerpc64le-linux-gnu/bitcoin-d3e5779f110a-powerpc64le-linux-gnu.tar.gz
  ade3eb3f496a5904d12cb09095e7f05c69961d759aba1acc82e01bcfcc2c6ef1  guix-build-d3e5779f110a/output/riscv64-linux-gnu/SHA256SUMS.part
  05a9d0921b5706e210fb0b5a1a70890604ee22a7bbdc25f011cd1bbff1420120  guix-build-d3e5779f110a/output/riscv64-linux-gnu/bitcoin-d3e5779f110a-riscv64-linux-gnu-debug.tar.gz
  fb897b1b527450da6a033802c218722d69176c2572c321ad3e18b2ef3bf398ed  guix-build-d3e5779f110a/output/riscv64-linux-gnu/bitcoin-d3e5779f110a-riscv64-linux-gnu.tar.gz
  a76da4a5d3fd3ed98adbe1e4f1b1c974937e401801d65b55a14e21339b072f9d  guix-build-d3e5779f110a/output/x86_64-apple-darwin/SHA256SUMS.part
  3eb30a851466d4dcf08fbf1d94071d8d6a253fee52540ab109ae9cf629cb36d2  guix-build-d3e5779f110a/output/x86_64-apple-darwin/bitcoin-d3e5779f110a-x86_64-apple-darwin-unsigned.dmg
  6a350a4ea7afbdfba07d1d51039d9c3da5f5ff0a3d4e40095006d0cb571a5c65  guix-build-d3e5779f110a/output/x86_64-apple-darwin/bitcoin-d3e5779f110a-x86_64-apple-darwin-unsigned.tar.gz
  32761153c68431c3bf1eb6638caa00bd4d7daf7372cdea2a9cc711c4c1f68326  guix-build-d3e5779f110a/output/x86_64-apple-darwin/bitcoin-d3e5779f110a-x86_64-apple-darwin.tar.gz
  79fa0b6b25a74c4681679a04ceb9fab1ae0e339ed9e44a30e3c84ee019255e85  guix-build-d3e5779f110a/output/x86_64-linux-gnu/SHA256SUMS.part
  dd496a120f99a818fe453f23f607dd064c855e6d5d199870ce3ab924aca6fed0  guix-build-d3e5779f110a/output/x86_64-linux-gnu/bitcoin-d3e5779f110a-x86_64-linux-gnu-debug.tar.gz
  6dc03d9ad75126ad67c29afbf3ef12e99a97f0792cd8fac6407a2f4ca911f9d1  guix-build-d3e5779f110a/output/x86_64-linux-gnu/bitcoin-d3e5779f110a-x86_64-linux-gnu.tar.gz
  32fb6e76ed757d9d4693c3ef2cea7bd7b9151cadbf9976ed64ab1e3a7ee71e9e  guix-build-d3e5779f110a/output/x86_64-w64-mingw32/SHA256SUMS.part
  93bbc60e546571e14268e25efcf83375c5ec4ed13c546cbf5b76d849a0f50f35  guix-build-d3e5779f110a/output/x86_64-w64-mingw32/bitcoin-d3e5779f110a-win64-debug.zip
  b85932e7cde8616c2d7342735ad8c6247d556fa86ccee8d0ecd326debeb93d8c  guix-build-d3e5779f110a/output/x86_64-w64-mingw32/bitcoin-d3e5779f110a-win64-setup-unsigned.exe
  f77431e1d5b7a356c065dd81383222116cf8254bda0666a169b25415cfba1810  guix-build-d3e5779f110a/output/x86_64-w64-mingw32/bitcoin-d3e5779f110a-win64-unsigned.tar.gz
  8d1a43386648ef5d97aae806f0e38f7fa3ec5f39b30b495fdcf6be6590a392eb  guix-build-d3e5779f110a/output/x86_64-w64-mingw32/bitcoin-d3e5779f110a-win64.zip
  ```

ACKs for top commit:
  fanquake:
    ACK d3e5779f11

Tree-SHA512: bd32442c81146a5558a4f5bc79e1eb841e7c920551289a2c910f8a697c0a70ea26926e9ca7cdb486176b772ee42544559939417a2bdf3ab6cc4b65dba816a087
2022-11-25 17:17:28 +00:00
fanquake
32599766ca Merge bitcoin/bitcoin#26335: Guix documentation improvements
10660c0c60 doc: move Guix uninstall instructions to INSTALL.md (Sjors Provoost)
68fab72a8c guix: OpenSSL test failure workaround (Sjors Provoost)
d612dca852 guix: reminder to migrate guix-daemon-original customization (Sjors Provoost)
8aa460cd02 guix: add guile-gnutls and guile-json to install list (Sjors Provoost)
9b9991e026 guix: recommend mounting a tmpfs on /tmp (Sjors Provoost)
682283445e guix: bump recommended hash for manual installation (Sjors Provoost)

Pull request description:

  I'm manually installing Guix on a fresh Ubuntu machine. Will be pushing more documentation fixes to this PR as I run into things.

  1. Bump minimum hash to match time-machine bump in #25099. It's not necessary for the root Guix version to match the time-machine version in our build, because `guix build` will automatically perform an upgrade for the user, but imo it's better to get any build issues (in Guix itself) over with while the user is going though `INSTALL.md`, rather than during their first Guix build (of Bitcoin Core).
  2. Recommend mapping a tmpfs to /tmp upfront, rather than in the troubleshooting section
  3. Add `guile-gnutls` and `guile-json` to the table of stuff to install (avoids having to find out in the `./configure` phase)
  4. Improve systemd doc
  5. Workaround OpenSSL v1.1.1l and v1.1.1n test failure (change machine time)
  6. Move uninstallation instructions to INSTALL.md, drop unused footnote / links

ACKs for top commit:
  jamesob:
    ACK 10660c0c60

Tree-SHA512: ff1278b16f03ea9c63e23e97a852340ab824d5f6c64645cb70237dd828b9a439b4133b60cd2b89672573f6546e99419021d092e236f731908158a7aa6473b0ef
2022-11-25 16:48:20 +00:00
fanquake
93cae70d87 Merge bitcoin/bitcoin#26561: fuzz: Move-only net utils
fa3b2cf277 fuzz: Move-only net utils (MarcoFalke)

Pull request description:

  This should speed up fuzz builds when `src/test/fuzz/util.h` is modified. Also, it makes sense on its own.

ACKs for top commit:
  dergoegge:
    ACK fa3b2cf277

Tree-SHA512: 03d6abeb728ac8eb3f28167e8ac43d8d6e7e1b1738ec14f58a36e17502081fdde2d56f2d47a9e11b991754667e83b2eb22d154e394c0c1c4ffa0945db86b7e21
2022-11-25 16:47:17 +00:00
fanquake
bc67215b29 Merge bitcoin/bitcoin#26558: doc: add tr() descriptor example to deriveaddresses
92a4ed05d1 doc: add tr() descriptor example to deriveaddresses (FractalEncrypt)

Pull request description:

  This simple PR adds a missing tr() descriptor example to the `help deriveaddresses` examples.

  - The functionality added in https://github.com/bitcoin/bitcoin/pull/24043 is a significant departure from legacy multisig address creation, yet there is no corresponding tr() descriptor example in the help.
  - Having this example in combination with the examples in the descriptors documentation will be helpful to users.

  I needed this information to correctly create a tr multisig address but was unable. I had to leave the software and use a 3rd party site to ask two separate questions ([1](https://bitcoin.stackexchange.com/questions/115700/how-do-i-create-a-taproot-multisig-address-requiring-21-of-210-keys-to-spend), [2](https://bitcoin.stackexchange.com/questions/115742/signing-psbts-to-spend-from-taproot-multisig-address)) to create an address using the new functionality.

  Note: This specific example is not provided in the [descriptors.md ](https://github.com/bitcoin/bitcoin/blob/master/doc/descriptors.md) documentation, though there is a similar example with `sortedmulti_a. `

ACKs for top commit:
  instagibbs:
    ACK 92a4ed05d1
  kouloumos:
    ACK 92a4ed05d1
  w0xlt:
    ACK 92a4ed05d1

Tree-SHA512: 8fb052bd469718157cb64439b885f8b0ecfb5a798535a02bae0a5dc748cd554a3e5ffdd9fe4acaef16156eadb59e1b2bcde7356e811397225f2783a84c8b112f
2022-11-25 16:39:40 +00:00
josibake
54dd8f51ce ci: use ci_exec_root for clang install 2022-11-25 14:13:29 +01:00
@RandyMcMillan
dddfa028ff contrib/builder-keys/keys.txt: remove unavailable key
remove unavailable builder-key:
A2FD494D0021AA9B4FA58F759102B7AE654A4A5A Ilyas Ridhuan (IlyasRidhuan)
2022-11-23 21:29:33 -05:00
Martin Zumsande
956c67059c refactor, doc: Improve SetupAddressRelay call in version processing
This code was a bit hard to understand, so make it less dense and
add more explanations. Doesn't change behavior.

Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2022-11-23 16:11:43 -05:00
MarcoFalke
fa3b2cf277 fuzz: Move-only net utils 2022-11-23 17:26:01 +01:00
FractalEncrypt
92a4ed05d1 doc: add tr() descriptor example to deriveaddresses
add a tr() descriptor example to the help deriveaddresses examples
2022-11-23 10:17:29 -05:00
fanquake
38d06e1561 Merge bitcoin/bitcoin#26383: test: Add feature_taproot case involving invalid internal pubkey
5d413c8e79 Add feature_taproot case involved invalid internal pubkey (Pieter Wuille)

Pull request description:

  Add a test case to feature_taproot which involves an output that is (incorrectly) constructed, using an invalid internal public key and valid script tree. It is designed to detect cases where the script path spending validation logic does not detect this case, and instead treats the internal public key as the point at infinity.

  Equivalent unit test case added in https://github.com/bitcoin-core/qa-assets/pull/98.

ACKs for top commit:
  instagibbs:
    ACK 5d413c8e79
  aureleoules:
    reACK 5d413c8e79

Tree-SHA512: dfa014e383cd2743f3c9a996e1f2a2fceb9e244edf4b05dc0c110c4ba32a87684482222907805a4ca998aebcb42a197bb3e7967bfb5f0554fe9f1e5aa5463603
2022-11-22 16:31:05 +00:00
MarcoFalke
85892f77c9 Merge bitcoin/bitcoin#25900: ci: run docker wrapper with a non-root user
849f20a6d3 ci: create and use non-root user for docker image (josibake)

Pull request description:

  Previously, everything in the ci docker image ran as the root user. This would lead to certain directories (`ci/scratch`, `depends`) being owned by `root` after running the ci locally which would lead to annoying behavior such as subsequent guix builds failing due to `depends/` being owned by root.

  This PR adds a non-root user in the container and chowns the mounted working directory. All the `docker exec` commands now run as the non-root user, except for the few that still need to run as root (mainly, installing packages).

  To test this I checked out a fresh copy of the repo, applied my changes, ran the CI, and verified all the local file permissions were unchanged after the CI was finished running.

ACKs for top commit:
  hebasto:
    ACK 849f20a6d3, tested on Ubuntu 22.04 by running commands as follows:

Tree-SHA512: 734dca0f36157fce5fab243b4ff657fc17ba980e8e4e4644305f41002ff21bd5cef02c306ea1e0b5c841d4c07c095e8e4be16722e6a38c890717c60a3f5ec62a
2022-11-22 12:46:40 +01:00
fanquake
1ef498f75b Merge bitcoin/bitcoin#26520: doc: test: update/fix TestShell example instructions
31d0067f8b doc: test: update/fix TestShell example instructions (Sebastian Falbesoner)

Pull request description:

  This PR tackles two issues in the TestShell documentation:
  - add missing instruction for creating a wallet prior to the `getnewaddress` call (needed as there is no default wallet created anymore since v0.21)
  - fix `generatetoaddress` call syntax (the scripted-diff in commit fa0b916971 only worked for tests using `BitcoinTestFramework`)

ACKs for top commit:
  fanquake:
    ACK 31d0067f8b - current instructions don't work. These do.

Tree-SHA512: d2b7808a06892ad16728cb2b6d4a72b255ad711d27fe98b1de562f80444e7bb25d73296abdde4308162fe3be702864e2f7b7dbbbb000fe54c709951c09e6c730
2022-11-22 10:53:04 +00:00
MarcoFalke
164027f824 Merge bitcoin/bitcoin#26100: doc: clarify that NetPermissionFlags::Implicit is only about whitelists
f362920c2c doc: clarify that NetPermissionFlags::Implicit is only about whitelists (Vasil Dimov)

Pull request description:

  `NetPermissionFlags::Implicit` applies just to connections from `-whitebind` or `-whitelist`, clarify that in its comment.

ACKs for top commit:
  Zero-1729:
    crACK f362920c2c
  aureleoules:
    ACK f362920c2c
  hernanmarino:
    re ACK f362920c2c

Tree-SHA512: 03f6f8be221c6819bdd0b5b56b69b4e3a6dd25e5ca5a247eeb1261113144b9b74cf064a0b7815317782a0a18365dd3dab97963bd238e9b231dbe7e1cf0395683
2022-11-22 11:44:43 +01:00
MarcoFalke
7eeae5c023 Merge bitcoin/bitcoin#26553: test: Fix intermittent failure in rpc_net.py
0f6cd72237 test: Fix intermittent failure in rpc_net.py (Martin Zumsande)

Pull request description:

  Fixes #26552.

  The problem was that calling `disconnect_p2ps` waits until `self.num_test_p2p_connections() == 0`.
  `num_test_p2p_connections()` checks the field `subver` in `getpeerinfo` to distinguish p2p nodes from full nodes.
  However, if we are dealing with a p2p connection that has never sent a version, the node has never received the special subversion and the wait is ineffective (we continue even though the disconnection is not yet completed).

  Fix this by not using `disconnect_p2ps`.

ACKs for top commit:
  MarcoFalke:
    ACK 0f6cd72237

Tree-SHA512: ebdc78498db6971ae2f9b494dc76b35de46155bf191ce82ee04162592d0d9ec1272901992406d530fa46fb52cd815c4b91350824578292df14986584bc60b90a
2022-11-22 11:22:01 +01:00
fanquake
1b680948d4 Merge bitcoin/bitcoin#26292: util: move threadinterrupt into util/
b89530483d util: move threadinterrupt into util (fanquake)

Pull request description:

  Alongside thread and threadnames. It's part of libbitcoin_util.

ACKs for top commit:
  ryanofsky:
    Code review ACK b89530483d. No changes since last review other than rebase
  theuni:
    ACK b89530483d.

Tree-SHA512: 0421f4d1881ec295272446804b27d16bf63e6b62b272f8bb52bfecde9ae6605e8109ed16294690d3e3ce4b15cc5e7c4046f99442df73adb10bdf069d3fb165aa
2022-11-22 09:52:53 +00:00
fanquake
542a2b567d Merge bitcoin/bitcoin#26376: test: Use type-safe NodeSeconds for TestMemPoolEntryHelper
fa2d01470a test: Use type-safe NodeSeconds for TestMemPoolEntryHelper (MacroFake)

Pull request description:

  test-only refactor to drop the deprecated `GetTime` in favour of the type-safe alternative

ACKs for top commit:
  aureleoules:
    ACK fa2d01470a - verified that there is no behavior change

Tree-SHA512: 5b64dae19c7bba9e8d90377c85891bc86f60ffbe67ea28d5ed3bd38f6dc30d3fbfba00bf49a16792922bddf83a52c632b6e5e5d8ffe1619fd9bf63effc60d59a
2022-11-22 09:50:33 +00:00
Martin Zumsande
0f6cd72237 test: Fix intermittent failure in rpc_net.py
The wait in disconnect_p2ps checked for subver, which
is unavailable for the peer that didn't send a version msg.
2022-11-21 18:10:25 -05:00
furszy
13d9760829 test: load wallet, coverage for crypted keys
Adds test coverage for the wallet's crypted key loading from db process.
The following scenarios are covered:

1) "All ckeys checksums valid" test:
   Loads an encrypted wallet with all the crypted keys with a valid checksum and
   verifies that 'CWallet::Unlock' doesn't force an entire crypted keys re-write.

   (we force a complete ckeys re-write if we find any missing crypted key checksum
    during the wallet loading process)

2) "Missing checksum in one ckey" test:
   Verifies that loading up a wallet with, at least one, 'ckey' with no checksum
   triggers a complete re-write of the crypted keys.

3) "Invalid ckey checksum error" test:
   Verifies that loading up a ckey with an invalid checksum stops the wallet loading
   process with a corruption error.

4) "Invalid ckey pubkey error" test:
   Verifies that loading up a ckey with an invalid pubkey stops the wallet loading
   process with a corruption error.
2022-11-21 17:30:00 -03:00
furszy
373c99633e refactor: move DuplicateMockDatabase to wallet/test/util.h 2022-11-21 17:30:00 -03:00
furszy
ee7a984f85 refactor: unify test/util/wallet.h with wallet/test/util.h
files share the same purpose, and we shouldn't have wallet code
inside the test directory.

This later is needed to use wallet util functions in the bench
and test binaries without be forced to duplicate them.
2022-11-21 17:30:00 -03:00
Pieter Wuille
5d413c8e79 Add feature_taproot case involved invalid internal pubkey 2022-11-21 14:00:43 -05:00
Russell O'Connor
dee89438b8 Abstract out ComputeTapbranchHash 2022-11-21 12:38:53 -05:00
Russell O'Connor
8e3fc99427 Do not use CScript for tapleaf scripts until the tapleaf version is known
Prevents use of CScript methods until the tapleaf is known to be a tapscript.
2022-11-21 12:38:53 -05:00
josibake
849f20a6d3 ci: create and use non-root user for docker image
Running all commands as the root user in the docker image
will change local file permissions in the ci and depends directory.

Add a non-root user to the container and use this user whenever
possible when running docker exec commands.
2022-11-21 18:11:28 +01:00
MacroFake
60a00889b0 Merge bitcoin/bitcoin#26545: test: Remove unused sanitizer suppressions
fadb714039 test: Remove unused sanitizer suppressions (MacroFake)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK fadb714039
  hebasto:
    ACK fadb714039

Tree-SHA512: 28d4d2eacdbad567434f0f792dbcaa424344dbd13e8686c72f1cbf8a79343b6ee9661246b13ab3f295757269861405315f42180ef49b69411d0b45f9b7796dd7
2022-11-21 17:27:34 +01:00
fanquake
01a66e21ef Merge bitcoin/bitcoin#26546: test: remove unused class NodePongAdd1
40bdc8a6e4 test: remove unused class `NodePongAdd1` (Sebastian Falbesoner)

Pull request description:

  This class was introduced in commit fa3365430c ("net: Use mockable time for ping/pong, add tests"), but actually never used.

ACKs for top commit:
  stickies-v:
    ACK 40bdc8a6e

Tree-SHA512: b5a6552e4f2e0b7e368a071cc53b9a8e6f5d1950565a9fda8eb1971a01d8be0541d066842723ef44174fe8189925fa36f2defb6d7bf8d104abc77de410cc4c13
2022-11-21 15:32:23 +00:00
MacroFake
0968c51401 Merge bitcoin/bitcoin#26497: fuzz: Make ConsumeNetAddr always produce valid onion addresses
0eeb9b0442 [fuzz] Move ConsumeNetAddr to fuzz/util/net.h (dergoegge)
291c8697d4 [fuzz] Make ConsumeNetAddr produce valid onion addresses (dergoegge)
c9ba3f836e [netaddress] Make OnionToString public (dergoegge)

Pull request description:

  The chance that the fuzzer is able to guess a valid onion address is probably slim, as they are Base32 encoded and include a checksum.  Right now, any target using `ConsumeNetAddr` would have a hard time uncovering bugs that require valid onion addresses as input.

  This PR makes `ConsumeNetAddr` produce valid onion addresses by using the 32 bytes given by the fuzzer as the pubkey for the onion address and forming a valid address according to the torv3 spec.

ACKs for top commit:
  vasild:
    ACK 0eeb9b0442
  brunoerg:
    ACK 0eeb9b0442

Tree-SHA512: 7c687a4d12f9659559be8f0c3cd4265167d1261d419cfd3d503fd7c7f207cc0db745220f02fb1737e4a5700ea7429311cfc0b42e6c15968ce6a85f8813c7e1d8
2022-11-21 14:35:20 +01:00
Sebastian Falbesoner
40bdc8a6e4 test: remove unused class NodePongAdd1
This class was introduced in commit
fa3365430c ("net: Use mockable time for
ping/pong, add tests"), but actually never used.
2022-11-21 13:03:35 +01:00
MacroFake
fadb714039 test: Remove unused sanitizer suppressions 2022-11-21 12:13:42 +01:00
MacroFake
df2f16666c Merge bitcoin/bitcoin#26508: RPC/Blockchain: Minor improvements for scanblocks & scantxoutset docs/errors
f9869843a6 RPC/blockchain: scan{blocks,txoutset>: Further doc improvements (Luke Dashjr)
54b45e155e RPC/Blockchain: Clarify invalid-action error in scanblocks & scantxoutset (Luke Dashjr)

Pull request description:

  * Clarify invalid-action error in scanblocks & scantxoutset
  * Mention action=='start' only returns after scan completes (already in scantxoutset)
  * Document `relevant_blocks`

ACKs for top commit:
  kristapsk:
    utACK f9869843a6
  aureleoules:
    ACK f9869843a6
  MarnixCroes:
    ACK f9869843a6

Tree-SHA512: a37c9cc8a9a2f59376e8d8ed7dbf5e140eb3fefb4b7c19a23fc8190f3aef060bda1f0d5d06dc81cd7dca9e871d65f6c8094bab6e8d42e0bcef0fc7ffd2342d09
2022-11-21 11:32:36 +01:00
MacroFake
295f617988 Merge bitcoin/bitcoin#26505: doc: -getinfo help - grammar correction
cc597bd56d src/bitcoin-cli.cpp: -getinfo help - grammar correction (@RandyMcMillan)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: a5321968d0d377e1481170b4220a1319bf9040ec198b27c011609a5b7a81e9193500b750980c7de423b8b99655ed0f7772a9621e0b230aa6cc5d7b48167ed4f9
2022-11-21 09:34:06 +01:00
glozow
d0b1f613c2 Merge bitcoin/bitcoin#17786: refactor: Nuke policy/fees->mempool circular dependencies
c8dc0e3eaa refactor: Inline `CTxMemPoolEntry` class's functions (Hennadii Stepanov)
75bbe594e5 refactor: Move `CTxMemPoolEntry` class to its own module (Hennadii Stepanov)

Pull request description:

  This PR:
  - gets rid of the `policy/fees` -> `txmempool` -> `policy/fees` circular dependency
  - is an alternative to #13949, which nukes only one circular dependency

ACKs for top commit:
  ryanofsky:
    Code review ACK c8dc0e3eaa. Just include and whitespace changes since last review, and there's a moveonly commit now so it's very easy to review
  theStack:
    Code-review ACK c8dc0e3eaa
  glozow:
    utACK c8dc0e3eaa, agree these changes are an improvement.

Tree-SHA512: 36ece824e6ed3ab1a1e198b30a906c8ac12de24545f840eb046958a17315ac9260c7de26e11e2fbab7208adc3d74918db7a7e389444130f8810548ca2e81af41
2022-11-18 17:04:49 -08:00
Andrew Chow
aeb395dcdb Merge bitcoin/bitcoin#25315: Add warning on first startup if free disk space is less than necessary
6630a1e844 Add warning on first startup if free disk space is less than necessary (Ben Woosley)

Pull request description:

  This reworks/revives https://github.com/bitcoin/bitcoin/pull/15848 to add a check for low disk space on first startup and issue a warning if disk space is below the expected space required to accommodate the blocks.

  This PR was fashioned by a team of developers at the [bitcoin++](https://www.btcplusplus.dev/) conference workshop: "[Let's contribute to Bitcoin Core](https://sched.co/12P6Z)"

  Fixes #15813

ACKs for top commit:
  achow101:
    ACK 6630a1e844
  willcl-ark:
    tACK 6630a1e844 rebased on master. Warning shows on first start but not on restart after some blocks have been downloaded.
  aureleoules:
    ACK 6630a1e844
  pablomartin4btc:
    re-ACK 6630a1e844
  hernanmarino:
    ReACK 6630a1e844

Tree-SHA512: 0f18acabdf2b514e96e2eea8f304960b952226b83dc91334cf7d1f6355ea2f257aaec0ee38d43ac36435385ecd918333d20657c35a8a7407e7cf2680ccb643bb
2022-11-18 11:33:10 -05:00
furszy
cc5a5e8121 wallet: bugfix, invalid crypted key "checksum_valid" set
At wallet load time, we set the crypted key "checksum_valid" variable always to false.
Which, on every wallet decryption call, forces the process to re-write the entire ckeys to db when
it's not needed.
2022-11-18 11:38:56 -03:00
MacroFake
256120d2da Merge bitcoin/bitcoin#26519: test: Add getpeerinfo test for missing version message
fa68d086f3 test: Add getpeerinfo test for missing version message (MacroFake)

Pull request description:

  There seems to be a lot of discussion about behaviour/code that is completely untested.

  Fix this by adding a test. The test documents the current behaviour and helps to detect when the behaviour changes in the future.

ACKs for top commit:
  jonatack:
    ACK fa68d086f3
  mzumsande:
    Code Review ACK fa68d086f3

Tree-SHA512: d092b30d5bdb46712c91a7c5bd2d0c82a0da281f1460967aa4e32c648b15d8d97870ded9565a90af34874eb468aad8b99694a2485af6807994e7cfc05482aa8c
2022-11-18 10:52:51 +01:00
MacroFake
b3c76ab757 Merge bitcoin/bitcoin#26524: doc: add 24.0 release notes
b93beef5ed doc: Mac -> macOS in release notes template (fanquake)
2747adb68a doc: Add 24.0 release notes (fanquake)

Pull request description:

  Same as past releases.

ACKs for top commit:
  stickies-v:
    ACK b93beef5ed

Tree-SHA512: c28bc7286f330a6058ae266b238468044439457ff5b9df191232d91dc17b8092facd6ed72accec8bc9db10f055f7bb7e06700cc1ed0bd045fc15f612bc023a46
2022-11-18 10:26:43 +01:00
MacroFake
cfee93f68a Merge bitcoin/bitcoin#26487: log: improve some validation log messages to include hashPrevBlock
ac410e6fc0 log: improve some validation log messages to include hashPrevBlock (Skuli Dulfari)

Pull request description:

  When there is an issue with a previous block the current log messages do not indicate hashPrevBlock. Adding it makes debugging easier.

ACKs for top commit:
  stickies-v:
    ACK ac410e6fc0
  aureleoules:
    reACK ac410e6fc0
  theStack:
    ACK ac410e6fc0

Tree-SHA512: d91481321f4474bb4fdf6ad55d1c897437b631b0a12308815c4ac5b053c8a76726e2d93f2aa0701e8cfd48fba7fad19ef5ffca3c67d3aa973dc593df806f1757
2022-11-18 09:54:33 +01:00
Skuli Dulfari
ac410e6fc0 log: improve some validation log messages to include hashPrevBlock
When there is an issue with a previous block the current log messages do
not indicate hashPrevBlock. Adding it makes debugging easier.
2022-11-17 16:45:15 +00:00
dergoegge
0eeb9b0442 [fuzz] Move ConsumeNetAddr to fuzz/util/net.h 2022-11-17 14:52:45 +00:00
dergoegge
291c8697d4 [fuzz] Make ConsumeNetAddr produce valid onion addresses 2022-11-17 14:52:45 +00:00
dergoegge
c9ba3f836e [netaddress] Make OnionToString public 2022-11-17 14:52:45 +00:00
fanquake
b93beef5ed doc: Mac -> macOS in release notes template
https://github.com/bitcoin/bitcoin/pull/26503#discussion_r1023782259
2022-11-17 14:43:56 +00:00
fanquake
2747adb68a doc: Add 24.0 release notes 2022-11-17 14:38:22 +00:00
Hennadii Stepanov
fb01af6c77 Merge bitcoin-core/gui#680: Fixes MacOS 13 segfault by preventing certain notifications after main window is destroyed
8a5014cd8a Fixes bitcoin#26490 by preventing notifications (John Moffett)

Pull request description:

  This is a PR to address https://github.com/bitcoin/bitcoin/issues/26490

  The menu bar currently subscribes to window focus change notifications to enable or disable certain menu options in response to the window status.

  Notifications are automatically unsubscribed (disconnected in Qt parlance) if the sender is deleted -- in this case, the sender is the QTApplication object (`qApp`). However, MacOS 13 sends a window focus change notification *after* the main window has been destroyed but *before* `qApp` has been fully destroyed.

  Since the menu bar is deleted in the main window's destructor, it no longer exists when it receives these notifications (in two different places via lambda expressions). The solution is to pass the main window (`this`) as context when subscribing to the notifications. In this [overloaded version](https://doc.qt.io/qt-5/qobject.html#connect-1) of `connect`, Qt automatically unsubscribes to notifications if the sender OR context (here the main window object) is destroyed. Since the spurious notifications are sent after the main window object is destroyed, this change prevents them from being sent.

  Tested on Mac OS 13 and 12 only.

ACKs for top commit:
  hebasto:
    ACK 8a5014cd8a

Tree-SHA512: 3dff0a252fe0e93dd68cf5503135ecf6a72bcf385ba38407d6021ab77cca323f8bbe58aeca90ec124aa2a22ab9d35b706946179ac3b5d171c96a7010de51a090
2022-11-17 14:30:11 +00:00
brunoerg
741c215b5f test: remove unused vars in feature_block 2022-11-17 10:37:21 -03:00
Sebastian Falbesoner
31d0067f8b doc: test: update/fix TestShell example instructions
Tackles two issues in the TestShell documentation:
- add missing instruction for creating a wallet prior to the
  `getnewaddress` call (needed as there is no default wallet created
  anymore since v0.21)
- fix `generatetoaddress` call syntax (the scripted-diff in commit
  fa0b916971 only worked for tests using
  `BitcoinTestFramework`)
2022-11-17 12:04:13 +01:00
MacroFake
fa68d086f3 test: Add getpeerinfo test for missing version message 2022-11-17 11:20:33 +01:00
MacroFake
82fe672ea0 Merge bitcoin-core/gui#681: Fix Transaction Relay tooltip text in Peers details window
7a53033303 Fix Transaction Relay tooltip text in Peers details window (Jon Atack)

Pull request description:

  as a value of N/A could occur due to a lock or a disconnection race but not during connection setup -- see https://github.com/bitcoin/bitcoin/pull/26457#pullrequestreview-1181641835.  Credit to Martin Zumsande for finding this.

ACKs for top commit:
  jarolrod:
    ACK 7a53033303

Tree-SHA512: 031779567e927f05f6fae02394a8c97ba5c45ba9fffd7f1e2c006e152df5f724d92a06f18a4c2540436476eca6b40a3a5cbc4421666cd576439b823668acfcfb
2022-11-17 10:22:36 +01:00
Randall Naar
6fb102c9f3 test: Changed small_txpuzzle_randfee to return the virtual size instead of the transaction hex for feerate calculation. 2022-11-17 00:11:08 -05:00
Jon Atack
7a53033303 Fix Transaction Relay tooltip text in Peers details window
as a value of N/A could occur due to a lock or a disconnection race
but not during connection setup.
2022-11-16 20:54:54 -08:00
Hennadii Stepanov
c8dc0e3eaa refactor: Inline CTxMemPoolEntry class's functions 2022-11-16 20:17:05 +00:00
Hennadii Stepanov
75bbe594e5 refactor: Move CTxMemPoolEntry class to its own module
This change nukes the policy/fees->mempool circular dependency.

Easy to review using `diff --color-moved=dimmed-zebra`.
2022-11-16 20:16:07 +00:00
Aurèle Oulès
07dfbb5bb8 Make static nLastFlush and nLastWrite Chainstate members 2022-11-16 16:51:53 +01:00
furszy
212ccdf2c2 wallet: AvailableCoins, add arg to include/skip locked coins 2022-11-16 12:14:42 -03:00
MacroFake
6863ad79a6 Merge bitcoin/bitcoin#25112: util: Move error message formatting of NonFatalCheckError to cpp
2222ec71fd util: Move error message formatting of NonFatalCheckError to cpp (MacroFake)

Pull request description:

  This allows to strip down the header file.

ACKs for top commit:
  hebasto:
    re-ACK 2222ec71fd, only rebased and suggested changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/25112#pullrequestreview-1182361605).
  aureleoules:
    ACK 2222ec71fd

Tree-SHA512: 313b3c891bb000cf606df1793b068f93df99915a254fbd67a45f003d440cce7355cdcc6b196f35757cc02d3697970d30e9de0d675f2aa8eb74107c13d663927a
2022-11-16 15:04:43 +01:00
MacroFake
2222ec71fd util: Move error message formatting of NonFatalCheckError to cpp
This allows to strip down the header file
2022-11-16 12:21:33 +01:00
Andrew Chow
f0c646f026 Merge bitcoin/bitcoin#25730: RPC: listunspent, add "include immature coinbase" flag
fa84df1f03 scripted-diff: wallet: rename AvailableCoinsParams members to snake_case (furszy)
61c2265629 wallet: group AvailableCoins filtering parameters in a single struct (furszy)
f0f6a3577b RPC: listunspent, add "include immature coinbase" flag (furszy)

Pull request description:

  Simple PR; adds a "include_immature_coinbase" flag to `listunspent` to include the immature coinbase UTXOs on the response.  Requested by #25728.

ACKs for top commit:
  danielabrozzoni:
    reACK fa84df1f03
  achow101:
    ACK fa84df1f03
  aureleoules:
    reACK fa84df1f03
  kouloumos:
    reACK fa84df1f03
  theStack:
    Code-review ACK fa84df1f03

Tree-SHA512: 0f3544cb8cfd0378a5c74594480f78e9e919c6cfb73a83e0f3112f8a0132a9147cf846f999eab522cea9ef5bd3ffd60690ea2ca367dde457b0554d7f38aec792
2022-11-15 19:53:04 -05:00
Luke Dashjr
f9869843a6 RPC/blockchain: scan{blocks,txoutset>: Further doc improvements 2022-11-16 00:43:11 +00:00
Luke Dashjr
54b45e155e RPC/Blockchain: Clarify invalid-action error in scanblocks & scantxoutset 2022-11-16 00:43:07 +00:00
Andrew Chow
5602cc7ccf Merge bitcoin/bitcoin#16981: Improve runtime performance of --reindex
db929893ef Faster -reindex by initially deserializing only headers (Larry Ruane)
c72de9990a util: add CBufferedFile::SkipTo() to move ahead in the stream (Larry Ruane)
48a68908ba Add LoadExternalBlockFile() benchmark (Larry Ruane)

Pull request description:

  ### Background
  During the first part of reindexing, `LoadExternalBlockFile()` sequentially reads raw blocks from the `blocks/blk00nnn.dat` files (rather than receiving them from peers, as with initial block download) and eventually adds all of them to the block index. When an individual block is initially read, it can't be immediately added unless all its ancestors have been added, which is rare (only about 8% of the time), because the blocks are not sorted by height. When the block can't be immediately added to the block index, its disk location is saved in a map so it can be added later. When its parent is later added to the block index, `LoadExternalBlockFile()` reads and deserializes the block from disk a second time and adds it to the block index. Most blocks (92%) get deserialized twice.

  ### This PR
  During the initial read, it's rarely useful to deserialize the entire block; only the header is needed to determine if the block can be added to the block index immediately. This change to `LoadExternalBlockFile()` initially deserializes only a block's header, then deserializes the entire block only if it can be added immediately. This reduces reindex time on mainnet by 7 hours on a Raspberry Pi, which translates to around a 25% reduction in the first part of reindexing (adding blocks to the index), and about a 6% reduction in overall reindex time.

  Summary: The performance gain is the result of deserializing each block only once, except its header which is deserialized twice, but the header is only 80 bytes.

ACKs for top commit:
  andrewtoth:
    ACK db929893ef
  achow101:
    ACK db929893ef
  aureleoules:
    ACK db929893ef - minor changes and new benchmark since last review
  theStack:
    re-ACK db929893ef
  stickies-v:
    re-ACK db929893e

Tree-SHA512: 5a5377192c11edb5b662e18f511c9beb8f250bc88aeadf2f404c92c3232a7617bade50477ebf16c0602b9bd3b68306d3ee7615de58acfd8cae664d28bb7b0136
2022-11-15 19:23:39 -05:00
MacroFake
547a963628 Merge bitcoin/bitcoin#26489: test: Split overly large util_tests.cpp file
fa4ec1be51 test: Split overly large util_tests.cpp file (MacroFake)

Pull request description:

  The file has ~3kLOC and is slow to compile.

  Fix both issues by splitting it. (On my machine the compilation goes from 25 seconds previously to 17+10 seconds for the two smaller files)

  To review, `--color-moved=dimmed-zebra` can be used.

ACKs for top commit:
  RandyMcMillan:
    ACK fa4ec1be51 for:
  shaavan:
    ACK fa4ec1be51
  aureleoules:
    ACK fa4ec1be51

Tree-SHA512: 4719439c7ee0c6c06b6f6ccf07b3a037c0cae58b1bd6e6e929ebfeab8403be3d1905581669ed733bff0cbf4e385c27ae58d519ce031e145e6889bd5bce1c1d03
2022-11-15 21:48:07 +01:00
John Moffett
8a5014cd8a Fixes bitcoin#26490 by preventing notifications
MacOS 13 sends a window focus change notification after the main
window has been destroyed but before the QTApplication has been
destroyed. This results in the menu bar receiving a notification
despite it no longer existing. The solution is to pass the main
window as context when subscribing to the notifications. Qt
automatically unsubscribes to notifications if the sender OR
context is destroyed.
2022-11-15 10:41:03 -05:00
@RandyMcMillan
cc597bd56d src/bitcoin-cli.cpp: -getinfo help - grammar correction 2022-11-15 09:41:23 -05:00
MacroFake
fa10f193b5 test: Set default in add_wallet_options if only one type can be chosen 2022-11-15 10:03:56 +01:00
James O'Beirne
e4be0e9b06 test: add -maxtipage test for the maximum allowable value 2022-11-14 10:33:17 -05:00
Hennadii Stepanov
f97892d21a doc: Mention required workload when building with MSVC 2022-11-14 14:46:02 +00:00
James O'Beirne
a451e832b4 fix: validation: cast now() to seconds for maxtipage comparison
Since faf44876db, the maxtipage comparison
in IsInitialBlockDownload() has been broken, since the NodeClock::now()
time_point is in the system's native denomination (micrcoseconds).

Without this patch, specifying the maximum allowable -maxtipage
(9223372036854775807) results in a SIGABRT crash.

Co-authored-by: MacroFake <falke.marco@gmail.com>
2022-11-14 09:45:33 -05:00
MacroFake
fa4ec1be51 test: Split overly large util_tests.cpp file 2022-11-14 14:22:43 +01:00
MacroFake
48174c0f28 Merge bitcoin/bitcoin#26240: rpc: Adjust RPCTypeCheckObj error string
2dede9f675 Adjust RPCTypeCheckObj error string (Leonardo Araujo)

Pull request description:

  Unifies the JSON type error strings as mentioned in #26214. Also refer to #25737.

ACKs for top commit:
  furszy:
    ACK 2dede9f6

Tree-SHA512: c918889e347ba32cb6d0e33c0de5956c2077dd40c996151e16741b0c4983ff098c60258206ded76ad7bbec4876c780c6abb494a97e4f1e05717d28a59b9167a6
2022-11-14 12:09:06 +01:00
Gleb Naumenko
46339d29b1 test, refactor: Reorder sendtxrcncl tests for better readability 2022-11-14 12:04:08 +02:00
Gleb Naumenko
14263c13f1 p2p, refactor: Extend logs for unexpected sendtxrcncl 2022-11-14 12:04:07 +02:00
Gleb Naumenko
87493e112e p2p, test, refactor: Minor code improvements 2022-11-14 11:49:49 +02:00
Gleb Naumenko
00c5dec818 p2p: Clarify sendtxrcncl policies 2022-11-14 11:40:17 +02:00
Gleb Naumenko
ac6ee5ba21 test: Expand unit and functional tests for txreconciliation 2022-11-14 11:40:15 +02:00
Gleb Naumenko
bc84e24a4f p2p, refactor: Switch to enum class for ReconciliationRegisterResult
While doing this, add a new value: ALREADY_REGISTERED.
2022-11-14 11:37:28 +02:00
MacroFake
59e00c7e03 Merge bitcoin/bitcoin#25714: univalue: Avoid std::string copies
fa09525751 univalue: string_view test (MacroFake)
1111c7e3f1 univalue: Avoid std::string copies (MacroFake)

Pull request description:

  This shouldn't matter too much, unless a really large string is pushed into a json struct, but I think it also clarifies the code.

ACKs for top commit:
  martinus:
    Code review ACK fa09525751
  aureleoules:
    reACK fa09525751
  ryanofsky:
    Code review ACK fa09525751

Tree-SHA512: 74c441912bd0b00cdb9ea7890121f71ae5d62a7594e7d29aa402c9e3f033710c5d3afb27a37c552e6513804b249aa37e375ce013a3db853a25d1fd7b6e6cd3a8
2022-11-14 10:17:54 +01:00
Andrew Chow
7ef730ca84 Merge bitcoin/bitcoin#26483: test: Don't pass add_to_wallet option to walletcreatefundedpsbt
737c285f69 test: Don't pass add_to_wallet option to walletcreatefundedpsbt (Ryan Ofsky)

Pull request description:

  It's not a documented option. Noticed while working on #19762

ACKs for top commit:
  achow101:
    ACK 737c285f69

Tree-SHA512: 1bf4186fae4390233b2f23389eb6c515c7f0209f12553592df5166e75c452ccd1fb125d9246047c08cff0b869fdda7793812d15da01441e2c4777514446f3ed6
2022-11-10 15:13:58 -05:00
Ryan Ofsky
737c285f69 test: Don't pass add_to_wallet option to walletcreatefundedpsbt
It's not a documented option. Noticed while working on #19762.
2022-11-10 11:48:50 -05:00
Hennadii Stepanov
f1e89597c8 test: Drop no longer required bench output redirection 2022-11-10 16:26:44 +00:00
Hennadii Stepanov
4dbcdf26a3 bench: Suppress output when running with -sanity-check option
This change allows to simplify CI tests, and makes it easier to
integrate the `bench_bitcoin` binary into CMake custom targets or
commands, as `COMMAND` does not support output redirection
2022-11-10 16:26:34 +00:00
MacroFake
555519d082 test: Remove wallet option from non-wallet tests
Review note: The changes are complete, because self.options.descriptors
is set to None in parse_args (test_framework.py).

A value of None implies -disablewallet, see the previous commit.

So if a call to add_wallet_options is missing, it will lead to a test
failure when the wallet is compiled in.
2022-11-10 17:19:13 +01:00
MacroFake
fac8d59d31 test: Set -disablewallet when no wallet has been compiled
self.descriptors is None when no wallet has been compiled, so it is safe
to completely disable the wallet. This change will enhance a future
commit.
2022-11-10 14:39:26 +01:00
MacroFake
fa68937b89 test: Make requires_wallet private
The bool is only used to call a public helper, which some tests already
do. So use the public helper in all tests consistently and make the
confusingly named bool private.
2022-11-10 10:01:31 +01:00
Gleb Naumenko
a60f729e29 p2p: Drop roles from sendtxrcncl
This feature was currently redundant (although could have provided
more flexibility in the future), and already been causing confusion.
2022-11-10 09:21:57 +02:00
MacroFake
9dce30194b Merge bitcoin/bitcoin#26472: test: add missing bech32m / BIP86 test-cases to wallet_descriptor.py
887d85e43d test: add missing bech32m / BIP86 test-cases to wallet_descriptor.py (Sebastian Falbesoner)

Pull request description:

  This small PR adds missing "bech32m" address type / BIP86 checks w.r.t. to the `getnewaddress`/`getrawchangeaddress` RPC and descriptor export functionality to the functional test `wallet_descriptor.py`.

ACKs for top commit:
  shaavan:
    ACK 887d85e43d
  kristapsk:
    ACK 887d85e43d

Tree-SHA512: 05b443ae14138769dc3c87a0178f21db2698fa5bcbeaa953c50ed0c9cf5dcd1effcf4afd09551ca9f4ce73898a7882caaf4c57078767beb6a6a65eb3a662726d
2022-11-09 15:34:19 +01:00
MacroFake
44ca5d5e87 Merge bitcoin/bitcoin#26473: test: Avoid collision with valid path names in getarg_tests/logargs
c8f91478c1 test: Avoid collision with valid path names in `getarg_tests/logargs` (Hennadii Stepanov)

Pull request description:

  This PR prevents test failure when "private" is a part of a valid path.

  For example, `/private/var` is a valid path on macOS for temporary files, which in turn causes test failure on CI for tests managed by the [CTest](https://github.com/bitcoin/bitcoin/pull/25797) framework.

ACKs for top commit:
  MarcoFalke:
    ACK c8f91478c1

Tree-SHA512: 09d257f8fa6be903ec8092b2ae92887a4bec2d05085c76c110637657f4a4bfe2714bf87e2e4727719b3624c8fa4c835ce2ca259c2c9c93033837f997b2057e4f
2022-11-09 12:14:31 +01:00
Hennadii Stepanov
c8f91478c1 test: Avoid collision with valid path names in getarg_tests/logargs 2022-11-09 09:30:21 +00:00
Sebastian Falbesoner
887d85e43d test: add missing bech32m / BIP86 test-cases to wallet_descriptor.py 2022-11-08 18:43:39 +01:00
Hennadii Stepanov
d3e5779f11 guix: Drop non-existent directories 2022-11-08 13:13:45 +00:00
Hennadii Stepanov
43c4afc407 guix: Drop no longer used prepend_to_search_env_var()
It was introduced in c1ae726a13, and it
has no longer been used since 1dd8cbfbc6.
2022-11-08 13:13:44 +00:00
Sebastian Falbesoner
3198e4239e test: check that loading descriptor wallet with legacy entries throws error 2022-11-08 12:29:16 +01:00
Sebastian Falbesoner
349ed2a0ee wallet: throw error if legacy entries are present on loading descriptor wallets
In the wallet key-value-loading routine, most legacy type entries
require a LegacyScriptPubKeyMan instance after successful
deserialization. On a descriptor wallet, creating that (via method
`GetOrCreateLegacyScriptPubKeyMan`) fails and then leads to a
null-pointer dereference crash. Fix this by throwing an error if
if the wallet flags indicate that we have a descriptor wallet and there
is a legacy entry found.
2022-11-08 12:29:12 +01:00
Gleb Naumenko
6772cbf69c tests: stabilize sendtxrcncl test 2022-11-08 12:18:48 +02:00
Vasil Dimov
f362920c2c doc: clarify that NetPermissionFlags::Implicit is only about whitelists
Co-authored-by: Jon Atack <jon@atack.com>
2022-11-07 15:20:20 +01:00
MacroFake
fa09525751 univalue: string_view test 2022-11-07 09:41:19 +01:00
Ryan Ofsky
d8b12a75db rpc: Allow named and positional arguments to be used together
It's nice to be able to use named options and positional arguments together.

Most shell tools accept both, and python functions combine options and
arguments allowing them to be passed with even more flexibility. This change
adds support for python's approach so as a motivating example:

    bitcoin-cli -named createwallet wallet_name=mywallet load_on_startup=1

Can be shortened to:

    bitcoin-cli -named createwallet mywallet load_on_startup=1

JSON-RPC standard doesn't have a convention for passing named and positional
parameters together, so this implementation makes one up and interprets any
unused "args" named parameter as a positional parameter array.
2022-11-05 05:32:39 -04:00
MacroFake
50422b770a Merge bitcoin/bitcoin#26419: log: mempool: log removal reason in validation interface
25ef049d60 log: mempool: log removal reason in validation interface (James O'Beirne)

Pull request description:

  Currently the exact reason a transaction is removed from the mempool isn't logged. It is sometimes detectable from context, but adding the `reason` to the validation interface logs (where it is already passed) seems like an easy way to disambiguate.

  For example in the case of mempool expiry, the logs look like this:

  ```
  [validationinterface.cpp:220] [TransactionRemovedFromMempool] [validation] Enqueuing TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
  [txmempool.cpp:1050] [RemoveUnbroadcastTx] [mempool] Removed <txid> from set of unbroadcast txns before confirmation that txn was sent out
  [validationinterface.cpp:220] [operator()] [validation] TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
  [validation.cpp:267] [LimitMempoolSize] [mempool] Expired 1 transactions from the memory pool
  ```

  There is no context-free way to know $txid was evicted on the basis of expiry. This change will make that case (and probably others) clear.

ACKs for top commit:
  0xB10C:
    ACK 25ef049d60

Tree-SHA512: 9890f9fa16f66c8a9296798d8c28993e1b81da17cf592946f2abc22041f0b30b0911ab86a0c48d4aa46b9a8b3f7f5de67778649ac48c97740b0a09aa6816e0af
2022-11-05 10:32:39 +01:00
MacroFake
ce57dbac90 Merge bitcoin/bitcoin#26449: rpc: doc: add missing option "bech32m" for change_type parameters
c3b1fe59db rpc: doc: add missing option "bech32m" for `change_type` parameters (Sebastian Falbesoner)

Pull request description:

  Affects the help of the `fundrawtransaction`, `send` and `walletcreatefundedpsbt` RPCs.

  This was found by manually inspecting the results of `$ git grep p2sh-segwit.*bech32`.

ACKs for top commit:
  achow101:
    ACK c3b1fe59db

Tree-SHA512: a3f1f8fde5905c80e1b95bd042ca0bc73d08c1c0e79c52ab0d6d12d7afdd4aa288afb41e12279fcea328a396f3d0a5564018170c0a11c5aa26dc6d44d2a62b1c
2022-11-05 09:24:59 +01:00
fanquake
ae6bb6e71e Merge bitcoin/bitcoin#26418: Fix signing of multi_a and rawtr scripts with wallets that only have corresponding keys
0de30ed509 tests: Test Taproot PSBT signing with keys in other descriptor (Andrew Chow)
6efcdf6b7f tests: Use new wallets for each test in wallet_taproot.py (Andrew Chow)
8781a1b6bb psbt: Include output pubkey in additional pubkeys to sign (Andrew Chow)
323890d0d7 sign: Fill in taproot pubkey info for all script path sigs (Andrew Chow)

Pull request description:

  A user reported on [stackexchange](https://bitcoin.stackexchange.com/q/115742/48884) that they were unable to sign for a `multi_a` script using a wallet that only had the corresponding keys (i.e. it did not have the `multi_a()` descriptor). This PR fixes this issue.

  Additionally, `wallet_taproot.py` is modified to test for this scenario by having another wallet in `do_test_psbt` which contains descriptors that only have the keys involved in the descriptor being tested. `wallet_taproot.py` was also modified to create new wallets for each test case rather than sharing wallets throughout as the sharing could result in the signing wallet having the keys in a different descriptor and accidentally result in failing to detect a test failure.

  The changes to the test also revealed a similar issue with `rawtr()` descriptors, which has also been fixed by checking if a descriptor can produce a `SigningProvider` for the Taproot output pubkey.

ACKs for top commit:
  instagibbs:
    crACK 0de30ed509
  darosior:
    ACK 0de30ed509

Tree-SHA512: 12e131dd8afd93da7b1288c9054de2415a228d4477b97102da3ee4e82ce9de20b186260c3085a4b7b067bd8b74400751dcadf153f113db83abc59e7466e69f14
2022-11-04 15:54:13 +00:00
MacroFake
e42ba134f4 Merge bitcoin/bitcoin#26448: test: fix intermittent failure in p2p_sendtxrcncl.py
74d975318a test: fix intermittent failure in p2p_sendtxrcncl.py (Martin Zumsande)

Pull request description:

  `p2p_sendtxrcncl.py` currently fails intermittently in the CI, see e.g. https://cirrus-ci.com/task/5511952184115200?logs=ci#L4024

  I believe that this is related to the reuse of the parameter `p2p_idx=2` of `add_outbound_p2p_connection` in this test: When we call `peer_disconnect`, we don't wait until the node has completed the disconnection. So there is a race between setting up the next connection (next  `addconnection` RPC), and if the old one hasn't been removed and has an identical port like the new one (because we didn't increment `p2p_idx`), `CConnman::OpenNetworkConnection` just [returns](5274f32437/src/net.cpp (L1976)) without establishing a connection, and the test fails.

  Fix this by using distinct `disconnect_p2ps` instead of `peer_disconnect`, which waits for the disconnect to complete. We can then use the same value for `p2p_idx` everywhere.

ACKs for top commit:
  MarcoFalke:
    review ACK 74d975318a

Tree-SHA512: f99f2550b6b320c0a2416a475c1cf189c009fce3a5abf1d4462486e1bfe309e2c3fd4228a4009b0ca38cb77465ce85e3d22298719eb07302fa0a72fbab0e0668
2022-11-04 16:50:42 +01:00
fanquake
83cf055bef Merge bitcoin/bitcoin#26443: doc: mention BIP86 in doc/bips.md
303fb8ff45 doc: mention BIP86 in doc/bips.md (Sebastian Falbesoner)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 303fb8ff45

Tree-SHA512: f30de5be3c789d315d0118594671e9f6a5c7b6e9ec7b7f1c9428f582eeff13946c37ebae26910a2134091e284f30499fcc62923f873418d0ba46a0322af998ad
2022-11-04 15:48:39 +00:00
James O'Beirne
25ef049d60 log: mempool: log removal reason in validation interface
Currently the exact reason a transaction is removed from the mempool isn't
logged. It is sometimes detectable from context, but adding the `reason` to
the validation interface logs (where it is already passed) seems like an easy
way to disambiguate.

For example, in the case of mempool expiry, the logs look like this:

```
[validationinterface.cpp:220] [TransactionRemovedFromMempool] [validation] Enqueuing TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
[txmempool.cpp:1050] [RemoveUnbroadcastTx] [mempool] Removed <txid> from set of unbroadcast txns before confirmation that txn was sent out
[validationinterface.cpp:220] [operator()] [validation] TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
[validation.cpp:267] [LimitMempoolSize] [mempool] Expired 1 transactions from the memory pool
```

There is no context-free way to know $txid was evicted on the basis of expiry.
This change will make that case (and probably others) clear.
2022-11-04 09:38:39 -04:00
Martin Zumsande
74d975318a test: fix intermittent failure in p2p_sendtxrcncl.py
Using disconnect_p2ps instead of peer_disconnect makes
the node wait for the disconnect to complete. As a result,
we can reuse p2p_idx=0 in the add_outbound_p2p_connection calls.
2022-11-03 16:41:50 -04:00
Sebastian Falbesoner
c3b1fe59db rpc: doc: add missing option "bech32m" for change_type parameters
Affects the help of the `fundrawtransaction`, `send` and
`walletcratefundedpsbt` RPCs.
2022-11-03 19:05:21 +01:00
Sjors Provoost
10660c0c60 doc: move Guix uninstall instructions to INSTALL.md
Also drop unused links.
2022-11-03 14:20:44 +01:00
MacroFake
28653a596a Merge bitcoin/bitcoin#26445: .python-version: bump patch version to 3.6.15
29fa38a41a .python-version: bump patch version to 3.6.15 (Sjors Provoost)

Pull request description:

  I'm unable to build Python 3.6.12 using PyEnv on macOS 13.0 as well as Ubuntu 22.10. Bumping the patch version fixes that issue on both systems.

  A workaround is to add `.python-version` to your local git excludes and then do `pyenv local 3.6.15`, but this won't persist when you switch branches. Another workaround is to disable `pyenv`, but then you're potentially not running the test suite against the oldest supported Python version.

ACKs for top commit:
  MarcoFalke:
    concept ACK 29fa38a41a

Tree-SHA512: 60ecf76c957456fe4ece9e92c30866b27aba32d5bdb1696495281c2f7ef7a1fc30226e8992d55bf6f45e430b7588e05fbe49de7138aefab5597dd52bd4f021c0
2022-11-03 13:30:13 +01:00
Sjors Provoost
68fab72a8c guix: OpenSSL test failure workaround
Co-Authored-By: Stephan Oeste <emzy@emzy.de>
2022-11-03 12:52:13 +01:00
Sjors Provoost
d612dca852 guix: reminder to migrate guix-daemon-original customization 2022-11-03 12:52:12 +01:00
Sjors Provoost
8aa460cd02 guix: add guile-gnutls and guile-json to install list
They are mentioned in the figure above, but having them in the table makes it easier to (apt) install everything required.
2022-11-03 12:52:12 +01:00
Sjors Provoost
9b9991e026 guix: recommend mounting a tmpfs on /tmp
This was already suggested in the troubleshooting section, but recommending it upfront would prevent the issue in the first place and speed up builds.
2022-11-03 12:52:12 +01:00
Hennadii Stepanov
29ef26ae25 build: Drop unneeded linking of contrib/devtools/ scripts
Th build system targets run those scripts from the top source directory.
2022-11-03 11:48:29 +00:00
Hennadii Stepanov
77779c3717 script: Improve test-{security,symbol}-check.py robustness
This change allows to use the `test-{security,symbol}-check.py` scripts
when building out of source tree with no need to link scripts into the
build directory.
2022-11-03 11:26:00 +00:00
fanquake
2a7c9984db Merge bitcoin/bitcoin#25248: refactor: Add LIFETIMEBOUND / -Wdangling-gsl to Assert()
fa3ea81c3e refactor: Add LIFETIMEBOUND / -Wdangling-gsl to Assert() (MacroFake)

Pull request description:

  Currently compiles clean, but I think it may still be useful.

  Can be tested by adding an `&`:

  ```diff
  diff --git a/src/test/util_tests.cpp b/src/test/util_tests.cpp
  index 5766fff92d..300c1ec60f 100644
  --- a/src/test/util_tests.cpp
  +++ b/src/test/util_tests.cpp
  @@ -125,7 +125,7 @@ BOOST_AUTO_TEST_CASE(util_check)

       // Check -Wdangling-gsl does not trigger when copying the int. (It would
       // trigger on "const int&")
  -    const int nine{*Assert(std::optional<int>{9})};
  +    const int& nine{*Assert(std::optional<int>{9})};
       BOOST_CHECK_EQUAL(9, nine);
   }

  ```

  Output:
  ```
  test/util_tests.cpp:128:29: warning: object backing the pointer will be destroyed at the end of the full-expression [-Wdangling-gsl]
      const int& nine{*Assert(std::optional<int>{9})};
                              ^~~~~~~~~~~~~~~~~~~~~
  ./util/check.h:75:50: note: expanded from macro 'Assert'
  #define Assert(val) inline_assertion_check<true>(val, __FILE__, __LINE__, __func__, #val)
                                                   ^~~
  1 warning generated.

ACKs for top commit:
  jonatack:
    ACK fa3ea81c3e
  theuni:
    ACK fa3ea81c3e

Tree-SHA512: 17dea4d75f2ee2bf6e1b6a6f6d8f439711c777df0390574e8d8edb6ac9ee807a135341e4439050bd6a15ecc4097a1ba9a7ab15d27541ebf70a4e081fa6871877
2022-11-03 10:29:05 +00:00
Sjors Provoost
29fa38a41a .python-version: bump patch version to 3.6.15 2022-11-03 09:26:27 +01:00
MacroFake
5274f32437 Merge bitcoin/bitcoin#26417: test: fix intermittent failure in feature_index_prune.py
201b9a02fd test: fix intermittent failure in feature_index_prune.py (Martin Zumsande)

Pull request description:

  I can't reproduce the error from #26630 locally, but from analying the logs I think the problem is the following:

  After calling `sync_blocks`, we didn't check that the indexes have caught up to the tip before performing the manual pruning. This could possibly lead to prune blockers with a lower height than the expected 2489, which do appear in the logs of the failed CI runs, e.g.
   - `2022-10-27T21:14:17.703920Z [C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\src\validation.cpp:2395] [FlushStateToDisk] [prune] coinstatsindex limited pruning to height 2488` ([Cirrus](https://cirrus-ci.com/task/5443742333665280?logs=functional_tests#L2506))

  So, this should be fixed by a call to `sync_index`.
  Fixes #26330

ACKs for top commit:
  brunoerg:
    crACK 201b9a02fd

Tree-SHA512: fb7023c9eb2ba6d0e69e059a401453cbdf63abc6804543dffcf36ba9f93c9cd13209e57aa5536d94b2e420c9d4cd0b1a7eff1adadd19aa7b3c33f592502e1bc0
2022-11-02 15:00:40 +01:00
MacroFake
39f026b1ec Merge bitcoin/bitcoin#26396: net: Avoid SetTxRelay for feeler connections
fa24239a1c net: Avoid SetTxRelay for feeler connections (MacroFake)

Pull request description:

  Seems odd to reserve memory for the struct (the heaviest member being `m_tx_inventory_known_filter`) when it is never used.

  This also avoids sending out `msg_sendtxrcncl` before disconnecting. This shouldn't matter, as other messages, such as `msg_wtxidrelay`, `msg_sendaddrv2`, `msg_verack` or `msg_getaddr` are still sent. Though, it allows to test the changes here as a side-effect.

ACKs for top commit:
  naumenkogs:
    ACK fa24239a1c
  vasild:
    ACK fa24239a1c
  jonatack:
    ACK fa24239a1c
  mzumsande:
    ACK fa24239a1c

Tree-SHA512: d7604c7eb4df8f2de811e600bdd312440ee03e508d3a0f09ae79f7f2d3eeec663bfd47a2d079fa50b756d61e35dfa998de068a7b9afaf35378fa0e62a538263d
2022-11-02 08:07:28 +01:00
Sebastian Falbesoner
303fb8ff45 doc: mention BIP86 in doc/bips.md 2022-11-01 20:50:51 +01:00
MacroFake
bf0cb43990 Merge bitcoin/bitcoin#26437: test: remove unused CHANGE_{XPRV,XPUB} constants
f1ee974e8e test: remove unused `CHANGE_{XPRV,XPUB}` constants (Sebastian Falbesoner)

Pull request description:

  These constants exist since the introduction of the functional test wallet_taproot.py (2667366aaa), but they have never been used.

ACKs for top commit:
  instagibbs:
    ACK f1ee974e8e
  brunoerg:
    ACK f1ee974e8e

Tree-SHA512: 2c6f8cd537c35af59c4cbe6abb07386b7fb9e4662c7bed27ad8935e004f9bf452705fb1caef4f5ea1f4ad509745afba966005f95a18cf2b59e00b7739a5df7f8
2022-11-01 16:26:09 +01:00
Sebastian Falbesoner
f1ee974e8e test: remove unused CHANGE_{XPRV,XPUB} constants
These constants exist since the introduction of the functional test
wallet_taproot.py (2667366aaa), but they
have never been used.
2022-11-01 13:57:11 +01:00
fanquake
5668ccec1d Merge bitcoin/bitcoin#25548: gui: Check for readlink buffer overflow and handle gracefully
e049fd76f0 Bugfix: Check for readlink buffer overflow and handle gracefully (Luke Dashjr)

Pull request description:

  If readlink returns the size of the buffer, an overflow may have (safely) occurred.
  Pass a buffer size of MAX_PATH+1 (the size of the actual buffer) to detect this scenario.

ACKs for top commit:
  hebasto:
    ACK e049fd76f0.

Tree-SHA512: 188bace79cbe556efe7782e46b870c02729b07b104a9316b0f7d50013504972e85baf507403d2d6060bb2bf3e13f40d735bddd18255d97a60810208c3de87691
2022-11-01 11:09:17 +00:00
fanquake
b89530483d util: move threadinterrupt into util 2022-11-01 10:14:49 +00:00
fanquake
c041d8f2c9 Merge bitcoin/bitcoin#26360: build: remove threadinterrupt from libbitcoinkernel
afbcd227dd build: remove threadinterrupt from libbitcoinkernel (fanquake)

Pull request description:

  Extracted from #26292.

ACKs for top commit:
  hebasto:
    ACK afbcd227dd, tested on Ubuntu 22.04.
  ryanofsky:
    Code review ACK afbcd227dd

Tree-SHA512: 9d355f0e417561be41cdd0674a8f94c9ffe3ecfb4063bb9c90f1032cb9d471be11d4fa26de40993e3a411e015272201551fbbb3d3c2b43e4c17bf49386a2741c
2022-11-01 10:12:43 +00:00
fanquake
27e76afe24 Merge bitcoin/bitcoin#26294: build: move util/url to common/url
3a0b352c63 refactor: move url.h/cpp from lib util to lib common (fanquake)
058eb69ce4 build: add missing event cflags to libbitcoin_util (fanquake)

Pull request description:

  Move `util/url` to `common/url`.

  Also add missing `event_*` flags to `libbitcoin_util`. #26293 + the commit dropping boost cppflags from `libbitcoin_util` shows this issue. i.e:
  ```bash
    CXX      util/libbitcoin_util_a-url.o
  util/url.cpp:7:10: fatal error: 'event2/http.h' file not found
  #include <event2/http.h>
           ^~~~~~~~~~~~~~~
  1 error generated.
  ```

ACKs for top commit:
  hebasto:
    ACK 3a0b352c63
  ryanofsky:
    Code review ACK 3a0b352c63

Tree-SHA512: 600a76fd334267a02d332df9b67891a38d3fd7f5baf8a82b2447879b3bc65eab2552d2c081c0a5f1ec927bf80df7fc1f0cbbdda4cb76994b46dadf260b8e1cb3
2022-11-01 10:05:49 +00:00
fanquake
d08b63baa0 Merge bitcoin/bitcoin#26373: Update minisketch subtree to latest upstream
e9f1d8c272 Squashed 'src/minisketch/' changes from 47f0a2d26f..a571ba20f9 (Hennadii Stepanov)

Pull request description:

  Contains:
  - https://github.com/sipa/minisketch/pull/68
  - https://github.com/sipa/minisketch/pull/69
  - https://github.com/sipa/minisketch/pull/70
  - https://github.com/sipa/minisketch/pull/72

  Fixes bitcoin/bitcoin#26262.

  Required for bitcoin/bitcoin#25797.

  Guix builds on `arm64`:
  ```
  28af3bdd17924bb5279934add1fe34f5664181a1c99f33793725adc35f90d778  guix-build-45a0f4e01448/output/aarch64-linux-gnu/SHA256SUMS.part
  c9cdcb107de5b683221b0e2a3796814548414cdba27be198516a2b401ada90c3  guix-build-45a0f4e01448/output/aarch64-linux-gnu/bitcoin-45a0f4e01448-aarch64-linux-gnu-debug.tar.gz
  ce1e916cc7574f555c04ed5fa9e9a47a1e4d2d9d1cb82a040269e8c32aca9194  guix-build-45a0f4e01448/output/aarch64-linux-gnu/bitcoin-45a0f4e01448-aarch64-linux-gnu.tar.gz
  26404d3479108bcb8ede4ae7f3d37dbb64a335f0cd5bce7ee901d171a1ef2b0b  guix-build-45a0f4e01448/output/arm-linux-gnueabihf/SHA256SUMS.part
  0fe32aee5836baed86ad25cbddb43c5a9d2ecbbe5cc954f3a76670aa52ff3c6f  guix-build-45a0f4e01448/output/arm-linux-gnueabihf/bitcoin-45a0f4e01448-arm-linux-gnueabihf-debug.tar.gz
  41f312cbf26a0e51e6867e644beb3d085ebd4221dc83ea84a97b708e6d230b6d  guix-build-45a0f4e01448/output/arm-linux-gnueabihf/bitcoin-45a0f4e01448-arm-linux-gnueabihf.tar.gz
  c05ba8235e05c171ecdbacba87dfc1db34d1971286b853c26e3de52a6b0ea1cc  guix-build-45a0f4e01448/output/arm64-apple-darwin/SHA256SUMS.part
  9d1dae2b73573d34950684269c41f2504761f2df19f24033513f6664b70b1aad  guix-build-45a0f4e01448/output/arm64-apple-darwin/bitcoin-45a0f4e01448-arm64-apple-darwin-unsigned.dmg
  1ff742346bd04b7edf27e096286f4863fcfec984b99caf265a64fe72b298b36a  guix-build-45a0f4e01448/output/arm64-apple-darwin/bitcoin-45a0f4e01448-arm64-apple-darwin-unsigned.tar.gz
  74a001f16ba382c22acb9979ee5d905af42f4ca3613fc6e8fa66fec994fb0850  guix-build-45a0f4e01448/output/arm64-apple-darwin/bitcoin-45a0f4e01448-arm64-apple-darwin.tar.gz
  ec2f70a0b8a128d3ca69307150714c400620199b506a411b0a4c0a506cc3ed66  guix-build-45a0f4e01448/output/dist-archive/bitcoin-45a0f4e01448.tar.gz
  9a67ea74e1b5ef3d08b6a59217ecf8d193234c76f81157192fabe7a9b679af0c  guix-build-45a0f4e01448/output/powerpc64-linux-gnu/SHA256SUMS.part
  95b458427d73ed81f4a4f85be0dfe81a9a48dc254be65ef158de93d32f07a17a  guix-build-45a0f4e01448/output/powerpc64-linux-gnu/bitcoin-45a0f4e01448-powerpc64-linux-gnu-debug.tar.gz
  2b269c630f069cc42da5f361d13f5b684fbacb49641f44ed6687379a1e9281b0  guix-build-45a0f4e01448/output/powerpc64-linux-gnu/bitcoin-45a0f4e01448-powerpc64-linux-gnu.tar.gz
  33d576b9be022108db6d76c41c1f72c865b5bb38439c6da1cbceac757a1be47f  guix-build-45a0f4e01448/output/powerpc64le-linux-gnu/SHA256SUMS.part
  053dc9cecc978601701459109eec82f3bcce4042d794c520fd4bd8cd4e17eff3  guix-build-45a0f4e01448/output/powerpc64le-linux-gnu/bitcoin-45a0f4e01448-powerpc64le-linux-gnu-debug.tar.gz
  32946d589103c763c2fc67c0d8fa13a38b9194df76cd31755a761a7b9ad856a6  guix-build-45a0f4e01448/output/powerpc64le-linux-gnu/bitcoin-45a0f4e01448-powerpc64le-linux-gnu.tar.gz
  7d1be372c84f9403333462e82d7800dfb1812b14c1ed228dacf605dff9aab75c  guix-build-45a0f4e01448/output/riscv64-linux-gnu/SHA256SUMS.part
  3338bcaf6b8ffd9c155c055a0a18d039e1173212e128561fdea493c263d6ecb9  guix-build-45a0f4e01448/output/riscv64-linux-gnu/bitcoin-45a0f4e01448-riscv64-linux-gnu-debug.tar.gz
  7735eefa84630d16d8d37ea23c734578c6fb6540d4359b78d0d1508887e76042  guix-build-45a0f4e01448/output/riscv64-linux-gnu/bitcoin-45a0f4e01448-riscv64-linux-gnu.tar.gz
  c3a96d6758631b8e8b4f5bb8035e5442e1fc5b831c7a0ddedb26473b025b2503  guix-build-45a0f4e01448/output/x86_64-apple-darwin/SHA256SUMS.part
  b6c87738e6e561bfc4052464e93a6d03a87c51e8f27e4df8ae7458c9828e28cb  guix-build-45a0f4e01448/output/x86_64-apple-darwin/bitcoin-45a0f4e01448-x86_64-apple-darwin-unsigned.dmg
  0c16f9435c3d572970400ff4683705b297eda3dd37a682619371771da8efbb24  guix-build-45a0f4e01448/output/x86_64-apple-darwin/bitcoin-45a0f4e01448-x86_64-apple-darwin-unsigned.tar.gz
  22e06590afe07b992f6f4e46d5568499403937606d16540b121f23e133270560  guix-build-45a0f4e01448/output/x86_64-apple-darwin/bitcoin-45a0f4e01448-x86_64-apple-darwin.tar.gz
  56ddb44760f71fd1c1ea06c7174bf035b8b73c32569d69a1401d130551db43e5  guix-build-45a0f4e01448/output/x86_64-linux-gnu/SHA256SUMS.part
  1c8a27fab5d9d0f3a5449f04ae8a7e693c5f144948f6d0c2dbf397e04ce9ebfa  guix-build-45a0f4e01448/output/x86_64-linux-gnu/bitcoin-45a0f4e01448-x86_64-linux-gnu-debug.tar.gz
  66aca3767407f6c2e77b8b93edc58e71c00bed518a75362e3c2f5009739c666c  guix-build-45a0f4e01448/output/x86_64-linux-gnu/bitcoin-45a0f4e01448-x86_64-linux-gnu.tar.gz
  7ac04c1d5f2dbb6c2c392add9c4e2c5c838c718910b920fe6691a5e9881065b9  guix-build-45a0f4e01448/output/x86_64-w64-mingw32/SHA256SUMS.part
  2c51afbcf5645516d01d5f730e9883d9ed3cb90b9b9baf90792ecdb2b3eb21cf  guix-build-45a0f4e01448/output/x86_64-w64-mingw32/bitcoin-45a0f4e01448-win64-debug.zip
  81738a0690e954d6850f210fa3933ca5de8800078734f8b0810b5e7fe3e63e5d  guix-build-45a0f4e01448/output/x86_64-w64-mingw32/bitcoin-45a0f4e01448-win64-setup-unsigned.exe
  ecade9ee23ad156e057d466afe79943f008de316973738efdf5ae33d2a527631  guix-build-45a0f4e01448/output/x86_64-w64-mingw32/bitcoin-45a0f4e01448-win64-unsigned.tar.gz
  3b17762d01825a3f7d5aecf34cf038938d9ee467e63ea3f34efd1dfa2af5e284  guix-build-45a0f4e01448/output/x86_64-w64-mingw32/bitcoin-45a0f4e01448-win64.zip
  ```

ACKs for top commit:
  fanquake:
    ACK 45a0f4e014

Tree-SHA512: e48d79aa0e3563aeae981ddf1281420fff9b739af43e3d77f13cace158724ff01a7622762aadfd093dcd18822a2c1b76d1c548ae64faf727ba93044887b560fb
2022-11-01 08:29:57 +00:00
fanquake
43e813cab2 Merge bitcoin/bitcoin#26387: p2p: TryLowWorkHeadersSync follow-ups
784b023191 [net processing] Simplify use of IsContinuationOfLowWorkHeadersSync in TryLowWorkHeaderSync (dergoegge)
e891aabf5a [net processing] Fixup TryLowWorkHeadersSync comment (dergoegge)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/26355#discussion_r1003561481 and https://github.com/bitcoin/bitcoin/pull/26355#discussion_r1004554187

ACKs for top commit:
  hernanmarino:
    ACK 784b023191
  brunoerg:
    crACK 784b023191
  mzumsande:
    ACK 784b023191

Tree-SHA512: b47ac0d78a09ca3a1806e38c5d2e2fcf1e5f0668f202450b5079c5cb168e168ac6828c0948d23f3610696375134986d75ef3c6098858173023bcb743aec8004c
2022-10-31 15:35:21 +00:00
fanquake
4766cd1981 Merge bitcoin/bitcoin#24051: Bugfix: configure: bitcoin-{cli,tx,util} don't need UPnP, NAT-PMP, or ZMQ
98868633d1 Bugfix: configure: bitcoin-{cli,tx,util} don't need UPnP, NAT-PMP, or ZMQ (Luke Dashjr)

Pull request description:

  As with #23345, these other tools likewise don't use various deps.

ACKs for top commit:
  achow101:
    ACK 98868633d1

Tree-SHA512: 4be056b8e0c9f69834229aa257187457de1bc34214d320b770834e21ecc1f0ca7aa7b9689fba525928947bfabbb461528795f709014fb9618b82f088fe64f271
2022-10-31 11:51:15 +00:00
MacroFake
2856dee808 Merge bitcoin/bitcoin#26402: doc: Fix typos
180eac0f73 Fix: typos (omahs)

Pull request description:

  Fix: typos

ACKs for top commit:
  aureleoules:
    ACK 180eac0f73

Tree-SHA512: 23150f3408a2eb9cb298c6add16d1bcb149da277238786f053c2896c803f896b07555b3dc71e15cd8d390023800c8e006f931f415f887bab69ee5bd4b0420581
2022-10-31 11:46:43 +01:00
fanquake
3a0b352c63 refactor: move url.h/cpp from lib util to lib common 2022-10-31 10:17:04 +00:00
fanquake
058eb69ce4 build: add missing event cflags to libbitcoin_util
The fact that this is missing is currently masked by the inclusion of
BOOST_CPPFLAGS.
2022-10-31 10:10:53 +00:00
MacroFake
c75c0d8e11 Merge bitcoin/bitcoin#26424: doc: correct deriveaddresses RPC name
0f38524c31 doc: correct deriveaddresses RPC name (Bitcoin Hodler)

Pull request description:

  There never was a `deriveaddress` RPC, from what I can tell. It was always called `deriveaddresses` (plural).

ACKs for top commit:
  theStack:
    ACK 0f38524c31
  Zero-1729:
    ACK 0f38524c31

Tree-SHA512: 3f405f5479a0d39cf150fd80b4d854ffe4eef718a358202c619e34a08d98c1252b82fc70d106cdf2215dc5a50c6f6cd5e26fe7ed87156f6b08f8e97d963affb7
2022-10-31 08:42:50 +01:00
Bitcoin Hodler
0f38524c31 doc: correct deriveaddresses RPC name 2022-10-30 18:11:45 +00:00
Douglas Chimento
f86697163e rpc: Return fee and prevout(s) to getrawtransaction
* Add optional fee response in BTC to getrawtransaction
* Add optional prevout(s) response to getrawtransaction showing utxos being spent
* Add getrawtransaction_verbosity functional test to validate fields
2022-10-30 14:06:15 +02:00
fanquake
1914e470e3 build: copy config.{guess,sub} post autogen in zmq package
Otherwise our config.guess and config.sub will be copied over. This
problem has been masked by the fact that modern systems ship with
versions that recognise all the triplets we use (namely
arm64-apple-darwin). However building on ubuntu 20.04 surfaces the
issue.

Fixes #26420.
2022-10-29 15:51:10 +01:00
furszy
fa84df1f03 scripted-diff: wallet: rename AvailableCoinsParams members to snake_case
-BEGIN VERIFY SCRIPT-

sed -i 's/nMinimumAmount/min_amount/g' $(git grep -l nMinimumAmount)
sed -i 's/nMaximumAmount/max_amount/g' $(git grep -l nMaximumAmount)
sed -i 's/nMinimumSumAmount/min_sum_amount/g' $(git grep -l nMinimumSumAmount)
sed -i 's/nMaximumCount/max_count/g' $(git grep -l nMaximumCount)

-END VERIFY SCRIPT-
2022-10-29 08:51:34 -03:00
furszy
61c2265629 wallet: group AvailableCoins filtering parameters in a single struct
Plus clean callers that use the params default values
2022-10-29 08:50:38 -03:00
furszy
f0f6a3577b RPC: listunspent, add "include immature coinbase" flag
so we can return the immature coinbase UTXOs as well.
2022-10-29 08:45:12 -03:00
MacroFake
4f270d2b63 Merge bitcoin/bitcoin#26404: test: fix intermittent failure in rpc_getblockfrompeer.py
8a9f1e4d18 test: fix intermittent failure in rpc_getblockfrompeer.py (Martin Zumsande)

Pull request description:

  Fixes an intermittent failure in `rpc_getblockfrompeer.py` observed in https://cirrus-ci.com/task/6610115527704576 by adding a sync to make sure the node has processed the header we sent it before we query it for the corresponding block.

  Fixes #26412

ACKs for top commit:
  jonatack:
    ACK 8a9f1e4d18

Tree-SHA512: f6188ab3cfd863034e44e9806d0d99a8781462bec94141501aefc71589153481ffb144e126326ab81807c2b2c93de7f4aac5d09dbcf26c4512a176e06fc6e5f8
2022-10-29 11:14:10 +02:00
MacroFake
984a01589b Merge bitcoin/bitcoin#26408: test: Remove spam from debug log
ef97b89902 Exclude rand from debug log (Jeff Ruane)

Pull request description:

  Currently, `debug.log` is spammed with messages like this from `random.cpp` when functional tests are run.

  ```
  2022-10-25T19:24:34.787663Z [scheduler] [random.cpp:519] [SeedPeriodic] [rand] Feeding 36565 bytes of dynamic environment data into RNG
  ```

  These logs are not useful for debugging and decrease the signal-to-noise ratio of the logs, so they should be suppressed by excluding the `rand` category, as the `libevent` and `leveldb` categories currently are.

ACKs for top commit:
  LarryRuane:
    ACK ef97b89902
  kouloumos:
     ACK ef97b89902, confirmed that this log level is only used in `random.cpp` and indeed it seems that it doesn't add any value to the debug.log during functional tests.
  satsie:
    ACK ef97b89902
  theStack:
    ACK ef97b89902

Tree-SHA512: 5cea384a3197f0ec77efa9efc77822914450ecf5546606568bbd432c3536040c772c57aef58d3bb083a2e5e756f690766fa1fb382ab1973748db238108a58746
2022-10-29 09:59:23 +02:00
Andrew Chow
0de30ed509 tests: Test Taproot PSBT signing with keys in other descriptor
Test that the same keys included in other descriptors will still be able
to sign a PSBT that requires those keys.
2022-10-28 20:03:22 -04:00
Andrew Chow
6efcdf6b7f tests: Use new wallets for each test in wallet_taproot.py
To avoid a wallet potentially being able to sign a transaction using
keys from descriptors imported in previous tests, make new wallets for
each test case rather than sharing them.
2022-10-28 20:03:22 -04:00
Andrew Chow
8781a1b6bb psbt: Include output pubkey in additional pubkeys to sign
In addition to the pubkeys in hd_keypaths and tap_bip32_keypaths, also
see if the descriptor can produce a SigningProvider for the output
pubkey.

Also slightly refactors this area to reduce code duplication.
2022-10-28 20:03:22 -04:00
Andrew Chow
323890d0d7 sign: Fill in taproot pubkey info for all script path sigs
Taproot pubkey info was not being added for multi_a signing. The filling
of this info is moved into the common function CreateTaprootScriptSig so
that any signing of taproot scripts will include the pubkey info.
2022-10-28 20:03:22 -04:00
Andrew Chow
8b050762b1 Merge bitcoin/bitcoin#26409: refactor: Silence GCC Wmissing-field-initializers in ChainstateManagerOpts
fa29ef00ad refactor: Silence GCC Wmissing-field-initializers in ChainstateManagerOpts (MacroFake)

Pull request description:

  The `std::optional` fields in the struct that fall back to chain param defaults if not provided should be initialized to `std::nullopt`. This already happens with the current code.

  However, for consistency with `check_block_index` and to silence a GCC warning, add the "missing" `{}`.

ACKs for top commit:
  achow101:
    ACK fa29ef00ad
  hebasto:
    ACK fa29ef00ad, tested on Ubuntu 22.04 + GCC 11.3.
  jonatack:
    ACK fa29ef00ad

Tree-SHA512: bdec9c56df5d601a5616e107fed48737b13b0a7242b6526092fb682b5016544a4bc08666b60304c668d44c6f7ac69d3788093d921382c1d6c577c1f9fe31fc50
2022-10-28 15:37:17 -04:00
Martin Zumsande
201b9a02fd test: fix intermittent failure in feature_index_prune.py
After syncing the blocks, we didn't check that the
indexes have caught up to the tip before manually pruning.
This could lead to prune blockers lower thatn the expected height.
2022-10-28 15:25:12 -04:00
brunoerg
33fdfc7986 test: perturb anchors.dat to test it doesn't throw an error during initialization 2022-10-28 11:47:03 -03:00
Martin Zumsande
8a9f1e4d18 test: fix intermittent failure in rpc_getblockfrompeer.py
by adding a sync to make sure the node has received the
header before we query it for the block
2022-10-28 10:17:50 -04:00
kouloumos
0b78110f73 test: Move tx creation to create_self_transfer_multi 2022-10-28 16:19:55 +03:00
MacroFake
1bad29fe02 Merge bitcoin/bitcoin#26377: test: Make system_tests/run_command test locale and platform agnostic
884304e6c6 test: Make `system_tests/run_command` locale agnostic (Hennadii Stepanov)

Pull request description:

  Fixes bitcoin/bitcoin#26368.

ACKs for top commit:
  Sjors:
    tACK 884304e6c6

Tree-SHA512: 76d4941e02b3b119dcf4dacbe60ef45a9dc8cf775bdb31b5291cd8147665285d41caaf1f5688abdfc9a47c393ddb535af7b11af839660d30ef30f1ca0d936133
2022-10-28 11:32:13 +02:00
MacroFake
fa29ef00ad refactor: Silence GCC Wmissing-field-initializers in ChainstateManagerOpts 2022-10-28 11:08:20 +02:00
omahs
180eac0f73 Fix: typos
Fix: typos

Fix: typos

Fix: typos
2022-10-28 09:39:36 +02:00
Jeff Ruane
ef97b89902 Exclude rand from debug log
Currently, debug.log is spammed with messages from random.cpp
when functional tests are run. These logs are not useful for
debugging, and decrease the signal to noise ratio of the logs.
2022-10-28 01:15:41 -06:00
Andrew Chow
f37bd15d47 Merge bitcoin/bitcoin#25685: wallet: Faster transaction creation by removing pre-set-inputs fetching responsibility from Coin Selection
3fcb545ab2 bench: benchmark transaction creation process (furszy)
a8a75346d7 wallet: SelectCoins, return early if target is covered by preset-inputs (furszy)
f41712a734 wallet: simplify preset inputs selection target check (furszy)
5baedc3351 wallet: remove fetch pre-selected-inputs responsibility from SelectCoins (furszy)
295852f619 wallet: encapsulate pre-selected-inputs lookup into its own function (furszy)
37e7887cb4 wallet: skip manually selected coins from 'AvailableCoins' result (furszy)
94c0766b0c wallet: skip available coins fetch if "other inputs" are disallowed (furszy)

Pull request description:

  #### # Context (Current Flow on Master)

  In the transaction creation process, in order to select which coins the new transaction will spend,
  we first obtain all the available coins known by the wallet, which means walking-through the
  wallet txes map, gathering the ones that fulfill certain spendability requirements in a vector.

  This coins vector is then provided to the Coin Selection process, which first checks if the user
  has manually selected any input (which could be internal, aka known by the wallet, or external),
  and if it does, it fetches them by searching each of them inside the wallet and/or inside the
  Coin Control external tx data.

  Then, after finding the pre-selected-inputs and gathering them in a vector, the Coin Selection
  process walks-through the entire available coins vector once more just to erase coins that are
  in both vectors. So the Coin Selection process doesn’t pick them twice (duplicate inputs inside
  the same transaction).

  #### # Process Workflow Changes

  Now, a new method, `FetchCoins` will be responsible for:
  1) Lookup the user pre-selected-inputs (which can be internal or external).
  2) And, fetch the available coins in the wallet (excluding the already fetched ones).

  Which will occur prior to the Coin Selection process. Which allows us to never include the
  pre-selected-inputs inside the available coins vector in the first place, as well as doing other
  nice improvements (written below).

  So, Coin Selection can perform its main responsibility without mixing it with having to fetch
  internal/external coins nor any slow and unneeded duplicate coins verification.

  #### # Summarizing the Improvements:

  1) If any pre-selected-input lookup fail, the process will return the error right away.
      (before, the wallet was fetching all the wallet available coins, walking through the
      entire txes map, and then failing for an invalid pre-selected-input inside SelectCoins)

  2) The pre-selected-inputs lookup failure causes are properly described on the return error.
      (before, we were returning an "Insufficient Funds" error for everything, even if the failure
      was due a not solvable external input)

  3) **Faster Coin Selection**: no longer need to "remove the pre-set inputs from the available coins
      vector so that Coin Selection doesn't pick them" (which meant to loop-over the entire
      available coins vector at Coin Selection time, erasing duplicate coins that were pre-selected).

      Now, the available coins vector, which is built after the pre-selected-inputs fetching,
      doesn’t include the already selected inputs in the first place.

  4) **Faster transaction creation** for transactions that only use manually selected inputs.

      We now will return early, as soon as we finish fetching the pre-selected-inputs and
      not perform the resources expensive calculation of walking-through the entire wallet
      txes map to obtain the available coins (coins that we will not use).

  ---------------------------

  Added a new bench (f6d0bb2) measuring the transaction creation process, for a wallet with ~250k UTXO, only using the pre-selected-inputs inside coin control. Setting `m_allow_other_inputs=false` to disallow the wallet to include coins automatically.

  #### Result on this PR (tip f6d0bb2d):

  |               ns/op |                op/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |        1,048,675.00 |              953.58 |    0.3% |      0.06 | `WalletCreateTransaction`

  vs

  #### Result on master (tip 4a4289e2):

  |               ns/op |                op/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |       96,373,458.20 |               10.38 |    0.2% |      5.30 | `WalletCreateTransaction`

  The benchmark took to run in master: **96.37 milliseconds**, while in this PR: **1 millisecond**  🚀 .

ACKs for top commit:
  S3RK:
    Code Review ACK 3fcb545ab2
  achow101:
    ACK 3fcb545ab2
  aureleoules:
    reACK 3fcb545ab2

Tree-SHA512: 42f833e92f40c348007ca565a4c98039e6f1ff25d8322bc2b27115824744779baf0b0a38452e4e2cdcba45076473f1028079bbd0f670020481ec5d3db42e4731
2022-10-27 17:48:58 -04:00
Martin Zumsande
39b93649c4 test: add functional test for IBD stalling logic 2022-10-27 15:10:34 -04:00
Andrew Chow
551c8e9526 Merge bitcoin/bitcoin#26349: rpc: make address field optional list{transactions, sinceblock} response
eb679a7896 rpc: make `address` field optional (w0xlt)

Pull request description:

  Close https://github.com/bitcoin/bitcoin/issues/26338.

  This PR makes optional the `address` field in the response of `listtransactions` and `listsinceblock` RPC.
  And adds two tests that fail on master, but not on this branch.

ACKs for top commit:
  achow101:
    ACK eb679a7896
  aureleoules:
    ACK eb679a7896

Tree-SHA512: b267439626e2ec3134ae790c849949a4c40ef0cebd20092e8187be3db0a61941b2da10bbbba92ca880b8369f46c1aaa806d057eaa5159325f65cbec7cb33c52f
2022-10-27 13:17:39 -04:00
MacroFake
bd478890c5 Merge bitcoin/bitcoin#26388: ci: Use macos-ventura-xcode:14.1 image for "macOS native" task
da16893474 ci: Use `macos-ventura-xcode:14.1` image for "macOS native" task (Hennadii Stepanov)
702836530f ci: Make `getopt` path architecture agnostic (Hennadii Stepanov)

Pull request description:

  The "macOS native" CI task always uses the recent OS image.

  This PR updates it up to the recent macOS release.

  Cirrus Labs [stopped](https://github.com/bitcoin/bitcoin/pull/25160#issuecomment-1162829773) updating macOS images for `x86_64`, therefore, an `arm64` image been used.

  Also `make test-security-check` has been dropped as it ["isn't even expected to pass"](https://github.com/bitcoin/bitcoin/issues/26386#issuecomment-1290318628) on `arm64` in CI.

ACKs for top commit:
  Sjors:
    utACK da16893

Tree-SHA512: 36785d33b7f11b3cdbc53bcfbf97d88bf821fad248c825982dd9f8e3413809a4ef11190eaf950e60fdf479b62ff66920c35d9ea42d534723f015742eec7e19b6
2022-10-27 16:15:20 +02:00
MacroFake
fa24239a1c net: Avoid SetTxRelay for feeler connections 2022-10-27 16:09:33 +02:00
Hennadii Stepanov
39710f5635 Merge bitcoin-core/gui#665: Switch to the selected wallet after loading
b8b59ff9fe gui: update the screen after loading wallet (w0xlt)

Pull request description:

  Currently, the user loads a wallet and the screen does not switch to the selected wallet after loading (File -> Open Wallet -> wallet name).

  This PR changes that by making the `OpenWalletActivity::opened` signal connection a `Qt::QueuedConnection` type.

ACKs for top commit:
  jarolrod:
    ACK b8b59ff9fe
  hebasto:
    ACK b8b59ff9fe, tested on Ubuntu 22.04.

Tree-SHA512: 43cd755638b643f481014a7933a0af25df2d109e859cb5f878bc04e562950d550716fa38465140060e28526b2441688580cbcbe4ec6819566b4f95162ca5e527
2022-10-27 13:56:42 +01:00
Hennadii Stepanov
1b228497fa qt: Drop no longer used SplashScreen::finish() slot 2022-10-27 12:58:48 +01:00
Hennadii Stepanov
10811afff4 qt: Drop no longer used BitcoinApplication::splashFinished() signal 2022-10-27 12:58:48 +01:00
Hennadii Stepanov
5299cfe371 qt: Delete splash screen widget explicitly
This ensures that during shutdown, including failed initialization, the
`SplashScreen::m_connected_wallet_handlers` is deleted before the wallet
context is.
2022-10-27 12:58:48 +01:00
glozow
2242de16cc Merge bitcoin/bitcoin#26394: Fix typo in comment SHA256->SHA512
0cc23fc603 Fix typo in comment SHA256->SHA512 (Elichai Turkel)

Pull request description:

  The comment says it's the SHA-256 state, while it's actually the SHA-512 state

ACKs for top commit:
  andrewtoth:
    ACK 0cc23fc603
  aureleoules:
    ACK 0cc23fc603

Tree-SHA512: 4e390ceefb847d3bbe4f5caab390a4fdd14892fe443f58c32b08b3444fccd611cff22938c3dfa611dfd2497736f779fae4165497b4208e48aa8fc9d2236f943b
2022-10-27 11:02:22 +01:00
furszy
3fcb545ab2 bench: benchmark transaction creation process
Goal 1:
Benchmark the transaction creation process for pre-selected-inputs only.
Setting `m_allow_other_inputs=false` to disallow the wallet to include coins automatically.

Goal 2:
Benchmark the transaction creation process for pre-selected-inputs and coin selection.

-----------------------

Benchmark Setup:
1) Generates a 5k blockchain, loading the wallet with 5k transactions with two outputs each.
2) Fetch 4 random UTXO from the wallet's available coins and pre-select them as inputs inside CoinControl.

Benchmark (Goal 1):
Call `CreateTransaction` providing the coin control, who has set `m_allow_other_inputs=false` and
the manually selected coins.

Benchmark (Goal 2):
Call `CreateTransaction` providing the coin control, who has set `m_allow_other_inputs=true` and
the manually selected coins.
2022-10-26 15:54:31 -03:00
furszy
a8a75346d7 wallet: SelectCoins, return early if target is covered by preset-inputs 2022-10-26 15:54:31 -03:00
furszy
f41712a734 wallet: simplify preset inputs selection target check
we are already computing the preset inputs total amount inside `PreSelectedInputs::Insert`,
which internally decides whether to use the effective value or the raw output value based on
the 'subtract_fee_outputs' flag.
2022-10-26 15:54:31 -03:00
furszy
5baedc3351 wallet: remove fetch pre-selected-inputs responsibility from SelectCoins
so if there is an error in any of the pre-set coins, we can fail right away
without computing the wallet available coins set (calling `AvailableCoins`)
which is a slow operation as it goes through the entire wallet's txes map.

----------------------

And to make the Coin Selection flow cleared, have decoupled SelectCoins in two functions:

1) AutomaticCoinSelection.
2) SelectCoins.

1) AutomaticCoinSelection:
   Receives a set of coins and selects the best subset of them to
   cover the target amount.

2) SelectCoins
   In charge of select all the user manually selected coins first ("pre-set inputs"), and
   if coin_control 'm_allow_other_inputs=true', call 'AutomaticCoinSelection' to select a
   subset of coins owned by the wallet to cover for the target - preset_inputs.total_amount
   remaining value.
2022-10-26 15:54:31 -03:00
furszy
295852f619 wallet: encapsulate pre-selected-inputs lookup into its own function
First step towards decoupling the pre-selected-inputs fetching functionality
from `SelectCoins`. Which, will let us not waste resources calculating the
available coins if one of the pre-set inputs has an error.

(right now, if one of the pre-set inputs is invalid, we first walk through
the entire wallet txes map just to end up failing right after it finish)
2022-10-26 15:52:35 -03:00
furszy
37e7887cb4 wallet: skip manually selected coins from 'AvailableCoins' result
No need to walk through the entire wallet's txes map just to get
coins that we could have gotten by just doing a simple map.find(out.hash).
(Which is what we are doing inside `SelectCoins` anyway)
2022-10-26 15:52:35 -03:00
furszy
94c0766b0c wallet: skip available coins fetch if "other inputs" are disallowed
no need to waste resources calculating the wallet available coins if
they are not going to be used.

The 'm_allow_other_inputs=true` default value change is to correct
an ugly misleading behavior:

The tx creation process was having a workaround patch to automatically
fall back to select coins from the wallet if `m_allow_other_inputs=false`
(previous default value) and no manual inputs were selected.

This could be seen in master in flows like `sendtoaddress`, `sendmany`
and even the GUI, where the `m_allow_other_inputs` value isn't customized
and the wallet still selects and adds coins to the tx internally.
2022-10-26 15:47:51 -03:00
MacroFake
ec92d23fb8 Merge bitcoin/bitcoin#26395: rpc: add missing lock around chainman.ActiveTip()
f5ff3d773c rpc: add missing lock around chainman.ActiveTip() (Andrew Toth)

Pull request description:

  https://github.com/bitcoin/bitcoin/pull/23927 seems to have missed a lock around `chainman.ActiveChain()`.

ACKs for top commit:
  aureleoules:
    ACK f5ff3d773c

Tree-SHA512: 3f116ca44c1b2bc0c7042698249ea3417dfb7c0bb81158a7ceecd087f1e02baa89948f9bb7924b1757798a1691a7de6e886aa72a0a9e227c13a3f512cc59d6c9
2022-10-26 18:05:30 +02:00
Andrew Toth
f5ff3d773c rpc: add missing lock around chainman.ActiveTip() 2022-10-26 11:46:39 -04:00
Andrew Chow
e25de33e7b Merge bitcoin/bitcoin#26341: test: add BIP158 false-positive element check in rpc_scanblocks.py
fa54d3011e test: check for false-positives in rpc_scanblocks.py (Sebastian Falbesoner)
3bca6cd61a test: add compact block filter (BIP158) helper routines (Sebastian Falbesoner)
25ee74dd11 test: add SipHash implementation for generic data in Python (Sebastian Falbesoner)

Pull request description:

  This PR adds a fixed false-positive element check to the functional test rpc_scanblocks.py by using a pre-calculated scriptPubKey that collides with the regtest genesis block's coinbase output. Note that determining a BIP158 false-positive at runtime would also be possible, but take too long (we'd need to create and check ~800k output scripts on average, which took at least 2 minutes on average on my machine).

  The introduced check is related to issue #26322 and more concretely inspired by PR #26325 which introduces an "accurate" mode that filters out these false-positives. The introduced cryptography routines (siphash for generic data) and helpers (BIP158 ranged hash calculation, relevant scriptPubKey per block determination) could potentially also be useful for more tests in the future that involve compact block filters.

ACKs for top commit:
  achow101:
    ACK fa54d3011e

Tree-SHA512: c6af50864146028228d197fb022ba2ff24d1ef48dc7d171bccfb21e62dd50ac80db5fae0c53f5d205edabd48b3493c7aa2040f628a223e68df086ec2243e5a93
2022-10-26 11:46:20 -04:00
Andrew Chow
88502ecf08 Merge bitcoin/bitcoin#23927: rpc: Pruning nodes can not fetch blocks before syncing past their height
5826bf546e test: Add test for getblockfrompeer on syncing pruned nodes (Fabian Jahr)
7fa851fba8 rpc: Pruned nodes can not fetch unsynced blocks (Fabian Jahr)

Pull request description:

  This PR prevents `getblockfrompeer` from getting used on blocks that the node has not synced past yet if the node is in running in prune mode.

  ### Problem

  While a node is still catching up to the tip that it is aware of via the headers, the user can currently use  to fetch blocks close to or at the tip. These blocks are stored in the block/rev file that otherwise contains blocks the node is receiving as part of the syncing process.

  This creates a problem for pruned nodes: The files containing a fetched block are not pruned during syncing because they contain a block close to the tip. This means the entire file (~130MB) will not be pruned until the tip has moved on far enough from the fetched block. In extreme cases with heavy pruning (like 550) and multiple blocks being fetched this could mean that the disc usage far exceeds what the user expects, potentially running out of space.

  ### Approach

  There would be certainly other approaches that could fix the problem while still allowing the current behavior, but all of the ideas I came up with seemed like overkill for a niche problem on a new RPC where it's still unclear how and how much it will be used.

  ### Testing

  So far I did not see a simple enough way to test this I am still looking into it and if it's complex will potentially add it in a follow-up. What would be needed is a way to have a node fetch headers but not sync the blocks yet, that seems like a pattern that could be generally useful.

  To manually reproduce the problematic behavior:
  1. Start a node with current `master` with `-prune=550` and an empty/new datadir, Testnet and Mainnet should both work.
  2. While the node is syncing run `getblockfrompeer` on the current tip and a few other recent blocks.
  3. Go to your datadir and observe the blocks folder: There should be a few full `blk*.dat` and `rev*.dat` files that are not being pruned. When you "pinned" a few of these files the blocks folder should be significantly above the target size of 550MB.

ACKs for top commit:
  Sjors:
    utACK 5826bf546e
  achow101:
    ACK 5826bf546e
  aureleoules:
    tACK 5826bf546e

Tree-SHA512: aa3f477ec755a9df2331c047cb10b3cd08292522bf6ad7a36a7ea36d7eba4894b84de8bd23003c9baea5ac0c53b77142c3c2819ae7528cece9d10a0d06c850d8
2022-10-26 11:27:31 -04:00
Andrew Chow
48af307481 Merge bitcoin/bitcoin#25957: wallet: fast rescan with BIP157 block filters for descriptor wallets
0582932260 test: add test for fast rescan using block filters (top-up detection) (Sebastian Falbesoner)
ca48a4694f rpc: doc: mention rescan speedup using `blockfilterindex=1` in affected wallet RPCs (Sebastian Falbesoner)
3449880b49 wallet: fast rescan: show log message for every non-skipped block (Sebastian Falbesoner)
935c6c4b23 wallet: take use of `FastWalletRescanFilter` (Sebastian Falbesoner)
70b3513904 wallet: add `FastWalletRescanFilter` class for speeding up rescans (Sebastian Falbesoner)
c051026586 wallet: add method for retrieving the end range for a ScriptPubKeyMan (Sebastian Falbesoner)
845279132b wallet: support fetching scriptPubKeys with minimum descriptor range index (Sebastian Falbesoner)
088e38d3bb add chain interface methods for using BIP 157 block filters (Sebastian Falbesoner)

Pull request description:

  ## Description

  This PR is another take of using BIP 157 block filters (enabled by `-blockfilterindex=1`) for faster wallet rescans and is a modern revival of #15845. For reviewers new to this topic I can highly recommend to read the corresponding PR review club (https://bitcoincore.reviews/15845).

  The basic idea is to skip blocks for deeper inspection (i.e. looking at every single tx for matches) if our block filter doesn't match any of the block's spent or created UTXOs are relevant for our wallet. Note that there can be false-positives (see https://bitcoincore.reviews/15845#l-199 for a PR review club discussion about false-positive rates), but no false-negatives, i.e. it is safe to skip blocks if the filter doesn't match; if the filter *does* match even though there are no wallet-relevant txs in the block, no harm is done, only a little more time is spent extra.

  In contrast to #15845, this solution only supports descriptor wallets, which are way more widespread now than back in the time >3 years ago. With that approach, we don't have to ever derive the relevant scriptPubKeys ourselves from keys before populating the filter, and can instead shift the full responsibility to that to the `DescriptorScriptPubKeyMan` which already takes care of that automatically. Compared to legacy wallets, the `IsMine` logic for descriptor wallets is as trivial as checking if a scriptPubKey is included in the ScriptPubKeyMan's set of scriptPubKeys (`m_map_script_pub_keys`): e191fac4f3/src/wallet/scriptpubkeyman.cpp (L1703-L1710)

  One of the unaddressed issues of #15845 was that [the filter was only created once outside the loop](https://github.com/bitcoin/bitcoin/pull/15845#discussion_r343265997) and as such didn't take into account possible top-ups that have happened. This is solved here by keeping a state of ranged `DescriptorScriptPubKeyMan`'s descriptor end ranges and check at each iteration whether that range has increased since last time. If yes, we update the filter with all scriptPubKeys that have been added since the last filter update with a range index equal or higher than the last end range. Note that finding new scriptPubKeys could be made more efficient than linearly iterating through the whole `m_script_pub_keys` map (e.g. by introducing a bidirectional map), but this would mean introducing additional complexity and state and it's probably not worth it at this time, considering that the performance gain is already significant.

  Output scripts from non-ranged `DescriptorScriptPubKeyMan`s (i.e. ones with a fixed set of output scripts that is never extended) are added only once when the filter is created first.

  ## Benchmark results

  Obviously, the speed-up indirectly correlates with the wallet tx frequency in the scanned range: the more blocks contain wallet-related transactions, the less blocks can be skipped due to block filter detection.

  In a [simple benchmark](https://github.com/theStack/bitcoin/blob/fast_rescan_functional_test_benchmark/test/functional/pr25957_benchmark.py), a regtest chain with 1008 blocks (corresponding to 1 week) is mined with 20000 scriptPubKeys contained (25 txs * 800 outputs) each. The blocks each have a weight of ~2500000 WUs and hence are about 62.5% full. A global constant `WALLET_TX_BLOCK_FREQUENCY` defines how often wallet-related txs are included in a block. The created descriptor wallet (default setting of `keypool=1000`, we have 8*1000 = 8000 scriptPubKeys at the start) is backuped via the `backupwallet` RPC before the mining starts and imported via `restorewallet` RPC after. The measured time for taking this import process (which involves a rescan) once with block filters (`-blockfilterindex=1`) and once without block filters (`-blockfilterindex=0`) yield the relevant result numbers for the benchmark.

  The following table lists the results, sorted from worst-case (all blocks contain wallte-relevant txs, 0% can be skipped) to best-case (no blocks contain walltet-relevant txs, 100% can be skipped) where the frequencies have been picked arbitrarily:

  wallet-related tx frequency; 1 tx per...    | ratio of irrelevant blocks  | w/o filters | with filters | speed gain
  --------------------------------------------|-----------------------------|-------------|--------------|-------------
  ~ 10 minutes (every block)                  |              0%             |   56.806s   |   63.554s    |  ~0.9x
  ~ 20 minutes (every 2nd block)              |           50% (1/2)         |   58.896s   |   36.076s    |  ~1.6x
  ~ 30 minutes (every 3rd block)              |          66.67% (2/3)       |   56.781s   |   25.430s    |  ~2.2x
  ~ 1 hour (every 6th block)                  |          83.33% (5/6)       |   58.193s   |   15.786s    |  ~3.7x
  ~ 6 hours (every 36th block)                |          97.22% (35/36)     |   57.500s   |    6.935s    |  ~8.3x
  ~ 1 day (every 144th block)                 |         99.31% (143/144)    |   68.881s   |    6.107s    | ~11.3x
    (no txs)                                  |              100%           |   58.529s   |    5.630s    | ~10.4x

  Since even the (rather unrealistic) worst-case scenario of having wallet-related txs in _every_ block of the rescan range obviously doesn't take significantly longer, I'd argue it's reasonable to always take advantage of block filters if they are available and there's no need to provide an option for the user.

  Feedback about the general approach (but also about details like naming, where I struggled a lot) would be greatly appreciated. Thanks fly out to furszy for discussing this subject and patiently answering basic question about descriptor wallets!

ACKs for top commit:
  achow101:
    ACK 0582932260
  Sjors:
    re-utACK 0582932260
  aureleoules:
    ACK 0582932260 - minor changes, documentation and updated test since last review
  w0xlt:
    re-ACK 0582932260

Tree-SHA512: 3289ba6e4572726e915d19f3e8b251d12a4cec8c96d041589956c484b5575e3708b14f6e1e121b05fe98aff1c8724de4564a5a9123f876967d33343cbef242e1
2022-10-26 11:19:19 -04:00
Elichai Turkel
0cc23fc603 Fix typo in comment SHA256->SHA512 2022-10-26 15:55:29 +03:00
MacroFake
69b10212ea Merge bitcoin/bitcoin#26381: test: Fix intermittent issue in p2p_sendtxrcncl.py
fa3da8307b test: Check debug log as well in p2p_sendtxrcncl.py (MacroFake)
fae0439486 test: Check correct disconnect reason in p2p_sendtxrcncl.py (MacroFake)
fa590cfaae test: Fix intermittent issue in p2p_sendtxrcncl.py (MacroFake)

Pull request description:

  Should fix https://github.com/bitcoin/bitcoin/issues/26364

  I can't reproduce this, but my guess would be that `PeerNoVerack::on_version`, which sends the `wtxidrelay` message, is executed in the event loop and thus may run after the main thread sending `msg_verack`.

  Also, fix another bug.

  Finally, add some `assert_debug_log` to ensure the right code branch is executed (and not some random, unrelated disconnect).

ACKs for top commit:
  naumenkogs:
    ACK fa3da8307b

Tree-SHA512: ee2988372223ea22e94e9783ef6d37114a3945991b6d60f0157917f3850fb7077c566564c3771d915ee74ab28202a3b73c6d89ddec6e2c918462db9a3c02e6cf
2022-10-26 12:36:11 +02:00
dergoegge
784b023191 [net processing] Simplify use of IsContinuationOfLowWorkHeadersSync in TryLowWorkHeaderSync
`m_headers_sync` is already reset in IsContinuationOfLowWorkHeadersSync
if there is a failure, so there is no need to also reset in
TryLowWorkHeaderSync.
2022-10-26 11:12:03 +01:00
MacroFake
a1fff275e7 Merge bitcoin/bitcoin#25704: refactor: Remove almost all validation option globals
aaaa7bd0ba iwyu: Add missing includes (MacroFake)
fa9ebec096 Remove g_parallel_script_checks (MacroFake)
fa7c834b9f Move ::fCheckBlockIndex into ChainstateManager (MacroFake)
fa43188d86 Move ::fCheckpointsEnabled into ChainstateManager (MacroFake)
cccca83099 Move ::nMinimumChainWork into ChainstateManager (MacroFake)
fa29d0b57c Move ::hashAssumeValid into ChainstateManager (MacroFake)
faf44876db Move ::nMaxTipAge into ChainstateManager (MacroFake)

Pull request description:

  It seems preferable to assign globals to a class (in this case `ChainstateManager`), than to leave them dangling. This should clarify scope for code-readers, as well as clarifying unit test behaviour.

ACKs for top commit:
  dergoegge:
    Code review ACK aaaa7bd0ba
  ryanofsky:
    Code review ACK aaaa7bd0ba. No changes since last review, other than rebase
  aureleoules:
    reACK aaaa7bd0ba

Tree-SHA512: 83ec3ba0fb4f1dad95810d4bd4e578454e0718dc1bdd3a794cc4e48aa819b6f5dad4ac4edab3719bdfd5f89cbe23c2740a50fd56c1ff81c99e521c5f6d4e898d
2022-10-26 11:41:57 +02:00
MacroFake
cf288377c0 Merge bitcoin/bitcoin#26275: Fix crash on deriveaddresses when index is 2147483647 (2^31-1)
9153ff3e27 rpc: add non-regression test about deriveaddresses crash when index is 2147483647 (muxator)
addf9d6502 rpc: fix crash in deriveaddresses when derivation index is 2147483647 (muxator)

Pull request description:

  This PR is a proposal for fixing #26274 (better described there).

  The problem is due to a signed int wrapping when the `index` parameter of the `deriveaddresses` RPC call has the value `2^31-1`.

  ```C++
  for (int i = range_begin; i <= range_end; ++i) {
  ```

  * the first commit adds a "temporary" test case (`test/functional/rpc_deriveaddresses_crash.py`) that shows the crash, and can be used to generate a core dump;
  * the second commit fixes the problem giving an explicit size to the `i` variable in a for loop, from `int` to `int64_t`. The same commit also removes the ephemeral test case and adds a passing test to `test/functional/rpc_deriveaddresses.py`, in order to prevent future regressions.

  This is my first submission to this project and I do not know its conventions. Please advise if something needs to be changed.

ACKs for top commit:
  achow101:
    ACK 9153ff3e27

Tree-SHA512: 0477b57b15dc2c682cf539d6002f100d44a8c7e668041aa3340c39dcdbd40e083c75dec6896b6c076b044a01c2e5254272ae6696d8a1467539391926f270940a
2022-10-26 10:12:27 +02:00
w0xlt
eb679a7896 rpc: make address field optional 2022-10-26 01:18:28 -03:00
fanquake
28cf756971 Merge bitcoin/bitcoin#23578: Add external signer taproot support
796b020c37 wallet: add taproot support to external signer (Sjors Provoost)

Pull request description:

  Builds on #22558 (merged on 2022-06-28).

  [HWI 2.1.0](https://github.com/bitcoin-core/HWI/releases/tag/2.1.0) or newer is required to import and use taproot descriptors. Older versions will work, but won't import a taproot descriptor.

  Tested with HWI 2.1.1:
  * Trezor T (firmware v2.5.1) on Signet: signs, change detection works
  * Ledger Nano S (firmware 2.1.0, Bitcoin app 2.0.6): signs, change detection works

  Only the most basic `tr(key)` descriptor is supported, script path spending is completely untested (if it works at all).

ACKs for top commit:
  jb55:
    utACK 796b020c37
  achow101:
    ACK 796b020c37

Tree-SHA512: 6dcb7eeb45421a3bbf2bdabeacd29979867db69077d7bf192bb77faa4bfefe446487b8df07bc40f9457009a88e598bdc09f769e6106fed2833ace7ef205a157a
2022-10-26 11:10:23 +08:00
Sebastian Falbesoner
0582932260 test: add test for fast rescan using block filters (top-up detection) 2022-10-25 15:57:39 +02:00
Sebastian Falbesoner
ca48a4694f rpc: doc: mention rescan speedup using blockfilterindex=1 in affected wallet RPCs 2022-10-25 15:57:39 +02:00
Sebastian Falbesoner
3449880b49 wallet: fast rescan: show log message for every non-skipped block
For that purpose, a new logging category BCLog::SCAN is introduced.
2022-10-25 15:57:38 +02:00
Sebastian Falbesoner
935c6c4b23 wallet: take use of FastWalletRescanFilter
Can be reviewed with `--ignore-all-space`.
2022-10-25 15:57:38 +02:00
Sebastian Falbesoner
70b3513904 wallet: add FastWalletRescanFilter class for speeding up rescans
This only supports wallet descriptors right now.
2022-10-25 15:57:38 +02:00
Sebastian Falbesoner
c051026586 wallet: add method for retrieving the end range for a ScriptPubKeyMan 2022-10-25 15:57:38 +02:00
Sebastian Falbesoner
845279132b wallet: support fetching scriptPubKeys with minimum descriptor range index
This extra method will be needed for updating the filter set for
faster wallet rescans; after an internal top-up has happened, we only
want to add the newly created scriptPubKeys.
2022-10-25 15:57:38 +02:00
Sebastian Falbesoner
088e38d3bb add chain interface methods for using BIP 157 block filters
This is useful for speeding up wallet rescans and is based on an
earlier version from PR #15845 ("wallet: Fast rescan with BIP157 block
filters"), which was never merged.

Co-authored-by: MacroFake <falke.marco@gmail.com>
2022-10-25 15:57:28 +02:00
Hennadii Stepanov
da16893474 ci: Use macos-ventura-xcode:14.1 image for "macOS native" task 2022-10-25 13:39:03 +01:00
MacroFake
fa3da8307b test: Check debug log as well in p2p_sendtxrcncl.py 2022-10-25 13:26:36 +02:00
MacroFake
fae0439486 test: Check correct disconnect reason in p2p_sendtxrcncl.py
Previously it disconnected due to "sendtxrcncl received after verack",
now it disconnects for the correct reason.
2022-10-25 13:26:29 +02:00
Hennadii Stepanov
702836530f ci: Make getopt path architecture agnostic 2022-10-25 09:49:07 +01:00
fanquake
bfce05cc34 Merge bitcoin/bitcoin#26372: build, msvc: Drop no longer required macro definitions for leveldb
393be86724 build, msvc: Drop no longer required macro definitions for leveldb (Hennadii Stepanov)

Pull request description:

  Since levedb v1.21:
  - the `__STDC_LIMIT_MACROS` macro definition is unneeded;
    commit: [50fbc87e8c62a816d6afd4740e0652a13ac6dc3e](50fbc87e8c)

  - the `LEVELDB_ATOMIC_PRESENT` macro is unused;
    commit: [04f39105c5a418905da8b7657ca244d672c99d3b](04f39105c5)

ACKs for top commit:
  sipsorcery:
    tACK 393be86724.
  jarolrod:
    crACK 393be86724

Tree-SHA512: 845bb310474cf17f9f0f70a585054a222914313a0fc28c42a023133250efbb9ab7fb2d1ebb355943233b6fd24d5bddfb73fbbf5dce71ea232ae652af8b695149
2022-10-25 10:07:16 +08:00
dergoegge
e891aabf5a [net processing] Fixup TryLowWorkHeadersSync comment 2022-10-24 22:05:59 +01:00
Martin Zumsande
0565951f34 p2p: Make block stalling timeout adaptive
This makes the stalling detection mechanism (previously a fixed
timeout of 2s) adaptive:
If we disconnect a peer for stalling, double the timeout for the
next peer - and let it slowly relax back to its default
value each time the tip advances. (Idea by Pieter Wuille)

This makes situations more unlikely in which we'd keep on
disconnecting many of our peers for stalling, even though our
own bandwidth is insufficient to download a block in 2 seconds.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2022-10-24 16:15:22 -04:00
Larry Ruane
db929893ef Faster -reindex by initially deserializing only headers
When a block is initially read from a blk*.dat file during reindexing,
it can be added to the block index only if all of its ancestor blocks
have been added, which is rare. If the block's ancestors have not been
added, the block must be re-read from disk later when it can be added.

This commit: During the initial block read, deserialize only its header,
rather than the entire block, since this is sufficient to determine
if its parent (and thus all its ancestors) has been added. This is a
performance improvement.
2022-10-24 13:02:37 -06:00
Larry Ruane
c72de9990a util: add CBufferedFile::SkipTo() to move ahead in the stream
SkipTo() reads data from the file into the CBufferedFile object
(memory), but, unlike this object's read() method, SkipTo() doesn't
transfer data into a caller's memory buffer. This is useful because
after skipping forward in the stream in this way, the user can, if
needed, rewind the stream (SetPos()) and access the object's memory
buffer including ranges that were skipped over (without needing to
read from the disk file).
2022-10-24 13:02:37 -06:00
Larry Ruane
48a68908ba Add LoadExternalBlockFile() benchmark 2022-10-24 13:02:35 -06:00
MacroFake
1c5c951713 Merge bitcoin/bitcoin#26380: Revert "test: check importing wallets when blocks are pruned throw an error"
e1eadaa72d Revert "test: check importing wallets when blocks are pruned throw an error" (Aurèle Oulès)

Pull request description:

  The test doesn't pass (not detected by the normal CI, because it is an extended test):

  ```
  Traceback (most recent call last):
    File "/tmp/cirrus-ci-build/bitcoin-core/test/functional/test_framework/test_framework.py", line 133, in main
      self.run_test()
    File "/tmp/cirrus-ci-build/bitcoin-core/test/functional/feature_pruning.py", line 480, in run_test
      self.wallet_test()
    File "/tmp/cirrus-ci-build/bitcoin-core/test/functional/feature_pruning.py", line 361, in wallet_test
      assert_raises_rpc_error(-4, "Importing wallets is disabled when blocks are pruned", self.nodes[2].importwallet, "abc")
    File "/tmp/cirrus-ci-build/bitcoin-core/test/functional/test_framework/util.py", line 130, in assert_raises_rpc_error
      assert try_rpc(code, message, fun, *args, **kwds), "No exception raised"
    File "/tmp/cirrus-ci-build/bitcoin-core/test/functional/test_framework/util.py", line 145, in try_rpc
      raise AssertionError(
  AssertionError: Expected substring not found in error message:
  substring: 'Importing wallets is disabled when blocks are pruned'
  error message: 'Only legacy wallets are supported by this command'.
  ```

  So revert it for now, which will be done anyway in https://github.com/bitcoin/bitcoin/pull/24865/commits. (This commit is taken from there)

ACKs for top commit:
  andrewtoth:
    ACK e1eadaa72d

Tree-SHA512: 10f556ea1aa97dc9cb64f91055977eecb9534f658170aabb4909c3e85ca6c20588c7a021219356fab678e0e2bec999d347facd00054f07a9445ad393e6353b4c
2022-10-24 16:51:52 +02:00
glozow
3d0fca1288 Merge bitcoin/bitcoin#26355: p2p: Handle IsContinuationOfLowWorkHeadersSync return value correctly when new headers sync is started
7ad15d1100 [net processing] Handle IsContinuationOfLowWorkHeadersSync return value correctly when new headers sync is started (dergoegge)

Pull request description:

  This PR fixes a bug in the headers sync logic that enables submitting headers to a nodes block index that don't lead to a chain that surpasses our DoS limit.

  The issue is that we ignore the return value on [the first `IsContinuationOfLowWorkHeadersSync` call after a new headers sync is started](fabc031048/src/net_processing.cpp (L2553-L2568)), which leads to us passing headers to [`ProcessNewBlockHeaders`](fabc031048/src/net_processing.cpp (L2856)) when that initial `IsContinuationOfLowWorkHeadersSync` call returns `false`. One easy way (maybe the only?) to trigger this is by sending 2000 headers where the last header has a different `nBits` value than the prior headers (which fails the pre-sync logic [here](fabc031048/src/headerssync.cpp (L189))). Those 2000 headers will be passed to `ProcessNewBlockHeaders`.

  I haven't included a test here so far because we can't test this without changing the default value for `CRegTestParams::consensus.fPowAllowMinDifficultyBlocks` or doing some more involved refactoring.

ACKs for top commit:
  sipa:
    ACK 7ad15d1100
  glozow:
    ACK 7ad15d1100

Tree-SHA512: 9aabb8bf3700401e79863d0accda0befd2a83c4d469a53f97d827e51139e2f826aee08cdfbc8866b311b153f61fdac9b7aa515fcfa2a21c5e2812c2bf3c03664
2022-10-24 15:38:37 +01:00
MacroFake
fa590cfaae test: Fix intermittent issue in p2p_sendtxrcncl.py 2022-10-24 16:07:23 +02:00
MacroFake
fa579f3063 refactor: Pass reference to last header, not pointer
It is never a nullptr, otherwise an assertion would fire in
UpdatePeerStateForReceivedHeaders.

Passing a reference makes the code easier to read and less brittle.
2022-10-24 14:51:35 +02:00
Hennadii Stepanov
884304e6c6 test: Make system_tests/run_command locale agnostic 2022-10-24 13:36:04 +01:00
MacroFake
fa2d01470a test: Use type-safe NodeSeconds for TestMemPoolEntryHelper 2022-10-24 11:33:33 +02:00
MacroFake
3db23fd821 Merge bitcoin-core/gui#676: Update peers window "Transaction Relay" label and tooltip
a079103c94 gui: update peers window "Transaction Relay" label and tooltip (Jon Atack)

Pull request description:

  to current v24.0 p2p behavior.  Similar updates have been made to RPC getpeerinfo and CLI -netinfo.

Top commit has no ACKs.

Tree-SHA512: 400a794f655f799eefcb77c479fef6bcd3f81aede2af54a4a9bcb7c0c783e2e3f18bc5fd2484a79e8c30af279747a05fc0ebb69dbc47375d4c55b16ceba97b99
2022-10-24 10:42:47 +02:00
MacroFake
8fb3fd2ba4 Merge bitcoin-core/gui#673: Use fallback value for Version and User Agent during peer connection
c2a21c0670 gui: use fallback value for Version and User Agent during peer connection (Jon Atack)

Pull request description:

  During connection setup for a peer, getpeerinfo returns `"version": 0, "subver": ""` and the GUI Peers window displays 0 and an empty field, respectively.

  Give these fields the same behavior as the other fields in the GUI Peers window: display the fallback value in `src/qt/forms/debugwindow.ui` (i.e. `N/A`) until a valid result is available after the peer connection completes.

  An alternative would be to display nothing for both, as is the case currently for User Agent.

ACKs for top commit:
  jarolrod:
    ACK c2a21c0670
  furszy:
    code ACK c2a21c06

Tree-SHA512: 4f0060fa9abde120a2bb48c9dcc87894d9bb70c33e6ab43b22400a4bcd0ceff0fa098adf7f385b0a7a4cf5d7053463b36fe1232e19a8d5025eecd8db9833f73b
2022-10-24 10:40:05 +02:00
MacroFake
c05673577d Merge bitcoin/bitcoin#26358: doc: Rearrange a few lines in the dependency graph of libraries
1184a66347 doc: Rearrange some lines in the dependency graph of libraries (Stacie Waleyko)

Pull request description:

  In this PR, I've attempted to improve readability in the [dependency graph of libraries](https://github.com/bitcoin/bitcoin/blob/master/doc/design/libraries.md) by untangling a few crossed lines. I'm not sure if this is that big of an improvement but  wanted to throw it out there.

  I used an extremely scientific method of manually counting the number of crossed lines in the original diagram and got 15. This PR reduces that number down to about 10.

  I also changed the curve of the lines to "basis" which rounds the edges out. Again, not sure if it really is that much of an improvement, but it seems marginally easier on the eyes.

  Here is what the new graph looks like rendered:

  ![Screenshot from 2022-10-20 22-09-30](https://user-images.githubusercontent.com/1823216/197095545-5fc90cce-a817-4db2-a0f5-1a8a95380b70.png)

  The changes can be verified independently with [Mermaid](https://mermaid-js.github.io/mermaid/#/), with the easiest way being the online editor: https://mermaid.live/

  I did try moving some more stuff around, particularly the top level of library callers, but was not able to simplify the graph any further.

ACKs for top commit:
  shaavan:
    ACK 1184a66347

Tree-SHA512: 61d33d68c1e6fa354aebdda5e06e9c7a722ca20886c6acc30dd08af7133d737130d7a646d87f9e5a8ae0bc5a5aabfbc64ded9ee04dfeed8f23d948444add916b
2022-10-24 10:28:03 +02:00
fanquake
50cc8ef5a7 Merge bitcoin/bitcoin#26302: refactor: Use type-safe time point for CWallet::m_next_resend
fa51cc9651 refactor: Use type-safe time point for CWallet::m_next_resend (MacroFake)

Pull request description:

  `GetTime` is not type-safe, thus deprecated, see 75cbbfa279/src/util/time.h (L62-L70)

ACKs for top commit:
  shaavan:
    Code Review ACK fa51cc9651
  aureleoules:
    ACK fa51cc9651

Tree-SHA512: 030de10070518580763ea75079442e2f934c54d3083be3ebe35e7f1bc6db2096745bb46d95aa1e6efe29ced30a048acfe5cd999178e6787b7647dfbec5ecb444
2022-10-24 10:11:13 +08:00
Hennadii Stepanov
45a0f4e014 Update minisketch subtree to latest upstream 2022-10-23 15:03:04 +01:00
Hennadii Stepanov
e9f1d8c272 Squashed 'src/minisketch/' changes from 47f0a2d26f..a571ba20f9
a571ba20f9 Merge sipa/minisketch#68: Add missed `#include <string>`
b9a7f7e2bc Merge sipa/minisketch#69: refactor: Drop unused `total` local variables
8a5af94edc Merge sipa/minisketch#70: build: Remove `-Qunused-arguments` workaround for clang + ccache
c36f1f03a3 Merge sipa/minisketch#72: Fix MSVC implementation of `CountBits()` function
0078bedda6 Ignore `HAVE_CLZ` macro when building with MSVC
1c772918c4 Fix MSVC implementation of `CountBits()` function
98f87c55f4 build: Remove `-Qunused-arguments` workaround for clang + ccache
11a1e25c81 refactor: Drop unused `total` local variables
ed6c8fcfd9 Add missed `#include <string>`

git-subtree-dir: src/minisketch
git-subtree-split: a571ba20f9dd1accab6a2309d066369878042ca6
2022-10-23 15:00:20 +01:00
Hennadii Stepanov
393be86724 build, msvc: Drop no longer required macro definitions for leveldb
Since levedb v1.21:
- the `__STDC_LIMIT_MACROS` macro definition is unneeded;
  commit: 50fbc87e8c62a816d6afd4740e0652a13ac6dc3e

- the `LEVELDB_ATOMIC_PRESENT` macro is unused;
  commit: 04f39105c5a418905da8b7657ca244d672c99d3b
2022-10-23 08:05:43 +01:00
Fabian Jahr
d885bb2f6e test: Test exclusion of OP_RETURN from getblockstats 2022-10-23 01:33:41 +02:00
Fabian Jahr
ba9d288b24 test: Fix getblockstats test data generator
The getblockstats RPC functional test is using previously generated test data that is part of the repository. That test data can be regenerated by running the test file with `--gen-test-data` which invokes the `generate_test_data()` function. That function still relied on the old wallet behavior of having a default wallet to work. Because of this the function was broken and this change fixes this. The fact that this was broken did was not noticed previously because the function is not used by the automated test suite by default.
2022-10-23 01:33:41 +02:00
Fabian Jahr
2ca5a496c2 rpc: Improve getblockstats
- Fix getblockstats for block height 0 which previously returned an error.
- Introduce alternative utxo_*_actual statistics which exclude unspendables: Genesis block, BIP30, unspendable outputs
- Update test data
- Explicitly test Genesis block results
2022-10-23 01:33:41 +02:00
Fabian Jahr
cb94db119f validation, index: Add unspendable coinbase helper functions
Making the checks to identify BIP30 available outside of validation.cpp is needed for reporting and tracking statistics on specific blocks and the UTXO set correctly.
2022-10-23 01:33:36 +02:00
dergoegge
7ad15d1100 [net processing] Handle IsContinuationOfLowWorkHeadersSync return value correctly when new headers sync is started 2022-10-21 11:05:34 +01:00
MacroFake
6d40484684 Merge bitcoin/bitcoin#26352: doc: add scanblocks to list of descriptor RPCs
ff138f9cf1 doc: add `scanblocks` to list of descriptor RPCs (Sebastian Falbesoner)

Pull request description:

  This is a tiny documentation follow-up to #23549.

ACKs for top commit:
  aureleoules:
    ACK ff138f9cf1
  shaavan:
    ACK ff138f9cf1

Tree-SHA512: cc45b78c13ec4aa5bac688648f8e83f04a9ae54376e67371b280428f0253e2585cf0575fa51e3835d4c13c940559bfcdd88d7504bf97a81b2a73bb34a0db7577
2022-10-21 11:32:13 +02:00
MacroFake
8c5c98db47 Merge bitcoin/bitcoin#26248: net: Set relay in version msg to peers with relay permission in -blocksonly mode
dddd1acf58 net: Set relay in version msg to peers with relay permission (MacroFake)

Pull request description:

  Seems odd to set the `relay` permission in -blocksonly mode and also ask the peer not to relay transactions.

ACKs for top commit:
  dergoegge:
    ACK dddd1acf58
  naumenkogs:
    ACK dddd1acf58
  mzumsande:
    ACK dddd1acf58

Tree-SHA512: 7bb0e964993ea4982747ae2801fe963ff88586e2ded03015b60ab83172b5b61f2d50e9cde9d7711b7ab207f8639467ecafc4d011ea151ec6c82c722f510f4df7
2022-10-21 11:18:48 +02:00
fanquake
afbcd227dd build: remove threadinterrupt from libbitcoinkernel 2022-10-21 16:40:55 +08:00
fanquake
f2859c3aa8 Merge bitcoin/bitcoin#25727: util, config: error on startup if conf or reindex are set in config file
deba6fe315 test: update feature_config_args.py (josibake)
2e3826cbcd util: warn if reindex is used in conf (josibake)
5e744f4238 util: disallow setting conf in bitcoin.conf (josibake)

Pull request description:

  In help from `bitcoind -h` it specifes that `conf` can only be used from the commandline. However, if `conf` is set in a `bitcoin.conf` file, there is no error and from reading the logs it seems as if the `conf=<other file>` is being used, despite it being ignored. To recreate, you can setup a `bitcoin.conf` file in the default directory, add `conf=<some other file>.conf` and in the separate config file set whichever config value you want and verify that it is being ignored. alternatively, if you set `includeconf=<some other file>.conf` , your config in `<some other file>` will be picked up.

  This PR fixes this by having the node error when reading the config file if `conf=` is set.

  Additionally, it was mentioned in a recent [PR review club](https://bitcoincore.reviews/24858) that if `reindex=1` is set in the config file, the node will reindex on every startup, which is undesirable:
   ```irc
  17:14 <larryruane> michaelfolkson: Reindex is requested by the user (node operator) as a configuration option (command line or in the config file, tho you probably would never put it in the file, or else it would reindex on every startup!)
  ```

  This PR also has a commit to warn if `reindex=1` is set in the config file.

ACKs for top commit:
  hebasto:
    ACK deba6fe315, tested on Ubuntu 22.04.
  aureleoules:
    tACK deba6fe315
  ryanofsky:
    Code review ACK deba6fe315.

Tree-SHA512: 619fd0aa14e98af1166d6beb92651f5ba3f10d38b8ee132957f094f19c3a37313d9f4d7be2e4019f3fc9a2ca5fa42d03eb539ad820e27efec7ee58a26eb520b1
2022-10-21 16:39:44 +08:00
fanquake
6da45649c2 Merge bitcoin/bitcoin#26259: test: Test year 2106 block timestamps
fafc96aaf4 test: Test year 2106 block timestamps (MacroFake)

Pull request description:

  Alternative to https://github.com/bitcoin/bitcoin/pull/21362 that closes https://github.com/bitcoin/bitcoin/issues/21356

ACKs for top commit:
  Sjors:
    utACK fafc96a

Tree-SHA512: 196d98f42d6f7f0222312b7bd1c68b3bd30cb6f0cbaccb900cfc5fcc689494adb2a7d7d6023c1ff1e8cf871047ec37eeca41386e31029d99cabf9343b4fd2a03
2022-10-21 16:29:52 +08:00
fanquake
085f83940d Merge bitcoin/bitcoin#26344: wallet: Fix sendall with watchonly wallets and specified inputs
315fd4dbab test: Test for out of bounds vout in sendall (Andrew Chow)
b132c85650 wallet: Check utxo prevout index out of bounds in sendall (Andrew Chow)
708b72b715 test: Test that sendall works with watchonly spending specific utxos (Andrew Chow)
6bcd7e2a3b wallet: Correctly check ismine for sendall (Andrew Chow)

Pull request description:

  The `sendall` RPC would previously fail when used with a watchonly wallet and specified inputs. This failure was caused by checking isminetype equality with ISMINE_ALL rather than a bitwise AND as IsMine can never return ISMINE_ALL.

  Also added a test.

ACKs for top commit:
  w0xlt:
    ACK 315fd4dbab
  furszy:
    ACK 315fd4db

Tree-SHA512: fb55cf6524e789964770b803f401027319f0351433ea084ffa7c5e6f1797567a608c956b7f7c5bd542aa172c4b7b38b07d0976f5ec587569efead27266e8664c
2022-10-21 16:24:15 +08:00
Sjors Provoost
682283445e guix: bump recommended hash for manual installation
Tracks time-machine bump in 298389e3b5.
2022-10-21 09:50:19 +02:00
Stacie Waleyko
1184a66347 doc: Rearrange some lines in the dependency graph of libraries 2022-10-20 22:09:54 -04:00
Sebastian Falbesoner
ff138f9cf1 doc: add scanblocks to list of descriptor RPCs 2022-10-20 23:24:16 +02:00
Andrew Chow
315fd4dbab test: Test for out of bounds vout in sendall 2022-10-20 13:25:13 -04:00
Andrew Chow
b132c85650 wallet: Check utxo prevout index out of bounds in sendall 2022-10-20 13:24:57 -04:00
Andrew Chow
708b72b715 test: Test that sendall works with watchonly spending specific utxos 2022-10-20 13:21:03 -04:00
Andrew Chow
fabc031048 Merge bitcoin/bitcoin#26158: bench: add "priority level" to the benchmark framework
3e9d0bea8d build: only run high priority benchmarks in 'make check' (furszy)
466b54bd4a bench: surround main() execution with try/catch (furszy)
3da7cd2a76 bench: explicitly make all current benchmarks "high" priority (furszy)
05b8c76232 bench: add "priority level" to the benchmark framework (furszy)
f1593780b8 bench: place benchmark implementation inside benchmark namespace (furszy)

Pull request description:

  This is from today's meeting, a simple "priority level" for the benchmark framework.

  Will allow us to run certain benchmarks while skip non-prioritized ones in `make check`.

  By default, `bench_bitcoin` will run all the benchmarks. `make check`will only run the high priority ones,
  and have marked all the existent benchmarks as "high priority" to retain the current behavior.

  Could test it by modifying any benchmark priority to something different from "high", and
  run `bench_bitcoin -priority-level=high` and/or `bench_bitcoin -priority-level=medium,low`
  (the first command will skip the modified bench while the second one will include it).

  Note: the second commit could be avoided by having a default arg value for the priority
  level but.. an explicit set in every `BENCHMARK` macro call makes it less error-prone.

ACKs for top commit:
  kouloumos:
    re-ACK 3e9d0bea8d
  achow101:
    ACK 3e9d0bea8d
  theStack:
    re-ACK 3e9d0bea8d
  stickies-v:
    re-ACK 3e9d0bea8d

Tree-SHA512: ece59bf424c5fc1db335f84caa507476fb8ad8c6151880f1f8289562e17023aae5b5e7de03e8cbba6337bf09215f9be331e9ef51c791c43bce43f7446813b054
2022-10-20 11:05:03 -04:00
furszy
3e9d0bea8d build: only run high priority benchmarks in 'make check' 2022-10-20 10:21:05 -03:00
furszy
466b54bd4a bench: surround main() execution with try/catch
so we have a cleaner exit on internal runtime errors.
e.g. an unknown priority level.
2022-10-20 10:21:04 -03:00
furszy
3da7cd2a76 bench: explicitly make all current benchmarks "high" priority
no-functional changes. Only have set the priority level explicitly
on every BENCHMARK macro call.
2022-10-20 10:21:04 -03:00
furszy
05b8c76232 bench: add "priority level" to the benchmark framework
Will allow us to run certain benchmarks while skip
non-prioritized ones in 'make check'.
2022-10-20 10:21:04 -03:00
MacroFake
fafc96aaf4 test: Test year 2106 block timestamps
* Use maximum timestamp in getblocktemplate test
* Mine block with maximum timestamp and MTP in blockchain test
2022-10-20 14:45:50 +02:00
Aurèle Oulès
04609284ad rpc: Improve error when wallet is already loaded 2022-10-20 11:51:37 +02:00
fanquake
2ac71d20b2 Merge bitcoin/bitcoin#25595: Verify PSBT inputs rather than check for fields being empty
e133264c5b Add test for PSBT input verification (Greg Sanders)
d25699280a Verify PSBT inputs rather than check for fields being empty (Greg Sanders)

Pull request description:

  In a few keys spots, PSBT finality is checked by looking for non-empty witness data.

  This complicates a couple things:
  1) Empty data can be valid in certain cases
  2) User may be passed bogus final data by a counterparty during PSBT work happening, and end up with incorrect signatures that they may not be able to check in other contexts if the UTXO doesn't exist yet in chain/mempool, timelocks, etc.

  On the whole I think these heavier checks are worth it in case someone is actually assuming the signatures are correct if our API is saying so.

ACKs for top commit:
  achow101:
    ACK e133264c5b

Tree-SHA512: 9de4fbb0be1257b081781f5df908fd55666e3acd5c4e36beb3b3f2f5a6aed69ff77068c44cde6127e159e773293fd9ced4c0bb47e693969f337e74dc8af030da
2022-10-20 08:13:14 +08:00
Sebastian Falbesoner
fa54d3011e test: check for false-positives in rpc_scanblocks.py 2022-10-20 01:33:17 +02:00
Sebastian Falbesoner
3bca6cd61a test: add compact block filter (BIP158) helper routines
By now, we add one helper for calculating ranged hashes and another one
for finding relevant scriptPubKeys given a block.
2022-10-20 01:33:17 +02:00
Sebastian Falbesoner
25ee74dd11 test: add SipHash implementation for generic data in Python
We will need this in the next commit to calculate ranged hashes
of scriptPubKeys as defined in BIP158.
2022-10-20 01:32:48 +02:00
Andrew Chow
6bcd7e2a3b wallet: Correctly check ismine for sendall
sendall should be using a bitwise AND for sendall's IsMine check rather
than an equality as IsMine will never return ISMINE_ALL.
2022-10-19 15:13:11 -04:00
Aurèle Oulès
e1eadaa72d Revert "test: check importing wallets when blocks are pruned throw an error"
This reverts commit 4aff7a48a4.
2022-10-19 16:51:47 +02:00
Aurèle Oulès
ed52e71176 Periodically check disk space to avoid corruption 2022-10-19 10:41:15 +02:00
Aurèle Oulès
7fe537f7a4 Implement CCoinsViewErrorCatcher::HaveCoin 2022-10-19 10:41:15 +02:00
MacroFake
a97791d9fb Merge bitcoin/bitcoin#25830: refactor: Replace m_params with chainman.GetParams()
5d3f98d278 refactor: Replace m_params with chainman.GetParams() (Aurèle Oulès)

Pull request description:

  Fixes a TODO introduced in #24595.
  Removes `m_params` from `CChainState` class and replaces it with `m_chainman.GetParams()`.

ACKs for top commit:
  MarcoFalke:
    review ACK 5d3f98d278 🌎

Tree-SHA512: de0fe31450d281cc7307c0d820495e86c93c7998e77a148db2c703da66cff1059e6560c041f1864913c42075aa24d259c2623d45e929ca0a8056ed330a9f9978
2022-10-19 10:04:34 +02:00
MacroFake
003050dfaf Merge bitcoin/bitcoin#26286: test: Remove unused txmempool include from tests
1c48dae76f test: Use C++11 member initializers for TestMemPoolEntryHelper (MacroFake)
fad7f2239c test: Remove unused txmempool include from tests (MacroFake)

Pull request description:

  Seems odd to include this heavy header in all tests despite it only being used in a few tests.

  Can be reviewed with `--color-moved=dimmed-zebra --ignore-all-space`

ACKs for top commit:
  aureleoules:
    reACK 1c48dae76f
  hebasto:
    ACK 1c48dae76f, I have reviewed the code and it looks OK, I agree it can be merged.
  w0xlt:
    ACK 1c48dae76f

Tree-SHA512: 31f2808d04ec33bfc2409832b8e59e6c870eaa98fbcf879e1c786492c7d07134711b30f8290bdb34e1b8f7b8f2f11dae8e10c64e7eb31f584b2f5c58fcc7743b
2022-10-19 09:41:47 +02:00
MacroFake
bbe2655309 Merge bitcoin/bitcoin#26142: Use PACKAGE_NAME in messages rather than hardcoding "Bitcoin Core"
b147322a7a Use `PACKAGE_NAME` in messages rather than hardcoding "Bitcoin Core" (Hennadii Stepanov)

Pull request description:

  Usually, we do not hardcode "Bitcoin Core" in the user-faced messages.

  See:
  - bitcoin/bitcoin#18646
  - bitcoin/bitcoin#19282

  Also grammar has been improved -- singular instead of plural.

ACKs for top commit:
  jarolrod:
    ACK b147322a7a

Tree-SHA512: b135c18703dfdd7b63d4cb27d1ac48f6a9dbf69382142ae381f33bf561cbf57477a11d1c73263aa834f705206d7dd5716df2523d38ed0d4cfec8babc38bb017a
2022-10-19 09:22:22 +02:00
MacroFake
c102a558e1 Merge bitcoin/bitcoin#26179: bench: Add missed ECCVerifyHandle instance
f09d47b263 bench: Add missed `ECCVerifyHandle` instance (Hennadii Stepanov)

Pull request description:

  To clearly observe the lack of an `ECCVerifyHandle` instance,
  - apply the following diff:
  ```diff
  --- a/src/Makefile.bench.include
  +++ b/src/Makefile.bench.include
  @@ -19,11 +19,9 @@ bench_bench_bitcoin_SOURCES = \
     bench/bench.h \
     bench/bench_bitcoin.cpp \
     bench/block_assemble.cpp \
  -  bench/ccoins_caching.cpp \
     bench/chacha20.cpp \
     bench/chacha_poly_aead.cpp \
     bench/checkblock.cpp \
  -  bench/checkqueue.cpp \
     bench/crypto_hash.cpp \
     bench/data.cpp \
     bench/data.h \
  @@ -46,8 +44,7 @@ bench_bench_bitcoin_SOURCES = \
     bench/rpc_blockchain.cpp \
     bench/rpc_mempool.cpp \
     bench/strencodings.cpp \
  -  bench/util_time.cpp \
  -  bench/verify_script.cpp
  +  bench/util_time.cpp

   nodist_bench_bench_bitcoin_SOURCES = $(GENERATED_BENCH_FILES)

  ```
  - then
  ```
  $ ./autogen
  $ ./configure
  $ make clean
  $ make
  ```
  - then
  ```
  $ ./src/bench/bench_bitcoin -filter=ExpandDescriptor
  bench_bitcoin: pubkey.cpp:296: bool CPubKey::IsFullyValid() const: Assertion `secp256k1_context_verify && "secp256k1_context_verify must be initialized to use CPubKey."' failed.
  Aborted (core dumped)
  ```

ACKs for top commit:
  achow101:
    ACK f09d47b263
  w0xlt:
    ACK f09d47b263

Tree-SHA512: e1f33f88d427c57fe31d5810d12e9f46fed2911f5736208ebf7d4a968de0dd8c1f6b73a0d1093316da117dd3bcfda5dde6e41d6c95fcdb99bdea62e19df5ad20
2022-10-19 09:16:29 +02:00
MacroFake
0aa641f5f4 Merge bitcoin/bitcoin#26206: test: check importing wallets when blocks are pruned throw an error
4aff7a48a4 test: check importing wallets when blocks are pruned throw an error (brunoerg)

Pull request description:

  This PR adds test coverage for the following error:
  437b608df2/src/wallet/rpc/backup.cpp (L513-L518)

ACKs for top commit:
  andrewtoth:
    ACK 4aff7a48a4

Tree-SHA512: fbbf6056cb3759f726b8a5ff25fca51bf47e973e5d655ec164e2bec88e2dbd3b243677869d2cf33af268ea635ca0f2e9f737c4734077fc5a936ac3a24ad4b88b
2022-10-19 09:11:47 +02:00
Jon Atack
a079103c94 gui: update peers window "Transaction Relay" label and tooltip
to current v24.0 p2p behavior
2022-10-18 15:26:52 -07:00
Dhruv Mehta
2555a3950f p2p: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified 2022-10-18 09:51:36 -07:00
MacroFake
1c48dae76f test: Use C++11 member initializers for TestMemPoolEntryHelper
Co-authored-by: Aurèle Oulès <aurele@oules.com>
2022-10-18 17:32:44 +02:00
MacroFake
aaaa7bd0ba iwyu: Add missing includes 2022-10-18 14:12:52 +02:00
MacroFake
fa9ebec096 Remove g_parallel_script_checks 2022-10-18 14:12:42 +02:00
MacroFake
fa7c834b9f Move ::fCheckBlockIndex into ChainstateManager
This changes the flag for the bitcoin-chainstate executable. Previously
it was false, now it is the chain's default value (still false for the
main chain).
2022-10-18 14:11:48 +02:00
MacroFake
fa43188d86 Move ::fCheckpointsEnabled into ChainstateManager 2022-10-18 14:10:50 +02:00
MacroFake
cccca83099 Move ::nMinimumChainWork into ChainstateManager
This changes the minimum chain work for the bitcoin-chainstate
executable. Previously it was uint256{}, now it is the chain's default
minimum chain work.
2022-10-18 14:09:17 +02:00
MacroFake
fa29d0b57c Move ::hashAssumeValid into ChainstateManager
This changes the assumed valid block for the bitcoin-chainstate
executable. Previously it was uint256{}, now it is defaultAssumeValid.
2022-10-18 14:08:49 +02:00
MacroFake
faf44876db Move ::nMaxTipAge into ChainstateManager 2022-10-18 14:07:59 +02:00
MacroFake
fad7f2239c test: Remove unused txmempool include from tests 2022-10-18 14:02:09 +02:00
fanquake
5974c49f90 Merge bitcoin/bitcoin#26321: Adjust .tx/config for new Transifex CLI
d6adbb7ee1 Adjust `.tx/config` for new Transifex CLI (Hennadii Stepanov)

Pull request description:

  The old Transifex Command-Line Tool is considered deprecated (as of January 2022) and will sunset on Nov 30, 2022.

  See: https://github.com/transifex/cli/blob/devel/README.md#migrating-from-older-versions-of-the-client

  An accompanying PR: https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/142

ACKs for top commit:
  jarolrod:
    ACK d6adbb7ee1

Tree-SHA512: aa63158e02dc136be1669f3ff9ca28a990da215227520a197968ebe1a0a6fa73526cbd6d8fd7eed608eb06ad242c66598826c58e124c1fc53beaa28ae6b53c37
2022-10-18 19:58:24 +08:00
MacroFake
ba441d493c Merge bitcoin/bitcoin#26313: doc: consolidate library documentation to libraries.md
af781bf4b2 doc: fix typo in doc/libraries.md (fanquake)
9e9ae6101f doc: remove library commentary from src/Makefile.am (fanquake)

Pull request description:

  Deduplicate the makefile comments, in favour of doc/libraries.md. I think a single, more comprehensive source of truth is preferable. Diagrams are also useful. Came up in https://github.com/bitcoin/bitcoin/pull/26292#issuecomment-1275094478.

ACKs for top commit:
  ryanofsky:
    Code review ACK af781bf4b2, nice cleanups
  hebasto:
    ACK af781bf4b2, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: df61ed1394102221701ae2dfa42886dfabe9d9fd7f601b794e2195f93d8f7c2a1cd1c000a77d0a969b42328e8ebc0387755c57291837b283fdf376dbd98fdda1
2022-10-18 13:44:56 +02:00
glozow
e7a0e96271 Merge bitcoin/bitcoin#23443: p2p: Erlay support signaling
e56d1d2afd test: Add functional tests for sendtxrcncl message from outbound (Gleb Naumenko)
cfcef60779 test: Add functional tests for sendtxrcncl from inbound (Gleb Naumenko)
b99ee9d22d test: Add unit tests for reconciliation negotiation (Gleb Naumenko)
f63f1d3f4b p2p: clear txreconciliation state for non-wtxid peers (Gleb Naumenko)
88d326c8e3 p2p: Finish negotiating reconciliation support (Gleb Naumenko)
36cf6bf216 Add helper to see if a peer is registered for reconciliations (Gleb Naumenko)
4470acf076 p2p: Forget peer's reconciliation state on disconnect (Gleb Naumenko)
3fcf78ee6a p2p: Announce reconciliation support (Gleb Naumenko)
24e36fac0a log: Add tx reconciliation log category (Gleb Naumenko)

Pull request description:

  This is a part of the Erlay project:
  - [parent PR](https://github.com/bitcoin/bitcoin/pull/21515)
  - [associated BIP-330](https://github.com/bitcoin/bips/pull/1376).

  -------

  This PR adds a new p2p message `sendtxrcncl` signaling for reconciliation support.
  Before sending that message, a node is supposed to "pre-register" the peer by generating and storing an associated reconciliation salt component.
  Once the salts are exchanged within this new message, nodes "register" each other for future reconciliations by computing and storing the aggregate salt, along with the reconciliation parameters based on the connection direction.

ACKs for top commit:
  dergoegge:
    re-ACK e56d1d2afd
  sipa:
    re-ACK e56d1d2afd. No differences with a rebase of previously reviewed e91690e67dad180c7fb9bed0409a9c4567d3e5df.
  mzumsande:
    re-ACK e56d1d2afd
  vasild:
    ACK e56d1d2afd

Tree-SHA512: 0db953b7347364e2496ebca3bfe6a27ac336307eec698242523a18336fcfc7a1ab87e3b09ce8b2bdf800ebbb1c9d33736ffdb8f5672f93735318789aa4a45f39
2022-10-17 18:10:56 +01:00
Greg Sanders
e133264c5b Add test for PSBT input verification 2022-10-17 11:13:30 -04:00
Greg Sanders
d25699280a Verify PSBT inputs rather than check for fields being empty 2022-10-17 11:13:02 -04:00
Andrew Chow
a52ff619a4 Merge bitcoin/bitcoin#25985: Revert "build: Use Homebrew's sqlite package if it is available"
d216d714aa Revert "build: Use Homebrew's sqlite package if it is available" (fanquake)

Pull request description:

  This reverts ee7b84e63c from #20527.

  That change was made without any rationale, maybe other than, a brew
  installed version might be newer, and that's "better". However when
  building from source on macOS, it just results in drastically worse
  performance, and issues / confusion like #25724.

  The difference in performance can be observed using the example from https://github.com/bitcoin/bitcoin/issues/25724#issuecomment-1213554922,
  but minified i.e:
  ```bash
  time src/bitcoin-cli createwallet speedy true
  time src/bitcoin-cli importdescriptors '[
    {"desc":"raw(00145846369f3d6ba366d6f5a903fb5cf4dca3763c0e)#k9wh6v62","timestamp":"now"},
    {"desc":"raw(001420800aabf13f3a4c4ce3ce4c66cecf1d17f21a6e)#6m0hlfh4","timestamp":"now"},
    {"desc":"raw(0014c6bf9715e06d73ebf9b3b02d5cc48d24d8bbabc1)#wyavh36r","timestamp":"now"},
    {"desc":"raw(00141ba7807b3f46af113beaea5c698428ce7138cd8a)#jctdsups","timestamp":"now"},
    {"desc":"raw(00140c1bd27f10fff01b36ddf3c1febaa1acff19b080)#9s6nc3pk","timestamp":"now"},
    {"desc":"raw(00141226e31987e4bc2e63c0ee12908f675e40464b20)#9pp7qm39","timestamp":"now"},
    {"desc":"raw(0014f73f149f7503960a5e849c6ee7a8a8c336f631cb)#qtkxv9fc","timestamp":"now"},
    {"desc":"raw(0014c8ccb4d81ffc769fc5fdd8d7eed69b0e0cae5749)#hn39qayv","timestamp":"now"},
    {"desc":"raw(001498565aead2d67a22a6021d55210f2a917fc22169)#6ar3vwsx","timestamp":"now"},
    {"desc":"raw(001403013248ac0cd9eabe176cad162cda2a19f771e1)#4m47mukd","timestamp":"now"}
  ]'
  ```

  Running master, when building from souce and using brew installed
  sqlite, this takes ~3.4s. With this PR, the same operation takes ~0.3s.

  Resolves the "build from source" portion of #25724. Building from
  depends is still not ideal, however I have some other changes that might
  help improve things in that case.

  Related performance issue reports:
  * https://github.com/bitcoindevkit/bdk/issues/749
  * https://bitcoin.stackexchange.com/questions/113898/bitcoin-v23-is-10-times-slower-than-v22-on-macos-for-basic-regtest-tests
  * https://github.com/bitcoin/bitcoin/issues/25724
  * https://github.com/bitcoin/bitcoin/pull/25985#issuecomment-1245942400

ACKs for top commit:
  achow101:
    ACK d216d714aa
  jarolrod:
    ACK d216d714aa
  hebasto:
    ACK d216d714aa, I have reviewed the code and it looks OK, I agree it can be merged. No conflicts with our build [docs](d216d714aa/doc/build-osx.md (descriptor-wallet-support)).

Tree-SHA512: 1bb4b44385b11fa9fe66edd7449278f9e47a6cc679b7111f9adf17db94c34e29c9cceafc917454e134420db40b24b56da29226af6f43e6dbeff822b79b77ed60
2022-10-17 10:29:41 -04:00
Gleb Naumenko
e56d1d2afd test: Add functional tests for sendtxrcncl message from outbound 2022-10-17 12:36:14 +03:00
Gleb Naumenko
cfcef60779 test: Add functional tests for sendtxrcncl from inbound 2022-10-17 12:36:13 +03:00
Gleb Naumenko
b99ee9d22d test: Add unit tests for reconciliation negotiation 2022-10-17 12:35:44 +03:00
Gleb Naumenko
f63f1d3f4b p2p: clear txreconciliation state for non-wtxid peers
We optimistically pre-register a peer for txreconciliations
upon sending txreconciliation support announcement.
But if, at VERACK, we realize that the peer never sent
WTXIDRELAY message, we should unregister the peer
from txreconciliations, because txreconciliations rely on wtxids.
2022-10-17 12:35:44 +03:00
Gleb Naumenko
88d326c8e3 p2p: Finish negotiating reconciliation support
Once we received a reconciliation announcement support
message from a peer and it doesn't violate our protocol,
we store the negotiated parameters which will be used
for future reconciliations.
2022-10-17 12:35:44 +03:00
Gleb Naumenko
36cf6bf216 Add helper to see if a peer is registered for reconciliations 2022-10-17 12:35:44 +03:00
Gleb Naumenko
4470acf076 p2p: Forget peer's reconciliation state on disconnect 2022-10-17 12:35:44 +03:00
Gleb Naumenko
3fcf78ee6a p2p: Announce reconciliation support
If we're connecting to the peer which might support
transaction reconciliation, we announce we want to reconcile
with them.

We store the reconciliation salt so that when the peer
responds with their salt, we are able to compute the
full reconciliation salt.

This behavior is enabled with a CLI flag.
2022-10-17 12:35:43 +03:00
Gleb Naumenko
24e36fac0a log: Add tx reconciliation log category 2022-10-17 12:00:59 +03:00
Andrew Chow
c35b91afdc Merge bitcoin/bitcoin#26319: add 0xb10c builder key
5165ae1405 add 0xb10c builder key (0xb10c)

Pull request description:

  I've been asked to add my key given my [activity as GUIX builder](https://github.com/bitcoin-core/guix.sigs/commits?author=0xB10C).

ACKs for top commit:
  achow101:
    ACK 5165ae1405
  1440000bytes:
    ACK 5165ae1405
  hebasto:
    ACK 5165ae1405, the added fingerprint is the same as one in my local gpg output.

Tree-SHA512: 794b01c87dec5139cd9dd3a1ec7ca4dd21351b16b46a4ea64c3be0e569ff20a301cdfa45873663f446e0a59d6319950f77c32f776260cff63e176b81ed262be3
2022-10-15 14:17:48 -04:00
Hennadii Stepanov
d6adbb7ee1 Adjust .tx/config for new Transifex CLI
The old Transifex Command-Line Tool is considered deprecated (as of
January 2022) and will sunset on Nov 30, 2022.

See: https://github.com/transifex/cli/blob/devel/README.md#migrating-from-older-versions-of-the-client

An accompanying PR: https://github.com/bitcoin-core/bitcoin-maintainer-tools/pull/142
2022-10-15 19:11:39 +01:00
0xb10c
5165ae1405 add 0xb10c builder key 2022-10-15 18:43:07 +02:00
Vasil Dimov
4163093d63 wallet: use Mutex for g_sqlite_mutex instead of GlobalMutex
Using `Mutex` provides stronger guarantee than `GlobalMutex` wrt Clang's
thread safety analysis. Thus it is better to reduce the usage of
`GlobalMutex` in favor of `Mutex`.

Using `Mutex` for `g_sqlite_mutex` is ok because its usage is limited in
`wallet/sqlite.cpp` and it does not require propagating the negative
annotations to not relevant code.
2022-10-14 14:36:12 +02:00
fanquake
af781bf4b2 doc: fix typo in doc/libraries.md 2022-10-14 17:07:39 +08:00
fanquake
9e9ae6101f doc: remove library commentary from src/Makefile.am
This duplicates and is less explanatory than doc/libraries.md.
2022-10-14 17:07:36 +08:00
MacroFake
3f1f5f6f1e Merge bitcoin/bitcoin#26307: test: use MiniWallet for rpc_scanblocks.py
ae3626ea52 test: use MiniWallet for rpc_scanblocks.py (Sebastian Falbesoner)

Pull request description:

  This is a small follow-up for #23549 which introduced `scanblocks`. Since that RPC doesn't need the wallet, we can switch the functional test to use MiniWallet.

ACKs for top commit:
  MarcoFalke:
    review ACK ae3626ea52

Tree-SHA512: e0b0088103e059b29719299c58fd5173b1cff58cb73025a9a33ad493cd0ac50ba25a5f790e471c00a09b4dca80f3c011174ca4e0c8f34746c39831f5823dc8ba
2022-10-14 10:21:32 +02:00
fanquake
3b85e17b49 Merge bitcoin/bitcoin#26297: ci: Use all available CPUs for functional tests in "Win64 native" task
6fbd173d8a ci: Use all available CPUs for functional tests in "Win64 native" task (Hennadii Stepanov)

Pull request description:

  On the [master](https://cirrus-ci.com/task/5422842484359168) branch:
  ![Screenshot from 2022-10-12 09-45-58](https://user-images.githubusercontent.com/32963518/195296883-3852ea09-7345-4166-b855-7704dcd87202.png)

  This [PR](https://cirrus-ci.com/task/6392972617973760) branch:
  ![Screenshot from 2022-10-12 11-11-15](https://user-images.githubusercontent.com/32963518/195315902-f667874a-8aeb-4f2f-bdc3-5ba432ae9353.png)

  Also consider "CPU Usage" charts provided by CI.

  Overlooked in cda62657e9 (bitcoin/bitcoin#25929).

ACKs for top commit:
  hebasto:
    Indeed. Reverted back to 6fbd173d8a ([pr26297.01](https://github.com/hebasto/bitcoin/commits/pr26297.01)), which was already [ACKed](https://github.com/bitcoin/bitcoin/pull/26297#pullrequestreview-1138724890) by @aureleoules.
  aureleoules:
    ACK 6fbd173d8a
  jarolrod:
    ACK 6fbd173d8a
  shaavan:
    ACK 6fbd173d8a

Tree-SHA512: ddd4b41af95bd735f881a3b2c64ee308de2725381f770e313e66555f929d88c8848c98cc5fcd15dfa6845b5dd84ca6c8764ef5d01602b0a62041820856af2b98
2022-10-14 11:37:44 +08:00
Sebastian Falbesoner
ae3626ea52 test: use MiniWallet for rpc_scanblocks.py 2022-10-13 22:07:59 +02:00
Andrew Chow
deeb70a165 Merge bitcoin/bitcoin#26306: add lock annotation for FeeFilterRounder::round()
cbb2da8fcf add lock annotation for FeeFilterRounder::round() (glozow)

Pull request description:

  CI failure from #24407: https://github.com/bitcoin/bitcoin/runs/8876014446

  Calling `WITH_LOCK()` on a non-recursive mutex requires not holding it beforehand.

ACKs for top commit:
  achow101:
    ACK cbb2da8fcf
  dergoegge:
    ACK cbb2da8fcf
  hebasto:
    ACK cbb2da8fcf, tested on Ubuntu 22.04 with clang 14.0.

Tree-SHA512: d6782ee48442b9d64d58a54c1ec7c53822ab051bf9728b44d6a0e05f1953e90f16420d349379345845db203fbad4e1f5750d9070adcb7daa18f12359a29488ca
2022-10-13 16:02:13 -04:00
glozow
cbb2da8fcf add lock annotation for FeeFilterRounder::round()
Calling WITH_LOCK() on a non-recursive mutex requires not holding it beforehand.

Co-authored-by: Niklas Gögge <n.goeggi@gmail.com>
2022-10-13 14:42:59 -04:00
Andrew Chow
0384b19414 Merge bitcoin/bitcoin#24851: init: ignore BIP-30 verification in DisconnectBlock for problematic blocks
e899d4ca6f init: limit bip30 exceptions to coinbase txs (Chris Geihsler)
511eb7fdea Ignore problematic blocks in DisconnectBlock (Chris Geihsler)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/22596

  When using checklevel=4, block verification fails because of duplicate coinbase transactions involving blocks 91812 and 91722. There was already a check in place within `ConnectBlock` to ignore the problematic blocks, but `DisconnectBlock` did not contain a similar check to ignore these blocks when called from `VerifyDB`.

  By ignoring these two blocks in `DisconnectBlock`, the block verification process succeeds at checklevel=4.

  (Note to reviewers: this is my first contribution to Bitcoin Core, so any feedback is most welcome. Thanks in advance for reviewing!)

  ## Steps to reproduce:

  Use the following bitcoin.conf file and start bitcoind. I only used block data through block ~100000 so that the verification process was much faster.

  ```
  assumevalid=0
  checkblocks=0
  checklevel=4
  ```

  Without this change, you will see the following error when the blocks are verified:

  ```
  2022-04-14T02:56:44Z init message: Verifying blocks…
  2022-04-14T02:56:44Z Verifying last 101881 blocks at level 4
  2022-04-14T02:56:44Z [0%]...[10%]...[20%]...[30%]...[40%]...ERROR: VerifyDB(): *** coin database inconsistencies found (last 10160 blocks, 142571 good transactions before that)

  2022-04-14T02:57:01Z : Corrupted block database detected.
  Please restart with -reindex or -reindex-chainstate to recover.
  : Corrupted block database detected.
  Please restart with -reindex or -reindex-chainstate to recover.
  ```

  With this change, you will see this instead:

  ```
  2022-04-14T02:32:29Z init message: Verifying blocks…
  2022-04-14T02:32:29Z Verifying last 101746 blocks at level 4
  2022-04-14T02:32:29Z [0%]...[10%]...[20%]...[30%]...[40%]...[50%]...[60%]...[70%]...[80%]...[90%]...[DONE].
  2022-04-14T02:32:48Z No coin database inconsistencies in last 101746 blocks (226126 transactions)
  ```

ACKs for top commit:
  laanwj:
    Code review ACK e899d4ca6f
  achow101:
    ACK e899d4ca6f
  jamesob:
    (Biased) ACK e899d4ca6f ([`jamesob/ackr/24851.2.seejee.init_ignore_bip_30_verif`](https://github.com/jamesob/bitcoin/tree/ackr/24851.2.seejee.init_ignore_bip_30_verif))

Tree-SHA512: d2f6d25e9619aee32c1a73fe846b1b587698eaa5a4994fa6424f1038f45654f9fd52b74a69843cc84d90168d74827130ccf8e9201502f5d52281acdb20429291
2022-10-13 14:15:28 -04:00
Andrew Chow
92be831847 Merge bitcoin/bitcoin#25412: rest: add /deploymentinfo endpoint
a8250e30f1 doc: add release note about `/rest/deploymentinfo` (brunoerg)
5c96020024 doc: add `/deploymentinfo` in REST-interface (brunoerg)
3e44bee08e test: add coverage for `/rest/deploymentinfo` (brunoerg)
91497031cb rest: add `/deploymentinfo` (brunoerg)

Pull request description:

  #23508 added a new RPC named `getdeploymentinfo`, it moved the softfork section from `getblockchaininfo` into this new one. In the REST interface, we have an endpoint named`/rest/chaininfo.json` (which refers to `getblockchaininfo`), so, this PR adds a new REST endpoint named `/deploymentinfo` which refers to `getdeploymentinfo`.

  You can use it by passing a block hash, e.g: '/rest/deploymentinfo/<BLOCKHASH>.json' or you can use it without passing a block hash to get the 'deploymentinfo' for the last block.

ACKs for top commit:
  jonatack:
    re-ACK a8250e30f1 rebase-only since my last review at c65f82bb
  achow101:
    ACK a8250e30f1
  stickies-v:
    re-ACK a8250e30f1

Tree-SHA512: 0735183b6828d51a72ed0e2be5a09b314ac4693f548982c6e9adaa0ef07a55aa428d3b2d1b1de70b83169811a663a8624b686166e5797f624dcc00178b9796e6
2022-10-13 13:30:55 -04:00
Andrew Chow
0bac04b758 Merge bitcoin/bitcoin#24407: fees: make the class FeeFilterRounder thread-safe
8173f160e0 style: rename variables to match coding style (Vasil Dimov)
8b4ad203d0 fees: make FeeFilterRounder::feeset const (Vasil Dimov)
e7a5bf6be7 fees: make the class FeeFilterRounder thread-safe (Vasil Dimov)

Pull request description:

  Make the class `FeeFilterRounder` thread-safe so that its methods can be called concurrently by different threads on the same object. Currently it has just one method (`round()`).

  The second commit is optional, but it improves readability, showing that the `feeset` member will never be changed, thus does not need protection from concurrent access.

ACKs for top commit:
  jonatack:
    re-ACK 8173f160e0
  laanwj:
    Code review ACK 8173f160e0
  promag:
    Code review ACK 8173f160e0

Tree-SHA512: 94b809997c485c0d114fa702d0406b980be8eaaebcfefa56808ed670aa943959c2f16cfd0ef72b4752fe2a409a23af1b4b7f2f236e51212957759569e3bbbefd
2022-10-13 11:57:07 -04:00
Andrew Chow
5ff3d1e5ce Merge bitcoin/bitcoin#24269: test: add functional test for -discover
bff05bd745 test: add functional test for -discover (brunoerg)

Pull request description:

  This PR adds a functional test for `-discover`. It tests different scenarios where `localaddresses` should be empty or may contain the addresses. Obs: `localaddresses` is not always accurate, so it's not possible to ensure (100%) it will contain any addresses.

  515200298b/src/init.cpp (L449)

  Obs: See #24258  - It adds test coverage for this field but for nodes with proxy.

ACKs for top commit:
  mzumsande:
    Code review ACK bff05bd745
  achow101:
    ACK bff05bd745
  rajarshimaitra:
    tACK bff05bd745

Tree-SHA512: 8782497c146bce1ba86fda6146f3847465d7069f2cb6b84f2afc8f3b43efa813442bffe7447e9ce02adee304100b60365409bf0e5d875dfb880038442feec2a6
2022-10-13 11:47:42 -04:00
Andrew Chow
1dec90d95b Merge bitcoin/bitcoin#25526: wallet: avoid double keypool TopUp() call on descriptor wallets
bfb9b94ebe wallet: remove duplicate descriptor type check in GetNewDestination (furszy)
76b982a4a5 wallet: remove unused `nAccountingEntryNumber` field (furszy)
599ff5adfc wallet: avoid double TopUp() calls on descriptor wallets (furszy)

Pull request description:

  Found it while was digging over a `getnewaddress` timeout on the functional test suite.

  ### Context:

  We are calling `TopUp()` twice in the following flows for descriptor wallets:

  A) `CWallet::GetNewDestination`:
     1) Calls spk_man->TopUp()
     2) Calls spk_man->GetNewDestination() --> which, after the basic script checks, calls TopUp() again.

  B) `CWallet::GetReservedDestination`:
     1) Calls spk_man->TopUp()
     2) Calls spk_man->GetReservedDestination() --> which calls to GetNewDestination (which calls to TopUp again).

  ### Changes:

  Move `TopUp()` responsibility from the wallet class to each scriptpubkeyman.
  So each spkm can decide to call it or not after perform the basic checks
  for the new destination request.

  Aside from that, remove the unused `nAccountingEntryNumber` wallet field. And a duplicated descriptor type check in `GetNewDestination`

ACKs for top commit:
  aureleoules:
    re-ACK bfb9b94ebe.
  achow101:
    ACK bfb9b94ebe
  theStack:
    Code-review ACK bfb9b94ebe

Tree-SHA512: 3ab73f37729e50d6c6a4434f676855bc1fb404619d63c03e5b06ce61c292c09c59d64cb1aa3bd9277b06f26988956991d62c90f9d835884f41ed500b43a12058
2022-10-13 11:27:38 -04:00
Andrew Chow
cb9764b686 Merge bitcoin/bitcoin#26109: rpc, doc: getpeerinfo updates
a3789c700b Improve getpeerinfo pingtime, minping, and pingwait help docs (Jon Atack)
df660ddb1c Update getpeerinfo/-netinfo/TxRelay#m_relay_txs relaytxes docs (for v24 backport) (Jon Atack)
1f448542e7 Always return getpeerinfo "minfeefilter" field (for v24 backport) (Jon Atack)
9cd6682545 Make getpeerinfo field order consistent with its help (for v24 backport) (Jon Atack)

Pull request description:

  Various updates and fixups, mostly targeting v24. Please refer to the commit messages for details.

ACKs for top commit:
  achow101:
    ACK a3789c700b
  brunoerg:
    ACK a3789c700b
  vasild:
    ACK a3789c700b

Tree-SHA512: b8586a9b83c1b18786b5ac1fc1dba91573c13225fc2cfc8d078f4220967c95056354f6be13327f33b4fcf3e9d5310fa4e1bdc93102cbd6574f956698993a54bf
2022-10-13 11:07:33 -04:00
Andrew Chow
bc2b1f0fe2 Merge bitcoin/bitcoin#23549: Add scanblocks RPC call (attempt 2)
626b7c8493 fuzz: add scanblocks as safe for fuzzing (James O'Beirne)
94fe5453c7 test: rpc: add scanblocks functional test (Jonas Schnelli)
6ef2566b68 rpc: add scanblocks - scan for relevant blocks with descriptors (Jonas Schnelli)
a4258f6e81 rpc: move-only: consolidate blockchain scan args (James O'Beirne)

Pull request description:

  Revives #20664. All feedback from the previous PR has either been responded to inline or incorporated here.

  ---

  Major changes from Jonas' PR:
  - consolidated arguments for scantxoutset/scanblocks
  - substantial cleanup of the functional test

  Here's the range-diff (`git range-diff master jonasschnelli/2020/12/filterblocks_rpc jamesob/2021-11-scanblocks`): https://gist.github.com/jamesob/aa4a975344209f0316444b8de2ec1d18

  ### Original PR description

  > The `scanblocks` RPC call allows one to get relevant blockhashes from a set of descriptors by scanning all blockfilters in a given range.
  >
  > **Example:**
  >
  > `scanblocks start '["addr(<bitcoin_address>)"]' 661000` (returns relevant blockhashes for `<bitcoin_address>` from blockrange 661000->tip)
  >
  > ## Why is this useful?
  > **Fast wallet rescans**: get the relevant blocks and only rescan those via `rescanblockchain getblockheader(<hash>)[height] getblockheader(<hash>)[height])`. A future PR may add an option to allow to provide an array of blockhashes to `rescanblockchain`.
  >
  > **prune wallet rescans**: (_needs additional changes_): together with a call to fetch blocks from the p2p network if they have been pruned, it would allow to rescan wallets back to the genesis block in pruned mode (relevant #15946).
  >
  > **SPV mode** (_needs additional changes_): it would be possible to build the blockfilterindex from the p2p network (rather then deriving them from the blocks) and thus allow some sort of hybrid-SPV mode with moderate bandwidth consumption (related #9483)

ACKs for top commit:
  furszy:
    diff re-ACK 626b7c8

Tree-SHA512: f84e4dcb851b122b39e9700c58fbc31e899cdcf9b587df9505eaf1f45578cc4253e89ce2a45d1ff21bd213e31ddeedbbcad2c80810f46755b30acc17b07e2873
2022-10-13 10:48:16 -04:00
Andrew Chow
6912a28f08 Merge bitcoin/bitcoin#25667: assumeutxo: snapshot initialization
bf95976061 doc: add note about snapshot chainstate init (James O'Beirne)
e4d7995286 test: add testcases for snapshot initialization (James O'Beirne)
cced4e7336 test: move-only-ish: factor out LoadVerifyActivateChainstate() (James O'Beirne)
51fc9241c0 test: allow on-disk coins and block tree dbs in tests (James O'Beirne)
3c361391b8 test: add reset_chainstate parameter for snapshot unittests (James O'Beirne)
00b357c215 validation: add ResetChainstates() (James O'Beirne)
3a29dfbfb2 move-only: test: make snapshot chainstate setup reusable (James O'Beirne)
8153bd9247 blockmanager: avoid undefined behavior during FlushBlockFile (James O'Beirne)
ad67ff377c validation: remove snapshot datadirs upon validation failure (James O'Beirne)
34d1590331 add utilities for deleting on-disk leveldb data (James O'Beirne)
252abd1e8b init: add utxo snapshot detection (James O'Beirne)
f9f1735f13 validation: rename snapshot chainstate dir (James O'Beirne)
d14bebf100 db: add StoragePath to CDBWrapper/CCoinsViewDB (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: https://github.com/bitcoin/bitcoin/pull/15606)

  ---

  Half of the replacement for #24232. The original PR grew larger than expected throughout the review process.

  This change adds the ability to initialize a snapshot-based chainstate during init if one is detected on disk. This is of course unused as of now (aside from in unittests) given that we haven't yet enabled actually loading snapshots.

  Don't be scared! There are some big move-only commits in here.

  Accompanying changes include:

  - moving the snapshot coinsdb directory from being called `chainstate_[base blockhash]` to `chainstate_snapshot`, since we only support one snapshot in use at a time. This simplifies some logic, but it necessitates writing that base blockhash out to a file within the coinsdb dir. See [discussion here](https://github.com/bitcoin/bitcoin/pull/24232#discussion_r832762880).
  - adding a simple fix in `FlushBlockFile()` that avoids a crash when attemping to flush to disk before `LoadBlockIndexDB()` is called, which happens when calling `MaybeRebalanceCaches()` during multiple chainstate init.
  - improving the unittest to allow testing with on-disk chainstates - necessary to test a simulated restart and re-initialization.

ACKs for top commit:
  naumenkogs:
    utACK bf95976061
  ariard:
    Code Review ACK bf9597606
  ryanofsky:
    Code review ACK bf95976061. Changes since last review: rebasing, switching from CAutoFile to AutoFile, adding comments, switching from BOOST_CHECK to Assert in test util, using chainman.GetMutex() in tests, destroying one ChainstateManager before creating a new one in tests
  fjahr:
    utACK bf95976061
  aureleoules:
    ACK bf95976061

Tree-SHA512: 15ae75caf19f8d12a12d2647c52897904d27b265a7af6b4ae7b858592eeadb8f9da6c2394b6baebec90adc28742c053e3eb506119577dae7c1e722ebb3b7bcc0
2022-10-13 10:19:27 -04:00
glozow
147d64dbdf Merge bitcoin/bitcoin#25858: psbt: Only include PSBT_OUT_TAP_TREE when the output has a script path
9e386afb67 tests: Test that PSBT_OUT_TAP_TREE is included correctly (Andrew Chow)
30ff25cf37 psbt: Only include m_tap_tree if it has scripts (Andrew Chow)
0577d423ad psbt: Change m_tap_tree to store just the tuples (Andrew Chow)
22c051ca70 tests: Test that PSBT_OUT_TAP_TREE is combined correctly (Andrew Chow)
7df6e1bb77 psbt: Fix merging of m_tap_tree (Andrew Chow)
0652dc53b2 [BugFix]: Do not allow deserializing PSBT with empty PSBT_OUT_TAP_TREE (Jeremy Rubin)

Pull request description:

  PSBT_OUT_TAP_TREE should not be included for outputs that do not have such a tree. This should be disallowed during parsing, as well as prior to serialization when the field is populated during updating.

  Also added some test cases.

  Alternative to #25856

ACKs for top commit:
  instagibbs:
    ACK 9e386afb67
  darosior:
    ACK 9e386afb67

Tree-SHA512: ce5c02a69752d176dbd967c1e8d30129b1905c8f186aeeef034576c1de82059271a1ee846bd040f5be4e66bb77ba711dcf14ac1e597c5707d7e7e2293f6cfefb
2022-10-13 09:40:27 -04:00
klementtan
d96d97ad30 doc: Add release note for shutdownnotify. 2022-10-13 21:22:52 +08:00
klementtan
0bd73e2c45 util: Add -shutdownnotify option. 2022-10-13 21:22:50 +08:00
MacroFake
fa51cc9651 refactor: Use type-safe time point for CWallet::m_next_resend 2022-10-13 14:49:18 +02:00
MacroFake
75cbbfa279 Merge bitcoin/bitcoin#26299: ci: Use Multi-ToolTask in "Win64 native" task
98c389ccff ci: Use Multi-ToolTask in "Win64 native" task (Hennadii Stepanov)

Pull request description:

  See https://devblogs.microsoft.com/cppblog/improved-parallelism-in-msbuild/.

  Build time (`ccache` cache has been _invalidated_ to ensure equal initial conditions):
  - on the [master](https://cirrus-ci.com/task/4900469905555456) branch:
  ![image](https://user-images.githubusercontent.com/32963518/195401896-c714cb08-2a41-4eed-afb1-14992f5a9152.png)

  - this [PR](https://cirrus-ci.com/task/5840011785404416) changes:
  ![image](https://user-images.githubusercontent.com/32963518/195412308-3dd823bb-0ecd-481f-8ab8-05643fd4f7c0.png)

  Also consider "CPU Usage" charts provided by CI.

ACKs for top commit:
  aureleoules:
    ACK 98c389ccff 🎉
  jarolrod:
    ACK 98c389ccff

Tree-SHA512: 03b7ff48322ab8ab74ca1646df994fb6766ad4dd656f3b89e5ef5bc4756c185354689f44e5e1aa3b193aab895402b4d032b53fa9aa437455d9800e0294d86836
2022-10-13 13:26:01 +02:00
fanquake
a5f95bafcd Merge bitcoin/bitcoin#25964: build: fix mingw miniupnpc cflags
859644b3c8 build: set D_WIN32_WINNT=0x0601 for mingw miniupnpc (fanquake)
8e2d93ff0f build: fix cflags passing for mingw miniupnpc (fanquake)

Pull request description:

  Pulls in a patch I've upstreamed to miniupnpc so that we properly pass our cflags when building it for mingw. See https://github.com/miniupnp/miniupnp/pull/619. Also set `D_WIN32_WINNT` to `0x0601` to match libevent, configure etc. Previously it was being set to `0X501`.

  Guix Build (x86_64 / arm64):
  ```bash
  39a66c473a45b83ca85500b32ccf8f30d4ae80f965ca064566ee9fd84a51964b  guix-build-859644b3c855/output/aarch64-linux-gnu/SHA256SUMS.part
  7b0515e422f350cb23f4f0b2f87eaa1b30d1c80389da6f1cbe700794902c88e9  guix-build-859644b3c855/output/aarch64-linux-gnu/bitcoin-859644b3c855-aarch64-linux-gnu-debug.tar.gz
  192253fb387a2216b6d63d47a18e34bfa284874488d7ebc6ba656ca76a905519  guix-build-859644b3c855/output/aarch64-linux-gnu/bitcoin-859644b3c855-aarch64-linux-gnu.tar.gz
  0a8b5c77928a46e62dff85cf73ca9e932560533b99ef1ec374be00516f9e1183  guix-build-859644b3c855/output/arm-linux-gnueabihf/SHA256SUMS.part
  d1b01b36d7092d63ab84877e05a973d915d177dbc618fe00eeaff86295032750  guix-build-859644b3c855/output/arm-linux-gnueabihf/bitcoin-859644b3c855-arm-linux-gnueabihf-debug.tar.gz
  50d9cd81a4a37fbd5c22ee8f1b8398a836879bda1b514a9ed3d0bcd6fd3de41f  guix-build-859644b3c855/output/arm-linux-gnueabihf/bitcoin-859644b3c855-arm-linux-gnueabihf.tar.gz
  e590d6dc6687c744b4067af330a7fe44110da4972c46f0262c39ce03e2aa6ac5  guix-build-859644b3c855/output/arm64-apple-darwin/SHA256SUMS.part
  6b9b97a1f6ead6d2b70d706ba39b11f36c2929962afc0d52404f2341d412d4e7  guix-build-859644b3c855/output/arm64-apple-darwin/bitcoin-859644b3c855-arm64-apple-darwin-unsigned.dmg
  de6be985a4e1b11c6450c388b54be4fff3dc3a78e528cb628623ee4a8ea249f4  guix-build-859644b3c855/output/arm64-apple-darwin/bitcoin-859644b3c855-arm64-apple-darwin-unsigned.tar.gz
  940024658b9387040ceb26535dbd1ed7edb3709106f6e25d5d3720ed90bbbb2e  guix-build-859644b3c855/output/arm64-apple-darwin/bitcoin-859644b3c855-arm64-apple-darwin.tar.gz
  67c8fcdd31dca595e5c6b72e597a135718dd50ce1f062cb18d181520c13d3013  guix-build-859644b3c855/output/dist-archive/bitcoin-859644b3c855.tar.gz
  4b1954953913d1387589873a8e9dc9765f0f300c125270046da95a23c43aa069  guix-build-859644b3c855/output/powerpc64-linux-gnu/SHA256SUMS.part
  5b12ab7e3a7fc162912e67e026646ee5d4c92ef804525504f188d9a569af7d67  guix-build-859644b3c855/output/powerpc64-linux-gnu/bitcoin-859644b3c855-powerpc64-linux-gnu-debug.tar.gz
  fa4debb24dbb4c9c515ad3bfa2e5cb9bd686f1062157bd4480d076ef35d7ea9d  guix-build-859644b3c855/output/powerpc64-linux-gnu/bitcoin-859644b3c855-powerpc64-linux-gnu.tar.gz
  bafad2f56aad4edd25e49e4cc658811cf87f32a2a2e013672659d410d7a8cac8  guix-build-859644b3c855/output/powerpc64le-linux-gnu/SHA256SUMS.part
  96318cac800dc7ee86229754a76047dececd62ecd962362e6dbd1d694bc8c17a  guix-build-859644b3c855/output/powerpc64le-linux-gnu/bitcoin-859644b3c855-powerpc64le-linux-gnu-debug.tar.gz
  461415ee7bf67e2c59bfce3a09736b3b0ecdc7c81751ce2bed19369237450154  guix-build-859644b3c855/output/powerpc64le-linux-gnu/bitcoin-859644b3c855-powerpc64le-linux-gnu.tar.gz
  5c9d19f6af5d1fab7de7496bbdcd1cd266abd6c0e2c226b362dab2582c94a33a  guix-build-859644b3c855/output/riscv64-linux-gnu/SHA256SUMS.part
  04492f5ce121ba09672119cc8861ee30ecdb814b34726c9c5d7971c528209a55  guix-build-859644b3c855/output/riscv64-linux-gnu/bitcoin-859644b3c855-riscv64-linux-gnu-debug.tar.gz
  5fb28111b49e73fc53db2805500c275a2e6321c3e180695eb813675629dcd64c  guix-build-859644b3c855/output/riscv64-linux-gnu/bitcoin-859644b3c855-riscv64-linux-gnu.tar.gz
  3f48c1c2ba77d4fda725225f6c9b5ab7f3aae244c8abb354407cc73d6547d983  guix-build-859644b3c855/output/x86_64-apple-darwin/SHA256SUMS.part
  6dcfb5a4af350466fb7f9319e02fd4bcef66e015116c9eda8aff03b3ac53d109  guix-build-859644b3c855/output/x86_64-apple-darwin/bitcoin-859644b3c855-x86_64-apple-darwin-unsigned.dmg
  87112cc5f2d02c16614f6d8df41af6f3ea3c765eb6d196d68d2b514b6bd317bc  guix-build-859644b3c855/output/x86_64-apple-darwin/bitcoin-859644b3c855-x86_64-apple-darwin-unsigned.tar.gz
  b245a4d4881a679f2c91e0c7fd5466d6b93313289e609dbce4e1009b7591332e  guix-build-859644b3c855/output/x86_64-apple-darwin/bitcoin-859644b3c855-x86_64-apple-darwin.tar.gz
  a4c3449aa9d8a8e1c8b0532f6e74845c40f90ad4186f480d5bb2750e184efc10  guix-build-859644b3c855/output/x86_64-linux-gnu/SHA256SUMS.part
  12a32f25d6ef7f60023dd3053b21e31131350b6afa01aeab25e53d1928b1a0f5  guix-build-859644b3c855/output/x86_64-linux-gnu/bitcoin-859644b3c855-x86_64-linux-gnu-debug.tar.gz
  93d603d89c9251e93992e3c0e176b094d981b020ef1d4bbfbff21806e50b962d  guix-build-859644b3c855/output/x86_64-linux-gnu/bitcoin-859644b3c855-x86_64-linux-gnu.tar.gz
  79df55f4ca8f972665fd55b43550b2ab9ffafa8fc8de1335b324f4af047c6788  guix-build-859644b3c855/output/x86_64-w64-mingw32/SHA256SUMS.part
  8d837e96595ab7337736da334d940ffc9d15215141f176804a528fe9e21f490a  guix-build-859644b3c855/output/x86_64-w64-mingw32/bitcoin-859644b3c855-win64-debug.zip
  6c9541524f1d54eceb3265c6e79d62502fdc0c2e5263719a0ca357988d7ed718  guix-build-859644b3c855/output/x86_64-w64-mingw32/bitcoin-859644b3c855-win64-setup-unsigned.exe
  7566ab4ee53092e81c3079db955d85c8d574cbde2be21526d45619076ffcd264  guix-build-859644b3c855/output/x86_64-w64-mingw32/bitcoin-859644b3c855-win64-unsigned.tar.gz
  32164cfa7c06ead63305485653f37d74c6ada82d28b79f58e66faf6e72e130bb  guix-build-859644b3c855/output/x86_64-w64-mingw32/bitcoin-859644b3c855-win64.zip
  ```

ACKs for top commit:
  jarolrod:
    re-ACK 859644b
  hebasto:
    ACK 859644b3c8, I've verified introduced changes in compiler flags, including the case with `DEBUG=1`.

Tree-SHA512: 6e181ced7e474a80aa191663b08dc594179a0593b8e2d1e4b7c8683794fd7de8d37faedb9a36997645ce6a2a6151e1461678b4db95170fc9b1fcadd6e1bddbe5
2022-10-13 13:13:17 +08:00
fanquake
aa6fb37acc Merge bitcoin/bitcoin#26205: wallet: #25768 follow ups
b01682a812 refactor: revert m_next_resend to not be std::atomic (stickies-v)
9245f45670 wallet: only update m_next_resend when actually resending (stickies-v)
7fbde8af5c refactor: carve out tx resend timer logic into ShouldResend (stickies-v)
01f3534632 refactor: remove unused locks for ResubmitWalletTransactions (stickies-v)
c6e8e11fb0 wallet: fix capitalization in docstring (stickies-v)

Pull request description:

  This PR addresses the outstanding comments/issues from #25768:

  - capitalization [typo](https://github.com/bitcoin/bitcoin/pull/25768#discussion_r958572522) in docstring
  - remove [unused locks](01f3534632) that we previously needed for `ReacceptWalletTransactions()`
  - before #25768, only `ResendWalletTransactions()` would reset `m_next_resend` (formerly called `nNextResend`). By unifying it with `ReacceptWalletTransactions()` into `ResubmitWalletTransactions()`, the number of callsites that would reset the `m_next_resend` timer increased
    - since `m_next_resend` is only used in case of `relay=true` (formerly `ResendWalletTransactions()`), this is unintuitive
    - it leads to [unexpected behaviour](https://github.com/bitcoin/bitcoin/pull/25768#issuecomment-1252619427) such as transactions potentially never being rebroadcasted.
    - it makes the ResubmitWalletTransactions()` logic [more complicated than strictly necessary](https://github.com/bitcoin/bitcoin/pull/25768#discussion_r962828563)
    - since #25768, we relied on an earlier call of `ResubmitWalletTransactions(relay=false, force=true)` to initialize `m_next_resend()`, I think we can more elegantly do that by just providing `m_next_resend` with a default value
    - just to highlight: this commit introduces behaviour change

  Note: the `if (!fBroadcastTransactions)` in `CWallet:ShouldResend()` is duplicated on purpose, since it potentially avoids the slightly more expensive `if (!chain().isReadyToBroadcast())` check afterwards. I don't have a strong view on it, so happy to remove that additional check to reduce the diff, too.

ACKs for top commit:
  aureleoules:
    ACK b01682a812
  achow101:
    ACK b01682a812

Tree-SHA512: ac5f1d8858f8dd736dd1480f385984d660c1916b62a42562317020e8f9fd6a30bd8f23d973d47e4c9480d744c5ba39fdbefd69568a5eb0589a8422d7e5971c1c
2022-10-13 12:09:44 +08:00
fanquake
7e5fe03461 Merge bitcoin/bitcoin#24467: doc: minor improvements in getutxos REST endpoint synopsis
c456302d42 doc: minor improvements in getutxos REST endpoint synopsis (Sebastian Falbesoner)

Pull request description:

  Describing an optional sub-path as `<checkmempool>` in the synopsis could be misleading as the angle brackets normally indicate that the field has to be replaced a custom value. Clarify that by showing two variants instead, similar to the `block` endpoint with the `notxdetails` option:
  ```
  #### Blocks
  `GET /rest/block/<BLOCK-HASH>.<bin|hex|json>`
  `GET /rest/block/notxdetails/<BLOCK-HASH>.<bin|hex|json>`
  ```

  Further improvements:
  - uppercase `<TXID>` and `<N>`, to match the description of the other endpoints
  - s/getutxo command/getutxos endpoint/
  - describe what the `checkmempool` option does
  - s/serialisation/serialization/ (the US spelling is more dominant than the UK spelling in the project, and there is indeed no other instance of the string "serialis*" in the source tree, except once in a release note)
  - link to BIP64 within the text instead of only showing bare URL
  - mention that BIP64 is only relevant for `bin` and `hex` output formats
  - show two endpoint formats of the block section as list

ACKs for top commit:
  stickies-v:
    ACK c456302d42 - also checked that current master (cc12b8947) doesn't have any other lines changes that would require updates as per the outlined improvement points.

Tree-SHA512: b025aac0812397f5fbf78c805c13aeb5afa6862a049d13c0b101178799cdaff1ccd3abc368a5c103ea6ebf17cdff76584c54638d0f8d303d81ade2d71443d305
2022-10-13 11:54:52 +08:00
fanquake
422efcad36 Merge bitcoin/bitcoin#26188: test: silence TSAN false positive in coinstatsindex_initial_sync
861cb3fadc test: move SyncWithValidationInterfaceQueue() before Stop() in txindex_tests (Vasil Dimov)
6526dc3b78 test: silence TSAN false positive in coinstatsindex_initial_sync (Vasil Dimov)

Pull request description:

  Silence false positives from TSAN about unsynchronized calls to `BaseIndex::~BaseIndex()` and `BaseIndex::SetBestBlockIndex()`. They are synchronized, but beyond the comprehension of TSAN - by `SyncWithValidationInterfaceQueue()`, called from `BaseIndex::BlockUntilSyncedToCurrentChain()`.

  Fixes https://github.com/bitcoin/bitcoin/issues/25365

ACKs for top commit:
  MarcoFalke:
    review ACK 861cb3fadc
  ryanofsky:
    Code review ACK 861cb3fadc. Just comment change since last review.

Tree-SHA512: 8c30fdf2fd11d54e9adfa68a67185ab820bd7bd9f7f3ad6456e7e6d219fa9cf6d34b41e98e723eae86cb0c1baef7f3fc57b1b011a13dc3fe3d78334b9b5596de
2022-10-13 11:42:27 +08:00
glozow
7e1007a3c6 Merge bitcoin/bitcoin#25421: net: convert standalone IsSelectableSocket() and SetSocketNonBlocking() to Sock methods
b527b54950 net: convert standalone SetSocketNonBlocking() to Sock::SetNonBlocking() (Vasil Dimov)
29f66f7682 moveonly: move SetSocketNonBlocking() from netbase to util/sock (Vasil Dimov)
b4bac55679 net: convert standalone IsSelectableSocket() to Sock::IsSelectable() (Vasil Dimov)
5db7d2ca0a moveonly: move IsSelectableSocket() from compat.h to sock.{h,cpp} (Vasil Dimov)

Pull request description:

  _This is a piece of #21878, chopped off to ease review._

  * convert standalone `IsSelectableSocket()` to `Sock::IsSelectable()`
  * convert standalone `SetSocketNonBlocking()` to `Sock::SetNonBlocking()`

  This further encapsulates syscalls inside the `Sock` class and makes the callers mockable.

ACKs for top commit:
  jonatack:
    ACK b527b54950 review/debug build/unit tests at each commit, cross-referenced the changes with `man select` and `man errno`, ran a signet node on the last commit with ip4/ip6//tor/i2p/cjdns and network connections were nominal
  dergoegge:
    Code review ACK b527b54950

Tree-SHA512: af783ce558c7a89e173f7ab323fb3517103d765c19b5d14de29f64706b4e1fea3653492e8ea73ae972699986aaddf2ae72c7cfaa7dad7614254283083b7d2632
2022-10-12 15:49:02 -04:00
glozow
cc12b8947b Merge bitcoin/bitcoin#24858: incorrect blk file size calculation during reindex results in recoverable blk file corruption
bcb0cacac2 reindex, log, test: fixes #21379 (mruddy)

Pull request description:

  Fixes #21379.

  The blocks/blk?????.dat files are mutated and become increasingly malformed, or corrupt, as a result of running the re-indexing process.
  The mutations occur after the re-indexing process has finished, as new blocks are appended, but are a result of a re-indexing process miscalculation that lingers in the block manager's `m_blockfile_info` `nSize` data until node restart.
  These additions to the blk files are non-fatal, but also not desirable.
  That is, this is a form of data corruption that the reading code is lenient enough to process (it skips the extra bytes), but it adds some scary looking log messages as it encounters them.

  The summary of the problem is that the re-index process double counts the size of the serialization header (magic message start bytes [4 bytes] + length [4 bytes] = 8 bytes) while calculating the blk data file size (both values already account for the serialization header's size, hence why it is over accounted).

  This bug manifests itself in a few different ways, after re-indexing, when a new block from a peer is processed:
  1. If the new block will not fit into the last blk file processed while re-indexing, while remaining under the 128MiB limit, then the blk file is flushed to disk and truncated to a size that is 8 greater than it should be. The truncation adds zero bytes (see `FlatFileSeq::Flush` and `TruncateFile`).
  1. If the last blk file processed while re-indexing has logical space for the new block under the 128 MiB limit:
      1. If the blk file was not already large enough to hold the new block, then the zeros are, in effect, added by `fseek` when the file is opened for writing. Eight zero bytes are added to the end of the last blk file just before the new block is written. This happens because the write offset is 8 too great due to the miscalculation. The result is 8 zero bytes between the end of the last block and the beginning of the next block's magic + length + block.
      1. If the blk file was already large enough to hold the new block, then the current existing file contents remain in the 8 byte gap between the end of the last block and the beginning of the next block's magic + length + block. Commonly, when this occcurs, it is due to the blk file containing blocks that are not connected to the block tree during reindex and are thus left behind by the reindex process and later overwritten when new blocks are added. The orphaned blocks can be valid blocks, but due to the nature of concurrent block download, the parent may not have been retrieved and written by the time the node was previously shutdown.

ACKs for top commit:
  LarryRuane:
    tested code-review ACK bcb0cacac2
  ryanofsky:
    Code review ACK bcb0cacac2. This is a disturbing bug with an easy fix which seems well-worth merging.
  mzumsande:
    ACK bcb0cacac2 (reviewed code and did some testing, I agree that it fixes the bug).
  w0xlt:
    tACK bcb0cacac2

Tree-SHA512: acc97927ea712916506772550451136b0f1e5404e92df24cc05e405bb09eb6fe7c3011af3dd34a7723c3db17fda657ae85fa314387e43833791e9169c0febe51
2022-10-12 14:13:54 -04:00
Hennadii Stepanov
98c389ccff ci: Use Multi-ToolTask in "Win64 native" task
See https://devblogs.microsoft.com/cppblog/improved-parallelism-in-msbuild/
2022-10-12 11:50:57 +01:00
fanquake
1d277f4223 Merge bitcoin/bitcoin#26280: rpc: Return coinbase flag in scantxoutset
fa08663344 rpc: Return coinbase flag in scantxoutset (MacroFake)

Pull request description:

  I guess it can't hurt to return this for someone that wants to know it

ACKs for top commit:
  aureleoules:
    ACK fa08663344
  shaavan:
    ACK fa08663344

Tree-SHA512: 04c554b3ed9877bab93ffcf0c1a4430cd41b30c5f4f3bf462a518fc8b3d68832dd85a29e81bd805eaa16e987856933d7a888a8c126f670bb2844bbd5ca1bf902
2022-10-12 10:28:32 +08:00
fanquake
5fc3939850 Merge bitcoin/bitcoin#22087: Validate port-options
04526787b5 Validate `port` options (amadeuszpawlik)
f8387c4234 Validate port value in `SplitHostPort` (amadeuszpawlik)

Pull request description:

  Validate `port`-options, so that invalid values are rejected early in the startup.
  Ports are `uint16_t`s, which effectively limits a port's value to <=65535. As discussed in https://github.com/bitcoin/bitcoin/pull/24116 and https://github.com/bitcoin/bitcoin/pull/24344, port "0" is considered invalid too.
  Proposed in https://github.com/bitcoin/bitcoin/issues/21893#issuecomment-835784223

  The `SplitHostPort(std::string in, uint16_t& portOut, std::string& hostOut)` now returns a bool that indicates whether the port value was set and within the allowed range. This is an improvement that can be used not only for port validation of options at startup, but also in rpc calls, etc,

ACKs for top commit:
  luke-jr:
    utACK 04526787b5
  ryanofsky:
    Code review ACK 04526787b5. Just suggested changes since last review: reverting some SplitHostPort changes, adding release notes, avoiding 'GetArgs[0]` problem.

Tree-SHA512: f1ac80bf98520b287a6413ceadb41bc3a93c491955de9b9319ee1298ac0ab982751905762a287e748997ead6198a8bb7a3bc8817ac9e3d2468e11ab4a0f8496d
2022-10-12 08:59:18 +08:00
Hennadii Stepanov
6fbd173d8a ci: Use all available CPUs for functional tests in "Win64 native" task 2022-10-11 22:10:19 +01:00
Anthony Towns
7082ce3e88 scripted-diff: rename and de-globalise g_cs_orphans
-BEGIN VERIFY SCRIPT-
sed -i -e 's/static RecursiveMutex/mutable Mutex/' src/txorphanage.h
sed -i -e '/RecursiveMutex/d' src/txorphanage.cpp
sed -i -e 's/g_cs_orphans/m_mutex/g' $(git grep -l g_cs_orphans src/)
-END VERIFY SCRIPT-
2022-10-11 23:35:37 +10:00
Anthony Towns
733d85f79c Move all g_cs_orphans locking to txorphanage 2022-10-11 23:35:32 +10:00
Vasil Dimov
861cb3fadc test: move SyncWithValidationInterfaceQueue() before Stop() in txindex_tests
So that the call order is the same as in coinstatsindex_tests.
2022-10-11 09:45:55 +02:00
Vasil Dimov
6526dc3b78 test: silence TSAN false positive in coinstatsindex_initial_sync
Fixes https://github.com/bitcoin/bitcoin/issues/25365
2022-10-11 09:45:54 +02:00
Anthony Towns
a936f41a5d txorphanage: make m_peer_work_set private 2022-10-11 14:05:09 +10:00
Anthony Towns
3614819864 txorphange: move orphan workset to txorphanage 2022-10-11 14:04:49 +10:00
fanquake
2e77dff744 Merge bitcoin/bitcoin#25676: sync: simplify and remove unused code from sync.h
75c3f9f880 sync: rename AnnotatedMixin::UniqueLock to AnnotatedMixin::unique_lock (Vasil Dimov)
8d9ee8efe8 sync: remove DebugLock alias template (Vasil Dimov)
4b2e16763f sync: avoid confusing name overlap (Mutex) (Vasil Dimov)
9d7ae4b66c sync: remove unused template parameter from ::UniqueLock (Vasil Dimov)
11c190e3f1 sync: simplify MaybeCheckNotHeld() definitions by using a template (Vasil Dimov)

Pull request description:

  Summary:

  * Reduce 4 of the `MaybeCheckNotHeld()` definitions to 2 by using a template.
  * Remove unused template parameter from `::UniqueLock`.
  * Use `MutexType` instead of `Mutex` for a template parameter name to avoid overlap/confusion with the `Mutex` class.
  * Rename `AnnotatedMixin::UniqueLock` to `AnnotatedMixin::unique_lock` to avoid overlap/confusion with the global `UniqueLock` and for consistency with `UniqueLock::reverse_lock`.

  The first commit `sync: simplify MaybeCheckNotHeld() definitions by using a template` is also part of https://github.com/bitcoin/bitcoin/pull/25390

ACKs for top commit:
  aureleoules:
    ACK 75c3f9f880 - LGTM
  ryanofsky:
    Code review ACK 75c3f9f880. Nice cleanups! Just suggested changes since last review: keeping UniqueLock name and fixing a missed rename in a code comment

Tree-SHA512: ec261f6a444bdfe4f06e844b57b3606fdd9b2f842647cae15266d9729970d87585c808d482fbba0b31c33a4aa03527c36e282c92b28d9052711f75a7048c96f1
2022-10-11 09:18:55 +08:00
Leonardo Araujo
2dede9f675 Adjust RPCTypeCheckObj error string 2022-10-10 18:08:00 -03:00
Ben Woosley
6630a1e844 Add warning on first startup if free disk space is less than necessary
To accommodate the expected blocks data.

Co-authored-by: Antoine Poinsot <darosior@protonmail.com>
Co-authored-by: benthecarman <benthecarman@live.com>
Co-authored-by: Justin Litchfield <litch@me.com>
Co-authored-by: Liran Cohen <c.liran.c@gmail.com>
Co-authored-by: Ryan Loomba <ryan.loomba@gmail.com>
Co-authored-by: Buck Perley <bucko.perley@gmail.com>
Co-authored-by: bajjer <bajjer@bajjer.xyz>
Co-authored-by: Suhail Saqan <suhail.saqan@gmail.com>
Co-authored-by: Christopher Sweeney <sweeney.chris@gmail.com>
Co-authored-by: Alyssa <orbitalturtle@protonmail.com>
Co-authored-by: Ben Schroth <ben@styng.social>
Co-authored-by: Jason Hester <mail@jason-hester.me>
Co-authored-by: Matt Clough <Matt.clough@pm.me>
Co-authored-by: Elise Schedler <eliseschedler@gmail.com>
Co-authored-by: ghander <cen254@gmail.com>
Co-authored-by: PopeLaz <btclz@fastmail.com>
Co-authored-by: Aurèle Oulès <hello@aureleoules.com>
2022-10-10 16:58:14 -04:00
MacroFake
9ca39d69df Merge bitcoin/bitcoin#26254: iwyu: Add zmq source files
13afcc0cd4 iwyu: Add zmq source files (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK 13afcc0cd4

Tree-SHA512: 7af95e991fc2782aeba2edfef0a2f75f9c361058295586adb062087aa31c47cfcce2425aee9dd5153e18e018cf1f9272c9617c671b7262db55f241526c3fcb15
2022-10-10 18:08:45 +02:00
Aurèle Oulès
5d3f98d278 refactor: Replace m_params with chainman.GetParams()
Fixes a TODO introduced in #24595.
2022-10-10 17:43:45 +02:00
Hennadii Stepanov
13afcc0cd4 iwyu: Add zmq source files 2022-10-10 15:44:02 +01:00
fanquake
57c192767b Merge bitcoin/bitcoin#25322: build: Fix capnp package build for Android
8b8edc25c1 build: Specify native binaries explicitly when building `capnp` package (Hennadii Stepanov)
a413595c37 build: Fix `capnp` package build for Android (Hennadii Stepanov)

Pull request description:

  On master (e3c08eb620):
  ```
  $ make -C depends capnp MULTIPROCESS=1 HOST=aarch64-linux-android ANDROID_SDK=$ANDROID_HOME ANDROID_NDK=$ANDROID_HOME/ndk/23.2.8568313 ANDROID_API_LEVEL=28 ANDROID_TOOLCHAIN_BIN=$ANDROID_HOME/ndk/23.2.8568313/toolchains/llvm/prebuilt/linux-x86_64/bin
  ...
  ld: error: unable to find library -lkj
  ...
  ```

  This PR fixes this error, and also improves configuring according to the docs.

ACKs for top commit:
  ryanofsky:
    Code review ACK 8b8edc25c1. I'd be a little curious to know what causes the error and how `--disable-shared` fixes it, but these changes all look good

Tree-SHA512: 1b07b75f2a83932d8dc1f007e42a67d8327bd5fe4566f554dab4599e2a1e04b0144648790a1fd2ab1c295dba728586035aa0ebdbe5cf49df048ec87736895aaf
2022-10-10 21:04:32 +08:00
MacroFake
239757409b Merge bitcoin/bitcoin#26118: log: Use steady clock for bench logging
fabf1cdb20 Use steady clock for bench logging (MacroFake)
faed342a23 scripted-diff: Rename time symbols (MacroFake)

Pull request description:

  Instead of using `0.001` and similar constants to "convert" an int64_t to milliseconds, use the type-safe `Ticks<>` helper. Also, use steady clock instead of system clock, since the durations are used for benchmarking.

ACKs for top commit:
  fanquake:
    ACK fabf1cdb20 - validation bench output still looks sane.

Tree-SHA512: e6525b5fdad6045ca500c56014897d7428ad288aaf375933d3b5939feddf257f6910d562eb66ebcde9186bef9a604ee8d763a318253838318d59df2a285be7c2
2022-10-10 12:00:34 +02:00
fanquake
866dd664a1 Merge bitcoin/bitcoin#26196: kernel: move RunCommandParseJSON to its own file
43b8777dc3 refactor: move run_command from util to common (Cory Fields)
192325a77d kernel: move RunCommandParseJSON to its own file (Cory Fields)

Pull request description:

  Because libbitcoinkernel does not include this new object, this has the side-effect of eliminating its unnecessary `boost::process` dependency.

  This leaves libbitcoinkernel with 3 remaining boost dependencies:
  - `boost::date_time` for `util/time.cpp`, which I'll separate out next. Exactly like this PR.
  - `boost::signals2` for which I have a POC re-implementation here: https://github.com/theuni/bitcoin/commits/replace-boost-signals
  - `boost::multi_index` which I'm not sure about yet.

ACKs for top commit:
  ryanofsky:
    Code review ACK 43b8777dc3. Could consider squashing the two commits, so the code just moves once instead of twice.
  fanquake:
    ACK 43b8777dc3

Tree-SHA512: f2a46cac34aaadfb8a1442316152ad354f6990021b82c78d80cae9fd43cd026209ffd62132eaa99d5d0f8cf34e996b6737d318a9d9a3f1d2ff8d17d697abf26d
2022-10-10 17:58:18 +08:00
fanquake
869342f7fa Merge bitcoin/bitcoin#26282: wallet: have prune error take precedence over assumedvalid
1c36bafc5f wallet: have prune error take precedence over assumedvalid (James O'Beirne)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/pull/23997#discussion_r891412739.

  From Russ Yanofsky:

  > Agree with all of Marco's points here and think this should be updated
  >
  > If havePrune and hasAssumedValidChain are both true, better to show havePrune error message.  Assumed-valid error message is vague and not very actionable.  Would suggest "Error loading wallet. Wallet requires blocks to be downloaded, and software does not currently support loading wallets while blocks are being downloaded out of order though assumeutxo snapshots. Wallet should be able to load successfully after node sync reaches height {block_height}"

ACKs for top commit:
  MarcoFalke:
    ACK 1c36bafc5f
  aureleoules:
    ACK 1c36bafc5f

Tree-SHA512: bfb0024bb962525cbbd392ade3c0331a8b0525e7f2f2ab52b2dbb9b6dd6311070d85ecb762a7689db84a30991971865698ab6fec187206e6a92133790c5a91dc
2022-10-10 17:04:43 +08:00
fanquake
9eaa5dbc81 Merge bitcoin/bitcoin#25073: test: Cleanup miner_tests
faa15527d7 test: Use dedicated mempool in TestBasicMining (MacroFake)
fafab384a0 test: Use dedicated mempool in TestPackageSelection (MacroFake)
fa4055d79c test: Use dedicated mempool in TestPrioritisedMining (MacroFake)
fa29218285 test: Pass mempool reference to AssemblerForTest (MacroFake)

Pull request description:

  This cleans up the miner tests:

  * Removes duplicate/redundant and thus confusing chainparams object.
  * Uses a fresh mempool for each subtest instead of using the "global" one from the testing setup. This makes it easier to follow the tests in smaller scopes. Also it makes sure the mempool is truly cleared by reconstructing it. Finally, this removes calls to `clear`, see https://github.com/bitcoin/bitcoin/pull/19909

ACKs for top commit:
  glozow:
    utACK faa15527d7

Tree-SHA512: ced1260f6ab70fba74b0fac7ff4fc7adfddcd2f3bee785249d2a4a9055ac253eff9090edbda7a17e72a71a81b56ff708d5ff64e1f57ebc7b7747d6c88fec51e3
2022-10-10 16:45:39 +08:00
MacroFake
9f44f2e32f Merge bitcoin/bitcoin#26284: Fix comment typos
adb1714426 Fix comment typos in scriptpubkeyman.cpp, wallet.cpp, wallet.h (Dimitris Tsapakidis)

Pull request description:

  Fixes a number of comment typos found in the code.

Top commit has no ACKs.

Tree-SHA512: c2c996b66d33ecf0ee734b76303a0f2444e184d2f3ff6931768712ca51011ad51e54336c33a2ff55133766d20ae6adcbb14ddc754dde58b1fe9167d68f54fec5
2022-10-10 09:32:51 +02:00
fanquake
515c9c4c2e Merge bitcoin/bitcoin#26183: build: split ARM crc & crypto extension checks
20adaeaef5 build: split ARM crc & crypto extension checks (fanquake)

Pull request description:

  We currently perform the same check twice, to put the same set of flags in two different variables. Split the checks so we test for the `crc` and `crypto` extensions independently.

  If we don't want to split, we should just delete the second `AX_CHECK_COMPILE_FLAG` check, and set `ARM_CRC_CXXFLAGS` & `ARM_SHANI_CXXFLAGS` at the same time.

  Guix Build:
  ```bash
  045392a6a4f538723b7759c67eeafd832735de7294b72b3a7f488d05a13711f7  guix-build-20adaeaef5fa/output/aarch64-linux-gnu/SHA256SUMS.part
  054fda86577d757788a1c87508268402535fcbe869240309a2c91997234389cf  guix-build-20adaeaef5fa/output/aarch64-linux-gnu/bitcoin-20adaeaef5fa-aarch64-linux-gnu-debug.tar.gz
  92dc2513b2b6d87c0869ae18493fd9d0e2690b5b02bfd4310d54f4d394cfccdf  guix-build-20adaeaef5fa/output/aarch64-linux-gnu/bitcoin-20adaeaef5fa-aarch64-linux-gnu.tar.gz
  2515cfc708cc6ce0e650ca00c49de8dad856b54741ddc0c195845fc6ce2d67db  guix-build-20adaeaef5fa/output/arm-linux-gnueabihf/SHA256SUMS.part
  fa0a956365e62b484f66dcf9763a02858db5c7e99317861819a54a15589ced80  guix-build-20adaeaef5fa/output/arm-linux-gnueabihf/bitcoin-20adaeaef5fa-arm-linux-gnueabihf-debug.tar.gz
  1b3ddf2b1bbdc7632696ca78908e69b4fd156ccf7afa8078b5541d2ac10ab931  guix-build-20adaeaef5fa/output/arm-linux-gnueabihf/bitcoin-20adaeaef5fa-arm-linux-gnueabihf.tar.gz
  f87d8e23df60b208a631f6642f6c2cc0fc8e4e5e9563b36d1de9d371f22a69d9  guix-build-20adaeaef5fa/output/arm64-apple-darwin/SHA256SUMS.part
  c24ac07bfa935fd40358823d95ef01128a03b80deec6b2cb8bed122994e8adc2  guix-build-20adaeaef5fa/output/arm64-apple-darwin/bitcoin-20adaeaef5fa-arm64-apple-darwin-unsigned.dmg
  696660e030accadc27901dfb4e120aa2fefefa8cc2a33ae887e3c98e5d4795f5  guix-build-20adaeaef5fa/output/arm64-apple-darwin/bitcoin-20adaeaef5fa-arm64-apple-darwin-unsigned.tar.gz
  30dcd3f543781ac0e07e36336c2901a25a0829e0d1425c25b3c7aba1d0e5420e  guix-build-20adaeaef5fa/output/arm64-apple-darwin/bitcoin-20adaeaef5fa-arm64-apple-darwin.tar.gz
  4d63db45f28fcb99aa8f3b30cf06afef80dd308a8d2fdf874752accb3f341258  guix-build-20adaeaef5fa/output/dist-archive/bitcoin-20adaeaef5fa.tar.gz
  eb208b98b3118e9f8240aab91c7ecb2f9b778109bc19d81d0ba73b3e35aa1123  guix-build-20adaeaef5fa/output/powerpc64-linux-gnu/SHA256SUMS.part
  8b0de7008b1932ed18d3ab71ca309dc4919096e226e0a7197bd192e1ba96da82  guix-build-20adaeaef5fa/output/powerpc64-linux-gnu/bitcoin-20adaeaef5fa-powerpc64-linux-gnu-debug.tar.gz
  bcbc269cc4b5883397c516ef3ef6df564f4a81c240d5afcf912a2bf9554ff148  guix-build-20adaeaef5fa/output/powerpc64-linux-gnu/bitcoin-20adaeaef5fa-powerpc64-linux-gnu.tar.gz
  e5f7fd823056449a495a68d18fe941b472479bc59d9d4d11a041a4e2cc4044ec  guix-build-20adaeaef5fa/output/powerpc64le-linux-gnu/SHA256SUMS.part
  73ee7e786372b32ab840f0c00ca0479ddd022b3d37219cd929cb49f744c174e3  guix-build-20adaeaef5fa/output/powerpc64le-linux-gnu/bitcoin-20adaeaef5fa-powerpc64le-linux-gnu-debug.tar.gz
  08f64c9aae4d9beef88d8fbae8ad0152517de74bedc88540775c4f757c8b6b9a  guix-build-20adaeaef5fa/output/powerpc64le-linux-gnu/bitcoin-20adaeaef5fa-powerpc64le-linux-gnu.tar.gz
  fe3c28fdb1ee9d5b6ca3ba4510d61c052567edb3b93fdde929ed197072c0fd66  guix-build-20adaeaef5fa/output/riscv64-linux-gnu/SHA256SUMS.part
  890d6b96edcc431620eede6239dec51368aff917010e03dabeb29d6a672d7a28  guix-build-20adaeaef5fa/output/riscv64-linux-gnu/bitcoin-20adaeaef5fa-riscv64-linux-gnu-debug.tar.gz
  df1fc0c9af4799cfe170444e21965f2a600aa193fdd0da542fedceeb3081b194  guix-build-20adaeaef5fa/output/riscv64-linux-gnu/bitcoin-20adaeaef5fa-riscv64-linux-gnu.tar.gz
  f69cae0b2d0eadb336cf314a888b1e0bed241f38954fe58ca9c9c2d00e49b74e  guix-build-20adaeaef5fa/output/x86_64-apple-darwin/SHA256SUMS.part
  acc5fa9725bba738d10bb8b1e7df2d8a7b0e648015e1c046f67451d343f68224  guix-build-20adaeaef5fa/output/x86_64-apple-darwin/bitcoin-20adaeaef5fa-x86_64-apple-darwin-unsigned.dmg
  7e4d8cb6d74434ba9084f487187c49cd5a4138c9ae03a6c2236cdffadb236bc8  guix-build-20adaeaef5fa/output/x86_64-apple-darwin/bitcoin-20adaeaef5fa-x86_64-apple-darwin-unsigned.tar.gz
  8d93add28b20dfb2a556d3867cfbf218db336d7eefee6ab6f76a1bb4dd4ae20b  guix-build-20adaeaef5fa/output/x86_64-apple-darwin/bitcoin-20adaeaef5fa-x86_64-apple-darwin.tar.gz
  ba0863eda963db706d2880daa8bc526e6332097010fa7227f513a2d715b6cd6c  guix-build-20adaeaef5fa/output/x86_64-linux-gnu/SHA256SUMS.part
  6915794f3cdc8ad9b305b6baa58f89f7493097b88c0af190d52d93457a17e8d8  guix-build-20adaeaef5fa/output/x86_64-linux-gnu/bitcoin-20adaeaef5fa-x86_64-linux-gnu-debug.tar.gz
  467b05298058ec507c3b247c423f3ea7e027ecf62e45d7ae4b81160118bc0d02  guix-build-20adaeaef5fa/output/x86_64-linux-gnu/bitcoin-20adaeaef5fa-x86_64-linux-gnu.tar.gz
  51a534803deaabcbba27d82359ef46e4d5b9e7b121ab71e1975c2a0d1c4c6f45  guix-build-20adaeaef5fa/output/x86_64-w64-mingw32/SHA256SUMS.part
  c9c5496f20bac01ed6439746aff9ca3dd55708718902c898e99f3d5741b167a3  guix-build-20adaeaef5fa/output/x86_64-w64-mingw32/bitcoin-20adaeaef5fa-win64-debug.zip
  cfaac54be36789927f83172c0af44c50648f63df7cdc9d81774a170e5ab6e3e5  guix-build-20adaeaef5fa/output/x86_64-w64-mingw32/bitcoin-20adaeaef5fa-win64-setup-unsigned.exe
  759b79660c291dcc7da88088de3bb666162fed5c9d94bb24f10cef6e781c565f  guix-build-20adaeaef5fa/output/x86_64-w64-mingw32/bitcoin-20adaeaef5fa-win64-unsigned.tar.gz
  f0124333d384ff6962e2131c7b2814bf5c968e77b63ff1b2c7d19cb4c571757c  guix-build-20adaeaef5fa/output/x86_64-w64-mingw32/bitcoin-20adaeaef5fa-win64.zip
  ```

ACKs for top commit:
  jarolrod:
    ACK 20adaeaef5

Tree-SHA512: 8b515b95ba4d41ca2ce91448339841dcfb80feb028e9e3bc67a72e72d93669e1257534c11286489a60ae240f6ad6e68f56615818fefd1c09a07a1bee4976fa6e
2022-10-10 15:30:51 +08:00
fanquake
857f07dfd0 Merge bitcoin/bitcoin#26277: test: Remove confusing DUMMY_P2WPKH_SCRIPT
fa8a305ddd test: Remove confusing DUMMY_P2WPKH_SCRIPT (MacroFake)

Pull request description:

  It is confusing because, it is *not* a P2WPKH script, and it is nonstandard.

  See also https://github.com/bitcoin/bitcoin/pull/26265/files#r989827855

  Fix all issues by removing it, and also remove the no longer needed `-acceptnonstdtxn` setting from the test.

ACKs for top commit:
  instagibbs:
    ACK fa8a305ddd
  theStack:
    Code-review ACK fa8a305ddd 📜

Tree-SHA512: 64f3e0009b055e4fd4428b20f3e85582e1608e9b06e500b8fbfeb91fc35ce510e69d051e8f48ce35d0320067793e12f4423b214cc1f68c217a5872e0ad97d211
2022-10-10 15:28:48 +08:00
Vasil Dimov
75c3f9f880 sync: rename AnnotatedMixin::UniqueLock to AnnotatedMixin::unique_lock
This avoids confusion with the global `UniqueLock` and the snake case
is consistent with `UniqueLock::reverse_lock.
2022-10-10 09:20:59 +02:00
Vasil Dimov
8d9ee8efe8 sync: remove DebugLock alias template
Use `UniqueLock` directly. Type deduction works just fine from the first
argument to the constructor of `UniqueLock`, so there is no need to
repeat

```cpp
UniqueLock<typename std::remove_reference<typename std::remove_pointer<decltype(cs)>::type>::type>
```

five times in the `LOCK` macros. Just `UniqueLock` suffices.
2022-10-10 09:20:58 +02:00
Vasil Dimov
4b2e16763f sync: avoid confusing name overlap (Mutex)
Use `MutexType` instead of `Mutex` for the template parameter of
`UniqueLock` because there is already a class named `Mutex` and the
naming overlap is confusing. `MutexType` is used elsewhere in `sync.h`.
2022-10-10 09:20:45 +02:00
fanquake
4175c332b9 Merge bitcoin/bitcoin#26215: index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability
8891949bdc index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability (Ryan Ofsky)

Pull request description:

  Since commit f08c9fb0c6 from PR https://github.com/bitcoin/bitcoin/pull/21726, index  `BlockUntilSyncedToCurrentChain` behavior has been less reliable, and there has also been a race condition in the `coinstatsindex_initial_sync` unit test.

  It seems better for `BlockUntilSyncedToCurrentChain` to actually wait for the last connected block to be fully processed, than to be able to return before prune locks are set, so this switches the order of `m_best_block_index = block;` and `UpdatePruneLock` statements in `SetBestBlockIndex` to make it more reliable.

  Also since commit f08c9fb0c6, there has been a race condition in the `coinstatsindex_initial_sync` test. Before that commit, the atomic index best block pointer `m_best_block_index` was updated as the last step of `BaseIndex::BlockConnected`, so `BlockUntilSyncedToCurrentChain` could safely be used in tests to wait for the last `BlockConnected` notification to be finished before stopping and destroying the index.  But after that commit, calling `BlockUntilSyncedToCurrentChain` is no longer sufficient, and there is a race between the test shutdown code which destroys the index object and the new code introduced in that commit calling `AllowPrune()` and `GetName()` on the index object. Reproducibility instructions for this are in https://github.com/bitcoin/bitcoin/issues/25365#issuecomment-1259744133

  This commit fixes the `coinstatsindex_initial_sync` race condition, even though it will require an additional change to silence TSAN false positives, https://github.com/bitcoin/bitcoin/pull/26188, after it is fixed. So this partially addresses but does not resolve the bug reporting TSAN errors https://github.com/bitcoin/bitcoin/issues/25365.

  There is no known race condition outside of test code currently, because the bitcoind `Shutdown` function calls `FlushBackgroundCallbacks` not `BlockUntilSyncedToCurrentChain` to safely shut down.

  Co-authored-by: vasild
  Co-authored-by: MarcoFalke

ACKs for top commit:
  mzumsande:
    re-ACK 8891949bdc

Tree-SHA512: 52e29e3772a0c92873c54e5ffb31dd66a909b68a2031b7585713cd1d976811289c98bd9bb41679a8689062f03be4f97bb8368696e789caa4607c2fd8b1fe289b
2022-10-10 14:23:00 +08:00
fanquake
cf3db7c256 Merge bitcoin/bitcoin#26258: refactor: Remove unused CDataStream::rdbuf method
fabbbe32ee Remove unused CDataStream::rdbuf method (MacroFake)

Pull request description:

  It is unused and seems unlikely to be ever used.

ACKs for top commit:
  theStack:
    Code-review ACK fabbbe32ee
  aureleoules:
    ACK fabbbe32ee

Tree-SHA512: 5804642658f96a0fb51482ebf3a062bb0f997c1e0527455afa4aceeeb6c1ad139a98b14a7c8a0909daba733a83bdc24fcadad45060ead4be6eb3dc3e66c129e2
2022-10-10 14:14:15 +08:00
glozow
d33c5894e9 Merge bitcoin/bitcoin#26103: refactor: mempool: use CTxMemPool::Limits
33b12e5df6 docs: improve docs where MemPoolLimits is used (stickies-v)
6945853c0b test: use NoLimits() in MempoolIndexingTest (stickies-v)
3a86f24a4c refactor: mempool: use CTxMempool::Limits (stickies-v)
b85af25f87 refactor: mempool: add MemPoolLimits::NoLimits() (stickies-v)

Pull request description:

  Mempool currently considers 4 limits regarding ancestor and descendant count and size, which get passed around between functions quite a bit. This PR uses `CTxMemPool::Limits` introduced in https://github.com/bitcoin/bitcoin/pull/25290 to simplify those signatures and callsites.

  The purpose of this PR is to improve readability and maintenance, without behaviour change.

  As noted in the first commit "refactor: mempool: change MemPoolLimits members to uint", we currently have an underflow issue where a user could pass a negative `-limitancestorsize`, which is eventually cast to an unsigned integer. This behaviour already exists. Because it's orthogonal and to minimize scope, I think this should be fixed in a separate PR.

ACKs for top commit:
  hebasto:
    ACK 33b12e5df6, I have reviewed the code and it looks OK, I agree it can be merged.
  glozow:
    reACK 33b12e5df6

Tree-SHA512: 591c6dcee1894f1c3ca28b34a680eeadcf0d40cda92451b4a422c03087b27d682b5e30ba4367abd75a99b5ccb115b7884b0026958d3c7dddab030549db5a4056
2022-10-09 10:28:32 -04:00
Dimitris Tsapakidis
adb1714426 Fix comment typos in scriptpubkeyman.cpp, wallet.cpp, wallet.h
Fix comment typos:
sigature -> signature
ponter -> pointer
it's key -> its key
2022-10-09 17:19:59 +03:00
glozow
ec8016eba7 Merge bitcoin/bitcoin#26281: docs: fix m_children to be a member of CTxMemPoolEntry
01bf4af4f2 docs: fix m_children to be a member of CTxMemPoolEntry (stickies-v)

Pull request description:

  Small documentation fix to reflect that `m_children` [is a member](73b61717a9/src/txmempool.h (L99)) of `CTxMemPoolEntry`, not `CTxMemPool`

ACKs for top commit:
  hebasto:
    ACK 01bf4af4f2, wrong wording was introduced in bitcoin/bitcoin#19478.
  glozow:
    ACK 01bf4af4f2

Tree-SHA512: b66c43b92fda44682b1f67c43073ca9e133a6dc03cd28253e571e67170531138c20b22ffdb08f312fb2d47a1f869b876611646b54325c8b614d12049befad578
2022-10-09 10:17:02 -04:00
James O'Beirne
1c36bafc5f wallet: have prune error take precedence over assumedvalid
From Russ Yanofsky:

"Agree with all of Marco's points here and think this should be updated

If havePrune and hasAssumedValidChain are both true, better to show
havePrune error message.  Assumed-valid error message is vague and not
very actionable.  Would suggest "Error loading wallet. Wallet requires
blocks to be downloaded, and software does not currently support loading
wallets while blocks are being downloaded out of order though assumeutxo
snapshots. Wallet should be able to load successfully after node sync
reaches height {block_height}"

Co-authored-by: MacroFake <MarcoFalke@gmail.com>
Co-authored-by: Russ Yanofsky <russ@yanofsky.org>
2022-10-07 15:19:31 -04:00
stickies-v
01bf4af4f2 docs: fix m_children to be a member of CTxMemPoolEntry 2022-10-07 15:06:45 +01:00
MacroFake
fa08663344 rpc: Return coinbase flag in scantxoutset 2022-10-07 15:04:28 +02:00
MacroFake
fa8a305ddd test: Remove confusing DUMMY_P2WPKH_SCRIPT 2022-10-07 13:11:05 +02:00
Anthony Towns
6f8e442ba6 net_processing: Localise orphan_work_set handling to ProcessOrphanTx 2022-10-07 14:41:24 +10:00
Anthony Towns
0027174b39 net_processing: move ProcessOrphanTx docs to declaration 2022-10-07 14:40:50 +10:00
Anthony Towns
9910ed755c net_processing: Pass a Peer& to ProcessOrphanTx 2022-10-07 14:40:26 +10:00
Anthony Towns
89e2e0da0b net_processing: move extra transactions to msgproc mutex
Previously vExtraTxnForCompact and vExtraTxnForCompactIt were protected
by g_cs_orphans; protect them by g_msgproc_mutex instead, as they
are only used during message processing.
2022-10-07 14:40:03 +10:00
Anthony Towns
ff8d44d196 Remove unnecessary includes of txorphange.h 2022-10-07 14:39:40 +10:00
josibake
deba6fe315 test: update feature_config_args.py
add two new test cases for conf and reindex
2022-10-06 18:20:30 -04:00
josibake
2e3826cbcd util: warn if reindex is used in conf
using reindex in a conf file can lead to the node reindexing on every restart.
we still allow it but throw a warning.
2022-10-06 18:20:30 -04:00
josibake
5e744f4238 util: disallow setting conf in bitcoin.conf
Help from `bitcoind -h` states that conf can only be used from the commandline.
However, if conf is set in a bitcoin.conf file, it is ignored but there is no error.

Show an error to user if conf is set in a .conf file and prompt them to use
`includeconf` if they wish to specify additional config files.

Adds `IsConfSupported` function to allow for easily adding conf options
to disallow or throw warnings for.
2022-10-06 18:20:14 -04:00
furszy
e43a547a36 refactor: wallet, do not translate init arguments names 2022-10-06 18:00:08 -03:00
Andrew Chow
9e386afb67 tests: Test that PSBT_OUT_TAP_TREE is included correctly 2022-10-06 15:32:51 -04:00
Andrew Chow
30ff25cf37 psbt: Only include m_tap_tree if it has scripts 2022-10-06 15:32:51 -04:00
Andrew Chow
0577d423ad psbt: Change m_tap_tree to store just the tuples
Instead of having an entire TaprootBuilder which may or may not be
complete, and could potentially have future changes that interact oddly
with taproot tree tuples, have m_tap_tree be just the tuples.

When needed in other a TaprootBuilder for actual use, the tuples will be
added to a a TaprootBuilder that, in the future, can take in whatever
other data is needed as well.
2022-10-06 15:32:51 -04:00
Andrew Chow
22c051ca70 tests: Test that PSBT_OUT_TAP_TREE is combined correctly 2022-10-06 15:32:45 -04:00
Andrew Chow
7df6e1bb77 psbt: Fix merging of m_tap_tree
Merging should be checking that the current PSBTOutput doesn't have a
taptree and the other one's is copied over. The original merging had
this inverted and would remove m_tap_tree if the other did not have it.
2022-10-06 15:19:30 -04:00
MacroFake
73b61717a9 Merge bitcoin/bitcoin#26272: test: Prevent UB in minisketch_tests.cpp
97007e2b9b test: Prevent UB in `minisketch_tests.cpp` (Hennadii Stepanov)

Pull request description:

  [`std::optional::operator*`](https://en.cppreference.com/w/cpp/utility/optional/operator*), which follows after the changed line, can cause UB.

  This PR addresses https://github.com/bitcoin/bitcoin/issues/26262#issuecomment-1268855418

ACKs for top commit:
  stickies-v:
    ACK 97007e2b9b

Tree-SHA512: a7dde8dac0cbdfa362fa1158b4564eccff9405852612227d581690c9a34084b3467ae6d4c0269262688d75339dcea90aaa38fccbba9be92d2643c2113860f3d6
2022-10-06 16:01:17 +02:00
Hennadii Stepanov
97007e2b9b test: Prevent UB in minisketch_tests.cpp 2022-10-06 12:50:54 +01:00
glozow
292f652d53 Merge bitcoin/bitcoin#24364: refactor: remove duplicate code from BlockAssembler
0f40d65321 refactor: remove duplicate code from BlockAssembler (James O'Beirne)

Pull request description:

  Found while reminding myself how transactions are chosen for blocks. Take it or leave it!

ACKs for top commit:
  glozow:
    ACK 0f40d65321
  theStack:
    Concept and code-review ACK 0f40d65321

Tree-SHA512: 8a2694e670ce3fe897ab8f64f64c8df5f8487fc1264527a3abbcba0e5b921fb693416497ccd62508295bc33f202c65556b91b6af463acb91aab43138d2492c14
2022-10-06 12:50:33 +01:00
muxator
9153ff3e27 rpc: add non-regression test about deriveaddresses crash when index is 2147483647
This test would cause a crash in bitcoind (see #26274) if the fix given in the
previous commit was not applied.
2022-10-06 12:03:36 +02:00
muxator
addf9d6502 rpc: fix crash in deriveaddresses when derivation index is 2147483647
2147483647 is the maximum positive value of a signed int32, and - currently -
the maximum value that the deriveaddresses bitcoin RPC call accepts as
derivation index due to its input validation routines.

Before this change, when the derivation index (and thus range_end) reached
std::numeric_limits<int_32_t>::max(), the "i" variable in the for cycle (which
is declared as int, and as such 32 bits in size on most platforms) would be
incremented at the end of the first iteration and then warp back to
-2147483648. This caused SIGABRT in bitcoind and a core dump.

This change assigns "i" an explicit size of 64 bits on every platform,
sidestepping the problem.

Fixes #26274.
2022-10-06 22:17:49 +02:00
amadeuszpawlik
04526787b5 Validate port options
Check `port` options for invalid values (ports are parsed as uint16, so
in practice values >65535 are invalid; port 0 is undefined and therefore
considered invalid too). This allows for an early rejection of faulty
values and an supplying an informative message to the user.

Splits tests in `feature_proxy.py` to cover both invalid `hostname`
and `port` values.

Adds a release-note as previously valid `-port` and `-rpcport` values
can now result in errors.
2022-10-05 19:30:15 +02:00
amadeuszpawlik
f8387c4234 Validate port value in SplitHostPort
Forward the validation of the port from `ParseUInt16(...)`.
Consider port 0 as invalid.
Add suitable test for the `SplitHostPort` function.
Add doxygen description to the `SplitHostPort` function.
2022-10-05 19:24:04 +02:00
Ryan Ofsky
8891949bdc index: Improve BaseIndex::BlockUntilSyncedToCurrentChain reliability
Since commit f08c9fb0c6 from PR
https://github.com/bitcoin/bitcoin/pull/21726, index
`BlockUntilSyncedToCurrentChain` behavior has been less reliable, and there has
also been a race condition in the `coinstatsindex_initial_sync` unit test.

It seems better for `BlockUntilSyncedToCurrentChain` to actually wait for the
last connected block to be fully processed, than to be able to return before
prune locks are set, so this switches the order of `m_best_block_index =
block;` and `UpdatePruneLock` statements in `SetBestBlockIndex` to make it more
reliable.

Also since commit f08c9fb0c6, there has been a
race condition in the `coinstatsindex_initial_sync` test. Before that commit,
the atomic index best block pointer `m_best_block_index` was updated as the
last step of `BaseIndex::BlockConnected`, so `BlockUntilSyncedToCurrentChain`
could safely be used in tests to wait for the last `BlockConnected`
notification to be finished before stopping and destroying the index. But
after that commit, calling `BlockUntilSyncedToCurrentChain` is no longer
sufficient, and there is a race between the test shutdown code which destroys
the index object and the new code introduced in that commit calling
`AllowPrune()` and `GetName()` on the index object. Reproducibility
instructions for this are in
https://github.com/bitcoin/bitcoin/issues/25365#issuecomment-1259744133

This commit fixes the `coinstatsindex_initial_sync` race condition, even though
it will require an additional change to silence TSAN false positives,
https://github.com/bitcoin/bitcoin/pull/26188, after it is fixed. So this
partially addresses but does not resolve the bug reporting TSAN errors
https://github.com/bitcoin/bitcoin/issues/25365.

There is no known race condition outside of test code currently, because the
bitcoind `Shutdown` function calls `FlushBackgroundCallbacks` not
`BlockUntilSyncedToCurrentChain` to safely shut down.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
Co-authored-by: MacroFake <falke.marco@gmail.com>
2022-10-05 11:06:58 -04:00
MacroFake
5e82b9ba96 Merge bitcoin/bitcoin#26252: refactor: Make 64-bit shift explicit
5c5b85d0e7 refactor: Make 64-bit shift explicit (Hennadii Stepanov)

Pull request description:

  [`std::array::at()`](https://en.cppreference.com/w/cpp/container/array/at) expects an argument of the `size_t` type. This PR avoids implicit type conversion (for both 64-bit and 32-bit systems).

  Also it enables MSVC warning [C4334](https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4334) for all codebase.

ACKs for top commit:
  MarcoFalke:
    ACK 5c5b85d0e7 🚎
  jonatack:
    Code review ACK 5c5b85d0e7

Tree-SHA512: fda850a42068f2ada9f877fac9ff8af1e22b5dcb3e708f5b95c316e77c52c72d33cd9ec6507a7f5d1731d1afdf5af6dc65025d388cc480f82c46f4d88ef2d306
2022-10-05 15:46:23 +02:00
MacroFake
fabbbe32ee Remove unused CDataStream::rdbuf method
It is unused and seems unlikely to be ever used.
2022-10-05 15:29:36 +02:00
Andrew Toth
1ff5d61dfd doc: add mempool/contents rest verbose and mempool_sequence args 2022-10-05 09:25:07 -04:00
Andrew Toth
52a31dccc9 tests: mempool/contents verbose and mempool_sequence query params tests 2022-10-05 09:25:07 -04:00
Andrew Toth
a518fff0f2 rest: add verbose and mempool_sequence query params for mempool/contents 2022-10-05 09:25:07 -04:00
Jon Atack
c2a21c0670 gui: use fallback value for Version and User Agent during peer connection
During connection setup for a peer, getpeerinfo returns "version": 0, "subver": ""
and the GUI Peers window displays 0 and an empty field, respectively.

Give these fields the same behavior as the other fields in the GUI Peers window:
display the fallback value in src/qt/forms/debugwindow.ui (i.e. "N/A") until a
valid result is available after the peer connection completes.
2022-10-05 15:19:01 +02:00
stickies-v
33b12e5df6 docs: improve docs where MemPoolLimits is used 2022-10-05 13:09:08 +01:00
stickies-v
6945853c0b test: use NoLimits() in MempoolIndexingTest
The (100, 1000000, 1000, 1000000) limits are arbitrarily high and
don't restrict anything, they are just meant to calculate ancestors
properly. Using NoLimits() makes this intent more clear and simplifies
the code.
2022-10-05 13:07:11 +01:00
stickies-v
3a86f24a4c refactor: mempool: use CTxMempool::Limits
Simplifies function signatures by removing repetition of all the
ancestor/descendant limits,  and increases readability by being
more verbose by naming the limits, while still reducing the LoC.
2022-10-05 13:07:11 +01:00
stickies-v
b85af25f87 refactor: mempool: add MemPoolLimits::NoLimits()
There are quite a few places in the codebase that require us to
construct a CTxMemPool without limits on ancestors and descendants.
This helper function allows us to get rid of all that duplication.
2022-10-05 13:07:11 +01:00
MacroFake
faa15527d7 test: Use dedicated mempool in TestBasicMining
No need for a shared mempool. Also remove unused chainparams parameter.

Can be reviewed with --ignore-all-space
2022-10-05 13:36:57 +02:00
MacroFake
fafab384a0 test: Use dedicated mempool in TestPackageSelection
No need for a shared mempool. Also remove unused chainparams parameter.
2022-10-05 13:36:56 +02:00
MacroFake
fa4055d79c test: Use dedicated mempool in TestPrioritisedMining
No need for a shared mempool. Also remove unused chainparams parameter.
2022-10-05 13:35:18 +02:00
MacroFake
fa29218285 test: Pass mempool reference to AssemblerForTest 2022-10-05 13:34:36 +02:00
MacroFake
b2e6d37b51 Merge bitcoin/bitcoin#26256: ci: Remove clang-format from lint task
fa04376554 Remove clang-format from lint task (MacroFake)

Pull request description:

  clang-format could be used in scripted diffs, but remained largely unused.

  So remove the install bloat, as it is unlikely to be used in the future.

ACKs for top commit:
  fanquake:
    ACK fa04376554
  hebasto:
    ACK fa04376554

Tree-SHA512: e0a3ee47d2aa2565dd34676914c558c985eaeb522a05f10bcaac115871edcf0d7f101b517e4d452ca5223c40b18ad02883c31e2da3d1f4ff86464a9af0097b11
2022-10-05 11:52:46 +02:00
MacroFake
fa04376554 Remove clang-format from lint task
clang-format could be used in scripted diffs, but remained largely
unused.
2022-10-05 10:52:42 +02:00
MacroFake
d3cdd37d92 Merge bitcoin/bitcoin#26250: fuzz: add mempool_utils.cpp
8a6b6dfcd8 fuzz: pass max fee into ConsumeTxMemPoolEntry (fanquake)
eb15569280 fuzz: add util/mempool/h.cpp (fanquake)

Pull request description:

  Moving the heavy (Boost) mempool code out of fuzz/util.h. Means that (for ex) a crypto_common fuzz unit doesn't need to care about seeing endless Boost headers. This results in a ~10% speedup (for me) when compiling the fuzz tests. Your results may vary.

ACKs for top commit:
  MarcoFalke:
    review ACK 8a6b6dfcd8 🍮

Tree-SHA512: 27dc9d9581ac0b1b319cc0dc08fe5f8fbf9269386a5cb23f6fd5d8231bf015ed942ab4414d8001220541be0013756354578ddab1fec607c6fba04daf421bc870
2022-10-05 10:29:00 +02:00
Cory Fields
43b8777dc3 refactor: move run_command from util to common
Quoting ryanofsky: "util can be the library for things included in the kernel
which the kernel can depend on, and common can be the library for other code
that needs to be shared internally, but should not be part of the kernel or
shared externally."
2022-10-04 21:21:05 +00:00
Hennadii Stepanov
5c5b85d0e7 refactor: Make 64-bit shift explicit
Also this change enables MSVC warning C4334 for all codebase.

https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/compiler-warning-level-3-c4334
2022-10-04 21:49:07 +01:00
fanquake
a23f8c8978 Merge bitcoin/bitcoin#26234: ci: Allow PIP_PACKAGES on centos
fa6054e952 ci: Allow PIP_PACKAGES on centos (MacroFake)
fac085a05c ci: Remove unused package (MacroFake)

Pull request description:

  This was added in 7fc5e865b9 but I can't see a reason why this should be forbidden.

  This is also needed for other changes (bumping the minimum python version).

ACKs for top commit:
  hebasto:
    re-ACK fa6054e952

Tree-SHA512: e8ead9ee00079024eb1e8c6e7b31c78cf2a3392159b444765c2ea9a58bed2a7550bf71083210692a45bb8ed7896cb882b72bf70baa13a2384864b2b510b73005
2022-10-04 21:39:29 +01:00
fanquake
b6565b65aa Merge bitcoin/bitcoin#26243: test: Remove unused fCheckpointsEnabled from miner_tests
fa9436e908 test: Remove unused fCheckpointsEnabled from miner_tests (MacroFake)

Pull request description:

  The earliest checkpoint is at height 11111, so this can't possibly have any impact on this test.

ACKs for top commit:
  fanquake:
    ACK fa9436e908 - given the low number of blocks, having the additional check in `ContextualCheckBlockHeader()` enabled should be a no-op, so disabling and re-enabling is dead code.

Tree-SHA512: 7d1b952c297c915e9588761f82f5006cf5186b7ff30e8a1c702302e0b44afe536bde9eda8acf2995825ae01d2ad9d2393ae2feefb29f15676aaf71881941579b
2022-10-04 21:17:07 +01:00
fanquake
8a6b6dfcd8 fuzz: pass max fee into ConsumeTxMemPoolEntry 2022-10-04 21:12:50 +01:00
fanquake
eb15569280 fuzz: add util/mempool/h.cpp
Moving the mempool code (Boost) out of util.h, results in a ~10% speedup
(for me) when compiling the fuzz tests.
2022-10-04 21:12:50 +01:00
fanquake
d919e8d574 Merge bitcoin/bitcoin#26244: build, msvc: Enable C4834 warning
f3e40c481a build, msvc: Enable C4834 warning (Hennadii Stepanov)

Pull request description:

  Since bitcoin/bitcoin#26189 our codebase is  C4834 warning free.

  See https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/c4834.

ACKs for top commit:
  sipsorcery:
    tACK f3e40c481a.

Tree-SHA512: 2dd8fdb5d6c0d4eaf7ffec82039411963010ecea559adfe7e2b5587f467135bd665e0cef5c748e4d6d9ba3c2038d6a5c79980972884adbaaac3bda3b2fbbc408
2022-10-04 21:09:58 +01:00
James O'Beirne
626b7c8493 fuzz: add scanblocks as safe for fuzzing 2022-10-04 13:51:35 -04:00
Jonas Schnelli
94fe5453c7 test: rpc: add scanblocks functional test
Co-authored-by: James O'Beirne <james.obeirne@gmail.com>
2022-10-04 13:51:35 -04:00
Jonas Schnelli
6ef2566b68 rpc: add scanblocks - scan for relevant blocks with descriptors
Co-authored-by: James O'Beirne <james.obeirne@gmail.com>
2022-10-04 13:51:35 -04:00
James O'Beirne
a4258f6e81 rpc: move-only: consolidate blockchain scan args
For later reuse in `scanblocks`.
2022-10-04 13:51:33 -04:00
MacroFake
2d8d9aecac Merge bitcoin/bitcoin#26249: ci: Workaround Windows filesystem executable bit loss
b8d361ab6f ci: Workaround Windows filesystem executable bit loss (Hennadii Stepanov)

Pull request description:

  Fixes https://cirrus-ci.com/task/5946581265416192:
  ```
  From https://github.com/bitcoin/bitcoin
   * branch                refs/pull/26103/merge -> FETCH_HEAD
  error: Your local changes to the following files would be overwritten by checkout:
  ci/lint/04_install.sh
  ci/lint/06_script.sh
  contrib/devtools/gen-manpages.py
  contrib/devtools/symbol-check.py
  contrib/signet/getcoins.py
  contrib/signet/miner
  test/functional/feature_proxy.py
  test/functional/feature_taproot.py
  test/functional/interface_rest.py
  test/functional/mining_prioritisetransaction.py
  test/functional/rpc_blockchain.py
  test/functional/rpc_fundrawtransaction.py
  test/functional/rpc_help.py
  test/functional/rpc_rawtransaction.py
  test/functional/test_framework/test_framework.py
  test/functional/test_runner.py
  test/functional/wallet_basic.py
  test/functional/wallet_bumpfee.py
  test/functional/wallet_hd.py
  test/functional/wallet_importmulti.py
  test/functional/wallet_listdescriptors.py
  test/functional/wallet_multiwallet.py
  test/functional/wallet_resendwallettransactions.py
  test/functional/wallet_sendall.py
  test/lint/lint-locale-dependence.py
  Please commit your changes or stash them before you switch branches.
  Aborting
  ```

Top commit has no ACKs.

Tree-SHA512: 2b33d882a515bb17c7c2ae8cfe73541483cdc15e736909afaf42befc8f648dba5dc83ff58ebd6d38a5650a8eca01907ae6c61537927ac9718bd9582d8501647d
2022-10-04 16:40:08 +02:00
Hennadii Stepanov
b8d361ab6f ci: Workaround Windows filesystem executable bit loss 2022-10-04 15:20:32 +01:00
MacroFake
dddd1acf58 net: Set relay in version msg to peers with relay permission 2022-10-04 16:07:00 +02:00
Cory Fields
192325a77d kernel: move RunCommandParseJSON to its own file
Because libbitcoinkernel does not include this new object, this has the
side-effect of eliminating the unnecessary boost::process dependency.
2022-10-04 13:51:40 +00:00
fanquake
4fa50dd055 Merge bitcoin/bitcoin#26236: ci: Use same merge_script implementation for Windows as for all
37cf472063 ci: Use same `merge_script` implementation for Windows as for all (Hennadii Stepanov)
ac1d99240a ci: Move `git config` commands into script where they are used (Hennadii Stepanov)

Pull request description:

  This PR is a continuation of bitcoin/bitcoin#26202 and it suggests the same approach for the "Win64 native" CI task.

Top commit has no ACKs.

Tree-SHA512: 3154c9f30bc62549d738dc337e24f66614420417c349770c8381cc29f825f75695c9abbbb8dc57abbfda1375bf353f7c68e1a3766fd6d2440792e2d7fb68e15e
2022-10-04 13:45:40 +01:00
fanquake
92e9fc6d0d Merge bitcoin/bitcoin#26128: doc: add missing historical release notes
cb075d245e doc: add historical 0.21.2 release notes (fanquake)
699f3429c6 doc: add historical 0.20.2 release notes (fanquake)

Pull request description:

  0.20.2 and 0.21.2 are missing from master.

ACKs for top commit:
  jarolrod:
    ACK cb075d245e

Tree-SHA512: f05fb2e5b589cd60581e724182c6f32f992a85e6dc41f5a91d5c6941869ff4c0a7f28a405b3dfc71d9660c1385a1a13d220aa50b2d69c7787cb7974a3e4bf814
2022-10-04 12:51:50 +01:00
Hennadii Stepanov
f3e40c481a build, msvc: Enable C4834 warning
https://docs.microsoft.com/en-us/cpp/error-messages/compiler-warnings/c4834
2022-10-04 12:03:26 +01:00
MacroFake
fa9436e908 test: Remove unused fCheckpointsEnabled from miner_tests
The earliest checkpoint is at height 11111, so this can't possibly have
any impact on this test.
2022-10-04 12:40:19 +02:00
MacroFake
914c00074b Merge bitcoin/bitcoin#26235: refactor: move *index constants out of validation
7d14577d0f refactor: move DEFAULT_BLOCKFILTERINDEX from val to blockfilterindex (fanquake)
c87d569189 refactor: move DEFAULT_COINSTATSINDEX from validation to coinstatsindex (fanquake)
2bfc1e6aaa refactor: move DEFAULT_TXINDEX from validation to txindex (fanquake)

Pull request description:

  Move `*index` default constants out of `validation.h`.

ACKs for top commit:
  stickies-v:
    re-ACK 7d14577d0f
  aureleoules:
    ACK 7d14577d0f

Tree-SHA512: 3021db1a63ceb714dee4b91f755d1fb9a6633adb6f1081e34e4179900e7543e3a7b06fe47507d580a3a2caf52f7ede784cb36716d521c76b0404bdc798f0186a
2022-10-04 12:36:13 +02:00
MacroFake
fa6054e952 ci: Allow PIP_PACKAGES on centos
This was added in 7fc5e865b9 but I can't
see a reason why this should be forbidden.
2022-10-04 11:53:31 +02:00
MacroFake
fac085a05c ci: Remove unused package
Address feedback from https://github.com/bitcoin/bitcoin/pull/24561/files#r985719812
2022-10-04 11:51:33 +02:00
MacroFake
f65a2c6f57 Merge bitcoin/bitcoin#26237: kernel: remove util/bytevectorhash.cpp
4bee62e9b8 kernel: remove util/bytevectorhash.cpp (fanquake)

Pull request description:

  This is no-longer used.

ACKs for top commit:
  hebasto:
    ACK 4bee62e9b8, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 4d61f87b640ef3c759008631433b3e6d2bd2ac54bbe0b287f32ea1569760048f17a66cfe846b94ec458a7db5d064be6da59299b9280572a3dc649df60760c63f
2022-10-04 11:34:23 +02:00
fanquake
44a29758a0 Merge bitcoin/bitcoin#26209: Update leveldb subtree
1a463c70a3 Squashed 'src/leveldb/' changes from 22f1e4a02f..e2f10b4e47 (fanquake)

Pull request description:

  Pulls in https://github.com/bitcoin-core/leveldb-subtree/pull/34: win32: fix -Wmissing-field-initializers warnings
  Related to #26090, #25972.

  Guix Build:
  ```bash
  a8c49e700309e9268ad0a503e7813775f58daf37813501775ae5831eab0cf26a  guix-build-13601da17e7a/output/aarch64-linux-gnu/SHA256SUMS.part
  3a1722403f8daf8ea152c72004fb1021f8a7241ee7f51fb0e6adeeb50881f001  guix-build-13601da17e7a/output/aarch64-linux-gnu/bitcoin-13601da17e7a-aarch64-linux-gnu-debug.tar.gz
  3b7c1f2361ef4832c305ff4b0d9eb7419f787f8cf537952d7c20be549debe0af  guix-build-13601da17e7a/output/aarch64-linux-gnu/bitcoin-13601da17e7a-aarch64-linux-gnu.tar.gz
  329cb5bc932621785a6aeca43af307279c47d87a4983ee3d59636f8e3dc27afa  guix-build-13601da17e7a/output/arm-linux-gnueabihf/SHA256SUMS.part
  4030f344c4c31cc0877c11d3b89f6ea93f930a4ae7e7976d49ad5123cc31a367  guix-build-13601da17e7a/output/arm-linux-gnueabihf/bitcoin-13601da17e7a-arm-linux-gnueabihf-debug.tar.gz
  ab50945c416b256b4c92ea0a50c0b7ec241e4c1f5803a4e7cb4ba83a6c5db888  guix-build-13601da17e7a/output/arm-linux-gnueabihf/bitcoin-13601da17e7a-arm-linux-gnueabihf.tar.gz
  604557cc73c4065697255c6971547a46cd0d6ad1cdf74261d4c73d002abe8c46  guix-build-13601da17e7a/output/arm64-apple-darwin/SHA256SUMS.part
  11e781c64fab7933df4a9b4f56357ad18dd8f149b3c9195bf2d310b58355851d  guix-build-13601da17e7a/output/arm64-apple-darwin/bitcoin-13601da17e7a-arm64-apple-darwin-unsigned.dmg
  e3fe9e278310f3dc97f463bb8cf2b930b587dba5095e18a7f38c4ef76338daad  guix-build-13601da17e7a/output/arm64-apple-darwin/bitcoin-13601da17e7a-arm64-apple-darwin-unsigned.tar.gz
  00828a9bfd74013a439b7f0254bd72904b99ff1e3a9503ef6307ee57de81c5eb  guix-build-13601da17e7a/output/arm64-apple-darwin/bitcoin-13601da17e7a-arm64-apple-darwin.tar.gz
  d80032d2509a982dc81790b3ef3cd8d44551b41047082ebc506f01bbb8830de1  guix-build-13601da17e7a/output/dist-archive/bitcoin-13601da17e7a.tar.gz
  61655cdf7c081c43275da57e5b42c21e33d7864378f3784652969ffdf7c3c2b2  guix-build-13601da17e7a/output/powerpc64-linux-gnu/SHA256SUMS.part
  3d213e2f25ca72dc0c3d372c0ad3f6803865825a233b29dd80b5cff3e3963621  guix-build-13601da17e7a/output/powerpc64-linux-gnu/bitcoin-13601da17e7a-powerpc64-linux-gnu-debug.tar.gz
  ef8cad747428b6b46c5d2c598fb7896b320a5321cf9a0c9dba2209c638bfbc7b  guix-build-13601da17e7a/output/powerpc64-linux-gnu/bitcoin-13601da17e7a-powerpc64-linux-gnu.tar.gz
  2302e022394ca46570c70bc5977f35bbed258bfbb19c2c6d80f1982a725c9e35  guix-build-13601da17e7a/output/powerpc64le-linux-gnu/SHA256SUMS.part
  5ca3104b7d546c7d905f82dfd92b5350af754e3973b533adc50c6cbde8f59f96  guix-build-13601da17e7a/output/powerpc64le-linux-gnu/bitcoin-13601da17e7a-powerpc64le-linux-gnu-debug.tar.gz
  2781824d8d08f8ac6663925db8b2d1b50b80165e67ed69df86580438284f0c3b  guix-build-13601da17e7a/output/powerpc64le-linux-gnu/bitcoin-13601da17e7a-powerpc64le-linux-gnu.tar.gz
  24a73f91c5ab9e72f2471cd7b31482c702a3de8dade9aff6b76af34253748cec  guix-build-13601da17e7a/output/riscv64-linux-gnu/SHA256SUMS.part
  b09658af4bd24eb5e76bacbfc7f8750a8941f117511b5c55a44ef1beffec8367  guix-build-13601da17e7a/output/riscv64-linux-gnu/bitcoin-13601da17e7a-riscv64-linux-gnu-debug.tar.gz
  56bde3f3df115dbf801944e4f923a08752fd05cda0bbf0e0e3c915387f98e4a8  guix-build-13601da17e7a/output/riscv64-linux-gnu/bitcoin-13601da17e7a-riscv64-linux-gnu.tar.gz
  ca9a5acb332fb2ffda42e44d84b58e273a769e94f0abaaa0623ed1d1c5267dde  guix-build-13601da17e7a/output/x86_64-apple-darwin/SHA256SUMS.part
  ccae805bca5d4075d1bccbaeca7ac40bb4dac0a97f4fd4c3c80b2d6134d6acb0  guix-build-13601da17e7a/output/x86_64-apple-darwin/bitcoin-13601da17e7a-x86_64-apple-darwin-unsigned.dmg
  b12c9682b69b3662150abd6680bb60f9fef656a1fdc3cd21951c2da571be40cc  guix-build-13601da17e7a/output/x86_64-apple-darwin/bitcoin-13601da17e7a-x86_64-apple-darwin-unsigned.tar.gz
  fd3378e4d60f0ef3ae1b18f526e96ed9e838df4def021de332dd0b6cd20ed0c4  guix-build-13601da17e7a/output/x86_64-apple-darwin/bitcoin-13601da17e7a-x86_64-apple-darwin.tar.gz
  47c7af42153850a94d70de6489389f6faf8772d6adc88269705a12e6b894e84e  guix-build-13601da17e7a/output/x86_64-linux-gnu/SHA256SUMS.part
  876898d116041680ea6fa3bbce1becb05e45ea5bdbaf973a1465fde4bd2df26d  guix-build-13601da17e7a/output/x86_64-linux-gnu/bitcoin-13601da17e7a-x86_64-linux-gnu-debug.tar.gz
  56671e22f25ba695c7315de916c6a2d1b57d5f00982b331c68a43912ac3ebc44  guix-build-13601da17e7a/output/x86_64-linux-gnu/bitcoin-13601da17e7a-x86_64-linux-gnu.tar.gz
  f5b9b7d83ad34b7b6c3cb98b9d7878795a1f6fb7b422ffb02b23266beb76d3a6  guix-build-13601da17e7a/output/x86_64-w64-mingw32/SHA256SUMS.part
  a72cfb4ce41cad3bb24df2e4bb318edd05370e12fc31f9ba21637b3fd412dccd  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64-debug.zip
  4b175eac7355715884be279dd4595665912b62815f1d0186ca9874cb9e6424cb  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64-setup-unsigned.exe
  c45d927f0adedc3eb0a01cee0f9fa734c17d9e9a6692b0f416a3830c007b52cc  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64-unsigned.tar.gz
  360a5b8806c72dbfcfed616d5cec78606461d664e21fb28a44bd473a2b7c9326  guix-build-13601da17e7a/output/x86_64-w64-mingw32/bitcoin-13601da17e7a-win64.zip
  ```

ACKs for top commit:
  hebasto:
    ACK 13601da17e, I have reviewed the code and it looks OK, I agree it can be merged.
  theuni:
    ACK 13601da17e

Tree-SHA512: 1450618714a456d8969fa5bfc3ed700452e0737213af50656a0a1e0764e6063390fb528eb1889d8bf1e02e451f601f0a5bc63a02ac34ef10aeb7dc80fe41acd1
2022-10-04 10:20:26 +01:00
glozow
cda6c79190 Merge bitcoin/bitcoin#26203: wallet: Use correct effective value when checking target
d0d9cf7aea test: Check external coin effective value is used in CoinSelection (Aurèle Oulès)
76b79c1a17 wallet: Use correct effective value when checking target (Aurèle Oulès)

Pull request description:

  Fixes #26185. The following assert failed because it was not checked in the parent function.

  2bd9aa5a44/src/wallet/coinselection.cpp (L391)

ACKs for top commit:
  glozow:
    reACK d0d9cf7aea
  furszy:
    ACK d0d9cf7a

Tree-SHA512: e126daba1115e9d143f2a582c6953e7ea55e96853b6e819c7744fd7a23668f7d9854681d43ef55d8774655bc54e7e87c1c9fccd746d9e30fbf3caa82ef808ae9
2022-10-04 09:57:17 +01:00
Hennadii Stepanov
37cf472063 ci: Use same merge_script implementation for Windows as for all 2022-10-04 08:51:55 +01:00
Hennadii Stepanov
ac1d99240a ci: Move git config commands into script where they are used 2022-10-04 08:51:55 +01:00
w0xlt
17cad44851 test: refactor RPCPackagesTest to use MiniWallet 2022-10-03 22:37:40 -03:00
fanquake
4bee62e9b8 kernel: remove util/bytevectorhash.cpp 2022-10-03 18:49:07 +01:00
fanquake
1730f6cb23 Merge bitcoin/bitcoin#26189: refactor: Do not discard try_lock() return value
30cc1c6609 refactor: Drop `owns_lock()` call (Hennadii Stepanov)
bff4e068b6 refactor: Do not discard `try_lock()` return value (Hennadii Stepanov)

Pull request description:

  Microsoft's C++ Standard Library uses the `[[nodiscard]]` attribute for `try_lock()`.
  See: https://github.com/microsoft/STL/blob/main/stl/inc/mutex

  This change allows to drop the current suppression for the warning C4838 and helps to prevent the upcoming warning C4858.
  See: 539c26c923

  Fixes bitcoin/bitcoin#26017.

  Split from bitcoin/bitcoin#25819.

ACKs for top commit:
  vasild:
    ACK 30cc1c6609

Tree-SHA512: ce17404e1c78af4f763129753caf8e5a0e1c91ba398778fe912f9fcc56a847e8112460d1a1a35bf905a593b7d8e0b16c6b099ad74976b67dca5f4f3eda6ff621
2022-10-03 18:21:35 +01:00
fanquake
7d14577d0f refactor: move DEFAULT_BLOCKFILTERINDEX from val to blockfilterindex 2022-10-03 18:19:40 +01:00
fanquake
c87d569189 refactor: move DEFAULT_COINSTATSINDEX from validation to coinstatsindex 2022-10-03 18:19:39 +01:00
fanquake
2bfc1e6aaa refactor: move DEFAULT_TXINDEX from validation to txindex 2022-10-03 18:19:39 +01:00
fanquake
b92b12e8f3 Merge bitcoin/bitcoin#25735: net: remove useless call to IsReachable() from CConnman::Bind()
9cbfe40d8a net: remove useless call to IsReachable() from CConnman::Bind() (Vasil Dimov)

Pull request description:

  `CConnman::Bind()` is called without `BF_EXPLICIT` only when passed
  either `0.0.0.0` or `::`. For those addresses `IsReachable()` is always
  true (regardless of the `-onlynet=` setting!), meaning that the `if`
  condition never evaluates to true.

  `IsReachable()` is always true for the "any" IPv4 and IPv6 addresses
  because `CNetAddr::GetNetwork()` returns `NET_UNROUTABLE` instead of
  `NET_IPV4` or `NET_IPV6` and the network `NET_UNROUTABLE` is always
  considered reachable.

  It follows that `BF_EXPLICIT` is unnecessary, remove it too.

ACKs for top commit:
  naumenkogs:
    ACK 9cbfe40d8a
  aureleoules:
    ACK 9cbfe40d8a
  mzumsande:
    ACK 9cbfe40d8a

Tree-SHA512: 4e53ee8a73ddd133fd4ff25635135b65e5c19d1fc56fe5c30337406560664616c0adff414dca47602948919f34c81073aae6bfc2871509f3912663d86750928e
2022-10-03 18:16:10 +01:00
MacroFake
3baa0f5a60 Merge bitcoin/bitcoin#26230: ci: Run bench_bitcoin.exe --sanity-check in "Win64 native" task
91bee4d898 ci: Run `bench_bitcoin.exe --sanity-check` in "Win64 native" task (Hennadii Stepanov)

Pull request description:

  This PR adds [`--sanity-check`](https://github.com/bitcoin/bitcoin/pull/25107) flag to `src\bench_bitcoin.exe` invocation as its results are been discarded.

  Also a better name used for the script as it follows GNU's `make check`.

ACKs for top commit:
  fanquake:
    ACK 91bee4d898

Tree-SHA512: fd5feeda72d1ef46c5fbfc2aa5c042ab2e3de7772546379da4596306b5658ab95f62939fba237c0bd7a1b09c85de20fc1cd9e5df1efe11bdae50d4a7b8081f74
2022-10-03 13:54:21 +02:00
Hennadii Stepanov
30cc1c6609 refactor: Drop owns_lock() call
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2022-10-03 12:26:37 +01:00
fanquake
c21b32ccd1 Merge bitcoin/bitcoin#26198: refactor: move Boost Datetime usage to wallet
079cf88c0d refactor: move Boost datetime usage to wallet (fanquake)

Pull request description:

  This means we don't need Boost Datetime in a `--disable-wallet` build, and it isn't included in the kernel (via time.h/cpp). Split from a larger boost removal branch/effort.

ACKs for top commit:
  hebasto:
    re-ACK 079cf88c0d
  aureleoules:
    re-ACK 079cf88c0d - rebased and two additional unit tests since my last review.
  jarolrod:
    crACK 079cf88c0d

Tree-SHA512: c84f47158a4f21902f211c059d8c4bd55ffe95a256835deee723653be08cca49eeddfc33a2316b0cd31805e81cf77eaa39c6c9dcff4cda11a26ba4c1c143974e
2022-10-03 11:13:12 +01:00
willcl-ark
874c861885 doc: Add I2P bandwidth guidance to i2p.md
Adds some general guidance on lowering bandwidth usage when using I2P
routers.
2022-10-03 10:36:45 +01:00
MacroFake
132d98a0a6 Merge bitcoin/bitcoin#26229: test: Use proper Boost macros instead of assertions
5c9a27a46f test: Use proper Boost macros instead of assertions (Hennadii Stepanov)

Pull request description:

  On the master branch:
  ```
  $ src/test/test_bitcoin -l test_suite -t banman_tests
  Running 1 test case...
  ...
  Test case banman_tests/file did not check any assertions
  ...
  ```

  This PR suggests to use proper Boost [macros](https://www.boost.org/doc/libs/1_80_0/libs/test/doc/html/boost_test/utf_reference/testing_tool_ref.html).

Top commit has no ACKs.

Tree-SHA512: e0c8e5e6371acd0e0a80070fffdf1445f264c62499f8d9811822994c89735a913c18c8ed730495578400abdd93d2d500345504f2a9246401d53fb2f9f71be8c5
2022-10-03 11:36:08 +02:00
fanquake
6c5ef5d460 Merge bitcoin/bitcoin#26208: signet/miner: reduce default interblock interval limit to 30min
51a08f41ff signet/miner: reduce default interblock interval limit to 30min (Anthony Towns)

Pull request description:

  Reduces the cap on the time between blocks from 60 minutes to 30 minutes, and makes it configurable.

Top commit has no ACKs.

Tree-SHA512: 7b880c50e47d055a2737c057fab190017748849d264c6c39dde465959a544d502221d12c6307d4de693f51badb4779b9c147e48775ede6ec6613e808067ab279
2022-10-03 09:14:22 +01:00
fanquake
25742aa3ed Merge bitcoin/bitcoin#26231: doc: bump bips.md up-to-date version to v24.0
a9d20eeceb doc: bump bips.md up-to-date version to v24.0 (Sebastian Falbesoner)

Pull request description:

  This is a trivial follow-up to #26124.

ACKs for top commit:
  jarolrod:
    ACK a9d20eeceb

Tree-SHA512: 24c17c72498f96f9122d8fb041f1f6f63bd186e25ac3cb5a661bb1993106c6632f5efd95a15d19681004d30d38eca2d2a16b383a7a1f1c3db17f887ae1fcd02a
2022-10-03 09:06:10 +01:00
Sebastian Falbesoner
a9d20eeceb doc: bump bips.md up-to-date version to v24.0
This is a trivial follow-up to #26124.
2022-10-03 01:25:31 +02:00
Hennadii Stepanov
91bee4d898 ci: Run bench_bitcoin.exe --sanity-check in "Win64 native" task
Also a better name used for the script as it follows GNU's `make check`.
2022-10-03 00:23:33 +01:00
Hennadii Stepanov
5c9a27a46f test: Use proper Boost macros instead of assertions 2022-10-03 00:00:31 +01:00
fanquake
93001b16a4 Merge bitcoin/bitcoin#26216: fuzz: Limit outpoints.size in txorphan target to avoid OOM
fa5752da6a fuzz: Limit outpoints.size in txorphan target to avoid OOM (MacroFake)

Pull request description:

  Fixes https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=52008

ACKs for top commit:
  fanquake:
    ACK fa5752da6a

Tree-SHA512: f010c0eabb72ad4bbf428954f6f978e88d6d15ec3ee77536334b11c0ca605377bdaa40ecf1984f027a430d62f05e9201775f5a6b047ffa38563aeefc04958a1f
2022-10-02 16:37:40 +01:00
fanquake
13ec2c1913 Merge bitcoin/bitcoin#24084: doc: add information about status code 404 for some endpoints (rest)
0811cbfc28 doc: add info about status code 404 for some rest endpoints (brunoerg)

Pull request description:

  This PR adds an explanation about status code 404 for 2 endpoints (`/rest/tx/ `and `/rest/blockhashbyheight/`) in`REST-interface.md`. There are other endpoints that already cover it.

ACKs for top commit:
  [deleted]:
    reACK 0811cbfc28
  shaavan:
    ACK 0811cbfc28

Tree-SHA512: a01ac6653f706b7a7e4a4679a2b81e448381f31460ac4bcfc179af6186401cffae7b49a82f3a52c89e556acd5c16c159ce752c7a678177900ddf2e4e5c72fe6b
2022-10-02 16:29:31 +01:00
MacroFake
abf20902d0 Merge bitcoin/bitcoin#26212: contrib: Fix capture_output in getcoins.py
be59bd17ec contrib: Fix capture_output in getcoins.py (willcl-ark)

Pull request description:

  Our minimum required Python version 3.6.12 does not support `capture_output` as a subprocess.run argument; this was added in python 3.7.

  We can emulate it by setting both stdout and stderr to `subprocess.PIPE`

Top commit has no ACKs.

Tree-SHA512: 3158bfe657e2d5b263c9337a7ac1dd830e7e119854ec54a40a9e511329fe5bed073350bf70c66b5a6cc9a27dc8eb0e062bbac3969640fd7d6475379726003c98
2022-10-02 16:50:14 +02:00
MacroFake
3a230f4ce1 Merge bitcoin/bitcoin#26211: ci: Bump vcpkg to the latest version 2022.09.27
281e7c715d ci: Bump vcpkg to the latest version `2022.09.27` (Hennadii Stepanov)

Pull request description:

  Dependency changes in [vcpkg](https://github.com/microsoft/vcpkg) (2022.06.16.1 - [2022.09.27](https://github.com/microsoft/vcpkg/releases/tag/2022.09.27)):
   - boost 1.79.0#0 -> 1.80.0#0
   - sqlite3 3.37.2#1 -> 3.39.2#0
   - zeromq 4.3.4#5 -> 4.3.4#6

  The recent update was in bitcoin/bitcoin#25460.

ACKs for top commit:
  sipsorcery:
    tACK 281e7c715d.

Tree-SHA512: 624e2506eb16fb37d1ca0a71caa12e64f8709c0ddd280e3d1e0f6a8fa2a3667b0f8a2f52d553e096c9f8cd50e4e220e23a23fdb97076d7bcdfab0951e94909a1
2022-10-02 14:01:54 +02:00
Aurèle Oulès
d0d9cf7aea test: Check external coin effective value is used in CoinSelection 2022-10-02 01:34:45 +02:00
Aurèle Oulès
76b79c1a17 wallet: Use correct effective value when checking target 2022-10-02 01:34:25 +02:00
fanquake
079cf88c0d refactor: move Boost datetime usage to wallet
This means we don't need datetime in a --disable-wallet build, and it
isn't included in the kernel.
2022-10-01 11:41:53 +01:00
MacroFake
f59e91511a Merge bitcoin/bitcoin#26131: log: log RPC port on startup
9d14f27bdd log: log RPC port on startup (James O'Beirne)

Pull request description:

  I just spent a few hours trying to figure out why "18444" wasn't getting me to regtest's RPC server. I'm not the sharpest tool in the shed, but I was maybe understandably confused because "Bound to 127.0.0.1:18445" appears in the logs, which I assumed was the P2P port.

  This change logs the RPC listening address by default on startup, which seems like a basic piece of information that shouldn't be buried under `-debug`.

ACKs for top commit:
  dergoegge:
    ACK 9d14f27bdd
  jarolrod:
    ACK 9d14f27bdd
  aureleoules:
    ACK 9d14f27bdd

Tree-SHA512: 5c86f018c0b8d6264abf878c921afe53033b23ab4cf289276bb1ed28fdf591c9d8871a4baa4098c363cb2aa9a637d2e4e18e56b14dfc7d767ee40757d7ff2e7c
2022-09-30 16:29:22 +02:00
stickies-v
b01682a812 refactor: revert m_next_resend to not be std::atomic
Since m_next_resend is now only called from MaybeResendWalletTxs()
we don't have any potential race conditions anymore, so the usage
of std::atomic can be reverted.
2022-09-30 14:47:39 +01:00
stickies-v
9245f45670 wallet: only update m_next_resend when actually resending
We only want to relay our resubmitted transactions once every 12-36h.
By separating the timer update logic out of ResubmitWalletTransactions
and into MaybeResendWalletTxs we avoid non-relay calls (previously in
the separate ReacceptWalletTransactions function) from resetting that
timer.
2022-09-30 14:47:39 +01:00
stickies-v
7fbde8af5c refactor: carve out tx resend timer logic into ShouldResend
Moves the logic of whether or not transactions should actually be
resent out of the function that's resending them. This reduces
responsibilities of ResubmitWalletTransactions and allows
carving out the updating of m_next_resend in a future commit.
2022-09-30 14:47:38 +01:00
willcl-ark
be59bd17ec contrib: Fix capture_output in getcoins.py
Our required Python version 3.6.12 does not support `capture_output` as
a subprocess.run argument; this was added in python 3.7.

We can emulate it by setting stdout and stderr to subprocess.PIPE
2022-09-30 13:20:48 +01:00
Hennadii Stepanov
281e7c715d ci: Bump vcpkg to the latest version 2022.09.27
Dependency changes (2022.06.16.1 - 2022.09.27):
 - boost 1.79.0#0 -> 1.80.0#0
 - sqlite3 3.37.2#1 -> 3.39.2#0
 - zeromq 4.3.4#5 -> 4.3.4#6
2022-09-30 12:53:50 +01:00
brunoerg
bff05bd745 test: add functional test for -discover 2022-09-30 08:26:45 -03:00
MacroFake
33eef562a3 Merge bitcoin/bitcoin#26074: refactor: Set RPCArg options with designated initializers
fa2c72dda0 rpc: Set RPCArg options with designated initializers (MacroFake)

Pull request description:

  For optional constructor arguments, use a new struct. This comes with two benefits:
  * Earlier unused optional arguments can be omitted
  * Designated initializers can be used

ACKs for top commit:
  stickies-v:
    re-ACK fa2c72dda0

Tree-SHA512: 2a0619548187cc7437fee2466ac4780746490622f202659f53641be01bc2a1fea4416d1a77f3e963bf7c4cce62899b61fab0b9683440cf82f68be44f63826658
2022-09-30 10:06:14 +02:00
brunoerg
4aff7a48a4 test: check importing wallets when blocks are pruned throw an error 2022-09-29 14:43:43 -03:00
MacroFake
437b608df2 Merge bitcoin/bitcoin#26202: ci: Use remote pull/merge ref instead of local git merge
fad7281d78 ci: Use remote pull/merge ref instead of local git merge (MacroFake)

Pull request description:

  The merge strategy on the remote may be different than the local one. This may cause local merges to be different or fail completely. Fix this by using the result of the remote merge.

  Fixes https://github.com/bitcoin/bitcoin/issues/26163

ACKs for top commit:
  hebasto:
    ACK fad7281d78, I regularly use the same commands locally.

Tree-SHA512: 0febbf5db8c1536e31b374a7599a92037ca814174809075f42c7c7c4e1daaab5b3df09cf82f2de0d1e847c41eb30e770daaf7a85287f5d8d43ebd642d1234d3c
2022-09-29 19:15:00 +02:00
stickies-v
01f3534632 refactor: remove unused locks for ResubmitWalletTransactions
ReacceptWalletTransactions is replaced by ResubmitWalletTransactions
which already handles acquiring the necessary locks internally.
2022-09-29 17:32:53 +01:00
stickies-v
c6e8e11fb0 wallet: fix capitalization in docstring 2022-09-29 17:32:52 +01:00
fanquake
13601da17e Update leveldb-subtree subtree to latest upstream 2022-09-29 16:43:30 +01:00
fanquake
1a463c70a3 Squashed 'src/leveldb/' changes from 22f1e4a02f..e2f10b4e47
e2f10b4e47 Merge bitcoin-core/leveldb-subtree#34: win32: fix -Wmissing-field-initializers warnings
12c52b392d win32: fix -Wmissing-field-initializers warnings

git-subtree-dir: src/leveldb
git-subtree-split: e2f10b4e47bc950a81bc96d1c6db3a8048216642
2022-09-29 16:43:30 +01:00
MacroFake
fad7281d78 ci: Use remote pull/merge ref instead of local git merge
The merge strategy on the remote may be different than the local one.
This may cause local merges to be different or fail completely. Fix this
by using the result of the remote merge.
2022-09-29 13:20:38 +02:00
Anthony Towns
51a08f41ff signet/miner: reduce default interblock interval limit to 30min
Also allow the operator to change it, if desired, without having
to edit the code.
2022-09-29 14:02:58 +10:00
Gleb Naumenko
3c43d9db1e p2p: Don't self-advertise during VERSION processing
Previously, we would prepare to self-announce to a new peer while
parsing a VERSION message from that peer. This is redundant, because we
do something very similar in MaybeSendAddr(), which is called from
SendMessages() after the version handshake is finished.

There are a couple of differences:

1) MaybeSendAddr() self-advertises to all peers we do address relay with,
   not just outbound ones.
2) GetLocalAddrForPeer() called from MaybeSendAddr() makes a
   probabilistic decision to either advertise
   what they think we are or what we think we are, while
   PushAddress(self) on VERSION deterministically only does
   the former if the address from the latter is unroutable.
3) During VERSION processing, we haven't received a potential sendaddrv2 message
   from our peer yet, so self-advertisements with addresses from addrV2-only networks
   would always be dropped in PushAddress().

Since it's confusing to have two slightly different mechanisms for self-advertising,
and the one in MaybeSendAddr() is better, remove the one in VERSION.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2022-09-28 17:41:56 -04:00
furszy
f1593780b8 bench: place benchmark implementation inside benchmark namespace 2022-09-28 13:27:51 -03:00
glozow
b2da6dd943 Merge bitcoin/bitcoin#26138: test: Avoid race in disconnect_nodes helper
faeea28753 test: Avoid race in disconnect_nodes helper (MacroFake)

Pull request description:

  Also wait for the other node to notice the closed socket. Otherwise, the other node is not able to use the connect helper.

  Fixes https://github.com/bitcoin/bitcoin/issues/26014

ACKs for top commit:
  stickies-v:
    ACK faeea2875
  glozow:
    ACK faeea28753

Tree-SHA512: 2f0fa6812c0519aba3eaf21f0c70073b768fcd4dad23989d57e138ee9057a7da1a6b281645e9bff4051259cdca51568700e066491ac6b6daae99f30e395159ca
2022-09-28 16:21:19 +01:00
MacroFake
291e363ce5 Merge bitcoin/bitcoin#26195: ci: Use git2.34 for lint task
d8ded8bc08 ci: Use git2.34 for lint task (MacroFake)

Pull request description:

  Since most maintainers use a recent version of git that uses the `ort` strategy by default (https://git-scm.com/docs/merge-strategies/2.34.0), bump git for the lint taks as well.

  Fixes https://github.com/bitcoin/bitcoin/pull/26130#issuecomment-1260499544

ACKs for top commit:
  fanquake:
    ACK d8ded8bc08 - seems fine for now, and to keep python3.6 around. When we bump to >= Jammy in future we'll have to pick from Python3.10+.

Tree-SHA512: 5a9c40b1c242678a7f92e641db026309b3e2e99d7d032778c98eeb56f7abd65f9e0a24f9b2ccf0350d5c0286d50f1ac5969e4249beaa5ffc4b00d06ca8b141bc
2022-09-28 09:25:22 +00:00
MacroFake
d8ded8bc08 ci: Use git2.34 for lint task 2022-09-28 19:58:13 +02:00
Hennadii Stepanov
b147322a7a Use PACKAGE_NAME in messages rather than hardcoding "Bitcoin Core" 2022-09-27 23:04:37 +01:00
Hennadii Stepanov
bff4e068b6 refactor: Do not discard try_lock() return value
Microsoft's C++ Standard Library uses the `[[nodiscard]]` attribute for
`try_lock()`.
See: https://github.com/microsoft/STL/blob/main/stl/inc/mutex

This change allows to drop the current suppression for the warning C4838
and helps to prevent the upcoming warning C4858.
See: 539c26c923
2022-09-27 22:35:16 +01:00
glozow
9fcdb9f3a0 Merge bitcoin/bitcoin#26172: p2p: ProcessHeadersMessage(): fix received_new_header
bdcafb9133 p2p: ProcessHeadersMessage(): fix received_new_header (Larry Ruane)

Pull request description:

  Follow-up to #25717. The commit "Utilize anti-DoS headers download strategy" changed how this bool variable is computed, so that its value is now the opposite of what it should be.

  Prior to #25717:
  ```
  bool received_new_header{WITH_LOCK(::cs_main, return m_chainman.m_blockman.LookupBlockIndex(headers.back().GetHash()) == nullptr)};
  ```
  After #25717 (simplified):
  ```
  {
      LOCK(cs_main);
      last_received_header = m_chainman.m_blockman.LookupBlockIndex(headers.back().GetHash());
  }
  bool received_new_header{last_received_header != nullptr};
  ```

ACKs for top commit:
  dergoegge:
    ACK bdcafb9133
  glozow:
    ACK bdcafb9133, I believe this is correct and don't see anything to suggest the switch was intentional.
  stickies-v:
    ACK bdcafb9133

Tree-SHA512: 35c12762f1429585a0b1c15053e310e83efb28c3d8cbf4092fad9fe81c893f6d766df1f2b20624882acb9654d0539a0c871f587d7090dc2a198115adf59db3ec
2022-09-27 11:02:44 +01:00
MacroFake
eeac05aa22 Merge bitcoin/bitcoin#26156: test: check that listdescriptors descriptor strings are sorted
810c3dc7ef doc, rpc: mention that `listdescriptors` result is sorted by string representation (Sebastian Falbesoner)
d99af861d0 test: check that `listdescriptors` descriptor strings are sorted (Sebastian Falbesoner)

Pull request description:

  This small PR adds a test for the change introduced in PR #25931 ("rpc: sort listdescriptors result", commit 50996241f2). The correctness of the test can easily be verified by commenting out the `std::sort` call in the `listdescriptors` RPC implementation:
  ```diff
  diff --git a/src/wallet/rpc/backup.cpp b/src/wallet/rpc/backup.cpp
  index 09c74ea2da..3ed1a69b26 100644
  --- a/src/wallet/rpc/backup.cpp
  +++ b/src/wallet/rpc/backup.cpp
  @@ -1829,9 +1829,11 @@ RPCHelpMan listdescriptors()
           });
       }

  +    /*
       std::sort(wallet_descriptors.begin(), wallet_descriptors.end(), [](const auto& a, const auto& b) {
           return a.descriptor < b.descriptor;
       });
  +    */

       UniValue descriptors(UniValue::VARR);
       for (const WalletDescInfo& info : wallet_descriptors) {

  ```
  leading to a fail of the functional test `wallet_listdescriptors.py`.

ACKs for top commit:
  jarolrod:
    ACK 810c3dc7ef
  aureleoules:
    ACK 810c3dc7ef

Tree-SHA512: 31770e3149b8a0251ecfa8662a2270c149f778eb910985f48a91d6a5d288b7b1c2244f9f1b798ebe3f1aa9f0b935cb4d6f12d5d28f78bcde3c4a61af76d11d0a
2022-09-27 09:27:00 +00:00
Sebastian Falbesoner
810c3dc7ef doc, rpc: mention that listdescriptors result is sorted by string representation 2022-09-26 15:16:01 +02:00
fanquake
20adaeaef5 build: split ARM crc & crypto extension checks
We currently perform the same check twice, to put the same set of flags
in two different variables. Split the checks so we test for crc and crypto
extensions independently.

If we don't want to split, we should just delete the second AX_CHECK_COMPILE_FLAG
check, and set ARM_CRC_CXXFLAGS & ARM_CRC_CXXFLAGS at the same time.
2022-09-26 11:23:03 +01:00
fanquake
f227e153e8 Merge bitcoin/bitcoin#26161: build: remove unused MSVC defines
73ae72e603 build: remove unused MSVC defines (fanquake)

Pull request description:

  Most of these aren't unused in the codebase at all, let alone for MSVC.

ACKs for top commit:
  sipsorcery:
    tACK 73ae72e603.

Tree-SHA512: 6a8aef69902d5941fba741777c19176b96d11a4c75cf27bdf3881019fc650e8f60dd7960a1bc9469a24c4575f7d34cf675afdcf4bcb240c32cab1c9c6ab286db
2022-09-26 11:20:13 +01:00
Hennadii Stepanov
f09d47b263 bench: Add missed ECCVerifyHandle instance 2022-09-26 11:03:36 +01:00
MacroFake
ec13810a00 Merge bitcoin/bitcoin#26159: build: Remove stdlib.h from header checks
553ff452c0 build: remove stdlib.h from header checks (fanquake)
a63d4cb26a refactor: use <cstdlib> over stdlib.h (fanquake)

Pull request description:

  We already use a mix of `<cstlib>` and `stdlib.h` unconditionally throughout
  the codebase.

  Us checking this header also duplicates work already done by autotools.
  Currently stdlib.h is checked for 3 times during a ./configure run, after
  this change, at least it's only twice.

  Similar to #26150.

ACKs for top commit:
  kristapsk:
    ACK 553ff452c0
  TheCharlatan:
    ACK 553ff452c0

Tree-SHA512: 0a43d39d3df180a1614dbd3a1ee1531b0969ffe4a0c09dfe9d2f3f0ec16196b5fd7523309f6722936a8c8b20908508724e1903e939dd81c3b4538d85d0f42953
2022-09-25 11:45:41 +00:00
MacroFake
0cfbb171bd Merge bitcoin/bitcoin#26130: Bugfix: Wallet: Lock cs_wallet for SignMessage
a60d9eb9e6 Bugfix: Wallet: Lock cs_wallet for SignMessage (Luke Dashjr)

Pull request description:

  cs_desc_main is typically locked within scope of a cs_wallet lock, but:

  CWallet::IsLocked locks cs_wallet
  ...called from DescriptorScriptPubKeyMan::GetKeys
  ...called from DescriptorScriptPubKeyMan::GetSigningProvider which locks cs_desc_main first, but has no access to cs_wallet ...called from DescriptorScriptPubKeyMan::SignMessage ...called from CWallet::SignMessage which can access and lock cs_wallet

  Resolve the out of order locks by grabbing cs_wallet in CWallet::SignMessage first

  -------------

  Note this is currently only an issue for the GUI (which lacks sufficient testing apparently), but can be reproduced by #26082 (CI fails as a result)

ACKs for top commit:
  achow101:
    ACK a60d9eb9e6
  w0xlt:
    ACK a60d9eb9e6

Tree-SHA512: 60f6959b0ceaf4d9339ba1a47154734034b637c41b1f9e26748a2dbbc3a2a95fc3696019103c55ae70c91d910ba8f3d7f4e27d263030eb60b689f290c4d82ea9
2022-09-24 14:02:13 +00:00
Larry Ruane
bdcafb9133 p2p: ProcessHeadersMessage(): fix received_new_header
Follow-up to #25717. The commit "Utilize anti-DoS headers download
strategy" changed how this bool variable is computed, so that its value
is now the opposite of what it should be.
2022-09-24 00:07:46 -06:00
fanquake
73ae72e603 build: remove unused MSVC defines
Most of these aren't unused in the codebase at all, let alone for MSVC.
2022-09-23 16:30:24 +01:00
fanquake
553ff452c0 build: remove stdlib.h from header checks
We already use a mix of <cstdlib> and stdlib.h unconditionally throughout
the codebase.

Us checking this header also duplicates work already done by autotools.
Currently stdlib.h is checked for 3 times during a ./configure run, after
this change, at least it's only twice.
2022-09-23 10:49:33 +01:00
fanquake
a63d4cb26a refactor: use <cstdlib> over stdlib.h
We currently use both. Consolidate on the former.
2022-09-23 10:48:47 +01:00
Sebastian Falbesoner
d99af861d0 test: check that listdescriptors descriptor strings are sorted
Tests the change introduced in PR #25931 ("rpc: sort listdescriptors
result", commit 50996241f2).
2022-09-22 19:29:43 +02:00
Jon Atack
a3789c700b Improve getpeerinfo pingtime, minping, and pingwait help docs 2022-09-22 16:45:48 +02:00
Jon Atack
df660ddb1c Update getpeerinfo/-netinfo/TxRelay#m_relay_txs relaytxes docs (for v24 backport)
to the current p2p behavior.  We only initialize the Peer::TxRelay m_relay_txs
data structure if it isn't an outbound block-relay-only connection and fRelay=true
(the peer wishes to receive tx announcements) or we're offering NODE_BLOOM to this peer.
2022-09-22 16:45:32 +02:00
Jon Atack
1f448542e7 Always return getpeerinfo "minfeefilter" field (for v24 backport)
with its pre-existing v23 default value of 0.
2022-09-22 16:45:20 +02:00
Jon Atack
9cd6682545 Make getpeerinfo field order consistent with its help (for v24 backport)
This also keeps it consistent with the last release (v23)
2022-09-22 16:44:38 +02:00
fanquake
100949af0e Merge bitcoin/bitcoin#26012: fuzz: Avoid timeout in bitdeque fuzz target
fa4ba04c15 fuzz: Remove no-op call to get() (MacroFake)
fa642286b8 fuzz: Avoid timeout in bitdeque fuzz target (MacroFake)

Pull request description:

  I'd guess that any bug should be discoverable within `10` ops. However, `900` seems also better than no limit at all, which causes timeouts such as https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=50892

ACKs for top commit:
  sipa:
    ACK fa4ba04c15

Tree-SHA512: f6bd25e78d5f04c6f88e9300c2fa3d0993a0911cb0fd1b414077adc0edde1a06ad72af5e2f50f0ab1324f91999ae57d879686c545b2e6c19ae7f637a8804bd48
2022-09-22 14:55:43 +01:00
fanquake
590d206444 Merge bitcoin/bitcoin#26150: build: remove stdio.h from header checks
55aad5f3a9 build: remove stdio.h from header checks (fanquake)
b95633121b refactor: use <cstdio> over stdio.h (fanquake)

Pull request description:

  We already use a mix of `<cstdio>` and `stdio.h` unconditionally throughout
  the codebase.

  Us checking this header also duplicates work already done by autotools.
  Currently `stdio.h` is checked for 3 times during a ./configure run, after
  this change, at least it's only twice.

ACKs for top commit:
  TheCharlatan:
    ACK 55aad5f3a9
  kristapsk:
    ACK 55aad5f3a9

Tree-SHA512: a83cc724528ab92aacfa53048b12fcccec3962637ca7fad30f6c610365edeb0e951f74e37832ad7d3f79ca9b8d7203cb10165c89d0e4b63eeda7a970dab82dfb
2022-09-22 14:45:13 +01:00
fanquake
859644b3c8 build: set D_WIN32_WINNT=0x0601 for mingw miniupnpc
This matches configure, and what we set for libevent etc.
2022-09-22 14:36:05 +01:00
fanquake
8e2d93ff0f build: fix cflags passing for mingw miniupnpc
Cherry-pick of a patch upstreamed to miniupnpc, see here:
https://github.com/miniupnp/miniupnp/pull/619.
2022-09-22 14:24:52 +01:00
Andrew Chow
2b2c970627 Merge bitcoin/bitcoin#26149: Fix assert failure in miniscript string parsing
648f6950cd Correct sanity-checking script_size calculation (Pieter Wuille)

Pull request description:

  Fix a bug in the script_size sanity-check in the miniscript string parser, found by oss-fuzz in https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=51636, and introduced in e8cc2e4afc (#25540).

  This bug would cause an assertion failure when feeding a miniscript with a `thresh(k,...)` fragment, with k >= 128, to an RPC.

ACKs for top commit:
  darosior:
    utACK 648f6950cd
  achow101:
    ACK 648f6950cd

Tree-SHA512: d86a0721758cd1e42ef02050b542f0935efdc19447a1ca76a3ade96352a6ee8261eef3d4a5cbdec77bf0ad14dfed42e9eb6bd4246b816a9f6f06d786900da9e7
2022-09-21 13:17:07 -04:00
fanquake
74e54cc2a3 Merge bitcoin/bitcoin#25917: depends: libnatpmp 07004b97cf691774efebe70404cf22201e4d330d
65471008e0 depends: libnatpmp 07004b97cf691774efebe70404cf22201e4d330d (fanquake)

Pull request description:

  This pulls in two changes I've upstreamed:
  Support for pkg-config: https://github.com/miniupnp/libnatpmp/pull/19
  Suppressing a deprecation warning: https://github.com/miniupnp/libnatpmp/pull/28

  and other upstream bug fixes.

  Somewhat related to #22644.

  Guix Build (arm64):
  ```bash
  4a4b8e1cb5070482e53b3b869c5f6e19b892619184cf58315a3af368e13a52a2  guix-build-65471008e0a4/output/arm-linux-gnueabihf/SHA256SUMS.part
  88b51f0f261f84b58da60a5c8df251b551379b811a255df9982ff496c70fc8b7  guix-build-65471008e0a4/output/arm-linux-gnueabihf/bitcoin-65471008e0a4-arm-linux-gnueabihf-debug.tar.gz
  33a564914be6f4134d8e8a5f31a2bb97798a79bae8f75151b35564238d6f288e  guix-build-65471008e0a4/output/arm-linux-gnueabihf/bitcoin-65471008e0a4-arm-linux-gnueabihf.tar.gz
  ea429260a2b5ac333f70b2b8b8697a4f76eee79cf6998d18c7de7b60818d8fe5  guix-build-65471008e0a4/output/arm64-apple-darwin/SHA256SUMS.part
  304677e040592a0aee947c733657ea6c4dd8613f7c3a9834314a68e5f7a53325  guix-build-65471008e0a4/output/arm64-apple-darwin/bitcoin-65471008e0a4-arm64-apple-darwin-unsigned.dmg
  da171a60370ad1da6cfc1566fc4e16d09f6e896002b0ac9758fb4044eb1033c1  guix-build-65471008e0a4/output/arm64-apple-darwin/bitcoin-65471008e0a4-arm64-apple-darwin-unsigned.tar.gz
  45c439319a9e2c07de8796dcfd182cdeb5600ad31f8f6b6dc383cc2914f49ab8  guix-build-65471008e0a4/output/arm64-apple-darwin/bitcoin-65471008e0a4-arm64-apple-darwin.tar.gz
  120311e7c807b2641b5e7024e1b85fac0800680bccd924016f417b026aa90d25  guix-build-65471008e0a4/output/dist-archive/bitcoin-65471008e0a4.tar.gz
  5272264317c93359c158c688cf0d0b6e70a6d9ec4b2e447b9fb9fcaf1950f6bc  guix-build-65471008e0a4/output/powerpc64-linux-gnu/SHA256SUMS.part
  98a0d14b8d05974d70f0d5243bfc2fec6aea30b9d8375f950f42a21d5e034065  guix-build-65471008e0a4/output/powerpc64-linux-gnu/bitcoin-65471008e0a4-powerpc64-linux-gnu-debug.tar.gz
  8e07410ad2e7e204c5e84a81894c5398b34f36cc9e1ba386bf0963f09755818b  guix-build-65471008e0a4/output/powerpc64-linux-gnu/bitcoin-65471008e0a4-powerpc64-linux-gnu.tar.gz
  47ae8b948127f03e43a4c7e83a3611b650e26da50936ea9ce0247e01b29463f3  guix-build-65471008e0a4/output/powerpc64le-linux-gnu/SHA256SUMS.part
  04e92527df8c7bb9f78c29b152d920830be0ef26d01e0ee1b32d52bb68fc9717  guix-build-65471008e0a4/output/powerpc64le-linux-gnu/bitcoin-65471008e0a4-powerpc64le-linux-gnu-debug.tar.gz
  ddb09291c9e270343b56e66fae33fd792eb2ec7e5a138337ec1608642dd3a983  guix-build-65471008e0a4/output/powerpc64le-linux-gnu/bitcoin-65471008e0a4-powerpc64le-linux-gnu.tar.gz
  de3cfcf47abed3d4d3881086eba94ee2c34d9078dc53639cc6321c2bd1041073  guix-build-65471008e0a4/output/riscv64-linux-gnu/SHA256SUMS.part
  fecd6a307b66fd54fa5792ce14e1a4d50df5646cdf6cc51d5b355fb1384f5105  guix-build-65471008e0a4/output/riscv64-linux-gnu/bitcoin-65471008e0a4-riscv64-linux-gnu-debug.tar.gz
  f2c5a85fb17ad9605809b4ac01eba56cd573744697b9c942d293b42a623a2832  guix-build-65471008e0a4/output/riscv64-linux-gnu/bitcoin-65471008e0a4-riscv64-linux-gnu.tar.gz
  4cd0b05e96491d9b0fe7eff78aab885a2f0d62f141950117759fecceaeba5412  guix-build-65471008e0a4/output/x86_64-apple-darwin/SHA256SUMS.part
  bae30549d8044e1aa11bae6d57321a7a3a765c8b87ead9cdcc8abf9f22a564a3  guix-build-65471008e0a4/output/x86_64-apple-darwin/bitcoin-65471008e0a4-x86_64-apple-darwin-unsigned.dmg
  435187b492d6df77d4fa7cd0f77ad08c587c4bd463295be802b6e222ee92a2fa  guix-build-65471008e0a4/output/x86_64-apple-darwin/bitcoin-65471008e0a4-x86_64-apple-darwin-unsigned.tar.gz
  b9cf808ec88368a9b0e1883253a9d8c6f123bf7f61df40e34e5e4b8959846d65  guix-build-65471008e0a4/output/x86_64-apple-darwin/bitcoin-65471008e0a4-x86_64-apple-darwin.tar.gz
  b9d9a46f7d3188cd517a180fd24fdb38b8c695175fe9ac75c3b3eb2450d9f5d5  guix-build-65471008e0a4/output/x86_64-linux-gnu/SHA256SUMS.part
  fd601d6821fbe5faa1eb44b9f07046187f491a83fbf1e0fd603be212bfea3385  guix-build-65471008e0a4/output/x86_64-linux-gnu/bitcoin-65471008e0a4-x86_64-linux-gnu-debug.tar.gz
  98ae9d653a8fdeda055468dbf674ee46b0727e1c5b768b1765fc4451defed36e  guix-build-65471008e0a4/output/x86_64-linux-gnu/bitcoin-65471008e0a4-x86_64-linux-gnu.tar.gz
  821de40cbc96c27bb1f0fbcfb46c4e801f202c2dfd7aa9ad88daf6ced2be90e5  guix-build-65471008e0a4/output/x86_64-w64-mingw32/SHA256SUMS.part
  0f2425ea0a17c9aa80cf7d2eb316b9ebe76f49de777635ac37f9e6e21bcd650e  guix-build-65471008e0a4/output/x86_64-w64-mingw32/bitcoin-65471008e0a4-win64-debug.zip
  046ede05868c153ec1d87b8dad4b6494913ea3e952edee86b068d367c57fa7ec  guix-build-65471008e0a4/output/x86_64-w64-mingw32/bitcoin-65471008e0a4-win64-setup-unsigned.exe
  62b6e5ebb085735838ada1f328d6e092d7fb726caad6e049340bfa783897cfa1  guix-build-65471008e0a4/output/x86_64-w64-mingw32/bitcoin-65471008e0a4-win64-unsigned.tar.gz
  265f9814a7fa3333395f34a5dceafd11d50d170aa5303386aa6f8244542d070a  guix-build-65471008e0a4/output/x86_64-w64-mingw32/bitcoin-65471008e0a4-win64.zip
  ```

ACKs for top commit:
  theuni:
    Simple bump ACK 65471008e0. From a quick glance the upstream changes appear to be minimal and sane.
  hebasto:
    ACK 65471008e0

Tree-SHA512: 61541b7dcde611f5bafe5b77977403dab86fe24f0bf4bfb79ab7123bac8b7c4dcad53993d18ab40964756699a77952c8ecc5a0416055c9e436fc34867f7f9cf6
2022-09-21 17:04:16 +01:00
fanquake
55aad5f3a9 build: remove stdio.h from header checks
We already use a mix of <cstdio> and stdio.h unconditionally throughout
the codebase.

Us checking this header also duplicates work already done by autotools.
Currently stdio.h is checked for 3 times during a ./configure run, after
this change, at least it's only twice.
2022-09-21 16:53:18 +01:00
fanquake
b95633121b refactor: use <cstdio> over stdio.h
We currently use both. Consolidate on the former.
2022-09-21 16:53:11 +01:00
fanquake
4e15a288c3 Merge bitcoin/bitcoin#26135: build: remove strings.h from header checks
eb6026b90f build: remove strings.h from header checks (fanquake)

Pull request description:

  We don't include `strings.h` anywhere.

  This is also already checked for by autoconf, so us checking for it just means a 3rd existence check during `./configure`.

ACKs for top commit:
  TheCharlatan:
    Code review ACK eb6026b90f
  hebasto:
    ACK eb6026b90f, tested on macOS 12.6:

Tree-SHA512: 4036c21b2f659140e9f471b4d24336fe925c6c010e2ced36e1f606d9c76dea236d086d15a884eb8f95381b39322abeecab973b10532527005fdadd095411e358
2022-09-21 16:29:50 +01:00
Andrew Chow
25cd47de71 Merge bitcoin/bitcoin#25933: wallet: AvailableCoins, simplify output script type acquisition
58b7df3caa wallet: AvailableCoins, simplify output script type acquisition (furszy)

Pull request description:

  There is an unnecessary `ExtractDestination()` call and subsequent result parse into an `CScriptID`.

  The `Solver()` call, which we are already doing below anyway, retrieves the script type and, in the P2SH case, the program id.

ACKs for top commit:
  achow101:
    ACK 58b7df3caa
  aureleoules:
    re-ACK 58b7df3caa
  rajarshimaitra:
    ACK 58b7df3caa
  w0xlt:
    ACK 58b7df3caa

Tree-SHA512: 51080766877c34cb2232ee3a1cb6b6a62b829c9297c67b99577742b94854a737a74d248015a4603ca9b6cd0a3c9e1d6d78673ff3cc9fc65dd82deea72dc537fd
2022-09-21 11:27:37 -04:00
Pieter Wuille
648f6950cd Correct sanity-checking script_size calculation 2022-09-21 09:17:45 -04:00
Ryan Ofsky
9d3127b11e Add settings.json prune-prev, proxy-prev, onion-prev settings
This provides a way for the GUI settings dialog box to retain previous pruning
and proxy settings when they are disabled, as requested by vasild:

https://github.com/bitcoin/bitcoin/pull/15936#pullrequestreview-937685759
https://github.com/bitcoin/bitcoin/pull/15936#discussion_r850568749
https://github.com/bitcoin/bitcoin/pull/15936#discussion_r852998379

Importantly, while this PR changes the settings.json format, it changes it in a
fully backwards compatible way, so previous versious of bitcoind and bitcoin-qt
will correctly interpret prune, proxy, and onion settins written by new
versions of bitcoin-qt.
2022-09-21 06:25:47 -04:00
fanquake
80d1598133 Merge bitcoin/bitcoin#26099: build: remove duplicate / unneeded libs from bench_bitcoin
ff7c81f63a build: remove duplicate / unneeded libs from bench_bitcoin (fanquake)

Pull request description:

  EVENT_*_LIBS are already in LDADD.
  Move wallet libs into the wallet conditional, similar to zmq.

ACKs for top commit:
  theuni:
    ACK ff7c81f63a

Tree-SHA512: 6bd92f03478d56cd38645e38c0e6c4614cdf9c745124069d0d1d80483d76f5c656e1749061455ba04c619684513a063dda3f8f4bd09fe7b66911714d83592f25
2022-09-21 11:25:47 +01:00
fanquake
b1f44ecdcd Merge bitcoin/bitcoin#25737: rpc: treat univalue type check error as RPC_TYPE_ERROR, not RPC_MISC_ERROR
e68d380797 rpc: remove unneeded RPCTypeCheckArgument checks (furszy)
55566630c6 rpc: treat univalue type check error as RPC_TYPE_ERROR, not RPC_MISC_ERROR (furszy)

Pull request description:

  Same rationale as #26039, tackling another angle of the problem.

  #### Context
  We have the same univalue type error checking code spread/duplicated few times:
  `RPCTypeCheckObj`, `RPCTypeCheckArgument`, `UniValue::checkType`.

  In the first two functions, we are properly returning an `RPC_TYPE_ERROR` while in `UniValue::checkType`
  we are throwing an `std::runtime_error` which is caught by the RPC server request handler, who invalidly
  treats it as `RPC_MISC_ERROR` (which is a generic error return code that provides no information to the user).

  #### Proposed Changes

  Throw a custom exception from `Univalue::checkType` (instead of a plain
  `std::runtime_error`) and catch it on the RPC server request handler.

  So we properly return `RPC_TYPE_ERROR` (-3) on every arg type error and
  not the general `RPC_MISC_ERROR` (-1).

  This will allow us to remove all the `RPCTypeCheckArgument` calls. As them are redundant since #25629.

Top commit has no ACKs.

Tree-SHA512: 4e4c41851fd4e2b01a2d8b94e71513f9831f810768ebd89684caca4901e87d3677980003949bcce441f9ca607a1b38a5894839b6c492f5947b8bab8cd9423ba6
2022-09-21 11:19:44 +01:00
fanquake
3c537f1cc8 Merge bitcoin/bitcoin#25873: depends: Boost 1.80.0
cc8dff5f8f depends: Boost 1.80.0 (fanquake)

Pull request description:

  Mostly misc bug fixes and improvements, to the continually decreasing parts of Boost that we actually use. See: https://www.boost.org/users/history/version_1_80_0.html.

  Includes some boring upstreamed changes, i.e https://github.com/boostorg/multi_index/pull/57, https://github.com/boostorg/signals2/pull/60 that aid #24742. Getting Boost modules to drop their usage of deprecated (redirect) headers means we can prune them from our depends tree.

  Also a requirement for #25696.

  Guix Build (x86_64):
  ```bash
  e793a5ac9372b8fce6e19916be840eee99735bccdedf44b9bca006fd9ac8c395  guix-build-cc8dff5f8ff8/output/aarch64-linux-gnu/SHA256SUMS.part
  001e0382f2b05e12f0ec5eaf09e001e31313ee3ab5367b0ba135ea2d7b863bf6  guix-build-cc8dff5f8ff8/output/aarch64-linux-gnu/bitcoin-cc8dff5f8ff8-aarch64-linux-gnu-debug.tar.gz
  a50229534b41eebd6c44001b56eb2be35b50d5f3a3b161a8fa46d7558b79693e  guix-build-cc8dff5f8ff8/output/aarch64-linux-gnu/bitcoin-cc8dff5f8ff8-aarch64-linux-gnu.tar.gz
  d54d16ae4d5ef2bb0d0dcebbd6d1ec4b0f0976063bf66222a320e68340f902d5  guix-build-cc8dff5f8ff8/output/arm-linux-gnueabihf/SHA256SUMS.part
  04c464043af256ee05d565040df9bb438151d4e041aa46f800fc567eba111c2f  guix-build-cc8dff5f8ff8/output/arm-linux-gnueabihf/bitcoin-cc8dff5f8ff8-arm-linux-gnueabihf-debug.tar.gz
  097cd2f633bf6d3b8e922d81ce9f6dfe793589418e394abfa4e183861f5ad236  guix-build-cc8dff5f8ff8/output/arm-linux-gnueabihf/bitcoin-cc8dff5f8ff8-arm-linux-gnueabihf.tar.gz
  30c8074725b0701b1a781685f50a23ba038297b2633599f40792157daadc13cb  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/SHA256SUMS.part
  df6036e89c7e8746badcbcf805ec4f84070847833c562bc2185c15904b1b909a  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/bitcoin-cc8dff5f8ff8-arm64-apple-darwin-unsigned.dmg
  8421c957b2b83c4afc3bc7f65f93798755acc99ac5921bd89b367c22c5b064d8  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/bitcoin-cc8dff5f8ff8-arm64-apple-darwin-unsigned.tar.gz
  076e8e185bc01dd43feb3fcbcec04eff5ca435b0fdf47c6f05008feec1cf11ac  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/bitcoin-cc8dff5f8ff8-arm64-apple-darwin.tar.gz
  03de7ffcf8d98c61c3a5b763e9bcc6310c9895950f49768da2eeba4871d8478a  guix-build-cc8dff5f8ff8/output/dist-archive/bitcoin-cc8dff5f8ff8.tar.gz
  80ffe674a5fa86cfcf3f47eb8103dbfbc97dd74f793cf785c1347469f19b281f  guix-build-cc8dff5f8ff8/output/powerpc64-linux-gnu/SHA256SUMS.part
  8ce2024c3fa65e6cd613ef526f90c5904675012b22eb39ccf94081d91e14340a  guix-build-cc8dff5f8ff8/output/powerpc64-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64-linux-gnu-debug.tar.gz
  89dc31c0306b0f1c79e13e5f705b73233695a802fd4946aac03364c45e7de984  guix-build-cc8dff5f8ff8/output/powerpc64-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64-linux-gnu.tar.gz
  7ed1b82ba3d4c1a8ddc24c6f650aa6b2d1be08580b2ec32cfec0fd682828d797  guix-build-cc8dff5f8ff8/output/powerpc64le-linux-gnu/SHA256SUMS.part
  01c7f706b236b342f4e0b5043c5788168806338cdaefd5541b04118665c7e40b  guix-build-cc8dff5f8ff8/output/powerpc64le-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64le-linux-gnu-debug.tar.gz
  01767af5b190d0be4a075a5d038644b7dd0d1fc5f0b69698acaca92bfef960d0  guix-build-cc8dff5f8ff8/output/powerpc64le-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64le-linux-gnu.tar.gz
  414de662e0bb3df6a5c8fc1a3576c90a943998b802d97070e31bde434fc6a7f1  guix-build-cc8dff5f8ff8/output/riscv64-linux-gnu/SHA256SUMS.part
  db55ddf8051e6dc80180f3eee12a7370bed1140959c34fb5de8d88f0344b23e2  guix-build-cc8dff5f8ff8/output/riscv64-linux-gnu/bitcoin-cc8dff5f8ff8-riscv64-linux-gnu-debug.tar.gz
  bb19f15545d5a52e800ffb7f141915ab7083fb4fab9a80269f6714b28a294502  guix-build-cc8dff5f8ff8/output/riscv64-linux-gnu/bitcoin-cc8dff5f8ff8-riscv64-linux-gnu.tar.gz
  d9241782d6e596ae02500ffe062501f80b064357bece5d10f7fd4d218240c3a1  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/SHA256SUMS.part
  1c3258a573e849a8efe6fce535a4f8737fb3a076ebe74ee29ca1e84a9113d24f  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/bitcoin-cc8dff5f8ff8-x86_64-apple-darwin-unsigned.dmg
  7e6d562dd636fcacae88bedca45b49a879901c0fc1309ea1812aba59bbbcb5d0  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/bitcoin-cc8dff5f8ff8-x86_64-apple-darwin-unsigned.tar.gz
  c4bc9c27466504417a4bd581cc024b8c5a370a51c8349c1f067752c98c2c14bc  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/bitcoin-cc8dff5f8ff8-x86_64-apple-darwin.tar.gz
  59aaf11181373efd2e281004ce968967a26fa95f90c5a25f44cc4c687e9ddb0a  guix-build-cc8dff5f8ff8/output/x86_64-linux-gnu/SHA256SUMS.part
  fbcde4bfe21314104c7e6036e1f7f4b3ef0e41a7546fef5d1f2a6456b955778d  guix-build-cc8dff5f8ff8/output/x86_64-linux-gnu/bitcoin-cc8dff5f8ff8-x86_64-linux-gnu-debug.tar.gz
  b1302146a0e96f7faa150d764aa0ca92b46e887a886532ee7fc2b2cc63f174c5  guix-build-cc8dff5f8ff8/output/x86_64-linux-gnu/bitcoin-cc8dff5f8ff8-x86_64-linux-gnu.tar.gz
  319b52c1a62a9cdad2e3f1fac8dd22458be2a9c1e6a0d60b33cb27272d69e52a  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/SHA256SUMS.part
  11300b916588cb060ac06e074b94bc5da852ab36446df903045ad593dce5056b  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64-debug.zip
  045fcb6ca721bdefb7490b3452f28449cb2b0449721dbbb20c174be76f96e1a3  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64-setup-unsigned.exe
  38c826537c8054a35103e5ab7ca4f97ca98551f23bcbadb0532f6ca3444e0731  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64-unsigned.tar.gz
  7c5f75c5a0b9b98540c8c779a6fc6f5e98d7de792d3a218e4ad7a68fa4027385  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64.zip
  ```

  Guix Build (arm64):
  ```bash
  d54d16ae4d5ef2bb0d0dcebbd6d1ec4b0f0976063bf66222a320e68340f902d5  guix-build-cc8dff5f8ff8/output/arm-linux-gnueabihf/SHA256SUMS.part
  04c464043af256ee05d565040df9bb438151d4e041aa46f800fc567eba111c2f  guix-build-cc8dff5f8ff8/output/arm-linux-gnueabihf/bitcoin-cc8dff5f8ff8-arm-linux-gnueabihf-debug.tar.gz
  097cd2f633bf6d3b8e922d81ce9f6dfe793589418e394abfa4e183861f5ad236  guix-build-cc8dff5f8ff8/output/arm-linux-gnueabihf/bitcoin-cc8dff5f8ff8-arm-linux-gnueabihf.tar.gz
  30c8074725b0701b1a781685f50a23ba038297b2633599f40792157daadc13cb  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/SHA256SUMS.part
  df6036e89c7e8746badcbcf805ec4f84070847833c562bc2185c15904b1b909a  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/bitcoin-cc8dff5f8ff8-arm64-apple-darwin-unsigned.dmg
  8421c957b2b83c4afc3bc7f65f93798755acc99ac5921bd89b367c22c5b064d8  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/bitcoin-cc8dff5f8ff8-arm64-apple-darwin-unsigned.tar.gz
  076e8e185bc01dd43feb3fcbcec04eff5ca435b0fdf47c6f05008feec1cf11ac  guix-build-cc8dff5f8ff8/output/arm64-apple-darwin/bitcoin-cc8dff5f8ff8-arm64-apple-darwin.tar.gz
  03de7ffcf8d98c61c3a5b763e9bcc6310c9895950f49768da2eeba4871d8478a  guix-build-cc8dff5f8ff8/output/dist-archive/bitcoin-cc8dff5f8ff8.tar.gz
  80ffe674a5fa86cfcf3f47eb8103dbfbc97dd74f793cf785c1347469f19b281f  guix-build-cc8dff5f8ff8/output/powerpc64-linux-gnu/SHA256SUMS.part
  8ce2024c3fa65e6cd613ef526f90c5904675012b22eb39ccf94081d91e14340a  guix-build-cc8dff5f8ff8/output/powerpc64-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64-linux-gnu-debug.tar.gz
  89dc31c0306b0f1c79e13e5f705b73233695a802fd4946aac03364c45e7de984  guix-build-cc8dff5f8ff8/output/powerpc64-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64-linux-gnu.tar.gz
  7ed1b82ba3d4c1a8ddc24c6f650aa6b2d1be08580b2ec32cfec0fd682828d797  guix-build-cc8dff5f8ff8/output/powerpc64le-linux-gnu/SHA256SUMS.part
  01c7f706b236b342f4e0b5043c5788168806338cdaefd5541b04118665c7e40b  guix-build-cc8dff5f8ff8/output/powerpc64le-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64le-linux-gnu-debug.tar.gz
  01767af5b190d0be4a075a5d038644b7dd0d1fc5f0b69698acaca92bfef960d0  guix-build-cc8dff5f8ff8/output/powerpc64le-linux-gnu/bitcoin-cc8dff5f8ff8-powerpc64le-linux-gnu.tar.gz
  414de662e0bb3df6a5c8fc1a3576c90a943998b802d97070e31bde434fc6a7f1  guix-build-cc8dff5f8ff8/output/riscv64-linux-gnu/SHA256SUMS.part
  db55ddf8051e6dc80180f3eee12a7370bed1140959c34fb5de8d88f0344b23e2  guix-build-cc8dff5f8ff8/output/riscv64-linux-gnu/bitcoin-cc8dff5f8ff8-riscv64-linux-gnu-debug.tar.gz
  bb19f15545d5a52e800ffb7f141915ab7083fb4fab9a80269f6714b28a294502  guix-build-cc8dff5f8ff8/output/riscv64-linux-gnu/bitcoin-cc8dff5f8ff8-riscv64-linux-gnu.tar.gz
  d9241782d6e596ae02500ffe062501f80b064357bece5d10f7fd4d218240c3a1  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/SHA256SUMS.part
  1c3258a573e849a8efe6fce535a4f8737fb3a076ebe74ee29ca1e84a9113d24f  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/bitcoin-cc8dff5f8ff8-x86_64-apple-darwin-unsigned.dmg
  7e6d562dd636fcacae88bedca45b49a879901c0fc1309ea1812aba59bbbcb5d0  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/bitcoin-cc8dff5f8ff8-x86_64-apple-darwin-unsigned.tar.gz
  c4bc9c27466504417a4bd581cc024b8c5a370a51c8349c1f067752c98c2c14bc  guix-build-cc8dff5f8ff8/output/x86_64-apple-darwin/bitcoin-cc8dff5f8ff8-x86_64-apple-darwin.tar.gz
  59aaf11181373efd2e281004ce968967a26fa95f90c5a25f44cc4c687e9ddb0a  guix-build-cc8dff5f8ff8/output/x86_64-linux-gnu/SHA256SUMS.part
  fbcde4bfe21314104c7e6036e1f7f4b3ef0e41a7546fef5d1f2a6456b955778d  guix-build-cc8dff5f8ff8/output/x86_64-linux-gnu/bitcoin-cc8dff5f8ff8-x86_64-linux-gnu-debug.tar.gz
  b1302146a0e96f7faa150d764aa0ca92b46e887a886532ee7fc2b2cc63f174c5  guix-build-cc8dff5f8ff8/output/x86_64-linux-gnu/bitcoin-cc8dff5f8ff8-x86_64-linux-gnu.tar.gz
  319b52c1a62a9cdad2e3f1fac8dd22458be2a9c1e6a0d60b33cb27272d69e52a  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/SHA256SUMS.part
  11300b916588cb060ac06e074b94bc5da852ab36446df903045ad593dce5056b  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64-debug.zip
  045fcb6ca721bdefb7490b3452f28449cb2b0449721dbbb20c174be76f96e1a3  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64-setup-unsigned.exe
  38c826537c8054a35103e5ab7ca4f97ca98551f23bcbadb0532f6ca3444e0731  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64-unsigned.tar.gz
  7c5f75c5a0b9b98540c8c779a6fc6f5e98d7de792d3a218e4ad7a68fa4027385  guix-build-cc8dff5f8ff8/output/x86_64-w64-mingw32/bitcoin-cc8dff5f8ff8-win64.zip
  ```

ACKs for top commit:
  gruve-p:
    ACK cc8dff5f8f
  hebasto:
    ACK cc8dff5f8f
  jarolrod:
    ACK cc8dff5f8f

Tree-SHA512: d34a4c3e84efe9cade6de9d7ba5aafc0a5c6efce83e313552a248f2b1eb49dc032a50cf4f1c7eb4767e754a7b67f75a129be19e36a892776b16001c31f6725be
2022-09-21 11:14:36 +01:00
fanquake
97f865bb76 Merge bitcoin/bitcoin#25989: init: abort if i2p/cjdns are chosen via -onlynet but are unreachable
68209a7b5c rpc: make addpeeraddress work with cjdns addresses (Martin Zumsande)
a8a9ed67cc init: Abort if i2p/cjdns are chosen via -onlynet but unreachable (Martin Zumsande)

Pull request description:

  If the networks i2p / cjdns are chosen via `-onlynet` but the user forgot to provide `-i2psam` / `-cjdnsreachable`, no outbound connections will be made - it would be nice to inform the user about that.
  The solution proposed here mimics existing behavior for `-onlynet=onion` and non-specified `-onion`/`-proxy` where we already abort with an InitError - if reviewers would prefer to just print a warning, please say so.

  The second commit adds CJDNS support to the debug-only `addpeeraddress` RPC allowing to add CJDNS addresses to addrman for testing and debug purposes. (if `-cjdnsreachable=1`)

  This is the result of an [IRC discussion](https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2022-09-01#848066;) with vasild.

ACKs for top commit:
  vasild:
    ACK 68209a7b5c
  dergoegge:
    ACK 68209a7b5c

Tree-SHA512: 6db9787f01820190f14f90a0b39e4206603421eb7521f792879094d8bbf4d4d0bfd70665eadcc40994ac7941a15ab5a8d65c4779fba5634c0e6fa66eb0972b8d
2022-09-21 11:00:47 +01:00
MacroFake
7184fb866f Merge bitcoin/bitcoin#26143: test: wait for the expected basic block filter index in interface_rest
aee29c5d9e test: wait for the expected basic block filter index in `interface_rest` (brunoerg)

Pull request description:

  Fixes #26098

  Wait for the expected 'basic block filter index' to not cause issues when calling `/blockfilterheaders/basic/`, like:
  9bd842a592/src/rest.cpp (L423-L424)

Top commit has no ACKs.

Tree-SHA512: 36abe28a76cd01ce7ac1ae9258ce1a9a5473d985c498d915c1130256bc800d0d1207708a195b78bbcb00478ec9c373a2fbfeb26c1fddeb35abe8b253e0308058
2022-09-21 09:55:07 +02:00
Andrew Chow
9e2a2b88d5 Merge bitcoin/bitcoin#26132: wallet: Fix nNextResend data race in ResubmitWalletTransactions
fad61573ed Fix nNextResend data race in ResubmitWalletTransactions (MacroFake)

Pull request description:

  Now that `ResubmitWalletTransactions` is called from more than one thread, it is no longer thread-safe.

  Introduced in 5291933fed.

ACKs for top commit:
  achow101:
    ACK fad61573ed
  jonatack:
    ACK fad61573ed
  stickies-v:
    However, I think the current data race UB fix in fad61573e is the most critical to get into v24, so: ACK fad61573e - but open to further improvements.

Tree-SHA512: 54da2ed1c5f44e33588ac1d21ce26908fcf0bfe785c28ba8f6a479389b5ab7a0b32b016d4c482a2ccb405e0686efb61ffe23e427f5e589dc7d2b3c7469978977
2022-09-20 18:48:06 -04:00
brunoerg
aee29c5d9e test: wait for the expected basic block filter index in interface_rest 2022-09-20 14:10:34 -03:00
MacroFake
9bd842a592 Merge bitcoin/bitcoin#26127: test: check that bumping tx with already spent coin fails
74eb194f81 test: check that bumping tx with already spent coin fails (Sebastian Falbesoner)

Pull request description:

  This PR adds missing coverage for the `bumpfee` RPC,  for the case that a wallet transaction is passed with an input that is already spent:
  0b02ce914e/src/wallet/feebumper.cpp (L182-L186)

  This is achieved by simply creating a transaction with a wallet and then mining it (I'm not aware of any other scenario how this could be achieved). Additionally, two RPC throw checks are changed in the test to be more specific:
  0b02ce914e/src/wallet/feebumper.cpp (L42-L45)
  0b02ce914e/src/wallet/feebumper.cpp (L47-L50)

ACKs for top commit:
  glozow:
    ACK 74eb194f81

Tree-SHA512: 487d0e30a7cc5e2a5f63424ab6aed2963e05e47e2649fb1ad2289c4b48ad488f2dae5c27bf50e532e7eb2f2f5bf0340ed7dda985d14473f31dec0d757bb56324
2022-09-20 18:44:35 +02:00
Andrew Chow
fc4017552c Merge bitcoin/bitcoin#26116: rpc: Allow importmulti watchonly imports with locked wallet
2c03465dfa test: Test watchonly imports with passphrase-locked wallet (Aurèle Oulès)
1fcf9e6e81 rpc: Allow importmulti watchonly imports with locked wallet (Aurèle Oulès)

Pull request description:

  Allows watch-only imports on locked wallets with `importmulti`.
  Also adds a test.

  Fixes #17867.

ACKs for top commit:
  achow101:
    ACK 2c03465dfa
  kristapsk:
    re-ACK 2c03465dfa
  theStack:
    re-ACK 2c03465dfa

Tree-SHA512: 9978d6e59a230c0d160efd312c671cf59458797387d6622b6bf5c9e0681c1fcfebedb3d834fa9314dc5a1eda97e3295696352eacbeab9b43a46b942990087035
2022-09-20 12:00:02 -04:00
MacroFake
faeea28753 test: Avoid race in disconnect_nodes helper 2022-09-20 15:48:05 +02:00
fanquake
5b6f0f31fa Merge bitcoin/bitcoin#26036: net: add NetEventsInterface::g_msgproc_mutex
d575a675cc net_processing: add thread safety annotation for m_highest_fast_announce (Anthony Towns)
0ae7987f68 net_processing: add thread safety annotations for PeerManagerImpl members accessed only via the msgproc thread (Anthony Towns)
a66a7ccb82 net_processing: add thread safety annotations for Peer members accessed only via the msgproc thread (Anthony Towns)
bf12abe454 net: drop cs_sendProcessing (Anthony Towns)
1e78f566d5 net: add NetEventsInterface::g_msgproc_mutex (Anthony Towns)

Pull request description:

  There are many cases where we assume message processing is single-threaded in order for how we access node-related memory to be safe. Add an explicit mutex that we can use to document this, which allows the compiler to catch any cases where we try to access that memory from other threads and break that assumption.

ACKs for top commit:
  MarcoFalke:
    review ACK d575a675cc 📽
  dergoegge:
    Code review ACK d575a675cc
  w0xlt:
    ACK d575a675cc
  vasild:
    ACK d575a675cc modulo the missing runtime checks

Tree-SHA512: b886d1aa4adf318ae64e32ccaf3d508dbb79d6eed3f1fa9d8b2ed96f3c72a3d38cd0f12e05826c9832a2a1302988adfd2b43ea9691aa844f37d8f5c37ff20e05
2022-09-20 14:18:23 +01:00
fanquake
eb6026b90f build: remove strings.h from header checks
We don't include strings.h anywhere.

This is also already checked for by autoconf, so us checking for it just
means a 3rd existence check during ./configure.
2022-09-20 12:06:36 +01:00
MacroFake
fad61573ed Fix nNextResend data race in ResubmitWalletTransactions 2022-09-20 11:49:57 +02:00
MacroFake
71ac70d877 Merge bitcoin/bitcoin#26095: script: bump codespell to 2.2.1, update ignored words and fix spelling
b6a65568df Fix issues identified by codespell 2.2.1 and update ignored words (Jon Atack)
8f2010de6e Bump codespell version to 2.2.1 (Jon Atack)

Pull request description:

  as well as one in `test/lint/lint-locale-dependence.py` not seen by the spelling linter.

  Can be tested locally by running `test/lint/lint-spelling.py` on this branch versus on master and by checking the CI linter result.

ACKs for top commit:
  satsie:
    ACK b6a65568df

Tree-SHA512: ab4ba029a9a5de5926fa5d336bd3b21245acf0649c6aa69a48c223bd22327e13beb32e970f66f54db58cd318731b643e1c7ace9a89776ed2a069cddc02363b71
2022-09-20 11:22:22 +02:00
fanquake
cb075d245e doc: add historical 0.21.2 release notes 2022-09-20 09:30:10 +01:00
fanquake
699f3429c6 doc: add historical 0.20.2 release notes 2022-09-20 09:30:04 +01:00
MacroFake
d76a423809 Merge bitcoin/bitcoin#26067: util: improve bitcoin-wallet exit codes
fa2b8ae0a2 util: improve bitcoin-wallet exit codes (MacroFake)

Pull request description:

  Refactors `bitcoin-wallet` so that it doesn't return a non-zero exit code by default, and makes the option handling more inline with the other binaries. i.e outputting `Error: too few parameters` if you don't pass any options.

  Fixing this means we can check the process output in `gen-manpages.py`; which addresses the remaining [review comment](https://github.com/bitcoin/bitcoin/pull/24263#discussion_r806126705) from #24263.

Top commit has no ACKs.

Tree-SHA512: 80bd8098faefb4401ca1e4d49937ef6c960cf60ce0e7fb9dc38904fbc2fd92e319ec04570381da84943b7477845bf6be00e977f4c0451b247a6698662ce8f1bf
2022-09-20 09:54:04 +02:00
James O'Beirne
9d14f27bdd log: log RPC port on startup
I just spent a few hours trying to figure out why "18444" wasn't getting
me to regtest's RPC server. I'm not the sharpest tool in the shed, but I
was maybe understandably confused because "Bound to
127.0.0.1:18445" appears in the logs, which I assumed was the P2P port.

This change logs the RPC listening address, which seems like a basic
piece of information that shouldn't be buried in debug logs.
2022-09-19 21:43:26 -04:00
Luke Dashjr
a60d9eb9e6 Bugfix: Wallet: Lock cs_wallet for SignMessage
cs_desc_main is typically locked within scope of a cs_wallet lock, but:

CWallet::IsLocked locks cs_wallet
...called from DescriptorScriptPubKeyMan::GetKeys
...called from DescriptorScriptPubKeyMan::GetSigningProvider which locks cs_desc_main first, but has no access to cs_wallet
...called from DescriptorScriptPubKeyMan::SignMessage
...called from CWallet::SignMessage which can access and lock cs_wallet

Resolve the out of order locks by grabbing cs_wallet in CWallet::SignMessage first
2022-09-20 00:46:27 +00:00
Sebastian Falbesoner
74eb194f81 test: check that bumping tx with already spent coin fails 2022-09-19 20:01:49 +02:00
glozow
0b02ce914e Merge bitcoin/bitcoin#26124: docs: Add 371 to bips.md
d3d6a18f71 docs: Add 371 to bips.md (Andrew Chow)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK d3d6a18f71

Tree-SHA512: dc25742c4fe46d4bb80023802823e61b8081964c5ccd6627dcbe531727e0442926b402cf909937ccf5713a0d0f917cb0eab2c743330afb57216b54325d7431f0
2022-09-19 17:55:09 +01:00
Andrew Chow
d3d6a18f71 docs: Add 371 to bips.md 2022-09-19 12:20:09 -04:00
MacroFake
9843a8c1f3 Merge bitcoin/bitcoin#26119: doc: Move -permitbaremultisig to the relay help category
faf5bb87da doc: Move -permitbaremultisig to the relay help category (MacroFake)

Pull request description:

  This transaction relay setting doesn't have anything to do with establishing p2p connections.

ACKs for top commit:
  glozow:
    concept ACK faf5bb87da
  luke-jr:
    utACK faf5bb87da, but (nit) prefer to keep it on a single line

Tree-SHA512: 436fcff5191c346fe16b3208411886e3239fb5819322673a45cf0c0f9a3070563d312da8bb5d5f6060b36d305e59e5b58928526d5042cb3dac29ce7740d17c1c
2022-09-19 18:12:34 +02:00
MacroFake
6e11e20340 Merge bitcoin/bitcoin#26121: build: Bump version to 24.99
0567b5650e build: Bump version to 24.99 (fanquake)

Pull request description:

  24.x has been branched off: https://github.com/bitcoin/bitcoin/tree/24.x.

  On the master branch, bump to 24.99 (pre-25.0).

ACKs for top commit:
  theStack:
    ACK 0567b5650e

Tree-SHA512: 5207363c707d9fe2b392cafc823fd31b99f06f87b0ad0a20841c78fd6e3ad6f030d9ebf995b1ab3b3843a1dca7f86e65889120ee0f490cb80d175de666f5ff75
2022-09-19 18:10:12 +02:00
fanquake
0567b5650e build: Bump version to 24.99
On the master branch, bump to 24.99 (pre-25.0).
2022-09-19 16:48:08 +01:00
Martin Zumsande
68209a7b5c rpc: make addpeeraddress work with cjdns addresses
This allows us to add cjdns addresses to addrman for
testing and debug purposes (if -cjdnsreachable is true)
2022-09-19 11:06:43 -04:00
Martin Zumsande
a8a9ed67cc init: Abort if i2p/cjdns are chosen via -onlynet but unreachable
...because -i2psam or -cjdnsreachable are not provided.
This mimics existing behavior for -onlynet=onion and non-specified proxy.
2022-09-19 11:06:43 -04:00
MacroFake
faf5bb87da doc: Move -permitbaremultisig to the relay help category 2022-09-19 14:09:34 +02:00
Aurèle Oulès
2c03465dfa test: Test watchonly imports with passphrase-locked wallet 2022-09-19 13:54:55 +02:00
MacroFake
fabf1cdb20 Use steady clock for bench logging 2022-09-19 11:51:34 +02:00
MacroFake
faed342a23 scripted-diff: Rename time symbols
-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ':(exclude)src/versionbits.cpp') ; }

 ren nStart                 time_start
 ren nTimeStart             time_start
 ren nTimeReadFromDiskTotal time_read_from_disk_total
 ren nTimeConnectTotal      time_connect_total
 ren nTimeFlush             time_flush
 ren nTimeChainState        time_chainstate
 ren nTimePostConnect       time_post_connect
 ren nTimeCheck             time_check
 ren nTimeForks             time_forks
 ren nTimeConnect           time_connect
 ren nTimeVerify            time_verify
 ren nTimeUndo              time_undo
 ren nTimeIndex             time_index
 ren nTimeTotal             time_total
 ren nTime1                 time_1
 ren nTime2                 time_2
 ren nTime3                 time_3
 ren nTime4                 time_4
 ren nTime5                 time_5
 ren nTime6                 time_6

 ren nBlocksTotal num_blocks_total

 # Newline after semicolon
 perl -0777 -pi -e 's/; time_connect_total/;\n        time_connect_total/g' src/validation.cpp
 perl -0777 -pi -e 's/; time_/;\n    time_/g'                               src/validation.cpp

-END VERIFY SCRIPT-
2022-09-19 10:45:49 +02:00
Aurèle Oulès
1fcf9e6e81 rpc: Allow importmulti watchonly imports with locked wallet 2022-09-17 21:38:55 +02:00
furszy
58b7df3caa wallet: AvailableCoins, simplify output script type acquisition 2022-09-17 10:29:30 -03:00
yancy
81d4a2b14f refactor: Move feerate comparison invariant outside of the loop 2022-09-17 10:07:51 +02:00
yancy
365aca4045 refactor: Simplify feerate comparison statement 2022-09-16 14:29:05 +02:00
furszy
e68d380797 rpc: remove unneeded RPCTypeCheckArgument checks
No-behavior change.

Since #25629, we check the univalue type internally.
2022-09-15 10:45:18 -03:00
furszy
55566630c6 rpc: treat univalue type check error as RPC_TYPE_ERROR, not RPC_MISC_ERROR
By throwing a custom exception from `Univalue::checkType` (instead of a plain
std::runtime_error) and catching it on the RPC server request handler.

So we properly return RPC_TYPE_ERROR (-3) on arg type errors and
not the general RPC_MISC_ERROR (-1).
2022-09-15 10:24:53 -03:00
Jon Atack
b6a65568df Fix issues identified by codespell 2.2.1 and update ignored words
and also fix spelling in test/lint/lint-locale-dependence.py not caught by the
spelling linter and fix up a paragraph we are touching here in test/README.md.
2022-09-15 13:03:40 +02:00
Jon Atack
8f2010de6e Bump codespell version to 2.2.1 2022-09-15 12:53:32 +02:00
Anthony Towns
d575a675cc net_processing: add thread safety annotation for m_highest_fast_announce 2022-09-15 20:28:55 +10:00
Anthony Towns
0ae7987f68 net_processing: add thread safety annotations for PeerManagerImpl members accessed only via the msgproc thread 2022-09-15 20:28:55 +10:00
Anthony Towns
a66a7ccb82 net_processing: add thread safety annotations for Peer members accessed only via the msgproc thread 2022-09-15 20:28:55 +10:00
fanquake
ff7c81f63a build: remove duplicate / unneeded libs from bench_bitcoin
EVENT_*_LIBS are already in LDADD.
Move wallet libs into the wallet conditional, similar to zmq.
2022-09-15 10:29:12 +01:00
Anthony Towns
bf12abe454 net: drop cs_sendProcessing
SendMessages() is now protected g_msgproc_mutex; so this additional
per-node mutex is redundant.
2022-09-15 14:44:42 +10:00
Anthony Towns
1e78f566d5 net: add NetEventsInterface::g_msgproc_mutex
There are many cases where we assume message processing is
single-threaded in order for how we access node-related memory to be
safe. Add an explicit mutex that we can use to document this, which allows
the compiler to catch any cases where we try to access that memory from
other threads and break that assumption.
2022-09-15 14:44:38 +10:00
Vasil Dimov
9d7ae4b66c sync: remove unused template parameter from ::UniqueLock
The template parameter `typename Base = typename Mutex::UniqueLock` is
not used, so remove it. Use internally defined type `Base` to avoid
repetitions of `Mutex::UniqueLock`.
2022-09-14 14:17:09 +02:00
Vasil Dimov
11c190e3f1 sync: simplify MaybeCheckNotHeld() definitions by using a template
Reduce 4 of the `MaybeCheckNotHeld()` definitions to 2 by using a
template. This also makes the function usable for other
[BasicLockable](https://en.cppreference.com/w/cpp/named_req/BasicLockable)
types.
2022-09-14 14:17:01 +02:00
James O'Beirne
bf95976061 doc: add note about snapshot chainstate init 2022-09-13 13:31:37 -04:00
James O'Beirne
e4d7995286 test: add testcases for snapshot initialization 2022-09-13 13:31:23 -04:00
James O'Beirne
cced4e7336 test: move-only-ish: factor out LoadVerifyActivateChainstate()
in TestingSetup(). This is used in the following commit to test
reinitializing chainstates after snapshot validation and cleanup.

Best reviewed with `git diff --color-moved=dimmed-zebra`.
2022-09-13 13:31:23 -04:00
James O'Beirne
51fc9241c0 test: allow on-disk coins and block tree dbs in tests
Used when testing cleanup of on-disk chainstate data for snapshot
testcases. Also necessary for simulating node restart in .cpp tests.
2022-09-13 13:31:23 -04:00
James O'Beirne
3c361391b8 test: add reset_chainstate parameter for snapshot unittests
This CreateAndActivateUTXOSnapshot parameter is necessary once we
perform snapshot completion within ABC, since the existing UpdateTip
test will fail because the IBD chain that has generated the snapshot
will exceed the base of the snapshot.

Being able to test snapshots being loaded into a mostly-uninitialized
datadir allows for more realistic unittest scenarios.
2022-09-13 13:31:21 -04:00
James O'Beirne
00b357c215 validation: add ResetChainstates()
Necessary for the following test commit.
2022-09-13 13:30:28 -04:00
James O'Beirne
3a29dfbfb2 move-only: test: make snapshot chainstate setup reusable
For use in next commit.

Most easily reviewed with
`--color-moved=dimmed_zebra --color-moved-ws=allow-indentation-change`.
2022-09-13 13:30:25 -04:00
James O'Beirne
8153bd9247 blockmanager: avoid undefined behavior during FlushBlockFile
If we call FlushBlockFile() without having intitialized the block index
with LoadBlockIndexDB(), we may be indexing into an empty vector.

Specifically this is an issue when we call MaybeRebalanceCaches() during
chainstate init before the block index has been loaded, which calls
FlushBlockFile().

Also add an assert to avoid undefined behavior.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2022-09-13 13:30:25 -04:00
James O'Beirne
ad67ff377c validation: remove snapshot datadirs upon validation failure
If a UTXO snapshot fails to validate, don't leave the resulting datadir
on disk as this will confuse initialization on next startup and we'll
get an assertion error.
2022-09-13 13:30:25 -04:00
James O'Beirne
34d1590331 add utilities for deleting on-disk leveldb data
Used in later commits to remove leveldb directories for
- invalid snapshot chainstates, and
- background-vaildation chainstates that have finished serving their
  purpose.
2022-09-13 13:30:25 -04:00
James O'Beirne
252abd1e8b init: add utxo snapshot detection
Add functionality for activating a snapshot-based chainstate if one is
detected on-disk.

Also cautiously initialize chainstate cache usages so that we don't
somehow blow past our cache allowances during initialization, then
rebalance at the end of init.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2022-09-13 13:30:14 -04:00
James O'Beirne
f9f1735f13 validation: rename snapshot chainstate dir
This changes the snapshot's leveldb chainstate dir name from
`chainstate_[blockhash]` to `chainstate_snapshot`. This simplifies
later logic that loads snapshot data, and enforces the limitation
of a single snapshot at any given time.

Since we still need to persis the blockhash of the base block, we
write that out to a file (`chainstate_snapshot/base_blockhash`) for
later use during initialization, so that we can reinitialize the
snapshot chainstate.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2022-09-13 13:30:12 -04:00
James O'Beirne
d14bebf100 db: add StoragePath to CDBWrapper/CCoinsViewDB
This is used in subsequent commits. It allows us to clean up UTXO
snapshot chainstate after background validation completes.
2022-09-13 12:38:06 -04:00
MacroFake
fa2c72dda0 rpc: Set RPCArg options with designated initializers 2022-09-13 18:37:15 +02:00
fanquake
5b2529b269 build: fix depends bdb compilation for BSDs
Currently, building bdb for *BSD HOSTs in depends fails with:
```bash
libtool: compile:  clang -m64 -c -I. -I../dist/./.. -I/home/ubuntu/bitcoin/depends/x86_64-unknown-freebsd/include -D_THREAD_SAFE -pipe -std=c11 -O2 -Wno-error=implicit-function-declaration -Wno-error=format-security ../dist/./../mutex/mut_tas.c  -fPIC -DPIC -o mut_tas.o
In file included from ../dist/./../mutex/mut_tas.c:11:
In file included from ./db_int.h:884:
In file included from ../dist/./../dbinc/mutex.h:15:
../dist/./../dbinc/mutex_int.h:932:2: error: unknown type name 'pthread_rwlock_t'
        MUTEX_FIELDS                    /* Opaque thread mutex structures. */
        ^
../dist/./../dbinc/mutex_int.h:65:3: note: expanded from macro 'MUTEX_FIELDS'
                pthread_rwlock_t rwlock;        /* Read/write lock */   \
                ^
1 error generated.
```

Defining `_XOPEN_SOURCE` >=600 fixes access to the missing
`pthread_rwlock_t` definitions.
2022-09-13 12:13:17 +01:00
MacroFake
fa2b8ae0a2 util: improve bitcoin-wallet exit codes 2022-09-12 13:11:18 +02:00
sinetek
5669afb80e fs: drop old WSL1 hack. 2022-09-09 22:07:17 +02:00
w0xlt
b8b59ff9fe gui: update the screen after loading wallet 2022-09-07 23:26:35 -03:00
MacroFake
fa4ba04c15 fuzz: Remove no-op call to get() 2022-09-06 14:38:19 +02:00
yancy
b942c94d15 test: Change coinselection parameter location to make tests independent 2022-09-06 13:56:30 +02:00
Sjors Provoost
f39d9269eb rpc: warn that nodes ignore requests for old stale blocks
This is an anti-fingerprinting measure. See BlockRequestAllowed in net_processing.

It has been around since 2014, but alternative clients might still serve these blocks.

See also: d8b4b49667, 85da07a5a0, a2be3b66b5, 3788a8479b
2022-09-06 11:22:56 +02:00
MacroFake
fa642286b8 fuzz: Avoid timeout in bitdeque fuzz target 2022-09-05 15:38:33 +02:00
brunoerg
cca4f82b82 test: add coverage for rpc error when trying to rescan beyond pruned data 2022-09-05 10:21:41 -03:00
MacroFake
1111c7e3f1 univalue: Avoid std::string copies 2022-09-05 14:24:14 +02:00
fanquake
d216d714aa Revert "build: Use Homebrew's sqlite package if it is available"
This reverts ee7b84e63c from #20527.
This change was made without any rationale, maybe other than a brew
installed version might be newer, and that's "better". However when
building from source on macOS, it just results in drastically worse
perofrmance, and results in issues / confusions like #25724.

Resolves the "build from source" portion of #25724. Building from
depends is still not ideal, however I have some other changes that might
help improve things in that case.

The difference in performance can be observed using the example from
https://github.com/bitcoin/bitcoin/issues/25724#issuecomment-1213554922,
but minified to only 10 descriptors. i.e:
```bash
time src/bitcoin-cli createwallet speedy true
time src/bitcoin-cli importdescriptors '[
  {"desc":"raw(00145846369f3d6ba366d6f5a903fb5cf4dca3763c0e)#k9wh6v62","timestamp":"now"},
  {"desc":"raw(001420800aabf13f3a4c4ce3ce4c66cecf1d17f21a6e)#6m0hlfh4","timestamp":"now"},
  {"desc":"raw(0014c6bf9715e06d73ebf9b3b02d5cc48d24d8bbabc1)#wyavh36r","timestamp":"now"},
  {"desc":"raw(00141ba7807b3f46af113beaea5c698428ce7138cd8a)#jctdsups","timestamp":"now"},
  {"desc":"raw(00140c1bd27f10fff01b36ddf3c1febaa1acff19b080)#9s6nc3pk","timestamp":"now"},
  {"desc":"raw(00141226e31987e4bc2e63c0ee12908f675e40464b20)#9pp7qm39","timestamp":"now"},
  {"desc":"raw(0014f73f149f7503960a5e849c6ee7a8a8c336f631cb)#qtkxv9fc","timestamp":"now"},
  {"desc":"raw(0014c8ccb4d81ffc769fc5fdd8d7eed69b0e0cae5749)#hn39qayv","timestamp":"now"},
  {"desc":"raw(001498565aead2d67a22a6021d55210f2a917fc22169)#6ar3vwsx","timestamp":"now"},
  {"desc":"raw(001403013248ac0cd9eabe176cad162cda2a19f771e1)#4m47mukd","timestamp":"now"}
]'
```

Running master, when building from souce and using brew installed
sqlite, this takes ~3.4s. With this PR, the same operation takes ~0.3s.
2022-09-02 14:51:27 +01:00
yancy
fb1c6c14c1 test: Remove redundant test 2022-08-31 14:20:37 +02:00
w0xlt
1b77db2653 test: add ismine test for descriptor scriptpubkeyman 2022-08-27 18:59:13 -03:00
fanquake
65471008e0 depends: libnatpmp 07004b97cf691774efebe70404cf22201e4d330d
This pulls in two changes I've upstreamed:
Support for pkg-config: https://github.com/miniupnp/libnatpmp/pull/19
Suppressing a deprecation warning: https://github.com/miniupnp/libnatpmp/pull/28

Somewhat related to #22644.
2022-08-24 16:03:21 +01:00
Vasil Dimov
9cbfe40d8a net: remove useless call to IsReachable() from CConnman::Bind()
`CConnman::Bind()` is called without `BF_EXPLICIT` only when passed
either `0.0.0.0` or `::`. For those addresses `IsReachable()` is always
true (regardless of the `-onlynet=` setting!), meaning that the `if`
condition never evaluates to true.

`IsReachable()` is always true for the "any" IPv4 and IPv6 addresses
because `CNetAddr::GetNetwork()` returns `NET_UNROUTABLE` instead of
`NET_IPV4` or `NET_IPV6` and the network `NET_UNROUTABLE` is always
considered reachable.

It follows that `BF_EXPLICIT` is unnecessary, remove it too.
2022-08-22 14:16:49 +02:00
fanquake
cc8dff5f8f depends: Boost 1.80.0
https://www.boost.org/users/history/version_1_80_0.html
2022-08-19 10:47:30 +01:00
brunoerg
a8250e30f1 doc: add release note about /rest/deploymentinfo 2022-08-16 19:21:51 -03:00
brunoerg
5c96020024 doc: add /deploymentinfo in REST-interface 2022-08-16 19:21:51 -03:00
brunoerg
3e44bee08e test: add coverage for /rest/deploymentinfo 2022-08-16 19:21:51 -03:00
brunoerg
91497031cb rest: add /deploymentinfo 2022-08-16 19:21:46 -03:00
Jeremy Rubin
0652dc53b2 [BugFix]: Do not allow deserializing PSBT with empty PSBT_OUT_TAP_TREE 2022-08-16 15:21:58 -04:00
Andrew Chow
fdb8dc8a5a gui: Show watchonly balance only for Legacy wallets
Descriptor wallets do not have a watchonly balance as wallets are
designated watchonly or not. Thus we should not be displaying the empty
watchonly balance for descriptor wallets.
2022-08-14 15:58:47 -04:00
furszy
bfb9b94ebe wallet: remove duplicate descriptor type check in GetNewDestination 2022-08-12 12:42:20 -03:00
furszy
76b982a4a5 wallet: remove unused nAccountingEntryNumber field 2022-08-12 12:36:07 -03:00
furszy
599ff5adfc wallet: avoid double TopUp() calls on descriptor wallets
Move TopUp() responsibility from the wallet class to each scriptpubkeyman.
So each spkm can decide to call it or not after perform the basic checks
for the new destination request.

Reason:

We were calling it twice in the following flows for descriptor wallets:

A) CWallet::GetNewDestination:
   1) Calls spk_man->TopUp()
   2) Calls spk_man->GetNewDestination() --> which, after the basic script checks, calls TopUp() again.

B) CWallet::GetReservedDestination:
   1) Calls spk_man->TopUp()
   2) Calls spk_man->GetReservedDestination() --> which calls to GetNewDestination (which calls to TopUp again).
2022-08-12 12:36:04 -03:00
MacroFake
fa3ea81c3e refactor: Add LIFETIMEBOUND / -Wdangling-gsl to Assert() 2022-08-02 13:40:42 +02:00
Hennadii Stepanov
6d58117a31 build: Build minisketch test in make check, not in make 2022-07-21 10:42:09 +01:00
Vasil Dimov
b527b54950 net: convert standalone SetSocketNonBlocking() to Sock::SetNonBlocking()
This further encapsulates syscalls inside the `Sock` class.

Co-authored-by: practicalswift <practicalswift@users.noreply.github.com>
2022-07-20 16:26:24 +02:00
Vasil Dimov
29f66f7682 moveonly: move SetSocketNonBlocking() from netbase to util/sock
To be converted to a method of the `Sock` class.
2022-07-20 16:26:24 +02:00
Vasil Dimov
b4bac55679 net: convert standalone IsSelectableSocket() to Sock::IsSelectable()
This makes the callers mockable.
2022-07-20 16:26:23 +02:00
Vasil Dimov
5db7d2ca0a moveonly: move IsSelectableSocket() from compat.h to sock.{h,cpp}
To be converted to a method of the `Sock` class.
2022-07-20 16:26:19 +02:00
Luke Dashjr
e049fd76f0 Bugfix: Check for readlink buffer overflow and handle gracefully
If readlink returns the size of the buffer, an overflow may have (safely) occurred.
Pass a buffer size of MAX_PATH+1 (the size of the actual buffer) to detect this scenario.
2022-07-05 23:44:18 +00:00
Sjors Provoost
796b020c37 wallet: add taproot support to external signer 2022-06-28 17:15:25 +02:00
Hennadii Stepanov
b59b31ae0b build: Drop redundant qt/bitcoin.cpp
This file was included in #9457, but now it is a part of the
BITCOIN_QT_BASE_CPP.
2022-06-12 14:33:16 +02:00
Hennadii Stepanov
d90ad5a42e build: Include qt sources for parsing with extract_strings.py 2022-06-12 14:31:39 +02:00
Hennadii Stepanov
8b8edc25c1 build: Specify native binaries explicitly when building capnp package
From `configure --help`:
  --with-external-capnp   use the system capnp binary (or the one specified
                          with $CAPNP) instead of compiling a new one (useful
                          for cross-compiling)
2022-06-09 15:56:52 +02:00
Hennadii Stepanov
a413595c37 build: Fix capnp package build for Android 2022-06-09 15:56:52 +02:00
Fabian Jahr
5826bf546e test: Add test for getblockfrompeer on syncing pruned nodes 2022-06-06 01:36:16 +02:00
Fabian Jahr
7fa851fba8 rpc: Pruned nodes can not fetch unsynced blocks
While a node is still catching up to the tip that it is aware of via the headers, the user can currently use  to fetch blocks close to the tip. These blocks are stored in the current block/rev file which otherwise contains blocks the node is receiving as part of the syncing process.

This creates a problem for pruned nodes: The files containing a fetched block are not pruned during syncing because they contain a block close to the tip. This means the entire file will not be pruned until the tip have moved on far enough from the fetched block. In extreme cases with heavy pruning (550) and multiple blocks being fetched this could mean that the disc usage far exceeds what the user expects, potentially running out of space.
2022-06-06 01:34:11 +02:00
mruddy
bcb0cacac2 reindex, log, test: fixes #21379
This fixes a blk file size calculation made during reindex that results in increased blk file malformity.
The fix is to avoid double counting the size of the serialization header during reindex.
This adds a unit test to reproduce the bug before the fix and to ensure that it does not recur.
These changes include a log message change also so as to not be as alarming. This is a common and recoverable
data corruption. These messages can now be filtered by the debug log reindex category.
2022-05-07 07:11:29 -04:00
Sjors Provoost
2c07cfacd1 gui: bumpfee signer support
Specifically this enables the Send button in the fee bump dialog for wallets with external signer support. Similar to 2efdfb88aa.
2022-04-25 18:14:28 +02:00
Sjors Provoost
7e02a33297 rpc: bumpfee signer support 2022-04-25 18:13:23 +02:00
Sjors Provoost
304ece9945 rpc: document bools in FillPSBT() calls 2022-04-25 17:41:26 +02:00
Vasil Dimov
8173f160e0 style: rename variables to match coding style
Rename the variables that were touched by the previous commit (split
logical from style changes).

minIncrementalFee -> min_incremental_fee
minFeeLimit -> min_fee_limit
bucketBoundary -> bucket_boundary
feeset -> fee_set
FeeFilterRounder::feeset -> FeeFilterRounder::m_fee_set
2022-04-18 10:40:07 +02:00
Vasil Dimov
8b4ad203d0 fees: make FeeFilterRounder::feeset const
It is only set in the constructor, thus improve readability by marking
it as `const` and setting it from the initializer list using a helper
function to derive its value.

The idea was suggested by Anthony Towns <aj@erisian.com.au> in
https://github.com/bitcoin/bitcoin/pull/19268#discussion_r439929792
2022-04-18 10:40:06 +02:00
Vasil Dimov
e7a5bf6be7 fees: make the class FeeFilterRounder thread-safe
So that its methods can be called concurrently by different threads on
the same object. Currently it has just one method (`round()`).

Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2022-04-18 10:40:05 +02:00
Hennadii Stepanov
a3a2bd9e8a ci: Drop no longer needed package-specific flags 2022-04-16 08:59:13 +02:00
Hennadii Stepanov
071eef1e97 build: Propagate user-defined flags to host packages 2022-04-16 08:57:22 +02:00
Chris Geihsler
e899d4ca6f init: limit bip30 exceptions to coinbase txs
Co-authored-by: James O'Beirne <james.obeirne@pm.me>
2022-04-14 12:04:33 -04:00
Chris Geihsler
511eb7fdea Ignore problematic blocks in DisconnectBlock
When using checklevel=4, block verification fails because of duplicate coinbase transactions
involving blocks 91812 and 91722. There was already a check in place for ConnectBlock to
ignore the problematic blocks, but DisconnectBlock did not contain a similar check.

This change ignores the blocks where these inconsistencies surface so
that block verification will succeed at checklevel=4.
2022-04-13 23:05:40 -04:00
Sebastian Falbesoner
c456302d42 doc: minor improvements in getutxos REST endpoint synopsis
Describing an optional sub-path as <checkmempool> in the synopsis could
be misleading as the angle brackets normally indicate that the field has
to be replaced a custom value. Clarify that by showing two variants
instead, similar to the block endpoint with the notxdetails option.

Further improvements:

- uppercase <TXID> and <N>, to match the description of the other endpoints
- s/getutxo command/getutxos endpoint/
- describe what the checkmempool option does
- s/serialisation/serialization/ (the US spelling is more dominant than
  the UK spelling in the project, and there is indeed no other instance
  of the string "serialis*" in the source tree, except once in a release
  note)
- link to BIP64 within the text instead of only showing bare URL
- mention that BIP64 is only relevant for bin and hex output formats
- show two endpoint formats of the block section as list
2022-03-30 17:27:03 +02:00
James O'Beirne
0f40d65321 refactor: remove duplicate code from BlockAssembler 2022-02-16 21:17:21 -05:00
Pasta
75347236f2 docs: document c-style cast prohibition 2022-02-02 00:31:47 +07:00
brunoerg
0811cbfc28 doc: add info about status code 404 for some rest endpoints 2022-01-24 08:39:42 -03:00
Luke Dashjr
98868633d1 Bugfix: configure: bitcoin-{cli,tx,util} don't need UPnP, NAT-PMP, or ZMQ 2022-01-12 22:20:13 +00:00
Jeffrey Czyz
f36d1d5b89 Use void* throughout support/lockedpool.h
Replace uses of char* with void* in Arena's member variables. Instead,
cast to char* where needed in the implementation.

Certain compiler environments disallow std::hash<char*> specializations
to prevent hashing the pointer's value instead of the string contents.
Thus, compilation fails when std::unordered_map is keyed by char*.

Explicitly using void* is a workaround in such environments. For
consistency, void* is used throughout all member variables similarly to
the public interface.
2019-11-20 18:19:13 -08:00
1685 changed files with 248949 additions and 76709 deletions

View File

@@ -1,20 +1,50 @@
env: # Global defaults
CIRRUS_CLONE_DEPTH: 1
PACKAGE_MANAGER_INSTALL: "apt-get update && apt-get install -y"
MAKEJOBS: "-j10"
TEST_RUNNER_PORT_MIN: "14000" # Must be larger than 12321, which is used for the http cache. See https://cirrus-ci.org/guide/writing-tasks/#http-cache
CI_FAILFAST_TEST_LEAVE_DANGLING: "1" # Cirrus CI does not care about dangling process and setting this variable avoids killing the CI script itself on error
CCACHE_SIZE: "200M"
CI_FAILFAST_TEST_LEAVE_DANGLING: "1" # Cirrus CI does not care about dangling processes and setting this variable avoids killing the CI script itself on error
CCACHE_MAXSIZE: "200M"
CCACHE_DIR: "/tmp/ccache_dir"
CCACHE_NOHASHDIR: "1" # Debug info might contain a stale path if the build dir changes, but this is fine
cirrus_ephemeral_worker_template_env: &CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
DANGER_RUN_CI_ON_HOST: "1" # Containers will be discarded after the run, so there is no risk that the ci scripts modify the system
persistent_worker_template_env: &PERSISTENT_WORKER_TEMPLATE_ENV
RESTART_CI_DOCKER_BEFORE_RUN: "1"
persistent_worker_template: &PERSISTENT_WORKER_TEMPLATE
persistent_worker: {} # https://cirrus-ci.org/guide/persistent-workers/
# https://cirrus-ci.org/guide/persistent-workers/
#
# It is possible to select a specific persistent worker by label. Refer to the
# Cirrus CI docs for more details.
#
# Generally, a persistent worker must run Ubuntu 23.04+ or Debian 12+.
# Specifically,
# - apt-get is required due to PACKAGE_MANAGER_INSTALL
# - podman-docker-4.1+ is required due to the use of `podman` when
# RESTART_CI_DOCKER_BEFORE_RUN is set and 4.1+ due to the bugfix in 4.1
# (https://github.com/bitcoin/bitcoin/pull/21652#issuecomment-1657098200)
# - The ./ci/ depedencies (with cirrus-cli) should be installed:
#
# ```
# apt update && apt install screen python3 bash podman-docker curl -y && curl -L -o cirrus "https://github.com/cirruslabs/cirrus-cli/releases/latest/download/cirrus-linux-$(dpkg --print-architecture)" && mv cirrus /usr/local/bin/cirrus && chmod +x /usr/local/bin/cirrus
# ```
#
# - There are no strict requirements on the hardware, because having less CPUs
# runs the same CI script (maybe slower). To avoid rare and intermittent OOM
# due to short memory usage spikes, it is recommended to add (and persist)
# swap:
#
# ```
# fallocate -l 16G /swapfile_ci && chmod 600 /swapfile_ci && mkswap /swapfile_ci && swapon /swapfile_ci && ( echo '/swapfile_ci none swap sw 0 0' | tee -a /etc/fstab )
# ```
#
# - To register the persistent worker, open a `screen` session and run:
#
# ```
# RESTART_CI_DOCKER_BEFORE_RUN=1 screen cirrus worker run --labels type=todo_fill_in_type --token todo_fill_in_token
# ```
#
# The following specific types should exist, with the following requirements:
# - small: For an x86_64 machine, recommended to have 2 CPUs and 8 GB of memory.
# - medium: For an x86_64 machine, recommended to have 4 CPUs and 16 GB of memory.
# - noble: For a machine running the Linux kernel shipped with exaclty Ubuntu Noble 24.04. The machine is recommended to have 4 CPUs and 16 GB of memory.
# - arm64: For an aarch64 machine, recommended to have 2 CPUs and 8 GB of memory.
# https://cirrus-ci.org/guide/tips-and-tricks/#sharing-configuration-between-tasks
filter_template: &FILTER_TEMPLATE
@@ -24,41 +54,20 @@ filter_template: &FILTER_TEMPLATE
base_template: &BASE_TEMPLATE
<< : *FILTER_TEMPLATE
merge_base_script:
# Unconditionally install git (used in fingerprint_script) and set the
# default git author name (used in verify-commits.py)
# Unconditionally install git (used in fingerprint_script).
- bash -c "$PACKAGE_MANAGER_INSTALL git"
- git config --global user.email "ci@ci.ci"
- git config --global user.name "ci"
- if [ "$CIRRUS_PR" = "" ]; then exit 0; fi
- git fetch $CIRRUS_REPO_CLONE_URL $CIRRUS_BASE_BRANCH
- git merge FETCH_HEAD # Merge base to detect silent merge conflicts
- git fetch --depth=1 $CIRRUS_REPO_CLONE_URL "pull/${CIRRUS_PR}/merge"
- git checkout FETCH_HEAD # Use merged changes to detect silent merge conflicts
# Also, the merge commit is used to lint COMMIT_RANGE="HEAD~..HEAD"
main_template: &MAIN_TEMPLATE
timeout_in: 120m # https://cirrus-ci.org/faq/#instance-timed-out
ccache_cache:
folder: "/tmp/ccache_dir"
ci_script:
- ./ci/test_run_all.sh
global_task_template: &GLOBAL_TASK_TEMPLATE
<< : *BASE_TEMPLATE
container:
# https://cirrus-ci.org/faq/#are-there-any-limits
# Each project has 16 CPU in total, assign 2 to each container, so that 8 tasks run in parallel
cpu: 2
greedy: true
memory: 8G # Set to 8GB to avoid OOM. https://cirrus-ci.org/guide/linux/#linux-containers
depends_built_cache:
folder: "depends/built"
fingerprint_script: echo $CIRRUS_TASK_NAME $(git rev-list -1 HEAD ./depends)
<< : *MAIN_TEMPLATE
macos_native_task_template: &MACOS_NATIVE_TASK_TEMPLATE
<< : *BASE_TEMPLATE
check_clang_script:
- clang --version
brew_install_script:
- brew install boost libevent qt@5 miniupnpc libnatpmp ccache zeromq qrencode libtool automake gnu-getopt
<< : *MAIN_TEMPLATE
compute_credits_template: &CREDITS_TEMPLATE
@@ -67,284 +76,131 @@ compute_credits_template: &CREDITS_TEMPLATE
use_compute_credits: $CIRRUS_REPO_FULL_NAME == 'bitcoin/bitcoin' && $CIRRUS_PR != ""
task:
name: 'lint [bionic]'
name: 'lint'
<< : *BASE_TEMPLATE
container:
image: ubuntu:bionic # For python 3.6, oldest supported version according to doc/dependencies.md
image: debian:bookworm
cpu: 1
memory: 1G
# For faster CI feedback, immediately schedule the linters
<< : *CREDITS_TEMPLATE
python_cache:
folder: "/python_build"
fingerprint_script: cat .python-version /etc/os-release
unshallow_script:
- git fetch --unshallow --no-tags
lint_script:
- ./ci/lint_run_all.sh
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
task:
name: 'tidy [jammy]'
name: 'tidy'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:jammy
cpu: 2
memory: 5G
# For faster CI feedback, immediately schedule the linters
<< : *CREDITS_TEMPLATE
persistent_worker:
labels:
type: medium
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_tidy.sh"
task:
name: "Win64 native [vs2022]"
<< : *FILTER_TEMPLATE
windows_container:
cpu: 6
memory: 12G
image: cirrusci/windowsservercore:visualstudio2022
timeout_in: 120m
env:
PATH: 'C:\jom;C:\Python39;C:\Python39\Scripts;C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin;%PATH%'
PYTHONUTF8: 1
CI_VCPKG_TAG: '2022.06.16.1'
VCPKG_DOWNLOADS: 'C:\Users\ContainerAdministrator\AppData\Local\vcpkg\downloads'
VCPKG_DEFAULT_BINARY_CACHE: 'C:\Users\ContainerAdministrator\AppData\Local\vcpkg\archives'
CCACHE_DIR: 'C:\Users\ContainerAdministrator\AppData\Local\ccache'
WRAPPED_CL: 'C:\Users\ContainerAdministrator\AppData\Local\Temp\cirrus-ci-build\ci\test\wrapped-cl.bat'
QT_DOWNLOAD_URL: 'https://download.qt.io/official_releases/qt/5.15/5.15.5/single/qt-everywhere-opensource-src-5.15.5.zip'
QT_LOCAL_PATH: 'C:\qt-everywhere-opensource-src-5.15.5.zip'
QT_SOURCE_DIR: 'C:\qt-everywhere-src-5.15.5'
QTBASEDIR: 'C:\Qt_static'
x64_NATIVE_TOOLS: '"C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\VC\Auxiliary\Build\vcvars64.bat"'
QT_CONFIGURE_COMMAND: '..\configure -release -silent -opensource -confirm-license -opengl desktop -static -static-runtime -mp -qt-zlib -qt-pcre -qt-libpng -nomake examples -nomake tests -nomake tools -no-angle -no-dbus -no-gif -no-gtk -no-ico -no-icu -no-libjpeg -no-libudev -no-sql-sqlite -no-sql-odbc -no-sqlite -no-vulkan -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip doc -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtlottie -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquick3d -skip qtquickcontrols -skip qtquickcontrols2 -skip qtquicktimeline -skip qtremoteobjects -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtsvg -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebglplugin -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -no-openssl -no-feature-bearermanagement -no-feature-printdialog -no-feature-printer -no-feature-printpreviewdialog -no-feature-printpreviewwidget -no-feature-sql -no-feature-sqlmodel -no-feature-textbrowser -no-feature-textmarkdownwriter -no-feature-textodfwriter -no-feature-xml'
IgnoreWarnIntDirInTempDetected: 'true'
merge_script:
- git config --global user.email "ci@ci.ci"
- git config --global user.name "ci"
# Windows filesystem loses the executable bit, and all of the executable
# files are considered "modified" now. It will break the following `git merge`
# command. The next two commands make git ignore this issue.
- git config core.filemode false
- git reset --hard
- PowerShell -NoLogo -Command if ($env:CIRRUS_PR -ne $null) { git fetch $env:CIRRUS_REPO_CLONE_URL $env:CIRRUS_BASE_BRANCH; git merge FETCH_HEAD; }
msvc_qt_built_cache:
folder: "%QTBASEDIR%"
reupload_on_changes: false
fingerprint_script:
- echo %QT_DOWNLOAD_URL% %QT_CONFIGURE_COMMAND%
- msbuild -version
populate_script:
- curl -L -o C:\jom.zip http://download.qt.io/official_releases/jom/jom.zip
- mkdir C:\jom
- tar -xf C:\jom.zip -C C:\jom
- curl -L -o %QT_LOCAL_PATH% %QT_DOWNLOAD_URL%
- tar -xf %QT_LOCAL_PATH% -C C:\
- '%x64_NATIVE_TOOLS%'
- cd %QT_SOURCE_DIR%
- mkdir build
- cd build
- '%QT_CONFIGURE_COMMAND% -prefix %QTBASEDIR%'
- jom
- jom install
vcpkg_tools_cache:
folder: '%VCPKG_DOWNLOADS%\tools'
reupload_on_changes: false
fingerprint_script:
- echo %CI_VCPKG_TAG%
- msbuild -version
vcpkg_binary_cache:
folder: '%VCPKG_DEFAULT_BINARY_CACHE%'
reupload_on_changes: true
fingerprint_script:
- echo %CI_VCPKG_TAG%
- type build_msvc\vcpkg.json
- msbuild -version
populate_script:
- mkdir %VCPKG_DEFAULT_BINARY_CACHE%
ccache_cache:
folder: '%CCACHE_DIR%'
install_tools_script:
- choco install --yes --no-progress ccache --version=4.6.1
- choco install --yes --no-progress python3 --version=3.9.6
- pip install zmq
- ccache --version
- python -VV
install_vcpkg_script:
- cd ..
- git clone --quiet https://github.com/microsoft/vcpkg.git
- cd vcpkg
- git -c advice.detachedHead=false checkout %CI_VCPKG_TAG%
- .\bootstrap-vcpkg -disableMetrics
- echo set(VCPKG_BUILD_TYPE release) >> triplets\x64-windows-static.cmake
- .\vcpkg integrate install
- .\vcpkg version
build_script:
- '%x64_NATIVE_TOOLS%'
- cd %CIRRUS_WORKING_DIR%
- ccache --zero-stats --max-size=%CCACHE_SIZE%
- python build_msvc\msvc-autogen.py
- msbuild build_msvc\bitcoin.sln -property:CLToolExe=%WRAPPED_CL% -property:Configuration=Release -maxCpuCount -verbosity:minimal -noLogo
- ccache --show-stats
unit_tests_script:
- src\test_bitcoin.exe -l test_suite
- src\bench_bitcoin.exe > NUL
- python test\util\test_runner.py
- python test\util\rpcauth-test.py
functional_tests_script:
# Increase the dynamic port range to the maximum allowed value to mitigate "OSError: [WinError 10048] Only one usage of each socket address (protocol/network address/port) is normally permitted".
# See: https://docs.microsoft.com/en-us/biztalk/technical-guides/settings-that-can-be-modified-to-improve-network-performance
- netsh int ipv4 set dynamicport tcp start=1025 num=64511
- netsh int ipv6 set dynamicport tcp start=1025 num=64511
# Exclude feature_dbcrash for now due to timeout
- python test\functional\test_runner.py --nocleanup --ci --quiet --combinedlogslen=4000 --jobs=4 --timeout-factor=8 --extended --exclude feature_dbcrash
task:
name: 'ARM [unit tests, no functional tests] [bullseye]'
name: 'ARM, unit tests, no functional tests'
<< : *GLOBAL_TASK_TEMPLATE
arm_container:
image: debian:bullseye
cpu: 2
memory: 8G
persistent_worker:
labels:
type: arm64 # Use arm64 worker to sidestep qemu and avoid a slow CI: https://github.com/bitcoin/bitcoin/pull/28087#issuecomment-1649399453
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_arm.sh"
QEMU_USER_CMD: "" # Disable qemu and run the test natively
task:
name: 'Win64 [unit tests, no gui tests, no boost::process, no functional tests] [jammy]'
name: 'Win64, unit tests, no gui tests, no boost::process, no functional tests'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:jammy
persistent_worker:
labels:
type: small
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_win64.sh"
task:
name: '32-bit + dash [gui] [CentOS 8]'
name: '32-bit CentOS, dash, gui'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: quay.io/centos/centos:stream8
persistent_worker:
labels:
type: small
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
PACKAGE_MANAGER_INSTALL: "yum install -y"
FILE_ENV: "./ci/test/00_setup_env_i686_centos.sh"
task:
name: '[previous releases, uses qt5 dev package and some depends packages, DEBUG] [unsigned char] [buster]'
previous_releases_cache:
folder: "releases"
name: 'previous releases, qt5 dev package and depends packages, DEBUG'
<< : *GLOBAL_TASK_TEMPLATE
<< : *PERSISTENT_WORKER_TEMPLATE
persistent_worker:
labels:
type: small
env:
<< : *PERSISTENT_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_qt5.sh"
task:
name: '[TSan, depends, gui] [jammy]'
name: 'TSan, depends, gui'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:jammy
cpu: 6 # Increase CPU and Memory to avoid timeout
memory: 24G
persistent_worker:
labels:
type: medium
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_tsan.sh"
task:
name: '[MSan, depends] [focal]'
name: 'MSan, depends'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
persistent_worker:
labels:
type: small
timeout_in: 300m # Use longer timeout for the *rare* case where a full build (llvm + msan + depends + ...) needs to be done.
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_msan.sh"
MAKEJOBS: "-j4" # Avoid excessive memory use due to MSan
task:
name: '[ASan + LSan + UBSan + integer, no depends, USDT] [jammy]'
name: 'ASan + LSan + UBSan + integer, no depends, USDT'
enable_bpfcc_script:
# In the image build step, no external environment variables are available,
# so any settings will need to be written to the settings env file:
- sed -i "s|\${CIRRUS_CI}|true|g" ./ci/test/00_setup_env_native_asan.sh
<< : *GLOBAL_TASK_TEMPLATE
# We can't use a 'container' for the USDT interface tests as the CirrusCI
# containers don't have privileges to hook into bitcoind. CirrusCI uses
# Google Compute Engine instances: https://cirrus-ci.org/guide/custom-vms/
# Images can be found here: https://cloud.google.com/compute/docs/images/os-details
compute_engine_instance:
image_project: ubuntu-os-cloud
image: family/ubuntu-2204-lts # when upgrading, check if we can drop "ADD_UNTRUSTED_BPFCC_PPA"
cpu: 4
memory: 12G
persistent_worker:
labels:
type: noble # Must use this specific worker (needed for USDT functional tests)
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
HOME: /root/ # Only needed for compute_engine_instance
FILE_ENV: "./ci/test/00_setup_env_native_asan.sh"
MAKEJOBS: "-j4" # Avoid excessive memory use
task:
name: '[fuzzer,address,undefined,integer, no depends] [jammy]'
name: 'fuzzer,address,undefined,integer, no depends'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:jammy
cpu: 4 # Increase CPU and memory to avoid timeout
memory: 16G
persistent_worker:
labels:
type: medium
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_fuzz.sh"
task:
name: '[multiprocess, i686, DEBUG] [focal]'
name: 'multiprocess, i686, DEBUG'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:focal
cpu: 4
memory: 16G # The default memory is sometimes just a bit too small, so double everything
persistent_worker:
labels:
type: medium
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_i686_multiprocess.sh"
task:
name: '[no wallet, libbitcoinkernel] [bionic]'
name: 'no wallet, libbitcoinkernel'
<< : *GLOBAL_TASK_TEMPLATE
container:
image: ubuntu:bionic
persistent_worker:
labels:
type: small
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_native_nowallet_libbitcoinkernel.sh"
task:
name: 'macOS 10.15 [gui, no tests] [focal]'
<< : *BASE_TEMPLATE
macos_sdk_cache:
folder: "depends/SDKs/$MACOS_SDK"
fingerprint_key: "$MACOS_SDK"
<< : *MAIN_TEMPLATE
container:
image: ubuntu:focal
name: 'macOS-cross 11.0, gui, no tests'
<< : *GLOBAL_TASK_TEMPLATE
persistent_worker:
labels:
type: small
env:
MACOS_SDK: "Xcode-12.2-12B45b-extracted-SDK-with-libcxx-headers"
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_mac.sh"
task:
name: 'macOS 12 native x86_64 [gui, system sqlite] [no depends]'
macos_instance:
# Use latest image, but hardcode version to avoid silent upgrades (and breaks)
image: monterey-xcode-13.3 # https://cirrus-ci.org/guide/macOS
<< : *MACOS_NATIVE_TASK_TEMPLATE
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
CI_USE_APT_INSTALL: "no"
PACKAGE_MANAGER_INSTALL: "echo" # Nothing to do
FILE_ENV: "./ci/test/00_setup_env_mac_native_x86_64.sh"
task:
name: 'ARM64 Android APK [focal]'
<< : *BASE_TEMPLATE
android_sdk_cache:
folder: "depends/SDKs/android"
fingerprint_key: "ANDROID_API_LEVEL=28 ANDROID_BUILD_TOOLS_VERSION=28.0.3 ANDROID_NDK_VERSION=23.2.8568313"
depends_sources_cache:
folder: "depends/sources"
fingerprint_script: git rev-list -1 HEAD ./depends
<< : *MAIN_TEMPLATE
container:
image: ubuntu:focal
env:
<< : *CIRRUS_EPHEMERAL_WORKER_TEMPLATE_ENV
FILE_ENV: "./ci/test/00_setup_env_android.sh"

View File

@@ -1,27 +0,0 @@
<!-- This issue tracker is only for technical issues related to Bitcoin Core.
General bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange at https://bitcoin.stackexchange.com.
For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/.
If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue!
Any report, issue or feature request related to the GUI should be reported at
https://github.com/bitcoin-core/gui/issues/
-->
<!-- Describe the issue -->
<!--- What behavior did you expect? -->
<!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->
<!--- How reliably can you reproduce the issue, what are the steps to do so? -->
<!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->
<!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->
<!-- GUI-related issue? What is your operating system and its version? If Linux, what is your desktop environment and graphical shell? -->
<!-- Any extra information that might be useful in the debugging process. -->
<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

93
.github/ISSUE_TEMPLATE/bug.yml vendored Normal file
View File

@@ -0,0 +1,93 @@
name: Bug report
description: Submit a new bug report.
labels: [bug]
body:
- type: markdown
attributes:
value: |
## This issue tracker is only for technical issues related to Bitcoin Core.
* General bitcoin questions and/or support requests should use Bitcoin StackExchange at https://bitcoin.stackexchange.com.
* For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/.
* If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running `memtest` and observe CPU temperature with a load-test tool such as `linpack` before creating an issue.
----
- type: checkboxes
attributes:
label: Is there an existing issue for this?
description: Please search to see if an issue already exists for the bug you encountered.
options:
- label: I have searched the existing issues
required: true
- type: textarea
id: current-behaviour
attributes:
label: Current behaviour
description: Tell us what went wrong
validations:
required: true
- type: textarea
id: expected-behaviour
attributes:
label: Expected behaviour
description: Tell us what you expected to happen
validations:
required: true
- type: textarea
id: reproduction-steps
attributes:
label: Steps to reproduce
description: |
Tell us how to reproduce your bug. Please attach related screenshots if necessary.
* Run-time or compile-time configuration options
* Actions taken
validations:
required: true
- type: textarea
id: logs
attributes:
label: Relevant log output
description: |
Please copy and paste any relevant log output or attach a debug log file.
You can find the debug.log in your [data dir.](https://github.com/bitcoin/bitcoin/blob/master/doc/files.md#data-directory-location)
Please be aware that the debug log might contain personally identifying information.
validations:
required: false
- type: dropdown
attributes:
label: How did you obtain Bitcoin Core
multiple: false
options:
- Compiled from source
- Pre-built binaries
- Package manager
- Other
validations:
required: true
- type: input
id: core-version
attributes:
label: What version of Bitcoin Core are you using?
description: Run `bitcoind --version` or in Bitcoin-QT use `Help > About Bitcoin Core`
placeholder: e.g. v24.0.1 or master@e1bf547
validations:
required: true
- type: input
id: os
attributes:
label: Operating system and version
placeholder: e.g. "MacOS Ventura 13.2" or "Ubuntu 22.04 LTS"
validations:
required: true
- type: textarea
id: machine-specs
attributes:
label: Machine specifications
description: |
What are the specifications of the host machine?
e.g. OS/CPU and disk type, network connectivity
validations:
required: false

View File

@@ -1,41 +0,0 @@
---
name: Bug report
about: Create a report to help us improve (use this for suspected bugs only, if not sure, open a regular issue below)
title: ''
labels: Bug
assignees: ''
---
<!-- This issue tracker is only for technical issues related to Bitcoin Core.
General bitcoin questions and/or support requests are best directed to the Bitcoin StackExchange at https://bitcoin.stackexchange.com.
For reporting security issues, please read instructions at https://bitcoincore.org/en/contact/.
If the node is "stuck" during sync or giving "block checksum mismatch" errors, please ensure your hardware is stable by running memtest and observe CPU temperature with a load-test tool such as linpack before creating an issue! -->
<!-- Describe the issue -->
**Expected behavior**
<!--- What behavior did you expect? -->
**Actual behavior**
<!--- What was the actual behavior (provide screenshots if the issue is GUI-related)? -->
**To reproduce**
<!--- How reliably can you reproduce the issue, what are the steps to do so? -->
**System information**
<!-- What version of Bitcoin Core are you using, where did you get it (website, self-compiled, etc)? -->
<!-- What type of machine are you observing the error on (OS/CPU and disk type)? -->
<!-- GUI-related issue? What is your operating system and its version? If Linux, what is your desktop environment and graphical shell? -->
<!-- Any extra information that might be useful in the debugging process. -->
<!--- This is normally the contents of a `debug.log` or `config.log` file. Raw text or a link to a pastebin type site are preferred. -->

8
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Bitcoin Core Security Policy
url: https://github.com/bitcoin/bitcoin/blob/master/SECURITY.md
about: View security policy
- name: Bitcoin Core Developers
url: https://bitcoincore.org
about: Bitcoin Core homepage

View File

@@ -1,20 +0,0 @@
---
name: Feature request
about: Suggest an idea for this project
title: ''
labels: Feature
assignees: ''
---
**Is your feature request related to a problem? Please describe.**
<!-- A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] -->
**Describe the solution you'd like**
<!-- A clear and concise description of what you want to happen. -->
**Describe alternatives you've considered**
<!-- A clear and concise description of any alternative solutions or features you've considered. -->
**Additional context**
<!-- Add any other context or screenshots about the feature request here. -->

View File

@@ -0,0 +1,36 @@
name: Feature Request
description: Suggest an idea for this project.
labels: [Feature]
body:
- type: textarea
id: feature
attributes:
label: Please describe the feature you'd like to see added.
description: Attach screenshots or logs if applicable.
validations:
required: true
- type: textarea
id: related-problem
attributes:
label: Is your feature related to a problem, if so please describe it.
description: Attach screenshots or logs if applicable.
validations:
required: false
- type: textarea
id: solution
attributes:
label: Describe the solution you'd like
validations:
required: false
- type: textarea
id: alternatives
attributes:
label: Describe any alternatives you've considered
validations:
required: false
- type: textarea
id: additional-context
attributes:
label: Please leave any additional context
validations:
required: false

View File

@@ -1,22 +0,0 @@
---
name: Good first issue
about: '(Regular devs only): Suggest a new good first issue'
title: ''
labels: ''
assignees: ''
---
<!-- Needs the label "good first issue" assigned manually before or after opening -->
<!-- A good first issue is an uncontroversial issue, that has a relatively unique and obvious solution -->
<!-- Motivate the issue and explain the solution briefly -->
#### Useful skills:
<!-- (For example, “C++11 std::thread”, “Qt5 GUI and async GUI design” or “basic understanding of Bitcoin mining and the Bitcoin Core RPC interface”.) -->
#### Want to work on this issue?
For guidance on contributing, please read [CONTRIBUTING.md](https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md) before opening your pull request.

View File

@@ -0,0 +1,44 @@
name: Good First Issue
description: (Regular devs only) Suggest a new good first issue
labels: [good first issue]
body:
- type: markdown
attributes:
value: |
Please add the label "good first issue" manually before or after opening
A good first issue is an uncontroversial issue, that has a relatively unique and obvious solution
Motivate the issue and explain the solution briefly
- type: textarea
id: motivation
attributes:
label: Motivation
description: Motivate the issue
validations:
required: true
- type: textarea
id: solution
attributes:
label: Possible solution
description: Describe a possible solution
validations:
required: false
- type: textarea
id: useful-skills
attributes:
label: Useful Skills
description: For example, “`std::thread`”, “Qt5 GUI and async GUI design” or “basic understanding of Bitcoin mining and the Bitcoin Core RPC interface”.
value: |
* Compiling Bitcoin Core from source
* Running the C++ unit tests and the Python functional tests
* ...
- type: textarea
attributes:
label: Guidance for new contributors
description: Please leave this to automatically add the footer for new contributors
value: |
Want to work on this issue?
For guidance on contributing, please read [CONTRIBUTING.md](https://github.com/bitcoin/bitcoin/blob/master/CONTRIBUTING.md) before opening your pull request.

View File

@@ -1,11 +0,0 @@
---
name: An issue or feature request related to the GUI
about: Any report, issue or feature request related to the GUI should be reported at https://github.com/bitcoin-core/gui/issues/
title: Any report, issue or feature request related to the GUI should be reported at https://github.com/bitcoin-core/gui/issues/
labels: GUI
assignees: ''
---
Any report, issue or feature request related to the GUI should be reported at
https://github.com/bitcoin-core/gui/issues/

18
.github/ISSUE_TEMPLATE/gui_issue.yml vendored Normal file
View File

@@ -0,0 +1,18 @@
name: Issue or feature request related to the GUI
description: Any report, issue or feature request related to the GUI
labels: [GUI]
body:
- type: checkboxes
id: acknowledgement
attributes:
label: Issues, reports or feature requests related to the GUI should be opened directly on the GUI repo
description: https://github.com/bitcoin-core/gui/issues/
options:
- label: I still think this issue should be opened here
required: true
- type: textarea
id: gui-request
attributes:
label: Report
validations:
required: true

344
.github/workflows/ci.yml vendored Normal file
View File

@@ -0,0 +1,344 @@
# Copyright (c) 2023 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
name: CI
on:
# See: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#pull_request.
pull_request:
# See: https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#push.
push:
branches:
- '**'
tags-ignore:
- '**'
concurrency:
group: ${{ github.event_name != 'pull_request' && github.run_id || github.ref }}
cancel-in-progress: true
env:
DANGER_RUN_CI_ON_HOST: 1
CI_FAILFAST_TEST_LEAVE_DANGLING: 1 # GHA does not care about dangling processes and setting this variable avoids killing the CI script itself on error
MAKEJOBS: '-j10'
jobs:
test-each-commit:
name: 'test each commit'
runs-on: ubuntu-22.04
if: github.event_name == 'pull_request' && github.event.pull_request.commits != 1
timeout-minutes: 360 # Use maximum time, see https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idtimeout-minutes. Assuming a worst case time of 1 hour per commit, this leads to a --max-count=6 below.
env:
MAX_COUNT: 6
steps:
- name: Determine fetch depth
run: echo "FETCH_DEPTH=$((${{ github.event.pull_request.commits }} + 2))" >> "$GITHUB_ENV"
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: ${{ env.FETCH_DEPTH }}
- name: Determine commit range
run: |
# Checkout HEAD~ and find the test base commit
# Checkout HEAD~ because it would be wasteful to rerun tests on the PR
# head commit that are already run by other jobs.
git checkout HEAD~
# Figure out test base commit by listing ancestors of HEAD, excluding
# ancestors of the most recent merge commit, limiting the list to the
# newest MAX_COUNT ancestors, ordering it from oldest to newest, and
# taking the first one.
#
# If the branch contains up to MAX_COUNT ancestor commits after the
# most recent merge commit, all of those commits will be tested. If it
# contains more, only the most recent MAX_COUNT commits will be
# tested.
#
# In the command below, the ^@ suffix is used to refer to all parents
# of the merge commit as described in:
# https://git-scm.com/docs/git-rev-parse#_other_rev_parent_shorthand_notations
# and the ^ prefix is used to exclude these parents and all their
# ancestors from the rev-list output as described in:
# https://git-scm.com/docs/git-rev-list
echo "TEST_BASE=$(git rev-list -n$((${{ env.MAX_COUNT }} + 1)) --reverse HEAD ^$(git rev-list -n1 --merges HEAD)^@ | head -1)" >> "$GITHUB_ENV"
- run: |
sudo apt-get update
sudo apt-get install clang ccache build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev qtbase5-dev qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
- name: Compile and run tests
run: |
# Run tests on commits after the last merge commit and before the PR head commit
# Use clang++, because it is a bit faster and uses less memory than g++
git rebase --exec "echo Running test-one-commit on \$( git log -1 ) && ./autogen.sh && CC=clang CXX=clang++ ./configure && make clean && make -j $(nproc) check && ./test/functional/test_runner.py -j $(( $(nproc) * 2 ))" ${{ env.TEST_BASE }}
macos-native-x86_64:
name: 'macOS 13 native, x86_64, no depends, sqlite only, gui'
# Use latest image, but hardcode version to avoid silent upgrades (and breaks).
# See: https://github.com/actions/runner-images#available-images.
runs-on: macos-13 # Use M1 once available https://github.com/github/roadmap/issues/528
# No need to run on the read-only mirror, unless it is a PR.
if: github.repository != 'bitcoin-core/gui' || github.event_name == 'pull_request'
timeout-minutes: 120
env:
FILE_ENV: './ci/test/00_setup_env_mac_native.sh'
BASE_ROOT_DIR: ${{ github.workspace }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Clang version
run: |
sudo xcode-select --switch /Applications/Xcode_15.0.app
clang --version
- name: Install Homebrew packages
env:
HOMEBREW_NO_INSTALLED_DEPENDENTS_CHECK: 1
run: |
# A workaround for "The `brew link` step did not complete successfully" error.
brew install python@3 || brew link --overwrite python@3
brew install automake libtool pkg-config gnu-getopt ccache boost libevent miniupnpc libnatpmp zeromq qt@5 qrencode
- name: Set Ccache directory
run: echo "CCACHE_DIR=${RUNNER_TEMP}/ccache_dir" >> "$GITHUB_ENV"
- name: Restore Ccache cache
id: ccache-cache
uses: actions/cache/restore@v3
with:
path: ${{ env.CCACHE_DIR }}
key: ${{ github.job }}-ccache-${{ github.run_id }}
restore-keys: ${{ github.job }}-ccache-
- name: CI script
run: ./ci/test_run_all.sh
- name: Save Ccache cache
uses: actions/cache/save@v3
if: github.event_name != 'pull_request' && steps.ccache-cache.outputs.cache-hit != 'true'
with:
path: ${{ env.CCACHE_DIR }}
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
key: ${{ github.job }}-ccache-${{ github.run_id }}
win64-native:
name: 'Win64 native, VS 2022'
# Use latest image, but hardcode version to avoid silent upgrades (and breaks).
# See: https://github.com/actions/runner-images#available-images.
runs-on: windows-2022
# No need to run on the read-only mirror, unless it is a PR.
if: github.repository != 'bitcoin-core/gui' || github.event_name == 'pull_request'
env:
CCACHE_MAXSIZE: '200M'
CI_CCACHE_VERSION: '4.7.5'
CI_QT_CONF: '-release -silent -opensource -confirm-license -opengl desktop -static -static-runtime -mp -qt-zlib -qt-pcre -qt-libpng -nomake examples -nomake tests -nomake tools -no-angle -no-dbus -no-gif -no-gtk -no-ico -no-icu -no-libjpeg -no-libudev -no-sql-sqlite -no-sql-odbc -no-sqlite -no-vulkan -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcharts -skip qtconnectivity -skip qtdatavis3d -skip qtdeclarative -skip doc -skip qtdoc -skip qtgamepad -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtlottie -skip qtmacextras -skip qtmultimedia -skip qtnetworkauth -skip qtpurchasing -skip qtquick3d -skip qtquickcontrols -skip qtquickcontrols2 -skip qtquicktimeline -skip qtremoteobjects -skip qtscript -skip qtscxml -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtspeech -skip qtsvg -skip qtvirtualkeyboard -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebglplugin -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -no-openssl -no-feature-bearermanagement -no-feature-printdialog -no-feature-printer -no-feature-printpreviewdialog -no-feature-printpreviewwidget -no-feature-sql -no-feature-sqlmodel -no-feature-textbrowser -no-feature-textmarkdownwriter -no-feature-textodfwriter -no-feature-xml'
CI_QT_DIR: 'qt-everywhere-src-5.15.10'
CI_QT_URL: 'https://download.qt.io/official_releases/qt/5.15/5.15.10/single/qt-everywhere-opensource-src-5.15.10.zip'
PYTHONUTF8: 1
TEST_RUNNER_TIMEOUT_FACTOR: 40
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Fix Visual Studio installation
# Avoid toolset ambiguity that MSVC can't handle.
run: |
Set-Location "C:\Program Files (x86)\Microsoft Visual Studio\Installer\"
$InstallPath = "C:\Program Files\Microsoft Visual Studio\2022\Enterprise"
$componentsToRemove= @(
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ARM.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ARM"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ARM64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ARM64"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ATL.ARM.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ATL.ARM"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ATL.ARM64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ATL.ARM64"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ATL.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.ATL"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.MFC.ARM.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.MFC.ARM"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.MFC.ARM64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.MFC.ARM64"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.MFC.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.MFC"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.x86.x64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.37.17.7.x86.x64"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ARM"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ARM.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ARM64"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ARM64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ATL"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ATL.ARM"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ATL.ARM.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ATL.ARM64"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ATL.ARM64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.ATL.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.MFC"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.MFC.ARM"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.MFC.ARM.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.MFC.ARM64"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.MFC.ARM64.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.MFC.Spectre"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.x86.x64"
"Microsoft.VisualStudio.Component.VC.14.38.17.8.x86.x64.Spectre"
)
[string]$workloadArgs = $componentsToRemove | ForEach-Object {" --remove " + $_}
$Arguments = ('/c', "vs_installer.exe", 'modify', '--installPath', "`"$InstallPath`"",$workloadArgs, '--quiet', '--norestart', '--nocache')
# should be run twice
$process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
$process = Start-Process -FilePath cmd.exe -ArgumentList $Arguments -Wait -PassThru -WindowStyle Hidden
- name: Configure Developer Command Prompt for Microsoft Visual C++
# Using microsoft/setup-msbuild is not enough.
uses: ilammy/msvc-dev-cmd@v1
with:
arch: x64
- name: Check MSBuild and Qt
run: |
msbuild -version | Out-File -FilePath "$env:GITHUB_WORKSPACE\msbuild_version"
Get-Content -Path "$env:GITHUB_WORKSPACE\msbuild_version"
$env:CI_QT_URL | Out-File -FilePath "$env:GITHUB_WORKSPACE\qt_url"
$env:CI_QT_CONF | Out-File -FilePath "$env:GITHUB_WORKSPACE\qt_conf"
- name: Restore static Qt cache
id: static-qt-cache
uses: actions/cache/restore@v3
with:
path: C:\Qt_static
key: ${{ github.job }}-static-qt-${{ hashFiles('msbuild_version', 'qt_url', 'qt_conf') }}
- name: Build static Qt. Download
if: steps.static-qt-cache.outputs.cache-hit != 'true'
shell: cmd
run: |
curl --location --output C:\qt-src.zip %CI_QT_URL%
choco install --yes --no-progress jom
- name: Build static Qt. Expand source archive
if: steps.static-qt-cache.outputs.cache-hit != 'true'
shell: cmd
run: tar -xf C:\qt-src.zip -C C:\
- name: Build static Qt. Create build directory
if: steps.static-qt-cache.outputs.cache-hit != 'true'
run: |
Rename-Item -Path "C:\$env:CI_QT_DIR" -NewName "C:\qt-src"
New-Item -ItemType Directory -Path "C:\qt-src\build"
- name: Build static Qt. Configure
if: steps.static-qt-cache.outputs.cache-hit != 'true'
working-directory: C:\qt-src\build
shell: cmd
run: ..\configure %CI_QT_CONF% -prefix C:\Qt_static
- name: Build static Qt. Build
if: steps.static-qt-cache.outputs.cache-hit != 'true'
working-directory: C:\qt-src\build
shell: cmd
run: jom
- name: Build static Qt. Install
if: steps.static-qt-cache.outputs.cache-hit != 'true'
working-directory: C:\qt-src\build
shell: cmd
run: jom install
- name: Save static Qt cache
if: steps.static-qt-cache.outputs.cache-hit != 'true'
uses: actions/cache/save@v3
with:
path: C:\Qt_static
key: ${{ github.job }}-static-qt-${{ hashFiles('msbuild_version', 'qt_url', 'qt_conf') }}
- name: Ccache installation cache
id: ccache-installation-cache
uses: actions/cache@v3
with:
path: |
C:\ProgramData\chocolatey\lib\ccache
C:\ProgramData\chocolatey\bin\ccache.exe
C:\ccache\cl.exe
key: ${{ github.job }}-ccache-installation-${{ env.CI_CCACHE_VERSION }}
- name: Install Ccache
if: steps.ccache-installation-cache.outputs.cache-hit != 'true'
run: |
choco install --yes --no-progress ccache --version=$env:CI_CCACHE_VERSION
New-Item -ItemType Directory -Path "C:\ccache"
Copy-Item -Path "$env:ChocolateyInstall\lib\ccache\tools\ccache-$env:CI_CCACHE_VERSION-windows-x86_64\ccache.exe" -Destination "C:\ccache\cl.exe"
- name: Restore Ccache cache
id: ccache-cache
uses: actions/cache/restore@v3
with:
path: ~/AppData/Local/ccache
key: ${{ github.job }}-ccache-${{ github.run_id }}
restore-keys: ${{ github.job }}-ccache-
- name: Using vcpkg with MSBuild
run: |
Set-Location "$env:VCPKG_INSTALLATION_ROOT"
Add-Content -Path "triplets\x64-windows-static.cmake" -Value "set(VCPKG_BUILD_TYPE release)"
vcpkg --vcpkg-root "$env:VCPKG_INSTALLATION_ROOT" integrate install
git rev-parse HEAD | Out-File -FilePath "$env:GITHUB_WORKSPACE\vcpkg_commit"
Get-Content -Path "$env:GITHUB_WORKSPACE\vcpkg_commit"
- name: vcpkg tools cache
uses: actions/cache@v3
with:
path: C:/vcpkg/downloads/tools
key: ${{ github.job }}-vcpkg-tools
- name: vcpkg binary cache
uses: actions/cache@v3
with:
path: ~/AppData/Local/vcpkg/archives
key: ${{ github.job }}-vcpkg-binary-${{ hashFiles('vcpkg_commit', 'msbuild_version', 'build_msvc/vcpkg.json') }}
- name: Generate project files
run: py -3 build_msvc\msvc-autogen.py
- name: Build
shell: cmd
run: |
ccache --zero-stats
msbuild build_msvc\bitcoin.sln -property:CLToolPath=C:\ccache;CLToolExe=cl.exe;UseMultiToolTask=true;Configuration=Release -maxCpuCount -verbosity:minimal -noLogo
- name: Ccache stats
run: ccache --show-stats
- name: Save Ccache cache
uses: actions/cache/save@v3
if: github.event_name != 'pull_request' && steps.ccache-cache.outputs.cache-hit != 'true'
with:
path: ~/AppData/Local/ccache
# https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache
key: ${{ github.job }}-ccache-${{ github.run_id }}
- name: Run unit tests
run: src\test_bitcoin.exe -l test_suite
- name: Run benchmarks
run: src\bench_bitcoin.exe -sanity-check
- name: Run util tests
run: py -3 test\util\test_runner.py
- name: Run rpcauth test
run: py -3 test\util\rpcauth-test.py
- name: Run functional tests
# Don't run functional tests for pull requests.
# The test suit regularly fails to complete in windows native github
# actions as a child process stops making progress. The root cause has
# not yet been determined.
# Discussed in https://github.com/bitcoin/bitcoin/pull/28509
if: github.event_name != 'pull_request'
run: py -3 test\functional\test_runner.py --jobs $env:NUMBER_OF_PROCESSORS --ci --quiet --tmpdirprefix=$env:RUNNER_TEMP --combinedlogslen=99999999 --timeout-factor=$env:TEST_RUNNER_TIMEOUT_FACTOR --extended

2
.gitignore vendored
View File

@@ -74,7 +74,7 @@ src/qt/bitcoin-qt.includes
*.log
*.trs
*.dmg
*.zip
*.json.h
*.raw.h

View File

@@ -1 +1 @@
3.6.12
3.8.17

View File

@@ -107,7 +107,7 @@ each_dict_entry_on_separate_line=True
i18n_comment=
# The i18n function call names. The presence of this function stops
# reformattting on that line, because the string it has cannot be moved
# reformatting on that line, because the string it has cannot be moved
# away from the i18n comment.
i18n_function_call=

View File

@@ -1,7 +1,7 @@
[main]
host = https://www.transifex.com
[bitcoin.qt-translation-024x]
[o:bitcoin:p:bitcoin:r:qt-translation-026x]
file_filter = src/qt/locale/bitcoin_<lang>.xlf
source_file = src/qt/locale/bitcoin_en.xlf
source_lang = en

View File

@@ -1,7 +1,7 @@
The MIT License (MIT)
Copyright (c) 2009-2022 The Bitcoin Core developers
Copyright (c) 2009-2022 Bitcoin Developers
Copyright (c) 2009-2023 The Bitcoin Core developers
Copyright (c) 2009-2023 Bitcoin Developers
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

View File

@@ -37,7 +37,7 @@ space := $(empty) $(empty)
OSX_APP=Bitcoin-Qt.app
OSX_VOLNAME = $(subst $(space),-,$(PACKAGE_NAME))
OSX_DMG = $(OSX_VOLNAME).dmg
OSX_ZIP = $(OSX_VOLNAME).zip
OSX_DEPLOY_SCRIPT=$(top_srcdir)/contrib/macdeploy/macdeployqtplus
OSX_INSTALLER_ICONS=$(top_srcdir)/src/qt/res/icons/bitcoin.icns
OSX_PLIST=$(top_builddir)/share/qt/Info.plist #not installed
@@ -124,15 +124,16 @@ osx_volname:
echo $(OSX_VOLNAME) >$@
if BUILD_DARWIN
$(OSX_DMG): $(OSX_APP_BUILT) $(OSX_PACKAGING)
$(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR) -dmg
$(OSX_ZIP): $(OSX_APP_BUILT) $(OSX_PACKAGING)
$(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR) -zip
deploydir: $(OSX_DMG)
deploydir: $(OSX_ZIP)
else !BUILD_DARWIN
APP_DIST_DIR=$(top_builddir)/dist
$(OSX_DMG): deploydir
$(XORRISOFS) -D -l -V "$(OSX_VOLNAME)" -no-pad -r -dir-mode 0755 -o $@ $(APP_DIST_DIR) -- $(if $(SOURCE_DATE_EPOCH),-volume_date all_file_dates =$(SOURCE_DATE_EPOCH))
$(OSX_ZIP): deploydir
if [ -n "$(SOURCE_DATE_EPOCH)" ]; then find $(APP_DIST_DIR) -exec touch -d @$(SOURCE_DATE_EPOCH) {} +; fi
cd $(APP_DIST_DIR) && find . | sort | $(ZIP) -X@ $@
$(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt: $(OSX_APP_BUILT) $(OSX_PACKAGING)
INSTALL_NAME_TOOL=$(INSTALL_NAME_TOOL) OTOOL=$(OTOOL) STRIP=$(STRIP) $(PYTHON) $(OSX_DEPLOY_SCRIPT) $(OSX_APP) $(OSX_VOLNAME) -translations-dir=$(QT_TRANSLATION_DIR)
@@ -140,7 +141,7 @@ $(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt: $(OSX_APP_BUILT) $(OSX_PAC
deploydir: $(APP_DIST_DIR)/$(OSX_APP)/Contents/MacOS/Bitcoin-Qt
endif !BUILD_DARWIN
deploy: $(OSX_DMG)
deploy: $(OSX_ZIP)
endif
$(BITCOIN_QT_BIN): FORCE
@@ -313,7 +314,7 @@ EXTRA_DIST += \
test/util/data/txcreatesignv2.hex \
test/util/rpcauth-test.py
CLEANFILES = $(OSX_DMG) $(BITCOIN_WIN_INSTALLER)
CLEANFILES = $(OSX_ZIP) $(BITCOIN_WIN_INSTALLER)
DISTCHECK_CONFIGURE_FLAGS = --enable-man

View File

@@ -1,17 +0,0 @@
# ==============================================================================
# Bitcoin Core REVIEWERS
# ==============================================================================
# Configuration of automated review requests for the bitcoin/bitcoin repo
# via DrahtBot.
# Order is not important; if a modified file or directory matches a fnmatch,
# the reviewer will be mentioned in a PR comment requesting a review.
# Regular contributors are free to add their names to specific directories or
# files provided that they are willing to provide a review.
# Absence from this list should not be interpreted as a discouragement to
# review a pull request. Peer review is always welcome and is a critical
# component of the progress of the codebase. Information on peer review
# guidelines can be found in the CONTRIBUTING.md doc.

View File

@@ -14,3 +14,16 @@ fi
command -v autoreconf >/dev/null || \
(echo "configuration failed, please install autoconf first" && exit 1)
autoreconf --install --force --warnings=all
if expr "'$(build-aux/config.guess --timestamp)" \< "'$(depends/config.guess --timestamp)" > /dev/null; then
chmod ug+w build-aux/config.guess
chmod ug+w src/secp256k1/build-aux/config.guess
cp depends/config.guess build-aux
cp depends/config.guess src/secp256k1/build-aux
fi
if expr "'$(build-aux/config.sub --timestamp)" \< "'$(depends/config.sub --timestamp)" > /dev/null; then
chmod ug+w build-aux/config.sub
chmod ug+w src/secp256k1/build-aux/config.sub
cp depends/config.sub build-aux
cp depends/config.sub src/secp256k1/build-aux
fi

View File

@@ -8,7 +8,7 @@
#
# DESCRIPTION
#
# Test for the Boost C++ libraries of a particular version (or newer)
# Test for the Boost C++ headers of a particular version (or newer)
#
# If no path to the installed boost library is given the macro searchs
# under /usr, /usr/local, /opt, /opt/local and /opt/homebrew and evaluates
@@ -17,12 +17,14 @@
#
# This macro calls:
#
# AC_SUBST(BOOST_CPPFLAGS) / AC_SUBST(BOOST_LDFLAGS)
# AC_SUBST(BOOST_CPPFLAGS)
#
# And sets:
#
# HAVE_BOOST
#
# Note that this macro has been modified compared to upstream.
#
# LICENSE
#
# Copyright (c) 2008 Thomas Porschberg <thomas@randspringer.de>
@@ -59,26 +61,10 @@ AC_ARG_WITH([boost],
],
[want_boost="yes"])
AC_ARG_WITH([boost-libdir],
[AS_HELP_STRING([--with-boost-libdir=LIB_DIR],
[Force given directory for boost libraries.
Note that this will override library path detection,
so use this parameter only if default library detection fails
and you know exactly where your boost libraries are located.])],
[
AS_IF([test -d "$withval"],
[_AX_BOOST_BASE_boost_lib_path="$withval"],
[AC_MSG_ERROR([--with-boost-libdir expected directory name])])
],
[_AX_BOOST_BASE_boost_lib_path=""])
BOOST_LDFLAGS=""
BOOST_CPPFLAGS=""
AS_IF([test "x$want_boost" = "xyes"],
[_AX_BOOST_BASE_RUNDETECT([$1],[$2],[$3])])
AC_SUBST(BOOST_CPPFLAGS)
AC_SUBST(BOOST_LDFLAGS)
])
@@ -139,7 +125,6 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
AC_MSG_CHECKING([for boostlib >= $1 ($WANT_BOOST_VERSION) lib path in "$_AX_BOOST_BASE_boost_path/$_AX_BOOST_BASE_boost_path_tmp"])
AS_IF([test -d "$_AX_BOOST_BASE_boost_path/$_AX_BOOST_BASE_boost_path_tmp" && test -r "$_AX_BOOST_BASE_boost_path/$_AX_BOOST_BASE_boost_path_tmp" ],[
AC_MSG_RESULT([yes])
BOOST_LDFLAGS="-L$_AX_BOOST_BASE_boost_path/$_AX_BOOST_BASE_boost_path_tmp";
break;
],
[AC_MSG_RESULT([no])])
@@ -156,27 +141,17 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
for libsubdir in $search_libsubdirs ; do
if ls "$_AX_BOOST_BASE_boost_path_tmp/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi
done
BOOST_LDFLAGS="-L$_AX_BOOST_BASE_boost_path_tmp/$libsubdir"
BOOST_CPPFLAGS="-I$_AX_BOOST_BASE_boost_path_tmp/include"
break;
fi
done
])
dnl overwrite ld flags if we have required special directory with
dnl --with-boost-libdir parameter
AS_IF([test "x$_AX_BOOST_BASE_boost_lib_path" != "x"],
[BOOST_LDFLAGS="-L$_AX_BOOST_BASE_boost_lib_path"])
AC_MSG_CHECKING([for boostlib >= $1 ($WANT_BOOST_VERSION)])
AC_MSG_CHECKING([for Boost headers >= $1 ($WANT_BOOST_VERSION)])
CPPFLAGS_SAVED="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS_SAVED="$LDFLAGS"
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_REQUIRE([AC_PROG_CXX])
AC_LANG_PUSH(C++)
AC_COMPILE_IFELSE([_AX_BOOST_BASE_PROGRAM($WANT_BOOST_VERSION)],[
@@ -193,11 +168,8 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
dnl built and installed without the --layout=system option or for a staged(not installed) version
if test "x$succeeded" != "xyes" ; then
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
BOOST_CPPFLAGS=
if test -z "$_AX_BOOST_BASE_boost_lib_path" ; then
BOOST_LDFLAGS=
fi
_version=0
if test -n "$_AX_BOOST_BASE_boost_path" ; then
if test -d "$_AX_BOOST_BASE_boost_path" && test -r "$_AX_BOOST_BASE_boost_path"; then
@@ -216,14 +188,6 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
BOOST_CPPFLAGS="-I$_AX_BOOST_BASE_boost_path"
fi
fi
dnl if we found something and BOOST_LDFLAGS was unset before
dnl (because "$_AX_BOOST_BASE_boost_lib_path" = ""), set it here.
if test -n "$BOOST_CPPFLAGS" && test -z "$BOOST_LDFLAGS"; then
for libsubdir in $libsubdirs ; do
if ls "$_AX_BOOST_BASE_boost_path/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi
done
BOOST_LDFLAGS="-L$_AX_BOOST_BASE_boost_path/$libsubdir"
fi
fi
else
if test "x$cross_compiling" != "xyes" ; then
@@ -242,12 +206,6 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
VERSION_UNDERSCORE=`echo $_version | sed 's/\./_/'`
BOOST_CPPFLAGS="-I$best_path/include/boost-$VERSION_UNDERSCORE"
if test -z "$_AX_BOOST_BASE_boost_lib_path" ; then
for libsubdir in $libsubdirs ; do
if ls "$best_path/$libsubdir/libboost_"* >/dev/null 2>&1 ; then break; fi
done
BOOST_LDFLAGS="-L$best_path/$libsubdir"
fi
fi
if test -n "$BOOST_ROOT" ; then
@@ -259,10 +217,9 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
stage_version=`echo $version_dir | sed 's/boost_//' | sed 's/_/./g'`
stage_version_shorten=`expr $stage_version : '\([[0-9]]*\.[[0-9]]*\)'`
V_CHECK=`expr $stage_version_shorten \>\= $_version`
if test "x$V_CHECK" = "x1" && test -z "$_AX_BOOST_BASE_boost_lib_path" ; then
if test "x$V_CHECK" = "x1" ; then
AC_MSG_NOTICE(We will use a staged boost library from $BOOST_ROOT)
BOOST_CPPFLAGS="-I$BOOST_ROOT"
BOOST_LDFLAGS="-L$BOOST_ROOT/stage/$libsubdir"
fi
fi
fi
@@ -270,8 +227,6 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
export CPPFLAGS
LDFLAGS="$LDFLAGS $BOOST_LDFLAGS"
export LDFLAGS
AC_LANG_PUSH(C++)
AC_COMPILE_IFELSE([_AX_BOOST_BASE_PROGRAM($WANT_BOOST_VERSION)],[
@@ -298,6 +253,4 @@ AC_DEFUN([_AX_BOOST_BASE_RUNDETECT],[
fi
CPPFLAGS="$CPPFLAGS_SAVED"
LDFLAGS="$LDFLAGS_SAVED"
])

View File

@@ -1,47 +0,0 @@
dnl Copyright (c) 2022 The Bitcoin Core developers
dnl Distributed under the MIT software license, see the accompanying
dnl file COPYING or http://www.opensource.org/licenses/mit-license.php.
# GCC 8.1 and earlier requires -lstdc++fs
# Clang 8.0.0 (libc++) and earlier requires -lc++fs
m4_define([_CHECK_FILESYSTEM_testbody], [[
#include <filesystem>
namespace fs = std::filesystem;
int main() {
(void)fs::current_path().root_name();
return 0;
}
]])
AC_DEFUN([CHECK_FILESYSTEM], [
AC_LANG_PUSH(C++)
AC_MSG_CHECKING([whether std::filesystem can be used without link library])
AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_FILESYSTEM_testbody])],[
AC_MSG_RESULT([yes])
],[
AC_MSG_RESULT([no])
SAVED_LIBS="$LIBS"
LIBS="$SAVED_LIBS -lstdc++fs"
AC_MSG_CHECKING([whether std::filesystem needs -lstdc++fs])
AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_FILESYSTEM_testbody])],[
AC_MSG_RESULT([yes])
],[
AC_MSG_RESULT([no])
AC_MSG_CHECKING([whether std::filesystem needs -lc++fs])
LIBS="$SAVED_LIBS -lc++fs"
AC_LINK_IFELSE([AC_LANG_SOURCE([_CHECK_FILESYSTEM_testbody])],[
AC_MSG_RESULT([yes])
],[
AC_MSG_FAILURE([cannot figure out how to use std::filesystem])
])
])
])
AC_LANG_POP
])

View File

@@ -7,7 +7,9 @@ Visual Studio 2022 is minimum required to build Bitcoin Core.
Solution and project files to build with `msbuild` or Visual Studio can be found in the `build_msvc` directory.
To build Bitcoin Core from the command-line, it is sufficient to only install the Visual Studio Build Tools component.
To build Bitcoin Core from the command-line, it is sufficient to only install the [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/) component.
The "Desktop development with C++" workload must be installed as well.
Building with Visual Studio is an alternative to the Linux based [cross-compiler build](../doc/build-windows.md).
@@ -15,7 +17,7 @@ Building with Visual Studio is an alternative to the Linux based [cross-compiler
Prerequisites
---------------------
To build [dependencies](../doc/dependencies.md) (except for [Qt](#qt)),
the default approach is to use the [vcpkg](https://docs.microsoft.com/en-us/cpp/vcpkg) package manager from Microsoft:
the default approach is to use the [vcpkg](https://vcpkg.io) package manager from Microsoft:
1. [Install](https://vcpkg.io/en/getting-started.html) vcpkg.
@@ -30,7 +32,7 @@ Qt
---------------------
To build Bitcoin Core with the GUI, a static build of Qt is required.
1. Download a single ZIP archive of Qt source code from https://download.qt.io/official_releases/qt/ (e.g., [`qt-everywhere-opensource-src-5.15.5.zip`](https://download.qt.io/official_releases/qt/5.15/5.15.5/single/qt-everywhere-opensource-src-5.15.5.zip)), and expand it into a dedicated folder. The following instructions assume that this folder is `C:\dev\qt-source`.
1. Download a single ZIP archive of Qt source code from https://download.qt.io/official_releases/qt/ (e.g., [`qt-everywhere-opensource-src-5.15.10.zip`](https://download.qt.io/official_releases/qt/5.15/5.15.10/single/qt-everywhere-opensource-src-5.15.10.zip)), and expand it into a dedicated folder. The following instructions assume that this folder is `C:\dev\qt-source`.
2. Open "x64 Native Tools Command Prompt for VS 2022", and input the following commands:
```cmd
@@ -67,7 +69,7 @@ Alternatively, open the `build_msvc/bitcoin.sln` file in Visual Studio.
Security
---------------------
[Base address randomization](https://docs.microsoft.com/en-us/cpp/build/reference/dynamicbase-use-address-space-layout-randomization?view=msvc-160) is used to make Bitcoin Core more secure. When building Bitcoin using the `build_msvc` process base address randomization can be disabled by editing `common.init.vcproj` to change `RandomizedBaseAddress` from `true` to `false` and then rebuilding the project.
[Base address randomization](https://learn.microsoft.com/en-us/cpp/build/reference/dynamicbase-use-address-space-layout-randomization) is used to make Bitcoin Core more secure. When building Bitcoin using the `build_msvc` process base address randomization can be disabled by editing `common.init.vcproj` to change `RandomizedBaseAddress` from `true` to `false` and then rebuilding the project.
To check if `bitcoind` has `RandomizedBaseAddress` enabled or disabled run

View File

@@ -12,7 +12,7 @@
@SOURCE_FILES@
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">

View File

@@ -15,6 +15,9 @@
<ProjectReference Include="..\libbitcoin_cli\libbitcoin_cli.vcxproj">
<Project>{0667528c-d734-4009-adf9-c0d6c4a5a5a6}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">
<Project>{7c87e378-df58-482e-aa2f-1bc129bc19ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_crypto\libbitcoin_crypto.vcxproj">
<Project>{6190199c-6cf4-4dad-bfbd-93fa72a760c1}</Project>
</ProjectReference>

View File

@@ -13,7 +13,7 @@
<ResourceCompile Include="..\..\src\qt\res\bitcoin-qt-res.rc" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_cli\libbitcoin_cli.vcxproj">

View File

@@ -12,7 +12,7 @@
<ClCompile Include="..\..\src\bitcoin-tx.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">

View File

@@ -12,7 +12,7 @@
<ClCompile Include="..\..\src\bitcoin-util.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">

View File

@@ -15,7 +15,7 @@
</ClCompile>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">

View File

@@ -2,7 +2,7 @@ Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 16
VisualStudioVersion = 16.0.28803.452
MinimumVisualStudioVersion = 10.0.40219.1
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbitcoinconsensus", "libbitcoinconsensus\libbitcoinconsensus.vcxproj", "{2B384FA8-9EE1-4544-93CB-0D733C25E8CE}"
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "libbitcoin_consensus", "libbitcoin_consensus\libbitcoin_consensus.vcxproj", "{2B384FA8-9EE1-4544-93CB-0D733C25E8CE}"
EndProject
Project("{8BC9CEB8-8B4A-11D0-8D11-00A0C91BC942}") = "bitcoind", "bitcoind\bitcoind.vcxproj", "{D4513DDF-6013-44DC-ADCC-12EAF6D1F038}"
EndProject

View File

@@ -38,21 +38,12 @@
/* Define to 1 to enable SQLite wallet */
#define USE_SQLITE 1
/* Define to 1 to enable ZMQ functions */
/* Define this symbol to enable ZMQ functions */
#define ENABLE_ZMQ 1
/* define if the Boost library is available */
#define HAVE_BOOST /**/
/* define if external signer support is enabled (requires Boost::Process) */
#define ENABLE_EXTERNAL_SIGNER /**/
/* Define this symbol if the consensus lib has been built */
#define HAVE_CONSENSUS_LIB 1
/* define if the compiler supports basic C++20 syntax */
#define HAVE_CXX20 1
/* Define to 1 if you have the declaration of `be16toh', and to 0 if you
don't. */
#define HAVE_DECL_BE16TOH 0
@@ -121,49 +112,9 @@
*/
#define HAVE_DECL_SETSID 0
/* Define to 1 if you have the declaration of `strerror_r', and to 0 if you
don't. */
#define HAVE_DECL_STRERROR_R 0
/* Define if the dllexport attribute is supported. */
#define HAVE_DLLEXPORT_ATTRIBUTE 1
/* Define to 1 if you have the <inttypes.h> header file. */
#define HAVE_INTTYPES_H 1
/* Define to 1 if you have the <memory.h> header file. */
#define HAVE_MEMORY_H 1
/* Define to 1 if you have the <miniupnpc/miniupnpc.h> header file. */
#define HAVE_MINIUPNPC_MINIUPNPC_H 1
/* Define to 1 if you have the <miniupnpc/upnpcommands.h> header file. */
#define HAVE_MINIUPNPC_UPNPCOMMANDS_H 1
/* Define to 1 if you have the <miniupnpc/upnperrors.h> header file. */
#define HAVE_MINIUPNPC_UPNPERRORS_H 1
/* Define to 1 if you have the <stdint.h> header file. */
#define HAVE_STDINT_H 1
/* Define to 1 if you have the <stdio.h> header file. */
#define HAVE_STDIO_H 1
/* Define to 1 if you have the <stdlib.h> header file. */
#define HAVE_STDLIB_H 1
/* Define to 1 if you have the <strings.h> header file. */
#define HAVE_STRINGS_H 1
/* Define to 1 if you have the <string.h> header file. */
#define HAVE_STRING_H 1
/* Define to 1 if you have the <sys/stat.h> header file. */
#define HAVE_SYS_STAT_H 1
/* Define to 1 if you have the <sys/types.h> header file. */
#define HAVE_SYS_TYPES_H 1
/* Define to the address where bug reports for this package should be sent. */
#define PACKAGE_BUGREPORT "https://github.com/bitcoin/bitcoin/issues"

View File

@@ -15,7 +15,7 @@
</ClCompile>
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_common\libbitcoin_common.vcxproj">

View File

@@ -88,9 +88,9 @@
<WarningLevel>Level3</WarningLevel>
<PrecompiledHeader>NotUsing</PrecompiledHeader>
<AdditionalOptions>/utf-8 /Zc:__cplusplus /std:c++20 %(AdditionalOptions)</AdditionalOptions>
<DisableSpecificWarnings>4018;4244;4267;4334;4715;4805;4834</DisableSpecificWarnings>
<DisableSpecificWarnings>4018;4244;4267;4715;4805</DisableSpecificWarnings>
<TreatWarningAsError>true</TreatWarningAsError>
<PreprocessorDefinitions>_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING;ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;_WIN32_IE=0x0501;WIN32_LEAN_AND_MEAN;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>_SILENCE_CXX17_CODECVT_HEADER_DEPRECATION_WARNING;SECP256K1_STATIC;ZMQ_STATIC;NOMINMAX;WIN32;HAVE_CONFIG_H;_CRT_SECURE_NO_WARNINGS;_SCL_SECURE_NO_WARNINGS;_CONSOLE;_WIN32_WINNT=0x0601;_WIN32_IE=0x0501;WIN32_LEAN_AND_MEAN;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src;..\..\src\minisketch\include;..\..\src\univalue\include;..\..\src\secp256k1\include;..\..\src\leveldb\include;..\..\src\leveldb\helpers\memenv;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>
<Link>

View File

@@ -8,6 +8,7 @@
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\common\url.cpp" />
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -11,15 +11,6 @@
<ClCompile Include="..\..\src\arith_uint256.cpp" />
<ClCompile Include="..\..\src\consensus\merkle.cpp" />
<ClCompile Include="..\..\src\consensus\tx_check.cpp" />
<ClCompile Include="..\..\src\crypto\aes.cpp" />
<ClCompile Include="..\..\src\crypto\chacha20.cpp" />
<ClCompile Include="..\..\src\crypto\hmac_sha256.cpp" />
<ClCompile Include="..\..\src\crypto\hmac_sha512.cpp" />
<ClCompile Include="..\..\src\crypto\ripemd160.cpp" />
<ClCompile Include="..\..\src\crypto\sha1.cpp" />
<ClCompile Include="..\..\src\crypto\sha256.cpp" />
<ClCompile Include="..\..\src\crypto\sha256_sse4.cpp" />
<ClCompile Include="..\..\src\crypto\sha512.cpp" />
<ClCompile Include="..\..\src\hash.cpp" />
<ClCompile Include="..\..\src\primitives\block.cpp" />
<ClCompile Include="..\..\src\primitives\transaction.cpp" />

View File

@@ -8,7 +8,6 @@
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\util\url.cpp" />
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -50,7 +50,7 @@
</ItemGroup>
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>HAVE_CRC32C=0;HAVE_SNAPPY=0;__STDC_LIMIT_MACROS;LEVELDB_IS_BIG_ENDIAN=0;_UNICODE;UNICODE;_CRT_NONSTDC_NO_DEPRECATE;LEVELDB_PLATFORM_WINDOWS;LEVELDB_ATOMIC_PRESENT;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>HAVE_CRC32C=0;HAVE_SNAPPY=0;LEVELDB_IS_BIG_ENDIAN=0;_UNICODE;UNICODE;_CRT_NONSTDC_NO_DEPRECATE;LEVELDB_PLATFORM_WINDOWS;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<DisableSpecificWarnings>4244;4267</DisableSpecificWarnings>
<AdditionalIncludeDirectories>..\..\src\leveldb;..\..\src\leveldb\include;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
</ClCompile>

View File

@@ -14,7 +14,8 @@
</ItemGroup>
<ItemDefinitionGroup>
<ClCompile>
<PreprocessorDefinitions>ENABLE_MODULE_ECDH;ENABLE_MODULE_RECOVERY;ENABLE_MODULE_EXTRAKEYS;ENABLE_MODULE_SCHNORRSIG;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<PreprocessorDefinitions>ENABLE_MODULE_RECOVERY;ENABLE_MODULE_EXTRAKEYS;ENABLE_MODULE_SCHNORRSIG;ENABLE_MODULE_ELLSWIFT;%(PreprocessorDefinitions)</PreprocessorDefinitions>
<UndefinePreprocessorDefinitions>USE_ASM_X86_64;%(UndefinePreprocessorDefinitions)</UndefinePreprocessorDefinitions>
<AdditionalIncludeDirectories>..\..\src\secp256k1;%(AdditionalIncludeDirectories)</AdditionalIncludeDirectories>
<DisableSpecificWarnings>4146;4244;4267;4334</DisableSpecificWarnings>
</ClCompile>

View File

@@ -1,15 +0,0 @@
/**********************************************************************
* Copyright (c) 2013, 2014 Pieter Wuille *
* Distributed under the MIT software license, see the accompanying *
* file COPYING or http://www.opensource.org/licenses/mit-license.php.*
**********************************************************************/
#ifndef BITCOIN_LIBSECP256K1_CONFIG_H
#define BITCOIN_LIBSECP256K1_CONFIG_H
#undef USE_ASM_X86_64
#define ECMULT_GEN_PREC_BITS 4
#define ECMULT_WINDOW_SIZE 15
#endif // BITCOIN_LIBSECP256K1_CONFIG_H

View File

@@ -8,6 +8,7 @@
<ConfigurationType>StaticLibrary</ConfigurationType>
</PropertyGroup>
<ItemGroup>
<ClCompile Include="..\..\src\wallet\test\util.cpp" />
@SOURCE_FILES@
</ItemGroup>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />

View File

@@ -1,5 +1,5 @@
#!/usr/bin/env python3
# Copyright (c) 2016-2021 The Bitcoin Core developers
# Copyright (c) 2016-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -111,7 +111,6 @@ def main():
set_properties(vcxproj_filename, '@SOURCE_FILES@\n', content)
parse_config_into_btc_config()
copyfile(os.path.join(SOURCE_DIR,'../build_msvc/bitcoin_config.h'), os.path.join(SOURCE_DIR, 'config/bitcoin-config.h'))
copyfile(os.path.join(SOURCE_DIR,'../build_msvc/libsecp256k1_config.h'), os.path.join(SOURCE_DIR, 'secp256k1/src/libsecp256k1-config.h'))
if __name__ == '__main__':
main()

View File

@@ -10,13 +10,18 @@
<ItemGroup>
<ClCompile Include="..\..\src\init\bitcoin-qt.cpp" />
<ClCompile Include="..\..\src\test\util\setup_common.cpp" />
<ClCompile Include="..\..\src\wallet\test\util.cpp">
<ObjectFileName>$(IntDir)wallet_test_util.obj</ObjectFileName>
</ClCompile>
<ClCompile Include="..\..\src\qt\test\addressbooktests.cpp" />
<ClCompile Include="..\..\src\qt\test\apptests.cpp" />
<ClCompile Include="..\..\src\qt\test\optiontests.cpp" />
<ClCompile Include="..\..\src\qt\test\rpcnestedtests.cpp" />
<ClCompile Include="..\..\src\qt\test\test_main.cpp" />
<ClCompile Include="..\..\src\qt\test\uritests.cpp" />
<ClCompile Include="..\..\src\qt\test\util.cpp" />
<ClCompile Include="..\..\src\qt\test\util.cpp">
<ObjectFileName>$(IntDir)qt_test_util.obj</ObjectFileName>
</ClCompile>
<ClCompile Include="..\..\src\qt\test\wallettests.cpp" />
<ClCompile Include="..\..\src\wallet\test\wallet_test_fixture.cpp" />
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_addressbooktests.cpp" />
@@ -27,7 +32,7 @@
<ClCompile Include="$(GeneratedFilesOutDir)\moc\moc_wallettests.cpp" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_cli\libbitcoin_cli.vcxproj">
@@ -54,6 +59,9 @@
<ProjectReference Include="..\libbitcoin_zmq\libbitcoin_zmq.vcxproj">
<Project>{792d487f-f14c-49fc-a9de-3fc150f31c3f}</Project>
</ProjectReference>
<ProjectReference Include="..\libtest_util\libtest_util.vcxproj">
<Project>{1e065f03-3566-47d0-8fa9-daa72b084e7d}</Project>
</ProjectReference>
<ProjectReference Include="..\libleveldb\libleveldb.vcxproj">
<Project>{18430fef-6b61-4c53-b396-718e02850f1b}</Project>
</ProjectReference>

View File

@@ -22,7 +22,7 @@
<ProjectReference Include="..\libminisketch\libminisketch.vcxproj">
<Project>{542007e3-be0d-4b0d-a6b0-aa8813e2558d}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoinconsensus\libbitcoinconsensus.vcxproj">
<ProjectReference Include="..\libbitcoin_consensus\libbitcoin_consensus.vcxproj">
<Project>{2b384fa8-9ee1-4544-93cb-0d733c25e8ce}</Project>
</ProjectReference>
<ProjectReference Include="..\libbitcoin_cli\libbitcoin_cli.vcxproj">
@@ -68,7 +68,7 @@
<RawTestFile Include="..\..\src\test\data\*.raw" />
</ItemGroup>
<HeaderFromHexdump RawFilePath="%(RawTestFile.FullPath)" HeaderFilePath="%(RawTestFile.FullPath).h" SourceHeader="static unsigned const char %(RawTestFile.Filename)_raw[] = {" SourceFooter="};" />
<HeaderFromHexdump RawFilePath="%(JsonTestFile.FullPath)" HeaderFilePath="%(JsonTestFile.FullPath).h" SourceHeader="namespace json_tests{ static unsigned const char %(JsonTestFile.Filename)[] = {" SourceFooter="};}" />
<HeaderFromHexdump RawFilePath="%(JsonTestFile.FullPath)" HeaderFilePath="%(JsonTestFile.FullPath).h" SourceHeader="#include &lt;string&gt;&#x0D;&#x0A;namespace json_tests{ static const std::string %(JsonTestFile.Filename){" SourceFooter="};}" />
</Target>
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.props" />
<Import Project="$(VCTargetsPath)\Microsoft.Cpp.targets" />

View File

@@ -13,5 +13,12 @@
"features": ["thread"]
},
"zeromq"
],
"builtin-baseline": "f14984af3738e69f197bf0e647a8dca12de92996",
"overrides": [
{
"name": "libevent",
"version": "2.1.12#7"
}
]
}

View File

@@ -8,29 +8,23 @@ Be aware that the tests will be built and run in-place, so please run at your ow
If the repository is not a fresh git clone, you might have to clean files from previous builds or test runs first.
The ci needs to perform various sysadmin tasks such as installing packages or writing to the user's home directory.
While most of the actions are done inside a docker container, this is not possible for all. Thus, cache directories,
such as the depends cache, previous release binaries, or ccache, are mounted as read-write into the docker container. While it should be fine to run
While it should be fine to run
the ci system locally on you development box, the ci scripts can generally be assumed to have received less review and
testing compared to other parts of the codebase. If you want to keep the work tree clean, you might want to run the ci
system in a virtual machine with a Linux operating system of your choice.
To allow for a wide range of tested environments, but also ensure reproducibility to some extent, the test stage
requires `docker` to be installed. To install all requirements on Ubuntu, run
requires `bash`, `docker`, and `python3` to be installed. To install all requirements on Ubuntu, run
```
sudo apt install docker.io bash
sudo apt install bash docker.io python3
```
To run the default test stage,
It is recommended to run the ci system in a clean env. To run the test stage
with a specific configuration,
```
./ci/test_run_all.sh
```
To run the test stage with a specific configuration,
```
FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
```
### Configurations
@@ -45,14 +39,11 @@ the system package manager to install build dependencies. This guarantees that
the tester is using the same versions as the release builds, which also use
`./depends`.
If no `FILE_ENV` has been specified or values are left out, `00_setup_env.sh`
is used as the default configuration with fallback values.
It is also possible to force a specific configuration without modifying the
file. For example,
```
MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh
env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'MAKEJOBS="-j1" FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
```
The files starting with `0n` (`n` greater than 0) are the scripts that are run

View File

@@ -1,22 +1,47 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
${CI_RETRY_EXE} apt-get update
${CI_RETRY_EXE} apt-get install -y clang-format-9 python3-pip curl git gawk jq
update-alternatives --install /usr/bin/clang-format clang-format "$(which clang-format-9 )" 100
update-alternatives --install /usr/bin/clang-format-diff clang-format-diff "$(which clang-format-diff-9)" 100
export PATH=$PWD/ci/retry:$PATH
${CI_RETRY_EXE} pip3 install codespell==2.1.0
${CI_RETRY_EXE} pip3 install flake8==4.0.1
${CI_RETRY_EXE} pip3 install mypy==0.942
${CI_RETRY_EXE} pip3 install pyzmq==22.3.0
${CI_RETRY_EXE} pip3 install vulture==2.3
${CI_RETRY_EXE} apt-get update
# Lint dependencies:
# - curl/xz-utils (to install shellcheck)
# - git (used in many lint scripts)
# - gpg (used by verify-commits)
${CI_RETRY_EXE} apt-get install -y curl xz-utils git gpg
PYTHON_PATH="/python_build"
if [ ! -d "${PYTHON_PATH}/bin" ]; then
(
${CI_RETRY_EXE} git clone https://github.com/pyenv/pyenv.git
cd pyenv/plugins/python-build || exit 1
./install.sh
)
# For dependencies see https://github.com/pyenv/pyenv/wiki#suggested-build-environment
${CI_RETRY_EXE} apt-get install -y build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev \
clang
env CC=clang python-build "$(cat "./.python-version")" "${PYTHON_PATH}"
fi
export PATH="${PYTHON_PATH}/bin:${PATH}"
command -v python3
python3 --version
${CI_RETRY_EXE} pip3 install \
codespell==2.2.5 \
flake8==6.1.0 \
lief==0.13.2 \
mypy==1.4.1 \
pyzmq==25.1.0 \
vulture==2.6
SHELLCHECK_VERSION=v0.8.0
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | tar --xz -xf - --directory /tmp/
export PATH="/tmp/shellcheck-${SHELLCHECK_VERSION}:${PATH}"
curl -sL "https://github.com/koalaman/shellcheck/releases/download/${SHELLCHECK_VERSION}/shellcheck-${SHELLCHECK_VERSION}.linux.x86_64.tar.xz" | \
tar --xz -xf - --directory /tmp/
mv "/tmp/shellcheck-${SHELLCHECK_VERSION}/shellcheck" /usr/bin/

View File

@@ -1,15 +1,25 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C
GIT_HEAD=$(git rev-parse HEAD)
if [ -n "$CIRRUS_PR" ]; then
COMMIT_RANGE="${CIRRUS_BASE_SHA}..$GIT_HEAD"
set -ex
if [ -n "$LOCAL_BRANCH" ]; then
# To faithfully recreate CI linting locally, specify all commits on the current
# branch.
COMMIT_RANGE="$(git merge-base HEAD master)..HEAD"
elif [ -n "$CIRRUS_PR" ]; then
COMMIT_RANGE="HEAD~..HEAD"
echo
git log --no-merges --oneline "$COMMIT_RANGE"
echo
test/lint/commit-script-check.sh "$COMMIT_RANGE"
else
COMMIT_RANGE="SKIP_EMPTY_NOT_A_PR"
fi
export COMMIT_RANGE
@@ -31,11 +41,8 @@ if [ "$CIRRUS_REPO_FULL_NAME" = "bitcoin/bitcoin" ] && [ "$CIRRUS_PR" = "" ] ; t
git log HEAD~10 -1 --format='%H' > ./contrib/verify-commits/trusted-sha512-root-commit
git log HEAD~10 -1 --format='%H' > ./contrib/verify-commits/trusted-git-root
mapfile -t KEYS < contrib/verify-commits/trusted-keys
git config user.email "ci@ci.ci"
git config user.name "ci"
${CI_RETRY_EXE} gpg --keyserver hkps://keys.openpgp.org --recv-keys "${KEYS[@]}" &&
./contrib/verify-commits/verify-commits.py;
fi
if [ -n "$COMMIT_RANGE" ]; then
echo
git log --no-merges --oneline "$COMMIT_RANGE"
fi

19
ci/lint/container-entrypoint.sh Executable file
View File

@@ -0,0 +1,19 @@
#!/usr/bin/env bash
#
# Copyright (c) The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or https://opensource.org/license/mit/.
export LC_ALL=C
# Fixes permission issues when there is a container UID/GID mismatch with the owner
# of the mounted bitcoin src dir.
git config --global --add safe.directory /bitcoin
export PATH="/python_build/bin:${PATH}"
if [ -z "$1" ]; then
LOCAL_BRANCH=1 bash -ic "./ci/lint/06_script.sh"
else
exec "$@"
fi

23
ci/lint_imagefile Normal file
View File

@@ -0,0 +1,23 @@
# Copyright (c) The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or https://opensource.org/license/mit/.
# See test/lint/README.md for usage.
FROM debian:bookworm
ENV DEBIAN_FRONTEND=noninteractive
ENV LC_ALL=C.UTF-8
COPY ./.python-version /.python-version
COPY ./ci/lint/container-entrypoint.sh /entrypoint.sh
COPY ./ci/lint/04_install.sh /install.sh
RUN /install.sh && \
echo 'alias lint="./ci/lint/06_script.sh"' >> ~/.bashrc && \
chmod 755 /entrypoint.sh && \
rm -rf /var/lib/apt/lists/*
WORKDIR /bitcoin
ENTRYPOINT ["/entrypoint.sh"]

View File

@@ -8,4 +8,5 @@ export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/lint/04_install.sh
set -o errexit; source ./ci/lint/06_script.sh
set -o errexit
./ci/lint/06_script.sh

View File

@@ -1,22 +1,30 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
# The root dir.
set -ex
# The source root dir, usually from git, usually read-only.
# The ci system copies this folder.
# This is where the depends build is done.
BASE_ROOT_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )"/../../ >/dev/null 2>&1 && pwd )
export BASE_ROOT_DIR
BASE_READ_ONLY_DIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )"/../../ >/dev/null 2>&1 && pwd )
export BASE_READ_ONLY_DIR
# The destination root dir inside the container.
# This folder will also hold any SDKs.
# This folder only exists on the ci guest and will be a copy of BASE_READ_ONLY_DIR
export BASE_ROOT_DIR="${BASE_ROOT_DIR:-/ci_container_base}"
# The depends dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
# This folder exists only on the ci guest, and on the ci host as a volume.
export DEPENDS_DIR=${DEPENDS_DIR:-$BASE_ROOT_DIR/depends}
# A folder for the ci system to put temporary files (ccache, datadirs for tests, ...)
# This folder only exists on the ci host.
# A folder for the ci system to put temporary files (build result, datadirs for tests, ...)
# This folder only exists on the ci guest.
export BASE_SCRATCH_DIR=${BASE_SCRATCH_DIR:-$BASE_ROOT_DIR/ci/scratch}
# A folder for the ci system to put executables.
# This folder only exists on the ci guest.
export BINS_SCRATCH_DIR="${BASE_SCRATCH_DIR}/bins/"
echo "Setting specific values in env"
if [ -n "${FILE_ENV}" ]; then
@@ -28,46 +36,38 @@ fi
echo "Fallback to default values in env (if not yet set)"
# The number of parallel jobs to pass down to make and test_runner.py
export MAKEJOBS=${MAKEJOBS:--j4}
# What host to compile for. See also ./depends/README.md
# Tests that need cross-compilation export the appropriate HOST.
# Tests that run natively guess the host
export HOST=${HOST:-$("$BASE_ROOT_DIR/depends/config.guess")}
# Whether to prefer BusyBox over GNU utilities
export USE_BUSY_BOX=${USE_BUSY_BOX:-false}
export RUN_UNIT_TESTS=${RUN_UNIT_TESTS:-true}
export RUN_FUNCTIONAL_TESTS=${RUN_FUNCTIONAL_TESTS:-true}
export RUN_TIDY=${RUN_TIDY:-false}
export RUN_SECURITY_TESTS=${RUN_SECURITY_TESTS:-false}
# By how much to scale the test_runner timeouts (option --timeout-factor).
# This is needed because some ci machines have slow CPU or disk, so sanitizers
# might be slow or a reindex might be waiting on disk IO.
export TEST_RUNNER_TIMEOUT_FACTOR=${TEST_RUNNER_TIMEOUT_FACTOR:-40}
export TEST_RUNNER_ENV=${TEST_RUNNER_ENV:-}
export RUN_FUZZ_TESTS=${RUN_FUZZ_TESTS:-false}
export EXPECTED_TESTS_DURATION_IN_SECONDS=${EXPECTED_TESTS_DURATION_IN_SECONDS:-1000}
export CONTAINER_NAME=${CONTAINER_NAME:-ci_unnamed}
export DOCKER_NAME_TAG=${DOCKER_NAME_TAG:-ubuntu:20.04}
# Randomize test order.
# See https://www.boost.org/doc/libs/1_71_0/libs/test/doc/html/boost_test/utf_reference/rt_param_reference/random.html
export BOOST_TEST_RANDOM=${BOOST_TEST_RANDOM:-1}
# See man 7 debconf
export DEBIAN_FRONTEND=noninteractive
export CCACHE_SIZE=${CCACHE_SIZE:-100M}
export CCACHE_MAXSIZE=${CCACHE_MAXSIZE:-100M}
export CCACHE_TEMPDIR=${CCACHE_TEMPDIR:-/tmp/.ccache-temp}
export CCACHE_COMPRESS=${CCACHE_COMPRESS:-1}
# The cache dir.
# This folder exists on the ci host and ci guest. Changes are propagated back and forth.
# This folder exists only on the ci guest, and on the ci host as a volume.
export CCACHE_DIR=${CCACHE_DIR:-$BASE_SCRATCH_DIR/.ccache}
# Folder where the build result is put (bin and lib).
export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_SCRATCH_DIR/out/$HOST}
export BASE_OUTDIR=${BASE_OUTDIR:-$BASE_SCRATCH_DIR/out}
# Folder where the build is done (dist and out-of-tree build).
export BASE_BUILD_DIR=${BASE_BUILD_DIR:-$BASE_SCRATCH_DIR/build}
export PREVIOUS_RELEASES_DIR=${PREVIOUS_RELEASES_DIR:-$BASE_ROOT_DIR/releases/$HOST}
export SDK_URL=${SDK_URL:-https://bitcoincore.org/depends-sources/sdks}
export DOCKER_PACKAGES=${DOCKER_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps bison}
# The folder for previous release binaries.
# This folder exists only on the ci guest, and on the ci host as a volume.
export PREVIOUS_RELEASES_DIR=${PREVIOUS_RELEASES_DIR:-$BASE_ROOT_DIR/prev_releases}
export CI_BASE_PACKAGES=${CI_BASE_PACKAGES:-build-essential libtool autotools-dev automake pkg-config bsdmainutils curl ca-certificates ccache python3 rsync git procps bison e2fsprogs}
export GOAL=${GOAL:-install}
export DIR_QA_ASSETS=${DIR_QA_ASSETS:-${BASE_SCRATCH_DIR}/qa-assets}
export PATH=${BASE_ROOT_DIR}/ci/retry:$PATH
export CI_RETRY_EXE=${CI_RETRY_EXE:-"retry --"}

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -9,7 +9,7 @@ export LC_ALL=C.UTF-8
export HOST=aarch64-linux-android
export PACKAGES="unzip openjdk-8-jdk gradle"
export CONTAINER_NAME=ci_android
export DOCKER_NAME_TAG="ubuntu:focal"
export CI_IMAGE_NAME_TAG="docker.io/amd64/ubuntu:22.04"
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false

View File

@@ -7,18 +7,10 @@
export LC_ALL=C.UTF-8
export HOST=arm-linux-gnueabihf
# The host arch is unknown, so we run the tests through qemu.
# If the host is arm and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
if [ -z ${QEMU_USER_CMD+x} ]; then export QEMU_USER_CMD="${QEMU_USER_CMD:-"qemu-arm -L /usr/arm-linux-gnueabihf/"}"; fi
export DPKG_ADD_ARCH="armhf"
export PACKAGES="python3-zmq g++-arm-linux-gnueabihf busybox libc6:armhf libstdc++6:armhf libfontconfig1:armhf libxcb1:armhf"
if [ -n "$QEMU_USER_CMD" ]; then
# Likely cross-compiling, so install the needed gcc and qemu-user
export PACKAGES="$PACKAGES qemu-user"
fi
export CONTAINER_NAME=ci_arm_linux
# Use debian to avoid 404 apt errors when cross compiling
export DOCKER_NAME_TAG="debian:bullseye"
export CI_IMAGE_NAME_TAG="docker.io/arm64v8/debian:bookworm"
export USE_BUSY_BOX=true
export RUN_UNIT_TESTS=true
export RUN_FUNCTIONAL_TESTS=false

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -8,9 +8,10 @@ export LC_ALL=C.UTF-8
export HOST=i686-pc-linux-gnu
export CONTAINER_NAME=ci_i686_centos
export DOCKER_NAME_TAG=quay.io/centos/centos:stream8
export DOCKER_PACKAGES="gcc-c++ glibc-devel.x86_64 libstdc++-devel.x86_64 glibc-devel.i686 libstdc++-devel.i686 ccache libtool make git python3 python3-zmq which patch lbzip2 xz procps-ng dash rsync coreutils bison"
export CI_IMAGE_NAME_TAG="quay.io/centos/amd64:stream9"
export CI_BASE_PACKAGES="gcc-c++ glibc-devel.x86_64 libstdc++-devel.x86_64 glibc-devel.i686 libstdc++-devel.i686 ccache libtool make git python3 python3-pip which patch lbzip2 xz procps-ng dash rsync coreutils bison util-linux e2fsprogs"
export PIP_PACKAGES="pyzmq"
export GOAL="install"
export NO_WERROR=1 # Suppress error: #warning _FORTIFY_SOURCE > 2 is treated like 2 on this platform [-Werror=cpp]
export BITCOIN_CONFIG="--enable-zmq --with-gui=qt5 --enable-reduce-exports"
export CONFIG_SHELL="/bin/dash"
export TEST_RUNNER_ENV="LC_ALL=en_US.UTF-8"

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
@@ -8,10 +8,10 @@ export LC_ALL=C.UTF-8
export HOST=i686-pc-linux-gnu
export CONTAINER_NAME=ci_i686_multiprocess
export DOCKER_NAME_TAG=ubuntu:20.04
export PACKAGES="cmake python3 python3-pip llvm clang g++-multilib"
export CI_IMAGE_NAME_TAG="docker.io/amd64/ubuntu:20.04"
export PACKAGES="cmake llvm clang g++-multilib"
export DEP_OPTS="DEBUG=1 MULTIPROCESS=1"
export GOAL="install"
export BITCOIN_CONFIG="--enable-debug CC='clang -m32' CXX='clang++ -m32' LDFLAGS='--rtlib=compiler-rt -lgcc_s'"
export BITCOIN_CONFIG="--enable-debug CC='clang -m32' CXX='clang++ -m32' \
LDFLAGS='--rtlib=compiler-rt -lgcc_s' CPPFLAGS='-DBOOST_MULTI_INDEX_ENABLE_SAFE_MODE'"
export TEST_RUNNER_ENV="BITCOIND=bitcoin-node"
export TEST_RUNNER_EXTRA="--nosandbox"

View File

@@ -1,18 +1,23 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export SDK_URL=${SDK_URL:-https://bitcoincore.org/depends-sources/sdks}
export CONTAINER_NAME=ci_macos_cross
export DOCKER_NAME_TAG=ubuntu:20.04 # Check that Focal can cross-compile to macos
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:22.04"
export HOST=x86_64-apple-darwin
export PACKAGES="cmake libz-dev libtinfo5 python3-setuptools xorriso"
export PACKAGES="cmake libz-dev python3-setuptools zip"
export XCODE_VERSION=12.2
export XCODE_BUILD_ID=12B45b
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export GOAL="deploy"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports"
# False-positive warning is fixed with clang 17, remove this when that version
# can be used.
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports LDFLAGS=-Wno-error=unused-command-line-argument"

View File

@@ -0,0 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-apple-darwin
# Homebrew's python@3.12 is marked as externally managed (PEP 668).
# Therefore, `--break-system-packages` is needed.
export PIP_PACKAGES="--break-system-packages zmq"
export GOAL="install"
export BITCOIN_CONFIG="--with-gui --with-miniupnpc --with-natpmp --enable-reduce-exports"
export CI_OS_NAME="macos"
export NO_DEPENDS=1
export OSX_SDK=""
export CCACHE_MAXSIZE=400M
export RUN_FUZZ_TESTS=true
export FUZZ_TESTS_CONFIG="--exclude banman" # https://github.com/bitcoin/bitcoin/issues/27924

View File

@@ -1,17 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=x86_64-apple-darwin
export PIP_PACKAGES="zmq lief"
export GOAL="install"
export BITCOIN_CONFIG="--with-gui --enable-reduce-exports"
export CI_OS_NAME="macos"
export NO_DEPENDS=1
export OSX_SDK=""
export CCACHE_SIZE=300M
export RUN_SECURITY_TESTS="true"

View File

@@ -1,19 +1,26 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
# We install an up-to-date 'bpfcc-tools' package from an untrusted PPA.
# This can be dropped with the next Ubuntu or Debian release that includes up-to-date packages.
# See the if-then in ci/test/04_install.sh too.
export ADD_UNTRUSTED_BPFCC_PPA=true
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
# Only install BCC tracing packages in Cirrus CI.
if [[ "${CIRRUS_CI}" == "true" ]]; then
BPFCC_PACKAGE="bpfcc-tools linux-headers-$(uname --kernel-release)"
export CI_CONTAINER_CAP="--privileged -v /sys/kernel:/sys/kernel:rw"
else
BPFCC_PACKAGE=""
export CI_CONTAINER_CAP="--cap-add SYS_PTRACE" # If run with (ASan + LSan), the container needs access to ptrace (https://github.com/google/sanitizers/issues/764)
fi
export CONTAINER_NAME=ci_native_asan
export PACKAGES="systemtap-sdt-dev bpfcc-tools clang llvm python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev bsdmainutils libboost-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev"
export DOCKER_NAME_TAG=ubuntu:22.04 # May not run in docker unless --enable-usdt is dropped
export PACKAGES="systemtap-sdt-dev clang-17 llvm-17 libclang-rt-17-dev python3-zmq qtbase5-dev qttools5-dev-tools libevent-dev libboost-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libqrencode-dev libsqlite3-dev ${BPFCC_PACKAGE}"
export NO_DEPENDS=1
export GOAL="install"
export BITCOIN_CONFIG="--enable-usdt --enable-zmq --with-incompatible-bdb --with-gui=qt5 CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' --with-sanitizers=address,integer,undefined CC=clang CXX=clang++"
export BITCOIN_CONFIG="--enable-c++20 --enable-usdt --enable-zmq --with-incompatible-bdb --with-gui=qt5 \
CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER' \
--with-sanitizers=address,float-divide-by-zero,integer,undefined \
CC='clang-17 -ftrivial-auto-var-init=pattern' CXX='clang++-17 -ftrivial-auto-var-init=pattern'"

View File

@@ -1,18 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:22.04"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
export CONTAINER_NAME=ci_native_fuzz
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-dev libsqlite3-dev"
export PACKAGES="clang-17 llvm-17 libclang-rt-17-dev libevent-dev libboost-dev libsqlite3-dev"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export RUN_FUZZ_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,address,undefined,integer CC='clang -ftrivial-auto-var-init=pattern' CXX='clang++ -ftrivial-auto-var-init=pattern'"
export CCACHE_SIZE=200M
export CI_CONTAINER_CAP="--cap-add SYS_PTRACE" # If run with (ASan + LSan), the container needs access to ptrace (https://github.com/google/sanitizers/issues/764)
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,address,undefined,float-divide-by-zero,integer \
CC='clang-17 -ftrivial-auto-var-init=pattern' CXX='clang++-17 -ftrivial-auto-var-init=pattern'"
export CCACHE_MAXSIZE=200M

View File

@@ -1,25 +1,25 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
LIBCXX_DIR="${BASE_SCRATCH_DIR}/msan/build/"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:22.04"
LIBCXX_DIR="/msan/cxx_build/"
export MSAN_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls"
LIBCXX_FLAGS="-nostdinc++ -stdlib=libc++ -L${LIBCXX_DIR}lib -lc++abi -I${LIBCXX_DIR}include -I${LIBCXX_DIR}include/c++/v1 -lpthread -Wl,-rpath,${LIBCXX_DIR}lib -Wno-unused-command-line-argument"
LIBCXX_FLAGS="-nostdinc++ -nostdlib++ -isystem ${LIBCXX_DIR}include/c++/v1 -L${LIBCXX_DIR}lib -Wl,-rpath,${LIBCXX_DIR}lib -lc++ -lc++abi -lpthread -Wno-unused-command-line-argument"
export MSAN_AND_LIBCXX_FLAGS="${MSAN_FLAGS} ${LIBCXX_FLAGS}"
export CONTAINER_NAME="ci_native_msan"
export PACKAGES="clang-12 llvm-12 cmake"
export CONTAINER_NAME="ci_native_fuzz_msan"
export PACKAGES="cmake ninja-build"
# BDB generates false-positives and will be removed in future
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' libevent_cflags='${MSAN_FLAGS}' sqlite_cflags='${MSAN_FLAGS}' zeromq_cxxflags='-std=c++17 ${MSAN_AND_LIBCXX_FLAGS}'"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,memory --disable-hardening --with-asm=no --prefix=${DEPENDS_DIR}/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer,memory --disable-hardening --with-asm=no CFLAGS='${MSAN_FLAGS}' CPPFLAGS='-DBOOST_MULTI_INDEX_ENABLE_SAFE_MODE' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export USE_MEMORY_SANITIZER="true"
export RUN_UNIT_TESTS="false"
export RUN_FUNCTIONAL_TESTS="false"
export RUN_FUZZ_TESTS=true
export CCACHE_SIZE=250M
export CCACHE_MAXSIZE=250M

View File

@@ -1,20 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:22.04"
export CI_IMAGE_NAME_TAG="docker.io/debian:bookworm"
export CONTAINER_NAME=ci_native_fuzz_valgrind
export PACKAGES="clang llvm python3 libevent-dev bsdmainutils libboost-dev libsqlite3-dev valgrind"
export PACKAGES="clang llvm libclang-rt-dev libevent-dev libboost-dev libsqlite3-dev valgrind"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export RUN_FUZZ_TESTS=true
export FUZZ_TESTS_CONFIG="--valgrind"
export GOAL="install"
# Temporarily pin dwarf 4, until valgrind can understand clang's dwarf 5
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer CC=clang CXX=clang++ CFLAGS='-gdwarf-4' CXXFLAGS='-gdwarf-4'"
export CCACHE_SIZE=200M
# Temporarily pin dwarf 4, until using Valgrind 3.20 or later
export BITCOIN_CONFIG="--enable-fuzz --with-sanitizers=fuzzer CC='clang -gdwarf-4' CXX='clang++ -gdwarf-4'"
export CCACHE_MAXSIZE=200M

View File

@@ -1,23 +1,23 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:20.04"
LIBCXX_DIR="${BASE_SCRATCH_DIR}/msan/build/"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:22.04"
LIBCXX_DIR="/msan/cxx_build/"
export MSAN_FLAGS="-fsanitize=memory -fsanitize-memory-track-origins=2 -fno-omit-frame-pointer -g -O1 -fno-optimize-sibling-calls"
LIBCXX_FLAGS="-nostdinc++ -stdlib=libc++ -L${LIBCXX_DIR}lib -lc++abi -I${LIBCXX_DIR}include -I${LIBCXX_DIR}include/c++/v1 -lpthread -Wl,-rpath,${LIBCXX_DIR}lib -Wno-unused-command-line-argument"
LIBCXX_FLAGS="-nostdinc++ -nostdlib++ -isystem ${LIBCXX_DIR}include/c++/v1 -L${LIBCXX_DIR}lib -Wl,-rpath,${LIBCXX_DIR}lib -lc++ -lc++abi -lpthread -Wno-unused-command-line-argument"
export MSAN_AND_LIBCXX_FLAGS="${MSAN_FLAGS} ${LIBCXX_FLAGS}"
export CONTAINER_NAME="ci_native_msan"
export PACKAGES="clang-12 llvm-12 cmake"
export PACKAGES="cmake ninja-build"
# BDB generates false-positives and will be removed in future
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC='clang' CXX='clang++' CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}' libevent_cflags='${MSAN_FLAGS}' sqlite_cflags='${MSAN_FLAGS}' zeromq_cxxflags='-std=c++17 ${MSAN_AND_LIBCXX_FLAGS}'"
export DEP_OPTS="NO_BDB=1 NO_QT=1 CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export GOAL="install"
export BITCOIN_CONFIG="--with-sanitizers=memory --disable-hardening --with-asm=no --prefix=${DEPENDS_DIR}/x86_64-pc-linux-gnu/ CC=clang CXX=clang++ CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export BITCOIN_CONFIG="--with-sanitizers=memory --disable-hardening --with-asm=no CFLAGS='${MSAN_FLAGS}' CXXFLAGS='${MSAN_AND_LIBCXX_FLAGS}'"
export USE_MEMORY_SANITIZER="true"
export RUN_FUNCTIONAL_TESTS="false"
export CCACHE_SIZE=250M
export CCACHE_MAXSIZE=250M

View File

@@ -1,14 +1,15 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_nowallet_libbitcoinkernel
export DOCKER_NAME_TAG=ubuntu:18.04 # Use bionic to have one config run the tests in python3.6, see doc/dependencies.md
export PACKAGES="python3-zmq clang-8 llvm-8 libc++abi-8-dev libc++-8-dev" # Use clang-8 to test C++17 compatibility, see doc/dependencies.md
export DEP_OPTS="NO_WALLET=1 CC=clang-8 CXX='clang++-8 -stdlib=libc++'"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:20.04"
# Use minimum supported python3.8 and clang-10, see doc/dependencies.md
export PACKAGES="python3-zmq clang-10 llvm-10 libc++abi-10-dev libc++-10-dev"
export DEP_OPTS="NO_WALLET=1 CC=clang-10 CXX='clang++-10 -stdlib=libc++'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-reduce-exports CC=clang-8 CXX='clang++-8 -stdlib=libc++' --enable-experimental-util-chainstate --with-experimental-kernel-lib --enable-shared"
export BITCOIN_CONFIG="--enable-reduce-exports --enable-experimental-util-chainstate --with-experimental-kernel-lib --enable-shared"

View File

@@ -1,19 +1,21 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_qt5
export DOCKER_NAME_TAG=debian:buster # Check that buster gcc-8 can compile our C++17 and run our functional tests in python3, see doc/dependencies.md
export PACKAGES="gcc-8 g++-8 python3-zmq qtbase5-dev qttools5-dev-tools libdbus-1-dev libharfbuzz-dev"
export DEP_OPTS="NO_QT=1 NO_UPNP=1 NO_NATPMP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1 CC=gcc-8 CXX=g++-8"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:20.04"
# Use minimum supported python3.8 and gcc-9, see doc/dependencies.md
export PACKAGES="gcc-9 g++-9 python3-zmq qtbase5-dev qttools5-dev-tools libdbus-1-dev libharfbuzz-dev"
export DEP_OPTS="NO_QT=1 NO_UPNP=1 NO_NATPMP=1 DEBUG=1 ALLOW_HOST_PACKAGES=1 CC=gcc-9 CXX=g++-9"
export TEST_RUNNER_EXTRA="--previous-releases --coverage --extended --exclude feature_dbcrash" # Run extended tests so that coverage does not fail, but exclude the very slow dbcrash
export RUN_UNIT_TESTS_SEQUENTIAL="true"
export RUN_UNIT_TESTS="false"
export GOAL="install"
export NO_WERROR=1 # -Werror=maybe-uninitialized
export DOWNLOAD_PREVIOUS_RELEASES="true"
export BITCOIN_CONFIG="--enable-zmq --with-libs=no --with-gui=qt5 --enable-reduce-exports \
--enable-debug CFLAGS=\"-g0 -O2 -funsigned-char\" CXXFLAGS=\"-g0 -O2 -funsigned-char\" CC=gcc-8 CXX=g++-8"
export BITCOIN_CONFIG="--enable-zmq --with-libs=no --with-gui=qt5 --enable-reduce-exports --enable-debug \
CFLAGS=\"-g0 -O2 -funsigned-char\" CPPFLAGS='-DBOOST_MULTI_INDEX_ENABLE_SAFE_MODE' CXXFLAGS=\"-g0 -O2 -funsigned-char\""

View File

@@ -1,19 +1,20 @@
#!/usr/bin/env bash
#
# Copyright (c) 2022 The Bitcoin Core developers
# Copyright (c) 2023-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:22.04"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
export CONTAINER_NAME=ci_native_tidy
export PACKAGES="clang libclang-dev llvm-dev clang-tidy bear cmake libevent-dev libboost-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev systemtap-sdt-dev libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libqrencode-dev libsqlite3-dev libdb++-dev"
export TIDY_LLVM_V="17"
export PACKAGES="clang-${TIDY_LLVM_V} libclang-${TIDY_LLVM_V}-dev llvm-${TIDY_LLVM_V}-dev libomp-${TIDY_LLVM_V}-dev clang-tidy-${TIDY_LLVM_V} jq bear cmake libevent-dev libboost-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev systemtap-sdt-dev qtbase5-dev qttools5-dev qttools5-dev-tools libqrencode-dev libsqlite3-dev libdb++-dev"
export NO_DEPENDS=1
export RUN_UNIT_TESTS=false
export RUN_FUNCTIONAL_TESTS=false
export RUN_FUZZ_TESTS=false
export RUN_TIDY=true
export GOAL="install"
export BITCOIN_CONFIG="CC=clang CXX=clang++ --enable-c++20 --with-incompatible-bdb --disable-hardening CFLAGS='-O0 -g0' CXXFLAGS='-O0 -g0'"
export CCACHE_SIZE=200M
export BITCOIN_CONFIG="CC=clang-${TIDY_LLVM_V} CXX=clang++-${TIDY_LLVM_V} --with-incompatible-bdb --disable-hardening CFLAGS='-O0 -g0' CXXFLAGS='-O0 -g0 -I/usr/lib/llvm-${TIDY_LLVM_V}/lib/clang/${TIDY_LLVM_V}/include'"
export CCACHE_MAXSIZE=200M

View File

@@ -1,14 +1,14 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_native_tsan
export DOCKER_NAME_TAG=ubuntu:22.04
export PACKAGES="clang-13 llvm-13 libc++abi-13-dev libc++-13-dev python3-zmq"
export DEP_OPTS="CC=clang-13 CXX='clang++-13 -stdlib=libc++'"
export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
export PACKAGES="clang-17 llvm-17 libclang-rt-17-dev libc++abi-17-dev libc++-17-dev python3-zmq"
export DEP_OPTS="CC=clang-17 CXX='clang++-17 -stdlib=libc++'"
export GOAL="install"
export BITCOIN_CONFIG="--enable-zmq CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER -DDEBUG_LOCKCONTENTION' CXXFLAGS='-g' --with-sanitizers=thread CC=clang-13 CXX='clang++-13 -stdlib=libc++'"
export BITCOIN_CONFIG="--enable-zmq CPPFLAGS='-DARENA_DEBUG -DDEBUG_LOCKORDER -DDEBUG_LOCKCONTENTION' CXXFLAGS='-g' --with-sanitizers=thread"

View File

@@ -1,17 +1,17 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export DOCKER_NAME_TAG="ubuntu:22.04"
export CI_IMAGE_NAME_TAG="docker.io/debian:bookworm"
export CONTAINER_NAME=ci_native_valgrind
export PACKAGES="valgrind clang llvm python3-zmq libevent-dev bsdmainutils libboost-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libsqlite3-dev"
export PACKAGES="valgrind clang llvm libclang-rt-dev python3-zmq libevent-dev libboost-dev libdb5.3++-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev libsqlite3-dev"
export USE_VALGRIND=1
export NO_DEPENDS=1
export TEST_RUNNER_EXTRA="--nosandbox --exclude feature_init,rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
export TEST_RUNNER_EXTRA="--exclude feature_init,rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
export GOAL="install"
# Temporarily pin dwarf 4, until valgrind can understand clang's dwarf 5
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no CC=clang CXX=clang++ CFLAGS='-gdwarf-4' CXXFLAGS='-gdwarf-4'" # TODO enable GUI
# Temporarily pin dwarf 4, until using Valgrind 3.20 or later
export BITCOIN_CONFIG="--enable-zmq --with-incompatible-bdb --with-gui=no CC='clang -gdwarf-4' CXX='clang++ -gdwarf-4'" # TODO enable GUI

View File

@@ -1,26 +1,16 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export HOST=s390x-linux-gnu
# The host arch is unknown, so we run the tests through qemu.
# If the host is s390x and wants to run the tests natively, it can set QEMU_USER_CMD to the empty string.
if [ -z ${QEMU_USER_CMD+x} ]; then export QEMU_USER_CMD="${QEMU_USER_CMD:-"qemu-s390x"}"; fi
export PACKAGES="python3-zmq"
if [ -n "$QEMU_USER_CMD" ]; then
# Likely cross-compiling, so install the needed gcc and qemu-user
export DPKG_ADD_ARCH="s390x"
export PACKAGES="$PACKAGES g++-s390x-linux-gnu qemu-user libc6:s390x libstdc++6:s390x"
fi
# Use debian to avoid 404 apt errors
export CONTAINER_NAME=ci_s390x
export DOCKER_NAME_TAG="debian:bookworm"
export TEST_RUNNER_ENV="LC_ALL=C"
export TEST_RUNNER_EXTRA="--exclude feature_init,rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
export CI_IMAGE_NAME_TAG="docker.io/s390x/ubuntu:24.04"
export TEST_RUNNER_EXTRA="--exclude rpc_bind,feature_bind_extra" # Excluded for now, see https://github.com/bitcoin/bitcoin/issues/17765#issuecomment-602068547
export RUN_FUNCTIONAL_TESTS=true
export GOAL="install"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-gui-tests" # GUI tests disabled for now, see https://github.com/bitcoin/bitcoin/issues/23730
export BITCOIN_CONFIG="--enable-reduce-exports"

View File

@@ -1,16 +1,19 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2021 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CONTAINER_NAME=ci_win64
export DOCKER_NAME_TAG=ubuntu:22.04 # Check that Jammy can cross-compile to win64
export CI_IMAGE_NAME_TAG="docker.io/amd64/ubuntu:22.04" # Check that Jammy can cross-compile to win64
export HOST=x86_64-w64-mingw32
export DPKG_ADD_ARCH="i386"
export PACKAGES="python3 nsis g++-mingw-w64-x86-64-posix wine-binfmt wine64 wine32 file"
export PACKAGES="nsis g++-mingw-w64-x86-64-posix wine-binfmt wine64 wine32 file"
export RUN_FUNCTIONAL_TESTS=false
export GOAL="deploy"
export BITCOIN_CONFIG="--enable-reduce-exports --disable-external-signer --disable-gui-tests"
# Prior to 11.0.0, the mingw-w64 headers were missing noreturn attributes, causing warnings when
# cross-compiling for Windows. https://sourceforge.net/p/mingw-w64/bugs/306/
# https://github.com/mingw-w64/mingw-w64/commit/1690994f515910a31b9fb7c7bd3a52d4ba987abe
export BITCOIN_CONFIG="--enable-reduce-exports --enable-external-signer --disable-gui-tests CXXFLAGS=-Wno-return-type"

99
ci/test/01_base_install.sh Executable file
View File

@@ -0,0 +1,99 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -ex
CFG_DONE="ci.base-install-done" # Use a global git setting to remember whether this script ran to avoid running it twice
if [ "$(git config --global ${CFG_DONE})" == "true" ]; then
echo "Skip base install"
exit 0
fi
if [ -n "$DPKG_ADD_ARCH" ]; then
dpkg --add-architecture "$DPKG_ADD_ARCH"
fi
if [[ $CI_IMAGE_NAME_TAG == *centos* ]]; then
bash -c "dnf -y install epel-release"
bash -c "dnf -y --allowerasing install $CI_BASE_PACKAGES $PACKAGES"
elif [ "$CI_OS_NAME" != "macos" ]; then
if [[ -n "${APPEND_APT_SOURCES_LIST}" ]]; then
echo "${APPEND_APT_SOURCES_LIST}" >> /etc/apt/sources.list
fi
${CI_RETRY_EXE} apt-get update
${CI_RETRY_EXE} bash -c "apt-get install --no-install-recommends --no-upgrade -y $PACKAGES $CI_BASE_PACKAGES"
fi
if [ -n "$PIP_PACKAGES" ]; then
# shellcheck disable=SC2086
${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
${CI_RETRY_EXE} git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-17.0.2 /msan/llvm-project
cmake -G Ninja -B /msan/clang_build/ \
-DLLVM_ENABLE_PROJECTS="clang" \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_TARGETS_TO_BUILD=Native \
-DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind" \
-S /msan/llvm-project/llvm
ninja -C /msan/clang_build/ "-j$( nproc )" # Use nproc, because MAKEJOBS is the default in docker image builds
ninja -C /msan/clang_build/ install-runtimes
update-alternatives --install /usr/bin/clang++ clang++ /msan/clang_build/bin/clang++ 100
update-alternatives --install /usr/bin/clang clang /msan/clang_build/bin/clang 100
update-alternatives --install /usr/bin/llvm-symbolizer llvm-symbolizer /msan/clang_build/bin/llvm-symbolizer 100
cmake -G Ninja -B /msan/cxx_build/ \
-DLLVM_ENABLE_RUNTIMES='libcxx;libcxxabi' \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_USE_SANITIZER=MemoryWithOrigins \
-DCMAKE_C_COMPILER=clang \
-DCMAKE_CXX_COMPILER=clang++ \
-DLLVM_TARGETS_TO_BUILD=Native \
-DLLVM_ENABLE_PER_TARGET_RUNTIME_DIR=OFF \
-DLIBCXX_HARDENING_MODE=debug \
-S /msan/llvm-project/runtimes
ninja -C /msan/cxx_build/ "-j$( nproc )" # Use nproc, because MAKEJOBS is the default in docker image builds
fi
if [[ "${RUN_TIDY}" == "true" ]]; then
${CI_RETRY_EXE} git clone https://github.com/include-what-you-use/include-what-you-use -b master /include-what-you-use
git -C /include-what-you-use checkout a138eaac254e5a472464e31d5ec418fe6e6f1fc7
cmake -B /iwyu-build/ -G 'Unix Makefiles' -DCMAKE_PREFIX_PATH=/usr/lib/llvm-"${TIDY_LLVM_V}" -S /include-what-you-use
make -C /iwyu-build/ install "-j$( nproc )" # Use nproc, because MAKEJOBS is the default in docker image builds
fi
mkdir -p "${DEPENDS_DIR}/SDKs" "${DEPENDS_DIR}/sdk-sources"
OSX_SDK_BASENAME="Xcode-${XCODE_VERSION}-${XCODE_BUILD_ID}-extracted-SDK-with-libcxx-headers"
if [ -n "$XCODE_VERSION" ] && [ ! -d "${DEPENDS_DIR}/SDKs/${OSX_SDK_BASENAME}" ]; then
OSX_SDK_FILENAME="${OSX_SDK_BASENAME}.tar.gz"
OSX_SDK_PATH="${DEPENDS_DIR}/sdk-sources/${OSX_SDK_FILENAME}"
if [ ! -f "$OSX_SDK_PATH" ]; then
${CI_RETRY_EXE} curl --location --fail "${SDK_URL}/${OSX_SDK_FILENAME}" -o "$OSX_SDK_PATH"
fi
tar -C "${DEPENDS_DIR}/SDKs" -xf "$OSX_SDK_PATH"
fi
if [ -n "$ANDROID_HOME" ] && [ ! -d "$ANDROID_HOME" ]; then
ANDROID_TOOLS_PATH=${DEPENDS_DIR}/sdk-sources/android-tools.zip
if [ ! -f "$ANDROID_TOOLS_PATH" ]; then
${CI_RETRY_EXE} curl --location --fail "${ANDROID_TOOLS_URL}" -o "$ANDROID_TOOLS_PATH"
fi
mkdir -p "$ANDROID_HOME"
unzip -o "$ANDROID_TOOLS_PATH" -d "$ANDROID_HOME"
yes | "${ANDROID_HOME}"/cmdline-tools/bin/sdkmanager --sdk_root="${ANDROID_HOME}" --install "build-tools;${ANDROID_BUILD_TOOLS_VERSION}" "platform-tools" "platforms;android-31" "platforms;android-${ANDROID_API_LEVEL}" "ndk;${ANDROID_NDK_VERSION}"
fi
git config --global ${CFG_DONE} "true"

92
ci/test/02_run_container.sh Executable file
View File

@@ -0,0 +1,92 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
export CI_IMAGE_LABEL="bitcoin-ci-test"
set -ex
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
# Export all env vars to avoid missing some.
# Though, exclude those with newlines to avoid parsing problems.
python3 -c 'import os; [print(f"{key}={value}") for key, value in os.environ.items() if "\n" not in value and "HOME" != key and "PATH" != key and "USER" != key]' | tee /tmp/env
# System-dependent env vars must be kept as is. So read them from the container.
docker run --rm "${CI_IMAGE_NAME_TAG}" bash -c "env | grep --extended-regexp '^(HOME|PATH|USER)='" | tee --append /tmp/env
echo "Creating $CI_IMAGE_NAME_TAG container to run in"
DOCKER_BUILDKIT=1 docker build \
--file "${BASE_READ_ONLY_DIR}/ci/test_imagefile" \
--build-arg "CI_IMAGE_NAME_TAG=${CI_IMAGE_NAME_TAG}" \
--build-arg "FILE_ENV=${FILE_ENV}" \
--label="${CI_IMAGE_LABEL}" \
--tag="${CONTAINER_NAME}" \
"${BASE_READ_ONLY_DIR}"
docker volume create "${CONTAINER_NAME}_ccache" || true
docker volume create "${CONTAINER_NAME}_depends" || true
docker volume create "${CONTAINER_NAME}_depends_sources" || true
docker volume create "${CONTAINER_NAME}_depends_SDKs_android" || true
docker volume create "${CONTAINER_NAME}_previous_releases" || true
if [ -n "${RESTART_CI_DOCKER_BEFORE_RUN}" ] ; then
echo "Restart docker before run to stop and clear all containers started with --rm"
podman container rm --force --all # Similar to "systemctl restart docker"
# Still prune everything in case the filtered pruning doesn't work, or if labels were not set
# on a previous run. Belt and suspenders approach, should be fine to remove in the future.
# Prune images used by --external containers (e.g. build containers) when
# using podman.
echo "Prune all dangling images"
podman image prune --force --external
fi
echo "Prune all dangling $CI_IMAGE_LABEL images"
# When detecting podman-docker, `--external` should be added.
docker image prune --force --filter "label=$CI_IMAGE_LABEL"
# shellcheck disable=SC2086
CI_CONTAINER_ID=$(docker run --cap-add LINUX_IMMUTABLE $CI_CONTAINER_CAP --rm --interactive --detach --tty \
--mount "type=bind,src=$BASE_READ_ONLY_DIR,dst=$BASE_READ_ONLY_DIR,readonly" \
--mount "type=volume,src=${CONTAINER_NAME}_ccache,dst=$CCACHE_DIR" \
--mount "type=volume,src=${CONTAINER_NAME}_depends,dst=$DEPENDS_DIR/built" \
--mount "type=volume,src=${CONTAINER_NAME}_depends_sources,dst=$DEPENDS_DIR/sources" \
--mount "type=volume,src=${CONTAINER_NAME}_depends_SDKs_android,dst=$DEPENDS_DIR/SDKs/android" \
--mount "type=volume,src=${CONTAINER_NAME}_previous_releases,dst=$PREVIOUS_RELEASES_DIR" \
--env-file /tmp/env \
--name "$CONTAINER_NAME" \
"$CONTAINER_NAME")
export CI_CONTAINER_ID
export CI_EXEC_CMD_PREFIX="docker exec ${CI_CONTAINER_ID}"
else
echo "Running on host system without docker wrapper"
echo "Create missing folders"
mkdir -p "${CCACHE_DIR}"
mkdir -p "${PREVIOUS_RELEASES_DIR}"
fi
if [ "$CI_OS_NAME" == "macos" ]; then
IN_GETOPT_BIN="$(brew --prefix gnu-getopt)/bin/getopt"
export IN_GETOPT_BIN
fi
CI_EXEC () {
$CI_EXEC_CMD_PREFIX bash -c "export PATH=${BINS_SCRATCH_DIR}:${BASE_ROOT_DIR}/ci/retry:\$PATH && cd \"${BASE_ROOT_DIR}\" && $*"
}
export -f CI_EXEC
# Normalize all folders to BASE_ROOT_DIR
CI_EXEC rsync --archive --stats --human-readable "${BASE_READ_ONLY_DIR}/" "${BASE_ROOT_DIR}" || echo "Nothing to copy from ${BASE_READ_ONLY_DIR}/"
CI_EXEC "${BASE_ROOT_DIR}/ci/test/01_base_install.sh"
# Fixes permission issues when there is a container UID/GID mismatch with the owner
# of the git source code directory.
CI_EXEC git config --global --add safe.directory \"*\"
CI_EXEC mkdir -p "${BINS_SCRATCH_DIR}"
CI_EXEC "${BASE_ROOT_DIR}/ci/test/06_script_b.sh"
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
echo "Stop and remove CI container by ID"
docker container kill "${CI_CONTAINER_ID}"
fi

View File

@@ -1,147 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
if [[ $QEMU_USER_CMD == qemu-s390* ]]; then
export LC_ALL=C
fi
if [ "$CI_OS_NAME" == "macos" ]; then
sudo -H pip3 install --upgrade pip
# shellcheck disable=SC2086
IN_GETOPT_BIN="/usr/local/opt/gnu-getopt/bin/getopt" ${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
# Create folders that are mounted into the docker
mkdir -p "${CCACHE_DIR}"
mkdir -p "${PREVIOUS_RELEASES_DIR}"
export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1"
export LSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/lsan"
export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:halt_on_error=1:log_path=${BASE_SCRATCH_DIR}/sanitizer-output/tsan"
export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"
env | grep -E '^(BITCOIN_CONFIG|BASE_|QEMU_|CCACHE_|LC_ALL|BOOST_TEST_RANDOM|DEBIAN_FRONTEND|CONFIG_SHELL|(ASAN|LSAN|TSAN|UBSAN)_OPTIONS|PREVIOUS_RELEASES_DIR)' | tee /tmp/env
if [[ $BITCOIN_CONFIG = *--with-sanitizers=*address* ]]; then # If ran with (ASan + LSan), Docker needs access to ptrace (https://github.com/google/sanitizers/issues/764)
DOCKER_ADMIN="--cap-add SYS_PTRACE"
fi
export P_CI_DIR="$PWD"
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
echo "Creating $DOCKER_NAME_TAG container to run in"
${CI_RETRY_EXE} docker pull "$DOCKER_NAME_TAG"
if [ -n "${RESTART_CI_DOCKER_BEFORE_RUN}" ] ; then
echo "Restart docker before run to stop and clear all containers started with --rm"
systemctl restart docker
fi
# shellcheck disable=SC2086
DOCKER_ID=$(docker run $DOCKER_ADMIN --rm --interactive --detach --tty \
--mount type=bind,src=$BASE_ROOT_DIR,dst=/ro_base,readonly \
--mount type=bind,src=$CCACHE_DIR,dst=$CCACHE_DIR \
--mount type=bind,src=$DEPENDS_DIR,dst=$DEPENDS_DIR \
--mount type=bind,src=$PREVIOUS_RELEASES_DIR,dst=$PREVIOUS_RELEASES_DIR \
-w $BASE_ROOT_DIR \
--env-file /tmp/env \
--name $CONTAINER_NAME \
$DOCKER_NAME_TAG)
export DOCKER_CI_CMD_PREFIX="docker exec $DOCKER_ID"
else
echo "Running on host system without docker wrapper"
fi
CI_EXEC () {
$DOCKER_CI_CMD_PREFIX bash -c "export PATH=$BASE_SCRATCH_DIR/bins/:\$PATH && cd \"$P_CI_DIR\" && $*"
}
export -f CI_EXEC
if [ -n "$DPKG_ADD_ARCH" ]; then
CI_EXEC dpkg --add-architecture "$DPKG_ADD_ARCH"
fi
if [[ $DOCKER_NAME_TAG == *centos* ]]; then
${CI_RETRY_EXE} CI_EXEC dnf -y install epel-release
${CI_RETRY_EXE} CI_EXEC dnf -y --allowerasing install "$DOCKER_PACKAGES" "$PACKAGES"
elif [ "$CI_USE_APT_INSTALL" != "no" ]; then
if [[ "${ADD_UNTRUSTED_BPFCC_PPA}" == "true" ]]; then
# Ubuntu 22.04 LTS and Debian 11 both have an outdated bpfcc-tools packages.
# The iovisor PPA is outdated as well. The next Ubuntu and Debian releases will contain updated
# packages. Meanwhile, use an untrusted PPA to install an up-to-date version of the bpfcc-tools
# package.
# TODO: drop this once we can use newer images in GCE
CI_EXEC add-apt-repository ppa:hadret/bpfcc
fi
${CI_RETRY_EXE} CI_EXEC apt-get update
${CI_RETRY_EXE} CI_EXEC apt-get install --no-install-recommends --no-upgrade -y "$PACKAGES" "$DOCKER_PACKAGES"
if [ -n "$PIP_PACKAGES" ]; then
# shellcheck disable=SC2086
${CI_RETRY_EXE} pip3 install --user $PIP_PACKAGES
fi
fi
if [ "$CI_OS_NAME" == "macos" ]; then
top -l 1 -s 0 | awk ' /PhysMem/ {print}'
echo "Number of CPUs: $(sysctl -n hw.logicalcpu)"
else
CI_EXEC free -m -h
CI_EXEC echo "Number of CPUs \(nproc\):" \$\(nproc\)
CI_EXEC echo "$(lscpu | grep Endian)"
fi
CI_EXEC echo "Free disk space:"
CI_EXEC df -h
if [ "$RUN_FUZZ_TESTS" = "true" ]; then
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
if [ ! -d "$DIR_FUZZ_IN" ]; then
CI_EXEC git clone --depth=1 https://github.com/bitcoin-core/qa-assets "${DIR_QA_ASSETS}"
fi
elif [ "$RUN_UNIT_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
export DIR_UNIT_TEST_DATA=${DIR_QA_ASSETS}/unit_test_data/
if [ ! -d "$DIR_UNIT_TEST_DATA" ]; then
CI_EXEC mkdir -p "$DIR_UNIT_TEST_DATA"
CI_EXEC curl --location --fail https://github.com/bitcoin-core/qa-assets/raw/main/unit_test_data/script_assets_test.json -o "${DIR_UNIT_TEST_DATA}/script_assets_test.json"
fi
fi
CI_EXEC mkdir -p "${BASE_SCRATCH_DIR}/sanitizer-output/"
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
CI_EXEC "update-alternatives --install /usr/bin/clang++ clang++ \$(which clang++-12) 100"
CI_EXEC "update-alternatives --install /usr/bin/clang clang \$(which clang-12) 100"
CI_EXEC "mkdir -p ${BASE_SCRATCH_DIR}/msan/build/"
CI_EXEC "git clone --depth=1 https://github.com/llvm/llvm-project -b llvmorg-12.0.0 ${BASE_SCRATCH_DIR}/msan/llvm-project"
CI_EXEC "cd ${BASE_SCRATCH_DIR}/msan/build/ && cmake -DLLVM_ENABLE_PROJECTS='libcxx;libcxxabi' -DCMAKE_BUILD_TYPE=Release -DLLVM_USE_SANITIZER=MemoryWithOrigins -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DLLVM_TARGETS_TO_BUILD=X86 ../llvm-project/llvm/"
CI_EXEC "cd ${BASE_SCRATCH_DIR}/msan/build/ && make $MAKEJOBS cxx"
fi
if [[ "${RUN_TIDY}" == "true" ]]; then
export DIR_IWYU="${BASE_SCRATCH_DIR}/iwyu"
if [ ! -d "${DIR_IWYU}" ]; then
CI_EXEC "mkdir -p ${DIR_IWYU}/build/"
CI_EXEC "git clone --depth=1 https://github.com/include-what-you-use/include-what-you-use -b clang_14 ${DIR_IWYU}/include-what-you-use"
CI_EXEC "cd ${DIR_IWYU}/build && cmake -G 'Unix Makefiles' -DCMAKE_PREFIX_PATH=/usr/lib/llvm-14 ../include-what-you-use"
CI_EXEC "cd ${DIR_IWYU}/build && make install $MAKEJOBS"
fi
fi
if [ -z "$DANGER_RUN_CI_ON_HOST" ]; then
echo "Create $BASE_ROOT_DIR"
CI_EXEC rsync -a /ro_base/ "$BASE_ROOT_DIR"
fi
if [ "$USE_BUSY_BOX" = "true" ]; then
echo "Setup to use BusyBox utils"
CI_EXEC mkdir -p "${BASE_SCRATCH_DIR}/bins/"
# tar excluded for now because it requires passing in the exact archive type in ./depends (fixed in later BusyBox version)
# find excluded for now because it does not recognize the -delete option in ./depends (fixed in later BusyBox version)
# ar excluded for now because it does not recognize the -q option in ./depends (unknown if fixed)
# shellcheck disable=SC1010
CI_EXEC for util in \$\(busybox --list \| grep -v "^ar$" \| grep -v "^tar$" \| grep -v "^find$"\)\; do ln -s \$\(command -v busybox\) "${BASE_SCRATCH_DIR}/bins/\$util"\; done
# Print BusyBox version
CI_EXEC patch --help
fi

View File

@@ -1,52 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
# Make sure default datadir does not exist and is never read by creating a dummy file
if [ "$CI_OS_NAME" == "macos" ]; then
echo > "${HOME}/Library/Application Support/Bitcoin"
else
CI_EXEC echo \> \$HOME/.bitcoin
fi
CI_EXEC mkdir -p "${DEPENDS_DIR}/SDKs" "${DEPENDS_DIR}/sdk-sources"
OSX_SDK_BASENAME="Xcode-${XCODE_VERSION}-${XCODE_BUILD_ID}-extracted-SDK-with-libcxx-headers"
if [ -n "$XCODE_VERSION" ] && [ ! -d "${DEPENDS_DIR}/SDKs/${OSX_SDK_BASENAME}" ]; then
OSX_SDK_FILENAME="${OSX_SDK_BASENAME}.tar.gz"
OSX_SDK_PATH="${DEPENDS_DIR}/sdk-sources/${OSX_SDK_FILENAME}"
if [ ! -f "$OSX_SDK_PATH" ]; then
CI_EXEC curl --location --fail "${SDK_URL}/${OSX_SDK_FILENAME}" -o "$OSX_SDK_PATH"
fi
CI_EXEC tar -C "${DEPENDS_DIR}/SDKs" -xf "$OSX_SDK_PATH"
fi
if [ -n "$ANDROID_HOME" ] && [ ! -d "$ANDROID_HOME" ]; then
ANDROID_TOOLS_PATH=${DEPENDS_DIR}/sdk-sources/android-tools.zip
if [ ! -f "$ANDROID_TOOLS_PATH" ]; then
CI_EXEC curl --location --fail "${ANDROID_TOOLS_URL}" -o "$ANDROID_TOOLS_PATH"
fi
CI_EXEC mkdir -p "$ANDROID_HOME"
CI_EXEC unzip -o "$ANDROID_TOOLS_PATH" -d "$ANDROID_HOME"
CI_EXEC "yes | ${ANDROID_HOME}/cmdline-tools/bin/sdkmanager --sdk_root=\"${ANDROID_HOME}\" --install \"build-tools;${ANDROID_BUILD_TOOLS_VERSION}\" \"platform-tools\" \"platforms;android-${ANDROID_API_LEVEL}\" \"ndk;${ANDROID_NDK_VERSION}\""
fi
if [ -z "$NO_DEPENDS" ]; then
if [[ $DOCKER_NAME_TAG == *centos* ]]; then
# CentOS has problems building the depends if the config shell is not explicitly set
# (i.e. for libevent a Makefile with an empty SHELL variable is generated, leading to
# an error as the first command is executed)
SHELL_OPTS="LC_ALL=en_US.UTF-8 CONFIG_SHELL=/bin/dash"
else
SHELL_OPTS="CONFIG_SHELL="
fi
CI_EXEC "$SHELL_OPTS" make "$MAKEJOBS" -C depends HOST="$HOST" "$DEP_OPTS" LOG=1
fi
if [ "$DOWNLOAD_PREVIOUS_RELEASES" = "true" ]; then
CI_EXEC test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR"
fi

View File

@@ -1,65 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
BITCOIN_CONFIG_ALL="--enable-suppress-external-warnings --disable-dependency-tracking --prefix=$DEPENDS_DIR/$HOST"
if [ -z "$NO_WERROR" ]; then
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-werror"
fi
CI_EXEC "ccache --zero-stats --max-size=$CCACHE_SIZE"
PRINT_CCACHE_STATISTICS="ccache --version | head -n 1 && ccache --show-stats"
if [ -n "$ANDROID_TOOLS_URL" ]; then
CI_EXEC make distclean || true
CI_EXEC ./autogen.sh
CI_EXEC ./configure "$BITCOIN_CONFIG_ALL" "$BITCOIN_CONFIG" || ( (CI_EXEC cat config.log) && false)
CI_EXEC "make $MAKEJOBS && cd src/qt && ANDROID_HOME=${ANDROID_HOME} ANDROID_NDK_HOME=${ANDROID_NDK_HOME} make apk"
CI_EXEC "${PRINT_CCACHE_STATISTICS}"
exit 0
fi
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-external-signer --bindir=$BASE_OUTDIR/bin --libdir=$BASE_OUTDIR/lib"
if [ -n "$CONFIG_SHELL" ]; then
CI_EXEC "$CONFIG_SHELL" -c "./autogen.sh"
else
CI_EXEC ./autogen.sh
fi
CI_EXEC mkdir -p "${BASE_BUILD_DIR}"
export P_CI_DIR="${BASE_BUILD_DIR}"
CI_EXEC "${BASE_ROOT_DIR}/configure" --cache-file=config.cache "$BITCOIN_CONFIG_ALL" "$BITCOIN_CONFIG" || ( (CI_EXEC cat config.log) && false)
CI_EXEC make distdir VERSION="$HOST"
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST"
CI_EXEC ./configure --cache-file=../config.cache "$BITCOIN_CONFIG_ALL" "$BITCOIN_CONFIG" || ( (CI_EXEC cat config.log) && false)
set -o errtrace
trap 'CI_EXEC "cat ${BASE_SCRATCH_DIR}/sanitizer-output/* 2> /dev/null"' ERR
if [[ ${USE_MEMORY_SANITIZER} == "true" ]]; then
# MemorySanitizer (MSAN) does not support tracking memory initialization done by
# using the Linux getrandom syscall. Avoid using getrandom by undefining
# HAVE_SYS_GETRANDOM. See https://github.com/google/sanitizers/issues/852 for
# details.
CI_EXEC 'grep -v HAVE_SYS_GETRANDOM src/config/bitcoin-config.h > src/config/bitcoin-config.h.tmp && mv src/config/bitcoin-config.h.tmp src/config/bitcoin-config.h'
fi
if [[ "${RUN_TIDY}" == "true" ]]; then
MAYBE_BEAR="bear --config src/.bear-tidy-config"
MAYBE_TOKEN="--"
fi
CI_EXEC "${MAYBE_BEAR}" "${MAYBE_TOKEN}" make "$MAKEJOBS" "$GOAL" || ( echo "Build failure. Verbose build follows." && CI_EXEC make "$GOAL" V=1 ; false )
CI_EXEC "${PRINT_CCACHE_STATISTICS}"
CI_EXEC du -sh "${DEPENDS_DIR}"/*/
CI_EXEC du -sh "${PREVIOUS_RELEASES_DIR}"

View File

@@ -1,80 +1,204 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Copyright (c) 2018-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
if [[ $HOST = *-mingw32 ]]; then
# Generate all binaries, so that they can be wrapped
CI_EXEC make "$MAKEJOBS" -C src/secp256k1 VERBOSE=1
CI_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-wine.sh"
set -ex
export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1"
export LSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/lsan"
export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:halt_on_error=1:log_path=${BASE_SCRATCH_DIR}/sanitizer-output/tsan"
export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"
if [ "$CI_OS_NAME" == "macos" ]; then
top -l 1 -s 0 | awk ' /PhysMem/ {print}'
echo "Number of CPUs: $(sysctl -n hw.logicalcpu)"
else
free -m -h
echo "Number of CPUs (nproc): $(nproc)"
echo "System info: $(uname --kernel-name --kernel-release)"
lscpu
fi
echo "Free disk space:"
df -h
# What host to compile for. See also ./depends/README.md
# Tests that need cross-compilation export the appropriate HOST.
# Tests that run natively guess the host
export HOST=${HOST:-$("$BASE_ROOT_DIR/depends/config.guess")}
(
# compact->outputs[i].file_size is uninitialized memory, so reading it is UB.
# The statistic bytes_written is only used for logging, which is disabled in
# CI, so as a temporary minimal fix to work around UB and CI failures, leave
# bytes_written unmodified.
# See https://github.com/bitcoin/bitcoin/pull/28359#issuecomment-1698694748
echo 'diff --git a/src/leveldb/db/db_impl.cc b/src/leveldb/db/db_impl.cc
index 65e31724bc..f61b471953 100644
--- a/src/leveldb/db/db_impl.cc
+++ b/src/leveldb/db/db_impl.cc
@@ -1028,9 +1028,6 @@ Status DBImpl::DoCompactionWork(CompactionState* compact) {
stats.bytes_read += compact->compaction->input(which, i)->file_size;
}
}
- for (size_t i = 0; i < compact->outputs.size(); i++) {
- stats.bytes_written += compact->outputs[i].file_size;
- }
mutex_.Lock();
stats_[compact->compaction->level() + 1].Add(stats);' | patch -p1
git diff
)
if [ "$RUN_FUZZ_TESTS" = "true" ]; then
export DIR_FUZZ_IN=${DIR_QA_ASSETS}/fuzz_seed_corpus/
if [ ! -d "$DIR_FUZZ_IN" ]; then
${CI_RETRY_EXE} git clone --depth=1 https://github.com/bitcoin-core/qa-assets "${DIR_QA_ASSETS}"
fi
(
cd "${DIR_QA_ASSETS}"
echo "Using qa-assets repo from commit ..."
git log -1
)
elif [ "$RUN_UNIT_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
export DIR_UNIT_TEST_DATA=${DIR_QA_ASSETS}/unit_test_data/
if [ ! -d "$DIR_UNIT_TEST_DATA" ]; then
mkdir -p "$DIR_UNIT_TEST_DATA"
${CI_RETRY_EXE} curl --location --fail https://github.com/bitcoin-core/qa-assets/raw/main/unit_test_data/script_assets_test.json -o "${DIR_UNIT_TEST_DATA}/script_assets_test.json"
fi
fi
if [ -n "$QEMU_USER_CMD" ]; then
mkdir -p "${BASE_SCRATCH_DIR}/sanitizer-output/"
if [ "$USE_BUSY_BOX" = "true" ]; then
echo "Setup to use BusyBox utils"
# tar excluded for now because it requires passing in the exact archive type in ./depends (fixed in later BusyBox version)
# ar excluded for now because it does not recognize the -q option in ./depends (unknown if fixed)
for util in $(busybox --list | grep -v "^ar$" | grep -v "^tar$" ); do ln -s "$(command -v busybox)" "${BINS_SCRATCH_DIR}/$util"; done
# Print BusyBox version
patch --help
fi
# Make sure default datadir does not exist and is never read by creating a dummy file
if [ "$CI_OS_NAME" == "macos" ]; then
echo > "${HOME}/Library/Application Support/Bitcoin"
else
echo > "${HOME}/.bitcoin"
fi
if [ -z "$NO_DEPENDS" ]; then
if [[ $CI_IMAGE_NAME_TAG == *centos* ]]; then
SHELL_OPTS="CONFIG_SHELL=/bin/dash"
else
SHELL_OPTS="CONFIG_SHELL="
fi
bash -c "$SHELL_OPTS make $MAKEJOBS -C depends HOST=$HOST $DEP_OPTS LOG=1"
fi
if [ "$DOWNLOAD_PREVIOUS_RELEASES" = "true" ]; then
test/get_previous_releases.py -b -t "$PREVIOUS_RELEASES_DIR"
fi
BITCOIN_CONFIG_ALL="--disable-dependency-tracking"
if [ -z "$NO_DEPENDS" ]; then
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} CONFIG_SITE=$DEPENDS_DIR/$HOST/share/config.site"
fi
if [ -z "$NO_WERROR" ]; then
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-werror"
fi
ccache --zero-stats
PRINT_CCACHE_STATISTICS="ccache --version | head -n 1 && ccache --show-stats"
if [ -n "$ANDROID_TOOLS_URL" ]; then
make distclean || true
./autogen.sh
bash -c "./configure $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG" || ( (cat config.log) && false)
make "${MAKEJOBS}" && cd src/qt && ANDROID_HOME=${ANDROID_HOME} ANDROID_NDK_HOME=${ANDROID_NDK_HOME} make apk
bash -c "${PRINT_CCACHE_STATISTICS}"
exit 0
fi
BITCOIN_CONFIG_ALL="${BITCOIN_CONFIG_ALL} --enable-external-signer --prefix=$BASE_OUTDIR"
if [ -n "$CONFIG_SHELL" ]; then
"$CONFIG_SHELL" -c "./autogen.sh"
else
./autogen.sh
fi
mkdir -p "${BASE_BUILD_DIR}"
cd "${BASE_BUILD_DIR}"
bash -c "${BASE_ROOT_DIR}/configure --cache-file=config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG" || ( (cat config.log) && false)
make distdir VERSION="$HOST"
cd "${BASE_BUILD_DIR}/bitcoin-$HOST"
bash -c "./configure --cache-file=../config.cache $BITCOIN_CONFIG_ALL $BITCOIN_CONFIG" || ( (cat config.log) && false)
if [[ "${RUN_TIDY}" == "true" ]]; then
MAYBE_BEAR="bear --config src/.bear-tidy-config"
MAYBE_TOKEN="--"
fi
bash -c "${MAYBE_BEAR} ${MAYBE_TOKEN} make $MAKEJOBS $GOAL" || ( echo "Build failure. Verbose build follows." && make "$GOAL" V=1 ; false )
bash -c "${PRINT_CCACHE_STATISTICS}"
du -sh "${DEPENDS_DIR}"/*/
du -sh "${PREVIOUS_RELEASES_DIR}"
if [[ $HOST = *-mingw32 ]]; then
# Generate all binaries, so that they can be wrapped
CI_EXEC make "$MAKEJOBS" -C src/secp256k1 VERBOSE=1
CI_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-qemu.sh"
make "$MAKEJOBS" -C src/secp256k1 VERBOSE=1
make "$MAKEJOBS" -C src minisketch/test.exe VERBOSE=1
"${BASE_ROOT_DIR}/ci/test/wrap-wine.sh"
fi
if [ -n "$USE_VALGRIND" ]; then
CI_EXEC "${BASE_ROOT_DIR}/ci/test/wrap-valgrind.sh"
"${BASE_ROOT_DIR}/ci/test/wrap-valgrind.sh"
fi
if [ "$RUN_UNIT_TESTS" = "true" ]; then
CI_EXEC "${TEST_RUNNER_ENV}" DIR_UNIT_TEST_DATA="${DIR_UNIT_TEST_DATA}" LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" make "$MAKEJOBS" check VERBOSE=1
bash -c "${TEST_RUNNER_ENV} DIR_UNIT_TEST_DATA=${DIR_UNIT_TEST_DATA} LD_LIBRARY_PATH=${DEPENDS_DIR}/${HOST}/lib make $MAKEJOBS check VERBOSE=1"
fi
if [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]; then
CI_EXEC "${TEST_RUNNER_ENV}" DIR_UNIT_TEST_DATA="${DIR_UNIT_TEST_DATA}" LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" "${BASE_OUTDIR}/bin/test_bitcoin" --catch_system_errors=no -l test_suite
bash -c "${TEST_RUNNER_ENV} DIR_UNIT_TEST_DATA=${DIR_UNIT_TEST_DATA} LD_LIBRARY_PATH=${DEPENDS_DIR}/${HOST}/lib ${BASE_OUTDIR}/bin/test_bitcoin --catch_system_errors=no -l test_suite"
fi
if [ "$RUN_FUNCTIONAL_TESTS" = "true" ]; then
CI_EXEC LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" "${TEST_RUNNER_ENV}" test/functional/test_runner.py --ci "$MAKEJOBS" --tmpdirprefix "${BASE_SCRATCH_DIR}/test_runner/" --ansi --combinedlogslen=4000 --timeout-factor="${TEST_RUNNER_TIMEOUT_FACTOR}" "${TEST_RUNNER_EXTRA}" --quiet --failfast
bash -c "LD_LIBRARY_PATH=${DEPENDS_DIR}/${HOST}/lib ${TEST_RUNNER_ENV} test/functional/test_runner.py --ci $MAKEJOBS --tmpdirprefix ${BASE_SCRATCH_DIR}/test_runner/ --ansi --combinedlogslen=99999999 --timeout-factor=${TEST_RUNNER_TIMEOUT_FACTOR} ${TEST_RUNNER_EXTRA} --quiet --failfast"
fi
if [ "${RUN_TIDY}" = "true" ]; then
set -eo pipefail
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST/src/"
( CI_EXEC run-clang-tidy -quiet "${MAKEJOBS}" ) | grep -C5 "error"
export P_CI_DIR="${BASE_BUILD_DIR}/bitcoin-$HOST/"
CI_EXEC "python3 ${DIR_IWYU}/include-what-you-use/iwyu_tool.py"\
" src/compat"\
" src/dbwrapper.cpp"\
" src/init"\
" src/kernel"\
" src/node/chainstate.cpp"\
" src/node/mempool_args.cpp"\
" src/node/validation_cache_args.cpp"\
" src/policy/feerate.cpp"\
" src/policy/packages.cpp"\
" src/policy/settings.cpp"\
" src/primitives/transaction.cpp"\
" src/rpc/fees.cpp"\
" src/rpc/signmessage.cpp"\
" src/test/fuzz/txorphan.cpp"\
" src/threadinterrupt.cpp"\
" src/util/bip32.cpp"\
" src/util/bytevectorhash.cpp"\
" src/util/error.cpp"\
" src/util/getuniquepath.cpp"\
" src/util/hasher.cpp"\
" src/util/message.cpp"\
" src/util/moneystr.cpp"\
" src/util/serfloat.cpp"\
" src/util/spanparsing.cpp"\
" src/util/strencodings.cpp"\
" src/util/string.cpp"\
" src/util/syserror.cpp"\
" src/util/url.cpp"\
" -p . ${MAKEJOBS} -- -Xiwyu --cxx17ns -Xiwyu --mapping_file=${BASE_BUILD_DIR}/bitcoin-$HOST/contrib/devtools/iwyu/bitcoin.core.imp"
fi
cmake -B /tidy-build -DLLVM_DIR=/usr/lib/llvm-"${TIDY_LLVM_V}"/cmake -DCMAKE_BUILD_TYPE=Release -S "${BASE_ROOT_DIR}"/contrib/devtools/bitcoin-tidy
cmake --build /tidy-build "$MAKEJOBS"
cmake --build /tidy-build --target bitcoin-tidy-tests "$MAKEJOBS"
if [ "$RUN_SECURITY_TESTS" = "true" ]; then
CI_EXEC make test-security-check
set -eo pipefail
cd "${BASE_BUILD_DIR}/bitcoin-$HOST/src/"
( run-clang-tidy-"${TIDY_LLVM_V}" -quiet -load="/tidy-build/libbitcoin-tidy.so" "${MAKEJOBS}" ) | grep -C5 "error"
# Filter out files by regex here, because regex may not be
# accepted in src/.bear-tidy-config
# Filter out:
# * qt qrc and moc generated files
jq 'map(select(.file | test("src/qt/qrc_.*\\.cpp$|/moc_.*\\.cpp$") | not))' ../compile_commands.json > tmp.json
mv tmp.json ../compile_commands.json
cd "${BASE_BUILD_DIR}/bitcoin-$HOST/"
python3 "/include-what-you-use/iwyu_tool.py" \
-p . "${MAKEJOBS}" \
-- -Xiwyu --cxx17ns -Xiwyu --mapping_file="${BASE_BUILD_DIR}/bitcoin-$HOST/contrib/devtools/iwyu/bitcoin.core.imp" \
-Xiwyu --max_line_length=160 \
2>&1 | tee /tmp/iwyu_ci.out
cd "${BASE_ROOT_DIR}/src"
python3 "/include-what-you-use/fix_includes.py" --nosafe_headers < /tmp/iwyu_ci.out
git --no-pager diff
fi
if [ "$RUN_FUZZ_TESTS" = "true" ]; then
CI_EXEC LD_LIBRARY_PATH="${DEPENDS_DIR}/${HOST}/lib" test/fuzz/test_runner.py "${FUZZ_TESTS_CONFIG}" "$MAKEJOBS" -l DEBUG "${DIR_FUZZ_IN}"
bash -c "LD_LIBRARY_PATH=${DEPENDS_DIR}/${HOST}/lib test/fuzz/test_runner.py ${FUZZ_TESTS_CONFIG} $MAKEJOBS -l DEBUG ${DIR_FUZZ_IN} --empty_min_time=60"
fi

View File

@@ -1,18 +0,0 @@
#!/usr/bin/env bash
#
# Copyright (c) 2018-2021 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
for b_name in {"${BASE_OUTDIR}/bin"/*,src/secp256k1/*tests,src/minisketch/test{,-verify},src/univalue/{test_json,unitester,object}}; do
# shellcheck disable=SC2044
for b in $(find "${BASE_ROOT_DIR}" -executable -type f -name "$(basename "$b_name")"); do
echo "Wrap $b ..."
mv "$b" "${b}_orig"
echo '#!/usr/bin/env bash' > "$b"
echo "$QEMU_USER_CMD \"${b}_orig\" \"\$@\"" >> "$b"
chmod +x "$b"
done
done

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
#
# Copyright (c) 2020-2021 The Bitcoin Core developers
# Copyright (c) 2020-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.

View File

@@ -1 +0,0 @@
ccache cl %*

16
ci/test_imagefile Normal file
View File

@@ -0,0 +1,16 @@
# Copyright (c) The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or https://opensource.org/license/mit/.
# See ci/README.md for usage.
ARG CI_IMAGE_NAME_TAG
FROM ${CI_IMAGE_NAME_TAG}
ARG FILE_ENV
ENV FILE_ENV=${FILE_ENV}
COPY ./ci/retry/retry /usr/bin/retry
COPY ./ci/test/00_setup_env.sh ./${FILE_ENV} ./ci/test/01_base_install.sh /ci_container_base/ci/test/
RUN ["bash", "-c", "cd /ci_container_base/ && set -o errexit && source ./ci/test/00_setup_env.sh && ./ci/test/01_base_install.sh"]

View File

@@ -1,13 +1,11 @@
#!/usr/bin/env bash
#
# Copyright (c) 2019-2020 The Bitcoin Core developers
# Copyright (c) 2019-present The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
export LC_ALL=C.UTF-8
set -o errexit; source ./ci/test/00_setup_env.sh
set -o errexit; source ./ci/test/04_install.sh
set -o errexit; source ./ci/test/05_before_script.sh
set -o errexit; source ./ci/test/06_script_a.sh
set -o errexit; source ./ci/test/06_script_b.sh
set -o errexit
"./ci/test/02_run_container.sh"

View File

@@ -1,10 +1,10 @@
AC_PREREQ([2.69])
define(_CLIENT_VERSION_MAJOR, 23)
define(_CLIENT_VERSION_MINOR, 99)
define(_CLIENT_VERSION_MAJOR, 26)
define(_CLIENT_VERSION_MINOR, 2)
define(_CLIENT_VERSION_BUILD, 0)
define(_CLIENT_VERSION_RC, 0)
define(_CLIENT_VERSION_IS_RELEASE, false)
define(_COPYRIGHT_YEAR, 2022)
define(_CLIENT_VERSION_IS_RELEASE, true)
define(_COPYRIGHT_YEAR, 2024)
define(_COPYRIGHT_HOLDERS,[The %s developers])
define(_COPYRIGHT_HOLDERS_SUBSTITUTION,[[Bitcoin Core]])
AC_INIT([Bitcoin Core],m4_join([.], _CLIENT_VERSION_MAJOR, _CLIENT_VERSION_MINOR, _CLIENT_VERSION_BUILD)m4_if(_CLIENT_VERSION_RC, [0], [], [rc]_CLIENT_VERSION_RC),[https://github.com/bitcoin/bitcoin/issues],[bitcoin],[https://bitcoincore.org/])
@@ -70,21 +70,32 @@ else
fi
AC_PROG_CXX
dnl By default, libtool for mingw refuses to link static libs into a dll for
dnl fear of mixing pic/non-pic objects, and import/export complications. Since
dnl we have those under control, re-enable that functionality.
dnl libtool overrides
case $host in
*mingw*)
dnl By default, libtool for mingw refuses to link static libs into a dll for
dnl fear of mixing pic/non-pic objects, and import/export complications. Since
dnl we have those under control, re-enable that functionality.
lt_cv_deplibs_check_method="pass_all"
dnl Remove unwanted -DDLL_EXPORT from these variables.
dnl We do not use this macro, but system headers may export unwanted symbols
dnl if it's set.
lt_cv_prog_compiler_pic="-DPIC"
lt_cv_prog_compiler_pic_CXX="-DPIC"
;;
*darwin*)
dnl Because it prints a verbose warning, lld fails the following check
dnl for "-Wl,-single_module" from libtool.m4:
dnl # If there is a non-empty error log, and "single_module"
dnl # appears in it, assume the flag caused a linker warning
dnl "-single_module" works fine on ld64 and lld, so just bypass the test.
dnl Failure to set this to "yes" causes libtool to use a very broken
dnl link-line for shared libs.
lt_cv_apple_cc_single_mod="yes"
;;
esac
AC_ARG_WITH([seccomp],
[AS_HELP_STRING([--with-seccomp],
[enable experimental syscall sandbox feature (-sandbox), default is yes if seccomp-bpf is detected under Linux x86_64])],
[seccomp_found=$withval],
[seccomp_found=auto])
AC_ARG_ENABLE([c++20],
[AS_HELP_STRING([--enable-c++20],
[enable compilation in c++20 mode (disabled by default)])],
@@ -98,9 +109,6 @@ else
AX_CXX_COMPILE_STDCXX([20], [noext], [mandatory])
fi
dnl check if additional link flags are required for std::filesystem
CHECK_FILESYSTEM
dnl Unless the user specified OBJCXX, force it to be the same as CXX. This ensures
dnl that we get the same -std flags for both.
m4_ifdef([AC_PROG_OBJCXX],[
@@ -120,8 +128,8 @@ AC_PATH_TOOL([AR], [ar])
AC_PATH_TOOL([GCOV], [gcov])
AC_PATH_TOOL([LLVM_COV], [llvm-cov])
AC_PATH_PROG([LCOV], [lcov])
dnl Python 3.6 is specified in .python-version and should be used if available, see doc/dependencies.md
AC_PATH_PROGS([PYTHON], [python3.6 python3.7 python3.8 python3.9 python3.10 python3.11 python3 python])
dnl Python 3.8 is specified in .python-version and should be used if available, see doc/dependencies.md
AC_PATH_PROGS([PYTHON], [python3.8 python3.9 python3.10 python3.11 python3.12 python3 python])
AC_PATH_PROG([GENHTML], [genhtml])
AC_PATH_PROG([GIT], [git])
AC_PATH_PROG([CCACHE], [ccache])
@@ -163,24 +171,12 @@ AC_ARG_WITH([miniupnpc],
[use_upnp=$withval],
[use_upnp=auto])
AC_ARG_ENABLE([upnp-default],
[AS_HELP_STRING([--enable-upnp-default],
[if UPNP is enabled, turn it on at startup (default is no)])],
[use_upnp_default=$enableval],
[use_upnp_default=no])
AC_ARG_WITH([natpmp],
[AS_HELP_STRING([--with-natpmp],
[enable NAT-PMP (default is yes if libnatpmp is found)])],
[use_natpmp=$withval],
[use_natpmp=auto])
AC_ARG_ENABLE([natpmp-default],
[AS_HELP_STRING([--enable-natpmp-default],
[if NAT-PMP is enabled, turn it on at startup (default is no)])],
[use_natpmp_default=$enableval],
[use_natpmp_default=no])
AC_ARG_ENABLE(tests,
AS_HELP_STRING([--disable-tests],[do not compile tests (default is to compile)]),
[use_tests=$enableval],
@@ -242,10 +238,10 @@ dnl May be useful if warnings from external headers clutter the build output
dnl too much, so that it becomes difficult to spot Bitcoin Core warnings
dnl or if they cause a build failure with --enable-werror.
AC_ARG_ENABLE([suppress-external-warnings],
[AS_HELP_STRING([--enable-suppress-external-warnings],
[Suppress warnings from external headers (default is no)])],
[AS_HELP_STRING([--disable-suppress-external-warnings],
[Do not suppress warnings from external headers (default is to suppress)])],
[suppress_external_warnings=$enableval],
[suppress_external_warnings=no])
[suppress_external_warnings=yes])
AC_ARG_ENABLE([lcov],
[AS_HELP_STRING([--enable-lcov],
@@ -431,12 +427,6 @@ if test "$enable_werror" = "yes"; then
AC_MSG_ERROR([enable-werror set but -Werror is not usable])
fi
ERROR_CXXFLAGS=$CXXFLAG_WERROR
dnl -Wreturn-type is broken in GCC for MinGW-w64.
dnl https://sourceforge.net/p/mingw-w64/bugs/306/
AX_CHECK_COMPILE_FLAG([-Werror=return-type], [], [ERROR_CXXFLAGS="$ERROR_CXXFLAGS -Wno-error=return-type"], [$CXXFLAG_WERROR],
[AC_LANG_SOURCE([[#include <cassert>
int f(){ assert(false); }]])])
fi
if test "$CXXFLAGS_overridden" = "no"; then
@@ -448,7 +438,7 @@ if test "$CXXFLAGS_overridden" = "no"; then
AX_CHECK_COMPILE_FLAG([-Wvla], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wvla"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wshadow-field], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wshadow-field"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wthread-safety], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wthread-safety"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wloop-analysis], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wrange-loop-analysis"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wloop-analysis], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wloop-analysis"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wredundant-decls], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wredundant-decls"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wunused-member-function], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wunused-member-function"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-Wdate-time], [WARN_CXXFLAGS="$WARN_CXXFLAGS -Wdate-time"], [], [$CXXFLAG_WERROR])
@@ -481,6 +471,12 @@ fi
dnl Don't allow extended (non-ASCII) symbols in identifiers. This is easier for code review.
AX_CHECK_COMPILE_FLAG([-fno-extended-identifiers], [CORE_CXXFLAGS="$CORE_CXXFLAGS -fno-extended-identifiers"], [], [$CXXFLAG_WERROR])
dnl Currently all versions of gcc are subject to a class of bugs, see the
dnl gccbug_90348 test case (only reproduces on GCC 11 and earlier) and
dnl https://gcc.gnu.org/bugzilla/show_bug.cgi?id=111843. To work around that, set
dnl -fstack-reuse=none for all gcc builds. (Only gcc understands this flag)
AX_CHECK_COMPILE_FLAG([-fstack-reuse=none], [CORE_CXXFLAGS="$CORE_CXXFLAGS -fstack-reuse=none"])
enable_arm_crc=no
enable_arm_shani=no
enable_sse42=no
@@ -589,7 +585,7 @@ CXXFLAGS="$TEMP_CXXFLAGS"
# ARM
AX_CHECK_COMPILE_FLAG([-march=armv8-a+crc+crypto], [ARM_CRC_CXXFLAGS="-march=armv8-a+crc+crypto"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-march=armv8-a+crc+crypto], [ARM_SHANI_CXXFLAGS="-march=armv8-a+crc+crypto"], [], [$CXXFLAG_WERROR])
AX_CHECK_COMPILE_FLAG([-march=armv8-a+crypto], [ARM_SHANI_CXXFLAGS="-march=armv8-a+crypto"], [], [$CXXFLAG_WERROR])
TEMP_CXXFLAGS="$CXXFLAGS"
CXXFLAGS="$ARM_CRC_CXXFLAGS $CXXFLAGS"
@@ -718,6 +714,8 @@ case $host in
AC_MSG_ERROR([windres not found])
fi
CORE_CPPFLAGS="$CORE_CPPFLAGS -DSECP256K1_STATIC"
CORE_CPPFLAGS="$CORE_CPPFLAGS -D_MT -DWIN32 -D_WINDOWS -D_WIN32_WINNT=0x0601 -D_WIN32_IE=0x0501 -DWIN32_LEAN_AND_MEAN"
dnl Prevent the definition of min/max macros.
dnl We always want to use the standard library.
@@ -733,6 +731,10 @@ case $host in
dnl We require Windows 7 (NT 6.1) or later
AX_CHECK_LINK_FLAG([-Wl,--major-subsystem-version -Wl,6 -Wl,--minor-subsystem-version -Wl,1], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,--major-subsystem-version -Wl,6 -Wl,--minor-subsystem-version -Wl,1"], [], [$LDFLAG_WERROR])
dnl Avoid the use of aligned vector instructions when building for Windows.
dnl See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=54412.
AX_CHECK_COMPILE_FLAG([-Wa,-muse-unaligned-vector-move], [CORE_CXXFLAGS="$CORE_CXXFLAGS -Wa,-muse-unaligned-vector-move"], [], [$CXXFLAG_WERROR])
;;
*darwin*)
TARGET_OS=darwin
@@ -753,7 +755,7 @@ case $host in
dnl option to system-ify all /usr/local/include paths without adding it to the list
dnl of search paths in case it's not already there.
if test "$suppress_external_warnings" != "no"; then
AX_CHECK_PREPROC_FLAG([-Xclang -internal-isystem/usr/local/include], [CORE_CPPFLAGS="$CORE_CPPFLAGS -Xclang -internal-isystem/usr/local/include"], [], [$CXXFLAG_WERROR])
AX_CHECK_PREPROC_FLAG([-Xclang -internal-isystem -Xclang /usr/local/include/], [CORE_CPPFLAGS="$CORE_CPPFLAGS -Xclang -internal-isystem -Xclang /usr/local/include/"], [], [$CXXFLAG_WERROR])
fi
if test "$use_bdb" != "no" && $BREW list --versions berkeley-db@4 >/dev/null && test "$BDB_CFLAGS" = "" && test "$BDB_LIBS" = ""; then
@@ -763,10 +765,6 @@ case $host in
BDB_LIBS="-L$bdb_prefix/lib -ldb_cxx-4.8"
fi
if test "$use_sqlite" != "no" && $BREW list --versions sqlite3 >/dev/null; then
export PKG_CONFIG_PATH="$($BREW --prefix sqlite3 2>/dev/null)/lib/pkgconfig:$PKG_CONFIG_PATH"
fi
if $BREW list --versions qt@5 >/dev/null; then
export PKG_CONFIG_PATH="$($BREW --prefix qt@5 2>/dev/null)/lib/pkgconfig:$PKG_CONFIG_PATH"
fi
@@ -807,7 +805,7 @@ case $host in
AC_PATH_TOOL([DSYMUTIL], [dsymutil], [dsymutil])
AC_PATH_TOOL([INSTALL_NAME_TOOL], [install_name_tool], [install_name_tool])
AC_PATH_TOOL([OTOOL], [otool], [otool])
AC_PATH_PROGS([XORRISOFS], [xorrisofs], [xorrisofs])
AC_PATH_PROG([ZIP], [zip], [zip])
dnl libtool will try to strip the static lib, which is a problem for
dnl cross-builds because strip attempts to call a hard-coded ld,
@@ -927,8 +925,6 @@ if test "$ac_cv_sys_large_files" != "" &&
CORE_CPPFLAGS="$CORE_CPPFLAGS -D_LARGE_FILES=$ac_cv_sys_large_files"
fi
AC_SEARCH_LIBS([clock_gettime],[rt])
if test "$enable_gprof" = "yes"; then
dnl -pg is incompatible with -pie. Since hardening and profiling together doesn't make sense,
dnl we simply make them mutually exclusive here. Additionally, hardened toolchains may force
@@ -951,10 +947,6 @@ if test "$TARGET_OS" != "windows"; then
AX_CHECK_COMPILE_FLAG([-fPIC], [PIC_FLAGS="-fPIC"])
fi
dnl All versions of gcc that we commonly use for building are subject to bug
dnl https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90348. To work around that, set
dnl -fstack-reuse=none for all gcc builds. (Only gcc understands this flag)
AX_CHECK_COMPILE_FLAG([-fstack-reuse=none], [HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -fstack-reuse=none"])
if test "$use_hardening" != "no"; then
use_hardening=yes
AX_CHECK_COMPILE_FLAG([-Wstack-protector], [HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -Wstack-protector"])
@@ -964,7 +956,8 @@ if test "$use_hardening" != "no"; then
case $host in
*mingw*)
dnl stack-clash-protection doesn't currently work, and likely should just be skipped for Windows.
dnl stack-clash-protection doesn't compile with GCC 10 and earlier.
dnl In any case, it is a no-op for Windows.
dnl See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90458 for more details.
;;
*)
@@ -972,16 +965,21 @@ if test "$use_hardening" != "no"; then
;;
esac
case $host in
*aarch64*)
AX_CHECK_COMPILE_FLAG([-mbranch-protection=bti], [HARDENED_CXXFLAGS="$HARDENED_CXXFLAGS -mbranch-protection=bti"])
;;
esac
dnl When enable_debug is yes, all optimizations are disabled.
dnl However, FORTIFY_SOURCE requires that there is some level of optimization, otherwise it does nothing and just creates a compiler warning.
dnl Since FORTIFY_SOURCE is a no-op without optimizations, do not enable it when enable_debug is yes.
if test "$enable_debug" != "yes"; then
AX_CHECK_PREPROC_FLAG([-D_FORTIFY_SOURCE=2],[
AX_CHECK_PREPROC_FLAG([-D_FORTIFY_SOURCE=3],[
AX_CHECK_PREPROC_FLAG([-U_FORTIFY_SOURCE],[
HARDENED_CPPFLAGS="$HARDENED_CPPFLAGS -U_FORTIFY_SOURCE"
])
HARDENED_CPPFLAGS="$HARDENED_CPPFLAGS -D_FORTIFY_SOURCE=2"
HARDENED_CPPFLAGS="$HARDENED_CPPFLAGS -D_FORTIFY_SOURCE=3"
])
fi
@@ -1008,9 +1006,10 @@ if test "$TARGET_OS" = "darwin"; then
AX_CHECK_LINK_FLAG([-Wl,-dead_strip], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,-dead_strip"], [], [$LDFLAG_WERROR])
AX_CHECK_LINK_FLAG([-Wl,-dead_strip_dylibs], [CORE_LDFLAGS="$CORE_LDFLAGS -Wl,-dead_strip_dylibs"], [], [$LDFLAG_WERROR])
AX_CHECK_LINK_FLAG([-Wl,-bind_at_load], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-bind_at_load"], [], [$LDFLAG_WERROR])
AX_CHECK_LINK_FLAG([-Wl,-fixup_chains], [HARDENED_LDFLAGS="$HARDENED_LDFLAGS -Wl,-fixup_chains"], [], [$LDFLAG_WERROR])
fi
AC_CHECK_HEADERS([endian.h sys/endian.h byteswap.h stdio.h stdlib.h unistd.h strings.h sys/types.h sys/stat.h sys/select.h sys/prctl.h sys/sysctl.h vm/vm_param.h sys/vmmeter.h sys/resources.h])
AC_CHECK_HEADERS([endian.h sys/endian.h byteswap.h sys/select.h sys/prctl.h sys/sysctl.h vm/vm_param.h sys/vmmeter.h sys/resources.h])
AC_CHECK_DECLS([getifaddrs, freeifaddrs],[CHECK_SOCKET],,
[#include <sys/types.h>
@@ -1072,7 +1071,7 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <malloc.h>]],
dnl Check for posix_fallocate
AC_MSG_CHECKING([for posix_fallocate])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
// same as in src/util/system.cpp
// same as in src/util/fs_helpers.cpp
#ifdef __linux__
#ifdef _POSIX_C_SOURCE
#undef _POSIX_C_SOURCE
@@ -1170,17 +1169,16 @@ AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <ctime>]],
)
dnl Check for different ways of gathering OS randomness
AC_MSG_CHECKING([for Linux getrandom syscall])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <unistd.h>
#include <sys/syscall.h>
#include <linux/random.h>]],
[[ syscall(SYS_getrandom, nullptr, 32, 0); ]])],
[ AC_MSG_RESULT([yes]); AC_DEFINE([HAVE_SYS_GETRANDOM], [1], [Define this symbol if the Linux getrandom system call is available]) ],
AC_MSG_CHECKING([for Linux getrandom function])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <sys/random.h>]],
[[ getrandom(nullptr, 32, 0); ]])],
[ AC_MSG_RESULT([yes]); AC_DEFINE([HAVE_GETRANDOM], [1], [Define this symbol if the Linux getrandom function call is available]) ],
[ AC_MSG_RESULT([no])]
)
AC_MSG_CHECKING([for getentropy via random.h])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <unistd.h>
AC_MSG_CHECKING([for getentropy via sys/random.h])
AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[
#include <sys/random.h>]],
[[ getentropy(nullptr, 32) ]])],
[ AC_MSG_RESULT([yes]); AC_DEFINE([HAVE_GETENTROPY_RAND], [1], [Define this symbol if the BSD getentropy system call is available with sys/random.h]) ],
@@ -1409,7 +1407,9 @@ if test "$use_usdt" != "no"; then
AC_COMPILE_IFELSE([
AC_LANG_PROGRAM(
[#include <sys/sdt.h>],
[DTRACE_PROBE("context", "event");]
[DTRACE_PROBE(context, event);
int a, b, c, d, e, f, g;
DTRACE_PROBE7(context, event, a, b, c, d, e, f, g);]
)],
[AC_MSG_RESULT([yes]); AC_DEFINE([ENABLE_TRACING], [1], [Define to 1 to enable tracepoints for Userspace, Statically Defined Tracing])],
[AC_MSG_RESULT([no]); use_usdt=no;]
@@ -1417,7 +1417,7 @@ if test "$use_usdt" != "no"; then
fi
AM_CONDITIONAL([ENABLE_USDT_TRACEPOINTS], [test "$use_usdt" = "yes"])
if test "$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_bench$use_tests" = "nonononononono"; then
if test "$build_bitcoind$bitcoin_enable_qt$use_bench$use_tests" = "nononono"; then
use_upnp=no
use_natpmp=no
use_zmq=no
@@ -1432,14 +1432,15 @@ if test "$use_upnp" != "no"; then
[AC_CHECK_LIB([miniupnpc], [upnpDiscover], [MINIUPNPC_LIBS="$MINIUPNPC_LIBS -lminiupnpc"], [have_miniupnpc=no], [$MINIUPNPC_LIBS])],
[have_miniupnpc=no]
)
dnl The minimum supported miniUPnPc API version is set to 10. This keeps compatibility
dnl with Ubuntu 16.04 LTS and Debian 8 libminiupnpc-dev packages.
dnl The minimum supported miniUPnPc API version is set to 17. This excludes
dnl versions with known vulnerabilities.
if test "$have_miniupnpc" != "no"; then
AC_MSG_CHECKING([whether miniUPnPc API version is supported])
AC_PREPROC_IFELSE([AC_LANG_PROGRAM([[
@%:@include <miniupnpc/miniupnpc.h>
]], [[
#if MINIUPNPC_API_VERSION >= 10
#if MINIUPNPC_API_VERSION >= 17
// Everything is okay
#else
# error miniUPnPc API version is too old
@@ -1448,7 +1449,7 @@ if test "$use_upnp" != "no"; then
AC_MSG_RESULT([yes])
],[
AC_MSG_RESULT([no])
AC_MSG_WARN([miniUPnPc API version < 10 is unsupported, disabling UPnP support.])
AC_MSG_WARN([miniUPnPc API version < 17 is unsupported, disabling UPnP support.])
have_miniupnpc=no
])
fi
@@ -1465,7 +1466,7 @@ if test "$use_natpmp" != "no"; then
CPPFLAGS="$TEMP_CPPFLAGS"
fi
if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench" = "nonononononono"; then
if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoind$bitcoin_enable_qt$use_tests$use_bench$enable_fuzz_binary" = "nonononononononono"; then
use_boost=no
else
use_boost=yes
@@ -1483,13 +1484,11 @@ if test "$use_boost" = "yes"; then
BOOST_CPPFLAGS="$BOOST_CPPFLAGS -DBOOST_MULTI_INDEX_DISABLE_SERIALIZATION"
dnl Prevent use of std::unary_function, which was removed in C++17,
dnl and will generate warnings with newer compilers.
dnl See: https://github.com/boostorg/container_hash/issues/22.
BOOST_CPPFLAGS="$BOOST_CPPFLAGS -DBOOST_NO_CXX98_FUNCTION_BASE"
if test "$enable_debug" = "yes" || test "$enable_fuzz" = "yes"; then
BOOST_CPPFLAGS="$BOOST_CPPFLAGS -DBOOST_MULTI_INDEX_ENABLE_SAFE_MODE"
fi
dnl and will generate warnings with newer compilers for Boost
dnl older than 1.80.
dnl See: https://github.com/boostorg/config/pull/430.
AX_CHECK_PREPROC_FLAG([-DBOOST_NO_CXX98_FUNCTION_BASE], [BOOST_CPPFLAGS="$BOOST_CPPFLAGS -DBOOST_NO_CXX98_FUNCTION_BASE"], [], [$CXXFLAG_WERROR],
[AC_LANG_PROGRAM([[#include <boost/config.hpp>]])])
if test "$suppress_external_warnings" != "no"; then
BOOST_CPPFLAGS=SUPPRESS_WARNINGS($BOOST_CPPFLAGS)
@@ -1497,78 +1496,48 @@ if test "$use_boost" = "yes"; then
fi
if test "$use_external_signer" != "no"; then
case $host in
*mingw*)
dnl Boost Process uses Boost Filesystem when targeting Windows. Also,
dnl since Boost 1.71.0, Process does not work with mingw-w64 without
dnl workarounds. See 67669ab425b52a2b6be3d2f3b3b7e3939b676a2c.
if test "$use_external_signer" = "yes"; then
AC_MSG_ERROR([External signing is not supported on Windows])
fi
use_external_signer="no";
;;
*)
AC_MSG_CHECKING([whether Boost.Process can be used])
TEMP_CXXFLAGS="$CXXFLAGS"
dnl Boost 1.78 requires the following workaround.
dnl See: https://github.com/boostorg/process/issues/235
CXXFLAGS="$CXXFLAGS -Wno-error=narrowing"
TEMP_CPPFLAGS="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
TEMP_LDFLAGS="$LDFLAGS"
dnl Boost 1.73 and older require the following workaround.
LDFLAGS="$LDFLAGS $PTHREAD_CFLAGS"
AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include <boost/process.hpp>]])],
[have_boost_process="yes"],
[have_boost_process="no"])
LDFLAGS="$TEMP_LDFLAGS"
CPPFLAGS="$TEMP_CPPFLAGS"
CXXFLAGS="$TEMP_CXXFLAGS"
AC_MSG_RESULT([$have_boost_process])
if test "$have_boost_process" = "yes"; then
use_external_signer="yes"
AC_DEFINE([ENABLE_EXTERNAL_SIGNER], [1], [Define if external signer support is enabled])
else
if test "$use_external_signer" = "yes"; then
AC_MSG_ERROR([External signing is not supported for this Boost version])
fi
use_external_signer="no";
fi
;;
esac
AC_MSG_CHECKING([whether Boost.Process can be used])
TEMP_CXXFLAGS="$CXXFLAGS"
dnl Boost 1.78 requires the following workaround.
dnl See: https://github.com/boostorg/process/issues/235
CXXFLAGS="$CXXFLAGS -Wno-error=narrowing"
TEMP_CPPFLAGS="$CPPFLAGS"
CPPFLAGS="$CPPFLAGS $BOOST_CPPFLAGS"
TEMP_LDFLAGS="$LDFLAGS"
dnl Boost 1.73 and older require the following workaround.
LDFLAGS="$LDFLAGS $PTHREAD_CFLAGS"
AC_LINK_IFELSE([AC_LANG_PROGRAM([[
#define BOOST_PROCESS_USE_STD_FS
#include <boost/process.hpp>
]],[[
namespace bp = boost::process;
bp::opstream stdin_stream;
bp::ipstream stdout_stream;
bp::child c("dummy", bp::std_out > stdout_stream, bp::std_err > stdout_stream, bp::std_in < stdin_stream);
stdin_stream << std::string{"test"} << std::endl;
if (c.running()) c.terminate();
c.wait();
c.exit_code();
]])],
[have_boost_process="yes"],
[have_boost_process="no"])
LDFLAGS="$TEMP_LDFLAGS"
CPPFLAGS="$TEMP_CPPFLAGS"
CXXFLAGS="$TEMP_CXXFLAGS"
AC_MSG_RESULT([$have_boost_process])
if test "$have_boost_process" = "yes"; then
use_external_signer="yes"
AC_DEFINE([ENABLE_EXTERNAL_SIGNER], [1], [Define if external signer support is enabled])
AC_DEFINE([BOOST_PROCESS_USE_STD_FS], [1], [Defined to avoid Boost::Process trying to use Boost Filesystem])
else
if test "$use_external_signer" = "yes"; then
AC_MSG_ERROR([External signing is not supported for this Boost version])
fi
use_external_signer="no";
fi
fi
AM_CONDITIONAL([ENABLE_EXTERNAL_SIGNER], [test "$use_external_signer" = "yes"])
dnl Do not compile with syscall sandbox support when compiling under the sanitizers.
dnl The sanitizers introduce use of syscalls that are not typically used in bitcoind
dnl (such as execve when the sanitizers execute llvm-symbolizer).
if test "$use_sanitizers" != ""; then
AC_MSG_WARN([Specifying --with-sanitizers forces --without-seccomp since the sanitizers introduce use of syscalls not allowed by the bitcoind syscall sandbox (-sandbox=<mode>).])
seccomp_found=no
fi
if test "$seccomp_found" != "no"; then
AC_MSG_CHECKING([for seccomp-bpf (Linux x86-64)])
AC_PREPROC_IFELSE([AC_LANG_PROGRAM([[
@%:@include <linux/seccomp.h>
]], [[
#if !defined(__x86_64__)
# error Syscall sandbox is an experimental feature currently available only under Linux x86-64.
#endif
]])],[
AC_MSG_RESULT([yes])
seccomp_found="yes"
AC_DEFINE([USE_SYSCALL_SANDBOX], [1], [Define this symbol to build with syscall sandbox support.])
],[
AC_MSG_RESULT([no])
seccomp_found="no"
])
fi
dnl Currently only enable -sandbox=<mode> feature if seccomp is found.
dnl In the future, sandboxing could be also be supported with other
dnl sandboxing mechanisms besides seccomp.
use_syscall_sandbox=$seccomp_found
AM_CONDITIONAL([ENABLE_SYSCALL_SANDBOX], [test "$use_syscall_sandbox" != "no"])
dnl Check for reduced exports
if test "$use_reduce_exports" = "yes"; then
AX_CHECK_COMPILE_FLAG([-fvisibility=hidden], [CORE_CXXFLAGS="$CORE_CXXFLAGS -fvisibility=hidden"],
@@ -1627,12 +1596,9 @@ dnl ZMQ check
if test "$use_zmq" = "yes"; then
PKG_CHECK_MODULES([ZMQ], [libzmq >= 4],
AC_DEFINE([ENABLE_ZMQ], [1], [Define to 1 to enable ZMQ functions]),
[AC_DEFINE([ENABLE_ZMQ], [0], [Define to 1 to enable ZMQ functions])
AC_MSG_WARN([libzmq version 4.x or greater not found, disabling])
AC_DEFINE([ENABLE_ZMQ], [1], [Define this symbol to enable ZMQ functions]),
[AC_MSG_WARN([libzmq version 4.x or greater not found, disabling])
use_zmq=no])
else
AC_DEFINE_UNQUOTED([ENABLE_ZMQ], [0], [Define to 1 to enable ZMQ functions])
fi
if test "$use_zmq" = "yes"; then
@@ -1644,6 +1610,8 @@ if test "$use_zmq" = "yes"; then
esac
fi
AM_CONDITIONAL([ENABLE_ZMQ], [test "$use_zmq" = "yes"])
dnl libmultiprocess library check
libmultiprocess_found=no
@@ -1769,17 +1737,10 @@ if test "$have_miniupnpc" = "no"; then
else
if test "$use_upnp" != "no"; then
AC_MSG_RESULT([yes])
AC_MSG_CHECKING([whether to build with UPnP enabled by default])
use_upnp=yes
upnp_setting=0
if test "$use_upnp_default" != "no"; then
use_upnp_default=yes
upnp_setting=1
fi
AC_MSG_RESULT([$use_upnp_default])
AC_DEFINE_UNQUOTED([USE_UPNP],[$upnp_setting],[UPnP support not compiled if undefined, otherwise value (0 or 1) determines default state])
AC_DEFINE([USE_UPNP], [1], [Define to 1 if UPnP support should be compiled in.])
if test "$TARGET_OS" = "windows"; then
MINIUPNPC_CPPFLAGS="$MINIUPNPC_CPPFLAGS -DSTATICLIB -DMINIUPNP_STATICLIB"
MINIUPNPC_CPPFLAGS="$MINIUPNPC_CPPFLAGS -DMINIUPNP_STATICLIB"
fi
else
AC_MSG_RESULT([no])
@@ -1797,15 +1758,8 @@ if test "$have_natpmp" = "no"; then
else
if test "$use_natpmp" != "no"; then
AC_MSG_RESULT([yes])
AC_MSG_CHECKING([whether to build with NAT-PMP enabled by default])
use_natpmp=yes
natpmp_setting=0
if test "$use_natpmp_default" != "no"; then
use_natpmp_default=yes
natpmp_setting=1
fi
AC_MSG_RESULT($use_natpmp_default)
AC_DEFINE_UNQUOTED([USE_NATPMP], [$natpmp_setting], [NAT-PMP support not compiled if undefined, otherwise value (0 or 1) determines default state])
AC_DEFINE([USE_NATPMP], [1], [Define to 1 if UPnP support should be compiled in.])
if test "$TARGET_OS" = "windows"; then
NATPMP_CPPFLAGS="$NATPMP_CPPFLAGS -DSTATICLIB -DNATPMP_STATICLIB"
fi
@@ -1852,8 +1806,6 @@ if test "$bitcoin_enable_qt" != "no"; then
fi
fi
AM_CONDITIONAL([ENABLE_ZMQ], [test "$use_zmq" = "yes"])
AC_MSG_CHECKING([whether to build test_bitcoin])
if test "$use_tests" = "yes"; then
if test "$enable_fuzz" = "yes"; then
@@ -1874,7 +1826,7 @@ else
AC_MSG_RESULT([no])
fi
if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_libs$build_bitcoind$bitcoin_enable_qt$enable_fuzz_binary$use_bench$use_tests" = "nonononononononono"; then
if test "$build_bitcoin_wallet$build_bitcoin_cli$build_bitcoin_tx$build_bitcoin_util$build_bitcoin_libs$build_bitcoind$bitcoin_enable_qt$enable_fuzz_binary$use_bench$use_tests" = "nononononononononono"; then
AC_MSG_ERROR([No targets! Please specify at least one of: --with-utils --with-libs --with-daemon --with-gui --enable-fuzz(-binary) --enable-bench or --enable-tests])
fi
@@ -1988,13 +1940,8 @@ AC_SUBST(HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR)
AC_CONFIG_FILES([Makefile src/Makefile doc/man/Makefile share/setup.nsi share/qt/Info.plist test/config.ini])
AC_CONFIG_FILES([contrib/devtools/split-debug.sh],[chmod +x contrib/devtools/split-debug.sh])
AM_COND_IF([HAVE_DOXYGEN], [AC_CONFIG_FILES([doc/Doxyfile])])
AC_CONFIG_LINKS([contrib/devtools/security-check.py:contrib/devtools/security-check.py])
AC_CONFIG_LINKS([contrib/devtools/symbol-check.py:contrib/devtools/symbol-check.py])
AC_CONFIG_LINKS([contrib/devtools/test-security-check.py:contrib/devtools/test-security-check.py])
AC_CONFIG_LINKS([contrib/devtools/test-symbol-check.py:contrib/devtools/test-symbol-check.py])
AC_CONFIG_LINKS([contrib/devtools/iwyu/bitcoin.core.imp:contrib/devtools/iwyu/bitcoin.core.imp])
AC_CONFIG_LINKS([contrib/filter-lcov.py:contrib/filter-lcov.py])
AC_CONFIG_LINKS([contrib/macdeploy/background.tiff:contrib/macdeploy/background.tiff])
AC_CONFIG_LINKS([src/.bear-tidy-config:src/.bear-tidy-config])
AC_CONFIG_LINKS([src/.clang-tidy:src/.clang-tidy])
AC_CONFIG_LINKS([test/functional/test_runner.py:test/functional/test_runner.py])
@@ -2012,15 +1959,7 @@ CPPFLAGS_TEMP="$CPPFLAGS"
unset CPPFLAGS
CPPFLAGS="$CPPFLAGS_TEMP"
LDFLAGS_TEMP="$LDFLAGS"
unset LDFLAGS
LDFLAGS="$LDFLAGS_TEMP"
LIBS_TEMP="$LIBS"
unset LIBS
LIBS="$LIBS_TEMP"
ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --enable-module-recovery --enable-module-schnorrsig"
ac_configure_args="${ac_configure_args} --disable-shared --with-pic --enable-benchmark=no --enable-module-recovery --disable-module-ecdh"
AC_CONFIG_SUBDIRS([src/secp256k1])
AC_OUTPUT
@@ -2037,7 +1976,6 @@ echo
echo "Options used to compile and link:"
echo " external signer = $use_external_signer"
echo " multiprocess = $build_multiprocess"
echo " with experimental syscall sandbox support = $use_syscall_sandbox"
echo " with libs = $build_bitcoin_libs"
echo " with wallet = $enable_wallet"
if test "$enable_wallet" != "no"; then

View File

@@ -26,9 +26,6 @@ The [Debian](/contrib/debian) subfolder contains the copyright file.
All other packaging related files can be found in the [bitcoin-core/packaging](https://github.com/bitcoin-core/packaging) repository.
### [Builder keys](/contrib/builder-keys)
PGP keys used for signing Bitcoin Core [release](/doc/release-process.md) results.
### [MacDeploy](/contrib/macdeploy) ###
Scripts and notes for Mac builds.
@@ -38,5 +35,11 @@ Test and Verify Tools
### [TestGen](/contrib/testgen) ###
Utilities to generate test vectors for the data-driven Bitcoin tests.
### [Verify Binaries](/contrib/verifybinaries) ###
### [Verify-Binaries](/contrib/verify-binaries) ###
This script attempts to download and verify the signature file SHA256SUMS.asc from bitcoin.org.
Command Line Tools
---------------------
### [Completions](/contrib/completions) ###
Shell completions for bash and fish.

View File

@@ -1,33 +0,0 @@
## PGP keys of builders and Developers
The file `keys.txt` contains fingerprints of the public keys of builders and
active developers.
The associated keys are mainly used to sign git commits or the build results
of Guix builds.
The most recent version of each pgp key can be found on most pgp key servers.
Fetch the latest version from the key server to see if any key was revoked in
the meantime.
To fetch the latest version of all pgp keys in your gpg homedir,
```sh
gpg --refresh-keys
```
To fetch keys of builders and active developers, feed the list of fingerprints
of the primary keys into gpg:
On \*NIX:
```sh
while read fingerprint keyholder_name; do gpg --keyserver hkps://keys.openpgp.org --recv-keys ${fingerprint}; done < ./keys.txt
```
On Windows (requires Gpg4win >= 4.0.0):
```
FOR /F "tokens=1" %i IN (keys.txt) DO gpg --keyserver hkps://keys.openpgp.org --recv-keys %i
```
Add your key to the list if you provided Guix attestations for two major or
minor releases of Bitcoin Core.

View File

@@ -1,57 +0,0 @@
9D3CC86A72F8494342EA5FD10A41BDC3F4FAFF1C Aaron Clauson (sipsorcery)
617C90010B3BD370B0AC7D424BB42E31C79111B8 Akira Takizawa (akx20000)
E944AE667CF960B1004BC32FCA662BE18B877A60 Andreas Schildbach (aschildbach)
152812300785C96444D3334D17565732E08E5E41 Andrew Chow (achow101)
590B7292695AFFA5B672CBB2E13FC145CD3F4304 Antoine Poinsot (darosior)
0AD83877C1F0CD1EE9BD660AD7CC770B81FD22A8 Ben Carman (benthecarman)
912FD3228387123DC97E0E57D5566241A0295FA9 BtcDrak (btcdrak)
04017A2A6D9A0CCDC81D8EC296AB007F1A7ED999 Carl Dong (dongcarl)
C519EBCF3B926298946783EFF6430754120EC2F4 Christian Decker (cdecker)
18AE2F798E0D239755DA4FD24B79F986CBDF8736 Chun Kuan Le (ken2812221)
101598DC823C1B5F9A6624ABA5E0907A0380E6C3 CoinForensics (CoinForensics)
F20F56EF6A067F70E8A5C99FFF95FAA971697405 centaur (centaur)
C060A6635913D98A3587D7DB1C2491FFEB0EF770 Cory Fields (cfields)
BF6273FAEF7CC0BA1F562E50989F6B3048A116B5 Dev Random (devrandom)
6D3170C1DC2C6FD0AEEBCA6743811D1A26623924 Douglas Roark (droark)
948444FCE03B05BA5AB0591EC37B1C1D44C786EE Duncan Dean (dunxen)
1C6621605EC50319C463D56C7F81D87985D61612 Emanuele Cisbani (cisba)
9A1689B60D1B3CCE9262307A2F40A9BF167FBA47 Erik Mossberg (erkmos)
D35176BE9264832E4ACA8986BF0792FBE95DC863 fivepiece (fivepiece)
6F993B250557E7B016ADE5713BDCDA2D87A881D9 Fuzzbawls (Fuzzbawls)
01CDF4627A3B88AAE4A571C87588242FBE38D3A8 Gavin Andresen (gavinandresen)
6B002C6EA3F91B1B0DF0C9BC8F617F1200A6D25C Gloria Zhao (glozow)
D1DBF2C4B96F2DEBF4C16654410108112E7EA81F Hennadii Stepanov (hebasto)
A2FD494D0021AA9B4FA58F759102B7AE654A4A5A Ilyas Ridhuan (IlyasRidhuan)
2688F5A9A4BE0F295E921E8A25F27A38A47AD566 James O'Beirne (jamesob)
D3F22A3A4C366C2DCB66D3722DA9C5A7FA81EA35 Jarol Rodriguez (jarolrod)
7480909378D544EA6B6DCEB7535B12980BB8A4D3 Jeffri H Frontz (jhfrontz)
D3CC177286005BB8FF673294C5242A1AB3936517 jl2012 (jl2012)
82921A4B88FD454B7EB8CE3C796C4109063D4EAF Jon Atack (jonatack)
32EE5C4C3FA15CCADB46ABE529D4BCB6416F53EC Jonas Schnelli (jonasschnelli)
4B4E840451149DD7FB0D633477DFAB5C3108B9A8 Jorge Timon (jtimon)
C42AFF7C61B3E44A1454CD3557AF762DB3353322 Karl-Johan Alm (kallewoof)
70A1D47DD44F59DF8B22244333E472FE870C7E5D Kristaps Kaupe (kristapsk)
30DE693AE0DE9E37B3E7EB6BBFF0F67810C1EED1 Lisa Neigut (niftynei)
E463A93F5F3117EEDE6C7316BD02942421F4889F Luke Dashjr (luke-jr)
B8B3F1C0E58C15DB6A81D30C3648A882F4316B9B Marco Falke (marco)
07DF3E57A548CCFB7530709189BBB8663E2E65CE Matt Corallo (BlueMatt)
CA03882CB1FC067B5D3ACFE4D300116E1C875A3D MeshCollider (meshcollider)
E777299FC265DD04793070EB944D35F9AC3DB76A Michael Ford (fanquake)
AD5764F4ADCE1B99BDFD179E12335A271D4D62EC Michael Tidwell (miketwenty1)
9692B91BBF0E8D34DFD33B1882C5C009628ECF0C Michagogo (michagogo)
C57E4B42223FDE851D4F69DD28DF2724F241D8EE midnightmagic (midnightmagic)
F4FC70F07310028424EFC20A8E4256593F177720 Oliver Gugger (guggero, Oliver Gugger)
D62A803E27E7F43486035ADBBCD04D8E9CCCAC2A Paul Rabahy (prab)
37EC7D7B0A217CDB4B4E007E7FAB114267E4FA04 Peter Todd (petertodd)
D762373D24904A3E42F33B08B9A408E71DAAC974 Pieter Wuille [Location: Leuven, Belgium] (sipa)
133EAC179436F14A5CF1B794860FEB804E669320 Pieter Wuille (sipa)
6A8F9C266528E25AEB1D7731C2371D91CB716EA7 Sebastian Falbesoner (theStack)
A8FC55F3B04BA3146F3492E79303B33A305224CB Sebastian Kung (TheCharlatan)
ED9BDF7AD6A55E232E84524257FF9BDBCC301009 Sjors Provoost (sjors)
867345026B6763E8B07EE73AB6737117397F5C4F Stephan Oeste (Emzy)
9EDAFF80E080659604F4A76B2EBB056FD847F8A7 Stephan Oeste (Emzy)
6DEEF79B050C4072509B743F8C275BC595448867 Tomas Kanocz (KanoczTomas)
AEC1884398647C47413C1C3FB1179EB7347DC10D Warren Togami (wtogami)
74E2DEF5D77260B98BC19438099BAD163C70FBFA Will Clark (will8clark)
79D00BAC68B56D422F945A8F8E3A8F3247DBCBBF Willy Ko (willyko)
71A3B16735405025D447E8F274810B012346C9A6 Wladimir J. van der Laan (laanwj)

View File

@@ -1,5 +1,5 @@
# bash programmable completion for bitcoin-cli(1)
# Copyright (c) 2012-2019 The Bitcoin Core developers
# Copyright (c) 2012-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.

View File

@@ -1,5 +1,5 @@
# bash programmable completion for bitcoin-tx(1)
# Copyright (c) 2016 The Bitcoin Core developers
# Copyright (c) 2016-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.

View File

@@ -1,5 +1,5 @@
# bash programmable completion for bitcoind(1) and bitcoin-qt(1)
# Copyright (c) 2012-2019 The Bitcoin Core developers
# Copyright (c) 2012-2022 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.

View File

@@ -0,0 +1,99 @@
# Disable files from being included in completions by default
complete --command bitcoin-cli --no-files
function __fish_bitcoin_cli_get_commands_helper
set --local cmd (commandline -oc)
# Don't return commands if '-help or -?' in commandline
if string match --quiet --regex -- '^-help$|^-\?$' $cmd
return
end
# Strip help cmd from token to avoid duplication errors
set --local cmd (string match --invert --regex -- '^help$' $cmd)
# Strip -stdin* options to avoid waiting for input while we fetch completions
# TODO: this appears to be broken when run as tab completion (requires ctrl+c to exit)
set --local cmd (string match --invert --regex -- '^-stdin.*$' $cmd)
# Match, format and return commands
for command in ($cmd help 2>&1 | string match --invert -r '^\=\=.*' | string match --invert -r '^\\s*$')
echo $command
end
end
function __fish_bitcoin_cli_get_commands
argparse 'nohelp' 'commandsonly' -- $argv
set --local commands
# Exclude description, exclude help
if set -q _flag_nohelp; and set -q _flag_commandsonly
set --append commands (__fish_bitcoin_cli_get_commands_helper | string replace -r ' .*$' '' | string match --invert -r 'help')
# Include description, exclude help
else if set -q _flag_nohelp
set --append commands (__fish_bitcoin_cli_get_commands_helper | string replace ' ' \t | string match --invert -r 'help')
# Exclude description, include help
else if set -q _flag_commandsonly
set --append commands (__fish_bitcoin_cli_get_commands_helper | string replace -r ' .*$' '')
# Include description, include help
else
set --append commands (__fish_bitcoin_cli_get_commands_helper | string replace ' ' \t)
end
if string match -q -r '^.*error.*$' $commands[1]
# RPC offline or RPC wallet not loaded
return
else
for command in $commands
echo $command
end
end
end
function __fish_bitcoin_cli_get_options
argparse 'nofiles' -- $argv
set --local cmd (commandline -oc)
# Don't return options if '-help or -?' in commandline
if string match --quiet --regex -- '^-help$|-\?$' $cmd
return
end
set --local options
if set -q _flag_nofiles
set --append options ($cmd -help 2>&1 | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=' | string match --invert -r '^.*=$')
else
set --append options ($cmd -help 2>&1 | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=' | string match -r '^.*=$')
end
for option in $options
echo $option
end
end
# Add options with file completion
# Don't offer after a command is given
complete \
--command bitcoin-cli \
--no-files \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_cli_get_commands --commandsonly)" \
--arguments "(__fish_bitcoin_cli_get_options)"
# Enable file completions only if the commandline now contains a `*.=` style option
complete --command bitcoin-cli \
--condition 'string match --regex -- ".*=" (commandline -pt)' \
--force-files
# Add options without file completion
# Don't offer after a command is given
complete \
--command bitcoin-cli \
--no-files \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_cli_get_commands --commandsonly)" \
--arguments "(__fish_bitcoin_cli_get_options --nofiles)"
# Add commands
# Permit command completions after `bitcoin-cli help` but not after other commands
complete \
--command bitcoin-cli \
--no-files \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_cli_get_commands --commandsonly --nohelp)" \
--arguments "(__fish_bitcoin_cli_get_commands)"

View File

@@ -0,0 +1,35 @@
# Disable files from being included in completions by default
complete --command bitcoin-qt --no-files
# Extract options
function __fish_bitcoinqt_get_options
argparse 'nofiles' -- $argv
set --local cmd (commandline -opc)[1]
set --local options
if set -q _flag_nofiles
set --append options ($cmd -help-debug | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=' | string match --invert -r '^.*=$')
else
set --append options ($cmd -help-debug | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=' | string match -r '^.*=$')
end
for option in $options
echo $option
end
end
# Add options with file completion
complete \
--command bitcoin-qt \
--arguments "(__fish_bitcoinqt_get_options)"
# Enable file completions only if the commandline now contains a `*.=` style option
complete -c bitcoin-qt \
--condition 'string match --regex -- ".*=" (commandline -pt)' \
--force-files
# Add options without file completion
complete \
--command bitcoin-qt \
--arguments "(__fish_bitcoinqt_get_options --nofiles)"

View File

@@ -0,0 +1,65 @@
# Disable files from being included in completions by default
complete --command bitcoin-tx --no-files
# Modified version of __fish_seen_subcommand_from
# Uses regex to detect cmd= syntax
function __fish_bitcoin_seen_cmd
set -l cmd (commandline -oc)
set -e cmd[1]
for i in $cmd
for j in $argv
if string match --quiet --regex -- "^$j.*" $i
return 0
end
end
end
return 1
end
# Extract options
function __fish_bitcoin_tx_get_options
set --local cmd (commandline -oc)[1]
if string match --quiet --regex -- '^-help$|-\?$' $cmd
return
end
for option in ($cmd -help 2>&1 | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=')
echo $option
end
end
# Extract commands
function __fish_bitcoin_tx_get_commands
argparse 'commandsonly' -- $argv
set --local cmd (commandline -oc)[1]
set --local commands
if set -q _flag_commandsonly
set --append commands ($cmd -help | sed -e '1,/Commands:/d' -e 's/=/=\t/' -e 's/(=/=/' -e '/^ [a-z]/ p' -e d | string replace -r '\ \ ' '' | string replace -r '=.*' '')
else
set --append commands ($cmd -help | sed -e '1,/Commands:/d' -e 's/=/=\t/' -e 's/(=/=/' -e '/^ [a-z]/ p' -e d | string replace -r '\ \ ' '')
end
for command in $commands
echo $command
end
end
# Add options
complete \
--command bitcoin-tx \
--condition "not __fish_bitcoin_seen_cmd (__fish_bitcoin_tx_get_commands --commandsonly)" \
--arguments "(__fish_bitcoin_tx_get_options)" \
--no-files
# Add commands
complete \
--command bitcoin-tx \
--arguments "(__fish_bitcoin_tx_get_commands)" \
--no-files
# Add file completions for load and set commands
complete \
--command bitcoin-tx \
--condition 'string match --regex -- "(load|set)=" (commandline -pt)' \
--force-files

View File

@@ -0,0 +1,38 @@
# Disable files from being included in completions by default
complete --command bitcoin-util --no-files
# Extract options
function __fish_bitcoin_util_get_options
set --local cmd (commandline -opc)[1]
set --local options
set --append options ($cmd -help 2>&1 | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=')
for option in $options
echo $option
end
end
# Extract commands
function __fish_bitcoin_util_get_commands
set --local cmd (commandline -opc)[1]
set --local commands
set --append commands ($cmd -help | sed -e '1,/Commands:/d' -e 's/=/=\t/' -e 's/(=/=/' -e '/^ [a-z]/ p' -e d | string replace -r '\ \ ' '')
for command in $commands
echo $command
end
end
# Add options
complete \
--command bitcoin-util \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_util_get_commands)" \
--arguments "(__fish_bitcoin_util_get_options)"
# Add commands
complete \
--command bitcoin-util \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_util_get_commands)" \
--arguments "(__fish_bitcoin_util_get_commands)"

View File

@@ -0,0 +1,35 @@
# Disable files from being included in completions by default
complete --command bitcoin-wallet --no-files
# Extract options
function __fish_bitcoin_wallet_get_options
set --local cmd (commandline -opc)[1]
for option in ($cmd -help 2>&1 | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=')
echo $option
end
end
# Extract commands
function __fish_bitcoin_wallet_get_commands
set --local cmd (commandline -opc)[1]
for command in ($cmd -help | sed -e '1,/Commands:/d' -e 's/=/=\t/' -e 's/(=/=/' -e '/^ [a-z]/ p' -e d | string replace -r '\ \ ' '')
echo $command
end
end
# Add options
complete \
--command bitcoin-wallet \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_wallet_get_commands)" \
--arguments "(__fish_bitcoin_wallet_get_options)"
# Add commands
complete \
--command bitcoin-wallet \
--condition "not __fish_seen_subcommand_from (__fish_bitcoin_wallet_get_commands)" \
--arguments "(__fish_bitcoin_wallet_get_commands)"
# Add file completions for load and set commands
complete --command bitcoin-wallet \
--condition "string match -r -- '(dumpfile|datadir)*=' (commandline -pt)" \
--force-files

View File

@@ -0,0 +1,35 @@
# Disable files from being included in completions by default
complete --command bitcoind --no-files
# Extract options
function __fish_bitcoind_get_options
argparse 'nofiles' -- $argv
set --local cmd (commandline -opc)[1]
set --local options
if set -q _flag_nofiles
set --append options ($cmd -help-debug | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=' | string match --invert -r '^.*=$')
else
set --append options ($cmd -help-debug | string match -r '^ -.*' | string replace -r ' -' '-' | string replace -r '=.*' '=' | string match -r '^.*=$')
end
for option in $options
echo $option
end
end
# Add options with file completion
complete \
--command bitcoind \
--arguments "(__fish_bitcoind_get_options)"
# Enable file completions only if the commandline now contains a `*.=` style option
complete --command bitcoind \
--condition 'string match --regex -- ".*=" (commandline -pt)' \
--force-files
# Add options without file completion
complete \
--command bitcoind \
--arguments "(__fish_bitcoind_get_options --nofiles)"

View File

@@ -5,24 +5,16 @@ Upstream-Contact: Satoshi Nakamoto <satoshin@gmx.com>
Source: https://github.com/bitcoin/bitcoin
Files: *
Copyright: 2009-2022, Bitcoin Core Developers
Copyright: 2009-2023, Bitcoin Core Developers
License: Expat
Comment: The Bitcoin Core Developers encompasses the current developers listed on bitcoin.org,
as well as the numerous contributors to the project.
Comment: The Bitcoin Core Developers encompasses all contributors to the
project, listed in the release notes or the git log.
Files: debian/*
Copyright: 2010-2011, Jonas Smedegaard <dr@jones.dk>
2011, Matt Corallo <matt@bluematt.me>
License: GPL-2+
Files: src/secp256k1/build-aux/m4/ax_jni_include_dir.m4
Copyright: 2008 Don Anderson <dda@sleepycat.com>
License: GNU-All-permissive-License
Files: src/secp256k1/build-aux/m4/ax_prog_cc_for_build.m4
Copyright: 2008 Paolo Bonzini <bonzini@gnu.org>
License: GNU-All-permissive-License
Files: src/qt/res/icons/add.png
src/qt/res/icons/address-book.png
src/qt/res/icons/chevron.png
@@ -112,12 +104,6 @@ License: Expat
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
License: GNU-All-permissive-License
Copying and distribution of this file, with or without modification, are
permitted in any medium without royalty provided the copyright notice
and this notice are preserved. This file is offered as-is, without any
warranty.
License: GPL-2+
This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the

View File

@@ -83,13 +83,23 @@ A small script to automatically create manpages in ../../doc/man by running the
This requires help2man which can be found at: https://www.gnu.org/software/help2man/
With in-tree builds this tool can be run from any directory within the
repostitory. To use this tool with out-of-tree builds set `BUILDDIR`. For
repository. To use this tool with out-of-tree builds set `BUILDDIR`. For
example:
```bash
BUILDDIR=$PWD/build contrib/devtools/gen-manpages.py
```
headerssync-params.py
=====================
A script to generate optimal parameters for the headerssync module (src/headerssync.cpp). It takes no command-line
options, as all its configuration is set at the top of the file. It runs many times faster inside PyPy. Invocation:
```bash
pypy3 contrib/devtools/headerssync-params.py
```
gen-bitcoin-conf.sh
===================

View File

@@ -0,0 +1,56 @@
cmake_minimum_required(VERSION 3.9)
project(bitcoin-tidy VERSION 1.0.0 DESCRIPTION "clang-tidy checks for Bitcoin Core")
include(GNUInstallDirs)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED True)
set(CMAKE_CXX_EXTENSIONS False)
# TODO: Figure out how to avoid the terminfo check
find_package(LLVM REQUIRED CONFIG)
find_program(CLANG_TIDY_EXE NAMES "clang-tidy-${LLVM_VERSION_MAJOR}" "clang-tidy" HINTS ${LLVM_TOOLS_BINARY_DIR})
message(STATUS "Found LLVM ${LLVM_PACKAGE_VERSION}")
message(STATUS "Found clang-tidy: ${CLANG_TIDY_EXE}")
add_library(bitcoin-tidy MODULE bitcoin-tidy.cpp logprintf.cpp)
target_include_directories(bitcoin-tidy SYSTEM PRIVATE ${LLVM_INCLUDE_DIRS})
# Disable RTTI and exceptions as necessary
if (MSVC)
target_compile_options(bitcoin-tidy PRIVATE /GR-)
else()
target_compile_options(bitcoin-tidy PRIVATE -fno-rtti)
target_compile_options(bitcoin-tidy PRIVATE -fno-exceptions)
endif()
if(CMAKE_HOST_APPLE)
# ld64 expects no undefined symbols by default
target_link_options(bitcoin-tidy PRIVATE -Wl,-flat_namespace)
target_link_options(bitcoin-tidy PRIVATE -Wl,-undefined -Wl,suppress)
endif()
# Add warnings
if (MSVC)
target_compile_options(bitcoin-tidy PRIVATE /W4)
else()
target_compile_options(bitcoin-tidy PRIVATE -Wall)
target_compile_options(bitcoin-tidy PRIVATE -Wextra)
endif()
if(CMAKE_VERSION VERSION_LESS 3.27)
set(CLANG_TIDY_COMMAND "${CLANG_TIDY_EXE}" "--load=${CMAKE_BINARY_DIR}/${CMAKE_SHARED_MODULE_PREFIX}bitcoin-tidy${CMAKE_SHARED_MODULE_SUFFIX}" "-checks=-*,bitcoin-*")
else()
# CLANG_TIDY_COMMAND supports generator expressions as of 3.27
set(CLANG_TIDY_COMMAND "${CLANG_TIDY_EXE}" "--load=$<TARGET_FILE:bitcoin-tidy>" "-checks=-*,bitcoin-*")
endif()
# Create a dummy library that runs clang-tidy tests as a side-effect of building
add_library(bitcoin-tidy-tests OBJECT EXCLUDE_FROM_ALL example_logprintf.cpp)
add_dependencies(bitcoin-tidy-tests bitcoin-tidy)
set_target_properties(bitcoin-tidy-tests PROPERTIES CXX_CLANG_TIDY "${CLANG_TIDY_COMMAND}")
install(TARGETS bitcoin-tidy LIBRARY DESTINATION ${CMAKE_INSTALL_LIBDIR})

View File

@@ -0,0 +1,11 @@
# Bitcoin Tidy
Example Usage:
```bash
cmake -S . -B build -DLLVM_DIR=$(llvm-config --cmakedir) -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
cmake --build build --target bitcoin-tidy-tests -j$(nproc)
```

View File

@@ -0,0 +1,22 @@
// Copyright (c) 2023 Bitcoin Developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include "logprintf.h"
#include <clang-tidy/ClangTidyModule.h>
#include <clang-tidy/ClangTidyModuleRegistry.h>
class BitcoinModule final : public clang::tidy::ClangTidyModule
{
public:
void addCheckFactories(clang::tidy::ClangTidyCheckFactories& CheckFactories) override
{
CheckFactories.registerCheck<bitcoin::LogPrintfCheck>("bitcoin-unterminated-logprintf");
}
};
static clang::tidy::ClangTidyModuleRegistry::Add<BitcoinModule>
X("bitcoin-module", "Adds bitcoin checks.");
volatile int BitcoinModuleAnchorSource = 0;

View File

@@ -0,0 +1,108 @@
// Copyright (c) 2023 Bitcoin Developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <string>
// Test for bitcoin-unterminated-logprintf
enum LogFlags {
NONE
};
enum Level {
None
};
template <typename... Args>
static inline void LogPrintf_(const std::string& logging_function, const std::string& source_file, const int source_line, const LogFlags flag, const Level level, const char* fmt, const Args&... args)
{
}
#define LogPrintLevel_(category, level, ...) LogPrintf_(__func__, __FILE__, __LINE__, category, level, __VA_ARGS__)
#define LogPrintf(...) LogPrintLevel_(LogFlags::NONE, Level::None, __VA_ARGS__)
#define LogPrint(category, ...) \
do { \
LogPrintf(__VA_ARGS__); \
} while (0)
class CWallet
{
std::string GetDisplayName() const
{
return "default wallet";
}
public:
template <typename... Params>
void WalletLogPrintf(const char* fmt, Params... parameters) const
{
LogPrintf(("%s " + std::string{fmt}).c_str(), GetDisplayName(), parameters...);
};
};
struct ScriptPubKeyMan
{
std::string GetDisplayName() const
{
return "default wallet";
}
template <typename... Params>
void WalletLogPrintf(const char* fmt, Params... parameters) const
{
LogPrintf(("%s " + std::string{fmt}).c_str(), GetDisplayName(), parameters...);
};
};
void good_func()
{
LogPrintf("hello world!\n");
}
void good_func2()
{
CWallet wallet;
wallet.WalletLogPrintf("hi\n");
ScriptPubKeyMan spkm;
spkm.WalletLogPrintf("hi\n");
const CWallet& walletref = wallet;
walletref.WalletLogPrintf("hi\n");
auto* walletptr = new CWallet();
walletptr->WalletLogPrintf("hi\n");
delete walletptr;
}
void bad_func()
{
LogPrintf("hello world!");
}
void bad_func2()
{
LogPrintf("");
}
void bad_func3()
{
// Ending in "..." has no special meaning.
LogPrintf("hello world!...");
}
void bad_func4_ignored()
{
LogPrintf("hello world!"); // NOLINT(bitcoin-unterminated-logprintf)
}
void bad_func5()
{
CWallet wallet;
wallet.WalletLogPrintf("hi");
ScriptPubKeyMan spkm;
spkm.WalletLogPrintf("hi");
const CWallet& walletref = wallet;
walletref.WalletLogPrintf("hi");
auto* walletptr = new CWallet();
walletptr->WalletLogPrintf("hi");
delete walletptr;
}

Some files were not shown because too many files have changed in this diff Show More