41194 Commits

Author SHA1 Message Date
Ava Chow
76a33be21d
Merge bitcoin/bitcoin#28307: rpc, wallet: fix incorrect segwit redeem script size limit
2451a217dd2c21b6d2f2b2699ceddd0bf9073019 test: addmultisigaddress, coverage for script size limits (furszy)
53302a09817e5b799d345dfea432546a55a9d727 bugfix: addmultisigaddress, add unsupported operation for redeem scripts over 520 bytes (furszy)
9be6065cc03f2408f290a332b203eef9c9cebf24 test: coverage for 16-20 segwit multisig scripts (furszy)
9d9a91c4ea6b3bb32ef4131bca86f1d6683fc901 rpc: bugfix, incorrect segwit redeem script size used in signrawtransactionwithkey (furszy)
0c9fedfc45fa7cbd6801ca5fd756863ec9a6911c fix incorrect multisig redeem script size limit for segwit (furszy)
f7a173b5785cda460470df9a74a0e0f94d7f9a18 test: rpc_createmultisig, decouple 'test_sortedmulti_descriptors_bip67' (furszy)
4f33dbd8f8c0e29f37b04e6af6d2c7905ecceaf6 test: rpc_createmultisig, decouple 'test_mixing_uncompressed_and_compressed_keys' (furszy)
25a81705d376e8c96dad45436ae3fca975b3daf5 test: rpc_createmultisig, remove unnecessary checkbalances() (furszy)
b5a328943362cfac6e90fd4e1b167c357d53b7d4 test: refactor, multiple cleanups in rpc_createmultisig.py (furszy)
3635d432681847313c098f9827483372a840e70f test: rpc_createmultisig, remove manual wallet initialization (furszy)

Pull request description:

  Fixing https://github.com/bitcoin/bitcoin/issues/28250#issuecomment-1674830104 and more.

  Currently, redeem scripts longer than 520 bytes, which are technically valid under segwit rules, have flaws in the following processes:
  1) The multisig creation process fails to deduce the output descriptor, resulting in the generation of an incorrect descriptor. Additionally, the accompanying user warning is also inaccurate.
  2) The `signrawtransactionwithkey` RPC command fail to sign them.
  3) The legacy wallet `addmultisigaddress` wrongly discards them.

  The issue arises because most of these flows are utilizing the legacy spkm keystore, which imposes
  the [p2sh max redeem script size rule](ded6873340/src/script/signingprovider.cpp (L160)) on all scripts. Which blocks segwit redeem scripts longer than
  the max element size in all the previously mentioned processes (`createmultisig`, `addmultisigaddress`, and
  `signrawtransactionwithkey`).

  This PR fixes the problem, enabling the creation of multisig output descriptors involving more than 15 keys and
  allowing the signing of these scripts, along with other post-segwit redeem scripts that surpass the 520-byte
  p2sh limit.

  Important note:
  Instead of adding support for these longer redeem scripts in the legacy wallet, an "unsupported operation"
  error has been added. The reasons behind this decision are:

  1) The introduction of this feature brings about a compatibility-breaking change that requires downgrade
      protection; older wallets would be unable to interact with these "new" legacy wallets.

  2) Considering the ongoing deprecation of the legacy spkm, this issue provides another compelling
      reason to transition towards descriptors.

  Testing notes:
  To easily verify each of the fixes, I decoupled the tests into standalone commits. So they can be
  cherry-picked on top of master. Where `rpc_createmultisig.py` (with and without the `--legacy-wallet`
  arg) will fail without the bugs fixes commits.

  Extra note:
  The initial commits improves the `rpc_createmultisig.py` test in many ways. I found this test very
  antiquated, screaming for an update and cleanup.

ACKs for top commit:
  pinheadmz:
    ACK 2451a217dd2c21b6d2f2b2699ceddd0bf9073019
  theStack:
    Code-review ACK 2451a217dd2c21b6d2f2b2699ceddd0bf9073019
  achow101:
    ACK 2451a217dd2c21b6d2f2b2699ceddd0bf9073019

Tree-SHA512: 71794533cbd46b3a1079fb4e9d190d3ea3b615de0cbfa443466e14f05e4616ca90e12ce2bf07113515ea8113e64a560ad572bb9ea9d4835b6fb67b6ae596167f
2024-06-04 21:39:49 -04:00
Ava Chow
b3a61bd7b1
Merge bitcoin/bitcoin#28074: fuzz: wallet, add target for Crypter
d7290d662f494503f28e087dd728b492c0bb2c5f fuzz: wallet, add target for Crypter (Ayush Singh)

Pull request description:

  This PR adds fuzz coverage for `wallet/crypter`.

  Motivation: Issue [27272](https://github.com/bitcoin/bitcoin/issues/27272#issue-1628327906)

  I ran this for a long time with Sanitizers on and had no crashes; the average `exec/sec` also looks good to me. However, I would really appreciate it if some of the reviewers could try it on their machines too, and give their feedback.

ACKs for top commit:
  maflcko:
    utACK d7290d662f494503f28e087dd728b492c0bb2c5f
  achow101:
    ACK d7290d662f494503f28e087dd728b492c0bb2c5f
  brunoerg:
    utACK d7290d662f494503f28e087dd728b492c0bb2c5f

Tree-SHA512: f5c496cabdd3263a7e1ad49eeff702725336f76bf19a82e5dbbead082e990889dd43c851d0d2d6ab740f44b8ec2aa06defd9ff6b02be68b5f8b4eaf963f88599
2024-06-04 21:26:42 -04:00
Ava Chow
55cf34a5c3
Merge bitcoin/bitcoin#30047: refactor: Model the bech32 charlimit as an Enum
7f3f6c6dc80247e6dfb0d406dc53bc8198f029fd refactor: replace hardcoded numbers (Lőrinc)
5676aec1e1a6d2c6fd3099e120e263a0a7def089 refactor: Model the bech32 charlimit as an Enum (josibake)

Pull request description:

  Broken out from #28122

  ---

  Bech32(m) was defined with a 90 character limit so that certain guarantees for error detection could be made for segwit addresses (see https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki#checksum-design).

  However, there is nothing about the encoding scheme itself that requires a limit of 90 and in practice bech32(m) is being used without the 90 char limit (e.g. lightning invoices, silent payments). Further, increasing the character limit doesn't do away with error detection, it simply changes the guarantee.

  The primary motivation for this change is for being able to parse BIP352 v0 silent payment addresses (see 622c7a98b9), which require up to 118 characters. In addition to BIP352, modeling the character limit as an enum allows us to easily support new address types that use bech32m and specify their own character limit.

ACKs for top commit:
  paplorinc:
    re-ACK 7f3f6c6dc80247e6dfb0d406dc53bc8198f029fd
  achow101:
    ACK 7f3f6c6dc80247e6dfb0d406dc53bc8198f029fd
  theuni:
    utACK 7f3f6c6dc80247e6dfb0d406dc53bc8198f029fd

Tree-SHA512: 9c793d657448c1f795093b9f7d4d6dfa431598f48d54e1c899a69fb2f43aeb68b40ca2ff08864eefeeb6627d4171877234b5df0056ff2a2b84415bc3558bd280
2024-06-04 20:32:25 -04:00
Ava Chow
09fe1435d9
Merge bitcoin/bitcoin#29997: rpc: Remove index-based Arg accessor
fa3169b0732d7eb4b9166e7ecc6b7cfb669a9b54 rpc: Remove index-based Arg accessor (MarcoFalke)

Pull request description:

  The index-based Arg accessor is redundant with the name-based one. It does not provide any benefit to the code reader, or otherwise, so remove it.

ACKs for top commit:
  stickies-v:
    re-ACK fa3169b0732d7eb4b9166e7ecc6b7cfb669a9b54, addressed doc nits
  achow101:
    ACK fa3169b0732d7eb4b9166e7ecc6b7cfb669a9b54
  ryanofsky:
    Code review ACK fa3169b0732d7eb4b9166e7ecc6b7cfb669a9b54. One changes since last review are some documentation improvements

Tree-SHA512: f9da1c049dbf38c3b47a8caf8d24d195c2d4b88c7ec45a9ccfb78f1e39f29cb86869f84b308f6e49856b074c06604ab634c90eb89c9c93d2a8169e070aa1bd40
2024-06-04 20:11:59 -04:00
Ava Chow
56ea8ed3d3
Merge bitcoin/bitcoin#29428: test: Assumeutxo: snapshots with less work should not be loaded
df6dc2aaaeffc664006b86ee8c8797dc484ec40e test: Assumeutxo: snapshots with less work should not be loaded (Hernan Marino)

Pull request description:

  This PR adds a test which checks that snapshots with less accumulated work than the node's active chain, should not be loaded and return with an error. Although in a different context of discussion the missing test was detect in a thread in https://github.com/bitcoin/bitcoin/pull/29394 (see https://github.com/bitcoin/bitcoin/pull/29394#discussion_r1484122214)

ACKs for top commit:
  maflcko:
    utACK df6dc2aaaeffc664006b86ee8c8797dc484ec40e
  kevkevinpal:
    utACK [df6dc2a](df6dc2aaae)
  achow101:
    ACK df6dc2aaaeffc664006b86ee8c8797dc484ec40e
  alfonsoromanz:
    Re ACK df6dc2aaaeffc664006b86ee8c8797dc484ec40e. Make is successful and the test passes.

Tree-SHA512: 07a394b4b288cc8ad3f66ed4e70dcda468db18113e9442eb7215cf491768432d55efaaa5b79d633094917e05475a30f0c5e4f64f8f2da293ba306891b4485560
2024-06-04 19:13:03 -04:00
Ava Chow
a937458904
Merge bitcoin/bitcoin#30154: doc: update mention of generating bitcoin.conf
9013e2b97e8f50d2be63ce740c42d0b0e0b9b7f2 Link to gen-bitcoin-conf.sh instead of bitcoin.conf placeholder (Epic Curious)

Pull request description:

  Closes #30153.

  This PR updates `doc/init.md` to mention generating an example bitcoin.conf instead of referencing the placeholder `share/examples/bitcoin.conf`. Also changes the code-formatted text to a markdown link.

  ## Background

  - Two years ago, `share/examples/bitcoin.conf` was replaced with [a placeholder file](b483084d86). To see an example `bitcoin.conf`, the user now runs the `contrib/devtools/gen-bitcoin-conf.sh` script, which replaces the placeholder file with the parsed contents of `bitcoind --help`.

  - The instructions in `init.md` about an example `bitcoin.conf` haven't changed significantly since they were [added almost 10 years ago](234bfbf6a5/doc/init.md (L39)). They should be updated to improve clarity.

ACKs for top commit:
  edilmedeiros:
    ACK 9013e2b97e8f50d2be63ce740c42d0b0e0b9b7f2
  kevkevinpal:
    reACK [9013e2b](9013e2b97e)
  achow101:
    ACK 9013e2b97e8f50d2be63ce740c42d0b0e0b9b7f2
  stickies-v:
    ACK 9013e2b97e8f50d2be63ce740c42d0b0e0b9b7f2
  tdb3:
    ACK for 9013e2b97e8f50d2be63ce740c42d0b0e0b9b7f2

Tree-SHA512: 5ac5ad672ad181d574e19e29c3727fb9e5373282444fae09b42d113d5c8915cb2829d496212638cdc4b70540b7e1794a751fcdc9539f956a594cddd70c8fd747
2024-06-04 19:00:59 -04:00
Ava Chow
e54c392356
Merge bitcoin/bitcoin#28979: wallet, rpc: document and update sendall behavior around unconfirmed inputs
71aae72e1fc998b2629d68a7301d85dc1b65641e test: test sendall does ancestor aware funding (ishaanam)
36757941a05b65c2b61a83820afdf5effd8fc9a2 wallet, rpc: implement ancestor aware funding for sendall (ishaanam)
544131f3fba9ea07fee29f9d3ee0116cd5d8a5b2 rpc, test: test sendall spends unconfirmed change and unconfirmed inputs when specified (ishaanam)

Pull request description:

  This PR:
  - Adds a functional test that `sendall` spends unconfirmed change
  - Adds a functional test that `sendall` spends regular unconfirmed inputs when specified by user
  - Adds ancestor aware funding to `sendall` by using `calculateCombinedBumpFee` and adjusting the effective value accordingly
  - Adds a functional test for ancestor aware funding in `sendall`

ACKs for top commit:
  S3RK:
    ACK 71aae72e1fc998b2629d68a7301d85dc1b65641e
  achow101:
    ACK 71aae72e1fc998b2629d68a7301d85dc1b65641e
  furszy:
    ACK 71aae72e1f

Tree-SHA512: acaeb7c65166ce53123a1d6cb5012197202246acc02ef9f37a28154cc93afdbd77c25e840ab79bdc7e0b88904014a43ab1ddea79d5337dc310ea210634ab61f0
2024-06-04 18:46:47 -04:00
Ava Chow
701b0cf2f3
Merge bitcoin/bitcoin#28366: Fix waste calculation in SelectionResult
bd34dd85e7b8b4cc26d2173d84bbeda2e9c27624 Use `exact_target` shorthand in coinselector_tests (Murch)
7aa7e30441fe77bf8e8092916e36b004bbbfe2a7 Fold GetSelectionWaste() into ComputeAndSetWaste() (Murch)

Pull request description:

  PR #26152 moved waste calculation into SelectionResult to be able to correct the waste score on basis of the bump_fee_group_discount for overlapping ancestries. This left two functions with largely overlapping purpose, where one was simply a wrapper of the other. This PR cleans up the overlap, and fixes the double-meaning of `change_cost` where the `GetChange()` function assumed that no change was created when `change_cost` was set to 0. This behavior was exploited in a bunch of tests, but is problematic, because a `change_cost` of 0 is permitted with custom settings for feerate and discard_feerate (i.e. when they’re both 0).

ACKs for top commit:
  achow101:
    ACK bd34dd85e7b8b4cc26d2173d84bbeda2e9c27624
  furszy:
    Code ACK bd34dd85e7b8b4cc26d2173d84bbeda2e9c27624
  ismaelsadeeq:
    Code Review ACK bd34dd85e7b8b4cc26d2173d84bbeda2e9c27624

Tree-SHA512: 83a2688d45d719dc61a64b5180fe136107faccf401a59df65245c05d701748a03e85ed56fde8c9b7ef39a3ab54374dd3718c559bda5b3f55dafedfd7fed25161
2024-06-04 18:37:18 -04:00
merge-script
d39f15a8a5
Merge bitcoin/bitcoin#30211: fuzz: Make FuzzedSock fuzz friendlier
22d0f1a27ef7733b51b3c2138a8d01713df8f248 [fuzz] Avoid endless waiting in FuzzedSock::{Wait,WaitMany} (marcofleon)
a7fceda68bb62fe3d9060fcf52e33b2f64a2acf9 [fuzz] Make peeking through FuzzedSock::Recv fuzzer friendly (dergoegge)
865cdf3692590bc6b1121524fe1bee188788b791 [fuzz] Use fuzzer friendly ConsumeRandomLengthByteVector in FuzzedSock::Recv (dergoegge)

Pull request description:

  `FuzzedSock` has a few issues that block a fuzzer from making progress. See commit messages for details.

ACKs for top commit:
  marcofleon:
    Tested ACK 22d0f1a27ef7733b51b3c2138a8d01713df8f248
  brunoerg:
    utACK 22d0f1a27ef7733b51b3c2138a8d01713df8f248

Tree-SHA512: 2d66fc94ba58b6652ae234bd1dcd33b7d685b5054fe83e0cd624b053dd51519c23148f43a865ab8c8bc5fc2dc25e701952831b99159687474978a90348faa4c5
2024-06-04 14:56:47 +01:00
merge-script
9efc2af3be
Merge bitcoin/bitcoin#30217: depends: Update Boost download link
ffbc173ca1ed6b93de8bf3f88b8aed0743f4916c depends: Update Boost download link (Hennadii Stepanov)

Pull request description:

  The Boost has [migrated](https://github.com/boostorg/boost-tasks/pull/3) their downloads from from boostorg.jfrog.io to archives.boost.io.

  So do we.

  FWIW, the download speed at my location is much better :)

ACKs for top commit:
  maflcko:
    ACK ffbc173ca1ed6b93de8bf3f88b8aed0743f4916c

Tree-SHA512: 3f66675c390510ecfdacf2ac8af4a0a6bdbdf6fbfe01ed66fbe1b11bc9e935709e5492a754711cd788352c5853738fce755afd0c0480f36d0f8af31fcc108263
2024-06-04 09:11:46 +01:00
merge-script
f7a6d34449
Merge bitcoin/bitcoin#30215: doc: JSON-RPC request Content-Type is application/json
3c08e11c3ea4499e8d20609e2417cac859b3e98e doc: JSON-RPC request Content-Type is application/json (Luke Dashjr)

Pull request description:

  Specify json content type in RPC examples.

  Picks up #29946. Which needed rebasing and the commit message fixing,

ACKs for top commit:
  laanwj:
    ACK 3c08e11c3ea4499e8d20609e2417cac859b3e98e
  tdb3:
    ACK for 3c08e11c3ea4499e8d20609e2417cac859b3e98e

Tree-SHA512: 770bbbc0fb324cb63628980b13583cabf02e75079851850170587fb6eca41a70b01dcedaf1926bb6488eb9816a3cc6616fe8cee8c4b7e09aa39b7df5834ca0ec
2024-06-03 14:41:34 +01:00
merge-script
c065ae8469
Merge bitcoin/bitcoin#30134: fuzz: add more coverage for ScriptPubKeyMan
e3249f21111f1dd4beb66f10af933c34a36c30ac fuzz: add more coverage for `ScriptPubKeyMan` (brunoerg)

Pull request description:

  This PR adds more coverage for `ScriptPubKeyMan`:

  - Check `GetKey` and `HasPrivKey` after adding descriptor key.
  - Cover `GetEndRange` and `GetKeyPoolSize`.
  - Cover `MarkUnusedAddresses` with the scripts from ScriptPubKeys and `GetMetadata` with the destinations from them.

ACKs for top commit:
  marcofleon:
    Tested ACK e3249f21111f1dd4beb66f10af933c34a36c30ac. I ran the updated harness for ~9 hours on an empty corpus, generated a coverage report, and checked that the new functions mentioned were hit. Coverage of `scriptpubkeyman.cpp` increased.
  murchandamus:
    Tested ACK e3249f21111f1dd4beb66f10af933c34a36c30ac

Tree-SHA512: cfab91f6c8401174842e79209c0e9225c08f011fe9b41d0a58bcec716ae4545eaf803867f899ed7b5fbcefea45711f91894e36df082ba19732dd310cd9e61a79
2024-06-03 14:01:47 +01:00
merge-script
e40df5468d
Merge bitcoin/bitcoin#30216: build: Fix building fuzz binary on on SunOS / illumos
3299abce948f205bb1354993614b669189f9b89f build: Fix building `fuzz` binary on on SunOS / illumos (Hennadii Stepanov)

Pull request description:

  On master branch @ 457e1846d2bf6ef9d54b9ba1a330ba8bbff13091, building the `fuzz` binary fails:
  ```
  $ ./autogen.sh
  $ ./configure
  $ gmake -C src test/fuzz/fuzz
  < snip >
    CXX      test/fuzz/fuzz-http_request.o
  test/fuzz/http_request.cpp:13:10: fatal error: event2/buffer.h: No such file or directory
     13 | #include <event2/buffer.h>
        |          ^~~~~~~~~~~~~~~~~
  compilation terminated.
  gmake: *** [Makefile:17138: test/fuzz/fuzz-http_request.o] Error 1
  gmake: Leaving directory '/export/home/hebasto/git/bitcoin/src'
  ```

  The testing system:
  ```
  $ uname -a
  SunOS openindiana 5.11 illumos-82079dec87 i86pc i386 i86pc
  ```

  This PR fixes this issue.

ACKs for top commit:
  maflcko:
    ACK 3299abce948f205bb1354993614b669189f9b89f

Tree-SHA512: 43048cf0d3db47d71263da179e07225afd901ed2039ee4d17314ff7b581ab36f41282fde3b1210926cecda546320dc573937c564520f61fbb236c2b9914ed0d4
2024-06-03 12:44:06 +01:00
marcofleon
22d0f1a27e [fuzz] Avoid endless waiting in FuzzedSock::{Wait,WaitMany}
Currently, when the FuzzedDataProvider of a FuzzedSock runs out of data,
FuzzedSock::Wait and WaitMany will simulate endless waiting as the
requested events are never simulated as occured.

Fix this by simulating event occurence when ConsumeBool() returns false
(e.g. when the data provider runs out).

Co-authored-by: dergoegge <n.goeggi@gmail.com>
2024-06-03 10:32:43 +01:00
dergoegge
a7fceda68b [fuzz] Make peeking through FuzzedSock::Recv fuzzer friendly
FuzzedSock only supports peeking at one byte at a time, which is not
fuzzer friendly when trying to receive long data.

Fix this by supporting peek data of arbitrary length instead of only one
byte.
2024-06-03 10:32:43 +01:00
merge-script
80bdd4b6be
Merge bitcoin/bitcoin#30167: doc, rpc: Release notes and follow-ups for #29612
efc1b5be8a4696c0db19ba18316b2d4ed09e10f2 test: Add coverage for txid coins count check when loading snapshot (Fabian Jahr)
6b6084850b8c2ebcdbeecdb406e8732adaa6d23c assumeutxo: Add network magic ctor param to SnapshotMetadata (Fabian Jahr)
1f1f9984555d49f07ae20cb3a8153a177c546beb assumeutxo: Deserialize trailing byte instead of Txid (Fabian Jahr)
359967e310794e0bbdbe2bca38ee440a88bc4f43 doc: Add release notes for #29612 (Fabian Jahr)

Pull request description:

  This adds release notes for #29612 and addresses post-merge review comments.

ACKs for top commit:
  maflcko:
    utACK efc1b5be8a4696c0db19ba18316b2d4ed09e10f2
  theStack:
    utACK efc1b5be8a4696c0db19ba18316b2d4ed09e10f2

Tree-SHA512: 3b270202e4f7b2576090ef1d970fd54a6840d96fc3621dddd28e888fb8696a97ff69af2e000bcee3b364316ca3f6e2a9b2f1694c6184f0e704dc487823127ce4
2024-06-03 10:29:14 +01:00
merge-script
f7c0ddff46
Merge bitcoin/bitcoin#30192: build: remove --enable-lcov-branch-coverage
cbd4640ede92a1a5d7b7c1367eb7c00a9f476c62 build: remove --enable-lcov-branch-coverage (fanquake)

Pull request description:

  This supports lcov `2.x` in the sense that we are no-longer hardcoding version specific options, and instead will use the `LCOV_OPTS` variable (which is the more flexible thing to do in any case). It's also quite likely that devs are already having to pass extra options to lcov `2.x`, given it's more stringent in terms of coverage generation and error checking. See this thread for an example: https://github.com/linux-test-project/lcov/issues/238.

  Tested on one machine (LCOV 2.0, gcc 13.2) with:
  ```bash
  ./autogen.sh
  ./configure --enable-lcov CXXFLAGS="-fprofile-update=prefer-atomic" LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch"
  make
  make cov
  <snip>
  Processing file src/netaddress.cpp
    lines=521 hit=480 functions=72 hit=72 branches=675 hit=499
  Overall coverage rate:
    lines......: 81.8% (79362 of 97002 lines)
    functions......: 77.8% (10356 of 13310 functions)
    branches......: 49.6% (130628 of 263196 branches)
  ```

  and another machine (LCOV 2.1, Clang 18.1.3) with:
  ```bash
  ./autogen.sh
  ./configure --enable-lcov CC=clang CXX=clang++ LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch,inconsistent"
  make
  make cov
  <snip>
      Processing file src/util/strencodings.cpp
        lines=315 hit=311 functions=38 hit=38 branches=425 hit=357
      Overall coverage rate:
        source files: 622
        lines.......: 79.8% (70311 of 88132 lines)
        functions...: 78.1% (13968 of 17881 functions)
        branches....: 44.5% (157551 of 354317 branches)
      Message summary:
        101 warning messages:
          count: 1
          inconsistent: 100
        3528 ignore messages:
          inconsistent: 3528
  ```

  Related to #28468.

ACKs for top commit:
  theuni:
    utACK cbd4640ede92a1a5d7b7c1367eb7c00a9f476c62
  hebasto:
    ACK cbd4640ede92a1a5d7b7c1367eb7c00a9f476c62, tested on Ubuntu 22.04.

Tree-SHA512: 94eb01e0e236a480052749f6107b1d0d2e4f6f70a8eefd55fa9ba3d2f72996c9e8a0f28340698b7ac82e7a71e9cf799b7a53ddb6e435e5e9795f5f98a18820f7
2024-06-03 10:06:15 +01:00
merge-script
e18accc5f5
Merge bitcoin/bitcoin#30186: fuzz: increase txorphan harness stability
8defc182a31d828ad0f53ebf7e3be9e9cfc42d09 scripted-diff: Replace nNextSweep with m_next_sweep (marcofleon)
0048680467e15037023ceae44bc2dc8309f82f39 increase txorphan harness stability (marcofleon)

Pull request description:

  This moves `nNextSweep` from being a static variable in `LimitOrphans` to being a member of the `TxOrphanage` class. This improves the stability of the `txorphan` fuzz harness, as each orphanage (created every iteration) now has its own value for `nNextSweep`.

ACKs for top commit:
  maflcko:
    utACK 8defc182a31d828ad0f53ebf7e3be9e9cfc42d09
  dergoegge:
    Code review ACK 8defc182a31d828ad0f53ebf7e3be9e9cfc42d09
  glozow:
    utACK 8defc182a31d828ad0f53ebf7e3be9e9cfc42d09, I can rebase on this pretty easily

Tree-SHA512: 54d4a5074def764f6c895308b94e417662d2f21f157925421131745f22743907df59971f4ce545063658cd74ec133792cdd8df96ae3e69af8314e9b0ff899d48
2024-06-03 09:59:54 +01:00
Hennadii Stepanov
ffbc173ca1
depends: Update Boost download link
See: https://github.com/boostorg/boost-tasks/pull/3
2024-06-03 09:49:34 +01:00
Hennadii Stepanov
3299abce94
build: Fix building fuzz binary on on SunOS / illumos 2024-06-02 19:51:22 +01:00
Luke Dashjr
3c08e11c3e
doc: JSON-RPC request Content-Type is application/json
Specify json content type in RPC examples
2024-05-31 16:44:47 +01:00
merge-script
457e1846d2
Merge bitcoin/bitcoin#30204: depends: consolidate dependency docs
a27e1ceb9f9c9239af9b0a151c84a57573ea646a depends: consolidate dependency docs (fanquake)

Pull request description:

  Adds missing `g++` for macOS. This is needed by Qt:
  ```bash
  Configuring qt...
  Creating qmake...
  gmake[1]: Entering directory '/bitcoin/depends/work/build/arm64-apple-darwin/qt/5.15.14-4bca24c8f89/qtbase/qmake'
  gmake[1]: g++: No such file or directory
  gmake[1]: *** [Makefile:250: main.o] Error 127
  ```

  `xz-utils` was also missing (but generally already installed), and is needed for the `.tar.xz` tarballs.

  Remove `bsdmainutils`, as this is only needed by the main build system (for tests), and isn't needed to complete a depends build.

ACKs for top commit:
  maflcko:
    ACK a27e1ceb9f9c9239af9b0a151c84a57573ea646a

Tree-SHA512: 720c31d4d4c9b86fda4aace405d528193714dd3e526f38d5b8a83e4b676a433b9c891f01d86d673be9ac848458eda8a89b0981003a42eaa6d97bacc2e914396a
2024-05-31 15:30:29 +01:00
merge-script
3ac1465cd9
Merge bitcoin/bitcoin#30198: depends: qt 5.15.14 and fix macOS build with Clang 18
0a3631fc352eda849290db940844e5ef723436df depends: fix Qt macOS build with Clang 18 (fanquake)
b018bd779dd9b2d46c10254a1cf5b819ae8e027c depends: qt 5.15.14 (fanquake)

Pull request description:

  Also adds a patch to Qts internal libpng, to fix compilation using Clang 18, when targetting macOS. I'd like to get this patched, so we can continue working on removing `FORCE_USE_SYSTEM_CLANG` (#30201); otherwise builds will be broken using the default Clang (`18`) on the current Ubuntu LTS (`24.04`).

  With this PR, anyone using Ubuntu 24.04 should be able to `apt install clang llvm lld`, and then cross-compile for macOS using:
  ```bash
  # clang --version
  Ubuntu clang version 18.1.3 (1)
  make -C depends HOST=arm64-apple-darwin FORCE_USE_SYSTEM_CLANG=1
  ./autogen.sh
  CONFIG_SITE=/path/to/depends/arm64-apple-darwin/share/config.site ./configure
  make
  # file src/qt/bitcoin-qt
  src/qt/bitcoin-qt: Mach-O 64-bit arm64 executable, flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|WEAK_DEFINES|BINDS_TO_WEAK|PIE|HAS_TLV_DESCRIPTORS>
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 0a3631fc352eda849290db940844e5ef723436df
  theuni:
    utACK 0a3631fc352eda849290db940844e5ef723436df
  hebasto:
    ACK 0a3631fc352eda849290db940844e5ef723436df, a new patch indeed fixes cross-compiling on Ubuntu 24.04 with `FORCE_USE_SYSTEM_CLANG=1`.

Tree-SHA512: 711d321b1efbb1aeef802d1d7e72fff8f4e28aa2420d19df9db6f4449fc7d281e1d08ba242ce20122dfe21129e713bd59e7e6ade0b67d7271eea18b39ceb9283
2024-05-31 15:28:32 +01:00
dergoegge
865cdf3692 [fuzz] Use fuzzer friendly ConsumeRandomLengthByteVector in FuzzedSock::Recv
See comment on FuzzedDataProvider::ConsumeRandomLengthString.
2024-05-31 14:48:29 +01:00
fanquake
a27e1ceb9f
depends: consolidate dependency docs
Adds missing `g++` for macOS. This is needed by Qt:
```bash
Configuring qt...
Creating qmake...
gmake[1]: Entering directory '/bitcoin/depends/work/build/arm64-apple-darwin/qt/5.15.14-4bca24c8f89/qtbase/qmake'
gmake[1]: g++: No such file or directory
gmake[1]: *** [Makefile:250: main.o] Error 127
```

`xz-utils` was also missing (but generally already installed), and is
needed for the `.tar.xz` tarballs.

Remove bsdmainutils, as this is only needed by the main build system
(for tests), and isn't needed to complete a depends build.
2024-05-31 09:30:24 +01:00
merge-script
62f7f59ff4
Merge bitcoin/bitcoin#30199: clang-tidy: Add bugprone-move-forwarding-reference check
88cdb5967f093cf96e9184a48c0d9e34cea9d341 clang-tidy: Add `bugprone-move-forwarding-reference` check (Hennadii Stepanov)

Pull request description:

  This PR adds [`bugprone-move-forwarding-reference`](https://clang.llvm.org/extra/clang-tidy/checks/bugprone/move-forwarding-reference.html) to the clang-tidy checks.

ACKs for top commit:
  maflcko:
    utACK 88cdb5967f093cf96e9184a48c0d9e34cea9d341

Tree-SHA512: 8366c895085d0656a4491035aa8863c9dca12885c2bdf0392bebc63d6f6f5473ec263594e5fde70a3c211e95d19b9cd98e2c574ced91b4c970cce0edce40bceb
2024-05-30 15:32:08 +01:00
merge-script
0a7c650fcd
Merge bitcoin/bitcoin#30034: ci: add markdown link check job
4b7d9842691046b01f0c08d69f924ddb62ccc4c6 lint: add markdown hyperlink checker (willcl-ark)

Pull request description:

  Potential followup to: #30025

  This should prevent us reintroducing broken markdown links.

  It does not test "online" (external) links, only those within this repo. Both relative and absolute links are parsed successfully if they resolve.

ACKs for top commit:
  maflcko:
    re-utACK 4b7d9842691046b01f0c08d69f924ddb62ccc4c6
  davidgumberg:
    reACK 4b7d984269

Tree-SHA512: 9bc40d700b73499c046bb76157bc139f32ec3850f64ef813bbf7f18f9c01a253abe6a857d6f559890165f2bd26e7742c05d86232cd9b8efb33ff85d735f4f095
2024-05-30 12:36:09 +01:00
fanquake
0a3631fc35
depends: fix Qt macOS build with Clang 18
Patch Qts internal libpng to resolve the failure.

I would like to have this patched, so we can continue working on the
removal of `FORCE_USE_SYSTEM_CLANG`. Otherwise builds will be broken using
the default clang (18) on the current Ubuntu LTS (24.04).
2024-05-30 12:00:05 +01:00
Hennadii Stepanov
88cdb5967f
clang-tidy: Add bugprone-move-forwarding-reference check 2024-05-30 11:33:59 +01:00
fanquake
cbd4640ede
build: remove --enable-lcov-branch-coverage
This supports lcov 2.x in the sense that we are no-longer hardcoding
version specific options, and instead will use the `LCOV_OPTS` variable
(which is the more correct/flexible thing to do in any case). It's also
quite likely that devs are already having to pass extra options to lcov
2.x, given it's more stringent in terms of coverage generation and error
checking. See this thread for an example:
https://github.com/linux-test-project/lcov/issues/238.

Added an example to the developer notes.

Tested on one machine (LCOV 2.0, gcc 13.2) with:
```bash
./autogen.sh
./configure --enable-lcov CXXFLAGS="-fprofile-update=prefer-atomic" LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch"
make
make cov
<snip>
Processing file src/netaddress.cpp
  lines=521 hit=480 functions=72 hit=72 branches=675 hit=499
Overall coverage rate:
  lines......: 81.8% (79362 of 97002 lines)
  functions......: 77.8% (10356 of 13310 functions)
  branches......: 49.6% (130628 of 263196 branches)
```

and another machine (LCOV 2.1, Clang 18.1.3) with:
```bash
./autogen.sh
./configure --enable-lcov CC=clang CXX=clang++ LCOV_OPTS="--rc branch_coverage=1 --ignore-errors mismatch,inconsistent"
make
make cov
<snip>
Processing file src/util/strencodings.cpp
  lines=315 hit=311 functions=38 hit=38 branches=425 hit=357
Overall coverage rate:
  source files: 622
  lines.......: 79.8% (70311 of 88132 lines)
  functions...: 78.1% (13968 of 17881 functions)
  branches....: 44.5% (157551 of 354317 branches)
Message summary:
  101 warning messages:
    count: 1
    inconsistent: 100
  3528 ignore messages:
    inconsistent: 3528
```
2024-05-30 10:36:55 +01:00
fanquake
b018bd779d
depends: qt 5.15.14 2024-05-30 09:29:11 +01:00
merge-script
f61ede574c
Merge bitcoin/bitcoin#30049: build, test, doc: Temporarily remove Android-related stuff
5deb0b024e14c7c63d405c651d1ca323560a1c21 build, test, doc: Temporarily remove Android-related stuff (Hennadii Stepanov)

Pull request description:

  Previously, our Android builds were geared towards generating APKs, which relied on Qt. However, after migrating to C++20, compiling for Android became unfeasible due to Qt 5.15's compatibility limitations with NDK only up to r25, which includes an outdated embedded libc++ (see https://github.com/bitcoin/bitcoin/issues/29360).

  All removed stuff will be reinstated after migrating the build system to CMake and upgrading Qt to version 6.x.

  This PR makes possible a clean migration to the CMake-based build system as it removes code, which is not used at this moment.

ACKs for top commit:
  vasild:
    ACK 5deb0b024e14c7c63d405c651d1ca323560a1c21
  fanquake:
    ACK 5deb0b024e14c7c63d405c651d1ca323560a1c21 - given none of this is currently tested/wont compile. Can be revisted in future.

Tree-SHA512: 3bc2ccfe881e11cc1d78c27acd6f1d86cfba86821ef3bb5eca2e80d978fdfa13659ec82284dcaadc507e2394524dea91d4b8f81d0030c1cef9708df8be76bf07
2024-05-30 09:25:42 +01:00
marcofleon
8defc182a3 scripted-diff: Replace nNextSweep with m_next_sweep
-BEGIN VERIFY SCRIPT-
sed -i 's/nNextSweep/m_next_sweep/g' $(git grep -l 'nNextSweep')
-END VERIFY SCRIPT-

fixing to match style
2024-05-29 09:02:07 -07:00
marcofleon
0048680467 increase txorphan harness stability
initialize variable
2024-05-29 08:22:11 -07:00
merge-script
10164916f7
Merge bitcoin/bitcoin#30122: bench: enable wallet creation benchmarks on all platforms
7c8abf3c2001152423da06d25f9f4906611685ea bench: bugfix, properly release wallet before erasing directory (furszy)

Pull request description:

  Simple fix for #29816.

  Since the wallet is appended to the global `WalletContext` during
  creation, merely calling `reset()` on the benchmark shared_pointer
  is insufficient to destruct the wallet. This no destruction of the
  wallet object results in keeping the db connection open, which
  was causes the `fs::remove_all()` failure on Windows.

ACKs for top commit:
  maflcko:
    utACK 7c8abf3c2001152423da06d25f9f4906611685ea
  kevkevinpal:
    utACK [7c8abf3](7c8abf3c20)
  hebasto:
    re-ACK 7c8abf3c2001152423da06d25f9f4906611685ea, I agree with changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/30122#pullrequestreview-2061694682).

Tree-SHA512: 279df65bea8f7aa02af0a2efed62dca9bf9b29cb748eb369c602d223e08a8a907dea7b1bffbd3dab91b1656c1d91b18a9a0534bc3f153bd751414b0e6230b3a4
2024-05-29 10:32:02 +01:00
merge-script
46d3477b5b
Merge bitcoin/bitcoin#30172: fuzz: Handle missing BDBRO errors
9ddf39dd87a3729ceedaa05a207621a02c532536 fuzz: Handle missing BDBRO errors (Ava Chow)

Pull request description:

  Adds error messages that were not being handled. Also removes error messages that no longer exist.

  Fixes #30166

ACKs for top commit:
  dergoegge:
    reACK 9ddf39dd87a3729ceedaa05a207621a02c532536
  TheCharlatan:
    ACK 9ddf39dd87a3729ceedaa05a207621a02c532536

Tree-SHA512: 2597536a1e5d030653dfcb02fd892f7492f5a091def787f6cbd421b8bca9544847684a498e9458ea99ae7de5a8a6d91532ff904d1e39222d324939d31d2eb3f0
2024-05-29 10:27:12 +01:00
Ava Chow
9ddf39dd87 fuzz: Handle missing BDBRO errors
Adds error messages that were not being handled. Also removes error
messages that no longer exist.
2024-05-29 05:01:21 -04:00
merge-script
be100cf4c7
Merge bitcoin/bitcoin#21778: build: LLD based macOS toolchain
e8c25e8a35e333e90514945c592557615641553f guix: drop binutils from macOS env (fanquake)
555fddf646265f7e57a416dc64b171f2c9460e20 guix: use GUIX_LD_WRAPPER_DISABLE_RPATH for all HOSTS (fanquake)
9ec238d0f3c7b97aec83dbeb4c7a6950c8f5125e guix: remove ZERO_AR_DATE export (fanquake)
f836f7e9b3e091eb27cdefb624e2a6f2a921fa55 depends: remove cctools & libtapi (fanquake)
4a0536c5d96688729f8c885060c83cb12d72a8c5 build: switch to using lld for macOS builds (fanquake)
c6a6b2d6fd4e3a01b095dc98645f819ebabf1931 build: add lld into macOS build environment(s) (fanquake)
437e908ebd1bf9473ef924de5aec57bdc26de6dd depends: swap cctools-x for llvm-x (fanquake)
bab287d1bab2c02b5fab3285f2678c15316d31c2 depends: don't use -no_warning_for_no_symbols in macOS qt build (fanquake)

Pull request description:

  This switches us to using a [LLD](https://lld.llvm.org/) based toolchain for macOS builds.

  ### Benefits
  * Less complicated macOS toolchain.
  * No longer beholden to Apple releasing it's [source](https://opensource.apple.com/source/) for [cctools](https://opensource.apple.com/source/cctools/), [ld64](https://opensource.apple.com/source/ld64/) & [libtapi](https://opensource.apple.com/source/tapi/).
  * No more reliance on third parties to modify those sources for us. i.e [apple-libtapi](https://github.com/tpoechtrager/apple-libtapi), [cctools-port](https://github.com/tpoechtrager/cctools-port) (cctools + ld64).

ACKs for top commit:
  theuni:
    Tentative ACK e8c25e8a35e333e90514945c592557615641553f.

Tree-SHA512: ec73304e8a2cd4c71041f7863d7d2e4e0408787299fb4fa3745076853156e8f64e4742e16f30d65e3a27f1e9c0d19cdf802248366b72a4fcb4ea821f92bb7a00
2024-05-29 09:42:05 +01:00
merge-script
417b6cecee
Merge bitcoin/bitcoin#30156: fuzz: More accurate coverage reports
949abebea0059edd929b653b4b475a5880fc0a3e [fuzz] Avoid collecting initialization coverage (dergoegge)

Pull request description:

  Our coverage reports include coverage of initialization code, which can be misleading when trying to evaluate the coverage a fuzz harness achieves through fuzzing alone.

  This PR proposes to make fuzz coverage reports more accurate by resetting coverage counters after initialization code has been run. This makes it easier to evaluate which code was actually reached through fuzzing (e.g. to spot fuzz blockers).

ACKs for top commit:
  maflcko:
    utACK 949abebea0059edd929b653b4b475a5880fc0a3e
  brunoerg:
    nice, utACK 949abebea0059edd929b653b4b475a5880fc0a3e

Tree-SHA512: c8579bda4f3d71d199b9331fbe6316fce375a906743d0bc216bb94958dc03fdc9a951ea50cfeb487494a75668ae3c16471a82f7e5fdd912d781dc29d063e2c5b
2024-05-29 09:34:48 +01:00
merge-script
58956028e4
Merge bitcoin/bitcoin#30170: refactor: Use type-safe time in txorphanage
fa6d4891c7815025ab1cd58d0906466af31bb648 refactor: Use type-safe time in txorphanage (MarcoFalke)

Pull request description:

  This allows to remove manual conversions like multiplication by `60`, and uses a type-safe type instead of a raw `int64_t`.

ACKs for top commit:
  epiccurious:
    utACK fa6d4891c7815025ab1cd58d0906466af31bb648.
  dergoegge:
    Code review ACK fa6d4891c7815025ab1cd58d0906466af31bb648
  brunoerg:
    utACK fa6d4891c7815025ab1cd58d0906466af31bb648

Tree-SHA512: c187d0e579b1131afcef8c901f5662c18ab867fa2a99fbb13b67bb1e10b2047128194bfef8329cde0d51e1c35d6227ae292b823968f37ea9422975e46e01846a
2024-05-29 09:22:24 +01:00
Murch
bd34dd85e7 Use exact_target shorthand in coinselector_tests 2024-05-28 10:14:17 -04:00
Murch
7aa7e30441 Fold GetSelectionWaste() into ComputeAndSetWaste()
Both `GetSelectionWaste()` and `ComputeAndSetWaste()` now are part of
`SelectionResult`. Instead of `ComputeAndSetWaste()` being a wrapper for
`GetSelectionWaste()`, we combine them to a new function
`RecalculateWaste()`.

As I was combining the logic of the two functions, I noticed that
`GetSelectionWaste()` was making the odd assumption that the
`change_cost` being set to zero means that no change is created.
However, if we build transactions at a feerate of zero with the
`discard_feerate` also set to zero, we'd organically have a
`change_cost` of zero, even when we create change on a transaction.

This commit cleans up this duplicate meaning of `change_cost` and relies
on `GetChange()` to figure out whether there is change on basis of the
`min_viable_change` and whatever is left after deducting fees.

Since this broke a bunch of tests that relied on the double-meaning of
`change_cost` a bunch of tests had to be fixed.
2024-05-24 14:53:54 -04:00
Ava Chow
327f08bb0c
Merge bitcoin/bitcoin#30169: fuzz: Fix wallet_bdb_parser stdlib error matching
fac72985292b516919a216d9a78cf84418cd7f96 fuzz: Fix wallet_bdb_parser stdlib error matching (MarcoFalke)

Pull request description:

  The stdlib error string is an implementation detail and can not be relied upon.

  Ref: `libc++abi: terminating due to uncaught exception of type std::runtime_error: AutoFile::read: end of file: unspecified iostream_category error`

ACKs for top commit:
  achow101:
    ACK fac72985292b516919a216d9a78cf84418cd7f96

Tree-SHA512: 588acc71a05d97855d6bb65380411e8486692536434eadee7697de09f80b128ff2f90a31fd0e8384d084b554d2f3978efd076082e070e721cf05b07c94cc83b1
2024-05-24 13:15:47 -04:00
Fabian Jahr
efc1b5be8a
test: Add coverage for txid coins count check when loading snapshot 2024-05-24 18:44:05 +02:00
Fabian Jahr
6b6084850b
assumeutxo: Add network magic ctor param to SnapshotMetadata
This prevents SnapshotMetadata from using any globals implicitly.
2024-05-24 18:44:02 +02:00
MarcoFalke
fac7298529
fuzz: Fix wallet_bdb_parser stdlib error matching 2024-05-24 14:21:30 +02:00
willcl-ark
4b7d984269
lint: add markdown hyperlink checker
This adds a markdown hyperlink check task to the lint test_runner. It
relies on having the [`mlc`](https://crates.io/crates/mlc) binary found
on $PATH, but will fail with `success` if the binary is not found.

`mlc` is also added to the ci/04_install.sh script run by the
containerfile.

Note that broken markdown hyperlinks will be detected in untracked
markdown files found in a dirty working directory (including e.g.
.venv).
2024-05-24 12:11:50 +01:00
Fabian Jahr
1f1f998455
assumeutxo: Deserialize trailing byte instead of Txid 2024-05-24 12:42:09 +02:00
Fabian Jahr
359967e310
doc: Add release notes for #29612 2024-05-24 12:42:06 +02:00
glozow
4c387cb64f
Merge bitcoin/bitcoin#30072: refactor prep for package rbf
2fd34ba504957331f5a08614b6e1f8317260f04d Add sanity checks for various ATMPArgs booleans (Greg Sanders)
20d8936d8bb6137a5a3722d34e0d7ae756031009 [refactor] make some members MemPoolAccept-wide (glozow)
cbbfe719b223b9e05398227cef68c99eb97670bd cpfp carveout is excluded in packages (glozow)
69f7ab05bafec1cf06fd7a58351f78e32bbfa2cf Add m_allow_sibling_eviction as separate ATMPArgs flag (Greg Sanders)
57ee3029ddadaee5cb48224e0a87f704b7971bd8 Add description for m_test_accept (Greg Sanders)

Pull request description:

  First few commits of https://github.com/bitcoin/bitcoin/pull/28984 to set the stage for the package RBF logic.

  These refactors are preparation for evaluating an RBF in a multi-proposed-transaction context instead of only a single proposed transaction. Also, carveouts and sibling evictions only should work in single RBF cases so add logic to preclude multi-tx cases in the future.

  No behavior changes aside from bailing earlier from failed carve-outs.

ACKs for top commit:
  glozow:
    reACK 2fd34ba504957331f5a08614b6e1f8317260f04d via range-diff
  sr-gi:
    utACK [2fd34ba](2fd34ba504)
  theStack:
    re-ACK 2fd34ba504957331f5a08614b6e1f8317260f04d

Tree-SHA512: 5071c5b8d9b8d2c9faa278c8c4df31de288cb407a68e4d55544c588caff6c86160cce7825453549c6ed69e29d9ccb5ee2d4a518b18f563bfb12f2ced073fe42a
2024-05-24 10:24:50 +01:00