39925 Commits

Author SHA1 Message Date
fanquake
cf937b2068
fuzz: also set MSAN_SYMBOLIZER_PATH 2024-01-26 13:56:09 +00:00
fanquake
e3b68b3b83
Merge bitcoin/bitcoin#28875: build: Pass sanitize flags to instrument libsecp256k1 code
cbea49c0d32badb975fbf22d44f8e25cc7972af7 build: Pass sanitize flags to instrument `libsecp256k1` code (Hennadii Stepanov)

Pull request description:

  This PR is a revived https://github.com/bitcoin/bitcoin/pull/27991 with an addressed [comment](https://github.com/bitcoin/bitcoin/pull/27991#discussion_r1252148488).

  Fixes https://github.com/bitcoin/bitcoin/issues/27990.

  Might be tested as follows:
  ```
  $ ./autogen.sh && ./configure --enable-fuzz --with-sanitizers=fuzzer CC=clang-13 CXX=clang++-13
  $ make clean > /dev/null && make
  $ objdump --disassemble=secp256k1_xonly_pubkey_serialize src/test/fuzz/fuzz | grep __sanitizer_cov
   1953bd0:e8 bb c6 05 ff       call   9b0290 <__sanitizer_cov_trace_const_cmp8>
   1953d32:e8 69 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
   1953d58:e8 43 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
   1953d82:e8 19 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
  ```

ACKs for top commit:
  fanquake:
    ACK cbea49c0d32badb975fbf22d44f8e25cc7972af7
  dergoegge:
    reACK cbea49c0d32badb975fbf22d44f8e25cc7972af7

Tree-SHA512: 801994e75b711d20eaf0d675f378da07d693f4a7de026efd93860f5f1deabed855a83eca3561725263e4fe605fcc5f91eb73c021ec91c831864e6deb575e3885
2024-01-26 11:31:34 +00:00
Ava Chow
717103bcce
Merge bitcoin/bitcoin#29315: refactor: Compile unreachable walletdb code
fa3373d3adbace7e4665cf391363319a55a09a96 refactor: Compile unreachable code (MarcoFalke)

Pull request description:

  When unreachable code isn't compiled, compile failures are not detected.

  Fix this by leaving it unreachable, but compiling it.

  Fixes https://github.com/bitcoin/bitcoin/pull/28999#discussion_r1465010916

ACKs for top commit:
  achow101:
    ACK fa3373d3adbace7e4665cf391363319a55a09a96
  ryanofsky:
    Code review ACK fa3373d3adbace7e4665cf391363319a55a09a96. This looks good, and should prevent code in the else blocks from accidentally breaking.

Tree-SHA512: 3a3764915dfc935bf5d7a48f1ca151dcbac340c1cbdce8236b24ae9b4f04d6ee9771ed058ca60bcbca6e19d13671de3517f828a8f7ab6444c7cc4e3538d1ba4e
2024-01-25 17:16:09 -05:00
Ava Chow
36720994a4
Merge bitcoin/bitcoin#20827: During IBD, prune as much as possible until we get close to where we will eventually keep blocks
d298ff8b62b2624ed390c8a2f905c888ffc956ff During IBD, prune as much as possible until we get close to where we will eventually keep blocks (Luke Dashjr)

Pull request description:

  This should reduce pruning flushes even more, speeding up IBD with pruning on systems that have a sufficient dbcache.

  Assumes 1 MB per block between tip and best header chain. Simply adds this to the buffer pruning is trying to leave available, which results in pruning almost everything up until we get close to where we need to be keeping blocks.

ACKs for top commit:
  andrewtoth:
    ACK d298ff8b62b2624ed390c8a2f905c888ffc956ff
  fjahr:
    utACK d298ff8b62b2624ed390c8a2f905c888ffc956ff
  achow101:
    ACK d298ff8b62b2624ed390c8a2f905c888ffc956ff

Tree-SHA512: 2a482376bfb177e2ba7c2f0bb0b58b02efdb38b34755a18d1fc3e869df5959c85b6f1009e1386fa8b89c4f90d520383e36bd3e21dec221042315134efb1a455b
2024-01-25 15:20:17 -05:00
fanquake
ac923e70e7
Merge bitcoin/bitcoin#29287: depends: Do not override CFLAGS when building SQLite with DEBUG=1
5fb8f0f80fc41cc636da56864195244d8fd9116e depends: Do not override CFLAGS when building SQLite with DEBUG=1 (Hennadii Stepanov)
2b0dd88f1ce9084324dc54db578fade9c926fd71 depends: Ensure definitions are passed when building SQLite with DEBUG=1 (Hennadii Stepanov)

Pull request description:

  The `--enable-debug` configure option for the SQLite package does two things:
  ```autoconf
  #-----------------------------------------------------------------------
  #   --enable-debug
  #
  AC_ARG_ENABLE(debug, [AS_HELP_STRING(
    [--enable-debug], [build with debugging features enabled [default=no]])],
    [], [])
  AC_MSG_CHECKING([Build type])
  if test x"$enable_debug" = "xyes"; then
    BUILD_CFLAGS="$BUILD_CFLAGS -DSQLITE_DEBUG -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE"
    CFLAGS="-g -O0"
    AC_MSG_RESULT([debug])
  else
    AC_MSG_RESULT([release])
  fi
  #-----------------------------------------------------------------------
  ```

  It adds three preprocessor definitions and overrides `CFLAGS` with `"-g -O0"`. The latter breaks the user's ability to provide sanitizer and LTO flags.

  This PR might be especially useful for OSS-Fuzz where `DEBUG=1` has been used since https://github.com/google/oss-fuzz/pull/10503.

  Also it makes a workaround for building SQLite for 32-bit unneeded. For details, please refer to https://github.com/hebasto/oss-fuzz/tree/240120-sqlite.

  Changes in https://github.com/bitcoin/bitcoin/pull/29282 might not be strictly required now. However, I consider them an improvement.

ACKs for top commit:
  fanquake:
    ACK 5fb8f0f80fc41cc636da56864195244d8fd9116e - downstream is also green, so i'll fixup the PR there.

Tree-SHA512: 8593d8a0237ebb270d5da763fb65ed642ab8ed0d44e57704a34154621f49e3d5c58b462cc0070251fa1ba556c58a3c7d3620530d6839dc6dc9e0887010330eca
2024-01-25 15:46:56 +00:00
MarcoFalke
fa3373d3ad
refactor: Compile unreachable code
When unreachable code isn't compiled, compile failures are not detected.

Fix this by leaving it unreachable, but compiling it.

Fixes https://github.com/bitcoin/bitcoin/pull/28999#discussion_r1465010916

Can be reviewed with --ignore-all-space
2024-01-25 16:25:55 +01:00
fanquake
7699a1aab8
Merge bitcoin/bitcoin#29313: ci: Update cache action
ec25e745420fce5fd3e14b0c39e6f475d918d5ad ci: Update cache action (Hennadii Stepanov)

Pull request description:

  This PR fixes deprecation [warnings](https://github.com/bitcoin/bitcoin/actions/runs/7652979339) for Node.js 16 actions in the GHA CI:
  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/ea7b0708-8b2f-446f-a16d-ecc2c8a1da45)

  See:
  - https://github.com/marketplace/actions/cache
  - https://github.com/actions/cache/releases/tag/v4.0.0

Top commit has no ACKs.

Tree-SHA512: 48503abab5d188d6fac2a1ead62512c217a831f611c4dce0e05666d72fac4db26f947cbe9a42fda0307cbdcb9aa0bd4b4d7a15ac2c14c757f92ba2916da0020b
2024-01-25 14:23:41 +00:00
Hennadii Stepanov
5fb8f0f80f
depends: Do not override CFLAGS when building SQLite with DEBUG=1
The `--enable-debug` configure option for the SQLite package does two
things. It adds three preprocessor definitions and overrides CFLAGS with
"-g -O0". The latter breaks the user's ability to provide sanitizer and
LTO flags.
2024-01-25 12:25:27 +00:00
Hennadii Stepanov
2b0dd88f1c
depends: Ensure definitions are passed when building SQLite with DEBUG=1
The SQLite build system overrides the `CFLAGS` when is configured with
the `--enable-debug` option.
2024-01-25 12:23:49 +00:00
Hennadii Stepanov
ec25e74542
ci: Update cache action
This change fixes deprecation warnings for Node.js 16 actions in the GHA
CI.

See:
- https://github.com/marketplace/actions/cache
- https://github.com/actions/cache/releases/tag/v4.0.0
2024-01-25 11:55:57 +00:00
fanquake
4ad83ef09b
Merge bitcoin/bitcoin#29205: build: always set -g -O2 in CORE_CXXFLAGS
00c1e2aa4496b5f038ae5199dbd16d8313766533 build: fix optimisation flags used for --coverage (fanquake)
1dc2c9b385f8345c588449848149b8e470653afc ci: cleanup C*FLAG usage in Valgrind jobs (fanquake)
6cc2a38c1388b696e9c28a08c6bd9c93da4fa6b8 build: add sanitizer flags to configure output (fanquake)
08cd5aca18f0774258c7c459773b9e8b386d48ef build: always set -g -O2 in CORE_CXXFLAGS (fanquake)

Pull request description:

  Rather than trying to sporadically rely on / override Autoconf default behaviour. Just always override (if unset), and always set the flags we want (which are the same as the Autoconf defaults).

  Removes the need for duplicate code to clear (if not overridden) `CXXFLAGS`.

  Fixes cases of "missing" `-O2`. i.e this PR when running a Valgrind CI job with changes here:
  ```bash
  CXXFLAGS        =  -g -O2  -fdebug-prefix-map=$(abs_top_srcdir)=.  -Wstack-protector -fstack-protector-all -mbranch-protection=bti   -Werror  -fsanitize=fuzzer  -gdwarf-4
  ```

  Fixes configure output to reflect actual compilation flag ordering, so it's useful.

  Note that if we do still end up with a duplicate "-g -O2" when compiling, that has no effect, and I don't really thinks it's something worth trying to optimize.

ACKs for top commit:
  TheCharlatan:
    lgtm ACK 00c1e2aa4496b5f038ae5199dbd16d8313766533
  hebasto:
    ACK 00c1e2aa4496b5f038ae5199dbd16d8313766533, I have reviewed the code and it looks OK. Also tested `ci/test/00_setup_env_native_valgrind.sh`.
  theuni:
    ACK 00c1e2aa4496b5f038ae5199dbd16d8313766533

Tree-SHA512: cf6c7acf813ba10b198561e83eb72e9b2532a39cb1767c452d031e82921dcd42a47b129735b24c4e36131fd0c8fe7457f7cae870c1e011cdfdd430bdc4d4912b
2024-01-25 10:12:56 +00:00
Ava Chow
207220ce8b
Merge bitcoin/bitcoin#29302: wallet: clarify replaced_by_txid and replaces_txid in help output
ff54314d4abed3bf9a78daf785a01c63af15c69d wallet: clarify replaced_by_txid and replaces_txid in help output (marco)

Pull request description:

  Resolves issue #27781

ACKs for top commit:
  achow101:
    ACK ff54314d4abed3bf9a78daf785a01c63af15c69d
  ryanofsky:
    Code review ACK ff54314d4abed3bf9a78daf785a01c63af15c69d. Seems like a helpful clarification

Tree-SHA512: b13a0e24505dfaee083467ac6f357b96460b5d1841dc29c4df4a503c290d379cef3d50fcc76f33bbc95741f484dd9d2461b0c2e8bdebf57a8a72edfbeece2a79
2024-01-24 13:04:27 -05:00
fanquake
ea4ddd8652
Merge bitcoin/bitcoin#29304: fuzz: Exit and log stderr for parse_test_list errors
9d09c873a50d344e2a9cb35fe246a52688b9fa34 fuzz: Exit and log stderr for parse_test_list errors (dergoegge)

Pull request description:

  We should log all errors that occur when attempting to print the harness list in the fuzz test runner.

ACKs for top commit:
  maflcko:
    lgtm ACK 9d09c873a50d344e2a9cb35fe246a52688b9fa34

Tree-SHA512: 50471b732c8cbe287dacba14487e7c8a5826f146432d93aa3bb55d063a8ba158d01641d6cb1360241dd4cd54ef5e045b0412f9cc34d06c181134921d1f1ceced
2024-01-24 15:14:16 +00:00
dergoegge
9d09c873a5 fuzz: Exit and log stderr for parse_test_list errors 2024-01-24 11:42:30 +00:00
marco
ff54314d4a wallet: clarify replaced_by_txid and replaces_txid in help output 2024-01-23 17:34:16 -07:00
Ava Chow
e69796c79c
Merge bitcoin/bitcoin#28560: wallet, rpc: FundTransaction refactor
18ad1b9142e91cef2f5c6a693eeb2d0fbb8c517d refactor: pass CRecipient to FundTransaction (josibake)
5ad19668dbcc47486d1c18f711cea3d8a9d2e7e2 refactor: simplify `CreateRecipients` (josibake)
47353a608dc6e20e5fd2ca53850d6f9aa3240d4a refactor: remove out param from `ParseRecipients` (josibake)
f7384b921c3460c7a3cc7827a68b2c613bd98f8e refactor: move parsing to new function (josibake)
6f569ac903e5ddaac275996a5d0c31b2220b7b81 refactor: move normalization to new function (josibake)
435fe5cd96599c518e26efe444c9d94d1277996b test: add tests for fundrawtx and sendmany rpcs (josibake)

Pull request description:

  ## Motivation

  The primary motivation for this PR is to enable `FundTransaction` to take a vector of `CRecipient` objects to allow passing BIP352 silent payment addresses to RPCs that use `FundTransaction` (e.g. `send`, `walletcreatefundedpsbt`). To do that, SFFO logic needs to be moved out of `FundTransaction` so the `CRecipient` objects with the correct SFFO information can be created and then passed to `FundTransaction`.

  As a secondary motivation, this PR moves the SFFO stuff closer to the caller, making the code cleaner and easier to understand. This is done by having a single function which parses RPC inputs for SFFO and consistently using the `set<int>` method for communicating SFFO.

  I'm also not convinced we need to pass a full `CMutableTx` object to `FundTransaction`, but I'm leaving that for a follow-up PR/discussion, as its not a blocker for silent payments.

ACKs for top commit:
  S3RK:
    reACK 18ad1b9142e91cef2f5c6a693eeb2d0fbb8c517d
  josibake:
    > According to my `range-diff` nothing changed. reACK [18ad1b9](18ad1b9142)
  achow101:
    ACK 18ad1b9142e91cef2f5c6a693eeb2d0fbb8c517d

Tree-SHA512: d61f017cf7d98489ef216475b68693fd77e7b53a26a6477dcd73e7e5ceff5036b2d21476e377839e710bb73644759d42c4f9f4b14ed96b3e56ed87b07aa6d1a7
2024-01-23 16:40:58 -05:00
Ava Chow
2f218c664b
Merge bitcoin/bitcoin#28921: multiprocess: Add basic type conversion hooks
6acec6b9ff02b91de132bb1575d75908a8a2d27b multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fce72e0c79849109ee5d7afe707991b3512 multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee239211a5287fbc361c0eb158b105ae8c8db test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)

Pull request description:

  Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.

  The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  achow101:
    ACK 6acec6b9ff02b91de132bb1575d75908a8a2d27b
  dergoegge:
    reACK 6acec6b9ff02b91de132bb1575d75908a8a2d27b

Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
2024-01-23 16:22:29 -05:00
Ava Chow
874c8bdb9e
Merge bitcoin/bitcoin#29144: init: handle empty settings file gracefully
e9014042a6bed8c16cc9a31fc35cb709d4b3c766 settings: add auto-generated warning msg for editing the file manually (furszy)
966f5de99a9f5da05c91378ad1e8ea8ed37ac3b3 init: improve corrupted/empty settings file error msg (furszy)

Pull request description:

  Small and simple issue reported [here](https://community.umbrel.com/t/bitcoin-docker-container-keeps-restarting/2144).

  Improving a confusing situation reported by users who did not understand why a
  settings parsing error occurred when the file was empty and did not know how to solve it.

  Empty setting file could be due (1) corruption or (2) an user manually cleaning up the file content.
  In both scenarios, the 'Unable to parse settings file' error does not help the user move forward.

ACKs for top commit:
  achow101:
    ACK e9014042a6bed8c16cc9a31fc35cb709d4b3c766
  hebasto:
    re-ACK e9014042a6bed8c16cc9a31fc35cb709d4b3c766.
  ryanofsky:
    Code review ACK e9014042a6bed8c16cc9a31fc35cb709d4b3c766. Just whitespace formatting changes and shortening a test string literal since last review
  shaavan:
    Code review ACK e9014042a6bed8c16cc9a31fc35cb709d4b3c766

Tree-SHA512: 2910654c6b9e9112de391eedb8e46980280f822fa3059724dd278db7436804dd27fae628d2003f2c6ac1599b07ac5c589af016be693486e949f558515e662bec
2024-01-23 15:14:03 -05:00
Ava Chow
6f732ffc3c
Merge bitcoin/bitcoin#28774: wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it
32a9f13cb805ecf6aebb5cf4e5d92b8a47c4548b wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Vasil Dimov)

Pull request description:

  `CWallet::GetEncryptionKey()` would return a reference to the internal
  `CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

  Returning a copy would be a shorter solution, but could have security
  implications of the master key remaining somewhere in the memory even
  after `CWallet::Lock()` (the current calls to
  `CWallet::GetEncryptionKey()` are safe, but that is not future proof).

  So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
  change the `GetEncryptionKey()` method to provide the encryption
  key to a given callback:
  `m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

  This silences the following (clang 18):

  ```
  wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
   3520 |     return vMasterKey;
        |            ^
  ```

  ---
  _Previously this PR modified both ArgsManager and wallet code. But the ArgsManager commit 856c88776f was merged in https://github.com/bitcoin/bitcoin/pull/29040 so now this only affects wallet code. The previous PR description was:_

  Avoid this unsafe pattern from `ArgsManager` and `CWallet`:

  ```cpp
  class A
  {
      Mutex mutex;
      Foo member GUARDED_BY(mutex);
      const Foo& Get()
      {
          LOCK(mutex);
          return member;
      } // callers of `Get()` will have access to `member` without owning the mutex.
  ```

ACKs for top commit:
  achow101:
    ACK 32a9f13cb805ecf6aebb5cf4e5d92b8a47c4548b
  ryanofsky:
    Code review ACK 32a9f13cb805ecf6aebb5cf4e5d92b8a47c4548b. This seems like a potentially real race condition, and the fix here is pretty simple.
  furszy:
    ACK 32a9f13c

Tree-SHA512: 133da84691642afc1a73cf14ad004a7266cb4be1a6a3ec634d131dca5dbcdef52522c1d5eb04f5b6c4e06e1fc3e6ac57315f8fe1e207b464ca025c2b4edefdc1
2024-01-23 15:05:23 -05:00
Ava Chow
7cb7759b25
Merge bitcoin/bitcoin#29272: wallet: fix coin selection tracing to return -1 when no change pos
d55fdb1a495190e213b1b5127f5d91e4a409765e Move TRACEx parameters to seperate lines (Richard Myers)
2d58629ee63eebc760e2a9226afcd0c46d3ec2bd wallet: fix coin selection tracing to return -1 when no change pos (Richard Myers)

Pull request description:

  This is a bugfix for from when [optional was introduced](758501b713)  for `change_pos` in the wallet. When optional `change_pos` is unset, we should return -1 and not 0.

  I added two new checks to the `test/functional/interface_usdt_coinselection.py` which adds coverage for the situations when `normal_create_tx_internal` and `aps_create_tx_internal` events occur with no change.

  You can reproduce this bug using the coin-selection-simulation scripts as described in [issue #16](https://github.com/achow101/coin-selection-simulation/issues/16). You can also run the `interface_usdt_coinselection.py` test  without the changes to `wallet/spend.cpp`.

ACKs for top commit:
  0xB10C:
    ACK d55fdb1a495190e213b1b5127f5d91e4a409765e
  achow101:
    ACK d55fdb1a495190e213b1b5127f5d91e4a409765e
  murchandamus:
    ACK d55fdb1a495190e213b1b5127f5d91e4a409765e

Tree-SHA512: 6efac3b756bdf51debbcb759dc3c4b7a4304626bc047b70025cec02f3a04937ace7712e9558ac71e560fd136005a98c518ac5bb4b90c3282d776beccd0de9749
2024-01-23 14:33:43 -05:00
fanquake
f1ab078ed7
Merge bitcoin/bitcoin#29276: depends: Update libmultiprocess library to fix C++20 macos build error
b8105b3ed7c97cd6545dba99d0e13c33f183e450 depends: Update libmultiprocess library to fix C++20 macos build error (Ryan Ofsky)

Pull request description:

  Fixes #29248

  The std::result_of type was removed in c++20, but was being referenced in some old, unused code in the library. The issue was fixed in:

  - https://github.com/chaincodelabs/libmultiprocess/pull/91

  This update also includes other recent libmultiprocess changes to improve C++20 support and fix build issues:

  - https://github.com/chaincodelabs/libmultiprocess/pull/89
  - https://github.com/chaincodelabs/libmultiprocess/pull/90
  - https://github.com/chaincodelabs/libmultiprocess/pull/93

ACKs for top commit:
  fanquake:
    ACK b8105b3ed7c97cd6545dba99d0e13c33f183e450.

Tree-SHA512: 2ca64b5fc27be752baba38df4b4faf62152e18c70ead6e0e063f1cb0c25dd5d924dec7ebfd7f8bbd651ae50eb35e8d8b591a9847c36f22558b5f5effccf56536
2024-01-23 17:06:57 +00:00
fanquake
8c9dceb962
Merge bitcoin/bitcoin#29291: Add test for negative transaction version w/ CSV to tx_valid.json
97181decf5726aab6c5cd01b3e1964072f2531ff Add test for negative transaction version w/ CSV to tx_valid.json (Chris Stewart)

Pull request description:

  This PR adds a static test vector corresponding to the bug found in various implementations of the bitcoin protocol discovered by dergoegge

  For more information see:

  https://delvingbitcoin.org/t/disclosure-btcd-consensus-bugs-due-to-usage-of-signed-transaction-version/455

ACKs for top commit:
  darosior:
    ACK 97181decf5726aab6c5cd01b3e1964072f2531ff
  dergoegge:
    ACK 97181decf5726aab6c5cd01b3e1964072f2531ff

Tree-SHA512: 92bbcd3cd10a569757b4de91e1b2bcfebc2b75ddb0160be36d8e512a6fa4623cced1aba93bd1cc044962cd2b10e1d184ef109ccdfe3cfcf85cf4b9585d80d115
2024-01-23 16:53:37 +00:00
Ryan Ofsky
b8105b3ed7 depends: Update libmultiprocess library to fix C++20 macos build error
Fixes #29248

The std::result_of type was removed in c++20, but was being referenced in some
old, unused code in the library. The issue was fixed in:

https://github.com/chaincodelabs/libmultiprocess/pull/91 util: Drop Bind, BindTuple, ComposeFn, GetFn, and ThrowFn helpers

This update also includes other recent libmultiprocess changes to improve C++20
support and fix build issues:

https://github.com/chaincodelabs/libmultiprocess/pull/89 pkgconfig: Drop -std=c++17 compile flag
https://github.com/chaincodelabs/libmultiprocess/pull/90 pkgconfig: Use @CMAKE_INSTALL_LIBDIR@ variable
https://github.com/chaincodelabs/libmultiprocess/pull/93 Fix support for vector<bool> serialization with libc++
2024-01-22 11:47:13 -05:00
furszy
e9014042a6
settings: add auto-generated warning msg for editing the file manually
Hopefully, refraining users from modifying the file unless they are
certain about the potential consequences.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
furszy
966f5de99a
init: improve corrupted/empty settings file error msg
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
glozow
651fb034d8
Merge bitcoin/bitcoin#29260: refactor: remove CTxMemPool::queryHashes()
282b12ddb0aeb1f0991dd9f45c6b5c5c079652ec refactor: remove CTxMemPool::queryHashes() (stickies-v)

Pull request description:

  `CTxMemPool::queryHashes()` is only used in `MempoolToJSON()`, where it can just as easily be replaced with the more general `CTxMemPool::entryAll()`. No behaviour change, just cleans up the code.

ACKs for top commit:
  dergoegge:
    Code review ACK 282b12ddb0aeb1f0991dd9f45c6b5c5c079652ec
  TheCharlatan:
    ACK 282b12ddb0aeb1f0991dd9f45c6b5c5c079652ec
  glozow:
    ACK 282b12ddb0aeb1f0991dd9f45c6b5c5c079652ec. Looks like there's no conflicts.

Tree-SHA512: 16160dec8e1f2457fa0f62dc96d2d2efd92c4bab810ecdb0e08918b8e85a667702c8e41421eeb4ea6abe92a5956a2a39a7a6368514973b78be0d22de2ad299b2
2024-01-22 10:03:57 +00:00
Richard Myers
d55fdb1a49
Move TRACEx parameters to seperate lines 2024-01-20 14:58:17 +01:00
Richard Myers
2d58629ee6
wallet: fix coin selection tracing to return -1 when no change pos 2024-01-20 14:56:41 +01:00
josibake
18ad1b9142
refactor: pass CRecipient to FundTransaction
Instead turning tx.vout into a vector of `CRecipient`, make `FundTransaction`
take a `CRecipient` vector directly. This allows us to remove SFFO logic from
the wrapper RPC `FundTransaction` since the `CRecipient` objects have already
been created with the correct SFFO values. This also allows us to remove
SFFO from both `FundTransaction` function signatures.

This sets us up in a future PR to be able to use these RPCs with BIP352
static payment codes.
2024-01-19 15:04:56 +01:00
josibake
5ad19668db
refactor: simplify CreateRecipients
Move validation logic out of `CreateRecipients` and instead take the
already validated outputs from `ParseOutputs` as an input.

Move SFFO parsing out of `CreateRecipients` into a new function,
`InterpretSubtractFeeFromOutputsInstructions`. This takes the SFFO instructions
from `sendmany` and `sendtoaddress` and turns them into a set of integers.
In a later commit, we will also move the SFFO parsing logic from
`FundTransaction` into this function.

Worth noting: a user can pass duplicate addresses and addresses that dont exist
in the transaction outputs as SFFO args to `sendmany` and `sendtoaddress`
without triggering a warning. This behavior is preserved in to keep this commit
strictly a refactor.
2024-01-19 15:04:56 +01:00
josibake
47353a608d
refactor: remove out param from ParseRecipients
Have `ParseRecipients` return a vector of `CRecipients` and rename to `CreateRecipients`.
2024-01-19 15:04:56 +01:00
josibake
f7384b921c
refactor: move parsing to new function
Move the parsing and validation out of `AddOutputs` into its own function,
`ParseOutputs`. This allows us to re-use this logic in `ParseRecipients` in a
later commit, where the code is currently duplicated.

The new `ParseOutputs` function returns a CTxDestination,CAmount tuples.
This allows the caller to then translate the validated outputs into
either CRecipients or CTxOuts.
2024-01-19 15:04:56 +01:00
josibake
6f569ac903
refactor: move normalization to new function
Move the univalue formatting logic out of AddOutputs and into its own function,
`NormalizeOutputs`. This allows us to re-use this logic in later commits.
2024-01-19 15:04:56 +01:00
josibake
435fe5cd96
test: add tests for fundrawtx and sendmany rpcs
If the serialized transaction passed to `fundrawtransaction` contains
duplicates, they will be deserialized and added to the transaction. Add
a test to ensure this behavior is not changed during the refactor.

A user can pass any number of duplicated and unrelated addresses as an
SFFO argument to `sendmany` and the RPC will not throw an error (note,
all the rest of the RPCs which take SFFO as an argument will error if
the user passes duplicates or specifies outputs not present in the
transaction). Add a test to ensure this behavior is not changed during
the refactor.
2024-01-19 15:04:56 +01:00
fanquake
03752444cd
Merge bitcoin/bitcoin#29249: depends: add NM output to gen_id
6ec2813cd88d5f0b955d746e4711a8ad256ea47f depends: add NM output to gen_id (fanquake)

Pull request description:

  `NM` is part of the current toolset, and can be set by the user. Include it in `gen_id`.

ACKs for top commit:
  TheCharlatan:
    Re-ACK 6ec2813cd88d5f0b955d746e4711a8ad256ea47f

Tree-SHA512: 2ada61e03783f9eb441f285ef5da50557ad729cb52ce2d2c4b2c38103dab29920a26262d4545fd2ac7fbf1cedc4902cd2359833544fbc0debf829c12a63e9769
2024-01-19 13:17:16 +00:00
Hennadii Stepanov
cbea49c0d3
build: Pass sanitize flags to instrument libsecp256k1 code
Also a new UBSan suppression has been added.
2024-01-19 10:08:41 +00:00
stickies-v
282b12ddb0
refactor: remove CTxMemPool::queryHashes()
Its only usage can easily be replaced with CTxMemPool::entryAll()
2024-01-18 21:54:56 +00:00
Ava Chow
5f3a0574c4
Merge bitcoin/bitcoin#29262: rpc: Fix race in loadtxoutset
5555d8db3313f893609eb0cf549bb597361d4466 test: Use blocks_path where possible (MarcoFalke)
fa9108941fa1a0e83484114e2d8a99d264c2ad09 rpc: Fix race in loadtxoutset (MarcoFalke)

Pull request description:

  The tip may have advanced, also if it did not, there is no reason to
  have two variables point to the same block.

  Fixes https://github.com/bitcoin/bitcoin/pull/27596#discussion_r1344694600

ACKs for top commit:
  achow101:
    ACK 5555d8db3313f893609eb0cf549bb597361d4466
  pablomartin4btc:
    ACK 5555d8db3313f893609eb0cf549bb597361d4466
  BrandonOdiwuor:
    Code Review ACK 5555d8db3313f893609eb0cf549bb597361d4466

Tree-SHA512: 23a82924a915b61bb1adab8ad20ec8914139c8ee647817af34ca27ee310a2e45833d8b285503e0feebe63e4667193d6d98cfcbbc1509bf40712225e04dd19e8b
2024-01-18 13:17:35 -05:00
Ava Chow
ac3901ebd0
Merge bitcoin/bitcoin#29228: test: Remove all-lint.py script
fa2b95cf3f5148d27a8fd4fb3763ca1fc139bdd9 test: Remove all-lint.py script (MarcoFalke)
fadb06c361bbda78192d89fd06195f0aa5418c56 doc: move-only lint docs to one place (MarcoFalke)

Pull request description:

  Seems confusing to have a test runner that calls another runner (`all-lint.py`), which calls a subset of the lint tests.

  Fix that by just calling this subset of lint tests in the test runner directly, and remove the now unused `all-lint.py`.

  To run all lint checks locally, refer to the documentation: https://github.com/bitcoin/bitcoin/blob/master/test/lint/README.md#running-locally

ACKs for top commit:
  kevkevinpal:
    ACK [fa2b95c](fa2b95cf3f)
  achow101:
    ACK fa2b95cf3f5148d27a8fd4fb3763ca1fc139bdd9
  TheCharlatan:
    ACK fa2b95cf3f5148d27a8fd4fb3763ca1fc139bdd9
  pablomartin4btc:
    tACK fa2b95cf3f5148d27a8fd4fb3763ca1fc139bdd9
  brunoerg:
    utACK fa2b95cf3f5148d27a8fd4fb3763ca1fc139bdd9

Tree-SHA512: 43fac9acb4e9a6744d695dd49c7202e19ab4bf480f4cccff768647d0157a065f40e6ad70b9f6a65ba42048cc5fa9834365aa8e7aa0ed64c09e0cd4eb8dccb831
2024-01-18 13:02:15 -05:00
Vasil Dimov
32a9f13cb8
wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it
`CWallet::GetEncryptionKey()` would return a reference to the internal
`CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

Returning a copy would be a shorter solution, but could have security
implications of the master key remaining somewhere in the memory even
after `CWallet::Lock()` (the current calls to
`CWallet::GetEncryptionKey()` are safe, but that is not future proof).

So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
change the `GetEncryptionKey()` method to provide the encryption
key to a given callback:
`m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

This silences the following (clang 18):

```
wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
 3520 |     return vMasterKey;
      |            ^
```
2024-01-18 18:12:59 +01:00
fanquake
03c5b0064d
Merge bitcoin/bitcoin#29085: refactor: C++20: Use std::rotl
60446285436da62adef1c0a9b11c3336d82b4d89 crypto, hash: replace custom rotl32 with std::rotl (Fabian Jahr)

Pull request description:

  While exploring some C++20 changes and checking against our code I found this potential improvement:

  1. We can replace our custom implementation of `rotl32` in crypto/chacha20 with `std::rotl` from the [new `bit` header](https://en.cppreference.com/w/cpp/header/bit).

ACKs for top commit:
  fanquake:
    ACK 60446285436da62adef1c0a9b11c3336d82b4d89

Tree-SHA512: db55b366f20fca2ef62e5f10a838f8a709d531678c35c1dba20898754029c788a2fd47995208ed6d187cf814109a7ca397bc2c301504500aee79da04c95d6895
2024-01-18 09:40:44 +00:00
fanquake
3d52cedb49
Merge bitcoin/bitcoin#29251: contrib: Update clang-format-diff
52149b7a2c2b48ed4a4c0900c74cda4bb52a1ea5 contrib: Fix clang-format-diff.py lint errors (TheCharlatan)
008e81e025d64c33e1e71b5a2fe63dfdf6b31437 contrib: Latest clang-format-diff.py script (TheCharlatan)

Pull request description:

  This was taken from 900bb318b5/clang/tools/clang-format/clang-format-diff.py and is useful for systems where clang tools are shipped with a version suffix.

ACKs for top commit:
  maflcko:
    lgtm ACK 52149b7a2c2b48ed4a4c0900c74cda4bb52a1ea5  🌱

Tree-SHA512: cad720b283650e36c5b3ec597278112899ae6413a42c79b6296c58242000f32ae6ab7ed732a3a0f30f1f2586d7e3c0d1ef856a4821e28272bae6f428ed9497e9
2024-01-17 16:08:10 +00:00
fanquake
514268170b
Merge bitcoin/bitcoin#29133: refactor: Allow std::span construction from CKey
fa96d937116682f32613d31a3ae7d6f652e8146d refactor: Allow std::span construction from CKey (MarcoFalke)
999962d68d47e1e630d689aca880f41635c004cb Add missing XOnlyPubKey::data() to get mutable data (MarcoFalke)

Pull request description:

  Is is possible to construct a `Span` from a reference to a `CKey`. However, the same is not possible with `std::span`.

  Fix that.

ACKs for top commit:
  shaavan:
    ReACK fa96d937116682f32613d31a3ae7d6f652e8146d
  willcl-ark:
    ACK fa96d937116682f32613d31a3ae7d6f652e8146d

Tree-SHA512: 44fccdce5f32bc16b44f3b1bd32e86d9eabfd09bca6abe79f2d6db0cb0b5e4aaeaff710f023cb21ccde9315d2007d55f1b43f29416e81bceeeabe3948f673d3a
2024-01-17 16:00:32 +00:00
MarcoFalke
5555d8db33
test: Use blocks_path where possible 2024-01-17 16:48:47 +01:00
MarcoFalke
fa9108941f
rpc: Fix race in loadtxoutset
The tip may have advanced, also if it did not, there is no reason to
have two variables point to the same block.
2024-01-17 16:48:42 +01:00
fanquake
c818607ed5
Merge bitcoin/bitcoin#29233: build: depends move macOS C(XX) FLAGS out of C & CXX
cbc9bf11fe84deb96daf9b97a8e7499979360db2 build: move -mlinker-version to *FLAGS (fanquake)
42b2283765e28003ba58cf2043870d1e5df5dfe0 depends: deduplicate use of mmacosx-version-min in macOS build (fanquake)

Pull request description:

  Move some C/CXX FLAGS out of C/CXX. The remaining flags are host/SDK related, and will need some more thought.
  This is more correct in any case, and simplifies future changes.
  Related to #21778.

ACKs for top commit:
  theuni:
    utACK cbc9bf11fe84deb96daf9b97a8e7499979360db2
  TheCharlatan:
    ACK cbc9bf11fe84deb96daf9b97a8e7499979360db2

Tree-SHA512: 373216c1de32375faddc161ecc09b14fed0e0994cbd5ed62c862c38a5aea80c7f1740f32f36b8a60ac1caf749309750d22164c50b89916f41838d6453296ac4a
2024-01-17 10:46:16 +00:00
TheCharlatan
52149b7a2c
contrib: Fix clang-format-diff.py lint errors
We assume to be using python3, so don't check for it. This removes a
type error on the line `from io import BytesIO as StringIO`.

Specify the encoding as "utf8" when opening a file.
2024-01-17 11:10:15 +01:00
Ava Chow
8106b268cd
Merge bitcoin/bitcoin#29239: rpc: Make v2transport default for addnode RPC when enabled
3ba815b42db74804e341ce15f648c2b297af55ca Make v2transport default for addnode RPC when enabled (Pieter Wuille)

Pull request description:

  Since #29058, several types of manually configured connections will attempt v2 connections when `-v2transport` is enabled, except for the `addnode` RPC, as that one has an explicit argument to enable or disable.

  Make the default for that RPC match the `-v2transport` setting so the behavior matches that of other manual connections from a user perspective.

ACKs for top commit:
  achow101:
    ACK 3ba815b42db74804e341ce15f648c2b297af55ca
  kristapsk:
    ACK 3ba815b42db74804e341ce15f648c2b297af55ca
  theStack:
    Code-review ACK 3ba815b42db74804e341ce15f648c2b297af55ca

Tree-SHA512: 31ef48cf1e533abb17866020378c004df929e626074dc98b3229fb60a66de58435e95c8fda8d1b463e1208aa39d1f42d239818e7e58595a3738089920598befc
2024-01-16 16:50:03 -05:00
Ava Chow
a3fb1f80ac
Merge bitcoin/bitcoin#28791: snapshots: don't core dump when running -checkblockindex after loadtxoutset
cdc6ac4126b31426261605a757c52ea2dbfb2a81 snapshots: don't core dump when running -checkblockindex after `loadtxoutset` (Mark Friedenbach)

Pull request description:

  Transaction counts aren't known for block history loaded from a snapshot. If you start with `-checkblockindex` after loading a snapshot, the bitcoin daemon will core dump. The test suite does not check for this because all the snapshots have no non-coinbase transactions (all blocks prior to the snapshot are assumed to have `nTx = 1`).

  Recommend for backport to 26.x

ACKs for top commit:
  fjahr:
    utACK cdc6ac4126b31426261605a757c52ea2dbfb2a81
  achow101:
    ACK cdc6ac4126b31426261605a757c52ea2dbfb2a81
  pablomartin4btc:
    tACK cdc6ac4126b31426261605a757c52ea2dbfb2a81

Tree-SHA512: f7488a85cc29056e2ac443ce8f34aea4dfde6ba246efce82235d6a4dca2dca4344f07b93c93424b4addcb83e4cb2ae49a3ebb37d89840d42d2aeea35904cab04
2024-01-16 15:02:53 -05:00
Ava Chow
5711da6588
Merge bitcoin/bitcoin#29213: doc, test: test and explain service flag handling
74ebd4d1359edce82a134dfcd3da9840f8d206e2 doc, test: Test and explain service flag handling (Martin Zumsande)

Pull request description:

  Service flags received from the peer-to-peer network are handled differently, depending on how we receive them.
  If received directly from an outbound peer the flags belong to, they replace existing flags.
  If received via gossip relay (so that anyone could send them), new flags are added, but existing ones but cannot be overwritten.

  Document that and add test coverage for it.

ACKs for top commit:
  achow101:
    ACK 74ebd4d1359edce82a134dfcd3da9840f8d206e2
  furszy:
    ACK 74ebd4d1359edce82a134dfcd3da9840f8d206e2
  brunoerg:
    utACK 74ebd4d1359edce82a134dfcd3da9840f8d206e2

Tree-SHA512: 604adc3304b8e3cb1a10dfd017025c10b029bebd3ef533f96bcb5856fee5d4396a9aed4949908b8e7ef267ad21320d1814dd80f88426330c5c9c2c529c497591
2024-01-16 13:35:45 -05:00