Commit Graph

946 Commits

Author SHA1 Message Date
Ryan Ofsky
c864a4c194 Simplify fs::path by dropping filename() and make_preferred() overloads
These overloads were needed to allow passing `fs::path` objects directly to
libstdc++'s `fstream` constructors, but after the previous commit, there is no
longer any remaining code that does pass `fs::path` objects to `fstream`
constructors. Writing new code which does this is also discouraged because the
standard has been updated in https://wg21.link/lwg3430 to disallow it.

Dropping these also means its no longer possible to pass `fs::path` arguments
directly to `fstream::open` in libstdc++, which is somewhat unfortunate but not
a big loss because it is already not possible to pass them to the constructor.
So this commit updates `fstream::open` calls.

Additionally, this change required updates to src/bitcoin.cpp since it was
relying on the overloaded filename() method.
2025-10-06 12:14:02 -04:00
Ryan Ofsky
b0113afd44 Fix windows libc++ fs::path fstream compile errors
As reported by hebasto in https://github.com/bitcoin/bitcoin/issues/33545,
newer libc++ versions implementing https://wg21.link/lwg3430 will no longer
implicitly convert `fs::path` objects to `std::filesystem::path` objects when
constructing `std::ifstream` and `std::ofstream` types.

This is not a problem in Unix systems since `fs::path` objects use
`std::string` as their native string type, but it causes compile errors on
Windows which use `std::wstring` as their string type, since `fstream`s can't
be constructed from `wstring`s.

Fix the windows libc++ compile errors by adding a new `fs::path::std_path()`
method and using it construct `fstream`s more portably.

Additionally, delete `fs::path`'s implicit `native_string` conversion so these
errors will not go undetected in the future, even though there is not currently
a CI job testing Windows libc++ builds.
2025-10-06 11:25:56 -04:00
Ava Chow
f41f97240c Merge bitcoin/bitcoin#28584: Fuzz: extend CConnman tests
0802398e74 fuzz: make it possible to mock (fuzz) CThreadInterrupt (Vasil Dimov)
6d9e5d130d fuzz: add CConnman::SocketHandler() to the tests (Vasil Dimov)
3265df63a4 fuzz: add CConnman::InitBinds() to the tests (Vasil Dimov)
91cbf4dbd8 fuzz: add CConnman::CreateNodeFromAcceptedSocket() to the tests (Vasil Dimov)
50da7432ec fuzz: add CConnman::OpenNetworkConnection() to the tests (Vasil Dimov)
e6a917c8f8 fuzz: add Fuzzed NetEventsInterface and use it in connman tests (Vasil Dimov)
e883b37768 fuzz: set the output argument of FuzzedSock::Accept() (Vasil Dimov)

Pull request description:

  Extend `CConnman` fuzz tests to also exercise the methods `OpenNetworkConnection()`, `CreateNodeFromAcceptedSocket()`, `InitBinds()` and `SocketHandler()`.

  Previously fuzzing those methods would have resulted in real socket functions being called in the operating system which is undesirable during fuzzing. Now that https://github.com/bitcoin/bitcoin/pull/21878 is complete all those are mocked to a fuzzed socket and a fuzzed DNS resolver (see how `CreateSock` and `g_dns_lookup` are replaced in the first commit).

ACKs for top commit:
  achow101:
    ACK 0802398e74
  jonatack:
    Review re-ACK 0802398e74
  dergoegge:
    Code review ACK 0802398e74

Tree-SHA512: a717d4e79f42bacf2b029c821fdc265e10e4e5c41af77cd4cb452cc5720ec83c62789d5b3dfafd39a22cc8c0500b18169aa7864d497dded729a32ab863dd6c4d
2025-09-30 15:59:09 -07:00
merge-script
9a5ba154be Merge bitcoin/bitcoin#33310: trace: Workaround GCC bug compiling with old systemtap
93a29ff283 trace: Workaround GCC bug compiling with old systemtap (Luke Dashjr)

Pull request description:

ACKs for top commit:
  0xB10C:
    lgtm ACK 93a29ff283 - I did not test this.

Tree-SHA512: 9ce9ed8b7733af721134462073a3417e52d67e9e9853eebbddfa795842b381de98e28756ebfa6652536cbfdd08181142eccd198f4dc00a57d8748801b362b4b7
2025-09-12 11:49:38 +01:00
Luke Dashjr
93a29ff283 trace: Workaround GCC bug compiling with old systemtap 2025-09-04 19:25:42 +00:00
merge-script
9b1a7c3e8d Merge bitcoin/bitcoin#33116: refactor: Convert uint256 to Txid
de0675f9de refactor: Move `transaction_identifier.h` to primitives (marcofleon)
6f068f65de Remove implicit uint256 conversion and comparison (marcofleon)
9c24cda72e refactor: Convert remaining instances from uint256 to Txid (marcofleon)
d2ecd6815d policy, refactor: Convert uint256 to Txid (marcofleon)
f6c0d1d231 mempool, refactor: Convert uint256 to Txid (marcofleon)
aeb0f78330 refactor: Convert `mini_miner` from uint256 to Txid (marcofleon)
326f244724 refactor: Convert RPCs and `merkleblock` from uint256 to Txid (marcofleon)
49b3d3a92a Clean up `FindTxForGetData` (marcofleon)

Pull request description:

  This is the final leg of the [type safety refactor](https://github.com/bitcoin/bitcoin/pull/32189).

  All of these changes are straightforward `uint256` --> `Txid` along with any necessary explicit conversions. Also, `transaction_identifier.h` is moved to primitives in the last commit, as `Txid` and `Wtxid` become fundamental types after this PR.

ACKs for top commit:
  stickies-v:
    re-ACK de0675f9de, no changes since a20724d926d5844168c6a13fa8293df8c8927efe except address review nits.
  janb84:
    re ACK de0675f9de
  dergoegge:
    re-ACK de0675f9de
  theStack:
    Code-review ACK de0675f9de

Tree-SHA512: 2413160fca7ab146a8d79d18ce3afcf7384cacc73c513d41928904aa453b4dd7a350064cee71e9c5d015da5904c7c81ac17603e50a47441ebc5b0c653235dd08
2025-08-13 14:50:51 -04:00
marcofleon
de0675f9de refactor: Move transaction_identifier.h to primitives
Moves the file from `src/util` to `src/primitives`. Now that the
refactor is complete, Txid and Wtxid are fundamental types, so it
makes sense for them to reside in `src/primitives`.
2025-08-11 16:47:51 +01:00
marcofleon
6f068f65de Remove implicit uint256 conversion and comparison 2025-08-11 16:47:51 +01:00
marcofleon
9c24cda72e refactor: Convert remaining instances from uint256 to Txid
These remaining miscellaneous changes were identified by commenting out
the `operator const uint256&` conversion and the `Compare(const uint256&)`
method from `transaction_identifier.h`.
2025-08-11 16:47:43 +01:00
willcl-ark
db3228042b util: detect and warn when using exFAT on macOS
exFAT is known to cause corruption on macOS. See #28552.

Therefore we should warn when using this fs format for either the blocks
or data directories on macOS.

Co-authored-by: l0rinc <pap.lorinc@gmail.com>
2025-08-08 19:21:06 +01:00
Ava Chow
d767503b6a Merge bitcoin/bitcoin#33039: refactor,test: follow-ups to multi-byte block obfuscation
86e3a0a8cb refactor: standardize obfuscation memory alignment (Lőrinc)
13f00345c0 refactor: write `Obfuscation` object when new key is generated in dbwrapper (Lőrinc)
e5b1b7c557 refactor: rename `OBFUSCATION_KEY_KEY` (Lőrinc)
298bf95105 refactor: simplify `Obfuscation::HexKey` (Lőrinc)
2dea045425 test: make `obfuscation_serialize` more thorough (Lőrinc)
a17d8202c3 test: merge xor_roundtrip_random_chunks and xor_bytes_reference (Lőrinc)

Pull request description:

  Follow up for https://github.com/bitcoin/bitcoin/pull/31144
  Applied the remaining comments in separate commits - except for the last one where I could group them.
  Please see the commit messages for more context.

ACKs for top commit:
  achow101:
    ACK 86e3a0a8cb
  ryanofsky:
    Code review ACK 86e3a0a8cb, just tweaking key write assert as suggested
  hodlinator:
    ACK 86e3a0a8cb

Tree-SHA512: 967510a141fbb57bf9d088d92b554cf2fffc2f6aa0eab756cbae3230f53e9b04ceebcc6fea5f3383c01ad41985ecde5b5686c64a771ca9deae3497b9b88c1c8b
2025-08-06 15:46:18 -07:00
merge-script
9617a42fdb Merge bitcoin/bitcoin#32581: allocators: Apply manual ASan poisoning to PoolResource
ad132761fc [allocators] Apply manual ASan poisoning to PoolResource (dergoegge)

Pull request description:

  Currently ASan will not detect use-after-free issues for memory allocated by a `PoolResource`. This is because ASan is only aware of the memory chunks allocated by `PoolResource` but not the individual "sub-chunks" within.

  E.g. this test will not produce an ASan error even though the referenced coin has been deallocated:

  ```c++
  diff --git a/src/test/coins_tests.cpp b/src/test/coins_tests.cpp
  index c46144b34b..aa6ca15ce1 100644
  --- a/src/test/coins_tests.cpp
  +++ b/src/test/coins_tests.cpp
  @@ -508,6 +508,17 @@ BOOST_FIXTURE_TEST_CASE(updatecoins_simulation_test, UpdateTest)
       BOOST_CHECK(spent_a_duplicate_coinbase);
   }

  +BOOST_AUTO_TEST_CASE(asan_uaf)
  +{
  +    CCoinsMapMemoryResource cache_coins_memory_resource{};
  +    CCoinsMap map(0, SaltedOutpointHasher(/*deterministic=*/true), CCoinsMap::key_equal{}, &cache_coins_memory_resource);
  +    COutPoint outpoint{};
  +    map.emplace(outpoint, Coin{});
  +    auto& coin = map.at(outpoint);
  +    map.erase(outpoint);
  +    coin.coin.nHeight = 1;
  +}
  +
   BOOST_AUTO_TEST_CASE(ccoins_serialization)
   {
       // Good example
  ```

  Fix this by applying [manual ASan poisoning](https://github.com/google/sanitizers/wiki/AddressSanitizerManualPoisoning) for memory allocated by `PoolResource`:

  * Newly allocated chunks are poisoned as a whole
  * "Sub-chunks" are unpoisoned/re-poisoned during allocation/deallocation

  With the poisoning applied, ASan catches the issue in the test above:
  ```
  $ ./build_unit/bin/test_bitcoin --run_test="coins_tests/asan_uaf"
  Running 1 test case...
  =================================================================
  ==366064==ERROR: AddressSanitizer: use-after-poison on address 0x7f99c3204870 at pc 0x55569dab6f8a bp 0x7ffe0210e4d0 sp 0x7ffe0210e4c8
  READ of size 4 at 0x7f99c3204870 thread T0 (b-test)
  ```

ACKs for top commit:
  achow101:
    ACK ad132761fc
  marcofleon:
    code review ACK ad132761fc

Tree-SHA512: eb5e80bfa9509225e784151807bd8aa21fb0826ca1781dfe81b1d60bd3766019384ea3f9cb8e53398fde2f4e994a9c201b5a9962b4d279d7e52bb60e8961be11
2025-08-05 10:47:01 +01:00
merge-script
5ee4e79669 Merge bitcoin/bitcoin#31244: descriptors: MuSig2
5fe7915c86 doc: Add musig() example (Ava Chow)
d576079ab4 tests: Test musig() parsing (Ava Chow)
a53924bee3 descriptor: Parse musig() key expressions (Ava Chow)
9473e9606c descriptors: Move DeriveType parsing into its own function (Ava Chow)
4af0dca096 descriptor: Add MuSigPubkeyProvider (Ava Chow)
d00d95437d Add MuSig2 Keyagg Cache helper functions (Ava Chow)
8ecea91bf2 sign: Add GetMuSig2ParticipantPubkeys to SigningProvider (Ava Chow)
fac0ee0bfc build: Enable secp256k1 musig module (Ava Chow)
1894f97503 descriptors: Add PubkeyProvider::IsBIP32() (Ava Chow)
12bc1d0b1e util/string: Allow Split to include the separator (Ava Chow)
8811312571 script/parsing: Allow Const to not skip the found constant (Ava Chow)
5fe4c66462 XOnlyPubKey: Add GetCPubKeys (Ava Chow)

Pull request description:

  Implements parsing of BIP 390 `musig()` descriptors.

  Split from #29675

ACKs for top commit:
  w0xlt:
    reACK 5fe7915c86
  rkrux:
    ACK 5fe7915c86
  theStack:
    re-ACK 5fe7915c86 🎹
  Sjors:
    ACK 5fe7915c86

Tree-SHA512: a5be6288e277187fb9a1e2adf4e9822b46b1b8380d732b2fabd53f317c839aecb1971b04410486cbd65047fbc20956675d4d676f56caa37a44ff0e4d12b9b081
2025-07-31 16:51:39 -04:00
merge-script
dc78ed2140 Merge bitcoin/bitcoin#33005: refactor: GenTxid type safety followups
94b39ce738 refactor: Change `m_tx_inventory_to_send` from `std::set<GenTxid>` to `std::set<Wtxid>` (marcofleon)
a9819b0e9d refactor: Change `FindTxForGetData` to take GenTxid instead of CInv (marcofleon)
d588575ed1 refactor: miscellaneous GenTxid followups (marcofleon)

Pull request description:

  This is a followup to https://github.com/bitcoin/bitcoin/pull/32631.

  Addresses:
  https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2200291621
  https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2199951996
  https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2201874252
  https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2201918072
  https://github.com/bitcoin/bitcoin/pull/32631#issuecomment-3063383775

ACKs for top commit:
  glozow:
    ACK 94b39ce738
  maflcko:
    review ACK 94b39ce738 🎲
  stickies-v:
    ACK 94b39ce738

Tree-SHA512: 3c88656b2e4e676653db87df5b1b694e1b1f40d89d7b825dad068e57c9c9f8add098ba797413274bd992b1c1fdec94c794ab3fd086d2a9561f5de92ae9f6d942
2025-07-30 14:20:03 -04:00
merge-script
75a5c8258e Merge bitcoin/bitcoin#33063: util: Revert "common: Close non-std fds before exec in RunCommandJSON"
faa1c3e80d Revert "Merge bitcoin/bitcoin#32343: common: Close non-std fds before exec in RunCommandJSON" (MarcoFalke)

Pull request description:

  After a fork() in a multithreaded program, the child can safely
  call only async-signal-safe functions (see [signal-safety(7)](https://www.man7.org/linux/man-pages/man7/signal-safety.7.html))
  until such time as it calls execv.

  The standard library (`std` namespace) is not async-signal-safe. Also, `throw`, isn't.

  There was an alternative implementation using `readdir` (https://github.com/bitcoin/bitcoin/pull/32529), but that isn't async-signal-safe either, and that implementation was still using `throw`.

  So temporarily revert this feature.

  A follow-up in the future can add it back, using only async-signal-safe functions, or by using a different approach.

  Fixes https://github.com/bitcoin/bitcoin/issues/32524
  Fixes https://github.com/bitcoin/bitcoin/issues/33015
  Fixes https://github.com/bitcoin/bitcoin/issues/32855

  For reference, a failure can manifest in the GCC debug mode:

  * While `fork`ing, a debug mode mutex is held (by any other thread).
  * The `fork`ed child tries to use the stdard libary before `execv` and deadlocks.

  This may look like the following:

  ```
  (gdb) thread apply all bt

  Thread 1 (Thread 0xf58f4b40 (LWP 774911) "b-httpworker.2"):
  #0  0xf7f4f589 in __kernel_vsyscall ()
  #1  0xf79e467e in ?? () from /lib32/libc.so.6
  #2  0xf79eb582 in pthread_mutex_lock () from /lib32/libc.so.6
  #3  0xf7d93bf2 in ?? () from /lib32/libstdc++.so.6
  #4  0xf7d93f36 in __gnu_debug::_Safe_iterator_base::_M_attach(__gnu_debug::_Safe_sequence_base*, bool) () from /lib32/libstdc++.so.6
  #5  0x5668810a in __gnu_debug::_Safe_iterator_base::_Safe_iterator_base (this=0xf58f13ac, __seq=0xf58f13f8, __constant=false) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_base.h:91
  #6  0x56ddfb50 in __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<int*, std::__cxx1998::vector<int, std::allocator<int> > >, std::__debug::vector<int, std::allocator<int> >, std::forward_iterator_tag>::_Safe_iterator (this=0xf58f13a8, __i=3, __seq=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_iterator.h:162
  #7  0x56ddfacb in __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<int*, std::__cxx1998::vector<int, std::allocator<int> > >, std::__debug::vector<int, std::allocator<int> >, std::bidirectional_iterator_tag>::_Safe_iterator (this=0xf58f13a8, __i=3, __seq=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_iterator.h:539
  #8  0x56ddfa5b in __gnu_debug::_Safe_iterator<__gnu_cxx::__normal_iterator<int*, std::__cxx1998::vector<int, std::allocator<int> > >, std::__debug::vector<int, std::allocator<int> >, std::random_access_iterator_tag>::_Safe_iterator (this=0xf58f13a8, __i=3, __seq=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/safe_iterator.h:687
  #9  0x56ddd3f6 in std::__debug::vector<int, std::allocator<int> >::begin (this=0xf58f13f8) at /bin/../lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/debug/vector:300
  #10 0x57d83701 in subprocess::detail::Child::execute_child (this=0xf58f156c) at ./util/subprocess.h:1372
  #11 0x57d80a7c in subprocess::Popen::execute_process (this=0xf58f1cd8) at ./util/subprocess.h:1231
  #12 0x57d6d2b4 in subprocess::Popen::Popen<subprocess::input, subprocess::output, subprocess::error, subprocess::close_fds> (this=0xf58f1cd8, cmd_args="fake.py enumerate", args=..., args=..., args=..., args=...) at ./util/subprocess.h:964
  #13 0x57d6b597 in RunCommandParseJSON (str_command="fake.py enumerate", str_std_in="") at ./common/run_command.cpp:27
  #14 0x57a90547 in ExternalSigner::Enumerate (command="fake.py", signers=std::__debug::vector of length 0, capacity 0, chain="regtest") at ./external_signer.cpp:28
  #15 0x56defdab in enumeratesigners()::$_0::operator()(RPCHelpMan const&, JSONRPCRequest const&) const (this=0xf58f2ba0, self=..., request=...) at ./rpc/external_signer.cpp:51
  ...
  (truncated, only one thread exists)
  ```

ACKs for top commit:
  fanquake:
    ACK faa1c3e80d
  darosior:
    ACK faa1c3e80d

Tree-SHA512: 602da5f2eba08d7fe01ba19baf411e287ae27fe2d4b82f41734e05b7b1d938ce94cc0041e86ba677284fa92838e96ebee687023ff28047e2b036fd9a53567e0a
2025-07-26 10:20:58 +01:00
MarcoFalke
faa1c3e80d Revert "Merge bitcoin/bitcoin#32343: common: Close non-std fds before exec in RunCommandJSON"
This reverts commit 31d3eebfb9, reversing
changes made to 4b26ca0e2f.
2025-07-25 15:30:42 +02:00
MarcoFalke
face8123fd log: [refactor] Use info level for init logs
This refactor does not change behavior.
2025-07-25 09:50:50 +02:00
Lőrinc
86e3a0a8cb refactor: standardize obfuscation memory alignment
See:
* https://github.com/bitcoin/bitcoin/pull/31144#discussion_r2216962117
* https://github.com/bitcoin/bitcoin/pull/31144#discussion_r2220277161
* https://github.com/bitcoin/bitcoin/pull/31144#discussion_r2210851772

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2025-07-23 12:56:48 -07:00
marcofleon
d588575ed1 refactor: miscellaneous GenTxid followups
Addresses a few comments from #32631:
https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2199951996
https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2201874252
https://github.com/bitcoin/bitcoin/pull/32631#discussion_r2201918072
2025-07-23 15:01:18 +01:00
Lőrinc
298bf95105 refactor: simplify Obfuscation::HexKey
See: https://github.com/bitcoin/bitcoin/pull/31144#discussion_r2215746554

Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
2025-07-22 10:26:15 -07:00
Lőrinc
248b6a27c3 optimization: peel align-head and unroll body to 64 bytes
Benchmarks indicated that obfuscating multiple bytes already gives an order of magnitude speed-up, but:
* GCC still emitted scalar code;
* Clang’s auto-vectorized loop ran on the slow unaligned-load path.

Fix contains:
* peeling the misaligned head enabled the hot loop starting at an 8-byte address;
* `std::assume_aligned<8>` tells the optimizer the promise holds - required to keep Apple Clang happy;
* manually unrolling the body to 64 bytes enabled GCC to auto-vectorize.

Note that `target.size() > KEY_SIZE` condition is just an optimization, the aligned and unaligned loops work without it as well - it's why the alignment calculation still contains `std::min`.

>  C++ compiler .......................... GNU 14.2.0

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                0.03 |   32,464,658,919.11 |    0.0% |            0.50 |            0.11 |  4.474 |           0.08 |    0.0% |      5.29 | `ObfuscationBench`

> C++ compiler .......................... Clang 20.1.7

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                0.02 |   41,231,547,045.17 |    0.0% |            0.30 |            0.09 |  3.463 |           0.02 |    0.0% |      5.47 | `ObfuscationBench`

Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
2025-07-16 14:37:19 -07:00
Lőrinc
e7114fc6dc optimization: migrate fixed-size obfuscation from std::vector<std::byte> to uint64_t
All former `std::vector<std::byte>` keys were replaced with `uint64_t` (we still serialize them as vectors but convert immediately to `uint64_t` on load).
This is why some tests still generate vector keys and convert them to `uint64_t` later instead of generating them directly.

In `Obfuscation::Unserialize` we can safely throw an `std::ios_base::failure` since during mempool fuzzing `mempool_persist.cpp#L141` catches and ignored these errors.

>  C++ compiler .......................... GNU 14.2.0

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                0.04 |   28,365,698,819.44 |    0.0% |            0.34 |            0.13 |  2.714 |           0.07 |    0.0% |      5.33 | `ObfuscationBench`

> C++ compiler .......................... Clang 20.1.7

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                0.08 |   13,012,464,203.00 |    0.0% |            0.65 |            0.28 |  2.338 |           0.13 |    0.8% |      5.50 | `ObfuscationBench`

Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-07-16 14:33:07 -07:00
Lőrinc
478d40afc6 refactor: encapsulate vector/array keys into Obfuscation 2025-07-16 14:33:07 -07:00
Lőrinc
377aab8e5a refactor: move util::Xor to Obfuscation().Xor
This is meant to focus the usages to narrow the scope of the obfuscation optimization.

`Obfuscation::Xor` is mostly a move.

Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
2025-07-16 14:33:07 -07:00
Lőrinc
54ab0bd64c refactor: commit to 8 byte obfuscation keys
Since 31 byte xor-keys are not used in the codebase, using the common size (8 bytes) makes the benchmarks more realistic.

Co-authored-by: maflcko <6399679+maflcko@users.noreply.github.com>
2025-07-16 13:19:18 -07:00
merge-script
23e15d40b9 Merge bitcoin/bitcoin#32631: refactor: Convert GenTxid to std::variant
a60f863d3e scripted-diff: Replace GenTxidVariant with GenTxid (marcofleon)
c8ba199598 Remove old GenTxid class (marcofleon)
072a198ea4 Convert remaining instances of GenTxid to GenTxidVariant (marcofleon)
1b528391c7 Convert `txrequest` to GenTxidVariant (marcofleon)
bde4579b07 Convert `txdownloadman_impl` to GenTxidVariant (marcofleon)
c876a892ec Replace GenTxid with Txid/Wtxid overloads in `txmempool` (marcofleon)
de858ce2be move-only: make GetInfo a private CTxMemPool member (stickies-v)
eee473d9f3 Convert `CompareInvMempoolOrder` to GenTxidVariant (marcofleon)
243553d590 refactor: replace get_iter_from_wtxid with GetIter(const Wtxid&) (stickies-v)
fcf92fd640 refactor: make CTxMemPool::GetIter strongly typed (marcofleon)
11d28f21bb Implement GenTxid as a variant (marcofleon)

Pull request description:

  Part of the [type safety refactor](https://github.com/bitcoin/bitcoin/pull/32189).

  This PR changes the GenTxid class to a variant, which holds both Txids and Wtxids. This provides compile-time type safety and eliminates the manual type check (bool m_is_wtxid). Variables that can be either a Txid or a Wtxid are now using the new GenTxid variant, instead of uint256.

ACKs for top commit:
  w0xlt:
    ACK a60f863d3e
  dergoegge:
    Code review ACK a60f863d3e
  maflcko:
    review ACK a60f863d3e 🎽
  theStack:
    Code-review ACK a60f863d3e

Tree-SHA512: da9b73b7bdffee2eb9281a409205519ac330d3336094d17681896703fbca8099608782c9c85801e388e4d90af5af8abf1f34931f57bbbe6e9674d802d6066047
2025-07-11 13:47:19 -04:00
MarcoFalke
fa6b45fa8e Add SetMockTime for time_point types 2025-07-09 13:57:54 +02:00
marcofleon
a60f863d3e scripted-diff: Replace GenTxidVariant with GenTxid
-BEGIN VERIFY SCRIPT-
sed -i 's/GenTxidVariant/GenTxid/g' $(git grep -l 'GenTxidVariant')
-END VERIFY SCRIPT-
2025-07-08 20:00:51 +01:00
marcofleon
c876a892ec Replace GenTxid with Txid/Wtxid overloads in txmempool
Co-authored-by: stickies-v <stickies-v@protonmail.com>
2025-07-08 19:31:02 +01:00
marcofleon
11d28f21bb Implement GenTxid as a variant
Reimplements the GenTxid class as a variant for better type safety.
Also adds two temporary functions to the old GenTxid class that
convert to and from the new variant.
2025-06-25 23:08:15 +01:00
Ava Chow
12bc1d0b1e util/string: Allow Split to include the separator
When splitting a string, sometimes the separator needs to be included.
Split will now optionally include the separator at the end of the left
side of the splits, i.e. it appears at the end of the splits, except
for the last one.

Specifically, for musig() descriptors, Split is used to separate a
musig() from any derivation path that follows it by splitting on the
closing parentheses. Since that parentheses is needed for Func() and
Expr(), Split() needs to preserve the end parentheses instead of
discarding it.
2025-06-11 11:09:42 -07:00
Vasil Dimov
0802398e74 fuzz: make it possible to mock (fuzz) CThreadInterrupt
* Make the methods of `CThreadInterrupt` virtual and store a pointer to
  it in `CConnman`, thus making it possible to override with a mocked
  instance.
* Initialize `CConnman::m_interrupt_net` from the constructor, making it
  possible for callers to supply mocked version.
* Introduce `FuzzedThreadInterrupt` and `ConsumeThreadInterrupt()` and
  use them in `src/test/fuzz/connman.cpp` and `src/test/fuzz/i2p.cpp`.

This improves the CPU utilization of the `connman` fuzz test.

As a nice side effect, the `std::shared_ptr` used for
`CConnman::m_interrupt_net` resolves the possible lifetime issues with
it (see the removed comment for that variable).
2025-06-09 14:17:33 +02:00
MarcoFalke
fa9ca13f35 refactor: Sort includes of touched source files 2025-06-03 19:56:55 +02:00
MarcoFalke
facb152697 scripted-diff: Bump copyright headers after include changes
Historically, the headers have been bumped some time after a file has
been touched. Do it now to avoid having to touch them again in the
future for that reason.

-BEGIN VERIFY SCRIPT-
 sed -i --regexp-extended 's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' $( git show --pretty="" --name-only HEAD~0 )
-END VERIFY SCRIPT-
2025-06-03 15:13:57 +02:00
MarcoFalke
fae71d30f7 clang-tidy: Apply modernize-deprecated-headers
This can be reproduced according to the developer notes with something
like

( cd ./src/ && ../contrib/devtools/run-clang-tidy.py -p ../bld-cmake -fix -j $(nproc) )

Also, the header related changes were done manually.
2025-06-03 15:13:54 +02:00
merge-script
053bda5d9f Merge bitcoin/bitcoin#32460: fs: remove _POSIX_C_SOURCE defining
24e5fd3bed fs: remove _POSIX_C_SOURCE defining (fanquake)

Pull request description:

  On Linux systems, `_POSIX_C_SOURCE` will default to `200809L` (since glibc 2.10). There's currently no reason for us to undefine it, and then set it to an earlier value. Also tested with musl libc.

  I think if anything, the project should be settings macros like `_POSIX_C_SOURCE`, globally.

ACKs for top commit:
  hebasto:
    re-ACK 24e5fd3bed, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/32460#pullrequestreview-2854183748).

Tree-SHA512: 920d60058821992193616e0c73c2f7e4230a9e3ccb9d71d16493ae69696c868f4325d3dd2d4e8388749080c187aa7b205493b3e2c6986ad37440e591ebe107e1
2025-05-30 10:21:01 +01:00
Ava Chow
012f347685 Merge bitcoin/bitcoin#31375: multiprocess: Add bitcoin wrapper executable
a5ac43d98d doc: Add release notes describing bitcoin wrapper executable (Ryan Ofsky)
258bda80c0 doc: Mention bitcoin wrapper executable in documentation (Ryan Ofsky)
d2739d75c9 build: add bitcoin.exe to windows installer (Sjors Provoost)
ba649c0006 ci: Run multiprocess tests through wrapper executable (Ryan Ofsky)
29bdd743bb test: Support BITCOIN_CMD environment variable (Ryan Ofsky)
9c8c68891b multiprocess: Add bitcoin wrapper executable (Ryan Ofsky)
5076d20fdb util: Add cross-platform ExecVp and GetExePath functions (Ryan Ofsky)

Pull request description:

  Intended to make bitcoin command line features more discoverable and allow installing new multiprocess binaries in libexec/ instead of bin/ so they don't cause confusion.

  Idea and implementation of this were discussed in https://github.com/bitcoin/bitcoin/issues/30983.

  ---

  Initial implementation of this feature is deliberately minimal so the UX can evolve in response to feedback and there are not too many details to debate and discuss in a single PR. But many improvements are possible or planned:

  - Adding manpage and bash completions.
  - Showing nicer error messages that detect if an executable isn't installed and suggest how to fix [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2073194474)
  - Showing wrapper command lines in subcommand in help output [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2077800405). This could be done conditionally as suggested in the comment or be unconditional.
  - Showing wrapper command lines in subcommand error output. There is a bitcoin-cli error pointed out in [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091152243) that is needlessly confusing.
  - Integrating help so `bitcoin help subcommand` invokes `bitcoin subcommand -h`. `bitcoin -h subcommand` should also be supported and be equivalent [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2093116725)
  - Adding support for `bitcoin-util` subcommands. Ideal interface would probably be more like `bitcoin grind` not `bitcoin util grind` but this has been punted for now. Supporting subcommands directly would require some ArgsManager modifications
  - Adding a dedicated python functional test for the wrapper. Right now there is some CI coverage by setting the `BITCOIN_CMD` variable, but this doesn't cover things like the help output and version output, and support for different directory layouts.
  - Better `--multiprocess` (`-m`) / `--monolithic` (`-M`) default selection. Right now, default is monolithic but it probably makes sense to chose more intelligently depending on whether -ipc options are enabled and what binaries are available.
  - Maybe parsing `bitcoin.conf` and supporting options to control wrapper behavior like custom locations or preferences or aliases.
  - Better command command line usability. Allow combining short options like (`-ah`). Allow fuzzy matching of subcommands or suggestions if you misspell. (suggested by stickies in review club)
  - Not directly related to this PR but `bitcoin-cli named` implementation used by the wrapper should do a better job disambiguating named arguments from base64 arguments ending in = as pointed out in [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091886628)

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722). A review club meeting for it took place in https://bitcoincore.reviews/31375

ACKs for top commit:
  Sjors:
    utACK a5ac43d98d
  achow101:
    ACK a5ac43d98d
  vasild:
    ACK a5ac43d98d
  theStack:
    ACK a5ac43d98d
  ismaelsadeeq:
    fwiw my last review implied an ACK a5ac43d98d
  hodlinator:
    ACK a5ac43d98d

Tree-SHA512: 570e6a4ff8bd79ef6554da3d01f36c0a7c6d2dd7dace8f8732eca98f4a8bc2284474a9beadeba783114fe2f3dd08b2041b3da7753bae0b7f881ec50668cb821f
2025-05-27 12:38:19 -07:00
dergoegge
ad132761fc [allocators] Apply manual ASan poisoning to PoolResource 2025-05-23 10:09:39 +01:00
merge-script
35bf3f8839 Merge bitcoin/bitcoin#32400: random: Use modern Windows randomness functions
6b4bcc1623 random: Use modern Windows randomness functions (David Gumberg)

Pull request description:

  This change resolves #32391 and is a follow-up to #14089.

  The old randomness API has been deprecated and will be removed at some point according to Microsoft.[^1] This PR removes all uses of that API from Bitcoin Core code, but the deprecated API is still invoked in Bitcoin Core binaries compiled after this PR because of upstream use, see this comment: https://github.com/bitcoin/bitcoin/pull/32400#issuecomment-2846972614.

  For reference on `BCryptGenRandom`, see: https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom.

  [`STATUS_SUCCESS`](https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55) gets defined here since including `ntstatus.h` is [more trouble](70f149b9a1/examples/examples_util.h (L19-L28)) than it's worth.

  [^1]: https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontextw & https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom

ACKs for top commit:
  laanwj:
    re-ACK 6b4bcc1623
  fanquake:
    ACK 6b4bcc1623

Tree-SHA512: ddd9093669dfd6ff0eee7e5e6a9c7dce798d03dd9a81dcc2e668e9b84779b7adab3105a7f0c8038e54accf28f19fe211628e13b3fc2200caa5b423f766725e37
2025-05-22 12:12:57 +01:00
fanquake
24e5fd3bed fs: remove _POSIX_C_SOURCE defining
On Linux systems, `_POSIX_C_SOURCE` will default to `200809L` (since
glibc 2.10). There's currently no reason for us to undefine it, and then
set it to an earlier value. Also tested with musl libc.

I think if anything, the project should be settings macros like
`_POSIX_C_SOURCE`, globally.
2025-05-21 15:58:11 +01:00
merge-script
0a56ed1ac8 Merge bitcoin/bitcoin#32567: subprocess: Backport upstream changes
e63a7034f0 subprocess: Don't add an extra whitespace at end of Windows command line (laanwj)

Pull request description:

  A list of the backported PRs:
  - https://github.com/arun11299/cpp-subprocess/pull/119

  The following PRs were skipped for backporting:
  - https://github.com/arun11299/cpp-subprocess/pull/118 because there is no changes in the header code.

  Required for https://github.com/bitcoin/bitcoin/pull/32566.

ACKs for top commit:
  laanwj:
    Code review ACK e63a7034f0

Tree-SHA512: 69a74aa7f9c611a9ec910e27161c5e9e147067d37f8335953cd3875fcc88dc840a2f7b206bb603f22507159e406b1449f1dc4702fffe890bb824672641b4feed
2025-05-21 12:08:36 +01:00
merge-script
fad009af49 Merge bitcoin/bitcoin#32520: Remove legacy Parse(U)Int*
faf55fc80b doc: Remove ParseInt mentions in documentation (MarcoFalke)
3333282933 refactor: Remove unused Parse(U)Int* (MarcoFalke)
fa84e6c36c bitcoin-tx: Reject + sign in MutateTxDel* (MarcoFalke)
face2519fa bitcoin-tx: Reject + sign in vout parsing (MarcoFalke)
fa8acaf0b9 bitcoin-tx: Reject + sign in replaceable parsing (MarcoFalke)
faff25a558 bitcoin-tx: Reject + sign in locktime (MarcoFalke)
dddd9e5fe3 bitcoin-tx: Reject + sign in nversion parsing (MarcoFalke)
fab06ac037 rest: Use SAFE_CHARS_URI in SanitizeString error msg (MarcoFalke)
8888bb499d rest: Reject + sign in /blockhashbyheight/ (MarcoFalke)
fafd43c691 test: Reject + sign when parsing regtest deployment params (MarcoFalke)
fa123afa0e Reject + sign when checking -ipcfd (MarcoFalke)
fa479857ed Reject + sign in SplitHostPort (MarcoFalke)
fab4c2967d net: Reject + sign when parsing subnet mask (MarcoFalke)
fa89652e68 init: Reject + sign in -*port parsing (MarcoFalke)
fa9c45577d cli: Reject + sign in -netinfo level parsing (MarcoFalke)
fa98041325 refactor: Use ToIntegral in CreateFromDump (MarcoFalke)
fa23ed7fc2 refactor: Use ToIntegral in ParseHDKeypath (MarcoFalke)

Pull request description:

  The legacy int parsing is problematic, because it accepts the `+` sign for unsigned integers. In all cases this is either:

  * Useless, because the `+` sign was already rejected.
  * Erroneous and inconsistent, when third party parsers reject it. (C.f. https://github.com/bitcoin/bitcoin/pull/32365)
  * Confusing, because the `+` sign is  neither documented, nor can it be assumed to be present.

  Fix all issues by removing the legacy int parsing.

ACKs for top commit:
  stickies-v:
    re-ACK faf55fc80b
  brunoerg:
    code review ACK faf55fc80b

Tree-SHA512: a311ab6a58fe02a37741c1800feb3dcfad92377b4bfb61b433b2393f52ba89ef45d00940972b2767b213a3dd7b59e5e35d5b659c586eacdfe4e565a77b12b19f
2025-05-20 15:55:38 +01:00
laanwj
e63a7034f0 subprocess: Don't add an extra whitespace at end of Windows command line
The windows code adds an unnecessary extra space to the command line.
This can cause subtle issues, so avoid it.

Github-Pull: arun11299/cpp-subprocess#119
Rebased-From: 777cfa77d1f84bb08b3e445d5f7fc6c87282223b
2025-05-20 12:10:10 +01:00
fanquake
c7c3bfadfc doc: add & amend copyright headers 2025-05-20 09:43:21 +01:00
MarcoFalke
faf55fc80b doc: Remove ParseInt mentions in documentation
In the dev notes, remove the whole section, because:

* ParseDouble was removed in commit
  fa9d72a794
* The locale-dependent atoi is already checked by
  test/lint/lint-locale-dependence.py

Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
2025-05-20 06:50:50 +02:00
fanquake
7193245cd6 doc: remove For ... comments
We don't add or maintain these, and they are of little value, as
well as having the effect of polluting diffs.

They are also wrong, i.e DEFAULT_SCRIPTCHECK_THREADS is not in
validation.h.
2025-05-19 16:40:33 +01:00
MarcoFalke
3333282933 refactor: Remove unused Parse(U)Int* 2025-05-19 17:16:13 +02:00
MarcoFalke
fa479857ed Reject + sign in SplitHostPort
It is better to reject it with an error. For example,

$ bitcoin-cli -rpcconnect=127.0.0.1:+23501 -getinfo
error: Invalid port provided in -rpcconnect: 127.0.0.1:+23501
2025-05-15 22:12:14 +02:00
MarcoFalke
fa23ed7fc2 refactor: Use ToIntegral in ParseHDKeypath
ToIntegral<uint32_t> only accepts numbers, so just use that to replace
the equivalent but more verbose way with find_first_not_of+ParseUInt32.
2025-05-15 19:33:58 +02:00
David Gumberg
6b4bcc1623 random: Use modern Windows randomness functions
The old randomness API has been deprecated and may be removed soon.[^1]

For reference on `BCryptGenRandom`, see: https://learn.microsoft.com/en-us/windows/win32/api/bcrypt/nf-bcrypt-bcryptgenrandom.

`STATUS_SUCCESS`[^2] gets defined here since including `ntstatus.h` is
more trouble than it's worth. [^3]

[^1]: https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptacquirecontextw & https://learn.microsoft.com/en-us/windows/win32/api/wincrypt/nf-wincrypt-cryptgenrandom
[^2]: https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-erref/596a1078-e883-4972-9bbc-49e60bebca55
[^3]: See 70f149b9a1/examples/examples_util.h (L19-L28)
2025-05-14 22:55:10 -07:00