Commit Graph

6287 Commits

Author SHA1 Message Date
bensig
08ed802bab doc: fix double-word typos in comments 2025-12-30 12:12:26 -08:00
merge-script
7249bcc4f8 Merge bitcoin/bitcoin#34119: contrib: remove copyright_header.py
ba6315d2f6 contrib: remove copyright_header.py (fanquake)
3e4765ee10 scripted-diff: [doc] Unify stale copyright headers (fanquake)

Pull request description:

  After #34084, our copyright headers shouldn't need "managing"; so remove the Python script.

ACKs for top commit:
  fjahr:
    ACK ba6315d2f6
  rkrux:
    crACK [ba6315d](ba6315d2f6)
  janb84:
    ACK ba6315d2f6

Tree-SHA512: c0d6ed0a71803c5ae6c70260fa4162bea1f1b24cf6fc9b58e018bb9d6a673d2d59c25a17deda067a268024e3757081e3a214680e1e626d71c0debc5066d5f623
2025-12-29 07:03:02 -08:00
merge-script
eb0594e23f Merge bitcoin/bitcoin#33891: kernel: Expose reusable PrecomputedTransactionData in script validation
44e006d438 [kernel] Expose reusable PrecomputedTransactionData in script valid (Josh Doman)

Pull request description:

  This PR exposes a reusable `PrecomputedTransactionData` object in script validation using libkernel.

  Currently, libkernel computes `PrecomputedTransactionData` each time `btck_script_pubkey_verify` is called, exposing clients to quadratic hashing when validating a transaction with multiple inputs. By externalizing `PrecomputedTransactionData` and making it reusable, libkernel can eliminate this attack vector.

  I discussed this problem in [this issue](https://github.com/TheCharlatan/rust-bitcoinkernel/issues/46). The design of this PR is inspired by @sedited's comments.

  The PR introduces three new APIs for managing the `btck_PrecomputedTransactionData` object:
  ```c
  /**
   * @brief Create precomputed transaction data for script verification.
   *
   * @param[in] tx_to             Non-null.
   * @param[in] spent_outputs     Nullable for non-taproot verification. Points to an array of
   *                              outputs spent by the transaction.
   * @param[in] spent_outputs_len Length of the spent_outputs array.
   * @return                      The precomputed data, or null on error.
   */
  btck_PrecomputedTransactionData* btck_precomputed_transaction_data_create(
      const btck_Transaction* tx_to,
      const btck_TransactionOutput** spent_outputs, size_t spent_outputs_len) BITCOINKERNEL_ARG_NONNULL(1);

  /**
   * @brief Copy precomputed transaction data.
   *
   * @param[in] precomputed_txdata  Non-null.
   * @return                      The copied precomputed transaction data.
   */
  btck_PrecomputedTransactionData* btck_precomputed_transaction_data_copy(
      const btck_PrecomputedTransactionData* precomputed_txdata) BITCOINKERNEL_ARG_NONNULL(1);

  /**
   * Destroy the precomputed transaction data.
   */
  void btck_precomputed_transaction_data_destroy(btck_PrecomputedTransactionData* precomputed_txdata);
  ```

  The PR also modifies `btck_script_pubkey_verify` so that it accepts `precomputed_txdata` instead of `spent_outputs`:
  ```c
  /**
   * @brief Verify if the input at input_index of tx_to spends the script pubkey
   * under the constraints specified by flags. If the
   * `btck_ScriptVerificationFlags_WITNESS` flag is set in the flags bitfield, the
   * amount parameter is used. If the taproot flag is set, the precomputed data
   * must contain the spent outputs.
   *
   * @param[in] script_pubkey      Non-null, script pubkey to be spent.
   * @param[in] amount             Amount of the script pubkey's associated output. May be zero if
   *                               the witness flag is not set.
   * @param[in] tx_to              Non-null, transaction spending the script_pubkey.
   * @param[in] precomputed_txdata Nullable if the taproot flag is not set. Otherwise, precomputed data
   *                               for tx_to with the spent outputs must be provided.
   * @param[in] input_index        Index of the input in tx_to spending the script_pubkey.
   * @param[in] flags              Bitfield of btck_ScriptVerificationFlags controlling validation constraints.
   * @param[out] status            Nullable, will be set to an error code if the operation fails, or OK otherwise.
   * @return                       1 if the script is valid, 0 otherwise.
   */
  int btck_script_pubkey_verify(
      const btck_ScriptPubkey* script_pubkey,
      int64_t amount,
      const btck_Transaction* tx_to,
      const btck_PrecomputedTransactionData* precomputed_txdata,
      unsigned int input_index,
      btck_ScriptVerificationFlags flags,
      btck_ScriptVerifyStatus* status) BITCOINKERNEL_ARG_NONNULL(1, 3);
  ```

  As before, an error is thrown if the taproot flag is set and `spent_outputs` is not provided in `precomputed_txdata` (or `precomputed_txdata` is null). For simple single-input non-taproot verification, `precomputed_txdata` may be null, and the kernel will construct the precomputed data on-the-fly.

  Both the C++ wrapper and the test suite are updated with the new API. Tests cover both `precomputed_txdata` reuse and nullability.

  Appreciate feedback on this concept / approach!

ACKs for top commit:
  sedited:
    Re-ACK 44e006d438
  stringintech:
    ACK 44e006d

Tree-SHA512: 1ed435173e6ff4ec82bc603194cf182c685cb79f167439a442b9b179a32f6c189c358f04d4cb56d153fab04e3424a11b73c31680e42b87b8a6efcc3ccefc366c
2025-12-27 16:20:43 +00:00
merge-script
ec4ff99a22 Merge bitcoin/bitcoin#33892: policy: allow <minrelay txns in package context if paid for by cpfp
e44dec027c add release note about supporing non-TRUC <minrelay txns (Greg Sanders)
1488315d76 policy: Allow any transaction version with < minrelay (Greg Sanders)

Pull request description:

  Prior to cluster mempool, a policy was in place that
  disallowed non-TRUC transactions from being
  TX_RECONSIDERABLE in a package setting if it was below
  minrelay. This was meant to simplify reasoning about mempool
  trimming requirements with non-trivial transaction
  topologies in the mempool. This is no longer a concern
  post-cluster mempool, so this is relaxed.

  In effect, this makes 0-value parent transactions relayable
  through the network without the TRUC restrictions and
  thus the anti-pinning protections.

ACKs for top commit:
  ajtowns:
    ACK e44dec027c - lgtm
  ismaelsadeeq:
    ACK e44dec027c

Tree-SHA512: 6fd1a2429c55ca844d9bd669ea797e29eca3f544f0b5d3484743d3c1cdf4364f7c7a058aaf707bcfd94b84c621bea03228cb39487cbc23912b9e0980a1e5b451
2025-12-27 16:13:19 +00:00
Josh Doman
44e006d438 [kernel] Expose reusable PrecomputedTransactionData in script valid 2025-12-23 18:48:56 -05:00
fanquake
3e4765ee10 scripted-diff: [doc] Unify stale copyright headers
-BEGIN VERIFY SCRIPT-

sed --in-place --regexp-extended \
       's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' \
       $( git grep -l 'The Bitcoin Core developers' -- ':(exclude)COPYING' ':(exclude)src/ipc/libmultiprocess' ':(exclude)src/minisketch' )

-END VERIFY SCRIPT-
2025-12-19 16:58:36 +00:00
merge-script
7f295e1d9b Merge bitcoin/bitcoin#34084: scripted-diff: [doc] Unify stale copyright headers
fa4cb13b52 test: [doc] Manually unify stale headers (MarcoFalke)
fa5f297748 scripted-diff: [doc] Unify stale copyright headers (MarcoFalke)

Pull request description:

  Historically, the upper year range in file headers was bumped manually
  or with a script.

  This has many issues:

  * The script is causing churn. See for example commit 306ccd4, or
    drive-by first-time contributions bumping them one-by-one. (A few from
    this year: https://github.com/bitcoin/bitcoin/pull/32008,
    https://github.com/bitcoin/bitcoin/pull/31642,
    https://github.com/bitcoin/bitcoin/pull/32963, ...)
  * Some, or likely most, upper year values were wrong. Reasons for
    incorrect dates could be code moves, cherry-picks, or simply bugs in
    the script.
  * The upper range is not needed for anything.
  * Anyone who wants to find the initial file creation date, or file
    history, can use `git log` or `git blame` to get more accurate
    results.
  * Many places are already using the `-present` suffix, with the meaning
    that the upper range is omitted.

  To fix all issues, this bumps the upper range of the copyright headers
  to `-present`.

  Further notes:

  * Obviously, the yearly 4-line bump commit for the build system (c.f.
    b537a2c02a) is fine and will remain.
  * For new code, the date range can be fully omitted, as it is done
    already by some developers. Obviously, developers are free to pick
    whatever style they want. One can list the commits for each style.
  * For example, to list all commits that use `-present`:
    `git log --format='%an (%ae) [%h: %s]' -S 'present The Bitcoin'`.
  * Alternatively, to list all commits that use no range at all:
    `git log --format='%an (%ae) [%h: %s]' -S '(c) The Bitcoin'`.

  <!--
  * The lower range can be wrong as well, so it could be omitted as well,
    but this is left for a follow-up. A previous attempt was in
    https://github.com/bitcoin/bitcoin/pull/26817.

ACKs for top commit:
  l0rinc:
    ACK fa4cb13b52
  rkrux:
    re-ACK fa4cb13b52
  janb84:
    ACK fa4cb13b52

Tree-SHA512: e5132781bdc4417d1e2922809b27ef4cf0abb37ffb68c65aab8a5391d3c917b61a18928ec2ec2c75ef5184cb79a5b8c8290d63e949220dbeab3bd2c0dfbdc4c5
2025-12-19 16:56:02 +00:00
merge-script
5e7931af35 Merge bitcoin/bitcoin#34095: refactor: enable readability-container-contains clang-tidy rule
1e94e562f7 refactor: enable `readability-container-contains` clang-tidy rule (Lőrinc)
fd9f1accbd Fix compilation for old Boost versions (Lőrinc)

Pull request description:

  Replace the last few instances of `.count() != 0` and `.count() == 0` and bare `count()` patterns with the more expressive C++20 `.contains()` method:

  * `std::set<std::string>` in `getblocktemplate` RPC;
  * `std::map<std::string, ...>` in `transaction_tests`;
  * other bare `std::unordered_set` and `std::map` count calls.

  Also fixes https://github.com/bitcoin/bitcoin/issues/34101 by reverting `boost::multi_index::contains` calls not available in our minimum supported version.

  With no remaining violations, enable the `readability-container-contains` clang-tidy check to prevent future regressions.

  Follow-up to https://github.com/bitcoin/bitcoin/pull/33192

ACKs for top commit:
  hebasto:
    ACK 1e94e562f7.
  pablomartin4btc:
    re-ACK 1e94e562f7
  janb84:
    ACK 1e94e562f7
  rkrux:
    re-ACK 1e94e562f7

Tree-SHA512: d54a7821d319bf0d60b6c3a870917464a7d5b9279c6a86708c03a3516ec23bbf18f0e83de62b3b2b1607de96e1470f1144b4918d69a6c770e6b7e09863e7dbac
2025-12-19 15:55:18 +00:00
merge-script
74d6efe9c2 Merge bitcoin/bitcoin#34106: doc: add missing copyright headers
aeb7ccb937 doc: add missing copyright headers (fanquake)
68a7cb8f8b contrib: output copyright in generate-seeds.py (fanquake)

Pull request description:

  Takes care of some queries from #34084.

ACKs for top commit:
  rkrux:
    tACK aeb7ccb937
  janb84:
    ACK aeb7ccb937

Tree-SHA512: 12bb8fe58e0e84d4e1dcba94f95da2ebb0518208023459cb8ca81c1f95715749a6009cda8fe140dcde809fe35bdae10ee55b5eeea2cacfa30c58b1caefb2b521
2025-12-19 15:18:35 +00:00
Lőrinc
1e94e562f7 refactor: enable readability-container-contains clang-tidy rule
Replace the last few instances of `.count() != 0` and `.count() == 0` and `.count()` patterns with the more expressive C++20 `.contains()` method:

* `std::set<std::string>` in `getblocktemplate` RPC;
* `std::map<std::string, ...>` in `transaction_tests`;
* other bare `std::unordered_set` and `std::map` count calls.

With no remaining violations, enable the `readability-container-contains`
clang-tidy check to prevent future regressions.
2025-12-18 22:38:02 +01:00
Pieter Wuille
75bdb925f4 clusterlin: drop support for improvable chunking (simplification)
With MergeLinearizations() gone and the LIMO-based Linearize() replaced by SFL, we do not
need a class (LinearizationChunking) that can maintain an incrementally-improving chunk
set anymore.

Replace it with a function (ChunkLinearizationInfo) that just computes the chunks as
SetInfos once, and returns them as a vector. This simplifies several call sites too.
2025-12-18 16:01:31 -05:00
Pieter Wuille
91399a7912 clusterlin: remove unused MergeLinearizations (cleanup)
This ended up never being used in txgraph.
2025-12-18 16:01:31 -05:00
Pieter Wuille
13aad26b78 clusterlin: randomize various decisions in SFL (feature)
This introduces a local RNG inside the SFL state, which is used to randomize
various decisions inside the algorithm, in order to make it hard to create
pathological clusters which predictably have bad performance.

The decisions being randomized are:
* When deciding what chunk to attempt to split, the queue order is
  randomized.
* When deciding which dependency to split on, a uniformly random one is
  chosen among those with higher top feerate than bottom feerate within
  the chosen chunk.
* When deciding which chunks to merge, a uniformly random one among those
  with the higher feerate difference is picked.
* When merging two chunks, a uniformly random dependency between them is
  now activated.
* When making the state topological, the queue of chunks to process is
  randomized.
2025-12-18 16:01:31 -05:00
Pieter Wuille
ddbfa4dfac clusterlin: keep FIFO queue of improvable chunks (preparation)
This introduces a queue of chunks that still need processing, in both
MakeTopological() and OptimizationStep(). This is simultaneously:
* A preparation for introducing randomization, by allowing permuting the
  queue.
* An improvement to the fairness of suboptimal solutions, by distributing
  the work more fairly over chunks.
* An optimization, by avoiding retrying chunks over and over again which
  are already known to be optimal.
2025-12-18 16:01:31 -05:00
Pieter Wuille
3efc94d656 clusterlin: replace cluster linearization with SFL (feature)
This replaces the existing LIMO linearization algorithm (which internally uses
ancestor set finding and candidate set finding) with the much more performant
spanning-forest linearization algorithm.

This removes the old candidate-set search algorithm, and several of its tests,
benchmarks, and needed utility code.

The worst case time per cost is similar to the previous algorithm, so
ACCEPTABLE_ITERS is unchanged.
2025-12-18 16:01:31 -05:00
Pieter Wuille
6a8fa821b8 clusterlin: add support for loading existing linearization (feature) 2025-12-18 16:01:22 -05:00
Pieter Wuille
da48ed9f34 clusterlin: ReadLinearization for non-topological (tests)
Rather than using an ad-hoc no-dependency copy of the graph when a potentially
non-topological linearization is needed in the clusterlin fuzz test, add this
directly as a feature in ReadLinearization().

This is preparation for a later commit where another use for such a function
is added.
2025-12-18 15:49:07 -05:00
Pieter Wuille
c461259fb6 clusterlin: add class implementing SFL state (preparation)
This adds a data structure representing the optimization state for the spanning-forest
linearization algorithm (SFL), plus a fuzz test for its correctness.

This is preparation for switching over Linearize() to use this algorithm.

See https://delvingbitcoin.org/t/spanning-forest-cluster-linearization/1419 for
a description of the algorithm.
2025-12-18 15:49:01 -05:00
Pieter Wuille
86dd550a9b clusterlin: add known-correct optimal linearization tests (tests) 2025-12-18 14:17:28 -05:00
fanquake
aeb7ccb937 doc: add missing copyright headers 2025-12-18 16:28:13 +00:00
merge-script
516ae5ede4 Merge bitcoin/bitcoin#31533: fuzz: Add fuzz target for block index tree and related validation events
db2d39f642 fuzz: add subtest for re-downloading a previously pruned block (Eugene Siegel)
45f5b2dac3 fuzz: Add fuzzer for block index (Martin Zumsande)
c011e3aa54 test: Wrap validation functions with TestChainstateManager (Martin Zumsande)

Pull request description:

  This adds a fuzz target for the block index and various events in validation that interact with it.

  It can create arbitrary tree-like structure of block indexes, simulating (so far) the following events:
  - Adding a header
  - Receiving the full block (may be valid or not)
  - `ActivateBestChain()` - Reorging the chain to a new chain tip (possibly encountering invalid blocks on the way)
  - Pruning a block in the best chain
  - Receiving a previously pruned block again (`getblockfrompeer`)

  It might be interesting / possible to extend this to more events, such as dealing with more than one chainstate (assumeutxo).

  The test skips all actual validation of header/ block / transaction data by just simulating the outcome, and also doesn't interact with the data directory.
  The main goal is to ensure the integrity of the block index tree in all fuzzed constellations, by calling `CheckBlockIndex()` at the end of each iteration.

  Compared to #29158 this approach has a more limited scope (by skipping all actual validation), but it is fast - it doesn't do a full init sequence on each iteration, but "cleans up" after itself by resetting the global validation state after each iteration.

ACKs for top commit:
  Crypt-iQ:
    reACK db2d39f642
  maflcko:
    review ACK db2d39f642 🍶
  sedited:
    Re-ACK db2d39f642

Tree-SHA512: 76cd5f8f4d7d7258620b46d7438bad4508c3bdc98825b48b60f694b5a9838e2b2cf4967c0ead181f86f66f4939ddfe552471851b9d18f84f584c03dd7e09fc43
2025-12-18 15:26:42 +00:00
merge-script
80b1b5917d Merge bitcoin/bitcoin#34088: log: Use __func__ for -logsourcelocations
facd3d56cc log: Use `__func__` for -logsourcelocations (MarcoFalke)

Pull request description:

  The `-logsourcelocations` option was recently changed to print the full function signature, as a side-effect of moving toward `std::source_location` internally.

  This is fine, but at least for me, it makes debugging functional test failures harder, because the log is just so massively verbose, with questionable benefit.

  I think the historically used file name, line number, and plain `__func__` name are more than sufficient for `-logsourcelocations`.

  So switch back to using that.

  For reference, a verbose log may look like:

  ```
  ...
  node0 2025-12-17T07:28:37.528146Z [init] [checkqueue.h:147] [CCheckQueue<T, R>::CCheckQueue(unsigned int, int) [with T = CScriptCheck; R = std::pair<ScriptError_t, std::__cxx11::basic_string<char> >]] Script verificatio
  n uses 1 additional threads
  ...
  ```

  I don't think there is value in printing stuff, like the (anon) namespace, the class template args, or the functionn (template) args. The following should be more than sufficient:

  ```
  ...
  node0 2025-12-17T09:45:57.017122Z [init] [checkqueue.h:147] [CCheckQueue] Script verification uses 1 additional threads
  ...

ACKs for top commit:
  ajtowns:
    ACK facd3d56cc -- those long signatures are terrible
  stickies-v:
    ACK facd3d56cc

Tree-SHA512: 22fd1f0074fc6e85754967f9219659f57c905005a2bea9176f0b439abec324d7e6c2f875c8951934a3b11ef7e9d7e38d5d5d307e2bd1e000bc27ee85635cd668
2025-12-18 12:17:20 +00:00
merge-script
8d38b6f5f1 Merge bitcoin/bitcoin#34091: fuzz: doc: remove any mention to address_deserialize_v2
caf4843a59 fuzz: doc: remove any mention to address_deserialize_v2 (brunoerg)

Pull request description:

  We don't have `address_deserialize_v2` target anymore since fac81affb5 (we used to have `address_deserialize_v1_notime`, `address_deserialize_v1_withtime` and `address_deserialize_v2` but now we only have a single `address_deserialize` target) so it removes any mention to it.

ACKs for top commit:
  maflcko:
    review ACK caf4843a59 🎾
  marcofleon:
    ACK caf4843a59

Tree-SHA512: 539d69edbfe4ca11eb0701ed5c789ad81976e3e85e8a229e39e9dc1b1c72264f01d10a1c16d0a3bb4a354794412dc8b625298f4f72430905a00b65faeaa37d6b
2025-12-18 11:35:41 +00:00
Ryan Ofsky
ab513103df Merge bitcoin/bitcoin#33192: refactor: unify container presence checks
d9319b06cf refactor: unify container presence checks - non-trivial counts (Lőrinc)
039307554e refactor: unify container presence checks - trivial counts (Lőrinc)
8bb9219b63 refactor: unify container presence checks - find (Lőrinc)

Pull request description:

  ### Summary
  Instead of counting occurrences in sets and maps, the C++20 `::contains` method expresses the intent unambiguously and can return early on first encounter.

  ### Context
  Applied clang‑tidy's [readability‑container‑contains](https://clang.llvm.org/extra/clang-tidy/checks/readability/container-contains.html) check, though many cases required manual changes since tidy couldn't fix them automatically.

  ### Changes
  The changes made here were:

  | From                   | To               |
  |------------------------|------------------|
  | `m.find(k) == m.end()` | `!m.contains(k)` |
  | `m.find(k) != m.end()` | `m.contains(k)`  |
  | `m.count(k)`           | `m.contains(k)`  |
  | `!m.count(k)`          | `!m.contains(k)` |
  | `m.count(k) == 0`      | `!m.contains(k)` |
  | `m.count(k) != 1`      | `!m.contains(k)` |
  | `m.count(k) == 1`      | `m.contains(k)`  |
  | `m.count(k) < 1`       | `!m.contains(k)`  |
  | `m.count(k) > 0`       | `m.contains(k)`  |
  | `m.count(k) != 0`      | `m.contains(k)`  |

  > Note that `== 1`/`!= 1`/`< 1` only apply to simple [maps](https://en.cppreference.com/w/cpp/container/map/contains)/[sets](https://en.cppreference.com/w/cpp/container/set/contains) and had to be changed manually.

  There are many other cases that could have been changed, but we've reverted most of those to reduce conflict with other open PRs.

  -----

  <details>
  <summary>clang-tidy command on Mac</summary>

  ```bash
  rm -rfd build && \
  cmake -B build \
    -DCMAKE_C_COMPILER="$(brew --prefix llvm)/bin/clang" \
    -DCMAKE_CXX_COMPILER="$(brew --prefix llvm)/bin/clang++" \
    -DCMAKE_OSX_SYSROOT="$(xcrun --show-sdk-path)" \
    -DCMAKE_C_FLAGS="-target arm64-apple-macos11" \
    -DCMAKE_CXX_FLAGS="-target arm64-apple-macos11" \
    -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DBUILD_BENCH=ON -DBUILD_FUZZ_BINARY=ON -DBUILD_FOR_FUZZING=ON

   "$(brew --prefix llvm)/bin/run-clang-tidy" -quiet -p build -j$(nproc) -checks='-*,readability-container-contains' | grep -v 'clang-tidy'
  ```

  </details>

  Note: this is a take 2 of https://github.com/bitcoin/bitcoin/pull/33094 with fewer contentious changes.

ACKs for top commit:
  optout21:
    reACK d9319b06cf
  sedited:
    ACK d9319b06cf
  janb84:
    re ACK d9319b06cf
  pablomartin4btc:
    re-ACK d9319b06cf
  ryanofsky:
    Code review ACK d9319b06cf. I manually reviewed the full change, and it seems there are a lot of positive comments about this and no more very significant conflicts, so I will merge it shortly.

Tree-SHA512: e4415221676cfb88413ccc446e5f4369df7a55b6642347277667b973f515c3c8ee5bfa9ee0022479c8de945c89fbc9ff61bd8ba086e70f30298cbc1762610fe1
2025-12-17 16:17:29 -05:00
MarcoFalke
facd3d56cc log: Use __func__ for -logsourcelocations 2025-12-17 18:35:49 +01:00
merge-script
e5c600dc0e Merge bitcoin/bitcoin#34063: Make transaction_indentifier hex string constructor evaluated at comptime
5ac3579520 refactor: Add compile-time-checked hex txid (rustaceanrob)

Pull request description:

  Suggested by l0rinc as a comment in #34004.

  There are tests that utilize `FromHex` that will only fail during runtime if malformed. Adds a compile time constructor that can be caught by LSPs.

ACKs for top commit:
  l0rinc:
    ACK 5ac3579520
  maflcko:
    review ACK 5ac3579520 🦎
  rkrux:
    crACK 5ac3579520

Tree-SHA512: b0bae2bf0b8cd8c9a90765a14c46146313cf8b224a29d58a253e65ca95c4205c0beddea9c49ae58901e72c8c5202b91695d074ffb1c48e448d2e5606eb1bd5b4
2025-12-17 17:16:30 +00:00
brunoerg
caf4843a59 fuzz: doc: remove any mention to address_deserialize_v2 2025-12-17 11:57:11 -03:00
MarcoFalke
fa5f297748 scripted-diff: [doc] Unify stale copyright headers
-BEGIN VERIFY SCRIPT-

 sed --in-place --regexp-extended \
   's;( 20[0-2][0-9])(-20[0-2][0-9])? The Bitcoin Core developers;\1-present The Bitcoin Core developers;g' \
   $( git grep -l 'The Bitcoin Core developers' -- ':(exclude)COPYING' ':(exclude)src/ipc/libmultiprocess' ':(exclude)src/minisketch' )

-END VERIFY SCRIPT-
2025-12-16 22:21:15 +01:00
Eugene Siegel
db2d39f642 fuzz: add subtest for re-downloading a previously pruned block
This imitates the use of the getblockfrompeer rpc.
Note that currently pruning is limited to blocks in the active chain.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2025-12-16 11:25:46 -05:00
Martin Zumsande
45f5b2dac3 fuzz: Add fuzzer for block index
This fuzz target creates arbitrary tree-like structure of indices,
simulating the following events:
- Adding a header to the block tree db
- Receiving the full block (may be valid or not)
- Reorging to a new chain tip (possibly encountering invalid blocks on
  the way)
- pruning
The test skips all actual validation of header/ block / transaction data
by just simulating the outcome, and also doesn't interact with the data directory.

The main goal is to test the integrity of the block index tree in
all fuzzed constellations, by calling CheckBlockIndex()
at the end of each iteration.
2025-12-16 11:25:46 -05:00
Martin Zumsande
c011e3aa54 test: Wrap validation functions with TestChainstateManager
This allows to access them in the fuzz test in the next commit
without making them public.

Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2025-12-16 11:25:46 -05:00
merge-script
13891a8a68 Merge bitcoin/bitcoin#34050: fuzz: exercise ComputeMerkleRoot without mutated parameter
7e9de20c0c fuzz: exercise `ComputeMerkleRoot` without mutated parameter (Lőrinc)

Pull request description:

  The `mutated` parameter in `ComputeMerkleRoot` unlocks a different path that was always exercised in the fuzz test.
  Adjusted to be fuzzer to pass `nullptr` as well to make sure that path is also tested: 24ed820d4f/src/consensus/merkle.cpp (L49-L53)

  Follow-up to https://github.com/bitcoin/bitcoin/pull/33805#discussion_r2589073735

ACKs for top commit:
  frankomosh:
    ACK [7e9de20](7e9de20c0c)
  hodlinator:
    ACK 7e9de20c0c
  sedited:
    ACK 7e9de20c0c

Tree-SHA512: bf27029ac04003447b24a95544ec863f9ceca6c28d51ea811dd6ca2b412a2a780bb9fdbcdc82719f39dd710a746eb2446263e8377d67a8be52a1694571d03498
2025-12-16 14:25:55 +00:00
merge-script
4f11ef058b Merge bitcoin/bitcoin#30214: refactor: Improve assumeutxo state representation
82be652e40 doc: Improve ChainstateManager documentation, use consistent terms (Ryan Ofsky)
af455dcb39 refactor: Simplify pruning functions (TheCharlatan)
ae85c495f1 refactor: Delete ChainstateManager::GetAll() method (Ryan Ofsky)
6a572dbda9 refactor: Add ChainstateManager::ActivateBestChains() method (Ryan Ofsky)
491d827d52 refactor: Add ChainstateManager::m_chainstates member (Ryan Ofsky)
e514fe6116 refactor: Delete ChainstateManager::SnapshotBlockhash() method (Ryan Ofsky)
ee35250683 refactor: Delete ChainstateManager::IsSnapshotValidated() method (Ryan Ofsky)
d9e82299fc refactor: Delete ChainstateManager::IsSnapshotActive() method (Ryan Ofsky)
4dfe383912 refactor: Convert ChainstateRole enum to struct (Ryan Ofsky)
352ad27fc1 refactor: Add ChainstateManager::ValidatedChainstate() method (Ryan Ofsky)
a229cb9477 refactor: Add ChainstateManager::CurrentChainstate() method (Ryan Ofsky)
a9b7f5614c refactor: Add Chainstate::StoragePath() method (Ryan Ofsky)
840bd2ef23 refactor: Pass chainstate parameters to MaybeCompleteSnapshotValidation (Ryan Ofsky)
1598a15aed refactor: Deduplicate Chainstate activation code (Ryan Ofsky)
9fe927b6d6 refactor: Add Chainstate m_assumeutxo and m_target_utxohash members (Ryan Ofsky)
6082c84713 refactor: Add Chainstate::m_target_blockhash member (Ryan Ofsky)
de00e87548 test: Fix broken chainstatemanager_snapshot_init check (Ryan Ofsky)

Pull request description:

  This PR contains the first part of #28608, which tries to make assumeutxo code more maintainable, and improve it by not locking `cs_main` for a long time when the snapshot block is connected, and by deleting the snapshot validation chainstate when it is no longer used, instead of waiting until the next restart.

  The changes in this PR are just refactoring. They make `Chainstate` objects self-contained, so for example, it is possible to determine what blocks to connect to a chainstate without querying `ChainstateManager`, and to determine whether a Chainstate is validated without basing it on inferences like `&cs != &ActiveChainstate()` or `GetAll().size() == 1`.

  The PR also tries to make assumeutxo terminology less confusing, using "current chainstate" to refer to the chainstate targeting the current network tip, and "historical chainstate" to refer to the chainstate downloading old blocks and validating the assumeutxo snapshot. It removes uses of the terms "active chainstate," "usable chainstate," "disabled chainstate," "ibd chainstate," and "snapshot chainstate" which are confusing for various reasons.

ACKs for top commit:
  maflcko:
    re-review ACK 82be652e40 🕍
  fjahr:
    re-ACK 82be652e40
  sedited:
    Re-ACK 82be652e40

Tree-SHA512: 81c67abba9fc5bb170e32b7bf8a1e4f7b5592315b4ef720be916d5f1f5a7088c0c59cfb697744dd385552f58aa31ee36176bae6a6e465723e65861089a1252e5
2025-12-16 14:03:34 +00:00
merge-script
2210feb446 Merge bitcoin/bitcoin#34051: log: Remove brittle and confusing LogPrintLevel
fa8a5d215c log: Remove brittle and confusing LogPrintLevel (MarcoFalke)
fac24bbec8 test: Clarify logging_SeverityLevels test (MarcoFalke)
f273167661 ipc: separate log statements per level (stickies-v)
94c51ae540 libevent: separate log statements per level (stickies-v)

Pull request description:

  `LogPrintLevel` has many issues:

  * It encourages to log several levels in one source location. This is problematic, because all levels (even warnings and errors) will be rate limited equally for the same location.
  * Its warning and error logs are specially formatted compared to all other warning and error logs in the codebase, making them harder to spot (both in the debug log and in the code).
  * It is verbose to type and read.
  * It is confusing, because the majority of code uses the `Log$LEVEL(...)` macros. Having less ways to achieve the same makes the code more consistent and easier to review.

  Fix all issues by removing it

ACKs for top commit:
  stickies-v:
    re-ACK fa8a5d215c
  ajtowns:
    ACK fa8a5d215c
  pablomartin4btc:
    re-ACK fa8a5d215c

Tree-SHA512: 9fbb04962d9c26e566338694a7725b3c0e88ef733322d890bcc6aeddb45266c754e7c885c69bbfebd1588cc09912c6784cfc00e69882f1271a8c87d201490478
2025-12-14 12:30:48 +00:00
rustaceanrob
5ac3579520 refactor: Add compile-time-checked hex txid
Suggested by @l0rinc in #34004

Message by @l0rinc:

This adds a consteval constructor to transaction_identifier (Txid/Wtxid) to allow parsing hex strings at compile-time.
This replaces runtime FromHex checks in tests, ensuring that malformed hardcoded hashes cause build failures rather than runtime test failures.

Test variables are explicitly marked constexpr. This is required to workaround a regression in GCC 14 (Bug 117501) where the compiler incorrectly flags consteval initialization of non-constexpr variables as "statements with no effect".

GCC Bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=117501
Reproducer: https://godbolt.org/z/xb5TMaPs6

Co-authored-by: l0rinc <pap.lorinc@gmail.com>
2025-12-13 18:23:48 +00:00
MarcoFalke
fa8a5d215c log: Remove brittle and confusing LogPrintLevel 2025-12-13 13:43:24 +01:00
MarcoFalke
fac24bbec8 test: Clarify logging_SeverityLevels test
The test was a bit confusing, because it just referred to the "global
log level" without explicitly specifying what it is. The level is set
though the LogSetup constructor. However, it is easier to follow unit
tests, if they are self-contained. So just set the level to Debug
explicitly here.

Also, add a new debug_3 log, to further document the intended behavior
of the unit test.

Also, replace the LogPrintLevel with the shorter and exact replacements
LogTrace and LogDebug.
2025-12-13 12:50:12 +01:00
marcofleon
a70a14a3f4 refactor: Separate out logic for building a tree-shaped dependency graph 2025-12-12 16:09:53 +01:00
marcofleon
ce29d7d626 fuzz: Fix variable in clusterlin_postlinearize_tree check
The test intends to verify that running `PostLinearize` a
second time on a tree-structured graph doesn't change the
result. But `PostLinearize` was being called on the original
variable, not the copy. So the check was comparing the
unmodified copy against itself, which is useless.

Fix by post-linearizing the correct variable.
2025-12-12 15:04:10 +00:00
marcofleon
876e2849b4 fuzz: Fix incorrect loop bounds in clusterlin_postlinearize_tree
The dependency graphs generated by this test can have holes
(unused indices) in them. This means some of the transactions
were skipped when using `depgraph_gen.TxCount()` as the upper
bound of the loop. Switch to using `depgraph.Positions()` to
correctly handle sparse graphs.
2025-12-12 15:02:26 +00:00
merge-script
938d7aacab Merge bitcoin/bitcoin#33657: rest: allow reading partial block data from storage
07135290c1 rest: allow reading partial block data from storage (Roman Zeyde)
4e2af1c065 blockstorage: allow reading partial block data from storage (Roman Zeyde)
f2fd1aa21c blockstorage: return an error code from `ReadRawBlock()` (Roman Zeyde)

Pull request description:

  It allows fetching specific transactions using an external index, following https://github.com/bitcoin/bitcoin/pull/32541#issuecomment-3267485313.

  Currently, electrs and other indexers map between an address/scripthash to the list of the relevant transactions.

  However, in order to fetch those transactions from bitcoind, electrs relies on reading the whole block and post-filtering for a specific transaction[^1]. Other indexers use a `txindex` to fetch a transaction using its txid [^2][^3][^4].

  The above approach has significant storage and CPU overhead, since the `txid` is a pseudo-random 32-byte value. Also, mainnet `txindex` takes ~60GB today.

  This PR is adding support for using the transaction's position within its block to be able to fetch it directly using [REST API](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md), using the following HTTP request:

  ```
  GET /rest/blockpart/BLOCKHASH.bin?offset=OFFSET&size=SIZE
  ```

  - The offsets' index can be encoded much more efficiently ([~1.3GB today](https://github.com/romanz/bindex-rs/pull/66#issuecomment-3508476436)).

  - Address history query performance can be tested on mainnet using [1BitcoinEaterAddressDontSendf59kuE](https://mempool.space/address/1BitcoinEaterAddressDontSendf59kuE) - assuming warm OS block cache, [it takes <1s to fetch 5200 txs, i.e. <0.2ms per tx](https://github.com/romanz/bindex-rs/pull/66#issuecomment-3508476436) with [bindex](https://github.com/romanz/bindex-rs).

  - Only binary and hex response formats are supported.

  [^1]: https://github.com/romanz/electrs/blob/master/doc/schema.md
  [^2]: https://github.com/Blockstream/electrs/blob/new-index/doc/schema.md#txstore
  [^3]: https://github.com/spesmilo/electrumx/blob/master/docs/HOWTO.rst#prerequisites
  [^4]: https://github.com/cculianu/Fulcrum/blob/master/README.md#requirements

ACKs for top commit:
  maflcko:
    review ACK 07135290c1 🏪
  l0rinc:
    ACK 07135290c1
  hodlinator:
    re-ACK 07135290c1

Tree-SHA512: bcce7bf4b9a3e5e920ab5a83e656f50d5d7840cdde6b7147d329cf578f8a2db555fc1aa5334e8ee64d5630d25839ece77a2cf421c6c3ac1fa379bb453163bd4f
2025-12-12 13:22:00 +00:00
merge-script
597b8be223 Merge bitcoin/bitcoin#34025: net: Waste less time in socket handling
5f5c1ea019 net: Cache -capturemessages setting (Anthony Towns)
cea443e246 net: Pass time to InactivityChecks fuctions (Anthony Towns)

Pull request description:

  Cuts out some wasted time in net socket handling. First, only calculates the current time once every 50ms, rather than once for each peer, which given we only care about second-level precision seems more than adequate. Second, caches the value of the `-capturemessages` setting in `CConnman` rather than re-evaluating it every time we invoke `PushMessaage`.

ACKs for top commit:
  maflcko:
    review ACK 5f5c1ea019 🏣
  vasild:
    ACK 5f5c1ea019
  sedited:
    ACK 5f5c1ea019
  mzumsande:
    ACK 5f5c1ea019

Tree-SHA512: 0194143a3a4481c6355ac9eab27ce6ae4bed5db1d483ba5d06288dd92f195ccb9f0f055a9eb9d7e16e9bbf72f145eca1ff17c6700ee9aa42730103a8f047b32c
2025-12-12 10:49:59 +00:00
Ryan Ofsky
ae85c495f1 refactor: Delete ChainstateManager::GetAll() method
Just use m_chainstates array instead.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
e514fe6116 refactor: Delete ChainstateManager::SnapshotBlockhash() method
SnapshotBlockhash() is only called two places outside of tests, and is used
redundantly in some tests, checking the same field as other checks. Simplify by
dropping the method and using the m_from_snapshot_blockhash field directly.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
ee35250683 refactor: Delete ChainstateManager::IsSnapshotValidated() method
IsSnapshotValidated() is only called one place outside of tests, and is use
redundantly in some tests, asserting that a snapshot is not validated when a
snapshot chainstate does not even exist. Simplify by dropping the method and
checking Chainstate m_assumeutxo field directly.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
d9e82299fc refactor: Delete ChainstateManager::IsSnapshotActive() method
IsSnapshotActive() method is only called one place outside of tests and
asserts, and is confusing because it returns true even after the snapshot is
fully validated.

The documentation which said this "implies that a background validation
chainstate is also in use" is also incorrect, because after the snapshot is
validated, the background chainstate gets disabled and IsUsable() would return
false.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
4dfe383912 refactor: Convert ChainstateRole enum to struct
Change ChainstateRole parameter passed to wallets and indexes. Wallets and
indexes need to know whether chainstate is historical and whether it is fully
validated. They should not be aware of the assumeutxo snapshot validation
process.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
a9b7f5614c refactor: Add Chainstate::StoragePath() method
Use to simplify code determining the chainstate leveldb paths. New method is
the now the only code that needs to figure out the storage path, so the path
doesn't need to be constructed multiple places and backed out of leveldb.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
840bd2ef23 refactor: Pass chainstate parameters to MaybeCompleteSnapshotValidation
Remove hardcoded references to m_ibd_chainstate and m_snapshot_chainstate so
MaybeCompleteSnapshotValidation function can be simpler and focus on validating
the snapshot without dealing with internal ChainstateManager states.

This is a step towards being able to validate the snapshot outside of
ActivateBestChain loop so cs_main is not locked for minutes when the snapshot
block is connected.
2025-12-12 06:49:59 -04:00
Ryan Ofsky
1598a15aed refactor: Deduplicate Chainstate activation code
Move duplicate code from ChainstateManager::ActivateSnapshot and
ChainstateManager::ActivateExistingSnapshot methods to a new
ChainstateManager::AddChainstate method.

The "AddChainstate" method name doesn't mention snapshots even though it is
only used to add snapshot chainstates now, because it becomes more generalized
in a later commit in this PR ("refactor: Add ChainstateManager::m_chainstates
member")
2025-12-12 06:49:59 -04:00