Commit Graph

29261 Commits

Author SHA1 Message Date
TheCharlatan
f5d5d1213c kernel: Add function to read block undo data from disk to C header
This adds functions for reading the undo data from disk with a retrieved
block tree entry. The undo data of a block contains all the spent
script pubkeys of all the transactions in a block. For ease of
understanding the undo data is renamed to spent outputs with seperate
data structures exposed for a block's and a transaction's spent outputs.

In normal operations undo data is used during re-orgs. This data might
also be useful for building external indexes, or to scan for silent
payment transactions.

Internally the block undo data contains a vector of transaction undo
data which contains a vector of the coins consumed. The coins are all
int the order of the transaction inputs of the consuming transactions.
Each coin can be used to retrieve a transaction output and in turn a
script pubkey and amount.

This translates to the three-level hierarchy the api provides: Block
spent outputs contain transaction spent outputs, which contain
individual coins. Each coin includes the associated output, the height
of the block is contained in, and whether it is from a coinbase
transaction.
2025-11-04 08:32:10 +01:00
TheCharlatan
09d0f62638 kernel: Add functions to read block from disk to C header
This adds functions for reading a block from disk with a retrieved block
tree entry. External services that wish to build their own index, or
analyze blocks can use this to retrieve block data.

The block tree can now be traversed from the tip backwards. This is
guaranteed to work, since the chainstate maintains an internal block
tree index in memory and every block (besides the genesis) has an
ancestor.

The user can use this function to iterate through all blocks in the
chain (starting from the tip). The tip is retrieved from a separate
`Chain` object, which allows distinguishing whether entries are
currently in the best chain. Once the block tree entry for the genesis
block is reached a nullptr is returned if the user attempts to get the
previous entry.
2025-11-04 08:32:09 +01:00
TheCharlatan
a263a4caf2 kernel: Add function for copying block data to C header
This adds a function for streaming bytes into a user-owned data
structure.

Use it in the tests for verifying the implementation of the validation
interface's `BlockChecked` method.
2025-11-04 08:32:08 +01:00
TheCharlatan
b30e15f432 kernel: Add functions for the block validation state to C header
These allow for the interpretation of the data in a `BlockChecked`
validation interface callback. The validation state passed through
`BlockChecked` is the source of truth for the validity of a block (the
mode). It is
also useful to get richer information in case a block failed to
validate (the result).
2025-11-04 08:32:07 +01:00
TheCharlatan
aa262da7bc kernel: Add validation interface to C header
This adds the infrastructure required to process validation events. For
now the external validation interface only has support for the
`BlockChecked` , `NewPoWValidBlock`, `BlockConnected`, and
`BlockDisconnected` callback. Support for the other internal
validation interface methods can be added in the future.

The validation interface follows an architecture for defining its
callbacks and ownership that is similar to the notifications.

The task runner is created internally with a context, which itself
internally creates a unique ValidationSignals object. When the user
creates a new chainstate manager the validation signals are internally
passed to the chainstate manager through the context.

A validation interface can register for validation events with a
context. Internally the passed in validation interface is registerd with
the validation signals of a context.

The callbacks block any further validation execution when they are
called. It is up to the user to either multiplex them, or use them
otherwise in a multithreaded mechanism to make processing the validation
events non-blocking.

I.e. for a synchronous mechanism, the user executes instructions
directly at the end of the callback function:

```mermaid
sequenceDiagram
    participant V as Validation
    participant C as Callback
    V->>C: Call callback
    Note over C: Process event (blocks)
    C-->>V: Return
    Note over V: Validation resumes

```

To avoid blocking, the user can submit the data to e.g. a worker thread
or event manager, so processing happens asynchronously:

```mermaid
sequenceDiagram
    participant V as Validation
    participant C as Callback
    participant W as Worker Thread
    V->>C: Call callback
    C->>W: Submit to worker thread
    C-->>V: Return immediately
    Note over V: Validation continues
    Note over W: Process event async
```
2025-11-04 08:32:06 +01:00
TheCharlatan
d27e27758d kernel: Add interrupt function to C header
Calling interrupt can halt long-running functions associated with
objects that were created through the passed-in context.
2025-11-04 08:32:06 +01:00
TheCharlatan
1976b13be9 kernel: Add import blocks function to C header
Add `btck_import_blocks` to import block data and rebuild indexes. The
function can either reindex all existing block files if the indexes were
previously wiped through the chainstate manager options, or import
blocks from specified file paths.
2025-11-04 08:32:05 +01:00
TheCharlatan
a747ca1f51 kernel: Add chainstate load options for in-memory dbs in C header
This allows a user to run the kernel without creating on-disk files for
the block tree and chainstate indexes. This is potentially useful in
scenarios where the user needs to do some ephemeral validation
operations.

One specific use case is when linearizing the blocks on disk. The block
files store blocks out of order, so a program may utilize the library
and its header to read the blocks with one chainstate manager, and then
write them back in order, and without orphans, with another chainstate
maanger. To save disk resources and if the indexes are not required once
done, it may be beneficial to keep the indexes in memory for the
chainstate manager that writes the blocks back again.
2025-11-04 08:32:04 +01:00
TheCharlatan
070e77732c kernel: Add options for reindexing in C header
Adds options for wiping the chainstate and block tree indexes to the
chainstate manager options. In combination and once the
`*_import_blocks(...)` function is added in a later commit, this
triggers a reindex. For now, it just wipes the existing data.
2025-11-04 08:32:03 +01:00
TheCharlatan
ad80abc73d kernel: Add block validation to C header
The added function allows the user process and validate a given block
with the chainstate manager. The *_process_block(...) function does some
preliminary checks on the block before passing it to
`ProcessNewBlock(...)`. These are similar to the checks in the
`submitblock()` rpc.

Richer processing of the block validation result will be made available
in the following commits through the validation interface.

The commits also adds a utility for deserializing a `CBlock`
(`kernel_block_create()`) that may then be passed to the library for
processing.

The tests exercise the function for both mainnet and regtest. The
commit also adds the data of 206 regtest blocks (some blocks also
contain transactions).
2025-11-04 08:32:02 +01:00
TheCharlatan
cb1590b05e kernel: Add chainstate loading when instantiating a ChainstateManager
The library will now internally load the chainstate when a new
ChainstateManager is instantiated.

Options for controlling details of loading the chainstate will be added
over the next few commits.
2025-11-04 08:32:01 +01:00
TheCharlatan
e2c1bd3d71 kernel: Add chainstate manager option for setting worker threads
Re-use the same pattern used for the context options. This allows users
to set the number of threads used in the validation thread pool.
2025-11-04 08:32:00 +01:00
TheCharlatan
65571c36a2 kernel: Add chainstate manager object to C header
This is the main driver class for anything validation related, so expose
it here.

Creating the chainstate manager options will currently also trigger the
creation of their respectively configured directories.

The chainstate manager and block manager options are consolidated into a
single object. The kernel might eventually introduce a separate block
manager object for the purposes of being a light-weight block store
reader.

The chainstate manager will associate with the context with which it was
created for the duration of its lifetime and it keeps it in memory with
a shared pointer.

The tests now also create dedicated temporary directories. This is
similar to the behaviour in the existing unit test framework.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2025-11-04 08:31:59 +01:00
TheCharlatan
c62f657ba3 kernel: Add notifications context option to C header
The notifications are used for notifying on connected blocks and on
warning and fatal error conditions.

The user of the C header may define callbacks that gets passed to the
internal notification object in the
`kernel_NotificationInterfaceCallbacks` struct.

Each of the callbacks take a `user_data` argument that gets populated
from the `user_data` value in the struct. It can be used to recreate the
structure containing the callbacks on the user's side, or to give the
callbacks additional contextual information.
2025-11-04 08:31:58 +01:00
TheCharlatan
9e1bac4585 kernel: Add chain params context option to C header
As a first option, add the chainparams. For now these can only be
instantiated with default values. In future they may be expanded to take
their own options for regtest and signet configurations.

This commit also introduces a unique pattern for setting the option
values when calling the `*_set(...)` function.
2025-11-04 08:31:58 +01:00
TheCharlatan
337ea860df kernel: Add kernel library context object
The context introduced here holds the objects that will be required for
running validation tasks, such as the chosen chain parameters, callbacks
for validation events, and interrupt handling. These will be used by the
chainstate manager introduced in subsequent commits.

This commit also introduces conventions for defining option objects. A
common pattern throughout the C header will be:
```
options = object_option_create();
object = object_create(options);
```
This allows for more consistent usage of a "builder pattern" for
objects where options can be configured independently from
instantiation.
2025-11-04 08:31:57 +01:00
TheCharlatan
28d679bad9 kernel: Add logging to kernel library C header
Exposing logging in the kernel library allows users to follow
operations. Users of the C header can use
`kernel_logging_connection_create(...)` to pass a callback function to
Bitcoin Core's internal logger. Additionally the level and category can
be globally configured.

By default, the logger buffers messages until
`kernel_loggin_connection_create(...)` is called. If the user does not
want any logging messages, it is recommended that
`kernel_disable_logging()` is called, which permanently disables the
logging and any buffering of messages.

Co-authored-by: stringintech <stringintech@gmail.com>
2025-11-04 08:31:56 +01:00
TheCharlatan
2cf136dec4 kernel: Introduce initial kernel C header API
As a first step, implement the equivalent of what was implemented in the
now deprecated libbitcoinconsensus header. Also add a test binary to
exercise the header and library.

Unlike the deprecated libbitcoinconsensus the kernel library can now use
the hardware-accelerated sha256 implementations thanks for its
statically-initialzed context. The functions kept around for
backwards-compatibility in the libbitcoinconsensus header are not ported
over. As a new header, it should not be burdened by previous
implementations. Also add a new error code for handling invalid flag
combinations, which would otherwise cause a crash.

The macros used in the new C header were adapted from the libsecp256k1
header.

To make use of the C header from C++ code, a C++ header is also
introduced for wrapping the C header. This makes it safer and easier to
use from C++ code.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2025-11-04 08:31:51 +01:00
Hennadii Stepanov
3bb30658e6 Merge bitcoin/bitcoin#32380: Modernize use of UTF-8 in Windows code
53e4951a5b Switch to ANSI Windows API in `fsbridge::fopen()` function (Hennadii Stepanov)
dbe770d921 Switch to ANSI Windows API in `Win32ErrorString()` function (Hennadii Stepanov)
06d0be4e22 Remove no longer necessary `WinCmdLineArgs` class (Hennadii Stepanov)
f366408492 cmake: Set process code page to UTF-8 on Windows (Hennadii Stepanov)
dccbb17806 Set minimum supported Windows version to 1903 (May 2019 Update) (Hennadii Stepanov)

Pull request description:

  The main goal is to remove [deprecated](https://github.com/bitcoin/bitcoin/issues/32361) code (removed in C++26).

  This PR employs Microsoft's modern [approach](https://learn.microsoft.com/en-us/windows/apps/design/globalizing/use-utf8-code-page) to handling UTF-8:
  > Until recently, Windows has emphasized "Unicode" -W variants over -A APIs. However, recent releases have used the ANSI code page and -A APIs as a means to introduce UTF-8 support to apps. If the ANSI code page is configured for UTF-8, then -A APIs typically operate in UTF-8. This model has the benefit of supporting existing code built with -A APIs without any code changes.

  TODO:
  - [x] Handle application manifests properly when building with MSVC.
  - [x] Bump the minimum supported Windows version to 1903 (May 2019 Update).
  - [x] Remove all remaining use cases of the deprecated `std:wstring_convert`.
      - The instance in `subprocess.h` will be addressed in a follow-up PR, as additional tests are likely needed.
      - The usage in `common/system.cpp` is handled in https://github.com/bitcoin/bitcoin/pull/32566.

  Resolves partially https://github.com/bitcoin/bitcoin/issues/32361.

ACKs for top commit:
  laanwj:
    re-ACK 53e4951a5b
  hodlinator:
    re-ACK 53e4951a5b
  davidgumberg:
    untested crACK 53e4951a5b

Tree-SHA512: 0dbe9badca8b979ac2b4814fea6e4a7e53c423a1c96cb76ce894253137d3640a87631a5b22b9645e8f0c2a36a107122eb19ed8e92978c17384ffa8b9ab9993b5
2025-10-28 22:41:07 +00:00
merge-script
1abc8fa308 Merge bitcoin/bitcoin#33218: refactor: rename fees.{h,cpp} to fees/block_policy_estimator.{h,cpp}
1a7fb5eeee fees: return current block height in estimateSmartFee (ismaelsadeeq)
ab49480d9b fees: rename fees_args to block_policy_estimator_args (ismaelsadeeq)
06db08a435 fees: refactor: rename fees to block_policy_estimator (ismaelsadeeq)
6dfdd7e034 fees: refactor: rename policy_fee_tests.cpp to feerounder_tests.cpp (ismaelsadeeq)

Pull request description:

  This PR is a simple refactoring that does four things:

  1. Renames `test/policy_fee_tests.cpp` to `test/feerounder_tests.cpp`.
  2. Renames `policy/fees.{h,cpp}` to `policy/fees/block_policy_estimator.{h,cpp}`.
  3. Renames `policy/fees_args.cpp` to `policy/fees/block_policy_estimator_args.cpp`.
  4. Modifies `estimateSmartFee` to return the block height at which the estimate was made by adding a `best_height` unsigned int value to the `FeeCalculation` struct.

  **Motivation**

  In preparation for adding a new fee estimator, the `fees` directory is created so we can organize code into `block_policy_estimator` and `mempool` because

  a) It would be clunky to add more code directly under `fees`.
  b) Having `policy/fees.{h,cpp}` and `policy/mempool.{h,cpp}` would also be undesirable.

  Therefore, it makes sense to structure the it as `policy/fees/block_policy_estimator`, `policy/fees/mempool`, etc.
  Hence test file were also updated accordingly.

  The current block height is also returned because later in #30157 we log the height at which each estimate is made (at the debug log category of  fee estimation :) ). This feature is particularly useful for empirical data analysis.

ACKs for top commit:
  maflcko:
    re-ACK 1a7fb5eeee 🐤
  polespinasa:
    re ACK 1a7fb5eeee
  willcl-ark:
    ACK 1a7fb5eeee
  janb84:
    re ACK 1a7fb5eeee

Tree-SHA512: fef7ace2a9f262ec0361fb7a46df5108afc46b5c4b059caadf2fd114740aefbb2592389d11646c13d0e28bf0ef2cfcfbab3e659c4d4288b8ebe64725fd1963c0
2025-10-28 11:57:59 -04:00
Hennadii Stepanov
24434c1284 Merge bitcoin/bitcoin#31308: ci, iwyu: Treat warnings as errors for specific directories
02d2b5a11c ci, iwyu: Treat warnings as errors for specific directories (Hennadii Stepanov)
57a3eac387 refactor: Fix includes in `index` directory (Hennadii Stepanov)
bdb8eadcdc refactor: Fix includes in `crypto` directory (Hennadii Stepanov)
56f2a689a2 ci: Do not patch `leveldb` to workaround UB in "tidy" CI job (Hennadii Stepanov)

Pull request description:

  This PR is the first step towards treating IWYU warnings as errors. At this stage, it applies only to the `crypto` and `index` directories.

ACKs for top commit:
  maflcko:
    re-ACK 02d2b5a11c 💮
  ryanofsky:
    Code review ACK 02d2b5a11c. Just rebased and update tidy patch comment again since last review
  willcl-ark:
    ACK 02d2b5a11c

Tree-SHA512: 1c966e01c47bf3e7d225faa3b819367f757430e2d71e9582fa82d67307aabe3f0d76f69346ee180192e7f5ab194ecc58d2b8ecf178eab26ba3309a6b55bff4b6
2025-10-28 11:52:27 +00:00
Ava Chow
80bb7012be Merge bitcoin/bitcoin#31514: wallet: allow label for non-ranged external descriptor (if internal=false) & disallow label for ranged descriptors
664657ed13 bugfix: disallow label for ranged descriptors & allow external non-ranged descriptors to have label (scgbckbone)

Pull request description:

  Motivation:
  * ranged descriptors MUST not be able to have label (current impl allows it)
  * external non-ranged descriptor MUST be able to have label (current impl disallows it, **if** `internal=false` is provided via importdescriptor user data)

  Repro steps:
  * create blank wallet and import descriptors
  * external has `label=test` (not internal)
  ```
      conn = bitcoind.create_wallet(wallet_name=w_name, disable_private_keys=True, blank=True,
                                    passphrase=None, avoid_reuse=False, descriptors=True)
      descriptors = [
          {
              "timestamp": "now",
              "label": "test",
              "active": True,
              "desc": "wpkh([0f056943/84h/1h/0h]tpubDC7jGaaSE66Pn4dgtbAAstde4bCyhSUs4r3P8WhMVvPByvcRrzrwqSvpF9Ghx83Z1LfVugGRrSBko5UEKELCz9HoMv5qKmGq3fqnnbS5E9r/0/*)#erexmnep",
              "internal": False
          },
          {
              "desc": "wpkh([0f056943/84h/1h/0h]tpubDC7jGaaSE66Pn4dgtbAAstde4bCyhSUs4r3P8WhMVvPByvcRrzrwqSvpF9Ghx83Z1LfVugGRrSBko5UEKELCz9HoMv5qKmGq3fqnnbS5E9r/1/*)#ghu8xxfe",
              "active": True,
              "internal": True,
              "timestamp": "now"
          },
      ]
      r = conn.importdescriptors(descriptors)
      print(r)
  ```
  response:
  ```
  [{'error': {'code': -8,
              'message': 'Internal addresses should not have a label'},
    'success': False,
    'warnings': ['Range not given, using default keypool range']},
   {'success': True,
    'warnings': ['Range not given, using default keypool range']}]
  ```
  But in above, ONLY external has a label.

  If you remove `internal: False` from external descriptor import object - it will import no problem:
  ```
  [{'success': True,
    'warnings': ['Range not given, using default keypool range']},
   {'success': True,
    'warnings': ['Range not given, using default keypool range']}]

  ```
  Even tho it should NOT, as the descriptor is ranged. Current implementation relies on checking user provided data to decide whether desc is ranged.

ACKs for top commit:
  achow101:
    ACK 664657ed13
  rkrux:
    lgtm crACK 664657ed13

Tree-SHA512: 9e70aea620019c29950ba417d4ae38d65cd94a4f6fcabbc021d67b031de1c44c27d6f6f5cb7e6950a099eb6e58bed9be764d4c6347195daeccb14a5d95c123b2
2025-10-27 13:56:45 -07:00
merge-script
5e1f626ac3 Merge bitcoin/bitcoin#32504: test: descriptor: cover invalid multi/multi_a cases
58e55b17e6 test: descriptor: cover invalid multi/multi_a cases (brunoerg)

Pull request description:

  This PR adds test coverage for invalid `multi()` and `multi_a()` cases, see:

  1. 53eb5593f0/src/script/descriptor.cpp (L1819-L1821)

  2.  53eb5593f0/src/script/descriptor.cpp (L1835-L1837)

  3. 53eb5593f0/src/script/descriptor.cpp (L1838-L1840)

  We could also exercise to exceed the number of keys - 20 for `multi` and 999 for `multi_a`.

ACKs for top commit:
  maflcko:
    lgtm ACK 58e55b17e6
  darosior:
    utACK 58e55b17e6
  glozow:
    ACK 58e55b17e6

Tree-SHA512: 0983e9c70e4bef13fa21b2e22e17c2e86eda0950f6271a42b24b91eef22c3277659a862a78bd511c9e14c92859070b3bf2968cfa24de0a1397de1f824946c757
2025-10-27 15:39:36 -04:00
merge-script
56e9703968 Merge bitcoin/bitcoin#29640: Fix tiebreak when loading blocks from disk (and add tests for comparing chain ties)
0465574c12 test: Fixes send_blocks_and_test docs (Sergi Delgado Segura)
09c95f21e7 test: Adds block tiebreak over restarts tests (Sergi Delgado Segura)
18524b072e Make nSequenceId init value constants (Sergi Delgado Segura)
8b91883a23 Set the same best tip on restart if two candidates have the same work (Sergi Delgado Segura)
5370bed21e test: add functional test for complex reorgs (Pieter Wuille)
ab145cb3b4 Updates CBlockIndexWorkComparator outdated comment (Sergi Delgado Segura)

Pull request description:

  This PR grabs some interesting bits from https://github.com/bitcoin/bitcoin/pull/29284 and fixes some edge cases in how block tiebreaks are dealt with.

  ## Regarding #29284

  The main functionality from the PR was dropped given it was not an issue anymore, however, reviewers pointed out some comments were outdated https://github.com/bitcoin/bitcoin/pull/29284#discussion_r1522023578 (which to my understanding may have led to thinking that there was still an issue) it also added test coverage for the aforementioned case which was already passing on master and is useful to keep.

  ## New functionality

  While reviewing the superseded PR, it was noticed that blocks that are loaded from disk may face a similar issue (check https://github.com/bitcoin/bitcoin/pull/29284#issuecomment-1994317785 for more context).

  The issue comes from how tiebreaks for equal work blocks are handled: if two blocks have the same amount of work, the one that is activatable first wins, that is, the one for which we have all its data (and all of its ancestors'). The variable that keeps track of this, within `CBlockIndex` is `nSequenceId`, which is not persisted over restarts. This means that when a node is restarted, all blocks loaded from disk are defaulted the same `nSequenceId`: 0.
  Now, when trying to decide what chain is best on loading blocks from disk, the previous tiebreaker rule is not decisive anymore, so the `CBlockIndexWorkComparator` has to default to its last rule: whatever block is loaded first (has a smaller memory address).

  This means that if multiple same work tip candidates were available before restarting the node, it could be the case that the selected chain tip after restarting does not match the one before.

  Therefore, the way `nSequenceId` is initialized is changed to:

  - 0 for blocks that belong to the previously known best chain
  - 1 to all other blocks loaded from disk

ACKs for top commit:
  sipa:
    utACK 0465574c12
  TheCharlatan:
    ACK 0465574c12
  furszy:
    Tested ACK 0465574c12.

Tree-SHA512: 161da814da03ce10c34d27d79a315460a9c98d019b85ee35bc5daa991ed3b6a2e69a829e421fc70d093a83cf7a2e403763041e594df39ed1991445e54c16532a
2025-10-27 12:17:37 -04:00
merge-script
9bd9ec00b2 Merge bitcoin/bitcoin#33688: test: Update BIP324 test vectors
51877f2fc5 test: Update BIP324 test vectors (Tim Ruffing)

Pull request description:

  This updates the hardcoded test vectors from BIP324. The test vectors had to be regenerated (in the aux files of the BIP) because there was a bug in the script used for generating them (https://github.com/bitcoin/bips/pull/2016).

ACKs for top commit:
  jonatack:
    ACK 51877f2fc5
  theStack:
    ACK 51877f2fc5

Tree-SHA512: 59f4075e286067b11fce98667c860f3083b6cca8a2e49da8783ccdce8e32c34fd3e1943191d24dcf5bb68d8a2540726d99f7c29e8b0f104032ccb82423ca8d82
2025-10-27 11:30:49 +01:00
ismaelsadeeq
1a7fb5eeee fees: return current block height in estimateSmartFee 2025-10-27 10:44:18 +01:00
ismaelsadeeq
ab49480d9b fees: rename fees_args to block_policy_estimator_args
- Also move them to policy/fees/ and update includes
- Note: the block_policy_estimator_args.h include in block_policy_estimator_args.cpp was done manually.
2025-10-27 10:44:18 +01:00
ismaelsadeeq
06db08a435 fees: refactor: rename fees to block_policy_estimator
- Also move it to policy/fees and update the includes
2025-10-27 10:41:02 +01:00
ismaelsadeeq
6dfdd7e034 fees: refactor: rename policy_fee_tests.cpp to feerounder_tests.cpp
- Also remame the test suite name to match the new name.
2025-10-27 10:34:21 +01:00
Ava Chow
f54ffb4bc1 Merge bitcoin/bitcoin#32813: clang-format: make formatting deterministic for different formatter versions
13f36c020f clang-format: regenerate configs (Lőrinc)

Pull request description:

  Updates `.clang-format` file to reflect [latest supported Clang-Format standards](https://releases.llvm.org/16.0.0/tools/clang/docs/ClangFormatStyleOptions.html) while preserving most of the existing formatting behavior.

  Note that [`AfterStruct` brace placement](https://github.com/bitcoin/bitcoin/pull/32414#discussion_r2072678126) was originally aligned here with `AfterClass`, but was reverted by reviewer demand.

ACKs for top commit:
  maflcko:
    re-ACK 13f36c020f 🖼
  achow101:
    ACK 13f36c020f
  hodlinator:
    re-ACK 13f36c020f

Tree-SHA512: 02bd9d8a22a9af268297aeddd1f2b2cce079fddd0e1f764d6e9650bb614cb7bcfbd20b38d6e4e5db1744b3dd1ba540380010c085f2cbc0be8aa936f21d27d8de
2025-10-24 13:25:00 -07:00
Ava Chow
1916c51cd8 Merge bitcoin/bitcoin#33210: fuzz: enhance wallet_fees by mocking mempool stuff
5ded99a7f0 fuzz: MockMempoolMinFee in wallet_fees (brunoerg)
c9a7a198d9 test: move MockMempoolMinFee to util/txmempool (brunoerg)
adf67eb21b fuzz: create FeeEstimatorTestingSetup to set fee_estimator (brunoerg)
ff10a37e99 fuzz: mock CBlockPolicyEstimator in wallet_fuzz (brunoerg)
f591c3beca fees: make estimateSmartFee/HighestTargetTracked virtual for mocking (brunoerg)
19273d0705 fuzz: set mempool options in wallet_fees (brunoerg)

Pull request description:

  Some functions in `wallet/fees.cpp` (fuzzed by the wallet_fees target) depends on some mempool stuff - e.g. relay current min fee, smart fee and max blocks estimation, relay dust fee and other ones. For better fuzzing of it, it would be great to have these values/interactions. That said, this PR enhances the `wallet_fees` target by:

  - Setting mempool options - `min_relay_feerate`,  `dust_relay_feerate` and `incremental_relay_feerate` - when creating the `CTxMemPool`.
  - Creates a `ConsumeMempoolMinFee` function which is used to have a mempool min fee (similar approach from `MockMempoolMinFee` from unit test).
  - Mock `CBlockPolicyEstimator` - estimateSmartFee/HighestTagretTracket functions, especifically. It's better to mock it then trying to interact to CBlockPolicyEstimator in order to have some effective values due to performance.

  Note that I created `FeeEstimatorTestingSetup` because we cannot set `m_node.fee_estimator` in `ChainTestingSetup` since fae8c73d9e.

ACKs for top commit:
  maflcko:
    re-ACK 5ded99a7f0 🎯
  ismaelsadeeq:
    Code review ACK 5ded99a7f0

Tree-SHA512: 13d2af042098afd237ef349437021ea841069d93d4c3e3a32e1b562c027d00c727f375426709d34421092993398caf7ba8ff19077982cb6f470f8938a44e7754
2025-10-24 11:43:42 -07:00
Ava Chow
0eb554728c Merge bitcoin/bitcoin#33336: log: print every script verification state change
45bd891465 log: split assumevalid ancestry-failure-reason message (Lőrinc)
6c13a38ab5 log: separate script verification reasons (Lőrinc)
f2ea6f04e7 refactor: untangle assumevalid decision branches (Lőrinc)
9bc298556c validation: log initial script verification state (Lőrinc)
4fad4e992c test: add assumevalid scenarios scaffold (Lőrinc)
91ac64b0a6 log: reword `signature validations` to `script verification` in `assumevalid` log (Lőrinc)

Pull request description:

  ### Summary

  Users can encounter cases where script checks are unexpectedly enabled (e.g. after reindex, or when `assumevalid`/`minimumchainwork` gates fail). Without an explicit line, they must infer state from the absence of a message, which is incomplete and error-prone.
  The existing "Assuming ancestors of block …" line does not reliably indicate whether script checks are actually enabled, which makes debugging/benchmarking confusing.

  ### What this changes

  We make the initial **script-verification** state explicit and log **why** checks are enabled to avoid confusion.
  * Always log the first script-verification state on startup, **before** the first `UpdateTip`.
  * Flatten the nested `assumevalid` conditionals into a linear gating sequence for readability.
  * Extend the functional test to assert the old behavior with the new reason strings.

  This is a **logging-only** test change it shouldn't change any other behavior.

  ### Example output

  The state (with reason) is logged at startup and whenever the reason changes, e.g.:

  * `Disabling script verification at block #904336 (000000000000000000014106b2082b1a18aaf3091e8b337c6fed110db8c56620).`
  * `Enabling script verification at block #912527 (000000000000000000010bb6aa3ecabd7d41738463b6c6621776c2e40dbe738a): block too recent relative to best header.`
  * `Enabling script verification at block #912684 (00000000000000000001375cf7b90b2b86e559d05ed92ca764d376702ead3858): block height above assumevalid height.`

  ------

  Follow-up to https://github.com/bitcoin/bitcoin/pull/32975#discussion_r2329269037

ACKs for top commit:
  Eunovo:
    re-ACK 45bd891465
  achow101:
    ACK 45bd891465
  hodlinator:
    re-ACK 45bd891465
  yuvicc:
    ACK 45bd891465
  andrewtoth:
    ACK 45bd891465
  ajtowns:
    ACK 45bd891465

Tree-SHA512: 58328d7c418a6fe18f1c7fe1dd31955bb6fce8b928b0df693f6200807932eb5933146300af886a80a1d922228d93faf531145186dae55ad4ad1f691970732eca
2025-10-24 11:00:35 -07:00
Ava Chow
c6c4edf324 Merge bitcoin/bitcoin#32983: rpc: refactor: use string_view in Arg/MaybeArg
b63428ac9c rpc: refactor: use more (Maybe)Arg<std::string_view> (stickies-v)
037830ca0d refactor: increase string_view usage (stickies-v)
b3bf18f0ba rpc: refactor: use string_view in Arg/MaybeArg (stickies-v)

Pull request description:

  The `RPCHelpMan::{Arg,MaybeArg}` helpers avoid copying (potentially) large strings by returning them as `const std::string*` (`MaybeArg`) or `const std::string&` (`Arg`). For `MaybeArg`, this has the not-so-nice effect that users need to deal with raw pointers, potentially also requiring new functions (e.g. [`EnsureUniqueWalletName` ](d127b25199 (diff-d8bfcfbdd5fa7d5c52d38c1fe5eeac9ce5c5a794cdfaf683585140fa70a32374R32))) with raw pointers being implemented.

  This PR aims to improve on this by returning a trivially copyable `std::string_view` (`Arg`) or `std::optional<std::string_view>` (`MaybeArg`), modernizing the interface without introducing any additional copying overhead. In doing so, it also generalizes whether we return by value or by pointer/reference using `std::is_trivially_copyable_v` instead of defining the types manually.

  In cases where functions currently take a `const std::string&` and it would be too much work / touching consensus logic to update them (`signmessage.cpp`), a `std::string` copy is made (which was already happening anyway).

  The last 2 commits increase usage of the `{Arg,MaybeArg}<std::string_view>` helpers, and could be dropped/pruned if anything turns out to be controversial - I just think it's a nice little cleanup.

ACKs for top commit:
  maflcko:
    re-ACK b63428ac9c 🎉
  achow101:
    ACK b63428ac9c
  pablomartin4btc:
    re-ACK [b63428a](b63428ac9c)
  w0xlt:
    reACK b63428ac9c

Tree-SHA512: b4942c353a1658c22a88d8c9b402c288ad35265a3b88aa2072b1f9b6d921cd073194ed4b00b807cb48ca440f47c87ef3d8e0dd1a5d814be58fc7743f26288277
2025-10-24 10:33:51 -07:00
Ava Chow
00ad998d95 Merge bitcoin/bitcoin#33252: p2p: add DifferenceFormatter fuzz target and invariant check
65a10fc3c5 p2p: add assertion for BlockTransactionsRequest indexes (frankomosh)
58be359f6b fuzz: add a target for DifferenceFormatter Class (frankomosh)

Pull request description:

  Adds a fuzz test for the [`DifferenceFormatter`](e3f416dbf7/src/blockencodings.h (L22-L42)) (used in [`BlockTransactionsRequest`](https://github.com/bitcoin/bitcoin/blob/master/src/blockencodings.h#L44-L54), [BIP 152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki)). The DifferenceFormatter class implements differential encoding for compact block transactions (BIP 152). This PR ensures that its strictly-monotonic property is maintained. It complements the tests in [`blocktransactionsrequest_deserialize`](9703b7e6d5/src/test/fuzz/deserialize.cpp (L314)).

  Additionally, there's an added invariant check after GETBLOCKTXN deserialization in `net_processing.cpp`.

ACKs for top commit:
  Crypt-iQ:
    tACK 65a10fc3c5
  achow101:
    ACK 65a10fc3c5
  dergoegge:
    Code review ACK 65a10fc3c5

Tree-SHA512: 70659cf045e99bb5f753763c7ddac094cb2883c202c899276cbe616889afa053b2d5e831f99d6386d4d1e4118cd35fa0b14b54667853fe067f6efe2eb77b4097
2025-10-24 10:12:11 -07:00
merge-script
af78d36512 Merge bitcoin/bitcoin#32588: util: Abort on failing CHECK_NONFATAL in debug builds
fa37153288 util: Abort on failing CHECK_NONFATAL in debug builds (MarcoFalke)
fa0dc4bdff test: Allow testing of check failures (MarcoFalke)
faeb58fe66 refactor: Set G_ABORT_ON_FAILED_ASSUME when G_FUZZING_BUILD (MarcoFalke)

Pull request description:

  A failing `CHECK_NONFATAL` will throw an exception. This is fine and even desired in production builds, because the program may catch the exception and give the user a way to easily report the bug upstream.

  However, in debug development builds, exceptions for internal bugs are problematic:

  * The exception could accidentally be caught and silently ignored
  * The exception does not include a full stacktrace, possibly making debugging harder

  Fix all issues by turning the exception into an abort in debug builds.

  This can be tested by reverting the hunks to `src/rpc/node.cpp` and `test/functional/rpc_misc.py` and then running the functional or fuzz tests.

ACKs for top commit:
  achow101:
    ACK fa37153288
  ryanofsky:
    Code review ACK fa37153288, just catching subprocess.CalledProcessError in test fixing up a comment since last review
  stickies-v:
    ACK fa37153288

Tree-SHA512: 2d892b838ccef6f9b25a066e7c2f6cd6f5acc94aad1d91fce62308983bd3f5c5d724897a76de4e3cc5c3678ddadc87e2ee8c87362965373526038e598dfb0101
2025-10-24 04:41:24 +02:00
Tim Ruffing
51877f2fc5 test: Update BIP324 test vectors
based on https://github.com/bitcoin/bips/pull/2016
2025-10-23 14:20:17 +02:00
merge-script
161864a038 Merge bitcoin/bitcoin#32579: p2p: Correct unrealistic headerssync unit test behavior
cc5dda1de3 headerssync: Make HeadersSyncState more flexible and move constants (Hodlinator)
8fd1c2893e test(headerssync): Test returning of pow_validated_headers behavior (Hodlinator)
7b00643ef5 test(headerssync): headers_sync_chainwork test improvements (Hodlinator)
04eeb9578c doc(test): Improve comments (Hodlinator)
fe896f8faa refactor(test): Store HeadersSyncState on the stack (Hodlinator)
f03686892a refactor(test): Break up headers_sync_state (Hodlinator)
e984618d0b refactor(headerssync): Process spans of headers (Hodlinator)
a4ac9915a9 refactor(headerssync): Extract test constants ahead of breakup into functions (Hodlinator)

Pull request description:

  ### Background

  As part of the release process we often run *contrib/devtools/headerssync-params.py* and increase the values of the constants `HEADER_COMMITMENT_PERIOD` and `REDOWNLOAD_BUFFER_SIZE` in *src/headerssync.cpp* as per *doc/release-process.md* (example: 11a2d3a63e). This helps fine tune the memory consumption per `HeadersSyncState`-instance in the face of malicious peers.

  (The `REDOWNLOAD_BUFFER_SIZE`/`HEADER_COMMITMENT_PERIOD` ratio determines how many Headers Sync commitment bits must match between PRESYNC & REDOWNLOAD phases before we start permanently storing headers from a peer. For more details see comments in *src/headerssync.h* and *contrib/devtools/headerssync-params.py*).

  ### Problem: Not feeding back headers until completing sync

  During v30 release process #33274 made `REDOWNLOAD_BUFFER_SIZE` exceed the `target_blocks` constant used to control the length of chains generated for testing Headers Sync (`15000`, *headers_sync_chainwork_tests.cpp*).

  The `HeadersSyncState::m_redownloaded_headers`-buffer now does not reach the `REDOWNLOAD_BUFFER_SIZE`-threshold during those unit tests. As a consequence `HeadersSyncState::PopHeadersReadyForAcceptance()` will not start feeding back headers until the PoW threshold has been met. While this will not cause the unit test to start failing on master, it means we have gone from testing behavior that resembles mainnet (way more than `REDOWNLOAD_BUFFER_SIZE` headers to reach the PoW limit), to behavior that is not possible/expected there.

  ### Solution

  Avoid testing this unrealistic condition of completing Headers Sync before reaching `REDOWNLOAD_BUFFER_SIZE` by making tests able to define their own values through the new `HeadersSyncParams` instead of having them hard-coded for all chains & tests.

  ### Commits

  * First 6 commits refactor and improve the unit tests in order to clarify latter changes.
  * We then add checks for the behavior around the `REDOWNLOAD_BUFFER_SIZE` threshold.
  * The main change: we extract the section from *headerssync.cpp* containing the constants to *kernel/chainparams.cpp*, making `HeadersSyncState` no longer hard-coded to mainnet.

  ### Notes

  This PR used to be called "headerssync: Preempt unrealistic unit test behavior".

ACKs for top commit:
  l0rinc:
    reACK cc5dda1de3
  marcofleon:
    code review ACK cc5dda1de3
  danielabrozzoni:
    reACK cc5dda1de3

Tree-SHA512: ccc824dcbbb8ad5ae98c3bf5808b38467aac0230739898a758c9b939eecd74f982df088fa0ba81cc1c1732f19a607b135a6e9577bb9fcf7f8570567ce92f66e6
2025-10-23 06:19:50 -04:00
merge-script
99cb2054bd Merge bitcoin/bitcoin#33600: refactor: Construct g_verify_flag_names on first use
faa9d10c84 refactor: Construct g_verify_flag_names on first use (MarcoFalke)

Pull request description:

  The current usage of the `g_verify_flag_names` map seems fine and I can not see a static initialization order fiasco here.

  However, it seems brittle to hope this remains the case in the future. Also, it triggers a msan false-positive in the fuzz CI task. (C.f https://github.com/bitcoin-core/qa-assets/actions/runs/18352815555/job/52413137315?pr=241#step:7:5245)

  So just apply the "Construct on first use" idiom.

ACKs for top commit:
  kevkevinpal:
    ACK [faa9d10](faa9d10c84)
  ajtowns:
    ACK faa9d10c84
  janb84:
    lgtm ACK faa9d10c84
  stickies-v:
    ACK faa9d10c84

Tree-SHA512: 6685dfc91c99a8245722e07fac99a7a6d58586c30964be7ccd74a176dfbf00c6255c8594621e2909640763924f51d3efd4ce65ed65eaeeb1d05c2fd01fe63604
2025-10-23 05:55:55 -04:00
merge-script
211bf6c975 Merge bitcoin/bitcoin#33566: miner: fix empty mempool case for waitNext()
8f7673257a miner: fix empty mempool case for waitNext() (Sjors Provoost)

Pull request description:

  Block template fees are calculated by looping over `new_tmpl->vTxFees` and return (early) once the `fee_threshold` is exceeded.

  This left an edge case when the mempool is empty, which this commit fixes and adds a test for.

  Also update `test/functional/interface_ipc.py` to reflect the new behavior,

  Fixes https://github.com/Sjors/sv2-tp/issues/9

ACKs for top commit:
  optout21:
    ACK 8f7673257a
  cedwies:
    tACK 8f76732
  sipa:
    utACK 8f7673257a
  zaidmstrr:
    Concept ACK [8f76732](8f7673257a)

Tree-SHA512: ef200fe95e96f810e425283bc37f945c4bf5efa16f4b74820b8a07968f30c5146bca213a372124be84b48beead5dfd35f2b5d10d188d0a465f847ebab61de10a
2025-10-23 05:49:29 -04:00
Hennadii Stepanov
98c4994d0f Merge bitcoin/bitcoin#33570: randomenv: Fix MinGW dllimport warning for environ
9610b0d1e2 randomenv: Fix MinGW dllimport warning for `environ` (Lőrinc)

Pull request description:

  Related to https://github.com/bitcoin/bitcoin/pull/33550#issuecomment-3378978210

  Extends 7703884 to guard environ declaration on all Windows builds, not just MSVC.

  In the `mingw-w64` headers (used by `llvm-mingw`), `environ` is defined as a macro which  expands through [`_environ`](cdb052f1d4/mingw-w64-headers/crt/stdlib.h (L262-L264)) to `(* __p__environ())`, a call to a `dllimport` function, causing the same inconsistent linkage warning as MSVC.

  Use `WIN32` instead of `_MSC_VER` to match the platform-specific guards already used throughout the file.

  The warning occurs with `llvm-mingw` (both `UCRT` and `MSVCRT` variants as tested by Hebasto), but not with the `mingw-w64` toolchain currently used in CI (as mentioned by fanquake).

  ----

  The error was reproduced by adding a temporary [nightly build](https://github.com/l0rinc/bitcoin-core-nightly/pull/4) pointing to https://github.com/l0rinc/bitcoin/pull/45. On `master` the failure can be seen in https://github.com/l0rinc/bitcoin-core-nightly/pull/2

  before:
  https://github.com/l0rinc/bitcoin-core-nightly/actions/runs/18327936488/job/52196728885?pr=2

  <details>
  <summary>Details</summary>

  ```
  /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/src/randomenv.cpp:61:15: warning: '__p__environ' redeclared without 'dllimport' attribute: previous 'dllimport' ignored [-Winconsistent-dllimport]
     61 | extern char** environ; // NOLINT(readability-redundant-declaration): Necessary on some platforms
        |               ^
  /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:656:17: note: expanded from macro 'environ'
    656 | #define environ _environ
        |                 ^
  /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:225:21: note: expanded from macro '_environ'
    225 | #define _environ (* __p__environ())
        |                     ^
  /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:221:27: note: previous declaration is here
    221 |   _CRTIMP char ***__cdecl __p__environ(void);
        |                           ^
  /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/stdlib.h:221:3: note: previous attribute is here
    221 |   _CRTIMP char ***__cdecl __p__environ(void);
        |   ^
  /home/runner/work/bitcoin-core-nightly/bitcoin-core-nightly/llvm_mingw_toolchain/aarch64-w64-mingw32/include/_mingw.h:52:40: note: expanded from macro '_CRTIMP'
     52 | #      define _CRTIMP  __attribute__ ((__dllimport__))
        |                                        ^
  1 warning generated.
  ```

  </details>

  after:
  https://github.com/l0rinc/bitcoin-core-nightly/actions/runs/18329616268/job/52201940831?pr=4

  <details>
  <summary>Details</summary>

  ```
  [ 28%] Building CXX object src/util/CMakeFiles/bitcoin_util.dir/__/randomenv.cpp.obj
  ```

  </details>

  Note that there are some other remaining warnings in the logs that will be fixed in separate PRs

ACKs for top commit:
  sipa:
    utACK 9610b0d1e2 if this makes the compilers happy
  laanwj:
    Code review ACK 9610b0d1e2
  hebasto:
    re-ACK 9610b0d1e2.

Tree-SHA512: a9e39d288b663ed24cbbbae228850e6f02d417d8781a3ac3d0b3db0b7ff734bbd62fddb9f57b8f77daab4e9c016ff66906ebc5fb2de7635ef539ef7f4dc2eaba
2025-10-22 11:51:05 +02:00
Hennadii Stepanov
3fee0754a2 Merge bitcoin/bitcoin#33550: Fix windows libc++ fs::path fstream compile errors
c864a4c194 Simplify fs::path by dropping filename() and make_preferred() overloads (Ryan Ofsky)
b0113afd44 Fix windows libc++ fs::path fstream compile errors (Ryan Ofsky)

Pull request description:

  Drop support for passing `fs::path` directly to `std::ifstream` and `std::ofstream` constructors and `open()` functions, because as reported by hebasto in https://github.com/bitcoin/bitcoin/issues/33545, after https://wg21.link/lwg3430 there is no way this can continue to work in windows builds, and there are already compile errors compiling for windows with newer versions of libc++.

  Instead, add an `fs::path::std_path()` method that returns `std::filesystem::path` references and use it where needed.

ACKs for top commit:
  hebasto:
    ACK c864a4c194.
  l0rinc:
    Code review ACK c864a4c194
  maflcko:
    re-ACK c864a4c194 🌥

Tree-SHA512: d22372692ab86244e2b2caf4c5e9c9acbd9ba38df5411606b75e428474eabead152fc7ca1afe0bb0df6b818351211a70487e94b40a17b68db5aa757604a0ddf6
2025-10-22 10:10:27 +02:00
merge-script
0eeae4d174 Merge bitcoin/bitcoin#33625: Update secp256k1 subtree to latest master
3cbf7cb3e6 Squashed 'src/secp256k1/' changes from b9313c6e1a..d543c0d917 (fanquake)

Pull request description:

  Updates the subtree to d543c0d917
  Related to #33284.

ACKs for top commit:
  hebasto:
    ACK 879c21045e.
  janb84:
    ACK 879c21045e

Tree-SHA512: 1802cd84959b5c935170792f458651f30431fe8340ead7966ff381c1c0c3a9f6c21bbb8dd96a07482ffed49642ded49e80b61802e688b8351956b111dffd5a78
2025-10-19 15:45:47 +01:00
merge-script
d30f149360 Merge bitcoin/bitcoin#33630: doc: correct topology requirements in submitpackage helptext
3d22282564 [doc] correct topology requirements in submitpackage helptext (glozow)

Pull request description:

  This doc is outdated since #31385. Also made it explicit that a singleton is ok.

  Can be backported to 30.x, but doesn't need to be backported earlier ("if any" covers #31096).

ACKs for top commit:
  janb84:
    ACK 3d22282564
  instagibbs:
    ACK 3d22282564

Tree-SHA512: 95e40630a5b2a571029c0657c20a5e2a1cf1789913b868cee314c1a9fcb9a09fccdd3c87f3f15a8eb95c5ff9b83f8adee0661f86619bf21965866b6f6a76dfd0
2025-10-17 15:16:07 +01:00
glozow
3d22282564 [doc] correct topology requirements in submitpackage helptext 2025-10-17 09:29:16 -04:00
fanquake
54ffe3de5b Update leveldb subtree to latest master 2025-10-16 13:49:49 +01:00
merge-script
e14451ac87 Merge bitcoin/bitcoin#33469: TxGraph: change m_excluded_clusters
9b43428c96 TxGraph: change m_excluded_clusters (Greg Sanders)

Pull request description:

  Change BlockBuilderImpl's m_excluded_clusters to unordered set since ordering is not used.

  Change the set to a set of sequence numbers for a modest stability increase under fuzz testing.

ACKs for top commit:
  sipa:
    ACK 9b43428c96
  marcofleon:
    tACK 9b43428c96
  glozow:
    ACK 9b43428c96

Tree-SHA512: 140a492af93f3eff756847a8168aab2624bb7df407f177dde6f3b07e9db2d0ced6b125e2b126f4957ccd054272056bedf74f9f0e64a80d90c16fd94e0fa86a44
2025-10-15 10:00:49 -04:00
merge-script
f76e1ae389 Merge bitcoin/bitcoin#32313: coins: fix cachedCoinsUsage accounting in CCoinsViewCache
24d861da78 coins: only adjust `cachedCoinsUsage` on `EmplaceCoinInternalDANGER` insert (Lőrinc)
d7c9d6c291 coins: fix `cachedCoinsUsage` accounting to prevent underflow (Lőrinc)
39cf8bb3d0 refactor: remove redundant usage tracking from `CoinsViewCacheCursor` (Lőrinc)
67cff8bec9 refactor: assert newly-created parent cache entry has zero memory usage (Lőrinc)

Pull request description:

  ### Summary

  This PR fixes `cachedCoinsUsage` accounting bugs in `CCoinsViewCache` that caused UBSan `unsigned-integer-overflow` violations during testing. The issues stemmed from incorrect decrement timing in `AddCoin()`, unconditional reset in `Flush()` on failure, and incorrect increment in `EmplaceCoinInternalDANGER()` when insertion fails.

  ### Problems Fixed

  **1. `AddCoin()` underflow on exception**
  - Previously decremented `cachedCoinsUsage` *before* the `possible_overwrite` validation
  - If validation threw, the map entry remained unchanged but counter was decremented
  - This corrupted accounting and later caused underflow
  - **Impact**: Test-only in current codebase, but unsound accounting that could affect future changes

  **2. `Flush()` accounting drift on failure**
  - Unconditionally reset `cachedCoinsUsage` to 0, even when `BatchWrite()` failed
  - Left the map populated while the counter read zero
  - **Impact**: Test-only (production `BatchWrite()` returns `true`), but broke accounting consistency

  **3. Cursor redundant usage tracking**
  - `CoinsViewCacheCursor::NextAndMaybeErase()` subtracted usage when erasing spent entries
  - However, `SpendCoin()` already decremented and cleared the `scriptPubKey`, leaving `DynamicMemoryUsage()` at 0
  - **Impact**: Redundant code that obscured actual accounting behavior

  **4. `EmplaceCoinInternalDANGER()` double-counting**
  - Incremented `cachedCoinsUsage` even when `try_emplace` did not insert (duplicate key)
  - Inflated the counter on duplicate attempts
  - **Impact**: Mostly test-reachable (AssumeUTXO doesn't overwrite in production), but incorrect accounting

  ### Testing

  To reproduce the historical UBSan failures on the referenced baseline and to verify the fix, run:
  ```
  MAKEJOBS="-j$(nproc)" FILE_ENV="./ci/test/00_setup_env_native_fuzz.sh" ./ci/test_run_all.sh
  ```

  The change was tested with the related unit and fuzz test, and asserted before/after each `cachedCoinsUsage` change (in production code and fuzz) that the calculations are still correct by recalculating them from scratch.

  <details>
  <summary>Details</summary>

  ```C++
  bool CCoinsViewCache::CacheUsageValid() const
  {
      size_t actual{0};
      for (auto& entry : cacheCoins | std::views::values) actual += entry.coin.DynamicMemoryUsage();
      return actual == cachedCoinsUsage;
  }
  ```
  or
  ```patch
  diff --git a/src/coins.cpp b/src/coins.cpp
  --- a/src/coins.cpp(revision fd3b1a7f4bb2ac527f23d4eb4cfa40a3215906e5)
  +++ b/src/coins.cpp(revision 872a05633bfdbd06ad82190d7fe34b42d13ebfe9)
  @@ -96,6 +96,7 @@
           fresh = !it->second.IsDirty();
       }
       if (!inserted) {
  +        Assert(cachedCoinsUsage >= it->second.coin.DynamicMemoryUsage());
           cachedCoinsUsage -= it->second.coin.DynamicMemoryUsage();
       }
       it->second.coin = std::move(coin);
  @@ -133,6 +134,7 @@
   bool CCoinsViewCache::SpendCoin(const COutPoint &outpoint, Coin* moveout) {
       CCoinsMap::iterator it = FetchCoin(outpoint);
       if (it == cacheCoins.end()) return false;
  +    Assert(cachedCoinsUsage >= it->second.coin.DynamicMemoryUsage());
       cachedCoinsUsage -= it->second.coin.DynamicMemoryUsage();
       TRACEPOINT(utxocache, spent,
              outpoint.hash.data(),
  @@ -226,10 +228,12 @@
               if (itUs->second.IsFresh() && it->second.coin.IsSpent()) {
                   // The grandparent cache does not have an entry, and the coin
                   // has been spent. We can just delete it from the parent cache.
  +                Assert(cachedCoinsUsage >= itUs->second.coin.DynamicMemoryUsage());
                   cachedCoinsUsage -= itUs->second.coin.DynamicMemoryUsage();
                   cacheCoins.erase(itUs);
               } else {
                   // A normal modification.
  +                Assert(cachedCoinsUsage >= itUs->second.coin.DynamicMemoryUsage());
                   cachedCoinsUsage -= itUs->second.coin.DynamicMemoryUsage();
                   if (cursor.WillErase(*it)) {
                       // Since this entry will be erased,
  @@ -279,6 +283,7 @@
   {
       CCoinsMap::iterator it = cacheCoins.find(hash);
       if (it != cacheCoins.end() && !it->second.IsDirty() && !it->second.IsFresh()) {
  +        Assert(cachedCoinsUsage >= it->second.coin.DynamicMemoryUsage());
           cachedCoinsUsage -= it->second.coin.DynamicMemoryUsage();
           TRACEPOINT(utxocache, uncache,
                  hash.hash.data(),
  ```

  </details>

ACKs for top commit:
  optout21:
    reACK 24d861da78
  andrewtoth:
    ACK 24d861da78
  sipa:
    ACK 24d861da78
  w0xlt:
    ACK 24d861da78

Tree-SHA512: ff1b756b46220f278ab6c850626a0f376bed64389ef7f66a95c994e1c7cceec1d1843d2b24e8deabe10e2bdade2a274d9654ac60eb2b9bf471a71db8a2ff496c
2025-10-15 09:48:04 -04:00
merge-script
b1f8a13702 Merge bitcoin/bitcoin#33624: test: P2SH sig ops are only counted with SCRIPT_VERIFY_P2SH
3a10d700bc test: P2SH sig ops are only counted with `SCRIPT_VERIFY_P2SH` flag (brunoerg)

Pull request description:

  This PR adds a test case for `GetTransactionSigOpCost` to check that P2SH sig ops are only counted when `SCRIPT_VERIFY_P2SH` flag is set.

  Kills the following [mutant](https://corecheck.dev/mutation/src/consensus/tx_verify.cpp#L150):

  ```diff
  diff --git a/src/consensus/tx_verify.cpp b/src/consensus/tx_verify.cpp
  index 9d09872597..cc7cdaaf8f 100644
  --- a/src/consensus/tx_verify.cpp
  +++ b/src/consensus/tx_verify.cpp
  @@ -147,7 +147,7 @@ int64_t GetTransactionSigOpCost(const CTransaction& tx, const CCoinsViewCache& i
       if (tx.IsCoinBase())
           return nSigOps;

  -    if (flags & SCRIPT_VERIFY_P2SH) {
  +    if (1==1) {
           nSigOps += GetP2SHSigOpCount(tx, inputs) * WITNESS_SCALE_FACTOR;
       }
  ```

ACKs for top commit:
  l0rinc:
    Tested ACK 3a10d700bc
  maflcko:
    re-lgtm ACK 3a10d700bc
  instagibbs:
    ACK 3a10d700bc
  janb84:
    tested ACK 3a10d700bc

Tree-SHA512: f560b4f9f2ce5c5fdd0a86e7e1f8ea27a8c6fda0327a6186a0c21e2c06ef13beeb017686db1688cace68812a01701abe46e8e1a095afefc6f2aed6ed96ba8288
2025-10-15 09:55:49 +01:00
fanquake
879c21045e Update secp256k1 subtree to latest master 2025-10-15 09:49:11 +01:00
fanquake
3cbf7cb3e6 Squashed 'src/secp256k1/' changes from b9313c6e1a..d543c0d917
d543c0d917 Merge bitcoin-core/secp256k1#1734: Introduce (mini) unit test framework
f44c1ebd96 Merge bitcoin-core/secp256k1#1719: ci: DRY workflow using anchors
a44a339384 Merge bitcoin-core/secp256k1#1750: ci: Use clang-snapshot in "MSan" job
15d014804e ci: Drop default for `inputs.command` in `run-in-docker-action`
1decc49a1f ci: Use YAML anchor and aliases for repeated "CI script" steps
dff1bc107d ci, refactor: Generalize use of `matrix.configuration.env_vars`
4b644da199 ci: Use YAML anchor and aliases for repeated "Print logs" steps
a889cd93df ci: Bump `actions/checkout` version
574c2f3080 ci: Use YAML anchor and aliases for repeated "Checkout" steps
53585f93b7 ci: Use clang-snapshot in "MSan" job
6894c964f3 Fix Clang 21+ `-Wuninitialized-const-pointer` warning when using MSan
2b7337f63a Merge bitcoin-core/secp256k1#1756: ci: Fix image caching and apply other improvements
f163c35897 ci: Set `DEBIAN_FRONTEND=noninteractive`
70ae177ca0 ci: Bump `docker/build-push-action` version
b2a95a420f ci: Drop `tags` input for `docker/build-push-action`
122014edb3 ci: Add `scope` parameter to `cache-{to,from}` options
2f4546ce56 test: add --log option to display tests execution
95b9953ea4 test: Add option to display all available tests
953f7b0088 test: support running specific tests/modules targets
0302c1a3d7 test: add --help for command-line options
9ec3bfe22d test: adapt modules to the new test infrastructure
48789dafc2 test: introduce (mini) unit test framework
baa265429f Merge bitcoin-core/secp256k1#1727: docs: Clarify that callback can be called more than once
4d90585fea docs: Improve API docs of _context_set_illegal_callback
895f53d1cf docs: Clarify that callback can be called more than once
de6af6ae35 Merge bitcoin-core/secp256k1#1748: bench: improve context creation in ECDH benchmark
5817885153 Merge bitcoin-core/secp256k1#1749: build: Fix warnings in x86_64 assembly check
ab560078aa build: Fix warnings in x86_64 assembly check
10dab907e7 Merge bitcoin-core/secp256k1#1741: doc: clarify API doc of `secp256k1_ecdsa_recover` return value
dfe284ed2d bench: improve context creation in ECDH benchmark
7321bdf27b doc: clarify API doc of `secp256k1_ecdsa_recover` return value
b475654302 Merge bitcoin-core/secp256k1#1745: test: introduce group order byte-array constant for deduplication
9cce703863 refactor: move 'gettime_i64()' to tests_common.h
0c91c56041 test: introduce group order byte-array constant for deduplication
88be4e8d86 Merge bitcoin-core/secp256k1#1735: musig: Invalidate secnonce in secp256k1_musig_partial_sign
36e76952cb Merge bitcoin-core/secp256k1#1738: check-abi: remove support for obsolete CMake library output location (src/libsecp256k1.so)
399b582a5f Split memclear into two versions
4985ac0f89 Merge bitcoin-core/secp256k1#1737: doc: mention ctx requirement for `_ellswift_create` (not secp256k1_context_static)
7ebaa134a7 check-abi: remove support for obsolete CMake library output location (src/libsecp256k1.so)
806de38bfc doc: mention ctx requirement for `_ellswift_create` (not secp256k1_context_static)
03fb60ad2e Merge bitcoin-core/secp256k1#1681: doc: Recommend clang-cl when building on Windows
d93380fb35 Merge bitcoin-core/secp256k1#1731: schnorrsig: Securely clear buf containing k or its negation
8113671f80 Merge bitcoin-core/secp256k1#1729: hash: Use size_t instead of int for RFC6979 outlen copy
325d65a8cf Rename and clear var containing k or -k
960ba5f9c6 Use size_t instead of int for RFC6979 outlen copy
737912430d ci: Add more tests for clang-cl
7379a5bed3 doc: Recommend clang-cl when building on Windows
f36afb8b3d Merge bitcoin-core/secp256k1#1725: tests: refactor tagged hash verification
5153cf1c91 tests: refactor tagged hash tests
d2dcf52091 Merge bitcoin-core/secp256k1#1726: docs: fix broken link to Tromer's cache.pdf paper
489a43d1bf docs: fix broken link to eprint cache.pdf paper
d599714147 Merge bitcoin-core/secp256k1#1722: docs: Exclude modules' `bench_impl.h` headers from coverage report
0458def51e doc: Add `--gcov-ignore-parse-errors=all` option to `gcovr` invocations
1aecce5936 doc: Add `--merge-mode-functions=separate` option to `gcovr` invocations
106a7cbf41 doc: Exclude modules' `bench_impl.h` headers from coverage report
a9e955d3ea autotools, docs: Adjust help string for `--enable-coverage` option
e523e4f90e Merge bitcoin-core/secp256k1#1720: chore(ci): Fix typo in Dockerfile comment
24ba8ff168 chore(ci): Fix typo in Dockerfile comment
74b8068c5d Merge bitcoin-core/secp256k1#1717: test: update wycheproof test vectors
c25c3c8a88 test: update wycheproof test vectors
20e3b44746 Merge bitcoin-core/secp256k1#1688: cmake: Avoid contaminating parent project's cache with `BUILD_SHARED_LIBS`
2c076d907a Merge bitcoin-core/secp256k1#1711: tests: update Wycheproof
7b07b22957 cmake: Avoid contaminating parent project's cache with BUILD_SHARED_LIBS
5433648ca0 Fix typos and spellings
9ea54c69b7 tests: update Wycheproof files

git-subtree-dir: src/secp256k1
git-subtree-split: d543c0d917a76a201578948701cc30ef336e0fe6
2025-10-15 09:49:11 +01:00