Commit Graph

1483 Commits

Author SHA1 Message Date
MarcoFalke
fa30b34026 test: Do not pass tests on unhandled exceptions
This adds a missing catch for BaseException (e.g. SystemExit), which
would otherwise be silently ignored.

Also, remove the redundant other catches, which are just calling
log.exception with a redundant log message.
2025-07-17 17:04:57 +02:00
Sjors Provoost
83bb414557 test: less ambiguous error if bitcoind is missing
Before this change, when a functional test is run without building
the source, the error message suggested that previous release binaries
were missing.

When no previous release version is set, make the error message more
specifically about bitcoind.
2025-07-08 13:19:48 +02:00
merge-script
fc543f94a9 Merge bitcoin/bitcoin#32385: test: refactor out same-txid-diff-wtxid tx to reuse in other tests
afaaba69ed test: refactor out same-txid-diff-wtxid tx to reuse in other tests (stratospher)

Pull request description:

  It's useful to easily create transactions with same txid, different wtxid and valid witness for testing scenarios in other places in the codebase (ex: private broadcast connections, see https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2055915118)

  So refactor out the current `same-txid-diff-wtxid` transaction in `mempool_accept_wtxid.py` so that it can be reused.

ACKs for top commit:
  maflcko:
    review ACK afaaba69ed 📎
  theStack:
    ACK afaaba69ed
  vasild:
    ACK afaaba69ed

Tree-SHA512: 0fc51ac326725d4abe76a15b6b5be55d070b96c303c444f4dd31c2b0a82f266836382389a123a7f6a71aa35e61fbfae27f843b31cc19474e49f3dc82f36ebf73
2025-07-07 16:37:04 -04:00
merge-script
6251949443 Merge bitcoin/bitcoin#32290: test: allow all functional tests to be run or skipped with --usecli
666016e56b ci: use --usecli in one of the CI jobs (Martin Zumsande)
7ea248a020 test: Disable several (sub)tests with cli (Martin Zumsande)
f420b6356b test: skip subtests that check for wrong types with cli (Martin Zumsande)
6530d0015b test: add function to convert to json for height_or_hash params (Martin Zumsande)
54d28722ba test: Don't send empty named args with cli (Martin Zumsande)
cca422060e test: convert tuple to json for cli (Martin Zumsande)
af34e98086 test: make rpc_psbt.py usable with --usecli (Martin Zumsande)
8f8ce9e174 test: rename .rpc to ._rpc and remove unnecessary uses (Martin Zumsande)
5b08885986 test: enable functional tests with large rpc args for cli (Martin Zumsande)
7d5352ac73 test: use -stdin for large rpc commands (Martin Zumsande)
6c364e0c10 test: Enable various tests for usage with cli (Martin Zumsande)

Pull request description:

  Fixes #32264

  I looked into all current failures listed in the issue, as well all tests that are already disabled for the cli with `self.supports_cli = False`. There are several reasons why existing tests fail  with `--usecli` on many systems, the most important ones are:

  - Most common reason is that the test executes a RPC call with a large arg that exceeds `MAX_ARG_STRLEN` of the OS, which is usually 128kb on linux: This is fixed by using `-stdin` for these large calls (idea by 0xB10C)
  - they test specifically the rpc interface - nothing to do there except disabling.
  - Some functional test submit wrong types to params on purpose to test the error message (which is different when using the cli) - deactivated these specific subtests locally for the cli when there is just one or two of them, deactivated the entire tests when there are more spots
  - When python sets `None` for an arg, the cli converts this to 'null' in `arg_to_cli`. This is fine e.g. for boolean args, but doesn't work for strings where it's interpreted as the string 'null'. Bypass this for named args by not including args in case the value is `None` for the cli is used (it's effectively the same as leaving the optional arg out).
  -  the `height_or_hash` param used in some RPC needs to be converted to a JSON (effectively adding full quotes).
  - Some tests were marked with `self.supports_cli = False` in the past but run fine on master today - enabled those.

  In total, this PR fixes all tests that fail on master and reduces the number of tests that are deactivated (`self.supports_cli = False`) from 40 to 21.
  It also adds `--usecli` to one CI job (multiprocess, i686, DEBUG) to detect regressions.

ACKs for top commit:
  maflcko:
    re-ACK 666016e56b 🔀
  pinheadmz:
    re-ACK 666016e56b

Tree-SHA512: 7a1efd212649ca100b236a1239294d40ecd36e2720e3b173a230b14545bb40b135111db7fed8a0d1448120f5387da146a03f1912e2028c8d03a0b6a3ca8761b0
2025-07-03 10:20:03 +01:00
Ava Chow
ce000c8ee0 Merge bitcoin/bitcoin#32219: test: enabling wallet migration functional test on windows
941b8f54c0 ci: run get_previous_releases as part of test cross win job (Max Edwards)
5e2182140b test: increment mocked time for migrating wallet backups (Max Edwards)
5174565802 ci: disable feature_unsupported_utxo_db functional test (Max Edwards)
3dc90d69a6 test: remove mempool.dat before copying (Max Edwards)
67a6b20d50 test: add windows support to get previous releases script (Max Edwards)
1a1b478ca3 scripted-diff: rename tarball to archive (Max Edwards)
4f06dc8484 test: remove building from source from get prev releases script (Max Edwards)

Pull request description:

  This PR updates the `test/get_previous_releases.py` script to also work on Windows by changing to be pure python rather than using unix tools such as `curl` and `tar`.

  This enables additional functional tests to run such as `wallet_migration.py`, `mempool_compatability.py` and `wallet_backwards_compatibility.py`.

  Unfortunately `feature_unsupported_utxo_db.py` _could_ run but this test requires Bitcoin `v0.14.3` which will not run under windows with emojis in the data directory (as the functional test runner has by default) . This test could be run as it's own step in the ci workflow file and would pass but as it's quite an old version / feature I have assumed it's not worth worrying about and best just to exclude.

  Two tests needed to be slightly modified to run under windows. Both were issues with trying to overwrite a file that already exists which windows seems to be more strict on than the unix based systems.

  Finally, building from source has been dropped from the `get_previous_releases.py` script. This had not been updated after the move to cmake and so it was assumed that nobody could have been using that feature.

ACKs for top commit:
  maflcko:
    re-ACK 941b8f54c0 🍪
  achow101:
    ACK 941b8f54c0
  hodlinator:
    re-ACK 941b8f54c0

Tree-SHA512: 22933d0ec278b9b0ffcd2a8e90026e1a3631b00186e7f78bd65be925049021e319367d488c36a82ab526a07b264bac18c2777f87ca1174b231ed49fed56d11cb
2025-07-01 13:58:54 -07:00
merge-script
6e5b67a370 Merge bitcoin/bitcoin#32697: test: Turn util/test_runner into functional test
fa21631595 test: Use self.log (MarcoFalke)
fa346f7797 test: Move error string into exception (MarcoFalke)
fa1986181f test: Remove useless catch-throw (MarcoFalke)
fa2f1c55b7 move-only util data to test/functional/data/util (MarcoFalke)
faa18bf287 test: Turn util/test_runner into functional test (MarcoFalke)
fa955154c7 test: Add missing skip_if_no_bitcoin_tx (MarcoFalke)
fac9db6eb0 test: Add missing tx util to Binaries (MarcoFalke)
fa91835ec6 test: Use lowercase env var as attribute name (MarcoFalke)
fac49094cd test: Remove duplicate ConfigParser (MarcoFalke)

Pull request description:

  The `test/util/test_runner.py` has many issues:

  * The boilerplate for the test runner is duplicate or inconsistent with the other (functional) tests. For example, logging options, `ConfigParser` handling, `Binaries` handling ...
  * The cmake/ci behavior is brittle and can silently fail, as explained in https://github.com/bitcoin/bitcoin/issues/31476
  * corecheck (and likely other places that manually run the tests) completely forget to run it
  * If the test is manually called, it runs single threaded, when it could just run in parallel with the other functional tests

  Fix all issues by removing the util test_runner and moving the test logic into a new functional test file.

ACKs for top commit:
  janb84:
    re ACK fa21631595
  brunoerg:
    re-ACK fa21631595
  hebasto:
    re-ACK fa21631595, additional feedback has been addressed since my previous [review](https://github.com/bitcoin/bitcoin/pull/32697#pullrequestreview-2940350432).

Tree-SHA512: 694e647887801f002843a74011035d5ed3dfed091d3f0ae18e812a16a4680e04e60e50de0a92af7e047e8ddd6ff5a7834c690f16fd42b74ebc1674bf9989406f
2025-06-30 13:45:14 -04:00
MarcoFalke
fa3f100010 test: Use msg_generic in p2p_ping.py 2025-06-30 14:50:55 +02:00
dergoegge
7dc43ea503 test: Add msgtype to msg_generic slots 2025-06-30 11:13:11 +01:00
Ava Chow
b3bb4031ab Merge bitcoin/bitcoin#32540: rest: fetch spent transaction outputs by blockhash
c48846ec41 doc: add release notes for #32540 (Roman Zeyde)
d4e212e8a6 rest: fetch spent transaction outputs by blockhash (Roman Zeyde)

Pull request description:

  Today, it is possible to fetch a block's spent prevouts in order to build an external index by using the `/rest/block/BLOCKHASH.json` endpoint. However, its performance is low due to JSON serialization overhead.

  We can significantly optimize it by adding a new [REST API](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md) endpoint, using a binary response format (returning a collection of spent txout lists, one per each block transaction):

  ```
  $ BLOCKHASH=00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054

  $ ab -k -c 1 -n 100 http://localhost:8332/rest/block/$BLOCKHASH.json
  Document Length:        13278152 bytes
  Requests per second:    3.53 [#/sec] (mean)
  Time per request:       283.569 [ms] (mean)

  $ ab -k -c 1 -n 10000 http://localhost:8332/rest/spenttxouts/$BLOCKHASH.bin
  Document Length:        195591 bytes
  Requests per second:    254.47 [#/sec] (mean)
  Time per request:       3.930 [ms] (mean)
  ```

  Currently, this PR is being used and tested by Bindex[^1].

  This PR would allow to improve the performance of external indexers such as electrs[^2], ElectrumX[^3], Fulcrum[^4] and Blockbook[^5].

  [^1]: https://github.com/romanz/bindex-rs
  [^2]: https://github.com/romanz/electrs (also [blockstream.info](https://github.com/Blockstream/electrs) and [mempool.space](https://github.com/mempool/electrs) forks)
  [^3]: https://github.com/spesmilo/electrumx
  [^4]: https://github.com/cculianu/Fulcrum
  [^5]: https://github.com/trezor/blockbook

ACKs for top commit:
  maflcko:
    re-ACK c48846ec41 📶
  TheCharlatan:
    Re-ACK c48846ec41
  achow101:
    ACK c48846ec41

Tree-SHA512: cf423541be90d6615289760494ae849b7239b69427036db6cc528ac81df10900f514471d81a460125522c5ffa31e9747ddfca187a1f93151e4ae77fe773c6b7b
2025-06-27 14:44:41 -07:00
Max Edwards
4f06dc8484 test: remove building from source from get prev releases script
Using the get_previous_releases.py script to build from source only works for
releases prior to v29 due to removal of Autotools (in favor of CMake). It also
does not support building on Windows, and we are adding support for downloading
Windows release binaries in later commits of this PR.

As there were no complaints during review, it is assumed nobody uses this
functionality.
2025-06-25 20:56:22 +01:00
Martin Zumsande
6530d0015b test: add function to convert to json for height_or_hash params
This is necessary for bitcoin-cli because a string without quotes
is not a valid json.
2025-06-23 17:42:56 -04:00
Martin Zumsande
54d28722ba test: Don't send empty named args with cli
If python passed None for an optional (i.e. 'null' is
sent), this will lead to the arg being interpreted as not
provided by bitcoind - except for string args, for which the arg is
interpreted as as 'null' string. Bypass this by not sending
named args to bitcoin-cli - so that the default value will
actually be used.

Also drops an unnecessary str() conversion, kwargs keys
are always strings.
2025-06-23 17:42:56 -04:00
Martin Zumsande
cca422060e test: convert tuple to json for cli
This makes it possible to run rpc_createmultisig.py with --usecli.
2025-06-23 17:42:56 -04:00
Martin Zumsande
8f8ce9e174 test: rename .rpc to ._rpc and remove unnecessary uses
It is usually not necessary, and makes it impossible to use --usecli
2025-06-23 17:42:55 -04:00
Martin Zumsande
7d5352ac73 test: use -stdin for large rpc commands
Because of the MAX_ARG_STRLEN limit (128kb on most systems)
for args, these would usually fail. As a workaround, use
-stdin for these large calls. Idea by 0xB10C.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-06-23 17:42:55 -04:00
Ava Chow
daf393b3f1 Merge bitcoin/bitcoin#32642: test: update BIP340 test vectors and implementation (variable-length messages)
b184f5c87c test: update BIP340 test vectors and implementation (variable-length messages) (Sebastian Falbesoner)

Pull request description:

  This PR updates the Schnorr signatures implementation in the functional test framework to the latest BIP changes (see https://github.com/bitcoin/bips/pull/1446,commit 200f9b26fe0a2f235a2af8b30c4be9f12f6bc9cb) and syncs the [test vectors](https://github.com/bitcoin/bips/blob/master/bip-0340/test-vectors.csv) accordingly. Practically, we probably don't need non-32-bytes message signing/verifying any time soon, but it seems good practice anyways to update.

ACKs for top commit:
  stratospher:
    ACK b184f5c.
  achow101:
    ACK b184f5c87c
  real-or-random:
    utACK b184f5c87c
  jonasnick:
    utACK b184f5c87c

Tree-SHA512: b566823aa0f1cd7151215178c57551d772b338d022ccb2807a0df2670df6d59c4b63a6fc936708ccf2922c7e59f474f544adaafc4aea731bfd896250c0d45fa6
2025-06-20 14:45:34 -07:00
stratospher
afaaba69ed test: refactor out same-txid-diff-wtxid tx to reuse in other tests
useful to easily create transactions with same txid, different
wtxid and valid witness for testing scenarios in other places
(ex: private broadcast connections)
2025-06-16 14:48:17 +05:30
MarcoFalke
fa955154c7 test: Add missing skip_if_no_bitcoin_tx 2025-06-13 13:03:48 +02:00
MarcoFalke
fac9db6eb0 test: Add missing tx util to Binaries 2025-06-13 09:48:38 +02:00
MarcoFalke
fa91835ec6 test: Use lowercase env var as attribute name
This is a refactor.
2025-06-13 09:48:32 +02:00
MarcoFalke
fac49094cd test: Remove duplicate ConfigParser
It is sufficient to parse once.
2025-06-13 09:48:28 +02:00
Ava Chow
1473f69924 Merge bitcoin/bitcoin#32421: test: refactor: overhaul (w)txid determination for CTransaction objects
4ef6253017 test: avoid unneeded (w)txid hex -> integer conversions (Sebastian Falbesoner)
472f3770ae scripted-diff: test: rename CTransaction `.getwtxid()` -> `wtxid_hex` for consistency (Sebastian Falbesoner)
81af4334e8 test: rename CTransaction `.sha256` -> `.txid_int` for consistency (Sebastian Falbesoner)
ce83924237 test: rename CTransaction `.rehash()`/`.hash` -> `.txid_hex` for consistency (Sebastian Falbesoner)
e9cdaefb0a test: introduce and use CTransaction `.wtxid_int` property (Sebastian Falbesoner)
9b3dce24a3 test: remove bare CTransaction `.rehash()`/`.calc_sha256()` calls (Sebastian Falbesoner)
a2724e3ea3 test: remove txid caching in CTransaction class (Sebastian Falbesoner)

Pull request description:

  In the functional test framework, determining a (w)txid for a `CTransaction` instance is currently rather confusing and footgunny due to inconsistent naming/interfaces (see table below) and statefulness involved. This PR aims to improve that by:
  * removing the (w)txid caching mechanism, in order to avoid the need to call additional rehashing functions (`.rehash()`/`.calculate_sha256()`, see first two commits and https://github.com/bitcoin/bitcoin/pull/32050#discussion_r1993286997). This change in theory decreases the performance, as the involved serialization and hashing involved might be called more often than previously, but I couldn't find a functional test where this leads to a measurable run-time increase on my machine.
  * introduce consistent naming that shows the type of the returned txid, i.e. hex string vs. test-framework-internal representation [currently integers] (see remaining commits)

  Summary table showing (w)txid determaination before/after this PR:

  | Task                   | master                 | PR           |
  |:-----------------------|:-----------------------|:-------------|
  | get TXID (hex string)  | `.rehash()` / `.hash`[1] | `.txid_hex`  |
  | get TXID (integer)     | `.sha256`[1]             | `.txid_int`  |
  | get WTXID (hex string) | `.getwtxid()`          | `.wtxid_hex` |
  | get WTXID (integer)    | `.calc_sha256(True)`   | `.wtxid_int` |

  Unfortunately, most renames can't be done with a scripted-diff, as the property names (`.hash`, `.sha256`) are also used for blocks and other message types. The PR is rather invasive and touches a lot of files, but I think it's worth to do it, also to make life easier for new contributors. Future tasks like e.g. doing the same overhaul for block (header) objects or getting rid of the integer representation (see https://github.com/bitcoin/bitcoin/pull/32050) become easier should become easier after this one.

  [1] = returned value might be out-of-date, if rehashing function wasn't called after modification

ACKs for top commit:
  maflcko:
    re-ACK 4ef6253017 🏈
  achow101:
    ACK 4ef6253017
  marcofleon:
    code review ACK 4ef6253017

Tree-SHA512: 4b472c31d169966b6f6878911a8404d25bf3e503b6e8ef30f36a7415d21ad4bc1265083af2d3ead6edfcd9fac9ccb0a8be57e1b0739ad431b836413070d7d583
2025-06-11 14:21:11 -07:00
Sebastian Falbesoner
472f3770ae scripted-diff: test: rename CTransaction .getwtxid() -> wtxid_hex for consistency
-BEGIN VERIFY SCRIPT-

sed -i "s|def getwtxid|@property\n    def wtxid_hex|g" ./test/functional/test_framework/messages.py
sed -i "s|getwtxid()|wtxid_hex|g" $(git grep -l getwtxid)

-END VERIFY SCRIPT-
2025-06-11 00:52:25 +02:00
Sebastian Falbesoner
81af4334e8 test: rename CTransaction .sha256 -> .txid_int for consistency
Note that we unfortunately can't use a scripted diff here, as the same
property name is also used for `CBlockHeader`/`CBlock` instances.
2025-06-11 00:52:25 +02:00
Sebastian Falbesoner
ce83924237 test: rename CTransaction .rehash()/.hash -> .txid_hex for consistency
Note that we unfortunately can't use a scripted diff here, as the same
property and method name is also used for `CBlockHeader`/`CBlock` instances.
2025-06-11 00:49:10 +02:00
Ava Chow
d978a43d05 Merge bitcoin/bitcoin#32408: tests: Expand HTTP coverage to assert libevent behavior
f16c8c67bf tests: Expand HTTP coverage to assert libevent behavior (Matthew Zipkin)

Pull request description:

  These commits are cherry-picked from #32061  and part of a project to [remove libevent](https://github.com/bitcoin/bitcoin/issues/31194).

  This PR only adds functional tests to `interface_http` to cover some HTTP server behaviors we inherit from libevent, in order to maintain those behaviors when we replace libevent with our own HTTP server.

  1. Pipelining: The server must respond to requests from a client in the order in which they were received [RFC 7230 6.3.2](https://www.rfc-editor.org/rfc/rfc7230#section-6.3.2)
  2. `-rpcservertimeout` config option which sets the amount of time the server will keep an idle client connection alive
  3. "Chunked" Transfer-Encoding: Allows a client to send a request in pieces, without the `Content-Length` header [RFC 7230 4.1](https://www.rfc-editor.org/rfc/rfc7230#section-4.1)

ACKs for top commit:
  achow101:
    ACK f16c8c67bf
  vasild:
    ACK f16c8c67bf
  polespinasa:
    ACK f16c8c67bf
  fjahr:
    utACK f16c8c67bf

Tree-SHA512: 405b59431b4d2bf118fde04b270865dee06ef980ab120d9cc1dce28e5d65dfd880a57055b407009d22f4de614bc3eebdb3e203bcd39e86cb14fbfd62195ed06a
2025-06-09 13:08:25 -07:00
Sebastian Falbesoner
e9cdaefb0a test: introduce and use CTransaction .wtxid_int property
This commits removes the `.calc_sha256` method from the CTransaction
and introduces a property `.wtxid_int` property as replacement.
2025-06-09 17:28:28 +02:00
Sebastian Falbesoner
9b3dce24a3 test: remove bare CTransaction .rehash()/.calc_sha256() calls
Since the previous commit, CTransaction object calls to the
methods `.rehash()` and `.calc_sha256()` are effectively no-ops
if the returned value is not used, so we can just remove them.
2025-06-09 17:28:24 +02:00
Sebastian Falbesoner
a2724e3ea3 test: remove txid caching in CTransaction class
Rather than txids (represented by the fields `.sha256` and `.hash`)
being stateful, simply compute them on-the-fly. This ensures that
the correct values are always returned and takes the burden of
rehashing from test writers, making the code shorter overall.
In a first step, the fields are kept at the same name with @property
functions as drop-in replacements, for a minimal diff. In later commits,
the names are changed to be more descriptive and indicating the return
type of the txid.
2025-06-09 16:55:04 +02:00
merge-script
f3bbc74664 Merge bitcoin/bitcoin#32406: policy: uncap datacarrier by default
a189d63618 add release note for datacarriersize default change (Greg Sanders)
a141e1bf50 Add more OP_RETURN mempool acceptance functional tests (Peter Todd)
0b4048c733 datacarrier: deprecate startup arguments for future removal (Greg Sanders)
63091b79e7 test: remove unnecessary -datacarriersize args from tests (Greg Sanders)
9f36962b07 policy: uncap datacarrier by default (Greg Sanders)

Pull request description:

  Retains the `-datacarrier*` args, marks them as deprecated, and does not require another startup argument for multiple OP_RETURN outputs.

  If a user has set `-datacarriersize` the value is "budgeted" across all seen OP_RETURN output scriptPubKeys. In other words the total script bytes stays the same, but can be spread across any number of outputs. This is done to not introduce an additional argument to support multiple outputs.

  I do not advise people use the option with custom arguments and it is marked as deprecated to not mislead as a promise to offer it forever. The argument itself can be removed in some future release to clean up the code and minimize footguns for users.

ACKs for top commit:
  stickies-v:
    re-ACK a189d63618
  Sjors:
    re-ACK a189d63618
  polespinasa:
    re-ACK a189d63618
  hodlinator:
    re-ACK a189d63618
  ajtowns:
    reACK a189d63618
  mzumsande:
    re-ACK a189d63618
  petertodd:
    ACK a189d63618
  theStack:
    re-ACK a189d63618
  1440000bytes:
    re-ACK a189d63618
  willcl-ark:
    ACK a189d63618
  dergoegge:
    ACK a189d63618
  fanquake:
    ACK a189d63618
  murchandamus:
    ACK a189d63618
  darosior:
    Concept ACK a189d63618.

Tree-SHA512: 3da2f1ef2f50884d4da7e50df2121bf175cb826edaa14ba7c3068a6d5b2a70beb426edc55d50338ee1d9686b9f74fdf9e10d30fb26a023a718dd82fa1e77b038
2025-06-09 08:23:56 -04:00
Roman Zeyde
d4e212e8a6 rest: fetch spent transaction outputs by blockhash
Today, it is possible to fetch a block's spent prevouts in order to
build an external index by using the `/rest/block/HASH.json` endpoint.
However, its performance is low due to JSON serialization overhead.

We can significantly optimize it by adding a new REST endpoint, using
a binary response format:

```
$ BLOCKHASH=00000000000000000002a7c4c1e48d76c5a37902165a270156b7a8d72728a054

$ ab -k -c 1 -n 100 http://localhost:8332/rest/block/$BLOCKHASH.json
Document Length:        13278152 bytes
Requests per second:    3.53 [#/sec] (mean)
Time per request:       283.569 [ms] (mean)

$ ab -k -c 1 -n 10000 http://localhost:8332/rest/spentoutputs/$BLOCKHASH.bin
Document Length:        195591 bytes
Requests per second:    254.47 [#/sec] (mean)
Time per request:       3.930 [ms] (mean)
```

Currently, this PR is being used and tested by Bindex:

 * https://github.com/romanz/bindex-rs

This PR would allow to improve the performance of external indexers
such as electrs, ElectrumX, Fulcrum and Blockbook:

 * https://github.com/romanz/electrs (also https://github.com/Blockstream/electrs and https://github.com/mempool/electrs)
 * https://github.com/spesmilo/electrumx
 * https://github.com/cculianu/Fulcrum
 * https://github.com/trezor/blockbook
2025-06-07 20:17:50 +03:00
Martin Zumsande
ed179e0a65 test: apply microsecond precision to test framework logging
This makes it easier to compare test and bitcoind logs -
combine_logs.py orders by timestamp and will be more precise.
2025-06-03 14:02:01 -04:00
Ava Chow
2d819fa4df Merge bitcoin/bitcoin#29032: signet: omit commitment for some trivial challenges
6ee32aaaca test: signet tool genpsbt and solvepsbt commands (Sjors Provoost)
0a99d99fe4 signet: miner skips PSBT step for OP_TRUE (Sjors Provoost)
cdfb70e5a6 signet: split decode_psbt miner helper (Sjors Provoost)

Pull request description:

  [BIP325](https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki) mentions the following rule:

  > In the special case where an empty solution is valid (ie scriptSig and scriptWitness are both empty) this additional commitment can optionally be left out. This special case is to allow non-signet-aware block generation code to be used to test a custom signet chain where the challenge is trivially true.

  Such a signet can be created using e.g. `-signetchallenge=51` (`OP_TRUE`). However `contrib/signet/miner` won't omit the commitment.

  This PR improves the miner by skipping the PSBT for known trivial scripts (just `OP_TRUE` and trivial pushes for now). This prevents it from appending the 4 byte signet header to the witness commitment, as allowed by the above rule.

  ---

  Previously the script would fail with `PSBT signing failed`, making it difficult to mine. This is no longer the case.

ACKs for top commit:
  achow101:
    ACK 6ee32aaaca
  theStack:
    re-ACK 6ee32aaaca
  danielabrozzoni:
    ACK 6ee32aaaca

Tree-SHA512: e47fbf471f2909286a6c1c073799ea388b9c19551afcce96cf9af45cc48d25c02f1e48e08861a88b604361e2c107a759d5baf393da8a37360de419f31651758a
2025-06-02 15:24:34 -07:00
Greg Sanders
63091b79e7 test: remove unnecessary -datacarriersize args from tests 2025-05-30 10:14:18 -04:00
Greg Sanders
9f36962b07 policy: uncap datacarrier by default
Datacarrier output script sizes and output counts are now
uncapped by default.

To avoid introducing another startup argument, we modify the
OP_RETURN accounting to "budget" the spk sizes.

If a user has set a custom default, this results in that
budget being spent over the sum of all OP_RETURN outputs'
scripts in the transaction, no longer capping the number
of OP_RETURN outputs themselves. This should allow a
superset of current behavior while respecting the passed
argument in terms of total arbitrary data storage.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2025-05-30 10:12:38 -04:00
Sebastian Falbesoner
b184f5c87c test: update BIP340 test vectors and implementation (variable-length messages)
See https://github.com/bitcoin/bips/pull/1446,
commit 200f9b26fe0a2f235a2af8b30c4be9f12f6bc9cb
2025-05-30 01:47:09 +02:00
Ava Chow
5471e29d05 Merge bitcoin/bitcoin#32304: test: test MAX_SCRIPT_SIZE for block validity
b1ea542ae6 test: test MAX_SCRIPT_SIZE for block validity (Greg Sanders)

Pull request description:

  I don't believe there are direct tests for this.

ACKs for top commit:
  achow101:
    ACK b1ea542ae6
  TheCharlatan:
    ACK b1ea542ae6
  theStack:
    ACK b1ea542ae6

Tree-SHA512: 1d7d3eab9c54977844bf2ca1aa403b070aae0f818db2fb5cae367d1c4d12f1e403b6fdec224af769a2ebb648cbca8bfd0d7df5db2a89fccf256c9c244484eba2
2025-05-29 14:32:10 -07:00
Ava Chow
ce46000712 Merge bitcoin/bitcoin#32509: qa: feature_framework_startup_failures.py fixes & improvements (#30660 follow-up)
bf950c4544 qa: Improve suppressed errors output (Hodlinator)
075352ec8e qa: assert_raises_message() - search in str(e) (Hodlinator)
bd8ebbc4ab qa: Make --timeout-factor=0 result in a smaller factor (Hodlinator)
d8f05e7bf3 qa: Fix dormant bug caused by multiple --tmpdir (Hodlinator)

Pull request description:

  * Handle multiple `--tmpdir` args properly.
  * Handle `--timeout-factor=0` properly (fixes #32506).
  * Improve readability of expected error message (`assert_raises_message()`).
  * Make suppressed error output less confusing (`wait_for_rpc_connection()`).

ACKs for top commit:
  i-am-yuvi:
    re-ACK bf950c4544
  ismaelsadeeq:
    Code review and tested ACK bf950c45
  achow101:
    ACK bf950c4544
  janb84:
    LGTM ACK bf950c4544

Tree-SHA512: 8993f53e962231adef9cad92594dc6a8b8e979b1571d1381cd0fffa1929833b2163c68c03b6a6e29995006a3c3121822ec25ac7725e71ccdab8876ac24aae86c
2025-05-27 14:45:11 -07:00
Ava Chow
012f347685 Merge bitcoin/bitcoin#31375: multiprocess: Add bitcoin wrapper executable
a5ac43d98d doc: Add release notes describing bitcoin wrapper executable (Ryan Ofsky)
258bda80c0 doc: Mention bitcoin wrapper executable in documentation (Ryan Ofsky)
d2739d75c9 build: add bitcoin.exe to windows installer (Sjors Provoost)
ba649c0006 ci: Run multiprocess tests through wrapper executable (Ryan Ofsky)
29bdd743bb test: Support BITCOIN_CMD environment variable (Ryan Ofsky)
9c8c68891b multiprocess: Add bitcoin wrapper executable (Ryan Ofsky)
5076d20fdb util: Add cross-platform ExecVp and GetExePath functions (Ryan Ofsky)

Pull request description:

  Intended to make bitcoin command line features more discoverable and allow installing new multiprocess binaries in libexec/ instead of bin/ so they don't cause confusion.

  Idea and implementation of this were discussed in https://github.com/bitcoin/bitcoin/issues/30983.

  ---

  Initial implementation of this feature is deliberately minimal so the UX can evolve in response to feedback and there are not too many details to debate and discuss in a single PR. But many improvements are possible or planned:

  - Adding manpage and bash completions.
  - Showing nicer error messages that detect if an executable isn't installed and suggest how to fix [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2073194474)
  - Showing wrapper command lines in subcommand in help output [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2077800405). This could be done conditionally as suggested in the comment or be unconditional.
  - Showing wrapper command lines in subcommand error output. There is a bitcoin-cli error pointed out in [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091152243) that is needlessly confusing.
  - Integrating help so `bitcoin help subcommand` invokes `bitcoin subcommand -h`. `bitcoin -h subcommand` should also be supported and be equivalent [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2093116725)
  - Adding support for `bitcoin-util` subcommands. Ideal interface would probably be more like `bitcoin grind` not `bitcoin util grind` but this has been punted for now. Supporting subcommands directly would require some ArgsManager modifications
  - Adding a dedicated python functional test for the wrapper. Right now there is some CI coverage by setting the `BITCOIN_CMD` variable, but this doesn't cover things like the help output and version output, and support for different directory layouts.
  - Better `--multiprocess` (`-m`) / `--monolithic` (`-M`) default selection. Right now, default is monolithic but it probably makes sense to chose more intelligently depending on whether -ipc options are enabled and what binaries are available.
  - Maybe parsing `bitcoin.conf` and supporting options to control wrapper behavior like custom locations or preferences or aliases.
  - Better command command line usability. Allow combining short options like (`-ah`). Allow fuzzy matching of subcommands or suggestions if you misspell. (suggested by stickies in review club)
  - Not directly related to this PR but `bitcoin-cli named` implementation used by the wrapper should do a better job disambiguating named arguments from base64 arguments ending in = as pointed out in [(comment)](https://github.com/bitcoin/bitcoin/pull/31375#discussion_r2091886628)

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722). A review club meeting for it took place in https://bitcoincore.reviews/31375

ACKs for top commit:
  Sjors:
    utACK a5ac43d98d
  achow101:
    ACK a5ac43d98d
  vasild:
    ACK a5ac43d98d
  theStack:
    ACK a5ac43d98d
  ismaelsadeeq:
    fwiw my last review implied an ACK a5ac43d98d
  hodlinator:
    ACK a5ac43d98d

Tree-SHA512: 570e6a4ff8bd79ef6554da3d01f36c0a7c6d2dd7dace8f8732eca98f4a8bc2284474a9beadeba783114fe2f3dd08b2041b3da7753bae0b7f881ec50668cb821f
2025-05-27 12:38:19 -07:00
merge-script
d2c9fc84e1 Merge bitcoin/bitcoin#32533: test: properly check for per-tx sigops limit
7bc64a8859 test: properly check for per-tx sigops limit (Sebastian Falbesoner)

Pull request description:

  Currently the per-tx sigops limit standardness check (bounded by `MAX_STANDARD_TX_SIGOPS_COST`, throwing "bad-txns-too-many-sigops" if exceeded):
  3f83c744ac/src/validation.cpp (L925-L927)

  is only indirectly tested with the much higher per-block consensus limit (`MAX_BLOCK_SIGOPS_COST`):
  3f83c744ac/test/functional/data/invalid_txs.py (L236-L242)

  I.e. an increase in the per-tx limit up to the per-block one would still pass all of our tests. Refine that by splitting up the invalid tx template `TooManySigops` in a per-block and a per-tx template.

  The involved functional tests taking use of these templates are `feature_block.py` and `p2p_invalid_txs.py`. Can be tested by applying e.g.
  ```diff
  diff --git a/src/policy/policy.h b/src/policy/policy.h
  index 2151ec13dd..e5766d2a55 100644
  --- a/src/policy/policy.h
  +++ b/src/policy/policy.h
  @@ -37,7 +37,7 @@ static constexpr unsigned int MIN_STANDARD_TX_NONWITNESS_SIZE{65};
   /** Maximum number of signature check operations in an IsStandard() P2SH script */
   static constexpr unsigned int MAX_P2SH_SIGOPS{15};
   /** The maximum number of sigops we're willing to relay/mine in a single tx */
  -static constexpr unsigned int MAX_STANDARD_TX_SIGOPS_COST{MAX_BLOCK_SIGOPS_COST/5};
  +static constexpr unsigned int MAX_STANDARD_TX_SIGOPS_COST{MAX_BLOCK_SIGOPS_COST/5 + 4};
   /** Default for -incrementalrelayfee, which sets the minimum feerate increase for mempool limiting or replacement **/
   static constexpr unsigned int DEFAULT_INCREMENTAL_RELAY_FEE{1000};
   /** Default for -bytespersigop */
  diff --git a/test/functional/mempool_accept.py b/test/functional/mempool_accept.py
  ```
  where the tests succeed on master, but fail on this PR.

  (Found by diving deeper into the jungle of current sig-ops limit, as preparation for reviewing the [BIP 54](https://github.com/bitcoin/bips/blob/master/bip-0054.md) draft and related preparatory PRs like #32521).

ACKs for top commit:
  fjahr:
    tACK 7bc64a8859
  tapcrafter:
    tACK 7bc64a8859
  darosior:
    ACK 7bc64a8859
  instagibbs:
    crACK 7bc64a8859

Tree-SHA512: 1365409349664a76a1d46b2fa358c0d0609fb17fffdd549423d22b61749481282c928be3c2fb428725735c82d319b4279f703bde01e94e4aec14bab206abb8cf
2025-05-22 10:06:03 -04:00
Matthew Zipkin
f16c8c67bf tests: Expand HTTP coverage to assert libevent behavior
Covers:
- http pipelining
- rpcservertimeout

  Testing this requires adding an option to TestNode to force
  the test framework to establish a new HTTP connection for
  every RPC. Otherwise, attempting to reuse a persistent connection
  would cause framework RPCs during startup and shutdown to fail.

- "chunked" Transfer-Encoding
2025-05-21 10:47:23 -04:00
Sjors Provoost
0a99d99fe4 signet: miner skips PSBT step for OP_TRUE 2025-05-19 09:45:35 +02:00
merge-script
7710a31f0c Merge bitcoin/bitcoin#32452: test: Remove legacy wallet RPC overloads
b104d44227 test: Remove RPCOverloadWrapper (Ava Chow)
4d32c19516 test: Replace importpubkey (Ava Chow)
fe838dd391 test: Replace usage of addmultisigaddress (Ava Chow)
d314207779 test: Replace usage of importaddress (Ava Chow)
fcc457573f test: Replace importprivkey with wallet_importprivkey (Ava Chow)
94c87bbbd0 test: Remove unnecessary importprivkey from wallet_createwallet (Ava Chow)

Pull request description:

  `RPCOverloadWrapper` implemented overloads for legacy wallet only RPCs so that the same function call could be used within tests for both legacy wallets and descriptor wallets. With legacy wallets now removed, there is no need to continue to have these overloads.

  For `importaddress`, `addmultisigaddress`, and `importpubkey`, the uses of these are converted to `importdescriptors`.

  For `importprivkey`, a new helper function `wallet_imporprivkey` is introduced that does what the overload did. This is mainly to reduce verbosity as `importprivkey` was more widely used throughout the tests.

  Some tests that used these RPCs are now also no longer relevant and have been removed.

ACKs for top commit:
  Sjors:
    ACK b104d44227
  pablomartin4btc:
    cr ACK b104d44227
  rkrux:
    ACK b104d44227
  w0xlt:
    ACK b104d44227

Tree-SHA512: ded2f73829e2ce28466d4a9738eb382783ad990daee5d1859dbc4d354e6f8eec0c483ed5ecb1287fe0dd24ac332065b733a30d71b126b841bd7cd49e9a094b6d
2025-05-17 10:19:35 +01:00
Sebastian Falbesoner
7bc64a8859 test: properly check for per-tx sigops limit
Currently the per-tx sigops limit standardness check (bounded by
`MAX_STANDARD_TX_SIGOPS_COST`, throwing "bad-txns-too-many-sigops"
if exceeded) is only indirectly tested with the much higher per-block
consensus limit (`MAX_BLOCK_SIGOPS_COST`), i.e. an increase in the
limit would still pass all tests. Refine that by splitting up the invalid
tx template `TooManySigops` in a per-block and a per-tx one.

The involved functional tests taking use of these templates are
`feature_block.py` and `p2p_invalid_txs.py`.
2025-05-16 16:35:42 +02:00
Hodlinator
bf950c4544 qa: Improve suppressed errors output
Original discussion: https://github.com/bitcoin/bitcoin/pull/30660#discussion_r2080132673
2025-05-16 09:20:51 +02:00
Hodlinator
075352ec8e qa: assert_raises_message() - search in str(e)
repr() is annoying because it requires escaping special characters, use str() instead.

Original discussion: https://github.com/bitcoin/bitcoin/pull/30660#discussion_r2080292165
2025-05-16 09:20:51 +02:00
Hodlinator
bd8ebbc4ab qa: Make --timeout-factor=0 result in a smaller factor
Would otherwise cause an OverflowError in feature_framework_startup_failures.py when calling subprocess.run() with 60 * factor.

Fixes #32506

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2025-05-16 09:20:51 +02:00
merge-script
c779ee3a40 Merge bitcoin/bitcoin#32492: test: add skip_if_running_under_valgrind()
75a185ea3d test: add skip_if_running_under_valgrind() (fanquake)

Pull request description:

  Enable it in the USDT tests. The context (from 0xB10C):

  > every time the tracepoint is reached a SIGTRAP is fired.
  > No matter the tracepoint contents, even with an empty one.
  > Valgrind intercepts SIGTRAP and aborts.

  See discussion in #32374.

ACKs for top commit:
  maflcko:
    lgtm ACK 75a185ea3d
  willcl-ark:
    ACK 75a185ea3d

Tree-SHA512: 7f45c3049ab39cc514024067bd6ac26598e99202c114b48459834c26c2e1273fa58af693878298e628a10c561b954850e49e76b39567b771bb0c0534a063a524
2025-05-15 10:17:50 +01:00
Ava Chow
53eb5593f0 Merge bitcoin/bitcoin#32305: test: add test for decoding PSBT with MuSig2 PSBT key types (BIP 373)
4b24186756 test: add test for decoding PSBT with MuSig2 PSBT key types (BIP 373) (Sebastian Falbesoner)
8ba245cb83 test: add constants for MuSig2 PSBT key types (BIP 373) (Sebastian Falbesoner)

Pull request description:

  This PR is a follow-up to #31247 (see https://github.com/bitcoin/bitcoin/pull/31247#pullrequestreview-2427834909) and adds a functional test for decoding PSBTs (using the `decodepsbt` RPC) with MuSig2 per-input and per-output types. The first commit adds the new MuSig2 key types to the test frameworks and extends the PSBT serialization to cope with lists of bytestrings.

ACKs for top commit:
  achow101:
    ACK 4b24186756
  rkrux:
    re-ACK 4b24186

Tree-SHA512: f12919f71b3fff74df1d7ddaa8db455b1b139f7abd51d7f3fa5d750fc7dd613454b438c4e0dedad679476d414fa1da43ef1121e486b0bdfd97d5ef8bdf37f060
2025-05-14 13:19:43 -07:00
fanquake
75a185ea3d test: add skip_if_running_under_valgrind()
Enable it in the USDT tests. The context (from 0xB10C):

> every time the tracepoint is reached a SIGTRAP is fired.
> No matter the tracepoint contents, even with an empty one.
> Valgrind intercepts SIGTRAP and aborts.

See discussion in #32374.
2025-05-14 10:55:26 +01:00