Simplify the witness malleation test helper by converting the
ValidWitnessMalleatedTx class to a standalone function
build_malleated_tx_package() and updating call sites.
Co-authored-by: rkrux <rkrux.connect@gmail.com>
fa3df52712 bench: Require semicolon after BENCHMARK(foo) (MarcoFalke)
fa8938f08c bench: Remove incorrect __LINE__ in BENCHMARK macro (MarcoFalke)
fa51a28a94 scripted-diff: Remove priority_level from BENCHMARK macro (MarcoFalke)
fa790c3eea bench: Remove -priority-level= option (MarcoFalke)
Pull request description:
The option was added in https://github.com/bitcoin/bitcoin/pull/26158, when the project was using an autotools-based build system. However, in the meantime this option is unused:
* First, commit 27f11217ca removed the option from one CI task
* Then https://github.com/bitcoin/bitcoin/pull/32310 removed the option from CMakeList.txt, because:
* they only run as a sanity check (fastest version)
* no one otherwise runs them, not even CI
* issues have been missed due to this
Finally, after commit 0ad4376a49, I don't see a single reason to keep this option, so remove it.
Also, there is a commit to turn a silent ignore of duplicate bench names into an error.
ACKs for top commit:
achow101:
ACK fa3df52712
l0rinc:
ACK fa3df52712
hebasto:
re-ACK fa3df52712, only suggested changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/34210#pullrequestreview-3652414135).
Tree-SHA512: 68a314bff551fa878196d5a615d41d71e1c8c504135e6fc555659aa9f0c8786957d49ba038448e933554a8bc54caea2ddd7d628042c5627bf3bf37628210f8fb
ce63d37ebe test: use dynamic port allocation to avoid test conflicts (woltx)
Pull request description:
Use `port=0` for dynamic port allocation in test framework components to avoid intermittent "address already in use" errors when running tests concurrently or when ports are stuck in TIME_WAIT state. Example: https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2634509304
Changes:
- Update `socks5.py` and `p2p.py` to support dynamic port allocation
- Convert `feature_proxy.py` and `feature_anchors.py` to use `port=0`
ACKs for top commit:
achow101:
ACK ce63d37ebe
vasild:
ACK ce63d37ebe
mzumsande:
re-ACK ce63d37ebe
Tree-SHA512: 4efcedca3bde209fbd1bdc2a4ae04b7d53515587d86e421ce61064f78c675c71b45d9782b514c5e7cfc0e92842c947d49f7a3fddb03fe619fcdec9b565f0ecbd
7b5d256af4 test: Add bitcoin-chainstate test for assumeutxo functionality (stringintech)
2bc3265649 Fix `ChainstateManager::AddChainstate()` assertion crash (stringintech)
5f3d6bdb66 Add regtest support to bitcoin-chainstate tool (stringintech)
Pull request description:
This PR adds functional test coverage for the bitcoin-chainstate tool loading a datadir initialized with an assumeutxo snapshot.
The PR also includes:
- Fix for assertion crash in `ChainstateManager::AddChainstate()` when `prev_chainstate` has no initialized mempool (required for the test to pass)
- `-regtest` flag support for bitcoin-chainstate to enable the testing
This work started while experimenting with the bitcoin-chainstate tool and how the kernel API (#30595) behaved when loading a datadir containing assumeutxo data, during the time that PR was still under review. sedited suggested opening a PR to add this test coverage.
ACKs for top commit:
achow101:
ACK 7b5d256af4
theStack:
Concept and code-review ACK 7b5d256af4
sedited:
Re-ACK 7b5d256af4
Tree-SHA512: 5d3b0050cf2d53144b5f65451c991d5e212117b4541ae1368ecf58fde5f3cca4f018aad6ae32257b9ebb1c28b926424fbcff496ba5487cdc4eb456cea6db8b24
792e2edf57 p2p: first addr self-announcement in separate msg (0xb10c)
Pull request description:
This makes sure the initial address self-announcement a node sends to a peer happends in a separate P2P message. This has benefits for both inbound and outbound connections:
For inbound connections from a peer to us, previously, we might send the self-announcement along with our response to a GETADDR request. However, the self-announcement might replace an address from the GETADDR response. This isn't clean.
For outbound connections from us to a peer, previously, it could have happend that we send the self-announcement along with other addresses. Since shortly after connection open, the peer might only have one rate-limiting token for us, and the addresses are shuffeld on arrival, it's possible that the self-announcement gets rate-limited. However, note that these rate-limitings seem to be rare in practice.
This is inspired by and based on https://github.com/bitcoin/bitcoin/pull/33699#issuecomment-3462287763. The discussion there should be helpful for reviewers.
ACKs for top commit:
bensig:
ACK 792e2edf57
achow101:
ACK 792e2edf57
fjahr:
Code review ACK 792e2edf57
frankomosh:
Code Review ACK [792e2ed](792e2edf57)
Tree-SHA512: e3d39b1e3ae6208b54df4b36c624a32d70a442e01681f49e0c8a65076a818b5bf203c2e51011dc32edbbe3637b3c0b5f18de26e3461c288aa3806646a209a260
8fb5e5f41d test: check wallet rescan properly in feature_pruning (brunoerg)
9b57c8d2bd test: fix feature_pruning when built without wallet (brunoerg)
Pull request description:
Fixes#34175
In `feature_pruning`, the`wallet_test` doesn't require any specific wallet functionality and this test is important for one of next ones (`test_scanblocks_pruned`). The reason is that it synchronizes the node 5 and, without this sync, `test_scanblocks_pruned` will fail since we expect `scanblocks` to fail due to `Block not available (pruned data)` and it doesn't happen without this sync.
ACKs for top commit:
achow101:
ACK 8fb5e5f41d
furszy:
utACK 8fb5e5f41d
musaHaruna:
Tested ACK [8fb5e5f](8fb5e5f41d)
w0xlt:
ACK 8fb5e5f41d
Tree-SHA512: 812afbf4343a7493e2169eb6735fce25692d5cb19972abafc772b8c05a64b9c7027f6675cd084f345977e916e62a722d671f90831bbdc51683e0cd253fa482f0
de4242f474 refactor: Use reference for chain_start in HeadersSyncState (Daniela Brozzoni)
e37555e540 refactor: Use initializer list in CompressedHeader (Daniela Brozzoni)
0488bdfefe refactor: Remove unused parameter in ReportHeadersPresync (Daniela Brozzoni)
256246a9fa refactor: Remove redundant parameter from CheckHeadersPoW (Daniela Brozzoni)
ca0243e3a6 refactor: Remove useless CBlock::GetBlockHeader (Pieter Wuille)
4568652222 refactor: Use std::span in HasValidProofOfWork (Daniela Brozzoni)
4066bfe561 refactor: Compute work from headers without CBlockIndex (Daniela Brozzoni)
0bf6139e19 p2p: Avoid an IsAncestorOfBestHeaderOrTip call (Pieter Wuille)
Pull request description:
This is a partial* revival of #25968
It contains a list of most-unrelated simplifications and optimizations to the code merged in #25717:
- Avoid an IsAncestorOfBestHeaderOrTip call: Just don't call this function when it won't have any effect.
- Compute work from headers without CBlockIndex: Avoid the need to construct a CBlockIndex object just to compute work for a header, when its nBits value suffices for that. Also use some Spans where possible.
- Remove useless CBlock::GetBlockHeader: There is no need for a function to convert a CBlock to a CBlockHeader, as it's a child class of it.
It also contains the following code cleanups, which were suggested by reviewers in #25968:
- Remove redundant parameter from CheckHeadersPoW: No need to pass consensusParams, as CheckHeadersPow already has access to m_chainparams.GetConsensus()
- Remove unused parameter in ReportHeadersPresync
- Use initializer list in CompressedHeader, also make GetFullHeader const
- Use reference for chain_start in HeadersSyncState: chain_start can never be null, so it's better to pass it as a reference rather than a raw pointer
*I decided to leave out three commits that were in #25968 (4e7ac7b94d, ab52fb4e95, 7f1cf440ca), since they're a bit more involved, and I'm a new contributor. If this PR gets merged, I'll comment under #25968 to note that these three commits are still up for grabs :)
ACKs for top commit:
l0rinc:
ACK de4242f474
polespinasa:
re-ACK de4242f474
sipa:
ACK de4242f474
achow101:
ACK de4242f474
hodlinator:
re-ACK de4242f474
Tree-SHA512: 1de4f3ce0854a196712505f2b52ccb985856f5133769552bf37375225ea8664a3a7a6a9578c4fd461e935cd94a7cbbb08f15751a1da7651f8962c866146d9d4b
f62568c97c ci: Install `pyzmq` for functional tests on Windows (Hennadii Stepanov)
Pull request description:
This PR enables `interface_zmq.py` on Windows by installing the required `pyzmq` package.
ACKs for top commit:
maflcko:
lgtm ACK f62568c97c
Tree-SHA512: a57728ef82f1ff15a3ed1a659e98ad2d10f69693772649eaa97e1803e18f2f68e99782be3cb149322d1bf72607ec103e1837db1db39e9bd8c2a0a0863463635d
7fc465ece8 doc: fix incorrect description of `PackageMempoolChecks` (ismaelsadeeq)
1412b779ad refactor: execute `PackageMempoolChecks` during package rbf only (ismaelsadeeq)
Pull request description:
This is a simple PR that fixes the incorrect description of what is done in `PackageMempoolChecks`
> // Enforce package mempool ancestor/descendant limits (distinct from individual
> // ancestor/descendant limits done in PreChecks) and run Package RBF checks.
After cluster mempool, we no longer enforce ancestor/descendant limits in both `PreChecks` and `PackageMempoolChecks`; instead, cluster limit is enforced in `PackageMempoolChecks`.
This PR fixes the incorrect comment by;
- Making it clear why it is necessary to have two calls of `CheckMempoolPolicyLimts` in both `PackageMempoolChecks` and after in `AcceptMultipleTransactionsInternal` by executing `PackageMempoolChecks` only during package RBF only. No need to jump into the next subroutine when there is no conflict.
- Renames `PackageMempoolChecks` to `PackageRBFChecks`; the method name is self-explanatory now, hence no need for a description comment.
ACKs for top commit:
yashbhutwala:
ACK 7fc465ece8
instagibbs:
ACK 7fc465ece8
glozow:
utACK 7fc465ece8
Tree-SHA512: 38655f9d05be54cadd224fad376da9871a85efc7801306b58d4f51aee658036cdce2ab406143a3439d7211fc9bb0fc86bd330852e8926d79660944872b8fae8d
fa2959e16d test: Fail on self-check warnings in test_runner.py (MarcoFalke)
Pull request description:
I don't see a reason to start running the tests, if the test_runner detects warnings during the self-check.
Usually, this will just lead to a possibly confusing test failure after some wasted time anyway.
So just fail fast before even running any tests.
If there was a reason to ignore the warnings, a new option could trivially be added:
```py
parser.add_argument("--ignore-self-check-warnings", dest="ignore_warnings", default=False, action="store_true",
help="Ignore test runner warnings about self-checks before running the tests")
```
However, I don't see the need.
ACKs for top commit:
hodlinator:
utACK fa2959e16d
Tree-SHA512: 3f2e1af9ba06cd805c69e8c40149214a8a13af286ba6315c346e23d0f1ddae8e850103b6967d2d9799a095f6a8ec8802c3c773af8d7123598e8887f56c764fb5
fa5032f0e4 ci: Run feature_unsupported_utxo_db.py on Windows (MarcoFalke)
Pull request description:
The feature_unsupported_utxo_db.py test is skipped on Windows, because the ancient exe used in the test does not support UTF-8.
One way to fix this could be to rework the test_runner to supply an ASCII temp dir. However, this is tedious and probably not worth it for a test that is close to being deleted: The test only checks that a `-reindex-chainstate` is sufficient, but at this point, it seems fine to accept users having to go through a full `-reindex`, or even full re-download.
I am not deleting it just yet, because it is still minimally useful: E.g. https://github.com/bitcoin/bitcoin/pull/31674#issuecomment-2599106040. Though, I think the next time an issue pops up with this test, it can be considered for deletion.
ACKs for top commit:
hebasto:
ACK fa5032f0e4.
Tree-SHA512: eb3be204a1238caa93c7c2947171ed3c1c2717ccf95e5a58102791dbecfcd8710e0d89f001e266c192734bb4326ec6ca6e0df3128ad0d1e76cfe0f34f486a08e
fa1d17d56c refactor: Use uint64_t over size_t for serialize corruption check in fees.dat (MarcoFalke)
Pull request description:
Serialization should not behave differently on different architectures. See also the related commit 3789215f73.
However, on fees.dat file corruption, 32-bit builds may run into an unsigned integer overflow and report the wrong corruption reason, or may even silently continue after the corruption.
This is a bit hard to reproduce, because 32-bit platforms are rare and most of them don't support running the unsigned integer overflow sanitizer. So the possible options to reproduce are:
* Run on armhf and manually annotate the code to detect the overflow
* Run on i386 with the integer sanitizer (possibly via `podman run -it --rm --platform linux/i386 'debian:trixie'`)
* Run the integer sanitizer on any 64-bit platform and manually replace type in the affected line by `uint32_t`
Afterwards, the steps to reproduce are:
```
export DEBIAN_FRONTEND=noninteractive && apt update && apt install curl wget htop git vim ccache -y && git clone https://github.com/bitcoin/bitcoin.git --depth=1 ./b-c && cd b-c && apt install build-essential cmake pkg-config python3-zmq libzmq3-dev libevent-dev libboost-dev libsqlite3-dev systemtap-sdt-dev libcapnp-dev capnproto libqrencode-dev qt6-tools-dev qt6-l10n-tools qt6-base-dev clang llvm libc++-dev libc++abi-dev -y
cmake -B ./bld-cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_COMPILER='clang' -DCMAKE_CXX_COMPILER='clang++' -DSANITIZERS=undefined,integer,float-divide-by-zero --preset=dev-mode
cmake --build ./bld-cmake --parallel $(nproc)
curl -fLO '6074731370'
UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1" FUZZ=policy_estimator_io ./bld-cmake/bin/fuzz ./607473137013139e3676e30ec4b29639e673fa9b
```
The output will be something like:
```
/b-c/src/policy/fees/block_policy_estimator.cpp:448:25: runtime error: unsigned integer overflow: 346685954 * 219 cannot be represented in type 'unsigned int'
#0 0x5b0b1bbe in TxConfirmStats::Read(AutoFile&, unsigned int) /b-c/bld-cmake/src/./policy/fees/block_policy_estimator.cpp:448:25
#1 0x5b0b7d3f in CBlockPolicyEstimator::Read(AutoFile&) /b-c/bld-cmake/src/./policy/fees/block_policy_estimator.cpp:1037:29
#2 0x592a9783 in policy_estimator_io_fuzz_target(std::span<unsigned char const, 4294967295u>) /b-c/bld-cmake/src/test/fuzz/./test/fuzz/policy_estimator_io.cpp:32:32
#3 0x5896ba8e in void std::__invoke_impl<void, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>>(std::__invoke_other, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>&&) /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:61:14
#4 0x5896b8eb in std::enable_if<is_invocable_r_v<void, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>>, void>::type std::__invoke_r<void, void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>>(void (*&)(std::span<unsigned char const, 4294967295u>), std::span<unsigned char const, 4294967295u>&&) /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/invoke.h:111:2
#5 0x5896b44b in std::_Function_handler<void (std::span<unsigned char const, 4294967295u>), void (*)(std::span<unsigned char const, 4294967295u>)>::_M_invoke(std::_Any_data const&, std::span<unsigned char const, 4294967295u>&&) /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/std_function.h:290:9
#6 0x59845c95 in std::function<void (std::span<unsigned char const, 4294967295u>)>::operator()(std::span<unsigned char const, 4294967295u>) const /usr/lib/gcc/i686-linux-gnu/14/../../../../include/c++/14/bits/std_function.h:591:9
#7 0x5983a0da in test_one_input(std::span<unsigned char const, 4294967295u>) /b-c/bld-cmake/src/test/fuzz/util/./test/fuzz/fuzz.cpp:88:5
#8 0x5983cb80 in main /b-c/bld-cmake/src/test/fuzz/util/./test/fuzz/fuzz.cpp:271:13
#9 0xf75aecc2 (/lib/i386-linux-gnu/libc.so.6+0x24cc2) (BuildId: 2dc5f2945fad35c1b07d1a5a32520b3c41afaa75)
#10 0xf75aed87 in __libc_start_main (/lib/i386-linux-gnu/libc.so.6+0x24d87) (BuildId: 2dc5f2945fad35c1b07d1a5a32520b3c41afaa75)
#11 0x58932db6 in _start (/b-c/bld-cmake/bin/fuzz+0x235ddb6) (BuildId: 7d8d83a77923f14e99c0de64acbc5f5bfc2cce9b)
SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow /b-c/src/policy/fees/block_policy_estimator.cpp:448:25
```
Note: This is marked a "refactor", because the code change does not affect 64-bit builds, and on the still remaining rare 32-bit builds today it is extremely unlikely to happen in production.
ACKs for top commit:
bensig:
ACK fa1d17d56c
ismaelsadeeq:
utACK fa1d17d56c
luke-jr:
Also, utACK fa1d17d56c as an improvement.
Tree-SHA512: 696bf8e0dbe4777c84cb90e313c7f8f9ee90d4b3e64de1222f8472b2d9d0f3a0f6f027fda743dd6ca8c6aab94f404db7a65bb562a76000d9c33a8a39de28d8d4
2f5b1c5f80 psbt: Fix `PSBTInputSignedAndVerified` bounds `assert` (Lőrinc)
Pull request description:
This PR fixes an off-by-one in a debug assertion in `PSBTInputSignedAndVerified`.
The function indexes `psbt.inputs[input_index]`, so the assertion must not allow indexing at `psbt.inputs.size()`.
Found during review: https://github.com/bitcoin/bitcoin/pull/31650#discussion_r2685892867
ACKs for top commit:
optout21:
utACK 2f5b1c5f80
maflcko:
lgtm ACK 2f5b1c5f80
achow101:
ACK 2f5b1c5f80
Tree-SHA512: cec613a9a38358d5caa243197d746baa129aebfd7fe697689f28e652f94c4683873c4676d5eb2eb909ea19de5e5f6e54ecc5f3162384a48f6f38a59273667689
facaf56214 contrib: Remove unused functions (MarcoFalke)
Pull request description:
* `remove_files` is unused since 5668c6473a
* `download_lines_with_urllib` is unused since it was introduced in 37c9fb7a59
* `determine_wellknown_cmd` is unused since 76c090145e
ACKs for top commit:
fjahr:
ACK facaf56214
bensig:
ACK facaf56214 straightforward
l0rinc:
Reproduced it locally, ACK facaf56214
Tree-SHA512: c7d4880944b57108429e31541f1b97a98101cc06f1be716d6028f4193b6f087d9af06fa87e95ab224e027d574c6bd793cf06e540cdb8997805e1470e0c4e77a2
2a746500fa ci: migrate some jobs to Debian Trixie, use GCC 14 (fanquake)
fb0e6edfe8 guix: Apply SSA generation patch to maintain determinism (Mara van der Laan)
34909799fe guix: use GCC 14.3.0 over 13.3.0 (fanquake)
47be9122a7 guix: disable gprofng in GCC (fanquake)
ea29329eb7 guix: build GCC with --enable-host-bind-now (fanquake)
6f54e267d0 guix: disable libquadmath in GCC (fanquake)
7735901ed2 guix: disable building libgomp in GCC (fanquake)
Pull request description:
Switching to using GCC 14.x for release builds has come up multiple times recently. It will eventually be needed for #25573, and could also be useful for #30210.
ACKs for top commit:
hebasto:
ACK 2a746500fa. I have reviewed the code and it looks OK. The new GCC patch looks reasonable.
theuni:
utACK 2a746500fa
sedited:
ACK 2a746500fa
Tree-SHA512: 56912bed19386f06d52fb94e0ef6d96f5415ab2de8b5e94890806d7cc0b937a3c4b11cc161aa2e06ca2fd3c392ef7501c91688e0897e1c1c51aafa963f3e50d9
ab41492c6b test: Prevent loop from running out of utxos in bip68 test (Fabian Jahr)
Pull request description:
This tries to fix#34205
I stared at the test code quite a bit and initially suspected some `MiniWallet` internals to be the issue but I think that was the wrong direction and there is simply a very small chance that the loop in `test_sequence_lock_confirmed_inputs` runs out of available utxos: We are starting out with 200-250 utxos and run the loop 400 times. If a transaction is accepted it could have up to 10 inputs but it always has only one output, so the pool is depleting in this case. And it's actually even worse because the output produced is not recognized as spendable by the `MiniWallet` because it is not using the correct output script. However, only a small fraction of transactions are actually accepted, which is why this issue almost never occurs. I did some extra printing and usually we end up with >100 utxos still available by the end of the test. But there is a small chance that too many transactions are accepted and then we can run out of utxos.
I considered two fixes: The first was a break at the beginning of the loop `if available_utxos == 0: break`, this would work fine but I went with the second option: Simply creating the output with the correct output script so that `MiniWallet` recognizes it as spendable. This minimal replentishment of available utxos ensures that at worst we should get a few 1 input, 1 ouput transactions by the end but we should never run out of available utxos. I didn't look back in history but I suspect that this is how it was intended before `MiniWallet` introduced.
Also moves the `random` import in the same function to the top of the file.
ACKs for top commit:
maflcko:
lgtm ACK ab41492c6b
bensig:
ACK ab41492c6b
darosior:
ACK ab41492c6b
Tree-SHA512: d3ce56b669d011257a4a6967923f56011dbd03362576f564b29464639391851a09113f84b5ca2902911be7aa0923ccc9f402d13e6d673fd089dfe2b1f113ae4d
fa8d56f9f0 fuzz: Reject too large descriptor leaf sizes in scriptpubkeyman target (MarcoFalke)
fabac1b395 fuzz: Reject some more "expensive" descriptors in the scriptpubkeyman target (MarcoFalke)
333333356f fuzz: [refactor] Use std::span over FuzzBufferType in descriptor utils (MarcoFalke)
Pull request description:
Accepting "expensive" fuzz inputs which have no real use-case is problematic, because it prevents the fuzz engine from spending time on the next useful fuzz input.
Also, this may lead to problems, where the fuzz target can not be run at all on some platforms. See https://github.com/bitcoin/bitcoin/issues/34110.
Fixes https://github.com/bitcoin/bitcoin/issues/34110 by rejecting those useless and expensive inputs (via the third commit)
Can be tested by running the input and checking the time before and after the changes here:
```
curl -fLO '1cf91e0c6b'
FUZZ=scriptpubkeyman time ./bld-cmake/bin/fuzz ./1cf91e0c6bfff9dafcd4db5b0ba36b1e906f4cf5
```
Also, the second commit fixes https://github.com/bitcoin/bitcoin/issues/31066.
ACKs for top commit:
brunoerg:
code review ACK fa8d56f9f0
marcofleon:
ACK fa8d56f9f0
sipa:
ACK fa8d56f9f0
Tree-SHA512: e683cb89c3047358add438508c173f1cf647827bcadc3564ad42c757e4c99b8e9b777213fd38ebeb46f4c89a72363e0642f47435e20df3960eaeb5b8257dbd32
6bb66fcccb test: Improve code coverage for pubkey checks (billymcbip)
Pull request description:
Cover these branches in `IsCompressedOrUncompressedPubKey` and `IsCompressedPubKey`:
- `Non-canonical public key: invalid length for uncompressed key`
- `Non-canonical public key: invalid length for compressed key`
- `Non-canonical public key: invalid prefix for compressed key`
See the missed branches here: https://maflcko.github.io/b-c-cov/total.coverage/src/script/interpreter.cpp.gcov.html
`script_tests` succeed on my end.
ACKs for top commit:
maflcko:
ACK 6bb66fcccb🌑
rkrux:
code review ACK 6bb66fcccb
darosior:
ACK 6bb66fcccb
Tree-SHA512: f9b8acdc8bbe95559d594e74ed721d27be715754717b1557796168a6e81ce56d5bc20c40da4c0906ef9e1edcd88f202f000e34d8331d9be8d2694067a98996c6
997e7b4d7c init: Fix non-zero code on interrupt (sedited)
Pull request description:
Reported by dergoegge on irc.
An interrupt does not create a failure exit code during normal operation. This should also be the case when interrupt is triggered during initialization. However a failure exit code is currently returned if an interrupt occurs during init. Fix this by making `AppInitMain` return true instead of false on interrupt, which further up the call stack currently sets the `EXIT_FAILURE` code. Also add a check for the interrupt condition during GUI startup. Returning `EXIT_SUCCESS` seems to be the usual behaviour for daemons, see the discussion on IRC for this: https://www.erisian.com.au/bitcoin-core-dev/log-2026-01-08.html#l-146 .
Best reviewed with `--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`.
ACKs for top commit:
maflcko:
review ACK 997e7b4d7c🔺
janb84:
ACK 997e7b4d7c
dergoegge:
utACK 997e7b4d7c
Tree-SHA512: c9542e95d9312567e029426a329144b5bc638d8ebc9c966e0246c1bb728d40f56ca425b00c446f5d238067e629c2337d0fe78bcc5a8760424d2ec38a5578e115
8ac134be5e contrib: verify-commits sha1 exceptions (Ava Chow)
Pull request description:
Allow specific commits to not require the sha1 check.
Fixes the CI for aeaa67a9ea which uses SHA1 somewhere in the key's self signatures.
ACKs for top commit:
dergoegge:
utACK 8ac134be5e
darosior:
utACK 8ac134be5e. Looks good to me provided we want to avoid the temporary red CI on master branch.
Tree-SHA512: cafd080ee35e4f735cbe9281ac2a73488c2f9e4acccd39563e933a3775def35b6e8080a6a772e45e64d90275013e430765a29e87116130ff456257c5ec8a2810
c5825d4b7f qa: Require `--exclude` for each excluded test (Hennadii Stepanov)
Pull request description:
This PR allows a long `--exclude ...` argument in the `test/functional/test_runner.py` invocation to be split across multiple lines, with optional per-line explanatory comments. I found this useful for the CI scripts in https://github.com/hebasto/bitcoin-core-nightly.
ACKs for top commit:
l0rinc:
tested ACK c5825d4b7f
maflcko:
review ACK c5825d4b7f🛄
achow101:
ACK c5825d4b7f
rkrux:
ACK c5825d4b7f
Tree-SHA512: bcf42848516197978b65df8a8bc68e036a62c9afc6158274eac74a325dc01991eb063a042f940c53ea15a7feb18d4bdfc45d8c71f0ef20c76140b12e07ba3ac5
- We no longer enforce ancestor/descendant count limit
in both PreChecks and PackageMempoolChecks.
- This commit fixes the incorrect comment by just renaming
`PackageMempoolChecks` to `PackageRBFChecks`
- The method name is self explanatory now; hence no need
for a description comment.
48f57bb35b mining: add new getCoinbaseTx() returning a struct (Sjors Provoost)
d59b4cdb57 mining: rename getCoinbaseTx() to ..RawTx() (Sjors Provoost)
Pull request description:
The first commit renames `getCoinbaseTx()` to `getCoinbaseRawTx()` to reflect that it returns a serialised transaction. This does not impact IPC clients, because they do not use the function name.
The second commit then introduces a replacement `getCoinbase()` that provides a struct with everything clients need to construct a coinbase. This avoids clients having to parse and manipulate our dummy transaction.
Deprecate but don't remove `getCoinbaseRawTx()`, `getCoinbaseCommitment()` and `getWitnessCommitmentIndex()`.
After this change we can drop these deprecated methods, which in turn would allow us to clear the dummy transaction from the `getBlock()` result. But that is left for a followup to keep this PR focussed. See https://github.com/Sjors/bitcoin/pull/106 for an approach.
Expand the `interface_ipc.py` functional test to document its usage.
Can be tested using:
- https://github.com/stratum-mining/sv2-tp/pull/59
ACKs for top commit:
ryanofsky:
Code review ACK 48f57bb35b. Just rebased and addressed comments and dropped coinbase tx "template" suffix, which is a nice change
ismaelsadeeq:
code review ACK 48f57bb35b
vasild:
ACK 48f57bb35b
Tree-SHA512: c4f1d752777fb3086a1a0b7b8b06e4205dbe2f3adb41f218855ad1dee952adccc263cf82acd3bf9300cc83c2c64cebd2b27f66a69beee32d325b9a85e3643b0d
This makes the code more consistent.
Also, use "using BenchFunction = ..." while touching the header.
Also, fixup the whitespace after and earlier scripted-diff.
Duplicate benchmarks with the same name are not supported. Expanding the
name with __LINE__ is confusing and brittle, because it makes duplication
bugs silent.
Fix this twofold:
* By enforcing unique benchmarks at compile-time and link-time. For
example, a link failure may now look like:
"mold: error: duplicate symbol: bench_runner_AddrManAdd"
* By enforcing unique benchmarks at run-time. This should never happen,
due to the build-failure, but a failure may look like:
"Assertion `benchmarks().try_emplace(std::move(name), std::move(func)).second' failed."
8937221304 doc: add release notes for 29415 (Vasil Dimov)
582016fa5f test: add unit test for the private broadcast storage (Vasil Dimov)
e74d54e048 test: add functional test for private broadcast (Vasil Dimov)
818b780a05 rpc: use private broadcast from sendrawtransaction RPC if -privatebroadcast is ON (Vasil Dimov)
eab595f9cf net_processing: retry private broadcast (Vasil Dimov)
37b79f9c39 net_processing: stop private broadcast of a transaction after round-trip (Vasil Dimov)
2de53eee74 net_processing: handle ConnectionType::PRIVATE_BROADCAST connections (Vasil Dimov)
30a9853ad3 net_processing: move a debug check in VERACK processing earlier (Vasil Dimov)
d1092e5d48 net_processing: modernize PushNodeVersion() (Vasil Dimov)
9937a12a2f net_processing: move the debug log about receiving VERSION earlier (Vasil Dimov)
a098f37b9e net_processing: reorder the code that handles the VERSION message (Vasil Dimov)
679ce3a0b8 net_processing: store transactions for private broadcast in PeerManager (Vasil Dimov)
a3faa6f944 node: extend node::TxBroadcast with a 3rd option (Vasil Dimov)
95c051e210 net_processing: rename RelayTransaction() to better describe what it does (Vasil Dimov)
bb49d26032 net: implement opening PRIVATE_BROADCAST connections (Vasil Dimov)
01dad4efe2 net: introduce a new connection type for private broadcast (Vasil Dimov)
94aaa5d31b init: introduce a new option to enable/disable private broadcast (Vasil Dimov)
d6ee490e0a log: introduce a new category for private broadcast (Vasil Dimov)
Pull request description:
_Parts of this PR are isolated in independent smaller PRs to ease review:_
* [x] _https://github.com/bitcoin/bitcoin/pull/29420_
* [x] _https://github.com/bitcoin/bitcoin/pull/33454_
* [x] _https://github.com/bitcoin/bitcoin/pull/33567_
* [x] _https://github.com/bitcoin/bitcoin/pull/33793_
---
To improve privacy, broadcast locally submitted transactions (from the `sendrawtransaction` RPC) to the P2P network only via Tor or I2P short-lived connections, or to IPv4/IPv6 peers but through the Tor network.
* Introduce a new connection type for private broadcast of transactions with the following properties:
* started whenever there are local transactions to be sent
* opened to Tor or I2P peers or IPv4/IPv6 via the Tor proxy
* opened regardless of max connections limits
* after handshake is completed one local transaction is pushed to the peer, `PING` is sent and after receiving `PONG` the connection is closed
* ignore all incoming messages after handshake is completed (except `PONG`)
* Broadcast transactions submitted via `sendrawtransaction` using this new mechanism, to a few peers. Keep doing this until we receive back this transaction from one of our ordinary peers (this takes about 1 second on mainnet).
* The transaction is stored in peerman and does not enter the mempool.
* Once we get an `INV` from one of our ordinary peers, then the normal flow executes: we request the transaction with `GETDATA`, receive it with a `TX` message, put it in our mempool and broadcast it to all our existent connections (as if we see it for the first time).
* After we receive the full transaction as a `TX` message, in reply to our `GETDATA` request, only then consider the transaction has propagated through the network and remove it from the storage in peerman, ending the private broadcast attempts.
The messages exchange should look like this:
```
tx-sender >--- connect -------> tx-recipient
tx-sender >--- VERSION -------> tx-recipient (dummy VERSION with no revealing data)
tx-sender <--- VERSION -------< tx-recipient
tx-sender <--- WTXIDRELAY ----< tx-recipient (maybe)
tx-sender <--- SENDADDRV2 ----< tx-recipient (maybe)
tx-sender <--- SENDTXRCNCL ---< tx-recipient (maybe)
tx-sender <--- VERACK --------< tx-recipient
tx-sender >--- VERACK --------> tx-recipient
tx-sender >--- INV/TX --------> tx-recipient
tx-sender <--- GETDATA/TX ----< tx-recipient
tx-sender >--- TX ------------> tx-recipient
tx-sender >--- PING ----------> tx-recipient
tx-sender <--- PONG ----------< tx-recipient
tx-sender disconnects
```
Whenever a new transaction is received from `sendrawtransaction` RPC, the node will send it to a few (`NUM_PRIVATE_BROADCAST_PER_TX`) recipients right away. If after some time we still have not heard anything about the transaction from the network, then it will be sent to 1 more peer (see `PeerManagerImpl::ReattemptPrivateBroadcast()`).
A few considerations:
* The short-lived private broadcast connections are very cheap and fast wrt network traffic. It is expected that some of those peers could blackhole the transaction. Just one honest/proper peer is enough for successful propagation.
* The peers that receive the transaction could deduce that this is initial transaction broadcast from the transaction originator. This is ok, they can't identify the sender.
---
<details>
<summary>How to test this?</summary>
Thank you, @stratospher and @andrewtoth!
Start `bitcoind` with `-privatebroadcast=1 -debug=privatebroadcast`.
Create a wallet and get a new address, go to the Signet faucet and request some coins to that address:
```bash
build/bin/bitcoin-cli -chain="signet" createwallet test
build/bin/bitcoin-cli -chain="signet" getnewaddress
```
Get a new address for the test transaction recipient:
```bash
build/bin/bitcoin-cli -chain="signet" loadwallet test
new_address=$(build/bin/bitcoin-cli -chain="signet" getnewaddress)
```
Create the transaction:
```bash
# Option 1: `createrawtransaction` and `signrawtransactionwithwallet`:
txid=$(build/bin/bitcoin-cli -chain="signet" listunspent | jq -r '.[0] | .txid')
vout=$(build/bin/bitcoin-cli -chain="signet" listunspent | jq -r '.[0] | .vout')
echo "txid: $txid"
echo "vout: $vout"
tx=$(build/bin/bitcoin-cli -chain="signet" createrawtransaction "[{\"txid\": \"$txid\", \"vout\": $vout}]" "[{\"$new_address\": 0.00001000}]" 0 false)
echo "tx: $tx"
signed_tx=$(build/bin/bitcoin-cli -chain="signet" signrawtransactionwithwallet "$tx" | jq -r '.hex')
echo "signed_tx: $signed_tx"
# OR Option 2: `walletcreatefundedpsbt` and `walletprocesspsbt`:
# This makes it not have to worry about inputs and also automatically sends back change to the wallet.
# Start `bitcoind` with `-fallbackfee=0.00003000` for instance for 3 sat/vbyte fee.
psbt=$(build/bin/bitcoin-cli -chain="signet" walletcreatefundedpsbt "[]" "[{\"$new_address\": 0.00001000}]" | jq -r '.psbt')
echo "psbt: $psbt"
signed_tx=$(build/bin/bitcoin-cli -chain="signet" walletprocesspsbt "$psbt" | jq -r '.hex')
echo "signed_tx: $signed_tx"
```
Finally, send the transaction:
```bash
raw_tx=$(build/bin/bitcoin-cli -chain="signet" sendrawtransaction "$signed_tx")
echo "raw_tx: $raw_tx"
```
</details>
---
<details>
<summary>High-level explanation of the commits</summary>
* New logging category and config option to enable private broadcast
* `log: introduce a new category for private broadcast`
* `init: introduce a new option to enable/disable private broadcast`
* Implement the private broadcast connection handling on the `CConnman` side:
* `net: introduce a new connection type for private broadcast`
* `net: implement opening PRIVATE_BROADCAST connections`
* Prepare `BroadcastTransaction()` for private broadcast requests:
* `net_processing: rename RelayTransaction to better describe what it does`
* `node: extend node::TxBroadcast with a 3rd option`
* `net_processing: store transactions for private broadcast in PeerManager`
* Implement the private broadcast connection handling on the `PeerManager` side:
* `net_processing: reorder the code that handles the VERSION message`
* `net_processing: move the debug log about receiving VERSION earlier`
* `net_processing: modernize PushNodeVersion()`
* `net_processing: move a debug check in VERACK processing earlier`
* `net_processing: handle ConnectionType::PRIVATE_BROADCAST connections`
* `net_processing: stop private broadcast of a transaction after round-trip`
* `net_processing: retry private broadcast`
* Engage the new functionality from `sendrawtransaction`:
* `rpc: use private broadcast from sendrawtransaction RPC if -privatebroadcast is ON`
* New tests:
* `test: add functional test for private broadcast`
* `test: add unit test for the private broadcast storage`
</details>
---
**This PR would resolve the following issues:**
https://github.com/bitcoin/bitcoin/issues/3828 Clients leak IPs if they are recipients of a transaction
https://github.com/bitcoin/bitcoin/issues/14692 Can't configure bitocoind to only send tx via Tor but receive clearnet transactions
https://github.com/bitcoin/bitcoin/issues/19042 Tor-only transaction broadcast onlynet=onion alternative
https://github.com/bitcoin/bitcoin/issues/24557 Option for receive events with all networks, but send transactions and/or blocks only with anonymous network[s]?
https://github.com/bitcoin/bitcoin/issues/25450 Ability to broadcast wallet transactions only via dedicated oneshot Tor connections
https://github.com/bitcoin/bitcoin/issues/32235 Tor: TX circuit isolation
**Issues that are related, but (maybe?) not to be resolved by this PR:**
https://github.com/bitcoin/bitcoin/issues/21876 Broadcast a transaction to specific nodes
https://github.com/bitcoin/bitcoin/issues/28636 new RPC: sendrawtransactiontopeer
---
Further extensions:
* Have the wallet do the private broadcast as well, https://github.com/bitcoin/bitcoin/issues/11887 would have to be resolved.
* Have the `submitpackage` RPC do the private broadcast as well, [draft diff in the comment below](https://github.com/bitcoin/bitcoin/pull/29415#pullrequestreview-2972293733), thanks ismaelsadeeq!
* Add some stats via RPC, so that the user can better monitor what is going on during and after the broadcast. Currently this can be done via the debug log, but that is not convenient.
* Make the private broadcast storage, currently in peerman, persistent over node restarts.
* Add (optional) random delay before starting to broadcast the transaction in order to avoid correlating unrelated transactions based on the time when they were broadcast. Suggested independently of this PR [here](https://github.com/bitcoin/bitcoin/issues/30471).
* Consider periodically sending transactions that did not originate from the node as decoy, discussed [here](https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2035414972).
* Consider waiting for peer's FEEFILTER message and if the transaction that was sent to the peer is below that threshold, then assume the peer is going to drop it. Then use this knowledge to retry more aggressively with another peer, instead of the current 10 min. See [comment below](https://github.com/bitcoin/bitcoin/pull/29415#issuecomment-3258611648).
* It may make sense to be able to override the default policy -- eg so submitrawtransaction can go straight to the mempool and relay, even if txs are normally privately relayed. See [comment below](https://github.com/bitcoin/bitcoin/pull/29415#issuecomment-3427086681).
* As a side effect we have a new metric available - the time it takes for a transaction to reach a random node in the network (from the point of view of the private broadcast recipient the tx originator is a random node somewhere in the network). This can be useful for monitoring, unrelated to privacy characteristics of this feature.
---
_A previous incarnation of this can be found at https://github.com/bitcoin/bitcoin/pull/27509. It puts the transaction in the mempool and (tries to) hide it from the outside observers. This turned out to be too error prone or maybe even impossible._
ACKs for top commit:
l0rinc:
code review diff ACK 8937221304
andrewtoth:
ACK 8937221304
pinheadmz:
ACK 8937221304
w0xlt:
ACK 8937221304 with nit https://github.com/bitcoin/bitcoin/pull/29415#discussion_r2654849875
mzumsande:
re-ACK 8937221304
Tree-SHA512: d51dadc865c2eb080c903cbe2f669e69a967e5f9fc64e9a20a68f39a67bf0db6ac2ad682af7fa24ef9f0942a41c89959341a16ba7b616475e1c5ab8e563b9b96
44b12cdb11 doc: add 433 (Pay to Anchor) to bips.md (Sebastian Falbesoner)
Pull request description:
See PR https://github.com/bitcoin/bips/pull/1982 (merged recently) and #30352 (first released in v28.0).
ACKs for top commit:
instagibbs:
ACK 44b12cdb11
glozow:
ACK 44b12cdb11
Tree-SHA512: 0389bfd9b57a354ef15eff726783c83969586a73de52d27b6d3b042dba14c1361af29303b6ce42330a841310811a81c5e45b5c7d25e307787f1cfbfd75ab3e62
Use port=0 for dynamic port allocation in test framework components
to avoid "address already in use" errors from concurrent tests or
ports stuck in TIME_WAIT state from previous test runs.
Changes:
- socks5.py: Update conf.addr after bind() to reflect actual port
- p2p.py: Retrieve actual port after create_server() when port=0
- feature_proxy.py: Use port=0 for all SOCKS5 proxy servers
- feature_anchors.py: Use port=0 for onion proxy server
- No need to jump into the next subroutine when there is no conflict.
- This makes it clear why it is necessary to have two calls of
CheckMempoolPolicyLimts in both PackageMempoolChecks and after in
AcceptMultipleTransactionsInternal, there is a possibilty that we
we want to accept multiple transaction but they are not conflicting
with any in-mempool transaction, in that case also we want to check
that they do not bust the cluster limits.
2cade5d5d1 [miniminer] stop assuming ancestor fees >= self fees (glozow)
Pull request description:
These assertions exist to detect double-deducting values when we update descendants. However, negative fees are possible with `prioritisetransaction` so it doesn't make sense to check this.
Leave the check for sizes because those are never negative.
Fixes#34234
ACKs for top commit:
instagibbs:
ACK 2cade5d5d1
dergoegge:
utACK 2cade5d5d1
Tree-SHA512: 935bbc8bd9a0d508eea43bb49aa43c22735e3f2c1012598f6843e229c13b76e44f9fd3eb8b61c437fa0b32353b4e7b15afa3e31002bdfa382d3d711d16419fde
