faca7ac13215fd88b420feea8f06d7404f8fd067 ci: Bump fuzz task timeout (MarcoFalke)
Pull request description:
The fuzz task seems to be the most CPU intense task (going through GB of data through all fuzz inputs for all fuzz targets).
Normally, the task takes 44 minutes (example https://cirrus-ci.com/task/5077976091459584), but under higher load, it may take longer (https://cirrus-ci.com/task/5966231095738368).
I tried to move it to GHA to see how it compares, but it will be even slower there: https://github.com/maflcko/bitcoin-core-with-ci/actions/runs/13182526514/job/36796629409.
The CI machines were recently updated to increase the CI performance, so in theory they could be updated again, but this can take some time and seems like the wrong fix anyway, because it will just hide the problem:
Ideally fuzzing is fast and when evaluating a fuzz input takes more than 10 seconds, it feels more like a slow unit test loop. So ideally fuzz timeouts should be fixed (https://github.com/bitcoin/bitcoin/issues/31066, https://github.com/bitcoin/bitcoin/issues/30498, ...). However, this can also take time.
So temporarily bump the fuzz timeout for now.
ACKs for top commit:
dergoegge:
ACK faca7ac13215fd88b420feea8f06d7404f8fd067
Tree-SHA512: cfb06d14712d94be6b28a17eee821dcc762453e8efbd9376200f8a0e784a55c2140e45ac48bee9b71ef6e85ae7345155dddc1239cbf0cd4bc02583848fe46308
No change for now, moving from map of NodeId->workset to
NodeId->PeerOrphanInfo struct that holds the workset.
In future commits, we will start tracking more things per-peer in the
orphanage.
2f27c910869e301b7e7669e81a0878da64e49957 qt: Update the `src/qt/locale/bitcoin_en.xlf` translation source file (Hennadii Stepanov)
864386a7444fb5cf16613956ce8bf335f51b67d5 cmake: Ensure generated sources are up to date for `translate` target (Hennadii Stepanov)
2b51dd384b4a2655ee066e8bccd254270d0f5f6c Update Transifex slug for 29.x (Hennadii Stepanov)
Pull request description:
This PR follows our [Release Process](864386a744/doc/release-process.md).
It is required to open Transifex translations for v29.0, as scheduled in https://github.com/bitcoin/bitcoin/issues/31029.
The previous similar PR: https://github.com/bitcoin/bitcoin/pull/30548.
**Notes for reviewers:**
1. This is the first release process conducted after migrating the build system to CMake. This revealed a bug, which is fixed in the second commit
2. To reproduce the diff in the third commit, follow these steps:
```
gmake -C depends -j $(nproc) MULTIPROCESS=1
cmake --preset dev-mode --toolchain depends/$(./depends/config.guess)/toolchain.cmake
cmake --build build_dev_mode --target translate
```
ACKs for top commit:
stickies-v:
ACK 2f27c910869e301b7e7669e81a0878da64e49957
Tree-SHA512: 325ce2418f218b82cc3b0a6c727473963455680cdf6383a85768613ed9e485974b2e52bd5b2e7a7472ad8ebe40bccb2884764d7f9e83dc10a587cd7892e0028b
Some of the networking constants defined in p2p_tx_download.py are more generally
defined in p2p.py
Also, rename the remaining ones to match ones defined in txdownloadman
Some sources might be generated, and while they likely do not contain
any translatable strings, this change generalizes the approach to
include generated sources in the translation process as well.
1973a9e4f1dfba57051135d6e1bca80979de3879 test: fixes p2p_ibd_txrelay wait time (Sergi Delgado Segura)
Pull request description:
`p2p_ibd_txrelay` expects no GETDATA to have been received by a peer after announcing a transaction. The reason is that the node is doing IBD, so transaction requests are not replied to. However, the way this is checked is wrong, and the check will pass even if the node **was not** in IBD.
This is due to the mocktime not being properly initialized, so the check is always performed earlier than it should, making it impossible for the request to be there.
This can be checked by modifying the test so the peer **is not doing IBD**, and checking how the test succeeds on that assert (even though it fails later on, given the nature of the test):
```diff
index 882f5b5c13..3a69ae5860 100755
--- a/test/functional/p2p_ibd_txrelay.py
+++ b/test/functional/p2p_ibd_txrelay.py
@@ -34,7 +34,7 @@ NORMAL_FEE_FILTER = Decimal(100) / COIN
class P2PIBDTxRelayTest(BitcoinTestFramework):
def set_test_params(self):
- self.setup_clean_chain = True
+ # self.setup_clean_chain = True
self.num_nodes = 2
self.extra_args = [
["-minrelaytxfee={}".format(NORMAL_FEE_FILTER)],
@@ -43,9 +43,11 @@ class P2PIBDTxRelayTest(BitcoinTestFramework):
def run_test(self):
self.log.info("Check that nodes set minfilter to MAX_MONEY while still in IBD")
- for node in self.nodes:
- assert node.getblockchaininfo()['initialblockdownload']
- self.wait_until(lambda: all(peer['minfeefilter'] == MAX_FEE_FILTER for peer in node.getpeerinfo()))
+ # for node in self.nodes:
+ # assert node.getblockchaininfo()['initialblockdownload']
+ # self.wait_until(lambda: all(peer['minfeefilter'] == MAX_FEE_FILTER for peer in node.getpeerinfo()))
```
ACKs for top commit:
i-am-yuvi:
ACK 1973a9e4f1dfba57051135d6e1bca80979de3879
glozow:
ACK 1973a9e4f1dfba57051135d6e1bca80979de3879
Tree-SHA512: c4b3afe9927c5480671ebf5c1f6ee5fc7e3aeefeb13c210fa83587a6c126e1a8e40ad8e46587537d0f4bf06a36bbf2310ca065d685d4d9286e5a446b8d5b2235
3f4b104b1b7e1b87c0be8e395e02b6ae3c5d7b08 test: make sure we are on sync with a peer before checking if they have sent a message (Sergi Delgado Segura)
Pull request description:
p2p_orphan_handling checks whether a message has not been requested slightly too soon, making the check always succeed. This passes unnoticed since the expected result is for the message to not have been received, but it will make the test not catch a relevant change that should make it fail.
An easy way to check this is the case is to modify one of the test cases to force a request within the expected time, and check how the request is not seen. After the change, the test would crash as expected:
```diff
index 963d92485c..30ab5f2035 100755
--- a/test/functional/p2p_orphan_handling.py
+++ b/test/functional/p2p_orphan_handling.py
@@ -186,9 +185,12 @@ class OrphanHandlingTest(BitcoinTestFramework):
parent_inv = CInv(t=MSG_WTX, h=int(tx_parent_arrives["tx"].getwtxid(), 16))
assert_equal(len(peer_spy.get_invs()), 0)
peer_spy.assert_no_immediate_response(msg_getdata([parent_inv]))
+ txid = 0xdeadbeef
+ peer_spy.send_and_ping(msg_inv([CInv(t=MSG_WTX, h=txid)]))
# Request would be scheduled with this delay because it is not a preferred relay peer.
self.nodes[0].bumpmocktime(NONPREF_PEER_TX_DELAY)
+ peer_spy.assert_never_requested(int(txid))
peer_spy.assert_never_requested(int(tx_parent_arrives["txid"], 16))
peer_spy.assert_never_requested(int(tx_parent_doesnt_arrive["txid"], 16))
# Request would be scheduled with this delay because it is by txid.
```
It is worth noting that this is not seen in the cases where the message is expected to be received, because in such cases `assert_never_requested` is always after a `wait_....` method, which is already waiting for the node to sync on their end.
ACKs for top commit:
i-am-yuvi:
ACK 3f4b104b1b7e1b87c0be8e395e02b6ae3c5d7b08
instagibbs:
ACK 3f4b104b1b7e1b87c0be8e395e02b6ae3c5d7b08
glozow:
ACK 3f4b104b1b7e1b87c0be8e395e02b6ae3c5d7b08
Tree-SHA512: 321a6605d630bed2217b6374e999dbb84da14138263dd8adf65fe3a6cd7981a50c873beced9cf05cb6d747a912e91017c58e7d4323d25449c87d83095ff4cba9
e3622a969293feea75cfadc8f7c6083edcd6d8de tracing: document that peer addrs can be >68 chars (0xb10c)
b19b526758f055733e1c21809cf975169fdd39b0 tracing: log_p2p_connections.bt example (0xb10c)
caa5486574baf805b36c8abc873554aee4ba82b7 tracing: connection closed tracepoint (0xb10c)
b2ad6ede95ea66e8677b31d614e183953966db54 tracing: add misbehaving conn tracepoint (0xb10c)
68c1ef4f19bc4ebe0ca1af95ac378732c4fe6d22 tracing: add inbound connection eviction tracepoint (0xb10c)
4d61d52f4387622701cdad4bb0fb115127021106 tracing: add outbound connection tracepoint (0xb10c)
85b2603eec634257cd3b398900dbb92251db71e6 tracing: add inbound connection tracepoint (0xb10c)
Pull request description:
This adds five new tracepoints with documentation and tests for network connections:
- established connections with `net:inbound_connection` and `net:outbound_connection`
- closed connections (both closed by us or by the peer) with `net:closed_connnection`
- inbound connections that we choose to evict with `net:evicted_inbound_connection`
- connections that are misbehaving and punished with `net:misbehaving_connection`
I've been using these tracepoints for a few months now to monitor connection lifetimes, re-connection frequency by IP and netgroup, misbehavior, peer discouragement, and eviction and more. Together with the two existing P2P message tracepoints they allow for a good overview of local P2P network activity. Also sort-of addresses https://github.com/bitcoin/bitcoin/pull/22006#discussion_r636775863.
I've been back and forth on which arguments to include. For example, `net:evicted_connection` could also include some of the eviction metrics like e.g. `last_block_time`, `min_ping_time`, ... but I've left them out for now. If wanted, this can be added here or in a follow-up. I've tried to minimize a potential performance impact by measuring executed instructions with `gdb` where possible (method described [here](https://github.com/bitcoin/bitcoin/pull/23724#issuecomment-996919963)). I don't think a few hundred extra instructions are too crucial, as connection opens/closes aren't too frequent (compared to e.g. P2P messages). Note: e.g. `CreateNodeFromAcceptedSocket()` usually executes between 80k and 90k instructions for each new inbound connection.
| tracepoint | instructions |
|----------------------------|--------------------------------------------------------|
| net:inbound_connection | 390 ins |
| net:outbound_connection | between 700 and 1000 ins |
| net:closed_connnection | 473 ins |
| net:evicted_inbound_connection | not measured; likely similar to net:closed_connnection |
| net:misbehaving_connection | not measured |
Also added a bpftrace (tested with v0.14.1) `log_p2p_connections.bt` example script that produces output similar to:
```
Attaching 6 probes...
Logging opened, closed, misbehaving, and evicted P2P connections
OUTBOUND conn to 127.0.0.1:15287: id=0, type=block-relay-only, network=0, total_out=1
INBOUND conn from 127.0.0.1:45324: id=1, type=inbound, network=0, total_in=1
MISBEHAVING conn id=1, message='getdata message size = 50001'
CLOSED conn to 127.0.0.1:15287: id=0, type=block-relay-only, network=0, established=1231006505
EVICTED conn to 127.0.0.1:45324: id=1, type=inbound, network=0, established=1612312312
```
ACKs for top commit:
laanwj:
re-ACK e3622a969293feea75cfadc8f7c6083edcd6d8de
vasild:
ACK e3622a969293feea75cfadc8f7c6083edcd6d8de
sipa:
utACK e3622a969293feea75cfadc8f7c6083edcd6d8de
Tree-SHA512: 1032dcac6fe0ced981715606f82c2db47016407d3accb8f216c978f010da9bc20453e24a167dcc95287f4783b48562ffb90f645bf230990e3df1b9b9a6d4e5d0
The `-ffile-prefix-map` compiler option implicitly enables
`-fprofile-prefix-map` in GCC or `-fcoverage-prefix-map` in Clang, which
can cause issues with coverage builds.
This change ensures that only the options necessary for build
reproducibility are applied.
723440c5b8eb3a815c80bfb37ad195b5448b25ed test framework, wallet: rename get_scriptPubKey method to get_output_script (Alfonso Roman Zubeldia)
fa0232a3e07ad6d11b4d4aaec93e9531ac3274f3 test: add validation for gettxout RPC response (Alfonso Roman Zubeldia)
Pull request description:
Added a new test in `test/functional/rpc_blockchain.py` to validate the gettxout RPC response. This new test ensures all response elements are verified, including `bestblock`, `confirmations`, `value`, `coinbase`, and `scriptPubKey` details.
Also renamed the method `get_scriptPubKey` from `test/functional/test_framework/wallet.py` to the modern name `get_output_script` as suggested by maflcko (https://github.com/bitcoin/bitcoin/pull/30226#discussion_r1925491846)
ACKs for top commit:
fjahr:
reACK 723440c5b8eb3a815c80bfb37ad195b5448b25ed
maflcko:
lgtm ACK 723440c5b8eb3a815c80bfb37ad195b5448b25ed
brunoerg:
code review ACK 723440c5b8eb3a815c80bfb37ad195b5448b25ed
Tree-SHA512: 3384578909d2e7548cef302c5b8a9fed5b82dfc942892503ad4a05e73f5cceafad1eab3af9a27e54aef3db7631f8935298d6b882c70d2f02a9a75b8e3c209b6c
846a1387280fa584f70ccb1f5d0198339b065528 func test: Expand tx download preference tests (Greg Sanders)
Pull request description:
1. Check that outbound nodes are treated the same as whitelisted connections for
the purposes of `getdata` delays
2. Add test case that demonstrates download retries are preferentially
given to outbound (preferred) connections
even when multiple announcements are
considered ready.
`NUM_INBOUND` is a magic number large enough that it should fail over 90% of the time
if the underlying outbound->preferred->PriorityComputer logic was broken. Bumping this
to 100 peers cost another 14 seconds locally for the sub-test, so I made it pretty small.
ACKs for top commit:
i-am-yuvi:
tACK 846a1387280fa584f70ccb1f5d0198339b065528 good catch
maflcko:
ACK 846a1387280fa584f70ccb1f5d0198339b065528 🍕
marcofleon:
lgtm ACK 846a1387280fa584f70ccb1f5d0198339b065528
Tree-SHA512: 337aa4dc33b5c0abeb4fe7e4cd5e389f7f53ae25dd991ba26615c16999872542391993020122fd255af4c7163f76c1d1feb2f2d6114f12a364c0360d4d52b8c3
7e0db87d4fff996c086f6e86b62338c98ef30c55 test: added additional coverage to waitforblock and waitforblockheight rpc's (kevkevinpal)
Pull request description:
Similar to https://github.com/bitcoin/bitcoin/pull/31746
This adds test coverage to the `waitforblock` and `waitforblockheight` rpc's by adding a test to assert we get an rpc error if we include a negative timeout
ACKs for top commit:
Sjors:
utACK 7e0db87d4fff996c086f6e86b62338c98ef30c55
Prabhat1308:
ACK [7e0db87](7e0db87d4f)
brunoerg:
code review ACK 7e0db87d4fff996c086f6e86b62338c98ef30c55
BrandonOdiwuor:
Code Review ACK 7e0db87d4fff996c086f6e86b62338c98ef30c55
Tree-SHA512: c3b1b3a525e91e0092b783d73d2401126e3b8792a394be00374258f20cf3d619498e6625d3aad5e5ced29509c5eac828ee03c4ee66ba405b91e89be7e8b07311
d9c8aacce38ab593ea9277976eb64ccadd7d062f depends, refactor: Avoid hardcoding `host_prefix` in toolchain file (Hennadii Stepanov)
Pull request description:
This PR allows the entire `depends/<host_prefix>` directory to be relocatable.
Only `libevent` package configuration files are non-relocatable for the version `2.1.12-stable` we use now. However, this issue has been fixed upstream in 1f1593ff27 and friends.
Fixes https://github.com/bitcoin/bitcoin/issues/31050.
ACKs for top commit:
theuni:
Neat. utACK d9c8aacce38ab593ea9277976eb64ccadd7d062f
ryanofsky:
Code review ACK d9c8aacce38ab593ea9277976eb64ccadd7d062f
Tree-SHA512: c4c340722e63fc1da36fba2b15f025a6e1d477da1991194d678f546f2f3ea9454e2f0ff054aae6ae6c332a0781a597c3ce63b9018b46b8c258033f0d19efbef3
- The reserved weight of the coinbase transaction is an estimate and
may not reflect the exact value; it can be lower.
- It should be clear that `currentblockweight` includes the reserved coinbase transaction weight.
whereas `currentblocktx` does not account for the coinbase transaction count.
- Also clarify `m_last_block_num_txs` and `m_last_block_weight`
- This commit renamed coinbase_max_additional_weight to block_reserved_weight.
- Also clarify that the reservation is for block header, transaction count
and coinbase transaction.
faf8fc5487d409eeff7b7b260eabb6929a7b7a5f lint: Call lint_commit_msg from test_runner (MarcoFalke)
fa99728b0c8b3cac7056fa554fab7a8a4624a2de lint: Move commit range printing to test_runner (MarcoFalke)
fa673cf3449f4e71501814bf99c2e2bbb49b8fcb lint: Call lint_scripted_diff from test_runner (MarcoFalke)
Pull request description:
The lint `commit-script-check.sh` can not be called from the test_runner at all and must be called manually. Also, some checks require `COMMIT_RANGE` to be set.
Fix all issues by moving two lint checks into the test_runner. Also, the proper commit range is passed to the checks by the test_runner, so that the user no longer has to do it.
ACKs for top commit:
kevkevinpal:
reACK [faf8fc5](faf8fc5487)
willcl-ark:
tACK faf8fc5487d409eeff7b7b260eabb6929a7b7a5f
Tree-SHA512: 78018adc618d997508c226c9eee0a4fada3899cdfd91587132ab1c0389aea69127bafc3a900e90e30aca2c6bae9dcd6e6188ef287e91413bc63ee66fb078b1af
3e97ff9c5eaa3160426ba112930b047404c54c9e gui, psbt: Use SIGHASH_DEFAULT when signing PSBTs (Ava Chow)
Pull request description:
SIGHASH_DEFAULT should be used to indicate SIGHASH_DEFAULT for taproot inputs, and SIGHASH_ALL for all other input types. This avoids adding an unnecessary byte to the end of all Taproot signatures added to PSBTs signed in the GUI.
See also bitcoin/bitcoin#22514
ACKs for top commit:
Sjors:
utACK 3e97ff9c5eaa3160426ba112930b047404c54c9e
pablomartin4btc:
utACK 3e97ff9c5eaa3160426ba112930b047404c54c9e
hebasto:
ACK 3e97ff9c5eaa3160426ba112930b047404c54c9e, I have reviewed the code and it looks OK.
Tree-SHA512: f96f26b3a6959865cf23039afb5ffb7e454fb52ee39c510583851caf00a8a383cde69bc7e90db536addbdd498a02f4b001cbaf509d6d53c5f8601b3933786f6c
9d2d9f7ce29636f08322df70cf6abec8e0ca3727 rpc: Include assumeutxo as a failure reason of rescanblockchain (Fabian Jahr)
595edee169045b6735b76ff9721677f0e43f13e5 test, assumeutxo: import descriptors during background sync (Alfonso Roman Zubeldia)
d73ae603d44f93e4d6c5116f235dd11a0bdbf89c rpc: Improve importdescriptor RPC error messages (Fabian Jahr)
27f99b6d63b7ca2d4fcb9db3e88ed66c024c59d5 validation: Don't assume m_chain_tx_count in GuessVerificationProgress (Fabian Jahr)
42d5d5336319aaf0f07345037db78239d9e012fc interfaces: Add helper function for wallet on pruning (Fabian Jahr)
Pull request description:
A test that is added as part of #30455 uncovered this issue: The `GuessVerificationProgress` function is used during during descriptor import and relies on `m_chain_tx_count`. In #29370 an [`Assume` was added](0fd915ee6b) expecting the `m_chaint_tx_count` to be set. However, as the test uncovered, `GuessVerificationProgress` is called with background sync blocks that have `m_chaint_tx_count = 0` when they have not been downloaded and processed yet.
The simple fix is to remove the `Assume`. Users should not be thrown off by the `Internal bug detected` error. The behavior of `importdescriptor` is kept consistent with the behavior for blocks missing due to pruning.
The test by alfonsoromanz is cherry-picked here to show that the [CI errors](https://cirrus-ci.com/task/5110045812195328?logs=ci#L2535) should be fixed by this change.
This PR also improves error messages returned by the `importdescriptors` and `rescanblockchain` RPCs. The error message now changes depending on the situation of the node, i.e. if pruning is happening or an assumutxo backgroundsync is active.
ACKs for top commit:
achow101:
ACK 9d2d9f7ce29636f08322df70cf6abec8e0ca3727
mzumsande:
Code Review ACK 9d2d9f7ce29636f08322df70cf6abec8e0ca3727
furszy:
Code review ACK 9d2d9f7ce29636f08322df70cf6abec8e0ca3727
Tree-SHA512: b841a9b371e5eb8eb3bfebca35645ff2fdded7a3e5e06308d46a33a51ca42cc4c258028c9958fbbb6cda9bb990e07ab8d8504dd9ec6705ef78afe0435912b365
0cdddeb2240d1f33c8b2dd28bb0c9d84d9420e3d kernel: Move block tree db open to BlockManager constructor (TheCharlatan)
7fbb1bc44b1461f008284533f1667677e729f0c0 kernel: Move block tree db open to block manager (TheCharlatan)
57ba59c0cdf20de322afabe4a132ad17e483ce77 refactor: Remove redundant reindex check (TheCharlatan)
Pull request description:
Before this change the block tree db was needlessly re-opened during startup when loading a completed snapshot. Improve this by letting the block manager open it on construction. This also simplifies the test code a bit.
The change was initially motivated to make it easier for users of the kernel library to instantiate a BlockManager that may be used to read data from disk without loading the block index into a cache.
ACKs for top commit:
maflcko:
re-ACK 0cdddeb2240d1f33c8b2dd28bb0c9d84d9420e3d 🏪
achow101:
ACK 0cdddeb2240d1f33c8b2dd28bb0c9d84d9420e3d
mzumsande:
re-ACK 0cdddeb2240d1f33c8b2dd28bb0c9d84d9420e3d
Tree-SHA512: fe3d557a725367e549e6a0659f64259cfef6aaa565ec867d9a177be0143ff18a2c4a20dd57e35e15f97cf870df476d88c05b03b6a7d9e8d51c568d9eda8947ef
