5c3e4d8b293fab06d2311a863c675a392f24e383 doc: add a section about using MSan (Antoine Poinsot)
Pull request description:
Just a couple lines in a subsection of the sanitizers section mentioning that using the memory sanitizer is a bit more involve than other sanitizers, describing the steps and pointing to an example.
ACKs for top commit:
fanquake:
ACK 5c3e4d8b293fab06d2311a863c675a392f24e383
dergoegge:
ACK 5c3e4d8b293fab06d2311a863c675a392f24e383
Tree-SHA512: 4ff73c2dd0f25cb96148e54bd867b8d340bd0fbc9b9a736a705125039352eb1d40bd724f9f262a44d3dbd1bea8f03166cf30e571d882fec02ceb1dd399ef7422
a4df12323c4e9230bca58562ba17ecee4233f8fe doc: add release notes (Sjors Provoost)
c75872ffdd98ce9f04fb8489515d1b63853f03b4 test: use DIFF_1_N_BITS in tool_signet_miner (tdb3)
4131f322ac0abe43639065362cd3c4ea36d2c5c3 test: check difficulty adjustment using alternate mainnet (Sjors Provoost)
c4f68c12e228818f655352d17d038dcc7ba1db3a Use OP_0 for BIP34 padding in signet and tests (Sjors Provoost)
cf0a62878be214cd4ec779aab214221b27b769b6 rpc: add next to getmininginfo (Sjors Provoost)
2d18a078a2d9eaa53b8b7acc7600141c69f0d742 rpc: add target and bits to getchainstates (Sjors Provoost)
f153f57acc9f9a6f84af161d5bed9aa9965abaa3 rpc: add target and bits to getblockchaininfo (Sjors Provoost)
baa504fdfaff4a9f61bc939035df5d5f2978cfd7 rpc: add target to getmininginfo result (Sjors Provoost)
2a7bfebd5e788e1d9e7e07a9f1b8e3625a0301cd Add target to getblock(header) in RPC and REST (Sjors Provoost)
341f93251677fee66c822f414b75499e8b3b31f6 rpc: add GetTarget helper (Sjors Provoost)
d20d96fa41ce706ccc480b4f3143438ce0720348 test: use REGTEST_N_BITS in feature_block (tdb3)
7ddbed4f9fc0c90bfed244a71194740a4a1fa1be rpc: add nBits to getmininginfo (Sjors Provoost)
ba7b9f3d7bf5a1ad395262b080e832f5c9958e4d build: move pow and chain to bitcoin_common (Sjors Provoost)
c4cc9e3e9df2733260942e0513dd8478d2a104da consensus: add DeriveTarget() to pow.h (Sjors Provoost)
Pull request description:
**tl&dr for consensus-code only reviewers**: the first commit splits `CheckProofOfWorkImpl()` in order to create a `DeriveTarget()` helper. The rest of this PR does not touch consensus code.
There are three ways to represent the proof-of-work in a block:
1. nBits
2. Difficulty
3. Target
The latter notation is useful when you want to compare share work against either the pool target (to get paid) or network difficulty (found an actual block). E.g. for difficulty 1 which corresponds to an nBits value of `0x00ffff`:
```
share hash: f6b973257df982284715b0c7a20640dad709d22b0b1a58f2f88d35886ea5ac45
target: 7fffff0000000000000000000000000000000000000000000000000000000000
```
It's immediately clear that the share is invalid because the hash is above the target.
This type of logging is mostly done by the pool software. It's a nice extra convenience, but not very important. It impacts the following RPC calls:
1. `getmininginfo` displays the `target` for the tip block
2. `getblock` and `getblockheader` display the `target` for a specific block (ditto for their REST equivalents)
The `getdifficulty` method is a bit useless in its current state, because what miners really want to know if the difficulty for the _next_ block. So I added a boolean argument `next` to `getdifficulty`. (These values are typically the same, except for the first block in a retarget period. On testnet3 / testnet4 they change when no block is found after 20 minutes).
Similarly I added a `next` object to `getmininginfo` which shows `bit`, `difficulty` and `target` for the next block.
In order to test the difficulty transition, an alternate mainnet chain with 2016 blocks was generated and used in `mining_mainnet.py`. The chain is deterministic except for its timestamp and nonce values, which are stored in `mainnet_alt.json`.
As described at the top, this PR introduces a helper method `DeriveTarget()` which is split out from `CheckProofOfWorkImpl`. The proposed `checkblock` RPC in #31564 needs this helper method internally to figure out the consensus target.
Finally, this PR moves `pow.cpp` and `chain.cpp` from `bitcoin_node` to `bitcoin_common`, in order to give `rpc/util.cpp` (which lives in `bitcoin_common`) access to `pow.h`.
ACKs for top commit:
ismaelsadeeq:
re-ACK a4df12323c4e9230bca58562ba17ecee4233f8fe
tdb3:
code review re ACK a4df12323c4e9230bca58562ba17ecee4233f8fe
ryanofsky:
Code review ACK a4df12323c4e9230bca58562ba17ecee4233f8fe. Only overall changes since last review were dropping new `gettarget` method and dropping changes to `getdifficulty`, but there were also various internal changes splitting and rearranging commits.
Tree-SHA512: edef5633590379c4be007ac96fd1deda8a5b9562ca6ff19fe377cb552b5166f3890d158554c249ab8345977a06da5df07866c9f42ac43ee83dfe3830c61cd169
e94c9d171239c1fc44fa9c77a4595ecd626b767f [doc] Amend notes on benchmarking (dergoegge)
Pull request description:
This gives some more context on the motivation and larger picture of benchmarks.
ACKs for top commit:
l0rinc:
ACK e94c9d171239c1fc44fa9c77a4595ecd626b767f
instagibbs:
reACK e94c9d171239c1fc44fa9c77a4595ecd626b767f
darosior:
reACK e94c9d171239c1fc44fa9c77a4595ecd626b767f
brunoerg:
reACK e94c9d171239c1fc44fa9c77a4595ecd626b767f
Tree-SHA512: 2cbf51f283f2efc0938e7021ae48db51fe89caf9ef9780821e99fa745dff839e2d202ca956ce6cc48b8319db304069728e77883feefe486264eb1783a0610c93
This is similar in structure to test/functional/wallet_multisig_descriptor_psbt.py
both in code and concept. It should serve as some integration testing for
Miniscript descriptors, and also documents a simple multisig that starts as 4-of-4
and decays to 3-of-4, 2-of-4, and finally 1-of-4 at block heights (I think in the
real world aligning this to halvenings would be nice).
- Move "Clang" and "GCC" from the table to a new "Compiler" heading,
indicating either is required.
- Move "CMake" into the required table.
- Move "Python" into the optional table.
- Merge the optional dependencies into one table. Removed sub-headers
are put into parentheses behind the dependency name in the first
column.
- Replace the whitespace in the "Minimum required" column of "qrencode"
with "N/A" for consistency.
- Add missing info for the "systemtap" dependency.
- Add context for "Linux Kernel" dependency in parentheses behind the
dependency name.
31a0e5f0905bfc6b22ceaaeca53466dfd74967ab depends: Qt 5.15.16 (fanquake)
Pull request description:
Contains a handful of miscellaneous bug fixes.
We can drop a few of our patches.
See https://github.com/qt/qtbase/compare/v5.15.14-lts-lgpl...v5.15.16-lts-lgpl.
ACKs for top commit:
hebasto:
ACK 31a0e5f0905bfc6b22ceaaeca53466dfd74967ab.
TheCharlatan:
ACK 31a0e5f0905bfc6b22ceaaeca53466dfd74967ab
Tree-SHA512: dd7b3332dd6ecb95189bc72364883425fb8869e03850791d2ee92555a37046c7abaaee16575a0396f1ce9674856b894563dbd36868c2cf46f9fee48028fd967b
According to the description for pkg-config, "pkgconf is a
replacement for pkg-config, providing additional functionality
while also maintaining compatibility. This package only provides
a dependency link to the pkgconf package to help with package
upgrades. It can be safely removed."
Thus several scripts and markdown files are updated.
fa029a78780fd00e1d3ce1bebb81a95857bfbb94 doc: Clarify min macOS and Xcode version (MarcoFalke)
Pull request description:
Two minor doc fixups:
* Clarify that `macOS 13.0+` means `macOS 13+`, indicating that on any major version, only the latest security release is supported.
* Clarify that the Xcode version was selected based on the minimum required macOS version and the minimum required clang version.
ACKs for top commit:
jarolrod:
ACK fa029a78780fd00e1d3ce1bebb81a95857bfbb94
hebasto:
re-ACK fa029a78780fd00e1d3ce1bebb81a95857bfbb94.
theuni:
ACK fa029a78780fd00e1d3ce1bebb81a95857bfbb94
Tree-SHA512: d34910fcc22e57021d7642938e5886419d2b711e1062cbc4fc3da48baf07377231f9d7b394e22ccb17e830d058c8c797dbd1bbffcc7c8828601bb500e1154a9e
0a76c292ac8fa29166a7ec6efda1fafce86af0d3 doc: Install `net/py-pyzmq` port on FreeBSD for `interface_zmq.py` (Hennadii Stepanov)
Pull request description:
On FreeBSD, Python's `zmq` module is provided as a separate port.
This PR updates the FreeBSD Build Guide to include this port, enabling the `interface_zmq.py` functional test.
ACKs for top commit:
maflcko:
lgtm ACK 0a76c292ac8fa29166a7ec6efda1fafce86af0d3
vasild:
ACK 0a76c292ac8fa29166a7ec6efda1fafce86af0d3
Tree-SHA512: c13eada3e870149f47348145d6a29f41125ac75efd88eabe6dd2d0429e0377ed280e76a764cfaf627498c1d07b9135a995cc644146fa666bc3bfa0eb2c86e88b
2bdaf52ed1259fd3bec22b680e12563fcee0a8b3 doc: Update NetBSD Build Guide (Hennadii Stepanov)
Pull request description:
This PR:
1. Updates the documented NetBSD version.
2. Adds the optional ZeroMQ package to align the guide with other *BSD systems.
3. Updates the Python version to meet the minimum requirement specified in https://github.com/bitcoin/bitcoin/pull/30527.
4. Suggests to Install [`net/py-zmq`](https://ftp.netbsd.org/pub/pkgsrc/current/pkgsrc/net/py-zmq/index.html) package to enable the `interface_zmq.py` functional test.
5. Fix a formatting issue.
See also the recent NetBSD nightly build at https://github.com/hebasto/bitcoin-core-nightly/actions/runs/12554769828/job/35003929261.
ACKs for top commit:
tdb3:
ACK 2bdaf52ed1259fd3bec22b680e12563fcee0a8b3
Tree-SHA512: 16562e7325dd92e44fa641c8d4e64df69ee76175fc8eba61da06775d4a751307825a6ffe1743fb614a591ba1d33d197ea6b7f9111f5a5b335f6b257bb4868bf6
b6f0593f43304f4ff31e8b68558ceeb1b588403c doc: add release note about testmempoolaccept debug-message (Matthew Zipkin)
f9cac635237142090271022164fa5d58e014493d test: cover testmempoolaccept debug-message in RBF test (Matthew Zipkin)
f9650e18ea6edb41c0136cc2ec3c7e0aba1bf83a rbf: remove unecessary newline at end of error string (Matthew Zipkin)
221c789e91696569fa34dbd162d26e98cf9cab64 rpc: include verbose reject-details field in testmempoolaccept response (Matthew Zipkin)
Pull request description:
Adds a new field `reject-details` in `testmempoolaccept` responses to include `m_debug_message` from `ValidationState`. This string is the complete error message thrown by the mempool in response to `sendrawtransaction`.
The extra verbosity is helpful to consumers of `testmempoolaccept`, which is sort of a debug tool anyway.
example:
>
> {
> "txid": "07d7a59a7bdad4c3a5070659ea04147c9b755ad9e173c52b6a38e017abf0f5b8",
> "wtxid": "5dc243b1b92ee2f5a43134eb3e23449be03d1abb3d7f3c03c836ed0f13c50185",
> "allowed": false,
> "reject-reason": "insufficient fee",
> "reject-details": "insufficient fee, rejecting replacement 07d7a59a7bdad4c3a5070659ea04147c9b755ad9e173c52b6a38e017abf0f5b8; new feerate 0.00300000 BTC/kvB <= old feerate 0.00300000 BTC/kvB"
> }
ACKs for top commit:
rkrux:
re-ACK b6f0593f43304f4ff31e8b68558ceeb1b588403c
glozow:
ACK b6f0593f43304f4ff31e8b68558ceeb1b588403c
Tree-SHA512: 340b8023d59cefa84598879c4efdb7c399a3f62da126e87c595523f302e53d33098fc69da9c5f8c92b7580dc75466c66cea372051f935b197265648fe15c43a3
1. Update the documented NetBSD version.
2. Add the optional ZeroMQ package to align the guide with other *BSD
systems.
3. Update the Python version to meet the minimum requirement specified
in https://github.com/bitcoin/bitcoin/pull/30527.
4. Install `net/py-zmq` package to enable the `interface_zmq.py`
functional test.
5. Fix a formatting issue.
1dd3af8fbc350c6f1efa8ae6449e67e1b42ccff4 Add release note for #31223 (Martin Zumsande)
997757dd2b4d7b20b17299fbd21970b2efb8bbc8 test: add functional test for -port behavior (Martin Zumsande)
0e2b12b92a28a2949e75bf50f31563f52e647d6e net, init: derive default onion port if a user specified a -port (Martin Zumsande)
Pull request description:
This resolves#31133 (setups with multiple local nodes each using a different `-port` no longer working with v28.0, see the issue description for more details) by deriving the default onion listening port to be the value specified by `-port` incremented by 1 (idea by vasild / laanwj).
Note that with this fix, the chosen `-port` values of two local nodes cannot be adjacent, otherwise there will be port collisions again.
From the discussion in the linked issue, this was the most popular option, followed by doing nothing and telling affected users to change their setups to use `-bind` instead of `-port`. But more opinions are certainly welcome!
I think that if we decide to do something about the problem described in the issue, we should do so soon (in 28.1.), so I opened this PR.
Fixes#31133
ACKs for top commit:
achow101:
ACK 1dd3af8fbc350c6f1efa8ae6449e67e1b42ccff4
laanwj:
Tested ACK 1dd3af8fbc350c6f1efa8ae6449e67e1b42ccff4
tdb3:
Code review ACK 1dd3af8fbc350c6f1efa8ae6449e67e1b42ccff4
Tree-SHA512: 37fda2b23bbedcab5df3a401cf5afce66ae5318fb78f9660f83e3fd075b528e8156d7a0903f9a12ffe97ab5d83860587116b74af28670a1f4c2f0d1be4999f40
32fc59796f74a2941772b5ec2755b1319132cd9c rpc: Allow single transaction through submitpackage (glozow)
Pull request description:
There's no particular reason to restrict single transaction submissions with submitpackage. This change relaxes the RPC checks as enables the `AcceptPackage` flow to accept packages of a single transaction.
Resolves#31085
ACKs for top commit:
naumenkogs:
ACK 32fc59796f
achow101:
ACK 32fc59796f74a2941772b5ec2755b1319132cd9c
glozow:
ACK 32fc59796f74a2941772b5ec2755b1319132cd9c
Tree-SHA512: ffed353bfdca610ffcfd53b40b76da05ffc26df6bac4b0421492e067bede930380e03399d2e2d1d17f0e88fb91cd8eb376e3aabebbabcc724590bf068d09807c
73db95c65c1d372822166045ca8b9f173d5fd883 kernel: Make bitcoin-chainstate's block validation mirror submitblock's (TheCharlatan)
bb53ce9bdae2f02d7bd95cf5d8ca4ccf5136466a tests: Add functional test for submitting a previously pruned block (Greg Sanders)
1f7fc738255205a64374686aca9a4c53089360f1 rpc: Remove submitblock duplicate pre-check (TheCharlatan)
e62a8abd7df21795dcd173773f689b6d4c8feab6 rpc: Remove submitblock invalid-duplicate precheck (TheCharlatan)
36dbebafb9b54764005e6fffa7ad28d4cadfe5e4 rpc: Remove submitblock coinbase pre-check (TheCharlatan)
Pull request description:
With the introduction of a mining ipc interface and the potential future introduction of a kernel library API it becomes increasingly important to offer common behaviour between them. An example of this is ProcessNewBlock, which is used by ipc, rpc, net_processing and (potentially) the kernel library. Having divergent behaviour on suggested pre-checks and checks for these functions is confusing to both developers and users and is a maintenance burden.
The rpc interface for ProcessNewBlock (submitblock) currently pre-checks if the block has a coinbase transaction and whether it has been processed before. While the current example binary for how to use the kernel library, bitcoin-chainstate, imitates these checks, the other interfaces do not.
The coinbase check is repeated again early during ProcessNewBlock. Pre-checking it may also shadow more fundamental problems with a block. In most cases the block header is checked first, before validating the transactions. Checking the coinbase first therefore masks potential issues with the header. Fix this by removing the pre-check.
Similary the duplicate checks are repeated early in the contextual checks of ProcessNewBlock. If duplicate blocks are detected much of their validation is skipped. Depending on the constitution of the block, validating the merkle root of the block is part of the more intensive workload when validating a block. This could be an argument for moving the pre-checks into block processing. In net_processing this would have a smaller effect however, since the block mutation check, which also validates the merkle root, is done before.
Testing spamming a node with valid, but duplicate unrequested blocks seems to exhaust a CPU thread, but does not seem to significantly impact keeping up with the tip. The benefits of adding these checks to net_processing are questionable, especially since there are other ways to trigger the more CPU-intensive checks without submitting a duplicate block. Since these DOS concerns apply even less to the RPC interface, which does not have banning mechanics built in, remove them too.
Finally, also remove the pre-checks from `bitcoin-chainstate.cpp`.
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).
ACKs for top commit:
Sjors:
re-utACK 73db95c65c1d372822166045ca8b9f173d5fd883
achow101:
ACK 73db95c65c1d372822166045ca8b9f173d5fd883
instagibbs:
ACK 73db95c65c1d372822166045ca8b9f173d5fd883
mzumsande:
ACK 73db95c65c1d372822166045ca8b9f173d5fd883
Tree-SHA512: 2d02e851cf402ecf6a1968c058df3576aac407e200cbf922a1a6391b7f97b4f42c6d9f6b0a78b9d1af0a6d40bdd529a7b11a1e6d88885bd7b8b090f6d1411861
8bf1b3039cb5b396e7e6d3ac075656952edd56d5 doc: Use more precise anchor links to Xcode SDK extraction (Jeremy Rand)
Pull request description:
The "SDK Extraction" section is what users presumably are looking for when they follow these links.
ACKs for top commit:
fanquake:
ACK 8bf1b3039cb5b396e7e6d3ac075656952edd56d5
Tree-SHA512: 38669a6b171aa102bb80f5b3a343bd6a067c6921c454f6d18087c5add8016eea2ba8196036f9968f0a9b7df1f642c96ff6c657338c32e775beb04038497cde1f
37a5c5d83664c31d83fc649d3c8c858bd5f10f21 doc: update descriptors.md for getdescriptoractivity (James O'Beirne)
ee3ce6a4f4d35afe7fcab16eff419a6788b02170 test: rpc: add no address case for getdescriptoractivity (James O'Beirne)
811f76f3a511d20750046319b390e225a1151caa rpc: add getdescriptoractivity (James O'Beirne)
25fe087de59e967ce968d35ed77138325eb9a9fa rpc: move-only: move ScriptPubKeyDoc to utils (James O'Beirne)
Pull request description:
The RPC command `scanblocks` provides a useful way to get a set of blockhashes that have activity relevant to a set of descriptors (`relevant_blocks`). However actually extracting the activity from those blocks is left as an exercise to the end user.
This process involves not only generating the (potentially ranged) set of scripts for the descriptor set on the client side (maybe via `deriveaddresses`), but then the user must retrieve each block's contents one-by-one using `getblock <hash>`, which is transmitted over a network link. And that's all before they perform the actual search over block content. There's even more work required to incorporate unconfirmed transactions.
This PR introduces an RPC `getdescriptoractivity` that [dovetails](https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2024-08-16#1046393;) with `scanblocks` output, handling the process described above. Users specify the blockhashes (perhaps from `relevant_blocks`) and a set of descriptors; they are then given all spend/receive activity in that set of blocks.
This is a very useful tool when implementing lightweight wallets that want neither to require a third-party indexer like electrs, nor the overhead of creating and managing watch-only wallets in Core. This allows Core to be more easily used in a "stateless" manner by wallets, with potentially many nodes interchangeably acting as backends.
### Example usage
```
% ./src/bitcoin-cli scanblocks start \
'["addr(bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t)"]' \
857263
{
"from_height": 857263,
"to_height": 858263,
"relevant_blocks": [
"00000000000000000002bc5cc78f5b0913a5230a8f4b0d5060bc9a60900a5a88",
"00000000000000000001c5291ed6a40c06d3db5c8fb738567654b24a14b24ecb"
],
"completed": true
}
% ./src/bitcoin-cli getdescriptoractivity \
'["00000000000000000002bc5cc78f5b0913a5230a8f4b0d5060bc9a60900a5a88", "00000000000000000001c5291ed6a40c06d3db5c8fb738567654b24a14b24ecb"]' \
'["addr(bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t)"]'
{
"activity": [
{
"type": "receive",
"amount": 0.00002900,
"blockhash": "00000000000000000002bc5cc78f5b0913a5230a8f4b0d5060bc9a60900a5a88",
"height": 857907,
"txid": "c9d34f202c1f66d80cae76f305350f5fdde910b97cf6ae6bf79f5bcf2a337d06",
"vout": 254,
"output_spk": {
"asm": "1 7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"desc": "rawtr(7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b)#yewcd80j",
"hex": "51207e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"address": "bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t",
"type": "witness_v1_taproot"
}
},
{
"type": "spend",
"amount": 0.00002900,
"blockhash": "00000000000000000001c5291ed6a40c06d3db5c8fb738567654b24a14b24ecb",
"height": 858260,
"spend_txid": "7f61d1b248d4ee46376f9c6df272f63fbb0c17039381fb23ca5d90473b823c36",
"spend_vin": 0,
"prevout_txid": "c9d34f202c1f66d80cae76f305350f5fdde910b97cf6ae6bf79f5bcf2a337d06",
"prevout_vout": 254,
"prevout_spk": {
"asm": "1 7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"desc": "rawtr(7e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b)#yewcd80j",
"hex": "51207e02f613a8d427f5f55ff62bddc47ccfb394953e57fdcb9a8add58af3124698b",
"address": "bc1p0cp0vyag6snlta2l7c4am3rue7eef9f72l7uhx52m4v27vfydx9s8tfs7t",
"type": "witness_v1_taproot"
}
}
]
}
```
ACKs for top commit:
instagibbs:
reACK 37a5c5d83664c31d83fc649d3c8c858bd5f10f21
achow101:
ACK 37a5c5d83664c31d83fc649d3c8c858bd5f10f21
tdb3:
Code review and light retest ACK 37a5c5d83664c31d83fc649d3c8c858bd5f10f21
rkrux:
re-ACK 37a5c5d83664c31d83fc649d3c8c858bd5f10f21
Tree-SHA512: 04aa51e329c6c2ed72464b9886281d5ebd7511a8a8e184ea81249033a4dad535a12829b1010afc2da79b344ea8b5ab8ed47e426d0bf2eb78ab395d20b1da8dbb
ee1128ead846698db5e5633f193883837f2fbc64 doc: update stack-clash-protection comment re mingw-w64 (fanquake)
bf47448f152316145d9abb9b8abc3b564194fe46 test: drop check for Windows < 10 (fanquake)
35b898c47f8af6807c4a5f404af165c663c81a99 release: target Windows 10 or later (fanquake)
398754e70bc96b86ad0327fbe70fafdf27bb4e35 depends: target Windows 10 when building for mingw-w64 (fanquake)
Pull request description:
Follows up to https://github.com/bitcoin/bitcoin/pull/31048#discussion_r1803165670.
We definitely cannot claim that Bitcoin Core is "supported and extensively tested on" on Windows 7.
Note that #30997 is also increasing the minimum required Windows version (for the GUI) to 10.
ACKs for top commit:
hodlinator:
cr-ACK ee1128ead846698db5e5633f193883837f2fbc64
davidgumberg:
ACK ee1128ead8
achow101:
ACK ee1128ead846698db5e5633f193883837f2fbc64
hebasto:
re-ACK ee1128ead846698db5e5633f193883837f2fbc64, only rebased, a commit message and a comment have been amended since my recent [review](https://github.com/bitcoin/bitcoin/pull/31172#pullrequestreview-2415452160).
TheCharlatan:
ACK ee1128ead846698db5e5633f193883837f2fbc64
Tree-SHA512: 245e0bac3d63414d919a1948661fef4ff79359faaacaf19d64abd91cc62e822797fb1cf3379e340bfdf9a85c0b88fd99a90eda450dd4218b6213ab78aefb1374
And under the hood suppoert single transactions
in AcceptPackage. This simplifies user experience
and paves the way for reducing number of codepaths
for transaction acceptance in the future.
Co-Authored-By: instagibbs <gsanders87@gmail.com>
The duplicate checks are repeated early in the contextual checks of
ProcessNewBlock. If duplicate blocks are detected much of their
validation is skipped. Depending on the constitution of the block,
validating the merkle root of the block is part of the more intensive
workload when validating a block. This could be an argument for moving
the pre-checks into block processing. In net_processing this would have
a smaller effect however, since the block mutation check, which also
validates the merkle root, is done before.
A side effect of this change is that a duplicate block is persisted
again on disk even when pruning is activated. This is similar to the
behaviour with getblockfrompeer. Add a release note for this change in
behaviour.
Testing spamming a node with valid, but duplicate unrequested blocks
seems to exhaust a CPU thread, but does not seem to significantly impact
keeping up with the tip. The benefits of adding these checks to
net_processing are questionable, especially since there are other ways
to trigger the more CPU-intensive checks without submitting a duplicate
block. Since these DOS concerns apply even less to the RPC interface,
which does not have banning mechanics built in, remove them too.
---
With the introduction of a mining ipc interface and the potential future
introduction of a kernel library API it becomes increasingly important
to offer common behaviour between them. An example of this is
ProcessNewBlock, which is used by ipc, rpc, net_processing and
(potentially) the kernel library. Having divergent behaviour on
suggested pre-checks and checks for these functions is confusing to both
developers and users and is a maintenance burden.
The rpc interface for ProcessNewBlock (submitblock) currently pre-checks
if the block has a coinbase transaction and whether it has been
processed before. While the current example binary for how to use the
kernel library, bitcoin-chainstate, imitates these checks, the other
interfaces do not.
Brew has migrated to using the later:
```bash
brew info pkg-config
==> pkgconf: stable 2.3.0 (bottled), HEAD
Package compiler and linker metadata toolkit
https://github.com/pkgconf/pkgconf
```
5736d1ddacc4019101e7a5170dd25efbc63b622a tracing: pass if replaced by tx/pkg to tracepoint (0xb10c)
a4ec07f1944999c2eead41d08d7dd4fc3aa71243 doc: add comments for CTxMemPool::ChangeSet (Suhas Daftuar)
83f814b1d1100baac9dca9c176f89b0ec2555dbc Remove m_all_conflicts from SubPackageState (Suhas Daftuar)
d3c8e7dfb63f7986a1f9654ea2393aabe3cd78da Ensure that we don't add duplicate transactions in rbf fuzz tests (Suhas Daftuar)
d7dc9fd2f7bc675256687b9c55fdbec9cc8ac781 Move CalculateChunksForRBF() to the mempool changeset (Suhas Daftuar)
284a1d33f1dcbc3b3404ea40a948ff6600239613 Move prioritisation into changeset (Suhas Daftuar)
446b08b599bc492bbec10ccc2292aee6f90c58e7 Don't distinguish between direct conflicts and all conflicts when doing cluster-size-2-rbf checks (Suhas Daftuar)
b53041021abc4f9ee7203341413e8676e2d5a7ca Duplicate transactions are not permitted within a changeset (Suhas Daftuar)
b447416fddcb8c8647391502cca3dbfd1552e02e Public mempool removal methods Assume() no changeset is outstanding (Suhas Daftuar)
2b30f4d36c86f775ac637b171d27d42a02309c5b Make RemoveStaged() private (Suhas Daftuar)
18829194ca68152ac0b38d34e94b9265ee74c410 Enforce that there is only one changeset at a time (Suhas Daftuar)
7fb62f7db60c7d793828ae45f87bc3f5c63cc989 Apply mempool changeset transactions directly into the mempool (Suhas Daftuar)
34b6c5833d11ea84fbd4b891e06408f6f4ca6fac Clean up FinalizeSubpackage to avoid workspace-specific information (Suhas Daftuar)
57983b8add72a04721d3f2050c063a3c4d8683ed Move LimitMempoolSize to take place outside FinalizeSubpackage (Suhas Daftuar)
01e145b9758f1df14a7ea18058ba9577bf88e459 Move changeset from workspace to subpackage (Suhas Daftuar)
802214c0832de00f24268183f7763fa984ba7903 Introduce mempool changesets (Suhas Daftuar)
87d92fa340195d9c87be3d023ca133b90b3b7d4e test: Add unit test coverage of package rbf + prioritisetransaction (Suhas Daftuar)
15d982f91e6b0f145c9dd4edf29827cfabb37a3f Add package hash to package-rbf log message (Suhas Daftuar)
Pull request description:
part of cluster mempool: #30289
It became clear while working on cluster mempool that it would be helpful for transaction validation if we could consider a full set of proposed changes to the mempool -- consisting of a set of transactions to add, and a set of transactions (ie conflicts) to simultaneously remove -- and perform calculations on what the mempool would look like if the proposed changes were to be applied. Two specific examples of where we'd like to do this:
- Determining if ancestor/descendant/TRUC limits would be violated (in the future, cluster limits) if either a single transaction or a package of transactions were to be accepted
- Determining if an RBF would make the mempool "better", however that idea is defined, both in the single transaction and package of transaction cases
In preparation for cluster mempool, I have pulled this reworking of the mempool interface out of #28676 so it can be reviewed on its own. I have not re-implemented ancestor/descendant limits to be run through the changeset, since with cluster mempool those limits will be going away, so this seems like wasted effort. However, I have rebased #28676 on top of this branch so reviewers can see what the new mempool interface could look like in the cluster mempool setting.
There are some minor behavior changes here, which I believe are inconsequential:
- In the package validation setting, transactions would be added to the mempool before the `ConsensusScriptChecks()` are run. In theory, `ConsensusScriptChecks()` should always pass if the `PolicyScriptChecks()` have passed and it's just a belt-and-suspenders for us, but if somehow they were to diverge then there could be some small behavior change from adding transactions and then removing them, versus never adding them at all.
- The error reporting on `CheckConflictTopology()` has slightly changed due to no longer distinguishing between direct conflicts and indirect conflicts. I believe this should be entirely inconsequential because there shouldn't be a logical difference between those two ideas from the perspective of this function, but I did have to update some error strings in some tests.
- Because, in a package setting, RBFs now happen as part of the entire package being accepted, the logging has changed slightly because we do not know which transaction specifically evicted a given removed transaction.
- Specifically, the "package hash" is now used to reference the set of transactions that are being accepted, rather than any single txid. The log message relating to package RBF that happen in the `TXPACKAGES` category has been updated as well to include the package hash, so that it's possible to see which specific set of transactions are being referenced by that package hash.
- Relatedly, the tracepoint logging in the package rbf case has been updated as well to reference the package hash, rather than a transaction hash.
ACKs for top commit:
naumenkogs:
ACK 5736d1ddac
instagibbs:
ACK 5736d1ddacc4019101e7a5170dd25efbc63b622a
ismaelsadeeq:
reACK 5736d1ddacc4019101e7a5170dd25efbc63b622a
glozow:
ACK 5736d1ddacc
Tree-SHA512: 21810872e082920d337c89ac406085aa71c5f8e5151ab07aedf41e6601f60a909b22fbf462ef3b735d5d5881e9b76142c53957158e674dd5dfe6f6aabbdf630b
ac286e0d1bd4c8dbe6d21cbe8c0a131ad8ba1854 doc: Fix grammatical errors in multisig-tutorial.md (secp512k2)
Pull request description:
This pull request fixes grammatical errors in the `multisig-tutorial.md` document.
ACKs for top commit:
Abdulkbk:
ACK ac286e0d1bd4c8dbe6d21cbe8c0a131ad8ba1854
Tree-SHA512: 684fe16e802431109957b9cde441353edeb16ffffde4282310c1a6f104adffc53347d00a2cf3a5969a78803f3177d6056ca37d3b7e8be52c2ec43ec0b9fcf4d9
The mempool:replaced tracepoint now reports either a txid or a
package hash (previously it always was a txid). To let users know
if a txid or package hash is passed, a boolean argument is added
the the tracepoint.
In the functional test, a ctypes.Structure class for MempoolReplaced
is introduced as Python warns the following when not explcitly
casting it to a ctype:
Type: 'bool' not recognized. Please define the data with ctypes manually.
5a96767e3f531ba9e8a676eec47727421f9f589f depends, libevent: Do not install *.pc files and remove patches for them (Hennadii Stepanov)
ffda355b5a2113fa0f7db8015f7b08bf1351e245 cmake, refactor: Move `HAVE_EVHTTP_...` to `libevent` interface (Hennadii Stepanov)
b619bdc3303217f4415342fe60e586e18fa48308 cmake: Revamp `FindLibevent` module (Hennadii Stepanov)
Pull request description:
This PR generalizes the use of `find_package` / `pkg_check_modules`, prioritizing the former.
Addresses https://github.com/bitcoin/bitcoin/pull/30903#issuecomment-2444700876:
> We should also follow up with refactoring the libevent module, to more generically use CMake/pkg-config, rather than restricting the CMake usage to `vcpkg`. At that point, we'd likely be able to dump pkg-config for the depends path entirely.
Similar to https://github.com/bitcoin/bitcoin/pull/30903.
ACKs for top commit:
fanquake:
ACK 5a96767e3f531ba9e8a676eec47727421f9f589f
Tree-SHA512: 181020c16ccd2821e718c73f264badcdc5e62980c4a8d9691e759efe2ea00da2326e26308d1dcfdeac01e9e27930428ecace9f36941deee951b751b138d7266c
ec375de39ff8e0d44fc026618a234e37019e46c1 doc: Add missing 'blank=true' option in offline-signing-tutorial.md (secp512k2)
Pull request description:
Issue:
The text mentions that the `createwallet` command should use the options `disable_private_keys=true, blank=true`, but the provided command only includes `disable_private_keys=true`, missing the `blank=true` option.
Correction:
Added `blank=true` to the command to match the options described in the text.
Explanation:
The `blank=true` option is necessary to create a blank wallet. Including this option ensures the command matches the options specified in the text.
ACKs for top commit:
fanquake:
ACK ec375de39ff8e0d44fc026618a234e37019e46c1
Tree-SHA512: 8c145e3ef1598c5e13f2aa89e496f76bfe2fc6f47d5e740963acad18dd1f782655a822dc234862af8e5a08060ab7fe1039a3dcfa68455a9143fe2d849975927c
5e3b444022c354ad4d69e3142f7bc98d723c9e29 doc: Fix missing comma in JSON example in REST-interface.md (secp512k2)
Pull request description:
This pull request addresses a minor issues in the REST-interface.md documentation:
Missing Comma in JSON Example: In the "Query UTXO set" section, a missing comma after the "desc" field in the JSON example has been added to ensure valid JSON syntax.
ACKs for top commit:
maflcko:
lgtm ACK 5e3b444022c354ad4d69e3142f7bc98d723c9e29
Abdulkbk:
ACK 5e3b444022c354ad4d69e3142f7bc98d723c9e29
Tree-SHA512: d2d479c8a991d3380d16b7b140a375a90dca0fce0a024a4b8ccf842d703398fde14ae972349f5fbd2e0ce26aa6cd6d07c0262d9c09ddc4c6c466527cfbe0e1f1
0de3e96e333090548a43e5e870c4cb8941d6baf1 tracing: use bitcoind pid in bcc tracing examples (0xb10c)
411c6cfc6c2e488e407f057b646730e63806ed8a tracing: only prepare tracepoint args if attached (0xb10c)
d524c1ec06643208c189089089e84f6e1cd0abad tracing: dedup TRACE macros & rename to TRACEPOINT (0xb10c)
Pull request description:
Currently, if the tracepoints are compiled (e.g. in depends and release builds), we always prepare the tracepoint arguments regardless of the tracepoints being used or not. We made sure that the argument preparation is as cheap as possible, but we can almost completely eliminate any overhead for users not interested in the tracepoints (the vast majority), by gating the tracepoint argument preparation with an `if(something is attached to this tracepoint)`. To achieve this, we use the optional semaphore feature provided by SystemTap.
The first commit simplifies and deduplicates our tracepoint macros from 13 TRACEx macros to a single TRACEPOINT macro. This makes them easier to use and also avoids more duplicate macro definitions in the second commit.
The Linux tracing tools I'm aware of (bcc, bpftrace, libbpf, and systemtap) all support the semaphore gating feature. Thus, all existing tracepoints and their argument preparation is gated in the second commit. For details, please refer to the commit messages and the updated documentation in `doc/tracing.md`.
Also adding unit tests that include all tracepoint macros to make sure there are no compiler problems with them (e.g. some varadiac extension not supported).
Reviewers might want to check:
- Do the tracepoints still work for you? Do the examples in `contrib/tracing/` run on your system (as bpftrace frequently breaks on every new version, please test master too if it should't work for you)? Do the CI interface tests still pass?
- Is the new documentation clear?
- The `TRACEPOINT_SEMAPHORE(event, context)` macros places global variables in our global namespace. Is this something we strictly want to avoid or maybe move to all `TRACEPOINT_SEMAPHORE`s to a separate .cpp file or even namespace? I like having the `TRACEPOINT_SEMAPHORE()` in same file as the `TRACEPOINT()`, but open for suggestion on alternative approaches.
- Are newly added tracepoints in the unit tests visible when using `readelf -n build/src/test/test_bitcoin`? You can run the new unit tests with `./build/src/test/test_bitcoin --run_test=util_trace_tests* --log_level=all`.
<details><summary>Two of the added unit tests demonstrate that we are only processing the tracepoint arguments when attached by having a test-failure condition in the tracepoint argument preparation. The following bpftrace script can be used to demonstrate that the tests do indeed fail when attached to the tracepoints.</summary>
`fail_tests.bt`:
```c
#!/usr/bin/env bpftrace
usdt:./build/src/test/test_bitcoin:test:check_if_attached {
printf("the 'check_if_attached' test should have failed\n");
}
usdt:./build/src/test/test_bitcoin:test:expensive_section {
printf("the 'expensive_section' test should have failed\n");
}
```
Run the unit tests with `./build/src/test/test_bitcoin` and start `bpftrace fail_tests.bt -p $(pidof test_bitcoin)` in a separate terminal. The unit tests should fail with:
```
Running 594 test cases...
test/util_trace_tests.cpp(31): error: in "util_trace_tests/test_tracepoint_check_if_attached": check false has failed
test/util_trace_tests.cpp(51): error: in "util_trace_tests/test_tracepoint_manual_tracepoint_active_check": check false has failed
*** 2 failures are detected in the test module "Bitcoin Core Test Suite"
```
</details>
These links might provide more contextual information for reviewers:
- [How SystemTap Userspace Probes Work by eklitzke](https://eklitzke.org/how-sytemtap-userspace-probes-work) (actually an example on Bitcoin Core; mentions that with semaphores "the overhead for an untraced process is effectively zero.")
- [libbpf comment on USDT semaphore handling](1596a09b5d/src/usdt.c (L83-L92)) (can recommend the whole comment for background on how the tracepoints and tracing tools work together)
- https://sourceware.org/systemtap/wiki/UserSpaceProbeImplementation#Semaphore_Handling
ACKs for top commit:
willcl-ark:
utACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
laanwj:
re-ACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
jb55:
utACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
vasild:
ACK 0de3e96e333090548a43e5e870c4cb8941d6baf1
Tree-SHA512: 0e5e0dc5e0353beaf5c446e4be03d447e64228b1be71ee9972fde1d6fac3fac71a9d73c6ce4fa68975f87db2b2bf6eee2009921a2a145e24d83a475d007a559b
