2b6e767d96 doc: archive release notes for v31.1 (fanquake)
Pull request description:
v31.1 has been tagged: https://github.com/bitcoin/bitcoin/releases/tag/v31.1/.
ACKs for top commit:
willcl-ark:
ACK 2b6e767d96
Tree-SHA512: 29e30534e56ffc0ea6ad077bdded05792bcc1ce3dd277332208658d1ae2b8aa9ef30289fed72be25370fc62621e71a0d2da563fe58726541d17dfe5d365a2470
31abaa264c doc: add an AI contribution policy (will)
Pull request description:
This policy, adapted from ripgrep, f0cec341ab/AI_POLICY.md who in turn adapted it from uv c5187e200d/AI_POLICY.md, works as a reasonable and pragmatic AI contribution policy at this point in time.
It codifies roughly how the project is currently operating, it's expectations when Ai is being used, and what we don't wish to see.
Link to the document directly from the new PR and issue helptext.
ACKs for top commit:
Sjors:
re-ACK 31abaa264c
achow101:
ACK 31abaa264c
sedited:
Re-ACK 31abaa264c
l0rinc:
ACK 31abaa264c
Tree-SHA512: 667bda2d02717889ee6878438b4e4c7155025ae6933ac748b49f7ca2a04c94515bdd04d522a778828995c3e328780521c1734dd0d8cca4711a702fc9f242756f
68cb7840d2 doc: improve offline-signing-tutorial after 32489 (Pablo Martin)
Pull request description:
General improvements noted in the #32489 review and deferred by the author:
- Remove [a stale NOTE](https://github.com/bitcoin/bitcoin/pull/32489#discussion_r3484961398) referencing `walletcreatefundedpsbt`; the tutorial was updated to use the send RPC instead.
- [Fix](https://github.com/bitcoin/bitcoin/pull/32489#discussion_r3484961536) `listtransactions` example output from `{...}` to `[...]`; the RPC returns a JSON array, not an object.
ACKs for top commit:
polespinasa:
ACK 68cb7840d2
Tree-SHA512: 0615f042a98f68d1a3bd71bf04ad0f66aa88b7011572b58e2b349623da0ac334d5d388caeee16e1bd922650e5aa6a9fa7d9d00da9a70d227dad22a78dd9e6b76
4e29de719e private broadcast: add release note for limited cap (Gregory Sanders)
cbf8c107c1 Release cs_main between individual private tx re-attempts (Greg Sanders)
5aea3d0373 private broadcast: limit outstanding txs to count of 10,000 (Gregory Sanders)
Pull request description:
Add a belt-and-suspenders feature, limit the amount of memory and cpu possible when unlucky or simply misconfigured. The worst case limit is roughly 400kB * 10,000 = 4GB, regardless of usage pattern.
Before this change, sheer volume of broadcasts, mismatches in standardness rules, or simply fee mismatches may result in unbounded growth of memory usage. As the feature may be expanded in the future, explicit bounds helps reasoning going forward.
ACKs for top commit:
frankomosh:
tACK 4e29de719e. Ran private_broadcast_tests and p2p_private_broadcast_cap.py. Great to have an explicit bound as the belt-and-suspenders against unbounded queue growth.
vasild:
ACK 4e29de719e
andrewtoth:
ACK 4e29de719e
stickies-v:
ACK 4e29de719e
Tree-SHA512: 18161755f37d07cca185a09e782dbe2fd0025b8befd4f6660e988865cc3a9b705d41769b816161e8142fe6ce31a56e0288bd78efc25135cedfc47fc855011799
cddbad325d doc: Add release notes for 32489 (exportwatchonlywallet RPC) (Pablo Martin)
Pull request description:
This is a follow-up to #32489.
ACKs for top commit:
polespinasa:
ACK cddbad325d
Tree-SHA512: bfc14c1d8576395caea9636fcc57182c7fe027e397b8529d60901f7310e30acfe9157fbc0e11f6ad8c16c96cb96df9d7d70d0965c5e8a06d6fa3e9e7b5a65533
General improvements noted in the #32489 review and deferred by the
author:
- Remove a stale NOTE referencing walletcreatefundedpsbt; the tutorial
was updated to use the send RPC instead.
- Fix listtransactions example output from {...} to [...]; the RPC
returns a JSON array, not an object.
9b2ef81757 doc: add release notes for #33671 (getbalances nonmempool field) (Pablo Martin)
Pull request description:
This is a follow-up to #33671.
Top commit has no ACKs.
Tree-SHA512: 0572b121ff74b3a455355f03d3297f5c779313d7d7e1ad258d46f2d44980d6140280cbd8c367828e6db7b8243d7cc485bde7367edae4415afef0db7b70c23713
a15bdc0598 doc: update offline-signing-tutorial to use exportwatchonlywallet rpc (Pol Espinasa)
a388076401 test: Test for exportwatchonlywallet (Ava Chow)
d053e3e5c8 wallet, rpc: Add exportwatchonlywallet RPC (Ava Chow)
444878efef wallet: Add CWallet::ExportWatchOnly (Ava Chow)
f9273f01db wallet: Move listdescriptors retrieving from RPC to CWallet (Ava Chow)
a1c83789a7 wallet: Write new descriptor's cache in AddWalletDescriptor (Ava Chow)
1e996640e6 wallet: Use Descriptor::CanSelfExpand() in CanGetAddresses() (Ava Chow)
d2ee9227da descriptor: Add CanSelfExpand() (Ava Chow)
Pull request description:
Currently, if a user wants to use an airgapped setup, they need to manually create the watchonly wallet that will live on the online node by importing the public descriptors. This PR introduces `exportwatchonlywallet` which will create a wallet file with the public descriptors to avoid exposing the specific internals to the user. Additionally, this RPC will copy any existing labels, transactions, and wallet flags. This ensures that the exported watchonly wallet is almost entirely a copy of the original wallet but without private keys.
ACKs for top commit:
polespinasa:
lgtm ACK a15bdc0598
Sjors:
re-utACK a15bdc0598
pablomartin4btc:
re-ACK [a15bdc0](a15bdc0598)
w0xlt:
lgtm reACK a15bdc0598
Tree-SHA512: cfc59415ad9aa13d1445cf2a85db1c051215496b6edcf5a8db463499b2b51b92ee7bf840b709035dff7635f9d0c533423bceb58c851f220500e1ea254d12f3b8
Document project expectations for AI-assisted contributions so contributors
understand when AI use is acceptable and when it creates review or moderation
burden.
Link to the document directly from the new PR and issues helptext.
Document that newly indexed `txospenderindex` entries use less disk space and that existing indexes remain readable.
Users only need to rebuild the index if they want previously indexed entries rewritten with the smaller marker.
3765b428d1 logging: More fully remove libevent log category (Ryan Ofsky)
Pull request description:
Libevent log category was partially removed in 39e9099da5, and this commit extends that with the following changes:
- Stops showing libevent in the list of supported log categories in `bitcoind -help` and `bitcoin-cli help logging` output.
- Stops returning `"libevent": false` in `logging` RPC output.
It's not good to treat libevent as a supported log category when it can't be enabled and trying to enable it results in warnings.
There's also no need to define an unused LIBEVENT constant value and keep more complicated logic for dealing with deprecated log categories, so this change also simplifies code internally.
ACKs for top commit:
l0rinc:
code review ACK 3765b428d1
pinheadmz:
ACK 3765b428d1
sedited:
ACK 3765b428d1
Tree-SHA512: 09e9514f905bb0a79d870689af491886baaa31fa19f2ad6aef4283fa20c2fa6ce8d384178139227aeeabffabff6e83d254114daaeefbbfe2c6172b9da8871298
0cdd817a82 add release note (Pol Espinasa)
517d37ce3e test: tests wallet migration with load_wallet disabled (Pol Espinasa)
b98dd63da7 rpc: Add load_wallet argument to migratewallet RPC (Pol Espinasa)
4acd063ba6 wallet: make loading the wallet after migrating optional (Pol Espinasa)
97d08d62ba refactor: store wallet names to MigrationResult (Pol Espinasa)
Pull request description:
This PR is motivated by this [Stack Exchange question](https://bitcoin.stackexchange.com/questions/130713/bitcoin-core-quickest-method-legacy-descriptor-wallet-migration).
Long story short, someone who has a node pruned before his legacy wallet birthday, is unable to migrate the wallet as it is not possible to load it.
Loading is not necessary for migration, and migrating without wanting to use the wallet in that node is a valid use-case.
This PR adds a new RPC argument to `migratewallet` that allow the user disabling the wallet loading.
Second commits adds tests for it.
Follow-up: Add an option to the GUI to not load the wallet after migrating.
ACKs for top commit:
achow101:
ACK 0cdd817a82
w0xlt:
ACK 0cdd817a82
pablomartin4btc:
ACK 0cdd817a82
Tree-SHA512: 8389599e63603b1a532e1bfba0b6c652653386c001f5a881bd49843302b74ff4dbaa4131b5b377c24f483d42e0e70a92b96f760244e3c2e2b44ce08cd04ca1e0
Libevent log category was partially removed in 39e9099da5, and this
commit extends that with the following changes:
- Stops showing libevent in the list of supported log categories in
`bitcoind -help` and `bitcoin-cli help logging` output.
- Stops returning `"libevent": false` in `logging` RPC output.
It's not good to treat libevent as a supported log category when it
can't be enabled and trying to enable it results in warnings.
There's also no need to define an unused LIBEVENT constant value and
keep more complicated logic for dealing with deprecated log categories,
so this change also simplifies code internally.
Co-authored-by: David Gumberg <davidzgumberg@gmail.com>
Co-authored-by: l0rinc <pap.lorinc@gmail.com>
2fe34808fa wallet: reject sendtoaddress and sendmany for external signers (Sjors Provoost)
bd5a32f7db doc: add taproot descriptor to getdescriptors example (woltx)
7131c82937 doc: clarify which commands receive --chain, --fingerprint and --stdin (woltx)
4fdd4d8d29 doc: replace stale signtransaction wording with current signtx flow (woltx)
fab92257fe doc, rpc: document enumerate model field and fingerprint deduplication (woltx)
Pull request description:
This PR aligns the external signer documentation with current behavior, and makes one previously implicit behavior explicit.
Per review feedback, each commit fixes a limited set of issues:
* **doc, rpc: document enumerate model field and fingerprint deduplication** — the `enumerate` response uses the optional `model` field, which Bitcoin Core maps to the `name` field of the `enumeratesigners` RPC result. Duplicate fingerprints are skipped, and wallet operations require exactly one connected signer.
* **doc: replace stale signtransaction wording with current signtx flow** — spending from an external signer wallet uses `send`/`sendall` (and `bumpfee` for fee-bumping), which invoke `<cmd> --stdin` and pass the `signtx` subcommand and PSBT over stdin.
* **doc: clarify which commands receive --chain, --fingerprint and --stdin** — mark `--chain` and `--fingerprint` as required except for `enumerate`, keep `--stdin` required for protocol flexibility, and match the order and form of the actual invocations in the usage examples.
* **doc: add taproot descriptor to getdescriptors example** — show the BIP86 `tr()` descriptor alongside the other address types.
* **wallet: reject sendtoaddress and sendmany for external signers** — return a specific error instead of the misleading "Private keys are disabled for this wallet", with functional test coverage. Cherry-picked from #33112 (thanks Sjors).
How the documentation went stale:
* The `enumerate` example has shown a `name` field since external signer support landed in #16546, but the implementation has always read `model`.
* `sendtoaddress`/`sendmany` external signer support was effectively precluded by #21201, which was merged a few days before #16546, so the interaction was missed in review and the documented `signtransaction` flow never existed in this form.
* Fingerprint deduplication was added in #35251.
* The documentation was last updated in #33765.
ACKs for top commit:
Sjors:
ACK 2fe34808fa
optout21:
ACK 2fe34808fa
naiyoma:
ACK 2fe34808fa
Tree-SHA512: 86859d2f81ac337f3b4b6578c6ee0151ffb76b8374dfa58e28e00ce4eb69dc200cd6bd2d0a99f73d0475c3824d6ac1cb9e2542b119ca124dd835132dc95cd023
`BaseIndex` currently uses the same name for logs, `getindexinfo`, prune locks, and the sync thread.
Linux truncates system thread names to 15 visible bytes after the `b-` prefix, so long indexer names are clipped in system tools.
Pass a separate thread name explicitly at each `BaseIndex` call site and shorten the OS-visible indexer names to `txidx`, `blkfltbscidx`, `coinstatsidx`, and `txospenderidx`.
Add an `Assume` to `ThreadRename()` so future OS-visible thread names keep fitting the same Linux limit.
The public index names, `getindexinfo` keys, command-line options, and on-disk paths stay unchanged.
Co-authored-by: winterrdog <winterrdog@users.noreply.github.com>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: sedited <seb.kung@gmail.com>
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
Thread names with numeric suffixes are easier to scan when the suffixes use a fixed width.
Format `ThreadPool` and script-check worker suffixes as two digits while keeping the compact dotted convention.
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
ThreadPool workers currently format their names as `name_pool_N`.
Linux truncates the system thread name to 15 visible bytes, so `b-http_pool_N` spends much of that space on the suffix.
Use a dotted numeric suffix so HTTP worker names become `b-http.N`, leaving more room for longer pool names.
This feature was broken for more than a year and no developer apparently
even noticed, so one can conclude that it is largely unused; it seems
thus reasonable to remove it to reduce maintenance burden.
Wallets import descriptors for all supported address types in a
BIP44/49/84/86 compatible manner, so show the BIP86 tr() descriptor in
the example response as well.
Bitcoin Core passes --chain and --fingerprint to every signer command
except enumerate, so mark them "(required except for enumerate)" and
name the current commands only as examples. Keep --stdin documented as
required for all commands so the interface stays flexible, noting that
Bitcoin Core currently only uses it for signtx.
Add the missing flags to the getdescriptors and displayaddress usage
examples and the corresponding doxygen comments, matching the order and
form of the actual invocations, add testnet4 to the chain name lists,
and drop the getaddressinfo implementation detail from the
walletdisplayaddress description.
The protocol documentation still described a "signtransaction" command
driven by sendtoaddress and sendmany. Those RPCs never gained external
signer support: it was effectively precluded by #21201, which was
merged a few days before external signer support landed in #16546, so
the interaction was missed in review (#33112 has a commit making the
rejection explicit). Spending from an external signer wallet uses
send/sendall (and bumpfee for fee-bumping), which invoke the signer
with `<cmd> --stdin` and pass the `signtx` subcommand and PSBT over
stdin.
Update the spending example and the protocol description accordingly,
using `bitcoin rpc` for the example since it enables -named by default.
The external signer "enumerate" response uses the optional "model"
field, not "name". Document that Bitcoin Core maps it to the "name"
field of the enumeratesigners RPC result, and that signers with
duplicate master key fingerprints are skipped.
Also document that wallet operations require exactly one connected
signer.
Co-authored-by: optout <13562139+optout21@users.noreply.github.com>
Co-authored-by: naiyoma <lankas.aurelia@gmail.com>
BIP434 defines FEATURE messages which are sent between VERSION and VERACK
to indicate support for new P2P protocol features. This commit provides
the infrastructure for easily using BIP434 negotiation when implementing
such new P2P protocol features. Note that advertised protocol version
is bumped to 70017, as per BIP434's specification.
bf0d257c11 net: un-default the OpenNetworkConnection()'s proxy_override argument (Eugene Siegel)
5a3756d150 test: add a regression test for private broadcast v1 retries (Vasil Dimov)
ab35a028ed test: make reusable filling of a node's addrman (Vasil Dimov)
2333be9cbc test: make reusable starting a standalone P2P listener (Vasil Dimov)
2ffa81fac4 test: make reusable SOCKS5 server starting (Vasil Dimov)
32d072a49f doc: add release notes for #35319 (Vasil Dimov)
d01b461f71 net: ensure no direct private broadcast connections (Vasil Dimov)
fd230f942d net: use the proxy if overriden when doing v2->v1 reconnections (Vasil Dimov)
Pull request description:
This PR includes https://github.com/bitcoin/bitcoin/pull/35319 and on top of that adds a regression functional test.
The functional test exercises the relevant code paths without modifying non-test code. To do that it does:
* Add a bunch of IPv4 addresses to the node's addrman (they will be added without P2P_V2 flag).
* Get them to report P2P_V2 in their service flags and connect to each one, so that the flags
in addrman are updated to contain P2P_V2.
* Get one successful connection to a Tor peer (.onion) so that bitcoind assumes the configured
Tor proxy works and is indeed a proxy to the Tor network. This will make it open private
broadcast connections also to IPv4 addresses via that proxy.
* Start some private broadcast connections.
* Remember the destination IPv4 address of the first connection and get it to fail the v2
transport.
* Wait for a subsequent connection also through the Tor proxy to the same IPv4 and expect
it to be v1, i.e. the v2->v1 downgrade retry.
The test fails without the fix - the v1 retry never arrives to the Tor proxy. And passes with the fix. The fix is in the first commit here and in https://github.com/bitcoin/bitcoin/pull/35319, can remove it by `git show fd230f942d | git apply -R`.
ACKs for top commit:
Crypt-iQ:
reACK bf0d257c11
andrewtoth:
ACK bf0d257c11
instagibbs:
ACK bf0d257c11
sedited:
utACK bf0d257c11
Tree-SHA512: 11e89be36577199e0312e5e63efeac04e295faaba1cf1c13a30e683d35f473c8dbb419d1897b0333c2e993c10637adecafcf90fe08c812065c793cbc903744c9
107d4178d9 versionbits: update VersionBitsCache doc comment to match current behaviour (Antoine Poinsot)
94e3ac0b21 doc: release notes and bips doc update for #34779 (Antoine Poinsot)
1d5240574a qa: test we don't warn for ignored unknown version bits deployments (Antoine Poinsot)
f802edf57c versionbits: Limit live activation params and activation warnings per BIP323 (Anthony Towns)
Pull request description:
This implements https://github.com/bitcoin/bips/pull/2116, which repurposes 24 version bits as extra nonce space for miners rather than soft fork deployment coordination. 24 bits allows a miner to perform up to 72 PH before needing a fresh job from its controller. The current 16 bits in use by miners only allow up to 280 TH, which [apparently led some ASIC designers to start rolling the timestamp field](https://github.com/bitaxeorg/ESP-Miner/pull/1553#issuecomment-3937736319) on their beefier machines.
Mailing list discussion available [here](https://gnusha.org/pi/bitcoindev/6fa0cb45-37d6-4b41-9ff8-03730fd96d6e@mattcorallo.com/). A previous shot at this is https://github.com/bitcoin/bitcoin/pull/13972 (with a smaller extranonce space).
This change only affects the warning logic.
ACKs for top commit:
ajtowns:
ACK 107d4178d9
achow101:
ACK 107d4178d9
sedited:
Re-ACK 107d4178d9
optout21:
ACK 107d4178d9
Tree-SHA512: cfaf5d7de1e8c020a4d7f4b1096b6c3e0e3b41ea840a4652ebcdabc345c5c557161c8304f1d7d6de541a2bf1df3c855ad7b64e49dd8c8af3937876d134bb5aba
fa787043f5 doc: Compress doc/build-unix.md dependency package names into table (MarcoFalke)
Pull request description:
Currently, `doc/build-unix.md` is tediously verbose, because for several Linux distros, it has exact duplicate sections with only the package names adjusted.
This is hard to maintain, and review. Also, it is hard to read, and hard to use, because a one-line copy-paste does not work to fetch the list of packages.
Fix all issues by compressing the 150+ lines into a small table and a short description.
ACKs for top commit:
achow101:
ACK fa787043f5
darosior:
ACK fa787043f5
sedited:
ACK fa787043f5
hebasto:
re-ACK fa787043f5.
Tree-SHA512: a6d7f18392ab5a0d468387ffe4335f71ae9656a100ede3de4118e3ef28814e5a70202ffa55eb0218a07effcde220b680499ea9068fd54b91cac42fca8699febf
f701cd159a doc: fix typo in release notes of #34917 (rkrux)
7bc39e3d08 wallet, test: add wallet_deprecated_rbf.py for walletrbf deprecated keys & options (rkrux)
2cbbcb5659 wallet, test: remove -deprecatedrpc=bip125 from wallet_send.py (rkrux)
307134bd7e wallet, test: remove -deprecatedrpc=bip125 from wallet_migration.py (rkrux)
3ec550d168 wallet, test: remove -deprecatedrpc=bip125 from wallet_basic.py (rkrux)
a52ea9bff9 wallet, test: remove -walletrbf startup option from wallet_backwards_compatibility.py (rkrux)
42330922dd wallet, test: remove -walletrbf startup option from wallet_backwards_compatibility.py (rkrux)
8cb6e405d8 wallet, test: remove -walletrbf startup option from wallet_listtransactions.py (rkrux)
0ee94b2fef wallet, test: remove -deprecatedrpc=bip125 from wallet_listtransactions.py (rkrux)
5e833e068d wallet, test: -walletrbf startup option from wallet_bumpfee.py (rkrux)
a2a2b1745f wallet, test: remove -walletrbf startup option from rpc_psbt.py (rkrux)
a3fe455a95 wallet: refactor to read -walletrbf only once instead of twice (rkrux)
Pull request description:
Prerequisite to #35404 and #35405.
All these changes address the points raised in the review of PR #34917
here: https://github.com/bitcoin/bitcoin/pull/34917#pullrequestreview-4362148900.
Essentially updating the existing wallet functional tests without using
the -deprecatedrpc=bip125 and -walletrbf startup options. Instead,
these two are added and tested via a singular new
wallet_deprecated_rbf.py test that can be removed easily later when
these startup options are completely removed from the wallet post
deprecation.
ACKs for top commit:
maflcko:
review ACK f701cd159a🌄
achow101:
ACK f701cd159a
Tree-SHA512: 700785062b5de8ee3b6c4f50570b769d56c6c4960f2b6e2a2e71be8085c6b51eaeb34fb158fae76f812fe82791aaa0c0277f964f0472cb0784b86caabe6d4ec9
d5adb9d09b doc: fix doxygen links to threads in developer-notes.md (Matthew Zipkin)
Pull request description:
The "threads" section of `developer-notes.md` has links to anchor tags in the code generated by doxygen. As far as I can tell this was introduced in #18645 and changes to this section of this document have continued the pattern. The problem is, the content at `https://doxygen.bitcoincore.org` gets re-rendered daily and those anchor tags are generated internally by doxygen, so they are all broken now.
This PR adds doxygen syntax `\anchor XXXX` comments in the code where functions that run in these threads are defined, and then those stable, human-readable anchor tags are applied to the links in the doc.
I have generated the doxygen output from this branch, hosted it on my own web server, and created a modified `developer-notes.md` with these anchor tags and my server as host for demonstration:
https://gist.github.com/pinheadmz/ed3dda7d3c8d589e3989040519190b84#threads
Just note when looking at this:
- `main` is at the bottom of the html page so it might not look right at first
- `initload` is a lambda inside `AppInitMain` so thats where doxygen renders the anchor
ACKs for top commit:
fanquake:
ACK d5adb9d09b
rkrux:
lgtm ACK d5adb9d
Tree-SHA512: c5517823a2d668b01318b3dae3d76fdd9db8a74d8c721aeb748e4f4a6cb56cb4d24e34b2590a41f8553992005cab368fca4ce322a4f204cec16ce338337ae9ee
2e9fdcc6da doc: add feature deprecation and removal process to developer notes (Guillermo Fernandes)
Pull request description:
Closes#31980
Adds a dedicated **"Feature deprecation and removal process"** section to `doc/developer-notes.md` covering the full deprecation lifecycle for all major feature categories.
## What's added
**General principles**
- Grace period is one major release (deprecated in N, removed in N+1)
- Deprecation and removal both require release notes
- Deprecated features should remain accessible via a re-enable flag during the grace period
**Per-category guidance covering:**
- RPC methods and fields (`-deprecatedrpc=<feature>` pattern, help text requirements, worked example pointing to #31278)
- Startup options (`LogWarning`/`InitWarning` on use, help text update)
- REST interface (document in `doc/REST-interface.md`)
- ZMQ (document in `doc/zmq.md`)
- Wallet settings (defer to RPC or startup option process depending on exposure)
This consolidates the process that currently exists only implicitly across PRs and issue discussions into one place for contributors to reference.
ACKs for top commit:
maflcko:
lgtm ACK 2e9fdcc6da
polespinasa:
ACK 2e9fdcc6da
stickies-v:
ACK 2e9fdcc6da
sedited:
ACK 2e9fdcc6da
Tree-SHA512: 1d43df410664a45f937bcbd250664f13379168ca90e3024bea506e21a88177e201dcb4fadade705735099e3b8aaa2102a3080ad005bffb3aecb8f08d530d4277
1e5d3b4f0d doc: add release note for mining option validation (Sjors Provoost)
0317f52022 ci: enforce iwyu for touched files (Sjors Provoost)
8c58f63578 refactor: have mining files include what they use (Sjors Provoost)
3bb6498fb0 mining: store block create options in NodeContext (Sjors Provoost)
4637cd157d mining: reject invalid block create options (Sjors Provoost)
8daac1d6eb mining: add block create option helpers (Sjors Provoost)
128da7c3ff miner: add block_max_weight to BlockCreateOptions (Sjors Provoost)
fa81e51eae mining: parse block creation args in mining_args (Sjors Provoost)
020166080c mining: use interface for tests, bench and fuzzers (Sjors Provoost)
44082bea47 interfaces: make Mining use const NodeContext (Sjors Provoost)
d4368e059c move-only: add node/mining_types.h (Sjors Provoost)
6aeb1fbea2 test: cover IPC blockmaxweight policy (Sjors Provoost)
63b23ea1e9 test: regression test for waitNext mining policy (Sjors Provoost)
24750f8b31 test: add createNewBlock failure helper (Sjors Provoost)
63ee9cd15b test: misc interface_ipc_mining.py improvements (Sjors Provoost)
Pull request description:
Although this PR is primarily a refactor, _there are behavior changes_ documented in the release note:
- the IPC mining interface now rejects out-of-range block template options instead of silently clamping them;
- startup now rejects `-blockmaxweight` values lower than `-blockreservedweight`, instead of allowing them to be clamped later.
The interaction between node startup options like `-blockreservedweight` and runtime options, especially those passed via IPC, is confusing.
They're combined in `BlockAssembler::Options`, which this PR gets rid of in favour of `BlockCreateOptions`.
`BlockCreateOptions` is used by interface clients. As before, IPC clients have access to a safe / sane subset, whereas RPC and test code can use all fields. The same type is also used to store mining defaults parsed once during node startup in `NodeContext`.
The maximum block weight setting (`block_max_weight`) is optional. When read from startup options it matches `-blockmaxweight`; when provided by callers it is a runtime override. `Merge()` fills unset fields from startup defaults while preserving caller-provided values.
This all happens in commits `mining: add block create option helpers` and `mining: store block create options in NodeContext`, and requires some preparation to keep things easy to review.
We get rid of `BlockAssembler::Options` but this is used in many tests. Since large churn is inevitable, we might as well switch all tests, bench and fuzzers over to the Mining interface. The `mining: use interface for tests, bench and fuzzers` commit does that, dramatically reducing direct use of `BlockAssembler`. Two exceptions are documented in the commit message. Because `test_block_validity` wasn't available via the interface and the block_assemble benchmark needs it, it's moved from `BlockAssembler::Options` to `BlockCreateOptions` (still not exposed via IPC).
We need access to mining related structs from both the miner and node initialization code. To avoid having to pull in all of `BlockAssembler` for the latter, the `move-only: add node/mining_types.h` commit introduces `node/mining_types.h` and moves `BlockCreateOptions`, `BlockWaitOptions` and `BlockCheckOptions` there from `src/node/types.h`.
I considered also moving `DEFAULT_BLOCK_MAX_WEIGHT`, `DEFAULT_BLOCK_RESERVED_WEIGHT`, `MINIMUM_BLOCK_RESERVED_WEIGHT` and `DEFAULT_BLOCK_MIN_TX_FEE` there from `policy.h`, since they are distinct from relay policy and not needed by the kernel. But this seems more appropriate for a follow-up and requires additional discussion.
---
I kept variable renaming and other formatting changes to a minimum to ease review with `--color-moved=dimmed-zebra`.
## Commit summary
Tests and test cleanup:
- `test: misc interface_ipc_mining.py improvements`
- `test: add assert_create_fails helper`
- `test: regression test for waitNext mining policy`
- `test: cover IPC blockmaxweight policy`
Refactoring test/bench/fuzz callers:
- `interfaces: make Mining use const NodeContext`
- `mining: use interface for tests, bench and fuzzers`
Moving mining interface types:
- `move-only: add node/mining_types.h`
Separating startup defaults from runtime options:
- `mining: parse block creation args in mining_args`: adds `node/mining_args.{h,cpp}` and moves mining option parsing out of `init.cpp`, without storing the parsed values yet.
- `miner: add block_max_weight to BlockCreateOptions`: moves the runtime maximum block weight setting into `BlockCreateOptions` as an optional value, so it can later be defaulted from startup args when unset.
- `mining: add block create option helpers`: centralizes block template option defaulting and merging, removes `BlockAssembler::Options`, and preserves behavior except for dropping the `Specified ` prefix from startup option error messages.
- `mining: reject invalid block create options`: checks typed `BlockCreateOptions` before block template creation, so invalid runtime options are rejected instead of silently clamped. Startup validation also rejects `-blockmaxweight` values lower than `-blockreservedweight`.
- `mining: store block create options in NodeContext`: stores the startup mining options in `NodeContext` as `BlockCreateOptions`, so startup defaults and runtime overrides can be merged with the same option type.
Include hygiene, CI and release note:
- `refactor: have mining files include what they use`
- `ci: enforce iwyu for touched files`
- `doc: add release note for mining option validation`
ACKs for top commit:
w0xlt:
reACK 1e5d3b4f0d
sedited:
ACK 1e5d3b4f0d
ryanofsky:
Code review ACK 1e5d3b4f0d. Looks good, thanks for the updates!
Tree-SHA512: 28c715023cb78f02775caa787b243c994bd0f8ce4559afc8db9301e93400ebbc74963626a4afe65ae15bcc16b9192d051a745839f4c804848d50746ea5a224b4
